2023-08-25T09:21:08.037 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-25T09:21:08.038 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315 --name rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/orig.config.yaml 2023-08-25T09:21:08.083 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T09:21:08.168 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315 branch: wip-rishabh-2023aug3-b5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: null first_in_suite: false job_id: '7379315' kernel: flavor: default kdb: true sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-2023aug3-b5 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: test-nfs-pr-52556 sha1: 4b369cf18ed1391a426ab4ae86da834e9c074f81 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 sleep_before_teardown: 0 suite: fs suite_branch: test-nfs-pr-52556 suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 4b369cf18ed1391a426ab4ae86da834e9c074f81 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL3R/U3OG/FM9IFxeKz3TYfxUwwqbXNkJq2WN7SE1FnMDOfR60dexyJU4bMzBueqLKnZTZ/jAGKwpHHxo5L981I= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6by/2gfhYuyGvhCvl1oZimmuV0/1tQO9dMYvoeQ22S/c4hi+VkSUZiLArWbdLYvzk4bQdC3sIdT45aZwfMh+U= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-25_06:38:25 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-25T09:21:08.169 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa; will attempt to use it 2023-08-25T09:21:08.170 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa/tasks 2023-08-25T09:21:08.170 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-25T09:21:08.172 INFO:teuthology.task.internal:Checking packages... 2023-08-25T09:21:08.201 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ba6b39e4f4f77fb95c747099b53db0467d435f56' 2023-08-25T09:21:08.202 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-25T09:21:08.202 INFO:teuthology.packaging:ref: None 2023-08-25T09:21:08.202 INFO:teuthology.packaging:tag: None 2023-08-25T09:21:08.203 INFO:teuthology.packaging:branch: wip-rishabh-2023aug3-b5 2023-08-25T09:21:08.203 INFO:teuthology.packaging:sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T09:21:08.203 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-2023aug3-b5 2023-08-25T09:21:08.421 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5703.gba6b39e4 2023-08-25T09:21:08.421 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-25T09:21:08.428 INFO:teuthology.task.internal:no buildpackages task found 2023-08-25T09:21:08.429 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-25T09:21:08.455 INFO:teuthology.task.internal:Saving configuration 2023-08-25T09:21:08.471 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-25T09:21:08.480 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T09:21:08.506 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 09:16:06.610146', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL3R/U3OG/FM9IFxeKz3TYfxUwwqbXNkJq2WN7SE1FnMDOfR60dexyJU4bMzBueqLKnZTZ/jAGKwpHHxo5L981I='} 2023-08-25T09:21:08.527 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 09:16:06.608777', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6by/2gfhYuyGvhCvl1oZimmuV0/1tQO9dMYvoeQ22S/c4hi+VkSUZiLArWbdLYvzk4bQdC3sIdT45aZwfMh+U='} 2023-08-25T09:21:08.527 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-25T09:21:08.533 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-08-25T09:21:08.534 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-08-25T09:21:08.534 INFO:teuthology.run_tasks:Running task console_log... 2023-08-25T09:21:08.635 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6dd4357e50>, signals=[15]) 2023-08-25T09:21:08.636 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-25T09:21:08.642 INFO:teuthology.task.internal:Opening connections... 2023-08-25T09:21:08.642 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:21:08.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:21:08.729 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:21:08.730 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:21:08.812 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-25T09:21:08.818 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2023-08-25T09:21:08.836 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2023-08-25T09:21:08.837 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2023-08-25T09:21:08.893 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2023-08-25T09:21:08.893 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2023-08-25T09:21:08.894 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2023-08-25T09:21:08.894 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2023-08-25T09:21:08.894 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2023-08-25T09:21:08.894 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2023-08-25T09:21:08.895 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-25T09:21:08.895 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2023-08-25T09:21:08.895 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2023-08-25T09:21:08.896 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-25T09:21:08.896 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2023-08-25T09:21:08.896 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T09:21:08.896 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-25T09:21:08.897 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-25T09:21:08.898 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2023-08-25T09:21:08.925 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2023-08-25T09:21:08.943 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2023-08-25T09:21:08.943 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2023-08-25T09:21:09.000 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2023-08-25T09:21:09.001 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2023-08-25T09:21:09.001 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2023-08-25T09:21:09.001 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2023-08-25T09:21:09.001 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2023-08-25T09:21:09.002 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T09:21:09.003 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-08-25T09:21:09.003 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-08-25T09:21:09.003 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2023-08-25T09:21:09.025 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-25T09:21:09.036 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-25T09:21:09.061 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-25T09:21:09.062 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2023-08-25T09:21:09.064 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2023-08-25T09:21:09.081 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-25T09:21:09.088 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-25T09:21:09.088 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2023-08-25T09:21:09.122 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2023-08-25T09:21:09.157 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-25T09:21:09.216 INFO:teuthology.run_tasks:Running task kernel... 2023-08-25T09:21:09.232 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'} 2023-08-25T09:21:09.232 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}}, timeout 300 2023-08-25T09:21:09.233 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T09:21:09.233 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T09:21:09.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:21:09.250 DEBUG:teuthology.task.kernel:sha1 for mon.a is a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:09.250 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "a6a69670761025e039dc5ae86619b0cf04c46dc2"... 2023-08-25T09:21:09.250 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-08-25T09:21:09.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:21:09.252 DEBUG:teuthology.task.kernel:sha1 for mon.b is a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:09.252 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "a6a69670761025e039dc5ae86619b0cf04c46dc2"... 2023-08-25T09:21:09.252 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2023-08-25T09:21:09.306 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-239.el9.x86_64 2023-08-25T09:21:09.307 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-239.el9.x86_64 vs a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:09.307 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-25T09:21:09.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:09.310 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-239.el9.x86_64 2023-08-25T09:21:09.311 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-239.el9.x86_64 vs a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:09.311 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-08-25T09:21:09.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:09.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum upgrade -y linux-firmware 2023-08-25T09:21:09.537 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum upgrade -y linux-firmware 2023-08-25T09:21:11.568 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 10 MB/s | 6.2 MB 00:00 2023-08-25T09:21:11.603 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 12 MB/s | 6.2 MB 00:00 2023-08-25T09:21:12.870 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2023-08-25T09:21:12.904 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 6.6 MB/s | 2.0 MB 00:00 2023-08-25T09:21:13.653 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 7.8 MB 00:00 2023-08-25T09:21:16.210 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 7.8 MB 00:03 2023-08-25T09:21:16.579 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 18 MB 00:01 2023-08-25T09:21:18.011 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 22 MB/s | 18 MB 00:00 2023-08-25T09:21:20.071 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 14 kB 00:00 2023-08-25T09:21:20.972 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 23 MB/s | 18 MB 00:00 2023-08-25T09:21:21.388 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 14 kB 00:00 2023-08-25T09:21:22.172 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 26 MB/s | 18 MB 00:00 2023-08-25T09:21:26.961 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-08-25T09:21:26.968 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-08-25T09:21:26.968 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout: linux-firmware noarch 20230814-139.el9 CentOS-BaseOS 307 M 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2023-08-25T09:21:26.969 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 1 Package 2023-08-25T09:21:26.970 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:21:26.970 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 307 M 2023-08-25T09:21:26.970 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-08-25T09:21:28.209 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-08-25T09:21:28.215 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-08-25T09:21:28.215 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-08-25T09:21:28.216 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-08-25T09:21:28.216 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2023-08-25T09:21:28.216 INFO:teuthology.orchestra.run.smithi155.stdout: linux-firmware noarch 20230814-139.el9 CentOS-BaseOS 307 M 2023-08-25T09:21:28.216 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:21:28.216 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-08-25T09:21:28.216 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2023-08-25T09:21:28.217 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 1 Package 2023-08-25T09:21:28.217 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:21:28.217 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 307 M 2023-08-25T09:21:28.217 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-08-25T09:21:39.612 INFO:teuthology.orchestra.run.smithi070.stdout:linux-firmware-20230814-139.el9.noarch.rpm 24 MB/s | 307 MB 00:12 2023-08-25T09:21:39.613 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-08-25T09:21:39.613 INFO:teuthology.orchestra.run.smithi070.stdout:Total 24 MB/s | 307 MB 00:12 2023-08-25T09:21:39.614 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-08-25T09:21:39.693 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-08-25T09:21:39.694 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-08-25T09:21:41.634 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-08-25T09:21:41.635 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-08-25T09:21:41.989 INFO:teuthology.orchestra.run.smithi155.stdout:linux-firmware-20230814-139.el9.noarch.rpm 22 MB/s | 307 MB 00:13 2023-08-25T09:21:41.991 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-08-25T09:21:41.991 INFO:teuthology.orchestra.run.smithi155.stdout:Total 22 MB/s | 307 MB 00:13 2023-08-25T09:21:41.992 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-08-25T09:21:42.068 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-08-25T09:21:42.069 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-08-25T09:21:44.002 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-08-25T09:21:44.002 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-08-25T09:21:47.524 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-08-25T09:21:48.395 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : linux-firmware-20230814-139.el9.noarch 1/2 2023-08-25T09:21:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-25T09:21:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: linux-firmware-20221214-129.el9.noarch 2/2 2023-08-25T09:21:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 1/2 2023-08-25T09:21:50.108 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-08-25T09:21:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-25T09:21:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:21:50.375 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2023-08-25T09:21:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: linux-firmware-20230814-139.el9.noarch 2023-08-25T09:21:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:21:50.376 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-08-25T09:21:50.712 INFO:teuthology.task.kernel:Downloading kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on mon.a... 2023-08-25T09:21:50.713 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:50.868 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/9/flavors/default/x86_64/ 2023-08-25T09:21:50.869 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/9/flavors/default/x86_64/ --input-file=- 2023-08-25T09:21:50.872 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi070.front.sepia.ceph.com to complete... 2023-08-25T09:21:50.974 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : linux-firmware-20230814-139.el9.noarch 1/2 2023-08-25T09:21:51.080 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-25T09:21:52.415 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: linux-firmware-20221214-129.el9.noarch 2/2 2023-08-25T09:21:52.415 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 1/2 2023-08-25T09:21:52.873 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : linux-firmware-20221214-129.el9.noarch 2/2 2023-08-25T09:21:52.873 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:21:52.874 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2023-08-25T09:21:52.874 INFO:teuthology.orchestra.run.smithi155.stdout: linux-firmware-20230814-139.el9.noarch 2023-08-25T09:21:52.874 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:21:52.874 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-08-25T09:21:53.250 INFO:teuthology.task.kernel:Downloading kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on mon.b... 2023-08-25T09:21:53.250 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a6a69670761025e039dc5ae86619b0cf04c46dc2 2023-08-25T09:21:53.467 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/9/flavors/default/x86_64/ 2023-08-25T09:21:53.467 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/9/flavors/default/x86_64/ --input-file=- 2023-08-25T09:21:53.469 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi155.front.sepia.ceph.com to complete... 2023-08-25T09:22:04.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25 09:22:04 URL:https://4.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm [319273036/319273036] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-25T09:22:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:FINISHED --2023-08-25 09:22:04-- 2023-08-25T09:22:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:Total wall clock time: 14s 2023-08-25T09:22:04.658 INFO:teuthology.orchestra.run.smithi070.stderr:Downloaded: 1 files, 304M in 13s (22.6 MB/s) 2023-08-25T09:22:04.660 INFO:teuthology.task.kernel:Installing kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on mon.a... 2023-08-25T09:22:04.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-25T09:22:06.496 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying... ######################################## 2023-08-25T09:22:06.536 INFO:teuthology.orchestra.run.smithi070.stdout:Preparing... ######################################## 2023-08-25T09:22:06.585 INFO:teuthology.orchestra.run.smithi070.stdout:Updating / installing... 2023-08-25T09:22:12.593 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25 09:22:12 URL:https://4.chacra.ceph.com/r/kernel/testing/a6a69670761025e039dc5ae86619b0cf04c46dc2/centos/9/flavors/default/x86_64/kernel-6.5.0_rc7_ga6a696707610-1.x86_64.rpm [319273036/319273036] -> "/tmp/kernel.x86_64.rpm" [1] 2023-08-25T09:22:12.594 INFO:teuthology.orchestra.run.smithi155.stderr:FINISHED --2023-08-25 09:22:12-- 2023-08-25T09:22:12.594 INFO:teuthology.orchestra.run.smithi155.stderr:Total wall clock time: 19s 2023-08-25T09:22:12.594 INFO:teuthology.orchestra.run.smithi155.stderr:Downloaded: 1 files, 304M in 19s (16.3 MB/s) 2023-08-25T09:22:12.596 INFO:teuthology.task.kernel:Installing kernel a6a69670761025e039dc5ae86619b0cf04c46dc2 on mon.b... 2023-08-25T09:22:12.596 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-08-25T09:22:13.573 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-6.5.0_rc7_ga6a696707610-1 ######################################## 2023-08-25T09:22:14.472 INFO:teuthology.orchestra.run.smithi155.stdout:Verifying... ######################################## 2023-08-25T09:22:14.527 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing... ######################################## 2023-08-25T09:22:14.577 INFO:teuthology.orchestra.run.smithi155.stdout:Updating / installing... 2023-08-25T09:22:21.607 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-6.5.0_rc7_ga6a696707610-1 ######################################## 2023-08-25T09:22:53.446 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-25T09:22:53.447 DEBUG:teuthology.orchestra.run.smithi070:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-25T09:22:53.564 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.564 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/config-6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/boot/vmlinuz-6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto 2023-08-25T09:22:53.565 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-25T09:22:53.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-25T09:22:53.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-25T09:22:53.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-25T09:22:53.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-25T09:22:53.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-25T09:22:53.566 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-25T09:22:53.567 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-25T09:22:53.568 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-25T09:22:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm.ko 2023-08-25T09:22:53.570 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto 2023-08-25T09:22:53.570 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ansi_cprng.ko 2023-08-25T09:22:53.570 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/anubis.ko 2023-08-25T09:22:53.570 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/arc4.ko 2023-08-25T09:22:53.570 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_memcpy.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_pq.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_tx.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_xor.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/raid6test.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authenc.ko 2023-08-25T09:22:53.571 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authencesn.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blake2b_generic.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_common.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_generic.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/camellia_generic.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast5_generic.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast6_generic.ko 2023-08-25T09:22:53.572 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast_common.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ccm.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cmac.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crc32_generic.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_engine.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_user.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cts.ko 2023-08-25T09:22:53.573 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/des_generic.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/dh_generic.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/echainiv.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/essiv.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/fcrypt.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/khazad.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4.ko 2023-08-25T09:22:53.574 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4hc.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/md4.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/michael_mic.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcbc.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcrypt.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/rmd160.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/seed.ko 2023-08-25T09:22:53.575 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/serpent_generic.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tcrypt.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tea.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_common.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_generic.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/vmac.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/wp512.ko 2023-08-25T09:22:53.576 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xcbc.ko 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xor.ko 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xxhash_generic.ko 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_pad.ko 2023-08-25T09:22:53.577 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/custom_method.ko 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/ec_sys.ko 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/platform_profile.ko 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbs.ko 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbshc.ko 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/video.ko 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata 2023-08-25T09:22:53.578 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/acard-ahci.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ahci_platform.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ata_generic.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/libahci_platform.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_acpi.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ali.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_amd.ko 2023-08-25T09:22:53.579 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_artop.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atiixp.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atp867x.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd640.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd64x.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cypress.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_efar.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt366.ko 2023-08-25T09:22:53.580 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt37x.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it8213.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it821x.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_jmicron.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_marvell.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_mpiix.ko 2023-08-25T09:22:53.581 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_netcell.ko 2023-08-25T09:22:53.582 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ninja32.ko 2023-08-25T09:22:53.582 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87410.ko 2023-08-25T09:22:53.582 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87415.ko 2023-08-25T09:22:53.582 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_oldpiix.ko 2023-08-25T09:22:53.582 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_opti.ko 2023-08-25T09:22:53.582 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_optidma.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pcmcia.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_piccolo.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_rdc.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sch.ko 2023-08-25T09:22:53.583 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_serverworks.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sil680.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sis.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sl82c105.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_triflex.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_via.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pdc_adma.ko 2023-08-25T09:22:53.584 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_inic162x.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_mv.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_nv.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_promise.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_qstor.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil24.ko 2023-08-25T09:22:53.585 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sis.ko 2023-08-25T09:22:53.586 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_svw.ko 2023-08-25T09:22:53.586 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sx4.ko 2023-08-25T09:22:53.586 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_uli.ko 2023-08-25T09:22:53.586 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_via.ko 2023-08-25T09:22:53.586 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_vsc.ko 2023-08-25T09:22:53.586 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay/ks0108.ko 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma 2023-08-25T09:22:53.587 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma/bcma.ko 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe/aoe.ko 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/brd.ko 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd/drbd.ko 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/floppy.ko 2023-08-25T09:22:53.588 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/loop.ko 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/nbd.ko 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk/null_blk.ko 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/pktcdvd.ko 2023-08-25T09:22:53.589 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/rbd.ko 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/virtio_blk.ko 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkfront.ko 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram/zram.ko 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char 2023-08-25T09:22:53.590 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hangcheck-timer.ko 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/via-rng.ko 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-25T09:22:53.591 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-25T09:22:53.592 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/lp.ko 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave/mwave.ko 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ppdev.ko 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tlclk.ko 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm.ko 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-25T09:22:53.593 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/uv_mmtimer.ko 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/virtio_console.ko 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq 2023-08-25T09:22:53.594 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp 2023-08-25T09:22:53.595 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp.ko 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-25T09:22:53.596 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-aes.ko 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-sha.ko 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca/dca.ko 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq 2023-08-25T09:22:53.597 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac 2023-08-25T09:22:53.598 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/amd64_edac.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/e752x_edac.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/edac_mce_amd.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3000_edac.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3200_edac.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5100_edac.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5400_edac.ko 2023-08-25T09:22:53.599 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7300_edac.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7core_edac.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i82975x_edac.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/ie31200_edac.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/sb_edac.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/x38_edac.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/edd.ko 2023-08-25T09:22:53.600 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-ich.ko 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-appleir.ko 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-aureal.ko 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-axff.ko 2023-08-25T09:22:53.601 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-dr.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elecom.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elo.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-emsff.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gaff.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gt683r.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gyration.ko 2023-08-25T09:22:53.602 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtekff.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-hyperv.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-icade.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-keytouch.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-kye.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lcpower.ko 2023-08-25T09:22:53.603 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-led.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lenovo.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-multitouch.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-ortek.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-penmount.ko 2023-08-25T09:22:53.604 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-petalynx.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-picolcd.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-pl.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-plantronics.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-primax.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-rmi.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-common.ko 2023-08-25T09:22:53.605 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-25T09:22:53.606 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-saitek.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-samsung.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sjoy.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sony.ko 2023-08-25T09:22:53.607 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-speedlink.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-steelseries.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sunplus.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tivo.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tmff.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-topseed.ko 2023-08-25T09:22:53.608 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-twinhan.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-uclogic.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-waltop.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-wiimote.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-xinmo.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zpff.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zydacron.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/uhid.ko 2023-08-25T09:22:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/wacom.ko 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_balloon.ko 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_utils.ko 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_vmbus.ko 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru.ko 2023-08-25T09:22:53.610 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru3.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7414.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7418.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adc128d818.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1025.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1026.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1029.ko 2023-08-25T09:22:53.611 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1031.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm9240.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ads7828.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7410.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7411.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7462.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7470.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7475.ko 2023-08-25T09:22:53.612 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7x10.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/amc6821.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/applesmc.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asb100.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asc7621.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/atxp1.ko 2023-08-25T09:22:53.613 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/coretemp.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dme1737.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds1621.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds620.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc1403.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc6w201.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71805f.ko 2023-08-25T09:22:53.614 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71882fg.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f75375s.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fam15h_power.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fschmd.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g760a.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g762.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl518sm.ko 2023-08-25T09:22:53.615 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl520sm.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5500_temp.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5k_amb.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmaem.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmpex.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina209.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina2xx.ko 2023-08-25T09:22:53.616 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/it87.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k10temp.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k8temp.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lineage-pem.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm63.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm73.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm75.ko 2023-08-25T09:22:53.617 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm77.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm78.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm80.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm83.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm85.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm87.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm90.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm92.ko 2023-08-25T09:22:53.618 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm93.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95234.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95241.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95245.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc2945.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4151.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4215.ko 2023-08-25T09:22:53.619 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4222.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4245.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4260.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4261.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max16065.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1619.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1668.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max197.ko 2023-08-25T09:22:53.620 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6639.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6650.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6697.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/mcp3021.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6683.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775-core.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775.ko 2023-08-25T09:22:53.621 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct7802.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87360.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87427.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pcf8591.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-25T09:22:53.622 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-25T09:22:53.623 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/powr1220.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5627.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5636.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht15.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht21.ko 2023-08-25T09:22:53.624 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/shtc1.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sis5595.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47b397.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m1.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m192.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/thmc50.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp102.ko 2023-08-25T09:22:53.625 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp103.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp401.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp421.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via-cputemp.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via686a.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt1211.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt8231.ko 2023-08-25T09:22:53.626 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627ehf.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627hf.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83781d.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83791d.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83792d.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83793.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83795.ko 2023-08-25T09:22:53.627 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l785ts.ko 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l786ng.ko 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses 2023-08-25T09:22:53.628 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-25T09:22:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-25T09:22:53.630 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-25T09:22:53.631 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-dev.ko 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-mux.ko 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-smbus.ko 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-stub.ko 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core 2023-08-25T09:22:53.632 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_core.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw 2023-08-25T09:22:53.633 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca 2023-08-25T09:22:53.634 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-25T09:22:53.635 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-25T09:22:53.636 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/gameport.ko 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/lightning.ko 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/ns558.ko 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/joydev.ko 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse 2023-08-25T09:22:53.637 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/appletouch.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/bcm5974.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/cyapatp.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/sermouse.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-25T09:22:53.638 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/altera_ps2.ko 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/arc_ps2.ko 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/serio_raw.ko 2023-08-25T09:22:53.639 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/sparse-keymap.ko 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-blinkm.ko 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lm3530.ko 2023-08-25T09:22:53.640 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lp3944.ko 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-ss4200.ko 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-25T09:22:53.641 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache/bcache.ko 2023-08-25T09:22:53.642 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-bio-prison.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache-smq.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-crypt.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-delay.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-flakey.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-log-userspace.ko 2023-08-25T09:22:53.643 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-multipath.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-queue-length.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-raid.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-round-robin.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-service-time.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-switch.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-thin-pool.ko 2023-08-25T09:22:53.644 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-verity.ko 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/faulty.ko 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/linear.ko 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/multipath.ko 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid0.ko 2023-08-25T09:22:53.645 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid1.ko 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid10.ko 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid456.ko 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptbase.ko 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptctl.ko 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptfc.ko 2023-08-25T09:22:53.646 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptlan.ko 2023-08-25T09:22:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptsas.ko 2023-08-25T09:22:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptscsih.ko 2023-08-25T09:22:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptspi.ko 2023-08-25T09:22:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd 2023-08-25T09:22:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_ich.ko 2023-08-25T09:22:53.647 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_sch.ko 2023-08-25T09:22:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/mfd-core.ko 2023-08-25T09:22:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/sm501.ko 2023-08-25T09:22:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/viperboard.ko 2023-08-25T09:22:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/vx855.ko 2023-08-25T09:22:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/wl1273-core.ko 2023-08-25T09:22:53.648 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds9802als.ko 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds990x.ko 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/bh1770glc.ko 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710 2023-08-25T09:22:53.649 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710/cb710.ko 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/at24.ko 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/max6875.ko 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/enclosure.ko 2023-08-25T09:22:53.650 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/hpilo.ko 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29003.ko 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29020.ko 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei 2023-08-25T09:22:53.651 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-me.ko 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-txe.ko 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei.ko 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-25T09:22:53.652 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_7xx1.ko 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_core.ko 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tsl2550.ko 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_balloon.ko 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci 2023-08-25T09:22:53.653 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding/bonding.ko 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-25T09:22:53.654 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dummy.ko 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/eql.ko 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com 2023-08-25T09:22:53.655 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-25T09:22:53.656 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-25T09:22:53.657 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros 2023-08-25T09:22:53.658 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-25T09:22:53.659 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-25T09:22:53.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom 2023-08-25T09:22:53.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-25T09:22:53.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-25T09:22:53.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-25T09:22:53.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-25T09:22:53.660 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio 2023-08-25T09:22:53.661 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-25T09:22:53.662 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip 2023-08-25T09:22:53.663 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink 2023-08-25T09:22:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dnet.ko 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ethoc.ko 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/fealnx.ko 2023-08-25T09:22:53.665 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k 2023-08-25T09:22:53.666 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf 2023-08-25T09:22:53.667 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/jme.ko 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell 2023-08-25T09:22:53.668 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-25T09:22:53.669 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi 2023-08-25T09:22:53.670 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-25T09:22:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-25T09:22:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion 2023-08-25T09:22:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-25T09:22:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia 2023-08-25T09:22:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-25T09:22:53.671 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-25T09:22:53.672 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-25T09:22:53.673 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis 2023-08-25T09:22:53.674 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-25T09:22:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-25T09:22:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc 2023-08-25T09:22:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-25T09:22:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-25T09:22:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-25T09:22:53.675 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-25T09:22:53.676 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-25T09:22:53.677 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom 2023-08-25T09:22:53.678 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/geneve.ko 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ifb.ko 2023-08-25T09:22:53.679 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvlan.ko 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvtap.ko 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio.ko 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-25T09:22:53.680 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mii.ko 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/net_failover.ko 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/netconsole.ko 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/nlmon.ko 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs 2023-08-25T09:22:53.681 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/amd.ko 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-25T09:22:53.682 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm87xx.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/broadcom.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/cicada.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/davicom.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/dp83640.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/et1011c.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/icplus.ko 2023-08-25T09:22:53.683 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/lxt.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/marvell.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/micrel.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/national.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/phylink.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/qsemi.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/realtek.ko 2023-08-25T09:22:53.684 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/smsc.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/ste10Xp.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/vitesse.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_async.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-25T09:22:53.685 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppoe.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppox.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pptp.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slhc.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slip.ko 2023-08-25T09:22:53.686 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/sungem_phy.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tap.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_random.ko 2023-08-25T09:22:53.687 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tun.ko 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/veth.ko 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/virtio_net.ko 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan 2023-08-25T09:22:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan/vxlan.ko 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netfront.ko 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb/ntb.ko 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme 2023-08-25T09:22:53.689 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme-core.ko 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme.ko 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport.ko 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_cs.ko 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_pc.ko 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_serial.ko 2023-08-25T09:22:53.690 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/xen-pcifront.ko 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia 2023-08-25T09:22:53.691 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/i82092.ko 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/pd6729.ko 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86 2023-08-25T09:22:53.692 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acerhdf.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-25T09:22:53.693 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/hdaps.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel_ips.ko 2023-08-25T09:22:53.694 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-25T09:22:53.695 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi.ko 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients 2023-08-25T09:22:53.696 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps_parport.ko 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/pps_core.ko 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp.ko 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp_kvm.ko 2023-08-25T09:22:53.697 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-25T09:22:53.698 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-em3027.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-25T09:22:53.699 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-max6900.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-25T09:22:53.700 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-25T09:22:53.701 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-x1205.ko 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-9xxx.ko 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-sas.ko 2023-08-25T09:22:53.702 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-xxxx.ko 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/BusLogic.ko 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/a100u2w.ko 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/advansys.ko 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx 2023-08-25T09:22:53.703 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/am53c974.ko 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/atp870u.ko 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi 2023-08-25T09:22:53.704 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa/bfa.ko 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-25T09:22:53.705 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ch.ko 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-25T09:22:53.706 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dc395x.ko 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-25T09:22:53.707 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dmx3191d.ko 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esp_scsi.ko 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-25T09:22:53.708 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic/fnic.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hpsa.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hptiop.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hv_storvsc.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/initio.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ipr.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ips.ko 2023-08-25T09:22:53.709 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci 2023-08-25T09:22:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci/isci.ko 2023-08-25T09:22:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-25T09:22:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-25T09:22:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc 2023-08-25T09:22:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc/libfc.ko 2023-08-25T09:22:53.710 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi.ko 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas/libsas.ko 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid 2023-08-25T09:22:53.711 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid.ko 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas 2023-08-25T09:22:53.712 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvumi.ko 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pmcraid.ko 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla1280.ko 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-25T09:22:53.713 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/raid_class.ko 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_debug.ko 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-25T09:22:53.714 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-25T09:22:53.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-25T09:22:53.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-25T09:22:53.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ses.ko 2023-08-25T09:22:53.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/st.ko 2023-08-25T09:22:53.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/stex.ko 2023-08-25T09:22:53.715 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/virtio_scsi.ko 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/wd719x.ko 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb/ssb.ko 2023-08-25T09:22:53.716 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_file.ko 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_iblock.ko 2023-08-25T09:22:53.717 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_mod.ko 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_pscsi.ko 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_user.ko 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal 2023-08-25T09:22:53.718 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-25T09:22:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-25T09:22:53.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-25T09:22:53.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-25T09:22:53.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-25T09:22:53.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-25T09:22:53.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-25T09:22:53.720 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_gsm.ko 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_hdlc.ko 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/nozomi.ko 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial 2023-08-25T09:22:53.721 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/arc_uart.ko 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/synclink_gt.ko 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs 2023-08-25T09:22:53.722 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio.ko 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_aec.ko 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_cif.ko 2023-08-25T09:22:53.723 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_pci_generic.ko 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_sercos3.ko 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-acm.ko 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usblp.ko 2023-08-25T09:22:53.724 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usbtmc.ko 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/mdc800.ko 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/microtek.ko 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/adutux.ko 2023-08-25T09:22:53.725 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/appledisplay.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi26.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi62.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ezusb.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/idmouse.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/iowarrior.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-25T09:22:53.726 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ldusb.ko 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/legousbtower.ko 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usb3503.ko 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usblcd.ko 2023-08-25T09:22:53.727 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/uss720.ko 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/yurex.ko 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy/phy-generic.ko 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/aircable.ko 2023-08-25T09:22:53.728 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ark3116.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ch341.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cp210x.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cyberjack.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/empeg.ko 2023-08-25T09:22:53.729 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_ti.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipaq.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipw.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ir-usb.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-25T09:22:53.730 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mct_u232.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7720.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7840.ko 2023-08-25T09:22:53.731 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/navman.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/omninet.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/opticon.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/option.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/oti6858.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/pl2303.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcaux.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcserial.ko 2023-08-25T09:22:53.732 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/quatech2.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/safe_serial.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/sierra.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ssu100.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/symbolserial.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-25T09:22:53.733 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_debug.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/visor.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/whiteheat.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/uas.ko 2023-08-25T09:22:53.734 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-25T09:22:53.735 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-karma.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/usb-storage.ko 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip 2023-08-25T09:22:53.736 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-25T09:22:53.737 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio.ko 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost.ko 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_net.ko 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_scsi.ko 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video 2023-08-25T09:22:53.738 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/apple_bl.ko 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/lcd.ko 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio.ko 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_balloon.ko 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_mmio.ko 2023-08-25T09:22:53.739 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci.ko 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_ring.ko 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2482.ko 2023-08-25T09:22:53.740 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2490.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-25T09:22:53.741 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/wire.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-25T09:22:53.742 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/hpwdt.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/i6300esb.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ib700wdt.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ibmasr.ko 2023-08-25T09:22:53.743 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it87_wdt.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/machzwd.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/nv_tco.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-25T09:22:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/softdog.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/via_wdt.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-25T09:22:53.745 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/wdt_pci.ko 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/xen_wdt.ko 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-evtchn.ko 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntalloc.ko 2023-08-25T09:22:53.746 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntdev.ko 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-privcmd.ko 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p 2023-08-25T09:22:53.747 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p/9p.ko 2023-08-25T09:22:53.748 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs 2023-08-25T09:22:53.748 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs/affs.ko 2023-08-25T09:22:53.748 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs 2023-08-25T09:22:53.748 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs/befs.ko 2023-08-25T09:22:53.748 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/binfmt_misc.ko 2023-08-25T09:22:53.748 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs/btrfs.ko 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles/cachefiles.ko 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph/ceph.ko 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda 2023-08-25T09:22:53.749 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda/coda.ko 2023-08-25T09:22:53.750 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs 2023-08-25T09:22:53.750 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs/cramfs.ko 2023-08-25T09:22:53.750 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm 2023-08-25T09:22:53.750 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm/dlm.ko 2023-08-25T09:22:53.750 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs/f2fs.ko 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/fat.ko 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/msdos.ko 2023-08-25T09:22:53.751 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/vfat.ko 2023-08-25T09:22:53.752 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache 2023-08-25T09:22:53.752 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache/fscache.ko 2023-08-25T09:22:53.752 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse 2023-08-25T09:22:53.752 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/cuse.ko 2023-08-25T09:22:53.752 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/fuse.ko 2023-08-25T09:22:53.752 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2/gfs2.ko 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs/hfs.ko 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus/hfsplus.ko 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs 2023-08-25T09:22:53.753 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs/isofs.ko 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs/jfs.ko 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd/lockd.ko 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix/minix.ko 2023-08-25T09:22:53.754 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs/netfs.ko 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout 2023-08-25T09:22:53.755 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfs.ko 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv3.ko 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv4.ko 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/grace.ko 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/nfs_acl.ko 2023-08-25T09:22:53.756 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd/nfsd.ko 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2/nilfs2.ko 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-celtic.ko 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-centeuro.ko 2023-08-25T09:22:53.757 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-croatian.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-cyrillic.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-gaelic.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-greek.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-iceland.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-inuit.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-roman.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-romanian.ko 2023-08-25T09:22:53.758 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-turkish.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1250.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1251.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1255.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp737.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp775.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp850.ko 2023-08-25T09:22:53.759 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp852.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp855.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp857.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp860.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp861.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp862.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp863.ko 2023-08-25T09:22:53.760 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp864.ko 2023-08-25T09:22:53.761 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp865.ko 2023-08-25T09:22:53.761 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp866.ko 2023-08-25T09:22:53.761 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp869.ko 2023-08-25T09:22:53.761 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp874.ko 2023-08-25T09:22:53.761 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp932.ko 2023-08-25T09:22:53.761 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp936.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp949.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp950.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_euc-jp.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-1.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-13.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-14.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-15.ko 2023-08-25T09:22:53.762 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-2.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-3.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-4.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-5.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-6.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-7.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-9.ko 2023-08-25T09:22:53.763 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-r.ko 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-ru.ko 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-u.ko 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_utf8.ko 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-25T09:22:53.764 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2.ko 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-25T09:22:53.765 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs/overlay.ko 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore/ramoops.ko 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs/reiserfs.ko 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs 2023-08-25T09:22:53.766 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs/romfs.ko 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client/cifs.ko 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_arc4.ko 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_md4.ko 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs 2023-08-25T09:22:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs/squashfs.ko 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv/sysv.ko 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf/udf.ko 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs/ufs.ko 2023-08-25T09:22:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs/xfs.ko 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu/rcutorture.ko 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/torture.ko 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace 2023-08-25T09:22:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/asn1_encoder.ko 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc-itu-t.ko 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc8.ko 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libarc4.ko 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libdes.ko 2023-08-25T09:22:53.770 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/libcrc32c.ko 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lru_cache.ko 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4_compress.ko 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4hc_compress.ko 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math/cordic.ko 2023-08-25T09:22:53.771 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6/raid6_pq.ko 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_bm.ko 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_fsm.ko 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_kmp.ko 2023-08-25T09:22:53.772 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd/zstd_compress.ko 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm/hwpoison-inject.ko 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/6lowpan.ko 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_dest.ko 2023-08-25T09:22:53.773 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_fragment.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_hop.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_mobility.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_routing.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_udp.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/garp.ko 2023-08-25T09:22:53.774 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/mrp.ko 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/p8022.ko 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/psnap.ko 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/stp.ko 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q/8021q.ko 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet.ko 2023-08-25T09:22:53.775 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_fd.ko 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_rdma.ko 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_virtio.ko 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv/batman-adv.ko 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/br_netfilter.ko 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/bridge.ko 2023-08-25T09:22:53.776 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-25T09:22:53.777 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-25T09:22:53.778 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtables.ko 2023-08-25T09:22:53.779 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph/libceph.ko 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/failover.ko 2023-08-25T09:22:53.780 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/pktgen.ko 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp.ko 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_diag.ko 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv4.ko 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv6.ko 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver/dns_resolver.ko 2023-08-25T09:22:53.781 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/dsa_core.ko 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_brcm.ko 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_dsa.ko 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_none.ko 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_trailer.ko 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154 2023-08-25T09:22:53.782 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154.ko 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ah4.ko 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/esp4.ko 2023-08-25T09:22:53.783 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/fou.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/gre.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/inet_diag.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_gre.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_tunnel.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_vti.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipcomp.ko 2023-08-25T09:22:53.784 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipip.ko 2023-08-25T09:22:53.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter 2023-08-25T09:22:53.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-25T09:22:53.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-25T09:22:53.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-25T09:22:53.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-25T09:22:53.785 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-25T09:22:53.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-25T09:22:53.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-25T09:22:53.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-25T09:22:53.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-25T09:22:53.786 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-25T09:22:53.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-25T09:22:53.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-25T09:22:53.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-25T09:22:53.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-25T09:22:53.787 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-25T09:22:53.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-25T09:22:53.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-25T09:22:53.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-25T09:22:53.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-25T09:22:53.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-25T09:22:53.788 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_bic.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_dctcp.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_diag.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_highspeed.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_htcp.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_hybla.ko 2023-08-25T09:22:53.789 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_illinois.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_lp.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_scalable.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_vegas.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_veno.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_westwood.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_yeah.ko 2023-08-25T09:22:53.790 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tunnel4.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_diag.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_tunnel.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ah6.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/esp6.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/fou6.ko 2023-08-25T09:22:53.791 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_tunnel.ko 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_vti.ko 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ipcomp6.ko 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-25T09:22:53.792 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-25T09:22:53.793 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-25T09:22:53.794 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/sit.ko 2023-08-25T09:22:53.795 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/tunnel6.ko 2023-08-25T09:22:53.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-25T09:22:53.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key 2023-08-25T09:22:53.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key/af_key.ko 2023-08-25T09:22:53.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp 2023-08-25T09:22:53.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_core.ko 2023-08-25T09:22:53.796 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_eth.ko 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip.ko 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip6.ko 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_netlink.ko 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ppp.ko 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc 2023-08-25T09:22:53.797 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc/llc.ko 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154/mac802154.ko 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls/mpls_gso.ko 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset 2023-08-25T09:22:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-25T09:22:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-25T09:22:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-25T09:22:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-25T09:22:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-25T09:22:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-25T09:22:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-25T09:22:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-25T09:22:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-25T09:22:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conncount.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-25T09:22:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-25T09:22:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-25T09:22:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-25T09:22:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-25T09:22:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-25T09:22:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-25T09:22:53.804 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table.ko 2023-08-25T09:22:53.805 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-25T09:22:53.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_log_syslog.ko 2023-08-25T09:22:53.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat.ko 2023-08-25T09:22:53.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-25T09:22:53.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-25T09:22:53.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_irc.ko 2023-08-25T09:22:53.806 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_sip.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_tables.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_log.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-25T09:22:53.807 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_chain_nat.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_compat.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_ct.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fib.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-25T09:22:53.808 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_hash.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_limit.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_log.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_masq.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_nat.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_numgen.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_queue.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_quota.ko 2023-08-25T09:22:53.809 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_redir.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject_inet.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_socket.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_tunnel.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_AUDIT.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-25T09:22:53.810 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CT.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_DSCP.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HL.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HMARK.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LED.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LOG.ko 2023-08-25T09:22:53.811 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-25T09:22:53.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NETMAP.ko 2023-08-25T09:22:53.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFLOG.ko 2023-08-25T09:22:53.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-25T09:22:53.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_RATEEST.ko 2023-08-25T09:22:53.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-25T09:22:53.812 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_SECMARK.ko 2023-08-25T09:22:53.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-25T09:22:53.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-25T09:22:53.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TEE.ko 2023-08-25T09:22:53.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TPROXY.ko 2023-08-25T09:22:53.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TRACE.ko 2023-08-25T09:22:53.813 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_addrtype.ko 2023-08-25T09:22:53.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_bpf.ko 2023-08-25T09:22:53.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cgroup.ko 2023-08-25T09:22:53.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cluster.ko 2023-08-25T09:22:53.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_comment.ko 2023-08-25T09:22:53.814 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connbytes.ko 2023-08-25T09:22:53.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlabel.ko 2023-08-25T09:22:53.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlimit.ko 2023-08-25T09:22:53.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connmark.ko 2023-08-25T09:22:53.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_conntrack.ko 2023-08-25T09:22:53.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cpu.ko 2023-08-25T09:22:53.815 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dccp.ko 2023-08-25T09:22:53.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_devgroup.ko 2023-08-25T09:22:53.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dscp.ko 2023-08-25T09:22:53.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ecn.ko 2023-08-25T09:22:53.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_esp.ko 2023-08-25T09:22:53.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hashlimit.ko 2023-08-25T09:22:53.816 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_helper.ko 2023-08-25T09:22:53.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hl.ko 2023-08-25T09:22:53.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipcomp.ko 2023-08-25T09:22:53.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_iprange.ko 2023-08-25T09:22:53.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipvs.ko 2023-08-25T09:22:53.817 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_l2tp.ko 2023-08-25T09:22:53.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_length.ko 2023-08-25T09:22:53.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_limit.ko 2023-08-25T09:22:53.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mac.ko 2023-08-25T09:22:53.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mark.ko 2023-08-25T09:22:53.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_multiport.ko 2023-08-25T09:22:53.818 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nat.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nfacct.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_osf.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_owner.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_physdev.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_pkttype.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_policy.ko 2023-08-25T09:22:53.819 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_quota.ko 2023-08-25T09:22:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_rateest.ko 2023-08-25T09:22:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_realm.ko 2023-08-25T09:22:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_recent.ko 2023-08-25T09:22:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_sctp.ko 2023-08-25T09:22:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_set.ko 2023-08-25T09:22:53.820 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_socket.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_state.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_statistic.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_string.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_tcpmss.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_time.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_u32.ko 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink 2023-08-25T09:22:53.821 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink/netlink_diag.ko 2023-08-25T09:22:53.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh 2023-08-25T09:22:53.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh/nsh.ko 2023-08-25T09:22:53.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch 2023-08-25T09:22:53.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/openvswitch.ko 2023-08-25T09:22:53.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-geneve.ko 2023-08-25T09:22:53.822 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-gre.ko 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-vxlan.ko 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet/af_packet_diag.ko 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds.ko 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_rdma.ko 2023-08-25T09:22:53.823 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_tcp.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_csum.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_gact.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_ipt.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_mirred.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_nat.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_pedit.ko 2023-08-25T09:22:53.824 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_police.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_simple.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_skbedit.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_vlan.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_basic.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_bpf.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_flow.ko 2023-08-25T09:22:53.825 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_fw.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_route.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_u32.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_cmp.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_ipset.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_meta.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_nbyte.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_text.ko 2023-08-25T09:22:53.826 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_u32.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_choke.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_codel.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_drr.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_fq.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_gred.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hfsc.ko 2023-08-25T09:22:53.827 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hhf.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_htb.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_ingress.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio_lib.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_multiq.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_netem.ko 2023-08-25T09:22:53.828 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_pie.ko 2023-08-25T09:22:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_plug.ko 2023-08-25T09:22:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_prio.ko 2023-08-25T09:22:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_qfq.ko 2023-08-25T09:22:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_red.ko 2023-08-25T09:22:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfb.ko 2023-08-25T09:22:53.829 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfq.ko 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_tbf.ko 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_teql.ko 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp.ko 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp_diag.ko 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss 2023-08-25T09:22:53.830 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/sunrpc.ko 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/diag.ko 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/tipc.ko 2023-08-25T09:22:53.831 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix/unix_diag.ko 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock.ko 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-25T09:22:53.832 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys 2023-08-25T09:22:53.833 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys/trusted.ko 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib/irqbypass.ko 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias.bin 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.alias.bin 2023-08-25T09:22:53.834 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.bin 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.modinfo 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep.bin 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.devname 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.order 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.softdep 2023-08-25T09:22:53.835 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols 2023-08-25T09:22:53.836 INFO:teuthology.orchestra.run.smithi070.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols.bin 2023-08-25T09:22:53.837 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.837 DEBUG:teuthology.orchestra.run.smithi070:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-25T09:22:53.862 INFO:teuthology.orchestra.run.smithi070.stdout:postinstall scriptlet (using /bin/sh): 2023-08-25T09:22:53.863 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-ga6a696707610 -a -r /boot/System.map-6.5.0-rc7-ga6a696707610 ]; then 2023-08-25T09:22:53.863 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:22:53.863 INFO:teuthology.orchestra.run.smithi070.stdout:cp /boot/System.map-6.5.0-rc7-ga6a696707610 /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:22:53.863 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.863 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/installkernel 6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:22:53.864 INFO:teuthology.orchestra.run.smithi070.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:22:53.864 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-08-25T09:22:53.864 INFO:teuthology.orchestra.run.smithi070.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-25T09:22:53.864 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-25T09:22:53.864 INFO:teuthology.orchestra.run.smithi070.stdout:new-kernel-pkg --remove 6.5.0-rc7-ga6a696707610 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-ga6a696707610.img 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-install remove 6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.865 INFO:teuthology.orchestra.run.smithi070.stdout:fi 2023-08-25T09:22:53.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2023-08-25T09:22:53.958 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2023-08-25T09:22:53.959 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2023-08-25T09:22:53.959 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2023-08-25T09:22:53.960 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 53.el9 2023-08-25T09:22:53.960 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2023-08-25T09:22:53.960 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-25T09:22:53.960 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2023-08-25T09:22:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8551923 2023-08-25T09:22:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2023-08-25T09:22:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-25T09:22:53.961 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-25T09:22:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-25T09:22:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-25T09:22:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2023-08-25T09:22:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2023-08-25T09:22:53.962 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-25T09:22:53.963 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2023-08-25T09:22:53.963 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2023-08-25T09:22:53.963 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:22:53.963 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-25T09:22:53.964 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-25T09:22:53.964 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-25T09:22:53.964 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2023-08-25T09:22:53.965 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:22:53.965 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2023-08-25T09:22:53.966 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-25T09:22:53.966 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 6.5.0-rc7-ga6a696707610 2023-08-25T09:22:53.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-25T09:22:54.558 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2023-08-25T09:22:55.859 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-25T09:22:55.900 INFO:teuthology.orchestra.run.smithi070.stderr:done 2023-08-25T09:22:55.902 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2023-08-25T09:22:55.918 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.1QJuBWdcfs 2023-08-25T09:22:55.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1QJuBWdcfs 2023-08-25T09:22:55.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.1QJuBWdcfs 2023-08-25T09:22:56.140 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.1QJuBWdcfs is 6KB 2023-08-25T09:22:56.188 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.1QJuBWdcfs 2023-08-25T09:22:56.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2023-08-25T09:22:56.266 INFO:teuthology.orchestra.run.smithi070.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-25T09:22:56.266 INFO:teuthology.orchestra.run.smithi070.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-25T09:22:56.267 INFO:teuthology.orchestra.run.smithi070.stdout:e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc7-ga6a696707610.conf 2023-08-25T09:22:56.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc7-ga6a696707610 2023-08-25T09:22:56.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2023-08-25T09:23:01.475 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-08-25T09:23:01.476 DEBUG:teuthology.orchestra.run.smithi155:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-08-25T09:23:01.548 INFO:teuthology.orchestra.run.smithi155.stdout:/boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.548 INFO:teuthology.orchestra.run.smithi155.stdout:/boot/config-6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.548 INFO:teuthology.orchestra.run.smithi155.stdout:/boot/vmlinuz-6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-08-25T09:23:01.549 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crc32c-intel.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-08-25T09:23:01.550 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-avx2.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-08-25T09:23:01.551 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce 2023-08-25T09:23:01.552 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-amd.ko 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm-intel.ko 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/arch/x86/kvm/kvm.ko 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ansi_cprng.ko 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/anubis.ko 2023-08-25T09:23:01.553 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/arc4.ko 2023-08-25T09:23:01.554 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx 2023-08-25T09:23:01.554 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_memcpy.ko 2023-08-25T09:23:01.554 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_pq.ko 2023-08-25T09:23:01.554 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_raid6_recov.ko 2023-08-25T09:23:01.555 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_tx.ko 2023-08-25T09:23:01.555 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/async_xor.ko 2023-08-25T09:23:01.555 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/async_tx/raid6test.ko 2023-08-25T09:23:01.555 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authenc.ko 2023-08-25T09:23:01.556 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/authencesn.ko 2023-08-25T09:23:01.556 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blake2b_generic.ko 2023-08-25T09:23:01.556 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_common.ko 2023-08-25T09:23:01.557 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/blowfish_generic.ko 2023-08-25T09:23:01.557 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/camellia_generic.ko 2023-08-25T09:23:01.557 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast5_generic.ko 2023-08-25T09:23:01.558 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast6_generic.ko 2023-08-25T09:23:01.558 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cast_common.ko 2023-08-25T09:23:01.558 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/ccm.ko 2023-08-25T09:23:01.558 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cmac.ko 2023-08-25T09:23:01.558 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crc32_generic.ko 2023-08-25T09:23:01.559 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_engine.ko 2023-08-25T09:23:01.559 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/crypto_user.ko 2023-08-25T09:23:01.559 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/cts.ko 2023-08-25T09:23:01.559 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/des_generic.ko 2023-08-25T09:23:01.560 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/dh_generic.ko 2023-08-25T09:23:01.560 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/echainiv.ko 2023-08-25T09:23:01.560 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/essiv.ko 2023-08-25T09:23:01.560 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/fcrypt.ko 2023-08-25T09:23:01.561 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/khazad.ko 2023-08-25T09:23:01.561 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4.ko 2023-08-25T09:23:01.561 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/lz4hc.ko 2023-08-25T09:23:01.561 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/md4.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/michael_mic.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcbc.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/pcrypt.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/rmd160.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/seed.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/serpent_generic.ko 2023-08-25T09:23:01.562 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tcrypt.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/tea.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_common.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/twofish_generic.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/vmac.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/wp512.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xcbc.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xor.ko 2023-08-25T09:23:01.563 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/crypto/xxhash_generic.ko 2023-08-25T09:23:01.564 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers 2023-08-25T09:23:01.564 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi 2023-08-25T09:23:01.564 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_ipmi.ko 2023-08-25T09:23:01.564 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/acpi_pad.ko 2023-08-25T09:23:01.564 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/custom_method.ko 2023-08-25T09:23:01.564 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/ec_sys.ko 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/platform_profile.ko 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbs.ko 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/sbshc.ko 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/acpi/video.ko 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/acard-ahci.ko 2023-08-25T09:23:01.565 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ahci_platform.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/ata_generic.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/libahci_platform.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_acpi.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ali.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_amd.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_artop.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atiixp.ko 2023-08-25T09:23:01.566 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_atp867x.ko 2023-08-25T09:23:01.567 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd640.ko 2023-08-25T09:23:01.567 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cmd64x.ko 2023-08-25T09:23:01.567 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_cypress.ko 2023-08-25T09:23:01.567 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_efar.ko 2023-08-25T09:23:01.567 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt366.ko 2023-08-25T09:23:01.568 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt37x.ko 2023-08-25T09:23:01.568 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x2n.ko 2023-08-25T09:23:01.568 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_hpt3x3.ko 2023-08-25T09:23:01.568 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it8213.ko 2023-08-25T09:23:01.568 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_it821x.ko 2023-08-25T09:23:01.568 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_jmicron.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_marvell.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_mpiix.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_netcell.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ninja32.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87410.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_ns87415.ko 2023-08-25T09:23:01.569 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_oldpiix.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_opti.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_optidma.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pcmcia.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc2027x.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_piccolo.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_rdc.ko 2023-08-25T09:23:01.570 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sch.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_serverworks.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sil680.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sis.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_sl82c105.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_triflex.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pata_via.ko 2023-08-25T09:23:01.571 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/pdc_adma.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_inic162x.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_mv.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_nv.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_promise.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_qstor.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sil24.ko 2023-08-25T09:23:01.572 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sis.ko 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_svw.ko 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_sx4.ko 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_uli.ko 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_via.ko 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ata/sata_vsc.ko 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay 2023-08-25T09:23:01.573 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/auxdisplay/ks0108.ko 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-i2c.ko 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/base/regmap/regmap-mmio.ko 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/bcma/bcma.ko 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block 2023-08-25T09:23:01.574 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/aoe/aoe.ko 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/brd.ko 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/drbd/drbd.ko 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/floppy.ko 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/loop.ko 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx 2023-08-25T09:23:01.575 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/nbd.ko 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/null_blk/null_blk.ko 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/pktcdvd.ko 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/rbd.ko 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/virtio_blk.ko 2023-08-25T09:23:01.576 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/xen-blkfront.ko 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/block/zram/zram.ko 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hangcheck-timer.ko 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random 2023-08-25T09:23:01.577 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/amd-rng.ko 2023-08-25T09:23:01.578 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/intel-rng.ko 2023-08-25T09:23:01.578 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-08-25T09:23:01.578 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/via-rng.ko 2023-08-25T09:23:01.578 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/hw_random/virtio-rng.ko 2023-08-25T09:23:01.578 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi 2023-08-25T09:23:01.578 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_si.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/lp.ko 2023-08-25T09:23:01.579 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/mwave/mwave.ko 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/ppdev.ko 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tlclk.ko 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm.ko 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_atmel.ko 2023-08-25T09:23:01.580 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_infineon.ko 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_nsc.ko 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis.ko 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/uv_mmtimer.ko 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/char/virtio_console.ko 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq 2023-08-25T09:23:01.581 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/p4-clockmod.ko 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/powernow-k8.ko 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/cpufreq/speedstep-lib.ko 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp 2023-08-25T09:23:01.582 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/ccp/ccp.ko 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-08-25T09:23:01.583 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-08-25T09:23:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-aes.ko 2023-08-25T09:23:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/padlock-sha.ko 2023-08-25T09:23:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio 2023-08-25T09:23:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-08-25T09:23:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dca/dca.ko 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/dw/dw_dmac.ko 2023-08-25T09:23:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat 2023-08-25T09:23:01.586 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/dma/ioat/ioatdma.ko 2023-08-25T09:23:01.586 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac 2023-08-25T09:23:01.586 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/amd64_edac.ko 2023-08-25T09:23:01.586 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/e752x_edac.ko 2023-08-25T09:23:01.587 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/edac_mce_amd.ko 2023-08-25T09:23:01.587 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3000_edac.ko 2023-08-25T09:23:01.587 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i3200_edac.ko 2023-08-25T09:23:01.588 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5100_edac.ko 2023-08-25T09:23:01.588 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i5400_edac.ko 2023-08-25T09:23:01.588 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7300_edac.ko 2023-08-25T09:23:01.588 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i7core_edac.ko 2023-08-25T09:23:01.588 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/i82975x_edac.ko 2023-08-25T09:23:01.589 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/ie31200_edac.ko 2023-08-25T09:23:01.589 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/sb_edac.ko 2023-08-25T09:23:01.589 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/edac/x38_edac.ko 2023-08-25T09:23:01.589 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware 2023-08-25T09:23:01.589 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/edd.ko 2023-08-25T09:23:01.589 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/firmware/iscsi_ibft.ko 2023-08-25T09:23:01.590 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio 2023-08-25T09:23:01.590 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-ich.ko 2023-08-25T09:23:01.590 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/gpio/gpio-viperboard.ko 2023-08-25T09:23:01.590 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-appleir.ko 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-aureal.ko 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-axff.ko 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-dr.ko 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elecom.ko 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-elo.ko 2023-08-25T09:23:01.591 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-emsff.ko 2023-08-25T09:23:01.592 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gaff.ko 2023-08-25T09:23:01.592 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gt683r.ko 2023-08-25T09:23:01.592 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-gyration.ko 2023-08-25T09:23:01.592 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-kbd.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtek-mouse.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-holtekff.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-hyperv.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-icade.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-keytouch.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-kye.ko 2023-08-25T09:23:01.593 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lcpower.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-led.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-lenovo.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-dj.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-multitouch.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-ortek.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-penmount.ko 2023-08-25T09:23:01.594 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-petalynx.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-picolcd.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-pl.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-plantronics.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-primax.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-rmi.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-arvo.ko 2023-08-25T09:23:01.595 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-common.ko 2023-08-25T09:23:01.596 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-isku.ko 2023-08-25T09:23:01.596 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kone.ko 2023-08-25T09:23:01.596 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-08-25T09:23:01.596 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-konepure.ko 2023-08-25T09:23:01.597 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-08-25T09:23:01.597 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-lua.ko 2023-08-25T09:23:01.597 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-pyra.ko 2023-08-25T09:23:01.597 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-ryos.ko 2023-08-25T09:23:01.598 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat-savu.ko 2023-08-25T09:23:01.598 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-roccat.ko 2023-08-25T09:23:01.598 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-saitek.ko 2023-08-25T09:23:01.598 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-samsung.ko 2023-08-25T09:23:01.598 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sensor-hub.ko 2023-08-25T09:23:01.599 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sjoy.ko 2023-08-25T09:23:01.599 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sony.ko 2023-08-25T09:23:01.599 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-speedlink.ko 2023-08-25T09:23:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-steelseries.ko 2023-08-25T09:23:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-sunplus.ko 2023-08-25T09:23:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-thrustmaster.ko 2023-08-25T09:23:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tivo.ko 2023-08-25T09:23:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-tmff.ko 2023-08-25T09:23:01.600 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-topseed.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-twinhan.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-uclogic.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-waltop.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-wiimote.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-xinmo.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zpff.ko 2023-08-25T09:23:01.601 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/hid-zydacron.ko 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/uhid.ko 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hid/wacom.ko 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_balloon.ko 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_utils.ko 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hv/hv_vmbus.ko 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon 2023-08-25T09:23:01.602 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/abituguru3.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/acpi_power_meter.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7414.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ad7418.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adc128d818.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1025.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1026.ko 2023-08-25T09:23:01.603 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1029.ko 2023-08-25T09:23:01.604 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm1031.ko 2023-08-25T09:23:01.604 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adm9240.ko 2023-08-25T09:23:01.604 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ads7828.ko 2023-08-25T09:23:01.604 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7410.ko 2023-08-25T09:23:01.604 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7411.ko 2023-08-25T09:23:01.605 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7462.ko 2023-08-25T09:23:01.605 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7470.ko 2023-08-25T09:23:01.605 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7475.ko 2023-08-25T09:23:01.605 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/adt7x10.ko 2023-08-25T09:23:01.605 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/amc6821.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/applesmc.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asb100.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asc7621.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/asus_atk0110.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/atxp1.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/coretemp.ko 2023-08-25T09:23:01.606 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-08-25T09:23:01.607 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/dme1737.ko 2023-08-25T09:23:01.607 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds1621.ko 2023-08-25T09:23:01.607 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ds620.ko 2023-08-25T09:23:01.607 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc1403.ko 2023-08-25T09:23:01.607 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/emc6w201.ko 2023-08-25T09:23:01.607 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71805f.ko 2023-08-25T09:23:01.608 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f71882fg.ko 2023-08-25T09:23:01.608 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/f75375s.ko 2023-08-25T09:23:01.608 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fam15h_power.ko 2023-08-25T09:23:01.608 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/fschmd.ko 2023-08-25T09:23:01.608 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g760a.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/g762.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl518sm.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/gl520sm.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/hwmon-vid.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5500_temp.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/i5k_amb.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmaem.ko 2023-08-25T09:23:01.609 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ibmpex.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina209.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ina2xx.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/it87.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k10temp.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/k8temp.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lineage-pem.ko 2023-08-25T09:23:01.610 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm63.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm73.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm75.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm77.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm78.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm80.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm83.ko 2023-08-25T09:23:01.611 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm85.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm87.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm90.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm92.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm93.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95234.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95241.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/lm95245.ko 2023-08-25T09:23:01.612 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc2945.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4151.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4215.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4222.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4245.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4260.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/ltc4261.ko 2023-08-25T09:23:01.613 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max16065.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1619.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max1668.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max197.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6639.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6650.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/max6697.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/mcp3021.ko 2023-08-25T09:23:01.614 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6683.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775-core.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct6775.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/nct7802.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87360.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pc87427.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pcf8591.ko 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus 2023-08-25T09:23:01.615 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-08-25T09:23:01.616 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-08-25T09:23:01.616 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-08-25T09:23:01.616 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max16064.ko 2023-08-25T09:23:01.616 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max34440.ko 2023-08-25T09:23:01.616 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/max8688.ko 2023-08-25T09:23:01.616 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/powr1220.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5627.ko 2023-08-25T09:23:01.617 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch5636.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sch56xx-common.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht15.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sht21.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/shtc1.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/sis5595.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47b397.ko 2023-08-25T09:23:01.618 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m1.ko 2023-08-25T09:23:01.619 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/smsc47m192.ko 2023-08-25T09:23:01.619 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/thmc50.ko 2023-08-25T09:23:01.619 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp102.ko 2023-08-25T09:23:01.619 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp103.ko 2023-08-25T09:23:01.619 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp401.ko 2023-08-25T09:23:01.619 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/tmp421.ko 2023-08-25T09:23:01.620 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via-cputemp.ko 2023-08-25T09:23:01.620 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/via686a.ko 2023-08-25T09:23:01.620 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt1211.ko 2023-08-25T09:23:01.620 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/vt8231.ko 2023-08-25T09:23:01.620 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627ehf.ko 2023-08-25T09:23:01.620 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83627hf.ko 2023-08-25T09:23:01.621 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83781d.ko 2023-08-25T09:23:01.621 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83791d.ko 2023-08-25T09:23:01.621 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83792d.ko 2023-08-25T09:23:01.621 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83793.ko 2023-08-25T09:23:01.621 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83795.ko 2023-08-25T09:23:01.621 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l785ts.ko 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/hwmon/w83l786ng.ko 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses 2023-08-25T09:23:01.622 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-08-25T09:23:01.623 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-08-25T09:23:01.623 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-08-25T09:23:01.623 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-08-25T09:23:01.624 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-08-25T09:23:01.624 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-08-25T09:23:01.624 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-08-25T09:23:01.624 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-08-25T09:23:01.625 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-i801.ko 2023-08-25T09:23:01.625 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-isch.ko 2023-08-25T09:23:01.625 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-08-25T09:23:01.625 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-08-25T09:23:01.626 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-08-25T09:23:01.626 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-parport.ko 2023-08-25T09:23:01.626 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-08-25T09:23:01.627 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-08-25T09:23:01.627 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-08-25T09:23:01.627 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-08-25T09:23:01.627 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-08-25T09:23:01.628 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-08-25T09:23:01.628 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-via.ko 2023-08-25T09:23:01.628 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-08-25T09:23:01.628 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-08-25T09:23:01.629 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-dev.ko 2023-08-25T09:23:01.629 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-mux.ko 2023-08-25T09:23:01.629 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-08-25T09:23:01.629 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-smbus.ko 2023-08-25T09:23:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/i2c/i2c-stub.ko 2023-08-25T09:23:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband 2023-08-25T09:23:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core 2023-08-25T09:23:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_cm.ko 2023-08-25T09:23:01.630 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_core.ko 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_umad.ko 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/iw_cm.ko 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_cm.ko 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4 2023-08-25T09:23:01.631 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-08-25T09:23:01.632 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp 2023-08-25T09:23:01.633 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/fm801-gp.ko 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/gameport.ko 2023-08-25T09:23:01.634 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/lightning.ko 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/gameport/ns558.ko 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/joydev.ko 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/keyboard/gpio_keys.ko 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/appletouch.ko 2023-08-25T09:23:01.635 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/bcm5974.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/cyapatp.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/elan_i2c.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/sermouse.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/synaptics_usb.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/mouse/vsxxxaa.ko 2023-08-25T09:23:01.636 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4 2023-08-25T09:23:01.637 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/rmi4/rmi_core.ko 2023-08-25T09:23:01.637 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio 2023-08-25T09:23:01.637 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/altera_ps2.ko 2023-08-25T09:23:01.637 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/arc_ps2.ko 2023-08-25T09:23:01.637 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-08-25T09:23:01.638 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/serio/serio_raw.ko 2023-08-25T09:23:01.638 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/input/sparse-keymap.ko 2023-08-25T09:23:01.638 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu 2023-08-25T09:23:01.638 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd 2023-08-25T09:23:01.639 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/iommu/amd/iommu_v2.ko 2023-08-25T09:23:01.639 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds 2023-08-25T09:23:01.639 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-blinkm.ko 2023-08-25T09:23:01.640 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lm3530.ko 2023-08-25T09:23:01.640 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-lp3944.ko 2023-08-25T09:23:01.640 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/leds-ss4200.ko 2023-08-25T09:23:01.640 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger 2023-08-25T09:23:01.641 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-08-25T09:23:01.641 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-08-25T09:23:01.641 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-08-25T09:23:01.641 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-08-25T09:23:01.642 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-08-25T09:23:01.642 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-08-25T09:23:01.642 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-08-25T09:23:01.643 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-08-25T09:23:01.643 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md 2023-08-25T09:23:01.643 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache 2023-08-25T09:23:01.643 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/bcache/bcache.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-bio-prison.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache-smq.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-cache.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-crypt.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-delay.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-flakey.ko 2023-08-25T09:23:01.644 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-log-userspace.ko 2023-08-25T09:23:01.645 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-multipath.ko 2023-08-25T09:23:01.645 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-queue-length.ko 2023-08-25T09:23:01.645 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-raid.ko 2023-08-25T09:23:01.645 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-round-robin.ko 2023-08-25T09:23:01.645 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-service-time.ko 2023-08-25T09:23:01.645 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-switch.ko 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-thin-pool.ko 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/dm-verity.ko 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/faulty.ko 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/linear.ko 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/multipath.ko 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data 2023-08-25T09:23:01.646 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid0.ko 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid1.ko 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid10.ko 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/md/raid456.ko 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion 2023-08-25T09:23:01.647 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptbase.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptctl.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptfc.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptlan.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptsas.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptscsih.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/message/fusion/mptspi.ko 2023-08-25T09:23:01.648 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_ich.ko 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/lpc_sch.ko 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/mfd-core.ko 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/sm501.ko 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/viperboard.ko 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/vx855.ko 2023-08-25T09:23:01.649 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/mfd/wl1273-core.ko 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds9802als.ko 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/apds990x.ko 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/bh1770glc.ko 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710 2023-08-25T09:23:01.650 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/cb710/cb710.ko 2023-08-25T09:23:01.651 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom 2023-08-25T09:23:01.651 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/at24.ko 2023-08-25T09:23:01.651 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom.ko 2023-08-25T09:23:01.651 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-08-25T09:23:01.651 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/eeprom/max6875.ko 2023-08-25T09:23:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/enclosure.ko 2023-08-25T09:23:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/hpilo.ko 2023-08-25T09:23:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29003.ko 2023-08-25T09:23:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/isl29020.ko 2023-08-25T09:23:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d 2023-08-25T09:23:01.652 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-me.ko 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei-txe.ko 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/mei/mei.ko 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru 2023-08-25T09:23:01.653 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-gru/gru.ko 2023-08-25T09:23:01.654 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp 2023-08-25T09:23:01.654 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xp.ko 2023-08-25T09:23:01.654 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpc.ko 2023-08-25T09:23:01.655 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-08-25T09:23:01.655 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_7xx1.ko 2023-08-25T09:23:01.655 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tifm_core.ko 2023-08-25T09:23:01.655 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/tsl2550.ko 2023-08-25T09:23:01.656 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_balloon.ko 2023-08-25T09:23:01.656 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci 2023-08-25T09:23:01.656 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-08-25T09:23:01.656 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net 2023-08-25T09:23:01.657 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding 2023-08-25T09:23:01.657 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/bonding/bonding.ko 2023-08-25T09:23:01.657 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa 2023-08-25T09:23:01.657 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53 2023-08-25T09:23:01.657 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/b53/b53_common.ko 2023-08-25T09:23:01.657 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/bcm-sf2.ko 2023-08-25T09:23:01.658 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6060.ko 2023-08-25T09:23:01.658 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx 2023-08-25T09:23:01.658 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-08-25T09:23:01.658 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/dummy.ko 2023-08-25T09:23:01.659 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/eql.ko 2023-08-25T09:23:01.659 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet 2023-08-25T09:23:01.659 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com 2023-08-25T09:23:01.659 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-08-25T09:23:01.659 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-08-25T09:23:01.659 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-08-25T09:23:01.660 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-08-25T09:23:01.660 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390 2023-08-25T09:23:01.660 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/8390.ko 2023-08-25T09:23:01.660 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-08-25T09:23:01.660 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-08-25T09:23:01.660 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/agere/et131x.ko 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-08-25T09:23:01.661 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx 2023-08-25T09:23:01.662 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-08-25T09:23:01.663 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-08-25T09:23:01.664 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet 2023-08-25T09:23:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-08-25T09:23:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-08-25T09:23:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade 2023-08-25T09:23:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna 2023-08-25T09:23:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-08-25T09:23:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-08-25T09:23:01.666 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip 2023-08-25T09:23:01.667 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-08-25T09:23:01.668 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-08-25T09:23:01.668 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-08-25T09:23:01.668 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-08-25T09:23:01.668 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-08-25T09:23:01.668 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-08-25T09:23:01.668 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink 2023-08-25T09:23:01.669 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-08-25T09:23:01.669 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-08-25T09:23:01.669 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/dnet.ko 2023-08-25T09:23:01.669 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex 2023-08-25T09:23:01.669 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet 2023-08-25T09:23:01.669 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-08-25T09:23:01.670 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ethoc.ko 2023-08-25T09:23:01.670 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/fealnx.ko 2023-08-25T09:23:01.670 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel 2023-08-25T09:23:01.670 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e100.ko 2023-08-25T09:23:01.670 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000 2023-08-25T09:23:01.670 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf 2023-08-25T09:23:01.671 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-08-25T09:23:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf 2023-08-25T09:23:01.673 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-08-25T09:23:01.673 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/jme.ko 2023-08-25T09:23:01.673 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell 2023-08-25T09:23:01.674 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-08-25T09:23:01.674 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/skge.ko 2023-08-25T09:23:01.674 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-08-25T09:23:01.674 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox 2023-08-25T09:23:01.674 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4 2023-08-25T09:23:01.674 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom 2023-08-25T09:23:01.675 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-08-25T09:23:01.676 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-08-25T09:23:01.677 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-08-25T09:23:01.678 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-08-25T09:23:01.678 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-08-25T09:23:01.678 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc 2023-08-25T09:23:01.678 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-08-25T09:23:01.679 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek 2023-08-25T09:23:01.679 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-08-25T09:23:01.679 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-08-25T09:23:01.680 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/atp.ko 2023-08-25T09:23:01.680 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-08-25T09:23:01.680 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker 2023-08-25T09:23:01.680 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-08-25T09:23:01.681 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc 2023-08-25T09:23:01.681 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-08-25T09:23:01.681 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan 2023-08-25T09:23:01.681 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-08-25T09:23:01.682 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis 2023-08-25T09:23:01.682 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis190.ko 2023-08-25T09:23:01.682 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sis/sis900.ko 2023-08-25T09:23:01.682 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc 2023-08-25T09:23:01.683 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-08-25T09:23:01.683 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-08-25T09:23:01.683 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-08-25T09:23:01.684 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-08-25T09:23:01.684 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro 2023-08-25T09:23:01.684 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac 2023-08-25T09:23:01.684 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-08-25T09:23:01.685 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-08-25T09:23:01.685 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun 2023-08-25T09:23:01.685 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/cassini.ko 2023-08-25T09:23:01.685 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/niu.ko 2023-08-25T09:23:01.685 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sungem.ko 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/ti/tlan.ko 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via 2023-08-25T09:23:01.686 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/geneve.ko 2023-08-25T09:23:01.687 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ieee802154/fakelb.ko 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ifb.ko 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ipvlan/ipvlan.ko 2023-08-25T09:23:01.688 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvlan.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/macvtap.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mdio/mdio-regmap.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/mii.ko 2023-08-25T09:23:01.689 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/net_failover.ko 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/netconsole.ko 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/nlmon.ko 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs-lynx.ko 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy 2023-08-25T09:23:01.690 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/amd.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm7xxx.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/bcm87xx.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/broadcom.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/cicada.ko 2023-08-25T09:23:01.691 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/davicom.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/dp83640.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/et1011c.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/icplus.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/lxt.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/marvell.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/micrel.ko 2023-08-25T09:23:01.692 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/national.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/phylink.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/qsemi.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/realtek.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/smsc.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/ste10Xp.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/phy/vitesse.ko 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp 2023-08-25T09:23:01.693 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/bsd_comp.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_async.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_deflate.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_generic.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_mppe.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/ppp_synctty.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppoe.ko 2023-08-25T09:23:01.694 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pppox.ko 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/ppp/pptp.ko 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slhc.ko 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/slip/slip.ko 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/sungem_phy.ko 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tap.ko 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team 2023-08-25T09:23:01.695 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_activebackup.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_broadcast.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_random.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/tun.ko 2023-08-25T09:23:01.696 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/veth.ko 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/virtio_net.ko 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/vxlan/vxlan.ko 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netback/xen-netback.ko 2023-08-25T09:23:01.697 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/net/xen-netfront.ko 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ntb/ntb.ko 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme-core.ko 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/nvme/host/nvme.ko 2023-08-25T09:23:01.698 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport.ko 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_cs.ko 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_pc.ko 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/parport/parport_serial.ko 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-08-25T09:23:01.699 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie 2023-08-25T09:23:01.700 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/pcie/aer_inject.ko 2023-08-25T09:23:01.700 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pci/xen-pcifront.ko 2023-08-25T09:23:01.700 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia 2023-08-25T09:23:01.700 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/i82092.ko 2023-08-25T09:23:01.700 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/pd6729.ko 2023-08-25T09:23:01.700 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pcmcia/yenta_socket.ko 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acer-wmi.ko 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/acerhdf.ko 2023-08-25T09:23:01.701 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/apple-gmux.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-laptop.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/asus-wmi.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/classmate-laptop.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-08-25T09:23:01.702 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/hdaps.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/intel_ips.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/msi-wmi.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/mxm-wmi.ko 2023-08-25T09:23:01.703 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-laptop.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/samsung-q10.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/topstar-laptop.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/toshiba_haps.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi-bmof.ko 2023-08-25T09:23:01.704 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/platform/x86/wmi.ko 2023-08-25T09:23:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap 2023-08-25T09:23:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_common.ko 2023-08-25T09:23:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/powercap/intel_rapl_msr.ko 2023-08-25T09:23:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps 2023-08-25T09:23:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients 2023-08-25T09:23:01.705 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-gpio.ko 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps-ldisc.ko 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/clients/pps_parport.ko 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/pps/pps_core.ko 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp.ko 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ptp/ptp_kvm.ko 2023-08-25T09:23:01.706 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq32k.ko 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-bq4802.ko 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1286.ko 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1307.ko 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1374.ko 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1511.ko 2023-08-25T09:23:01.707 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1553.ko 2023-08-25T09:23:01.708 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1672.ko 2023-08-25T09:23:01.708 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds1742.ko 2023-08-25T09:23:01.708 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds2404.ko 2023-08-25T09:23:01.708 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-ds3232.ko 2023-08-25T09:23:01.708 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-em3027.ko 2023-08-25T09:23:01.708 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-fm3130.ko 2023-08-25T09:23:01.709 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl12022.ko 2023-08-25T09:23:01.709 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-isl1208.ko 2023-08-25T09:23:01.709 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m41t80.ko 2023-08-25T09:23:01.709 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t35.ko 2023-08-25T09:23:01.709 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-m48t59.ko 2023-08-25T09:23:01.709 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-max6900.ko 2023-08-25T09:23:01.710 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-msm6242.ko 2023-08-25T09:23:01.710 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf2127.ko 2023-08-25T09:23:01.710 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf85063.ko 2023-08-25T09:23:01.710 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8523.ko 2023-08-25T09:23:01.710 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8563.ko 2023-08-25T09:23:01.711 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-pcf8583.ko 2023-08-25T09:23:01.711 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rp5c01.ko 2023-08-25T09:23:01.711 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rs5c372.ko 2023-08-25T09:23:01.711 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-08-25T09:23:01.711 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8025.ko 2023-08-25T09:23:01.711 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-rx8581.ko 2023-08-25T09:23:01.712 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-08-25T09:23:01.712 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/rtc/rtc-x1205.ko 2023-08-25T09:23:01.712 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi 2023-08-25T09:23:01.712 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-9xxx.ko 2023-08-25T09:23:01.713 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-sas.ko 2023-08-25T09:23:01.713 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/3w-xxxx.ko 2023-08-25T09:23:01.713 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/BusLogic.ko 2023-08-25T09:23:01.714 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/a100u2w.ko 2023-08-25T09:23:01.714 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid 2023-08-25T09:23:01.714 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aacraid/aacraid.ko 2023-08-25T09:23:01.714 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/advansys.ko 2023-08-25T09:23:01.715 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx 2023-08-25T09:23:01.715 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-08-25T09:23:01.715 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-08-25T09:23:01.715 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/am53c974.ko 2023-08-25T09:23:01.716 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr 2023-08-25T09:23:01.716 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-08-25T09:23:01.716 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/atp870u.ko 2023-08-25T09:23:01.716 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi 2023-08-25T09:23:01.717 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-08-25T09:23:01.717 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa 2023-08-25T09:23:01.717 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bfa/bfa.ko 2023-08-25T09:23:01.717 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc 2023-08-25T09:23:01.718 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-08-25T09:23:01.718 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i 2023-08-25T09:23:01.718 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-08-25T09:23:01.718 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ch.ko 2023-08-25T09:23:01.718 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/csiostor/csiostor.ko 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-08-25T09:23:01.719 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dc395x.ko 2023-08-25T09:23:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler 2023-08-25T09:23:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-08-25T09:23:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-08-25T09:23:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-08-25T09:23:01.720 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-08-25T09:23:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/dmx3191d.ko 2023-08-25T09:23:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r 2023-08-25T09:23:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esas2r/esas2r.ko 2023-08-25T09:23:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/esp_scsi.ko 2023-08-25T09:23:01.721 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe 2023-08-25T09:23:01.722 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/fcoe.ko 2023-08-25T09:23:01.722 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-08-25T09:23:01.722 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic 2023-08-25T09:23:01.722 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/fnic/fnic.ko 2023-08-25T09:23:01.723 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hpsa.ko 2023-08-25T09:23:01.723 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hptiop.ko 2023-08-25T09:23:01.723 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/hv_storvsc.ko 2023-08-25T09:23:01.723 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/initio.ko 2023-08-25T09:23:01.724 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ipr.ko 2023-08-25T09:23:01.724 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ips.ko 2023-08-25T09:23:01.724 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci 2023-08-25T09:23:01.725 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/isci/isci.ko 2023-08-25T09:23:01.725 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-08-25T09:23:01.725 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/iscsi_tcp.ko 2023-08-25T09:23:01.725 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc 2023-08-25T09:23:01.726 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libfc/libfc.ko 2023-08-25T09:23:01.726 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi.ko 2023-08-25T09:23:01.726 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libiscsi_tcp.ko 2023-08-25T09:23:01.726 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas 2023-08-25T09:23:01.727 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/libsas/libsas.ko 2023-08-25T09:23:01.727 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc 2023-08-25T09:23:01.727 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/lpfc/lpfc.ko 2023-08-25T09:23:01.727 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid 2023-08-25T09:23:01.728 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid.ko 2023-08-25T09:23:01.728 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-08-25T09:23:01.728 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-08-25T09:23:01.729 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-08-25T09:23:01.729 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas 2023-08-25T09:23:01.729 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-08-25T09:23:01.729 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas 2023-08-25T09:23:01.729 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvsas/mvsas.ko 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/mvumi.ko 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/pmcraid.ko 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla1280.ko 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx 2023-08-25T09:23:01.730 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/raid_class.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_debug.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_fc.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-08-25T09:23:01.731 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_sas.ko 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_spi.ko 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/scsi_transport_srp.ko 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/ses.ko 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/st.ko 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/stex.ko 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2 2023-08-25T09:23:01.732 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/virtio_scsi.ko 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/vmw_pvscsi.ko 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/scsi/wd719x.ko 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ssb/ssb.ko 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target 2023-08-25T09:23:01.733 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/loopback/tcm_loop.ko 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_file.ko 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_iblock.ko 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_mod.ko 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_pscsi.ko 2023-08-25T09:23:01.734 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/target_core_user.ko 2023-08-25T09:23:01.735 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc 2023-08-25T09:23:01.735 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-08-25T09:23:01.735 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal 2023-08-25T09:23:01.736 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel 2023-08-25T09:23:01.736 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal 2023-08-25T09:23:01.736 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-08-25T09:23:01.736 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-08-25T09:23:01.737 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-08-25T09:23:01.737 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-08-25T09:23:01.737 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-08-25T09:23:01.737 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-08-25T09:23:01.738 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-08-25T09:23:01.738 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-08-25T09:23:01.738 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-08-25T09:23:01.738 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-08-25T09:23:01.739 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-08-25T09:23:01.739 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-08-25T09:23:01.739 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-08-25T09:23:01.740 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-08-25T09:23:01.740 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-08-25T09:23:01.740 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty 2023-08-25T09:23:01.740 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_gsm.ko 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/n_hdlc.ko 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/nozomi.ko 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250 2023-08-25T09:23:01.741 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/arc_uart.ko 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/serial/jsm/jsm.ko 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/tty/synclink_gt.ko 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core 2023-08-25T09:23:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/core/ufshcd-core.ko 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio.ko 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_aec.ko 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_cif.ko 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_pci_generic.ko 2023-08-25T09:23:01.743 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/uio/uio_sercos3.ko 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-acm.ko 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/cdc-wdm.ko 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usblp.ko 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/class/usbtmc.ko 2023-08-25T09:23:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/host/sl811-hcd.ko 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/mdc800.ko 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/image/microtek.ko 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/adutux.ko 2023-08-25T09:23:01.745 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/appledisplay.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi26.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/emi62.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ezusb.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/idmouse.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/iowarrior.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/isight_firmware.ko 2023-08-25T09:23:01.746 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/ldusb.ko 2023-08-25T09:23:01.747 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/legousbtower.ko 2023-08-25T09:23:01.747 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga 2023-08-25T09:23:01.747 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-08-25T09:23:01.747 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/trancevibrator.ko 2023-08-25T09:23:01.747 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usb3503.ko 2023-08-25T09:23:01.747 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usblcd.ko 2023-08-25T09:23:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/usbsevseg.ko 2023-08-25T09:23:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/uss720.ko 2023-08-25T09:23:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/misc/yurex.ko 2023-08-25T09:23:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy 2023-08-25T09:23:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/phy/phy-generic.ko 2023-08-25T09:23:01.748 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial 2023-08-25T09:23:01.749 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/aircable.ko 2023-08-25T09:23:01.749 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ark3116.ko 2023-08-25T09:23:01.749 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/belkin_sa.ko 2023-08-25T09:23:01.749 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ch341.ko 2023-08-25T09:23:01.749 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cp210x.ko 2023-08-25T09:23:01.749 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cyberjack.ko 2023-08-25T09:23:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/cypress_m8.ko 2023-08-25T09:23:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/digi_acceleport.ko 2023-08-25T09:23:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/empeg.ko 2023-08-25T09:23:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ftdi_sio.ko 2023-08-25T09:23:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/garmin_gps.ko 2023-08-25T09:23:01.750 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_edgeport.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/io_ti.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipaq.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ipw.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ir-usb.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan.ko 2023-08-25T09:23:01.751 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/keyspan_pda.ko 2023-08-25T09:23:01.752 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kl5kusb105.ko 2023-08-25T09:23:01.752 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/kobil_sct.ko 2023-08-25T09:23:01.752 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mct_u232.ko 2023-08-25T09:23:01.752 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7720.ko 2023-08-25T09:23:01.752 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/mos7840.ko 2023-08-25T09:23:01.752 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/navman.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/omninet.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/opticon.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/option.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/oti6858.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/pl2303.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcaux.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/qcserial.ko 2023-08-25T09:23:01.753 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/quatech2.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/safe_serial.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/sierra.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/spcp8x5.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ssu100.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/symbolserial.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-08-25T09:23:01.754 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_debug.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/usb_wwan.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/visor.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/whiteheat.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/serial/xsens_mt.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/uas.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-alauda.ko 2023-08-25T09:23:01.755 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-cypress.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-datafab.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-freecom.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-isd200.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-karma.ko 2023-08-25T09:23:01.756 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-onetouch.ko 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-realtek.ko 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr09.ko 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-sddr55.ko 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/ums-usbat.ko 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/storage/usb-storage.ko 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip 2023-08-25T09:23:01.757 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-core.ko 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/usbip-host.ko 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/pci/vfio-pci.ko 2023-08-25T09:23:01.758 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio.ko 2023-08-25T09:23:01.759 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-08-25T09:23:01.759 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost 2023-08-25T09:23:01.759 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost.ko 2023-08-25T09:23:01.759 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_iotlb.ko 2023-08-25T09:23:01.759 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_net.ko 2023-08-25T09:23:01.759 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/vhost/vhost_scsi.ko 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/apple_bl.ko 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/lcd.ko 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/video/backlight/platform_lcd.ko 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio.ko 2023-08-25T09:23:01.760 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_balloon.ko 2023-08-25T09:23:01.761 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_mmio.ko 2023-08-25T09:23:01.761 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci.ko 2023-08-25T09:23:01.761 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-08-25T09:23:01.761 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-08-25T09:23:01.761 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/virtio/virtio_ring.ko 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2482.ko 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/masters/ds2490.ko 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-08-25T09:23:01.762 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-08-25T09:23:01.763 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_smem.ko 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/slaves/w1_therm.ko 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/w1/wire.ko 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim1535_wdt.ko 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/alim7101_wdt.ko 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/f71808e_wdt.ko 2023-08-25T09:23:01.764 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/hpwdt.ko 2023-08-25T09:23:01.765 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/i6300esb.ko 2023-08-25T09:23:01.765 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-08-25T09:23:01.765 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/iTCO_wdt.ko 2023-08-25T09:23:01.765 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ib700wdt.ko 2023-08-25T09:23:01.765 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ibmasr.ko 2023-08-25T09:23:01.766 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-08-25T09:23:01.766 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it8712f_wdt.ko 2023-08-25T09:23:01.766 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/it87_wdt.ko 2023-08-25T09:23:01.766 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/machzwd.ko 2023-08-25T09:23:01.767 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/nv_tco.ko 2023-08-25T09:23:01.767 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_pci.ko 2023-08-25T09:23:01.767 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/pcwd_usb.ko 2023-08-25T09:23:01.767 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-08-25T09:23:01.768 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sch311x_wdt.ko 2023-08-25T09:23:01.768 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/softdog.ko 2023-08-25T09:23:01.768 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/sp5100_tco.ko 2023-08-25T09:23:01.769 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/via_wdt.ko 2023-08-25T09:23:01.769 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-08-25T09:23:01.769 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83877f_wdt.ko 2023-08-25T09:23:01.770 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/w83977f_wdt.ko 2023-08-25T09:23:01.770 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/wdt_pci.ko 2023-08-25T09:23:01.770 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/watchdog/xen_wdt.ko 2023-08-25T09:23:01.770 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen 2023-08-25T09:23:01.771 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-acpi-processor.ko 2023-08-25T09:23:01.771 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-evtchn.ko 2023-08-25T09:23:01.771 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntalloc.ko 2023-08-25T09:23:01.772 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-gntdev.ko 2023-08-25T09:23:01.772 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback 2023-08-25T09:23:01.772 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-08-25T09:23:01.772 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xen-privcmd.ko 2023-08-25T09:23:01.773 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs 2023-08-25T09:23:01.773 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/drivers/xen/xenfs/xenfs.ko 2023-08-25T09:23:01.773 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs 2023-08-25T09:23:01.773 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p 2023-08-25T09:23:01.774 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/9p/9p.ko 2023-08-25T09:23:01.774 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs 2023-08-25T09:23:01.774 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/affs/affs.ko 2023-08-25T09:23:01.774 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs 2023-08-25T09:23:01.774 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/befs/befs.ko 2023-08-25T09:23:01.775 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/binfmt_misc.ko 2023-08-25T09:23:01.775 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs 2023-08-25T09:23:01.775 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/btrfs/btrfs.ko 2023-08-25T09:23:01.776 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles 2023-08-25T09:23:01.776 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cachefiles/cachefiles.ko 2023-08-25T09:23:01.776 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph 2023-08-25T09:23:01.776 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ceph/ceph.ko 2023-08-25T09:23:01.776 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/coda/coda.ko 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/cramfs/cramfs.ko 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/dlm/dlm.ko 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs 2023-08-25T09:23:01.777 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ecryptfs/ecryptfs.ko 2023-08-25T09:23:01.778 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs 2023-08-25T09:23:01.778 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/f2fs/f2fs.ko 2023-08-25T09:23:01.778 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat 2023-08-25T09:23:01.778 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/fat.ko 2023-08-25T09:23:01.778 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/msdos.ko 2023-08-25T09:23:01.778 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fat/vfat.ko 2023-08-25T09:23:01.779 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache 2023-08-25T09:23:01.779 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fscache/fscache.ko 2023-08-25T09:23:01.779 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse 2023-08-25T09:23:01.780 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/cuse.ko 2023-08-25T09:23:01.780 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/fuse/fuse.ko 2023-08-25T09:23:01.780 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2 2023-08-25T09:23:01.780 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/gfs2/gfs2.ko 2023-08-25T09:23:01.781 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs 2023-08-25T09:23:01.781 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfs/hfs.ko 2023-08-25T09:23:01.781 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus 2023-08-25T09:23:01.782 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/hfsplus/hfsplus.ko 2023-08-25T09:23:01.782 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs 2023-08-25T09:23:01.782 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/isofs/isofs.ko 2023-08-25T09:23:01.782 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs 2023-08-25T09:23:01.783 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/jfs/jfs.ko 2023-08-25T09:23:01.783 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd 2023-08-25T09:23:01.783 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/lockd/lockd.ko 2023-08-25T09:23:01.784 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix 2023-08-25T09:23:01.784 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/minix/minix.ko 2023-08-25T09:23:01.784 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs 2023-08-25T09:23:01.785 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/netfs/netfs.ko 2023-08-25T09:23:01.785 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs 2023-08-25T09:23:01.785 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout 2023-08-25T09:23:01.785 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-08-25T09:23:01.786 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout 2023-08-25T09:23:01.786 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-08-25T09:23:01.786 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout 2023-08-25T09:23:01.786 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-08-25T09:23:01.787 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfs.ko 2023-08-25T09:23:01.787 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv3.ko 2023-08-25T09:23:01.787 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs/nfsv4.ko 2023-08-25T09:23:01.787 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common 2023-08-25T09:23:01.787 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/grace.ko 2023-08-25T09:23:01.787 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfs_common/nfs_acl.ko 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nfsd/nfsd.ko 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nilfs2/nilfs2.ko 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-celtic.ko 2023-08-25T09:23:01.788 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-centeuro.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-croatian.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-cyrillic.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-gaelic.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-greek.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-iceland.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-inuit.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-roman.ko 2023-08-25T09:23:01.789 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-romanian.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/mac-turkish.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1250.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1251.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp1255.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp737.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp775.ko 2023-08-25T09:23:01.790 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp850.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp852.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp855.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp857.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp860.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp861.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp862.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp863.ko 2023-08-25T09:23:01.791 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp864.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp865.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp866.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp869.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp874.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp932.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp936.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp949.ko 2023-08-25T09:23:01.792 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_cp950.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_euc-jp.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-1.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-13.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-14.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-15.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-2.ko 2023-08-25T09:23:01.793 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-3.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-4.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-5.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-6.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-7.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_iso8859-9.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-r.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-ru.ko 2023-08-25T09:23:01.794 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_koi8-u.ko 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/nls/nls_utf8.ko 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs 2023-08-25T09:23:01.795 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2.ko 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/overlayfs/overlay.ko 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore 2023-08-25T09:23:01.796 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/pstore/ramoops.ko 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/reiserfs/reiserfs.ko 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/romfs/romfs.ko 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client 2023-08-25T09:23:01.797 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/client/cifs.ko 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_arc4.ko 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/smb/common/cifs_md4.ko 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/squashfs/squashfs.ko 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/sysv/sysv.ko 2023-08-25T09:23:01.798 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/udf/udf.ko 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/ufs/ufs.ko 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/fs/xfs/xfs.ko 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu 2023-08-25T09:23:01.799 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/rcu/rcutorture.ko 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/torture.ko 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/asn1_encoder.ko 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc-itu-t.ko 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crc8.ko 2023-08-25T09:23:01.800 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libarc4.ko 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/crypto/libdes.ko 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/libcrc32c.ko 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lru_cache.ko 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4_compress.ko 2023-08-25T09:23:01.801 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/lz4/lz4hc_compress.ko 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/math/cordic.ko 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/raid6/raid6_pq.ko 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/reed_solomon/reed_solomon.ko 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_bm.ko 2023-08-25T09:23:01.802 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_fsm.ko 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/ts_kmp.ko 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/lib/zstd/zstd_compress.ko 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/mm/hwpoison-inject.ko 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan 2023-08-25T09:23:01.803 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/6lowpan.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_dest.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_fragment.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_hop.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_ipv6.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_mobility.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_routing.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/6lowpan/nhc_udp.ko 2023-08-25T09:23:01.804 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/garp.ko 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/mrp.ko 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/p8022.ko 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/psnap.ko 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/802/stp.ko 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q 2023-08-25T09:23:01.805 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/8021q/8021q.ko 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet.ko 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_fd.ko 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_rdma.ko 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/9p/9pnet_virtio.ko 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/batman-adv/batman-adv.ko 2023-08-25T09:23:01.806 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/br_netfilter.ko 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/bridge.ko 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_among.ko 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arp.ko 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-08-25T09:23:01.807 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_limit.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_log.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-08-25T09:23:01.808 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_snat.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_stp.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-08-25T09:23:01.809 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/ebtables.ko 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ceph/libceph.ko 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core 2023-08-25T09:23:01.810 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/failover.ko 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/core/pktgen.ko 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp.ko 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_diag.ko 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv4.ko 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dccp/dccp_ipv6.ko 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver 2023-08-25T09:23:01.811 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dns_resolver/dns_resolver.ko 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/dsa_core.ko 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_brcm.ko 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_dsa.ko 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_none.ko 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/dsa/tag_trailer.ko 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154 2023-08-25T09:23:01.812 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154.ko 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ieee802154/ieee802154_socket.ko 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ah4.ko 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/esp4.ko 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/fou.ko 2023-08-25T09:23:01.813 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/gre.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/inet_diag.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_gre.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_tunnel.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ip_vti.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipcomp.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/ipip.ko 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter 2023-08-25T09:23:01.814 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arp_tables.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-08-25T09:23:01.815 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/iptable_security.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-08-25T09:23:01.816 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_bic.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_dctcp.ko 2023-08-25T09:23:01.817 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_diag.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_highspeed.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_htcp.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_hybla.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_illinois.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_lp.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_scalable.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_vegas.ko 2023-08-25T09:23:01.818 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_veno.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_westwood.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tcp_yeah.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/tunnel4.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_diag.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/udp_tunnel.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv4/xfrm4_tunnel.ko 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6 2023-08-25T09:23:01.819 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ah6.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/esp6.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/fou6.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_tunnel.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ip6_vti.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/ipcomp6.ko 2023-08-25T09:23:01.820 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-08-25T09:23:01.821 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-08-25T09:23:01.822 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-08-25T09:23:01.823 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/sit.ko 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/tunnel6.ko 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/ipv6/xfrm6_tunnel.ko 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/key/af_key.ko 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp 2023-08-25T09:23:01.824 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_core.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_debugfs.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_eth.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ip6.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_netlink.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/l2tp/l2tp_ppp.ko 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc 2023-08-25T09:23:01.825 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/llc/llc.ko 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mac802154/mac802154.ko 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/mpls/mpls_gso.ko 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set.ko 2023-08-25T09:23:01.826 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-08-25T09:23:01.827 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-08-25T09:23:01.828 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-08-25T09:23:01.829 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conncount.ko 2023-08-25T09:23:01.830 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_h323.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_irc.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-08-25T09:23:01.831 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sane.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_sip.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_dup_netdev.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table.ko 2023-08-25T09:23:01.832 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_flow_table_inet.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_log_syslog.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_amanda.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_ftp.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_irc.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_sip.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_nat_tftp.ko 2023-08-25T09:23:01.833 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_synproxy_core.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nf_tables.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_acct.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_log.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_osf.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nfnetlink_queue.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_chain_nat.ko 2023-08-25T09:23:01.834 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_compat.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_ct.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_dup_netdev.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fib.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_fwd_netdev.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_hash.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_limit.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_log.ko 2023-08-25T09:23:01.835 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_masq.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_nat.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_numgen.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_queue.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_quota.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_redir.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject.ko 2023-08-25T09:23:01.836 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_reject_inet.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_socket.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/nft_tunnel.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_AUDIT.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CHECKSUM.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CLASSIFY.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_CT.ko 2023-08-25T09:23:01.837 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_DSCP.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HL.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_HMARK.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_IDLETIMER.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LED.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_LOG.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_MASQUERADE.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NETMAP.ko 2023-08-25T09:23:01.838 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFLOG.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_NFQUEUE.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_RATEEST.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_REDIRECT.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_SECMARK.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPMSS.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TEE.ko 2023-08-25T09:23:01.839 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TPROXY.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_TRACE.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_addrtype.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_bpf.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cgroup.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cluster.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_comment.ko 2023-08-25T09:23:01.840 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connbytes.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlabel.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connlimit.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_connmark.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_conntrack.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_cpu.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dccp.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_devgroup.ko 2023-08-25T09:23:01.841 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_dscp.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ecn.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_esp.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hashlimit.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_helper.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_hl.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipcomp.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_iprange.ko 2023-08-25T09:23:01.842 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_ipvs.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_l2tp.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_length.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_limit.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mac.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_mark.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_multiport.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nat.ko 2023-08-25T09:23:01.843 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_nfacct.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_osf.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_owner.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_physdev.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_pkttype.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_policy.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_quota.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_rateest.ko 2023-08-25T09:23:01.844 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_realm.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_recent.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_sctp.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_set.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_socket.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_state.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_statistic.ko 2023-08-25T09:23:01.845 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_string.ko 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_tcpmss.ko 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_time.ko 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netfilter/xt_u32.ko 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/netlink/netlink_diag.ko 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/nsh/nsh.ko 2023-08-25T09:23:01.846 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/openvswitch.ko 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-geneve.ko 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-gre.ko 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/openvswitch/vport-vxlan.ko 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/packet/af_packet_diag.ko 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds 2023-08-25T09:23:01.847 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_rdma.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/rds/rds_tcp.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_csum.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_gact.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_ipt.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_mirred.ko 2023-08-25T09:23:01.848 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_nat.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_pedit.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_police.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_simple.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_skbedit.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/act_vlan.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_basic.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_bpf.ko 2023-08-25T09:23:01.849 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_flow.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_fw.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_route.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/cls_u32.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_cmp.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_ipset.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_meta.ko 2023-08-25T09:23:01.850 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_nbyte.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_text.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/em_u32.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_choke.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_codel.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_drr.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_fq.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_gred.ko 2023-08-25T09:23:01.851 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hfsc.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_hhf.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_htb.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_ingress.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_mqprio_lib.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_multiq.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_netem.ko 2023-08-25T09:23:01.852 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_pie.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_plug.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_prio.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_qfq.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_red.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfb.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_sfq.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_tbf.ko 2023-08-25T09:23:01.853 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sched/sch_teql.ko 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp.ko 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sctp/sctp_diag.ko 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-08-25T09:23:01.854 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/sunrpc.ko 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/diag.ko 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/tipc/tipc.ko 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/unix/unix_diag.ko 2023-08-25T09:23:01.855 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock.ko 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_diag.ko 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/vmw_vsock/vsock_loopback.ko 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm 2023-08-25T09:23:01.856 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/net/xfrm/xfrm_ipcomp.ko 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/security/keys/trusted-keys/trusted.ko 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt 2023-08-25T09:23:01.857 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/kernel/virt/lib/irqbypass.ko 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.alias.bin 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.alias.bin 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.bin 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.builtin.modinfo 2023-08-25T09:23:01.858 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep 2023-08-25T09:23:01.859 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.dep.bin 2023-08-25T09:23:01.859 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.devname 2023-08-25T09:23:01.859 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.order 2023-08-25T09:23:01.859 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.softdep 2023-08-25T09:23:01.859 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols 2023-08-25T09:23:01.859 INFO:teuthology.orchestra.run.smithi155.stdout:/lib/modules/6.5.0-rc7-ga6a696707610/modules.symbols.bin 2023-08-25T09:23:01.860 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.860 DEBUG:teuthology.orchestra.run.smithi155:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-08-25T09:23:01.885 INFO:teuthology.orchestra.run.smithi155.stdout:postinstall scriptlet (using /bin/sh): 2023-08-25T09:23:01.885 INFO:teuthology.orchestra.run.smithi155.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-ga6a696707610 -a -r /boot/System.map-6.5.0-rc7-ga6a696707610 ]; then 2023-08-25T09:23:01.885 INFO:teuthology.orchestra.run.smithi155.stdout:cp /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:23:01.885 INFO:teuthology.orchestra.run.smithi155.stdout:cp /boot/System.map-6.5.0-rc7-ga6a696707610 /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:23:01.885 INFO:teuthology.orchestra.run.smithi155.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-ga6a696707610 /boot/System.map-6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:/sbin/installkernel 6.5.0-rc7-ga6a696707610 /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-ga6a696707610-rpm /boot/.System.map-6.5.0-rc7-ga6a696707610-rpm 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:preuninstall scriptlet (using /bin/sh): 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:new-kernel-pkg --remove 6.5.0-rc7-ga6a696707610 --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-ga6a696707610.img 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-08-25T09:23:01.886 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-install remove 6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.887 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2023-08-25T09:23:01.887 INFO:teuthology.orchestra.run.smithi155.stdout:postuninstall scriptlet (using /bin/sh): 2023-08-25T09:23:01.887 INFO:teuthology.orchestra.run.smithi155.stdout:if [ -x /sbin/update-bootloader ]; then 2023-08-25T09:23:01.887 INFO:teuthology.orchestra.run.smithi155.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.887 INFO:teuthology.orchestra.run.smithi155.stdout:fi 2023-08-25T09:23:01.888 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -qi grub2-tools 2023-08-25T09:23:01.967 INFO:teuthology.orchestra.run.smithi155.stdout:Name : grub2-tools 2023-08-25T09:23:01.967 INFO:teuthology.orchestra.run.smithi155.stdout:Epoch : 1 2023-08-25T09:23:01.967 INFO:teuthology.orchestra.run.smithi155.stdout:Version : 2.06 2023-08-25T09:23:01.967 INFO:teuthology.orchestra.run.smithi155.stdout:Release : 53.el9 2023-08-25T09:23:01.967 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: x86_64 2023-08-25T09:23:01.967 INFO:teuthology.orchestra.run.smithi155.stdout:Install Date: Mon 30 Jan 2023 10:38:40 AM UTC 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:Group : Unspecified 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:Size : 8551923 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:License : GPLv3+ 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:Signature : RSA/SHA256, Wed 18 Jan 2023 04:40:07 PM UTC, Key ID 05b555b38483c65d 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:Source RPM : grub2-2.06-53.el9.src.rpm 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:Build Date : Fri 13 Jan 2023 09:07:28 PM UTC 2023-08-25T09:23:01.968 INFO:teuthology.orchestra.run.smithi155.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:Packager : builder@centos.org 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:Vendor : CentOS 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:URL : http://www.gnu.org/software/grub/ 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:Summary : Support tools for GRUB. 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:Description : 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-08-25T09:23:01.969 INFO:teuthology.orchestra.run.smithi155.stdout:customizable bootloader with modular architecture. It supports a rich 2023-08-25T09:23:01.970 INFO:teuthology.orchestra.run.smithi155.stdout:variety of kernel formats, file systems, computer architectures and 2023-08-25T09:23:01.970 INFO:teuthology.orchestra.run.smithi155.stdout:hardware devices. 2023-08-25T09:23:01.970 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:23:01.970 INFO:teuthology.orchestra.run.smithi155.stdout:This subpackage provides tools for support of all platforms. 2023-08-25T09:23:01.971 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-08-25T09:23:01.971 INFO:teuthology.task.kernel:Updating grub on smithi155 to boot 6.5.0-rc7-ga6a696707610 2023-08-25T09:23:01.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-08-25T09:23:02.565 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2023-08-25T09:23:03.978 INFO:teuthology.orchestra.run.smithi155.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-08-25T09:23:04.023 INFO:teuthology.orchestra.run.smithi155.stderr:done 2023-08-25T09:23:04.034 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2023-08-25T09:23:04.048 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.4T5LllfCuM 2023-08-25T09:23:04.049 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4T5LllfCuM 2023-08-25T09:23:04.117 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.4T5LllfCuM 2023-08-25T09:23:04.270 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.4T5LllfCuM is 6KB 2023-08-25T09:23:04.319 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.4T5LllfCuM 2023-08-25T09:23:04.333 DEBUG:teuthology.orchestra.run.smithi155:> sudo /bin/ls /boot/loader/entries || true 2023-08-25T09:23:04.397 INFO:teuthology.orchestra.run.smithi155.stdout:e13cba1d25f14c51a1d04da8db823a4e-0-rescue.conf 2023-08-25T09:23:04.398 INFO:teuthology.orchestra.run.smithi155.stdout:e13cba1d25f14c51a1d04da8db823a4e-5.14.0-239.el9.x86_64.conf 2023-08-25T09:23:04.398 INFO:teuthology.orchestra.run.smithi155.stdout:e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc7-ga6a696707610.conf 2023-08-25T09:23:04.399 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-set-default e13cba1d25f14c51a1d04da8db823a4e-6.5.0-rc7-ga6a696707610 2023-08-25T09:23:04.604 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2023-08-25T09:23:26.476 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-08-25T09:23:26.477 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-08-25T09:23:26.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:23:34.613 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-08-25T09:23:34.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:23:34.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:23:44.884 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-08-25T09:23:53.072 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-08-25T09:23:53.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-08-25T09:23:53.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:23:56.944 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.70 2023-08-25T09:24:02.076 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:24:02.077 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:24:05.140 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-08-25T09:24:08.952 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-08-25T09:24:08.953 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:24:17.148 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:24:17.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:24:35.568 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2023-08-25T09:24:40.755 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-08-25T09:24:41.413 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-08-25T09:24:41.413 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc7-ga6a696707610"... 2023-08-25T09:24:41.414 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2023-08-25T09:24:41.457 INFO:teuthology.orchestra.run.smithi070.stdout:6.5.0-rc7-ga6a696707610 2023-08-25T09:24:41.458 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-ga6a696707610 vs 6.5.0-rc7-ga6a696707610 2023-08-25T09:24:41.458 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-25T09:24:41.458 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-25T09:24:42.459 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-08-25T09:24:42.460 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-25T09:24:42.726 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2023-08-25T09:24:42.769 DEBUG:teuthology.parallel:result is None 2023-08-25T09:24:50.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:24:50.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:24:50.961 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-08-25T09:24:51.519 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:24:51.519 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc7-ga6a696707610"... 2023-08-25T09:24:51.520 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2023-08-25T09:24:51.557 INFO:teuthology.orchestra.run.smithi155.stdout:6.5.0-rc7-ga6a696707610 2023-08-25T09:24:51.557 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-ga6a696707610 vs 6.5.0-rc7-ga6a696707610 2023-08-25T09:24:51.557 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-08-25T09:24:51.558 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-08-25T09:24:52.559 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-08-25T09:24:52.574 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-08-25T09:24:52.709 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2023-08-25T09:24:52.764 DEBUG:teuthology.parallel:result is None 2023-08-25T09:24:52.765 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-25T09:24:52.779 INFO:teuthology.task.internal:Creating test directory... 2023-08-25T09:24:52.780 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T09:24:52.792 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T09:24:52.879 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-25T09:24:52.888 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-25T09:24:52.914 INFO:teuthology.task.internal:Creating archive directory... 2023-08-25T09:24:52.914 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T09:24:52.921 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T09:24:53.029 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-25T09:24:53.038 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-25T09:24:53.039 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T09:24:53.046 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T09:24:53.159 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T09:24:53.197 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T09:24:53.228 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T09:24:53.303 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T09:24:53.309 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-25T09:24:53.324 INFO:teuthology.task.internal:Configuring sudo... 2023-08-25T09:24:53.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T09:24:53.331 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T09:24:53.467 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-25T09:24:53.480 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-25T09:24:53.481 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T09:24:53.486 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T09:24:53.530 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T09:24:53.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T09:24:53.744 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T09:24:53.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T09:24:53.903 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:24:53.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T09:24:53.984 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T09:24:54.067 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T09:24:54.152 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T09:24:54.224 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T09:24:54.302 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:24:54.302 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T09:24:54.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2023-08-25T09:24:54.389 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2023-08-25T09:24:54.508 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:24:54.535 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:24:54.800 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-25T09:24:54.816 INFO:teuthology.task.internal:Starting timer... 2023-08-25T09:24:54.816 INFO:teuthology.run_tasks:Running task pcp... 2023-08-25T09:24:54.848 INFO:teuthology.run_tasks:Running task selinux... 2023-08-25T09:24:54.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2023-08-25T09:24:55.317 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2023-08-25T09:24:55.321 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2023-08-25T09:24:55.429 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2023-08-25T09:24:55.432 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-25T09:24:55.432 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2023-08-25T09:24:55.509 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2023-08-25T09:24:55.510 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2023-08-25T09:24:55.553 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2023-08-25T09:24:55.553 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2023-08-25T09:24:55.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:24:55.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:24:55.640 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:24:55.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:24:55.728 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-25T09:24:55.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2023-08-25T09:24:55.844 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2023-08-25T09:24:55.929 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-25T09:24:55.941 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-08-25T09:24:55.942 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-25T09:24:55.971 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-25T09:24:55.972 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-25T09:36:01.175 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2023-08-25T09:36:01.204 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-08-25T09:36:01.205 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:36:01.346 DEBUG:teuthology.orchestra.run.smithi070:> true 2023-08-25T09:36:01.504 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2023-08-25T09:36:01.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:36:01.505 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T09:36:01.642 DEBUG:teuthology.orchestra.run.smithi155:> true 2023-08-25T09:36:01.799 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2023-08-25T09:36:01.800 INFO:teuthology.run_tasks:Running task clock... 2023-08-25T09:36:01.811 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-25T09:36:01.811 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T09:36:01.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:36:01.821 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T09:36:01.821 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:36:01.918 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T09:36:01.930 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T09:36:01.968 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T09:36:01.979 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T09:36:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2023-08-25T09:36:02.069 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2023-08-25T09:36:02.103 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2023-08-25T09:36:02.108 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2023-08-25T09:36:02.150 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T09:36:02.159 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T09:36:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T09:36:02.213 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T09:36:02.340 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2023-08-25T09:36:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2023-08-25T09:36:02.348 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:36:02.349 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-08-25T09:36:02.349 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.349 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.349 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.349 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.350 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:36:02.350 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-08-25T09:36:02.350 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.350 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.350 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.351 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:36:02.352 INFO:teuthology.run_tasks:Running task install... 2023-08-25T09:36:02.363 DEBUG:teuthology.task.install:project ceph 2023-08-25T09:36:02.364 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}} 2023-08-25T09:36:02.364 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'} 2023-08-25T09:36:02.364 INFO:teuthology.task.install:Using flavor: default 2023-08-25T09:36:02.374 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-25T09:36:02.375 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-25T09:36:02.375 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T09:36:02.376 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T09:36:02.379 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T09:36:02.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T09:36:02.518 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/ 2023-08-25T09:36:02.518 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T09:36:02.763 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/ 2023-08-25T09:36:02.763 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T09:36:02.799 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T09:36:02.800 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:36:02.800 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T09:36:02.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T09:36:02.869 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:36:02.869 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T09:36:02.905 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T09:36:02.906 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T09:36:02.970 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T09:36:02.971 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T09:36:03.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T09:36:03.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T09:36:03.184 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T09:36:03.240 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T09:36:03.281 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2023-08-25T09:36:03.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2023-08-25T09:36:03.333 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2023-08-25T09:36:03.337 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2023-08-25T09:36:03.655 INFO:teuthology.orchestra.run.smithi070.stdout:72 files removed 2023-08-25T09:36:03.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T09:36:03.708 INFO:teuthology.orchestra.run.smithi155.stdout:72 files removed 2023-08-25T09:36:03.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T09:36:04.580 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 172 kB/s | 78 kB 00:00 2023-08-25T09:36:04.759 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 137 kB/s | 78 kB 00:00 2023-08-25T09:36:05.155 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-08-25T09:36:05.318 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-08-25T09:36:05.668 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2023-08-25T09:36:05.878 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2023-08-25T09:36:06.265 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - AppStream 12 MB/s | 6.2 MB 00:00 2023-08-25T09:36:06.497 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 11 MB/s | 6.2 MB 00:00 2023-08-25T09:36:07.794 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2023-08-25T09:36:08.017 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2023-08-25T09:36:09.398 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 7.8 MB 00:01 2023-08-25T09:36:09.571 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 7.8 MB 00:01 2023-08-25T09:36:11.439 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 22 MB/s | 18 MB 00:00 2023-08-25T09:36:17.540 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 5.7 MB 00:02 2023-08-25T09:36:19.016 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 14 kB 00:00 2023-08-25T09:36:20.109 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 19 MB/s | 18 MB 00:00 2023-08-25T09:36:25.397 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 18 MB 00:14 2023-08-25T09:36:27.163 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-25T09:36:27.165 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-25T09:36:27.186 INFO:teuthology.orchestra.run.smithi155.stdout:Package NetworkManager-1:1.41.8-1.el9.x86_64 is already installed. 2023-08-25T09:36:27.187 INFO:teuthology.orchestra.run.smithi155.stdout:Package iproute-6.0.0-2.el9.x86_64 is already installed. 2023-08-25T09:36:27.188 INFO:teuthology.orchestra.run.smithi155.stdout:Package util-linux-2.37.4-9.el9.x86_64 is already installed. 2023-08-25T09:36:27.190 INFO:teuthology.orchestra.run.smithi155.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-08-25T09:36:27.192 INFO:teuthology.orchestra.run.smithi155.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-08-25T09:36:27.196 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsdump-3.1.10-1.el9.x86_64 is already installed. 2023-08-25T09:36:27.197 INFO:teuthology.orchestra.run.smithi155.stdout:Package xfsprogs-5.14.2-1.el9.x86_64 is already installed. 2023-08-25T09:36:27.279 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2023-08-25T09:36:27.305 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2023-08-25T09:36:27.305 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 6.5 k 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 5.2 M 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 850 k 2023-08-25T09:36:27.306 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 141 k 2023-08-25T09:36:27.307 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 1.5 M 2023-08-25T09:36:27.307 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 132 k 2023-08-25T09:36:27.307 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 1.7 M 2023-08-25T09:36:27.308 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 7.4 M 2023-08-25T09:36:27.308 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 47 k 2023-08-25T09:36:27.308 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 8.2 M 2023-08-25T09:36:27.308 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 40 M 2023-08-25T09:36:27.309 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 261 k 2023-08-25T09:36:27.309 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 225 k 2023-08-25T09:36:27.309 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 218 k 2023-08-25T09:36:27.310 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 18 k 2023-08-25T09:36:27.310 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 CentOS-AppStream 21 k 2023-08-25T09:36:27.310 INFO:teuthology.orchestra.run.smithi155.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-08-25T09:36:27.310 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-08-25T09:36:27.311 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 31 k 2023-08-25T09:36:27.311 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 667 k 2023-08-25T09:36:27.311 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 126 k 2023-08-25T09:36:27.311 INFO:teuthology.orchestra.run.smithi155.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-08-25T09:36:27.312 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel x86_64 2.37.4-14.el9 CentOS-AppStream 24 k 2023-08-25T09:36:27.312 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-08-25T09:36:27.312 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-08-25T09:36:27.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-08-25T09:36:27.313 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-08-25T09:36:27.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 161 k 2023-08-25T09:36:27.313 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 320 k 2023-08-25T09:36:27.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 297 k 2023-08-25T09:36:27.314 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 99 k 2023-08-25T09:36:27.314 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 85 k 2023-08-25T09:36:27.314 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 3.0 M 2023-08-25T09:36:27.315 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 169 k 2023-08-25T09:36:27.315 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-08-25T09:36:27.315 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2023-08-25T09:36:27.315 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager x86_64 1:1.44.0-2.el9 CentOS-BaseOS 2.2 M 2023-08-25T09:36:27.315 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm x86_64 1:1.44.0-2.el9 CentOS-BaseOS 1.8 M 2023-08-25T09:36:27.316 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team x86_64 1:1.44.0-2.el9 CentOS-BaseOS 41 k 2023-08-25T09:36:27.316 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui x86_64 1:1.44.0-2.el9 CentOS-BaseOS 245 k 2023-08-25T09:36:27.316 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-debuginfod-client x86_64 0.189-3.el9 CentOS-BaseOS 36 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf x86_64 0.189-3.el9 CentOS-BaseOS 192 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs x86_64 0.189-3.el9 CentOS-BaseOS 254 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: iproute x86_64 6.2.0-5.el9 CentOS-BaseOS 785 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-tc x86_64 6.2.0-5.el9 CentOS-BaseOS 448 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid x86_64 2.37.4-14.el9 CentOS-BaseOS 107 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk x86_64 2.37.4-14.el9 CentOS-BaseOS 154 k 2023-08-25T09:36:27.317 INFO:teuthology.orchestra.run.smithi155.stdout: libmount x86_64 2.37.4-14.el9 CentOS-BaseOS 133 k 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 3.3 M 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 3.0 M 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols x86_64 2.37.4-14.el9 CentOS-BaseOS 63 k 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid x86_64 2.37.4-14.el9 CentOS-BaseOS 28 k 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: openssl x86_64 1:3.0.7-24.el9 CentOS-BaseOS 1.2 M 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs x86_64 1:3.0.7-24.el9 CentOS-BaseOS 2.1 M 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux x86_64 2.37.4-14.el9 CentOS-BaseOS 2.2 M 2023-08-25T09:36:27.318 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-core x86_64 2.37.4-14.el9 CentOS-BaseOS 433 k 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user x86_64 2.37.4-14.el9 CentOS-BaseOS 30 k 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump x86_64 3.1.12-3.el9 CentOS-BaseOS 321 k 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs x86_64 5.19.0-4.el9 CentOS-BaseOS 1.0 M 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 19 M 2023-08-25T09:36:27.319 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 23 k 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 2.1 M 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 242 k 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 4.4 M 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 18 M 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 15 k 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 24 k 2023-08-25T09:36:27.320 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-25T09:36:27.321 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 164 k 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-25T09:36:27.322 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 473 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 4.7 M 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-25T09:36:27.323 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-25T09:36:27.324 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-25T09:36:27.324 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-25T09:36:27.324 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-08-25T09:36:27.324 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-25T09:36:27.324 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-25T09:36:27.324 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-25T09:36:27.325 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-25T09:36:27.325 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 45 k 2023-08-25T09:36:27.325 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 127 k 2023-08-25T09:36:27.325 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-25T09:36:27.325 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-25T09:36:27.325 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-25T09:36:27.326 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-25T09:36:27.326 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-25T09:36:27.326 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-25T09:36:27.326 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-25T09:36:27.327 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-25T09:36:27.328 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-25T09:36:27.328 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-25T09:36:27.328 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-25T09:36:27.328 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-25T09:36:27.328 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-25T09:36:27.328 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-25T09:36:27.329 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-25T09:36:27.330 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-25T09:36:27.331 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-08-25T09:36:27.332 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel x86_64 1.2.11-36.el9 appstream 45 k 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout:Install 131 Packages 2023-08-25T09:36:27.333 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 23 Packages 2023-08-25T09:36:27.334 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:36:27.334 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 206 M 2023-08-25T09:36:27.334 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2023-08-25T09:36:27.636 INFO:teuthology.orchestra.run.smithi155.stdout:(1/154): ceph-18.0.0-5703.gba6b39e4.el9.x86_64. 32 kB/s | 6.5 kB 00:00 2023-08-25T09:36:27.870 INFO:teuthology.orchestra.run.smithi155.stdout:(2/154): ceph-fuse-18.0.0-5703.gba6b39e4.el9.x8 3.6 MB/s | 850 kB 00:00 2023-08-25T09:36:27.961 INFO:teuthology.orchestra.run.smithi155.stdout:(3/154): ceph-immutable-object-cache-18.0.0-570 1.5 MB/s | 141 kB 00:00 2023-08-25T09:36:28.111 INFO:teuthology.orchestra.run.smithi155.stdout:(4/154): ceph-base-18.0.0-5703.gba6b39e4.el9.x8 7.6 MB/s | 5.2 MB 00:00 2023-08-25T09:36:28.245 INFO:teuthology.orchestra.run.smithi155.stdout:(5/154): ceph-mds-18.0.0-5703.gba6b39e4.el9.x86 7.4 MB/s | 2.1 MB 00:00 2023-08-25T09:36:28.320 INFO:teuthology.orchestra.run.smithi155.stdout:(6/154): ceph-mgr-18.0.0-5703.gba6b39e4.el9.x86 7.0 MB/s | 1.5 MB 00:00 2023-08-25T09:36:28.854 INFO:teuthology.orchestra.run.smithi155.stdout:(7/154): ceph-mon-18.0.0-5703.gba6b39e4.el9.x86 7.2 MB/s | 4.4 MB 00:00 2023-08-25T09:36:29.804 INFO:teuthology.orchestra.run.smithi155.stdout:(8/154): ceph-osd-18.0.0-5703.gba6b39e4.el9.x86 12 MB/s | 18 MB 00:01 2023-08-25T09:36:29.854 INFO:teuthology.orchestra.run.smithi155.stdout:(9/154): ceph-selinux-18.0.0-5703.gba6b39e4.el9 495 kB/s | 24 kB 00:00 2023-08-25T09:36:30.038 INFO:teuthology.orchestra.run.smithi155.stdout:(10/154): ceph-radosgw-18.0.0-5703.gba6b39e4.el 6.9 MB/s | 8.2 MB 00:01 2023-08-25T09:36:30.121 INFO:teuthology.orchestra.run.smithi155.stdout:(11/154): cephfs-mirror-18.0.0-5703.gba6b39e4.e 2.6 MB/s | 218 kB 00:00 2023-08-25T09:36:30.205 INFO:teuthology.orchestra.run.smithi155.stdout:(12/154): libcephfs-devel-18.0.0-5703.gba6b39e4 379 kB/s | 31 kB 00:00 2023-08-25T09:36:30.455 INFO:teuthology.orchestra.run.smithi155.stdout:(13/154): ceph-common-18.0.0-5703.gba6b39e4.el9 6.2 MB/s | 19 MB 00:03 2023-08-25T09:36:30.514 INFO:teuthology.orchestra.run.smithi155.stdout:(14/154): libcephfs2-18.0.0-5703.gba6b39e4.el9. 2.1 MB/s | 667 kB 00:00 2023-08-25T09:36:30.564 INFO:teuthology.orchestra.run.smithi155.stdout:(15/154): libcephsqlite-18.0.0-5703.gba6b39e4.e 1.5 MB/s | 164 kB 00:00 2023-08-25T09:36:30.606 INFO:teuthology.orchestra.run.smithi155.stdout:(16/154): librados-devel-18.0.0-5703.gba6b39e4. 1.4 MB/s | 126 kB 00:00 2023-08-25T09:36:30.690 INFO:teuthology.orchestra.run.smithi155.stdout:(17/154): libradosstriper1-18.0.0-5703.gba6b39e 3.7 MB/s | 473 kB 00:00 2023-08-25T09:36:30.773 INFO:teuthology.orchestra.run.smithi155.stdout:(18/154): python3-ceph-argparse-18.0.0-5703.gba 548 kB/s | 45 kB 00:00 2023-08-25T09:36:30.890 INFO:teuthology.orchestra.run.smithi155.stdout:(19/154): python3-ceph-common-18.0.0-5703.gba6b 1.1 MB/s | 127 kB 00:00 2023-08-25T09:36:31.007 INFO:teuthology.orchestra.run.smithi155.stdout:(20/154): python3-cephfs-18.0.0-5703.gba6b39e4. 1.4 MB/s | 161 kB 00:00 2023-08-25T09:36:31.132 INFO:teuthology.orchestra.run.smithi155.stdout:(21/154): librgw2-18.0.0-5703.gba6b39e4.el9.x86 8.9 MB/s | 4.7 MB 00:00 2023-08-25T09:36:31.183 INFO:teuthology.orchestra.run.smithi155.stdout:(22/154): python3-rados-18.0.0-5703.gba6b39e4.e 1.8 MB/s | 320 kB 00:00 2023-08-25T09:36:31.233 INFO:teuthology.orchestra.run.smithi155.stdout:(23/154): python3-rbd-18.0.0-5703.gba6b39e4.el9 2.9 MB/s | 297 kB 00:00 2023-08-25T09:36:31.283 INFO:teuthology.orchestra.run.smithi155.stdout:(24/154): python3-rgw-18.0.0-5703.gba6b39e4.el9 997 kB/s | 99 kB 00:00 2023-08-25T09:36:31.317 INFO:teuthology.orchestra.run.smithi155.stdout:(25/154): rbd-fuse-18.0.0-5703.gba6b39e4.el9.x8 1.0 MB/s | 85 kB 00:00 2023-08-25T09:36:31.404 INFO:teuthology.orchestra.run.smithi155.stdout:(26/154): rbd-nbd-18.0.0-5703.gba6b39e4.el9.x86 1.9 MB/s | 169 kB 00:00 2023-08-25T09:36:31.451 INFO:teuthology.orchestra.run.smithi155.stdout:(27/154): ceph-grafana-dashboards-18.0.0-5703.g 509 kB/s | 23 kB 00:00 2023-08-25T09:36:31.518 INFO:teuthology.orchestra.run.smithi155.stdout:(28/154): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 2.0 MB/s | 132 kB 00:00 2023-08-25T09:36:31.618 INFO:teuthology.orchestra.run.smithi155.stdout:(29/154): rbd-mirror-18.0.0-5703.gba6b39e4.el9. 9.1 MB/s | 3.0 MB 00:00 2023-08-25T09:36:31.727 INFO:teuthology.orchestra.run.smithi155.stdout:(30/154): ceph-mgr-dashboard-18.0.0-5703.gba6b3 8.4 MB/s | 1.7 MB 00:00 2023-08-25T09:36:31.810 INFO:teuthology.orchestra.run.smithi155.stdout:(31/154): ceph-mgr-modules-core-18.0.0-5703.gba 2.9 MB/s | 242 kB 00:00 2023-08-25T09:36:31.886 INFO:teuthology.orchestra.run.smithi155.stdout:(32/154): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 640 kB/s | 47 kB 00:00 2023-08-25T09:36:31.953 INFO:teuthology.orchestra.run.smithi155.stdout:(33/154): ceph-prometheus-alerts-18.0.0-5703.gb 222 kB/s | 15 kB 00:00 2023-08-25T09:36:32.053 INFO:teuthology.orchestra.run.smithi155.stdout:(34/154): ceph-volume-18.0.0-5703.gba6b39e4.el9 2.6 MB/s | 261 kB 00:00 2023-08-25T09:36:32.145 INFO:teuthology.orchestra.run.smithi155.stdout:(35/154): cephadm-18.0.0-5703.gba6b39e4.el9.noa 2.4 MB/s | 225 kB 00:00 2023-08-25T09:36:32.228 INFO:teuthology.orchestra.run.smithi155.stdout:(36/154): cephfs-top-18.0.0-5703.gba6b39e4.el9. 219 kB/s | 18 kB 00:00 2023-08-25T09:36:32.854 INFO:teuthology.orchestra.run.smithi155.stdout:(37/154): ceph-test-18.0.0-5703.gba6b39e4.el9.x 13 MB/s | 40 MB 00:02 2023-08-25T09:36:32.896 INFO:teuthology.orchestra.run.smithi155.stdout:(38/154): automake-1.16.2-8.el9.noarch.rpm 994 kB/s | 662 kB 00:00 2023-08-25T09:36:32.971 INFO:teuthology.orchestra.run.smithi155.stdout:(39/154): bison-3.7.4-5.el9.x86_64.rpm 7.9 MB/s | 920 kB 00:00 2023-08-25T09:36:33.013 INFO:teuthology.orchestra.run.smithi155.stdout:(40/154): boost-program-options-1.75.0-8.el9.x8 974 kB/s | 106 kB 00:00 2023-08-25T09:36:33.038 INFO:teuthology.orchestra.run.smithi155.stdout:(41/154): elfutils-libelf-devel-0.189-3.el9.x86 327 kB/s | 21 kB 00:00 2023-08-25T09:36:33.080 INFO:teuthology.orchestra.run.smithi155.stdout:(42/154): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 307 kB 00:00 2023-08-25T09:36:33.231 INFO:teuthology.orchestra.run.smithi155.stdout:(43/154): ceph-mgr-diskprediction-local-18.0.0- 4.6 MB/s | 7.4 MB 00:01 2023-08-25T09:36:33.255 INFO:teuthology.orchestra.run.smithi155.stdout:(44/154): flexiblas-3.0.4-8.el9.x86_64.rpm 148 kB/s | 32 kB 00:00 2023-08-25T09:36:33.381 INFO:teuthology.orchestra.run.smithi155.stdout:(45/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 10 MB/s | 3.0 MB 00:00 2023-08-25T09:36:33.406 INFO:teuthology.orchestra.run.smithi155.stdout:(46/154): flexiblas-openblas-openmp-3.0.4-8.el9 128 kB/s | 17 kB 00:00 2023-08-25T09:36:33.448 INFO:teuthology.orchestra.run.smithi155.stdout:(47/154): jq-1.6-15.el9.x86_64.rpm 1.0 MB/s | 186 kB 00:00 2023-08-25T09:36:33.498 INFO:teuthology.orchestra.run.smithi155.stdout:(48/154): libaio-devel-0.3.111-13.el9.x86_64.rp 207 kB/s | 10 kB 00:00 2023-08-25T09:36:33.523 INFO:teuthology.orchestra.run.smithi155.stdout:(49/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 44 kB 00:00 2023-08-25T09:36:33.566 INFO:teuthology.orchestra.run.smithi155.stdout:(50/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 159 kB 00:00 2023-08-25T09:36:33.628 INFO:teuthology.orchestra.run.smithi155.stdout:(51/154): libtool-2.4.6-45.el9.x86_64.rpm 5.8 MB/s | 578 kB 00:00 2023-08-25T09:36:33.682 INFO:teuthology.orchestra.run.smithi155.stdout:(52/154): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2023-08-25T09:36:33.707 INFO:teuthology.orchestra.run.smithi155.stdout:(53/154): libuuid-devel-2.37.4-14.el9.x86_64.rp 181 kB/s | 24 kB 00:00 2023-08-25T09:36:33.733 INFO:teuthology.orchestra.run.smithi155.stdout:(54/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 458 kB/s | 46 kB 00:00 2023-08-25T09:36:33.780 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 1.2 MB/s | 5.7 MB 00:04 2023-08-25T09:36:33.792 INFO:teuthology.orchestra.run.smithi155.stdout:(55/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2023-08-25T09:36:33.834 INFO:teuthology.orchestra.run.smithi155.stdout:(56/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2023-08-25T09:36:33.850 INFO:teuthology.orchestra.run.smithi155.stdout:(57/154): openblas-0.3.21-2.el9.x86_64.rpm 298 kB/s | 34 kB 00:00 2023-08-25T09:36:34.018 INFO:teuthology.orchestra.run.smithi155.stdout:(58/154): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2023-08-25T09:36:34.109 INFO:teuthology.orchestra.run.smithi155.stdout:(59/154): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.5 MB 00:00 2023-08-25T09:36:34.210 INFO:teuthology.orchestra.run.smithi155.stdout:(60/154): openssl-devel-3.0.7-24.el9.x86_64.rpm 8.7 MB/s | 3.0 MB 00:00 2023-08-25T09:36:34.260 INFO:teuthology.orchestra.run.smithi155.stdout:(61/154): postgresql-contrib-13.11-1.el9.x86_64 3.4 MB/s | 811 kB 00:00 2023-08-25T09:36:34.302 INFO:teuthology.orchestra.run.smithi155.stdout:(62/154): postgresql-private-libs-13.11-1.el9.x 782 kB/s | 134 kB 00:00 2023-08-25T09:36:34.535 INFO:teuthology.orchestra.run.smithi155.stdout:(63/154): postgresql-server-13.11-1.el9.x86_64. 21 MB/s | 5.7 MB 00:00 2023-08-25T09:36:34.644 INFO:teuthology.orchestra.run.smithi155.stdout:(64/154): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 5.8 MB 00:00 2023-08-25T09:36:34.686 INFO:teuthology.orchestra.run.smithi155.stdout:(65/154): python3-jinja2-2.11.3-4.el9.noarch.rp 608 kB/s | 228 kB 00:00 2023-08-25T09:36:34.703 INFO:teuthology.orchestra.run.smithi155.stdout:(66/154): python3-jsonpatch-1.21-16.el9.noarch. 150 kB/s | 24 kB 00:00 2023-08-25T09:36:34.720 INFO:teuthology.orchestra.run.smithi155.stdout:(67/154): python3-jsonpointer-2.0-4.el9.noarch. 565 kB/s | 18 kB 00:00 2023-08-25T09:36:34.753 INFO:teuthology.orchestra.run.smithi155.stdout:(68/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 144 kB 00:00 2023-08-25T09:36:34.770 INFO:teuthology.orchestra.run.smithi155.stdout:(69/154): python3-markupsafe-1.1.1-12.el9.x86_6 489 kB/s | 32 kB 00:00 2023-08-25T09:36:34.820 INFO:teuthology.orchestra.run.smithi155.stdout:(70/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 217 kB 00:00 2023-08-25T09:36:34.979 INFO:teuthology.orchestra.run.smithi155.stdout:(71/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2023-08-25T09:36:35.013 INFO:teuthology.orchestra.run.smithi155.stdout:(72/154): python3-oauthlib-3.1.1-5.el9.noarch.r 741 kB/s | 174 kB 00:00 2023-08-25T09:36:35.062 INFO:teuthology.orchestra.run.smithi155.stdout:(73/154): python3-prettytable-0.7.2-27.el9.noar 182 kB/s | 41 kB 00:00 2023-08-25T09:36:35.113 INFO:teuthology.orchestra.run.smithi155.stdout:(74/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2023-08-25T09:36:35.155 INFO:teuthology.orchestra.run.smithi155.stdout:(75/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 211 kB 00:00 2023-08-25T09:36:35.347 INFO:teuthology.orchestra.run.smithi155.stdout:(76/154): python3-pytz-2021.1-5.el9.noarch.rpm 168 kB/s | 47 kB 00:00 2023-08-25T09:36:35.388 INFO:teuthology.orchestra.run.smithi155.stdout:(77/154): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2023-08-25T09:36:35.413 INFO:teuthology.orchestra.run.smithi155.stdout:(78/154): python3-toml-0.10.2-6.el9.noarch.rpm 595 kB/s | 39 kB 00:00 2023-08-25T09:36:35.514 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 14 kB 00:00 2023-08-25T09:36:35.781 INFO:teuthology.orchestra.run.smithi155.stdout:(79/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2023-08-25T09:36:35.806 INFO:teuthology.orchestra.run.smithi155.stdout:(80/154): uuid-1.6.2-55.el9.x86_64.rpm 151 kB/s | 56 kB 00:00 2023-08-25T09:36:35.856 INFO:teuthology.orchestra.run.smithi155.stdout:(81/154): socat-1.7.4.1-5.el9.x86_64.rpm 706 kB/s | 300 kB 00:00 2023-08-25T09:36:35.890 INFO:teuthology.orchestra.run.smithi155.stdout:(82/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.0 MB/s | 85 kB 00:00 2023-08-25T09:36:35.906 INFO:teuthology.orchestra.run.smithi155.stdout:(83/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 651 kB/s | 63 kB 00:00 2023-08-25T09:36:35.940 INFO:teuthology.orchestra.run.smithi155.stdout:(84/154): fuse-2.9.9-15.el9.x86_64.rpm 968 kB/s | 79 kB 00:00 2023-08-25T09:36:35.982 INFO:teuthology.orchestra.run.smithi155.stdout:(85/154): libquadmath-11.4.1-2.1.el9.x86_64.rpm 2.6 MB/s | 197 kB 00:00 2023-08-25T09:36:36.049 INFO:teuthology.orchestra.run.smithi155.stdout:(86/154): libgfortran-11.4.1-2.1.el9.x86_64.rpm 5.6 MB/s | 807 kB 00:00 2023-08-25T09:36:36.082 INFO:teuthology.orchestra.run.smithi155.stdout:(87/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 241 kB 00:00 2023-08-25T09:36:36.149 INFO:teuthology.orchestra.run.smithi155.stdout:(88/154): python3-cryptography-36.0.1-4.el9.x86 7.3 MB/s | 1.1 MB 00:00 2023-08-25T09:36:36.192 INFO:teuthology.orchestra.run.smithi155.stdout:(89/154): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2023-08-25T09:36:36.225 INFO:teuthology.orchestra.run.smithi155.stdout:(90/154): python3-ply-3.11-14.el9.noarch.rpm 791 kB/s | 103 kB 00:00 2023-08-25T09:36:36.350 INFO:teuthology.orchestra.run.smithi155.stdout:(91/154): zlib-devel-1.2.11-36.el9.x86_64.rpm 288 kB/s | 45 kB 00:00 2023-08-25T09:36:36.450 INFO:teuthology.orchestra.run.smithi155.stdout:(92/154): python3-devel-3.9.16-1.el9.x86_64.rpm 827 kB/s | 247 kB 00:00 2023-08-25T09:36:36.993 INFO:teuthology.orchestra.run.smithi155.stdout:(93/154): fmt-8.1.1-5.el9.x86_64.rpm 144 kB/s | 111 kB 00:00 2023-08-25T09:36:37.118 INFO:teuthology.orchestra.run.smithi155.stdout:(94/154): libarrow-doc-9.0.0-9.el9.noarch.rpm 197 kB/s | 24 kB 00:00 2023-08-25T09:36:37.160 INFO:teuthology.orchestra.run.smithi155.stdout:(95/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 382 kB/s | 309 kB 00:00 2023-08-25T09:36:37.260 INFO:teuthology.orchestra.run.smithi155.stdout:(96/154): liboath-2.6.7-2.el9.x86_64.rpm 347 kB/s | 49 kB 00:00 2023-08-25T09:36:37.310 INFO:teuthology.orchestra.run.smithi155.stdout:(97/154): libunwind-1.6.2-1.el9.x86_64.rpm 452 kB/s | 67 kB 00:00 2023-08-25T09:36:37.594 INFO:teuthology.orchestra.run.smithi155.stdout:(98/154): libarrow-9.0.0-9.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2023-08-25T09:36:37.646 INFO:teuthology.orchestra.run.smithi155.stdout:(99/154): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.2 MB/s | 837 kB 00:00 2023-08-25T09:36:37.707 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 18 MB 00:02 2023-08-25T09:36:37.720 INFO:teuthology.orchestra.run.smithi155.stdout:(100/154): python3-asyncssh-2.13.2-3.el9.noarch 1.3 MB/s | 548 kB 00:00 2023-08-25T09:36:37.745 INFO:teuthology.orchestra.run.smithi155.stdout:(101/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 300 kB/s | 43 kB 00:00 2023-08-25T09:36:37.770 INFO:teuthology.orchestra.run.smithi155.stdout:(102/154): python3-cachetools-4.2.4-1.el9.noarc 265 kB/s | 32 kB 00:00 2023-08-25T09:36:37.820 INFO:teuthology.orchestra.run.smithi155.stdout:(103/154): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2023-08-25T09:36:37.871 INFO:teuthology.orchestra.run.smithi155.stdout:(104/154): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2023-08-25T09:36:37.929 INFO:teuthology.orchestra.run.smithi155.stdout:(105/154): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2023-08-25T09:36:37.971 INFO:teuthology.orchestra.run.smithi155.stdout:(106/154): python3-google-auth-2.22.0-1.el9.noa 1.3 MB/s | 197 kB 00:00 2023-08-25T09:36:37.996 INFO:teuthology.orchestra.run.smithi155.stdout:(107/154): python3-jaraco-8.2.1-3.el9.noarch.rp 88 kB/s | 11 kB 00:00 2023-08-25T09:36:38.038 INFO:teuthology.orchestra.run.smithi155.stdout:(108/154): python3-jaraco-classes-3.2.1-5.el9.n 166 kB/s | 18 kB 00:00 2023-08-25T09:36:38.080 INFO:teuthology.orchestra.run.smithi155.stdout:(109/154): python3-jaraco-collections-3.0.0-8.e 214 kB/s | 23 kB 00:00 2023-08-25T09:36:38.105 INFO:teuthology.orchestra.run.smithi155.stdout:(110/154): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2023-08-25T09:36:38.139 INFO:teuthology.orchestra.run.smithi155.stdout:(111/154): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2023-08-25T09:36:38.181 INFO:teuthology.orchestra.run.smithi155.stdout:(112/154): python3-jwt+crypto-2.4.0-1.el9.noarc 91 kB/s | 9.0 kB 00:00 2023-08-25T09:36:38.206 INFO:teuthology.orchestra.run.smithi155.stdout:(113/154): python3-jwt-2.4.0-1.el9.noarch.rpm 409 kB/s | 41 kB 00:00 2023-08-25T09:36:38.331 INFO:teuthology.orchestra.run.smithi155.stdout:(114/154): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2023-08-25T09:36:38.357 INFO:teuthology.orchestra.run.smithi155.stdout:(115/154): python3-logutils-0.3.5-21.el9.noarch 264 kB/s | 46 kB 00:00 2023-08-25T09:36:38.390 INFO:teuthology.orchestra.run.smithi155.stdout:(116/154): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2023-08-25T09:36:38.432 INFO:teuthology.orchestra.run.smithi155.stdout:(117/154): python3-natsort-7.1.1-5.el9.noarch.r 586 kB/s | 58 kB 00:00 2023-08-25T09:36:38.516 INFO:teuthology.orchestra.run.smithi155.stdout:(118/154): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2023-08-25T09:36:38.533 INFO:teuthology.orchestra.run.smithi155.stdout:(119/154): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2023-08-25T09:36:38.592 INFO:teuthology.orchestra.run.smithi155.stdout:(120/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 567 kB/s | 90 kB 00:00 2023-08-25T09:36:38.609 INFO:teuthology.orchestra.run.smithi155.stdout:(121/154): python3-repoze-lru-0.7-16.el9.noarch 340 kB/s | 31 kB 00:00 2023-08-25T09:36:38.725 INFO:teuthology.orchestra.run.smithi155.stdout:(122/154): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2023-08-25T09:36:38.750 INFO:teuthology.orchestra.run.smithi155.stdout:(123/154): python3-rsa-4.9-2.el9.noarch.rpm 375 kB/s | 59 kB 00:00 2023-08-25T09:36:38.777 INFO:teuthology.orchestra.run.smithi155.stdout:(124/154): python3-tempora-5.0.0-2.el9.noarch.r 215 kB/s | 36 kB 00:00 2023-08-25T09:36:38.834 INFO:teuthology.orchestra.run.smithi155.stdout:(125/154): python3-typing-extensions-4.4.0-2.el 477 kB/s | 51 kB 00:00 2023-08-25T09:36:38.893 INFO:teuthology.orchestra.run.smithi155.stdout:(126/154): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2023-08-25T09:36:38.935 INFO:teuthology.orchestra.run.smithi155.stdout:(127/154): python3-websocket-client-1.2.3-2.el9 569 kB/s | 90 kB 00:00 2023-08-25T09:36:38.977 INFO:teuthology.orchestra.run.smithi155.stdout:(128/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2023-08-25T09:36:39.002 INFO:teuthology.orchestra.run.smithi155.stdout:(129/154): python3-zc-lockfile-2.0-10.el9.noarc 186 kB/s | 20 kB 00:00 2023-08-25T09:36:39.061 INFO:teuthology.orchestra.run.smithi155.stdout:(130/154): re2-20211101-2.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2023-08-25T09:36:39.161 INFO:teuthology.orchestra.run.smithi155.stdout:(131/154): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2023-08-25T09:36:39.370 INFO:teuthology.orchestra.run.smithi155.stdout:(132/154): NetworkManager-1.44.0-2.el9.x86_64.r 11 MB/s | 2.2 MB 00:00 2023-08-25T09:36:39.470 INFO:teuthology.orchestra.run.smithi155.stdout:(133/154): NetworkManager-libnm-1.44.0-2.el9.x8 18 MB/s | 1.8 MB 00:00 2023-08-25T09:36:39.512 INFO:teuthology.orchestra.run.smithi155.stdout:(134/154): NetworkManager-team-1.44.0-2.el9.x86 1.0 MB/s | 41 kB 00:00 2023-08-25T09:36:39.562 INFO:teuthology.orchestra.run.smithi155.stdout:(135/154): NetworkManager-tui-1.44.0-2.el9.x86_ 4.9 MB/s | 245 kB 00:00 2023-08-25T09:36:39.596 INFO:teuthology.orchestra.run.smithi155.stdout:(136/154): elfutils-debuginfod-client-0.189-3.e 1.1 MB/s | 36 kB 00:00 2023-08-25T09:36:39.646 INFO:teuthology.orchestra.run.smithi155.stdout:(137/154): elfutils-libelf-0.189-3.el9.x86_64.r 3.8 MB/s | 192 kB 00:00 2023-08-25T09:36:39.738 INFO:teuthology.orchestra.run.smithi155.stdout:(138/154): librbd1-18.0.0-5703.gba6b39e4.el9.x8 4.5 MB/s | 3.0 MB 00:00 2023-08-25T09:36:39.805 INFO:teuthology.orchestra.run.smithi155.stdout:(139/154): elfutils-libs-0.189-3.el9.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2023-08-25T09:36:39.864 INFO:teuthology.orchestra.run.smithi155.stdout:(140/154): iproute-6.2.0-5.el9.x86_64.rpm 6.3 MB/s | 785 kB 00:00 2023-08-25T09:36:39.914 INFO:teuthology.orchestra.run.smithi155.stdout:(141/154): iproute-tc-6.2.0-5.el9.x86_64.rpm 4.4 MB/s | 448 kB 00:00 2023-08-25T09:36:39.948 INFO:teuthology.orchestra.run.smithi155.stdout:(142/154): libblkid-2.37.4-14.el9.x86_64.rpm 1.3 MB/s | 107 kB 00:00 2023-08-25T09:36:39.990 INFO:teuthology.orchestra.run.smithi155.stdout:(143/154): libfdisk-2.37.4-14.el9.x86_64.rpm 2.0 MB/s | 154 kB 00:00 2023-08-25T09:36:40.023 INFO:teuthology.orchestra.run.smithi155.stdout:(144/154): libmount-2.37.4-14.el9.x86_64.rpm 1.8 MB/s | 133 kB 00:00 2023-08-25T09:36:40.049 INFO:teuthology.orchestra.run.smithi155.stdout:(145/154): libsmartcols-2.37.4-14.el9.x86_64.rp 1.1 MB/s | 63 kB 00:00 2023-08-25T09:36:40.074 INFO:teuthology.orchestra.run.smithi155.stdout:(146/154): libuuid-2.37.4-14.el9.x86_64.rpm 571 kB/s | 28 kB 00:00 2023-08-25T09:36:40.141 INFO:teuthology.orchestra.run.smithi155.stdout:(147/154): openssl-3.0.7-24.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2023-08-25T09:36:40.233 INFO:teuthology.orchestra.run.smithi155.stdout:(148/154): openssl-libs-3.0.7-24.el9.x86_64.rpm 15 MB/s | 2.1 MB 00:00 2023-08-25T09:36:40.325 INFO:teuthology.orchestra.run.smithi155.stdout:(149/154): util-linux-2.37.4-14.el9.x86_64.rpm 13 MB/s | 2.2 MB 00:00 2023-08-25T09:36:40.375 INFO:teuthology.orchestra.run.smithi155.stdout:(150/154): util-linux-core-2.37.4-14.el9.x86_64 3.3 MB/s | 433 kB 00:00 2023-08-25T09:36:40.400 INFO:teuthology.orchestra.run.smithi155.stdout:(151/154): util-linux-user-2.37.4-14.el9.x86_64 427 kB/s | 30 kB 00:00 2023-08-25T09:36:40.442 INFO:teuthology.orchestra.run.smithi155.stdout:(152/154): xfsdump-3.1.12-3.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2023-08-25T09:36:40.509 INFO:teuthology.orchestra.run.smithi155.stdout:(153/154): xfsprogs-5.19.0-4.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2023-08-25T09:36:40.593 INFO:teuthology.orchestra.run.smithi155.stdout:(154/154): librados2-18.0.0-5703.gba6b39e4.el9. 2.1 MB/s | 3.3 MB 00:01 2023-08-25T09:36:40.606 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2023-08-25T09:36:40.607 INFO:teuthology.orchestra.run.smithi155.stdout:Total 15 MB/s | 206 MB 00:13 2023-08-25T09:36:40.700 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2023-08-25T09:36:41.015 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2023-08-25T09:36:41.015 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2023-08-25T09:36:43.108 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2023-08-25T09:36:43.109 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2023-08-25T09:36:44.810 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-25T09:36:44.812 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-08-25T09:36:44.832 INFO:teuthology.orchestra.run.smithi070.stdout:Package NetworkManager-1:1.41.8-1.el9.x86_64 is already installed. 2023-08-25T09:36:44.834 INFO:teuthology.orchestra.run.smithi070.stdout:Package iproute-6.0.0-2.el9.x86_64 is already installed. 2023-08-25T09:36:44.835 INFO:teuthology.orchestra.run.smithi070.stdout:Package util-linux-2.37.4-9.el9.x86_64 is already installed. 2023-08-25T09:36:44.836 INFO:teuthology.orchestra.run.smithi070.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-08-25T09:36:44.839 INFO:teuthology.orchestra.run.smithi070.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-08-25T09:36:44.843 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsdump-3.1.10-1.el9.x86_64 is already installed. 2023-08-25T09:36:44.844 INFO:teuthology.orchestra.run.smithi070.stdout:Package xfsprogs-5.14.2-1.el9.x86_64 is already installed. 2023-08-25T09:36:44.927 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2023-08-25T09:36:44.956 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2023-08-25T09:36:44.956 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2023-08-25T09:36:44.956 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2023-08-25T09:36:44.956 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2023-08-25T09:36:44.957 INFO:teuthology.orchestra.run.smithi070.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-08-25T09:36:44.957 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 6.5 k 2023-08-25T09:36:44.957 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 5.2 M 2023-08-25T09:36:44.957 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 850 k 2023-08-25T09:36:44.957 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 141 k 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 1.5 M 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 132 k 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 1.7 M 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 7.4 M 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 47 k 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 8.2 M 2023-08-25T09:36:44.958 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 40 M 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 261 k 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 225 k 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 218 k 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 18 k 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 CentOS-AppStream 21 k 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-08-25T09:36:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 31 k 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 667 k 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 126 k 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel x86_64 2.37.4-14.el9 CentOS-AppStream 24 k 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel x86_64 1:3.0.7-24.el9 CentOS-AppStream 3.0 M 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-08-25T09:36:44.960 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 161 k 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 320 k 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 297 k 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 99 k 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 85 k 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 3.0 M 2023-08-25T09:36:44.961 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 169 k 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager x86_64 1:1.44.0-2.el9 CentOS-BaseOS 2.2 M 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm x86_64 1:1.44.0-2.el9 CentOS-BaseOS 1.8 M 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team x86_64 1:1.44.0-2.el9 CentOS-BaseOS 41 k 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui x86_64 1:1.44.0-2.el9 CentOS-BaseOS 245 k 2023-08-25T09:36:44.962 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-debuginfod-client x86_64 0.189-3.el9 CentOS-BaseOS 36 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf x86_64 0.189-3.el9 CentOS-BaseOS 192 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libs x86_64 0.189-3.el9 CentOS-BaseOS 254 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: iproute x86_64 6.2.0-5.el9 CentOS-BaseOS 785 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: iproute-tc x86_64 6.2.0-5.el9 CentOS-BaseOS 448 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: libblkid x86_64 2.37.4-14.el9 CentOS-BaseOS 107 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: libfdisk x86_64 2.37.4-14.el9 CentOS-BaseOS 154 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: libmount x86_64 2.37.4-14.el9 CentOS-BaseOS 133 k 2023-08-25T09:36:44.963 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 3.3 M 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 3.0 M 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: libsmartcols x86_64 2.37.4-14.el9 CentOS-BaseOS 63 k 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid x86_64 2.37.4-14.el9 CentOS-BaseOS 28 k 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: openssl x86_64 1:3.0.7-24.el9 CentOS-BaseOS 1.2 M 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-libs x86_64 1:3.0.7-24.el9 CentOS-BaseOS 2.1 M 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: util-linux x86_64 2.37.4-14.el9 CentOS-BaseOS 2.2 M 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: util-linux-core x86_64 2.37.4-14.el9 CentOS-BaseOS 433 k 2023-08-25T09:36:44.964 INFO:teuthology.orchestra.run.smithi070.stdout: util-linux-user x86_64 2.37.4-14.el9 CentOS-BaseOS 30 k 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: xfsdump x86_64 3.1.12-3.el9 CentOS-BaseOS 321 k 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs x86_64 5.19.0-4.el9 CentOS-BaseOS 1.0 M 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 19 M 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 23 k 2023-08-25T09:36:44.965 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 2.1 M 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 242 k 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 4.4 M 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 18 M 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el9 ceph-noarch 15 k 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 24 k 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-08-25T09:36:44.966 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: jq x86_64 1.6-15.el9 CentOS-AppStream 186 k 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-08-25T09:36:44.967 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 164 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.4.1-2.1.el9 CentOS-BaseOS 807 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.4.1-2.1.el9 CentOS-BaseOS 197 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 473 k 2023-08-25T09:36:44.968 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 4.7 M 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-08-25T09:36:44.969 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 45 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el9 ceph 127 k 2023-08-25T09:36:44.970 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.16-1.el9 appstream 247 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.22.0-1.el9 epel 197 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-08-25T09:36:44.971 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-08-25T09:36:44.972 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-08-25T09:36:44.973 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-08-25T09:36:44.974 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-08-25T09:36:44.975 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-2.el9 epel 192 k 2023-08-25T09:36:44.976 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel x86_64 1.2.11-36.el9 appstream 45 k 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:36:44.977 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2023-08-25T09:36:44.978 INFO:teuthology.orchestra.run.smithi070.stdout:============================================================================================ 2023-08-25T09:36:44.978 INFO:teuthology.orchestra.run.smithi070.stdout:Install 131 Packages 2023-08-25T09:36:44.978 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 23 Packages 2023-08-25T09:36:44.978 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:36:44.979 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 206 M 2023-08-25T09:36:44.979 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2023-08-25T09:36:45.989 INFO:teuthology.orchestra.run.smithi070.stdout:(1/154): ceph-18.0.0-5703.gba6b39e4.el9.x86_64. 18 kB/s | 6.5 kB 00:00 2023-08-25T09:36:46.584 INFO:teuthology.orchestra.run.smithi070.stdout:(2/154): ceph-fuse-18.0.0-5703.gba6b39e4.el9.x8 1.4 MB/s | 850 kB 00:00 2023-08-25T09:36:46.709 INFO:teuthology.orchestra.run.smithi070.stdout:(3/154): ceph-immutable-object-cache-18.0.0-570 1.1 MB/s | 141 kB 00:00 2023-08-25T09:36:47.001 INFO:teuthology.orchestra.run.smithi070.stdout:(4/154): ceph-common-18.0.0-5703.gba6b39e4.el9. 14 MB/s | 19 MB 00:01 2023-08-25T09:36:47.168 INFO:teuthology.orchestra.run.smithi070.stdout:(5/154): ceph-mgr-18.0.0-5703.gba6b39e4.el9.x86 8.8 MB/s | 1.5 MB 00:00 2023-08-25T09:36:47.402 INFO:teuthology.orchestra.run.smithi070.stdout:(6/154): ceph-base-18.0.0-5703.gba6b39e4.el9.x8 2.9 MB/s | 5.2 MB 00:01 2023-08-25T09:36:47.485 INFO:teuthology.orchestra.run.smithi070.stdout:(7/154): ceph-mds-18.0.0-5703.gba6b39e4.el9.x86 2.7 MB/s | 2.1 MB 00:00 2023-08-25T09:36:47.627 INFO:teuthology.orchestra.run.smithi070.stdout:(8/154): ceph-mon-18.0.0-5703.gba6b39e4.el9.x86 9.5 MB/s | 4.4 MB 00:00 2023-08-25T09:36:47.677 INFO:teuthology.orchestra.run.smithi070.stdout:(9/154): ceph-selinux-18.0.0-5703.gba6b39e4.el9 493 kB/s | 24 kB 00:00 2023-08-25T09:36:49.386 INFO:teuthology.orchestra.run.smithi070.stdout:(10/154): ceph-osd-18.0.0-5703.gba6b39e4.el9.x8 8.9 MB/s | 18 MB 00:01 2023-08-25T09:36:49.561 INFO:teuthology.orchestra.run.smithi070.stdout:(11/154): ceph-radosgw-18.0.0-5703.gba6b39e4.el 3.9 MB/s | 8.2 MB 00:02 2023-08-25T09:36:49.603 INFO:teuthology.orchestra.run.smithi070.stdout:(12/154): cephfs-mirror-18.0.0-5703.gba6b39e4.e 1.0 MB/s | 218 kB 00:00 2023-08-25T09:36:49.628 INFO:teuthology.orchestra.run.smithi070.stdout:(13/154): libcephfs-devel-18.0.0-5703.gba6b39e4 474 kB/s | 31 kB 00:00 2023-08-25T09:36:49.694 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2023-08-25T09:36:49.712 INFO:teuthology.orchestra.run.smithi070.stdout:(14/154): libcephsqlite-18.0.0-5703.gba6b39e4.e 1.9 MB/s | 164 kB 00:00 2023-08-25T09:36:49.771 INFO:teuthology.orchestra.run.smithi070.stdout:(15/154): libcephfs2-18.0.0-5703.gba6b39e4.el9. 3.9 MB/s | 667 kB 00:00 2023-08-25T09:36:49.809 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-libs-1:3.0.7-24.el9.x86_64 1/177 2023-08-25T09:36:49.812 INFO:teuthology.orchestra.run.smithi070.stdout:(16/154): librados-devel-18.0.0-5703.gba6b39e4. 1.2 MB/s | 126 kB 00:00 2023-08-25T09:36:49.863 INFO:teuthology.orchestra.run.smithi070.stdout:(17/154): libradosstriper1-18.0.0-5703.gba6b39e 5.1 MB/s | 473 kB 00:00 2023-08-25T09:36:49.873 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libuuid-2.37.4-14.el9.x86_64 2/177 2023-08-25T09:36:49.928 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libblkid-2.37.4-14.el9.x86_64 3/177 2023-08-25T09:36:49.930 INFO:teuthology.orchestra.run.smithi070.stdout:(18/154): python3-ceph-argparse-18.0.0-5703.gba 688 kB/s | 45 kB 00:00 2023-08-25T09:36:50.038 INFO:teuthology.orchestra.run.smithi070.stdout:(19/154): python3-ceph-common-18.0.0-5703.gba6b 1.2 MB/s | 127 kB 00:00 2023-08-25T09:36:50.046 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libblkid-2.37.4-14.el9.x86_64 3/177 2023-08-25T09:36:50.131 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/177 2023-08-25T09:36:50.147 INFO:teuthology.orchestra.run.smithi070.stdout:(20/154): python3-cephfs-18.0.0-5703.gba6b39e4. 1.5 MB/s | 161 kB 00:00 2023-08-25T09:36:50.212 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libelf-0.189-3.el9.x86_64 5/177 2023-08-25T09:36:50.264 INFO:teuthology.orchestra.run.smithi070.stdout:(21/154): python3-rados-18.0.0-5703.gba6b39e4.e 2.7 MB/s | 320 kB 00:00 2023-08-25T09:36:50.303 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/177 2023-08-25T09:36:50.373 INFO:teuthology.orchestra.run.smithi070.stdout:(22/154): python3-rbd-18.0.0-5703.gba6b39e4.el9 2.7 MB/s | 297 kB 00:00 2023-08-25T09:36:50.390 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/177 2023-08-25T09:36:50.436 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/177 2023-08-25T09:36:50.596 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/177 2023-08-25T09:36:50.865 INFO:teuthology.orchestra.run.smithi070.stdout:(23/154): ceph-test-18.0.0-5703.gba6b39e4.el9.x 13 MB/s | 40 MB 00:03 2023-08-25T09:36:50.907 INFO:teuthology.orchestra.run.smithi070.stdout:(24/154): python3-rgw-18.0.0-5703.gba6b39e4.el9 186 kB/s | 99 kB 00:00 2023-08-25T09:36:50.940 INFO:teuthology.orchestra.run.smithi070.stdout:(25/154): rbd-fuse-18.0.0-5703.gba6b39e4.el9.x8 1.1 MB/s | 85 kB 00:00 2023-08-25T09:36:51.065 INFO:teuthology.orchestra.run.smithi070.stdout:(26/154): librgw2-18.0.0-5703.gba6b39e4.el9.x86 3.7 MB/s | 4.7 MB 00:01 2023-08-25T09:36:51.107 INFO:teuthology.orchestra.run.smithi070.stdout:(27/154): rbd-nbd-18.0.0-5703.gba6b39e4.el9.x86 1.0 MB/s | 169 kB 00:00 2023-08-25T09:36:51.133 INFO:teuthology.orchestra.run.smithi070.stdout:(28/154): ceph-grafana-dashboards-18.0.0-5703.g 349 kB/s | 23 kB 00:00 2023-08-25T09:36:51.183 INFO:teuthology.orchestra.run.smithi070.stdout:(29/154): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 1.7 MB/s | 132 kB 00:00 2023-08-25T09:36:51.204 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/177 2023-08-25T09:36:51.275 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 11/177 2023-08-25T09:36:51.311 INFO:teuthology.orchestra.run.smithi070.stdout:(30/154): rbd-mirror-18.0.0-5703.gba6b39e4.el9. 7.5 MB/s | 3.0 MB 00:00 2023-08-25T09:36:51.411 INFO:teuthology.orchestra.run.smithi070.stdout:(31/154): ceph-mgr-modules-core-18.0.0-5703.gba 2.4 MB/s | 242 kB 00:00 2023-08-25T09:36:51.438 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 11/177 2023-08-25T09:36:51.485 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 12/177 2023-08-25T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout:(32/154): ceph-mgr-dashboard-18.0.0-5703.gba6b3 4.5 MB/s | 1.7 MB 00:00 2023-08-25T09:36:51.545 INFO:teuthology.orchestra.run.smithi070.stdout:(33/154): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 358 kB/s | 47 kB 00:00 2023-08-25T09:36:51.586 INFO:teuthology.orchestra.run.smithi070.stdout:(34/154): ceph-prometheus-alerts-18.0.0-5703.gb 222 kB/s | 15 kB 00:00 2023-08-25T09:36:51.602 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 12/177 2023-08-25T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout:(35/154): ceph-volume-18.0.0-5703.gba6b39e4.el9 2.8 MB/s | 261 kB 00:00 2023-08-25T09:36:51.644 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el9.x86_64 13/177 2023-08-25T09:36:51.687 INFO:teuthology.orchestra.run.smithi070.stdout:(36/154): cephadm-18.0.0-5703.gba6b39e4.el9.noa 2.2 MB/s | 225 kB 00:00 2023-08-25T09:36:51.713 INFO:teuthology.orchestra.run.smithi070.stdout:(37/154): cephfs-top-18.0.0-5703.gba6b39e4.el9. 244 kB/s | 18 kB 00:00 2023-08-25T09:36:51.863 INFO:teuthology.orchestra.run.smithi070.stdout:(38/154): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-08-25T09:36:51.950 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/177 2023-08-25T09:36:52.022 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/177 2023-08-25T09:36:52.056 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 16/177 2023-08-25T09:36:52.121 INFO:teuthology.orchestra.run.smithi070.stdout:(39/154): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 662 kB 00:00 2023-08-25T09:36:52.163 INFO:teuthology.orchestra.run.smithi070.stdout:(40/154): boost-program-options-1.75.0-8.el9.x8 571 kB/s | 106 kB 00:00 2023-08-25T09:36:52.199 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 16/177 2023-08-25T09:36:52.213 INFO:teuthology.orchestra.run.smithi070.stdout:(41/154): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 920 kB 00:00 2023-08-25T09:36:52.231 INFO:teuthology.orchestra.run.smithi070.stdout:(42/154): elfutils-libelf-devel-0.189-3.el9.x86 205 kB/s | 21 kB 00:00 2023-08-25T09:36:52.240 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86 17/177 2023-08-25T09:36:52.289 INFO:teuthology.orchestra.run.smithi070.stdout:(43/154): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 307 kB 00:00 2023-08-25T09:36:52.331 INFO:teuthology.orchestra.run.smithi070.stdout:(44/154): flexiblas-3.0.4-8.el9.x86_64.rpm 329 kB/s | 32 kB 00:00 2023-08-25T09:36:52.332 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86 17/177 2023-08-25T09:36:52.387 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libmount-2.37.4-14.el9.x86_64 18/177 2023-08-25T09:36:52.431 INFO:teuthology.orchestra.run.smithi070.stdout:(45/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2023-08-25T09:36:52.448 INFO:teuthology.orchestra.run.smithi070.stdout:(46/154): flexiblas-openblas-openmp-3.0.4-8.el9 109 kB/s | 17 kB 00:00 2023-08-25T09:36:52.521 INFO:teuthology.orchestra.run.smithi070.stdout:(47/154): jq-1.6-15.el9.x86_64.rpm 1.0 MB/s | 186 kB 00:00 2023-08-25T09:36:52.548 INFO:teuthology.orchestra.run.smithi070.stdout:(48/154): libaio-devel-0.3.111-13.el9.x86_64.rp 103 kB/s | 10 kB 00:00 2023-08-25T09:36:52.582 INFO:teuthology.orchestra.run.smithi070.stdout:(49/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 159 kB 00:00 2023-08-25T09:36:52.599 INFO:teuthology.orchestra.run.smithi070.stdout:(50/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 577 kB/s | 44 kB 00:00 2023-08-25T09:36:52.657 INFO:teuthology.orchestra.run.smithi070.stdout:(51/154): librdkafka-1.6.1-102.el9.x86_64.rpm 8.8 MB/s | 661 kB 00:00 2023-08-25T09:36:52.708 INFO:teuthology.orchestra.run.smithi070.stdout:(52/154): libtool-2.4.6-45.el9.x86_64.rpm 4.6 MB/s | 578 kB 00:00 2023-08-25T09:36:52.715 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libsmartcols-2.37.4-14.el9.x86_64 19/177 2023-08-25T09:36:52.725 INFO:teuthology.orchestra.run.smithi070.stdout:(53/154): libuuid-devel-2.37.4-14.el9.x86_64.rp 213 kB/s | 24 kB 00:00 2023-08-25T09:36:52.750 INFO:teuthology.orchestra.run.smithi070.stdout:(54/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 513 kB/s | 46 kB 00:00 2023-08-25T09:36:52.800 INFO:teuthology.orchestra.run.smithi070.stdout:(55/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2023-08-25T09:36:52.850 INFO:teuthology.orchestra.run.smithi070.stdout:(56/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2023-08-25T09:36:52.867 INFO:teuthology.orchestra.run.smithi070.stdout:(57/154): openblas-0.3.21-2.el9.x86_64.rpm 294 kB/s | 34 kB 00:00 2023-08-25T09:36:52.934 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 20/177 2023-08-25T09:36:53.007 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 20/177 2023-08-25T09:36:53.017 INFO:teuthology.orchestra.run.smithi070.stdout:(58/154): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2023-08-25T09:36:53.126 INFO:teuthology.orchestra.run.smithi070.stdout:(59/154): postgresql-13.11-1.el9.x86_64.rpm 6.3 MB/s | 1.5 MB 00:00 2023-08-25T09:36:53.217 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 21/177 2023-08-25T09:36:53.234 INFO:teuthology.orchestra.run.smithi070.stdout:(60/154): openssl-devel-3.0.7-24.el9.x86_64.rpm 8.5 MB/s | 3.0 MB 00:00 2023-08-25T09:36:53.288 INFO:teuthology.orchestra.run.smithi070.stdout:(61/154): postgresql-contrib-13.11-1.el9.x86_64 3.0 MB/s | 811 kB 00:00 2023-08-25T09:36:53.290 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-1:1.44.0-2.el9.x86_64 21/177 2023-08-25T09:36:53.335 INFO:teuthology.orchestra.run.smithi070.stdout:(62/154): postgresql-private-libs-13.11-1.el9.x 694 kB/s | 134 kB 00:00 2023-08-25T09:36:53.385 INFO:teuthology.orchestra.run.smithi070.stdout:(63/154): python3-jinja2-2.11.3-4.el9.noarch.rp 4.6 MB/s | 228 kB 00:00 2023-08-25T09:36:53.569 INFO:teuthology.orchestra.run.smithi070.stdout:(64/154): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 5.8 MB 00:00 2023-08-25T09:36:53.707 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 21/177 2023-08-25T09:36:53.719 INFO:teuthology.orchestra.run.smithi070.stdout:(65/154): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2023-08-25T09:36:53.744 INFO:teuthology.orchestra.run.smithi070.stdout:(66/154): python3-jsonpatch-1.21-16.el9.noarch. 69 kB/s | 24 kB 00:00 2023-08-25T09:36:53.761 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/177 2023-08-25T09:36:53.769 INFO:teuthology.orchestra.run.smithi070.stdout:(67/154): python3-jsonpointer-2.0-4.el9.noarch. 98 kB/s | 18 kB 00:00 2023-08-25T09:36:53.803 INFO:teuthology.orchestra.run.smithi070.stdout:(68/154): python3-markupsafe-1.1.1-12.el9.x86_6 996 kB/s | 32 kB 00:00 2023-08-25T09:36:53.814 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/177 2023-08-25T09:36:53.862 INFO:teuthology.orchestra.run.smithi070.stdout:(69/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 144 kB 00:00 2023-08-25T09:36:53.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 24/177 2023-08-25T09:36:53.953 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 25/177 2023-08-25T09:36:53.996 INFO:teuthology.orchestra.run.smithi070.stdout:(70/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2023-08-25T09:36:54.037 INFO:teuthology.orchestra.run.smithi070.stdout:(71/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.0 MB/s | 217 kB 00:00 2023-08-25T09:36:54.079 INFO:teuthology.orchestra.run.smithi070.stdout:(72/154): python3-oauthlib-3.1.1-5.el9.noarch.r 867 kB/s | 174 kB 00:00 2023-08-25T09:36:54.080 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/177 2023-08-25T09:36:54.121 INFO:teuthology.orchestra.run.smithi070.stdout:(73/154): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 41 kB 00:00 2023-08-25T09:36:54.150 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/177 2023-08-25T09:36:54.163 INFO:teuthology.orchestra.run.smithi070.stdout:(74/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2023-08-25T09:36:54.192 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/177 2023-08-25T09:36:54.196 INFO:teuthology.orchestra.run.smithi070.stdout:(75/154): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 211 kB 00:00 2023-08-25T09:36:54.242 INFO:teuthology.orchestra.run.smithi070.stdout:(76/154): python3-pytz-2021.1-5.el9.noarch.rpm 406 kB/s | 47 kB 00:00 2023-08-25T09:36:54.297 INFO:teuthology.orchestra.run.smithi070.stdout:(77/154): python3-requests-oauthlib-1.3.0-12.el 433 kB/s | 43 kB 00:00 2023-08-25T09:36:54.317 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/177 2023-08-25T09:36:54.371 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el9. 30/177 2023-08-25T09:36:54.439 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 31/177 2023-08-25T09:36:54.578 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el9.x86_6 32/177 2023-08-25T09:36:54.664 INFO:teuthology.orchestra.run.smithi070.stdout:(78/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2023-08-25T09:36:54.697 INFO:teuthology.orchestra.run.smithi070.stdout:(79/154): python3-toml-0.10.2-6.el9.noarch.rpm 88 kB/s | 39 kB 00:00 2023-08-25T09:36:54.731 INFO:teuthology.orchestra.run.smithi070.stdout:(80/154): socat-1.7.4.1-5.el9.x86_64.rpm 718 kB/s | 300 kB 00:00 2023-08-25T09:36:54.756 INFO:teuthology.orchestra.run.smithi070.stdout:(81/154): uuid-1.6.2-55.el9.x86_64.rpm 643 kB/s | 56 kB 00:00 2023-08-25T09:36:54.781 INFO:teuthology.orchestra.run.smithi070.stdout:(82/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2023-08-25T09:36:54.783 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/177 2023-08-25T09:36:54.840 INFO:teuthology.orchestra.run.smithi070.stdout:(83/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 795 kB/s | 85 kB 00:00 2023-08-25T09:36:54.856 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/177 2023-08-25T09:36:54.874 INFO:teuthology.orchestra.run.smithi070.stdout:(84/154): fuse-2.9.9-15.el9.x86_64.rpm 680 kB/s | 79 kB 00:00 2023-08-25T09:36:54.932 INFO:teuthology.orchestra.run.smithi070.stdout:(85/154): libgfortran-11.4.1-2.1.el9.x86_64.rpm 5.3 MB/s | 807 kB 00:00 2023-08-25T09:36:54.974 INFO:teuthology.orchestra.run.smithi070.stdout:(86/154): libquadmath-11.4.1-2.1.el9.x86_64.rpm 2.1 MB/s | 197 kB 00:00 2023-08-25T09:36:55.008 INFO:teuthology.orchestra.run.smithi070.stdout:(87/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 241 kB 00:00 2023-08-25T09:36:55.074 INFO:teuthology.orchestra.run.smithi070.stdout:(88/154): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.1 MB 00:00 2023-08-25T09:36:55.116 INFO:teuthology.orchestra.run.smithi070.stdout:(89/154): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2023-08-25T09:36:55.150 INFO:teuthology.orchestra.run.smithi070.stdout:(90/154): python3-pycparser-2.20-6.el9.noarch.r 960 kB/s | 124 kB 00:00 2023-08-25T09:36:55.359 INFO:teuthology.orchestra.run.smithi070.stdout:(91/154): zlib-devel-1.2.11-36.el9.x86_64.rpm 217 kB/s | 45 kB 00:00 2023-08-25T09:36:55.417 INFO:teuthology.orchestra.run.smithi070.stdout:(92/154): python3-devel-3.9.16-1.el9.x86_64.rpm 724 kB/s | 247 kB 00:00 2023-08-25T09:36:55.521 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/177 2023-08-25T09:36:55.636 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/177 2023-08-25T09:36:55.756 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/177 2023-08-25T09:36:55.813 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-core-2.37.4-14.el9.x86_64 38/177 2023-08-25T09:36:55.854 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: util-linux-core-2.37.4-14.el9.x86_64 38/177 2023-08-25T09:36:55.854 INFO:teuthology.orchestra.run.smithi070.stdout:(93/154): fmt-8.1.1-5.el9.x86_64.rpm 158 kB/s | 111 kB 00:00 2023-08-25T09:36:55.912 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el9.x86_6 39/177 2023-08-25T09:36:55.970 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/177 2023-08-25T09:36:55.976 INFO:teuthology.orchestra.run.smithi070.stdout:(94/154): libarrow-doc-9.0.0-9.el9.noarch.rpm 196 kB/s | 24 kB 00:00 2023-08-25T09:36:56.118 INFO:teuthology.orchestra.run.smithi070.stdout:(95/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 407 kB/s | 309 kB 00:00 2023-08-25T09:36:56.135 INFO:teuthology.orchestra.run.smithi070.stdout:(96/154): liboath-2.6.7-2.el9.x86_64.rpm 307 kB/s | 49 kB 00:00 2023-08-25T09:36:56.159 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-libs-0.189-3.el9.x86_64 41/177 2023-08-25T09:36:56.244 INFO:teuthology.orchestra.run.smithi070.stdout:(97/154): libunwind-1.6.2-1.el9.x86_64.rpm 541 kB/s | 67 kB 00:00 2023-08-25T09:36:56.249 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-6.2.0-5.el9.x86_64 42/177 2023-08-25T09:36:56.577 INFO:teuthology.orchestra.run.smithi070.stdout:(98/154): libarrow-9.0.0-9.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2023-08-25T09:36:56.628 INFO:teuthology.orchestra.run.smithi070.stdout:(99/154): python3-asyncssh-2.13.2-3.el9.noarch. 1.4 MB/s | 548 kB 00:00 2023-08-25T09:36:56.637 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : libfdisk-2.37.4-14.el9.x86_64 43/177 2023-08-25T09:36:56.686 INFO:teuthology.orchestra.run.smithi070.stdout:(100/154): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.5 MB/s | 837 kB 00:00 2023-08-25T09:36:56.703 INFO:teuthology.orchestra.run.smithi070.stdout:(101/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2023-08-25T09:36:56.720 INFO:teuthology.orchestra.run.smithi070.stdout:(102/154): python3-cachetools-4.2.4-1.el9.noarc 352 kB/s | 32 kB 00:00 2023-08-25T09:36:56.779 INFO:teuthology.orchestra.run.smithi070.stdout:(103/154): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2023-08-25T09:36:56.837 INFO:teuthology.orchestra.run.smithi070.stdout:(104/154): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2023-08-25T09:36:56.868 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-2.37.4-14.el9.x86_64 44/177 2023-08-25T09:36:56.889 INFO:teuthology.orchestra.run.smithi070.stdout:(105/154): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2023-08-25T09:36:56.929 INFO:teuthology.orchestra.run.smithi070.stdout:(106/154): python3-google-auth-2.22.0-1.el9.noa 1.3 MB/s | 197 kB 00:00 2023-08-25T09:36:56.955 INFO:teuthology.orchestra.run.smithi070.stdout:(107/154): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2023-08-25T09:36:56.968 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsprogs-5.19.0-4.el9.x86_64 45/177 2023-08-25T09:36:56.988 INFO:teuthology.orchestra.run.smithi070.stdout:(108/154): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2023-08-25T09:36:57.030 INFO:teuthology.orchestra.run.smithi070.stdout:(109/154): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2023-08-25T09:36:57.035 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libuuid-devel-2.37.4-14.el9.x86_64 46/177 2023-08-25T09:36:57.055 INFO:teuthology.orchestra.run.smithi070.stdout:(110/154): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2023-08-25T09:36:57.114 INFO:teuthology.orchestra.run.smithi070.stdout:(111/154): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2023-08-25T09:36:57.148 INFO:teuthology.orchestra.run.smithi070.stdout:(112/154): python3-jwt+crypto-2.4.0-1.el9.noarc 77 kB/s | 9.0 kB 00:00 2023-08-25T09:36:57.173 INFO:teuthology.orchestra.run.smithi070.stdout:(113/154): python3-jwt-2.4.0-1.el9.noarch.rpm 351 kB/s | 41 kB 00:00 2023-08-25T09:36:57.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/177 2023-08-25T09:36:57.265 INFO:teuthology.orchestra.run.smithi070.stdout:(114/154): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2023-08-25T09:36:57.290 INFO:teuthology.orchestra.run.smithi070.stdout:(115/154): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2023-08-25T09:36:57.310 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : openssl-1:3.0.7-24.el9.x86_64 48/177 2023-08-25T09:36:57.324 INFO:teuthology.orchestra.run.smithi070.stdout:(116/154): python3-more-itertools-8.12.0-2.el9. 525 kB/s | 79 kB 00:00 2023-08-25T09:36:57.374 INFO:teuthology.orchestra.run.smithi070.stdout:(117/154): python3-natsort-7.1.1-5.el9.noarch.r 535 kB/s | 58 kB 00:00 2023-08-25T09:36:57.424 INFO:teuthology.orchestra.run.smithi070.stdout:(118/154): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2023-08-25T09:36:57.449 INFO:teuthology.orchestra.run.smithi070.stdout:(119/154): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2023-08-25T09:36:57.472 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-2.el9.x86_64 49/177 2023-08-25T09:36:57.491 INFO:teuthology.orchestra.run.smithi070.stdout:(120/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 772 kB/s | 90 kB 00:00 2023-08-25T09:36:57.514 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 50/177 2023-08-25T09:36:57.533 INFO:teuthology.orchestra.run.smithi070.stdout:(121/154): python3-repoze-lru-0.7-16.el9.noarch 284 kB/s | 31 kB 00:00 2023-08-25T09:36:57.583 INFO:teuthology.orchestra.run.smithi070.stdout:(122/154): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2023-08-25T09:36:57.602 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/177 2023-08-25T09:36:57.612 INFO:teuthology.orchestra.run.smithi070.stdout:(123/154): python3-rsa-4.9-2.el9.noarch.rpm 507 kB/s | 59 kB 00:00 2023-08-25T09:36:57.642 INFO:teuthology.orchestra.run.smithi070.stdout:(124/154): python3-tempora-5.0.0-2.el9.noarch.r 331 kB/s | 36 kB 00:00 2023-08-25T09:36:57.708 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/177 2023-08-25T09:36:57.718 INFO:teuthology.orchestra.run.smithi070.stdout:(125/154): python3-typing-extensions-4.4.0-2.el 383 kB/s | 51 kB 00:00 2023-08-25T09:36:57.759 INFO:teuthology.orchestra.run.smithi070.stdout:(126/154): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2023-08-25T09:36:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/177 2023-08-25T09:36:57.811 INFO:teuthology.orchestra.run.smithi070.stdout:(127/154): python3-websocket-client-1.2.3-2.el9 537 kB/s | 90 kB 00:00 2023-08-25T09:36:57.839 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/177 2023-08-25T09:36:57.869 INFO:teuthology.orchestra.run.smithi070.stdout:(128/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2023-08-25T09:36:57.894 INFO:teuthology.orchestra.run.smithi070.stdout:(129/154): python3-zc-lockfile-2.0-10.el9.noarc 150 kB/s | 20 kB 00:00 2023-08-25T09:36:57.922 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/177 2023-08-25T09:36:57.944 INFO:teuthology.orchestra.run.smithi070.stdout:(130/154): re2-20211101-2.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-08-25T09:36:57.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/177 2023-08-25T09:36:58.066 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/177 2023-08-25T09:36:58.136 INFO:teuthology.orchestra.run.smithi070.stdout:(131/154): thrift-0.15.0-2.el9.x86_64.rpm 5.9 MB/s | 1.6 MB 00:00 2023-08-25T09:36:58.271 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/177 2023-08-25T09:36:58.281 INFO:teuthology.orchestra.run.smithi070.stdout:(132/154): NetworkManager-1.44.0-2.el9.x86_64.r 16 MB/s | 2.2 MB 00:00 2023-08-25T09:36:58.329 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/177 2023-08-25T09:36:58.378 INFO:teuthology.orchestra.run.smithi070.stdout:(133/154): NetworkManager-libnm-1.44.0-2.el9.x8 19 MB/s | 1.8 MB 00:00 2023-08-25T09:36:58.391 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/177 2023-08-25T09:36:58.404 INFO:teuthology.orchestra.run.smithi070.stdout:(134/154): NetworkManager-team-1.44.0-2.el9.x86 1.7 MB/s | 41 kB 00:00 2023-08-25T09:36:58.462 INFO:teuthology.orchestra.run.smithi070.stdout:(135/154): NetworkManager-tui-1.44.0-2.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-08-25T09:36:58.496 INFO:teuthology.orchestra.run.smithi070.stdout:(136/154): elfutils-debuginfod-client-0.189-3.e 1.1 MB/s | 36 kB 00:00 2023-08-25T09:36:58.531 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/177 2023-08-25T09:36:58.578 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 62/177 2023-08-25T09:36:58.588 INFO:teuthology.orchestra.run.smithi070.stdout:(137/154): librbd1-18.0.0-5703.gba6b39e4.el9.x8 4.7 MB/s | 3.0 MB 00:00 2023-08-25T09:36:58.630 INFO:teuthology.orchestra.run.smithi070.stdout:(138/154): elfutils-libelf-0.189-3.el9.x86_64.r 1.4 MB/s | 192 kB 00:00 2023-08-25T09:36:58.633 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/177 2023-08-25T09:36:58.672 INFO:teuthology.orchestra.run.smithi070.stdout:(139/154): elfutils-libs-0.189-3.el9.x86_64.rpm 3.1 MB/s | 254 kB 00:00 2023-08-25T09:36:58.677 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/177 2023-08-25T09:36:58.722 INFO:teuthology.orchestra.run.smithi070.stdout:(140/154): iproute-6.2.0-5.el9.x86_64.rpm 8.7 MB/s | 785 kB 00:00 2023-08-25T09:36:58.725 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 65/177 2023-08-25T09:36:58.772 INFO:teuthology.orchestra.run.smithi070.stdout:(141/154): iproute-tc-6.2.0-5.el9.x86_64.rpm 4.8 MB/s | 448 kB 00:00 2023-08-25T09:36:58.784 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fmt-8.1.1-5.el9.x86_64 66/177 2023-08-25T09:36:58.806 INFO:teuthology.orchestra.run.smithi070.stdout:(142/154): libblkid-2.37.4-14.el9.x86_64.rpm 1.3 MB/s | 107 kB 00:00 2023-08-25T09:36:58.865 INFO:teuthology.orchestra.run.smithi070.stdout:(143/154): libfdisk-2.37.4-14.el9.x86_64.rpm 1.7 MB/s | 154 kB 00:00 2023-08-25T09:36:58.898 INFO:teuthology.orchestra.run.smithi070.stdout:(144/154): libmount-2.37.4-14.el9.x86_64.rpm 1.5 MB/s | 133 kB 00:00 2023-08-25T09:36:58.924 INFO:teuthology.orchestra.run.smithi070.stdout:(145/154): libsmartcols-2.37.4-14.el9.x86_64.rp 1.1 MB/s | 63 kB 00:00 2023-08-25T09:36:58.949 INFO:teuthology.orchestra.run.smithi070.stdout:(146/154): libuuid-2.37.4-14.el9.x86_64.rpm 572 kB/s | 28 kB 00:00 2023-08-25T09:36:58.977 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : zlib-devel-1.2.11-36.el9.x86_64 67/177 2023-08-25T09:36:59.016 INFO:teuthology.orchestra.run.smithi070.stdout:(147/154): openssl-3.0.7-24.el9.x86_64.rpm 13 MB/s | 1.2 MB 00:00 2023-08-25T09:36:59.044 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 68/177 2023-08-25T09:36:59.108 INFO:teuthology.orchestra.run.smithi070.stdout:(148/154): openssl-libs-3.0.7-24.el9.x86_64.rpm 15 MB/s | 2.1 MB 00:00 2023-08-25T09:36:59.132 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 69/177 2023-08-25T09:36:59.238 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 70/177 2023-08-25T09:36:59.358 INFO:teuthology.orchestra.run.smithi070.stdout:(149/154): util-linux-2.37.4-14.el9.x86_64.rpm 6.7 MB/s | 2.2 MB 00:00 2023-08-25T09:36:59.430 INFO:teuthology.orchestra.run.smithi070.stdout:(150/154): util-linux-core-2.37.4-14.el9.x86_64 1.4 MB/s | 433 kB 00:00 2023-08-25T09:36:59.459 INFO:teuthology.orchestra.run.smithi070.stdout:(151/154): util-linux-user-2.37.4-14.el9.x86_64 314 kB/s | 30 kB 00:00 2023-08-25T09:36:59.516 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 71/177 2023-08-25T09:36:59.550 INFO:teuthology.orchestra.run.smithi070.stdout:(152/154): librados2-18.0.0-5703.gba6b39e4.el9. 2.0 MB/s | 3.3 MB 00:01 2023-08-25T09:36:59.588 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 72/177 2023-08-25T09:36:59.592 INFO:teuthology.orchestra.run.smithi070.stdout:(153/154): xfsdump-3.1.12-3.el9.x86_64.rpm 1.9 MB/s | 321 kB 00:00 2023-08-25T09:36:59.659 INFO:teuthology.orchestra.run.smithi070.stdout:(154/154): xfsprogs-5.19.0-4.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2023-08-25T09:36:59.677 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2023-08-25T09:36:59.678 INFO:teuthology.orchestra.run.smithi070.stdout:Total 14 MB/s | 206 MB 00:14 2023-08-25T09:36:59.774 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2023-08-25T09:36:59.783 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 73/177 2023-08-25T09:36:59.849 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 74/177 2023-08-25T09:36:59.889 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 75/177 2023-08-25T09:36:59.946 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 76/177 2023-08-25T09:37:00.007 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 77/177 2023-08-25T09:37:00.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-15.el9.x86_64 78/177 2023-08-25T09:37:00.085 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2023-08-25T09:37:00.085 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2023-08-25T09:37:00.161 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/177 2023-08-25T09:37:00.245 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/177 2023-08-25T09:37:00.320 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/177 2023-08-25T09:37:00.471 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/177 2023-08-25T09:37:00.512 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/177 2023-08-25T09:37:00.555 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/177 2023-08-25T09:37:00.619 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/177 2023-08-25T09:37:00.676 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/177 2023-08-25T09:37:00.857 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/177 2023-08-25T09:37:01.143 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/177 2023-08-25T09:37:01.342 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/177 2023-08-25T09:37:01.415 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/177 2023-08-25T09:37:02.212 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2023-08-25T09:37:02.213 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2023-08-25T09:37:02.432 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/177 2023-08-25T09:37:02.536 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/177 2023-08-25T09:37:02.587 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/177 2023-08-25T09:37:02.630 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/177 2023-08-25T09:37:02.911 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/177 2023-08-25T09:37:02.949 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/177 2023-08-25T09:37:03.063 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/177 2023-08-25T09:37:03.894 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/177 2023-08-25T09:37:04.043 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/177 2023-08-25T09:37:05.600 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/177 2023-08-25T09:37:05.737 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/177 2023-08-25T09:37:05.784 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 102/177 2023-08-25T09:37:05.823 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-15.el9.x86_64 103/177 2023-08-25T09:37:05.872 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/177 2023-08-25T09:37:06.256 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/177 2023-08-25T09:37:06.293 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 106/177 2023-08-25T09:37:06.413 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 106/177 2023-08-25T09:37:06.450 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 107/177 2023-08-25T09:37:06.523 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/177 2023-08-25T09:37:07.023 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 109/177 2023-08-25T09:37:07.063 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 110/177 2023-08-25T09:37:07.153 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 110/177 2023-08-25T09:37:07.191 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 111/177 2023-08-25T09:37:08.711 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2023-08-25T09:37:08.815 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : openssl-libs-1:3.0.7-24.el9.x86_64 1/177 2023-08-25T09:37:08.880 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libuuid-2.37.4-14.el9.x86_64 2/177 2023-08-25T09:37:08.935 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libblkid-2.37.4-14.el9.x86_64 3/177 2023-08-25T09:37:09.052 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libblkid-2.37.4-14.el9.x86_64 3/177 2023-08-25T09:37:09.134 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/177 2023-08-25T09:37:09.243 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : elfutils-libelf-0.189-3.el9.x86_64 5/177 2023-08-25T09:37:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 6/177 2023-08-25T09:37:09.464 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/177 2023-08-25T09:37:09.570 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/177 2023-08-25T09:37:09.669 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 112/177 2023-08-25T09:37:09.688 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/177 2023-08-25T09:37:09.719 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 112/177 2023-08-25T09:37:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/177 2023-08-25T09:37:10.224 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 11/177 2023-08-25T09:37:10.350 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 112/177 2023-08-25T09:37:10.382 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 11/177 2023-08-25T09:37:10.414 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 113/177 2023-08-25T09:37:10.425 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 12/177 2023-08-25T09:37:10.559 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 12/177 2023-08-25T09:37:10.597 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 113/177 2023-08-25T09:37:10.598 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-25T09:37:10.598 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-25T09:37:10.598 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:10.606 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el9.x86_64 13/177 2023-08-25T09:37:10.657 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 114/177 2023-08-25T09:37:10.867 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/177 2023-08-25T09:37:10.961 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/177 2023-08-25T09:37:10.996 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 16/177 2023-08-25T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 16/177 2023-08-25T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86 17/177 2023-08-25T09:37:11.262 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86 17/177 2023-08-25T09:37:11.326 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libmount-2.37.4-14.el9.x86_64 18/177 2023-08-25T09:37:11.684 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libsmartcols-2.37.4-14.el9.x86_64 19/177 2023-08-25T09:37:11.877 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 20/177 2023-08-25T09:37:11.965 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 20/177 2023-08-25T09:37:12.167 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 21/177 2023-08-25T09:37:12.246 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-1:1.44.0-2.el9.x86_64 21/177 2023-08-25T09:37:12.725 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 21/177 2023-08-25T09:37:12.768 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/177 2023-08-25T09:37:12.813 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/177 2023-08-25T09:37:12.889 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.4.1-2.1.el9.x86_64 24/177 2023-08-25T09:37:12.967 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.4.1-2.1.el9.x86_64 25/177 2023-08-25T09:37:13.105 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/177 2023-08-25T09:37:13.172 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/177 2023-08-25T09:37:13.215 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/177 2023-08-25T09:37:13.369 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/177 2023-08-25T09:37:13.420 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el9. 30/177 2023-08-25T09:37:13.481 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 31/177 2023-08-25T09:37:14.004 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el9.x86_6 32/177 2023-08-25T09:37:14.199 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/177 2023-08-25T09:37:14.279 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/177 2023-08-25T09:37:14.956 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/177 2023-08-25T09:37:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/177 2023-08-25T09:37:15.206 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 37/177 2023-08-25T09:37:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : util-linux-core-2.37.4-14.el9.x86_64 38/177 2023-08-25T09:37:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: util-linux-core-2.37.4-14.el9.x86_64 38/177 2023-08-25T09:37:15.379 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el9.x86_6 39/177 2023-08-25T09:37:15.438 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/177 2023-08-25T09:37:15.674 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : elfutils-libs-0.189-3.el9.x86_64 41/177 2023-08-25T09:37:15.764 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : iproute-6.2.0-5.el9.x86_64 42/177 2023-08-25T09:37:16.142 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : libfdisk-2.37.4-14.el9.x86_64 43/177 2023-08-25T09:37:16.363 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : util-linux-2.37.4-14.el9.x86_64 44/177 2023-08-25T09:37:16.472 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : xfsprogs-5.19.0-4.el9.x86_64 45/177 2023-08-25T09:37:16.539 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libuuid-devel-2.37.4-14.el9.x86_64 46/177 2023-08-25T09:37:16.755 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/177 2023-08-25T09:37:16.832 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : openssl-1:3.0.7-24.el9.x86_64 48/177 2023-08-25T09:37:16.997 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-2.el9.x86_64 49/177 2023-08-25T09:37:17.052 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 50/177 2023-08-25T09:37:17.146 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/177 2023-08-25T09:37:17.260 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/177 2023-08-25T09:37:17.311 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/177 2023-08-25T09:37:17.384 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/177 2023-08-25T09:37:17.464 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/177 2023-08-25T09:37:17.544 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/177 2023-08-25T09:37:17.620 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/177 2023-08-25T09:37:17.831 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/177 2023-08-25T09:37:17.892 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/177 2023-08-25T09:37:17.963 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/177 2023-08-25T09:37:18.113 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/177 2023-08-25T09:37:18.165 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.22.0-1.el9.noarch 62/177 2023-08-25T09:37:18.230 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/177 2023-08-25T09:37:18.282 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/177 2023-08-25T09:37:18.331 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 65/177 2023-08-25T09:37:18.383 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fmt-8.1.1-5.el9.x86_64 66/177 2023-08-25T09:37:18.566 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : zlib-devel-1.2.11-36.el9.x86_64 67/177 2023-08-25T09:37:18.631 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.16-1.el9.x86_64 68/177 2023-08-25T09:37:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 69/177 2023-08-25T09:37:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 70/177 2023-08-25T09:37:19.123 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 71/177 2023-08-25T09:37:19.199 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 72/177 2023-08-25T09:37:19.389 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 73/177 2023-08-25T09:37:19.462 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 74/177 2023-08-25T09:37:19.502 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 75/177 2023-08-25T09:37:19.552 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 76/177 2023-08-25T09:37:19.602 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 77/177 2023-08-25T09:37:19.697 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-15.el9.x86_64 78/177 2023-08-25T09:37:19.769 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 79/177 2023-08-25T09:37:19.857 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : uuid-1.6.2-55.el9.x86_64 80/177 2023-08-25T09:37:19.914 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/177 2023-08-25T09:37:20.065 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/177 2023-08-25T09:37:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/177 2023-08-25T09:37:20.168 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/177 2023-08-25T09:37:20.272 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/177 2023-08-25T09:37:20.340 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/177 2023-08-25T09:37:20.512 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/177 2023-08-25T09:37:20.569 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/177 2023-08-25T09:37:20.723 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/177 2023-08-25T09:37:20.798 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/177 2023-08-25T09:37:21.761 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/177 2023-08-25T09:37:21.823 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/177 2023-08-25T09:37:21.864 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/177 2023-08-25T09:37:21.906 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/177 2023-08-25T09:37:22.207 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.21-2.el9.x86_64 95/177 2023-08-25T09:37:22.252 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 96/177 2023-08-25T09:37:22.356 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 97/177 2023-08-25T09:37:23.176 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 98/177 2023-08-25T09:37:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 99/177 2023-08-25T09:37:23.314 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 114/177 2023-08-25T09:37:23.314 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2023-08-25T09:37:23.314 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2023-08-25T09:37:23.315 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2023-08-25T09:37:23.315 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2023-08-25T09:37:23.315 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2023-08-25T09:37:23.315 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2023-08-25T09:37:23.316 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2023-08-25T09:37:23.316 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:23.550 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 115/177 2023-08-25T09:37:23.616 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 115/177 2023-08-25T09:37:23.617 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:23.617 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-25T09:37:23.617 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:24.873 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 100/177 2023-08-25T09:37:24.964 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 101/177 2023-08-25T09:37:25.018 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 102/177 2023-08-25T09:37:25.074 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : jq-1.6-15.el9.x86_64 103/177 2023-08-25T09:37:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/177 2023-08-25T09:37:25.354 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 116/177 2023-08-25T09:37:25.467 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 116/177 2023-08-25T09:37:25.468 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:25.468 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-25T09:37:25.468 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-25T09:37:25.468 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-25T09:37:25.468 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:25.524 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/177 2023-08-25T09:37:25.571 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 106/177 2023-08-25T09:37:25.691 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 106/177 2023-08-25T09:37:25.709 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 117/177 2023-08-25T09:37:25.735 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 107/177 2023-08-25T09:37:25.793 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 117/177 2023-08-25T09:37:25.793 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:25.793 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-25T09:37:25.794 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-25T09:37:25.794 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-25T09:37:25.794 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:25.799 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/177 2023-08-25T09:37:26.293 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 109/177 2023-08-25T09:37:26.294 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 118/177 2023-08-25T09:37:26.340 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 110/177 2023-08-25T09:37:26.379 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 118/177 2023-08-25T09:37:26.380 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:26.380 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-25T09:37:26.380 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-25T09:37:26.380 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-25T09:37:26.380 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 110/177 2023-08-25T09:37:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 111/177 2023-08-25T09:37:26.512 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : automake-1.16.2-8.el9.noarch 119/177 2023-08-25T09:37:27.096 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 120/177 2023-08-25T09:37:27.165 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 120/177 2023-08-25T09:37:27.220 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 121/177 2023-08-25T09:37:27.344 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 122/177 2023-08-25T09:37:27.398 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noa 123/177 2023-08-25T09:37:27.727 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noa 123/177 2023-08-25T09:37:27.775 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.n 124/177 2023-08-25T09:37:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 112/177 2023-08-25T09:37:28.939 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.n 124/177 2023-08-25T09:37:28.963 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 112/177 2023-08-25T09:37:28.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 125/177 2023-08-25T09:37:29.119 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 125/177 2023-08-25T09:37:29.301 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 126/177 2023-08-25T09:37:29.336 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 127/177 2023-08-25T09:37:29.416 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 127/177 2023-08-25T09:37:29.416 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:29.417 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-25T09:37:29.417 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-25T09:37:29.417 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-25T09:37:29.417 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:29.470 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 128/177 2023-08-25T09:37:29.509 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 128/177 2023-08-25T09:37:29.600 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 112/177 2023-08-25T09:37:29.610 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el9.x86_64 129/177 2023-08-25T09:37:29.657 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 113/177 2023-08-25T09:37:29.829 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 113/177 2023-08-25T09:37:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-08-25T09:37:29.829 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-08-25T09:37:29.829 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:29.900 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 114/177 2023-08-25T09:37:30.522 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/177 2023-08-25T09:37:30.572 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 131/177 2023-08-25T09:37:30.659 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 131/177 2023-08-25T09:37:30.659 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:30.660 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-25T09:37:30.660 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-25T09:37:30.660 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-25T09:37:30.660 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:30.749 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 132/177 2023-08-25T09:37:30.850 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 132/177 2023-08-25T09:37:30.850 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:30.850 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-25T09:37:30.850 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:30.940 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 133/177 2023-08-25T09:37:31.050 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 133/177 2023-08-25T09:37:31.050 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:31.050 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-08-25T09:37:31.050 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-25T09:37:31.051 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-25T09:37:31.051 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:31.434 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 134/177 2023-08-25T09:37:31.522 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 134/177 2023-08-25T09:37:31.523 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:31.523 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-25T09:37:31.523 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-25T09:37:31.523 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-25T09:37:31.523 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:35.696 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el9.x86_64 135/177 2023-08-25T09:37:35.762 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 136/177 2023-08-25T09:37:35.819 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 137/177 2023-08-25T09:37:36.123 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 138/177 2023-08-25T09:37:36.253 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 139/177 2023-08-25T09:37:36.310 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 140/177 2023-08-25T09:37:36.370 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 141/177 2023-08-25T09:37:36.452 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : xfsdump-3.1.12-3.el9.x86_64 142/177 2023-08-25T09:37:36.583 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : util-linux-user-2.37.4-14.el9.x86_64 143/177 2023-08-25T09:37:36.645 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : iproute-tc-6.2.0-5.el9.x86_64 144/177 2023-08-25T09:37:36.699 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/177 2023-08-25T09:37:36.743 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el9.x86_ 146/177 2023-08-25T09:37:36.812 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/177 2023-08-25T09:37:36.868 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/177 2023-08-25T09:37:37.858 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-25T09:37:37.935 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-25T09:37:38.009 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-25T09:37:39.364 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el9.noarch 150/177 2023-08-25T09:37:39.455 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 151/177 2023-08-25T09:37:39.507 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/177 2023-08-25T09:37:39.644 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/177 2023-08-25T09:37:39.691 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/177 2023-08-25T09:37:39.739 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-tui-1:1.41.8-1.el9.x86_64 155/177 2023-08-25T09:37:39.792 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-debuginfod-client-0.188-3.el9.x86_64 156/177 2023-08-25T09:37:39.851 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libs-0.188-3.el9.x86_64 157/177 2023-08-25T09:37:39.852 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-25T09:37:40.022 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-25T09:37:40.079 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-1:3.0.7-2.el9.x86_64 159/177 2023-08-25T09:37:40.080 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-25T09:37:40.225 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-25T09:37:40.280 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsdump-3.1.10-1.el9.x86_64 161/177 2023-08-25T09:37:40.343 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : xfsprogs-5.14.2-1.el9.x86_64 162/177 2023-08-25T09:37:40.439 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-tc-6.0.0-2.el9.x86_64 163/177 2023-08-25T09:37:40.491 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : iproute-6.0.0-2.el9.x86_64 164/177 2023-08-25T09:37:40.569 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-user-2.37.4-9.el9.x86_64 165/177 2023-08-25T09:37:40.662 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-2.37.4-9.el9.x86_64 166/177 2023-08-25T09:37:40.728 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : util-linux-core-2.37.4-9.el9.x86_64 167/177 2023-08-25T09:37:40.770 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libfdisk-2.37.4-9.el9.x86_64 168/177 2023-08-25T09:37:40.813 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libmount-2.37.4-9.el9.x86_64 169/177 2023-08-25T09:37:40.854 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libblkid-2.37.4-9.el9.x86_64 170/177 2023-08-25T09:37:40.904 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-team-1:1.41.8-1.el9.x86_64 171/177 2023-08-25T09:37:40.943 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-25T09:37:40.943 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-25T09:37:41.280 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-25T09:37:41.280 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-25T09:37:41.443 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-25T09:37:41.482 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libuuid-2.37.4-9.el9.x86_64 174/177 2023-08-25T09:37:41.524 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : libsmartcols-2.37.4-9.el9.x86_64 175/177 2023-08-25T09:37:41.568 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : elfutils-libelf-0.188-3.el9.x86_64 176/177 2023-08-25T09:37:41.729 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-25T09:37:42.197 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-25T09:37:42.198 INFO:teuthology.orchestra.run.smithi155.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-25T09:37:42.198 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:42.594 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 114/177 2023-08-25T09:37:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2023-08-25T09:37:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2023-08-25T09:37:42.595 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2023-08-25T09:37:42.595 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2023-08-25T09:37:42.595 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2023-08-25T09:37:42.595 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2023-08-25T09:37:42.595 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2023-08-25T09:37:42.595 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 115/177 2023-08-25T09:37:42.886 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 115/177 2023-08-25T09:37:42.886 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:42.886 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-08-25T09:37:42.887 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:44.608 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 116/177 2023-08-25T09:37:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 116/177 2023-08-25T09:37:44.693 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:44.693 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-08-25T09:37:44.693 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-25T09:37:44.693 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-08-25T09:37:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:44.959 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 117/177 2023-08-25T09:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 117/177 2023-08-25T09:37:45.042 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-08-25T09:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-25T09:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-08-25T09:37:45.043 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:45.551 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 118/177 2023-08-25T09:37:45.657 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 118/177 2023-08-25T09:37:45.658 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:45.658 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-08-25T09:37:45.658 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-25T09:37:45.659 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-08-25T09:37:45.659 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:45.795 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : automake-1.16.2-8.el9.noarch 119/177 2023-08-25T09:37:46.431 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 120/177 2023-08-25T09:37:46.471 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 120/177 2023-08-25T09:37:46.527 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 121/177 2023-08-25T09:37:46.665 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 122/177 2023-08-25T09:37:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noa 123/177 2023-08-25T09:37:46.961 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el9.x86_64 1/177 2023-08-25T09:37:46.961 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 2/177 2023-08-25T09:37:46.961 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 3/177 2023-08-25T09:37:46.961 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 4/177 2023-08-25T09:37:46.961 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 6/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 7/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 8/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 9/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 10/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 11/177 2023-08-25T09:37:46.962 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el9.x86_64 12/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 13/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el9.x86_ 14/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 15/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 16/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el9.x86_6 17/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86 18/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 19/177 2023-08-25T09:37:46.963 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el9. 21/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el9.x86_6 22/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el9.x86_64 23/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 24/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 25/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 26/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 27/177 2023-08-25T09:37:46.964 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 28/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noa 30/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.n 31/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 34/177 2023-08-25T09:37:46.965 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 36/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 37/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el9.noarch 38/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 42/177 2023-08-25T09:37:46.966 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-15.el9.x86_64 47/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/177 2023-08-25T09:37:46.967 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/177 2023-08-25T09:37:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/177 2023-08-25T09:37:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-devel-2.37.4-14.el9.x86_64 53/177 2023-08-25T09:37:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/177 2023-08-25T09:37:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/177 2023-08-25T09:37:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/177 2023-08-25T09:37:46.968 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/177 2023-08-25T09:37:46.969 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/177 2023-08-25T09:37:46.969 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/177 2023-08-25T09:37:46.969 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/177 2023-08-25T09:37:46.969 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/177 2023-08-25T09:37:46.969 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/177 2023-08-25T09:37:46.969 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/177 2023-08-25T09:37:46.970 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/177 2023-08-25T09:37:46.970 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/177 2023-08-25T09:37:46.970 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/177 2023-08-25T09:37:46.970 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/177 2023-08-25T09:37:46.970 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/177 2023-08-25T09:37:46.970 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/177 2023-08-25T09:37:46.971 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/177 2023-08-25T09:37:46.971 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/177 2023-08-25T09:37:46.971 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/177 2023-08-25T09:37:46.971 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/177 2023-08-25T09:37:46.971 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/177 2023-08-25T09:37:46.971 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/177 2023-08-25T09:37:46.972 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 85/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 86/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/177 2023-08-25T09:37:46.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 91/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : zlib-devel-1.2.11-36.el9.x86_64 92/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 93/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 94/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 95/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 96/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 97/177 2023-08-25T09:37:46.974 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 98/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 99/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/177 2023-08-25T09:37:46.975 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 106/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 107/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 108/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 109/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 110/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 111/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 112/177 2023-08-25T09:37:46.976 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 113/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/177 2023-08-25T09:37:46.977 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/177 2023-08-25T09:37:46.978 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-2.el9.x86_64 130/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 132/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 134/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/177 2023-08-25T09:37:46.979 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 136/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-1:1.41.8-1.el9.x86_64 137/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 138/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 139/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 140/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-team-1:1.41.8-1.el9.x86_64 141/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 142/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : NetworkManager-tui-1:1.41.8-1.el9.x86_64 143/177 2023-08-25T09:37:46.980 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el9.x86_64 145/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libelf-0.188-3.el9.x86_64 147/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : elfutils-libs-0.188-3.el9.x86_64 149/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 150/177 2023-08-25T09:37:46.981 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-6.0.0-2.el9.x86_64 151/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 152/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : iproute-tc-6.0.0-2.el9.x86_64 153/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.37.4-14.el9.x86_64 154/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libblkid-2.37.4-9.el9.x86_64 155/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.37.4-14.el9.x86_64 156/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libfdisk-2.37.4-9.el9.x86_64 157/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.37.4-14.el9.x86_64 158/177 2023-08-25T09:37:46.982 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libmount-2.37.4-9.el9.x86_64 159/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.37.4-14.el9.x86_64 160/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libsmartcols-2.37.4-9.el9.x86_64 161/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.37.4-14.el9.x86_64 162/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libuuid-2.37.4-9.el9.x86_64 163/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 164/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-1:3.0.7-2.el9.x86_64 165/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 166/177 2023-08-25T09:37:46.983 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openssl-libs-1:3.0.7-2.el9.x86_64 167/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.37.4-14.el9.x86_64 168/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-2.37.4-9.el9.x86_64 169/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-core-2.37.4-14.el9.x86_64 170/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-core-2.37.4-9.el9.x86_64 171/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.37.4-14.el9.x86_64 172/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : util-linux-user-2.37.4-9.el9.x86_64 173/177 2023-08-25T09:37:46.984 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 174/177 2023-08-25T09:37:46.985 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsdump-3.1.10-1.el9.x86_64 175/177 2023-08-25T09:37:46.985 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 176/177 2023-08-25T09:37:47.022 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noa 123/177 2023-08-25T09:37:47.075 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.n 124/177 2023-08-25T09:37:48.229 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.n 124/177 2023-08-25T09:37:48.267 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 125/177 2023-08-25T09:37:48.406 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 125/177 2023-08-25T09:37:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 126/177 2023-08-25T09:37:48.627 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 127/177 2023-08-25T09:37:48.729 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 127/177 2023-08-25T09:37:48.729 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:48.729 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-08-25T09:37:48.730 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-25T09:37:48.730 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-08-25T09:37:48.730 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:48.794 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 128/177 2023-08-25T09:37:48.835 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 128/177 2023-08-25T09:37:48.932 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el9.x86_64 129/177 2023-08-25T09:37:49.366 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xfsprogs-5.14.2-1.el9.x86_64 177/177 2023-08-25T09:37:49.366 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:49.366 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2023-08-25T09:37:49.366 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-1:1.44.0-2.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-team-1:1.44.0-2.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: NetworkManager-tui-1:1.44.0-2.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-debuginfod-client-0.189-3.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-0.189-3.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libs-0.189-3.el9.x86_64 2023-08-25T09:37:49.367 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-6.2.0-5.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: iproute-tc-6.2.0-5.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: libblkid-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: libfdisk-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: libmount-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.368 INFO:teuthology.orchestra.run.smithi155.stdout: libsmartcols-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-1:3.0.7-24.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-libs-1:3.0.7-24.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-core-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: util-linux-user-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.369 INFO:teuthology.orchestra.run.smithi155.stdout: xfsdump-3.1.12-3.el9.x86_64 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-5.19.0-4.el9.x86_64 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: automake-1.16.2-8.el9.noarch 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: bison-3.7.4-5.el9.x86_64 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.370 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.371 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.372 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:37:49.373 INFO:teuthology.orchestra.run.smithi155.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: flex-2.6.4-9.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: fuse-2.9.9-15.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-25T09:37:49.374 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-15.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-25T09:37:49.375 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-25T09:37:49.376 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: libtool-2.4.6-45.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: libuuid-devel-2.37.4-14.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-25T09:37:49.377 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-13.11-1.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: postgresql-server-13.11-1.el9.x86_64 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-25T09:37:49.378 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-25T09:37:49.379 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-25T09:37:49.380 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-25T09:37:49.380 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-25T09:37:49.380 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-25T09:37:49.380 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-25T09:37:49.380 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-25T09:37:49.380 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-25T09:37:49.381 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-25T09:37:49.382 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-25T09:37:49.383 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-25T09:37:49.384 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-25T09:37:49.385 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-2.el9.x86_64 2023-08-25T09:37:49.386 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-25T09:37:49.387 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-25T09:37:49.387 INFO:teuthology.orchestra.run.smithi155.stdout: uuid-1.6.2-55.el9.x86_64 2023-08-25T09:37:49.387 INFO:teuthology.orchestra.run.smithi155.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-08-25T09:37:49.387 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-25T09:37:49.388 INFO:teuthology.orchestra.run.smithi155.stdout: zlib-devel-1.2.11-36.el9.x86_64 2023-08-25T09:37:49.388 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:37:49.388 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2023-08-25T09:37:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/177 2023-08-25T09:37:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 131/177 2023-08-25T09:37:49.937 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 131/177 2023-08-25T09:37:49.937 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:49.937 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-08-25T09:37:49.937 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-25T09:37:49.937 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-08-25T09:37:49.938 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 132/177 2023-08-25T09:37:50.044 DEBUG:teuthology.parallel:result is None 2023-08-25T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 132/177 2023-08-25T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-08-25T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:50.214 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 133/177 2023-08-25T09:37:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 133/177 2023-08-25T09:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-08-25T09:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-25T09:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-08-25T09:37:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:50.715 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 134/177 2023-08-25T09:37:50.816 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 134/177 2023-08-25T09:37:50.817 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-08-25T09:37:50.817 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-08-25T09:37:50.817 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-25T09:37:50.818 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-08-25T09:37:50.818 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:37:55.002 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el9.x86_64 135/177 2023-08-25T09:37:55.058 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 136/177 2023-08-25T09:37:55.108 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 137/177 2023-08-25T09:37:55.406 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 138/177 2023-08-25T09:37:55.542 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 139/177 2023-08-25T09:37:55.601 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 140/177 2023-08-25T09:37:55.668 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 141/177 2023-08-25T09:37:55.742 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : xfsdump-3.1.12-3.el9.x86_64 142/177 2023-08-25T09:37:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : util-linux-user-2.37.4-14.el9.x86_64 143/177 2023-08-25T09:37:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : iproute-tc-6.2.0-5.el9.x86_64 144/177 2023-08-25T09:37:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el9.x86_64 145/177 2023-08-25T09:37:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el9.x86_ 146/177 2023-08-25T09:37:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/177 2023-08-25T09:37:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/177 2023-08-25T09:37:57.096 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-25T09:37:57.146 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-25T09:37:57.217 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/177 2023-08-25T09:37:58.541 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el9.noarch 150/177 2023-08-25T09:37:58.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 151/177 2023-08-25T09:37:58.713 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/177 2023-08-25T09:37:58.882 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/177 2023-08-25T09:37:58.942 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/177 2023-08-25T09:37:58.988 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-tui-1:1.41.8-1.el9.x86_64 155/177 2023-08-25T09:37:59.043 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : elfutils-debuginfod-client-0.188-3.el9.x86_64 156/177 2023-08-25T09:37:59.095 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : elfutils-libs-0.188-3.el9.x86_64 157/177 2023-08-25T09:37:59.095 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-25T09:37:59.271 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/177 2023-08-25T09:37:59.355 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : openssl-1:3.0.7-2.el9.x86_64 159/177 2023-08-25T09:37:59.355 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-25T09:37:59.500 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/177 2023-08-25T09:37:59.604 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : xfsdump-3.1.10-1.el9.x86_64 161/177 2023-08-25T09:37:59.669 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : xfsprogs-5.14.2-1.el9.x86_64 162/177 2023-08-25T09:37:59.732 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : iproute-tc-6.0.0-2.el9.x86_64 163/177 2023-08-25T09:37:59.792 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : iproute-6.0.0-2.el9.x86_64 164/177 2023-08-25T09:37:59.868 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : util-linux-user-2.37.4-9.el9.x86_64 165/177 2023-08-25T09:37:59.953 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : util-linux-2.37.4-9.el9.x86_64 166/177 2023-08-25T09:38:00.012 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : util-linux-core-2.37.4-9.el9.x86_64 167/177 2023-08-25T09:38:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libfdisk-2.37.4-9.el9.x86_64 168/177 2023-08-25T09:38:00.121 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libmount-2.37.4-9.el9.x86_64 169/177 2023-08-25T09:38:00.163 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libblkid-2.37.4-9.el9.x86_64 170/177 2023-08-25T09:38:00.204 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-team-1:1.41.8-1.el9.x86_64 171/177 2023-08-25T09:38:00.245 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-25T09:38:00.245 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-25T09:38:00.595 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-1:1.41.8-1.el9.x86_64 172/177 2023-08-25T09:38:00.596 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-25T09:38:00.752 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: NetworkManager-libnm-1:1.41.8-1.el9.x86_64 173/177 2023-08-25T09:38:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libuuid-2.37.4-9.el9.x86_64 174/177 2023-08-25T09:38:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : libsmartcols-2.37.4-9.el9.x86_64 175/177 2023-08-25T09:38:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : elfutils-libelf-0.188-3.el9.x86_64 176/177 2023-08-25T09:38:01.023 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-25T09:38:01.484 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: openssl-libs-1:3.0.7-2.el9.x86_64 177/177 2023-08-25T09:38:01.484 INFO:teuthology.orchestra.run.smithi070.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-08-25T09:38:01.484 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:06.376 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el9.x86_64 1/177 2023-08-25T09:38:06.376 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 2/177 2023-08-25T09:38:06.376 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 3/177 2023-08-25T09:38:06.376 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 4/177 2023-08-25T09:38:06.376 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 6/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 7/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 8/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 9/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 10/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 11/177 2023-08-25T09:38:06.377 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el9.x86_64 12/177 2023-08-25T09:38:06.378 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 13/177 2023-08-25T09:38:06.378 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el9.x86_ 14/177 2023-08-25T09:38:06.378 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 15/177 2023-08-25T09:38:06.378 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 16/177 2023-08-25T09:38:06.378 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el9.x86_6 17/177 2023-08-25T09:38:06.378 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86 18/177 2023-08-25T09:38:06.379 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 19/177 2023-08-25T09:38:06.379 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/177 2023-08-25T09:38:06.379 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el9. 21/177 2023-08-25T09:38:06.379 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el9.x86_6 22/177 2023-08-25T09:38:06.380 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el9.x86_64 23/177 2023-08-25T09:38:06.380 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 24/177 2023-08-25T09:38:06.380 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 25/177 2023-08-25T09:38:06.381 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 26/177 2023-08-25T09:38:06.381 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 27/177 2023-08-25T09:38:06.381 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 28/177 2023-08-25T09:38:06.381 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/177 2023-08-25T09:38:06.382 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noa 30/177 2023-08-25T09:38:06.382 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.n 31/177 2023-08-25T09:38:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/177 2023-08-25T09:38:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/177 2023-08-25T09:38:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 34/177 2023-08-25T09:38:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/177 2023-08-25T09:38:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 36/177 2023-08-25T09:38:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 37/177 2023-08-25T09:38:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el9.noarch 38/177 2023-08-25T09:38:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/177 2023-08-25T09:38:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/177 2023-08-25T09:38:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/177 2023-08-25T09:38:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 42/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : jq-1.6-15.el9.x86_64 47/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/177 2023-08-25T09:38:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/177 2023-08-25T09:38:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/177 2023-08-25T09:38:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/177 2023-08-25T09:38:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/177 2023-08-25T09:38:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-devel-2.37.4-14.el9.x86_64 53/177 2023-08-25T09:38:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 59/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/177 2023-08-25T09:38:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/177 2023-08-25T09:38:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/177 2023-08-25T09:38:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/177 2023-08-25T09:38:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/177 2023-08-25T09:38:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 65/177 2023-08-25T09:38:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 66/177 2023-08-25T09:38:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 67/177 2023-08-25T09:38:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 68/177 2023-08-25T09:38:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 69/177 2023-08-25T09:38:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 70/177 2023-08-25T09:38:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 71/177 2023-08-25T09:38:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 72/177 2023-08-25T09:38:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/177 2023-08-25T09:38:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 74/177 2023-08-25T09:38:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 75/177 2023-08-25T09:38:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 76/177 2023-08-25T09:38:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 77/177 2023-08-25T09:38:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 78/177 2023-08-25T09:38:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 79/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 81/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 82/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.4.1-2.1.el9.x86_64 85/177 2023-08-25T09:38:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.4.1-2.1.el9.x86_64 86/177 2023-08-25T09:38:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/177 2023-08-25T09:38:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/177 2023-08-25T09:38:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/177 2023-08-25T09:38:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/177 2023-08-25T09:38:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.16-1.el9.x86_64 91/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : zlib-devel-1.2.11-36.el9.x86_64 92/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 93/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 94/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 95/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 96/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 97/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 98/177 2023-08-25T09:38:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 99/177 2023-08-25T09:38:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/177 2023-08-25T09:38:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/177 2023-08-25T09:38:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/177 2023-08-25T09:38:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/177 2023-08-25T09:38:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/177 2023-08-25T09:38:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.22.0-1.el9.noarch 106/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 107/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 108/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 109/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 110/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 111/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 112/177 2023-08-25T09:38:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 113/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/177 2023-08-25T09:38:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/177 2023-08-25T09:38:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/177 2023-08-25T09:38:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/177 2023-08-25T09:38:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/177 2023-08-25T09:38:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-2.el9.x86_64 130/177 2023-08-25T09:38:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/177 2023-08-25T09:38:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 132/177 2023-08-25T09:38:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 134/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 136/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-1:1.41.8-1.el9.x86_64 137/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 138/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-libnm-1:1.41.8-1.el9.x86_64 139/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 140/177 2023-08-25T09:38:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-team-1:1.41.8-1.el9.x86_64 141/177 2023-08-25T09:38:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 142/177 2023-08-25T09:38:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : NetworkManager-tui-1:1.41.8-1.el9.x86_64 143/177 2023-08-25T09:38:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 144/177 2023-08-25T09:38:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el9.x86_64 145/177 2023-08-25T09:38:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 146/177 2023-08-25T09:38:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libelf-0.188-3.el9.x86_64 147/177 2023-08-25T09:38:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 148/177 2023-08-25T09:38:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : elfutils-libs-0.188-3.el9.x86_64 149/177 2023-08-25T09:38:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 150/177 2023-08-25T09:38:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-6.0.0-2.el9.x86_64 151/177 2023-08-25T09:38:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 152/177 2023-08-25T09:38:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : iproute-tc-6.0.0-2.el9.x86_64 153/177 2023-08-25T09:38:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libblkid-2.37.4-14.el9.x86_64 154/177 2023-08-25T09:38:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libblkid-2.37.4-9.el9.x86_64 155/177 2023-08-25T09:38:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libfdisk-2.37.4-14.el9.x86_64 156/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libfdisk-2.37.4-9.el9.x86_64 157/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libmount-2.37.4-14.el9.x86_64 158/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libmount-2.37.4-9.el9.x86_64 159/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libsmartcols-2.37.4-14.el9.x86_64 160/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libsmartcols-2.37.4-9.el9.x86_64 161/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-2.37.4-14.el9.x86_64 162/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libuuid-2.37.4-9.el9.x86_64 163/177 2023-08-25T09:38:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 164/177 2023-08-25T09:38:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-1:3.0.7-2.el9.x86_64 165/177 2023-08-25T09:38:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 166/177 2023-08-25T09:38:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openssl-libs-1:3.0.7-2.el9.x86_64 167/177 2023-08-25T09:38:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : util-linux-2.37.4-14.el9.x86_64 168/177 2023-08-25T09:38:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : util-linux-2.37.4-9.el9.x86_64 169/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : util-linux-core-2.37.4-14.el9.x86_64 170/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : util-linux-core-2.37.4-9.el9.x86_64 171/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : util-linux-user-2.37.4-14.el9.x86_64 172/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : util-linux-user-2.37.4-9.el9.x86_64 173/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 174/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsdump-3.1.10-1.el9.x86_64 175/177 2023-08-25T09:38:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 176/177 2023-08-25T09:38:08.772 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xfsprogs-5.14.2-1.el9.x86_64 177/177 2023-08-25T09:38:08.773 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:08.773 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2023-08-25T09:38:08.773 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-1:1.44.0-2.el9.x86_64 2023-08-25T09:38:08.773 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 2023-08-25T09:38:08.773 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-team-1:1.44.0-2.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: NetworkManager-tui-1:1.44.0-2.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-debuginfod-client-0.189-3.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-0.189-3.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libs-0.189-3.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: iproute-6.2.0-5.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: iproute-tc-6.2.0-5.el9.x86_64 2023-08-25T09:38:08.774 INFO:teuthology.orchestra.run.smithi070.stdout: libblkid-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.775 INFO:teuthology.orchestra.run.smithi070.stdout: libfdisk-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.775 INFO:teuthology.orchestra.run.smithi070.stdout: libmount-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.775 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.775 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.775 INFO:teuthology.orchestra.run.smithi070.stdout: libsmartcols-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.776 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.776 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-1:3.0.7-24.el9.x86_64 2023-08-25T09:38:08.776 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-libs-1:3.0.7-24.el9.x86_64 2023-08-25T09:38:08.776 INFO:teuthology.orchestra.run.smithi070.stdout: util-linux-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.777 INFO:teuthology.orchestra.run.smithi070.stdout: util-linux-core-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.777 INFO:teuthology.orchestra.run.smithi070.stdout: util-linux-user-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.777 INFO:teuthology.orchestra.run.smithi070.stdout: xfsdump-3.1.12-3.el9.x86_64 2023-08-25T09:38:08.777 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-5.19.0-4.el9.x86_64 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout: automake-1.16.2-8.el9.noarch 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout: bison-3.7.4-5.el9.x86_64 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.778 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.779 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.780 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.780 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.780 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.780 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.780 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.781 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.782 INFO:teuthology.orchestra.run.smithi070.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el9.noarch 2023-08-25T09:38:08.782 INFO:teuthology.orchestra.run.smithi070.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-08-25T09:38:08.782 INFO:teuthology.orchestra.run.smithi070.stdout: flex-2.6.4-9.el9.x86_64 2023-08-25T09:38:08.782 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-08-25T09:38:08.782 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-08-25T09:38:08.782 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-08-25T09:38:08.783 INFO:teuthology.orchestra.run.smithi070.stdout: fmt-8.1.1-5.el9.x86_64 2023-08-25T09:38:08.783 INFO:teuthology.orchestra.run.smithi070.stdout: fuse-2.9.9-15.el9.x86_64 2023-08-25T09:38:08.783 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-08-25T09:38:08.783 INFO:teuthology.orchestra.run.smithi070.stdout: jq-1.6-15.el9.x86_64 2023-08-25T09:38:08.784 INFO:teuthology.orchestra.run.smithi070.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-08-25T09:38:08.784 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-9.el9.x86_64 2023-08-25T09:38:08.784 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-08-25T09:38:08.784 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.784 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.785 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.785 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.4.1-2.1.el9.x86_64 2023-08-25T09:38:08.785 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2023-08-25T09:38:08.785 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-08-25T09:38:08.785 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.4.1-2.1.el9.x86_64 2023-08-25T09:38:08.785 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-08-25T09:38:08.786 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.786 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.786 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-08-25T09:38:08.786 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.786 INFO:teuthology.orchestra.run.smithi070.stdout: libtool-2.4.6-45.el9.x86_64 2023-08-25T09:38:08.787 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2023-08-25T09:38:08.787 INFO:teuthology.orchestra.run.smithi070.stdout: libuuid-devel-2.37.4-14.el9.x86_64 2023-08-25T09:38:08.787 INFO:teuthology.orchestra.run.smithi070.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-08-25T09:38:08.787 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-08-25T09:38:08.787 INFO:teuthology.orchestra.run.smithi070.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-08-25T09:38:08.787 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.21-2.el9.x86_64 2023-08-25T09:38:08.788 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-08-25T09:38:08.788 INFO:teuthology.orchestra.run.smithi070.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-08-25T09:38:08.788 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-08-25T09:38:08.788 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-13.11-1.el9.x86_64 2023-08-25T09:38:08.788 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-08-25T09:38:08.788 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: postgresql-server-13.11-1.el9.x86_64 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.789 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-08-25T09:38:08.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-08-25T09:38:08.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-08-25T09:38:08.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-08-25T09:38:08.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-08-25T09:38:08.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.16-1.el9.x86_64 2023-08-25T09:38:08.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.22.0-1.el9.noarch 2023-08-25T09:38:08.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-08-25T09:38:08.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-08-25T09:38:08.792 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-08-25T09:38:08.792 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-08-25T09:38:08.792 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-08-25T09:38:08.792 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-08-25T09:38:08.792 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-08-25T09:38:08.793 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-08-25T09:38:08.793 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-08-25T09:38:08.793 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-08-25T09:38:08.793 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-08-25T09:38:08.793 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-08-25T09:38:08.794 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-08-25T09:38:08.795 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-08-25T09:38:08.795 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2023-08-25T09:38:08.795 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2023-08-25T09:38:08.795 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-08-25T09:38:08.795 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-08-25T09:38:08.795 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-08-25T09:38:08.796 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-08-25T09:38:08.797 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.797 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2023-08-25T09:38:08.797 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2023-08-25T09:38:08.797 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-08-25T09:38:08.797 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-08-25T09:38:08.797 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.7-6.el9.noarch 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.798 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el9.x86_64 2023-08-25T09:38:08.799 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-2.el9.x86_64 2023-08-25T09:38:08.799 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-5.el9.x86_64 2023-08-25T09:38:08.799 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2023-08-25T09:38:08.799 INFO:teuthology.orchestra.run.smithi070.stdout: uuid-1.6.2-55.el9.x86_64 2023-08-25T09:38:08.800 INFO:teuthology.orchestra.run.smithi070.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-08-25T09:38:08.800 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-08-25T09:38:08.800 INFO:teuthology.orchestra.run.smithi070.stdout: zlib-devel-1.2.11-36.el9.x86_64 2023-08-25T09:38:08.801 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:08.801 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2023-08-25T09:38:09.521 DEBUG:teuthology.parallel:result is None 2023-08-25T09:38:09.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T09:38:09.746 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T09:38:09.805 INFO:teuthology.orchestra.run.smithi070.stdout:18.0.0-5703.gba6b39e4.el9 2023-08-25T09:38:09.806 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el9 2023-08-25T09:38:09.807 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T09:38:09.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T09:38:09.951 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T09:38:10.021 INFO:teuthology.orchestra.run.smithi155.stdout:18.0.0-5703.gba6b39e4.el9 2023-08-25T09:38:10.022 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el9 2023-08-25T09:38:10.023 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T09:38:10.024 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-25T09:38:10.025 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:10.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T09:38:10.107 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:10.107 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T09:38:10.212 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-25T09:38:10.213 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:10.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T09:38:10.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T09:38:10.406 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:10.406 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T09:38:10.510 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T09:38:10.607 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-25T09:38:10.608 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:10.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T09:38:10.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T09:38:10.804 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:10.805 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T09:38:10.910 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T09:38:11.050 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-25T09:38:11.050 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:11.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T09:38:11.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T09:38:11.247 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:11.247 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T09:38:11.351 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T09:38:11.449 INFO:teuthology.run_tasks:Running task ceph... 2023-08-25T09:38:11.522 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-25T09:38:11.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /var/log/ceph 2023-08-25T09:38:11.529 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /var/log/ceph 2023-08-25T09:38:11.630 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-25T09:38:11.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-25T09:38:11.636 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-25T09:38:11.727 INFO:tasks.ceph:Creating extra log directories... 2023-08-25T09:38:11.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-25T09:38:11.733 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-25T09:38:11.840 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-08-25T09:38:11.841 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-08-25T09:38:11.841 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-08-25T09:38:11.841 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:11.842 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-25T09:38:11.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-25T09:38:12.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-25T09:38:12.183 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:12.183 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-25T09:38:12.243 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-25T09:38:12.519 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-25T09:38:12.630 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-25T09:38:12.630 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-25T09:38:12.630 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315', 'branch': 'wip-rishabh-2023aug3-b5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': None, 'first_in_suite': False, 'job_id': '7379315', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023aug3-b5'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'test-nfs-pr-52556', 'sha1': '4b369cf18ed1391a426ab4ae86da834e9c074f81'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'test-nfs-pr-52556', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_4b369cf18ed1391a426ab4ae86da834e9c074f81/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '4b369cf18ed1391a426ab4ae86da834e9c074f81', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi070.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL3R/U3OG/FM9IFxeKz3TYfxUwwqbXNkJq2WN7SE1FnMDOfR60dexyJU4bMzBueqLKnZTZ/jAGKwpHHxo5L981I=', 'smithi155.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6by/2gfhYuyGvhCvl1oZimmuV0/1tQO9dMYvoeQ22S/c4hi+VkSUZiLArWbdLYvzk4bQdC3sIdT45aZwfMh+U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '449a1bc2027504e7b3c3d7b30fa4178906581da7', 'timestamp': '2023-08-25_06:38:25', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2224'} 2023-08-25T09:38:12.631 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-25T09:38:12.637 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-25T09:38:12.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m0777 -- /var/run/ceph 2023-08-25T09:38:12.723 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m0777 -- /var/run/ceph 2023-08-25T09:38:12.822 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:12.823 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T09:38:12.864 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T09:38:12.864 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2023-08-25T09:38:12.946 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T09:38:12.947 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:12.947 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 622 Links: 1 2023-08-25T09:38:12.947 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:12.947 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:12.947 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-08-25 09:38:01.972400181 +0000 2023-08-25T09:38:12.947 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-08-25 09:35:11.089954623 +0000 2023-08-25T09:38:12.948 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-08-25 09:35:11.089954623 +0000 2023-08-25T09:38:12.948 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-08-25 09:35:11.089954623 +0000 2023-08-25T09:38:12.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T09:38:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-08-25T09:38:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-08-25T09:38:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000305787 s, 1.7 MB/s 2023-08-25T09:38:13.066 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T09:38:13.109 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2023-08-25T09:38:13.193 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T09:38:13.194 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:13.194 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 641 Links: 1 2023-08-25T09:38:13.194 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:13.196 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:13.196 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-08-25 09:38:01.974400123 +0000 2023-08-25T09:38:13.197 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-08-25 09:35:12.100925274 +0000 2023-08-25T09:38:13.197 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-08-25 09:35:12.100925274 +0000 2023-08-25T09:38:13.197 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-08-25 09:35:12.100925274 +0000 2023-08-25T09:38:13.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T09:38:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-08-25T09:38:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-08-25T09:38:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000208303 s, 2.5 MB/s 2023-08-25T09:38:13.307 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T09:38:13.351 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2023-08-25T09:38:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T09:38:13.433 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:13.433 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 660 Links: 1 2023-08-25T09:38:13.433 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:13.433 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:13.433 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-08-25 09:38:01.974400123 +0000 2023-08-25T09:38:13.434 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-08-25 09:35:13.107896040 +0000 2023-08-25T09:38:13.434 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-08-25 09:35:13.107896040 +0000 2023-08-25T09:38:13.434 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-08-25 09:35:13.107896040 +0000 2023-08-25T09:38:13.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T09:38:13.543 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-08-25T09:38:13.544 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-08-25T09:38:13.544 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000206858 s, 2.5 MB/s 2023-08-25T09:38:13.546 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T09:38:13.587 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2023-08-25T09:38:13.670 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 679 Links: 1 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2023-08-25 09:38:01.975400094 +0000 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2023-08-25 09:35:14.135866197 +0000 2023-08-25T09:38:13.671 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2023-08-25 09:35:14.135866197 +0000 2023-08-25T09:38:13.672 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2023-08-25 09:35:14.135866197 +0000 2023-08-25T09:38:13.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T09:38:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-08-25T09:38:13.782 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2023-08-25T09:38:13.782 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000186931 s, 2.7 MB/s 2023-08-25T09:38:13.784 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T09:38:13.826 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:13.826 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:13.827 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T09:38:13.887 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T09:38:13.887 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2023-08-25T09:38:13.943 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T09:38:13.943 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:13.943 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 625 Links: 1 2023-08-25T09:38:13.943 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:13.944 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:13.944 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-08-25 09:37:42.642405558 +0000 2023-08-25T09:38:13.944 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-08-25 09:35:11.127019445 +0000 2023-08-25T09:38:13.944 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-08-25 09:35:11.127019445 +0000 2023-08-25T09:38:13.944 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2023-08-25 09:35:11.127019445 +0000 2023-08-25T09:38:13.945 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T09:38:14.039 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-08-25T09:38:14.039 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-08-25T09:38:14.040 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000342263 s, 1.5 MB/s 2023-08-25T09:38:14.044 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T09:38:14.104 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2023-08-25T09:38:14.158 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T09:38:14.158 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:14.158 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 642 Links: 1 2023-08-25T09:38:14.158 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:14.159 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:14.159 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-08-25 09:37:42.643405538 +0000 2023-08-25T09:38:14.159 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-08-25 09:35:12.144999189 +0000 2023-08-25T09:38:14.159 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-08-25 09:35:12.144999189 +0000 2023-08-25T09:38:14.159 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2023-08-25 09:35:12.144999189 +0000 2023-08-25T09:38:14.159 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T09:38:14.248 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-08-25T09:38:14.249 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-08-25T09:38:14.249 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000352766 s, 1.5 MB/s 2023-08-25T09:38:14.252 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T09:38:14.308 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2023-08-25T09:38:14.362 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T09:38:14.362 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:14.363 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 659 Links: 1 2023-08-25T09:38:14.363 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:14.363 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:14.363 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-08-25 09:37:42.644405517 +0000 2023-08-25T09:38:14.364 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-08-25 09:35:13.190978377 +0000 2023-08-25T09:38:14.364 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-08-25 09:35:13.190978377 +0000 2023-08-25T09:38:14.364 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2023-08-25 09:35:13.190978377 +0000 2023-08-25T09:38:14.365 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T09:38:14.455 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-08-25T09:38:14.455 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-08-25T09:38:14.455 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000318354 s, 1.6 MB/s 2023-08-25T09:38:14.458 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T09:38:14.518 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2023-08-25T09:38:14.570 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T09:38:14.571 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T09:38:14.571 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 677 Links: 1 2023-08-25T09:38:14.571 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T09:38:14.571 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T09:38:14.571 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2023-08-25 09:37:42.645405497 +0000 2023-08-25T09:38:14.571 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2023-08-25 09:35:14.202958240 +0000 2023-08-25T09:38:14.572 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2023-08-25 09:35:14.202958240 +0000 2023-08-25T09:38:14.572 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2023-08-25 09:35:14.202958240 +0000 2023-08-25T09:38:14.572 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T09:38:14.658 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2023-08-25T09:38:14.659 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2023-08-25T09:38:14.660 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000332945 s, 1.5 MB/s 2023-08-25T09:38:14.662 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T09:38:14.715 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:14.715 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-25T09:38:14.716 INFO:tasks.ceph:Generating config... 2023-08-25T09:38:14.718 INFO:tasks.ceph:[client] client mount timeout = 600 2023-08-25T09:38:14.718 INFO:tasks.ceph:[client] debug client = 20 2023-08-25T09:38:14.718 INFO:tasks.ceph:[client] debug ms = 1 2023-08-25T09:38:14.719 INFO:tasks.ceph:[client] pid file = 2023-08-25T09:38:14.719 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-08-25T09:38:14.720 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-08-25T09:38:14.720 INFO:tasks.ceph:[client.0] debug client = 20 2023-08-25T09:38:14.720 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-08-25T09:38:14.720 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-08-25T09:38:14.721 INFO:tasks.ceph:[client.1] debug client = 20 2023-08-25T09:38:14.721 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-08-25T09:38:14.721 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-08-25T09:38:14.722 INFO:tasks.ceph:[mds] debug mds = 20 2023-08-25T09:38:14.722 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-08-25T09:38:14.722 INFO:tasks.ceph:[mds] debug ms = 1 2023-08-25T09:38:14.723 INFO:tasks.ceph:[mds] mds debug frag = True 2023-08-25T09:38:14.723 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-08-25T09:38:14.723 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-08-25T09:38:14.723 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-08-25T09:38:14.724 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-08-25T09:38:14.724 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-08-25T09:38:14.725 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-08-25T09:38:14.725 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-25T09:38:14.725 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-25T09:38:14.725 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-25T09:38:14.725 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-25T09:38:14.726 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-25T09:38:14.726 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-08-25T09:38:14.726 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-08-25T09:38:14.726 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-08-25T09:38:14.726 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-08-25T09:38:14.727 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-08-25T09:38:14.727 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-08-25T09:38:14.727 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-08-25T09:38:14.728 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-08-25T09:38:14.728 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-25T09:38:14.729 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-25T09:38:14.729 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-08-25T09:38:14.730 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-08-25T09:38:14.730 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-08-25T09:38:14.731 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-08-25T09:38:14.731 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-08-25T09:38:14.731 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-08-25T09:38:14.732 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-08-25T09:38:14.732 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-08-25T09:38:14.733 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-08-25T09:38:14.734 INFO:tasks.ceph:Setting up mon.a... 2023-08-25T09:38:14.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-25T09:38:14.894 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.keyring 2023-08-25T09:38:14.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-25T09:38:15.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-25T09:38:15.129 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.70'), ('mon.b', '172.21.15.155'), ('mon.c', '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]')] 2023-08-25T09:38:15.129 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.70,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-08-25T09:38:15.131 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:15.131 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-25T09:38:15.185 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.70 --add b 172.21.15.155 --addv c '[v2:172.21.15.155:3301,v1:172.21.15.155:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:15.290 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:15.290 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: generated fsid 64dd9a7e-0cdf-46fb-8c89-fab611563c75 2023-08-25T09:38:15.290 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = pacific 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 0 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:fsid 64dd9a7e-0cdf-46fb-8c89-fab611563c75 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:last_changed 2023-08-25T09:38:15.289014+0000 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:created 2023-08-25T09:38:15.289014+0000 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:min_mon_release 16 (pacific) 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:election_strategy: 1 2023-08-25T09:38:15.291 INFO:teuthology.orchestra.run.smithi070.stdout:0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2023-08-25T09:38:15.292 INFO:teuthology.orchestra.run.smithi070.stdout:1: [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] mon.b 2023-08-25T09:38:15.292 INFO:teuthology.orchestra.run.smithi070.stdout:2: [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] mon.c 2023-08-25T09:38:15.292 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-08-25T09:38:15.294 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-25T09:38:15.347 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 64dd9a7e-0cdf-46fb-8c89-fab611563c75... 2023-08-25T09:38:15.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-25T09:38:15.354 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-25T09:38:15.519 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2023-08-25T09:38:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: chdir = "" 2023-08-25T09:38:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-25T09:38:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: auth supported = cephx 2023-08-25T09:38:15.520 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: filestore xattr use omap = true 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: mon clock drift allowed = 1.000 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: osd crush chooseleaf type = 0 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: auth debug = true 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.521 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on old message = true 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: ms die on bug = true 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: mon pg warn max object skew = 0 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: # disable pg_autoscaler by default for new pools 2023-08-25T09:38:15.522 INFO:teuthology.orchestra.run.smithi070.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default size = 2 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow primary affinity = true 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd allow pg remap = true 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on legacy crush tunables = false 2023-08-25T09:38:15.523 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on crush straw calc version zero = false 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on no sortbitwise = false 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on osd down out interval zero = false 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: mon warn on too few osds = false 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: mon_allow_pool_size_one = true 2023-08-25T09:38:15.524 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: osd default data pool replay window = 5 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: mon allow pool delete = true 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.525 INFO:teuthology.orchestra.run.smithi070.stdout: mon cluster log file level = debug 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: debug asserts on shutdown = true 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: mon health detail to clog = false 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: mon host = "172.21.15.70,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = 64dd9a7e-0cdf-46fb-8c89-fab611563c75 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout:[osd] 2023-08-25T09:38:15.526 INFO:teuthology.orchestra.run.smithi070.stdout: osd journal size = 100 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub load threshold = 5.0 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: osd scrub max interval = 600 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: osd mclock profile = high_recovery_ops 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: osd recover clone overlap = true 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: osd recovery max chunk = 1048576 2023-08-25T09:38:15.527 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug shutdown = true 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug op order = true 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug verify stray on activate = true 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: osd open classes on start = true 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug pg log writeout = true 2023-08-25T09:38:15.528 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: osd deep scrub update digest min age = 30 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: osd map max advance = 10 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: journal zero on create = true 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.529 INFO:teuthology.orchestra.run.smithi070.stdout: filestore ondisk finisher threads = 3 2023-08-25T09:38:15.530 INFO:teuthology.orchestra.run.smithi070.stdout: filestore apply finisher threads = 3 2023-08-25T09:38:15.530 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: bdev debug aio = true 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: osd debug misdirected ops = true 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore block size = 96636764160 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: bluestore fsck on mount = True 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluefs = 20 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: debug bluestore = 20 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-08-25T09:38:15.531 INFO:teuthology.orchestra.run.smithi070.stdout: debug osd = 20 2023-08-25T09:38:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: debug rocksdb = 10 2023-08-25T09:38:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: memstore device bytes = 200000000 2023-08-25T09:38:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull_ratio = 0.85 2023-08-25T09:38:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.9 2023-08-25T09:38:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.8 2023-08-25T09:38:15.532 INFO:teuthology.orchestra.run.smithi070.stdout: osd failsafe full ratio = 1.0 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: osd mon report interval = 5 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: osd objectstore = memstore 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr] 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: debug mgr = 20 2023-08-25T09:38:15.533 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: mgr/telemetry/nag = false 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout:[mon] 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-08-25T09:38:15.534 INFO:teuthology.orchestra.run.smithi070.stdout: debug mon = 20 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: debug paxos = 20 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: debug auth = 20 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: mon data avail warn = 5 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: mon mgr mkfs grace = 240 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min pgs per osd = 4 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd reporter subtree level = osd 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd prime pg temp = true 2023-08-25T09:38:15.535 INFO:teuthology.orchestra.run.smithi070.stdout: mon reweight min bytes per osd = 10 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: auth mon ticket ttl = 660 # 11m 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: auth service ticket ttl = 240 # 4m 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: # don't complain about insecure global_id in the test suite 2023-08-25T09:38:15.536 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: # 1m isn't quite enough 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: mon_down_mkfs_grace = 2m 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: mon_warn_on_filestore_osds = false 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: mon op complaint time = 120 2023-08-25T09:38:15.537 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd backfillfull ratio = 0.6 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd full ratio = 0.7 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: mon osd nearfull ratio = 0.6 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout:[client] 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: rgw cache enabled = true 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable ops log = true 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: rgw enable usage log = true 2023-08-25T09:38:15.538 INFO:teuthology.orchestra.run.smithi070.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: client mount timeout = 600 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: pid file = "" 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2023-08-25T09:38:15.539 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.a] 2023-08-25T09:38:15.540 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.b] 2023-08-25T09:38:15.540 INFO:teuthology.orchestra.run.smithi070.stdout:[mon.c] 2023-08-25T09:38:15.540 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2023-08-25T09:38:15.540 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: debug objectcacher = 20 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout: debug objecter = 20 2023-08-25T09:38:15.541 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2023-08-25T09:38:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: debug client = 20 2023-08-25T09:38:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: debug objectcacher = 20 2023-08-25T09:38:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: debug objecter = 20 2023-08-25T09:38:15.542 INFO:teuthology.orchestra.run.smithi070.stdout:[mds] 2023-08-25T09:38:15.542 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds = 20 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: debug mds balancer = 20 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: debug ms = 1 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug frag = True 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: mds debug scatterstat = True 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: mds op complaint time = 180 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: mds verify scatter = True 2023-08-25T09:38:15.543 INFO:teuthology.orchestra.run.smithi070.stdout: osd op complaint time = 180 2023-08-25T09:38:15.544 INFO:teuthology.orchestra.run.smithi070.stdout: rados mon op timeout = 900 2023-08-25T09:38:15.544 INFO:teuthology.orchestra.run.smithi070.stdout: rados osd op timeout = 900 2023-08-25T09:38:15.545 INFO:teuthology.orchestra.run.smithi155.stdout:[global] 2023-08-25T09:38:15.545 INFO:teuthology.orchestra.run.smithi155.stdout: chdir = "" 2023-08-25T09:38:15.545 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-25T09:38:15.545 INFO:teuthology.orchestra.run.smithi155.stdout: auth supported = cephx 2023-08-25T09:38:15.545 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: filestore xattr use omap = true 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: mon clock drift allowed = 1.000 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: osd crush chooseleaf type = 0 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: auth debug = true 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.546 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on old message = true 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: ms die on bug = true 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: mon pg warn max object skew = 0 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: # disable pg_autoscaler by default for new pools 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-25T09:38:15.547 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default size = 2 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow primary affinity = true 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd allow pg remap = true 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on legacy crush tunables = false 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on crush straw calc version zero = false 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on no sortbitwise = false 2023-08-25T09:38:15.548 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on osd down out interval zero = false 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: mon warn on too few osds = false 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: mon_allow_pool_size_one = true 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.549 INFO:teuthology.orchestra.run.smithi155.stdout: osd default data pool replay window = 5 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: mon allow pool delete = true 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: mon cluster log file level = debug 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: debug asserts on shutdown = true 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: mon health detail to clog = false 2023-08-25T09:38:15.550 INFO:teuthology.orchestra.run.smithi155.stdout: mon host = "172.21.15.70,172.21.15.155,[v2:172.21.15.155:3301,v1:172.21.15.155:6790]" 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: fsid = 64dd9a7e-0cdf-46fb-8c89-fab611563c75 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout:[osd] 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: osd journal size = 100 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub load threshold = 5.0 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: osd scrub max interval = 600 2023-08-25T09:38:15.551 INFO:teuthology.orchestra.run.smithi155.stdout: osd mclock profile = high_recovery_ops 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: osd recover clone overlap = true 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: osd recovery max chunk = 1048576 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug shutdown = true 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug op order = true 2023-08-25T09:38:15.552 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug verify stray on activate = true 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: osd open classes on start = true 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug pg log writeout = true 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: osd deep scrub update digest min age = 30 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: osd map max advance = 10 2023-08-25T09:38:15.553 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: journal zero on create = true 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: filestore ondisk finisher threads = 3 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: filestore apply finisher threads = 3 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: bdev debug aio = true 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: osd debug misdirected ops = true 2023-08-25T09:38:15.554 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore block size = 96636764160 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: bluestore fsck on mount = True 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluefs = 20 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: debug bluestore = 20 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: debug osd = 20 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: debug rocksdb = 10 2023-08-25T09:38:15.555 INFO:teuthology.orchestra.run.smithi155.stdout: memstore device bytes = 200000000 2023-08-25T09:38:15.556 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull_ratio = 0.85 2023-08-25T09:38:15.556 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.9 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.8 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout: osd failsafe full ratio = 1.0 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout: osd mon report interval = 5 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout: osd objectstore = memstore 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.557 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr] 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: debug mgr = 20 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: mgr/telemetry/nag = false 2023-08-25T09:38:15.558 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout:[mon] 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: debug mon = 20 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: debug paxos = 20 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: debug auth = 20 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: mon data avail warn = 5 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: mon mgr mkfs grace = 240 2023-08-25T09:38:15.559 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min pgs per osd = 4 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd reporter subtree level = osd 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd prime pg temp = true 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: mon reweight min bytes per osd = 10 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: auth mon ticket ttl = 660 # 11m 2023-08-25T09:38:15.560 INFO:teuthology.orchestra.run.smithi155.stdout: auth service ticket ttl = 240 # 4m 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: # don't complain about insecure global_id in the test suite 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: # 1m isn't quite enough 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: mon_down_mkfs_grace = 2m 2023-08-25T09:38:15.561 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout: mon_warn_on_filestore_osds = false 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout: mon op complaint time = 120 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd backfillfull ratio = 0.6 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd full ratio = 0.7 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout: mon osd nearfull ratio = 0.6 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout: 2023-08-25T09:38:15.562 INFO:teuthology.orchestra.run.smithi155.stdout:[client] 2023-08-25T09:38:15.563 INFO:teuthology.orchestra.run.smithi155.stdout: rgw cache enabled = true 2023-08-25T09:38:15.563 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable ops log = true 2023-08-25T09:38:15.563 INFO:teuthology.orchestra.run.smithi155.stdout: rgw enable usage log = true 2023-08-25T09:38:15.563 INFO:teuthology.orchestra.run.smithi155.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-25T09:38:15.563 INFO:teuthology.orchestra.run.smithi155.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout: client mount timeout = 600 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout: pid file = "" 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2023-08-25T09:38:15.564 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.a] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.b] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[mon.c] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2023-08-25T09:38:15.565 INFO:teuthology.orchestra.run.smithi155.stdout: debug objectcacher = 20 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: debug objecter = 20 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: debug client = 20 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: debug objectcacher = 20 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: debug objecter = 20 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout:[mds] 2023-08-25T09:38:15.566 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds = 20 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: debug mds balancer = 20 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: debug ms = 1 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug frag = True 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: mds debug scatterstat = True 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: mds op complaint time = 180 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: mds verify scatter = True 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: osd op complaint time = 180 2023-08-25T09:38:15.567 INFO:teuthology.orchestra.run.smithi155.stdout: rados mon op timeout = 900 2023-08-25T09:38:15.568 INFO:teuthology.orchestra.run.smithi155.stdout: rados osd op timeout = 900 2023-08-25T09:38:15.571 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-25T09:38:15.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-25T09:38:15.719 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-25T09:38:15.719 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:15.720 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-25T09:38:15.776 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:15.776 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-25T09:38:15.826 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:38:15.827 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:15.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-25T09:38:15.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-25T09:38:15.939 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:15.940 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:16.034 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:38:16.034 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:16.034 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-25T09:38:16.034 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-25T09:38:16.201 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:16.201 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:16.300 INFO:tasks.ceph:Setting up mon nodes... 2023-08-25T09:38:16.300 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-25T09:38:16.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-08-25T09:38:16.495 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-08-25T09:38:16.505 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-25T09:38:16.704 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-25T09:38:16.714 INFO:tasks.ceph:Setting up mds nodes... 2023-08-25T09:38:16.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-08-25T09:38:16.911 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-08-25T09:38:16.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-08-25T09:38:17.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-08-25T09:38:17.209 INFO:teuthology.orchestra.run.smithi070.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-08-25T09:38:17.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-08-25T09:38:17.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-08-25T09:38:17.510 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-08-25T09:38:17.519 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-08-25T09:38:17.612 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-08-25T09:38:17.790 INFO:teuthology.orchestra.run.smithi155.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-08-25T09:38:17.800 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-08-25T09:38:17.900 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-25T09:38:17.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-25T09:38:18.036 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-25T09:38:18.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-08-25T09:38:18.243 INFO:teuthology.orchestra.run.smithi070.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-08-25T09:38:18.310 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-25T09:38:18.447 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-25T09:38:18.516 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-08-25T09:38:18.643 INFO:teuthology.orchestra.run.smithi155.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-08-25T09:38:18.712 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-25T09:38:18.713 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi070.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi155.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-25T09:38:18.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-25T09:38:18.814 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:18.815 INFO:tasks.ceph:role: osd.0 2023-08-25T09:38:18.815 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:38:18.816 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-25T09:38:18.906 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:18.907 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:18.908 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:18.908 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:18.940 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-08-25T09:38:18.946 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-08-25T09:38:18.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-25T09:38:19.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-25T09:38:19.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-25T09:38:19.232 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:19.233 INFO:tasks.ceph:role: osd.1 2023-08-25T09:38:19.233 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:38:19.233 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-25T09:38:19.326 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:19.327 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:19.327 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:19.327 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:19.327 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:19.327 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:19.327 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:19.328 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:19.328 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:19.328 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:19.361 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-08-25T09:38:19.367 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-08-25T09:38:19.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-25T09:38:19.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-25T09:38:19.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-25T09:38:19.651 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:19.652 INFO:tasks.ceph:role: osd.2 2023-08-25T09:38:19.652 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:38:19.653 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-25T09:38:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:19.744 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:19.744 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:19.744 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:19.745 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:19.745 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:19.745 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:19.745 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:19.745 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:19.777 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-08-25T09:38:19.783 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-08-25T09:38:19.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-08-25T09:38:19.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-25T09:38:19.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-25T09:38:20.071 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:20.071 INFO:tasks.ceph:role: osd.3 2023-08-25T09:38:20.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:38:20.072 DEBUG:teuthology.orchestra.run.smithi070:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-25T09:38:20.160 INFO:teuthology.orchestra.run.smithi070.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:20.160 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:20.160 INFO:teuthology.orchestra.run.smithi070.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:20.161 INFO:teuthology.orchestra.run.smithi070.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:20.193 INFO:teuthology.orchestra.run.smithi070.stdout:Discarding blocks...Done. 2023-08-25T09:38:20.198 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi070.front.sepia.ceph.com -o noatime 2023-08-25T09:38:20.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-08-25T09:38:20.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-25T09:38:20.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:20.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:20.546+0000 7f670c225500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-08-25T09:38:20.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:20.547+0000 7f670c225500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-08-25T09:38:20.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:20.550+0000 7f670c225500 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-08-25T09:38:20.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-25T09:38:20.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:20.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:20.781+0000 7f317320c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-08-25T09:38:20.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:20.781+0000 7f317320c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-08-25T09:38:20.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:20.783+0000 7f317320c500 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-08-25T09:38:20.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-25T09:38:20.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:21.029+0000 7fa3f4e13500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-08-25T09:38:21.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:21.030+0000 7fa3f4e13500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-08-25T09:38:21.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:21.032+0000 7fa3f4e13500 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-08-25T09:38:21.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-25T09:38:21.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:21.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:21.280+0000 7f940b341500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-08-25T09:38:21.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:21.280+0000 7f940b341500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-08-25T09:38:21.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:21.282+0000 7f940b341500 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-08-25T09:38:21.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-25T09:38:21.389 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-08-25T09:38:21.493 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:21.494 INFO:tasks.ceph:role: osd.4 2023-08-25T09:38:21.494 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:38:21.494 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-25T09:38:21.592 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:21.592 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:21.593 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:21.594 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:22.100 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-08-25T09:38:22.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-08-25T09:38:22.108 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-08-25T09:38:22.301 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-08-25T09:38:22.411 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-08-25T09:38:22.506 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:22.507 INFO:tasks.ceph:role: osd.5 2023-08-25T09:38:22.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:38:22.507 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-25T09:38:22.609 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:22.609 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:22.609 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:22.609 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:22.610 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:22.610 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:22.610 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:23.091 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-08-25T09:38:23.098 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-08-25T09:38:23.098 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-08-25T09:38:23.245 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-08-25T09:38:23.344 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-08-25T09:38:23.435 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:23.436 INFO:tasks.ceph:role: osd.6 2023-08-25T09:38:23.436 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:38:23.436 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-25T09:38:23.538 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:23.539 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:23.539 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:23.540 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:23.540 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:23.540 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:23.540 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:24.037 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-08-25T09:38:24.043 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-08-25T09:38:24.044 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-08-25T09:38:24.227 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-08-25T09:38:24.332 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-08-25T09:38:24.424 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T09:38:24.425 INFO:tasks.ceph:role: osd.7 2023-08-25T09:38:24.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:38:24.426 DEBUG:teuthology.orchestra.run.smithi155:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-25T09:38:24.528 INFO:teuthology.orchestra.run.smithi155.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T09:38:24.528 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: = sunit=0 swidth=0 blks 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-08-25T09:38:24.529 INFO:teuthology.orchestra.run.smithi155.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T09:38:24.530 INFO:teuthology.orchestra.run.smithi155.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T09:38:25.001 INFO:teuthology.orchestra.run.smithi155.stdout:Discarding blocks...Done. 2023-08-25T09:38:25.010 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi155.front.sepia.ceph.com -o noatime 2023-08-25T09:38:25.010 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-08-25T09:38:25.159 DEBUG:teuthology.orchestra.run.smithi155:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-08-25T09:38:25.260 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:25.419 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.417+0000 7fe37c61d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-08-25T09:38:25.424 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.422+0000 7fe37c61d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-08-25T09:38:25.434 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.432+0000 7fe37c61d500 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-08-25T09:38:25.469 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-08-25T09:38:25.567 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:25.710 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.708+0000 7f7c47021500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-08-25T09:38:25.713 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.712+0000 7f7c47021500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-08-25T09:38:25.723 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.722+0000 7f7c47021500 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-08-25T09:38:25.749 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-08-25T09:38:25.845 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:25.987 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.986+0000 7f9c79bba500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-08-25T09:38:25.991 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.989+0000 7f9c79bba500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-08-25T09:38:25.999 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:25.998+0000 7f9c79bba500 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-08-25T09:38:26.022 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-08-25T09:38:26.113 DEBUG:teuthology.orchestra.run.smithi155:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:26.248 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:26.246+0000 7fa17353c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-08-25T09:38:26.248 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:26.246+0000 7fa17353c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-08-25T09:38:26.250 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:26.248+0000 7fa17353c500 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-08-25T09:38:26.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-08-25T09:38:26.363 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-25T09:38:26.364 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-08-25T09:38:26.467 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-08-25T09:38:26.561 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-08-25T09:38:26.661 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-25T09:38:26.757 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-25T09:38:26.857 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-25T09:38:26.951 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:26.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-25T09:38:27.045 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.046 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-25T09:38:27.155 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.155 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-08-25T09:38:27.250 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.251 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-08-25T09:38:27.346 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.347 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-08-25T09:38:27.444 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.444 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-08-25T09:38:27.541 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.541 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-08-25T09:38:27.641 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.641 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-08-25T09:38:27.735 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:27.735 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-25T09:38:27.796 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:27.797 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-08-25T09:38:27.850 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.850 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-25T09:38:27.910 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:27.910 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-08-25T09:38:27.965 INFO:tasks.ceph:Adding keys to all mons... 2023-08-25T09:38:27.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-25T09:38:27.975 DEBUG:teuthology.orchestra.run.smithi155:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-25T09:38:28.057 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.y] 2023-08-25T09:38:28.057 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg== 2023-08-25T09:38:28.057 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.a] 2023-08-25T09:38:28.057 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ== 2023-08-25T09:38:28.057 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.c] 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ== 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.0] 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA== 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.1] 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA== 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.2] 2023-08-25T09:38:28.058 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA== 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.3] 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw== 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout:[mgr.x] 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ== 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.b] 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA== 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout:[mds.d] 2023-08-25T09:38:28.059 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw== 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.4] 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w== 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.5] 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ== 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.6] 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw== 2023-08-25T09:38:28.060 INFO:teuthology.orchestra.run.smithi070.stdout:[osd.7] 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg== 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout:[client.2] 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g== 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout:[client.1] 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g== 2023-08-25T09:38:28.061 INFO:teuthology.orchestra.run.smithi070.stdout:[client.3] 2023-08-25T09:38:28.062 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA== 2023-08-25T09:38:28.070 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.y] 2023-08-25T09:38:28.070 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg== 2023-08-25T09:38:28.071 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.a] 2023-08-25T09:38:28.071 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ== 2023-08-25T09:38:28.071 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.c] 2023-08-25T09:38:28.071 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ== 2023-08-25T09:38:28.071 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.0] 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA== 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.1] 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA== 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.2] 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA== 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.3] 2023-08-25T09:38:28.072 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw== 2023-08-25T09:38:28.073 INFO:teuthology.orchestra.run.smithi155.stdout:[mgr.x] 2023-08-25T09:38:28.073 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ== 2023-08-25T09:38:28.073 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.b] 2023-08-25T09:38:28.073 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA== 2023-08-25T09:38:28.073 INFO:teuthology.orchestra.run.smithi155.stdout:[mds.d] 2023-08-25T09:38:28.073 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw== 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.4] 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w== 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.5] 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ== 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.6] 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw== 2023-08-25T09:38:28.074 INFO:teuthology.orchestra.run.smithi155.stdout:[osd.7] 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg== 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout:[client.0] 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout:[client.2] 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g== 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2023-08-25T09:38:28.075 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g== 2023-08-25T09:38:28.076 INFO:teuthology.orchestra.run.smithi155.stdout:[client.3] 2023-08-25T09:38:28.076 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA== 2023-08-25T09:38:28.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T09:38:28.112 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T09:38:28.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:28.272 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:28.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:28.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:28.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:28.576 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:28.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:28.721 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:28.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:28.865 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.023 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T09:38:29.182 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T09:38:29.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:29.328 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:29.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:29.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T09:38:29.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.637 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.797 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:29.958 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:30.104 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:30.114 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T09:38:30.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.277 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.427 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.587 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.748 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T09:38:30.899 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-25T09:38:30.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-25T09:38:30.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-25T09:38:31.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-25T09:38:31.243 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-08-25T09:38:31.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-25T09:38:31.515 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-08-25T09:38:31.599 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-08-25T09:38:31.692 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-25T09:38:31.847 DEBUG:teuthology.orchestra.run.smithi155:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-08-25T09:38:31.927 DEBUG:teuthology.orchestra.run.smithi070:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:31.937 DEBUG:teuthology.orchestra.run.smithi155:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-25T09:38:31.992 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-25T09:38:31.992 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-25T09:38:31.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-25T09:38:31.999 INFO:tasks.ceph.mon.a:Started 2023-08-25T09:38:32.000 INFO:tasks.ceph.mon.b:Restarting daemon 2023-08-25T09:38:32.000 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-08-25T09:38:32.006 INFO:tasks.ceph.mon.b:Started 2023-08-25T09:38:32.007 INFO:tasks.ceph.mon.c:Restarting daemon 2023-08-25T09:38:32.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-08-25T09:38:32.013 INFO:tasks.ceph.mon.c:Started 2023-08-25T09:38:32.013 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-25T09:38:32.014 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-08-25T09:38:32.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-08-25T09:38:32.023 INFO:tasks.ceph.mgr.y:Started 2023-08-25T09:38:32.024 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-25T09:38:32.024 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-25T09:38:32.030 INFO:tasks.ceph.mgr.x:Started 2023-08-25T09:38:32.030 INFO:tasks.ceph:Setting crush tunables to default 2023-08-25T09:38:32.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd crush tunables default 2023-08-25T09:38:32.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.205+0000 7f37e4aec640 1 Processor -- start 2023-08-25T09:38:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.205+0000 7f37e4aec640 1 -- start start 2023-08-25T09:38:32.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.206+0000 7f37e4aec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.206+0000 7f37e4aec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.206+0000 7f37e4aec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e01606d0 con 0x7f37e015c910 2023-08-25T09:38:32.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.206+0000 7f37e4aec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e0069f60 con 0x7f37e006f3b0 2023-08-25T09:38:32.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.206+0000 7f37e4aec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e006f7e0 con 0x7f37e015ccf0 2023-08-25T09:38:32.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37de59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f37e015c910 0x7f37e0069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38914/0 (socket says 172.21.15.70:38914) 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42982/0 (socket says 172.21.15.70:42982) 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 -- 172.21.15.70:0/3650348970 learned_addr learned my addr 172.21.15.70:0/3650348970 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 msgr2=0x7f37e00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 msgr2=0x7f37e00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T09:38:32.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 msgr2=0x7f37e006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-25T09:38:32.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 msgr2=0x7f37e006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-25T09:38:32.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.207+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T09:38:32.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.208+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T09:38:32.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.408+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.409+0000 7f37ded9d640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 msgr2=0x7f37e00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-08-25T09:38:32.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.409+0000 7f37ded9d640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 msgr2=0x7f37e00b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-25T09:38:32.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.409+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T09:38:32.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.409+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T09:38:32.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.409+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.410+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 msgr2=0x7f37e006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-25T09:38:32.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.410+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 msgr2=0x7f37e006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-25T09:38:32.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.410+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T09:38:32.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.410+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-08-25T09:38:32.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.445+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908425234 0 0) 0x7f37e01606d0 con 0x7f37e015c910 2023-08-25T09:38:32.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.445+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37cc003520 con 0x7f37e015c910 2023-08-25T09:38:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.448+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506623424 0 0) 0x7f37cc003520 con 0x7f37e015c910 2023-08-25T09:38:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.448+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 msgr2=0x7f37e006ee70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T09:38:32.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.448+0000 7f37ddd9b640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.155:3300/0 conn(0x7f37e015ccf0 0x7f37e006ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 msgr2=0x7f37e00b2280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T09:38:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 --2- 172.21.15.70:0/3650348970 >> v2:172.21.15.70:3300/0 conn(0x7f37e006f3b0 0x7f37e00b2280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e00cfc40 con 0x7f37e015c910 2023-08-25T09:38:32.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f37d0001670 con 0x7f37e015c910 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 >> v1:172.21.15.70:6789/0 conn(0x7f37e015c910 legacy=0x7f37e0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0039c0 0x7f37cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37cc0047c0 0x7f37cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.449+0000 7f37ddd9b640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37cc005180 0x7f37cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.450+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0069f60 con 0x7f37cc0047c0 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.450+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e006f7e0 con 0x7f37cc0039c0 2023-08-25T09:38:32.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.450+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc00c0d0 con 0x7f37cc005180 2023-08-25T09:38:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.450+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37cc0047c0 0x7f37cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37de59c640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0039c0 0x7f37cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37cc005180 0x7f37cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37cc005180 msgr2=0x7f37cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:32.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37cc005180 0x7f37cc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0039c0 msgr2=0x7f37cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:32.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0039c0 0x7f37cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.451+0000 7f37d7fff640 1 -- 172.21.15.70:0/3650348970 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc003520 con 0x7f37cc0047c0 2023-08-25T09:38:32.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.452+0000 7f37ded9d640 1 -- 172.21.15.70:0/3650348970 reap_dead start 2023-08-25T09:38:32.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.453+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37cc0047c0 0x7f37cc004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37c8006fd0 tx=0x7f37cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6d6ead3546814 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:32.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.454+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c802c7e0 con 0x7f37cc0047c0 2023-08-25T09:38:32.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.454+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c8030070 con 0x7f37cc0047c0 2023-08-25T09:38:32.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.454+0000 7f37ddd9b640 1 -- 172.21.15.70:0/3650348970 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c802cb80 con 0x7f37cc0047c0 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 -- 172.21.15.70:0/3650348970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37cc0047c0 msgr2=0x7f37cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37cc0047c0 0x7f37cc004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f37c8006fd0 tx=0x7f37cc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 -- 172.21.15.70:0/3650348970 shutdown_connections 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 --2- 172.21.15.70:0/3650348970 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37cc0047c0 0x7f37cc004c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 -- 172.21.15.70:0/3650348970 >> 172.21.15.70:0/3650348970 conn(0x7f37e00cd280 msgr2=0x7f37e00cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 -- 172.21.15.70:0/3650348970 shutdown_connections 2023-08-25T09:38:32.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.455+0000 7f37e4aec640 1 -- 172.21.15.70:0/3650348970 wait complete. 2023-08-25T09:38:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.458+0000 7f37e4aec640 1 Processor -- start 2023-08-25T09:38:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.458+0000 7f37e4aec640 1 -- start start 2023-08-25T09:38:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37e4aec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0047c0 0x7f37e01586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37e4aec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 0x7f37e00c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37e4aec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37e00acd00 0x7f37e00ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:32.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37e4aec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0158be0 con 0x7f37e0158d60 2023-08-25T09:38:32.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37e4aec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e00b44f0 con 0x7f37cc0047c0 2023-08-25T09:38:32.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37e4aec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e0159540 con 0x7f37e00acd00 2023-08-25T09:38:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 0x7f37e00c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 0x7f37e00c6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43004/0 (socket says 172.21.15.70:43004) 2023-08-25T09:38:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.459+0000 7f37d7fff640 1 -- 172.21.15.70:0/3342161243 learned_addr learned my addr 172.21.15.70:0/3342161243 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37de59c640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0047c0 0x7f37e01586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37ded9d640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37e00acd00 0x7f37e00ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37d7fff640 1 -- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37e00acd00 msgr2=0x7f37e00ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37e00acd00 0x7f37e00ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37d7fff640 1 -- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0047c0 msgr2=0x7f37e01586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0047c0 0x7f37e01586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37d7fff640 1 -- 172.21.15.70:0/3342161243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0159260 con 0x7f37e0158d60 2023-08-25T09:38:32.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.460+0000 7f37d7fff640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 0x7f37e00c6850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37c802c180 tx=0x7f37c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.462+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c803f850 con 0x7f37e0158d60 2023-08-25T09:38:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.462+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e00b3ea0 con 0x7f37e0158d60 2023-08-25T09:38:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.462+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c803f9b0 con 0x7f37e0158d60 2023-08-25T09:38:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.462+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c803fd60 con 0x7f37e0158d60 2023-08-25T09:38:32.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.463+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e00c6d90 con 0x7f37e0158d60 2023-08-25T09:38:32.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.463+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f37c8049700 con 0x7f37e0158d60 2023-08-25T09:38:32.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.464+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f37c8036720 con 0x7f37e0158d60 2023-08-25T09:38:32.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.464+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e00d0490 con 0x7f37e0158d60 2023-08-25T09:38:32.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.470+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f37c8067b00 con 0x7f37e0158d60 2023-08-25T09:38:32.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.544+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f37e00b4660 con 0x7f37e0158d60 2023-08-25T09:38:32.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.548+0000 7f37d6ffd640 1 -- 172.21.15.70:0/3342161243 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f37c8056020 con 0x7f37e0158d60 2023-08-25T09:38:32.550 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2023-08-25T09:38:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 msgr2=0x7f37e00c6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:32.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 0x7f37e00c6850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f37c802c180 tx=0x7f37c80046c0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 shutdown_connections 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37e00acd00 0x7f37e00ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37e0158d60 0x7f37e00c6850 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 --2- 172.21.15.70:0/3342161243 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37cc0047c0 0x7f37e01586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 >> 172.21.15.70:0/3342161243 conn(0x7f37e00cd280 msgr2=0x7f37e0160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 shutdown_connections 2023-08-25T09:38:32.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:32.550+0000 7f37e4aec640 1 -- 172.21.15.70:0/3342161243 wait complete. 2023-08-25T09:38:32.568 INFO:tasks.ceph:check_enable_crimson: False 2023-08-25T09:38:32.568 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-25T09:38:32.569 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:32.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-25T09:38:32.654 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:32.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-25T09:38:32.656 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:32.654+0000 7eff8dc16040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T09:38:32.664 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:32.662+0000 7f74a2c15040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T09:38:32.736 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:32.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-25T09:38:32.872 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:38:32.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-25T09:38:32.953 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:32.953 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-08-25T09:38:33.043 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:33.044 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-08-25T09:38:33.136 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:33.137 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-08-25T09:38:33.264 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:38:33.265 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-08-25T09:38:33.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-08-25T09:38:33.600 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.598+0000 7f5d4a114640 1 Processor -- start 2023-08-25T09:38:33.601 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.599+0000 7f5d4a114640 1 -- start start 2023-08-25T09:38:33.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d4a114640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d4a114640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d440b19a0 0x7f5d440d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d4a114640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d440cd900 con 0x7f5d4415ca20 2023-08-25T09:38:33.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d4a114640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4415ce00 con 0x7f5d440b19a0 2023-08-25T09:38:33.602 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d4a114640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44160fb0 con 0x7f5d440cf9a0 2023-08-25T09:38:33.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d48911640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d49112640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5d4415ca20 0x7f5d440cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:59566/0 (socket says 172.21.15.155:59566) 2023-08-25T09:38:33.603 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.600+0000 7f5d49112640 1 -- 172.21.15.155:0/1362314846 learned_addr learned my addr 172.21.15.155:0/1362314846 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-08-25T09:38:33.605 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.601+0000 7f5d49913640 1 --2- 172.21.15.155:0/1362314846 >> v2:172.21.15.70:3300/0 conn(0x7f5d440b19a0 0x7f5d440d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.605 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.601+0000 7f5d48911640 1 -- 172.21.15.155:0/1362314846 >> v2:172.21.15.70:3300/0 conn(0x7f5d440b19a0 msgr2=0x7f5d440d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:33.605 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.601+0000 7f5d3bfff640 1 -- 172.21.15.155:0/1362314846 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023614630 0 0) 0x7f5d440cd900 con 0x7f5d4415ca20 2023-08-25T09:38:33.605 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.601+0000 7f5d48911640 1 --2- 172.21.15.155:0/1362314846 >> v2:172.21.15.70:3300/0 conn(0x7f5d440b19a0 0x7f5d440d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.605 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.601+0000 7f5d48911640 1 -- 172.21.15.155:0/1362314846 >> v1:172.21.15.155:6789/0 conn(0x7f5d4415ca20 legacy=0x7f5d440cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.601+0000 7f5d48911640 1 -- 172.21.15.155:0/1362314846 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d44161290 con 0x7f5d440cf9a0 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.602+0000 7f5d48911640 1 --2- 172.21.15.155:0/1362314846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440d0de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5d340097a0 tx=0x7f5d3402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55938226383eb102 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.602+0000 7f5d3bfff640 1 -- 172.21.15.155:0/1362314846 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34031830 con 0x7f5d440cf9a0 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.602+0000 7f5d3bfff640 1 -- 172.21.15.155:0/1362314846 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d34031990 con 0x7f5d440cf9a0 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.602+0000 7f5d3bfff640 1 -- 172.21.15.155:0/1362314846 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34031d20 con 0x7f5d440cf9a0 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 -- 172.21.15.155:0/1362314846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 msgr2=0x7f5d440d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.606 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1362314846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440d0de0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5d340097a0 tx=0x7f5d3402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.607 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 -- 172.21.15.155:0/1362314846 shutdown_connections 2023-08-25T09:38:33.607 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1362314846 >> v2:172.21.15.70:3300/0 conn(0x7f5d440b19a0 0x7f5d440d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.607 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1362314846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440d0de0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.607 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 -- 172.21.15.155:0/1362314846 >> 172.21.15.155:0/1362314846 conn(0x7f5d440704a0 msgr2=0x7f5d440b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:33.607 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 -- 172.21.15.155:0/1362314846 shutdown_connections 2023-08-25T09:38:33.607 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.603+0000 7f5d4a114640 1 -- 172.21.15.155:0/1362314846 wait complete. 2023-08-25T09:38:33.609 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.605+0000 7f5d4a114640 1 Processor -- start 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.605+0000 7f5d4a114640 1 -- start start 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d4a114640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d440b19a0 0x7f5d44151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d4a114640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d4a114640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 0x7f5d440b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d4a114640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441f3430 con 0x7f5d440b19a0 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d4a114640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4415d130 con 0x7f5d4415ca20 2023-08-25T09:38:33.610 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d4a114640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4415d410 con 0x7f5d440cf9a0 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d48911640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d49913640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 0x7f5d440b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d48911640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:47264/0 (socket says 172.21.15.155:47264) 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d49913640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 0x7f5d440b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:43984/0 (socket says 172.21.15.155:43984) 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d48911640 1 -- 172.21.15.155:0/1881391326 learned_addr learned my addr 172.21.15.155:0/1881391326 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.606+0000 7f5d49112640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d440b19a0 0x7f5d44151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.611 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d49913640 1 -- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 msgr2=0x7f5d440b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.612 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d49913640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.612 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d49913640 1 -- 172.21.15.155:0/1881391326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d440b19a0 msgr2=0x7f5d44151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.612 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d49913640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d440b19a0 0x7f5d44151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.612 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d49913640 1 -- 172.21.15.155:0/1881391326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34009400 con 0x7f5d4415ca20 2023-08-25T09:38:33.612 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d49913640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 0x7f5d440b3d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007d90 tx=0x7f5d3c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c018070 con 0x7f5d4415ca20 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00edd0 con 0x7f5d4415ca20 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4415d6f0 con 0x7f5d4415ca20 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c003bb0 con 0x7f5d4415ca20 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.607+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d440cb270 con 0x7f5d4415ca20 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.608+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5d3c014720 con 0x7f5d4415ca20 2023-08-25T09:38:33.613 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.608+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5d3c01d8f0 con 0x7f5d4415ca20 2023-08-25T09:38:33.614 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.609+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4415dc10 con 0x7f5d4415ca20 2023-08-25T09:38:33.614 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.612+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5d4415dc10 con 0x7f5d4415ca20 2023-08-25T09:38:33.652 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:33.650+0000 7eff8dc16040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T09:38:33.658 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:33.656+0000 7f74a2c15040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T09:38:33.705 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.703+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f5d440b1760 con 0x7f5d4415ca20 2023-08-25T09:38:33.712 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.711+0000 7f5d39ffb640 1 -- 172.21.15.155:0/1881391326 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5d3c02c0b0 con 0x7f5d4415ca20 2023-08-25T09:38:33.714 INFO:teuthology.orchestra.run.smithi155.stdout:0 2023-08-25T09:38:33.714 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 msgr2=0x7f5d440b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.715 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 0x7f5d440b3d80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007d90 tx=0x7f5d3c00e4b0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.715 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 shutdown_connections 2023-08-25T09:38:33.715 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d4415ca20 0x7f5d440b3d80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.715 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d440cf9a0 0x7f5d440b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 --2- 172.21.15.155:0/1881391326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d440b19a0 0x7f5d44151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 >> 172.21.15.155:0/1881391326 conn(0x7f5d440704a0 msgr2=0x7f5d440b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:33.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 shutdown_connections 2023-08-25T09:38:33.716 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.712+0000 7f5d4a114640 1 -- 172.21.15.155:0/1881391326 wait complete. 2023-08-25T09:38:33.730 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-08-25T09:38:33.808 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:33.806+0000 7eff8dc16040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T09:38:33.824 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:33.822+0000 7f74a2c15040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T09:38:33.864 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:33.863+0000 7eff8dc16040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T09:38:33.884 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:33.882+0000 7f74a2c15040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T09:38:33.916 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:33.914+0000 7eff8dc16040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T09:38:33.926 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.924+0000 7faf918cc640 1 Processor -- start 2023-08-25T09:38:33.926 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.924+0000 7faf918cc640 1 -- start start 2023-08-25T09:38:33.928 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.926+0000 7faf918cc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf8c0cb8d0 0x7faf8c06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.928 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.926+0000 7faf918cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.926+0000 7faf918cc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf8c072d60 0x7faf8c160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.926+0000 7faf918cc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8c0731f0 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.926+0000 7faf918cc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf8c0cf4f0 con 0x7faf8c072d60 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.926+0000 7faf918cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0cebc0 con 0x7faf8c0b1e00 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf8c072d60 0x7faf8c160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf83fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.929 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf8c072d60 0x7faf8c160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:43990/0 (socket says 172.21.15.155:43990) 2023-08-25T09:38:33.930 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 -- 172.21.15.155:0/2441723053 learned_addr learned my addr 172.21.15.155:0/2441723053 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:33.930 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8affd640 1 --2- 172.21.15.155:0/2441723053 >> v2:172.21.15.70:3300/0 conn(0x7faf8c0cb8d0 0x7faf8c06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.931 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 -- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 msgr2=0x7faf8c072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:33.931 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 --2- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.931 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 -- 172.21.15.155:0/2441723053 >> v2:172.21.15.70:3300/0 conn(0x7faf8c0cb8d0 msgr2=0x7faf8c06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:33.931 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 --2- 172.21.15.155:0/2441723053 >> v2:172.21.15.70:3300/0 conn(0x7faf8c0cb8d0 0x7faf8c06aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.932 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.927+0000 7faf8b7fe640 1 -- 172.21.15.155:0/2441723053 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0ceea0 con 0x7faf8c072d60 2023-08-25T09:38:33.932 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.928+0000 7faf8b7fe640 1 --2- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c072d60 0x7faf8c160970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faf8c073470 tx=0x7faf8402be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b64b2abb28a69f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:33.933 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.928+0000 7faf8a7fc640 1 -- 172.21.15.155:0/2441723053 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8402c9f0 con 0x7faf8c072d60 2023-08-25T09:38:33.933 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.928+0000 7faf8a7fc640 1 -- 172.21.15.155:0/2441723053 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8402cb50 con 0x7faf8c072d60 2023-08-25T09:38:33.933 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.928+0000 7faf8a7fc640 1 -- 172.21.15.155:0/2441723053 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840343f0 con 0x7faf8c072d60 2023-08-25T09:38:33.933 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 -- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c072d60 msgr2=0x7faf8c160970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.933 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 --2- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c072d60 0x7faf8c160970 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7faf8c073470 tx=0x7faf8402be50 comp rx=0 tx=0).stop 2023-08-25T09:38:33.933 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 -- 172.21.15.155:0/2441723053 shutdown_connections 2023-08-25T09:38:33.934 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 --2- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c072d60 0x7faf8c160970 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.934 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 --2- 172.21.15.155:0/2441723053 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.934 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 --2- 172.21.15.155:0/2441723053 >> v2:172.21.15.70:3300/0 conn(0x7faf8c0cb8d0 0x7faf8c06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.934 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.929+0000 7faf918cc640 1 -- 172.21.15.155:0/2441723053 >> 172.21.15.155:0/2441723053 conn(0x7faf8c0b1270 msgr2=0x7faf8c0b1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:33.934 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.930+0000 7faf918cc640 1 -- 172.21.15.155:0/2441723053 shutdown_connections 2023-08-25T09:38:33.934 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.930+0000 7faf918cc640 1 -- 172.21.15.155:0/2441723053 wait complete. 2023-08-25T09:38:33.935 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.932+0000 7faf918cc640 1 Processor -- start 2023-08-25T09:38:33.935 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.933+0000 7faf918cc640 1 -- start start 2023-08-25T09:38:33.937 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf918cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8c072d60 0x7faf8c1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.937 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf918cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.937 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf918cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 0x7faf8c1f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:33.937 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf918cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0c7320 con 0x7faf8c072d60 2023-08-25T09:38:33.937 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf918cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c1f6b90 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.937 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf918cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0c7530 con 0x7faf8c0b1e00 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf8b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 0x7faf8c1f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf83fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf8b7fe640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 0x7faf8c1f5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:43994/0 (socket says 172.21.15.155:43994) 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.934+0000 7faf8b7fe640 1 -- 172.21.15.155:0/1898091953 learned_addr learned my addr 172.21.15.155:0/1898091953 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8affd640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8c072d60 0x7faf8c1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8b7fe640 1 -- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 msgr2=0x7faf8c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.938 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8b7fe640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.939 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8b7fe640 1 -- 172.21.15.155:0/1898091953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8c072d60 msgr2=0x7faf8c1570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:33.939 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8b7fe640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8c072d60 0x7faf8c1570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:33.939 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8b7fe640 1 -- 172.21.15.155:0/1898091953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf84009b30 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.939 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8affd640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8c072d60 0x7faf8c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:33.939 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf83fff640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:33.941 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.935+0000 7faf8b7fe640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 0x7faf8c1f5e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf84036040 tx=0x7faf84004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:33.941 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.936+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84009510 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.941 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.936+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8403e450 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.936+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf8c0c7810 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.936+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8403e7e0 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.936+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c0c94c0 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.937+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faf8403ec40 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.937+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7faf84048e40 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.937+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c04e420 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.942 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:33.940+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7faf84066b00 con 0x7faf8c0cb8d0 2023-08-25T09:38:33.944 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:33.943+0000 7f74a2c15040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T09:38:34.041 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.037+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7faf8c0b2ec0 con 0x7faf8c0cb8d0 2023-08-25T09:38:34.046 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.044+0000 7faf88ff9640 1 -- 172.21.15.155:0/1898091953 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7faf84055030 con 0x7faf8c0cb8d0 2023-08-25T09:38:34.047 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:38:34.048 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 msgr2=0x7faf8c1f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.048 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 0x7faf8c1f5e10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faf84036040 tx=0x7faf84004a40 comp rx=0 tx=0).stop 2023-08-25T09:38:34.048 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 shutdown_connections 2023-08-25T09:38:34.049 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf8c0cb8d0 0x7faf8c1f5e10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.049 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf8c0b1e00 0x7faf8c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.049 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 --2- 172.21.15.155:0/1898091953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf8c072d60 0x7faf8c1570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.050 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 >> 172.21.15.155:0/1898091953 conn(0x7faf8c0b1270 msgr2=0x7faf8c0b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.050 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 shutdown_connections 2023-08-25T09:38:34.050 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.045+0000 7faf918cc640 1 -- 172.21.15.155:0/1898091953 wait complete. 2023-08-25T09:38:34.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-08-25T09:38:34.263 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.261+0000 7fc2d6a26640 1 Processor -- start 2023-08-25T09:38:34.263 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.261+0000 7fc2d6a26640 1 -- start start 2023-08-25T09:38:34.264 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.262+0000 7fc2d6a26640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2d0069850 0x7fc2d0069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2d6a26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d015cef0 0x7fc2d00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2d6a26640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d00d1ac0 con 0x7fc2d00d1f50 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2d6a26640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d00b2040 con 0x7fc2d0069850 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2d6a26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d006a170 con 0x7fc2d015cef0 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2d0069850 0x7fc2d0069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2d0069850 0x7fc2d0069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:43998/0 (socket says 172.21.15.155:43998) 2023-08-25T09:38:34.265 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2d4fc3640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc2d00d1f50 0x7fc2d00b5aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:59578/0 (socket says 172.21.15.155:59578) 2023-08-25T09:38:34.266 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cffff640 1 -- 172.21.15.155:0/3457969091 learned_addr learned my addr 172.21.15.155:0/3457969091 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cf7fe640 1 --2- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d015cef0 0x7fc2d00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cffff640 1 -- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d015cef0 msgr2=0x7fc2d00b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cffff640 1 --2- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d015cef0 0x7fc2d00b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.263+0000 7fc2cffff640 1 -- 172.21.15.155:0/3457969091 >> v1:172.21.15.155:6789/0 conn(0x7fc2d00d1f50 legacy=0x7fc2d00b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.264+0000 7fc2cffff640 1 -- 172.21.15.155:0/3457969091 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00d0eb0 con 0x7fc2d0069850 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.265+0000 7fc2cffff640 1 --2- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d0069850 0x7fc2d0069c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f5a5a2e2c45f9e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:34.267 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.265+0000 7fc2ceffd640 1 -- 172.21.15.155:0/3457969091 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c002f830 con 0x7fc2d0069850 2023-08-25T09:38:34.268 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.265+0000 7fc2ceffd640 1 -- 172.21.15.155:0/3457969091 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c002f990 con 0x7fc2d0069850 2023-08-25T09:38:34.268 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.265+0000 7fc2ceffd640 1 -- 172.21.15.155:0/3457969091 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c002fd20 con 0x7fc2d0069850 2023-08-25T09:38:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 -- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d0069850 msgr2=0x7fc2d0069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d0069850 0x7fc2d0069c30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c002ba10 comp rx=0 tx=0).stop 2023-08-25T09:38:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 -- 172.21.15.155:0/3457969091 shutdown_connections 2023-08-25T09:38:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d015cef0 0x7fc2d00b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/3457969091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d0069850 0x7fc2d0069c30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 -- 172.21.15.155:0/3457969091 >> 172.21.15.155:0/3457969091 conn(0x7fc2d0070290 msgr2=0x7fc2d00706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 -- 172.21.15.155:0/3457969091 shutdown_connections 2023-08-25T09:38:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.266+0000 7fc2d6a26640 1 -- 172.21.15.155:0/3457969091 wait complete. 2023-08-25T09:38:34.271 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.268+0000 7fc2d6a26640 1 Processor -- start 2023-08-25T09:38:34.271 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.268+0000 7fc2d6a26640 1 -- start start 2023-08-25T09:38:34.271 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d6a26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 0x7fc2d015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d6a26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d00d1f50 0x7fc2d0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d6a26640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d015cef0 0x7fc2d0147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d6a26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0159e70 con 0x7fc2d00d1f50 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d6a26640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d015a150 con 0x7fc2d015cef0 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d6a26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d015a430 con 0x7fc2d0069850 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 0x7fc2d015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.272 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 0x7fc2d015ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:47294/0 (socket says 172.21.15.155:47294) 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 -- 172.21.15.155:0/2411838651 learned_addr learned my addr 172.21.15.155:0/2411838651 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2d4fc3640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d015cef0 0x7fc2d0147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cf7fe640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d00d1f50 0x7fc2d0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 -- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d015cef0 msgr2=0x7fc2d0147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d015cef0 0x7fc2d0147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 -- 172.21.15.155:0/2411838651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d00d1f50 msgr2=0x7fc2d0159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.273 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d00d1f50 0x7fc2d0159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.274 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cffff640 1 -- 172.21.15.155:0/2411838651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c00094d0 con 0x7fc2d0069850 2023-08-25T09:38:34.274 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.269+0000 7fc2cf7fe640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d00d1f50 0x7fc2d0159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:34.274 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2cffff640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 0x7fc2d015ff20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0000c00 tx=0x7fc2c0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:34.275 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0037c40 con 0x7fc2d0069850 2023-08-25T09:38:34.275 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0037da0 con 0x7fc2d0069850 2023-08-25T09:38:34.275 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d015a710 con 0x7fc2d0069850 2023-08-25T09:38:34.276 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0041640 con 0x7fc2d0069850 2023-08-25T09:38:34.276 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d015af10 con 0x7fc2d0069850 2023-08-25T09:38:34.276 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.270+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc2c0041aa0 con 0x7fc2d0069850 2023-08-25T09:38:34.277 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.271+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc2c003c070 con 0x7fc2d0069850 2023-08-25T09:38:34.277 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.271+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d004e420 con 0x7fc2d0069850 2023-08-25T09:38:34.277 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.274+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc2c0041d50 con 0x7fc2d0069850 2023-08-25T09:38:34.355 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.353+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7fc2d00cca90 con 0x7fc2d0069850 2023-08-25T09:38:34.362 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.361+0000 7fc2ccff9640 1 -- 172.21.15.155:0/2411838651 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc2c004b460 con 0x7fc2d0069850 2023-08-25T09:38:34.363 INFO:teuthology.orchestra.run.smithi155.stdout:2 2023-08-25T09:38:34.364 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 msgr2=0x7fc2d015ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 0x7fc2d015ff20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0000c00 tx=0x7fc2c0004730 comp rx=0 tx=0).stop 2023-08-25T09:38:34.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 shutdown_connections 2023-08-25T09:38:34.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d015cef0 0x7fc2d0147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d00d1f50 0x7fc2d0159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.365 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 --2- 172.21.15.155:0/2411838651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d0069850 0x7fc2d015ff20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 >> 172.21.15.155:0/2411838651 conn(0x7fc2d0070290 msgr2=0x7fc2d00c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 shutdown_connections 2023-08-25T09:38:34.366 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.362+0000 7fc2d6a26640 1 -- 172.21.15.155:0/2411838651 wait complete. 2023-08-25T09:38:34.381 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-08-25T09:38:34.564 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.562+0000 7f232ac34640 1 Processor -- start 2023-08-25T09:38:34.564 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.562+0000 7f232ac34640 1 -- start start 2023-08-25T09:38:34.566 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f232ac34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2324069850 0x7f2324069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.567 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f232ac34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f232415cef0 0x7f23240b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.567 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f232ac34640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23240d1ac0 con 0x7f23240d1f50 2023-08-25T09:38:34.567 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f232ac34640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23240b2040 con 0x7f2324069850 2023-08-25T09:38:34.567 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f232ac34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f232406a170 con 0x7f232415cef0 2023-08-25T09:38:34.568 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f23291d1640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f23240d1f50 0x7f23240b5aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:59590/0 (socket says 172.21.15.155:59590) 2023-08-25T09:38:34.568 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f2323fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f232415cef0 0x7f23240b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.568 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.563+0000 7f23291d1640 1 -- 172.21.15.155:0/1645918126 learned_addr learned my addr 172.21.15.155:0/1645918126 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-08-25T09:38:34.568 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f23289d0640 1 --2- 172.21.15.155:0/1645918126 >> v2:172.21.15.70:3300/0 conn(0x7f2324069850 0x7f2324069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.568 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f23237fe640 1 -- 172.21.15.155:0/1645918126 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959231190 0 0) 0x7f23240d1ac0 con 0x7f23240d1f50 2023-08-25T09:38:34.569 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f23237fe640 1 -- 172.21.15.155:0/1645918126 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2310003520 con 0x7f23240d1f50 2023-08-25T09:38:34.569 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f2323fff640 1 -- 172.21.15.155:0/1645918126 >> v2:172.21.15.70:3300/0 conn(0x7f2324069850 msgr2=0x7f2324069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:34.569 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f2323fff640 1 --2- 172.21.15.155:0/1645918126 >> v2:172.21.15.70:3300/0 conn(0x7f2324069850 0x7f2324069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.569 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f2323fff640 1 -- 172.21.15.155:0/1645918126 >> v1:172.21.15.155:6789/0 conn(0x7f23240d1f50 legacy=0x7f23240b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.570 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f2323fff640 1 -- 172.21.15.155:0/1645918126 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23240d0eb0 con 0x7f232415cef0 2023-08-25T09:38:34.570 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.564+0000 7f23289d0640 1 --2- 172.21.15.155:0/1645918126 >> v2:172.21.15.70:3300/0 conn(0x7f2324069850 0x7f2324069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:34.575 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.566+0000 7f2323fff640 1 --2- 172.21.15.155:0/1645918126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f232415cef0 0x7f23240b8300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2314009d90 tx=0x7f231402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7251f7e6fdca5b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:34.575 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.566+0000 7f23237fe640 1 -- 172.21.15.155:0/1645918126 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231402cce0 con 0x7f232415cef0 2023-08-25T09:38:34.575 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.566+0000 7f23237fe640 1 -- 172.21.15.155:0/1645918126 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231402ce40 con 0x7f232415cef0 2023-08-25T09:38:34.575 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.566+0000 7f23237fe640 1 -- 172.21.15.155:0/1645918126 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314034640 con 0x7f232415cef0 2023-08-25T09:38:34.575 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 -- 172.21.15.155:0/1645918126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f232415cef0 msgr2=0x7f23240b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.575 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 --2- 172.21.15.155:0/1645918126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f232415cef0 0x7f23240b8300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2314009d90 tx=0x7f231402bf30 comp rx=0 tx=0).stop 2023-08-25T09:38:34.576 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 -- 172.21.15.155:0/1645918126 shutdown_connections 2023-08-25T09:38:34.576 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 --2- 172.21.15.155:0/1645918126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f232415cef0 0x7f23240b8300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.576 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 --2- 172.21.15.155:0/1645918126 >> v2:172.21.15.70:3300/0 conn(0x7f2324069850 0x7f2324069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.576 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 -- 172.21.15.155:0/1645918126 >> 172.21.15.155:0/1645918126 conn(0x7f2324070290 msgr2=0x7f23240706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.576 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 -- 172.21.15.155:0/1645918126 shutdown_connections 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.567+0000 7f232ac34640 1 -- 172.21.15.155:0/1645918126 wait complete. 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.569+0000 7f232ac34640 1 Processor -- start 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.569+0000 7f232ac34640 1 -- start start 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f232ac34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 0x7f2324158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f232ac34640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23240d1f50 0x7f232415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f232ac34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f232415cef0 0x7f23240b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f232ac34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f232415d810 con 0x7f232415cef0 2023-08-25T09:38:34.577 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f232ac34640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23240b4e60 con 0x7f23240d1f50 2023-08-25T09:38:34.579 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f232ac34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23240b5140 con 0x7f2324069850 2023-08-25T09:38:34.580 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f23289d0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 0x7f2324158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.580 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f23289d0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 0x7f2324158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:47316/0 (socket says 172.21.15.155:47316) 2023-08-25T09:38:34.580 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f23289d0640 1 -- 172.21.15.155:0/2988577087 learned_addr learned my addr 172.21.15.155:0/2988577087 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:34.581 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.570+0000 7f23289d0640 1 -- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23240d1f50 msgr2=0x7f232415d2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T09:38:34.581 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23289d0640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23240d1f50 0x7f232415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.581 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23289d0640 1 -- 172.21.15.155:0/2988577087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f232415cef0 msgr2=0x7f23240b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23289d0640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f232415cef0 0x7f23240b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23289d0640 1 -- 172.21.15.155:0/2988577087 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314009a70 con 0x7f2324069850 2023-08-25T09:38:34.582 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23289d0640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 0x7f2324158100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f230c00d7d0 tx=0x7f230c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:34.585 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c004030 con 0x7f2324069850 2023-08-25T09:38:34.585 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c004190 con 0x7f2324069850 2023-08-25T09:38:34.585 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c004520 con 0x7f2324069850 2023-08-25T09:38:34.585 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23241f30f0 con 0x7f2324069850 2023-08-25T09:38:34.585 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.571+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2324157290 con 0x7f2324069850 2023-08-25T09:38:34.585 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.572+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f230c00b9c0 con 0x7f2324069850 2023-08-25T09:38:34.586 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.572+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f230c00bc70 con 0x7f2324069850 2023-08-25T09:38:34.586 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.572+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23241f2f50 con 0x7f2324069850 2023-08-25T09:38:34.586 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.577+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f23241f2f50 con 0x7f2324069850 2023-08-25T09:38:34.671 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.669+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f2324157480 con 0x7f2324069850 2023-08-25T09:38:34.676 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.675+0000 7f23217fa640 1 -- 172.21.15.155:0/2988577087 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f230c003d40 con 0x7f2324069850 2023-08-25T09:38:34.677 INFO:teuthology.orchestra.run.smithi155.stdout:3 2023-08-25T09:38:34.680 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 msgr2=0x7f2324158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.680 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 0x7f2324158100 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f230c00d7d0 tx=0x7f230c00dc90 comp rx=0 tx=0).stop 2023-08-25T09:38:34.680 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 shutdown_connections 2023-08-25T09:38:34.680 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f232415cef0 0x7f23240b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.681 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23240d1f50 0x7f232415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.681 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 --2- 172.21.15.155:0/2988577087 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2324069850 0x7f2324158100 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.681 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.676+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 >> 172.21.15.155:0/2988577087 conn(0x7f2324070290 msgr2=0x7f23240c6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.681 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.677+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 shutdown_connections 2023-08-25T09:38:34.681 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.677+0000 7f232ac34640 1 -- 172.21.15.155:0/2988577087 wait complete. 2023-08-25T09:38:34.694 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-08-25T09:38:34.864 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:34.863+0000 7eff8dc16040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T09:38:34.873 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.871+0000 7f9048d34640 1 Processor -- start 2023-08-25T09:38:34.874 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.871+0000 7f9048d34640 1 -- start start 2023-08-25T09:38:34.876 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9048d34640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f90440cf9a0 0x7f90440d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.876 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9048d34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f90440b19a0 0x7f90440d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9048d34640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90440cd900 con 0x7f904415ca20 2023-08-25T09:38:34.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9048d34640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904415ce00 con 0x7f90440b19a0 2023-08-25T09:38:34.877 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9048d34640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9044160fb0 con 0x7f90440cf9a0 2023-08-25T09:38:34.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9042ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f90440cf9a0 0x7f90440d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9042ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f90440cf9a0 0x7f90440d0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44028/0 (socket says 172.21.15.155:44028) 2023-08-25T09:38:34.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.873+0000 7f9042ffd640 1 -- 172.21.15.155:0/1827031645 learned_addr learned my addr 172.21.15.155:0/1827031645 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:34.878 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.874+0000 7f9042ffd640 1 -- 172.21.15.155:0/1827031645 >> v2:172.21.15.70:3300/0 conn(0x7f90440b19a0 msgr2=0x7f90440d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:34.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.874+0000 7f9042ffd640 1 --2- 172.21.15.155:0/1827031645 >> v2:172.21.15.70:3300/0 conn(0x7f90440b19a0 0x7f90440d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.874+0000 7f9042ffd640 1 -- 172.21.15.155:0/1827031645 >> v1:172.21.15.70:6789/0 conn(0x7f904415ca20 legacy=0x7f90440cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:34.879 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.874+0000 7f9042ffd640 1 -- 172.21.15.155:0/1827031645 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9044161290 con 0x7f90440cf9a0 2023-08-25T09:38:34.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.875+0000 7f9042ffd640 1 --2- 172.21.15.155:0/1827031645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f90440d0de0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90440d39a0 tx=0x7f903802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b4a5c7d30d4bb7d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:34.885 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.875+0000 7f90427fc640 1 -- 172.21.15.155:0/1827031645 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038031870 con 0x7f90440cf9a0 2023-08-25T09:38:34.886 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.875+0000 7f90427fc640 1 -- 172.21.15.155:0/1827031645 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90380319d0 con 0x7f90440cf9a0 2023-08-25T09:38:34.886 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.875+0000 7f90427fc640 1 -- 172.21.15.155:0/1827031645 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038031d60 con 0x7f90440cf9a0 2023-08-25T09:38:34.888 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.875+0000 7f9048d34640 1 -- 172.21.15.155:0/1827031645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 msgr2=0x7f90440d0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.888 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.875+0000 7f9048d34640 1 --2- 172.21.15.155:0/1827031645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f90440d0de0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f90440d39a0 tx=0x7f903802b910 comp rx=0 tx=0).stop 2023-08-25T09:38:34.889 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.876+0000 7f9048d34640 1 -- 172.21.15.155:0/1827031645 shutdown_connections 2023-08-25T09:38:34.889 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.876+0000 7f9048d34640 1 --2- 172.21.15.155:0/1827031645 >> v2:172.21.15.70:3300/0 conn(0x7f90440b19a0 0x7f90440d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.889 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.876+0000 7f9048d34640 1 --2- 172.21.15.155:0/1827031645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f90440d0de0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.889 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.876+0000 7f9048d34640 1 -- 172.21.15.155:0/1827031645 >> 172.21.15.155:0/1827031645 conn(0x7f90440704a0 msgr2=0x7f90440b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.890 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.876+0000 7f9048d34640 1 -- 172.21.15.155:0/1827031645 shutdown_connections 2023-08-25T09:38:34.890 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.876+0000 7f9048d34640 1 -- 172.21.15.155:0/1827031645 wait complete. 2023-08-25T09:38:34.890 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.877+0000 7f9048d34640 1 Processor -- start 2023-08-25T09:38:34.891 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.878+0000 7f9048d34640 1 -- start start 2023-08-25T09:38:34.891 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.878+0000 7f9048d34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90440b19a0 0x7f90440c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.891 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9048d34640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f904415f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.891 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9048d34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f904415ca20 0x7f90440d2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:34.892 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9048d34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904415fd80 con 0x7f904415ca20 2023-08-25T09:38:34.892 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9048d34640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440ca7f0 con 0x7f90440cf9a0 2023-08-25T09:38:34.892 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9048d34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440c9f50 con 0x7f90440b19a0 2023-08-25T09:38:34.893 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f904415f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.893 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f90437fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90440b19a0 0x7f90440c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.893 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9042ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f904415f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44036/0 (socket says 172.21.15.155:44036) 2023-08-25T09:38:34.894 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9042ffd640 1 -- 172.21.15.155:0/3202991862 learned_addr learned my addr 172.21.15.155:0/3202991862 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:34.894 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f9043fff640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f904415ca20 0x7f90440d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:34.894 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f90437fe640 1 -- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 msgr2=0x7f904415f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f90437fe640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f904415f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f90437fe640 1 -- 172.21.15.155:0/3202991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f904415ca20 msgr2=0x7f90440d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f90437fe640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f904415ca20 0x7f90440d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.879+0000 7f90437fe640 1 -- 172.21.15.155:0/3202991862 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90380094d0 con 0x7f90440b19a0 2023-08-25T09:38:34.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f90437fe640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90440b19a0 0x7f90440c9970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f903400b3e0 tx=0x7f903400b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:34.896 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034011830 con 0x7f90440b19a0 2023-08-25T09:38:34.896 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9034011990 con 0x7f90440b19a0 2023-08-25T09:38:34.896 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034011d40 con 0x7f90440b19a0 2023-08-25T09:38:34.896 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440ca230 con 0x7f90440b19a0 2023-08-25T09:38:34.896 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440c6e50 con 0x7f90440b19a0 2023-08-25T09:38:34.897 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f903401a700 con 0x7f90440b19a0 2023-08-25T09:38:34.897 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.880+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9034019680 con 0x7f90440b19a0 2023-08-25T09:38:34.897 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.881+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90440c6cb0 con 0x7f90440b19a0 2023-08-25T09:38:34.897 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.883+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f9034039b00 con 0x7f90440b19a0 2023-08-25T09:38:34.911 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:34.909+0000 7f74a2c15040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T09:38:34.928 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:34.926+0000 7eff8dc16040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T09:38:34.963 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.961+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f90440c7040 con 0x7f90440b19a0 2023-08-25T09:38:34.969 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.967+0000 7f902bfff640 1 -- 172.21.15.155:0/3202991862 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9034028030 con 0x7f90440b19a0 2023-08-25T09:38:34.969 INFO:teuthology.orchestra.run.smithi155.stdout:4 2023-08-25T09:38:34.970 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90440b19a0 msgr2=0x7f90440c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:34.971 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90440b19a0 0x7f90440c9970 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f903400b3e0 tx=0x7f903400b8a0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.971 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 shutdown_connections 2023-08-25T09:38:34.971 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f904415ca20 0x7f90440d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.971 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f90440cf9a0 0x7f904415f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.971 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 --2- 172.21.15.155:0/3202991862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90440b19a0 0x7f90440c9970 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:34.972 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 >> 172.21.15.155:0/3202991862 conn(0x7f90440704a0 msgr2=0x7f90440b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:34.972 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 shutdown_connections 2023-08-25T09:38:34.972 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:34.969+0000 7f9048d34640 1 -- 172.21.15.155:0/3202991862 wait complete. 2023-08-25T09:38:34.976 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:34.975+0000 7f74a2c15040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T09:38:34.982 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:34.981+0000 7eff8dc16040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T09:38:34.988 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-08-25T09:38:35.042 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:35.041+0000 7f74a2c15040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T09:38:35.109 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:35.107+0000 7eff8dc16040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T09:38:35.169 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.167+0000 7fe7756e1640 1 Processor -- start 2023-08-25T09:38:35.169 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.167+0000 7fe7756e1640 1 -- start start 2023-08-25T09:38:35.170 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe7756e1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7700cf630 0x7fe7700cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe7756e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cbae0 0x7fe7700cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe7756e1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7700cb650 con 0x7fe77015cee0 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe7756e1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7700d0020 con 0x7fe7700cf630 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe7756e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cfa90 con 0x7fe7700cbae0 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe76f7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cbae0 0x7fe7700cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe76effd640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe77015cee0 0x7fe7700cd100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:59602/0 (socket says 172.21.15.155:59602) 2023-08-25T09:38:35.171 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe76effd640 1 -- 172.21.15.155:0/335786429 learned_addr learned my addr 172.21.15.155:0/335786429 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-08-25T09:38:35.172 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe767fff640 1 --2- 172.21.15.155:0/335786429 >> v2:172.21.15.155:3300/0 conn(0x7fe7700cf630 0x7fe7700cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.172 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.169+0000 7fe76f7fe640 1 -- 172.21.15.155:0/335786429 >> v2:172.21.15.155:3300/0 conn(0x7fe7700cf630 msgr2=0x7fe7700cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:35.172 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.170+0000 7fe76f7fe640 1 --2- 172.21.15.155:0/335786429 >> v2:172.21.15.155:3300/0 conn(0x7fe7700cf630 0x7fe7700cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.173 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.170+0000 7fe76f7fe640 1 -- 172.21.15.155:0/335786429 >> v1:172.21.15.155:6789/0 conn(0x7fe77015cee0 legacy=0x7fe7700cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.173 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.170+0000 7fe76f7fe640 1 -- 172.21.15.155:0/335786429 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700cfd40 con 0x7fe7700cbae0 2023-08-25T09:38:35.173 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.170+0000 7fe767fff640 1 --2- 172.21.15.155:0/335786429 >> v2:172.21.15.155:3300/0 conn(0x7fe7700cf630 0x7fe7700cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:35.174 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.171+0000 7fe76f7fe640 1 --2- 172.21.15.155:0/335786429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cbae0 0x7fe7700cbf50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cfdface58c30df server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:35.174 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.171+0000 7fe76e7fc640 1 -- 172.21.15.155:0/335786429 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76802cbc0 con 0x7fe7700cbae0 2023-08-25T09:38:35.174 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe76e7fc640 1 -- 172.21.15.155:0/335786429 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76802cd20 con 0x7fe7700cbae0 2023-08-25T09:38:35.174 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe76e7fc640 1 -- 172.21.15.155:0/335786429 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768034660 con 0x7fe7700cbae0 2023-08-25T09:38:35.174 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 -- 172.21.15.155:0/335786429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cbae0 msgr2=0x7fe7700cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.174 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 --2- 172.21.15.155:0/335786429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cbae0 0x7fe7700cbf50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe768009d90 tx=0x7fe76802be10 comp rx=0 tx=0).stop 2023-08-25T09:38:35.175 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 -- 172.21.15.155:0/335786429 shutdown_connections 2023-08-25T09:38:35.175 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 --2- 172.21.15.155:0/335786429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cbae0 0x7fe7700cbf50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.175 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 --2- 172.21.15.155:0/335786429 >> v2:172.21.15.155:3300/0 conn(0x7fe7700cf630 0x7fe7700cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.175 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 -- 172.21.15.155:0/335786429 >> 172.21.15.155:0/335786429 conn(0x7fe7700d0880 msgr2=0x7fe7700d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:35.175 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 -- 172.21.15.155:0/335786429 shutdown_connections 2023-08-25T09:38:35.176 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.172+0000 7fe7756e1640 1 -- 172.21.15.155:0/335786429 wait complete. 2023-08-25T09:38:35.176 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.174+0000 7fe7756e1640 1 Processor -- start 2023-08-25T09:38:35.176 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.174+0000 7fe7756e1640 1 -- start start 2023-08-25T09:38:35.177 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:35.175+0000 7f74a2c15040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T09:38:35.178 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe7756e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 0x7fe7700c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.179 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe7756e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cf630 0x7fe7701f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.179 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe7756e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77015cee0 0x7fe7701ef360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.179 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe7756e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c8ed0 con 0x7fe77015cee0 2023-08-25T09:38:35.179 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe7756e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c8250 con 0x7fe7700cbae0 2023-08-25T09:38:35.179 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe7756e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c8530 con 0x7fe7700cf630 2023-08-25T09:38:35.179 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe76effd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 0x7fe7700c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.180 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe76effd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 0x7fe7700c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44054/0 (socket says 172.21.15.155:44054) 2023-08-25T09:38:35.180 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe76effd640 1 -- 172.21.15.155:0/2876839365 learned_addr learned my addr 172.21.15.155:0/2876839365 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:35.180 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.175+0000 7fe767fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cf630 0x7fe7701f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.180 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76effd640 1 -- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cf630 msgr2=0x7fe7701f5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.180 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76f7fe640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77015cee0 0x7fe7701ef360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76effd640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cf630 0x7fe7701f5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76effd640 1 -- 172.21.15.155:0/2876839365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77015cee0 msgr2=0x7fe7701ef360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76effd640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77015cee0 0x7fe7701ef360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76effd640 1 -- 172.21.15.155:0/2876839365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe758009400 con 0x7fe7700cbae0 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe767fff640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cf630 0x7fe7701f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe76effd640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 0x7fe7700c7950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c430 tx=0x7fe76000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600074e0 con 0x7fe7700cbae0 2023-08-25T09:38:35.181 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760007640 con 0x7fe7700cbae0 2023-08-25T09:38:35.182 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7600059a0 con 0x7fe7700cbae0 2023-08-25T09:38:35.182 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.176+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe768009a70 con 0x7fe7700cbae0 2023-08-25T09:38:35.182 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.177+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7700c8af0 con 0x7fe7700cbae0 2023-08-25T09:38:35.182 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.177+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe760012400 con 0x7fe7700cbae0 2023-08-25T09:38:35.182 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.177+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fe760012c30 con 0x7fe7700cbae0 2023-08-25T09:38:35.183 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.178+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7700c9010 con 0x7fe7700cbae0 2023-08-25T09:38:35.183 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.180+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe7700c9010 con 0x7fe7700cbae0 2023-08-25T09:38:35.267 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:35.265+0000 7eff8dc16040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T09:38:35.268 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.266+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7fe77004e420 con 0x7fe7700cbae0 2023-08-25T09:38:35.274 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.272+0000 7fe7677fe640 1 -- 172.21.15.155:0/2876839365 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe760021060 con 0x7fe7700cbae0 2023-08-25T09:38:35.275 INFO:teuthology.orchestra.run.smithi155.stdout:5 2023-08-25T09:38:35.276 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 msgr2=0x7fe7700c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.276 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 0x7fe7700c7950 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c430 tx=0x7fe76000c8f0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.277 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 shutdown_connections 2023-08-25T09:38:35.277 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe77015cee0 0x7fe7701ef360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.277 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7700cf630 0x7fe7701f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.278 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 --2- 172.21.15.155:0/2876839365 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7700cbae0 0x7fe7700c7950 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.278 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.273+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 >> 172.21.15.155:0/2876839365 conn(0x7fe7700d0880 msgr2=0x7fe77006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:35.278 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.274+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 shutdown_connections 2023-08-25T09:38:35.278 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.274+0000 7fe7756e1640 1 -- 172.21.15.155:0/2876839365 wait complete. 2023-08-25T09:38:35.293 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-08-25T09:38:35.363 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:35.361+0000 7f74a2c15040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T09:38:35.376 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:35.374+0000 7eff8dc16040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T09:38:35.427 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:35.424+0000 7eff8dc16040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T09:38:35.480 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:35.478+0000 7f74a2c15040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T09:38:35.487 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.485+0000 7fb73bfd5640 1 Processor -- start 2023-08-25T09:38:35.488 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.485+0000 7fb73bfd5640 1 -- start start 2023-08-25T09:38:35.490 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb73bfd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb734069850 0x7fb734069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.491 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb73bfd5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7340d1f50 0x7fb7340d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.491 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb73bfd5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340d1ac0 con 0x7fb73415cef0 2023-08-25T09:38:35.491 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb73bfd5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340b8a10 con 0x7fb734069850 2023-08-25T09:38:35.491 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb73bfd5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340b2040 con 0x7fb7340d1f50 2023-08-25T09:38:35.491 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb73a572640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7340d1f50 0x7fb7340d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb739570640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb73415cef0 0x7fb7340b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:59604/0 (socket says 172.21.15.155:59604) 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.487+0000 7fb739570640 1 -- 172.21.15.155:0/1418723212 learned_addr learned my addr 172.21.15.155:0/1418723212 (peer_addr_for_me v1:172.21.15.155:0/0) 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.488+0000 7fb739d71640 1 --2- 172.21.15.155:0/1418723212 >> v2:172.21.15.70:3300/0 conn(0x7fb734069850 0x7fb734069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.488+0000 7fb73a572640 1 -- 172.21.15.155:0/1418723212 >> v2:172.21.15.70:3300/0 conn(0x7fb734069850 msgr2=0x7fb734069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.488+0000 7fb73a572640 1 --2- 172.21.15.155:0/1418723212 >> v2:172.21.15.70:3300/0 conn(0x7fb734069850 0x7fb734069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.488+0000 7fb738d6f640 1 -- 172.21.15.155:0/1418723212 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028098684 0 0) 0x7fb7340d1ac0 con 0x7fb73415cef0 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.488+0000 7fb73a572640 1 -- 172.21.15.155:0/1418723212 >> v1:172.21.15.155:6789/0 conn(0x7fb73415cef0 legacy=0x7fb7340b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.492 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.488+0000 7fb73a572640 1 -- 172.21.15.155:0/1418723212 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73406a170 con 0x7fb7340d1f50 2023-08-25T09:38:35.494 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.489+0000 7fb73a572640 1 --2- 172.21.15.155:0/1418723212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7340d1f50 0x7fb7340d23e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009d90 tx=0x7fb72c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9acc0b0328a0e73 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:35.494 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb738d6f640 1 -- 172.21.15.155:0/1418723212 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c02cc50 con 0x7fb7340d1f50 2023-08-25T09:38:35.494 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb738d6f640 1 -- 172.21.15.155:0/1418723212 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c02cdb0 con 0x7fb7340d1f50 2023-08-25T09:38:35.494 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb738d6f640 1 -- 172.21.15.155:0/1418723212 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c034660 con 0x7fb7340d1f50 2023-08-25T09:38:35.494 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb73bfd5640 1 -- 172.21.15.155:0/1418723212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7340d1f50 msgr2=0x7fb7340d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.494 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/1418723212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7340d1f50 0x7fb7340d23e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009d90 tx=0x7fb72c02bea0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb73bfd5640 1 -- 172.21.15.155:0/1418723212 shutdown_connections 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/1418723212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7340d1f50 0x7fb7340d23e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/1418723212 >> v2:172.21.15.70:3300/0 conn(0x7fb734069850 0x7fb734069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.490+0000 7fb73bfd5640 1 -- 172.21.15.155:0/1418723212 >> 172.21.15.155:0/1418723212 conn(0x7fb734070290 msgr2=0x7fb7340706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.491+0000 7fb73bfd5640 1 -- 172.21.15.155:0/1418723212 shutdown_connections 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.491+0000 7fb73bfd5640 1 -- 172.21.15.155:0/1418723212 wait complete. 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.492+0000 7fb73bfd5640 1 Processor -- start 2023-08-25T09:38:35.495 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.492+0000 7fb73bfd5640 1 -- start start 2023-08-25T09:38:35.497 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73bfd5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 0x7fb734159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.497 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73bfd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340d1f50 0x7fb7340c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.497 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73bfd5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb73415cef0 0x7fb7340c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73bfd5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340b51c0 con 0x7fb7340d1f50 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73bfd5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340ca640 con 0x7fb734069850 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73bfd5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340ca920 con 0x7fb73415cef0 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb739d71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 0x7fb734159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb73a572640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb73415cef0 0x7fb7340c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb739d71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 0x7fb734159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44068/0 (socket says 172.21.15.155:44068) 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.493+0000 7fb739d71640 1 -- 172.21.15.155:0/2829210490 learned_addr learned my addr 172.21.15.155:0/2829210490 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:35.498 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb739570640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340d1f50 0x7fb7340c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb73a572640 1 -- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 msgr2=0x7fb734159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.504 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb73a572640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 0x7fb734159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb73a572640 1 -- 172.21.15.155:0/2829210490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340d1f50 msgr2=0x7fb7340c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb73a572640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340d1f50 0x7fb7340c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb73a572640 1 -- 172.21.15.155:0/2829210490 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009a70 con 0x7fb73415cef0 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb739d71640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 0x7fb734159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb73a572640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb73415cef0 0x7fb7340c9fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb72c033000 tx=0x7fb72c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c040650 con 0x7fb73415cef0 2023-08-25T09:38:35.505 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c0407b0 con 0x7fb73415cef0 2023-08-25T09:38:35.506 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.494+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c040b60 con 0x7fb73415cef0 2023-08-25T09:38:35.506 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.495+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340cac00 con 0x7fb73415cef0 2023-08-25T09:38:35.506 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.495+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7340b2420 con 0x7fb73415cef0 2023-08-25T09:38:35.506 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.495+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb72c049400 con 0x7fb73415cef0 2023-08-25T09:38:35.506 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.495+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fb72c03e9f0 con 0x7fb73415cef0 2023-08-25T09:38:35.506 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.496+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7340b2270 con 0x7fb73415cef0 2023-08-25T09:38:35.507 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.499+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fb72c044020 con 0x7fb73415cef0 2023-08-25T09:38:35.535 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:35.533+0000 7f74a2c15040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T09:38:35.579 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.577+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7fb73404e420 con 0x7fb73415cef0 2023-08-25T09:38:35.586 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.584+0000 7fb72a7fc640 1 -- 172.21.15.155:0/2829210490 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb72c039070 con 0x7fb73415cef0 2023-08-25T09:38:35.587 INFO:teuthology.orchestra.run.smithi155.stdout:6 2023-08-25T09:38:35.588 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb73415cef0 msgr2=0x7fb7340c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.588 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb73415cef0 0x7fb7340c9fa0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb72c033000 tx=0x7fb72c004a40 comp rx=0 tx=0).stop 2023-08-25T09:38:35.588 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 shutdown_connections 2023-08-25T09:38:35.588 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb73415cef0 0x7fb7340c9fa0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.588 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7340d1f50 0x7fb7340c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 --2- 172.21.15.155:0/2829210490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb734069850 0x7fb734159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 >> 172.21.15.155:0/2829210490 conn(0x7fb734070290 msgr2=0x7fb73415fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:35.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 shutdown_connections 2023-08-25T09:38:35.589 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.585+0000 7fb73bfd5640 1 -- 172.21.15.155:0/2829210490 wait complete. 2023-08-25T09:38:35.594 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:35.592+0000 7eff8dc16040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T09:38:35.604 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-08-25T09:38:35.709 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:35.707+0000 7f74a2c15040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T09:38:35.791 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.790+0000 7ff3bf56a640 1 Processor -- start 2023-08-25T09:38:35.792 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.790+0000 7ff3bf56a640 1 -- start start 2023-08-25T09:38:35.793 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.791+0000 7ff3bf56a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3b8069850 0x7ff3b8069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.793 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.791+0000 7ff3bf56a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3b80d1f50 0x7ff3b80d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.793 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.791+0000 7ff3bf56a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80d1ac0 con 0x7ff3b815cef0 2023-08-25T09:38:35.794 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.791+0000 7ff3bf56a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b80b8a10 con 0x7ff3b8069850 2023-08-25T09:38:35.794 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.791+0000 7ff3bf56a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b80b2040 con 0x7ff3b80d1f50 2023-08-25T09:38:35.794 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.791+0000 7ff3bdb07640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3b80d1f50 0x7ff3b80d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.794 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bdb07640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3b80d1f50 0x7ff3b80d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.155:44080/0 (socket says 172.21.15.155:44080) 2023-08-25T09:38:35.794 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bdb07640 1 -- 172.21.15.155:0/3694139780 learned_addr learned my addr 172.21.15.155:0/3694139780 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:35.795 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bd306640 1 --2- 172.21.15.155:0/3694139780 >> v2:172.21.15.70:3300/0 conn(0x7ff3b8069850 0x7ff3b8069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.795 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bdb07640 1 -- 172.21.15.155:0/3694139780 >> v2:172.21.15.70:3300/0 conn(0x7ff3b8069850 msgr2=0x7ff3b8069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:35.795 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bdb07640 1 --2- 172.21.15.155:0/3694139780 >> v2:172.21.15.70:3300/0 conn(0x7ff3b8069850 0x7ff3b8069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.795 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bdb07640 1 -- 172.21.15.155:0/3694139780 >> v1:172.21.15.155:6789/0 conn(0x7ff3b815cef0 legacy=0x7ff3b80b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.796 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.792+0000 7ff3bdb07640 1 -- 172.21.15.155:0/3694139780 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b806a170 con 0x7ff3b80d1f50 2023-08-25T09:38:35.796 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.793+0000 7ff3bdb07640 1 --2- 172.21.15.155:0/3694139780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80d23e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80b1cb0 tx=0x7ff3b002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81796eca90f5f79 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:35.796 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.793+0000 7ff3affff640 1 -- 172.21.15.155:0/3694139780 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b002cb20 con 0x7ff3b80d1f50 2023-08-25T09:38:35.796 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.793+0000 7ff3affff640 1 -- 172.21.15.155:0/3694139780 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b002cc80 con 0x7ff3b80d1f50 2023-08-25T09:38:35.796 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.793+0000 7ff3affff640 1 -- 172.21.15.155:0/3694139780 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b00343f0 con 0x7ff3b80d1f50 2023-08-25T09:38:35.797 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 -- 172.21.15.155:0/3694139780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b80d1f50 msgr2=0x7ff3b80d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.797 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/3694139780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80d23e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80b1cb0 tx=0x7ff3b002bf80 comp rx=0 tx=0).stop 2023-08-25T09:38:35.797 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 -- 172.21.15.155:0/3694139780 shutdown_connections 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/3694139780 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80d23e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/3694139780 >> v2:172.21.15.70:3300/0 conn(0x7ff3b8069850 0x7ff3b8069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 -- 172.21.15.155:0/3694139780 >> 172.21.15.155:0/3694139780 conn(0x7ff3b8070290 msgr2=0x7ff3b80706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 -- 172.21.15.155:0/3694139780 shutdown_connections 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.794+0000 7ff3bf56a640 1 -- 172.21.15.155:0/3694139780 wait complete. 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.796+0000 7ff3bf56a640 1 Processor -- start 2023-08-25T09:38:35.798 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.796+0000 7ff3bf56a640 1 -- start start 2023-08-25T09:38:35.803 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bf56a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 0x7ff3b81f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.803 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bf56a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.804 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bf56a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b815cef0 0x7ff3b80c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:35.804 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bf56a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81f2260 con 0x7ff3b80d1f50 2023-08-25T09:38:35.804 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bf56a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b814c6e0 con 0x7ff3b815cef0 2023-08-25T09:38:35.804 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bf56a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b814cf80 con 0x7ff3b8069850 2023-08-25T09:38:35.805 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bd306640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 0x7ff3b81f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.805 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bdb07640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b815cef0 0x7ff3b80c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.805 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bd306640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 0x7ff3b81f1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.155:47366/0 (socket says 172.21.15.155:47366) 2023-08-25T09:38:35.805 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bd306640 1 -- 172.21.15.155:0/1407775586 learned_addr learned my addr 172.21.15.155:0/1407775586 (peer_addr_for_me v2:172.21.15.155:0/0) 2023-08-25T09:38:35.805 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bcb05640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:35.806 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.797+0000 7ff3bdb07640 1 -- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 msgr2=0x7ff3b81f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.806 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bdb07640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 0x7ff3b81f1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.806 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bdb07640 1 -- 172.21.15.155:0/1407775586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b80d1f50 msgr2=0x7ff3b80c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.806 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bdb07640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.806 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bdb07640 1 -- 172.21.15.155:0/1407775586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009b30 con 0x7ff3b815cef0 2023-08-25T09:38:35.806 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bcb05640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bd306640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 0x7ff3b81f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bdb07640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b815cef0 0x7ff3b80c75a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0032000 tx=0x7ff3b0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0009510 con 0x7ff3b815cef0 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b003e450 con 0x7ff3b815cef0 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b003e800 con 0x7ff3b815cef0 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b80ca400 con 0x7ff3b815cef0 2023-08-25T09:38:35.807 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.798+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b814cc10 con 0x7ff3b815cef0 2023-08-25T09:38:35.808 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.799+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff3b003ec60 con 0x7ff3b815cef0 2023-08-25T09:38:35.808 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.799+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff3b0048400 con 0x7ff3b815cef0 2023-08-25T09:38:35.808 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.799+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b814d0f0 con 0x7ff3b815cef0 2023-08-25T09:38:35.809 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.803+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7ff3b0043020 con 0x7ff3b815cef0 2023-08-25T09:38:35.883 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.881+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7ff3b804e420 con 0x7ff3b815cef0 2023-08-25T09:38:35.890 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.888+0000 7ff3adffb640 1 -- 172.21.15.155:0/1407775586 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff3b0048950 con 0x7ff3b815cef0 2023-08-25T09:38:35.891 INFO:teuthology.orchestra.run.smithi155.stdout:7 2023-08-25T09:38:35.892 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b815cef0 msgr2=0x7ff3b80c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:35.893 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b815cef0 0x7ff3b80c75a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0032000 tx=0x7ff3b0004a40 comp rx=0 tx=0).stop 2023-08-25T09:38:35.893 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 shutdown_connections 2023-08-25T09:38:35.893 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b815cef0 0x7ff3b80c75a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.894 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b80d1f50 0x7ff3b80c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.894 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 --2- 172.21.15.155:0/1407775586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3b8069850 0x7ff3b81f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:35.894 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 >> 172.21.15.155:0/1407775586 conn(0x7ff3b8070290 msgr2=0x7ff3b815f310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:35.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 shutdown_connections 2023-08-25T09:38:35.895 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:38:35.889+0000 7ff3bf56a640 1 -- 172.21.15.155:0/1407775586 wait complete. 2023-08-25T09:38:35.907 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-25T09:38:35.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-08-25T09:38:35.918 INFO:tasks.ceph.osd.0:Started 2023-08-25T09:38:35.918 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-25T09:38:35.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-08-25T09:38:35.927 INFO:tasks.ceph.osd.1:Started 2023-08-25T09:38:35.928 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-25T09:38:35.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-08-25T09:38:35.934 INFO:tasks.ceph.osd.2:Started 2023-08-25T09:38:35.935 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-25T09:38:35.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-08-25T09:38:35.942 INFO:tasks.ceph.osd.3:Started 2023-08-25T09:38:35.943 INFO:tasks.ceph.osd.4:Restarting daemon 2023-08-25T09:38:35.943 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-08-25T09:38:35.952 INFO:tasks.ceph.osd.4:Started 2023-08-25T09:38:35.952 INFO:tasks.ceph.osd.5:Restarting daemon 2023-08-25T09:38:35.952 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-08-25T09:38:35.960 INFO:tasks.ceph.osd.5:Started 2023-08-25T09:38:35.961 INFO:tasks.ceph.osd.6:Restarting daemon 2023-08-25T09:38:35.961 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-08-25T09:38:35.967 INFO:tasks.ceph.osd.6:Started 2023-08-25T09:38:35.967 INFO:tasks.ceph.osd.7:Restarting daemon 2023-08-25T09:38:35.967 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-08-25T09:38:35.974 INFO:tasks.ceph.osd.7:Started 2023-08-25T09:38:35.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:36.105 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.104+0000 7eff8dc16040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T09:38:36.118 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-08-25T09:38:36.117+0000 7fa03d116500 -1 Falling back to public interface 2023-08-25T09:38:36.121 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-08-25T09:38:36.120+0000 7f618ba0c500 -1 Falling back to public interface 2023-08-25T09:38:36.129 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-08-25T09:38:36.127+0000 7f0c2ee2b500 -1 Falling back to public interface 2023-08-25T09:38:36.135 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-08-25T09:38:36.133+0000 7f21dee0e500 -1 Falling back to public interface 2023-08-25T09:38:36.141 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-08-25T09:38:36.139+0000 7f37b4b36500 -1 Falling back to public interface 2023-08-25T09:38:36.143 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-08-25T09:38:36.142+0000 7f618ba0c500 -1 osd.4 0 log_to_monitors true 2023-08-25T09:38:36.159 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-08-25T09:38:36.157+0000 7fa03d116500 -1 osd.1 0 log_to_monitors true 2023-08-25T09:38:36.161 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-08-25T09:38:36.160+0000 7f0c2ee2b500 -1 osd.0 0 log_to_monitors true 2023-08-25T09:38:36.162 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-08-25T09:38:36.161+0000 7f0ed929a500 -1 Falling back to public interface 2023-08-25T09:38:36.164 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-08-25T09:38:36.162+0000 7f7d2161f500 -1 Falling back to public interface 2023-08-25T09:38:36.164 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-08-25T09:38:36.162+0000 7f37b4b36500 -1 osd.7 0 log_to_monitors true 2023-08-25T09:38:36.169 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-08-25T09:38:36.167+0000 7f758dea9500 -1 Falling back to public interface 2023-08-25T09:38:36.178 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-08-25T09:38:36.176+0000 7f21dee0e500 -1 osd.2 0 log_to_monitors true 2023-08-25T09:38:36.186 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-08-25T09:38:36.185+0000 7f0ed929a500 -1 osd.6 0 log_to_monitors true 2023-08-25T09:38:36.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.187+0000 7f5b78ef3640 1 Processor -- start 2023-08-25T09:38:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.187+0000 7f5b78ef3640 1 -- start start 2023-08-25T09:38:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b78ef3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b7415cbd0 0x7f5b740cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b7259c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b7415cbd0 0x7f5b740cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b71d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5b741606c0 0x7f5b740cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38930/0 (socket says 172.21.15.70:38930) 2023-08-25T09:38:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b71d9b640 1 -- 172.21.15.70:0/3679875242 learned_addr learned my addr 172.21.15.70:0/3679875242 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7415cfb0 con 0x7f5b741606c0 2023-08-25T09:38:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b740cd4e0 con 0x7f5b740d2a00 2023-08-25T09:38:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.189+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b74160f10 con 0x7f5b7415cbd0 2023-08-25T09:38:36.192 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-08-25T09:38:36.190+0000 7f7d2161f500 -1 osd.5 0 log_to_monitors true 2023-08-25T09:38:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.190+0000 7f5b7159a640 1 -- 172.21.15.70:0/3679875242 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665154705 0 0) 0x7f5b7415cfb0 con 0x7f5b741606c0 2023-08-25T09:38:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.190+0000 7f5b7159a640 1 -- 172.21.15.70:0/3679875242 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b5c003520 con 0x7f5b741606c0 2023-08-25T09:38:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.190+0000 7f5b7259c640 1 -- 172.21.15.70:0/3679875242 >> v1:172.21.15.155:6789/0 conn(0x7f5b740d2a00 legacy=0x7f5b740b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.190+0000 7f5b7259c640 1 -- 172.21.15.70:0/3679875242 >> v1:172.21.15.70:6789/0 conn(0x7f5b741606c0 legacy=0x7f5b740cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.190+0000 7f5b7259c640 1 -- 172.21.15.70:0/3679875242 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b741611f0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.191+0000 7f5b7259c640 1 --2- 172.21.15.70:0/3679875242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cffb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b6402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef6fed797cffceb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b7159a640 1 -- 172.21.15.70:0/3679875242 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64030610 con 0x7f5b7415cbd0 2023-08-25T09:38:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b7159a640 1 -- 172.21.15.70:0/3679875242 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64030770 con 0x7f5b7415cbd0 2023-08-25T09:38:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b7159a640 1 -- 172.21.15.70:0/3679875242 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64030ae0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7415cbd0 msgr2=0x7f5b740cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b78ef3640 1 --2- 172.21.15.70:0/3679875242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cffb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5b640097a0 tx=0x7f5b6402b960 comp rx=0 tx=0).stop 2023-08-25T09:38:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 shutdown_connections 2023-08-25T09:38:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.192+0000 7f5b78ef3640 1 --2- 172.21.15.70:0/3679875242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cffb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.193+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 >> 172.21.15.70:0/3679875242 conn(0x7f5b74070260 msgr2=0x7f5b74070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.193+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 shutdown_connections 2023-08-25T09:38:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.193+0000 7f5b78ef3640 1 -- 172.21.15.70:0/3679875242 wait complete. 2023-08-25T09:38:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.194+0000 7f5b78ef3640 1 Processor -- start 2023-08-25T09:38:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.194+0000 7f5b78ef3640 1 -- start start 2023-08-25T09:38:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.195+0000 7f5b78ef3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b740d2a00 0x7f5b740b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b78ef3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b78ef3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b741606c0 0x7f5b741f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b78ef3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b741f5ab0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b78ef3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b74159750 con 0x7f5b740d2a00 2023-08-25T09:38:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b78ef3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b74159a30 con 0x7f5b741606c0 2023-08-25T09:38:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b7259c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b740d2a00 0x7f5b740b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b71d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b7259c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b740d2a00 0x7f5b740b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:49954/0 (socket says 172.21.15.70:49954) 2023-08-25T09:38:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b7259c640 1 -- 172.21.15.70:0/1293042647 learned_addr learned my addr 172.21.15.70:0/1293042647 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b72d9d640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b741606c0 0x7f5b741f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b71d9b640 1 -- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b741606c0 msgr2=0x7f5b741f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b71d9b640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b741606c0 0x7f5b741f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b71d9b640 1 -- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b740d2a00 msgr2=0x7f5b740b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b71d9b640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b740d2a00 0x7f5b740b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.196+0000 7f5b71d9b640 1 -- 172.21.15.70:0/1293042647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009400 con 0x7f5b7415cbd0 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b71d9b640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cb600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800c3c0 tx=0x7f5b68001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68005ce0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68005e40 con 0x7f5b7415cbd0 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680072d0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b74159d10 con 0x7f5b7415cbd0 2023-08-25T09:38:36.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5b68012400 con 0x7f5b7415cbd0 2023-08-25T09:38:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.197+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b74159370 con 0x7f5b7415cbd0 2023-08-25T09:38:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.198+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5b6801b450 con 0x7f5b7415cbd0 2023-08-25T09:38:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.199+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b741f5bf0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.201+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f5b741f5bf0 con 0x7f5b7415cbd0 2023-08-25T09:38:36.205 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-08-25T09:38:36.204+0000 7f758dea9500 -1 osd.3 0 log_to_monitors true 2023-08-25T09:38:36.238 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.237+0000 7f74a2c15040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T09:38:36.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.267+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b7404e420 con 0x7f5b7415cbd0 2023-08-25T09:38:36.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.267+0000 7f5b62ffd640 1 -- 172.21.15.70:0/1293042647 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5b68017070 con 0x7f5b7415cbd0 2023-08-25T09:38:36.269 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:36.269 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":10,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:35.884043+0000","last_up_change":"0.000000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:36.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.268+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b7415cbd0 msgr2=0x7f5b740cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:36.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.268+0000 7f5b78ef3640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cb600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800c3c0 tx=0x7f5b68001e70 comp rx=0 tx=0).stop 2023-08-25T09:38:36.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.268+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 shutdown_connections 2023-08-25T09:38:36.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.268+0000 7f5b78ef3640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b741606c0 0x7f5b741f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:36.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.268+0000 7f5b78ef3640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b7415cbd0 0x7f5b740cb600 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:36.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.268+0000 7f5b78ef3640 1 --2- 172.21.15.70:0/1293042647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b740d2a00 0x7f5b740b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:36.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.269+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 >> 172.21.15.70:0/1293042647 conn(0x7f5b74070260 msgr2=0x7f5b740b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:36.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.269+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 shutdown_connections 2023-08-25T09:38:36.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:36.269+0000 7f5b78ef3640 1 -- 172.21.15.70:0/1293042647 wait complete. 2023-08-25T09:38:36.285 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-25T09:38:36.286 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-25T09:38:36.369 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.367+0000 7eff8dc16040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T09:38:36.430 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.428+0000 7eff8dc16040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T09:38:36.460 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.458+0000 7f74a2c15040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T09:38:36.484 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.483+0000 7eff8dc16040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T09:38:36.521 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.519+0000 7f74a2c15040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T09:38:36.545 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.543+0000 7eff8dc16040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T09:38:36.574 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.573+0000 7f74a2c15040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T09:38:36.634 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.632+0000 7f74a2c15040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T09:38:36.843 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.842+0000 7eff8dc16040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T09:38:36.905 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.903+0000 7eff8dc16040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T09:38:36.933 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.931+0000 7f74a2c15040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T09:38:36.960 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:36.958+0000 7eff8dc16040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T09:38:36.993 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:36.991+0000 7f74a2c15040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T09:38:37.057 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:37.055+0000 7f74a2c15040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T09:38:37.091 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:37.122 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:37.120+0000 7eff8dc16040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T09:38:37.175 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:37.173+0000 7eff8dc16040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T09:38:37.220 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:37.218+0000 7f74a2c15040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T09:38:37.238 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:37.236+0000 7eff8dc16040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T09:38:37.260 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.259+0000 7f6acaa45640 1 Processor -- start 2023-08-25T09:38:37.261 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.259+0000 7f6acaa45640 1 -- start start 2023-08-25T09:38:37.263 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6acaa45640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac40c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:37.263 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6acaa45640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6ac40b1720 0x7f6ac4072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:37.263 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6acaa45640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ac406d170 con 0x7f6ac40b1e00 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6acaa45640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ac406a060 con 0x7f6ac40b1720 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6acaa45640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4069850 con 0x7f6ac415cb80 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6aca244640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6ac40b1e00 0x7f6ac40cab80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38934/0 (socket says 172.21.15.70:38934) 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6aca244640 1 -- 172.21.15.70:0/183186027 learned_addr learned my addr 172.21.15.70:0/183186027 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6ac9242640 1 --2- 172.21.15.70:0/183186027 >> v2:172.21.15.155:3300/0 conn(0x7f6ac40b1720 0x7f6ac4072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.261+0000 7f6ac9a43640 1 --2- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac40c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:37.264 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.262+0000 7f6ac8a41640 1 -- 172.21.15.70:0/183186027 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087877058 0 0) 0x7f6ac406d170 con 0x7f6ac40b1e00 2023-08-25T09:38:37.265 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.262+0000 7f6ac8a41640 1 -- 172.21.15.70:0/183186027 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab0003520 con 0x7f6ac40b1e00 2023-08-25T09:38:37.265 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.262+0000 7f6ac9242640 1 -- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 msgr2=0x7f6ac40c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:37.265 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.262+0000 7f6ac9242640 1 --2- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac40c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.265 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.262+0000 7f6ac9242640 1 -- 172.21.15.70:0/183186027 >> v1:172.21.15.70:6789/0 conn(0x7f6ac40b1e00 legacy=0x7f6ac40cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:37.265 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.262+0000 7f6ac9242640 1 -- 172.21.15.70:0/183186027 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac40c8b10 con 0x7f6ac40b1720 2023-08-25T09:38:37.267 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.263+0000 7f6ac9242640 1 --2- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1720 0x7f6ac4072820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40097a0 tx=0x7f6ab402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b570f6e54115552 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:37.267 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.263+0000 7f6ac8a41640 1 -- 172.21.15.70:0/183186027 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4031830 con 0x7f6ac40b1720 2023-08-25T09:38:37.267 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.263+0000 7f6ac8a41640 1 -- 172.21.15.70:0/183186027 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab4031990 con 0x7f6ac40b1720 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.263+0000 7f6ac8a41640 1 -- 172.21.15.70:0/183186027 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4031d00 con 0x7f6ac40b1720 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 -- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1720 msgr2=0x7f6ac4072820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 --2- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1720 0x7f6ac4072820 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40097a0 tx=0x7f6ab402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 -- 172.21.15.70:0/183186027 shutdown_connections 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 --2- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1720 0x7f6ac4072820 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 --2- 172.21.15.70:0/183186027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac40c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.268 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 -- 172.21.15.70:0/183186027 >> 172.21.15.70:0/183186027 conn(0x7f6ac40cc0a0 msgr2=0x7f6ac40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:37.269 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 -- 172.21.15.70:0/183186027 shutdown_connections 2023-08-25T09:38:37.269 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.264+0000 7f6acaa45640 1 -- 172.21.15.70:0/183186027 wait complete. 2023-08-25T09:38:37.271 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.266+0000 7f6acaa45640 1 Processor -- start 2023-08-25T09:38:37.271 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.266+0000 7f6acaa45640 1 -- start start 2023-08-25T09:38:37.271 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6acaa45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 0x7f6ac4153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:37.272 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6acaa45640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1e00 0x7f6ac40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:37.272 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6acaa45640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac406bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:37.272 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6acaa45640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac4154280 con 0x7f6ac40b1720 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6acaa45640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac415af00 con 0x7f6ac40b1e00 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6acaa45640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac41e62d0 con 0x7f6ac415cb80 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6ac9a43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 0x7f6ac4153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6ac9a43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 0x7f6ac4153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43100/0 (socket says 172.21.15.70:43100) 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6ac9a43640 1 -- 172.21.15.70:0/852357383 learned_addr learned my addr 172.21.15.70:0/852357383 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.267+0000 7f6ac9242640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1e00 0x7f6ac40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:37.273 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6aca244640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac406bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6ac9a43640 1 -- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 msgr2=0x7f6ac406bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6ac9a43640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac406bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6ac9a43640 1 -- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1e00 msgr2=0x7f6ac40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6ac9a43640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1e00 0x7f6ac40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6ac9a43640 1 -- 172.21.15.70:0/852357383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab4009400 con 0x7f6ac40b1720 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6ac9a43640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 0x7f6ac4153d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9e0 tx=0x7f6ac000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0009670 con 0x7f6ac40b1720 2023-08-25T09:38:37.274 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac000c9d0 con 0x7f6ac40b1720 2023-08-25T09:38:37.275 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac000cd80 con 0x7f6ac40b1720 2023-08-25T09:38:37.275 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ac41e7130 con 0x7f6ac40b1720 2023-08-25T09:38:37.275 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.268+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac41e6800 con 0x7f6ac40b1720 2023-08-25T09:38:37.275 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.269+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6ac0005c90 con 0x7f6ac40b1720 2023-08-25T09:38:37.275 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.269+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7f6ac0013c30 con 0x7f6ac40b1720 2023-08-25T09:38:37.275 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.269+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ac4066fb0 con 0x7f6ac40b1720 2023-08-25T09:38:37.276 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:37.269+0000 7f74a2c15040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T09:38:37.276 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.272+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6ac0021060 con 0x7f6ac40b1720 2023-08-25T09:38:37.294 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:37.292+0000 7eff8dc16040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T09:38:37.306 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.304+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mgrmap(e 2) v1 ==== 44122+0+0 (secure 0 0 0) 0x7f6ac0006000 con 0x7f6ac40b1720 2023-08-25T09:38:37.335 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:37.333+0000 7f74a2c15040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T09:38:37.349 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.347+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ac406b130 con 0x7f6ac40b1720 2023-08-25T09:38:37.350 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.349+0000 7f6aba7fc640 1 -- 172.21.15.70:0/852357383 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6ac0017020 con 0x7f6ac40b1720 2023-08-25T09:38:37.350 INFO:teuthology.misc.health.smithi070.stdout: 2023-08-25T09:38:37.351 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":11,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:36.886525+0000","last_up_change":"0.000000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:37.351 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 msgr2=0x7f6ac4153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:37.352 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 0x7f6ac4153d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e9e0 tx=0x7f6ac000eea0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.352 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 shutdown_connections 2023-08-25T09:38:37.352 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ac415cb80 0x7f6ac406bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.352 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ac40b1e00 0x7f6ac40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.352 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 --2- 172.21.15.70:0/852357383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ac40b1720 0x7f6ac4153d40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:37.352 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 >> 172.21.15.70:0/852357383 conn(0x7f6ac40cc0a0 msgr2=0x7f6ac415bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:37.353 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 shutdown_connections 2023-08-25T09:38:37.353 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:37.350+0000 7f6acaa45640 1 -- 172.21.15.70:0/852357383 wait complete. 2023-08-25T09:38:37.367 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-25T09:38:37.391 INFO:tasks.ceph.mgr.x.smithi155.stderr:2023-08-25T09:38:37.389+0000 7f74a2c15040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T09:38:37.952 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-08-25T09:38:37.950+0000 7f7589e35640 -1 osd.3 0 waiting for initial osdmap 2023-08-25T09:38:37.955 INFO:tasks.ceph.osd.3.smithi070.stderr:2023-08-25T09:38:37.953+0000 7f7583469640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.961 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-08-25T09:38:37.960+0000 7fa0398b8640 -1 osd.1 0 waiting for initial osdmap 2023-08-25T09:38:37.963 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-08-25T09:38:37.962+0000 7f0ed5226640 -1 osd.6 0 waiting for initial osdmap 2023-08-25T09:38:37.964 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-08-25T09:38:37.962+0000 7f37b0ac2640 -1 osd.7 0 waiting for initial osdmap 2023-08-25T09:38:37.965 INFO:tasks.ceph.osd.1.smithi070.stderr:2023-08-25T09:38:37.963+0000 7fa0326d6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.965 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-08-25T09:38:37.964+0000 7f7d1cf5f640 -1 osd.5 0 waiting for initial osdmap 2023-08-25T09:38:37.966 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-08-25T09:38:37.964+0000 7f6187447640 -1 osd.4 0 waiting for initial osdmap 2023-08-25T09:38:37.968 INFO:tasks.ceph.osd.6.smithi155.stderr:2023-08-25T09:38:37.966+0000 7f0ece85a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.969 INFO:tasks.ceph.osd.5.smithi155.stderr:2023-08-25T09:38:37.968+0000 7f7d16593640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.970 INFO:tasks.ceph.osd.4.smithi155.stderr:2023-08-25T09:38:37.968+0000 7f6180a7b640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.970 INFO:tasks.ceph.osd.7.smithi155.stderr:2023-08-25T09:38:37.968+0000 7f37aa0f6640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.970 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-08-25T09:38:37.969+0000 7f21da7dc640 -1 osd.2 0 waiting for initial osdmap 2023-08-25T09:38:37.971 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-08-25T09:38:37.969+0000 7f0c2a307640 -1 osd.0 0 waiting for initial osdmap 2023-08-25T09:38:37.973 INFO:tasks.ceph.osd.0.smithi070.stderr:2023-08-25T09:38:37.972+0000 7f0c2393b640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:37.973 INFO:tasks.ceph.osd.2.smithi070.stderr:2023-08-25T09:38:37.972+0000 7f21d3e10640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T09:38:38.348 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:38.347+0000 7eff2f775640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-25T09:38:38.400 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:38:38.349+0000 7eff2f775640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-08-25T09:38:42.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:38:42.642 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:38:44.174 DEBUG:teuthology.orchestra.run.smithi070:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:44.359 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.357+0000 7f021a742640 1 Processor -- start 2023-08-25T09:38:44.359 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.358+0000 7f021a742640 1 -- start start 2023-08-25T09:38:44.361 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.359+0000 7f021a742640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f021415e830 0x7f021406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.361 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.359+0000 7f021a742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 0x7f021415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.361 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.359+0000 7f021a742640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02140b1b60 con 0x7f021415c6d0 2023-08-25T09:38:44.362 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.359+0000 7f021a742640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f021406bd80 con 0x7f021415e830 2023-08-25T09:38:44.362 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.359+0000 7f021a742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021406b7c0 con 0x7f021406b2b0 2023-08-25T09:38:44.362 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f0218cdf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f021415c6d0 0x7f02140b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52416/0 (socket says 172.21.15.70:52416) 2023-08-25T09:38:44.362 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f0218cdf640 1 -- 172.21.15.70:0/2831287165 learned_addr learned my addr 172.21.15.70:0/2831287165 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:44.365 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f0213fff640 1 --2- 172.21.15.70:0/2831287165 >> v2:172.21.15.155:3300/0 conn(0x7f021415e830 0x7f021406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.365 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f020bfff640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 0x7f021415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.365 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195981460 0 0) 0x7f02140b1b60 con 0x7f021415c6d0 2023-08-25T09:38:44.365 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0200003520 con 0x7f021415c6d0 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168117841 0 0) 0x7f0200003520 con 0x7f021415c6d0 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.360+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 msgr2=0x7f021415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 0x7f021415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 >> v2:172.21.15.155:3300/0 conn(0x7f021415e830 msgr2=0x7f021406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 --2- 172.21.15.70:0/2831287165 >> v2:172.21.15.155:3300/0 conn(0x7f021415e830 0x7f021406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021406baa0 con 0x7f021415c6d0 2023-08-25T09:38:44.366 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f020c0019f0 con 0x7f021415c6d0 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 >> v1:172.21.15.70:6789/0 conn(0x7f021415c6d0 legacy=0x7f02140b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 0x7f0200003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200003ef0 0x7f02000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200004d00 0x7f0200005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021406bd80 con 0x7f0200003ef0 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021406b7c0 con 0x7f0200004d00 2023-08-25T09:38:44.367 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020000b980 con 0x7f021406b2b0 2023-08-25T09:38:44.368 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f020bfff640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200003ef0 0x7f02000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.368 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f0213fff640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 0x7f0200003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.369 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f020bfff640 1 -- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 msgr2=0x7f0200003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.369 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f020bfff640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f021406b2b0 0x7f0200003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.369 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f020bfff640 1 -- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200004d00 msgr2=0x7f0200005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.370 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f020bfff640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200004d00 0x7f0200005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.370 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.361+0000 7f020bfff640 1 -- 172.21.15.70:0/2831287165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200003520 con 0x7f0200003ef0 2023-08-25T09:38:44.370 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.362+0000 7f020bfff640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200003ef0 0x7f02000047c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009a90 tx=0x7f02000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509e87a76d3f4636 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:44.371 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02c6b0 con 0x7f0200003ef0 2023-08-25T09:38:44.371 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc02c810 con 0x7f0200003ef0 2023-08-25T09:38:44.371 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f02137fe640 1 -- 172.21.15.70:0/2831287165 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02cba0 con 0x7f0200003ef0 2023-08-25T09:38:44.371 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f021a742640 1 -- 172.21.15.70:0/2831287165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200003ef0 msgr2=0x7f02000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.372 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f021a742640 1 --2- 172.21.15.70:0/2831287165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0200003ef0 0x7f02000047c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009a90 tx=0x7f02000057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.372 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f0218cdf640 1 -- 172.21.15.70:0/2831287165 reap_dead start 2023-08-25T09:38:44.372 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f021a742640 1 -- 172.21.15.70:0/2831287165 shutdown_connections 2023-08-25T09:38:44.372 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f021a742640 1 -- 172.21.15.70:0/2831287165 >> 172.21.15.70:0/2831287165 conn(0x7f0214069850 msgr2=0x7f0214069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:44.372 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f021a742640 1 -- 172.21.15.70:0/2831287165 shutdown_connections 2023-08-25T09:38:44.373 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.363+0000 7f021a742640 1 -- 172.21.15.70:0/2831287165 wait complete. 2023-08-25T09:38:44.375 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.365+0000 7f021a742640 1 Processor -- start 2023-08-25T09:38:44.375 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.365+0000 7f021a742640 1 -- start start 2023-08-25T09:38:44.375 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f021a742640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200003ef0 0x7f02141468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.375 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f021a742640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 0x7f0214159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.375 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f021a742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0214159b90 0x7f02140c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f021a742640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214147c90 con 0x7f0214146df0 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f021a742640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021415a020 con 0x7f0200003ef0 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f021a742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02141472f0 con 0x7f0214159b90 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.366+0000 7f020bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 0x7f0214159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 0x7f0214159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36110/0 (socket says 172.21.15.70:36110) 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 -- 172.21.15.70:0/2105465948 learned_addr learned my addr 172.21.15.70:0/2105465948 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:44.376 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f0218cdf640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0214159b90 0x7f02140c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f0213fff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200003ef0 0x7f02141468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 -- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0214159b90 msgr2=0x7f02140c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0214159b90 0x7f02140c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 -- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200003ef0 msgr2=0x7f02141468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200003ef0 0x7f02141468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 -- 172.21.15.70:0/2105465948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02141475d0 con 0x7f0214146df0 2023-08-25T09:38:44.377 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f0213fff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200003ef0 0x7f02141468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:44.378 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.367+0000 7f020bfff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 0x7f0214159650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01fc02c2b0 tx=0x7f01fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:44.378 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.368+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc003810 con 0x7f0214146df0 2023-08-25T09:38:44.378 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.368+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc003970 con 0x7f0214146df0 2023-08-25T09:38:44.378 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.368+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02141478b0 con 0x7f0214146df0 2023-08-25T09:38:44.378 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.368+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc003ce0 con 0x7f0214146df0 2023-08-25T09:38:44.378 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.368+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02140ca3e0 con 0x7f0214146df0 2023-08-25T09:38:44.379 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.369+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021404e420 con 0x7f0214146df0 2023-08-25T09:38:44.379 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.370+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f01fc047450 con 0x7f0214146df0 2023-08-25T09:38:44.380 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.371+0000 7f02117fa640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01f406a050 0x7f01f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.380 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.371+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f01fc0aa350 con 0x7f0214146df0 2023-08-25T09:38:44.380 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.371+0000 7f0213fff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01f406a050 0x7f01f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.380 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.371+0000 7f0213fff640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01f406a050 0x7f01f406c510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f02141580c0 tx=0x7f0204008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:44.381 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.376+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f01fc076020 con 0x7f0214146df0 2023-08-25T09:38:44.566 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.564+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02140b2d30 con 0x7f0214146df0 2023-08-25T09:38:44.568 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.567+0000 7f02117fa640 1 -- 172.21.15.70:0/2105465948 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12838 (secure 0 0 0) 0x7f01fc0044e0 con 0x7f0214146df0 2023-08-25T09:38:44.569 INFO:teuthology.misc.health.smithi070.stdout: 2023-08-25T09:38:44.569 INFO:teuthology.misc.health.smithi070.stdout:{"epoch":16,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:41.958804+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:44.571 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01f406a050 msgr2=0x7f01f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.571 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01f406a050 0x7f01f406c510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f02141580c0 tx=0x7f0204008040 comp rx=0 tx=0).stop 2023-08-25T09:38:44.571 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 msgr2=0x7f0214159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.571 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 0x7f0214159650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f01fc02c2b0 tx=0x7f01fc004730 comp rx=0 tx=0).stop 2023-08-25T09:38:44.571 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 shutdown_connections 2023-08-25T09:38:44.572 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01f406a050 0x7f01f406c510 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.572 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0214159b90 0x7f02140c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.572 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0214146df0 0x7f0214159650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.572 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 --2- 172.21.15.70:0/2105465948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0200003ef0 0x7f02141468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.572 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.569+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 >> 172.21.15.70:0/2105465948 conn(0x7f0214069850 msgr2=0x7f021406f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:44.572 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.570+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 shutdown_connections 2023-08-25T09:38:44.573 INFO:teuthology.misc.health.smithi070.stderr:2023-08-25T09:38:44.570+0000 7f021a742640 1 -- 172.21.15.70:0/2105465948 wait complete. 2023-08-25T09:38:44.585 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-08-25T09:38:44.586 INFO:tasks.ceph:Creating RBD pool 2023-08-25T09:38:44.586 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-25T09:38:44.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.789+0000 7fc19b95d640 1 Processor -- start 2023-08-25T09:38:44.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.789+0000 7fc19b95d640 1 -- start start 2023-08-25T09:38:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.790+0000 7fc19b95d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc194069850 0x7fc19415e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.790+0000 7fc19b95d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 0x7fc194069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc19b95d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1940ac880 con 0x7fc194160be0 2023-08-25T09:38:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc19b95d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1940ad400 con 0x7fc194069850 2023-08-25T09:38:44.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc19b95d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940ad890 con 0x7fc1940c9620 2023-08-25T09:38:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc199efa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc194160be0 0x7fc194161040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52428/0 (socket says 172.21.15.70:52428) 2023-08-25T09:38:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc199efa640 1 -- 172.21.15.70:0/671752196 learned_addr learned my addr 172.21.15.70:0/671752196 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc1996f9640 1 --2- 172.21.15.70:0/671752196 >> v2:172.21.15.155:3300/0 conn(0x7fc194069850 0x7fc19415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc198ef8640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 0x7fc194069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.791+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76566398 0 0) 0x7fc1940ac880 con 0x7fc194160be0 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003520 con 0x7fc194160be0 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414578511 0 0) 0x7fc180003520 con 0x7fc194160be0 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 msgr2=0x7fc194069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 0x7fc194069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 >> v2:172.21.15.155:3300/0 conn(0x7fc194069850 msgr2=0x7fc19415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 --2- 172.21.15.70:0/671752196 >> v2:172.21.15.155:3300/0 conn(0x7fc194069850 0x7fc19415e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940acad0 con 0x7fc194160be0 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc18c0019f0 con 0x7fc194160be0 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 >> v1:172.21.15.70:6789/0 conn(0x7fc194160be0 legacy=0x7fc194161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc180003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 0x7fc1800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc180004d00 0x7fc180005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940ad400 con 0x7fc180003970 2023-08-25T09:38:44.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940ad890 con 0x7fc180004d00 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000b980 con 0x7fc1940c9620 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.792+0000 7fc1996f9640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc180003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc199efa640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc180004d00 0x7fc180005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc198ef8640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 0x7fc1800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc1996f9640 1 -- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 msgr2=0x7fc1800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc1996f9640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c9620 0x7fc1800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc1996f9640 1 -- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc180004d00 msgr2=0x7fc180005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc1996f9640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc180004d00 0x7fc180005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.793+0000 7fc1996f9640 1 -- 172.21.15.70:0/671752196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180003520 con 0x7fc180003970 2023-08-25T09:38:44.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.794+0000 7fc1996f9640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc180003d90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc190009ac0 tx=0x7fc1800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b8f96f7b1edddc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:44.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.794+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19002c710 con 0x7fc180003970 2023-08-25T09:38:44.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19002c870 con 0x7fc180003970 2023-08-25T09:38:44.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc18bfff640 1 -- 172.21.15.70:0/671752196 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19002cc00 con 0x7fc180003970 2023-08-25T09:38:44.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc19b95d640 1 -- 172.21.15.70:0/671752196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 msgr2=0x7fc180003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc19b95d640 1 --2- 172.21.15.70:0/671752196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc180003d90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc190009ac0 tx=0x7fc1800057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc199efa640 1 -- 172.21.15.70:0/671752196 reap_dead start 2023-08-25T09:38:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc19b95d640 1 -- 172.21.15.70:0/671752196 shutdown_connections 2023-08-25T09:38:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc19b95d640 1 -- 172.21.15.70:0/671752196 >> 172.21.15.70:0/671752196 conn(0x7fc1940cb0d0 msgr2=0x7fc1940cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc19b95d640 1 -- 172.21.15.70:0/671752196 shutdown_connections 2023-08-25T09:38:44.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.795+0000 7fc19b95d640 1 -- 172.21.15.70:0/671752196 wait complete. 2023-08-25T09:38:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.797+0000 7fc19b95d640 1 Processor -- start 2023-08-25T09:38:44.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.797+0000 7fc19b95d640 1 -- start start 2023-08-25T09:38:44.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.798+0000 7fc19b95d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc194156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.798+0000 7fc19b95d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c48a0 0x7fc1940c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.798+0000 7fc19b95d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1940c24e0 0x7fc1940c2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.798+0000 7fc19b95d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940c69e0 con 0x7fc180003970 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.798+0000 7fc19b95d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940c80b0 con 0x7fc1940c24e0 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.798+0000 7fc19b95d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc194156f70 con 0x7fc1940c48a0 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc194156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc194156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36118/0 (socket says 172.21.15.70:36118) 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 -- 172.21.15.70:0/391148197 learned_addr learned my addr 172.21.15.70:0/391148197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc199efa640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1940c24e0 0x7fc1940c2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc198ef8640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c48a0 0x7fc1940c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 -- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c48a0 msgr2=0x7fc1940c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c48a0 0x7fc1940c1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 -- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1940c24e0 msgr2=0x7fc1940c2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:44.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1940c24e0 0x7fc1940c2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 -- 172.21.15.70:0/391148197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc194157250 con 0x7fc180003970 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.799+0000 7fc1996f9640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc194156600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc19002c2b0 tx=0x7fc190004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.800+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190003810 con 0x7fc180003970 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.800+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19002de90 con 0x7fc180003970 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.800+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc194157530 con 0x7fc180003970 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.800+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190003bb0 con 0x7fc180003970 2023-08-25T09:38:44.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.800+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc194156b40 con 0x7fc180003970 2023-08-25T09:38:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.801+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc194156cd0 con 0x7fc180003970 2023-08-25T09:38:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.802+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc190003d10 con 0x7fc180003970 2023-08-25T09:38:44.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.803+0000 7fc189ffb640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc17806a050 0x7fc17806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.803+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1900aa080 con 0x7fc180003970 2023-08-25T09:38:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.803+0000 7fc198ef8640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc17806a050 0x7fc17806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:44.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.804+0000 7fc198ef8640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc17806a050 0x7fc17806c510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc19415c970 tx=0x7fc184008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:44.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.807+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc190075d40 con 0x7fc180003970 2023-08-25T09:38:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:44.997+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fc194066fb0 con 0x7fc180003970 2023-08-25T09:38:46.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.007+0000 7fc189ffb640 1 -- 172.21.15.70:0/391148197 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fc190079360 con 0x7fc180003970 2023-08-25T09:38:46.009 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'rbd' created 2023-08-25T09:38:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.011+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc17806a050 msgr2=0x7fc17806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.011+0000 7fc19b95d640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc17806a050 0x7fc17806c510 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc19415c970 tx=0x7fc184008040 comp rx=0 tx=0).stop 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.011+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 msgr2=0x7fc194156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.011+0000 7fc19b95d640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc194156600 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc19002c2b0 tx=0x7fc190004730 comp rx=0 tx=0).stop 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 shutdown_connections 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc17806a050 0x7fc17806c510 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1940c24e0 0x7fc1940c2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1940c48a0 0x7fc1940c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 --2- 172.21.15.70:0/391148197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc180003970 0x7fc194156600 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 >> 172.21.15.70:0/391148197 conn(0x7fc1940cb0d0 msgr2=0x7fc19406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 shutdown_connections 2023-08-25T09:38:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.012+0000 7fc19b95d640 1 -- 172.21.15.70:0/391148197 wait complete. 2023-08-25T09:38:46.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-25T09:38:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.241+0000 7f570c014640 1 Processor -- start 2023-08-25T09:38:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.241+0000 7f570c014640 1 -- start start 2023-08-25T09:38:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f570c014640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57041608c0 0x7f5704069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f570c014640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f570406ae50 0x7f570406b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f570c014640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040cff30 con 0x7f570406a830 2023-08-25T09:38:46.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f570c014640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570406b840 con 0x7f570406ae50 2023-08-25T09:38:46.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f570c014640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57040ccbd0 con 0x7f57041608c0 2023-08-25T09:38:46.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f5709db0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f570406a830 0x7f57040d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52438/0 (socket says 172.21.15.70:52438) 2023-08-25T09:38:46.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.243+0000 7f5709db0640 1 -- 172.21.15.70:0/1933467367 learned_addr learned my addr 172.21.15.70:0/1933467367 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:46.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f57095af640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57041608c0 0x7f5704069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f570a5b1640 1 --2- 172.21.15.70:0/1933467367 >> v2:172.21.15.155:3300/0 conn(0x7f570406ae50 0x7f570406b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357904014 0 0) 0x7f57040cff30 con 0x7f570406a830 2023-08-25T09:38:46.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f0003520 con 0x7f570406a830 2023-08-25T09:38:46.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030318369 0 0) 0x7f56f0003520 con 0x7f570406a830 2023-08-25T09:38:46.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57041608c0 msgr2=0x7f5704069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:46.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57041608c0 0x7f5704069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 >> v2:172.21.15.155:3300/0 conn(0x7f570406ae50 msgr2=0x7f570406b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:46.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 --2- 172.21.15.70:0/1933467367 >> v2:172.21.15.155:3300/0 conn(0x7f570406ae50 0x7f570406b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570406ac10 con 0x7f570406a830 2023-08-25T09:38:46.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f57000017f0 con 0x7f570406a830 2023-08-25T09:38:46.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 >> v1:172.21.15.70:6789/0 conn(0x7f570406a830 legacy=0x7f57040d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:46.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f56f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56f00047c0 0x7f56f0004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56f0005140 0x7f56f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570406b840 con 0x7f56f0003970 2023-08-25T09:38:46.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040ccbd0 con 0x7f56f0005140 2023-08-25T09:38:46.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00045c0 con 0x7f56f00047c0 2023-08-25T09:38:46.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.244+0000 7f5709db0640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f56f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f5709db0640 1 -- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56f00047c0 msgr2=0x7f56f0004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f570a5b1640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56f0005140 0x7f56f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f57095af640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56f00047c0 0x7f56f0004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f5709db0640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f56f00047c0 0x7f56f0004c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f5709db0640 1 -- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56f0005140 msgr2=0x7f56f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f5709db0640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f56f0005140 0x7f56f000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.245+0000 7f5709db0640 1 -- 172.21.15.70:0/1933467367 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0003520 con 0x7f56f0003970 2023-08-25T09:38:46.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.246+0000 7f5709db0640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f56f0003d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f570000c1c0 tx=0x7f56f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d7b05c7b999c6d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:46.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.246+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570002cad0 con 0x7f56f0003970 2023-08-25T09:38:46.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.246+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570002cc30 con 0x7f56f0003970 2023-08-25T09:38:46.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.246+0000 7f5708dae640 1 -- 172.21.15.70:0/1933467367 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700007070 con 0x7f56f0003970 2023-08-25T09:38:46.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570c014640 1 -- 172.21.15.70:0/1933467367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 msgr2=0x7f56f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:46.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570c014640 1 --2- 172.21.15.70:0/1933467367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f56f0003d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f570000c1c0 tx=0x7f56f00057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570a5b1640 1 -- 172.21.15.70:0/1933467367 reap_dead start 2023-08-25T09:38:46.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570c014640 1 -- 172.21.15.70:0/1933467367 shutdown_connections 2023-08-25T09:38:46.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570c014640 1 -- 172.21.15.70:0/1933467367 >> 172.21.15.70:0/1933467367 conn(0x7f570406cf40 msgr2=0x7f570406d320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:46.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570c014640 1 -- 172.21.15.70:0/1933467367 shutdown_connections 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.247+0000 7f570c014640 1 -- 172.21.15.70:0/1933467367 wait complete. 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.249+0000 7f570c014640 1 Processor -- start 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.249+0000 7f570c014640 1 -- start start 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f570c014640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f57040c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f570c014640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57040b6410 0x7f570414de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f570c014640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57040c4460 0x7f57040c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f570c014640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570414e390 con 0x7f56f0003970 2023-08-25T09:38:46.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f570c014640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040b3ad0 con 0x7f57040b6410 2023-08-25T09:38:46.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f570c014640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57040b3550 con 0x7f57040c4460 2023-08-25T09:38:46.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f5709db0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f57040c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f5709db0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f57040c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36128/0 (socket says 172.21.15.70:36128) 2023-08-25T09:38:46.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.250+0000 7f5709db0640 1 -- 172.21.15.70:0/1087443413 learned_addr learned my addr 172.21.15.70:0/1087443413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:46.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f5709db0640 1 -- 172.21.15.70:0/1087443413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57040c4460 msgr2=0x7f57040c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:46.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f5709db0640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57040c4460 0x7f57040c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f5709db0640 1 -- 172.21.15.70:0/1087443413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57040b6410 msgr2=0x7f570414de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f5709db0640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57040b6410 0x7f570414de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f5709db0640 1 -- 172.21.15.70:0/1087443413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040b37f0 con 0x7f56f0003970 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f5709db0640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f57040c71a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f570002c470 tx=0x7f5700001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570003c040 con 0x7f56f0003970 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700005bb0 con 0x7f56f0003970 2023-08-25T09:38:46.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570002ca50 con 0x7f56f0003970 2023-08-25T09:38:46.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.251+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57040b5c10 con 0x7f56f0003970 2023-08-25T09:38:46.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.252+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570415c480 con 0x7f56f0003970 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.253+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570404e420 con 0x7f56f0003970 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.254+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5700032070 con 0x7f56f0003970 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.254+0000 7f56fa7fc640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f56e806a080 0x7f56e806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.255+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f57000a9230 con 0x7f56f0003970 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.255+0000 7f57095af640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f56e806a080 0x7f56e806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.255+0000 7f57095af640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f56e806a080 0x7f56e806c540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f57040cf470 tx=0x7f56f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:46.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.259+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5700074d30 con 0x7f56f0003970 2023-08-25T09:38:46.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:46.448+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5704070f70 con 0x7f56f0003970 2023-08-25T09:38:47.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.017+0000 7f56fa7fc640 1 -- 172.21.15.70:0/1087443413 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5700078350 con 0x7f56f0003970 2023-08-25T09:38:47.020 INFO:teuthology.orchestra.run.smithi070.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-25T09:38:47.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.021+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f56e806a080 msgr2=0x7f56e806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.021+0000 7f570c014640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f56e806a080 0x7f56e806c540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f57040cf470 tx=0x7f56f4008040 comp rx=0 tx=0).stop 2023-08-25T09:38:47.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.021+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 msgr2=0x7f57040c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.021+0000 7f570c014640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f57040c71a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f570002c470 tx=0x7f5700001b80 comp rx=0 tx=0).stop 2023-08-25T09:38:47.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 shutdown_connections 2023-08-25T09:38:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f56e806a080 0x7f56e806c540 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57040c4460 0x7f57040c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57040b6410 0x7f570414de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 --2- 172.21.15.70:0/1087443413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f56f0003970 0x7f57040c71a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 >> 172.21.15.70:0/1087443413 conn(0x7f570406cf40 msgr2=0x7f5704069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:47.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.022+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 shutdown_connections 2023-08-25T09:38:47.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.023+0000 7f570c014640 1 -- 172.21.15.70:0/1087443413 wait complete. 2023-08-25T09:38:47.044 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-25T09:38:47.045 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:38:47.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:38:47.055 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:38:47.055 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:38:47.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:38:47.063 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:38:47.064 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:38:47.064 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:38:47.073 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:38:47.073 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:38:47.073 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:38:47.081 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:38:47.082 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-08-25T09:38:47.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:47.213 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:38:47.217 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:38:47.239 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:38:47.253 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:38:47.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.274+0000 7f4682b55640 1 Processor -- start 2023-08-25T09:38:47.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.274+0000 7f4682b55640 1 -- start start 2023-08-25T09:38:47.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.275+0000 7f4682b55640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f467c15ec10 0x7f467c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.275+0000 7f4682b55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467c15c6d0 0x7f467c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.275+0000 7f4682b55640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f467c0b1b60 con 0x7f467c15e830 2023-08-25T09:38:47.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f4682b55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c06bd80 con 0x7f467c15c6d0 2023-08-25T09:38:47.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f4682b55640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467c06b7c0 con 0x7f467c15ec10 2023-08-25T09:38:47.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467c15c6d0 0x7f467c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f467c15c6d0 0x7f467c0b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36158/0 (socket says 172.21.15.70:36158) 2023-08-25T09:38:47.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 -- 172.21.15.70:0/1879990831 learned_addr learned my addr 172.21.15.70:0/1879990831 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:47.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f4673fff640 1 --2- 172.21.15.70:0/1879990831 >> v2:172.21.15.155:3300/0 conn(0x7f467c15ec10 0x7f467c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 -- 172.21.15.70:0/1879990831 >> v2:172.21.15.155:3300/0 conn(0x7f467c15ec10 msgr2=0x7f467c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:47.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 --2- 172.21.15.70:0/1879990831 >> v2:172.21.15.155:3300/0 conn(0x7f467c15ec10 0x7f467c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 -- 172.21.15.70:0/1879990831 >> v1:172.21.15.155:6789/0 conn(0x7f467c15e830 legacy=0x7f467c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:47.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f46810f2640 1 -- 172.21.15.70:0/1879990831 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c06baa0 con 0x7f467c15c6d0 2023-08-25T09:38:47.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.276+0000 7f4673fff640 1 --2- 172.21.15.70:0/1879990831 >> v2:172.21.15.155:3300/0 conn(0x7f467c15ec10 0x7f467c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.277+0000 7f46810f2640 1 --2- 172.21.15.70:0/1879990831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c0b1ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f467c0cf6f0 tx=0x7f467402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f22dfc478b90b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f46737fe640 1 -- 172.21.15.70:0/1879990831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674031870 con 0x7f467c15c6d0 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f46737fe640 1 -- 172.21.15.70:0/1879990831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46740319d0 con 0x7f467c15c6d0 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f46737fe640 1 -- 172.21.15.70:0/1879990831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674031d60 con 0x7f467c15c6d0 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 -- 172.21.15.70:0/1879990831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 msgr2=0x7f467c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 --2- 172.21.15.70:0/1879990831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c0b1ff0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f467c0cf6f0 tx=0x7f467402b910 comp rx=0 tx=0).stop 2023-08-25T09:38:47.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 -- 172.21.15.70:0/1879990831 shutdown_connections 2023-08-25T09:38:47.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 --2- 172.21.15.70:0/1879990831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c0b1ff0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 --2- 172.21.15.70:0/1879990831 >> v2:172.21.15.155:3300/0 conn(0x7f467c15ec10 0x7f467c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 -- 172.21.15.70:0/1879990831 >> 172.21.15.70:0/1879990831 conn(0x7f467c069850 msgr2=0x7f467c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:47.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 -- 172.21.15.70:0/1879990831 shutdown_connections 2023-08-25T09:38:47.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.278+0000 7f4682b55640 1 -- 172.21.15.70:0/1879990831 wait complete. 2023-08-25T09:38:47.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.280+0000 7f4682b55640 1 Processor -- start 2023-08-25T09:38:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.280+0000 7f4682b55640 1 -- start start 2023-08-25T09:38:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.281+0000 7f4682b55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.281+0000 7f4682b55640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f467c15e830 0x7f467c0c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.281+0000 7f4682b55640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f467c15ec10 0x7f467c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.281+0000 7f4682b55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c153db0 con 0x7f467c15c6d0 2023-08-25T09:38:47.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.281+0000 7f4682b55640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0c98e0 con 0x7f467c15e830 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.281+0000 7f4682b55640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467c0c7f10 con 0x7f467c15ec10 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c1f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36170/0 (socket says 172.21.15.70:36170) 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 -- 172.21.15.70:0/4151809557 learned_addr learned my addr 172.21.15.70:0/4151809557 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46810f2640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f467c15ec10 0x7f467c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f4673fff640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f467c15e830 0x7f467c0c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 -- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f467c15ec10 msgr2=0x7f467c0c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f467c15ec10 0x7f467c0c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 -- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f467c15e830 msgr2=0x7f467c0c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f467c15e830 0x7f467c0c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 -- 172.21.15.70:0/4151809557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740094d0 con 0x7f467c15c6d0 2023-08-25T09:38:47.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46810f2640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f467c15ec10 0x7f467c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:47.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.282+0000 7f46808f1640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c1f59a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f467800c8a0 tx=0x7f467800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.283+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678003b70 con 0x7f467c15c6d0 2023-08-25T09:38:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.283+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678003cf0 con 0x7f467c15c6d0 2023-08-25T09:38:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.283+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800aa90 con 0x7f467c15c6d0 2023-08-25T09:38:47.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.283+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f467c1f5f80 con 0x7f467c15c6d0 2023-08-25T09:38:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.283+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467c1f6540 con 0x7f467c15c6d0 2023-08-25T09:38:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.284+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f467c0cf3b0 con 0x7f467c15c6d0 2023-08-25T09:38:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.285+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f467800abf0 con 0x7f467c15c6d0 2023-08-25T09:38:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.286+0000 7f46717fa640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f466806a080 0x7f466806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.286+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f46780881a0 con 0x7f467c15c6d0 2023-08-25T09:38:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.286+0000 7f4673fff640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f466806a080 0x7f466806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.287+0000 7f4673fff640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f466806a080 0x7f466806c540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f467c1561a0 tx=0x7f466c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:47.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.291+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4678053b40 con 0x7f467c15c6d0 2023-08-25T09:38:47.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.479+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f467c04e420 con 0x7f467c15c6d0 2023-08-25T09:38:47.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.481+0000 7f46717fa640 1 -- 172.21.15.70:0/4151809557 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14663 (secure 0 0 0) 0x7f4678003e60 con 0x7f467c15c6d0 2023-08-25T09:38:47.483 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:47.483 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:47.002380+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:47.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f466806a080 msgr2=0x7f466806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f466806a080 0x7f466806c540 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f467c1561a0 tx=0x7f466c008040 comp rx=0 tx=0).stop 2023-08-25T09:38:47.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 msgr2=0x7f467c1f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c1f59a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f467800c8a0 tx=0x7f467800cd60 comp rx=0 tx=0).stop 2023-08-25T09:38:47.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 shutdown_connections 2023-08-25T09:38:47.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f466806a080 0x7f466806c540 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f467c15ec10 0x7f467c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f467c15e830 0x7f467c0c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 --2- 172.21.15.70:0/4151809557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f467c15c6d0 0x7f467c1f59a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 >> 172.21.15.70:0/4151809557 conn(0x7f467c069850 msgr2=0x7f467c0b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:47.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 shutdown_connections 2023-08-25T09:38:47.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.484+0000 7f4682b55640 1 -- 172.21.15.70:0/4151809557 wait complete. 2023-08-25T09:38:47.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:38:47.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:38:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.712+0000 7ff83c3cf640 1 Processor -- start 2023-08-25T09:38:47.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.712+0000 7ff83c3cf640 1 -- start start 2023-08-25T09:38:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83c3cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff834069f60 0x7ff8340ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83c3cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8340c2550 con 0x7ff83415ae50 2023-08-25T09:38:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83c3cf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8340ce330 con 0x7ff8340c2110 2023-08-25T09:38:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83c3cf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8340cc5f0 con 0x7ff834069f60 2023-08-25T09:38:47.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83a16b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff83415ae50 0x7ff8340cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52460/0 (socket says 172.21.15.70:52460) 2023-08-25T09:38:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83a16b640 1 -- 172.21.15.70:0/3321171065 learned_addr learned my addr 172.21.15.70:0/3321171065 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83a96c640 1 --2- 172.21.15.70:0/3321171065 >> v2:172.21.15.70:3300/0 conn(0x7ff834069f60 0x7ff8340ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83a96c640 1 -- 172.21.15.70:0/3321171065 >> v1:172.21.15.155:6789/0 conn(0x7ff8340c2110 legacy=0x7ff834069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83a96c640 1 -- 172.21.15.70:0/3321171065 >> v1:172.21.15.70:6789/0 conn(0x7ff83415ae50 legacy=0x7ff8340cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff839169640 1 -- 172.21.15.70:0/3321171065 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883771897 0 0) 0x7ff8340c2550 con 0x7ff83415ae50 2023-08-25T09:38:47.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.714+0000 7ff83a96c640 1 -- 172.21.15.70:0/3321171065 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff834070650 con 0x7ff834069f60 2023-08-25T09:38:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.715+0000 7ff83a96c640 1 --2- 172.21.15.70:0/3321171065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff834069f60 0x7ff8340ccad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009d90 tx=0x7ff82c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd990be19f55eb98 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff839169640 1 -- 172.21.15.70:0/3321171065 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c02cd00 con 0x7ff834069f60 2023-08-25T09:38:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff839169640 1 -- 172.21.15.70:0/3321171065 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c02ce60 con 0x7ff834069f60 2023-08-25T09:38:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff839169640 1 -- 172.21.15.70:0/3321171065 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c034660 con 0x7ff834069f60 2023-08-25T09:38:47.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 -- 172.21.15.70:0/3321171065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff834069f60 msgr2=0x7ff8340ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/3321171065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff834069f60 0x7ff8340ccad0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff82c009d90 tx=0x7ff82c02bf50 comp rx=0 tx=0).stop 2023-08-25T09:38:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 -- 172.21.15.70:0/3321171065 shutdown_connections 2023-08-25T09:38:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/3321171065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff834069f60 0x7ff8340ccad0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 -- 172.21.15.70:0/3321171065 >> 172.21.15.70:0/3321171065 conn(0x7ff8340cb0d0 msgr2=0x7ff8340cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:47.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 -- 172.21.15.70:0/3321171065 shutdown_connections 2023-08-25T09:38:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.716+0000 7ff83c3cf640 1 -- 172.21.15.70:0/3321171065 wait complete. 2023-08-25T09:38:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.718+0000 7ff83c3cf640 1 Processor -- start 2023-08-25T09:38:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.718+0000 7ff83c3cf640 1 -- start start 2023-08-25T09:38:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83c3cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff834069f60 0x7ff8341f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83c3cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 0x7ff8340c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83c3cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff83415ae50 0x7ff8340ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83c3cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c9820 con 0x7ff8340c2110 2023-08-25T09:38:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83c3cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340b0ee0 con 0x7ff834069f60 2023-08-25T09:38:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83c3cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8340c8c10 con 0x7ff83415ae50 2023-08-25T09:38:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 0x7ff8340c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 0x7ff8340c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36190/0 (socket says 172.21.15.70:36190) 2023-08-25T09:38:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 -- 172.21.15.70:0/2888170922 learned_addr learned my addr 172.21.15.70:0/2888170922 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 -- 172.21.15.70:0/2888170922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff83415ae50 msgr2=0x7ff8340ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff83415ae50 0x7ff8340ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 -- 172.21.15.70:0/2888170922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff834069f60 msgr2=0x7ff8341f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff834069f60 0x7ff8341f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.719+0000 7ff83996a640 1 -- 172.21.15.70:0/2888170922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c009a70 con 0x7ff8340c2110 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.720+0000 7ff83996a640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 0x7ff8340c9e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff82400c410 tx=0x7ff82400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.720+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240035f0 con 0x7ff8340c2110 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.720+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff824003750 con 0x7ff8340c2110 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.720+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824003b40 con 0x7ff8340c2110 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.720+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8340c8ef0 con 0x7ff8340c2110 2023-08-25T09:38:47.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.720+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8340c94b0 con 0x7ff8340c2110 2023-08-25T09:38:47.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.721+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8340c9990 con 0x7ff8340c2110 2023-08-25T09:38:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.722+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff824003ca0 con 0x7ff8340c2110 2023-08-25T09:38:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.723+0000 7ff82affd640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff81806a050 0x7ff81806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.723+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff8240874f0 con 0x7ff8340c2110 2023-08-25T09:38:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.723+0000 7ff83a16b640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff81806a050 0x7ff81806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:47.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.724+0000 7ff83a16b640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff81806a050 0x7ff81806c510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff830007fd0 tx=0x7ff830007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:47.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.727+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff824052e90 con 0x7ff8340c2110 2023-08-25T09:38:47.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.905+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff834066fb0 con 0x7ff8340c2110 2023-08-25T09:38:47.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.905+0000 7ff82affd640 1 -- 172.21.15.70:0/2888170922 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8240564b0 con 0x7ff8340c2110 2023-08-25T09:38:47.907 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:38:47.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff81806a050 msgr2=0x7ff81806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff81806a050 0x7ff81806c510 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff830007fd0 tx=0x7ff830007960 comp rx=0 tx=0).stop 2023-08-25T09:38:47.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 msgr2=0x7ff8340c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 0x7ff8340c9e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff82400c410 tx=0x7ff82400c8d0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 shutdown_connections 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff81806a050 0x7ff81806c510 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff83415ae50 0x7ff8340ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8340c2110 0x7ff8340c9e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 --2- 172.21.15.70:0/2888170922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff834069f60 0x7ff8341f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:47.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 >> 172.21.15.70:0/2888170922 conn(0x7ff8340cb0d0 msgr2=0x7ff8340cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 shutdown_connections 2023-08-25T09:38:47.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:47.908+0000 7ff83c3cf640 1 -- 172.21.15.70:0/2888170922 wait complete. 2023-08-25T09:38:47.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:38:48.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.131+0000 7fbd853a4640 1 Processor -- start 2023-08-25T09:38:48.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.132+0000 7fbd853a4640 1 -- start start 2023-08-25T09:38:48.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd853a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbd800b2850 0x7fbd800cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd853a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 0x7fbd800c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd853a4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd800cd800 con 0x7fbd800b2470 2023-08-25T09:38:48.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd853a4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd8015ad20 con 0x7fbd800b2850 2023-08-25T09:38:48.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd853a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015a110 con 0x7fbd800c8d20 2023-08-25T09:38:48.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd7ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbd800b2470 0x7fbd80159930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52476/0 (socket says 172.21.15.70:52476) 2023-08-25T09:38:48.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.133+0000 7fbd7ffff640 1 -- 172.21.15.70:0/2520506857 learned_addr learned my addr 172.21.15.70:0/2520506857 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:48.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385326017 0 0) 0x7fbd800cd800 con 0x7fbd800b2470 2023-08-25T09:38:48.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd6c005730 con 0x7fbd800b2470 2023-08-25T09:38:48.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2520506857 >> v2:172.21.15.155:3300/0 conn(0x7fbd800b2850 0x7fbd800cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd84ba3640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 0x7fbd800c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364014593 0 0) 0x7fbd6c005730 con 0x7fbd800b2470 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 msgr2=0x7fbd800c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 0x7fbd800c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 >> v2:172.21.15.155:3300/0 conn(0x7fbd800b2850 msgr2=0x7fbd800cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 --2- 172.21.15.70:0/2520506857 >> v2:172.21.15.155:3300/0 conn(0x7fbd800b2850 0x7fbd800cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8015a3f0 con 0x7fbd800b2470 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fbd700017f0 con 0x7fbd800b2470 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2520506857 >> v2:172.21.15.155:3300/0 conn(0x7fbd800b2850 0x7fbd800cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:48.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 >> v1:172.21.15.70:6789/0 conn(0x7fbd800b2470 legacy=0x7fbd80159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd6c005b80 0x7fbd6c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd6c0069d0 0x7fbd6c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 0x7fbd6c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015ad20 con 0x7fbd6c0069d0 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015a110 con 0x7fbd6c005b80 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.134+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c00dc70 con 0x7fbd800c8d20 2023-08-25T09:38:48.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd6c0069d0 0x7fbd6c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd7ffff640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd6c005b80 0x7fbd6c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd84ba3640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 0x7fbd6c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd777fe640 1 -- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 msgr2=0x7fbd6c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd800c8d20 0x7fbd6c007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd777fe640 1 -- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd6c005b80 msgr2=0x7fbd6c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd6c005b80 0x7fbd6c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.135+0000 7fbd777fe640 1 -- 172.21.15.70:0/2520506857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c005730 con 0x7fbd6c0069d0 2023-08-25T09:38:48.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.136+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd6c0069d0 0x7fbd6c006e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006fd0 tx=0x7fbd6c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e552db5fb0390578 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:48.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.136+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004230 con 0x7fbd6c0069d0 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.136+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd68004390 con 0x7fbd6c0069d0 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.136+0000 7fbd7f7fe640 1 -- 172.21.15.70:0/2520506857 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003830 con 0x7fbd6c0069d0 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd853a4640 1 -- 172.21.15.70:0/2520506857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd6c0069d0 msgr2=0x7fbd6c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2520506857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd6c0069d0 0x7fbd6c006e50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd68006fd0 tx=0x7fbd6c007a00 comp rx=0 tx=0).stop 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd84ba3640 1 -- 172.21.15.70:0/2520506857 reap_dead start 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd853a4640 1 -- 172.21.15.70:0/2520506857 shutdown_connections 2023-08-25T09:38:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd853a4640 1 -- 172.21.15.70:0/2520506857 >> 172.21.15.70:0/2520506857 conn(0x7fbd800ce830 msgr2=0x7fbd800cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd853a4640 1 -- 172.21.15.70:0/2520506857 shutdown_connections 2023-08-25T09:38:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.137+0000 7fbd853a4640 1 -- 172.21.15.70:0/2520506857 wait complete. 2023-08-25T09:38:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.139+0000 7fbd853a4640 1 Processor -- start 2023-08-25T09:38:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.139+0000 7fbd853a4640 1 -- start start 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd853a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd6c0069d0 0x7fbd800c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd853a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8009b240 0x7fbd80070730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd853a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 0x7fbd800c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd853a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800c7fb0 con 0x7fbd8009b790 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd853a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800c54f0 con 0x7fbd8009b240 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd853a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800c7610 con 0x7fbd6c0069d0 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd84ba3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 0x7fbd800c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd84ba3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 0x7fbd800c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36208/0 (socket says 172.21.15.70:36208) 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.140+0000 7fbd84ba3640 1 -- 172.21.15.70:0/2672851803 learned_addr learned my addr 172.21.15.70:0/2672851803 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd7ffff640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd6c0069d0 0x7fbd800c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd84ba3640 1 -- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd6c0069d0 msgr2=0x7fbd800c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd777fe640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8009b240 0x7fbd80070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd84ba3640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd6c0069d0 0x7fbd800c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd84ba3640 1 -- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8009b240 msgr2=0x7fbd80070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd84ba3640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8009b240 0x7fbd80070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd84ba3640 1 -- 172.21.15.70:0/2672851803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800c5790 con 0x7fbd8009b790 2023-08-25T09:38:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd84ba3640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 0x7fbd800c9130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd800c9100 tx=0x7fbd78009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78007c10 con 0x7fbd8009b790 2023-08-25T09:38:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7800f050 con 0x7fbd8009b790 2023-08-25T09:38:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd800c78f0 con 0x7fbd8009b790 2023-08-25T09:38:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.141+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78014410 con 0x7fbd8009b790 2023-08-25T09:38:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.142+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd800c7c60 con 0x7fbd8009b790 2023-08-25T09:38:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.143+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd80061460 con 0x7fbd8009b790 2023-08-25T09:38:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.144+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbd78014570 con 0x7fbd8009b790 2023-08-25T09:38:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.145+0000 7fbd7d7fa640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbd6006a010 0x7fbd6006c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.145+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbd78088500 con 0x7fbd8009b790 2023-08-25T09:38:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.145+0000 7fbd7ffff640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbd6006a010 0x7fbd6006c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.145+0000 7fbd7ffff640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbd6006a010 0x7fbd6006c4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd700011d0 tx=0x7fbd70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:48.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.149+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbd78053fd0 con 0x7fbd8009b790 2023-08-25T09:38:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.338+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd80071d30 con 0x7fbd8009b790 2023-08-25T09:38:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.339+0000 7fbd7d7fa640 1 -- 172.21.15.70:0/2672851803 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd780575f0 con 0x7fbd8009b790 2023-08-25T09:38:48.341 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:38:48.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbd6006a010 msgr2=0x7fbd6006c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbd6006a010 0x7fbd6006c4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbd700011d0 tx=0x7fbd70009040 comp rx=0 tx=0).stop 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 msgr2=0x7fbd800c9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 0x7fbd800c9130 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd800c9100 tx=0x7fbd78009d20 comp rx=0 tx=0).stop 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 shutdown_connections 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbd6006a010 0x7fbd6006c4d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbd8009b790 0x7fbd800c9130 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbd8009b240 0x7fbd80070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 --2- 172.21.15.70:0/2672851803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbd6c0069d0 0x7fbd800c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 >> 172.21.15.70:0/2672851803 conn(0x7fbd800ce830 msgr2=0x7fbd800ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:48.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 shutdown_connections 2023-08-25T09:38:48.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.342+0000 7fbd853a4640 1 -- 172.21.15.70:0/2672851803 wait complete. 2023-08-25T09:38:48.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.566+0000 7f8487fff640 1 Processor -- start 2023-08-25T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.566+0000 7f8487fff640 1 -- start start 2023-08-25T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8487fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f848815ca00 0x7f84880cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8487fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848806ba40 con 0x7f84880acd50 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8487fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8488160680 con 0x7f84880c5120 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8487fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8488160960 con 0x7f848815ca00 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8486ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f848815ca00 0x7f84880cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8486ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f848815ca00 0x7f84880cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36218/0 (socket says 172.21.15.70:36218) 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f8486ffd640 1 -- 172.21.15.70:0/3682922354 learned_addr learned my addr 172.21.15.70:0/3682922354 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.568+0000 7f84877fe640 1 --1- 172.21.15.70:0/3682922354 >> v1:172.21.15.70:6789/0 conn(0x7f84880acd50 0x7f848806b330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52484/0 (socket says 172.21.15.70:52484) 2023-08-25T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.569+0000 7f8486ffd640 1 -- 172.21.15.70:0/3682922354 >> v1:172.21.15.155:6789/0 conn(0x7f84880c5120 legacy=0x7f848806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.569+0000 7f8486ffd640 1 -- 172.21.15.70:0/3682922354 >> v1:172.21.15.70:6789/0 conn(0x7f84880acd50 legacy=0x7f848806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.569+0000 7f8486ffd640 1 -- 172.21.15.70:0/3682922354 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8488160c40 con 0x7f848815ca00 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.569+0000 7f8485ffb640 1 -- 172.21.15.70:0/3682922354 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855418394 0 0) 0x7f848806ba40 con 0x7f84880acd50 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.570+0000 7f8486ffd640 1 --2- 172.21.15.70:0/3682922354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848815ca00 0x7f84880cceb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8478009870 tx=0x7f847802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838859f34b73abb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.570+0000 7f8485ffb640 1 -- 172.21.15.70:0/3682922354 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478030850 con 0x7f848815ca00 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.570+0000 7f8485ffb640 1 -- 172.21.15.70:0/3682922354 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84780309b0 con 0x7f848815ca00 2023-08-25T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.570+0000 7f8485ffb640 1 -- 172.21.15.70:0/3682922354 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478030d40 con 0x7f848815ca00 2023-08-25T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.570+0000 7f8487fff640 1 -- 172.21.15.70:0/3682922354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848815ca00 msgr2=0x7f84880cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.570+0000 7f8487fff640 1 --2- 172.21.15.70:0/3682922354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848815ca00 0x7f84880cceb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8478009870 tx=0x7f847802b990 comp rx=0 tx=0).stop 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.571+0000 7f8487fff640 1 -- 172.21.15.70:0/3682922354 shutdown_connections 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.571+0000 7f8487fff640 1 --2- 172.21.15.70:0/3682922354 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848815ca00 0x7f84880cceb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.571+0000 7f8487fff640 1 -- 172.21.15.70:0/3682922354 >> 172.21.15.70:0/3682922354 conn(0x7f84880affd0 msgr2=0x7f84880b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.571+0000 7f8487fff640 1 -- 172.21.15.70:0/3682922354 shutdown_connections 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.571+0000 7f8487fff640 1 -- 172.21.15.70:0/3682922354 wait complete. 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.572+0000 7f8487fff640 1 Processor -- start 2023-08-25T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.573+0000 7f8487fff640 1 -- start start 2023-08-25T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.573+0000 7f8487fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84880acd50 0x7f84880c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f8487fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 0x7f84880cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f8487fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f848815ca00 0x7f84880ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f8487fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c44d0 con 0x7f84880c5120 2023-08-25T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f8487fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c2420 con 0x7f848815ca00 2023-08-25T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f8487fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c3e10 con 0x7f84880acd50 2023-08-25T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 0x7f84880cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 0x7f84880cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36220/0 (socket says 172.21.15.70:36220) 2023-08-25T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 -- 172.21.15.70:0/1388657974 learned_addr learned my addr 172.21.15.70:0/1388657974 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 -- 172.21.15.70:0/1388657974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84880acd50 msgr2=0x7f84880c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84880acd50 0x7f84880c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 -- 172.21.15.70:0/1388657974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f848815ca00 msgr2=0x7f84880ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f848815ca00 0x7f84880ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 -- 172.21.15.70:0/1388657974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780094d0 con 0x7f84880c5120 2023-08-25T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.574+0000 7f84867fc640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 0x7f84880cc9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c430 tx=0x7f847c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.575+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c011790 con 0x7f84880c5120 2023-08-25T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.575+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c0118f0 con 0x7f84880c5120 2023-08-25T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.575+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84880c40f0 con 0x7f84880c5120 2023-08-25T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.575+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c011ce0 con 0x7f84880c5120 2023-08-25T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.575+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880c3ac0 con 0x7f84880c5120 2023-08-25T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.576+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8488066fb0 con 0x7f84880c5120 2023-08-25T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.577+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f847c01c460 con 0x7f84880c5120 2023-08-25T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.578+0000 7f846f7fe640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847406a050 0x7f847406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.578+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f847c0882a0 con 0x7f84880c5120 2023-08-25T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.578+0000 7f8486ffd640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847406a050 0x7f847406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.579+0000 7f8486ffd640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847406a050 0x7f847406c510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f847803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.582+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f847c053c40 con 0x7f84880c5120 2023-08-25T09:38:48.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.806+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84880c4610 con 0x7f84880c5120 2023-08-25T09:38:48.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.807+0000 7f846f7fe640 1 -- 172.21.15.70:0/1388657974 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f84880c4610 con 0x7f84880c5120 2023-08-25T09:38:48.808 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:48.809 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-25T09:38:48.809 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 3 2023-08-25T09:38:48.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847406a050 msgr2=0x7f847406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847406a050 0x7f847406c510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f847803b040 comp rx=0 tx=0).stop 2023-08-25T09:38:48.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 msgr2=0x7f84880cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:48.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 0x7f84880cc9c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c430 tx=0x7f847c00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 shutdown_connections 2023-08-25T09:38:48.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847406a050 0x7f847406c510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f848815ca00 0x7f84880ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84880c5120 0x7f84880cc9c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 --2- 172.21.15.70:0/1388657974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84880acd50 0x7f84880c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:48.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 >> 172.21.15.70:0/1388657974 conn(0x7f84880affd0 msgr2=0x7f84880c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:48.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 shutdown_connections 2023-08-25T09:38:48.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:48.809+0000 7f8487fff640 1 -- 172.21.15.70:0/1388657974 wait complete. 2023-08-25T09:38:48.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:49.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.034+0000 7f764ae83640 1 Processor -- start 2023-08-25T09:38:49.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.034+0000 7f764ae83640 1 -- start start 2023-08-25T09:38:49.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f764ae83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f764415cbd0 0x7f76440cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f764ae83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76440d2a00 0x7f76440d2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f764ae83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764415cfb0 con 0x7f76441606c0 2023-08-25T09:38:49.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f764ae83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440d3390 con 0x7f76440d2a00 2023-08-25T09:38:49.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f764ae83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76440cd4e0 con 0x7f764415cbd0 2023-08-25T09:38:49.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f7643fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f76441606c0 0x7f76440cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37924/0 (socket says 172.21.15.70:37924) 2023-08-25T09:38:49.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f7643fff640 1 -- 172.21.15.70:0/1395583346 learned_addr learned my addr 172.21.15.70:0/1395583346 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:49.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.036+0000 7f7649420640 1 --2- 172.21.15.70:0/1395583346 >> v2:172.21.15.70:3300/0 conn(0x7f76440d2a00 0x7f76440d2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f7648c1f640 1 --2- 172.21.15.70:0/1395583346 >> v2:172.21.15.155:3300/0 conn(0x7f764415cbd0 0x7f76440cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f76437fe640 1 -- 172.21.15.70:0/1395583346 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149162464 0 0) 0x7f764415cfb0 con 0x7f76441606c0 2023-08-25T09:38:49.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f76437fe640 1 -- 172.21.15.70:0/1395583346 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7634003520 con 0x7f76441606c0 2023-08-25T09:38:49.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f7649420640 1 -- 172.21.15.70:0/1395583346 >> v2:172.21.15.155:3300/0 conn(0x7f764415cbd0 msgr2=0x7f76440cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f7649420640 1 --2- 172.21.15.70:0/1395583346 >> v2:172.21.15.155:3300/0 conn(0x7f764415cbd0 0x7f76440cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f7649420640 1 -- 172.21.15.70:0/1395583346 >> v1:172.21.15.70:6789/0 conn(0x7f76441606c0 legacy=0x7f76440cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.037+0000 7f7649420640 1 -- 172.21.15.70:0/1395583346 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644160f10 con 0x7f76440d2a00 2023-08-25T09:38:49.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.038+0000 7f7649420640 1 --2- 172.21.15.70:0/1395583346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76440d2a00 0x7f76440d2e50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f764408b900 tx=0x7f763c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66825e08d7cf3959 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.038+0000 7f76437fe640 1 -- 172.21.15.70:0/1395583346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c02cc80 con 0x7f76440d2a00 2023-08-25T09:38:49.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.038+0000 7f76437fe640 1 -- 172.21.15.70:0/1395583346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c02cde0 con 0x7f76440d2a00 2023-08-25T09:38:49.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.038+0000 7f76437fe640 1 -- 172.21.15.70:0/1395583346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c034660 con 0x7f76440d2a00 2023-08-25T09:38:49.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 -- 172.21.15.70:0/1395583346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76440d2a00 msgr2=0x7f76440d2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 --2- 172.21.15.70:0/1395583346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76440d2a00 0x7f76440d2e50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f764408b900 tx=0x7f763c02bed0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 -- 172.21.15.70:0/1395583346 shutdown_connections 2023-08-25T09:38:49.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 --2- 172.21.15.70:0/1395583346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76440d2a00 0x7f76440d2e50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 --2- 172.21.15.70:0/1395583346 >> v2:172.21.15.155:3300/0 conn(0x7f764415cbd0 0x7f76440cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 -- 172.21.15.70:0/1395583346 >> 172.21.15.70:0/1395583346 conn(0x7f7644070260 msgr2=0x7f7644070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:49.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 -- 172.21.15.70:0/1395583346 shutdown_connections 2023-08-25T09:38:49.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.039+0000 7f764ae83640 1 -- 172.21.15.70:0/1395583346 wait complete. 2023-08-25T09:38:49.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.041+0000 7f764ae83640 1 Processor -- start 2023-08-25T09:38:49.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.041+0000 7f764ae83640 1 -- start start 2023-08-25T09:38:49.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f764ae83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76440d2a00 0x7f764415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f764ae83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f764415cbd0 0x7f76440b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f764ae83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 0x7f76440c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f764ae83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764415a830 con 0x7f76441606c0 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f764ae83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440b7cd0 con 0x7f76440d2a00 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f764ae83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764415aa80 con 0x7f764415cbd0 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f7649420640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 0x7f76440c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f7649420640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 0x7f76440c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56478/0 (socket says 172.21.15.70:56478) 2023-08-25T09:38:49.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.042+0000 7f7649420640 1 -- 172.21.15.70:0/2107719444 learned_addr learned my addr 172.21.15.70:0/2107719444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7648c1f640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76440d2a00 0x7f764415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7643fff640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f764415cbd0 0x7f76440b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7649420640 1 -- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f764415cbd0 msgr2=0x7f76440b7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7649420640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f764415cbd0 0x7f76440b7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7649420640 1 -- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76440d2a00 msgr2=0x7f764415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7649420640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76440d2a00 0x7f764415a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7649420640 1 -- 172.21.15.70:0/2107719444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c009a70 con 0x7f76441606c0 2023-08-25T09:38:49.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7643fff640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f764415cbd0 0x7f76440b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7648c1f640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76440d2a00 0x7f764415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f7649420640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 0x7f76440c7f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f763c02c4e0 tx=0x7f763c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c040650 con 0x7f76441606c0 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c0407b0 con 0x7f76441606c0 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c040b60 con 0x7f76441606c0 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764415ad60 con 0x7f76441606c0 2023-08-25T09:38:49.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.043+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764415b320 con 0x7f76441606c0 2023-08-25T09:38:49.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.044+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764404e420 con 0x7f76441606c0 2023-08-25T09:38:49.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.045+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f763c040cc0 con 0x7f76441606c0 2023-08-25T09:38:49.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.046+0000 7f76417fa640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f762806a050 0x7f762806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.046+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f763c0aaff0 con 0x7f76441606c0 2023-08-25T09:38:49.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.046+0000 7f7648c1f640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f762806a050 0x7f762806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.047+0000 7f7648c1f640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f762806a050 0x7f762806c510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7638006fd0 tx=0x7f7638008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.050+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f763c076960 con 0x7f76441606c0 2023-08-25T09:38:49.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.242+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7644070900 con 0x7f76441606c0 2023-08-25T09:38:49.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.243+0000 7f76417fa640 1 -- 172.21.15.70:0/2107719444 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14650 (secure 0 0 0) 0x7f763c009450 con 0x7f76441606c0 2023-08-25T09:38:49.245 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:49.245 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":19,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:48.014265+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:49.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.246+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f762806a050 msgr2=0x7f762806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.246+0000 7f764ae83640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f762806a050 0x7f762806c510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7638006fd0 tx=0x7f7638008040 comp rx=0 tx=0).stop 2023-08-25T09:38:49.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.246+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 msgr2=0x7f76440c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.246+0000 7f764ae83640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 0x7f76440c7f30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f763c02c4e0 tx=0x7f763c004a40 comp rx=0 tx=0).stop 2023-08-25T09:38:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 shutdown_connections 2023-08-25T09:38:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f762806a050 0x7f762806c510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76441606c0 0x7f76440c7f30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f764415cbd0 0x7f76440b7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 --2- 172.21.15.70:0/2107719444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f76440d2a00 0x7f764415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 >> 172.21.15.70:0/2107719444 conn(0x7f7644070260 msgr2=0x7f76440d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 shutdown_connections 2023-08-25T09:38:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.247+0000 7f764ae83640 1 -- 172.21.15.70:0/2107719444 wait complete. 2023-08-25T09:38:49.265 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:38:49.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:38:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.484+0000 7f8486ea7640 1 Processor -- start 2023-08-25T09:38:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.485+0000 7f8486ea7640 1 -- start start 2023-08-25T09:38:49.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8486ea7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8486ea7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f84800cdc20 0x7f8480069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8486ea7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800cc5f0 con 0x7f84800ce060 2023-08-25T09:38:49.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8486ea7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8480070650 con 0x7f84800cdc20 2023-08-25T09:38:49.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8486ea7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8480070930 con 0x7f848015ae50 2023-08-25T09:38:49.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8485444640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f84800ce060 0x7f84800ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37926/0 (socket says 172.21.15.70:37926) 2023-08-25T09:38:49.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.486+0000 7f8485444640 1 -- 172.21.15.70:0/2304484010 learned_addr learned my addr 172.21.15.70:0/2304484010 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:49.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f8484c43640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847ffff640 1 --2- 172.21.15.70:0/2304484010 >> v2:172.21.15.155:3300/0 conn(0x7f84800cdc20 0x7f8480069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543476957 0 0) 0x7f84800cc5f0 con 0x7f84800ce060 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8468003520 con 0x7f84800ce060 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3027081038 0 0) 0x7f8468003520 con 0x7f84800ce060 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 msgr2=0x7f84800c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 >> v2:172.21.15.155:3300/0 conn(0x7f84800cdc20 msgr2=0x7f8480069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2304484010 >> v2:172.21.15.155:3300/0 conn(0x7f84800cdc20 0x7f8480069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480070c10 con 0x7f84800ce060 2023-08-25T09:38:49.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8478001670 con 0x7f84800ce060 2023-08-25T09:38:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 >> v1:172.21.15.70:6789/0 conn(0x7f84800ce060 legacy=0x7f84800ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 0x7f8468003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468003ef0 0x7f84680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8468004d00 0x7f8468005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480070650 con 0x7f8468004d00 2023-08-25T09:38:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480070930 con 0x7f8468003ef0 2023-08-25T09:38:49.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.487+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846800b9b0 con 0x7f848015ae50 2023-08-25T09:38:49.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8485444640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8468004d00 0x7f8468005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8484c43640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 0x7f8468003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8485444640 1 -- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 msgr2=0x7f8468003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8485444640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848015ae50 0x7f8468003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f847ffff640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468003ef0 0x7f84680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8485444640 1 -- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468003ef0 msgr2=0x7f84680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8485444640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468003ef0 0x7f84680047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.488+0000 7f8485444640 1 -- 172.21.15.70:0/2304484010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468003520 con 0x7f8468004d00 2023-08-25T09:38:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.489+0000 7f8485444640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8468004d00 0x7f8468005180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f847800c210 tx=0x7f84680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e715664abf8b704 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.489+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478005ad0 con 0x7f8468004d00 2023-08-25T09:38:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478005c30 con 0x7f8468004d00 2023-08-25T09:38:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f847f7fe640 1 -- 172.21.15.70:0/2304484010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847802caf0 con 0x7f8468004d00 2023-08-25T09:38:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8486ea7640 1 -- 172.21.15.70:0/2304484010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8468004d00 msgr2=0x7f8468005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8486ea7640 1 --2- 172.21.15.70:0/2304484010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8468004d00 0x7f8468005180 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f847800c210 tx=0x7f84680057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8485444640 1 -- 172.21.15.70:0/2304484010 reap_dead start 2023-08-25T09:38:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8486ea7640 1 -- 172.21.15.70:0/2304484010 shutdown_connections 2023-08-25T09:38:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8486ea7640 1 -- 172.21.15.70:0/2304484010 >> 172.21.15.70:0/2304484010 conn(0x7f84800cb0d0 msgr2=0x7f84800cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8486ea7640 1 -- 172.21.15.70:0/2304484010 shutdown_connections 2023-08-25T09:38:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.490+0000 7f8486ea7640 1 -- 172.21.15.70:0/2304484010 wait complete. 2023-08-25T09:38:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.492+0000 7f8486ea7640 1 Processor -- start 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.492+0000 7f8486ea7640 1 -- start start 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.493+0000 7f8486ea7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468004d00 0x7f848014a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.493+0000 7f8486ea7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848014a5f0 0x7f84801473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.493+0000 7f8486ea7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 0x7f84800c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.493+0000 7f8486ea7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848014aa70 con 0x7f84801478e0 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.493+0000 7f8486ea7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8480147d90 con 0x7f8468004d00 2023-08-25T09:38:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.493+0000 7f8486ea7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800c9920 con 0x7f848014a5f0 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 0x7f84800c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 0x7f84800c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56490/0 (socket says 172.21.15.70:56490) 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 -- 172.21.15.70:0/150143554 learned_addr learned my addr 172.21.15.70:0/150143554 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f847ffff640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848014a5f0 0x7f84801473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8484c43640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468004d00 0x7f848014a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 -- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848014a5f0 msgr2=0x7f84801473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848014a5f0 0x7f84801473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 -- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468004d00 msgr2=0x7f848014a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468004d00 0x7f848014a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 -- 172.21.15.70:0/150143554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800c9c00 con 0x7f84801478e0 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f847ffff640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848014a5f0 0x7f84801473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f8485444640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 0x7f84800c8c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8478001930 tx=0x7f8478001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.494+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847803c040 con 0x7f84801478e0 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.495+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847802cce0 con 0x7f84801478e0 2023-08-25T09:38:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.495+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800c9ee0 con 0x7f84801478e0 2023-08-25T09:38:49.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.495+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478007270 con 0x7f84801478e0 2023-08-25T09:38:49.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.495+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800ca4a0 con 0x7f84801478e0 2023-08-25T09:38:49.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.496+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84800ca630 con 0x7f84801478e0 2023-08-25T09:38:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.497+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8478032070 con 0x7f84801478e0 2023-08-25T09:38:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.497+0000 7f847d7fa640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a050 0x7f846c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.498+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f84780a92b0 con 0x7f84801478e0 2023-08-25T09:38:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.498+0000 7f8484c43640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a050 0x7f846c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.498+0000 7f8484c43640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a050 0x7f846c06c510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8470007380 tx=0x7f847000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.502+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8478074d70 con 0x7f84801478e0 2023-08-25T09:38:49.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.689+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8480066fb0 con 0x7f84801478e0 2023-08-25T09:38:49.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.690+0000 7f847d7fa640 1 -- 172.21.15.70:0/150143554 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8478001c50 con 0x7f84801478e0 2023-08-25T09:38:49.693 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:38:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a050 msgr2=0x7f846c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a050 0x7f846c06c510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8470007380 tx=0x7f847000b040 comp rx=0 tx=0).stop 2023-08-25T09:38:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 msgr2=0x7f84800c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 0x7f84800c8c10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8478001930 tx=0x7f8478001a70 comp rx=0 tx=0).stop 2023-08-25T09:38:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 shutdown_connections 2023-08-25T09:38:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a050 0x7f846c06c510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f84801478e0 0x7f84800c8c10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f848014a5f0 0x7f84801473a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 --2- 172.21.15.70:0/150143554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8468004d00 0x7f848014a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.692+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 >> 172.21.15.70:0/150143554 conn(0x7f84800cb0d0 msgr2=0x7f84800c3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.693+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 shutdown_connections 2023-08-25T09:38:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.693+0000 7f8486ea7640 1 -- 172.21.15.70:0/150143554 wait complete. 2023-08-25T09:38:49.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:38:49.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.921+0000 7f197baad640 1 Processor -- start 2023-08-25T09:38:49.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.921+0000 7f197baad640 1 -- start start 2023-08-25T09:38:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.922+0000 7f197baad640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1974159bf0 0x7f197415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.922+0000 7f197baad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f197406f0e0 0x7f1974070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f197baad640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197415e780 con 0x7f19740cd660 2023-08-25T09:38:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f197baad640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197406f8e0 con 0x7f1974159bf0 2023-08-25T09:38:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f197baad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197406fbc0 con 0x7f197406f0e0 2023-08-25T09:38:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1974159bf0 0x7f197415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1974159bf0 0x7f197415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56502/0 (socket says 172.21.15.70:56502) 2023-08-25T09:38:49.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 -- 172.21.15.70:0/1020267081 learned_addr learned my addr 172.21.15.70:0/1020267081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:49.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979048640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f197406f0e0 0x7f1974070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 -- 172.21.15.70:0/1020267081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f197406f0e0 msgr2=0x7f1974070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f197406f0e0 0x7f1974070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 -- 172.21.15.70:0/1020267081 >> v1:172.21.15.155:6789/0 conn(0x7f19740cd660 legacy=0x7f19740c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979849640 1 -- 172.21.15.70:0/1020267081 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197406fea0 con 0x7f1974159bf0 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.923+0000 7f1979048640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f197406f0e0 0x7f1974070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.924+0000 7f1979849640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1974159bf0 0x7f197415e240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1970009870 tx=0x7f197002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc4505125d9ffc1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f1978847640 1 -- 172.21.15.70:0/1020267081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970031640 con 0x7f1974159bf0 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f1978847640 1 -- 172.21.15.70:0/1020267081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19700317a0 con 0x7f1974159bf0 2023-08-25T09:38:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f1978847640 1 -- 172.21.15.70:0/1020267081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970031b30 con 0x7f1974159bf0 2023-08-25T09:38:49.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 -- 172.21.15.70:0/1020267081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1974159bf0 msgr2=0x7f197415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1974159bf0 0x7f197415e240 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1970009870 tx=0x7f197002b8f0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 -- 172.21.15.70:0/1020267081 shutdown_connections 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f197406f0e0 0x7f1974070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 --2- 172.21.15.70:0/1020267081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1974159bf0 0x7f197415e240 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 -- 172.21.15.70:0/1020267081 >> 172.21.15.70:0/1020267081 conn(0x7f19740cb0d0 msgr2=0x7f19740cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 -- 172.21.15.70:0/1020267081 shutdown_connections 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.925+0000 7f197baad640 1 -- 172.21.15.70:0/1020267081 wait complete. 2023-08-25T09:38:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.927+0000 7f197baad640 1 Processor -- start 2023-08-25T09:38:49.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.927+0000 7f197baad640 1 -- start start 2023-08-25T09:38:49.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f197baad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197406f0e0 0x7f19740c2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f197baad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 0x7f19740c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f197baad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1974159bf0 0x7f1974157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f197baad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19741526d0 con 0x7f19740cd660 2023-08-25T09:38:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f197baad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19740c48f0 con 0x7f197406f0e0 2023-08-25T09:38:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f197baad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19740c4bd0 con 0x7f1974159bf0 2023-08-25T09:38:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f1979048640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 0x7f19740c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f1979048640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 0x7f19740c3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56508/0 (socket says 172.21.15.70:56508) 2023-08-25T09:38:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.928+0000 7f1979048640 1 -- 172.21.15.70:0/3668730445 learned_addr learned my addr 172.21.15.70:0/3668730445 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f197a04a640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1974159bf0 0x7f1974157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979849640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197406f0e0 0x7f19740c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979048640 1 -- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1974159bf0 msgr2=0x7f1974157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979048640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1974159bf0 0x7f1974157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979048640 1 -- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197406f0e0 msgr2=0x7f19740c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979048640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197406f0e0 0x7f19740c2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979048640 1 -- 172.21.15.70:0/3668730445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700094d0 con 0x7f19740cd660 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f197a04a640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1974159bf0 0x7f1974157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f1979048640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 0x7f19740c3e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f196400c7f0 tx=0x7f196400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1964007a30 con 0x7f19740cd660 2023-08-25T09:38:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1964007b90 con 0x7f19740cd660 2023-08-25T09:38:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196400ce40 con 0x7f19740cd660 2023-08-25T09:38:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.929+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19740c4eb0 con 0x7f19740cd660 2023-08-25T09:38:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.930+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1974150820 con 0x7f19740cd660 2023-08-25T09:38:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.931+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1974150a40 con 0x7f19740cd660 2023-08-25T09:38:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.931+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f19640113f0 con 0x7f19740cd660 2023-08-25T09:38:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.933+0000 7f196a7fc640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f195806a050 0x7f195806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.933+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1964088420 con 0x7f19740cd660 2023-08-25T09:38:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.933+0000 7f1979849640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f195806a050 0x7f195806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.934+0000 7f1979849640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f195806a050 0x7f195806c510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1970036000 tx=0x7f197003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:49.937+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1964053dd0 con 0x7f19740cd660 2023-08-25T09:38:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.132+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1974070e40 con 0x7f19740cd660 2023-08-25T09:38:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.133+0000 7f196a7fc640 1 -- 172.21.15.70:0/3668730445 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f19640573f0 con 0x7f19740cd660 2023-08-25T09:38:50.136 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:38:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f195806a050 msgr2=0x7f195806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f195806a050 0x7f195806c510 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1970036000 tx=0x7f197003c040 comp rx=0 tx=0).stop 2023-08-25T09:38:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 msgr2=0x7f19740c3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 0x7f19740c3e80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f196400c7f0 tx=0x7f196400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 shutdown_connections 2023-08-25T09:38:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f195806a050 0x7f195806c510 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1974159bf0 0x7f1974157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19740cd660 0x7f19740c3e80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 --2- 172.21.15.70:0/3668730445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f197406f0e0 0x7f19740c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 >> 172.21.15.70:0/3668730445 conn(0x7f19740cb0d0 msgr2=0x7f1974158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.136+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 shutdown_connections 2023-08-25T09:38:50.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.137+0000 7f197baad640 1 -- 172.21.15.70:0/3668730445 wait complete. 2023-08-25T09:38:50.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:38:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.371+0000 7ff9152a7640 1 Processor -- start 2023-08-25T09:38:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.371+0000 7ff9152a7640 1 -- start start 2023-08-25T09:38:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff9152a7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9100cea30 0x7ff9100b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff9152a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9100ac870 0x7ff9100b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff9152a7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91006f050 con 0x7ff91015cb40 2023-08-25T09:38:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff9152a7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91006f330 con 0x7ff9100cea30 2023-08-25T09:38:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff9152a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91006f610 con 0x7ff9100ac870 2023-08-25T09:38:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff907fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9100cea30 0x7ff9100b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff907fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff9100cea30 0x7ff9100b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56518/0 (socket says 172.21.15.70:56518) 2023-08-25T09:38:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff907fff640 1 -- 172.21.15.70:0/2173621734 learned_addr learned my addr 172.21.15.70:0/2173621734 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.373+0000 7ff90f7fe640 1 --2- 172.21.15.70:0/2173621734 >> v2:172.21.15.155:3300/0 conn(0x7ff9100ac870 0x7ff9100b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.374+0000 7ff907fff640 1 -- 172.21.15.70:0/2173621734 >> v2:172.21.15.155:3300/0 conn(0x7ff9100ac870 msgr2=0x7ff9100b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.374+0000 7ff907fff640 1 --2- 172.21.15.70:0/2173621734 >> v2:172.21.15.155:3300/0 conn(0x7ff9100ac870 0x7ff9100b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.374+0000 7ff907fff640 1 -- 172.21.15.70:0/2173621734 >> v1:172.21.15.155:6789/0 conn(0x7ff91015cb40 legacy=0x7ff9100ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.374+0000 7ff907fff640 1 -- 172.21.15.70:0/2173621734 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91006f8f0 con 0x7ff9100cea30 2023-08-25T09:38:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.375+0000 7ff907fff640 1 --2- 172.21.15.70:0/2173621734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9100cea30 0x7ff9100b1b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9100adbd0 tx=0x7ff8f802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2120645926f1a9e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff90e7fc640 1 -- 172.21.15.70:0/2173621734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8031870 con 0x7ff9100cea30 2023-08-25T09:38:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff90e7fc640 1 -- 172.21.15.70:0/2173621734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f80319d0 con 0x7ff9100cea30 2023-08-25T09:38:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff90e7fc640 1 -- 172.21.15.70:0/2173621734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8031d60 con 0x7ff9100cea30 2023-08-25T09:38:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff9152a7640 1 -- 172.21.15.70:0/2173621734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9100cea30 msgr2=0x7ff9100b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2173621734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9100cea30 0x7ff9100b1b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9100adbd0 tx=0x7ff8f802b910 comp rx=0 tx=0).stop 2023-08-25T09:38:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff9152a7640 1 -- 172.21.15.70:0/2173621734 shutdown_connections 2023-08-25T09:38:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2173621734 >> v2:172.21.15.155:3300/0 conn(0x7ff9100ac870 0x7ff9100b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2173621734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9100cea30 0x7ff9100b1b30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.376+0000 7ff9152a7640 1 -- 172.21.15.70:0/2173621734 >> 172.21.15.70:0/2173621734 conn(0x7ff910069850 msgr2=0x7ff910069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.377+0000 7ff9152a7640 1 -- 172.21.15.70:0/2173621734 shutdown_connections 2023-08-25T09:38:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.377+0000 7ff9152a7640 1 -- 172.21.15.70:0/2173621734 wait complete. 2023-08-25T09:38:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.378+0000 7ff9152a7640 1 Processor -- start 2023-08-25T09:38:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.379+0000 7ff9152a7640 1 -- start start 2023-08-25T09:38:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.379+0000 7ff9152a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9100ac870 0x7ff9101f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff9152a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9100cea30 0x7ff9100af390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff9152a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 0x7ff9100af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff9152a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff910145330 con 0x7ff91015cb40 2023-08-25T09:38:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff9152a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff910160e70 con 0x7ff9100cea30 2023-08-25T09:38:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff9152a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff910145580 con 0x7ff9100ac870 2023-08-25T09:38:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 0x7ff9100af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 0x7ff9100af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56528/0 (socket says 172.21.15.70:56528) 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 -- 172.21.15.70:0/2872341556 learned_addr learned my addr 172.21.15.70:0/2872341556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90effd640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9100ac870 0x7ff9101f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff907fff640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9100cea30 0x7ff9100af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 -- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9100ac870 msgr2=0x7ff9101f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9100ac870 0x7ff9101f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 -- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9100cea30 msgr2=0x7ff9100af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9100cea30 0x7ff9100af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff90f7fe640 1 -- 172.21.15.70:0/2872341556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80094d0 con 0x7ff91015cb40 2023-08-25T09:38:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.380+0000 7ff907fff640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9100cea30 0x7ff9100af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:50.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.381+0000 7ff90f7fe640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 0x7ff9100af8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff908007e40 tx=0x7ff90800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.381+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908014620 con 0x7ff91015cb40 2023-08-25T09:38:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.381+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90800ee90 con 0x7ff91015cb40 2023-08-25T09:38:50.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.381+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080149e0 con 0x7ff91015cb40 2023-08-25T09:38:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.381+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff910145860 con 0x7ff91015cb40 2023-08-25T09:38:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.381+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9101f6290 con 0x7ff91015cb40 2023-08-25T09:38:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.382+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff910145d80 con 0x7ff91015cb40 2023-08-25T09:38:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.383+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff908014b40 con 0x7ff91015cb40 2023-08-25T09:38:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.384+0000 7ff9077fe640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8f406a050 0x7ff8f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.384+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff908088c50 con 0x7ff91015cb40 2023-08-25T09:38:50.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.384+0000 7ff90effd640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8f406a050 0x7ff8f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.385+0000 7ff90effd640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8f406a050 0x7ff8f406c510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:50.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.388+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff9080545f0 con 0x7ff91015cb40 2023-08-25T09:38:50.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.581+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9101f6420 con 0x7ff91015cb40 2023-08-25T09:38:50.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.582+0000 7ff9077fe640 1 -- 172.21.15.70:0/2872341556 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7ff908057c10 con 0x7ff91015cb40 2023-08-25T09:38:50.585 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:50.585 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:38:50.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:38:50.586 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:38:50.586 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:38:50.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:38:50.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:38:50.587 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:38:50.587 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:38:50.587 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:38:50.588 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:38:50.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8f406a050 msgr2=0x7ff8f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8f406a050 0x7ff8f406c510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff900006fd0 tx=0x7ff900008040 comp rx=0 tx=0).stop 2023-08-25T09:38:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 msgr2=0x7ff9100af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 0x7ff9100af8d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff908007e40 tx=0x7ff90800e4b0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 shutdown_connections 2023-08-25T09:38:50.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8f406a050 0x7ff8f406c510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff91015cb40 0x7ff9100af8d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9100cea30 0x7ff9100af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 --2- 172.21.15.70:0/2872341556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9100ac870 0x7ff9101f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 >> 172.21.15.70:0/2872341556 conn(0x7ff910069850 msgr2=0x7ff910159020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 shutdown_connections 2023-08-25T09:38:50.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.585+0000 7ff9152a7640 1 -- 172.21.15.70:0/2872341556 wait complete. 2023-08-25T09:38:50.607 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:38:50.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:38:50.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.820+0000 7f238a176640 1 Processor -- start 2023-08-25T09:38:50.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.820+0000 7f238a176640 1 -- start start 2023-08-25T09:38:50.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.821+0000 7f238a176640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142740 0x7f23840cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.821+0000 7f238a176640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f238406aa70 0x7f238406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.821+0000 7f238a176640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238415d320 con 0x7f2384142c30 2023-08-25T09:38:50.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.821+0000 7f238a176640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238406b440 con 0x7f238406aa70 2023-08-25T09:38:50.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f238a176640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23840ccd40 con 0x7f2384142740 2023-08-25T09:38:50.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2383fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f238406aa70 0x7f238406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2382ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2384142c30 0x7f238415db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37930/0 (socket says 172.21.15.70:37930) 2023-08-25T09:38:50.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2382ffd640 1 -- 172.21.15.70:0/3269315849 learned_addr learned my addr 172.21.15.70:0/3269315849 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:50.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f23837fe640 1 --2- 172.21.15.70:0/3269315849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142740 0x7f23840cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2383fff640 1 -- 172.21.15.70:0/3269315849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142740 msgr2=0x7f23840cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2383fff640 1 --2- 172.21.15.70:0/3269315849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142740 0x7f23840cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2383fff640 1 -- 172.21.15.70:0/3269315849 >> v1:172.21.15.70:6789/0 conn(0x7f2384142c30 legacy=0x7f238415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f23827fc640 1 -- 172.21.15.70:0/3269315849 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131096721 0 0) 0x7f238415d320 con 0x7f2384142c30 2023-08-25T09:38:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.822+0000 7f2383fff640 1 -- 172.21.15.70:0/3269315849 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840cd020 con 0x7f238406aa70 2023-08-25T09:38:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.823+0000 7f2383fff640 1 --2- 172.21.15.70:0/3269315849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238406af00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f237c009d90 tx=0x7f237c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4043dbadcd08912b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f23827fc640 1 -- 172.21.15.70:0/3269315849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c02cc50 con 0x7f238406aa70 2023-08-25T09:38:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f23827fc640 1 -- 172.21.15.70:0/3269315849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c02cdb0 con 0x7f238406aa70 2023-08-25T09:38:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f23827fc640 1 -- 172.21.15.70:0/3269315849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c034660 con 0x7f238406aa70 2023-08-25T09:38:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f238a176640 1 -- 172.21.15.70:0/3269315849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 msgr2=0x7f238406af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f238a176640 1 --2- 172.21.15.70:0/3269315849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238406af00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f237c009d90 tx=0x7f237c02bea0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f238a176640 1 -- 172.21.15.70:0/3269315849 shutdown_connections 2023-08-25T09:38:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f238a176640 1 --2- 172.21.15.70:0/3269315849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238406af00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f238a176640 1 --2- 172.21.15.70:0/3269315849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142740 0x7f23840cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.824+0000 7f238a176640 1 -- 172.21.15.70:0/3269315849 >> 172.21.15.70:0/3269315849 conn(0x7f2384069c70 msgr2=0x7f238406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.825+0000 7f238a176640 1 -- 172.21.15.70:0/3269315849 shutdown_connections 2023-08-25T09:38:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.825+0000 7f238a176640 1 -- 172.21.15.70:0/3269315849 wait complete. 2023-08-25T09:38:50.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.826+0000 7f238a176640 1 Processor -- start 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.826+0000 7f238a176640 1 -- start start 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.827+0000 7f238a176640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238413bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.827+0000 7f238a176640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2384142740 0x7f238415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.827+0000 7f238a176640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142c30 0x7f238413ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.827+0000 7f238a176640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238413dd60 con 0x7f238406aa70 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.827+0000 7f238a176640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238413e040 con 0x7f2384142740 2023-08-25T09:38:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.827+0000 7f238a176640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238413e320 con 0x7f2384142c30 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238413bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238413bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56542/0 (socket says 172.21.15.70:56542) 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 -- 172.21.15.70:0/2377036267 learned_addr learned my addr 172.21.15.70:0/2377036267 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 -- 172.21.15.70:0/2377036267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142c30 msgr2=0x7f238413ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142c30 0x7f238413ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 -- 172.21.15.70:0/2377036267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2384142740 msgr2=0x7f238415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2384142740 0x7f238415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 -- 172.21.15.70:0/2377036267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c009a70 con 0x7f238406aa70 2023-08-25T09:38:50.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.828+0000 7f23837fe640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238413bf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f237400d800 tx=0x7f237400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:50.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.829+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237400b6c0 con 0x7f238406aa70 2023-08-25T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.829+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237400b820 con 0x7f238406aa70 2023-08-25T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.829+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237400bbb0 con 0x7f238406aa70 2023-08-25T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.829+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238413e600 con 0x7f238406aa70 2023-08-25T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.829+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238413c4b0 con 0x7f238406aa70 2023-08-25T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.830+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238404e420 con 0x7f238406aa70 2023-08-25T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.831+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f237400bd10 con 0x7f238406aa70 2023-08-25T09:38:50.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.832+0000 7f236bfff640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f237006a050 0x7f237006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:50.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.832+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2374088780 con 0x7f238406aa70 2023-08-25T09:38:50.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.832+0000 7f2382ffd640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f237006a050 0x7f237006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:50.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.832+0000 7f2382ffd640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f237006a050 0x7f237006c510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2378007fd0 tx=0x7f2378007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:50.836+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23740541b0 con 0x7f238406aa70 2023-08-25T09:38:51.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.012+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f23841406e0 con 0x7f238406aa70 2023-08-25T09:38:51.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.118+0000 7f236bfff640 1 -- 172.21.15.70:0/2377036267 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f23740577d0 con 0x7f238406aa70 2023-08-25T09:38:51.122 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:38:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f237006a050 msgr2=0x7f237006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f237006a050 0x7f237006c510 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2378007fd0 tx=0x7f2378007920 comp rx=0 tx=0).stop 2023-08-25T09:38:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 msgr2=0x7f238413bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238413bf70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f237400d800 tx=0x7f237400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 shutdown_connections 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f237006a050 0x7f237006c510 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2384142c30 0x7f238413ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2384142740 0x7f238415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 --2- 172.21.15.70:0/2377036267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f238406aa70 0x7f238413bf70 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f237400d800 tx=0x7f237400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.122+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 >> 172.21.15.70:0/2377036267 conn(0x7f2384069c70 msgr2=0x7f2384158e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:51.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.123+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 shutdown_connections 2023-08-25T09:38:51.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.124+0000 7f238a176640 1 -- 172.21.15.70:0/2377036267 wait complete. 2023-08-25T09:38:51.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:38:51.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.363+0000 7f09288ec640 1 Processor -- start 2023-08-25T09:38:51.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.363+0000 7f09288ec640 1 -- start start 2023-08-25T09:38:51.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f09288ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09240c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:51.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f09288ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 0x7f09240ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:51.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f09288ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09240cc5f0 con 0x7f09240cdc20 2023-08-25T09:38:51.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f09288ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0924070650 con 0x7f0924069f60 2023-08-25T09:38:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f09288ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0924070930 con 0x7f092415ae50 2023-08-25T09:38:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f092259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09240c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:51.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.365+0000 7f0922d9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 0x7f09240ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:51.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f0922d9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 0x7f09240ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:60490/0 (socket says 172.21.15.70:60490) 2023-08-25T09:38:51.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f092259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09240c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46578/0 (socket says 172.21.15.70:46578) 2023-08-25T09:38:51.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f0922d9d640 1 -- 172.21.15.70:0/880228470 learned_addr learned my addr 172.21.15.70:0/880228470 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:51.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f092259c640 1 -- 172.21.15.70:0/880228470 >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 msgr2=0x7f09240ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:51.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f092259c640 1 --2- 172.21.15.70:0/880228470 >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 0x7f09240ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f092259c640 1 -- 172.21.15.70:0/880228470 >> v1:172.21.15.155:6789/0 conn(0x7f09240cdc20 legacy=0x7f0924069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:51.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f092259c640 1 -- 172.21.15.70:0/880228470 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0924070c10 con 0x7f092415ae50 2023-08-25T09:38:51.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.366+0000 7f0922d9d640 1 --2- 172.21.15.70:0/880228470 >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 0x7f09240ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:51.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.368+0000 7f092259c640 1 --2- 172.21.15.70:0/880228470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09240c2110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09140097a0 tx=0x7f091402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e0fb6b55b26da8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:51.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.368+0000 7f092159a640 1 -- 172.21.15.70:0/880228470 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914030800 con 0x7f092415ae50 2023-08-25T09:38:51.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.368+0000 7f092159a640 1 -- 172.21.15.70:0/880228470 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914030960 con 0x7f092415ae50 2023-08-25T09:38:51.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.368+0000 7f092159a640 1 -- 172.21.15.70:0/880228470 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914030cf0 con 0x7f092415ae50 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 -- 172.21.15.70:0/880228470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 msgr2=0x7f09240c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 --2- 172.21.15.70:0/880228470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09240c2110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f09140097a0 tx=0x7f091402b940 comp rx=0 tx=0).stop 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 -- 172.21.15.70:0/880228470 shutdown_connections 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 --2- 172.21.15.70:0/880228470 >> v2:172.21.15.155:3300/0 conn(0x7f0924069f60 0x7f09240ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 --2- 172.21.15.70:0/880228470 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09240c2110 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 -- 172.21.15.70:0/880228470 >> 172.21.15.70:0/880228470 conn(0x7f09240cb0d0 msgr2=0x7f09240cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 -- 172.21.15.70:0/880228470 shutdown_connections 2023-08-25T09:38:51.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.369+0000 7f09288ec640 1 -- 172.21.15.70:0/880228470 wait complete. 2023-08-25T09:38:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.370+0000 7f09288ec640 1 Processor -- start 2023-08-25T09:38:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.371+0000 7f09288ec640 1 -- start start 2023-08-25T09:38:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.371+0000 7f09288ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0924069f60 0x7f09241f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.371+0000 7f09288ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 0x7f09240b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.371+0000 7f09288ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09241f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f09288ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240c9670 con 0x7f09240cdc20 2023-08-25T09:38:51.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f09288ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b3c90 con 0x7f0924069f60 2023-08-25T09:38:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f09288ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240c8d40 con 0x7f092415ae50 2023-08-25T09:38:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 0x7f09240b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 0x7f09240b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56552/0 (socket says 172.21.15.70:56552) 2023-08-25T09:38:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 -- 172.21.15.70:0/741979894 learned_addr learned my addr 172.21.15.70:0/741979894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:51.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0922d9d640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09241f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:51.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f092259c640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0924069f60 0x7f09241f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:51.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 -- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 msgr2=0x7f09241f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09241f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 -- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0924069f60 msgr2=0x7f09241f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0924069f60 0x7f09241f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.372+0000 7f0921d9b640 1 -- 172.21.15.70:0/741979894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c009b30 con 0x7f09240cdc20 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.373+0000 7f0921d9b640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 0x7f09240b33b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f091800c400 tx=0x7f091800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.373+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09180052d0 con 0x7f09240cdc20 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.373+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0918005450 con 0x7f09240cdc20 2023-08-25T09:38:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.373+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918007260 con 0x7f09240cdc20 2023-08-25T09:38:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.373+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0914009400 con 0x7f09240cdc20 2023-08-25T09:38:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.373+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09240c9300 con 0x7f09240cdc20 2023-08-25T09:38:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.374+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0924066fb0 con 0x7f09240cdc20 2023-08-25T09:38:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.375+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f09180073c0 con 0x7f09240cdc20 2023-08-25T09:38:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.376+0000 7f0912ffd640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090406a050 0x7f090406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.376+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f0918087830 con 0x7f09240cdc20 2023-08-25T09:38:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.376+0000 7f092259c640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090406a050 0x7f090406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.377+0000 7f092259c640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090406a050 0x7f090406c510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0914036000 tx=0x7f091403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.381+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09180530f0 con 0x7f09240cdc20 2023-08-25T09:38:51.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:51.571+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f09240b40e0 con 0x7f09240cdc20 2023-08-25T09:38:52.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.124+0000 7f0912ffd640 1 -- 172.21.15.70:0/741979894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0918056710 con 0x7f09240cdc20 2023-08-25T09:38:52.128 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:38:52.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.132+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090406a050 msgr2=0x7f090406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:52.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.132+0000 7f09288ec640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090406a050 0x7f090406c510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0914036000 tx=0x7f091403b040 comp rx=0 tx=0).stop 2023-08-25T09:38:52.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.132+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 msgr2=0x7f09240b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:52.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.132+0000 7f09288ec640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 0x7f09240b33b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f091800c400 tx=0x7f091800c8c0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.139+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 shutdown_connections 2023-08-25T09:38:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.139+0000 7f09288ec640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090406a050 0x7f090406c510 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.139+0000 7f09288ec640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f092415ae50 0x7f09241f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.139+0000 7f09288ec640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09240cdc20 0x7f09240b33b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.139+0000 7f09288ec640 1 --2- 172.21.15.70:0/741979894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0924069f60 0x7f09241f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.139+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 >> 172.21.15.70:0/741979894 conn(0x7f09240cb0d0 msgr2=0x7f09240c1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:52.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.143+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 shutdown_connections 2023-08-25T09:38:52.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.143+0000 7f09288ec640 1 -- 172.21.15.70:0/741979894 wait complete. 2023-08-25T09:38:52.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:38:52.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.359+0000 7f715d347640 1 Processor -- start 2023-08-25T09:38:52.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.359+0000 7f715d347640 1 -- start start 2023-08-25T09:38:52.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.361+0000 7f715d347640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71580cd850 0x7f71580cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:52.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.361+0000 7f715d347640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71580d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:52.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.361+0000 7f715d347640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71580b6cc0 con 0x7f71580b8510 2023-08-25T09:38:52.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.361+0000 7f715d347640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71580ce170 con 0x7f71580cd850 2023-08-25T09:38:52.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.361+0000 7f715d347640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71580d1500 con 0x7f715815cfe0 2023-08-25T09:38:52.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71580d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:52.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f7156ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71580cd850 0x7f71580cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:52.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71580d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46594/0 (socket says 172.21.15.70:46594) 2023-08-25T09:38:52.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 -- 172.21.15.70:0/2458545454 learned_addr learned my addr 172.21.15.70:0/2458545454 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:52.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 -- 172.21.15.70:0/2458545454 >> v2:172.21.15.155:3300/0 conn(0x7f71580cd850 msgr2=0x7f71580cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:52.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 --2- 172.21.15.70:0/2458545454 >> v2:172.21.15.155:3300/0 conn(0x7f71580cd850 0x7f71580cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 -- 172.21.15.70:0/2458545454 >> v1:172.21.15.155:6789/0 conn(0x7f71580b8510 legacy=0x7f71580b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:52.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.362+0000 7f714e7fc640 1 -- 172.21.15.70:0/2458545454 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580d17e0 con 0x7f715815cfe0 2023-08-25T09:38:52.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.364+0000 7f714e7fc640 1 --2- 172.21.15.70:0/2458545454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71580d0c00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7140009870 tx=0x7f714002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7ba3950e72bce10 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:52.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.364+0000 7f71567fc640 1 -- 172.21.15.70:0/2458545454 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140030840 con 0x7f715815cfe0 2023-08-25T09:38:52.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.364+0000 7f71567fc640 1 -- 172.21.15.70:0/2458545454 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71400309a0 con 0x7f715815cfe0 2023-08-25T09:38:52.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.364+0000 7f71567fc640 1 -- 172.21.15.70:0/2458545454 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140030d30 con 0x7f715815cfe0 2023-08-25T09:38:52.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 -- 172.21.15.70:0/2458545454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 msgr2=0x7f71580d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:52.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 --2- 172.21.15.70:0/2458545454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71580d0c00 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7140009870 tx=0x7f714002b980 comp rx=0 tx=0).stop 2023-08-25T09:38:52.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 -- 172.21.15.70:0/2458545454 shutdown_connections 2023-08-25T09:38:52.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 --2- 172.21.15.70:0/2458545454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71580d0c00 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 --2- 172.21.15.70:0/2458545454 >> v2:172.21.15.155:3300/0 conn(0x7f71580cd850 0x7f71580cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 -- 172.21.15.70:0/2458545454 >> 172.21.15.70:0/2458545454 conn(0x7f7158069850 msgr2=0x7f7158069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:52.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 -- 172.21.15.70:0/2458545454 shutdown_connections 2023-08-25T09:38:52.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.365+0000 7f715d347640 1 -- 172.21.15.70:0/2458545454 wait complete. 2023-08-25T09:38:52.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.367+0000 7f715d347640 1 Processor -- start 2023-08-25T09:38:52.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.367+0000 7f715d347640 1 -- start start 2023-08-25T09:38:52.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f715d347640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71580b8510 0x7f71580cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:52.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f715d347640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 0x7f715815e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:52.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f715d347640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71581f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:52.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f715d347640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580c9be0 con 0x7f71580cd850 2023-08-25T09:38:52.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f715d347640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715815ec60 con 0x7f71580b8510 2023-08-25T09:38:52.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f715d347640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715815ef40 con 0x7f715815cfe0 2023-08-25T09:38:52.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f714e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 0x7f715815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:52.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f714e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 0x7f715815e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56568/0 (socket says 172.21.15.70:56568) 2023-08-25T09:38:52.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.368+0000 7f714e7fc640 1 -- 172.21.15.70:0/3508963113 learned_addr learned my addr 172.21.15.70:0/3508963113 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:52.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f71577fe640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71581f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:52.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f7156ffd640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71580b8510 0x7f71580cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714e7fc640 1 -- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 msgr2=0x7f71581f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714e7fc640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71581f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714e7fc640 1 -- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71580b8510 msgr2=0x7f71580cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714e7fc640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71580b8510 0x7f71580cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714e7fc640 1 -- 172.21.15.70:0/3508963113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400094d0 con 0x7f71580cd850 2023-08-25T09:38:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714e7fc640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 0x7f715815e410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7140000c00 tx=0x7f7140004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.369+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140039c90 con 0x7f71580cd850 2023-08-25T09:38:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.370+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7140039df0 con 0x7f71580cd850 2023-08-25T09:38:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.370+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140043640 con 0x7f71580cd850 2023-08-25T09:38:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.370+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715815f220 con 0x7f71580cd850 2023-08-25T09:38:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.370+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715815e950 con 0x7f71580cd850 2023-08-25T09:38:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.371+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715806a700 con 0x7f71580cd850 2023-08-25T09:38:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.372+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f71400437a0 con 0x7f71580cd850 2023-08-25T09:38:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.373+0000 7f714ffff640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713806a080 0x7f713806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.373+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f714003e070 con 0x7f71580cd850 2023-08-25T09:38:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.374+0000 7f7156ffd640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713806a080 0x7f713806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.374+0000 7f7156ffd640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713806a080 0x7f713806c540 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f71480074a0 tx=0x7f714800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.378+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7140076c40 con 0x7f71580cd850 2023-08-25T09:38:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:52.605+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f71580ca070 con 0x7f71580cd850 2023-08-25T09:38:53.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.185+0000 7f714ffff640 1 -- 172.21.15.70:0/3508963113 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f71580ca070 con 0x7f71580cd850 2023-08-25T09:38:53.188 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 3 and data pool 4 2023-08-25T09:38:53.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713806a080 msgr2=0x7f713806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713806a080 0x7f713806c540 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f71480074a0 tx=0x7f714800b040 comp rx=0 tx=0).stop 2023-08-25T09:38:53.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 msgr2=0x7f715815e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 0x7f715815e410 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7140000c00 tx=0x7f7140004730 comp rx=0 tx=0).stop 2023-08-25T09:38:53.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 shutdown_connections 2023-08-25T09:38:53.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713806a080 0x7f713806c540 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f715815cfe0 0x7f71581f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71580cd850 0x7f715815e410 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 --2- 172.21.15.70:0/3508963113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71580b8510 0x7f71580cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 >> 172.21.15.70:0/3508963113 conn(0x7f7158069850 msgr2=0x7f71580c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:53.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 shutdown_connections 2023-08-25T09:38:53.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.188+0000 7f715d347640 1 -- 172.21.15.70:0/3508963113 wait complete. 2023-08-25T09:38:53.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.419+0000 7fb67d760640 1 Processor -- start 2023-08-25T09:38:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.420+0000 7fb67d760640 1 -- start start 2023-08-25T09:38:53.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67d760640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb67815cb80 0x7fb6780c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67d760640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb678072f30 0x7fb6780cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67d760640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67806d170 con 0x7fb6780b1720 2023-08-25T09:38:53.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67d760640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67806a060 con 0x7fb678072f30 2023-08-25T09:38:53.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67d760640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb678069850 con 0x7fb67815cb80 2023-08-25T09:38:53.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67cf5f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb678072f30 0x7fb6780cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67cf5f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb678072f30 0x7fb6780cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56578/0 (socket says 172.21.15.70:56578) 2023-08-25T09:38:53.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.421+0000 7fb67cf5f640 1 -- 172.21.15.70:0/1069602109 learned_addr learned my addr 172.21.15.70:0/1069602109 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:53.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.422+0000 7fb677fff640 1 --2- 172.21.15.70:0/1069602109 >> v2:172.21.15.155:3300/0 conn(0x7fb67815cb80 0x7fb6780c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.422+0000 7fb67cf5f640 1 -- 172.21.15.70:0/1069602109 >> v2:172.21.15.155:3300/0 conn(0x7fb67815cb80 msgr2=0x7fb6780c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:53.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.422+0000 7fb67cf5f640 1 --2- 172.21.15.70:0/1069602109 >> v2:172.21.15.155:3300/0 conn(0x7fb67815cb80 0x7fb6780c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.422+0000 7fb67cf5f640 1 -- 172.21.15.70:0/1069602109 >> v1:172.21.15.70:6789/0 conn(0x7fb6780b1720 legacy=0x7fb678072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.422+0000 7fb67cf5f640 1 -- 172.21.15.70:0/1069602109 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780c8b10 con 0x7fb678072f30 2023-08-25T09:38:53.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.422+0000 7fb676ffd640 1 -- 172.21.15.70:0/1069602109 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936437884 0 0) 0x7fb67806d170 con 0x7fb6780b1720 2023-08-25T09:38:53.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.423+0000 7fb67cf5f640 1 --2- 172.21.15.70:0/1069602109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb6780cab80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb678073bc0 tx=0x7fb67002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ea4c9b6eee63b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:53.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.423+0000 7fb676ffd640 1 -- 172.21.15.70:0/1069602109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67002cb20 con 0x7fb678072f30 2023-08-25T09:38:53.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.423+0000 7fb676ffd640 1 -- 172.21.15.70:0/1069602109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67002cc80 con 0x7fb678072f30 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.423+0000 7fb676ffd640 1 -- 172.21.15.70:0/1069602109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6700343f0 con 0x7fb678072f30 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 -- 172.21.15.70:0/1069602109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 msgr2=0x7fb6780cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 --2- 172.21.15.70:0/1069602109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb6780cab80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb678073bc0 tx=0x7fb67002bf80 comp rx=0 tx=0).stop 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 -- 172.21.15.70:0/1069602109 shutdown_connections 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 --2- 172.21.15.70:0/1069602109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb6780cab80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 --2- 172.21.15.70:0/1069602109 >> v2:172.21.15.155:3300/0 conn(0x7fb67815cb80 0x7fb6780c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 -- 172.21.15.70:0/1069602109 >> 172.21.15.70:0/1069602109 conn(0x7fb6780cc0a0 msgr2=0x7fb6780cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:53.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 -- 172.21.15.70:0/1069602109 shutdown_connections 2023-08-25T09:38:53.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.424+0000 7fb67d760640 1 -- 172.21.15.70:0/1069602109 wait complete. 2023-08-25T09:38:53.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.426+0000 7fb67d760640 1 Processor -- start 2023-08-25T09:38:53.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.426+0000 7fb67d760640 1 -- start start 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67d760640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb678158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67d760640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6780b1720 0x7fb67815a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67d760640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67815cb80 0x7fb678152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67d760640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6781529d0 con 0x7fb678072f30 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67d760640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6781553e0 con 0x7fb67815cb80 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67d760640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb678159e60 con 0x7fb6780b1720 2023-08-25T09:38:53.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb677fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb678158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb677fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb678158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56584/0 (socket says 172.21.15.70:56584) 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb677fff640 1 -- 172.21.15.70:0/1364465651 learned_addr learned my addr 172.21.15.70:0/1364465651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb677fff640 1 -- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6780b1720 msgr2=0x7fb67815a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb6777fe640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6780b1720 0x7fb67815a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb677fff640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6780b1720 0x7fb67815a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb67cf5f640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67815cb80 0x7fb678152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.427+0000 7fb677fff640 1 -- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67815cb80 msgr2=0x7fb678152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb677fff640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67815cb80 0x7fb678152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb677fff640 1 -- 172.21.15.70:0/1364465651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb670009b30 con 0x7fb678072f30 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb6777fe640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6780b1720 0x7fb67815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb677fff640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb678158c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb66800d5f0 tx=0x7fb66800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680125f0 con 0x7fb678072f30 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668012750 con 0x7fb678072f30 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668012b00 con 0x7fb678072f30 2023-08-25T09:38:53.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6781592e0 con 0x7fb678072f30 2023-08-25T09:38:53.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.428+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6781598a0 con 0x7fb678072f30 2023-08-25T09:38:53.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.429+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb678066fb0 con 0x7fb678072f30 2023-08-25T09:38:53.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.430+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb668012c60 con 0x7fb678072f30 2023-08-25T09:38:53.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.431+0000 7fb674ff9640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66406a050 0x7fb66406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.431+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb668088a60 con 0x7fb678072f30 2023-08-25T09:38:53.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.431+0000 7fb6777fe640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66406a050 0x7fb66406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.432+0000 7fb6777fe640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66406a050 0x7fb66406c510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb678072ed0 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:53.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.435+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb668054140 con 0x7fb678072f30 2023-08-25T09:38:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.624+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6780b2310 con 0x7fb678072f30 2023-08-25T09:38:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.626+0000 7fb674ff9640 1 -- 172.21.15.70:0/1364465651 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18378 (secure 0 0 0) 0x7fb668057760 con 0x7fb678072f30 2023-08-25T09:38:53.629 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:53.629 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":22,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:52.607578+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:53.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66406a050 msgr2=0x7fb66406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66406a050 0x7fb66406c510 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb678072ed0 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2023-08-25T09:38:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 msgr2=0x7fb678158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb678158c50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb66800d5f0 tx=0x7fb66800dab0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 shutdown_connections 2023-08-25T09:38:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66406a050 0x7fb66406c510 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67815cb80 0x7fb678152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6780b1720 0x7fb67815a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 --2- 172.21.15.70:0/1364465651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb678072f30 0x7fb678158c50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 >> 172.21.15.70:0/1364465651 conn(0x7fb6780cc0a0 msgr2=0x7fb6780ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 shutdown_connections 2023-08-25T09:38:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.629+0000 7fb67d760640 1 -- 172.21.15.70:0/1364465651 wait complete. 2023-08-25T09:38:53.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:53.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.868+0000 7f2b293c4640 1 Processor -- start 2023-08-25T09:38:53.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.868+0000 7f2b293c4640 1 -- start start 2023-08-25T09:38:53.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.869+0000 7f2b293c4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b240ce5b0 0x7f2b240ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.869+0000 7f2b293c4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2415aa10 0x7f2b240b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.869+0000 7f2b293c4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b240ceed0 con 0x7f2b240b2680 2023-08-25T09:38:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.869+0000 7f2b293c4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b240b2b10 con 0x7f2b240ce5b0 2023-08-25T09:38:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.869+0000 7f2b293c4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b24159d10 con 0x7f2b2415aa10 2023-08-25T09:38:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b240ce5b0 0x7f2b240ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2b240ce5b0 0x7f2b240ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56588/0 (socket says 172.21.15.70:56588) 2023-08-25T09:38:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 -- 172.21.15.70:0/3436493916 learned_addr learned my addr 172.21.15.70:0/3436493916 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b237fe640 1 --1- 172.21.15.70:0/3436493916 >> v1:172.21.15.70:6789/0 conn(0x7f2b240b2680 0x7f2b2406be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37962/0 (socket says 172.21.15.70:37962) 2023-08-25T09:38:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b1a7fc640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2415aa10 0x7f2b240b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 -- 172.21.15.70:0/3436493916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2415aa10 msgr2=0x7f2b240b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2415aa10 0x7f2b240b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 -- 172.21.15.70:0/3436493916 >> v1:172.21.15.70:6789/0 conn(0x7f2b240b2680 legacy=0x7f2b2406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b22ffd640 1 -- 172.21.15.70:0/3436493916 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2415d3f0 con 0x7f2b240ce5b0 2023-08-25T09:38:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b1a7fc640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2415aa10 0x7f2b240b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.870+0000 7f2b227fc640 1 -- 172.21.15.70:0/3436493916 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 703794368 0 0) 0x7f2b240ceed0 con 0x7f2b240b2680 2023-08-25T09:38:53.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.871+0000 7f2b22ffd640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240ce5b0 0x7f2b240ce990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009870 tx=0x7f2b1402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe033a8813e69ba6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.871+0000 7f2b227fc640 1 -- 172.21.15.70:0/3436493916 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14030840 con 0x7f2b240ce5b0 2023-08-25T09:38:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.871+0000 7f2b227fc640 1 -- 172.21.15.70:0/3436493916 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b140309a0 con 0x7f2b240ce5b0 2023-08-25T09:38:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.871+0000 7f2b227fc640 1 -- 172.21.15.70:0/3436493916 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14030d30 con 0x7f2b240ce5b0 2023-08-25T09:38:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 -- 172.21.15.70:0/3436493916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240ce5b0 msgr2=0x7f2b240ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240ce5b0 0x7f2b240ce990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009870 tx=0x7f2b1402b980 comp rx=0 tx=0).stop 2023-08-25T09:38:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 -- 172.21.15.70:0/3436493916 shutdown_connections 2023-08-25T09:38:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b2415aa10 0x7f2b240b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 --2- 172.21.15.70:0/3436493916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240ce5b0 0x7f2b240ce990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 -- 172.21.15.70:0/3436493916 >> 172.21.15.70:0/3436493916 conn(0x7f2b24069850 msgr2=0x7f2b24069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 -- 172.21.15.70:0/3436493916 shutdown_connections 2023-08-25T09:38:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.872+0000 7f2b293c4640 1 -- 172.21.15.70:0/3436493916 wait complete. 2023-08-25T09:38:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.874+0000 7f2b293c4640 1 Processor -- start 2023-08-25T09:38:53.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.874+0000 7f2b293c4640 1 -- start start 2023-08-25T09:38:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b293c4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 0x7f2b24148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b293c4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b240ce5b0 0x7f2b240b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b293c4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2415aa10 0x7f2b24156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b293c4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240b06b0 con 0x7f2b240b2680 2023-08-25T09:38:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b293c4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b241486d0 con 0x7f2b2415aa10 2023-08-25T09:38:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b293c4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240afd80 con 0x7f2b240ce5b0 2023-08-25T09:38:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b22ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 0x7f2b24148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b22ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 0x7f2b24148190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56590/0 (socket says 172.21.15.70:56590) 2023-08-25T09:38:53.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.875+0000 7f2b22ffd640 1 -- 172.21.15.70:0/1284260002 learned_addr learned my addr 172.21.15.70:0/1284260002 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:53.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b22ffd640 1 -- 172.21.15.70:0/1284260002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b240ce5b0 msgr2=0x7f2b240b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b22ffd640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b240ce5b0 0x7f2b240b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b22ffd640 1 -- 172.21.15.70:0/1284260002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2415aa10 msgr2=0x7f2b24156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b22ffd640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2415aa10 0x7f2b24156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b22ffd640 1 -- 172.21.15.70:0/1284260002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b140094d0 con 0x7f2b240b2680 2023-08-25T09:38:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b22ffd640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 0x7f2b24148190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000c00 tx=0x7f2b14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14030810 con 0x7f2b240b2680 2023-08-25T09:38:53.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14043400 con 0x7f2b240b2680 2023-08-25T09:38:53.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b140437d0 con 0x7f2b240b2680 2023-08-25T09:38:53.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.876+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b240b0060 con 0x7f2b240b2680 2023-08-25T09:38:53.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.877+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b241e3c90 con 0x7f2b240b2680 2023-08-25T09:38:53.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.878+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2404e420 con 0x7f2b240b2680 2023-08-25T09:38:53.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.879+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2b14043930 con 0x7f2b240b2680 2023-08-25T09:38:53.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.880+0000 7f2b1bfff640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2b0406a080 0x7f2b0406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:53.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.880+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b1403e070 con 0x7f2b240b2680 2023-08-25T09:38:53.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.880+0000 7f2b1a7fc640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2b0406a080 0x7f2b0406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:53.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.881+0000 7f2b1a7fc640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2b0406a080 0x7f2b0406c540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006fd0 tx=0x7f2b0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:53.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:53.884+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2b14076f20 con 0x7f2b240b2680 2023-08-25T09:38:54.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.074+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b240cae50 con 0x7f2b240b2680 2023-08-25T09:38:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.080+0000 7f2b1bfff640 1 -- 172.21.15.70:0/1284260002 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f2b1407a540 con 0x7f2b240b2680 2023-08-25T09:38:54.083 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:54.084 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2b0406a080 msgr2=0x7f2b0406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2b0406a080 0x7f2b0406c540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c006fd0 tx=0x7f2b0c008040 comp rx=0 tx=0).stop 2023-08-25T09:38:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 msgr2=0x7f2b24148190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 0x7f2b24148190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000c00 tx=0x7f2b14004730 comp rx=0 tx=0).stop 2023-08-25T09:38:54.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 shutdown_connections 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2b0406a080 0x7f2b0406c540 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2b2415aa10 0x7f2b24156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b240ce5b0 0x7f2b240b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 --2- 172.21.15.70:0/1284260002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b240b2680 0x7f2b24148190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.083+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 >> 172.21.15.70:0/1284260002 conn(0x7f2b24069850 msgr2=0x7f2b24069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.084+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 shutdown_connections 2023-08-25T09:38:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.084+0000 7f2b293c4640 1 -- 172.21.15.70:0/1284260002 wait complete. 2023-08-25T09:38:54.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:38:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.321+0000 7f4d21606640 1 Processor -- start 2023-08-25T09:38:54.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.321+0000 7f4d21606640 1 -- start start 2023-08-25T09:38:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d21606640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d21606640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d1c06f0e0 0x7f4d1c070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d21606640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c15e780 con 0x7f4d1c0cd660 2023-08-25T09:38:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d21606640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c06f8e0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d21606640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c06fbc0 con 0x7f4d1c159bf0 2023-08-25T09:38:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d1b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4d1c0cd660 0x7f4d1c0c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37966/0 (socket says 172.21.15.70:37966) 2023-08-25T09:38:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d1b7fe640 1 -- 172.21.15.70:0/745393830 learned_addr learned my addr 172.21.15.70:0/745393830 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.323+0000 7f4d1a7fc640 1 --2- 172.21.15.70:0/745393830 >> v2:172.21.15.70:3300/0 conn(0x7f4d1c06f0e0 0x7f4d1c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.324+0000 7f4d1affd640 1 --2- 172.21.15.70:0/745393830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.324+0000 7f4d1a7fc640 1 -- 172.21.15.70:0/745393830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 msgr2=0x7f4d1c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:54.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.324+0000 7f4d1a7fc640 1 --2- 172.21.15.70:0/745393830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.324+0000 7f4d19ffb640 1 -- 172.21.15.70:0/745393830 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472223041 0 0) 0x7f4d1c15e780 con 0x7f4d1c0cd660 2023-08-25T09:38:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.324+0000 7f4d1a7fc640 1 -- 172.21.15.70:0/745393830 >> v1:172.21.15.70:6789/0 conn(0x7f4d1c0cd660 legacy=0x7f4d1c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.324+0000 7f4d1a7fc640 1 -- 172.21.15.70:0/745393830 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c06fea0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.325+0000 7f4d1a7fc640 1 --2- 172.21.15.70:0/745393830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c070980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d040097a0 tx=0x7f4d0402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130b348c3d216b4d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.325+0000 7f4d19ffb640 1 -- 172.21.15.70:0/745393830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0402d040 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.325+0000 7f4d19ffb640 1 -- 172.21.15.70:0/745393830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d04031810 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.325+0000 7f4d19ffb640 1 -- 172.21.15.70:0/745393830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04031ba0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 -- 172.21.15.70:0/745393830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 msgr2=0x7f4d1c070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 --2- 172.21.15.70:0/745393830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c070980 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d040097a0 tx=0x7f4d0402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 -- 172.21.15.70:0/745393830 shutdown_connections 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 --2- 172.21.15.70:0/745393830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c070980 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 --2- 172.21.15.70:0/745393830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 -- 172.21.15.70:0/745393830 >> 172.21.15.70:0/745393830 conn(0x7f4d1c0cb0d0 msgr2=0x7f4d1c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 -- 172.21.15.70:0/745393830 shutdown_connections 2023-08-25T09:38:54.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.326+0000 7f4d21606640 1 -- 172.21.15.70:0/745393830 wait complete. 2023-08-25T09:38:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.328+0000 7f4d21606640 1 Processor -- start 2023-08-25T09:38:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.328+0000 7f4d21606640 1 -- start start 2023-08-25T09:38:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d21606640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c1f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d21606640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c0cd660 0x7f4d1c1e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d21606640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c1e2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:54.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d21606640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1e5c40 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d21606640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c3ee0 con 0x7f4d1c0cd660 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d21606640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1e5ec0 con 0x7f4d1c159bf0 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c1f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c1f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56612/0 (socket says 172.21.15.70:56612) 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 -- 172.21.15.70:0/2842698207 learned_addr learned my addr 172.21.15.70:0/2842698207 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 -- 172.21.15.70:0/2842698207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 msgr2=0x7f4d1c1e2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c1e2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 -- 172.21.15.70:0/2842698207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c0cd660 msgr2=0x7f4d1c1e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c0cd660 0x7f4d1c1e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.329+0000 7f4d1affd640 1 -- 172.21.15.70:0/2842698207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04009400 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.330+0000 7f4d1affd640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c1f1ac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00d5f0 tx=0x7f4d0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.330+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c0125f0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.330+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c012750 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.330+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c012b00 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.330+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c1e61a0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.330+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c1f2000 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.331+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c070e70 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.332+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4d0c012c60 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.333+0000 7f4d137fe640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cfc06a050 0x7f4cfc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.333+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d0c088ac0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.333+0000 7f4d1a7fc640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cfc06a050 0x7f4cfc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:54.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.334+0000 7f4d1a7fc640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cfc06a050 0x7f4cfc06c510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d04036000 tx=0x7f4d04030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:54.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.337+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4d0c0541b0 con 0x7f4d1c06f0e0 2023-08-25T09:38:54.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:54.573+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4d1c066fb0 con 0x7f4d1c06f0e0 2023-08-25T09:38:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.205+0000 7f4d137fe640 1 -- 172.21.15.70:0/2842698207 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4d0c0577d0 con 0x7f4d1c06f0e0 2023-08-25T09:38:55.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cfc06a050 msgr2=0x7f4cfc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cfc06a050 0x7f4cfc06c510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d04036000 tx=0x7f4d04030000 comp rx=0 tx=0).stop 2023-08-25T09:38:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 msgr2=0x7f4d1c1f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c1f1ac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00d5f0 tx=0x7f4d0c00dab0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 shutdown_connections 2023-08-25T09:38:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cfc06a050 0x7f4cfc06c510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d1c159bf0 0x7f4d1c1e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d1c0cd660 0x7f4d1c1e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 --2- 172.21.15.70:0/2842698207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d1c06f0e0 0x7f4d1c1f1ac0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 >> 172.21.15.70:0/2842698207 conn(0x7f4d1c0cb0d0 msgr2=0x7f4d1c0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 shutdown_connections 2023-08-25T09:38:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.209+0000 7f4d21606640 1 -- 172.21.15.70:0/2842698207 wait complete. 2023-08-25T09:38:55.231 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2023-08-25T09:38:55.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:38:55.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.448+0000 7fcd8c799640 1 Processor -- start 2023-08-25T09:38:55.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.448+0000 7fcd8c799640 1 -- start start 2023-08-25T09:38:55.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd8c799640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd840c2110 0x7fcd84069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd8c799640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd84069d90 0x7fcd840ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd8c799640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd840c2550 con 0x7fcd8415ae50 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd8c799640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8406a200 con 0x7fcd840c2110 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd8c799640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd840ce330 con 0x7fcd84069d90 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd89d34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd840c2110 0x7fcd84069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd89d34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd840c2110 0x7fcd84069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56614/0 (socket says 172.21.15.70:56614) 2023-08-25T09:38:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd89d34640 1 -- 172.21.15.70:0/3023128371 learned_addr learned my addr 172.21.15.70:0/3023128371 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:55.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.450+0000 7fcd8ad36640 1 --2- 172.21.15.70:0/3023128371 >> v2:172.21.15.155:3300/0 conn(0x7fcd84069d90 0x7fcd840ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.451+0000 7fcd89d34640 1 -- 172.21.15.70:0/3023128371 >> v2:172.21.15.155:3300/0 conn(0x7fcd84069d90 msgr2=0x7fcd840ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.451+0000 7fcd89d34640 1 --2- 172.21.15.70:0/3023128371 >> v2:172.21.15.155:3300/0 conn(0x7fcd84069d90 0x7fcd840ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.451+0000 7fcd89d34640 1 -- 172.21.15.70:0/3023128371 >> v1:172.21.15.155:6789/0 conn(0x7fcd8415ae50 legacy=0x7fcd840cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.451+0000 7fcd89d34640 1 -- 172.21.15.70:0/3023128371 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd840cc5f0 con 0x7fcd840c2110 2023-08-25T09:38:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.451+0000 7fcd8ad36640 1 --2- 172.21.15.70:0/3023128371 >> v2:172.21.15.155:3300/0 conn(0x7fcd84069d90 0x7fcd840ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:55.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.452+0000 7fcd89d34640 1 --2- 172.21.15.70:0/3023128371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd840c2110 0x7fcd84069850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd840b00d0 tx=0x7fcd7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1928fb7b3eb2477 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.452+0000 7fcd89533640 1 -- 172.21.15.70:0/3023128371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74030860 con 0x7fcd840c2110 2023-08-25T09:38:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.452+0000 7fcd89533640 1 -- 172.21.15.70:0/3023128371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd740309c0 con 0x7fcd840c2110 2023-08-25T09:38:55.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.452+0000 7fcd89533640 1 -- 172.21.15.70:0/3023128371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74030d50 con 0x7fcd840c2110 2023-08-25T09:38:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 -- 172.21.15.70:0/3023128371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd840c2110 msgr2=0x7fcd84069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 --2- 172.21.15.70:0/3023128371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd840c2110 0x7fcd84069850 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd840b00d0 tx=0x7fcd7402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 -- 172.21.15.70:0/3023128371 shutdown_connections 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 --2- 172.21.15.70:0/3023128371 >> v2:172.21.15.155:3300/0 conn(0x7fcd84069d90 0x7fcd840ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 --2- 172.21.15.70:0/3023128371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd840c2110 0x7fcd84069850 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 -- 172.21.15.70:0/3023128371 >> 172.21.15.70:0/3023128371 conn(0x7fcd840cb0d0 msgr2=0x7fcd840cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 -- 172.21.15.70:0/3023128371 shutdown_connections 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.453+0000 7fcd8c799640 1 -- 172.21.15.70:0/3023128371 wait complete. 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.455+0000 7fcd8c799640 1 Processor -- start 2023-08-25T09:38:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.455+0000 7fcd8c799640 1 -- start start 2023-08-25T09:38:55.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8c799640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 0x7fcd8406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:55.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8c799640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd840c2110 0x7fcd840acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:55.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8c799640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8415ae50 0x7fcd841589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:55.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8c799640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840af1a0 con 0x7fcd84069d90 2023-08-25T09:38:55.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8c799640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840af440 con 0x7fcd8415ae50 2023-08-25T09:38:55.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8c799640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd840af720 con 0x7fcd840c2110 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 0x7fcd8406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 0x7fcd8406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56618/0 (socket says 172.21.15.70:56618) 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 -- 172.21.15.70:0/1220872020 learned_addr learned my addr 172.21.15.70:0/1220872020 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 -- 172.21.15.70:0/1220872020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd840c2110 msgr2=0x7fcd840acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd840c2110 0x7fcd840acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 -- 172.21.15.70:0/1220872020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8415ae50 msgr2=0x7fcd841589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:55.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8415ae50 0x7fcd841589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:55.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.456+0000 7fcd8a535640 1 -- 172.21.15.70:0/1220872020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740094d0 con 0x7fcd84069d90 2023-08-25T09:38:55.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.457+0000 7fcd8a535640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 0x7fcd8406b4e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b3e0 tx=0x7fcd8000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.457+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80011830 con 0x7fcd84069d90 2023-08-25T09:38:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.457+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd800119b0 con 0x7fcd84069d90 2023-08-25T09:38:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.457+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80011d60 con 0x7fcd84069d90 2023-08-25T09:38:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.457+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd840afa00 con 0x7fcd84069d90 2023-08-25T09:38:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.457+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8415d0c0 con 0x7fcd84069d90 2023-08-25T09:38:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.458+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd84066fb0 con 0x7fcd84069d90 2023-08-25T09:38:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.459+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcd8001a400 con 0x7fcd84069d90 2023-08-25T09:38:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.460+0000 7fcd7affd640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd6806a050 0x7fcd6806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.460+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd800887c0 con 0x7fcd84069d90 2023-08-25T09:38:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.460+0000 7fcd89d34640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd6806a050 0x7fcd6806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:55.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.461+0000 7fcd89d34640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd6806a050 0x7fcd6806c510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd74036000 tx=0x7fcd7403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:55.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.464+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd80053e10 con 0x7fcd84069d90 2023-08-25T09:38:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:55.689+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcd8406a4b0 con 0x7fcd84069d90 2023-08-25T09:38:56.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.220+0000 7fcd7affd640 1 -- 172.21.15.70:0/1220872020 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcd80057430 con 0x7fcd84069d90 2023-08-25T09:38:56.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.223+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd6806a050 msgr2=0x7fcd6806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.223+0000 7fcd8c799640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd6806a050 0x7fcd6806c510 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd74036000 tx=0x7fcd7403b040 comp rx=0 tx=0).stop 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.223+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 msgr2=0x7fcd8406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.223+0000 7fcd8c799640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 0x7fcd8406b4e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd8000b3e0 tx=0x7fcd8000b8a0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 shutdown_connections 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd6806a050 0x7fcd6806c510 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd8415ae50 0x7fcd841589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd840c2110 0x7fcd840acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 --2- 172.21.15.70:0/1220872020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd84069d90 0x7fcd8406b4e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 >> 172.21.15.70:0/1220872020 conn(0x7fcd840cb0d0 msgr2=0x7fcd840c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 shutdown_connections 2023-08-25T09:38:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.224+0000 7fcd8c799640 1 -- 172.21.15.70:0/1220872020 wait complete. 2023-08-25T09:38:56.246 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:38:56.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:38:56.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.460+0000 7f0d93fff640 1 Processor -- start 2023-08-25T09:38:56.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.460+0000 7f0d93fff640 1 -- start start 2023-08-25T09:38:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.461+0000 7f0d93fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d94159840 0x7f0d940b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 0x7f0d940cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d93fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d940b4450 con 0x7f0d9415d3d0 2023-08-25T09:38:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d93fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d940c8b60 con 0x7f0d94159840 2023-08-25T09:38:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d93fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9415d890 con 0x7f0d940cadb0 2023-08-25T09:38:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d927fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d9415d3d0 0x7f0d940b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37978/0 (socket says 172.21.15.70:37978) 2023-08-25T09:38:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d927fc640 1 -- 172.21.15.70:0/1923474758 learned_addr learned my addr 172.21.15.70:0/1923474758 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:56.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d937fe640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 0x7f0d940cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d92ffd640 1 --2- 172.21.15.70:0/1923474758 >> v2:172.21.15.155:3300/0 conn(0x7f0d94159840 0x7f0d940b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763414868 0 0) 0x7f0d940b4450 con 0x7f0d9415d3d0 2023-08-25T09:38:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.462+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d80005730 con 0x7f0d9415d3d0 2023-08-25T09:38:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913794767 0 0) 0x7f0d80005730 con 0x7f0d9415d3d0 2023-08-25T09:38:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 msgr2=0x7f0d940cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 0x7f0d940cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 >> v2:172.21.15.155:3300/0 conn(0x7f0d94159840 msgr2=0x7f0d940b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 --2- 172.21.15.70:0/1923474758 >> v2:172.21.15.155:3300/0 conn(0x7f0d94159840 0x7f0d940b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9415db70 con 0x7f0d9415d3d0 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0d7c001690 con 0x7f0d9415d3d0 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 >> v1:172.21.15.70:6789/0 conn(0x7f0d9415d3d0 legacy=0x7f0d940b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80005b80 0x7f0d80005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 0x7f0d800069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d80006f10 0x7f0d80007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940c8b60 con 0x7f0d80006f10 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9415d890 con 0x7f0d80005b80 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8000db90 con 0x7f0d940cadb0 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d937fe640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d80006f10 0x7f0d80007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d927fc640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 0x7f0d800069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.463+0000 7f0d92ffd640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80005b80 0x7f0d80005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.464+0000 7f0d937fe640 1 -- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 msgr2=0x7f0d800069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.464+0000 7f0d937fe640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d940cadb0 0x7f0d800069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.464+0000 7f0d937fe640 1 -- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80005b80 msgr2=0x7f0d80005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.464+0000 7f0d937fe640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d80005b80 0x7f0d80005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.464+0000 7f0d937fe640 1 -- 172.21.15.70:0/1923474758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80005730 con 0x7f0d80006f10 2023-08-25T09:38:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d937fe640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d80006f10 0x7f0d80007390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009af0 tx=0x7f0d80007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab601e2fb9670ea server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c02c6b0 con 0x7f0d80006f10 2023-08-25T09:38:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c02c810 con 0x7f0d80006f10 2023-08-25T09:38:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d91ffb640 1 -- 172.21.15.70:0/1923474758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c02cba0 con 0x7f0d80006f10 2023-08-25T09:38:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d93fff640 1 -- 172.21.15.70:0/1923474758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d80006f10 msgr2=0x7f0d80007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d93fff640 1 --2- 172.21.15.70:0/1923474758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d80006f10 0x7f0d80007390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009af0 tx=0x7f0d80007a00 comp rx=0 tx=0).stop 2023-08-25T09:38:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.465+0000 7f0d937fe640 1 -- 172.21.15.70:0/1923474758 reap_dead start 2023-08-25T09:38:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.466+0000 7f0d93fff640 1 -- 172.21.15.70:0/1923474758 shutdown_connections 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.466+0000 7f0d93fff640 1 -- 172.21.15.70:0/1923474758 >> 172.21.15.70:0/1923474758 conn(0x7f0d940affd0 msgr2=0x7f0d940b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.466+0000 7f0d93fff640 1 -- 172.21.15.70:0/1923474758 shutdown_connections 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.466+0000 7f0d93fff640 1 -- 172.21.15.70:0/1923474758 wait complete. 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.468+0000 7f0d93fff640 1 Processor -- start 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.468+0000 7f0d93fff640 1 -- start start 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d80006f10 0x7f0d941f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 0x7f0d94070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d93fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d94156a50 0x7f0d94071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d93fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94071650 con 0x7f0d9406f380 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d93fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d941f1d70 con 0x7f0d94156a50 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d93fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d941559b0 con 0x7f0d80006f10 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 0x7f0d94070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 0x7f0d94070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56628/0 (socket says 172.21.15.70:56628) 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 -- 172.21.15.70:0/2068813404 learned_addr learned my addr 172.21.15.70:0/2068813404 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d92ffd640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d80006f10 0x7f0d941f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 -- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d80006f10 msgr2=0x7f0d941f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d80006f10 0x7f0d941f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 -- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d94156a50 msgr2=0x7f0d94071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d94156a50 0x7f0d94071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.469+0000 7f0d927fc640 1 -- 172.21.15.70:0/2068813404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d94155c90 con 0x7f0d9406f380 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.470+0000 7f0d927fc640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 0x7f0d94070bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00e7f0 tx=0x7f0d7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.470+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00c7f0 con 0x7f0d9406f380 2023-08-25T09:38:56.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.470+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00c950 con 0x7f0d9406f380 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.470+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00cd00 con 0x7f0d9406f380 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.470+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d94155f70 con 0x7f0d9406f380 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.470+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d94156530 con 0x7f0d9406f380 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.471+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d94066fb0 con 0x7f0d9406f380 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.472+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0d7c01d460 con 0x7f0d9406f380 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.473+0000 7f0d8b7fe640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d7406a080 0x7f0d7406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.473+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d7c0885f0 con 0x7f0d9406f380 2023-08-25T09:38:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.473+0000 7f0d92ffd640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d7406a080 0x7f0d7406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.474+0000 7f0d92ffd640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d7406a080 0x7f0d7406c540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d84007520 tx=0x7f0d8400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.477+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0d7c053bb0 con 0x7f0d9406f380 2023-08-25T09:38:56.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.710+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d9406b9b0 con 0x7f0d9406f380 2023-08-25T09:38:56.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.711+0000 7f0d8b7fe640 1 -- 172.21.15.70:0/2068813404 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7f0d7c00ce60 con 0x7f0d9406f380 2023-08-25T09:38:56.713 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:56.714 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:38:56.714 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:38:56.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d7406a080 msgr2=0x7f0d7406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d7406a080 0x7f0d7406c540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0d84007520 tx=0x7f0d8400b040 comp rx=0 tx=0).stop 2023-08-25T09:38:56.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 msgr2=0x7f0d94070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 0x7f0d94070bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00e7f0 tx=0x7f0d7c00ecb0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 shutdown_connections 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d7406a080 0x7f0d7406c540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d94156a50 0x7f0d94071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d9406f380 0x7f0d94070bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 --2- 172.21.15.70:0/2068813404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d80006f10 0x7f0d941f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 >> 172.21.15.70:0/2068813404 conn(0x7f0d940affd0 msgr2=0x7f0d94158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:56.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 shutdown_connections 2023-08-25T09:38:56.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.714+0000 7f0d93fff640 1 -- 172.21.15.70:0/2068813404 wait complete. 2023-08-25T09:38:56.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.939+0000 7f472bc1f640 1 Processor -- start 2023-08-25T09:38:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.939+0000 7f472bc1f640 1 -- start start 2023-08-25T09:38:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.940+0000 7f472bc1f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f472406d3d0 0x7f472406d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.940+0000 7f472bc1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f472406def0 0x7f4724166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.940+0000 7f472bc1f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f472406e320 con 0x7f472415cc00 2023-08-25T09:38:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.940+0000 7f472bc1f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472415cfe0 con 0x7f472406def0 2023-08-25T09:38:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.940+0000 7f472bc1f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47240cfdb0 con 0x7f472406d3d0 2023-08-25T09:38:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f472406def0 0x7f4724166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f472406def0 0x7f4724166760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56638/0 (socket says 172.21.15.70:56638) 2023-08-25T09:38:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 -- 172.21.15.70:0/83616236 learned_addr learned my addr 172.21.15.70:0/83616236 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f47291ba640 1 --2- 172.21.15.70:0/83616236 >> v2:172.21.15.155:3300/0 conn(0x7f472406d3d0 0x7f472406d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 -- 172.21.15.70:0/83616236 >> v2:172.21.15.155:3300/0 conn(0x7f472406d3d0 msgr2=0x7f472406d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 --2- 172.21.15.70:0/83616236 >> v2:172.21.15.155:3300/0 conn(0x7f472406d3d0 0x7f472406d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 -- 172.21.15.70:0/83616236 >> v1:172.21.15.155:6789/0 conn(0x7f472415cc00 legacy=0x7f47240cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.941+0000 7f472a1bc640 1 -- 172.21.15.70:0/83616236 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724069850 con 0x7f472406def0 2023-08-25T09:38:56.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472a1bc640 1 --2- 172.21.15.70:0/83616236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472406def0 0x7f4724166760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47240ce3c0 tx=0x7f471c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1079e38b26187b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f47289b9640 1 -- 172.21.15.70:0/83616236 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c031870 con 0x7f472406def0 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f47289b9640 1 -- 172.21.15.70:0/83616236 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c0319d0 con 0x7f472406def0 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f47289b9640 1 -- 172.21.15.70:0/83616236 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c031d60 con 0x7f472406def0 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472bc1f640 1 -- 172.21.15.70:0/83616236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472406def0 msgr2=0x7f4724166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472bc1f640 1 --2- 172.21.15.70:0/83616236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472406def0 0x7f4724166760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f47240ce3c0 tx=0x7f471c02b910 comp rx=0 tx=0).stop 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472bc1f640 1 -- 172.21.15.70:0/83616236 shutdown_connections 2023-08-25T09:38:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472bc1f640 1 --2- 172.21.15.70:0/83616236 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472406def0 0x7f4724166760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472bc1f640 1 --2- 172.21.15.70:0/83616236 >> v2:172.21.15.155:3300/0 conn(0x7f472406d3d0 0x7f472406d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.943+0000 7f472bc1f640 1 -- 172.21.15.70:0/83616236 >> 172.21.15.70:0/83616236 conn(0x7f47240d0860 msgr2=0x7f47240d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:56.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.944+0000 7f472bc1f640 1 -- 172.21.15.70:0/83616236 shutdown_connections 2023-08-25T09:38:56.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.944+0000 7f472bc1f640 1 -- 172.21.15.70:0/83616236 wait complete. 2023-08-25T09:38:56.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.945+0000 7f472bc1f640 1 Processor -- start 2023-08-25T09:38:56.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.945+0000 7f472bc1f640 1 -- start start 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.946+0000 7f472bc1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472406d3d0 0x7f47240c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.946+0000 7f472bc1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472406def0 0x7f47240adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.946+0000 7f472bc1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 0x7f47241499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.946+0000 7f472bc1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c8ec0 con 0x7f472415cc00 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.946+0000 7f472bc1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c9a00 con 0x7f472406d3d0 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.946+0000 7f472bc1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240c9160 con 0x7f472406def0 2023-08-25T09:38:56.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 0x7f47241499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 0x7f47241499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56648/0 (socket says 172.21.15.70:56648) 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 -- 172.21.15.70:0/3027702853 learned_addr learned my addr 172.21.15.70:0/3027702853 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 -- 172.21.15.70:0/3027702853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472406def0 msgr2=0x7f47240adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472406def0 0x7f47240adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 -- 172.21.15.70:0/3027702853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472406d3d0 msgr2=0x7f47240c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472406d3d0 0x7f47240c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:56.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 -- 172.21.15.70:0/3027702853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c0094d0 con 0x7f472415cc00 2023-08-25T09:38:56.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.947+0000 7f472a1bc640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 0x7f47241499f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f471c0090b0 tx=0x7f471c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:56.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.948+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c03ac60 con 0x7f472415cc00 2023-08-25T09:38:56.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.948+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c03adc0 con 0x7f472415cc00 2023-08-25T09:38:56.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.948+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c044630 con 0x7f472415cc00 2023-08-25T09:38:56.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.948+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47240c9440 con 0x7f472415cc00 2023-08-25T09:38:56.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.948+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47240c5a00 con 0x7f472415cc00 2023-08-25T09:38:56.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.949+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47240c5860 con 0x7f472415cc00 2023-08-25T09:38:56.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.950+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f471c04d450 con 0x7f472415cc00 2023-08-25T09:38:56.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.951+0000 7f47227fc640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f471006a050 0x7f471006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:56.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.951+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f471c02d040 con 0x7f472415cc00 2023-08-25T09:38:56.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.951+0000 7f47299bb640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f471006a050 0x7f471006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:56.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.951+0000 7f47299bb640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f471006a050 0x7f471006c510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f471400ab90 tx=0x7f4714009230 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:56.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:56.955+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f47240c5860 con 0x7f472415cc00 2023-08-25T09:38:57.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.148+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47240c5860 con 0x7f472415cc00 2023-08-25T09:38:57.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.151+0000 7f47227fc640 1 -- 172.21.15.70:0/3027702853 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f47240c5860 con 0x7f472415cc00 2023-08-25T09:38:57.155 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:57.155 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f471006a050 msgr2=0x7f471006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f471006a050 0x7f471006c510 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f471400ab90 tx=0x7f4714009230 comp rx=0 tx=0).stop 2023-08-25T09:38:57.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 msgr2=0x7f47241499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 0x7f47241499f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f471c0090b0 tx=0x7f471c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 shutdown_connections 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f471006a050 0x7f471006c510 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f472415cc00 0x7f47241499f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f472406def0 0x7f47240adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.153+0000 7f472bc1f640 1 --2- 172.21.15.70:0/3027702853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f472406d3d0 0x7f47240c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.154+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 >> 172.21.15.70:0/3027702853 conn(0x7f47240d0860 msgr2=0x7f47240b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:57.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.154+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 shutdown_connections 2023-08-25T09:38:57.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.154+0000 7f472bc1f640 1 -- 172.21.15.70:0/3027702853 wait complete. 2023-08-25T09:38:57.176 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:38:57.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:38:57.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.392+0000 7f551389b640 1 Processor -- start 2023-08-25T09:38:57.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.392+0000 7f551389b640 1 -- start start 2023-08-25T09:38:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.394+0000 7f551389b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f550c160ff0 0x7f550c06b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.394+0000 7f551389b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c0ce5f0 0x7f550c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.394+0000 7f551389b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f550c1613d0 0x7f550c161800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.394+0000 7f551389b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c06ad70 con 0x7f550c1613d0 2023-08-25T09:38:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.394+0000 7f551389b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c0cefb0 con 0x7f550c160ff0 2023-08-25T09:38:57.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.394+0000 7f551389b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1620b0 con 0x7f550c0ce5f0 2023-08-25T09:38:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f550c1613d0 0x7f550c161800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f550c1613d0 0x7f550c161800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56656/0 (socket says 172.21.15.70:56656) 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 -- 172.21.15.70:0/3525427135 learned_addr learned my addr 172.21.15.70:0/3525427135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511637640 1 --2- 172.21.15.70:0/3525427135 >> v2:172.21.15.155:3300/0 conn(0x7f550c160ff0 0x7f550c06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5510e36640 1 --2- 172.21.15.70:0/3525427135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c0ce5f0 0x7f550c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 -- 172.21.15.70:0/3525427135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c0ce5f0 msgr2=0x7f550c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 --2- 172.21.15.70:0/3525427135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c0ce5f0 0x7f550c0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 -- 172.21.15.70:0/3525427135 >> v2:172.21.15.155:3300/0 conn(0x7f550c160ff0 msgr2=0x7f550c06b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:57.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 --2- 172.21.15.70:0/3525427135 >> v2:172.21.15.155:3300/0 conn(0x7f550c160ff0 0x7f550c06b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511e38640 1 -- 172.21.15.70:0/3525427135 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c162390 con 0x7f550c1613d0 2023-08-25T09:38:57.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.395+0000 7f5511637640 1 --2- 172.21.15.70:0/3525427135 >> v2:172.21.15.155:3300/0 conn(0x7f550c160ff0 0x7f550c06b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.396+0000 7f5511e38640 1 --2- 172.21.15.70:0/3525427135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c1613d0 0x7f550c161800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f550c06c5e0 tx=0x7f550402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88700ee8c807a033 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f5503fff640 1 -- 172.21.15.70:0/3525427135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550402ca80 con 0x7f550c1613d0 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f5503fff640 1 -- 172.21.15.70:0/3525427135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550402cbe0 con 0x7f550c1613d0 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f5503fff640 1 -- 172.21.15.70:0/3525427135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55040343f0 con 0x7f550c1613d0 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 -- 172.21.15.70:0/3525427135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c1613d0 msgr2=0x7f550c161800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 --2- 172.21.15.70:0/3525427135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c1613d0 0x7f550c161800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f550c06c5e0 tx=0x7f550402bee0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 -- 172.21.15.70:0/3525427135 shutdown_connections 2023-08-25T09:38:57.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 --2- 172.21.15.70:0/3525427135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c1613d0 0x7f550c161800 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 --2- 172.21.15.70:0/3525427135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c0ce5f0 0x7f550c0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 --2- 172.21.15.70:0/3525427135 >> v2:172.21.15.155:3300/0 conn(0x7f550c160ff0 0x7f550c06b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.397+0000 7f551389b640 1 -- 172.21.15.70:0/3525427135 >> 172.21.15.70:0/3525427135 conn(0x7f550c069850 msgr2=0x7f550c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:57.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.398+0000 7f551389b640 1 -- 172.21.15.70:0/3525427135 shutdown_connections 2023-08-25T09:38:57.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.398+0000 7f551389b640 1 -- 172.21.15.70:0/3525427135 wait complete. 2023-08-25T09:38:57.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.399+0000 7f551389b640 1 Processor -- start 2023-08-25T09:38:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 -- start start 2023-08-25T09:38:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 0x7f550c1f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550c160ff0 0x7f550c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c1613d0 0x7f550c1fa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1f6b50 con 0x7f550c0ce5f0 2023-08-25T09:38:57.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1fab90 con 0x7f550c160ff0 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.400+0000 7f551389b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1fae70 con 0x7f550c1613d0 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 0x7f550c1f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 0x7f550c1f7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56672/0 (socket says 172.21.15.70:56672) 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 -- 172.21.15.70:0/1515215636 learned_addr learned my addr 172.21.15.70:0/1515215636 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 -- 172.21.15.70:0/1515215636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c1613d0 msgr2=0x7f550c1fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c1613d0 0x7f550c1fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 -- 172.21.15.70:0/1515215636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550c160ff0 msgr2=0x7f550c15c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550c160ff0 0x7f550c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 -- 172.21.15.70:0/1515215636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504009b30 con 0x7f550c0ce5f0 2023-08-25T09:38:57.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.401+0000 7f5511637640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 0x7f550c1f7540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f550800e7e0 tx=0x7f550800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:57.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.402+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508004030 con 0x7f550c0ce5f0 2023-08-25T09:38:57.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.402+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508004190 con 0x7f550c0ce5f0 2023-08-25T09:38:57.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.402+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c1f6d40 con 0x7f550c0ce5f0 2023-08-25T09:38:57.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.402+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550800c6d0 con 0x7f550c0ce5f0 2023-08-25T09:38:57.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.402+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c1f7a80 con 0x7f550c0ce5f0 2023-08-25T09:38:57.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.403+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f550c1fb0b0 con 0x7f550c0ce5f0 2023-08-25T09:38:57.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.404+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f550800c830 con 0x7f550c0ce5f0 2023-08-25T09:38:57.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.405+0000 7f5501ffb640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54f006a050 0x7f54f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.405+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55080882f0 con 0x7f550c0ce5f0 2023-08-25T09:38:57.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.405+0000 7f5510e36640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54f006a050 0x7f54f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.406+0000 7f5510e36640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54f006a050 0x7f54f006c510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54fc006fd0 tx=0x7f54fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:57.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.409+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f55080539d0 con 0x7f550c0ce5f0 2023-08-25T09:38:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.596+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f550c06b260 con 0x7f54f006a050 2023-08-25T09:38:57.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.606+0000 7f5501ffb640 1 -- 172.21.15.70:0/1515215636 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280680 (secure 0 0 0) 0x7f54fc00be30 con 0x7f54f006a050 2023-08-25T09:38:57.614 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:57.616 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:38:57.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54f006a050 msgr2=0x7f54f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54f006a050 0x7f54f006c510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54fc006fd0 tx=0x7f54fc008040 comp rx=0 tx=0).stop 2023-08-25T09:38:57.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 msgr2=0x7f550c1f7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 0x7f550c1f7540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f550800e7e0 tx=0x7f550800eca0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 shutdown_connections 2023-08-25T09:38:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54f006a050 0x7f54f006c510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f550c1613d0 0x7f550c1fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f550c160ff0 0x7f550c15c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 --2- 172.21.15.70:0/1515215636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f550c0ce5f0 0x7f550c1f7540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 >> 172.21.15.70:0/1515215636 conn(0x7f550c069850 msgr2=0x7f550c06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:57.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.611+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 shutdown_connections 2023-08-25T09:38:57.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.612+0000 7f551389b640 1 -- 172.21.15.70:0/1515215636 wait complete. 2023-08-25T09:38:57.633 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-08-25T09:38:57.318621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1830,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560669,"statfs":{"total":1600000000,"available":1598125544,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002266"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169943+0000","last_change":"2023-08-25T09:38:53.105814+0000","last_active":"2023-08-25T09:38:54.169943+0000","last_peered":"2023-08-25T09:38:54.169943+0000","last_clean":"2023-08-25T09:38:54.169943+0000","last_became_active":"2023-08-25T09:38:53.103612+0000","last_became_peered":"2023-08-25T09:38:53.103612+0000","last_unstale":"2023-08-25T09:38:54.169943+0000","last_undegraded":"2023-08-25T09:38:54.169943+0000","last_fullsized":"2023-08-25T09:38:54.169943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185531+0000","last_change":"2023-08-25T09:38:53.103005+0000","last_active":"2023-08-25T09:38:54.185531+0000","last_peered":"2023-08-25T09:38:54.185531+0000","last_clean":"2023-08-25T09:38:54.185531+0000","last_became_active":"2023-08-25T09:38:53.102634+0000","last_became_peered":"2023-08-25T09:38:53.102634+0000","last_unstale":"2023-08-25T09:38:54.185531+0000","last_undegraded":"2023-08-25T09:38:54.185531+0000","last_fullsized":"2023-08-25T09:38:54.185531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214701+0000","last_change":"2023-08-25T09:38:53.111188+0000","last_active":"2023-08-25T09:38:54.214701+0000","last_peered":"2023-08-25T09:38:54.214701+0000","last_clean":"2023-08-25T09:38:54.214701+0000","last_became_active":"2023-08-25T09:38:53.108190+0000","last_became_peered":"2023-08-25T09:38:53.108190+0000","last_unstale":"2023-08-25T09:38:54.214701+0000","last_undegraded":"2023-08-25T09:38:54.214701+0000","last_fullsized":"2023-08-25T09:38:54.214701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:16.468452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214312+0000","last_change":"2023-08-25T09:38:53.103968+0000","last_active":"2023-08-25T09:38:54.214312+0000","last_peered":"2023-08-25T09:38:54.214312+0000","last_clean":"2023-08-25T09:38:54.214312+0000","last_became_active":"2023-08-25T09:38:53.103676+0000","last_became_peered":"2023-08-25T09:38:53.103676+0000","last_unstale":"2023-08-25T09:38:54.214312+0000","last_undegraded":"2023-08-25T09:38:54.214312+0000","last_fullsized":"2023-08-25T09:38:54.214312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170566+0000","last_change":"2023-08-25T09:38:53.110486+0000","last_active":"2023-08-25T09:38:54.170566+0000","last_peered":"2023-08-25T09:38:54.170566+0000","last_clean":"2023-08-25T09:38:54.170566+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.170566+0000","last_undegraded":"2023-08-25T09:38:54.170566+0000","last_fullsized":"2023-08-25T09:38:54.170566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:15.681428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077139+0000","last_change":"2023-08-25T09:38:53.082883+0000","last_active":"2023-08-25T09:38:54.077139+0000","last_peered":"2023-08-25T09:38:54.077139+0000","last_clean":"2023-08-25T09:38:54.077139+0000","last_became_active":"2023-08-25T09:38:53.081892+0000","last_became_peered":"2023-08-25T09:38:53.081892+0000","last_unstale":"2023-08-25T09:38:54.077139+0000","last_undegraded":"2023-08-25T09:38:54.077139+0000","last_fullsized":"2023-08-25T09:38:54.077139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:25:18.218904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077432+0000","last_change":"2023-08-25T09:38:53.094986+0000","last_active":"2023-08-25T09:38:54.077432+0000","last_peered":"2023-08-25T09:38:54.077432+0000","last_clean":"2023-08-25T09:38:54.077432+0000","last_became_active":"2023-08-25T09:38:53.094533+0000","last_became_peered":"2023-08-25T09:38:53.094533+0000","last_unstale":"2023-08-25T09:38:54.077432+0000","last_undegraded":"2023-08-25T09:38:54.077432+0000","last_fullsized":"2023-08-25T09:38:54.077432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:44.551022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167144+0000","last_change":"2023-08-25T09:38:53.097497+0000","last_active":"2023-08-25T09:38:54.167144+0000","last_peered":"2023-08-25T09:38:54.167144+0000","last_clean":"2023-08-25T09:38:54.167144+0000","last_became_active":"2023-08-25T09:38:53.096835+0000","last_became_peered":"2023-08-25T09:38:53.096835+0000","last_unstale":"2023-08-25T09:38:54.167144+0000","last_undegraded":"2023-08-25T09:38:54.167144+0000","last_fullsized":"2023-08-25T09:38:54.167144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168759+0000","last_change":"2023-08-25T09:38:53.111823+0000","last_active":"2023-08-25T09:38:54.168759+0000","last_peered":"2023-08-25T09:38:54.168759+0000","last_clean":"2023-08-25T09:38:54.168759+0000","last_became_active":"2023-08-25T09:38:53.107075+0000","last_became_peered":"2023-08-25T09:38:53.107075+0000","last_unstale":"2023-08-25T09:38:54.168759+0000","last_undegraded":"2023-08-25T09:38:54.168759+0000","last_fullsized":"2023-08-25T09:38:54.168759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167237+0000","last_change":"2023-08-25T09:38:53.097405+0000","last_active":"2023-08-25T09:38:54.167237+0000","last_peered":"2023-08-25T09:38:54.167237+0000","last_clean":"2023-08-25T09:38:54.167237+0000","last_became_active":"2023-08-25T09:38:53.097063+0000","last_became_peered":"2023-08-25T09:38:53.097063+0000","last_unstale":"2023-08-25T09:38:54.167237+0000","last_undegraded":"2023-08-25T09:38:54.167237+0000","last_fullsized":"2023-08-25T09:38:54.167237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167432+0000","last_change":"2023-08-25T09:38:53.097713+0000","last_active":"2023-08-25T09:38:54.167432+0000","last_peered":"2023-08-25T09:38:54.167432+0000","last_clean":"2023-08-25T09:38:54.167432+0000","last_became_active":"2023-08-25T09:38:53.097250+0000","last_became_peered":"2023-08-25T09:38:53.097250+0000","last_unstale":"2023-08-25T09:38:54.167432+0000","last_undegraded":"2023-08-25T09:38:54.167432+0000","last_fullsized":"2023-08-25T09:38:54.167432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170076+0000","last_change":"2023-08-25T09:38:53.098525+0000","last_active":"2023-08-25T09:38:54.170076+0000","last_peered":"2023-08-25T09:38:54.170076+0000","last_clean":"2023-08-25T09:38:54.170076+0000","last_became_active":"2023-08-25T09:38:53.097701+0000","last_became_peered":"2023-08-25T09:38:53.097701+0000","last_unstale":"2023-08-25T09:38:54.170076+0000","last_undegraded":"2023-08-25T09:38:54.170076+0000","last_fullsized":"2023-08-25T09:38:54.170076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079156+0000","last_change":"2023-08-25T09:38:53.110156+0000","last_active":"2023-08-25T09:38:54.079156+0000","last_peered":"2023-08-25T09:38:54.079156+0000","last_clean":"2023-08-25T09:38:54.079156+0000","last_became_active":"2023-08-25T09:38:53.109921+0000","last_became_peered":"2023-08-25T09:38:53.109921+0000","last_unstale":"2023-08-25T09:38:54.079156+0000","last_undegraded":"2023-08-25T09:38:54.079156+0000","last_fullsized":"2023-08-25T09:38:54.079156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167535+0000","last_change":"2023-08-25T09:38:53.102316+0000","last_active":"2023-08-25T09:38:54.167535+0000","last_peered":"2023-08-25T09:38:54.167535+0000","last_clean":"2023-08-25T09:38:54.167535+0000","last_became_active":"2023-08-25T09:38:53.100943+0000","last_became_peered":"2023-08-25T09:38:53.100943+0000","last_unstale":"2023-08-25T09:38:54.167535+0000","last_undegraded":"2023-08-25T09:38:54.167535+0000","last_fullsized":"2023-08-25T09:38:54.167535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214539+0000","last_change":"2023-08-25T09:38:53.111165+0000","last_active":"2023-08-25T09:38:54.214539+0000","last_peered":"2023-08-25T09:38:54.214539+0000","last_clean":"2023-08-25T09:38:54.214539+0000","last_became_active":"2023-08-25T09:38:53.108285+0000","last_became_peered":"2023-08-25T09:38:53.108285+0000","last_unstale":"2023-08-25T09:38:54.214539+0000","last_undegraded":"2023-08-25T09:38:54.214539+0000","last_fullsized":"2023-08-25T09:38:54.214539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077362+0000","last_change":"2023-08-25T09:38:53.089749+0000","last_active":"2023-08-25T09:38:54.077362+0000","last_peered":"2023-08-25T09:38:54.077362+0000","last_clean":"2023-08-25T09:38:54.077362+0000","last_became_active":"2023-08-25T09:38:53.088753+0000","last_became_peered":"2023-08-25T09:38:53.088753+0000","last_unstale":"2023-08-25T09:38:54.077362+0000","last_undegraded":"2023-08-25T09:38:54.077362+0000","last_fullsized":"2023-08-25T09:38:54.077362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:35.174323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078331+0000","last_change":"2023-08-25T09:38:53.100818+0000","last_active":"2023-08-25T09:38:54.078331+0000","last_peered":"2023-08-25T09:38:54.078331+0000","last_clean":"2023-08-25T09:38:54.078331+0000","last_became_active":"2023-08-25T09:38:53.100555+0000","last_became_peered":"2023-08-25T09:38:53.100555+0000","last_unstale":"2023-08-25T09:38:54.078331+0000","last_undegraded":"2023-08-25T09:38:54.078331+0000","last_fullsized":"2023-08-25T09:38:54.078331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215037+0000","last_change":"2023-08-25T09:38:53.105302+0000","last_active":"2023-08-25T09:38:54.215037+0000","last_peered":"2023-08-25T09:38:54.215037+0000","last_clean":"2023-08-25T09:38:54.215037+0000","last_became_active":"2023-08-25T09:38:53.103402+0000","last_became_peered":"2023-08-25T09:38:53.103402+0000","last_unstale":"2023-08-25T09:38:54.215037+0000","last_undegraded":"2023-08-25T09:38:54.215037+0000","last_fullsized":"2023-08-25T09:38:54.215037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171407+0000","last_change":"2023-08-25T09:38:53.099130+0000","last_active":"2023-08-25T09:38:54.171407+0000","last_peered":"2023-08-25T09:38:54.171407+0000","last_clean":"2023-08-25T09:38:54.171407+0000","last_became_active":"2023-08-25T09:38:53.094676+0000","last_became_peered":"2023-08-25T09:38:53.094676+0000","last_unstale":"2023-08-25T09:38:54.171407+0000","last_undegraded":"2023-08-25T09:38:54.171407+0000","last_fullsized":"2023-08-25T09:38:54.171407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:05.648183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167405+0000","last_change":"2023-08-25T09:38:53.111895+0000","last_active":"2023-08-25T09:38:54.167405+0000","last_peered":"2023-08-25T09:38:54.167405+0000","last_clean":"2023-08-25T09:38:54.167405+0000","last_became_active":"2023-08-25T09:38:53.111626+0000","last_became_peered":"2023-08-25T09:38:53.111626+0000","last_unstale":"2023-08-25T09:38:54.167405+0000","last_undegraded":"2023-08-25T09:38:54.167405+0000","last_fullsized":"2023-08-25T09:38:54.167405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167554+0000","last_change":"2023-08-25T09:38:53.104999+0000","last_active":"2023-08-25T09:38:54.167554+0000","last_peered":"2023-08-25T09:38:54.167554+0000","last_clean":"2023-08-25T09:38:54.167554+0000","last_became_active":"2023-08-25T09:38:53.104760+0000","last_became_peered":"2023-08-25T09:38:53.104760+0000","last_unstale":"2023-08-25T09:38:54.167554+0000","last_undegraded":"2023-08-25T09:38:54.167554+0000","last_fullsized":"2023-08-25T09:38:54.167554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:17.840144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078687+0000","last_change":"2023-08-25T09:38:53.106873+0000","last_active":"2023-08-25T09:38:54.078687+0000","last_peered":"2023-08-25T09:38:54.078687+0000","last_clean":"2023-08-25T09:38:54.078687+0000","last_became_active":"2023-08-25T09:38:53.106581+0000","last_became_peered":"2023-08-25T09:38:53.106581+0000","last_unstale":"2023-08-25T09:38:54.078687+0000","last_undegraded":"2023-08-25T09:38:54.078687+0000","last_fullsized":"2023-08-25T09:38:54.078687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168645+0000","last_change":"2023-08-25T09:38:53.112057+0000","last_active":"2023-08-25T09:38:54.168645+0000","last_peered":"2023-08-25T09:38:54.168645+0000","last_clean":"2023-08-25T09:38:54.168645+0000","last_became_active":"2023-08-25T09:38:53.111710+0000","last_became_peered":"2023-08-25T09:38:53.111710+0000","last_unstale":"2023-08-25T09:38:54.168645+0000","last_undegraded":"2023-08-25T09:38:54.168645+0000","last_fullsized":"2023-08-25T09:38:54.168645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167656+0000","last_change":"2023-08-25T09:38:53.101206+0000","last_active":"2023-08-25T09:38:54.167656+0000","last_peered":"2023-08-25T09:38:54.167656+0000","last_clean":"2023-08-25T09:38:54.167656+0000","last_became_active":"2023-08-25T09:38:53.100575+0000","last_became_peered":"2023-08-25T09:38:53.100575+0000","last_unstale":"2023-08-25T09:38:54.167656+0000","last_undegraded":"2023-08-25T09:38:54.167656+0000","last_fullsized":"2023-08-25T09:38:54.167656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:44.735256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170902+0000","last_change":"2023-08-25T09:38:53.110658+0000","last_active":"2023-08-25T09:38:54.170902+0000","last_peered":"2023-08-25T09:38:54.170902+0000","last_clean":"2023-08-25T09:38:54.170902+0000","last_became_active":"2023-08-25T09:38:53.107553+0000","last_became_peered":"2023-08-25T09:38:53.107553+0000","last_unstale":"2023-08-25T09:38:54.170902+0000","last_undegraded":"2023-08-25T09:38:54.170902+0000","last_fullsized":"2023-08-25T09:38:54.170902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:35.169281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078368+0000","last_change":"2023-08-25T09:38:53.106509+0000","last_active":"2023-08-25T09:38:54.078368+0000","last_peered":"2023-08-25T09:38:54.078368+0000","last_clean":"2023-08-25T09:38:54.078368+0000","last_became_active":"2023-08-25T09:38:53.106227+0000","last_became_peered":"2023-08-25T09:38:53.106227+0000","last_unstale":"2023-08-25T09:38:54.078368+0000","last_undegraded":"2023-08-25T09:38:54.078368+0000","last_fullsized":"2023-08-25T09:38:54.078368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077761+0000","last_change":"2023-08-25T09:38:53.094823+0000","last_active":"2023-08-25T09:38:54.077761+0000","last_peered":"2023-08-25T09:38:54.077761+0000","last_clean":"2023-08-25T09:38:54.077761+0000","last_became_active":"2023-08-25T09:38:53.094160+0000","last_became_peered":"2023-08-25T09:38:53.094160+0000","last_unstale":"2023-08-25T09:38:54.077761+0000","last_undegraded":"2023-08-25T09:38:54.077761+0000","last_fullsized":"2023-08-25T09:38:54.077761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:05:17.873255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169761+0000","last_change":"2023-08-25T09:38:53.109739+0000","last_active":"2023-08-25T09:38:54.169761+0000","last_peered":"2023-08-25T09:38:54.169761+0000","last_clean":"2023-08-25T09:38:54.169761+0000","last_became_active":"2023-08-25T09:38:53.106937+0000","last_became_peered":"2023-08-25T09:38:53.106937+0000","last_unstale":"2023-08-25T09:38:54.169761+0000","last_undegraded":"2023-08-25T09:38:54.169761+0000","last_fullsized":"2023-08-25T09:38:54.169761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170229+0000","last_change":"2023-08-25T09:38:53.102490+0000","last_active":"2023-08-25T09:38:54.170229+0000","last_peered":"2023-08-25T09:38:54.170229+0000","last_clean":"2023-08-25T09:38:54.170229+0000","last_became_active":"2023-08-25T09:38:53.101594+0000","last_became_peered":"2023-08-25T09:38:53.101594+0000","last_unstale":"2023-08-25T09:38:54.170229+0000","last_undegraded":"2023-08-25T09:38:54.170229+0000","last_fullsized":"2023-08-25T09:38:54.170229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214623+0000","last_change":"2023-08-25T09:38:53.092816+0000","last_active":"2023-08-25T09:38:54.214623+0000","last_peered":"2023-08-25T09:38:54.214623+0000","last_clean":"2023-08-25T09:38:54.214623+0000","last_became_active":"2023-08-25T09:38:53.092392+0000","last_became_peered":"2023-08-25T09:38:53.092392+0000","last_unstale":"2023-08-25T09:38:54.214623+0000","last_undegraded":"2023-08-25T09:38:54.214623+0000","last_fullsized":"2023-08-25T09:38:54.214623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079002+0000","last_change":"2023-08-25T09:38:53.104234+0000","last_active":"2023-08-25T09:38:54.079002+0000","last_peered":"2023-08-25T09:38:54.079002+0000","last_clean":"2023-08-25T09:38:54.079002+0000","last_became_active":"2023-08-25T09:38:53.104004+0000","last_became_peered":"2023-08-25T09:38:53.104004+0000","last_unstale":"2023-08-25T09:38:54.079002+0000","last_undegraded":"2023-08-25T09:38:54.079002+0000","last_fullsized":"2023-08-25T09:38:54.079002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167124+0000","last_change":"2023-08-25T09:38:53.106635+0000","last_active":"2023-08-25T09:38:54.167124+0000","last_peered":"2023-08-25T09:38:54.167124+0000","last_clean":"2023-08-25T09:38:54.167124+0000","last_became_active":"2023-08-25T09:38:53.106105+0000","last_became_peered":"2023-08-25T09:38:53.106105+0000","last_unstale":"2023-08-25T09:38:54.167124+0000","last_undegraded":"2023-08-25T09:38:54.167124+0000","last_fullsized":"2023-08-25T09:38:54.167124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078322+0000","last_change":"2023-08-25T09:38:53.108366+0000","last_active":"2023-08-25T09:38:54.078322+0000","last_peered":"2023-08-25T09:38:54.078322+0000","last_clean":"2023-08-25T09:38:54.078322+0000","last_became_active":"2023-08-25T09:38:53.108123+0000","last_became_peered":"2023-08-25T09:38:53.108123+0000","last_unstale":"2023-08-25T09:38:54.078322+0000","last_undegraded":"2023-08-25T09:38:54.078322+0000","last_fullsized":"2023-08-25T09:38:54.078322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077197+0000","last_change":"2023-08-25T09:38:52.142687+0000","last_active":"2023-08-25T09:38:54.077197+0000","last_peered":"2023-08-25T09:38:54.077197+0000","last_clean":"2023-08-25T09:38:54.077197+0000","last_became_active":"2023-08-25T09:38:52.140330+0000","last_became_peered":"2023-08-25T09:38:52.140330+0000","last_unstale":"2023-08-25T09:38:54.077197+0000","last_undegraded":"2023-08-25T09:38:54.077197+0000","last_fullsized":"2023-08-25T09:38:54.077197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:42.836964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076792+0000","last_change":"2023-08-25T09:38:53.094431+0000","last_active":"2023-08-25T09:38:54.076792+0000","last_peered":"2023-08-25T09:38:54.076792+0000","last_clean":"2023-08-25T09:38:54.076792+0000","last_became_active":"2023-08-25T09:38:53.093756+0000","last_became_peered":"2023-08-25T09:38:53.093756+0000","last_unstale":"2023-08-25T09:38:54.076792+0000","last_undegraded":"2023-08-25T09:38:54.076792+0000","last_fullsized":"2023-08-25T09:38:54.076792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:32.683806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079876+0000","last_change":"2023-08-25T09:38:52.138724+0000","last_active":"2023-08-25T09:38:54.079876+0000","last_peered":"2023-08-25T09:38:54.079876+0000","last_clean":"2023-08-25T09:38:54.079876+0000","last_became_active":"2023-08-25T09:38:52.138323+0000","last_became_peered":"2023-08-25T09:38:52.138323+0000","last_unstale":"2023-08-25T09:38:54.079876+0000","last_undegraded":"2023-08-25T09:38:54.079876+0000","last_fullsized":"2023-08-25T09:38:54.079876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170651+0000","last_change":"2023-08-25T09:38:53.098073+0000","last_active":"2023-08-25T09:38:54.170651+0000","last_peered":"2023-08-25T09:38:54.170651+0000","last_clean":"2023-08-25T09:38:54.170651+0000","last_became_active":"2023-08-25T09:38:53.089135+0000","last_became_peered":"2023-08-25T09:38:53.089135+0000","last_unstale":"2023-08-25T09:38:54.170651+0000","last_undegraded":"2023-08-25T09:38:54.170651+0000","last_fullsized":"2023-08-25T09:38:54.170651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078034+0000","last_change":"2023-08-25T09:38:52.146382+0000","last_active":"2023-08-25T09:38:54.078034+0000","last_peered":"2023-08-25T09:38:54.078034+0000","last_clean":"2023-08-25T09:38:54.078034+0000","last_became_active":"2023-08-25T09:38:52.145954+0000","last_became_peered":"2023-08-25T09:38:52.145954+0000","last_unstale":"2023-08-25T09:38:54.078034+0000","last_undegraded":"2023-08-25T09:38:54.078034+0000","last_fullsized":"2023-08-25T09:38:54.078034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:02.701588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169581+0000","last_change":"2023-08-25T09:38:53.106145+0000","last_active":"2023-08-25T09:38:54.169581+0000","last_peered":"2023-08-25T09:38:54.169581+0000","last_clean":"2023-08-25T09:38:54.169581+0000","last_became_active":"2023-08-25T09:38:53.105502+0000","last_became_peered":"2023-08-25T09:38:53.105502+0000","last_unstale":"2023-08-25T09:38:54.169581+0000","last_undegraded":"2023-08-25T09:38:54.169581+0000","last_fullsized":"2023-08-25T09:38:54.169581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:01.061628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167472+0000","last_change":"2023-08-25T09:38:52.148928+0000","last_active":"2023-08-25T09:38:54.167472+0000","last_peered":"2023-08-25T09:38:54.167472+0000","last_clean":"2023-08-25T09:38:54.167472+0000","last_became_active":"2023-08-25T09:38:52.148227+0000","last_became_peered":"2023-08-25T09:38:52.148227+0000","last_unstale":"2023-08-25T09:38:54.167472+0000","last_undegraded":"2023-08-25T09:38:54.167472+0000","last_fullsized":"2023-08-25T09:38:54.167472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214435+0000","last_change":"2023-08-25T09:38:52.138014+0000","last_active":"2023-08-25T09:38:54.214435+0000","last_peered":"2023-08-25T09:38:54.214435+0000","last_clean":"2023-08-25T09:38:54.214435+0000","last_became_active":"2023-08-25T09:38:52.137777+0000","last_became_peered":"2023-08-25T09:38:52.137777+0000","last_unstale":"2023-08-25T09:38:54.214435+0000","last_undegraded":"2023-08-25T09:38:54.214435+0000","last_fullsized":"2023-08-25T09:38:54.214435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:10.214913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077886+0000","last_change":"2023-08-25T09:38:53.100050+0000","last_active":"2023-08-25T09:38:54.077886+0000","last_peered":"2023-08-25T09:38:54.077886+0000","last_clean":"2023-08-25T09:38:54.077886+0000","last_became_active":"2023-08-25T09:38:53.099406+0000","last_became_peered":"2023-08-25T09:38:53.099406+0000","last_unstale":"2023-08-25T09:38:54.077886+0000","last_undegraded":"2023-08-25T09:38:54.077886+0000","last_fullsized":"2023-08-25T09:38:54.077886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:46.330038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186222+0000","last_change":"2023-08-25T09:38:52.145560+0000","last_active":"2023-08-25T09:38:54.186222+0000","last_peered":"2023-08-25T09:38:54.186222+0000","last_clean":"2023-08-25T09:38:54.186222+0000","last_became_active":"2023-08-25T09:38:52.145324+0000","last_became_peered":"2023-08-25T09:38:52.145324+0000","last_unstale":"2023-08-25T09:38:54.186222+0000","last_undegraded":"2023-08-25T09:38:54.186222+0000","last_fullsized":"2023-08-25T09:38:54.186222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:47.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167557+0000","last_change":"2023-08-25T09:38:53.105196+0000","last_active":"2023-08-25T09:38:54.167557+0000","last_peered":"2023-08-25T09:38:54.167557+0000","last_clean":"2023-08-25T09:38:54.167557+0000","last_became_active":"2023-08-25T09:38:53.104948+0000","last_became_peered":"2023-08-25T09:38:53.104948+0000","last_unstale":"2023-08-25T09:38:54.167557+0000","last_undegraded":"2023-08-25T09:38:54.167557+0000","last_fullsized":"2023-08-25T09:38:54.167557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078179+0000","last_change":"2023-08-25T09:38:52.140961+0000","last_active":"2023-08-25T09:38:54.078179+0000","last_peered":"2023-08-25T09:38:54.078179+0000","last_clean":"2023-08-25T09:38:54.078179+0000","last_became_active":"2023-08-25T09:38:52.137710+0000","last_became_peered":"2023-08-25T09:38:52.137710+0000","last_unstale":"2023-08-25T09:38:54.078179+0000","last_undegraded":"2023-08-25T09:38:54.078179+0000","last_fullsized":"2023-08-25T09:38:54.078179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:09.490525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.083467+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.082859+0000","last_became_peered":"2023-08-25T09:38:53.082859+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215256+0000","last_change":"2023-08-25T09:38:52.148012+0000","last_active":"2023-08-25T09:38:54.215256+0000","last_peered":"2023-08-25T09:38:54.215256+0000","last_clean":"2023-08-25T09:38:54.215256+0000","last_became_active":"2023-08-25T09:38:52.147757+0000","last_became_peered":"2023-08-25T09:38:52.147757+0000","last_unstale":"2023-08-25T09:38:54.215256+0000","last_undegraded":"2023-08-25T09:38:54.215256+0000","last_fullsized":"2023-08-25T09:38:54.215256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:36.644423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170501+0000","last_change":"2023-08-25T09:38:53.110777+0000","last_active":"2023-08-25T09:38:54.170501+0000","last_peered":"2023-08-25T09:38:54.170501+0000","last_clean":"2023-08-25T09:38:54.170501+0000","last_became_active":"2023-08-25T09:38:53.108970+0000","last_became_peered":"2023-08-25T09:38:53.108970+0000","last_unstale":"2023-08-25T09:38:54.170501+0000","last_undegraded":"2023-08-25T09:38:54.170501+0000","last_fullsized":"2023-08-25T09:38:54.170501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080573+0000","last_change":"2023-08-25T09:38:52.144190+0000","last_active":"2023-08-25T09:38:54.080573+0000","last_peered":"2023-08-25T09:38:54.080573+0000","last_clean":"2023-08-25T09:38:54.080573+0000","last_became_active":"2023-08-25T09:38:52.143875+0000","last_became_peered":"2023-08-25T09:38:52.143875+0000","last_unstale":"2023-08-25T09:38:54.080573+0000","last_undegraded":"2023-08-25T09:38:54.080573+0000","last_fullsized":"2023-08-25T09:38:54.080573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215099+0000","last_change":"2023-08-25T09:38:53.106602+0000","last_active":"2023-08-25T09:38:54.215099+0000","last_peered":"2023-08-25T09:38:54.215099+0000","last_clean":"2023-08-25T09:38:54.215099+0000","last_became_active":"2023-08-25T09:38:53.106340+0000","last_became_peered":"2023-08-25T09:38:53.106340+0000","last_unstale":"2023-08-25T09:38:54.215099+0000","last_undegraded":"2023-08-25T09:38:54.215099+0000","last_fullsized":"2023-08-25T09:38:54.215099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076776+0000","last_change":"2023-08-25T09:38:52.140744+0000","last_active":"2023-08-25T09:38:54.076776+0000","last_peered":"2023-08-25T09:38:54.076776+0000","last_clean":"2023-08-25T09:38:54.076776+0000","last_became_active":"2023-08-25T09:38:52.140450+0000","last_became_peered":"2023-08-25T09:38:52.140450+0000","last_unstale":"2023-08-25T09:38:54.076776+0000","last_undegraded":"2023-08-25T09:38:54.076776+0000","last_fullsized":"2023-08-25T09:38:54.076776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:54:37.632752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077689+0000","last_change":"2023-08-25T09:38:53.100210+0000","last_active":"2023-08-25T09:38:54.077689+0000","last_peered":"2023-08-25T09:38:54.077689+0000","last_clean":"2023-08-25T09:38:54.077689+0000","last_became_active":"2023-08-25T09:38:53.099792+0000","last_became_peered":"2023-08-25T09:38:53.099792+0000","last_unstale":"2023-08-25T09:38:54.077689+0000","last_undegraded":"2023-08-25T09:38:54.077689+0000","last_fullsized":"2023-08-25T09:38:54.077689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:37.979410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077328+0000","last_change":"2023-08-25T09:38:52.132704+0000","last_active":"2023-08-25T09:38:54.077328+0000","last_peered":"2023-08-25T09:38:54.077328+0000","last_clean":"2023-08-25T09:38:54.077328+0000","last_became_active":"2023-08-25T09:38:52.132045+0000","last_became_peered":"2023-08-25T09:38:52.132045+0000","last_unstale":"2023-08-25T09:38:54.077328+0000","last_undegraded":"2023-08-25T09:38:54.077328+0000","last_fullsized":"2023-08-25T09:38:54.077328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:32:14.124629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186144+0000","last_change":"2023-08-25T09:38:53.095472+0000","last_active":"2023-08-25T09:38:54.186144+0000","last_peered":"2023-08-25T09:38:54.186144+0000","last_clean":"2023-08-25T09:38:54.186144+0000","last_became_active":"2023-08-25T09:38:53.095117+0000","last_became_peered":"2023-08-25T09:38:53.095117+0000","last_unstale":"2023-08-25T09:38:54.186144+0000","last_undegraded":"2023-08-25T09:38:54.186144+0000","last_fullsized":"2023-08-25T09:38:54.186144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169438+0000","last_change":"2023-08-25T09:38:52.142123+0000","last_active":"2023-08-25T09:38:54.169438+0000","last_peered":"2023-08-25T09:38:54.169438+0000","last_clean":"2023-08-25T09:38:54.169438+0000","last_became_active":"2023-08-25T09:38:52.141349+0000","last_became_peered":"2023-08-25T09:38:52.141349+0000","last_unstale":"2023-08-25T09:38:54.169438+0000","last_undegraded":"2023-08-25T09:38:54.169438+0000","last_fullsized":"2023-08-25T09:38:54.169438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078490+0000","last_change":"2023-08-25T09:38:53.101599+0000","last_active":"2023-08-25T09:38:54.078490+0000","last_peered":"2023-08-25T09:38:54.078490+0000","last_clean":"2023-08-25T09:38:54.078490+0000","last_became_active":"2023-08-25T09:38:53.101348+0000","last_became_peered":"2023-08-25T09:38:53.101348+0000","last_unstale":"2023-08-25T09:38:54.078490+0000","last_undegraded":"2023-08-25T09:38:54.078490+0000","last_fullsized":"2023-08-25T09:38:54.078490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167829+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:38:54.167829+0000","last_peered":"2023-08-25T09:38:54.167829+0000","last_clean":"2023-08-25T09:38:54.167829+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:38:54.167829+0000","last_undegraded":"2023-08-25T09:38:54.167829+0000","last_fullsized":"2023-08-25T09:38:54.167829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:32.241680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078306+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:38:54.078306+0000","last_peered":"2023-08-25T09:38:54.078306+0000","last_clean":"2023-08-25T09:38:54.078306+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:38:54.078306+0000","last_undegraded":"2023-08-25T09:38:54.078306+0000","last_fullsized":"2023-08-25T09:38:54.078306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:40:49.888170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170349+0000","last_change":"2023-08-25T09:38:52.145925+0000","last_active":"2023-08-25T09:38:54.170349+0000","last_peered":"2023-08-25T09:38:54.170349+0000","last_clean":"2023-08-25T09:38:54.170349+0000","last_became_active":"2023-08-25T09:38:52.145572+0000","last_became_peered":"2023-08-25T09:38:52.145572+0000","last_unstale":"2023-08-25T09:38:54.170349+0000","last_undegraded":"2023-08-25T09:38:54.170349+0000","last_fullsized":"2023-08-25T09:38:54.170349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:19:18.032882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076523+0000","last_change":"2023-08-25T09:38:53.098547+0000","last_active":"2023-08-25T09:38:54.076523+0000","last_peered":"2023-08-25T09:38:54.076523+0000","last_clean":"2023-08-25T09:38:54.076523+0000","last_became_active":"2023-08-25T09:38:53.098162+0000","last_became_peered":"2023-08-25T09:38:53.098162+0000","last_unstale":"2023-08-25T09:38:54.076523+0000","last_undegraded":"2023-08-25T09:38:54.076523+0000","last_fullsized":"2023-08-25T09:38:54.076523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:14.282266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076947+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:38:54.076947+0000","last_peered":"2023-08-25T09:38:54.076947+0000","last_clean":"2023-08-25T09:38:54.076947+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:38:54.076947+0000","last_undegraded":"2023-08-25T09:38:54.076947+0000","last_fullsized":"2023-08-25T09:38:54.076947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:41.461997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079667+0000","last_change":"2023-08-25T09:38:52.146575+0000","last_active":"2023-08-25T09:38:54.079667+0000","last_peered":"2023-08-25T09:38:54.079667+0000","last_clean":"2023-08-25T09:38:54.079667+0000","last_became_active":"2023-08-25T09:38:52.146263+0000","last_became_peered":"2023-08-25T09:38:52.146263+0000","last_unstale":"2023-08-25T09:38:54.079667+0000","last_undegraded":"2023-08-25T09:38:54.079667+0000","last_fullsized":"2023-08-25T09:38:54.079667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:58.354866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170732+0000","last_change":"2023-08-25T09:38:53.110355+0000","last_active":"2023-08-25T09:38:54.170732+0000","last_peered":"2023-08-25T09:38:54.170732+0000","last_clean":"2023-08-25T09:38:54.170732+0000","last_became_active":"2023-08-25T09:38:53.107934+0000","last_became_peered":"2023-08-25T09:38:53.107934+0000","last_unstale":"2023-08-25T09:38:54.170732+0000","last_undegraded":"2023-08-25T09:38:54.170732+0000","last_fullsized":"2023-08-25T09:38:54.170732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080971+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:38:54.080971+0000","last_peered":"2023-08-25T09:38:54.080971+0000","last_clean":"2023-08-25T09:38:54.080971+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:38:54.080971+0000","last_undegraded":"2023-08-25T09:38:54.080971+0000","last_fullsized":"2023-08-25T09:38:54.080971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:01.142595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170917+0000","last_change":"2023-08-25T09:38:52.149368+0000","last_active":"2023-08-25T09:38:54.170917+0000","last_peered":"2023-08-25T09:38:54.170917+0000","last_clean":"2023-08-25T09:38:54.170917+0000","last_became_active":"2023-08-25T09:38:52.149118+0000","last_became_peered":"2023-08-25T09:38:52.149118+0000","last_unstale":"2023-08-25T09:38:54.170917+0000","last_undegraded":"2023-08-25T09:38:54.170917+0000","last_fullsized":"2023-08-25T09:38:54.170917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:16.835823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078872+0000","last_change":"2023-08-25T09:38:53.108258+0000","last_active":"2023-08-25T09:38:54.078872+0000","last_peered":"2023-08-25T09:38:54.078872+0000","last_clean":"2023-08-25T09:38:54.078872+0000","last_became_active":"2023-08-25T09:38:53.107991+0000","last_became_peered":"2023-08-25T09:38:53.107991+0000","last_unstale":"2023-08-25T09:38:54.078872+0000","last_undegraded":"2023-08-25T09:38:54.078872+0000","last_fullsized":"2023-08-25T09:38:54.078872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078433+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:38:54.078433+0000","last_peered":"2023-08-25T09:38:54.078433+0000","last_clean":"2023-08-25T09:38:54.078433+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:38:54.078433+0000","last_undegraded":"2023-08-25T09:38:54.078433+0000","last_fullsized":"2023-08-25T09:38:54.078433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:17.906346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215184+0000","last_change":"2023-08-25T09:38:52.146434+0000","last_active":"2023-08-25T09:38:54.215184+0000","last_peered":"2023-08-25T09:38:54.215184+0000","last_clean":"2023-08-25T09:38:54.215184+0000","last_became_active":"2023-08-25T09:38:52.146126+0000","last_became_peered":"2023-08-25T09:38:52.146126+0000","last_unstale":"2023-08-25T09:38:54.215184+0000","last_undegraded":"2023-08-25T09:38:54.215184+0000","last_fullsized":"2023-08-25T09:38:54.215184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079123+0000","last_change":"2023-08-25T09:38:53.097778+0000","last_active":"2023-08-25T09:38:54.079123+0000","last_peered":"2023-08-25T09:38:54.079123+0000","last_clean":"2023-08-25T09:38:54.079123+0000","last_became_active":"2023-08-25T09:38:53.097182+0000","last_became_peered":"2023-08-25T09:38:53.097182+0000","last_unstale":"2023-08-25T09:38:54.079123+0000","last_undegraded":"2023-08-25T09:38:54.079123+0000","last_fullsized":"2023-08-25T09:38:54.079123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185740+0000","last_change":"2023-08-25T09:38:52.143868+0000","last_active":"2023-08-25T09:38:54.185740+0000","last_peered":"2023-08-25T09:38:54.185740+0000","last_clean":"2023-08-25T09:38:54.185740+0000","last_became_active":"2023-08-25T09:38:52.143584+0000","last_became_peered":"2023-08-25T09:38:52.143584+0000","last_unstale":"2023-08-25T09:38:54.185740+0000","last_undegraded":"2023-08-25T09:38:54.185740+0000","last_fullsized":"2023-08-25T09:38:54.185740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171239+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:38:54.171239+0000","last_peered":"2023-08-25T09:38:54.171239+0000","last_clean":"2023-08-25T09:38:54.171239+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:38:54.171239+0000","last_undegraded":"2023-08-25T09:38:54.171239+0000","last_fullsized":"2023-08-25T09:38:54.171239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:38.295927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079243+0000","last_change":"2023-08-25T09:38:53.098704+0000","last_active":"2023-08-25T09:38:54.079243+0000","last_peered":"2023-08-25T09:38:54.079243+0000","last_clean":"2023-08-25T09:38:54.079243+0000","last_became_active":"2023-08-25T09:38:53.098360+0000","last_became_peered":"2023-08-25T09:38:53.098360+0000","last_unstale":"2023-08-25T09:38:54.079243+0000","last_undegraded":"2023-08-25T09:38:54.079243+0000","last_fullsized":"2023-08-25T09:38:54.079243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079454+0000","last_change":"2023-08-25T09:38:52.146452+0000","last_active":"2023-08-25T09:38:54.079454+0000","last_peered":"2023-08-25T09:38:54.079454+0000","last_clean":"2023-08-25T09:38:54.079454+0000","last_became_active":"2023-08-25T09:38:52.146193+0000","last_became_peered":"2023-08-25T09:38:52.146193+0000","last_unstale":"2023-08-25T09:38:54.079454+0000","last_undegraded":"2023-08-25T09:38:54.079454+0000","last_fullsized":"2023-08-25T09:38:54.079454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076841+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:38:54.076841+0000","last_peered":"2023-08-25T09:38:54.076841+0000","last_clean":"2023-08-25T09:38:54.076841+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:38:54.076841+0000","last_undegraded":"2023-08-25T09:38:54.076841+0000","last_fullsized":"2023-08-25T09:38:54.076841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:16.377836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079556+0000","last_change":"2023-08-25T09:38:53.109546+0000","last_active":"2023-08-25T09:38:54.079556+0000","last_peered":"2023-08-25T09:38:54.079556+0000","last_clean":"2023-08-25T09:38:54.079556+0000","last_became_active":"2023-08-25T09:38:53.109238+0000","last_became_peered":"2023-08-25T09:38:53.109238+0000","last_unstale":"2023-08-25T09:38:54.079556+0000","last_undegraded":"2023-08-25T09:38:54.079556+0000","last_fullsized":"2023-08-25T09:38:54.079556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170022+0000","last_change":"2023-08-25T09:38:52.144544+0000","last_active":"2023-08-25T09:38:54.170022+0000","last_peered":"2023-08-25T09:38:54.170022+0000","last_clean":"2023-08-25T09:38:54.170022+0000","last_became_active":"2023-08-25T09:38:52.144095+0000","last_became_peered":"2023-08-25T09:38:52.144095+0000","last_unstale":"2023-08-25T09:38:54.170022+0000","last_undegraded":"2023-08-25T09:38:54.170022+0000","last_fullsized":"2023-08-25T09:38:54.170022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:08.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169271+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:38:54.169271+0000","last_peered":"2023-08-25T09:38:54.169271+0000","last_clean":"2023-08-25T09:38:54.169271+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:38:54.169271+0000","last_undegraded":"2023-08-25T09:38:54.169271+0000","last_fullsized":"2023-08-25T09:38:54.169271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:11.717086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169145+0000","last_change":"2023-08-25T09:38:53.112168+0000","last_active":"2023-08-25T09:38:54.169145+0000","last_peered":"2023-08-25T09:38:54.169145+0000","last_clean":"2023-08-25T09:38:54.169145+0000","last_became_active":"2023-08-25T09:38:53.107808+0000","last_became_peered":"2023-08-25T09:38:53.107808+0000","last_unstale":"2023-08-25T09:38:54.169145+0000","last_undegraded":"2023-08-25T09:38:54.169145+0000","last_fullsized":"2023-08-25T09:38:54.169145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169044+0000","last_change":"2023-08-25T09:38:52.143505+0000","last_active":"2023-08-25T09:38:54.169044+0000","last_peered":"2023-08-25T09:38:54.169044+0000","last_clean":"2023-08-25T09:38:54.169044+0000","last_became_active":"2023-08-25T09:38:52.143175+0000","last_became_peered":"2023-08-25T09:38:52.143175+0000","last_unstale":"2023-08-25T09:38:54.169044+0000","last_undegraded":"2023-08-25T09:38:54.169044+0000","last_fullsized":"2023-08-25T09:38:54.169044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:51.819062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078207+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:38:54.078207+0000","last_peered":"2023-08-25T09:38:54.078207+0000","last_clean":"2023-08-25T09:38:54.078207+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:38:54.078207+0000","last_undegraded":"2023-08-25T09:38:54.078207+0000","last_fullsized":"2023-08-25T09:38:54.078207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:28:24.021135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185610+0000","last_change":"2023-08-25T09:38:53.095808+0000","last_active":"2023-08-25T09:38:54.185610+0000","last_peered":"2023-08-25T09:38:54.185610+0000","last_clean":"2023-08-25T09:38:54.185610+0000","last_became_active":"2023-08-25T09:38:53.095427+0000","last_became_peered":"2023-08-25T09:38:53.095427+0000","last_unstale":"2023-08-25T09:38:54.185610+0000","last_undegraded":"2023-08-25T09:38:54.185610+0000","last_fullsized":"2023-08-25T09:38:54.185610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186067+0000","last_change":"2023-08-25T09:38:53.094102+0000","last_active":"2023-08-25T09:38:54.186067+0000","last_peered":"2023-08-25T09:38:54.186067+0000","last_clean":"2023-08-25T09:38:54.186067+0000","last_became_active":"2023-08-25T09:38:53.093658+0000","last_became_peered":"2023-08-25T09:38:53.093658+0000","last_unstale":"2023-08-25T09:38:54.186067+0000","last_undegraded":"2023-08-25T09:38:54.186067+0000","last_fullsized":"2023-08-25T09:38:54.186067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076645+0000","last_change":"2023-08-25T09:38:52.145307+0000","last_active":"2023-08-25T09:38:54.076645+0000","last_peered":"2023-08-25T09:38:54.076645+0000","last_clean":"2023-08-25T09:38:54.076645+0000","last_became_active":"2023-08-25T09:38:52.145076+0000","last_became_peered":"2023-08-25T09:38:52.145076+0000","last_unstale":"2023-08-25T09:38:54.076645+0000","last_undegraded":"2023-08-25T09:38:54.076645+0000","last_fullsized":"2023-08-25T09:38:54.076645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:16.527497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168761+0000","last_change":"2023-08-25T09:38:53.105922+0000","last_active":"2023-08-25T09:38:54.168761+0000","last_peered":"2023-08-25T09:38:54.168761+0000","last_clean":"2023-08-25T09:38:54.168761+0000","last_became_active":"2023-08-25T09:38:53.105488+0000","last_became_peered":"2023-08-25T09:38:53.105488+0000","last_unstale":"2023-08-25T09:38:54.168761+0000","last_undegraded":"2023-08-25T09:38:54.168761+0000","last_fullsized":"2023-08-25T09:38:54.168761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215307+0000","last_change":"2023-08-25T09:38:52.142639+0000","last_active":"2023-08-25T09:38:54.215307+0000","last_peered":"2023-08-25T09:38:54.215307+0000","last_clean":"2023-08-25T09:38:54.215307+0000","last_became_active":"2023-08-25T09:38:52.142406+0000","last_became_peered":"2023-08-25T09:38:52.142406+0000","last_unstale":"2023-08-25T09:38:54.215307+0000","last_undegraded":"2023-08-25T09:38:54.215307+0000","last_fullsized":"2023-08-25T09:38:54.215307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:49.561545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186057+0000","last_change":"2023-08-25T09:38:52.145084+0000","last_active":"2023-08-25T09:38:54.186057+0000","last_peered":"2023-08-25T09:38:54.186057+0000","last_clean":"2023-08-25T09:38:54.186057+0000","last_became_active":"2023-08-25T09:38:52.144835+0000","last_became_peered":"2023-08-25T09:38:52.144835+0000","last_unstale":"2023-08-25T09:38:54.186057+0000","last_undegraded":"2023-08-25T09:38:54.186057+0000","last_fullsized":"2023-08-25T09:38:54.186057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169334+0000","last_change":"2023-08-25T09:38:53.107353+0000","last_active":"2023-08-25T09:38:54.169334+0000","last_peered":"2023-08-25T09:38:54.169334+0000","last_clean":"2023-08-25T09:38:54.169334+0000","last_became_active":"2023-08-25T09:38:53.106537+0000","last_became_peered":"2023-08-25T09:38:53.106537+0000","last_unstale":"2023-08-25T09:38:54.169334+0000","last_undegraded":"2023-08-25T09:38:54.169334+0000","last_fullsized":"2023-08-25T09:38:54.169334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170432+0000","last_change":"2023-08-25T09:38:52.142676+0000","last_active":"2023-08-25T09:38:54.170432+0000","last_peered":"2023-08-25T09:38:54.170432+0000","last_clean":"2023-08-25T09:38:54.170432+0000","last_became_active":"2023-08-25T09:38:52.142266+0000","last_became_peered":"2023-08-25T09:38:52.142266+0000","last_unstale":"2023-08-25T09:38:54.170432+0000","last_undegraded":"2023-08-25T09:38:54.170432+0000","last_fullsized":"2023-08-25T09:38:54.170432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:05.330534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080254+0000","last_change":"2023-08-25T09:38:53.092162+0000","last_active":"2023-08-25T09:38:54.080254+0000","last_peered":"2023-08-25T09:38:54.080254+0000","last_clean":"2023-08-25T09:38:54.080254+0000","last_became_active":"2023-08-25T09:38:53.091630+0000","last_became_peered":"2023-08-25T09:38:53.091630+0000","last_unstale":"2023-08-25T09:38:54.080254+0000","last_undegraded":"2023-08-25T09:38:54.080254+0000","last_fullsized":"2023-08-25T09:38:54.080254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076680+0000","last_change":"2023-08-25T09:38:53.104609+0000","last_active":"2023-08-25T09:38:54.076680+0000","last_peered":"2023-08-25T09:38:54.076680+0000","last_clean":"2023-08-25T09:38:54.076680+0000","last_became_active":"2023-08-25T09:38:53.104257+0000","last_became_peered":"2023-08-25T09:38:53.104257+0000","last_unstale":"2023-08-25T09:38:54.076680+0000","last_undegraded":"2023-08-25T09:38:54.076680+0000","last_fullsized":"2023-08-25T09:38:54.076680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:15.255372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171027+0000","last_change":"2023-08-25T09:38:52.149252+0000","last_active":"2023-08-25T09:38:54.171027+0000","last_peered":"2023-08-25T09:38:54.171027+0000","last_clean":"2023-08-25T09:38:54.171027+0000","last_became_active":"2023-08-25T09:38:52.148775+0000","last_became_peered":"2023-08-25T09:38:52.148775+0000","last_unstale":"2023-08-25T09:38:54.171027+0000","last_undegraded":"2023-08-25T09:38:54.171027+0000","last_fullsized":"2023-08-25T09:38:54.171027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169503+0000","last_change":"2023-08-25T09:38:53.112427+0000","last_active":"2023-08-25T09:38:54.169503+0000","last_peered":"2023-08-25T09:38:54.169503+0000","last_clean":"2023-08-25T09:38:54.169503+0000","last_became_active":"2023-08-25T09:38:53.110947+0000","last_became_peered":"2023-08-25T09:38:53.110947+0000","last_unstale":"2023-08-25T09:38:54.169503+0000","last_undegraded":"2023-08-25T09:38:54.169503+0000","last_fullsized":"2023-08-25T09:38:54.169503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:35.927820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169802+0000","last_change":"2023-08-25T09:38:52.140188+0000","last_active":"2023-08-25T09:38:54.169802+0000","last_peered":"2023-08-25T09:38:54.169802+0000","last_clean":"2023-08-25T09:38:54.169802+0000","last_became_active":"2023-08-25T09:38:52.139523+0000","last_became_peered":"2023-08-25T09:38:52.139523+0000","last_unstale":"2023-08-25T09:38:54.169802+0000","last_undegraded":"2023-08-25T09:38:54.169802+0000","last_fullsized":"2023-08-25T09:38:54.169802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:04.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078279+0000","last_change":"2023-08-25T09:38:53.089895+0000","last_active":"2023-08-25T09:38:54.078279+0000","last_peered":"2023-08-25T09:38:54.078279+0000","last_clean":"2023-08-25T09:38:54.078279+0000","last_became_active":"2023-08-25T09:38:53.089211+0000","last_became_peered":"2023-08-25T09:38:53.089211+0000","last_unstale":"2023-08-25T09:38:54.078279+0000","last_undegraded":"2023-08-25T09:38:54.078279+0000","last_fullsized":"2023-08-25T09:38:54.078279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167322+0000","last_change":"2023-08-25T09:38:52.149088+0000","last_active":"2023-08-25T09:38:54.167322+0000","last_peered":"2023-08-25T09:38:54.167322+0000","last_clean":"2023-08-25T09:38:54.167322+0000","last_became_active":"2023-08-25T09:38:52.148577+0000","last_became_peered":"2023-08-25T09:38:52.148577+0000","last_unstale":"2023-08-25T09:38:54.167322+0000","last_undegraded":"2023-08-25T09:38:54.167322+0000","last_fullsized":"2023-08-25T09:38:54.167322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:48.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169133+0000","last_change":"2023-08-25T09:38:53.105736+0000","last_active":"2023-08-25T09:38:54.169133+0000","last_peered":"2023-08-25T09:38:54.169133+0000","last_clean":"2023-08-25T09:38:54.169133+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.169133+0000","last_undegraded":"2023-08-25T09:38:54.169133+0000","last_fullsized":"2023-08-25T09:38:54.169133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:14.964126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169659+0000","last_change":"2023-08-25T09:38:52.144372+0000","last_active":"2023-08-25T09:38:54.169659+0000","last_peered":"2023-08-25T09:38:54.169659+0000","last_clean":"2023-08-25T09:38:54.169659+0000","last_became_active":"2023-08-25T09:38:52.143646+0000","last_became_peered":"2023-08-25T09:38:52.143646+0000","last_unstale":"2023-08-25T09:38:54.169659+0000","last_undegraded":"2023-08-25T09:38:54.169659+0000","last_fullsized":"2023-08-25T09:38:54.169659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:43.414664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185820+0000","last_change":"2023-08-25T09:38:53.091214+0000","last_active":"2023-08-25T09:38:54.185820+0000","last_peered":"2023-08-25T09:38:54.185820+0000","last_clean":"2023-08-25T09:38:54.185820+0000","last_became_active":"2023-08-25T09:38:53.090817+0000","last_became_peered":"2023-08-25T09:38:53.090817+0000","last_unstale":"2023-08-25T09:38:54.185820+0000","last_undegraded":"2023-08-25T09:38:54.185820+0000","last_fullsized":"2023-08-25T09:38:54.185820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167705+0000","last_change":"2023-08-25T09:38:52.139178+0000","last_active":"2023-08-25T09:38:54.167705+0000","last_peered":"2023-08-25T09:38:54.167705+0000","last_clean":"2023-08-25T09:38:54.167705+0000","last_became_active":"2023-08-25T09:38:52.138923+0000","last_became_peered":"2023-08-25T09:38:52.138923+0000","last_unstale":"2023-08-25T09:38:54.167705+0000","last_undegraded":"2023-08-25T09:38:54.167705+0000","last_fullsized":"2023-08-25T09:38:54.167705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.098793+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.097585+0000","last_became_peered":"2023-08-25T09:38:53.097585+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077130+0000","last_change":"2023-08-25T09:38:52.143009+0000","last_active":"2023-08-25T09:38:54.077130+0000","last_peered":"2023-08-25T09:38:54.077130+0000","last_clean":"2023-08-25T09:38:54.077130+0000","last_became_active":"2023-08-25T09:38:52.140881+0000","last_became_peered":"2023-08-25T09:38:52.140881+0000","last_unstale":"2023-08-25T09:38:54.077130+0000","last_undegraded":"2023-08-25T09:38:54.077130+0000","last_fullsized":"2023-08-25T09:38:54.077130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:19.948794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078140+0000","last_change":"2023-08-25T09:38:53.100041+0000","last_active":"2023-08-25T09:38:54.078140+0000","last_peered":"2023-08-25T09:38:54.078140+0000","last_clean":"2023-08-25T09:38:54.078140+0000","last_became_active":"2023-08-25T09:38:53.099764+0000","last_became_peered":"2023-08-25T09:38:53.099764+0000","last_unstale":"2023-08-25T09:38:54.078140+0000","last_undegraded":"2023-08-25T09:38:54.078140+0000","last_fullsized":"2023-08-25T09:38:54.078140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:13:51.695417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170584+0000","last_change":"2023-08-25T09:38:52.150397+0000","last_active":"2023-08-25T09:38:54.170584+0000","last_peered":"2023-08-25T09:38:54.170584+0000","last_clean":"2023-08-25T09:38:54.170584+0000","last_became_active":"2023-08-25T09:38:52.150134+0000","last_became_peered":"2023-08-25T09:38:52.150134+0000","last_unstale":"2023-08-25T09:38:54.170584+0000","last_undegraded":"2023-08-25T09:38:54.170584+0000","last_fullsized":"2023-08-25T09:38:54.170584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185984+0000","last_change":"2023-08-25T09:38:53.099998+0000","last_active":"2023-08-25T09:38:54.185984+0000","last_peered":"2023-08-25T09:38:54.185984+0000","last_clean":"2023-08-25T09:38:54.185984+0000","last_became_active":"2023-08-25T09:38:53.099612+0000","last_became_peered":"2023-08-25T09:38:53.099612+0000","last_unstale":"2023-08-25T09:38:54.185984+0000","last_undegraded":"2023-08-25T09:38:54.185984+0000","last_fullsized":"2023-08-25T09:38:54.185984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170948+0000","last_change":"2023-08-25T09:38:52.147242+0000","last_active":"2023-08-25T09:38:54.170948+0000","last_peered":"2023-08-25T09:38:54.170948+0000","last_clean":"2023-08-25T09:38:54.170948+0000","last_became_active":"2023-08-25T09:38:52.146238+0000","last_became_peered":"2023-08-25T09:38:52.146238+0000","last_unstale":"2023-08-25T09:38:54.170948+0000","last_undegraded":"2023-08-25T09:38:54.170948+0000","last_fullsized":"2023-08-25T09:38:54.170948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910561,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":657,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199326264,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2829999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":80,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195231,"statfs":{"total":200000000,"available":199917214,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":89,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195223,"statfs":{"total":200000000,"available":199908845,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":82,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195230,"statfs":{"total":200000000,"available":199915602,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7989999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":664,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194647,"statfs":{"total":200000000,"available":199319519,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":80,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195231,"statfs":{"total":200000000,"available":199917068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]}],"pool_statfs":[]}} 2023-08-25T09:38:57.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:38:57.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.891+0000 7f8fe250c640 1 Processor -- start 2023-08-25T09:38:57.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.892+0000 7f8fe250c640 1 -- start start 2023-08-25T09:38:57.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fe250c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc15aa10 0x7f8fdc0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fe250c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8fdc0b2680 0x7f8fdc06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fe250c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fdc15d3f0 con 0x7f8fdc0ce5b0 2023-08-25T09:38:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fe250c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fdc0b2b10 con 0x7f8fdc0b2680 2023-08-25T09:38:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fe250c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc159d10 con 0x7f8fdc15aa10 2023-08-25T09:38:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fdbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8fdc0ce5b0 0x7f8fdc0ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37992/0 (socket says 172.21.15.70:37992) 2023-08-25T09:38:57.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.893+0000 7f8fdbfff640 1 -- 172.21.15.70:0/3898913534 learned_addr learned my addr 172.21.15.70:0/3898913534 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:57.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fd3fff640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc15aa10 0x7f8fdc0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/3898913534 >> v2:172.21.15.155:3300/0 conn(0x7f8fdc0b2680 0x7f8fdc06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059733520 0 0) 0x7f8fdc15d3f0 con 0x7f8fdc0ce5b0 2023-08-25T09:38:57.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc8005730 con 0x7f8fdc0ce5b0 2023-08-25T09:38:57.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833091611 0 0) 0x7f8fc8005730 con 0x7f8fdc0ce5b0 2023-08-25T09:38:57.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc15aa10 msgr2=0x7f8fdc0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc15aa10 0x7f8fdc0b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 >> v2:172.21.15.155:3300/0 conn(0x7f8fdc0b2680 msgr2=0x7f8fdc06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3898913534 >> v2:172.21.15.155:3300/0 conn(0x7f8fdc0b2680 0x7f8fdc06f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0b2d20 con 0x7f8fdc0ce5b0 2023-08-25T09:38:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8fcc001670 con 0x7f8fdc0ce5b0 2023-08-25T09:38:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 >> v1:172.21.15.70:6789/0 conn(0x7f8fdc0ce5b0 legacy=0x7f8fdc0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc8005b80 0x7f8fc8005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc80069d0 0x7f8fc8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc8007390 0x7f8fc800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc0b2b10 con 0x7f8fc8007390 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc159d10 con 0x7f8fc80069d0 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc800e1d0 con 0x7f8fc8005b80 2023-08-25T09:38:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fd3fff640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc15aa10 0x7f8fdc0b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:57.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.894+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/3898913534 >> v2:172.21.15.155:3300/0 conn(0x7f8fdc0b2680 0x7f8fdc06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:38:57.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc8007390 0x7f8fc800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fd3fff640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc80069d0 0x7f8fc8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fe0aa9640 1 -- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc8005b80 msgr2=0x7f8fc8005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:57.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc8005b80 0x7f8fc8005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fe0aa9640 1 -- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc80069d0 msgr2=0x7f8fc8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc80069d0 0x7f8fc8006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.895+0000 7f8fe0aa9640 1 -- 172.21.15.70:0/3898913534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc8005730 con 0x7f8fc8007390 2023-08-25T09:38:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.896+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc8007390 0x7f8fc800db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4007410 tx=0x7f8fc8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a611218cc5214fa5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.896+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd402cdc0 con 0x7f8fc8007390 2023-08-25T09:38:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.896+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd4034440 con 0x7f8fc8007390 2023-08-25T09:38:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.896+0000 7f8fdb7fe640 1 -- 172.21.15.70:0/3898913534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40347b0 con 0x7f8fc8007390 2023-08-25T09:38:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe250c640 1 -- 172.21.15.70:0/3898913534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc8007390 msgr2=0x7f8fc800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe250c640 1 --2- 172.21.15.70:0/3898913534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc8007390 0x7f8fc800db90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4007410 tx=0x7f8fc8007a00 comp rx=0 tx=0).stop 2023-08-25T09:38:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe0aa9640 1 -- 172.21.15.70:0/3898913534 reap_dead start 2023-08-25T09:38:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe250c640 1 -- 172.21.15.70:0/3898913534 shutdown_connections 2023-08-25T09:38:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe250c640 1 -- 172.21.15.70:0/3898913534 >> 172.21.15.70:0/3898913534 conn(0x7f8fdc069850 msgr2=0x7f8fdc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe250c640 1 -- 172.21.15.70:0/3898913534 shutdown_connections 2023-08-25T09:38:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.897+0000 7f8fe250c640 1 -- 172.21.15.70:0/3898913534 wait complete. 2023-08-25T09:38:57.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.899+0000 7f8fe250c640 1 Processor -- start 2023-08-25T09:38:57.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.899+0000 7f8fe250c640 1 -- start start 2023-08-25T09:38:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fe250c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc8007390 0x7f8fdc13e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fe250c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 0x7f8fdc152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fe250c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc156900 0x7f8fdc156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fe250c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc153330 con 0x7f8fdc152970 2023-08-25T09:38:57.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fe250c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1578a0 con 0x7f8fc8007390 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fe250c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fdc1575c0 con 0x7f8fdc156900 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fd3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 0x7f8fdc152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fd3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 0x7f8fdc152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56680/0 (socket says 172.21.15.70:56680) 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.900+0000 7f8fd3fff640 1 -- 172.21.15.70:0/1721916116 learned_addr learned my addr 172.21.15.70:0/1721916116 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fe0aa9640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc156900 0x7f8fdc156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd3fff640 1 -- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc156900 msgr2=0x7f8fdc156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd3fff640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc156900 0x7f8fdc156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd3fff640 1 -- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc8007390 msgr2=0x7f8fdc13e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:57.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc8007390 0x7f8fdc13e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd3fff640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc8007390 0x7f8fdc13e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:57.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd3fff640 1 -- 172.21.15.70:0/1721916116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0aebe0 con 0x7f8fdc152970 2023-08-25T09:38:57.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd3fff640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 0x7f8fdc152df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4006d40 tx=0x7f8fc400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:57.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.901+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4014890 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.902+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc400c030 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.902+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4014bf0 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.902+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc0aeec0 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.902+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fdc1572c0 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.903+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fdc04e420 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.904+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8fc4014d50 con 0x7f8fdc152970 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.905+0000 7f8fd97fa640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fbc06a080 0x7f8fbc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:57.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.905+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8fc4088620 con 0x7f8fdc152970 2023-08-25T09:38:57.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.905+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fbc06a080 0x7f8fbc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:57.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.906+0000 7f8fdbfff640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fbc06a080 0x7f8fbc06c540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc007ab0 tx=0x7f8fcc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:57.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:57.909+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fc4053da0 con 0x7f8fdc152970 2023-08-25T09:38:58.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.092+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fdc15d6b0 con 0x7f8fbc06a080 2023-08-25T09:38:58.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.101+0000 7f8fd97fa640 1 -- 172.21.15.70:0/1721916116 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280680 (secure 0 0 0) 0x7f8fcc00d940 con 0x7f8fbc06a080 2023-08-25T09:38:58.109 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:58.111 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:38:58.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fbc06a080 msgr2=0x7f8fbc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fbc06a080 0x7f8fbc06c540 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc007ab0 tx=0x7f8fcc009040 comp rx=0 tx=0).stop 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 msgr2=0x7f8fdc152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 0x7f8fdc152df0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4006d40 tx=0x7f8fc400f920 comp rx=0 tx=0).stop 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 shutdown_connections 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fbc06a080 0x7f8fbc06c540 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fdc156900 0x7f8fdc156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fdc152970 0x7f8fdc152df0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 --2- 172.21.15.70:0/1721916116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc8007390 0x7f8fdc13e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.108+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 >> 172.21.15.70:0/1721916116 conn(0x7f8fdc069850 msgr2=0x7f8fdc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:58.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.109+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 shutdown_connections 2023-08-25T09:38:58.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.109+0000 7f8fe250c640 1 -- 172.21.15.70:0/1721916116 wait complete. 2023-08-25T09:38:58.130 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-08-25T09:38:57.318621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1830,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560669,"statfs":{"total":1600000000,"available":1598125544,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002266"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169943+0000","last_change":"2023-08-25T09:38:53.105814+0000","last_active":"2023-08-25T09:38:54.169943+0000","last_peered":"2023-08-25T09:38:54.169943+0000","last_clean":"2023-08-25T09:38:54.169943+0000","last_became_active":"2023-08-25T09:38:53.103612+0000","last_became_peered":"2023-08-25T09:38:53.103612+0000","last_unstale":"2023-08-25T09:38:54.169943+0000","last_undegraded":"2023-08-25T09:38:54.169943+0000","last_fullsized":"2023-08-25T09:38:54.169943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185531+0000","last_change":"2023-08-25T09:38:53.103005+0000","last_active":"2023-08-25T09:38:54.185531+0000","last_peered":"2023-08-25T09:38:54.185531+0000","last_clean":"2023-08-25T09:38:54.185531+0000","last_became_active":"2023-08-25T09:38:53.102634+0000","last_became_peered":"2023-08-25T09:38:53.102634+0000","last_unstale":"2023-08-25T09:38:54.185531+0000","last_undegraded":"2023-08-25T09:38:54.185531+0000","last_fullsized":"2023-08-25T09:38:54.185531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214701+0000","last_change":"2023-08-25T09:38:53.111188+0000","last_active":"2023-08-25T09:38:54.214701+0000","last_peered":"2023-08-25T09:38:54.214701+0000","last_clean":"2023-08-25T09:38:54.214701+0000","last_became_active":"2023-08-25T09:38:53.108190+0000","last_became_peered":"2023-08-25T09:38:53.108190+0000","last_unstale":"2023-08-25T09:38:54.214701+0000","last_undegraded":"2023-08-25T09:38:54.214701+0000","last_fullsized":"2023-08-25T09:38:54.214701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:16.468452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214312+0000","last_change":"2023-08-25T09:38:53.103968+0000","last_active":"2023-08-25T09:38:54.214312+0000","last_peered":"2023-08-25T09:38:54.214312+0000","last_clean":"2023-08-25T09:38:54.214312+0000","last_became_active":"2023-08-25T09:38:53.103676+0000","last_became_peered":"2023-08-25T09:38:53.103676+0000","last_unstale":"2023-08-25T09:38:54.214312+0000","last_undegraded":"2023-08-25T09:38:54.214312+0000","last_fullsized":"2023-08-25T09:38:54.214312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170566+0000","last_change":"2023-08-25T09:38:53.110486+0000","last_active":"2023-08-25T09:38:54.170566+0000","last_peered":"2023-08-25T09:38:54.170566+0000","last_clean":"2023-08-25T09:38:54.170566+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.170566+0000","last_undegraded":"2023-08-25T09:38:54.170566+0000","last_fullsized":"2023-08-25T09:38:54.170566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:15.681428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077139+0000","last_change":"2023-08-25T09:38:53.082883+0000","last_active":"2023-08-25T09:38:54.077139+0000","last_peered":"2023-08-25T09:38:54.077139+0000","last_clean":"2023-08-25T09:38:54.077139+0000","last_became_active":"2023-08-25T09:38:53.081892+0000","last_became_peered":"2023-08-25T09:38:53.081892+0000","last_unstale":"2023-08-25T09:38:54.077139+0000","last_undegraded":"2023-08-25T09:38:54.077139+0000","last_fullsized":"2023-08-25T09:38:54.077139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:25:18.218904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077432+0000","last_change":"2023-08-25T09:38:53.094986+0000","last_active":"2023-08-25T09:38:54.077432+0000","last_peered":"2023-08-25T09:38:54.077432+0000","last_clean":"2023-08-25T09:38:54.077432+0000","last_became_active":"2023-08-25T09:38:53.094533+0000","last_became_peered":"2023-08-25T09:38:53.094533+0000","last_unstale":"2023-08-25T09:38:54.077432+0000","last_undegraded":"2023-08-25T09:38:54.077432+0000","last_fullsized":"2023-08-25T09:38:54.077432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:44.551022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167144+0000","last_change":"2023-08-25T09:38:53.097497+0000","last_active":"2023-08-25T09:38:54.167144+0000","last_peered":"2023-08-25T09:38:54.167144+0000","last_clean":"2023-08-25T09:38:54.167144+0000","last_became_active":"2023-08-25T09:38:53.096835+0000","last_became_peered":"2023-08-25T09:38:53.096835+0000","last_unstale":"2023-08-25T09:38:54.167144+0000","last_undegraded":"2023-08-25T09:38:54.167144+0000","last_fullsized":"2023-08-25T09:38:54.167144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168759+0000","last_change":"2023-08-25T09:38:53.111823+0000","last_active":"2023-08-25T09:38:54.168759+0000","last_peered":"2023-08-25T09:38:54.168759+0000","last_clean":"2023-08-25T09:38:54.168759+0000","last_became_active":"2023-08-25T09:38:53.107075+0000","last_became_peered":"2023-08-25T09:38:53.107075+0000","last_unstale":"2023-08-25T09:38:54.168759+0000","last_undegraded":"2023-08-25T09:38:54.168759+0000","last_fullsized":"2023-08-25T09:38:54.168759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167237+0000","last_change":"2023-08-25T09:38:53.097405+0000","last_active":"2023-08-25T09:38:54.167237+0000","last_peered":"2023-08-25T09:38:54.167237+0000","last_clean":"2023-08-25T09:38:54.167237+0000","last_became_active":"2023-08-25T09:38:53.097063+0000","last_became_peered":"2023-08-25T09:38:53.097063+0000","last_unstale":"2023-08-25T09:38:54.167237+0000","last_undegraded":"2023-08-25T09:38:54.167237+0000","last_fullsized":"2023-08-25T09:38:54.167237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167432+0000","last_change":"2023-08-25T09:38:53.097713+0000","last_active":"2023-08-25T09:38:54.167432+0000","last_peered":"2023-08-25T09:38:54.167432+0000","last_clean":"2023-08-25T09:38:54.167432+0000","last_became_active":"2023-08-25T09:38:53.097250+0000","last_became_peered":"2023-08-25T09:38:53.097250+0000","last_unstale":"2023-08-25T09:38:54.167432+0000","last_undegraded":"2023-08-25T09:38:54.167432+0000","last_fullsized":"2023-08-25T09:38:54.167432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170076+0000","last_change":"2023-08-25T09:38:53.098525+0000","last_active":"2023-08-25T09:38:54.170076+0000","last_peered":"2023-08-25T09:38:54.170076+0000","last_clean":"2023-08-25T09:38:54.170076+0000","last_became_active":"2023-08-25T09:38:53.097701+0000","last_became_peered":"2023-08-25T09:38:53.097701+0000","last_unstale":"2023-08-25T09:38:54.170076+0000","last_undegraded":"2023-08-25T09:38:54.170076+0000","last_fullsized":"2023-08-25T09:38:54.170076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079156+0000","last_change":"2023-08-25T09:38:53.110156+0000","last_active":"2023-08-25T09:38:54.079156+0000","last_peered":"2023-08-25T09:38:54.079156+0000","last_clean":"2023-08-25T09:38:54.079156+0000","last_became_active":"2023-08-25T09:38:53.109921+0000","last_became_peered":"2023-08-25T09:38:53.109921+0000","last_unstale":"2023-08-25T09:38:54.079156+0000","last_undegraded":"2023-08-25T09:38:54.079156+0000","last_fullsized":"2023-08-25T09:38:54.079156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167535+0000","last_change":"2023-08-25T09:38:53.102316+0000","last_active":"2023-08-25T09:38:54.167535+0000","last_peered":"2023-08-25T09:38:54.167535+0000","last_clean":"2023-08-25T09:38:54.167535+0000","last_became_active":"2023-08-25T09:38:53.100943+0000","last_became_peered":"2023-08-25T09:38:53.100943+0000","last_unstale":"2023-08-25T09:38:54.167535+0000","last_undegraded":"2023-08-25T09:38:54.167535+0000","last_fullsized":"2023-08-25T09:38:54.167535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214539+0000","last_change":"2023-08-25T09:38:53.111165+0000","last_active":"2023-08-25T09:38:54.214539+0000","last_peered":"2023-08-25T09:38:54.214539+0000","last_clean":"2023-08-25T09:38:54.214539+0000","last_became_active":"2023-08-25T09:38:53.108285+0000","last_became_peered":"2023-08-25T09:38:53.108285+0000","last_unstale":"2023-08-25T09:38:54.214539+0000","last_undegraded":"2023-08-25T09:38:54.214539+0000","last_fullsized":"2023-08-25T09:38:54.214539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077362+0000","last_change":"2023-08-25T09:38:53.089749+0000","last_active":"2023-08-25T09:38:54.077362+0000","last_peered":"2023-08-25T09:38:54.077362+0000","last_clean":"2023-08-25T09:38:54.077362+0000","last_became_active":"2023-08-25T09:38:53.088753+0000","last_became_peered":"2023-08-25T09:38:53.088753+0000","last_unstale":"2023-08-25T09:38:54.077362+0000","last_undegraded":"2023-08-25T09:38:54.077362+0000","last_fullsized":"2023-08-25T09:38:54.077362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:35.174323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078331+0000","last_change":"2023-08-25T09:38:53.100818+0000","last_active":"2023-08-25T09:38:54.078331+0000","last_peered":"2023-08-25T09:38:54.078331+0000","last_clean":"2023-08-25T09:38:54.078331+0000","last_became_active":"2023-08-25T09:38:53.100555+0000","last_became_peered":"2023-08-25T09:38:53.100555+0000","last_unstale":"2023-08-25T09:38:54.078331+0000","last_undegraded":"2023-08-25T09:38:54.078331+0000","last_fullsized":"2023-08-25T09:38:54.078331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215037+0000","last_change":"2023-08-25T09:38:53.105302+0000","last_active":"2023-08-25T09:38:54.215037+0000","last_peered":"2023-08-25T09:38:54.215037+0000","last_clean":"2023-08-25T09:38:54.215037+0000","last_became_active":"2023-08-25T09:38:53.103402+0000","last_became_peered":"2023-08-25T09:38:53.103402+0000","last_unstale":"2023-08-25T09:38:54.215037+0000","last_undegraded":"2023-08-25T09:38:54.215037+0000","last_fullsized":"2023-08-25T09:38:54.215037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171407+0000","last_change":"2023-08-25T09:38:53.099130+0000","last_active":"2023-08-25T09:38:54.171407+0000","last_peered":"2023-08-25T09:38:54.171407+0000","last_clean":"2023-08-25T09:38:54.171407+0000","last_became_active":"2023-08-25T09:38:53.094676+0000","last_became_peered":"2023-08-25T09:38:53.094676+0000","last_unstale":"2023-08-25T09:38:54.171407+0000","last_undegraded":"2023-08-25T09:38:54.171407+0000","last_fullsized":"2023-08-25T09:38:54.171407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:05.648183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167405+0000","last_change":"2023-08-25T09:38:53.111895+0000","last_active":"2023-08-25T09:38:54.167405+0000","last_peered":"2023-08-25T09:38:54.167405+0000","last_clean":"2023-08-25T09:38:54.167405+0000","last_became_active":"2023-08-25T09:38:53.111626+0000","last_became_peered":"2023-08-25T09:38:53.111626+0000","last_unstale":"2023-08-25T09:38:54.167405+0000","last_undegraded":"2023-08-25T09:38:54.167405+0000","last_fullsized":"2023-08-25T09:38:54.167405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167554+0000","last_change":"2023-08-25T09:38:53.104999+0000","last_active":"2023-08-25T09:38:54.167554+0000","last_peered":"2023-08-25T09:38:54.167554+0000","last_clean":"2023-08-25T09:38:54.167554+0000","last_became_active":"2023-08-25T09:38:53.104760+0000","last_became_peered":"2023-08-25T09:38:53.104760+0000","last_unstale":"2023-08-25T09:38:54.167554+0000","last_undegraded":"2023-08-25T09:38:54.167554+0000","last_fullsized":"2023-08-25T09:38:54.167554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:17.840144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078687+0000","last_change":"2023-08-25T09:38:53.106873+0000","last_active":"2023-08-25T09:38:54.078687+0000","last_peered":"2023-08-25T09:38:54.078687+0000","last_clean":"2023-08-25T09:38:54.078687+0000","last_became_active":"2023-08-25T09:38:53.106581+0000","last_became_peered":"2023-08-25T09:38:53.106581+0000","last_unstale":"2023-08-25T09:38:54.078687+0000","last_undegraded":"2023-08-25T09:38:54.078687+0000","last_fullsized":"2023-08-25T09:38:54.078687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168645+0000","last_change":"2023-08-25T09:38:53.112057+0000","last_active":"2023-08-25T09:38:54.168645+0000","last_peered":"2023-08-25T09:38:54.168645+0000","last_clean":"2023-08-25T09:38:54.168645+0000","last_became_active":"2023-08-25T09:38:53.111710+0000","last_became_peered":"2023-08-25T09:38:53.111710+0000","last_unstale":"2023-08-25T09:38:54.168645+0000","last_undegraded":"2023-08-25T09:38:54.168645+0000","last_fullsized":"2023-08-25T09:38:54.168645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167656+0000","last_change":"2023-08-25T09:38:53.101206+0000","last_active":"2023-08-25T09:38:54.167656+0000","last_peered":"2023-08-25T09:38:54.167656+0000","last_clean":"2023-08-25T09:38:54.167656+0000","last_became_active":"2023-08-25T09:38:53.100575+0000","last_became_peered":"2023-08-25T09:38:53.100575+0000","last_unstale":"2023-08-25T09:38:54.167656+0000","last_undegraded":"2023-08-25T09:38:54.167656+0000","last_fullsized":"2023-08-25T09:38:54.167656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:44.735256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170902+0000","last_change":"2023-08-25T09:38:53.110658+0000","last_active":"2023-08-25T09:38:54.170902+0000","last_peered":"2023-08-25T09:38:54.170902+0000","last_clean":"2023-08-25T09:38:54.170902+0000","last_became_active":"2023-08-25T09:38:53.107553+0000","last_became_peered":"2023-08-25T09:38:53.107553+0000","last_unstale":"2023-08-25T09:38:54.170902+0000","last_undegraded":"2023-08-25T09:38:54.170902+0000","last_fullsized":"2023-08-25T09:38:54.170902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:35.169281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078368+0000","last_change":"2023-08-25T09:38:53.106509+0000","last_active":"2023-08-25T09:38:54.078368+0000","last_peered":"2023-08-25T09:38:54.078368+0000","last_clean":"2023-08-25T09:38:54.078368+0000","last_became_active":"2023-08-25T09:38:53.106227+0000","last_became_peered":"2023-08-25T09:38:53.106227+0000","last_unstale":"2023-08-25T09:38:54.078368+0000","last_undegraded":"2023-08-25T09:38:54.078368+0000","last_fullsized":"2023-08-25T09:38:54.078368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077761+0000","last_change":"2023-08-25T09:38:53.094823+0000","last_active":"2023-08-25T09:38:54.077761+0000","last_peered":"2023-08-25T09:38:54.077761+0000","last_clean":"2023-08-25T09:38:54.077761+0000","last_became_active":"2023-08-25T09:38:53.094160+0000","last_became_peered":"2023-08-25T09:38:53.094160+0000","last_unstale":"2023-08-25T09:38:54.077761+0000","last_undegraded":"2023-08-25T09:38:54.077761+0000","last_fullsized":"2023-08-25T09:38:54.077761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:05:17.873255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169761+0000","last_change":"2023-08-25T09:38:53.109739+0000","last_active":"2023-08-25T09:38:54.169761+0000","last_peered":"2023-08-25T09:38:54.169761+0000","last_clean":"2023-08-25T09:38:54.169761+0000","last_became_active":"2023-08-25T09:38:53.106937+0000","last_became_peered":"2023-08-25T09:38:53.106937+0000","last_unstale":"2023-08-25T09:38:54.169761+0000","last_undegraded":"2023-08-25T09:38:54.169761+0000","last_fullsized":"2023-08-25T09:38:54.169761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170229+0000","last_change":"2023-08-25T09:38:53.102490+0000","last_active":"2023-08-25T09:38:54.170229+0000","last_peered":"2023-08-25T09:38:54.170229+0000","last_clean":"2023-08-25T09:38:54.170229+0000","last_became_active":"2023-08-25T09:38:53.101594+0000","last_became_peered":"2023-08-25T09:38:53.101594+0000","last_unstale":"2023-08-25T09:38:54.170229+0000","last_undegraded":"2023-08-25T09:38:54.170229+0000","last_fullsized":"2023-08-25T09:38:54.170229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214623+0000","last_change":"2023-08-25T09:38:53.092816+0000","last_active":"2023-08-25T09:38:54.214623+0000","last_peered":"2023-08-25T09:38:54.214623+0000","last_clean":"2023-08-25T09:38:54.214623+0000","last_became_active":"2023-08-25T09:38:53.092392+0000","last_became_peered":"2023-08-25T09:38:53.092392+0000","last_unstale":"2023-08-25T09:38:54.214623+0000","last_undegraded":"2023-08-25T09:38:54.214623+0000","last_fullsized":"2023-08-25T09:38:54.214623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079002+0000","last_change":"2023-08-25T09:38:53.104234+0000","last_active":"2023-08-25T09:38:54.079002+0000","last_peered":"2023-08-25T09:38:54.079002+0000","last_clean":"2023-08-25T09:38:54.079002+0000","last_became_active":"2023-08-25T09:38:53.104004+0000","last_became_peered":"2023-08-25T09:38:53.104004+0000","last_unstale":"2023-08-25T09:38:54.079002+0000","last_undegraded":"2023-08-25T09:38:54.079002+0000","last_fullsized":"2023-08-25T09:38:54.079002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167124+0000","last_change":"2023-08-25T09:38:53.106635+0000","last_active":"2023-08-25T09:38:54.167124+0000","last_peered":"2023-08-25T09:38:54.167124+0000","last_clean":"2023-08-25T09:38:54.167124+0000","last_became_active":"2023-08-25T09:38:53.106105+0000","last_became_peered":"2023-08-25T09:38:53.106105+0000","last_unstale":"2023-08-25T09:38:54.167124+0000","last_undegraded":"2023-08-25T09:38:54.167124+0000","last_fullsized":"2023-08-25T09:38:54.167124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078322+0000","last_change":"2023-08-25T09:38:53.108366+0000","last_active":"2023-08-25T09:38:54.078322+0000","last_peered":"2023-08-25T09:38:54.078322+0000","last_clean":"2023-08-25T09:38:54.078322+0000","last_became_active":"2023-08-25T09:38:53.108123+0000","last_became_peered":"2023-08-25T09:38:53.108123+0000","last_unstale":"2023-08-25T09:38:54.078322+0000","last_undegraded":"2023-08-25T09:38:54.078322+0000","last_fullsized":"2023-08-25T09:38:54.078322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077197+0000","last_change":"2023-08-25T09:38:52.142687+0000","last_active":"2023-08-25T09:38:54.077197+0000","last_peered":"2023-08-25T09:38:54.077197+0000","last_clean":"2023-08-25T09:38:54.077197+0000","last_became_active":"2023-08-25T09:38:52.140330+0000","last_became_peered":"2023-08-25T09:38:52.140330+0000","last_unstale":"2023-08-25T09:38:54.077197+0000","last_undegraded":"2023-08-25T09:38:54.077197+0000","last_fullsized":"2023-08-25T09:38:54.077197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:42.836964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076792+0000","last_change":"2023-08-25T09:38:53.094431+0000","last_active":"2023-08-25T09:38:54.076792+0000","last_peered":"2023-08-25T09:38:54.076792+0000","last_clean":"2023-08-25T09:38:54.076792+0000","last_became_active":"2023-08-25T09:38:53.093756+0000","last_became_peered":"2023-08-25T09:38:53.093756+0000","last_unstale":"2023-08-25T09:38:54.076792+0000","last_undegraded":"2023-08-25T09:38:54.076792+0000","last_fullsized":"2023-08-25T09:38:54.076792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:32.683806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079876+0000","last_change":"2023-08-25T09:38:52.138724+0000","last_active":"2023-08-25T09:38:54.079876+0000","last_peered":"2023-08-25T09:38:54.079876+0000","last_clean":"2023-08-25T09:38:54.079876+0000","last_became_active":"2023-08-25T09:38:52.138323+0000","last_became_peered":"2023-08-25T09:38:52.138323+0000","last_unstale":"2023-08-25T09:38:54.079876+0000","last_undegraded":"2023-08-25T09:38:54.079876+0000","last_fullsized":"2023-08-25T09:38:54.079876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170651+0000","last_change":"2023-08-25T09:38:53.098073+0000","last_active":"2023-08-25T09:38:54.170651+0000","last_peered":"2023-08-25T09:38:54.170651+0000","last_clean":"2023-08-25T09:38:54.170651+0000","last_became_active":"2023-08-25T09:38:53.089135+0000","last_became_peered":"2023-08-25T09:38:53.089135+0000","last_unstale":"2023-08-25T09:38:54.170651+0000","last_undegraded":"2023-08-25T09:38:54.170651+0000","last_fullsized":"2023-08-25T09:38:54.170651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078034+0000","last_change":"2023-08-25T09:38:52.146382+0000","last_active":"2023-08-25T09:38:54.078034+0000","last_peered":"2023-08-25T09:38:54.078034+0000","last_clean":"2023-08-25T09:38:54.078034+0000","last_became_active":"2023-08-25T09:38:52.145954+0000","last_became_peered":"2023-08-25T09:38:52.145954+0000","last_unstale":"2023-08-25T09:38:54.078034+0000","last_undegraded":"2023-08-25T09:38:54.078034+0000","last_fullsized":"2023-08-25T09:38:54.078034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:02.701588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169581+0000","last_change":"2023-08-25T09:38:53.106145+0000","last_active":"2023-08-25T09:38:54.169581+0000","last_peered":"2023-08-25T09:38:54.169581+0000","last_clean":"2023-08-25T09:38:54.169581+0000","last_became_active":"2023-08-25T09:38:53.105502+0000","last_became_peered":"2023-08-25T09:38:53.105502+0000","last_unstale":"2023-08-25T09:38:54.169581+0000","last_undegraded":"2023-08-25T09:38:54.169581+0000","last_fullsized":"2023-08-25T09:38:54.169581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:01.061628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167472+0000","last_change":"2023-08-25T09:38:52.148928+0000","last_active":"2023-08-25T09:38:54.167472+0000","last_peered":"2023-08-25T09:38:54.167472+0000","last_clean":"2023-08-25T09:38:54.167472+0000","last_became_active":"2023-08-25T09:38:52.148227+0000","last_became_peered":"2023-08-25T09:38:52.148227+0000","last_unstale":"2023-08-25T09:38:54.167472+0000","last_undegraded":"2023-08-25T09:38:54.167472+0000","last_fullsized":"2023-08-25T09:38:54.167472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214435+0000","last_change":"2023-08-25T09:38:52.138014+0000","last_active":"2023-08-25T09:38:54.214435+0000","last_peered":"2023-08-25T09:38:54.214435+0000","last_clean":"2023-08-25T09:38:54.214435+0000","last_became_active":"2023-08-25T09:38:52.137777+0000","last_became_peered":"2023-08-25T09:38:52.137777+0000","last_unstale":"2023-08-25T09:38:54.214435+0000","last_undegraded":"2023-08-25T09:38:54.214435+0000","last_fullsized":"2023-08-25T09:38:54.214435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:10.214913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077886+0000","last_change":"2023-08-25T09:38:53.100050+0000","last_active":"2023-08-25T09:38:54.077886+0000","last_peered":"2023-08-25T09:38:54.077886+0000","last_clean":"2023-08-25T09:38:54.077886+0000","last_became_active":"2023-08-25T09:38:53.099406+0000","last_became_peered":"2023-08-25T09:38:53.099406+0000","last_unstale":"2023-08-25T09:38:54.077886+0000","last_undegraded":"2023-08-25T09:38:54.077886+0000","last_fullsized":"2023-08-25T09:38:54.077886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:46.330038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186222+0000","last_change":"2023-08-25T09:38:52.145560+0000","last_active":"2023-08-25T09:38:54.186222+0000","last_peered":"2023-08-25T09:38:54.186222+0000","last_clean":"2023-08-25T09:38:54.186222+0000","last_became_active":"2023-08-25T09:38:52.145324+0000","last_became_peered":"2023-08-25T09:38:52.145324+0000","last_unstale":"2023-08-25T09:38:54.186222+0000","last_undegraded":"2023-08-25T09:38:54.186222+0000","last_fullsized":"2023-08-25T09:38:54.186222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:47.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167557+0000","last_change":"2023-08-25T09:38:53.105196+0000","last_active":"2023-08-25T09:38:54.167557+0000","last_peered":"2023-08-25T09:38:54.167557+0000","last_clean":"2023-08-25T09:38:54.167557+0000","last_became_active":"2023-08-25T09:38:53.104948+0000","last_became_peered":"2023-08-25T09:38:53.104948+0000","last_unstale":"2023-08-25T09:38:54.167557+0000","last_undegraded":"2023-08-25T09:38:54.167557+0000","last_fullsized":"2023-08-25T09:38:54.167557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078179+0000","last_change":"2023-08-25T09:38:52.140961+0000","last_active":"2023-08-25T09:38:54.078179+0000","last_peered":"2023-08-25T09:38:54.078179+0000","last_clean":"2023-08-25T09:38:54.078179+0000","last_became_active":"2023-08-25T09:38:52.137710+0000","last_became_peered":"2023-08-25T09:38:52.137710+0000","last_unstale":"2023-08-25T09:38:54.078179+0000","last_undegraded":"2023-08-25T09:38:54.078179+0000","last_fullsized":"2023-08-25T09:38:54.078179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:09.490525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.083467+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.082859+0000","last_became_peered":"2023-08-25T09:38:53.082859+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215256+0000","last_change":"2023-08-25T09:38:52.148012+0000","last_active":"2023-08-25T09:38:54.215256+0000","last_peered":"2023-08-25T09:38:54.215256+0000","last_clean":"2023-08-25T09:38:54.215256+0000","last_became_active":"2023-08-25T09:38:52.147757+0000","last_became_peered":"2023-08-25T09:38:52.147757+0000","last_unstale":"2023-08-25T09:38:54.215256+0000","last_undegraded":"2023-08-25T09:38:54.215256+0000","last_fullsized":"2023-08-25T09:38:54.215256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:36.644423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170501+0000","last_change":"2023-08-25T09:38:53.110777+0000","last_active":"2023-08-25T09:38:54.170501+0000","last_peered":"2023-08-25T09:38:54.170501+0000","last_clean":"2023-08-25T09:38:54.170501+0000","last_became_active":"2023-08-25T09:38:53.108970+0000","last_became_peered":"2023-08-25T09:38:53.108970+0000","last_unstale":"2023-08-25T09:38:54.170501+0000","last_undegraded":"2023-08-25T09:38:54.170501+0000","last_fullsized":"2023-08-25T09:38:54.170501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080573+0000","last_change":"2023-08-25T09:38:52.144190+0000","last_active":"2023-08-25T09:38:54.080573+0000","last_peered":"2023-08-25T09:38:54.080573+0000","last_clean":"2023-08-25T09:38:54.080573+0000","last_became_active":"2023-08-25T09:38:52.143875+0000","last_became_peered":"2023-08-25T09:38:52.143875+0000","last_unstale":"2023-08-25T09:38:54.080573+0000","last_undegraded":"2023-08-25T09:38:54.080573+0000","last_fullsized":"2023-08-25T09:38:54.080573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215099+0000","last_change":"2023-08-25T09:38:53.106602+0000","last_active":"2023-08-25T09:38:54.215099+0000","last_peered":"2023-08-25T09:38:54.215099+0000","last_clean":"2023-08-25T09:38:54.215099+0000","last_became_active":"2023-08-25T09:38:53.106340+0000","last_became_peered":"2023-08-25T09:38:53.106340+0000","last_unstale":"2023-08-25T09:38:54.215099+0000","last_undegraded":"2023-08-25T09:38:54.215099+0000","last_fullsized":"2023-08-25T09:38:54.215099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076776+0000","last_change":"2023-08-25T09:38:52.140744+0000","last_active":"2023-08-25T09:38:54.076776+0000","last_peered":"2023-08-25T09:38:54.076776+0000","last_clean":"2023-08-25T09:38:54.076776+0000","last_became_active":"2023-08-25T09:38:52.140450+0000","last_became_peered":"2023-08-25T09:38:52.140450+0000","last_unstale":"2023-08-25T09:38:54.076776+0000","last_undegraded":"2023-08-25T09:38:54.076776+0000","last_fullsized":"2023-08-25T09:38:54.076776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:54:37.632752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077689+0000","last_change":"2023-08-25T09:38:53.100210+0000","last_active":"2023-08-25T09:38:54.077689+0000","last_peered":"2023-08-25T09:38:54.077689+0000","last_clean":"2023-08-25T09:38:54.077689+0000","last_became_active":"2023-08-25T09:38:53.099792+0000","last_became_peered":"2023-08-25T09:38:53.099792+0000","last_unstale":"2023-08-25T09:38:54.077689+0000","last_undegraded":"2023-08-25T09:38:54.077689+0000","last_fullsized":"2023-08-25T09:38:54.077689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:37.979410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077328+0000","last_change":"2023-08-25T09:38:52.132704+0000","last_active":"2023-08-25T09:38:54.077328+0000","last_peered":"2023-08-25T09:38:54.077328+0000","last_clean":"2023-08-25T09:38:54.077328+0000","last_became_active":"2023-08-25T09:38:52.132045+0000","last_became_peered":"2023-08-25T09:38:52.132045+0000","last_unstale":"2023-08-25T09:38:54.077328+0000","last_undegraded":"2023-08-25T09:38:54.077328+0000","last_fullsized":"2023-08-25T09:38:54.077328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:32:14.124629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186144+0000","last_change":"2023-08-25T09:38:53.095472+0000","last_active":"2023-08-25T09:38:54.186144+0000","last_peered":"2023-08-25T09:38:54.186144+0000","last_clean":"2023-08-25T09:38:54.186144+0000","last_became_active":"2023-08-25T09:38:53.095117+0000","last_became_peered":"2023-08-25T09:38:53.095117+0000","last_unstale":"2023-08-25T09:38:54.186144+0000","last_undegraded":"2023-08-25T09:38:54.186144+0000","last_fullsized":"2023-08-25T09:38:54.186144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169438+0000","last_change":"2023-08-25T09:38:52.142123+0000","last_active":"2023-08-25T09:38:54.169438+0000","last_peered":"2023-08-25T09:38:54.169438+0000","last_clean":"2023-08-25T09:38:54.169438+0000","last_became_active":"2023-08-25T09:38:52.141349+0000","last_became_peered":"2023-08-25T09:38:52.141349+0000","last_unstale":"2023-08-25T09:38:54.169438+0000","last_undegraded":"2023-08-25T09:38:54.169438+0000","last_fullsized":"2023-08-25T09:38:54.169438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078490+0000","last_change":"2023-08-25T09:38:53.101599+0000","last_active":"2023-08-25T09:38:54.078490+0000","last_peered":"2023-08-25T09:38:54.078490+0000","last_clean":"2023-08-25T09:38:54.078490+0000","last_became_active":"2023-08-25T09:38:53.101348+0000","last_became_peered":"2023-08-25T09:38:53.101348+0000","last_unstale":"2023-08-25T09:38:54.078490+0000","last_undegraded":"2023-08-25T09:38:54.078490+0000","last_fullsized":"2023-08-25T09:38:54.078490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167829+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:38:54.167829+0000","last_peered":"2023-08-25T09:38:54.167829+0000","last_clean":"2023-08-25T09:38:54.167829+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:38:54.167829+0000","last_undegraded":"2023-08-25T09:38:54.167829+0000","last_fullsized":"2023-08-25T09:38:54.167829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:32.241680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078306+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:38:54.078306+0000","last_peered":"2023-08-25T09:38:54.078306+0000","last_clean":"2023-08-25T09:38:54.078306+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:38:54.078306+0000","last_undegraded":"2023-08-25T09:38:54.078306+0000","last_fullsized":"2023-08-25T09:38:54.078306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:40:49.888170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170349+0000","last_change":"2023-08-25T09:38:52.145925+0000","last_active":"2023-08-25T09:38:54.170349+0000","last_peered":"2023-08-25T09:38:54.170349+0000","last_clean":"2023-08-25T09:38:54.170349+0000","last_became_active":"2023-08-25T09:38:52.145572+0000","last_became_peered":"2023-08-25T09:38:52.145572+0000","last_unstale":"2023-08-25T09:38:54.170349+0000","last_undegraded":"2023-08-25T09:38:54.170349+0000","last_fullsized":"2023-08-25T09:38:54.170349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:19:18.032882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076523+0000","last_change":"2023-08-25T09:38:53.098547+0000","last_active":"2023-08-25T09:38:54.076523+0000","last_peered":"2023-08-25T09:38:54.076523+0000","last_clean":"2023-08-25T09:38:54.076523+0000","last_became_active":"2023-08-25T09:38:53.098162+0000","last_became_peered":"2023-08-25T09:38:53.098162+0000","last_unstale":"2023-08-25T09:38:54.076523+0000","last_undegraded":"2023-08-25T09:38:54.076523+0000","last_fullsized":"2023-08-25T09:38:54.076523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:14.282266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076947+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:38:54.076947+0000","last_peered":"2023-08-25T09:38:54.076947+0000","last_clean":"2023-08-25T09:38:54.076947+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:38:54.076947+0000","last_undegraded":"2023-08-25T09:38:54.076947+0000","last_fullsized":"2023-08-25T09:38:54.076947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:41.461997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079667+0000","last_change":"2023-08-25T09:38:52.146575+0000","last_active":"2023-08-25T09:38:54.079667+0000","last_peered":"2023-08-25T09:38:54.079667+0000","last_clean":"2023-08-25T09:38:54.079667+0000","last_became_active":"2023-08-25T09:38:52.146263+0000","last_became_peered":"2023-08-25T09:38:52.146263+0000","last_unstale":"2023-08-25T09:38:54.079667+0000","last_undegraded":"2023-08-25T09:38:54.079667+0000","last_fullsized":"2023-08-25T09:38:54.079667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:58.354866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170732+0000","last_change":"2023-08-25T09:38:53.110355+0000","last_active":"2023-08-25T09:38:54.170732+0000","last_peered":"2023-08-25T09:38:54.170732+0000","last_clean":"2023-08-25T09:38:54.170732+0000","last_became_active":"2023-08-25T09:38:53.107934+0000","last_became_peered":"2023-08-25T09:38:53.107934+0000","last_unstale":"2023-08-25T09:38:54.170732+0000","last_undegraded":"2023-08-25T09:38:54.170732+0000","last_fullsized":"2023-08-25T09:38:54.170732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080971+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:38:54.080971+0000","last_peered":"2023-08-25T09:38:54.080971+0000","last_clean":"2023-08-25T09:38:54.080971+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:38:54.080971+0000","last_undegraded":"2023-08-25T09:38:54.080971+0000","last_fullsized":"2023-08-25T09:38:54.080971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:01.142595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170917+0000","last_change":"2023-08-25T09:38:52.149368+0000","last_active":"2023-08-25T09:38:54.170917+0000","last_peered":"2023-08-25T09:38:54.170917+0000","last_clean":"2023-08-25T09:38:54.170917+0000","last_became_active":"2023-08-25T09:38:52.149118+0000","last_became_peered":"2023-08-25T09:38:52.149118+0000","last_unstale":"2023-08-25T09:38:54.170917+0000","last_undegraded":"2023-08-25T09:38:54.170917+0000","last_fullsized":"2023-08-25T09:38:54.170917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:16.835823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078872+0000","last_change":"2023-08-25T09:38:53.108258+0000","last_active":"2023-08-25T09:38:54.078872+0000","last_peered":"2023-08-25T09:38:54.078872+0000","last_clean":"2023-08-25T09:38:54.078872+0000","last_became_active":"2023-08-25T09:38:53.107991+0000","last_became_peered":"2023-08-25T09:38:53.107991+0000","last_unstale":"2023-08-25T09:38:54.078872+0000","last_undegraded":"2023-08-25T09:38:54.078872+0000","last_fullsized":"2023-08-25T09:38:54.078872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078433+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:38:54.078433+0000","last_peered":"2023-08-25T09:38:54.078433+0000","last_clean":"2023-08-25T09:38:54.078433+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:38:54.078433+0000","last_undegraded":"2023-08-25T09:38:54.078433+0000","last_fullsized":"2023-08-25T09:38:54.078433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:17.906346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215184+0000","last_change":"2023-08-25T09:38:52.146434+0000","last_active":"2023-08-25T09:38:54.215184+0000","last_peered":"2023-08-25T09:38:54.215184+0000","last_clean":"2023-08-25T09:38:54.215184+0000","last_became_active":"2023-08-25T09:38:52.146126+0000","last_became_peered":"2023-08-25T09:38:52.146126+0000","last_unstale":"2023-08-25T09:38:54.215184+0000","last_undegraded":"2023-08-25T09:38:54.215184+0000","last_fullsized":"2023-08-25T09:38:54.215184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079123+0000","last_change":"2023-08-25T09:38:53.097778+0000","last_active":"2023-08-25T09:38:54.079123+0000","last_peered":"2023-08-25T09:38:54.079123+0000","last_clean":"2023-08-25T09:38:54.079123+0000","last_became_active":"2023-08-25T09:38:53.097182+0000","last_became_peered":"2023-08-25T09:38:53.097182+0000","last_unstale":"2023-08-25T09:38:54.079123+0000","last_undegraded":"2023-08-25T09:38:54.079123+0000","last_fullsized":"2023-08-25T09:38:54.079123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185740+0000","last_change":"2023-08-25T09:38:52.143868+0000","last_active":"2023-08-25T09:38:54.185740+0000","last_peered":"2023-08-25T09:38:54.185740+0000","last_clean":"2023-08-25T09:38:54.185740+0000","last_became_active":"2023-08-25T09:38:52.143584+0000","last_became_peered":"2023-08-25T09:38:52.143584+0000","last_unstale":"2023-08-25T09:38:54.185740+0000","last_undegraded":"2023-08-25T09:38:54.185740+0000","last_fullsized":"2023-08-25T09:38:54.185740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171239+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:38:54.171239+0000","last_peered":"2023-08-25T09:38:54.171239+0000","last_clean":"2023-08-25T09:38:54.171239+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:38:54.171239+0000","last_undegraded":"2023-08-25T09:38:54.171239+0000","last_fullsized":"2023-08-25T09:38:54.171239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:38.295927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079243+0000","last_change":"2023-08-25T09:38:53.098704+0000","last_active":"2023-08-25T09:38:54.079243+0000","last_peered":"2023-08-25T09:38:54.079243+0000","last_clean":"2023-08-25T09:38:54.079243+0000","last_became_active":"2023-08-25T09:38:53.098360+0000","last_became_peered":"2023-08-25T09:38:53.098360+0000","last_unstale":"2023-08-25T09:38:54.079243+0000","last_undegraded":"2023-08-25T09:38:54.079243+0000","last_fullsized":"2023-08-25T09:38:54.079243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079454+0000","last_change":"2023-08-25T09:38:52.146452+0000","last_active":"2023-08-25T09:38:54.079454+0000","last_peered":"2023-08-25T09:38:54.079454+0000","last_clean":"2023-08-25T09:38:54.079454+0000","last_became_active":"2023-08-25T09:38:52.146193+0000","last_became_peered":"2023-08-25T09:38:52.146193+0000","last_unstale":"2023-08-25T09:38:54.079454+0000","last_undegraded":"2023-08-25T09:38:54.079454+0000","last_fullsized":"2023-08-25T09:38:54.079454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076841+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:38:54.076841+0000","last_peered":"2023-08-25T09:38:54.076841+0000","last_clean":"2023-08-25T09:38:54.076841+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:38:54.076841+0000","last_undegraded":"2023-08-25T09:38:54.076841+0000","last_fullsized":"2023-08-25T09:38:54.076841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:16.377836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079556+0000","last_change":"2023-08-25T09:38:53.109546+0000","last_active":"2023-08-25T09:38:54.079556+0000","last_peered":"2023-08-25T09:38:54.079556+0000","last_clean":"2023-08-25T09:38:54.079556+0000","last_became_active":"2023-08-25T09:38:53.109238+0000","last_became_peered":"2023-08-25T09:38:53.109238+0000","last_unstale":"2023-08-25T09:38:54.079556+0000","last_undegraded":"2023-08-25T09:38:54.079556+0000","last_fullsized":"2023-08-25T09:38:54.079556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170022+0000","last_change":"2023-08-25T09:38:52.144544+0000","last_active":"2023-08-25T09:38:54.170022+0000","last_peered":"2023-08-25T09:38:54.170022+0000","last_clean":"2023-08-25T09:38:54.170022+0000","last_became_active":"2023-08-25T09:38:52.144095+0000","last_became_peered":"2023-08-25T09:38:52.144095+0000","last_unstale":"2023-08-25T09:38:54.170022+0000","last_undegraded":"2023-08-25T09:38:54.170022+0000","last_fullsized":"2023-08-25T09:38:54.170022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:08.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169271+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:38:54.169271+0000","last_peered":"2023-08-25T09:38:54.169271+0000","last_clean":"2023-08-25T09:38:54.169271+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:38:54.169271+0000","last_undegraded":"2023-08-25T09:38:54.169271+0000","last_fullsized":"2023-08-25T09:38:54.169271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:11.717086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169145+0000","last_change":"2023-08-25T09:38:53.112168+0000","last_active":"2023-08-25T09:38:54.169145+0000","last_peered":"2023-08-25T09:38:54.169145+0000","last_clean":"2023-08-25T09:38:54.169145+0000","last_became_active":"2023-08-25T09:38:53.107808+0000","last_became_peered":"2023-08-25T09:38:53.107808+0000","last_unstale":"2023-08-25T09:38:54.169145+0000","last_undegraded":"2023-08-25T09:38:54.169145+0000","last_fullsized":"2023-08-25T09:38:54.169145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169044+0000","last_change":"2023-08-25T09:38:52.143505+0000","last_active":"2023-08-25T09:38:54.169044+0000","last_peered":"2023-08-25T09:38:54.169044+0000","last_clean":"2023-08-25T09:38:54.169044+0000","last_became_active":"2023-08-25T09:38:52.143175+0000","last_became_peered":"2023-08-25T09:38:52.143175+0000","last_unstale":"2023-08-25T09:38:54.169044+0000","last_undegraded":"2023-08-25T09:38:54.169044+0000","last_fullsized":"2023-08-25T09:38:54.169044+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:51.819062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078207+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:38:54.078207+0000","last_peered":"2023-08-25T09:38:54.078207+0000","last_clean":"2023-08-25T09:38:54.078207+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:38:54.078207+0000","last_undegraded":"2023-08-25T09:38:54.078207+0000","last_fullsized":"2023-08-25T09:38:54.078207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:28:24.021135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185610+0000","last_change":"2023-08-25T09:38:53.095808+0000","last_active":"2023-08-25T09:38:54.185610+0000","last_peered":"2023-08-25T09:38:54.185610+0000","last_clean":"2023-08-25T09:38:54.185610+0000","last_became_active":"2023-08-25T09:38:53.095427+0000","last_became_peered":"2023-08-25T09:38:53.095427+0000","last_unstale":"2023-08-25T09:38:54.185610+0000","last_undegraded":"2023-08-25T09:38:54.185610+0000","last_fullsized":"2023-08-25T09:38:54.185610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186067+0000","last_change":"2023-08-25T09:38:53.094102+0000","last_active":"2023-08-25T09:38:54.186067+0000","last_peered":"2023-08-25T09:38:54.186067+0000","last_clean":"2023-08-25T09:38:54.186067+0000","last_became_active":"2023-08-25T09:38:53.093658+0000","last_became_peered":"2023-08-25T09:38:53.093658+0000","last_unstale":"2023-08-25T09:38:54.186067+0000","last_undegraded":"2023-08-25T09:38:54.186067+0000","last_fullsized":"2023-08-25T09:38:54.186067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076645+0000","last_change":"2023-08-25T09:38:52.145307+0000","last_active":"2023-08-25T09:38:54.076645+0000","last_peered":"2023-08-25T09:38:54.076645+0000","last_clean":"2023-08-25T09:38:54.076645+0000","last_became_active":"2023-08-25T09:38:52.145076+0000","last_became_peered":"2023-08-25T09:38:52.145076+0000","last_unstale":"2023-08-25T09:38:54.076645+0000","last_undegraded":"2023-08-25T09:38:54.076645+0000","last_fullsized":"2023-08-25T09:38:54.076645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:16.527497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168761+0000","last_change":"2023-08-25T09:38:53.105922+0000","last_active":"2023-08-25T09:38:54.168761+0000","last_peered":"2023-08-25T09:38:54.168761+0000","last_clean":"2023-08-25T09:38:54.168761+0000","last_became_active":"2023-08-25T09:38:53.105488+0000","last_became_peered":"2023-08-25T09:38:53.105488+0000","last_unstale":"2023-08-25T09:38:54.168761+0000","last_undegraded":"2023-08-25T09:38:54.168761+0000","last_fullsized":"2023-08-25T09:38:54.168761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215307+0000","last_change":"2023-08-25T09:38:52.142639+0000","last_active":"2023-08-25T09:38:54.215307+0000","last_peered":"2023-08-25T09:38:54.215307+0000","last_clean":"2023-08-25T09:38:54.215307+0000","last_became_active":"2023-08-25T09:38:52.142406+0000","last_became_peered":"2023-08-25T09:38:52.142406+0000","last_unstale":"2023-08-25T09:38:54.215307+0000","last_undegraded":"2023-08-25T09:38:54.215307+0000","last_fullsized":"2023-08-25T09:38:54.215307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:49.561545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186057+0000","last_change":"2023-08-25T09:38:52.145084+0000","last_active":"2023-08-25T09:38:54.186057+0000","last_peered":"2023-08-25T09:38:54.186057+0000","last_clean":"2023-08-25T09:38:54.186057+0000","last_became_active":"2023-08-25T09:38:52.144835+0000","last_became_peered":"2023-08-25T09:38:52.144835+0000","last_unstale":"2023-08-25T09:38:54.186057+0000","last_undegraded":"2023-08-25T09:38:54.186057+0000","last_fullsized":"2023-08-25T09:38:54.186057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169334+0000","last_change":"2023-08-25T09:38:53.107353+0000","last_active":"2023-08-25T09:38:54.169334+0000","last_peered":"2023-08-25T09:38:54.169334+0000","last_clean":"2023-08-25T09:38:54.169334+0000","last_became_active":"2023-08-25T09:38:53.106537+0000","last_became_peered":"2023-08-25T09:38:53.106537+0000","last_unstale":"2023-08-25T09:38:54.169334+0000","last_undegraded":"2023-08-25T09:38:54.169334+0000","last_fullsized":"2023-08-25T09:38:54.169334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170432+0000","last_change":"2023-08-25T09:38:52.142676+0000","last_active":"2023-08-25T09:38:54.170432+0000","last_peered":"2023-08-25T09:38:54.170432+0000","last_clean":"2023-08-25T09:38:54.170432+0000","last_became_active":"2023-08-25T09:38:52.142266+0000","last_became_peered":"2023-08-25T09:38:52.142266+0000","last_unstale":"2023-08-25T09:38:54.170432+0000","last_undegraded":"2023-08-25T09:38:54.170432+0000","last_fullsized":"2023-08-25T09:38:54.170432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:05.330534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080254+0000","last_change":"2023-08-25T09:38:53.092162+0000","last_active":"2023-08-25T09:38:54.080254+0000","last_peered":"2023-08-25T09:38:54.080254+0000","last_clean":"2023-08-25T09:38:54.080254+0000","last_became_active":"2023-08-25T09:38:53.091630+0000","last_became_peered":"2023-08-25T09:38:53.091630+0000","last_unstale":"2023-08-25T09:38:54.080254+0000","last_undegraded":"2023-08-25T09:38:54.080254+0000","last_fullsized":"2023-08-25T09:38:54.080254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076680+0000","last_change":"2023-08-25T09:38:53.104609+0000","last_active":"2023-08-25T09:38:54.076680+0000","last_peered":"2023-08-25T09:38:54.076680+0000","last_clean":"2023-08-25T09:38:54.076680+0000","last_became_active":"2023-08-25T09:38:53.104257+0000","last_became_peered":"2023-08-25T09:38:53.104257+0000","last_unstale":"2023-08-25T09:38:54.076680+0000","last_undegraded":"2023-08-25T09:38:54.076680+0000","last_fullsized":"2023-08-25T09:38:54.076680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:15.255372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171027+0000","last_change":"2023-08-25T09:38:52.149252+0000","last_active":"2023-08-25T09:38:54.171027+0000","last_peered":"2023-08-25T09:38:54.171027+0000","last_clean":"2023-08-25T09:38:54.171027+0000","last_became_active":"2023-08-25T09:38:52.148775+0000","last_became_peered":"2023-08-25T09:38:52.148775+0000","last_unstale":"2023-08-25T09:38:54.171027+0000","last_undegraded":"2023-08-25T09:38:54.171027+0000","last_fullsized":"2023-08-25T09:38:54.171027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169503+0000","last_change":"2023-08-25T09:38:53.112427+0000","last_active":"2023-08-25T09:38:54.169503+0000","last_peered":"2023-08-25T09:38:54.169503+0000","last_clean":"2023-08-25T09:38:54.169503+0000","last_became_active":"2023-08-25T09:38:53.110947+0000","last_became_peered":"2023-08-25T09:38:53.110947+0000","last_unstale":"2023-08-25T09:38:54.169503+0000","last_undegraded":"2023-08-25T09:38:54.169503+0000","last_fullsized":"2023-08-25T09:38:54.169503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:35.927820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169802+0000","last_change":"2023-08-25T09:38:52.140188+0000","last_active":"2023-08-25T09:38:54.169802+0000","last_peered":"2023-08-25T09:38:54.169802+0000","last_clean":"2023-08-25T09:38:54.169802+0000","last_became_active":"2023-08-25T09:38:52.139523+0000","last_became_peered":"2023-08-25T09:38:52.139523+0000","last_unstale":"2023-08-25T09:38:54.169802+0000","last_undegraded":"2023-08-25T09:38:54.169802+0000","last_fullsized":"2023-08-25T09:38:54.169802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:04.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078279+0000","last_change":"2023-08-25T09:38:53.089895+0000","last_active":"2023-08-25T09:38:54.078279+0000","last_peered":"2023-08-25T09:38:54.078279+0000","last_clean":"2023-08-25T09:38:54.078279+0000","last_became_active":"2023-08-25T09:38:53.089211+0000","last_became_peered":"2023-08-25T09:38:53.089211+0000","last_unstale":"2023-08-25T09:38:54.078279+0000","last_undegraded":"2023-08-25T09:38:54.078279+0000","last_fullsized":"2023-08-25T09:38:54.078279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167322+0000","last_change":"2023-08-25T09:38:52.149088+0000","last_active":"2023-08-25T09:38:54.167322+0000","last_peered":"2023-08-25T09:38:54.167322+0000","last_clean":"2023-08-25T09:38:54.167322+0000","last_became_active":"2023-08-25T09:38:52.148577+0000","last_became_peered":"2023-08-25T09:38:52.148577+0000","last_unstale":"2023-08-25T09:38:54.167322+0000","last_undegraded":"2023-08-25T09:38:54.167322+0000","last_fullsized":"2023-08-25T09:38:54.167322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:48.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169133+0000","last_change":"2023-08-25T09:38:53.105736+0000","last_active":"2023-08-25T09:38:54.169133+0000","last_peered":"2023-08-25T09:38:54.169133+0000","last_clean":"2023-08-25T09:38:54.169133+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.169133+0000","last_undegraded":"2023-08-25T09:38:54.169133+0000","last_fullsized":"2023-08-25T09:38:54.169133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:14.964126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169659+0000","last_change":"2023-08-25T09:38:52.144372+0000","last_active":"2023-08-25T09:38:54.169659+0000","last_peered":"2023-08-25T09:38:54.169659+0000","last_clean":"2023-08-25T09:38:54.169659+0000","last_became_active":"2023-08-25T09:38:52.143646+0000","last_became_peered":"2023-08-25T09:38:52.143646+0000","last_unstale":"2023-08-25T09:38:54.169659+0000","last_undegraded":"2023-08-25T09:38:54.169659+0000","last_fullsized":"2023-08-25T09:38:54.169659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:43.414664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185820+0000","last_change":"2023-08-25T09:38:53.091214+0000","last_active":"2023-08-25T09:38:54.185820+0000","last_peered":"2023-08-25T09:38:54.185820+0000","last_clean":"2023-08-25T09:38:54.185820+0000","last_became_active":"2023-08-25T09:38:53.090817+0000","last_became_peered":"2023-08-25T09:38:53.090817+0000","last_unstale":"2023-08-25T09:38:54.185820+0000","last_undegraded":"2023-08-25T09:38:54.185820+0000","last_fullsized":"2023-08-25T09:38:54.185820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167705+0000","last_change":"2023-08-25T09:38:52.139178+0000","last_active":"2023-08-25T09:38:54.167705+0000","last_peered":"2023-08-25T09:38:54.167705+0000","last_clean":"2023-08-25T09:38:54.167705+0000","last_became_active":"2023-08-25T09:38:52.138923+0000","last_became_peered":"2023-08-25T09:38:52.138923+0000","last_unstale":"2023-08-25T09:38:54.167705+0000","last_undegraded":"2023-08-25T09:38:54.167705+0000","last_fullsized":"2023-08-25T09:38:54.167705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.098793+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.097585+0000","last_became_peered":"2023-08-25T09:38:53.097585+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077130+0000","last_change":"2023-08-25T09:38:52.143009+0000","last_active":"2023-08-25T09:38:54.077130+0000","last_peered":"2023-08-25T09:38:54.077130+0000","last_clean":"2023-08-25T09:38:54.077130+0000","last_became_active":"2023-08-25T09:38:52.140881+0000","last_became_peered":"2023-08-25T09:38:52.140881+0000","last_unstale":"2023-08-25T09:38:54.077130+0000","last_undegraded":"2023-08-25T09:38:54.077130+0000","last_fullsized":"2023-08-25T09:38:54.077130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:19.948794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078140+0000","last_change":"2023-08-25T09:38:53.100041+0000","last_active":"2023-08-25T09:38:54.078140+0000","last_peered":"2023-08-25T09:38:54.078140+0000","last_clean":"2023-08-25T09:38:54.078140+0000","last_became_active":"2023-08-25T09:38:53.099764+0000","last_became_peered":"2023-08-25T09:38:53.099764+0000","last_unstale":"2023-08-25T09:38:54.078140+0000","last_undegraded":"2023-08-25T09:38:54.078140+0000","last_fullsized":"2023-08-25T09:38:54.078140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:13:51.695417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170584+0000","last_change":"2023-08-25T09:38:52.150397+0000","last_active":"2023-08-25T09:38:54.170584+0000","last_peered":"2023-08-25T09:38:54.170584+0000","last_clean":"2023-08-25T09:38:54.170584+0000","last_became_active":"2023-08-25T09:38:52.150134+0000","last_became_peered":"2023-08-25T09:38:52.150134+0000","last_unstale":"2023-08-25T09:38:54.170584+0000","last_undegraded":"2023-08-25T09:38:54.170584+0000","last_fullsized":"2023-08-25T09:38:54.170584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185984+0000","last_change":"2023-08-25T09:38:53.099998+0000","last_active":"2023-08-25T09:38:54.185984+0000","last_peered":"2023-08-25T09:38:54.185984+0000","last_clean":"2023-08-25T09:38:54.185984+0000","last_became_active":"2023-08-25T09:38:53.099612+0000","last_became_peered":"2023-08-25T09:38:53.099612+0000","last_unstale":"2023-08-25T09:38:54.185984+0000","last_undegraded":"2023-08-25T09:38:54.185984+0000","last_fullsized":"2023-08-25T09:38:54.185984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170948+0000","last_change":"2023-08-25T09:38:52.147242+0000","last_active":"2023-08-25T09:38:54.170948+0000","last_peered":"2023-08-25T09:38:54.170948+0000","last_clean":"2023-08-25T09:38:54.170948+0000","last_became_active":"2023-08-25T09:38:52.146238+0000","last_became_peered":"2023-08-25T09:38:52.146238+0000","last_unstale":"2023-08-25T09:38:54.170948+0000","last_undegraded":"2023-08-25T09:38:54.170948+0000","last_fullsized":"2023-08-25T09:38:54.170948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910561,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.456}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.216}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":657,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199326264,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1779999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2929999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2829999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":80,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195231,"statfs":{"total":200000000,"available":199917214,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":89,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195223,"statfs":{"total":200000000,"available":199908845,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":82,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195230,"statfs":{"total":200000000,"available":199915602,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1759999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7989999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":664,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194647,"statfs":{"total":200000000,"available":199319519,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":80,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195231,"statfs":{"total":200000000,"available":199917068,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3169999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]}]}],"pool_statfs":[]}} 2023-08-25T09:38:58.138 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:38:58.138 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-25T09:38:58.139 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-25T09:38:58.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-25T09:38:58.139 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-25T09:38:58.249 INFO:tasks.ceph.mds.d.smithi155.stderr:2023-08-25T09:38:58.247+0000 7f9fa0eb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:38:58.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.390+0000 7f6037f12640 1 Processor -- start 2023-08-25T09:38:58.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.391+0000 7f6037f12640 1 -- start start 2023-08-25T09:38:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.392+0000 7f6037f12640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60300cffb0 0x7f60300cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.392+0000 7f6037f12640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 0x7f60300d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.392+0000 7f6037f12640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603015cfb0 con 0x7f603015cbd0 2023-08-25T09:38:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.392+0000 7f6037f12640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300cd4e0 con 0x7f60300cffb0 2023-08-25T09:38:58.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.392+0000 7f6037f12640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6030160f10 con 0x7f60300cdeb0 2023-08-25T09:38:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.393+0000 7f60354ad640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60300cffb0 0x7f60300cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.393+0000 7f60364af640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 0x7f60300d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.393+0000 7f60364af640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 0x7f60300d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46726/0 (socket says 172.21.15.70:46726) 2023-08-25T09:38:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.393+0000 7f60364af640 1 -- 172.21.15.70:0/3901235976 learned_addr learned my addr 172.21.15.70:0/3901235976 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.394+0000 7f60364af640 1 -- 172.21.15.70:0/3901235976 >> v2:172.21.15.155:3300/0 conn(0x7f60300cffb0 msgr2=0x7f60300cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.394+0000 7f60364af640 1 --2- 172.21.15.70:0/3901235976 >> v2:172.21.15.155:3300/0 conn(0x7f60300cffb0 0x7f60300cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.394+0000 7f60364af640 1 -- 172.21.15.70:0/3901235976 >> v1:172.21.15.155:6789/0 conn(0x7f603015cbd0 legacy=0x7f60301606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.394+0000 7f60364af640 1 -- 172.21.15.70:0/3901235976 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60301611f0 con 0x7f60300cdeb0 2023-08-25T09:38:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.395+0000 7f60364af640 1 --2- 172.21.15.70:0/3901235976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 0x7f60300d2a00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6028009e50 tx=0x7f602802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e3af6862949ac4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6034cac640 1 -- 172.21.15.70:0/3901235976 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602802cb00 con 0x7f60300cdeb0 2023-08-25T09:38:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6034cac640 1 -- 172.21.15.70:0/3901235976 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602802cc60 con 0x7f60300cdeb0 2023-08-25T09:38:58.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6034cac640 1 -- 172.21.15.70:0/3901235976 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280343f0 con 0x7f60300cdeb0 2023-08-25T09:38:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6037f12640 1 -- 172.21.15.70:0/3901235976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 msgr2=0x7f60300d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6037f12640 1 --2- 172.21.15.70:0/3901235976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 0x7f60300d2a00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6028009e50 tx=0x7f602802bf60 comp rx=0 tx=0).stop 2023-08-25T09:38:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6037f12640 1 -- 172.21.15.70:0/3901235976 shutdown_connections 2023-08-25T09:38:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6037f12640 1 --2- 172.21.15.70:0/3901235976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60300cdeb0 0x7f60300d2a00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6037f12640 1 --2- 172.21.15.70:0/3901235976 >> v2:172.21.15.155:3300/0 conn(0x7f60300cffb0 0x7f60300cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.396+0000 7f6037f12640 1 -- 172.21.15.70:0/3901235976 >> 172.21.15.70:0/3901235976 conn(0x7f6030070260 msgr2=0x7f6030070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.397+0000 7f6037f12640 1 -- 172.21.15.70:0/3901235976 shutdown_connections 2023-08-25T09:38:58.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.397+0000 7f6037f12640 1 -- 172.21.15.70:0/3901235976 wait complete. 2023-08-25T09:38:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.399+0000 7f6037f12640 1 Processor -- start 2023-08-25T09:38:58.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.399+0000 7f6037f12640 1 -- start start 2023-08-25T09:38:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.400+0000 7f6037f12640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60300cdeb0 0x7f6030153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.400+0000 7f6037f12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 0x7f6030150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.400+0000 7f6037f12640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603015cbd0 0x7f60301516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.400+0000 7f6037f12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6030152b90 con 0x7f60300cffb0 2023-08-25T09:38:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.400+0000 7f6037f12640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60301514a0 con 0x7f60300cdeb0 2023-08-25T09:38:58.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.400+0000 7f6037f12640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6030151f80 con 0x7f603015cbd0 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 0x7f6030150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 0x7f6030150f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56690/0 (socket says 172.21.15.70:56690) 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 -- 172.21.15.70:0/2218953744 learned_addr learned my addr 172.21.15.70:0/2218953744 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60364af640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603015cbd0 0x7f60301516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f6035cae640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60300cdeb0 0x7f6030153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 -- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603015cbd0 msgr2=0x7f60301516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603015cbd0 0x7f60301516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 -- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60300cdeb0 msgr2=0x7f6030153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60300cdeb0 0x7f6030153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f60354ad640 1 -- 172.21.15.70:0/2218953744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028009b30 con 0x7f60300cffb0 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.401+0000 7f6035cae640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60300cdeb0 0x7f6030153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.402+0000 7f60354ad640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 0x7f6030150f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9e0 tx=0x7f602000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.402+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020009670 con 0x7f60300cffb0 2023-08-25T09:38:58.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.402+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602000c9d0 con 0x7f60300cffb0 2023-08-25T09:38:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.402+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000cd80 con 0x7f60300cffb0 2023-08-25T09:38:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.402+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6030152260 con 0x7f60300cffb0 2023-08-25T09:38:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.402+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6030152820 con 0x7f60300cffb0 2023-08-25T09:38:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.403+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6030152d40 con 0x7f60300cffb0 2023-08-25T09:38:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.404+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6020013470 con 0x7f60300cffb0 2023-08-25T09:38:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.405+0000 7f60267fc640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601406a050 0x7f601406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.405+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6020088840 con 0x7f60300cffb0 2023-08-25T09:38:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.405+0000 7f6035cae640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601406a050 0x7f601406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.406+0000 7f6035cae640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601406a050 0x7f601406c510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f602c000f60 tx=0x7f602c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:58.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.410+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6020053f20 con 0x7f60300cffb0 2023-08-25T09:38:58.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.648+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f603004e420 con 0x7f60300cffb0 2023-08-25T09:38:58.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.656+0000 7f60267fc640 1 -- 172.21.15.70:0/2218953744 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169492 (secure 0 0 0) 0x7f6020057540 con 0x7f60300cffb0 2023-08-25T09:38:58.662 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:58.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601406a050 msgr2=0x7f601406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601406a050 0x7f601406c510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f602c000f60 tx=0x7f602c009040 comp rx=0 tx=0).stop 2023-08-25T09:38:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 msgr2=0x7f6030150f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 0x7f6030150f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f602000e9e0 tx=0x7f602000eea0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 shutdown_connections 2023-08-25T09:38:58.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601406a050 0x7f601406c510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603015cbd0 0x7f60301516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60300cffb0 0x7f6030150f60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 --2- 172.21.15.70:0/2218953744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60300cdeb0 0x7f6030153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.660+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 >> 172.21.15.70:0/2218953744 conn(0x7f6030070260 msgr2=0x7f60300b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.661+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 shutdown_connections 2023-08-25T09:38:58.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.661+0000 7f6037f12640 1 -- 172.21.15.70:0/2218953744 wait complete. 2023-08-25T09:38:58.681 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":5,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2630723694},{"type":"v1","addr":"172.21.15.70:6833","nonce":2630723694}]},"active_addr":"172.21.15.70:6833/2630723694","active_change":"2023-08-25T09:38:37.295695+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4113,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3447656693}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1323776345}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":2529502140}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1573430604}]}]} 2023-08-25T09:38:58.685 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-25T09:38:58.686 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-25T09:38:58.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:58.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.891+0000 7f53f359e640 1 Processor -- start 2023-08-25T09:38:58.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.891+0000 7f53f359e640 1 -- start start 2023-08-25T09:38:58.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53f40cad20 0x7f53f40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f415e7c0 con 0x7f53f40b12f0 2023-08-25T09:38:58.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f359e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f4071b50 con 0x7f53f415a7c0 2023-08-25T09:38:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f40cb6f0 con 0x7f53f40cad20 2023-08-25T09:38:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53ebfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f53f40b12f0 0x7f53f415ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38002/0 (socket says 172.21.15.70:38002) 2023-08-25T09:38:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53ebfff640 1 -- 172.21.15.70:0/1382094639 learned_addr learned my addr 172.21.15.70:0/1382094639 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:58.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f2d9d640 1 --2- 172.21.15.70:0/1382094639 >> v2:172.21.15.70:3300/0 conn(0x7f53f40cad20 0x7f53f40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f1d9b640 1 -- 172.21.15.70:0/1382094639 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181110921 0 0) 0x7f53f415e7c0 con 0x7f53f40b12f0 2023-08-25T09:38:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f1d9b640 1 -- 172.21.15.70:0/1382094639 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e0003520 con 0x7f53f40b12f0 2023-08-25T09:38:58.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.893+0000 7f53f2d9d640 1 -- 172.21.15.70:0/1382094639 >> v1:172.21.15.155:6789/0 conn(0x7f53f415a7c0 legacy=0x7f53f40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:58.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.894+0000 7f53f2d9d640 1 -- 172.21.15.70:0/1382094639 >> v1:172.21.15.70:6789/0 conn(0x7f53f40b12f0 legacy=0x7f53f415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.894+0000 7f53f2d9d640 1 -- 172.21.15.70:0/1382094639 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f415e490 con 0x7f53f40cad20 2023-08-25T09:38:58.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.895+0000 7f53f2d9d640 1 --2- 172.21.15.70:0/1382094639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f40cb1b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009d90 tx=0x7f53ec02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b19e1e4941b8a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:58.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.895+0000 7f53f1d9b640 1 -- 172.21.15.70:0/1382094639 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec02ccf0 con 0x7f53f40cad20 2023-08-25T09:38:58.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.895+0000 7f53f1d9b640 1 -- 172.21.15.70:0/1382094639 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec02ce50 con 0x7f53f40cad20 2023-08-25T09:38:58.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.895+0000 7f53f1d9b640 1 -- 172.21.15.70:0/1382094639 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec034660 con 0x7f53f40cad20 2023-08-25T09:38:58.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.895+0000 7f53f359e640 1 -- 172.21.15.70:0/1382094639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 msgr2=0x7f53f40cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.895+0000 7f53f359e640 1 --2- 172.21.15.70:0/1382094639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f40cb1b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009d90 tx=0x7f53ec02bf40 comp rx=0 tx=0).stop 2023-08-25T09:38:58.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.896+0000 7f53f359e640 1 -- 172.21.15.70:0/1382094639 shutdown_connections 2023-08-25T09:38:58.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.896+0000 7f53f359e640 1 --2- 172.21.15.70:0/1382094639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f40cb1b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.896+0000 7f53f359e640 1 -- 172.21.15.70:0/1382094639 >> 172.21.15.70:0/1382094639 conn(0x7f53f40affd0 msgr2=0x7f53f40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:58.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.896+0000 7f53f359e640 1 -- 172.21.15.70:0/1382094639 shutdown_connections 2023-08-25T09:38:58.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.896+0000 7f53f359e640 1 -- 172.21.15.70:0/1382094639 wait complete. 2023-08-25T09:38:58.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.897+0000 7f53f359e640 1 Processor -- start 2023-08-25T09:38:58.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.897+0000 7f53f359e640 1 -- start start 2023-08-25T09:38:58.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.898+0000 7f53f359e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f40b12f0 0x7f53f414a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.898+0000 7f53f359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f41f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.898+0000 7f53f359e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f415a7c0 0x7f53f41f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.898+0000 7f53f359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41f60f0 con 0x7f53f40cad20 2023-08-25T09:38:58.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.898+0000 7f53f359e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41f63d0 con 0x7f53f40b12f0 2023-08-25T09:38:58.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.898+0000 7f53f359e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41f66b0 con 0x7f53f415a7c0 2023-08-25T09:38:58.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f41f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f41f57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56708/0 (socket says 172.21.15.70:56708) 2023-08-25T09:38:58.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 -- 172.21.15.70:0/3734059964 learned_addr learned my addr 172.21.15.70:0/3734059964 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:58.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 -- 172.21.15.70:0/3734059964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f415a7c0 msgr2=0x7f53f41f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f415a7c0 0x7f53f41f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 -- 172.21.15.70:0/3734059964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f40b12f0 msgr2=0x7f53f414a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:58.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f40b12f0 0x7f53f414a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:58.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 -- 172.21.15.70:0/3734059964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec009a70 con 0x7f53f40cad20 2023-08-25T09:38:58.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.899+0000 7f53ebfff640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f41f57f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c410 tx=0x7f53dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:58.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.900+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc005ce0 con 0x7f53f40cad20 2023-08-25T09:38:58.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.900+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc005e40 con 0x7f53f40cad20 2023-08-25T09:38:58.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.900+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0072f0 con 0x7f53f40cad20 2023-08-25T09:38:58.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.900+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f41f2310 con 0x7f53f40cad20 2023-08-25T09:38:58.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.900+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f41f5d30 con 0x7f53f40cad20 2023-08-25T09:38:58.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.901+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f41f5ec0 con 0x7f53f40cad20 2023-08-25T09:38:58.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.902+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f53dc007510 con 0x7f53f40cad20 2023-08-25T09:38:58.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.903+0000 7f53eaffd640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f53d406a050 0x7f53d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:58.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.903+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53dc087ce0 con 0x7f53f40cad20 2023-08-25T09:38:58.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.903+0000 7f53f259c640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f53d406a050 0x7f53d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:58.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.904+0000 7f53f259c640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f53d406a050 0x7f53d406c510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000f60 tx=0x7f53e40079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:58.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:58.908+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f53dc0533f0 con 0x7f53f40cad20 2023-08-25T09:38:59.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.098+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53f41f2830 con 0x7f53f40cad20 2023-08-25T09:38:59.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.100+0000 7f53eaffd640 1 -- 172.21.15.70:0/3734059964 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f53f41f2830 con 0x7f53f40cad20 2023-08-25T09:38:59.103 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:59.103 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:59.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f53d406a050 msgr2=0x7f53d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f53d406a050 0x7f53d406c510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000f60 tx=0x7f53e40079b0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 msgr2=0x7f53f41f57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f41f57f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c410 tx=0x7f53dc00c8d0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 shutdown_connections 2023-08-25T09:38:59.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f53d406a050 0x7f53d406c510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.102+0000 7f53f359e640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53f415a7c0 0x7f53f41f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.103+0000 7f53f359e640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53f40cad20 0x7f53f41f57f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.103+0000 7f53f359e640 1 --2- 172.21.15.70:0/3734059964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53f40b12f0 0x7f53f414a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.103+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 >> 172.21.15.70:0/3734059964 conn(0x7f53f40affd0 msgr2=0x7f53f4159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.103+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 shutdown_connections 2023-08-25T09:38:59.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.103+0000 7f53f359e640 1 -- 172.21.15.70:0/3734059964 wait complete. 2023-08-25T09:38:59.124 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-25T09:38:59.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:38:59.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.335+0000 7fbc6221f640 1 Processor -- start 2023-08-25T09:38:59.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.336+0000 7fbc6221f640 1 -- start start 2023-08-25T09:38:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc6221f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbc5c15cc00 0x7fbc5c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc6221f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c15cfe0 con 0x7fbc5c0cf580 2023-08-25T09:38:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc6221f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c0cfdb0 con 0x7fbc5c06e0c0 2023-08-25T09:38:59.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc6221f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c069850 con 0x7fbc5c15cc00 2023-08-25T09:38:59.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc5affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbc5c0cf580 0x7fbc5c06d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35410/0 (socket says 172.21.15.70:35410) 2023-08-25T09:38:59.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc5affd640 1 -- 172.21.15.70:0/2485426000 learned_addr learned my addr 172.21.15.70:0/2485426000 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:59.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.337+0000 7fbc5b7fe640 1 --2- 172.21.15.70:0/2485426000 >> v2:172.21.15.155:3300/0 conn(0x7fbc5c15cc00 0x7fbc5c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671700032 0 0) 0x7fbc5c15cfe0 con 0x7fbc5c0cf580 2023-08-25T09:38:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc48003520 con 0x7fbc5c0cf580 2023-08-25T09:38:59.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041315367 0 0) 0x7fbc48003520 con 0x7fbc5c0cf580 2023-08-25T09:38:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 >> v2:172.21.15.155:3300/0 conn(0x7fbc5c15cc00 msgr2=0x7fbc5c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 --2- 172.21.15.70:0/2485426000 >> v2:172.21.15.155:3300/0 conn(0x7fbc5c15cc00 0x7fbc5c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 >> v1:172.21.15.155:6789/0 conn(0x7fbc5c06e0c0 legacy=0x7fbc5c166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c069b30 con 0x7fbc5c0cf580 2023-08-25T09:38:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fbc44001670 con 0x7fbc5c0cf580 2023-08-25T09:38:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 >> v1:172.21.15.70:6789/0 conn(0x7fbc5c0cf580 legacy=0x7fbc5c06d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48003970 0x7fbc48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc480047c0 0x7fbc48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc48005180 0x7fbc4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c069850 con 0x7fbc480047c0 2023-08-25T09:38:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4800c020 con 0x7fbc48003970 2023-08-25T09:38:59.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4800c1e0 con 0x7fbc48005180 2023-08-25T09:38:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.338+0000 7fbc5affd640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc480047c0 0x7fbc48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.339+0000 7fbc5affd640 1 -- 172.21.15.70:0/2485426000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc48005180 msgr2=0x7fbc4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.339+0000 7fbc5affd640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc48005180 0x7fbc4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.339+0000 7fbc5affd640 1 -- 172.21.15.70:0/2485426000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48003970 msgr2=0x7fbc48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.339+0000 7fbc5affd640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc48003970 0x7fbc48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.339+0000 7fbc5affd640 1 -- 172.21.15.70:0/2485426000 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48003520 con 0x7fbc480047c0 2023-08-25T09:38:59.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.339+0000 7fbc5bfff640 1 -- 172.21.15.70:0/2485426000 reap_dead start 2023-08-25T09:38:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.340+0000 7fbc5affd640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc480047c0 0x7fbc48004c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c240 tx=0x7fbc480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ef998b9ed78a95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.340+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44005ad0 con 0x7fbc480047c0 2023-08-25T09:38:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.340+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc44005c30 con 0x7fbc480047c0 2023-08-25T09:38:59.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.340+0000 7fbc5a7fc640 1 -- 172.21.15.70:0/2485426000 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4402caf0 con 0x7fbc480047c0 2023-08-25T09:38:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 -- 172.21.15.70:0/2485426000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc480047c0 msgr2=0x7fbc48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc480047c0 0x7fbc48004c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c240 tx=0x7fbc480057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 -- 172.21.15.70:0/2485426000 shutdown_connections 2023-08-25T09:38:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2485426000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc480047c0 0x7fbc48004c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 -- 172.21.15.70:0/2485426000 >> 172.21.15.70:0/2485426000 conn(0x7fbc5c0d0860 msgr2=0x7fbc5c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 -- 172.21.15.70:0/2485426000 shutdown_connections 2023-08-25T09:38:59.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.341+0000 7fbc6221f640 1 -- 172.21.15.70:0/2485426000 wait complete. 2023-08-25T09:38:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.343+0000 7fbc6221f640 1 Processor -- start 2023-08-25T09:38:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.343+0000 7fbc6221f640 1 -- start start 2023-08-25T09:38:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc6221f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc480047c0 0x7fbc5c14fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc6221f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc5c0cc270 0x7fbc5c0cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc6221f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 0x7fbc5c0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc6221f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0ca4c0 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc6221f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0cccf0 con 0x7fbc480047c0 2023-08-25T09:38:59.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc6221f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0ccfd0 con 0x7fbc5c0cc270 2023-08-25T09:38:59.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 0x7fbc5c0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 0x7fbc5c0c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47082/0 (socket says 172.21.15.70:47082) 2023-08-25T09:38:59.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 -- 172.21.15.70:0/2074167345 learned_addr learned my addr 172.21.15.70:0/2074167345 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5b7fe640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc480047c0 0x7fbc5c14fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5affd640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc5c0cc270 0x7fbc5c0cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 -- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc5c0cc270 msgr2=0x7fbc5c0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc5c0cc270 0x7fbc5c0cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 -- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc480047c0 msgr2=0x7fbc5c14fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.344+0000 7fbc5bfff640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc480047c0 0x7fbc5c14fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc5bfff640 1 -- 172.21.15.70:0/2074167345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c0cd2b0 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc5affd640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc5c0cc270 0x7fbc5c0cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:59.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc5bfff640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 0x7fbc5c0c9f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0cae40 tx=0x7fbc5400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54004220 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc540043a0 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc54007310 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c0cd590 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.345+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c1501a0 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.346+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c065cb0 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.347+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbc54007470 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.349+0000 7fbc53fff640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc3c06a050 0x7fbc3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.349+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc54087500 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.349+0000 7fbc5b7fe640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc3c06a050 0x7fbc3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.349+0000 7fbc5b7fe640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc3c06a050 0x7fbc3c06c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007520 tx=0x7fbc4c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.353+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbc54052c80 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.529+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc5c04e420 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.532+0000 7fbc53fff640 1 -- 172.21.15.70:0/2074167345 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fbc540562a0 con 0x7fbc5c0c9b00 2023-08-25T09:38:59.535 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:38:59.535 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:38:59.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.534+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc3c06a050 msgr2=0x7fbc3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.534+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc3c06a050 0x7fbc3c06c510 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007520 tx=0x7fbc4c00b040 comp rx=0 tx=0).stop 2023-08-25T09:38:59.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 msgr2=0x7fbc5c0c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 0x7fbc5c0c9f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c0cae40 tx=0x7fbc5400c820 comp rx=0 tx=0).stop 2023-08-25T09:38:59.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 shutdown_connections 2023-08-25T09:38:59.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc3c06a050 0x7fbc3c06c510 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc5c0c9b00 0x7fbc5c0c9f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc5c0cc270 0x7fbc5c0cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 --2- 172.21.15.70:0/2074167345 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc480047c0 0x7fbc5c14fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 >> 172.21.15.70:0/2074167345 conn(0x7fbc5c0d0860 msgr2=0x7fbc5c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 shutdown_connections 2023-08-25T09:38:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.535+0000 7fbc6221f640 1 -- 172.21.15.70:0/2074167345 wait complete. 2023-08-25T09:38:59.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-25T09:38:59.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-25T09:38:59.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-25T09:38:59.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-25T09:38:59.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-08-25T09:38:59.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-08-25T09:38:59.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-08-25T09:38:59.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-08-25T09:38:59.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.854+0000 7f1e791cf640 1 Processor -- start 2023-08-25T09:38:59.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.854+0000 7f1e791cf640 1 -- start start 2023-08-25T09:38:59.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.856+0000 7f1e791cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740aff00 0x7f1e740b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.856+0000 7f1e791cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7406f1a0 con 0x7f1e740ca470 2023-08-25T09:38:59.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.856+0000 7f1e791cf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740cc260 con 0x7f1e7415c8b0 2023-08-25T09:38:59.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.856+0000 7f1e791cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740b08d0 con 0x7f1e740aff00 2023-08-25T09:38:59.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.857+0000 7f1e7359e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1e740ca470 0x7f1e740cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35418/0 (socket says 172.21.15.70:35418) 2023-08-25T09:38:59.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.857+0000 7f1e7359e640 1 -- 172.21.15.70:0/1167449454 learned_addr learned my addr 172.21.15.70:0/1167449454 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:59.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.857+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202036767 0 0) 0x7f1e7406f1a0 con 0x7f1e740ca470 2023-08-25T09:38:59.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.857+0000 7f1e7259c640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740aff00 0x7f1e740b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.857+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e5c003520 con 0x7f1e740ca470 2023-08-25T09:38:59.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833542409 0 0) 0x7f1e5c003520 con 0x7f1e740ca470 2023-08-25T09:38:59.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740aff00 msgr2=0x7f1e740b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740aff00 0x7f1e740b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 >> v1:172.21.15.155:6789/0 conn(0x7f1e7415c8b0 legacy=0x7f1e7406ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740cc540 con 0x7f1e740ca470 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1e6c0019f0 con 0x7f1e740ca470 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 >> v1:172.21.15.70:6789/0 conn(0x7f1e740ca470 legacy=0x7f1e740cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e5c0039a0 0x7f1e5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740b08d0 con 0x7f1e5c0039a0 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c00c050 con 0x7f1e5c005180 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.858+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c00c210 con 0x7f1e5c0047c0 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.859+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e5c0039a0 0x7f1e5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.859+0000 7f1e7359e640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.859+0000 7f1e7259c640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e72d9d640 1 -- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e5c0047c0 msgr2=0x7f1e5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e72d9d640 1 -- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5c005180 msgr2=0x7f1e5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e72d9d640 1 -- 172.21.15.70:0/1167449454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e5c003520 con 0x7f1e5c0039a0 2023-08-25T09:38:59.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e7359e640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e5c005180 0x7f1e5c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:59.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.860+0000 7f1e7259c640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e5c0047c0 0x7f1e5c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:38:59.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.862+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e5c0039a0 0x7f1e5c003dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e68001750 tx=0x7f1e5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b34aa9a309509a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.862+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680052d0 con 0x7f1e5c0039a0 2023-08-25T09:38:59.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.863+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6802e050 con 0x7f1e5c0039a0 2023-08-25T09:38:59.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.863+0000 7f1e71d9b640 1 -- 172.21.15.70:0/1167449454 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680052d0 con 0x7f1e5c0039a0 2023-08-25T09:38:59.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.863+0000 7f2e404a7640 1 Processor -- start 2023-08-25T09:38:59.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.863+0000 7f2e404a7640 1 -- start start 2023-08-25T09:38:59.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e791cf640 1 -- 172.21.15.70:0/1167449454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e5c0039a0 msgr2=0x7f1e5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e791cf640 1 --2- 172.21.15.70:0/1167449454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e5c0039a0 0x7f1e5c003dc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1e68001750 tx=0x7f1e5c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e7359e640 1 -- 172.21.15.70:0/1167449454 reap_dead start 2023-08-25T09:38:59.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e791cf640 1 -- 172.21.15.70:0/1167449454 shutdown_connections 2023-08-25T09:38:59.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e791cf640 1 -- 172.21.15.70:0/1167449454 >> 172.21.15.70:0/1167449454 conn(0x7f1e74069c40 msgr2=0x7f1e7406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e791cf640 1 -- 172.21.15.70:0/1167449454 shutdown_connections 2023-08-25T09:38:59.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.865+0000 7f2e404a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380ca770 0x7f2e3815adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.865+0000 7f2e404a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e38159c30 0x7f2e380aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.866+0000 7f2e404a7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e380b2940 con 0x7f2e380b0440 2023-08-25T09:38:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.866+0000 7f2e404a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e380b08d0 con 0x7f2e38159c30 2023-08-25T09:38:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.866+0000 7f2e404a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380cab50 con 0x7f2e380ca770 2023-08-25T09:38:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.866+0000 7f2e3da42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e38159c30 0x7f2e380aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.866+0000 7f2e3da42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e38159c30 0x7f2e380aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38382/0 (socket says 172.21.15.70:38382) 2023-08-25T09:38:59.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.866+0000 7f2e3da42640 1 -- 172.21.15.70:0/2952856163 learned_addr learned my addr 172.21.15.70:0/2952856163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.864+0000 7f1e791cf640 1 -- 172.21.15.70:0/1167449454 wait complete. 2023-08-25T09:38:59.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7fb68ab0f640 1 Processor -- start 2023-08-25T09:38:59.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7fb68ab0f640 1 -- start start 2023-08-25T09:38:59.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f2e3e243640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380ca770 0x7f2e3815adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f2e3da42640 1 -- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380ca770 msgr2=0x7f2e3815adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f2e3da42640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380ca770 0x7f2e3815adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f2e3da42640 1 -- 172.21.15.70:0/2952856163 >> v1:172.21.15.70:6789/0 conn(0x7f2e380b0440 legacy=0x7f2e380b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f2e3da42640 1 -- 172.21.15.70:0/2952856163 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38160240 con 0x7f2e38159c30 2023-08-25T09:38:59.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f2e3e243640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380ca770 0x7f2e3815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:38:59.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.867+0000 7f1e791cf640 1 Processor -- start 2023-08-25T09:38:59.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.868+0000 7f1e791cf640 1 -- start start 2023-08-25T09:38:59.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7fb68ab0f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb67c007600 0x7fb67c005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f2e3da42640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e38159c30 0x7f2e380aff00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=420e2f34da42cd07 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7fb68ab0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c007170 con 0x7fb67c050f00 2023-08-25T09:38:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7fb68ab0f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c006560 con 0x7fb67c0e11c0 2023-08-25T09:38:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7fb68ab0f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c006840 con 0x7fb67c007600 2023-08-25T09:38:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f1e791cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e74201950 0x7f1e7415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f1e791cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e741ff6b0 0x7f1e740c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f1e791cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740c4580 0x7f1e741fb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f1e791cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c4a30 con 0x7f1e74201950 2023-08-25T09:38:59.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f1e791cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74202090 con 0x7f1e740c4580 2023-08-25T09:38:59.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.869+0000 7f1e791cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74202330 con 0x7f1e741ff6b0 2023-08-25T09:38:59.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f2e3d241640 1 -- 172.21.15.70:0/2952856163 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34030840 con 0x7f2e38159c30 2023-08-25T09:38:59.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7fb6890ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb67c050f00 0x7fb67c04bbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35434/0 (socket says 172.21.15.70:35434) 2023-08-25T09:38:59.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7fb6890ac640 1 -- 172.21.15.70:0/3229055246 learned_addr learned my addr 172.21.15.70:0/3229055246 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:59.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f2e3d241640 1 -- 172.21.15.70:0/2952856163 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e340309a0 con 0x7f2e38159c30 2023-08-25T09:38:59.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7fb683fff640 1 --2- 172.21.15.70:0/3229055246 >> v2:172.21.15.155:3300/0 conn(0x7fb67c007600 0x7fb67c005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f1e7259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e741ff6b0 0x7f1e740c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f1e7259c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e741ff6b0 0x7f1e740c4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44672/0 (socket says 172.21.15.70:44672) 2023-08-25T09:38:59.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f1e7259c640 1 -- 172.21.15.70:0/553215651 learned_addr learned my addr 172.21.15.70:0/553215651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f1e7359e640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740c4580 0x7f1e741fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e74201950 0x7f1e7415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7fb6837fe640 1 -- 172.21.15.70:0/3229055246 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842390223 0 0) 0x7fb67c007170 con 0x7fb67c050f00 2023-08-25T09:38:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.870+0000 7fb6837fe640 1 -- 172.21.15.70:0/3229055246 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb674003520 con 0x7fb67c050f00 2023-08-25T09:38:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7fb6837fe640 1 -- 172.21.15.70:0/3229055246 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3121206575 0 0) 0x7fb674003520 con 0x7fb67c050f00 2023-08-25T09:38:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7fb683fff640 1 -- 172.21.15.70:0/3229055246 >> v1:172.21.15.155:6789/0 conn(0x7fb67c0e11c0 legacy=0x7fb67c039e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T09:38:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7fb683fff640 1 -- 172.21.15.70:0/3229055246 >> v1:172.21.15.70:6789/0 conn(0x7fb67c050f00 legacy=0x7fb67c04bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7fb683fff640 1 -- 172.21.15.70:0/3229055246 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c006b20 con 0x7fb67c007600 2023-08-25T09:38:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7f2e404a7640 1 -- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e38159c30 msgr2=0x7f2e380aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7f2e404a7640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e38159c30 0x7f2e380aff00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b980 comp rx=0 tx=0).stop 2023-08-25T09:38:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7f2e404a7640 1 -- 172.21.15.70:0/2952856163 shutdown_connections 2023-08-25T09:38:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7f2e404a7640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e38159c30 0x7f2e380aff00 secure :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009870 tx=0x7f2e3402b980 comp rx=0 tx=0).stop 2023-08-25T09:38:59.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7f2e404a7640 1 --2- 172.21.15.70:0/2952856163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380ca770 0x7f2e3815adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.871+0000 7f2e404a7640 1 -- 172.21.15.70:0/2952856163 >> 172.21.15.70:0/2952856163 conn(0x7f2e380caf00 msgr2=0x7f2e380cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f1e72d9d640 1 -- 172.21.15.70:0/553215651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e741ff6b0 msgr2=0x7f1e740c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e741ff6b0 0x7f1e740c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f1e72d9d640 1 -- 172.21.15.70:0/553215651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740c4580 msgr2=0x7f1e741fb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e740c4580 0x7f1e741fb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f1e72d9d640 1 -- 172.21.15.70:0/553215651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74202610 con 0x7f1e74201950 2023-08-25T09:38:59.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f1e72d9d640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e74201950 0x7f1e7415eb60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e68001720 tx=0x7f1e68001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f2e404a7640 1 -- 172.21.15.70:0/2952856163 shutdown_connections 2023-08-25T09:38:59.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.872+0000 7f2e404a7640 1 -- 172.21.15.70:0/2952856163 wait complete. 2023-08-25T09:38:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.873+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680052d0 con 0x7f1e74201950 2023-08-25T09:38:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.873+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6803d640 con 0x7f1e74201950 2023-08-25T09:38:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.873+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6803d7f0 con 0x7f1e74201950 2023-08-25T09:38:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.873+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e742028f0 con 0x7f1e74201950 2023-08-25T09:38:59.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.873+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e74201d30 con 0x7f1e74201950 2023-08-25T09:38:59.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.873+0000 7fb683fff640 1 --2- 172.21.15.70:0/3229055246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c007600 0x7fb67c005ec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6700097a0 tx=0x7fb67002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aabfa4af6cae2591 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.874+0000 7fb6837fe640 1 -- 172.21.15.70:0/3229055246 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68404e420 con 0x7fb67c007600 2023-08-25T09:38:59.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.874+0000 7fb6837fe640 1 -- 172.21.15.70:0/3229055246 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67002e050 con 0x7fb67c007600 2023-08-25T09:38:59.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.874+0000 7fb6837fe640 1 -- 172.21.15.70:0/3229055246 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb670030730 con 0x7fb67c007600 2023-08-25T09:38:59.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 -- 172.21.15.70:0/3229055246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c007600 msgr2=0x7fb67c005ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 --2- 172.21.15.70:0/3229055246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c007600 0x7fb67c005ec0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb6700097a0 tx=0x7fb67002b790 comp rx=0 tx=0).stop 2023-08-25T09:38:59.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e742024a0 con 0x7f1e74201950 2023-08-25T09:38:59.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 -- 172.21.15.70:0/3229055246 shutdown_connections 2023-08-25T09:38:59.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 --2- 172.21.15.70:0/3229055246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c007600 0x7fb67c005ec0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 -- 172.21.15.70:0/3229055246 >> 172.21.15.70:0/3229055246 conn(0x7fb67c009fe0 msgr2=0x7fb67c00a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7f2e404a7640 1 Processor -- start 2023-08-25T09:38:59.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 -- 172.21.15.70:0/3229055246 shutdown_connections 2023-08-25T09:38:59.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fa8ee1a4640 1 Processor -- start 2023-08-25T09:38:59.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fb68ab0f640 1 -- 172.21.15.70:0/3229055246 wait complete. 2023-08-25T09:38:59.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7f2e404a7640 1 -- start start 2023-08-25T09:38:59.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.876+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e6803d950 con 0x7f1e74201950 2023-08-25T09:38:59.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.877+0000 7f2e404a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e380b0440 0x7f2e38156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.875+0000 7fa8ee1a4640 1 -- start start 2023-08-25T09:38:59.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.877+0000 7f2e404a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e380ca770 0x7f2e380c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.877+0000 7fa8ee1a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e00e78b0 0x7fa8e004f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.877+0000 7fa8ee1a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8e00eb2c0 0x7fa8e00eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.877+0000 7fa8ee1a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8e0052fe0 0x7fa8e0053470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ee1a4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e0050340 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ee1a4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e0006480 con 0x7fa8e0052fe0 2023-08-25T09:38:59.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ee1a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00ebc40 con 0x7fa8e00e78b0 2023-08-25T09:38:59.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fb68ab0f640 1 Processor -- start 2023-08-25T09:38:59.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e404a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380afaa0 0x7f2e380aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fb68ab0f640 1 -- start start 2023-08-25T09:38:59.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e404a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38147b60 con 0x7f2e380b0440 2023-08-25T09:38:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e404a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38155120 con 0x7f2e380ca770 2023-08-25T09:38:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e404a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38155400 con 0x7f2e380afaa0 2023-08-25T09:38:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e3e243640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e380b0440 0x7f2e38156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e3e243640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e380b0440 0x7f2e38156da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47104/0 (socket says 172.21.15.70:47104) 2023-08-25T09:38:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e3e243640 1 -- 172.21.15.70:0/370201679 learned_addr learned my addr 172.21.15.70:0/370201679 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e3da42640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e380ca770 0x7f2e380c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7f2e3ea44640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380afaa0 0x7f2e380aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e3e243640 1 -- 172.21.15.70:0/370201679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380afaa0 msgr2=0x7f2e380aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e3e243640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e380afaa0 0x7f2e380aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e3e243640 1 -- 172.21.15.70:0/370201679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e380ca770 msgr2=0x7f2e380c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e3e243640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e380ca770 0x7f2e380c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e3e243640 1 -- 172.21.15.70:0/370201679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e340094d0 con 0x7f2e380b0440 2023-08-25T09:38:59.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ec9a1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8e00eb2c0 0x7fa8e00eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ed1a2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e00e78b0 0x7fa8e004f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ec9a1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8e00eb2c0 0x7fa8e00eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47108/0 (socket says 172.21.15.70:47108) 2023-08-25T09:38:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.878+0000 7fa8ec9a1640 1 -- 172.21.15.70:0/3571981970 learned_addr learned my addr 172.21.15.70:0/3571981970 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/3571981970 >> v2:172.21.15.155:3300/0 conn(0x7fa8e0052fe0 0x7fa8e0053470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e3e243640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e380b0440 0x7f2e38156da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c7f0 tx=0x7f2e2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fb68ab0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb67c007600 0x7fb67c04a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7fb68ab0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb67c050f00 0x7fb67c04e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28007820 con 0x7f2e380b0440 2023-08-25T09:38:59.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e28007980 con 0x7f2e380b0440 2023-08-25T09:38:59.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28007d30 con 0x7f2e380b0440 2023-08-25T09:38:59.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e381556e0 con 0x7f2e380b0440 2023-08-25T09:38:59.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7fb68ab0f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c0e11c0 0x7fb67c04e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e380c4bd0 con 0x7f2e380b0440 2023-08-25T09:38:59.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7fb68ab0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c04bbc0 con 0x7fb67c050f00 2023-08-25T09:38:59.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7fb68ab0f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c04be60 con 0x7fb67c0e11c0 2023-08-25T09:38:59.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7fb68ab0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c04c140 con 0x7fb67c007600 2023-08-25T09:38:59.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7fb6890ac640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c0e11c0 0x7fb67c04e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f1e5b7fe640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a150 0x7f1e6006c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e680a9a00 con 0x7f1e74201950 2023-08-25T09:38:59.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f1e5b7fe640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7f1e60072990 0x7f1e60074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.880+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 --> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e60075460 con 0x7f1e60072990 2023-08-25T09:38:59.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e00e78b0 msgr2=0x7fa8e004f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e00e78b0 0x7fa8e004f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/3571981970 >> v2:172.21.15.70:3300/0 conn(0x7fa8e00eb2c0 msgr2=0x7fa8e00eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/3571981970 >> v2:172.21.15.70:3300/0 conn(0x7fa8e00eb2c0 0x7fa8e00eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.879+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/3571981970 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e004fd80 con 0x7fa8e0052fe0 2023-08-25T09:38:59.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.881+0000 7f64fdb09640 1 Processor -- start 2023-08-25T09:38:59.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.881+0000 7f1e7259c640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a150 0x7f1e6006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.881+0000 7f1e7359e640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7f1e60072990 0x7f1e60074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.882+0000 7fb6888ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb67c007600 0x7fb67c04a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.882+0000 7fb6888ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb67c007600 0x7fb67c04a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44706/0 (socket says 172.21.15.70:44706) 2023-08-25T09:38:59.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.882+0000 7fb6888ab640 1 -- 172.21.15.70:0/2732836833 learned_addr learned my addr 172.21.15.70:0/2732836833 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.881+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2e38155570 con 0x7f2e380b0440 2023-08-25T09:38:59.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7f1e7359e640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7f1e60072990 0x7f1e60074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7f1e7259c640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a150 0x7f1e6006c610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e740c4d70 tx=0x7f1e64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fb6888ab640 1 -- 172.21.15.70:0/2732836833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c0e11c0 msgr2=0x7fb67c04e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fb6888ab640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb67c0e11c0 0x7fb67c04e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fb6888ab640 1 -- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb67c050f00 msgr2=0x7fb67c04e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fb6888ab640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb67c050f00 0x7fb67c04e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fb6888ab640 1 -- 172.21.15.70:0/2732836833 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb670009400 con 0x7fb67c007600 2023-08-25T09:38:59.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.881+0000 7f64fdb09640 1 -- start start 2023-08-25T09:38:59.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fb6888ab640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb67c007600 0x7fb67c04a810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb67800b3e0 tx=0x7fb67800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e0052fe0 0x7fa8e0053470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0037af0 tx=0x7fa8d802b7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=630e03ebce10526c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e68075180 con 0x7f1e74201950 2023-08-25T09:38:59.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== osd.0 v2:172.21.15.70:6808/3728228047 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f1e60075460 con 0x7f1e60072990 2023-08-25T09:38:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678011830 con 0x7fb67c007600 2023-08-25T09:38:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6780119b0 con 0x7fb67c007600 2023-08-25T09:38:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678011d60 con 0x7fb67c007600 2023-08-25T09:38:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c04c420 con 0x7fb67c007600 2023-08-25T09:38:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c04ad50 con 0x7fb67c007600 2023-08-25T09:38:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.885+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e280113f0 con 0x7f2e380b0440 2023-08-25T09:38:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.883+0000 7f64fdb09640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f64f00eb2c0 0x7f64f00eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7f64fdb09640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f0050340 con 0x7f64f0006910 2023-08-25T09:38:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7f64fdb09640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64f0006480 con 0x7f64f00e78b0 2023-08-25T09:38:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.884+0000 7f64fdb09640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f00ebc40 con 0x7f64f00eb2c0 2023-08-25T09:38:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.885+0000 7f64fcb07640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f64f00e78b0 0x7f64f004f260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:45824/0 (socket says 172.21.15.70:45824) 2023-08-25T09:38:59.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.885+0000 7f64fcb07640 1 -- 172.21.15.70:0/182687748 learned_addr learned my addr 172.21.15.70:0/182687748 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:38:59.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.885+0000 7fa8dffff640 1 -- 172.21.15.70:0/3571981970 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8031750 con 0x7fa8e0052fe0 2023-08-25T09:38:59.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.885+0000 7fa8dffff640 1 -- 172.21.15.70:0/3571981970 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d80318b0 con 0x7fa8e0052fe0 2023-08-25T09:38:59.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.885+0000 7fa8dffff640 1 -- 172.21.15.70:0/3571981970 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8031c80 con 0x7fa8e0052fe0 2023-08-25T09:38:59.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.886+0000 7f64f7fff640 1 --2- 172.21.15.70:0/182687748 >> v2:172.21.15.70:3300/0 conn(0x7f64f00eb2c0 0x7f64f00eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.886+0000 7f2e2effd640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e1c06a150 0x7f2e1c06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.886+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e0052fe0 msgr2=0x7fa8e0053470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.886+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e0052fe0 0x7fa8e0053470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0037af0 tx=0x7fa8d802b7f0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e280885c0 con 0x7f2e380b0440 2023-08-25T09:38:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/3571981970 shutdown_connections 2023-08-25T09:38:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e0052fe0 0x7fa8e0053470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/3571981970 >> v2:172.21.15.70:3300/0 conn(0x7fa8e00eb2c0 0x7fa8e00eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/3571981970 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e00e78b0 0x7fa8e004f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/3571981970 >> 172.21.15.70:0/3571981970 conn(0x7fa8e0054c10 msgr2=0x7fa8e0054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7f2e2effd640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f2e1c072a10 0x7f2e1c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 --> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e1c0754e0 con 0x7f2e1c072a10 2023-08-25T09:38:59.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e2808adf0 con 0x7f2e380b0440 2023-08-25T09:38:59.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7f2e3da42640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e1c06a150 0x7f2e1c06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb664000fc0 con 0x7fb67c007600 2023-08-25T09:38:59.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7f2e3ea44640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f2e1c072a10 0x7f2e1c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/3571981970 shutdown_connections 2023-08-25T09:38:59.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.887+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/3571981970 wait complete. 2023-08-25T09:38:59.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.888+0000 7f2e3da42640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e1c06a150 0x7f2e1c06c610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e34036000 tx=0x7f2e34008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.889+0000 7f2e3ea44640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f2e1c072a10 0x7f2e1c074e30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.888+0000 7f64f7fff640 1 -- 172.21.15.70:0/182687748 >> v1:172.21.15.155:6789/0 conn(0x7f64f00e78b0 legacy=0x7f64f004f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.888+0000 7f64f7fff640 1 -- 172.21.15.70:0/182687748 >> v1:172.21.15.70:6789/0 conn(0x7f64f0006910 legacy=0x7f64f0006da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.888+0000 7f64f77fe640 1 -- 172.21.15.70:0/182687748 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774108454 0 0) 0x7f64f0006480 con 0x7f64f00e78b0 2023-08-25T09:38:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.888+0000 7f7f0d2aa640 1 Processor -- start 2023-08-25T09:38:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.889+0000 7f7f0d2aa640 1 -- start start 2023-08-25T09:38:59.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.889+0000 7f64f7fff640 1 -- 172.21.15.70:0/182687748 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f004fd80 con 0x7f64f00eb2c0 2023-08-25T09:38:59.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.890+0000 7fa8ee1a4640 1 Processor -- start 2023-08-25T09:38:59.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.890+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== osd.2 v2:172.21.15.70:6815/284471511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f2e1c0754e0 con 0x7f2e1c072a10 2023-08-25T09:38:59.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.890+0000 7fa8ee1a4640 1 -- start start 2023-08-25T09:38:59.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f7f0d2aa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f080cb220 0x7f7f080b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f7f0d2aa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f080cb600 0x7f7f080cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f7f0d2aa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080c9b10 con 0x7f7f080b1b70 2023-08-25T09:38:59.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f7f0d2aa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f08161660 con 0x7f7f080cb600 2023-08-25T09:38:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f7f0d2aa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f080ccc20 con 0x7f7f080cb220 2023-08-25T09:38:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f1d5e870640 1 Processor -- start 2023-08-25T09:38:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7fa8ee1a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e0052fe0 0x7fa8e00e1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ee1a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e00e78b0 0x7fa8e00e2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ee1a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e00eb2c0 0x7fa8e00e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f7f07fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f080cb220 0x7f7f080b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f7f07fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f080cb220 0x7f7f080b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38420/0 (socket says 172.21.15.70:38420) 2023-08-25T09:38:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f7f07fff640 1 -- 172.21.15.70:0/34081878 learned_addr learned my addr 172.21.15.70:0/34081878 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ee1a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00e8940 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ee1a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00e7d30 con 0x7fa8e00e78b0 2023-08-25T09:38:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ee1a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e00e8010 con 0x7fa8e0052fe0 2023-08-25T09:38:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f64f7fff640 1 --2- 172.21.15.70:0/182687748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00eb2c0 0x7f64f00eb700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64e40097a0 tx=0x7f64e402b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18eafa0df1e1cd42 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb67801a400 con 0x7fb67c007600 2023-08-25T09:38:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fb6817fa640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66806a120 0x7fb66806c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb678088c60 con 0x7fb67c007600 2023-08-25T09:38:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.893+0000 7fb6817fa640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fb6680729e0 0x7fb668074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.893+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 --> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb6680754b0 con 0x7fb6680729e0 2023-08-25T09:38:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.893+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb678089010 con 0x7fb67c007600 2023-08-25T09:38:59.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ed9a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e00eb2c0 0x7fa8e00e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ed9a3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e00eb2c0 0x7fa8e00e8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47154/0 (socket says 172.21.15.70:47154) 2023-08-25T09:38:59.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/2158309051 learned_addr learned my addr 172.21.15.70:0/2158309051 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.893+0000 7fa8ed1a2640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e0052fe0 0x7fa8e00e1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.893+0000 7fa8ec9a1640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e00e78b0 0x7fa8e00e2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f7f07fff640 1 -- 172.21.15.70:0/34081878 >> v2:172.21.15.70:3300/0 conn(0x7f7f080cb600 msgr2=0x7f7f080cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f7f07fff640 1 --2- 172.21.15.70:0/34081878 >> v2:172.21.15.70:3300/0 conn(0x7f7f080cb600 0x7f7f080cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f7f07fff640 1 -- 172.21.15.70:0/34081878 >> v1:172.21.15.155:6789/0 conn(0x7f7f080b1b70 legacy=0x7f7f080b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.892+0000 7f7f07fff640 1 -- 172.21.15.70:0/34081878 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080ccf00 con 0x7f7f080cb220 2023-08-25T09:38:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e0052fe0 msgr2=0x7fa8e00e1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e0052fe0 0x7fa8e00e1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e00e78b0 msgr2=0x7fa8e00e2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e00e78b0 0x7fa8e00e2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/2158309051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8009400 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f7f07fff640 1 --2- 172.21.15.70:0/34081878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb220 0x7f7f080b1630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f080cbce0 tx=0x7f7ef802b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278adbe97cacb9db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fb6890ac640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fb6680729e0 0x7fb668074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7fb683fff640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66806a120 0x7fb66806c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.891+0000 7f1d5e870640 1 -- start start 2023-08-25T09:38:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f18355a4640 1 Processor -- start 2023-08-25T09:38:59.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f1d5e870640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 0x7f1d580b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f1d5e870640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1d580c1450 0x7f1d580ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f1d5e870640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d580b15a0 con 0x7f1d580cea70 2023-08-25T09:38:59.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f1d5e870640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d580cef00 con 0x7f1d580c1450 2023-08-25T09:38:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f1d5e870640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580c1cd0 con 0x7f1d581610b0 2023-08-25T09:38:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f1d5d86e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 0x7f1d580b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f1d5d86e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 0x7f1d580b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44732/0 (socket says 172.21.15.70:44732) 2023-08-25T09:38:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f1d5d86e640 1 -- 172.21.15.70:0/3417767067 learned_addr learned my addr 172.21.15.70:0/3417767067 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f7f06ffd640 1 -- 172.21.15.70:0/34081878 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8030880 con 0x7f7f080cb220 2023-08-25T09:38:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f7f06ffd640 1 -- 172.21.15.70:0/34081878 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef80309e0 con 0x7f7f080cb220 2023-08-25T09:38:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f7f06ffd640 1 -- 172.21.15.70:0/34081878 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8030db0 con 0x7f7f080cb220 2023-08-25T09:38:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7fb683fff640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66806a120 0x7fb66806c5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb670034000 tx=0x7fb67002d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f64f77fe640 1 -- 172.21.15.70:0/182687748 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4030710 con 0x7f64f00eb2c0 2023-08-25T09:38:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f64f77fe640 1 -- 172.21.15.70:0/182687748 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e402e050 con 0x7f64f00eb2c0 2023-08-25T09:38:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.894+0000 7f64f77fe640 1 -- 172.21.15.70:0/182687748 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4030a90 con 0x7f64f00eb2c0 2023-08-25T09:38:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f64fdb09640 1 -- 172.21.15.70:0/182687748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00eb2c0 msgr2=0x7f64f00eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f64fdb09640 1 --2- 172.21.15.70:0/182687748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00eb2c0 0x7f64f00eb700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f64e40097a0 tx=0x7f64e402b6e0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7fa8ed9a3640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e00eb2c0 0x7fa8e00e8dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009890 tx=0x7fa8d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f1d5d86e640 1 -- 172.21.15.70:0/3417767067 >> v2:172.21.15.155:3300/0 conn(0x7f1d580c1450 msgr2=0x7f1d580ce530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T09:38:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f1d5d86e640 1 --2- 172.21.15.70:0/3417767067 >> v2:172.21.15.155:3300/0 conn(0x7f1d580c1450 0x7f1d580ce530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f1d5c86c640 1 -- 172.21.15.70:0/3417767067 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352559039 0 0) 0x7f1d580b15a0 con 0x7f1d580cea70 2023-08-25T09:38:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f1d5d86e640 1 -- 172.21.15.70:0/3417767067 >> v1:172.21.15.155:6789/0 conn(0x7f1d580cea70 legacy=0x7f1d580caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f1d5d86e640 1 -- 172.21.15.70:0/3417767067 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d580cc310 con 0x7f1d581610b0 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/34081878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb220 msgr2=0x7f7f080b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 --2- 172.21.15.70:0/34081878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb220 0x7f7f080b1630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f080cbce0 tx=0x7f7ef802b9c0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/34081878 shutdown_connections 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 --2- 172.21.15.70:0/34081878 >> v2:172.21.15.70:3300/0 conn(0x7f7f080cb600 0x7f7f080cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 --2- 172.21.15.70:0/34081878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb220 0x7f7f080b1630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/34081878 >> 172.21.15.70:0/34081878 conn(0x7f7f0806ac60 msgr2=0x7f7f0806b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/34081878 shutdown_connections 2023-08-25T09:38:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.896+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/34081878 wait complete. 2023-08-25T09:38:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.895+0000 7f18355a4640 1 -- start start 2023-08-25T09:38:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7fb6890ac640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fb6680729e0 0x7fb668074e00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7f64fdb09640 1 -- 172.21.15.70:0/182687748 shutdown_connections 2023-08-25T09:38:59.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7f64fdb09640 1 --2- 172.21.15.70:0/182687748 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00eb2c0 0x7f64f00eb700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7f64fdb09640 1 -- 172.21.15.70:0/182687748 >> 172.21.15.70:0/182687748 conn(0x7f64f0054c10 msgr2=0x7f64f0054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8033980 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8033ae0 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8033e70 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.897+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e00e82f0 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== osd.1 v2:172.21.15.70:6800/1147342635 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fb6680754b0 con 0x7fb6680729e0 2023-08-25T09:38:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7f18355a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f182804fcf0 0x7f182803a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7f18355a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280383e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7f7f0d2aa640 1 Processor -- start 2023-08-25T09:38:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7f182e7fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f182804fcf0 0x7f182803a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7f182e7fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f182804fcf0 0x7f182803a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38438/0 (socket says 172.21.15.70:38438) 2023-08-25T09:38:59.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.898+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e00e24e0 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f182f7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f182f7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280383e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44744/0 (socket says 172.21.15.70:44744) 2023-08-25T09:38:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f182e7fc640 1 -- 172.21.15.70:0/2419739671 learned_addr learned my addr 172.21.15.70:0/2419739671 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f64fdb09640 1 -- 172.21.15.70:0/182687748 shutdown_connections 2023-08-25T09:38:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7fa8d37fe640 1 -- 172.21.15.70:0/2158309051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8c0000fc0 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18280077c0 con 0x7f1828050230 2023-08-25T09:38:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182803a690 con 0x7f182804fcf0 2023-08-25T09:38:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1828006bb0 con 0x7f182800ab00 2023-08-25T09:38:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.900+0000 7f7f0d2aa640 1 -- start start 2023-08-25T09:38:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7f7f0d2aa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f080b1b70 0x7f7f08159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8d803a040 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7f7f0d2aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f080cb220 0x7f7f08162ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7f7f0d2aa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb600 0x7f7f081eaae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7f7f0d2aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08163940 con 0x7f7f080cb220 2023-08-25T09:38:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7f7f0d2aa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08163c20 con 0x7f7f080cb600 2023-08-25T09:38:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.901+0000 7f7f0d2aa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08163f00 con 0x7f7f080b1b70 2023-08-25T09:38:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.899+0000 7f64fdb09640 1 -- 172.21.15.70:0/182687748 wait complete. 2023-08-25T09:38:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7f64fdb09640 1 Processor -- start 2023-08-25T09:38:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7fa8ddffb640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8cc06a0d0 0x7fa8cc06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8d80a9f60 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.903+0000 7fa8ddffb640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7fa8cc072990 0x7fa8cc074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7f1d5d86e640 1 --2- 172.21.15.70:0/3417767067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 0x7f1d580b3f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009870 tx=0x7f1d5402b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d318fa00cd46e8f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7f7f077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f080cb220 0x7f7f08162ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7f7f077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f080cb220 0x7f7f08162ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47166/0 (socket says 172.21.15.70:47166) 2023-08-25T09:38:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.902+0000 7f7f077fe640 1 -- 172.21.15.70:0/3975311347 learned_addr learned my addr 172.21.15.70:0/3975311347 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.900+0000 7f182e7fc640 1 -- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 msgr2=0x7f18280383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:38:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.900+0000 7f182e7fc640 1 --2- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.900+0000 7f182e7fc640 1 -- 172.21.15.70:0/2419739671 >> v1:172.21.15.155:6789/0 conn(0x7f1828050230 legacy=0x7f182804f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.900+0000 7f182e7fc640 1 -- 172.21.15.70:0/2419739671 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828006e90 con 0x7f182804fcf0 2023-08-25T09:38:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.903+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 --> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8cc075460 con 0x7fa8cc072990 2023-08-25T09:38:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.903+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d80ac540 con 0x7fa8e00eb2c0 2023-08-25T09:38:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.903+0000 7f7f0caa9640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb600 0x7f7f081eaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.903+0000 7fa8ed1a2640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8cc06a0d0 0x7fa8cc06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f7f077fe640 1 -- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f080b1b70 msgr2=0x7f7f08159bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T09:38:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7fa8ed1a2640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8cc06a0d0 0x7fa8cc06c590 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80799c0 tx=0x7fa8e808c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.903+0000 7f64fdb09640 1 -- start start 2023-08-25T09:38:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7fa8ec9a1640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7fa8cc072990 0x7fa8cc074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f1d5c86c640 1 -- 172.21.15.70:0/3417767067 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54030860 con 0x7f1d581610b0 2023-08-25T09:38:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f1d5c86c640 1 -- 172.21.15.70:0/3417767067 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d540309c0 con 0x7f1d581610b0 2023-08-25T09:38:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f1d5c86c640 1 -- 172.21.15.70:0/3417767067 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d54030d50 con 0x7f1d581610b0 2023-08-25T09:38:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f7f077fe640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f080b1b70 0x7f7f08159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f7f077fe640 1 -- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb600 msgr2=0x7f7f081eaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f7f077fe640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f080cb600 0x7f7f081eaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f7f077fe640 1 -- 172.21.15.70:0/3975311347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80094d0 con 0x7f7f080cb220 2023-08-25T09:38:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f64fdb09640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f0006910 0x7f64f00e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f64fdb09640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00e78b0 0x7f64f01884d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.904+0000 7f7f077fe640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f080cb220 0x7f7f08162ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b410 tx=0x7f7efc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fdb09640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f00eb2c0 0x7f64f00e3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fdb09640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0187430 con 0x7f64f00e78b0 2023-08-25T09:38:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fdb09640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f0187710 con 0x7f64f00eb2c0 2023-08-25T09:38:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fdb09640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f01879f0 con 0x7f64f0006910 2023-08-25T09:38:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 -- 172.21.15.70:0/3417767067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 msgr2=0x7f1d580b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 --2- 172.21.15.70:0/3417767067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 0x7f1d580b3f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009870 tx=0x7f1d5402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fcb07640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f0006910 0x7f64f00e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fcb07640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f0006910 0x7f64f00e2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44770/0 (socket says 172.21.15.70:44770) 2023-08-25T09:38:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fcb07640 1 -- 172.21.15.70:0/2528478156 learned_addr learned my addr 172.21.15.70:0/2528478156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 -- 172.21.15.70:0/3417767067 shutdown_connections 2023-08-25T09:38:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 --2- 172.21.15.70:0/3417767067 >> v2:172.21.15.155:3300/0 conn(0x7f1d580c1450 0x7f1d580ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 --2- 172.21.15.70:0/3417767067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d581610b0 0x7f1d580b3f40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 -- 172.21.15.70:0/3417767067 >> 172.21.15.70:0/3417767067 conn(0x7f1d58069eb0 msgr2=0x7f1d580b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64f7fff640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00e78b0 0x7f64f01884d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f1d5e870640 1 -- 172.21.15.70:0/3417767067 shutdown_connections 2023-08-25T09:38:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc011830 con 0x7f7f080cb220 2023-08-25T09:38:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc0119b0 con 0x7f7f080cb220 2023-08-25T09:38:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc011d60 con 0x7f7f080cb220 2023-08-25T09:38:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/3975311347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f081641e0 con 0x7f7f080cb220 2023-08-25T09:38:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.905+0000 7f64fd308640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f00eb2c0 0x7f64f00e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f182e7fc640 1 --2- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182804fcf0 0x7f182803a150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18240098a0 tx=0x7f182402ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4af2e831fa5221 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f64f7fff640 1 -- 172.21.15.70:0/2528478156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f0006910 msgr2=0x7f64f00e2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f64f7fff640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f0006910 0x7f64f00e2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f64f7fff640 1 -- 172.21.15.70:0/2528478156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f00eb2c0 msgr2=0x7f64f00e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f64f7fff640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f00eb2c0 0x7f64f00e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f64f7fff640 1 -- 172.21.15.70:0/2528478156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4009400 con 0x7f64f00e78b0 2023-08-25T09:38:59.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7f64f7fff640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00e78b0 0x7f64f01884d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64e4032000 tx=0x7f64e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.906+0000 7f1d5e870640 1 -- 172.21.15.70:0/3417767067 wait complete. 2023-08-25T09:38:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7fa8ec9a1640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7fa8cc072990 0x7fa8cc074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/3975311347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f08071ad0 con 0x7f7f080cb220 2023-08-25T09:38:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4032910 con 0x7f64f00e78b0 2023-08-25T09:38:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f0187cd0 con 0x7f64f00e78b0 2023-08-25T09:38:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4032a90 con 0x7f64f00e78b0 2023-08-25T09:38:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4032e40 con 0x7f64f00e78b0 2023-08-25T09:38:59.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f00e30d0 con 0x7f64f00e78b0 2023-08-25T09:38:59.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f1d5e870640 1 Processor -- start 2023-08-25T09:38:59.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f1d5e870640 1 -- start start 2023-08-25T09:38:59.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7f182dffb640 1 -- 172.21.15.70:0/2419739671 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182402f870 con 0x7f182804fcf0 2023-08-25T09:38:59.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7f182dffb640 1 -- 172.21.15.70:0/2419739671 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182402f9d0 con 0x7f182804fcf0 2023-08-25T09:38:59.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.907+0000 7f182dffb640 1 -- 172.21.15.70:0/2419739671 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182402fda0 con 0x7f182804fcf0 2023-08-25T09:38:59.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182804fcf0 msgr2=0x7f182803a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f18355a4640 1 --2- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182804fcf0 0x7f182803a150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18240098a0 tx=0x7f182402ba50 comp rx=0 tx=0).stop 2023-08-25T09:38:59.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.908+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/3975311347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f080cd4c0 con 0x7f7f080cb220 2023-08-25T09:38:59.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.909+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== osd.4 v2:172.21.15.155:6800/4222826791 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fa8cc075460 con 0x7fa8cc072990 2023-08-25T09:38:59.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5e870640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d580c1450 0x7f1d5814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5e870640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d580cea70 0x7f1d580b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.909+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 shutdown_connections 2023-08-25T09:38:59.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.909+0000 7f18355a4640 1 --2- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.909+0000 7f18355a4640 1 --2- 172.21.15.70:0/2419739671 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f182804fcf0 0x7f182803a150 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f18240098a0 tx=0x7f182402ba50 comp rx=0 tx=0).stop 2023-08-25T09:38:59.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.909+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 >> 172.21.15.70:0/2419739671 conn(0x7f1828009fe0 msgr2=0x7f182800a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5e870640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d581610b0 0x7f1d5815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5e870640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d581462f0 con 0x7f1d581610b0 2023-08-25T09:38:59.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5e870640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d58161490 con 0x7f1d580c1450 2023-08-25T09:38:59.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5e870640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d581616e0 con 0x7f1d580cea70 2023-08-25T09:38:59.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64f0187880 con 0x7f64f00e78b0 2023-08-25T09:38:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.912+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 --> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e740ccb00 con 0x7f1e60072990 2023-08-25T09:38:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f1e5b7fe640 1 -- 172.21.15.70:0/553215651 <== osd.0 v2:172.21.15.70:6808/3728228047 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e6c001ec0 con 0x7f1e60072990 2023-08-25T09:38:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.912+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f64e4039040 con 0x7f64f00e78b0 2023-08-25T09:38:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f64f57fa640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64dc06a150 0x7f64dc06c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64e40aa340 con 0x7f64f00e78b0 2023-08-25T09:38:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f64f57fa640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7f64dc072990 0x7f64dc074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5d06d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d580cea70 0x7f1d580b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5d06d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d580cea70 0x7f1d580b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44786/0 (socket says 172.21.15.70:44786) 2023-08-25T09:38:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f1d5d06d640 1 -- 172.21.15.70:0/3395800333 learned_addr learned my addr 172.21.15.70:0/3395800333 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f1d5d86e640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d580c1450 0x7f1d5814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f1d5d06d640 1 -- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d580c1450 msgr2=0x7f1d5814d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f1d5d06d640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1d580c1450 0x7f1d5814d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f1d5d06d640 1 -- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d581610b0 msgr2=0x7f1d5815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f1d5d06d640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1d581610b0 0x7f1d5815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f1d5d06d640 1 -- 172.21.15.70:0/3395800333 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540094d0 con 0x7f1d580cea70 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.912+0000 7f1d5d06d640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d580cea70 0x7f1d580b1780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800b480 tx=0x7f1d4800b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48010870 con 0x7f1d580cea70 2023-08-25T09:38:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d480109f0 con 0x7f1d580cea70 2023-08-25T09:38:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48010da0 con 0x7f1d580cea70 2023-08-25T09:38:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5814cd00 con 0x7f1d580cea70 2023-08-25T09:38:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5815a950 con 0x7f1d580cea70 2023-08-25T09:38:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.911+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7efc01a400 con 0x7f7f080cb220 2023-08-25T09:38:59.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 shutdown_connections 2023-08-25T09:38:59.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.910+0000 7f18355a4640 1 -- 172.21.15.70:0/2419739671 wait complete. 2023-08-25T09:38:59.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f18355a4640 1 Processor -- start 2023-08-25T09:38:59.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f7f04ff9640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ef406a120 0x7f7ef406c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7efc088680 con 0x7f7f080cb220 2023-08-25T09:38:59.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f7f04ff9640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f7ef4072960 0x7f7ef4074d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.913+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 --> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ef4075430 con 0x7f7ef4072960 2023-08-25T09:38:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f7f07fff640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ef406a120 0x7f7ef406c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f64fcb07640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64dc06a150 0x7f64dc06c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 --> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64dc075460 con 0x7f64dc072990 2023-08-25T09:38:59.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f64fd308640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7f64dc072990 0x7f64dc074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.914+0000 7f7f0caa9640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f7ef4072960 0x7f7ef4074d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.915+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d580cc8d0 con 0x7f1d580cea70 2023-08-25T09:38:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.915+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e4075b50 con 0x7f64f00e78b0 2023-08-25T09:38:59.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.915+0000 7f18355a4640 1 -- start start 2023-08-25T09:38:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.915+0000 7f64fd308640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7f64dc072990 0x7f64dc074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.915+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7efc009230 con 0x7f7f080cb220 2023-08-25T09:38:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f7f0caa9640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f7ef4072960 0x7f7ef4074d80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f64fcb07640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64dc06a150 0x7f64dc06c610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64ec001320 tx=0x7f64ec0012b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f7f07fff640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ef406a120 0x7f7ef406c5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8036000 tx=0x7f7ef803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f18355a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280deae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== osd.3 v2:172.21.15.70:6824/2671981919 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f64dc075460 con 0x7f64dc072990 2023-08-25T09:38:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7f1e60072990 msgr2=0x7f1e60074db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f1e791cf640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7f1e60072990 0x7f1e60074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.916+0000 7f18355a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18280364a0 0x7f182817b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f18355a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1828036920 0x7f182817b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f18355a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182818a1d0 con 0x7f1828036920 2023-08-25T09:38:59.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f182effd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f182effd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280deae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44796/0 (socket says 172.21.15.70:44796) 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f182effd640 1 -- 172.21.15.70:0/858948076 learned_addr learned my addr 172.21.15.70:0/858948076 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f182f7fe640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1828036920 0x7f182817b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1828189650 con 0x7f18280364a0 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1828189930 con 0x7f182800ab00 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.917+0000 7f182e7fc640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18280364a0 0x7f182817b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f182f7fe640 1 -- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 msgr2=0x7f18280deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f182f7fe640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f182800ab00 0x7f18280deae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f182f7fe640 1 -- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18280364a0 msgr2=0x7f182817b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f182f7fe640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18280364a0 0x7f182817b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f182f7fe640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18240094d0 con 0x7f1828036920 2023-08-25T09:38:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== osd.6 v2:172.21.15.155:6816/4156222745 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f7ef4075430 con 0x7f7ef4072960 2023-08-25T09:38:59.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.918+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d48019450 con 0x7f1d580cea70 2023-08-25T09:38:59.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.919+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a150 msgr2=0x7f1e6006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.919+0000 7f1e791cf640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a150 0x7f1e6006c610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e740c4d70 tx=0x7f1e64008040 comp rx=0 tx=0).stop 2023-08-25T09:38:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.920+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e74201950 msgr2=0x7f1e7415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.920+0000 7f1e791cf640 1 --2- 172.21.15.70:0/553215651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e74201950 0x7f1e7415eb60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1e68001720 tx=0x7f1e68001120 comp rx=0 tx=0).stop 2023-08-25T09:38:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.920+0000 7f1e7359e640 1 -- 172.21.15.70:0/553215651 reap_dead start 2023-08-25T09:38:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.919+0000 7f182f7fe640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1828036920 0x7f182817b870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f183002c740 tx=0x7f1830066c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.919+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18300739a0 con 0x7f1828036920 2023-08-25T09:38:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.919+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830073b00 con 0x7f1828036920 2023-08-25T09:38:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.919+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830073e70 con 0x7f1828036920 2023-08-25T09:38:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.920+0000 7f1d4e7fc640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1d3c06a0d0 0x7f1d3c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.920+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d48088a20 con 0x7f1d580cea70 2023-08-25T09:38:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1d4e7fc640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f1d3c072990 0x7f1d3c074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 --> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d3c075460 con 0x7f1d3c072990 2023-08-25T09:38:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d48088d90 con 0x7f1d580cea70 2023-08-25T09:38:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 shutdown_connections 2023-08-25T09:38:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 >> 172.21.15.70:0/553215651 conn(0x7f1e74069c40 msgr2=0x7f1e740654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 shutdown_connections 2023-08-25T09:38:59.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1d5e06f640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f1d3c072990 0x7f1d3c074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f1d5d86e640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1d3c06a0d0 0x7f1d3c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1828189c10 con 0x7f1828036920 2023-08-25T09:38:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.921+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18280df020 con 0x7f1828036920 2023-08-25T09:38:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.922+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 --> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2e3815a7d0 con 0x7f2e1c072a10 2023-08-25T09:38:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.922+0000 7f1d5d86e640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1d3c06a0d0 0x7f1d3c06c590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d54036000 tx=0x7f1d5403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.922+0000 7f1d5e06f640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f1d3c072990 0x7f1d3c074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.923+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1828007450 con 0x7f1828036920 2023-08-25T09:38:59.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.923+0000 7f1e791cf640 1 -- 172.21.15.70:0/553215651 wait complete. 2023-08-25T09:38:59.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.923+0000 7f2e2effd640 1 -- 172.21.15.70:0/370201679 <== osd.2 v2:172.21.15.70:6815/284471511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2e30001e10 con 0x7f2e1c072a10 2023-08-25T09:38:59.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.924+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f2e1c072a10 msgr2=0x7f2e1c074e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.924+0000 7f2e404a7640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f2e1c072a10 0x7f2e1c074e30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.924+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18300933f0 con 0x7f1828036920 2023-08-25T09:38:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.924+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== osd.5 v2:172.21.15.155:6818/1708871082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f1d3c075460 con 0x7f1d3c072990 2023-08-25T09:38:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.924+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e1c06a150 msgr2=0x7f2e1c06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.924+0000 7f2e404a7640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e1c06a150 0x7f2e1c06c610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2e34036000 tx=0x7f2e34008030 comp rx=0 tx=0).stop 2023-08-25T09:38:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e380b0440 msgr2=0x7f2e38156da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f2e404a7640 1 --2- 172.21.15.70:0/370201679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e380b0440 0x7f2e38156da0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c7f0 tx=0x7f2e2800ccb0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f2e3ea44640 1 -- 172.21.15.70:0/370201679 reap_dead start 2023-08-25T09:38:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 shutdown_connections 2023-08-25T09:38:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f18177fe640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181c06a0d0 0x7f181c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 >> 172.21.15.70:0/370201679 conn(0x7f2e380caf00 msgr2=0x7f2e381539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.925+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18301015b0 con 0x7f1828036920 2023-08-25T09:38:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f18177fe640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7f181c072990 0x7f181c074db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:38:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f181c075460 con 0x7f181c072990 2023-08-25T09:38:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1830101930 con 0x7f1828036920 2023-08-25T09:38:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f182effd640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181c06a0d0 0x7f181c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 shutdown_connections 2023-08-25T09:38:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f2e404a7640 1 -- 172.21.15.70:0/370201679 wait complete. 2023-08-25T09:38:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.926+0000 7f182e7fc640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7f181c072990 0x7f181c074db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:38:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.927+0000 7f182effd640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181c06a0d0 0x7f181c06c590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1820007fd0 tx=0x7f1820009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.927+0000 7f182e7fc640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7f181c072990 0x7f181c074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:38:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.929+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== osd.7 v2:172.21.15.155:6808/324773794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f181c075460 con 0x7f181c072990 2023-08-25T09:38:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.932+0000 7fa8d37fe640 1 -- 172.21.15.70:0/2158309051 --> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8c0002cb0 con 0x7fa8cc072990 2023-08-25T09:38:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.934+0000 7fa8ddffb640 1 -- 172.21.15.70:0/2158309051 <== osd.4 v2:172.21.15.155:6800/4222826791 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8c0002cb0 con 0x7fa8cc072990 2023-08-25T09:38:59.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.936+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 --> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb664002cb0 con 0x7fb6680729e0 2023-08-25T09:38:59.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.937+0000 7fb6817fa640 1 -- 172.21.15.70:0/2732836833 <== osd.1 v2:172.21.15.70:6800/1147342635 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb664002cb0 con 0x7fb6680729e0 2023-08-25T09:38:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.937+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fb6680729e0 msgr2=0x7fb668074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fb6680729e0 0x7fb668074e00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66806a120 msgr2=0x7fb66806c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb66806a120 0x7fb66806c5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb670034000 tx=0x7fb67002d000 comp rx=0 tx=0).stop 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb67c007600 msgr2=0x7fb67c04a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 --2- 172.21.15.70:0/2732836833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb67c007600 0x7fb67c04a810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb67800b3e0 tx=0x7fb67800b8a0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb6890ac640 1 -- 172.21.15.70:0/2732836833 reap_dead start 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 shutdown_connections 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 >> 172.21.15.70:0/2732836833 conn(0x7fb67c009fe0 msgr2=0x7fb67c00a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 shutdown_connections 2023-08-25T09:38:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.939+0000 7fb68ab0f640 1 -- 172.21.15.70:0/2732836833 wait complete. 2023-08-25T09:38:59.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.945+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 --> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64f0006110 con 0x7f64dc072990 2023-08-25T09:38:59.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.945+0000 7f7f0d2aa640 1 -- 172.21.15.70:0/3975311347 --> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f080cd1e0 con 0x7f7ef4072960 2023-08-25T09:38:59.989 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-08-25T09:38:59.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-25T09:38:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.946+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7fa8cc072990 msgr2=0x7fa8cc074db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.946+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7fa8cc072990 0x7fa8cc074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.946+0000 7f7f04ff9640 1 -- 172.21.15.70:0/3975311347 <== osd.6 v2:172.21.15.155:6816/4156222745 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f00005110 con 0x7f7ef4072960 2023-08-25T09:38:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7f64f57fa640 1 -- 172.21.15.70:0/2528478156 <== osd.3 v2:172.21.15.70:6824/2671981919 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f64f804e140 con 0x7f64dc072990 2023-08-25T09:38:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.950+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7f64dc072990 msgr2=0x7f64dc074db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.950+0000 7f64fdb09640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7f64dc072990 0x7f64dc074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.950+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64dc06a150 msgr2=0x7f64dc06c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.950+0000 7f64fdb09640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64dc06a150 0x7f64dc06c610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64ec001320 tx=0x7f64ec0012b0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.951+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00e78b0 msgr2=0x7f64f01884d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.951+0000 7f64fdb09640 1 --2- 172.21.15.70:0/2528478156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f00e78b0 0x7f64f01884d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f64e4032000 tx=0x7f64e40046c0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.951+0000 7f64fd308640 1 -- 172.21.15.70:0/2528478156 reap_dead start 2023-08-25T09:38:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.951+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 shutdown_connections 2023-08-25T09:38:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.951+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 >> 172.21.15.70:0/2528478156 conn(0x7f64f0054c10 msgr2=0x7f64f004bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8cc06a0d0 msgr2=0x7fa8cc06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8cc06a0d0 0x7fa8cc06c590 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80799c0 tx=0x7fa8e808c040 comp rx=0 tx=0).stop 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e00eb2c0 msgr2=0x7fa8e00e8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ee1a4640 1 --2- 172.21.15.70:0/2158309051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e00eb2c0 0x7fa8e00e8dd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009890 tx=0x7fa8d80046c0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ed9a3640 1 -- 172.21.15.70:0/2158309051 reap_dead start 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 shutdown_connections 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 >> 172.21.15.70:0/2158309051 conn(0x7fa8e0054c10 msgr2=0x7fa8e004c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.950+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 shutdown_connections 2023-08-25T09:38:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.950+0000 7fa8ee1a4640 1 -- 172.21.15.70:0/2158309051 wait complete. 2023-08-25T09:38:59.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.948+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f7ef4072960 msgr2=0x7f7ef4074d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.948+0000 7f7eee7fc640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f7ef4072960 0x7f7ef4074d80 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.948+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ef406a120 msgr2=0x7f7ef406c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.948+0000 7f7eee7fc640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ef406a120 0x7f7ef406c5e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8036000 tx=0x7f7ef803b040 comp rx=0 tx=0).stop 2023-08-25T09:38:59.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.948+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f080cb220 msgr2=0x7f7f08162ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:38:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.948+0000 7f7eee7fc640 1 --2- 172.21.15.70:0/3975311347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f080cb220 0x7f7f08162ea0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b410 tx=0x7f7efc00b8d0 comp rx=0 tx=0).stop 2023-08-25T09:38:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.949+0000 7f7f0caa9640 1 -- 172.21.15.70:0/3975311347 reap_dead start 2023-08-25T09:38:59.995 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:38:59.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-25T09:38:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.953+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 shutdown_connections 2023-08-25T09:38:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.953+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 >> 172.21.15.70:0/3975311347 conn(0x7f7f0806ac60 msgr2=0x7f7f080af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:38:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.953+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 shutdown_connections 2023-08-25T09:38:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.954+0000 7f64fdb09640 1 -- 172.21.15.70:0/2528478156 wait complete. 2023-08-25T09:38:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.954+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 shutdown_connections 2023-08-25T09:38:59.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.954+0000 7f7eee7fc640 1 -- 172.21.15.70:0/3975311347 wait complete. 2023-08-25T09:38:59.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.954+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 --> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1828007170 con 0x7f181c072990 2023-08-25T09:38:59.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.955+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 --> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d580cc5f0 con 0x7f1d3c072990 2023-08-25T09:38:59.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.955+0000 7f18177fe640 1 -- 172.21.15.70:0/858948076 <== osd.7 v2:172.21.15.155:6808/324773794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f182402fb40 con 0x7f181c072990 2023-08-25T09:38:59.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f1d4e7fc640 1 -- 172.21.15.70:0/3395800333 <== osd.5 v2:172.21.15.155:6818/1708871082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1d50001870 con 0x7f1d3c072990 2023-08-25T09:38:59.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7f181c072990 msgr2=0x7f181c074db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f18355a4640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7f181c072990 0x7f181c074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f1d3c072990 msgr2=0x7f1d3c074db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f1d5e870640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f1d3c072990 0x7f1d3c074db0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181c06a0d0 msgr2=0x7f181c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f18355a4640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181c06a0d0 0x7f181c06c590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1820007fd0 tx=0x7f1820009040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1828036920 msgr2=0x7f182817b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f18355a4640 1 --2- 172.21.15.70:0/858948076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1828036920 0x7f182817b870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f183002c740 tx=0x7f1830066c70 comp rx=0 tx=0).stop 2023-08-25T09:39:00.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1d3c06a0d0 msgr2=0x7f1d3c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f1d5e870640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1d3c06a0d0 0x7f1d3c06c590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d54036000 tx=0x7f1d5403b040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.956+0000 7f182f7fe640 1 -- 172.21.15.70:0/858948076 reap_dead start 2023-08-25T09:39:00.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d580cea70 msgr2=0x7f1d580b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 shutdown_connections 2023-08-25T09:39:00.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 >> 172.21.15.70:0/858948076 conn(0x7f1828009fe0 msgr2=0x7f182804d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f1d5e870640 1 --2- 172.21.15.70:0/3395800333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1d580cea70 0x7f1d580b1780 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800b480 tx=0x7f1d4800b940 comp rx=0 tx=0).stop 2023-08-25T09:39:00.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 shutdown_connections 2023-08-25T09:39:00.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f1d5e06f640 1 -- 172.21.15.70:0/3395800333 reap_dead start 2023-08-25T09:39:00.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f18355a4640 1 -- 172.21.15.70:0/858948076 wait complete. 2023-08-25T09:39:00.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 shutdown_connections 2023-08-25T09:39:00.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.957+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 >> 172.21.15.70:0/3395800333 conn(0x7f1d58069eb0 msgr2=0x7f1d580caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.958+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 shutdown_connections 2023-08-25T09:39:00.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:38:59.958+0000 7f1d5e870640 1 -- 172.21.15.70:0/3395800333 wait complete. 2023-08-25T09:39:00.006 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:00.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-25T09:39:00.007 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:00.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-25T09:39:00.008 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:00.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-25T09:39:00.009 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:00.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-25T09:39:00.010 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:00.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-25T09:39:00.011 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:00.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-25T09:39:00.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.335+0000 7f03abc04640 1 Processor -- start 2023-08-25T09:39:00.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.335+0000 7f03abc04640 1 -- start start 2023-08-25T09:39:00.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.334+0000 7f6c150d7640 1 Processor -- start 2023-08-25T09:39:00.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.334+0000 7f6c150d7640 1 -- start start 2023-08-25T09:39:00.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.336+0000 7f6c150d7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c100cb220 0x7f6c100b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.336+0000 7f6c150d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c100c9950 0x7f6c100cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.336+0000 7f6c150d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100cd1d0 con 0x7f6c100cb600 2023-08-25T09:39:00.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.336+0000 7f6c150d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100cc5c0 con 0x7f6c100c9950 2023-08-25T09:39:00.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.336+0000 7f6c150d7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100cc8a0 con 0x7f6c100cb220 2023-08-25T09:39:00.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6c100cb220 0x7f6c100b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c100c9950 0x7f6c100cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6c100c9950 0x7f6c100cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47202/0 (socket says 172.21.15.70:47202) 2023-08-25T09:39:00.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f03abc04640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f03a40caf00 0x7f03a40cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f03abc04640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03a41598f0 0x7f03a40aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f03abc04640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a40b0440 0x7f03a415c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f03abc04640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a40b08d0 con 0x7f03a41598f0 2023-08-25T09:39:00.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f03abc04640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a415d500 con 0x7f03a40caf00 2023-08-25T09:39:00.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f03abc04640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40cd030 con 0x7f03a40b0440 2023-08-25T09:39:00.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/751412570 learned_addr learned my addr 172.21.15.70:0/751412570 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/751412570 >> v2:172.21.15.155:3300/0 conn(0x7f6c100cb220 msgr2=0x7f6c100b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0f7fe640 1 --2- 172.21.15.70:0/751412570 >> v2:172.21.15.155:3300/0 conn(0x7f6c100cb220 0x7f6c100b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.337+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/751412570 >> v1:172.21.15.70:6789/0 conn(0x7f6c100cb600 legacy=0x7f6c100cdba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T09:39:00.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.338+0000 7f03a99a0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f03a40caf00 0x7f03a40cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.338+0000 7f03a99a0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f03a40caf00 0x7f03a40cd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38508/0 (socket says 172.21.15.70:38508) 2023-08-25T09:39:00.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.338+0000 7f03a919f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03a41598f0 0x7f03a40aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.338+0000 7f03a919f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f03a41598f0 0x7f03a40aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47210/0 (socket says 172.21.15.70:47210) 2023-08-25T09:39:00.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.338+0000 7f03a919f640 1 -- 172.21.15.70:0/3352583841 learned_addr learned my addr 172.21.15.70:0/3352583841 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.338+0000 7f03aa1a1640 1 --2- 172.21.15.70:0/3352583841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a40b0440 0x7f03a415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.339+0000 7f03a919f640 1 -- 172.21.15.70:0/3352583841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a40b0440 msgr2=0x7f03a415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.339+0000 7f03a919f640 1 --2- 172.21.15.70:0/3352583841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a40b0440 0x7f03a415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.339+0000 7f03a919f640 1 -- 172.21.15.70:0/3352583841 >> v2:172.21.15.155:3300/0 conn(0x7f03a40caf00 msgr2=0x7f03a40cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.339+0000 7f03a919f640 1 --2- 172.21.15.70:0/3352583841 >> v2:172.21.15.155:3300/0 conn(0x7f03a40caf00 0x7f03a40cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.339+0000 7f03a919f640 1 -- 172.21.15.70:0/3352583841 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40cc9e0 con 0x7f03a41598f0 2023-08-25T09:39:00.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.340+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/751412570 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100ccb80 con 0x7f6c100c9950 2023-08-25T09:39:00.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.341+0000 7f03a919f640 1 --2- 172.21.15.70:0/3352583841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a41598f0 0x7f03a40aff00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03a40b0ac0 tx=0x7f039c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b3d0d115d8a8a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.341+0000 7f6c0f7fe640 1 --2- 172.21.15.70:0/751412570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100c9950 0x7f6c100cd660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c100cbce0 tx=0x7f6bf800b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1ac437f2292482 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.342+0000 7f6c0effd640 1 -- 172.21.15.70:0/751412570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf802c9c0 con 0x7f6c100c9950 2023-08-25T09:39:00.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.342+0000 7f6c0effd640 1 -- 172.21.15.70:0/751412570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf8030070 con 0x7f6c100c9950 2023-08-25T09:39:00.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.342+0000 7f6c0effd640 1 -- 172.21.15.70:0/751412570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf802cd60 con 0x7f6c100c9950 2023-08-25T09:39:00.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.342+0000 7f03a899e640 1 -- 172.21.15.70:0/3352583841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c02c9f0 con 0x7f03a41598f0 2023-08-25T09:39:00.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.342+0000 7f03a899e640 1 -- 172.21.15.70:0/3352583841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c02cb50 con 0x7f03a41598f0 2023-08-25T09:39:00.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.342+0000 7f03a899e640 1 -- 172.21.15.70:0/3352583841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0343f0 con 0x7f03a41598f0 2023-08-25T09:39:00.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.343+0000 7f6c150d7640 1 -- 172.21.15.70:0/751412570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100c9950 msgr2=0x7f6c100cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.343+0000 7f6c150d7640 1 --2- 172.21.15.70:0/751412570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100c9950 0x7f6c100cd660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c100cbce0 tx=0x7f6bf800b4d0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.343+0000 7f03abc04640 1 -- 172.21.15.70:0/3352583841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a41598f0 msgr2=0x7f03a40aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.343+0000 7f03abc04640 1 --2- 172.21.15.70:0/3352583841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a41598f0 0x7f03a40aff00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f03a40b0ac0 tx=0x7f039c02be50 comp rx=0 tx=0).stop 2023-08-25T09:39:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f6c150d7640 1 -- 172.21.15.70:0/751412570 shutdown_connections 2023-08-25T09:39:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f6c150d7640 1 --2- 172.21.15.70:0/751412570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100c9950 0x7f6c100cd660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f6c150d7640 1 --2- 172.21.15.70:0/751412570 >> v2:172.21.15.155:3300/0 conn(0x7f6c100cb220 0x7f6c100b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f6c150d7640 1 -- 172.21.15.70:0/751412570 >> 172.21.15.70:0/751412570 conn(0x7f6c1006ac60 msgr2=0x7f6c1006b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f03abc04640 1 -- 172.21.15.70:0/3352583841 shutdown_connections 2023-08-25T09:39:00.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f03abc04640 1 --2- 172.21.15.70:0/3352583841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a40b0440 0x7f03a415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f03abc04640 1 --2- 172.21.15.70:0/3352583841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a41598f0 0x7f03a40aff00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f03abc04640 1 --2- 172.21.15.70:0/3352583841 >> v2:172.21.15.155:3300/0 conn(0x7f03a40caf00 0x7f03a40cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f03abc04640 1 -- 172.21.15.70:0/3352583841 >> 172.21.15.70:0/3352583841 conn(0x7f03a4069c40 msgr2=0x7f03a406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f6c150d7640 1 -- 172.21.15.70:0/751412570 shutdown_connections 2023-08-25T09:39:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.344+0000 7f6c150d7640 1 -- 172.21.15.70:0/751412570 wait complete. 2023-08-25T09:39:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.347+0000 7f6c150d7640 1 Processor -- start 2023-08-25T09:39:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.347+0000 7f03abc04640 1 -- 172.21.15.70:0/3352583841 shutdown_connections 2023-08-25T09:39:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.349+0000 7fd882362640 1 Processor -- start 2023-08-25T09:39:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7fd882362640 1 -- start start 2023-08-25T09:39:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.348+0000 7f03abc04640 1 -- 172.21.15.70:0/3352583841 wait complete. 2023-08-25T09:39:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.349+0000 7f6c150d7640 1 -- start start 2023-08-25T09:39:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f03abc04640 1 Processor -- start 2023-08-25T09:39:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.351+0000 7f03abc04640 1 -- start start 2023-08-25T09:39:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f6c150d7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c100c9950 0x7f6c100c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f6c150d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100cb220 0x7f6c100c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f6c150d7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c100cb600 0x7f6c100c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f6c150d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100c7300 con 0x7f6c100cb220 2023-08-25T09:39:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f6c150d7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10213d90 con 0x7f6c100cb600 2023-08-25T09:39:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.350+0000 7f6c150d7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10210a00 con 0x7f6c100c9950 2023-08-25T09:39:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.351+0000 7f6c148d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c100cb600 0x7f6c100c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.351+0000 7f6c148d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c100cb600 0x7f6c100c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38512/0 (socket says 172.21.15.70:38512) 2023-08-25T09:39:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.351+0000 7f6c148d6640 1 -- 172.21.15.70:0/1074160019 learned_addr learned my addr 172.21.15.70:0/1074160019 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7f6c0f7fe640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100cb220 0x7f6c100c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7f6c0ffff640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c100c9950 0x7f6c100c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7fd882362640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8740e11d0 0x7fd874038780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7f03abc04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a40b0440 0x7f03a40af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7f03abc04640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03a40caf00 0x7f03a414a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7f03a99a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a40b0440 0x7f03a40af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f03a99a0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a40b0440 0x7f03a40af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47226/0 (socket says 172.21.15.70:47226) 2023-08-25T09:39:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.352+0000 7fd882362640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd8740e15b0 0x7fd87403aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd882362640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd874055480 con 0x7fd8740542a0 2023-08-25T09:39:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd881360640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8740e11d0 0x7fd874038780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd882362640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd874056920 con 0x7fd8740e11d0 2023-08-25T09:39:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd882362640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd874055290 con 0x7fd8740e15b0 2023-08-25T09:39:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd881360640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8740e11d0 0x7fd874038780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47220/0 (socket says 172.21.15.70:47220) 2023-08-25T09:39:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c100c9950 msgr2=0x7f6c100c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f6c0f7fe640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c100c9950 0x7f6c100c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c100cb600 msgr2=0x7f6c100c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f6c0f7fe640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c100cb600 0x7f6c100c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd881360640 1 -- 172.21.15.70:0/406433269 learned_addr learned my addr 172.21.15.70:0/406433269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f6c0f7fe640 1 -- 172.21.15.70:0/1074160019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf8005fc0 con 0x7f6c100cb220 2023-08-25T09:39:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fd881b61640 1 --2- 172.21.15.70:0/406433269 >> v2:172.21.15.155:3300/0 conn(0x7fd8740e15b0 0x7fd87403aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f03a99a0640 1 -- 172.21.15.70:0/65685545 learned_addr learned my addr 172.21.15.70:0/65685545 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03abc04640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a41598f0 0x7f03a414ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4202560 con 0x7f03a40b0440 2023-08-25T09:39:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4202800 con 0x7f03a40caf00 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4202ae0 con 0x7f03a41598f0 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03a919f640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03a40caf00 0x7f03a414a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03a919f640 1 -- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a41598f0 msgr2=0x7f03a414ad60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03a919f640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a41598f0 0x7f03a414ad60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03a919f640 1 -- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a40b0440 msgr2=0x7f03a40af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03a919f640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a40b0440 0x7f03a40af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f03a919f640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c009b30 con 0x7f03a40caf00 2023-08-25T09:39:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7f6c0f7fe640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100cb220 0x7f6c100c6790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c10066140 tx=0x7f6bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800a020 con 0x7f6c100cb220 2023-08-25T09:39:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf800a1c0 con 0x7f6c100cb220 2023-08-25T09:39:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8003a60 con 0x7f6c100cb220 2023-08-25T09:39:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10212fd0 con 0x7f6c100cb220 2023-08-25T09:39:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c10213590 con 0x7f6c100cb220 2023-08-25T09:39:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7fd881360640 1 -- 172.21.15.70:0/406433269 >> v2:172.21.15.155:3300/0 conn(0x7fd8740e15b0 msgr2=0x7fd87403aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7fd881360640 1 --2- 172.21.15.70:0/406433269 >> v2:172.21.15.155:3300/0 conn(0x7fd8740e15b0 0x7fd87403aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7fd881360640 1 -- 172.21.15.70:0/406433269 >> v1:172.21.15.155:6789/0 conn(0x7fd8740542a0 legacy=0x7fd874051680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7fd881360640 1 -- 172.21.15.70:0/406433269 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd874056540 con 0x7fd8740e11d0 2023-08-25T09:39:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.355+0000 7f03a919f640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03a40caf00 0x7f03a414a820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f039c032000 tx=0x7f039c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fd881360640 1 --2- 172.21.15.70:0/406433269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e11d0 0x7fd874038780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8740043b0 tx=0x7fd87802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33a09608196e305 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c10066fb0 con 0x7f6c100cb220 2023-08-25T09:39:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.353+0000 7fe14aee7640 1 Processor -- start 2023-08-25T09:39:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fd873fff640 1 -- 172.21.15.70:0/406433269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878031890 con 0x7fd8740e11d0 2023-08-25T09:39:00.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fd873fff640 1 -- 172.21.15.70:0/406433269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8780319f0 con 0x7fd8740e11d0 2023-08-25T09:39:00.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fd873fff640 1 -- 172.21.15.70:0/406433269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878031d60 con 0x7fd8740e11d0 2023-08-25T09:39:00.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.354+0000 7fe14aee7640 1 -- start start 2023-08-25T09:39:00.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fe14aee7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe1440aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fe14aee7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe144164750 0x7fe1440b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fe14aee7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440ccab0 con 0x7fe144164370 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fe14aee7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440ccd90 con 0x7fe144164750 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fe14aee7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440cd070 con 0x7fe1440cc050 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.356+0000 7fe143fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe1440aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe143fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe1440aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44834/0 (socket says 172.21.15.70:44834) 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe143fff640 1 -- 172.21.15.70:0/1944585674 learned_addr learned my addr 172.21.15.70:0/1944585674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe149484640 1 --2- 172.21.15.70:0/1944585674 >> v2:172.21.15.70:3300/0 conn(0x7fe144164750 0x7fe1440b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe143fff640 1 -- 172.21.15.70:0/1944585674 >> v2:172.21.15.70:3300/0 conn(0x7fe144164750 msgr2=0x7fe1440b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe143fff640 1 --2- 172.21.15.70:0/1944585674 >> v2:172.21.15.70:3300/0 conn(0x7fe144164750 0x7fe1440b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe143fff640 1 -- 172.21.15.70:0/1944585674 >> v1:172.21.15.155:6789/0 conn(0x7fe144164370 legacy=0x7fe1440cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.357+0000 7fe143fff640 1 -- 172.21.15.70:0/1944585674 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440cd350 con 0x7fe1440cc050 2023-08-25T09:39:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 -- 172.21.15.70:0/406433269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e11d0 msgr2=0x7fd874038780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 --2- 172.21.15.70:0/406433269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e11d0 0x7fd874038780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8740043b0 tx=0x7fd87802b930 comp rx=0 tx=0).stop 2023-08-25T09:39:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fe149484640 1 --2- 172.21.15.70:0/1944585674 >> v2:172.21.15.70:3300/0 conn(0x7fe144164750 0x7fe1440b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 -- 172.21.15.70:0/406433269 shutdown_connections 2023-08-25T09:39:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 --2- 172.21.15.70:0/406433269 >> v2:172.21.15.155:3300/0 conn(0x7fd8740e15b0 0x7fd87403aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 --2- 172.21.15.70:0/406433269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e11d0 0x7fd874038780 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8740043b0 tx=0x7fd87802b930 comp rx=0 tx=0).stop 2023-08-25T09:39:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 -- 172.21.15.70:0/406433269 >> 172.21.15.70:0/406433269 conn(0x7fd874053780 msgr2=0x7fd874053b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 -- 172.21.15.70:0/406433269 shutdown_connections 2023-08-25T09:39:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7fd882362640 1 -- 172.21.15.70:0/406433269 wait complete. 2023-08-25T09:39:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c040670 con 0x7f03a40caf00 2023-08-25T09:39:00.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c0407d0 con 0x7f03a40caf00 2023-08-25T09:39:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c040b80 con 0x7f03a40caf00 2023-08-25T09:39:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a4202dc0 con 0x7f03a40caf00 2023-08-25T09:39:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40b26f0 con 0x7f03a40caf00 2023-08-25T09:39:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.358+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6bf8003bc0 con 0x7f6c100cb220 2023-08-25T09:39:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.360+0000 7fe143fff640 1 --2- 172.21.15.70:0/1944585674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe1440aff00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009e50 tx=0x7fe13c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f5e6e78e3d43bce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.361+0000 7fd882362640 1 Processor -- start 2023-08-25T09:39:00.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.360+0000 7f6c0cff9640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6bf006a050 0x7f6bf006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.360+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6bf80ab130 con 0x7f6c100cb220 2023-08-25T09:39:00.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.361+0000 7f6c0ffff640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6bf006a050 0x7f6bf006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7f6c0ffff640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6bf006a050 0x7f6bf006c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c10089800 tx=0x7f6c08010040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038c005250 con 0x7f03a40caf00 2023-08-25T09:39:00.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.361+0000 7fd882362640 1 -- start start 2023-08-25T09:39:00.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.361+0000 7fe1437fe640 1 -- 172.21.15.70:0/1944585674 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c02cb00 con 0x7fe1440cc050 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.361+0000 7fe1437fe640 1 -- 172.21.15.70:0/1944585674 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c02cc60 con 0x7fe1440cc050 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 -- 172.21.15.70:0/1944585674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 msgr2=0x7fe1440aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 --2- 172.21.15.70:0/1944585674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe1440aff00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009e50 tx=0x7fe13c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 -- 172.21.15.70:0/1944585674 shutdown_connections 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 --2- 172.21.15.70:0/1944585674 >> v2:172.21.15.70:3300/0 conn(0x7fe144164750 0x7fe1440b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 --2- 172.21.15.70:0/1944585674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe1440aff00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 -- 172.21.15.70:0/1944585674 >> 172.21.15.70:0/1944585674 conn(0x7fe144069c40 msgr2=0x7fe14406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.362+0000 7fe14aee7640 1 -- 172.21.15.70:0/1944585674 shutdown_connections 2023-08-25T09:39:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.363+0000 7fd882362640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 0x7fd874051ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fd881360640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 0x7fd874051ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fd881360640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 0x7fd874051ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47234/0 (socket says 172.21.15.70:47234) 2023-08-25T09:39:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fd881360640 1 -- 172.21.15.70:0/3569794207 learned_addr learned my addr 172.21.15.70:0/3569794207 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd87404eee0 0x7fd8740dca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fa95d767640 1 Processor -- start 2023-08-25T09:39:00.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.363+0000 7fe14aee7640 1 -- 172.21.15.70:0/1944585674 wait complete. 2023-08-25T09:39:00.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd874037710 0x7fd874037ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.364+0000 7fa95d767640 1 -- start start 2023-08-25T09:39:00.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740372f0 con 0x7fd8740e15b0 2023-08-25T09:39:00.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fd880b5f640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd87404eee0 0x7fd8740dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fe14aee7640 1 Processor -- start 2023-08-25T09:39:00.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd874036d00 con 0x7fd87404eee0 2023-08-25T09:39:00.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fd881b61640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd874037710 0x7fd874037ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd874038a60 con 0x7fd874037710 2023-08-25T09:39:00.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95d767640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9581610b0 0x7fa9580b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95d767640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa9580c1450 0x7fa9580ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95d767640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa9580cea70 0x7fa9580caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95d767640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9580cef00 con 0x7fa9580cea70 2023-08-25T09:39:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95d767640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9580b15a0 con 0x7fa9580c1450 2023-08-25T09:39:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95d767640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9580c1cd0 con 0x7fa9581610b0 2023-08-25T09:39:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6bf8076940 con 0x7f6c100cb220 2023-08-25T09:39:00.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95cf66640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa9580cea70 0x7fa9580caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95cf66640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa9580cea70 0x7fa9580caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47236/0 (socket says 172.21.15.70:47236) 2023-08-25T09:39:00.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fa95cf66640 1 -- 172.21.15.70:0/1292650672 learned_addr learned my addr 172.21.15.70:0/1292650672 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fd881360640 1 -- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd874037710 msgr2=0x7fd874037ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fd881360640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd874037710 0x7fd874037ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fd881360640 1 -- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd87404eee0 msgr2=0x7fd8740dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fd881360640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd87404eee0 0x7fd8740dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.366+0000 7fd881360640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780094d0 con 0x7fd8740e15b0 2023-08-25T09:39:00.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.365+0000 7fe14aee7640 1 -- start start 2023-08-25T09:39:00.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fe14aee7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe144151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa9577fe640 1 --2- 172.21.15.70:0/1292650672 >> v2:172.21.15.155:3300/0 conn(0x7fa9580c1450 0x7fa9580ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fe14aee7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe144164370 0x7fe1440affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa95cf66640 1 -- 172.21.15.70:0/1292650672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9581610b0 msgr2=0x7fa9580b3f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T09:39:00.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa95cf66640 1 --2- 172.21.15.70:0/1292650672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9581610b0 0x7fa9580b3f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa95cf66640 1 -- 172.21.15.70:0/1292650672 >> v2:172.21.15.155:3300/0 conn(0x7fa9580c1450 msgr2=0x7fa9580ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa95cf66640 1 --2- 172.21.15.70:0/1292650672 >> v2:172.21.15.155:3300/0 conn(0x7fa9580c1450 0x7fa9580ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa95cf66640 1 -- 172.21.15.70:0/1292650672 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9580cc310 con 0x7fa9580cea70 2023-08-25T09:39:00.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fa9577fe640 1 --2- 172.21.15.70:0/1292650672 >> v2:172.21.15.155:3300/0 conn(0x7fa9580c1450 0x7fa9580ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:00.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fe148c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe144151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fe148c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe144151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44852/0 (socket says 172.21.15.70:44852) 2023-08-25T09:39:00.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fd881360640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 0x7fd874051ec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8780090b0 tx=0x7fd8780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fd880b5f640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd87404eee0 0x7fd8740dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:00.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fd881b61640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd874037710 0x7fd874037ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:00.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.367+0000 7fe143fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe144164370 0x7fe1440affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fe143fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe144164370 0x7fe1440affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47242/0 (socket says 172.21.15.70:47242) 2023-08-25T09:39:00.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fe148c83640 1 -- 172.21.15.70:0/965033180 learned_addr learned my addr 172.21.15.70:0/965033180 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144164750 0x7fe1440ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440afba0 con 0x7fe144164370 2023-08-25T09:39:00.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440aef20 con 0x7fe144164750 2023-08-25T09:39:00.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440af200 con 0x7fe1440cc050 2023-08-25T09:39:00.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878003a20 con 0x7fd8740e15b0 2023-08-25T09:39:00.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87803f070 con 0x7fd8740e15b0 2023-08-25T09:39:00.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878003d80 con 0x7fd8740e15b0 2023-08-25T09:39:00.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd874038cb0 con 0x7fd8740e15b0 2023-08-25T09:39:00.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.368+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd874036f10 con 0x7fd8740e15b0 2023-08-25T09:39:00.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.369+0000 7fa95cf66640 1 --2- 172.21.15.70:0/1292650672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa9580caf60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa95806cda0 tx=0x7fa95002bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d233d22171e47 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.369+0000 7fa956ffd640 1 -- 172.21.15.70:0/1292650672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95002caa0 con 0x7fa9580cea70 2023-08-25T09:39:00.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.369+0000 7fa956ffd640 1 -- 172.21.15.70:0/1292650672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95002cc00 con 0x7fa9580cea70 2023-08-25T09:39:00.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.369+0000 7fa956ffd640 1 -- 172.21.15.70:0/1292650672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9500343f0 con 0x7fa9580cea70 2023-08-25T09:39:00.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fe148c83640 1 -- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144164750 msgr2=0x7fe1440ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fe148c83640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144164750 0x7fe1440ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fe148c83640 1 -- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe144164370 msgr2=0x7fe1440affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fe148c83640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe144164370 0x7fe1440affa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fe148c83640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13c009b30 con 0x7fe1440cc050 2023-08-25T09:39:00.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f039c040ce0 con 0x7f03a40caf00 2023-08-25T09:39:00.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fb953de8640 1 Processor -- start 2023-08-25T09:39:00.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fa95d767640 1 -- 172.21.15.70:0/1292650672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 msgr2=0x7fa9580caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.370+0000 7fa95d767640 1 --2- 172.21.15.70:0/1292650672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa9580caf60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa95806cda0 tx=0x7fa95002bf00 comp rx=0 tx=0).stop 2023-08-25T09:39:00.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fb953de8640 1 -- start start 2023-08-25T09:39:00.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 -- 172.21.15.70:0/1292650672 shutdown_connections 2023-08-25T09:39:00.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 --2- 172.21.15.70:0/1292650672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa9580caf60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 --2- 172.21.15.70:0/1292650672 >> v2:172.21.15.155:3300/0 conn(0x7fa9580c1450 0x7fa9580ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 --2- 172.21.15.70:0/1292650672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9581610b0 0x7fa9580b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 -- 172.21.15.70:0/1292650672 >> 172.21.15.70:0/1292650672 conn(0x7fa958069eb0 msgr2=0x7fa9580b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fe148c83640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe144151a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe13800c430 tx=0x7fe13800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 -- 172.21.15.70:0/1292650672 shutdown_connections 2023-08-25T09:39:00.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7fa95d767640 1 -- 172.21.15.70:0/1292650672 wait complete. 2023-08-25T09:39:00.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7f039a7fc640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f038806a0a0 0x7f038806c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.371+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f039c0ab6e0 con 0x7f03a40caf00 2023-08-25T09:39:00.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7f03a99a0640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f038806a0a0 0x7f038806c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138011790 con 0x7fe1440cc050 2023-08-25T09:39:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1380118f0 con 0x7fe1440cc050 2023-08-25T09:39:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138011ce0 con 0x7fe1440cc050 2023-08-25T09:39:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440af4e0 con 0x7fe1440cc050 2023-08-25T09:39:00.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fb953de8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c15c8b0 0x7fb94c06ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fb953de8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb94c0aff00 0x7fb94c0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fb953de8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb94c0ca470 0x7fb94c0ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fb953de8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0cc050 con 0x7fb94c0ca470 2023-08-25T09:39:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fb953de8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c0b08a0 con 0x7fb94c0aff00 2023-08-25T09:39:00.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.372+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14414fe40 con 0x7fe1440cc050 2023-08-25T09:39:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.373+0000 7fb953de8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0cae20 con 0x7fb94c15c8b0 2023-08-25T09:39:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.373+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874037430 con 0x7fd8740e15b0 2023-08-25T09:39:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.373+0000 7fb951b84640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c15c8b0 0x7fb94c06ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.373+0000 7fb951383640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb94c0aff00 0x7fb94c0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.373+0000 7fb951383640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb94c0aff00 0x7fb94c0b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38576/0 (socket says 172.21.15.70:38576) 2023-08-25T09:39:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fb951383640 1 -- 172.21.15.70:0/2323087140 learned_addr learned my addr 172.21.15.70:0/2323087140 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.373+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd878046480 con 0x7fd8740e15b0 2023-08-25T09:39:00.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fa95d767640 1 Processor -- start 2023-08-25T09:39:00.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7f03a99a0640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f038806a0a0 0x7f038806c560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009c00 tx=0x7f03a0009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fb951383640 1 -- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c15c8b0 msgr2=0x7fb94c06ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fb951383640 1 --2- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c15c8b0 0x7fb94c06ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fb951383640 1 -- 172.21.15.70:0/2323087140 >> v2:172.21.15.70:3300/0 conn(0x7fb94c0ca470 msgr2=0x7fb94c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fb951383640 1 --2- 172.21.15.70:0/2323087140 >> v2:172.21.15.70:3300/0 conn(0x7fb94c0ca470 0x7fb94c0ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fb951383640 1 -- 172.21.15.70:0/2323087140 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0cc260 con 0x7fb94c0aff00 2023-08-25T09:39:00.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.375+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe144066fb0 con 0x7fe1440cc050 2023-08-25T09:39:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.374+0000 7fa95d767640 1 -- start start 2023-08-25T09:39:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.375+0000 7fd871ffb640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd86006a0a0 0x7fd86006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.375+0000 7fa95d767640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9580c1450 0x7fa958071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.375+0000 7fd880b5f640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd86006a0a0 0x7fd86006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.375+0000 7fa95d767640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa95806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa95d767640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581610b0 0x7fa95806b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd878041070 con 0x7fd8740e15b0 2023-08-25T09:39:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fd880b5f640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd86006a0a0 0x7fd86006c560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd86c007fd0 tx=0x7fd86c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.377+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe13801c460 con 0x7fe1440cc050 2023-08-25T09:39:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa95d767640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa958071a70 con 0x7fa9580cea70 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa95d767640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95806f530 con 0x7fa9581610b0 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa95d767640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95806f810 con 0x7fa9580c1450 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa95806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa95806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47254/0 (socket says 172.21.15.70:47254) 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 -- 172.21.15.70:0/4227438147 learned_addr learned my addr 172.21.15.70:0/4227438147 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa957fff640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9580c1450 0x7fa958071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 -- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9580c1450 msgr2=0x7fa958071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9580c1450 0x7fa958071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 -- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581610b0 msgr2=0x7fa95806b200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T09:39:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581610b0 0x7fa95806b200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fa9577fe640 1 -- 172.21.15.70:0/4227438147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa950009b30 con 0x7fa9580cea70 2023-08-25T09:39:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.377+0000 7fa9577fe640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa95806eab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00d7d0 tx=0x7fa94c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.376+0000 7fb951383640 1 --2- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0aff00 0x7fb94c0b0360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb94c06bc60 tx=0x7fb94402bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47708f163fc2ee3c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.378+0000 7fe1417fa640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe13006a0a0 0x7fe13006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.378+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1380883c0 con 0x7fe1440cc050 2023-08-25T09:39:00.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.378+0000 7fe143fff640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe13006a0a0 0x7fe13006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.379+0000 7fe143fff640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe13006a0a0 0x7fe13006c560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe13c035000 tx=0x7fe13c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.379+0000 7f271982d640 1 Processor -- start 2023-08-25T09:39:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.380+0000 7f271982d640 1 -- start start 2023-08-25T09:39:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f271982d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27140cffb0 0x7f27140cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f271982d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27140d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f271982d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271415cfb0 con 0x7f271415cbd0 2023-08-25T09:39:00.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f271982d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27140cd4e0 con 0x7f27140cffb0 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f2712ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f271415cbd0 0x7f27141606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35450/0 (socket says 172.21.15.70:35450) 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f2712ffd640 1 -- 172.21.15.70:0/57988599 learned_addr learned my addr 172.21.15.70:0/57988599 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.381+0000 7f271982d640 1 -- 172.21.15.70:0/57988599 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2714160f10 con 0x7f27140cdeb0 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.382+0000 7f27137fe640 1 --2- 172.21.15.70:0/57988599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27140d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.382+0000 7f27127fc640 1 --2- 172.21.15.70:0/57988599 >> v2:172.21.15.155:3300/0 conn(0x7f27140cffb0 0x7f27140cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.382+0000 7f27137fe640 1 -- 172.21.15.70:0/57988599 >> v2:172.21.15.155:3300/0 conn(0x7f27140cffb0 msgr2=0x7f27140cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.382+0000 7f27137fe640 1 --2- 172.21.15.70:0/57988599 >> v2:172.21.15.155:3300/0 conn(0x7f27140cffb0 0x7f27140cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.382+0000 7f27137fe640 1 -- 172.21.15.70:0/57988599 >> v1:172.21.15.70:6789/0 conn(0x7f271415cbd0 legacy=0x7f27141606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.382+0000 7f27137fe640 1 -- 172.21.15.70:0/57988599 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27141611f0 con 0x7f27140cdeb0 2023-08-25T09:39:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.383+0000 7ffbc7fff640 1 Processor -- start 2023-08-25T09:39:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.383+0000 7ffbc7fff640 1 -- start start 2023-08-25T09:39:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.384+0000 7f27137fe640 1 --2- 172.21.15.70:0/57988599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27140d2a00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f270c009d90 tx=0x7f270c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4288995722b992 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7ffbc7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbc815a8c0 0x7ffbc80cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004030 con 0x7fa9580cea70 2023-08-25T09:39:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c004190 con 0x7fa9580cea70 2023-08-25T09:39:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c004520 con 0x7fa9580cea70 2023-08-25T09:39:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95806faf0 con 0x7fa9580cea70 2023-08-25T09:39:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7ffbc7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffbc80cea20 0x7ffbc80cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9580700b0 con 0x7fa9580cea70 2023-08-25T09:39:00.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd874037430 con 0x7fd8740e15b0 2023-08-25T09:39:00.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7ffbc7fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc806f500 con 0x7ffbc80b6090 2023-08-25T09:39:00.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7ffbc7fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc815aca0 con 0x7ffbc815a8c0 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.385+0000 7ffbc7fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc80d0480 con 0x7ffbc80cea20 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.386+0000 7ffbc6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbc815a8c0 0x7ffbc80cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.386+0000 7ffbc6ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbc815a8c0 0x7ffbc80cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47258/0 (socket says 172.21.15.70:47258) 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.386+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/4032608976 learned_addr learned my addr 172.21.15.70:0/4032608976 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.386+0000 7ffbc77fe640 1 --2- 172.21.15.70:0/4032608976 >> v2:172.21.15.155:3300/0 conn(0x7ffbc80cea20 0x7ffbc80cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.387+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/4032608976 >> v2:172.21.15.155:3300/0 conn(0x7ffbc80cea20 msgr2=0x7ffbc80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.387+0000 7ffbc6ffd640 1 --2- 172.21.15.70:0/4032608976 >> v2:172.21.15.155:3300/0 conn(0x7ffbc80cea20 0x7ffbc80cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.387+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/4032608976 >> v1:172.21.15.70:6789/0 conn(0x7ffbc80b6090 legacy=0x7ffbc80ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.387+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/4032608976 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc806e680 con 0x7ffbc815a8c0 2023-08-25T09:39:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.387+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa958066fb0 con 0x7fa9580cea70 2023-08-25T09:39:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.388+0000 7ffbc6ffd640 1 --2- 172.21.15.70:0/4032608976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc815a8c0 0x7ffbc80cff40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80cdfb0 tx=0x7ffbb802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6e8d0c60258c46 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.388+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa94c0096a0 con 0x7fa9580cea70 2023-08-25T09:39:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.388+0000 7ffbc5ffb640 1 -- 172.21.15.70:0/4032608976 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8004230 con 0x7ffbc815a8c0 2023-08-25T09:39:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7ffbc5ffb640 1 -- 172.21.15.70:0/4032608976 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb8004390 con 0x7ffbc815a8c0 2023-08-25T09:39:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7ffbc5ffb640 1 -- 172.21.15.70:0/4032608976 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8030840 con 0x7ffbc815a8c0 2023-08-25T09:39:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7fa954ff9640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa94406a0d0 0x7fa94406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa94c0886c0 con 0x7fa9580cea70 2023-08-25T09:39:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7fa957fff640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa94406a0d0 0x7fa94406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7ffbc7fff640 1 -- 172.21.15.70:0/4032608976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc815a8c0 msgr2=0x7ffbc80cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.389+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/4032608976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc815a8c0 0x7ffbc80cff40 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80cdfb0 tx=0x7ffbb802b980 comp rx=0 tx=0).stop 2023-08-25T09:39:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7ffbc7fff640 1 -- 172.21.15.70:0/4032608976 shutdown_connections 2023-08-25T09:39:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/4032608976 >> v2:172.21.15.155:3300/0 conn(0x7ffbc80cea20 0x7ffbc80cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/4032608976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc815a8c0 0x7ffbc80cff40 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7ffbc7fff640 1 -- 172.21.15.70:0/4032608976 >> 172.21.15.70:0/4032608976 conn(0x7ffbc806ce70 msgr2=0x7ffbc806d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f039c076eb0 con 0x7f03a40caf00 2023-08-25T09:39:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7fa957fff640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa94406a0d0 0x7fa94406c590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa948006fd0 tx=0x7fa948008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7fb950b82640 1 -- 172.21.15.70:0/2323087140 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94402caa0 con 0x7fb94c0aff00 2023-08-25T09:39:00.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7fb950b82640 1 -- 172.21.15.70:0/2323087140 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94402cc00 con 0x7fb94c0aff00 2023-08-25T09:39:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.390+0000 7fb950b82640 1 -- 172.21.15.70:0/2323087140 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9440343f0 con 0x7fb94c0aff00 2023-08-25T09:39:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7ffbc7fff640 1 -- 172.21.15.70:0/4032608976 shutdown_connections 2023-08-25T09:39:00.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7ffbc7fff640 1 -- 172.21.15.70:0/4032608976 wait complete. 2023-08-25T09:39:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 -- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0aff00 msgr2=0x7fb94c0b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 --2- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0aff00 0x7fb94c0b0360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb94c06bc60 tx=0x7fb94402bf00 comp rx=0 tx=0).stop 2023-08-25T09:39:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 -- 172.21.15.70:0/2323087140 shutdown_connections 2023-08-25T09:39:00.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 --2- 172.21.15.70:0/2323087140 >> v2:172.21.15.70:3300/0 conn(0x7fb94c0ca470 0x7fb94c0ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 --2- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0aff00 0x7fb94c0b0360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 --2- 172.21.15.70:0/2323087140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c15c8b0 0x7fb94c06ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.391+0000 7fb953de8640 1 -- 172.21.15.70:0/2323087140 >> 172.21.15.70:0/2323087140 conn(0x7fb94c069c40 msgr2=0x7fb94c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.392+0000 7fb953de8640 1 -- 172.21.15.70:0/2323087140 shutdown_connections 2023-08-25T09:39:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.392+0000 7fb953de8640 1 -- 172.21.15.70:0/2323087140 wait complete. 2023-08-25T09:39:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.392+0000 7ffbc7fff640 1 Processor -- start 2023-08-25T09:39:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.393+0000 7ffbc7fff640 1 -- start start 2023-08-25T09:39:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 0x7ffbc80c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7fb953de8640 1 Processor -- start 2023-08-25T09:39:00.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 0x7ffbc80c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 0x7ffbc80c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47266/0 (socket says 172.21.15.70:47266) 2023-08-25T09:39:00.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7fb953de8640 1 -- start start 2023-08-25T09:39:00.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/3381517370 learned_addr learned my addr 172.21.15.70:0/3381517370 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbc80cea20 0x7ffbc815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbc815a8c0 0x7ffbc815b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80c8020 con 0x7ffbc80b6090 2023-08-25T09:39:00.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80c6680 con 0x7ffbc80cea20 2023-08-25T09:39:00.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.394+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8158cd0 con 0x7ffbc815a8c0 2023-08-25T09:39:00.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc67fc640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbc80cea20 0x7ffbc815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc77fe640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbc815a8c0 0x7ffbc815b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbc815a8c0 msgr2=0x7ffbc815b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7fb953de8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c0aff00 0x7fb94c1f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc6ffd640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbc815a8c0 0x7ffbc815b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbc80cea20 msgr2=0x7ffbc815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc6ffd640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbc80cea20 0x7ffbc815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc6ffd640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb8009460 con 0x7ffbc80b6090 2023-08-25T09:39:00.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7fb953de8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0ca470 0x7fb94c0c2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc77fe640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbc815a8c0 0x7ffbc815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:00.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7fb953de8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 0x7fb94c1518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb953de8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1f6290 con 0x7fb94c15c8b0 2023-08-25T09:39:00.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb953de8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0c29b0 con 0x7fb94c0ca470 2023-08-25T09:39:00.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.395+0000 7ffbc6ffd640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 0x7ffbc80c7ae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009940 tx=0x7ffbb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb953de8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c15ed20 con 0x7fb94c0aff00 2023-08-25T09:39:00.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 0x7fb94c1518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 0x7fb94c1518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47276/0 (socket says 172.21.15.70:47276) 2023-08-25T09:39:00.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 -- 172.21.15.70:0/1278138717 learned_addr learned my addr 172.21.15.70:0/1278138717 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb951b84640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c0aff00 0x7fb94c1f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb951383640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0ca470 0x7fb94c0c2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 -- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c0aff00 msgr2=0x7fb94c1f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c0aff00 0x7fb94c1f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 -- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0ca470 msgr2=0x7fb94c0c2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0ca470 0x7fb94c0c2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb952385640 1 -- 172.21.15.70:0/1278138717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944009b30 con 0x7fb94c15c8b0 2023-08-25T09:39:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb951b84640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c0aff00 0x7fb94c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.396+0000 7fb951383640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0ca470 0x7fb94c0c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.397+0000 7fb952385640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 0x7fb94c1518c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009fd0 tx=0x7fb93c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb803d070 con 0x7ffbc80b6090 2023-08-25T09:39:00.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb8004500 con 0x7ffbc80b6090 2023-08-25T09:39:00.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc80cac90 con 0x7ffbc80b6090 2023-08-25T09:39:00.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c011830 con 0x7fb94c15c8b0 2023-08-25T09:39:00.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa94c053d90 con 0x7fa9580cea70 2023-08-25T09:39:00.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc80c6030 con 0x7ffbc80b6090 2023-08-25T09:39:00.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.400+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8030e00 con 0x7ffbc80b6090 2023-08-25T09:39:00.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.398+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0b14f0 con 0x7fb94c15c8b0 2023-08-25T09:39:00.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.400+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c0b1ab0 con 0x7fb94c15c8b0 2023-08-25T09:39:00.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.401+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe138053b20 con 0x7fe1440cc050 2023-08-25T09:39:00.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.400+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c011990 con 0x7fb94c15c8b0 2023-08-25T09:39:00.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.400+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c011d80 con 0x7fb94c15c8b0 2023-08-25T09:39:00.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.401+0000 7f2711ffb640 1 -- 172.21.15.70:0/57988599 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c02cc50 con 0x7f27140cdeb0 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.401+0000 7f2711ffb640 1 -- 172.21.15.70:0/57988599 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c02cdb0 con 0x7f27140cdeb0 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.401+0000 7f2711ffb640 1 -- 172.21.15.70:0/57988599 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c034660 con 0x7f27140cdeb0 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 -- 172.21.15.70:0/57988599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 msgr2=0x7f27140d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 --2- 172.21.15.70:0/57988599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27140d2a00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f270c009d90 tx=0x7f270c02bea0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 -- 172.21.15.70:0/57988599 shutdown_connections 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 --2- 172.21.15.70:0/57988599 >> v2:172.21.15.155:3300/0 conn(0x7f27140cffb0 0x7f27140cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 --2- 172.21.15.70:0/57988599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27140d2a00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 -- 172.21.15.70:0/57988599 >> 172.21.15.70:0/57988599 conn(0x7f2714070260 msgr2=0x7f2714070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 -- 172.21.15.70:0/57988599 shutdown_connections 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.402+0000 7f271982d640 1 -- 172.21.15.70:0/57988599 wait complete. 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.404+0000 7f271982d640 1 Processor -- start 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.404+0000 7f271982d640 1 -- start start 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.405+0000 7f271982d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27141f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.405+0000 7f271982d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 0x7f27140cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.405+0000 7f271982d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271415cbd0 0x7f27140d0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f271982d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140cac50 con 0x7f27140cffb0 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f271982d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27140ca040 con 0x7f271415cbd0 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f271982d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27140ca320 con 0x7f27140cdeb0 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f27127fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 0x7f27140cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f27127fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 0x7f27140cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47290/0 (socket says 172.21.15.70:47290) 2023-08-25T09:39:00.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f27127fc640 1 -- 172.21.15.70:0/1963497081 learned_addr learned my addr 172.21.15.70:0/1963497081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:00.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7f2712ffd640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27141f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.405+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb93c010380 con 0x7fb94c15c8b0 2023-08-25T09:39:00.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7fb9427fc640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb93006a080 0x7fb93006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.406+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb93c088320 con 0x7fb94c15c8b0 2023-08-25T09:39:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27137fe640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271415cbd0 0x7f27140d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27127fc640 1 -- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 msgr2=0x7f27141f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27127fc640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27141f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27127fc640 1 -- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271415cbd0 msgr2=0x7f27140d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27127fc640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271415cbd0 0x7f27140d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27127fc640 1 -- 172.21.15.70:0/1963497081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c009a70 con 0x7f27140cffb0 2023-08-25T09:39:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27137fe640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271415cbd0 0x7f27140d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7fb951b84640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb93006a080 0x7fb93006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.407+0000 7f27127fc640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 0x7f27140cb8e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f270800e9b0 tx=0x7f270800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.408+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708009670 con 0x7f27140cffb0 2023-08-25T09:39:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.408+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270800c9a0 con 0x7f27140cffb0 2023-08-25T09:39:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.408+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270800cd50 con 0x7f27140cffb0 2023-08-25T09:39:00.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.408+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27140ca600 con 0x7f27140cffb0 2023-08-25T09:39:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.408+0000 7fb951b84640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb93006a080 0x7fb93006c540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb94c15fdb0 tx=0x7fb948008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.408+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27140cbe20 con 0x7f27140cffb0 2023-08-25T09:39:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.410+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2714151830 con 0x7f27140cffb0 2023-08-25T09:39:00.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.411+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc8066fb0 con 0x7ffbc80b6090 2023-08-25T09:39:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.413+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2708013470 con 0x7f27140cffb0 2023-08-25T09:39:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.413+0000 7f26fbfff640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f270006a080 0x7f270006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.413+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2708088790 con 0x7f27140cffb0 2023-08-25T09:39:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.414+0000 7f2712ffd640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f270006a080 0x7f270006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.415+0000 7f2712ffd640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f270006a080 0x7f270006c540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2704005880 tx=0x7f2704009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.413+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffbb8003610 con 0x7ffbc80b6090 2023-08-25T09:39:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.415+0000 7ffbaf7fe640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb406a050 0x7ffbb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:00.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.415+0000 7ffbc67fc640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb406a050 0x7ffbb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:00.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.416+0000 7ffbc67fc640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb406a050 0x7ffbb406c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8159ed0 tx=0x7ffbbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:00.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.417+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbb807d7c0 con 0x7ffbc80b6090 2023-08-25T09:39:00.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.417+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb934005250 con 0x7fb94c15c8b0 2023-08-25T09:39:00.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.422+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2714151830 con 0x7f27140cffb0 2023-08-25T09:39:00.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.440+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffbb8075df0 con 0x7ffbc80b6090 2023-08-25T09:39:00.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.449+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb93c053990 con 0x7fb94c15c8b0 2023-08-25T09:39:00.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.778+0000 7f03abc04640 1 -- 172.21.15.70:0/65685545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f038c0057d0 con 0x7f03a40caf00 2023-08-25T09:39:00.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.779+0000 7f039a7fc640 1 -- 172.21.15.70:0/65685545 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f039c07a4d0 con 0x7f03a40caf00 2023-08-25T09:39:00.784 INFO:teuthology.orchestra.run.smithi070.stdout:55834574853 2023-08-25T09:39:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.779+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c10071430 con 0x7f6c100cb220 2023-08-25T09:39:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.780+0000 7f6c0cff9640 1 -- 172.21.15.70:0/1074160019 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6bf8079f60 con 0x7f6c100cb220 2023-08-25T09:39:00.785 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f0387fff640 1 -- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f038806a0a0 msgr2=0x7f038806c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f0387fff640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f038806a0a0 0x7f038806c560 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009c00 tx=0x7f03a0009190 comp rx=0 tx=0).stop 2023-08-25T09:39:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6bf006a050 msgr2=0x7f6bf006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6bf006a050 0x7f6bf006c510 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6c10089800 tx=0x7f6c08010040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100cb220 msgr2=0x7f6c100c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100cb220 0x7f6c100c6790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c10066140 tx=0x7f6bf80046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 shutdown_connections 2023-08-25T09:39:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6bf006a050 0x7f6bf006c510 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c100cb600 0x7f6c100c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c100cb220 0x7f6c100c6790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 --2- 172.21.15.70:0/1074160019 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c100c9950 0x7f6c100c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.783+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 >> 172.21.15.70:0/1074160019 conn(0x7f6c1006ac60 msgr2=0x7f6c1006eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.784+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 shutdown_connections 2023-08-25T09:39:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.784+0000 7f6c150d7640 1 -- 172.21.15.70:0/1074160019 wait complete. 2023-08-25T09:39:00.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.784+0000 7f0387fff640 1 -- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03a40caf00 msgr2=0x7f03a414a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.784+0000 7f0387fff640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03a40caf00 0x7f03a414a820 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f039c032000 tx=0x7f039c004a40 comp rx=0 tx=0).stop 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.784+0000 7f0387fff640 1 -- 172.21.15.70:0/65685545 shutdown_connections 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.784+0000 7f0387fff640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f038806a0a0 0x7f038806c560 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.785+0000 7f0387fff640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f03a41598f0 0x7f03a414ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.785+0000 7f0387fff640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f03a40caf00 0x7f03a414a820 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.785+0000 7f0387fff640 1 --2- 172.21.15.70:0/65685545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f03a40b0440 0x7f03a40af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.785+0000 7f0387fff640 1 -- 172.21.15.70:0/65685545 >> 172.21.15.70:0/65685545 conn(0x7f03a4069c40 msgr2=0x7f03a406ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.785+0000 7f0387fff640 1 -- 172.21.15.70:0/65685545 shutdown_connections 2023-08-25T09:39:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.786+0000 7f0387fff640 1 -- 172.21.15.70:0/65685545 wait complete. 2023-08-25T09:39:00.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.793+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe14406a430 con 0x7fe1440cc050 2023-08-25T09:39:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.794+0000 7fe1417fa640 1 -- 172.21.15.70:0/965033180 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe138057140 con 0x7fe1440cc050 2023-08-25T09:39:00.797 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.798+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe13006a0a0 msgr2=0x7fe13006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.798+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe13006a0a0 0x7fe13006c560 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe13c035000 tx=0x7fe13c030000 comp rx=0 tx=0).stop 2023-08-25T09:39:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.799+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 msgr2=0x7fe144151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.799+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe144151a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe13800c430 tx=0x7fe13800c8f0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.799+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 shutdown_connections 2023-08-25T09:39:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.799+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe13006a0a0 0x7fe13006c560 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.800+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe144164750 0x7fe1440ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.800+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe144164370 0x7fe1440affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.800+0000 7fe14aee7640 1 --2- 172.21.15.70:0/965033180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1440cc050 0x7fe144151a40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.800+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 >> 172.21.15.70:0/965033180 conn(0x7fe144069c40 msgr2=0x7fe144155c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.800+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 shutdown_connections 2023-08-25T09:39:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.801+0000 7fe14aee7640 1 -- 172.21.15.70:0/965033180 wait complete. 2023-08-25T09:39:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.802+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd87400ad50 con 0x7fd8740e15b0 2023-08-25T09:39:00.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.803+0000 7fd871ffb640 1 -- 172.21.15.70:0/3569794207 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd87400ad50 con 0x7fd8740e15b0 2023-08-25T09:39:00.805 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.809+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa9580c8ce0 con 0x7fa9580cea70 2023-08-25T09:39:00.812 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2023-08-25T09:39:00.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.809+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd86006a0a0 msgr2=0x7fd86006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.809+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd86006a0a0 0x7fd86006c560 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd86c007fd0 tx=0x7fd86c009040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.809+0000 7fa954ff9640 1 -- 172.21.15.70:0/4227438147 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa94c0573b0 con 0x7fa9580cea70 2023-08-25T09:39:00.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 msgr2=0x7fd874051ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 0x7fd874051ec0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8780090b0 tx=0x7fd8780046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 shutdown_connections 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd86006a0a0 0x7fd86006c560 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd874037710 0x7fd874037ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd87404eee0 0x7fd8740dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 --2- 172.21.15.70:0/3569794207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8740e15b0 0x7fd874051ec0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 >> 172.21.15.70:0/3569794207 conn(0x7fd874053780 msgr2=0x7fd8740e03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.810+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 shutdown_connections 2023-08-25T09:39:00.815 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.811+0000 7fd882362640 1 -- 172.21.15.70:0/3569794207 wait complete. 2023-08-25T09:39:00.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.812+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa94406a0d0 msgr2=0x7fa94406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.813+0000 7fa95d767640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa94406a0d0 0x7fa94406c590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa948006fd0 tx=0x7fa948008040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.813+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 msgr2=0x7fa95806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.813+0000 7fa95d767640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa95806eab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00d7d0 tx=0x7fa94c00dc90 comp rx=0 tx=0).stop 2023-08-25T09:39:00.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.2 2023-08-25T09:39:00.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.814+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 shutdown_connections 2023-08-25T09:39:00.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.814+0000 7fa95d767640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa94406a0d0 0x7fa94406c590 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.814+0000 7fa95d767640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9581610b0 0x7fa95806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.814+0000 7fa95d767640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa9580cea70 0x7fa95806eab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.814+0000 7fa95d767640 1 --2- 172.21.15.70:0/4227438147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa9580c1450 0x7fa958071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.814+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 >> 172.21.15.70:0/4227438147 conn(0x7fa958069eb0 msgr2=0x7fa9580ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.816+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 shutdown_connections 2023-08-25T09:39:00.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.816+0000 7fa95d767640 1 -- 172.21.15.70:0/4227438147 wait complete. 2023-08-25T09:39:00.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.824+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2714151830 con 0x7f27140cffb0 2023-08-25T09:39:00.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.825+0000 7f26fbfff640 1 -- 172.21.15.70:0/1963497081 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2714151830 con 0x7f27140cffb0 2023-08-25T09:39:00.827 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f270006a080 msgr2=0x7f270006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f270006a080 0x7f270006c540 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2704005880 tx=0x7f2704009040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 msgr2=0x7f27140cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 0x7f27140cb8e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f270800e9b0 tx=0x7f270800ee70 comp rx=0 tx=0).stop 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 shutdown_connections 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f270006a080 0x7f270006c540 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f271415cbd0 0x7f27140d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27140cffb0 0x7f27140cb8e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 --2- 172.21.15.70:0/1963497081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27140cdeb0 0x7f27141f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 >> 172.21.15.70:0/1963497081 conn(0x7f2714070260 msgr2=0x7f271415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.827+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 shutdown_connections 2023-08-25T09:39:00.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.828+0000 7f271982d640 1 -- 172.21.15.70:0/1963497081 wait complete. 2023-08-25T09:39:00.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.4 2023-08-25T09:39:00.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.1 2023-08-25T09:39:00.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.5 2023-08-25T09:39:00.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.843+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ffbc8065a50 con 0x7ffbc80b6090 2023-08-25T09:39:00.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.844+0000 7ffbaf7fe640 1 -- 172.21.15.70:0/3381517370 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffbb8079410 con 0x7ffbc80b6090 2023-08-25T09:39:00.846 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2023-08-25T09:39:00.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb406a050 msgr2=0x7ffbb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb406a050 0x7ffbb406c510 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8159ed0 tx=0x7ffbbc009040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 msgr2=0x7ffbc80c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 0x7ffbc80c7ae0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8009940 tx=0x7ffbb80046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 shutdown_connections 2023-08-25T09:39:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb406a050 0x7ffbb406c510 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbc815a8c0 0x7ffbc815b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbc80cea20 0x7ffbc815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3381517370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbc80b6090 0x7ffbc80c7ae0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 >> 172.21.15.70:0/3381517370 conn(0x7ffbc806ce70 msgr2=0x7ffbc80cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 shutdown_connections 2023-08-25T09:39:00.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.846+0000 7ffbc7fff640 1 -- 172.21.15.70:0/3381517370 wait complete. 2023-08-25T09:39:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.853+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb934005500 con 0x7fb94c15c8b0 2023-08-25T09:39:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.853+0000 7fb9427fc640 1 -- 172.21.15.70:0/1278138717 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb93c056fb0 con 0x7fb94c15c8b0 2023-08-25T09:39:00.856 INFO:teuthology.orchestra.run.smithi070.stdout:55834574852 2023-08-25T09:39:00.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb93006a080 msgr2=0x7fb93006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb93006a080 0x7fb93006c540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb94c15fdb0 tx=0x7fb948008040 comp rx=0 tx=0).stop 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 msgr2=0x7fb94c1518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 0x7fb94c1518c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009fd0 tx=0x7fb93c00b8e0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 shutdown_connections 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb93006a080 0x7fb93006c540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb94c15c8b0 0x7fb94c1518c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb94c0ca470 0x7fb94c0c2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 --2- 172.21.15.70:0/1278138717 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb94c0aff00 0x7fb94c1f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.855+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 >> 172.21.15.70:0/1278138717 conn(0x7fb94c069c40 msgr2=0x7fb94c0654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.856+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 shutdown_connections 2023-08-25T09:39:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:00.856+0000 7fb953de8640 1 -- 172.21.15.70:0/1278138717 wait complete. 2023-08-25T09:39:00.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.6 2023-08-25T09:39:00.873 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2023-08-25T09:39:01.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-25T09:39:01.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-25T09:39:01.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-25T09:39:01.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-25T09:39:01.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-25T09:39:01.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-25T09:39:01.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-25T09:39:01.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-25T09:39:02.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.105+0000 7f82f9c57640 1 Processor -- start 2023-08-25T09:39:02.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.105+0000 7f82f9c57640 1 -- start start 2023-08-25T09:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.107+0000 7f82f9c57640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f82f40b0610 0x7f82f40cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.107+0000 7f82f9c57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.107+0000 7f82f9c57640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f415d500 con 0x7f82f4159b60 2023-08-25T09:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.107+0000 7f82f9c57640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40cd030 con 0x7f82f40b0610 2023-08-25T09:39:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.107+0000 7f82f9c57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4159f40 con 0x7f82f415c790 2023-08-25T09:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.108+0000 7f82f37fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f82f4159b60 0x7f82f40aff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35478/0 (socket says 172.21.15.70:35478) 2023-08-25T09:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.108+0000 7f82f2ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f82f40b0610 0x7f82f40cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.108+0000 7f82f37fe640 1 -- 172.21.15.70:0/2690403529 learned_addr learned my addr 172.21.15.70:0/2690403529 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.108+0000 7f82f2ffd640 1 -- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 msgr2=0x7f82f415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.109+0000 7f82f2ffd640 1 --2- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.109+0000 7f82f2ffd640 1 -- 172.21.15.70:0/2690403529 >> v1:172.21.15.70:6789/0 conn(0x7f82f4159b60 legacy=0x7f82f40aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.109+0000 7f82f2ffd640 1 -- 172.21.15.70:0/2690403529 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40cc700 con 0x7f82f40b0610 2023-08-25T09:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.110+0000 7f82f2ffd640 1 --2- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f40b0610 0x7f82f40cd4c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005d90 tx=0x7f82ec02dd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=616489ab99843a38 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.111+0000 7f82f27fc640 1 -- 172.21.15.70:0/2690403529 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec02eae0 con 0x7f82f40b0610 2023-08-25T09:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.111+0000 7f82f27fc640 1 -- 172.21.15.70:0/2690403529 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ec02ec40 con 0x7f82f40b0610 2023-08-25T09:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.111+0000 7f82f27fc640 1 -- 172.21.15.70:0/2690403529 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ec036440 con 0x7f82f40b0610 2023-08-25T09:39:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.112+0000 7f82f9c57640 1 -- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f40b0610 msgr2=0x7f82f40cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.112+0000 7f82f9c57640 1 --2- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f40b0610 0x7f82f40cd4c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f82ec005d90 tx=0x7f82ec02dd30 comp rx=0 tx=0).stop 2023-08-25T09:39:02.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.120+0000 7f82f9c57640 1 -- 172.21.15.70:0/2690403529 shutdown_connections 2023-08-25T09:39:02.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.120+0000 7f82f9c57640 1 --2- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f40b0610 0x7f82f40cd4c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.120+0000 7f82f9c57640 1 --2- 172.21.15.70:0/2690403529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f415cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.120+0000 7f82f9c57640 1 -- 172.21.15.70:0/2690403529 >> 172.21.15.70:0/2690403529 conn(0x7f82f4069c40 msgr2=0x7f82f406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.120+0000 7f82f9c57640 1 -- 172.21.15.70:0/2690403529 shutdown_connections 2023-08-25T09:39:02.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.120+0000 7f82f9c57640 1 -- 172.21.15.70:0/2690403529 wait complete. 2023-08-25T09:39:02.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.123+0000 7f82f9c57640 1 Processor -- start 2023-08-25T09:39:02.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.123+0000 7f82f9c57640 1 -- start start 2023-08-25T09:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.125+0000 7f82f9c57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 0x7f82f40c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.126+0000 7f82f37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 0x7f82f40c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.126+0000 7f82f37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 0x7f82f40c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47294/0 (socket says 172.21.15.70:47294) 2023-08-25T09:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.126+0000 7f82f37fe640 1 -- 172.21.15.70:0/3835888791 learned_addr learned my addr 172.21.15.70:0/3835888791 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.126+0000 7f82f9c57640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4159b60 0x7f82f40710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.127+0000 7f82f9c57640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f40c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.127+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41fef90 con 0x7f82f40b0610 2023-08-25T09:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.127+0000 7f82f2ffd640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4159b60 0x7f82f40710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.128+0000 7f82f3fff640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f40c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.128+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41ff270 con 0x7f82f4159b60 2023-08-25T09:39:02.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.128+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41ff550 con 0x7f82f415c790 2023-08-25T09:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.129+0000 7f82f37fe640 1 -- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 msgr2=0x7f82f40c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.129+0000 7f82f37fe640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f40c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.129+0000 7f82f37fe640 1 -- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4159b60 msgr2=0x7f82f40710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.129+0000 7f82f37fe640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4159b60 0x7f82f40710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.129+0000 7f82f37fe640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ec00c0e0 con 0x7f82f40b0610 2023-08-25T09:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.130+0000 7f82f37fe640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 0x7f82f40c3f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f82e400c4a0 tx=0x7f82e400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.131+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4005ce0 con 0x7f82f40b0610 2023-08-25T09:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.131+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f41ff830 con 0x7f82f40b0610 2023-08-25T09:39:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.132+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f4071630 con 0x7f82f40b0610 2023-08-25T09:39:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.135+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e4005e40 con 0x7f82f40b0610 2023-08-25T09:39:02.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.136+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e4007270 con 0x7f82f40b0610 2023-08-25T09:39:02.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.136+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f82e4007490 con 0x7f82f40b0610 2023-08-25T09:39:02.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.138+0000 7f82dbfff640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82e006a1a0 0x7f82e006c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.138+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82e4087b90 con 0x7f82f40b0610 2023-08-25T09:39:02.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.139+0000 7f82f2ffd640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82e006a1a0 0x7f82e006c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.140+0000 7f82f2ffd640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82e006a1a0 0x7f82e006c660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82ec038000 tx=0x7f82ec032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.141+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d4005250 con 0x7f82f40b0610 2023-08-25T09:39:02.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.150+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f82e4053310 con 0x7f82f40b0610 2023-08-25T09:39:02.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.167+0000 7fb8b475d640 1 Processor -- start 2023-08-25T09:39:02.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7f818a142640 1 Processor -- start 2023-08-25T09:39:02.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.167+0000 7fb8b475d640 1 -- start start 2023-08-25T09:39:02.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7fb8b475d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac0cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7f818a142640 1 -- start start 2023-08-25T09:39:02.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7fb8b475d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb8ac15cea0 0x7fb8ac0b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7fb8b475d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ac0b08a0 con 0x7fb8ac0aff00 2023-08-25T09:39:02.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7fb8b475d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ac15d500 con 0x7fb8ac15cea0 2023-08-25T09:39:02.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.168+0000 7fb8b475d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0cd030 con 0x7fb8ac159b60 2023-08-25T09:39:02.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7fb8b1cf8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb8ac0aff00 0x7fb8ac15c790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35498/0 (socket says 172.21.15.70:35498) 2023-08-25T09:39:02.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7fb8b1cf8640 1 -- 172.21.15.70:0/32502450 learned_addr learned my addr 172.21.15.70:0/32502450 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:02.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.162+0000 7f18ca34e640 1 Processor -- start 2023-08-25T09:39:02.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.162+0000 7f18ca34e640 1 -- start start 2023-08-25T09:39:02.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.164+0000 7f18ca34e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18c416cc80 0x7f18c408a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7fb8b24f9640 1 --2- 172.21.15.70:0/32502450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac0cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7fb8b2cfa640 1 --2- 172.21.15.70:0/32502450 >> v2:172.21.15.155:3300/0 conn(0x7fb8ac15cea0 0x7fb8ac0b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.164+0000 7f18ca34e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40cca90 con 0x7f18c416d060 2023-08-25T09:39:02.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.164+0000 7f18ca34e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40b3da0 con 0x7f18c40cba60 2023-08-25T09:39:02.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.164+0000 7f18ca34e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c40b1480 con 0x7f18c416cc80 2023-08-25T09:39:02.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.165+0000 7f18c37fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18c416d060 0x7f18c40cb350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35494/0 (socket says 172.21.15.70:35494) 2023-08-25T09:39:02.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.165+0000 7f18c37fe640 1 -- 172.21.15.70:0/2675537573 learned_addr learned my addr 172.21.15.70:0/2675537573 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:02.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.166+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253992647 0 0) 0x7f18c40cca90 con 0x7f18c416d060 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18ac003970 con 0x7f18c416d060 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916809674 0 0) 0x7f18c40b3da0 con 0x7f18c40cba60 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.169+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c40cca90 con 0x7f18c40cba60 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.166+0000 7f18c3fff640 1 --2- 172.21.15.70:0/2675537573 >> v2:172.21.15.70:3300/0 conn(0x7f18c416cc80 0x7f18c408a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2962684283 0 0) 0x7f18ac003970 con 0x7f18c416d060 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 >> v2:172.21.15.70:3300/0 conn(0x7f18c416cc80 msgr2=0x7f18c408a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/2675537573 >> v2:172.21.15.70:3300/0 conn(0x7f18c416cc80 0x7f18c408a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 >> v1:172.21.15.155:6789/0 conn(0x7f18c40cba60 legacy=0x7f18c40ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c40b4080 con 0x7f18c416d060 2023-08-25T09:39:02.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7fb8b24f9640 1 -- 172.21.15.70:0/32502450 >> v2:172.21.15.155:3300/0 conn(0x7fb8ac15cea0 msgr2=0x7fb8ac0b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7fb8b24f9640 1 --2- 172.21.15.70:0/32502450 >> v2:172.21.15.155:3300/0 conn(0x7fb8ac15cea0 0x7fb8ac0b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f18bc0019f0 con 0x7f18c416d060 2023-08-25T09:39:02.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 >> v1:172.21.15.70:6789/0 conn(0x7f18c416d060 legacy=0x7f18c40cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003bb0 0x7f18ac003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac0045b0 0x7f18ac004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18ac005150 0x7f18ac005600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c40b1480 con 0x7f18ac003bb0 2023-08-25T09:39:02.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac00c010 con 0x7f18ac005150 2023-08-25T09:39:02.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ac00c1d0 con 0x7f18ac0045b0 2023-08-25T09:39:02.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7fb8b14f7640 1 -- 172.21.15.70:0/32502450 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906202167 0 0) 0x7fb8ac0b08a0 con 0x7fb8ac0aff00 2023-08-25T09:39:02.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7f18c88eb640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18ac005150 0x7f18ac005600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7fb8b24f9640 1 -- 172.21.15.70:0/32502450 >> v1:172.21.15.70:6789/0 conn(0x7fb8ac0aff00 legacy=0x7fb8ac15c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.170+0000 7fb8b24f9640 1 -- 172.21.15.70:0/32502450 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac159f40 con 0x7fb8ac159b60 2023-08-25T09:39:02.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c3fff640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003bb0 0x7f18ac003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c37fe640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac0045b0 0x7f18ac004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c88eb640 1 -- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac0045b0 msgr2=0x7f18ac004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c88eb640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac0045b0 0x7f18ac004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c88eb640 1 -- 172.21.15.70:0/2675537573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003bb0 msgr2=0x7f18ac003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c88eb640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18ac003bb0 0x7f18ac003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.171+0000 7f18c88eb640 1 -- 172.21.15.70:0/2675537573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c40b4080 con 0x7f18ac005150 2023-08-25T09:39:02.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.172+0000 7f1010599640 1 Processor -- start 2023-08-25T09:39:02.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.172+0000 7f818a142640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81840b0610 0x7f81840cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.172+0000 7f1010599640 1 -- start start 2023-08-25T09:39:02.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.172+0000 7fb8b24f9640 1 --2- 172.21.15.70:0/32502450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac0cd4c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80097d0 tx=0x7fb8a802b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749cb0fee3df6692 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.172+0000 7fb0c9112640 1 Processor -- start 2023-08-25T09:39:02.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f18c88eb640 1 -- 172.21.15.70:0/2675537573 reap_dead start 2023-08-25T09:39:02.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f818a142640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f818415c790 0x7f818415cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f81837fe640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f8184159b60 0x7f81840aff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:45882/0 (socket says 172.21.15.70:45882) 2023-08-25T09:39:02.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f81837fe640 1 -- 172.21.15.70:0/1115547888 learned_addr learned my addr 172.21.15.70:0/1115547888 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:02.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f818a142640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818415d500 con 0x7f8184159b60 2023-08-25T09:39:02.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81840cd030 con 0x7f81840b0610 2023-08-25T09:39:02.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8184159f40 con 0x7f818415c790 2023-08-25T09:39:02.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7fb0c9112640 1 -- start start 2023-08-25T09:39:02.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f8182ffd640 1 --2- 172.21.15.70:0/1115547888 >> v2:172.21.15.70:3300/0 conn(0x7f81840b0610 0x7f81840cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7fb8b14f7640 1 -- 172.21.15.70:0/32502450 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80316f0 con 0x7fb8ac159b60 2023-08-25T09:39:02.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7fb8b14f7640 1 -- 172.21.15.70:0/32502450 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8031850 con 0x7fb8ac159b60 2023-08-25T09:39:02.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7fb8b14f7640 1 -- 172.21.15.70:0/32502450 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8031be0 con 0x7fb8ac159b60 2023-08-25T09:39:02.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7fb8b475d640 1 -- 172.21.15.70:0/32502450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 msgr2=0x7fb8ac0cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7fb8b475d640 1 --2- 172.21.15.70:0/32502450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac0cd4c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80097d0 tx=0x7fb8a802b790 comp rx=0 tx=0).stop 2023-08-25T09:39:02.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f1010599640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10000061b0 0x7f100003eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f1010599640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10000588d0 0x7f1000057050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f1010599640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1000058d60 con 0x7f100003c0b0 2023-08-25T09:39:02.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f1010599640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1000057590 con 0x7f10000061b0 2023-08-25T09:39:02.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f1010599640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1000057830 con 0x7f10000588d0 2023-08-25T09:39:02.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f18c88eb640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18ac005150 0x7f18ac005600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18b4001890 tx=0x7f18ac005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f740d3e9a531cf66 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.173+0000 7f8183fff640 1 --2- 172.21.15.70:0/1115547888 >> v2:172.21.15.155:3300/0 conn(0x7f818415c790 0x7f818415cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f8182ffd640 1 -- 172.21.15.70:0/1115547888 >> v2:172.21.15.155:3300/0 conn(0x7f818415c790 msgr2=0x7f818415cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f8182ffd640 1 --2- 172.21.15.70:0/1115547888 >> v2:172.21.15.155:3300/0 conn(0x7f818415c790 0x7f818415cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f8182ffd640 1 -- 172.21.15.70:0/1115547888 >> v1:172.21.15.155:6789/0 conn(0x7f8184159b60 legacy=0x7f81840aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.174+0000 7f8182ffd640 1 -- 172.21.15.70:0/1115547888 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840cc700 con 0x7f81840b0610 2023-08-25T09:39:02.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb8b475d640 1 -- 172.21.15.70:0/32502450 shutdown_connections 2023-08-25T09:39:02.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb8b475d640 1 --2- 172.21.15.70:0/32502450 >> v2:172.21.15.155:3300/0 conn(0x7fb8ac15cea0 0x7fb8ac0b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb8b475d640 1 --2- 172.21.15.70:0/32502450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac0cd4c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb8b475d640 1 -- 172.21.15.70:0/32502450 >> 172.21.15.70:0/32502450 conn(0x7fb8ac069c40 msgr2=0x7fb8ac06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f100e335640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10000061b0 0x7f100003eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f100e335640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10000061b0 0x7f100003eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38720/0 (socket says 172.21.15.70:38720) 2023-08-25T09:39:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f100e335640 1 -- 172.21.15.70:0/463978991 learned_addr learned my addr 172.21.15.70:0/463978991 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb0c9112640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0c40b4ae0 0x7fb0c40cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb0c9112640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0c40b57a0 0x7fb0c40cd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb0c9112640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c40d16a0 con 0x7fb0c415cc50 2023-08-25T09:39:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb0c9112640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c40d26f0 con 0x7fb0c40b4ae0 2023-08-25T09:39:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb0c9112640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c40d0a90 con 0x7fb0c40b57a0 2023-08-25T09:39:02.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b40052d0 con 0x7f18ac005150 2023-08-25T09:39:02.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb8b475d640 1 -- 172.21.15.70:0/32502450 shutdown_connections 2023-08-25T09:39:02.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b4005430 con 0x7f18ac005150 2023-08-25T09:39:02.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7f18c2ffd640 1 -- 172.21.15.70:0/2675537573 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4003830 con 0x7f18ac005150 2023-08-25T09:39:02.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.175+0000 7fb8b475d640 1 -- 172.21.15.70:0/32502450 wait complete. 2023-08-25T09:39:02.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f100db34640 1 --2- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10000588d0 0x7f1000057050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0c40b4ae0 0x7fb0c40cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f100e335640 1 -- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10000588d0 msgr2=0x7f1000057050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f100e335640 1 --2- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10000588d0 0x7f1000057050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f100e335640 1 -- 172.21.15.70:0/463978991 >> v1:172.21.15.155:6789/0 conn(0x7f100003c0b0 legacy=0x7f100003c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0c40b4ae0 0x7fb0c40cd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47340/0 (socket says 172.21.15.70:47340) 2023-08-25T09:39:02.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 -- 172.21.15.70:0/154119174 learned_addr learned my addr 172.21.15.70:0/154119174 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c359e640 1 --2- 172.21.15.70:0/154119174 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40b57a0 0x7fb0c40cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f18ca34e640 1 -- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18ac005150 msgr2=0x7f18ac005600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f18ca34e640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18ac005150 0x7f18ac005600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f18b4001890 tx=0x7f18ac005c40 comp rx=0 tx=0).stop 2023-08-25T09:39:02.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f100e335640 1 -- 172.21.15.70:0/463978991 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000057b10 con 0x7f10000061b0 2023-08-25T09:39:02.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7f100d333640 1 -- 172.21.15.70:0/463978991 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469558444 0 0) 0x7f0ff8001110 con 0x7f100003c0b0 2023-08-25T09:39:02.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 -- 172.21.15.70:0/154119174 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40b57a0 msgr2=0x7fb0c40cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 --2- 172.21.15.70:0/154119174 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40b57a0 0x7fb0c40cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 -- 172.21.15.70:0/154119174 >> v1:172.21.15.155:6789/0 conn(0x7fb0c415cc50 legacy=0x7fb0c40b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.176+0000 7fb0c259c640 1 -- 172.21.15.70:0/154119174 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40d0d70 con 0x7fb0c40b4ae0 2023-08-25T09:39:02.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7f18ca34e640 1 -- 172.21.15.70:0/2675537573 shutdown_connections 2023-08-25T09:39:02.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7f18ca34e640 1 --2- 172.21.15.70:0/2675537573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18ac005150 0x7f18ac005600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7f18ca34e640 1 -- 172.21.15.70:0/2675537573 >> 172.21.15.70:0/2675537573 conn(0x7f18c4069c40 msgr2=0x7f18c406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7f8182ffd640 1 --2- 172.21.15.70:0/1115547888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81840b0610 0x7f81840cd4c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81780087a0 tx=0x7f817802ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d77ac6961f62a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7fbe9deb0640 1 Processor -- start 2023-08-25T09:39:02.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7f100e335640 1 --2- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000061b0 0x7f100003eb40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1008079c80 tx=0x7f10080a5970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0326ac07b36c331 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7f18ca34e640 1 -- 172.21.15.70:0/2675537573 shutdown_connections 2023-08-25T09:39:02.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7f100d333640 1 -- 172.21.15.70:0/463978991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080aa6e0 con 0x7f10000061b0 2023-08-25T09:39:02.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7f100d333640 1 -- 172.21.15.70:0/463978991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10080aa840 con 0x7f10000061b0 2023-08-25T09:39:02.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7f100d333640 1 -- 172.21.15.70:0/463978991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080aacb0 con 0x7f10000061b0 2023-08-25T09:39:02.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7fb8b475d640 1 Processor -- start 2023-08-25T09:39:02.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7fb0c259c640 1 --2- 172.21.15.70:0/154119174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c40cd1d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b5410 tx=0x7fb0b802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a23f06d32839d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7f81827fc640 1 -- 172.21.15.70:0/1115547888 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817802eb70 con 0x7f81840b0610 2023-08-25T09:39:02.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7f81827fc640 1 -- 172.21.15.70:0/1115547888 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8178032070 con 0x7f81840b0610 2023-08-25T09:39:02.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7f81827fc640 1 -- 172.21.15.70:0/1115547888 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8178038400 con 0x7f81840b0610 2023-08-25T09:39:02.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fb0c1d9b640 1 -- 172.21.15.70:0/154119174 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8031880 con 0x7fb0c40b4ae0 2023-08-25T09:39:02.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fb0c1d9b640 1 -- 172.21.15.70:0/154119174 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b80319e0 con 0x7fb0c40b4ae0 2023-08-25T09:39:02.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fb0c1d9b640 1 -- 172.21.15.70:0/154119174 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8031d70 con 0x7fb0c40b4ae0 2023-08-25T09:39:02.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81840b0610 msgr2=0x7f81840cd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f818a142640 1 --2- 172.21.15.70:0/1115547888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81840b0610 0x7f81840cd4c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81780087a0 tx=0x7f817802ddc0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7fb8b475d640 1 -- start start 2023-08-25T09:39:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb8b475d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8ac0aff00 0x7fb8ac1436f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 shutdown_connections 2023-08-25T09:39:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f818a142640 1 --2- 172.21.15.70:0/1115547888 >> v2:172.21.15.155:3300/0 conn(0x7f818415c790 0x7f818415cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f818a142640 1 --2- 172.21.15.70:0/1115547888 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81840b0610 0x7f81840cd4c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 >> 172.21.15.70:0/1115547888 conn(0x7f8184069c40 msgr2=0x7f818406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 -- 172.21.15.70:0/154119174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b4ae0 msgr2=0x7fb0c40cd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 --2- 172.21.15.70:0/154119174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c40cd1d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40b5410 tx=0x7fb0b802b920 comp rx=0 tx=0).stop 2023-08-25T09:39:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 -- 172.21.15.70:0/154119174 shutdown_connections 2023-08-25T09:39:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 --2- 172.21.15.70:0/154119174 >> v2:172.21.15.155:3300/0 conn(0x7fb0c40b57a0 0x7fb0c40cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 --2- 172.21.15.70:0/154119174 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c40cd1d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 -- 172.21.15.70:0/154119174 >> 172.21.15.70:0/154119174 conn(0x7fb0c40cf3d0 msgr2=0x7fb0c40cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.177+0000 7fbe9deb0640 1 -- start start 2023-08-25T09:39:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fbe9deb0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbe900df500 0x7fbe90007580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fbe9deb0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe90005fe0 0x7fbe90054260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fbe9deb0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe900064e0 con 0x7fbe9004fc70 2023-08-25T09:39:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fbe9deb0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe900df8e0 con 0x7fbe900df500 2023-08-25T09:39:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.179+0000 7fbe9deb0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe900067c0 con 0x7fbe90005fe0 2023-08-25T09:39:02.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.178+0000 7f18ca34e640 1 -- 172.21.15.70:0/2675537573 wait complete. 2023-08-25T09:39:02.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f18ca34e640 1 Processor -- start 2023-08-25T09:39:02.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7fb0c9112640 1 -- 172.21.15.70:0/154119174 shutdown_connections 2023-08-25T09:39:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fb0c9112640 1 -- 172.21.15.70:0/154119174 wait complete. 2023-08-25T09:39:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fb8b475d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f1010599640 1 -- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000061b0 msgr2=0x7f100003eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.180+0000 7f1010599640 1 --2- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000061b0 0x7f100003eb40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1008079c80 tx=0x7f10080a5970 comp rx=0 tx=0).stop 2023-08-25T09:39:02.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fb8b475d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8ac15cea0 0x7fb8ac071190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 shutdown_connections 2023-08-25T09:39:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fbe9d6af640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fbe9004fc70 0x7fbe900e2f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:45912/0 (socket says 172.21.15.70:45912) 2023-08-25T09:39:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fbe9d6af640 1 -- 172.21.15.70:0/2180231124 learned_addr learned my addr 172.21.15.70:0/2180231124 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fbe9ceae640 1 --2- 172.21.15.70:0/2180231124 >> v2:172.21.15.155:3300/0 conn(0x7fbe900df500 0x7fbe90007580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7fb8b1cf8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f1010599640 1 -- 172.21.15.70:0/463978991 shutdown_connections 2023-08-25T09:39:02.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f1010599640 1 --2- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10000588d0 0x7f1000057050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f1010599640 1 --2- 172.21.15.70:0/463978991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000061b0 0x7f100003eb40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f1010599640 1 -- 172.21.15.70:0/463978991 >> 172.21.15.70:0/463978991 conn(0x7f1000055a20 msgr2=0x7f1000055e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fbe97fff640 1 --2- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe90005fe0 0x7fbe90054260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fb8b1cf8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac06f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44966/0 (socket says 172.21.15.70:44966) 2023-08-25T09:39:02.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fb8b1cf8640 1 -- 172.21.15.70:0/819283853 learned_addr learned my addr 172.21.15.70:0/819283853 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0ae780 con 0x7fb8ac0aff00 2023-08-25T09:39:02.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7f18ca34e640 1 -- start start 2023-08-25T09:39:02.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0c45a0 con 0x7fb8ac15cea0 2023-08-25T09:39:02.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac0ade70 con 0x7fb8ac159b60 2023-08-25T09:39:02.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fb8b2cfa640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8ac15cea0 0x7fb8ac071190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f1010599640 1 -- 172.21.15.70:0/463978991 shutdown_connections 2023-08-25T09:39:02.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f18ca34e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 0x7f18c40adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f18ca34e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c4167400 0x7f18c40c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18ca34e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c4167840 0x7f18c4070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18ca34e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4070880 con 0x7f18c4167840 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18ca34e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c406fc70 con 0x7f18c4167400 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18ca34e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c406ff50 con 0x7f18ac005150 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 0x7f18c40adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 0x7f18c40adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44970/0 (socket says 172.21.15.70:44970) 2023-08-25T09:39:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18c3fff640 1 -- 172.21.15.70:0/3344710943 learned_addr learned my addr 172.21.15.70:0/3344710943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7f18c37fe640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c4167400 0x7f18c40c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f18c88eb640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c4167840 0x7f18c4070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.181+0000 7f818a142640 1 -- 172.21.15.70:0/1115547888 wait complete. 2023-08-25T09:39:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f818a142640 1 Processor -- start 2023-08-25T09:39:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7fb8b1cf8640 1 -- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8ac15cea0 msgr2=0x7fb8ac071190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7fb8b1cf8640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8ac15cea0 0x7fb8ac071190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7fb8b1cf8640 1 -- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8ac0aff00 msgr2=0x7fb8ac1436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7fb8b1cf8640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8ac0aff00 0x7fb8ac1436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7fb8b1cf8640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a8009400 con 0x7fb8ac159b60 2023-08-25T09:39:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7fb0c9112640 1 Processor -- start 2023-08-25T09:39:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7fbe9ceae640 1 -- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe90005fe0 msgr2=0x7fbe90054260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7fbe9ceae640 1 --2- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe90005fe0 0x7fbe90054260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7fbe9ceae640 1 -- 172.21.15.70:0/2180231124 >> v1:172.21.15.155:6789/0 conn(0x7fbe9004fc70 legacy=0x7fbe900e2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.183+0000 7fbe9ceae640 1 -- 172.21.15.70:0/2180231124 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90006aa0 con 0x7fbe900df500 2023-08-25T09:39:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7fb0c9112640 1 -- start start 2023-08-25T09:39:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f18c3fff640 1 -- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c4167400 msgr2=0x7f18c40c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f18c3fff640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c4167400 0x7f18c40c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f18c3fff640 1 -- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c4167840 msgr2=0x7f18c4070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f18c3fff640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c4167840 0x7f18c4070d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7f18c3fff640 1 -- 172.21.15.70:0/3344710943 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c4070230 con 0x7f18ac005150 2023-08-25T09:39:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.184+0000 7fb8b1cf8640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac06f110 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c4a0 tx=0x7fb89c00c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.182+0000 7f1010599640 1 -- 172.21.15.70:0/463978991 wait complete. 2023-08-25T09:39:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.185+0000 7f1010599640 1 Processor -- start 2023-08-25T09:39:02.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.185+0000 7f18c3fff640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 0x7f18c40adee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b950 tx=0x7f18b800be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.185+0000 7fbe9ceae640 1 --2- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900df500 0x7fbe90007580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a940 tx=0x7fbe84030170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2b5a207fa76ddc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.185+0000 7f1010599640 1 -- start start 2023-08-25T09:39:02.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.185+0000 7f818a142640 1 -- start start 2023-08-25T09:39:02.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.186+0000 7f818a142640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 0x7f818414daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.186+0000 7fb0c9112640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c4144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.186+0000 7f1010599640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 0x7f10000dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.186+0000 7f818a142640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8184159b60 0x7f8184150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.186+0000 7f1010599640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f100003c0b0 0x7f100016b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f1010599640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 0x7f100003da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7fb0c9112640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b57a0 0x7fb0c40c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f1010599640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100016d1e0 con 0x7f10000061b0 2023-08-25T09:39:02.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f1010599640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100017cfd0 con 0x7f10000588d0 2023-08-25T09:39:02.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f1010599640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10000e0370 con 0x7f100003c0b0 2023-08-25T09:39:02.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f81837fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 0x7f818414daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f81837fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 0x7f818414daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:44984/0 (socket says 172.21.15.70:44984) 2023-08-25T09:39:02.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f8182ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8184159b60 0x7f8184150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7fb0c9112640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0c415cc50 0x7fb0c415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f8182ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8184159b60 0x7f8184150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47356/0 (socket says 172.21.15.70:47356) 2023-08-25T09:39:02.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f81837fe640 1 -- 172.21.15.70:0/3774748694 learned_addr learned my addr 172.21.15.70:0/3774748694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818415c790 0x7f818414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7fb0c2d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c4144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7fb0c2d9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c4144b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38772/0 (socket says 172.21.15.70:38772) 2023-08-25T09:39:02.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818414eb70 con 0x7f8184159b60 2023-08-25T09:39:02.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818414e240 con 0x7f818415c790 2023-08-25T09:39:02.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818414e520 con 0x7f81840b0610 2023-08-25T09:39:02.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7fb0c2d9d640 1 -- 172.21.15.70:0/3421092761 learned_addr learned my addr 172.21.15.70:0/3421092761 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f100db34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f100003c0b0 0x7f100016b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f100e335640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 0x7f10000dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f100eb36640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 0x7f100003da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f100eb36640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 0x7f100003da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38788/0 (socket says 172.21.15.70:38788) 2023-08-25T09:39:02.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f100e335640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 0x7f10000dfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47368/0 (socket says 172.21.15.70:47368) 2023-08-25T09:39:02.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7f100eb36640 1 -- 172.21.15.70:0/1985742391 learned_addr learned my addr 172.21.15.70:0/1985742391 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.187+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40c7520 con 0x7fb0c40b57a0 2023-08-25T09:39:02.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cc130 con 0x7fb0c40b4ae0 2023-08-25T09:39:02.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cc3d0 con 0x7fb0c415cc50 2023-08-25T09:39:02.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7fb0c259c640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b57a0 0x7fb0c40c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7fb0c359e640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0c415cc50 0x7fb0c415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f81837fe640 1 -- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818415c790 msgr2=0x7f818414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f81837fe640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818415c790 0x7f818414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f81837fe640 1 -- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8184159b60 msgr2=0x7f8184150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f81837fe640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8184159b60 0x7f8184150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f81837fe640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178008480 con 0x7f81840b0610 2023-08-25T09:39:02.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f100eb36640 1 -- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f100003c0b0 msgr2=0x7f100016b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f100eb36640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f100003c0b0 0x7f100016b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f100eb36640 1 -- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 msgr2=0x7f10000dfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f100eb36640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 0x7f10000dfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.188+0000 7f100eb36640 1 -- 172.21.15.70:0/1985742391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000e1880 con 0x7f10000588d0 2023-08-25T09:39:02.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8012ab0 con 0x7f18ac005150 2023-08-25T09:39:02.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7f100e335640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 0x7f10000dfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:39:02.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7fb0c259c640 1 -- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0c415cc50 msgr2=0x7fb0c415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7fb0c259c640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0c415cc50 0x7fb0c415f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7fb0c259c640 1 -- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40b4ae0 msgr2=0x7fb0c4144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7fb0c259c640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c4144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7fb0c259c640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80094d0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c4070510 con 0x7f18ac005150 2023-08-25T09:39:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c40ae420 con 0x7f18ac005150 2023-08-25T09:39:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8013040 con 0x7f18ac005150 2023-08-25T09:39:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8012e30 con 0x7f18ac005150 2023-08-25T09:39:02.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.189+0000 7fb0c259c640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b57a0 0x7fb0c40c9a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8033000 tx=0x7fb0b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c0074e0 con 0x7fb8ac159b60 2023-08-25T09:39:02.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c007640 con 0x7fb8ac159b60 2023-08-25T09:39:02.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c004a00 con 0x7fb8ac159b60 2023-08-25T09:39:02.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac147e60 con 0x7fb8ac159b60 2023-08-25T09:39:02.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fbe977fe640 1 -- 172.21.15.70:0/2180231124 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84038440 con 0x7fbe900df500 2023-08-25T09:39:02.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fbe977fe640 1 -- 172.21.15.70:0/2180231124 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe840385a0 con 0x7fbe900df500 2023-08-25T09:39:02.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fbe977fe640 1 -- 172.21.15.70:0/2180231124 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84038970 con 0x7fbe900df500 2023-08-25T09:39:02.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7f100eb36640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 0x7f100003da00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c390 tx=0x7f0ff800c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7f81837fe640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 0x7f818414daa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f817400c400 tx=0x7f817400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b803ad20 con 0x7fb0c40b57a0 2023-08-25T09:39:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b80443f0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80447a0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80035f0 con 0x7f10000588d0 2023-08-25T09:39:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff8003770 con 0x7f10000588d0 2023-08-25T09:39:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8003ba0 con 0x7f10000588d0 2023-08-25T09:39:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174005ce0 con 0x7f81840b0610 2023-08-25T09:39:02.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174005e40 con 0x7f81840b0610 2023-08-25T09:39:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81740072d0 con 0x7f81840b0610 2023-08-25T09:39:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c40cc6b0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7fbe9deb0640 1 -- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900df500 msgr2=0x7fbe90007580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900df500 0x7fbe90007580 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400a940 tx=0x7fbe84030170 comp rx=0 tx=0).stop 2023-08-25T09:39:02.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.191+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a4005250 con 0x7f18ac005150 2023-08-25T09:39:02.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.190+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ac0ae0c0 con 0x7fb8ac159b60 2023-08-25T09:39:02.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.193+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c4145080 con 0x7fb0c40b57a0 2023-08-25T09:39:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.193+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100016d410 con 0x7f10000588d0 2023-08-25T09:39:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.193+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10000e12c0 con 0x7f10000588d0 2023-08-25T09:39:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818414e800 con 0x7f81840b0610 2023-08-25T09:39:02.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818414f540 con 0x7f81840b0610 2023-08-25T09:39:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fbe9deb0640 1 -- 172.21.15.70:0/2180231124 shutdown_connections 2023-08-25T09:39:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe90005fe0 0x7fbe90054260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/2180231124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe900df500 0x7fbe90007580 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fbe9deb0640 1 -- 172.21.15.70:0/2180231124 >> 172.21.15.70:0/2180231124 conn(0x7fbe90054c10 msgr2=0x7fbe90055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18b8021460 con 0x7f18ac005150 2023-08-25T09:39:02.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fbe9deb0640 1 -- 172.21.15.70:0/2180231124 shutdown_connections 2023-08-25T09:39:02.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.195+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb894005250 con 0x7fb8ac159b60 2023-08-25T09:39:02.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.195+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100016d5d0 con 0x7f10000588d0 2023-08-25T09:39:02.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.195+0000 7f18c0ff9640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0b80303d0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.195+0000 7fb0ab7fe640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0b006a0d0 0x7fb0b006c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.196+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18b8088940 con 0x7f18ac005150 2023-08-25T09:39:02.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.196+0000 7f18c37fe640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.196+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0b8079d90 con 0x7fb0c40b57a0 2023-08-25T09:39:02.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.196+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb89c004b60 con 0x7fb8ac159b60 2023-08-25T09:39:02.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.197+0000 7f18c37fe640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18c406a2b0 tx=0x7f18bc002140 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.196+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818414f790 con 0x7f81840b0610 2023-08-25T09:39:02.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.197+0000 7fb8a2ffd640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb89006a120 0x7fb89006c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.197+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb89c087a20 con 0x7fb8ac159b60 2023-08-25T09:39:02.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.197+0000 7fb0c2d9d640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0b006a0d0 0x7fb0b006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.197+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c40c76d0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.194+0000 7fbe9deb0640 1 -- 172.21.15.70:0/2180231124 wait complete. 2023-08-25T09:39:02.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.197+0000 7fbe9deb0640 1 Processor -- start 2023-08-25T09:39:02.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.198+0000 7fb0c2d9d640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0b006a0d0 0x7fb0b006c590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc001ef0 tx=0x7fb0bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.198+0000 7fb8b24f9640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb89006a120 0x7fb89006c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.199+0000 7fb8b24f9640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb89006a120 0x7fb89006c5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8036000 tx=0x7fb8a8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ff8003d00 con 0x7f10000588d0 2023-08-25T09:39:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.199+0000 7fbe9deb0640 1 -- start start 2023-08-25T09:39:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7fbe9deb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 0x7fbe900d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7fbe9deb0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9004fc70 0x7fbe9017c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7fbe9deb0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe900df500 0x7fbe900396d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7fbe9deb0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90185750 con 0x7fbe90005fe0 2023-08-25T09:39:02.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7fbe9deb0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90186570 con 0x7fbe9004fc70 2023-08-25T09:39:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.200+0000 7fbe9deb0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe90185960 con 0x7fbe900df500 2023-08-25T09:39:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7fbe9ceae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 0x7fbe900d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7f0ffeffd640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fec06a0a0 0x7f0fec06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ff80878d0 con 0x7f10000588d0 2023-08-25T09:39:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7f100e335640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fec06a0a0 0x7f0fec06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7f100e335640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fec06a0a0 0x7f0fec06c560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1000008630 tx=0x7f10080ab000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7fbe9ceae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 0x7fbe900d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47390/0 (socket says 172.21.15.70:47390) 2023-08-25T09:39:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7fbe9ceae640 1 -- 172.21.15.70:0/1380495849 learned_addr learned my addr 172.21.15.70:0/1380495849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.201+0000 7fbe9d6af640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe900df500 0x7fbe900396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7fbe97fff640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9004fc70 0x7fbe9017c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7fbe9ceae640 1 -- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe900df500 msgr2=0x7fbe900396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7fbe9ceae640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe900df500 0x7fbe900396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7fbe9ceae640 1 -- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9004fc70 msgr2=0x7fbe9017c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7fbe9ceae640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9004fc70 0x7fbe9017c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.202+0000 7fbe9ceae640 1 -- 172.21.15.70:0/1380495849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8400a5a0 con 0x7fbe90005fe0 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.203+0000 7fbe9ceae640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 0x7fbe900d27f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe84030780 tx=0x7fbe840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.203+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84038440 con 0x7fbe90005fe0 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.203+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe90185c40 con 0x7fbe90005fe0 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.204+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe90186200 con 0x7fbe90005fe0 2023-08-25T09:39:02.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.204+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0c40c76d0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.204+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84013540 con 0x7fbe90005fe0 2023-08-25T09:39:02.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.204+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84039ea0 con 0x7fbe90005fe0 2023-08-25T09:39:02.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.205+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78005250 con 0x7fbe90005fe0 2023-08-25T09:39:02.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.207+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbe8400f9e0 con 0x7fbe90005fe0 2023-08-25T09:39:02.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.207+0000 7fbe957fa640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7c06a050 0x7fbe7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.208+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe840aae00 con 0x7fbe90005fe0 2023-08-25T09:39:02.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.208+0000 7fbe97fff640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7c06a050 0x7fbe7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.208+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18b8054040 con 0x7f18ac005150 2023-08-25T09:39:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.209+0000 7fbe97fff640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7c06a050 0x7fbe7c06c510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe90048670 tx=0x7fbe8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.212+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f81740074f0 con 0x7f81840b0610 2023-08-25T09:39:02.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.213+0000 7f816bfff640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f817006a0a0 0x7f817006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.214+0000 7f8182ffd640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f817006a0a0 0x7f817006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.214+0000 7f8182ffd640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f817006a0a0 0x7f817006c560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8178039000 tx=0x7f817800a040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.216+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbe84076450 con 0x7fbe90005fe0 2023-08-25T09:39:02.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.216+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0ff8053150 con 0x7f10000588d0 2023-08-25T09:39:02.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.221+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f817405b180 con 0x7f81840b0610 2023-08-25T09:39:02.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.222+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb89c053230 con 0x7fb8ac159b60 2023-08-25T09:39:02.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.235+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f81740537b0 con 0x7f81840b0610 2023-08-25T09:39:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.253+0000 7fe750975640 1 Processor -- start 2023-08-25T09:39:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.253+0000 7fe750975640 1 -- start start 2023-08-25T09:39:02.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.255+0000 7fe750975640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c1589c0 0x7fe74c06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.255+0000 7fe750975640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe74c15c640 0x7fe74c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.255+0000 7fe750975640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe74c06bcf0 con 0x7fe74c06c410 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.255+0000 7fe750975640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe74c06b450 con 0x7fe74c15c640 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.255+0000 7fe750975640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c06b730 con 0x7fe74c1589c0 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.256+0000 7fe74a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe74c15c640 0x7fe74c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.256+0000 7fe74a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe74c15c640 0x7fe74c0cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47404/0 (socket says 172.21.15.70:47404) 2023-08-25T09:39:02.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.256+0000 7fe74a59c640 1 -- 172.21.15.70:0/785207242 learned_addr learned my addr 172.21.15.70:0/785207242 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.256+0000 7fe74ad9d640 1 --2- 172.21.15.70:0/785207242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c1589c0 0x7fe74c06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.257+0000 7fe74a59c640 1 -- 172.21.15.70:0/785207242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c1589c0 msgr2=0x7fe74c06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.257+0000 7fe74a59c640 1 --2- 172.21.15.70:0/785207242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c1589c0 0x7fe74c06ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.257+0000 7fe74a59c640 1 -- 172.21.15.70:0/785207242 >> v1:172.21.15.70:6789/0 conn(0x7fe74c06c410 legacy=0x7fe74c0c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.257+0000 7fe74a59c640 1 -- 172.21.15.70:0/785207242 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c06ba10 con 0x7fe74c15c640 2023-08-25T09:39:02.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.259+0000 7fe74a59c640 1 --2- 172.21.15.70:0/785207242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c15c640 0x7fe74c0cc950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe74400d370 tx=0x7fe74402e1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78ab253675affaf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.263+0000 7fe749d9b640 1 -- 172.21.15.70:0/785207242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744038400 con 0x7fe74c15c640 2023-08-25T09:39:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.264+0000 7fe749d9b640 1 -- 172.21.15.70:0/785207242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe744032070 con 0x7fe74c15c640 2023-08-25T09:39:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.264+0000 7fe749d9b640 1 -- 172.21.15.70:0/785207242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744038780 con 0x7fe74c15c640 2023-08-25T09:39:02.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.265+0000 7fe750975640 1 -- 172.21.15.70:0/785207242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c15c640 msgr2=0x7fe74c0cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.265+0000 7fe750975640 1 --2- 172.21.15.70:0/785207242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c15c640 0x7fe74c0cc950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe74400d370 tx=0x7fe74402e1f0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.266+0000 7fe750975640 1 -- 172.21.15.70:0/785207242 shutdown_connections 2023-08-25T09:39:02.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.266+0000 7fe750975640 1 --2- 172.21.15.70:0/785207242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c15c640 0x7fe74c0cc950 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.266+0000 7fe750975640 1 --2- 172.21.15.70:0/785207242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c1589c0 0x7fe74c06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.266+0000 7fe750975640 1 -- 172.21.15.70:0/785207242 >> 172.21.15.70:0/785207242 conn(0x7fe74c0ce4a0 msgr2=0x7fe74c0ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.266+0000 7fe750975640 1 -- 172.21.15.70:0/785207242 shutdown_connections 2023-08-25T09:39:02.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.266+0000 7fe750975640 1 -- 172.21.15.70:0/785207242 wait complete. 2023-08-25T09:39:02.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.269+0000 7fe750975640 1 Processor -- start 2023-08-25T09:39:02.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.269+0000 7fe750975640 1 -- start start 2023-08-25T09:39:02.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.271+0000 7fe750975640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe74c06c410 0x7fe74c0c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.271+0000 7fe750975640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c1589c0 0x7fe74c1f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.271+0000 7fe750975640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c15c640 0x7fe74c0c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe750975640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c0c8990 con 0x7fe74c1589c0 2023-08-25T09:39:02.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe74ad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe74c06c410 0x7fe74c0c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe74a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c1589c0 0x7fe74c1f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe74ad9d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe74c06c410 0x7fe74c0c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38800/0 (socket says 172.21.15.70:38800) 2023-08-25T09:39:02.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe74ad9d640 1 -- 172.21.15.70:0/1380952014 learned_addr learned my addr 172.21.15.70:0/1380952014 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe74b59e640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c15c640 0x7fe74c0c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c1f2320 con 0x7fe74c06c410 2023-08-25T09:39:02.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.272+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74c0c83d0 con 0x7fe74c15c640 2023-08-25T09:39:02.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.273+0000 7fe74a59c640 1 -- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c15c640 msgr2=0x7fe74c0c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.273+0000 7fe74a59c640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c15c640 0x7fe74c0c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.273+0000 7fe74a59c640 1 -- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe74c06c410 msgr2=0x7fe74c0c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.273+0000 7fe74a59c640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe74c06c410 0x7fe74c0c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.273+0000 7fe74a59c640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74400d050 con 0x7fe74c1589c0 2023-08-25T09:39:02.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.274+0000 7fe74a59c640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c1589c0 0x7fe74c1f1de0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe744036000 tx=0x7fe744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.275+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744041840 con 0x7fe74c1589c0 2023-08-25T09:39:02.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.275+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74c0c86b0 con 0x7fe74c1589c0 2023-08-25T09:39:02.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.276+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74c1f18c0 con 0x7fe74c1589c0 2023-08-25T09:39:02.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.279+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7440419c0 con 0x7fe74c1589c0 2023-08-25T09:39:02.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.280+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744041d70 con 0x7fe74c1589c0 2023-08-25T09:39:02.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.280+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe74404b400 con 0x7fe74c1589c0 2023-08-25T09:39:02.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.282+0000 7fe73b7fe640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe73006a170 0x7fe73006c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.282+0000 7fe74ad9d640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe73006a170 0x7fe73006c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.284+0000 7fe74ad9d640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe73006a170 0x7fe73006c630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe73c007a60 tx=0x7fe73c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.284+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe7440ae710 con 0x7fe74c1589c0 2023-08-25T09:39:02.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.285+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe728005250 con 0x7fe74c1589c0 2023-08-25T09:39:02.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.295+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe744079f20 con 0x7fe74c1589c0 2023-08-25T09:39:02.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.547+0000 7f82f9c57640 1 -- 172.21.15.70:0/3835888791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f82d4005500 con 0x7f82f40b0610 2023-08-25T09:39:02.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.547+0000 7f82dbfff640 1 -- 172.21.15.70:0/3835888791 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f82e4017020 con 0x7f82f40b0610 2023-08-25T09:39:02.552 INFO:teuthology.orchestra.run.smithi070.stdout:55834574855 2023-08-25T09:39:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 -- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82e006a1a0 msgr2=0x7f82e006c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82e006a1a0 0x7f82e006c660 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f82ec038000 tx=0x7f82ec032000 comp rx=0 tx=0).stop 2023-08-25T09:39:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 -- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 msgr2=0x7f82f40c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 0x7f82f40c3f90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f82e400c4a0 tx=0x7f82e400c960 comp rx=0 tx=0).stop 2023-08-25T09:39:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 -- 172.21.15.70:0/3835888791 shutdown_connections 2023-08-25T09:39:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82e006a1a0 0x7f82e006c660 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f415c790 0x7f82f40c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f4159b60 0x7f82f40710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 --2- 172.21.15.70:0/3835888791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f40b0610 0x7f82f40c3f90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.552+0000 7f82d9ffb640 1 -- 172.21.15.70:0/3835888791 >> 172.21.15.70:0/3835888791 conn(0x7f82f4069c40 msgr2=0x7f82f40afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.553+0000 7f82d9ffb640 1 -- 172.21.15.70:0/3835888791 shutdown_connections 2023-08-25T09:39:02.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.553+0000 7f82d9ffb640 1 -- 172.21.15.70:0/3835888791 wait complete. 2023-08-25T09:39:02.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-08-25T09:39:02.581 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.607+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f818414e660 con 0x7f81840b0610 2023-08-25T09:39:02.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.608+0000 7f816bfff640 1 -- 172.21.15.70:0/3774748694 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f818414e660 con 0x7f81840b0610 2023-08-25T09:39:02.609 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f817006a0a0 msgr2=0x7f817006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f817006a0a0 0x7f817006c560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8178039000 tx=0x7f817800a040 comp rx=0 tx=0).stop 2023-08-25T09:39:02.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 msgr2=0x7f818414daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 0x7f818414daa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f817400c400 tx=0x7f817400c8c0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 shutdown_connections 2023-08-25T09:39:02.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f817006a0a0 0x7f817006c560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.610+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f818415c790 0x7f818414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.611+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8184159b60 0x7f8184150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.611+0000 7f818a142640 1 --2- 172.21.15.70:0/3774748694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81840b0610 0x7f818414daa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.611+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 >> 172.21.15.70:0/3774748694 conn(0x7f8184069c40 msgr2=0x7f81840aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.611+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 shutdown_connections 2023-08-25T09:39:02.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.611+0000 7f818a142640 1 -- 172.21.15.70:0/3774748694 wait complete. 2023-08-25T09:39:02.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.612+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbe78005500 con 0x7fbe90005fe0 2023-08-25T09:39:02.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.613+0000 7fbe957fa640 1 -- 172.21.15.70:0/1380495849 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe84079a70 con 0x7fbe90005fe0 2023-08-25T09:39:02.615 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.616+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7c06a050 msgr2=0x7fbe7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.616+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7c06a050 0x7fbe7c06c510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe90048670 tx=0x7fbe8c008040 comp rx=0 tx=0).stop 2023-08-25T09:39:02.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.616+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 msgr2=0x7fbe900d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.616+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 0x7fbe900d27f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe84030780 tx=0x7fbe840046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 shutdown_connections 2023-08-25T09:39:02.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7c06a050 0x7fbe7c06c510 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe900df500 0x7fbe900396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe9004fc70 0x7fbe9017c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 --2- 172.21.15.70:0/1380495849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe90005fe0 0x7fbe900d27f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 >> 172.21.15.70:0/1380495849 conn(0x7fbe90054c10 msgr2=0x7fbe900381d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 shutdown_connections 2023-08-25T09:39:02.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fbe9deb0640 1 -- 172.21.15.70:0/1380495849 wait complete. 2023-08-25T09:39:02.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.617+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb0c40c76d0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.618+0000 7fb0ab7fe640 1 -- 172.21.15.70:0/3421092761 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb0c40c76d0 con 0x7fb0c40b57a0 2023-08-25T09:39:02.621 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.622+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0b006a0d0 msgr2=0x7fb0b006c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.622+0000 7fb0c9112640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0b006a0d0 0x7fb0b006c590 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc001ef0 tx=0x7fb0bc009040 comp rx=0 tx=0).stop 2023-08-25T09:39:02.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.622+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b57a0 msgr2=0x7fb0c40c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.622+0000 7fb0c9112640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b57a0 0x7fb0c40c9a20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8033000 tx=0x7fb0b80046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.622+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 shutdown_connections 2023-08-25T09:39:02.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0b006a0d0 0x7fb0b006c590 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0c415cc50 0x7fb0c415f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0c40b57a0 0x7fb0c40c9a20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 --2- 172.21.15.70:0/3421092761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0c40b4ae0 0x7fb0c4144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 >> 172.21.15.70:0/3421092761 conn(0x7fb0c40cf3d0 msgr2=0x7fb0c40b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 shutdown_connections 2023-08-25T09:39:02.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.623+0000 7fb0c9112640 1 -- 172.21.15.70:0/3421092761 wait complete. 2023-08-25T09:39:02.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.630+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1000058d60 con 0x7f10000588d0 2023-08-25T09:39:02.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.631+0000 7f0ffeffd640 1 -- 172.21.15.70:0/1985742391 <== mon.1 v2:172.21.15.155:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1000058d60 con 0x7f10000588d0 2023-08-25T09:39:02.633 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-08-25T09:39:02.634 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fec06a0a0 msgr2=0x7f0fec06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fec06a0a0 0x7f0fec06c560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1000008630 tx=0x7f10080ab000 comp rx=0 tx=0).stop 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 msgr2=0x7f100003da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 0x7f100003da00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800c390 tx=0x7f0ff800c850 comp rx=0 tx=0).stop 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 shutdown_connections 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fec06a0a0 0x7f0fec06c560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10000588d0 0x7f100003da00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f100003c0b0 0x7f100016b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 --2- 172.21.15.70:0/1985742391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10000061b0 0x7f10000dfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 >> 172.21.15.70:0/1985742391 conn(0x7f1000055a20 msgr2=0x7f1000050070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 shutdown_connections 2023-08-25T09:39:02.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.635+0000 7f1010599640 1 -- 172.21.15.70:0/1985742391 wait complete. 2023-08-25T09:39:02.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.636+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f18a40050c0 con 0x7f18ac005150 2023-08-25T09:39:02.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.637+0000 7f18c0ff9640 1 -- 172.21.15.70:0/3344710943 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f18b8057660 con 0x7f18ac005150 2023-08-25T09:39:02.639 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-08-25T09:39:02.641 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 msgr2=0x7f18b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f18c406a2b0 tx=0x7f18bc002140 comp rx=0 tx=0).stop 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 msgr2=0x7f18c40adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 0x7f18c40adee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b950 tx=0x7f18b800be10 comp rx=0 tx=0).stop 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 shutdown_connections 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c4167840 0x7f18c4070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c4167400 0x7f18c40c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 --2- 172.21.15.70:0/3344710943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18ac005150 0x7f18c40adee0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 >> 172.21.15.70:0/3344710943 conn(0x7f18c4069c40 msgr2=0x7f18c40ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 shutdown_connections 2023-08-25T09:39:02.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.641+0000 7f18ca34e640 1 -- 172.21.15.70:0/3344710943 wait complete. 2023-08-25T09:39:02.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.644+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb894005500 con 0x7fb8ac159b60 2023-08-25T09:39:02.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.645+0000 7fb8a2ffd640 1 -- 172.21.15.70:0/819283853 <== mon.2 v2:172.21.15.155:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb89c018020 con 0x7fb8ac159b60 2023-08-25T09:39:02.646 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.647+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb89006a120 msgr2=0x7fb89006c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.647+0000 7fb8b475d640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb89006a120 0x7fb89006c5e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8036000 tx=0x7fb8a8030000 comp rx=0 tx=0).stop 2023-08-25T09:39:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.647+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 msgr2=0x7fb8ac06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.647+0000 7fb8b475d640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac06f110 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb89c00c4a0 tx=0x7fb89c00c960 comp rx=0 tx=0).stop 2023-08-25T09:39:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 shutdown_connections 2023-08-25T09:39:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb89006a120 0x7fb89006c5e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8ac15cea0 0x7fb8ac071190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8ac159b60 0x7fb8ac06f110 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 --2- 172.21.15.70:0/819283853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8ac0aff00 0x7fb8ac1436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 >> 172.21.15.70:0/819283853 conn(0x7fb8ac069c40 msgr2=0x7fb8ac150430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 shutdown_connections 2023-08-25T09:39:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.648+0000 7fb8b475d640 1 -- 172.21.15.70:0/819283853 wait complete. 2023-08-25T09:39:02.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-08-25T09:39:02.652 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.661 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-08-25T09:39:02.661 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-08-25T09:39:02.664 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-08-25T09:39:02.666 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.667+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe728005500 con 0x7fe74c1589c0 2023-08-25T09:39:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.667+0000 7fe73b7fe640 1 -- 172.21.15.70:0/1380952014 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe744047070 con 0x7fe74c1589c0 2023-08-25T09:39:02.668 INFO:teuthology.orchestra.run.smithi070.stdout:55834574854 2023-08-25T09:39:02.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe73006a170 msgr2=0x7fe73006c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe73006a170 0x7fe73006c630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe73c007a60 tx=0x7fe73c008040 comp rx=0 tx=0).stop 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c1589c0 msgr2=0x7fe74c1f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c1589c0 0x7fe74c1f1de0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe744036000 tx=0x7fe744004a40 comp rx=0 tx=0).stop 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 shutdown_connections 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe73006a170 0x7fe73006c630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe74c15c640 0x7fe74c0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe74c1589c0 0x7fe74c1f1de0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 --2- 172.21.15.70:0/1380952014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe74c06c410 0x7fe74c0c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 >> 172.21.15.70:0/1380952014 conn(0x7fe74c0ce4a0 msgr2=0x7fe74c0ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 shutdown_connections 2023-08-25T09:39:02.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.669+0000 7fe750975640 1 -- 172.21.15.70:0/1380952014 wait complete. 2023-08-25T09:39:02.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-08-25T09:39:02.686 DEBUG:teuthology.parallel:result is None 2023-08-25T09:39:02.686 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-25T09:39:02.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:39:02.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.936+0000 7f5bdf55d640 1 Processor -- start 2023-08-25T09:39:02.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.936+0000 7f5bdf55d640 1 -- start start 2023-08-25T09:39:02.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdf55d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 0x7f5bd80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdf55d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5bd80b2b40 0x7f5bd80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdf55d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd806f050 con 0x7f5bd80ac870 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdf55d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd806f330 con 0x7f5bd80b2b40 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdf55d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd806f610 con 0x7f5bd815cb40 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdcaf8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5bd80ac870 0x7f5bd8160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35514/0 (socket says 172.21.15.70:35514) 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdcaf8640 1 -- 172.21.15.70:0/2430897010 learned_addr learned my addr 172.21.15.70:0/2430897010 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bddafa640 1 --2- 172.21.15.70:0/2430897010 >> v2:172.21.15.155:3300/0 conn(0x7f5bd80b2b40 0x7f5bd80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 0x7f5bd80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.938+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630297645 0 0) 0x7f5bd806f050 con 0x7f5bd80ac870 2023-08-25T09:39:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc4003520 con 0x7f5bd80ac870 2023-08-25T09:39:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3374806089 0 0) 0x7f5bc4003520 con 0x7f5bd80ac870 2023-08-25T09:39:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 msgr2=0x7f5bd80ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 0x7f5bd80ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 >> v2:172.21.15.155:3300/0 conn(0x7f5bd80b2b40 msgr2=0x7f5bd80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 --2- 172.21.15.70:0/2430897010 >> v2:172.21.15.155:3300/0 conn(0x7f5bd80b2b40 0x7f5bd80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd806f8f0 con 0x7f5bd80ac870 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5bc8001670 con 0x7f5bd80ac870 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 >> v1:172.21.15.70:6789/0 conn(0x7f5bd80ac870 legacy=0x7f5bd8160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 0x7f5bc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4004d00 0x7f5bc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd806f330 con 0x7f5bc4003970 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd806f610 con 0x7f5bc4004d00 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc400b980 con 0x7f5bd815cb40 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bddafa640 1 --2- 172.21.15.70:0/2430897010 >> v2:172.21.15.155:3300/0 conn(0x7f5bd80b2b40 0x7f5bd80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.939+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.940+0000 7f5bdd2f9640 1 -- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 msgr2=0x7f5bc40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.940+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd815cb40 0x7f5bc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.940+0000 7f5bdd2f9640 1 -- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4004d00 msgr2=0x7f5bc4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.940+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4004d00 0x7f5bc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.940+0000 7f5bdd2f9640 1 -- 172.21.15.70:0/2430897010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc4003520 con 0x7f5bc4003970 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009af0 tx=0x7f5bc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a75cbb08d6e107 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd402c710 con 0x7f5bc4003970 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd402c870 con 0x7f5bc4003970 2023-08-25T09:39:02.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bcffff640 1 -- 172.21.15.70:0/2430897010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd402cc00 con 0x7f5bc4003970 2023-08-25T09:39:02.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bdf55d640 1 -- 172.21.15.70:0/2430897010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc4003970 msgr2=0x7f5bc4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/2430897010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bc4003970 0x7f5bc4003d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009af0 tx=0x7f5bc40057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bddafa640 1 -- 172.21.15.70:0/2430897010 reap_dead start 2023-08-25T09:39:02.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bdf55d640 1 -- 172.21.15.70:0/2430897010 shutdown_connections 2023-08-25T09:39:02.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.941+0000 7f5bdf55d640 1 -- 172.21.15.70:0/2430897010 >> 172.21.15.70:0/2430897010 conn(0x7f5bd8069850 msgr2=0x7f5bd8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:02.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.942+0000 7f5bdf55d640 1 -- 172.21.15.70:0/2430897010 shutdown_connections 2023-08-25T09:39:02.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.942+0000 7f5bdf55d640 1 -- 172.21.15.70:0/2430897010 wait complete. 2023-08-25T09:39:02.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.943+0000 7f5bdf55d640 1 Processor -- start 2023-08-25T09:39:02.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.943+0000 7f5bdf55d640 1 -- start start 2023-08-25T09:39:02.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdf55d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4003970 0x7f5bd8146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdf55d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 0x7f5bd815fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdf55d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd814e9c0 0x7f5bd814ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdf55d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8160370 con 0x7f5bd815f9b0 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdf55d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd814b400 con 0x7f5bc4003970 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdf55d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd815f050 con 0x7f5bd814e9c0 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdcaf8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 0x7f5bd815fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdcaf8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 0x7f5bd815fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47440/0 (socket says 172.21.15.70:47440) 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.944+0000 7f5bdcaf8640 1 -- 172.21.15.70:0/375284914 learned_addr learned my addr 172.21.15.70:0/375284914 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4003970 0x7f5bd8146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdcaf8640 1 -- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd814e9c0 msgr2=0x7f5bd814ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bddafa640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd814e9c0 0x7f5bd814ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdcaf8640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd814e9c0 0x7f5bd814ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdcaf8640 1 -- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4003970 msgr2=0x7f5bd8146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdcaf8640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4003970 0x7f5bd8146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdcaf8640 1 -- 172.21.15.70:0/375284914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd815e4d0 con 0x7f5bd815f9b0 2023-08-25T09:39:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdcaf8640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 0x7f5bd815fe30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c840 tx=0x7f5bc800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc80074e0 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc8007660 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc8004a20 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd815e7b0 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.945+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd815ed70 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.946+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd814e6e0 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.947+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5bc8004b80 con 0x7f5bd815f9b0 2023-08-25T09:39:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.948+0000 7f5bcdffb640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bbc06a080 0x7f5bbc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.948+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5bc8087a50 con 0x7f5bd815f9b0 2023-08-25T09:39:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.948+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bbc06a080 0x7f5bbc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.949+0000 7f5bdd2f9640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bbc06a080 0x7f5bbc06c540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4036000 tx=0x7f5bd4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:02.952+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5bd814e6e0 con 0x7f5bd815f9b0 2023-08-25T09:39:03.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.123+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bd806ece0 con 0x7f5bbc06a080 2023-08-25T09:39:03.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.131+0000 7f5bcdffb640 1 -- 172.21.15.70:0/375284914 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280649 (secure 0 0 0) 0x7f5bd4034440 con 0x7f5bbc06a080 2023-08-25T09:39:03.133 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:03.137 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:39:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bbc06a080 msgr2=0x7f5bbc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bbc06a080 0x7f5bbc06c540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4036000 tx=0x7f5bd4030000 comp rx=0 tx=0).stop 2023-08-25T09:39:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 msgr2=0x7f5bd815fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 0x7f5bd815fe30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5bc800c840 tx=0x7f5bc800cd00 comp rx=0 tx=0).stop 2023-08-25T09:39:03.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 shutdown_connections 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bbc06a080 0x7f5bbc06c540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd814e9c0 0x7f5bd814ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd815f9b0 0x7f5bd815fe30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 --2- 172.21.15.70:0/375284914 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bc4003970 0x7f5bd8146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.137+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 >> 172.21.15.70:0/375284914 conn(0x7f5bd8069850 msgr2=0x7f5bd80add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.138+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 shutdown_connections 2023-08-25T09:39:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.138+0000 7f5bdf55d640 1 -- 172.21.15.70:0/375284914 wait complete. 2023-08-25T09:39:03.156 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-08-25T09:39:01.319755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1856,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560643,"statfs":{"total":1600000000,"available":1598098932,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.248505"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169943+0000","last_change":"2023-08-25T09:38:53.105814+0000","last_active":"2023-08-25T09:38:54.169943+0000","last_peered":"2023-08-25T09:38:54.169943+0000","last_clean":"2023-08-25T09:38:54.169943+0000","last_became_active":"2023-08-25T09:38:53.103612+0000","last_became_peered":"2023-08-25T09:38:53.103612+0000","last_unstale":"2023-08-25T09:38:54.169943+0000","last_undegraded":"2023-08-25T09:38:54.169943+0000","last_fullsized":"2023-08-25T09:38:54.169943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185531+0000","last_change":"2023-08-25T09:38:53.103005+0000","last_active":"2023-08-25T09:38:54.185531+0000","last_peered":"2023-08-25T09:38:54.185531+0000","last_clean":"2023-08-25T09:38:54.185531+0000","last_became_active":"2023-08-25T09:38:53.102634+0000","last_became_peered":"2023-08-25T09:38:53.102634+0000","last_unstale":"2023-08-25T09:38:54.185531+0000","last_undegraded":"2023-08-25T09:38:54.185531+0000","last_fullsized":"2023-08-25T09:38:54.185531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214701+0000","last_change":"2023-08-25T09:38:53.111188+0000","last_active":"2023-08-25T09:38:54.214701+0000","last_peered":"2023-08-25T09:38:54.214701+0000","last_clean":"2023-08-25T09:38:54.214701+0000","last_became_active":"2023-08-25T09:38:53.108190+0000","last_became_peered":"2023-08-25T09:38:53.108190+0000","last_unstale":"2023-08-25T09:38:54.214701+0000","last_undegraded":"2023-08-25T09:38:54.214701+0000","last_fullsized":"2023-08-25T09:38:54.214701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:16.468452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214312+0000","last_change":"2023-08-25T09:38:53.103968+0000","last_active":"2023-08-25T09:38:54.214312+0000","last_peered":"2023-08-25T09:38:54.214312+0000","last_clean":"2023-08-25T09:38:54.214312+0000","last_became_active":"2023-08-25T09:38:53.103676+0000","last_became_peered":"2023-08-25T09:38:53.103676+0000","last_unstale":"2023-08-25T09:38:54.214312+0000","last_undegraded":"2023-08-25T09:38:54.214312+0000","last_fullsized":"2023-08-25T09:38:54.214312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170566+0000","last_change":"2023-08-25T09:38:53.110486+0000","last_active":"2023-08-25T09:38:54.170566+0000","last_peered":"2023-08-25T09:38:54.170566+0000","last_clean":"2023-08-25T09:38:54.170566+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.170566+0000","last_undegraded":"2023-08-25T09:38:54.170566+0000","last_fullsized":"2023-08-25T09:38:54.170566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:15.681428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077139+0000","last_change":"2023-08-25T09:38:53.082883+0000","last_active":"2023-08-25T09:38:54.077139+0000","last_peered":"2023-08-25T09:38:54.077139+0000","last_clean":"2023-08-25T09:38:54.077139+0000","last_became_active":"2023-08-25T09:38:53.081892+0000","last_became_peered":"2023-08-25T09:38:53.081892+0000","last_unstale":"2023-08-25T09:38:54.077139+0000","last_undegraded":"2023-08-25T09:38:54.077139+0000","last_fullsized":"2023-08-25T09:38:54.077139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:25:18.218904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077432+0000","last_change":"2023-08-25T09:38:53.094986+0000","last_active":"2023-08-25T09:38:54.077432+0000","last_peered":"2023-08-25T09:38:54.077432+0000","last_clean":"2023-08-25T09:38:54.077432+0000","last_became_active":"2023-08-25T09:38:53.094533+0000","last_became_peered":"2023-08-25T09:38:53.094533+0000","last_unstale":"2023-08-25T09:38:54.077432+0000","last_undegraded":"2023-08-25T09:38:54.077432+0000","last_fullsized":"2023-08-25T09:38:54.077432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:44.551022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167144+0000","last_change":"2023-08-25T09:38:53.097497+0000","last_active":"2023-08-25T09:38:54.167144+0000","last_peered":"2023-08-25T09:38:54.167144+0000","last_clean":"2023-08-25T09:38:54.167144+0000","last_became_active":"2023-08-25T09:38:53.096835+0000","last_became_peered":"2023-08-25T09:38:53.096835+0000","last_unstale":"2023-08-25T09:38:54.167144+0000","last_undegraded":"2023-08-25T09:38:54.167144+0000","last_fullsized":"2023-08-25T09:38:54.167144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168759+0000","last_change":"2023-08-25T09:38:53.111823+0000","last_active":"2023-08-25T09:38:54.168759+0000","last_peered":"2023-08-25T09:38:54.168759+0000","last_clean":"2023-08-25T09:38:54.168759+0000","last_became_active":"2023-08-25T09:38:53.107075+0000","last_became_peered":"2023-08-25T09:38:53.107075+0000","last_unstale":"2023-08-25T09:38:54.168759+0000","last_undegraded":"2023-08-25T09:38:54.168759+0000","last_fullsized":"2023-08-25T09:38:54.168759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167237+0000","last_change":"2023-08-25T09:38:53.097405+0000","last_active":"2023-08-25T09:38:54.167237+0000","last_peered":"2023-08-25T09:38:54.167237+0000","last_clean":"2023-08-25T09:38:54.167237+0000","last_became_active":"2023-08-25T09:38:53.097063+0000","last_became_peered":"2023-08-25T09:38:53.097063+0000","last_unstale":"2023-08-25T09:38:54.167237+0000","last_undegraded":"2023-08-25T09:38:54.167237+0000","last_fullsized":"2023-08-25T09:38:54.167237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167432+0000","last_change":"2023-08-25T09:38:53.097713+0000","last_active":"2023-08-25T09:38:54.167432+0000","last_peered":"2023-08-25T09:38:54.167432+0000","last_clean":"2023-08-25T09:38:54.167432+0000","last_became_active":"2023-08-25T09:38:53.097250+0000","last_became_peered":"2023-08-25T09:38:53.097250+0000","last_unstale":"2023-08-25T09:38:54.167432+0000","last_undegraded":"2023-08-25T09:38:54.167432+0000","last_fullsized":"2023-08-25T09:38:54.167432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170076+0000","last_change":"2023-08-25T09:38:53.098525+0000","last_active":"2023-08-25T09:38:54.170076+0000","last_peered":"2023-08-25T09:38:54.170076+0000","last_clean":"2023-08-25T09:38:54.170076+0000","last_became_active":"2023-08-25T09:38:53.097701+0000","last_became_peered":"2023-08-25T09:38:53.097701+0000","last_unstale":"2023-08-25T09:38:54.170076+0000","last_undegraded":"2023-08-25T09:38:54.170076+0000","last_fullsized":"2023-08-25T09:38:54.170076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079156+0000","last_change":"2023-08-25T09:38:53.110156+0000","last_active":"2023-08-25T09:38:54.079156+0000","last_peered":"2023-08-25T09:38:54.079156+0000","last_clean":"2023-08-25T09:38:54.079156+0000","last_became_active":"2023-08-25T09:38:53.109921+0000","last_became_peered":"2023-08-25T09:38:53.109921+0000","last_unstale":"2023-08-25T09:38:54.079156+0000","last_undegraded":"2023-08-25T09:38:54.079156+0000","last_fullsized":"2023-08-25T09:38:54.079156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167535+0000","last_change":"2023-08-25T09:38:53.102316+0000","last_active":"2023-08-25T09:38:54.167535+0000","last_peered":"2023-08-25T09:38:54.167535+0000","last_clean":"2023-08-25T09:38:54.167535+0000","last_became_active":"2023-08-25T09:38:53.100943+0000","last_became_peered":"2023-08-25T09:38:53.100943+0000","last_unstale":"2023-08-25T09:38:54.167535+0000","last_undegraded":"2023-08-25T09:38:54.167535+0000","last_fullsized":"2023-08-25T09:38:54.167535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214539+0000","last_change":"2023-08-25T09:38:53.111165+0000","last_active":"2023-08-25T09:38:54.214539+0000","last_peered":"2023-08-25T09:38:54.214539+0000","last_clean":"2023-08-25T09:38:54.214539+0000","last_became_active":"2023-08-25T09:38:53.108285+0000","last_became_peered":"2023-08-25T09:38:53.108285+0000","last_unstale":"2023-08-25T09:38:54.214539+0000","last_undegraded":"2023-08-25T09:38:54.214539+0000","last_fullsized":"2023-08-25T09:38:54.214539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077362+0000","last_change":"2023-08-25T09:38:53.089749+0000","last_active":"2023-08-25T09:38:54.077362+0000","last_peered":"2023-08-25T09:38:54.077362+0000","last_clean":"2023-08-25T09:38:54.077362+0000","last_became_active":"2023-08-25T09:38:53.088753+0000","last_became_peered":"2023-08-25T09:38:53.088753+0000","last_unstale":"2023-08-25T09:38:54.077362+0000","last_undegraded":"2023-08-25T09:38:54.077362+0000","last_fullsized":"2023-08-25T09:38:54.077362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:35.174323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078331+0000","last_change":"2023-08-25T09:38:53.100818+0000","last_active":"2023-08-25T09:38:54.078331+0000","last_peered":"2023-08-25T09:38:54.078331+0000","last_clean":"2023-08-25T09:38:54.078331+0000","last_became_active":"2023-08-25T09:38:53.100555+0000","last_became_peered":"2023-08-25T09:38:53.100555+0000","last_unstale":"2023-08-25T09:38:54.078331+0000","last_undegraded":"2023-08-25T09:38:54.078331+0000","last_fullsized":"2023-08-25T09:38:54.078331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215037+0000","last_change":"2023-08-25T09:38:53.105302+0000","last_active":"2023-08-25T09:38:54.215037+0000","last_peered":"2023-08-25T09:38:54.215037+0000","last_clean":"2023-08-25T09:38:54.215037+0000","last_became_active":"2023-08-25T09:38:53.103402+0000","last_became_peered":"2023-08-25T09:38:53.103402+0000","last_unstale":"2023-08-25T09:38:54.215037+0000","last_undegraded":"2023-08-25T09:38:54.215037+0000","last_fullsized":"2023-08-25T09:38:54.215037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171407+0000","last_change":"2023-08-25T09:38:53.099130+0000","last_active":"2023-08-25T09:38:54.171407+0000","last_peered":"2023-08-25T09:38:54.171407+0000","last_clean":"2023-08-25T09:38:54.171407+0000","last_became_active":"2023-08-25T09:38:53.094676+0000","last_became_peered":"2023-08-25T09:38:53.094676+0000","last_unstale":"2023-08-25T09:38:54.171407+0000","last_undegraded":"2023-08-25T09:38:54.171407+0000","last_fullsized":"2023-08-25T09:38:54.171407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:05.648183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167405+0000","last_change":"2023-08-25T09:38:53.111895+0000","last_active":"2023-08-25T09:38:54.167405+0000","last_peered":"2023-08-25T09:38:54.167405+0000","last_clean":"2023-08-25T09:38:54.167405+0000","last_became_active":"2023-08-25T09:38:53.111626+0000","last_became_peered":"2023-08-25T09:38:53.111626+0000","last_unstale":"2023-08-25T09:38:54.167405+0000","last_undegraded":"2023-08-25T09:38:54.167405+0000","last_fullsized":"2023-08-25T09:38:54.167405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167554+0000","last_change":"2023-08-25T09:38:53.104999+0000","last_active":"2023-08-25T09:38:54.167554+0000","last_peered":"2023-08-25T09:38:54.167554+0000","last_clean":"2023-08-25T09:38:54.167554+0000","last_became_active":"2023-08-25T09:38:53.104760+0000","last_became_peered":"2023-08-25T09:38:53.104760+0000","last_unstale":"2023-08-25T09:38:54.167554+0000","last_undegraded":"2023-08-25T09:38:54.167554+0000","last_fullsized":"2023-08-25T09:38:54.167554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:17.840144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078687+0000","last_change":"2023-08-25T09:38:53.106873+0000","last_active":"2023-08-25T09:38:54.078687+0000","last_peered":"2023-08-25T09:38:54.078687+0000","last_clean":"2023-08-25T09:38:54.078687+0000","last_became_active":"2023-08-25T09:38:53.106581+0000","last_became_peered":"2023-08-25T09:38:53.106581+0000","last_unstale":"2023-08-25T09:38:54.078687+0000","last_undegraded":"2023-08-25T09:38:54.078687+0000","last_fullsized":"2023-08-25T09:38:54.078687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168645+0000","last_change":"2023-08-25T09:38:53.112057+0000","last_active":"2023-08-25T09:38:54.168645+0000","last_peered":"2023-08-25T09:38:54.168645+0000","last_clean":"2023-08-25T09:38:54.168645+0000","last_became_active":"2023-08-25T09:38:53.111710+0000","last_became_peered":"2023-08-25T09:38:53.111710+0000","last_unstale":"2023-08-25T09:38:54.168645+0000","last_undegraded":"2023-08-25T09:38:54.168645+0000","last_fullsized":"2023-08-25T09:38:54.168645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167656+0000","last_change":"2023-08-25T09:38:53.101206+0000","last_active":"2023-08-25T09:38:54.167656+0000","last_peered":"2023-08-25T09:38:54.167656+0000","last_clean":"2023-08-25T09:38:54.167656+0000","last_became_active":"2023-08-25T09:38:53.100575+0000","last_became_peered":"2023-08-25T09:38:53.100575+0000","last_unstale":"2023-08-25T09:38:54.167656+0000","last_undegraded":"2023-08-25T09:38:54.167656+0000","last_fullsized":"2023-08-25T09:38:54.167656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:44.735256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170902+0000","last_change":"2023-08-25T09:38:53.110658+0000","last_active":"2023-08-25T09:38:54.170902+0000","last_peered":"2023-08-25T09:38:54.170902+0000","last_clean":"2023-08-25T09:38:54.170902+0000","last_became_active":"2023-08-25T09:38:53.107553+0000","last_became_peered":"2023-08-25T09:38:53.107553+0000","last_unstale":"2023-08-25T09:38:54.170902+0000","last_undegraded":"2023-08-25T09:38:54.170902+0000","last_fullsized":"2023-08-25T09:38:54.170902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:35.169281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078368+0000","last_change":"2023-08-25T09:38:53.106509+0000","last_active":"2023-08-25T09:38:54.078368+0000","last_peered":"2023-08-25T09:38:54.078368+0000","last_clean":"2023-08-25T09:38:54.078368+0000","last_became_active":"2023-08-25T09:38:53.106227+0000","last_became_peered":"2023-08-25T09:38:53.106227+0000","last_unstale":"2023-08-25T09:38:54.078368+0000","last_undegraded":"2023-08-25T09:38:54.078368+0000","last_fullsized":"2023-08-25T09:38:54.078368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077761+0000","last_change":"2023-08-25T09:38:53.094823+0000","last_active":"2023-08-25T09:38:54.077761+0000","last_peered":"2023-08-25T09:38:54.077761+0000","last_clean":"2023-08-25T09:38:54.077761+0000","last_became_active":"2023-08-25T09:38:53.094160+0000","last_became_peered":"2023-08-25T09:38:53.094160+0000","last_unstale":"2023-08-25T09:38:54.077761+0000","last_undegraded":"2023-08-25T09:38:54.077761+0000","last_fullsized":"2023-08-25T09:38:54.077761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:05:17.873255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169761+0000","last_change":"2023-08-25T09:38:53.109739+0000","last_active":"2023-08-25T09:38:54.169761+0000","last_peered":"2023-08-25T09:38:54.169761+0000","last_clean":"2023-08-25T09:38:54.169761+0000","last_became_active":"2023-08-25T09:38:53.106937+0000","last_became_peered":"2023-08-25T09:38:53.106937+0000","last_unstale":"2023-08-25T09:38:54.169761+0000","last_undegraded":"2023-08-25T09:38:54.169761+0000","last_fullsized":"2023-08-25T09:38:54.169761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170229+0000","last_change":"2023-08-25T09:38:53.102490+0000","last_active":"2023-08-25T09:38:54.170229+0000","last_peered":"2023-08-25T09:38:54.170229+0000","last_clean":"2023-08-25T09:38:54.170229+0000","last_became_active":"2023-08-25T09:38:53.101594+0000","last_became_peered":"2023-08-25T09:38:53.101594+0000","last_unstale":"2023-08-25T09:38:54.170229+0000","last_undegraded":"2023-08-25T09:38:54.170229+0000","last_fullsized":"2023-08-25T09:38:54.170229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214623+0000","last_change":"2023-08-25T09:38:53.092816+0000","last_active":"2023-08-25T09:38:54.214623+0000","last_peered":"2023-08-25T09:38:54.214623+0000","last_clean":"2023-08-25T09:38:54.214623+0000","last_became_active":"2023-08-25T09:38:53.092392+0000","last_became_peered":"2023-08-25T09:38:53.092392+0000","last_unstale":"2023-08-25T09:38:54.214623+0000","last_undegraded":"2023-08-25T09:38:54.214623+0000","last_fullsized":"2023-08-25T09:38:54.214623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079002+0000","last_change":"2023-08-25T09:38:53.104234+0000","last_active":"2023-08-25T09:38:54.079002+0000","last_peered":"2023-08-25T09:38:54.079002+0000","last_clean":"2023-08-25T09:38:54.079002+0000","last_became_active":"2023-08-25T09:38:53.104004+0000","last_became_peered":"2023-08-25T09:38:53.104004+0000","last_unstale":"2023-08-25T09:38:54.079002+0000","last_undegraded":"2023-08-25T09:38:54.079002+0000","last_fullsized":"2023-08-25T09:38:54.079002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167124+0000","last_change":"2023-08-25T09:38:53.106635+0000","last_active":"2023-08-25T09:38:54.167124+0000","last_peered":"2023-08-25T09:38:54.167124+0000","last_clean":"2023-08-25T09:38:54.167124+0000","last_became_active":"2023-08-25T09:38:53.106105+0000","last_became_peered":"2023-08-25T09:38:53.106105+0000","last_unstale":"2023-08-25T09:38:54.167124+0000","last_undegraded":"2023-08-25T09:38:54.167124+0000","last_fullsized":"2023-08-25T09:38:54.167124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078322+0000","last_change":"2023-08-25T09:38:53.108366+0000","last_active":"2023-08-25T09:38:54.078322+0000","last_peered":"2023-08-25T09:38:54.078322+0000","last_clean":"2023-08-25T09:38:54.078322+0000","last_became_active":"2023-08-25T09:38:53.108123+0000","last_became_peered":"2023-08-25T09:38:53.108123+0000","last_unstale":"2023-08-25T09:38:54.078322+0000","last_undegraded":"2023-08-25T09:38:54.078322+0000","last_fullsized":"2023-08-25T09:38:54.078322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077197+0000","last_change":"2023-08-25T09:38:52.142687+0000","last_active":"2023-08-25T09:38:54.077197+0000","last_peered":"2023-08-25T09:38:54.077197+0000","last_clean":"2023-08-25T09:38:54.077197+0000","last_became_active":"2023-08-25T09:38:52.140330+0000","last_became_peered":"2023-08-25T09:38:52.140330+0000","last_unstale":"2023-08-25T09:38:54.077197+0000","last_undegraded":"2023-08-25T09:38:54.077197+0000","last_fullsized":"2023-08-25T09:38:54.077197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:42.836964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076792+0000","last_change":"2023-08-25T09:38:53.094431+0000","last_active":"2023-08-25T09:38:54.076792+0000","last_peered":"2023-08-25T09:38:54.076792+0000","last_clean":"2023-08-25T09:38:54.076792+0000","last_became_active":"2023-08-25T09:38:53.093756+0000","last_became_peered":"2023-08-25T09:38:53.093756+0000","last_unstale":"2023-08-25T09:38:54.076792+0000","last_undegraded":"2023-08-25T09:38:54.076792+0000","last_fullsized":"2023-08-25T09:38:54.076792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:32.683806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079876+0000","last_change":"2023-08-25T09:38:52.138724+0000","last_active":"2023-08-25T09:38:54.079876+0000","last_peered":"2023-08-25T09:38:54.079876+0000","last_clean":"2023-08-25T09:38:54.079876+0000","last_became_active":"2023-08-25T09:38:52.138323+0000","last_became_peered":"2023-08-25T09:38:52.138323+0000","last_unstale":"2023-08-25T09:38:54.079876+0000","last_undegraded":"2023-08-25T09:38:54.079876+0000","last_fullsized":"2023-08-25T09:38:54.079876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170651+0000","last_change":"2023-08-25T09:38:53.098073+0000","last_active":"2023-08-25T09:38:54.170651+0000","last_peered":"2023-08-25T09:38:54.170651+0000","last_clean":"2023-08-25T09:38:54.170651+0000","last_became_active":"2023-08-25T09:38:53.089135+0000","last_became_peered":"2023-08-25T09:38:53.089135+0000","last_unstale":"2023-08-25T09:38:54.170651+0000","last_undegraded":"2023-08-25T09:38:54.170651+0000","last_fullsized":"2023-08-25T09:38:54.170651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078034+0000","last_change":"2023-08-25T09:38:52.146382+0000","last_active":"2023-08-25T09:38:54.078034+0000","last_peered":"2023-08-25T09:38:54.078034+0000","last_clean":"2023-08-25T09:38:54.078034+0000","last_became_active":"2023-08-25T09:38:52.145954+0000","last_became_peered":"2023-08-25T09:38:52.145954+0000","last_unstale":"2023-08-25T09:38:54.078034+0000","last_undegraded":"2023-08-25T09:38:54.078034+0000","last_fullsized":"2023-08-25T09:38:54.078034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:02.701588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169581+0000","last_change":"2023-08-25T09:38:53.106145+0000","last_active":"2023-08-25T09:38:54.169581+0000","last_peered":"2023-08-25T09:38:54.169581+0000","last_clean":"2023-08-25T09:38:54.169581+0000","last_became_active":"2023-08-25T09:38:53.105502+0000","last_became_peered":"2023-08-25T09:38:53.105502+0000","last_unstale":"2023-08-25T09:38:54.169581+0000","last_undegraded":"2023-08-25T09:38:54.169581+0000","last_fullsized":"2023-08-25T09:38:54.169581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:01.061628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167472+0000","last_change":"2023-08-25T09:38:52.148928+0000","last_active":"2023-08-25T09:38:54.167472+0000","last_peered":"2023-08-25T09:38:54.167472+0000","last_clean":"2023-08-25T09:38:54.167472+0000","last_became_active":"2023-08-25T09:38:52.148227+0000","last_became_peered":"2023-08-25T09:38:52.148227+0000","last_unstale":"2023-08-25T09:38:54.167472+0000","last_undegraded":"2023-08-25T09:38:54.167472+0000","last_fullsized":"2023-08-25T09:38:54.167472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214435+0000","last_change":"2023-08-25T09:38:52.138014+0000","last_active":"2023-08-25T09:38:54.214435+0000","last_peered":"2023-08-25T09:38:54.214435+0000","last_clean":"2023-08-25T09:38:54.214435+0000","last_became_active":"2023-08-25T09:38:52.137777+0000","last_became_peered":"2023-08-25T09:38:52.137777+0000","last_unstale":"2023-08-25T09:38:54.214435+0000","last_undegraded":"2023-08-25T09:38:54.214435+0000","last_fullsized":"2023-08-25T09:38:54.214435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:10.214913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077886+0000","last_change":"2023-08-25T09:38:53.100050+0000","last_active":"2023-08-25T09:38:54.077886+0000","last_peered":"2023-08-25T09:38:54.077886+0000","last_clean":"2023-08-25T09:38:54.077886+0000","last_became_active":"2023-08-25T09:38:53.099406+0000","last_became_peered":"2023-08-25T09:38:53.099406+0000","last_unstale":"2023-08-25T09:38:54.077886+0000","last_undegraded":"2023-08-25T09:38:54.077886+0000","last_fullsized":"2023-08-25T09:38:54.077886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:46.330038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186222+0000","last_change":"2023-08-25T09:38:52.145560+0000","last_active":"2023-08-25T09:38:54.186222+0000","last_peered":"2023-08-25T09:38:54.186222+0000","last_clean":"2023-08-25T09:38:54.186222+0000","last_became_active":"2023-08-25T09:38:52.145324+0000","last_became_peered":"2023-08-25T09:38:52.145324+0000","last_unstale":"2023-08-25T09:38:54.186222+0000","last_undegraded":"2023-08-25T09:38:54.186222+0000","last_fullsized":"2023-08-25T09:38:54.186222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:47.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167557+0000","last_change":"2023-08-25T09:38:53.105196+0000","last_active":"2023-08-25T09:38:54.167557+0000","last_peered":"2023-08-25T09:38:54.167557+0000","last_clean":"2023-08-25T09:38:54.167557+0000","last_became_active":"2023-08-25T09:38:53.104948+0000","last_became_peered":"2023-08-25T09:38:53.104948+0000","last_unstale":"2023-08-25T09:38:54.167557+0000","last_undegraded":"2023-08-25T09:38:54.167557+0000","last_fullsized":"2023-08-25T09:38:54.167557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078179+0000","last_change":"2023-08-25T09:38:52.140961+0000","last_active":"2023-08-25T09:38:54.078179+0000","last_peered":"2023-08-25T09:38:54.078179+0000","last_clean":"2023-08-25T09:38:54.078179+0000","last_became_active":"2023-08-25T09:38:52.137710+0000","last_became_peered":"2023-08-25T09:38:52.137710+0000","last_unstale":"2023-08-25T09:38:54.078179+0000","last_undegraded":"2023-08-25T09:38:54.078179+0000","last_fullsized":"2023-08-25T09:38:54.078179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:09.490525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.083467+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.082859+0000","last_became_peered":"2023-08-25T09:38:53.082859+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215256+0000","last_change":"2023-08-25T09:38:52.148012+0000","last_active":"2023-08-25T09:38:54.215256+0000","last_peered":"2023-08-25T09:38:54.215256+0000","last_clean":"2023-08-25T09:38:54.215256+0000","last_became_active":"2023-08-25T09:38:52.147757+0000","last_became_peered":"2023-08-25T09:38:52.147757+0000","last_unstale":"2023-08-25T09:38:54.215256+0000","last_undegraded":"2023-08-25T09:38:54.215256+0000","last_fullsized":"2023-08-25T09:38:54.215256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:36.644423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170501+0000","last_change":"2023-08-25T09:38:53.110777+0000","last_active":"2023-08-25T09:38:54.170501+0000","last_peered":"2023-08-25T09:38:54.170501+0000","last_clean":"2023-08-25T09:38:54.170501+0000","last_became_active":"2023-08-25T09:38:53.108970+0000","last_became_peered":"2023-08-25T09:38:53.108970+0000","last_unstale":"2023-08-25T09:38:54.170501+0000","last_undegraded":"2023-08-25T09:38:54.170501+0000","last_fullsized":"2023-08-25T09:38:54.170501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080573+0000","last_change":"2023-08-25T09:38:52.144190+0000","last_active":"2023-08-25T09:38:54.080573+0000","last_peered":"2023-08-25T09:38:54.080573+0000","last_clean":"2023-08-25T09:38:54.080573+0000","last_became_active":"2023-08-25T09:38:52.143875+0000","last_became_peered":"2023-08-25T09:38:52.143875+0000","last_unstale":"2023-08-25T09:38:54.080573+0000","last_undegraded":"2023-08-25T09:38:54.080573+0000","last_fullsized":"2023-08-25T09:38:54.080573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215099+0000","last_change":"2023-08-25T09:38:53.106602+0000","last_active":"2023-08-25T09:38:54.215099+0000","last_peered":"2023-08-25T09:38:54.215099+0000","last_clean":"2023-08-25T09:38:54.215099+0000","last_became_active":"2023-08-25T09:38:53.106340+0000","last_became_peered":"2023-08-25T09:38:53.106340+0000","last_unstale":"2023-08-25T09:38:54.215099+0000","last_undegraded":"2023-08-25T09:38:54.215099+0000","last_fullsized":"2023-08-25T09:38:54.215099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076776+0000","last_change":"2023-08-25T09:38:52.140744+0000","last_active":"2023-08-25T09:38:54.076776+0000","last_peered":"2023-08-25T09:38:54.076776+0000","last_clean":"2023-08-25T09:38:54.076776+0000","last_became_active":"2023-08-25T09:38:52.140450+0000","last_became_peered":"2023-08-25T09:38:52.140450+0000","last_unstale":"2023-08-25T09:38:54.076776+0000","last_undegraded":"2023-08-25T09:38:54.076776+0000","last_fullsized":"2023-08-25T09:38:54.076776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:54:37.632752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077689+0000","last_change":"2023-08-25T09:38:53.100210+0000","last_active":"2023-08-25T09:38:54.077689+0000","last_peered":"2023-08-25T09:38:54.077689+0000","last_clean":"2023-08-25T09:38:54.077689+0000","last_became_active":"2023-08-25T09:38:53.099792+0000","last_became_peered":"2023-08-25T09:38:53.099792+0000","last_unstale":"2023-08-25T09:38:54.077689+0000","last_undegraded":"2023-08-25T09:38:54.077689+0000","last_fullsized":"2023-08-25T09:38:54.077689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:37.979410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077328+0000","last_change":"2023-08-25T09:38:52.132704+0000","last_active":"2023-08-25T09:38:54.077328+0000","last_peered":"2023-08-25T09:38:54.077328+0000","last_clean":"2023-08-25T09:38:54.077328+0000","last_became_active":"2023-08-25T09:38:52.132045+0000","last_became_peered":"2023-08-25T09:38:52.132045+0000","last_unstale":"2023-08-25T09:38:54.077328+0000","last_undegraded":"2023-08-25T09:38:54.077328+0000","last_fullsized":"2023-08-25T09:38:54.077328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:32:14.124629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186144+0000","last_change":"2023-08-25T09:38:53.095472+0000","last_active":"2023-08-25T09:38:54.186144+0000","last_peered":"2023-08-25T09:38:54.186144+0000","last_clean":"2023-08-25T09:38:54.186144+0000","last_became_active":"2023-08-25T09:38:53.095117+0000","last_became_peered":"2023-08-25T09:38:53.095117+0000","last_unstale":"2023-08-25T09:38:54.186144+0000","last_undegraded":"2023-08-25T09:38:54.186144+0000","last_fullsized":"2023-08-25T09:38:54.186144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169438+0000","last_change":"2023-08-25T09:38:52.142123+0000","last_active":"2023-08-25T09:38:54.169438+0000","last_peered":"2023-08-25T09:38:54.169438+0000","last_clean":"2023-08-25T09:38:54.169438+0000","last_became_active":"2023-08-25T09:38:52.141349+0000","last_became_peered":"2023-08-25T09:38:52.141349+0000","last_unstale":"2023-08-25T09:38:54.169438+0000","last_undegraded":"2023-08-25T09:38:54.169438+0000","last_fullsized":"2023-08-25T09:38:54.169438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078490+0000","last_change":"2023-08-25T09:38:53.101599+0000","last_active":"2023-08-25T09:38:54.078490+0000","last_peered":"2023-08-25T09:38:54.078490+0000","last_clean":"2023-08-25T09:38:54.078490+0000","last_became_active":"2023-08-25T09:38:53.101348+0000","last_became_peered":"2023-08-25T09:38:53.101348+0000","last_unstale":"2023-08-25T09:38:54.078490+0000","last_undegraded":"2023-08-25T09:38:54.078490+0000","last_fullsized":"2023-08-25T09:38:54.078490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167829+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:38:54.167829+0000","last_peered":"2023-08-25T09:38:54.167829+0000","last_clean":"2023-08-25T09:38:54.167829+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:38:54.167829+0000","last_undegraded":"2023-08-25T09:38:54.167829+0000","last_fullsized":"2023-08-25T09:38:54.167829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:32.241680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078306+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:38:54.078306+0000","last_peered":"2023-08-25T09:38:54.078306+0000","last_clean":"2023-08-25T09:38:54.078306+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:38:54.078306+0000","last_undegraded":"2023-08-25T09:38:54.078306+0000","last_fullsized":"2023-08-25T09:38:54.078306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:40:49.888170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170349+0000","last_change":"2023-08-25T09:38:52.145925+0000","last_active":"2023-08-25T09:38:54.170349+0000","last_peered":"2023-08-25T09:38:54.170349+0000","last_clean":"2023-08-25T09:38:54.170349+0000","last_became_active":"2023-08-25T09:38:52.145572+0000","last_became_peered":"2023-08-25T09:38:52.145572+0000","last_unstale":"2023-08-25T09:38:54.170349+0000","last_undegraded":"2023-08-25T09:38:54.170349+0000","last_fullsized":"2023-08-25T09:38:54.170349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:19:18.032882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076523+0000","last_change":"2023-08-25T09:38:53.098547+0000","last_active":"2023-08-25T09:38:54.076523+0000","last_peered":"2023-08-25T09:38:54.076523+0000","last_clean":"2023-08-25T09:38:54.076523+0000","last_became_active":"2023-08-25T09:38:53.098162+0000","last_became_peered":"2023-08-25T09:38:53.098162+0000","last_unstale":"2023-08-25T09:38:54.076523+0000","last_undegraded":"2023-08-25T09:38:54.076523+0000","last_fullsized":"2023-08-25T09:38:54.076523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:14.282266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076947+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:38:54.076947+0000","last_peered":"2023-08-25T09:38:54.076947+0000","last_clean":"2023-08-25T09:38:54.076947+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:38:54.076947+0000","last_undegraded":"2023-08-25T09:38:54.076947+0000","last_fullsized":"2023-08-25T09:38:54.076947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:41.461997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079667+0000","last_change":"2023-08-25T09:38:52.146575+0000","last_active":"2023-08-25T09:38:54.079667+0000","last_peered":"2023-08-25T09:38:54.079667+0000","last_clean":"2023-08-25T09:38:54.079667+0000","last_became_active":"2023-08-25T09:38:52.146263+0000","last_became_peered":"2023-08-25T09:38:52.146263+0000","last_unstale":"2023-08-25T09:38:54.079667+0000","last_undegraded":"2023-08-25T09:38:54.079667+0000","last_fullsized":"2023-08-25T09:38:54.079667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:58.354866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170732+0000","last_change":"2023-08-25T09:38:53.110355+0000","last_active":"2023-08-25T09:38:54.170732+0000","last_peered":"2023-08-25T09:38:54.170732+0000","last_clean":"2023-08-25T09:38:54.170732+0000","last_became_active":"2023-08-25T09:38:53.107934+0000","last_became_peered":"2023-08-25T09:38:53.107934+0000","last_unstale":"2023-08-25T09:38:54.170732+0000","last_undegraded":"2023-08-25T09:38:54.170732+0000","last_fullsized":"2023-08-25T09:38:54.170732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080971+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:38:54.080971+0000","last_peered":"2023-08-25T09:38:54.080971+0000","last_clean":"2023-08-25T09:38:54.080971+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:38:54.080971+0000","last_undegraded":"2023-08-25T09:38:54.080971+0000","last_fullsized":"2023-08-25T09:38:54.080971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:01.142595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170917+0000","last_change":"2023-08-25T09:38:52.149368+0000","last_active":"2023-08-25T09:38:54.170917+0000","last_peered":"2023-08-25T09:38:54.170917+0000","last_clean":"2023-08-25T09:38:54.170917+0000","last_became_active":"2023-08-25T09:38:52.149118+0000","last_became_peered":"2023-08-25T09:38:52.149118+0000","last_unstale":"2023-08-25T09:38:54.170917+0000","last_undegraded":"2023-08-25T09:38:54.170917+0000","last_fullsized":"2023-08-25T09:38:54.170917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:16.835823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078872+0000","last_change":"2023-08-25T09:38:53.108258+0000","last_active":"2023-08-25T09:38:54.078872+0000","last_peered":"2023-08-25T09:38:54.078872+0000","last_clean":"2023-08-25T09:38:54.078872+0000","last_became_active":"2023-08-25T09:38:53.107991+0000","last_became_peered":"2023-08-25T09:38:53.107991+0000","last_unstale":"2023-08-25T09:38:54.078872+0000","last_undegraded":"2023-08-25T09:38:54.078872+0000","last_fullsized":"2023-08-25T09:38:54.078872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078433+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:38:54.078433+0000","last_peered":"2023-08-25T09:38:54.078433+0000","last_clean":"2023-08-25T09:38:54.078433+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:38:54.078433+0000","last_undegraded":"2023-08-25T09:38:54.078433+0000","last_fullsized":"2023-08-25T09:38:54.078433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:17.906346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215184+0000","last_change":"2023-08-25T09:38:52.146434+0000","last_active":"2023-08-25T09:38:54.215184+0000","last_peered":"2023-08-25T09:38:54.215184+0000","last_clean":"2023-08-25T09:38:54.215184+0000","last_became_active":"2023-08-25T09:38:52.146126+0000","last_became_peered":"2023-08-25T09:38:52.146126+0000","last_unstale":"2023-08-25T09:38:54.215184+0000","last_undegraded":"2023-08-25T09:38:54.215184+0000","last_fullsized":"2023-08-25T09:38:54.215184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079123+0000","last_change":"2023-08-25T09:38:53.097778+0000","last_active":"2023-08-25T09:38:54.079123+0000","last_peered":"2023-08-25T09:38:54.079123+0000","last_clean":"2023-08-25T09:38:54.079123+0000","last_became_active":"2023-08-25T09:38:53.097182+0000","last_became_peered":"2023-08-25T09:38:53.097182+0000","last_unstale":"2023-08-25T09:38:54.079123+0000","last_undegraded":"2023-08-25T09:38:54.079123+0000","last_fullsized":"2023-08-25T09:38:54.079123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185740+0000","last_change":"2023-08-25T09:38:52.143868+0000","last_active":"2023-08-25T09:38:54.185740+0000","last_peered":"2023-08-25T09:38:54.185740+0000","last_clean":"2023-08-25T09:38:54.185740+0000","last_became_active":"2023-08-25T09:38:52.143584+0000","last_became_peered":"2023-08-25T09:38:52.143584+0000","last_unstale":"2023-08-25T09:38:54.185740+0000","last_undegraded":"2023-08-25T09:38:54.185740+0000","last_fullsized":"2023-08-25T09:38:54.185740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171239+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:38:54.171239+0000","last_peered":"2023-08-25T09:38:54.171239+0000","last_clean":"2023-08-25T09:38:54.171239+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:38:54.171239+0000","last_undegraded":"2023-08-25T09:38:54.171239+0000","last_fullsized":"2023-08-25T09:38:54.171239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:38.295927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079243+0000","last_change":"2023-08-25T09:38:53.098704+0000","last_active":"2023-08-25T09:38:54.079243+0000","last_peered":"2023-08-25T09:38:54.079243+0000","last_clean":"2023-08-25T09:38:54.079243+0000","last_became_active":"2023-08-25T09:38:53.098360+0000","last_became_peered":"2023-08-25T09:38:53.098360+0000","last_unstale":"2023-08-25T09:38:54.079243+0000","last_undegraded":"2023-08-25T09:38:54.079243+0000","last_fullsized":"2023-08-25T09:38:54.079243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079454+0000","last_change":"2023-08-25T09:38:52.146452+0000","last_active":"2023-08-25T09:38:54.079454+0000","last_peered":"2023-08-25T09:38:54.079454+0000","last_clean":"2023-08-25T09:38:54.079454+0000","last_became_active":"2023-08-25T09:38:52.146193+0000","last_became_peered":"2023-08-25T09:38:52.146193+0000","last_unstale":"2023-08-25T09:38:54.079454+0000","last_undegraded":"2023-08-25T09:38:54.079454+0000","last_fullsized":"2023-08-25T09:38:54.079454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076841+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:38:54.076841+0000","last_peered":"2023-08-25T09:38:54.076841+0000","last_clean":"2023-08-25T09:38:54.076841+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:38:54.076841+0000","last_undegraded":"2023-08-25T09:38:54.076841+0000","last_fullsized":"2023-08-25T09:38:54.076841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:16.377836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079556+0000","last_change":"2023-08-25T09:38:53.109546+0000","last_active":"2023-08-25T09:38:54.079556+0000","last_peered":"2023-08-25T09:38:54.079556+0000","last_clean":"2023-08-25T09:38:54.079556+0000","last_became_active":"2023-08-25T09:38:53.109238+0000","last_became_peered":"2023-08-25T09:38:53.109238+0000","last_unstale":"2023-08-25T09:38:54.079556+0000","last_undegraded":"2023-08-25T09:38:54.079556+0000","last_fullsized":"2023-08-25T09:38:54.079556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170022+0000","last_change":"2023-08-25T09:38:52.144544+0000","last_active":"2023-08-25T09:38:54.170022+0000","last_peered":"2023-08-25T09:38:54.170022+0000","last_clean":"2023-08-25T09:38:54.170022+0000","last_became_active":"2023-08-25T09:38:52.144095+0000","last_became_peered":"2023-08-25T09:38:52.144095+0000","last_unstale":"2023-08-25T09:38:54.170022+0000","last_undegraded":"2023-08-25T09:38:54.170022+0000","last_fullsized":"2023-08-25T09:38:54.170022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:08.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169271+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:38:54.169271+0000","last_peered":"2023-08-25T09:38:54.169271+0000","last_clean":"2023-08-25T09:38:54.169271+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:38:54.169271+0000","last_undegraded":"2023-08-25T09:38:54.169271+0000","last_fullsized":"2023-08-25T09:38:54.169271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:11.717086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169145+0000","last_change":"2023-08-25T09:38:53.112168+0000","last_active":"2023-08-25T09:38:54.169145+0000","last_peered":"2023-08-25T09:38:54.169145+0000","last_clean":"2023-08-25T09:38:54.169145+0000","last_became_active":"2023-08-25T09:38:53.107808+0000","last_became_peered":"2023-08-25T09:38:53.107808+0000","last_unstale":"2023-08-25T09:38:54.169145+0000","last_undegraded":"2023-08-25T09:38:54.169145+0000","last_fullsized":"2023-08-25T09:38:54.169145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.337779+0000","last_change":"2023-08-25T09:38:52.143505+0000","last_active":"2023-08-25T09:38:54.337779+0000","last_peered":"2023-08-25T09:38:54.337779+0000","last_clean":"2023-08-25T09:38:54.337779+0000","last_became_active":"2023-08-25T09:38:52.143175+0000","last_became_peered":"2023-08-25T09:38:52.143175+0000","last_unstale":"2023-08-25T09:38:54.337779+0000","last_undegraded":"2023-08-25T09:38:54.337779+0000","last_fullsized":"2023-08-25T09:38:54.337779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:51.819062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078207+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:38:54.078207+0000","last_peered":"2023-08-25T09:38:54.078207+0000","last_clean":"2023-08-25T09:38:54.078207+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:38:54.078207+0000","last_undegraded":"2023-08-25T09:38:54.078207+0000","last_fullsized":"2023-08-25T09:38:54.078207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:28:24.021135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185610+0000","last_change":"2023-08-25T09:38:53.095808+0000","last_active":"2023-08-25T09:38:54.185610+0000","last_peered":"2023-08-25T09:38:54.185610+0000","last_clean":"2023-08-25T09:38:54.185610+0000","last_became_active":"2023-08-25T09:38:53.095427+0000","last_became_peered":"2023-08-25T09:38:53.095427+0000","last_unstale":"2023-08-25T09:38:54.185610+0000","last_undegraded":"2023-08-25T09:38:54.185610+0000","last_fullsized":"2023-08-25T09:38:54.185610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186067+0000","last_change":"2023-08-25T09:38:53.094102+0000","last_active":"2023-08-25T09:38:54.186067+0000","last_peered":"2023-08-25T09:38:54.186067+0000","last_clean":"2023-08-25T09:38:54.186067+0000","last_became_active":"2023-08-25T09:38:53.093658+0000","last_became_peered":"2023-08-25T09:38:53.093658+0000","last_unstale":"2023-08-25T09:38:54.186067+0000","last_undegraded":"2023-08-25T09:38:54.186067+0000","last_fullsized":"2023-08-25T09:38:54.186067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076645+0000","last_change":"2023-08-25T09:38:52.145307+0000","last_active":"2023-08-25T09:38:54.076645+0000","last_peered":"2023-08-25T09:38:54.076645+0000","last_clean":"2023-08-25T09:38:54.076645+0000","last_became_active":"2023-08-25T09:38:52.145076+0000","last_became_peered":"2023-08-25T09:38:52.145076+0000","last_unstale":"2023-08-25T09:38:54.076645+0000","last_undegraded":"2023-08-25T09:38:54.076645+0000","last_fullsized":"2023-08-25T09:38:54.076645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:16.527497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168761+0000","last_change":"2023-08-25T09:38:53.105922+0000","last_active":"2023-08-25T09:38:54.168761+0000","last_peered":"2023-08-25T09:38:54.168761+0000","last_clean":"2023-08-25T09:38:54.168761+0000","last_became_active":"2023-08-25T09:38:53.105488+0000","last_became_peered":"2023-08-25T09:38:53.105488+0000","last_unstale":"2023-08-25T09:38:54.168761+0000","last_undegraded":"2023-08-25T09:38:54.168761+0000","last_fullsized":"2023-08-25T09:38:54.168761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215307+0000","last_change":"2023-08-25T09:38:52.142639+0000","last_active":"2023-08-25T09:38:54.215307+0000","last_peered":"2023-08-25T09:38:54.215307+0000","last_clean":"2023-08-25T09:38:54.215307+0000","last_became_active":"2023-08-25T09:38:52.142406+0000","last_became_peered":"2023-08-25T09:38:52.142406+0000","last_unstale":"2023-08-25T09:38:54.215307+0000","last_undegraded":"2023-08-25T09:38:54.215307+0000","last_fullsized":"2023-08-25T09:38:54.215307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:49.561545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186057+0000","last_change":"2023-08-25T09:38:52.145084+0000","last_active":"2023-08-25T09:38:54.186057+0000","last_peered":"2023-08-25T09:38:54.186057+0000","last_clean":"2023-08-25T09:38:54.186057+0000","last_became_active":"2023-08-25T09:38:52.144835+0000","last_became_peered":"2023-08-25T09:38:52.144835+0000","last_unstale":"2023-08-25T09:38:54.186057+0000","last_undegraded":"2023-08-25T09:38:54.186057+0000","last_fullsized":"2023-08-25T09:38:54.186057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169334+0000","last_change":"2023-08-25T09:38:53.107353+0000","last_active":"2023-08-25T09:38:54.169334+0000","last_peered":"2023-08-25T09:38:54.169334+0000","last_clean":"2023-08-25T09:38:54.169334+0000","last_became_active":"2023-08-25T09:38:53.106537+0000","last_became_peered":"2023-08-25T09:38:53.106537+0000","last_unstale":"2023-08-25T09:38:54.169334+0000","last_undegraded":"2023-08-25T09:38:54.169334+0000","last_fullsized":"2023-08-25T09:38:54.169334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170432+0000","last_change":"2023-08-25T09:38:52.142676+0000","last_active":"2023-08-25T09:38:54.170432+0000","last_peered":"2023-08-25T09:38:54.170432+0000","last_clean":"2023-08-25T09:38:54.170432+0000","last_became_active":"2023-08-25T09:38:52.142266+0000","last_became_peered":"2023-08-25T09:38:52.142266+0000","last_unstale":"2023-08-25T09:38:54.170432+0000","last_undegraded":"2023-08-25T09:38:54.170432+0000","last_fullsized":"2023-08-25T09:38:54.170432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:05.330534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080254+0000","last_change":"2023-08-25T09:38:53.092162+0000","last_active":"2023-08-25T09:38:54.080254+0000","last_peered":"2023-08-25T09:38:54.080254+0000","last_clean":"2023-08-25T09:38:54.080254+0000","last_became_active":"2023-08-25T09:38:53.091630+0000","last_became_peered":"2023-08-25T09:38:53.091630+0000","last_unstale":"2023-08-25T09:38:54.080254+0000","last_undegraded":"2023-08-25T09:38:54.080254+0000","last_fullsized":"2023-08-25T09:38:54.080254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076680+0000","last_change":"2023-08-25T09:38:53.104609+0000","last_active":"2023-08-25T09:38:54.076680+0000","last_peered":"2023-08-25T09:38:54.076680+0000","last_clean":"2023-08-25T09:38:54.076680+0000","last_became_active":"2023-08-25T09:38:53.104257+0000","last_became_peered":"2023-08-25T09:38:53.104257+0000","last_unstale":"2023-08-25T09:38:54.076680+0000","last_undegraded":"2023-08-25T09:38:54.076680+0000","last_fullsized":"2023-08-25T09:38:54.076680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:15.255372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171027+0000","last_change":"2023-08-25T09:38:52.149252+0000","last_active":"2023-08-25T09:38:54.171027+0000","last_peered":"2023-08-25T09:38:54.171027+0000","last_clean":"2023-08-25T09:38:54.171027+0000","last_became_active":"2023-08-25T09:38:52.148775+0000","last_became_peered":"2023-08-25T09:38:52.148775+0000","last_unstale":"2023-08-25T09:38:54.171027+0000","last_undegraded":"2023-08-25T09:38:54.171027+0000","last_fullsized":"2023-08-25T09:38:54.171027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169503+0000","last_change":"2023-08-25T09:38:53.112427+0000","last_active":"2023-08-25T09:38:54.169503+0000","last_peered":"2023-08-25T09:38:54.169503+0000","last_clean":"2023-08-25T09:38:54.169503+0000","last_became_active":"2023-08-25T09:38:53.110947+0000","last_became_peered":"2023-08-25T09:38:53.110947+0000","last_unstale":"2023-08-25T09:38:54.169503+0000","last_undegraded":"2023-08-25T09:38:54.169503+0000","last_fullsized":"2023-08-25T09:38:54.169503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:35.927820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169802+0000","last_change":"2023-08-25T09:38:52.140188+0000","last_active":"2023-08-25T09:38:54.169802+0000","last_peered":"2023-08-25T09:38:54.169802+0000","last_clean":"2023-08-25T09:38:54.169802+0000","last_became_active":"2023-08-25T09:38:52.139523+0000","last_became_peered":"2023-08-25T09:38:52.139523+0000","last_unstale":"2023-08-25T09:38:54.169802+0000","last_undegraded":"2023-08-25T09:38:54.169802+0000","last_fullsized":"2023-08-25T09:38:54.169802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:04.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078279+0000","last_change":"2023-08-25T09:38:53.089895+0000","last_active":"2023-08-25T09:38:54.078279+0000","last_peered":"2023-08-25T09:38:54.078279+0000","last_clean":"2023-08-25T09:38:54.078279+0000","last_became_active":"2023-08-25T09:38:53.089211+0000","last_became_peered":"2023-08-25T09:38:53.089211+0000","last_unstale":"2023-08-25T09:38:54.078279+0000","last_undegraded":"2023-08-25T09:38:54.078279+0000","last_fullsized":"2023-08-25T09:38:54.078279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167322+0000","last_change":"2023-08-25T09:38:52.149088+0000","last_active":"2023-08-25T09:38:54.167322+0000","last_peered":"2023-08-25T09:38:54.167322+0000","last_clean":"2023-08-25T09:38:54.167322+0000","last_became_active":"2023-08-25T09:38:52.148577+0000","last_became_peered":"2023-08-25T09:38:52.148577+0000","last_unstale":"2023-08-25T09:38:54.167322+0000","last_undegraded":"2023-08-25T09:38:54.167322+0000","last_fullsized":"2023-08-25T09:38:54.167322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:48.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169133+0000","last_change":"2023-08-25T09:38:53.105736+0000","last_active":"2023-08-25T09:38:54.169133+0000","last_peered":"2023-08-25T09:38:54.169133+0000","last_clean":"2023-08-25T09:38:54.169133+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.169133+0000","last_undegraded":"2023-08-25T09:38:54.169133+0000","last_fullsized":"2023-08-25T09:38:54.169133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:14.964126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169659+0000","last_change":"2023-08-25T09:38:52.144372+0000","last_active":"2023-08-25T09:38:54.169659+0000","last_peered":"2023-08-25T09:38:54.169659+0000","last_clean":"2023-08-25T09:38:54.169659+0000","last_became_active":"2023-08-25T09:38:52.143646+0000","last_became_peered":"2023-08-25T09:38:52.143646+0000","last_unstale":"2023-08-25T09:38:54.169659+0000","last_undegraded":"2023-08-25T09:38:54.169659+0000","last_fullsized":"2023-08-25T09:38:54.169659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:43.414664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185820+0000","last_change":"2023-08-25T09:38:53.091214+0000","last_active":"2023-08-25T09:38:54.185820+0000","last_peered":"2023-08-25T09:38:54.185820+0000","last_clean":"2023-08-25T09:38:54.185820+0000","last_became_active":"2023-08-25T09:38:53.090817+0000","last_became_peered":"2023-08-25T09:38:53.090817+0000","last_unstale":"2023-08-25T09:38:54.185820+0000","last_undegraded":"2023-08-25T09:38:54.185820+0000","last_fullsized":"2023-08-25T09:38:54.185820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167705+0000","last_change":"2023-08-25T09:38:52.139178+0000","last_active":"2023-08-25T09:38:54.167705+0000","last_peered":"2023-08-25T09:38:54.167705+0000","last_clean":"2023-08-25T09:38:54.167705+0000","last_became_active":"2023-08-25T09:38:52.138923+0000","last_became_peered":"2023-08-25T09:38:52.138923+0000","last_unstale":"2023-08-25T09:38:54.167705+0000","last_undegraded":"2023-08-25T09:38:54.167705+0000","last_fullsized":"2023-08-25T09:38:54.167705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.098793+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.097585+0000","last_became_peered":"2023-08-25T09:38:53.097585+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077130+0000","last_change":"2023-08-25T09:38:52.143009+0000","last_active":"2023-08-25T09:38:54.077130+0000","last_peered":"2023-08-25T09:38:54.077130+0000","last_clean":"2023-08-25T09:38:54.077130+0000","last_became_active":"2023-08-25T09:38:52.140881+0000","last_became_peered":"2023-08-25T09:38:52.140881+0000","last_unstale":"2023-08-25T09:38:54.077130+0000","last_undegraded":"2023-08-25T09:38:54.077130+0000","last_fullsized":"2023-08-25T09:38:54.077130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:19.948794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078140+0000","last_change":"2023-08-25T09:38:53.100041+0000","last_active":"2023-08-25T09:38:54.078140+0000","last_peered":"2023-08-25T09:38:54.078140+0000","last_clean":"2023-08-25T09:38:54.078140+0000","last_became_active":"2023-08-25T09:38:53.099764+0000","last_became_peered":"2023-08-25T09:38:53.099764+0000","last_unstale":"2023-08-25T09:38:54.078140+0000","last_undegraded":"2023-08-25T09:38:54.078140+0000","last_fullsized":"2023-08-25T09:38:54.078140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:13:51.695417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170584+0000","last_change":"2023-08-25T09:38:52.150397+0000","last_active":"2023-08-25T09:38:54.170584+0000","last_peered":"2023-08-25T09:38:54.170584+0000","last_clean":"2023-08-25T09:38:54.170584+0000","last_became_active":"2023-08-25T09:38:52.150134+0000","last_became_peered":"2023-08-25T09:38:52.150134+0000","last_unstale":"2023-08-25T09:38:54.170584+0000","last_undegraded":"2023-08-25T09:38:54.170584+0000","last_fullsized":"2023-08-25T09:38:54.170584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185984+0000","last_change":"2023-08-25T09:38:53.099998+0000","last_active":"2023-08-25T09:38:54.185984+0000","last_peered":"2023-08-25T09:38:54.185984+0000","last_clean":"2023-08-25T09:38:54.185984+0000","last_became_active":"2023-08-25T09:38:53.099612+0000","last_became_peered":"2023-08-25T09:38:53.099612+0000","last_unstale":"2023-08-25T09:38:54.185984+0000","last_undegraded":"2023-08-25T09:38:54.185984+0000","last_fullsized":"2023-08-25T09:38:54.185984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170948+0000","last_change":"2023-08-25T09:38:52.147242+0000","last_active":"2023-08-25T09:38:54.170948+0000","last_peered":"2023-08-25T09:38:54.170948+0000","last_clean":"2023-08-25T09:38:54.170948+0000","last_became_active":"2023-08-25T09:38:52.146238+0000","last_became_peered":"2023-08-25T09:38:52.146238+0000","last_unstale":"2023-08-25T09:38:54.170948+0000","last_undegraded":"2023-08-25T09:38:54.170948+0000","last_fullsized":"2023-08-25T09:38:54.170948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910561,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":664,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194648,"statfs":{"total":200000000,"available":199319611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8540000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910561,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":89,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195223,"statfs":{"total":200000000,"available":199908845,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":88,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195223,"statfs":{"total":200000000,"available":199908949,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.383}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":664,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194647,"statfs":{"total":200000000,"available":199319519,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910415,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]}],"pool_statfs":[]}} 2023-08-25T09:39:03.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:39:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.410+0000 7faccf8fa640 1 Processor -- start 2023-08-25T09:39:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.410+0000 7faccf8fa640 1 -- start start 2023-08-25T09:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccf8fa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7facc815e830 0x7facc806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccf8fa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc815ec10 0x7facc80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccf8fa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc80b1b60 con 0x7facc806b2b0 2023-08-25T09:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccf8fa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc806bd80 con 0x7facc815ec10 2023-08-25T09:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccf8fa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc806b7c0 con 0x7facc815e830 2023-08-25T09:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc815ec10 0x7facc80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc815ec10 0x7facc80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47444/0 (socket says 172.21.15.70:47444) 2023-08-25T09:39:03.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 -- 172.21.15.70:0/3599208397 learned_addr learned my addr 172.21.15.70:0/3599208397 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccd696640 1 --2- 172.21.15.70:0/3599208397 >> v2:172.21.15.155:3300/0 conn(0x7facc815e830 0x7facc806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 -- 172.21.15.70:0/3599208397 >> v2:172.21.15.155:3300/0 conn(0x7facc815e830 msgr2=0x7facc806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 --2- 172.21.15.70:0/3599208397 >> v2:172.21.15.155:3300/0 conn(0x7facc815e830 0x7facc806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 -- 172.21.15.70:0/3599208397 >> v1:172.21.15.155:6789/0 conn(0x7facc806b2b0 legacy=0x7facc815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccde97640 1 -- 172.21.15.70:0/3599208397 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc806baa0 con 0x7facc815ec10 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.412+0000 7faccd696640 1 --2- 172.21.15.70:0/3599208397 >> v2:172.21.15.155:3300/0 conn(0x7facc815e830 0x7facc806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.413+0000 7faccde97640 1 --2- 172.21.15.70:0/3599208397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc815ec10 0x7facc80b1ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7facc806bf10 tx=0x7facc002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2779d980c6e84c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7facbffff640 1 -- 172.21.15.70:0/3599208397 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc002ca80 con 0x7facc815ec10 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7facbffff640 1 -- 172.21.15.70:0/3599208397 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc002cbe0 con 0x7facc815ec10 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7facbffff640 1 -- 172.21.15.70:0/3599208397 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00343f0 con 0x7facc815ec10 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7faccf8fa640 1 -- 172.21.15.70:0/3599208397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc815ec10 msgr2=0x7facc80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7faccf8fa640 1 --2- 172.21.15.70:0/3599208397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc815ec10 0x7facc80b1ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7facc806bf10 tx=0x7facc002bee0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7faccf8fa640 1 -- 172.21.15.70:0/3599208397 shutdown_connections 2023-08-25T09:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7faccf8fa640 1 --2- 172.21.15.70:0/3599208397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc815ec10 0x7facc80b1ff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7faccf8fa640 1 --2- 172.21.15.70:0/3599208397 >> v2:172.21.15.155:3300/0 conn(0x7facc815e830 0x7facc806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.414+0000 7faccf8fa640 1 -- 172.21.15.70:0/3599208397 >> 172.21.15.70:0/3599208397 conn(0x7facc8069850 msgr2=0x7facc8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.415+0000 7faccf8fa640 1 -- 172.21.15.70:0/3599208397 shutdown_connections 2023-08-25T09:39:03.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.415+0000 7faccf8fa640 1 -- 172.21.15.70:0/3599208397 wait complete. 2023-08-25T09:39:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.416+0000 7faccf8fa640 1 Processor -- start 2023-08-25T09:39:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.416+0000 7faccf8fa640 1 -- start start 2023-08-25T09:39:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccf8fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 0x7facc8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccf8fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc815e830 0x7facc80c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccf8fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc815ec10 0x7facc8159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccf8fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c9d90 con 0x7facc806b2b0 2023-08-25T09:39:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccf8fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c79d0 con 0x7facc815e830 2023-08-25T09:39:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccf8fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc80c8100 con 0x7facc815ec10 2023-08-25T09:39:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccd696640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 0x7facc8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccd696640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 0x7facc8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47460/0 (socket says 172.21.15.70:47460) 2023-08-25T09:39:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.417+0000 7faccd696640 1 -- 172.21.15.70:0/1126660887 learned_addr learned my addr 172.21.15.70:0/1126660887 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccd696640 1 -- 172.21.15.70:0/1126660887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc815ec10 msgr2=0x7facc8159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccd696640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc815ec10 0x7facc8159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccd696640 1 -- 172.21.15.70:0/1126660887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc815e830 msgr2=0x7facc80c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccd696640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc815e830 0x7facc80c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccd696640 1 -- 172.21.15.70:0/1126660887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc0009b30 con 0x7facc806b2b0 2023-08-25T09:39:03.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccd696640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 0x7facc8153450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7facc400c7f0 tx=0x7facc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:03.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4007a30 con 0x7facc806b2b0 2023-08-25T09:39:03.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc4007b90 con 0x7facc806b2b0 2023-08-25T09:39:03.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc400ce40 con 0x7facc806b2b0 2023-08-25T09:39:03.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc815ca20 con 0x7facc806b2b0 2023-08-25T09:39:03.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.418+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc8154160 con 0x7facc806b2b0 2023-08-25T09:39:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.419+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc804e420 con 0x7facc806b2b0 2023-08-25T09:39:03.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.420+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7facc40113f0 con 0x7facc806b2b0 2023-08-25T09:39:03.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.421+0000 7facbdffb640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7facac06a050 0x7facac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.421+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7facc4088710 con 0x7facc806b2b0 2023-08-25T09:39:03.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.421+0000 7faccce95640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7facac06a050 0x7facac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.422+0000 7faccce95640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7facac06a050 0x7facac06c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facc81561d0 tx=0x7facb8007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:03.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.425+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7facc4053df0 con 0x7facc806b2b0 2023-08-25T09:39:03.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.594+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facc815c8a0 con 0x7facac06a050 2023-08-25T09:39:03.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.615+0000 7facbdffb640 1 -- 172.21.15.70:0/1126660887 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280650 (secure 0 0 0) 0x7facb80050d0 con 0x7facac06a050 2023-08-25T09:39:03.618 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:03.621 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:39:03.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7facac06a050 msgr2=0x7facac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7facac06a050 0x7facac06c510 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7facc81561d0 tx=0x7facb8007990 comp rx=0 tx=0).stop 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 msgr2=0x7facc8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 0x7facc8153450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7facc400c7f0 tx=0x7facc400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 shutdown_connections 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7facac06a050 0x7facac06c510 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc815ec10 0x7facc8159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc815e830 0x7facc80c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 --2- 172.21.15.70:0/1126660887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc806b2b0 0x7facc8153450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 >> 172.21.15.70:0/1126660887 conn(0x7facc8069850 msgr2=0x7facc80b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:03.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 shutdown_connections 2023-08-25T09:39:03.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.621+0000 7faccf8fa640 1 -- 172.21.15.70:0/1126660887 wait complete. 2023-08-25T09:39:03.639 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-25T09:39:03.320649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1856,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560643,"statfs":{"total":1600000000,"available":1598098932,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003552"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169943+0000","last_change":"2023-08-25T09:38:53.105814+0000","last_active":"2023-08-25T09:38:54.169943+0000","last_peered":"2023-08-25T09:38:54.169943+0000","last_clean":"2023-08-25T09:38:54.169943+0000","last_became_active":"2023-08-25T09:38:53.103612+0000","last_became_peered":"2023-08-25T09:38:53.103612+0000","last_unstale":"2023-08-25T09:38:54.169943+0000","last_undegraded":"2023-08-25T09:38:54.169943+0000","last_fullsized":"2023-08-25T09:38:54.169943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185531+0000","last_change":"2023-08-25T09:38:53.103005+0000","last_active":"2023-08-25T09:38:54.185531+0000","last_peered":"2023-08-25T09:38:54.185531+0000","last_clean":"2023-08-25T09:38:54.185531+0000","last_became_active":"2023-08-25T09:38:53.102634+0000","last_became_peered":"2023-08-25T09:38:53.102634+0000","last_unstale":"2023-08-25T09:38:54.185531+0000","last_undegraded":"2023-08-25T09:38:54.185531+0000","last_fullsized":"2023-08-25T09:38:54.185531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214701+0000","last_change":"2023-08-25T09:38:53.111188+0000","last_active":"2023-08-25T09:38:54.214701+0000","last_peered":"2023-08-25T09:38:54.214701+0000","last_clean":"2023-08-25T09:38:54.214701+0000","last_became_active":"2023-08-25T09:38:53.108190+0000","last_became_peered":"2023-08-25T09:38:53.108190+0000","last_unstale":"2023-08-25T09:38:54.214701+0000","last_undegraded":"2023-08-25T09:38:54.214701+0000","last_fullsized":"2023-08-25T09:38:54.214701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:16.468452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214312+0000","last_change":"2023-08-25T09:38:53.103968+0000","last_active":"2023-08-25T09:38:54.214312+0000","last_peered":"2023-08-25T09:38:54.214312+0000","last_clean":"2023-08-25T09:38:54.214312+0000","last_became_active":"2023-08-25T09:38:53.103676+0000","last_became_peered":"2023-08-25T09:38:53.103676+0000","last_unstale":"2023-08-25T09:38:54.214312+0000","last_undegraded":"2023-08-25T09:38:54.214312+0000","last_fullsized":"2023-08-25T09:38:54.214312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170566+0000","last_change":"2023-08-25T09:38:53.110486+0000","last_active":"2023-08-25T09:38:54.170566+0000","last_peered":"2023-08-25T09:38:54.170566+0000","last_clean":"2023-08-25T09:38:54.170566+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.170566+0000","last_undegraded":"2023-08-25T09:38:54.170566+0000","last_fullsized":"2023-08-25T09:38:54.170566+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:15.681428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077139+0000","last_change":"2023-08-25T09:38:53.082883+0000","last_active":"2023-08-25T09:38:54.077139+0000","last_peered":"2023-08-25T09:38:54.077139+0000","last_clean":"2023-08-25T09:38:54.077139+0000","last_became_active":"2023-08-25T09:38:53.081892+0000","last_became_peered":"2023-08-25T09:38:53.081892+0000","last_unstale":"2023-08-25T09:38:54.077139+0000","last_undegraded":"2023-08-25T09:38:54.077139+0000","last_fullsized":"2023-08-25T09:38:54.077139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:25:18.218904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077432+0000","last_change":"2023-08-25T09:38:53.094986+0000","last_active":"2023-08-25T09:38:54.077432+0000","last_peered":"2023-08-25T09:38:54.077432+0000","last_clean":"2023-08-25T09:38:54.077432+0000","last_became_active":"2023-08-25T09:38:53.094533+0000","last_became_peered":"2023-08-25T09:38:53.094533+0000","last_unstale":"2023-08-25T09:38:54.077432+0000","last_undegraded":"2023-08-25T09:38:54.077432+0000","last_fullsized":"2023-08-25T09:38:54.077432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:44.551022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167144+0000","last_change":"2023-08-25T09:38:53.097497+0000","last_active":"2023-08-25T09:38:54.167144+0000","last_peered":"2023-08-25T09:38:54.167144+0000","last_clean":"2023-08-25T09:38:54.167144+0000","last_became_active":"2023-08-25T09:38:53.096835+0000","last_became_peered":"2023-08-25T09:38:53.096835+0000","last_unstale":"2023-08-25T09:38:54.167144+0000","last_undegraded":"2023-08-25T09:38:54.167144+0000","last_fullsized":"2023-08-25T09:38:54.167144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168759+0000","last_change":"2023-08-25T09:38:53.111823+0000","last_active":"2023-08-25T09:38:54.168759+0000","last_peered":"2023-08-25T09:38:54.168759+0000","last_clean":"2023-08-25T09:38:54.168759+0000","last_became_active":"2023-08-25T09:38:53.107075+0000","last_became_peered":"2023-08-25T09:38:53.107075+0000","last_unstale":"2023-08-25T09:38:54.168759+0000","last_undegraded":"2023-08-25T09:38:54.168759+0000","last_fullsized":"2023-08-25T09:38:54.168759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167237+0000","last_change":"2023-08-25T09:38:53.097405+0000","last_active":"2023-08-25T09:38:54.167237+0000","last_peered":"2023-08-25T09:38:54.167237+0000","last_clean":"2023-08-25T09:38:54.167237+0000","last_became_active":"2023-08-25T09:38:53.097063+0000","last_became_peered":"2023-08-25T09:38:53.097063+0000","last_unstale":"2023-08-25T09:38:54.167237+0000","last_undegraded":"2023-08-25T09:38:54.167237+0000","last_fullsized":"2023-08-25T09:38:54.167237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167432+0000","last_change":"2023-08-25T09:38:53.097713+0000","last_active":"2023-08-25T09:38:54.167432+0000","last_peered":"2023-08-25T09:38:54.167432+0000","last_clean":"2023-08-25T09:38:54.167432+0000","last_became_active":"2023-08-25T09:38:53.097250+0000","last_became_peered":"2023-08-25T09:38:53.097250+0000","last_unstale":"2023-08-25T09:38:54.167432+0000","last_undegraded":"2023-08-25T09:38:54.167432+0000","last_fullsized":"2023-08-25T09:38:54.167432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170076+0000","last_change":"2023-08-25T09:38:53.098525+0000","last_active":"2023-08-25T09:38:54.170076+0000","last_peered":"2023-08-25T09:38:54.170076+0000","last_clean":"2023-08-25T09:38:54.170076+0000","last_became_active":"2023-08-25T09:38:53.097701+0000","last_became_peered":"2023-08-25T09:38:53.097701+0000","last_unstale":"2023-08-25T09:38:54.170076+0000","last_undegraded":"2023-08-25T09:38:54.170076+0000","last_fullsized":"2023-08-25T09:38:54.170076+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079156+0000","last_change":"2023-08-25T09:38:53.110156+0000","last_active":"2023-08-25T09:38:54.079156+0000","last_peered":"2023-08-25T09:38:54.079156+0000","last_clean":"2023-08-25T09:38:54.079156+0000","last_became_active":"2023-08-25T09:38:53.109921+0000","last_became_peered":"2023-08-25T09:38:53.109921+0000","last_unstale":"2023-08-25T09:38:54.079156+0000","last_undegraded":"2023-08-25T09:38:54.079156+0000","last_fullsized":"2023-08-25T09:38:54.079156+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167535+0000","last_change":"2023-08-25T09:38:53.102316+0000","last_active":"2023-08-25T09:38:54.167535+0000","last_peered":"2023-08-25T09:38:54.167535+0000","last_clean":"2023-08-25T09:38:54.167535+0000","last_became_active":"2023-08-25T09:38:53.100943+0000","last_became_peered":"2023-08-25T09:38:53.100943+0000","last_unstale":"2023-08-25T09:38:54.167535+0000","last_undegraded":"2023-08-25T09:38:54.167535+0000","last_fullsized":"2023-08-25T09:38:54.167535+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214539+0000","last_change":"2023-08-25T09:38:53.111165+0000","last_active":"2023-08-25T09:38:54.214539+0000","last_peered":"2023-08-25T09:38:54.214539+0000","last_clean":"2023-08-25T09:38:54.214539+0000","last_became_active":"2023-08-25T09:38:53.108285+0000","last_became_peered":"2023-08-25T09:38:53.108285+0000","last_unstale":"2023-08-25T09:38:54.214539+0000","last_undegraded":"2023-08-25T09:38:54.214539+0000","last_fullsized":"2023-08-25T09:38:54.214539+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077362+0000","last_change":"2023-08-25T09:38:53.089749+0000","last_active":"2023-08-25T09:38:54.077362+0000","last_peered":"2023-08-25T09:38:54.077362+0000","last_clean":"2023-08-25T09:38:54.077362+0000","last_became_active":"2023-08-25T09:38:53.088753+0000","last_became_peered":"2023-08-25T09:38:53.088753+0000","last_unstale":"2023-08-25T09:38:54.077362+0000","last_undegraded":"2023-08-25T09:38:54.077362+0000","last_fullsized":"2023-08-25T09:38:54.077362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:35.174323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078331+0000","last_change":"2023-08-25T09:38:53.100818+0000","last_active":"2023-08-25T09:38:54.078331+0000","last_peered":"2023-08-25T09:38:54.078331+0000","last_clean":"2023-08-25T09:38:54.078331+0000","last_became_active":"2023-08-25T09:38:53.100555+0000","last_became_peered":"2023-08-25T09:38:53.100555+0000","last_unstale":"2023-08-25T09:38:54.078331+0000","last_undegraded":"2023-08-25T09:38:54.078331+0000","last_fullsized":"2023-08-25T09:38:54.078331+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215037+0000","last_change":"2023-08-25T09:38:53.105302+0000","last_active":"2023-08-25T09:38:54.215037+0000","last_peered":"2023-08-25T09:38:54.215037+0000","last_clean":"2023-08-25T09:38:54.215037+0000","last_became_active":"2023-08-25T09:38:53.103402+0000","last_became_peered":"2023-08-25T09:38:53.103402+0000","last_unstale":"2023-08-25T09:38:54.215037+0000","last_undegraded":"2023-08-25T09:38:54.215037+0000","last_fullsized":"2023-08-25T09:38:54.215037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171407+0000","last_change":"2023-08-25T09:38:53.099130+0000","last_active":"2023-08-25T09:38:54.171407+0000","last_peered":"2023-08-25T09:38:54.171407+0000","last_clean":"2023-08-25T09:38:54.171407+0000","last_became_active":"2023-08-25T09:38:53.094676+0000","last_became_peered":"2023-08-25T09:38:53.094676+0000","last_unstale":"2023-08-25T09:38:54.171407+0000","last_undegraded":"2023-08-25T09:38:54.171407+0000","last_fullsized":"2023-08-25T09:38:54.171407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:05.648183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167405+0000","last_change":"2023-08-25T09:38:53.111895+0000","last_active":"2023-08-25T09:38:54.167405+0000","last_peered":"2023-08-25T09:38:54.167405+0000","last_clean":"2023-08-25T09:38:54.167405+0000","last_became_active":"2023-08-25T09:38:53.111626+0000","last_became_peered":"2023-08-25T09:38:53.111626+0000","last_unstale":"2023-08-25T09:38:54.167405+0000","last_undegraded":"2023-08-25T09:38:54.167405+0000","last_fullsized":"2023-08-25T09:38:54.167405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167554+0000","last_change":"2023-08-25T09:38:53.104999+0000","last_active":"2023-08-25T09:38:54.167554+0000","last_peered":"2023-08-25T09:38:54.167554+0000","last_clean":"2023-08-25T09:38:54.167554+0000","last_became_active":"2023-08-25T09:38:53.104760+0000","last_became_peered":"2023-08-25T09:38:53.104760+0000","last_unstale":"2023-08-25T09:38:54.167554+0000","last_undegraded":"2023-08-25T09:38:54.167554+0000","last_fullsized":"2023-08-25T09:38:54.167554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:17.840144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078687+0000","last_change":"2023-08-25T09:38:53.106873+0000","last_active":"2023-08-25T09:38:54.078687+0000","last_peered":"2023-08-25T09:38:54.078687+0000","last_clean":"2023-08-25T09:38:54.078687+0000","last_became_active":"2023-08-25T09:38:53.106581+0000","last_became_peered":"2023-08-25T09:38:53.106581+0000","last_unstale":"2023-08-25T09:38:54.078687+0000","last_undegraded":"2023-08-25T09:38:54.078687+0000","last_fullsized":"2023-08-25T09:38:54.078687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:22:28.576325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168645+0000","last_change":"2023-08-25T09:38:53.112057+0000","last_active":"2023-08-25T09:38:54.168645+0000","last_peered":"2023-08-25T09:38:54.168645+0000","last_clean":"2023-08-25T09:38:54.168645+0000","last_became_active":"2023-08-25T09:38:53.111710+0000","last_became_peered":"2023-08-25T09:38:53.111710+0000","last_unstale":"2023-08-25T09:38:54.168645+0000","last_undegraded":"2023-08-25T09:38:54.168645+0000","last_fullsized":"2023-08-25T09:38:54.168645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:48.369874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167656+0000","last_change":"2023-08-25T09:38:53.101206+0000","last_active":"2023-08-25T09:38:54.167656+0000","last_peered":"2023-08-25T09:38:54.167656+0000","last_clean":"2023-08-25T09:38:54.167656+0000","last_became_active":"2023-08-25T09:38:53.100575+0000","last_became_peered":"2023-08-25T09:38:53.100575+0000","last_unstale":"2023-08-25T09:38:54.167656+0000","last_undegraded":"2023-08-25T09:38:54.167656+0000","last_fullsized":"2023-08-25T09:38:54.167656+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:44.735256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170902+0000","last_change":"2023-08-25T09:38:53.110658+0000","last_active":"2023-08-25T09:38:54.170902+0000","last_peered":"2023-08-25T09:38:54.170902+0000","last_clean":"2023-08-25T09:38:54.170902+0000","last_became_active":"2023-08-25T09:38:53.107553+0000","last_became_peered":"2023-08-25T09:38:53.107553+0000","last_unstale":"2023-08-25T09:38:54.170902+0000","last_undegraded":"2023-08-25T09:38:54.170902+0000","last_fullsized":"2023-08-25T09:38:54.170902+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:35.169281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078368+0000","last_change":"2023-08-25T09:38:53.106509+0000","last_active":"2023-08-25T09:38:54.078368+0000","last_peered":"2023-08-25T09:38:54.078368+0000","last_clean":"2023-08-25T09:38:54.078368+0000","last_became_active":"2023-08-25T09:38:53.106227+0000","last_became_peered":"2023-08-25T09:38:53.106227+0000","last_unstale":"2023-08-25T09:38:54.078368+0000","last_undegraded":"2023-08-25T09:38:54.078368+0000","last_fullsized":"2023-08-25T09:38:54.078368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077761+0000","last_change":"2023-08-25T09:38:53.094823+0000","last_active":"2023-08-25T09:38:54.077761+0000","last_peered":"2023-08-25T09:38:54.077761+0000","last_clean":"2023-08-25T09:38:54.077761+0000","last_became_active":"2023-08-25T09:38:53.094160+0000","last_became_peered":"2023-08-25T09:38:53.094160+0000","last_unstale":"2023-08-25T09:38:54.077761+0000","last_undegraded":"2023-08-25T09:38:54.077761+0000","last_fullsized":"2023-08-25T09:38:54.077761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:05:17.873255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169761+0000","last_change":"2023-08-25T09:38:53.109739+0000","last_active":"2023-08-25T09:38:54.169761+0000","last_peered":"2023-08-25T09:38:54.169761+0000","last_clean":"2023-08-25T09:38:54.169761+0000","last_became_active":"2023-08-25T09:38:53.106937+0000","last_became_peered":"2023-08-25T09:38:53.106937+0000","last_unstale":"2023-08-25T09:38:54.169761+0000","last_undegraded":"2023-08-25T09:38:54.169761+0000","last_fullsized":"2023-08-25T09:38:54.169761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170229+0000","last_change":"2023-08-25T09:38:53.102490+0000","last_active":"2023-08-25T09:38:54.170229+0000","last_peered":"2023-08-25T09:38:54.170229+0000","last_clean":"2023-08-25T09:38:54.170229+0000","last_became_active":"2023-08-25T09:38:53.101594+0000","last_became_peered":"2023-08-25T09:38:53.101594+0000","last_unstale":"2023-08-25T09:38:54.170229+0000","last_undegraded":"2023-08-25T09:38:54.170229+0000","last_fullsized":"2023-08-25T09:38:54.170229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:49.023690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214623+0000","last_change":"2023-08-25T09:38:53.092816+0000","last_active":"2023-08-25T09:38:54.214623+0000","last_peered":"2023-08-25T09:38:54.214623+0000","last_clean":"2023-08-25T09:38:54.214623+0000","last_became_active":"2023-08-25T09:38:53.092392+0000","last_became_peered":"2023-08-25T09:38:53.092392+0000","last_unstale":"2023-08-25T09:38:54.214623+0000","last_undegraded":"2023-08-25T09:38:54.214623+0000","last_fullsized":"2023-08-25T09:38:54.214623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079002+0000","last_change":"2023-08-25T09:38:53.104234+0000","last_active":"2023-08-25T09:38:54.079002+0000","last_peered":"2023-08-25T09:38:54.079002+0000","last_clean":"2023-08-25T09:38:54.079002+0000","last_became_active":"2023-08-25T09:38:53.104004+0000","last_became_peered":"2023-08-25T09:38:53.104004+0000","last_unstale":"2023-08-25T09:38:54.079002+0000","last_undegraded":"2023-08-25T09:38:54.079002+0000","last_fullsized":"2023-08-25T09:38:54.079002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167124+0000","last_change":"2023-08-25T09:38:53.106635+0000","last_active":"2023-08-25T09:38:54.167124+0000","last_peered":"2023-08-25T09:38:54.167124+0000","last_clean":"2023-08-25T09:38:54.167124+0000","last_became_active":"2023-08-25T09:38:53.106105+0000","last_became_peered":"2023-08-25T09:38:53.106105+0000","last_unstale":"2023-08-25T09:38:54.167124+0000","last_undegraded":"2023-08-25T09:38:54.167124+0000","last_fullsized":"2023-08-25T09:38:54.167124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078322+0000","last_change":"2023-08-25T09:38:53.108366+0000","last_active":"2023-08-25T09:38:54.078322+0000","last_peered":"2023-08-25T09:38:54.078322+0000","last_clean":"2023-08-25T09:38:54.078322+0000","last_became_active":"2023-08-25T09:38:53.108123+0000","last_became_peered":"2023-08-25T09:38:53.108123+0000","last_unstale":"2023-08-25T09:38:54.078322+0000","last_undegraded":"2023-08-25T09:38:54.078322+0000","last_fullsized":"2023-08-25T09:38:54.078322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077197+0000","last_change":"2023-08-25T09:38:52.142687+0000","last_active":"2023-08-25T09:38:54.077197+0000","last_peered":"2023-08-25T09:38:54.077197+0000","last_clean":"2023-08-25T09:38:54.077197+0000","last_became_active":"2023-08-25T09:38:52.140330+0000","last_became_peered":"2023-08-25T09:38:52.140330+0000","last_unstale":"2023-08-25T09:38:54.077197+0000","last_undegraded":"2023-08-25T09:38:54.077197+0000","last_fullsized":"2023-08-25T09:38:54.077197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:42.836964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076792+0000","last_change":"2023-08-25T09:38:53.094431+0000","last_active":"2023-08-25T09:38:54.076792+0000","last_peered":"2023-08-25T09:38:54.076792+0000","last_clean":"2023-08-25T09:38:54.076792+0000","last_became_active":"2023-08-25T09:38:53.093756+0000","last_became_peered":"2023-08-25T09:38:53.093756+0000","last_unstale":"2023-08-25T09:38:54.076792+0000","last_undegraded":"2023-08-25T09:38:54.076792+0000","last_fullsized":"2023-08-25T09:38:54.076792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:32.683806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079876+0000","last_change":"2023-08-25T09:38:52.138724+0000","last_active":"2023-08-25T09:38:54.079876+0000","last_peered":"2023-08-25T09:38:54.079876+0000","last_clean":"2023-08-25T09:38:54.079876+0000","last_became_active":"2023-08-25T09:38:52.138323+0000","last_became_peered":"2023-08-25T09:38:52.138323+0000","last_unstale":"2023-08-25T09:38:54.079876+0000","last_undegraded":"2023-08-25T09:38:54.079876+0000","last_fullsized":"2023-08-25T09:38:54.079876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170651+0000","last_change":"2023-08-25T09:38:53.098073+0000","last_active":"2023-08-25T09:38:54.170651+0000","last_peered":"2023-08-25T09:38:54.170651+0000","last_clean":"2023-08-25T09:38:54.170651+0000","last_became_active":"2023-08-25T09:38:53.089135+0000","last_became_peered":"2023-08-25T09:38:53.089135+0000","last_unstale":"2023-08-25T09:38:54.170651+0000","last_undegraded":"2023-08-25T09:38:54.170651+0000","last_fullsized":"2023-08-25T09:38:54.170651+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078034+0000","last_change":"2023-08-25T09:38:52.146382+0000","last_active":"2023-08-25T09:38:54.078034+0000","last_peered":"2023-08-25T09:38:54.078034+0000","last_clean":"2023-08-25T09:38:54.078034+0000","last_became_active":"2023-08-25T09:38:52.145954+0000","last_became_peered":"2023-08-25T09:38:52.145954+0000","last_unstale":"2023-08-25T09:38:54.078034+0000","last_undegraded":"2023-08-25T09:38:54.078034+0000","last_fullsized":"2023-08-25T09:38:54.078034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:02.701588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169581+0000","last_change":"2023-08-25T09:38:53.106145+0000","last_active":"2023-08-25T09:38:54.169581+0000","last_peered":"2023-08-25T09:38:54.169581+0000","last_clean":"2023-08-25T09:38:54.169581+0000","last_became_active":"2023-08-25T09:38:53.105502+0000","last_became_peered":"2023-08-25T09:38:53.105502+0000","last_unstale":"2023-08-25T09:38:54.169581+0000","last_undegraded":"2023-08-25T09:38:54.169581+0000","last_fullsized":"2023-08-25T09:38:54.169581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:01.061628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167472+0000","last_change":"2023-08-25T09:38:52.148928+0000","last_active":"2023-08-25T09:38:54.167472+0000","last_peered":"2023-08-25T09:38:54.167472+0000","last_clean":"2023-08-25T09:38:54.167472+0000","last_became_active":"2023-08-25T09:38:52.148227+0000","last_became_peered":"2023-08-25T09:38:52.148227+0000","last_unstale":"2023-08-25T09:38:54.167472+0000","last_undegraded":"2023-08-25T09:38:54.167472+0000","last_fullsized":"2023-08-25T09:38:54.167472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.214435+0000","last_change":"2023-08-25T09:38:52.138014+0000","last_active":"2023-08-25T09:38:54.214435+0000","last_peered":"2023-08-25T09:38:54.214435+0000","last_clean":"2023-08-25T09:38:54.214435+0000","last_became_active":"2023-08-25T09:38:52.137777+0000","last_became_peered":"2023-08-25T09:38:52.137777+0000","last_unstale":"2023-08-25T09:38:54.214435+0000","last_undegraded":"2023-08-25T09:38:54.214435+0000","last_fullsized":"2023-08-25T09:38:54.214435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:10.214913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077886+0000","last_change":"2023-08-25T09:38:53.100050+0000","last_active":"2023-08-25T09:38:54.077886+0000","last_peered":"2023-08-25T09:38:54.077886+0000","last_clean":"2023-08-25T09:38:54.077886+0000","last_became_active":"2023-08-25T09:38:53.099406+0000","last_became_peered":"2023-08-25T09:38:53.099406+0000","last_unstale":"2023-08-25T09:38:54.077886+0000","last_undegraded":"2023-08-25T09:38:54.077886+0000","last_fullsized":"2023-08-25T09:38:54.077886+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:46.330038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186222+0000","last_change":"2023-08-25T09:38:52.145560+0000","last_active":"2023-08-25T09:38:54.186222+0000","last_peered":"2023-08-25T09:38:54.186222+0000","last_clean":"2023-08-25T09:38:54.186222+0000","last_became_active":"2023-08-25T09:38:52.145324+0000","last_became_peered":"2023-08-25T09:38:52.145324+0000","last_unstale":"2023-08-25T09:38:54.186222+0000","last_undegraded":"2023-08-25T09:38:54.186222+0000","last_fullsized":"2023-08-25T09:38:54.186222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:50:47.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167557+0000","last_change":"2023-08-25T09:38:53.105196+0000","last_active":"2023-08-25T09:38:54.167557+0000","last_peered":"2023-08-25T09:38:54.167557+0000","last_clean":"2023-08-25T09:38:54.167557+0000","last_became_active":"2023-08-25T09:38:53.104948+0000","last_became_peered":"2023-08-25T09:38:53.104948+0000","last_unstale":"2023-08-25T09:38:54.167557+0000","last_undegraded":"2023-08-25T09:38:54.167557+0000","last_fullsized":"2023-08-25T09:38:54.167557+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078179+0000","last_change":"2023-08-25T09:38:52.140961+0000","last_active":"2023-08-25T09:38:54.078179+0000","last_peered":"2023-08-25T09:38:54.078179+0000","last_clean":"2023-08-25T09:38:54.078179+0000","last_became_active":"2023-08-25T09:38:52.137710+0000","last_became_peered":"2023-08-25T09:38:52.137710+0000","last_unstale":"2023-08-25T09:38:54.078179+0000","last_undegraded":"2023-08-25T09:38:54.078179+0000","last_fullsized":"2023-08-25T09:38:54.078179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:09.490525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.083467+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.082859+0000","last_became_peered":"2023-08-25T09:38:53.082859+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215256+0000","last_change":"2023-08-25T09:38:52.148012+0000","last_active":"2023-08-25T09:38:54.215256+0000","last_peered":"2023-08-25T09:38:54.215256+0000","last_clean":"2023-08-25T09:38:54.215256+0000","last_became_active":"2023-08-25T09:38:52.147757+0000","last_became_peered":"2023-08-25T09:38:52.147757+0000","last_unstale":"2023-08-25T09:38:54.215256+0000","last_undegraded":"2023-08-25T09:38:54.215256+0000","last_fullsized":"2023-08-25T09:38:54.215256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:36.644423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170501+0000","last_change":"2023-08-25T09:38:53.110777+0000","last_active":"2023-08-25T09:38:54.170501+0000","last_peered":"2023-08-25T09:38:54.170501+0000","last_clean":"2023-08-25T09:38:54.170501+0000","last_became_active":"2023-08-25T09:38:53.108970+0000","last_became_peered":"2023-08-25T09:38:53.108970+0000","last_unstale":"2023-08-25T09:38:54.170501+0000","last_undegraded":"2023-08-25T09:38:54.170501+0000","last_fullsized":"2023-08-25T09:38:54.170501+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:52.823383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080573+0000","last_change":"2023-08-25T09:38:52.144190+0000","last_active":"2023-08-25T09:38:54.080573+0000","last_peered":"2023-08-25T09:38:54.080573+0000","last_clean":"2023-08-25T09:38:54.080573+0000","last_became_active":"2023-08-25T09:38:52.143875+0000","last_became_peered":"2023-08-25T09:38:52.143875+0000","last_unstale":"2023-08-25T09:38:54.080573+0000","last_undegraded":"2023-08-25T09:38:54.080573+0000","last_fullsized":"2023-08-25T09:38:54.080573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215099+0000","last_change":"2023-08-25T09:38:53.106602+0000","last_active":"2023-08-25T09:38:54.215099+0000","last_peered":"2023-08-25T09:38:54.215099+0000","last_clean":"2023-08-25T09:38:54.215099+0000","last_became_active":"2023-08-25T09:38:53.106340+0000","last_became_peered":"2023-08-25T09:38:53.106340+0000","last_unstale":"2023-08-25T09:38:54.215099+0000","last_undegraded":"2023-08-25T09:38:54.215099+0000","last_fullsized":"2023-08-25T09:38:54.215099+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076776+0000","last_change":"2023-08-25T09:38:52.140744+0000","last_active":"2023-08-25T09:38:54.076776+0000","last_peered":"2023-08-25T09:38:54.076776+0000","last_clean":"2023-08-25T09:38:54.076776+0000","last_became_active":"2023-08-25T09:38:52.140450+0000","last_became_peered":"2023-08-25T09:38:52.140450+0000","last_unstale":"2023-08-25T09:38:54.076776+0000","last_undegraded":"2023-08-25T09:38:54.076776+0000","last_fullsized":"2023-08-25T09:38:54.076776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:54:37.632752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077689+0000","last_change":"2023-08-25T09:38:53.100210+0000","last_active":"2023-08-25T09:38:54.077689+0000","last_peered":"2023-08-25T09:38:54.077689+0000","last_clean":"2023-08-25T09:38:54.077689+0000","last_became_active":"2023-08-25T09:38:53.099792+0000","last_became_peered":"2023-08-25T09:38:53.099792+0000","last_unstale":"2023-08-25T09:38:54.077689+0000","last_undegraded":"2023-08-25T09:38:54.077689+0000","last_fullsized":"2023-08-25T09:38:54.077689+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:37.979410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077328+0000","last_change":"2023-08-25T09:38:52.132704+0000","last_active":"2023-08-25T09:38:54.077328+0000","last_peered":"2023-08-25T09:38:54.077328+0000","last_clean":"2023-08-25T09:38:54.077328+0000","last_became_active":"2023-08-25T09:38:52.132045+0000","last_became_peered":"2023-08-25T09:38:52.132045+0000","last_unstale":"2023-08-25T09:38:54.077328+0000","last_undegraded":"2023-08-25T09:38:54.077328+0000","last_fullsized":"2023-08-25T09:38:54.077328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:32:14.124629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186144+0000","last_change":"2023-08-25T09:38:53.095472+0000","last_active":"2023-08-25T09:38:54.186144+0000","last_peered":"2023-08-25T09:38:54.186144+0000","last_clean":"2023-08-25T09:38:54.186144+0000","last_became_active":"2023-08-25T09:38:53.095117+0000","last_became_peered":"2023-08-25T09:38:53.095117+0000","last_unstale":"2023-08-25T09:38:54.186144+0000","last_undegraded":"2023-08-25T09:38:54.186144+0000","last_fullsized":"2023-08-25T09:38:54.186144+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:50.565866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169438+0000","last_change":"2023-08-25T09:38:52.142123+0000","last_active":"2023-08-25T09:38:54.169438+0000","last_peered":"2023-08-25T09:38:54.169438+0000","last_clean":"2023-08-25T09:38:54.169438+0000","last_became_active":"2023-08-25T09:38:52.141349+0000","last_became_peered":"2023-08-25T09:38:52.141349+0000","last_unstale":"2023-08-25T09:38:54.169438+0000","last_undegraded":"2023-08-25T09:38:54.169438+0000","last_fullsized":"2023-08-25T09:38:54.169438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078490+0000","last_change":"2023-08-25T09:38:53.101599+0000","last_active":"2023-08-25T09:38:54.078490+0000","last_peered":"2023-08-25T09:38:54.078490+0000","last_clean":"2023-08-25T09:38:54.078490+0000","last_became_active":"2023-08-25T09:38:53.101348+0000","last_became_peered":"2023-08-25T09:38:53.101348+0000","last_unstale":"2023-08-25T09:38:54.078490+0000","last_undegraded":"2023-08-25T09:38:54.078490+0000","last_fullsized":"2023-08-25T09:38:54.078490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167829+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:38:54.167829+0000","last_peered":"2023-08-25T09:38:54.167829+0000","last_clean":"2023-08-25T09:38:54.167829+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:38:54.167829+0000","last_undegraded":"2023-08-25T09:38:54.167829+0000","last_fullsized":"2023-08-25T09:38:54.167829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:32.241680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078306+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:38:54.078306+0000","last_peered":"2023-08-25T09:38:54.078306+0000","last_clean":"2023-08-25T09:38:54.078306+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:38:54.078306+0000","last_undegraded":"2023-08-25T09:38:54.078306+0000","last_fullsized":"2023-08-25T09:38:54.078306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:40:49.888170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170349+0000","last_change":"2023-08-25T09:38:52.145925+0000","last_active":"2023-08-25T09:38:54.170349+0000","last_peered":"2023-08-25T09:38:54.170349+0000","last_clean":"2023-08-25T09:38:54.170349+0000","last_became_active":"2023-08-25T09:38:52.145572+0000","last_became_peered":"2023-08-25T09:38:52.145572+0000","last_unstale":"2023-08-25T09:38:54.170349+0000","last_undegraded":"2023-08-25T09:38:54.170349+0000","last_fullsized":"2023-08-25T09:38:54.170349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:19:18.032882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076523+0000","last_change":"2023-08-25T09:38:53.098547+0000","last_active":"2023-08-25T09:38:54.076523+0000","last_peered":"2023-08-25T09:38:54.076523+0000","last_clean":"2023-08-25T09:38:54.076523+0000","last_became_active":"2023-08-25T09:38:53.098162+0000","last_became_peered":"2023-08-25T09:38:53.098162+0000","last_unstale":"2023-08-25T09:38:54.076523+0000","last_undegraded":"2023-08-25T09:38:54.076523+0000","last_fullsized":"2023-08-25T09:38:54.076523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:14.282266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076947+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:38:54.076947+0000","last_peered":"2023-08-25T09:38:54.076947+0000","last_clean":"2023-08-25T09:38:54.076947+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:38:54.076947+0000","last_undegraded":"2023-08-25T09:38:54.076947+0000","last_fullsized":"2023-08-25T09:38:54.076947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:41.461997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079667+0000","last_change":"2023-08-25T09:38:52.146575+0000","last_active":"2023-08-25T09:38:54.079667+0000","last_peered":"2023-08-25T09:38:54.079667+0000","last_clean":"2023-08-25T09:38:54.079667+0000","last_became_active":"2023-08-25T09:38:52.146263+0000","last_became_peered":"2023-08-25T09:38:52.146263+0000","last_unstale":"2023-08-25T09:38:54.079667+0000","last_undegraded":"2023-08-25T09:38:54.079667+0000","last_fullsized":"2023-08-25T09:38:54.079667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:58.354866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170732+0000","last_change":"2023-08-25T09:38:53.110355+0000","last_active":"2023-08-25T09:38:54.170732+0000","last_peered":"2023-08-25T09:38:54.170732+0000","last_clean":"2023-08-25T09:38:54.170732+0000","last_became_active":"2023-08-25T09:38:53.107934+0000","last_became_peered":"2023-08-25T09:38:53.107934+0000","last_unstale":"2023-08-25T09:38:54.170732+0000","last_undegraded":"2023-08-25T09:38:54.170732+0000","last_fullsized":"2023-08-25T09:38:54.170732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080971+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:38:54.080971+0000","last_peered":"2023-08-25T09:38:54.080971+0000","last_clean":"2023-08-25T09:38:54.080971+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:38:54.080971+0000","last_undegraded":"2023-08-25T09:38:54.080971+0000","last_fullsized":"2023-08-25T09:38:54.080971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:01.142595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170917+0000","last_change":"2023-08-25T09:38:52.149368+0000","last_active":"2023-08-25T09:38:54.170917+0000","last_peered":"2023-08-25T09:38:54.170917+0000","last_clean":"2023-08-25T09:38:54.170917+0000","last_became_active":"2023-08-25T09:38:52.149118+0000","last_became_peered":"2023-08-25T09:38:52.149118+0000","last_unstale":"2023-08-25T09:38:54.170917+0000","last_undegraded":"2023-08-25T09:38:54.170917+0000","last_fullsized":"2023-08-25T09:38:54.170917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:16.835823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078872+0000","last_change":"2023-08-25T09:38:53.108258+0000","last_active":"2023-08-25T09:38:54.078872+0000","last_peered":"2023-08-25T09:38:54.078872+0000","last_clean":"2023-08-25T09:38:54.078872+0000","last_became_active":"2023-08-25T09:38:53.107991+0000","last_became_peered":"2023-08-25T09:38:53.107991+0000","last_unstale":"2023-08-25T09:38:54.078872+0000","last_undegraded":"2023-08-25T09:38:54.078872+0000","last_fullsized":"2023-08-25T09:38:54.078872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:57.646414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078433+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:38:54.078433+0000","last_peered":"2023-08-25T09:38:54.078433+0000","last_clean":"2023-08-25T09:38:54.078433+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:38:54.078433+0000","last_undegraded":"2023-08-25T09:38:54.078433+0000","last_fullsized":"2023-08-25T09:38:54.078433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:17.906346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215184+0000","last_change":"2023-08-25T09:38:52.146434+0000","last_active":"2023-08-25T09:38:54.215184+0000","last_peered":"2023-08-25T09:38:54.215184+0000","last_clean":"2023-08-25T09:38:54.215184+0000","last_became_active":"2023-08-25T09:38:52.146126+0000","last_became_peered":"2023-08-25T09:38:52.146126+0000","last_unstale":"2023-08-25T09:38:54.215184+0000","last_undegraded":"2023-08-25T09:38:54.215184+0000","last_fullsized":"2023-08-25T09:38:54.215184+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079123+0000","last_change":"2023-08-25T09:38:53.097778+0000","last_active":"2023-08-25T09:38:54.079123+0000","last_peered":"2023-08-25T09:38:54.079123+0000","last_clean":"2023-08-25T09:38:54.079123+0000","last_became_active":"2023-08-25T09:38:53.097182+0000","last_became_peered":"2023-08-25T09:38:53.097182+0000","last_unstale":"2023-08-25T09:38:54.079123+0000","last_undegraded":"2023-08-25T09:38:54.079123+0000","last_fullsized":"2023-08-25T09:38:54.079123+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:11.219234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185740+0000","last_change":"2023-08-25T09:38:52.143868+0000","last_active":"2023-08-25T09:38:54.185740+0000","last_peered":"2023-08-25T09:38:54.185740+0000","last_clean":"2023-08-25T09:38:54.185740+0000","last_became_active":"2023-08-25T09:38:52.143584+0000","last_became_peered":"2023-08-25T09:38:52.143584+0000","last_unstale":"2023-08-25T09:38:54.185740+0000","last_undegraded":"2023-08-25T09:38:54.185740+0000","last_fullsized":"2023-08-25T09:38:54.185740+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171239+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:38:54.171239+0000","last_peered":"2023-08-25T09:38:54.171239+0000","last_clean":"2023-08-25T09:38:54.171239+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:38:54.171239+0000","last_undegraded":"2023-08-25T09:38:54.171239+0000","last_fullsized":"2023-08-25T09:38:54.171239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:38.295927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079243+0000","last_change":"2023-08-25T09:38:53.098704+0000","last_active":"2023-08-25T09:38:54.079243+0000","last_peered":"2023-08-25T09:38:54.079243+0000","last_clean":"2023-08-25T09:38:54.079243+0000","last_became_active":"2023-08-25T09:38:53.098360+0000","last_became_peered":"2023-08-25T09:38:53.098360+0000","last_unstale":"2023-08-25T09:38:54.079243+0000","last_undegraded":"2023-08-25T09:38:54.079243+0000","last_fullsized":"2023-08-25T09:38:54.079243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079454+0000","last_change":"2023-08-25T09:38:52.146452+0000","last_active":"2023-08-25T09:38:54.079454+0000","last_peered":"2023-08-25T09:38:54.079454+0000","last_clean":"2023-08-25T09:38:54.079454+0000","last_became_active":"2023-08-25T09:38:52.146193+0000","last_became_peered":"2023-08-25T09:38:52.146193+0000","last_unstale":"2023-08-25T09:38:54.079454+0000","last_undegraded":"2023-08-25T09:38:54.079454+0000","last_fullsized":"2023-08-25T09:38:54.079454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:15.464131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076841+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:38:54.076841+0000","last_peered":"2023-08-25T09:38:54.076841+0000","last_clean":"2023-08-25T09:38:54.076841+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:38:54.076841+0000","last_undegraded":"2023-08-25T09:38:54.076841+0000","last_fullsized":"2023-08-25T09:38:54.076841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:16.377836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.079556+0000","last_change":"2023-08-25T09:38:53.109546+0000","last_active":"2023-08-25T09:38:54.079556+0000","last_peered":"2023-08-25T09:38:54.079556+0000","last_clean":"2023-08-25T09:38:54.079556+0000","last_became_active":"2023-08-25T09:38:53.109238+0000","last_became_peered":"2023-08-25T09:38:53.109238+0000","last_unstale":"2023-08-25T09:38:54.079556+0000","last_undegraded":"2023-08-25T09:38:54.079556+0000","last_fullsized":"2023-08-25T09:38:54.079556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170022+0000","last_change":"2023-08-25T09:38:52.144544+0000","last_active":"2023-08-25T09:38:54.170022+0000","last_peered":"2023-08-25T09:38:54.170022+0000","last_clean":"2023-08-25T09:38:54.170022+0000","last_became_active":"2023-08-25T09:38:52.144095+0000","last_became_peered":"2023-08-25T09:38:52.144095+0000","last_unstale":"2023-08-25T09:38:54.170022+0000","last_undegraded":"2023-08-25T09:38:54.170022+0000","last_fullsized":"2023-08-25T09:38:54.170022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:08.611235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169271+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:38:54.169271+0000","last_peered":"2023-08-25T09:38:54.169271+0000","last_clean":"2023-08-25T09:38:54.169271+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:38:54.169271+0000","last_undegraded":"2023-08-25T09:38:54.169271+0000","last_fullsized":"2023-08-25T09:38:54.169271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:11.717086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169145+0000","last_change":"2023-08-25T09:38:53.112168+0000","last_active":"2023-08-25T09:38:54.169145+0000","last_peered":"2023-08-25T09:38:54.169145+0000","last_clean":"2023-08-25T09:38:54.169145+0000","last_became_active":"2023-08-25T09:38:53.107808+0000","last_became_peered":"2023-08-25T09:38:53.107808+0000","last_unstale":"2023-08-25T09:38:54.169145+0000","last_undegraded":"2023-08-25T09:38:54.169145+0000","last_fullsized":"2023-08-25T09:38:54.169145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:34.812577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.337779+0000","last_change":"2023-08-25T09:38:52.143505+0000","last_active":"2023-08-25T09:38:54.337779+0000","last_peered":"2023-08-25T09:38:54.337779+0000","last_clean":"2023-08-25T09:38:54.337779+0000","last_became_active":"2023-08-25T09:38:52.143175+0000","last_became_peered":"2023-08-25T09:38:52.143175+0000","last_unstale":"2023-08-25T09:38:54.337779+0000","last_undegraded":"2023-08-25T09:38:54.337779+0000","last_fullsized":"2023-08-25T09:38:54.337779+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:38:51.819062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078207+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:38:54.078207+0000","last_peered":"2023-08-25T09:38:54.078207+0000","last_clean":"2023-08-25T09:38:54.078207+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:38:54.078207+0000","last_undegraded":"2023-08-25T09:38:54.078207+0000","last_fullsized":"2023-08-25T09:38:54.078207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:28:24.021135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185610+0000","last_change":"2023-08-25T09:38:53.095808+0000","last_active":"2023-08-25T09:38:54.185610+0000","last_peered":"2023-08-25T09:38:54.185610+0000","last_clean":"2023-08-25T09:38:54.185610+0000","last_became_active":"2023-08-25T09:38:53.095427+0000","last_became_peered":"2023-08-25T09:38:53.095427+0000","last_unstale":"2023-08-25T09:38:54.185610+0000","last_undegraded":"2023-08-25T09:38:54.185610+0000","last_fullsized":"2023-08-25T09:38:54.185610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186067+0000","last_change":"2023-08-25T09:38:53.094102+0000","last_active":"2023-08-25T09:38:54.186067+0000","last_peered":"2023-08-25T09:38:54.186067+0000","last_clean":"2023-08-25T09:38:54.186067+0000","last_became_active":"2023-08-25T09:38:53.093658+0000","last_became_peered":"2023-08-25T09:38:53.093658+0000","last_unstale":"2023-08-25T09:38:54.186067+0000","last_undegraded":"2023-08-25T09:38:54.186067+0000","last_fullsized":"2023-08-25T09:38:54.186067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:27:37.648744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076645+0000","last_change":"2023-08-25T09:38:52.145307+0000","last_active":"2023-08-25T09:38:54.076645+0000","last_peered":"2023-08-25T09:38:54.076645+0000","last_clean":"2023-08-25T09:38:54.076645+0000","last_became_active":"2023-08-25T09:38:52.145076+0000","last_became_peered":"2023-08-25T09:38:52.145076+0000","last_unstale":"2023-08-25T09:38:54.076645+0000","last_undegraded":"2023-08-25T09:38:54.076645+0000","last_fullsized":"2023-08-25T09:38:54.076645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:16.527497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.168761+0000","last_change":"2023-08-25T09:38:53.105922+0000","last_active":"2023-08-25T09:38:54.168761+0000","last_peered":"2023-08-25T09:38:54.168761+0000","last_clean":"2023-08-25T09:38:54.168761+0000","last_became_active":"2023-08-25T09:38:53.105488+0000","last_became_peered":"2023-08-25T09:38:53.105488+0000","last_unstale":"2023-08-25T09:38:54.168761+0000","last_undegraded":"2023-08-25T09:38:54.168761+0000","last_fullsized":"2023-08-25T09:38:54.168761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:06.334854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.215307+0000","last_change":"2023-08-25T09:38:52.142639+0000","last_active":"2023-08-25T09:38:54.215307+0000","last_peered":"2023-08-25T09:38:54.215307+0000","last_clean":"2023-08-25T09:38:54.215307+0000","last_became_active":"2023-08-25T09:38:52.142406+0000","last_became_peered":"2023-08-25T09:38:52.142406+0000","last_unstale":"2023-08-25T09:38:54.215307+0000","last_undegraded":"2023-08-25T09:38:54.215307+0000","last_fullsized":"2023-08-25T09:38:54.215307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:49.561545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.186057+0000","last_change":"2023-08-25T09:38:52.145084+0000","last_active":"2023-08-25T09:38:54.186057+0000","last_peered":"2023-08-25T09:38:54.186057+0000","last_clean":"2023-08-25T09:38:54.186057+0000","last_became_active":"2023-08-25T09:38:52.144835+0000","last_became_peered":"2023-08-25T09:38:52.144835+0000","last_unstale":"2023-08-25T09:38:54.186057+0000","last_undegraded":"2023-08-25T09:38:54.186057+0000","last_fullsized":"2023-08-25T09:38:54.186057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:00.057308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169334+0000","last_change":"2023-08-25T09:38:53.107353+0000","last_active":"2023-08-25T09:38:54.169334+0000","last_peered":"2023-08-25T09:38:54.169334+0000","last_clean":"2023-08-25T09:38:54.169334+0000","last_became_active":"2023-08-25T09:38:53.106537+0000","last_became_peered":"2023-08-25T09:38:53.106537+0000","last_unstale":"2023-08-25T09:38:54.169334+0000","last_undegraded":"2023-08-25T09:38:54.169334+0000","last_fullsized":"2023-08-25T09:38:54.169334+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:53.892642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170432+0000","last_change":"2023-08-25T09:38:52.142676+0000","last_active":"2023-08-25T09:38:54.170432+0000","last_peered":"2023-08-25T09:38:54.170432+0000","last_clean":"2023-08-25T09:38:54.170432+0000","last_became_active":"2023-08-25T09:38:52.142266+0000","last_became_peered":"2023-08-25T09:38:52.142266+0000","last_unstale":"2023-08-25T09:38:54.170432+0000","last_undegraded":"2023-08-25T09:38:54.170432+0000","last_fullsized":"2023-08-25T09:38:54.170432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:05.330534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.080254+0000","last_change":"2023-08-25T09:38:53.092162+0000","last_active":"2023-08-25T09:38:54.080254+0000","last_peered":"2023-08-25T09:38:54.080254+0000","last_clean":"2023-08-25T09:38:54.080254+0000","last_became_active":"2023-08-25T09:38:53.091630+0000","last_became_peered":"2023-08-25T09:38:53.091630+0000","last_unstale":"2023-08-25T09:38:54.080254+0000","last_undegraded":"2023-08-25T09:38:54.080254+0000","last_fullsized":"2023-08-25T09:38:54.080254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:49.502775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.076680+0000","last_change":"2023-08-25T09:38:53.104609+0000","last_active":"2023-08-25T09:38:54.076680+0000","last_peered":"2023-08-25T09:38:54.076680+0000","last_clean":"2023-08-25T09:38:54.076680+0000","last_became_active":"2023-08-25T09:38:53.104257+0000","last_became_peered":"2023-08-25T09:38:53.104257+0000","last_unstale":"2023-08-25T09:38:54.076680+0000","last_undegraded":"2023-08-25T09:38:54.076680+0000","last_fullsized":"2023-08-25T09:38:54.076680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:15.255372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.171027+0000","last_change":"2023-08-25T09:38:52.149252+0000","last_active":"2023-08-25T09:38:54.171027+0000","last_peered":"2023-08-25T09:38:54.171027+0000","last_clean":"2023-08-25T09:38:54.171027+0000","last_became_active":"2023-08-25T09:38:52.148775+0000","last_became_peered":"2023-08-25T09:38:52.148775+0000","last_unstale":"2023-08-25T09:38:54.171027+0000","last_undegraded":"2023-08-25T09:38:54.171027+0000","last_fullsized":"2023-08-25T09:38:54.171027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169503+0000","last_change":"2023-08-25T09:38:53.112427+0000","last_active":"2023-08-25T09:38:54.169503+0000","last_peered":"2023-08-25T09:38:54.169503+0000","last_clean":"2023-08-25T09:38:54.169503+0000","last_became_active":"2023-08-25T09:38:53.110947+0000","last_became_peered":"2023-08-25T09:38:53.110947+0000","last_unstale":"2023-08-25T09:38:54.169503+0000","last_undegraded":"2023-08-25T09:38:54.169503+0000","last_fullsized":"2023-08-25T09:38:54.169503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:35.927820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169802+0000","last_change":"2023-08-25T09:38:52.140188+0000","last_active":"2023-08-25T09:38:54.169802+0000","last_peered":"2023-08-25T09:38:54.169802+0000","last_clean":"2023-08-25T09:38:54.169802+0000","last_became_active":"2023-08-25T09:38:52.139523+0000","last_became_peered":"2023-08-25T09:38:52.139523+0000","last_unstale":"2023-08-25T09:38:54.169802+0000","last_undegraded":"2023-08-25T09:38:54.169802+0000","last_fullsized":"2023-08-25T09:38:54.169802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:04.643863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078279+0000","last_change":"2023-08-25T09:38:53.089895+0000","last_active":"2023-08-25T09:38:54.078279+0000","last_peered":"2023-08-25T09:38:54.078279+0000","last_clean":"2023-08-25T09:38:54.078279+0000","last_became_active":"2023-08-25T09:38:53.089211+0000","last_became_peered":"2023-08-25T09:38:53.089211+0000","last_unstale":"2023-08-25T09:38:54.078279+0000","last_undegraded":"2023-08-25T09:38:54.078279+0000","last_fullsized":"2023-08-25T09:38:54.078279+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:57.744266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167322+0000","last_change":"2023-08-25T09:38:52.149088+0000","last_active":"2023-08-25T09:38:54.167322+0000","last_peered":"2023-08-25T09:38:54.167322+0000","last_clean":"2023-08-25T09:38:54.167322+0000","last_became_active":"2023-08-25T09:38:52.148577+0000","last_became_peered":"2023-08-25T09:38:52.148577+0000","last_unstale":"2023-08-25T09:38:54.167322+0000","last_undegraded":"2023-08-25T09:38:54.167322+0000","last_fullsized":"2023-08-25T09:38:54.167322+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:48.019370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169133+0000","last_change":"2023-08-25T09:38:53.105736+0000","last_active":"2023-08-25T09:38:54.169133+0000","last_peered":"2023-08-25T09:38:54.169133+0000","last_clean":"2023-08-25T09:38:54.169133+0000","last_became_active":"2023-08-25T09:38:53.105084+0000","last_became_peered":"2023-08-25T09:38:53.105084+0000","last_unstale":"2023-08-25T09:38:54.169133+0000","last_undegraded":"2023-08-25T09:38:54.169133+0000","last_fullsized":"2023-08-25T09:38:54.169133+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:14.964126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.169659+0000","last_change":"2023-08-25T09:38:52.144372+0000","last_active":"2023-08-25T09:38:54.169659+0000","last_peered":"2023-08-25T09:38:54.169659+0000","last_clean":"2023-08-25T09:38:54.169659+0000","last_became_active":"2023-08-25T09:38:52.143646+0000","last_became_peered":"2023-08-25T09:38:52.143646+0000","last_unstale":"2023-08-25T09:38:54.169659+0000","last_undegraded":"2023-08-25T09:38:54.169659+0000","last_fullsized":"2023-08-25T09:38:54.169659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:43.414664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185820+0000","last_change":"2023-08-25T09:38:53.091214+0000","last_active":"2023-08-25T09:38:54.185820+0000","last_peered":"2023-08-25T09:38:54.185820+0000","last_clean":"2023-08-25T09:38:54.185820+0000","last_became_active":"2023-08-25T09:38:53.090817+0000","last_became_peered":"2023-08-25T09:38:53.090817+0000","last_unstale":"2023-08-25T09:38:54.185820+0000","last_undegraded":"2023-08-25T09:38:54.185820+0000","last_fullsized":"2023-08-25T09:38:54.185820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:07.265653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.167705+0000","last_change":"2023-08-25T09:38:52.139178+0000","last_active":"2023-08-25T09:38:54.167705+0000","last_peered":"2023-08-25T09:38:54.167705+0000","last_clean":"2023-08-25T09:38:54.167705+0000","last_became_active":"2023-08-25T09:38:52.138923+0000","last_became_peered":"2023-08-25T09:38:52.138923+0000","last_unstale":"2023-08-25T09:38:54.167705+0000","last_undegraded":"2023-08-25T09:38:54.167705+0000","last_fullsized":"2023-08-25T09:38:54.167705+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:47:34.923499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185462+0000","last_change":"2023-08-25T09:38:53.098793+0000","last_active":"2023-08-25T09:38:54.185462+0000","last_peered":"2023-08-25T09:38:54.185462+0000","last_clean":"2023-08-25T09:38:54.185462+0000","last_became_active":"2023-08-25T09:38:53.097585+0000","last_became_peered":"2023-08-25T09:38:53.097585+0000","last_unstale":"2023-08-25T09:38:54.185462+0000","last_undegraded":"2023-08-25T09:38:54.185462+0000","last_fullsized":"2023-08-25T09:38:54.185462+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:35.228835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.077130+0000","last_change":"2023-08-25T09:38:52.143009+0000","last_active":"2023-08-25T09:38:54.077130+0000","last_peered":"2023-08-25T09:38:54.077130+0000","last_clean":"2023-08-25T09:38:54.077130+0000","last_became_active":"2023-08-25T09:38:52.140881+0000","last_became_peered":"2023-08-25T09:38:52.140881+0000","last_unstale":"2023-08-25T09:38:54.077130+0000","last_undegraded":"2023-08-25T09:38:54.077130+0000","last_fullsized":"2023-08-25T09:38:54.077130+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:19.948794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.078140+0000","last_change":"2023-08-25T09:38:53.100041+0000","last_active":"2023-08-25T09:38:54.078140+0000","last_peered":"2023-08-25T09:38:54.078140+0000","last_clean":"2023-08-25T09:38:54.078140+0000","last_became_active":"2023-08-25T09:38:53.099764+0000","last_became_peered":"2023-08-25T09:38:53.099764+0000","last_unstale":"2023-08-25T09:38:54.078140+0000","last_undegraded":"2023-08-25T09:38:54.078140+0000","last_fullsized":"2023-08-25T09:38:54.078140+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:13:51.695417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170584+0000","last_change":"2023-08-25T09:38:52.150397+0000","last_active":"2023-08-25T09:38:54.170584+0000","last_peered":"2023-08-25T09:38:54.170584+0000","last_clean":"2023-08-25T09:38:54.170584+0000","last_became_active":"2023-08-25T09:38:52.150134+0000","last_became_peered":"2023-08-25T09:38:52.150134+0000","last_unstale":"2023-08-25T09:38:54.170584+0000","last_undegraded":"2023-08-25T09:38:54.170584+0000","last_fullsized":"2023-08-25T09:38:54.170584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:13.959805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.185984+0000","last_change":"2023-08-25T09:38:53.099998+0000","last_active":"2023-08-25T09:38:54.185984+0000","last_peered":"2023-08-25T09:38:54.185984+0000","last_clean":"2023-08-25T09:38:54.185984+0000","last_became_active":"2023-08-25T09:38:53.099612+0000","last_became_peered":"2023-08-25T09:38:53.099612+0000","last_unstale":"2023-08-25T09:38:54.185984+0000","last_undegraded":"2023-08-25T09:38:54.185984+0000","last_fullsized":"2023-08-25T09:38:54.185984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:52.113645+0000","last_clean_scrub_stamp":"2023-08-25T09:38:52.113645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:06:59.359186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T09:38:54.170948+0000","last_change":"2023-08-25T09:38:52.147242+0000","last_active":"2023-08-25T09:38:54.170948+0000","last_peered":"2023-08-25T09:38:54.170948+0000","last_clean":"2023-08-25T09:38:54.170948+0000","last_became_active":"2023-08-25T09:38:52.146238+0000","last_became_peered":"2023-08-25T09:38:52.146238+0000","last_unstale":"2023-08-25T09:38:54.170948+0000","last_undegraded":"2023-08-25T09:38:54.170948+0000","last_fullsized":"2023-08-25T09:38:54.170948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:51.109325+0000","last_clean_scrub_stamp":"2023-08-25T09:38:51.109325+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:43.730935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910561,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3089999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3009999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3360000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5449999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":664,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194648,"statfs":{"total":200000000,"available":199319611,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.788}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8540000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910561,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":89,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195223,"statfs":{"total":200000000,"available":199908845,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0329999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96499999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":88,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195223,"statfs":{"total":200000000,"available":199908949,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.383}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.474}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":664,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194647,"statfs":{"total":200000000,"available":199319519,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.395}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1279999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2370000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":87,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195225,"statfs":{"total":200000000,"available":199910415,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]}]}],"pool_statfs":[]}} 2023-08-25T09:39:03.645 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-25T09:39:03.646 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-25T09:39:03.646 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-08-25T09:39:03.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-08-25T09:39:03.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.897+0000 7f8403f47640 1 Processor -- start 2023-08-25T09:39:03.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.897+0000 7f8403f47640 1 -- start start 2023-08-25T09:39:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8403f47640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 0x7f83fc0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8403f47640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0b12b0 con 0x7f83fc15acd0 2023-08-25T09:39:03.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8403f47640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc06ae20 con 0x7f83fc0ca780 2023-08-25T09:39:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8403f47640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0b47e0 con 0x7f83fc0b4400 2023-08-25T09:39:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8401ce3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f83fc15acd0 0x7f83fc15b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35518/0 (socket says 172.21.15.70:35518) 2023-08-25T09:39:03.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8401ce3640 1 -- 172.21.15.70:0/1734604352 learned_addr learned my addr 172.21.15.70:0/1734604352 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32046520 0 0) 0x7f83fc0b12b0 con 0x7f83fc15acd0 2023-08-25T09:39:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e8003520 con 0x7f83fc15acd0 2023-08-25T09:39:03.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.899+0000 7f84024e4640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 0x7f83fc0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4272078428 0 0) 0x7f83e8003520 con 0x7f83fc15acd0 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 msgr2=0x7f83fc0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 0x7f83fc0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 >> v1:172.21.15.155:6789/0 conn(0x7f83fc0ca780 legacy=0x7f83fc0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc15b7c0 con 0x7f83fc15acd0 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f83f80017f0 con 0x7f83fc15acd0 2023-08-25T09:39:03.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 >> v1:172.21.15.70:6789/0 conn(0x7f83fc15acd0 legacy=0x7f83fc15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e80039a0 0x7f83e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 0x7f83e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e8004d00 0x7f83e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0b47e0 con 0x7f83e8004d00 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800bae0 con 0x7f83e80039a0 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800bca0 con 0x7f83fc0b4400 2023-08-25T09:39:03.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f84024e4640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e8004d00 0x7f83e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.900+0000 7f8401ce3640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e80039a0 0x7f83e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.901+0000 7f84024e4640 1 -- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 msgr2=0x7f83e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.901+0000 7f84024e4640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83fc0b4400 0x7f83e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.901+0000 7f84024e4640 1 -- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e80039a0 msgr2=0x7f83e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.901+0000 7f84024e4640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e80039a0 0x7f83e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.901+0000 7f84024e4640 1 -- 172.21.15.70:0/1734604352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8003520 con 0x7f83e8004d00 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f84024e4640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e8004d00 0x7f83e8005180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009fc0 tx=0x7f83e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7144ea93022f704 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f402cc60 con 0x7f83e8004d00 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f400f040 con 0x7f83e8004d00 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f8400ce1640 1 -- 172.21.15.70:0/1734604352 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4034440 con 0x7f83e8004d00 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f8403f47640 1 -- 172.21.15.70:0/1734604352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e8004d00 msgr2=0x7f83e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f8403f47640 1 --2- 172.21.15.70:0/1734604352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e8004d00 0x7f83e8005180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009fc0 tx=0x7f83e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.902+0000 7f84024e4640 1 -- 172.21.15.70:0/1734604352 reap_dead start 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.903+0000 7f8403f47640 1 -- 172.21.15.70:0/1734604352 shutdown_connections 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.903+0000 7f8403f47640 1 -- 172.21.15.70:0/1734604352 >> 172.21.15.70:0/1734604352 conn(0x7f83fc0ce760 msgr2=0x7f83fc0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.903+0000 7f8403f47640 1 -- 172.21.15.70:0/1734604352 shutdown_connections 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.903+0000 7f8403f47640 1 -- 172.21.15.70:0/1734604352 wait complete. 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.904+0000 7f8403f47640 1 Processor -- start 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.904+0000 7f8403f47640 1 -- start start 2023-08-25T09:39:03.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.905+0000 7f8403f47640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e8004d00 0x7f83fc143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.905+0000 7f8403f47640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc1561d0 0x7f83fc1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.905+0000 7f8403f47640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 0x7f83fc14dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.905+0000 7f8403f47640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc153030 con 0x7f83fc151f30 2023-08-25T09:39:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.905+0000 7f8403f47640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc143760 con 0x7f83fc1561d0 2023-08-25T09:39:03.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.905+0000 7f8403f47640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc1523b0 con 0x7f83e8004d00 2023-08-25T09:39:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 0x7f83fc14dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 0x7f83fc14dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47482/0 (socket says 172.21.15.70:47482) 2023-08-25T09:39:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 -- 172.21.15.70:0/1589642344 learned_addr learned my addr 172.21.15.70:0/1589642344 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:03.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f8401ce3640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e8004d00 0x7f83fc143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 -- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e8004d00 msgr2=0x7f83fc143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84014e2640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc1561d0 0x7f83fc1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e8004d00 0x7f83fc143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 -- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc1561d0 msgr2=0x7f83fc1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc1561d0 0x7f83fc1519f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 -- 172.21.15.70:0/1589642344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc152690 con 0x7f83fc151f30 2023-08-25T09:39:03.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.906+0000 7f84024e4640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 0x7f83fc14dca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83f4037040 tx=0x7f83f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.907+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4043070 con 0x7f83fc151f30 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.907+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4034b50 con 0x7f83fc151f30 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.907+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4003990 con 0x7f83fc151f30 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.907+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc152970 con 0x7f83fc151f30 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.907+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc14e1e0 con 0x7f83fc151f30 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.908+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc04e420 con 0x7f83fc151f30 2023-08-25T09:39:03.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.909+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f83f403e440 con 0x7f83fc151f30 2023-08-25T09:39:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.910+0000 7f83f27fc640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83e006a050 0x7f83e006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.910+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83f40aa0a0 con 0x7f83fc151f30 2023-08-25T09:39:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.910+0000 7f8401ce3640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83e006a050 0x7f83e006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:03.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.911+0000 7f8401ce3640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83e006a050 0x7f83e006c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83f80011d0 tx=0x7f83f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:03.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:03.915+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f83f40757a0 con 0x7f83fc151f30 2023-08-25T09:39:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.128+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83fc06b620 con 0x7f83fc151f30 2023-08-25T09:39:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.129+0000 7f83f27fc640 1 -- 172.21.15.70:0/1589642344 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f83f4039080 con 0x7f83fc151f30 2023-08-25T09:39:04.131 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:04.131 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-25T09:39:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83e006a050 msgr2=0x7f83e006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83e006a050 0x7f83e006c510 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f83f80011d0 tx=0x7f83f8009040 comp rx=0 tx=0).stop 2023-08-25T09:39:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 msgr2=0x7f83fc14dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 0x7f83fc14dca0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83f4037040 tx=0x7f83f4009750 comp rx=0 tx=0).stop 2023-08-25T09:39:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 shutdown_connections 2023-08-25T09:39:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83e006a050 0x7f83e006c510 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83fc151f30 0x7f83fc14dca0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83fc1561d0 0x7f83fc1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 --2- 172.21.15.70:0/1589642344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e8004d00 0x7f83fc143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.131+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 >> 172.21.15.70:0/1589642344 conn(0x7f83fc0ce760 msgr2=0x7f83fc0cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.132+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 shutdown_connections 2023-08-25T09:39:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.132+0000 7f8403f47640 1 -- 172.21.15.70:0/1589642344 wait complete. 2023-08-25T09:39:04.152 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-08-25T09:39:04.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:04.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.353+0000 7fa61b401640 1 Processor -- start 2023-08-25T09:39:04.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.353+0000 7fa61b401640 1 -- start start 2023-08-25T09:39:04.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.354+0000 7fa61b401640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6140b12f0 0x7fa61415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.354+0000 7fa61b401640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa61415f190 0x7fa6140cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.354+0000 7fa61b401640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61415e7c0 con 0x7fa61415a7c0 2023-08-25T09:39:04.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.354+0000 7fa61b401640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa614071b50 con 0x7fa6140b12f0 2023-08-25T09:39:04.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.354+0000 7fa61b401640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61415e490 con 0x7fa61415f190 2023-08-25T09:39:04.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa61a3ff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa61415a7c0 0x7fa6140b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35526/0 (socket says 172.21.15.70:35526) 2023-08-25T09:39:04.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa61a3ff640 1 -- 172.21.15.70:0/3168656515 learned_addr learned my addr 172.21.15.70:0/3168656515 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:04.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa619bfe640 1 --2- 172.21.15.70:0/3168656515 >> v2:172.21.15.70:3300/0 conn(0x7fa6140b12f0 0x7fa61415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa61ac00640 1 --2- 172.21.15.70:0/3168656515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa61415f190 0x7fa6140cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa619bfe640 1 -- 172.21.15.70:0/3168656515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa61415f190 msgr2=0x7fa6140cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:04.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa619bfe640 1 --2- 172.21.15.70:0/3168656515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa61415f190 0x7fa6140cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa619bfe640 1 -- 172.21.15.70:0/3168656515 >> v1:172.21.15.70:6789/0 conn(0x7fa61415a7c0 legacy=0x7fa6140b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa619bfe640 1 -- 172.21.15.70:0/3168656515 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6140b3d60 con 0x7fa6140b12f0 2023-08-25T09:39:04.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.355+0000 7fa6193fd640 1 -- 172.21.15.70:0/3168656515 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460652453 0 0) 0x7fa61415e7c0 con 0x7fa61415a7c0 2023-08-25T09:39:04.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.356+0000 7fa619bfe640 1 --2- 172.21.15.70:0/3168656515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6140b12f0 0x7fa61415ec50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097a0 tx=0x7fa60402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f366dc447e76851b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:04.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.356+0000 7fa6193fd640 1 -- 172.21.15.70:0/3168656515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604031830 con 0x7fa6140b12f0 2023-08-25T09:39:04.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.356+0000 7fa6193fd640 1 -- 172.21.15.70:0/3168656515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604031990 con 0x7fa6140b12f0 2023-08-25T09:39:04.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.356+0000 7fa6193fd640 1 -- 172.21.15.70:0/3168656515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa604031d20 con 0x7fa6140b12f0 2023-08-25T09:39:04.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 -- 172.21.15.70:0/3168656515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6140b12f0 msgr2=0x7fa61415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 --2- 172.21.15.70:0/3168656515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6140b12f0 0x7fa61415ec50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa6040097a0 tx=0x7fa60402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 -- 172.21.15.70:0/3168656515 shutdown_connections 2023-08-25T09:39:04.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 --2- 172.21.15.70:0/3168656515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6140b12f0 0x7fa61415ec50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 --2- 172.21.15.70:0/3168656515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa61415f190 0x7fa6140cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 -- 172.21.15.70:0/3168656515 >> 172.21.15.70:0/3168656515 conn(0x7fa6140affd0 msgr2=0x7fa6140b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:04.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 -- 172.21.15.70:0/3168656515 shutdown_connections 2023-08-25T09:39:04.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.357+0000 7fa61b401640 1 -- 172.21.15.70:0/3168656515 wait complete. 2023-08-25T09:39:04.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.359+0000 7fa61b401640 1 Processor -- start 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.359+0000 7fa61b401640 1 -- start start 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61b401640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6140b12f0 0x7fa6140c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61b401640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa61415a7c0 0x7fa6140acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61b401640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 0x7fa6141f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61b401640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6140c2150 con 0x7fa61415f190 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61b401640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6141f2f70 con 0x7fa61415a7c0 2023-08-25T09:39:04.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61b401640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6141f3250 con 0x7fa6140b12f0 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61ac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 0x7fa6141f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61ac00640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 0x7fa6141f24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47494/0 (socket says 172.21.15.70:47494) 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61ac00640 1 -- 172.21.15.70:0/3501058901 learned_addr learned my addr 172.21.15.70:0/3501058901 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.360+0000 7fa61a3ff640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6140b12f0 0x7fa6140c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61ac00640 1 -- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6140b12f0 msgr2=0x7fa6140c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61ac00640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6140b12f0 0x7fa6140c2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa619bfe640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa61415a7c0 0x7fa6140acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61ac00640 1 -- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa61415a7c0 msgr2=0x7fa6140acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61ac00640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa61415a7c0 0x7fa6140acaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61ac00640 1 -- 172.21.15.70:0/3501058901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604009400 con 0x7fa61415f190 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61ac00640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 0x7fa6141f24b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00cc30 tx=0x7fa60c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00ce40 con 0x7fa61415f190 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c003990 con 0x7fa61415f190 2023-08-25T09:39:04.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c003d20 con 0x7fa61415f190 2023-08-25T09:39:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6140c2410 con 0x7fa61415f190 2023-08-25T09:39:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.361+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6140c2e70 con 0x7fa61415f190 2023-08-25T09:39:04.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.363+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6141f3470 con 0x7fa61415f190 2023-08-25T09:39:04.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.363+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa60c003e80 con 0x7fa61415f190 2023-08-25T09:39:04.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.364+0000 7fa60affd640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5f806a050 0x7fa5f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.364+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa60c088a20 con 0x7fa61415f190 2023-08-25T09:39:04.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.365+0000 7fa61a3ff640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5f806a050 0x7fa5f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.365+0000 7fa61a3ff640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5f806a050 0x7fa5f806c510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa610007fd0 tx=0x7fa610009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:04.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.368+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa60c054070 con 0x7fa61415f190 2023-08-25T09:39:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.558+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6140c2600 con 0x7fa61415f190 2023-08-25T09:39:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.560+0000 7fa60affd640 1 -- 172.21.15.70:0/3501058901 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fa60c01d020 con 0x7fa61415f190 2023-08-25T09:39:04.563 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:04.563 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5f806a050 msgr2=0x7fa5f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5f806a050 0x7fa5f806c510 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa610007fd0 tx=0x7fa610009040 comp rx=0 tx=0).stop 2023-08-25T09:39:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 msgr2=0x7fa6141f24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 0x7fa6141f24b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00cc30 tx=0x7fa60c00a5c0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 shutdown_connections 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5f806a050 0x7fa5f806c510 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa61415f190 0x7fa6141f24b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa61415a7c0 0x7fa6140acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 --2- 172.21.15.70:0/3501058901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6140b12f0 0x7fa6140c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.563+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 >> 172.21.15.70:0/3501058901 conn(0x7fa6140affd0 msgr2=0x7fa614158940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.564+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 shutdown_connections 2023-08-25T09:39:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.564+0000 7fa61b401640 1 -- 172.21.15.70:0/3501058901 wait complete. 2023-08-25T09:39:04.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:04.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:04.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.796+0000 7f25594e1640 1 Processor -- start 2023-08-25T09:39:04.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.796+0000 7f25594e1640 1 -- start start 2023-08-25T09:39:04.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25594e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2554160290 0x7f25540cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25594e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f25540d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25594e1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2554071c10 con 0x7f2554160670 2023-08-25T09:39:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25594e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2554071000 con 0x7f2554160290 2023-08-25T09:39:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25594e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25540712e0 con 0x7f2554069850 2023-08-25T09:39:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f2552ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2554160290 0x7f25540cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25527fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2554160670 0x7f25540cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35534/0 (socket says 172.21.15.70:35534) 2023-08-25T09:39:04.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25527fc640 1 -- 172.21.15.70:0/708680274 learned_addr learned my addr 172.21.15.70:0/708680274 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:04.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f2552ffd640 1 --2- 172.21.15.70:0/708680274 >> v2:172.21.15.70:3300/0 conn(0x7f2554160290 0x7f25540cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47510/0 (socket says 172.21.15.70:47510) 2023-08-25T09:39:04.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f25537fe640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f25540d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f2552ffd640 1 -- 172.21.15.70:0/708680274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 msgr2=0x7f25540d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:04.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f2552ffd640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f25540d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.798+0000 7f2552ffd640 1 -- 172.21.15.70:0/708680274 >> v1:172.21.15.70:6789/0 conn(0x7f2554160670 legacy=0x7f25540cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.799+0000 7f2552ffd640 1 -- 172.21.15.70:0/708680274 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25540715c0 con 0x7f2554160290 2023-08-25T09:39:04.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.799+0000 7f25537fe640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f25540d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:04.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.799+0000 7f2551ffb640 1 -- 172.21.15.70:0/708680274 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425621756 0 0) 0x7f2554071c10 con 0x7f2554160670 2023-08-25T09:39:04.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f2552ffd640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160290 0x7f25540cd360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f25440097a0 tx=0x7f254402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a478c61ad84b890e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:04.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f2551ffb640 1 -- 172.21.15.70:0/708680274 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254402d040 con 0x7f2554160290 2023-08-25T09:39:04.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f2551ffb640 1 -- 172.21.15.70:0/708680274 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2544031810 con 0x7f2554160290 2023-08-25T09:39:04.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f2551ffb640 1 -- 172.21.15.70:0/708680274 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544031ba0 con 0x7f2554160290 2023-08-25T09:39:04.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f25594e1640 1 -- 172.21.15.70:0/708680274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160290 msgr2=0x7f25540cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f25594e1640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160290 0x7f25540cd360 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f25440097a0 tx=0x7f254402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f25594e1640 1 -- 172.21.15.70:0/708680274 shutdown_connections 2023-08-25T09:39:04.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f25594e1640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f25540d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f25594e1640 1 --2- 172.21.15.70:0/708680274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160290 0x7f25540cd360 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.800+0000 7f25594e1640 1 -- 172.21.15.70:0/708680274 >> 172.21.15.70:0/708680274 conn(0x7f255406ba60 msgr2=0x7f255406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:04.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.801+0000 7f25594e1640 1 -- 172.21.15.70:0/708680274 shutdown_connections 2023-08-25T09:39:04.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.801+0000 7f25594e1640 1 -- 172.21.15.70:0/708680274 wait complete. 2023-08-25T09:39:04.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.802+0000 7f25594e1640 1 Processor -- start 2023-08-25T09:39:04.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.802+0000 7f25594e1640 1 -- start start 2023-08-25T09:39:04.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.803+0000 7f25594e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f2554154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.803+0000 7f25594e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554160290 0x7f25541f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.803+0000 7f25594e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 0x7f2554154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.803+0000 7f25594e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25540b0680 con 0x7f2554160670 2023-08-25T09:39:04.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.803+0000 7f25594e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25540ca700 con 0x7f2554160290 2023-08-25T09:39:04.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.803+0000 7f25594e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25541f68d0 con 0x7f2554069850 2023-08-25T09:39:04.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 0x7f2554154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 0x7f2554154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47514/0 (socket says 172.21.15.70:47514) 2023-08-25T09:39:04.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 -- 172.21.15.70:0/457373661 learned_addr learned my addr 172.21.15.70:0/457373661 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:04.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 -- 172.21.15.70:0/457373661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 msgr2=0x7f2554154860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f2554154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 -- 172.21.15.70:0/457373661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554160290 msgr2=0x7f25541f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554160290 0x7f25541f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 -- 172.21.15.70:0/457373661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544009400 con 0x7f2554160670 2023-08-25T09:39:04.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f25537fe640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 0x7f2554154060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f254c00c820 tx=0x7f254c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:04.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c007a30 con 0x7f2554160670 2023-08-25T09:39:04.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c007b90 con 0x7f2554160670 2023-08-25T09:39:04.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.804+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00ce10 con 0x7f2554160670 2023-08-25T09:39:04.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.805+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25540c7ae0 con 0x7f2554160670 2023-08-25T09:39:04.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.805+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25540ca390 con 0x7f2554160670 2023-08-25T09:39:04.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.806+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25540c8000 con 0x7f2554160670 2023-08-25T09:39:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.807+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f254c0113f0 con 0x7f2554160670 2023-08-25T09:39:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.808+0000 7f253b7fe640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f254006a0a0 0x7f254006c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.808+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f254c088700 con 0x7f2554160670 2023-08-25T09:39:04.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.808+0000 7f2552ffd640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f254006a0a0 0x7f254006c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.809+0000 7f2552ffd640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f254006a0a0 0x7f254006c560 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2544036000 tx=0x7f2544030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:04.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.812+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f25540c8000 con 0x7f2554160670 2023-08-25T09:39:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.985+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25540ca1f0 con 0x7f2554160670 2023-08-25T09:39:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.986+0000 7f253b7fe640 1 -- 172.21.15.70:0/457373661 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25540ca1f0 con 0x7f2554160670 2023-08-25T09:39:04.988 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.988+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f254006a0a0 msgr2=0x7f254006c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.988+0000 7f25594e1640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f254006a0a0 0x7f254006c560 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2544036000 tx=0x7f2544030000 comp rx=0 tx=0).stop 2023-08-25T09:39:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.988+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 msgr2=0x7f2554154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.988+0000 7f25594e1640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 0x7f2554154060 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f254c00c820 tx=0x7f254c00cce0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 shutdown_connections 2023-08-25T09:39:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f254006a0a0 0x7f254006c560 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2554160670 0x7f2554154060 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2554160290 0x7f25541f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 --2- 172.21.15.70:0/457373661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2554069850 0x7f2554154860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 >> 172.21.15.70:0/457373661 conn(0x7f255406ba60 msgr2=0x7f25540ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 shutdown_connections 2023-08-25T09:39:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:04.989+0000 7f25594e1640 1 -- 172.21.15.70:0/457373661 wait complete. 2023-08-25T09:39:05.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.209+0000 7f816196b640 1 Processor -- start 2023-08-25T09:39:05.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.209+0000 7f816196b640 1 -- start start 2023-08-25T09:39:05.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.210+0000 7f816196b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f815c0c2110 0x7f815c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.210+0000 7f816196b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c0c2550 con 0x7f815c15ae50 2023-08-25T09:39:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f816196b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f815c0ce330 con 0x7f815c069d90 2023-08-25T09:39:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f816196b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815c06a200 con 0x7f815c0c2110 2023-08-25T09:39:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f815a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f815c0c2110 0x7f815c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f815a7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f815c0c2110 0x7f815c069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47518/0 (socket says 172.21.15.70:47518) 2023-08-25T09:39:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f815a7fc640 1 -- 172.21.15.70:0/68102937 learned_addr learned my addr 172.21.15.70:0/68102937 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f815a7fc640 1 -- 172.21.15.70:0/68102937 >> v1:172.21.15.155:6789/0 conn(0x7f815c069d90 legacy=0x7f815c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f815a7fc640 1 -- 172.21.15.70:0/68102937 >> v1:172.21.15.70:6789/0 conn(0x7f815c15ae50 legacy=0x7f815c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.211+0000 7f815a7fc640 1 -- 172.21.15.70:0/68102937 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0cc5f0 con 0x7f815c0c2110 2023-08-25T09:39:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.212+0000 7f815a7fc640 1 --2- 172.21.15.70:0/68102937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c0c2110 0x7f815c069850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8150009870 tx=0x7f815002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6f43f67df48c63 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.212+0000 7f8159ffb640 1 -- 172.21.15.70:0/68102937 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815002f850 con 0x7f815c0c2110 2023-08-25T09:39:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.212+0000 7f8159ffb640 1 -- 172.21.15.70:0/68102937 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815002f9b0 con 0x7f815c0c2110 2023-08-25T09:39:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f8159ffb640 1 -- 172.21.15.70:0/68102937 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815002fd40 con 0x7f815c0c2110 2023-08-25T09:39:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 -- 172.21.15.70:0/68102937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c0c2110 msgr2=0x7f815c069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 --2- 172.21.15.70:0/68102937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c0c2110 0x7f815c069850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8150009870 tx=0x7f815002ba30 comp rx=0 tx=0).stop 2023-08-25T09:39:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 -- 172.21.15.70:0/68102937 shutdown_connections 2023-08-25T09:39:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 --2- 172.21.15.70:0/68102937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c0c2110 0x7f815c069850 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 -- 172.21.15.70:0/68102937 >> 172.21.15.70:0/68102937 conn(0x7f815c0cb0d0 msgr2=0x7f815c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 -- 172.21.15.70:0/68102937 shutdown_connections 2023-08-25T09:39:05.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.213+0000 7f816196b640 1 -- 172.21.15.70:0/68102937 wait complete. 2023-08-25T09:39:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.215+0000 7f816196b640 1 Processor -- start 2023-08-25T09:39:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.215+0000 7f816196b640 1 -- start start 2023-08-25T09:39:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f816196b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 0x7f815c1f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f816196b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f815c0c2110 0x7f815c0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f816196b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f815c15ae50 0x7f815c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f816196b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c6170 con 0x7f815c069d90 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f816196b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c6450 con 0x7f815c15ae50 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f816196b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c0c6730 con 0x7f815c0c2110 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f815affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 0x7f815c1f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f815affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 0x7f815c1f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47520/0 (socket says 172.21.15.70:47520) 2023-08-25T09:39:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.216+0000 7f815affd640 1 -- 172.21.15.70:0/307310077 learned_addr learned my addr 172.21.15.70:0/307310077 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f815affd640 1 -- 172.21.15.70:0/307310077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f815c0c2110 msgr2=0x7f815c0c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f815affd640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f815c0c2110 0x7f815c0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f815affd640 1 -- 172.21.15.70:0/307310077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f815c15ae50 msgr2=0x7f815c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f815affd640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f815c15ae50 0x7f815c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f815affd640 1 -- 172.21.15.70:0/307310077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500094d0 con 0x7f815c069d90 2023-08-25T09:39:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f815affd640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 0x7f815c1f1ee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f814c00c400 tx=0x7f814c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:05.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0052d0 con 0x7f815c069d90 2023-08-25T09:39:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c005430 con 0x7f815c069d90 2023-08-25T09:39:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c003830 con 0x7f815c069d90 2023-08-25T09:39:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.217+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c0c6a10 con 0x7f815c069d90 2023-08-25T09:39:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.218+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c1f2420 con 0x7f815c069d90 2023-08-25T09:39:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.219+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c1f5e00 con 0x7f815c069d90 2023-08-25T09:39:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.219+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f814c003990 con 0x7f815c069d90 2023-08-25T09:39:05.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.220+0000 7f8143fff640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f814806a050 0x7f814806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.220+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f814c087b30 con 0x7f815c069d90 2023-08-25T09:39:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.220+0000 7f815a7fc640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f814806a050 0x7f814806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.221+0000 7f815a7fc640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f814806a050 0x7f814806c510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8150036000 tx=0x7f8150039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.224+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f815c1f5e00 con 0x7f815c069d90 2023-08-25T09:39:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.420+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f815c1f5e00 con 0x7f815c069d90 2023-08-25T09:39:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.421+0000 7f8143fff640 1 -- 172.21.15.70:0/307310077 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f814c017020 con 0x7f815c069d90 2023-08-25T09:39:05.423 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f814806a050 msgr2=0x7f814806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f814806a050 0x7f814806c510 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8150036000 tx=0x7f8150039040 comp rx=0 tx=0).stop 2023-08-25T09:39:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 msgr2=0x7f815c1f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 0x7f815c1f1ee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f814c00c400 tx=0x7f814c00c8c0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 shutdown_connections 2023-08-25T09:39:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f814806a050 0x7f814806c510 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f815c15ae50 0x7f815c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f815c0c2110 0x7f815c0c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 --2- 172.21.15.70:0/307310077 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f815c069d90 0x7f815c1f1ee0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 >> 172.21.15.70:0/307310077 conn(0x7f815c0cb0d0 msgr2=0x7f815c0c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 shutdown_connections 2023-08-25T09:39:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.424+0000 7f816196b640 1 -- 172.21.15.70:0/307310077 wait complete. 2023-08-25T09:39:05.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:05.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.641+0000 7f4949f0c640 1 Processor -- start 2023-08-25T09:39:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.641+0000 7f4949f0c640 1 -- start start 2023-08-25T09:39:05.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.642+0000 7f4949f0c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4944159bf0 0x7f494415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.642+0000 7f4949f0c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494415e780 con 0x7f49440cd660 2023-08-25T09:39:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.642+0000 7f4949f0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494406f8e0 con 0x7f494406f0e0 2023-08-25T09:39:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.642+0000 7f4949f0c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f494406fbc0 con 0x7f4944159bf0 2023-08-25T09:39:05.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f49437fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4944159bf0 0x7f494415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f4943fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f49440cd660 0x7f49440c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35560/0 (socket says 172.21.15.70:35560) 2023-08-25T09:39:05.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f49437fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4944159bf0 0x7f494415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47530/0 (socket says 172.21.15.70:47530) 2023-08-25T09:39:05.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f4943fff640 1 -- 172.21.15.70:0/1722139581 learned_addr learned my addr 172.21.15.70:0/1722139581 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f49437fe640 1 -- 172.21.15.70:0/1722139581 >> v1:172.21.15.155:6789/0 conn(0x7f494406f0e0 legacy=0x7f4944070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:05.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f49437fe640 1 -- 172.21.15.70:0/1722139581 >> v1:172.21.15.70:6789/0 conn(0x7f49440cd660 legacy=0x7f49440c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f49437fe640 1 -- 172.21.15.70:0/1722139581 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494406fea0 con 0x7f4944159bf0 2023-08-25T09:39:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.643+0000 7f49427fc640 1 -- 172.21.15.70:0/1722139581 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613725503 0 0) 0x7f494415e780 con 0x7f49440cd660 2023-08-25T09:39:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.644+0000 7f49437fe640 1 --2- 172.21.15.70:0/1722139581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f494415e240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49340097a0 tx=0x7f493402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b9233afc8c05f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:05.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f49427fc640 1 -- 172.21.15.70:0/1722139581 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934031830 con 0x7f4944159bf0 2023-08-25T09:39:05.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f49427fc640 1 -- 172.21.15.70:0/1722139581 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934031990 con 0x7f4944159bf0 2023-08-25T09:39:05.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f49427fc640 1 -- 172.21.15.70:0/1722139581 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934031d20 con 0x7f4944159bf0 2023-08-25T09:39:05.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 -- 172.21.15.70:0/1722139581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 msgr2=0x7f494415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 --2- 172.21.15.70:0/1722139581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f494415e240 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f49340097a0 tx=0x7f493402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 -- 172.21.15.70:0/1722139581 shutdown_connections 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 --2- 172.21.15.70:0/1722139581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f494415e240 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 -- 172.21.15.70:0/1722139581 >> 172.21.15.70:0/1722139581 conn(0x7f49440cb0d0 msgr2=0x7f49440cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 -- 172.21.15.70:0/1722139581 shutdown_connections 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.645+0000 7f4949f0c640 1 -- 172.21.15.70:0/1722139581 wait complete. 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.647+0000 7f4949f0c640 1 Processor -- start 2023-08-25T09:39:05.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.647+0000 7f4949f0c640 1 -- start start 2023-08-25T09:39:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4949f0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f494406f0e0 0x7f49440c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4949f0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49440cd660 0x7f49440c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4949f0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f49440c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4949f0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49441495e0 con 0x7f4944159bf0 2023-08-25T09:39:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4949f0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49440c8370 con 0x7f494406f0e0 2023-08-25T09:39:05.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4949f0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440ae100 con 0x7f49440cd660 2023-08-25T09:39:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4943fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f49440c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4943fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f49440c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47540/0 (socket says 172.21.15.70:47540) 2023-08-25T09:39:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.648+0000 7f4943fff640 1 -- 172.21.15.70:0/738056718 learned_addr learned my addr 172.21.15.70:0/738056718 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:05.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4943fff640 1 -- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49440cd660 msgr2=0x7f49440c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4943fff640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49440cd660 0x7f49440c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4943fff640 1 -- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f494406f0e0 msgr2=0x7f49440c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f49437fe640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f494406f0e0 0x7f49440c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4943fff640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f494406f0e0 0x7f49440c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4943fff640 1 -- 172.21.15.70:0/738056718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4934009400 con 0x7f4944159bf0 2023-08-25T09:39:05.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4943fff640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f49440c4090 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c7e0 tx=0x7f493c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:05.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00a640 con 0x7f4944159bf0 2023-08-25T09:39:05.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c00a7c0 con 0x7f4944159bf0 2023-08-25T09:39:05.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00ab70 con 0x7f4944159bf0 2023-08-25T09:39:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.649+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49440ae3e0 con 0x7f4944159bf0 2023-08-25T09:39:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.650+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49440ae9a0 con 0x7f4944159bf0 2023-08-25T09:39:05.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.651+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944070160 con 0x7f4944159bf0 2023-08-25T09:39:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.651+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f493c00acd0 con 0x7f4944159bf0 2023-08-25T09:39:05.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.652+0000 7f492bfff640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.652+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f493c087c30 con 0x7f4944159bf0 2023-08-25T09:39:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.653+0000 7f49437fe640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.653+0000 7f49437fe640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4934036000 tx=0x7f4934008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:05.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.657+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4944070160 con 0x7f4944159bf0 2023-08-25T09:39:05.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.844+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f494406f540 con 0x7f4944159bf0 2023-08-25T09:39:05.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.845+0000 7f492bfff640 1 -- 172.21.15.70:0/738056718 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f494406f540 con 0x7f4944159bf0 2023-08-25T09:39:05.847 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:05.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.847+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 msgr2=0x7f493006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.847+0000 7f4949f0c640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4934036000 tx=0x7f4934008030 comp rx=0 tx=0).stop 2023-08-25T09:39:05.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.847+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 msgr2=0x7f49440c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.847+0000 7f4949f0c640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f49440c4090 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c7e0 tx=0x7f493c00cca0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 shutdown_connections 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4944159bf0 0x7f49440c4090 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f49440cd660 0x7f49440c3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 --2- 172.21.15.70:0/738056718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f494406f0e0 0x7f49440c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:05.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 >> 172.21.15.70:0/738056718 conn(0x7f49440cb0d0 msgr2=0x7f4944153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:05.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 shutdown_connections 2023-08-25T09:39:05.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:05.848+0000 7f4949f0c640 1 -- 172.21.15.70:0/738056718 wait complete. 2023-08-25T09:39:05.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.083+0000 7f6aab342640 1 Processor -- start 2023-08-25T09:39:06.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.083+0000 7f6aab342640 1 -- start start 2023-08-25T09:39:06.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aab342640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6aa40b0320 0x7f6aa40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aab342640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa40adbb0 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aab342640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa40c90a0 con 0x7f6aa415a920 2023-08-25T09:39:06.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aab342640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa40c9380 con 0x7f6aa40b0320 2023-08-25T09:39:06.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aaa340640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6aa40b0320 0x7f6aa40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aaa340640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6aa40b0320 0x7f6aa40c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47554/0 (socket says 172.21.15.70:47554) 2023-08-25T09:39:06.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aaa340640 1 -- 172.21.15.70:0/2240372348 learned_addr learned my addr 172.21.15.70:0/2240372348 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:06.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aaa340640 1 -- 172.21.15.70:0/2240372348 >> v1:172.21.15.155:6789/0 conn(0x7f6aa415a920 legacy=0x7f6aa40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aaa340640 1 -- 172.21.15.70:0/2240372348 >> v1:172.21.15.70:6789/0 conn(0x7f6aa40b1ce0 legacy=0x7f6aa40ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.085+0000 7f6aaa340640 1 -- 172.21.15.70:0/2240372348 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa40c9660 con 0x7f6aa40b0320 2023-08-25T09:39:06.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.087+0000 7f6aaa340640 1 --2- 172.21.15.70:0/2240372348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b0320 0x7f6aa40c9e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009850 tx=0x7f6aa002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7ca0be36496a7f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.087+0000 7f6aa933e640 1 -- 172.21.15.70:0/2240372348 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0004230 con 0x7f6aa40b0320 2023-08-25T09:39:06.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.087+0000 7f6aa933e640 1 -- 172.21.15.70:0/2240372348 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa0004390 con 0x7f6aa40b0320 2023-08-25T09:39:06.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.087+0000 7f6aa933e640 1 -- 172.21.15.70:0/2240372348 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0030840 con 0x7f6aa40b0320 2023-08-25T09:39:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.087+0000 7f6aab342640 1 -- 172.21.15.70:0/2240372348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b0320 msgr2=0x7f6aa40c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.087+0000 7f6aab342640 1 --2- 172.21.15.70:0/2240372348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b0320 0x7f6aa40c9e60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009850 tx=0x7f6aa002b980 comp rx=0 tx=0).stop 2023-08-25T09:39:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.088+0000 7f6aab342640 1 -- 172.21.15.70:0/2240372348 shutdown_connections 2023-08-25T09:39:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.088+0000 7f6aab342640 1 --2- 172.21.15.70:0/2240372348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b0320 0x7f6aa40c9e60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.088+0000 7f6aab342640 1 -- 172.21.15.70:0/2240372348 >> 172.21.15.70:0/2240372348 conn(0x7f6aa40ce830 msgr2=0x7f6aa40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:06.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.088+0000 7f6aab342640 1 -- 172.21.15.70:0/2240372348 shutdown_connections 2023-08-25T09:39:06.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.088+0000 7f6aab342640 1 -- 172.21.15.70:0/2240372348 wait complete. 2023-08-25T09:39:06.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.089+0000 7f6aab342640 1 Processor -- start 2023-08-25T09:39:06.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.089+0000 7f6aab342640 1 -- start start 2023-08-25T09:39:06.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.090+0000 7f6aab342640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa40b0320 0x7f6aa41f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.090+0000 7f6aab342640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 0x7f6aa40c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.090+0000 7f6aab342640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa415a920 0x7f6aa40c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.090+0000 7f6aab342640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40698e0 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.090+0000 7f6aab342640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa4157b70 con 0x7f6aa40b0320 2023-08-25T09:39:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.090+0000 7f6aab342640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa40c78a0 con 0x7f6aa415a920 2023-08-25T09:39:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 0x7f6aa40c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 0x7f6aa40c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47568/0 (socket says 172.21.15.70:47568) 2023-08-25T09:39:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 -- 172.21.15.70:0/2869408516 learned_addr learned my addr 172.21.15.70:0/2869408516 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:06.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 -- 172.21.15.70:0/2869408516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa415a920 msgr2=0x7f6aa40c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa415a920 0x7f6aa40c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 -- 172.21.15.70:0/2869408516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa40b0320 msgr2=0x7f6aa41f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa40b0320 0x7f6aa41f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 -- 172.21.15.70:0/2869408516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0009460 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6aa9b3f640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 0x7f6aa40c64a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400b3c0 tx=0x7f6a9400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.091+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011830 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.092+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a94011990 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.092+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94011d60 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.092+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa41f2d60 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.092+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa41f3320 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.093+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa4061460 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.094+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6a9401b460 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.095+0000 7f6a9affd640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a8806a050 0x7f6a8806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.095+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a94088320 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.095+0000 7f6aaa340640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a8806a050 0x7f6a8806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.095+0000 7f6aaa340640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a8806a050 0x7f6a8806c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0035000 tx=0x7f6aa0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.099+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6a94053a10 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.287+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6aa4069a20 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.290+0000 7f6a9affd640 1 -- 172.21.15.70:0/2869408516 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6a94057030 con 0x7f6aa40b1ce0 2023-08-25T09:39:06.292 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:06.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a8806a050 msgr2=0x7f6a8806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a8806a050 0x7f6a8806c510 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0035000 tx=0x7f6aa0030000 comp rx=0 tx=0).stop 2023-08-25T09:39:06.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 msgr2=0x7f6aa40c64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 0x7f6aa40c64a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400b3c0 tx=0x7f6a9400b880 comp rx=0 tx=0).stop 2023-08-25T09:39:06.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 shutdown_connections 2023-08-25T09:39:06.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a8806a050 0x7f6a8806c510 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6aa415a920 0x7f6aa40c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6aa40b1ce0 0x7f6aa40c64a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 --2- 172.21.15.70:0/2869408516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6aa40b0320 0x7f6aa41f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.293+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 >> 172.21.15.70:0/2869408516 conn(0x7f6aa40ce830 msgr2=0x7f6aa40654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:06.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.294+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 shutdown_connections 2023-08-25T09:39:06.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.294+0000 7f6aab342640 1 -- 172.21.15.70:0/2869408516 wait complete. 2023-08-25T09:39:06.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:39:06.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.518+0000 7f16df7b5640 1 Processor -- start 2023-08-25T09:39:06.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.518+0000 7f16df7b5640 1 -- start start 2023-08-25T09:39:06.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.519+0000 7f16df7b5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f16d815e830 0x7f16d806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.519+0000 7f16df7b5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16d806b2b0 0x7f16d815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16df7b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815c6d0 0x7f16d80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16df7b5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d806bd80 con 0x7f16d806b2b0 2023-08-25T09:39:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16df7b5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d80b1b60 con 0x7f16d815e830 2023-08-25T09:39:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16df7b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d806b7c0 con 0x7f16d815c6d0 2023-08-25T09:39:06.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16d806b2b0 0x7f16d815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16d806b2b0 0x7f16d815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47572/0 (socket says 172.21.15.70:47572) 2023-08-25T09:39:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 -- 172.21.15.70:0/3762931358 learned_addr learned my addr 172.21.15.70:0/3762931358 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 -- 172.21.15.70:0/3762931358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815c6d0 msgr2=0x7f16d80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dd551640 1 --2- 172.21.15.70:0/3762931358 >> v2:172.21.15.155:3300/0 conn(0x7f16d815e830 0x7f16d806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 --2- 172.21.15.70:0/3762931358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815c6d0 0x7f16d80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 -- 172.21.15.70:0/3762931358 >> v2:172.21.15.155:3300/0 conn(0x7f16d815e830 msgr2=0x7f16d806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 --2- 172.21.15.70:0/3762931358 >> v2:172.21.15.155:3300/0 conn(0x7f16d815e830 0x7f16d806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dcd50640 1 -- 172.21.15.70:0/3762931358 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d806baa0 con 0x7f16d806b2b0 2023-08-25T09:39:06.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.520+0000 7f16dd551640 1 --2- 172.21.15.70:0/3762931358 >> v2:172.21.15.155:3300/0 conn(0x7f16d815e830 0x7f16d806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:06.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.521+0000 7f16dcd50640 1 --2- 172.21.15.70:0/3762931358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d806b2b0 0x7f16d815c190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f16d806bf10 tx=0x7f16c802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a89eb9f9f720da9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16cffff640 1 -- 172.21.15.70:0/3762931358 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8031860 con 0x7f16d806b2b0 2023-08-25T09:39:06.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16cffff640 1 -- 172.21.15.70:0/3762931358 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c80319c0 con 0x7f16d806b2b0 2023-08-25T09:39:06.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16cffff640 1 -- 172.21.15.70:0/3762931358 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8031d50 con 0x7f16d806b2b0 2023-08-25T09:39:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 -- 172.21.15.70:0/3762931358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d806b2b0 msgr2=0x7f16d815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 --2- 172.21.15.70:0/3762931358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d806b2b0 0x7f16d815c190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f16d806bf10 tx=0x7f16c802b900 comp rx=0 tx=0).stop 2023-08-25T09:39:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 -- 172.21.15.70:0/3762931358 shutdown_connections 2023-08-25T09:39:06.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 --2- 172.21.15.70:0/3762931358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815c6d0 0x7f16d80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 --2- 172.21.15.70:0/3762931358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d806b2b0 0x7f16d815c190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 --2- 172.21.15.70:0/3762931358 >> v2:172.21.15.155:3300/0 conn(0x7f16d815e830 0x7f16d806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.522+0000 7f16df7b5640 1 -- 172.21.15.70:0/3762931358 >> 172.21.15.70:0/3762931358 conn(0x7f16d8069850 msgr2=0x7f16d8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:06.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.523+0000 7f16df7b5640 1 -- 172.21.15.70:0/3762931358 shutdown_connections 2023-08-25T09:39:06.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.523+0000 7f16df7b5640 1 -- 172.21.15.70:0/3762931358 wait complete. 2023-08-25T09:39:06.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.524+0000 7f16df7b5640 1 Processor -- start 2023-08-25T09:39:06.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.524+0000 7f16df7b5640 1 -- start start 2023-08-25T09:39:06.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.525+0000 7f16df7b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16d806b2b0 0x7f16d81550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.525+0000 7f16df7b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 0x7f16d80ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.525+0000 7f16df7b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815e830 0x7f16d815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.525+0000 7f16df7b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c99a0 con 0x7f16d815c6d0 2023-08-25T09:39:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.525+0000 7f16df7b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c8d90 con 0x7f16d806b2b0 2023-08-25T09:39:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.525+0000 7f16df7b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c9070 con 0x7f16d815e830 2023-08-25T09:39:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 0x7f16d80ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 0x7f16d80ca290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47580/0 (socket says 172.21.15.70:47580) 2023-08-25T09:39:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 -- 172.21.15.70:0/2297160613 learned_addr learned my addr 172.21.15.70:0/2297160613 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dd551640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16d806b2b0 0x7f16d81550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16ddd52640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815e830 0x7f16d815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 -- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815e830 msgr2=0x7f16d815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815e830 0x7f16d815ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 -- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16d806b2b0 msgr2=0x7f16d81550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16d806b2b0 0x7f16d81550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 -- 172.21.15.70:0/2297160613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c80094d0 con 0x7f16d815c6d0 2023-08-25T09:39:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dd551640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16d806b2b0 0x7f16d81550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16ddd52640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815e830 0x7f16d815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:06.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.526+0000 7f16dcd50640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 0x7f16d80ca290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16c8000c00 tx=0x7f16c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.527+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8031830 con 0x7f16d815c6d0 2023-08-25T09:39:06.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.527+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c80443f0 con 0x7f16d815c6d0 2023-08-25T09:39:06.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.527+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80447a0 con 0x7f16d815c6d0 2023-08-25T09:39:06.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.527+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d80c9350 con 0x7f16d815c6d0 2023-08-25T09:39:06.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.527+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d80c8a20 con 0x7f16d815c6d0 2023-08-25T09:39:06.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.528+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d80c9b50 con 0x7f16d815c6d0 2023-08-25T09:39:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.529+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f16c804d450 con 0x7f16d815c6d0 2023-08-25T09:39:06.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.530+0000 7f16cdffb640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f16bc06a050 0x7f16bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.530+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16c80aae00 con 0x7f16d815c6d0 2023-08-25T09:39:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.530+0000 7f16dd551640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f16bc06a050 0x7f16bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.531+0000 7f16dd551640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f16bc06a050 0x7f16bc06c510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16d4006fd0 tx=0x7f16d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.534+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f16d80c9b50 con 0x7f16d815c6d0 2023-08-25T09:39:06.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.756+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16d815bf50 con 0x7f16d815c6d0 2023-08-25T09:39:06.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.757+0000 7f16cdffb640 1 -- 172.21.15.70:0/2297160613 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7f16c8035090 con 0x7f16d815c6d0 2023-08-25T09:39:06.759 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:06.759 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:39:06.759 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:39:06.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.759+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f16bc06a050 msgr2=0x7f16bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.759+0000 7f16df7b5640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f16bc06a050 0x7f16bc06c510 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f16d4006fd0 tx=0x7f16d4008040 comp rx=0 tx=0).stop 2023-08-25T09:39:06.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.759+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 msgr2=0x7f16d80ca290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.759+0000 7f16df7b5640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 0x7f16d80ca290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f16c8000c00 tx=0x7f16c80046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 shutdown_connections 2023-08-25T09:39:06.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f16bc06a050 0x7f16bc06c510 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16d815e830 0x7f16d815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16d815c6d0 0x7f16d80ca290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 --2- 172.21.15.70:0/2297160613 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16d806b2b0 0x7f16d81550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 >> 172.21.15.70:0/2297160613 conn(0x7f16d8069850 msgr2=0x7f16d80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:06.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 shutdown_connections 2023-08-25T09:39:06.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.760+0000 7f16df7b5640 1 -- 172.21.15.70:0/2297160613 wait complete. 2023-08-25T09:39:06.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:06.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.979+0000 7f0923383640 1 Processor -- start 2023-08-25T09:39:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.979+0000 7f0923383640 1 -- start start 2023-08-25T09:39:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f0923383640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f091c15ec10 0x7f091c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f0923383640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 0x7f091c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f0923383640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091c0b1b60 con 0x7f091c15e830 2023-08-25T09:39:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f0923383640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091c06bd80 con 0x7f091c15ec10 2023-08-25T09:39:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f0923383640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c06b7c0 con 0x7f091c15c6d0 2023-08-25T09:39:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f092111f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f091c15e830 0x7f091c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35588/0 (socket says 172.21.15.70:35588) 2023-08-25T09:39:06.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f092111f640 1 -- 172.21.15.70:0/1152442583 learned_addr learned my addr 172.21.15.70:0/1152442583 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f0921920640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 0x7f091c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.981+0000 7f092091e640 1 --2- 172.21.15.70:0/1152442583 >> v2:172.21.15.155:3300/0 conn(0x7f091c15ec10 0x7f091c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222239024 0 0) 0x7f091c0b1b60 con 0x7f091c15e830 2023-08-25T09:39:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0908003520 con 0x7f091c15e830 2023-08-25T09:39:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2038806205 0 0) 0x7f0908003520 con 0x7f091c15e830 2023-08-25T09:39:06.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 msgr2=0x7f091c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 0x7f091c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 >> v2:172.21.15.155:3300/0 conn(0x7f091c15ec10 msgr2=0x7f091c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 --2- 172.21.15.70:0/1152442583 >> v2:172.21.15.155:3300/0 conn(0x7f091c15ec10 0x7f091c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c06baa0 con 0x7f091c15e830 2023-08-25T09:39:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0918001670 con 0x7f091c15e830 2023-08-25T09:39:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 >> v1:172.21.15.70:6789/0 conn(0x7f091c15e830 legacy=0x7f091c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0908003970 0x7f0908003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09080047c0 0x7f0908004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 0x7f0908005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c06bd80 con 0x7f0908003970 2023-08-25T09:39:06.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c06b7c0 con 0x7f09080047c0 2023-08-25T09:39:06.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090800b980 con 0x7f091c15c6d0 2023-08-25T09:39:06.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.982+0000 7f092111f640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0908003970 0x7f0908003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092091e640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09080047c0 0x7f0908004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:06.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092111f640 1 -- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 msgr2=0x7f0908005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092111f640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c15c6d0 0x7f0908005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092111f640 1 -- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09080047c0 msgr2=0x7f0908004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:06.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092111f640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09080047c0 0x7f0908004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092111f640 1 -- 172.21.15.70:0/1152442583 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908003520 con 0x7f0908003970 2023-08-25T09:39:06.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.983+0000 7f092091e640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09080047c0 0x7f0908004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:06.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.984+0000 7f092111f640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0908003970 0x7f0908003d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f091800c240 tx=0x7f09080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb266bf202881a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.984+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091802cb00 con 0x7f0908003970 2023-08-25T09:39:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.984+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091802cc60 con 0x7f0908003970 2023-08-25T09:39:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.984+0000 7f0913fff640 1 -- 172.21.15.70:0/1152442583 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918005af0 con 0x7f0908003970 2023-08-25T09:39:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0923383640 1 -- 172.21.15.70:0/1152442583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0908003970 msgr2=0x7f0908003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:06.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0923383640 1 --2- 172.21.15.70:0/1152442583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0908003970 0x7f0908003d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f091800c240 tx=0x7f09080057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0921920640 1 -- 172.21.15.70:0/1152442583 reap_dead start 2023-08-25T09:39:06.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0923383640 1 -- 172.21.15.70:0/1152442583 shutdown_connections 2023-08-25T09:39:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0923383640 1 -- 172.21.15.70:0/1152442583 >> 172.21.15.70:0/1152442583 conn(0x7f091c069850 msgr2=0x7f091c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:06.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0923383640 1 -- 172.21.15.70:0/1152442583 shutdown_connections 2023-08-25T09:39:06.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.985+0000 7f0923383640 1 -- 172.21.15.70:0/1152442583 wait complete. 2023-08-25T09:39:06.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.987+0000 7f0923383640 1 Processor -- start 2023-08-25T09:39:06.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.987+0000 7f0923383640 1 -- start start 2023-08-25T09:39:07.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0923383640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0908003970 0x7f091c1f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0923383640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 0x7f091c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0923383640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c1f2530 0x7f091c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0923383640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1f3460 con 0x7f091c0af430 2023-08-25T09:39:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0923383640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1f2ea0 con 0x7f0908003970 2023-08-25T09:39:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0923383640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091c1f3180 con 0x7f091c1f2530 2023-08-25T09:39:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 0x7f091c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 0x7f091c14b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47594/0 (socket says 172.21.15.70:47594) 2023-08-25T09:39:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 -- 172.21.15.70:0/3662231404 learned_addr learned my addr 172.21.15.70:0/3662231404 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:07.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0921920640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c1f2530 0x7f091c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092111f640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0908003970 0x7f091c1f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 -- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c1f2530 msgr2=0x7f091c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c1f2530 0x7f091c15ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 -- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0908003970 msgr2=0x7f091c1f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0908003970 0x7f091c1f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092091e640 1 -- 172.21.15.70:0/3662231404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c1f5c50 con 0x7f091c0af430 2023-08-25T09:39:07.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f0921920640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c1f2530 0x7f091c15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:07.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.988+0000 7f092111f640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0908003970 0x7f091c1f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:07.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.989+0000 7f092091e640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 0x7f091c14b020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f090c010880 tx=0x7f090c010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.989+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00eb60 con 0x7f091c0af430 2023-08-25T09:39:07.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.989+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00ecc0 con 0x7f091c0af430 2023-08-25T09:39:07.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.989+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c01c400 con 0x7f091c0af430 2023-08-25T09:39:07.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.989+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c1f5f30 con 0x7f091c0af430 2023-08-25T09:39:07.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.989+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c14b560 con 0x7f091c0af430 2023-08-25T09:39:07.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.990+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c04e420 con 0x7f091c0af430 2023-08-25T09:39:07.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.991+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f090c01c560 con 0x7f091c0af430 2023-08-25T09:39:07.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.992+0000 7f0911ffb640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090006a080 0x7f090006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.992+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f090c088af0 con 0x7f091c0af430 2023-08-25T09:39:07.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.992+0000 7f092111f640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090006a080 0x7f090006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.993+0000 7f092111f640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090006a080 0x7f090006c540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0918034000 tx=0x7f0918007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:06.996+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f090c054280 con 0x7f091c0af430 2023-08-25T09:39:07.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.185+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f091c06c620 con 0x7f091c0af430 2023-08-25T09:39:07.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.187+0000 7f0911ffb640 1 -- 172.21.15.70:0/3662231404 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f090c0578a0 con 0x7f091c0af430 2023-08-25T09:39:07.189 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:07.189 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:07.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.189+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090006a080 msgr2=0x7f090006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.189+0000 7f0923383640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090006a080 0x7f090006c540 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0918034000 tx=0x7f0918007000 comp rx=0 tx=0).stop 2023-08-25T09:39:07.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.189+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 msgr2=0x7f091c14b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.189+0000 7f0923383640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 0x7f091c14b020 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f090c010880 tx=0x7f090c010d40 comp rx=0 tx=0).stop 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 shutdown_connections 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f090006a080 0x7f090006c540 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091c1f2530 0x7f091c15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091c0af430 0x7f091c14b020 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 --2- 172.21.15.70:0/3662231404 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0908003970 0x7f091c1f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 >> 172.21.15.70:0/3662231404 conn(0x7f091c069850 msgr2=0x7f091c06ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:07.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 shutdown_connections 2023-08-25T09:39:07.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.190+0000 7f0923383640 1 -- 172.21.15.70:0/3662231404 wait complete. 2023-08-25T09:39:07.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:07.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.426+0000 7f71fbd42640 1 Processor -- start 2023-08-25T09:39:07.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.426+0000 7f71fbd42640 1 -- start start 2023-08-25T09:39:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.427+0000 7f71fbd42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 0x7f71f40c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.427+0000 7f71fbd42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71f40cdc20 0x7f71f4069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.427+0000 7f71fbd42640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f40cc5f0 con 0x7f71f40ce060 2023-08-25T09:39:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.427+0000 7f71fbd42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f4070650 con 0x7f71f40cdc20 2023-08-25T09:39:07.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.427+0000 7f71fbd42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4070930 con 0x7f71f415ae50 2023-08-25T09:39:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.428+0000 7f71f92dd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71f40cdc20 0x7f71f4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.428+0000 7f71f9ade640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 0x7f71f40c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.428+0000 7f71f9ade640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 0x7f71f40c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:45194/0 (socket says 172.21.15.70:45194) 2023-08-25T09:39:07.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.428+0000 7f71f9ade640 1 -- 172.21.15.70:0/3308384957 learned_addr learned my addr 172.21.15.70:0/3308384957 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.428+0000 7f71f92dd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71f40cdc20 0x7f71f4069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38980/0 (socket says 172.21.15.70:38980) 2023-08-25T09:39:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.429+0000 7f71f9ade640 1 -- 172.21.15.70:0/3308384957 >> v2:172.21.15.155:3300/0 conn(0x7f71f40cdc20 msgr2=0x7f71f4069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.429+0000 7f71f9ade640 1 --2- 172.21.15.70:0/3308384957 >> v2:172.21.15.155:3300/0 conn(0x7f71f40cdc20 0x7f71f4069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.429+0000 7f71f9ade640 1 -- 172.21.15.70:0/3308384957 >> v1:172.21.15.155:6789/0 conn(0x7f71f40ce060 legacy=0x7f71f40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.429+0000 7f71f9ade640 1 -- 172.21.15.70:0/3308384957 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4070c10 con 0x7f71f415ae50 2023-08-25T09:39:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.430+0000 7f71f9ade640 1 --2- 172.21.15.70:0/3308384957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 0x7f71f40c2110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009870 tx=0x7f71f002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf5f7d75aed0779a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.431+0000 7f71f8adc640 1 -- 172.21.15.70:0/3308384957 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f002f830 con 0x7f71f415ae50 2023-08-25T09:39:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.431+0000 7f71f8adc640 1 -- 172.21.15.70:0/3308384957 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f002f990 con 0x7f71f415ae50 2023-08-25T09:39:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.431+0000 7f71f8adc640 1 -- 172.21.15.70:0/3308384957 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f002fd20 con 0x7f71f415ae50 2023-08-25T09:39:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.431+0000 7f71fbd42640 1 -- 172.21.15.70:0/3308384957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 msgr2=0x7f71f40c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.431+0000 7f71fbd42640 1 --2- 172.21.15.70:0/3308384957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 0x7f71f40c2110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f71f0009870 tx=0x7f71f002ba10 comp rx=0 tx=0).stop 2023-08-25T09:39:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.432+0000 7f71fbd42640 1 -- 172.21.15.70:0/3308384957 shutdown_connections 2023-08-25T09:39:07.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.432+0000 7f71fbd42640 1 --2- 172.21.15.70:0/3308384957 >> v2:172.21.15.155:3300/0 conn(0x7f71f40cdc20 0x7f71f4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.432+0000 7f71fbd42640 1 --2- 172.21.15.70:0/3308384957 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f415ae50 0x7f71f40c2110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.432+0000 7f71fbd42640 1 -- 172.21.15.70:0/3308384957 >> 172.21.15.70:0/3308384957 conn(0x7f71f40cb0d0 msgr2=0x7f71f40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.432+0000 7f71fbd42640 1 -- 172.21.15.70:0/3308384957 shutdown_connections 2023-08-25T09:39:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.432+0000 7f71fbd42640 1 -- 172.21.15.70:0/3308384957 wait complete. 2023-08-25T09:39:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.434+0000 7f71fbd42640 1 Processor -- start 2023-08-25T09:39:07.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.434+0000 7f71fbd42640 1 -- start start 2023-08-25T09:39:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71fbd42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 0x7f71f40c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71fbd42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f40ce060 0x7f71f40c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71fbd42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71f415ae50 0x7f71f4069850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71fbd42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c8530 con 0x7f71f40cdc20 2023-08-25T09:39:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71fbd42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c5be0 con 0x7f71f415ae50 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71fbd42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c9f00 con 0x7f71f40ce060 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.435+0000 7f71f9ade640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 0x7f71f40c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 0x7f71f40c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47602/0 (socket says 172.21.15.70:47602) 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 -- 172.21.15.70:0/2388796783 learned_addr learned my addr 172.21.15.70:0/2388796783 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 -- 172.21.15.70:0/2388796783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f40ce060 msgr2=0x7f71f40c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f40ce060 0x7f71f40c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 -- 172.21.15.70:0/2388796783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71f415ae50 msgr2=0x7f71f4069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71f415ae50 0x7f71f4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 -- 172.21.15.70:0/2388796783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00094d0 con 0x7f71f40cdc20 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.436+0000 7f71f9ade640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 0x7f71f40c7ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71f002ba50 tx=0x7f71f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.437+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f003b040 con 0x7f71f40cdc20 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.437+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f0037c40 con 0x7f71f40cdc20 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.437+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0041400 con 0x7f71f40cdc20 2023-08-25T09:39:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.437+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f40c9660 con 0x7f71f40cdc20 2023-08-25T09:39:07.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.437+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f40c9c20 con 0x7f71f40cdc20 2023-08-25T09:39:07.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.438+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f4066fb0 con 0x7f71f40cdc20 2023-08-25T09:39:07.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.439+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f71f0041560 con 0x7f71f40cdc20 2023-08-25T09:39:07.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.440+0000 7f71ea7fc640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71d806a050 0x7f71d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.440+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71f00ab1b0 con 0x7f71f40cdc20 2023-08-25T09:39:07.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.440+0000 7f71f92dd640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71d806a050 0x7f71d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.441+0000 7f71f92dd640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71d806a050 0x7f71d806c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f71e4006a90 tx=0x7f71e4006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.444+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f71f0076890 con 0x7f71f40cdc20 2023-08-25T09:39:07.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.632+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71f415a470 con 0x7f71f40cdc20 2023-08-25T09:39:07.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.632+0000 7f71ea7fc640 1 -- 172.21.15.70:0/2388796783 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71f0079eb0 con 0x7f71f40cdc20 2023-08-25T09:39:07.634 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:07.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71d806a050 msgr2=0x7f71d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71d806a050 0x7f71d806c510 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f71e4006a90 tx=0x7f71e4006a20 comp rx=0 tx=0).stop 2023-08-25T09:39:07.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 msgr2=0x7f71f40c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 0x7f71f40c7ff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f71f002ba50 tx=0x7f71f00046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 shutdown_connections 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71d806a050 0x7f71d806c510 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71f415ae50 0x7f71f4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71f40ce060 0x7f71f40c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 --2- 172.21.15.70:0/2388796783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71f40cdc20 0x7f71f40c7ff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 >> 172.21.15.70:0/2388796783 conn(0x7f71f40cb0d0 msgr2=0x7f71f40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 shutdown_connections 2023-08-25T09:39:07.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.635+0000 7f71fbd42640 1 -- 172.21.15.70:0/2388796783 wait complete. 2023-08-25T09:39:07.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:07.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.869+0000 7f67de799640 1 Processor -- start 2023-08-25T09:39:07.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.869+0000 7f67de799640 1 -- start start 2023-08-25T09:39:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.871+0000 7f67de799640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67d8072820 0x7f67d80aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.871+0000 7f67de799640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67d80ad280 0x7f67d8072c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.871+0000 7f67de799640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d806bc10 0x7f67d806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.871+0000 7f67de799640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d80b38c0 con 0x7f67d8072820 2023-08-25T09:39:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.871+0000 7f67de799640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d8160620 con 0x7f67d80ad280 2023-08-25T09:39:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.871+0000 7f67de799640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d80af180 con 0x7f67d806bc10 2023-08-25T09:39:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67d8072820 0x7f67d80aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67d8072820 0x7f67d80aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47616/0 (socket says 172.21.15.70:47616) 2023-08-25T09:39:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 -- 172.21.15.70:0/840354826 learned_addr learned my addr 172.21.15.70:0/840354826 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67dcd36640 1 --2- 172.21.15.70:0/840354826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d806bc10 0x7f67d806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 -- 172.21.15.70:0/840354826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d806bc10 msgr2=0x7f67d806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:07.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d77fe640 1 --2- 172.21.15.70:0/840354826 >> v2:172.21.15.155:3300/0 conn(0x7f67d80ad280 0x7f67d8072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 --2- 172.21.15.70:0/840354826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d806bc10 0x7f67d806c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 -- 172.21.15.70:0/840354826 >> v2:172.21.15.155:3300/0 conn(0x7f67d80ad280 msgr2=0x7f67d8072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 --2- 172.21.15.70:0/840354826 >> v2:172.21.15.155:3300/0 conn(0x7f67d80ad280 0x7f67d8072c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d7fff640 1 -- 172.21.15.70:0/840354826 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8073140 con 0x7f67d8072820 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.872+0000 7f67d77fe640 1 --2- 172.21.15.70:0/840354826 >> v2:172.21.15.155:3300/0 conn(0x7f67d80ad280 0x7f67d8072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.873+0000 7f67d7fff640 1 --2- 172.21.15.70:0/840354826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d8072820 0x7f67d80aec40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f67d80733c0 tx=0x7f67c802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e9ad1051200fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.874+0000 7f67d6ffd640 1 -- 172.21.15.70:0/840354826 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c802c9f0 con 0x7f67d8072820 2023-08-25T09:39:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.874+0000 7f67d6ffd640 1 -- 172.21.15.70:0/840354826 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c802cb50 con 0x7f67d8072820 2023-08-25T09:39:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.874+0000 7f67d6ffd640 1 -- 172.21.15.70:0/840354826 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80343f0 con 0x7f67d8072820 2023-08-25T09:39:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 -- 172.21.15.70:0/840354826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d8072820 msgr2=0x7f67d80aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 --2- 172.21.15.70:0/840354826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d8072820 0x7f67d80aec40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f67d80733c0 tx=0x7f67c802be50 comp rx=0 tx=0).stop 2023-08-25T09:39:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 -- 172.21.15.70:0/840354826 shutdown_connections 2023-08-25T09:39:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 --2- 172.21.15.70:0/840354826 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d806bc10 0x7f67d806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 --2- 172.21.15.70:0/840354826 >> v2:172.21.15.155:3300/0 conn(0x7f67d80ad280 0x7f67d8072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 --2- 172.21.15.70:0/840354826 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d8072820 0x7f67d80aec40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 -- 172.21.15.70:0/840354826 >> 172.21.15.70:0/840354826 conn(0x7f67d80cfc70 msgr2=0x7f67d80d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 -- 172.21.15.70:0/840354826 shutdown_connections 2023-08-25T09:39:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.875+0000 7f67de799640 1 -- 172.21.15.70:0/840354826 wait complete. 2023-08-25T09:39:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.876+0000 7f67de799640 1 Processor -- start 2023-08-25T09:39:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.877+0000 7f67de799640 1 -- start start 2023-08-25T09:39:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.877+0000 7f67de799640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 0x7f67d8148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67de799640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d8072820 0x7f67d80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67de799640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d80ad280 0x7f67d80cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67de799640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8155a10 con 0x7f67d806bc10 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67de799640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8156870 con 0x7f67d80ad280 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67de799640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d8155c60 con 0x7f67d8072820 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 0x7f67d8148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 0x7f67d8148ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47620/0 (socket says 172.21.15.70:47620) 2023-08-25T09:39:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 -- 172.21.15.70:0/3321449494 learned_addr learned my addr 172.21.15.70:0/3321449494 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 -- 172.21.15.70:0/3321449494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d8072820 msgr2=0x7f67d80ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d8072820 0x7f67d80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 -- 172.21.15.70:0/3321449494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d80ad280 msgr2=0x7f67d80cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d80ad280 0x7f67d80cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.878+0000 7f67d7fff640 1 -- 172.21.15.70:0/3321449494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8009b30 con 0x7f67d806bc10 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.879+0000 7f67d7fff640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 0x7f67d8148ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000f80 tx=0x7f67c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.879+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c803e450 con 0x7f67d806bc10 2023-08-25T09:39:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.879+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c803e5f0 con 0x7f67d806bc10 2023-08-25T09:39:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.879+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d8155f40 con 0x7f67d806bc10 2023-08-25T09:39:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.879+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c803e9a0 con 0x7f67d806bc10 2023-08-25T09:39:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.879+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d8156500 con 0x7f67d806bc10 2023-08-25T09:39:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.880+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d81566c0 con 0x7f67d806bc10 2023-08-25T09:39:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.881+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67c803eb00 con 0x7f67d806bc10 2023-08-25T09:39:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.882+0000 7f67d4ff9640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67c406a080 0x7f67c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.882+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67c80ab5b0 con 0x7f67d806bc10 2023-08-25T09:39:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.882+0000 7f67d77fe640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67c406a080 0x7f67c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.883+0000 7f67d77fe640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67c406a080 0x7f67c406c540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006fd0 tx=0x7f67cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:07.886+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f67c8076dd0 con 0x7f67d806bc10 2023-08-25T09:39:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.073+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f67d8066fb0 con 0x7f67d806bc10 2023-08-25T09:39:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.074+0000 7f67d4ff9640 1 -- 172.21.15.70:0/3321449494 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f67c803edb0 con 0x7f67d806bc10 2023-08-25T09:39:08.076 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67c406a080 msgr2=0x7f67c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67c406a080 0x7f67c406c540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f67cc006fd0 tx=0x7f67cc008040 comp rx=0 tx=0).stop 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 msgr2=0x7f67d8148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 0x7f67d8148ed0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f67c8000f80 tx=0x7f67c8004a40 comp rx=0 tx=0).stop 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 shutdown_connections 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67c406a080 0x7f67c406c540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67d80ad280 0x7f67d80cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67d8072820 0x7f67d80ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 --2- 172.21.15.70:0/3321449494 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67d806bc10 0x7f67d8148ed0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 >> 172.21.15.70:0/3321449494 conn(0x7f67d80cfc70 msgr2=0x7f67d806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:08.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 shutdown_connections 2023-08-25T09:39:08.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.077+0000 7f67de799640 1 -- 172.21.15.70:0/3321449494 wait complete. 2023-08-25T09:39:08.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:08.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.299+0000 7f0f7274f640 1 Processor -- start 2023-08-25T09:39:08.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.299+0000 7f0f7274f640 1 -- start start 2023-08-25T09:39:08.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f7274f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f6c15cfe0 0x7f0f6c0d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f7274f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0d1500 con 0x7f0f6c0cd850 2023-08-25T09:39:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f7274f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f6c0d17e0 con 0x7f0f6c0b8510 2023-08-25T09:39:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f7274f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6c0d1ac0 con 0x7f0f6c15cfe0 2023-08-25T09:39:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f6bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0f6c0cd850 0x7f0f6c0cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35598/0 (socket says 172.21.15.70:35598) 2023-08-25T09:39:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f6bfff640 1 -- 172.21.15.70:0/2273842828 learned_addr learned my addr 172.21.15.70:0/2273842828 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:08.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.301+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1110795249 0 0) 0x7f0f6c0d1500 con 0x7f0f6c0cd850 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f54003520 con 0x7f0f6c0cd850 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2913470378 0 0) 0x7f0f54003520 con 0x7f0f6c0cd850 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6b7fe640 1 --2- 172.21.15.70:0/2273842828 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c15cfe0 0x7f0f6c0d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c15cfe0 msgr2=0x7f0f6c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 --2- 172.21.15.70:0/2273842828 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c15cfe0 0x7f0f6c0d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 >> v1:172.21.15.155:6789/0 conn(0x7f0f6c0b8510 legacy=0x7f0f6c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:08.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0d1da0 con 0x7f0f6c0cd850 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0f5c001670 con 0x7f0f6c0cd850 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6b7fe640 1 --2- 172.21.15.70:0/2273842828 >> v2:172.21.15.155:3300/0 conn(0x7f0f6c15cfe0 0x7f0f6c0d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 >> v1:172.21.15.70:6789/0 conn(0x7f0f6c0cd850 legacy=0x7f0f6c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f54003970 0x7f0f54003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540047c0 0x7f0f54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f540052a0 0x7f0f5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c0d1ac0 con 0x7f0f540052a0 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5400c1a0 con 0x7f0f540047c0 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5400c360 con 0x7f0f54003970 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.302+0000 7f0f70cec640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f540052a0 0x7f0f5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.303+0000 7f0f70cec640 1 -- 172.21.15.70:0/2273842828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f54003970 msgr2=0x7f0f54003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.303+0000 7f0f70cec640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f54003970 0x7f0f54003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.303+0000 7f0f70cec640 1 -- 172.21.15.70:0/2273842828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540047c0 msgr2=0x7f0f54004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:08.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.303+0000 7f0f70cec640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540047c0 0x7f0f54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.303+0000 7f0f70cec640 1 -- 172.21.15.70:0/2273842828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54003520 con 0x7f0f540052a0 2023-08-25T09:39:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.304+0000 7f0f70cec640 1 -- 172.21.15.70:0/2273842828 reap_dead start 2023-08-25T09:39:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.304+0000 7f0f70cec640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f540052a0 0x7f0f5400b980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f640044c0 tx=0x7f0f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed7f2dd6bb3edd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.304+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f640305f0 con 0x7f0f540052a0 2023-08-25T09:39:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.304+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64030770 con 0x7f0f540052a0 2023-08-25T09:39:08.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.304+0000 7f0f6affd640 1 -- 172.21.15.70:0/2273842828 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64030b20 con 0x7f0f540052a0 2023-08-25T09:39:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 -- 172.21.15.70:0/2273842828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f540052a0 msgr2=0x7f0f5400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f540052a0 0x7f0f5400b980 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f640044c0 tx=0x7f0f540057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 -- 172.21.15.70:0/2273842828 shutdown_connections 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2273842828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f540052a0 0x7f0f5400b980 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 -- 172.21.15.70:0/2273842828 >> 172.21.15.70:0/2273842828 conn(0x7f0f6c069850 msgr2=0x7f0f6c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 -- 172.21.15.70:0/2273842828 shutdown_connections 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.305+0000 7f0f7274f640 1 -- 172.21.15.70:0/2273842828 wait complete. 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.307+0000 7f0f7274f640 1 Processor -- start 2023-08-25T09:39:08.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.307+0000 7f0f7274f640 1 -- start start 2023-08-25T09:39:08.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.308+0000 7f0f7274f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540052a0 0x7f0f6c1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.308+0000 7f0f7274f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 0x7f0f6c1f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.308+0000 7f0f7274f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c15d700 0x7f0f6c15db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.308+0000 7f0f7274f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c15e0d0 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.308+0000 7f0f7274f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1f54b0 con 0x7f0f540052a0 2023-08-25T09:39:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.308+0000 7f0f7274f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6c1f30c0 con 0x7f0f6c15d700 2023-08-25T09:39:08.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 0x7f0f6c1f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 0x7f0f6c1f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47640/0 (socket says 172.21.15.70:47640) 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 -- 172.21.15.70:0/2444209177 learned_addr learned my addr 172.21.15.70:0/2444209177 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f70cec640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c15d700 0x7f0f6c15db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6bfff640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540052a0 0x7f0f6c1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 -- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c15d700 msgr2=0x7f0f6c15db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c15d700 0x7f0f6c15db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 -- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540052a0 msgr2=0x7f0f6c1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540052a0 0x7f0f6c1f4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 -- 172.21.15.70:0/2444209177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f6c0b47d0 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.309+0000 7f0f6b7fe640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 0x7f0f6c1f6030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000c7f0 tx=0x7f0f6000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.310+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60007820 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.310+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f60007980 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.310+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6c1f2540 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.310+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60007d30 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.310+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6c1f2b00 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.311+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f6c066fb0 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.312+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0f600113f0 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.312+0000 7f0f68ff9640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0f5806a050 0x7f0f5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.313+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f600888d0 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.313+0000 7f0f6bfff640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0f5806a050 0x7f0f5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.313+0000 7f0f6bfff640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0f5806a050 0x7f0f5806c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000fb0 tx=0x7f0f5c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:08.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.317+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0f60053f20 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.504+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f6c0c98a0 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.504+0000 7f0f68ff9640 1 -- 172.21.15.70:0/2444209177 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f60057540 con 0x7f0f6c1f20a0 2023-08-25T09:39:08.506 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0f5806a050 msgr2=0x7f0f5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0f5806a050 0x7f0f5806c510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000fb0 tx=0x7f0f5c009040 comp rx=0 tx=0).stop 2023-08-25T09:39:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 msgr2=0x7f0f6c1f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 0x7f0f6c1f6030 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000c7f0 tx=0x7f0f6000ccb0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 shutdown_connections 2023-08-25T09:39:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0f5806a050 0x7f0f5806c510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f6c15d700 0x7f0f6c15db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f6c1f20a0 0x7f0f6c1f6030 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 --2- 172.21.15.70:0/2444209177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f540052a0 0x7f0f6c1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 >> 172.21.15.70:0/2444209177 conn(0x7f0f6c069850 msgr2=0x7f0f6c15bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 shutdown_connections 2023-08-25T09:39:08.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.507+0000 7f0f7274f640 1 -- 172.21.15.70:0/2444209177 wait complete. 2023-08-25T09:39:08.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:08.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.740+0000 7faf7f59e640 1 Processor -- start 2023-08-25T09:39:08.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.740+0000 7faf7f59e640 1 -- start start 2023-08-25T09:39:08.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf800c9680 0x7faf800c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7f59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800cd770 con 0x7faf800acbd0 2023-08-25T09:39:08.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7f59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800b2270 con 0x7faf80159020 2023-08-25T09:39:08.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800ccbd0 con 0x7faf800c9680 2023-08-25T09:39:08.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf800c9680 0x7faf800c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf800c9680 0x7faf800c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47652/0 (socket says 172.21.15.70:47652) 2023-08-25T09:39:08.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7e59c640 1 -- 172.21.15.70:0/2625052158 learned_addr learned my addr 172.21.15.70:0/2625052158 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:08.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.741+0000 7faf7ed9d640 1 --1- 172.21.15.70:0/2625052158 >> v1:172.21.15.70:6789/0 conn(0x7faf800acbd0 0x7faf800acfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35610/0 (socket says 172.21.15.70:35610) 2023-08-25T09:39:08.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.742+0000 7faf7e59c640 1 -- 172.21.15.70:0/2625052158 >> v1:172.21.15.155:6789/0 conn(0x7faf80159020 legacy=0x7faf800c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:08.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.742+0000 7faf7e59c640 1 -- 172.21.15.70:0/2625052158 >> v1:172.21.15.70:6789/0 conn(0x7faf800acbd0 legacy=0x7faf800acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.742+0000 7faf7e59c640 1 -- 172.21.15.70:0/2625052158 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf800c9fa0 con 0x7faf800c9680 2023-08-25T09:39:08.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.742+0000 7faf7dd9b640 1 -- 172.21.15.70:0/2625052158 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091248438 0 0) 0x7faf800cd770 con 0x7faf800acbd0 2023-08-25T09:39:08.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.743+0000 7faf7e59c640 1 --2- 172.21.15.70:0/2625052158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf800c9680 0x7faf800c9a60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf70009860 tx=0x7faf7002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2257c0add4f7212d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:08.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.743+0000 7faf7dd9b640 1 -- 172.21.15.70:0/2625052158 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70030840 con 0x7faf800c9680 2023-08-25T09:39:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.743+0000 7faf7dd9b640 1 -- 172.21.15.70:0/2625052158 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf700309a0 con 0x7faf800c9680 2023-08-25T09:39:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.743+0000 7faf7dd9b640 1 -- 172.21.15.70:0/2625052158 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70030d30 con 0x7faf800c9680 2023-08-25T09:39:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 -- 172.21.15.70:0/2625052158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf800c9680 msgr2=0x7faf800c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2625052158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf800c9680 0x7faf800c9a60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf70009860 tx=0x7faf7002b980 comp rx=0 tx=0).stop 2023-08-25T09:39:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 -- 172.21.15.70:0/2625052158 shutdown_connections 2023-08-25T09:39:08.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2625052158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf800c9680 0x7faf800c9a60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 -- 172.21.15.70:0/2625052158 >> 172.21.15.70:0/2625052158 conn(0x7faf800ce830 msgr2=0x7faf800cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 -- 172.21.15.70:0/2625052158 shutdown_connections 2023-08-25T09:39:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.744+0000 7faf7f59e640 1 -- 172.21.15.70:0/2625052158 wait complete. 2023-08-25T09:39:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.746+0000 7faf7f59e640 1 Processor -- start 2023-08-25T09:39:08.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.746+0000 7faf7f59e640 1 -- start start 2023-08-25T09:39:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7f59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf800acbd0 0x7faf800c1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7f59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf800c9680 0x7faf80156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 0x7faf800c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80156b60 con 0x7faf80159020 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7f59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80168920 con 0x7faf800c9680 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7f59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf801568c0 con 0x7faf800acbd0 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 0x7faf800c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 0x7faf800c3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47660/0 (socket says 172.21.15.70:47660) 2023-08-25T09:39:08.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.747+0000 7faf7ed9d640 1 -- 172.21.15.70:0/2294165476 learned_addr learned my addr 172.21.15.70:0/2294165476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7e59c640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf800acbd0 0x7faf800c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf75d9b640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf800c9680 0x7faf80156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7ed9d640 1 -- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf800acbd0 msgr2=0x7faf800c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7ed9d640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf800acbd0 0x7faf800c1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7ed9d640 1 -- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf800c9680 msgr2=0x7faf80156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7ed9d640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf800c9680 0x7faf80156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7ed9d640 1 -- 172.21.15.70:0/2294165476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf70009470 con 0x7faf80159020 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7ed9d640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 0x7faf800c3e70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c450 tx=0x7faf7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780074e0 con 0x7faf80159020 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf78007660 con 0x7faf80159020 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf801f33a0 con 0x7faf80159020 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.748+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78004a20 con 0x7faf80159020 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.749+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf801682d0 con 0x7faf80159020 2023-08-25T09:39:08.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.750+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf800ca970 con 0x7faf80159020 2023-08-25T09:39:08.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.751+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf78004b80 con 0x7faf80159020 2023-08-25T09:39:08.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.751+0000 7faf777fe640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf6006a050 0x7faf6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:08.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.752+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf78087720 con 0x7faf80159020 2023-08-25T09:39:08.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.752+0000 7faf7e59c640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf6006a050 0x7faf6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:08.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.752+0000 7faf7e59c640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf6006a050 0x7faf6006c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf70036000 tx=0x7faf7003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:08.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.756+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf78052d70 con 0x7faf80159020 2023-08-25T09:39:08.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.945+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf80168a90 con 0x7faf80159020 2023-08-25T09:39:08.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.946+0000 7faf777fe640 1 -- 172.21.15.70:0/2294165476 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf78056390 con 0x7faf80159020 2023-08-25T09:39:08.947 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:08.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf6006a050 msgr2=0x7faf6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf6006a050 0x7faf6006c510 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf70036000 tx=0x7faf7003b040 comp rx=0 tx=0).stop 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 msgr2=0x7faf800c3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 0x7faf800c3e70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faf7800c450 tx=0x7faf7800c910 comp rx=0 tx=0).stop 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 shutdown_connections 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf6006a050 0x7faf6006c510 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf80159020 0x7faf800c3e70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf800c9680 0x7faf80156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 --2- 172.21.15.70:0/2294165476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf800acbd0 0x7faf800c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:08.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 >> 172.21.15.70:0/2294165476 conn(0x7faf800ce830 msgr2=0x7faf800b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:08.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 shutdown_connections 2023-08-25T09:39:08.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:08.948+0000 7faf7f59e640 1 -- 172.21.15.70:0/2294165476 wait complete. 2023-08-25T09:39:08.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:39:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.177+0000 7f48b05e8640 1 Processor -- start 2023-08-25T09:39:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.177+0000 7f48b05e8640 1 -- start start 2023-08-25T09:39:09.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.178+0000 7f48b05e8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48a80b2070 0x7f48a81594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48b05e8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a806ae10 con 0x7f48a80709a0 2023-08-25T09:39:09.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48b05e8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a806b0b0 con 0x7f48a8159100 2023-08-25T09:39:09.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48b05e8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a806ba50 con 0x7f48a80b2070 2023-08-25T09:39:09.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48adb83640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f48a80709a0 0x7f48a80b1960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34342/0 (socket says 172.21.15.70:34342) 2023-08-25T09:39:09.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48adb83640 1 -- 172.21.15.70:0/297897603 learned_addr learned my addr 172.21.15.70:0/297897603 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48aeb85640 1 --2- 172.21.15.70:0/297897603 >> v2:172.21.15.70:3300/0 conn(0x7f48a80b2070 0x7f48a81594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48aeb85640 1 -- 172.21.15.70:0/297897603 >> v1:172.21.15.155:6789/0 conn(0x7f48a8159100 legacy=0x7f48a806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48aeb85640 1 -- 172.21.15.70:0/297897603 >> v1:172.21.15.70:6789/0 conn(0x7f48a80709a0 legacy=0x7f48a80b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48aeb85640 1 -- 172.21.15.70:0/297897603 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a806b390 con 0x7f48a80b2070 2023-08-25T09:39:09.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.179+0000 7f48ad382640 1 -- 172.21.15.70:0/297897603 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823124236 0 0) 0x7f48a806ae10 con 0x7f48a80709a0 2023-08-25T09:39:09.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.180+0000 7f48aeb85640 1 --2- 172.21.15.70:0/297897603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80b2070 0x7f48a81594e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009d90 tx=0x7f48a002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7cabb305e6eec0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48ad382640 1 -- 172.21.15.70:0/297897603 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a002cc60 con 0x7f48a80b2070 2023-08-25T09:39:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48ad382640 1 -- 172.21.15.70:0/297897603 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a002cdc0 con 0x7f48a80b2070 2023-08-25T09:39:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48ad382640 1 -- 172.21.15.70:0/297897603 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0034660 con 0x7f48a80b2070 2023-08-25T09:39:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 -- 172.21.15.70:0/297897603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80b2070 msgr2=0x7f48a81594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 --2- 172.21.15.70:0/297897603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80b2070 0x7f48a81594e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48a0009d90 tx=0x7f48a002beb0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 -- 172.21.15.70:0/297897603 shutdown_connections 2023-08-25T09:39:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 --2- 172.21.15.70:0/297897603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80b2070 0x7f48a81594e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 -- 172.21.15.70:0/297897603 >> 172.21.15.70:0/297897603 conn(0x7f48a8069850 msgr2=0x7f48a8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:09.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 -- 172.21.15.70:0/297897603 shutdown_connections 2023-08-25T09:39:09.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.181+0000 7f48b05e8640 1 -- 172.21.15.70:0/297897603 wait complete. 2023-08-25T09:39:09.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.183+0000 7f48b05e8640 1 Processor -- start 2023-08-25T09:39:09.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.183+0000 7f48b05e8640 1 -- start start 2023-08-25T09:39:09.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48b05e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 0x7f48a80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48b05e8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48a80b2070 0x7f48a80b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48b05e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48a8159100 0x7f48a80b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48b05e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815ba40 con 0x7f48a80709a0 2023-08-25T09:39:09.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48b05e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815a700 con 0x7f48a8159100 2023-08-25T09:39:09.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48b05e8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a815a9a0 con 0x7f48a80b2070 2023-08-25T09:39:09.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48ae384640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 0x7f48a80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48ae384640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 0x7f48a80cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35840/0 (socket says 172.21.15.70:35840) 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.184+0000 7f48ae384640 1 -- 172.21.15.70:0/1540752070 learned_addr learned my addr 172.21.15.70:0/1540752070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48aeb85640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48a8159100 0x7f48a80b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48ae384640 1 -- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48a80b2070 msgr2=0x7f48a80b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48ae384640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48a80b2070 0x7f48a80b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48ae384640 1 -- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48a8159100 msgr2=0x7f48a80b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48ae384640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48a8159100 0x7f48a80b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48ae384640 1 -- 172.21.15.70:0/1540752070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0009a70 con 0x7f48a80709a0 2023-08-25T09:39:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48ae384640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 0x7f48a80cd7f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c400 tx=0x7f48a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40052d0 con 0x7f48a80709a0 2023-08-25T09:39:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a4005430 con 0x7f48a80709a0 2023-08-25T09:39:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a815ac80 con 0x7f48a80709a0 2023-08-25T09:39:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4003830 con 0x7f48a80709a0 2023-08-25T09:39:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.185+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a815b240 con 0x7f48a80709a0 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.187+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a804e420 con 0x7f48a80709a0 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.187+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f48a4003990 con 0x7f48a80709a0 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.188+0000 7f489effd640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f488c06a050 0x7f488c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.188+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f48a4087bd0 con 0x7f48a80709a0 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.188+0000 7f48adb83640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f488c06a050 0x7f488c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.189+0000 7f48adb83640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f488c06a050 0x7f488c06c510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:09.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.192+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48a4053220 con 0x7f48a80709a0 2023-08-25T09:39:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.421+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48a80cabc0 con 0x7f48a80709a0 2023-08-25T09:39:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.423+0000 7f489effd640 1 -- 172.21.15.70:0/1540752070 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7f48a4056840 con 0x7f48a80709a0 2023-08-25T09:39:09.424 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:09.425 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:39:09.425 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:39:09.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.425+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f488c06a050 msgr2=0x7f488c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.425+0000 7f48b05e8640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f488c06a050 0x7f488c06c510 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4898007fd0 tx=0x7f4898007900 comp rx=0 tx=0).stop 2023-08-25T09:39:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.425+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 msgr2=0x7f48a80cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.425+0000 7f48b05e8640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 0x7f48a80cd7f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c400 tx=0x7f48a400c8c0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.425+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 shutdown_connections 2023-08-25T09:39:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.425+0000 7f48b05e8640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f488c06a050 0x7f488c06c510 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.426+0000 7f48b05e8640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48a8159100 0x7f48a80b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.426+0000 7f48b05e8640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48a80b2070 0x7f48a80b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.426+0000 7f48b05e8640 1 --2- 172.21.15.70:0/1540752070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48a80709a0 0x7f48a80cd7f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.426+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 >> 172.21.15.70:0/1540752070 conn(0x7f48a8069850 msgr2=0x7f48a80ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:09.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.426+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 shutdown_connections 2023-08-25T09:39:09.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.426+0000 7f48b05e8640 1 -- 172.21.15.70:0/1540752070 wait complete. 2023-08-25T09:39:09.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:09.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.651+0000 7f631ea38640 1 Processor -- start 2023-08-25T09:39:09.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.651+0000 7f631ea38640 1 -- start start 2023-08-25T09:39:09.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631ea38640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f631815cbd0 0x7f63180cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631ea38640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 0x7f63180cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631ea38640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631815cfb0 con 0x7f63180cdeb0 2023-08-25T09:39:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631ea38640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63180cd4e0 con 0x7f631815cbd0 2023-08-25T09:39:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631ea38640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6318160f10 con 0x7f63181606c0 2023-08-25T09:39:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631cfd5640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63180cdeb0 0x7f63180d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34356/0 (socket says 172.21.15.70:34356) 2023-08-25T09:39:09.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.653+0000 7f631cfd5640 1 -- 172.21.15.70:0/3760709767 learned_addr learned my addr 172.21.15.70:0/3760709767 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f6317fff640 1 --2- 172.21.15.70:0/3760709767 >> v2:172.21.15.155:3300/0 conn(0x7f631815cbd0 0x7f63180cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f630ffff640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 0x7f63180cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886699922 0 0) 0x7f631815cfb0 con 0x7f63180cdeb0 2023-08-25T09:39:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6304003520 con 0x7f63180cdeb0 2023-08-25T09:39:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2157471300 0 0) 0x7f6304003520 con 0x7f63180cdeb0 2023-08-25T09:39:09.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 msgr2=0x7f63180cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 0x7f63180cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 >> v2:172.21.15.155:3300/0 conn(0x7f631815cbd0 msgr2=0x7f63180cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 --2- 172.21.15.70:0/3760709767 >> v2:172.21.15.155:3300/0 conn(0x7f631815cbd0 0x7f63180cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63181611f0 con 0x7f63180cdeb0 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f63100019f0 con 0x7f63180cdeb0 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 >> v1:172.21.15.70:6789/0 conn(0x7f63180cdeb0 legacy=0x7f63180d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 0x7f6304003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f63040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6304004d00 0x7f6304005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63180cd4e0 con 0x7f6304003ef0 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318160f10 con 0x7f6304004d00 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.654+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630400b980 con 0x7f63181606c0 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.655+0000 7f630ffff640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f63040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.655+0000 7f630ffff640 1 -- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 msgr2=0x7f6304003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:09.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.655+0000 7f630ffff640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63181606c0 0x7f6304003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.655+0000 7f630ffff640 1 -- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6304004d00 msgr2=0x7f6304005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:09.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.655+0000 7f630ffff640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6304004d00 0x7f6304005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.655+0000 7f630ffff640 1 -- 172.21.15.70:0/3760709767 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6304003520 con 0x7f6304003ef0 2023-08-25T09:39:09.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.656+0000 7f630ffff640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f63040047c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6300009ac0 tx=0x7f63040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ccd666eced6421 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:09.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.656+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630002c6d0 con 0x7f6304003ef0 2023-08-25T09:39:09.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.656+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630002c830 con 0x7f6304003ef0 2023-08-25T09:39:09.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.656+0000 7f63177fe640 1 -- 172.21.15.70:0/3760709767 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630002cbc0 con 0x7f6304003ef0 2023-08-25T09:39:09.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631ea38640 1 -- 172.21.15.70:0/3760709767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 msgr2=0x7f63040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631ea38640 1 --2- 172.21.15.70:0/3760709767 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f63040047c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6300009ac0 tx=0x7f63040057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631cfd5640 1 -- 172.21.15.70:0/3760709767 reap_dead start 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631ea38640 1 -- 172.21.15.70:0/3760709767 shutdown_connections 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631ea38640 1 -- 172.21.15.70:0/3760709767 >> 172.21.15.70:0/3760709767 conn(0x7f6318070260 msgr2=0x7f6318070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631ea38640 1 -- 172.21.15.70:0/3760709767 shutdown_connections 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.657+0000 7f631ea38640 1 -- 172.21.15.70:0/3760709767 wait complete. 2023-08-25T09:39:09.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.659+0000 7f631ea38640 1 Processor -- start 2023-08-25T09:39:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.659+0000 7f631ea38640 1 -- start start 2023-08-25T09:39:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f631ea38640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f6318150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f631ea38640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f631815e1a0 0x7f631815da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f631ea38640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63180cbe40 0x7f63180cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f631ea38640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318151510 con 0x7f6304003ef0 2023-08-25T09:39:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f631ea38640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631815ffd0 con 0x7f631815e1a0 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f631ea38640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63180b60b0 con 0x7f63180cbe40 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f6317fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f6318150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f6317fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f6318150fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35852/0 (socket says 172.21.15.70:35852) 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.660+0000 7f6317fff640 1 -- 172.21.15.70:0/4100034566 learned_addr learned my addr 172.21.15.70:0/4100034566 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f630ffff640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f631815e1a0 0x7f631815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f6317fff640 1 -- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63180cbe40 msgr2=0x7f63180cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f6317fff640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63180cbe40 0x7f63180cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f6317fff640 1 -- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f631815e1a0 msgr2=0x7f631815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f6317fff640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f631815e1a0 0x7f631815da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f6317fff640 1 -- 172.21.15.70:0/4100034566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631815dfc0 con 0x7f6304003ef0 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f6317fff640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f6318150fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f630800e8c0 tx=0x7f630800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800c940 con 0x7f6304003ef0 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f630800cac0 con 0x7f6304003ef0 2023-08-25T09:39:09.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631815e740 con 0x7f6304003ef0 2023-08-25T09:39:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630800ce70 con 0x7f6304003ef0 2023-08-25T09:39:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.661+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631815ed00 con 0x7f6304003ef0 2023-08-25T09:39:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.662+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631804e420 con 0x7f6304003ef0 2023-08-25T09:39:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.663+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6308013470 con 0x7f6304003ef0 2023-08-25T09:39:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.664+0000 7f63157fa640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f806a080 0x7f62f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.665+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6308088770 con 0x7f6304003ef0 2023-08-25T09:39:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.665+0000 7f630ffff640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f806a080 0x7f62f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.665+0000 7f630ffff640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f806a080 0x7f62f806c540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6300036000 tx=0x7f6300030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:09.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.669+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6308057850 con 0x7f6304003ef0 2023-08-25T09:39:09.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.862+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63180d0710 con 0x7f6304003ef0 2023-08-25T09:39:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.864+0000 7f63157fa640 1 -- 172.21.15.70:0/4100034566 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f63080573e0 con 0x7f6304003ef0 2023-08-25T09:39:09.866 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:09.866 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f806a080 msgr2=0x7f62f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f806a080 0x7f62f806c540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6300036000 tx=0x7f6300030000 comp rx=0 tx=0).stop 2023-08-25T09:39:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 msgr2=0x7f6318150fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f6318150fd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f630800e8c0 tx=0x7f630800ed80 comp rx=0 tx=0).stop 2023-08-25T09:39:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 shutdown_connections 2023-08-25T09:39:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f806a080 0x7f62f806c540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63180cbe40 0x7f63180cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f631815e1a0 0x7f631815da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 --2- 172.21.15.70:0/4100034566 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6304003ef0 0x7f6318150fd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.866+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 >> 172.21.15.70:0/4100034566 conn(0x7f6318070260 msgr2=0x7f6318157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.867+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 shutdown_connections 2023-08-25T09:39:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:09.867+0000 7f631ea38640 1 -- 172.21.15.70:0/4100034566 wait complete. 2023-08-25T09:39:09.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:39:10.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.100+0000 7f5a2d20d640 1 Processor -- start 2023-08-25T09:39:10.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.100+0000 7f5a2d20d640 1 -- start start 2023-08-25T09:39:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2d20d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5a2815e830 0x7f5a2806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2d20d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a2815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2d20d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a280b1b60 con 0x7f5a2815c6d0 2023-08-25T09:39:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2d20d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2806bd80 con 0x7f5a2815e830 2023-08-25T09:39:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2d20d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806b7c0 con 0x7f5a2806b2b0 2023-08-25T09:39:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2759e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5a2815c6d0 0x7f5a280b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34372/0 (socket says 172.21.15.70:34372) 2023-08-25T09:39:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.102+0000 7f5a2759e640 1 -- 172.21.15.70:0/3636747266 learned_addr learned my addr 172.21.15.70:0/3636747266 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a1ffff640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a2815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/3636747266 >> v2:172.21.15.155:3300/0 conn(0x7f5a2815e830 0x7f5a2806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415401031 0 0) 0x7f5a280b1b60 con 0x7f5a2815c6d0 2023-08-25T09:39:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a14003520 con 0x7f5a2815c6d0 2023-08-25T09:39:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1553866246 0 0) 0x7f5a14003520 con 0x7f5a2815c6d0 2023-08-25T09:39:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 msgr2=0x7f5a2815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a2815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 >> v2:172.21.15.155:3300/0 conn(0x7f5a2815e830 msgr2=0x7f5a2806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 --2- 172.21.15.70:0/3636747266 >> v2:172.21.15.155:3300/0 conn(0x7f5a2815e830 0x7f5a2806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2806baa0 con 0x7f5a2815c6d0 2023-08-25T09:39:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5a200019f0 con 0x7f5a2815c6d0 2023-08-25T09:39:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 >> v1:172.21.15.70:6789/0 conn(0x7f5a2815c6d0 legacy=0x7f5a280b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14003970 0x7f5a14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a140047c0 0x7f5a14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.103+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806bd80 con 0x7f5a14003970 2023-08-25T09:39:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806b7c0 con 0x7f5a140047c0 2023-08-25T09:39:10.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1400b980 con 0x7f5a2806b2b0 2023-08-25T09:39:10.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14003970 0x7f5a14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a1ffff640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a140047c0 0x7f5a14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a26d9d640 1 -- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 msgr2=0x7f5a14005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:10.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a2759e640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a14005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a26d9d640 1 -- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a140047c0 msgr2=0x7f5a14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a140047c0 0x7f5a14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a26d9d640 1 -- 172.21.15.70:0/3636747266 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14003520 con 0x7f5a14003970 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.104+0000 7f5a2759e640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a2806b2b0 0x7f5a14005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.105+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14003970 0x7f5a14003d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009ac0 tx=0x7f5a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b678b58689f27 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:10.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1802c710 con 0x7f5a14003970 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1802c870 con 0x7f5a14003970 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2659c640 1 -- 172.21.15.70:0/3636747266 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1802cc00 con 0x7f5a14003970 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2d20d640 1 -- 172.21.15.70:0/3636747266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14003970 msgr2=0x7f5a14003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/3636747266 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a14003970 0x7f5a14003d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009ac0 tx=0x7f5a140057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2759e640 1 -- 172.21.15.70:0/3636747266 reap_dead start 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2d20d640 1 -- 172.21.15.70:0/3636747266 shutdown_connections 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2d20d640 1 -- 172.21.15.70:0/3636747266 >> 172.21.15.70:0/3636747266 conn(0x7f5a28069850 msgr2=0x7f5a28069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:10.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2d20d640 1 -- 172.21.15.70:0/3636747266 shutdown_connections 2023-08-25T09:39:10.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.106+0000 7f5a2d20d640 1 -- 172.21.15.70:0/3636747266 wait complete. 2023-08-25T09:39:10.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.108+0000 7f5a2d20d640 1 Processor -- start 2023-08-25T09:39:10.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.108+0000 7f5a2d20d640 1 -- start start 2023-08-25T09:39:10.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a2d20d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14003970 0x7f5a28150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a2d20d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 0x7f5a280c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a2d20d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a28150a50 0x7f5a28150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a2d20d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28150850 con 0x7f5a28155e40 2023-08-25T09:39:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a2d20d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281559b0 con 0x7f5a14003970 2023-08-25T09:39:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a2d20d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a28154da0 con 0x7f5a28150a50 2023-08-25T09:39:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 0x7f5a280c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a1ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 0x7f5a280c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35876/0 (socket says 172.21.15.70:35876) 2023-08-25T09:39:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.109+0000 7f5a1ffff640 1 -- 172.21.15.70:0/861184412 learned_addr learned my addr 172.21.15.70:0/861184412 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:10.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1ffff640 1 -- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a28150a50 msgr2=0x7f5a28150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1ffff640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a28150a50 0x7f5a28150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1ffff640 1 -- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14003970 msgr2=0x7f5a28150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14003970 0x7f5a28150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1ffff640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14003970 0x7f5a28150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1ffff640 1 -- 172.21.15.70:0/861184412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28155080 con 0x7f5a28155e40 2023-08-25T09:39:10.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1ffff640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 0x7f5a280c8250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5a10011560 tx=0x7f5a10011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004220 con 0x7f5a28155e40 2023-08-25T09:39:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10004380 con 0x7f5a28155e40 2023-08-25T09:39:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10015870 con 0x7f5a28155e40 2023-08-25T09:39:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.110+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a28155360 con 0x7f5a28155e40 2023-08-25T09:39:10.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.111+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a280ca050 con 0x7f5a28155e40 2023-08-25T09:39:10.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.112+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2804e420 con 0x7f5a28155e40 2023-08-25T09:39:10.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.113+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5a100159d0 con 0x7f5a28155e40 2023-08-25T09:39:10.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.114+0000 7f5a1f7fe640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0806a080 0x7f5a0806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.114+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a10088710 con 0x7f5a28155e40 2023-08-25T09:39:10.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.114+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0806a080 0x7f5a0806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.115+0000 7f5a26d9d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0806a080 0x7f5a0806c540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a18036000 tx=0x7f5a18030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:10.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.118+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5a10053e00 con 0x7f5a28155e40 2023-08-25T09:39:10.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.348+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a2806c620 con 0x7f5a28155e40 2023-08-25T09:39:10.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.349+0000 7f5a1f7fe640 1 -- 172.21.15.70:0/861184412 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7f5a10057420 con 0x7f5a28155e40 2023-08-25T09:39:10.351 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:10.352 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:39:10.352 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:39:10.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0806a080 msgr2=0x7f5a0806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0806a080 0x7f5a0806c540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a18036000 tx=0x7f5a18030000 comp rx=0 tx=0).stop 2023-08-25T09:39:10.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 msgr2=0x7f5a280c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 0x7f5a280c8250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5a10011560 tx=0x7f5a10011a20 comp rx=0 tx=0).stop 2023-08-25T09:39:10.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 shutdown_connections 2023-08-25T09:39:10.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0806a080 0x7f5a0806c540 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a28150a50 0x7f5a28150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a28155e40 0x7f5a280c8250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 --2- 172.21.15.70:0/861184412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a14003970 0x7f5a28150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 >> 172.21.15.70:0/861184412 conn(0x7f5a28069850 msgr2=0x7f5a280cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:10.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 shutdown_connections 2023-08-25T09:39:10.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.352+0000 7f5a2d20d640 1 -- 172.21.15.70:0/861184412 wait complete. 2023-08-25T09:39:10.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:38:52.608524+0000', 'modified': '2023-08-25T09:38:56.204697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6835/2636978427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6834', 'nonce': 2636978427}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 2636978427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:39:10.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:39:10.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-25T09:39:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.992+0000 7f03842ba640 1 Processor -- start 2023-08-25T09:39:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.992+0000 7f03842ba640 1 -- start start 2023-08-25T09:39:10.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f03842ba640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f037c0ca780 0x7f037c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f03842ba640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f037c0b3cf0 0x7f037c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f03842ba640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f037c0b12b0 con 0x7f037c15acd0 2023-08-25T09:39:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f03842ba640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f037c06c0e0 con 0x7f037c0ca780 2023-08-25T09:39:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f03842ba640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f037c06ae20 con 0x7f037c0b3cf0 2023-08-25T09:39:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0382056640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f037c15acd0 0x7f037c15b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34388/0 (socket says 172.21.15.70:34388) 2023-08-25T09:39:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0382056640 1 -- 172.21.15.70:0/3262845597 learned_addr learned my addr 172.21.15.70:0/3262845597 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:10.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0381855640 1 --2- 172.21.15.70:0/3262845597 >> v2:172.21.15.70:3300/0 conn(0x7f037c0ca780 0x7f037c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0381855640 1 -- 172.21.15.70:0/3262845597 >> v2:172.21.15.155:3300/0 conn(0x7f037c0b3cf0 msgr2=0x7f037c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0381855640 1 --2- 172.21.15.70:0/3262845597 >> v2:172.21.15.155:3300/0 conn(0x7f037c0b3cf0 0x7f037c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0381855640 1 -- 172.21.15.70:0/3262845597 >> v1:172.21.15.70:6789/0 conn(0x7f037c15acd0 legacy=0x7f037c15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0381855640 1 -- 172.21.15.70:0/3262845597 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037c15b7c0 con 0x7f037c0ca780 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.994+0000 7f0381054640 1 -- 172.21.15.70:0/3262845597 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367406790 0 0) 0x7f037c0b12b0 con 0x7f037c15acd0 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.995+0000 7f0381855640 1 --2- 172.21.15.70:0/3262845597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0ca780 0x7f037c0cac30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f037c0c8d30 tx=0x7f036c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fd958b66d65862 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f0381054640 1 -- 172.21.15.70:0/3262845597 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c02e070 con 0x7f037c0ca780 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f0381054640 1 -- 172.21.15.70:0/3262845597 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036c0315a0 con 0x7f037c0ca780 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f0381054640 1 -- 172.21.15.70:0/3262845597 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c031930 con 0x7f037c0ca780 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 -- 172.21.15.70:0/3262845597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0ca780 msgr2=0x7f037c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 --2- 172.21.15.70:0/3262845597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0ca780 0x7f037c0cac30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f037c0c8d30 tx=0x7f036c02b840 comp rx=0 tx=0).stop 2023-08-25T09:39:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 -- 172.21.15.70:0/3262845597 shutdown_connections 2023-08-25T09:39:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 --2- 172.21.15.70:0/3262845597 >> v2:172.21.15.155:3300/0 conn(0x7f037c0b3cf0 0x7f037c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 --2- 172.21.15.70:0/3262845597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0ca780 0x7f037c0cac30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 -- 172.21.15.70:0/3262845597 >> 172.21.15.70:0/3262845597 conn(0x7f037c0ce760 msgr2=0x7f037c0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 -- 172.21.15.70:0/3262845597 shutdown_connections 2023-08-25T09:39:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.996+0000 7f03842ba640 1 -- 172.21.15.70:0/3262845597 wait complete. 2023-08-25T09:39:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.998+0000 7f03842ba640 1 Processor -- start 2023-08-25T09:39:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.998+0000 7f03842ba640 1 -- start start 2023-08-25T09:39:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.999+0000 7f03842ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 0x7f037c1f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.999+0000 7f03842ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f037c0ca780 0x7f037c0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.999+0000 7f03842ba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f037c15acd0 0x7f037c0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.999+0000 7f03842ba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c1f1ce0 con 0x7f037c0b3cf0 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.999+0000 7f03842ba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037c151160 con 0x7f037c15acd0 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:10.999+0000 7f03842ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037c151440 con 0x7f037c0ca780 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 0x7f037c1f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 0x7f037c1f17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35882/0 (socket says 172.21.15.70:35882) 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 -- 172.21.15.70:0/129275526 learned_addr learned my addr 172.21.15.70:0/129275526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 -- 172.21.15.70:0/129275526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f037c0ca780 msgr2=0x7f037c0ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f037c0ca780 0x7f037c0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 -- 172.21.15.70:0/129275526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f037c15acd0 msgr2=0x7f037c0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f037c15acd0 0x7f037c0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 -- 172.21.15.70:0/129275526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c009400 con 0x7f037c0b3cf0 2023-08-25T09:39:11.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0382056640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 0x7f037c1f17a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0378001eb0 tx=0x7f037800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.000+0000 7f0372ffd640 1 -- 172.21.15.70:0/129275526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780074e0 con 0x7f037c0b3cf0 2023-08-25T09:39:11.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.001+0000 7f0372ffd640 1 -- 172.21.15.70:0/129275526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378007640 con 0x7f037c0b3cf0 2023-08-25T09:39:11.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.001+0000 7f0372ffd640 1 -- 172.21.15.70:0/129275526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780059e0 con 0x7f037c0b3cf0 2023-08-25T09:39:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.001+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037c151720 con 0x7f037c0b3cf0 2023-08-25T09:39:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.001+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037c153ab0 con 0x7f037c0b3cf0 2023-08-25T09:39:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.003+0000 7f0372ffd640 1 -- 172.21.15.70:0/129275526 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0378005b40 con 0x7f037c0b3cf0 2023-08-25T09:39:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.004+0000 7f0372ffd640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f036006a050 0x7f036006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.004+0000 7f0372ffd640 1 -- 172.21.15.70:0/129275526 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0378087be0 con 0x7f037c0b3cf0 2023-08-25T09:39:11.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.004+0000 7f0381855640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f036006a050 0x7f036006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.005+0000 7f0386e2e740 1 Processor -- start 2023-08-25T09:39:11.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.005+0000 7f0386e2e740 1 -- start start 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.005+0000 7f0381855640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f036006a050 0x7f036006c510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f036c036000 tx=0x7f036c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418a4130 0x5612418d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0386e2e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x561241817190 con 0x5612418d4150 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x561241814880 con 0x5612418a4130 2023-08-25T09:39:11.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x561241810170 con 0x5612418a7cd0 2023-08-25T09:39:11.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35884/0 (socket says 172.21.15.70:35884) 2023-08-25T09:39:11.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 -- 172.21.15.70:0/3862421096 learned_addr learned my addr 172.21.15.70:0/3862421096 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 -- 172.21.15.70:0/3862421096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 msgr2=0x5612418d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 -- 172.21.15.70:0/3862421096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418a4130 msgr2=0x5612418d13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418a4130 0x5612418d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.006+0000 7f0382056640 1 -- 172.21.15.70:0/3862421096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56124180e7f0 con 0x5612418d4150 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0382056640 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d4530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f037805e820 tx=0x7f037805af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d890bc44aba791 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f035ffff640 1 -- 172.21.15.70:0/3862421096 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378055180 con 0x5612418d4150 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f035ffff640 1 -- 172.21.15.70:0/3862421096 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03780553e0 con 0x5612418d4150 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f035ffff640 1 -- 172.21.15.70:0/3862421096 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378052e30 con 0x5612418d4150 2023-08-25T09:39:11.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 -- 172.21.15.70:0/3862421096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 msgr2=0x5612418d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d4530 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f037805e820 tx=0x7f037805af40 comp rx=0 tx=0).stop 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 -- 172.21.15.70:0/3862421096 shutdown_connections 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418a4130 0x5612418d13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3862421096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d4530 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 -- 172.21.15.70:0/3862421096 >> 172.21.15.70:0/3862421096 conn(0x5612418a63e0 msgr2=0x5612418eef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.007+0000 7f0386e2e740 1 -- 172.21.15.70:0/3862421096 shutdown_connections 2023-08-25T09:39:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.008+0000 7f0386e2e740 1 -- 172.21.15.70:0/3862421096 wait complete. 2023-08-25T09:39:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.008+0000 7f0386e2e740 1 -- start start 2023-08-25T09:39:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.008+0000 7f0386e2e740 1 Processor -- start 2023-08-25T09:39:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.009+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.009+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.009+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x5612418cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.009+0000 7f0386e2e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612417afd20 con 0x5612418d4150 2023-08-25T09:39:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.009+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612417af9c0 con 0x5612418d4b20 2023-08-25T09:39:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.009+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x561241575f20 con 0x5612418a7cd0 2023-08-25T09:39:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0386e2e740 10 client.0 _ll_register_callbacks cb 0x5612417c7630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:39:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35886/0 (socket says 172.21.15.70:35886) 2023-08-25T09:39:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 -- 172.21.15.70:0/945109991 learned_addr learned my addr 172.21.15.70:0/945109991 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0382056640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0382857640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x5612418cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 -- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 msgr2=0x5612418ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418ef240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 -- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 msgr2=0x5612418cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x5612418cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 -- 172.21.15.70:0/945109991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037c153ab0 con 0x5612418d4150 2023-08-25T09:39:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f0381855640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d7670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f036c02b880 tx=0x7f036c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80e9e3d6f36a463 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.010+0000 7f035d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:39:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f035d7fa640 1 -- 172.21.15.70:0/945109991 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c004220 con 0x5612418d4150 2023-08-25T09:39:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612417e4540 con 0x5612418d4150 2023-08-25T09:39:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f035d7fa640 1 -- 172.21.15.70:0/945109991 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f036c004380 con 0x5612418d4150 2023-08-25T09:39:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f035d7fa640 1 -- 172.21.15.70:0/945109991 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f036c032e90 con 0x5612418d4150 2023-08-25T09:39:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 10 client.4677 fetch_fsmap learned FSMap version 8 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561241807f30 con 0x5612418d4150 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f035d7fa640 1 -- 172.21.15.70:0/945109991 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5612417e4540 con 0x5612418d4150 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f035d7fa640 1 -- 172.21.15.70:0/945109991 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f036c003610 con 0x5612418d4150 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 10 client.4677 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 20 client.4677 populate_metadata read hostname 'smithi070' 2023-08-25T09:39:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241902dd0 0x5612419031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 4 client.4677 mds_command: new command op to 4276 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:39:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.011+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 --> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561241809150 con 0x561241902dd0 2023-08-25T09:39:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.012+0000 7f0382056640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241902dd0 0x5612419031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.013+0000 7f0382056640 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241902dd0 0x5612419031b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21997d87419b03c9 server_cookie=6f1a6ebba44faee8 in_seq=0 out_seq=0 2023-08-25T09:39:11.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.013+0000 7f035d7fa640 10 client.4677 ms_handle_connect on v2:172.21.15.155:6834/2636978427 2023-08-25T09:39:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.015+0000 7f035d7fa640 1 -- 172.21.15.70:0/945109991 <== mds.0 v2:172.21.15.155:6834/2636978427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561241809150 con 0x561241902dd0 2023-08-25T09:39:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.015+0000 7f035d7fa640 10 client.4677 handle_command_reply: tid=0 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.015+0000 7f0386e2e740 1 client.4677 shutdown 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.015+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 msgr2=0x5612418d7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.015+0000 7f0386e2e740 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d7670 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f036c02b880 tx=0x7f036c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 shutdown_connections 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241902dd0 0x5612419031b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x5612418cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418d4150 0x5612418d7670 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 --2- 172.21.15.70:0/945109991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418a7cd0 0x5612418ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f035d7fa640 0 client.4677 ms_handle_reset on v2:172.21.15.155:6834/2636978427 2023-08-25T09:39:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 >> 172.21.15.70:0/945109991 conn(0x5612418a4130 msgr2=0x5612418a8580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 shutdown_connections 2023-08-25T09:39:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 1 -- 172.21.15.70:0/945109991 wait complete. 2023-08-25T09:39:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.016+0000 7f0386e2e740 20 client.4677 trim_cache size 0 max 16384 2023-08-25T09:39:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 Processor -- start 2023-08-25T09:39:11.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 -- start start 2023-08-25T09:39:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4150 0x561241909720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4b20 0x5612418cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612418cb190 con 0x5612418a7cd0 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612418d0510 con 0x5612418d4150 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.033+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5612418cb9d0 con 0x5612418d4b20 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35898/0 (socket says 172.21.15.70:35898) 2023-08-25T09:39:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 -- 172.21.15.70:0/3705843869 learned_addr learned my addr 172.21.15.70:0/3705843869 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 -- 172.21.15.70:0/3705843869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4b20 msgr2=0x5612418cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4b20 0x5612418cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 -- 172.21.15.70:0/3705843869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4150 msgr2=0x561241909720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4150 0x561241909720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 -- 172.21.15.70:0/3705843869 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612417aea10 con 0x5612418a7cd0 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0382056640 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419091e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0378001130 tx=0x7f0378014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b42550f52faec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0370e1b640 1 -- 172.21.15.70:0/3705843869 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378053750 con 0x5612418a7cd0 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0370e1b640 1 -- 172.21.15.70:0/3705843869 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03780538b0 con 0x5612418a7cd0 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.034+0000 7f0370e1b640 1 -- 172.21.15.70:0/3705843869 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780752c0 con 0x5612418a7cd0 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 -- 172.21.15.70:0/3705843869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 msgr2=0x5612419091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419091e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0378001130 tx=0x7f0378014000 comp rx=0 tx=0).stop 2023-08-25T09:39:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 -- 172.21.15.70:0/3705843869 shutdown_connections 2023-08-25T09:39:11.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4b20 0x5612418cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4150 0x561241909720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 --2- 172.21.15.70:0/3705843869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419091e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 -- 172.21.15.70:0/3705843869 >> 172.21.15.70:0/3705843869 conn(0x5612418a4130 msgr2=0x5612418d5610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 -- 172.21.15.70:0/3705843869 shutdown_connections 2023-08-25T09:39:11.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.035+0000 7f0386e2e740 1 -- 172.21.15.70:0/3705843869 wait complete. 2023-08-25T09:39:11.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.036+0000 7f0386e2e740 1 -- start start 2023-08-25T09:39:11.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.036+0000 7f0386e2e740 1 Processor -- start 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4150 0x5612419091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x561241909720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612418a9c40 con 0x5612418a7cd0 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612418a3cf0 con 0x5612418d4b20 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x56124195c150 con 0x5612418d4150 2023-08-25T09:39:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0386e2e740 10 client.0 _ll_register_callbacks cb 0x7f0378056e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35906/0 (socket says 172.21.15.70:35906) 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 -- 172.21.15.70:0/808412011 learned_addr learned my addr 172.21.15.70:0/808412011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 -- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4150 msgr2=0x5612419091e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4150 0x5612419091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 -- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 msgr2=0x561241909720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x561241909720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.037+0000 7f0382056640 1 -- 172.21.15.70:0/808412011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56124180e7f0 con 0x5612418a7cd0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0382056640 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419551f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0378001130 tx=0x7f037800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6d892fb8e05a76 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f035d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f035d7fa640 1 -- 172.21.15.70:0/808412011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378064070 con 0x5612418a7cd0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f035d7fa640 1 -- 172.21.15.70:0/808412011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0378055200 con 0x5612418a7cd0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03780553e0 con 0x5612418a7cd0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f035d7fa640 1 -- 172.21.15.70:0/808412011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0378053750 con 0x5612418a7cd0 2023-08-25T09:39:11.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 10 client.4683 fetch_fsmap learned FSMap version 8 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x5612417d3de0 con 0x5612418a7cd0 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f035d7fa640 1 -- 172.21.15.70:0/808412011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03780553e0 con 0x5612418a7cd0 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f035d7fa640 1 -- 172.21.15.70:0/808412011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0378067020 con 0x5612418a7cd0 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 10 client.4683 fetch_fsmap finished waiting for FSMap version 8 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 10 client.4683 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 20 client.4683 populate_metadata read hostname 'smithi070' 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241958020 0x561241958400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 4 client.4683 mds_command: new command op to 4276 tid=0[{"prefix": "status"}] 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.038+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 --> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5612414bf5e0 con 0x561241958020 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.039+0000 7f0381855640 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241958020 0x561241958400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.040+0000 7f035d7fa640 10 client.4683 ms_handle_connect on v2:172.21.15.155:6834/2636978427 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.040+0000 7f0381855640 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241958020 0x561241958400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8353b8c7ce4cc3 server_cookie=6d94c7ebcf5146de in_seq=0 out_seq=0 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f035d7fa640 1 -- 172.21.15.70:0/808412011 <== mds.0 v2:172.21.15.155:6834/2636978427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5612414bf5e0 con 0x561241958020 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f035d7fa640 10 client.4683 handle_command_reply: tid=0 2023-08-25T09:39:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 client.4683 shutdown 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 msgr2=0x5612419551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419551f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0378001130 tx=0x7f037800a000 comp rx=0 tx=0).stop 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 shutdown_connections 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:6834/2636978427,v1:172.21.15.155:6835/2636978427] conn(0x561241958020 0x561241958400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5612418d4b20 0x561241909720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5612418d4150 0x5612419091e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 --2- 172.21.15.70:0/808412011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5612418a7cd0 0x5612419551f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f035d7fa640 0 client.4683 ms_handle_reset on v2:172.21.15.155:6834/2636978427 2023-08-25T09:39:11.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 >> 172.21.15.70:0/808412011 conn(0x5612418a4130 msgr2=0x5612418d5610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 shutdown_connections 2023-08-25T09:39:11.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 1 -- 172.21.15.70:0/808412011 wait complete. 2023-08-25T09:39:11.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.041+0000 7f0386e2e740 20 client.4683 trim_cache size 0 max 16384 2023-08-25T09:39:11.050 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 4276, 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 17.856807067999998, 2023-08-25T09:39:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 8, 2023-08-25T09:39:11.052 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 23, 2023-08-25T09:39:11.052 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 23, 2023-08-25T09:39:11.052 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 23.784901119000001 2023-08-25T09:39:11.052 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:39:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f036006a050 msgr2=0x7f036006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f036006a050 0x7f036006c510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f036c036000 tx=0x7f036c030000 comp rx=0 tx=0).stop 2023-08-25T09:39:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 msgr2=0x7f037c1f17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 0x7f037c1f17a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0378001eb0 tx=0x7f037800c890 comp rx=0 tx=0).stop 2023-08-25T09:39:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 shutdown_connections 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f036006a050 0x7f036006c510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f037c15acd0 0x7f037c0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f037c0ca780 0x7f037c0ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 --2- 172.21.15.70:0/129275526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f037c0b3cf0 0x7f037c1f17a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 >> 172.21.15.70:0/129275526 conn(0x7f037c0ce760 msgr2=0x7f037c0cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 shutdown_connections 2023-08-25T09:39:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.042+0000 7f03842ba640 1 -- 172.21.15.70:0/129275526 wait complete. 2023-08-25T09:39:11.058 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-08-25T09:39:11.089 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-08-25T09:39:11.090 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-08-25T09:39:11.090 INFO:tasks.ceph_fuse:client.0 config is {} 2023-08-25T09:39:11.090 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T09:39:11.091 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:11.091 INFO:tasks.ceph_fuse:client.2 config is {} 2023-08-25T09:39:11.091 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T09:39:11.092 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:11.092 INFO:tasks.ceph_fuse:client.1 config is {} 2023-08-25T09:39:11.092 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T09:39:11.092 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:11.092 INFO:tasks.ceph_fuse:client.3 config is {} 2023-08-25T09:39:11.093 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T09:39:11.093 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:11.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:11.093 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:39:11.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:11.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-08-25T09:39:11.248 INFO:teuthology.orchestra.run.smithi070.stderr:Cannot find device "ceph-brx" 2023-08-25T09:39:11.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:39:11.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:11.253 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:39:11.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:11.318 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2023-08-25T09:39:11.420 INFO:teuthology.orchestra.run.smithi155.stderr:Cannot find device "ceph-brx" 2023-08-25T09:39:11.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:39:11.424 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-08-25T09:39:11.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:39:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.645+0000 7f849d890640 1 Processor -- start 2023-08-25T09:39:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.645+0000 7f849d890640 1 -- start start 2023-08-25T09:39:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.646+0000 7f849d890640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f849815e830 0x7f849806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.646+0000 7f849d890640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849806b2b0 0x7f849815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.646+0000 7f849d890640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84980b1b60 con 0x7f849815c6d0 2023-08-25T09:39:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.646+0000 7f849d890640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f849806bd80 con 0x7f849815e830 2023-08-25T09:39:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.646+0000 7f849d890640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849806b7c0 con 0x7f849806b2b0 2023-08-25T09:39:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.647+0000 7f8496ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f849815e830 0x7f849806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.647+0000 7f8496ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f849815e830 0x7f849806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:52848/0 (socket says 172.21.15.70:52848) 2023-08-25T09:39:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.647+0000 7f8496ffd640 1 -- 172.21.15.70:0/3810350960 learned_addr learned my addr 172.21.15.70:0/3810350960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.647+0000 7f848e7fc640 1 --2- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849806b2b0 0x7f849815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.648+0000 7f8496ffd640 1 -- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849806b2b0 msgr2=0x7f849815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:11.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.648+0000 7f8496ffd640 1 --2- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849806b2b0 0x7f849815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.648+0000 7f8496ffd640 1 -- 172.21.15.70:0/3810350960 >> v1:172.21.15.155:6789/0 conn(0x7f849815c6d0 legacy=0x7f84980b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.648+0000 7f8496ffd640 1 -- 172.21.15.70:0/3810350960 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849806baa0 con 0x7f849815e830 2023-08-25T09:39:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.649+0000 7f8496ffd640 1 --2- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f849806ad70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8488009870 tx=0x7f848802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76795173d3c1a94d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f84967fc640 1 -- 172.21.15.70:0/3810350960 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488030840 con 0x7f849815e830 2023-08-25T09:39:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f84967fc640 1 -- 172.21.15.70:0/3810350960 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84880309a0 con 0x7f849815e830 2023-08-25T09:39:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f84967fc640 1 -- 172.21.15.70:0/3810350960 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488030d30 con 0x7f849815e830 2023-08-25T09:39:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f849d890640 1 -- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 msgr2=0x7f849806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f849d890640 1 --2- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f849806ad70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8488009870 tx=0x7f848802b980 comp rx=0 tx=0).stop 2023-08-25T09:39:11.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f849d890640 1 -- 172.21.15.70:0/3810350960 shutdown_connections 2023-08-25T09:39:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f849d890640 1 --2- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849806b2b0 0x7f849815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f849d890640 1 --2- 172.21.15.70:0/3810350960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f849806ad70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.650+0000 7f849d890640 1 -- 172.21.15.70:0/3810350960 >> 172.21.15.70:0/3810350960 conn(0x7f8498069850 msgr2=0x7f8498069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.651+0000 7f849d890640 1 -- 172.21.15.70:0/3810350960 shutdown_connections 2023-08-25T09:39:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.651+0000 7f849d890640 1 -- 172.21.15.70:0/3810350960 wait complete. 2023-08-25T09:39:11.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.652+0000 7f849d890640 1 Processor -- start 2023-08-25T09:39:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.653+0000 7f849d890640 1 -- start start 2023-08-25T09:39:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.653+0000 7f849d890640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 0x7f84981574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.653+0000 7f849d890640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849815c6d0 0x7f84980b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.653+0000 7f849d890640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f84980b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f849d890640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84980cd570 con 0x7f849806b2b0 2023-08-25T09:39:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f849d890640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84981579e0 con 0x7f849815e830 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f849d890640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84980cccd0 con 0x7f849815c6d0 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 0x7f84981574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 0x7f84981574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35910/0 (socket says 172.21.15.70:35910) 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 -- 172.21.15.70:0/3211122906 learned_addr learned my addr 172.21.15.70:0/3211122906 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f848e7fc640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849815c6d0 0x7f84980b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f84977fe640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f84980b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 -- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849815c6d0 msgr2=0x7f84980b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849815c6d0 0x7f84980b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 -- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 msgr2=0x7f84980b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f84980b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f8496ffd640 1 -- 172.21.15.70:0/3211122906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880094d0 con 0x7f849806b2b0 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f84977fe640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f84980b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.654+0000 7f848e7fc640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849815c6d0 0x7f84980b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.655+0000 7f8496ffd640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 0x7f84981574a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8488000c00 tx=0x7f8488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.655+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488039a90 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.655+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488039bf0 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.655+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84980ccfb0 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.655+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488043410 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.655+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84981e4460 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.656+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84981e42c0 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.657+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f848802f390 con 0x7f849806b2b0 2023-08-25T09:39:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.658+0000 7f8494ff9640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847806a050 0x7f847806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:11.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.658+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84880aa870 con 0x7f849806b2b0 2023-08-25T09:39:11.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.658+0000 7f848e7fc640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847806a050 0x7f847806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:11.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.659+0000 7f848e7fc640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847806a050 0x7f847806c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84980b2bb0 tx=0x7f8480008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:11.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.662+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f84981e42c0 con 0x7f849806b2b0 2023-08-25T09:39:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.888+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84981e42c0 con 0x7f849806b2b0 2023-08-25T09:39:11.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.888+0000 7f8494ff9640 1 -- 172.21.15.70:0/3211122906 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f84981e42c0 con 0x7f849806b2b0 2023-08-25T09:39:11.890 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:39:11.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847806a050 msgr2=0x7f847806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847806a050 0x7f847806c510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84980b2bb0 tx=0x7f8480008040 comp rx=0 tx=0).stop 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 msgr2=0x7f84981574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 0x7f84981574a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8488000c00 tx=0x7f8488004730 comp rx=0 tx=0).stop 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 shutdown_connections 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f847806a050 0x7f847806c510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f849815e830 0x7f84980b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f849815c6d0 0x7f84980b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 --2- 172.21.15.70:0/3211122906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f849806b2b0 0x7f84981574a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 >> 172.21.15.70:0/3211122906 conn(0x7f8498069850 msgr2=0x7f8498159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 shutdown_connections 2023-08-25T09:39:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:11.891+0000 7f849d890640 1 -- 172.21.15.70:0/3211122906 wait complete. 2023-08-25T09:39:11.911 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:39:11.911 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:39:11.911 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:39:11.912 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:39:11.912 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:11.912 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:39:11.912 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:39:11.912 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:39:11.913 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:39:11.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:11.913 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:39:11.961 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:11.963 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:11.963 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:39:11.964 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:11.964 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:11.964 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:39:11.965 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:11.965 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:11.965 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:39:11.965 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:39:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42324sec preferred_lft 42324sec 2023-08-25T09:39:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:39:11.966 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:11.967 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T09:39:11.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:11.967 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:11.968 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:11.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add name ceph-brx type bridge 2023-08-25T09:39:11.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr flush dev ceph-brx 2023-08-25T09:39:11.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set ceph-brx up 2023-08-25T09:39:11.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T09:39:11.969 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:12.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:12.475 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:39:12.556 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:39:12.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:12.561 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:39:12.622 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:39:12.623 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:39:12.623 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:39:12.623 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:39:12.623 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:39:12.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:12.624 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:12.624 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:12.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:39:12.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:39:12.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:39:12.625 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:12.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:39:12.823 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:39:13.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:13.397 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:39:13.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:13.455 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-08-25T09:39:13.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:13.509 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:13.510 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:13.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:39:13.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-25T09:39:13.511 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:13.656 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-25T09:39:13.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:13.657 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:13.657 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:13.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-25T09:39:13.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T09:39:13.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-25T09:39:13.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-25T09:39:13.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-25T09:39:13.658 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:13.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:13.935 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:13.935 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:13.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:39:13.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T09:39:13.937 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:14.098 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:39:14.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:14.098 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:14.148 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:39:14.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:14.149 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:14.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:39:14.296 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:14.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:14.387 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:14.391 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:14.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:14.393 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:39:14.443 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:39:14.444 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:39:14.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:39:14.527 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:14.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:14.598 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:14.601 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:14.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:14.602 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:39:14.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:39:14.635+0000 7fd2ed60e480 -1 init, newargv = 0x560a5a6c9080 newargc=15 2023-08-25T09:39:14.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[89436]: starting ceph client 2023-08-25T09:39:14.648 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[89436]: starting fuse 2023-08-25T09:39:15.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:39:15.854 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:15.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:15.953 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:15.957 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:15.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:15.958 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:39:16.014 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:39:16.015 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:39:16.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:39:16.016 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:39:16.016 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:39:16.016 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:39:16.016 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:39:16.017 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:16.017 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:39:16.017 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:39:16.017 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:39:16.018 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:39:16.018 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:16.018 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:39:16.018 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:39:16.019 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:39:16.019 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:39:16.019 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:16.019 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:39:16.019 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:39:16.020 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:39:16.020 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:39:16.020 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:39:16.020 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:39:16.021 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:39:16.021 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:39:16.021 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:16.021 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:39:16.022 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:39:16.198 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.89436.asok 2023-08-25T09:39:16.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:39:16.215 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.89436.asok 2023-08-25T09:39:16.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:16.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.89436.asok status 2023-08-25T09:39:16.453 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:39:16.453 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:39:16.453 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:39:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:39:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:39:16.454 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:39:16.455 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:39:16.455 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "89436", 2023-08-25T09:39:16.456 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:39:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:39:16.457 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:39:16.458 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 4412, 2023-08-25T09:39:16.458 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:39:16.458 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:39:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:39:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 4412 2023-08-25T09:39:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:16.459 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:39:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:39:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-08-25T09:39:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1322434897 2023-08-25T09:39:16.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:39:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:39:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:39:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:0", 2023-08-25T09:39:16.461 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1322434897 2023-08-25T09:39:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.4412 172.21.15.70:0/1322434897", 2023-08-25T09:39:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "172.21.15.70:0/1322434897", 2023-08-25T09:39:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:39:16.462 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 8, 2023-08-25T09:39:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 23, 2023-08-25T09:39:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:39:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:39:16.463 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:39:16.464 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:39:16.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:39:16.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.675+0000 7f35dd799640 1 Processor -- start 2023-08-25T09:39:16.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.675+0000 7f35dd799640 1 -- start start 2023-08-25T09:39:16.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35dd799640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35d80b1ff0 0x7f35d80b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:16.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35dd799640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d80b1b60 con 0x7f35d815ec10 2023-08-25T09:39:16.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35dd799640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d806b7c0 con 0x7f35d815e830 2023-08-25T09:39:16.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35dd799640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d806bd80 con 0x7f35d80b1ff0 2023-08-25T09:39:16.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35cffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f35d815ec10 0x7f35d815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34400/0 (socket says 172.21.15.70:34400) 2023-08-25T09:39:16.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35cffff640 1 -- 172.21.15.70:0/2119613872 learned_addr learned my addr 172.21.15.70:0/2119613872 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:16.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35d77fe640 1 --2- 172.21.15.70:0/2119613872 >> v2:172.21.15.70:3300/0 conn(0x7f35d80b1ff0 0x7f35d80b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:16.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35d67fc640 1 -- 172.21.15.70:0/2119613872 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525948617 0 0) 0x7f35d80b1b60 con 0x7f35d815ec10 2023-08-25T09:39:16.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.677+0000 7f35d67fc640 1 -- 172.21.15.70:0/2119613872 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c4003520 con 0x7f35d815ec10 2023-08-25T09:39:16.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.678+0000 7f35d77fe640 1 -- 172.21.15.70:0/2119613872 >> v1:172.21.15.155:6789/0 conn(0x7f35d815e830 legacy=0x7f35d806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:16.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.678+0000 7f35d77fe640 1 -- 172.21.15.70:0/2119613872 >> v1:172.21.15.70:6789/0 conn(0x7f35d815ec10 legacy=0x7f35d815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:16.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.678+0000 7f35d77fe640 1 -- 172.21.15.70:0/2119613872 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d806baa0 con 0x7f35d80b1ff0 2023-08-25T09:39:16.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.679+0000 7f35d77fe640 1 --2- 172.21.15.70:0/2119613872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d80b1ff0 0x7f35d80b2440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009d90 tx=0x7f35d002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcbae1622f6e8b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.679+0000 7f35d67fc640 1 -- 172.21.15.70:0/2119613872 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d002cc60 con 0x7f35d80b1ff0 2023-08-25T09:39:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.679+0000 7f35d67fc640 1 -- 172.21.15.70:0/2119613872 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d002cdc0 con 0x7f35d80b1ff0 2023-08-25T09:39:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.679+0000 7f35d67fc640 1 -- 172.21.15.70:0/2119613872 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0034660 con 0x7f35d80b1ff0 2023-08-25T09:39:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 -- 172.21.15.70:0/2119613872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d80b1ff0 msgr2=0x7f35d80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 --2- 172.21.15.70:0/2119613872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d80b1ff0 0x7f35d80b2440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009d90 tx=0x7f35d002beb0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 -- 172.21.15.70:0/2119613872 shutdown_connections 2023-08-25T09:39:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 --2- 172.21.15.70:0/2119613872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d80b1ff0 0x7f35d80b2440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 -- 172.21.15.70:0/2119613872 >> 172.21.15.70:0/2119613872 conn(0x7f35d8069850 msgr2=0x7f35d8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 -- 172.21.15.70:0/2119613872 shutdown_connections 2023-08-25T09:39:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.680+0000 7f35dd799640 1 -- 172.21.15.70:0/2119613872 wait complete. 2023-08-25T09:39:16.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.681+0000 7f35dd799640 1 Processor -- start 2023-08-25T09:39:16.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.682+0000 7f35dd799640 1 -- start start 2023-08-25T09:39:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35dd799640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d80b1ff0 0x7f35d8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35dd799640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 0x7f35d8157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35dd799640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d815ec10 0x7f35d8157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35dd799640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d81e6690 con 0x7f35d815e830 2023-08-25T09:39:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35dd799640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d8158190 con 0x7f35d815ec10 2023-08-25T09:39:16.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35dd799640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80ca840 con 0x7f35d80b1ff0 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 0x7f35d8157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 0x7f35d8157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35926/0 (socket says 172.21.15.70:35926) 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 -- 172.21.15.70:0/2513439085 learned_addr learned my addr 172.21.15.70:0/2513439085 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35d6ffd640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d80b1ff0 0x7f35d8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35d77fe640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d815ec10 0x7f35d8157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 -- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d80b1ff0 msgr2=0x7f35d8159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:16.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d80b1ff0 0x7f35d8159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 -- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d815ec10 msgr2=0x7f35d8157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d815ec10 0x7f35d8157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.683+0000 7f35cffff640 1 -- 172.21.15.70:0/2513439085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009a70 con 0x7f35d815e830 2023-08-25T09:39:16.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.684+0000 7f35cffff640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 0x7f35d8157710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c410 tx=0x7f35c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.684+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0005ce0 con 0x7f35d815e830 2023-08-25T09:39:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.684+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0005e40 con 0x7f35d815e830 2023-08-25T09:39:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.684+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c00072d0 con 0x7f35d815e830 2023-08-25T09:39:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.684+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d81e6ab0 con 0x7f35d815e830 2023-08-25T09:39:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.684+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d81e5fd0 con 0x7f35d815e830 2023-08-25T09:39:16.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.685+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d81e67d0 con 0x7f35d815e830 2023-08-25T09:39:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.686+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f35c00074f0 con 0x7f35d815e830 2023-08-25T09:39:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.687+0000 7f35cf7fe640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:16.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.687+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35c0087a40 con 0x7f35d815e830 2023-08-25T09:39:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.688+0000 7f35d6ffd640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:16.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.688+0000 7f35d6ffd640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35c8004ea0 tx=0x7f35c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:16.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.692+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35d81e67d0 con 0x7f35d815e830 2023-08-25T09:39:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.921+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f35d81e67d0 con 0x7f35d815e830 2023-08-25T09:39:16.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.922+0000 7f35cf7fe640 1 -- 172.21.15.70:0/2513439085 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f35c0017060 con 0x7f35d815e830 2023-08-25T09:39:16.924 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:39:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 msgr2=0x7f35b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35c8004ea0 tx=0x7f35c8009040 comp rx=0 tx=0).stop 2023-08-25T09:39:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 msgr2=0x7f35d8157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 0x7f35d8157710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c410 tx=0x7f35c000c8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 shutdown_connections 2023-08-25T09:39:16.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d815ec10 0x7f35d8157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d815e830 0x7f35d8157710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 --2- 172.21.15.70:0/2513439085 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d80b1ff0 0x7f35d8159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:16.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 >> 172.21.15.70:0/2513439085 conn(0x7f35d8069850 msgr2=0x7f35d8158d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:16.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 shutdown_connections 2023-08-25T09:39:16.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:16.925+0000 7f35dd799640 1 -- 172.21.15.70:0/2513439085 wait complete. 2023-08-25T09:39:16.944 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:39:16.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:39:16.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:39:16.945 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-08-25T09:39:16.945 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:16.945 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:39:16.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:39:16.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:39:16.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-08-25T09:39:16.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:16.947 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:39:16.999 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:17.000 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:17.000 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:17.000 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.000 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:17.000 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.001 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:17.001 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:17.001 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:39:17.001 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:17.001 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:17.001 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42319sec preferred_lft 42319sec 2023-08-25T09:39:17.002 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:39:17.003 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.004 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:39:17.004 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:39:17.004 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:39:17.004 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.006 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:39:17.099 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:39:17.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.104 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:39:17.159 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:39:17.159 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:39:17.159 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:39:17.160 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:39:17.160 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:39:17.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.161 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:17.162 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:17.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:39:17.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:39:17.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:39:17.163 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:17.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.345 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:39:17.395 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:39:17.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.396 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list-id 2023-08-25T09:39:17.446 INFO:teuthology.orchestra.run.smithi070.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-08-25T09:39:17.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.448 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:17.448 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:17.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:39:17.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-08-25T09:39:17.449 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:17.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-25T09:39:17.668 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:39:17.669 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 12:be:d6:d1:2f:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T09:39:17.670 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T09:39:17.670 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.670 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::10be:d6ff:fed1:2fe6/64 scope link 2023-08-25T09:39:17.670 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-25T09:39:17.760 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:39:17.761 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 12:be:d6:d1:2f:e6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T09:39:17.762 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T09:39:17.762 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.762 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::10be:d6ff:fed1:2fe6/64 scope link 2023-08-25T09:39:17.762 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:17.764 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-08-25T09:39:17.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:17.764 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:17.765 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:17.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-08-25T09:39:17.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-25T09:39:17.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-08-25T09:39:17.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-08-25T09:39:17.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-08-25T09:39:17.766 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:18.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:18.015 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:39:18.015 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:39:18.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.1 up 2023-08-25T09:39:18.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set dev brx.1 master ceph-brx 2023-08-25T09:39:18.016 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:39:18.183 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-08-25T09:39:18.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:18.184 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:18.233 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-08-25T09:39:18.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:18.234 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:18.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:39:18.379 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:18.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:18.468 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:18.472 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:18.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:18.473 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:39:18.528 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:39:18.529 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-08-25T09:39:18.529 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-08-25T09:39:18.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:39:18.614 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:18.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:18.684 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:18.686 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:18.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:18.687 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:39:18.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi070.stderr:2023-08-25T09:39:18.722+0000 7fda67e3f480 -1 init, newargv = 0x55675fcb5430 newargc=15 2023-08-25T09:39:18.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi070.stderr:ceph-fuse[90000]: starting ceph client 2023-08-25T09:39:18.730 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:39:18.731 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi070.stderr:ceph-fuse[90000]: starting fuse 2023-08-25T09:39:19.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:39:19.936 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:19.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:20.032 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:20.035 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:20.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:20.036 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:39:20.087 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:39:20.087 INFO:teuthology.orchestra.run.smithi070.stdout:41 2023-08-25T09:39:20.087 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 41] 2023-08-25T09:39:20.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:39:20.088 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:39:20.088 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:39:20.088 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:39:20.088 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:39:20.089 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:20.089 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:39:20.089 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-08-25T09:39:20.089 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:39:20.090 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-08-25T09:39:20.090 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:20.090 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:39:20.090 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:39:20.090 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:39:20.091 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:39:20.091 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:20.091 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:39:20.091 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:39:20.092 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:39:20.092 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:39:20.092 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:39:20.092 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:39:20.093 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:39:20.093 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:39:20.093 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:39:20.093 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.2")) 2023-08-25T09:39:20.093 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:39:20.269 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.2.90000.asok 2023-08-25T09:39:20.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:39:20.287 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.90000.asok 2023-08-25T09:39:20.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:20.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.90000.asok status 2023-08-25T09:39:20.528 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:39:20.529 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:39:20.529 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:39:20.529 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:39:20.529 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "2", 2023-08-25T09:39:20.529 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:39:20.529 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-08-25T09:39:20.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "90000", 2023-08-25T09:39:20.530 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:39:20.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:20.530 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:39:20.530 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:39:20.531 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 4707, 2023-08-25T09:39:20.531 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:39:20.531 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:39:20.531 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:39:20.531 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 4707 2023-08-25T09:39:20.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.2:0", 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3053974920 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:20.532 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:39:20.533 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:39:20.533 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.2:0", 2023-08-25T09:39:20.533 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3053974920 2023-08-25T09:39:20.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:39:20.533 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.4707 192.168.0.2:0/3053974920", 2023-08-25T09:39:20.533 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.2:0/3053974920", 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 8, 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 23, 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:39:20.534 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:39:20.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:39:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.756+0000 7f4d6ae83640 1 Processor -- start 2023-08-25T09:39:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.756+0000 7f4d6ae83640 1 -- start start 2023-08-25T09:39:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.758+0000 7f4d6ae83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 0x7f4d640ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.758+0000 7f4d6ae83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d641615b0 con 0x7f4d640b2b40 2023-08-25T09:39:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.758+0000 7f4d6ae83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6406f050 con 0x7f4d640ac870 2023-08-25T09:39:20.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.758+0000 7f4d6ae83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6406f330 con 0x7f4d6415cb40 2023-08-25T09:39:20.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.759+0000 7f4d69420640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4d640b2b40 0x7f4d64160ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51108/0 (socket says 172.21.15.70:51108) 2023-08-25T09:39:20.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.759+0000 7f4d69420640 1 -- 172.21.15.70:0/82047639 learned_addr learned my addr 172.21.15.70:0/82047639 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:20.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.759+0000 7f4d68c1f640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 0x7f4d640ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.759+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010970955 0 0) 0x7f4d641615b0 con 0x7f4d640b2b40 2023-08-25T09:39:20.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.759+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d54003520 con 0x7f4d640b2b40 2023-08-25T09:39:20.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190952304 0 0) 0x7f4d54003520 con 0x7f4d640b2b40 2023-08-25T09:39:20.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 msgr2=0x7f4d640ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:20.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 0x7f4d640ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 >> v1:172.21.15.155:6789/0 conn(0x7f4d640ac870 legacy=0x7f4d64160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6406f610 con 0x7f4d640b2b40 2023-08-25T09:39:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4d5c0019f0 con 0x7f4d640b2b40 2023-08-25T09:39:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 >> v1:172.21.15.70:6789/0 conn(0x7f4d640b2b40 legacy=0x7f4d64160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 0x7f4d540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54003ee0 0x7f4d540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d54004d00 0x7f4d54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6406f330 con 0x7f4d54004d00 2023-08-25T09:39:20.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5400b9e0 con 0x7f4d54003ee0 2023-08-25T09:39:20.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5400bba0 con 0x7f4d6415cb40 2023-08-25T09:39:20.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d69420640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d54004d00 0x7f4d54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d68c1f640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 0x7f4d540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.760+0000 7f4d63fff640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54003ee0 0x7f4d540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.761+0000 7f4d69420640 1 -- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 msgr2=0x7f4d540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:20.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.761+0000 7f4d69420640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415cb40 0x7f4d540039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:20.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.761+0000 7f4d69420640 1 -- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54003ee0 msgr2=0x7f4d540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:20.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.761+0000 7f4d69420640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54003ee0 0x7f4d540047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:20.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.761+0000 7f4d69420640 1 -- 172.21.15.70:0/82047639 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54003520 con 0x7f4d54004d00 2023-08-25T09:39:20.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.762+0000 7f4d69420640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d54004d00 0x7f4d54005180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4d640b3830 tx=0x7f4d540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fdfa431e7df9e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:20.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.762+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c034630 con 0x7f4d54004d00 2023-08-25T09:39:20.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.762+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c02f070 con 0x7f4d54004d00 2023-08-25T09:39:20.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.762+0000 7f4d637fe640 1 -- 172.21.15.70:0/82047639 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c034a30 con 0x7f4d54004d00 2023-08-25T09:39:20.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d6ae83640 1 -- 172.21.15.70:0/82047639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d54004d00 msgr2=0x7f4d54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:20.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/82047639 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d54004d00 0x7f4d54005180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4d640b3830 tx=0x7f4d540057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:20.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d69420640 1 -- 172.21.15.70:0/82047639 reap_dead start 2023-08-25T09:39:20.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d6ae83640 1 -- 172.21.15.70:0/82047639 shutdown_connections 2023-08-25T09:39:20.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d6ae83640 1 -- 172.21.15.70:0/82047639 >> 172.21.15.70:0/82047639 conn(0x7f4d64069850 msgr2=0x7f4d64069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:20.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d6ae83640 1 -- 172.21.15.70:0/82047639 shutdown_connections 2023-08-25T09:39:20.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.763+0000 7f4d6ae83640 1 -- 172.21.15.70:0/82047639 wait complete. 2023-08-25T09:39:20.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.765+0000 7f4d6ae83640 1 Processor -- start 2023-08-25T09:39:20.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.765+0000 7f4d6ae83640 1 -- start start 2023-08-25T09:39:20.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d6ae83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54004d00 0x7f4d641474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d6ae83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415a1c0 0x7f4d640c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d6ae83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 0x7f4d641583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d6ae83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d640c3fb0 con 0x7f4d640c3b00 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d6ae83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d641479e0 con 0x7f4d54004d00 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d6ae83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64159120 con 0x7f4d6415a1c0 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d69420640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 0x7f4d641583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d69420640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 0x7f4d641583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43184/0 (socket says 172.21.15.70:43184) 2023-08-25T09:39:20.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d69420640 1 -- 172.21.15.70:0/409581250 learned_addr learned my addr 172.21.15.70:0/409581250 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:20.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d68c1f640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54004d00 0x7f4d641474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.766+0000 7f4d63fff640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415a1c0 0x7f4d640c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d69420640 1 -- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415a1c0 msgr2=0x7f4d640c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:20.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d69420640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415a1c0 0x7f4d640c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:20.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d69420640 1 -- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54004d00 msgr2=0x7f4d641474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:20.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d69420640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54004d00 0x7f4d641474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:20.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d69420640 1 -- 172.21.15.70:0/409581250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64159400 con 0x7f4d640c3b00 2023-08-25T09:39:20.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d69420640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 0x7f4d641583b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00a920 tx=0x7f4d5c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:20.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c001a80 con 0x7f4d640c3b00 2023-08-25T09:39:20.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c034640 con 0x7f4d640c3b00 2023-08-25T09:39:20.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d641596e0 con 0x7f4d640c3b00 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c0349f0 con 0x7f4d640c3b00 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.767+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d64159ca0 con 0x7f4d640c3b00 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.769+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6404e420 con 0x7f4d640c3b00 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.769+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4d5c031030 con 0x7f4d640c3b00 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.770+0000 7f4d617fa640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d4806a050 0x7f4d4806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.770+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d5c0aa860 con 0x7f4d640c3b00 2023-08-25T09:39:20.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.771+0000 7f4d68c1f640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d4806a050 0x7f4d4806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:20.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.771+0000 7f4d68c1f640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d4806a050 0x7f4d4806c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009b00 tx=0x7f4d58008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:20.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:20.775+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4d5c075fd0 con 0x7f4d640c3b00 2023-08-25T09:39:21.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.002+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d6404e420 con 0x7f4d640c3b00 2023-08-25T09:39:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.003+0000 7f4d617fa640 1 -- 172.21.15.70:0/409581250 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d5c0795f0 con 0x7f4d640c3b00 2023-08-25T09:39:21.004 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:39:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.005+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d4806a050 msgr2=0x7f4d4806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.005+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d4806a050 0x7f4d4806c510 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009b00 tx=0x7f4d58008040 comp rx=0 tx=0).stop 2023-08-25T09:39:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 msgr2=0x7f4d641583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 0x7f4d641583b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c00a920 tx=0x7f4d5c001d80 comp rx=0 tx=0).stop 2023-08-25T09:39:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 shutdown_connections 2023-08-25T09:39:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d4806a050 0x7f4d4806c510 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d640c3b00 0x7f4d641583b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d6415a1c0 0x7f4d640c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 --2- 172.21.15.70:0/409581250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d54004d00 0x7f4d641474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 >> 172.21.15.70:0/409581250 conn(0x7f4d64069850 msgr2=0x7f4d6406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 shutdown_connections 2023-08-25T09:39:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:21.006+0000 7f4d6ae83640 1 -- 172.21.15.70:0/409581250 wait complete. 2023-08-25T09:39:21.025 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:39:21.026 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:39:21.026 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:39:21.026 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:39:21.027 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:21.027 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:39:21.028 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:39:21.028 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:39:21.028 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:39:21.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:21.029 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:39:21.092 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:21.092 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:21.092 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:39:21.093 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:21.094 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:39:21.095 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42328sec preferred_lft 42328sec 2023-08-25T09:39:21.095 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:39:21.095 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:21.095 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T09:39:21.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:21.096 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:21.096 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:21.096 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add name ceph-brx type bridge 2023-08-25T09:39:21.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr flush dev ceph-brx 2023-08-25T09:39:21.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set ceph-brx up 2023-08-25T09:39:21.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T09:39:21.097 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:21.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:21.600 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:39:21.681 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:39:21.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:21.685 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:39:21.742 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:39:21.742 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:39:21.743 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:39:21.743 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:39:21.743 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:39:21.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:21.744 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:21.744 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:21.744 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:39:21.745 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:39:21.745 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:39:21.745 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:22.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:22.565 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:39:22.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:22.622 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2023-08-25T09:39:22.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:22.669 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:22.669 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:22.669 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:39:22.670 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-08-25T09:39:22.670 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:22.860 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-08-25T09:39:22.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:22.861 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:22.862 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:22.862 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-08-25T09:39:22.862 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T09:39:22.862 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-08-25T09:39:22.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-08-25T09:39:22.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-08-25T09:39:22.863 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:23.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:23.161 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:23.161 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:23.162 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:39:23.162 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T09:39:23.163 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:23.332 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:39:23.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:23.333 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:23.377 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:39:23.377 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:23.378 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:23.476 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:39:23.563 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:23.563 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:23.652 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:23.655 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:23.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:23.657 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:39:23.710 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:39:23.711 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:39:23.717 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:39:23.795 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:23.796 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:23.867 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:23.870 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:23.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:23.871 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:39:23.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:39:23.905+0000 7f376f637480 -1 init, newargv = 0x55b6d0ee7fd0 newargc=15 2023-08-25T09:39:23.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[86634]: starting ceph client 2023-08-25T09:39:23.916 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[86634]: starting fuse 2023-08-25T09:39:25.017 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:39:25.126 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:25.126 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:25.220 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:25.224 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:25.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:25.225 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:39:25.275 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:39:25.276 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:39:25.276 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:39:25.277 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:39:25.277 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:39:25.277 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:39:25.278 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:39:25.278 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:25.278 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:39:25.278 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:39:25.279 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:39:25.279 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:39:25.279 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:25.279 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:39:25.280 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:39:25.280 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:39:25.280 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:39:25.280 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:25.280 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:39:25.281 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:39:25.281 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:39:25.281 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:39:25.281 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:39:25.282 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:39:25.282 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:39:25.282 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:39:25.282 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:25.282 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:39:25.283 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:39:25.454 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.86634.asok 2023-08-25T09:39:25.457 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:39:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:39:25.470 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.86634.asok 2023-08-25T09:39:25.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:25.471 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.86634.asok status 2023-08-25T09:39:25.712 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:39:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:39:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:39:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:39:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:39:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:39:25.713 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "86634", 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4427, 2023-08-25T09:39:25.714 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:39:25.715 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:39:25.715 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:39:25.715 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4427 2023-08-25T09:39:25.715 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:25.715 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:39:25.715 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:39:25.716 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:39:25.716 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 414963731 2023-08-25T09:39:25.716 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:39:25.716 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:25.716 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:39:25.716 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:39:25.717 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:39:25.717 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 414963731 2023-08-25T09:39:25.717 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:25.717 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4427 192.168.0.1:0/414963731", 2023-08-25T09:39:25.717 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/414963731", 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 8, 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 23, 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:39:25.718 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:39:25.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:39:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.949+0000 7f8980f12640 1 Processor -- start 2023-08-25T09:39:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.949+0000 7f8980f12640 1 -- start start 2023-08-25T09:39:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.950+0000 7f8980f12640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f897c0ce5b0 0x7f897c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.950+0000 7f8980f12640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f897c0b2850 0x7f897c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.950+0000 7f8980f12640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c0ceed0 con 0x7f897c15aa10 2023-08-25T09:39:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.950+0000 7f8980f12640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c159d10 con 0x7f897c0ce5b0 2023-08-25T09:39:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.950+0000 7f8980f12640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c15d3f0 con 0x7f897c0b2850 2023-08-25T09:39:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.950+0000 7f897a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f897c0ce5b0 0x7f897c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f8971d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f897c15aa10 0x7f897c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51116/0 (socket says 172.21.15.70:51116) 2023-08-25T09:39:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f897c0ce5b0 0x7f897c0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43186/0 (socket says 172.21.15.70:43186) 2023-08-25T09:39:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f8971d9b640 1 -- 172.21.15.70:0/4042208170 learned_addr learned my addr 172.21.15.70:0/4042208170 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897ad9d640 1 --2- 172.21.15.70:0/4042208170 >> v2:172.21.15.155:3300/0 conn(0x7f897c0b2850 0x7f897c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897a59c640 1 -- 172.21.15.70:0/4042208170 >> v2:172.21.15.155:3300/0 conn(0x7f897c0b2850 msgr2=0x7f897c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897a59c640 1 --2- 172.21.15.70:0/4042208170 >> v2:172.21.15.155:3300/0 conn(0x7f897c0b2850 0x7f897c06f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897a59c640 1 -- 172.21.15.70:0/4042208170 >> v1:172.21.15.70:6789/0 conn(0x7f897c15aa10 legacy=0x7f897c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897a59c640 1 -- 172.21.15.70:0/4042208170 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0b2d20 con 0x7f897c0ce5b0 2023-08-25T09:39:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f8979d9b640 1 -- 172.21.15.70:0/4042208170 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426637614 0 0) 0x7f897c0ceed0 con 0x7f897c15aa10 2023-08-25T09:39:25.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.951+0000 7f897ad9d640 1 --2- 172.21.15.70:0/4042208170 >> v2:172.21.15.155:3300/0 conn(0x7f897c0b2850 0x7f897c06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.952+0000 7f897a59c640 1 --2- 172.21.15.70:0/4042208170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ce990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f897c06a730 tx=0x7f896c02e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac293fb3df8843d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.952+0000 7f8979d9b640 1 -- 172.21.15.70:0/4042208170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c02edc0 con 0x7f897c0ce5b0 2023-08-25T09:39:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.952+0000 7f8979d9b640 1 -- 172.21.15.70:0/4042208170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896c036440 con 0x7f897c0ce5b0 2023-08-25T09:39:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.952+0000 7f8979d9b640 1 -- 172.21.15.70:0/4042208170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c0367d0 con 0x7f897c0ce5b0 2023-08-25T09:39:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 -- 172.21.15.70:0/4042208170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 msgr2=0x7f897c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 --2- 172.21.15.70:0/4042208170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ce990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f897c06a730 tx=0x7f896c02e010 comp rx=0 tx=0).stop 2023-08-25T09:39:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 -- 172.21.15.70:0/4042208170 shutdown_connections 2023-08-25T09:39:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 --2- 172.21.15.70:0/4042208170 >> v2:172.21.15.155:3300/0 conn(0x7f897c0b2850 0x7f897c06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 --2- 172.21.15.70:0/4042208170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ce990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 -- 172.21.15.70:0/4042208170 >> 172.21.15.70:0/4042208170 conn(0x7f897c069850 msgr2=0x7f897c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 -- 172.21.15.70:0/4042208170 shutdown_connections 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.953+0000 7f8980f12640 1 -- 172.21.15.70:0/4042208170 wait complete. 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.955+0000 7f8980f12640 1 Processor -- start 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.955+0000 7f8980f12640 1 -- start start 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8980f12640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c0b2850 0x7f897c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8980f12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8980f12640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c15aa10 0x7f897c0ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8980f12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b0ef0 con 0x7f897c0ce5b0 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8980f12640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b0320 con 0x7f897c0b2850 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8980f12640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0b05c0 con 0x7f897c15aa10 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8971d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43202/0 (socket says 172.21.15.70:43202) 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.956+0000 7f8971d9b640 1 -- 172.21.15.70:0/3926742201 learned_addr learned my addr 172.21.15.70:0/3926742201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f897a59c640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c0b2850 0x7f897c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8971d9b640 1 -- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c15aa10 msgr2=0x7f897c0ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8971d9b640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c15aa10 0x7f897c0ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8971d9b640 1 -- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c0b2850 msgr2=0x7f897c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8971d9b640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c0b2850 0x7f897c155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8971d9b640 1 -- 172.21.15.70:0/3926742201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c00b6a0 con 0x7f897c0ce5b0 2023-08-25T09:39:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8971d9b640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ae2c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f896400c430 tx=0x7f896400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:25.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640074e0 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964007660 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964004a20 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c0b08a0 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.957+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c152b60 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.958+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c152d80 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.959+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8964004b80 con 0x7f897c0ce5b0 2023-08-25T09:39:25.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.960+0000 7f89737fe640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.960+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8964088450 con 0x7f897c0ce5b0 2023-08-25T09:39:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.961+0000 7f897a59c640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.961+0000 7f897a59c640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f896c038000 tx=0x7f896c032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:25.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:25.965+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8964053aa0 con 0x7f897c0ce5b0 2023-08-25T09:39:26.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.191+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f897c04e420 con 0x7f897c0ce5b0 2023-08-25T09:39:26.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.192+0000 7f89737fe640 1 -- 172.21.15.70:0/3926742201 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f89640570c0 con 0x7f897c0ce5b0 2023-08-25T09:39:26.194 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 msgr2=0x7f896006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f896c038000 tx=0x7f896c032000 comp rx=0 tx=0).stop 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 msgr2=0x7f897c0ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ae2c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f896400c430 tx=0x7f896400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 shutdown_connections 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:26.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c15aa10 0x7f897c0ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0ce5b0 0x7f897c0ae2c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 --2- 172.21.15.70:0/3926742201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c0b2850 0x7f897c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:26.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 >> 172.21.15.70:0/3926742201 conn(0x7f897c069850 msgr2=0x7f897c070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:26.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 shutdown_connections 2023-08-25T09:39:26.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:26.194+0000 7f8980f12640 1 -- 172.21.15.70:0/3926742201 wait complete. 2023-08-25T09:39:26.216 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:39:26.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:39:26.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:39:26.217 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-08-25T09:39:26.217 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:26.217 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:39:26.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:39:26.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:39:26.218 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-08-25T09:39:26.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.218 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:39:26.279 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:26.279 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:26.280 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:39:26.281 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42323sec preferred_lft 42323sec 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:39:26.282 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.283 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:39:26.283 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.283 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:39:26.283 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:39:26.283 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:39:26.283 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.284 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:39:26.376 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:39:26.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.381 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:39:26.439 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:39:26.440 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:39:26.440 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:39:26.440 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:39:26.441 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:39:26.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.442 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:26.442 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:26.442 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:39:26.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:39:26.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:39:26.443 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:26.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.628 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:39:26.680 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:39:26.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.681 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list-id 2023-08-25T09:39:26.734 INFO:teuthology.orchestra.run.smithi155.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-08-25T09:39:26.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.735 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:26.735 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:26.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:39:26.735 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-08-25T09:39:26.735 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:26.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.884 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-25T09:39:26.972 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:26.973 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:26.973 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:26.973 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.973 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:26.973 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.974 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:39:26.974 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 16:ca:ba:1a:d3:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T09:39:26.974 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T09:39:26.974 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.974 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::14ca:baff:fe1a:d3ae/64 scope link 2023-08-25T09:39:26.974 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:26.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:26.977 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-25T09:39:27.072 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:39:27.072 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:39:27.072 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:39:27.073 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:27.073 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:39:27.073 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:27.073 INFO:teuthology.orchestra.run.smithi155.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:39:27.073 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 16:ca:ba:1a:d3:ae brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T09:39:27.074 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T09:39:27.074 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:27.074 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::14ca:baff:fe1a:d3ae/64 scope link 2023-08-25T09:39:27.074 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:39:27.075 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-08-25T09:39:27.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:27.076 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:27.076 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:27.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-08-25T09:39:27.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-25T09:39:27.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-08-25T09:39:27.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-08-25T09:39:27.077 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-08-25T09:39:27.078 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:27.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:27.356 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:39:27.357 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:39:27.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.1 up 2023-08-25T09:39:27.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set dev brx.1 master ceph-brx 2023-08-25T09:39:27.357 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:39:27.528 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-08-25T09:39:27.528 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:27.528 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:27.575 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-08-25T09:39:27.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:39:27.576 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:27.675 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:39:27.766 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:27.767 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:27.855 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:27.859 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:27.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:27.860 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:39:27.915 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:39:27.916 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-08-25T09:39:27.916 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-08-25T09:39:27.921 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:39:28.001 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:28.001 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:28.075 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:28.078 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:28.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:28.079 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:39:28.118 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:2023-08-25T09:39:28.117+0000 7f88a0829480 -1 init, newargv = 0x55a5259acc70 newargc=15 2023-08-25T09:39:28.119 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:39:28.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[87155]: starting ceph client 2023-08-25T09:39:28.126 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi155.stderr:ceph-fuse[87155]: starting fuse 2023-08-25T09:39:29.221 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:39:29.330 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:39:29.331 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:39:29.424 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:39:29.427 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:39:29.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:29.429 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:39:29.482 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:39:29.483 INFO:teuthology.orchestra.run.smithi155.stdout:41 2023-08-25T09:39:29.483 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 41] 2023-08-25T09:39:29.483 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:39:29.484 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:39:29.484 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:39:29.484 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:39:29.484 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:39:29.485 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:29.485 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:39:29.488 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-08-25T09:39:29.488 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:39:29.489 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-08-25T09:39:29.489 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:29.490 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:39:29.490 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:39:29.491 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:39:29.491 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:39:29.491 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:29.492 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:39:29.492 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:39:29.492 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:39:29.492 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:39:29.493 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:39:29.493 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:39:29.493 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:39:29.494 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:39:29.495 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:39:29.495 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.3")) 2023-08-25T09:39:29.496 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:39:29.676 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.3.87155.asok 2023-08-25T09:39:29.681 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:39:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:39:29.694 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.87155.asok 2023-08-25T09:39:29.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:29.695 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.87155.asok status 2023-08-25T09:39:29.939 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:39:29.940 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:39:29.940 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:39:29.940 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:39:29.941 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "3", 2023-08-25T09:39:29.941 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:39:29.941 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-08-25T09:39:29.942 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "87155", 2023-08-25T09:39:29.942 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:39:29.942 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:29.942 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:39:29.943 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:39:29.943 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4525, 2023-08-25T09:39:29.943 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:39:29.943 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:39:29.944 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:39:29.944 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4525 2023-08-25T09:39:29.944 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:29.944 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:39:29.944 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1946698184 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:39:29.945 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.2:0", 2023-08-25T09:39:29.946 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1946698184 2023-08-25T09:39:29.946 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:39:29.946 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4525 192.168.0.2:0/1946698184", 2023-08-25T09:39:29.946 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.2:0/1946698184", 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 8, 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 23, 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:39:29.947 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:39:29.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:29.954 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:39:29.954 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:30.015 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:39:30.016 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:30.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:30.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.116 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:39:30.116 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:30.172 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:39:30.173 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:30.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-08-25T09:39:30.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.274 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:39:30.275 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:30.336 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:39:30.336 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:30.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.337 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:30.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.430 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:39:30.431 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:30.481 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:39:30.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:30.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:30.482 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-08-25T09:39:30.559 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-08-25T09:39:30.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.799+0000 7f0975652640 1 Processor -- start 2023-08-25T09:39:30.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.799+0000 7f0975652640 1 -- start start 2023-08-25T09:39:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.800+0000 7f0975652640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09700b12f0 0x7f097015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.800+0000 7f0975652640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015f190 0x7f09700cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.800+0000 7f0975652640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097015e7c0 con 0x7f097015a7c0 2023-08-25T09:39:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0975652640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0970071b50 con 0x7f09700b12f0 2023-08-25T09:39:30.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0975652640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097015e490 con 0x7f097015f190 2023-08-25T09:39:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f096ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f097015a7c0 0x7f09700b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50038/0 (socket says 172.21.15.70:50038) 2023-08-25T09:39:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f096ffff640 1 -- 172.21.15.70:0/617603330 learned_addr learned my addr 172.21.15.70:0/617603330 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0967fff640 1 --2- 172.21.15.70:0/617603330 >> v2:172.21.15.70:3300/0 conn(0x7f09700b12f0 0x7f097015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:30.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0967fff640 1 -- 172.21.15.70:0/617603330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015f190 msgr2=0x7f09700cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0974e51640 1 --2- 172.21.15.70:0/617603330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015f190 0x7f09700cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0967fff640 1 --2- 172.21.15.70:0/617603330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015f190 0x7f09700cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0967fff640 1 -- 172.21.15.70:0/617603330 >> v1:172.21.15.70:6789/0 conn(0x7f097015a7c0 legacy=0x7f09700b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f0967fff640 1 -- 172.21.15.70:0/617603330 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700b3d60 con 0x7f09700b12f0 2023-08-25T09:39:30.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.801+0000 7f096f7fe640 1 -- 172.21.15.70:0/617603330 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979066084 0 0) 0x7f097015e7c0 con 0x7f097015a7c0 2023-08-25T09:39:30.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.802+0000 7f0967fff640 1 --2- 172.21.15.70:0/617603330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015ec50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09580097a0 tx=0x7f095802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db49c6baa408947b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:30.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f096f7fe640 1 -- 172.21.15.70:0/617603330 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095802d040 con 0x7f09700b12f0 2023-08-25T09:39:30.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f096f7fe640 1 -- 172.21.15.70:0/617603330 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0958031810 con 0x7f09700b12f0 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f096f7fe640 1 -- 172.21.15.70:0/617603330 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0958031ba0 con 0x7f09700b12f0 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 -- 172.21.15.70:0/617603330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 msgr2=0x7f097015ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 --2- 172.21.15.70:0/617603330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015ec50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f09580097a0 tx=0x7f095802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 -- 172.21.15.70:0/617603330 shutdown_connections 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 --2- 172.21.15.70:0/617603330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015ec50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 --2- 172.21.15.70:0/617603330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015f190 0x7f09700cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 -- 172.21.15.70:0/617603330 >> 172.21.15.70:0/617603330 conn(0x7f09700affd0 msgr2=0x7f09700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:30.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.803+0000 7f0975652640 1 -- 172.21.15.70:0/617603330 shutdown_connections 2023-08-25T09:39:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.804+0000 7f0975652640 1 -- 172.21.15.70:0/617603330 wait complete. 2023-08-25T09:39:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.805+0000 7f0975652640 1 Processor -- start 2023-08-25T09:39:30.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.805+0000 7f0975652640 1 -- start start 2023-08-25T09:39:30.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.806+0000 7f0975652640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.806+0000 7f0975652640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015a7c0 0x7f09701f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.806+0000 7f0975652640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f097015f190 0x7f09701f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.806+0000 7f0975652640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970070410 con 0x7f09700b12f0 2023-08-25T09:39:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.806+0000 7f0975652640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097006f800 con 0x7f097015f190 2023-08-25T09:39:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.806+0000 7f0975652640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097006fae0 con 0x7f097015a7c0 2023-08-25T09:39:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47342/0 (socket says 172.21.15.70:47342) 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 -- 172.21.15.70:0/3001688464 learned_addr learned my addr 172.21.15.70:0/3001688464 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 -- 172.21.15.70:0/3001688464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015a7c0 msgr2=0x7f09701f37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015a7c0 0x7f09701f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 -- 172.21.15.70:0/3001688464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f097015f190 msgr2=0x7f09701f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f097015f190 0x7f09701f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:30.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 -- 172.21.15.70:0/3001688464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958009400 con 0x7f09700b12f0 2023-08-25T09:39:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.807+0000 7f096ffff640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015aed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f096000c4d0 tx=0x7f096000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:30.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.808+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09600074e0 con 0x7f09700b12f0 2023-08-25T09:39:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.808+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0960007640 con 0x7f09700b12f0 2023-08-25T09:39:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.808+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f097006fdc0 con 0x7f09700b12f0 2023-08-25T09:39:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.808+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960004a20 con 0x7f09700b12f0 2023-08-25T09:39:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.808+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09700c2540 con 0x7f09700b12f0 2023-08-25T09:39:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.809+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0970066fb0 con 0x7f09700b12f0 2023-08-25T09:39:30.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.810+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0960004b80 con 0x7f09700b12f0 2023-08-25T09:39:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.811+0000 7f096d7fa640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f095006a050 0x7f095006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.811+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0960087b70 con 0x7f09700b12f0 2023-08-25T09:39:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.811+0000 7f0967fff640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f095006a050 0x7f095006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:30.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.812+0000 7f0967fff640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f095006a050 0x7f095006c510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0958036000 tx=0x7f0958030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:30.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:30.815+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09600531c0 con 0x7f09700b12f0 2023-08-25T09:39:31.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.003+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09700b0d00 con 0x7f09700b12f0 2023-08-25T09:39:31.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.006+0000 7f096d7fa640 1 -- 172.21.15.70:0/3001688464 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f0960018090 con 0x7f09700b12f0 2023-08-25T09:39:31.009 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:31.010 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:31.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f095006a050 msgr2=0x7f095006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f095006a050 0x7f095006c510 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0958036000 tx=0x7f0958030000 comp rx=0 tx=0).stop 2023-08-25T09:39:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 msgr2=0x7f097015aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015aed0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f096000c4d0 tx=0x7f096000c990 comp rx=0 tx=0).stop 2023-08-25T09:39:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 shutdown_connections 2023-08-25T09:39:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f095006a050 0x7f095006c510 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f097015f190 0x7f09701f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f097015a7c0 0x7f09701f37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 --2- 172.21.15.70:0/3001688464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09700b12f0 0x7f097015aed0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.008+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 >> 172.21.15.70:0/3001688464 conn(0x7f09700affd0 msgr2=0x7f09701588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.009+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 shutdown_connections 2023-08-25T09:39:31.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.009+0000 7f0975652640 1 -- 172.21.15.70:0/3001688464 wait complete. 2023-08-25T09:39:31.038 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:31.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.252+0000 7f8612e3f640 1 Processor -- start 2023-08-25T09:39:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.253+0000 7f8612e3f640 1 -- start start 2023-08-25T09:39:31.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.254+0000 7f8612e3f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f860c0cd850 0x7f860c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.254+0000 7f8612e3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 0x7f860c0d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.254+0000 7f8612e3f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c0b6cc0 con 0x7f860c0b8510 2023-08-25T09:39:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.254+0000 7f8612e3f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f860c0ce170 con 0x7f860c0cd850 2023-08-25T09:39:31.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.254+0000 7f8612e3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0d1500 con 0x7f860c15cfe0 2023-08-25T09:39:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f86113dc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f860c0b8510 0x7f860c0b4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50044/0 (socket says 172.21.15.70:50044) 2023-08-25T09:39:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f86113dc640 1 -- 172.21.15.70:0/1324329864 learned_addr learned my addr 172.21.15.70:0/1324329864 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:31.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f8610bdb640 1 --2- 172.21.15.70:0/1324329864 >> v2:172.21.15.155:3300/0 conn(0x7f860c0cd850 0x7f860c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860bfff640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 0x7f860c0d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485204728 0 0) 0x7f8604001160 con 0x7f860c0b8510 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85f4003520 con 0x7f860c0b8510 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1956884292 0 0) 0x7f85f4003520 con 0x7f860c0b8510 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 msgr2=0x7f860c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 0x7f860c0d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 >> v2:172.21.15.155:3300/0 conn(0x7f860c0cd850 msgr2=0x7f860c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 --2- 172.21.15.70:0/1324329864 >> v2:172.21.15.155:3300/0 conn(0x7f860c0cd850 0x7f860c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0d17e0 con 0x7f860c0b8510 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8604001850 con 0x7f860c0b8510 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.255+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 >> v1:172.21.15.70:6789/0 conn(0x7f860c0b8510 legacy=0x7f860c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f860b7fe640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85f4003970 0x7f85f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f860b7fe640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85f40047c0 0x7f85f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f860b7fe640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 0x7f85f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0ce170 con 0x7f85f4003970 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0d1500 con 0x7f85f40047c0 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f400b980 con 0x7f860c15cfe0 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f8610bdb640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85f4003970 0x7f85f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f8610bdb640 1 -- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 msgr2=0x7f85f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f8610bdb640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f860c15cfe0 0x7f85f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f8610bdb640 1 -- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85f40047c0 msgr2=0x7f85f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:31.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f8610bdb640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85f40047c0 0x7f85f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.256+0000 7f8610bdb640 1 -- 172.21.15.70:0/1324329864 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4003520 con 0x7f85f4003970 2023-08-25T09:39:31.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.257+0000 7f8610bdb640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85f4003970 0x7f85f4003d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fc0 tx=0x7f85f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc5b49d3c949642 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:31.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc02cc60 con 0x7f85f4003970 2023-08-25T09:39:31.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc00f040 con 0x7f85f4003970 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f860b7fe640 1 -- 172.21.15.70:0/1324329864 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc034440 con 0x7f85f4003970 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f8612e3f640 1 -- 172.21.15.70:0/1324329864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85f4003970 msgr2=0x7f85f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f8612e3f640 1 --2- 172.21.15.70:0/1324329864 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f85f4003970 0x7f85f4003d90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fc0 tx=0x7f85f40057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f86113dc640 1 -- 172.21.15.70:0/1324329864 reap_dead start 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f8612e3f640 1 -- 172.21.15.70:0/1324329864 shutdown_connections 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f8612e3f640 1 -- 172.21.15.70:0/1324329864 >> 172.21.15.70:0/1324329864 conn(0x7f860c069850 msgr2=0x7f860c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f8612e3f640 1 -- 172.21.15.70:0/1324329864 shutdown_connections 2023-08-25T09:39:31.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.258+0000 7f8612e3f640 1 -- 172.21.15.70:0/1324329864 wait complete. 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.260+0000 7f8612e3f640 1 Processor -- start 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.260+0000 7f8612e3f640 1 -- start start 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.261+0000 7f8612e3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85f4003970 0x7f860c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.261+0000 7f8612e3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f860c150e60 0x7f860c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.261+0000 7f8612e3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 0x7f860c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.261+0000 7f8612e3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b3c40 con 0x7f860c0b2ab0 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.261+0000 7f8612e3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b3030 con 0x7f860c150e60 2023-08-25T09:39:31.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.261+0000 7f8612e3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b3310 con 0x7f85f4003970 2023-08-25T09:39:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 0x7f860c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 0x7f860c15aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47352/0 (socket says 172.21.15.70:47352) 2023-08-25T09:39:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 -- 172.21.15.70:0/2151241806 learned_addr learned my addr 172.21.15.70:0/2151241806 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:31.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f8610bdb640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85f4003970 0x7f860c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 -- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85f4003970 msgr2=0x7f860c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f860bfff640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f860c150e60 0x7f860c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85f4003970 0x7f860c0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 -- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f860c150e60 msgr2=0x7f860c0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f860c150e60 0x7f860c0b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 -- 172.21.15.70:0/2151241806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0b35f0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.262+0000 7f86113dc640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 0x7f860c15aa60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8604000fb0 tx=0x7f860400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.263+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604007310 con 0x7f860c0b2ab0 2023-08-25T09:39:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.263+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860400ced0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.263+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86040076d0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.263+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c0b38d0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.263+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c1512e0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.264+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860c0b3db0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.265+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8604004830 con 0x7f860c0b2ab0 2023-08-25T09:39:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.266+0000 7f86097fa640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f85f806a080 0x7f85f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.266+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8604087990 con 0x7f860c0b2ab0 2023-08-25T09:39:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.266+0000 7f8610bdb640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f85f806a080 0x7f85f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.267+0000 7f8610bdb640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f85f806a080 0x7f85f806c540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85fc036000 tx=0x7f85fc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:31.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.270+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8604053110 con 0x7f860c0b2ab0 2023-08-25T09:39:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.461+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f860c0b6cc0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.462+0000 7f86097fa640 1 -- 172.21.15.70:0/2151241806 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f860c0b6cc0 con 0x7f860c0b2ab0 2023-08-25T09:39:31.465 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f85f806a080 msgr2=0x7f85f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f85f806a080 0x7f85f806c540 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f85fc036000 tx=0x7f85fc02e000 comp rx=0 tx=0).stop 2023-08-25T09:39:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 msgr2=0x7f860c15aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 0x7f860c15aa60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8604000fb0 tx=0x7f860400cb90 comp rx=0 tx=0).stop 2023-08-25T09:39:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 shutdown_connections 2023-08-25T09:39:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f85f806a080 0x7f85f806c540 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f860c0b2ab0 0x7f860c15aa60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f860c150e60 0x7f860c0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 --2- 172.21.15.70:0/2151241806 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f85f4003970 0x7f860c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 >> 172.21.15.70:0/2151241806 conn(0x7f860c069850 msgr2=0x7f860c0c8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:31.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 shutdown_connections 2023-08-25T09:39:31.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.465+0000 7f8612e3f640 1 -- 172.21.15.70:0/2151241806 wait complete. 2023-08-25T09:39:31.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:31.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.698+0000 7f525285e640 1 Processor -- start 2023-08-25T09:39:31.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.698+0000 7f525285e640 1 -- start start 2023-08-25T09:39:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.699+0000 7f525285e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f524c06ad30 0x7f524c06b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.699+0000 7f525285e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9790 0x7f524c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.699+0000 7f525285e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f524c0c9c10 0x7f524c0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.699+0000 7f525285e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c06b650 con 0x7f524c06ad30 2023-08-25T09:39:31.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.699+0000 7f525285e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c1597d0 con 0x7f524c0c9c10 2023-08-25T09:39:31.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.699+0000 7f525285e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c15a5b0 con 0x7f524c0c9790 2023-08-25T09:39:31.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.700+0000 7f524bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f524c06ad30 0x7f524c06b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.700+0000 7f524bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f524c06ad30 0x7f524c06b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47354/0 (socket says 172.21.15.70:47354) 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.700+0000 7f524bfff640 1 -- 172.21.15.70:0/4094805296 learned_addr learned my addr 172.21.15.70:0/4094805296 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.700+0000 7f5243fff640 1 --2- 172.21.15.70:0/4094805296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9790 0x7f524c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.700+0000 7f524bfff640 1 -- 172.21.15.70:0/4094805296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9790 msgr2=0x7f524c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.701+0000 7f524bfff640 1 --2- 172.21.15.70:0/4094805296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9790 0x7f524c0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.701+0000 7f524bfff640 1 -- 172.21.15.70:0/4094805296 >> v2:172.21.15.155:3300/0 conn(0x7f524c0c9c10 msgr2=0x7f524c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:31.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.701+0000 7f524bfff640 1 --2- 172.21.15.70:0/4094805296 >> v2:172.21.15.155:3300/0 conn(0x7f524c0c9c10 0x7f524c0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.701+0000 7f524bfff640 1 -- 172.21.15.70:0/4094805296 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c15a890 con 0x7f524c06ad30 2023-08-25T09:39:31.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.702+0000 7f524bfff640 1 --2- 172.21.15.70:0/4094805296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c06b110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f524c0c8ba0 tx=0x7f523c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca542b210e69ba91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:31.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.702+0000 7f524b7fe640 1 -- 172.21.15.70:0/4094805296 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c031890 con 0x7f524c06ad30 2023-08-25T09:39:31.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.702+0000 7f524b7fe640 1 -- 172.21.15.70:0/4094805296 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c0319f0 con 0x7f524c06ad30 2023-08-25T09:39:31.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.702+0000 7f524b7fe640 1 -- 172.21.15.70:0/4094805296 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c031d80 con 0x7f524c06ad30 2023-08-25T09:39:31.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 -- 172.21.15.70:0/4094805296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 msgr2=0x7f524c06b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 --2- 172.21.15.70:0/4094805296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c06b110 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f524c0c8ba0 tx=0x7f523c02b930 comp rx=0 tx=0).stop 2023-08-25T09:39:31.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 -- 172.21.15.70:0/4094805296 shutdown_connections 2023-08-25T09:39:31.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 --2- 172.21.15.70:0/4094805296 >> v2:172.21.15.155:3300/0 conn(0x7f524c0c9c10 0x7f524c0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 --2- 172.21.15.70:0/4094805296 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9790 0x7f524c0ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 --2- 172.21.15.70:0/4094805296 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c06b110 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 -- 172.21.15.70:0/4094805296 >> 172.21.15.70:0/4094805296 conn(0x7f524c0cb0d0 msgr2=0x7f524c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:31.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 -- 172.21.15.70:0/4094805296 shutdown_connections 2023-08-25T09:39:31.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.703+0000 7f525285e640 1 -- 172.21.15.70:0/4094805296 wait complete. 2023-08-25T09:39:31.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.704+0000 7f525285e640 1 Processor -- start 2023-08-25T09:39:31.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 -- start start 2023-08-25T09:39:31.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524c0c9790 0x7f524c1f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9c10 0x7f524c1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0c1fb0 con 0x7f524c06ad30 2023-08-25T09:39:31.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c15d190 con 0x7f524c0c9790 2023-08-25T09:39:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.705+0000 7f525285e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0c2200 con 0x7f524c0c9c10 2023-08-25T09:39:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c0c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47366/0 (socket says 172.21.15.70:47366) 2023-08-25T09:39:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 -- 172.21.15.70:0/4220161479 learned_addr learned my addr 172.21.15.70:0/4220161479 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f5243fff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524c0c9790 0x7f524c1f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 -- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9c10 msgr2=0x7f524c1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9c10 0x7f524c1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 -- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524c0c9790 msgr2=0x7f524c1f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524c0c9790 0x7f524c1f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f524bfff640 1 -- 172.21.15.70:0/4220161479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0094d0 con 0x7f524c06ad30 2023-08-25T09:39:31.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.706+0000 7f5243fff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524c0c9790 0x7f524c1f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:31.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.707+0000 7f524bfff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c0c49b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f523c000c00 tx=0x7f523c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:31.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.707+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c03ac60 con 0x7f524c06ad30 2023-08-25T09:39:31.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.707+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c03ade0 con 0x7f524c06ad30 2023-08-25T09:39:31.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.707+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c0c24e0 con 0x7f524c06ad30 2023-08-25T09:39:31.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.707+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c044630 con 0x7f524c06ad30 2023-08-25T09:39:31.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.707+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c15ce60 con 0x7f524c06ad30 2023-08-25T09:39:31.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.708+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c0c2a00 con 0x7f524c06ad30 2023-08-25T09:39:31.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.709+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f523c04d450 con 0x7f524c06ad30 2023-08-25T09:39:31.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.710+0000 7f52497fa640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f522c06a050 0x7f522c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.710+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f523c0aacc0 con 0x7f524c06ad30 2023-08-25T09:39:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.710+0000 7f5243fff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f522c06a050 0x7f522c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.711+0000 7f5243fff640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f522c06a050 0x7f522c06c510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f524c147710 tx=0x7f5234008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:31.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.714+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f524c0c2a00 con 0x7f524c06ad30 2023-08-25T09:39:31.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.902+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f524c0c2a00 con 0x7f524c06ad30 2023-08-25T09:39:31.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.902+0000 7f52497fa640 1 -- 172.21.15.70:0/4220161479 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f524c0c2a00 con 0x7f524c06ad30 2023-08-25T09:39:31.905 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f522c06a050 msgr2=0x7f522c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f522c06a050 0x7f522c06c510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f524c147710 tx=0x7f5234008040 comp rx=0 tx=0).stop 2023-08-25T09:39:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 msgr2=0x7f524c0c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c0c49b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f523c000c00 tx=0x7f523c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 shutdown_connections 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f522c06a050 0x7f522c06c510 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f524c0c9c10 0x7f524c1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f524c0c9790 0x7f524c1f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 --2- 172.21.15.70:0/4220161479 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f524c06ad30 0x7f524c0c49b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 >> 172.21.15.70:0/4220161479 conn(0x7f524c0cb0d0 msgr2=0x7f524c15abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 shutdown_connections 2023-08-25T09:39:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:31.905+0000 7f525285e640 1 -- 172.21.15.70:0/4220161479 wait complete. 2023-08-25T09:39:31.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.138+0000 7f70896ee640 1 Processor -- start 2023-08-25T09:39:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.138+0000 7f70896ee640 1 -- start start 2023-08-25T09:39:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.139+0000 7f70896ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.139+0000 7f70896ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70840b8510 0x7f70840b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.139+0000 7f70896ee640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70840d1500 con 0x7f70840cd850 2023-08-25T09:39:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.139+0000 7f70896ee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70840d17e0 con 0x7f70840b8510 2023-08-25T09:39:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.139+0000 7f70896ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840d1ac0 con 0x7f708415cfe0 2023-08-25T09:39:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f7082ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f70840cd850 0x7f70840cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50050/0 (socket says 172.21.15.70:50050) 2023-08-25T09:39:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f7082ffd640 1 -- 172.21.15.70:0/2398083928 learned_addr learned my addr 172.21.15.70:0/2398083928 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:32.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70837fe640 1 --2- 172.21.15.70:0/2398083928 >> v2:172.21.15.70:3300/0 conn(0x7f70840b8510 0x7f70840b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f707bfff640 1 --2- 172.21.15.70:0/2398083928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70827fc640 1 -- 172.21.15.70:0/2398083928 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745758654 0 0) 0x7f70840d1500 con 0x7f70840cd850 2023-08-25T09:39:32.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70827fc640 1 -- 172.21.15.70:0/2398083928 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7070003520 con 0x7f70840cd850 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70837fe640 1 -- 172.21.15.70:0/2398083928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 msgr2=0x7f70840d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70837fe640 1 --2- 172.21.15.70:0/2398083928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70837fe640 1 -- 172.21.15.70:0/2398083928 >> v1:172.21.15.70:6789/0 conn(0x7f70840cd850 legacy=0x7f70840cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.140+0000 7f70837fe640 1 -- 172.21.15.70:0/2398083928 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840d1da0 con 0x7f70840b8510 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.141+0000 7f70837fe640 1 --2- 172.21.15.70:0/2398083928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840b8510 0x7f70840b5cb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f707c009d90 tx=0x7f707c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b83ab5e14ddda8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70827fc640 1 -- 172.21.15.70:0/2398083928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c02cc60 con 0x7f70840b8510 2023-08-25T09:39:32.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70827fc640 1 -- 172.21.15.70:0/2398083928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c02cdc0 con 0x7f70840b8510 2023-08-25T09:39:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70827fc640 1 -- 172.21.15.70:0/2398083928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c034660 con 0x7f70840b8510 2023-08-25T09:39:32.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70896ee640 1 -- 172.21.15.70:0/2398083928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840b8510 msgr2=0x7f70840b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70896ee640 1 --2- 172.21.15.70:0/2398083928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840b8510 0x7f70840b5cb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f707c009d90 tx=0x7f707c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70896ee640 1 -- 172.21.15.70:0/2398083928 shutdown_connections 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70896ee640 1 --2- 172.21.15.70:0/2398083928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840b8510 0x7f70840b5cb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70896ee640 1 --2- 172.21.15.70:0/2398083928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.142+0000 7f70896ee640 1 -- 172.21.15.70:0/2398083928 >> 172.21.15.70:0/2398083928 conn(0x7f7084069850 msgr2=0x7f7084069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.143+0000 7f70896ee640 1 -- 172.21.15.70:0/2398083928 shutdown_connections 2023-08-25T09:39:32.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.143+0000 7f70896ee640 1 -- 172.21.15.70:0/2398083928 wait complete. 2023-08-25T09:39:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.145+0000 7f70896ee640 1 Processor -- start 2023-08-25T09:39:32.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.145+0000 7f70896ee640 1 -- start start 2023-08-25T09:39:32.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f70896ee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70840b8510 0x7f70840b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f70896ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 0x7f7084155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f70896ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f70896ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840cb330 con 0x7f70840cd850 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f70896ee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840cb610 con 0x7f70840b8510 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f70896ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840cb8f0 con 0x7f708415cfe0 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f707bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 0x7f7084155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f707bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 0x7f7084155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47392/0 (socket says 172.21.15.70:47392) 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.146+0000 7f707bfff640 1 -- 172.21.15.70:0/3818083364 learned_addr learned my addr 172.21.15.70:0/3818083364 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f70837fe640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707bfff640 1 -- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 msgr2=0x7f70840cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707bfff640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707bfff640 1 -- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70840b8510 msgr2=0x7f70840b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707bfff640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70840b8510 0x7f70840b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707bfff640 1 -- 172.21.15.70:0/3818083364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c009a70 con 0x7f70840cd850 2023-08-25T09:39:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707bfff640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 0x7f7084155f50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f706c00e840 tx=0x7f706c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c0094f0 con 0x7f70840cd850 2023-08-25T09:39:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c00c940 con 0x7f70840cd850 2023-08-25T09:39:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c00ccd0 con 0x7f70840cd850 2023-08-25T09:39:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.147+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70840cbbd0 con 0x7f70840cd850 2023-08-25T09:39:32.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.148+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70840b5410 con 0x7f70840cd850 2023-08-25T09:39:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.149+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708406a700 con 0x7f70840cd850 2023-08-25T09:39:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.150+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f706c013470 con 0x7f70840cd850 2023-08-25T09:39:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.151+0000 7f707b7fe640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f706406a050 0x7f706406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.151+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f706c088760 con 0x7f70840cd850 2023-08-25T09:39:32.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.151+0000 7f7082ffd640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f706406a050 0x7f706406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.152+0000 7f7082ffd640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f706406a050 0x7f706406c510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70841f3190 tx=0x7f7074007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.155+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f706c053db0 con 0x7f70840cd850 2023-08-25T09:39:32.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.329+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7084066fb0 con 0x7f70840cd850 2023-08-25T09:39:32.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.330+0000 7f707b7fe640 1 -- 172.21.15.70:0/3818083364 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f706c0573d0 con 0x7f70840cd850 2023-08-25T09:39:32.332 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:32.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f706406a050 msgr2=0x7f706406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f706406a050 0x7f706406c510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f70841f3190 tx=0x7f7074007930 comp rx=0 tx=0).stop 2023-08-25T09:39:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 msgr2=0x7f7084155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 0x7f7084155f50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f706c00e840 tx=0x7f706c00ed00 comp rx=0 tx=0).stop 2023-08-25T09:39:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 shutdown_connections 2023-08-25T09:39:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f706406a050 0x7f706406c510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f708415cfe0 0x7f70840cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70840cd850 0x7f7084155f50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 --2- 172.21.15.70:0/3818083364 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70840b8510 0x7f70840b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 >> 172.21.15.70:0/3818083364 conn(0x7f7084069850 msgr2=0x7f708415c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.332+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 shutdown_connections 2023-08-25T09:39:32.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.333+0000 7f70896ee640 1 -- 172.21.15.70:0/3818083364 wait complete. 2023-08-25T09:39:32.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:32.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.556+0000 7f166eba4640 1 Processor -- start 2023-08-25T09:39:32.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.556+0000 7f166eba4640 1 -- start start 2023-08-25T09:39:32.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166eba4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f1668069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166eba4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16680cca20 0x7f16680cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166eba4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16680ac880 con 0x7f1668069850 2023-08-25T09:39:32.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166eba4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166815e650 con 0x7f16680cca20 2023-08-25T09:39:32.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166eba4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16680ad400 con 0x7f166815e210 2023-08-25T09:39:32.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166d141640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16680cca20 0x7f16680cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166d141640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16680cca20 0x7f16680cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47394/0 (socket says 172.21.15.70:47394) 2023-08-25T09:39:32.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.558+0000 7f166d141640 1 -- 172.21.15.70:0/1650066692 learned_addr learned my addr 172.21.15.70:0/1650066692 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:32.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.559+0000 7f1667fff640 1 --2- 172.21.15.70:0/1650066692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f1668069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.559+0000 7f166d141640 1 -- 172.21.15.70:0/1650066692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 msgr2=0x7f1668069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:32.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.559+0000 7f166d141640 1 --2- 172.21.15.70:0/1650066692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f1668069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.559+0000 7f166d141640 1 -- 172.21.15.70:0/1650066692 >> v1:172.21.15.155:6789/0 conn(0x7f1668069850 legacy=0x7f16680c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.559+0000 7f166d141640 1 -- 172.21.15.70:0/1650066692 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680ad890 con 0x7f16680cca20 2023-08-25T09:39:32.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.560+0000 7f166d141640 1 --2- 172.21.15.70:0/1650066692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16680cca20 0x7f16680cce80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1660009e50 tx=0x7f166002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d67d729031f82d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.560+0000 7f16677fe640 1 -- 172.21.15.70:0/1650066692 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166002cb90 con 0x7f16680cca20 2023-08-25T09:39:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.560+0000 7f16677fe640 1 -- 172.21.15.70:0/1650066692 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166002ccf0 con 0x7f16680cca20 2023-08-25T09:39:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.560+0000 7f16677fe640 1 -- 172.21.15.70:0/1650066692 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660034610 con 0x7f16680cca20 2023-08-25T09:39:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 -- 172.21.15.70:0/1650066692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16680cca20 msgr2=0x7f16680cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 --2- 172.21.15.70:0/1650066692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16680cca20 0x7f16680cce80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1660009e50 tx=0x7f166002bff0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 -- 172.21.15.70:0/1650066692 shutdown_connections 2023-08-25T09:39:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 --2- 172.21.15.70:0/1650066692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16680cca20 0x7f16680cce80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 --2- 172.21.15.70:0/1650066692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f1668069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 -- 172.21.15.70:0/1650066692 >> 172.21.15.70:0/1650066692 conn(0x7f16680cb0d0 msgr2=0x7f16680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 -- 172.21.15.70:0/1650066692 shutdown_connections 2023-08-25T09:39:32.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.561+0000 7f166eba4640 1 -- 172.21.15.70:0/1650066692 wait complete. 2023-08-25T09:39:32.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.563+0000 7f166eba4640 1 Processor -- start 2023-08-25T09:39:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.563+0000 7f166eba4640 1 -- start start 2023-08-25T09:39:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.564+0000 7f166eba4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 0x7f166815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.564+0000 7f166eba4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16680cca20 0x7f166814ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.564+0000 7f166eba4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f16680c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.564+0000 7f166eba4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16680c4b00 con 0x7f1668069850 2023-08-25T09:39:32.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.564+0000 7f166eba4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166815aaa0 con 0x7f16680cca20 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.564+0000 7f166eba4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166815ad80 con 0x7f166815e210 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 0x7f166815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 0x7f166815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47408/0 (socket says 172.21.15.70:47408) 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 -- 172.21.15.70:0/4119562393 learned_addr learned my addr 172.21.15.70:0/4119562393 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f1667fff640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16680cca20 0x7f166814ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166d141640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f16680c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 -- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 msgr2=0x7f16680c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f16680c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 -- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16680cca20 msgr2=0x7f166814ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16680cca20 0x7f166814ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 -- 172.21.15.70:0/4119562393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1660009b30 con 0x7f1668069850 2023-08-25T09:39:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f1667fff640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16680cca20 0x7f166814ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:32.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.565+0000 7f166c940640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 0x7f166815a1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c450 tx=0x7f165c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.566+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c0052d0 con 0x7f1668069850 2023-08-25T09:39:32.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.566+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c005430 con 0x7f1668069850 2023-08-25T09:39:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.566+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c012870 con 0x7f1668069850 2023-08-25T09:39:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.566+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166815b060 con 0x7f1668069850 2023-08-25T09:39:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.566+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166815a720 con 0x7f1668069850 2023-08-25T09:39:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.567+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1668066fb0 con 0x7f1668069850 2023-08-25T09:39:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.568+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f165c0129d0 con 0x7f1668069850 2023-08-25T09:39:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.569+0000 7f16657fa640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164c06a050 0x7f164c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.569+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f165c088180 con 0x7f1668069850 2023-08-25T09:39:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.569+0000 7f1667fff640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164c06a050 0x7f164c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.570+0000 7f1667fff640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164c06a050 0x7f164c06c510 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1654006fd0 tx=0x7f1654008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.573+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f165c0537d0 con 0x7f1668069850 2023-08-25T09:39:32.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.747+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16680c20d0 con 0x7f1668069850 2023-08-25T09:39:32.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.748+0000 7f16657fa640 1 -- 172.21.15.70:0/4119562393 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f165c056df0 con 0x7f1668069850 2023-08-25T09:39:32.751 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:32.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.750+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164c06a050 msgr2=0x7f164c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.750+0000 7f166eba4640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164c06a050 0x7f164c06c510 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1654006fd0 tx=0x7f1654008040 comp rx=0 tx=0).stop 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.750+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 msgr2=0x7f166815a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 0x7f166815a1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c450 tx=0x7f165c00c910 comp rx=0 tx=0).stop 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 shutdown_connections 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164c06a050 0x7f164c06c510 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166815e210 0x7f16680c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16680cca20 0x7f166814ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 --2- 172.21.15.70:0/4119562393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1668069850 0x7f166815a1e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 >> 172.21.15.70:0/4119562393 conn(0x7f16680cb0d0 msgr2=0x7f166815ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:32.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 shutdown_connections 2023-08-25T09:39:32.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.751+0000 7f166eba4640 1 -- 172.21.15.70:0/4119562393 wait complete. 2023-08-25T09:39:32.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:32.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.968+0000 7fb70f204640 1 Processor -- start 2023-08-25T09:39:32.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.969+0000 7fb70f204640 1 -- start start 2023-08-25T09:39:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.970+0000 7fb70f204640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb70806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.970+0000 7fb70f204640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb70806b2b0 0x7fb70815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.970+0000 7fb70f204640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb70815c6d0 0x7fb7080b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.970+0000 7fb70f204640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70806bd80 con 0x7fb70806b2b0 2023-08-25T09:39:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.970+0000 7fb70f204640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080b1b60 con 0x7fb70815c6d0 2023-08-25T09:39:32.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.970+0000 7fb70f204640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70806b7c0 con 0x7fb70815e830 2023-08-25T09:39:32.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.971+0000 7fb707fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb70806b2b0 0x7fb70815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.971+0000 7fb70cfa0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb70806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.971+0000 7fb707fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb70806b2b0 0x7fb70815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47416/0 (socket says 172.21.15.70:47416) 2023-08-25T09:39:32.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.971+0000 7fb707fff640 1 -- 172.21.15.70:0/769544283 learned_addr learned my addr 172.21.15.70:0/769544283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.971+0000 7fb70d7a1640 1 --2- 172.21.15.70:0/769544283 >> v2:172.21.15.155:3300/0 conn(0x7fb70815c6d0 0x7fb7080b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.972+0000 7fb707fff640 1 -- 172.21.15.70:0/769544283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 msgr2=0x7fb70806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.972+0000 7fb707fff640 1 --2- 172.21.15.70:0/769544283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb70806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.972+0000 7fb707fff640 1 -- 172.21.15.70:0/769544283 >> v2:172.21.15.155:3300/0 conn(0x7fb70815c6d0 msgr2=0x7fb7080b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.972+0000 7fb707fff640 1 --2- 172.21.15.70:0/769544283 >> v2:172.21.15.155:3300/0 conn(0x7fb70815c6d0 0x7fb7080b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.972+0000 7fb707fff640 1 -- 172.21.15.70:0/769544283 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70806baa0 con 0x7fb70806b2b0 2023-08-25T09:39:32.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.973+0000 7fb707fff640 1 --2- 172.21.15.70:0/769544283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70815c190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb70806d150 tx=0x7fb6f402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975542185f7a0b5b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.973+0000 7fb7077fe640 1 -- 172.21.15.70:0/769544283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4031860 con 0x7fb70806b2b0 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.973+0000 7fb7077fe640 1 -- 172.21.15.70:0/769544283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f40319c0 con 0x7fb70806b2b0 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.973+0000 7fb7077fe640 1 -- 172.21.15.70:0/769544283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4031d50 con 0x7fb70806b2b0 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 -- 172.21.15.70:0/769544283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 msgr2=0x7fb70815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 --2- 172.21.15.70:0/769544283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70815c190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb70806d150 tx=0x7fb6f402b900 comp rx=0 tx=0).stop 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 -- 172.21.15.70:0/769544283 shutdown_connections 2023-08-25T09:39:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 --2- 172.21.15.70:0/769544283 >> v2:172.21.15.155:3300/0 conn(0x7fb70815c6d0 0x7fb7080b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 --2- 172.21.15.70:0/769544283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70815c190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 --2- 172.21.15.70:0/769544283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb70806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 -- 172.21.15.70:0/769544283 >> 172.21.15.70:0/769544283 conn(0x7fb708069850 msgr2=0x7fb708069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 -- 172.21.15.70:0/769544283 shutdown_connections 2023-08-25T09:39:32.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.974+0000 7fb70f204640 1 -- 172.21.15.70:0/769544283 wait complete. 2023-08-25T09:39:32.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.975+0000 7fb70f204640 1 Processor -- start 2023-08-25T09:39:32.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.976+0000 7fb70f204640 1 -- start start 2023-08-25T09:39:32.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70f204640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70f204640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70815c6d0 0x7fb7080a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70f204640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb7080a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70f204640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080c99c0 con 0x7fb70806b2b0 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70f204640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080c5770 con 0x7fb70815c6d0 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70f204640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080c96e0 con 0x7fb70815e830 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70814d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47430/0 (socket says 172.21.15.70:47430) 2023-08-25T09:39:32.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 -- 172.21.15.70:0/2837011271 learned_addr learned my addr 172.21.15.70:0/2837011271 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:32.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 -- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 msgr2=0x7fb7080a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb7080a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 -- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70815c6d0 msgr2=0x7fb7080a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb707fff640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70815c6d0 0x7fb7080a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70815c6d0 0x7fb7080a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb70cfa0640 1 -- 172.21.15.70:0/2837011271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40094d0 con 0x7fb70806b2b0 2023-08-25T09:39:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.977+0000 7fb707fff640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70815c6d0 0x7fb7080a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.978+0000 7fb70cfa0640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70814d500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e870 tx=0x7fb6fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.978+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc0094f0 con 0x7fb70806b2b0 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.978+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00c940 con 0x7fb70806b2b0 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.978+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00ccf0 con 0x7fb70806b2b0 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.978+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7080c5a10 con 0x7fb70806b2b0 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.978+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70814da40 con 0x7fb70806b2b0 2023-08-25T09:39:32.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.979+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7080c55d0 con 0x7fb70806b2b0 2023-08-25T09:39:32.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.980+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb6fc013470 con 0x7fb70806b2b0 2023-08-25T09:39:32.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.981+0000 7fb7057fa640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6ec06a050 0x7fb6ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:32.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.981+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6fc088650 con 0x7fb70806b2b0 2023-08-25T09:39:32.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.981+0000 7fb707fff640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6ec06a050 0x7fb6ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:32.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.982+0000 7fb707fff640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6ec06a050 0x7fb6ec06c510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4036000 tx=0x7fb6f403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:32.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:32.985+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7080c55d0 con 0x7fb70806b2b0 2023-08-25T09:39:33.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.183+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7080c55d0 con 0x7fb70806b2b0 2023-08-25T09:39:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.185+0000 7fb7057fa640 1 -- 172.21.15.70:0/2837011271 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fb7080c55d0 con 0x7fb70806b2b0 2023-08-25T09:39:33.193 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:33.193 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6ec06a050 msgr2=0x7fb6ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6ec06a050 0x7fb6ec06c510 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4036000 tx=0x7fb6f403c040 comp rx=0 tx=0).stop 2023-08-25T09:39:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 msgr2=0x7fb70814d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70814d500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e870 tx=0x7fb6fc00ed30 comp rx=0 tx=0).stop 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 shutdown_connections 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6ec06a050 0x7fb6ec06c510 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb70815e830 0x7fb7080a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb70815c6d0 0x7fb7080a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 --2- 172.21.15.70:0/2837011271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb70806b2b0 0x7fb70814d500 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 >> 172.21.15.70:0/2837011271 conn(0x7fb708069850 msgr2=0x7fb70815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 shutdown_connections 2023-08-25T09:39:33.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.188+0000 7fb70f204640 1 -- 172.21.15.70:0/2837011271 wait complete. 2023-08-25T09:39:33.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:33.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.427+0000 7f31b306a640 1 Processor -- start 2023-08-25T09:39:33.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.427+0000 7f31b306a640 1 -- start start 2023-08-25T09:39:33.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.428+0000 7f31b306a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31ac15a920 0x7f31ac0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.428+0000 7f31b306a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ac0adbb0 con 0x7f31ac0b1ce0 2023-08-25T09:39:33.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.428+0000 7f31b306a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ac0c90a0 con 0x7f31ac0b0320 2023-08-25T09:39:33.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.428+0000 7f31b306a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ac0c9380 con 0x7f31ac15a920 2023-08-25T09:39:33.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b2869640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f31ac0b1ce0 0x7f31ac0ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50064/0 (socket says 172.21.15.70:50064) 2023-08-25T09:39:33.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b2869640 1 -- 172.21.15.70:0/908840370 learned_addr learned my addr 172.21.15.70:0/908840370 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:33.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400512860 0 0) 0x7f31ac0adbb0 con 0x7f31ac0b1ce0 2023-08-25T09:39:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3198003520 con 0x7f31ac0b1ce0 2023-08-25T09:39:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1867640 1 --2- 172.21.15.70:0/908840370 >> v2:172.21.15.155:3300/0 conn(0x7f31ac15a920 0x7f31ac0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348144326 0 0) 0x7f3198003520 con 0x7f31ac0b1ce0 2023-08-25T09:39:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 >> v2:172.21.15.155:3300/0 conn(0x7f31ac15a920 msgr2=0x7f31ac0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1066640 1 --2- 172.21.15.70:0/908840370 >> v2:172.21.15.155:3300/0 conn(0x7f31ac15a920 0x7f31ac0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.429+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 >> v1:172.21.15.155:6789/0 conn(0x7f31ac0b0320 legacy=0x7f31ac0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac0c9660 con 0x7f31ac0b1ce0 2023-08-25T09:39:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f31a40019f0 con 0x7f31ac0b1ce0 2023-08-25T09:39:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 >> v1:172.21.15.70:6789/0 conn(0x7f31ac0b1ce0 legacy=0x7f31ac0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980039c0 0x7f3198003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31980047c0 0x7f3198004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980051d0 0x7f319800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0c9380 con 0x7f31980051d0 2023-08-25T09:39:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319800c120 con 0x7f31980047c0 2023-08-25T09:39:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319800c2e0 con 0x7f31980039c0 2023-08-25T09:39:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b2869640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980051d0 0x7f319800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b1867640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31980047c0 0x7f3198004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.430+0000 7f31b2068640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980039c0 0x7f3198003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.431+0000 7f31b2869640 1 -- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980039c0 msgr2=0x7f3198003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.431+0000 7f31b2869640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980039c0 0x7f3198003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.431+0000 7f31b2869640 1 -- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31980047c0 msgr2=0x7f3198004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.431+0000 7f31b2869640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31980047c0 0x7f3198004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.431+0000 7f31b2869640 1 -- 172.21.15.70:0/908840370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3198003520 con 0x7f31980051d0 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.431+0000 7f31b2068640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980039c0 0x7f3198003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.432+0000 7f31b2869640 1 -- 172.21.15.70:0/908840370 reap_dead start 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.432+0000 7f31b2869640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980051d0 0x7f319800b980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f31a400a760 tx=0x7f31980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7897d57a0e22e735 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.432+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4034630 con 0x7f31980051d0 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.432+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a402f070 con 0x7f31980051d0 2023-08-25T09:39:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.432+0000 7f31b1066640 1 -- 172.21.15.70:0/908840370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40349f0 con 0x7f31980051d0 2023-08-25T09:39:33.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 -- 172.21.15.70:0/908840370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980051d0 msgr2=0x7f319800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980051d0 0x7f319800b980 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f31a400a760 tx=0x7f31980057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 -- 172.21.15.70:0/908840370 shutdown_connections 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 --2- 172.21.15.70:0/908840370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31980051d0 0x7f319800b980 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 -- 172.21.15.70:0/908840370 >> 172.21.15.70:0/908840370 conn(0x7f31ac0ce830 msgr2=0x7f31ac0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 -- 172.21.15.70:0/908840370 shutdown_connections 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.433+0000 7f31b306a640 1 -- 172.21.15.70:0/908840370 wait complete. 2023-08-25T09:39:33.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.435+0000 7f31b306a640 1 Processor -- start 2023-08-25T09:39:33.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.435+0000 7f31b306a640 1 -- start start 2023-08-25T09:39:33.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b306a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980051d0 0x7f31ac157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b306a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 0x7f31ac069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b306a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31ac0704f0 0x7f31ac0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b306a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac070ee0 con 0x7f31ac15adf0 2023-08-25T09:39:33.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b306a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac157890 con 0x7f31ac0704f0 2023-08-25T09:39:33.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b306a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac06a860 con 0x7f31980051d0 2023-08-25T09:39:33.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 0x7f31ac069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 0x7f31ac069f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47446/0 (socket says 172.21.15.70:47446) 2023-08-25T09:39:33.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 -- 172.21.15.70:0/566126893 learned_addr learned my addr 172.21.15.70:0/566126893 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:33.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b2068640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980051d0 0x7f31ac157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b2869640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31ac0704f0 0x7f31ac0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 -- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980051d0 msgr2=0x7f31ac157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980051d0 0x7f31ac157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 -- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31ac0704f0 msgr2=0x7f31ac0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31ac0704f0 0x7f31ac0709a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.436+0000 7f31b1867640 1 -- 172.21.15.70:0/566126893 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac06ab40 con 0x7f31ac15adf0 2023-08-25T09:39:33.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.437+0000 7f31b1867640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 0x7f31ac069f10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f319c00f550 tx=0x7f319c00fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:33.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.437+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c013810 con 0x7f31ac15adf0 2023-08-25T09:39:33.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.437+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c013970 con 0x7f31ac15adf0 2023-08-25T09:39:33.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.437+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31ac06ae20 con 0x7f31ac15adf0 2023-08-25T09:39:33.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.437+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c013ce0 con 0x7f31ac15adf0 2023-08-25T09:39:33.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.437+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ac15c270 con 0x7f31ac15adf0 2023-08-25T09:39:33.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.438+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ac15c430 con 0x7f31ac15adf0 2023-08-25T09:39:33.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.439+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f319c01f460 con 0x7f31ac15adf0 2023-08-25T09:39:33.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.440+0000 7f31a2ffd640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f319006a050 0x7f319006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.440+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f319c088a10 con 0x7f31ac15adf0 2023-08-25T09:39:33.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.440+0000 7f31b2068640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f319006a050 0x7f319006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.441+0000 7f31b2068640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f319006a050 0x7f319006c510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f31a80015a0 tx=0x7f31a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:33.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.444+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f319c057af0 con 0x7f31ac15adf0 2023-08-25T09:39:33.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.622+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31ac061460 con 0x7f31ac15adf0 2023-08-25T09:39:33.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.623+0000 7f31a2ffd640 1 -- 172.21.15.70:0/566126893 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f319c057680 con 0x7f31ac15adf0 2023-08-25T09:39:33.626 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:33.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.625+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f319006a050 msgr2=0x7f319006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.625+0000 7f31b306a640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f319006a050 0x7f319006c510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f31a80015a0 tx=0x7f31a8009040 comp rx=0 tx=0).stop 2023-08-25T09:39:33.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 msgr2=0x7f31ac069f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 0x7f31ac069f10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f319c00f550 tx=0x7f319c00fa10 comp rx=0 tx=0).stop 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 shutdown_connections 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f319006a050 0x7f319006c510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31ac0704f0 0x7f31ac0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31ac15adf0 0x7f31ac069f10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 --2- 172.21.15.70:0/566126893 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31980051d0 0x7f31ac157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 >> 172.21.15.70:0/566126893 conn(0x7f31ac0ce830 msgr2=0x7f31ac0af230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:33.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 shutdown_connections 2023-08-25T09:39:33.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.626+0000 7f31b306a640 1 -- 172.21.15.70:0/566126893 wait complete. 2023-08-25T09:39:33.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:33.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.864+0000 7fa463547640 1 Processor -- start 2023-08-25T09:39:33.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.864+0000 7fa463547640 1 -- start start 2023-08-25T09:39:33.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.865+0000 7fa463547640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa45c15ae50 0x7fa45c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.865+0000 7fa463547640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c0cdc20 0x7fa45c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.865+0000 7fa463547640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa45c0cc5f0 con 0x7fa45c0ce060 2023-08-25T09:39:33.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.865+0000 7fa463547640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45c070650 con 0x7fa45c15ae50 2023-08-25T09:39:33.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.865+0000 7fa463547640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c070930 con 0x7fa45c0cdc20 2023-08-25T09:39:33.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.866+0000 7fa460ae2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c0cdc20 0x7fa45c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.866+0000 7fa4612e3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa45c15ae50 0x7fa45c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.866+0000 7fa461ae4640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa45c0ce060 0x7fa45c0ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:45358/0 (socket says 172.21.15.70:45358) 2023-08-25T09:39:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.866+0000 7fa461ae4640 1 -- 172.21.15.70:0/1217690991 learned_addr learned my addr 172.21.15.70:0/1217690991 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.866+0000 7fa4612e3640 1 --2- 172.21.15.70:0/1217690991 >> v2:172.21.15.155:3300/0 conn(0x7fa45c15ae50 0x7fa45c0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:48816/0 (socket says 172.21.15.70:48816) 2023-08-25T09:39:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.866+0000 7fa460ae2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c0cdc20 0x7fa45c069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:45910/0 (socket says 172.21.15.70:45910) 2023-08-25T09:39:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.867+0000 7fa4612e3640 1 -- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c0cdc20 msgr2=0x7fa45c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.867+0000 7fa4612e3640 1 --2- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c0cdc20 0x7fa45c069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.867+0000 7fa4612e3640 1 -- 172.21.15.70:0/1217690991 >> v1:172.21.15.155:6789/0 conn(0x7fa45c0ce060 legacy=0x7fa45c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.867+0000 7fa4612e3640 1 -- 172.21.15.70:0/1217690991 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c070c10 con 0x7fa45c15ae50 2023-08-25T09:39:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.868+0000 7fa4612e3640 1 --2- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c15ae50 0x7fa45c0c2110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa45802b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68c3a286e4b9c3bb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.869+0000 7fa453fff640 1 -- 172.21.15.70:0/1217690991 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458030800 con 0x7fa45c15ae50 2023-08-25T09:39:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.869+0000 7fa453fff640 1 -- 172.21.15.70:0/1217690991 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458030960 con 0x7fa45c15ae50 2023-08-25T09:39:33.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.869+0000 7fa453fff640 1 -- 172.21.15.70:0/1217690991 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458030cf0 con 0x7fa45c15ae50 2023-08-25T09:39:33.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 -- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c15ae50 msgr2=0x7fa45c0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 --2- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c15ae50 0x7fa45c0c2110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa4580097a0 tx=0x7fa45802b940 comp rx=0 tx=0).stop 2023-08-25T09:39:33.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 -- 172.21.15.70:0/1217690991 shutdown_connections 2023-08-25T09:39:33.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 --2- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c0cdc20 0x7fa45c069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 --2- 172.21.15.70:0/1217690991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c15ae50 0x7fa45c0c2110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 -- 172.21.15.70:0/1217690991 >> 172.21.15.70:0/1217690991 conn(0x7fa45c0cb0d0 msgr2=0x7fa45c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:33.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 -- 172.21.15.70:0/1217690991 shutdown_connections 2023-08-25T09:39:33.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.870+0000 7fa463547640 1 -- 172.21.15.70:0/1217690991 wait complete. 2023-08-25T09:39:33.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.872+0000 7fa463547640 1 Processor -- start 2023-08-25T09:39:33.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.872+0000 7fa463547640 1 -- start start 2023-08-25T09:39:33.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa463547640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 0x7fa45c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa463547640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c0ce060 0x7fa45c15e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa463547640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c15ae50 0x7fa45c0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa463547640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0b0f50 con 0x7fa45c0cdc20 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa463547640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0c7730 con 0x7fa45c0ce060 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa463547640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c0c6bb0 con 0x7fa45c15ae50 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa4612e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 0x7fa45c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa4612e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 0x7fa45c15dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47458/0 (socket says 172.21.15.70:47458) 2023-08-25T09:39:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.873+0000 7fa4612e3640 1 -- 172.21.15.70:0/2265407075 learned_addr learned my addr 172.21.15.70:0/2265407075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa460ae2640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c0ce060 0x7fa45c15e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa4612e3640 1 -- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c15ae50 msgr2=0x7fa45c0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa4612e3640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c15ae50 0x7fa45c0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa4612e3640 1 -- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c0ce060 msgr2=0x7fa45c15e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:33.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa4612e3640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c0ce060 0x7fa45c15e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:33.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa4612e3640 1 -- 172.21.15.70:0/2265407075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009400 con 0x7fa45c0cdc20 2023-08-25T09:39:33.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa4612e3640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 0x7fa45c15dc50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa45802bf50 tx=0x7fa4580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:33.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45803e070 con 0x7fa45c0cdc20 2023-08-25T09:39:33.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580399d0 con 0x7fa45c0cdc20 2023-08-25T09:39:33.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458039d60 con 0x7fa45c0cdc20 2023-08-25T09:39:33.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.874+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c0c6e90 con 0x7fa45c0cdc20 2023-08-25T09:39:33.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.875+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c0c7450 con 0x7fa45c0cdc20 2023-08-25T09:39:33.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.876+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c1f3560 con 0x7fa45c0cdc20 2023-08-25T09:39:33.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.877+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa4580423f0 con 0x7fa45c0cdc20 2023-08-25T09:39:33.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.877+0000 7fa451ffb640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa44006a050 0x7fa44006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:33.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.878+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4580aa100 con 0x7fa45c0cdc20 2023-08-25T09:39:33.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.878+0000 7fa460ae2640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa44006a050 0x7fa44006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:33.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.878+0000 7fa460ae2640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa44006a050 0x7fa44006c510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0c2180 tx=0x7fa44c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:33.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:33.882+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa45c1f3560 con 0x7fa45c0cdc20 2023-08-25T09:39:34.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.071+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa45c066fb0 con 0x7fa45c0cdc20 2023-08-25T09:39:34.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.072+0000 7fa451ffb640 1 -- 172.21.15.70:0/2265407075 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa458075750 con 0x7fa45c0cdc20 2023-08-25T09:39:34.074 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:34.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa44006a050 msgr2=0x7fa44006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa44006a050 0x7fa44006c510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0c2180 tx=0x7fa44c008040 comp rx=0 tx=0).stop 2023-08-25T09:39:34.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 msgr2=0x7fa45c15dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 0x7fa45c15dc50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa45802bf50 tx=0x7fa4580046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 shutdown_connections 2023-08-25T09:39:34.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa44006a050 0x7fa44006c510 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa45c15ae50 0x7fa45c0b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa45c0ce060 0x7fa45c15e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 --2- 172.21.15.70:0/2265407075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa45c0cdc20 0x7fa45c15dc50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 >> 172.21.15.70:0/2265407075 conn(0x7fa45c0cb0d0 msgr2=0x7fa45c0ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:34.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.074+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 shutdown_connections 2023-08-25T09:39:34.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.075+0000 7fa463547640 1 -- 172.21.15.70:0/2265407075 wait complete. 2023-08-25T09:39:34.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.306+0000 7fec11142640 1 Processor -- start 2023-08-25T09:39:34.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.306+0000 7fec11142640 1 -- start start 2023-08-25T09:39:34.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.308+0000 7fec11142640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fec0c159bf0 0x7fec0c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.308+0000 7fec11142640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c0cd660 0x7fec0c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.308+0000 7fec11142640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec0c15e780 con 0x7fec0c06f0e0 2023-08-25T09:39:34.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.308+0000 7fec11142640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0c0ce030 con 0x7fec0c159bf0 2023-08-25T09:39:34.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.308+0000 7fec11142640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c06f8e0 con 0x7fec0c0cd660 2023-08-25T09:39:34.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.309+0000 7fec0b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c0cd660 0x7fec0c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.309+0000 7fec0ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fec0c159bf0 0x7fec0c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.309+0000 7fec03fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fec0c06f0e0 0x7fec0c070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:45372/0 (socket says 172.21.15.70:45372) 2023-08-25T09:39:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.309+0000 7fec03fff640 1 -- 172.21.15.70:0/1874806858 learned_addr learned my addr 172.21.15.70:0/1874806858 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.310+0000 7fec0ad9d640 1 -- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c0cd660 msgr2=0x7fec0c0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.310+0000 7fec0ad9d640 1 --2- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c0cd660 0x7fec0c0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.310+0000 7fec0ad9d640 1 -- 172.21.15.70:0/1874806858 >> v1:172.21.15.155:6789/0 conn(0x7fec0c06f0e0 legacy=0x7fec0c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.310+0000 7fec0ad9d640 1 -- 172.21.15.70:0/1874806858 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c06fbc0 con 0x7fec0c159bf0 2023-08-25T09:39:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.311+0000 7fec0ad9d640 1 --2- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c159bf0 0x7fec0c15e240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7febfc0097a0 tx=0x7febfc02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e361438a7fbc697 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.311+0000 7fec0a59c640 1 -- 172.21.15.70:0/1874806858 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc02d040 con 0x7fec0c159bf0 2023-08-25T09:39:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.311+0000 7fec0a59c640 1 -- 172.21.15.70:0/1874806858 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc031810 con 0x7fec0c159bf0 2023-08-25T09:39:34.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.311+0000 7fec0a59c640 1 -- 172.21.15.70:0/1874806858 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc031ba0 con 0x7fec0c159bf0 2023-08-25T09:39:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 -- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c159bf0 msgr2=0x7fec0c15e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 --2- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c159bf0 0x7fec0c15e240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7febfc0097a0 tx=0x7febfc02b8b0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 -- 172.21.15.70:0/1874806858 shutdown_connections 2023-08-25T09:39:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 --2- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c0cd660 0x7fec0c0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 --2- 172.21.15.70:0/1874806858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c159bf0 0x7fec0c15e240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 -- 172.21.15.70:0/1874806858 >> 172.21.15.70:0/1874806858 conn(0x7fec0c0cb0d0 msgr2=0x7fec0c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 -- 172.21.15.70:0/1874806858 shutdown_connections 2023-08-25T09:39:34.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.312+0000 7fec11142640 1 -- 172.21.15.70:0/1874806858 wait complete. 2023-08-25T09:39:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.314+0000 7fec11142640 1 Processor -- start 2023-08-25T09:39:34.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.314+0000 7fec11142640 1 -- start start 2023-08-25T09:39:34.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec11142640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c06f0e0 0x7fec0c155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec11142640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 0x7fec0c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec11142640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c159bf0 0x7fec0c1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec11142640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c155c90 con 0x7fec0c0cd660 2023-08-25T09:39:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec11142640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c155ee0 con 0x7fec0c06f0e0 2023-08-25T09:39:34.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec11142640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c1561c0 con 0x7fec0c159bf0 2023-08-25T09:39:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 0x7fec0c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 0x7fec0c155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47470/0 (socket says 172.21.15.70:47470) 2023-08-25T09:39:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.315+0000 7fec03fff640 1 -- 172.21.15.70:0/3561533078 learned_addr learned my addr 172.21.15.70:0/3561533078 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec0ad9d640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c06f0e0 0x7fec0c155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec03fff640 1 -- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c159bf0 msgr2=0x7fec0c1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec0b59e640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c159bf0 0x7fec0c1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec03fff640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c159bf0 0x7fec0c1542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec03fff640 1 -- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c06f0e0 msgr2=0x7fec0c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec03fff640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c06f0e0 0x7fec0c155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec03fff640 1 -- 172.21.15.70:0/3561533078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc009400 con 0x7fec0c0cd660 2023-08-25T09:39:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec0ad9d640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c06f0e0 0x7fec0c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:34.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec03fff640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 0x7fec0c155750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febf400c4a0 tx=0x7febf400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4005ce0 con 0x7fec0c0cd660 2023-08-25T09:39:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf4005e40 con 0x7fec0c0cd660 2023-08-25T09:39:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.316+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4007270 con 0x7fec0c0cd660 2023-08-25T09:39:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.317+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c0c5690 con 0x7fec0c0cd660 2023-08-25T09:39:34.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.317+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c0c5c50 con 0x7fec0c0cd660 2023-08-25T09:39:34.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.318+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c066fb0 con 0x7fec0c0cd660 2023-08-25T09:39:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.319+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7febf4007490 con 0x7fec0c0cd660 2023-08-25T09:39:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.320+0000 7fec037fe640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7febec06a050 0x7febec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.320+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febf40879d0 con 0x7fec0c0cd660 2023-08-25T09:39:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.321+0000 7fec0ad9d640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7febec06a050 0x7febec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.321+0000 7fec0ad9d640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7febec06a050 0x7febec06c510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febfc036000 tx=0x7febfc03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:34.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.325+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7febf4053200 con 0x7fec0c0cd660 2023-08-25T09:39:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.516+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec0c0c5e40 con 0x7fec0c0cd660 2023-08-25T09:39:34.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.517+0000 7fec037fe640 1 -- 172.21.15.70:0/3561533078 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7febf4056820 con 0x7fec0c0cd660 2023-08-25T09:39:34.520 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:34.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7febec06a050 msgr2=0x7febec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7febec06a050 0x7febec06c510 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7febfc036000 tx=0x7febfc03c040 comp rx=0 tx=0).stop 2023-08-25T09:39:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 msgr2=0x7fec0c155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 0x7fec0c155750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7febf400c4a0 tx=0x7febf400c960 comp rx=0 tx=0).stop 2023-08-25T09:39:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 shutdown_connections 2023-08-25T09:39:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7febec06a050 0x7febec06c510 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fec0c159bf0 0x7fec0c1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fec0c0cd660 0x7fec0c155750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 --2- 172.21.15.70:0/3561533078 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fec0c06f0e0 0x7fec0c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 >> 172.21.15.70:0/3561533078 conn(0x7fec0c0cb0d0 msgr2=0x7fec0c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 shutdown_connections 2023-08-25T09:39:34.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.520+0000 7fec11142640 1 -- 172.21.15.70:0/3561533078 wait complete. 2023-08-25T09:39:34.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:34.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.764+0000 7f728b96b640 1 Processor -- start 2023-08-25T09:39:34.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.764+0000 7f728b96b640 1 -- start start 2023-08-25T09:39:34.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f728b96b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7284153d70 0x7f7284072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f728b96b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72840b2870 0x7f7284072d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f728b96b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840acc70 0x7f72840ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f728b96b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7284154150 con 0x7f72840b2870 2023-08-25T09:39:34.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f728b96b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728415d340 con 0x7f7284153d70 2023-08-25T09:39:34.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f728b96b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728415d620 con 0x7f72840acc70 2023-08-25T09:39:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f7288f06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72840b2870 0x7f7284072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f7288f06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f72840b2870 0x7f7284072d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47478/0 (socket says 172.21.15.70:47478) 2023-08-25T09:39:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f7288f06640 1 -- 172.21.15.70:0/700892922 learned_addr learned my addr 172.21.15.70:0/700892922 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f7289707640 1 --2- 172.21.15.70:0/700892922 >> v2:172.21.15.155:3300/0 conn(0x7f7284153d70 0x7f7284072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.766+0000 7f7289f08640 1 --2- 172.21.15.70:0/700892922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840acc70 0x7f72840ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.767+0000 7f7288f06640 1 -- 172.21.15.70:0/700892922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840acc70 msgr2=0x7f72840ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:34.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.767+0000 7f7288f06640 1 --2- 172.21.15.70:0/700892922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840acc70 0x7f72840ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.767+0000 7f7288f06640 1 -- 172.21.15.70:0/700892922 >> v2:172.21.15.155:3300/0 conn(0x7f7284153d70 msgr2=0x7f7284072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:34.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.767+0000 7f7288f06640 1 --2- 172.21.15.70:0/700892922 >> v2:172.21.15.155:3300/0 conn(0x7f7284153d70 0x7f7284072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.767+0000 7f7288f06640 1 -- 172.21.15.70:0/700892922 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728415d900 con 0x7f72840b2870 2023-08-25T09:39:34.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.768+0000 7f7288f06640 1 --2- 172.21.15.70:0/700892922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f7284072d60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72840735f0 tx=0x7f727402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33532701e3000486 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:34.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f727bfff640 1 -- 172.21.15.70:0/700892922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72740317e0 con 0x7f72840b2870 2023-08-25T09:39:34.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f727bfff640 1 -- 172.21.15.70:0/700892922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7274031940 con 0x7f72840b2870 2023-08-25T09:39:34.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f727bfff640 1 -- 172.21.15.70:0/700892922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274031cd0 con 0x7f72840b2870 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 -- 172.21.15.70:0/700892922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 msgr2=0x7f7284072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 --2- 172.21.15.70:0/700892922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f7284072d60 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f72840735f0 tx=0x7f727402b880 comp rx=0 tx=0).stop 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 -- 172.21.15.70:0/700892922 shutdown_connections 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 --2- 172.21.15.70:0/700892922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72840acc70 0x7f72840ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 --2- 172.21.15.70:0/700892922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f7284072d60 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 --2- 172.21.15.70:0/700892922 >> v2:172.21.15.155:3300/0 conn(0x7f7284153d70 0x7f7284072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 -- 172.21.15.70:0/700892922 >> 172.21.15.70:0/700892922 conn(0x7f72840cfc70 msgr2=0x7f72840d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:34.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 -- 172.21.15.70:0/700892922 shutdown_connections 2023-08-25T09:39:34.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.769+0000 7f728b96b640 1 -- 172.21.15.70:0/700892922 wait complete. 2023-08-25T09:39:34.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.771+0000 7f728b96b640 1 Processor -- start 2023-08-25T09:39:34.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.771+0000 7f728b96b640 1 -- start start 2023-08-25T09:39:34.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f728b96b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840acc70 0x7f72840ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f728b96b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f72840cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f728b96b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7284153d70 0x7f728414fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f728b96b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72840cf2f0 con 0x7f72840b2870 2023-08-25T09:39:34.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f728b96b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7284151080 con 0x7f72840acc70 2023-08-25T09:39:34.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f728b96b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7284151360 con 0x7f7284153d70 2023-08-25T09:39:34.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f7288f06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f72840cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f7288f06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f72840cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47496/0 (socket says 172.21.15.70:47496) 2023-08-25T09:39:34.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.772+0000 7f7288f06640 1 -- 172.21.15.70:0/4253688080 learned_addr learned my addr 172.21.15.70:0/4253688080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:34.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7289707640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840acc70 0x7f72840ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7289f08640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7284153d70 0x7f728414fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7288f06640 1 -- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7284153d70 msgr2=0x7f728414fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7288f06640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7284153d70 0x7f728414fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7288f06640 1 -- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840acc70 msgr2=0x7f72840ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7288f06640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840acc70 0x7f72840ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7288f06640 1 -- 172.21.15.70:0/4253688080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740094d0 con 0x7f72840b2870 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7288f06640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f72840cb000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7274000c00 tx=0x7f72740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.773+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274032880 con 0x7f72840b2870 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.774+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7274032a20 con 0x7f72840b2870 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.774+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274032dd0 con 0x7f72840b2870 2023-08-25T09:39:34.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.774+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7284151640 con 0x7f72840b2870 2023-08-25T09:39:34.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.774+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72840cefe0 con 0x7f72840b2870 2023-08-25T09:39:34.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.775+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72840738b0 con 0x7f72840b2870 2023-08-25T09:39:34.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.776+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f727404e450 con 0x7f72840b2870 2023-08-25T09:39:34.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.777+0000 7f7279ffb640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f726806a050 0x7f726806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:34.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.777+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72740aa6a0 con 0x7f72840b2870 2023-08-25T09:39:34.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.777+0000 7f7289707640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f726806a050 0x7f726806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:34.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.778+0000 7f7289707640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f726806a050 0x7f726806c510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f72800099c0 tx=0x7f7280003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:34.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.781+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7274075e50 con 0x7f72840b2870 2023-08-25T09:39:34.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.973+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7284066fb0 con 0x7f72840b2870 2023-08-25T09:39:34.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.975+0000 7f7279ffb640 1 -- 172.21.15.70:0/4253688080 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7274079470 con 0x7f72840b2870 2023-08-25T09:39:34.978 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:34.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f726806a050 msgr2=0x7f726806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f726806a050 0x7f726806c510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f72800099c0 tx=0x7f7280003990 comp rx=0 tx=0).stop 2023-08-25T09:39:34.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 msgr2=0x7f72840cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:34.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f72840cb000 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7274000c00 tx=0x7f72740046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 shutdown_connections 2023-08-25T09:39:34.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f726806a050 0x7f726806c510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7284153d70 0x7f728414fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72840b2870 0x7f72840cb000 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.977+0000 7f728b96b640 1 --2- 172.21.15.70:0/4253688080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72840acc70 0x7f72840ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.978+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 >> 172.21.15.70:0/4253688080 conn(0x7f72840cfc70 msgr2=0x7f72840d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.978+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 shutdown_connections 2023-08-25T09:39:34.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:34.978+0000 7f728b96b640 1 -- 172.21.15.70:0/4253688080 wait complete. 2023-08-25T09:39:35.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:35.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.211+0000 7f8d98638640 1 Processor -- start 2023-08-25T09:39:35.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.211+0000 7f8d98638640 1 -- start start 2023-08-25T09:39:35.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d98638640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d900ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d98638640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d900b2b40 0x7f8d900ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d98638640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9006f050 con 0x7f8d900ac870 2023-08-25T09:39:35.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d98638640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d9006f330 con 0x7f8d900b2b40 2023-08-25T09:39:35.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d98638640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9006f610 con 0x7f8d9015cb40 2023-08-25T09:39:35.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d96bd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d900b2b40 0x7f8d900ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d96bd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d900b2b40 0x7f8d900ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47498/0 (socket says 172.21.15.70:47498) 2023-08-25T09:39:35.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.213+0000 7f8d96bd5640 1 -- 172.21.15.70:0/770661291 learned_addr learned my addr 172.21.15.70:0/770661291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:35.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.214+0000 7f8d963d4640 1 --2- 172.21.15.70:0/770661291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d900ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.214+0000 7f8d96bd5640 1 -- 172.21.15.70:0/770661291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 msgr2=0x7f8d900ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:35.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.214+0000 7f8d96bd5640 1 --2- 172.21.15.70:0/770661291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d900ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.214+0000 7f8d96bd5640 1 -- 172.21.15.70:0/770661291 >> v1:172.21.15.155:6789/0 conn(0x7f8d900ac870 legacy=0x7f8d90160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:35.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.214+0000 7f8d96bd5640 1 -- 172.21.15.70:0/770661291 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9006f8f0 con 0x7f8d900b2b40 2023-08-25T09:39:35.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.215+0000 7f8d96bd5640 1 --2- 172.21.15.70:0/770661291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900b2b40 0x7f8d900ce860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009870 tx=0x7f8d8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5973554cdffe719c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.215+0000 7f8d953d2640 1 -- 172.21.15.70:0/770661291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88031640 con 0x7f8d900b2b40 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.215+0000 7f8d953d2640 1 -- 172.21.15.70:0/770661291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d880317a0 con 0x7f8d900b2b40 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.215+0000 7f8d953d2640 1 -- 172.21.15.70:0/770661291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88031b30 con 0x7f8d900b2b40 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 -- 172.21.15.70:0/770661291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900b2b40 msgr2=0x7f8d900ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 --2- 172.21.15.70:0/770661291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900b2b40 0x7f8d900ce860 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8d88009870 tx=0x7f8d8802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 -- 172.21.15.70:0/770661291 shutdown_connections 2023-08-25T09:39:35.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 --2- 172.21.15.70:0/770661291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900b2b40 0x7f8d900ce860 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 --2- 172.21.15.70:0/770661291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d900ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 -- 172.21.15.70:0/770661291 >> 172.21.15.70:0/770661291 conn(0x7f8d90069850 msgr2=0x7f8d90069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:35.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 -- 172.21.15.70:0/770661291 shutdown_connections 2023-08-25T09:39:35.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.216+0000 7f8d98638640 1 -- 172.21.15.70:0/770661291 wait complete. 2023-08-25T09:39:35.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.218+0000 7f8d98638640 1 Processor -- start 2023-08-25T09:39:35.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.218+0000 7f8d98638640 1 -- start start 2023-08-25T09:39:35.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d98638640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 0x7f8d9014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d98638640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d900b2b40 0x7f8d9015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d98638640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d90160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d98638640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9014bfa0 con 0x7f8d900ac870 2023-08-25T09:39:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d98638640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901f2ef0 con 0x7f8d900b2b40 2023-08-25T09:39:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d98638640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9014d730 con 0x7f8d9015cb40 2023-08-25T09:39:35.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d963d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 0x7f8d9014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d963d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 0x7f8d9014b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47508/0 (socket says 172.21.15.70:47508) 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.219+0000 7f8d963d4640 1 -- 172.21.15.70:0/1977899784 learned_addr learned my addr 172.21.15.70:0/1977899784 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d963d4640 1 -- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 msgr2=0x7f8d90160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d963d4640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d90160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d963d4640 1 -- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d900b2b40 msgr2=0x7f8d9015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d95bd3640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d900b2b40 0x7f8d9015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d963d4640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d900b2b40 0x7f8d9015d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d963d4640 1 -- 172.21.15.70:0/1977899784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d880094d0 con 0x7f8d900ac870 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d963d4640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 0x7f8d9014b1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000d800 tx=0x7f8d8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000b6c0 con 0x7f8d900ac870 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8000b820 con 0x7f8d900ac870 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000bbb0 con 0x7f8d900ac870 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.220+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9014da10 con 0x7f8d900ac870 2023-08-25T09:39:35.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.221+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d901f31d0 con 0x7f8d900ac870 2023-08-25T09:39:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.222+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d9014d590 con 0x7f8d900ac870 2023-08-25T09:39:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.222+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8d8000bd10 con 0x7f8d900ac870 2023-08-25T09:39:35.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.223+0000 7f8d8effd640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d7c06a050 0x7f8d7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.224+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d80088bc0 con 0x7f8d900ac870 2023-08-25T09:39:35.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.224+0000 7f8d95bd3640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d7c06a050 0x7f8d7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.224+0000 7f8d95bd3640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d7c06a050 0x7f8d7c06c510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8d90154010 tx=0x7f8d84007240 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:35.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.228+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8d9014d590 con 0x7f8d900ac870 2023-08-25T09:39:35.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.419+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d9014d590 con 0x7f8d900ac870 2023-08-25T09:39:35.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.423+0000 7f8d8effd640 1 -- 172.21.15.70:0/1977899784 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f8d9014d590 con 0x7f8d900ac870 2023-08-25T09:39:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:35.434 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:35.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d7c06a050 msgr2=0x7f8d7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d7c06a050 0x7f8d7c06c510 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8d90154010 tx=0x7f8d84007240 comp rx=0 tx=0).stop 2023-08-25T09:39:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 msgr2=0x7f8d9014b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 0x7f8d9014b1e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000d800 tx=0x7f8d8000dcc0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 shutdown_connections 2023-08-25T09:39:35.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d7c06a050 0x7f8d7c06c510 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d9015cb40 0x7f8d90160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d900b2b40 0x7f8d9015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 --2- 172.21.15.70:0/1977899784 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d900ac870 0x7f8d9014b1e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.426+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 >> 172.21.15.70:0/1977899784 conn(0x7f8d90069850 msgr2=0x7f8d9006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.427+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 shutdown_connections 2023-08-25T09:39:35.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.427+0000 7f8d98638640 1 -- 172.21.15.70:0/1977899784 wait complete. 2023-08-25T09:39:35.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:35.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:35.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.658+0000 7ff761742640 1 Processor -- start 2023-08-25T09:39:35.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.659+0000 7ff761742640 1 -- start start 2023-08-25T09:39:35.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff761742640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff75c0cd850 0x7ff75c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff761742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff761742640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0b6cc0 con 0x7ff75c0b8510 2023-08-25T09:39:35.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff761742640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0ce170 con 0x7ff75c0cd850 2023-08-25T09:39:35.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff761742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0d1500 con 0x7ff75c15cfe0 2023-08-25T09:39:35.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff75c0b8510 0x7ff75c0b4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50072/0 (socket says 172.21.15.70:50072) 2023-08-25T09:39:35.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75b7fe640 1 -- 172.21.15.70:0/1506657328 learned_addr learned my addr 172.21.15.70:0/1506657328 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:35.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75affd640 1 --2- 172.21.15.70:0/1506657328 >> v2:172.21.15.70:3300/0 conn(0x7ff75c0cd850 0x7ff75c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff753fff640 1 --2- 172.21.15.70:0/1506657328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75a7fc640 1 -- 172.21.15.70:0/1506657328 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795432194 0 0) 0x7ff7540014e0 con 0x7ff75c0b8510 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75a7fc640 1 -- 172.21.15.70:0/1506657328 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff748003520 con 0x7ff75c0b8510 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75affd640 1 -- 172.21.15.70:0/1506657328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 msgr2=0x7ff75c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75affd640 1 --2- 172.21.15.70:0/1506657328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75affd640 1 -- 172.21.15.70:0/1506657328 >> v1:172.21.15.70:6789/0 conn(0x7ff75c0b8510 legacy=0x7ff75c0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.661+0000 7ff75affd640 1 -- 172.21.15.70:0/1506657328 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0d17e0 con 0x7ff75c0cd850 2023-08-25T09:39:35.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.662+0000 7ff75affd640 1 --2- 172.21.15.70:0/1506657328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0cd850 0x7ff75c0cdc30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0097a0 tx=0x7ff74c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649df3b2695e36e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff75a7fc640 1 -- 172.21.15.70:0/1506657328 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c031830 con 0x7ff75c0cd850 2023-08-25T09:39:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff75a7fc640 1 -- 172.21.15.70:0/1506657328 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c031990 con 0x7ff75c0cd850 2023-08-25T09:39:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff75a7fc640 1 -- 172.21.15.70:0/1506657328 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c031d00 con 0x7ff75c0cd850 2023-08-25T09:39:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 -- 172.21.15.70:0/1506657328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0cd850 msgr2=0x7ff75c0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 --2- 172.21.15.70:0/1506657328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0cd850 0x7ff75c0cdc30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0097a0 tx=0x7ff74c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 -- 172.21.15.70:0/1506657328 shutdown_connections 2023-08-25T09:39:35.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 --2- 172.21.15.70:0/1506657328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 --2- 172.21.15.70:0/1506657328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0cd850 0x7ff75c0cdc30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 -- 172.21.15.70:0/1506657328 >> 172.21.15.70:0/1506657328 conn(0x7ff75c069850 msgr2=0x7ff75c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:35.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 -- 172.21.15.70:0/1506657328 shutdown_connections 2023-08-25T09:39:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.663+0000 7ff761742640 1 -- 172.21.15.70:0/1506657328 wait complete. 2023-08-25T09:39:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.665+0000 7ff761742640 1 Processor -- start 2023-08-25T09:39:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.665+0000 7ff761742640 1 -- start start 2023-08-25T09:39:35.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff761742640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 0x7ff75c1f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff761742640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff75c0cd850 0x7ff75c148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff761742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff761742640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c148dd0 con 0x7ff75c0b8510 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff761742640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0b4b80 con 0x7ff75c0cd850 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff761742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0b2ab0 con 0x7ff75c15cfe0 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff75affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 0x7ff75c1f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff75affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 0x7ff75c1f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47536/0 (socket says 172.21.15.70:47536) 2023-08-25T09:39:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.666+0000 7ff75affd640 1 -- 172.21.15.70:0/2683790242 learned_addr learned my addr 172.21.15.70:0/2683790242 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff75affd640 1 -- 172.21.15.70:0/2683790242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 msgr2=0x7ff75c0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff75affd640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff75affd640 1 -- 172.21.15.70:0/2683790242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff75c0cd850 msgr2=0x7ff75c148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff75affd640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff75c0cd850 0x7ff75c148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff75affd640 1 -- 172.21.15.70:0/2683790242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c009400 con 0x7ff75c0b8510 2023-08-25T09:39:35.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff75affd640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 0x7ff75c1f6170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff74c02bee0 tx=0x7ff74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:35.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c03a6f0 con 0x7ff75c0b8510 2023-08-25T09:39:35.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.667+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c03a870 con 0x7ff75c0b8510 2023-08-25T09:39:35.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.668+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0b4250 con 0x7ff75c0b8510 2023-08-25T09:39:35.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.668+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c03ac20 con 0x7ff75c0b8510 2023-08-25T09:39:35.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.668+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0b4810 con 0x7ff75c0b8510 2023-08-25T09:39:35.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.669+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c0d22c0 con 0x7ff75c0b8510 2023-08-25T09:39:35.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.670+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff74c04c3b0 con 0x7ff75c0b8510 2023-08-25T09:39:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.671+0000 7ff7537fe640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff73c06a050 0x7ff73c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.671+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff74c0aa360 con 0x7ff75c0b8510 2023-08-25T09:39:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.671+0000 7ff753fff640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff73c06a050 0x7ff73c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:35.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.672+0000 7ff753fff640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff73c06a050 0x7ff73c06c510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff75c15e510 tx=0x7ff744008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:35.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.676+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff74c075b70 con 0x7ff75c0b8510 2023-08-25T09:39:35.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.864+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff75c06a700 con 0x7ff75c0b8510 2023-08-25T09:39:35.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.866+0000 7ff7537fe640 1 -- 172.21.15.70:0/2683790242 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff74c079190 con 0x7ff75c0b8510 2023-08-25T09:39:35.868 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:35.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff73c06a050 msgr2=0x7ff73c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff73c06a050 0x7ff73c06c510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff75c15e510 tx=0x7ff744008040 comp rx=0 tx=0).stop 2023-08-25T09:39:35.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 msgr2=0x7ff75c1f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:35.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 0x7ff75c1f6170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff74c02bee0 tx=0x7ff74c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 shutdown_connections 2023-08-25T09:39:35.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff73c06a050 0x7ff73c06c510 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff75c15cfe0 0x7ff75c0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff75c0cd850 0x7ff75c148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 --2- 172.21.15.70:0/2683790242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff75c0b8510 0x7ff75c1f6170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:35.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 >> 172.21.15.70:0/2683790242 conn(0x7ff75c069850 msgr2=0x7ff75c0d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:35.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 shutdown_connections 2023-08-25T09:39:35.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:35.868+0000 7ff761742640 1 -- 172.21.15.70:0/2683790242 wait complete. 2023-08-25T09:39:35.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:36.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.105+0000 7fe0a0fc4640 1 Processor -- start 2023-08-25T09:39:36.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.105+0000 7fe0a0fc4640 1 -- start start 2023-08-25T09:39:36.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe0a0fc4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe09c15ae50 0x7fe09c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe0a0fc4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c0cdc20 0x7fe09c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe0a0fc4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe09c0cc5f0 con 0x7fe09c0ce060 2023-08-25T09:39:36.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe0a0fc4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c070650 con 0x7fe09c15ae50 2023-08-25T09:39:36.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe0a0fc4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c070930 con 0x7fe09c0cdc20 2023-08-25T09:39:36.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe09c15ae50 0x7fe09c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe09c15ae50 0x7fe09c0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47550/0 (socket says 172.21.15.70:47550) 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 -- 172.21.15.70:0/2661801807 learned_addr learned my addr 172.21.15.70:0/2661801807 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe091d9b640 1 --2- 172.21.15.70:0/2661801807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c0cdc20 0x7fe09c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 -- 172.21.15.70:0/2661801807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c0cdc20 msgr2=0x7fe09c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 --2- 172.21.15.70:0/2661801807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c0cdc20 0x7fe09c069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 -- 172.21.15.70:0/2661801807 >> v1:172.21.15.155:6789/0 conn(0x7fe09c0ce060 legacy=0x7fe09c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.106+0000 7fe09a59c640 1 -- 172.21.15.70:0/2661801807 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c070c10 con 0x7fe09c15ae50 2023-08-25T09:39:36.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.108+0000 7fe09a59c640 1 --2- 172.21.15.70:0/2661801807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c15ae50 0x7fe09c0c2110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009870 tx=0x7fe08c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c7d11bb9c73d27 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:36.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.108+0000 7fe099d9b640 1 -- 172.21.15.70:0/2661801807 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c030840 con 0x7fe09c15ae50 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.108+0000 7fe099d9b640 1 -- 172.21.15.70:0/2661801807 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c0309a0 con 0x7fe09c15ae50 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.108+0000 7fe099d9b640 1 -- 172.21.15.70:0/2661801807 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c030d30 con 0x7fe09c15ae50 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.108+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/2661801807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c15ae50 msgr2=0x7fe09c0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.108+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/2661801807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c15ae50 0x7fe09c0c2110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009870 tx=0x7fe08c02b980 comp rx=0 tx=0).stop 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.109+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/2661801807 shutdown_connections 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.109+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/2661801807 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c0cdc20 0x7fe09c069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.109+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/2661801807 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c15ae50 0x7fe09c0c2110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.109+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/2661801807 >> 172.21.15.70:0/2661801807 conn(0x7fe09c0cb0d0 msgr2=0x7fe09c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:36.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.109+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/2661801807 shutdown_connections 2023-08-25T09:39:36.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.109+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/2661801807 wait complete. 2023-08-25T09:39:36.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.110+0000 7fe0a0fc4640 1 Processor -- start 2023-08-25T09:39:36.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.111+0000 7fe0a0fc4640 1 -- start start 2023-08-25T09:39:36.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.111+0000 7fe0a0fc4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 0x7fe09c1f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe0a0fc4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe09c0ce060 0x7fe09c156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe0a0fc4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c15ae50 0x7fe09c0c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe0a0fc4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c7940 con 0x7fe09c0cdc20 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe0a0fc4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c7c20 con 0x7fe09c0ce060 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe0a0fc4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0c7f00 con 0x7fe09c15ae50 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 0x7fe09c1f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 0x7fe09c1f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47556/0 (socket says 172.21.15.70:47556) 2023-08-25T09:39:36.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 -- 172.21.15.70:0/3668587981 learned_addr learned my addr 172.21.15.70:0/3668587981 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:36.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 -- 172.21.15.70:0/3668587981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c15ae50 msgr2=0x7fe09c0c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c15ae50 0x7fe09c0c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 -- 172.21.15.70:0/3668587981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe09c0ce060 msgr2=0x7fe09c156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe09c0ce060 0x7fe09c156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 -- 172.21.15.70:0/3668587981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0094d0 con 0x7fe09c0cdc20 2023-08-25T09:39:36.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.112+0000 7fe09a59c640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 0x7fe09c1f2fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009870 tx=0x7fe08c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:36.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.113+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c039a90 con 0x7fe09c0cdc20 2023-08-25T09:39:36.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.113+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c039bf0 con 0x7fe09c0cdc20 2023-08-25T09:39:36.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.113+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c0433f0 con 0x7fe09c0cdc20 2023-08-25T09:39:36.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.113+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c0c81e0 con 0x7fe09c0cdc20 2023-08-25T09:39:36.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.113+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c156fc0 con 0x7fe09c0cdc20 2023-08-25T09:39:36.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.114+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c0c8700 con 0x7fe09c0cdc20 2023-08-25T09:39:36.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.115+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe08c042390 con 0x7fe09c0cdc20 2023-08-25T09:39:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.116+0000 7fe0937fe640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe07c06a050 0x7fe07c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.116+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe08c0aa7d0 con 0x7fe09c0cdc20 2023-08-25T09:39:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.116+0000 7fe091d9b640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe07c06a050 0x7fe07c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.117+0000 7fe091d9b640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe07c06a050 0x7fe07c06c510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0c8a00 tx=0x7fe084008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:36.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.120+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe09c0c8700 con 0x7fe09c0cdc20 2023-08-25T09:39:36.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.330+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe09c0c8700 con 0x7fe09c0cdc20 2023-08-25T09:39:36.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.331+0000 7fe0937fe640 1 -- 172.21.15.70:0/3668587981 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe08c034020 con 0x7fe09c0cdc20 2023-08-25T09:39:36.333 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:36.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe07c06a050 msgr2=0x7fe07c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe07c06a050 0x7fe07c06c510 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe09c0c8a00 tx=0x7fe084008040 comp rx=0 tx=0).stop 2023-08-25T09:39:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 msgr2=0x7fe09c1f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 0x7fe09c1f2fe0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009870 tx=0x7fe08c004730 comp rx=0 tx=0).stop 2023-08-25T09:39:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 shutdown_connections 2023-08-25T09:39:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe07c06a050 0x7fe07c06c510 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe09c15ae50 0x7fe09c0c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe09c0ce060 0x7fe09c156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 --2- 172.21.15.70:0/3668587981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe09c0cdc20 0x7fe09c1f2fe0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 >> 172.21.15.70:0/3668587981 conn(0x7fe09c0cb0d0 msgr2=0x7fe09c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 shutdown_connections 2023-08-25T09:39:36.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.334+0000 7fe0a0fc4640 1 -- 172.21.15.70:0/3668587981 wait complete. 2023-08-25T09:39:36.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:36.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.562+0000 7fdb6c634640 1 Processor -- start 2023-08-25T09:39:36.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.562+0000 7fdb6c634640 1 -- start start 2023-08-25T09:39:36.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.563+0000 7fdb6c634640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 0x7fdb6415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.563+0000 7fdb6c634640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb6406f0e0 0x7fdb64070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.563+0000 7fdb6c634640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6415e780 con 0x7fdb640cd660 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.563+0000 7fdb6c634640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6406f8e0 con 0x7fdb6406f0e0 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.563+0000 7fdb6c634640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6406fbc0 con 0x7fdb64159bf0 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.564+0000 7fdb6abd1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdb640cd660 0x7fdb640c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50074/0 (socket says 172.21.15.70:50074) 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.564+0000 7fdb6abd1640 1 -- 172.21.15.70:0/450580406 learned_addr learned my addr 172.21.15.70:0/450580406 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.564+0000 7fdb6a3d0640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 0x7fdb6415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.564+0000 7fdb69bcf640 1 --2- 172.21.15.70:0/450580406 >> v2:172.21.15.155:3300/0 conn(0x7fdb6406f0e0 0x7fdb64070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.564+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052095549 0 0) 0x7fdb6415e780 con 0x7fdb640cd660 2023-08-25T09:39:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.564+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb50005730 con 0x7fdb640cd660 2023-08-25T09:39:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3956878472 0 0) 0x7fdb50005730 con 0x7fdb640cd660 2023-08-25T09:39:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 msgr2=0x7fdb6415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 0x7fdb6415e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 >> v2:172.21.15.155:3300/0 conn(0x7fdb6406f0e0 msgr2=0x7fdb64070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:36.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 --2- 172.21.15.70:0/450580406 >> v2:172.21.15.155:3300/0 conn(0x7fdb6406f0e0 0x7fdb64070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6406fea0 con 0x7fdb640cd660 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fdb5c0019f0 con 0x7fdb640cd660 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb69bcf640 1 --2- 172.21.15.70:0/450580406 >> v2:172.21.15.155:3300/0 conn(0x7fdb6406f0e0 0x7fdb64070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 >> v1:172.21.15.70:6789/0 conn(0x7fdb640cd660 legacy=0x7fdb640c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 0x7fdb50005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb50006100 0x7fdb500069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb50007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6406f8e0 con 0x7fdb50006f10 2023-08-25T09:39:36.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6406fbc0 con 0x7fdb50006100 2023-08-25T09:39:36.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5000dbe0 con 0x7fdb64159bf0 2023-08-25T09:39:36.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.565+0000 7fdb6abd1640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb50007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.566+0000 7fdb6a3d0640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 0x7fdb50005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.566+0000 7fdb6abd1640 1 -- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 msgr2=0x7fdb50005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:36.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.566+0000 7fdb6abd1640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb64159bf0 0x7fdb50005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.566+0000 7fdb6abd1640 1 -- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb50006100 msgr2=0x7fdb500069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:36.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.566+0000 7fdb6abd1640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb50006100 0x7fdb500069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.566+0000 7fdb6abd1640 1 -- 172.21.15.70:0/450580406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50005730 con 0x7fdb50006f10 2023-08-25T09:39:36.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb6abd1640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb50007390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00a860 tx=0x7fdb50007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3742b6db1d04b01 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:36.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c034630 con 0x7fdb50006f10 2023-08-25T09:39:36.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c02f070 con 0x7fdb50006f10 2023-08-25T09:39:36.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb693ce640 1 -- 172.21.15.70:0/450580406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c034a50 con 0x7fdb50006f10 2023-08-25T09:39:36.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb6c634640 1 -- 172.21.15.70:0/450580406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 msgr2=0x7fdb50007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb6c634640 1 --2- 172.21.15.70:0/450580406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb50007390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00a860 tx=0x7fdb50007a00 comp rx=0 tx=0).stop 2023-08-25T09:39:36.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb6abd1640 1 -- 172.21.15.70:0/450580406 reap_dead start 2023-08-25T09:39:36.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb6c634640 1 -- 172.21.15.70:0/450580406 shutdown_connections 2023-08-25T09:39:36.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.567+0000 7fdb6c634640 1 -- 172.21.15.70:0/450580406 >> 172.21.15.70:0/450580406 conn(0x7fdb640cb0d0 msgr2=0x7fdb640cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:36.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.568+0000 7fdb6c634640 1 -- 172.21.15.70:0/450580406 shutdown_connections 2023-08-25T09:39:36.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.568+0000 7fdb6c634640 1 -- 172.21.15.70:0/450580406 wait complete. 2023-08-25T09:39:36.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.569+0000 7fdb6c634640 1 Processor -- start 2023-08-25T09:39:36.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.570+0000 7fdb6c634640 1 -- start start 2023-08-25T09:39:36.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.570+0000 7fdb6c634640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb641e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6c634640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb640c7e90 0x7fdb641f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6c634640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb640c8310 0x7fdb641f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6c634640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb641e6710 con 0x7fdb50006f10 2023-08-25T09:39:36.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6c634640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb641e69f0 con 0x7fdb640c8310 2023-08-25T09:39:36.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6c634640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb641e6cd0 con 0x7fdb640c7e90 2023-08-25T09:39:36.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb641e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb641e5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47582/0 (socket says 172.21.15.70:47582) 2023-08-25T09:39:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 -- 172.21.15.70:0/2779586154 learned_addr learned my addr 172.21.15.70:0/2779586154 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 -- 172.21.15.70:0/2779586154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb640c7e90 msgr2=0x7fdb641f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb640c7e90 0x7fdb641f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 -- 172.21.15.70:0/2779586154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb640c8310 msgr2=0x7fdb641f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb640c8310 0x7fdb641f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.571+0000 7fdb6a3d0640 1 -- 172.21.15.70:0/2779586154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb641e6fb0 con 0x7fdb50006f10 2023-08-25T09:39:36.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.572+0000 7fdb6a3d0640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb641e5e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb60010860 tx=0x7fdb60010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:36.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.572+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb60009610 con 0x7fdb50006f10 2023-08-25T09:39:36.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.572+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb6000eae0 con 0x7fdb50006f10 2023-08-25T09:39:36.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.572+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb641e7290 con 0x7fdb50006f10 2023-08-25T09:39:36.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.572+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb6000ee70 con 0x7fdb50006f10 2023-08-25T09:39:36.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.572+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb641e6340 con 0x7fdb50006f10 2023-08-25T09:39:36.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.573+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb64070e80 con 0x7fdb50006f10 2023-08-25T09:39:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.574+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdb6001c400 con 0x7fdb50006f10 2023-08-25T09:39:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.575+0000 7fdb5affd640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb4806a080 0x7fdb4806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.575+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb60088ab0 con 0x7fdb50006f10 2023-08-25T09:39:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.575+0000 7fdb69bcf640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb4806a080 0x7fdb4806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:36.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.576+0000 7fdb69bcf640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb4806a080 0x7fdb4806c540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb54006fd0 tx=0x7fdb54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:36.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.579+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdb60054250 con 0x7fdb50006f10 2023-08-25T09:39:36.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.766+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb641e7990 con 0x7fdb50006f10 2023-08-25T09:39:36.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.767+0000 7fdb5affd640 1 -- 172.21.15.70:0/2779586154 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb60057870 con 0x7fdb50006f10 2023-08-25T09:39:36.769 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:36.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb4806a080 msgr2=0x7fdb4806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb4806a080 0x7fdb4806c540 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdb54006fd0 tx=0x7fdb54008040 comp rx=0 tx=0).stop 2023-08-25T09:39:36.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 msgr2=0x7fdb641e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb641e5e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb60010860 tx=0x7fdb60010d20 comp rx=0 tx=0).stop 2023-08-25T09:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 shutdown_connections 2023-08-25T09:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb4806a080 0x7fdb4806c540 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb640c8310 0x7fdb641f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.769+0000 7fdb6c634640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb640c7e90 0x7fdb641f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.770+0000 7fdb6c634640 1 --2- 172.21.15.70:0/2779586154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb50006f10 0x7fdb641e5e00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.770+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 >> 172.21.15.70:0/2779586154 conn(0x7fdb640cb0d0 msgr2=0x7fdb640c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.770+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 shutdown_connections 2023-08-25T09:39:36.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:36.770+0000 7fdb6c634640 1 -- 172.21.15.70:0/2779586154 wait complete. 2023-08-25T09:39:36.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:37.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.006+0000 7f2082586640 1 Processor -- start 2023-08-25T09:39:37.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.006+0000 7f2082586640 1 -- start start 2023-08-25T09:39:37.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2082586640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f207c069850 0x7f207c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2082586640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f207c0c9620 0x7f207c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2082586640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207c0ac880 con 0x7f207c160be0 2023-08-25T09:39:37.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2082586640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0ad400 con 0x7f207c0c9620 2023-08-25T09:39:37.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2082586640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0ad890 con 0x7f207c069850 2023-08-25T09:39:37.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2073fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f207c0c9620 0x7f207c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2080b23640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f207c160be0 0x7f207c161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50080/0 (socket says 172.21.15.70:50080) 2023-08-25T09:39:37.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.008+0000 7f2080b23640 1 -- 172.21.15.70:0/1475963723 learned_addr learned my addr 172.21.15.70:0/1475963723 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:37.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.009+0000 7f207bfff640 1 --2- 172.21.15.70:0/1475963723 >> v2:172.21.15.155:3300/0 conn(0x7f207c069850 0x7f207c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.009+0000 7f2073fff640 1 -- 172.21.15.70:0/1475963723 >> v2:172.21.15.155:3300/0 conn(0x7f207c069850 msgr2=0x7f207c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.009+0000 7f2073fff640 1 --2- 172.21.15.70:0/1475963723 >> v2:172.21.15.155:3300/0 conn(0x7f207c069850 0x7f207c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.009+0000 7f2073fff640 1 -- 172.21.15.70:0/1475963723 >> v1:172.21.15.70:6789/0 conn(0x7f207c160be0 legacy=0x7f207c161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.009+0000 7f2073fff640 1 -- 172.21.15.70:0/1475963723 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0acad0 con 0x7f207c0c9620 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.009+0000 7f207b7fe640 1 -- 172.21.15.70:0/1475963723 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726141104 0 0) 0x7f207c0ac880 con 0x7f207c160be0 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.010+0000 7f2073fff640 1 --2- 172.21.15.70:0/1475963723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c069c30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f207c0cc0d0 tx=0x7f206402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d92fab7c2ca07c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.010+0000 7f207b7fe640 1 -- 172.21.15.70:0/1475963723 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064030610 con 0x7f207c0c9620 2023-08-25T09:39:37.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.010+0000 7f207b7fe640 1 -- 172.21.15.70:0/1475963723 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064030770 con 0x7f207c0c9620 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.010+0000 7f207b7fe640 1 -- 172.21.15.70:0/1475963723 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064030b00 con 0x7f207c0c9620 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 -- 172.21.15.70:0/1475963723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 msgr2=0x7f207c069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 --2- 172.21.15.70:0/1475963723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c069c30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f207c0cc0d0 tx=0x7f206402b960 comp rx=0 tx=0).stop 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 -- 172.21.15.70:0/1475963723 shutdown_connections 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 --2- 172.21.15.70:0/1475963723 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c069c30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 --2- 172.21.15.70:0/1475963723 >> v2:172.21.15.155:3300/0 conn(0x7f207c069850 0x7f207c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 -- 172.21.15.70:0/1475963723 >> 172.21.15.70:0/1475963723 conn(0x7f207c0cb0d0 msgr2=0x7f207c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:37.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 -- 172.21.15.70:0/1475963723 shutdown_connections 2023-08-25T09:39:37.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.011+0000 7f2082586640 1 -- 172.21.15.70:0/1475963723 wait complete. 2023-08-25T09:39:37.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.012+0000 7f2082586640 1 Processor -- start 2023-08-25T09:39:37.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.013+0000 7f2082586640 1 -- start start 2023-08-25T09:39:37.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.013+0000 7f2082586640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f207c069850 0x7f207c1f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2082586640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2082586640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f207c160be0 0x7f207c1f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2082586640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c1f4050 con 0x7f207c0c9620 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2082586640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0b0980 con 0x7f207c069850 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2082586640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0c5690 con 0x7f207c160be0 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c0b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47608/0 (socket says 172.21.15.70:47608) 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 -- 172.21.15.70:0/282428346 learned_addr learned my addr 172.21.15.70:0/282428346 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f207bfff640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f207c069850 0x7f207c1f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 -- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f207c160be0 msgr2=0x7f207c1f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f207c160be0 0x7f207c1f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 -- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f207c069850 msgr2=0x7f207c1f3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f207c069850 0x7f207c1f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.014+0000 7f2073fff640 1 -- 172.21.15.70:0/282428346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064009400 con 0x7f207c0c9620 2023-08-25T09:39:37.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.015+0000 7f2073fff640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c0b0080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2064009890 tx=0x7f20640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.015+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064039c90 con 0x7f207c0c9620 2023-08-25T09:39:37.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.015+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206403f040 con 0x7f207c0c9620 2023-08-25T09:39:37.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.015+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207c0c5970 con 0x7f207c0c9620 2023-08-25T09:39:37.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.015+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064006e10 con 0x7f207c0c9620 2023-08-25T09:39:37.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.015+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c0c5f30 con 0x7f207c0c9620 2023-08-25T09:39:37.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.016+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c066fb0 con 0x7f207c0c9620 2023-08-25T09:39:37.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.017+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2064004030 con 0x7f207c0c9620 2023-08-25T09:39:37.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.019+0000 7f20797fa640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f205c06a080 0x7f205c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.019+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f20640aa810 con 0x7f207c0c9620 2023-08-25T09:39:37.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.019+0000 7f207bfff640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f205c06a080 0x7f205c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.019+0000 7f207bfff640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f205c06a080 0x7f205c06c540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f206c006fd0 tx=0x7f206c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.023+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2064075f80 con 0x7f207c0c9620 2023-08-25T09:39:37.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.209+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f207c06a810 con 0x7f207c0c9620 2023-08-25T09:39:37.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.209+0000 7f20797fa640 1 -- 172.21.15.70:0/282428346 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20640795a0 con 0x7f207c0c9620 2023-08-25T09:39:37.211 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:37.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f205c06a080 msgr2=0x7f205c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f205c06a080 0x7f205c06c540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f206c006fd0 tx=0x7f206c008040 comp rx=0 tx=0).stop 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 msgr2=0x7f207c0b0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c0b0080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2064009890 tx=0x7f20640046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 shutdown_connections 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f205c06a080 0x7f205c06c540 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f207c160be0 0x7f207c1f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f207c0c9620 0x7f207c0b0080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 --2- 172.21.15.70:0/282428346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f207c069850 0x7f207c1f3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 >> 172.21.15.70:0/282428346 conn(0x7f207c0cb0d0 msgr2=0x7f207c15f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:37.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 shutdown_connections 2023-08-25T09:39:37.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.212+0000 7f2082586640 1 -- 172.21.15.70:0/282428346 wait complete. 2023-08-25T09:39:37.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:37.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.442+0000 7f632f408640 1 Processor -- start 2023-08-25T09:39:37.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.442+0000 7f632f408640 1 -- start start 2023-08-25T09:39:37.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632f408640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f632815a7c0 0x7f63280b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632f408640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f63280b12f0 0x7f632815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632f408640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632815e7c0 con 0x7f632815f190 2023-08-25T09:39:37.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632f408640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6328071b50 con 0x7f63280b12f0 2023-08-25T09:39:37.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632f408640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632815e490 con 0x7f632815a7c0 2023-08-25T09:39:37.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632ec07640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f632815f190 0x7f63280cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50090/0 (socket says 172.21.15.70:50090) 2023-08-25T09:39:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632ec07640 1 -- 172.21.15.70:0/2186082630 learned_addr learned my addr 172.21.15.70:0/2186082630 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.444+0000 7f632dc05640 1 --2- 172.21.15.70:0/2186082630 >> v2:172.21.15.70:3300/0 conn(0x7f63280b12f0 0x7f632815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632e406640 1 --2- 172.21.15.70:0/2186082630 >> v2:172.21.15.155:3300/0 conn(0x7f632815a7c0 0x7f63280b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632d404640 1 -- 172.21.15.70:0/2186082630 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45585677 0 0) 0x7f632815e7c0 con 0x7f632815f190 2023-08-25T09:39:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632d404640 1 -- 172.21.15.70:0/2186082630 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6314003520 con 0x7f632815f190 2023-08-25T09:39:37.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632dc05640 1 -- 172.21.15.70:0/2186082630 >> v2:172.21.15.155:3300/0 conn(0x7f632815a7c0 msgr2=0x7f63280b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632dc05640 1 --2- 172.21.15.70:0/2186082630 >> v2:172.21.15.155:3300/0 conn(0x7f632815a7c0 0x7f63280b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632dc05640 1 -- 172.21.15.70:0/2186082630 >> v1:172.21.15.70:6789/0 conn(0x7f632815f190 legacy=0x7f63280cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632dc05640 1 -- 172.21.15.70:0/2186082630 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280b3aa0 con 0x7f63280b12f0 2023-08-25T09:39:37.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.445+0000 7f632e406640 1 --2- 172.21.15.70:0/2186082630 >> v2:172.21.15.155:3300/0 conn(0x7f632815a7c0 0x7f63280b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.446+0000 7f632dc05640 1 --2- 172.21.15.70:0/2186082630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f632815ec50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63280ce280 tx=0x7f631802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe7b22aa128f37d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.446+0000 7f632d404640 1 -- 172.21.15.70:0/2186082630 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318031830 con 0x7f63280b12f0 2023-08-25T09:39:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632d404640 1 -- 172.21.15.70:0/2186082630 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6318031990 con 0x7f63280b12f0 2023-08-25T09:39:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632d404640 1 -- 172.21.15.70:0/2186082630 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318031d00 con 0x7f63280b12f0 2023-08-25T09:39:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 -- 172.21.15.70:0/2186082630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 msgr2=0x7f632815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 --2- 172.21.15.70:0/2186082630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f632815ec50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63280ce280 tx=0x7f631802b8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 -- 172.21.15.70:0/2186082630 shutdown_connections 2023-08-25T09:39:37.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 --2- 172.21.15.70:0/2186082630 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f632815ec50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 --2- 172.21.15.70:0/2186082630 >> v2:172.21.15.155:3300/0 conn(0x7f632815a7c0 0x7f63280b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 -- 172.21.15.70:0/2186082630 >> 172.21.15.70:0/2186082630 conn(0x7f63280affd0 msgr2=0x7f63280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 -- 172.21.15.70:0/2186082630 shutdown_connections 2023-08-25T09:39:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.447+0000 7f632f408640 1 -- 172.21.15.70:0/2186082630 wait complete. 2023-08-25T09:39:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.449+0000 7f632f408640 1 Processor -- start 2023-08-25T09:39:37.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.449+0000 7f632f408640 1 -- start start 2023-08-25T09:39:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632f408640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f63281f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632f408640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f632815a7c0 0x7f63281f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632f408640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632815f190 0x7f63280c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632f408640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280c8fd0 con 0x7f63280b12f0 2023-08-25T09:39:37.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632f408640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280cde40 con 0x7f632815a7c0 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632f408640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280cdb10 con 0x7f632815f190 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632e406640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f63281f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632e406640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f63281f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47626/0 (socket says 172.21.15.70:47626) 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.450+0000 7f632e406640 1 -- 172.21.15.70:0/26182472 learned_addr learned my addr 172.21.15.70:0/26182472 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632e406640 1 -- 172.21.15.70:0/26182472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632815f190 msgr2=0x7f63280c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632e406640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632815f190 0x7f63280c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632e406640 1 -- 172.21.15.70:0/26182472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f632815a7c0 msgr2=0x7f63281f4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632e406640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f632815a7c0 0x7f63281f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632e406640 1 -- 172.21.15.70:0/26182472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6318009400 con 0x7f63280b12f0 2023-08-25T09:39:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632e406640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f63281f2800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f632400c7f0 tx=0x7f632400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324007a30 con 0x7f63280b12f0 2023-08-25T09:39:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324007b90 con 0x7f63280b12f0 2023-08-25T09:39:37.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63280c8140 con 0x7f63280b12f0 2023-08-25T09:39:37.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.451+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632400ce40 con 0x7f63280b12f0 2023-08-25T09:39:37.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.452+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63281f5070 con 0x7f63280b12f0 2023-08-25T09:39:37.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.453+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6328066fb0 con 0x7f63280b12f0 2023-08-25T09:39:37.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.453+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f63240113f0 con 0x7f63280b12f0 2023-08-25T09:39:37.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.455+0000 7f631effd640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f630c06a050 0x7f630c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.455+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6324088780 con 0x7f63280b12f0 2023-08-25T09:39:37.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.455+0000 7f632dc05640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f630c06a050 0x7f630c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.456+0000 7f632dc05640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f630c06a050 0x7f630c06c510 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6318036000 tx=0x7f6318030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.460+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6324053dd0 con 0x7f63280b12f0 2023-08-25T09:39:37.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.648+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63280b0d00 con 0x7f63280b12f0 2023-08-25T09:39:37.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.650+0000 7f631effd640 1 -- 172.21.15.70:0/26182472 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f63240573f0 con 0x7f63280b12f0 2023-08-25T09:39:37.652 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:37.652 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:37.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f630c06a050 msgr2=0x7f630c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f630c06a050 0x7f630c06c510 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6318036000 tx=0x7f6318030000 comp rx=0 tx=0).stop 2023-08-25T09:39:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 msgr2=0x7f63281f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f63281f2800 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f632400c7f0 tx=0x7f632400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 shutdown_connections 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f630c06a050 0x7f630c06c510 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f632815f190 0x7f63280c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f632815a7c0 0x7f63281f4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 --2- 172.21.15.70:0/26182472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63280b12f0 0x7f63281f2800 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 >> 172.21.15.70:0/26182472 conn(0x7f63280affd0 msgr2=0x7f63280adee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 shutdown_connections 2023-08-25T09:39:37.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.652+0000 7f632f408640 1 -- 172.21.15.70:0/26182472 wait complete. 2023-08-25T09:39:37.673 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:37.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.881+0000 7f73ede5e640 1 Processor -- start 2023-08-25T09:39:37.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.881+0000 7f73ede5e640 1 -- start start 2023-08-25T09:39:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.882+0000 7f73ede5e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73e815ae50 0x7f73e80c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.882+0000 7f73ede5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 0x7f73e8069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.882+0000 7f73ede5e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e80cc5f0 con 0x7f73e80ce060 2023-08-25T09:39:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.882+0000 7f73ede5e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e8070650 con 0x7f73e815ae50 2023-08-25T09:39:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73ede5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8070930 con 0x7f73e80cdc20 2023-08-25T09:39:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73e7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f73e80ce060 0x7f73e80ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50104/0 (socket says 172.21.15.70:50104) 2023-08-25T09:39:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73e7fff640 1 -- 172.21.15.70:0/2995048672 learned_addr learned my addr 172.21.15.70:0/2995048672 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73e77fe640 1 --2- 172.21.15.70:0/2995048672 >> v2:172.21.15.155:3300/0 conn(0x7f73e815ae50 0x7f73e80c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324748116 0 0) 0x7f73e80cc5f0 con 0x7f73e80ce060 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73e6ffd640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 0x7f73e8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.883+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d0003520 con 0x7f73e80ce060 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1024166555 0 0) 0x7f73d0003520 con 0x7f73e80ce060 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 msgr2=0x7f73e8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 0x7f73e8069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 >> v2:172.21.15.155:3300/0 conn(0x7f73e815ae50 msgr2=0x7f73e80c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:37.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 --2- 172.21.15.70:0/2995048672 >> v2:172.21.15.155:3300/0 conn(0x7f73e815ae50 0x7f73e80c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8070c10 con 0x7f73e80ce060 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f73e00019f0 con 0x7f73e80ce060 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 >> v1:172.21.15.70:6789/0 conn(0x7f73e80ce060 legacy=0x7f73e80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0003970 0x7f73d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 0x7f73d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73d0004d00 0x7f73d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8070650 con 0x7f73d0004d00 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8070930 con 0x7f73d0003970 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d000b980 con 0x7f73e80cdc20 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.884+0000 7f73e7fff640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73d0004d00 0x7f73d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e77fe640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0003970 0x7f73d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e6ffd640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 0x7f73d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e7fff640 1 -- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 msgr2=0x7f73d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:37.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e7fff640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80cdc20 0x7f73d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e7fff640 1 -- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0003970 msgr2=0x7f73d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:37.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e7fff640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0003970 0x7f73d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.885+0000 7f73e7fff640 1 -- 172.21.15.70:0/2995048672 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0003520 con 0x7f73d0004d00 2023-08-25T09:39:37.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.886+0000 7f73e7fff640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73d0004d00 0x7f73d0005180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f73e80ce4f0 tx=0x7f73d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75af262ca52b8f52 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.886+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0034630 con 0x7f73d0004d00 2023-08-25T09:39:37.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.886+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e002f070 con 0x7f73d0004d00 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.886+0000 7f73e67fc640 1 -- 172.21.15.70:0/2995048672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e00349d0 con 0x7f73d0004d00 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73ede5e640 1 -- 172.21.15.70:0/2995048672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73d0004d00 msgr2=0x7f73d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73ede5e640 1 --2- 172.21.15.70:0/2995048672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73d0004d00 0x7f73d0005180 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f73e80ce4f0 tx=0x7f73d00057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73e7fff640 1 -- 172.21.15.70:0/2995048672 reap_dead start 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73ede5e640 1 -- 172.21.15.70:0/2995048672 shutdown_connections 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73ede5e640 1 -- 172.21.15.70:0/2995048672 >> 172.21.15.70:0/2995048672 conn(0x7f73e80cb0d0 msgr2=0x7f73e80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73ede5e640 1 -- 172.21.15.70:0/2995048672 shutdown_connections 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.887+0000 7f73ede5e640 1 -- 172.21.15.70:0/2995048672 wait complete. 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.889+0000 7f73ede5e640 1 Processor -- start 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.889+0000 7f73ede5e640 1 -- start start 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73ede5e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0004d00 0x7f73e815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73ede5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 0x7f73e8158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73ede5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80c6010 0x7f73e8158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73ede5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80c8a10 con 0x7f73e80b3b90 2023-08-25T09:39:37.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73ede5e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e815e5e0 con 0x7f73d0004d00 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73ede5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e815b3b0 con 0x7f73e80c6010 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73e6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 0x7f73e8158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73e6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 0x7f73e8158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47634/0 (socket says 172.21.15.70:47634) 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73e6ffd640 1 -- 172.21.15.70:0/3236535177 learned_addr learned my addr 172.21.15.70:0/3236535177 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.890+0000 7f73e77fe640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0004d00 0x7f73e815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e7fff640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80c6010 0x7f73e8158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e6ffd640 1 -- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80c6010 msgr2=0x7f73e8158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e6ffd640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80c6010 0x7f73e8158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e6ffd640 1 -- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0004d00 msgr2=0x7f73e815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:37.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e6ffd640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0004d00 0x7f73e815e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:37.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e6ffd640 1 -- 172.21.15.70:0/3236535177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80b4100 con 0x7f73e80b3b90 2023-08-25T09:39:37.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e7fff640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80c6010 0x7f73e8158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:37.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73e6ffd640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 0x7f73e8158610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73dc011600 tx=0x7f73dc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00f920 con 0x7f73e80b3b90 2023-08-25T09:39:37.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc00fa80 con 0x7f73e80b3b90 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc00fdf0 con 0x7f73e80b3b90 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.891+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e80c7f30 con 0x7f73e80b3b90 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.892+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e80c84f0 con 0x7f73e80b3b90 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.893+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8066fb0 con 0x7f73e80b3b90 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.894+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73dc015460 con 0x7f73e80b3b90 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.895+0000 7f73cffff640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73d406a080 0x7f73d406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:37.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.895+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73dc088a80 con 0x7f73e80b3b90 2023-08-25T09:39:37.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.895+0000 7f73e77fe640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73d406a080 0x7f73d406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:37.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.896+0000 7f73e77fe640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73d406a080 0x7f73d406c540 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009a10 tx=0x7f73d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:37.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:37.899+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73dc054170 con 0x7f73e80b3b90 2023-08-25T09:39:38.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.084+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73e815ca10 con 0x7f73e80b3b90 2023-08-25T09:39:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.085+0000 7f73cffff640 1 -- 172.21.15.70:0/3236535177 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73dc057790 con 0x7f73e80b3b90 2023-08-25T09:39:38.087 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73d406a080 msgr2=0x7f73d406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73d406a080 0x7f73d406c540 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009a10 tx=0x7f73d8008040 comp rx=0 tx=0).stop 2023-08-25T09:39:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 msgr2=0x7f73e8158610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 0x7f73e8158610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f73dc011600 tx=0x7f73dc011ac0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 shutdown_connections 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73d406a080 0x7f73d406c540 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73e80c6010 0x7f73e8158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73e80b3b90 0x7f73e8158610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 --2- 172.21.15.70:0/3236535177 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73d0004d00 0x7f73e815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.087+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 >> 172.21.15.70:0/3236535177 conn(0x7f73e80cb0d0 msgr2=0x7f73e806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.088+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 shutdown_connections 2023-08-25T09:39:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.088+0000 7f73ede5e640 1 -- 172.21.15.70:0/3236535177 wait complete. 2023-08-25T09:39:38.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:38.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.317+0000 7f558975d640 1 Processor -- start 2023-08-25T09:39:38.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.318+0000 7f558975d640 1 -- start start 2023-08-25T09:39:38.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.319+0000 7f558975d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f558415cfe0 0x7f55840d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.319+0000 7f558975d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f55840b8510 0x7f55840b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.319+0000 7f558975d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55840d1500 con 0x7f55840cd850 2023-08-25T09:39:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.319+0000 7f558975d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55840d17e0 con 0x7f558415cfe0 2023-08-25T09:39:38.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.319+0000 7f558975d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55840d1ac0 con 0x7f55840b8510 2023-08-25T09:39:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f558415cfe0 0x7f55840d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f558415cfe0 0x7f55840d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47636/0 (socket says 172.21.15.70:47636) 2023-08-25T09:39:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 -- 172.21.15.70:0/3052722238 learned_addr learned my addr 172.21.15.70:0/3052722238 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:38.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55837fe640 1 --2- 172.21.15.70:0/3052722238 >> v2:172.21.15.155:3300/0 conn(0x7f55840b8510 0x7f55840b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 -- 172.21.15.70:0/3052722238 >> v2:172.21.15.155:3300/0 conn(0x7f55840b8510 msgr2=0x7f55840b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:38.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 --2- 172.21.15.70:0/3052722238 >> v2:172.21.15.155:3300/0 conn(0x7f55840b8510 0x7f55840b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 -- 172.21.15.70:0/3052722238 >> v1:172.21.15.155:6789/0 conn(0x7f55840cd850 legacy=0x7f55840cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:38.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.320+0000 7f55827fc640 1 -- 172.21.15.70:0/3052722238 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840d1da0 con 0x7f558415cfe0 2023-08-25T09:39:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.321+0000 7f55827fc640 1 --2- 172.21.15.70:0/3052722238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f55840d0c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f558406a340 tx=0x7f557802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd50583516605378 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f5581ffb640 1 -- 172.21.15.70:0/3052722238 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557802ca80 con 0x7f558415cfe0 2023-08-25T09:39:38.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f5581ffb640 1 -- 172.21.15.70:0/3052722238 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557802cbe0 con 0x7f558415cfe0 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f5581ffb640 1 -- 172.21.15.70:0/3052722238 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55780343f0 con 0x7f558415cfe0 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f558975d640 1 -- 172.21.15.70:0/3052722238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 msgr2=0x7f55840d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f558975d640 1 --2- 172.21.15.70:0/3052722238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f55840d0c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f558406a340 tx=0x7f557802bee0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f558975d640 1 -- 172.21.15.70:0/3052722238 shutdown_connections 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f558975d640 1 --2- 172.21.15.70:0/3052722238 >> v2:172.21.15.155:3300/0 conn(0x7f55840b8510 0x7f55840b5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f558975d640 1 --2- 172.21.15.70:0/3052722238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f55840d0c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.322+0000 7f558975d640 1 -- 172.21.15.70:0/3052722238 >> 172.21.15.70:0/3052722238 conn(0x7f5584069850 msgr2=0x7f5584069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:38.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.323+0000 7f558975d640 1 -- 172.21.15.70:0/3052722238 shutdown_connections 2023-08-25T09:39:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.323+0000 7f558975d640 1 -- 172.21.15.70:0/3052722238 wait complete. 2023-08-25T09:39:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.324+0000 7f558975d640 1 Processor -- start 2023-08-25T09:39:38.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.324+0000 7f558975d640 1 -- start start 2023-08-25T09:39:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f558975d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840b8510 0x7f55841f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f558975d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840cd850 0x7f55841f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f558975d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f558415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f558975d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55841f63a0 con 0x7f558415cfe0 2023-08-25T09:39:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f558975d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840b3ed0 con 0x7f55840cd850 2023-08-25T09:39:38.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f558975d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55840b41b0 con 0x7f55840b8510 2023-08-25T09:39:38.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.325+0000 7f55837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f558415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f558415f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47650/0 (socket says 172.21.15.70:47650) 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 -- 172.21.15.70:0/1368931493 learned_addr learned my addr 172.21.15.70:0/1368931493 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 -- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840b8510 msgr2=0x7f55841f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f5582ffd640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840b8510 0x7f55841f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840b8510 0x7f55841f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 -- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840cd850 msgr2=0x7f55841f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840cd850 0x7f55841f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 -- 172.21.15.70:0/1368931493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009b30 con 0x7f558415cfe0 2023-08-25T09:39:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f55837fe640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f558415f3f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d800 tx=0x7f557c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c018070 con 0x7f558415cfe0 2023-08-25T09:39:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c0059e0 con 0x7f558415cfe0 2023-08-25T09:39:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55840b4490 con 0x7f558415cfe0 2023-08-25T09:39:38.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.326+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c005b90 con 0x7f558415cfe0 2023-08-25T09:39:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.327+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55840b4a50 con 0x7f558415cfe0 2023-08-25T09:39:38.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.328+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558406a700 con 0x7f558415cfe0 2023-08-25T09:39:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.328+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f557c00b6c0 con 0x7f558415cfe0 2023-08-25T09:39:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.329+0000 7f556b7fe640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.329+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f557c088500 con 0x7f558415cfe0 2023-08-25T09:39:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.330+0000 7f5582ffd640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.330+0000 7f5582ffd640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5574003d90 tx=0x7f5574009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:38.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.333+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f557c053be0 con 0x7f558415cfe0 2023-08-25T09:39:38.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.524+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f558415d3c0 con 0x7f558415cfe0 2023-08-25T09:39:38.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.525+0000 7f556b7fe640 1 -- 172.21.15.70:0/1368931493 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f558415d3c0 con 0x7f558415cfe0 2023-08-25T09:39:38.528 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:38.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 msgr2=0x7f557006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5574003d90 tx=0x7f5574009040 comp rx=0 tx=0).stop 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 msgr2=0x7f558415f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f558415f3f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f557c00d800 tx=0x7f557c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 shutdown_connections 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558415cfe0 0x7f558415f3f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55840cd850 0x7f55841f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 --2- 172.21.15.70:0/1368931493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55840b8510 0x7f55841f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 >> 172.21.15.70:0/1368931493 conn(0x7f5584069850 msgr2=0x7f55840b7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:38.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 shutdown_connections 2023-08-25T09:39:38.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.528+0000 7f558975d640 1 -- 172.21.15.70:0/1368931493 wait complete. 2023-08-25T09:39:38.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:38.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.759+0000 7f8ea4a7c640 1 Processor -- start 2023-08-25T09:39:38.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.759+0000 7f8ea4a7c640 1 -- start start 2023-08-25T09:39:38.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.760+0000 7f8ea4a7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 0x7f8ea00c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.760+0000 7f8ea4a7c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8ea00cdc20 0x7f8ea0069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8ea4a7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ea00cc5f0 con 0x7f8ea00ce060 2023-08-25T09:39:38.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8ea4a7c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea0070650 con 0x7f8ea00cdc20 2023-08-25T09:39:38.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8ea4a7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0070930 con 0x7f8ea015ae50 2023-08-25T09:39:38.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8e9ed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8ea00ce060 0x7f8ea00ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:50106/0 (socket says 172.21.15.70:50106) 2023-08-25T09:39:38.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8e9ed9d640 1 -- 172.21.15.70:0/3874337487 learned_addr learned my addr 172.21.15.70:0/3874337487 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:38.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 0x7f8ea00c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8e97fff640 1 --2- 172.21.15.70:0/3874337487 >> v2:172.21.15.155:3300/0 conn(0x7f8ea00cdc20 0x7f8ea0069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489202777 0 0) 0x7f8ea00cc5f0 con 0x7f8ea00ce060 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.761+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e8c003520 con 0x7f8ea00ce060 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1930827598 0 0) 0x7f8e8c003520 con 0x7f8ea00ce060 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 msgr2=0x7f8ea00c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 0x7f8ea00c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 >> v2:172.21.15.155:3300/0 conn(0x7f8ea00cdc20 msgr2=0x7f8ea0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:38.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 --2- 172.21.15.70:0/3874337487 >> v2:172.21.15.155:3300/0 conn(0x7f8ea00cdc20 0x7f8ea0069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea0070c10 con 0x7f8ea00ce060 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8e980019f0 con 0x7f8ea00ce060 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e97fff640 1 --2- 172.21.15.70:0/3874337487 >> v2:172.21.15.155:3300/0 conn(0x7f8ea00cdc20 0x7f8ea0069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 >> v1:172.21.15.70:6789/0 conn(0x7f8ea00ce060 legacy=0x7f8ea00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e8c003970 0x7f8e8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c0047c0 0x7f8e8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 0x7f8e8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0070650 con 0x7f8e8c003970 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0070930 con 0x7f8e8c0047c0 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c00bac0 con 0x7f8ea015ae50 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e8c003970 0x7f8e8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e9ed9d640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 0x7f8e8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.762+0000 7f8e97fff640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c0047c0 0x7f8e8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.763+0000 7f8e9e59c640 1 -- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 msgr2=0x7f8e8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:38.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.763+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015ae50 0x7f8e8c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.763+0000 7f8e9e59c640 1 -- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c0047c0 msgr2=0x7f8e8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:38.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.763+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c0047c0 0x7f8e8c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.763+0000 7f8e9e59c640 1 -- 172.21.15.70:0/3874337487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c003520 con 0x7f8e8c003970 2023-08-25T09:39:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.764+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e8c003970 0x7f8e8c003d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000ba00 tx=0x7f8e8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f3a1380fedf0f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.764+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900325f0 con 0x7f8e8c003970 2023-08-25T09:39:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.764+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90032750 con 0x7f8e8c003970 2023-08-25T09:39:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.764+0000 7f8e9dd9b640 1 -- 172.21.15.70:0/3874337487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90032ae0 con 0x7f8e8c003970 2023-08-25T09:39:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.764+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/3874337487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e8c003970 msgr2=0x7f8e8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.764+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/3874337487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e8c003970 0x7f8e8c003d90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000ba00 tx=0x7f8e8c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.765+0000 7f8e9ed9d640 1 -- 172.21.15.70:0/3874337487 reap_dead start 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.765+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/3874337487 shutdown_connections 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.765+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/3874337487 >> 172.21.15.70:0/3874337487 conn(0x7f8ea00cb0d0 msgr2=0x7f8ea00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.765+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/3874337487 shutdown_connections 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.765+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/3874337487 wait complete. 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.766+0000 7f8ea4a7c640 1 Processor -- start 2023-08-25T09:39:38.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.767+0000 7f8ea4a7c640 1 -- start start 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.767+0000 7f8ea4a7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c003970 0x7f8ea014fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.767+0000 7f8ea4a7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015dff0 0x7f8ea00b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.767+0000 7f8ea4a7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 0x7f8ea015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8ea4a7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea0150350 con 0x7f8ea00c65d0 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8ea4a7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea015b3b0 con 0x7f8e8c003970 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8ea4a7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea015b600 con 0x7f8ea015dff0 2023-08-25T09:39:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 0x7f8ea015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 0x7f8ea015c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47670/0 (socket says 172.21.15.70:47670) 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 -- 172.21.15.70:0/2388190331 learned_addr learned my addr 172.21.15.70:0/2388190331 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 -- 172.21.15.70:0/2388190331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015dff0 msgr2=0x7f8ea00b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015dff0 0x7f8ea00b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 -- 172.21.15.70:0/2388190331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c003970 msgr2=0x7f8ea014fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c003970 0x7f8ea014fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 -- 172.21.15.70:0/2388190331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea015b8e0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.768+0000 7f8e9ed9d640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 0x7f8ea015c3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e98007c40 tx=0x7f8e9800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:38.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.769+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800b8d0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.769+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800ba50 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.769+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800be00 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.769+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea015bbc0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.769+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea00b3c80 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.770+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea015c0e0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.771+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e98007110 con 0x7f8ea00c65d0 2023-08-25T09:39:38.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.772+0000 7f8e96ffd640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e8006a050 0x7f8e8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.772+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8e980889a0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.772+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e8006a050 0x7f8e8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:38.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.773+0000 7f8e9e59c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e8006a050 0x7f8e8006c510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0057a0 tx=0x7f8e90032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:38.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.777+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8ea015c0e0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.967+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ea015c0e0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.968+0000 7f8e96ffd640 1 -- 172.21.15.70:0/2388190331 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ea015c0e0 con 0x7f8ea00c65d0 2023-08-25T09:39:38.970 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:38.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e8006a050 msgr2=0x7f8e8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e8006a050 0x7f8e8006c510 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c0057a0 tx=0x7f8e90032000 comp rx=0 tx=0).stop 2023-08-25T09:39:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 msgr2=0x7f8ea015c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 0x7f8ea015c3c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8e98007c40 tx=0x7f8e9800b0d0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 shutdown_connections 2023-08-25T09:39:38.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e8006a050 0x7f8e8006c510 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ea00c65d0 0x7f8ea015c3c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ea015dff0 0x7f8ea00b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 --2- 172.21.15.70:0/2388190331 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e8c003970 0x7f8ea014fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:38.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 >> 172.21.15.70:0/2388190331 conn(0x7f8ea00cb0d0 msgr2=0x7f8ea00c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:38.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 shutdown_connections 2023-08-25T09:39:38.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:38.971+0000 7f8ea4a7c640 1 -- 172.21.15.70:0/2388190331 wait complete. 2023-08-25T09:39:38.990 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:39.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.205+0000 7fcd789d5640 1 Processor -- start 2023-08-25T09:39:39.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.206+0000 7fcd789d5640 1 -- start start 2023-08-25T09:39:39.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd789d5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0320 0x7fcd740c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd789d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd7415a920 0x7fcd740b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd789d5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd740adbb0 con 0x7fcd740b0700 2023-08-25T09:39:39.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd789d5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd740b2220 con 0x7fcd7415a920 2023-08-25T09:39:39.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd789d5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740c90a0 con 0x7fcd740b0320 2023-08-25T09:39:39.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd6bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd7415a920 0x7fcd740b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd6bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd7415a920 0x7fcd740b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42386/0 (socket says 172.21.15.70:42386) 2023-08-25T09:39:39.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd7359e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcd740b0700 0x7fcd740ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36272/0 (socket says 172.21.15.70:36272) 2023-08-25T09:39:39.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.207+0000 7fcd6bfff640 1 -- 172.21.15.70:0/2062068023 learned_addr learned my addr 172.21.15.70:0/2062068023 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:39.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.208+0000 7fcd72d9d640 1 --2- 172.21.15.70:0/2062068023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0320 0x7fcd740c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.208+0000 7fcd6bfff640 1 -- 172.21.15.70:0/2062068023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0320 msgr2=0x7fcd740c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:39.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.208+0000 7fcd6bfff640 1 --2- 172.21.15.70:0/2062068023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0320 0x7fcd740c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.208+0000 7fcd6bfff640 1 -- 172.21.15.70:0/2062068023 >> v1:172.21.15.70:6789/0 conn(0x7fcd740b0700 legacy=0x7fcd740ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.208+0000 7fcd6bfff640 1 -- 172.21.15.70:0/2062068023 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740c9380 con 0x7fcd7415a920 2023-08-25T09:39:39.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.209+0000 7fcd6bfff640 1 --2- 172.21.15.70:0/2062068023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740b1ce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009870 tx=0x7fcd5c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51839e1de9f668e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:39.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.209+0000 7fcd7259c640 1 -- 172.21.15.70:0/2062068023 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c030840 con 0x7fcd7415a920 2023-08-25T09:39:39.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.209+0000 7fcd7259c640 1 -- 172.21.15.70:0/2062068023 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c0309a0 con 0x7fcd7415a920 2023-08-25T09:39:39.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd7259c640 1 -- 172.21.15.70:0/2062068023 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c030d30 con 0x7fcd7415a920 2023-08-25T09:39:39.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 -- 172.21.15.70:0/2062068023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 msgr2=0x7fcd740b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 --2- 172.21.15.70:0/2062068023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740b1ce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009870 tx=0x7fcd5c02b980 comp rx=0 tx=0).stop 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 -- 172.21.15.70:0/2062068023 shutdown_connections 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 --2- 172.21.15.70:0/2062068023 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740b1ce0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 --2- 172.21.15.70:0/2062068023 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0320 0x7fcd740c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 -- 172.21.15.70:0/2062068023 >> 172.21.15.70:0/2062068023 conn(0x7fcd740ce830 msgr2=0x7fcd740cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 -- 172.21.15.70:0/2062068023 shutdown_connections 2023-08-25T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.210+0000 7fcd789d5640 1 -- 172.21.15.70:0/2062068023 wait complete. 2023-08-25T09:39:39.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.213+0000 7fcd789d5640 1 Processor -- start 2023-08-25T09:39:39.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.213+0000 7fcd789d5640 1 -- start start 2023-08-25T09:39:39.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd789d5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd740b0320 0x7fcd740ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd789d5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0700 0x7fcd740cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd789d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd789d5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7415dc50 con 0x7fcd7415a920 2023-08-25T09:39:39.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd789d5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd741f2f20 con 0x7fcd740b0320 2023-08-25T09:39:39.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd789d5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd741f2630 con 0x7fcd740b0700 2023-08-25T09:39:39.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd7359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd7359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42400/0 (socket says 172.21.15.70:42400) 2023-08-25T09:39:39.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.214+0000 7fcd7359e640 1 -- 172.21.15.70:0/3425948631 learned_addr learned my addr 172.21.15.70:0/3425948631 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:39.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd72d9d640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd740b0320 0x7fcd740ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd6bfff640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0700 0x7fcd740cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd7359e640 1 -- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0700 msgr2=0x7fcd740cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd7359e640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0700 0x7fcd740cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd7359e640 1 -- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd740b0320 msgr2=0x7fcd740ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd7359e640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd740b0320 0x7fcd740ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd7359e640 1 -- 172.21.15.70:0/3425948631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5c0094d0 con 0x7fcd7415a920 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd7359e640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740cdaf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c780 tx=0x7fcd6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00ce90 con 0x7fcd7415a920 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00a640 con 0x7fcd7415a920 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00a9f0 con 0x7fcd7415a920 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.215+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd741f33b0 con 0x7fcd7415a920 2023-08-25T09:39:39.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.216+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd741f2bb0 con 0x7fcd7415a920 2023-08-25T09:39:39.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.217+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd74061460 con 0x7fcd7415a920 2023-08-25T09:39:39.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.218+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcd6c00ab50 con 0x7fcd7415a920 2023-08-25T09:39:39.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.219+0000 7fcd6b7fe640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd5406a080 0x7fcd5406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.219+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd6c087c80 con 0x7fcd7415a920 2023-08-25T09:39:39.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.219+0000 7fcd72d9d640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd5406a080 0x7fcd5406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.220+0000 7fcd72d9d640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd5406a080 0x7fcd5406c540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b9b0 tx=0x7fcd64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:39.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.223+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd6c08e080 con 0x7fcd7415a920 2023-08-25T09:39:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.413+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd741f3060 con 0x7fcd7415a920 2023-08-25T09:39:39.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.413+0000 7fcd6b7fe640 1 -- 172.21.15.70:0/3425948631 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd6c017020 con 0x7fcd7415a920 2023-08-25T09:39:39.416 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd5406a080 msgr2=0x7fcd5406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd5406a080 0x7fcd5406c540 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b9b0 tx=0x7fcd64008040 comp rx=0 tx=0).stop 2023-08-25T09:39:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 msgr2=0x7fcd740cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740cdaf0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c780 tx=0x7fcd6c00cc40 comp rx=0 tx=0).stop 2023-08-25T09:39:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 shutdown_connections 2023-08-25T09:39:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd5406a080 0x7fcd5406c540 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd7415a920 0x7fcd740cdaf0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd740b0700 0x7fcd740cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 --2- 172.21.15.70:0/3425948631 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd740b0320 0x7fcd740ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 >> 172.21.15.70:0/3425948631 conn(0x7fcd740ce830 msgr2=0x7fcd74157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 shutdown_connections 2023-08-25T09:39:39.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.416+0000 7fcd789d5640 1 -- 172.21.15.70:0/3425948631 wait complete. 2023-08-25T09:39:39.510 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-08-25T09:39:39.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:39.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.735+0000 7f2fc2c2c640 1 Processor -- start 2023-08-25T09:39:39.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.735+0000 7f2fc2c2c640 1 -- start start 2023-08-25T09:39:39.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc2c2c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc15ca00 0x7f2fbc0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc2c2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fbc0c5120 0x7f2fbc0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc2c2c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2fbc0acd50 0x7f2fbc06ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc2c2c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc160680 con 0x7f2fbc0c5120 2023-08-25T09:39:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc2c2c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc160960 con 0x7f2fbc0acd50 2023-08-25T09:39:39.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc2c2c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc160c40 con 0x7f2fbc15ca00 2023-08-25T09:39:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc1429640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fbc0c5120 0x7f2fbc0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc1429640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fbc0c5120 0x7f2fbc0cd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42408/0 (socket says 172.21.15.70:42408) 2023-08-25T09:39:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.737+0000 7f2fc1429640 1 -- 172.21.15.70:0/88592128 learned_addr learned my addr 172.21.15.70:0/88592128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:39.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc242b640 1 --2- 172.21.15.70:0/88592128 >> v2:172.21.15.155:3300/0 conn(0x7f2fbc0acd50 0x7f2fbc06ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc1429640 1 -- 172.21.15.70:0/88592128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc15ca00 msgr2=0x7f2fbc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc1c2a640 1 --2- 172.21.15.70:0/88592128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc15ca00 0x7f2fbc0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc1429640 1 --2- 172.21.15.70:0/88592128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc15ca00 0x7f2fbc0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc1429640 1 -- 172.21.15.70:0/88592128 >> v2:172.21.15.155:3300/0 conn(0x7f2fbc0acd50 msgr2=0x7f2fbc06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc1429640 1 --2- 172.21.15.70:0/88592128 >> v2:172.21.15.155:3300/0 conn(0x7f2fbc0acd50 0x7f2fbc06ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc1429640 1 -- 172.21.15.70:0/88592128 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc160f20 con 0x7f2fbc0c5120 2023-08-25T09:39:39.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.738+0000 7f2fc242b640 1 --2- 172.21.15.70:0/88592128 >> v2:172.21.15.155:3300/0 conn(0x7f2fbc0acd50 0x7f2fbc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:39.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.739+0000 7f2fc1429640 1 --2- 172.21.15.70:0/88592128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc0cd3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc06ea00 tx=0x7f2fac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97db391113949939 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:39.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.739+0000 7f2fc0c28640 1 -- 172.21.15.70:0/88592128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac031870 con 0x7f2fbc0c5120 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.739+0000 7f2fc0c28640 1 -- 172.21.15.70:0/88592128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac0319d0 con 0x7f2fbc0c5120 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.739+0000 7f2fc0c28640 1 -- 172.21.15.70:0/88592128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac031d60 con 0x7f2fbc0c5120 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/88592128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc0c5120 msgr2=0x7f2fbc0cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/88592128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc0cd3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc06ea00 tx=0x7f2fac02b910 comp rx=0 tx=0).stop 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/88592128 shutdown_connections 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/88592128 >> v2:172.21.15.155:3300/0 conn(0x7f2fbc0acd50 0x7f2fbc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/88592128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc0cd3f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/88592128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc15ca00 0x7f2fbc0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/88592128 >> 172.21.15.70:0/88592128 conn(0x7f2fbc0affd0 msgr2=0x7f2fbc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:39.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/88592128 shutdown_connections 2023-08-25T09:39:39.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.740+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/88592128 wait complete. 2023-08-25T09:39:39.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.742+0000 7f2fc2c2c640 1 Processor -- start 2023-08-25T09:39:39.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.742+0000 7f2fc2c2c640 1 -- start start 2023-08-25T09:39:39.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc2c2c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc0acd50 0x7f2fbc1f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc2c2c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc2c2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 0x7f2fbc0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc2c2c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc146850 con 0x7f2fbc15ca00 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc2c2c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc15f670 con 0x7f2fbc0c5120 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc2c2c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc15f950 con 0x7f2fbc0acd50 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc242b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 0x7f2fbc0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc242b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 0x7f2fbc0ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42422/0 (socket says 172.21.15.70:42422) 2023-08-25T09:39:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc242b640 1 -- 172.21.15.70:0/1858282899 learned_addr learned my addr 172.21.15.70:0/1858282899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.743+0000 7f2fc1c2a640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc0acd50 0x7f2fbc1f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc1429640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc242b640 1 -- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc0acd50 msgr2=0x7f2fbc1f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc242b640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc0acd50 0x7f2fbc1f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc242b640 1 -- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fbc0c5120 msgr2=0x7f2fbc15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc242b640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc15ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc242b640 1 -- 172.21.15.70:0/1858282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac0094d0 con 0x7f2fbc15ca00 2023-08-25T09:39:39.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc242b640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 0x7f2fbc0ccbb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400cdd0 tx=0x7f2fb400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:39.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400d840 con 0x7f2fbc15ca00 2023-08-25T09:39:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb400d9a0 con 0x7f2fbc15ca00 2023-08-25T09:39:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400dd50 con 0x7f2fbc15ca00 2023-08-25T09:39:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.744+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc15fc30 con 0x7f2fbc15ca00 2023-08-25T09:39:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.745+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc15f1b0 con 0x7f2fbc15ca00 2023-08-25T09:39:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.746+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc15f400 con 0x7f2fbc15ca00 2023-08-25T09:39:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.746+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2fb401e3b0 con 0x7f2fbc15ca00 2023-08-25T09:39:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.748+0000 7f2fb27fc640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fa006a050 0x7f2fa006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.748+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fb4088f40 con 0x7f2fbc15ca00 2023-08-25T09:39:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.748+0000 7f2fc1c2a640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fa006a050 0x7f2fa006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.749+0000 7f2fc1c2a640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fa006a050 0x7f2fa006c510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8006fd0 tx=0x7f2fb8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.752+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2fb40546c0 con 0x7f2fbc15ca00 2023-08-25T09:39:39.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.941+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fbc066fb0 con 0x7f2fbc15ca00 2023-08-25T09:39:39.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.944+0000 7f2fb27fc640 1 -- 172.21.15.70:0/1858282899 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f2fb401a020 con 0x7f2fbc15ca00 2023-08-25T09:39:39.946 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:39.947 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:39.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fa006a050 msgr2=0x7f2fa006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fa006a050 0x7f2fa006c510 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8006fd0 tx=0x7f2fb8008040 comp rx=0 tx=0).stop 2023-08-25T09:39:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 msgr2=0x7f2fbc0ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 0x7f2fbc0ccbb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400cdd0 tx=0x7f2fb400a710 comp rx=0 tx=0).stop 2023-08-25T09:39:39.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 shutdown_connections 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fa006a050 0x7f2fa006c510 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fbc15ca00 0x7f2fbc0ccbb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fbc0c5120 0x7f2fbc15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 --2- 172.21.15.70:0/1858282899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fbc0acd50 0x7f2fbc1f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 >> 172.21.15.70:0/1858282899 conn(0x7f2fbc0affd0 msgr2=0x7f2fbc0af120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 shutdown_connections 2023-08-25T09:39:39.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:39.947+0000 7f2fc2c2c640 1 -- 172.21.15.70:0/1858282899 wait complete. 2023-08-25T09:39:39.968 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:39.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:40.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.185+0000 7f122ae83640 1 Processor -- start 2023-08-25T09:39:40.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.186+0000 7f122ae83640 1 -- start start 2023-08-25T09:39:40.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f122ae83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1224069850 0x7f122415e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f122ae83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f1224069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f122ae83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1224160be0 0x7f1224161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f122ae83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12240ac880 con 0x7f1224069850 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f122ae83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12240ad400 con 0x7f1224160be0 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f122ae83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240ad890 con 0x7f12240c9620 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1224069850 0x7f122415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1224069850 0x7f122415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42426/0 (socket says 172.21.15.70:42426) 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 -- 172.21.15.70:0/1341539844 learned_addr learned my addr 172.21.15.70:0/1341539844 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1223fff640 1 --2- 172.21.15.70:0/1341539844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f1224069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1229420640 1 --2- 172.21.15.70:0/1341539844 >> v2:172.21.15.155:3300/0 conn(0x7f1224160be0 0x7f1224161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 -- 172.21.15.70:0/1341539844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 msgr2=0x7f1224069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 --2- 172.21.15.70:0/1341539844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f1224069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 -- 172.21.15.70:0/1341539844 >> v2:172.21.15.155:3300/0 conn(0x7f1224160be0 msgr2=0x7f1224161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 --2- 172.21.15.70:0/1341539844 >> v2:172.21.15.155:3300/0 conn(0x7f1224160be0 0x7f1224161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.188+0000 7f1228c1f640 1 -- 172.21.15.70:0/1341539844 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12240acad0 con 0x7f1224069850 2023-08-25T09:39:40.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.189+0000 7f1229420640 1 --2- 172.21.15.70:0/1341539844 >> v2:172.21.15.155:3300/0 conn(0x7f1224160be0 0x7f1224161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:40.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.190+0000 7f1228c1f640 1 --2- 172.21.15.70:0/1341539844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f122415e210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f122406a7e0 tx=0x7f121802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356c061b385a908c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:40.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.190+0000 7f12237fe640 1 -- 172.21.15.70:0/1341539844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218030860 con 0x7f1224069850 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.190+0000 7f12237fe640 1 -- 172.21.15.70:0/1341539844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12180309c0 con 0x7f1224069850 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.190+0000 7f12237fe640 1 -- 172.21.15.70:0/1341539844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218030d50 con 0x7f1224069850 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 -- 172.21.15.70:0/1341539844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 msgr2=0x7f122415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 --2- 172.21.15.70:0/1341539844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f122415e210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f122406a7e0 tx=0x7f121802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 -- 172.21.15.70:0/1341539844 shutdown_connections 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 --2- 172.21.15.70:0/1341539844 >> v2:172.21.15.155:3300/0 conn(0x7f1224160be0 0x7f1224161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 --2- 172.21.15.70:0/1341539844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f1224069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 --2- 172.21.15.70:0/1341539844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f122415e210 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 -- 172.21.15.70:0/1341539844 >> 172.21.15.70:0/1341539844 conn(0x7f12240cb0d0 msgr2=0x7f12240cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:40.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 -- 172.21.15.70:0/1341539844 shutdown_connections 2023-08-25T09:39:40.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.191+0000 7f122ae83640 1 -- 172.21.15.70:0/1341539844 wait complete. 2023-08-25T09:39:40.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.193+0000 7f122ae83640 1 Processor -- start 2023-08-25T09:39:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.193+0000 7f122ae83640 1 -- start start 2023-08-25T09:39:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f122ae83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f1224149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f122ae83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f122406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f122ae83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1224160be0 0x7f1224157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f122ae83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b0f20 con 0x7f1224069850 2023-08-25T09:39:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f122ae83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b02e0 con 0x7f1224160be0 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f122ae83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12240b0580 con 0x7f12240c9620 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f1224149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f1224149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42436/0 (socket says 172.21.15.70:42436) 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 -- 172.21.15.70:0/1709995387 learned_addr learned my addr 172.21.15.70:0/1709995387 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 -- 172.21.15.70:0/1709995387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 msgr2=0x7f122406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f122406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 -- 172.21.15.70:0/1709995387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1224160be0 msgr2=0x7f1224157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1224160be0 0x7f1224157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.194+0000 7f1228c1f640 1 -- 172.21.15.70:0/1709995387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180094d0 con 0x7f1224069850 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.195+0000 7f1228c1f640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f1224149db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f121802b9e0 tx=0x7f1218004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.195+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218030850 con 0x7f1224069850 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.195+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1218043400 con 0x7f1224069850 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.195+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12180437b0 con 0x7f1224069850 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.195+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12240b0860 con 0x7f1224069850 2023-08-25T09:39:40.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.195+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122406ca20 con 0x7f1224069850 2023-08-25T09:39:40.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.196+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122406cbb0 con 0x7f1224069850 2023-08-25T09:39:40.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.197+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1218043910 con 0x7f1224069850 2023-08-25T09:39:40.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.198+0000 7f12217fa640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f120806a050 0x7f120806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.198+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f121803e070 con 0x7f1224069850 2023-08-25T09:39:40.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.199+0000 7f1223fff640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f120806a050 0x7f120806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.199+0000 7f1223fff640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f120806a050 0x7f120806c510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1210006b10 tx=0x7f1210008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:40.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.203+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1218077000 con 0x7f1224069850 2023-08-25T09:39:40.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.392+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1224066fb0 con 0x7f1224069850 2023-08-25T09:39:40.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.393+0000 7f12217fa640 1 -- 172.21.15.70:0/1709995387 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12180340a0 con 0x7f1224069850 2023-08-25T09:39:40.395 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:40.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f120806a050 msgr2=0x7f120806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f120806a050 0x7f120806c510 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1210006b10 tx=0x7f1210008040 comp rx=0 tx=0).stop 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 msgr2=0x7f1224149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f1224149db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f121802b9e0 tx=0x7f1218004730 comp rx=0 tx=0).stop 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 shutdown_connections 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f120806a050 0x7f120806c510 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1224160be0 0x7f1224157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12240c9620 0x7f122406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 --2- 172.21.15.70:0/1709995387 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1224069850 0x7f1224149db0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 >> 172.21.15.70:0/1709995387 conn(0x7f12240cb0d0 msgr2=0x7f122406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:40.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 shutdown_connections 2023-08-25T09:39:40.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.396+0000 7f122ae83640 1 -- 172.21.15.70:0/1709995387 wait complete. 2023-08-25T09:39:40.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:40.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.629+0000 7f7bb3312640 1 Processor -- start 2023-08-25T09:39:40.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.629+0000 7f7bb3312640 1 -- start start 2023-08-25T09:39:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb3312640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bac06a110 0x7f7bac0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb3312640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7bac06c510 0x7f7bac06c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb3312640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac0cd4e0 con 0x7f7bac069d30 2023-08-25T09:39:40.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb3312640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac0cc550 con 0x7f7bac06a110 2023-08-25T09:39:40.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb3312640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac0ccb10 con 0x7f7bac06c510 2023-08-25T09:39:40.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb08ad640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bac06a110 0x7f7bac0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb10ae640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7bac069d30 0x7f7bac0ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36278/0 (socket says 172.21.15.70:36278) 2023-08-25T09:39:40.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb10ae640 1 -- 172.21.15.70:0/1483541753 learned_addr learned my addr 172.21.15.70:0/1483541753 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:40.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb18af640 1 --2- 172.21.15.70:0/1483541753 >> v2:172.21.15.155:3300/0 conn(0x7f7bac06c510 0x7f7bac06c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb08ad640 1 -- 172.21.15.70:0/1483541753 >> v2:172.21.15.155:3300/0 conn(0x7f7bac06c510 msgr2=0x7f7bac06c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:40.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb08ad640 1 --2- 172.21.15.70:0/1483541753 >> v2:172.21.15.155:3300/0 conn(0x7f7bac06c510 0x7f7bac06c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb08ad640 1 -- 172.21.15.70:0/1483541753 >> v1:172.21.15.70:6789/0 conn(0x7f7bac069d30 legacy=0x7f7bac0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb08ad640 1 -- 172.21.15.70:0/1483541753 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0ade40 con 0x7f7bac06a110 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7ba3fff640 1 -- 172.21.15.70:0/1483541753 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060720376 0 0) 0x7f7ba8001110 con 0x7f7bac069d30 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.631+0000 7f7bb18af640 1 --2- 172.21.15.70:0/1483541753 >> v2:172.21.15.155:3300/0 conn(0x7f7bac06c510 0x7f7bac06c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.632+0000 7f7bb08ad640 1 --2- 172.21.15.70:0/1483541753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac06a110 0x7f7bac0ccfa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0097a0 tx=0x7f7b9c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e208f574ec354b2a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7ba3fff640 1 -- 172.21.15.70:0/1483541753 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c02e070 con 0x7f7bac06a110 2023-08-25T09:39:40.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7ba3fff640 1 -- 172.21.15.70:0/1483541753 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c0315a0 con 0x7f7bac06a110 2023-08-25T09:39:40.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7ba3fff640 1 -- 172.21.15.70:0/1483541753 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c031930 con 0x7f7bac06a110 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7bb3312640 1 -- 172.21.15.70:0/1483541753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac06a110 msgr2=0x7f7bac0ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7bb3312640 1 --2- 172.21.15.70:0/1483541753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac06a110 0x7f7bac0ccfa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0097a0 tx=0x7f7b9c02b840 comp rx=0 tx=0).stop 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7bb3312640 1 -- 172.21.15.70:0/1483541753 shutdown_connections 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7bb3312640 1 --2- 172.21.15.70:0/1483541753 >> v2:172.21.15.155:3300/0 conn(0x7f7bac06c510 0x7f7bac06c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7bb3312640 1 --2- 172.21.15.70:0/1483541753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac06a110 0x7f7bac0ccfa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.633+0000 7f7bb3312640 1 -- 172.21.15.70:0/1483541753 >> 172.21.15.70:0/1483541753 conn(0x7f7bac0cb0d0 msgr2=0x7f7bac0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:40.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.634+0000 7f7bb3312640 1 -- 172.21.15.70:0/1483541753 shutdown_connections 2023-08-25T09:39:40.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.634+0000 7f7bb3312640 1 -- 172.21.15.70:0/1483541753 wait complete. 2023-08-25T09:39:40.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.635+0000 7f7bb3312640 1 Processor -- start 2023-08-25T09:39:40.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.635+0000 7f7bb3312640 1 -- start start 2023-08-25T09:39:40.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.636+0000 7f7bb3312640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 0x7f7bac15fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.636+0000 7f7bb3312640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bac06a110 0x7f7bac0b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.636+0000 7f7bb3312640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bac06c510 0x7f7bac0c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.636+0000 7f7bb3312640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac0c5730 con 0x7f7bac069d30 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.636+0000 7f7bb3312640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac0c1c00 con 0x7f7bac06c510 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.636+0000 7f7bb3312640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac0b07b0 con 0x7f7bac06a110 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 0x7f7bac15fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 0x7f7bac15fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42450/0 (socket says 172.21.15.70:42450) 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 -- 172.21.15.70:0/3670685601 learned_addr learned my addr 172.21.15.70:0/3670685601 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 -- 172.21.15.70:0/3670685601 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bac06a110 msgr2=0x7f7bac0b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bac06a110 0x7f7bac0b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 -- 172.21.15.70:0/3670685601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bac06c510 msgr2=0x7f7bac0c16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bac06c510 0x7f7bac0c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 -- 172.21.15.70:0/3670685601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c009400 con 0x7f7bac069d30 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.637+0000 7f7bb10ae640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 0x7f7bac15fda0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c430 tx=0x7f7ba800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.638+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80074e0 con 0x7f7bac069d30 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.638+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8007660 con 0x7f7bac069d30 2023-08-25T09:39:40.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.638+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8004a20 con 0x7f7bac069d30 2023-08-25T09:39:40.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.638+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac0b0a90 con 0x7f7bac069d30 2023-08-25T09:39:40.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.638+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac0b0d70 con 0x7f7bac069d30 2023-08-25T09:39:40.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.639+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac066fb0 con 0x7f7bac069d30 2023-08-25T09:39:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.640+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ba8004b80 con 0x7f7bac069d30 2023-08-25T09:39:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.641+0000 7f7ba1ffb640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b9006a050 0x7f7b9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.642+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ba8088190 con 0x7f7bac069d30 2023-08-25T09:39:40.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.642+0000 7f7bb08ad640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b9006a050 0x7f7b9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:40.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.642+0000 7f7bb08ad640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b9006a050 0x7f7b9006c510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c036000 tx=0x7f7b9c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:40.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.646+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ba8053910 con 0x7f7bac069d30 2023-08-25T09:39:40.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.841+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7bac0cd4e0 con 0x7f7bac069d30 2023-08-25T09:39:40.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.841+0000 7f7ba1ffb640 1 -- 172.21.15.70:0/3670685601 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7bac0cd4e0 con 0x7f7bac069d30 2023-08-25T09:39:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:40.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b9006a050 msgr2=0x7f7b9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b9006a050 0x7f7b9006c510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c036000 tx=0x7f7b9c030000 comp rx=0 tx=0).stop 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 msgr2=0x7f7bac15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 0x7f7bac15fda0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c430 tx=0x7f7ba800c8f0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 shutdown_connections 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b9006a050 0x7f7b9006c510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bac06c510 0x7f7bac0c16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bac06a110 0x7f7bac0b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 --2- 172.21.15.70:0/3670685601 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bac069d30 0x7f7bac15fda0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.844+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 >> 172.21.15.70:0/3670685601 conn(0x7f7bac0cb0d0 msgr2=0x7f7bac070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.845+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 shutdown_connections 2023-08-25T09:39:40.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:40.845+0000 7f7bb3312640 1 -- 172.21.15.70:0/3670685601 wait complete. 2023-08-25T09:39:40.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:41.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.076+0000 7f4c798f3640 1 Processor -- start 2023-08-25T09:39:41.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.077+0000 7f4c798f3640 1 -- start start 2023-08-25T09:39:41.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.078+0000 7f4c798f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 0x7f4c740cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.078+0000 7f4c798f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4c74069850 0x7f4c740d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.078+0000 7f4c798f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c74071c10 con 0x7f4c74160670 2023-08-25T09:39:41.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.078+0000 7f4c798f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c74071000 con 0x7f4c74069850 2023-08-25T09:39:41.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.078+0000 7f4c798f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740712e0 con 0x7f4c74160290 2023-08-25T09:39:41.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c72ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 0x7f4c740cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c72ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 0x7f4c740cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:57520/0 (socket says 172.21.15.70:57520) 2023-08-25T09:39:41.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c72ffd640 1 -- 172.21.15.70:0/1297416504 learned_addr learned my addr 172.21.15.70:0/1297416504 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:41.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c727fc640 1 --1- 172.21.15.70:0/1297416504 >> v1:172.21.15.155:6789/0 conn(0x7f4c74160670 0x7f4c740cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:56300/0 (socket says 172.21.15.70:56300) 2023-08-25T09:39:41.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c737fe640 1 --2- 172.21.15.70:0/1297416504 >> v2:172.21.15.155:3300/0 conn(0x7f4c74069850 0x7f4c740d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c71ffb640 1 -- 172.21.15.70:0/1297416504 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486834770 0 0) 0x7f4c74071c10 con 0x7f4c74160670 2023-08-25T09:39:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.079+0000 7f4c71ffb640 1 -- 172.21.15.70:0/1297416504 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c60003520 con 0x7f4c74160670 2023-08-25T09:39:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.080+0000 7f4c72ffd640 1 -- 172.21.15.70:0/1297416504 >> v2:172.21.15.155:3300/0 conn(0x7f4c74069850 msgr2=0x7f4c740d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.080+0000 7f4c72ffd640 1 --2- 172.21.15.70:0/1297416504 >> v2:172.21.15.155:3300/0 conn(0x7f4c74069850 0x7f4c740d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.080+0000 7f4c72ffd640 1 -- 172.21.15.70:0/1297416504 >> v1:172.21.15.155:6789/0 conn(0x7f4c74160670 legacy=0x7f4c740cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.080+0000 7f4c72ffd640 1 -- 172.21.15.70:0/1297416504 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740715c0 con 0x7f4c74160290 2023-08-25T09:39:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.081+0000 7f4c72ffd640 1 --2- 172.21.15.70:0/1297416504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 0x7f4c740cd360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da68f62ad85299f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.081+0000 7f4c71ffb640 1 -- 172.21.15.70:0/1297416504 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64030840 con 0x7f4c74160290 2023-08-25T09:39:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.081+0000 7f4c71ffb640 1 -- 172.21.15.70:0/1297416504 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c640309a0 con 0x7f4c74160290 2023-08-25T09:39:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.081+0000 7f4c71ffb640 1 -- 172.21.15.70:0/1297416504 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64030d30 con 0x7f4c74160290 2023-08-25T09:39:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 -- 172.21.15.70:0/1297416504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 msgr2=0x7f4c740cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 --2- 172.21.15.70:0/1297416504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 0x7f4c740cd360 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009870 tx=0x7f4c6402b980 comp rx=0 tx=0).stop 2023-08-25T09:39:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 -- 172.21.15.70:0/1297416504 shutdown_connections 2023-08-25T09:39:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 --2- 172.21.15.70:0/1297416504 >> v2:172.21.15.155:3300/0 conn(0x7f4c74069850 0x7f4c740d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 --2- 172.21.15.70:0/1297416504 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74160290 0x7f4c740cd360 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 -- 172.21.15.70:0/1297416504 >> 172.21.15.70:0/1297416504 conn(0x7f4c7406ba60 msgr2=0x7f4c7406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 -- 172.21.15.70:0/1297416504 shutdown_connections 2023-08-25T09:39:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.082+0000 7f4c798f3640 1 -- 172.21.15.70:0/1297416504 wait complete. 2023-08-25T09:39:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.084+0000 7f4c798f3640 1 Processor -- start 2023-08-25T09:39:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.084+0000 7f4c798f3640 1 -- start start 2023-08-25T09:39:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c798f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74069850 0x7f4c741f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c798f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c74160290 0x7f4c740c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c798f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 0x7f4c741f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c798f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7415f4b0 con 0x7f4c74160670 2023-08-25T09:39:41.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c798f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741f6b30 con 0x7f4c74160290 2023-08-25T09:39:41.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c798f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c741582f0 con 0x7f4c74069850 2023-08-25T09:39:41.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.085+0000 7f4c737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 0x7f4c741f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 0x7f4c741f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42464/0 (socket says 172.21.15.70:42464) 2023-08-25T09:39:41.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 -- 172.21.15.70:0/4230586595 learned_addr learned my addr 172.21.15.70:0/4230586595 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:41.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c727fc640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c74160290 0x7f4c740c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c72ffd640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74069850 0x7f4c741f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 -- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74069850 msgr2=0x7f4c741f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74069850 0x7f4c741f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 -- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c74160290 msgr2=0x7f4c740c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c74160290 0x7f4c740c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 -- 172.21.15.70:0/4230586595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c640094d0 con 0x7f4c74160670 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c72ffd640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74069850 0x7f4c741f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.086+0000 7f4c737fe640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 0x7f4c741f5cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e930 tx=0x7f4c6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.087+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c009690 con 0x7f4c74160670 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.087+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00c980 con 0x7f4c74160670 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.087+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00cd30 con 0x7f4c74160670 2023-08-25T09:39:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.087+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c741585d0 con 0x7f4c74160670 2023-08-25T09:39:41.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.087+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c74158b90 con 0x7f4c74160670 2023-08-25T09:39:41.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.088+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c74158db0 con 0x7f4c74160670 2023-08-25T09:39:41.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.089+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4c6c013470 con 0x7f4c74160670 2023-08-25T09:39:41.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.091+0000 7f4c6bfff640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c5406a050 0x7f4c5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.091+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c6c088840 con 0x7f4c74160670 2023-08-25T09:39:41.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.091+0000 7f4c72ffd640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c5406a050 0x7f4c5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.092+0000 7f4c72ffd640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c5406a050 0x7f4c5406c510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c64036000 tx=0x7f4c6403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.095+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4c6c057a70 con 0x7f4c74160670 2023-08-25T09:39:41.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.287+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c74066fb0 con 0x7f4c74160670 2023-08-25T09:39:41.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.287+0000 7f4c6bfff640 1 -- 172.21.15.70:0/4230586595 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c6c057600 con 0x7f4c74160670 2023-08-25T09:39:41.289 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:41.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c5406a050 msgr2=0x7f4c5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c5406a050 0x7f4c5406c510 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c64036000 tx=0x7f4c6403b040 comp rx=0 tx=0).stop 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 msgr2=0x7f4c741f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 0x7f4c741f5cc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e930 tx=0x7f4c6c00edf0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 shutdown_connections 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c5406a050 0x7f4c5406c510 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c74160670 0x7f4c741f5cc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c74160290 0x7f4c740c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 --2- 172.21.15.70:0/4230586595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c74069850 0x7f4c741f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 >> 172.21.15.70:0/4230586595 conn(0x7f4c7406ba60 msgr2=0x7f4c740acb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 shutdown_connections 2023-08-25T09:39:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.290+0000 7f4c798f3640 1 -- 172.21.15.70:0/4230586595 wait complete. 2023-08-25T09:39:41.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:41.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.526+0000 7ff01b7cf640 1 Processor -- start 2023-08-25T09:39:41.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.526+0000 7ff01b7cf640 1 -- start start 2023-08-25T09:39:41.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.527+0000 7ff01b7cf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff01406a110 0x7ff0140ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.527+0000 7ff01b7cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140cd4e0 con 0x7ff01406c510 2023-08-25T09:39:41.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.527+0000 7ff01b7cf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0140cc550 con 0x7ff014069d30 2023-08-25T09:39:41.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.527+0000 7ff01b7cf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0140ccb10 con 0x7ff01406a110 2023-08-25T09:39:41.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.528+0000 7ff019d6c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff01406c510 0x7ff01406c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36288/0 (socket says 172.21.15.70:36288) 2023-08-25T09:39:41.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.528+0000 7ff019d6c640 1 -- 172.21.15.70:0/2024743699 learned_addr learned my addr 172.21.15.70:0/2024743699 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:41.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.528+0000 7ff018d6a640 1 --2- 172.21.15.70:0/2024743699 >> v2:172.21.15.155:3300/0 conn(0x7ff01406a110 0x7ff0140ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.528+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68687437 0 0) 0x7ff00c001490 con 0x7ff01406c510 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.528+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff000003520 con 0x7ff01406c510 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2068494912 0 0) 0x7ff000003520 con 0x7ff01406c510 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 >> v2:172.21.15.155:3300/0 conn(0x7ff01406a110 msgr2=0x7ff0140ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 --2- 172.21.15.70:0/2024743699 >> v2:172.21.15.155:3300/0 conn(0x7ff01406a110 0x7ff0140ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 >> v1:172.21.15.155:6789/0 conn(0x7ff014069d30 legacy=0x7ff0140ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140ade40 con 0x7ff01406c510 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff00c001b80 con 0x7ff01406c510 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 >> v1:172.21.15.70:6789/0 conn(0x7ff01406c510 legacy=0x7ff01406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff000003970 0x7ff000003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0000047c0 0x7ff000004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff000005180 0x7ff00000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140ccb10 con 0x7ff0000047c0 2023-08-25T09:39:41.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00000c120 con 0x7ff000003970 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00000c2e0 con 0x7ff000005180 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.529+0000 7ff018d6a640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0000047c0 0x7ff000004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff01956b640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff000003970 0x7ff000003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff019d6c640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff000005180 0x7ff00000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff018d6a640 1 -- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff000005180 msgr2=0x7ff00000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff018d6a640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff000005180 0x7ff00000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff018d6a640 1 -- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff000003970 msgr2=0x7ff000003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff018d6a640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff000003970 0x7ff000003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff018d6a640 1 -- 172.21.15.70:0/2024743699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000003520 con 0x7ff0000047c0 2023-08-25T09:39:41.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.530+0000 7ff019d6c640 1 -- 172.21.15.70:0/2024743699 reap_dead start 2023-08-25T09:39:41.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.531+0000 7ff018d6a640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0000047c0 0x7ff000004c40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff004009ac0 tx=0x7ff0000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef7cbd460f16997 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.531+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00402c6b0 con 0x7ff0000047c0 2023-08-25T09:39:41.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.531+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00402c810 con 0x7ff0000047c0 2023-08-25T09:39:41.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.531+0000 7ff00bfff640 1 -- 172.21.15.70:0/2024743699 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00402cba0 con 0x7ff0000047c0 2023-08-25T09:39:41.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 -- 172.21.15.70:0/2024743699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0000047c0 msgr2=0x7ff000004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0000047c0 0x7ff000004c40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff004009ac0 tx=0x7ff0000057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 -- 172.21.15.70:0/2024743699 shutdown_connections 2023-08-25T09:39:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/2024743699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0000047c0 0x7ff000004c40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 -- 172.21.15.70:0/2024743699 >> 172.21.15.70:0/2024743699 conn(0x7ff0140cb0d0 msgr2=0x7ff0140cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 -- 172.21.15.70:0/2024743699 shutdown_connections 2023-08-25T09:39:41.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.532+0000 7ff01b7cf640 1 -- 172.21.15.70:0/2024743699 wait complete. 2023-08-25T09:39:41.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.534+0000 7ff01b7cf640 1 Processor -- start 2023-08-25T09:39:41.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.534+0000 7ff01b7cf640 1 -- start start 2023-08-25T09:39:41.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff01b7cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0000047c0 0x7ff0140c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff01b7cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff014162f30 0x7ff0140b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff01b7cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 0x7ff0140b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff01b7cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01415e490 con 0x7ff014163370 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff01b7cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140c3920 con 0x7ff0000047c0 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff01b7cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0140c2a70 con 0x7ff014162f30 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff019d6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 0x7ff0140b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff019d6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 0x7ff0140b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42470/0 (socket says 172.21.15.70:42470) 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.535+0000 7ff019d6c640 1 -- 172.21.15.70:0/1575393651 learned_addr learned my addr 172.21.15.70:0/1575393651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:41.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff01956b640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0000047c0 0x7ff0140c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff018d6a640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff014162f30 0x7ff0140b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff019d6c640 1 -- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff014162f30 msgr2=0x7ff0140b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff019d6c640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff014162f30 0x7ff0140b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff019d6c640 1 -- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0000047c0 msgr2=0x7ff0140c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff019d6c640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0000047c0 0x7ff0140c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff019d6c640 1 -- 172.21.15.70:0/1575393651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0140c2d10 con 0x7ff014163370 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff019d6c640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 0x7ff0140b3dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff00c007c40 tx=0x7ff00c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c004d60 con 0x7ff014163370 2023-08-25T09:39:41.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c00bb80 con 0x7ff014163370 2023-08-25T09:39:41.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c007070 con 0x7ff014163370 2023-08-25T09:39:41.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.536+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0140c2ff0 con 0x7ff014163370 2023-08-25T09:39:41.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.537+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0140c35b0 con 0x7ff014163370 2023-08-25T09:39:41.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.538+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0140c3a90 con 0x7ff014163370 2023-08-25T09:39:41.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.538+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff00c0071d0 con 0x7ff014163370 2023-08-25T09:39:41.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.540+0000 7ff009ffb640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feff806a050 0x7feff806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.540+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff00c088740 con 0x7ff014163370 2023-08-25T09:39:41.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.540+0000 7ff01956b640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feff806a050 0x7feff806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.541+0000 7ff01956b640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feff806a050 0x7feff806c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff010007fd0 tx=0x7ff010009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.544+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff00c053e60 con 0x7ff014163370 2023-08-25T09:39:41.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.733+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0140cd4e0 con 0x7ff014163370 2023-08-25T09:39:41.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.735+0000 7ff009ffb640 1 -- 172.21.15.70:0/1575393651 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0140cd4e0 con 0x7ff014163370 2023-08-25T09:39:41.736 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:41.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feff806a050 msgr2=0x7feff806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feff806a050 0x7feff806c510 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff010007fd0 tx=0x7ff010009040 comp rx=0 tx=0).stop 2023-08-25T09:39:41.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 msgr2=0x7ff0140b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 0x7ff0140b3dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff00c007c40 tx=0x7ff00c00b4a0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 shutdown_connections 2023-08-25T09:39:41.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feff806a050 0x7feff806c510 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff014163370 0x7ff0140b3dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff014162f30 0x7ff0140b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 --2- 172.21.15.70:0/1575393651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0000047c0 0x7ff0140c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.737+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 >> 172.21.15.70:0/1575393651 conn(0x7ff0140cb0d0 msgr2=0x7ff014070890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:41.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.738+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 shutdown_connections 2023-08-25T09:39:41.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.738+0000 7ff01b7cf640 1 -- 172.21.15.70:0/1575393651 wait complete. 2023-08-25T09:39:41.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-08-25T09:39:41.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.965+0000 7fd4ce61f640 1 Processor -- start 2023-08-25T09:39:41.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.965+0000 7fd4ce61f640 1 -- start start 2023-08-25T09:39:41.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.967+0000 7fd4ce61f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815e830 0x7fd4c806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.967+0000 7fd4ce61f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 0x7fd4c80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.967+0000 7fd4ce61f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c80b1b60 con 0x7fd4c806b2b0 2023-08-25T09:39:41.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.967+0000 7fd4ce61f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c806bd80 con 0x7fd4c815ec10 2023-08-25T09:39:41.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.967+0000 7fd4ce61f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c806b7c0 con 0x7fd4c815e830 2023-08-25T09:39:41.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815e830 0x7fd4c806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4ccbbc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 0x7fd4c80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4ccbbc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 0x7fd4c80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:42628/0 (socket says 172.21.15.70:42628) 2023-08-25T09:39:41.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4ccbbc640 1 -- 172.21.15.70:0/3138632846 learned_addr learned my addr 172.21.15.70:0/3138632846 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:41.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4c7fff640 1 -- 172.21.15.70:0/3138632846 >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 msgr2=0x7fd4c80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:41.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4c7fff640 1 --2- 172.21.15.70:0/3138632846 >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 0x7fd4c80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4c7fff640 1 -- 172.21.15.70:0/3138632846 >> v1:172.21.15.155:6789/0 conn(0x7fd4c806b2b0 legacy=0x7fd4c815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4c7fff640 1 -- 172.21.15.70:0/3138632846 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c806baa0 con 0x7fd4c815e830 2023-08-25T09:39:41.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.968+0000 7fd4ccbbc640 1 --2- 172.21.15.70:0/3138632846 >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 0x7fd4c80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:39:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.970+0000 7fd4c7fff640 1 --2- 172.21.15.70:0/3138632846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815e830 0x7fd4c806ad70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80097a0 tx=0x7fd4b802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a0bbd00c936890 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.970+0000 7fd4c6ffd640 1 -- 172.21.15.70:0/3138632846 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b802d040 con 0x7fd4c815e830 2023-08-25T09:39:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.970+0000 7fd4c6ffd640 1 -- 172.21.15.70:0/3138632846 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b8031810 con 0x7fd4c815e830 2023-08-25T09:39:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.970+0000 7fd4c6ffd640 1 -- 172.21.15.70:0/3138632846 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8031ba0 con 0x7fd4c815e830 2023-08-25T09:39:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 -- 172.21.15.70:0/3138632846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815e830 msgr2=0x7fd4c806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/3138632846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815e830 0x7fd4c806ad70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80097a0 tx=0x7fd4b802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 -- 172.21.15.70:0/3138632846 shutdown_connections 2023-08-25T09:39:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/3138632846 >> v2:172.21.15.155:3300/0 conn(0x7fd4c815ec10 0x7fd4c80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/3138632846 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815e830 0x7fd4c806ad70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 -- 172.21.15.70:0/3138632846 >> 172.21.15.70:0/3138632846 conn(0x7fd4c8069850 msgr2=0x7fd4c8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 -- 172.21.15.70:0/3138632846 shutdown_connections 2023-08-25T09:39:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.971+0000 7fd4ce61f640 1 -- 172.21.15.70:0/3138632846 wait complete. 2023-08-25T09:39:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.974+0000 7fd4ce61f640 1 Processor -- start 2023-08-25T09:39:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.974+0000 7fd4ce61f640 1 -- start start 2023-08-25T09:39:41.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4ce61f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 0x7fd4c814b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4ce61f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c815e830 0x7fd4c80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4ce61f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815ec10 0x7fd4c815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4ce61f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c814b700 con 0x7fd4c806b2b0 2023-08-25T09:39:41.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4ce61f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c80c8130 con 0x7fd4c815e830 2023-08-25T09:39:41.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4ce61f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c8156350 con 0x7fd4c815ec10 2023-08-25T09:39:41.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 0x7fd4c814b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 0x7fd4c814b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42474/0 (socket says 172.21.15.70:42474) 2023-08-25T09:39:41.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.975+0000 7fd4c7fff640 1 -- 172.21.15.70:0/334520041 learned_addr learned my addr 172.21.15.70:0/334520041 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:41.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4ccbbc640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815ec10 0x7fd4c815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c7fff640 1 -- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815ec10 msgr2=0x7fd4c815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c77fe640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c815e830 0x7fd4c80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c7fff640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815ec10 0x7fd4c815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c7fff640 1 -- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c815e830 msgr2=0x7fd4c80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:41.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c7fff640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c815e830 0x7fd4c80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:41.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c7fff640 1 -- 172.21.15.70:0/334520041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00094d0 con 0x7fd4c806b2b0 2023-08-25T09:39:41.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c77fe640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c815e830 0x7fd4c80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.976+0000 7fd4c7fff640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 0x7fd4c814b1c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80098c0 tx=0x7fd4b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.977+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8006e10 con 0x7fd4c806b2b0 2023-08-25T09:39:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.977+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4b803ad60 con 0x7fd4c806b2b0 2023-08-25T09:39:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.977+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8044620 con 0x7fd4c806b2b0 2023-08-25T09:39:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.977+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b8009400 con 0x7fd4c806b2b0 2023-08-25T09:39:41.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.977+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c815ca20 con 0x7fd4c806b2b0 2023-08-25T09:39:41.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.978+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c80b17d0 con 0x7fd4c806b2b0 2023-08-25T09:39:41.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.979+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4b804d440 con 0x7fd4c806b2b0 2023-08-25T09:39:41.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.980+0000 7fd4c4ff9640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4a806a050 0x7fd4a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:41.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.980+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4b80aa930 con 0x7fd4c806b2b0 2023-08-25T09:39:41.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.980+0000 7fd4c77fe640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4a806a050 0x7fd4a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:41.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.981+0000 7fd4c77fe640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4a806a050 0x7fd4a806c510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80afc80 tx=0x7fd4b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:41.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:41.984+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd4c80b17d0 con 0x7fd4c806b2b0 2023-08-25T09:39:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.258+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fd4c80b17d0 con 0x7fd4c806b2b0 2023-08-25T09:39:42.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.465+0000 7fd4c4ff9640 1 -- 172.21.15.70:0/334520041 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v64) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd4c80b17d0 con 0x7fd4c806b2b0 2023-08-25T09:39:42.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4a806a050 msgr2=0x7fd4a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4a806a050 0x7fd4a806c510 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80afc80 tx=0x7fd4b0009040 comp rx=0 tx=0).stop 2023-08-25T09:39:42.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 msgr2=0x7fd4c814b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 0x7fd4c814b1c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80098c0 tx=0x7fd4b80046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 shutdown_connections 2023-08-25T09:39:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4a806a050 0x7fd4a806c510 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd4c815ec10 0x7fd4c815d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4c815e830 0x7fd4c80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 --2- 172.21.15.70:0/334520041 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4c806b2b0 0x7fd4c814b1c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.468+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 >> 172.21.15.70:0/334520041 conn(0x7fd4c8069850 msgr2=0x7fd4c80ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.469+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 shutdown_connections 2023-08-25T09:39:42.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.469+0000 7fd4ce61f640 1 -- 172.21.15.70:0/334520041 wait complete. 2023-08-25T09:39:42.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-08-25T09:39:42.712 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_objectstore":"memstore"} 2023-08-25T09:39:42.713 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-08-25T09:39:42.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:39:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.934+0000 7f126c90b640 1 Processor -- start 2023-08-25T09:39:42.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.934+0000 7f126c90b640 1 -- start start 2023-08-25T09:39:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.935+0000 7f126c90b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12680c9e60 0x7f12680b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.935+0000 7f126c90b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12680ca270 con 0x7f126815a920 2023-08-25T09:39:42.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.935+0000 7f126c90b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12680adbb0 con 0x7f12680b0320 2023-08-25T09:39:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.935+0000 7f126c90b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12680b2220 con 0x7f12680c9e60 2023-08-25T09:39:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.936+0000 7f126659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f126815a920 0x7f12680ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36304/0 (socket says 172.21.15.70:36304) 2023-08-25T09:39:42.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.936+0000 7f126659c640 1 -- 172.21.15.70:0/4029305368 learned_addr learned my addr 172.21.15.70:0/4029305368 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.936+0000 7f126759e640 1 --2- 172.21.15.70:0/4029305368 >> v2:172.21.15.155:3300/0 conn(0x7f12680c9e60 0x7f12680b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.936+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501734794 0 0) 0x7f12680ca270 con 0x7f126815a920 2023-08-25T09:39:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.936+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1250003520 con 0x7f126815a920 2023-08-25T09:39:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 787088597 0 0) 0x7f1250003520 con 0x7f126815a920 2023-08-25T09:39:42.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 >> v2:172.21.15.155:3300/0 conn(0x7f12680c9e60 msgr2=0x7f12680b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 --2- 172.21.15.70:0/4029305368 >> v2:172.21.15.155:3300/0 conn(0x7f12680c9e60 0x7f12680b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 >> v1:172.21.15.155:6789/0 conn(0x7f12680b0320 legacy=0x7f12680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680c90a0 con 0x7f126815a920 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f125c001670 con 0x7f126815a920 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 >> v1:172.21.15.70:6789/0 conn(0x7f126815a920 legacy=0x7f12680ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1250003970 0x7f1250003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12500047c0 0x7f1250004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1250005180 0x7f125000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125000c0a0 con 0x7f12500047c0 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125000c210 con 0x7f1250003970 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125000c3d0 con 0x7f1250005180 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f126659c640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12500047c0 0x7f1250004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.937+0000 7f1266d9d640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1250003970 0x7f1250003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126759e640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1250005180 0x7f125000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126659c640 1 -- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1250005180 msgr2=0x7f125000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126659c640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1250005180 0x7f125000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126659c640 1 -- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1250003970 msgr2=0x7f1250003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:42.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126659c640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1250003970 0x7f1250003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126659c640 1 -- 172.21.15.70:0/4029305368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250003520 con 0x7f12500047c0 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f1266d9d640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1250003970 0x7f1250003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.938+0000 7f126759e640 1 -- 172.21.15.70:0/4029305368 reap_dead start 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.939+0000 7f126659c640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12500047c0 0x7f1250004c40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f125c00c210 tx=0x7f12500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61c18ffe4ecafee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.939+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c005ad0 con 0x7f12500047c0 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.939+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c005c30 con 0x7f12500047c0 2023-08-25T09:39:42.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.939+0000 7f1265d9b640 1 -- 172.21.15.70:0/4029305368 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c02ca80 con 0x7f12500047c0 2023-08-25T09:39:42.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.939+0000 7f126c90b640 1 -- 172.21.15.70:0/4029305368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12500047c0 msgr2=0x7f1250004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.939+0000 7f126c90b640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12500047c0 0x7f1250004c40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f125c00c210 tx=0x7f12500057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.940+0000 7f126c90b640 1 -- 172.21.15.70:0/4029305368 shutdown_connections 2023-08-25T09:39:42.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.940+0000 7f126c90b640 1 --2- 172.21.15.70:0/4029305368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12500047c0 0x7f1250004c40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.940+0000 7f126c90b640 1 -- 172.21.15.70:0/4029305368 >> 172.21.15.70:0/4029305368 conn(0x7f12680ce830 msgr2=0x7f12680cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.940+0000 7f126c90b640 1 -- 172.21.15.70:0/4029305368 shutdown_connections 2023-08-25T09:39:42.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.940+0000 7f126c90b640 1 -- 172.21.15.70:0/4029305368 wait complete. 2023-08-25T09:39:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.941+0000 7f126c90b640 1 Processor -- start 2023-08-25T09:39:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.941+0000 7f126c90b640 1 -- start start 2023-08-25T09:39:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.942+0000 7f126c90b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500047c0 0x7f12680c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.942+0000 7f126c90b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12680c1e80 0x7f12680c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.942+0000 7f126c90b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 0x7f12681f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.942+0000 7f126c90b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268152d50 con 0x7f12680c2820 2023-08-25T09:39:42.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.942+0000 7f126c90b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680c2ca0 con 0x7f12500047c0 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.942+0000 7f126c90b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1268152ee0 con 0x7f12680c1e80 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 0x7f12681f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 0x7f12681f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42486/0 (socket says 172.21.15.70:42486) 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 -- 172.21.15.70:0/3169121283 learned_addr learned my addr 172.21.15.70:0/3169121283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126659c640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12680c1e80 0x7f12680c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 -- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12680c1e80 msgr2=0x7f12680c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12680c1e80 0x7f12680c22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 -- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500047c0 msgr2=0x7f12680c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500047c0 0x7f12680c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 -- 172.21.15.70:0/3169121283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680c4c20 con 0x7f12680c2820 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.943+0000 7f126759e640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 0x7f12681f3090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f12680c1e50 tx=0x7f1260009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.944+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260003c20 con 0x7f12680c2820 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.944+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1260007cf0 con 0x7f12680c2820 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.944+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1260014640 con 0x7f12680c2820 2023-08-25T09:39:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.944+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12681531a0 con 0x7f12680c2820 2023-08-25T09:39:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.944+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1268153760 con 0x7f12680c2820 2023-08-25T09:39:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.945+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1268061460 con 0x7f12680c2820 2023-08-25T09:39:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.946+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f126001d3b0 con 0x7f12680c2820 2023-08-25T09:39:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.947+0000 7f124f7fe640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f125406a050 0x7f125406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.947+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12600882f0 con 0x7f12680c2820 2023-08-25T09:39:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.947+0000 7f1266d9d640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f125406a050 0x7f125406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.948+0000 7f1266d9d640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f125406a050 0x7f125406c510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12580016c0 tx=0x7f1258009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:42.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:42.952+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1260057470 con 0x7f12680c2820 2023-08-25T09:39:42.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:39:42.963 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:39:43.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.134+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f12680ca720 con 0x7f12680c2820 2023-08-25T09:39:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.139+0000 7f124f7fe640 1 -- 172.21.15.70:0/3169121283 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1260007e60 con 0x7f12680c2820 2023-08-25T09:39:43.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.141+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f125406a050 msgr2=0x7f125406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.141+0000 7f126c90b640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f125406a050 0x7f125406c510 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f12580016c0 tx=0x7f1258009040 comp rx=0 tx=0).stop 2023-08-25T09:39:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.141+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 msgr2=0x7f12681f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.141+0000 7f126c90b640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 0x7f12681f3090 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f12680c1e50 tx=0x7f1260009cd0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 shutdown_connections 2023-08-25T09:39:43.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f125406a050 0x7f125406c510 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12680c2820 0x7f12681f3090 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12680c1e80 0x7f12680c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 --2- 172.21.15.70:0/3169121283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12500047c0 0x7f12680c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 >> 172.21.15.70:0/3169121283 conn(0x7f12680ce830 msgr2=0x7f12680af340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:43.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 shutdown_connections 2023-08-25T09:39:43.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.142+0000 7f126c90b640 1 -- 172.21.15.70:0/3169121283 wait complete. 2023-08-25T09:39:43.164 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:39:43.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:39:43.277 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:39:43.402 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:39:43.403 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:39:43.404 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:43.465 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:39:43.466 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:43.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:43.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.779+0000 7f9c7ffff640 1 Processor -- start 2023-08-25T09:39:43.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.779+0000 7f9c7ffff640 1 -- start start 2023-08-25T09:39:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c8015a7c0 0x7f9c800b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:43.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c800b12f0 0x7f9c8015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:43.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7ffff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8015e7c0 con 0x7f9c8015f190 2023-08-25T09:39:43.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7ffff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c80071b50 con 0x7f9c8015a7c0 2023-08-25T09:39:43.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8015e490 con 0x7f9c800b12f0 2023-08-25T09:39:43.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c8015a7c0 0x7f9c800b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c8015a7c0 0x7f9c800b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42500/0 (socket says 172.21.15.70:42500) 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c8015f190 0x7f9c800cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36320/0 (socket says 172.21.15.70:36320) 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7effd640 1 -- 172.21.15.70:0/2289604221 learned_addr learned my addr 172.21.15.70:0/2289604221 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7e7fc640 1 --2- 172.21.15.70:0/2289604221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c800b12f0 0x7f9c8015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7effd640 1 -- 172.21.15.70:0/2289604221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c800b12f0 msgr2=0x7f9c8015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.782+0000 7f9c7effd640 1 --2- 172.21.15.70:0/2289604221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c800b12f0 0x7f9c8015ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.782+0000 7f9c7effd640 1 -- 172.21.15.70:0/2289604221 >> v1:172.21.15.70:6789/0 conn(0x7f9c8015f190 legacy=0x7f9c800cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.782+0000 7f9c7effd640 1 -- 172.21.15.70:0/2289604221 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800b3aa0 con 0x7f9c8015a7c0 2023-08-25T09:39:43.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.781+0000 7f9c7dffb640 1 -- 172.21.15.70:0/2289604221 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258740377 0 0) 0x7f9c8015e7c0 con 0x7f9c8015f190 2023-08-25T09:39:43.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.783+0000 7f9c7effd640 1 --2- 172.21.15.70:0/2289604221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8015a7c0 0x7f9c800b3560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c7002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b694dc8ef400d86 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:43.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.783+0000 7f9c7dffb640 1 -- 172.21.15.70:0/2289604221 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70030840 con 0x7f9c8015a7c0 2023-08-25T09:39:43.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.783+0000 7f9c7dffb640 1 -- 172.21.15.70:0/2289604221 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c700309a0 con 0x7f9c8015a7c0 2023-08-25T09:39:43.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.783+0000 7f9c7dffb640 1 -- 172.21.15.70:0/2289604221 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70030d30 con 0x7f9c8015a7c0 2023-08-25T09:39:43.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.783+0000 7f9c7ffff640 1 -- 172.21.15.70:0/2289604221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8015a7c0 msgr2=0x7f9c800b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.783+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/2289604221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8015a7c0 0x7f9c800b3560 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c7002b980 comp rx=0 tx=0).stop 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.784+0000 7f9c7ffff640 1 -- 172.21.15.70:0/2289604221 shutdown_connections 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.784+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/2289604221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c800b12f0 0x7f9c8015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.784+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/2289604221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c8015a7c0 0x7f9c800b3560 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.784+0000 7f9c7ffff640 1 -- 172.21.15.70:0/2289604221 >> 172.21.15.70:0/2289604221 conn(0x7f9c800affd0 msgr2=0x7f9c800b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.784+0000 7f9c7ffff640 1 -- 172.21.15.70:0/2289604221 shutdown_connections 2023-08-25T09:39:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.784+0000 7f9c7ffff640 1 -- 172.21.15.70:0/2289604221 wait complete. 2023-08-25T09:39:43.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.785+0000 7f9c7ffff640 1 Processor -- start 2023-08-25T09:39:43.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.785+0000 7f9c7ffff640 1 -- start start 2023-08-25T09:39:43.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.786+0000 7f9c7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 0x7f9c801ef900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:43.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.786+0000 7f9c7ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c8015a7c0 0x7f9c800c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:43.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.786+0000 7f9c7ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c8015f190 0x7f9c800af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.786+0000 7f9c7ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800c6a60 con 0x7f9c800b12f0 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.786+0000 7f9c7ffff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800c1fd0 con 0x7f9c8015a7c0 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.786+0000 7f9c7ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800c22b0 con 0x7f9c8015f190 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 0x7f9c801ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 0x7f9c801ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42512/0 (socket says 172.21.15.70:42512) 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 -- 172.21.15.70:0/1125017926 learned_addr learned my addr 172.21.15.70:0/1125017926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 -- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c8015f190 msgr2=0x7f9c800af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c8015f190 0x7f9c800af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7e7fc640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c8015a7c0 0x7f9c800c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 -- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c8015a7c0 msgr2=0x7f9c800c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c8015a7c0 0x7f9c800c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 -- 172.21.15.70:0/1125017926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c700094d0 con 0x7f9c800b12f0 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.787+0000 7f9c7effd640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 0x7f9c801ef900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.788+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c700307f0 con 0x7f9c800b12f0 2023-08-25T09:39:43.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.788+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70043400 con 0x7f9c800b12f0 2023-08-25T09:39:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.788+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c700437b0 con 0x7f9c800b12f0 2023-08-25T09:39:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.788+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c800c2590 con 0x7f9c800b12f0 2023-08-25T09:39:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.788+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c800c8350 con 0x7f9c800b12f0 2023-08-25T09:39:43.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.789+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c800c2ab0 con 0x7f9c800b12f0 2023-08-25T09:39:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.790+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9c70043910 con 0x7f9c800b12f0 2023-08-25T09:39:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.791+0000 7f9c67fff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c6c06a050 0x7f9c6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.791+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c7003e070 con 0x7f9c800b12f0 2023-08-25T09:39:43.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.791+0000 7f9c7e7fc640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c6c06a050 0x7f9c6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:43.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.792+0000 7f9c7e7fc640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c6c06a050 0x7f9c6c06c510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c80152b20 tx=0x7f9c74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:43.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.795+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9c800c2ab0 con 0x7f9c800b12f0 2023-08-25T09:39:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.987+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c800c2ab0 con 0x7f9c800b12f0 2023-08-25T09:39:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.989+0000 7f9c67fff640 1 -- 172.21.15.70:0/1125017926 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f9c800c2ab0 con 0x7f9c800b12f0 2023-08-25T09:39:43.991 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:43.991 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c6c06a050 msgr2=0x7f9c6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c6c06a050 0x7f9c6c06c510 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c80152b20 tx=0x7f9c74008040 comp rx=0 tx=0).stop 2023-08-25T09:39:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 msgr2=0x7f9c801ef900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 0x7f9c801ef900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9c70009870 tx=0x7f9c70004730 comp rx=0 tx=0).stop 2023-08-25T09:39:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 shutdown_connections 2023-08-25T09:39:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c6c06a050 0x7f9c6c06c510 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c8015f190 0x7f9c800af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c8015a7c0 0x7f9c800c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 --2- 172.21.15.70:0/1125017926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c800b12f0 0x7f9c801ef900 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 >> 172.21.15.70:0/1125017926 conn(0x7f9c800affd0 msgr2=0x7f9c800b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 shutdown_connections 2023-08-25T09:39:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:43.992+0000 7f9c7ffff640 1 -- 172.21.15.70:0/1125017926 wait complete. 2023-08-25T09:39:44.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:44.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:44.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.227+0000 7fab504c5640 1 Processor -- start 2023-08-25T09:39:44.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.227+0000 7fab504c5640 1 -- start start 2023-08-25T09:39:44.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab504c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab48069f60 0x7fab480ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab504c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480c2550 con 0x7fab4815ae50 2023-08-25T09:39:44.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab504c5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab480ce330 con 0x7fab480c2110 2023-08-25T09:39:44.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab504c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab480cc5f0 con 0x7fab48069f60 2023-08-25T09:39:44.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab4e261640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fab4815ae50 0x7fab480cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36322/0 (socket says 172.21.15.70:36322) 2023-08-25T09:39:44.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab4e261640 1 -- 172.21.15.70:0/2256338593 learned_addr learned my addr 172.21.15.70:0/2256338593 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:44.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.229+0000 7fab4ea62640 1 --2- 172.21.15.70:0/2256338593 >> v2:172.21.15.70:3300/0 conn(0x7fab48069f60 0x7fab480ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.230+0000 7fab4d25f640 1 -- 172.21.15.70:0/2256338593 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193095947 0 0) 0x7fab480c2550 con 0x7fab4815ae50 2023-08-25T09:39:44.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.230+0000 7fab4d25f640 1 -- 172.21.15.70:0/2256338593 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab34003520 con 0x7fab4815ae50 2023-08-25T09:39:44.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.230+0000 7fab4ea62640 1 -- 172.21.15.70:0/2256338593 >> v1:172.21.15.155:6789/0 conn(0x7fab480c2110 legacy=0x7fab48069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.230+0000 7fab4ea62640 1 -- 172.21.15.70:0/2256338593 >> v1:172.21.15.70:6789/0 conn(0x7fab4815ae50 legacy=0x7fab480cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.230+0000 7fab4ea62640 1 -- 172.21.15.70:0/2256338593 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48070650 con 0x7fab48069f60 2023-08-25T09:39:44.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.231+0000 7fab4ea62640 1 --2- 172.21.15.70:0/2256338593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab48069f60 0x7fab480ccad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d90 tx=0x7fab4002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcff26947282979a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:44.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.231+0000 7fab4d25f640 1 -- 172.21.15.70:0/2256338593 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4002cd80 con 0x7fab48069f60 2023-08-25T09:39:44.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.231+0000 7fab4d25f640 1 -- 172.21.15.70:0/2256338593 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40034440 con 0x7fab48069f60 2023-08-25T09:39:44.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.231+0000 7fab4d25f640 1 -- 172.21.15.70:0/2256338593 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab400347b0 con 0x7fab48069f60 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 -- 172.21.15.70:0/2256338593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab48069f60 msgr2=0x7fab480ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 --2- 172.21.15.70:0/2256338593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab48069f60 0x7fab480ccad0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d90 tx=0x7fab4002bfd0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 -- 172.21.15.70:0/2256338593 shutdown_connections 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 --2- 172.21.15.70:0/2256338593 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab48069f60 0x7fab480ccad0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 -- 172.21.15.70:0/2256338593 >> 172.21.15.70:0/2256338593 conn(0x7fab480cb0d0 msgr2=0x7fab480cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 -- 172.21.15.70:0/2256338593 shutdown_connections 2023-08-25T09:39:44.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.232+0000 7fab504c5640 1 -- 172.21.15.70:0/2256338593 wait complete. 2023-08-25T09:39:44.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.234+0000 7fab504c5640 1 Processor -- start 2023-08-25T09:39:44.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.234+0000 7fab504c5640 1 -- start start 2023-08-25T09:39:44.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab504c5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab48069f60 0x7fab481f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab504c5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab480c2110 0x7fab481e4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab504c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 0x7fab481e4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab504c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab481f4b80 con 0x7fab4815ae50 2023-08-25T09:39:44.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab504c5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab481e56f0 con 0x7fab480c2110 2023-08-25T09:39:44.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab504c5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab481e4e50 con 0x7fab48069f60 2023-08-25T09:39:44.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab4ea62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 0x7fab481e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab4ea62640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 0x7fab481e4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42536/0 (socket says 172.21.15.70:42536) 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.235+0000 7fab4ea62640 1 -- 172.21.15.70:0/4270914894 learned_addr learned my addr 172.21.15.70:0/4270914894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4e261640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab48069f60 0x7fab481f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4da60640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab480c2110 0x7fab481e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4ea62640 1 -- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab48069f60 msgr2=0x7fab481f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4ea62640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab48069f60 0x7fab481f3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4ea62640 1 -- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab480c2110 msgr2=0x7fab481e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4ea62640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab480c2110 0x7fab481e4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4ea62640 1 -- 172.21.15.70:0/4270914894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab40009a70 con 0x7fab4815ae50 2023-08-25T09:39:44.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab4ea62640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 0x7fab481e4780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d90 tx=0x7fab40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:44.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40003b90 con 0x7fab4815ae50 2023-08-25T09:39:44.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40003d10 con 0x7fab4815ae50 2023-08-25T09:39:44.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40004270 con 0x7fab4815ae50 2023-08-25T09:39:44.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.236+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab481e5130 con 0x7fab4815ae50 2023-08-25T09:39:44.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.237+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab481e2590 con 0x7fab4815ae50 2023-08-25T09:39:44.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.238+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab480ae6d0 con 0x7fab4815ae50 2023-08-25T09:39:44.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.239+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fab4003b050 con 0x7fab4815ae50 2023-08-25T09:39:44.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.239+0000 7fab3effd640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab2c06a080 0x7fab2c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.240+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab400aaac0 con 0x7fab4815ae50 2023-08-25T09:39:44.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.240+0000 7fab4e261640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab2c06a080 0x7fab2c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.240+0000 7fab4e261640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab2c06a080 0x7fab2c06c540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab440011e0 tx=0x7fab44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:44.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.244+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fab400761a0 con 0x7fab4815ae50 2023-08-25T09:39:44.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.431+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab48066fb0 con 0x7fab4815ae50 2023-08-25T09:39:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.432+0000 7fab3effd640 1 -- 172.21.15.70:0/4270914894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab40039070 con 0x7fab4815ae50 2023-08-25T09:39:44.435 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab2c06a080 msgr2=0x7fab2c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab2c06a080 0x7fab2c06c540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fab440011e0 tx=0x7fab44009040 comp rx=0 tx=0).stop 2023-08-25T09:39:44.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 msgr2=0x7fab481e4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 0x7fab481e4780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d90 tx=0x7fab40004a40 comp rx=0 tx=0).stop 2023-08-25T09:39:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 shutdown_connections 2023-08-25T09:39:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab2c06a080 0x7fab2c06c540 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab4815ae50 0x7fab481e4780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab480c2110 0x7fab481e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 --2- 172.21.15.70:0/4270914894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab48069f60 0x7fab481f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 >> 172.21.15.70:0/4270914894 conn(0x7fab480cb0d0 msgr2=0x7fab48159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:44.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 shutdown_connections 2023-08-25T09:39:44.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.435+0000 7fab504c5640 1 -- 172.21.15.70:0/4270914894 wait complete. 2023-08-25T09:39:44.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:44.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.676+0000 7fc9c42ba640 1 Processor -- start 2023-08-25T09:39:44.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.676+0000 7fc9c42ba640 1 -- start start 2023-08-25T09:39:44.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c42ba640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc9bc0b0960 0x7fc9bc0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c42ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 0x7fc9bc06b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c42ba640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9bc0cb710 con 0x7fc9bc0ce110 2023-08-25T09:39:44.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c42ba640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9bc0cecb0 con 0x7fc9bc0b0960 2023-08-25T09:39:44.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c42ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0b1280 con 0x7fc9bc06ae60 2023-08-25T09:39:44.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c1855640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc9bc0ce110 0x7fc9bc0ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36326/0 (socket says 172.21.15.70:36326) 2023-08-25T09:39:44.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.678+0000 7fc9c1855640 1 -- 172.21.15.70:0/1546264009 learned_addr learned my addr 172.21.15.70:0/1546264009 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:44.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c2056640 1 --2- 172.21.15.70:0/1546264009 >> v2:172.21.15.155:3300/0 conn(0x7fc9bc0b0960 0x7fc9bc0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c2857640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 0x7fc9bc06b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612177830 0 0) 0x7fc9bc0cb710 con 0x7fc9bc0ce110 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a8003520 con 0x7fc9bc0ce110 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 835911284 0 0) 0x7fc9a8003520 con 0x7fc9bc0ce110 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 msgr2=0x7fc9bc06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 0x7fc9bc06b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 >> v2:172.21.15.155:3300/0 conn(0x7fc9bc0b0960 msgr2=0x7fc9bc0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 --2- 172.21.15.70:0/1546264009 >> v2:172.21.15.155:3300/0 conn(0x7fc9bc0b0960 0x7fc9bc0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc0ccb50 con 0x7fc9bc0ce110 2023-08-25T09:39:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc9ac001670 con 0x7fc9bc0ce110 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 >> v1:172.21.15.70:6789/0 conn(0x7fc9bc0ce110 legacy=0x7fc9bc0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 0x7fc9a8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0cecb0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.679+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc0b1280 con 0x7fc9a8004d00 2023-08-25T09:39:44.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a800b9d0 con 0x7fc9bc06ae60 2023-08-25T09:39:44.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1855640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c2056640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 0x7fc9a8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c2857640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1855640 1 -- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 msgr2=0x7fc9a8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:44.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1855640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc06ae60 0x7fc9a8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1855640 1 -- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8004d00 msgr2=0x7fc9a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:44.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1855640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a8004d00 0x7fc9a8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.680+0000 7fc9c1855640 1 -- 172.21.15.70:0/1546264009 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a8003520 con 0x7fc9a8003ef0 2023-08-25T09:39:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.681+0000 7fc9c1855640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9a80047c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c270 tx=0x7fc9a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ac8438b389cca7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac02cae0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ac02cc40 con 0x7fc9a8003ef0 2023-08-25T09:39:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c1054640 1 -- 172.21.15.70:0/1546264009 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac005af0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c42ba640 1 -- 172.21.15.70:0/1546264009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 msgr2=0x7fc9a80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/1546264009 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9a80047c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00c270 tx=0x7fc9a80057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c2857640 1 -- 172.21.15.70:0/1546264009 reap_dead start 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c42ba640 1 -- 172.21.15.70:0/1546264009 shutdown_connections 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c42ba640 1 -- 172.21.15.70:0/1546264009 >> 172.21.15.70:0/1546264009 conn(0x7fc9bc0cb960 msgr2=0x7fc9bc0cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c42ba640 1 -- 172.21.15.70:0/1546264009 shutdown_connections 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.682+0000 7fc9c42ba640 1 -- 172.21.15.70:0/1546264009 wait complete. 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.684+0000 7fc9c42ba640 1 Processor -- start 2023-08-25T09:39:44.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.684+0000 7fc9c42ba640 1 -- start start 2023-08-25T09:39:44.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c42ba640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9bc1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c42ba640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc158ec0 0x7fc9bc15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c42ba640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc1593f0 0x7fc9bc070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c42ba640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15da20 con 0x7fc9a8003ef0 2023-08-25T09:39:44.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c42ba640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15dd00 con 0x7fc9bc1593f0 2023-08-25T09:39:44.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c42ba640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9bc15dfe0 con 0x7fc9bc158ec0 2023-08-25T09:39:44.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.685+0000 7fc9c2056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9bc1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9bc1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42568/0 (socket says 172.21.15.70:42568) 2023-08-25T09:39:44.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 -- 172.21.15.70:0/922970224 learned_addr learned my addr 172.21.15.70:0/922970224 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:44.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2857640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc1593f0 0x7fc9bc070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c1855640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc158ec0 0x7fc9bc15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 -- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc158ec0 msgr2=0x7fc9bc15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc158ec0 0x7fc9bc15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 -- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc1593f0 msgr2=0x7fc9bc070ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:44.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc1593f0 0x7fc9bc070ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:44.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 -- 172.21.15.70:0/922970224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9bc15e2c0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c1855640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc158ec0 0x7fc9bc15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:44.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.686+0000 7fc9c2056640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9bc1586a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011600 tx=0x7fc9b8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:44.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.687+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8015870 con 0x7fc9a8003ef0 2023-08-25T09:39:44.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.687+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b80159d0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.687+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8015d40 con 0x7fc9a8003ef0 2023-08-25T09:39:44.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.687+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9bc15e5a0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.687+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9bc09b980 con 0x7fc9a8003ef0 2023-08-25T09:39:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.688+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9bc066fb0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.689+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9b800f490 con 0x7fc9a8003ef0 2023-08-25T09:39:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.690+0000 7fc9b2ffd640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9a006a050 0x7fc9a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.690+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9b8088a50 con 0x7fc9a8003ef0 2023-08-25T09:39:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.690+0000 7fc9c1855640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9a006a050 0x7fc9a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:44.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.691+0000 7fc9c1855640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9a006a050 0x7fc9a006c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac034000 tx=0x7fc9ac007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:44.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.694+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9b8057bc0 con 0x7fc9a8003ef0 2023-08-25T09:39:44.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.890+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9bc0cd6c0 con 0x7fc9a8003ef0 2023-08-25T09:39:45.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.891+0000 7fc9b2ffd640 1 -- 172.21.15.70:0/922970224 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc9b8057750 con 0x7fc9a8003ef0 2023-08-25T09:39:45.489 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:45.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.893+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9a006a050 msgr2=0x7fc9a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.893+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9a006a050 0x7fc9a006c510 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac034000 tx=0x7fc9ac007000 comp rx=0 tx=0).stop 2023-08-25T09:39:45.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.893+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 msgr2=0x7fc9bc1586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.893+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9bc1586a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8011600 tx=0x7fc9b8011ac0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 shutdown_connections 2023-08-25T09:39:45.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9a006a050 0x7fc9a006c510 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9bc1593f0 0x7fc9bc070ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9bc158ec0 0x7fc9bc15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 --2- 172.21.15.70:0/922970224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a8003ef0 0x7fc9bc1586a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 >> 172.21.15.70:0/922970224 conn(0x7fc9bc0cb960 msgr2=0x7fc9bc0af350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:45.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 shutdown_connections 2023-08-25T09:39:45.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:44.894+0000 7fc9c42ba640 1 -- 172.21.15.70:0/922970224 wait complete. 2023-08-25T09:39:45.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:45.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.715+0000 7f5a67c5e640 1 Processor -- start 2023-08-25T09:39:45.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.715+0000 7f5a67c5e640 1 -- start start 2023-08-25T09:39:45.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a67c5e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a60072f30 0x7f5a600acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:45.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a67c5e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5a600b2870 0x7f5a600ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:45.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a67c5e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6015d340 con 0x7f5a60153d70 2023-08-25T09:39:45.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a67c5e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a60154150 con 0x7f5a60072f30 2023-08-25T09:39:45.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a67c5e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6015d620 con 0x7f5a600b2870 2023-08-25T09:39:45.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a651f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a60072f30 0x7f5a600acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:45.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.717+0000 7f5a651f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5a60072f30 0x7f5a600acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42576/0 (socket says 172.21.15.70:42576) 2023-08-25T09:39:45.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.718+0000 7f5a651f9640 1 -- 172.21.15.70:0/1519984769 learned_addr learned my addr 172.21.15.70:0/1519984769 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:45.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.718+0000 7f5a651f9640 1 -- 172.21.15.70:0/1519984769 >> v2:172.21.15.155:3300/0 conn(0x7f5a600b2870 msgr2=0x7f5a600ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:45.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.718+0000 7f5a661fb640 1 --2- 172.21.15.70:0/1519984769 >> v2:172.21.15.155:3300/0 conn(0x7f5a600b2870 0x7f5a600ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:45.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.718+0000 7f5a651f9640 1 --2- 172.21.15.70:0/1519984769 >> v2:172.21.15.155:3300/0 conn(0x7f5a600b2870 0x7f5a600ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.718+0000 7f5a651f9640 1 -- 172.21.15.70:0/1519984769 >> v1:172.21.15.155:6789/0 conn(0x7f5a60153d70 legacy=0x7f5a60072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:45.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.718+0000 7f5a651f9640 1 -- 172.21.15.70:0/1519984769 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6015d900 con 0x7f5a60072f30 2023-08-25T09:39:45.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.719+0000 7f5a651f9640 1 --2- 172.21.15.70:0/1519984769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a600acc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5a600b4650 tx=0x7f5a5002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543c5b76bc41b267 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:45.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a649f8640 1 -- 172.21.15.70:0/1519984769 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a500317e0 con 0x7f5a60072f30 2023-08-25T09:39:45.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a649f8640 1 -- 172.21.15.70:0/1519984769 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a50031940 con 0x7f5a60072f30 2023-08-25T09:39:45.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a649f8640 1 -- 172.21.15.70:0/1519984769 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a50031cd0 con 0x7f5a60072f30 2023-08-25T09:39:45.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 -- 172.21.15.70:0/1519984769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 msgr2=0x7f5a600acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/1519984769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a600acc70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5a600b4650 tx=0x7f5a5002b880 comp rx=0 tx=0).stop 2023-08-25T09:39:45.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 -- 172.21.15.70:0/1519984769 shutdown_connections 2023-08-25T09:39:45.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/1519984769 >> v2:172.21.15.155:3300/0 conn(0x7f5a600b2870 0x7f5a600ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/1519984769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a600acc70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 -- 172.21.15.70:0/1519984769 >> 172.21.15.70:0/1519984769 conn(0x7f5a600cfc70 msgr2=0x7f5a600d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:45.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 -- 172.21.15.70:0/1519984769 shutdown_connections 2023-08-25T09:39:45.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.720+0000 7f5a67c5e640 1 -- 172.21.15.70:0/1519984769 wait complete. 2023-08-25T09:39:45.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.722+0000 7f5a67c5e640 1 Processor -- start 2023-08-25T09:39:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.722+0000 7f5a67c5e640 1 -- start start 2023-08-25T09:39:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a67c5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a60156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a67c5e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a600b2870 0x7f5a601f16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:45.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a67c5e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a60153d70 0x7f5a601f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a67c5e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60146460 con 0x7f5a60072f30 2023-08-25T09:39:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a67c5e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60142fd0 con 0x7f5a60153d70 2023-08-25T09:39:45.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a67c5e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60142cf0 con 0x7f5a600b2870 2023-08-25T09:39:45.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.723+0000 7f5a659fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a60156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:45.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a60156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42578/0 (socket says 172.21.15.70:42578) 2023-08-25T09:39:45.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 -- 172.21.15.70:0/3292765984 learned_addr learned my addr 172.21.15.70:0/3292765984 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:45.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a661fb640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a60153d70 0x7f5a601f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:45.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a651f9640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a600b2870 0x7f5a601f16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:45.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 -- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a600b2870 msgr2=0x7f5a601f16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a600b2870 0x7f5a601f16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 -- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a60153d70 msgr2=0x7f5a601f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a60153d70 0x7f5a601f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 -- 172.21.15.70:0/3292765984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500094d0 con 0x7f5a60072f30 2023-08-25T09:39:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a661fb640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a60153d70 0x7f5a601f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:45.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a651f9640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a600b2870 0x7f5a601f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.724+0000 7f5a659fa640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a60156ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b3f0 tx=0x7f5a5c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.725+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c011830 con 0x7f5a60072f30 2023-08-25T09:39:45.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.725+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c011990 con 0x7f5a60072f30 2023-08-25T09:39:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.725+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c011d40 con 0x7f5a60072f30 2023-08-25T09:39:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.725+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a600cd900 con 0x7f5a60072f30 2023-08-25T09:39:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.725+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a600cdec0 con 0x7f5a60072f30 2023-08-25T09:39:45.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.726+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a600738b0 con 0x7f5a60072f30 2023-08-25T09:39:45.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.727+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5a5c01a400 con 0x7f5a60072f30 2023-08-25T09:39:45.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.728+0000 7f5a567fc640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a4406a050 0x7f5a4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:45.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.728+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a5c0884d0 con 0x7f5a60072f30 2023-08-25T09:39:45.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.728+0000 7f5a651f9640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a4406a050 0x7f5a4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:45.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.729+0000 7f5a651f9640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a4406a050 0x7f5a4406c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a50036000 tx=0x7f5a5003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:45.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.732+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5a5c053b30 con 0x7f5a60072f30 2023-08-25T09:39:45.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.914+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a600cd760 con 0x7f5a60072f30 2023-08-25T09:39:45.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.915+0000 7f5a567fc640 1 -- 172.21.15.70:0/3292765984 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a600cd760 con 0x7f5a60072f30 2023-08-25T09:39:45.917 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:45.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a4406a050 msgr2=0x7f5a4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a4406a050 0x7f5a4406c510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5a50036000 tx=0x7f5a5003d040 comp rx=0 tx=0).stop 2023-08-25T09:39:45.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 msgr2=0x7f5a60156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:45.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a60156ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00b3f0 tx=0x7f5a5c00b8b0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 shutdown_connections 2023-08-25T09:39:45.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a4406a050 0x7f5a4406c510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a60153d70 0x7f5a601f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a600b2870 0x7f5a601f16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 --2- 172.21.15.70:0/3292765984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a60072f30 0x7f5a60156ae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:45.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 >> 172.21.15.70:0/3292765984 conn(0x7f5a600cfc70 msgr2=0x7f5a600acfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:45.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 shutdown_connections 2023-08-25T09:39:45.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:45.918+0000 7f5a67c5e640 1 -- 172.21.15.70:0/3292765984 wait complete. 2023-08-25T09:39:45.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:46.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.150+0000 7fd50b21f640 1 Processor -- start 2023-08-25T09:39:46.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.150+0000 7fd50b21f640 1 -- start start 2023-08-25T09:39:46.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd50b21f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50415ae50 0x7fd5040c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd50b21f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5040cdc20 0x7fd504069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd50b21f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5040cc5f0 con 0x7fd5040ce060 2023-08-25T09:39:46.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd50b21f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd504070650 con 0x7fd5040cdc20 2023-08-25T09:39:46.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd50b21f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd504070930 con 0x7fd50415ae50 2023-08-25T09:39:46.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5040cdc20 0x7fd504069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd5097bc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd5040ce060 0x7fd5040ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36340/0 (socket says 172.21.15.70:36340) 2023-08-25T09:39:46.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd5097bc640 1 -- 172.21.15.70:0/3048774502 learned_addr learned my addr 172.21.15.70:0/3048774502 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:46.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5040cdc20 0x7fd504069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42584/0 (socket says 172.21.15.70:42584) 2023-08-25T09:39:46.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd508fbb640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50415ae50 0x7fd5040c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fbfff640 1 -- 172.21.15.70:0/3048774502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50415ae50 msgr2=0x7fd5040c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:46.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fbfff640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50415ae50 0x7fd5040c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fbfff640 1 -- 172.21.15.70:0/3048774502 >> v1:172.21.15.70:6789/0 conn(0x7fd5040ce060 legacy=0x7fd5040ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fbfff640 1 -- 172.21.15.70:0/3048774502 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd504070c10 con 0x7fd5040cdc20 2023-08-25T09:39:46.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd4fb7fe640 1 -- 172.21.15.70:0/3048774502 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308499504 0 0) 0x7fd5040cc5f0 con 0x7fd5040ce060 2023-08-25T09:39:46.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.152+0000 7fd508fbb640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50415ae50 0x7fd5040c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:46.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd4fbfff640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5040cdc20 0x7fd504069850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40097a0 tx=0x7fd4f402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a0bb00ae2a0199 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:46.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd4fb7fe640 1 -- 172.21.15.70:0/3048774502 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4030800 con 0x7fd5040cdc20 2023-08-25T09:39:46.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd4fb7fe640 1 -- 172.21.15.70:0/3048774502 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4030960 con 0x7fd5040cdc20 2023-08-25T09:39:46.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd4fb7fe640 1 -- 172.21.15.70:0/3048774502 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4030cf0 con 0x7fd5040cdc20 2023-08-25T09:39:46.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd50b21f640 1 -- 172.21.15.70:0/3048774502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5040cdc20 msgr2=0x7fd504069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd50b21f640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5040cdc20 0x7fd504069850 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40097a0 tx=0x7fd4f402b940 comp rx=0 tx=0).stop 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd50b21f640 1 -- 172.21.15.70:0/3048774502 shutdown_connections 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd50b21f640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5040cdc20 0x7fd504069850 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd50b21f640 1 --2- 172.21.15.70:0/3048774502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50415ae50 0x7fd5040c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.154+0000 7fd50b21f640 1 -- 172.21.15.70:0/3048774502 >> 172.21.15.70:0/3048774502 conn(0x7fd5040cb0d0 msgr2=0x7fd5040cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.155+0000 7fd50b21f640 1 -- 172.21.15.70:0/3048774502 shutdown_connections 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.155+0000 7fd50b21f640 1 -- 172.21.15.70:0/3048774502 wait complete. 2023-08-25T09:39:46.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.156+0000 7fd50b21f640 1 Processor -- start 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.156+0000 7fd50b21f640 1 -- start start 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.157+0000 7fd50b21f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5040cdc20 0x7fd5041f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.157+0000 7fd50b21f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5040ce060 0x7fd5040adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.157+0000 7fd50b21f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 0x7fd5040ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.157+0000 7fd50b21f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040b2f00 con 0x7fd50415ae50 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.157+0000 7fd50b21f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040b2330 con 0x7fd5040ce060 2023-08-25T09:39:46.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.157+0000 7fd50b21f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5040b25d0 con 0x7fd5040cdc20 2023-08-25T09:39:46.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 0x7fd5040ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 0x7fd5040ae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42594/0 (socket says 172.21.15.70:42594) 2023-08-25T09:39:46.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 -- 172.21.15.70:0/1180326045 learned_addr learned my addr 172.21.15.70:0/1180326045 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:46.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd4fbfff640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5040ce060 0x7fd5040adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd508fbb640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5040cdc20 0x7fd5041f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 -- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5040cdc20 msgr2=0x7fd5041f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5040cdc20 0x7fd5041f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 -- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5040ce060 msgr2=0x7fd5040adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5040ce060 0x7fd5040adf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 -- 172.21.15.70:0/1180326045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4009400 con 0x7fd50415ae50 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.158+0000 7fd5097bc640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 0x7fd5040ae450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c430 tx=0x7fd4fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.159+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc0074e0 con 0x7fd50415ae50 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.159+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc007660 con 0x7fd50415ae50 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.159+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc004a20 con 0x7fd50415ae50 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.159+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5040b28b0 con 0x7fd50415ae50 2023-08-25T09:39:46.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.159+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5040b3390 con 0x7fd50415ae50 2023-08-25T09:39:46.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.160+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5040b3070 con 0x7fd50415ae50 2023-08-25T09:39:46.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.161+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4fc004b80 con 0x7fd50415ae50 2023-08-25T09:39:46.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.162+0000 7fd4f97fa640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4f006a080 0x7fd4f006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.162+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4fc088300 con 0x7fd50415ae50 2023-08-25T09:39:46.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.162+0000 7fd508fbb640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4f006a080 0x7fd4f006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.163+0000 7fd508fbb640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4f006a080 0x7fd4f006c540 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd500007510 tx=0x7fd5000074a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:46.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.166+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd4fc053b40 con 0x7fd50415ae50 2023-08-25T09:39:46.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.353+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd504066fb0 con 0x7fd50415ae50 2023-08-25T09:39:46.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.354+0000 7fd4f97fa640 1 -- 172.21.15.70:0/1180326045 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4fc057160 con 0x7fd50415ae50 2023-08-25T09:39:46.355 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:46.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4f006a080 msgr2=0x7fd4f006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4f006a080 0x7fd4f006c540 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd500007510 tx=0x7fd5000074a0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 msgr2=0x7fd5040ae450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 0x7fd5040ae450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00c430 tx=0x7fd4fc00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 shutdown_connections 2023-08-25T09:39:46.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4f006a080 0x7fd4f006c540 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd50415ae50 0x7fd5040ae450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd5040ce060 0x7fd5040adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 --2- 172.21.15.70:0/1180326045 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5040cdc20 0x7fd5041f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 >> 172.21.15.70:0/1180326045 conn(0x7fd5040cb0d0 msgr2=0x7fd5040af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:46.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 shutdown_connections 2023-08-25T09:39:46.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.356+0000 7fd50b21f640 1 -- 172.21.15.70:0/1180326045 wait complete. 2023-08-25T09:39:46.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:39:46.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.588+0000 7f91d8f1c640 1 Processor -- start 2023-08-25T09:39:46.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.588+0000 7f91d8f1c640 1 -- start start 2023-08-25T09:39:46.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d8f1c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d4073430 0x7f91d40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d8f1c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d4072820 con 0x7f91d40aee40 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d8f1c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d406bdc0 con 0x7f91d415ccb0 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d8f1c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d406c0a0 con 0x7f91d4073430 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d4073430 0x7f91d40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d4073430 0x7f91d40ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42598/0 (socket says 172.21.15.70:42598) 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91d2d9d640 1 -- 172.21.15.70:0/3854165187 learned_addr learned my addr 172.21.15.70:0/3854165187 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.590+0000 7f91cbfff640 1 --1- 172.21.15.70:0/3854165187 >> v1:172.21.15.70:6789/0 conn(0x7f91d40aee40 0x7f91d4072d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36356/0 (socket says 172.21.15.70:36356) 2023-08-25T09:39:46.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.591+0000 7f91d2d9d640 1 -- 172.21.15.70:0/3854165187 >> v1:172.21.15.155:6789/0 conn(0x7f91d415ccb0 legacy=0x7f91d415d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:46.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.591+0000 7f91d2d9d640 1 -- 172.21.15.70:0/3854165187 >> v1:172.21.15.70:6789/0 conn(0x7f91d40aee40 legacy=0x7f91d4072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.591+0000 7f91d2d9d640 1 -- 172.21.15.70:0/3854165187 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d406c380 con 0x7f91d4073430 2023-08-25T09:39:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.592+0000 7f91d2d9d640 1 --2- 172.21.15.70:0/3854165187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d4073430 0x7f91d40ce7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009e50 tx=0x7f91cc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375521270e934654 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.592+0000 7f91d1d9b640 1 -- 172.21.15.70:0/3854165187 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc02cb10 con 0x7f91d4073430 2023-08-25T09:39:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.592+0000 7f91d1d9b640 1 -- 172.21.15.70:0/3854165187 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc02cc70 con 0x7f91d4073430 2023-08-25T09:39:46.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.592+0000 7f91d1d9b640 1 -- 172.21.15.70:0/3854165187 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0343f0 con 0x7f91d4073430 2023-08-25T09:39:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 -- 172.21.15.70:0/3854165187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d4073430 msgr2=0x7f91d40ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/3854165187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d4073430 0x7f91d40ce7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009e50 tx=0x7f91cc02bf70 comp rx=0 tx=0).stop 2023-08-25T09:39:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 -- 172.21.15.70:0/3854165187 shutdown_connections 2023-08-25T09:39:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/3854165187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d4073430 0x7f91d40ce7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 -- 172.21.15.70:0/3854165187 >> 172.21.15.70:0/3854165187 conn(0x7f91d40b12c0 msgr2=0x7f91d40b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 -- 172.21.15.70:0/3854165187 shutdown_connections 2023-08-25T09:39:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.593+0000 7f91d8f1c640 1 -- 172.21.15.70:0/3854165187 wait complete. 2023-08-25T09:39:46.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.595+0000 7f91d8f1c640 1 Processor -- start 2023-08-25T09:39:46.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.595+0000 7f91d8f1c640 1 -- start start 2023-08-25T09:39:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.596+0000 7f91d8f1c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d4073430 0x7f91d40c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.596+0000 7f91d8f1c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 0x7f91d40c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d8f1c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40c9570 0x7f91d4147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d8f1c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d414a4a0 con 0x7f91d415ccb0 2023-08-25T09:39:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d8f1c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40c8550 con 0x7f91d4073430 2023-08-25T09:39:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d8f1c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40c8830 con 0x7f91d40c9570 2023-08-25T09:39:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 0x7f91d40c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 0x7f91d40c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42604/0 (socket says 172.21.15.70:42604) 2023-08-25T09:39:46.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 -- 172.21.15.70:0/1345996017 learned_addr learned my addr 172.21.15.70:0/1345996017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d259c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d4073430 0x7f91d40c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 -- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40c9570 msgr2=0x7f91d4147630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d2d9d640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40c9570 0x7f91d4147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40c9570 0x7f91d4147630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 -- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d4073430 msgr2=0x7f91d40c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d4073430 0x7f91d40c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91cbfff640 1 -- 172.21.15.70:0/1345996017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc009b30 con 0x7f91d415ccb0 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d259c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d4073430 0x7f91d40c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.597+0000 7f91d2d9d640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40c9570 0x7f91d4147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:46.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.598+0000 7f91cbfff640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 0x7f91d40c9030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c4d0 tx=0x7f91bc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.598+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc007480 con 0x7f91d415ccb0 2023-08-25T09:39:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.598+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc0075e0 con 0x7f91d415ccb0 2023-08-25T09:39:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.598+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0049e0 con 0x7f91d415ccb0 2023-08-25T09:39:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.598+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40c8b10 con 0x7f91d415ccb0 2023-08-25T09:39:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.598+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d40c6030 con 0x7f91d415ccb0 2023-08-25T09:39:46.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.599+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d40c8d00 con 0x7f91d415ccb0 2023-08-25T09:39:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.600+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f91bc004b40 con 0x7f91d415ccb0 2023-08-25T09:39:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.601+0000 7f91caffd640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91b406a050 0x7f91b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.601+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91bc087640 con 0x7f91d415ccb0 2023-08-25T09:39:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.601+0000 7f91d259c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91b406a050 0x7f91b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:46.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.602+0000 7f91d259c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91b406a050 0x7f91b406c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91c400a830 tx=0x7f91c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:46.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.605+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91bc052dd0 con 0x7f91d415ccb0 2023-08-25T09:39:46.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.807+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f91d404e420 con 0x7f91d415ccb0 2023-08-25T09:39:46.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.808+0000 7f91caffd640 1 -- 172.21.15.70:0/1345996017 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f91bc0563f0 con 0x7f91d415ccb0 2023-08-25T09:39:46.809 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:39:46.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91b406a050 msgr2=0x7f91b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91b406a050 0x7f91b406c510 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91c400a830 tx=0x7f91c4008040 comp rx=0 tx=0).stop 2023-08-25T09:39:46.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 msgr2=0x7f91d40c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:46.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 0x7f91d40c9030 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c4d0 tx=0x7f91bc00c990 comp rx=0 tx=0).stop 2023-08-25T09:39:46.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 shutdown_connections 2023-08-25T09:39:46.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91b406a050 0x7f91b406c510 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d40c9570 0x7f91d4147630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d415ccb0 0x7f91d40c9030 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.810+0000 7f91d8f1c640 1 --2- 172.21.15.70:0/1345996017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d4073430 0x7f91d40c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:46.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.811+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 >> 172.21.15.70:0/1345996017 conn(0x7f91d40b12c0 msgr2=0x7f91d40b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:46.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.811+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 shutdown_connections 2023-08-25T09:39:46.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:46.811+0000 7f91d8f1c640 1 -- 172.21.15.70:0/1345996017 wait complete. 2023-08-25T09:39:46.829 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:39:46.830 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:46.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:47.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.098+0000 7f74a67c2640 1 Processor -- start 2023-08-25T09:39:47.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.099+0000 7f74a67c2640 1 -- start start 2023-08-25T09:39:47.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.100+0000 7f74a67c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 0x7f74a006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.100+0000 7f74a67c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a00b1b60 con 0x7f74a006b2b0 2023-08-25T09:39:47.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.100+0000 7f74a67c2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a006b7c0 con 0x7f74a015ec10 2023-08-25T09:39:47.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.100+0000 7f74a67c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a006bd80 con 0x7f74a015e830 2023-08-25T09:39:47.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f7497fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f74a006b2b0 0x7f74a015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36372/0 (socket says 172.21.15.70:36372) 2023-08-25T09:39:47.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f7497fff640 1 -- 172.21.15.70:0/622207006 learned_addr learned my addr 172.21.15.70:0/622207006 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749ffff640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 0x7f74a006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475233521 0 0) 0x7f74a00b1b60 con 0x7f74a006b2b0 2023-08-25T09:39:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f748c003520 con 0x7f74a006b2b0 2023-08-25T09:39:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202846649 0 0) 0x7f748c003520 con 0x7f74a006b2b0 2023-08-25T09:39:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 msgr2=0x7f74a006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 0x7f74a006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 >> v1:172.21.15.155:6789/0 conn(0x7f74a015ec10 legacy=0x7f74a00b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a006baa0 con 0x7f74a006b2b0 2023-08-25T09:39:47.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7488001670 con 0x7f74a006b2b0 2023-08-25T09:39:47.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 >> v1:172.21.15.70:6789/0 conn(0x7f74a006b2b0 legacy=0x7f74a015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.101+0000 7f749f7fe640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0039a0 0x7f748c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749f7fe640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0047c0 0x7f748c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749f7fe640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 0x7f748c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a006bd80 con 0x7f748c0039a0 2023-08-25T09:39:47.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c00ba30 con 0x7f748c0047c0 2023-08-25T09:39:47.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c00bbf0 con 0x7f74a015e830 2023-08-25T09:39:47.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749ffff640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0039a0 0x7f748c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749ffff640 1 -- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 msgr2=0x7f748c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749ffff640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a015e830 0x7f748c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749ffff640 1 -- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0047c0 msgr2=0x7f748c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749ffff640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f748c0047c0 0x7f748c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.102+0000 7f749ffff640 1 -- 172.21.15.70:0/622207006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c003520 con 0x7f748c0039a0 2023-08-25T09:39:47.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.103+0000 7f749ffff640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0039a0 0x7f748c003dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7490009a90 tx=0x7f748c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cfd56a7c4bfa05 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:47.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.103+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749002c6b0 con 0x7f748c0039a0 2023-08-25T09:39:47.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749002c810 con 0x7f748c0039a0 2023-08-25T09:39:47.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f749f7fe640 1 -- 172.21.15.70:0/622207006 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749002cba0 con 0x7f748c0039a0 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a67c2640 1 -- 172.21.15.70:0/622207006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0039a0 msgr2=0x7f748c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a67c2640 1 --2- 172.21.15.70:0/622207006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f748c0039a0 0x7f748c003dc0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7490009a90 tx=0x7f748c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a4d5f640 1 -- 172.21.15.70:0/622207006 reap_dead start 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a67c2640 1 -- 172.21.15.70:0/622207006 shutdown_connections 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a67c2640 1 -- 172.21.15.70:0/622207006 >> 172.21.15.70:0/622207006 conn(0x7f74a0069850 msgr2=0x7f74a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a67c2640 1 -- 172.21.15.70:0/622207006 shutdown_connections 2023-08-25T09:39:47.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.104+0000 7f74a67c2640 1 -- 172.21.15.70:0/622207006 wait complete. 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.106+0000 7f74a67c2640 1 Processor -- start 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.106+0000 7f74a67c2640 1 -- start start 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.107+0000 7f74a67c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0039a0 0x7f74a014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.107+0000 7f74a67c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a01e6e00 0x7f74a01e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.107+0000 7f74a67c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 0x7f74a0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.107+0000 7f74a67c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a0159f90 con 0x7f74a01595d0 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.107+0000 7f74a67c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a01e78e0 con 0x7f74a01e6e00 2023-08-25T09:39:47.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.107+0000 7f74a67c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a01e7bc0 con 0x7f748c0039a0 2023-08-25T09:39:47.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 0x7f74a0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 0x7f74a0159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42614/0 (socket says 172.21.15.70:42614) 2023-08-25T09:39:47.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 -- 172.21.15.70:0/3665765418 learned_addr learned my addr 172.21.15.70:0/3665765418 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:47.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f749ffff640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0039a0 0x7f74a014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f7497fff640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a01e6e00 0x7f74a01e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 -- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0039a0 msgr2=0x7f74a014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0039a0 0x7f74a014f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 -- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a01e6e00 msgr2=0x7f74a01e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a01e6e00 0x7f74a01e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 -- 172.21.15.70:0/3665765418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a015ca20 con 0x7f74a01595d0 2023-08-25T09:39:47.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f7497fff640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a01e6e00 0x7f74a01e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:47.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.108+0000 7f74a4d5f640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 0x7f74a0159a50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74a006a6e0 tx=0x7f749800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:47.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.109+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7498005d40 con 0x7f74a01595d0 2023-08-25T09:39:47.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.109+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800a610 con 0x7f74a01595d0 2023-08-25T09:39:47.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.109+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a01e7ea0 con 0x7f74a01595d0 2023-08-25T09:39:47.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.109+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800a9c0 con 0x7f74a01595d0 2023-08-25T09:39:47.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.109+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a014f7d0 con 0x7f74a01595d0 2023-08-25T09:39:47.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.110+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a004e420 con 0x7f74a01595d0 2023-08-25T09:39:47.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.111+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f749800ab20 con 0x7f74a01595d0 2023-08-25T09:39:47.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.113+0000 7f749d7fa640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f748006a080 0x7f748006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.113+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7498087ab0 con 0x7f74a01595d0 2023-08-25T09:39:47.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.113+0000 7f749ffff640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f748006a080 0x7f748006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.114+0000 7f749ffff640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f748006a080 0x7f748006c540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7490036000 tx=0x7f7490030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:47.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.117+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7498053230 con 0x7f74a01595d0 2023-08-25T09:39:47.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.308+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74a006c620 con 0x7f74a01595d0 2023-08-25T09:39:47.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.311+0000 7f749d7fa640 1 -- 172.21.15.70:0/3665765418 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f7498017090 con 0x7f74a01595d0 2023-08-25T09:39:47.313 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:47.314 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:47.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f748006a080 msgr2=0x7f748006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f748006a080 0x7f748006c540 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7490036000 tx=0x7f7490030000 comp rx=0 tx=0).stop 2023-08-25T09:39:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 msgr2=0x7f74a0159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 0x7f74a0159a50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f74a006a6e0 tx=0x7f749800cb90 comp rx=0 tx=0).stop 2023-08-25T09:39:47.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 shutdown_connections 2023-08-25T09:39:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f748006a080 0x7f748006c540 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a01595d0 0x7f74a0159a50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a01e6e00 0x7f74a01e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 --2- 172.21.15.70:0/3665765418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f748c0039a0 0x7f74a014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.313+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 >> 172.21.15.70:0/3665765418 conn(0x7f74a0069850 msgr2=0x7f74a015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.314+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 shutdown_connections 2023-08-25T09:39:47.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.314+0000 7f74a67c2640 1 -- 172.21.15.70:0/3665765418 wait complete. 2023-08-25T09:39:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:47.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:47.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.550+0000 7f3c3d257640 1 Processor -- start 2023-08-25T09:39:47.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.551+0000 7f3c3d257640 1 -- start start 2023-08-25T09:39:47.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3d257640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3c380cd850 0x7f3c380cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3d257640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c3815cfe0 0x7f3c380d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3d257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380b8510 0x7f3c380b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3d257640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c380ce170 con 0x7f3c3815cfe0 2023-08-25T09:39:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3d257640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c380d1500 con 0x7f3c380cd850 2023-08-25T09:39:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3d257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380d17e0 con 0x7f3c380b8510 2023-08-25T09:39:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c3815cfe0 0x7f3c380d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3c3815cfe0 0x7f3c380d0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42616/0 (socket says 172.21.15.70:42616) 2023-08-25T09:39:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 -- 172.21.15.70:0/1430295059 learned_addr learned my addr 172.21.15.70:0/1430295059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:47.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 -- 172.21.15.70:0/1430295059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380b8510 msgr2=0x7f3c380b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 --2- 172.21.15.70:0/1430295059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380b8510 0x7f3c380b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 -- 172.21.15.70:0/1430295059 >> v2:172.21.15.155:3300/0 conn(0x7f3c380cd850 msgr2=0x7f3c380cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.552+0000 7f3c3659c640 1 --2- 172.21.15.70:0/1430295059 >> v2:172.21.15.155:3300/0 conn(0x7f3c380cd850 0x7f3c380cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.553+0000 7f3c3659c640 1 -- 172.21.15.70:0/1430295059 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c380d1ac0 con 0x7f3c3815cfe0 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.554+0000 7f3c3659c640 1 --2- 172.21.15.70:0/1430295059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c380d0c00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c380b5690 tx=0x7f3c2002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae22f48c39411447 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.554+0000 7f3c35d9b640 1 -- 172.21.15.70:0/1430295059 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20031880 con 0x7f3c3815cfe0 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.554+0000 7f3c35d9b640 1 -- 172.21.15.70:0/1430295059 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c200319e0 con 0x7f3c3815cfe0 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.554+0000 7f3c35d9b640 1 -- 172.21.15.70:0/1430295059 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20031d70 con 0x7f3c3815cfe0 2023-08-25T09:39:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.554+0000 7f3c3d257640 1 -- 172.21.15.70:0/1430295059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 msgr2=0x7f3c380d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.554+0000 7f3c3d257640 1 --2- 172.21.15.70:0/1430295059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c380d0c00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3c380b5690 tx=0x7f3c2002b920 comp rx=0 tx=0).stop 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 -- 172.21.15.70:0/1430295059 shutdown_connections 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 --2- 172.21.15.70:0/1430295059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380b8510 0x7f3c380b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 --2- 172.21.15.70:0/1430295059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c380d0c00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 --2- 172.21.15.70:0/1430295059 >> v2:172.21.15.155:3300/0 conn(0x7f3c380cd850 0x7f3c380cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 -- 172.21.15.70:0/1430295059 >> 172.21.15.70:0/1430295059 conn(0x7f3c38069850 msgr2=0x7f3c38069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:47.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 -- 172.21.15.70:0/1430295059 shutdown_connections 2023-08-25T09:39:47.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.555+0000 7f3c3d257640 1 -- 172.21.15.70:0/1430295059 wait complete. 2023-08-25T09:39:47.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.556+0000 7f3c3d257640 1 Processor -- start 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.556+0000 7f3c3d257640 1 -- start start 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.557+0000 7f3c3d257640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c380b8510 0x7f3c381f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.557+0000 7f3c3d257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380cd850 0x7f3c380ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.557+0000 7f3c3d257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c381f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.557+0000 7f3c3d257640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c381f30c0 con 0x7f3c3815cfe0 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.557+0000 7f3c3d257640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380b3740 con 0x7f3c380b8510 2023-08-25T09:39:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.557+0000 7f3c3d257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c380b3a20 con 0x7f3c380cd850 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c381f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c381f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42630/0 (socket says 172.21.15.70:42630) 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 -- 172.21.15.70:0/152690737 learned_addr learned my addr 172.21.15.70:0/152690737 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 -- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380cd850 msgr2=0x7f3c380ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c36d9d640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c380b8510 0x7f3c381f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380cd850 0x7f3c380ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 -- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c380b8510 msgr2=0x7f3c381f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c380b8510 0x7f3c381f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 -- 172.21.15.70:0/152690737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200094d0 con 0x7f3c3815cfe0 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.558+0000 7f3c3759e640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c381f4d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000b410 tx=0x7f3c3000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.559+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30005a10 con 0x7f3c3815cfe0 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.559+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c30005b70 con 0x7f3c3815cfe0 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.559+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30011630 con 0x7f3c3815cfe0 2023-08-25T09:39:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.559+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c380b3d00 con 0x7f3c3815cfe0 2023-08-25T09:39:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.559+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c380ca850 con 0x7f3c3815cfe0 2023-08-25T09:39:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.560+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c380b3ef0 con 0x7f3c3815cfe0 2023-08-25T09:39:47.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.561+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3c30011790 con 0x7f3c3815cfe0 2023-08-25T09:39:47.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.562+0000 7f3c2f7fe640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c1806a050 0x7f3c1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.562+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c30088140 con 0x7f3c3815cfe0 2023-08-25T09:39:47.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.562+0000 7f3c36d9d640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c1806a050 0x7f3c1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.563+0000 7f3c36d9d640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c1806a050 0x7f3c1806c510 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c380c8ea0 tx=0x7f3c28007310 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:47.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.567+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3c30053820 con 0x7f3c3815cfe0 2023-08-25T09:39:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.758+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c380ca850 con 0x7f3c3815cfe0 2023-08-25T09:39:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.759+0000 7f3c2f7fe640 1 -- 172.21.15.70:0/152690737 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c380ca850 con 0x7f3c3815cfe0 2023-08-25T09:39:47.761 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:47.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c1806a050 msgr2=0x7f3c1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c1806a050 0x7f3c1806c510 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c380c8ea0 tx=0x7f3c28007310 comp rx=0 tx=0).stop 2023-08-25T09:39:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 msgr2=0x7f3c381f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c381f4d10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000b410 tx=0x7f3c3000b8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 shutdown_connections 2023-08-25T09:39:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c1806a050 0x7f3c1806c510 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c3815cfe0 0x7f3c381f4d10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c380cd850 0x7f3c380ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 --2- 172.21.15.70:0/152690737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c380b8510 0x7f3c381f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.762+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 >> 172.21.15.70:0/152690737 conn(0x7f3c38069850 msgr2=0x7f3c380c7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:47.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.763+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 shutdown_connections 2023-08-25T09:39:47.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.763+0000 7f3c3d257640 1 -- 172.21.15.70:0/152690737 wait complete. 2023-08-25T09:39:47.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.988+0000 7f82f7fff640 1 Processor -- start 2023-08-25T09:39:47.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.988+0000 7f82f7fff640 1 -- start start 2023-08-25T09:39:47.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.989+0000 7f82f7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f82f815a920 0x7f82f80c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.989+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 0x7f82f80ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.989+0000 7f82f7fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f80ad8f0 con 0x7f82f80b0320 2023-08-25T09:39:47.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.989+0000 7f82f7fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f80adbb0 con 0x7f82f815a920 2023-08-25T09:39:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.989+0000 7f82f7fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80c90a0 con 0x7f82f80b1ce0 2023-08-25T09:39:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.990+0000 7f82f6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f82f80b0320 0x7f82f80b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36382/0 (socket says 172.21.15.70:36382) 2023-08-25T09:39:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.990+0000 7f82f6ffd640 1 -- 172.21.15.70:0/4109747429 learned_addr learned my addr 172.21.15.70:0/4109747429 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.990+0000 7f82effff640 1 --2- 172.21.15.70:0/4109747429 >> v2:172.21.15.155:3300/0 conn(0x7f82f815a920 0x7f82f80c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.990+0000 7f82f77fe640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 0x7f82f80ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.990+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425905753 0 0) 0x7f82f80ad8f0 con 0x7f82f80b0320 2023-08-25T09:39:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.990+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e4003520 con 0x7f82f80b0320 2023-08-25T09:39:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 455742266 0 0) 0x7f82e4003520 con 0x7f82f80b0320 2023-08-25T09:39:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 msgr2=0x7f82f80ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 0x7f82f80ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 >> v2:172.21.15.155:3300/0 conn(0x7f82f815a920 msgr2=0x7f82f80c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 --2- 172.21.15.70:0/4109747429 >> v2:172.21.15.155:3300/0 conn(0x7f82f815a920 0x7f82f80c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f80c9380 con 0x7f82f80b0320 2023-08-25T09:39:47.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f82e80017f0 con 0x7f82f80b0320 2023-08-25T09:39:47.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 >> v1:172.21.15.70:6789/0 conn(0x7f82f80b0320 legacy=0x7f82f80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e4003970 0x7f82e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40047c0 0x7f82e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 0x7f82e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80adbb0 con 0x7f82e4003970 2023-08-25T09:39:47.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f80c90a0 con 0x7f82e40047c0 2023-08-25T09:39:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e400b980 con 0x7f82f80b1ce0 2023-08-25T09:39:47.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e4003970 0x7f82e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82effff640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40047c0 0x7f82e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.991+0000 7f82f77fe640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 0x7f82e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:47.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.992+0000 7f82f6ffd640 1 -- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 msgr2=0x7f82e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.992+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80b1ce0 0x7f82e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:47.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.992+0000 7f82f6ffd640 1 -- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40047c0 msgr2=0x7f82e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.992+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82e40047c0 0x7f82e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.992+0000 7f82f6ffd640 1 -- 172.21.15.70:0/4109747429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4003520 con 0x7f82e4003970 2023-08-25T09:39:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.993+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e4003970 0x7f82e4003d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c240 tx=0x7f82e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4496aeebef12a12c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.993+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e802cb00 con 0x7f82e4003970 2023-08-25T09:39:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.993+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e802cc80 con 0x7f82e4003970 2023-08-25T09:39:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.993+0000 7f82f67fc640 1 -- 172.21.15.70:0/4109747429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8007070 con 0x7f82e4003970 2023-08-25T09:39:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f7fff640 1 -- 172.21.15.70:0/4109747429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e4003970 msgr2=0x7f82e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f7fff640 1 --2- 172.21.15.70:0/4109747429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82e4003970 0x7f82e4003d90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f82e800c240 tx=0x7f82e40057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f77fe640 1 -- 172.21.15.70:0/4109747429 reap_dead start 2023-08-25T09:39:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f7fff640 1 -- 172.21.15.70:0/4109747429 shutdown_connections 2023-08-25T09:39:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f7fff640 1 -- 172.21.15.70:0/4109747429 >> 172.21.15.70:0/4109747429 conn(0x7f82f80ce830 msgr2=0x7f82f80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:48.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f7fff640 1 -- 172.21.15.70:0/4109747429 shutdown_connections 2023-08-25T09:39:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.994+0000 7f82f7fff640 1 -- 172.21.15.70:0/4109747429 wait complete. 2023-08-25T09:39:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.996+0000 7f82f7fff640 1 Processor -- start 2023-08-25T09:39:48.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.996+0000 7f82f7fff640 1 -- start start 2023-08-25T09:39:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e4003970 0x7f82f81f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 0x7f82f8070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f7fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8156050 0x7f82f81564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f7fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f806c1d0 con 0x7f82f806b5f0 2023-08-25T09:39:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f7fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8156a10 con 0x7f82f8156050 2023-08-25T09:39:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f7fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f806c970 con 0x7f82e4003970 2023-08-25T09:39:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 0x7f82f8070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 0x7f82f8070ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42654/0 (socket says 172.21.15.70:42654) 2023-08-25T09:39:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 -- 172.21.15.70:0/1398613008 learned_addr learned my addr 172.21.15.70:0/1398613008 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:48.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f77fe640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8156050 0x7f82f81564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e4003970 0x7f82f81f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 -- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e4003970 msgr2=0x7f82f81f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e4003970 0x7f82f81f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 -- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8156050 msgr2=0x7f82f81564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8156050 0x7f82f81564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82effff640 1 -- 172.21.15.70:0/1398613008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f806c3b0 con 0x7f82f806b5f0 2023-08-25T09:39:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.997+0000 7f82f77fe640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8156050 0x7f82f81564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.998+0000 7f82effff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 0x7f82f8070ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82e0011560 tx=0x7f82e0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.998+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004220 con 0x7f82f806b5f0 2023-08-25T09:39:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.998+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e0004380 con 0x7f82f806b5f0 2023-08-25T09:39:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.998+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0015870 con 0x7f82f806b5f0 2023-08-25T09:39:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.998+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f806c690 con 0x7f82f806b5f0 2023-08-25T09:39:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.998+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f806ba50 con 0x7f82f806b5f0 2023-08-25T09:39:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:47.999+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f8061460 con 0x7f82f806b5f0 2023-08-25T09:39:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.000+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f82e00159d0 con 0x7f82f806b5f0 2023-08-25T09:39:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.001+0000 7f82ef7fe640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.001+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82e0088660 con 0x7f82f806b5f0 2023-08-25T09:39:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.001+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.002+0000 7f82f6ffd640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f82e8034000 tx=0x7f82e8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.005+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f82e0053de0 con 0x7f82f806b5f0 2023-08-25T09:39:48.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.195+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f82f80c5210 con 0x7f82f806b5f0 2023-08-25T09:39:48.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.196+0000 7f82ef7fe640 1 -- 172.21.15.70:0/1398613008 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f82e0057400 con 0x7f82f806b5f0 2023-08-25T09:39:48.198 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:48.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 msgr2=0x7f82d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f82e8034000 tx=0x7f82e8007000 comp rx=0 tx=0).stop 2023-08-25T09:39:48.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 msgr2=0x7f82f8070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 0x7f82f8070ef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f82e0011560 tx=0x7f82e0011a20 comp rx=0 tx=0).stop 2023-08-25T09:39:48.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 shutdown_connections 2023-08-25T09:39:48.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f8156050 0x7f82f81564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f806b5f0 0x7f82f8070ef0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 --2- 172.21.15.70:0/1398613008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82e4003970 0x7f82f81f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.199+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 >> 172.21.15.70:0/1398613008 conn(0x7f82f80ce830 msgr2=0x7f82f8159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:48.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.200+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 shutdown_connections 2023-08-25T09:39:48.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.200+0000 7f82f7fff640 1 -- 172.21.15.70:0/1398613008 wait complete. 2023-08-25T09:39:48.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:48.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.421+0000 7f0e810c2640 1 Processor -- start 2023-08-25T09:39:48.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.421+0000 7f0e810c2640 1 -- start start 2023-08-25T09:39:48.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e810c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c15ccd0 0x7f0e7c0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e810c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e7c0d2400 0x7f0e7c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e810c2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0e7c0d09f0 0x7f0e7c0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e810c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c15d0b0 con 0x7f0e7c0d2400 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e810c2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c0b58d0 con 0x7f0e7c0d09f0 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e810c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d0e80 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e7b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e7c0d2400 0x7f0e7c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e7b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e7c0d2400 0x7f0e7c0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42660/0 (socket says 172.21.15.70:42660) 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.423+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/1418988761 learned_addr learned my addr 172.21.15.70:0/1418988761 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e7bfff640 1 --2- 172.21.15.70:0/1418988761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c15ccd0 0x7f0e7c0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e808c1640 1 --2- 172.21.15.70:0/1418988761 >> v2:172.21.15.155:3300/0 conn(0x7f0e7c0d09f0 0x7f0e7c0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/1418988761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c15ccd0 msgr2=0x7f0e7c0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e7b7fe640 1 --2- 172.21.15.70:0/1418988761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c15ccd0 0x7f0e7c0d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/1418988761 >> v2:172.21.15.155:3300/0 conn(0x7f0e7c0d09f0 msgr2=0x7f0e7c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e7b7fe640 1 --2- 172.21.15.70:0/1418988761 >> v2:172.21.15.155:3300/0 conn(0x7f0e7c0d09f0 0x7f0e7c0cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.424+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/1418988761 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0d1160 con 0x7f0e7c0d2400 2023-08-25T09:39:48.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.425+0000 7f0e7b7fe640 1 --2- 172.21.15.70:0/1418988761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c0d2400 0x7f0e7c0cf2e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0b57b0 tx=0x7f0e7002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cc80c30b04cb87 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.425+0000 7f0e7affd640 1 -- 172.21.15.70:0/1418988761 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70031860 con 0x7f0e7c0d2400 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.425+0000 7f0e7affd640 1 -- 172.21.15.70:0/1418988761 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e700319c0 con 0x7f0e7c0d2400 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.425+0000 7f0e7affd640 1 -- 172.21.15.70:0/1418988761 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e70031d50 con 0x7f0e7c0d2400 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 -- 172.21.15.70:0/1418988761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c0d2400 msgr2=0x7f0e7c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 --2- 172.21.15.70:0/1418988761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c0d2400 0x7f0e7c0cf2e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c0b57b0 tx=0x7f0e7002b900 comp rx=0 tx=0).stop 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 -- 172.21.15.70:0/1418988761 shutdown_connections 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 --2- 172.21.15.70:0/1418988761 >> v2:172.21.15.155:3300/0 conn(0x7f0e7c0d09f0 0x7f0e7c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 --2- 172.21.15.70:0/1418988761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c0d2400 0x7f0e7c0cf2e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 --2- 172.21.15.70:0/1418988761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c15ccd0 0x7f0e7c0d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 -- 172.21.15.70:0/1418988761 >> 172.21.15.70:0/1418988761 conn(0x7f0e7c069850 msgr2=0x7f0e7c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:48.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 -- 172.21.15.70:0/1418988761 shutdown_connections 2023-08-25T09:39:48.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.426+0000 7f0e810c2640 1 -- 172.21.15.70:0/1418988761 wait complete. 2023-08-25T09:39:48.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.428+0000 7f0e810c2640 1 Processor -- start 2023-08-25T09:39:48.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.428+0000 7f0e810c2640 1 -- start start 2023-08-25T09:39:48.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e810c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c0d2400 0x7f0e7c14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e810c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 0x7f0e7c1e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e810c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e7c14f950 0x7f0e7c0b6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e810c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c14f250 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e810c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c151b40 con 0x7f0e7c14f950 2023-08-25T09:39:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e810c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0c9d80 con 0x7f0e7c0d2400 2023-08-25T09:39:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 0x7f0e7c1e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e7b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 0x7f0e7c1e5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42662/0 (socket says 172.21.15.70:42662) 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/4123777267 learned_addr learned my addr 172.21.15.70:0/4123777267 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e7bfff640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c0d2400 0x7f0e7c14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.429+0000 7f0e808c1640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e7c14f950 0x7f0e7c0b6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c0d2400 msgr2=0x7f0e7c14ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e7b7fe640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c0d2400 0x7f0e7c14ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e7c14f950 msgr2=0x7f0e7c0b6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e7b7fe640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e7c14f950 0x7f0e7c0b6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e7b7fe640 1 -- 172.21.15.70:0/4123777267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e700094d0 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e808c1640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e7c14f950 0x7f0e7c0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:48.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e7b7fe640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 0x7f0e7c1e5dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e7002b940 tx=0x7f0e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7003ad30 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e700443f0 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c1514f0 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.430+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e700447a0 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.431+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c1e6310 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.432+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c061460 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.433+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0e7004d450 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.434+0000 7f0e78ff9640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e6806a050 0x7f0e6806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.434+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e700aab10 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.434+0000 7f0e7bfff640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e6806a050 0x7f0e6806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.435+0000 7f0e7bfff640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e6806a050 0x7f0e6806c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c006fd0 tx=0x7f0e6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.438+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0e70076290 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.629+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e7c0b51b0 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.630+0000 7f0e78ff9640 1 -- 172.21.15.70:0/4123777267 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e70035020 con 0x7f0e7c15ccd0 2023-08-25T09:39:48.632 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:48.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.632+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e6806a050 msgr2=0x7f0e6806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e6806a050 0x7f0e6806c510 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c006fd0 tx=0x7f0e6c008040 comp rx=0 tx=0).stop 2023-08-25T09:39:48.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 msgr2=0x7f0e7c1e5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 0x7f0e7c1e5dd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e7002b940 tx=0x7f0e700046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 shutdown_connections 2023-08-25T09:39:48.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e6806a050 0x7f0e6806c510 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e7c14f950 0x7f0e7c0b6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e7c15ccd0 0x7f0e7c1e5dd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 --2- 172.21.15.70:0/4123777267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e7c0d2400 0x7f0e7c14ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 >> 172.21.15.70:0/4123777267 conn(0x7f0e7c069850 msgr2=0x7f0e7c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:48.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 shutdown_connections 2023-08-25T09:39:48.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.633+0000 7f0e810c2640 1 -- 172.21.15.70:0/4123777267 wait complete. 2023-08-25T09:39:48.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:48.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.866+0000 7f58de683640 1 Processor -- start 2023-08-25T09:39:48.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.866+0000 7f58de683640 1 -- start start 2023-08-25T09:39:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.868+0000 7f58de683640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58d8069850 0x7f58d815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.868+0000 7f58de683640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d8069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.868+0000 7f58de683640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80ac880 con 0x7f58d8160be0 2023-08-25T09:39:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.868+0000 7f58de683640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d80ad400 con 0x7f58d8069850 2023-08-25T09:39:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.868+0000 7f58de683640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80ad890 con 0x7f58d80c9620 2023-08-25T09:39:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58d8069850 0x7f58d815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58d8069850 0x7f58d815e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:42824/0 (socket says 172.21.15.70:42824) 2023-08-25T09:39:48.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d77fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d8069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 -- 172.21.15.70:0/1196638207 learned_addr learned my addr 172.21.15.70:0/1196638207 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58dcc20640 1 --1- 172.21.15.70:0/1196638207 >> v1:172.21.15.155:6789/0 conn(0x7f58d8160be0 0x7f58d8161040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:56376/0 (socket says 172.21.15.70:56376) 2023-08-25T09:39:48.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 -- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 msgr2=0x7f58d8069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 --2- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d8069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 -- 172.21.15.70:0/1196638207 >> v1:172.21.15.155:6789/0 conn(0x7f58d8160be0 legacy=0x7f58d8161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.869+0000 7f58d7fff640 1 -- 172.21.15.70:0/1196638207 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80acad0 con 0x7f58d8069850 2023-08-25T09:39:48.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.871+0000 7f58d7fff640 1 --2- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d815e210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009870 tx=0x7f58c802ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6824a6c25bef2d25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.872+0000 7f58d6ffd640 1 -- 172.21.15.70:0/1196638207 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c802f830 con 0x7f58d8069850 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.872+0000 7f58d6ffd640 1 -- 172.21.15.70:0/1196638207 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c802f990 con 0x7f58d8069850 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.872+0000 7f58d6ffd640 1 -- 172.21.15.70:0/1196638207 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c802fd20 con 0x7f58d8069850 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.872+0000 7f58de683640 1 -- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 msgr2=0x7f58d815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.872+0000 7f58de683640 1 --2- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d815e210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f58c8009870 tx=0x7f58c802ba10 comp rx=0 tx=0).stop 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.873+0000 7f58de683640 1 -- 172.21.15.70:0/1196638207 shutdown_connections 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.873+0000 7f58de683640 1 --2- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d8069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.873+0000 7f58de683640 1 --2- 172.21.15.70:0/1196638207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d815e210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.873+0000 7f58de683640 1 -- 172.21.15.70:0/1196638207 >> 172.21.15.70:0/1196638207 conn(0x7f58d80cb0d0 msgr2=0x7f58d80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:48.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.873+0000 7f58de683640 1 -- 172.21.15.70:0/1196638207 shutdown_connections 2023-08-25T09:39:48.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.873+0000 7f58de683640 1 -- 172.21.15.70:0/1196638207 wait complete. 2023-08-25T09:39:48.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.875+0000 7f58de683640 1 Processor -- start 2023-08-25T09:39:48.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.875+0000 7f58de683640 1 -- start start 2023-08-25T09:39:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58de683640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d81f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58de683640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d80c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58de683640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 0x7f58d815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58de683640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c70b0 con 0x7f58d8160be0 2023-08-25T09:39:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58de683640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80ae910 con 0x7f58d8069850 2023-08-25T09:39:48.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58de683640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c6530 con 0x7f58d80c9620 2023-08-25T09:39:48.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.876+0000 7f58dcc20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 0x7f58d815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 0x7f58d815ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42672/0 (socket says 172.21.15.70:42672) 2023-08-25T09:39:48.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 -- 172.21.15.70:0/419868580 learned_addr learned my addr 172.21.15.70:0/419868580 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:48.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58d7fff640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d81f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58d77fe640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d80c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 -- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 msgr2=0x7f58d80c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d80c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 -- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 msgr2=0x7f58d81f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d81f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 -- 172.21.15.70:0/419868580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c80094d0 con 0x7f58d8160be0 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58d7fff640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d81f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.877+0000 7f58dcc20640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 0x7f58d815ab10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58d000c730 tx=0x7f58d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.878+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000ce40 con 0x7f58d8160be0 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.878+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d000a640 con 0x7f58d8160be0 2023-08-25T09:39:48.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.878+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d80c6810 con 0x7f58d8160be0 2023-08-25T09:39:48.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.878+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000a9f0 con 0x7f58d8160be0 2023-08-25T09:39:48.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.878+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d80c6dd0 con 0x7f58d8160be0 2023-08-25T09:39:48.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.879+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d80aeb10 con 0x7f58d8160be0 2023-08-25T09:39:48.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.880+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f58d000ab50 con 0x7f58d8160be0 2023-08-25T09:39:48.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.881+0000 7f58d4ff9640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58c406a050 0x7f58c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:48.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.882+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58d0087d00 con 0x7f58d8160be0 2023-08-25T09:39:48.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.882+0000 7f58d7fff640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58c406a050 0x7f58c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:48.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.882+0000 7f58d7fff640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58c406a050 0x7f58c406c510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58c8036000 tx=0x7f58c8039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:48.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:48.886+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58d80aeb10 con 0x7f58d8160be0 2023-08-25T09:39:49.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.076+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58d80aead0 con 0x7f58d8160be0 2023-08-25T09:39:49.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.077+0000 7f58d4ff9640 1 -- 172.21.15.70:0/419868580 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58d0056e10 con 0x7f58d8160be0 2023-08-25T09:39:49.079 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:49.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58c406a050 msgr2=0x7f58c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58c406a050 0x7f58c406c510 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f58c8036000 tx=0x7f58c8039040 comp rx=0 tx=0).stop 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 msgr2=0x7f58d815ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 0x7f58d815ab10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f58d000c730 tx=0x7f58d000cbf0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 shutdown_connections 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58c406a050 0x7f58c406c510 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d8160be0 0x7f58d815ab10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c9620 0x7f58d80c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 --2- 172.21.15.70:0/419868580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58d8069850 0x7f58d81f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 >> 172.21.15.70:0/419868580 conn(0x7f58d80cb0d0 msgr2=0x7f58d815fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:49.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 shutdown_connections 2023-08-25T09:39:49.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.080+0000 7f58de683640 1 -- 172.21.15.70:0/419868580 wait complete. 2023-08-25T09:39:49.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:39:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.311+0000 7faf18a14640 1 Processor -- start 2023-08-25T09:39:49.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.311+0000 7faf18a14640 1 -- start start 2023-08-25T09:39:49.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf18a14640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf1415acd0 0x7faf1415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf18a14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf18a14640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf140b3cf0 0x7faf140b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf18a14640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1406c0e0 con 0x7faf1415acd0 2023-08-25T09:39:49.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf18a14640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf140b12b0 con 0x7faf140b3cf0 2023-08-25T09:39:49.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf18a14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf1406ae20 con 0x7faf140ca780 2023-08-25T09:39:49.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf1415acd0 0x7faf1415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf1259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf1415acd0 0x7faf1415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54704/0 (socket says 172.21.15.70:54704) 2023-08-25T09:39:49.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.313+0000 7faf1259c640 1 -- 172.21.15.70:0/2282228907 learned_addr learned my addr 172.21.15.70:0/2282228907 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf12d9d640 1 --2- 172.21.15.70:0/2282228907 >> v2:172.21.15.155:3300/0 conn(0x7faf140b3cf0 0x7faf140b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf11d9b640 1 --2- 172.21.15.70:0/2282228907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf1259c640 1 -- 172.21.15.70:0/2282228907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 msgr2=0x7faf140cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf1259c640 1 --2- 172.21.15.70:0/2282228907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf1259c640 1 -- 172.21.15.70:0/2282228907 >> v2:172.21.15.155:3300/0 conn(0x7faf140b3cf0 msgr2=0x7faf140b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf1259c640 1 --2- 172.21.15.70:0/2282228907 >> v2:172.21.15.155:3300/0 conn(0x7faf140b3cf0 0x7faf140b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.314+0000 7faf1259c640 1 -- 172.21.15.70:0/2282228907 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1406b910 con 0x7faf1415acd0 2023-08-25T09:39:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.315+0000 7faf1259c640 1 --2- 172.21.15.70:0/2282228907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1415b0b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf140b4990 tx=0x7faf0402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79e53bb98a0f6d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf1159a640 1 -- 172.21.15.70:0/2282228907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf040317c0 con 0x7faf1415acd0 2023-08-25T09:39:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf1159a640 1 -- 172.21.15.70:0/2282228907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf04031920 con 0x7faf1415acd0 2023-08-25T09:39:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf1159a640 1 -- 172.21.15.70:0/2282228907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04031cb0 con 0x7faf1415acd0 2023-08-25T09:39:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 -- 172.21.15.70:0/2282228907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 msgr2=0x7faf1415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 --2- 172.21.15.70:0/2282228907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1415b0b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faf140b4990 tx=0x7faf0402b860 comp rx=0 tx=0).stop 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 -- 172.21.15.70:0/2282228907 shutdown_connections 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 --2- 172.21.15.70:0/2282228907 >> v2:172.21.15.155:3300/0 conn(0x7faf140b3cf0 0x7faf140b4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 --2- 172.21.15.70:0/2282228907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 --2- 172.21.15.70:0/2282228907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1415b0b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.316+0000 7faf18a14640 1 -- 172.21.15.70:0/2282228907 >> 172.21.15.70:0/2282228907 conn(0x7faf140ce760 msgr2=0x7faf140ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.317+0000 7faf18a14640 1 -- 172.21.15.70:0/2282228907 shutdown_connections 2023-08-25T09:39:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.317+0000 7faf18a14640 1 -- 172.21.15.70:0/2282228907 wait complete. 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.318+0000 7faf18a14640 1 Processor -- start 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.318+0000 7faf18a14640 1 -- start start 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.319+0000 7faf18a14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf140b3cf0 0x7faf1413c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.319+0000 7faf18a14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.319+0000 7faf18a14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.319+0000 7faf18a14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1413cbb0 con 0x7faf1415acd0 2023-08-25T09:39:49.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.319+0000 7faf18a14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf140cd660 con 0x7faf140b3cf0 2023-08-25T09:39:49.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.319+0000 7faf18a14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf140cd940 con 0x7faf140ca780 2023-08-25T09:39:49.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1413d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54718/0 (socket says 172.21.15.70:54718) 2023-08-25T09:39:49.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 -- 172.21.15.70:0/659956669 learned_addr learned my addr 172.21.15.70:0/659956669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:49.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf1259c640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf140b3cf0 0x7faf1413c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf11d9b640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 -- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 msgr2=0x7faf140ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 -- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf140b3cf0 msgr2=0x7faf1413c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf140b3cf0 0x7faf1413c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 -- 172.21.15.70:0/659956669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009470 con 0x7faf1415acd0 2023-08-25T09:39:49.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.320+0000 7faf12d9d640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1413d0b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faf0c007d20 tx=0x7faf0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.321+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c014420 con 0x7faf1415acd0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.321+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c0145a0 con 0x7faf1415acd0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.321+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c014950 con 0x7faf1415acd0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.321+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf140cdc20 con 0x7faf1415acd0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.321+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1413c810 con 0x7faf1415acd0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.322+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1404e420 con 0x7faf1415acd0 2023-08-25T09:39:49.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.323+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf0c014ab0 con 0x7faf1415acd0 2023-08-25T09:39:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.324+0000 7faf02ffd640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faef406a050 0x7faef406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:49.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.324+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf0c088db0 con 0x7faf1415acd0 2023-08-25T09:39:49.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.324+0000 7faf1259c640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faef406a050 0x7faef406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:49.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.325+0000 7faf1259c640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faef406a050 0x7faef406c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf04036000 tx=0x7faf0403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:49.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.328+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf0c054530 con 0x7faf1415acd0 2023-08-25T09:39:49.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.533+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faf1406bec0 con 0x7faf1415acd0 2023-08-25T09:39:49.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.534+0000 7faf02ffd640 1 -- 172.21.15.70:0/659956669 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7faf0c057b50 con 0x7faf1415acd0 2023-08-25T09:39:49.535 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:39:49.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faef406a050 msgr2=0x7faef406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faef406a050 0x7faef406c510 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7faf04036000 tx=0x7faf0403d040 comp rx=0 tx=0).stop 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 msgr2=0x7faf1413d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1413d0b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faf0c007d20 tx=0x7faf0c00e4b0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 shutdown_connections 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faef406a050 0x7faef406c510 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf1415acd0 0x7faf1413d0b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf140ca780 0x7faf140ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 --2- 172.21.15.70:0/659956669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf140b3cf0 0x7faf1413c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 >> 172.21.15.70:0/659956669 conn(0x7faf140ce760 msgr2=0x7faf140b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 shutdown_connections 2023-08-25T09:39:49.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:49.537+0000 7faf18a14640 1 -- 172.21.15.70:0/659956669 wait complete. 2023-08-25T09:39:49.558 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:39:49.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:49.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:39:49.653 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:39:49.653 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:39:49.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:49.654 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:39:49.724 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:39:49.724 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:49.787 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:39:49.788 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:49.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:50.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.066+0000 7f9070a77640 1 Processor -- start 2023-08-25T09:39:50.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.066+0000 7f9070a77640 1 -- start start 2023-08-25T09:39:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f9070a77640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f906c15e830 0x7f906c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f9070a77640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0b1b60 con 0x7f906c15ec10 2023-08-25T09:39:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f9070a77640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c06b7c0 con 0x7f906c06b2b0 2023-08-25T09:39:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f9070a77640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c06bd80 con 0x7f906c15e830 2023-08-25T09:39:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f906ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f906c15ec10 0x7f906c0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41166/0 (socket says 172.21.15.70:41166) 2023-08-25T09:39:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f906ad9d640 1 -- 172.21.15.70:0/1523702693 learned_addr learned my addr 172.21.15.70:0/1523702693 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.068+0000 7f906a59c640 1 --2- 172.21.15.70:0/1523702693 >> v2:172.21.15.155:3300/0 conn(0x7f906c15e830 0x7f906c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 43973362 0 0) 0x7f906c0b1b60 con 0x7f906c15ec10 2023-08-25T09:39:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9058003520 con 0x7f906c15ec10 2023-08-25T09:39:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4215317983 0 0) 0x7f9058003520 con 0x7f906c15ec10 2023-08-25T09:39:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 >> v2:172.21.15.155:3300/0 conn(0x7f906c15e830 msgr2=0x7f906c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 --2- 172.21.15.70:0/1523702693 >> v2:172.21.15.155:3300/0 conn(0x7f906c15e830 0x7f906c06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 >> v1:172.21.15.155:6789/0 conn(0x7f906c06b2b0 legacy=0x7f906c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c06baa0 con 0x7f906c15ec10 2023-08-25T09:39:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f90640019f0 con 0x7f906c15ec10 2023-08-25T09:39:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 >> v1:172.21.15.70:6789/0 conn(0x7f906c15ec10 legacy=0x7f906c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9058003970 0x7f9058003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90580047c0 0x7f9058004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9058005180 0x7f905800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c06bd80 con 0x7f9058005180 2023-08-25T09:39:50.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905800c0d0 con 0x7f9058003970 2023-08-25T09:39:50.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905800c290 con 0x7f90580047c0 2023-08-25T09:39:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.069+0000 7f906ad9d640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9058005180 0x7f905800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.070+0000 7f906a59c640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9058003970 0x7f9058003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.070+0000 7f906ad9d640 1 -- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90580047c0 msgr2=0x7f9058004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.070+0000 7f906ad9d640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f90580047c0 0x7f9058004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.070+0000 7f906ad9d640 1 -- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9058003970 msgr2=0x7f9058003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.070+0000 7f906ad9d640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9058003970 0x7f9058003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.070+0000 7f906ad9d640 1 -- 172.21.15.70:0/1523702693 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058003520 con 0x7f9058005180 2023-08-25T09:39:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.071+0000 7f906ad9d640 1 -- 172.21.15.70:0/1523702693 reap_dead start 2023-08-25T09:39:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.071+0000 7f906ad9d640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9058005180 0x7f905800b980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f906400a830 tx=0x7f90580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4632ff2c63529552 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.071+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640072d0 con 0x7f9058005180 2023-08-25T09:39:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.071+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906402f070 con 0x7f9058005180 2023-08-25T09:39:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.071+0000 7f9069d9b640 1 -- 172.21.15.70:0/1523702693 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640076b0 con 0x7f9058005180 2023-08-25T09:39:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 -- 172.21.15.70:0/1523702693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9058005180 msgr2=0x7f905800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9058005180 0x7f905800b980 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f906400a830 tx=0x7f90580057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 -- 172.21.15.70:0/1523702693 shutdown_connections 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 --2- 172.21.15.70:0/1523702693 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9058005180 0x7f905800b980 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 -- 172.21.15.70:0/1523702693 >> 172.21.15.70:0/1523702693 conn(0x7f906c069850 msgr2=0x7f906c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 -- 172.21.15.70:0/1523702693 shutdown_connections 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.072+0000 7f9070a77640 1 -- 172.21.15.70:0/1523702693 wait complete. 2023-08-25T09:39:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.074+0000 7f9070a77640 1 Processor -- start 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.074+0000 7f9070a77640 1 -- start start 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f9070a77640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9058005180 0x7f906c144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f9070a77640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f906c147900 0x7f906c144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f9070a77640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 0x7f906c0ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f9070a77640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c06ece0 con 0x7f906c0ac870 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f9070a77640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0ad0f0 con 0x7f906c147900 2023-08-25T09:39:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f9070a77640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c06eec0 con 0x7f9058005180 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f906ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 0x7f906c0ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f906ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 0x7f906c0ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54746/0 (socket says 172.21.15.70:54746) 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f906ad9d640 1 -- 172.21.15.70:0/3784927390 learned_addr learned my addr 172.21.15.70:0/3784927390 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.075+0000 7f906a59c640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9058005180 0x7f906c144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f906ad9d640 1 -- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9058005180 msgr2=0x7f906c144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f906ad9d640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9058005180 0x7f906c144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f906ad9d640 1 -- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f906c147900 msgr2=0x7f906c144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9063fff640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f906c147900 0x7f906c144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f906ad9d640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f906c147900 0x7f906c144500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f906ad9d640 1 -- 172.21.15.70:0/3784927390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c06f1a0 con 0x7f906c0ac870 2023-08-25T09:39:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9063fff640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f906c147900 0x7f906c144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:50.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f906ad9d640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 0x7f906c0ae820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9064001580 tx=0x7f9064001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:50.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064004e20 con 0x7f906c0ac870 2023-08-25T09:39:50.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90640017f0 con 0x7f906c0ac870 2023-08-25T09:39:50.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064001b60 con 0x7f906c0ac870 2023-08-25T09:39:50.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c06f480 con 0x7f906c0ac870 2023-08-25T09:39:50.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.076+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c0aed60 con 0x7f906c0ac870 2023-08-25T09:39:50.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.077+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c04e420 con 0x7f906c0ac870 2023-08-25T09:39:50.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.078+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9064031030 con 0x7f906c0ac870 2023-08-25T09:39:50.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.079+0000 7f9062ffd640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f904c06a080 0x7f904c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:50.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.079+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f90640aa210 con 0x7f906c0ac870 2023-08-25T09:39:50.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.079+0000 7f906a59c640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f904c06a080 0x7f904c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:50.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.080+0000 7f906a59c640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f904c06a080 0x7f904c06c540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f905c006fd0 tx=0x7f905c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:50.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.083+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f90640758f0 con 0x7f906c0ac870 2023-08-25T09:39:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.273+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f906c04e420 con 0x7f906c0ac870 2023-08-25T09:39:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.274+0000 7f9062ffd640 1 -- 172.21.15.70:0/3784927390 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f9064078f10 con 0x7f906c0ac870 2023-08-25T09:39:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:50.278 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:50.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f904c06a080 msgr2=0x7f904c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f904c06a080 0x7f904c06c540 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f905c006fd0 tx=0x7f905c008040 comp rx=0 tx=0).stop 2023-08-25T09:39:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 msgr2=0x7f906c0ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 0x7f906c0ae820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9064001580 tx=0x7f9064001d80 comp rx=0 tx=0).stop 2023-08-25T09:39:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 shutdown_connections 2023-08-25T09:39:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f904c06a080 0x7f904c06c540 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f906c0ac870 0x7f906c0ae820 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f906c147900 0x7f906c144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 --2- 172.21.15.70:0/3784927390 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9058005180 0x7f906c144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:50.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 >> 172.21.15.70:0/3784927390 conn(0x7f906c069850 msgr2=0x7f906c0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:50.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 shutdown_connections 2023-08-25T09:39:50.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.277+0000 7f9070a77640 1 -- 172.21.15.70:0/3784927390 wait complete. 2023-08-25T09:39:50.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:50.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.521+0000 7ff0f8c57640 1 Processor -- start 2023-08-25T09:39:51.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.521+0000 7ff0f8c57640 1 -- start start 2023-08-25T09:39:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.522+0000 7ff0f8c57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff0f415ae50 0x7ff0f40c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.522+0000 7ff0f8c57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:51.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.522+0000 7ff0f8c57640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0f40cc5f0 con 0x7ff0f40ce060 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.522+0000 7ff0f8c57640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f4070650 con 0x7ff0f415ae50 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f8c57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f4070930 con 0x7ff0f40cdc20 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f259c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff0f415ae50 0x7ff0f40c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff0f40ce060 0x7ff0f40ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41170/0 (socket says 172.21.15.70:41170) 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f2d9d640 1 -- 172.21.15.70:0/115437280 learned_addr learned my addr 172.21.15.70:0/115437280 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0e9d9b640 1 --2- 172.21.15.70:0/115437280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:51.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f1d9b640 1 -- 172.21.15.70:0/115437280 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2478499009 0 0) 0x7ff0f40cc5f0 con 0x7ff0f40ce060 2023-08-25T09:39:51.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f1d9b640 1 -- 172.21.15.70:0/115437280 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0e0003520 con 0x7ff0f40ce060 2023-08-25T09:39:51.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f259c640 1 -- 172.21.15.70:0/115437280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 msgr2=0x7ff0f4069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:51.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f259c640 1 --2- 172.21.15.70:0/115437280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f259c640 1 -- 172.21.15.70:0/115437280 >> v1:172.21.15.70:6789/0 conn(0x7ff0f40ce060 legacy=0x7ff0f40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.523+0000 7ff0f259c640 1 -- 172.21.15.70:0/115437280 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4070c10 con 0x7ff0f415ae50 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.525+0000 7ff0f259c640 1 --2- 172.21.15.70:0/115437280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c2110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40097a0 tx=0x7ff0e402b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff36e5805d52f091 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.525+0000 7ff0f1d9b640 1 -- 172.21.15.70:0/115437280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4030800 con 0x7ff0f415ae50 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.525+0000 7ff0f1d9b640 1 -- 172.21.15.70:0/115437280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e4030960 con 0x7ff0f415ae50 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.525+0000 7ff0f1d9b640 1 -- 172.21.15.70:0/115437280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4030cf0 con 0x7ff0f415ae50 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.525+0000 7ff0f8c57640 1 -- 172.21.15.70:0/115437280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 msgr2=0x7ff0f40c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:51.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.525+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/115437280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c2110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40097a0 tx=0x7ff0e402b940 comp rx=0 tx=0).stop 2023-08-25T09:39:51.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.526+0000 7ff0f8c57640 1 -- 172.21.15.70:0/115437280 shutdown_connections 2023-08-25T09:39:51.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.526+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/115437280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.526+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/115437280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c2110 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.526+0000 7ff0f8c57640 1 -- 172.21.15.70:0/115437280 >> 172.21.15.70:0/115437280 conn(0x7ff0f40cb0d0 msgr2=0x7ff0f40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:51.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.526+0000 7ff0f8c57640 1 -- 172.21.15.70:0/115437280 shutdown_connections 2023-08-25T09:39:51.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.526+0000 7ff0f8c57640 1 -- 172.21.15.70:0/115437280 wait complete. 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.527+0000 7ff0f8c57640 1 Processor -- start 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.527+0000 7ff0f8c57640 1 -- start start 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.528+0000 7ff0f8c57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.528+0000 7ff0f8c57640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f40ce060 0x7ff0f40b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.528+0000 7ff0f8c57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.528+0000 7ff0f8c57640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b01c0 con 0x7ff0f415ae50 2023-08-25T09:39:51.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.528+0000 7ff0f8c57640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b3ed0 con 0x7ff0f40ce060 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.528+0000 7ff0f8c57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f40b0460 con 0x7ff0f40cdc20 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54752/0 (socket says 172.21.15.70:54752) 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 -- 172.21.15.70:0/3357315022 learned_addr learned my addr 172.21.15.70:0/3357315022 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 -- 172.21.15.70:0/3357315022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 msgr2=0x7ff0f4157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 -- 172.21.15.70:0/3357315022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f40ce060 msgr2=0x7ff0f40b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f40ce060 0x7ff0f40b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 -- 172.21.15.70:0/3357315022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e4009400 con 0x7ff0f415ae50 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.529+0000 7ff0f2d9d640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c9af0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c400 tx=0x7ff0ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.530+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec005ce0 con 0x7ff0f415ae50 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.530+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec005e40 con 0x7ff0f415ae50 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.530+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ec0072d0 con 0x7ff0f415ae50 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.530+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f40b0740 con 0x7ff0f415ae50 2023-08-25T09:39:51.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.530+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f40c8490 con 0x7ff0f415ae50 2023-08-25T09:39:51.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.531+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f40b0980 con 0x7ff0f415ae50 2023-08-25T09:39:51.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.532+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff0ec0074f0 con 0x7ff0f415ae50 2023-08-25T09:39:51.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.533+0000 7ff0eb7fe640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0d406a050 0x7ff0d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:51.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.533+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff0ec087a30 con 0x7ff0f415ae50 2023-08-25T09:39:51.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.533+0000 7ff0f259c640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0d406a050 0x7ff0d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:51.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.533+0000 7ff0f259c640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0d406a050 0x7ff0d406c510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4036000 tx=0x7ff0e4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:51.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.537+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff0ec053260 con 0x7ff0f415ae50 2023-08-25T09:39:51.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.727+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0f40c86e0 con 0x7ff0f415ae50 2023-08-25T09:39:51.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.728+0000 7ff0eb7fe640 1 -- 172.21.15.70:0/3357315022 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0ec056880 con 0x7ff0f415ae50 2023-08-25T09:39:51.849 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:51.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.730+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0d406a050 msgr2=0x7ff0d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:51.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.730+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0d406a050 0x7ff0d406c510 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4036000 tx=0x7ff0e4030000 comp rx=0 tx=0).stop 2023-08-25T09:39:51.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.730+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 msgr2=0x7ff0f40c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:51.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.730+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c9af0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec00c400 tx=0x7ff0ec00c8c0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 shutdown_connections 2023-08-25T09:39:51.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0d406a050 0x7ff0d406c510 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0f415ae50 0x7ff0f40c9af0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0f40ce060 0x7ff0f40b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 --2- 172.21.15.70:0/3357315022 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0f40cdc20 0x7ff0f4157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:51.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 >> 172.21.15.70:0/3357315022 conn(0x7ff0f40cb0d0 msgr2=0x7ff0f4159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:51.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 shutdown_connections 2023-08-25T09:39:51.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:50.731+0000 7ff0f8c57640 1 -- 172.21.15.70:0/3357315022 wait complete. 2023-08-25T09:39:51.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:52.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.079+0000 7f5c3adab640 1 Processor -- start 2023-08-25T09:39:52.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.079+0000 7f5c3adab640 1 -- start start 2023-08-25T09:39:52.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.081+0000 7f5c3adab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5c3415ae50 0x7f5c340c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.081+0000 7f5c3adab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c34069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.081+0000 7f5c3adab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c340cc5f0 con 0x7f5c340ce060 2023-08-25T09:39:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.081+0000 7f5c3adab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c34070650 con 0x7f5c3415ae50 2023-08-25T09:39:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.081+0000 7f5c3adab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34070930 con 0x7f5c340cdc20 2023-08-25T09:39:52.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.082+0000 7f5c39348640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5c340ce060 0x7f5c340ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41174/0 (socket says 172.21.15.70:41174) 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.082+0000 7f5c39348640 1 -- 172.21.15.70:0/2618730208 learned_addr learned my addr 172.21.15.70:0/2618730208 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.082+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c34069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.082+0000 7f5c38b47640 1 --2- 172.21.15.70:0/2618730208 >> v2:172.21.15.155:3300/0 conn(0x7f5c3415ae50 0x7f5c340c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.082+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54409265 0 0) 0x7f5c340cc5f0 con 0x7f5c340ce060 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.082+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c24003520 con 0x7f5c340ce060 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 904837440 0 0) 0x7f5c24003520 con 0x7f5c340ce060 2023-08-25T09:39:52.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 msgr2=0x7f5c34069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c34069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 >> v2:172.21.15.155:3300/0 conn(0x7f5c3415ae50 msgr2=0x7f5c340c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 --2- 172.21.15.70:0/2618730208 >> v2:172.21.15.155:3300/0 conn(0x7f5c3415ae50 0x7f5c340c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34070c10 con 0x7f5c340ce060 2023-08-25T09:39:52.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5c2c0019f0 con 0x7f5c340ce060 2023-08-25T09:39:52.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 >> v1:172.21.15.70:6789/0 conn(0x7f5c340ce060 legacy=0x7f5c340ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c24003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24003ef0 0x7f5c240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c24004d00 0x7f5c24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34070650 con 0x7f5c24003ef0 2023-08-25T09:39:52.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c34070930 con 0x7f5c24004d00 2023-08-25T09:39:52.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c2400ba60 con 0x7f5c340cdc20 2023-08-25T09:39:52.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24003ef0 0x7f5c240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c38b47640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c24003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.083+0000 7f5c39348640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c24004d00 0x7f5c24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.084+0000 7f5c33fff640 1 -- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 msgr2=0x7f5c24003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.084+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c24003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.084+0000 7f5c33fff640 1 -- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c24004d00 msgr2=0x7f5c24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.084+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c24004d00 0x7f5c24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.084+0000 7f5c33fff640 1 -- 172.21.15.70:0/2618730208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24003520 con 0x7f5c24003ef0 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.084+0000 7f5c38b47640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c340cdc20 0x7f5c24003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:52.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24003ef0 0x7f5c240047c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009ac0 tx=0x7f5c240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfec645ab1b9311 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2002c6b0 con 0x7f5c24003ef0 2023-08-25T09:39:52.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2002c810 con 0x7f5c24003ef0 2023-08-25T09:39:52.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c337fe640 1 -- 172.21.15.70:0/2618730208 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2002cba0 con 0x7f5c24003ef0 2023-08-25T09:39:52.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c3adab640 1 -- 172.21.15.70:0/2618730208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24003ef0 msgr2=0x7f5c240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2618730208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c24003ef0 0x7f5c240047c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009ac0 tx=0x7f5c240057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.085+0000 7f5c39348640 1 -- 172.21.15.70:0/2618730208 reap_dead start 2023-08-25T09:39:52.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.086+0000 7f5c3adab640 1 -- 172.21.15.70:0/2618730208 shutdown_connections 2023-08-25T09:39:52.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.086+0000 7f5c3adab640 1 -- 172.21.15.70:0/2618730208 >> 172.21.15.70:0/2618730208 conn(0x7f5c340cb0d0 msgr2=0x7f5c340cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:52.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.086+0000 7f5c3adab640 1 -- 172.21.15.70:0/2618730208 shutdown_connections 2023-08-25T09:39:52.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.086+0000 7f5c3adab640 1 -- 172.21.15.70:0/2618730208 wait complete. 2023-08-25T09:39:52.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.087+0000 7f5c3adab640 1 Processor -- start 2023-08-25T09:39:52.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.088+0000 7f5c3adab640 1 -- start start 2023-08-25T09:39:52.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c3adab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c24003ef0 0x7f5c34158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c3adab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c340c58a0 0x7f5c340c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c3adab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 0x7f5c3414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c3adab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340c6700 con 0x7f5c3414f650 2023-08-25T09:39:52.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c3adab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340c6220 con 0x7f5c340c58a0 2023-08-25T09:39:52.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c3adab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c340c6420 con 0x7f5c24003ef0 2023-08-25T09:39:52.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 0x7f5c3414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 0x7f5c3414fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54776/0 (socket says 172.21.15.70:54776) 2023-08-25T09:39:52.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 -- 172.21.15.70:0/2247332485 learned_addr learned my addr 172.21.15.70:0/2247332485 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:52.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c38b47640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c24003ef0 0x7f5c34158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c340c58a0 0x7f5c340c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 -- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c24003ef0 msgr2=0x7f5c34158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c24003ef0 0x7f5c34158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 -- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c340c58a0 msgr2=0x7f5c340c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c340c58a0 0x7f5c340c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c39348640 1 -- 172.21.15.70:0/2247332485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c341f2930 con 0x7f5c3414f650 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.089+0000 7f5c33fff640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c340c58a0 0x7f5c340c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.090+0000 7f5c39348640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 0x7f5c3414fae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5c340b00d0 tx=0x7f5c2c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.090+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c007ae0 con 0x7f5c3414f650 2023-08-25T09:39:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.090+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c2c00eda0 con 0x7f5c3414f650 2023-08-25T09:39:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.090+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c014670 con 0x7f5c3414f650 2023-08-25T09:39:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.090+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c341f2c10 con 0x7f5c3414f650 2023-08-25T09:39:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.090+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c341f25f0 con 0x7f5c3414f650 2023-08-25T09:39:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.091+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c341f3130 con 0x7f5c3414f650 2023-08-25T09:39:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.092+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5c2c0147d0 con 0x7f5c3414f650 2023-08-25T09:39:52.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.093+0000 7f5c317fa640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5c1806a050 0x7f5c1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.093+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c2c0883e0 con 0x7f5c3414f650 2023-08-25T09:39:52.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.094+0000 7f5c38b47640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5c1806a050 0x7f5c1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.094+0000 7f5c38b47640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5c1806a050 0x7f5c1806c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c28006fd0 tx=0x7f5c28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.098+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5c341f3130 con 0x7f5c3414f650 2023-08-25T09:39:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.283+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c341f3130 con 0x7f5c3414f650 2023-08-25T09:39:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.284+0000 7f5c317fa640 1 -- 172.21.15.70:0/2247332485 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c341f3130 con 0x7f5c3414f650 2023-08-25T09:39:52.285 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.286+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5c1806a050 msgr2=0x7f5c1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.286+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5c1806a050 0x7f5c1806c510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5c28006fd0 tx=0x7f5c28008040 comp rx=0 tx=0).stop 2023-08-25T09:39:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.286+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 msgr2=0x7f5c3414fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.286+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 0x7f5c3414fae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5c340b00d0 tx=0x7f5c2c00e530 comp rx=0 tx=0).stop 2023-08-25T09:39:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 shutdown_connections 2023-08-25T09:39:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5c1806a050 0x7f5c1806c510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5c3414f650 0x7f5c3414fae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5c340c58a0 0x7f5c340c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 --2- 172.21.15.70:0/2247332485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5c24003ef0 0x7f5c34158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 >> 172.21.15.70:0/2247332485 conn(0x7f5c340cb0d0 msgr2=0x7f5c340c45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:52.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 shutdown_connections 2023-08-25T09:39:52.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.287+0000 7f5c3adab640 1 -- 172.21.15.70:0/2247332485 wait complete. 2023-08-25T09:39:52.305 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:52.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.505+0000 7f77048ec640 1 Processor -- start 2023-08-25T09:39:52.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.505+0000 7f77048ec640 1 -- start start 2023-08-25T09:39:52.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f77048ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7700159bf0 0x7f770015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f77048ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77000cd660 0x7f77000cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f77048ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770015e780 con 0x7f770006f0e0 2023-08-25T09:39:52.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f77048ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77000ce030 con 0x7f77000cd660 2023-08-25T09:39:52.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f77048ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770006f8e0 con 0x7f7700159bf0 2023-08-25T09:39:52.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f76fed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77000cd660 0x7f77000cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f76fed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77000cd660 0x7f77000cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54780/0 (socket says 172.21.15.70:54780) 2023-08-25T09:39:52.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.507+0000 7f76fed9d640 1 -- 172.21.15.70:0/3915562939 learned_addr learned my addr 172.21.15.70:0/3915562939 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:52.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.508+0000 7f76fed9d640 1 -- 172.21.15.70:0/3915562939 >> v2:172.21.15.155:3300/0 conn(0x7f7700159bf0 msgr2=0x7f770015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.508+0000 7f76fed9d640 1 --2- 172.21.15.70:0/3915562939 >> v2:172.21.15.155:3300/0 conn(0x7f7700159bf0 0x7f770015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.508+0000 7f76fed9d640 1 -- 172.21.15.70:0/3915562939 >> v1:172.21.15.155:6789/0 conn(0x7f770006f0e0 legacy=0x7f7700070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.508+0000 7f76fed9d640 1 -- 172.21.15.70:0/3915562939 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770006fbc0 con 0x7f77000cd660 2023-08-25T09:39:52.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.509+0000 7f76fed9d640 1 --2- 172.21.15.70:0/3915562939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77000cd660 0x7f77000cdaf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f770006d1f0 tx=0x7f76f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f785555ac79f4c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.509+0000 7f76fdd9b640 1 -- 172.21.15.70:0/3915562939 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f802ca80 con 0x7f77000cd660 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.509+0000 7f76fdd9b640 1 -- 172.21.15.70:0/3915562939 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f802cbe0 con 0x7f77000cd660 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.509+0000 7f76fdd9b640 1 -- 172.21.15.70:0/3915562939 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f80343f0 con 0x7f77000cd660 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 -- 172.21.15.70:0/3915562939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77000cd660 msgr2=0x7f77000cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 --2- 172.21.15.70:0/3915562939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77000cd660 0x7f77000cdaf0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f770006d1f0 tx=0x7f76f802bee0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 -- 172.21.15.70:0/3915562939 shutdown_connections 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 --2- 172.21.15.70:0/3915562939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77000cd660 0x7f77000cdaf0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 --2- 172.21.15.70:0/3915562939 >> v2:172.21.15.155:3300/0 conn(0x7f7700159bf0 0x7f770015e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 -- 172.21.15.70:0/3915562939 >> 172.21.15.70:0/3915562939 conn(0x7f77000cb0d0 msgr2=0x7f77000cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:52.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 -- 172.21.15.70:0/3915562939 shutdown_connections 2023-08-25T09:39:52.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.510+0000 7f77048ec640 1 -- 172.21.15.70:0/3915562939 wait complete. 2023-08-25T09:39:52.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.511+0000 7f77048ec640 1 Processor -- start 2023-08-25T09:39:52.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.512+0000 7f77048ec640 1 -- start start 2023-08-25T09:39:52.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.512+0000 7f77048ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f770006f0e0 0x7f770014cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f77048ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77000cd660 0x7f7700156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f77048ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 0x7f7700156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f77048ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001f7010 con 0x7f7700159bf0 2023-08-25T09:39:52.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f77048ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77001f6400 con 0x7f770006f0e0 2023-08-25T09:39:52.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f77048ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77001f66e0 con 0x7f77000cd660 2023-08-25T09:39:52.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 0x7f7700156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 0x7f7700156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54794/0 (socket says 172.21.15.70:54794) 2023-08-25T09:39:52.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 -- 172.21.15.70:0/1640562406 learned_addr learned my addr 172.21.15.70:0/1640562406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:52.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 -- 172.21.15.70:0/1640562406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77000cd660 msgr2=0x7f7700156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77000cd660 0x7f7700156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 -- 172.21.15.70:0/1640562406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f770006f0e0 msgr2=0x7f770014cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f770006f0e0 0x7f770014cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.513+0000 7f76fed9d640 1 -- 172.21.15.70:0/1640562406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8009b30 con 0x7f7700159bf0 2023-08-25T09:39:52.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.514+0000 7f76fed9d640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 0x7f7700156bb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f76f8036040 tx=0x7f76f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.514+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f803e450 con 0x7f7700159bf0 2023-08-25T09:39:52.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.514+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f803e5b0 con 0x7f7700159bf0 2023-08-25T09:39:52.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.514+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f803e960 con 0x7f7700159bf0 2023-08-25T09:39:52.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.514+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77001f69c0 con 0x7f7700159bf0 2023-08-25T09:39:52.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.514+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770014d190 con 0x7f7700159bf0 2023-08-25T09:39:52.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.515+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770006fe80 con 0x7f7700159bf0 2023-08-25T09:39:52.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.516+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f76f802cd50 con 0x7f7700159bf0 2023-08-25T09:39:52.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.517+0000 7f76f6ffd640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f76e006a050 0x7f76e006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.517+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76f80ab150 con 0x7f7700159bf0 2023-08-25T09:39:52.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.517+0000 7f76fe59c640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f76e006a050 0x7f76e006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.518+0000 7f76fe59c640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f76e006a050 0x7f76e006c510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009710 tx=0x7f76f0006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.522+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f770006fe80 con 0x7f7700159bf0 2023-08-25T09:39:52.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.701+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f770006fe80 con 0x7f7700159bf0 2023-08-25T09:39:52.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.701+0000 7f76f6ffd640 1 -- 172.21.15.70:0/1640562406 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f770006fe80 con 0x7f7700159bf0 2023-08-25T09:39:52.703 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:52.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f76e006a050 msgr2=0x7f76e006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f76e006a050 0x7f76e006c510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009710 tx=0x7f76f0006c90 comp rx=0 tx=0).stop 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 msgr2=0x7f7700156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 0x7f7700156bb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f76f8036040 tx=0x7f76f8004a40 comp rx=0 tx=0).stop 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 shutdown_connections 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f76e006a050 0x7f76e006c510 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7700159bf0 0x7f7700156bb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77000cd660 0x7f7700156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 --2- 172.21.15.70:0/1640562406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f770006f0e0 0x7f770014cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 >> 172.21.15.70:0/1640562406 conn(0x7f77000cb0d0 msgr2=0x7f77000ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:52.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 shutdown_connections 2023-08-25T09:39:52.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.704+0000 7f77048ec640 1 -- 172.21.15.70:0/1640562406 wait complete. 2023-08-25T09:39:52.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.931+0000 7f1e76ef3640 1 Processor -- start 2023-08-25T09:39:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.931+0000 7f1e76ef3640 1 -- start start 2023-08-25T09:39:52.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.932+0000 7f1e76ef3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e700c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.932+0000 7f1e76ef3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e700b1ce0 0x7f1e700ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.932+0000 7f1e76ef3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e700b2170 con 0x7f1e7015a920 2023-08-25T09:39:52.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.932+0000 7f1e76ef3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e700ad8f0 con 0x7f1e700b1ce0 2023-08-25T09:39:52.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.932+0000 7f1e76ef3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700adbb0 con 0x7f1e700b0320 2023-08-25T09:39:52.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.933+0000 7f1e75ef1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e700c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.933+0000 7f1e75ef1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e700c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:56268/0 (socket says 172.21.15.70:56268) 2023-08-25T09:39:52.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.933+0000 7f1e75ef1640 1 -- 172.21.15.70:0/4228881819 learned_addr learned my addr 172.21.15.70:0/4228881819 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:52.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.933+0000 7f1e766f2640 1 --2- 172.21.15.70:0/4228881819 >> v2:172.21.15.155:3300/0 conn(0x7f1e700b1ce0 0x7f1e700ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.934+0000 7f1e75ef1640 1 -- 172.21.15.70:0/4228881819 >> v2:172.21.15.155:3300/0 conn(0x7f1e700b1ce0 msgr2=0x7f1e700ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.934+0000 7f1e75ef1640 1 --2- 172.21.15.70:0/4228881819 >> v2:172.21.15.155:3300/0 conn(0x7f1e700b1ce0 0x7f1e700ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.934+0000 7f1e75ef1640 1 -- 172.21.15.70:0/4228881819 >> v1:172.21.15.155:6789/0 conn(0x7f1e7015a920 legacy=0x7f1e700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.934+0000 7f1e75ef1640 1 -- 172.21.15.70:0/4228881819 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700c90a0 con 0x7f1e700b0320 2023-08-25T09:39:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.934+0000 7f1e74eef640 1 -- 172.21.15.70:0/4228881819 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417259321 0 0) 0x7f1e700b2170 con 0x7f1e7015a920 2023-08-25T09:39:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.935+0000 7f1e75ef1640 1 --2- 172.21.15.70:0/4228881819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e700c9e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009850 tx=0x7f1e6c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62d787865ca86c5d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e74eef640 1 -- 172.21.15.70:0/4228881819 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c004230 con 0x7f1e700b0320 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e74eef640 1 -- 172.21.15.70:0/4228881819 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c004390 con 0x7f1e700b0320 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e74eef640 1 -- 172.21.15.70:0/4228881819 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c003810 con 0x7f1e700b0320 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 -- 172.21.15.70:0/4228881819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 msgr2=0x7f1e700c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/4228881819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e700c9e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009850 tx=0x7f1e6c02b960 comp rx=0 tx=0).stop 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 -- 172.21.15.70:0/4228881819 shutdown_connections 2023-08-25T09:39:52.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/4228881819 >> v2:172.21.15.155:3300/0 conn(0x7f1e700b1ce0 0x7f1e700ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/4228881819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e700c9e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 -- 172.21.15.70:0/4228881819 >> 172.21.15.70:0/4228881819 conn(0x7f1e700ce830 msgr2=0x7f1e700cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 -- 172.21.15.70:0/4228881819 shutdown_connections 2023-08-25T09:39:52.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.936+0000 7f1e76ef3640 1 -- 172.21.15.70:0/4228881819 wait complete. 2023-08-25T09:39:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.938+0000 7f1e76ef3640 1 Processor -- start 2023-08-25T09:39:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.938+0000 7f1e76ef3640 1 -- start start 2023-08-25T09:39:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.939+0000 7f1e76ef3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e701ef9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.939+0000 7f1e76ef3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e700b1ce0 0x7f1e70155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.939+0000 7f1e76ef3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 0x7f1e701561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.939+0000 7f1e76ef3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70069a80 con 0x7f1e7015a920 2023-08-25T09:39:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.939+0000 7f1e76ef3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e701efee0 con 0x7f1e700b1ce0 2023-08-25T09:39:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.939+0000 7f1e76ef3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c4d40 con 0x7f1e700b0320 2023-08-25T09:39:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 0x7f1e701561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 0x7f1e701561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54798/0 (socket says 172.21.15.70:54798) 2023-08-25T09:39:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 -- 172.21.15.70:0/650578720 learned_addr learned my addr 172.21.15.70:0/650578720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e75ef1640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e701ef9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e756f0640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e700b1ce0 0x7f1e70155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 -- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 msgr2=0x7f1e701ef9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e701ef9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 -- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e700b1ce0 msgr2=0x7f1e70155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e700b1ce0 0x7f1e70155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.940+0000 7f1e766f2640 1 -- 172.21.15.70:0/650578720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c009460 con 0x7f1e7015a920 2023-08-25T09:39:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.941+0000 7f1e766f2640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 0x7f1e701561f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800efc0 tx=0x7f1e6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.941+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68014480 con 0x7f1e7015a920 2023-08-25T09:39:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.941+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800f040 con 0x7f1e7015a920 2023-08-25T09:39:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.941+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700c44a0 con 0x7f1e7015a920 2023-08-25T09:39:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.941+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68014820 con 0x7f1e7015a920 2023-08-25T09:39:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.941+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700c4a60 con 0x7f1e7015a920 2023-08-25T09:39:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.942+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70069bc0 con 0x7f1e7015a920 2023-08-25T09:39:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.943+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e68014980 con 0x7f1e7015a920 2023-08-25T09:39:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.944+0000 7f1e667fc640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5406a050 0x7f1e5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.945+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e68089410 con 0x7f1e7015a920 2023-08-25T09:39:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.945+0000 7f1e75ef1640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5406a050 0x7f1e5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:52.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.945+0000 7f1e75ef1640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5406a050 0x7f1e5406c510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c035000 tx=0x7f1e6c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:52.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:52.949+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e70069bc0 con 0x7f1e7015a920 2023-08-25T09:39:53.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.139+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e70069bc0 con 0x7f1e7015a920 2023-08-25T09:39:53.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.140+0000 7f1e667fc640 1 -- 172.21.15.70:0/650578720 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e70069bc0 con 0x7f1e7015a920 2023-08-25T09:39:53.146 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:53.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.142+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5406a050 msgr2=0x7f1e5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.142+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5406a050 0x7f1e5406c510 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c035000 tx=0x7f1e6c008030 comp rx=0 tx=0).stop 2023-08-25T09:39:53.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.142+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 msgr2=0x7f1e701561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.142+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 0x7f1e701561f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800efc0 tx=0x7f1e6800c490 comp rx=0 tx=0).stop 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 shutdown_connections 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5406a050 0x7f1e5406c510 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7015a920 0x7f1e701561f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e700b1ce0 0x7f1e70155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 --2- 172.21.15.70:0/650578720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e700b0320 0x7f1e701ef9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 >> 172.21.15.70:0/650578720 conn(0x7f1e700ce830 msgr2=0x7f1e70157900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:53.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 shutdown_connections 2023-08-25T09:39:53.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.143+0000 7f1e76ef3640 1 -- 172.21.15.70:0/650578720 wait complete. 2023-08-25T09:39:53.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:39:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.366+0000 7f762ec2d640 1 Processor -- start 2023-08-25T09:39:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.366+0000 7f762ec2d640 1 -- start start 2023-08-25T09:39:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762ec2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280ca780 0x7f76280cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762ec2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76280b3cf0 0x7f76280b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762ec2d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280b12b0 con 0x7f762815acd0 2023-08-25T09:39:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762ec2d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762806c0e0 con 0x7f76280b3cf0 2023-08-25T09:39:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762ec2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f762806ae20 con 0x7f76280ca780 2023-08-25T09:39:53.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762d1ca640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76280b3cf0 0x7f76280b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762d1ca640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f76280b3cf0 0x7f76280b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54812/0 (socket says 172.21.15.70:54812) 2023-08-25T09:39:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f762d1ca640 1 -- 172.21.15.70:0/2770609113 learned_addr learned my addr 172.21.15.70:0/2770609113 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.368+0000 7f7627fff640 1 --2- 172.21.15.70:0/2770609113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280ca780 0x7f76280cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.369+0000 7f762d1ca640 1 -- 172.21.15.70:0/2770609113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280ca780 msgr2=0x7f76280cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.369+0000 7f762d1ca640 1 --2- 172.21.15.70:0/2770609113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280ca780 0x7f76280cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.369+0000 7f762d1ca640 1 -- 172.21.15.70:0/2770609113 >> v1:172.21.15.155:6789/0 conn(0x7f762815acd0 legacy=0x7f762815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.369+0000 7f762d1ca640 1 -- 172.21.15.70:0/2770609113 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762815b7c0 con 0x7f76280b3cf0 2023-08-25T09:39:53.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.370+0000 7f762d1ca640 1 --2- 172.21.15.70:0/2770609113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280b3cf0 0x7f76280b4140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7620009e50 tx=0x7f762002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace12f79b465a424 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.370+0000 7f76277fe640 1 -- 172.21.15.70:0/2770609113 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762002ca70 con 0x7f76280b3cf0 2023-08-25T09:39:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.370+0000 7f76277fe640 1 -- 172.21.15.70:0/2770609113 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762002cbd0 con 0x7f76280b3cf0 2023-08-25T09:39:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.370+0000 7f76277fe640 1 -- 172.21.15.70:0/2770609113 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76200343f0 con 0x7f76280b3cf0 2023-08-25T09:39:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 -- 172.21.15.70:0/2770609113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280b3cf0 msgr2=0x7f76280b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2770609113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280b3cf0 0x7f76280b4140 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7620009e50 tx=0x7f762002bed0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 -- 172.21.15.70:0/2770609113 shutdown_connections 2023-08-25T09:39:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2770609113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280b3cf0 0x7f76280b4140 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2770609113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280ca780 0x7f76280cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 -- 172.21.15.70:0/2770609113 >> 172.21.15.70:0/2770609113 conn(0x7f76280ce760 msgr2=0x7f76280ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 -- 172.21.15.70:0/2770609113 shutdown_connections 2023-08-25T09:39:53.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.371+0000 7f762ec2d640 1 -- 172.21.15.70:0/2770609113 wait complete. 2023-08-25T09:39:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.373+0000 7f762ec2d640 1 Processor -- start 2023-08-25T09:39:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.373+0000 7f762ec2d640 1 -- start start 2023-08-25T09:39:53.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f762ec2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280b3cf0 0x7f762814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f762ec2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 0x7f762814b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f762ec2d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762815acd0 0x7f76280ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f762ec2d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762814f370 con 0x7f76280ca780 2023-08-25T09:39:53.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f762ec2d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762814e390 con 0x7f762815acd0 2023-08-25T09:39:53.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f762ec2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76280ad1e0 con 0x7f76280b3cf0 2023-08-25T09:39:53.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.374+0000 7f7627fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 0x7f762814b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 0x7f762814b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54816/0 (socket says 172.21.15.70:54816) 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 -- 172.21.15.70:0/2133846230 learned_addr learned my addr 172.21.15.70:0/2133846230 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f762c9c9640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280b3cf0 0x7f762814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 -- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280b3cf0 msgr2=0x7f762814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f762d1ca640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762815acd0 0x7f76280ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280b3cf0 0x7f762814ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 -- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762815acd0 msgr2=0x7f76280ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762815acd0 0x7f76280ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 -- 172.21.15.70:0/2133846230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009b30 con 0x7f76280ca780 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f762c9c9640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280b3cf0 0x7f762814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f7627fff640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 0x7f762814b970 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f761400d800 tx=0x7f761400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.375+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400b6c0 con 0x7f76280ca780 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.376+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761400b840 con 0x7f76280ca780 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.376+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761400bbf0 con 0x7f76280ca780 2023-08-25T09:39:53.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.376+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76280ad4c0 con 0x7f76280ca780 2023-08-25T09:39:53.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.376+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f762814e910 con 0x7f76280ca780 2023-08-25T09:39:53.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.377+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f762815e0f0 con 0x7f76280ca780 2023-08-25T09:39:53.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.378+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f761400bd50 con 0x7f76280ca780 2023-08-25T09:39:53.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.378+0000 7f76257fa640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f760c06a050 0x7f760c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.379+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7614088b40 con 0x7f76280ca780 2023-08-25T09:39:53.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.379+0000 7f762c9c9640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f760c06a050 0x7f760c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.379+0000 7f762c9c9640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f760c06a050 0x7f760c06c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f761c000fb0 tx=0x7f761c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:53.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.383+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7614054220 con 0x7f76280ca780 2023-08-25T09:39:53.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.573+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f762804e420 con 0x7f76280ca780 2023-08-25T09:39:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.573+0000 7f76257fa640 1 -- 172.21.15.70:0/2133846230 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7614057840 con 0x7f76280ca780 2023-08-25T09:39:53.575 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:39:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.575+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f760c06a050 msgr2=0x7f760c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.575+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f760c06a050 0x7f760c06c510 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f761c000fb0 tx=0x7f761c009040 comp rx=0 tx=0).stop 2023-08-25T09:39:53.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 msgr2=0x7f762814b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 0x7f762814b970 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f761400d800 tx=0x7f761400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 shutdown_connections 2023-08-25T09:39:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f760c06a050 0x7f760c06c510 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f762815acd0 0x7f76280ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f76280ca780 0x7f762814b970 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 --2- 172.21.15.70:0/2133846230 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76280b3cf0 0x7f762814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 >> 172.21.15.70:0/2133846230 conn(0x7f76280ce760 msgr2=0x7f76280ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 shutdown_connections 2023-08-25T09:39:53.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.576+0000 7f762ec2d640 1 -- 172.21.15.70:0/2133846230 wait complete. 2023-08-25T09:39:53.595 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:39:53.595 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:53.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:53.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.872+0000 7fa89e014640 1 Processor -- start 2023-08-25T09:39:53.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.872+0000 7fa89e014640 1 -- start start 2023-08-25T09:39:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.873+0000 7fa89e014640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8980ce5b0 0x7fa8980ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.873+0000 7fa89e014640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa89815aa10 0x7fa8980b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.873+0000 7fa89e014640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980b2aa0 con 0x7fa8980b2680 2023-08-25T09:39:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.873+0000 7fa89e014640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa898159d10 con 0x7fa89815aa10 2023-08-25T09:39:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.873+0000 7fa89e014640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980ceed0 con 0x7fa8980ce5b0 2023-08-25T09:39:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa89815aa10 0x7fa8980b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa89815aa10 0x7fa8980b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54826/0 (socket says 172.21.15.70:54826) 2023-08-25T09:39:53.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 -- 172.21.15.70:0/3240856781 learned_addr learned my addr 172.21.15.70:0/3240856781 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:53.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 -- 172.21.15.70:0/3240856781 >> v2:172.21.15.155:3300/0 conn(0x7fa8980ce5b0 msgr2=0x7fa8980ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:53.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 --2- 172.21.15.70:0/3240856781 >> v2:172.21.15.155:3300/0 conn(0x7fa8980ce5b0 0x7fa8980ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 -- 172.21.15.70:0/3240856781 >> v1:172.21.15.155:6789/0 conn(0x7fa8980b2680 legacy=0x7fa89806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:53.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.874+0000 7fa88ffff640 1 -- 172.21.15.70:0/3240856781 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89815d3f0 con 0x7fa89815aa10 2023-08-25T09:39:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.875+0000 7fa88ffff640 1 --2- 172.21.15.70:0/3240856781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa8980b2140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa89806f470 tx=0x7fa88002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea47b581ea9fc007 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa896ffd640 1 -- 172.21.15.70:0/3240856781 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800317e0 con 0x7fa89815aa10 2023-08-25T09:39:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa896ffd640 1 -- 172.21.15.70:0/3240856781 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880031940 con 0x7fa89815aa10 2023-08-25T09:39:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa896ffd640 1 -- 172.21.15.70:0/3240856781 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880031cd0 con 0x7fa89815aa10 2023-08-25T09:39:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 -- 172.21.15.70:0/3240856781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 msgr2=0x7fa8980b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 --2- 172.21.15.70:0/3240856781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa8980b2140 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa89806f470 tx=0x7fa88002b880 comp rx=0 tx=0).stop 2023-08-25T09:39:53.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 -- 172.21.15.70:0/3240856781 shutdown_connections 2023-08-25T09:39:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 --2- 172.21.15.70:0/3240856781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa8980b2140 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 --2- 172.21.15.70:0/3240856781 >> v2:172.21.15.155:3300/0 conn(0x7fa8980ce5b0 0x7fa8980ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 -- 172.21.15.70:0/3240856781 >> 172.21.15.70:0/3240856781 conn(0x7fa898069850 msgr2=0x7fa898069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.876+0000 7fa89e014640 1 -- 172.21.15.70:0/3240856781 shutdown_connections 2023-08-25T09:39:53.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.877+0000 7fa89e014640 1 -- 172.21.15.70:0/3240856781 wait complete. 2023-08-25T09:39:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.878+0000 7fa89e014640 1 Processor -- start 2023-08-25T09:39:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.878+0000 7fa89e014640 1 -- start start 2023-08-25T09:39:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.879+0000 7fa89e014640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8980b2680 0x7fa8980ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.879+0000 7fa89e014640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8980ce5b0 0x7fa898156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.879+0000 7fa89e014640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa898142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.879+0000 7fa89e014640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980af620 con 0x7fa89815aa10 2023-08-25T09:39:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.879+0000 7fa89e014640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa898156b60 con 0x7fa8980b2680 2023-08-25T09:39:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.879+0000 7fa89e014640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8980ae9a0 con 0x7fa8980ce5b0 2023-08-25T09:39:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa898142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa898142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54832/0 (socket says 172.21.15.70:54832) 2023-08-25T09:39:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 -- 172.21.15.70:0/1897956136 learned_addr learned my addr 172.21.15.70:0/1897956136 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:53.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa88ffff640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8980ce5b0 0x7fa898156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa8977fe640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8980b2680 0x7fa8980ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 -- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8980ce5b0 msgr2=0x7fa898156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8980ce5b0 0x7fa898156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 -- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8980b2680 msgr2=0x7fa8980ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8980b2680 0x7fa8980ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 -- 172.21.15.70:0/1897956136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800094d0 con 0x7fa89815aa10 2023-08-25T09:39:53.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa8977fe640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8980b2680 0x7fa8980ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa88ffff640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8980ce5b0 0x7fa898156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.880+0000 7fa897fff640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa898142050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c7b0 tx=0x7fa89000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.881+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890003d80 con 0x7fa89815aa10 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.881+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa890005070 con 0x7fa89815aa10 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.881+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890005420 con 0x7fa89815aa10 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.881+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8980aec80 con 0x7fa89815aa10 2023-08-25T09:39:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.881+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8980af240 con 0x7fa89815aa10 2023-08-25T09:39:53.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.882+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89804e420 con 0x7fa89815aa10 2023-08-25T09:39:53.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.883+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa890005580 con 0x7fa89815aa10 2023-08-25T09:39:53.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.884+0000 7fa894ff9640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa87806a050 0x7fa87806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:53.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.884+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8900879f0 con 0x7fa89815aa10 2023-08-25T09:39:53.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.884+0000 7fa8977fe640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa87806a050 0x7fa87806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:53.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.885+0000 7fa8977fe640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa87806a050 0x7fa87806c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa888006fd0 tx=0x7fa888008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:53.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:53.888+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa8900530d0 con 0x7fa89815aa10 2023-08-25T09:39:54.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.076+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8980cae50 con 0x7fa89815aa10 2023-08-25T09:39:54.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.077+0000 7fa894ff9640 1 -- 172.21.15.70:0/1897956136 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fa8900566f0 con 0x7fa89815aa10 2023-08-25T09:39:54.080 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:54.080 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:54.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa87806a050 msgr2=0x7fa87806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa87806a050 0x7fa87806c510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa888006fd0 tx=0x7fa888008040 comp rx=0 tx=0).stop 2023-08-25T09:39:54.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 msgr2=0x7fa898142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa898142050 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c7b0 tx=0x7fa89000cc70 comp rx=0 tx=0).stop 2023-08-25T09:39:54.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 shutdown_connections 2023-08-25T09:39:54.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa87806a050 0x7fa87806c510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa89815aa10 0x7fa898142050 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8980ce5b0 0x7fa898156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 --2- 172.21.15.70:0/1897956136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8980b2680 0x7fa8980ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.080+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 >> 172.21.15.70:0/1897956136 conn(0x7fa898069850 msgr2=0x7fa89806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.081+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 shutdown_connections 2023-08-25T09:39:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.081+0000 7fa89e014640 1 -- 172.21.15.70:0/1897956136 wait complete. 2023-08-25T09:39:54.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:54.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:54.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.311+0000 7fcbea098640 1 Processor -- start 2023-08-25T09:39:54.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.311+0000 7fcbea098640 1 -- start start 2023-08-25T09:39:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.312+0000 7fcbea098640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcbe40b0320 0x7fcbe40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.312+0000 7fcbea098640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.312+0000 7fcbea098640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe40ad8f0 con 0x7fcbe415a920 2023-08-25T09:39:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.312+0000 7fcbea098640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe40adbb0 con 0x7fcbe40b0320 2023-08-25T09:39:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.312+0000 7fcbea098640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe40c90a0 con 0x7fcbe40b1ce0 2023-08-25T09:39:54.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcbe40b0320 0x7fcbe40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcbe40b0320 0x7fcbe40c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54846/0 (socket says 172.21.15.70:54846) 2023-08-25T09:39:54.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 -- 172.21.15.70:0/4148866130 learned_addr learned my addr 172.21.15.70:0/4148866130 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:54.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 -- 172.21.15.70:0/4148866130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 msgr2=0x7fcbe40ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:54.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 --2- 172.21.15.70:0/4148866130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 -- 172.21.15.70:0/4148866130 >> v1:172.21.15.155:6789/0 conn(0x7fcbe415a920 legacy=0x7fcbe40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:54.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.313+0000 7fcbe9096640 1 -- 172.21.15.70:0/4148866130 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe40c9380 con 0x7fcbe40b0320 2023-08-25T09:39:54.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.314+0000 7fcbe9096640 1 --2- 172.21.15.70:0/4148866130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe40b0320 0x7fcbe40c9e60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009850 tx=0x7fcbe002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcb9bd0cd986439 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:54.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbd7fff640 1 -- 172.21.15.70:0/4148866130 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0004230 con 0x7fcbe40b0320 2023-08-25T09:39:54.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbd7fff640 1 -- 172.21.15.70:0/4148866130 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0004390 con 0x7fcbe40b0320 2023-08-25T09:39:54.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbd7fff640 1 -- 172.21.15.70:0/4148866130 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0031830 con 0x7fcbe40b0320 2023-08-25T09:39:54.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 -- 172.21.15.70:0/4148866130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe40b0320 msgr2=0x7fcbe40c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 --2- 172.21.15.70:0/4148866130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe40b0320 0x7fcbe40c9e60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009850 tx=0x7fcbe002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 -- 172.21.15.70:0/4148866130 shutdown_connections 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 --2- 172.21.15.70:0/4148866130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 --2- 172.21.15.70:0/4148866130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe40b0320 0x7fcbe40c9e60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 -- 172.21.15.70:0/4148866130 >> 172.21.15.70:0/4148866130 conn(0x7fcbe40ce830 msgr2=0x7fcbe40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 -- 172.21.15.70:0/4148866130 shutdown_connections 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.315+0000 7fcbea098640 1 -- 172.21.15.70:0/4148866130 wait complete. 2023-08-25T09:39:54.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.317+0000 7fcbea098640 1 Processor -- start 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.317+0000 7fcbea098640 1 -- start start 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbea098640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbe40b0320 0x7fcbe4154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbea098640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbea098640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 0x7fcbe40c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbea098640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe40c6e70 con 0x7fcbe415a920 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbea098640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe40c7cd0 con 0x7fcbe40b0320 2023-08-25T09:39:54.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbea098640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe40ab710 con 0x7fcbe40b1ce0 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbe9897640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 0x7fcbe40c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbe9897640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 0x7fcbe40c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54854/0 (socket says 172.21.15.70:54854) 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.318+0000 7fcbe9897640 1 -- 172.21.15.70:0/1878438357 learned_addr learned my addr 172.21.15.70:0/1878438357 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9096640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbe40b0320 0x7fcbe4154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe8895640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9897640 1 -- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 msgr2=0x7fcbe40c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9897640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40c2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9897640 1 -- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbe40b0320 msgr2=0x7fcbe4154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9897640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbe40b0320 0x7fcbe4154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9897640 1 -- 172.21.15.70:0/1878438357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe0009460 con 0x7fcbe415a920 2023-08-25T09:39:54.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbe9897640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 0x7fcbe40c2b70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00b9a0 tx=0x7fcbdc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00cba0 con 0x7fcbe415a920 2023-08-25T09:39:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc00cd20 con 0x7fcbe415a920 2023-08-25T09:39:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe40c70c0 con 0x7fcbe415a920 2023-08-25T09:39:54.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.319+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc014680 con 0x7fcbe415a920 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.320+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe40c7680 con 0x7fcbe415a920 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.321+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe4061460 con 0x7fcbe415a920 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.322+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcbdc007370 con 0x7fcbe415a920 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.323+0000 7fcbd5ffb640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbc806a050 0x7fcbc806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.323+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbdc088460 con 0x7fcbe415a920 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.323+0000 7fcbe9096640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbc806a050 0x7fcbc806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.324+0000 7fcbe9096640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbc806a050 0x7fcbc806c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0035000 tx=0x7fcbe0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.327+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcbdc053be0 con 0x7fcbe415a920 2023-08-25T09:39:54.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.516+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbe40c7e10 con 0x7fcbe415a920 2023-08-25T09:39:54.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.516+0000 7fcbd5ffb640 1 -- 172.21.15.70:0/1878438357 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbdc057200 con 0x7fcbe415a920 2023-08-25T09:39:54.518 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:54.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbc806a050 msgr2=0x7fcbc806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbc806a050 0x7fcbc806c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0035000 tx=0x7fcbe0008030 comp rx=0 tx=0).stop 2023-08-25T09:39:54.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 msgr2=0x7fcbe40c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 0x7fcbe40c2b70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00b9a0 tx=0x7fcbdc00be60 comp rx=0 tx=0).stop 2023-08-25T09:39:54.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 shutdown_connections 2023-08-25T09:39:54.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbc806a050 0x7fcbc806c510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcbe415a920 0x7fcbe40c2b70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcbe40b1ce0 0x7fcbe40c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 --2- 172.21.15.70:0/1878438357 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcbe40b0320 0x7fcbe4154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 >> 172.21.15.70:0/1878438357 conn(0x7fcbe40ce830 msgr2=0x7fcbe4158540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 shutdown_connections 2023-08-25T09:39:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.519+0000 7fcbea098640 1 -- 172.21.15.70:0/1878438357 wait complete. 2023-08-25T09:39:54.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:54.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.738+0000 7fc9c0045640 1 Processor -- start 2023-08-25T09:39:54.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.738+0000 7fc9c0045640 1 -- start start 2023-08-25T09:39:54.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.739+0000 7fc9c0045640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.739+0000 7fc9c0045640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc9b8069f60 0x7fc9b80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.739+0000 7fc9c0045640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b80cc5f0 con 0x7fc9b80cdc20 2023-08-25T09:39:54.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.739+0000 7fc9c0045640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b8070650 con 0x7fc9b8069f60 2023-08-25T09:39:54.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.739+0000 7fc9c0045640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b8070930 con 0x7fc9b815ae50 2023-08-25T09:39:54.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.740+0000 7fc9bdde1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.740+0000 7fc9bdde1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:56322/0 (socket says 172.21.15.70:56322) 2023-08-25T09:39:54.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.740+0000 7fc9bdde1640 1 -- 172.21.15.70:0/3554971370 learned_addr learned my addr 172.21.15.70:0/3554971370 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:54.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.740+0000 7fc9be5e2640 1 --2- 172.21.15.70:0/3554971370 >> v2:172.21.15.155:3300/0 conn(0x7fc9b8069f60 0x7fc9b80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.741+0000 7fc9bdde1640 1 -- 172.21.15.70:0/3554971370 >> v2:172.21.15.155:3300/0 conn(0x7fc9b8069f60 msgr2=0x7fc9b80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:54.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.741+0000 7fc9bdde1640 1 --2- 172.21.15.70:0/3554971370 >> v2:172.21.15.155:3300/0 conn(0x7fc9b8069f60 0x7fc9b80ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.741+0000 7fc9bcddf640 1 -- 172.21.15.70:0/3554971370 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218027678 0 0) 0x7fc9b80cc5f0 con 0x7fc9b80cdc20 2023-08-25T09:39:54.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.741+0000 7fc9bdde1640 1 -- 172.21.15.70:0/3554971370 >> v1:172.21.15.155:6789/0 conn(0x7fc9b80cdc20 legacy=0x7fc9b8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.741+0000 7fc9bdde1640 1 -- 172.21.15.70:0/3554971370 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8070c10 con 0x7fc9b815ae50 2023-08-25T09:39:54.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.742+0000 7fc9bdde1640 1 --2- 172.21.15.70:0/3554971370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80c2110 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009870 tx=0x7fc9b402ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b996b561c7d99b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:54.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.742+0000 7fc9bcddf640 1 -- 172.21.15.70:0/3554971370 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b402f830 con 0x7fc9b815ae50 2023-08-25T09:39:54.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.742+0000 7fc9bcddf640 1 -- 172.21.15.70:0/3554971370 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b402f990 con 0x7fc9b815ae50 2023-08-25T09:39:54.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.742+0000 7fc9bcddf640 1 -- 172.21.15.70:0/3554971370 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b402fd20 con 0x7fc9b815ae50 2023-08-25T09:39:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 -- 172.21.15.70:0/3554971370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 msgr2=0x7fc9b80c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 --2- 172.21.15.70:0/3554971370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80c2110 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4009870 tx=0x7fc9b402ba10 comp rx=0 tx=0).stop 2023-08-25T09:39:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 -- 172.21.15.70:0/3554971370 shutdown_connections 2023-08-25T09:39:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 --2- 172.21.15.70:0/3554971370 >> v2:172.21.15.155:3300/0 conn(0x7fc9b8069f60 0x7fc9b80ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 --2- 172.21.15.70:0/3554971370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80c2110 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 -- 172.21.15.70:0/3554971370 >> 172.21.15.70:0/3554971370 conn(0x7fc9b80cb0d0 msgr2=0x7fc9b80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 -- 172.21.15.70:0/3554971370 shutdown_connections 2023-08-25T09:39:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.743+0000 7fc9c0045640 1 -- 172.21.15.70:0/3554971370 wait complete. 2023-08-25T09:39:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.745+0000 7fc9c0045640 1 Processor -- start 2023-08-25T09:39:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.745+0000 7fc9c0045640 1 -- start start 2023-08-25T09:39:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9c0045640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 0x7fc9b81f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9c0045640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9b80cdc20 0x7fc9b815e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9c0045640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9c0045640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b815c5f0 con 0x7fc9b8069f60 2023-08-25T09:39:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9c0045640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b815c3f0 con 0x7fc9b80cdc20 2023-08-25T09:39:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9c0045640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b815e760 con 0x7fc9b815ae50 2023-08-25T09:39:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9bdde1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 0x7fc9b81f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9bdde1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 0x7fc9b81f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54870/0 (socket says 172.21.15.70:54870) 2023-08-25T09:39:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.746+0000 7fc9bdde1640 1 -- 172.21.15.70:0/812019625 learned_addr learned my addr 172.21.15.70:0/812019625 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9bdde1640 1 -- 172.21.15.70:0/812019625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 msgr2=0x7fc9b80ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9bdde1640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9bdde1640 1 -- 172.21.15.70:0/812019625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9b80cdc20 msgr2=0x7fc9b815e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9bdde1640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9b80cdc20 0x7fc9b815e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9bdde1640 1 -- 172.21.15.70:0/812019625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b40094d0 con 0x7fc9b8069f60 2023-08-25T09:39:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9bdde1640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 0x7fc9b81f22d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40090b0 tx=0x7fc9b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:54.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b403b040 con 0x7fc9b8069f60 2023-08-25T09:39:54.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b4037bc0 con 0x7fc9b8069f60 2023-08-25T09:39:54.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4004030 con 0x7fc9b8069f60 2023-08-25T09:39:54.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.747+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b81f2920 con 0x7fc9b8069f60 2023-08-25T09:39:54.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.748+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b81f2ee0 con 0x7fc9b8069f60 2023-08-25T09:39:54.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.749+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b8066fb0 con 0x7fc9b8069f60 2023-08-25T09:39:54.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.750+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9b4004190 con 0x7fc9b8069f60 2023-08-25T09:39:54.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.751+0000 7fc9ae7fc640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99c06a050 0x7fc99c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:54.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.751+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9b40ab430 con 0x7fc9b8069f60 2023-08-25T09:39:54.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.751+0000 7fc9bd5e0640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99c06a050 0x7fc99c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:54.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.751+0000 7fc9bd5e0640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99c06a050 0x7fc99c06c510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80cac80 tx=0x7fc9a8007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:54.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.755+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9b4076ba0 con 0x7fc9b8069f60 2023-08-25T09:39:54.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.946+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc9b80c30f0 con 0x7fc9b8069f60 2023-08-25T09:39:54.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.947+0000 7fc9ae7fc640 1 -- 172.21.15.70:0/812019625 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc9b407a1c0 con 0x7fc9b8069f60 2023-08-25T09:39:54.953 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:54.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99c06a050 msgr2=0x7fc99c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99c06a050 0x7fc99c06c510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80cac80 tx=0x7fc9a8007910 comp rx=0 tx=0).stop 2023-08-25T09:39:54.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 msgr2=0x7fc9b81f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:54.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 0x7fc9b81f22d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc9b40090b0 tx=0x7fc9b40046c0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 shutdown_connections 2023-08-25T09:39:54.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99c06a050 0x7fc99c06c510 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9b815ae50 0x7fc9b80ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9b80cdc20 0x7fc9b815e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 --2- 172.21.15.70:0/812019625 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9b8069f60 0x7fc9b81f22d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:54.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 >> 172.21.15.70:0/812019625 conn(0x7fc9b80cb0d0 msgr2=0x7fc9b80aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:54.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 shutdown_connections 2023-08-25T09:39:54.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:54.950+0000 7fc9c0045640 1 -- 172.21.15.70:0/812019625 wait complete. 2023-08-25T09:39:54.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:55.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.191+0000 7eff8dd6d640 1 Processor -- start 2023-08-25T09:39:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.191+0000 7eff8dd6d640 1 -- start start 2023-08-25T09:39:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8dd6d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160660 0x7eff8806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8dd6d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7eff88160ac0 0x7eff8806ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8dd6d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff88161040 con 0x7eff880cb880 2023-08-25T09:39:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8dd6d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff88161600 con 0x7eff88160ac0 2023-08-25T09:39:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8dd6d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8815d070 con 0x7eff88160660 2023-08-25T09:39:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8d56c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7eff88160ac0 0x7eff8806ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8d56c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7eff88160ac0 0x7eff8806ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54872/0 (socket says 172.21.15.70:54872) 2023-08-25T09:39:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff8d56c640 1 -- 172.21.15.70:0/1183715734 learned_addr learned my addr 172.21.15.70:0/1183715734 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.193+0000 7eff87fff640 1 --2- 172.21.15.70:0/1183715734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160660 0x7eff8806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.194+0000 7eff8d56c640 1 -- 172.21.15.70:0/1183715734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160660 msgr2=0x7eff8806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.194+0000 7eff8d56c640 1 --2- 172.21.15.70:0/1183715734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160660 0x7eff8806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.194+0000 7eff8d56c640 1 -- 172.21.15.70:0/1183715734 >> v1:172.21.15.155:6789/0 conn(0x7eff880cb880 legacy=0x7eff8815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.194+0000 7eff8d56c640 1 -- 172.21.15.70:0/1183715734 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff88161320 con 0x7eff88160ac0 2023-08-25T09:39:55.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.195+0000 7eff8d56c640 1 --2- 172.21.15.70:0/1183715734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160ac0 0x7eff8806ec60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff80009e50 tx=0x7eff8002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124f5612e3faeeed server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.195+0000 7eff877fe640 1 -- 172.21.15.70:0/1183715734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8002ca70 con 0x7eff88160ac0 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.195+0000 7eff877fe640 1 -- 172.21.15.70:0/1183715734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8002cbd0 con 0x7eff88160ac0 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.195+0000 7eff877fe640 1 -- 172.21.15.70:0/1183715734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff800343f0 con 0x7eff88160ac0 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 -- 172.21.15.70:0/1183715734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160ac0 msgr2=0x7eff8806ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/1183715734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160ac0 0x7eff8806ec60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7eff80009e50 tx=0x7eff8002bed0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 -- 172.21.15.70:0/1183715734 shutdown_connections 2023-08-25T09:39:55.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/1183715734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160ac0 0x7eff8806ec60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/1183715734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160660 0x7eff8806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 -- 172.21.15.70:0/1183715734 >> 172.21.15.70:0/1183715734 conn(0x7eff880d06b0 msgr2=0x7eff880d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:55.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 -- 172.21.15.70:0/1183715734 shutdown_connections 2023-08-25T09:39:55.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.196+0000 7eff8dd6d640 1 -- 172.21.15.70:0/1183715734 wait complete. 2023-08-25T09:39:55.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.198+0000 7eff8dd6d640 1 Processor -- start 2023-08-25T09:39:55.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.199+0000 7eff8dd6d640 1 -- start start 2023-08-25T09:39:55.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff8dd6d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff880cb880 0x7eff88156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff8dd6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 0x7eff8806b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff8dd6d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160ac0 0x7eff880aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff8dd6d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8806c230 con 0x7eff88160660 2023-08-25T09:39:55.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff8dd6d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8806bc70 con 0x7eff880cb880 2023-08-25T09:39:55.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff8dd6d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8806bf50 con 0x7eff88160ac0 2023-08-25T09:39:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff87fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 0x7eff8806b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff87fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 0x7eff8806b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54884/0 (socket says 172.21.15.70:54884) 2023-08-25T09:39:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.200+0000 7eff87fff640 1 -- 172.21.15.70:0/91844261 learned_addr learned my addr 172.21.15.70:0/91844261 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff87fff640 1 -- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160ac0 msgr2=0x7eff880aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff87fff640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160ac0 0x7eff880aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff87fff640 1 -- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff880cb880 msgr2=0x7eff88156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff8cd6b640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff880cb880 0x7eff88156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff87fff640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff880cb880 0x7eff88156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff87fff640 1 -- 172.21.15.70:0/91844261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80009b30 con 0x7eff88160660 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff8cd6b640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff880cb880 0x7eff88156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff87fff640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 0x7eff8806b2b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff7400d800 tx=0x7eff7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400b6c0 con 0x7eff88160660 2023-08-25T09:39:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7400b820 con 0x7eff88160660 2023-08-25T09:39:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff880b0d20 con 0x7eff88160660 2023-08-25T09:39:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400bbb0 con 0x7eff88160660 2023-08-25T09:39:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.201+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff88156e90 con 0x7eff88160660 2023-08-25T09:39:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.202+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff88061460 con 0x7eff88160660 2023-08-25T09:39:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.203+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7eff7400bd10 con 0x7eff88160660 2023-08-25T09:39:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.204+0000 7eff857fa640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff6c06a050 0x7eff6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.205+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff74088ac0 con 0x7eff88160660 2023-08-25T09:39:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.205+0000 7eff8cd6b640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff6c06a050 0x7eff6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.205+0000 7eff8cd6b640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff6c06a050 0x7eff6c06c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff7c003eb0 tx=0x7eff7c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.208+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7eff74057c40 con 0x7eff88160660 2023-08-25T09:39:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.398+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff880ad070 con 0x7eff88160660 2023-08-25T09:39:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.399+0000 7eff857fa640 1 -- 172.21.15.70:0/91844261 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff740577d0 con 0x7eff88160660 2023-08-25T09:39:55.401 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff6c06a050 msgr2=0x7eff6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff6c06a050 0x7eff6c06c510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff7c003eb0 tx=0x7eff7c009040 comp rx=0 tx=0).stop 2023-08-25T09:39:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 msgr2=0x7eff8806b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 0x7eff8806b2b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7eff7400d800 tx=0x7eff7400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 shutdown_connections 2023-08-25T09:39:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff6c06a050 0x7eff6c06c510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff88160ac0 0x7eff880aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff88160660 0x7eff8806b2b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 --2- 172.21.15.70:0/91844261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff880cb880 0x7eff88156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 >> 172.21.15.70:0/91844261 conn(0x7eff880d06b0 msgr2=0x7eff880b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 shutdown_connections 2023-08-25T09:39:55.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.402+0000 7eff8dd6d640 1 -- 172.21.15.70:0/91844261 wait complete. 2023-08-25T09:39:55.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:55.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.637+0000 7f5d37142640 1 Processor -- start 2023-08-25T09:39:55.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.637+0000 7f5d37142640 1 -- start start 2023-08-25T09:39:55.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.638+0000 7f5d37142640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d300c2110 0x7f5d30069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.638+0000 7f5d37142640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d300c2550 con 0x7f5d30069d90 2023-08-25T09:39:55.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.638+0000 7f5d37142640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d300ce330 con 0x7f5d3015ae50 2023-08-25T09:39:55.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.638+0000 7f5d37142640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3006a200 con 0x7f5d300c2110 2023-08-25T09:39:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d2ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d300c2110 0x7f5d30069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d356df640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5d30069d90 0x7f5d300ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41180/0 (socket says 172.21.15.70:41180) 2023-08-25T09:39:55.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d356df640 1 -- 172.21.15.70:0/432485333 learned_addr learned my addr 172.21.15.70:0/432485333 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d2ffff640 1 -- 172.21.15.70:0/432485333 >> v1:172.21.15.155:6789/0 conn(0x7f5d3015ae50 legacy=0x7f5d300cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d2ffff640 1 -- 172.21.15.70:0/432485333 >> v1:172.21.15.70:6789/0 conn(0x7f5d30069d90 legacy=0x7f5d300ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d2ffff640 1 -- 172.21.15.70:0/432485333 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d300cc5f0 con 0x7f5d300c2110 2023-08-25T09:39:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.639+0000 7f5d2f7fe640 1 -- 172.21.15.70:0/432485333 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47202337 0 0) 0x7f5d300c2550 con 0x7f5d30069d90 2023-08-25T09:39:55.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.640+0000 7f5d2ffff640 1 --2- 172.21.15.70:0/432485333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d300c2110 0x7f5d30069850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0097a0 tx=0x7f5d1c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf5d940ed40e566 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:55.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.641+0000 7f5d2f7fe640 1 -- 172.21.15.70:0/432485333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c030610 con 0x7f5d300c2110 2023-08-25T09:39:55.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.641+0000 7f5d2f7fe640 1 -- 172.21.15.70:0/432485333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c030770 con 0x7f5d300c2110 2023-08-25T09:39:55.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.641+0000 7f5d2f7fe640 1 -- 172.21.15.70:0/432485333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c030b00 con 0x7f5d300c2110 2023-08-25T09:39:55.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.641+0000 7f5d37142640 1 -- 172.21.15.70:0/432485333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d300c2110 msgr2=0x7f5d30069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.641+0000 7f5d37142640 1 --2- 172.21.15.70:0/432485333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d300c2110 0x7f5d30069850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c0097a0 tx=0x7f5d1c02b960 comp rx=0 tx=0).stop 2023-08-25T09:39:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.642+0000 7f5d37142640 1 -- 172.21.15.70:0/432485333 shutdown_connections 2023-08-25T09:39:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.642+0000 7f5d37142640 1 --2- 172.21.15.70:0/432485333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d300c2110 0x7f5d30069850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.642+0000 7f5d37142640 1 -- 172.21.15.70:0/432485333 >> 172.21.15.70:0/432485333 conn(0x7f5d300cb0d0 msgr2=0x7f5d300cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.642+0000 7f5d37142640 1 -- 172.21.15.70:0/432485333 shutdown_connections 2023-08-25T09:39:55.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.642+0000 7f5d37142640 1 -- 172.21.15.70:0/432485333 wait complete. 2023-08-25T09:39:55.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.643+0000 7f5d37142640 1 Processor -- start 2023-08-25T09:39:55.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.644+0000 7f5d37142640 1 -- start start 2023-08-25T09:39:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.644+0000 7f5d37142640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 0x7f5d3014ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.644+0000 7f5d37142640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d300c2110 0x7f5d30151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.644+0000 7f5d37142640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d3015ae50 0x7f5d3014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d37142640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3014c7b0 con 0x7f5d30069d90 2023-08-25T09:39:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d37142640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3014bba0 con 0x7f5d3015ae50 2023-08-25T09:39:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d37142640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3014be80 con 0x7f5d300c2110 2023-08-25T09:39:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 0x7f5d3014ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 0x7f5d3014ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54914/0 (socket says 172.21.15.70:54914) 2023-08-25T09:39:55.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 -- 172.21.15.70:0/1353958196 learned_addr learned my addr 172.21.15.70:0/1353958196 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 -- 172.21.15.70:0/1353958196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d300c2110 msgr2=0x7f5d30151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d300c2110 0x7f5d30151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 -- 172.21.15.70:0/1353958196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d3015ae50 msgr2=0x7f5d3014ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d3015ae50 0x7f5d3014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 -- 172.21.15.70:0/1353958196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c009400 con 0x7f5d30069d90 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.645+0000 7f5d34ede640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 0x7f5d3014ad30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b3f0 tx=0x7f5d2400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.646+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24011830 con 0x7f5d30069d90 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.646+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d24011990 con 0x7f5d30069d90 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.646+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24011d40 con 0x7f5d30069d90 2023-08-25T09:39:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.646+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3014c160 con 0x7f5d30069d90 2023-08-25T09:39:55.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.646+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3014b270 con 0x7f5d30069d90 2023-08-25T09:39:55.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.647+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3014b460 con 0x7f5d30069d90 2023-08-25T09:39:55.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.648+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d2401a400 con 0x7f5d30069d90 2023-08-25T09:39:55.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.649+0000 7f5d2d7fa640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d1406a050 0x7f5d1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:55.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.649+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d24088550 con 0x7f5d30069d90 2023-08-25T09:39:55.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.649+0000 7f5d2ffff640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d1406a050 0x7f5d1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:55.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.650+0000 7f5d2ffff640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d1406a050 0x7f5d1406c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c036000 tx=0x7f5d1c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.654+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d24053b10 con 0x7f5d30069d90 2023-08-25T09:39:55.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.844+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d30066fb0 con 0x7f5d30069d90 2023-08-25T09:39:55.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.845+0000 7f5d2d7fa640 1 -- 172.21.15.70:0/1353958196 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d24057130 con 0x7f5d30069d90 2023-08-25T09:39:55.847 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.847+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d1406a050 msgr2=0x7f5d1406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.847+0000 7f5d37142640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d1406a050 0x7f5d1406c510 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c036000 tx=0x7f5d1c03b040 comp rx=0 tx=0).stop 2023-08-25T09:39:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.847+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 msgr2=0x7f5d3014ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.847+0000 7f5d37142640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 0x7f5d3014ad30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b3f0 tx=0x7f5d2400b8b0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.847+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 shutdown_connections 2023-08-25T09:39:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d1406a050 0x7f5d1406c510 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d3015ae50 0x7f5d3014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d300c2110 0x7f5d30151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 --2- 172.21.15.70:0/1353958196 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d30069d90 0x7f5d3014ad30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 >> 172.21.15.70:0/1353958196 conn(0x7f5d300cb0d0 msgr2=0x7f5d300c1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 shutdown_connections 2023-08-25T09:39:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:55.848+0000 7f5d37142640 1 -- 172.21.15.70:0/1353958196 wait complete. 2023-08-25T09:39:55.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:39:56.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.080+0000 7fb2abcec640 1 Processor -- start 2023-08-25T09:39:56.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.081+0000 7fb2abcec640 1 -- start start 2023-08-25T09:39:56.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.082+0000 7fb2abcec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb2a415cbe0 0x7fb2a4160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.082+0000 7fb2abcec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2a4160ec0 0x7fb2a40cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.082+0000 7fb2abcec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a415cfc0 0x7fb2a40cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.082+0000 7fb2abcec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a40b5cb0 con 0x7fb2a4160ec0 2023-08-25T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.082+0000 7fb2abcec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a40d0c00 con 0x7fb2a415cbe0 2023-08-25T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.082+0000 7fb2abcec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a41613d0 con 0x7fb2a415cfc0 2023-08-25T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2aa289640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a415cfc0 0x7fb2a40cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9a88640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb2a415cbe0 0x7fb2a4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9287640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2a4160ec0 0x7fb2a40cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2aa289640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a415cfc0 0x7fb2a40cde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:56354/0 (socket says 172.21.15.70:56354) 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2aa289640 1 -- 172.21.15.70:0/60548578 learned_addr learned my addr 172.21.15.70:0/60548578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9287640 1 -- 172.21.15.70:0/60548578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a415cfc0 msgr2=0x7fb2a40cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9287640 1 --2- 172.21.15.70:0/60548578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a415cfc0 0x7fb2a40cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9287640 1 -- 172.21.15.70:0/60548578 >> v2:172.21.15.155:3300/0 conn(0x7fb2a415cbe0 msgr2=0x7fb2a4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9287640 1 --2- 172.21.15.70:0/60548578 >> v2:172.21.15.155:3300/0 conn(0x7fb2a415cbe0 0x7fb2a4160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.083+0000 7fb2a9287640 1 -- 172.21.15.70:0/60548578 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a41616b0 con 0x7fb2a4160ec0 2023-08-25T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.085+0000 7fb2a9287640 1 --2- 172.21.15.70:0/60548578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a4160ec0 0x7fb2a40cd8e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40b5610 tx=0x7fb29802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76909d7c46b370 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.085+0000 7fb2a8a86640 1 -- 172.21.15.70:0/60548578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29802e070 con 0x7fb2a4160ec0 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.085+0000 7fb2a8a86640 1 -- 172.21.15.70:0/60548578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2980315a0 con 0x7fb2a4160ec0 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.085+0000 7fb2a8a86640 1 -- 172.21.15.70:0/60548578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298031930 con 0x7fb2a4160ec0 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 -- 172.21.15.70:0/60548578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a4160ec0 msgr2=0x7fb2a40cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 --2- 172.21.15.70:0/60548578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a4160ec0 0x7fb2a40cd8e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40b5610 tx=0x7fb29802b840 comp rx=0 tx=0).stop 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 -- 172.21.15.70:0/60548578 shutdown_connections 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 --2- 172.21.15.70:0/60548578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a415cfc0 0x7fb2a40cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 --2- 172.21.15.70:0/60548578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a4160ec0 0x7fb2a40cd8e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 --2- 172.21.15.70:0/60548578 >> v2:172.21.15.155:3300/0 conn(0x7fb2a415cbe0 0x7fb2a4160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 -- 172.21.15.70:0/60548578 >> 172.21.15.70:0/60548578 conn(0x7fb2a4069850 msgr2=0x7fb2a4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 -- 172.21.15.70:0/60548578 shutdown_connections 2023-08-25T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.086+0000 7fb2abcec640 1 -- 172.21.15.70:0/60548578 wait complete. 2023-08-25T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.088+0000 7fb2abcec640 1 Processor -- start 2023-08-25T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.088+0000 7fb2abcec640 1 -- start start 2023-08-25T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2abcec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 0x7fb2a40b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2abcec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a415cfc0 0x7fb2a415fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2abcec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a4160ec0 0x7fb2a415ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2abcec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a415d480 con 0x7fb2a415cbe0 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2abcec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a415d760 con 0x7fb2a415cfc0 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2abcec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a40cb0b0 con 0x7fb2a4160ec0 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 0x7fb2a40b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 0x7fb2a40b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54936/0 (socket says 172.21.15.70:54936) 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 -- 172.21.15.70:0/2072905656 learned_addr learned my addr 172.21.15.70:0/2072905656 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 -- 172.21.15.70:0/2072905656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a4160ec0 msgr2=0x7fb2a415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a4160ec0 0x7fb2a415ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 -- 172.21.15.70:0/2072905656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a415cfc0 msgr2=0x7fb2a415fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a415cfc0 0x7fb2a415fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.089+0000 7fb2a9a88640 1 -- 172.21.15.70:0/2072905656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb298009400 con 0x7fb2a415cbe0 2023-08-25T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.090+0000 7fb2a9a88640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 0x7fb2a40b4b30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb29400eed0 tx=0x7fb29400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.090+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294010040 con 0x7fb2a415cbe0 2023-08-25T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.090+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29400bd90 con 0x7fb2a415cbe0 2023-08-25T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.090+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a40cb390 con 0x7fb2a415cbe0 2023-08-25T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.090+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294014620 con 0x7fb2a415cbe0 2023-08-25T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.090+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a4157fa0 con 0x7fb2a415cbe0 2023-08-25T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.091+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a40cb8b0 con 0x7fb2a415cbe0 2023-08-25T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.092+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb294014990 con 0x7fb2a415cbe0 2023-08-25T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.093+0000 7fb2a27fc640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb29006a050 0x7fb29006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.093+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb294088f30 con 0x7fb2a415cbe0 2023-08-25T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.093+0000 7fb2a9287640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb29006a050 0x7fb29006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.094+0000 7fb2a9287640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb29006a050 0x7fb29006c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb298036000 tx=0x7fb298030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.097+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb294054620 con 0x7fb2a415cbe0 2023-08-25T09:39:56.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.302+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb2a404e420 con 0x7fb2a415cbe0 2023-08-25T09:39:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.303+0000 7fb2a27fc640 1 -- 172.21.15.70:0/2072905656 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb294057c40 con 0x7fb2a415cbe0 2023-08-25T09:39:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:39:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb29006a050 msgr2=0x7fb29006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb29006a050 0x7fb29006c510 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb298036000 tx=0x7fb298030000 comp rx=0 tx=0).stop 2023-08-25T09:39:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 msgr2=0x7fb2a40b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 0x7fb2a40b4b30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb29400eed0 tx=0x7fb29400b480 comp rx=0 tx=0).stop 2023-08-25T09:39:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 shutdown_connections 2023-08-25T09:39:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb29006a050 0x7fb29006c510 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2a4160ec0 0x7fb2a415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2a415cfc0 0x7fb2a415fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 --2- 172.21.15.70:0/2072905656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2a415cbe0 0x7fb2a40b4b30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 >> 172.21.15.70:0/2072905656 conn(0x7fb2a4069850 msgr2=0x7fb2a415bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:56.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 shutdown_connections 2023-08-25T09:39:56.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.306+0000 7fb2abcec640 1 -- 172.21.15.70:0/2072905656 wait complete. 2023-08-25T09:39:56.327 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:39:56.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:56.329 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:39:56.436 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:39:56.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:39:56.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:39:56.437 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:39:56.503 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:39:56.504 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:56.567 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:39:56.568 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:56.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.834+0000 7fed56634640 1 Processor -- start 2023-08-25T09:39:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.835+0000 7fed56634640 1 -- start start 2023-08-25T09:39:56.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed56634640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed500cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed56634640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed501606c0 0x7fed500cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed56634640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed5015cfb0 con 0x7fed500cdeb0 2023-08-25T09:39:56.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed56634640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed500cd4e0 con 0x7fed501606c0 2023-08-25T09:39:56.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed56634640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed50160f10 con 0x7fed5015cbd0 2023-08-25T09:39:56.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed501606c0 0x7fed500cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fed501606c0 0x7fed500cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54950/0 (socket says 172.21.15.70:54950) 2023-08-25T09:39:56.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed47fff640 1 -- 172.21.15.70:0/3827564300 learned_addr learned my addr 172.21.15.70:0/3827564300 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:56.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed54bd1640 1 --1- 172.21.15.70:0/3827564300 >> v1:172.21.15.70:6789/0 conn(0x7fed500cdeb0 0x7fed500d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41182/0 (socket says 172.21.15.70:41182) 2023-08-25T09:39:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.836+0000 7fed4ffff640 1 --2- 172.21.15.70:0/3827564300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed500cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.837+0000 7fed47fff640 1 -- 172.21.15.70:0/3827564300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 msgr2=0x7fed500cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.837+0000 7fed47fff640 1 --2- 172.21.15.70:0/3827564300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed500cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.837+0000 7fed47fff640 1 -- 172.21.15.70:0/3827564300 >> v1:172.21.15.70:6789/0 conn(0x7fed500cdeb0 legacy=0x7fed500d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.837+0000 7fed47fff640 1 -- 172.21.15.70:0/3827564300 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed501611f0 con 0x7fed501606c0 2023-08-25T09:39:56.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.837+0000 7fed4f7fe640 1 -- 172.21.15.70:0/3827564300 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660265718 0 0) 0x7fed5015cfb0 con 0x7fed500cdeb0 2023-08-25T09:39:56.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.838+0000 7fed47fff640 1 --2- 172.21.15.70:0/3827564300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed501606c0 0x7fed500cd970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed38009870 tx=0x7fed3802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6805dca7153a6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:56.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.838+0000 7fed4f7fe640 1 -- 172.21.15.70:0/3827564300 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38030840 con 0x7fed501606c0 2023-08-25T09:39:56.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.838+0000 7fed4f7fe640 1 -- 172.21.15.70:0/3827564300 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed380309a0 con 0x7fed501606c0 2023-08-25T09:39:56.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.838+0000 7fed4f7fe640 1 -- 172.21.15.70:0/3827564300 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed38030d30 con 0x7fed501606c0 2023-08-25T09:39:56.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 -- 172.21.15.70:0/3827564300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed501606c0 msgr2=0x7fed500cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 --2- 172.21.15.70:0/3827564300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed501606c0 0x7fed500cd970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fed38009870 tx=0x7fed3802b980 comp rx=0 tx=0).stop 2023-08-25T09:39:56.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 -- 172.21.15.70:0/3827564300 shutdown_connections 2023-08-25T09:39:56.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 --2- 172.21.15.70:0/3827564300 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed501606c0 0x7fed500cd970 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 --2- 172.21.15.70:0/3827564300 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed500cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 -- 172.21.15.70:0/3827564300 >> 172.21.15.70:0/3827564300 conn(0x7fed50070260 msgr2=0x7fed50070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:56.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 -- 172.21.15.70:0/3827564300 shutdown_connections 2023-08-25T09:39:56.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.839+0000 7fed56634640 1 -- 172.21.15.70:0/3827564300 wait complete. 2023-08-25T09:39:56.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.841+0000 7fed56634640 1 Processor -- start 2023-08-25T09:39:56.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.841+0000 7fed56634640 1 -- start start 2023-08-25T09:39:56.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed56634640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 0x7fed501f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed56634640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed50155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed56634640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed501606c0 0x7fed500b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed56634640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed50160140 con 0x7fed500cdeb0 2023-08-25T09:39:56.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed56634640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed501f29f0 con 0x7fed501606c0 2023-08-25T09:39:56.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed56634640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed501f2c90 con 0x7fed5015cbd0 2023-08-25T09:39:56.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 0x7fed501f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 0x7fed501f24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54962/0 (socket says 172.21.15.70:54962) 2023-08-25T09:39:56.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 -- 172.21.15.70:0/3939493397 learned_addr learned my addr 172.21.15.70:0/3939493397 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:56.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 -- 172.21.15.70:0/3939493397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 msgr2=0x7fed50155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed50155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 -- 172.21.15.70:0/3939493397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed501606c0 msgr2=0x7fed500b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:56.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed501606c0 0x7fed500b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.842+0000 7fed4ffff640 1 -- 172.21.15.70:0/3939493397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed380094d0 con 0x7fed500cdeb0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.843+0000 7fed4ffff640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 0x7fed501f24b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed4000d800 tx=0x7fed4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.843+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000b6c0 con 0x7fed500cdeb0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.843+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4000b820 con 0x7fed500cdeb0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.843+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4000bbb0 con 0x7fed500cdeb0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.843+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed501f2f70 con 0x7fed500cdeb0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.843+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5015e160 con 0x7fed500cdeb0 2023-08-25T09:39:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.844+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5015e320 con 0x7fed500cdeb0 2023-08-25T09:39:56.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.845+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fed4000bd10 con 0x7fed500cdeb0 2023-08-25T09:39:56.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.846+0000 7fed4d7fa640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fed3006a050 0x7fed3006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:56.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.846+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed40088b30 con 0x7fed500cdeb0 2023-08-25T09:39:56.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.846+0000 7fed47fff640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fed3006a050 0x7fed3006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:56.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.847+0000 7fed47fff640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fed3006a050 0x7fed3006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed38036000 tx=0x7fed3803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:56.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:56.850+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fed40054210 con 0x7fed500cdeb0 2023-08-25T09:39:57.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.040+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed5004e420 con 0x7fed500cdeb0 2023-08-25T09:39:57.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.042+0000 7fed4d7fa640 1 -- 172.21.15.70:0/3939493397 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fed40057830 con 0x7fed500cdeb0 2023-08-25T09:39:57.044 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:57.044 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fed3006a050 msgr2=0x7fed3006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fed3006a050 0x7fed3006c510 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed38036000 tx=0x7fed3803b040 comp rx=0 tx=0).stop 2023-08-25T09:39:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 msgr2=0x7fed501f24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 0x7fed501f24b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fed4000d800 tx=0x7fed4000dcc0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 shutdown_connections 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fed3006a050 0x7fed3006c510 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fed501606c0 0x7fed500b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fed5015cbd0 0x7fed50155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 --2- 172.21.15.70:0/3939493397 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fed500cdeb0 0x7fed501f24b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.044+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 >> 172.21.15.70:0/3939493397 conn(0x7fed50070260 msgr2=0x7fed500b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.045+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 shutdown_connections 2023-08-25T09:39:57.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.045+0000 7fed56634640 1 -- 172.21.15.70:0/3939493397 wait complete. 2023-08-25T09:39:57.064 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:57.066 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:39:57.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.275+0000 7f72b1e06640 1 Processor -- start 2023-08-25T09:39:57.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.275+0000 7f72b1e06640 1 -- start start 2023-08-25T09:39:57.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.277+0000 7f72b1e06640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f72ac0cd660 0x7f72ac0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.277+0000 7f72b1e06640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ac06f8e0 con 0x7f72ac15e240 2023-08-25T09:39:57.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.277+0000 7f72b1e06640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ac06fbc0 con 0x7f72ac159bf0 2023-08-25T09:39:57.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.277+0000 7f72b1e06640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ac0ce030 con 0x7f72ac0cd660 2023-08-25T09:39:57.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.277+0000 7f72aaffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72ac15e240 0x7f72ac070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41186/0 (socket says 172.21.15.70:41186) 2023-08-25T09:39:57.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.277+0000 7f72aaffd640 1 -- 172.21.15.70:0/3756207047 learned_addr learned my addr 172.21.15.70:0/3756207047 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:57.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983909628 0 0) 0x7f72ac06f8e0 con 0x7f72ac15e240 2023-08-25T09:39:57.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72abfff640 1 --2- 172.21.15.70:0/3756207047 >> v2:172.21.15.155:3300/0 conn(0x7f72ac0cd660 0x7f72ac0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7294003520 con 0x7f72ac15e240 2023-08-25T09:39:57.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2441084320 0 0) 0x7f7294003520 con 0x7f72ac15e240 2023-08-25T09:39:57.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 >> v2:172.21.15.155:3300/0 conn(0x7f72ac0cd660 msgr2=0x7f72ac0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 --2- 172.21.15.70:0/3756207047 >> v2:172.21.15.155:3300/0 conn(0x7f72ac0cd660 0x7f72ac0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 >> v1:172.21.15.155:6789/0 conn(0x7f72ac159bf0 legacy=0x7f72ac06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac06fea0 con 0x7f72ac15e240 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f72a0001670 con 0x7f72ac15e240 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 >> v1:172.21.15.70:6789/0 conn(0x7f72ac15e240 legacy=0x7f72ac070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7294003970 0x7f7294003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72940047c0 0x7f7294004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294005180 0x7f729400fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0ce030 con 0x7f72940047c0 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7294010430 con 0x7f7294003970 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.278+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72940105f0 con 0x7f7294005180 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72aaffd640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72940047c0 0x7f7294004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72ab7fe640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7294003970 0x7f7294003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72abfff640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294005180 0x7f729400fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72aaffd640 1 -- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294005180 msgr2=0x7f729400fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:57.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72aaffd640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7294005180 0x7f729400fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72aaffd640 1 -- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7294003970 msgr2=0x7f7294003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:57.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72aaffd640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7294003970 0x7f7294003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72aaffd640 1 -- 172.21.15.70:0/3756207047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294003520 con 0x7f72940047c0 2023-08-25T09:39:57.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.279+0000 7f72abfff640 1 -- 172.21.15.70:0/3756207047 reap_dead start 2023-08-25T09:39:57.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.280+0000 7f72aaffd640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72940047c0 0x7f7294004c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c240 tx=0x7f72940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b1df03f48bd8dd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:57.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.281+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0005ad0 con 0x7f72940047c0 2023-08-25T09:39:57.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.281+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a0005c30 con 0x7f72940047c0 2023-08-25T09:39:57.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.281+0000 7f72aa7fc640 1 -- 172.21.15.70:0/3756207047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a002ca90 con 0x7f72940047c0 2023-08-25T09:39:57.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.281+0000 7f72b1e06640 1 -- 172.21.15.70:0/3756207047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72940047c0 msgr2=0x7f7294004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.281+0000 7f72b1e06640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72940047c0 0x7f7294004c40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c240 tx=0x7f72940057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.282+0000 7f72b1e06640 1 -- 172.21.15.70:0/3756207047 shutdown_connections 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.282+0000 7f72b1e06640 1 --2- 172.21.15.70:0/3756207047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72940047c0 0x7f7294004c40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.282+0000 7f72b1e06640 1 -- 172.21.15.70:0/3756207047 >> 172.21.15.70:0/3756207047 conn(0x7f72ac0cb0d0 msgr2=0x7f72ac0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.282+0000 7f72b1e06640 1 -- 172.21.15.70:0/3756207047 shutdown_connections 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.282+0000 7f72b1e06640 1 -- 172.21.15.70:0/3756207047 wait complete. 2023-08-25T09:39:57.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.283+0000 7f72b1e06640 1 Processor -- start 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.283+0000 7f72b1e06640 1 -- start start 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.284+0000 7f72b1e06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72940047c0 0x7f72ac146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.284+0000 7f72b1e06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 0x7f72ac143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.284+0000 7f72b1e06640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72ac0c8e70 0x7f72ac0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.284+0000 7f72b1e06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac146a50 con 0x7f72ac154ab0 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.284+0000 7f72b1e06640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac0c7e40 con 0x7f72ac0c8e70 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.284+0000 7f72b1e06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ac15b690 con 0x7f72940047c0 2023-08-25T09:39:57.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 0x7f72ac143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 0x7f72ac143060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54986/0 (socket says 172.21.15.70:54986) 2023-08-25T09:39:57.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 -- 172.21.15.70:0/2007568098 learned_addr learned my addr 172.21.15.70:0/2007568098 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:57.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72abfff640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72ac0c8e70 0x7f72ac0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72ab7fe640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72940047c0 0x7f72ac146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 -- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72940047c0 msgr2=0x7f72ac146470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72940047c0 0x7f72ac146470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 -- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72ac0c8e70 msgr2=0x7f72ac0c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72ac0c8e70 0x7f72ac0c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.285+0000 7f72aaffd640 1 -- 172.21.15.70:0/2007568098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac0c8610 con 0x7f72ac154ab0 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.286+0000 7f72aaffd640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 0x7f72ac143060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f72a0001930 tx=0x7f72a002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.286+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a003c040 con 0x7f72ac154ab0 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.286+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a002cc60 con 0x7f72ac154ab0 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.286+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0007070 con 0x7f72ac154ab0 2023-08-25T09:39:57.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.286+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ac0c8050 con 0x7f72ac154ab0 2023-08-25T09:39:57.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.286+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ac15b8a0 con 0x7f72ac154ab0 2023-08-25T09:39:57.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.287+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ac066fb0 con 0x7f72ac154ab0 2023-08-25T09:39:57.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.288+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f72a0032070 con 0x7f72ac154ab0 2023-08-25T09:39:57.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.290+0000 7f7293fff640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f729806a080 0x7f729806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.290+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72a00a9bb0 con 0x7f72ac154ab0 2023-08-25T09:39:57.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.290+0000 7f72ab7fe640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f729806a080 0x7f729806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.290+0000 7f72ab7fe640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f729806a080 0x7f729806c540 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72ac069270 tx=0x7f729c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:57.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.294+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f72a0075200 con 0x7f72ac154ab0 2023-08-25T09:39:57.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.483+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72ac06f0a0 con 0x7f72ac154ab0 2023-08-25T09:39:57.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.484+0000 7f7293fff640 1 -- 172.21.15.70:0/2007568098 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72a0078820 con 0x7f72ac154ab0 2023-08-25T09:39:57.487 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:39:57.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f729806a080 msgr2=0x7f729806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f729806a080 0x7f729806c540 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f72ac069270 tx=0x7f729c009040 comp rx=0 tx=0).stop 2023-08-25T09:39:57.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 msgr2=0x7f72ac143060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 0x7f72ac143060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f72a0001930 tx=0x7f72a002e070 comp rx=0 tx=0).stop 2023-08-25T09:39:57.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 shutdown_connections 2023-08-25T09:39:57.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f729806a080 0x7f729806c540 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72ac0c8e70 0x7f72ac0c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f72ac154ab0 0x7f72ac143060 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 --2- 172.21.15.70:0/2007568098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f72940047c0 0x7f72ac146470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 >> 172.21.15.70:0/2007568098 conn(0x7f72ac0cb0d0 msgr2=0x7f72ac0ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:57.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 shutdown_connections 2023-08-25T09:39:57.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.489+0000 7f72b1e06640 1 -- 172.21.15.70:0/2007568098 wait complete. 2023-08-25T09:39:57.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:39:57.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.726+0000 7f96b8e14640 1 Processor -- start 2023-08-25T09:39:57.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.726+0000 7f96b8e14640 1 -- start start 2023-08-25T09:39:57.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.727+0000 7f96b8e14640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f96b415ae50 0x7f96b40c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b8e14640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96b40cdc20 0x7f96b4069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b8e14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b40cc5f0 con 0x7f96b40ce060 2023-08-25T09:39:57.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b8e14640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b4070650 con 0x7f96b40cdc20 2023-08-25T09:39:57.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b8e14640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b4070930 con 0x7f96b415ae50 2023-08-25T09:39:57.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96abfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96b40cdc20 0x7f96b4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96abfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f96b40cdc20 0x7f96b4069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54998/0 (socket says 172.21.15.70:54998) 2023-08-25T09:39:57.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96abfff640 1 -- 172.21.15.70:0/275427340 learned_addr learned my addr 172.21.15.70:0/275427340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:57.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f96b40ce060 0x7f96b40ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41190/0 (socket says 172.21.15.70:41190) 2023-08-25T09:39:57.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b259c640 1 --2- 172.21.15.70:0/275427340 >> v2:172.21.15.155:3300/0 conn(0x7f96b415ae50 0x7f96b40c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96abfff640 1 -- 172.21.15.70:0/275427340 >> v2:172.21.15.155:3300/0 conn(0x7f96b415ae50 msgr2=0x7f96b40c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:57.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96abfff640 1 --2- 172.21.15.70:0/275427340 >> v2:172.21.15.155:3300/0 conn(0x7f96b415ae50 0x7f96b40c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.728+0000 7f96b1d9b640 1 -- 172.21.15.70:0/275427340 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628822543 0 0) 0x7f96b40cc5f0 con 0x7f96b40ce060 2023-08-25T09:39:57.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.729+0000 7f96abfff640 1 -- 172.21.15.70:0/275427340 >> v1:172.21.15.70:6789/0 conn(0x7f96b40ce060 legacy=0x7f96b40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.729+0000 7f96abfff640 1 -- 172.21.15.70:0/275427340 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4070c10 con 0x7f96b40cdc20 2023-08-25T09:39:57.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.730+0000 7f96abfff640 1 --2- 172.21.15.70:0/275427340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40cdc20 0x7f96b4069850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b1210 tx=0x7f969c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579707001a3e6671 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:57.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.730+0000 7f96b1d9b640 1 -- 172.21.15.70:0/275427340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c02f850 con 0x7f96b40cdc20 2023-08-25T09:39:57.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.730+0000 7f96b1d9b640 1 -- 172.21.15.70:0/275427340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c02f9b0 con 0x7f96b40cdc20 2023-08-25T09:39:57.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.730+0000 7f96b1d9b640 1 -- 172.21.15.70:0/275427340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c02fd40 con 0x7f96b40cdc20 2023-08-25T09:39:57.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.730+0000 7f96b8e14640 1 -- 172.21.15.70:0/275427340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40cdc20 msgr2=0x7f96b4069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.730+0000 7f96b8e14640 1 --2- 172.21.15.70:0/275427340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40cdc20 0x7f96b4069850 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96b40b1210 tx=0x7f969c02ba30 comp rx=0 tx=0).stop 2023-08-25T09:39:57.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.731+0000 7f96b8e14640 1 -- 172.21.15.70:0/275427340 shutdown_connections 2023-08-25T09:39:57.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.731+0000 7f96b8e14640 1 --2- 172.21.15.70:0/275427340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40cdc20 0x7f96b4069850 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.731+0000 7f96b8e14640 1 --2- 172.21.15.70:0/275427340 >> v2:172.21.15.155:3300/0 conn(0x7f96b415ae50 0x7f96b40c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.731+0000 7f96b8e14640 1 -- 172.21.15.70:0/275427340 >> 172.21.15.70:0/275427340 conn(0x7f96b40cb0d0 msgr2=0x7f96b40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:57.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.731+0000 7f96b8e14640 1 -- 172.21.15.70:0/275427340 shutdown_connections 2023-08-25T09:39:57.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.731+0000 7f96b8e14640 1 -- 172.21.15.70:0/275427340 wait complete. 2023-08-25T09:39:57.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.733+0000 7f96b8e14640 1 Processor -- start 2023-08-25T09:39:57.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.733+0000 7f96b8e14640 1 -- start start 2023-08-25T09:39:57.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b8e14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b40cdc20 0x7f96b41554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b8e14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 0x7f96b4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b8e14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b415ae50 0x7f96b40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b8e14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415bd00 con 0x7f96b40ce060 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b8e14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40c9690 con 0x7f96b415ae50 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b8e14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415bf30 con 0x7f96b40cdc20 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 0x7f96b4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 0x7f96b4155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55012/0 (socket says 172.21.15.70:55012) 2023-08-25T09:39:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96abfff640 1 -- 172.21.15.70:0/1011057471 learned_addr learned my addr 172.21.15.70:0/1011057471 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96b259c640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b40cdc20 0x7f96b41554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96abfff640 1 -- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b40cdc20 msgr2=0x7f96b41554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.734+0000 7f96abfff640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b40cdc20 0x7f96b41554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96abfff640 1 -- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b415ae50 msgr2=0x7f96b40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96abfff640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b415ae50 0x7f96b40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96abfff640 1 -- 172.21.15.70:0/1011057471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0094d0 con 0x7f96b40ce060 2023-08-25T09:39:57.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96abfff640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 0x7f96b4155a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f969c02ba70 tx=0x7f969c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:57.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c037d10 con 0x7f96b40ce060 2023-08-25T09:39:57.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969c037e90 con 0x7f96b40ce060 2023-08-25T09:39:57.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b4156040 con 0x7f96b40ce060 2023-08-25T09:39:57.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969c041640 con 0x7f96b40ce060 2023-08-25T09:39:57.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.735+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b4156600 con 0x7f96b40ce060 2023-08-25T09:39:57.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.736+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b415c0f0 con 0x7f96b40ce060 2023-08-25T09:39:57.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.737+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f969c0417a0 con 0x7f96b40ce060 2023-08-25T09:39:57.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.738+0000 7f96aaffd640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f969406a050 0x7f969406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:57.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.738+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f969c03c070 con 0x7f96b40ce060 2023-08-25T09:39:57.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.738+0000 7f96b259c640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f969406a050 0x7f969406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:57.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.739+0000 7f96b259c640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f969406a050 0x7f969406c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f96a4006fd0 tx=0x7f96a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:57.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.742+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f969c076ae0 con 0x7f96b40ce060 2023-08-25T09:39:57.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.929+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f96b4066fb0 con 0x7f96b40ce060 2023-08-25T09:39:57.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.930+0000 7f96aaffd640 1 -- 172.21.15.70:0/1011057471 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f969c07a100 con 0x7f96b40ce060 2023-08-25T09:39:57.932 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:39:57.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.932+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f969406a050 msgr2=0x7f969406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.932+0000 7f96b8e14640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f969406a050 0x7f969406c510 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f96a4006fd0 tx=0x7f96a4008040 comp rx=0 tx=0).stop 2023-08-25T09:39:57.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.932+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 msgr2=0x7f96b4155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:57.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.932+0000 7f96b8e14640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 0x7f96b4155a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f969c02ba70 tx=0x7f969c004730 comp rx=0 tx=0).stop 2023-08-25T09:39:57.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 shutdown_connections 2023-08-25T09:39:57.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f969406a050 0x7f969406c510 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96b415ae50 0x7f96b40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96b40ce060 0x7f96b4155a10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 --2- 172.21.15.70:0/1011057471 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96b40cdc20 0x7f96b41554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:57.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 >> 172.21.15.70:0/1011057471 conn(0x7f96b40cb0d0 msgr2=0x7f96b4158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:57.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 shutdown_connections 2023-08-25T09:39:57.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:57.933+0000 7f96b8e14640 1 -- 172.21.15.70:0/1011057471 wait complete. 2023-08-25T09:39:57.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:39:58.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.171+0000 7f61c23a4640 1 Processor -- start 2023-08-25T09:39:58.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.172+0000 7f61c23a4640 1 -- start start 2023-08-25T09:39:58.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61c23a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f61bc0c2110 0x7f61bc069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61c23a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 0x7f61bc0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61c23a4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc0c2550 con 0x7f61bc15ae50 2023-08-25T09:39:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61c23a4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61bc06a220 con 0x7f61bc0c2110 2023-08-25T09:39:58.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61c23a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0ce330 con 0x7f61bc069d90 2023-08-25T09:39:58.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61bbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f61bc15ae50 0x7f61bc0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41204/0 (socket says 172.21.15.70:41204) 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.173+0000 7f61bbfff640 1 -- 172.21.15.70:0/1250294735 learned_addr learned my addr 172.21.15.70:0/1250294735 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087105460 0 0) 0x7f61bc0c2550 con 0x7f61bc15ae50 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a8003520 con 0x7f61bc15ae50 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61c0941640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 0x7f61bc0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61b3fff640 1 --2- 172.21.15.70:0/1250294735 >> v2:172.21.15.155:3300/0 conn(0x7f61bc0c2110 0x7f61bc069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1040984199 0 0) 0x7f61a8003520 con 0x7f61bc15ae50 2023-08-25T09:39:58.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 msgr2=0x7f61bc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 0x7f61bc0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 >> v2:172.21.15.155:3300/0 conn(0x7f61bc0c2110 msgr2=0x7f61bc069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 --2- 172.21.15.70:0/1250294735 >> v2:172.21.15.155:3300/0 conn(0x7f61bc0c2110 0x7f61bc069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0cc5f0 con 0x7f61bc15ae50 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f61ac001670 con 0x7f61bc15ae50 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 >> v1:172.21.15.70:6789/0 conn(0x7f61bc15ae50 legacy=0x7f61bc0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 0x7f61a8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8003fa0 0x7f61a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a8004d00 0x7f61a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc06a220 con 0x7f61a8004d00 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0ce330 con 0x7f61a8003fa0 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.174+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a800ff20 con 0x7f61bc069d90 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61c0941640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a8004d00 0x7f61a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61bbfff640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 0x7f61a8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61b3fff640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8003fa0 0x7f61a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61c0941640 1 -- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 msgr2=0x7f61a8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61c0941640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc069d90 0x7f61a8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61c0941640 1 -- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8003fa0 msgr2=0x7f61a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61c0941640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8003fa0 0x7f61a80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.175+0000 7f61c0941640 1 -- 172.21.15.70:0/1250294735 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a8003520 con 0x7f61a8004d00 2023-08-25T09:39:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.176+0000 7f61c0941640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a8004d00 0x7f61a8005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f61b4007360 tx=0x7f61a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf5d693b7e14448 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:58.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.176+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b402ccd0 con 0x7f61a8004d00 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b402ce30 con 0x7f61a8004d00 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61bb7fe640 1 -- 172.21.15.70:0/1250294735 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4034660 con 0x7f61a8004d00 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c23a4640 1 -- 172.21.15.70:0/1250294735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a8004d00 msgr2=0x7f61a8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c23a4640 1 --2- 172.21.15.70:0/1250294735 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61a8004d00 0x7f61a8005180 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f61b4007360 tx=0x7f61a80057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c0941640 1 -- 172.21.15.70:0/1250294735 reap_dead start 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c23a4640 1 -- 172.21.15.70:0/1250294735 shutdown_connections 2023-08-25T09:39:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c23a4640 1 -- 172.21.15.70:0/1250294735 >> 172.21.15.70:0/1250294735 conn(0x7f61bc0cb0d0 msgr2=0x7f61bc0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c23a4640 1 -- 172.21.15.70:0/1250294735 shutdown_connections 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.177+0000 7f61c23a4640 1 -- 172.21.15.70:0/1250294735 wait complete. 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.179+0000 7f61c23a4640 1 Processor -- start 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.179+0000 7f61c23a4640 1 -- start start 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c23a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8004d00 0x7f61bc154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c23a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc0c4740 0x7f61bc0c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c23a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 0x7f61bc0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c23a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1553a0 con 0x7f61bc0cca80 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c23a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0cd440 con 0x7f61a8004d00 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c23a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0c8250 con 0x7f61bc0c4740 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c0941640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 0x7f61bc0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c0941640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 0x7f61bc0ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55038/0 (socket says 172.21.15.70:55038) 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.180+0000 7f61c0941640 1 -- 172.21.15.70:0/983953137 learned_addr learned my addr 172.21.15.70:0/983953137 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c0941640 1 -- 172.21.15.70:0/983953137 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc0c4740 msgr2=0x7f61bc0c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c0941640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc0c4740 0x7f61bc0c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c0941640 1 -- 172.21.15.70:0/983953137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8004d00 msgr2=0x7f61bc154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c0941640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8004d00 0x7f61bc154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c0941640 1 -- 172.21.15.70:0/983953137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0c6ab0 con 0x7f61bc0cca80 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c0941640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 0x7f61bc0ccf00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f61b4037040 tx=0x7f61b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4045070 con 0x7f61bc0cca80 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4034e00 con 0x7f61bc0cca80 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4003bd0 con 0x7f61bc0cca80 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.181+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc0c7640 con 0x7f61bc0cca80 2023-08-25T09:39:58.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.182+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc0c7c00 con 0x7f61bc0cca80 2023-08-25T09:39:58.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.183+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc066fb0 con 0x7f61bc0cca80 2023-08-25T09:39:58.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.184+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f61b403b040 con 0x7f61bc0cca80 2023-08-25T09:39:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.184+0000 7f61b97fa640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f619c06a080 0x7f619c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.185+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61b40a9e30 con 0x7f61bc0cca80 2023-08-25T09:39:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.185+0000 7f61bbfff640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f619c06a080 0x7f619c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.185+0000 7f61bbfff640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f619c06a080 0x7f619c06c540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0059f0 tx=0x7f61ac001d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:58.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.189+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f61b4075630 con 0x7f61bc0cca80 2023-08-25T09:39:58.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.381+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61bc0c8390 con 0x7f61bc0cca80 2023-08-25T09:39:58.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.382+0000 7f61b97fa640 1 -- 172.21.15.70:0/983953137 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61b4078c50 con 0x7f61bc0cca80 2023-08-25T09:39:58.384 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:39:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.384+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f619c06a080 msgr2=0x7f619c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.384+0000 7f61c23a4640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f619c06a080 0x7f619c06c540 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61ac0059f0 tx=0x7f61ac001d90 comp rx=0 tx=0).stop 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.384+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 msgr2=0x7f61bc0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.384+0000 7f61c23a4640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 0x7f61bc0ccf00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f61b4037040 tx=0x7f61b4004ab0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 shutdown_connections 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f619c06a080 0x7f619c06c540 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61bc0cca80 0x7f61bc0ccf00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61bc0c4740 0x7f61bc0c6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 --2- 172.21.15.70:0/983953137 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61a8004d00 0x7f61bc154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 >> 172.21.15.70:0/983953137 conn(0x7f61bc0cb0d0 msgr2=0x7f61bc156b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 shutdown_connections 2023-08-25T09:39:58.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.385+0000 7f61c23a4640 1 -- 172.21.15.70:0/983953137 wait complete. 2023-08-25T09:39:58.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:39:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.616+0000 7fb0ef59e640 1 Processor -- start 2023-08-25T09:39:58.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.616+0000 7fb0ef59e640 1 -- start start 2023-08-25T09:39:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.617+0000 7fb0ef59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0f00b0320 0x7fb0f00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.617+0000 7fb0ef59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0f00b1ce0 0x7fb0f00ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.617+0000 7fb0ef59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00ad8f0 con 0x7fb0f015a920 2023-08-25T09:39:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.617+0000 7fb0ef59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00adbb0 con 0x7fb0f00b0320 2023-08-25T09:39:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.617+0000 7fb0ef59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00c90a0 con 0x7fb0f00b1ce0 2023-08-25T09:39:58.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0f00b0320 0x7fb0f00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0f00b0320 0x7fb0f00c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55040/0 (socket says 172.21.15.70:55040) 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 -- 172.21.15.70:0/683637992 learned_addr learned my addr 172.21.15.70:0/683637992 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0eed9d640 1 --2- 172.21.15.70:0/683637992 >> v2:172.21.15.155:3300/0 conn(0x7fb0f00b1ce0 0x7fb0f00ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 -- 172.21.15.70:0/683637992 >> v2:172.21.15.155:3300/0 conn(0x7fb0f00b1ce0 msgr2=0x7fb0f00ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 --2- 172.21.15.70:0/683637992 >> v2:172.21.15.155:3300/0 conn(0x7fb0f00b1ce0 0x7fb0f00ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 -- 172.21.15.70:0/683637992 >> v1:172.21.15.155:6789/0 conn(0x7fb0f015a920 legacy=0x7fb0f00b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.618+0000 7fb0ee59c640 1 -- 172.21.15.70:0/683637992 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00c9380 con 0x7fb0f00b0320 2023-08-25T09:39:58.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.620+0000 7fb0ee59c640 1 --2- 172.21.15.70:0/683637992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f00b0320 0x7fb0f00c9e60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00c8f30 tx=0x7fb0e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d2018e4d816f5d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.620+0000 7fb0edd9b640 1 -- 172.21.15.70:0/683637992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0004230 con 0x7fb0f00b0320 2023-08-25T09:39:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.620+0000 7fb0edd9b640 1 -- 172.21.15.70:0/683637992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e0004390 con 0x7fb0f00b0320 2023-08-25T09:39:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.620+0000 7fb0edd9b640 1 -- 172.21.15.70:0/683637992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0003810 con 0x7fb0f00b0320 2023-08-25T09:39:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 -- 172.21.15.70:0/683637992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f00b0320 msgr2=0x7fb0f00c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/683637992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f00b0320 0x7fb0f00c9e60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00c8f30 tx=0x7fb0e002b8f0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 -- 172.21.15.70:0/683637992 shutdown_connections 2023-08-25T09:39:58.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/683637992 >> v2:172.21.15.155:3300/0 conn(0x7fb0f00b1ce0 0x7fb0f00ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/683637992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f00b0320 0x7fb0f00c9e60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 -- 172.21.15.70:0/683637992 >> 172.21.15.70:0/683637992 conn(0x7fb0f00ce830 msgr2=0x7fb0f00cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:58.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 -- 172.21.15.70:0/683637992 shutdown_connections 2023-08-25T09:39:58.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.621+0000 7fb0ef59e640 1 -- 172.21.15.70:0/683637992 wait complete. 2023-08-25T09:39:58.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.623+0000 7fb0ef59e640 1 Processor -- start 2023-08-25T09:39:58.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.623+0000 7fb0ef59e640 1 -- start start 2023-08-25T09:39:58.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0ef59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0f00b0320 0x7fb0f00979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0ef59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0f00b1ce0 0x7fb0f00c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0ef59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 0x7fb0f00c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0ef59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f0097f00 con 0x7fb0f015a920 2023-08-25T09:39:58.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0ef59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f006a260 con 0x7fb0f00b0320 2023-08-25T09:39:58.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0ef59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f006a540 con 0x7fb0f00b1ce0 2023-08-25T09:39:58.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0eed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 0x7fb0f00c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0eed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 0x7fb0f00c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55048/0 (socket says 172.21.15.70:55048) 2023-08-25T09:39:58.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.624+0000 7fb0eed9d640 1 -- 172.21.15.70:0/3599813136 learned_addr learned my addr 172.21.15.70:0/3599813136 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:58.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0ee59c640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0f00b0320 0x7fb0f00979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0eed9d640 1 -- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0f00b1ce0 msgr2=0x7fb0f00c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0e7fff640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0f00b1ce0 0x7fb0f00c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0eed9d640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0f00b1ce0 0x7fb0f00c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0eed9d640 1 -- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0f00b0320 msgr2=0x7fb0f00979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0eed9d640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0f00b0320 0x7fb0f00979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0eed9d640 1 -- 172.21.15.70:0/3599813136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e0009460 con 0x7fb0f015a920 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0e7fff640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0f00b1ce0 0x7fb0f00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0ee59c640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0f00b0320 0x7fb0f00979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0eed9d640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 0x7fb0f00c6b80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8007e40 tx=0x7fb0e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8014620 con 0x7fb0f015a920 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e800ee90 con 0x7fb0f015a920 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.625+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80149e0 con 0x7fb0f015a920 2023-08-25T09:39:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.626+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f006a820 con 0x7fb0f015a920 2023-08-25T09:39:58.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.626+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f006ade0 con 0x7fb0f015a920 2023-08-25T09:39:58.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.627+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f00c88d0 con 0x7fb0f015a920 2023-08-25T09:39:58.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.628+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0e8014b40 con 0x7fb0f015a920 2023-08-25T09:39:58.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.628+0000 7fb0e6ffd640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0d406a050 0x7fb0d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:58.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.629+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0e8089060 con 0x7fb0f015a920 2023-08-25T09:39:58.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.629+0000 7fb0ee59c640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0d406a050 0x7fb0d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:58.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.629+0000 7fb0ee59c640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0d406a050 0x7fb0d406c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0035000 tx=0x7fb0e0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:58.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.633+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0e8058180 con 0x7fb0f015a920 2023-08-25T09:39:58.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.805+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0f0061460 con 0x7fb0f015a920 2023-08-25T09:39:58.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.806+0000 7fb0e6ffd640 1 -- 172.21.15.70:0/3599813136 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0e8057d10 con 0x7fb0f015a920 2023-08-25T09:39:58.808 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:39:58.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.808+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0d406a050 msgr2=0x7fb0d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.808+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0d406a050 0x7fb0d406c510 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0035000 tx=0x7fb0e0008030 comp rx=0 tx=0).stop 2023-08-25T09:39:58.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.808+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 msgr2=0x7fb0f00c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:58.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.808+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 0x7fb0f00c6b80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8007e40 tx=0x7fb0e800e4b0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 shutdown_connections 2023-08-25T09:39:58.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0d406a050 0x7fb0d406c510 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0f015a920 0x7fb0f00c6b80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0f00b1ce0 0x7fb0f00c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 --2- 172.21.15.70:0/3599813136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0f00b0320 0x7fb0f00979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:58.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 >> 172.21.15.70:0/3599813136 conn(0x7fb0f00ce830 msgr2=0x7fb0f00af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:58.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 shutdown_connections 2023-08-25T09:39:58.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:58.809+0000 7fb0ef59e640 1 -- 172.21.15.70:0/3599813136 wait complete. 2023-08-25T09:39:58.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:39:59.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.039+0000 7fb6eed83640 1 Processor -- start 2023-08-25T09:39:59.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.039+0000 7fb6eed83640 1 -- start start 2023-08-25T09:39:59.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6eed83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb6e80b4400 0x7fb6e80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6eed83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e80b12b0 con 0x7fb6e815acd0 2023-08-25T09:39:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6eed83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e806ae20 con 0x7fb6e80ca780 2023-08-25T09:39:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6eed83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80b47e0 con 0x7fb6e80b4400 2023-08-25T09:39:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6ecb1f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb6e815acd0 0x7fb6e815b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54104/0 (socket says 172.21.15.70:54104) 2023-08-25T09:39:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6ecb1f640 1 -- 172.21.15.70:0/506243093 learned_addr learned my addr 172.21.15.70:0/506243093 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:59.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.041+0000 7fb6ed320640 1 --2- 172.21.15.70:0/506243093 >> v2:172.21.15.70:3300/0 conn(0x7fb6e80b4400 0x7fb6e80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.042+0000 7fb6e77fe640 1 -- 172.21.15.70:0/506243093 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272877274 0 0) 0x7fb6e80b12b0 con 0x7fb6e815acd0 2023-08-25T09:39:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.042+0000 7fb6ed320640 1 -- 172.21.15.70:0/506243093 >> v1:172.21.15.155:6789/0 conn(0x7fb6e80ca780 legacy=0x7fb6e80b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.042+0000 7fb6ed320640 1 -- 172.21.15.70:0/506243093 >> v1:172.21.15.70:6789/0 conn(0x7fb6e815acd0 legacy=0x7fb6e815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.042+0000 7fb6ed320640 1 -- 172.21.15.70:0/506243093 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e815b7c0 con 0x7fb6e80b4400 2023-08-25T09:39:59.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.043+0000 7fb6ed320640 1 --2- 172.21.15.70:0/506243093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80b4400 0x7fb6e80cac30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009d90 tx=0x7fb6e002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964b03d6d4a886db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.043+0000 7fb6e77fe640 1 -- 172.21.15.70:0/506243093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e002cc60 con 0x7fb6e80b4400 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.043+0000 7fb6e77fe640 1 -- 172.21.15.70:0/506243093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e002cdc0 con 0x7fb6e80b4400 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.043+0000 7fb6e77fe640 1 -- 172.21.15.70:0/506243093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0034660 con 0x7fb6e80b4400 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 -- 172.21.15.70:0/506243093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80b4400 msgr2=0x7fb6e80cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 --2- 172.21.15.70:0/506243093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80b4400 0x7fb6e80cac30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0009d90 tx=0x7fb6e002beb0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 -- 172.21.15.70:0/506243093 shutdown_connections 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 --2- 172.21.15.70:0/506243093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80b4400 0x7fb6e80cac30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 -- 172.21.15.70:0/506243093 >> 172.21.15.70:0/506243093 conn(0x7fb6e80ce760 msgr2=0x7fb6e80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:59.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 -- 172.21.15.70:0/506243093 shutdown_connections 2023-08-25T09:39:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.044+0000 7fb6eed83640 1 -- 172.21.15.70:0/506243093 wait complete. 2023-08-25T09:39:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.046+0000 7fb6eed83640 1 Processor -- start 2023-08-25T09:39:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.046+0000 7fb6eed83640 1 -- start start 2023-08-25T09:39:59.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6eed83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e80b4400 0x7fb6e80c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6eed83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 0x7fb6e80c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6eed83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e815acd0 0x7fb6e8147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6eed83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c62d0 con 0x7fb6e80ca780 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6eed83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c65b0 con 0x7fb6e815acd0 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6eed83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c6890 con 0x7fb6e80b4400 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 0x7fb6e80c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 0x7fb6e80c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47044/0 (socket says 172.21.15.70:47044) 2023-08-25T09:39:59.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6e7fff640 1 -- 172.21.15.70:0/3473014845 learned_addr learned my addr 172.21.15.70:0/3473014845 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:59.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6ecb1f640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e80b4400 0x7fb6e80c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6ed320640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e815acd0 0x7fb6e8147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6e7fff640 1 -- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e80b4400 msgr2=0x7fb6e80c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6e7fff640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e80b4400 0x7fb6e80c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.047+0000 7fb6e7fff640 1 -- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e815acd0 msgr2=0x7fb6e8147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6e7fff640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e815acd0 0x7fb6e8147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6e7fff640 1 -- 172.21.15.70:0/3473014845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e0009a70 con 0x7fb6e80ca780 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6e7fff640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 0x7fb6e80c42e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c400 tx=0x7fb6d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40052d0 con 0x7fb6e80ca780 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4005450 con 0x7fb6e80ca780 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4007260 con 0x7fb6e80ca780 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e80c6b70 con 0x7fb6e80ca780 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.048+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e80c3c50 con 0x7fb6e80ca780 2023-08-25T09:39:59.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.049+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e80c7090 con 0x7fb6e80ca780 2023-08-25T09:39:59.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.050+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb6d40073c0 con 0x7fb6e80ca780 2023-08-25T09:39:59.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.052+0000 7fb6e57fa640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6cc06a050 0x7fb6cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.052+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6d40879c0 con 0x7fb6e80ca780 2023-08-25T09:39:59.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.052+0000 7fb6ecb1f640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6cc06a050 0x7fb6cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.053+0000 7fb6ecb1f640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6cc06a050 0x7fb6cc06c510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806c410 tx=0x7fb6dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:59.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.056+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb6e80c7090 con 0x7fb6e80ca780 2023-08-25T09:39:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.257+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb6e80c7090 con 0x7fb6e80ca780 2023-08-25T09:39:59.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.258+0000 7fb6e57fa640 1 -- 172.21.15.70:0/3473014845 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb6e80c7090 con 0x7fb6e80ca780 2023-08-25T09:39:59.261 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:39:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6cc06a050 msgr2=0x7fb6cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6cc06a050 0x7fb6cc06c510 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806c410 tx=0x7fb6dc009040 comp rx=0 tx=0).stop 2023-08-25T09:39:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 msgr2=0x7fb6e80c42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 0x7fb6e80c42e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb6d400c400 tx=0x7fb6d400c8c0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 shutdown_connections 2023-08-25T09:39:59.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb6cc06a050 0x7fb6cc06c510 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb6e815acd0 0x7fb6e8147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb6e80ca780 0x7fb6e80c42e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 --2- 172.21.15.70:0/3473014845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb6e80b4400 0x7fb6e80c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 >> 172.21.15.70:0/3473014845 conn(0x7fb6e80ce760 msgr2=0x7fb6e80b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:59.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 shutdown_connections 2023-08-25T09:39:59.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.261+0000 7fb6eed83640 1 -- 172.21.15.70:0/3473014845 wait complete. 2023-08-25T09:39:59.283 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:39:59.284 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:39:59.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:39:59.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.583+0000 7f258b5e1640 1 Processor -- start 2023-08-25T09:39:59.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.583+0000 7f258b5e1640 1 -- start start 2023-08-25T09:39:59.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.585+0000 7f258b5e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 0x7f25840ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.585+0000 7f258b5e1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25841615b0 con 0x7f25840b2b40 2023-08-25T09:39:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.585+0000 7f258b5e1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258406f050 con 0x7f25840ac870 2023-08-25T09:39:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.585+0000 7f258b5e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258406f330 con 0x7f258415cb40 2023-08-25T09:39:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f2589b7e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f25840b2b40 0x7f2584160ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54110/0 (socket says 172.21.15.70:54110) 2023-08-25T09:39:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f2589b7e640 1 -- 172.21.15.70:0/1112282213 learned_addr learned my addr 172.21.15.70:0/1112282213 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f258937d640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 0x7f25840ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587109725 0 0) 0x7f25841615b0 con 0x7f25840b2b40 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2570003520 con 0x7f25840b2b40 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1037447679 0 0) 0x7f2570003520 con 0x7f25840b2b40 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 msgr2=0x7f25840ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 0x7f25840ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 >> v1:172.21.15.155:6789/0 conn(0x7f25840ac870 legacy=0x7f2584160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258406f610 con 0x7f25840b2b40 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.586+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f257c0019f0 con 0x7f25840b2b40 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 >> v1:172.21.15.70:6789/0 conn(0x7f25840b2b40 legacy=0x7f2584160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 0x7f25700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570003f20 0x7f25700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2570004d00 0x7f2570005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258406f330 con 0x7f2570004d00 2023-08-25T09:39:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257000ba60 con 0x7f2570003f20 2023-08-25T09:39:59.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257000bc20 con 0x7f258415cb40 2023-08-25T09:39:59.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f2589b7e640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2570004d00 0x7f2570005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f258937d640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 0x7f25700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f2589b7e640 1 -- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 msgr2=0x7f25700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f2589b7e640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415cb40 0x7f25700039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f2589b7e640 1 -- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570003f20 msgr2=0x7f25700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:39:59.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f2589b7e640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570003f20 0x7f25700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.587+0000 7f2589b7e640 1 -- 172.21.15.70:0/1112282213 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2570003520 con 0x7f2570004d00 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.588+0000 7f2589b7e640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2570004d00 0x7f2570005180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f257c00a800 tx=0x7f25700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b37e30c9758bd56 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0072d0 con 0x7f2570004d00 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c02f070 con 0x7f2570004d00 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f257bfff640 1 -- 172.21.15.70:0/1112282213 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c0076f0 con 0x7f2570004d00 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f258b5e1640 1 -- 172.21.15.70:0/1112282213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2570004d00 msgr2=0x7f2570005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f258b5e1640 1 --2- 172.21.15.70:0/1112282213 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2570004d00 0x7f2570005180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f257c00a800 tx=0x7f25700057f0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f2589b7e640 1 -- 172.21.15.70:0/1112282213 reap_dead start 2023-08-25T09:39:59.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f258b5e1640 1 -- 172.21.15.70:0/1112282213 shutdown_connections 2023-08-25T09:39:59.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.589+0000 7f258b5e1640 1 -- 172.21.15.70:0/1112282213 >> 172.21.15.70:0/1112282213 conn(0x7f2584069850 msgr2=0x7f2584069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:59.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.590+0000 7f258b5e1640 1 -- 172.21.15.70:0/1112282213 shutdown_connections 2023-08-25T09:39:59.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.590+0000 7f258b5e1640 1 -- 172.21.15.70:0/1112282213 wait complete. 2023-08-25T09:39:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.591+0000 7f258b5e1640 1 Processor -- start 2023-08-25T09:39:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 -- start start 2023-08-25T09:39:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570004d00 0x7f258415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 0x7f258414f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258414e000 0x7f258414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841585d0 con 0x7f258414d200 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258415b830 con 0x7f2570004d00 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.592+0000 7f258b5e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258414e5c0 con 0x7f258414e000 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 0x7f258414f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 0x7f258414f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47056/0 (socket says 172.21.15.70:47056) 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 -- 172.21.15.70:0/365955011 learned_addr learned my addr 172.21.15.70:0/365955011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 -- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258414e000 msgr2=0x7f258414fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258414e000 0x7f258414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f258937d640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570004d00 0x7f258415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 -- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570004d00 msgr2=0x7f258415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570004d00 0x7f258415b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.593+0000 7f2588b7c640 1 -- 172.21.15.70:0/365955011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258414e8a0 con 0x7f258414d200 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.594+0000 7f2588b7c640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 0x7f258414f820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f257400c3d0 tx=0x7f257400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.594+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740052d0 con 0x7f258414d200 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.594+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574005430 con 0x7f258414d200 2023-08-25T09:39:59.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.594+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574003830 con 0x7f258414d200 2023-08-25T09:39:59.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.594+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258414eb80 con 0x7f258414d200 2023-08-25T09:39:59.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.594+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841f5750 con 0x7f258414d200 2023-08-25T09:39:59.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.595+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258414f0a0 con 0x7f258414d200 2023-08-25T09:39:59.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.596+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2574003990 con 0x7f258414d200 2023-08-25T09:39:59.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.598+0000 7f2579ffb640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256806a050 0x7f256806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:39:59.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.598+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2574087bd0 con 0x7f258414d200 2023-08-25T09:39:59.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.598+0000 7f258937d640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256806a050 0x7f256806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:39:59.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.599+0000 7f258937d640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256806a050 0x7f256806c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2580009910 tx=0x7f2580008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:39:59.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.602+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f258414f0a0 con 0x7f258414d200 2023-08-25T09:39:59.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.796+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f258414f0a0 con 0x7f258414d200 2023-08-25T09:39:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.798+0000 7f2579ffb640 1 -- 172.21.15.70:0/365955011 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f258414f0a0 con 0x7f258414d200 2023-08-25T09:39:59.801 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:39:59.801 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:39:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.800+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256806a050 msgr2=0x7f256806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.800+0000 7f258b5e1640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256806a050 0x7f256806c510 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2580009910 tx=0x7f2580008040 comp rx=0 tx=0).stop 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 msgr2=0x7f258414f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 0x7f258414f820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f257400c3d0 tx=0x7f257400c890 comp rx=0 tx=0).stop 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 shutdown_connections 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256806a050 0x7f256806c510 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258414e000 0x7f258414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f258414d200 0x7f258414f820 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 --2- 172.21.15.70:0/365955011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2570004d00 0x7f258415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:39:59.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 >> 172.21.15.70:0/365955011 conn(0x7f2584069850 msgr2=0x7f258415ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:39:59.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 shutdown_connections 2023-08-25T09:39:59.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:39:59.801+0000 7f258b5e1640 1 -- 172.21.15.70:0/365955011 wait complete. 2023-08-25T09:39:59.821 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:39:59.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:00.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.030+0000 7f8d73234640 1 Processor -- start 2023-08-25T09:40:00.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.030+0000 7f8d73234640 1 -- start start 2023-08-25T09:40:00.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d73234640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d6c15ae50 0x7f8d6c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d73234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c0cdc20 0x7f8d6c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d73234640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6c0cc5f0 con 0x7f8d6c0ce060 2023-08-25T09:40:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d73234640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c070650 con 0x7f8d6c15ae50 2023-08-25T09:40:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d73234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c070930 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d70fd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d6c15ae50 0x7f8d6c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d70fd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d6c15ae50 0x7f8d6c0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47066/0 (socket says 172.21.15.70:47066) 2023-08-25T09:40:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.032+0000 7f8d70fd0640 1 -- 172.21.15.70:0/3664630533 learned_addr learned my addr 172.21.15.70:0/3664630533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:00.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.033+0000 7f8d70fd0640 1 -- 172.21.15.70:0/3664630533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c0cdc20 msgr2=0x7f8d6c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.033+0000 7f8d70fd0640 1 --2- 172.21.15.70:0/3664630533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c0cdc20 0x7f8d6c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.033+0000 7f8d70fd0640 1 -- 172.21.15.70:0/3664630533 >> v1:172.21.15.155:6789/0 conn(0x7f8d6c0ce060 legacy=0x7f8d6c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.033+0000 7f8d70fd0640 1 -- 172.21.15.70:0/3664630533 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c070c10 con 0x7f8d6c15ae50 2023-08-25T09:40:00.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.034+0000 7f8d70fd0640 1 --2- 172.21.15.70:0/3664630533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c15ae50 0x7f8d6c0c2110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d6002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a8a40316b05135 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.034+0000 7f8d6b7fe640 1 -- 172.21.15.70:0/3664630533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60030840 con 0x7f8d6c15ae50 2023-08-25T09:40:00.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d6b7fe640 1 -- 172.21.15.70:0/3664630533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d600309a0 con 0x7f8d6c15ae50 2023-08-25T09:40:00.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d6b7fe640 1 -- 172.21.15.70:0/3664630533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60030d30 con 0x7f8d6c15ae50 2023-08-25T09:40:00.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d73234640 1 -- 172.21.15.70:0/3664630533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c15ae50 msgr2=0x7f8d6c0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d73234640 1 --2- 172.21.15.70:0/3664630533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c15ae50 0x7f8d6c0c2110 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d6002b980 comp rx=0 tx=0).stop 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d73234640 1 -- 172.21.15.70:0/3664630533 shutdown_connections 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d73234640 1 --2- 172.21.15.70:0/3664630533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c0cdc20 0x7f8d6c069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d73234640 1 --2- 172.21.15.70:0/3664630533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c15ae50 0x7f8d6c0c2110 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.035+0000 7f8d73234640 1 -- 172.21.15.70:0/3664630533 >> 172.21.15.70:0/3664630533 conn(0x7f8d6c0cb0d0 msgr2=0x7f8d6c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.036+0000 7f8d73234640 1 -- 172.21.15.70:0/3664630533 shutdown_connections 2023-08-25T09:40:00.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.036+0000 7f8d73234640 1 -- 172.21.15.70:0/3664630533 wait complete. 2023-08-25T09:40:00.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.037+0000 7f8d73234640 1 Processor -- start 2023-08-25T09:40:00.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.037+0000 7f8d73234640 1 -- start start 2023-08-25T09:40:00.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.038+0000 7f8d73234640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 0x7f8d6c0b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.038+0000 7f8d73234640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d6c0ce060 0x7f8d6c1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.038+0000 7f8d73234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c15ae50 0x7f8d6c1f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.038+0000 7f8d73234640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0c8270 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.038+0000 7f8d73234640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0c75f0 con 0x7f8d6c0ce060 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.038+0000 7f8d73234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0c78d0 con 0x7f8d6c15ae50 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 0x7f8d6c0b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 0x7f8d6c0b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47074/0 (socket says 172.21.15.70:47074) 2023-08-25T09:40:00.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 -- 172.21.15.70:0/74618269 learned_addr learned my addr 172.21.15.70:0/74618269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 -- 172.21.15.70:0/74618269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c15ae50 msgr2=0x7f8d6c1f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c15ae50 0x7f8d6c1f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 -- 172.21.15.70:0/74618269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d6c0ce060 msgr2=0x7f8d6c1f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d6c0ce060 0x7f8d6c1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 -- 172.21.15.70:0/74618269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d600094d0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.039+0000 7f8d70fd0640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 0x7f8d6c0b3a50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.040+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d600307f0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.040+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60039d90 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.040+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60043630 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.040+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6c0c7bb0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.040+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c0c68a0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.041+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6c0b3f90 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.042+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8d60042390 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.043+0000 7f8d697fa640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d5006a050 0x7f8d5006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.043+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8d600aa9c0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.043+0000 7f8d6bfff640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d5006a050 0x7f8d5006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.044+0000 7f8d6bfff640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d5006a050 0x7f8d5006c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c06a220 tx=0x7f8d58006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.047+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8d600760a0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.235+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d6c0afe70 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.236+0000 7f8d697fa640 1 -- 172.21.15.70:0/74618269 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d600340a0 con 0x7f8d6c0cdc20 2023-08-25T09:40:00.239 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:00.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d5006a050 msgr2=0x7f8d5006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d5006a050 0x7f8d5006c510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c06a220 tx=0x7f8d58006c90 comp rx=0 tx=0).stop 2023-08-25T09:40:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 msgr2=0x7f8d6c0b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 0x7f8d6c0b3a50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d60004730 comp rx=0 tx=0).stop 2023-08-25T09:40:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 shutdown_connections 2023-08-25T09:40:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d5006a050 0x7f8d5006c510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d6c15ae50 0x7f8d6c1f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d6c0ce060 0x7f8d6c1f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 --2- 172.21.15.70:0/74618269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d6c0cdc20 0x7f8d6c0b3a50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.238+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 >> 172.21.15.70:0/74618269 conn(0x7f8d6c0cb0d0 msgr2=0x7f8d6c0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.239+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 shutdown_connections 2023-08-25T09:40:00.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.239+0000 7f8d73234640 1 -- 172.21.15.70:0/74618269 wait complete. 2023-08-25T09:40:00.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:00.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.472+0000 7f13bde3f640 1 Processor -- start 2023-08-25T09:40:00.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.472+0000 7f13bde3f640 1 -- start start 2023-08-25T09:40:00.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13bde3f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13b80ce8e0 0x7f13b80cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13bde3f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f13b806e090 0x7f13b8164520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13bde3f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b80cba70 con 0x7f13b80cdc80 2023-08-25T09:40:00.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13bde3f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b80cbd50 con 0x7f13b80ce8e0 2023-08-25T09:40:00.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13bde3f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b80cc030 con 0x7f13b806e090 2023-08-25T09:40:00.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13b77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13b80ce8e0 0x7f13b80cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13b77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13b80ce8e0 0x7f13b80cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47086/0 (socket says 172.21.15.70:47086) 2023-08-25T09:40:00.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13b77fe640 1 -- 172.21.15.70:0/1403910932 learned_addr learned my addr 172.21.15.70:0/1403910932 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:00.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.474+0000 7f13affff640 1 --1- 172.21.15.70:0/1403910932 >> v1:172.21.15.70:6789/0 conn(0x7f13b80cdc80 0x7f13b80cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54124/0 (socket says 172.21.15.70:54124) 2023-08-25T09:40:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b77fe640 1 -- 172.21.15.70:0/1403910932 >> v2:172.21.15.155:3300/0 conn(0x7f13b806e090 msgr2=0x7f13b8164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b7fff640 1 --2- 172.21.15.70:0/1403910932 >> v2:172.21.15.155:3300/0 conn(0x7f13b806e090 0x7f13b8164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b6ffd640 1 -- 172.21.15.70:0/1403910932 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696634840 0 0) 0x7f13b80cba70 con 0x7f13b80cdc80 2023-08-25T09:40:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b77fe640 1 --2- 172.21.15.70:0/1403910932 >> v2:172.21.15.155:3300/0 conn(0x7f13b806e090 0x7f13b8164520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b77fe640 1 -- 172.21.15.70:0/1403910932 >> v1:172.21.15.70:6789/0 conn(0x7f13b80cdc80 legacy=0x7f13b80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b77fe640 1 -- 172.21.15.70:0/1403910932 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b80cc310 con 0x7f13b80ce8e0 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.475+0000 7f13b7fff640 1 --2- 172.21.15.70:0/1403910932 >> v2:172.21.15.155:3300/0 conn(0x7f13b806e090 0x7f13b8164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.476+0000 7f13b77fe640 1 --2- 172.21.15.70:0/1403910932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80cb470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13b8070cb0 tx=0x7f13a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f71717123d84137 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.476+0000 7f13b6ffd640 1 -- 172.21.15.70:0/1403910932 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8004230 con 0x7f13b80ce8e0 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.476+0000 7f13b6ffd640 1 -- 172.21.15.70:0/1403910932 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a8004390 con 0x7f13b80ce8e0 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.476+0000 7f13b6ffd640 1 -- 172.21.15.70:0/1403910932 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8030840 con 0x7f13b80ce8e0 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 -- 172.21.15.70:0/1403910932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 msgr2=0x7f13b80cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 --2- 172.21.15.70:0/1403910932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80cb470 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f13b8070cb0 tx=0x7f13a802b980 comp rx=0 tx=0).stop 2023-08-25T09:40:00.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 -- 172.21.15.70:0/1403910932 shutdown_connections 2023-08-25T09:40:00.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 --2- 172.21.15.70:0/1403910932 >> v2:172.21.15.155:3300/0 conn(0x7f13b806e090 0x7f13b8164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 --2- 172.21.15.70:0/1403910932 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80cb470 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 -- 172.21.15.70:0/1403910932 >> 172.21.15.70:0/1403910932 conn(0x7f13b80d0860 msgr2=0x7f13b80d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 -- 172.21.15.70:0/1403910932 shutdown_connections 2023-08-25T09:40:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.477+0000 7f13bde3f640 1 -- 172.21.15.70:0/1403910932 wait complete. 2023-08-25T09:40:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.479+0000 7f13bde3f640 1 Processor -- start 2023-08-25T09:40:00.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.479+0000 7f13bde3f640 1 -- start start 2023-08-25T09:40:00.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13bde3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b806e090 0x7f13b80b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13bde3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b80cdc80 0x7f13b81d0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13bde3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13bde3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b80b5320 con 0x7f13b80ce8e0 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13bde3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b80c4200 con 0x7f13b80cdc80 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13bde3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b80b3d40 con 0x7f13b806e090 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47092/0 (socket says 172.21.15.70:47092) 2023-08-25T09:40:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 -- 172.21.15.70:0/3807294283 learned_addr learned my addr 172.21.15.70:0/3807294283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 -- 172.21.15.70:0/3807294283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b806e090 msgr2=0x7f13b80b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b806e090 0x7f13b80b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 -- 172.21.15.70:0/3807294283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b80cdc80 msgr2=0x7f13b81d0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b80cdc80 0x7f13b81d0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.480+0000 7f13b7fff640 1 -- 172.21.15.70:0/3807294283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a8009460 con 0x7f13b80ce8e0 2023-08-25T09:40:00.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.481+0000 7f13b7fff640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80b3480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f13b000cea0 tx=0x7f13b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.481+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003b90 con 0x7f13b80ce8e0 2023-08-25T09:40:00.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.481+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b0003d10 con 0x7f13b80ce8e0 2023-08-25T09:40:00.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.481+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b0004260 con 0x7f13b80ce8e0 2023-08-25T09:40:00.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.481+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b80b4020 con 0x7f13b80ce8e0 2023-08-25T09:40:00.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.481+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b80b48c0 con 0x7f13b80ce8e0 2023-08-25T09:40:00.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.482+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b8066fb0 con 0x7f13b80ce8e0 2023-08-25T09:40:00.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.483+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f13b00043c0 con 0x7f13b80ce8e0 2023-08-25T09:40:00.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.484+0000 7f13b4ff9640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f139806a050 0x7f139806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.484+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13b0088f40 con 0x7f13b80ce8e0 2023-08-25T09:40:00.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.484+0000 7f13b77fe640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f139806a050 0x7f139806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.485+0000 7f13b77fe640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f139806a050 0x7f139806c510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13a8035000 tx=0x7f13a8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.489+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13b0054500 con 0x7f13b80ce8e0 2023-08-25T09:40:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.677+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f13b80b4a80 con 0x7f13b80ce8e0 2023-08-25T09:40:00.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.678+0000 7f13b4ff9640 1 -- 172.21.15.70:0/3807294283 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f13b0057b20 con 0x7f13b80ce8e0 2023-08-25T09:40:00.681 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.680+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f139806a050 msgr2=0x7f139806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f139806a050 0x7f139806c510 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f13a8035000 tx=0x7f13a8030000 comp rx=0 tx=0).stop 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 msgr2=0x7f13b80b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80b3480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f13b000cea0 tx=0x7f13b000cf90 comp rx=0 tx=0).stop 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 shutdown_connections 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f139806a050 0x7f139806c510 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13b80ce8e0 0x7f13b80b3480 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13b80cdc80 0x7f13b81d0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 --2- 172.21.15.70:0/3807294283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13b806e090 0x7f13b80b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 >> 172.21.15.70:0/3807294283 conn(0x7f13b80d0860 msgr2=0x7f13b81e1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:00.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 shutdown_connections 2023-08-25T09:40:00.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.681+0000 7f13bde3f640 1 -- 172.21.15.70:0/3807294283 wait complete. 2023-08-25T09:40:00.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.917+0000 7f58dfe9b640 1 Processor -- start 2023-08-25T09:40:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.918+0000 7f58dfe9b640 1 -- start start 2023-08-25T09:40:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58dfe9b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58d8069f60 0x7f58d80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58dfe9b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80c2550 con 0x7f58d815ae50 2023-08-25T09:40:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58dfe9b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80ce330 con 0x7f58d80c2110 2023-08-25T09:40:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58dfe9b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d80cc5f0 con 0x7f58d8069f60 2023-08-25T09:40:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58ddc37640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f58d815ae50 0x7f58d80cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54130/0 (socket says 172.21.15.70:54130) 2023-08-25T09:40:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58ddc37640 1 -- 172.21.15.70:0/1424394492 learned_addr learned my addr 172.21.15.70:0/1424394492 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.919+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926928868 0 0) 0x7f58d80c2550 con 0x7f58d815ae50 2023-08-25T09:40:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c4003520 con 0x7f58d815ae50 2023-08-25T09:40:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58de438640 1 --2- 172.21.15.70:0/1424394492 >> v2:172.21.15.155:3300/0 conn(0x7f58d8069f60 0x7f58d80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377640491 0 0) 0x7f58c4003520 con 0x7f58d815ae50 2023-08-25T09:40:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 >> v2:172.21.15.155:3300/0 conn(0x7f58d8069f60 msgr2=0x7f58d80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 --2- 172.21.15.70:0/1424394492 >> v2:172.21.15.155:3300/0 conn(0x7f58d8069f60 0x7f58d80ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 >> v1:172.21.15.155:6789/0 conn(0x7f58d80c2110 legacy=0x7f58d8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d8070650 con 0x7f58d815ae50 2023-08-25T09:40:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f58d40019f0 con 0x7f58d815ae50 2023-08-25T09:40:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 >> v1:172.21.15.70:6789/0 conn(0x7f58d815ae50 legacy=0x7f58d80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4003970 0x7f58c4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c40047c0 0x7f58c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58c4005180 0x7f58c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80cc5f0 con 0x7f58c4005180 2023-08-25T09:40:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c400c090 con 0x7f58c4003970 2023-08-25T09:40:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c400c250 con 0x7f58c40047c0 2023-08-25T09:40:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.920+0000 7f58de438640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58c4005180 0x7f58c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58dd436640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c40047c0 0x7f58c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58ddc37640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4003970 0x7f58c4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58de438640 1 -- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c40047c0 msgr2=0x7f58c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58de438640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c40047c0 0x7f58c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58de438640 1 -- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4003970 msgr2=0x7f58c4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58de438640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4003970 0x7f58c4003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58de438640 1 -- 172.21.15.70:0/1424394492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c4003520 con 0x7f58c4005180 2023-08-25T09:40:00.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.921+0000 7f58dd436640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58c40047c0 0x7f58c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:00.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.922+0000 7f58de438640 1 -- 172.21.15.70:0/1424394492 reap_dead start 2023-08-25T09:40:00.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.922+0000 7f58de438640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58c4005180 0x7f58c400b980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58d815e980 tx=0x7f58c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3793f4cbc709e347 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.922+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0032630 con 0x7f58c4005180 2023-08-25T09:40:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.922+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d0032790 con 0x7f58c4005180 2023-08-25T09:40:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.922+0000 7f58dcc35640 1 -- 172.21.15.70:0/1424394492 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d0032b40 con 0x7f58c4005180 2023-08-25T09:40:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 -- 172.21.15.70:0/1424394492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58c4005180 msgr2=0x7f58c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58c4005180 0x7f58c400b980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58d815e980 tx=0x7f58c40057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 -- 172.21.15.70:0/1424394492 shutdown_connections 2023-08-25T09:40:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/1424394492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58c4005180 0x7f58c400b980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 -- 172.21.15.70:0/1424394492 >> 172.21.15.70:0/1424394492 conn(0x7f58d80cb0d0 msgr2=0x7f58d80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 -- 172.21.15.70:0/1424394492 shutdown_connections 2023-08-25T09:40:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.923+0000 7f58dfe9b640 1 -- 172.21.15.70:0/1424394492 wait complete. 2023-08-25T09:40:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.925+0000 7f58dfe9b640 1 Processor -- start 2023-08-25T09:40:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.925+0000 7f58dfe9b640 1 -- start start 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dfe9b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4005180 0x7f58d81e3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dfe9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 0x7f58d8158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dfe9b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c4750 0x7f58d81e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dfe9b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c36b0 con 0x7f58d80c30f0 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dfe9b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c3990 con 0x7f58c4005180 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dfe9b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80c3c70 con 0x7f58d80c4750 2023-08-25T09:40:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 0x7f58d8158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 0x7f58d8158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47098/0 (socket says 172.21.15.70:47098) 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 -- 172.21.15.70:0/3182605492 learned_addr learned my addr 172.21.15.70:0/3182605492 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58de438640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c4750 0x7f58d81e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58ddc37640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4005180 0x7f58d81e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 -- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c4750 msgr2=0x7f58d81e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c4750 0x7f58d81e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 -- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4005180 msgr2=0x7f58d81e3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4005180 0x7f58d81e3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58dd436640 1 -- 172.21.15.70:0/3182605492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80c3f50 con 0x7f58d80c30f0 2023-08-25T09:40:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.926+0000 7f58de438640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c4750 0x7f58d81e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.927+0000 7f58dd436640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 0x7f58d8158020 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f58c800d590 tx=0x7f58c800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.927+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8007220 con 0x7f58d80c30f0 2023-08-25T09:40:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.927+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c80073a0 con 0x7f58d80c30f0 2023-08-25T09:40:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.927+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d80c4230 con 0x7f58d80c30f0 2023-08-25T09:40:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.927+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c80067d0 con 0x7f58d80c30f0 2023-08-25T09:40:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.927+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d8158560 con 0x7f58d80c30f0 2023-08-25T09:40:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.928+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d80c4fc0 con 0x7f58d80c30f0 2023-08-25T09:40:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.929+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f58c8006930 con 0x7f58d80c30f0 2023-08-25T09:40:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.930+0000 7f58ce7fc640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58bc06a080 0x7f58bc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.930+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58c8088380 con 0x7f58d80c30f0 2023-08-25T09:40:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.931+0000 7f58ddc37640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58bc06a080 0x7f58bc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.931+0000 7f58ddc37640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58bc06a080 0x7f58bc06c540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58d4007c40 tx=0x7f58d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:00.934+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58c80539d0 con 0x7f58d80c30f0 2023-08-25T09:40:01.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.120+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58d8066fb0 con 0x7f58d80c30f0 2023-08-25T09:40:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.121+0000 7f58ce7fc640 1 -- 172.21.15.70:0/3182605492 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58c8056ff0 con 0x7f58d80c30f0 2023-08-25T09:40:01.123 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:01.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.123+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58bc06a080 msgr2=0x7f58bc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.123+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58bc06a080 0x7f58bc06c540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f58d4007c40 tx=0x7f58d4009040 comp rx=0 tx=0).stop 2023-08-25T09:40:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.123+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 msgr2=0x7f58d8158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.123+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 0x7f58d8158020 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f58c800d590 tx=0x7f58c800da50 comp rx=0 tx=0).stop 2023-08-25T09:40:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 shutdown_connections 2023-08-25T09:40:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58bc06a080 0x7f58bc06c540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58d80c4750 0x7f58d81e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58d80c30f0 0x7f58d8158020 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 --2- 172.21.15.70:0/3182605492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58c4005180 0x7f58d81e3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 >> 172.21.15.70:0/3182605492 conn(0x7f58d80cb0d0 msgr2=0x7f58d8159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 shutdown_connections 2023-08-25T09:40:01.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.124+0000 7f58dfe9b640 1 -- 172.21.15.70:0/3182605492 wait complete. 2023-08-25T09:40:01.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:01.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.352+0000 7f13feca7640 1 Processor -- start 2023-08-25T09:40:01.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.352+0000 7f13feca7640 1 -- start start 2023-08-25T09:40:01.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.353+0000 7f13feca7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13f8159bf0 0x7f13f815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.353+0000 7f13feca7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f806f0e0 0x7f13f8070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.353+0000 7f13feca7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f815e780 con 0x7f13f80cd660 2023-08-25T09:40:01.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.353+0000 7f13feca7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f806f8e0 con 0x7f13f8159bf0 2023-08-25T09:40:01.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.353+0000 7f13feca7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f806fbc0 con 0x7f13f806f0e0 2023-08-25T09:40:01.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13f8159bf0 0x7f13f815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f13f8159bf0 0x7f13f815e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47112/0 (socket says 172.21.15.70:47112) 2023-08-25T09:40:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 -- 172.21.15.70:0/3376423495 learned_addr learned my addr 172.21.15.70:0/3376423495 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 -- 172.21.15.70:0/3376423495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f806f0e0 msgr2=0x7f13f8070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 --2- 172.21.15.70:0/3376423495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f806f0e0 0x7f13f8070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 -- 172.21.15.70:0/3376423495 >> v1:172.21.15.155:6789/0 conn(0x7f13f80cd660 legacy=0x7f13f80c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.354+0000 7f13fca43640 1 -- 172.21.15.70:0/3376423495 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f806fea0 con 0x7f13f8159bf0 2023-08-25T09:40:01.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.355+0000 7f13fca43640 1 --2- 172.21.15.70:0/3376423495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8159bf0 0x7f13f815e240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009870 tx=0x7f13ec02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd6373ef43673d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.355+0000 7f13f77fe640 1 -- 172.21.15.70:0/3376423495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec031640 con 0x7f13f8159bf0 2023-08-25T09:40:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.355+0000 7f13f77fe640 1 -- 172.21.15.70:0/3376423495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec0317a0 con 0x7f13f8159bf0 2023-08-25T09:40:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.355+0000 7f13f77fe640 1 -- 172.21.15.70:0/3376423495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec031b30 con 0x7f13f8159bf0 2023-08-25T09:40:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 -- 172.21.15.70:0/3376423495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8159bf0 msgr2=0x7f13f815e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 --2- 172.21.15.70:0/3376423495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8159bf0 0x7f13f815e240 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009870 tx=0x7f13ec02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 -- 172.21.15.70:0/3376423495 shutdown_connections 2023-08-25T09:40:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 --2- 172.21.15.70:0/3376423495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f806f0e0 0x7f13f8070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 --2- 172.21.15.70:0/3376423495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f8159bf0 0x7f13f815e240 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 -- 172.21.15.70:0/3376423495 >> 172.21.15.70:0/3376423495 conn(0x7f13f80cb0d0 msgr2=0x7f13f80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 -- 172.21.15.70:0/3376423495 shutdown_connections 2023-08-25T09:40:01.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.356+0000 7f13feca7640 1 -- 172.21.15.70:0/3376423495 wait complete. 2023-08-25T09:40:01.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.358+0000 7f13feca7640 1 Processor -- start 2023-08-25T09:40:01.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.358+0000 7f13feca7640 1 -- start start 2023-08-25T09:40:01.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13feca7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f806f0e0 0x7f13f81f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13feca7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 0x7f13f81f0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13feca7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f8159bf0 0x7f13f81f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13feca7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f81580a0 con 0x7f13f80cd660 2023-08-25T09:40:01.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13feca7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f81f3470 con 0x7f13f806f0e0 2023-08-25T09:40:01.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13feca7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f81f3c50 con 0x7f13f8159bf0 2023-08-25T09:40:01.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 0x7f13f81f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 0x7f13f81f0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47114/0 (socket says 172.21.15.70:47114) 2023-08-25T09:40:01.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13f7fff640 1 -- 172.21.15.70:0/2733152329 learned_addr learned my addr 172.21.15.70:0/2733152329 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:01.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.359+0000 7f13f7fff640 1 -- 172.21.15.70:0/2733152329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f8159bf0 msgr2=0x7f13f81f1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f7fff640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f8159bf0 0x7f13f81f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f7fff640 1 -- 172.21.15.70:0/2733152329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f806f0e0 msgr2=0x7f13f81f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f7fff640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f806f0e0 0x7f13f81f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f7fff640 1 -- 172.21.15.70:0/2733152329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ec0094d0 con 0x7f13f80cd660 2023-08-25T09:40:01.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f7fff640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 0x7f13f81f0130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13e400b4a0 tx=0x7f13e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:01.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4010610 con 0x7f13f80cd660 2023-08-25T09:40:01.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e4010770 con 0x7f13f80cd660 2023-08-25T09:40:01.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f81f3f30 con 0x7f13f80cd660 2023-08-25T09:40:01.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e4010b20 con 0x7f13f80cd660 2023-08-25T09:40:01.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.360+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f81f44f0 con 0x7f13f80cd660 2023-08-25T09:40:01.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.361+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f8066fb0 con 0x7f13f80cd660 2023-08-25T09:40:01.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.362+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f13e4010c80 con 0x7f13f80cd660 2023-08-25T09:40:01.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.363+0000 7f13f57fa640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f13dc06a050 0x7f13dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.363+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13e40881c0 con 0x7f13f80cd660 2023-08-25T09:40:01.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.363+0000 7f13fca43640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f13dc06a050 0x7f13dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.364+0000 7f13fca43640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f13dc06a050 0x7f13dc06c510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13ec036000 tx=0x7f13ec030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:01.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.367+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13e40538a0 con 0x7f13f80cd660 2023-08-25T09:40:01.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.554+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13f81f0670 con 0x7f13f80cd660 2023-08-25T09:40:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.555+0000 7f13f57fa640 1 -- 172.21.15.70:0/2733152329 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f13e4056ec0 con 0x7f13f80cd660 2023-08-25T09:40:01.558 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f13dc06a050 msgr2=0x7f13dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f13dc06a050 0x7f13dc06c510 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f13ec036000 tx=0x7f13ec030000 comp rx=0 tx=0).stop 2023-08-25T09:40:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 msgr2=0x7f13f81f0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 0x7f13f81f0130 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f13e400b4a0 tx=0x7f13e400b960 comp rx=0 tx=0).stop 2023-08-25T09:40:01.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 shutdown_connections 2023-08-25T09:40:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f13dc06a050 0x7f13dc06c510 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f13f8159bf0 0x7f13f81f1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13f80cd660 0x7f13f81f0130 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 --2- 172.21.15.70:0/2733152329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13f806f0e0 0x7f13f81f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 >> 172.21.15.70:0/2733152329 conn(0x7f13f80cb0d0 msgr2=0x7f13f8069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 shutdown_connections 2023-08-25T09:40:01.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.558+0000 7f13feca7640 1 -- 172.21.15.70:0/2733152329 wait complete. 2023-08-25T09:40:01.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:40:01.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.788+0000 7fa3d7362640 1 Processor -- start 2023-08-25T09:40:01.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.788+0000 7fa3d7362640 1 -- start start 2023-08-25T09:40:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d7362640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3d015abb0 0x7fa3d00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d7362640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d015e4a0 con 0x7fa3d015a7d0 2023-08-25T09:40:01.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d7362640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d015f020 con 0x7fa3d0071850 2023-08-25T09:40:01.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d7362640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d00c53b0 con 0x7fa3d015abb0 2023-08-25T09:40:01.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d6360640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa3d015a7d0 0x7fa3d00b3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54136/0 (socket says 172.21.15.70:54136) 2023-08-25T09:40:01.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d6360640 1 -- 172.21.15.70:0/2934037544 learned_addr learned my addr 172.21.15.70:0/2934037544 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749126369 0 0) 0x7fa3d015e4a0 con 0x7fa3d015a7d0 2023-08-25T09:40:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d5b5f640 1 --2- 172.21.15.70:0/2934037544 >> v2:172.21.15.155:3300/0 conn(0x7fa3d015abb0 0x7fa3d00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.790+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3bc003520 con 0x7fa3d015a7d0 2023-08-25T09:40:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 879105674 0 0) 0x7fa3bc003520 con 0x7fa3d015a7d0 2023-08-25T09:40:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 >> v2:172.21.15.155:3300/0 conn(0x7fa3d015abb0 msgr2=0x7fa3d00b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:01.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 --2- 172.21.15.70:0/2934037544 >> v2:172.21.15.155:3300/0 conn(0x7fa3d015abb0 0x7fa3d00b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 >> v1:172.21.15.155:6789/0 conn(0x7fa3d0071850 legacy=0x7fa3d006ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d015e6f0 con 0x7fa3d015a7d0 2023-08-25T09:40:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa3cc001670 con 0x7fa3d015a7d0 2023-08-25T09:40:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 >> v1:172.21.15.70:6789/0 conn(0x7fa3d015a7d0 legacy=0x7fa3d00b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc0039e0 0x7fa3bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3bc0047c0 0x7fa3bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3bc005330 0x7fa3bc00b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc00c250 con 0x7fa3bc005330 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc00c3c0 con 0x7fa3bc0047c0 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc00c580 con 0x7fa3bc0039e0 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.791+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3bc005330 0x7fa3bc00b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6360640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc0039e0 0x7fa3bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d5b5f640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3bc0047c0 0x7fa3bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6b61640 1 -- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc0039e0 msgr2=0x7fa3bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc0039e0 0x7fa3bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6b61640 1 -- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3bc0047c0 msgr2=0x7fa3bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3bc0047c0 0x7fa3bc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6b61640 1 -- 172.21.15.70:0/2934037544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc003520 con 0x7fa3bc005330 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6360640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc0039e0 0x7fa3bc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:01.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.792+0000 7fa3d6b61640 1 -- 172.21.15.70:0/2934037544 reap_dead start 2023-08-25T09:40:01.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3bc005330 0x7fa3bc00b9a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8001e50 tx=0x7fa3bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e915594acce0b041 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8003b70 con 0x7fa3bc005330 2023-08-25T09:40:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c8003d10 con 0x7fa3bc005330 2023-08-25T09:40:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d535e640 1 -- 172.21.15.70:0/2934037544 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8005310 con 0x7fa3bc005330 2023-08-25T09:40:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d7362640 1 -- 172.21.15.70:0/2934037544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3bc005330 msgr2=0x7fa3bc00b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d7362640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3bc005330 0x7fa3bc00b9a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8001e50 tx=0x7fa3bc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d7362640 1 -- 172.21.15.70:0/2934037544 shutdown_connections 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d7362640 1 --2- 172.21.15.70:0/2934037544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3bc005330 0x7fa3bc00b9a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.793+0000 7fa3d7362640 1 -- 172.21.15.70:0/2934037544 >> 172.21.15.70:0/2934037544 conn(0x7fa3d00aff60 msgr2=0x7fa3d00b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.794+0000 7fa3d7362640 1 -- 172.21.15.70:0/2934037544 shutdown_connections 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.794+0000 7fa3d7362640 1 -- 172.21.15.70:0/2934037544 wait complete. 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.795+0000 7fa3d7362640 1 Processor -- start 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.795+0000 7fa3d7362640 1 -- start start 2023-08-25T09:40:01.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.796+0000 7fa3d7362640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc005330 0x7fa3d0155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.796+0000 7fa3d7362640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d01f24a0 0x7fa3d01f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.796+0000 7fa3d7362640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 0x7fa3d00ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.796+0000 7fa3d7362640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0155fa0 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.796+0000 7fa3d7362640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d00aeb90 con 0x7fa3d01f24a0 2023-08-25T09:40:01.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.796+0000 7fa3d7362640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d01f3fc0 con 0x7fa3bc005330 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 0x7fa3d00ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 0x7fa3d00ae650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47134/0 (socket says 172.21.15.70:47134) 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 -- 172.21.15.70:0/1549627212 learned_addr learned my addr 172.21.15.70:0/1549627212 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 -- 172.21.15.70:0/1549627212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc005330 msgr2=0x7fa3d0155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc005330 0x7fa3d0155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 -- 172.21.15.70:0/1549627212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d01f24a0 msgr2=0x7fa3d01f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d01f24a0 0x7fa3d01f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 -- 172.21.15.70:0/1549627212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d01f3990 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3d6b61640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 0x7fa3d00ae650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa3c802c060 tx=0x7fa3c80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.797+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8006d50 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.798+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c8003b80 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.798+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d01f2fd0 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.798+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8039680 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.798+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d01f3c50 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.799+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d0066fb0 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.800+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3c80397e0 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.801+0000 7fa3c6ffd640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b406a080 0x7fa3b406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.801+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3c80aa570 con 0x7fa3d00ae1d0 2023-08-25T09:40:01.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.801+0000 7fa3d6360640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b406a080 0x7fa3b406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:01.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.802+0000 7fa3d6360640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b406a080 0x7fa3b406c540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000ff0 tx=0x7fa3cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:01.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:01.805+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3c8075c70 con 0x7fa3d00ae1d0 2023-08-25T09:40:02.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.011+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3d01f4100 con 0x7fa3d00ae1d0 2023-08-25T09:40:02.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.012+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1549627212 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa3c8079290 con 0x7fa3d00ae1d0 2023-08-25T09:40:02.014 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:40:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b406a080 msgr2=0x7fa3b406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b406a080 0x7fa3b406c540 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc000ff0 tx=0x7fa3cc009040 comp rx=0 tx=0).stop 2023-08-25T09:40:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 msgr2=0x7fa3d00ae650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 0x7fa3d00ae650 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa3c802c060 tx=0x7fa3c80014a0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 shutdown_connections 2023-08-25T09:40:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b406a080 0x7fa3b406c540 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00ae1d0 0x7fa3d00ae650 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d01f24a0 0x7fa3d01f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 --2- 172.21.15.70:0/1549627212 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3bc005330 0x7fa3d0155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.014+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 >> 172.21.15.70:0/1549627212 conn(0x7fa3d00aff60 msgr2=0x7fa3d0159650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.015+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 shutdown_connections 2023-08-25T09:40:02.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.015+0000 7fa3d7362640 1 -- 172.21.15.70:0/1549627212 wait complete. 2023-08-25T09:40:02.034 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:40:02.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:02.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-08-25T09:40:02.128 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:40:02.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:40:02.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:02.129 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:40:02.186 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:40:02.186 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:02.247 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:40:02.247 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:02.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:02.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.521+0000 7fa1cebf3640 1 Processor -- start 2023-08-25T09:40:02.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.521+0000 7fa1cebf3640 1 -- start start 2023-08-25T09:40:02.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1cebf3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1cebf3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1c806b2b0 0x7fa1c815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1cebf3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c80b1b60 con 0x7fa1c815c6d0 2023-08-25T09:40:02.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1cebf3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c806bd80 con 0x7fa1c806b2b0 2023-08-25T09:40:02.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1cebf3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c806b7c0 con 0x7fa1c815e830 2023-08-25T09:40:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1bffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1c806b2b0 0x7fa1c815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1bffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1c806b2b0 0x7fa1c815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47140/0 (socket says 172.21.15.70:47140) 2023-08-25T09:40:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.523+0000 7fa1bffff640 1 -- 172.21.15.70:0/3555678481 learned_addr learned my addr 172.21.15.70:0/3555678481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.524+0000 7fa1cc98f640 1 --2- 172.21.15.70:0/3555678481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.524+0000 7fa1bffff640 1 -- 172.21.15.70:0/3555678481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 msgr2=0x7fa1c806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.524+0000 7fa1bffff640 1 --2- 172.21.15.70:0/3555678481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.524+0000 7fa1bffff640 1 -- 172.21.15.70:0/3555678481 >> v1:172.21.15.70:6789/0 conn(0x7fa1c815c6d0 legacy=0x7fa1c80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.524+0000 7fa1bffff640 1 -- 172.21.15.70:0/3555678481 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c806baa0 con 0x7fa1c806b2b0 2023-08-25T09:40:02.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.524+0000 7fa1bf7fe640 1 -- 172.21.15.70:0/3555678481 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048073234 0 0) 0x7fa1c80b1b60 con 0x7fa1c815c6d0 2023-08-25T09:40:02.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.525+0000 7fa1bffff640 1 --2- 172.21.15.70:0/3555678481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c815c190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009870 tx=0x7fa1b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c98d588b9749ca7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:02.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.525+0000 7fa1bf7fe640 1 -- 172.21.15.70:0/3555678481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8030840 con 0x7fa1c806b2b0 2023-08-25T09:40:02.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.525+0000 7fa1bf7fe640 1 -- 172.21.15.70:0/3555678481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b80309a0 con 0x7fa1c806b2b0 2023-08-25T09:40:02.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.525+0000 7fa1bf7fe640 1 -- 172.21.15.70:0/3555678481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8030d30 con 0x7fa1c806b2b0 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 -- 172.21.15.70:0/3555678481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 msgr2=0x7fa1c815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/3555678481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c815c190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8009870 tx=0x7fa1b802b980 comp rx=0 tx=0).stop 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 -- 172.21.15.70:0/3555678481 shutdown_connections 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/3555678481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c815c190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/3555678481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 -- 172.21.15.70:0/3555678481 >> 172.21.15.70:0/3555678481 conn(0x7fa1c8069850 msgr2=0x7fa1c8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:02.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 -- 172.21.15.70:0/3555678481 shutdown_connections 2023-08-25T09:40:02.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.526+0000 7fa1cebf3640 1 -- 172.21.15.70:0/3555678481 wait complete. 2023-08-25T09:40:02.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.528+0000 7fa1cebf3640 1 Processor -- start 2023-08-25T09:40:02.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.529+0000 7fa1cebf3640 1 -- start start 2023-08-25T09:40:02.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cebf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c81437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cebf3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c815c6d0 0x7fa1c8144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cebf3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cebf3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c81533d0 con 0x7fa1c806b2b0 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cebf3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8143d30 con 0x7fa1c815c6d0 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cebf3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8152b70 con 0x7fa1c815e830 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cc98f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c81437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cc98f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c81437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47156/0 (socket says 172.21.15.70:47156) 2023-08-25T09:40:02.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cc98f640 1 -- 172.21.15.70:0/4275930641 learned_addr learned my addr 172.21.15.70:0/4275930641 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:02.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1bffff640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c815c6d0 0x7fa1c8144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cc98f640 1 -- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 msgr2=0x7fa1c806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.530+0000 7fa1cd190640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cc98f640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cc98f640 1 -- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c815c6d0 msgr2=0x7fa1c8144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cc98f640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c815c6d0 0x7fa1c8144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cc98f640 1 -- 172.21.15.70:0/4275930641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b80094d0 con 0x7fa1c806b2b0 2023-08-25T09:40:02.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1bffff640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c815c6d0 0x7fa1c8144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cc98f640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c81437f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400d620 tx=0x7fa1c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400b620 con 0x7fa1c806b2b0 2023-08-25T09:40:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c400b780 con 0x7fa1c806b2b0 2023-08-25T09:40:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400bb30 con 0x7fa1c806b2b0 2023-08-25T09:40:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c8152e10 con 0x7fa1c806b2b0 2023-08-25T09:40:02.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.531+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c806f870 con 0x7fa1c806b2b0 2023-08-25T09:40:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.533+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c806fa30 con 0x7fa1c806b2b0 2023-08-25T09:40:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.533+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa1c400bc90 con 0x7fa1c806b2b0 2023-08-25T09:40:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.534+0000 7fa1bd7fa640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1b406a080 0x7fa1b406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.534+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa1c4088a50 con 0x7fa1c806b2b0 2023-08-25T09:40:02.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.535+0000 7fa1bffff640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1b406a080 0x7fa1b406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.535+0000 7fa1bffff640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1b406a080 0x7fa1b406c540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8036000 tx=0x7fa1b803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:02.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.539+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa1c4054140 con 0x7fa1c806b2b0 2023-08-25T09:40:02.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.727+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1c806f680 con 0x7fa1c806b2b0 2023-08-25T09:40:02.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.729+0000 7fa1bd7fa640 1 -- 172.21.15.70:0/4275930641 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fa1c4057760 con 0x7fa1c806b2b0 2023-08-25T09:40:02.733 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:02.733 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.732+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1b406a080 msgr2=0x7fa1b406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.732+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1b406a080 0x7fa1b406c540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1b8036000 tx=0x7fa1b803b040 comp rx=0 tx=0).stop 2023-08-25T09:40:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.732+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 msgr2=0x7fa1c81437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.732+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c81437f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400d620 tx=0x7fa1c400dae0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 shutdown_connections 2023-08-25T09:40:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1b406a080 0x7fa1b406c540 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1c815e830 0x7fa1c806fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1c815c6d0 0x7fa1c8144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 --2- 172.21.15.70:0/4275930641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1c806b2b0 0x7fa1c81437f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 >> 172.21.15.70:0/4275930641 conn(0x7fa1c8069850 msgr2=0x7fa1c80b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 shutdown_connections 2023-08-25T09:40:02.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.733+0000 7fa1cebf3640 1 -- 172.21.15.70:0/4275930641 wait complete. 2023-08-25T09:40:02.752 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:02.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.953+0000 7f77c3722640 1 Processor -- start 2023-08-25T09:40:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.953+0000 7f77c3722640 1 -- start start 2023-08-25T09:40:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.954+0000 7f77c3722640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160290 0x7f77bc0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.954+0000 7f77c3722640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77bc069850 0x7f77bc0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c3722640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77bc071c10 con 0x7f77bc160670 2023-08-25T09:40:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c3722640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77bc071000 con 0x7f77bc069850 2023-08-25T09:40:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c3722640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0712e0 con 0x7f77bc160290 2023-08-25T09:40:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77bc069850 0x7f77bc0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77bc069850 0x7f77bc0d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47158/0 (socket says 172.21.15.70:47158) 2023-08-25T09:40:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 -- 172.21.15.70:0/4039489415 learned_addr learned my addr 172.21.15.70:0/4039489415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c14be640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160290 0x7f77bc0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 -- 172.21.15.70:0/4039489415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160290 msgr2=0x7f77bc0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160290 0x7f77bc0cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 -- 172.21.15.70:0/4039489415 >> v1:172.21.15.155:6789/0 conn(0x7f77bc160670 legacy=0x7f77bc0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c1cbf640 1 -- 172.21.15.70:0/4039489415 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc0715c0 con 0x7f77bc069850 2023-08-25T09:40:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.955+0000 7f77c14be640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160290 0x7f77bc0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.957+0000 7f77c1cbf640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc069850 0x7f77bc0d0380 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009e50 tx=0x7f77b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c05e4417db7a7a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.957+0000 7f77b3fff640 1 -- 172.21.15.70:0/4039489415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b402ca80 con 0x7f77bc069850 2023-08-25T09:40:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.957+0000 7f77b3fff640 1 -- 172.21.15.70:0/4039489415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b402cbe0 con 0x7f77bc069850 2023-08-25T09:40:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.957+0000 7f77b3fff640 1 -- 172.21.15.70:0/4039489415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b40343f0 con 0x7f77bc069850 2023-08-25T09:40:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 -- 172.21.15.70:0/4039489415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc069850 msgr2=0x7f77bc0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc069850 0x7f77bc0d0380 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009e50 tx=0x7f77b402bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 -- 172.21.15.70:0/4039489415 shutdown_connections 2023-08-25T09:40:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc069850 0x7f77bc0d0380 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 --2- 172.21.15.70:0/4039489415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160290 0x7f77bc0cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 -- 172.21.15.70:0/4039489415 >> 172.21.15.70:0/4039489415 conn(0x7f77bc06ba60 msgr2=0x7f77bc06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 -- 172.21.15.70:0/4039489415 shutdown_connections 2023-08-25T09:40:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.958+0000 7f77c3722640 1 -- 172.21.15.70:0/4039489415 wait complete. 2023-08-25T09:40:02.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.960+0000 7f77c3722640 1 Processor -- start 2023-08-25T09:40:02.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.960+0000 7f77c3722640 1 -- start start 2023-08-25T09:40:02.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c3722640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77bc069850 0x7f77bc1f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c3722640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 0x7f77bc14dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c3722640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160670 0x7f77bc0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c3722640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc1f33a0 con 0x7f77bc160290 2023-08-25T09:40:02.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c3722640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0b0990 con 0x7f77bc069850 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c3722640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc0b0c70 con 0x7f77bc160670 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c0cbd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 0x7f77bc14dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c0cbd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 0x7f77bc14dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47162/0 (socket says 172.21.15.70:47162) 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.961+0000 7f77c0cbd640 1 -- 172.21.15.70:0/1364219121 learned_addr learned my addr 172.21.15.70:0/1364219121 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c14be640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77bc069850 0x7f77bc1f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c0cbd640 1 -- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160670 msgr2=0x7f77bc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c0cbd640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160670 0x7f77bc0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c0cbd640 1 -- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77bc069850 msgr2=0x7f77bc1f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:02.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c0cbd640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77bc069850 0x7f77bc1f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:02.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c0cbd640 1 -- 172.21.15.70:0/1364219121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b4009b30 con 0x7f77bc160290 2023-08-25T09:40:02.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c14be640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77bc069850 0x7f77bc1f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c0cbd640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 0x7f77bc14dd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b3f0 tx=0x7f77ac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac011830 con 0x7f77bc160290 2023-08-25T09:40:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ac011990 con 0x7f77bc160290 2023-08-25T09:40:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ac011d40 con 0x7f77bc160290 2023-08-25T09:40:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77bc0b0f50 con 0x7f77bc160290 2023-08-25T09:40:02.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.962+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77bc0b1510 con 0x7f77bc160290 2023-08-25T09:40:02.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.964+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc066fb0 con 0x7f77bc160290 2023-08-25T09:40:02.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.964+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77ac01a400 con 0x7f77bc160290 2023-08-25T09:40:02.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.965+0000 7f77b1ffb640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77a006a080 0x7f77a006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:02.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.965+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77ac0883c0 con 0x7f77bc160290 2023-08-25T09:40:02.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.966+0000 7f77c14be640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77a006a080 0x7f77a006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:02.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.966+0000 7f77c14be640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77a006a080 0x7f77a006c540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f77b8006fd0 tx=0x7f77b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:02.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:02.970+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77ac053aa0 con 0x7f77bc160290 2023-08-25T09:40:03.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.156+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77bc0b68f0 con 0x7f77bc160290 2023-08-25T09:40:03.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.157+0000 7f77b1ffb640 1 -- 172.21.15.70:0/1364219121 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77ac0570c0 con 0x7f77bc160290 2023-08-25T09:40:03.161 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77a006a080 msgr2=0x7f77a006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77a006a080 0x7f77a006c540 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f77b8006fd0 tx=0x7f77b8008040 comp rx=0 tx=0).stop 2023-08-25T09:40:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 msgr2=0x7f77bc14dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 0x7f77bc14dd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00b3f0 tx=0x7f77ac00b8b0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 shutdown_connections 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77a006a080 0x7f77a006c540 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77bc160670 0x7f77bc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77bc160290 0x7f77bc14dd50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 --2- 172.21.15.70:0/1364219121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77bc069850 0x7f77bc1f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 >> 172.21.15.70:0/1364219121 conn(0x7f77bc06ba60 msgr2=0x7f77bc0adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.160+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 shutdown_connections 2023-08-25T09:40:03.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.161+0000 7f77c3722640 1 -- 172.21.15.70:0/1364219121 wait complete. 2023-08-25T09:40:03.181 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:03.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.395+0000 7f0226f0f640 1 Processor -- start 2023-08-25T09:40:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.395+0000 7f0226f0f640 1 -- start start 2023-08-25T09:40:03.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.396+0000 7f0226f0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022006ad30 0x7f022006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.396+0000 7f0226f0f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02200c9790 0x7f02200ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.396+0000 7f0226f0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022006b650 con 0x7f02200c9c10 2023-08-25T09:40:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.396+0000 7f0226f0f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02201597d0 con 0x7f02200c9790 2023-08-25T09:40:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.396+0000 7f0226f0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022015a5b0 con 0x7f022006ad30 2023-08-25T09:40:03.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f021ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02200c9790 0x7f02200ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f02254ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02200c9c10 0x7f02200cc550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54160/0 (socket says 172.21.15.70:54160) 2023-08-25T09:40:03.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f02254ac640 1 -- 172.21.15.70:0/3943689862 learned_addr learned my addr 172.21.15.70:0/3943689862 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:03.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f0224cab640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022006ad30 0x7f022006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f021ffff640 1 -- 172.21.15.70:0/3943689862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022006ad30 msgr2=0x7f022006b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f021ffff640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022006ad30 0x7f022006b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f021ffff640 1 -- 172.21.15.70:0/3943689862 >> v1:172.21.15.70:6789/0 conn(0x7f02200c9c10 legacy=0x7f02200cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f021ffff640 1 -- 172.21.15.70:0/3943689862 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022015a890 con 0x7f02200c9790 2023-08-25T09:40:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.397+0000 7f0224cab640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022006ad30 0x7f022006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:03.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.398+0000 7f021ffff640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9790 0x7f02200ca2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f020c00a940 tx=0x7f020c02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b96508ae4be02d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:03.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.398+0000 7f021f7fe640 1 -- 172.21.15.70:0/3943689862 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c02db40 con 0x7f02200c9790 2023-08-25T09:40:03.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.398+0000 7f021f7fe640 1 -- 172.21.15.70:0/3943689862 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020c02dca0 con 0x7f02200c9790 2023-08-25T09:40:03.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.398+0000 7f021f7fe640 1 -- 172.21.15.70:0/3943689862 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020c035660 con 0x7f02200c9790 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 -- 172.21.15.70:0/3943689862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9790 msgr2=0x7f02200ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9790 0x7f02200ca2f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f020c00a940 tx=0x7f020c02cd90 comp rx=0 tx=0).stop 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 -- 172.21.15.70:0/3943689862 shutdown_connections 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9790 0x7f02200ca2f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 --2- 172.21.15.70:0/3943689862 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f022006ad30 0x7f022006b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 -- 172.21.15.70:0/3943689862 >> 172.21.15.70:0/3943689862 conn(0x7f02200cb0d0 msgr2=0x7f02200cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 -- 172.21.15.70:0/3943689862 shutdown_connections 2023-08-25T09:40:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.399+0000 7f0226f0f640 1 -- 172.21.15.70:0/3943689862 wait complete. 2023-08-25T09:40:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.401+0000 7f0226f0f640 1 Processor -- start 2023-08-25T09:40:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.401+0000 7f0226f0f640 1 -- start start 2023-08-25T09:40:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f0226f0f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022006ad30 0x7f022014f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f0226f0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02200c9790 0x7f02200c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f0226f0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 0x7f02200ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f0226f0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b3640 con 0x7f02200c9c10 2023-08-25T09:40:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f0226f0f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b3920 con 0x7f022006ad30 2023-08-25T09:40:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f0226f0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02200b3c00 con 0x7f02200c9790 2023-08-25T09:40:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f02254ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 0x7f02200ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f02254ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 0x7f02200ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47180/0 (socket says 172.21.15.70:47180) 2023-08-25T09:40:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.402+0000 7f02254ac640 1 -- 172.21.15.70:0/2703141209 learned_addr learned my addr 172.21.15.70:0/2703141209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f0224cab640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022006ad30 0x7f022014f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f021ffff640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02200c9790 0x7f02200c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f02254ac640 1 -- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02200c9790 msgr2=0x7f02200c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f02254ac640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02200c9790 0x7f02200c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f02254ac640 1 -- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022006ad30 msgr2=0x7f022014f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f02254ac640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022006ad30 0x7f022014f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f02254ac640 1 -- 172.21.15.70:0/2703141209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c00a5a0 con 0x7f02200c9c10 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f0224cab640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022006ad30 0x7f022014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f02254ac640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 0x7f02200ac880 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f021800c9f0 tx=0x7f021800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800ab70 con 0x7f02200c9c10 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021800acd0 con 0x7f02200c9c10 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218012600 con 0x7f02200c9c10 2023-08-25T09:40:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02200b3ee0 con 0x7f02200c9c10 2023-08-25T09:40:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.403+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02200c5bd0 con 0x7f02200c9c10 2023-08-25T09:40:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.404+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02200b40d0 con 0x7f02200c9c10 2023-08-25T09:40:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.405+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0218007070 con 0x7f02200c9c10 2023-08-25T09:40:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.406+0000 7f021d7fa640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f020406a080 0x7f020406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.406+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f02180882a0 con 0x7f02200c9c10 2023-08-25T09:40:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.406+0000 7f0224cab640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f020406a080 0x7f020406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.407+0000 7f0224cab640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f020406a080 0x7f020406c540 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0214006fd0 tx=0x7f0214008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.410+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0218053a30 con 0x7f02200c9c10 2023-08-25T09:40:03.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.584+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0220066fb0 con 0x7f02200c9c10 2023-08-25T09:40:03.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.584+0000 7f021d7fa640 1 -- 172.21.15.70:0/2703141209 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0218057050 con 0x7f02200c9c10 2023-08-25T09:40:03.587 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:03.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.586+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f020406a080 msgr2=0x7f020406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.586+0000 7f0226f0f640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f020406a080 0x7f020406c540 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0214006fd0 tx=0x7f0214008040 comp rx=0 tx=0).stop 2023-08-25T09:40:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.586+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 msgr2=0x7f02200ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.586+0000 7f0226f0f640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 0x7f02200ac880 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f021800c9f0 tx=0x7f021800ceb0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 shutdown_connections 2023-08-25T09:40:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f020406a080 0x7f020406c540 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02200c9c10 0x7f02200ac880 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02200c9790 0x7f02200c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 --2- 172.21.15.70:0/2703141209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f022006ad30 0x7f022014f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 >> 172.21.15.70:0/2703141209 conn(0x7f02200cb0d0 msgr2=0x7f02200cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 shutdown_connections 2023-08-25T09:40:03.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.587+0000 7f0226f0f640 1 -- 172.21.15.70:0/2703141209 wait complete. 2023-08-25T09:40:03.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.819+0000 7f6f7ea46640 1 Processor -- start 2023-08-25T09:40:03.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.819+0000 7f6f7ea46640 1 -- start start 2023-08-25T09:40:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f7ea46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6f7815ae50 0x7f6f780c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f7ea46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f780cdc20 0x7f6f78069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f7ea46640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780cc5f0 con 0x7f6f780ce060 2023-08-25T09:40:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f7ea46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f78070650 con 0x7f6f780cdc20 2023-08-25T09:40:03.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f7ea46640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f78070930 con 0x7f6f7815ae50 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f6ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f780cdc20 0x7f6f78069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f6ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6f780cdc20 0x7f6f78069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47186/0 (socket says 172.21.15.70:47186) 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.821+0000 7f6f6ffff640 1 -- 172.21.15.70:0/604926453 learned_addr learned my addr 172.21.15.70:0/604926453 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.822+0000 7f6f77fff640 1 --2- 172.21.15.70:0/604926453 >> v2:172.21.15.155:3300/0 conn(0x7f6f7815ae50 0x7f6f780c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.822+0000 7f6f6ffff640 1 -- 172.21.15.70:0/604926453 >> v2:172.21.15.155:3300/0 conn(0x7f6f7815ae50 msgr2=0x7f6f780c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.822+0000 7f6f6ffff640 1 --2- 172.21.15.70:0/604926453 >> v2:172.21.15.155:3300/0 conn(0x7f6f7815ae50 0x7f6f780c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.822+0000 7f6f6ffff640 1 -- 172.21.15.70:0/604926453 >> v1:172.21.15.155:6789/0 conn(0x7f6f780ce060 legacy=0x7f6f780ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.822+0000 7f6f6ffff640 1 -- 172.21.15.70:0/604926453 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78070c10 con 0x7f6f780cdc20 2023-08-25T09:40:03.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.822+0000 7f6f77fff640 1 --2- 172.21.15.70:0/604926453 >> v2:172.21.15.155:3300/0 conn(0x7f6f7815ae50 0x7f6f780c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.823+0000 7f6f6ffff640 1 --2- 172.21.15.70:0/604926453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f780cdc20 0x7f6f78069850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b1210 tx=0x7f6f6002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af76f0ac95ad29f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.823+0000 7f6f777fe640 1 -- 172.21.15.70:0/604926453 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60030860 con 0x7f6f780cdc20 2023-08-25T09:40:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.823+0000 7f6f777fe640 1 -- 172.21.15.70:0/604926453 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f600309c0 con 0x7f6f780cdc20 2023-08-25T09:40:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.823+0000 7f6f777fe640 1 -- 172.21.15.70:0/604926453 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60030d50 con 0x7f6f780cdc20 2023-08-25T09:40:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 -- 172.21.15.70:0/604926453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f780cdc20 msgr2=0x7f6f78069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/604926453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f780cdc20 0x7f6f78069850 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b1210 tx=0x7f6f6002b9a0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 -- 172.21.15.70:0/604926453 shutdown_connections 2023-08-25T09:40:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/604926453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f780cdc20 0x7f6f78069850 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/604926453 >> v2:172.21.15.155:3300/0 conn(0x7f6f7815ae50 0x7f6f780c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 -- 172.21.15.70:0/604926453 >> 172.21.15.70:0/604926453 conn(0x7f6f780cb0d0 msgr2=0x7f6f780cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 -- 172.21.15.70:0/604926453 shutdown_connections 2023-08-25T09:40:03.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.824+0000 7f6f7ea46640 1 -- 172.21.15.70:0/604926453 wait complete. 2023-08-25T09:40:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.826+0000 7f6f7ea46640 1 Processor -- start 2023-08-25T09:40:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.826+0000 7f6f7ea46640 1 -- start start 2023-08-25T09:40:03.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.827+0000 7f6f7ea46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f780cdc20 0x7f6f780b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.827+0000 7f6f7ea46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f780ce060 0x7f6f780c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.827+0000 7f6f7ea46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 0x7f6f780c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.827+0000 7f6f7ea46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780c6f60 con 0x7f6f7815ae50 2023-08-25T09:40:03.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.827+0000 7f6f7ea46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781541b0 con 0x7f6f780cdc20 2023-08-25T09:40:03.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.827+0000 7f6f7ea46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780c62e0 con 0x7f6f780ce060 2023-08-25T09:40:03.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 0x7f6f780c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 0x7f6f780c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47194/0 (socket says 172.21.15.70:47194) 2023-08-25T09:40:03.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 -- 172.21.15.70:0/3022470047 learned_addr learned my addr 172.21.15.70:0/3022470047 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 -- 172.21.15.70:0/3022470047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f780ce060 msgr2=0x7f6f780c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f780ce060 0x7f6f780c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 -- 172.21.15.70:0/3022470047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f780cdc20 msgr2=0x7f6f780b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f780cdc20 0x7f6f780b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 -- 172.21.15.70:0/3022470047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600094d0 con 0x7f6f7815ae50 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.828+0000 7f6f7cfe3640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 0x7f6f780c7d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000c780 tx=0x7f6f7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:03.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.829+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70003d80 con 0x7f6f7815ae50 2023-08-25T09:40:03.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.829+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7000ce90 con 0x7f6f7815ae50 2023-08-25T09:40:03.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.829+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700052b0 con 0x7f6f7815ae50 2023-08-25T09:40:03.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.829+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f780c65c0 con 0x7f6f7815ae50 2023-08-25T09:40:03.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.829+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f780c6b80 con 0x7f6f7815ae50 2023-08-25T09:40:03.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.830+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f780c70a0 con 0x7f6f7815ae50 2023-08-25T09:40:03.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.831+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6f70005410 con 0x7f6f7815ae50 2023-08-25T09:40:03.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.832+0000 7f6f757fa640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6f5806a050 0x7f6f5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:03.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.832+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6f70087b60 con 0x7f6f7815ae50 2023-08-25T09:40:03.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.832+0000 7f6f77fff640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6f5806a050 0x7f6f5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:03.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.833+0000 7f6f77fff640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6f5806a050 0x7f6f5806c510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006fd0 tx=0x7f6f68008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:03.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:03.838+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6f780c70a0 con 0x7f6f7815ae50 2023-08-25T09:40:04.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.028+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f780c70a0 con 0x7f6f7815ae50 2023-08-25T09:40:04.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.029+0000 7f6f757fa640 1 -- 172.21.15.70:0/3022470047 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f70017020 con 0x7f6f7815ae50 2023-08-25T09:40:04.031 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:04.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.031+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6f5806a050 msgr2=0x7f6f5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.031+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6f5806a050 0x7f6f5806c510 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f68006fd0 tx=0x7f6f68008040 comp rx=0 tx=0).stop 2023-08-25T09:40:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.031+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 msgr2=0x7f6f780c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.031+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 0x7f6f780c7d70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f7000c780 tx=0x7f6f7000cc40 comp rx=0 tx=0).stop 2023-08-25T09:40:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 shutdown_connections 2023-08-25T09:40:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6f5806a050 0x7f6f5806c510 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6f7815ae50 0x7f6f780c7d70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6f780ce060 0x7f6f780c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 --2- 172.21.15.70:0/3022470047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6f780cdc20 0x7f6f780b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 >> 172.21.15.70:0/3022470047 conn(0x7f6f780cb0d0 msgr2=0x7f6f780ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 shutdown_connections 2023-08-25T09:40:04.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.032+0000 7f6f7ea46640 1 -- 172.21.15.70:0/3022470047 wait complete. 2023-08-25T09:40:04.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:04.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.268+0000 7f8391fd6640 1 Processor -- start 2023-08-25T09:40:04.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.268+0000 7f8391fd6640 1 -- start start 2023-08-25T09:40:04.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f8391fd6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f838c072e70 0x7f838c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f8391fd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f838c06aea0 0x7f838c06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f8391fd6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c0ceb30 con 0x7f838c072a90 2023-08-25T09:40:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f8391fd6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0cee10 con 0x7f838c06aea0 2023-08-25T09:40:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f8391fd6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0cf0f0 con 0x7f838c072e70 2023-08-25T09:40:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f838bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f838c06aea0 0x7f838c06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f838bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f838c06aea0 0x7f838c06c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47208/0 (socket says 172.21.15.70:47208) 2023-08-25T09:40:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.270+0000 7f838bfff640 1 -- 172.21.15.70:0/4200732259 learned_addr learned my addr 172.21.15.70:0/4200732259 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.271+0000 7f838affd640 1 --2- 172.21.15.70:0/4200732259 >> v2:172.21.15.155:3300/0 conn(0x7f838c072e70 0x7f838c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.271+0000 7f838bfff640 1 -- 172.21.15.70:0/4200732259 >> v2:172.21.15.155:3300/0 conn(0x7f838c072e70 msgr2=0x7f838c06a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:04.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.271+0000 7f838bfff640 1 --2- 172.21.15.70:0/4200732259 >> v2:172.21.15.155:3300/0 conn(0x7f838c072e70 0x7f838c06a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.271+0000 7f838bfff640 1 -- 172.21.15.70:0/4200732259 >> v1:172.21.15.155:6789/0 conn(0x7f838c072a90 legacy=0x7f838c06b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:04.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.271+0000 7f838bfff640 1 -- 172.21.15.70:0/4200732259 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0cf3d0 con 0x7f838c06aea0 2023-08-25T09:40:04.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.272+0000 7f838bfff640 1 --2- 172.21.15.70:0/4200732259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c06aea0 0x7f838c06c250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8384009e50 tx=0x7f838402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d0f967a808b988 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:04.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.273+0000 7f838a7fc640 1 -- 172.21.15.70:0/4200732259 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838402ca70 con 0x7f838c06aea0 2023-08-25T09:40:04.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f838a7fc640 1 -- 172.21.15.70:0/4200732259 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838402cbd0 con 0x7f838c06aea0 2023-08-25T09:40:04.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f838a7fc640 1 -- 172.21.15.70:0/4200732259 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83840343f0 con 0x7f838c06aea0 2023-08-25T09:40:04.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f8391fd6640 1 -- 172.21.15.70:0/4200732259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c06aea0 msgr2=0x7f838c06c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4200732259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c06aea0 0x7f838c06c250 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8384009e50 tx=0x7f838402bed0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f8391fd6640 1 -- 172.21.15.70:0/4200732259 shutdown_connections 2023-08-25T09:40:04.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4200732259 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c06aea0 0x7f838c06c250 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4200732259 >> v2:172.21.15.155:3300/0 conn(0x7f838c072e70 0x7f838c06a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.274+0000 7f8391fd6640 1 -- 172.21.15.70:0/4200732259 >> 172.21.15.70:0/4200732259 conn(0x7f838c0cf9c0 msgr2=0x7f838c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:04.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.275+0000 7f8391fd6640 1 -- 172.21.15.70:0/4200732259 shutdown_connections 2023-08-25T09:40:04.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.275+0000 7f8391fd6640 1 -- 172.21.15.70:0/4200732259 wait complete. 2023-08-25T09:40:04.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.276+0000 7f8391fd6640 1 Processor -- start 2023-08-25T09:40:04.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.277+0000 7f8391fd6640 1 -- start start 2023-08-25T09:40:04.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.277+0000 7f8391fd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838c072a90 0x7f838c0c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f8391fd6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f838c072e70 0x7f838c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f8391fd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 0x7f838c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f8391fd6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c65b0 con 0x7f838c0c6120 2023-08-25T09:40:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f8391fd6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c4290 con 0x7f838c072e70 2023-08-25T09:40:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f8391fd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c4c60 con 0x7f838c072a90 2023-08-25T09:40:04.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 0x7f838c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 0x7f838c0ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47220/0 (socket says 172.21.15.70:47220) 2023-08-25T09:40:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 -- 172.21.15.70:0/4138371960 learned_addr learned my addr 172.21.15.70:0/4138371960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838b7fe640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838c072a90 0x7f838c0c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838affd640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f838c072e70 0x7f838c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 -- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838c072a90 msgr2=0x7f838c0c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838c072a90 0x7f838c0c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 -- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f838c072e70 msgr2=0x7f838c0c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f838c072e70 0x7f838c0c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838bfff640 1 -- 172.21.15.70:0/4138371960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8384009b30 con 0x7f838c0c6120 2023-08-25T09:40:04.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.278+0000 7f838affd640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f838c072e70 0x7f838c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:04.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.279+0000 7f838bfff640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 0x7f838c0ac880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8384036040 tx=0x7f8384004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:04.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.279+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384009510 con 0x7f838c0c6120 2023-08-25T09:40:04.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.279+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838403e450 con 0x7f838c0c6120 2023-08-25T09:40:04.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.279+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c0c3680 con 0x7f838c0c6120 2023-08-25T09:40:04.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.279+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838403e7e0 con 0x7f838c0c6120 2023-08-25T09:40:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.279+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c0c3c40 con 0x7f838c0c6120 2023-08-25T09:40:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.280+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c0c4420 con 0x7f838c0c6120 2023-08-25T09:40:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.281+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f838403e940 con 0x7f838c0c6120 2023-08-25T09:40:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.282+0000 7f8373fff640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f837806a050 0x7f837806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.282+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83840ab3b0 con 0x7f838c0c6120 2023-08-25T09:40:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.282+0000 7f838b7fe640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f837806a050 0x7f837806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.283+0000 7f838b7fe640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f837806a050 0x7f837806c510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f837c007fd0 tx=0x7f837c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:04.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.286+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8384076b20 con 0x7f838c0c6120 2023-08-25T09:40:04.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.474+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f838c066fb0 con 0x7f838c0c6120 2023-08-25T09:40:04.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.475+0000 7f8373fff640 1 -- 172.21.15.70:0/4138371960 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f838407a140 con 0x7f838c0c6120 2023-08-25T09:40:04.478 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:04.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.477+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f837806a050 msgr2=0x7f837806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.477+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f837806a050 0x7f837806c510 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f837c007fd0 tx=0x7f837c009040 comp rx=0 tx=0).stop 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 msgr2=0x7f838c0ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 0x7f838c0ac880 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8384036040 tx=0x7f8384004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 shutdown_connections 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f837806a050 0x7f837806c510 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f838c0c6120 0x7f838c0ac880 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f838c072e70 0x7f838c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 --2- 172.21.15.70:0/4138371960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f838c072a90 0x7f838c0c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 >> 172.21.15.70:0/4138371960 conn(0x7f838c0cf9c0 msgr2=0x7f838c15ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:04.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 shutdown_connections 2023-08-25T09:40:04.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.478+0000 7f8391fd6640 1 -- 172.21.15.70:0/4138371960 wait complete. 2023-08-25T09:40:04.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:40:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.709+0000 7f2532e8e640 1 Processor -- start 2023-08-25T09:40:04.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.710+0000 7f2532e8e640 1 -- start start 2023-08-25T09:40:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.711+0000 7f2532e8e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f252c06ad20 0x7f252c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.711+0000 7f2532e8e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f252c0caf30 0x7f252c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.711+0000 7f2532e8e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252c0cdd60 con 0x7f252c15cd90 2023-08-25T09:40:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.711+0000 7f2532e8e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c0cc9e0 con 0x7f252c06ad20 2023-08-25T09:40:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.711+0000 7f2532e8e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252c0cccc0 con 0x7f252c0caf30 2023-08-25T09:40:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f252c06ad20 0x7f252c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f252c06ad20 0x7f252c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47228/0 (socket says 172.21.15.70:47228) 2023-08-25T09:40:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 -- 172.21.15.70:0/2025798312 learned_addr learned my addr 172.21.15.70:0/2025798312 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f253142b640 1 --2- 172.21.15.70:0/2025798312 >> v2:172.21.15.155:3300/0 conn(0x7f252c0caf30 0x7f252c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 -- 172.21.15.70:0/2025798312 >> v2:172.21.15.155:3300/0 conn(0x7f252c0caf30 msgr2=0x7f252c0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 --2- 172.21.15.70:0/2025798312 >> v2:172.21.15.155:3300/0 conn(0x7f252c0caf30 0x7f252c0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 -- 172.21.15.70:0/2025798312 >> v1:172.21.15.155:6789/0 conn(0x7f252c15cd90 legacy=0x7f252c0718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:04.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.712+0000 7f2523fff640 1 -- 172.21.15.70:0/2025798312 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c0ccfa0 con 0x7f252c06ad20 2023-08-25T09:40:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.713+0000 7f2523fff640 1 --2- 172.21.15.70:0/2025798312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c06ad20 0x7f252c06b1b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f252c069ce0 tx=0x7f251c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8215c6363ebb3e96 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f25237fe640 1 -- 172.21.15.70:0/2025798312 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c031870 con 0x7f252c06ad20 2023-08-25T09:40:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f25237fe640 1 -- 172.21.15.70:0/2025798312 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c0319d0 con 0x7f252c06ad20 2023-08-25T09:40:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f25237fe640 1 -- 172.21.15.70:0/2025798312 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c031d60 con 0x7f252c06ad20 2023-08-25T09:40:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 -- 172.21.15.70:0/2025798312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c06ad20 msgr2=0x7f252c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 --2- 172.21.15.70:0/2025798312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c06ad20 0x7f252c06b1b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f252c069ce0 tx=0x7f251c02b910 comp rx=0 tx=0).stop 2023-08-25T09:40:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 -- 172.21.15.70:0/2025798312 shutdown_connections 2023-08-25T09:40:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 --2- 172.21.15.70:0/2025798312 >> v2:172.21.15.155:3300/0 conn(0x7f252c0caf30 0x7f252c0cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 --2- 172.21.15.70:0/2025798312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c06ad20 0x7f252c06b1b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 -- 172.21.15.70:0/2025798312 >> 172.21.15.70:0/2025798312 conn(0x7f252c06cf60 msgr2=0x7f252c06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.714+0000 7f2532e8e640 1 -- 172.21.15.70:0/2025798312 shutdown_connections 2023-08-25T09:40:04.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.715+0000 7f2532e8e640 1 -- 172.21.15.70:0/2025798312 wait complete. 2023-08-25T09:40:04.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.717+0000 7f2532e8e640 1 Processor -- start 2023-08-25T09:40:04.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.717+0000 7f2532e8e640 1 -- start start 2023-08-25T09:40:04.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f2532e8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f252c06ad20 0x7f252c1f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f2532e8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f252c0caf30 0x7f252c1f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f2532e8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 0x7f252c0c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f2532e8e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0c5c70 con 0x7f252c15cd90 2023-08-25T09:40:04.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f2532e8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0c5f50 con 0x7f252c0caf30 2023-08-25T09:40:04.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f2532e8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c0c79b0 con 0x7f252c06ad20 2023-08-25T09:40:04.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f253142b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 0x7f252c0c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f253142b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 0x7f252c0c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47242/0 (socket says 172.21.15.70:47242) 2023-08-25T09:40:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.718+0000 7f253142b640 1 -- 172.21.15.70:0/1470728293 learned_addr learned my addr 172.21.15.70:0/1470728293 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f2523fff640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f252c0caf30 0x7f252c1f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f253142b640 1 -- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f252c06ad20 msgr2=0x7f252c1f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f253142b640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f252c06ad20 0x7f252c1f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f253142b640 1 -- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f252c0caf30 msgr2=0x7f252c1f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f253142b640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f252c0caf30 0x7f252c1f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f253142b640 1 -- 172.21.15.70:0/1470728293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c0094d0 con 0x7f252c15cd90 2023-08-25T09:40:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f253142b640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 0x7f252c0c4eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2524007e40 tx=0x7f252400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2524014620 con 0x7f252c15cd90 2023-08-25T09:40:04.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252400ee90 con 0x7f252c15cd90 2023-08-25T09:40:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25240149e0 con 0x7f252c15cd90 2023-08-25T09:40:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.719+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252c0c7c90 con 0x7f252c15cd90 2023-08-25T09:40:04.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.720+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252c0ca550 con 0x7f252c15cd90 2023-08-25T09:40:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.721+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252c0c81b0 con 0x7f252c15cd90 2023-08-25T09:40:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.722+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2524014b40 con 0x7f252c15cd90 2023-08-25T09:40:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.722+0000 7f25217fa640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f251806a050 0x7f251806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.723+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2524088f00 con 0x7f252c15cd90 2023-08-25T09:40:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.723+0000 7f2530c2a640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f251806a050 0x7f251806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.723+0000 7f2530c2a640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f251806a050 0x7f251806c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2528006fd0 tx=0x7f2528006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:04.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.727+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f252c0c81b0 con 0x7f252c15cd90 2023-08-25T09:40:04.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.926+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f252c0c81b0 con 0x7f252c15cd90 2023-08-25T09:40:04.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.927+0000 7f25217fa640 1 -- 172.21.15.70:0/1470728293 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f252c0c81b0 con 0x7f252c15cd90 2023-08-25T09:40:04.929 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:40:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f251806a050 msgr2=0x7f251806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f251806a050 0x7f251806c510 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2528006fd0 tx=0x7f2528006bc0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 msgr2=0x7f252c0c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 0x7f252c0c4eb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2524007e40 tx=0x7f252400e4b0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 shutdown_connections 2023-08-25T09:40:04.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f251806a050 0x7f251806c510 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f252c15cd90 0x7f252c0c4eb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f252c0caf30 0x7f252c1f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 --2- 172.21.15.70:0/1470728293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f252c06ad20 0x7f252c1f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.929+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 >> 172.21.15.70:0/1470728293 conn(0x7f252c06cf60 msgr2=0x7f252c160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.930+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 shutdown_connections 2023-08-25T09:40:04.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:04.930+0000 7f2532e8e640 1 -- 172.21.15.70:0/1470728293 wait complete. 2023-08-25T09:40:04.951 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:40:04.952 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:05.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:05.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.237+0000 7fb106a46640 1 Processor -- start 2023-08-25T09:40:05.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.237+0000 7fb106a46640 1 -- start start 2023-08-25T09:40:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb106a46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb1000b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb106a46640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1000b1b60 con 0x7fb10015ec10 2023-08-25T09:40:05.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb106a46640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10006b7c0 con 0x7fb10015e830 2023-08-25T09:40:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb106a46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10006bd80 con 0x7fb1000b1ff0 2023-08-25T09:40:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb0f7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb10015ec10 0x7fb10015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54176/0 (socket says 172.21.15.70:54176) 2023-08-25T09:40:05.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb0f7fff640 1 -- 172.21.15.70:0/2151825516 learned_addr learned my addr 172.21.15.70:0/2151825516 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:05.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245160343 0 0) 0x7fb1000b1b60 con 0x7fb10015ec10 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ec003520 con 0x7fb10015ec10 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.239+0000 7fb104fe3640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb1000b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1506869518 0 0) 0x7fb0ec003520 con 0x7fb10015ec10 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 msgr2=0x7fb1000b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb1000b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 >> v1:172.21.15.155:6789/0 conn(0x7fb10015e830 legacy=0x7fb10006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10006baa0 con 0x7fb10015ec10 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb0e8001670 con 0x7fb10015ec10 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb104fe3640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb1000b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 >> v1:172.21.15.70:6789/0 conn(0x7fb10015ec10 legacy=0x7fb10015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb0ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ec004d50 0x7fb0ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10006bd80 con 0x7fb0ec0039a0 2023-08-25T09:40:05.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec00bb60 con 0x7fb0ec004d50 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec00bd20 con 0x7fb1000b1ff0 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.240+0000 7fb0fffff640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0f7fff640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb0ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb104fe3640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ec004d50 0x7fb0ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0fffff640 1 -- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 msgr2=0x7fb0ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0fffff640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb0ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0fffff640 1 -- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ec004d50 msgr2=0x7fb0ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:05.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0fffff640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ec004d50 0x7fb0ec0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0fffff640 1 -- 172.21.15.70:0/2151825516 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec003520 con 0x7fb0ec0039a0 2023-08-25T09:40:05.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb0f7fff640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1000b1ff0 0x7fb0ec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:05.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.241+0000 7fb104fe3640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0ec004d50 0x7fb0ec0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.242+0000 7fb0fffff640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0001720 tx=0x7fb0ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dd9832a4f225d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.242+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00052d0 con 0x7fb0ec0039a0 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.242+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f002e050 con 0x7fb0ec0039a0 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.242+0000 7fb0ff7fe640 1 -- 172.21.15.70:0/2151825516 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00035f0 con 0x7fb0ec0039a0 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb106a46640 1 -- 172.21.15.70:0/2151825516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 msgr2=0x7fb0ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb106a46640 1 --2- 172.21.15.70:0/2151825516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb0ec003dc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0001720 tx=0x7fb0ec0057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb104fe3640 1 -- 172.21.15.70:0/2151825516 reap_dead start 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb106a46640 1 -- 172.21.15.70:0/2151825516 shutdown_connections 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb106a46640 1 -- 172.21.15.70:0/2151825516 >> 172.21.15.70:0/2151825516 conn(0x7fb100069850 msgr2=0x7fb100069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb106a46640 1 -- 172.21.15.70:0/2151825516 shutdown_connections 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.243+0000 7fb106a46640 1 -- 172.21.15.70:0/2151825516 wait complete. 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.245+0000 7fb106a46640 1 Processor -- start 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.245+0000 7fb106a46640 1 -- start start 2023-08-25T09:40:05.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb106a46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb100159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb106a46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb100155da0 0x7fb1001556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb106a46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000c8150 0x7fb1000ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb106a46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb100159d90 con 0x7fb0ec0039a0 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb106a46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb100155c10 con 0x7fb1000c8150 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb106a46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb100156620 con 0x7fb100155da0 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb100159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb100159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47256/0 (socket says 172.21.15.70:47256) 2023-08-25T09:40:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 -- 172.21.15.70:0/1602450406 learned_addr learned my addr 172.21.15.70:0/1602450406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0f7fff640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb100155da0 0x7fb1001556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 -- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb100155da0 msgr2=0x7fb1001556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb100155da0 0x7fb1001556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 -- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000c8150 msgr2=0x7fb1000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb104fe3640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000c8150 0x7fb1000ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000c8150 0x7fb1000ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.246+0000 7fb0fffff640 1 -- 172.21.15.70:0/1602450406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000c9b80 con 0x7fb0ec0039a0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.247+0000 7fb0fffff640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb100159690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0008fd0 tx=0x7fb0f0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.247+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00076c0 con 0x7fb0ec0039a0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.247+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f0007a30 con 0x7fb0ec0039a0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.247+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1000c9e20 con 0x7fb0ec0039a0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.247+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0006c00 con 0x7fb0ec0039a0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.247+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1000c7c30 con 0x7fb0ec0039a0 2023-08-25T09:40:05.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.248+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10004e420 con 0x7fb0ec0039a0 2023-08-25T09:40:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.249+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0f00035f0 con 0x7fb0ec0039a0 2023-08-25T09:40:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.251+0000 7fb0fd7fa640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0e006a080 0x7fb0e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.251+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0f00aa810 con 0x7fb0ec0039a0 2023-08-25T09:40:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.251+0000 7fb0f7fff640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0e006a080 0x7fb0e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.251+0000 7fb0f7fff640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0e006a080 0x7fb0e006c540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8000ff0 tx=0x7fb0e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:05.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.255+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0f0079860 con 0x7fb0ec0039a0 2023-08-25T09:40:05.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.450+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb10006c620 con 0x7fb0ec0039a0 2023-08-25T09:40:05.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.451+0000 7fb0fd7fa640 1 -- 172.21.15.70:0/1602450406 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fb0f00793f0 con 0x7fb0ec0039a0 2023-08-25T09:40:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:05.455 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:05.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0e006a080 msgr2=0x7fb0e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0e006a080 0x7fb0e006c540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8000ff0 tx=0x7fb0e8009040 comp rx=0 tx=0).stop 2023-08-25T09:40:05.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 msgr2=0x7fb100159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb100159690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0008fd0 tx=0x7fb0f0001120 comp rx=0 tx=0).stop 2023-08-25T09:40:05.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 shutdown_connections 2023-08-25T09:40:05.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0e006a080 0x7fb0e006c540 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1000c8150 0x7fb1000ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb100155da0 0x7fb1001556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 --2- 172.21.15.70:0/1602450406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0ec0039a0 0x7fb100159690 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.454+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 >> 172.21.15.70:0/1602450406 conn(0x7fb100069850 msgr2=0x7fb10015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:05.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.455+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 shutdown_connections 2023-08-25T09:40:05.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.455+0000 7fb106a46640 1 -- 172.21.15.70:0/1602450406 wait complete. 2023-08-25T09:40:05.477 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:05.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.698+0000 7f1ec06a6640 1 Processor -- start 2023-08-25T09:40:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.698+0000 7f1ec06a6640 1 -- start start 2023-08-25T09:40:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ec06a6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 0x7f1eb80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ec06a6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb806f8e0 con 0x7f1eb815e240 2023-08-25T09:40:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ec06a6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb806fbc0 con 0x7f1eb8159bf0 2023-08-25T09:40:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ec06a6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80ce030 con 0x7f1eb80cd660 2023-08-25T09:40:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ebdc41640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1eb815e240 0x7f1eb8070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54178/0 (socket says 172.21.15.70:54178) 2023-08-25T09:40:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ebdc41640 1 -- 172.21.15.70:0/219118828 learned_addr learned my addr 172.21.15.70:0/219118828 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:05.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845563476 0 0) 0x7f1eb806f8e0 con 0x7f1eb815e240 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ebec43640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 0x7f1eb80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.700+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea4003520 con 0x7f1eb815e240 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2080918172 0 0) 0x7f1ea4003520 con 0x7f1eb815e240 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 msgr2=0x7f1eb80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 0x7f1eb80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 >> v1:172.21.15.155:6789/0 conn(0x7f1eb8159bf0 legacy=0x7f1eb806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb806fea0 con 0x7f1eb815e240 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1ea8001670 con 0x7f1eb815e240 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 >> v1:172.21.15.70:6789/0 conn(0x7f1eb815e240 legacy=0x7f1eb8070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40039a0 0x7f1ea4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 0x7f1ea40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ea4004d00 0x7f1ea4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80ce030 con 0x7f1ea40039a0 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea400bb10 con 0x7f1ea4004d00 2023-08-25T09:40:05.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea400bcd0 con 0x7f1eb80cd660 2023-08-25T09:40:05.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.701+0000 7f1ebe442640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40039a0 0x7f1ea4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.702+0000 7f1ebe442640 1 -- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 msgr2=0x7f1ea40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:05.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.702+0000 7f1ebe442640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb80cd660 0x7f1ea40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.702+0000 7f1ebe442640 1 -- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ea4004d00 msgr2=0x7f1ea4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:05.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.702+0000 7f1ebec43640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ea4004d00 0x7f1ea4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.702+0000 7f1ebe442640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1ea4004d00 0x7f1ea4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.702+0000 7f1ebe442640 1 -- 172.21.15.70:0/219118828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4003520 con 0x7f1ea40039a0 2023-08-25T09:40:05.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.703+0000 7f1ebe442640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40039a0 0x7f1ea4003dc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4001330 tx=0x7f1ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe0f11faa6c8548 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:05.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.703+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40052d0 con 0x7f1ea40039a0 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.703+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4005430 con 0x7f1ea40039a0 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.703+0000 7f1ebd440640 1 -- 172.21.15.70:0/219118828 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4003830 con 0x7f1ea40039a0 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.703+0000 7f1ec06a6640 1 -- 172.21.15.70:0/219118828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40039a0 msgr2=0x7f1ea4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.703+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/219118828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea40039a0 0x7f1ea4003dc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4001330 tx=0x7f1ea40057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.704+0000 7f1ebec43640 1 -- 172.21.15.70:0/219118828 reap_dead start 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.704+0000 7f1ec06a6640 1 -- 172.21.15.70:0/219118828 shutdown_connections 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.704+0000 7f1ec06a6640 1 -- 172.21.15.70:0/219118828 >> 172.21.15.70:0/219118828 conn(0x7f1eb80cb0d0 msgr2=0x7f1eb80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:05.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.704+0000 7f1ec06a6640 1 -- 172.21.15.70:0/219118828 shutdown_connections 2023-08-25T09:40:05.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.704+0000 7f1ec06a6640 1 -- 172.21.15.70:0/219118828 wait complete. 2023-08-25T09:40:05.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.706+0000 7f1ec06a6640 1 Processor -- start 2023-08-25T09:40:05.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.706+0000 7f1ec06a6640 1 -- start start 2023-08-25T09:40:05.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ec06a6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea40039a0 0x7f1eb8148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ec06a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815bb90 0x7f1eb80ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ec06a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 0x7f1eb80acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ec06a6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8149270 con 0x7f1eb815c010 2023-08-25T09:40:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ec06a6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb815d1d0 con 0x7f1eb815bb90 2023-08-25T09:40:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ec06a6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb815c5c0 con 0x7f1ea40039a0 2023-08-25T09:40:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 0x7f1eb80acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 0x7f1eb80acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47280/0 (socket says 172.21.15.70:47280) 2023-08-25T09:40:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 -- 172.21.15.70:0/1829465938 learned_addr learned my addr 172.21.15.70:0/1829465938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebe442640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea40039a0 0x7f1eb8148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebdc41640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815bb90 0x7f1eb80ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 -- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea40039a0 msgr2=0x7f1eb8148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea40039a0 0x7f1eb8148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 -- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815bb90 msgr2=0x7f1eb80ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815bb90 0x7f1eb80ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebec43640 1 -- 172.21.15.70:0/1829465938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb815c8a0 con 0x7f1eb815c010 2023-08-25T09:40:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.707+0000 7f1ebdc41640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815bb90 0x7f1eb80ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.708+0000 7f1ebec43640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 0x7f1eb80acdc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1eb806c800 tx=0x7f1eb0009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.708+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0007db0 con 0x7f1eb815c010 2023-08-25T09:40:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.708+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0014400 con 0x7f1eb815c010 2023-08-25T09:40:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.708+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb815cb80 con 0x7f1eb815c010 2023-08-25T09:40:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.708+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00147b0 con 0x7f1eb815c010 2023-08-25T09:40:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.708+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb8156680 con 0x7f1eb815c010 2023-08-25T09:40:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.709+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb81568a0 con 0x7f1eb815c010 2023-08-25T09:40:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.710+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1eb001d3b0 con 0x7f1eb815c010 2023-08-25T09:40:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.711+0000 7f1eaeffd640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.711+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1eb00882b0 con 0x7f1eb815c010 2023-08-25T09:40:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.711+0000 7f1ebe442640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.712+0000 7f1ebe442640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005040 tx=0x7f1eb4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.716+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1eb81568a0 con 0x7f1eb815c010 2023-08-25T09:40:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.901+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1eb80c5690 con 0x7f1eb815c010 2023-08-25T09:40:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.902+0000 7f1eaeffd640 1 -- 172.21.15.70:0/1829465938 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1eb80c5690 con 0x7f1eb815c010 2023-08-25T09:40:05.906 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:05.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.904+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 msgr2=0x7f1e9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.904+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4005040 tx=0x7f1eb4009030 comp rx=0 tx=0).stop 2023-08-25T09:40:05.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.904+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 msgr2=0x7f1eb80acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:05.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.904+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 0x7f1eb80acdc0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1eb806c800 tx=0x7f1eb0009dc0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 shutdown_connections 2023-08-25T09:40:05.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c010 0x7f1eb80acdc0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815bb90 0x7f1eb80ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 --2- 172.21.15.70:0/1829465938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea40039a0 0x7f1eb8148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:05.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 >> 172.21.15.70:0/1829465938 conn(0x7f1eb80cb0d0 msgr2=0x7f1eb80c9100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:05.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 shutdown_connections 2023-08-25T09:40:05.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:05.905+0000 7f1ec06a6640 1 -- 172.21.15.70:0/1829465938 wait complete. 2023-08-25T09:40:05.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.138+0000 7f359cb96640 1 Processor -- start 2023-08-25T09:40:06.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.138+0000 7f359cb96640 1 -- start start 2023-08-25T09:40:06.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359cb96640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f35980b1ed0 0x7f35980cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359cb96640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3598072820 0x7f3598072c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359cb96640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359806d250 con 0x7f35980cf040 2023-08-25T09:40:06.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359cb96640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359806ddd0 con 0x7f3598072820 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359cb96640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3598073160 con 0x7f35980b1ed0 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f3596d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3598072820 0x7f3598072c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f35980cf040 0x7f359806bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54194/0 (socket says 172.21.15.70:54194) 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.139+0000 7f359659c640 1 -- 172.21.15.70:0/3332384887 learned_addr learned my addr 172.21.15.70:0/3332384887 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f359559a640 1 -- 172.21.15.70:0/3332384887 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002689858 0 0) 0x7f359806d250 con 0x7f35980cf040 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f3595d9b640 1 --2- 172.21.15.70:0/3332384887 >> v2:172.21.15.155:3300/0 conn(0x7f35980b1ed0 0x7f35980cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f359559a640 1 -- 172.21.15.70:0/3332384887 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3584003520 con 0x7f35980cf040 2023-08-25T09:40:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f3596d9d640 1 -- 172.21.15.70:0/3332384887 >> v2:172.21.15.155:3300/0 conn(0x7f35980b1ed0 msgr2=0x7f35980cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f3596d9d640 1 --2- 172.21.15.70:0/3332384887 >> v2:172.21.15.155:3300/0 conn(0x7f35980b1ed0 0x7f35980cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f3596d9d640 1 -- 172.21.15.70:0/3332384887 >> v1:172.21.15.70:6789/0 conn(0x7f35980cf040 legacy=0x7f359806bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.140+0000 7f3596d9d640 1 -- 172.21.15.70:0/3332384887 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35980cf420 con 0x7f3598072820 2023-08-25T09:40:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.141+0000 7f3596d9d640 1 --2- 172.21.15.70:0/3332384887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598072820 0x7f3598072c20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f359806f7f0 tx=0x7f359002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce30cb2a921b470 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:06.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.141+0000 7f359559a640 1 -- 172.21.15.70:0/3332384887 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590031830 con 0x7f3598072820 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.141+0000 7f359559a640 1 -- 172.21.15.70:0/3332384887 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3590031990 con 0x7f3598072820 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.141+0000 7f359559a640 1 -- 172.21.15.70:0/3332384887 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590031d00 con 0x7f3598072820 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 -- 172.21.15.70:0/3332384887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598072820 msgr2=0x7f3598072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 --2- 172.21.15.70:0/3332384887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598072820 0x7f3598072c20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f359806f7f0 tx=0x7f359002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 -- 172.21.15.70:0/3332384887 shutdown_connections 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 --2- 172.21.15.70:0/3332384887 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598072820 0x7f3598072c20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 --2- 172.21.15.70:0/3332384887 >> v2:172.21.15.155:3300/0 conn(0x7f35980b1ed0 0x7f35980cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 -- 172.21.15.70:0/3332384887 >> 172.21.15.70:0/3332384887 conn(0x7f35980b12d0 msgr2=0x7f35980b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:06.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 -- 172.21.15.70:0/3332384887 shutdown_connections 2023-08-25T09:40:06.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.142+0000 7f359cb96640 1 -- 172.21.15.70:0/3332384887 wait complete. 2023-08-25T09:40:06.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.144+0000 7f359cb96640 1 Processor -- start 2023-08-25T09:40:06.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.144+0000 7f359cb96640 1 -- start start 2023-08-25T09:40:06.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f359cb96640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598072820 0x7f3598154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f359cb96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 0x7f3598154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f359cb96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980cf040 0x7f35980ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f359cb96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359815f120 con 0x7f35980b1ed0 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f359cb96640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359815e5a0 con 0x7f3598072820 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f359cb96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359815e880 con 0x7f35980cf040 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f3595d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 0x7f3598154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f3595d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 0x7f3598154cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47298/0 (socket says 172.21.15.70:47298) 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.145+0000 7f3595d9b640 1 -- 172.21.15.70:0/3572212719 learned_addr learned my addr 172.21.15.70:0/3572212719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f359659c640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598072820 0x7f3598154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f3595d9b640 1 -- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980cf040 msgr2=0x7f35980ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f3595d9b640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980cf040 0x7f35980ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f3595d9b640 1 -- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598072820 msgr2=0x7f3598154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f3595d9b640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598072820 0x7f3598154530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f3595d9b640 1 -- 172.21.15.70:0/3572212719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3590009400 con 0x7f35980b1ed0 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f3595d9b640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 0x7f3598154cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3580007ef0 tx=0x7f358000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580014620 con 0x7f35980b1ed0 2023-08-25T09:40:06.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3580014780 con 0x7f35980b1ed0 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359815eb60 con 0x7f35980b1ed0 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.146+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580014b30 con 0x7f35980b1ed0 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.147+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3598154a70 con 0x7f35980b1ed0 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.148+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3598066fb0 con 0x7f35980b1ed0 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.149+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3580014c90 con 0x7f35980b1ed0 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.150+0000 7f358effd640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a050 0x7f357806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.150+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3580088ec0 con 0x7f35980b1ed0 2023-08-25T09:40:06.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.150+0000 7f359659c640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a050 0x7f357806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.151+0000 7f359659c640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a050 0x7f357806c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35880011e0 tx=0x7f3588009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:06.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.154+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3580054640 con 0x7f35980b1ed0 2023-08-25T09:40:06.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.342+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35980c7f10 con 0x7f35980b1ed0 2023-08-25T09:40:06.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.343+0000 7f358effd640 1 -- 172.21.15.70:0/3572212719 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3580057c60 con 0x7f35980b1ed0 2023-08-25T09:40:06.346 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:06.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a050 msgr2=0x7f357806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a050 0x7f357806c510 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f35880011e0 tx=0x7f3588009040 comp rx=0 tx=0).stop 2023-08-25T09:40:06.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 msgr2=0x7f3598154cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 0x7f3598154cf0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3580007ef0 tx=0x7f358000e4b0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 shutdown_connections 2023-08-25T09:40:06.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a050 0x7f357806c510 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980cf040 0x7f35980ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980b1ed0 0x7f3598154cf0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 --2- 172.21.15.70:0/3572212719 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3598072820 0x7f3598154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 >> 172.21.15.70:0/3572212719 conn(0x7f35980b12d0 msgr2=0x7f359806c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:06.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 shutdown_connections 2023-08-25T09:40:06.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.346+0000 7f359cb96640 1 -- 172.21.15.70:0/3572212719 wait complete. 2023-08-25T09:40:06.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:06.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.583+0000 7f7b9ccbf640 1 Processor -- start 2023-08-25T09:40:06.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.583+0000 7f7b9ccbf640 1 -- start start 2023-08-25T09:40:06.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.584+0000 7f7b9ccbf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7b98159bf0 0x7f7b9815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.584+0000 7f7b9ccbf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b980cd660 0x7f7b980cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.584+0000 7f7b9ccbf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9815e780 con 0x7f7b9806f0e0 2023-08-25T09:40:06.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.584+0000 7f7b9ccbf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980ce030 con 0x7f7b980cd660 2023-08-25T09:40:06.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b9ccbf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9806f8e0 con 0x7f7b98159bf0 2023-08-25T09:40:06.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b980cd660 0x7f7b980cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7b980cd660 0x7f7b980cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47312/0 (socket says 172.21.15.70:47312) 2023-08-25T09:40:06.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 -- 172.21.15.70:0/4028675199 learned_addr learned my addr 172.21.15.70:0/4028675199 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:06.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b9659c640 1 --2- 172.21.15.70:0/4028675199 >> v2:172.21.15.155:3300/0 conn(0x7f7b98159bf0 0x7f7b9815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 -- 172.21.15.70:0/4028675199 >> v2:172.21.15.155:3300/0 conn(0x7f7b98159bf0 msgr2=0x7f7b9815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 --2- 172.21.15.70:0/4028675199 >> v2:172.21.15.155:3300/0 conn(0x7f7b98159bf0 0x7f7b9815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 -- 172.21.15.70:0/4028675199 >> v1:172.21.15.155:6789/0 conn(0x7f7b9806f0e0 legacy=0x7f7b98070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.585+0000 7f7b96d9d640 1 -- 172.21.15.70:0/4028675199 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9806fbc0 con 0x7f7b980cd660 2023-08-25T09:40:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.586+0000 7f7b96d9d640 1 --2- 172.21.15.70:0/4028675199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b980cd660 0x7f7b980cdaf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7b9806d1f0 tx=0x7f7b9002e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d1b4fd063f9a85 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b95d9b640 1 -- 172.21.15.70:0/4028675199 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9002ea90 con 0x7f7b980cd660 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b95d9b640 1 -- 172.21.15.70:0/4028675199 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9002ebf0 con 0x7f7b980cd660 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b95d9b640 1 -- 172.21.15.70:0/4028675199 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b90009600 con 0x7f7b980cd660 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/4028675199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b980cd660 msgr2=0x7f7b980cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/4028675199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b980cd660 0x7f7b980cdaf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7b9806d1f0 tx=0x7f7b9002e0f0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/4028675199 shutdown_connections 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/4028675199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b980cd660 0x7f7b980cdaf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/4028675199 >> v2:172.21.15.155:3300/0 conn(0x7f7b98159bf0 0x7f7b9815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.587+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/4028675199 >> 172.21.15.70:0/4028675199 conn(0x7f7b980cb0d0 msgr2=0x7f7b980cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.588+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/4028675199 shutdown_connections 2023-08-25T09:40:06.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.588+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/4028675199 wait complete. 2023-08-25T09:40:06.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.589+0000 7f7b9ccbf640 1 Processor -- start 2023-08-25T09:40:06.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.589+0000 7f7b9ccbf640 1 -- start start 2023-08-25T09:40:06.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.590+0000 7f7b9ccbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 0x7f7b980afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.590+0000 7f7b9ccbf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b980cd660 0x7f7b98151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.590+0000 7f7b9ccbf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b98159bf0 0x7f7b98151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.590+0000 7f7b9ccbf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981520f0 con 0x7f7b9806f0e0 2023-08-25T09:40:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.590+0000 7f7b9ccbf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c4e00 con 0x7f7b98159bf0 2023-08-25T09:40:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.590+0000 7f7b9ccbf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c7780 con 0x7f7b980cd660 2023-08-25T09:40:06.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 0x7f7b980afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 0x7f7b980afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47320/0 (socket says 172.21.15.70:47320) 2023-08-25T09:40:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 -- 172.21.15.70:0/913657163 learned_addr learned my addr 172.21.15.70:0/913657163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 -- 172.21.15.70:0/913657163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b980cd660 msgr2=0x7f7b98151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b980cd660 0x7f7b98151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 -- 172.21.15.70:0/913657163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b98159bf0 msgr2=0x7f7b98151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b98159bf0 0x7f7b98151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 -- 172.21.15.70:0/913657163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9000bd40 con 0x7f7b9806f0e0 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.591+0000 7f7b9659c640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 0x7f7b980afab0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800d800 tx=0x7f7b8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.592+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800b6c0 con 0x7f7b9806f0e0 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.592+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8800b820 con 0x7f7b9806f0e0 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.592+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980c7a60 con 0x7f7b9806f0e0 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.592+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800bbb0 con 0x7f7b9806f0e0 2023-08-25T09:40:06.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.592+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b980c7430 con 0x7f7b9806f0e0 2023-08-25T09:40:06.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.593+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b980c75f0 con 0x7f7b9806f0e0 2023-08-25T09:40:06.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.594+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7b8800bd10 con 0x7f7b9806f0e0 2023-08-25T09:40:06.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.595+0000 7f7b8effd640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b7806a050 0x7f7b7806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:06.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.596+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b88088b10 con 0x7f7b9806f0e0 2023-08-25T09:40:06.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.596+0000 7f7b8ffff640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b7806a050 0x7f7b7806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:06.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.596+0000 7f7b8ffff640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b7806a050 0x7f7b7806c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:06.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.600+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7b88057c90 con 0x7f7b9806f0e0 2023-08-25T09:40:06.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.789+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7b98066fb0 con 0x7f7b9806f0e0 2023-08-25T09:40:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.790+0000 7f7b8effd640 1 -- 172.21.15.70:0/913657163 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b88057820 con 0x7f7b9806f0e0 2023-08-25T09:40:06.793 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:06.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b7806a050 msgr2=0x7f7b7806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b7806a050 0x7f7b7806c510 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b80006fd0 tx=0x7f7b80006bc0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 msgr2=0x7f7b980afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:06.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 0x7f7b980afab0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800d800 tx=0x7f7b8800dcc0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 shutdown_connections 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7b7806a050 0x7f7b7806c510 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7b98159bf0 0x7f7b98151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7b980cd660 0x7f7b98151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 --2- 172.21.15.70:0/913657163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7b9806f0e0 0x7f7b980afab0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 >> 172.21.15.70:0/913657163 conn(0x7f7b980cb0d0 msgr2=0x7f7b98069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:06.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 shutdown_connections 2023-08-25T09:40:06.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:06.793+0000 7f7b9ccbf640 1 -- 172.21.15.70:0/913657163 wait complete. 2023-08-25T09:40:06.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.012+0000 7f22918ec640 1 Processor -- start 2023-08-25T09:40:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.013+0000 7f22918ec640 1 -- start start 2023-08-25T09:40:07.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.014+0000 7f22918ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f228c0cf3d0 0x7f228c0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.014+0000 7f22918ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c0cba70 con 0x7f228c06e090 2023-08-25T09:40:07.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.014+0000 7f22918ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c0cbd50 con 0x7f228c0ce8e0 2023-08-25T09:40:07.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.014+0000 7f22918ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c0cc030 con 0x7f228c0cf3d0 2023-08-25T09:40:07.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.014+0000 7f2283fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f228c06e090 0x7f228c0cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54204/0 (socket says 172.21.15.70:54204) 2023-08-25T09:40:07.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.014+0000 7f2283fff640 1 -- 172.21.15.70:0/3729512182 learned_addr learned my addr 172.21.15.70:0/3729512182 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:07.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228b7fe640 1 --2- 172.21.15.70:0/3729512182 >> v2:172.21.15.155:3300/0 conn(0x7f228c0cf3d0 0x7f228c0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254346571 0 0) 0x7f228c0cba70 con 0x7f228c06e090 2023-08-25T09:40:07.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f226c003520 con 0x7f228c06e090 2023-08-25T09:40:07.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2604209612 0 0) 0x7f226c003520 con 0x7f228c06e090 2023-08-25T09:40:07.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 >> v2:172.21.15.155:3300/0 conn(0x7f228c0cf3d0 msgr2=0x7f228c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 --2- 172.21.15.70:0/3729512182 >> v2:172.21.15.155:3300/0 conn(0x7f228c0cf3d0 0x7f228c0cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 >> v1:172.21.15.155:6789/0 conn(0x7f228c0ce8e0 legacy=0x7f228c0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c0cc310 con 0x7f228c06e090 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f2274001670 con 0x7f228c06e090 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 >> v1:172.21.15.70:6789/0 conn(0x7f228c06e090 legacy=0x7f228c0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c0039a0 0x7f226c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226c0047c0 0x7f226c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c005180 0x7f226c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0cc030 con 0x7f226c0047c0 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c00c050 con 0x7f226c005180 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.015+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c00c210 con 0x7f226c0039a0 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f2283fff640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226c0047c0 0x7f226c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f2283fff640 1 -- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c0039a0 msgr2=0x7f226c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f2283fff640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f226c0039a0 0x7f226c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f2283fff640 1 -- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c005180 msgr2=0x7f226c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:07.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f228b7fe640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c005180 0x7f226c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f2283fff640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c005180 0x7f226c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f2283fff640 1 -- 172.21.15.70:0/3729512182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c003520 con 0x7f226c0047c0 2023-08-25T09:40:07.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f228b7fe640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c005180 0x7f226c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:07.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.016+0000 7f228b7fe640 1 -- 172.21.15.70:0/3729512182 reap_dead start 2023-08-25T09:40:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.017+0000 7f2283fff640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226c0047c0 0x7f226c004c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f227400c210 tx=0x7f226c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2982284b9e1e9d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:07.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.017+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274005ad0 con 0x7f226c0047c0 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.017+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274005c30 con 0x7f226c0047c0 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.017+0000 7f228a7fc640 1 -- 172.21.15.70:0/3729512182 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227402ca50 con 0x7f226c0047c0 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 -- 172.21.15.70:0/3729512182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226c0047c0 msgr2=0x7f226c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226c0047c0 0x7f226c004c40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f227400c210 tx=0x7f226c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 -- 172.21.15.70:0/3729512182 shutdown_connections 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 --2- 172.21.15.70:0/3729512182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226c0047c0 0x7f226c004c40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 -- 172.21.15.70:0/3729512182 >> 172.21.15.70:0/3729512182 conn(0x7f228c0d0860 msgr2=0x7f228c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:07.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 -- 172.21.15.70:0/3729512182 shutdown_connections 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.018+0000 7f22918ec640 1 -- 172.21.15.70:0/3729512182 wait complete. 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.020+0000 7f22918ec640 1 Processor -- start 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.020+0000 7f22918ec640 1 -- start start 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f22918ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0047c0 0x7f228c1d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f22918ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f228c06b380 0x7f228c1d6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f22918ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 0x7f228c1e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f22918ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1d7370 con 0x7f228c1d8f50 2023-08-25T09:40:07.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f22918ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1d9750 con 0x7f226c0047c0 2023-08-25T09:40:07.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f22918ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c1d9a30 con 0x7f228c06b380 2023-08-25T09:40:07.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f228b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 0x7f228c1e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f228b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 0x7f228c1e32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47330/0 (socket says 172.21.15.70:47330) 2023-08-25T09:40:07.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.021+0000 7f228b7fe640 1 -- 172.21.15.70:0/857277107 learned_addr learned my addr 172.21.15.70:0/857277107 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:07.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f228b7fe640 1 -- 172.21.15.70:0/857277107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f228c06b380 msgr2=0x7f228c1d6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f228b7fe640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f228c06b380 0x7f228c1d6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f228b7fe640 1 -- 172.21.15.70:0/857277107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0047c0 msgr2=0x7f228c1d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f228b7fe640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0047c0 0x7f228c1d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f228b7fe640 1 -- 172.21.15.70:0/857277107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c1d9d10 con 0x7f228c1d8f50 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f228b7fe640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 0x7f228c1e32d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f228c06f670 tx=0x7f228400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284016420 con 0x7f228c1d8f50 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2284016580 con 0x7f228c1d8f50 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840168f0 con 0x7f228c1d8f50 2023-08-25T09:40:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c1d9ff0 con 0x7f228c1d8f50 2023-08-25T09:40:07.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.022+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c1d6da0 con 0x7f228c1d8f50 2023-08-25T09:40:07.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.024+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c1d74e0 con 0x7f228c1d8f50 2023-08-25T09:40:07.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.025+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2284016a50 con 0x7f228c1d8f50 2023-08-25T09:40:07.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.026+0000 7f2288ff9640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f227006a050 0x7f227006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.027+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22840894c0 con 0x7f228c1d8f50 2023-08-25T09:40:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.027+0000 7f228affd640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f227006a050 0x7f227006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.027+0000 7f228affd640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f227006a050 0x7f227006c510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f227c007fd0 tx=0x7f227c007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:07.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.031+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f228c1d74e0 con 0x7f228c1d8f50 2023-08-25T09:40:07.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.220+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f228c1d74b0 con 0x7f228c1d8f50 2023-08-25T09:40:07.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.221+0000 7f2288ff9640 1 -- 172.21.15.70:0/857277107 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2284058630 con 0x7f228c1d8f50 2023-08-25T09:40:07.224 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:07.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.224+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f227006a050 msgr2=0x7f227006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.224+0000 7f22918ec640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f227006a050 0x7f227006c510 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f227c007fd0 tx=0x7f227c007a00 comp rx=0 tx=0).stop 2023-08-25T09:40:07.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.224+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 msgr2=0x7f228c1e32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.224+0000 7f22918ec640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 0x7f228c1e32d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f228c06f670 tx=0x7f228400f520 comp rx=0 tx=0).stop 2023-08-25T09:40:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 shutdown_connections 2023-08-25T09:40:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f227006a050 0x7f227006c510 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f228c1d8f50 0x7f228c1e32d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f228c06b380 0x7f228c1d6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 --2- 172.21.15.70:0/857277107 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f226c0047c0 0x7f228c1d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 >> 172.21.15.70:0/857277107 conn(0x7f228c0d0860 msgr2=0x7f228c1dbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 shutdown_connections 2023-08-25T09:40:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.225+0000 7f22918ec640 1 -- 172.21.15.70:0/857277107 wait complete. 2023-08-25T09:40:07.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:40:07.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.456+0000 7f0623fff640 1 Processor -- start 2023-08-25T09:40:07.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.457+0000 7f0623fff640 1 -- start start 2023-08-25T09:40:07.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.458+0000 7f0623fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f0624070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.458+0000 7f0623fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06240cf210 0x7f06240cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.458+0000 7f0623fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06240b81d0 con 0x7f062415cf00 2023-08-25T09:40:07.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.458+0000 7f0623fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06241604f0 con 0x7f06240cf210 2023-08-25T09:40:07.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.458+0000 7f0623fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062415b250 con 0x7f06240b7df0 2023-08-25T09:40:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.459+0000 7f06237fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06240cf210 0x7f06240cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.459+0000 7f0622ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f0624070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.459+0000 7f06237fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06240cf210 0x7f06240cf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:51888/0 (socket says 172.21.15.70:51888) 2023-08-25T09:40:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.459+0000 7f06237fe640 1 -- 172.21.15.70:0/3729089129 learned_addr learned my addr 172.21.15.70:0/3729089129 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.460+0000 7f06237fe640 1 -- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 msgr2=0x7f0624070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.460+0000 7f06237fe640 1 --2- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f0624070880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.460+0000 7f06237fe640 1 -- 172.21.15.70:0/3729089129 >> v1:172.21.15.155:6789/0 conn(0x7f062415cf00 legacy=0x7f062415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.460+0000 7f06237fe640 1 -- 172.21.15.70:0/3729089129 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06240cfba0 con 0x7f06240cf210 2023-08-25T09:40:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.461+0000 7f06237fe640 1 --2- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240cf660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f061c009e50 tx=0x7f061c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc84eed124b1acd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0621ffb640 1 -- 172.21.15.70:0/3729089129 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c02cb00 con 0x7f06240cf210 2023-08-25T09:40:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0621ffb640 1 -- 172.21.15.70:0/3729089129 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c02cc60 con 0x7f06240cf210 2023-08-25T09:40:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0621ffb640 1 -- 172.21.15.70:0/3729089129 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0343f0 con 0x7f06240cf210 2023-08-25T09:40:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 -- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 msgr2=0x7f06240cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 --2- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240cf660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f061c009e50 tx=0x7f061c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:40:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 -- 172.21.15.70:0/3729089129 shutdown_connections 2023-08-25T09:40:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 --2- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240cf660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 --2- 172.21.15.70:0/3729089129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f0624070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 -- 172.21.15.70:0/3729089129 >> 172.21.15.70:0/3729089129 conn(0x7f06240b43a0 msgr2=0x7f06240b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 -- 172.21.15.70:0/3729089129 shutdown_connections 2023-08-25T09:40:07.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.462+0000 7f0623fff640 1 -- 172.21.15.70:0/3729089129 wait complete. 2023-08-25T09:40:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.464+0000 7f0623fff640 1 Processor -- start 2023-08-25T09:40:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.464+0000 7f0623fff640 1 -- start start 2023-08-25T09:40:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f0623fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f06240cc420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f0623fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f0623fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 0x7f062414e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f0623fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06240cb9c0 con 0x7f062415cf00 2023-08-25T09:40:07.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f0623fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062414f190 con 0x7f06240cf210 2023-08-25T09:40:07.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f0623fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06240cb020 con 0x7f06240b7df0 2023-08-25T09:40:07.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f06237fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 0x7f062414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f06237fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 0x7f062414e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47346/0 (socket says 172.21.15.70:47346) 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.465+0000 7f06237fe640 1 -- 172.21.15.70:0/2572144463 learned_addr learned my addr 172.21.15.70:0/2572144463 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06237fe640 1 -- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 msgr2=0x7f06240cc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f0622ffd640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f06240cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06227fc640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06237fe640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f06240cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06237fe640 1 -- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 msgr2=0x7f06240d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06237fe640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06237fe640 1 -- 172.21.15.70:0/2572144463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c009b30 con 0x7f062415cf00 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06227fc640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f06237fe640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 0x7f062414e610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f061c009e50 tx=0x7f061c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.466+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c009510 con 0x7f062415cf00 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.467+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061c03e450 con 0x7f062415cf00 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.467+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c03e800 con 0x7f062415cf00 2023-08-25T09:40:07.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.467+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06240cb300 con 0x7f062415cf00 2023-08-25T09:40:07.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.467+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062414eb50 con 0x7f062415cf00 2023-08-25T09:40:07.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.468+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0624066fb0 con 0x7f062415cf00 2023-08-25T09:40:07.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.469+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f061c03e960 con 0x7f062415cf00 2023-08-25T09:40:07.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.470+0000 7f061bfff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f060406a050 0x7f060406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:07.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.470+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f061c0ab240 con 0x7f062415cf00 2023-08-25T09:40:07.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.470+0000 7f0622ffd640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f060406a050 0x7f060406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:07.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.471+0000 7f0622ffd640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f060406a050 0x7f060406c510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f06240d24c0 tx=0x7f0614008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:07.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.475+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f061c0769c0 con 0x7f062415cf00 2023-08-25T09:40:07.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.673+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f062404dcb0 con 0x7f062415cf00 2023-08-25T09:40:07.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.674+0000 7f061bfff640 1 -- 172.21.15.70:0/2572144463 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f061c079fe0 con 0x7f062415cf00 2023-08-25T09:40:07.676 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:40:07.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f060406a050 msgr2=0x7f060406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f060406a050 0x7f060406c510 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f06240d24c0 tx=0x7f0614008040 comp rx=0 tx=0).stop 2023-08-25T09:40:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 msgr2=0x7f062414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 0x7f062414e610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f061c009e50 tx=0x7f061c004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 shutdown_connections 2023-08-25T09:40:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f060406a050 0x7f060406c510 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f062415cf00 0x7f062414e610 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06240cf210 0x7f06240d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 --2- 172.21.15.70:0/2572144463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06240b7df0 0x7f06240cc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.676+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 >> 172.21.15.70:0/2572144463 conn(0x7f06240b43a0 msgr2=0x7f062415bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.677+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 shutdown_connections 2023-08-25T09:40:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:07.677+0000 7f0623fff640 1 -- 172.21.15.70:0/2572144463 wait complete. 2023-08-25T09:40:07.696 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:40:07.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:07.698 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-08-25T09:40:07.796 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:40:07.797 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:40:07.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:07.798 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:40:07.844 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:40:07.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:08.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.058+0000 7fc8fc7e3640 1 Processor -- start 2023-08-25T09:40:08.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.058+0000 7fc8fc7e3640 1 -- start start 2023-08-25T09:40:08.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.060+0000 7fc8fc7e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f40cea30 0x7fc8f40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.060+0000 7fc8fc7e3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc8f40ac870 0x7fc8f40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.060+0000 7fc8fc7e3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f406f050 con 0x7fc8f415cb40 2023-08-25T09:40:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.060+0000 7fc8fc7e3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f406f330 con 0x7fc8f40ac870 2023-08-25T09:40:08.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.060+0000 7fc8fc7e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f406f610 con 0x7fc8f40cea30 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.061+0000 7fc8fad80640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc8f40ac870 0x7fc8f40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.061+0000 7fc8fad80640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc8f40ac870 0x7fc8f40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:51896/0 (socket says 172.21.15.70:51896) 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.061+0000 7fc8fad80640 1 -- 172.21.15.70:0/3203941186 learned_addr learned my addr 172.21.15.70:0/3203941186 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.061+0000 7fc8f9d7e640 1 --2- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f40cea30 0x7fc8f40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.062+0000 7fc8fad80640 1 -- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f40cea30 msgr2=0x7fc8f40b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.062+0000 7fc8fad80640 1 --2- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f40cea30 0x7fc8f40b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.062+0000 7fc8fad80640 1 -- 172.21.15.70:0/3203941186 >> v1:172.21.15.155:6789/0 conn(0x7fc8f415cb40 legacy=0x7fc8f40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.062+0000 7fc8fad80640 1 -- 172.21.15.70:0/3203941186 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f406f8f0 con 0x7fc8f40ac870 2023-08-25T09:40:08.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.063+0000 7fc8fad80640 1 --2- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009e50 tx=0x7fc8ec02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bb868cae0ee89c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.063+0000 7fc8f957d640 1 -- 172.21.15.70:0/3203941186 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec02cb00 con 0x7fc8f40ac870 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.063+0000 7fc8f957d640 1 -- 172.21.15.70:0/3203941186 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec02cc60 con 0x7fc8f40ac870 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.063+0000 7fc8f957d640 1 -- 172.21.15.70:0/3203941186 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0343f0 con 0x7fc8f40ac870 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40ac870 msgr2=0x7fc8f40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec009e50 tx=0x7fc8ec02bf60 comp rx=0 tx=0).stop 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3203941186 shutdown_connections 2023-08-25T09:40:08.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3203941186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f40cea30 0x7fc8f40b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3203941186 >> 172.21.15.70:0/3203941186 conn(0x7fc8f4069850 msgr2=0x7fc8f4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3203941186 shutdown_connections 2023-08-25T09:40:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.064+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3203941186 wait complete. 2023-08-25T09:40:08.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.066+0000 7fc8fc7e3640 1 Processor -- start 2023-08-25T09:40:08.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.067+0000 7fc8fc7e3640 1 -- start start 2023-08-25T09:40:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fc7e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fc7e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40cea30 0x7fc8f414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fc7e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f415cb40 0x7fc8f41f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fc7e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f413dbe0 con 0x7fc8f40ac870 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fc7e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f414ed20 con 0x7fc8f40cea30 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fc7e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f413de30 con 0x7fc8f415cb40 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fa57f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fa57f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47356/0 (socket says 172.21.15.70:47356) 2023-08-25T09:40:08.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.068+0000 7fc8fa57f640 1 -- 172.21.15.70:0/3929483485 learned_addr learned my addr 172.21.15.70:0/3929483485 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:08.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fa57f640 1 -- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f415cb40 msgr2=0x7fc8f41f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fa57f640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f415cb40 0x7fc8f41f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fa57f640 1 -- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40cea30 msgr2=0x7fc8f414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8f9d7e640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40cea30 0x7fc8f414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fa57f640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40cea30 0x7fc8f414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fa57f640 1 -- 172.21.15.70:0/3929483485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec009b30 con 0x7fc8f40ac870 2023-08-25T09:40:08.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8f9d7e640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40cea30 0x7fc8f414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fa57f640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c450 tx=0x7fc8f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f00052d0 con 0x7fc8f40ac870 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f0005430 con 0x7fc8f40ac870 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f413e110 con 0x7fc8f40ac870 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0012870 con 0x7fc8f40ac870 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.069+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f40b2a60 con 0x7fc8f40ac870 2023-08-25T09:40:08.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.070+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f413e630 con 0x7fc8f40ac870 2023-08-25T09:40:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.071+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc8f00129d0 con 0x7fc8f40ac870 2023-08-25T09:40:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.073+0000 7fc8eaffd640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8d806a050 0x7fc8d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.073+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8f0088170 con 0x7fc8f40ac870 2023-08-25T09:40:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.073+0000 7fc8f9d7e640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8d806a050 0x7fc8d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.073+0000 7fc8f9d7e640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8d806a050 0x7fc8d806c510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4070840 tx=0x7fc8e4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.077+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc8f413e630 con 0x7fc8f40ac870 2023-08-25T09:40:08.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.285+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8f40b2c20 con 0x7fc8f40ac870 2023-08-25T09:40:08.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.287+0000 7fc8eaffd640 1 -- 172.21.15.70:0/3929483485 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7fc8f00537c0 con 0x7fc8f40ac870 2023-08-25T09:40:08.290 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:08.290 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:40:08.290 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:40:08.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8d806a050 msgr2=0x7fc8d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8d806a050 0x7fc8d806c510 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4070840 tx=0x7fc8e4008040 comp rx=0 tx=0).stop 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 msgr2=0x7fc8f40b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2520 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000c450 tx=0x7fc8f000c910 comp rx=0 tx=0).stop 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 shutdown_connections 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8d806a050 0x7fc8d806c510 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8f415cb40 0x7fc8f41f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8f40cea30 0x7fc8f414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 --2- 172.21.15.70:0/3929483485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8f40ac870 0x7fc8f40b2520 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 >> 172.21.15.70:0/3929483485 conn(0x7fc8f4069850 msgr2=0x7fc8f40acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 shutdown_connections 2023-08-25T09:40:08.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.290+0000 7fc8fc7e3640 1 -- 172.21.15.70:0/3929483485 wait complete. 2023-08-25T09:40:08.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:08.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.521+0000 7fcfb20c5640 1 Processor -- start 2023-08-25T09:40:08.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.521+0000 7fcfb20c5640 1 -- start start 2023-08-25T09:40:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfb20c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcfac0b1ff0 0x7fcfac0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfb20c5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0b1b60 con 0x7fcfac15e830 2023-08-25T09:40:08.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfb20c5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac06b7c0 con 0x7fcfac15ec10 2023-08-25T09:40:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfb20c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac06bd80 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfab7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcfac15e830 0x7fcfac06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54206/0 (socket says 172.21.15.70:54206) 2023-08-25T09:40:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfab7fe640 1 -- 172.21.15.70:0/3738888036 learned_addr learned my addr 172.21.15.70:0/3738888036 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfabfff640 1 --2- 172.21.15.70:0/3738888036 >> v2:172.21.15.70:3300/0 conn(0x7fcfac0b1ff0 0x7fcfac0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfaaffd640 1 -- 172.21.15.70:0/3738888036 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955809625 0 0) 0x7fcfac0b1b60 con 0x7fcfac15e830 2023-08-25T09:40:08.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfaaffd640 1 -- 172.21.15.70:0/3738888036 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf8c003520 con 0x7fcfac15e830 2023-08-25T09:40:08.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfabfff640 1 -- 172.21.15.70:0/3738888036 >> v1:172.21.15.155:6789/0 conn(0x7fcfac15ec10 legacy=0x7fcfac15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:08.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfabfff640 1 -- 172.21.15.70:0/3738888036 >> v1:172.21.15.70:6789/0 conn(0x7fcfac15e830 legacy=0x7fcfac06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.523+0000 7fcfabfff640 1 -- 172.21.15.70:0/3738888036 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac06baa0 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfabfff640 1 --2- 172.21.15.70:0/3738888036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac0b2440 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009d90 tx=0x7fcfa402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ed2d8042da48d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfaaffd640 1 -- 172.21.15.70:0/3738888036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa402cc60 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfaaffd640 1 -- 172.21.15.70:0/3738888036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa402cdc0 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfaaffd640 1 -- 172.21.15.70:0/3738888036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4034640 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfb20c5640 1 -- 172.21.15.70:0/3738888036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 msgr2=0x7fcfac0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/3738888036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac0b2440 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009d90 tx=0x7fcfa402beb0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfb20c5640 1 -- 172.21.15.70:0/3738888036 shutdown_connections 2023-08-25T09:40:08.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/3738888036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac0b2440 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.525+0000 7fcfb20c5640 1 -- 172.21.15.70:0/3738888036 >> 172.21.15.70:0/3738888036 conn(0x7fcfac069850 msgr2=0x7fcfac069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:08.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.526+0000 7fcfb20c5640 1 -- 172.21.15.70:0/3738888036 shutdown_connections 2023-08-25T09:40:08.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.526+0000 7fcfb20c5640 1 -- 172.21.15.70:0/3738888036 wait complete. 2023-08-25T09:40:08.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.527+0000 7fcfb20c5640 1 Processor -- start 2023-08-25T09:40:08.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.527+0000 7fcfb20c5640 1 -- start start 2023-08-25T09:40:08.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfb20c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac15cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfb20c5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfac15e830 0x7fcfac15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfb20c5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfac15ec10 0x7fcfac1422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfb20c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac150360 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfb20c5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1510a0 con 0x7fcfac15ec10 2023-08-25T09:40:08.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfb20c5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0a8990 con 0x7fcfac15e830 2023-08-25T09:40:08.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.528+0000 7fcfab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac15cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac15cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47370/0 (socket says 172.21.15.70:47370) 2023-08-25T09:40:08.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 -- 172.21.15.70:0/1054121875 learned_addr learned my addr 172.21.15.70:0/1054121875 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:08.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 -- 172.21.15.70:0/1054121875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfac15e830 msgr2=0x7fcfac15d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfac15e830 0x7fcfac15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 -- 172.21.15.70:0/1054121875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfac15ec10 msgr2=0x7fcfac1422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfac15ec10 0x7fcfac1422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 -- 172.21.15.70:0/1054121875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4009a70 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfab7fe640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac15cbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c500 tx=0x7fcf9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c005ce0 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c005e40 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c007330 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.529+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac13fff0 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.530+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac1405b0 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.531+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac15c880 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.532+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf9c007550 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.532+0000 7fcfa8ff9640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf9006a050 0x7fcf9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.533+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf9c087cf0 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.533+0000 7fcfa3fff640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf9006a050 0x7fcf9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.533+0000 7fcfa3fff640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf9006a050 0x7fcf9006c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006fd0 tx=0x7fcf94006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.537+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcfac15c880 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.722+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfac15c880 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.724+0000 7fcfa8ff9640 1 -- 172.21.15.70:0/1054121875 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fcfac15c880 con 0x7fcfac0b1ff0 2023-08-25T09:40:08.727 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:08.727 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:08.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf9006a050 msgr2=0x7fcf9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf9006a050 0x7fcf9006c510 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf94006fd0 tx=0x7fcf94006bc0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 msgr2=0x7fcfac15cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac15cbb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c500 tx=0x7fcf9c00c9c0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 shutdown_connections 2023-08-25T09:40:08.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf9006a050 0x7fcf9006c510 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcfac15ec10 0x7fcfac1422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcfac15e830 0x7fcfac15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 --2- 172.21.15.70:0/1054121875 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcfac0b1ff0 0x7fcfac15cbb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.726+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 >> 172.21.15.70:0/1054121875 conn(0x7fcfac069850 msgr2=0x7fcfac15a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:08.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.727+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 shutdown_connections 2023-08-25T09:40:08.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.727+0000 7fcfb20c5640 1 -- 172.21.15.70:0/1054121875 wait complete. 2023-08-25T09:40:08.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:38:51.059079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:38:51.624756+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:08.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:08.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.962+0000 7fe13a309640 1 Processor -- start 2023-08-25T09:40:08.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.962+0000 7fe13a309640 1 -- start start 2023-08-25T09:40:08.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.963+0000 7fe13a309640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe1340b2850 0x7fe1340cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.963+0000 7fe13a309640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1340c8d20 0x7fe1340c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.963+0000 7fe13a309640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1340cd800 con 0x7fe1340b2470 2023-08-25T09:40:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe13a309640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13415ad20 con 0x7fe1340c8d20 2023-08-25T09:40:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe13a309640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13415a110 con 0x7fe1340b2850 2023-08-25T09:40:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1340c8d20 0x7fe1340c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1340c8d20 0x7fe1340c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47386/0 (socket says 172.21.15.70:47386) 2023-08-25T09:40:08.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 -- 172.21.15.70:0/520825962 learned_addr learned my addr 172.21.15.70:0/520825962 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:08.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe138b06640 1 --2- 172.21.15.70:0/520825962 >> v2:172.21.15.155:3300/0 conn(0x7fe1340b2850 0x7fe1340cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 -- 172.21.15.70:0/520825962 >> v2:172.21.15.155:3300/0 conn(0x7fe1340b2850 msgr2=0x7fe1340cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:08.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 --2- 172.21.15.70:0/520825962 >> v2:172.21.15.155:3300/0 conn(0x7fe1340b2850 0x7fe1340cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 -- 172.21.15.70:0/520825962 >> v1:172.21.15.155:6789/0 conn(0x7fe1340b2470 legacy=0x7fe134159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:08.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.964+0000 7fe139b08640 1 -- 172.21.15.70:0/520825962 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13415a3f0 con 0x7fe1340c8d20 2023-08-25T09:40:08.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.965+0000 7fe139b08640 1 --2- 172.21.15.70:0/520825962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340c8d20 0x7fe1340c91b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe1340ade40 tx=0x7fe12c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bff61fe6714333 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe12bfff640 1 -- 172.21.15.70:0/520825962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c02ca80 con 0x7fe1340c8d20 2023-08-25T09:40:08.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe12bfff640 1 -- 172.21.15.70:0/520825962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c02cbe0 con 0x7fe1340c8d20 2023-08-25T09:40:08.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe12bfff640 1 -- 172.21.15.70:0/520825962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0343f0 con 0x7fe1340c8d20 2023-08-25T09:40:08.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe13a309640 1 -- 172.21.15.70:0/520825962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340c8d20 msgr2=0x7fe1340c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe13a309640 1 --2- 172.21.15.70:0/520825962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340c8d20 0x7fe1340c91b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe1340ade40 tx=0x7fe12c02bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe13a309640 1 -- 172.21.15.70:0/520825962 shutdown_connections 2023-08-25T09:40:08.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe13a309640 1 --2- 172.21.15.70:0/520825962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340c8d20 0x7fe1340c91b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe13a309640 1 --2- 172.21.15.70:0/520825962 >> v2:172.21.15.155:3300/0 conn(0x7fe1340b2850 0x7fe1340cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.966+0000 7fe13a309640 1 -- 172.21.15.70:0/520825962 >> 172.21.15.70:0/520825962 conn(0x7fe1340ce830 msgr2=0x7fe1340cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:08.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.967+0000 7fe13a309640 1 -- 172.21.15.70:0/520825962 shutdown_connections 2023-08-25T09:40:08.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.967+0000 7fe13a309640 1 -- 172.21.15.70:0/520825962 wait complete. 2023-08-25T09:40:08.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.968+0000 7fe13a309640 1 Processor -- start 2023-08-25T09:40:08.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.968+0000 7fe13a309640 1 -- start start 2023-08-25T09:40:08.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.969+0000 7fe13a309640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1340b2470 0x7fe1341eda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.969+0000 7fe13a309640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 0x7fe1340c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.969+0000 7fe13a309640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1340c8d20 0x7fe1340b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.969+0000 7fe13a309640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340c49b0 con 0x7fe1340b2850 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.969+0000 7fe13a309640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13406f9e0 con 0x7fe1340c8d20 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.969+0000 7fe13a309640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13406edd0 con 0x7fe1340b2470 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 0x7fe1340c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 0x7fe1340c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47396/0 (socket says 172.21.15.70:47396) 2023-08-25T09:40:08.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 -- 172.21.15.70:0/1053272658 learned_addr learned my addr 172.21.15.70:0/1053272658 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 -- 172.21.15.70:0/1053272658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1340b2470 msgr2=0x7fe1341eda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1340b2470 0x7fe1341eda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 -- 172.21.15.70:0/1053272658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1340c8d20 msgr2=0x7fe1340b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1340c8d20 0x7fe1340b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 -- 172.21.15.70:0/1053272658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12c009b30 con 0x7fe1340b2850 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe138b06640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 0x7fe1340c3a00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe12400d800 tx=0x7fe12400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.970+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400b6c0 con 0x7fe1340b2850 2023-08-25T09:40:08.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.971+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12400b820 con 0x7fe1340b2850 2023-08-25T09:40:08.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.971+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12400bbb0 con 0x7fe1340b2850 2023-08-25T09:40:08.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.971+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13406f0b0 con 0x7fe1340b2850 2023-08-25T09:40:08.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.971+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13406f670 con 0x7fe1340b2850 2023-08-25T09:40:08.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.972+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe134061460 con 0x7fe1340b2850 2023-08-25T09:40:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.973+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe12400bd10 con 0x7fe1340b2850 2023-08-25T09:40:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.973+0000 7fe129ffb640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe11806a080 0x7fe11806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.974+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1240889e0 con 0x7fe1340b2850 2023-08-25T09:40:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.974+0000 7fe139307640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe11806a080 0x7fe11806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.974+0000 7fe139307640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe11806a080 0x7fe11806c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe130003d10 tx=0x7fe130007a00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:08.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:08.978+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe124054160 con 0x7fe1340b2850 2023-08-25T09:40:09.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.171+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe13406f800 con 0x7fe1340b2850 2023-08-25T09:40:09.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.171+0000 7fe129ffb640 1 -- 172.21.15.70:0/1053272658 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe124057780 con 0x7fe1340b2850 2023-08-25T09:40:09.174 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe11806a080 msgr2=0x7fe11806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe11806a080 0x7fe11806c540 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe130003d10 tx=0x7fe130007a00 comp rx=0 tx=0).stop 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 msgr2=0x7fe1340c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 0x7fe1340c3a00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe12400d800 tx=0x7fe12400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 shutdown_connections 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe11806a080 0x7fe11806c540 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1340c8d20 0x7fe1340b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1340b2850 0x7fe1340c3a00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 --2- 172.21.15.70:0/1053272658 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1340b2470 0x7fe1341eda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 >> 172.21.15.70:0/1053272658 conn(0x7fe1340ce830 msgr2=0x7fe1340c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.174+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 shutdown_connections 2023-08-25T09:40:09.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.175+0000 7fe13a309640 1 -- 172.21.15.70:0/1053272658 wait complete. 2023-08-25T09:40:09.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:09.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.410+0000 7fc20759e640 1 Processor -- start 2023-08-25T09:40:09.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.410+0000 7fc20759e640 1 -- start start 2023-08-25T09:40:09.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.411+0000 7fc20759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.411+0000 7fc20759e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2080c8d20 0x7fc2080c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.411+0000 7fc20759e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2080cd800 con 0x7fc2080b2470 2023-08-25T09:40:09.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.411+0000 7fc20759e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20815ad20 con 0x7fc2080c8d20 2023-08-25T09:40:09.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.411+0000 7fc20759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815a110 con 0x7fc2080b2850 2023-08-25T09:40:09.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2080c8d20 0x7fc2080c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2080c8d20 0x7fc2080c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42114/0 (socket says 172.21.15.70:42114) 2023-08-25T09:40:09.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 -- 172.21.15.70:0/3526395586 learned_addr learned my addr 172.21.15.70:0/3526395586 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:09.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc1fffff640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 -- 172.21.15.70:0/3526395586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 msgr2=0x7fc2080cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:09.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 -- 172.21.15.70:0/3526395586 >> v1:172.21.15.155:6789/0 conn(0x7fc2080b2470 legacy=0x7fc208159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:09.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc206d9d640 1 -- 172.21.15.70:0/3526395586 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20815a3f0 con 0x7fc2080c8d20 2023-08-25T09:40:09.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.412+0000 7fc1fffff640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:09.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.413+0000 7fc206d9d640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080c91b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc200009e50 tx=0x7fc20002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482e67bc9c8187a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:09.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc205d9b640 1 -- 172.21.15.70:0/3526395586 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20002ca80 con 0x7fc2080c8d20 2023-08-25T09:40:09.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc205d9b640 1 -- 172.21.15.70:0/3526395586 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20002cbe0 con 0x7fc2080c8d20 2023-08-25T09:40:09.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc205d9b640 1 -- 172.21.15.70:0/3526395586 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2000343f0 con 0x7fc2080c8d20 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 -- 172.21.15.70:0/3526395586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 msgr2=0x7fc2080c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080c91b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc200009e50 tx=0x7fc20002bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 -- 172.21.15.70:0/3526395586 shutdown_connections 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080c91b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 --2- 172.21.15.70:0/3526395586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 -- 172.21.15.70:0/3526395586 >> 172.21.15.70:0/3526395586 conn(0x7fc2080ce830 msgr2=0x7fc2080cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:09.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 -- 172.21.15.70:0/3526395586 shutdown_connections 2023-08-25T09:40:09.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.414+0000 7fc20759e640 1 -- 172.21.15.70:0/3526395586 wait complete. 2023-08-25T09:40:09.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.416+0000 7fc20759e640 1 Processor -- start 2023-08-25T09:40:09.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.416+0000 7fc20759e640 1 -- start start 2023-08-25T09:40:09.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.417+0000 7fc20759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2080b2470 0x7fc2080c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.417+0000 7fc20759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.417+0000 7fc20759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.417+0000 7fc20759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815dd20 con 0x7fc2080c8d20 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.417+0000 7fc20759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815e1b0 con 0x7fc2080b2470 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.417+0000 7fc20759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20815d110 con 0x7fc2080b2850 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42116/0 (socket says 172.21.15.70:42116) 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 -- 172.21.15.70:0/3607465531 learned_addr learned my addr 172.21.15.70:0/3607465531 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:09.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc1fffff640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc20659c640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2080b2470 0x7fc2080c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 -- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 msgr2=0x7fc2080af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 -- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2080b2470 msgr2=0x7fc2080c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2080b2470 0x7fc2080c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 -- 172.21.15.70:0/3607465531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200009b30 con 0x7fc2080c8d20 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc1fffff640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:09.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.418+0000 7fc206d9d640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080cc6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc200036040 tx=0x7fc200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.419+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200040670 con 0x7fc2080c8d20 2023-08-25T09:40:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.419+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2000407f0 con 0x7fc2080c8d20 2023-08-25T09:40:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.419+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200040ba0 con 0x7fc2080c8d20 2023-08-25T09:40:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.419+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20815d3f0 con 0x7fc2080c8d20 2023-08-25T09:40:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.419+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20815d9b0 con 0x7fc2080c8d20 2023-08-25T09:40:09.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.420+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20815ded0 con 0x7fc2080c8d20 2023-08-25T09:40:09.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.421+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc20002cd50 con 0x7fc2080c8d20 2023-08-25T09:40:09.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.422+0000 7fc1feffd640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1e806a080 0x7fc1e806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.422+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2000ab190 con 0x7fc2080c8d20 2023-08-25T09:40:09.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.422+0000 7fc20659c640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1e806a080 0x7fc1e806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.423+0000 7fc20659c640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1e806a080 0x7fc1e806c540 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc208069c40 tx=0x7fc1f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:09.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.426+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc20815ded0 con 0x7fc2080c8d20 2023-08-25T09:40:09.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.614+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc2080c65d0 con 0x7fc2080c8d20 2023-08-25T09:40:09.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.615+0000 7fc1feffd640 1 -- 172.21.15.70:0/3607465531 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc200076870 con 0x7fc2080c8d20 2023-08-25T09:40:09.617 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:09.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.617+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1e806a080 msgr2=0x7fc1e806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.617+0000 7fc20759e640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1e806a080 0x7fc1e806c540 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc208069c40 tx=0x7fc1f8009040 comp rx=0 tx=0).stop 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.617+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 msgr2=0x7fc2080cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.617+0000 7fc20759e640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080cc6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc200036040 tx=0x7fc200004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 shutdown_connections 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1e806a080 0x7fc1e806c540 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2080c8d20 0x7fc2080cc6d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2080b2850 0x7fc2080af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 --2- 172.21.15.70:0/3607465531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2080b2470 0x7fc2080c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 >> 172.21.15.70:0/3607465531 conn(0x7fc2080ce830 msgr2=0x7fc20806a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 shutdown_connections 2023-08-25T09:40:09.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.618+0000 7fc20759e640 1 -- 172.21.15.70:0/3607465531 wait complete. 2023-08-25T09:40:09.639 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:09.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.856+0000 7f1e4a54e640 1 Processor -- start 2023-08-25T09:40:09.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.856+0000 7f1e4a54e640 1 -- start start 2023-08-25T09:40:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e4a54e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e440c9790 0x7f1e440ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e4a54e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e440c9c40 0x7f1e440ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e4a54e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e441597d0 con 0x7f1e4406ad30 2023-08-25T09:40:09.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e4a54e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4415a5b0 con 0x7f1e440c9790 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e4a54e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4415a890 con 0x7f1e440c9c40 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e3bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e440c9790 0x7f1e440ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.857+0000 7f1e43fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1e4406ad30 0x7f1e4406b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39886/0 (socket says 172.21.15.70:39886) 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e43fff640 1 -- 172.21.15.70:0/4226382299 learned_addr learned my addr 172.21.15.70:0/4226382299 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e48aeb640 1 --2- 172.21.15.70:0/4226382299 >> v2:172.21.15.155:3300/0 conn(0x7f1e440c9c40 0x7f1e440ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e3bfff640 1 -- 172.21.15.70:0/4226382299 >> v2:172.21.15.155:3300/0 conn(0x7f1e440c9c40 msgr2=0x7f1e440ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:09.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e437fe640 1 -- 172.21.15.70:0/4226382299 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111400597 0 0) 0x7f1e441597d0 con 0x7f1e4406ad30 2023-08-25T09:40:09.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e3bfff640 1 --2- 172.21.15.70:0/4226382299 >> v2:172.21.15.155:3300/0 conn(0x7f1e440c9c40 0x7f1e440ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e3bfff640 1 -- 172.21.15.70:0/4226382299 >> v1:172.21.15.70:6789/0 conn(0x7f1e4406ad30 legacy=0x7f1e4406b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e3bfff640 1 -- 172.21.15.70:0/4226382299 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440c8cb0 con 0x7f1e440c9790 2023-08-25T09:40:09.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.858+0000 7f1e48aeb640 1 --2- 172.21.15.70:0/4226382299 >> v2:172.21.15.155:3300/0 conn(0x7f1e440c9c40 0x7f1e440ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:09.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.859+0000 7f1e3bfff640 1 --2- 172.21.15.70:0/4226382299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440ca2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0097a0 tx=0x7f1e2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3add616a4cd867f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:09.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.859+0000 7f1e437fe640 1 -- 172.21.15.70:0/4226382299 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c031830 con 0x7f1e440c9790 2023-08-25T09:40:09.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.859+0000 7f1e437fe640 1 -- 172.21.15.70:0/4226382299 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c031990 con 0x7f1e440c9790 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.859+0000 7f1e437fe640 1 -- 172.21.15.70:0/4226382299 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c031d20 con 0x7f1e440c9790 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 -- 172.21.15.70:0/4226382299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 msgr2=0x7f1e440ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/4226382299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440ca2f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0097a0 tx=0x7f1e2c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 -- 172.21.15.70:0/4226382299 shutdown_connections 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/4226382299 >> v2:172.21.15.155:3300/0 conn(0x7f1e440c9c40 0x7f1e440ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/4226382299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440ca2f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 -- 172.21.15.70:0/4226382299 >> 172.21.15.70:0/4226382299 conn(0x7f1e440cb0d0 msgr2=0x7f1e440cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:09.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 -- 172.21.15.70:0/4226382299 shutdown_connections 2023-08-25T09:40:09.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.860+0000 7f1e4a54e640 1 -- 172.21.15.70:0/4226382299 wait complete. 2023-08-25T09:40:09.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.862+0000 7f1e4a54e640 1 Processor -- start 2023-08-25T09:40:09.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.862+0000 7f1e4a54e640 1 -- start start 2023-08-25T09:40:09.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e4a54e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4406ad30 0x7f1e4415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e4a54e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e4a54e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e440c9c40 0x7f1e440cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e4a54e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4415d170 con 0x7f1e440c9790 2023-08-25T09:40:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e4a54e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4415c4f0 con 0x7f1e440c9c40 2023-08-25T09:40:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e4a54e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4415c7d0 con 0x7f1e4406ad30 2023-08-25T09:40:09.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e3bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e3bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42130/0 (socket says 172.21.15.70:42130) 2023-08-25T09:40:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.863+0000 7f1e3bfff640 1 -- 172.21.15.70:0/1000819730 learned_addr learned my addr 172.21.15.70:0/1000819730 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e43fff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4406ad30 0x7f1e4415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e3bfff640 1 -- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4406ad30 msgr2=0x7f1e4415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e48aeb640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e440c9c40 0x7f1e440cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e3bfff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4406ad30 0x7f1e4415d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e3bfff640 1 -- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e440c9c40 msgr2=0x7f1e440cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e3bfff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e440c9c40 0x7f1e440cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e3bfff640 1 -- 172.21.15.70:0/1000819730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c009400 con 0x7f1e440c9790 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e43fff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4406ad30 0x7f1e4415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e3bfff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440cb4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0098c0 tx=0x7f1e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c03ba70 con 0x7f1e440c9790 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c03bbf0 con 0x7f1e440c9790 2023-08-25T09:40:09.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0453f0 con 0x7f1e440c9790 2023-08-25T09:40:09.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.864+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4415cab0 con 0x7f1e440c9790 2023-08-25T09:40:09.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.865+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e440c8550 con 0x7f1e440c9790 2023-08-25T09:40:09.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.866+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4415dad0 con 0x7f1e440c9790 2023-08-25T09:40:09.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.867+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e2c04e450 con 0x7f1e440c9790 2023-08-25T09:40:09.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.868+0000 7f1e417fa640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e2406a050 0x7f1e2406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:09.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.868+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e2c0aa860 con 0x7f1e440c9790 2023-08-25T09:40:09.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.868+0000 7f1e43fff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e2406a050 0x7f1e2406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:09.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.869+0000 7f1e43fff640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e2406a050 0x7f1e2406c510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e34000f60 tx=0x7f1e34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:09.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:09.873+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e2c075f40 con 0x7f1e440c9790 2023-08-25T09:40:10.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.060+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e44066fb0 con 0x7f1e440c9790 2023-08-25T09:40:10.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.061+0000 7f1e417fa640 1 -- 172.21.15.70:0/1000819730 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e2c079560 con 0x7f1e440c9790 2023-08-25T09:40:10.064 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e2406a050 msgr2=0x7f1e2406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e2406a050 0x7f1e2406c510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1e34000f60 tx=0x7f1e34009040 comp rx=0 tx=0).stop 2023-08-25T09:40:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 msgr2=0x7f1e440cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440cb4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0098c0 tx=0x7f1e2c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 shutdown_connections 2023-08-25T09:40:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e2406a050 0x7f1e2406c510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e440c9c40 0x7f1e440cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e440c9790 0x7f1e440cb4b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 --2- 172.21.15.70:0/1000819730 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e4406ad30 0x7f1e4415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 >> 172.21.15.70:0/1000819730 conn(0x7f1e440cb0d0 msgr2=0x7f1e441553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 shutdown_connections 2023-08-25T09:40:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.064+0000 7f1e4a54e640 1 -- 172.21.15.70:0/1000819730 wait complete. 2023-08-25T09:40:10.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.301+0000 7fa56cc34640 1 Processor -- start 2023-08-25T09:40:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.301+0000 7fa56cc34640 1 -- start start 2023-08-25T09:40:10.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.302+0000 7fa56cc34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 0x7fa56815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.302+0000 7fa56cc34640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56815e780 con 0x7fa5680cd660 2023-08-25T09:40:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.302+0000 7fa56cc34640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56806f8e0 con 0x7fa56806f0e0 2023-08-25T09:40:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.302+0000 7fa56cc34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56806fbc0 con 0x7fa568159bf0 2023-08-25T09:40:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa566d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa5680cd660 0x7fa5680c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39892/0 (socket says 172.21.15.70:39892) 2023-08-25T09:40:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa566d9d640 1 -- 172.21.15.70:0/3402201513 learned_addr learned my addr 172.21.15.70:0/3402201513 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa56659c640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 0x7fa56815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092191175 0 0) 0x7fa56815e780 con 0x7fa5680cd660 2023-08-25T09:40:10.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa554003520 con 0x7fa5680cd660 2023-08-25T09:40:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1291228869 0 0) 0x7fa554003520 con 0x7fa5680cd660 2023-08-25T09:40:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 msgr2=0x7fa56815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa565d9b640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 0x7fa56815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.303+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 >> v1:172.21.15.155:6789/0 conn(0x7fa56806f0e0 legacy=0x7fa568070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56806fea0 con 0x7fa5680cd660 2023-08-25T09:40:10.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa5600019f0 con 0x7fa5680cd660 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 >> v1:172.21.15.70:6789/0 conn(0x7fa5680cd660 legacy=0x7fa5680c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5540039a0 0x7fa554003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 0x7fa5540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554004d00 0x7fa554005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56806fbc0 con 0x7fa5540039a0 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55400dc40 con 0x7fa554004d00 2023-08-25T09:40:10.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55400de00 con 0x7fa568159bf0 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa56659c640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5540039a0 0x7fa554003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa56659c640 1 -- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 msgr2=0x7fa5540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa56659c640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa568159bf0 0x7fa5540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa566d9d640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554004d00 0x7fa554005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa56659c640 1 -- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554004d00 msgr2=0x7fa554005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa56659c640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa554004d00 0x7fa554005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.304+0000 7fa56659c640 1 -- 172.21.15.70:0/3402201513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa554003520 con 0x7fa5540039a0 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.306+0000 7fa56659c640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5540039a0 0x7fa554003dc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa558009a90 tx=0x7fa5540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0e97137f8a164c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55802c6b0 con 0x7fa5540039a0 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55802c810 con 0x7fa5540039a0 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa565d9b640 1 -- 172.21.15.70:0/3402201513 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55802cba0 con 0x7fa5540039a0 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa56cc34640 1 -- 172.21.15.70:0/3402201513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5540039a0 msgr2=0x7fa554003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa56cc34640 1 --2- 172.21.15.70:0/3402201513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa5540039a0 0x7fa554003dc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa558009a90 tx=0x7fa5540057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa566d9d640 1 -- 172.21.15.70:0/3402201513 reap_dead start 2023-08-25T09:40:10.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa56cc34640 1 -- 172.21.15.70:0/3402201513 shutdown_connections 2023-08-25T09:40:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa56cc34640 1 -- 172.21.15.70:0/3402201513 >> 172.21.15.70:0/3402201513 conn(0x7fa5680cb0d0 msgr2=0x7fa5680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa56cc34640 1 -- 172.21.15.70:0/3402201513 shutdown_connections 2023-08-25T09:40:10.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.307+0000 7fa56cc34640 1 -- 172.21.15.70:0/3402201513 wait complete. 2023-08-25T09:40:10.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.309+0000 7fa56cc34640 1 Processor -- start 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.309+0000 7fa56cc34640 1 -- start start 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.310+0000 7fa56cc34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5540039a0 0x7fa5680c32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.310+0000 7fa56cc34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 0x7fa568144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.310+0000 7fa56cc34640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5680c5690 0x7fa5681575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.310+0000 7fa56cc34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5681f2880 con 0x7fa568151d40 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.310+0000 7fa56cc34640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa568154270 con 0x7fa5680c5690 2023-08-25T09:40:10.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.310+0000 7fa56cc34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa568154550 con 0x7fa5540039a0 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 0x7fa568144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 0x7fa568144ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42144/0 (socket says 172.21.15.70:42144) 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 -- 172.21.15.70:0/2391346037 learned_addr learned my addr 172.21.15.70:0/2391346037 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa56659c640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5540039a0 0x7fa5680c32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 -- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5540039a0 msgr2=0x7fa5680c32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa566d9d640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5680c5690 0x7fa5681575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5540039a0 0x7fa5680c32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 -- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5680c5690 msgr2=0x7fa5681575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5680c5690 0x7fa5681575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 -- 172.21.15.70:0/2391346037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568154830 con 0x7fa568151d40 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa56659c640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5540039a0 0x7fa5680c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.311+0000 7fa55ffff640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 0x7fa568144ac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c500 tx=0x7fa55000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.312+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5500052d0 con 0x7fa568151d40 2023-08-25T09:40:10.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.312+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550005430 con 0x7fa568151d40 2023-08-25T09:40:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.312+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa568154b10 con 0x7fa568151d40 2023-08-25T09:40:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.312+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550012870 con 0x7fa568151d40 2023-08-25T09:40:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.312+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa568153f40 con 0x7fa568151d40 2023-08-25T09:40:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.313+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa568066fb0 con 0x7fa568151d40 2023-08-25T09:40:10.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.314+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa5500129d0 con 0x7fa568151d40 2023-08-25T09:40:10.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.314+0000 7fa55effd640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa54806a080 0x7fa54806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.315+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa5500880f0 con 0x7fa568151d40 2023-08-25T09:40:10.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.315+0000 7fa56659c640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa54806a080 0x7fa54806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.315+0000 7fa56659c640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa54806a080 0x7fa54806c540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa558036000 tx=0x7fa558030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:10.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.319+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa550053870 con 0x7fa568151d40 2023-08-25T09:40:10.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.510+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5680c9ac0 con 0x7fa568151d40 2023-08-25T09:40:10.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.511+0000 7fa55effd640 1 -- 172.21.15.70:0/2391346037 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa550056e90 con 0x7fa568151d40 2023-08-25T09:40:10.513 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.513+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa54806a080 msgr2=0x7fa54806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.513+0000 7fa56cc34640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa54806a080 0x7fa54806c540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa558036000 tx=0x7fa558030000 comp rx=0 tx=0).stop 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.513+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 msgr2=0x7fa568144ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.513+0000 7fa56cc34640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 0x7fa568144ac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c500 tx=0x7fa55000c9c0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 shutdown_connections 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa54806a080 0x7fa54806c540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa5680c5690 0x7fa5681575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa568151d40 0x7fa568144ac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 --2- 172.21.15.70:0/2391346037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa5540039a0 0x7fa5680c32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 >> 172.21.15.70:0/2391346037 conn(0x7fa5680cb0d0 msgr2=0x7fa5680c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:10.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 shutdown_connections 2023-08-25T09:40:10.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.514+0000 7fa56cc34640 1 -- 172.21.15.70:0/2391346037 wait complete. 2023-08-25T09:40:10.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:10.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.748+0000 7fdbb6f37640 1 Processor -- start 2023-08-25T09:40:10.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.748+0000 7fdbb6f37640 1 -- start start 2023-08-25T09:40:10.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb6f37640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdbb0157170 0x7fdbb00afea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb6f37640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb00b03e0 0x7fdbb00aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb6f37640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb00c9f70 con 0x7fdbb00acf80 2023-08-25T09:40:10.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb6f37640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb00b0860 con 0x7fdbb0157170 2023-08-25T09:40:10.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb6f37640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00c9980 con 0x7fdbb00b03e0 2023-08-25T09:40:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdbb0157170 0x7fdbb00afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdbb0157170 0x7fdbb00afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42154/0 (socket says 172.21.15.70:42154) 2023-08-25T09:40:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 -- 172.21.15.70:0/1817749325 learned_addr learned my addr 172.21.15.70:0/1817749325 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 -- 172.21.15.70:0/1817749325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb00b03e0 msgr2=0x7fdbb00aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 --2- 172.21.15.70:0/1817749325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb00b03e0 0x7fdbb00aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 -- 172.21.15.70:0/1817749325 >> v1:172.21.15.155:6789/0 conn(0x7fdbb00acf80 legacy=0x7fdbb01605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.750+0000 7fdbb5f35640 1 -- 172.21.15.70:0/1817749325 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb00ad7b0 con 0x7fdbb0157170 2023-08-25T09:40:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.751+0000 7fdbb5f35640 1 --2- 172.21.15.70:0/1817749325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb0157170 0x7fdbb00afea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009870 tx=0x7fdbac02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8ce6f726ec7b77 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb4f33640 1 -- 172.21.15.70:0/1817749325 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac0317c0 con 0x7fdbb0157170 2023-08-25T09:40:10.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb4f33640 1 -- 172.21.15.70:0/1817749325 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbac031920 con 0x7fdbb0157170 2023-08-25T09:40:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb4f33640 1 -- 172.21.15.70:0/1817749325 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbac031cb0 con 0x7fdbb0157170 2023-08-25T09:40:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb6f37640 1 -- 172.21.15.70:0/1817749325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb0157170 msgr2=0x7fdbb00afea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/1817749325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb0157170 0x7fdbb00afea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdbac009870 tx=0x7fdbac02b860 comp rx=0 tx=0).stop 2023-08-25T09:40:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb6f37640 1 -- 172.21.15.70:0/1817749325 shutdown_connections 2023-08-25T09:40:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.752+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/1817749325 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb00b03e0 0x7fdbb00aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.753+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/1817749325 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb0157170 0x7fdbb00afea0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.753+0000 7fdbb6f37640 1 -- 172.21.15.70:0/1817749325 >> 172.21.15.70:0/1817749325 conn(0x7fdbb0069e80 msgr2=0x7fdbb006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.753+0000 7fdbb6f37640 1 -- 172.21.15.70:0/1817749325 shutdown_connections 2023-08-25T09:40:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.753+0000 7fdbb6f37640 1 -- 172.21.15.70:0/1817749325 wait complete. 2023-08-25T09:40:10.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.754+0000 7fdbb6f37640 1 Processor -- start 2023-08-25T09:40:10.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.755+0000 7fdbb6f37640 1 -- start start 2023-08-25T09:40:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.755+0000 7fdbb6f37640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00acf80 0x7fdbb01519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb6f37640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 0x7fdbb015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb6f37640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb0157170 0x7fdbb015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb6f37640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb015fe20 con 0x7fdbb00b03e0 2023-08-25T09:40:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb6f37640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00cdca0 con 0x7fdbb00acf80 2023-08-25T09:40:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb6f37640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb00cdf80 con 0x7fdbb0157170 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 0x7fdbb015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 0x7fdbb015f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42162/0 (socket says 172.21.15.70:42162) 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 -- 172.21.15.70:0/2259902989 learned_addr learned my addr 172.21.15.70:0/2259902989 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5f35640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00acf80 0x7fdbb01519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 -- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb0157170 msgr2=0x7fdbb015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb0157170 0x7fdbb015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 -- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00acf80 msgr2=0x7fdbb01519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00acf80 0x7fdbb01519e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.756+0000 7fdbb5734640 1 -- 172.21.15.70:0/2259902989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0094d0 con 0x7fdbb00b03e0 2023-08-25T09:40:10.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdbb5734640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 0x7fdbb015f3a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b410 tx=0x7fdba000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:10.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0011830 con 0x7fdbb00b03e0 2023-08-25T09:40:10.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba00119b0 con 0x7fdbb00b03e0 2023-08-25T09:40:10.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb00ce260 con 0x7fdbb00b03e0 2023-08-25T09:40:10.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0011d60 con 0x7fdbb00b03e0 2023-08-25T09:40:10.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb00ce820 con 0x7fdbb00b03e0 2023-08-25T09:40:10.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.758+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb0066fb0 con 0x7fdbb00b03e0 2023-08-25T09:40:10.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.759+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdba001a400 con 0x7fdbb00b03e0 2023-08-25T09:40:10.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.761+0000 7fdba67fc640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb9406a050 0x7fdb9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:10.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.761+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdba0088800 con 0x7fdbb00b03e0 2023-08-25T09:40:10.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.761+0000 7fdbb5f35640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb9406a050 0x7fdb9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:10.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.762+0000 7fdbb5f35640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb9406a050 0x7fdb9406c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdbac036000 tx=0x7fdbac03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.766+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdba0053e50 con 0x7fdbb00b03e0 2023-08-25T09:40:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.989+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbb00cb5f0 con 0x7fdbb00b03e0 2023-08-25T09:40:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.990+0000 7fdba67fc640 1 -- 172.21.15.70:0/2259902989 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7fdba0057470 con 0x7fdbb00b03e0 2023-08-25T09:40:10.992 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:10.992 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:40:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:40:10.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb9406a050 msgr2=0x7fdb9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb9406a050 0x7fdb9406c510 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdbac036000 tx=0x7fdbac03d040 comp rx=0 tx=0).stop 2023-08-25T09:40:10.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 msgr2=0x7fdbb015f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:10.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 0x7fdbb015f3a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdba000b410 tx=0x7fdba000b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 shutdown_connections 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb9406a050 0x7fdb9406c510 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdbb0157170 0x7fdbb015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdbb00b03e0 0x7fdbb015f3a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 --2- 172.21.15.70:0/2259902989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdbb00acf80 0x7fdbb01519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 >> 172.21.15.70:0/2259902989 conn(0x7fdbb0069e80 msgr2=0x7fdbb00ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.992+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 shutdown_connections 2023-08-25T09:40:10.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:10.993+0000 7fdbb6f37640 1 -- 172.21.15.70:0/2259902989 wait complete. 2023-08-25T09:40:11.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.160+0000 7ff276acc640 1 Processor -- start 2023-08-25T09:40:12.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.160+0000 7ff276acc640 1 -- start start 2023-08-25T09:40:12.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff276acc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff27015cb40 0x7ff2700ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff276acc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff2700ac870 0x7ff2700ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff276acc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27006f050 con 0x7ff2700b3880 2023-08-25T09:40:12.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff276acc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27006f330 con 0x7ff2700ac870 2023-08-25T09:40:12.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff276acc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27006f610 con 0x7ff27015cb40 2023-08-25T09:40:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff26ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff2700ac870 0x7ff2700ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff275069640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff2700b3880 0x7ff270160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39900/0 (socket says 172.21.15.70:39900) 2023-08-25T09:40:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.162+0000 7ff275069640 1 -- 172.21.15.70:0/1869961691 learned_addr learned my addr 172.21.15.70:0/1869961691 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:12.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff274868640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff27015cb40 0x7ff2700ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff26ffff640 1 -- 172.21.15.70:0/1869961691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff27015cb40 msgr2=0x7ff2700ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff26f7fe640 1 -- 172.21.15.70:0/1869961691 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093783879 0 0) 0x7ff27006f050 con 0x7ff2700b3880 2023-08-25T09:40:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff26ffff640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff27015cb40 0x7ff2700ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff26ffff640 1 -- 172.21.15.70:0/1869961691 >> v1:172.21.15.70:6789/0 conn(0x7ff2700b3880 legacy=0x7ff270160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff26ffff640 1 -- 172.21.15.70:0/1869961691 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27006f8f0 con 0x7ff2700ac870 2023-08-25T09:40:12.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.163+0000 7ff274868640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff27015cb40 0x7ff2700ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:12.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.164+0000 7ff26ffff640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700ac870 0x7ff2700ce860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff2580097a0 tx=0x7ff25802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ba436e045c3177 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.164+0000 7ff26f7fe640 1 -- 172.21.15.70:0/1869961691 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25802d040 con 0x7ff2700ac870 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.164+0000 7ff26f7fe640 1 -- 172.21.15.70:0/1869961691 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff258031810 con 0x7ff2700ac870 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.164+0000 7ff26f7fe640 1 -- 172.21.15.70:0/1869961691 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258031ba0 con 0x7ff2700ac870 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 -- 172.21.15.70:0/1869961691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700ac870 msgr2=0x7ff2700ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700ac870 0x7ff2700ce860 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff2580097a0 tx=0x7ff25802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 -- 172.21.15.70:0/1869961691 shutdown_connections 2023-08-25T09:40:12.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700ac870 0x7ff2700ce860 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 --2- 172.21.15.70:0/1869961691 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff27015cb40 0x7ff2700ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 -- 172.21.15.70:0/1869961691 >> 172.21.15.70:0/1869961691 conn(0x7ff270069850 msgr2=0x7ff270069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 -- 172.21.15.70:0/1869961691 shutdown_connections 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.165+0000 7ff276acc640 1 -- 172.21.15.70:0/1869961691 wait complete. 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.167+0000 7ff276acc640 1 Processor -- start 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.167+0000 7ff276acc640 1 -- start start 2023-08-25T09:40:12.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff276acc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2700ac870 0x7ff2701f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff276acc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 0x7ff27015b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff276acc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff27015cb40 0x7ff27015fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff276acc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27015a130 con 0x7ff2700b3880 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff276acc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2701f5470 con 0x7ff27015cb40 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff276acc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27015a380 con 0x7ff2700ac870 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 0x7ff27015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 0x7ff27015b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42178/0 (socket says 172.21.15.70:42178) 2023-08-25T09:40:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 -- 172.21.15.70:0/1570211938 learned_addr learned my addr 172.21.15.70:0/1570211938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff274868640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2700ac870 0x7ff2701f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff275069640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff27015cb40 0x7ff27015fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 -- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2700ac870 msgr2=0x7ff2701f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2700ac870 0x7ff2701f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 -- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff27015cb40 msgr2=0x7ff27015fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.168+0000 7ff26ffff640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff27015cb40 0x7ff27015fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff26ffff640 1 -- 172.21.15.70:0/1570211938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258009400 con 0x7ff2700b3880 2023-08-25T09:40:12.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff274868640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2700ac870 0x7ff2701f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:12.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff26ffff640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 0x7ff27015b420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098c0 tx=0x7ff2580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:12.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25803ad60 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2580423f0 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580427a0 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27015a660 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.169+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27015ac20 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.170+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27004e420 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.171+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff2580393b0 con 0x7ff2700b3880 2023-08-25T09:40:12.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.172+0000 7ff26d7fa640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff25006a050 0x7ff25006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.172+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2580aa350 con 0x7ff2700b3880 2023-08-25T09:40:12.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.172+0000 7ff274868640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff25006a050 0x7ff25006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.173+0000 7ff274868640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff25006a050 0x7ff25006c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2701f6770 tx=0x7ff260007550 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:12.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.176+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff258075a30 con 0x7ff2700b3880 2023-08-25T09:40:12.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.354+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff27006bd20 con 0x7ff2700b3880 2023-08-25T09:40:12.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.356+0000 7ff26d7fa640 1 -- 172.21.15.70:0/1570211938 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7ff258079050 con 0x7ff2700b3880 2023-08-25T09:40:12.359 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:12.359 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:12.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff25006a050 msgr2=0x7ff25006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff25006a050 0x7ff25006c510 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2701f6770 tx=0x7ff260007550 comp rx=0 tx=0).stop 2023-08-25T09:40:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 msgr2=0x7ff27015b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 0x7ff27015b420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098c0 tx=0x7ff2580046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 shutdown_connections 2023-08-25T09:40:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff25006a050 0x7ff25006c510 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff27015cb40 0x7ff27015fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2700b3880 0x7ff27015b420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 --2- 172.21.15.70:0/1570211938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2700ac870 0x7ff2701f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 >> 172.21.15.70:0/1570211938 conn(0x7ff270069850 msgr2=0x7ff27006c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 shutdown_connections 2023-08-25T09:40:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.359+0000 7ff276acc640 1 -- 172.21.15.70:0/1570211938 wait complete. 2023-08-25T09:40:12.380 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:40:12.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:12.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.579+0000 7ff577fff640 1 Processor -- start 2023-08-25T09:40:12.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.580+0000 7ff577fff640 1 -- start start 2023-08-25T09:40:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.581+0000 7ff577fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff57815a7c0 0x7ff5780b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.581+0000 7ff577fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5780b12f0 0x7ff57815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.581+0000 7ff577fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff57815e7c0 con 0x7ff57815f190 2023-08-25T09:40:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.581+0000 7ff577fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff578071b50 con 0x7ff5780b12f0 2023-08-25T09:40:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.581+0000 7ff577fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff57815e490 con 0x7ff57815a7c0 2023-08-25T09:40:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.581+0000 7ff56ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5780b12f0 0x7ff57815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff56ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5780b12f0 0x7ff57815ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42184/0 (socket says 172.21.15.70:42184) 2023-08-25T09:40:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff56ffff640 1 -- 172.21.15.70:0/3391110516 learned_addr learned my addr 172.21.15.70:0/3391110516 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:12.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff56ffff640 1 -- 172.21.15.70:0/3391110516 >> v2:172.21.15.155:3300/0 conn(0x7ff57815a7c0 msgr2=0x7ff5780b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:12.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff576ffd640 1 --2- 172.21.15.70:0/3391110516 >> v2:172.21.15.155:3300/0 conn(0x7ff57815a7c0 0x7ff5780b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff56ffff640 1 --2- 172.21.15.70:0/3391110516 >> v2:172.21.15.155:3300/0 conn(0x7ff57815a7c0 0x7ff5780b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff56ffff640 1 -- 172.21.15.70:0/3391110516 >> v1:172.21.15.155:6789/0 conn(0x7ff57815f190 legacy=0x7ff5780cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff56ffff640 1 -- 172.21.15.70:0/3391110516 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5780b3aa0 con 0x7ff5780b12f0 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.582+0000 7ff576ffd640 1 --2- 172.21.15.70:0/3391110516 >> v2:172.21.15.155:3300/0 conn(0x7ff57815a7c0 0x7ff5780b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.583+0000 7ff56ffff640 1 --2- 172.21.15.70:0/3391110516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5780b12f0 0x7ff57815ec50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff5780ce280 tx=0x7ff56002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedefccbe7d4dba8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.583+0000 7ff5767fc640 1 -- 172.21.15.70:0/3391110516 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560031870 con 0x7ff5780b12f0 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff5767fc640 1 -- 172.21.15.70:0/3391110516 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5600319d0 con 0x7ff5780b12f0 2023-08-25T09:40:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff5767fc640 1 -- 172.21.15.70:0/3391110516 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff560031d60 con 0x7ff5780b12f0 2023-08-25T09:40:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 -- 172.21.15.70:0/3391110516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5780b12f0 msgr2=0x7ff57815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 --2- 172.21.15.70:0/3391110516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5780b12f0 0x7ff57815ec50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff5780ce280 tx=0x7ff56002b910 comp rx=0 tx=0).stop 2023-08-25T09:40:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 -- 172.21.15.70:0/3391110516 shutdown_connections 2023-08-25T09:40:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 --2- 172.21.15.70:0/3391110516 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5780b12f0 0x7ff57815ec50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 --2- 172.21.15.70:0/3391110516 >> v2:172.21.15.155:3300/0 conn(0x7ff57815a7c0 0x7ff5780b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 -- 172.21.15.70:0/3391110516 >> 172.21.15.70:0/3391110516 conn(0x7ff5780affd0 msgr2=0x7ff5780b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:12.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 -- 172.21.15.70:0/3391110516 shutdown_connections 2023-08-25T09:40:12.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.584+0000 7ff577fff640 1 -- 172.21.15.70:0/3391110516 wait complete. 2023-08-25T09:40:12.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.586+0000 7ff577fff640 1 Processor -- start 2023-08-25T09:40:12.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.586+0000 7ff577fff640 1 -- start start 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff577fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5780b12f0 0x7ff5780cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff577fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815a7c0 0x7ff5781ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff577fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 0x7ff5780adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff577fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5781edf80 con 0x7ff57815f190 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff577fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff578153460 con 0x7ff5780b12f0 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff577fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5781ef6c0 con 0x7ff57815a7c0 2023-08-25T09:40:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff5777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 0x7ff5780adcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff5777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 0x7ff5780adcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42192/0 (socket says 172.21.15.70:42192) 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.587+0000 7ff5777fe640 1 -- 172.21.15.70:0/1089876840 learned_addr learned my addr 172.21.15.70:0/1089876840 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff576ffd640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5780b12f0 0x7ff5780cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff56ffff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815a7c0 0x7ff5781ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff5777fe640 1 -- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815a7c0 msgr2=0x7ff5781ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff5777fe640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815a7c0 0x7ff5781ef180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff5777fe640 1 -- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5780b12f0 msgr2=0x7ff5780cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff5777fe640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5780b12f0 0x7ff5780cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff5777fe640 1 -- 172.21.15.70:0/1089876840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5600094d0 con 0x7ff57815f190 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff576ffd640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5780b12f0 0x7ff5780cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff5777fe640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 0x7ff5780adcd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c780 tx=0x7ff57000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570003d80 con 0x7ff57815f190 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57000ce90 con 0x7ff57815f190 2023-08-25T09:40:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5780c7330 con 0x7ff57815f190 2023-08-25T09:40:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.588+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700052b0 con 0x7ff57815f190 2023-08-25T09:40:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.589+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5780c78f0 con 0x7ff57815f190 2023-08-25T09:40:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.590+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5781ee0f0 con 0x7ff57815f190 2023-08-25T09:40:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.591+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff570005410 con 0x7ff57815f190 2023-08-25T09:40:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.592+0000 7ff56f7fe640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff55806a050 0x7ff55806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.592+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff570087ea0 con 0x7ff57815f190 2023-08-25T09:40:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.592+0000 7ff576ffd640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff55806a050 0x7ff55806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.593+0000 7ff576ffd640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff55806a050 0x7ff55806c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff578159910 tx=0x7ff568008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:12.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.596+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff5700534f0 con 0x7ff57815f190 2023-08-25T09:40:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.825+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff578066fb0 con 0x7ff57815f190 2023-08-25T09:40:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.826+0000 7ff56f7fe640 1 -- 172.21.15.70:0/1089876840 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7ff570017070 con 0x7ff57815f190 2023-08-25T09:40:12.828 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:12.828 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:40:12.828 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:40:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff55806a050 msgr2=0x7ff55806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff55806a050 0x7ff55806c510 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff578159910 tx=0x7ff568008040 comp rx=0 tx=0).stop 2023-08-25T09:40:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 msgr2=0x7ff5780adcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:12.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 0x7ff5780adcd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c780 tx=0x7ff57000cc40 comp rx=0 tx=0).stop 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 shutdown_connections 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff55806a050 0x7ff55806c510 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff57815f190 0x7ff5780adcd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff57815a7c0 0x7ff5781ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 --2- 172.21.15.70:0/1089876840 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5780b12f0 0x7ff5780cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 >> 172.21.15.70:0/1089876840 conn(0x7ff5780affd0 msgr2=0x7ff5780c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:12.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 shutdown_connections 2023-08-25T09:40:12.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:12.829+0000 7ff577fff640 1 -- 172.21.15.70:0/1089876840 wait complete. 2023-08-25T09:40:12.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:13.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.053+0000 7f7a3f21f640 1 Processor -- start 2023-08-25T09:40:13.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.053+0000 7f7a3f21f640 1 -- start start 2023-08-25T09:40:13.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3f21f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7a380c4de0 0x7f7a381647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3f21f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3806bc20 con 0x7f7a38159820 2023-08-25T09:40:13.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3f21f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3815df00 con 0x7f7a380ccbb0 2023-08-25T09:40:13.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3f21f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3815e1e0 con 0x7f7a380c4de0 2023-08-25T09:40:13.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3cfbb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a38159820 0x7f7a3815d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39910/0 (socket says 172.21.15.70:39910) 2023-08-25T09:40:13.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3cfbb640 1 -- 172.21.15.70:0/2347455160 learned_addr learned my addr 172.21.15.70:0/2347455160 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:13.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.055+0000 7f7a3d7bc640 1 --2- 172.21.15.70:0/2347455160 >> v2:172.21.15.70:3300/0 conn(0x7f7a380c4de0 0x7f7a381647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.056+0000 7f7a3d7bc640 1 -- 172.21.15.70:0/2347455160 >> v1:172.21.15.155:6789/0 conn(0x7f7a380ccbb0 legacy=0x7f7a380c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:13.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.056+0000 7f7a3d7bc640 1 -- 172.21.15.70:0/2347455160 >> v1:172.21.15.70:6789/0 conn(0x7f7a38159820 legacy=0x7f7a3815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.056+0000 7f7a3d7bc640 1 -- 172.21.15.70:0/2347455160 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3806b8b0 con 0x7f7a380c4de0 2023-08-25T09:40:13.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.056+0000 7f7a377fe640 1 -- 172.21.15.70:0/2347455160 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3603135900 0 0) 0x7f7a3806bc20 con 0x7f7a38159820 2023-08-25T09:40:13.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.057+0000 7f7a3d7bc640 1 --2- 172.21.15.70:0/2347455160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a381647d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009d90 tx=0x7f7a3002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d470d5ff262af7e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:13.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.057+0000 7f7a377fe640 1 -- 172.21.15.70:0/2347455160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3002cc60 con 0x7f7a380c4de0 2023-08-25T09:40:13.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.057+0000 7f7a377fe640 1 -- 172.21.15.70:0/2347455160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3002cdc0 con 0x7f7a380c4de0 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.057+0000 7f7a377fe640 1 -- 172.21.15.70:0/2347455160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30034660 con 0x7f7a380c4de0 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2347455160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 msgr2=0x7f7a381647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2347455160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a381647d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7a30009d90 tx=0x7f7a3002beb0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2347455160 shutdown_connections 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2347455160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a381647d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2347455160 >> 172.21.15.70:0/2347455160 conn(0x7f7a380ce700 msgr2=0x7f7a380ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:13.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2347455160 shutdown_connections 2023-08-25T09:40:13.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.058+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2347455160 wait complete. 2023-08-25T09:40:13.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.060+0000 7f7a3f21f640 1 Processor -- start 2023-08-25T09:40:13.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.060+0000 7f7a3f21f640 1 -- start start 2023-08-25T09:40:13.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3f21f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a380b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3f21f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a380ccbb0 0x7f7a380c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3f21f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a38159820 0x7f7a380adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3f21f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b1fa0 con 0x7f7a380c4de0 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3f21f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b43e0 con 0x7f7a380ccbb0 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3f21f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380c6b80 con 0x7f7a38159820 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a380b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a380b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42210/0 (socket says 172.21.15.70:42210) 2023-08-25T09:40:13.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 -- 172.21.15.70:0/2026880917 learned_addr learned my addr 172.21.15.70:0/2026880917 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a37fff640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a380ccbb0 0x7f7a380c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 -- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a38159820 msgr2=0x7f7a380adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a38159820 0x7f7a380adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 -- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a380ccbb0 msgr2=0x7f7a380c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a380ccbb0 0x7f7a380c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.061+0000 7f7a3cfbb640 1 -- 172.21.15.70:0/2026880917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a30009a70 con 0x7f7a380c4de0 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.062+0000 7f7a3cfbb640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a380b3ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c430 tx=0x7f7a2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.062+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0074e0 con 0x7f7a380c4de0 2023-08-25T09:40:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.062+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c007660 con 0x7f7a380c4de0 2023-08-25T09:40:13.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.062+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c004a20 con 0x7f7a380c4de0 2023-08-25T09:40:13.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.062+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a380c6e60 con 0x7f7a380c4de0 2023-08-25T09:40:13.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.062+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a380c7420 con 0x7f7a380c4de0 2023-08-25T09:40:13.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.063+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3804e420 con 0x7f7a380c4de0 2023-08-25T09:40:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.064+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7a2c004b80 con 0x7f7a380c4de0 2023-08-25T09:40:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.065+0000 7f7a357fa640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7a1c06a080 0x7f7a1c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.065+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a2c088320 con 0x7f7a380c4de0 2023-08-25T09:40:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.066+0000 7f7a37fff640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7a1c06a080 0x7f7a1c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.066+0000 7f7a37fff640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7a1c06a080 0x7f7a1c06c540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7a380cad90 tx=0x7f7a24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.070+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7a2c053aa0 con 0x7f7a380c4de0 2023-08-25T09:40:13.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:40:13.205 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:40:13.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.300+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a380adbb0 con 0x7f7a380c4de0 2023-08-25T09:40:13.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.301+0000 7f7a357fa640 1 -- 172.21.15.70:0/2026880917 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4872 (secure 0 0 0) 0x7f7a2c0570c0 con 0x7f7a380c4de0 2023-08-25T09:40:13.303 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:13.303 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2774877887","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2774877887},{"type":"v1","addr":"172.21.15.155:6833","nonce":2774877887}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/520943521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":520943521},{"type":"v1","addr":"172.21.15.70:6835","nonce":520943521}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/3598251289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":3598251289},{"type":"v1","addr":"172.21.15.70:6837","nonce":3598251289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:38:52.608524+0000","modified":"2023-08-25T09:38:56.204697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6835/2636978427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2636978427},{"type":"v1","addr":"172.21.15.155:6835","nonce":2636978427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T09:40:13.304 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 8 2023-08-25T09:40:13.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7a1c06a080 msgr2=0x7f7a1c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7a1c06a080 0x7f7a1c06c540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7a380cad90 tx=0x7f7a24009040 comp rx=0 tx=0).stop 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 msgr2=0x7f7a380b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a380b3ea0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00c430 tx=0x7f7a2c00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 shutdown_connections 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7a1c06a080 0x7f7a1c06c540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a38159820 0x7f7a380adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a380ccbb0 0x7f7a380c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 --2- 172.21.15.70:0/2026880917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a380c4de0 0x7f7a380b3ea0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 >> 172.21.15.70:0/2026880917 conn(0x7f7a380ce700 msgr2=0x7f7a380c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 shutdown_connections 2023-08-25T09:40:13.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.304+0000 7f7a3f21f640 1 -- 172.21.15.70:0/2026880917 wait complete. 2023-08-25T09:40:13.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:13.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.586+0000 7f7396c46640 1 Processor -- start 2023-08-25T09:40:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.586+0000 7f7396c46640 1 -- start start 2023-08-25T09:40:13.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f7396c46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f739015cb40 0x7f73900ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f7396c46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73900b2b40 0x7f73900ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f7396c46640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f739006f050 con 0x7f73900ac870 2023-08-25T09:40:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f7396c46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f739006f330 con 0x7f739015cb40 2023-08-25T09:40:13.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f7396c46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739006f610 con 0x7f73900b2b40 2023-08-25T09:40:13.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f73949e2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f739015cb40 0x7f73900ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.588+0000 7f738ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f73900ac870 0x7f7390160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39922/0 (socket says 172.21.15.70:39922) 2023-08-25T09:40:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f738ffff640 1 -- 172.21.15.70:0/209872726 learned_addr learned my addr 172.21.15.70:0/209872726 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f73951e3640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73900b2b40 0x7f73900ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f738f7fe640 1 -- 172.21.15.70:0/209872726 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844266471 0 0) 0x7f739006f050 con 0x7f73900ac870 2023-08-25T09:40:13.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f738f7fe640 1 -- 172.21.15.70:0/209872726 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7380003520 con 0x7f73900ac870 2023-08-25T09:40:13.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f73949e2640 1 -- 172.21.15.70:0/209872726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73900b2b40 msgr2=0x7f73900ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:13.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f73949e2640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73900b2b40 0x7f73900ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f73949e2640 1 -- 172.21.15.70:0/209872726 >> v1:172.21.15.70:6789/0 conn(0x7f73900ac870 legacy=0x7f7390160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f73949e2640 1 -- 172.21.15.70:0/209872726 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739006f8f0 con 0x7f739015cb40 2023-08-25T09:40:13.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.589+0000 7f73951e3640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73900b2b40 0x7f73900ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:40:13.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.591+0000 7f73949e2640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f739015cb40 0x7f73900ce320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f73840097a0 tx=0x7f738402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c5ab0451a9d682 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:13.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.591+0000 7f738f7fe640 1 -- 172.21.15.70:0/209872726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384031830 con 0x7f739015cb40 2023-08-25T09:40:13.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.591+0000 7f738f7fe640 1 -- 172.21.15.70:0/209872726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384031990 con 0x7f739015cb40 2023-08-25T09:40:13.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.591+0000 7f738f7fe640 1 -- 172.21.15.70:0/209872726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384031d00 con 0x7f739015cb40 2023-08-25T09:40:13.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 -- 172.21.15.70:0/209872726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f739015cb40 msgr2=0x7f73900ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f739015cb40 0x7f73900ce320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f73840097a0 tx=0x7f738402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 -- 172.21.15.70:0/209872726 shutdown_connections 2023-08-25T09:40:13.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73900b2b40 0x7f73900ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 --2- 172.21.15.70:0/209872726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f739015cb40 0x7f73900ce320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 -- 172.21.15.70:0/209872726 >> 172.21.15.70:0/209872726 conn(0x7f7390069850 msgr2=0x7f7390069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:13.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 -- 172.21.15.70:0/209872726 shutdown_connections 2023-08-25T09:40:13.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.592+0000 7f7396c46640 1 -- 172.21.15.70:0/209872726 wait complete. 2023-08-25T09:40:13.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.595+0000 7f7396c46640 1 Processor -- start 2023-08-25T09:40:13.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.595+0000 7f7396c46640 1 -- start start 2023-08-25T09:40:13.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f7396c46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 0x7f73901501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f7396c46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73900b2b40 0x7f739015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f7396c46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f739015cb40 0x7f7390153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f7396c46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73901f2090 con 0x7f73900ac870 2023-08-25T09:40:13.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f7396c46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7390150730 con 0x7f73900b2b40 2023-08-25T09:40:13.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f7396c46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f739015e390 con 0x7f739015cb40 2023-08-25T09:40:13.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f73949e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 0x7f73901501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f73949e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 0x7f73901501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42236/0 (socket says 172.21.15.70:42236) 2023-08-25T09:40:13.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.596+0000 7f73949e2640 1 -- 172.21.15.70:0/1721236907 learned_addr learned my addr 172.21.15.70:0/1721236907 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:13.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f73949e2640 1 -- 172.21.15.70:0/1721236907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f739015cb40 msgr2=0x7f7390153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f73949e2640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f739015cb40 0x7f7390153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f73949e2640 1 -- 172.21.15.70:0/1721236907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73900b2b40 msgr2=0x7f739015f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f73949e2640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73900b2b40 0x7f739015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f73949e2640 1 -- 172.21.15.70:0/1721236907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7384009400 con 0x7f73900ac870 2023-08-25T09:40:13.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f73949e2640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 0x7f73901501f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7384034040 tx=0x7f73840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:13.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384004450 con 0x7f73900ac870 2023-08-25T09:40:13.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738403f070 con 0x7f73900ac870 2023-08-25T09:40:13.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384003810 con 0x7f73900ac870 2023-08-25T09:40:13.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.597+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739015e670 con 0x7f73900ac870 2023-08-25T09:40:13.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.598+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739015ec30 con 0x7f73900ac870 2023-08-25T09:40:13.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.599+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739004e420 con 0x7f73900ac870 2023-08-25T09:40:13.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.600+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7384003970 con 0x7f73900ac870 2023-08-25T09:40:13.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.601+0000 7f738d7fa640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f737406a050 0x7f737406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:13.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.602+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7384041070 con 0x7f73900ac870 2023-08-25T09:40:13.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.602+0000 7f738ffff640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f737406a050 0x7f737406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:13.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.602+0000 7f738ffff640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f737406a050 0x7f737406c510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f737c000f60 tx=0x7f737c0078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:13.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.606+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7384075d90 con 0x7f73900ac870 2023-08-25T09:40:13.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.801+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7390160a00 con 0x7f73900ac870 2023-08-25T09:40:13.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.804+0000 7f738d7fa640 1 -- 172.21.15.70:0/1721236907 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f73840793b0 con 0x7f73900ac870 2023-08-25T09:40:13.806 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:13.807 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":23,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:38:54.069940+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:38:51.059079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T09:38:51.624756+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:13.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f737406a050 msgr2=0x7f737406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f737406a050 0x7f737406c510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f737c000f60 tx=0x7f737c0078f0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 msgr2=0x7f73901501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 0x7f73901501f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7384034040 tx=0x7f73840046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 shutdown_connections 2023-08-25T09:40:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f737406a050 0x7f737406c510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f739015cb40 0x7f7390153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73900b2b40 0x7f739015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 --2- 172.21.15.70:0/1721236907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73900ac870 0x7f73901501f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:13.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 >> 172.21.15.70:0/1721236907 conn(0x7f7390069850 msgr2=0x7f7390160f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:13.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 shutdown_connections 2023-08-25T09:40:13.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:13.807+0000 7f7396c46640 1 -- 172.21.15.70:0/1721236907 wait complete. 2023-08-25T09:40:13.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:40:14.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.046+0000 7fadcc72f640 1 Processor -- start 2023-08-25T09:40:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.046+0000 7fadcc72f640 1 -- start start 2023-08-25T09:40:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadcc72f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadc4159aa0 0x7fadc4070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadcc72f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fadc415df10 0x7fadc4070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadcc72f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc406aec0 con 0x7fadc406bc80 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadcc72f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc4159e80 con 0x7fadc4159aa0 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadcc72f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc406b1a0 con 0x7fadc415df10 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadca4cb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadc4159aa0 0x7fadc4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadca4cb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fadc4159aa0 0x7fadc4070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42244/0 (socket says 172.21.15.70:42244) 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.048+0000 7fadca4cb640 1 -- 172.21.15.70:0/3929465737 learned_addr learned my addr 172.21.15.70:0/3929465737 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.049+0000 7fadca4cb640 1 -- 172.21.15.70:0/3929465737 >> v2:172.21.15.155:3300/0 conn(0x7fadc415df10 msgr2=0x7fadc4070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.049+0000 7fadca4cb640 1 --2- 172.21.15.70:0/3929465737 >> v2:172.21.15.155:3300/0 conn(0x7fadc415df10 0x7fadc4070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.049+0000 7fadca4cb640 1 -- 172.21.15.70:0/3929465737 >> v1:172.21.15.70:6789/0 conn(0x7fadc406bc80 legacy=0x7fadc40c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.049+0000 7fadca4cb640 1 -- 172.21.15.70:0/3929465737 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc406b480 con 0x7fadc4159aa0 2023-08-25T09:40:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.050+0000 7fadca4cb640 1 --2- 172.21.15.70:0/3929465737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc4070410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b4910 tx=0x7fadb402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592ec79fb7adf654 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.050+0000 7fadc94c9640 1 -- 172.21.15.70:0/3929465737 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402cb70 con 0x7fadc4159aa0 2023-08-25T09:40:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.050+0000 7fadc94c9640 1 -- 172.21.15.70:0/3929465737 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb402ccd0 con 0x7fadc4159aa0 2023-08-25T09:40:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.050+0000 7fadc94c9640 1 -- 172.21.15.70:0/3929465737 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4034610 con 0x7fadc4159aa0 2023-08-25T09:40:14.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 -- 172.21.15.70:0/3929465737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 msgr2=0x7fadc4070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 --2- 172.21.15.70:0/3929465737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc4070410 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fadc40b4910 tx=0x7fadb402bfd0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 -- 172.21.15.70:0/3929465737 shutdown_connections 2023-08-25T09:40:14.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 --2- 172.21.15.70:0/3929465737 >> v2:172.21.15.155:3300/0 conn(0x7fadc415df10 0x7fadc4070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 --2- 172.21.15.70:0/3929465737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc4070410 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 -- 172.21.15.70:0/3929465737 >> 172.21.15.70:0/3929465737 conn(0x7fadc40ce760 msgr2=0x7fadc40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:14.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 -- 172.21.15.70:0/3929465737 shutdown_connections 2023-08-25T09:40:14.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.051+0000 7fadcc72f640 1 -- 172.21.15.70:0/3929465737 wait complete. 2023-08-25T09:40:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.053+0000 7fadcc72f640 1 Processor -- start 2023-08-25T09:40:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.053+0000 7fadcc72f640 1 -- start start 2023-08-25T09:40:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadcc72f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadcc72f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadc40cb1e0 0x7fadc41f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadcc72f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadc4142150 0x7fadc4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadcc72f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41f70a0 con 0x7fadc4159aa0 2023-08-25T09:40:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadcc72f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40cb750 con 0x7fadc4142150 2023-08-25T09:40:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadcc72f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc40ccee0 con 0x7fadc40cb1e0 2023-08-25T09:40:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.054+0000 7fadca4cb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc40cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42252/0 (socket says 172.21.15.70:42252) 2023-08-25T09:40:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 -- 172.21.15.70:0/2133337605 learned_addr learned my addr 172.21.15.70:0/2133337605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:14.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 -- 172.21.15.70:0/2133337605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadc40cb1e0 msgr2=0x7fadc41f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadc40cb1e0 0x7fadc41f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 -- 172.21.15.70:0/2133337605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadc4142150 msgr2=0x7fadc4156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadc4142150 0x7fadc4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 -- 172.21.15.70:0/2133337605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4009b30 con 0x7fadc4159aa0 2023-08-25T09:40:14.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadca4cb640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc40cdca0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000f80 tx=0x7fadb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:14.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.055+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb403e450 con 0x7fadc4159aa0 2023-08-25T09:40:14.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.056+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb403e5f0 con 0x7fadc4159aa0 2023-08-25T09:40:14.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.056+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb403e9a0 con 0x7fadc4159aa0 2023-08-25T09:40:14.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.056+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc40cd1c0 con 0x7fadc4159aa0 2023-08-25T09:40:14.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.056+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc40cd780 con 0x7fadc4159aa0 2023-08-25T09:40:14.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.057+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc404e420 con 0x7fadc4159aa0 2023-08-25T09:40:14.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.058+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fadb403eb00 con 0x7fadc4159aa0 2023-08-25T09:40:14.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.059+0000 7fadc2ffd640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadb006a050 0x7fadb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.059+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadb402ce40 con 0x7fadc4159aa0 2023-08-25T09:40:14.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.059+0000 7fadc9cca640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadb006a050 0x7fadb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.060+0000 7fadc9cca640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadb006a050 0x7fadb006c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fadc41f60b0 tx=0x7fadb8007950 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:14.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.063+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fadb4076d50 con 0x7fadc4159aa0 2023-08-25T09:40:14.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.289+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fadc415ce80 con 0x7fadc4159aa0 2023-08-25T09:40:14.292 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:40:14.291+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:40:14.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.298+0000 7fadc2ffd640 1 -- 172.21.15.70:0/2133337605 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fadb403edb0 con 0x7fadc4159aa0 2023-08-25T09:40:14.301 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:40:14.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadb006a050 msgr2=0x7fadb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadb006a050 0x7fadb006c510 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fadc41f60b0 tx=0x7fadb8007950 comp rx=0 tx=0).stop 2023-08-25T09:40:14.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 msgr2=0x7fadc40cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc40cdca0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fadb4000f80 tx=0x7fadb4004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:14.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 shutdown_connections 2023-08-25T09:40:14.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadb006a050 0x7fadb006c510 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadc4142150 0x7fadc4156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fadc40cb1e0 0x7fadc41f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 --2- 172.21.15.70:0/2133337605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadc4159aa0 0x7fadc40cdca0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 >> 172.21.15.70:0/2133337605 conn(0x7fadc40ce760 msgr2=0x7fadc40ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:14.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 shutdown_connections 2023-08-25T09:40:14.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.301+0000 7fadcc72f640 1 -- 172.21.15.70:0/2133337605 wait complete. 2023-08-25T09:40:14.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:40:14.328 INFO:tasks.ceph.mds.d.smithi155.stderr: -4340> 2023-08-25T09:38:58.247+0000 7f9fa0eb8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:40:14.423 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:40:14.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.537+0000 7ff3294f3640 1 Processor -- start 2023-08-25T09:40:14.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.538+0000 7ff3294f3640 1 -- start start 2023-08-25T09:40:14.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.539+0000 7ff3294f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff32415ae90 0x7ff3240ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.539+0000 7ff3294f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32415be60 con 0x7ff3240ce410 2023-08-25T09:40:14.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.539+0000 7ff3294f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32406ee70 con 0x7ff32406f300 2023-08-25T09:40:14.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.539+0000 7ff3294f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3240ced30 con 0x7ff32415ae90 2023-08-25T09:40:14.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff322ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff3240ce410 0x7ff324070770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39940/0 (socket says 172.21.15.70:39940) 2023-08-25T09:40:14.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff322ffd640 1 -- 172.21.15.70:0/4285583151 learned_addr learned my addr 172.21.15.70:0/4285583151 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:14.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501026909 0 0) 0x7ff32415be60 con 0x7ff3240ce410 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff310003520 con 0x7ff3240ce410 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff31a7fc640 1 --2- 172.21.15.70:0/4285583151 >> v2:172.21.15.155:3300/0 conn(0x7ff32415ae90 0x7ff3240ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4252503554 0 0) 0x7ff310003520 con 0x7ff3240ce410 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 >> v2:172.21.15.155:3300/0 conn(0x7ff32415ae90 msgr2=0x7ff3240ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 --2- 172.21.15.70:0/4285583151 >> v2:172.21.15.155:3300/0 conn(0x7ff32415ae90 0x7ff3240ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 >> v1:172.21.15.155:6789/0 conn(0x7ff32406f300 legacy=0x7ff32406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:14.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32415b2f0 con 0x7ff3240ce410 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff314001670 con 0x7ff3240ce410 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 >> v1:172.21.15.70:6789/0 conn(0x7ff3240ce410 legacy=0x7ff324070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff310003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.540+0000 7ff3227fc640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3100047c0 0x7ff310004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff3227fc640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff310005180 0x7ff31000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240ced30 con 0x7ff310003970 2023-08-25T09:40:14.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31000c0b0 con 0x7ff3100047c0 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31000c270 con 0x7ff310005180 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff322ffd640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff310003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff31a7fc640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3100047c0 0x7ff310004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff3237fe640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff310005180 0x7ff31000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff322ffd640 1 -- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff310005180 msgr2=0x7ff31000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff322ffd640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff310005180 0x7ff31000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff322ffd640 1 -- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3100047c0 msgr2=0x7ff310004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff322ffd640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3100047c0 0x7ff310004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff322ffd640 1 -- 172.21.15.70:0/4285583151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff310003520 con 0x7ff310003970 2023-08-25T09:40:14.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.541+0000 7ff3237fe640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff310005180 0x7ff31000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:14.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.542+0000 7ff3237fe640 1 -- 172.21.15.70:0/4285583151 reap_dead start 2023-08-25T09:40:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.543+0000 7ff322ffd640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff310003d70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff314001eb0 tx=0x7ff3100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126caec9f0dc38f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.543+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314005ad0 con 0x7ff310003970 2023-08-25T09:40:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.543+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff314005c30 con 0x7ff310003970 2023-08-25T09:40:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.543+0000 7ff3227fc640 1 -- 172.21.15.70:0/4285583151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31402c890 con 0x7ff310003970 2023-08-25T09:40:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.543+0000 7ff3294f3640 1 -- 172.21.15.70:0/4285583151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 msgr2=0x7ff310003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.543+0000 7ff3294f3640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff310003d70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff314001eb0 tx=0x7ff3100057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.544+0000 7ff3294f3640 1 -- 172.21.15.70:0/4285583151 shutdown_connections 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.544+0000 7ff3294f3640 1 --2- 172.21.15.70:0/4285583151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff310003d70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.544+0000 7ff3294f3640 1 -- 172.21.15.70:0/4285583151 >> 172.21.15.70:0/4285583151 conn(0x7ff324069850 msgr2=0x7ff324069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.544+0000 7ff3294f3640 1 -- 172.21.15.70:0/4285583151 shutdown_connections 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.544+0000 7ff3294f3640 1 -- 172.21.15.70:0/4285583151 wait complete. 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.545+0000 7ff3294f3640 1 Processor -- start 2023-08-25T09:40:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.546+0000 7ff3294f3640 1 -- start start 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.546+0000 7ff3294f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff3240c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.546+0000 7ff3294f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32406c600 0x7ff324149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.546+0000 7ff3294f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff32406ca80 0x7ff32414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff3294f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3241f5af0 con 0x7ff310003970 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff3294f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3241f5dd0 con 0x7ff32406c600 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff3294f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3241f60b0 con 0x7ff32406ca80 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff3240c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff3240c35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42272/0 (socket says 172.21.15.70:42272) 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 -- 172.21.15.70:0/2777345311 learned_addr learned my addr 172.21.15.70:0/2777345311 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 -- 172.21.15.70:0/2777345311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff32406ca80 msgr2=0x7ff32414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff32406ca80 0x7ff32414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 -- 172.21.15.70:0/2777345311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32406c600 msgr2=0x7ff324149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32406c600 0x7ff324149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 -- 172.21.15.70:0/2777345311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3241f6390 con 0x7ff310003970 2023-08-25T09:40:14.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.547+0000 7ff322ffd640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff3240c35c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff314001eb0 tx=0x7ff31402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:14.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.548+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31403c040 con 0x7ff310003970 2023-08-25T09:40:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.548+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31402cca0 con 0x7ff310003970 2023-08-25T09:40:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.548+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314007270 con 0x7ff310003970 2023-08-25T09:40:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.548+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3241f6670 con 0x7ff310003970 2023-08-25T09:40:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.548+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240c3b00 con 0x7ff310003970 2023-08-25T09:40:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.549+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff32404e420 con 0x7ff310003970 2023-08-25T09:40:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.550+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff314032070 con 0x7ff310003970 2023-08-25T09:40:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.551+0000 7ff31bfff640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff30406a080 0x7ff30406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.551+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6148+0+0 (secure 0 0 0) 0x7ff3140a93a0 con 0x7ff310003970 2023-08-25T09:40:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.551+0000 7ff31a7fc640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff30406a080 0x7ff30406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.552+0000 7ff31a7fc640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff30406a080 0x7ff30406c540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff3240cab30 tx=0x7ff30c003a10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:14.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.555+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff314074b70 con 0x7ff310003970 2023-08-25T09:40:14.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:14.783+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff32406b1d0 con 0x7ff310003970 2023-08-25T09:40:15.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.347+0000 7ff31bfff640 1 -- 172.21.15.70:0/2777345311 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff314078190 con 0x7ff310003970 2023-08-25T09:40:15.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff30406a080 msgr2=0x7ff30406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:15.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff30406a080 0x7ff30406c540 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff3240cab30 tx=0x7ff30c003a10 comp rx=0 tx=0).stop 2023-08-25T09:40:15.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 msgr2=0x7ff3240c35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:15.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff3240c35c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff314001eb0 tx=0x7ff31402e070 comp rx=0 tx=0).stop 2023-08-25T09:40:15.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 shutdown_connections 2023-08-25T09:40:15.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff30406a080 0x7ff30406c540 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff32406ca80 0x7ff32414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff32406c600 0x7ff324149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 --2- 172.21.15.70:0/2777345311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff310003970 0x7ff3240c35c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.351+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 >> 172.21.15.70:0/2777345311 conn(0x7ff324069850 msgr2=0x7ff324158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.352+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 shutdown_connections 2023-08-25T09:40:15.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.352+0000 7ff3294f3640 1 -- 172.21.15.70:0/2777345311 wait complete. 2023-08-25T09:40:15.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:40:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.598+0000 7f874ca83640 1 Processor -- start 2023-08-25T09:40:15.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.599+0000 7f874ca83640 1 -- start start 2023-08-25T09:40:15.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.600+0000 7f874ca83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8748155120 0x7f87480cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:15.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.600+0000 7f874ca83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87480ca9f0 0x7f874815af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:15.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.600+0000 7f874ca83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874815e6f0 con 0x7f87480cd890 2023-08-25T09:40:15.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.600+0000 7f874ca83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87480cdd40 con 0x7f87480ca9f0 2023-08-25T09:40:15.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.600+0000 7f874ca83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87480caf30 con 0x7f8748155120 2023-08-25T09:40:15.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87480ca9f0 0x7f874815af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:15.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f87480ca9f0 0x7f874815af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42280/0 (socket says 172.21.15.70:42280) 2023-08-25T09:40:15.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 -- 172.21.15.70:0/2885070031 learned_addr learned my addr 172.21.15.70:0/2885070031 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:15.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 -- 172.21.15.70:0/2885070031 >> v2:172.21.15.155:3300/0 conn(0x7f8748155120 msgr2=0x7f87480cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:15.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f874659c640 1 --2- 172.21.15.70:0/2885070031 >> v2:172.21.15.155:3300/0 conn(0x7f8748155120 0x7f87480cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:15.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 --2- 172.21.15.70:0/2885070031 >> v2:172.21.15.155:3300/0 conn(0x7f8748155120 0x7f87480cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 -- 172.21.15.70:0/2885070031 >> v1:172.21.15.155:6789/0 conn(0x7f87480cd890 legacy=0x7f87480ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:15.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.601+0000 7f8746d9d640 1 -- 172.21.15.70:0/2885070031 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874815e9d0 con 0x7f87480ca9f0 2023-08-25T09:40:15.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.602+0000 7f8746d9d640 1 --2- 172.21.15.70:0/2885070031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87480ca9f0 0x7f874815af60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f87480ce610 tx=0x7f874002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dd9ecbbc535482 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:15.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.602+0000 7f8745d9b640 1 -- 172.21.15.70:0/2885070031 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874002ca80 con 0x7f87480ca9f0 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.602+0000 7f8745d9b640 1 -- 172.21.15.70:0/2885070031 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874002cbe0 con 0x7f87480ca9f0 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.602+0000 7f8745d9b640 1 -- 172.21.15.70:0/2885070031 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87400343f0 con 0x7f87480ca9f0 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 -- 172.21.15.70:0/2885070031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87480ca9f0 msgr2=0x7f874815af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 --2- 172.21.15.70:0/2885070031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87480ca9f0 0x7f874815af60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f87480ce610 tx=0x7f874002bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 -- 172.21.15.70:0/2885070031 shutdown_connections 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 --2- 172.21.15.70:0/2885070031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f87480ca9f0 0x7f874815af60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 --2- 172.21.15.70:0/2885070031 >> v2:172.21.15.155:3300/0 conn(0x7f8748155120 0x7f87480cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 -- 172.21.15.70:0/2885070031 >> 172.21.15.70:0/2885070031 conn(0x7f8748069c70 msgr2=0x7f874806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:15.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 -- 172.21.15.70:0/2885070031 shutdown_connections 2023-08-25T09:40:15.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.603+0000 7f874ca83640 1 -- 172.21.15.70:0/2885070031 wait complete. 2023-08-25T09:40:15.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.606+0000 7f874ca83640 1 Processor -- start 2023-08-25T09:40:15.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.606+0000 7f874ca83640 1 -- start start 2023-08-25T09:40:15.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f874ca83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87480ca9f0 0x7f874815d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:15.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f874ca83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87480cd890 0x7f8748149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:15.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f874ca83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 0x7f87481f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f874ca83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87481496a0 con 0x7f8748155120 2023-08-25T09:40:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f874ca83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87481f6420 con 0x7f87480cd890 2023-08-25T09:40:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f874ca83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87481f6700 con 0x7f87480ca9f0 2023-08-25T09:40:15.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f8746d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 0x7f87481f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:15.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.607+0000 7f8746d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 0x7f87481f5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42284/0 (socket says 172.21.15.70:42284) 2023-08-25T09:40:15.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 -- 172.21.15.70:0/1708017381 learned_addr learned my addr 172.21.15.70:0/1708017381 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f874659c640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87480ca9f0 0x7f874815d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:15.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f873ffff640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87480cd890 0x7f8748149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 -- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87480ca9f0 msgr2=0x7f874815d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87480ca9f0 0x7f874815d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 -- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87480cd890 msgr2=0x7f8748149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87480cd890 0x7f8748149160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 -- 172.21.15.70:0/1708017381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740009b30 con 0x7f8748155120 2023-08-25T09:40:15.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.608+0000 7f8746d9d640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 0x7f87481f5bd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8740036040 tx=0x7f8740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.609+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874003e450 con 0x7f8748155120 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.609+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874003e5d0 con 0x7f8748155120 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.609+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874003e980 con 0x7f8748155120 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.609+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87481f69e0 con 0x7f8748155120 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.609+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87481f25e0 con 0x7f8748155120 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.610+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87481f2b30 con 0x7f8748155120 2023-08-25T09:40:15.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.611+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f874002cd50 con 0x7f8748155120 2023-08-25T09:40:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.612+0000 7f873effd640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f872806a080 0x7f872806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.612+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f87400aaef0 con 0x7f8748155120 2023-08-25T09:40:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.612+0000 7f874659c640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f872806a080 0x7f872806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.613+0000 7f874659c640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f872806a080 0x7f872806c540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8738006fd0 tx=0x7f8738008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.616+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87481f2b30 con 0x7f8748155120 2023-08-25T09:40:15.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:15.812+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87481f2b30 con 0x7f8748155120 2023-08-25T09:40:16.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.359+0000 7f873effd640 1 -- 172.21.15.70:0/1708017381 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f87481f2b30 con 0x7f8748155120 2023-08-25T09:40:16.361 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:40:16.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f872806a080 msgr2=0x7f872806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f872806a080 0x7f872806c540 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8738006fd0 tx=0x7f8738008040 comp rx=0 tx=0).stop 2023-08-25T09:40:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 msgr2=0x7f87481f5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 0x7f87481f5bd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8740036040 tx=0x7f8740004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 shutdown_connections 2023-08-25T09:40:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f872806a080 0x7f872806c540 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8748155120 0x7f87481f5bd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87480cd890 0x7f8748149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 --2- 172.21.15.70:0/1708017381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87480ca9f0 0x7f874815d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 >> 172.21.15.70:0/1708017381 conn(0x7f8748069c70 msgr2=0x7f87480b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:16.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 shutdown_connections 2023-08-25T09:40:16.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.363+0000 7f874ca83640 1 -- 172.21.15.70:0/1708017381 wait complete. 2023-08-25T09:40:16.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:40:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.596+0000 7f09e37f3640 1 Processor -- start 2023-08-25T09:40:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.597+0000 7f09e37f3640 1 -- start start 2023-08-25T09:40:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.598+0000 7f09e37f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09dc15cd30 0x7f09dc06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.598+0000 7f09e37f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09dc0b0980 0x7f09dc06afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.598+0000 7f09e37f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09dc0b2300 con 0x7f09dc15c950 2023-08-25T09:40:16.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.598+0000 7f09e37f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc0b16f0 con 0x7f09dc0b0980 2023-08-25T09:40:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.598+0000 7f09e37f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09dc0b19d0 con 0x7f09dc15cd30 2023-08-25T09:40:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09dc0b0980 0x7f09dc06afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09dc0b0980 0x7f09dc06afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42288/0 (socket says 172.21.15.70:42288) 2023-08-25T09:40:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 -- 172.21.15.70:0/173857692 learned_addr learned my addr 172.21.15.70:0/173857692 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 -- 172.21.15.70:0/173857692 >> v2:172.21.15.155:3300/0 conn(0x7f09dc15cd30 msgr2=0x7f09dc06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e0d8e640 1 --2- 172.21.15.70:0/173857692 >> v2:172.21.15.155:3300/0 conn(0x7f09dc15cd30 0x7f09dc06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 --2- 172.21.15.70:0/173857692 >> v2:172.21.15.155:3300/0 conn(0x7f09dc15cd30 0x7f09dc06aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 -- 172.21.15.70:0/173857692 >> v1:172.21.15.155:6789/0 conn(0x7f09dc15c950 legacy=0x7f09dc0b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e1d90640 1 -- 172.21.15.70:0/173857692 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09dc0b1cb0 con 0x7f09dc0b0980 2023-08-25T09:40:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.599+0000 7f09e0d8e640 1 --2- 172.21.15.70:0/173857692 >> v2:172.21.15.155:3300/0 conn(0x7f09dc15cd30 0x7f09dc06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.600+0000 7f09e1d90640 1 --2- 172.21.15.70:0/173857692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc0b0980 0x7f09dc06afb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09dc06e860 tx=0x7f09d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b941bd48b0dfdd52 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09d3fff640 1 -- 172.21.15.70:0/173857692 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d402ca80 con 0x7f09dc0b0980 2023-08-25T09:40:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09d3fff640 1 -- 172.21.15.70:0/173857692 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d402cbe0 con 0x7f09dc0b0980 2023-08-25T09:40:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09d3fff640 1 -- 172.21.15.70:0/173857692 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d40343f0 con 0x7f09dc0b0980 2023-08-25T09:40:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09e37f3640 1 -- 172.21.15.70:0/173857692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc0b0980 msgr2=0x7f09dc06afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09e37f3640 1 --2- 172.21.15.70:0/173857692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc0b0980 0x7f09dc06afb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09dc06e860 tx=0x7f09d402bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09e37f3640 1 -- 172.21.15.70:0/173857692 shutdown_connections 2023-08-25T09:40:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09e37f3640 1 --2- 172.21.15.70:0/173857692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc0b0980 0x7f09dc06afb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09e37f3640 1 --2- 172.21.15.70:0/173857692 >> v2:172.21.15.155:3300/0 conn(0x7f09dc15cd30 0x7f09dc06aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.601+0000 7f09e37f3640 1 -- 172.21.15.70:0/173857692 >> 172.21.15.70:0/173857692 conn(0x7f09dc069c70 msgr2=0x7f09dc06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.602+0000 7f09e37f3640 1 -- 172.21.15.70:0/173857692 shutdown_connections 2023-08-25T09:40:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.602+0000 7f09e37f3640 1 -- 172.21.15.70:0/173857692 wait complete. 2023-08-25T09:40:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.603+0000 7f09e37f3640 1 Processor -- start 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.603+0000 7f09e37f3640 1 -- start start 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.604+0000 7f09e37f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09dc0b0980 0x7f09dc14bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.604+0000 7f09e37f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09dc15c950 0x7f09dc15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.604+0000 7f09e37f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 0x7f09dc15b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.604+0000 7f09e37f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc1597d0 con 0x7f09dc15cd30 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.604+0000 7f09e37f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc159ab0 con 0x7f09dc15c950 2023-08-25T09:40:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.604+0000 7f09e37f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc159d90 con 0x7f09dc0b0980 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 0x7f09dc15b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 0x7f09dc15b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42296/0 (socket says 172.21.15.70:42296) 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 -- 172.21.15.70:0/4170463542 learned_addr learned my addr 172.21.15.70:0/4170463542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e158f640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09dc0b0980 0x7f09dc14bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e0d8e640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09dc15c950 0x7f09dc15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 -- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09dc0b0980 msgr2=0x7f09dc14bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09dc0b0980 0x7f09dc14bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 -- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09dc15c950 msgr2=0x7f09dc15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09dc15c950 0x7f09dc15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e1d90640 1 -- 172.21.15.70:0/4170463542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d4009b30 con 0x7f09dc15cd30 2023-08-25T09:40:16.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.605+0000 7f09e0d8e640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09dc15c950 0x7f09dc15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:16.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.606+0000 7f09e1d90640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 0x7f09dc15b6a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09d4036040 tx=0x7f09d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:16.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.606+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d403e450 con 0x7f09dc15cd30 2023-08-25T09:40:16.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.606+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d403e5d0 con 0x7f09dc15cd30 2023-08-25T09:40:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.606+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d403e980 con 0x7f09dc15cd30 2023-08-25T09:40:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.606+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc15a070 con 0x7f09dc15cd30 2023-08-25T09:40:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.606+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc14c510 con 0x7f09dc15cd30 2023-08-25T09:40:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.607+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc15a530 con 0x7f09dc15cd30 2023-08-25T09:40:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.608+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f09d402cd50 con 0x7f09dc15cd30 2023-08-25T09:40:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.609+0000 7f09d1ffb640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f09c006a050 0x7f09c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.609+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f09d40ab070 con 0x7f09dc15cd30 2023-08-25T09:40:16.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.609+0000 7f09e158f640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f09c006a050 0x7f09c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.610+0000 7f09e158f640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f09c006a050 0x7f09c006c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0b0740 tx=0x7f09d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:16.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.613+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09d40dd990 con 0x7f09dc15cd30 2023-08-25T09:40:16.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:16.806+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f09dc04e420 con 0x7f09dc15cd30 2023-08-25T09:40:17.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.368+0000 7f09d1ffb640 1 -- 172.21.15.70:0/4170463542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f09d4043020 con 0x7f09dc15cd30 2023-08-25T09:40:17.370 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:40:17.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f09c006a050 msgr2=0x7f09c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f09c006a050 0x7f09c006c510 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f09dc0b0740 tx=0x7f09d8009040 comp rx=0 tx=0).stop 2023-08-25T09:40:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 msgr2=0x7f09dc15b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 0x7f09dc15b6a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f09d4036040 tx=0x7f09d4004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:17.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 shutdown_connections 2023-08-25T09:40:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f09c006a050 0x7f09c006c510 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09dc15cd30 0x7f09dc15b6a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09dc15c950 0x7f09dc15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 --2- 172.21.15.70:0/4170463542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09dc0b0980 0x7f09dc14bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 >> 172.21.15.70:0/4170463542 conn(0x7f09dc069c70 msgr2=0x7f09dc0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.372+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 shutdown_connections 2023-08-25T09:40:17.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.373+0000 7f09e37f3640 1 -- 172.21.15.70:0/4170463542 wait complete. 2023-08-25T09:40:17.395 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:40:17.396 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:40:17.396 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:40:17.396 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:40:17.397 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:40:17.397 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:40:17.397 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:40:17.397 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:40:17.398 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:40:17.398 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:40:17.398 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:40:17.399 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:40:17.399 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:40:17.400 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:40:17.401 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:40:17.401 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:40:17.415 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:40:17.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:40:17.418 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:40:17.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:40:17.422 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:40:17.422 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:40:17.423 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:40:17.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:40:17.466 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:40:17.467 DEBUG:teuthology.parallel:result is None 2023-08-25T09:40:17.471 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:40:17.472 DEBUG:teuthology.parallel:result is None 2023-08-25T09:40:17.475 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:40:17.475 DEBUG:teuthology.parallel:result is None 2023-08-25T09:40:17.478 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:40:17.478 DEBUG:teuthology.parallel:result is None 2023-08-25T09:40:17.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:40:17.632 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:40:17.633 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:40:17.636 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:40:17.637 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:40:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.673+0000 7f1113221640 1 Processor -- start 2023-08-25T09:40:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.673+0000 7f1113221640 1 -- start start 2023-08-25T09:40:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1113221640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f110c15cf00 0x7f110c070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1113221640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c0b81d0 con 0x7f110c0cf210 2023-08-25T09:40:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1113221640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c15d3c0 con 0x7f110c0b7df0 2023-08-25T09:40:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1113221640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c1604f0 con 0x7f110c15cf00 2023-08-25T09:40:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1112a20640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f110c0cf210 0x7f110c0cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39976/0 (socket says 172.21.15.70:39976) 2023-08-25T09:40:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1112a20640 1 -- 172.21.15.70:0/3242838954 learned_addr learned my addr 172.21.15.70:0/3242838954 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f1111a1e640 1 --2- 172.21.15.70:0/3242838954 >> v2:172.21.15.155:3300/0 conn(0x7f110c15cf00 0x7f110c070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f111121d640 1 -- 172.21.15.70:0/3242838954 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091764795 0 0) 0x7f110c0b81d0 con 0x7f110c0cf210 2023-08-25T09:40:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.675+0000 7f111121d640 1 -- 172.21.15.70:0/3242838954 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f8003520 con 0x7f110c0cf210 2023-08-25T09:40:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.676+0000 7f1111a1e640 1 -- 172.21.15.70:0/3242838954 >> v1:172.21.15.155:6789/0 conn(0x7f110c0b7df0 legacy=0x7f110c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.676+0000 7f1111a1e640 1 -- 172.21.15.70:0/3242838954 >> v1:172.21.15.70:6789/0 conn(0x7f110c0cf210 legacy=0x7f110c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.676+0000 7f1111a1e640 1 -- 172.21.15.70:0/3242838954 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c15d6a0 con 0x7f110c15cf00 2023-08-25T09:40:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.677+0000 7f1111a1e640 1 --2- 172.21.15.70:0/3242838954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c15cf00 0x7f110c070880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097a0 tx=0x7f10fc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ced36b9ff94c827 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.677+0000 7f111121d640 1 -- 172.21.15.70:0/3242838954 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc030610 con 0x7f110c15cf00 2023-08-25T09:40:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.677+0000 7f111121d640 1 -- 172.21.15.70:0/3242838954 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc030770 con 0x7f110c15cf00 2023-08-25T09:40:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.677+0000 7f111121d640 1 -- 172.21.15.70:0/3242838954 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc030b00 con 0x7f110c15cf00 2023-08-25T09:40:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 -- 172.21.15.70:0/3242838954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c15cf00 msgr2=0x7f110c070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 --2- 172.21.15.70:0/3242838954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c15cf00 0x7f110c070880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0097a0 tx=0x7f10fc02b960 comp rx=0 tx=0).stop 2023-08-25T09:40:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 -- 172.21.15.70:0/3242838954 shutdown_connections 2023-08-25T09:40:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 --2- 172.21.15.70:0/3242838954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c15cf00 0x7f110c070880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 -- 172.21.15.70:0/3242838954 >> 172.21.15.70:0/3242838954 conn(0x7f110c0b43a0 msgr2=0x7f110c0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:17.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 -- 172.21.15.70:0/3242838954 shutdown_connections 2023-08-25T09:40:17.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.678+0000 7f1113221640 1 -- 172.21.15.70:0/3242838954 wait complete. 2023-08-25T09:40:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.680+0000 7f1113221640 1 Processor -- start 2023-08-25T09:40:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.680+0000 7f1113221640 1 -- start start 2023-08-25T09:40:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1113221640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c0b7df0 0x7f110c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:17.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1113221640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f110c0cf210 0x7f110c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:17.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1113221640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 0x7f110c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1113221640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0b11c0 con 0x7f110c15cf00 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1113221640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0ca8b0 con 0x7f110c0b7df0 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1113221640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0cab50 con 0x7f110c0cf210 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 0x7f110c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 0x7f110c0ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42334/0 (socket says 172.21.15.70:42334) 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 -- 172.21.15.70:0/3524779261 learned_addr learned my addr 172.21.15.70:0/3524779261 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f111221f640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c0b7df0 0x7f110c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:17.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1111a1e640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f110c0cf210 0x7f110c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 -- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f110c0cf210 msgr2=0x7f110c0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f110c0cf210 0x7f110c0b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 -- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c0b7df0 msgr2=0x7f110c14db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:17.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c0b7df0 0x7f110c14db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1112a20640 1 -- 172.21.15.70:0/3524779261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10fc009400 con 0x7f110c15cf00 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f111221f640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c0b7df0 0x7f110c14db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.681+0000 7f1111a1e640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f110c0cf210 0x7f110c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.682+0000 7f1112a20640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 0x7f110c0ca370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f110400cb40 tx=0x7f110400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.682+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110400aca0 con 0x7f110c15cf00 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.682+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110400ae00 con 0x7f110c15cf00 2023-08-25T09:40:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.682+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c0cae30 con 0x7f110c15cf00 2023-08-25T09:40:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.682+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104012620 con 0x7f110c15cf00 2023-08-25T09:40:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.682+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c0cb3f0 con 0x7f110c15cf00 2023-08-25T09:40:17.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.683+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c071220 con 0x7f110c15cf00 2023-08-25T09:40:17.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.684+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1104007070 con 0x7f110c15cf00 2023-08-25T09:40:17.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.685+0000 7f1102ffd640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10f006a050 0x7f10f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:17.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.685+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1104088370 con 0x7f110c15cf00 2023-08-25T09:40:17.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.685+0000 7f111221f640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10f006a050 0x7f10f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:17.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.686+0000 7f111221f640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10f006a050 0x7f10f006c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1108007fd0 tx=0x7f1108009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:17.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.690+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1104053ce0 con 0x7f110c15cf00 2023-08-25T09:40:17.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:17.902+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f110c15b250 con 0x7f110c15cf00 2023-08-25T09:40:19.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.247+0000 7f1102ffd640 1 -- 172.21.15.70:0/3524779261 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1104057300 con 0x7f110c15cf00 2023-08-25T09:40:19.249 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:40:19.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.250+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10f006a050 msgr2=0x7f10f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.250+0000 7f1113221640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10f006a050 0x7f10f006c510 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1108007fd0 tx=0x7f1108009040 comp rx=0 tx=0).stop 2023-08-25T09:40:19.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.250+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 msgr2=0x7f110c0ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.250+0000 7f1113221640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 0x7f110c0ca370 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f110400cb40 tx=0x7f110400a060 comp rx=0 tx=0).stop 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 shutdown_connections 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10f006a050 0x7f10f006c510 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f110c15cf00 0x7f110c0ca370 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f110c0cf210 0x7f110c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 --2- 172.21.15.70:0/3524779261 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f110c0b7df0 0x7f110c14db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 >> 172.21.15.70:0/3524779261 conn(0x7f110c0b43a0 msgr2=0x7f110c15e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:19.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 shutdown_connections 2023-08-25T09:40:19.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.251+0000 7f1113221640 1 -- 172.21.15.70:0/3524779261 wait complete. 2023-08-25T09:40:19.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:40:19.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.487+0000 7fc21e07f640 1 Processor -- start 2023-08-25T09:40:19.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.487+0000 7fc21e07f640 1 -- start start 2023-08-25T09:40:19.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.488+0000 7fc21e07f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2180c92c0 0x7fc2180b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:19.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.488+0000 7fc21e07f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:19.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.488+0000 7fc21e07f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2180cd330 con 0x7fc2180b4150 2023-08-25T09:40:19.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.488+0000 7fc21e07f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2181615b0 con 0x7fc2180c92c0 2023-08-25T09:40:19.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.488+0000 7fc21e07f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b0e80 con 0x7fc2180c96a0 2023-08-25T09:40:19.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2180c92c0 0x7fc2180b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:19.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2180c92c0 0x7fc2180b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43724/0 (socket says 172.21.15.70:43724) 2023-08-25T09:40:19.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 -- 172.21.15.70:0/177829612 learned_addr learned my addr 172.21.15.70:0/177829612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:19.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 -- 172.21.15.70:0/177829612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 msgr2=0x7fc2180cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:19.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21c87c640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:19.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 -- 172.21.15.70:0/177829612 >> v1:172.21.15.70:6789/0 conn(0x7fc2180b4150 legacy=0x7fc2180cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21d07d640 1 -- 172.21.15.70:0/177829612 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180cc720 con 0x7fc2180c92c0 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.489+0000 7fc21c87c640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.490+0000 7fc21d07d640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180c92c0 0x7fc2180b12f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc214009870 tx=0x7fc21402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe134c9f52744b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc20ffff640 1 -- 172.21.15.70:0/177829612 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214030840 con 0x7fc2180c92c0 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc20ffff640 1 -- 172.21.15.70:0/177829612 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2140309a0 con 0x7fc2180c92c0 2023-08-25T09:40:19.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc20ffff640 1 -- 172.21.15.70:0/177829612 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214030d30 con 0x7fc2180c92c0 2023-08-25T09:40:19.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc21e07f640 1 -- 172.21.15.70:0/177829612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180c92c0 msgr2=0x7fc2180b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc21e07f640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180c92c0 0x7fc2180b12f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc214009870 tx=0x7fc21402b980 comp rx=0 tx=0).stop 2023-08-25T09:40:19.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc21e07f640 1 -- 172.21.15.70:0/177829612 shutdown_connections 2023-08-25T09:40:19.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc21e07f640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc21e07f640 1 --2- 172.21.15.70:0/177829612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180c92c0 0x7fc2180b12f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.491+0000 7fc21e07f640 1 -- 172.21.15.70:0/177829612 >> 172.21.15.70:0/177829612 conn(0x7fc2180affd0 msgr2=0x7fc2180b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:19.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.492+0000 7fc21e07f640 1 -- 172.21.15.70:0/177829612 shutdown_connections 2023-08-25T09:40:19.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.492+0000 7fc21e07f640 1 -- 172.21.15.70:0/177829612 wait complete. 2023-08-25T09:40:19.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.493+0000 7fc21e07f640 1 Processor -- start 2023-08-25T09:40:19.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.493+0000 7fc21e07f640 1 -- start start 2023-08-25T09:40:19.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.494+0000 7fc21e07f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 0x7fc2180c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:19.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.494+0000 7fc21e07f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2180c92c0 0x7fc21815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.494+0000 7fc21e07f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.494+0000 7fc21e07f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b32c0 con 0x7fc2180b4150 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.494+0000 7fc21e07f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180c6990 con 0x7fc2180c92c0 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.494+0000 7fc21e07f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180c5ab0 con 0x7fc2180c96a0 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 0x7fc2180c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 0x7fc2180c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43738/0 (socket says 172.21.15.70:43738) 2023-08-25T09:40:19.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 -- 172.21.15.70:0/1583652837 learned_addr learned my addr 172.21.15.70:0/1583652837 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:19.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 -- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 msgr2=0x7fc2180c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21c87c640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2180c92c0 0x7fc21815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:19.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 -- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2180c92c0 msgr2=0x7fc21815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2180c92c0 0x7fc21815e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 -- 172.21.15.70:0/1583652837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2140094d0 con 0x7fc2180b4150 2023-08-25T09:40:19.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.495+0000 7fc21d07d640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 0x7fc2180c6e20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc214000c00 tx=0x7fc214004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.496+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc214030810 con 0x7fc2180b4150 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.496+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc214043400 con 0x7fc2180b4150 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.496+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2140437d0 con 0x7fc2180b4150 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.496+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2180c5d80 con 0x7fc2180b4150 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.496+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2180c6340 con 0x7fc2180b4150 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.497+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc218066fb0 con 0x7fc2180b4150 2023-08-25T09:40:19.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.498+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc214043930 con 0x7fc2180b4150 2023-08-25T09:40:19.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.499+0000 7fc20dffb640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1fc06a050 0x7fc1fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:19.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.499+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc21403e070 con 0x7fc2180b4150 2023-08-25T09:40:19.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.499+0000 7fc21c87c640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1fc06a050 0x7fc1fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:19.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.500+0000 7fc21c87c640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1fc06a050 0x7fc1fc06c510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b3560 tx=0x7fc208008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:19.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.504+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc214076f50 con 0x7fc2180b4150 2023-08-25T09:40:19.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.760+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc2181eb8c0 con 0x7fc2180b4150 2023-08-25T09:40:19.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.761+0000 7fc20dffb640 1 -- 172.21.15.70:0/1583652837 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc214034090 con 0x7fc2180b4150 2023-08-25T09:40:19.764 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:19.764 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:19.764 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:40:19.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.765 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:40:19.765 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:40:19.766 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.766 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.766 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:40:19.767 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:40:19.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.767 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.768 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:40:19.768 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:40:19.768 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.769 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.769 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:40:19.769 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:40:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.771 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.771 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:40:19.771 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:40:19.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.771 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.772 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:40:19.772 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:40:19.773 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.773 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.773 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:40:19.773 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:40:19.773 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.774 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:40:19.774 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:40:19.774 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.775 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.777 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:40:19.777 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:40:19.777 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.777 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.777 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:40:19.778 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.779 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.779 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.779 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.780 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.783 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:40:19.783 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:40:19.784 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.784 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.784 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.785 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.785 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.786 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:40:19.786 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:40:19.786 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.786 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:40:19.787 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:40:19.787 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.788 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:40:19.788 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:40:19.788 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.789 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.789 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:40:19.789 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:40:19.789 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:40:19.790 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.790 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:40:19.791 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:40:19.791 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.791 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.791 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:40:19.792 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:40:19.792 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:40:19.792 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.793 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:40:19.793 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:40:19.793 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.793 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.794 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:40:19.794 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:40:19.794 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:40:19.794 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.795 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:40:19.795 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:40:19.795 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.796 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:40:19.796 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:40:19.796 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:40:19.796 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:40:19.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.797 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:40:19.797 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:40:19.798 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.798 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:40:19.798 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:40:19.798 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:40:19.798 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.799 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.799 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-mds", 2023-08-25T09:40:19.799 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCYduhk1LnmGBAA8vu2EWe3KZnK5YPjCJvy2g==", 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-mds" 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-mgr", 2023-08-25T09:40:19.800 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCYduhkSdHmGBAAHOV3ZpJDS+R49W9Xm8jhpQ==", 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-mgr" 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-osd", 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCYduhkTuTmGBAAZJ2FhZNbZ2ybM/alLDrgkQ==", 2023-08-25T09:40:19.801 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.802 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-osd" 2023-08-25T09:40:19.802 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.802 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-rbd", 2023-08-25T09:40:19.802 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCYduhkI/fmGBAAAlEHqnSGQ8etvldDT91b0g==", 2023-08-25T09:40:19.803 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.803 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-rbd" 2023-08-25T09:40:19.803 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-08-25T09:40:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCYduhkewrnGBAAiUhfU11L54YwlLA5Mg76KQ==", 2023-08-25T09:40:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-08-25T09:40:19.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.805 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.bootstrap-rgw", 2023-08-25T09:40:19.805 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCYduhkGx7nGBAAM1RM4KPX7PXZ8Uu3v9tQWQ==", 2023-08-25T09:40:19.805 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.806 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile bootstrap-rgw" 2023-08-25T09:40:19.806 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.807 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:40:19.807 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:40:19.807 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.807 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:40:19.807 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:40:19.808 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:40:19.809 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:19.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1fc06a050 msgr2=0x7fc1fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1fc06a050 0x7fc1fc06c510 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b3560 tx=0x7fc208008040 comp rx=0 tx=0).stop 2023-08-25T09:40:19.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 msgr2=0x7fc2180c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:19.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 0x7fc2180c6e20 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc214000c00 tx=0x7fc214004730 comp rx=0 tx=0).stop 2023-08-25T09:40:19.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 shutdown_connections 2023-08-25T09:40:19.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1fc06a050 0x7fc1fc06c510 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2180c96a0 0x7fc2180c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2180c92c0 0x7fc21815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 --2- 172.21.15.70:0/1583652837 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2180b4150 0x7fc2180c6e20 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:19.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 >> 172.21.15.70:0/1583652837 conn(0x7fc2180affd0 msgr2=0x7fc218160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:19.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.764+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 shutdown_connections 2023-08-25T09:40:19.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:19.765+0000 7fc21e07f640 1 -- 172.21.15.70:0/1583652837 wait complete. 2023-08-25T09:40:19.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-08-25T09:40:20.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.029+0000 7fa34340d640 1 Processor -- start 2023-08-25T09:40:20.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.030+0000 7fa34340d640 1 -- start start 2023-08-25T09:40:20.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa34340d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa33c06d3d0 0x7fa33c06d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa34340d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c06e320 con 0x7fa33c15cc00 2023-08-25T09:40:20.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa34340d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c15cfe0 con 0x7fa33c06def0 2023-08-25T09:40:20.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa34340d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c0cfdb0 con 0x7fa33c06d3d0 2023-08-25T09:40:20.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa3411a9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa33c15cc00 0x7fa33c0cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38440/0 (socket says 172.21.15.70:38440) 2023-08-25T09:40:20.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa3411a9640 1 -- 172.21.15.70:0/1443780446 learned_addr learned my addr 172.21.15.70:0/1443780446 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:20.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443135978 0 0) 0x7fa33c06e320 con 0x7fa33c15cc00 2023-08-25T09:40:20.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.031+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa324003520 con 0x7fa33c15cc00 2023-08-25T09:40:20.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa3409a8640 1 --2- 172.21.15.70:0/1443780446 >> v2:172.21.15.155:3300/0 conn(0x7fa33c06d3d0 0x7fa33c06d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544370186 0 0) 0x7fa324003520 con 0x7fa33c15cc00 2023-08-25T09:40:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 >> v2:172.21.15.155:3300/0 conn(0x7fa33c06d3d0 msgr2=0x7fa33c06d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 --2- 172.21.15.70:0/1443780446 >> v2:172.21.15.155:3300/0 conn(0x7fa33c06d3d0 0x7fa33c06d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 >> v1:172.21.15.155:6789/0 conn(0x7fa33c06def0 legacy=0x7fa33c166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c069850 con 0x7fa33c15cc00 2023-08-25T09:40:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa32c001b70 con 0x7fa33c15cc00 2023-08-25T09:40:20.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 >> v1:172.21.15.70:6789/0 conn(0x7fa33c15cc00 legacy=0x7fa33c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa324003970 0x7fa324003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3240047c0 0x7fa324004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa32400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0cfdb0 con 0x7fa324005180 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32400c080 con 0x7fa324003970 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32400c240 con 0x7fa3240047c0 2023-08-25T09:40:20.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa3419aa640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa32400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.032+0000 7fa3411a9640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa324003970 0x7fa324003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.033+0000 7fa3419aa640 1 -- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3240047c0 msgr2=0x7fa324004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.033+0000 7fa3419aa640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3240047c0 0x7fa324004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.033+0000 7fa3419aa640 1 -- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa324003970 msgr2=0x7fa324003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.033+0000 7fa3419aa640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa324003970 0x7fa324003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.033+0000 7fa3419aa640 1 -- 172.21.15.70:0/1443780446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c069850 con 0x7fa324005180 2023-08-25T09:40:20.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.033+0000 7fa3419aa640 1 -- 172.21.15.70:0/1443780446 reap_dead start 2023-08-25T09:40:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.034+0000 7fa3419aa640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa32400b980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa334003db0 tx=0x7fa3240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7af10318de7c298 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.034+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334007220 con 0x7fa324005180 2023-08-25T09:40:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.034+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33402e050 con 0x7fa324005180 2023-08-25T09:40:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.034+0000 7fa33bfff640 1 -- 172.21.15.70:0/1443780446 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3340075e0 con 0x7fa324005180 2023-08-25T09:40:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.034+0000 7fa34340d640 1 -- 172.21.15.70:0/1443780446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 msgr2=0x7fa32400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.034+0000 7fa34340d640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa32400b980 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa334003db0 tx=0x7fa3240057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.035+0000 7fa34340d640 1 -- 172.21.15.70:0/1443780446 shutdown_connections 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.035+0000 7fa34340d640 1 --2- 172.21.15.70:0/1443780446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa32400b980 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.035+0000 7fa34340d640 1 -- 172.21.15.70:0/1443780446 >> 172.21.15.70:0/1443780446 conn(0x7fa33c0d0860 msgr2=0x7fa33c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.035+0000 7fa34340d640 1 -- 172.21.15.70:0/1443780446 shutdown_connections 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.035+0000 7fa34340d640 1 -- 172.21.15.70:0/1443780446 wait complete. 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.036+0000 7fa34340d640 1 Processor -- start 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.036+0000 7fa34340d640 1 -- start start 2023-08-25T09:40:20.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa34340d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa33c159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa34340d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33c1f5240 0x7fa33c1f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa34340d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa33c0c5d60 0x7fa33c0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa34340d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0c6720 con 0x7fa324005180 2023-08-25T09:40:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa34340d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c0cb580 con 0x7fa33c1f5240 2023-08-25T09:40:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa34340d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c1599a0 con 0x7fa33c0c5d60 2023-08-25T09:40:20.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.037+0000 7fa3411a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa33c159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa33c159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43756/0 (socket says 172.21.15.70:43756) 2023-08-25T09:40:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 -- 172.21.15.70:0/1074266372 learned_addr learned my addr 172.21.15.70:0/1074266372 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 -- 172.21.15.70:0/1074266372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa33c0c5d60 msgr2=0x7fa33c0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa33c0c5d60 0x7fa33c0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 -- 172.21.15.70:0/1074266372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33c1f5240 msgr2=0x7fa33c1f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33c1f5240 0x7fa33c1f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 -- 172.21.15.70:0/1074266372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c1f5db0 con 0x7fa324005180 2023-08-25T09:40:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa3411a9640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa33c159460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00ebc0 tx=0x7fa32c00c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c00cbc0 con 0x7fa324005180 2023-08-25T09:40:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c00cd40 con 0x7fa324005180 2023-08-25T09:40:20.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.038+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c005d60 con 0x7fa324005180 2023-08-25T09:40:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.039+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c0caf30 con 0x7fa324005180 2023-08-25T09:40:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.039+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c159e30 con 0x7fa324005180 2023-08-25T09:40:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.040+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c04e420 con 0x7fa324005180 2023-08-25T09:40:20.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.041+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa32c0070d0 con 0x7fa324005180 2023-08-25T09:40:20.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.041+0000 7fa339ffb640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa32806a050 0x7fa32806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.041+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa32c0887d0 con 0x7fa324005180 2023-08-25T09:40:20.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.042+0000 7fa3409a8640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa32806a050 0x7fa32806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.042+0000 7fa3409a8640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa32806a050 0x7fa32806c510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa330006f00 tx=0x7fa330008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:20.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.046+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa32c054160 con 0x7fa324005180 2023-08-25T09:40:20.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.307+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fa33c0c5b40 con 0x7fa324005180 2023-08-25T09:40:20.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.314+0000 7fa339ffb640 1 -- 172.21.15.70:0/1074266372 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa32c057780 con 0x7fa324005180 2023-08-25T09:40:20.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa32806a050 msgr2=0x7fa32806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa32806a050 0x7fa32806c510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa330006f00 tx=0x7fa330008040 comp rx=0 tx=0).stop 2023-08-25T09:40:20.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 msgr2=0x7fa33c159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa33c159460 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00ebc0 tx=0x7fa32c00c4b0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 shutdown_connections 2023-08-25T09:40:20.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa32806a050 0x7fa32806c510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa33c0c5d60 0x7fa33c0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa33c1f5240 0x7fa33c1f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 --2- 172.21.15.70:0/1074266372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa324005180 0x7fa33c159460 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.316+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 >> 172.21.15.70:0/1074266372 conn(0x7fa33c0d0860 msgr2=0x7fa33c15bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.317+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 shutdown_connections 2023-08-25T09:40:20.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.317+0000 7fa34340d640 1 -- 172.21.15.70:0/1074266372 wait complete. 2023-08-25T09:40:20.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-08-25T09:40:20.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.549+0000 7f06654ec640 1 Processor -- start 2023-08-25T09:40:20.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.549+0000 7f06654ec640 1 -- start start 2023-08-25T09:40:20.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f06654ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015ec10 0x7f066015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f06654ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f066015c6d0 0x7f06600b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f06654ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06600b1b60 con 0x7f066015e830 2023-08-25T09:40:20.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f06654ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066006bd80 con 0x7f066015c6d0 2023-08-25T09:40:20.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f06654ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066006b7c0 con 0x7f066015ec10 2023-08-25T09:40:20.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f065f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f066015c6d0 0x7f06600b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f065f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f066015c6d0 0x7f06600b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43764/0 (socket says 172.21.15.70:43764) 2023-08-25T09:40:20.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.551+0000 7f065f7fe640 1 -- 172.21.15.70:0/41481251 learned_addr learned my addr 172.21.15.70:0/41481251 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:20.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.552+0000 7f065e7fc640 1 --2- 172.21.15.70:0/41481251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015ec10 0x7f066015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.552+0000 7f065f7fe640 1 -- 172.21.15.70:0/41481251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015ec10 msgr2=0x7f066015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:20.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.552+0000 7f065f7fe640 1 --2- 172.21.15.70:0/41481251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015ec10 0x7f066015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.552+0000 7f065f7fe640 1 -- 172.21.15.70:0/41481251 >> v1:172.21.15.155:6789/0 conn(0x7f066015e830 legacy=0x7f066006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:20.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.552+0000 7f065f7fe640 1 -- 172.21.15.70:0/41481251 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066006baa0 con 0x7f066015c6d0 2023-08-25T09:40:20.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.553+0000 7f065f7fe640 1 --2- 172.21.15.70:0/41481251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015c6d0 0x7f06600b1ff0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0658009e50 tx=0x7f065802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3e016e44ab15c9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.553+0000 7f065dffb640 1 -- 172.21.15.70:0/41481251 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065802ca80 con 0x7f066015c6d0 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.553+0000 7f065dffb640 1 -- 172.21.15.70:0/41481251 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065802cbe0 con 0x7f066015c6d0 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.553+0000 7f065dffb640 1 -- 172.21.15.70:0/41481251 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06580343f0 con 0x7f066015c6d0 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 -- 172.21.15.70:0/41481251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015c6d0 msgr2=0x7f06600b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 --2- 172.21.15.70:0/41481251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015c6d0 0x7f06600b1ff0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0658009e50 tx=0x7f065802bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 -- 172.21.15.70:0/41481251 shutdown_connections 2023-08-25T09:40:20.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 --2- 172.21.15.70:0/41481251 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015c6d0 0x7f06600b1ff0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 --2- 172.21.15.70:0/41481251 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015ec10 0x7f066015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 -- 172.21.15.70:0/41481251 >> 172.21.15.70:0/41481251 conn(0x7f0660069850 msgr2=0x7f0660069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 -- 172.21.15.70:0/41481251 shutdown_connections 2023-08-25T09:40:20.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.554+0000 7f06654ec640 1 -- 172.21.15.70:0/41481251 wait complete. 2023-08-25T09:40:20.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.556+0000 7f06654ec640 1 Processor -- start 2023-08-25T09:40:20.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.556+0000 7f06654ec640 1 -- start start 2023-08-25T09:40:20.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f06654ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015c6d0 0x7f06601593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f06654ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 0x7f06600cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f06654ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066015ec10 0x7f06600abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f06654ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06601f2490 con 0x7f066015e830 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f06654ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0660159900 con 0x7f066015ec10 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f06654ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600b00c0 con 0x7f066015c6d0 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f065e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 0x7f06600cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f065e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 0x7f06600cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43768/0 (socket says 172.21.15.70:43768) 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f065e7fc640 1 -- 172.21.15.70:0/438170112 learned_addr learned my addr 172.21.15.70:0/438170112 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.557+0000 7f065f7fe640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066015ec10 0x7f06600abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065effd640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015c6d0 0x7f06601593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065e7fc640 1 -- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015c6d0 msgr2=0x7f06601593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065e7fc640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015c6d0 0x7f06601593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065e7fc640 1 -- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066015ec10 msgr2=0x7f06600abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065e7fc640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066015ec10 0x7f06600abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065e7fc640 1 -- 172.21.15.70:0/438170112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0658009b30 con 0x7f066015e830 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065f7fe640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066015ec10 0x7f06600abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f065e7fc640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 0x7f06600cd970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f065400d5f0 tx=0x7f065400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:20.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540125f0 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654012750 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654012b00 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.558+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06600aec20 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.559+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06600af1e0 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.560+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066004e420 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.560+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0654012c60 con 0x7f066015e830 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.561+0000 7f06477fe640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064c06a080 0x7f064c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.561+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0654088670 con 0x7f066015e830 2023-08-25T09:40:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.562+0000 7f065effd640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064c06a080 0x7f064c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.562+0000 7f065effd640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064c06a080 0x7f064c06c540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06600abbd0 tx=0x7f0650009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:20.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.566+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f06540540a0 con 0x7f066015e830 2023-08-25T09:40:20.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.803+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f066006c620 con 0x7f066015e830 2023-08-25T09:40:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.811+0000 7f06477fe640 1 -- 172.21.15.70:0/438170112 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f06540576c0 con 0x7f066015e830 2023-08-25T09:40:20.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064c06a080 msgr2=0x7f064c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064c06a080 0x7f064c06c540 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06600abbd0 tx=0x7f0650009040 comp rx=0 tx=0).stop 2023-08-25T09:40:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 msgr2=0x7f06600cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 0x7f06600cd970 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f065400d5f0 tx=0x7f065400dab0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 shutdown_connections 2023-08-25T09:40:20.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064c06a080 0x7f064c06c540 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f066015ec10 0x7f06600abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f066015e830 0x7f06600cd970 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 --2- 172.21.15.70:0/438170112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015c6d0 0x7f06601593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:20.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 >> 172.21.15.70:0/438170112 conn(0x7f0660069850 msgr2=0x7f0660070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:20.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 shutdown_connections 2023-08-25T09:40:20.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:20.814+0000 7f06654ec640 1 -- 172.21.15.70:0/438170112 wait complete. 2023-08-25T09:40:20.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-08-25T09:40:21.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.034+0000 7f32312ed640 1 Processor -- start 2023-08-25T09:40:21.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.034+0000 7f32312ed640 1 -- start start 2023-08-25T09:40:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.035+0000 7f32312ed640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f322c0cec20 0x7f322c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f32312ed640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f322c071b40 0x7f322c0ca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f32312ed640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0b2130 con 0x7f322c15a840 2023-08-25T09:40:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f32312ed640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0cacb0 con 0x7f322c0cec20 2023-08-25T09:40:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f32312ed640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0b1520 con 0x7f322c071b40 2023-08-25T09:40:21.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f322c0cec20 0x7f322c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f322c0cec20 0x7f322c0cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43774/0 (socket says 172.21.15.70:43774) 2023-08-25T09:40:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 -- 172.21.15.70:0/1824016881 learned_addr learned my addr 172.21.15.70:0/1824016881 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3230aec640 1 --2- 172.21.15.70:0/1824016881 >> v2:172.21.15.155:3300/0 conn(0x7f322c071b40 0x7f322c0ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 -- 172.21.15.70:0/1824016881 >> v2:172.21.15.155:3300/0 conn(0x7f322c071b40 msgr2=0x7f322c0ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 --2- 172.21.15.70:0/1824016881 >> v2:172.21.15.155:3300/0 conn(0x7f322c071b40 0x7f322c0ca770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 -- 172.21.15.70:0/1824016881 >> v1:172.21.15.155:6789/0 conn(0x7f322c15a840 legacy=0x7f322c0ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.036+0000 7f3223fff640 1 -- 172.21.15.70:0/1824016881 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0b1800 con 0x7f322c0cec20 2023-08-25T09:40:21.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.037+0000 7f3223fff640 1 --2- 172.21.15.70:0/1824016881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c0cec20 0x7f322c0cae50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f322c0c8b70 tx=0x7f321402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb2f83600c1195a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:21.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f322b7fe640 1 -- 172.21.15.70:0/1824016881 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32140317e0 con 0x7f322c0cec20 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f322b7fe640 1 -- 172.21.15.70:0/1824016881 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214031940 con 0x7f322c0cec20 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f322b7fe640 1 -- 172.21.15.70:0/1824016881 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214031cd0 con 0x7f322c0cec20 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f32312ed640 1 -- 172.21.15.70:0/1824016881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c0cec20 msgr2=0x7f322c0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f32312ed640 1 --2- 172.21.15.70:0/1824016881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c0cec20 0x7f322c0cae50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f322c0c8b70 tx=0x7f321402b880 comp rx=0 tx=0).stop 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f32312ed640 1 -- 172.21.15.70:0/1824016881 shutdown_connections 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f32312ed640 1 --2- 172.21.15.70:0/1824016881 >> v2:172.21.15.155:3300/0 conn(0x7f322c071b40 0x7f322c0ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f32312ed640 1 --2- 172.21.15.70:0/1824016881 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c0cec20 0x7f322c0cae50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.038+0000 7f32312ed640 1 -- 172.21.15.70:0/1824016881 >> 172.21.15.70:0/1824016881 conn(0x7f322c069eb0 msgr2=0x7f322c0b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:21.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.039+0000 7f32312ed640 1 -- 172.21.15.70:0/1824016881 shutdown_connections 2023-08-25T09:40:21.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.039+0000 7f32312ed640 1 -- 172.21.15.70:0/1824016881 wait complete. 2023-08-25T09:40:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.040+0000 7f32312ed640 1 Processor -- start 2023-08-25T09:40:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.040+0000 7f32312ed640 1 -- start start 2023-08-25T09:40:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.041+0000 7f32312ed640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c071b40 0x7f322c0cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.041+0000 7f32312ed640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c0cec20 0x7f322c0aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.041+0000 7f32312ed640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 0x7f322c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.041+0000 7f32312ed640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1f67a0 con 0x7f322c15a840 2023-08-25T09:40:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.041+0000 7f32312ed640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1f5b90 con 0x7f322c071b40 2023-08-25T09:40:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.041+0000 7f32312ed640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c1f5e70 con 0x7f322c0cec20 2023-08-25T09:40:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 0x7f322c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 0x7f322c15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43780/0 (socket says 172.21.15.70:43780) 2023-08-25T09:40:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 -- 172.21.15.70:0/3271804407 learned_addr learned my addr 172.21.15.70:0/3271804407 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:21.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f322bfff640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c071b40 0x7f322c0cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 -- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c0cec20 msgr2=0x7f322c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c0cec20 0x7f322c0aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 -- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c071b40 msgr2=0x7f322c0cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c071b40 0x7f322c0cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 -- 172.21.15.70:0/3271804407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32140094d0 con 0x7f322c15a840 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.042+0000 7f3230aec640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 0x7f322c15c870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3224007e40 tx=0x7f322400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.043+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224014620 con 0x7f322c15a840 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.043+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400ee90 con 0x7f322c15a840 2023-08-25T09:40:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.043+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240149e0 con 0x7f322c15a840 2023-08-25T09:40:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.043+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c1f6150 con 0x7f322c15a840 2023-08-25T09:40:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.043+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0ce110 con 0x7f322c15a840 2023-08-25T09:40:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.044+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c0ac910 con 0x7f322c15a840 2023-08-25T09:40:21.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.045+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3224014b40 con 0x7f322c15a840 2023-08-25T09:40:21.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.045+0000 7f32297fa640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f320c06a050 0x7f320c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.046+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3224088c60 con 0x7f322c15a840 2023-08-25T09:40:21.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.046+0000 7f322bfff640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f320c06a050 0x7f320c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.046+0000 7f322bfff640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f320c06a050 0x7f320c06c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f321c006fd0 tx=0x7f321c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:21.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.050+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f322c0ac910 con 0x7f322c15a840 2023-08-25T09:40:21.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.302+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f322c0ac910 con 0x7f322c15a840 2023-08-25T09:40:21.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.310+0000 7f32297fa640 1 -- 172.21.15.70:0/3271804407 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f322c0ac910 con 0x7f322c15a840 2023-08-25T09:40:21.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.313+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f320c06a050 msgr2=0x7f320c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.313+0000 7f32312ed640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f320c06a050 0x7f320c06c510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f321c006fd0 tx=0x7f321c008040 comp rx=0 tx=0).stop 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.313+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 msgr2=0x7f322c15c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.313+0000 7f32312ed640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 0x7f322c15c870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3224007e40 tx=0x7f322400e4b0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 shutdown_connections 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f320c06a050 0x7f320c06c510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f322c15a840 0x7f322c15c870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f322c0cec20 0x7f322c0aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 --2- 172.21.15.70:0/3271804407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f322c071b40 0x7f322c0cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 >> 172.21.15.70:0/3271804407 conn(0x7f322c069eb0 msgr2=0x7f322c0c88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 shutdown_connections 2023-08-25T09:40:21.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.314+0000 7f32312ed640 1 -- 172.21.15.70:0/3271804407 wait complete. 2023-08-25T09:40:21.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-08-25T09:40:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.539+0000 7f2af3cec640 1 Processor -- start 2023-08-25T09:40:21.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.539+0000 7f2af3cec640 1 -- start start 2023-08-25T09:40:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af3cec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2aec15ec10 0x7f2aec15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af3cec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0b1b60 con 0x7f2aec15c6d0 2023-08-25T09:40:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af3cec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec06b7c0 con 0x7f2aec15e830 2023-08-25T09:40:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af3cec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec06bd80 con 0x7f2aec15ec10 2023-08-25T09:40:21.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af2289640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2aec15c6d0 0x7f2aec0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38446/0 (socket says 172.21.15.70:38446) 2023-08-25T09:40:21.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af2289640 1 -- 172.21.15.70:0/822897121 learned_addr learned my addr 172.21.15.70:0/822897121 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:21.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.541+0000 7f2af1287640 1 --2- 172.21.15.70:0/822897121 >> v2:172.21.15.155:3300/0 conn(0x7f2aec15ec10 0x7f2aec15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850277284 0 0) 0x7f2aec0b1b60 con 0x7f2aec15c6d0 2023-08-25T09:40:21.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2adc003520 con 0x7f2aec15c6d0 2023-08-25T09:40:21.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372509900 0 0) 0x7f2adc003520 con 0x7f2aec15c6d0 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 >> v2:172.21.15.155:3300/0 conn(0x7f2aec15ec10 msgr2=0x7f2aec15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 --2- 172.21.15.70:0/822897121 >> v2:172.21.15.155:3300/0 conn(0x7f2aec15ec10 0x7f2aec15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 >> v1:172.21.15.155:6789/0 conn(0x7f2aec15e830 legacy=0x7f2aec06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec06baa0 con 0x7f2aec15c6d0 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f2ae4001670 con 0x7f2aec15c6d0 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 >> v1:172.21.15.70:6789/0 conn(0x7f2aec15c6d0 legacy=0x7f2aec0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2adc003970 0x7f2adc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc0047c0 0x7f2adc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc005180 0x7f2adc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec06bd80 con 0x7f2adc005180 2023-08-25T09:40:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc00c120 con 0x7f2adc0047c0 2023-08-25T09:40:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.542+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2adc00c2e0 con 0x7f2adc003970 2023-08-25T09:40:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af2289640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc005180 0x7f2adc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af1287640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc0047c0 0x7f2adc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af1a88640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2adc003970 0x7f2adc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af2289640 1 -- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2adc003970 msgr2=0x7f2adc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af2289640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2adc003970 0x7f2adc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af2289640 1 -- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc0047c0 msgr2=0x7f2adc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af2289640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc0047c0 0x7f2adc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.543+0000 7f2af2289640 1 -- 172.21.15.70:0/822897121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc003520 con 0x7f2adc005180 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.544+0000 7f2af2289640 1 -- 172.21.15.70:0/822897121 reap_dead start 2023-08-25T09:40:21.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.544+0000 7f2af2289640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc005180 0x7f2adc00b980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c190 tx=0x7f2adc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0320e73ec179f2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.544+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4005ad0 con 0x7f2adc005180 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.544+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae4005c70 con 0x7f2adc005180 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.544+0000 7f2af0a86640 1 -- 172.21.15.70:0/822897121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae402c8a0 con 0x7f2adc005180 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 -- 172.21.15.70:0/822897121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc005180 msgr2=0x7f2adc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc005180 0x7f2adc00b980 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c190 tx=0x7f2adc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 -- 172.21.15.70:0/822897121 shutdown_connections 2023-08-25T09:40:21.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 --2- 172.21.15.70:0/822897121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2adc005180 0x7f2adc00b980 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 -- 172.21.15.70:0/822897121 >> 172.21.15.70:0/822897121 conn(0x7f2aec069850 msgr2=0x7f2aec069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 -- 172.21.15.70:0/822897121 shutdown_connections 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.545+0000 7f2af3cec640 1 -- 172.21.15.70:0/822897121 wait complete. 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.547+0000 7f2af3cec640 1 Processor -- start 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.547+0000 7f2af3cec640 1 -- start start 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af3cec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc005180 0x7f2aec0ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af3cec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 0x7f2aec0ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af3cec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0c5930 0x7f2aec0c5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af3cec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0c44e0 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af3cec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0c4900 con 0x7f2adc005180 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af3cec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0c62d0 con 0x7f2aec0c5930 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 0x7f2aec0ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 0x7f2aec0ce7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43800/0 (socket says 172.21.15.70:43800) 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 -- 172.21.15.70:0/2104006301 learned_addr learned my addr 172.21.15.70:0/2104006301 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 -- 172.21.15.70:0/2104006301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0c5930 msgr2=0x7f2aec0c5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0c5930 0x7f2aec0c5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 -- 172.21.15.70:0/2104006301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc005180 msgr2=0x7f2aec0ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc005180 0x7f2aec0ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.548+0000 7f2af1287640 1 -- 172.21.15.70:0/2104006301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0ced20 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.549+0000 7f2af1287640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 0x7f2aec0ce7e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8011590 tx=0x7f2ad8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.549+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8015930 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.549+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8015a90 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.549+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8015e40 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.549+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec153a90 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.549+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec0c3b40 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.550+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec04e420 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.551+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2ad801d400 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.552+0000 7f2aea7fc640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ad006a050 0x7f2ad006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.552+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2ad8088590 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.552+0000 7f2af1a88640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ad006a050 0x7f2ad006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:21.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.553+0000 7f2af1a88640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ad006a050 0x7f2ad006c510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00013d0 tx=0x7f2ae0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:21.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.556+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ad8053fc0 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.814+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2aec153a30 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.821+0000 7f2aea7fc640 1 -- 172.21.15.70:0/2104006301 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2ad80575e0 con 0x7f2aec0ce3a0 2023-08-25T09:40:21.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.823+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ad006a050 msgr2=0x7f2ad006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.823+0000 7f2af3cec640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ad006a050 0x7f2ad006c510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00013d0 tx=0x7f2ae0009040 comp rx=0 tx=0).stop 2023-08-25T09:40:21.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.823+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 msgr2=0x7f2aec0ce7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:21.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.823+0000 7f2af3cec640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 0x7f2aec0ce7e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8011590 tx=0x7f2ad8009f90 comp rx=0 tx=0).stop 2023-08-25T09:40:21.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.823+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 shutdown_connections 2023-08-25T09:40:21.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.823+0000 7f2af3cec640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ad006a050 0x7f2ad006c510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.824+0000 7f2af3cec640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2aec0c5930 0x7f2aec0c5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.824+0000 7f2af3cec640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2aec0ce3a0 0x7f2aec0ce7e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.824+0000 7f2af3cec640 1 --2- 172.21.15.70:0/2104006301 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2adc005180 0x7f2aec0ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:21.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.824+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 >> 172.21.15.70:0/2104006301 conn(0x7f2aec069850 msgr2=0x7f2aec0b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:21.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.824+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 shutdown_connections 2023-08-25T09:40:21.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:21.824+0000 7f2af3cec640 1 -- 172.21.15.70:0/2104006301 wait complete. 2023-08-25T09:40:21.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-08-25T09:40:22.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.055+0000 7fc77f6ec640 1 Processor -- start 2023-08-25T09:40:22.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.055+0000 7fc77f6ec640 1 -- start start 2023-08-25T09:40:22.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.057+0000 7fc77f6ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.057+0000 7fc77f6ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc7780ac870 0x7fc7780b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.057+0000 7fc77f6ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77806f050 con 0x7fc77815cb40 2023-08-25T09:40:22.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.057+0000 7fc77f6ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77806f330 con 0x7fc7780ac870 2023-08-25T09:40:22.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.057+0000 7fc77f6ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77806f610 con 0x7fc7780cea30 2023-08-25T09:40:22.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.058+0000 7fc77cc87640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.058+0000 7fc77dc89640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc7780ac870 0x7fc7780b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.058+0000 7fc77cc87640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:45612/0 (socket says 172.21.15.70:45612) 2023-08-25T09:40:22.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.058+0000 7fc77cc87640 1 -- 172.21.15.70:0/3488661386 learned_addr learned my addr 172.21.15.70:0/3488661386 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:22.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.059+0000 7fc77cc87640 1 -- 172.21.15.70:0/3488661386 >> v2:172.21.15.155:3300/0 conn(0x7fc7780ac870 msgr2=0x7fc7780b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:22.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.059+0000 7fc77cc87640 1 --2- 172.21.15.70:0/3488661386 >> v2:172.21.15.155:3300/0 conn(0x7fc7780ac870 0x7fc7780b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.059+0000 7fc77cc87640 1 -- 172.21.15.70:0/3488661386 >> v1:172.21.15.155:6789/0 conn(0x7fc77815cb40 legacy=0x7fc7780ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.059+0000 7fc77cc87640 1 -- 172.21.15.70:0/3488661386 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77806f8f0 con 0x7fc7780cea30 2023-08-25T09:40:22.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.060+0000 7fc77cc87640 1 --2- 172.21.15.70:0/3488661386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780b1b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc768009870 tx=0x7fc76802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c070699226879016 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc76ffff640 1 -- 172.21.15.70:0/3488661386 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768030840 con 0x7fc7780cea30 2023-08-25T09:40:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc76ffff640 1 -- 172.21.15.70:0/3488661386 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7680309a0 con 0x7fc7780cea30 2023-08-25T09:40:22.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc76ffff640 1 -- 172.21.15.70:0/3488661386 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc768030d30 con 0x7fc7780cea30 2023-08-25T09:40:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 -- 172.21.15.70:0/3488661386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 msgr2=0x7fc7780b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/3488661386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780b1b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc768009870 tx=0x7fc76802b980 comp rx=0 tx=0).stop 2023-08-25T09:40:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 -- 172.21.15.70:0/3488661386 shutdown_connections 2023-08-25T09:40:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/3488661386 >> v2:172.21.15.155:3300/0 conn(0x7fc7780ac870 0x7fc7780b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/3488661386 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780b1b30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 -- 172.21.15.70:0/3488661386 >> 172.21.15.70:0/3488661386 conn(0x7fc778069850 msgr2=0x7fc778069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:22.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 -- 172.21.15.70:0/3488661386 shutdown_connections 2023-08-25T09:40:22.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.061+0000 7fc77f6ec640 1 -- 172.21.15.70:0/3488661386 wait complete. 2023-08-25T09:40:22.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.063+0000 7fc77f6ec640 1 Processor -- start 2023-08-25T09:40:22.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.063+0000 7fc77f6ec640 1 -- start start 2023-08-25T09:40:22.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77f6ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 0x7fc7780cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77f6ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77f6ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc77815cb40 0x7fc7781590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77f6ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc778159620 con 0x7fc7780ac870 2023-08-25T09:40:22.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77f6ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7780ccf50 con 0x7fc77815cb40 2023-08-25T09:40:22.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77f6ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc778159850 con 0x7fc7780cea30 2023-08-25T09:40:22.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77d488640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 0x7fc7780cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77d488640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 0x7fc7780cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43802/0 (socket says 172.21.15.70:43802) 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.064+0000 7fc77d488640 1 -- 172.21.15.70:0/1139534960 learned_addr learned my addr 172.21.15.70:0/1139534960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77dc89640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc77815cb40 0x7fc7781590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77d488640 1 -- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 msgr2=0x7fc7780cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77d488640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77d488640 1 -- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc77815cb40 msgr2=0x7fc7781590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77d488640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc77815cb40 0x7fc7781590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77d488640 1 -- 172.21.15.70:0/1139534960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7680094d0 con 0x7fc7780ac870 2023-08-25T09:40:22.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77d488640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 0x7fc7780cb5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc77400c400 tx=0x7fc77400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7740052d0 con 0x7fc7780ac870 2023-08-25T09:40:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774005430 con 0x7fc7780ac870 2023-08-25T09:40:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7780cc340 con 0x7fc7780ac870 2023-08-25T09:40:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.065+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774003830 con 0x7fc7780ac870 2023-08-25T09:40:22.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.066+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7780cc900 con 0x7fc7780ac870 2023-08-25T09:40:22.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.067+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77804e420 con 0x7fc7780ac870 2023-08-25T09:40:22.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.068+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc774003990 con 0x7fc7780ac870 2023-08-25T09:40:22.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.068+0000 7fc76dffb640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc75c06a050 0x7fc75c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.069+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc774087880 con 0x7fc7780ac870 2023-08-25T09:40:22.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.069+0000 7fc77cc87640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc75c06a050 0x7fc75c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.069+0000 7fc77cc87640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc75c06a050 0x7fc75c06c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc768036000 tx=0x7fc76803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.073+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc774053220 con 0x7fc7780ac870 2023-08-25T09:40:22.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.330+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc7780b23e0 con 0x7fc7780ac870 2023-08-25T09:40:22.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.337+0000 7fc76dffb640 1 -- 172.21.15.70:0/1139534960 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc774056840 con 0x7fc7780ac870 2023-08-25T09:40:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc75c06a050 msgr2=0x7fc75c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc75c06a050 0x7fc75c06c510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc768036000 tx=0x7fc76803b040 comp rx=0 tx=0).stop 2023-08-25T09:40:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 msgr2=0x7fc7780cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 0x7fc7780cb5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc77400c400 tx=0x7fc77400c8c0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 shutdown_connections 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc75c06a050 0x7fc75c06c510 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc77815cb40 0x7fc7781590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc7780cea30 0x7fc7780cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 --2- 172.21.15.70:0/1139534960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc7780ac870 0x7fc7780cb5a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 >> 172.21.15.70:0/1139534960 conn(0x7fc778069850 msgr2=0x7fc7780b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 shutdown_connections 2023-08-25T09:40:22.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.340+0000 7fc77f6ec640 1 -- 172.21.15.70:0/1139534960 wait complete. 2023-08-25T09:40:22.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-08-25T09:40:22.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.578+0000 7fe4e5581640 1 Processor -- start 2023-08-25T09:40:22.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.578+0000 7fe4e5581640 1 -- start start 2023-08-25T09:40:22.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.579+0000 7fe4e5581640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4e00b4340 0x7fe4e00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.579+0000 7fe4e5581640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e00b7960 con 0x7fe4e015cc40 2023-08-25T09:40:22.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.579+0000 7fe4e5581640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e00b7df0 con 0x7fe4e00d0970 2023-08-25T09:40:22.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.579+0000 7fe4e5581640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e00b88f0 con 0x7fe4e00b4340 2023-08-25T09:40:22.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.580+0000 7fe4dffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe4e015cc40 0x7fe4e00b81e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38448/0 (socket says 172.21.15.70:38448) 2023-08-25T09:40:22.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.580+0000 7fe4dffff640 1 -- 172.21.15.70:0/3187458271 learned_addr learned my addr 172.21.15.70:0/3187458271 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:22.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.580+0000 7fe4df7fe640 1 --2- 172.21.15.70:0/3187458271 >> v2:172.21.15.70:3300/0 conn(0x7fe4e00b4340 0x7fe4e00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.580+0000 7fe4df7fe640 1 -- 172.21.15.70:0/3187458271 >> v1:172.21.15.155:6789/0 conn(0x7fe4e00d0970 legacy=0x7fe4e00d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:22.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.580+0000 7fe4df7fe640 1 -- 172.21.15.70:0/3187458271 >> v1:172.21.15.70:6789/0 conn(0x7fe4e015cc40 legacy=0x7fe4e00b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.580+0000 7fe4df7fe640 1 -- 172.21.15.70:0/3187458271 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e00b6d50 con 0x7fe4e00b4340 2023-08-25T09:40:22.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.581+0000 7fe4df7fe640 1 --2- 172.21.15.70:0/3187458271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e00b4340 0x7fe4e00b47d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009d90 tx=0x7fe4c802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ba4bca34a2fcb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:22.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4deffd640 1 -- 172.21.15.70:0/3187458271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c802cc60 con 0x7fe4e00b4340 2023-08-25T09:40:22.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4deffd640 1 -- 172.21.15.70:0/3187458271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c802cdc0 con 0x7fe4e00b4340 2023-08-25T09:40:22.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4deffd640 1 -- 172.21.15.70:0/3187458271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8034660 con 0x7fe4e00b4340 2023-08-25T09:40:22.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 -- 172.21.15.70:0/3187458271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e00b4340 msgr2=0x7fe4e00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 --2- 172.21.15.70:0/3187458271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e00b4340 0x7fe4e00b47d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009d90 tx=0x7fe4c802beb0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 -- 172.21.15.70:0/3187458271 shutdown_connections 2023-08-25T09:40:22.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 --2- 172.21.15.70:0/3187458271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e00b4340 0x7fe4e00b47d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 -- 172.21.15.70:0/3187458271 >> 172.21.15.70:0/3187458271 conn(0x7fe4e0070290 msgr2=0x7fe4e00706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:22.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 -- 172.21.15.70:0/3187458271 shutdown_connections 2023-08-25T09:40:22.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.582+0000 7fe4e5581640 1 -- 172.21.15.70:0/3187458271 wait complete. 2023-08-25T09:40:22.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.584+0000 7fe4e5581640 1 Processor -- start 2023-08-25T09:40:22.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.584+0000 7fe4e5581640 1 -- start start 2023-08-25T09:40:22.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e5581640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e00b4340 0x7fe4e00c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e5581640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e00d0970 0x7fe4e0153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e5581640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 0x7fe4e015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e5581640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e015f7d0 con 0x7fe4e015cc40 2023-08-25T09:40:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e5581640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c6570 con 0x7fe4e00d0970 2023-08-25T09:40:22.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e5581640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e00c6850 con 0x7fe4e00b4340 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.585+0000 7fe4e4d80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 0x7fe4e015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 0x7fe4e015edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43810/0 (socket says 172.21.15.70:43810) 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 -- 172.21.15.70:0/614449446 learned_addr learned my addr 172.21.15.70:0/614449446 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4df7fe640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e00d0970 0x7fe4e0153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 -- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e00b4340 msgr2=0x7fe4e00c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e00b4340 0x7fe4e00c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 -- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e00d0970 msgr2=0x7fe4e0153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e00d0970 0x7fe4e0153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 -- 172.21.15.70:0/614449446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8009a70 con 0x7fe4e015cc40 2023-08-25T09:40:22.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4df7fe640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e00d0970 0x7fe4e0153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:22.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.586+0000 7fe4e4d80640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 0x7fe4e015edf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c390 tx=0x7fe4d800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:22.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.587+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80035f0 con 0x7fe4e015cc40 2023-08-25T09:40:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.587+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8003750 con 0x7fe4e015cc40 2023-08-25T09:40:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.587+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8003b00 con 0x7fe4e015cc40 2023-08-25T09:40:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.587+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e00c6b30 con 0x7fe4e015cc40 2023-08-25T09:40:22.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.587+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e01543b0 con 0x7fe4e015cc40 2023-08-25T09:40:22.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.588+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e00c6ff0 con 0x7fe4e015cc40 2023-08-25T09:40:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.589+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe4d8003c60 con 0x7fe4e015cc40 2023-08-25T09:40:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.591+0000 7fe4dcff9640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe4c006a050 0x7fe4c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.591+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe4d8087570 con 0x7fe4e015cc40 2023-08-25T09:40:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.591+0000 7fe4dffff640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe4c006a050 0x7fe4c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:22.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.591+0000 7fe4dffff640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe4c006a050 0x7fe4c006c510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0001550 tx=0x7fe4d0007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:22.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.595+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe4d8052f10 con 0x7fe4e015cc40 2023-08-25T09:40:22.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.846+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe4e0066fb0 con 0x7fe4e015cc40 2023-08-25T09:40:22.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.852+0000 7fe4dcff9640 1 -- 172.21.15.70:0/614449446 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe4d8056530 con 0x7fe4e015cc40 2023-08-25T09:40:22.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe4c006a050 msgr2=0x7fe4c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe4c006a050 0x7fe4c006c510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0001550 tx=0x7fe4d0007a40 comp rx=0 tx=0).stop 2023-08-25T09:40:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 msgr2=0x7fe4e015edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 0x7fe4e015edf0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c390 tx=0x7fe4d800c850 comp rx=0 tx=0).stop 2023-08-25T09:40:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 shutdown_connections 2023-08-25T09:40:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe4c006a050 0x7fe4c006c510 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4e015cc40 0x7fe4e015edf0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4e00d0970 0x7fe4e0153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 --2- 172.21.15.70:0/614449446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4e00b4340 0x7fe4e00c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:22.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.855+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 >> 172.21.15.70:0/614449446 conn(0x7fe4e0070290 msgr2=0x7fe4e00b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:22.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.856+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 shutdown_connections 2023-08-25T09:40:22.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:22.856+0000 7fe4e5581640 1 -- 172.21.15.70:0/614449446 wait complete. 2023-08-25T09:40:22.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:23.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.071+0000 7f0763fff640 1 Processor -- start 2023-08-25T09:40:23.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.071+0000 7f0763fff640 1 -- start start 2023-08-25T09:40:23.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.072+0000 7f0763fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0764159840 0x7f07640b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.072+0000 7f0763fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f076415d3d0 0x7f07640b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.072+0000 7f0763fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076415d890 con 0x7f07640cadb0 2023-08-25T09:40:23.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.072+0000 7f0763fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640c8b60 con 0x7f076415d3d0 2023-08-25T09:40:23.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.072+0000 7f0763fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076415db70 con 0x7f0764159840 2023-08-25T09:40:23.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f076415d3d0 0x7f07640b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f076415d3d0 0x7f07640b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43824/0 (socket says 172.21.15.70:43824) 2023-08-25T09:40:23.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 -- 172.21.15.70:0/4279150706 learned_addr learned my addr 172.21.15.70:0/4279150706 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:23.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 -- 172.21.15.70:0/4279150706 >> v2:172.21.15.155:3300/0 conn(0x7f0764159840 msgr2=0x7f07640b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:23.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f0762ffd640 1 --2- 172.21.15.70:0/4279150706 >> v2:172.21.15.155:3300/0 conn(0x7f0764159840 0x7f07640b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 --2- 172.21.15.70:0/4279150706 >> v2:172.21.15.155:3300/0 conn(0x7f0764159840 0x7f07640b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 -- 172.21.15.70:0/4279150706 >> v1:172.21.15.70:6789/0 conn(0x7f07640cadb0 legacy=0x7f07640cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.073+0000 7f075bfff640 1 -- 172.21.15.70:0/4279150706 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076415de50 con 0x7f076415d3d0 2023-08-25T09:40:23.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.074+0000 7f075bfff640 1 --2- 172.21.15.70:0/4279150706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f076415d3d0 0x7f07640b3d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f076406bec0 tx=0x7f074c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f1a239b1dcfd64 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.074+0000 7f07627fc640 1 -- 172.21.15.70:0/4279150706 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c030870 con 0x7f076415d3d0 2023-08-25T09:40:23.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f07627fc640 1 -- 172.21.15.70:0/4279150706 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c0309d0 con 0x7f076415d3d0 2023-08-25T09:40:23.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f07627fc640 1 -- 172.21.15.70:0/4279150706 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c030d60 con 0x7f076415d3d0 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 -- 172.21.15.70:0/4279150706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f076415d3d0 msgr2=0x7f07640b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 --2- 172.21.15.70:0/4279150706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f076415d3d0 0x7f07640b3d40 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f076406bec0 tx=0x7f074c02b9b0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 -- 172.21.15.70:0/4279150706 shutdown_connections 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 --2- 172.21.15.70:0/4279150706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f076415d3d0 0x7f07640b3d40 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 --2- 172.21.15.70:0/4279150706 >> v2:172.21.15.155:3300/0 conn(0x7f0764159840 0x7f07640b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 -- 172.21.15.70:0/4279150706 >> 172.21.15.70:0/4279150706 conn(0x7f07640affd0 msgr2=0x7f07640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 -- 172.21.15.70:0/4279150706 shutdown_connections 2023-08-25T09:40:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.075+0000 7f0763fff640 1 -- 172.21.15.70:0/4279150706 wait complete. 2023-08-25T09:40:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.077+0000 7f0763fff640 1 Processor -- start 2023-08-25T09:40:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.077+0000 7f0763fff640 1 -- start start 2023-08-25T09:40:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0763fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 0x7f07641475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0763fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764159840 0x7f07641f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0763fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f076415d3d0 0x7f07640aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0763fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640701d0 con 0x7f07640cadb0 2023-08-25T09:40:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0763fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0764149f50 con 0x7f0764159840 2023-08-25T09:40:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0763fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0764147af0 con 0x7f076415d3d0 2023-08-25T09:40:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0762ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 0x7f07641475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0762ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 0x7f07641475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43838/0 (socket says 172.21.15.70:43838) 2023-08-25T09:40:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.078+0000 7f0762ffd640 1 -- 172.21.15.70:0/3423419813 learned_addr learned my addr 172.21.15.70:0/3423419813 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0762ffd640 1 -- 172.21.15.70:0/3423419813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f076415d3d0 msgr2=0x7f07640aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0762ffd640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f076415d3d0 0x7f07640aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0762ffd640 1 -- 172.21.15.70:0/3423419813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764159840 msgr2=0x7f07641f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0762ffd640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764159840 0x7f07641f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0762ffd640 1 -- 172.21.15.70:0/3423419813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0094d0 con 0x7f07640cadb0 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0762ffd640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 0x7f07641475b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f075400c820 tx=0x7f075400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754004220 con 0x7f07640cadb0 2023-08-25T09:40:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07540043a0 con 0x7f07640cadb0 2023-08-25T09:40:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754007a70 con 0x7f07640cadb0 2023-08-25T09:40:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640c6a50 con 0x7f07640cadb0 2023-08-25T09:40:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.079+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640c7010 con 0x7f07640cadb0 2023-08-25T09:40:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.081+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0764066fb0 con 0x7f07640cadb0 2023-08-25T09:40:23.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.081+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0754007bd0 con 0x7f07640cadb0 2023-08-25T09:40:23.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.082+0000 7f075b7fe640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f074406a050 0x7f074406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.082+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0754088040 con 0x7f07640cadb0 2023-08-25T09:40:23.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.082+0000 7f075bfff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f074406a050 0x7f074406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.083+0000 7f075bfff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f074406a050 0x7f074406c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f074c036000 tx=0x7f074c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.087+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0754053a80 con 0x7f07640cadb0 2023-08-25T09:40:23.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.274+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07640c9a90 con 0x7f07640cadb0 2023-08-25T09:40:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.276+0000 7f075b7fe640 1 -- 172.21.15.70:0/3423419813 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+14653 (secure 0 0 0) 0x7f07540570a0 con 0x7f07640cadb0 2023-08-25T09:40:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:23.279 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":31,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:19.239755+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:23.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f074406a050 msgr2=0x7f074406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f074406a050 0x7f074406c510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f074c036000 tx=0x7f074c03b040 comp rx=0 tx=0).stop 2023-08-25T09:40:23.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 msgr2=0x7f07641475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 0x7f07641475b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f075400c820 tx=0x7f075400cce0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 shutdown_connections 2023-08-25T09:40:23.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f074406a050 0x7f074406c510 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f076415d3d0 0x7f07640aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0764159840 0x7f07641f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 --2- 172.21.15.70:0/3423419813 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07640cadb0 0x7f07641475b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 >> 172.21.15.70:0/3423419813 conn(0x7f07640affd0 msgr2=0x7f076415af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:23.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 shutdown_connections 2023-08-25T09:40:23.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.279+0000 7f0763fff640 1 -- 172.21.15.70:0/3423419813 wait complete. 2023-08-25T09:40:23.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:23.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.516+0000 7f29f4c04640 1 Processor -- start 2023-08-25T09:40:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.516+0000 7f29f4c04640 1 -- start start 2023-08-25T09:40:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29f4c04640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29f0069f60 0x7f29f00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29f4c04640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00c2550 con 0x7f29f015ae50 2023-08-25T09:40:23.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29f4c04640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f00ce330 con 0x7f29f00c2110 2023-08-25T09:40:23.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29f4c04640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f00cc5f0 con 0x7f29f0069f60 2023-08-25T09:40:23.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29eed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29f0069f60 0x7f29f00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29eed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29f0069f60 0x7f29f00ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43854/0 (socket says 172.21.15.70:43854) 2023-08-25T09:40:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.518+0000 7f29eed9d640 1 -- 172.21.15.70:0/167577610 learned_addr learned my addr 172.21.15.70:0/167577610 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.519+0000 7f29eed9d640 1 -- 172.21.15.70:0/167577610 >> v1:172.21.15.155:6789/0 conn(0x7f29f00c2110 legacy=0x7f29f0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.519+0000 7f29eed9d640 1 -- 172.21.15.70:0/167577610 >> v1:172.21.15.70:6789/0 conn(0x7f29f015ae50 legacy=0x7f29f00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.519+0000 7f29eed9d640 1 -- 172.21.15.70:0/167577610 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f0070650 con 0x7f29f0069f60 2023-08-25T09:40:23.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.520+0000 7f29eed9d640 1 --2- 172.21.15.70:0/167577610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f0069f60 0x7f29f00ccad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009e50 tx=0x7f29e802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84231b4ba91fd968 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.520+0000 7f29ed59a640 1 -- 172.21.15.70:0/167577610 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e802cbb0 con 0x7f29f0069f60 2023-08-25T09:40:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.520+0000 7f29ed59a640 1 -- 172.21.15.70:0/167577610 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e802cd10 con 0x7f29f0069f60 2023-08-25T09:40:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.520+0000 7f29ed59a640 1 -- 172.21.15.70:0/167577610 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8034610 con 0x7f29f0069f60 2023-08-25T09:40:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 -- 172.21.15.70:0/167577610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f0069f60 msgr2=0x7f29f00ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 --2- 172.21.15.70:0/167577610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f0069f60 0x7f29f00ccad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009e50 tx=0x7f29e802c010 comp rx=0 tx=0).stop 2023-08-25T09:40:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 -- 172.21.15.70:0/167577610 shutdown_connections 2023-08-25T09:40:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 --2- 172.21.15.70:0/167577610 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f0069f60 0x7f29f00ccad0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 -- 172.21.15.70:0/167577610 >> 172.21.15.70:0/167577610 conn(0x7f29f00cb0d0 msgr2=0x7f29f00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 -- 172.21.15.70:0/167577610 shutdown_connections 2023-08-25T09:40:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.521+0000 7f29f4c04640 1 -- 172.21.15.70:0/167577610 wait complete. 2023-08-25T09:40:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.522+0000 7f29f4c04640 1 Processor -- start 2023-08-25T09:40:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.523+0000 7f29f4c04640 1 -- start start 2023-08-25T09:40:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.523+0000 7f29f4c04640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f0069f60 0x7f29f0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.523+0000 7f29f4c04640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 0x7f29f01f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29f4c04640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29f015ae50 0x7f29f00b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29f4c04640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01f45d0 con 0x7f29f00c2110 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29f4c04640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01554c0 con 0x7f29f0069f60 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29f4c04640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f01f3950 con 0x7f29f015ae50 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 0x7f29f01f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 0x7f29f01f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43860/0 (socket says 172.21.15.70:43860) 2023-08-25T09:40:23.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 -- 172.21.15.70:0/469648260 learned_addr learned my addr 172.21.15.70:0/469648260 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29eed9d640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29f015ae50 0x7f29f00b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29ee59c640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f0069f60 0x7f29f0154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 -- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29f015ae50 msgr2=0x7f29f00b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29f015ae50 0x7f29f00b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 -- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f0069f60 msgr2=0x7f29f0154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f0069f60 0x7f29f0154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.524+0000 7f29edd9b640 1 -- 172.21.15.70:0/469648260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8009b30 con 0x7f29f00c2110 2023-08-25T09:40:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.525+0000 7f29edd9b640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 0x7f29f01f3100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c430 tx=0x7f29e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.525+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4011790 con 0x7f29f00c2110 2023-08-25T09:40:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.525+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e40118f0 con 0x7f29f00c2110 2023-08-25T09:40:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.525+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4011ce0 con 0x7f29f00c2110 2023-08-25T09:40:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.525+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f01f3c30 con 0x7f29f00c2110 2023-08-25T09:40:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.525+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f01f41f0 con 0x7f29f00c2110 2023-08-25T09:40:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.526+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f01f4710 con 0x7f29f00c2110 2023-08-25T09:40:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.527+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29e401c460 con 0x7f29f00c2110 2023-08-25T09:40:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.528+0000 7f29deffd640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a080 0x7f29d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.528+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f29e40881f0 con 0x7f29f00c2110 2023-08-25T09:40:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.528+0000 7f29ee59c640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a080 0x7f29d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.529+0000 7f29ee59c640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a080 0x7f29d006c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f29e0000f60 tx=0x7f29e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.532+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29f01f4710 con 0x7f29f00c2110 2023-08-25T09:40:23.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.720+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29f01f4710 con 0x7f29f00c2110 2023-08-25T09:40:23.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.721+0000 7f29deffd640 1 -- 172.21.15.70:0/469648260 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29f01f4710 con 0x7f29f00c2110 2023-08-25T09:40:23.723 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:23.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a080 msgr2=0x7f29d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a080 0x7f29d006c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f29e0000f60 tx=0x7f29e0009040 comp rx=0 tx=0).stop 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 msgr2=0x7f29f01f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 0x7f29f01f3100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c430 tx=0x7f29e400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 shutdown_connections 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a080 0x7f29d006c540 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29f015ae50 0x7f29f00b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f00c2110 0x7f29f01f3100 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 --2- 172.21.15.70:0/469648260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f0069f60 0x7f29f0154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 >> 172.21.15.70:0/469648260 conn(0x7f29f00cb0d0 msgr2=0x7f29f00ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:23.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 shutdown_connections 2023-08-25T09:40:23.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.724+0000 7f29f4c04640 1 -- 172.21.15.70:0/469648260 wait complete. 2023-08-25T09:40:23.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:23.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.956+0000 7f38198f9640 1 Processor -- start 2023-08-25T09:40:23.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.956+0000 7f38198f9640 1 -- start start 2023-08-25T09:40:23.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.957+0000 7f38198f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 0x7f3814069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38198f9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3814069d90 0x7f38140ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38198f9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38140c2550 con 0x7f381415ae50 2023-08-25T09:40:23.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38198f9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381406a220 con 0x7f3814069d90 2023-08-25T09:40:23.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38198f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ce330 con 0x7f38140c2110 2023-08-25T09:40:23.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f3812ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f381415ae50 0x7f38140cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38464/0 (socket says 172.21.15.70:38464) 2023-08-25T09:40:23.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f3812ffd640 1 -- 172.21.15.70:0/995640260 learned_addr learned my addr 172.21.15.70:0/995640260 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:23.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f380bfff640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 0x7f3814069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788736721 0 0) 0x7f38140c2550 con 0x7f381415ae50 2023-08-25T09:40:23.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3800003520 con 0x7f381415ae50 2023-08-25T09:40:23.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.958+0000 7f38137fe640 1 --2- 172.21.15.70:0/995640260 >> v2:172.21.15.155:3300/0 conn(0x7f3814069d90 0x7f38140ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1696363800 0 0) 0x7f3800003520 con 0x7f381415ae50 2023-08-25T09:40:23.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 msgr2=0x7f3814069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:23.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 0x7f3814069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 >> v2:172.21.15.155:3300/0 conn(0x7f3814069d90 msgr2=0x7f38140ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:23.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 --2- 172.21.15.70:0/995640260 >> v2:172.21.15.155:3300/0 conn(0x7f3814069d90 0x7f38140ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140cc5f0 con 0x7f381415ae50 2023-08-25T09:40:23.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3804001670 con 0x7f381415ae50 2023-08-25T09:40:23.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 >> v1:172.21.15.70:6789/0 conn(0x7f381415ae50 legacy=0x7f38140cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 0x7f3800003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3800003ef0 0x7f38000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f3800005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381406a220 con 0x7f3800004d00 2023-08-25T09:40:23.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ce330 con 0x7f3800003ef0 2023-08-25T09:40:23.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380000ba60 con 0x7f38140c2110 2023-08-25T09:40:23.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38137fe640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f3800005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f380bfff640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3800003ef0 0x7f38000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f3812ffd640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 0x7f3800003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38137fe640 1 -- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 msgr2=0x7f3800003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:23.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.959+0000 7f38137fe640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140c2110 0x7f3800003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.960+0000 7f38137fe640 1 -- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3800003ef0 msgr2=0x7f38000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:23.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.960+0000 7f38137fe640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3800003ef0 0x7f38000047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.960+0000 7f38137fe640 1 -- 172.21.15.70:0/995640260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3800003520 con 0x7f3800004d00 2023-08-25T09:40:23.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38137fe640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f3800005180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f380c007360 tx=0x7f38000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d96cbf9380cc83 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c02ccf0 con 0x7f3800004d00 2023-08-25T09:40:23.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c02ce50 con 0x7f3800004d00 2023-08-25T09:40:23.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38127fc640 1 -- 172.21.15.70:0/995640260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c034660 con 0x7f3800004d00 2023-08-25T09:40:23.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38198f9640 1 -- 172.21.15.70:0/995640260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 msgr2=0x7f3800005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38198f9640 1 --2- 172.21.15.70:0/995640260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f3800005180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f380c007360 tx=0x7f38000057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38137fe640 1 -- 172.21.15.70:0/995640260 reap_dead start 2023-08-25T09:40:23.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38198f9640 1 -- 172.21.15.70:0/995640260 shutdown_connections 2023-08-25T09:40:23.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.961+0000 7f38198f9640 1 -- 172.21.15.70:0/995640260 >> 172.21.15.70:0/995640260 conn(0x7f38140cb0d0 msgr2=0x7f38140cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:23.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.962+0000 7f38198f9640 1 -- 172.21.15.70:0/995640260 shutdown_connections 2023-08-25T09:40:23.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.962+0000 7f38198f9640 1 -- 172.21.15.70:0/995640260 wait complete. 2023-08-25T09:40:23.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.963+0000 7f38198f9640 1 Processor -- start 2023-08-25T09:40:23.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.963+0000 7f38198f9640 1 -- start start 2023-08-25T09:40:23.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.964+0000 7f38198f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f381414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.964+0000 7f38198f9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38141580f0 0x7f3814158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.964+0000 7f38198f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140ac880 0x7f38140acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.964+0000 7f38198f9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140ad270 con 0x7f3800004d00 2023-08-25T09:40:23.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.964+0000 7f38198f9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141f1fe0 con 0x7f38141580f0 2023-08-25T09:40:23.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.964+0000 7f38198f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38141f2df0 con 0x7f38140ac880 2023-08-25T09:40:23.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f381414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f381414ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43874/0 (socket says 172.21.15.70:43874) 2023-08-25T09:40:23.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 -- 172.21.15.70:0/2010981018 learned_addr learned my addr 172.21.15.70:0/2010981018 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:23.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 -- 172.21.15.70:0/2010981018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140ac880 msgr2=0x7f38140acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140ac880 0x7f38140acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 -- 172.21.15.70:0/2010981018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38141580f0 msgr2=0x7f3814158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:23.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38141580f0 0x7f3814158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:23.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 -- 172.21.15.70:0/2010981018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381414f8b0 con 0x7f3800004d00 2023-08-25T09:40:23.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3812ffd640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f381414ee00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f380400d8e0 tx=0x7f380400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.965+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38040074e0 con 0x7f3800004d00 2023-08-25T09:40:23.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.966+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804007640 con 0x7f3800004d00 2023-08-25T09:40:23.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.966+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38141f21e0 con 0x7f3800004d00 2023-08-25T09:40:23.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.966+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804004a20 con 0x7f3800004d00 2023-08-25T09:40:23.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.966+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141f27a0 con 0x7f3800004d00 2023-08-25T09:40:23.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.967+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814066fb0 con 0x7f3800004d00 2023-08-25T09:40:23.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.968+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3804004b80 con 0x7f3800004d00 2023-08-25T09:40:23.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.968+0000 7f3810ff9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37f406a050 0x7f37f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:23.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.969+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3804087c20 con 0x7f3800004d00 2023-08-25T09:40:23.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.969+0000 7f380bfff640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37f406a050 0x7f37f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:23.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.969+0000 7f380bfff640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37f406a050 0x7f37f406c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38141530c0 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:23.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:23.973+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f38040571b0 con 0x7f3800004d00 2023-08-25T09:40:24.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.162+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f38140c1b30 con 0x7f3800004d00 2023-08-25T09:40:24.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.163+0000 7f3810ff9640 1 -- 172.21.15.70:0/2010981018 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7f38040190a0 con 0x7f3800004d00 2023-08-25T09:40:24.165 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:24.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37f406a050 msgr2=0x7f37f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37f406a050 0x7f37f406c510 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f38141530c0 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2023-08-25T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 msgr2=0x7f381414ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f381414ee00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f380400d8e0 tx=0x7f380400dda0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 shutdown_connections 2023-08-25T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37f406a050 0x7f37f406c510 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38140ac880 0x7f38140acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38141580f0 0x7f3814158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 --2- 172.21.15.70:0/2010981018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3800004d00 0x7f381414ee00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 >> 172.21.15.70:0/2010981018 conn(0x7f38140cb0d0 msgr2=0x7f381406c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.165+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 shutdown_connections 2023-08-25T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.166+0000 7f38198f9640 1 -- 172.21.15.70:0/2010981018 wait complete. 2023-08-25T09:40:24.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:40:24.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.399+0000 7f34b3fff640 1 Processor -- start 2023-08-25T09:40:24.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.399+0000 7f34b3fff640 1 -- start start 2023-08-25T09:40:24.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.400+0000 7f34b3fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f34b415a7c0 0x7f34b40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.400+0000 7f34b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f34b40b12f0 0x7f34b415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.400+0000 7f34b3fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b415e7c0 con 0x7f34b415f190 2023-08-25T09:40:24.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.400+0000 7f34b3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b4071b50 con 0x7f34b40b12f0 2023-08-25T09:40:24.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.400+0000 7f34b3fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b415e490 con 0x7f34b415a7c0 2023-08-25T09:40:24.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f34b40b12f0 0x7f34b415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f34b40b12f0 0x7f34b415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43890/0 (socket says 172.21.15.70:43890) 2023-08-25T09:40:24.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 -- 172.21.15.70:0/319601305 learned_addr learned my addr 172.21.15.70:0/319601305 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:24.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 -- 172.21.15.70:0/319601305 >> v2:172.21.15.155:3300/0 conn(0x7f34b415a7c0 msgr2=0x7f34b40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:24.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34b2ffd640 1 --2- 172.21.15.70:0/319601305 >> v2:172.21.15.155:3300/0 conn(0x7f34b415a7c0 0x7f34b40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 --2- 172.21.15.70:0/319601305 >> v2:172.21.15.155:3300/0 conn(0x7f34b415a7c0 0x7f34b40b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 -- 172.21.15.70:0/319601305 >> v1:172.21.15.155:6789/0 conn(0x7f34b415f190 legacy=0x7f34b40cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:24.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34aa7fc640 1 -- 172.21.15.70:0/319601305 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b40b3aa0 con 0x7f34b40b12f0 2023-08-25T09:40:24.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.401+0000 7f34b2ffd640 1 --2- 172.21.15.70:0/319601305 >> v2:172.21.15.155:3300/0 conn(0x7f34b415a7c0 0x7f34b40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:24.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.402+0000 7f34aa7fc640 1 --2- 172.21.15.70:0/319601305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b40b12f0 0x7f34b415ec50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34b40ce280 tx=0x7f349c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b763f6bc2a2a914b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:24.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b27fc640 1 -- 172.21.15.70:0/319601305 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c031870 con 0x7f34b40b12f0 2023-08-25T09:40:24.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b27fc640 1 -- 172.21.15.70:0/319601305 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c0319d0 con 0x7f34b40b12f0 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b27fc640 1 -- 172.21.15.70:0/319601305 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c031d60 con 0x7f34b40b12f0 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 -- 172.21.15.70:0/319601305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b40b12f0 msgr2=0x7f34b415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 --2- 172.21.15.70:0/319601305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b40b12f0 0x7f34b415ec50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f34b40ce280 tx=0x7f349c02b910 comp rx=0 tx=0).stop 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 -- 172.21.15.70:0/319601305 shutdown_connections 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 --2- 172.21.15.70:0/319601305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b40b12f0 0x7f34b415ec50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 --2- 172.21.15.70:0/319601305 >> v2:172.21.15.155:3300/0 conn(0x7f34b415a7c0 0x7f34b40b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 -- 172.21.15.70:0/319601305 >> 172.21.15.70:0/319601305 conn(0x7f34b40affd0 msgr2=0x7f34b40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:24.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 -- 172.21.15.70:0/319601305 shutdown_connections 2023-08-25T09:40:24.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.403+0000 7f34b3fff640 1 -- 172.21.15.70:0/319601305 wait complete. 2023-08-25T09:40:24.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.405+0000 7f34b3fff640 1 Processor -- start 2023-08-25T09:40:24.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.405+0000 7f34b3fff640 1 -- start start 2023-08-25T09:40:24.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34b40b12f0 0x7f34b4152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34b415a7c0 0x7f34b40c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 0x7f34b40c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40c6740 con 0x7f34b415f190 2023-08-25T09:40:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b3fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40ac920 con 0x7f34b415a7c0 2023-08-25T09:40:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4154c40 con 0x7f34b40b12f0 2023-08-25T09:40:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 0x7f34b40c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.406+0000 7f34b37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 0x7f34b40c70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43900/0 (socket says 172.21.15.70:43900) 2023-08-25T09:40:24.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 -- 172.21.15.70:0/3183483160 learned_addr learned my addr 172.21.15.70:0/3183483160 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b2ffd640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34b40b12f0 0x7f34b4152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34aa7fc640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34b415a7c0 0x7f34b40c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 -- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34b40b12f0 msgr2=0x7f34b4152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34b40b12f0 0x7f34b4152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 -- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34b415a7c0 msgr2=0x7f34b40c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34b415a7c0 0x7f34b40c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 -- 172.21.15.70:0/3183483160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c0094d0 con 0x7f34b415f190 2023-08-25T09:40:24.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b37fe640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 0x7f34b40c70e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c780 tx=0x7f34ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:24.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac003d80 con 0x7f34b415f190 2023-08-25T09:40:24.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac00ce90 con 0x7f34b415f190 2023-08-25T09:40:24.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0052b0 con 0x7f34b415f190 2023-08-25T09:40:24.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.407+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b4154f20 con 0x7f34b415f190 2023-08-25T09:40:24.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.408+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b41554e0 con 0x7f34b415f190 2023-08-25T09:40:24.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.409+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b40acaf0 con 0x7f34b415f190 2023-08-25T09:40:24.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.410+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f34ac005410 con 0x7f34b415f190 2023-08-25T09:40:24.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.411+0000 7f34b0ff9640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f349406a050 0x7f349406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.411+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f34ac087720 con 0x7f34b415f190 2023-08-25T09:40:24.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.411+0000 7f34b2ffd640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f349406a050 0x7f349406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.411+0000 7f34b2ffd640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f349406a050 0x7f349406c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f34b415e350 tx=0x7f34a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:24.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.415+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f34b40acaf0 con 0x7f34b415f190 2023-08-25T09:40:24.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.607+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f34b40aca90 con 0x7f34b415f190 2023-08-25T09:40:24.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.608+0000 7f34b0ff9640 1 -- 172.21.15.70:0/3183483160 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+126 (secure 0 0 0) 0x7f34ac017090 con 0x7f34b415f190 2023-08-25T09:40:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:24.611 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:40:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:40:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:40:24.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:40:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:40:24.612 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:24.613 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:40:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f349406a050 msgr2=0x7f349406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f349406a050 0x7f349406c510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f34b415e350 tx=0x7f34a4008040 comp rx=0 tx=0).stop 2023-08-25T09:40:24.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 msgr2=0x7f34b40c70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 0x7f34b40c70e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00c780 tx=0x7f34ac00cc40 comp rx=0 tx=0).stop 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 shutdown_connections 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f349406a050 0x7f349406c510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f34b415f190 0x7f34b40c70e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f34b415a7c0 0x7f34b40c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 --2- 172.21.15.70:0/3183483160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34b40b12f0 0x7f34b4152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.611+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 >> 172.21.15.70:0/3183483160 conn(0x7f34b40affd0 msgr2=0x7f34b4159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.612+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 shutdown_connections 2023-08-25T09:40:24.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.612+0000 7f34b3fff640 1 -- 172.21.15.70:0/3183483160 wait complete. 2023-08-25T09:40:24.631 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:40:24.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:40:24.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.842+0000 7f131e63f640 1 Processor -- start 2023-08-25T09:40:24.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.842+0000 7f131e63f640 1 -- start start 2023-08-25T09:40:24.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.843+0000 7f131e63f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 0x7f1318167a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.843+0000 7f131e63f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180b2300 con 0x7f131815c950 2023-08-25T09:40:24.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.843+0000 7f131e63f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13180b16f0 con 0x7f131815cd30 2023-08-25T09:40:24.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.843+0000 7f131e63f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13180b19d0 con 0x7f131806b180 2023-08-25T09:40:24.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.844+0000 7f1317fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f131815c950 0x7f13180b4190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38478/0 (socket says 172.21.15.70:38478) 2023-08-25T09:40:24.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.844+0000 7f1317fff640 1 -- 172.21.15.70:0/2567655901 learned_addr learned my addr 172.21.15.70:0/2567655901 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:24.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.844+0000 7f131cbdc640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 0x7f1318167a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.844+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778990750 0 0) 0x7f13180b2300 con 0x7f131815c950 2023-08-25T09:40:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.844+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1304003520 con 0x7f131815c950 2023-08-25T09:40:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310113892 0 0) 0x7f1304003520 con 0x7f131815c950 2023-08-25T09:40:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 msgr2=0x7f1318167a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 0x7f1318167a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 >> v1:172.21.15.155:6789/0 conn(0x7f131815cd30 legacy=0x7f131806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13180b1cb0 con 0x7f131815c950 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f13080019f0 con 0x7f131815c950 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 >> v1:172.21.15.70:6789/0 conn(0x7f131815c950 legacy=0x7f13180b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f1304003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13040047c0 0x7f1304004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 0x7f1304005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180b19d0 con 0x7f13040039a0 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130400ba60 con 0x7f13040047c0 2023-08-25T09:40:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130400bc20 con 0x7f131806b180 2023-08-25T09:40:24.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f1317fff640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f1304003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.845+0000 7f131cbdc640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 0x7f1304005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.846+0000 7f1317fff640 1 -- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 msgr2=0x7f1304005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:24.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.846+0000 7f1317fff640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f131806b180 0x7f1304005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.846+0000 7f1317fff640 1 -- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13040047c0 msgr2=0x7f1304004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:24.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.846+0000 7f1317fff640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f13040047c0 0x7f1304004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.846+0000 7f1317fff640 1 -- 172.21.15.70:0/2567655901 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1304003520 con 0x7f13040039a0 2023-08-25T09:40:24.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.847+0000 7f1317fff640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f1304003dc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f130800a800 tx=0x7f13040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350ae37fe790fcb5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:24.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.847+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080072d0 con 0x7f13040039a0 2023-08-25T09:40:24.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.847+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130802f070 con 0x7f13040039a0 2023-08-25T09:40:24.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.847+0000 7f1316ffd640 1 -- 172.21.15.70:0/2567655901 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308007670 con 0x7f13040039a0 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131e63f640 1 -- 172.21.15.70:0/2567655901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 msgr2=0x7f1304003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131e63f640 1 --2- 172.21.15.70:0/2567655901 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f1304003dc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f130800a800 tx=0x7f13040057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131cbdc640 1 -- 172.21.15.70:0/2567655901 reap_dead start 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131e63f640 1 -- 172.21.15.70:0/2567655901 shutdown_connections 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131e63f640 1 -- 172.21.15.70:0/2567655901 >> 172.21.15.70:0/2567655901 conn(0x7f1318069c70 msgr2=0x7f131806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131e63f640 1 -- 172.21.15.70:0/2567655901 shutdown_connections 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.848+0000 7f131e63f640 1 -- 172.21.15.70:0/2567655901 wait complete. 2023-08-25T09:40:24.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.850+0000 7f131e63f640 1 Processor -- start 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.850+0000 7f131e63f640 1 -- start start 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131e63f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f131815f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131e63f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1318154ad0 0x7f13181e6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131e63f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318144c80 0x7f1318145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131e63f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1318145670 con 0x7f13040039a0 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131e63f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131815f680 con 0x7f1318144c80 2023-08-25T09:40:24.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131e63f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1318153a30 con 0x7f1318154ad0 2023-08-25T09:40:24.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f131815f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f131815f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43920/0 (socket says 172.21.15.70:43920) 2023-08-25T09:40:24.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 -- 172.21.15.70:0/106138943 learned_addr learned my addr 172.21.15.70:0/106138943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:24.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131cbdc640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318144c80 0x7f1318145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 -- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1318154ad0 msgr2=0x7f13181e6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1318154ad0 0x7f13181e6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 -- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318144c80 msgr2=0x7f1318145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:24.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318144c80 0x7f1318145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:24.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f1317fff640 1 -- 172.21.15.70:0/106138943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318153d10 con 0x7f13040039a0 2023-08-25T09:40:24.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.851+0000 7f131cbdc640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318144c80 0x7f1318145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:24.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.852+0000 7f1317fff640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f131815f140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f130802cb20 tx=0x7f1308001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.852+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13080070d0 con 0x7f13040039a0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.852+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308007250 con 0x7f13040039a0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.852+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308007600 con 0x7f13040039a0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.852+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1318153ff0 con 0x7f13040039a0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.852+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13181545b0 con 0x7f13040039a0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.854+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1318154770 con 0x7f13040039a0 2023-08-25T09:40:24.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.854+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1308031030 con 0x7f13040039a0 2023-08-25T09:40:24.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.855+0000 7f1314ff9640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f12f806a080 0x7f12f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:24.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.856+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f13080a96b0 con 0x7f13040039a0 2023-08-25T09:40:24.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.856+0000 7f13177fe640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f12f806a080 0x7f12f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:24.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.856+0000 7f13177fe640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f12f806a080 0x7f12f806c540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13180b14b0 tx=0x7f1300007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:24.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:24.860+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f13080750f0 con 0x7f13040039a0 2023-08-25T09:40:25.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.048+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f131815c050 con 0x7f13040039a0 2023-08-25T09:40:25.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.412+0000 7f1314ff9640 1 -- 172.21.15.70:0/106138943 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1308078710 con 0x7f13040039a0 2023-08-25T09:40:25.414 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:40:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.415+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f12f806a080 msgr2=0x7f12f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.415+0000 7f131e63f640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f12f806a080 0x7f12f806c540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f13180b14b0 tx=0x7f1300007990 comp rx=0 tx=0).stop 2023-08-25T09:40:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 msgr2=0x7f131815f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f131815f140 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f130802cb20 tx=0x7f1308001d80 comp rx=0 tx=0).stop 2023-08-25T09:40:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 shutdown_connections 2023-08-25T09:40:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f12f806a080 0x7f12f806c540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1318144c80 0x7f1318145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1318154ad0 0x7f13181e6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 --2- 172.21.15.70:0/106138943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f13040039a0 0x7f131815f140 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 >> 172.21.15.70:0/106138943 conn(0x7f1318069c70 msgr2=0x7f13180b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:25.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 shutdown_connections 2023-08-25T09:40:25.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.416+0000 7f131e63f640 1 -- 172.21.15.70:0/106138943 wait complete. 2023-08-25T09:40:25.440 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:40:25.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.651+0000 7f4cfdd24640 1 Processor -- start 2023-08-25T09:40:25.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.651+0000 7f4cfdd24640 1 -- start start 2023-08-25T09:40:25.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.652+0000 7f4cfdd24640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4cf80c2110 0x7f4cf8069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4cfdd24640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80c2550 con 0x7f4cf8069d90 2023-08-25T09:40:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4cfdd24640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf80ce330 con 0x7f4cf815ae50 2023-08-25T09:40:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4cfdd24640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf806a200 con 0x7f4cf80c2110 2023-08-25T09:40:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4cf7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4cf8069d90 0x7f4cf80ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38494/0 (socket says 172.21.15.70:38494) 2023-08-25T09:40:25.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4cf7fff640 1 -- 172.21.15.70:0/859151235 learned_addr learned my addr 172.21.15.70:0/859151235 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4ceffff640 1 --2- 172.21.15.70:0/859151235 >> v2:172.21.15.155:3300/0 conn(0x7f4cf80c2110 0x7f4cf8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.653+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3572617533 0 0) 0x7f4cf80c2550 con 0x7f4cf8069d90 2023-08-25T09:40:25.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ce4003520 con 0x7f4cf8069d90 2023-08-25T09:40:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4194154266 0 0) 0x7f4ce4003520 con 0x7f4cf8069d90 2023-08-25T09:40:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 >> v2:172.21.15.155:3300/0 conn(0x7f4cf80c2110 msgr2=0x7f4cf8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:25.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 --2- 172.21.15.70:0/859151235 >> v2:172.21.15.155:3300/0 conn(0x7f4cf80c2110 0x7f4cf8069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 >> v1:172.21.15.155:6789/0 conn(0x7f4cf815ae50 legacy=0x7f4cf80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf80cc5f0 con 0x7f4cf8069d90 2023-08-25T09:40:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4cf00019f0 con 0x7f4cf8069d90 2023-08-25T09:40:25.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 >> v1:172.21.15.70:6789/0 conn(0x7f4cf8069d90 legacy=0x7f4cf80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce4003970 0x7f4ce4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ce40047c0 0x7f4ce4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4005180 0x7f4ce400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf806a200 con 0x7f4ce40047c0 2023-08-25T09:40:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce400c000 con 0x7f4ce4005180 2023-08-25T09:40:25.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce400c1c0 con 0x7f4ce4003970 2023-08-25T09:40:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.654+0000 7f4ceffff640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ce40047c0 0x7f4ce4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4cf7fff640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4005180 0x7f4ce400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4ceffff640 1 -- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce4003970 msgr2=0x7f4ce4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4ceffff640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce4003970 0x7f4ce4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4ceffff640 1 -- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4005180 msgr2=0x7f4ce400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4ceffff640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ce4005180 0x7f4ce400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4ceffff640 1 -- 172.21.15.70:0/859151235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce4003520 con 0x7f4ce40047c0 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.655+0000 7f4cf7fff640 1 -- 172.21.15.70:0/859151235 reap_dead start 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.656+0000 7f4ceffff640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ce40047c0 0x7f4ce4004c40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009a90 tx=0x7f4ce40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb98244f94934356 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.656+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce002c680 con 0x7f4ce40047c0 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.656+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce002c7e0 con 0x7f4ce40047c0 2023-08-25T09:40:25.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.656+0000 7f4cf6ffd640 1 -- 172.21.15.70:0/859151235 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce002cb70 con 0x7f4ce40047c0 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 -- 172.21.15.70:0/859151235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ce40047c0 msgr2=0x7f4ce4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ce40047c0 0x7f4ce4004c40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009a90 tx=0x7f4ce40057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 -- 172.21.15.70:0/859151235 shutdown_connections 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/859151235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ce40047c0 0x7f4ce4004c40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 -- 172.21.15.70:0/859151235 >> 172.21.15.70:0/859151235 conn(0x7f4cf80cb0d0 msgr2=0x7f4cf80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 -- 172.21.15.70:0/859151235 shutdown_connections 2023-08-25T09:40:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.657+0000 7f4cfdd24640 1 -- 172.21.15.70:0/859151235 wait complete. 2023-08-25T09:40:25.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.659+0000 7f4cfdd24640 1 Processor -- start 2023-08-25T09:40:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.659+0000 7f4cfdd24640 1 -- start start 2023-08-25T09:40:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cfdd24640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce40047c0 0x7f4cf8156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cfdd24640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf80ac880 0x7f4cf80acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cfdd24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 0x7f4cf815c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cfdd24640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80ad240 con 0x7f4cf815bc40 2023-08-25T09:40:25.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cfdd24640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf80c5d70 con 0x7f4cf80ac880 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cfdd24640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf815c970 con 0x7f4ce40047c0 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cf7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 0x7f4cf815c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cf7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 0x7f4cf815c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43934/0 (socket says 172.21.15.70:43934) 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.660+0000 7f4cf7fff640 1 -- 172.21.15.70:0/864052676 learned_addr learned my addr 172.21.15.70:0/864052676 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4ceffff640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf80ac880 0x7f4cf80acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf7fff640 1 -- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce40047c0 msgr2=0x7f4cf8156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf77fe640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce40047c0 0x7f4cf8156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf7fff640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce40047c0 0x7f4cf8156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf7fff640 1 -- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf80ac880 msgr2=0x7f4cf80acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf7fff640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf80ac880 0x7f4cf80acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf7fff640 1 -- 172.21.15.70:0/864052676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf815cc50 con 0x7f4cf815bc40 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4ceffff640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf80ac880 0x7f4cf80acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf7fff640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 0x7f4cf815c0c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8071c40 tx=0x7f4cf000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0004d60 con 0x7f4cf815bc40 2023-08-25T09:40:25.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf000bb20 con 0x7f4cf815bc40 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000bed0 con 0x7f4cf815bc40 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.661+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf815cf30 con 0x7f4cf815bc40 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.662+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf8157250 con 0x7f4cf815bc40 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.663+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf8066fb0 con 0x7f4cf815bc40 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.664+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4cf0007070 con 0x7f4cf815bc40 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.665+0000 7f4cf4ff9640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cd806a050 0x7f4cd806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:25.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.665+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f4cf0088860 con 0x7f4cf815bc40 2023-08-25T09:40:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.665+0000 7f4cf77fe640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cd806a050 0x7f4cd806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.666+0000 7f4cf77fe640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cd806a050 0x7f4cd806c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8069270 tx=0x7f4ce8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:25.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.670+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4cf0054000 con 0x7f4cf815bc40 2023-08-25T09:40:25.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:25.858+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4cf806bcf0 con 0x7f4cf815bc40 2023-08-25T09:40:26.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.431+0000 7f4cf4ff9640 1 -- 172.21.15.70:0/864052676 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4cf0057620 con 0x7f4cf815bc40 2023-08-25T09:40:26.436 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:40:26.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.441+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cd806a050 msgr2=0x7f4cd806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:26.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.441+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cd806a050 0x7f4cd806c510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8069270 tx=0x7f4ce8009040 comp rx=0 tx=0).stop 2023-08-25T09:40:26.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.441+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 msgr2=0x7f4cf815c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:26.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.441+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 0x7f4cf815c0c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8071c40 tx=0x7f4cf000b4a0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 shutdown_connections 2023-08-25T09:40:26.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4cd806a050 0x7f4cd806c510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4cf815bc40 0x7f4cf815c0c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4cf80ac880 0x7f4cf80acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 --2- 172.21.15.70:0/864052676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ce40047c0 0x7f4cf8156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 >> 172.21.15.70:0/864052676 conn(0x7f4cf80cb0d0 msgr2=0x7f4cf8158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:26.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 shutdown_connections 2023-08-25T09:40:26.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.443+0000 7f4cfdd24640 1 -- 172.21.15.70:0/864052676 wait complete. 2023-08-25T09:40:26.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:40:26.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.673+0000 7fd231d39640 1 Processor -- start 2023-08-25T09:40:26.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.673+0000 7fd231d39640 1 -- start start 2023-08-25T09:40:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.674+0000 7fd231d39640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.674+0000 7fd231d39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd22c0cd660 0x7fd22c0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.674+0000 7fd231d39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22c15e780 con 0x7fd22c06f0e0 2023-08-25T09:40:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.674+0000 7fd231d39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22c06f8e0 con 0x7fd22c0cd660 2023-08-25T09:40:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.674+0000 7fd231d39640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c06fbc0 con 0x7fd22c159bf0 2023-08-25T09:40:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd22c0cd660 0x7fd22c0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd22c0cd660 0x7fd22c0cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43944/0 (socket says 172.21.15.70:43944) 2023-08-25T09:40:26.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 -- 172.21.15.70:0/3774274128 learned_addr learned my addr 172.21.15.70:0/3774274128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22b7fe640 1 --2- 172.21.15.70:0/3774274128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 -- 172.21.15.70:0/3774274128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 msgr2=0x7fd22c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 --2- 172.21.15.70:0/3774274128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 -- 172.21.15.70:0/3774274128 >> v1:172.21.15.70:6789/0 conn(0x7fd22c06f0e0 legacy=0x7fd22c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.675+0000 7fd22bfff640 1 -- 172.21.15.70:0/3774274128 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0ce000 con 0x7fd22c0cd660 2023-08-25T09:40:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.676+0000 7fd22bfff640 1 --2- 172.21.15.70:0/3774274128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c0cdac0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd224009870 tx=0x7fd22402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfde6613c4888fc8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd22affd640 1 -- 172.21.15.70:0/3774274128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224030840 con 0x7fd22c0cd660 2023-08-25T09:40:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd22affd640 1 -- 172.21.15.70:0/3774274128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2240309a0 con 0x7fd22c0cd660 2023-08-25T09:40:26.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd22affd640 1 -- 172.21.15.70:0/3774274128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224030d30 con 0x7fd22c0cd660 2023-08-25T09:40:26.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 -- 172.21.15.70:0/3774274128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 msgr2=0x7fd22c0cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:26.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 --2- 172.21.15.70:0/3774274128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c0cdac0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd224009870 tx=0x7fd22402b980 comp rx=0 tx=0).stop 2023-08-25T09:40:26.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 -- 172.21.15.70:0/3774274128 shutdown_connections 2023-08-25T09:40:26.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 --2- 172.21.15.70:0/3774274128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c0cdac0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 --2- 172.21.15.70:0/3774274128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 -- 172.21.15.70:0/3774274128 >> 172.21.15.70:0/3774274128 conn(0x7fd22c0cb0d0 msgr2=0x7fd22c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:26.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.677+0000 7fd231d39640 1 -- 172.21.15.70:0/3774274128 shutdown_connections 2023-08-25T09:40:26.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.678+0000 7fd231d39640 1 -- 172.21.15.70:0/3774274128 wait complete. 2023-08-25T09:40:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.679+0000 7fd231d39640 1 Processor -- start 2023-08-25T09:40:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.679+0000 7fd231d39640 1 -- start start 2023-08-25T09:40:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.680+0000 7fd231d39640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd22c06f0e0 0x7fd22c0c1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.680+0000 7fd231d39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:26.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.680+0000 7fd231d39640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c06b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.680+0000 7fd231d39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0c1e50 con 0x7fd22c0cd660 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.680+0000 7fd231d39640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c1585b0 con 0x7fd22c06f0e0 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.680+0000 7fd231d39640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22c0c5730 con 0x7fd22c159bf0 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c06b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43946/0 (socket says 172.21.15.70:43946) 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 -- 172.21.15.70:0/13284661 learned_addr learned my addr 172.21.15.70:0/13284661 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:26.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd22b7fe640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd22c06f0e0 0x7fd22c0c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd22bfff640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c06b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 -- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 msgr2=0x7fd22c06b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c06b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 -- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd22c06f0e0 msgr2=0x7fd22c0c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:26.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd22c06f0e0 0x7fd22c0c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:26.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd222ffd640 1 -- 172.21.15.70:0/13284661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240094d0 con 0x7fd22c0cd660 2023-08-25T09:40:26.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.681+0000 7fd22b7fe640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd22c06f0e0 0x7fd22c0c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:26.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.682+0000 7fd222ffd640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c06b410 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd21400c870 tx=0x7fd21400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:26.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.682+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214003b70 con 0x7fd22c0cd660 2023-08-25T09:40:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.682+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214003cf0 con 0x7fd22c0cd660 2023-08-25T09:40:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.682+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22c14fbe0 con 0x7fd22c0cd660 2023-08-25T09:40:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.682+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400aa90 con 0x7fd22c0cd660 2023-08-25T09:40:26.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.682+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22c0c59d0 con 0x7fd22c0cd660 2023-08-25T09:40:26.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.683+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22c066fb0 con 0x7fd22c0cd660 2023-08-25T09:40:26.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.684+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd21400abf0 con 0x7fd22c0cd660 2023-08-25T09:40:26.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.685+0000 7fd228ff9640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd20c06a050 0x7fd20c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:26.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.686+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fd214088430 con 0x7fd22c0cd660 2023-08-25T09:40:26.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.686+0000 7fd22b7fe640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd20c06a050 0x7fd20c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:26.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.686+0000 7fd22b7fe640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd20c06a050 0x7fd20c06c510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd21c006fd0 tx=0x7fd21c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:26.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.690+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd214053b80 con 0x7fd22c0cd660 2023-08-25T09:40:26.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:26.915+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd22c157840 con 0x7fd22c0cd660 2023-08-25T09:40:27.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.430+0000 7fd228ff9640 1 -- 172.21.15.70:0/13284661 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd214003e60 con 0x7fd22c0cd660 2023-08-25T09:40:27.432 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 5 and data pool 6 2023-08-25T09:40:27.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.433+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd20c06a050 msgr2=0x7fd20c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.433+0000 7fd231d39640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd20c06a050 0x7fd20c06c510 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd21c006fd0 tx=0x7fd21c008040 comp rx=0 tx=0).stop 2023-08-25T09:40:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.433+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 msgr2=0x7fd22c06b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.433+0000 7fd231d39640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c06b410 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd21400c870 tx=0x7fd21400cd30 comp rx=0 tx=0).stop 2023-08-25T09:40:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.433+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 shutdown_connections 2023-08-25T09:40:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd20c06a050 0x7fd20c06c510 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd22c159bf0 0x7fd22c06b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd22c0cd660 0x7fd22c06b410 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 --2- 172.21.15.70:0/13284661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd22c06f0e0 0x7fd22c0c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 >> 172.21.15.70:0/13284661 conn(0x7fd22c0cb0d0 msgr2=0x7fd22c1564c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 shutdown_connections 2023-08-25T09:40:27.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.434+0000 7fd231d39640 1 -- 172.21.15.70:0/13284661 wait complete. 2023-08-25T09:40:27.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.671+0000 7fa22e737640 1 Processor -- start 2023-08-25T09:40:27.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.671+0000 7fa22e737640 1 -- start start 2023-08-25T09:40:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22e737640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa228159840 0x7fa2280b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22e737640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa2280cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22e737640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2280b4450 con 0x7fa22815d3d0 2023-08-25T09:40:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22e737640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2280c8b60 con 0x7fa228159840 2023-08-25T09:40:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22e737640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815d890 con 0x7fa2280cadb0 2023-08-25T09:40:27.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22d735640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa228159840 0x7fa2280b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22d735640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa228159840 0x7fa2280b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43962/0 (socket says 172.21.15.70:43962) 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22d735640 1 -- 172.21.15.70:0/94578797 learned_addr learned my addr 172.21.15.70:0/94578797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.673+0000 7fa22df36640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa2280cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.674+0000 7fa22d735640 1 -- 172.21.15.70:0/94578797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 msgr2=0x7fa2280cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.674+0000 7fa22d735640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa2280cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.674+0000 7fa22d735640 1 -- 172.21.15.70:0/94578797 >> v1:172.21.15.70:6789/0 conn(0x7fa22815d3d0 legacy=0x7fa2280b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.674+0000 7fa22d735640 1 -- 172.21.15.70:0/94578797 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22815db70 con 0x7fa228159840 2023-08-25T09:40:27.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.674+0000 7fa22df36640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa2280cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.675+0000 7fa22d735640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa2280b2390 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa224009870 tx=0x7fa22402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2bdfbb562cf682 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.675+0000 7fa21ffff640 1 -- 172.21.15.70:0/94578797 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224030840 con 0x7fa228159840 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.675+0000 7fa21ffff640 1 -- 172.21.15.70:0/94578797 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2240309a0 con 0x7fa228159840 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.675+0000 7fa21ffff640 1 -- 172.21.15.70:0/94578797 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224030d30 con 0x7fa228159840 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 -- 172.21.15.70:0/94578797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 msgr2=0x7fa2280b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa2280b2390 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa224009870 tx=0x7fa22402b980 comp rx=0 tx=0).stop 2023-08-25T09:40:27.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 -- 172.21.15.70:0/94578797 shutdown_connections 2023-08-25T09:40:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa2280cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 --2- 172.21.15.70:0/94578797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa2280b2390 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 -- 172.21.15.70:0/94578797 >> 172.21.15.70:0/94578797 conn(0x7fa2280affd0 msgr2=0x7fa2280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 -- 172.21.15.70:0/94578797 shutdown_connections 2023-08-25T09:40:27.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.676+0000 7fa22e737640 1 -- 172.21.15.70:0/94578797 wait complete. 2023-08-25T09:40:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.678+0000 7fa22e737640 1 Processor -- start 2023-08-25T09:40:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.678+0000 7fa22e737640 1 -- start start 2023-08-25T09:40:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22e737640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa22814fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22e737640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa228150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22e737640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa22815d3d0 0x7fa2281508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:27.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22e737640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa228071290 con 0x7fa228159840 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22e737640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280c8040 con 0x7fa22815d3d0 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22e737640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2280c8320 con 0x7fa2280cadb0 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22cf34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa228150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22cf34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa228150380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43974/0 (socket says 172.21.15.70:43974) 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22cf34640 1 -- 172.21.15.70:0/1769420063 learned_addr learned my addr 172.21.15.70:0/1769420063 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22d735640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa22814fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22cf34640 1 -- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 msgr2=0x7fa22814fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22cf34640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa22814fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22df36640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa22815d3d0 0x7fa2281508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.679+0000 7fa22cf34640 1 -- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa22815d3d0 msgr2=0x7fa2281508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa22cf34640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa22815d3d0 0x7fa2281508c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa22cf34640 1 -- 172.21.15.70:0/1769420063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2240094d0 con 0x7fa228159840 2023-08-25T09:40:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa22d735640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa22814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:27.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa22cf34640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa228150380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c390 tx=0x7fa21800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180052d0 con 0x7fa228159840 2023-08-25T09:40:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa218005430 con 0x7fa228159840 2023-08-25T09:40:27.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa218003830 con 0x7fa228159840 2023-08-25T09:40:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2280c77e0 con 0x7fa228159840 2023-08-25T09:40:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.680+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2280cf210 con 0x7fa228159840 2023-08-25T09:40:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.682+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa228066fb0 con 0x7fa228159840 2023-08-25T09:40:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.682+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa218003990 con 0x7fa228159840 2023-08-25T09:40:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.683+0000 7fa21dffb640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa20c06a080 0x7fa20c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.683+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa218087ab0 con 0x7fa228159840 2023-08-25T09:40:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.684+0000 7fa22d735640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa20c06a080 0x7fa20c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:27.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.684+0000 7fa22d735640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa20c06a080 0x7fa20c06c540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa224036000 tx=0x7fa22403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:27.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.688+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa2180531a0 con 0x7fa228159840 2023-08-25T09:40:27.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.876+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2280ad9f0 con 0x7fa228159840 2023-08-25T09:40:27.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.879+0000 7fa21dffb640 1 -- 172.21.15.70:0/1769420063 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18378 (secure 0 0 0) 0x7fa2180567c0 con 0x7fa228159840 2023-08-25T09:40:27.882 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:27.882 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":34,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:26.916905+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:27.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa20c06a080 msgr2=0x7fa20c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa20c06a080 0x7fa20c06c540 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa224036000 tx=0x7fa22403b040 comp rx=0 tx=0).stop 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 msgr2=0x7fa228150380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa228150380 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa21800c390 tx=0x7fa21800c850 comp rx=0 tx=0).stop 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 shutdown_connections 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa20c06a080 0x7fa20c06c540 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa22815d3d0 0x7fa2281508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa228159840 0x7fa228150380 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 --2- 172.21.15.70:0/1769420063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2280cadb0 0x7fa22814fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:27.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 >> 172.21.15.70:0/1769420063 conn(0x7fa2280affd0 msgr2=0x7fa2280ace60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:27.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 shutdown_connections 2023-08-25T09:40:27.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:27.882+0000 7fa22e737640 1 -- 172.21.15.70:0/1769420063 wait complete. 2023-08-25T09:40:27.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.116+0000 7f3388834640 1 Processor -- start 2023-08-25T09:40:28.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.116+0000 7f3388834640 1 -- start start 2023-08-25T09:40:28.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.118+0000 7f3388834640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 0x7f338406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.118+0000 7f3388834640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33840b1b60 con 0x7f338415ec10 2023-08-25T09:40:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.118+0000 7f3388834640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338406b7c0 con 0x7f338406b2b0 2023-08-25T09:40:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.118+0000 7f3388834640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338406bd80 con 0x7f338415e830 2023-08-25T09:40:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f3382d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f338415ec10 0x7f33840b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38512/0 (socket says 172.21.15.70:38512) 2023-08-25T09:40:28.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f3382d9d640 1 -- 172.21.15.70:0/1512542897 learned_addr learned my addr 172.21.15.70:0/1512542897 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338259c640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 0x7f338406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582535358 0 0) 0x7f33840b1b60 con 0x7f338415ec10 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3370003520 con 0x7f338415ec10 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4083288177 0 0) 0x7f3370003520 con 0x7f338415ec10 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 msgr2=0x7f338406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 0x7f338406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 >> v1:172.21.15.155:6789/0 conn(0x7f338406b2b0 legacy=0x7f338415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338406baa0 con 0x7f338415ec10 2023-08-25T09:40:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.119+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f337c001670 con 0x7f338415ec10 2023-08-25T09:40:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 >> v1:172.21.15.70:6789/0 conn(0x7f338415ec10 legacy=0x7f33840b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33700039a0 0x7f3370003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3370004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 0x7f3370005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338406bd80 con 0x7f33700047c0 2023-08-25T09:40:28.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337000ba30 con 0x7f33700039a0 2023-08-25T09:40:28.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337000bbf0 con 0x7f338415e830 2023-08-25T09:40:28.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3381d9b640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3370004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3382d9d640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 0x7f3370005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f338259c640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33700039a0 0x7f3370003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3381d9b640 1 -- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 msgr2=0x7f3370005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3381d9b640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f338415e830 0x7f3370005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3381d9b640 1 -- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33700039a0 msgr2=0x7f3370003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3381d9b640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33700039a0 0x7f3370003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.120+0000 7f3381d9b640 1 -- 172.21.15.70:0/1512542897 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370003520 con 0x7f33700047c0 2023-08-25T09:40:28.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f3381d9b640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3370004c40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f338406b6f0 tx=0x7f33700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ce51ee23922664 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:28.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c0052d0 con 0x7f33700047c0 2023-08-25T09:40:28.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c02e050 con 0x7f33700047c0 2023-08-25T09:40:28.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f338159a640 1 -- 172.21.15.70:0/1512542897 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c0035f0 con 0x7f33700047c0 2023-08-25T09:40:28.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f3388834640 1 -- 172.21.15.70:0/1512542897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 msgr2=0x7f3370004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f3388834640 1 --2- 172.21.15.70:0/1512542897 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3370004c40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f338406b6f0 tx=0x7f33700057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.122+0000 7f3382d9d640 1 -- 172.21.15.70:0/1512542897 reap_dead start 2023-08-25T09:40:28.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.123+0000 7f3388834640 1 -- 172.21.15.70:0/1512542897 shutdown_connections 2023-08-25T09:40:28.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.123+0000 7f3388834640 1 -- 172.21.15.70:0/1512542897 >> 172.21.15.70:0/1512542897 conn(0x7f3384069850 msgr2=0x7f3384069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:28.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.123+0000 7f3388834640 1 -- 172.21.15.70:0/1512542897 shutdown_connections 2023-08-25T09:40:28.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.123+0000 7f3388834640 1 -- 172.21.15.70:0/1512542897 wait complete. 2023-08-25T09:40:28.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.124+0000 7f3388834640 1 Processor -- start 2023-08-25T09:40:28.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 -- start start 2023-08-25T09:40:28.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3384147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3384153510 0x7f3384147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3384157610 0x7f33840c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3384158390 con 0x7f33700047c0 2023-08-25T09:40:28.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3384147650 con 0x7f3384153510 2023-08-25T09:40:28.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.125+0000 7f3388834640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33841539f0 con 0x7f3384157610 2023-08-25T09:40:28.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3384147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3384147930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43994/0 (socket says 172.21.15.70:43994) 2023-08-25T09:40:28.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 -- 172.21.15.70:0/3796750461 learned_addr learned my addr 172.21.15.70:0/3796750461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 -- 172.21.15.70:0/3796750461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3384157610 msgr2=0x7f33840c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3384157610 0x7f33840c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 -- 172.21.15.70:0/3796750461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3384153510 msgr2=0x7f3384147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3384153510 0x7f3384147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 -- 172.21.15.70:0/3796750461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338415ca20 con 0x7f33700047c0 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.126+0000 7f338259c640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3384147930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3374007f60 tx=0x7f3374007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:28.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.127+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337400e030 con 0x7f33700047c0 2023-08-25T09:40:28.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.127+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3374016470 con 0x7f33700047c0 2023-08-25T09:40:28.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.127+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3374016820 con 0x7f33700047c0 2023-08-25T09:40:28.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.127+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3384153cd0 con 0x7f33700047c0 2023-08-25T09:40:28.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.127+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3384153fb0 con 0x7f33700047c0 2023-08-25T09:40:28.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.128+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338404e420 con 0x7f33700047c0 2023-08-25T09:40:28.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.129+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3374016980 con 0x7f33700047c0 2023-08-25T09:40:28.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.130+0000 7f337affd640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a080 0x7f336406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.130+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f337400f070 con 0x7f33700047c0 2023-08-25T09:40:28.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.130+0000 7f3381d9b640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a080 0x7f336406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.131+0000 7f3381d9b640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a080 0x7f336406c540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f336c033000 tx=0x7f336c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:28.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.134+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3374054be0 con 0x7f33700047c0 2023-08-25T09:40:28.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.329+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f338406c620 con 0x7f33700047c0 2023-08-25T09:40:28.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.331+0000 7f337affd640 1 -- 172.21.15.70:0/3796750461 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18365 (secure 0 0 0) 0x7f3374058200 con 0x7f33700047c0 2023-08-25T09:40:28.333 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:28.334 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":35,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:28.214584+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a080 msgr2=0x7f336406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a080 0x7f336406c540 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f336c033000 tx=0x7f336c02f000 comp rx=0 tx=0).stop 2023-08-25T09:40:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 msgr2=0x7f3384147930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3384147930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3374007f60 tx=0x7f3374007f90 comp rx=0 tx=0).stop 2023-08-25T09:40:28.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 shutdown_connections 2023-08-25T09:40:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a080 0x7f336406c540 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3384157610 0x7f33840c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3384153510 0x7f3384147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 --2- 172.21.15.70:0/3796750461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33700047c0 0x7f3384147930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 >> 172.21.15.70:0/3796750461 conn(0x7f3384069850 msgr2=0x7f338415b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:28.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 shutdown_connections 2023-08-25T09:40:28.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.334+0000 7f3388834640 1 -- 172.21.15.70:0/3796750461 wait complete. 2023-08-25T09:40:28.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:40:28.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.577+0000 7f685b14e640 1 Processor -- start 2023-08-25T09:40:28.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.577+0000 7f685b14e640 1 -- start start 2023-08-25T09:40:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f685b14e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 0x7f6854070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f685b14e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f68540cd660 0x7f68540cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f685b14e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68540ce000 con 0x7f6854159bf0 2023-08-25T09:40:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f685b14e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685406f8e0 con 0x7f68540cd660 2023-08-25T09:40:28.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f685b14e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685406fbc0 con 0x7f685415e240 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f6858eea640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6854159bf0 0x7f685406f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38514/0 (socket says 172.21.15.70:38514) 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f6858eea640 1 -- 172.21.15.70:0/2766149170 learned_addr learned my addr 172.21.15.70:0/2766149170 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f68596eb640 1 --2- 172.21.15.70:0/2766149170 >> v2:172.21.15.70:3300/0 conn(0x7f68540cd660 0x7f68540cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f68596eb640 1 -- 172.21.15.70:0/2766149170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 msgr2=0x7f6854070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f68596eb640 1 --2- 172.21.15.70:0/2766149170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 0x7f6854070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f68596eb640 1 -- 172.21.15.70:0/2766149170 >> v1:172.21.15.70:6789/0 conn(0x7f6854159bf0 legacy=0x7f685406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.579+0000 7f68596eb640 1 -- 172.21.15.70:0/2766149170 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685406fea0 con 0x7f68540cd660 2023-08-25T09:40:28.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.581+0000 7f68596eb640 1 --2- 172.21.15.70:0/2766149170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68540cd660 0x7f68540cdac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f684c009d90 tx=0x7f684c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e299661c02534b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:28.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.581+0000 7f68537fe640 1 -- 172.21.15.70:0/2766149170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c02cc60 con 0x7f68540cd660 2023-08-25T09:40:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.581+0000 7f68537fe640 1 -- 172.21.15.70:0/2766149170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c02cdc0 con 0x7f68540cd660 2023-08-25T09:40:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.581+0000 7f68537fe640 1 -- 172.21.15.70:0/2766149170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c034660 con 0x7f68540cd660 2023-08-25T09:40:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.581+0000 7f685b14e640 1 -- 172.21.15.70:0/2766149170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68540cd660 msgr2=0x7f68540cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.581+0000 7f685b14e640 1 --2- 172.21.15.70:0/2766149170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68540cd660 0x7f68540cdac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f684c009d90 tx=0x7f684c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.582+0000 7f685b14e640 1 -- 172.21.15.70:0/2766149170 shutdown_connections 2023-08-25T09:40:28.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.582+0000 7f685b14e640 1 --2- 172.21.15.70:0/2766149170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68540cd660 0x7f68540cdac0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.582+0000 7f685b14e640 1 --2- 172.21.15.70:0/2766149170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 0x7f6854070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.582+0000 7f685b14e640 1 -- 172.21.15.70:0/2766149170 >> 172.21.15.70:0/2766149170 conn(0x7f68540cb0d0 msgr2=0x7f68540cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.582+0000 7f685b14e640 1 -- 172.21.15.70:0/2766149170 shutdown_connections 2023-08-25T09:40:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.582+0000 7f685b14e640 1 -- 172.21.15.70:0/2766149170 wait complete. 2023-08-25T09:40:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.584+0000 7f685b14e640 1 Processor -- start 2023-08-25T09:40:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.584+0000 7f685b14e640 1 -- start start 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f685b14e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68540cd660 0x7f68540af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f685b14e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 0x7f68540ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f685b14e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 0x7f68540acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f685b14e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540afe20 con 0x7f6854159bf0 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f685b14e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540afb40 con 0x7f68540cd660 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f685b14e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68540adf20 con 0x7f685415e240 2023-08-25T09:40:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 0x7f68540ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 0x7f68540ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44018/0 (socket says 172.21.15.70:44018) 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 -- 172.21.15.70:0/1128964799 learned_addr learned my addr 172.21.15.70:0/1128964799 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6858eea640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68540cd660 0x7f68540af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 -- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 msgr2=0x7f68540acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 0x7f68540acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 -- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68540cd660 msgr2=0x7f68540af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68540cd660 0x7f68540af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.585+0000 7f6853fff640 1 -- 172.21.15.70:0/1128964799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009a70 con 0x7f6854159bf0 2023-08-25T09:40:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.586+0000 7f6853fff640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 0x7f68540ac880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f684000b410 tx=0x7f684000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.586+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840011830 con 0x7f6854159bf0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.586+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400119b0 con 0x7f6854159bf0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.586+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840011d60 con 0x7f6854159bf0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.586+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68540ae200 con 0x7f6854159bf0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.586+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68540c8330 con 0x7f6854159bf0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.587+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68540ae720 con 0x7f6854159bf0 2023-08-25T09:40:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.588+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f684001a400 con 0x7f6854159bf0 2023-08-25T09:40:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.589+0000 7f68517fa640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f683806a050 0x7f683806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.589+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f68400886d0 con 0x7f6854159bf0 2023-08-25T09:40:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.589+0000 7f6858eea640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f683806a050 0x7f683806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.590+0000 7f6858eea640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f683806a050 0x7f683806c510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6848005880 tx=0x7f6848009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.593+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f68540ae720 con 0x7f6854159bf0 2023-08-25T09:40:28.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:28.822+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f68540ae720 con 0x7f6854159bf0 2023-08-25T09:40:29.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.461+0000 7f68517fa640 1 -- 172.21.15.70:0/1128964799 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f68540ae720 con 0x7f6854159bf0 2023-08-25T09:40:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.464+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f683806a050 msgr2=0x7f683806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:29.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.464+0000 7f685b14e640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f683806a050 0x7f683806c510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6848005880 tx=0x7f6848009040 comp rx=0 tx=0).stop 2023-08-25T09:40:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.464+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 msgr2=0x7f68540ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.464+0000 7f685b14e640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 0x7f68540ac880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f684000b410 tx=0x7f684000b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 shutdown_connections 2023-08-25T09:40:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f683806a050 0x7f683806c510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f685415e240 0x7f68540acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6854159bf0 0x7f68540ac880 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 --2- 172.21.15.70:0/1128964799 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f68540cd660 0x7f68540af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 >> 172.21.15.70:0/1128964799 conn(0x7f68540cb0d0 msgr2=0x7f68540b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 shutdown_connections 2023-08-25T09:40:29.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.465+0000 7f685b14e640 1 -- 172.21.15.70:0/1128964799 wait complete. 2023-08-25T09:40:29.488 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:40:29.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:40:29.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.707+0000 7fa8edf29640 1 Processor -- start 2023-08-25T09:40:29.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.707+0000 7fa8edf29640 1 -- start start 2023-08-25T09:40:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.708+0000 7fa8edf29640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e815dd90 0x7fa8e80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.708+0000 7fa8edf29640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8e80b0300 0x7fa8e80b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.708+0000 7fa8edf29640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e815e1f0 con 0x7fa8e8159650 2023-08-25T09:40:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.708+0000 7fa8edf29640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80cd650 con 0x7fa8e80b0300 2023-08-25T09:40:29.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.708+0000 7fa8edf29640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80b0cd0 con 0x7fa8e815dd90 2023-08-25T09:40:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.709+0000 7fa8ed728640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8e80b0300 0x7fa8e80b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.709+0000 7fa8e7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e815dd90 0x7fa8e80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:29.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.709+0000 7fa8ed728640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8e80b0300 0x7fa8e80b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:36196/0 (socket says 172.21.15.70:36196) 2023-08-25T09:40:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.709+0000 7fa8ed728640 1 -- 172.21.15.70:0/2257629735 learned_addr learned my addr 172.21.15.70:0/2257629735 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.710+0000 7fa8ed728640 1 -- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e815dd90 msgr2=0x7fa8e80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.710+0000 7fa8ed728640 1 --2- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e815dd90 0x7fa8e80ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.710+0000 7fa8ed728640 1 -- 172.21.15.70:0/2257629735 >> v1:172.21.15.155:6789/0 conn(0x7fa8e8159650 legacy=0x7fa8e80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:29.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.710+0000 7fa8ed728640 1 -- 172.21.15.70:0/2257629735 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80cd420 con 0x7fa8e80b0300 2023-08-25T09:40:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.711+0000 7fa8ed728640 1 --2- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80b0790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009870 tx=0x7fa8e002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569001a336788327 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8e77fe640 1 -- 172.21.15.70:0/2257629735 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0031640 con 0x7fa8e80b0300 2023-08-25T09:40:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8e77fe640 1 -- 172.21.15.70:0/2257629735 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e00317a0 con 0x7fa8e80b0300 2023-08-25T09:40:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8e77fe640 1 -- 172.21.15.70:0/2257629735 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0031b30 con 0x7fa8e80b0300 2023-08-25T09:40:29.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8edf29640 1 -- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 msgr2=0x7fa8e80b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8edf29640 1 --2- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80b0790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009870 tx=0x7fa8e002b8f0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8edf29640 1 -- 172.21.15.70:0/2257629735 shutdown_connections 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8edf29640 1 --2- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80b0790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8edf29640 1 --2- 172.21.15.70:0/2257629735 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e815dd90 0x7fa8e80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.712+0000 7fa8edf29640 1 -- 172.21.15.70:0/2257629735 >> 172.21.15.70:0/2257629735 conn(0x7fa8e80ce810 msgr2=0x7fa8e80cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.713+0000 7fa8edf29640 1 -- 172.21.15.70:0/2257629735 shutdown_connections 2023-08-25T09:40:29.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.713+0000 7fa8edf29640 1 -- 172.21.15.70:0/2257629735 wait complete. 2023-08-25T09:40:29.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.714+0000 7fa8edf29640 1 Processor -- start 2023-08-25T09:40:29.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 -- start start 2023-08-25T09:40:29.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e8159650 0x7fa8e81552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 0x7fa8e80c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e8155810 con 0x7fa8e815dd90 2023-08-25T09:40:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815ba10 con 0x7fa8e80b0300 2023-08-25T09:40:29.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.715+0000 7fa8edf29640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e815bcf0 con 0x7fa8e8159650 2023-08-25T09:40:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 0x7fa8e80c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 0x7fa8e80c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45786/0 (socket says 172.21.15.70:45786) 2023-08-25T09:40:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 -- 172.21.15.70:0/1574723943 learned_addr learned my addr 172.21.15.70:0/1574723943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ecf27640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 -- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e8159650 msgr2=0x7fa8e81552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:29.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e8159650 0x7fa8e81552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 -- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 msgr2=0x7fa8e80c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ed728640 1 -- 172.21.15.70:0/1574723943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e00094d0 con 0x7fa8e815dd90 2023-08-25T09:40:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.716+0000 7fa8ecf27640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.717+0000 7fa8ed728640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 0x7fa8e80c2ad0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009870 tx=0x7fa8e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.717+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e003a930 con 0x7fa8e815dd90 2023-08-25T09:40:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.717+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e003aab0 con 0x7fa8e815dd90 2023-08-25T09:40:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.717+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e003ae60 con 0x7fa8e815dd90 2023-08-25T09:40:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.717+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e815bfd0 con 0x7fa8e815dd90 2023-08-25T09:40:29.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.717+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e815c590 con 0x7fa8e815dd90 2023-08-25T09:40:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.718+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e815c780 con 0x7fa8e815dd90 2023-08-25T09:40:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.719+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8e002f050 con 0x7fa8e815dd90 2023-08-25T09:40:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.720+0000 7fa8e57fa640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8c806a080 0x7fa8c806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:29.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.721+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa8e00aa4e0 con 0x7fa8e815dd90 2023-08-25T09:40:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.721+0000 7fa8ecf27640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8c806a080 0x7fa8c806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:29.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.722+0000 7fa8ecf27640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8c806a080 0x7fa8c806c540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8007fd0 tx=0x7fa8d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:29.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.725+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa8e0075aa0 con 0x7fa8e815dd90 2023-08-25T09:40:29.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:29.953+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa8e8061460 con 0x7fa8e815dd90 2023-08-25T09:40:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.475+0000 7fa8e57fa640 1 -- 172.21.15.70:0/1574723943 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa8e00350c0 con 0x7fa8e815dd90 2023-08-25T09:40:30.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8c806a080 msgr2=0x7fa8c806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8c806a080 0x7fa8c806c540 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8007fd0 tx=0x7fa8d8009040 comp rx=0 tx=0).stop 2023-08-25T09:40:30.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 msgr2=0x7fa8e80c2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 0x7fa8e80c2ad0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009870 tx=0x7fa8e00046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 shutdown_connections 2023-08-25T09:40:30.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8c806a080 0x7fa8c806c540 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8e815dd90 0x7fa8e80c2ad0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8e8159650 0x7fa8e81552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 --2- 172.21.15.70:0/1574723943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8e80b0300 0x7fa8e80c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 >> 172.21.15.70:0/1574723943 conn(0x7fa8e80ce810 msgr2=0x7fa8e80ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:30.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 shutdown_connections 2023-08-25T09:40:30.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.479+0000 7fa8edf29640 1 -- 172.21.15.70:0/1574723943 wait complete. 2023-08-25T09:40:30.502 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:40:30.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:30.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.716+0000 7f5307c8e640 1 Processor -- start 2023-08-25T09:40:30.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.716+0000 7f5307c8e640 1 -- start start 2023-08-25T09:40:30.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.717+0000 7f5307c8e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53000b1ff0 0x7f53000b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.717+0000 7f5307c8e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53000b1b60 con 0x7f530015ec10 2023-08-25T09:40:30.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.717+0000 7f5307c8e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530006b7c0 con 0x7f530015e830 2023-08-25T09:40:30.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.717+0000 7f5307c8e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530006bd80 con 0x7f53000b1ff0 2023-08-25T09:40:30.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.718+0000 7f5305229640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f530015ec10 0x7f530015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38288/0 (socket says 172.21.15.70:38288) 2023-08-25T09:40:30.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.718+0000 7f5305229640 1 -- 172.21.15.70:0/4261453238 learned_addr learned my addr 172.21.15.70:0/4261453238 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:30.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.718+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758250492 0 0) 0x7f53000b1b60 con 0x7f530015ec10 2023-08-25T09:40:30.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.718+0000 7f530622b640 1 --2- 172.21.15.70:0/4261453238 >> v2:172.21.15.155:3300/0 conn(0x7f53000b1ff0 0x7f53000b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.718+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e8003520 con 0x7f530015ec10 2023-08-25T09:40:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3519279119 0 0) 0x7f52e8003520 con 0x7f530015ec10 2023-08-25T09:40:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 >> v2:172.21.15.155:3300/0 conn(0x7f53000b1ff0 msgr2=0x7f53000b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 --2- 172.21.15.70:0/4261453238 >> v2:172.21.15.155:3300/0 conn(0x7f53000b1ff0 0x7f53000b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 >> v1:172.21.15.155:6789/0 conn(0x7f530015e830 legacy=0x7f530006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530006baa0 con 0x7f530015ec10 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f52f4001670 con 0x7f530015ec10 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 >> v1:172.21.15.70:6789/0 conn(0x7f530015ec10 legacy=0x7f530015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e8003970 0x7f52e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e80047c0 0x7f52e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f52e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530006bd80 con 0x7f52e8005180 2023-08-25T09:40:30.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e800c0d0 con 0x7f52e80047c0 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e800c290 con 0x7f52e8003970 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f530622b640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f52e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5305229640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e80047c0 0x7f52e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f530622b640 1 -- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e8003970 msgr2=0x7f52e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.719+0000 7f5305a2a640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e8003970 0x7f52e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f530622b640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e8003970 0x7f52e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f530622b640 1 -- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e80047c0 msgr2=0x7f52e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:30.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f530622b640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e80047c0 0x7f52e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f530622b640 1 -- 172.21.15.70:0/4261453238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8003520 con 0x7f52e8005180 2023-08-25T09:40:30.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f5305229640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52e80047c0 0x7f52e8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:30.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f5305a2a640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52e8003970 0x7f52e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:30.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.720+0000 7f530622b640 1 -- 172.21.15.70:0/4261453238 reap_dead start 2023-08-25T09:40:30.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.721+0000 7f530622b640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f52e800b980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52f800ba30 tx=0x7f52e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f31069adaa0b65 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:30.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.721+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8004020 con 0x7f52e8005180 2023-08-25T09:40:30.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.721+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f80041c0 con 0x7f52e8005180 2023-08-25T09:40:30.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.721+0000 7f5304a28640 1 -- 172.21.15.70:0/4261453238 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8003610 con 0x7f52e8005180 2023-08-25T09:40:30.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.721+0000 7f5307c8e640 1 -- 172.21.15.70:0/4261453238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 msgr2=0x7f52e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.721+0000 7f5307c8e640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f52e800b980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f52f800ba30 tx=0x7f52e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.722+0000 7f5307c8e640 1 -- 172.21.15.70:0/4261453238 shutdown_connections 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.722+0000 7f5307c8e640 1 --2- 172.21.15.70:0/4261453238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f52e800b980 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.722+0000 7f5307c8e640 1 -- 172.21.15.70:0/4261453238 >> 172.21.15.70:0/4261453238 conn(0x7f5300069850 msgr2=0x7f5300069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.722+0000 7f5307c8e640 1 -- 172.21.15.70:0/4261453238 shutdown_connections 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.722+0000 7f5307c8e640 1 -- 172.21.15.70:0/4261453238 wait complete. 2023-08-25T09:40:30.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.723+0000 7f5307c8e640 1 Processor -- start 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.723+0000 7f5307c8e640 1 -- start start 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.724+0000 7f5307c8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f530014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.724+0000 7f5307c8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53000cd860 0x7f530015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.724+0000 7f5307c8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f530015c970 0x7f530014c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.724+0000 7f5307c8e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530015cdf0 con 0x7f52e8005180 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.724+0000 7f5307c8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530014ebd0 con 0x7f53000cd860 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.724+0000 7f5307c8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530014e050 con 0x7f530015c970 2023-08-25T09:40:30.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f530014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f530014f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45800/0 (socket says 172.21.15.70:45800) 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 -- 172.21.15.70:0/1350381446 learned_addr learned my addr 172.21.15.70:0/1350381446 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 -- 172.21.15.70:0/1350381446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f530015c970 msgr2=0x7f530014c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f530015c970 0x7f530014c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 -- 172.21.15.70:0/1350381446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53000cd860 msgr2=0x7f530015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53000cd860 0x7f530015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.725+0000 7f5305a2a640 1 -- 172.21.15.70:0/1350381446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530014e330 con 0x7f52e8005180 2023-08-25T09:40:30.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.726+0000 7f5305a2a640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f530014f580 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52f000d8e0 tx=0x7f52f000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:30.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.726+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0003da0 con 0x7f52e8005180 2023-08-25T09:40:30.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.726+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f000b6b0 con 0x7f52e8005180 2023-08-25T09:40:30.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.726+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f000ba80 con 0x7f52e8005180 2023-08-25T09:40:30.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.726+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530014e610 con 0x7f52e8005180 2023-08-25T09:40:30.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.726+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530014dce0 con 0x7f52e8005180 2023-08-25T09:40:30.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.727+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53001ee000 con 0x7f52e8005180 2023-08-25T09:40:30.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.728+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52f000bbe0 con 0x7f52e8005180 2023-08-25T09:40:30.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.729+0000 7f52fe7fc640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52ec06a050 0x7f52ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:30.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.729+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52f0088850 con 0x7f52e8005180 2023-08-25T09:40:30.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.730+0000 7f5305229640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52ec06a050 0x7f52ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:30.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.730+0000 7f5305229640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52ec06a050 0x7f52ec06c510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52f4001eb0 tx=0x7f52f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:30.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.734+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52f0053fd0 con 0x7f52e8005180 2023-08-25T09:40:30.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.964+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f530004e420 con 0x7f52e8005180 2023-08-25T09:40:30.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.965+0000 7f52fe7fc640 1 -- 172.21.15.70:0/1350381446 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f52f00575f0 con 0x7f52e8005180 2023-08-25T09:40:30.967 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:30.967 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:30.967 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:30.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52ec06a050 msgr2=0x7f52ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52ec06a050 0x7f52ec06c510 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52f4001eb0 tx=0x7f52f4009040 comp rx=0 tx=0).stop 2023-08-25T09:40:30.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 msgr2=0x7f530014f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:30.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f530014f580 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f52f000d8e0 tx=0x7f52f000dda0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 shutdown_connections 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52ec06a050 0x7f52ec06c510 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f530015c970 0x7f530014c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.967+0000 7f5307c8e640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53000cd860 0x7f530015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.968+0000 7f5307c8e640 1 --2- 172.21.15.70:0/1350381446 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52e8005180 0x7f530014f580 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.968+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 >> 172.21.15.70:0/1350381446 conn(0x7f5300069850 msgr2=0x7f53000b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:30.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.968+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 shutdown_connections 2023-08-25T09:40:30.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:30.968+0000 7f5307c8e640 1 -- 172.21.15.70:0/1350381446 wait complete. 2023-08-25T09:40:30.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:31.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.192+0000 7fc995199640 1 Processor -- start 2023-08-25T09:40:31.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.192+0000 7fc995199640 1 -- start start 2023-08-25T09:40:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.194+0000 7fc995199640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc9900709a0 0x7fc9900b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.194+0000 7fc995199640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99006ae10 con 0x7fc99006f0a0 2023-08-25T09:40:31.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.194+0000 7fc995199640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99006b0b0 con 0x7fc990159100 2023-08-25T09:40:31.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.194+0000 7fc995199640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99006ba50 con 0x7fc9900709a0 2023-08-25T09:40:31.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.194+0000 7fc98f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc99006f0a0 0x7fc9901594e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38294/0 (socket says 172.21.15.70:38294) 2023-08-25T09:40:31.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.194+0000 7fc98f59e640 1 -- 172.21.15.70:0/2050251342 learned_addr learned my addr 172.21.15.70:0/2050251342 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98e59c640 1 --2- 172.21.15.70:0/2050251342 >> v2:172.21.15.155:3300/0 conn(0x7fc9900709a0 0x7fc9900b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511123112 0 0) 0x7fc99006ae10 con 0x7fc99006f0a0 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc978003520 con 0x7fc99006f0a0 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1881051445 0 0) 0x7fc978003520 con 0x7fc99006f0a0 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 >> v2:172.21.15.155:3300/0 conn(0x7fc9900709a0 msgr2=0x7fc9900b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 --2- 172.21.15.70:0/2050251342 >> v2:172.21.15.155:3300/0 conn(0x7fc9900709a0 0x7fc9900b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 >> v1:172.21.15.155:6789/0 conn(0x7fc990159100 legacy=0x7fc99006be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99006b390 con 0x7fc99006f0a0 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc988001670 con 0x7fc99006f0a0 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 >> v1:172.21.15.70:6789/0 conn(0x7fc99006f0a0 legacy=0x7fc9901594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc978003970 0x7fc978003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc978004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc978005180 0x7fc97800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99006ba50 con 0x7fc9780047c0 2023-08-25T09:40:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97800c120 con 0x7fc978003970 2023-08-25T09:40:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.195+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97800c2e0 con 0x7fc978005180 2023-08-25T09:40:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98e59c640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc978004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98ed9d640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc978003970 0x7fc978003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98e59c640 1 -- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc978005180 msgr2=0x7fc97800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:31.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98e59c640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc978005180 0x7fc97800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98e59c640 1 -- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc978003970 msgr2=0x7fc978003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98e59c640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc978003970 0x7fc978003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98e59c640 1 -- 172.21.15.70:0/2050251342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978003520 con 0x7fc9780047c0 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.196+0000 7fc98f59e640 1 -- 172.21.15.70:0/2050251342 reap_dead start 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.197+0000 7fc98e59c640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc978004c40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc984009a90 tx=0x7fc9780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2588607ea2cd49dd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98402c680 con 0x7fc9780047c0 2023-08-25T09:40:31.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98402c7e0 con 0x7fc9780047c0 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc98dd9b640 1 -- 172.21.15.70:0/2050251342 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98402cb70 con 0x7fc9780047c0 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc995199640 1 -- 172.21.15.70:0/2050251342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 msgr2=0x7fc978004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc995199640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc978004c40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc984009a90 tx=0x7fc9780057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc995199640 1 -- 172.21.15.70:0/2050251342 shutdown_connections 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc995199640 1 --2- 172.21.15.70:0/2050251342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc978004c40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.198+0000 7fc995199640 1 -- 172.21.15.70:0/2050251342 >> 172.21.15.70:0/2050251342 conn(0x7fc990069850 msgr2=0x7fc990069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:31.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.199+0000 7fc995199640 1 -- 172.21.15.70:0/2050251342 shutdown_connections 2023-08-25T09:40:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.199+0000 7fc995199640 1 -- 172.21.15.70:0/2050251342 wait complete. 2023-08-25T09:40:31.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.200+0000 7fc995199640 1 Processor -- start 2023-08-25T09:40:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.200+0000 7fc995199640 1 -- start start 2023-08-25T09:40:31.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.201+0000 7fc995199640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc990140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.201+0000 7fc995199640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc990143bd0 0x7fc99015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.201+0000 7fc995199640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc99015bff0 0x7fc990159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.201+0000 7fc995199640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b14b0 con 0x7fc9780047c0 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.201+0000 7fc995199640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b1790 con 0x7fc99015bff0 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.201+0000 7fc995199640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900b1a70 con 0x7fc990143bd0 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc990140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc990140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45820/0 (socket says 172.21.15.70:45820) 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 -- 172.21.15.70:0/1258196280 learned_addr learned my addr 172.21.15.70:0/1258196280 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98f59e640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc99015bff0 0x7fc990159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 -- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc990143bd0 msgr2=0x7fc99015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc990143bd0 0x7fc99015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 -- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc99015bff0 msgr2=0x7fc990159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc99015bff0 0x7fc990159c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 -- 172.21.15.70:0/1258196280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900b1d50 con 0x7fc9780047c0 2023-08-25T09:40:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.202+0000 7fc98ed9d640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc990140cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc980000f80 tx=0x7fc98000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.203+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980003b80 con 0x7fc9780047c0 2023-08-25T09:40:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.203+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980003ce0 con 0x7fc9780047c0 2023-08-25T09:40:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.203+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800052f0 con 0x7fc9780047c0 2023-08-25T09:40:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.203+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9900b2030 con 0x7fc9780047c0 2023-08-25T09:40:31.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.203+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc990141210 con 0x7fc9780047c0 2023-08-25T09:40:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.204+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9900b0150 con 0x7fc9780047c0 2023-08-25T09:40:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.205+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc980005450 con 0x7fc9780047c0 2023-08-25T09:40:31.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.206+0000 7fc9777fe640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc97c06a050 0x7fc97c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.206+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9800885e0 con 0x7fc9780047c0 2023-08-25T09:40:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.206+0000 7fc98e59c640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc97c06a050 0x7fc97c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.207+0000 7fc98e59c640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc97c06a050 0x7fc97c06c510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc984036000 tx=0x7fc98402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:31.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.210+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc980053cd0 con 0x7fc9780047c0 2023-08-25T09:40:31.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.402+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc99004e420 con 0x7fc9780047c0 2023-08-25T09:40:31.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.404+0000 7fc9777fe640 1 -- 172.21.15.70:0/1258196280 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18365 (secure 0 0 0) 0x7fc9800572f0 con 0x7fc9780047c0 2023-08-25T09:40:31.407 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:31.407 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":35,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:28.214584+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:31.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc97c06a050 msgr2=0x7fc97c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc97c06a050 0x7fc97c06c510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc984036000 tx=0x7fc98402e000 comp rx=0 tx=0).stop 2023-08-25T09:40:31.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 msgr2=0x7fc990140cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc990140cd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc980000f80 tx=0x7fc98000ec80 comp rx=0 tx=0).stop 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 shutdown_connections 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc97c06a050 0x7fc97c06c510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc99015bff0 0x7fc990159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc990143bd0 0x7fc99015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 --2- 172.21.15.70:0/1258196280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9780047c0 0x7fc990140cd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 >> 172.21.15.70:0/1258196280 conn(0x7fc990069850 msgr2=0x7fc9900ad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:31.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 shutdown_connections 2023-08-25T09:40:31.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.407+0000 7fc995199640 1 -- 172.21.15.70:0/1258196280 wait complete. 2023-08-25T09:40:31.428 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:40:31.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:40:31.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.633+0000 7f6510b4e640 1 Processor -- start 2023-08-25T09:40:31.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.633+0000 7f6510b4e640 1 -- start start 2023-08-25T09:40:31.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.635+0000 7f6510b4e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f650c15ec10 0x7f650c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.635+0000 7f6510b4e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650c0b1b60 con 0x7f650c15c6d0 2023-08-25T09:40:31.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.635+0000 7f6510b4e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650c06b7c0 con 0x7f650c15e830 2023-08-25T09:40:31.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.635+0000 7f6510b4e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650c06bd80 con 0x7f650c15ec10 2023-08-25T09:40:31.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.635+0000 7f650ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f650c15c6d0 0x7f650c0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38308/0 (socket says 172.21.15.70:38308) 2023-08-25T09:40:31.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.635+0000 7f650ad9d640 1 -- 172.21.15.70:0/1610978165 learned_addr learned my addr 172.21.15.70:0/1610978165 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:31.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6503fff640 1 --2- 172.21.15.70:0/1610978165 >> v2:172.21.15.155:3300/0 conn(0x7f650c15ec10 0x7f650c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411162043 0 0) 0x7f650c0b1b60 con 0x7f650c15c6d0 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f8003520 con 0x7f650c15c6d0 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 700008294 0 0) 0x7f64f8003520 con 0x7f650c15c6d0 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 >> v2:172.21.15.155:3300/0 conn(0x7f650c15ec10 msgr2=0x7f650c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 --2- 172.21.15.70:0/1610978165 >> v2:172.21.15.155:3300/0 conn(0x7f650c15ec10 0x7f650c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 >> v1:172.21.15.155:6789/0 conn(0x7f650c15e830 legacy=0x7f650c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c06baa0 con 0x7f650c15c6d0 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f65040019f0 con 0x7f650c15c6d0 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 >> v1:172.21.15.70:6789/0 conn(0x7f650c15c6d0 legacy=0x7f650c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.636+0000 7f6509d9b640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8003970 0x7f64f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f6509d9b640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f80047c0 0x7f64f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f6509d9b640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f8005180 0x7f64f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c06bd80 con 0x7f64f8003970 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f800c120 con 0x7f64f80047c0 2023-08-25T09:40:31.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f800c2e0 con 0x7f64f8005180 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650a59c640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8003970 0x7f64f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f6503fff640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f80047c0 0x7f64f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650a59c640 1 -- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f8005180 msgr2=0x7f64f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650a59c640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f64f8005180 0x7f64f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650a59c640 1 -- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f80047c0 msgr2=0x7f64f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650a59c640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f80047c0 0x7f64f8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650ad9d640 1 -- 172.21.15.70:0/1610978165 reap_dead start 2023-08-25T09:40:31.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.637+0000 7f650a59c640 1 -- 172.21.15.70:0/1610978165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8003520 con 0x7f64f8003970 2023-08-25T09:40:31.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.638+0000 7f650a59c640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8003970 0x7f64f8003d90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f64fc001750 tx=0x7f64f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5bb26e96d06174f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc0052d0 con 0x7f64f8003970 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc02e050 con 0x7f64f8003970 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6509d9b640 1 -- 172.21.15.70:0/1610978165 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003830 con 0x7f64f8003970 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 -- 172.21.15.70:0/1610978165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8003970 msgr2=0x7f64f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8003970 0x7f64f8003d90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f64fc001750 tx=0x7f64f80057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 -- 172.21.15.70:0/1610978165 shutdown_connections 2023-08-25T09:40:31.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 --2- 172.21.15.70:0/1610978165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f64f8003970 0x7f64f8003d90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 -- 172.21.15.70:0/1610978165 >> 172.21.15.70:0/1610978165 conn(0x7f650c069850 msgr2=0x7f650c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 -- 172.21.15.70:0/1610978165 shutdown_connections 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.639+0000 7f6510b4e640 1 -- 172.21.15.70:0/1610978165 wait complete. 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.641+0000 7f6510b4e640 1 Processor -- start 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.641+0000 7f6510b4e640 1 -- start start 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.642+0000 7f6510b4e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f8003970 0x7f650c155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.642+0000 7f6510b4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 0x7f650c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.642+0000 7f6510b4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f650c0a95e0 0x7f650c0a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.642+0000 7f6510b4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c159c10 con 0x7f650c159250 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.642+0000 7f6510b4e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650c0aa310 con 0x7f64f8003970 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.642+0000 7f6510b4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c0aa5f0 con 0x7f650c0a95e0 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 0x7f650c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 0x7f650c1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45836/0 (socket says 172.21.15.70:45836) 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 -- 172.21.15.70:0/316050408 learned_addr learned my addr 172.21.15.70:0/316050408 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 -- 172.21.15.70:0/316050408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f650c0a95e0 msgr2=0x7f650c0a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f650c0a95e0 0x7f650c0a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 -- 172.21.15.70:0/316050408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f8003970 msgr2=0x7f650c155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f8003970 0x7f650c155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 -- 172.21.15.70:0/316050408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c0ab010 con 0x7f650c159250 2023-08-25T09:40:31.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6503fff640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 0x7f650c1596d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f64f40107c0 tx=0x7f64f4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.643+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400eb60 con 0x7f650c159250 2023-08-25T09:40:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.644+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f400ece0 con 0x7f650c159250 2023-08-25T09:40:31.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.644+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f401c640 con 0x7f650c159250 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.644+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c0ab2f0 con 0x7f650c159250 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.644+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650c0a9fd0 con 0x7f650c159250 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.645+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650c0b2940 con 0x7f650c159250 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.646+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f64f40143f0 con 0x7f650c159250 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.647+0000 7f6502ffd640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64ec06a050 0x7f64ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.647+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64f4088ad0 con 0x7f650c159250 2023-08-25T09:40:31.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.647+0000 7f650a59c640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64ec06a050 0x7f64ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:31.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.648+0000 7f650a59c640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64ec06a050 0x7f64ec06c510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64fc033000 tx=0x7f64fc02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:31.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.651+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f64f4054120 con 0x7f650c159250 2023-08-25T09:40:31.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.834+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f650c0b17f0 con 0x7f64ec06a050 2023-08-25T09:40:32.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.850+0000 7f6502ffd640 1 -- 172.21.15.70:0/316050408 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302939 (secure 0 0 0) 0x7f64fc0052d0 con 0x7f64ec06a050 2023-08-25T09:40:32.007 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:32.009 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:40:32.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.855+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64ec06a050 msgr2=0x7f64ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.855+0000 7f6510b4e640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64ec06a050 0x7f64ec06c510 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f64fc033000 tx=0x7f64fc02f000 comp rx=0 tx=0).stop 2023-08-25T09:40:32.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.855+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 msgr2=0x7f650c1596d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.855+0000 7f6510b4e640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 0x7f650c1596d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f64f40107c0 tx=0x7f64f4010c80 comp rx=0 tx=0).stop 2023-08-25T09:40:32.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.855+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 shutdown_connections 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f64ec06a050 0x7f64ec06c510 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f650c0a95e0 0x7f650c0a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f650c159250 0x7f650c1596d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 --2- 172.21.15.70:0/316050408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f64f8003970 0x7f650c155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 >> 172.21.15.70:0/316050408 conn(0x7f650c069850 msgr2=0x7f650c0b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 shutdown_connections 2023-08-25T09:40:32.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:31.856+0000 7f6510b4e640 1 -- 172.21.15.70:0/316050408 wait complete. 2023-08-25T09:40:32.012 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-08-25T09:40:31.350026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2392,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560107,"statfs":{"total":1600000000,"available":1597550189,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.942649"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.235603+0000","last_change":"2023-08-25T09:40:27.235517+0000","last_active":"2023-08-25T09:40:27.235603+0000","last_peered":"2023-08-25T09:40:27.235603+0000","last_clean":"2023-08-25T09:40:27.235603+0000","last_became_active":"2023-08-25T09:40:27.234495+0000","last_became_peered":"2023-08-25T09:40:27.234495+0000","last_unstale":"2023-08-25T09:40:27.235603+0000","last_undegraded":"2023-08-25T09:40:27.235603+0000","last_fullsized":"2023-08-25T09:40:27.235603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:00.171742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.237586+0000","last_change":"2023-08-25T09:40:27.237441+0000","last_active":"2023-08-25T09:40:27.237586+0000","last_peered":"2023-08-25T09:40:27.237586+0000","last_clean":"2023-08-25T09:40:27.237586+0000","last_became_active":"2023-08-25T09:40:27.235984+0000","last_became_peered":"2023-08-25T09:40:27.235984+0000","last_unstale":"2023-08-25T09:40:27.237586+0000","last_undegraded":"2023-08-25T09:40:27.237586+0000","last_fullsized":"2023-08-25T09:40:27.237586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:53.993727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251301+0000","last_change":"2023-08-25T09:40:27.251238+0000","last_active":"2023-08-25T09:40:27.251301+0000","last_peered":"2023-08-25T09:40:27.251301+0000","last_clean":"2023-08-25T09:40:27.251301+0000","last_became_active":"2023-08-25T09:40:27.250822+0000","last_became_peered":"2023-08-25T09:40:27.250822+0000","last_unstale":"2023-08-25T09:40:27.251301+0000","last_undegraded":"2023-08-25T09:40:27.251301+0000","last_fullsized":"2023-08-25T09:40:27.251301+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:04.631849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251442+0000","last_change":"2023-08-25T09:40:27.251383+0000","last_active":"2023-08-25T09:40:27.251442+0000","last_peered":"2023-08-25T09:40:27.251442+0000","last_clean":"2023-08-25T09:40:27.251442+0000","last_became_active":"2023-08-25T09:40:27.250952+0000","last_became_peered":"2023-08-25T09:40:27.250952+0000","last_unstale":"2023-08-25T09:40:27.251442+0000","last_undegraded":"2023-08-25T09:40:27.251442+0000","last_fullsized":"2023-08-25T09:40:27.251442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:13.257224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234652+0000","last_change":"2023-08-25T09:40:27.245778+0000","last_active":"2023-08-25T09:40:28.234652+0000","last_peered":"2023-08-25T09:40:28.234652+0000","last_clean":"2023-08-25T09:40:28.234652+0000","last_became_active":"2023-08-25T09:40:27.244955+0000","last_became_peered":"2023-08-25T09:40:27.244955+0000","last_unstale":"2023-08-25T09:40:28.234652+0000","last_undegraded":"2023-08-25T09:40:28.234652+0000","last_fullsized":"2023-08-25T09:40:28.234652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:17.033560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.236105+0000","last_change":"2023-08-25T09:40:27.235983+0000","last_active":"2023-08-25T09:40:27.236105+0000","last_peered":"2023-08-25T09:40:27.236105+0000","last_clean":"2023-08-25T09:40:27.236105+0000","last_became_active":"2023-08-25T09:40:27.235310+0000","last_became_peered":"2023-08-25T09:40:27.235310+0000","last_unstale":"2023-08-25T09:40:27.236105+0000","last_undegraded":"2023-08-25T09:40:27.236105+0000","last_fullsized":"2023-08-25T09:40:27.236105+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:18:58.163422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.247431+0000","last_change":"2023-08-25T09:40:27.247332+0000","last_active":"2023-08-25T09:40:27.247431+0000","last_peered":"2023-08-25T09:40:27.247431+0000","last_clean":"2023-08-25T09:40:27.247431+0000","last_became_active":"2023-08-25T09:40:27.246918+0000","last_became_peered":"2023-08-25T09:40:27.246918+0000","last_unstale":"2023-08-25T09:40:27.247431+0000","last_undegraded":"2023-08-25T09:40:27.247431+0000","last_fullsized":"2023-08-25T09:40:27.247431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:06:50.633155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244594+0000","last_change":"2023-08-25T09:40:27.244464+0000","last_active":"2023-08-25T09:40:27.244594+0000","last_peered":"2023-08-25T09:40:27.244594+0000","last_clean":"2023-08-25T09:40:27.244594+0000","last_became_active":"2023-08-25T09:40:27.243711+0000","last_became_peered":"2023-08-25T09:40:27.243711+0000","last_unstale":"2023-08-25T09:40:27.244594+0000","last_undegraded":"2023-08-25T09:40:27.244594+0000","last_fullsized":"2023-08-25T09:40:27.244594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:10.467343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234299+0000","last_change":"2023-08-25T09:40:27.232229+0000","last_active":"2023-08-25T09:40:28.234299+0000","last_peered":"2023-08-25T09:40:28.234299+0000","last_clean":"2023-08-25T09:40:28.234299+0000","last_became_active":"2023-08-25T09:40:27.230124+0000","last_became_peered":"2023-08-25T09:40:27.230124+0000","last_unstale":"2023-08-25T09:40:28.234299+0000","last_undegraded":"2023-08-25T09:40:28.234299+0000","last_fullsized":"2023-08-25T09:40:28.234299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:27.155616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.252791+0000","last_change":"2023-08-25T09:40:27.252695+0000","last_active":"2023-08-25T09:40:27.252791+0000","last_peered":"2023-08-25T09:40:27.252791+0000","last_clean":"2023-08-25T09:40:27.252791+0000","last_became_active":"2023-08-25T09:40:27.250521+0000","last_became_peered":"2023-08-25T09:40:27.250521+0000","last_unstale":"2023-08-25T09:40:27.252791+0000","last_undegraded":"2023-08-25T09:40:27.252791+0000","last_fullsized":"2023-08-25T09:40:27.252791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:09.522369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.253552+0000","last_change":"2023-08-25T09:40:27.252864+0000","last_active":"2023-08-25T09:40:27.253552+0000","last_peered":"2023-08-25T09:40:27.253552+0000","last_clean":"2023-08-25T09:40:27.253552+0000","last_became_active":"2023-08-25T09:40:27.250079+0000","last_became_peered":"2023-08-25T09:40:27.250079+0000","last_unstale":"2023-08-25T09:40:27.253552+0000","last_undegraded":"2023-08-25T09:40:27.253552+0000","last_fullsized":"2023-08-25T09:40:27.253552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:54:41.419105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234625+0000","last_change":"2023-08-25T09:40:27.240439+0000","last_active":"2023-08-25T09:40:28.234625+0000","last_peered":"2023-08-25T09:40:28.234625+0000","last_clean":"2023-08-25T09:40:28.234625+0000","last_became_active":"2023-08-25T09:40:27.239598+0000","last_became_peered":"2023-08-25T09:40:27.239598+0000","last_unstale":"2023-08-25T09:40:28.234625+0000","last_undegraded":"2023-08-25T09:40:28.234625+0000","last_fullsized":"2023-08-25T09:40:28.234625+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:40.711092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244217+0000","last_change":"2023-08-25T09:40:27.242649+0000","last_active":"2023-08-25T09:40:27.244217+0000","last_peered":"2023-08-25T09:40:27.244217+0000","last_clean":"2023-08-25T09:40:27.244217+0000","last_became_active":"2023-08-25T09:40:27.242141+0000","last_became_peered":"2023-08-25T09:40:27.242141+0000","last_unstale":"2023-08-25T09:40:27.244217+0000","last_undegraded":"2023-08-25T09:40:27.244217+0000","last_fullsized":"2023-08-25T09:40:27.244217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:35:52.785789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232677+0000","last_change":"2023-08-25T09:40:27.253854+0000","last_active":"2023-08-25T09:40:28.232677+0000","last_peered":"2023-08-25T09:40:28.232677+0000","last_clean":"2023-08-25T09:40:28.232677+0000","last_became_active":"2023-08-25T09:40:27.253569+0000","last_became_peered":"2023-08-25T09:40:27.253569+0000","last_unstale":"2023-08-25T09:40:28.232677+0000","last_undegraded":"2023-08-25T09:40:28.232677+0000","last_fullsized":"2023-08-25T09:40:28.232677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:55.421015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229312+0000","last_change":"2023-08-25T09:40:27.225902+0000","last_active":"2023-08-25T09:40:28.229312+0000","last_peered":"2023-08-25T09:40:28.229312+0000","last_clean":"2023-08-25T09:40:28.229312+0000","last_became_active":"2023-08-25T09:40:27.225241+0000","last_became_peered":"2023-08-25T09:40:27.225241+0000","last_unstale":"2023-08-25T09:40:28.229312+0000","last_undegraded":"2023-08-25T09:40:28.229312+0000","last_fullsized":"2023-08-25T09:40:28.229312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:31.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.246298+0000","last_change":"2023-08-25T09:40:27.246221+0000","last_active":"2023-08-25T09:40:27.246298+0000","last_peered":"2023-08-25T09:40:27.246298+0000","last_clean":"2023-08-25T09:40:27.246298+0000","last_became_active":"2023-08-25T09:40:27.245250+0000","last_became_peered":"2023-08-25T09:40:27.245250+0000","last_unstale":"2023-08-25T09:40:27.246298+0000","last_undegraded":"2023-08-25T09:40:27.246298+0000","last_fullsized":"2023-08-25T09:40:27.246298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:36:25.767167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229744+0000","last_change":"2023-08-25T09:40:27.249907+0000","last_active":"2023-08-25T09:40:28.229744+0000","last_peered":"2023-08-25T09:40:28.229744+0000","last_clean":"2023-08-25T09:40:28.229744+0000","last_became_active":"2023-08-25T09:40:27.249643+0000","last_became_peered":"2023-08-25T09:40:27.249643+0000","last_unstale":"2023-08-25T09:40:28.229744+0000","last_undegraded":"2023-08-25T09:40:28.229744+0000","last_fullsized":"2023-08-25T09:40:28.229744+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:01.770533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228965+0000","last_change":"2023-08-25T09:40:27.250612+0000","last_active":"2023-08-25T09:40:28.228965+0000","last_peered":"2023-08-25T09:40:28.228965+0000","last_clean":"2023-08-25T09:40:28.228965+0000","last_became_active":"2023-08-25T09:40:27.250306+0000","last_became_peered":"2023-08-25T09:40:27.250306+0000","last_unstale":"2023-08-25T09:40:28.228965+0000","last_undegraded":"2023-08-25T09:40:28.228965+0000","last_fullsized":"2023-08-25T09:40:28.228965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:34.191998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.240627+0000","last_change":"2023-08-25T09:40:27.240407+0000","last_active":"2023-08-25T09:40:27.240627+0000","last_peered":"2023-08-25T09:40:27.240627+0000","last_clean":"2023-08-25T09:40:27.240627+0000","last_became_active":"2023-08-25T09:40:27.239935+0000","last_became_peered":"2023-08-25T09:40:27.239935+0000","last_unstale":"2023-08-25T09:40:27.240627+0000","last_undegraded":"2023-08-25T09:40:27.240627+0000","last_fullsized":"2023-08-25T09:40:27.240627+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:37.766243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245860+0000","last_change":"2023-08-25T09:40:27.244252+0000","last_active":"2023-08-25T09:40:27.245860+0000","last_peered":"2023-08-25T09:40:27.245860+0000","last_clean":"2023-08-25T09:40:27.245860+0000","last_became_active":"2023-08-25T09:40:27.243961+0000","last_became_peered":"2023-08-25T09:40:27.243961+0000","last_unstale":"2023-08-25T09:40:27.245860+0000","last_undegraded":"2023-08-25T09:40:27.245860+0000","last_fullsized":"2023-08-25T09:40:27.245860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:33.793228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251868+0000","last_change":"2023-08-25T09:40:27.251472+0000","last_active":"2023-08-25T09:40:27.251868+0000","last_peered":"2023-08-25T09:40:27.251868+0000","last_clean":"2023-08-25T09:40:27.251868+0000","last_became_active":"2023-08-25T09:40:27.246490+0000","last_became_peered":"2023-08-25T09:40:27.246490+0000","last_unstale":"2023-08-25T09:40:27.251868+0000","last_undegraded":"2023-08-25T09:40:27.251868+0000","last_fullsized":"2023-08-25T09:40:27.251868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:18.035205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.248131+0000","last_change":"2023-08-25T09:40:27.247907+0000","last_active":"2023-08-25T09:40:27.248131+0000","last_peered":"2023-08-25T09:40:27.248131+0000","last_clean":"2023-08-25T09:40:27.248131+0000","last_became_active":"2023-08-25T09:40:27.245450+0000","last_became_peered":"2023-08-25T09:40:27.245450+0000","last_unstale":"2023-08-25T09:40:27.248131+0000","last_undegraded":"2023-08-25T09:40:27.248131+0000","last_fullsized":"2023-08-25T09:40:27.248131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:01.001152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229275+0000","last_change":"2023-08-25T09:40:27.240258+0000","last_active":"2023-08-25T09:40:28.229275+0000","last_peered":"2023-08-25T09:40:28.229275+0000","last_clean":"2023-08-25T09:40:28.229275+0000","last_became_active":"2023-08-25T09:40:27.238975+0000","last_became_peered":"2023-08-25T09:40:27.238975+0000","last_unstale":"2023-08-25T09:40:28.229275+0000","last_undegraded":"2023-08-25T09:40:28.229275+0000","last_fullsized":"2023-08-25T09:40:28.229275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:02.743173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.250719+0000","last_change":"2023-08-25T09:40:27.250649+0000","last_active":"2023-08-25T09:40:27.250719+0000","last_peered":"2023-08-25T09:40:27.250719+0000","last_clean":"2023-08-25T09:40:27.250719+0000","last_became_active":"2023-08-25T09:40:27.249259+0000","last_became_peered":"2023-08-25T09:40:27.249259+0000","last_unstale":"2023-08-25T09:40:27.250719+0000","last_undegraded":"2023-08-25T09:40:27.250719+0000","last_fullsized":"2023-08-25T09:40:27.250719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:44.759659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232021+0000","last_change":"2023-08-25T09:40:27.245788+0000","last_active":"2023-08-25T09:40:28.232021+0000","last_peered":"2023-08-25T09:40:28.232021+0000","last_clean":"2023-08-25T09:40:28.232021+0000","last_became_active":"2023-08-25T09:40:27.245409+0000","last_became_peered":"2023-08-25T09:40:27.245409+0000","last_unstale":"2023-08-25T09:40:28.232021+0000","last_undegraded":"2023-08-25T09:40:28.232021+0000","last_fullsized":"2023-08-25T09:40:28.232021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:44.399476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251083+0000","last_change":"2023-08-25T09:40:27.250984+0000","last_active":"2023-08-25T09:40:27.251083+0000","last_peered":"2023-08-25T09:40:27.251083+0000","last_clean":"2023-08-25T09:40:27.251083+0000","last_became_active":"2023-08-25T09:40:27.250503+0000","last_became_peered":"2023-08-25T09:40:27.250503+0000","last_unstale":"2023-08-25T09:40:27.251083+0000","last_undegraded":"2023-08-25T09:40:27.251083+0000","last_fullsized":"2023-08-25T09:40:27.251083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:06:53.168893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245832+0000","last_change":"2023-08-25T09:40:27.243524+0000","last_active":"2023-08-25T09:40:27.245832+0000","last_peered":"2023-08-25T09:40:27.245832+0000","last_clean":"2023-08-25T09:40:27.245832+0000","last_became_active":"2023-08-25T09:40:27.243078+0000","last_became_peered":"2023-08-25T09:40:27.243078+0000","last_unstale":"2023-08-25T09:40:27.245832+0000","last_undegraded":"2023-08-25T09:40:27.245832+0000","last_fullsized":"2023-08-25T09:40:27.245832+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:32.287541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244186+0000","last_change":"2023-08-25T09:40:27.242496+0000","last_active":"2023-08-25T09:40:27.244186+0000","last_peered":"2023-08-25T09:40:27.244186+0000","last_clean":"2023-08-25T09:40:27.244186+0000","last_became_active":"2023-08-25T09:40:27.241848+0000","last_became_peered":"2023-08-25T09:40:27.241848+0000","last_unstale":"2023-08-25T09:40:27.244186+0000","last_undegraded":"2023-08-25T09:40:27.244186+0000","last_fullsized":"2023-08-25T09:40:27.244186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:52:38.157083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229429+0000","last_change":"2023-08-25T09:40:27.232893+0000","last_active":"2023-08-25T09:40:28.229429+0000","last_peered":"2023-08-25T09:40:28.229429+0000","last_clean":"2023-08-25T09:40:28.229429+0000","last_became_active":"2023-08-25T09:40:27.232114+0000","last_became_peered":"2023-08-25T09:40:27.232114+0000","last_unstale":"2023-08-25T09:40:28.229429+0000","last_undegraded":"2023-08-25T09:40:28.229429+0000","last_fullsized":"2023-08-25T09:40:28.229429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:16:35.713957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229638+0000","last_change":"2023-08-25T09:40:27.232578+0000","last_active":"2023-08-25T09:40:28.229638+0000","last_peered":"2023-08-25T09:40:28.229638+0000","last_clean":"2023-08-25T09:40:28.229638+0000","last_became_active":"2023-08-25T09:40:27.231879+0000","last_became_peered":"2023-08-25T09:40:27.231879+0000","last_unstale":"2023-08-25T09:40:28.229638+0000","last_undegraded":"2023-08-25T09:40:28.229638+0000","last_fullsized":"2023-08-25T09:40:28.229638+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:49.537369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.246161+0000","last_change":"2023-08-25T09:40:27.246025+0000","last_active":"2023-08-25T09:40:27.246161+0000","last_peered":"2023-08-25T09:40:27.246161+0000","last_clean":"2023-08-25T09:40:27.246161+0000","last_became_active":"2023-08-25T09:40:27.244931+0000","last_became_peered":"2023-08-25T09:40:27.244931+0000","last_unstale":"2023-08-25T09:40:27.246161+0000","last_undegraded":"2023-08-25T09:40:27.246161+0000","last_fullsized":"2023-08-25T09:40:27.246161+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:14.272870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.250168+0000","last_change":"2023-08-25T09:40:27.250008+0000","last_active":"2023-08-25T09:40:27.250168+0000","last_peered":"2023-08-25T09:40:27.250168+0000","last_clean":"2023-08-25T09:40:27.250168+0000","last_became_active":"2023-08-25T09:40:27.249242+0000","last_became_peered":"2023-08-25T09:40:27.249242+0000","last_unstale":"2023-08-25T09:40:27.250168+0000","last_undegraded":"2023-08-25T09:40:27.250168+0000","last_fullsized":"2023-08-25T09:40:27.250168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:14.005564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228630+0000","last_change":"2023-08-25T09:40:27.247110+0000","last_active":"2023-08-25T09:40:28.228630+0000","last_peered":"2023-08-25T09:40:28.228630+0000","last_clean":"2023-08-25T09:40:28.228630+0000","last_became_active":"2023-08-25T09:40:27.246745+0000","last_became_peered":"2023-08-25T09:40:27.246745+0000","last_unstale":"2023-08-25T09:40:28.228630+0000","last_undegraded":"2023-08-25T09:40:28.228630+0000","last_fullsized":"2023-08-25T09:40:28.228630+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:15.172560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232298+0000","last_change":"2023-08-25T09:40:26.440045+0000","last_active":"2023-08-25T09:40:28.232298+0000","last_peered":"2023-08-25T09:40:28.232298+0000","last_clean":"2023-08-25T09:40:28.232298+0000","last_became_active":"2023-08-25T09:40:26.439784+0000","last_became_peered":"2023-08-25T09:40:26.439784+0000","last_unstale":"2023-08-25T09:40:28.232298+0000","last_undegraded":"2023-08-25T09:40:28.232298+0000","last_fullsized":"2023-08-25T09:40:28.232298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:49:28.490921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.241188+0000","last_change":"2023-08-25T09:40:27.241070+0000","last_active":"2023-08-25T09:40:27.241188+0000","last_peered":"2023-08-25T09:40:27.241188+0000","last_clean":"2023-08-25T09:40:27.241188+0000","last_became_active":"2023-08-25T09:40:27.239890+0000","last_became_peered":"2023-08-25T09:40:27.239890+0000","last_unstale":"2023-08-25T09:40:27.241188+0000","last_undegraded":"2023-08-25T09:40:27.241188+0000","last_fullsized":"2023-08-25T09:40:27.241188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:09.741049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.449633+0000","last_change":"2023-08-25T09:40:26.446455+0000","last_active":"2023-08-25T09:40:27.449633+0000","last_peered":"2023-08-25T09:40:27.449633+0000","last_clean":"2023-08-25T09:40:27.449633+0000","last_became_active":"2023-08-25T09:40:26.445291+0000","last_became_peered":"2023-08-25T09:40:26.445291+0000","last_unstale":"2023-08-25T09:40:27.449633+0000","last_undegraded":"2023-08-25T09:40:27.449633+0000","last_fullsized":"2023-08-25T09:40:27.449633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:32.696998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229793+0000","last_change":"2023-08-25T09:40:27.250075+0000","last_active":"2023-08-25T09:40:28.229793+0000","last_peered":"2023-08-25T09:40:28.229793+0000","last_clean":"2023-08-25T09:40:28.229793+0000","last_became_active":"2023-08-25T09:40:27.249635+0000","last_became_peered":"2023-08-25T09:40:27.249635+0000","last_unstale":"2023-08-25T09:40:28.229793+0000","last_undegraded":"2023-08-25T09:40:28.229793+0000","last_fullsized":"2023-08-25T09:40:28.229793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:42.048174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.447999+0000","last_change":"2023-08-25T09:40:26.453893+0000","last_active":"2023-08-25T09:40:27.447999+0000","last_peered":"2023-08-25T09:40:27.447999+0000","last_clean":"2023-08-25T09:40:27.447999+0000","last_became_active":"2023-08-25T09:40:26.452425+0000","last_became_peered":"2023-08-25T09:40:26.452425+0000","last_unstale":"2023-08-25T09:40:27.447999+0000","last_undegraded":"2023-08-25T09:40:27.447999+0000","last_fullsized":"2023-08-25T09:40:27.447999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:44:08.250945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.239607+0000","last_change":"2023-08-25T09:40:27.239458+0000","last_active":"2023-08-25T09:40:27.239607+0000","last_peered":"2023-08-25T09:40:27.239607+0000","last_clean":"2023-08-25T09:40:27.239607+0000","last_became_active":"2023-08-25T09:40:27.238127+0000","last_became_peered":"2023-08-25T09:40:27.238127+0000","last_unstale":"2023-08-25T09:40:27.239607+0000","last_undegraded":"2023-08-25T09:40:27.239607+0000","last_fullsized":"2023-08-25T09:40:27.239607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:17.133840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232423+0000","last_change":"2023-08-25T09:40:26.453980+0000","last_active":"2023-08-25T09:40:28.232423+0000","last_peered":"2023-08-25T09:40:28.232423+0000","last_clean":"2023-08-25T09:40:28.232423+0000","last_became_active":"2023-08-25T09:40:26.453721+0000","last_became_peered":"2023-08-25T09:40:26.453721+0000","last_unstale":"2023-08-25T09:40:28.232423+0000","last_undegraded":"2023-08-25T09:40:28.232423+0000","last_fullsized":"2023-08-25T09:40:28.232423+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:32.562036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.238635+0000","last_change":"2023-08-25T09:40:27.238322+0000","last_active":"2023-08-25T09:40:27.238635+0000","last_peered":"2023-08-25T09:40:27.238635+0000","last_clean":"2023-08-25T09:40:27.238635+0000","last_became_active":"2023-08-25T09:40:27.237962+0000","last_became_peered":"2023-08-25T09:40:27.237962+0000","last_unstale":"2023-08-25T09:40:27.238635+0000","last_undegraded":"2023-08-25T09:40:27.238635+0000","last_fullsized":"2023-08-25T09:40:27.238635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:46.020827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.231862+0000","last_change":"2023-08-25T09:40:26.451828+0000","last_active":"2023-08-25T09:40:28.231862+0000","last_peered":"2023-08-25T09:40:28.231862+0000","last_clean":"2023-08-25T09:40:28.231862+0000","last_became_active":"2023-08-25T09:40:26.451553+0000","last_became_peered":"2023-08-25T09:40:26.451553+0000","last_unstale":"2023-08-25T09:40:28.231862+0000","last_undegraded":"2023-08-25T09:40:28.231862+0000","last_fullsized":"2023-08-25T09:40:28.231862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:28:47.337275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.246392+0000","last_change":"2023-08-25T09:40:27.246200+0000","last_active":"2023-08-25T09:40:27.246392+0000","last_peered":"2023-08-25T09:40:27.246392+0000","last_clean":"2023-08-25T09:40:27.246392+0000","last_became_active":"2023-08-25T09:40:27.244520+0000","last_became_peered":"2023-08-25T09:40:27.244520+0000","last_unstale":"2023-08-25T09:40:27.246392+0000","last_undegraded":"2023-08-25T09:40:27.246392+0000","last_fullsized":"2023-08-25T09:40:27.246392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:53.425898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.446844+0000","last_change":"2023-08-25T09:40:26.447532+0000","last_active":"2023-08-25T09:40:27.446844+0000","last_peered":"2023-08-25T09:40:27.446844+0000","last_clean":"2023-08-25T09:40:27.446844+0000","last_became_active":"2023-08-25T09:40:26.447193+0000","last_became_peered":"2023-08-25T09:40:26.447193+0000","last_unstale":"2023-08-25T09:40:27.446844+0000","last_undegraded":"2023-08-25T09:40:27.446844+0000","last_fullsized":"2023-08-25T09:40:27.446844+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:34:27.723860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229575+0000","last_change":"2023-08-25T09:40:27.240487+0000","last_active":"2023-08-25T09:40:28.229575+0000","last_peered":"2023-08-25T09:40:28.229575+0000","last_clean":"2023-08-25T09:40:28.229575+0000","last_became_active":"2023-08-25T09:40:27.239763+0000","last_became_peered":"2023-08-25T09:40:27.239763+0000","last_unstale":"2023-08-25T09:40:28.229575+0000","last_undegraded":"2023-08-25T09:40:28.229575+0000","last_fullsized":"2023-08-25T09:40:28.229575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:40.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.236608+0000","last_change":"2023-08-25T09:40:26.453258+0000","last_active":"2023-08-25T09:40:27.236608+0000","last_peered":"2023-08-25T09:40:27.236608+0000","last_clean":"2023-08-25T09:40:27.236608+0000","last_became_active":"2023-08-25T09:40:26.451293+0000","last_became_peered":"2023-08-25T09:40:26.451293+0000","last_unstale":"2023-08-25T09:40:27.236608+0000","last_undegraded":"2023-08-25T09:40:27.236608+0000","last_fullsized":"2023-08-25T09:40:27.236608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:23:42.700979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249923+0000","last_change":"2023-08-25T09:40:27.249759+0000","last_active":"2023-08-25T09:40:27.249923+0000","last_peered":"2023-08-25T09:40:27.249923+0000","last_clean":"2023-08-25T09:40:27.249923+0000","last_became_active":"2023-08-25T09:40:27.248606+0000","last_became_peered":"2023-08-25T09:40:27.248606+0000","last_unstale":"2023-08-25T09:40:27.249923+0000","last_undegraded":"2023-08-25T09:40:27.249923+0000","last_fullsized":"2023-08-25T09:40:27.249923+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:32.287541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.231324+0000","last_change":"2023-08-25T09:40:26.444725+0000","last_active":"2023-08-25T09:40:27.231324+0000","last_peered":"2023-08-25T09:40:27.231324+0000","last_clean":"2023-08-25T09:40:27.231324+0000","last_became_active":"2023-08-25T09:40:26.444143+0000","last_became_peered":"2023-08-25T09:40:26.444143+0000","last_unstale":"2023-08-25T09:40:27.231324+0000","last_undegraded":"2023-08-25T09:40:27.231324+0000","last_fullsized":"2023-08-25T09:40:27.231324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:37:31.263967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251316+0000","last_change":"2023-08-25T09:40:27.251243+0000","last_active":"2023-08-25T09:40:27.251316+0000","last_peered":"2023-08-25T09:40:27.251316+0000","last_clean":"2023-08-25T09:40:27.251316+0000","last_became_active":"2023-08-25T09:40:27.250402+0000","last_became_peered":"2023-08-25T09:40:27.250402+0000","last_unstale":"2023-08-25T09:40:27.251316+0000","last_undegraded":"2023-08-25T09:40:27.251316+0000","last_fullsized":"2023-08-25T09:40:27.251316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:53:25.211130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.227452+0000","last_change":"2023-08-25T09:40:26.451421+0000","last_active":"2023-08-25T09:40:27.227452+0000","last_peered":"2023-08-25T09:40:27.227452+0000","last_clean":"2023-08-25T09:40:27.227452+0000","last_became_active":"2023-08-25T09:40:26.451104+0000","last_became_peered":"2023-08-25T09:40:26.451104+0000","last_unstale":"2023-08-25T09:40:27.227452+0000","last_undegraded":"2023-08-25T09:40:27.227452+0000","last_fullsized":"2023-08-25T09:40:27.227452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:49:47.084520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232044+0000","last_change":"2023-08-25T09:40:27.251131+0000","last_active":"2023-08-25T09:40:28.232044+0000","last_peered":"2023-08-25T09:40:28.232044+0000","last_clean":"2023-08-25T09:40:28.232044+0000","last_became_active":"2023-08-25T09:40:27.250801+0000","last_became_peered":"2023-08-25T09:40:27.250801+0000","last_unstale":"2023-08-25T09:40:28.232044+0000","last_undegraded":"2023-08-25T09:40:28.232044+0000","last_fullsized":"2023-08-25T09:40:28.232044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:16.054202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229405+0000","last_change":"2023-08-25T09:40:26.445211+0000","last_active":"2023-08-25T09:40:28.229405+0000","last_peered":"2023-08-25T09:40:28.229405+0000","last_clean":"2023-08-25T09:40:28.229405+0000","last_became_active":"2023-08-25T09:40:26.444959+0000","last_became_peered":"2023-08-25T09:40:26.444959+0000","last_unstale":"2023-08-25T09:40:28.229405+0000","last_undegraded":"2023-08-25T09:40:28.229405+0000","last_fullsized":"2023-08-25T09:40:28.229405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:22.438085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249873+0000","last_change":"2023-08-25T09:40:27.249745+0000","last_active":"2023-08-25T09:40:27.249873+0000","last_peered":"2023-08-25T09:40:27.249873+0000","last_clean":"2023-08-25T09:40:27.249873+0000","last_became_active":"2023-08-25T09:40:27.248763+0000","last_became_peered":"2023-08-25T09:40:27.248763+0000","last_unstale":"2023-08-25T09:40:27.249873+0000","last_undegraded":"2023-08-25T09:40:27.249873+0000","last_fullsized":"2023-08-25T09:40:27.249873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:31.717010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229926+0000","last_change":"2023-08-25T09:40:26.449338+0000","last_active":"2023-08-25T09:40:28.229926+0000","last_peered":"2023-08-25T09:40:28.229926+0000","last_clean":"2023-08-25T09:40:28.229926+0000","last_became_active":"2023-08-25T09:40:26.448092+0000","last_became_peered":"2023-08-25T09:40:26.448092+0000","last_unstale":"2023-08-25T09:40:28.229926+0000","last_undegraded":"2023-08-25T09:40:28.229926+0000","last_fullsized":"2023-08-25T09:40:28.229926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:18:58.605071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245949+0000","last_change":"2023-08-25T09:40:27.245865+0000","last_active":"2023-08-25T09:40:27.245949+0000","last_peered":"2023-08-25T09:40:27.245949+0000","last_clean":"2023-08-25T09:40:27.245949+0000","last_became_active":"2023-08-25T09:40:27.244521+0000","last_became_peered":"2023-08-25T09:40:27.244521+0000","last_unstale":"2023-08-25T09:40:27.245949+0000","last_undegraded":"2023-08-25T09:40:27.245949+0000","last_fullsized":"2023-08-25T09:40:27.245949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:33.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.450784+0000","last_change":"2023-08-25T09:40:26.453725+0000","last_active":"2023-08-25T09:40:27.450784+0000","last_peered":"2023-08-25T09:40:27.450784+0000","last_clean":"2023-08-25T09:40:27.450784+0000","last_became_active":"2023-08-25T09:40:26.451984+0000","last_became_peered":"2023-08-25T09:40:26.451984+0000","last_unstale":"2023-08-25T09:40:27.450784+0000","last_undegraded":"2023-08-25T09:40:27.450784+0000","last_fullsized":"2023-08-25T09:40:27.450784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:27.844837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.250502+0000","last_change":"2023-08-25T09:40:27.250422+0000","last_active":"2023-08-25T09:40:27.250502+0000","last_peered":"2023-08-25T09:40:27.250502+0000","last_clean":"2023-08-25T09:40:27.250502+0000","last_became_active":"2023-08-25T09:40:27.249377+0000","last_became_peered":"2023-08-25T09:40:27.249377+0000","last_unstale":"2023-08-25T09:40:27.250502+0000","last_undegraded":"2023-08-25T09:40:27.250502+0000","last_fullsized":"2023-08-25T09:40:27.250502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:23:06.089592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.449187+0000","last_change":"2023-08-25T09:40:26.454321+0000","last_active":"2023-08-25T09:40:27.449187+0000","last_peered":"2023-08-25T09:40:27.449187+0000","last_clean":"2023-08-25T09:40:27.449187+0000","last_became_active":"2023-08-25T09:40:26.453648+0000","last_became_peered":"2023-08-25T09:40:26.453648+0000","last_unstale":"2023-08-25T09:40:27.449187+0000","last_undegraded":"2023-08-25T09:40:27.449187+0000","last_fullsized":"2023-08-25T09:40:27.449187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:47:35.404450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249343+0000","last_change":"2023-08-25T09:40:27.249185+0000","last_active":"2023-08-25T09:40:27.249343+0000","last_peered":"2023-08-25T09:40:27.249343+0000","last_clean":"2023-08-25T09:40:27.249343+0000","last_became_active":"2023-08-25T09:40:27.247843+0000","last_became_peered":"2023-08-25T09:40:27.247843+0000","last_unstale":"2023-08-25T09:40:27.249343+0000","last_undegraded":"2023-08-25T09:40:27.249343+0000","last_fullsized":"2023-08-25T09:40:27.249343+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:53.425898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.222625+0000","last_change":"2023-08-25T09:40:26.446621+0000","last_active":"2023-08-25T09:40:27.222625+0000","last_peered":"2023-08-25T09:40:27.222625+0000","last_clean":"2023-08-25T09:40:27.222625+0000","last_became_active":"2023-08-25T09:40:26.445438+0000","last_became_peered":"2023-08-25T09:40:26.445438+0000","last_unstale":"2023-08-25T09:40:27.222625+0000","last_undegraded":"2023-08-25T09:40:27.222625+0000","last_fullsized":"2023-08-25T09:40:27.222625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:26:59.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.252673+0000","last_change":"2023-08-25T09:40:27.252197+0000","last_active":"2023-08-25T09:40:27.252673+0000","last_peered":"2023-08-25T09:40:27.252673+0000","last_clean":"2023-08-25T09:40:27.252673+0000","last_became_active":"2023-08-25T09:40:27.249544+0000","last_became_peered":"2023-08-25T09:40:27.249544+0000","last_unstale":"2023-08-25T09:40:27.252673+0000","last_undegraded":"2023-08-25T09:40:27.252673+0000","last_fullsized":"2023-08-25T09:40:27.252673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:53.993727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230377+0000","last_change":"2023-08-25T09:40:26.433779+0000","last_active":"2023-08-25T09:40:28.230377+0000","last_peered":"2023-08-25T09:40:28.230377+0000","last_clean":"2023-08-25T09:40:28.230377+0000","last_became_active":"2023-08-25T09:40:26.433504+0000","last_became_peered":"2023-08-25T09:40:26.433504+0000","last_unstale":"2023-08-25T09:40:28.230377+0000","last_undegraded":"2023-08-25T09:40:28.230377+0000","last_fullsized":"2023-08-25T09:40:28.230377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:58:26.635834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.235044+0000","last_change":"2023-08-25T09:40:27.231232+0000","last_active":"2023-08-25T09:40:28.235044+0000","last_peered":"2023-08-25T09:40:28.235044+0000","last_clean":"2023-08-25T09:40:28.235044+0000","last_became_active":"2023-08-25T09:40:27.230725+0000","last_became_peered":"2023-08-25T09:40:27.230725+0000","last_unstale":"2023-08-25T09:40:28.235044+0000","last_undegraded":"2023-08-25T09:40:28.235044+0000","last_fullsized":"2023-08-25T09:40:28.235044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:13.637819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232481+0000","last_change":"2023-08-25T09:40:26.440595+0000","last_active":"2023-08-25T09:40:28.232481+0000","last_peered":"2023-08-25T09:40:28.232481+0000","last_clean":"2023-08-25T09:40:28.232481+0000","last_became_active":"2023-08-25T09:40:26.440344+0000","last_became_peered":"2023-08-25T09:40:26.440344+0000","last_unstale":"2023-08-25T09:40:28.232481+0000","last_undegraded":"2023-08-25T09:40:28.232481+0000","last_fullsized":"2023-08-25T09:40:28.232481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:38:12.351776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249624+0000","last_change":"2023-08-25T09:40:27.249529+0000","last_active":"2023-08-25T09:40:27.249624+0000","last_peered":"2023-08-25T09:40:27.249624+0000","last_clean":"2023-08-25T09:40:27.249624+0000","last_became_active":"2023-08-25T09:40:27.248214+0000","last_became_peered":"2023-08-25T09:40:27.248214+0000","last_unstale":"2023-08-25T09:40:27.249624+0000","last_undegraded":"2023-08-25T09:40:27.249624+0000","last_fullsized":"2023-08-25T09:40:27.249624+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:18.035205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.447141+0000","last_change":"2023-08-25T09:40:26.442843+0000","last_active":"2023-08-25T09:40:27.447141+0000","last_peered":"2023-08-25T09:40:27.447141+0000","last_clean":"2023-08-25T09:40:27.447141+0000","last_became_active":"2023-08-25T09:40:26.442540+0000","last_became_peered":"2023-08-25T09:40:26.442540+0000","last_unstale":"2023-08-25T09:40:27.447141+0000","last_undegraded":"2023-08-25T09:40:27.447141+0000","last_fullsized":"2023-08-25T09:40:27.447141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:36.753372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245721+0000","last_change":"2023-08-25T09:40:27.245625+0000","last_active":"2023-08-25T09:40:27.245721+0000","last_peered":"2023-08-25T09:40:27.245721+0000","last_clean":"2023-08-25T09:40:27.245721+0000","last_became_active":"2023-08-25T09:40:27.243404+0000","last_became_peered":"2023-08-25T09:40:27.243404+0000","last_unstale":"2023-08-25T09:40:27.245721+0000","last_undegraded":"2023-08-25T09:40:27.245721+0000","last_fullsized":"2023-08-25T09:40:27.245721+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:13.637819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229968+0000","last_change":"2023-08-25T09:40:26.448676+0000","last_active":"2023-08-25T09:40:28.229968+0000","last_peered":"2023-08-25T09:40:28.229968+0000","last_clean":"2023-08-25T09:40:28.229968+0000","last_became_active":"2023-08-25T09:40:26.448068+0000","last_became_peered":"2023-08-25T09:40:26.448068+0000","last_unstale":"2023-08-25T09:40:28.229968+0000","last_undegraded":"2023-08-25T09:40:28.229968+0000","last_fullsized":"2023-08-25T09:40:28.229968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:14:18.703885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244295+0000","last_change":"2023-08-25T09:40:27.244124+0000","last_active":"2023-08-25T09:40:27.244295+0000","last_peered":"2023-08-25T09:40:27.244295+0000","last_clean":"2023-08-25T09:40:27.244295+0000","last_became_active":"2023-08-25T09:40:27.243499+0000","last_became_peered":"2023-08-25T09:40:27.243499+0000","last_unstale":"2023-08-25T09:40:27.244295+0000","last_undegraded":"2023-08-25T09:40:27.244295+0000","last_fullsized":"2023-08-25T09:40:27.244295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:05:50.380346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234899+0000","last_change":"2023-08-25T09:40:26.448882+0000","last_active":"2023-08-25T09:40:28.234899+0000","last_peered":"2023-08-25T09:40:28.234899+0000","last_clean":"2023-08-25T09:40:28.234899+0000","last_became_active":"2023-08-25T09:40:26.448612+0000","last_became_peered":"2023-08-25T09:40:26.448612+0000","last_unstale":"2023-08-25T09:40:28.234899+0000","last_undegraded":"2023-08-25T09:40:28.234899+0000","last_fullsized":"2023-08-25T09:40:28.234899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:00:58.640931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.240344+0000","last_change":"2023-08-25T09:40:27.240215+0000","last_active":"2023-08-25T09:40:27.240344+0000","last_peered":"2023-08-25T09:40:27.240344+0000","last_clean":"2023-08-25T09:40:27.240344+0000","last_became_active":"2023-08-25T09:40:27.239535+0000","last_became_peered":"2023-08-25T09:40:27.239535+0000","last_unstale":"2023-08-25T09:40:27.240344+0000","last_undegraded":"2023-08-25T09:40:27.240344+0000","last_fullsized":"2023-08-25T09:40:27.240344+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:33.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.229210+0000","last_change":"2023-08-25T09:40:26.449887+0000","last_active":"2023-08-25T09:40:27.229210+0000","last_peered":"2023-08-25T09:40:27.229210+0000","last_clean":"2023-08-25T09:40:27.229210+0000","last_became_active":"2023-08-25T09:40:26.449622+0000","last_became_peered":"2023-08-25T09:40:26.449622+0000","last_unstale":"2023-08-25T09:40:27.229210+0000","last_undegraded":"2023-08-25T09:40:27.229210+0000","last_fullsized":"2023-08-25T09:40:27.229210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:08.509499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228791+0000","last_change":"2023-08-25T09:40:27.225768+0000","last_active":"2023-08-25T09:40:28.228791+0000","last_peered":"2023-08-25T09:40:28.228791+0000","last_clean":"2023-08-25T09:40:28.228791+0000","last_became_active":"2023-08-25T09:40:27.225172+0000","last_became_peered":"2023-08-25T09:40:27.225172+0000","last_unstale":"2023-08-25T09:40:28.228791+0000","last_undegraded":"2023-08-25T09:40:28.228791+0000","last_fullsized":"2023-08-25T09:40:28.228791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:19.497902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.452137+0000","last_change":"2023-08-25T09:40:26.451165+0000","last_active":"2023-08-25T09:40:27.452137+0000","last_peered":"2023-08-25T09:40:27.452137+0000","last_clean":"2023-08-25T09:40:27.452137+0000","last_became_active":"2023-08-25T09:40:26.450877+0000","last_became_peered":"2023-08-25T09:40:26.450877+0000","last_unstale":"2023-08-25T09:40:27.452137+0000","last_undegraded":"2023-08-25T09:40:27.452137+0000","last_fullsized":"2023-08-25T09:40:27.452137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:13.259999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.248711+0000","last_change":"2023-08-25T09:40:27.248614+0000","last_active":"2023-08-25T09:40:27.248711+0000","last_peered":"2023-08-25T09:40:27.248711+0000","last_clean":"2023-08-25T09:40:27.248711+0000","last_became_active":"2023-08-25T09:40:27.248352+0000","last_became_peered":"2023-08-25T09:40:27.248352+0000","last_unstale":"2023-08-25T09:40:27.248711+0000","last_undegraded":"2023-08-25T09:40:27.248711+0000","last_fullsized":"2023-08-25T09:40:27.248711+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:26:54.224850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.222031+0000","last_change":"2023-08-25T09:40:26.445964+0000","last_active":"2023-08-25T09:40:27.222031+0000","last_peered":"2023-08-25T09:40:27.222031+0000","last_clean":"2023-08-25T09:40:27.222031+0000","last_became_active":"2023-08-25T09:40:26.444388+0000","last_became_peered":"2023-08-25T09:40:26.444388+0000","last_unstale":"2023-08-25T09:40:27.222031+0000","last_undegraded":"2023-08-25T09:40:27.222031+0000","last_fullsized":"2023-08-25T09:40:27.222031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:17:05.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.247251+0000","last_change":"2023-08-25T09:40:27.247160+0000","last_active":"2023-08-25T09:40:27.247251+0000","last_peered":"2023-08-25T09:40:27.247251+0000","last_clean":"2023-08-25T09:40:27.247251+0000","last_became_active":"2023-08-25T09:40:27.245757+0000","last_became_peered":"2023-08-25T09:40:27.245757+0000","last_unstale":"2023-08-25T09:40:27.247251+0000","last_undegraded":"2023-08-25T09:40:27.247251+0000","last_fullsized":"2023-08-25T09:40:27.247251+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:58:37.901592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.231714+0000","last_change":"2023-08-25T09:40:26.444486+0000","last_active":"2023-08-25T09:40:27.231714+0000","last_peered":"2023-08-25T09:40:27.231714+0000","last_clean":"2023-08-25T09:40:27.231714+0000","last_became_active":"2023-08-25T09:40:26.443652+0000","last_became_peered":"2023-08-25T09:40:26.443652+0000","last_unstale":"2023-08-25T09:40:27.231714+0000","last_undegraded":"2023-08-25T09:40:27.231714+0000","last_fullsized":"2023-08-25T09:40:27.231714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:36.095931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244258+0000","last_change":"2023-08-25T09:40:27.244114+0000","last_active":"2023-08-25T09:40:27.244258+0000","last_peered":"2023-08-25T09:40:27.244258+0000","last_clean":"2023-08-25T09:40:27.244258+0000","last_became_active":"2023-08-25T09:40:27.243355+0000","last_became_peered":"2023-08-25T09:40:27.243355+0000","last_unstale":"2023-08-25T09:40:27.244258+0000","last_undegraded":"2023-08-25T09:40:27.244258+0000","last_fullsized":"2023-08-25T09:40:27.244258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:44:22.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.222979+0000","last_change":"2023-08-25T09:40:26.454067+0000","last_active":"2023-08-25T09:40:27.222979+0000","last_peered":"2023-08-25T09:40:27.222979+0000","last_clean":"2023-08-25T09:40:27.222979+0000","last_became_active":"2023-08-25T09:40:26.453745+0000","last_became_peered":"2023-08-25T09:40:26.453745+0000","last_unstale":"2023-08-25T09:40:27.222979+0000","last_undegraded":"2023-08-25T09:40:27.222979+0000","last_fullsized":"2023-08-25T09:40:27.222979+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:58:36.888721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228713+0000","last_change":"2023-08-25T09:40:27.242691+0000","last_active":"2023-08-25T09:40:28.228713+0000","last_peered":"2023-08-25T09:40:28.228713+0000","last_clean":"2023-08-25T09:40:28.228713+0000","last_became_active":"2023-08-25T09:40:27.242038+0000","last_became_peered":"2023-08-25T09:40:27.242038+0000","last_unstale":"2023-08-25T09:40:28.228713+0000","last_undegraded":"2023-08-25T09:40:28.228713+0000","last_fullsized":"2023-08-25T09:40:28.228713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:35:27.213912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.234563+0000","last_change":"2023-08-25T09:40:26.452729+0000","last_active":"2023-08-25T09:40:27.234563+0000","last_peered":"2023-08-25T09:40:27.234563+0000","last_clean":"2023-08-25T09:40:27.234563+0000","last_became_active":"2023-08-25T09:40:26.452442+0000","last_became_peered":"2023-08-25T09:40:26.452442+0000","last_unstale":"2023-08-25T09:40:27.234563+0000","last_undegraded":"2023-08-25T09:40:27.234563+0000","last_fullsized":"2023-08-25T09:40:27.234563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:14:27.338263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.230347+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:40:27.230347+0000","last_peered":"2023-08-25T09:40:27.230347+0000","last_clean":"2023-08-25T09:40:27.230347+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:40:27.230347+0000","last_undegraded":"2023-08-25T09:40:27.230347+0000","last_fullsized":"2023-08-25T09:40:27.230347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:49:11.212026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228877+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:40:28.228877+0000","last_peered":"2023-08-25T09:40:28.228877+0000","last_clean":"2023-08-25T09:40:28.228877+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:40:28.228877+0000","last_undegraded":"2023-08-25T09:40:28.228877+0000","last_fullsized":"2023-08-25T09:40:28.228877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:46.734175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.235366+0000","last_change":"2023-08-25T09:40:27.246001+0000","last_active":"2023-08-25T09:40:28.235366+0000","last_peered":"2023-08-25T09:40:28.235366+0000","last_clean":"2023-08-25T09:40:28.235366+0000","last_became_active":"2023-08-25T09:40:27.245514+0000","last_became_peered":"2023-08-25T09:40:27.245514+0000","last_unstale":"2023-08-25T09:40:28.235366+0000","last_undegraded":"2023-08-25T09:40:28.235366+0000","last_fullsized":"2023-08-25T09:40:28.235366+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:33.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230085+0000","last_change":"2023-08-25T09:40:26.451368+0000","last_active":"2023-08-25T09:40:28.230085+0000","last_peered":"2023-08-25T09:40:28.230085+0000","last_clean":"2023-08-25T09:40:28.230085+0000","last_became_active":"2023-08-25T09:40:26.451070+0000","last_became_peered":"2023-08-25T09:40:26.451070+0000","last_unstale":"2023-08-25T09:40:28.230085+0000","last_undegraded":"2023-08-25T09:40:28.230085+0000","last_fullsized":"2023-08-25T09:40:28.230085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:09:20.744481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230443+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:40:28.230443+0000","last_peered":"2023-08-25T09:40:28.230443+0000","last_clean":"2023-08-25T09:40:28.230443+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:40:28.230443+0000","last_undegraded":"2023-08-25T09:40:28.230443+0000","last_fullsized":"2023-08-25T09:40:28.230443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:26.887559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.247222+0000","last_change":"2023-08-25T09:40:27.246478+0000","last_active":"2023-08-25T09:40:27.247222+0000","last_peered":"2023-08-25T09:40:27.247222+0000","last_clean":"2023-08-25T09:40:27.247222+0000","last_became_active":"2023-08-25T09:40:27.244949+0000","last_became_peered":"2023-08-25T09:40:27.244949+0000","last_unstale":"2023-08-25T09:40:27.247222+0000","last_undegraded":"2023-08-25T09:40:27.247222+0000","last_fullsized":"2023-08-25T09:40:27.247222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:17:06.713257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234692+0000","last_change":"2023-08-25T09:40:26.452544+0000","last_active":"2023-08-25T09:40:28.234692+0000","last_peered":"2023-08-25T09:40:28.234692+0000","last_clean":"2023-08-25T09:40:28.234692+0000","last_became_active":"2023-08-25T09:40:26.451907+0000","last_became_peered":"2023-08-25T09:40:26.451907+0000","last_unstale":"2023-08-25T09:40:28.234692+0000","last_undegraded":"2023-08-25T09:40:28.234692+0000","last_fullsized":"2023-08-25T09:40:28.234692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:01.406850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234952+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:40:28.234952+0000","last_peered":"2023-08-25T09:40:28.234952+0000","last_clean":"2023-08-25T09:40:28.234952+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:40:28.234952+0000","last_undegraded":"2023-08-25T09:40:28.234952+0000","last_fullsized":"2023-08-25T09:40:28.234952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:04:09.963659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.241874+0000","last_change":"2023-08-25T09:40:27.241743+0000","last_active":"2023-08-25T09:40:27.241874+0000","last_peered":"2023-08-25T09:40:27.241874+0000","last_clean":"2023-08-25T09:40:27.241874+0000","last_became_active":"2023-08-25T09:40:27.241404+0000","last_became_peered":"2023-08-25T09:40:27.241404+0000","last_unstale":"2023-08-25T09:40:27.241874+0000","last_undegraded":"2023-08-25T09:40:27.241874+0000","last_fullsized":"2023-08-25T09:40:27.241874+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:50:57.682960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234977+0000","last_change":"2023-08-25T09:40:26.457293+0000","last_active":"2023-08-25T09:40:28.234977+0000","last_peered":"2023-08-25T09:40:28.234977+0000","last_clean":"2023-08-25T09:40:28.234977+0000","last_became_active":"2023-08-25T09:40:26.457001+0000","last_became_peered":"2023-08-25T09:40:26.457001+0000","last_unstale":"2023-08-25T09:40:28.234977+0000","last_undegraded":"2023-08-25T09:40:28.234977+0000","last_fullsized":"2023-08-25T09:40:28.234977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:06:49.620284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230865+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:40:28.230865+0000","last_peered":"2023-08-25T09:40:28.230865+0000","last_clean":"2023-08-25T09:40:28.230865+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:40:28.230865+0000","last_undegraded":"2023-08-25T09:40:28.230865+0000","last_fullsized":"2023-08-25T09:40:28.230865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:57.530121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234640+0000","last_change":"2023-08-25T09:40:27.240120+0000","last_active":"2023-08-25T09:40:28.234640+0000","last_peered":"2023-08-25T09:40:28.234640+0000","last_clean":"2023-08-25T09:40:28.234640+0000","last_became_active":"2023-08-25T09:40:27.239752+0000","last_became_peered":"2023-08-25T09:40:27.239752+0000","last_unstale":"2023-08-25T09:40:28.234640+0000","last_undegraded":"2023-08-25T09:40:28.234640+0000","last_fullsized":"2023-08-25T09:40:28.234640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:42.649769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.227812+0000","last_change":"2023-08-25T09:40:26.452165+0000","last_active":"2023-08-25T09:40:27.227812+0000","last_peered":"2023-08-25T09:40:27.227812+0000","last_clean":"2023-08-25T09:40:27.227812+0000","last_became_active":"2023-08-25T09:40:26.451847+0000","last_became_peered":"2023-08-25T09:40:26.451847+0000","last_unstale":"2023-08-25T09:40:27.227812+0000","last_undegraded":"2023-08-25T09:40:27.227812+0000","last_fullsized":"2023-08-25T09:40:27.227812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:40:54.099861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.224417+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:40:27.224417+0000","last_peered":"2023-08-25T09:40:27.224417+0000","last_clean":"2023-08-25T09:40:27.224417+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:40:27.224417+0000","last_undegraded":"2023-08-25T09:40:27.224417+0000","last_fullsized":"2023-08-25T09:40:27.224417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:04:09.963659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.229294+0000","last_change":"2023-08-25T09:40:27.227785+0000","last_active":"2023-08-25T09:40:27.229294+0000","last_peered":"2023-08-25T09:40:27.229294+0000","last_clean":"2023-08-25T09:40:27.229294+0000","last_became_active":"2023-08-25T09:40:27.227153+0000","last_became_peered":"2023-08-25T09:40:27.227153+0000","last_unstale":"2023-08-25T09:40:27.229294+0000","last_undegraded":"2023-08-25T09:40:27.229294+0000","last_fullsized":"2023-08-25T09:40:27.229294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:21:50.535699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.233737+0000","last_change":"2023-08-25T09:40:26.451776+0000","last_active":"2023-08-25T09:40:27.233737+0000","last_peered":"2023-08-25T09:40:27.233737+0000","last_clean":"2023-08-25T09:40:27.233737+0000","last_became_active":"2023-08-25T09:40:26.451522+0000","last_became_peered":"2023-08-25T09:40:26.451522+0000","last_unstale":"2023-08-25T09:40:27.233737+0000","last_undegraded":"2023-08-25T09:40:27.233737+0000","last_fullsized":"2023-08-25T09:40:27.233737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:44.755044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230012+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:40:28.230012+0000","last_peered":"2023-08-25T09:40:28.230012+0000","last_clean":"2023-08-25T09:40:28.230012+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:40:28.230012+0000","last_undegraded":"2023-08-25T09:40:28.230012+0000","last_fullsized":"2023-08-25T09:40:28.230012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:05:18.585703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229087+0000","last_change":"2023-08-25T09:40:27.242878+0000","last_active":"2023-08-25T09:40:28.229087+0000","last_peered":"2023-08-25T09:40:28.229087+0000","last_clean":"2023-08-25T09:40:28.229087+0000","last_became_active":"2023-08-25T09:40:27.242417+0000","last_became_peered":"2023-08-25T09:40:27.242417+0000","last_unstale":"2023-08-25T09:40:28.229087+0000","last_undegraded":"2023-08-25T09:40:28.229087+0000","last_fullsized":"2023-08-25T09:40:28.229087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:23.006279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230151+0000","last_change":"2023-08-25T09:40:26.449575+0000","last_active":"2023-08-25T09:40:28.230151+0000","last_peered":"2023-08-25T09:40:28.230151+0000","last_clean":"2023-08-25T09:40:28.230151+0000","last_became_active":"2023-08-25T09:40:26.449039+0000","last_became_peered":"2023-08-25T09:40:26.449039+0000","last_unstale":"2023-08-25T09:40:28.230151+0000","last_undegraded":"2023-08-25T09:40:28.230151+0000","last_fullsized":"2023-08-25T09:40:28.230151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:42:56.357330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.235970+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:40:27.235970+0000","last_peered":"2023-08-25T09:40:27.235970+0000","last_clean":"2023-08-25T09:40:27.235970+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:40:27.235970+0000","last_undegraded":"2023-08-25T09:40:27.235970+0000","last_fullsized":"2023-08-25T09:40:27.235970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:23:41.616585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245795+0000","last_change":"2023-08-25T09:40:27.242527+0000","last_active":"2023-08-25T09:40:27.245795+0000","last_peered":"2023-08-25T09:40:27.245795+0000","last_clean":"2023-08-25T09:40:27.245795+0000","last_became_active":"2023-08-25T09:40:27.241958+0000","last_became_peered":"2023-08-25T09:40:27.241958+0000","last_unstale":"2023-08-25T09:40:27.245795+0000","last_undegraded":"2023-08-25T09:40:27.245795+0000","last_fullsized":"2023-08-25T09:40:27.245795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:18:58.163422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.236991+0000","last_change":"2023-08-25T09:40:26.455883+0000","last_active":"2023-08-25T09:40:27.236991+0000","last_peered":"2023-08-25T09:40:27.236991+0000","last_clean":"2023-08-25T09:40:27.236991+0000","last_became_active":"2023-08-25T09:40:26.452716+0000","last_became_peered":"2023-08-25T09:40:26.452716+0000","last_unstale":"2023-08-25T09:40:27.236991+0000","last_undegraded":"2023-08-25T09:40:27.236991+0000","last_fullsized":"2023-08-25T09:40:27.236991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:49:06.674070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230564+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:40:28.230564+0000","last_peered":"2023-08-25T09:40:28.230564+0000","last_clean":"2023-08-25T09:40:28.230564+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:40:28.230564+0000","last_undegraded":"2023-08-25T09:40:28.230564+0000","last_fullsized":"2023-08-25T09:40:28.230564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:27:58.374248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":158,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195154,"statfs":{"total":200000000,"available":199837740,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.98599999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.93100000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.194}]},{"osd":2,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.254}]},{"osd":3,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.875},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.5}]},{"osd":4,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3029999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.73},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.6499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":151,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195160,"statfs":{"total":200000000,"available":199844429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.3140000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.103}]},{"osd":3,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.367},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0600000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.234}]},{"osd":5,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.341},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.252}]},{"osd":7,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.286},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253513,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.327},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.1080000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.2749999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0589999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.383}]},{"osd":7,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.351}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195158,"statfs":{"total":200000000,"available":199842727,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.323},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.4079999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":8.5850000000000009,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.224}]},{"osd":2,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":4.1669999999999998,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.454}]},{"osd":3,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.278},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.4420000000000002,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.119}]},{"osd":6,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.863,"5min":2.863,"15min":2.863},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.3600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":158,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195154,"statfs":{"total":200000000,"available":199837720,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.099}]},{"osd":1,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.75}]},{"osd":4,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.069},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":3.496,"5min":3.496,"15min":3.496},"last":1.2769999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.399},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.3460000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":160,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195151,"statfs":{"total":200000000,"available":199835616,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.505}]},{"osd":3,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.9569999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":0.63},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1269999999999998,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.968,"5min":2.968,"15min":2.968},"last":0.746},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.194}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194584,"statfs":{"total":200000000,"available":199254073,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0469999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.26},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.496}]},{"osd":5,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.101},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.83299999999999996}]},{"osd":7,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.407},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.157}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":151,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195160,"statfs":{"total":200000000,"available":199844371,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":3.4889999999999999,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.044}]},{"osd":4,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.403},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.2609999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.216}]},{"osd":6,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.3380000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:40:32.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:40:32.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.240+0000 7f6b84512640 1 Processor -- start 2023-08-25T09:40:32.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.240+0000 7f6b84512640 1 -- start start 2023-08-25T09:40:32.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b84512640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b84512640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6b7c15aa10 0x7f6b7c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b84512640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b7c0b2680 0x7f6b7c06be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b84512640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c0b2b10 con 0x7f6b7c0b2680 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b84512640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c159d10 con 0x7f6b7c15aa10 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b84512640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ceed0 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b82aaf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b7c0b2680 0x7f6b7c06be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b82aaf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b7c0b2680 0x7f6b7c06be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45852/0 (socket says 172.21.15.70:45852) 2023-08-25T09:40:32.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b82aaf640 1 -- 172.21.15.70:0/1144219681 learned_addr learned my addr 172.21.15.70:0/1144219681 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:32.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b81aad640 1 --2- 172.21.15.70:0/1144219681 >> v2:172.21.15.155:3300/0 conn(0x7f6b7c15aa10 0x7f6b7c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.242+0000 7f6b822ae640 1 --2- 172.21.15.70:0/1144219681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.243+0000 7f6b82aaf640 1 -- 172.21.15.70:0/1144219681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0ce5b0 msgr2=0x7f6b7c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.243+0000 7f6b82aaf640 1 --2- 172.21.15.70:0/1144219681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.243+0000 7f6b82aaf640 1 -- 172.21.15.70:0/1144219681 >> v2:172.21.15.155:3300/0 conn(0x7f6b7c15aa10 msgr2=0x7f6b7c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.243+0000 7f6b82aaf640 1 --2- 172.21.15.70:0/1144219681 >> v2:172.21.15.155:3300/0 conn(0x7f6b7c15aa10 0x7f6b7c0b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.243+0000 7f6b82aaf640 1 -- 172.21.15.70:0/1144219681 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c15d3f0 con 0x7f6b7c0b2680 2023-08-25T09:40:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.244+0000 7f6b82aaf640 1 --2- 172.21.15.70:0/1144219681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0b2680 0x7f6b7c06be80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c06a7e0 tx=0x7f6b7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7828f11baf70f2a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:32.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.244+0000 7f6b812ac640 1 -- 172.21.15.70:0/1144219681 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7402ca80 con 0x7f6b7c0b2680 2023-08-25T09:40:32.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.244+0000 7f6b812ac640 1 -- 172.21.15.70:0/1144219681 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7402cbe0 con 0x7f6b7c0b2680 2023-08-25T09:40:32.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.244+0000 7f6b812ac640 1 -- 172.21.15.70:0/1144219681 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b740343f0 con 0x7f6b7c0b2680 2023-08-25T09:40:32.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 -- 172.21.15.70:0/1144219681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0b2680 msgr2=0x7f6b7c06be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 --2- 172.21.15.70:0/1144219681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0b2680 0x7f6b7c06be80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c06a7e0 tx=0x7f6b7402bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 -- 172.21.15.70:0/1144219681 shutdown_connections 2023-08-25T09:40:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 --2- 172.21.15.70:0/1144219681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0b2680 0x7f6b7c06be80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 --2- 172.21.15.70:0/1144219681 >> v2:172.21.15.155:3300/0 conn(0x7f6b7c15aa10 0x7f6b7c0b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 --2- 172.21.15.70:0/1144219681 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 -- 172.21.15.70:0/1144219681 >> 172.21.15.70:0/1144219681 conn(0x7f6b7c069850 msgr2=0x7f6b7c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 -- 172.21.15.70:0/1144219681 shutdown_connections 2023-08-25T09:40:32.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.245+0000 7f6b84512640 1 -- 172.21.15.70:0/1144219681 wait complete. 2023-08-25T09:40:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.247+0000 7f6b84512640 1 Processor -- start 2023-08-25T09:40:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.247+0000 7f6b84512640 1 -- start start 2023-08-25T09:40:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b84512640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0b2680 0x7f6b7c14d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b84512640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b84512640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b7c15aa10 0x7f6b7c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b84512640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ae2c0 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b84512640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0cbdb0 con 0x7f6b7c15aa10 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b84512640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0cc010 con 0x7f6b7c0b2680 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b81aad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b81aad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45860/0 (socket says 172.21.15.70:45860) 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.248+0000 7f6b81aad640 1 -- 172.21.15.70:0/2604237315 learned_addr learned my addr 172.21.15.70:0/2604237315 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:32.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b822ae640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0b2680 0x7f6b7c14d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b82aaf640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b7c15aa10 0x7f6b7c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b81aad640 1 -- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0b2680 msgr2=0x7f6b7c14d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b81aad640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0b2680 0x7f6b7c14d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b81aad640 1 -- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b7c15aa10 msgr2=0x7f6b7c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b81aad640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b7c15aa10 0x7f6b7c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b81aad640 1 -- 172.21.15.70:0/2604237315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74009b30 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b82aaf640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b7c15aa10 0x7f6b7c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:32.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b81aad640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0cdf60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d990 tx=0x7f6b6c00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c007a90 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c007bf0 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c010470 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c0cc2f0 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.249+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c0ae510 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.251+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c04e420 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.251+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6b6c0105d0 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.252+0000 7f6b72ffd640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b6006a050 0x7f6b6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.252+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b6c0887d0 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.253+0000 7f6b822ae640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b6006a050 0x7f6b6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.253+0000 7f6b822ae640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b6006a050 0x7f6b6006c510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6b78006fd0 tx=0x7f6b78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:32.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.257+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6b6c053f50 con 0x7f6b7c0ce5b0 2023-08-25T09:40:32.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.443+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b7c15d6b0 con 0x7f6b6006a050 2023-08-25T09:40:32.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.466+0000 7f6b72ffd640 1 -- 172.21.15.70:0/2604237315 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302939 (secure 0 0 0) 0x7f6b7800b200 con 0x7f6b6006a050 2023-08-25T09:40:32.469 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:32.473 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:40:32.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b6006a050 msgr2=0x7f6b6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b6006a050 0x7f6b6006c510 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6b78006fd0 tx=0x7f6b78008040 comp rx=0 tx=0).stop 2023-08-25T09:40:32.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 msgr2=0x7f6b7c0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0cdf60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d990 tx=0x7f6b6c00de50 comp rx=0 tx=0).stop 2023-08-25T09:40:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 shutdown_connections 2023-08-25T09:40:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b6006a050 0x7f6b6006c510 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b7c15aa10 0x7f6b7c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b7c0ce5b0 0x7f6b7c0cdf60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 --2- 172.21.15.70:0/2604237315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b7c0b2680 0x7f6b7c14d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.473+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 >> 172.21.15.70:0/2604237315 conn(0x7f6b7c069850 msgr2=0x7f6b7c156e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:32.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.474+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 shutdown_connections 2023-08-25T09:40:32.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.474+0000 7f6b84512640 1 -- 172.21.15.70:0/2604237315 wait complete. 2023-08-25T09:40:32.497 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-08-25T09:40:31.350026+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2392,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560107,"statfs":{"total":1600000000,"available":1597550189,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.942649"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.235603+0000","last_change":"2023-08-25T09:40:27.235517+0000","last_active":"2023-08-25T09:40:27.235603+0000","last_peered":"2023-08-25T09:40:27.235603+0000","last_clean":"2023-08-25T09:40:27.235603+0000","last_became_active":"2023-08-25T09:40:27.234495+0000","last_became_peered":"2023-08-25T09:40:27.234495+0000","last_unstale":"2023-08-25T09:40:27.235603+0000","last_undegraded":"2023-08-25T09:40:27.235603+0000","last_fullsized":"2023-08-25T09:40:27.235603+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:00.171742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.237586+0000","last_change":"2023-08-25T09:40:27.237441+0000","last_active":"2023-08-25T09:40:27.237586+0000","last_peered":"2023-08-25T09:40:27.237586+0000","last_clean":"2023-08-25T09:40:27.237586+0000","last_became_active":"2023-08-25T09:40:27.235984+0000","last_became_peered":"2023-08-25T09:40:27.235984+0000","last_unstale":"2023-08-25T09:40:27.237586+0000","last_undegraded":"2023-08-25T09:40:27.237586+0000","last_fullsized":"2023-08-25T09:40:27.237586+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:53.993727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251301+0000","last_change":"2023-08-25T09:40:27.251238+0000","last_active":"2023-08-25T09:40:27.251301+0000","last_peered":"2023-08-25T09:40:27.251301+0000","last_clean":"2023-08-25T09:40:27.251301+0000","last_became_active":"2023-08-25T09:40:27.250822+0000","last_became_peered":"2023-08-25T09:40:27.250822+0000","last_unstale":"2023-08-25T09:40:27.251301+0000","last_undegraded":"2023-08-25T09:40:27.251301+0000","last_fullsized":"2023-08-25T09:40:27.251301+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:04.631849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251442+0000","last_change":"2023-08-25T09:40:27.251383+0000","last_active":"2023-08-25T09:40:27.251442+0000","last_peered":"2023-08-25T09:40:27.251442+0000","last_clean":"2023-08-25T09:40:27.251442+0000","last_became_active":"2023-08-25T09:40:27.250952+0000","last_became_peered":"2023-08-25T09:40:27.250952+0000","last_unstale":"2023-08-25T09:40:27.251442+0000","last_undegraded":"2023-08-25T09:40:27.251442+0000","last_fullsized":"2023-08-25T09:40:27.251442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:13.257224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234652+0000","last_change":"2023-08-25T09:40:27.245778+0000","last_active":"2023-08-25T09:40:28.234652+0000","last_peered":"2023-08-25T09:40:28.234652+0000","last_clean":"2023-08-25T09:40:28.234652+0000","last_became_active":"2023-08-25T09:40:27.244955+0000","last_became_peered":"2023-08-25T09:40:27.244955+0000","last_unstale":"2023-08-25T09:40:28.234652+0000","last_undegraded":"2023-08-25T09:40:28.234652+0000","last_fullsized":"2023-08-25T09:40:28.234652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:17.033560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.236105+0000","last_change":"2023-08-25T09:40:27.235983+0000","last_active":"2023-08-25T09:40:27.236105+0000","last_peered":"2023-08-25T09:40:27.236105+0000","last_clean":"2023-08-25T09:40:27.236105+0000","last_became_active":"2023-08-25T09:40:27.235310+0000","last_became_peered":"2023-08-25T09:40:27.235310+0000","last_unstale":"2023-08-25T09:40:27.236105+0000","last_undegraded":"2023-08-25T09:40:27.236105+0000","last_fullsized":"2023-08-25T09:40:27.236105+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:18:58.163422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.247431+0000","last_change":"2023-08-25T09:40:27.247332+0000","last_active":"2023-08-25T09:40:27.247431+0000","last_peered":"2023-08-25T09:40:27.247431+0000","last_clean":"2023-08-25T09:40:27.247431+0000","last_became_active":"2023-08-25T09:40:27.246918+0000","last_became_peered":"2023-08-25T09:40:27.246918+0000","last_unstale":"2023-08-25T09:40:27.247431+0000","last_undegraded":"2023-08-25T09:40:27.247431+0000","last_fullsized":"2023-08-25T09:40:27.247431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:06:50.633155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244594+0000","last_change":"2023-08-25T09:40:27.244464+0000","last_active":"2023-08-25T09:40:27.244594+0000","last_peered":"2023-08-25T09:40:27.244594+0000","last_clean":"2023-08-25T09:40:27.244594+0000","last_became_active":"2023-08-25T09:40:27.243711+0000","last_became_peered":"2023-08-25T09:40:27.243711+0000","last_unstale":"2023-08-25T09:40:27.244594+0000","last_undegraded":"2023-08-25T09:40:27.244594+0000","last_fullsized":"2023-08-25T09:40:27.244594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:10.467343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234299+0000","last_change":"2023-08-25T09:40:27.232229+0000","last_active":"2023-08-25T09:40:28.234299+0000","last_peered":"2023-08-25T09:40:28.234299+0000","last_clean":"2023-08-25T09:40:28.234299+0000","last_became_active":"2023-08-25T09:40:27.230124+0000","last_became_peered":"2023-08-25T09:40:27.230124+0000","last_unstale":"2023-08-25T09:40:28.234299+0000","last_undegraded":"2023-08-25T09:40:28.234299+0000","last_fullsized":"2023-08-25T09:40:28.234299+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:27.155616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.252791+0000","last_change":"2023-08-25T09:40:27.252695+0000","last_active":"2023-08-25T09:40:27.252791+0000","last_peered":"2023-08-25T09:40:27.252791+0000","last_clean":"2023-08-25T09:40:27.252791+0000","last_became_active":"2023-08-25T09:40:27.250521+0000","last_became_peered":"2023-08-25T09:40:27.250521+0000","last_unstale":"2023-08-25T09:40:27.252791+0000","last_undegraded":"2023-08-25T09:40:27.252791+0000","last_fullsized":"2023-08-25T09:40:27.252791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:09.522369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.253552+0000","last_change":"2023-08-25T09:40:27.252864+0000","last_active":"2023-08-25T09:40:27.253552+0000","last_peered":"2023-08-25T09:40:27.253552+0000","last_clean":"2023-08-25T09:40:27.253552+0000","last_became_active":"2023-08-25T09:40:27.250079+0000","last_became_peered":"2023-08-25T09:40:27.250079+0000","last_unstale":"2023-08-25T09:40:27.253552+0000","last_undegraded":"2023-08-25T09:40:27.253552+0000","last_fullsized":"2023-08-25T09:40:27.253552+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:54:41.419105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234625+0000","last_change":"2023-08-25T09:40:27.240439+0000","last_active":"2023-08-25T09:40:28.234625+0000","last_peered":"2023-08-25T09:40:28.234625+0000","last_clean":"2023-08-25T09:40:28.234625+0000","last_became_active":"2023-08-25T09:40:27.239598+0000","last_became_peered":"2023-08-25T09:40:27.239598+0000","last_unstale":"2023-08-25T09:40:28.234625+0000","last_undegraded":"2023-08-25T09:40:28.234625+0000","last_fullsized":"2023-08-25T09:40:28.234625+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:40.711092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244217+0000","last_change":"2023-08-25T09:40:27.242649+0000","last_active":"2023-08-25T09:40:27.244217+0000","last_peered":"2023-08-25T09:40:27.244217+0000","last_clean":"2023-08-25T09:40:27.244217+0000","last_became_active":"2023-08-25T09:40:27.242141+0000","last_became_peered":"2023-08-25T09:40:27.242141+0000","last_unstale":"2023-08-25T09:40:27.244217+0000","last_undegraded":"2023-08-25T09:40:27.244217+0000","last_fullsized":"2023-08-25T09:40:27.244217+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:35:52.785789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232677+0000","last_change":"2023-08-25T09:40:27.253854+0000","last_active":"2023-08-25T09:40:28.232677+0000","last_peered":"2023-08-25T09:40:28.232677+0000","last_clean":"2023-08-25T09:40:28.232677+0000","last_became_active":"2023-08-25T09:40:27.253569+0000","last_became_peered":"2023-08-25T09:40:27.253569+0000","last_unstale":"2023-08-25T09:40:28.232677+0000","last_undegraded":"2023-08-25T09:40:28.232677+0000","last_fullsized":"2023-08-25T09:40:28.232677+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:55.421015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229312+0000","last_change":"2023-08-25T09:40:27.225902+0000","last_active":"2023-08-25T09:40:28.229312+0000","last_peered":"2023-08-25T09:40:28.229312+0000","last_clean":"2023-08-25T09:40:28.229312+0000","last_became_active":"2023-08-25T09:40:27.225241+0000","last_became_peered":"2023-08-25T09:40:27.225241+0000","last_unstale":"2023-08-25T09:40:28.229312+0000","last_undegraded":"2023-08-25T09:40:28.229312+0000","last_fullsized":"2023-08-25T09:40:28.229312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:31.531610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.246298+0000","last_change":"2023-08-25T09:40:27.246221+0000","last_active":"2023-08-25T09:40:27.246298+0000","last_peered":"2023-08-25T09:40:27.246298+0000","last_clean":"2023-08-25T09:40:27.246298+0000","last_became_active":"2023-08-25T09:40:27.245250+0000","last_became_peered":"2023-08-25T09:40:27.245250+0000","last_unstale":"2023-08-25T09:40:27.246298+0000","last_undegraded":"2023-08-25T09:40:27.246298+0000","last_fullsized":"2023-08-25T09:40:27.246298+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:36:25.767167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229744+0000","last_change":"2023-08-25T09:40:27.249907+0000","last_active":"2023-08-25T09:40:28.229744+0000","last_peered":"2023-08-25T09:40:28.229744+0000","last_clean":"2023-08-25T09:40:28.229744+0000","last_became_active":"2023-08-25T09:40:27.249643+0000","last_became_peered":"2023-08-25T09:40:27.249643+0000","last_unstale":"2023-08-25T09:40:28.229744+0000","last_undegraded":"2023-08-25T09:40:28.229744+0000","last_fullsized":"2023-08-25T09:40:28.229744+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:01.770533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228965+0000","last_change":"2023-08-25T09:40:27.250612+0000","last_active":"2023-08-25T09:40:28.228965+0000","last_peered":"2023-08-25T09:40:28.228965+0000","last_clean":"2023-08-25T09:40:28.228965+0000","last_became_active":"2023-08-25T09:40:27.250306+0000","last_became_peered":"2023-08-25T09:40:27.250306+0000","last_unstale":"2023-08-25T09:40:28.228965+0000","last_undegraded":"2023-08-25T09:40:28.228965+0000","last_fullsized":"2023-08-25T09:40:28.228965+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:34.191998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.240627+0000","last_change":"2023-08-25T09:40:27.240407+0000","last_active":"2023-08-25T09:40:27.240627+0000","last_peered":"2023-08-25T09:40:27.240627+0000","last_clean":"2023-08-25T09:40:27.240627+0000","last_became_active":"2023-08-25T09:40:27.239935+0000","last_became_peered":"2023-08-25T09:40:27.239935+0000","last_unstale":"2023-08-25T09:40:27.240627+0000","last_undegraded":"2023-08-25T09:40:27.240627+0000","last_fullsized":"2023-08-25T09:40:27.240627+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:37.766243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245860+0000","last_change":"2023-08-25T09:40:27.244252+0000","last_active":"2023-08-25T09:40:27.245860+0000","last_peered":"2023-08-25T09:40:27.245860+0000","last_clean":"2023-08-25T09:40:27.245860+0000","last_became_active":"2023-08-25T09:40:27.243961+0000","last_became_peered":"2023-08-25T09:40:27.243961+0000","last_unstale":"2023-08-25T09:40:27.245860+0000","last_undegraded":"2023-08-25T09:40:27.245860+0000","last_fullsized":"2023-08-25T09:40:27.245860+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:33.793228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251868+0000","last_change":"2023-08-25T09:40:27.251472+0000","last_active":"2023-08-25T09:40:27.251868+0000","last_peered":"2023-08-25T09:40:27.251868+0000","last_clean":"2023-08-25T09:40:27.251868+0000","last_became_active":"2023-08-25T09:40:27.246490+0000","last_became_peered":"2023-08-25T09:40:27.246490+0000","last_unstale":"2023-08-25T09:40:27.251868+0000","last_undegraded":"2023-08-25T09:40:27.251868+0000","last_fullsized":"2023-08-25T09:40:27.251868+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:18.035205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.248131+0000","last_change":"2023-08-25T09:40:27.247907+0000","last_active":"2023-08-25T09:40:27.248131+0000","last_peered":"2023-08-25T09:40:27.248131+0000","last_clean":"2023-08-25T09:40:27.248131+0000","last_became_active":"2023-08-25T09:40:27.245450+0000","last_became_peered":"2023-08-25T09:40:27.245450+0000","last_unstale":"2023-08-25T09:40:27.248131+0000","last_undegraded":"2023-08-25T09:40:27.248131+0000","last_fullsized":"2023-08-25T09:40:27.248131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:01.001152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229275+0000","last_change":"2023-08-25T09:40:27.240258+0000","last_active":"2023-08-25T09:40:28.229275+0000","last_peered":"2023-08-25T09:40:28.229275+0000","last_clean":"2023-08-25T09:40:28.229275+0000","last_became_active":"2023-08-25T09:40:27.238975+0000","last_became_peered":"2023-08-25T09:40:27.238975+0000","last_unstale":"2023-08-25T09:40:28.229275+0000","last_undegraded":"2023-08-25T09:40:28.229275+0000","last_fullsized":"2023-08-25T09:40:28.229275+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:02.743173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.250719+0000","last_change":"2023-08-25T09:40:27.250649+0000","last_active":"2023-08-25T09:40:27.250719+0000","last_peered":"2023-08-25T09:40:27.250719+0000","last_clean":"2023-08-25T09:40:27.250719+0000","last_became_active":"2023-08-25T09:40:27.249259+0000","last_became_peered":"2023-08-25T09:40:27.249259+0000","last_unstale":"2023-08-25T09:40:27.250719+0000","last_undegraded":"2023-08-25T09:40:27.250719+0000","last_fullsized":"2023-08-25T09:40:27.250719+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:44.759659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232021+0000","last_change":"2023-08-25T09:40:27.245788+0000","last_active":"2023-08-25T09:40:28.232021+0000","last_peered":"2023-08-25T09:40:28.232021+0000","last_clean":"2023-08-25T09:40:28.232021+0000","last_became_active":"2023-08-25T09:40:27.245409+0000","last_became_peered":"2023-08-25T09:40:27.245409+0000","last_unstale":"2023-08-25T09:40:28.232021+0000","last_undegraded":"2023-08-25T09:40:28.232021+0000","last_fullsized":"2023-08-25T09:40:28.232021+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:44.399476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251083+0000","last_change":"2023-08-25T09:40:27.250984+0000","last_active":"2023-08-25T09:40:27.251083+0000","last_peered":"2023-08-25T09:40:27.251083+0000","last_clean":"2023-08-25T09:40:27.251083+0000","last_became_active":"2023-08-25T09:40:27.250503+0000","last_became_peered":"2023-08-25T09:40:27.250503+0000","last_unstale":"2023-08-25T09:40:27.251083+0000","last_undegraded":"2023-08-25T09:40:27.251083+0000","last_fullsized":"2023-08-25T09:40:27.251083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:06:53.168893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245832+0000","last_change":"2023-08-25T09:40:27.243524+0000","last_active":"2023-08-25T09:40:27.245832+0000","last_peered":"2023-08-25T09:40:27.245832+0000","last_clean":"2023-08-25T09:40:27.245832+0000","last_became_active":"2023-08-25T09:40:27.243078+0000","last_became_peered":"2023-08-25T09:40:27.243078+0000","last_unstale":"2023-08-25T09:40:27.245832+0000","last_undegraded":"2023-08-25T09:40:27.245832+0000","last_fullsized":"2023-08-25T09:40:27.245832+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:32.287541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244186+0000","last_change":"2023-08-25T09:40:27.242496+0000","last_active":"2023-08-25T09:40:27.244186+0000","last_peered":"2023-08-25T09:40:27.244186+0000","last_clean":"2023-08-25T09:40:27.244186+0000","last_became_active":"2023-08-25T09:40:27.241848+0000","last_became_peered":"2023-08-25T09:40:27.241848+0000","last_unstale":"2023-08-25T09:40:27.244186+0000","last_undegraded":"2023-08-25T09:40:27.244186+0000","last_fullsized":"2023-08-25T09:40:27.244186+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:52:38.157083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229429+0000","last_change":"2023-08-25T09:40:27.232893+0000","last_active":"2023-08-25T09:40:28.229429+0000","last_peered":"2023-08-25T09:40:28.229429+0000","last_clean":"2023-08-25T09:40:28.229429+0000","last_became_active":"2023-08-25T09:40:27.232114+0000","last_became_peered":"2023-08-25T09:40:27.232114+0000","last_unstale":"2023-08-25T09:40:28.229429+0000","last_undegraded":"2023-08-25T09:40:28.229429+0000","last_fullsized":"2023-08-25T09:40:28.229429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:16:35.713957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229638+0000","last_change":"2023-08-25T09:40:27.232578+0000","last_active":"2023-08-25T09:40:28.229638+0000","last_peered":"2023-08-25T09:40:28.229638+0000","last_clean":"2023-08-25T09:40:28.229638+0000","last_became_active":"2023-08-25T09:40:27.231879+0000","last_became_peered":"2023-08-25T09:40:27.231879+0000","last_unstale":"2023-08-25T09:40:28.229638+0000","last_undegraded":"2023-08-25T09:40:28.229638+0000","last_fullsized":"2023-08-25T09:40:28.229638+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:49.537369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.246161+0000","last_change":"2023-08-25T09:40:27.246025+0000","last_active":"2023-08-25T09:40:27.246161+0000","last_peered":"2023-08-25T09:40:27.246161+0000","last_clean":"2023-08-25T09:40:27.246161+0000","last_became_active":"2023-08-25T09:40:27.244931+0000","last_became_peered":"2023-08-25T09:40:27.244931+0000","last_unstale":"2023-08-25T09:40:27.246161+0000","last_undegraded":"2023-08-25T09:40:27.246161+0000","last_fullsized":"2023-08-25T09:40:27.246161+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:14.272870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.250168+0000","last_change":"2023-08-25T09:40:27.250008+0000","last_active":"2023-08-25T09:40:27.250168+0000","last_peered":"2023-08-25T09:40:27.250168+0000","last_clean":"2023-08-25T09:40:27.250168+0000","last_became_active":"2023-08-25T09:40:27.249242+0000","last_became_peered":"2023-08-25T09:40:27.249242+0000","last_unstale":"2023-08-25T09:40:27.250168+0000","last_undegraded":"2023-08-25T09:40:27.250168+0000","last_fullsized":"2023-08-25T09:40:27.250168+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:14.005564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228630+0000","last_change":"2023-08-25T09:40:27.247110+0000","last_active":"2023-08-25T09:40:28.228630+0000","last_peered":"2023-08-25T09:40:28.228630+0000","last_clean":"2023-08-25T09:40:28.228630+0000","last_became_active":"2023-08-25T09:40:27.246745+0000","last_became_peered":"2023-08-25T09:40:27.246745+0000","last_unstale":"2023-08-25T09:40:28.228630+0000","last_undegraded":"2023-08-25T09:40:28.228630+0000","last_fullsized":"2023-08-25T09:40:28.228630+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:15.172560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232298+0000","last_change":"2023-08-25T09:40:26.440045+0000","last_active":"2023-08-25T09:40:28.232298+0000","last_peered":"2023-08-25T09:40:28.232298+0000","last_clean":"2023-08-25T09:40:28.232298+0000","last_became_active":"2023-08-25T09:40:26.439784+0000","last_became_peered":"2023-08-25T09:40:26.439784+0000","last_unstale":"2023-08-25T09:40:28.232298+0000","last_undegraded":"2023-08-25T09:40:28.232298+0000","last_fullsized":"2023-08-25T09:40:28.232298+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:49:28.490921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.241188+0000","last_change":"2023-08-25T09:40:27.241070+0000","last_active":"2023-08-25T09:40:27.241188+0000","last_peered":"2023-08-25T09:40:27.241188+0000","last_clean":"2023-08-25T09:40:27.241188+0000","last_became_active":"2023-08-25T09:40:27.239890+0000","last_became_peered":"2023-08-25T09:40:27.239890+0000","last_unstale":"2023-08-25T09:40:27.241188+0000","last_undegraded":"2023-08-25T09:40:27.241188+0000","last_fullsized":"2023-08-25T09:40:27.241188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:09.741049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.449633+0000","last_change":"2023-08-25T09:40:26.446455+0000","last_active":"2023-08-25T09:40:27.449633+0000","last_peered":"2023-08-25T09:40:27.449633+0000","last_clean":"2023-08-25T09:40:27.449633+0000","last_became_active":"2023-08-25T09:40:26.445291+0000","last_became_peered":"2023-08-25T09:40:26.445291+0000","last_unstale":"2023-08-25T09:40:27.449633+0000","last_undegraded":"2023-08-25T09:40:27.449633+0000","last_fullsized":"2023-08-25T09:40:27.449633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:32.696998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229793+0000","last_change":"2023-08-25T09:40:27.250075+0000","last_active":"2023-08-25T09:40:28.229793+0000","last_peered":"2023-08-25T09:40:28.229793+0000","last_clean":"2023-08-25T09:40:28.229793+0000","last_became_active":"2023-08-25T09:40:27.249635+0000","last_became_peered":"2023-08-25T09:40:27.249635+0000","last_unstale":"2023-08-25T09:40:28.229793+0000","last_undegraded":"2023-08-25T09:40:28.229793+0000","last_fullsized":"2023-08-25T09:40:28.229793+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:42.048174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.447999+0000","last_change":"2023-08-25T09:40:26.453893+0000","last_active":"2023-08-25T09:40:27.447999+0000","last_peered":"2023-08-25T09:40:27.447999+0000","last_clean":"2023-08-25T09:40:27.447999+0000","last_became_active":"2023-08-25T09:40:26.452425+0000","last_became_peered":"2023-08-25T09:40:26.452425+0000","last_unstale":"2023-08-25T09:40:27.447999+0000","last_undegraded":"2023-08-25T09:40:27.447999+0000","last_fullsized":"2023-08-25T09:40:27.447999+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:44:08.250945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.239607+0000","last_change":"2023-08-25T09:40:27.239458+0000","last_active":"2023-08-25T09:40:27.239607+0000","last_peered":"2023-08-25T09:40:27.239607+0000","last_clean":"2023-08-25T09:40:27.239607+0000","last_became_active":"2023-08-25T09:40:27.238127+0000","last_became_peered":"2023-08-25T09:40:27.238127+0000","last_unstale":"2023-08-25T09:40:27.239607+0000","last_undegraded":"2023-08-25T09:40:27.239607+0000","last_fullsized":"2023-08-25T09:40:27.239607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:17.133840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232423+0000","last_change":"2023-08-25T09:40:26.453980+0000","last_active":"2023-08-25T09:40:28.232423+0000","last_peered":"2023-08-25T09:40:28.232423+0000","last_clean":"2023-08-25T09:40:28.232423+0000","last_became_active":"2023-08-25T09:40:26.453721+0000","last_became_peered":"2023-08-25T09:40:26.453721+0000","last_unstale":"2023-08-25T09:40:28.232423+0000","last_undegraded":"2023-08-25T09:40:28.232423+0000","last_fullsized":"2023-08-25T09:40:28.232423+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:32.562036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.238635+0000","last_change":"2023-08-25T09:40:27.238322+0000","last_active":"2023-08-25T09:40:27.238635+0000","last_peered":"2023-08-25T09:40:27.238635+0000","last_clean":"2023-08-25T09:40:27.238635+0000","last_became_active":"2023-08-25T09:40:27.237962+0000","last_became_peered":"2023-08-25T09:40:27.237962+0000","last_unstale":"2023-08-25T09:40:27.238635+0000","last_undegraded":"2023-08-25T09:40:27.238635+0000","last_fullsized":"2023-08-25T09:40:27.238635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:46.020827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.231862+0000","last_change":"2023-08-25T09:40:26.451828+0000","last_active":"2023-08-25T09:40:28.231862+0000","last_peered":"2023-08-25T09:40:28.231862+0000","last_clean":"2023-08-25T09:40:28.231862+0000","last_became_active":"2023-08-25T09:40:26.451553+0000","last_became_peered":"2023-08-25T09:40:26.451553+0000","last_unstale":"2023-08-25T09:40:28.231862+0000","last_undegraded":"2023-08-25T09:40:28.231862+0000","last_fullsized":"2023-08-25T09:40:28.231862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:28:47.337275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.246392+0000","last_change":"2023-08-25T09:40:27.246200+0000","last_active":"2023-08-25T09:40:27.246392+0000","last_peered":"2023-08-25T09:40:27.246392+0000","last_clean":"2023-08-25T09:40:27.246392+0000","last_became_active":"2023-08-25T09:40:27.244520+0000","last_became_peered":"2023-08-25T09:40:27.244520+0000","last_unstale":"2023-08-25T09:40:27.246392+0000","last_undegraded":"2023-08-25T09:40:27.246392+0000","last_fullsized":"2023-08-25T09:40:27.246392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:53.425898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.446844+0000","last_change":"2023-08-25T09:40:26.447532+0000","last_active":"2023-08-25T09:40:27.446844+0000","last_peered":"2023-08-25T09:40:27.446844+0000","last_clean":"2023-08-25T09:40:27.446844+0000","last_became_active":"2023-08-25T09:40:26.447193+0000","last_became_peered":"2023-08-25T09:40:26.447193+0000","last_unstale":"2023-08-25T09:40:27.446844+0000","last_undegraded":"2023-08-25T09:40:27.446844+0000","last_fullsized":"2023-08-25T09:40:27.446844+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:34:27.723860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229575+0000","last_change":"2023-08-25T09:40:27.240487+0000","last_active":"2023-08-25T09:40:28.229575+0000","last_peered":"2023-08-25T09:40:28.229575+0000","last_clean":"2023-08-25T09:40:28.229575+0000","last_became_active":"2023-08-25T09:40:27.239763+0000","last_became_peered":"2023-08-25T09:40:27.239763+0000","last_unstale":"2023-08-25T09:40:28.229575+0000","last_undegraded":"2023-08-25T09:40:28.229575+0000","last_fullsized":"2023-08-25T09:40:28.229575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:40.922903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.236608+0000","last_change":"2023-08-25T09:40:26.453258+0000","last_active":"2023-08-25T09:40:27.236608+0000","last_peered":"2023-08-25T09:40:27.236608+0000","last_clean":"2023-08-25T09:40:27.236608+0000","last_became_active":"2023-08-25T09:40:26.451293+0000","last_became_peered":"2023-08-25T09:40:26.451293+0000","last_unstale":"2023-08-25T09:40:27.236608+0000","last_undegraded":"2023-08-25T09:40:27.236608+0000","last_fullsized":"2023-08-25T09:40:27.236608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:23:42.700979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249923+0000","last_change":"2023-08-25T09:40:27.249759+0000","last_active":"2023-08-25T09:40:27.249923+0000","last_peered":"2023-08-25T09:40:27.249923+0000","last_clean":"2023-08-25T09:40:27.249923+0000","last_became_active":"2023-08-25T09:40:27.248606+0000","last_became_peered":"2023-08-25T09:40:27.248606+0000","last_unstale":"2023-08-25T09:40:27.249923+0000","last_undegraded":"2023-08-25T09:40:27.249923+0000","last_fullsized":"2023-08-25T09:40:27.249923+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:32.287541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.231324+0000","last_change":"2023-08-25T09:40:26.444725+0000","last_active":"2023-08-25T09:40:27.231324+0000","last_peered":"2023-08-25T09:40:27.231324+0000","last_clean":"2023-08-25T09:40:27.231324+0000","last_became_active":"2023-08-25T09:40:26.444143+0000","last_became_peered":"2023-08-25T09:40:26.444143+0000","last_unstale":"2023-08-25T09:40:27.231324+0000","last_undegraded":"2023-08-25T09:40:27.231324+0000","last_fullsized":"2023-08-25T09:40:27.231324+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:37:31.263967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.251316+0000","last_change":"2023-08-25T09:40:27.251243+0000","last_active":"2023-08-25T09:40:27.251316+0000","last_peered":"2023-08-25T09:40:27.251316+0000","last_clean":"2023-08-25T09:40:27.251316+0000","last_became_active":"2023-08-25T09:40:27.250402+0000","last_became_peered":"2023-08-25T09:40:27.250402+0000","last_unstale":"2023-08-25T09:40:27.251316+0000","last_undegraded":"2023-08-25T09:40:27.251316+0000","last_fullsized":"2023-08-25T09:40:27.251316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:53:25.211130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.227452+0000","last_change":"2023-08-25T09:40:26.451421+0000","last_active":"2023-08-25T09:40:27.227452+0000","last_peered":"2023-08-25T09:40:27.227452+0000","last_clean":"2023-08-25T09:40:27.227452+0000","last_became_active":"2023-08-25T09:40:26.451104+0000","last_became_peered":"2023-08-25T09:40:26.451104+0000","last_unstale":"2023-08-25T09:40:27.227452+0000","last_undegraded":"2023-08-25T09:40:27.227452+0000","last_fullsized":"2023-08-25T09:40:27.227452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:49:47.084520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232044+0000","last_change":"2023-08-25T09:40:27.251131+0000","last_active":"2023-08-25T09:40:28.232044+0000","last_peered":"2023-08-25T09:40:28.232044+0000","last_clean":"2023-08-25T09:40:28.232044+0000","last_became_active":"2023-08-25T09:40:27.250801+0000","last_became_peered":"2023-08-25T09:40:27.250801+0000","last_unstale":"2023-08-25T09:40:28.232044+0000","last_undegraded":"2023-08-25T09:40:28.232044+0000","last_fullsized":"2023-08-25T09:40:28.232044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:16.054202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229405+0000","last_change":"2023-08-25T09:40:26.445211+0000","last_active":"2023-08-25T09:40:28.229405+0000","last_peered":"2023-08-25T09:40:28.229405+0000","last_clean":"2023-08-25T09:40:28.229405+0000","last_became_active":"2023-08-25T09:40:26.444959+0000","last_became_peered":"2023-08-25T09:40:26.444959+0000","last_unstale":"2023-08-25T09:40:28.229405+0000","last_undegraded":"2023-08-25T09:40:28.229405+0000","last_fullsized":"2023-08-25T09:40:28.229405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:22.438085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249873+0000","last_change":"2023-08-25T09:40:27.249745+0000","last_active":"2023-08-25T09:40:27.249873+0000","last_peered":"2023-08-25T09:40:27.249873+0000","last_clean":"2023-08-25T09:40:27.249873+0000","last_became_active":"2023-08-25T09:40:27.248763+0000","last_became_peered":"2023-08-25T09:40:27.248763+0000","last_unstale":"2023-08-25T09:40:27.249873+0000","last_undegraded":"2023-08-25T09:40:27.249873+0000","last_fullsized":"2023-08-25T09:40:27.249873+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:31.717010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229926+0000","last_change":"2023-08-25T09:40:26.449338+0000","last_active":"2023-08-25T09:40:28.229926+0000","last_peered":"2023-08-25T09:40:28.229926+0000","last_clean":"2023-08-25T09:40:28.229926+0000","last_became_active":"2023-08-25T09:40:26.448092+0000","last_became_peered":"2023-08-25T09:40:26.448092+0000","last_unstale":"2023-08-25T09:40:28.229926+0000","last_undegraded":"2023-08-25T09:40:28.229926+0000","last_fullsized":"2023-08-25T09:40:28.229926+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:18:58.605071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245949+0000","last_change":"2023-08-25T09:40:27.245865+0000","last_active":"2023-08-25T09:40:27.245949+0000","last_peered":"2023-08-25T09:40:27.245949+0000","last_clean":"2023-08-25T09:40:27.245949+0000","last_became_active":"2023-08-25T09:40:27.244521+0000","last_became_peered":"2023-08-25T09:40:27.244521+0000","last_unstale":"2023-08-25T09:40:27.245949+0000","last_undegraded":"2023-08-25T09:40:27.245949+0000","last_fullsized":"2023-08-25T09:40:27.245949+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:33.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.450784+0000","last_change":"2023-08-25T09:40:26.453725+0000","last_active":"2023-08-25T09:40:27.450784+0000","last_peered":"2023-08-25T09:40:27.450784+0000","last_clean":"2023-08-25T09:40:27.450784+0000","last_became_active":"2023-08-25T09:40:26.451984+0000","last_became_peered":"2023-08-25T09:40:26.451984+0000","last_unstale":"2023-08-25T09:40:27.450784+0000","last_undegraded":"2023-08-25T09:40:27.450784+0000","last_fullsized":"2023-08-25T09:40:27.450784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:27.844837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.250502+0000","last_change":"2023-08-25T09:40:27.250422+0000","last_active":"2023-08-25T09:40:27.250502+0000","last_peered":"2023-08-25T09:40:27.250502+0000","last_clean":"2023-08-25T09:40:27.250502+0000","last_became_active":"2023-08-25T09:40:27.249377+0000","last_became_peered":"2023-08-25T09:40:27.249377+0000","last_unstale":"2023-08-25T09:40:27.250502+0000","last_undegraded":"2023-08-25T09:40:27.250502+0000","last_fullsized":"2023-08-25T09:40:27.250502+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:23:06.089592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.449187+0000","last_change":"2023-08-25T09:40:26.454321+0000","last_active":"2023-08-25T09:40:27.449187+0000","last_peered":"2023-08-25T09:40:27.449187+0000","last_clean":"2023-08-25T09:40:27.449187+0000","last_became_active":"2023-08-25T09:40:26.453648+0000","last_became_peered":"2023-08-25T09:40:26.453648+0000","last_unstale":"2023-08-25T09:40:27.449187+0000","last_undegraded":"2023-08-25T09:40:27.449187+0000","last_fullsized":"2023-08-25T09:40:27.449187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:47:35.404450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249343+0000","last_change":"2023-08-25T09:40:27.249185+0000","last_active":"2023-08-25T09:40:27.249343+0000","last_peered":"2023-08-25T09:40:27.249343+0000","last_clean":"2023-08-25T09:40:27.249343+0000","last_became_active":"2023-08-25T09:40:27.247843+0000","last_became_peered":"2023-08-25T09:40:27.247843+0000","last_unstale":"2023-08-25T09:40:27.249343+0000","last_undegraded":"2023-08-25T09:40:27.249343+0000","last_fullsized":"2023-08-25T09:40:27.249343+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:53.425898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.222625+0000","last_change":"2023-08-25T09:40:26.446621+0000","last_active":"2023-08-25T09:40:27.222625+0000","last_peered":"2023-08-25T09:40:27.222625+0000","last_clean":"2023-08-25T09:40:27.222625+0000","last_became_active":"2023-08-25T09:40:26.445438+0000","last_became_peered":"2023-08-25T09:40:26.445438+0000","last_unstale":"2023-08-25T09:40:27.222625+0000","last_undegraded":"2023-08-25T09:40:27.222625+0000","last_fullsized":"2023-08-25T09:40:27.222625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:26:59.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.252673+0000","last_change":"2023-08-25T09:40:27.252197+0000","last_active":"2023-08-25T09:40:27.252673+0000","last_peered":"2023-08-25T09:40:27.252673+0000","last_clean":"2023-08-25T09:40:27.252673+0000","last_became_active":"2023-08-25T09:40:27.249544+0000","last_became_peered":"2023-08-25T09:40:27.249544+0000","last_unstale":"2023-08-25T09:40:27.252673+0000","last_undegraded":"2023-08-25T09:40:27.252673+0000","last_fullsized":"2023-08-25T09:40:27.252673+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:53.993727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230377+0000","last_change":"2023-08-25T09:40:26.433779+0000","last_active":"2023-08-25T09:40:28.230377+0000","last_peered":"2023-08-25T09:40:28.230377+0000","last_clean":"2023-08-25T09:40:28.230377+0000","last_became_active":"2023-08-25T09:40:26.433504+0000","last_became_peered":"2023-08-25T09:40:26.433504+0000","last_unstale":"2023-08-25T09:40:28.230377+0000","last_undegraded":"2023-08-25T09:40:28.230377+0000","last_fullsized":"2023-08-25T09:40:28.230377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:58:26.635834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.235044+0000","last_change":"2023-08-25T09:40:27.231232+0000","last_active":"2023-08-25T09:40:28.235044+0000","last_peered":"2023-08-25T09:40:28.235044+0000","last_clean":"2023-08-25T09:40:28.235044+0000","last_became_active":"2023-08-25T09:40:27.230725+0000","last_became_peered":"2023-08-25T09:40:27.230725+0000","last_unstale":"2023-08-25T09:40:28.235044+0000","last_undegraded":"2023-08-25T09:40:28.235044+0000","last_fullsized":"2023-08-25T09:40:28.235044+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:13.637819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.232481+0000","last_change":"2023-08-25T09:40:26.440595+0000","last_active":"2023-08-25T09:40:28.232481+0000","last_peered":"2023-08-25T09:40:28.232481+0000","last_clean":"2023-08-25T09:40:28.232481+0000","last_became_active":"2023-08-25T09:40:26.440344+0000","last_became_peered":"2023-08-25T09:40:26.440344+0000","last_unstale":"2023-08-25T09:40:28.232481+0000","last_undegraded":"2023-08-25T09:40:28.232481+0000","last_fullsized":"2023-08-25T09:40:28.232481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:38:12.351776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.249624+0000","last_change":"2023-08-25T09:40:27.249529+0000","last_active":"2023-08-25T09:40:27.249624+0000","last_peered":"2023-08-25T09:40:27.249624+0000","last_clean":"2023-08-25T09:40:27.249624+0000","last_became_active":"2023-08-25T09:40:27.248214+0000","last_became_peered":"2023-08-25T09:40:27.248214+0000","last_unstale":"2023-08-25T09:40:27.249624+0000","last_undegraded":"2023-08-25T09:40:27.249624+0000","last_fullsized":"2023-08-25T09:40:27.249624+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:18.035205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.447141+0000","last_change":"2023-08-25T09:40:26.442843+0000","last_active":"2023-08-25T09:40:27.447141+0000","last_peered":"2023-08-25T09:40:27.447141+0000","last_clean":"2023-08-25T09:40:27.447141+0000","last_became_active":"2023-08-25T09:40:26.442540+0000","last_became_peered":"2023-08-25T09:40:26.442540+0000","last_unstale":"2023-08-25T09:40:27.447141+0000","last_undegraded":"2023-08-25T09:40:27.447141+0000","last_fullsized":"2023-08-25T09:40:27.447141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:36.753372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245721+0000","last_change":"2023-08-25T09:40:27.245625+0000","last_active":"2023-08-25T09:40:27.245721+0000","last_peered":"2023-08-25T09:40:27.245721+0000","last_clean":"2023-08-25T09:40:27.245721+0000","last_became_active":"2023-08-25T09:40:27.243404+0000","last_became_peered":"2023-08-25T09:40:27.243404+0000","last_unstale":"2023-08-25T09:40:27.245721+0000","last_undegraded":"2023-08-25T09:40:27.245721+0000","last_fullsized":"2023-08-25T09:40:27.245721+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:13.637819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229968+0000","last_change":"2023-08-25T09:40:26.448676+0000","last_active":"2023-08-25T09:40:28.229968+0000","last_peered":"2023-08-25T09:40:28.229968+0000","last_clean":"2023-08-25T09:40:28.229968+0000","last_became_active":"2023-08-25T09:40:26.448068+0000","last_became_peered":"2023-08-25T09:40:26.448068+0000","last_unstale":"2023-08-25T09:40:28.229968+0000","last_undegraded":"2023-08-25T09:40:28.229968+0000","last_fullsized":"2023-08-25T09:40:28.229968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:14:18.703885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244295+0000","last_change":"2023-08-25T09:40:27.244124+0000","last_active":"2023-08-25T09:40:27.244295+0000","last_peered":"2023-08-25T09:40:27.244295+0000","last_clean":"2023-08-25T09:40:27.244295+0000","last_became_active":"2023-08-25T09:40:27.243499+0000","last_became_peered":"2023-08-25T09:40:27.243499+0000","last_unstale":"2023-08-25T09:40:27.244295+0000","last_undegraded":"2023-08-25T09:40:27.244295+0000","last_fullsized":"2023-08-25T09:40:27.244295+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:05:50.380346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234899+0000","last_change":"2023-08-25T09:40:26.448882+0000","last_active":"2023-08-25T09:40:28.234899+0000","last_peered":"2023-08-25T09:40:28.234899+0000","last_clean":"2023-08-25T09:40:28.234899+0000","last_became_active":"2023-08-25T09:40:26.448612+0000","last_became_peered":"2023-08-25T09:40:26.448612+0000","last_unstale":"2023-08-25T09:40:28.234899+0000","last_undegraded":"2023-08-25T09:40:28.234899+0000","last_fullsized":"2023-08-25T09:40:28.234899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:00:58.640931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.240344+0000","last_change":"2023-08-25T09:40:27.240215+0000","last_active":"2023-08-25T09:40:27.240344+0000","last_peered":"2023-08-25T09:40:27.240344+0000","last_clean":"2023-08-25T09:40:27.240344+0000","last_became_active":"2023-08-25T09:40:27.239535+0000","last_became_peered":"2023-08-25T09:40:27.239535+0000","last_unstale":"2023-08-25T09:40:27.240344+0000","last_undegraded":"2023-08-25T09:40:27.240344+0000","last_fullsized":"2023-08-25T09:40:27.240344+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:33.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.229210+0000","last_change":"2023-08-25T09:40:26.449887+0000","last_active":"2023-08-25T09:40:27.229210+0000","last_peered":"2023-08-25T09:40:27.229210+0000","last_clean":"2023-08-25T09:40:27.229210+0000","last_became_active":"2023-08-25T09:40:26.449622+0000","last_became_peered":"2023-08-25T09:40:26.449622+0000","last_unstale":"2023-08-25T09:40:27.229210+0000","last_undegraded":"2023-08-25T09:40:27.229210+0000","last_fullsized":"2023-08-25T09:40:27.229210+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:08.509499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228791+0000","last_change":"2023-08-25T09:40:27.225768+0000","last_active":"2023-08-25T09:40:28.228791+0000","last_peered":"2023-08-25T09:40:28.228791+0000","last_clean":"2023-08-25T09:40:28.228791+0000","last_became_active":"2023-08-25T09:40:27.225172+0000","last_became_peered":"2023-08-25T09:40:27.225172+0000","last_unstale":"2023-08-25T09:40:28.228791+0000","last_undegraded":"2023-08-25T09:40:28.228791+0000","last_fullsized":"2023-08-25T09:40:28.228791+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:19.497902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.452137+0000","last_change":"2023-08-25T09:40:26.451165+0000","last_active":"2023-08-25T09:40:27.452137+0000","last_peered":"2023-08-25T09:40:27.452137+0000","last_clean":"2023-08-25T09:40:27.452137+0000","last_became_active":"2023-08-25T09:40:26.450877+0000","last_became_peered":"2023-08-25T09:40:26.450877+0000","last_unstale":"2023-08-25T09:40:27.452137+0000","last_undegraded":"2023-08-25T09:40:27.452137+0000","last_fullsized":"2023-08-25T09:40:27.452137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:13.259999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.248711+0000","last_change":"2023-08-25T09:40:27.248614+0000","last_active":"2023-08-25T09:40:27.248711+0000","last_peered":"2023-08-25T09:40:27.248711+0000","last_clean":"2023-08-25T09:40:27.248711+0000","last_became_active":"2023-08-25T09:40:27.248352+0000","last_became_peered":"2023-08-25T09:40:27.248352+0000","last_unstale":"2023-08-25T09:40:27.248711+0000","last_undegraded":"2023-08-25T09:40:27.248711+0000","last_fullsized":"2023-08-25T09:40:27.248711+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:26:54.224850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.222031+0000","last_change":"2023-08-25T09:40:26.445964+0000","last_active":"2023-08-25T09:40:27.222031+0000","last_peered":"2023-08-25T09:40:27.222031+0000","last_clean":"2023-08-25T09:40:27.222031+0000","last_became_active":"2023-08-25T09:40:26.444388+0000","last_became_peered":"2023-08-25T09:40:26.444388+0000","last_unstale":"2023-08-25T09:40:27.222031+0000","last_undegraded":"2023-08-25T09:40:27.222031+0000","last_fullsized":"2023-08-25T09:40:27.222031+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:17:05.700386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.247251+0000","last_change":"2023-08-25T09:40:27.247160+0000","last_active":"2023-08-25T09:40:27.247251+0000","last_peered":"2023-08-25T09:40:27.247251+0000","last_clean":"2023-08-25T09:40:27.247251+0000","last_became_active":"2023-08-25T09:40:27.245757+0000","last_became_peered":"2023-08-25T09:40:27.245757+0000","last_unstale":"2023-08-25T09:40:27.247251+0000","last_undegraded":"2023-08-25T09:40:27.247251+0000","last_fullsized":"2023-08-25T09:40:27.247251+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:58:37.901592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.231714+0000","last_change":"2023-08-25T09:40:26.444486+0000","last_active":"2023-08-25T09:40:27.231714+0000","last_peered":"2023-08-25T09:40:27.231714+0000","last_clean":"2023-08-25T09:40:27.231714+0000","last_became_active":"2023-08-25T09:40:26.443652+0000","last_became_peered":"2023-08-25T09:40:26.443652+0000","last_unstale":"2023-08-25T09:40:27.231714+0000","last_undegraded":"2023-08-25T09:40:27.231714+0000","last_fullsized":"2023-08-25T09:40:27.231714+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:55:36.095931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.244258+0000","last_change":"2023-08-25T09:40:27.244114+0000","last_active":"2023-08-25T09:40:27.244258+0000","last_peered":"2023-08-25T09:40:27.244258+0000","last_clean":"2023-08-25T09:40:27.244258+0000","last_became_active":"2023-08-25T09:40:27.243355+0000","last_became_peered":"2023-08-25T09:40:27.243355+0000","last_unstale":"2023-08-25T09:40:27.244258+0000","last_undegraded":"2023-08-25T09:40:27.244258+0000","last_fullsized":"2023-08-25T09:40:27.244258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:44:22.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.222979+0000","last_change":"2023-08-25T09:40:26.454067+0000","last_active":"2023-08-25T09:40:27.222979+0000","last_peered":"2023-08-25T09:40:27.222979+0000","last_clean":"2023-08-25T09:40:27.222979+0000","last_became_active":"2023-08-25T09:40:26.453745+0000","last_became_peered":"2023-08-25T09:40:26.453745+0000","last_unstale":"2023-08-25T09:40:27.222979+0000","last_undegraded":"2023-08-25T09:40:27.222979+0000","last_fullsized":"2023-08-25T09:40:27.222979+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:58:36.888721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228713+0000","last_change":"2023-08-25T09:40:27.242691+0000","last_active":"2023-08-25T09:40:28.228713+0000","last_peered":"2023-08-25T09:40:28.228713+0000","last_clean":"2023-08-25T09:40:28.228713+0000","last_became_active":"2023-08-25T09:40:27.242038+0000","last_became_peered":"2023-08-25T09:40:27.242038+0000","last_unstale":"2023-08-25T09:40:28.228713+0000","last_undegraded":"2023-08-25T09:40:28.228713+0000","last_fullsized":"2023-08-25T09:40:28.228713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:35:27.213912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.234563+0000","last_change":"2023-08-25T09:40:26.452729+0000","last_active":"2023-08-25T09:40:27.234563+0000","last_peered":"2023-08-25T09:40:27.234563+0000","last_clean":"2023-08-25T09:40:27.234563+0000","last_became_active":"2023-08-25T09:40:26.452442+0000","last_became_peered":"2023-08-25T09:40:26.452442+0000","last_unstale":"2023-08-25T09:40:27.234563+0000","last_undegraded":"2023-08-25T09:40:27.234563+0000","last_fullsized":"2023-08-25T09:40:27.234563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:14:27.338263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":191,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.230347+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:40:27.230347+0000","last_peered":"2023-08-25T09:40:27.230347+0000","last_clean":"2023-08-25T09:40:27.230347+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:40:27.230347+0000","last_undegraded":"2023-08-25T09:40:27.230347+0000","last_fullsized":"2023-08-25T09:40:27.230347+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:49:11.212026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.228877+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:40:28.228877+0000","last_peered":"2023-08-25T09:40:28.228877+0000","last_clean":"2023-08-25T09:40:28.228877+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:40:28.228877+0000","last_undegraded":"2023-08-25T09:40:28.228877+0000","last_fullsized":"2023-08-25T09:40:28.228877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:46.734175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.235366+0000","last_change":"2023-08-25T09:40:27.246001+0000","last_active":"2023-08-25T09:40:28.235366+0000","last_peered":"2023-08-25T09:40:28.235366+0000","last_clean":"2023-08-25T09:40:28.235366+0000","last_became_active":"2023-08-25T09:40:27.245514+0000","last_became_peered":"2023-08-25T09:40:27.245514+0000","last_unstale":"2023-08-25T09:40:28.235366+0000","last_undegraded":"2023-08-25T09:40:28.235366+0000","last_fullsized":"2023-08-25T09:40:28.235366+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:33.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230085+0000","last_change":"2023-08-25T09:40:26.451368+0000","last_active":"2023-08-25T09:40:28.230085+0000","last_peered":"2023-08-25T09:40:28.230085+0000","last_clean":"2023-08-25T09:40:28.230085+0000","last_became_active":"2023-08-25T09:40:26.451070+0000","last_became_peered":"2023-08-25T09:40:26.451070+0000","last_unstale":"2023-08-25T09:40:28.230085+0000","last_undegraded":"2023-08-25T09:40:28.230085+0000","last_fullsized":"2023-08-25T09:40:28.230085+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:09:20.744481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230443+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:40:28.230443+0000","last_peered":"2023-08-25T09:40:28.230443+0000","last_clean":"2023-08-25T09:40:28.230443+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:40:28.230443+0000","last_undegraded":"2023-08-25T09:40:28.230443+0000","last_fullsized":"2023-08-25T09:40:28.230443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:26.887559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.247222+0000","last_change":"2023-08-25T09:40:27.246478+0000","last_active":"2023-08-25T09:40:27.247222+0000","last_peered":"2023-08-25T09:40:27.247222+0000","last_clean":"2023-08-25T09:40:27.247222+0000","last_became_active":"2023-08-25T09:40:27.244949+0000","last_became_peered":"2023-08-25T09:40:27.244949+0000","last_unstale":"2023-08-25T09:40:27.247222+0000","last_undegraded":"2023-08-25T09:40:27.247222+0000","last_fullsized":"2023-08-25T09:40:27.247222+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:17:06.713257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234692+0000","last_change":"2023-08-25T09:40:26.452544+0000","last_active":"2023-08-25T09:40:28.234692+0000","last_peered":"2023-08-25T09:40:28.234692+0000","last_clean":"2023-08-25T09:40:28.234692+0000","last_became_active":"2023-08-25T09:40:26.451907+0000","last_became_peered":"2023-08-25T09:40:26.451907+0000","last_unstale":"2023-08-25T09:40:28.234692+0000","last_undegraded":"2023-08-25T09:40:28.234692+0000","last_fullsized":"2023-08-25T09:40:28.234692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:01.406850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234952+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:40:28.234952+0000","last_peered":"2023-08-25T09:40:28.234952+0000","last_clean":"2023-08-25T09:40:28.234952+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:40:28.234952+0000","last_undegraded":"2023-08-25T09:40:28.234952+0000","last_fullsized":"2023-08-25T09:40:28.234952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:04:09.963659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.241874+0000","last_change":"2023-08-25T09:40:27.241743+0000","last_active":"2023-08-25T09:40:27.241874+0000","last_peered":"2023-08-25T09:40:27.241874+0000","last_clean":"2023-08-25T09:40:27.241874+0000","last_became_active":"2023-08-25T09:40:27.241404+0000","last_became_peered":"2023-08-25T09:40:27.241404+0000","last_unstale":"2023-08-25T09:40:27.241874+0000","last_undegraded":"2023-08-25T09:40:27.241874+0000","last_fullsized":"2023-08-25T09:40:27.241874+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:50:57.682960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234977+0000","last_change":"2023-08-25T09:40:26.457293+0000","last_active":"2023-08-25T09:40:28.234977+0000","last_peered":"2023-08-25T09:40:28.234977+0000","last_clean":"2023-08-25T09:40:28.234977+0000","last_became_active":"2023-08-25T09:40:26.457001+0000","last_became_peered":"2023-08-25T09:40:26.457001+0000","last_unstale":"2023-08-25T09:40:28.234977+0000","last_undegraded":"2023-08-25T09:40:28.234977+0000","last_fullsized":"2023-08-25T09:40:28.234977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:06:49.620284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230865+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:40:28.230865+0000","last_peered":"2023-08-25T09:40:28.230865+0000","last_clean":"2023-08-25T09:40:28.230865+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:40:28.230865+0000","last_undegraded":"2023-08-25T09:40:28.230865+0000","last_fullsized":"2023-08-25T09:40:28.230865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:57.530121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.234640+0000","last_change":"2023-08-25T09:40:27.240120+0000","last_active":"2023-08-25T09:40:28.234640+0000","last_peered":"2023-08-25T09:40:28.234640+0000","last_clean":"2023-08-25T09:40:28.234640+0000","last_became_active":"2023-08-25T09:40:27.239752+0000","last_became_peered":"2023-08-25T09:40:27.239752+0000","last_unstale":"2023-08-25T09:40:28.234640+0000","last_undegraded":"2023-08-25T09:40:28.234640+0000","last_fullsized":"2023-08-25T09:40:28.234640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:42.649769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.227812+0000","last_change":"2023-08-25T09:40:26.452165+0000","last_active":"2023-08-25T09:40:27.227812+0000","last_peered":"2023-08-25T09:40:27.227812+0000","last_clean":"2023-08-25T09:40:27.227812+0000","last_became_active":"2023-08-25T09:40:26.451847+0000","last_became_peered":"2023-08-25T09:40:26.451847+0000","last_unstale":"2023-08-25T09:40:27.227812+0000","last_undegraded":"2023-08-25T09:40:27.227812+0000","last_fullsized":"2023-08-25T09:40:27.227812+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:40:54.099861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.224417+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:40:27.224417+0000","last_peered":"2023-08-25T09:40:27.224417+0000","last_clean":"2023-08-25T09:40:27.224417+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:40:27.224417+0000","last_undegraded":"2023-08-25T09:40:27.224417+0000","last_fullsized":"2023-08-25T09:40:27.224417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:04:09.963659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.229294+0000","last_change":"2023-08-25T09:40:27.227785+0000","last_active":"2023-08-25T09:40:27.229294+0000","last_peered":"2023-08-25T09:40:27.229294+0000","last_clean":"2023-08-25T09:40:27.229294+0000","last_became_active":"2023-08-25T09:40:27.227153+0000","last_became_peered":"2023-08-25T09:40:27.227153+0000","last_unstale":"2023-08-25T09:40:27.229294+0000","last_undegraded":"2023-08-25T09:40:27.229294+0000","last_fullsized":"2023-08-25T09:40:27.229294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:21:50.535699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.233737+0000","last_change":"2023-08-25T09:40:26.451776+0000","last_active":"2023-08-25T09:40:27.233737+0000","last_peered":"2023-08-25T09:40:27.233737+0000","last_clean":"2023-08-25T09:40:27.233737+0000","last_became_active":"2023-08-25T09:40:26.451522+0000","last_became_peered":"2023-08-25T09:40:26.451522+0000","last_unstale":"2023-08-25T09:40:27.233737+0000","last_undegraded":"2023-08-25T09:40:27.233737+0000","last_fullsized":"2023-08-25T09:40:27.233737+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:55:44.755044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230012+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:40:28.230012+0000","last_peered":"2023-08-25T09:40:28.230012+0000","last_clean":"2023-08-25T09:40:28.230012+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:40:28.230012+0000","last_undegraded":"2023-08-25T09:40:28.230012+0000","last_fullsized":"2023-08-25T09:40:28.230012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:05:18.585703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.229087+0000","last_change":"2023-08-25T09:40:27.242878+0000","last_active":"2023-08-25T09:40:28.229087+0000","last_peered":"2023-08-25T09:40:28.229087+0000","last_clean":"2023-08-25T09:40:28.229087+0000","last_became_active":"2023-08-25T09:40:27.242417+0000","last_became_peered":"2023-08-25T09:40:27.242417+0000","last_unstale":"2023-08-25T09:40:28.229087+0000","last_undegraded":"2023-08-25T09:40:28.229087+0000","last_fullsized":"2023-08-25T09:40:28.229087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:23.006279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230151+0000","last_change":"2023-08-25T09:40:26.449575+0000","last_active":"2023-08-25T09:40:28.230151+0000","last_peered":"2023-08-25T09:40:28.230151+0000","last_clean":"2023-08-25T09:40:28.230151+0000","last_became_active":"2023-08-25T09:40:26.449039+0000","last_became_peered":"2023-08-25T09:40:26.449039+0000","last_unstale":"2023-08-25T09:40:28.230151+0000","last_undegraded":"2023-08-25T09:40:28.230151+0000","last_fullsized":"2023-08-25T09:40:28.230151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:42:56.357330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.235970+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:40:27.235970+0000","last_peered":"2023-08-25T09:40:27.235970+0000","last_clean":"2023-08-25T09:40:27.235970+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:40:27.235970+0000","last_undegraded":"2023-08-25T09:40:27.235970+0000","last_fullsized":"2023-08-25T09:40:27.235970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:23:41.616585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.245795+0000","last_change":"2023-08-25T09:40:27.242527+0000","last_active":"2023-08-25T09:40:27.245795+0000","last_peered":"2023-08-25T09:40:27.245795+0000","last_clean":"2023-08-25T09:40:27.245795+0000","last_became_active":"2023-08-25T09:40:27.241958+0000","last_became_peered":"2023-08-25T09:40:27.241958+0000","last_unstale":"2023-08-25T09:40:27.245795+0000","last_undegraded":"2023-08-25T09:40:27.245795+0000","last_fullsized":"2023-08-25T09:40:27.245795+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:26.407275+0000","last_clean_scrub_stamp":"2023-08-25T09:40:26.407275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:18:58.163422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-08-25T09:40:27.236991+0000","last_change":"2023-08-25T09:40:26.455883+0000","last_active":"2023-08-25T09:40:27.236991+0000","last_peered":"2023-08-25T09:40:27.236991+0000","last_clean":"2023-08-25T09:40:27.236991+0000","last_became_active":"2023-08-25T09:40:26.452716+0000","last_became_peered":"2023-08-25T09:40:26.452716+0000","last_unstale":"2023-08-25T09:40:27.236991+0000","last_undegraded":"2023-08-25T09:40:27.236991+0000","last_fullsized":"2023-08-25T09:40:27.236991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:40:25.394404+0000","last_clean_scrub_stamp":"2023-08-25T09:40:25.394404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:49:06.674070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":41,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-08-25T09:40:28.230564+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:40:28.230564+0000","last_peered":"2023-08-25T09:40:28.230564+0000","last_clean":"2023-08-25T09:40:28.230564+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:40:28.230564+0000","last_undegraded":"2023-08-25T09:40:28.230564+0000","last_fullsized":"2023-08-25T09:40:28.230564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:27:58.374248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":158,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195154,"statfs":{"total":200000000,"available":199837740,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.98599999999999999},{"interface":"front","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.93100000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.048,"5min":1.048,"15min":1.048},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.194}]},{"osd":2,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":1.254}]},{"osd":3,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.875},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.5}]},{"osd":4,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3029999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9099999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.73},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":1.6499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":151,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195160,"statfs":{"total":200000000,"available":199844429,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8620000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":1.3140000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.6319999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":1.103}]},{"osd":3,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.726,"5min":1.726,"15min":1.726},"last":1.367},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.0600000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.234}]},{"osd":5,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.341},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.252}]},{"osd":7,"last update":"Fri Aug 25 09:39:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.9730000000000001,"5min":1.9730000000000001,"15min":1.9730000000000001},"last":1.286},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.109,"5min":2.109,"15min":2.109},"last":1.4550000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194583,"statfs":{"total":200000000,"available":199253513,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.327},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.81799999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.1779999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.1080000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.2050000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.2749999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0589999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.383}]},{"osd":7,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.429,"15min":1.429},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.575,"15min":1.575},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3700000000000001,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.351}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":153,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195158,"statfs":{"total":200000000,"available":199842727,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":1.323},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.4079999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.48,"5min":1.48,"15min":1.48},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":8.5850000000000009,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.224}]},{"osd":2,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.349,"15min":1.349},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":4.1669999999999998,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":1.454}]},{"osd":3,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2,"15min":1.2},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.278},{"interface":"front","average":{"1min":1.375,"5min":1.375,"15min":1.375},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":1.1759999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.4420000000000002,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.7430000000000001,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.119}]},{"osd":6,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"max":{"1min":2.863,"5min":2.863,"15min":2.863},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.216,"5min":1.216,"15min":1.216},"max":{"1min":2.7930000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.3600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":158,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195154,"statfs":{"total":200000000,"available":199837720,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.099}]},{"osd":1,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.1579999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.3399999999999999,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.75}]},{"osd":4,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1949999999999998,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.2390000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.4060000000000001,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.3089999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.069},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":3.496,"5min":3.496,"15min":3.496},"last":1.2769999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.347,"15min":1.347},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":2.2989999999999999,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.399},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"last":1.3460000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":160,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195151,"statfs":{"total":200000000,"available":199835616,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.383,"5min":2.383,"15min":2.383},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":2.7309999999999999,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.45500000000000002}]},{"osd":1,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":2.3180000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.8599999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.505}]},{"osd":3,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":4.9569999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":0.63},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":3.0150000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.7639999999999998,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":0.81499999999999995}]},{"osd":5,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.1269999999999998,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"min":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"max":{"1min":2.6709999999999998,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":5.0599999999999996,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.1100000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:39:40 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":1.022,"5min":1.022,"15min":1.022},"max":{"1min":2.968,"5min":2.968,"15min":2.968},"last":0.746},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"min":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"max":{"1min":5.2990000000000004,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.194}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":728,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194584,"statfs":{"total":200000000,"available":199254073,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.0469999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.26},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":1.496}]},{"osd":5,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"min":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.101},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":0.83299999999999996}]},{"osd":7,"last update":"Fri Aug 25 09:39:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.3140000000000001,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":1.407},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.157}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":151,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195160,"statfs":{"total":200000000,"available":199844371,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.80400000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":3.4889999999999999,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.044}]},{"osd":4,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.403},{"interface":"front","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":1.948,"5min":1.948,"15min":1.948},"last":1.2609999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":3.3769999999999998,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.216}]},{"osd":6,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":1.3009999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:39:39 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.413,"15min":1.413},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.3380000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:40:32.504 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:40:32.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:40:32.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.762+0000 7fba6ffff640 1 Processor -- start 2023-08-25T09:40:32.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.762+0000 7fba6ffff640 1 -- start start 2023-08-25T09:40:32.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.763+0000 7fba6ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ca450 0x7fba700c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba700ad340 0x7fba7006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6ffff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7015ac10 con 0x7fba700b0300 2023-08-25T09:40:32.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6ffff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba700adb40 con 0x7fba700ad340 2023-08-25T09:40:32.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba700cac80 con 0x7fba700ca450 2023-08-25T09:40:32.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6f7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba700ad340 0x7fba7006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6e7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ca450 0x7fba700c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6f7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba700ad340 0x7fba7006e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:36314/0 (socket says 172.21.15.70:36314) 2023-08-25T09:40:32.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.764+0000 7fba6f7fe640 1 -- 172.21.15.70:0/2867996721 learned_addr learned my addr 172.21.15.70:0/2867996721 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:32.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.765+0000 7fba6f7fe640 1 -- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ca450 msgr2=0x7fba700c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:32.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.765+0000 7fba6f7fe640 1 --2- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ca450 0x7fba700c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.765+0000 7fba6f7fe640 1 -- 172.21.15.70:0/2867996721 >> v1:172.21.15.155:6789/0 conn(0x7fba700b0300 legacy=0x7fba700b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.765+0000 7fba6f7fe640 1 -- 172.21.15.70:0/2867996721 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba700c9110 con 0x7fba700ad340 2023-08-25T09:40:32.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.767+0000 7fba6f7fe640 1 --2- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700ad340 0x7fba7006e960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba68009e50 tx=0x7fba6802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9426a310989ef7c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:32.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.767+0000 7fba6dffb640 1 -- 172.21.15.70:0/2867996721 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6802ca70 con 0x7fba700ad340 2023-08-25T09:40:32.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.767+0000 7fba6dffb640 1 -- 172.21.15.70:0/2867996721 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6802cbd0 con 0x7fba700ad340 2023-08-25T09:40:32.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.767+0000 7fba6dffb640 1 -- 172.21.15.70:0/2867996721 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680343f0 con 0x7fba700ad340 2023-08-25T09:40:32.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 -- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700ad340 msgr2=0x7fba7006e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 --2- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700ad340 0x7fba7006e960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba68009e50 tx=0x7fba6802bed0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 -- 172.21.15.70:0/2867996721 shutdown_connections 2023-08-25T09:40:32.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 --2- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700ad340 0x7fba7006e960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 --2- 172.21.15.70:0/2867996721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ca450 0x7fba700c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 -- 172.21.15.70:0/2867996721 >> 172.21.15.70:0/2867996721 conn(0x7fba700ce810 msgr2=0x7fba700cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:32.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 -- 172.21.15.70:0/2867996721 shutdown_connections 2023-08-25T09:40:32.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.768+0000 7fba6ffff640 1 -- 172.21.15.70:0/2867996721 wait complete. 2023-08-25T09:40:32.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.770+0000 7fba6ffff640 1 Processor -- start 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.770+0000 7fba6ffff640 1 -- start start 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ad340 0x7fba7006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700b0300 0x7fba700c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 0x7fba700c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70151410 con 0x7fba700ca450 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6ffff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba70150e90 con 0x7fba700b0300 2023-08-25T09:40:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba70151130 con 0x7fba700ad340 2023-08-25T09:40:32.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 0x7fba700c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 0x7fba700c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45872/0 (socket says 172.21.15.70:45872) 2023-08-25T09:40:32.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6f7fe640 1 -- 172.21.15.70:0/4056157187 learned_addr learned my addr 172.21.15.70:0/4056157187 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:32.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6e7fc640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700b0300 0x7fba700c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6effd640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ad340 0x7fba7006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6f7fe640 1 -- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ad340 msgr2=0x7fba7006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.771+0000 7fba6f7fe640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ad340 0x7fba7006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6f7fe640 1 -- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700b0300 msgr2=0x7fba700c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6f7fe640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700b0300 0x7fba700c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6f7fe640 1 -- 172.21.15.70:0/4056157187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009b30 con 0x7fba700ca450 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6e7fc640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700b0300 0x7fba700c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6f7fe640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 0x7fba700c7af0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fba68009e50 tx=0x7fba68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68009510 con 0x7fba700ca450 2023-08-25T09:40:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6803e450 con 0x7fba700ca450 2023-08-25T09:40:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6803e800 con 0x7fba700ca450 2023-08-25T09:40:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7006b3e0 con 0x7fba700ca450 2023-08-25T09:40:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.772+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7006c120 con 0x7fba700ca450 2023-08-25T09:40:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.774+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba70061460 con 0x7fba700ca450 2023-08-25T09:40:32.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.774+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fba6803e960 con 0x7fba700ca450 2023-08-25T09:40:32.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.776+0000 7fba577fe640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba5c06a050 0x7fba5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:32.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.776+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba680ab270 con 0x7fba700ca450 2023-08-25T09:40:32.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.776+0000 7fba6effd640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba5c06a050 0x7fba5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:32.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.777+0000 7fba6effd640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba5c06a050 0x7fba5c06c510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba60000fb0 tx=0x7fba60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:32.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:32.780+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fba68076950 con 0x7fba700ca450 2023-08-25T09:40:33.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.037+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fba7006c980 con 0x7fba700ca450 2023-08-25T09:40:33.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.045+0000 7fba577fe640 1 -- 172.21.15.70:0/4056157187 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fba68079f70 con 0x7fba700ca450 2023-08-25T09:40:33.047 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:40:33.047 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:40:33.047 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:40:33.048 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:40:33.048 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:40:33.048 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:40:33.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba5c06a050 msgr2=0x7fba5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba5c06a050 0x7fba5c06c510 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba60000fb0 tx=0x7fba60009040 comp rx=0 tx=0).stop 2023-08-25T09:40:33.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 msgr2=0x7fba700c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 0x7fba700c7af0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fba68009e50 tx=0x7fba68004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:33.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 shutdown_connections 2023-08-25T09:40:33.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba5c06a050 0x7fba5c06c510 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba700ca450 0x7fba700c7af0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba700b0300 0x7fba700c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 --2- 172.21.15.70:0/4056157187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba700ad340 0x7fba7006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 >> 172.21.15.70:0/4056157187 conn(0x7fba700ce810 msgr2=0x7fba700b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.048+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 shutdown_connections 2023-08-25T09:40:33.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.049+0000 7fba6ffff640 1 -- 172.21.15.70:0/4056157187 wait complete. 2023-08-25T09:40:33.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.278+0000 7f2e9bfff640 1 Processor -- start 2023-08-25T09:40:33.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.279+0000 7f2e9bfff640 1 -- start start 2023-08-25T09:40:33.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.280+0000 7f2e9bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e9c0cb450 0x7f2e9c06f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.280+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.280+0000 7f2e9bfff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e9c06ed50 con 0x7f2e9c15cbc0 2023-08-25T09:40:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.280+0000 7f2e9bfff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0d0120 con 0x7f2e9c0cb450 2023-08-25T09:40:33.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.280+0000 7f2e9bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0d0400 con 0x7f2e9c071760 2023-08-25T09:40:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e9c0cb450 0x7f2e9c06f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2e9c0cb450 0x7f2e9c06f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:36328/0 (socket says 172.21.15.70:36328) 2023-08-25T09:40:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 -- 172.21.15.70:0/2752938511 learned_addr learned my addr 172.21.15.70:0/2752938511 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e9b7fe640 1 --2- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:33.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 -- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 msgr2=0x7f2e9c0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:33.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 --2- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c0b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 -- 172.21.15.70:0/2752938511 >> v1:172.21.15.155:6789/0 conn(0x7f2e9c15cbc0 legacy=0x7f2e9c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.281+0000 7f2e93fff640 1 -- 172.21.15.70:0/2752938511 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c06e700 con 0x7f2e9c0cb450 2023-08-25T09:40:33.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.283+0000 7f2e93fff640 1 --2- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06f1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009870 tx=0x7f2e8402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d32ddfcf84e0854 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.284+0000 7f2e9a7fc640 1 -- 172.21.15.70:0/2752938511 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84030840 con 0x7f2e9c0cb450 2023-08-25T09:40:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.284+0000 7f2e9a7fc640 1 -- 172.21.15.70:0/2752938511 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e840309a0 con 0x7f2e9c0cb450 2023-08-25T09:40:33.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.284+0000 7f2e9a7fc640 1 -- 172.21.15.70:0/2752938511 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84030d30 con 0x7f2e9c0cb450 2023-08-25T09:40:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 -- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c0cb450 msgr2=0x7f2e9c06f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06f1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009870 tx=0x7f2e8402b980 comp rx=0 tx=0).stop 2023-08-25T09:40:33.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 -- 172.21.15.70:0/2752938511 shutdown_connections 2023-08-25T09:40:33.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c0b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/2752938511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06f1e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 -- 172.21.15.70:0/2752938511 >> 172.21.15.70:0/2752938511 conn(0x7f2e9c0d06f0 msgr2=0x7f2e9c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:33.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 -- 172.21.15.70:0/2752938511 shutdown_connections 2023-08-25T09:40:33.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.285+0000 7f2e9bfff640 1 -- 172.21.15.70:0/2752938511 wait complete. 2023-08-25T09:40:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.287+0000 7f2e9bfff640 1 Processor -- start 2023-08-25T09:40:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.287+0000 7f2e9bfff640 1 -- start start 2023-08-25T09:40:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c15cbc0 0x7f2e9c06a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:33.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06b1d0 con 0x7f2e9c0cb450 2023-08-25T09:40:33.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9bfff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06b4b0 con 0x7f2e9c15cbc0 2023-08-25T09:40:33.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06b790 con 0x7f2e9c071760 2023-08-25T09:40:33.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e93fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:33.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e93fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45882/0 (socket says 172.21.15.70:45882) 2023-08-25T09:40:33.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e93fff640 1 -- 172.21.15.70:0/115186925 learned_addr learned my addr 172.21.15.70:0/115186925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:33.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9affd640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:33.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.288+0000 7f2e9b7fe640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c15cbc0 0x7f2e9c06a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e93fff640 1 -- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 msgr2=0x7f2e9c15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e93fff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c15fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e93fff640 1 -- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c15cbc0 msgr2=0x7f2e9c06a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e93fff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c15cbc0 0x7f2e9c06a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e93fff640 1 -- 172.21.15.70:0/115186925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840094d0 con 0x7f2e9c0cb450 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e9affd640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:33.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e9b7fe640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c15cbc0 0x7f2e9c06a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e93fff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06c270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2e84000c00 tx=0x7f2e84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84039c90 con 0x7f2e9c0cb450 2023-08-25T09:40:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e84039e10 con 0x7f2e9c0cb450 2023-08-25T09:40:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84043640 con 0x7f2e9c0cb450 2023-08-25T09:40:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c06ba70 con 0x7f2e9c0cb450 2023-08-25T09:40:33.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.289+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c06aec0 con 0x7f2e9c0cb450 2023-08-25T09:40:33.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.291+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c06bf90 con 0x7f2e9c0cb450 2023-08-25T09:40:33.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.291+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e840437a0 con 0x7f2e9c0cb450 2023-08-25T09:40:33.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.292+0000 7f2e937fe640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e7c06a050 0x7f2e7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:33.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.292+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e8403e070 con 0x7f2e9c0cb450 2023-08-25T09:40:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.293+0000 7f2e9affd640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e7c06a050 0x7f2e7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:33.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.293+0000 7f2e9affd640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e7c06a050 0x7f2e7c06c510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0015e0 tx=0x7f2e8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:33.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.296+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2e9c06bf90 con 0x7f2e9c0cb450 2023-08-25T09:40:33.388 INFO:tasks.ceph.mds.b.smithi155.stderr:2023-08-25T09:40:33.385+0000 7f22e0334640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:40:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.524+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e9c06bf90 con 0x7f2e9c0cb450 2023-08-25T09:40:33.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.525+0000 7f2e937fe640 1 -- 172.21.15.70:0/115186925 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f2e9c06bf90 con 0x7f2e9c0cb450 2023-08-25T09:40:33.527 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:33.527 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:33.528 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e7c06a050 msgr2=0x7f2e7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e7c06a050 0x7f2e7c06c510 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0015e0 tx=0x7f2e8c009040 comp rx=0 tx=0).stop 2023-08-25T09:40:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 msgr2=0x7f2e9c06c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06c270 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2e84000c00 tx=0x7f2e84004730 comp rx=0 tx=0).stop 2023-08-25T09:40:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 shutdown_connections 2023-08-25T09:40:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e7c06a050 0x7f2e7c06c510 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e9c15cbc0 0x7f2e9c06a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e9c0cb450 0x7f2e9c06c270 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 --2- 172.21.15.70:0/115186925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e9c071760 0x7f2e9c15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:33.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 >> 172.21.15.70:0/115186925 conn(0x7f2e9c0d06f0 msgr2=0x7f2e9c0b0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:33.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 shutdown_connections 2023-08-25T09:40:33.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:33.528+0000 7f2e9bfff640 1 -- 172.21.15.70:0/115186925 wait complete. 2023-08-25T09:40:33.951 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:40:26.917357+0000', 'modified': '2023-08-25T09:40:30.467270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5361}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5361': {'gid': 5361, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.155:6835/396571742', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6833', 'nonce': 396571742}, {'type': 'v1', 'addr': '172.21.15.155:6835', 'nonce': 396571742}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:40:33.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:40:33.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-25T09:40:34.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.177+0000 7f74bd66a640 1 Processor -- start 2023-08-25T09:40:34.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.177+0000 7f74bd66a640 1 -- start start 2023-08-25T09:40:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.178+0000 7f74bd66a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f74b80ca780 0x7f74b80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.178+0000 7f74bd66a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b80b12b0 con 0x7f74b80b3cf0 2023-08-25T09:40:34.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74bd66a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b806ae20 con 0x7f74b815acd0 2023-08-25T09:40:34.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74bd66a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b80b47e0 con 0x7f74b80ca780 2023-08-25T09:40:34.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74b77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f74b80b3cf0 0x7f74b80b40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38324/0 (socket says 172.21.15.70:38324) 2023-08-25T09:40:34.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74b77fe640 1 -- 172.21.15.70:0/1568121026 learned_addr learned my addr 172.21.15.70:0/1568121026 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:34.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74b67fc640 1 --2- 172.21.15.70:0/1568121026 >> v2:172.21.15.155:3300/0 conn(0x7f74b80ca780 0x7f74b80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3380243709 0 0) 0x7f74b80b12b0 con 0x7f74b80b3cf0 2023-08-25T09:40:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.179+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a0003520 con 0x7f74b80b3cf0 2023-08-25T09:40:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 128892075 0 0) 0x7f74a0003520 con 0x7f74b80b3cf0 2023-08-25T09:40:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 >> v2:172.21.15.155:3300/0 conn(0x7f74b80ca780 msgr2=0x7f74b80cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 --2- 172.21.15.70:0/1568121026 >> v2:172.21.15.155:3300/0 conn(0x7f74b80ca780 0x7f74b80cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 >> v1:172.21.15.155:6789/0 conn(0x7f74b815acd0 legacy=0x7f74b815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b815b7c0 con 0x7f74b80b3cf0 2023-08-25T09:40:34.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f74b00019f0 con 0x7f74b80b3cf0 2023-08-25T09:40:34.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 >> v1:172.21.15.70:6789/0 conn(0x7f74b80b3cf0 legacy=0x7f74b80b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a0003970 0x7f74a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a0005180 0x7f74a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80b47e0 con 0x7f74a00047c0 2023-08-25T09:40:34.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a000c120 con 0x7f74a0003970 2023-08-25T09:40:34.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a000c2e0 con 0x7f74a0005180 2023-08-25T09:40:34.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b67fc640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a0003970 0x7f74a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b67fc640 1 -- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a0005180 msgr2=0x7f74a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b67fc640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74a0005180 0x7f74a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b67fc640 1 -- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a0003970 msgr2=0x7f74a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b67fc640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74a0003970 0x7f74a0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.180+0000 7f74b67fc640 1 -- 172.21.15.70:0/1568121026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a0003520 con 0x7f74a00047c0 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.181+0000 7f74b77fe640 1 -- 172.21.15.70:0/1568121026 reap_dead start 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.181+0000 7f74b67fc640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74a0004c40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009aa0 tx=0x7f74a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f945592d8ab1e17a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac02c6d0 con 0x7f74a00047c0 2023-08-25T09:40:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac02c830 con 0x7f74a00047c0 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74b5ffb640 1 -- 172.21.15.70:0/1568121026 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac02cbe0 con 0x7f74a00047c0 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74bd66a640 1 -- 172.21.15.70:0/1568121026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 msgr2=0x7f74a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74bd66a640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74a0004c40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009aa0 tx=0x7f74a00057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74bd66a640 1 -- 172.21.15.70:0/1568121026 shutdown_connections 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74bd66a640 1 --2- 172.21.15.70:0/1568121026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74a0004c40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.182+0000 7f74bd66a640 1 -- 172.21.15.70:0/1568121026 >> 172.21.15.70:0/1568121026 conn(0x7f74b80ce760 msgr2=0x7f74b80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.183+0000 7f74bd66a640 1 -- 172.21.15.70:0/1568121026 shutdown_connections 2023-08-25T09:40:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.183+0000 7f74bd66a640 1 -- 172.21.15.70:0/1568121026 wait complete. 2023-08-25T09:40:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.184+0000 7f74bd66a640 1 Processor -- start 2023-08-25T09:40:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.184+0000 7f74bd66a640 1 -- start start 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.185+0000 7f74bd66a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74b80b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.185+0000 7f74bd66a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b81f2820 0x7f74b806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.185+0000 7f74bd66a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b8151be0 0x7f74b8152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.185+0000 7f74bd66a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8070760 con 0x7f74a00047c0 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.185+0000 7f74bd66a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b81525d0 con 0x7f74b8151be0 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.185+0000 7f74bd66a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b80702d0 con 0x7f74b81f2820 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74b80b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74b80b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45890/0 (socket says 172.21.15.70:45890) 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 -- 172.21.15.70:0/755172657 learned_addr learned my addr 172.21.15.70:0/755172657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 -- 172.21.15.70:0/755172657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b81f2820 msgr2=0x7f74b806ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b81f2820 0x7f74b806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 -- 172.21.15.70:0/755172657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b8151be0 msgr2=0x7f74b8152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b8151be0 0x7f74b8152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 -- 172.21.15.70:0/755172657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b806f6c0 con 0x7f74a00047c0 2023-08-25T09:40:34.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.186+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74b80b0060 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74a800d500 tx=0x7f74a800d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.187+0000 7f749f7fe640 1 -- 172.21.15.70:0/755172657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8005500 con 0x7f74a00047c0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.187+0000 7f749f7fe640 1 -- 172.21.15.70:0/755172657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8003660 con 0x7f74a00047c0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.187+0000 7f749f7fe640 1 -- 172.21.15.70:0/755172657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a80039d0 con 0x7f74a00047c0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.187+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b806f9a0 con 0x7f74a00047c0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.187+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b806ff60 con 0x7f74a00047c0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.190+0000 7f749f7fe640 1 -- 172.21.15.70:0/755172657 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f74a8003b30 con 0x7f74a00047c0 2023-08-25T09:40:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.191+0000 7f749f7fe640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f74a406a080 0x7f74a406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.191+0000 7f749f7fe640 1 -- 172.21.15.70:0/755172657 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74a80884f0 con 0x7f74a00047c0 2023-08-25T09:40:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.191+0000 7f74b67fc640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f74a406a080 0x7f74a406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.192+0000 7f74c01e4740 1 Processor -- start 2023-08-25T09:40:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.192+0000 7f74c01e4740 1 -- start start 2023-08-25T09:40:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.192+0000 7f74b67fc640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f74a406a080 0x7f74a406c540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74ac036000 tx=0x7f74ac02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c14530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be4130 0x55e8e1c113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1be7cd0 0x55e8e1c11900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74c01e4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1b57190 con 0x55e8e1c14150 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1b54880 con 0x55e8e1be4130 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1b50170 con 0x55e8e1be7cd0 2023-08-25T09:40:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c14530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c14530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45894/0 (socket says 172.21.15.70:45894) 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 -- 172.21.15.70:0/2349229075 learned_addr learned my addr 172.21.15.70:0/2349229075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 -- 172.21.15.70:0/2349229075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1be7cd0 msgr2=0x55e8e1c11900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1be7cd0 0x55e8e1c11900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 -- 172.21.15.70:0/2349229075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be4130 msgr2=0x55e8e1c113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be4130 0x55e8e1c113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.193+0000 7f74b6ffd640 1 -- 172.21.15.70:0/2349229075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8e1b4e7f0 con 0x55e8e1c14150 2023-08-25T09:40:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.194+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c14530 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f74a8007b20 tx=0x7f74a8057300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45dae6f1359ff49 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.194+0000 7f749d7fa640 1 -- 172.21.15.70:0/2349229075 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8055c30 con 0x55e8e1c14150 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.194+0000 7f749d7fa640 1 -- 172.21.15.70:0/2349229075 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8053fa0 con 0x55e8e1c14150 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.194+0000 7f749d7fa640 1 -- 172.21.15.70:0/2349229075 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a805f310 con 0x55e8e1c14150 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.194+0000 7f74c01e4740 1 -- 172.21.15.70:0/2349229075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 msgr2=0x55e8e1c14530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.194+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c14530 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f74a8007b20 tx=0x7f74a8057300 comp rx=0 tx=0).stop 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 -- 172.21.15.70:0/2349229075 shutdown_connections 2023-08-25T09:40:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1be7cd0 0x55e8e1c11900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be4130 0x55e8e1c113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2349229075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c14530 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 -- 172.21.15.70:0/2349229075 >> 172.21.15.70:0/2349229075 conn(0x55e8e1be63e0 msgr2=0x55e8e1c2ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 -- 172.21.15.70:0/2349229075 shutdown_connections 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 -- 172.21.15.70:0/2349229075 wait complete. 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.195+0000 7f74c01e4740 1 -- start start 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.196+0000 7f74c01e4740 1 Processor -- start 2023-08-25T09:40:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c09820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c43560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1aefd20 con 0x55e8e1c14150 2023-08-25T09:40:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1aef9c0 con 0x55e8e1be7cd0 2023-08-25T09:40:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e18b5f20 con 0x55e8e1c0dd40 2023-08-25T09:40:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74c01e4740 10 client.0 _ll_register_callbacks cb 0x55e8e1b07630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:40:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c43560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c43560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45910/0 (socket says 172.21.15.70:45910) 2023-08-25T09:40:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 -- 172.21.15.70:0/3894797328 learned_addr learned my addr 172.21.15.70:0/3894797328 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c09820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b77fe640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 -- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 msgr2=0x55e8e1c0e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 -- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 msgr2=0x55e8e1c09820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c09820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b67fc640 1 -- 172.21.15.70:0/3894797328 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b806ff60 con 0x55e8e1c14150 2023-08-25T09:40:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b77fe640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.197+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c09820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74b67fc640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c43560 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74ac037040 tx=0x7f74ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de9a2a9d3c6b589 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74967fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:40:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74967fc640 1 -- 172.21.15.70:0/3894797328 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac045070 con 0x55e8e1c14150 2023-08-25T09:40:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74967fc640 1 -- 172.21.15.70:0/3894797328 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74ac0348d0 con 0x55e8e1c14150 2023-08-25T09:40:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8e1b24540 con 0x55e8e1c14150 2023-08-25T09:40:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74967fc640 1 -- 172.21.15.70:0/3894797328 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74ac034a90 con 0x55e8e1c14150 2023-08-25T09:40:34.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74c01e4740 10 client.5532 fetch_fsmap learned FSMap version 16 2023-08-25T09:40:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.198+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e8e1b47f30 con 0x55e8e1c14150 2023-08-25T09:40:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74967fc640 1 -- 172.21.15.70:0/3894797328 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e8e1b24540 con 0x55e8e1c14150 2023-08-25T09:40:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74967fc640 1 -- 172.21.15.70:0/3894797328 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f74ac034bf0 con 0x55e8e1c14150 2023-08-25T09:40:34.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74c01e4740 10 client.5532 fetch_fsmap finished waiting for FSMap version 16 2023-08-25T09:40:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74c01e4740 10 client.5532 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T09:40:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74c01e4740 20 client.5532 populate_metadata read hostname 'smithi070' 2023-08-25T09:40:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1c0c060 0x55e8e1c0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74c01e4740 4 client.5532 mds_command: new command op to 5361 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:40:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.199+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 --> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8e1b49150 con 0x55e8e1c0c060 2023-08-25T09:40:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.200+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1c0c060 0x55e8e1c0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.201+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1c0c060 0x55e8e1c0c440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11c37cc22b847d4 server_cookie=9ca0e2cb1a921d88 in_seq=0 out_seq=0 2023-08-25T09:40:34.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.201+0000 7f74967fc640 10 client.5532 ms_handle_connect on v2:172.21.15.155:6833/396571742 2023-08-25T09:40:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74967fc640 1 -- 172.21.15.70:0/3894797328 <== mds.0 v2:172.21.15.155:6833/396571742 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e8e1b49150 con 0x55e8e1c0c060 2023-08-25T09:40:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74967fc640 10 client.5532 handle_command_reply: tid=0 2023-08-25T09:40:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 client.5532 shutdown 2023-08-25T09:40:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 msgr2=0x55e8e1c43560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c43560 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74ac037040 tx=0x7f74ac004730 comp rx=0 tx=0).stop 2023-08-25T09:40:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 shutdown_connections 2023-08-25T09:40:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1c0c060 0x55e8e1c0c440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c43560 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3894797328 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c09820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74967fc640 0 client.5532 ms_handle_reset on v2:172.21.15.155:6833/396571742 2023-08-25T09:40:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 >> 172.21.15.70:0/3894797328 conn(0x55e8e1be4130 msgr2=0x55e8e1be85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 shutdown_connections 2023-08-25T09:40:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 1 -- 172.21.15.70:0/3894797328 wait complete. 2023-08-25T09:40:34.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.203+0000 7f74c01e4740 20 client.5532 trim_cache size 0 max 16384 2023-08-25T09:40:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.220+0000 7f74c01e4740 1 Processor -- start 2023-08-25T09:40:34.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.220+0000 7f74c01e4740 1 -- start start 2023-08-25T09:40:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c2f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 0x55e8e1c10f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c14150 0x55e8e1c0cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74c01e4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1c10350 con 0x55e8e1c0dd40 2023-08-25T09:40:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1bdf970 con 0x55e8e1be7cd0 2023-08-25T09:40:34.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1c0d400 con 0x55e8e1c14150 2023-08-25T09:40:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74b67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 0x55e8e1c10f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74b67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 0x55e8e1c10f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45916/0 (socket says 172.21.15.70:45916) 2023-08-25T09:40:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.221+0000 7f74b67fc640 1 -- 172.21.15.70:0/2987549064 learned_addr learned my addr 172.21.15.70:0/2987549064 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b67fc640 1 -- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c14150 msgr2=0x55e8e1c0cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c2f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b67fc640 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c14150 0x55e8e1c0cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b67fc640 1 -- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 msgr2=0x55e8e1c2f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b67fc640 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c2f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b67fc640 1 -- 172.21.15.70:0/2987549064 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8e1beb020 con 0x55e8e1c0dd40 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f74b67fc640 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 0x55e8e1c10f00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00f120 tx=0x7f74ac045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a98986e26230f9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f749dffb640 1 -- 172.21.15.70:0/2987549064 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac034ad0 con 0x55e8e1c0dd40 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f749dffb640 1 -- 172.21.15.70:0/2987549064 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac034c30 con 0x55e8e1c0dd40 2023-08-25T09:40:34.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.222+0000 7f749dffb640 1 -- 172.21.15.70:0/2987549064 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac04a990 con 0x55e8e1c0dd40 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 -- 172.21.15.70:0/2987549064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 msgr2=0x55e8e1c10f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 0x55e8e1c10f00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00f120 tx=0x7f74ac045000 comp rx=0 tx=0).stop 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 -- 172.21.15.70:0/2987549064 shutdown_connections 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c14150 0x55e8e1c0cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c0dd40 0x55e8e1c10f00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 --2- 172.21.15.70:0/2987549064 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c2f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 -- 172.21.15.70:0/2987549064 >> 172.21.15.70:0/2987549064 conn(0x55e8e1be4130 msgr2=0x55e8e1c09630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 -- 172.21.15.70:0/2987549064 shutdown_connections 2023-08-25T09:40:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 -- 172.21.15.70:0/2987549064 wait complete. 2023-08-25T09:40:34.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.223+0000 7f74c01e4740 1 -- start start 2023-08-25T09:40:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.224+0000 7f74c01e4740 1 Processor -- start 2023-08-25T09:40:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c81c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1c48590 con 0x55e8e1c14150 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1c45800 con 0x55e8e1be7cd0 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e8e1c14f40 con 0x55e8e1c0dd40 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74c01e4740 10 client.0 _ll_register_callbacks cb 0x7f74a8057660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c0b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45928/0 (socket says 172.21.15.70:45928) 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 -- 172.21.15.70:0/3314765592 learned_addr learned my addr 172.21.15.70:0/3314765592 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:34.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c81c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b67fc640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 -- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 msgr2=0x55e8e1c0a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 -- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 msgr2=0x55e8e1c81c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c81c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.225+0000 7f74b77fe640 1 -- 172.21.15.70:0/3314765592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e8e1b47f30 con 0x55e8e1c14150 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74b77fe640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c0b040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f74b80cf590 tx=0x7f74b000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd72e7fe8ff262d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74967fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74967fc640 1 -- 172.21.15.70:0/3314765592 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b0004d60 con 0x55e8e1c14150 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74967fc640 1 -- 172.21.15.70:0/3314765592 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74b0013be0 con 0x55e8e1c14150 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74a8053fa0 con 0x55e8e1c14150 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74967fc640 1 -- 172.21.15.70:0/3314765592 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74b0007c80 con 0x55e8e1c14150 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74c01e4740 10 client.5538 fetch_fsmap learned FSMap version 16 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55e8e1aeea10 con 0x55e8e1c14150 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.226+0000 7f74967fc640 1 -- 172.21.15.70:0/3314765592 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8053fa0 con 0x55e8e1c14150 2023-08-25T09:40:34.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74967fc640 1 -- 172.21.15.70:0/3314765592 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f74b0013420 con 0x55e8e1c14150 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74c01e4740 10 client.5538 fetch_fsmap finished waiting for FSMap version 16 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74c01e4740 10 client.5538 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74c01e4740 20 client.5538 populate_metadata read hostname 'smithi070' 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1be97b0 0x55e8e1c47250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74c01e4740 4 client.5538 mds_command: new command op to 5361 tid=0[{"prefix": "status"}] 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 --> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e8e17ff5e0 con 0x55e8e1be97b0 2023-08-25T09:40:34.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.227+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1be97b0 0x55e8e1c47250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.228+0000 7f74b6ffd640 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1be97b0 0x55e8e1c47250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ee5eaa895aed9b server_cookie=1515c1183a19b1dd in_seq=0 out_seq=0 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.228+0000 7f74967fc640 10 client.5538 ms_handle_connect on v2:172.21.15.155:6833/396571742 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74967fc640 1 -- 172.21.15.70:0/3314765592 <== mds.0 v2:172.21.15.155:6833/396571742 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e8e17ff5e0 con 0x55e8e1be97b0 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74967fc640 10 client.5538 handle_command_reply: tid=0 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 client.5538 shutdown 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 msgr2=0x55e8e1c0b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c0b040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f74b80cf590 tx=0x7f74b000db60 comp rx=0 tx=0).stop 2023-08-25T09:40:34.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 shutdown_connections 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:6833/396571742,v1:172.21.15.155:6835/396571742] conn(0x55e8e1be97b0 0x55e8e1c47250 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55e8e1c14150 0x55e8e1c0b040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55e8e1c0dd40 0x55e8e1c0a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 --2- 172.21.15.70:0/3314765592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55e8e1be7cd0 0x55e8e1c81c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 >> 172.21.15.70:0/3314765592 conn(0x55e8e1be4130 msgr2=0x55e8e1c08cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74967fc640 0 client.5538 ms_handle_reset on v2:172.21.15.155:6833/396571742 2023-08-25T09:40:34.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 shutdown_connections 2023-08-25T09:40:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 1 -- 172.21.15.70:0/3314765592 wait complete. 2023-08-25T09:40:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.229+0000 7f74c01e4740 20 client.5538 trim_cache size 0 max 16384 2023-08-25T09:40:34.239 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:34.239 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:40:34.239 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5361, 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.8017706709999999, 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 16, 2023-08-25T09:40:34.240 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 35, 2023-08-25T09:40:34.241 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 35, 2023-08-25T09:40:34.241 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 16.589589904 2023-08-25T09:40:34.241 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f74a406a080 msgr2=0x7f74a406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f74a406a080 0x7f74a406c540 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f74ac036000 tx=0x7f74ac02e000 comp rx=0 tx=0).stop 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 msgr2=0x7f74b80b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74b80b0060 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74a800d500 tx=0x7f74a800d9c0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 shutdown_connections 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f74a406a080 0x7f74a406c540 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74b8151be0 0x7f74b8152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74b81f2820 0x7f74b806ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 --2- 172.21.15.70:0/755172657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74a00047c0 0x7f74b80b0060 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 >> 172.21.15.70:0/755172657 conn(0x7f74b80ce760 msgr2=0x7f74b815d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 shutdown_connections 2023-08-25T09:40:34.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.230+0000 7f74bd66a640 1 -- 172.21.15.70:0/755172657 wait complete. 2023-08-25T09:40:34.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:40:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.456+0000 7fa763fff640 1 Processor -- start 2023-08-25T09:40:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.456+0000 7fa763fff640 1 -- start start 2023-08-25T09:40:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.457+0000 7fa763fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa7640b12f0 0x7fa76415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.457+0000 7fa763fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76415e7c0 con 0x7fa76415f190 2023-08-25T09:40:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.457+0000 7fa763fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa764071b50 con 0x7fa76415a7c0 2023-08-25T09:40:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.457+0000 7fa763fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76415e490 con 0x7fa7640b12f0 2023-08-25T09:40:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.458+0000 7fa7637fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa76415f190 0x7fa7640cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38326/0 (socket says 172.21.15.70:38326) 2023-08-25T09:40:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.458+0000 7fa7637fe640 1 -- 172.21.15.70:0/3633066978 learned_addr learned my addr 172.21.15.70:0/3633066978 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.458+0000 7fa7627fc640 1 --2- 172.21.15.70:0/3633066978 >> v2:172.21.15.155:3300/0 conn(0x7fa7640b12f0 0x7fa76415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.458+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093933534 0 0) 0x7fa76415e7c0 con 0x7fa76415f190 2023-08-25T09:40:34.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.458+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa74c003520 con 0x7fa76415f190 2023-08-25T09:40:34.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000130747 0 0) 0x7fa74c003520 con 0x7fa76415f190 2023-08-25T09:40:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 >> v2:172.21.15.155:3300/0 conn(0x7fa7640b12f0 msgr2=0x7fa76415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 --2- 172.21.15.70:0/3633066978 >> v2:172.21.15.155:3300/0 conn(0x7fa7640b12f0 0x7fa76415ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 >> v1:172.21.15.155:6789/0 conn(0x7fa76415a7c0 legacy=0x7fa7640b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640b3d60 con 0x7fa76415f190 2023-08-25T09:40:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa75c0019f0 con 0x7fa76415f190 2023-08-25T09:40:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 >> v1:172.21.15.70:6789/0 conn(0x7fa76415f190 legacy=0x7fa7640cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c003970 0x7fa74c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0047c0 0x7fa74c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa74c005180 0x7fa74c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76415e490 con 0x7fa74c005180 2023-08-25T09:40:34.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c00c020 con 0x7fa74c0047c0 2023-08-25T09:40:34.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c00c1e0 con 0x7fa74c003970 2023-08-25T09:40:34.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7637fe640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa74c005180 0x7fa74c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7637fe640 1 -- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c003970 msgr2=0x7fa74c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7637fe640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c003970 0x7fa74c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7627fc640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0047c0 0x7fa74c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7637fe640 1 -- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0047c0 msgr2=0x7fa74c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:34.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7637fe640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0047c0 0x7fa74c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.459+0000 7fa7637fe640 1 -- 172.21.15.70:0/3633066978 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c003520 con 0x7fa74c005180 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.460+0000 7fa7627fc640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c0047c0 0x7fa74c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.460+0000 7fa7637fe640 1 -- 172.21.15.70:0/3633066978 reap_dead start 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.461+0000 7fa7637fe640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa74c005180 0x7fa74c00b980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00a820 tx=0x7fa74c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb89ea4048d1fc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.461+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0072d0 con 0x7fa74c005180 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.461+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c02f070 con 0x7fa74c005180 2023-08-25T09:40:34.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.461+0000 7fa761ffb640 1 -- 172.21.15.70:0/3633066978 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0076b0 con 0x7fa74c005180 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.461+0000 7fa763fff640 1 -- 172.21.15.70:0/3633066978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa74c005180 msgr2=0x7fa74c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.461+0000 7fa763fff640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa74c005180 0x7fa74c00b980 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00a820 tx=0x7fa74c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.462+0000 7fa763fff640 1 -- 172.21.15.70:0/3633066978 shutdown_connections 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.462+0000 7fa763fff640 1 --2- 172.21.15.70:0/3633066978 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa74c005180 0x7fa74c00b980 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.462+0000 7fa763fff640 1 -- 172.21.15.70:0/3633066978 >> 172.21.15.70:0/3633066978 conn(0x7fa7640affd0 msgr2=0x7fa7640b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.462+0000 7fa763fff640 1 -- 172.21.15.70:0/3633066978 shutdown_connections 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.462+0000 7fa763fff640 1 -- 172.21.15.70:0/3633066978 wait complete. 2023-08-25T09:40:34.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.463+0000 7fa763fff640 1 Processor -- start 2023-08-25T09:40:34.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.464+0000 7fa763fff640 1 -- start start 2023-08-25T09:40:34.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa763fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c005180 0x7fa7641f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa763fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 0x7fa7640c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa763fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7640ccce0 0x7fa7640cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa763fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640cd6a0 con 0x7fa76414fd80 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa763fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7640c79b0 con 0x7fa7640ccce0 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa763fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7641f2d20 con 0x7fa74c005180 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 0x7fa7640c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 0x7fa7640c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45938/0 (socket says 172.21.15.70:45938) 2023-08-25T09:40:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 -- 172.21.15.70:0/3603826267 learned_addr learned my addr 172.21.15.70:0/3603826267 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa762ffd640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c005180 0x7fa7641f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 -- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c005180 msgr2=0x7fa7641f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7637fe640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7640ccce0 0x7fa7640cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c005180 0x7fa7641f27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 -- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7640ccce0 msgr2=0x7fa7640cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7640ccce0 0x7fa7640cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.465+0000 7fa7627fc640 1 -- 172.21.15.70:0/3603826267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7640c6da0 con 0x7fa76414fd80 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.466+0000 7fa7627fc640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 0x7fa7640c7e40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa758010910 tx=0x7fa758010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.466+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75800ec20 con 0x7fa76414fd80 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.466+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75800ed80 con 0x7fa76414fd80 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.466+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7640c7080 con 0x7fa76414fd80 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.466+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75801c600 con 0x7fa76414fd80 2023-08-25T09:40:34.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.466+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7640c7640 con 0x7fa76414fd80 2023-08-25T09:40:34.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.467+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764066fb0 con 0x7fa76414fd80 2023-08-25T09:40:34.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.468+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa75801c760 con 0x7fa76414fd80 2023-08-25T09:40:34.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.470+0000 7fa74b7fe640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:34.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.470+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa758088b50 con 0x7fa76414fd80 2023-08-25T09:40:34.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.470+0000 7fa762ffd640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:34.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.470+0000 7fa762ffd640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa754001610 tx=0x7fa754009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:34.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.474+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa758054380 con 0x7fa76414fd80 2023-08-25T09:40:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.699+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7640b42f0 con 0x7fa76414fd80 2023-08-25T09:40:34.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.700+0000 7fa74b7fe640 1 -- 172.21.15.70:0/3603826267 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7580579a0 con 0x7fa76414fd80 2023-08-25T09:40:34.703 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 msgr2=0x7fa75006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa754001610 tx=0x7fa754009040 comp rx=0 tx=0).stop 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 msgr2=0x7fa7640c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 0x7fa7640c7e40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa758010910 tx=0x7fa758010dd0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 shutdown_connections 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7640ccce0 0x7fa7640cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76414fd80 0x7fa7640c7e40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 --2- 172.21.15.70:0/3603826267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c005180 0x7fa7641f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.702+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 >> 172.21.15.70:0/3603826267 conn(0x7fa7640affd0 msgr2=0x7fa7640c89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:34.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.703+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 shutdown_connections 2023-08-25T09:40:34.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:34.703+0000 7fa763fff640 1 -- 172.21.15.70:0/3603826267 wait complete. 2023-08-25T09:40:34.724 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:40:34.724 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:40:34.725 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:40:34.725 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:40:34.725 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:34.726 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:40:34.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:40:34.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:40:34.727 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:40:34.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:34.727 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:40:34.780 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:40:34.781 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:40:34.781 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:40:34.782 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.782 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:40:34.782 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.782 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:40:34.782 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:34.782 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:40:34.783 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:40:34.783 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:34.783 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:40:34.783 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:40:34.783 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:34.784 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:40:34.784 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:34.784 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:40:34.784 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42241sec preferred_lft 42241sec 2023-08-25T09:40:34.785 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:40:34.785 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.785 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:40:34.785 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:34.786 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:40:34.786 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.786 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:40:34.786 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.787 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:40:34.787 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:40:34.787 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:40:34.787 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.788 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:40:34.788 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:40:34.788 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:40:34.788 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:34.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:34.791 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:40:34.876 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:40:34.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:34.881 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:40:34.931 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:40:34.932 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:40:34.932 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:40:34.932 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:40:34.932 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:40:34.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:34.933 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:40:34.933 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:40:34.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:40:34.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:40:34.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:40:34.934 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:40:35.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:35.117 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:40:35.168 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:40:35.168 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:40:35.169 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:40:35.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:35.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:40:35.270 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:40:35.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:40:35.271 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:35.322 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:40:35.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:40:35.323 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:35.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:40:35.463 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:40:35.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:40:35.562 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:40:35.565 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:40:35.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:35.566 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:40:35.620 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:40:35.621 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:40:35.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:40:35.714 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:40:35.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:40:35.789 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:40:35.791 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:40:35.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:35.793 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:40:35.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:40:35.827+0000 7fc84a038480 -1 init, newargv = 0x55b7f7528ea0 newargc=15 2023-08-25T09:40:35.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[95809]: starting ceph client 2023-08-25T09:40:35.840 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[95809]: starting fuse 2023-08-25T09:40:36.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:40:37.051 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:40:37.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:40:37.154 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:40:37.158 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:40:37.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:37.160 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:40:37.213 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:40:37.215 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:40:37.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:40:37.217 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:40:37.217 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:40:37.218 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:37.218 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:40:37.219 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:37.219 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:40:37.219 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:40:37.220 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:40:37.220 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:40:37.221 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:37.221 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:40:37.221 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:40:37.222 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:40:37.222 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:40:37.222 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:37.223 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:40:37.223 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:40:37.223 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:40:37.223 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:40:37.224 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:40:37.225 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:40:37.226 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:40:37.226 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:40:37.227 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:37.228 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:40:37.229 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:37.414 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:37.431 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:37.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:37.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95809.asok status 2023-08-25T09:40:37.688 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:37.688 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:40:37.688 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:40:37.689 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:40:37.689 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:40:37.689 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:40:37.690 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:40:37.690 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "95809", 2023-08-25T09:40:37.690 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:40:37.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:37.691 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:40:37.691 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:40:37.692 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5553, 2023-08-25T09:40:37.692 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:40:37.693 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:40:37.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:40:37.693 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 5553 2023-08-25T09:40:37.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:37.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:37.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:37.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:37.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:37.695 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:37.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:37.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:37.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:37.697 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:37.697 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:37.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:37.697 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.5553 192.168.0.1:0/2574353996", 2023-08-25T09:40:37.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2574353996", 2023-08-25T09:40:37.698 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:40:37.698 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 16, 2023-08-25T09:40:37.698 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 35, 2023-08-25T09:40:37.698 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:40:37.698 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:40:37.699 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:40:37.699 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:37.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:37.703 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:40:37.704 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:37.765 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:40:37.765 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:37.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:37.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:37.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:40:38.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.079+0000 7fde110e7640 1 Processor -- start 2023-08-25T09:40:38.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.079+0000 7fde110e7640 1 -- start start 2023-08-25T09:40:38.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde110e7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 0x7fde0c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde110e7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde0c0b2850 0x7fde0c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde110e7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0c0ceed0 con 0x7fde0c15aa10 2023-08-25T09:40:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde110e7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0c159d10 con 0x7fde0c0b2850 2023-08-25T09:40:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde110e7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c15d3f0 con 0x7fde0c0ce5b0 2023-08-25T09:40:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde0259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fde0c15aa10 0x7fde0c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:38334/0 (socket says 172.21.15.70:38334) 2023-08-25T09:40:38.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.081+0000 7fde0259c640 1 -- 172.21.15.70:0/87660254 learned_addr learned my addr 172.21.15.70:0/87660254 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:38.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0b59e640 1 --2- 172.21.15.70:0/87660254 >> v2:172.21.15.155:3300/0 conn(0x7fde0c0b2850 0x7fde0c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0ad9d640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 0x7fde0c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504773938 0 0) 0x7fde0c0ceed0 con 0x7fde0c15aa10 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddf8003520 con 0x7fde0c15aa10 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3378625887 0 0) 0x7fddf8003520 con 0x7fde0c15aa10 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 msgr2=0x7fde0c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 0x7fde0c0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 >> v2:172.21.15.155:3300/0 conn(0x7fde0c0b2850 msgr2=0x7fde0c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 --2- 172.21.15.70:0/87660254 >> v2:172.21.15.155:3300/0 conn(0x7fde0c0b2850 0x7fde0c06f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0b2d20 con 0x7fde0c15aa10 2023-08-25T09:40:38.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fddf40019f0 con 0x7fde0c15aa10 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.082+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 >> v1:172.21.15.70:6789/0 conn(0x7fde0c15aa10 legacy=0x7fde0c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0a59c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddf8003970 0x7fddf8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0a59c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf80047c0 0x7fddf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0a59c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 0x7fddf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c159d10 con 0x7fddf80047c0 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c15d3f0 con 0x7fddf8003970 2023-08-25T09:40:38.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf800b980 con 0x7fde0c0ce5b0 2023-08-25T09:40:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0259c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf80047c0 0x7fddf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0ad9d640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddf8003970 0x7fddf8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0b59e640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 0x7fddf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0259c640 1 -- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 msgr2=0x7fddf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:38.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0259c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde0c0ce5b0 0x7fddf8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0259c640 1 -- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddf8003970 msgr2=0x7fddf8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:38.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0259c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddf8003970 0x7fddf8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.083+0000 7fde0259c640 1 -- 172.21.15.70:0/87660254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8003520 con 0x7fddf80047c0 2023-08-25T09:40:38.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.084+0000 7fde0259c640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf80047c0 0x7fddf8004c40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fddf400a860 tx=0x7fddf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7226e736ea237d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4034630 con 0x7fddf80047c0 2023-08-25T09:40:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf402f070 con 0x7fddf80047c0 2023-08-25T09:40:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde0a59c640 1 -- 172.21.15.70:0/87660254 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf40349f0 con 0x7fddf80047c0 2023-08-25T09:40:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde110e7640 1 -- 172.21.15.70:0/87660254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf80047c0 msgr2=0x7fddf8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde110e7640 1 --2- 172.21.15.70:0/87660254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddf80047c0 0x7fddf8004c40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fddf400a860 tx=0x7fddf80057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde0b59e640 1 -- 172.21.15.70:0/87660254 reap_dead start 2023-08-25T09:40:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde110e7640 1 -- 172.21.15.70:0/87660254 shutdown_connections 2023-08-25T09:40:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde110e7640 1 -- 172.21.15.70:0/87660254 >> 172.21.15.70:0/87660254 conn(0x7fde0c069850 msgr2=0x7fde0c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde110e7640 1 -- 172.21.15.70:0/87660254 shutdown_connections 2023-08-25T09:40:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.085+0000 7fde110e7640 1 -- 172.21.15.70:0/87660254 wait complete. 2023-08-25T09:40:38.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.087+0000 7fde110e7640 1 Processor -- start 2023-08-25T09:40:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.087+0000 7fde110e7640 1 -- start start 2023-08-25T09:40:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.088+0000 7fde110e7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf80047c0 0x7fde0c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.088+0000 7fde110e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 0x7fde0c152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.088+0000 7fde110e7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cd830 0x7fde0c0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.088+0000 7fde110e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c0ce200 con 0x7fde0c1552f0 2023-08-25T09:40:38.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.088+0000 7fde110e7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c153610 con 0x7fde0c0cd830 2023-08-25T09:40:38.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.088+0000 7fde110e7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0c1538f0 con 0x7fddf80047c0 2023-08-25T09:40:38.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 0x7fde0c152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 0x7fde0c152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45942/0 (socket says 172.21.15.70:45942) 2023-08-25T09:40:38.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 -- 172.21.15.70:0/1908885205 learned_addr learned my addr 172.21.15.70:0/1908885205 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:38.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 -- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf80047c0 msgr2=0x7fde0c1524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:38.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf80047c0 0x7fde0c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0b59e640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cd830 0x7fde0c0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 -- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cd830 msgr2=0x7fde0c0cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cd830 0x7fde0c0cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 -- 172.21.15.70:0/1908885205 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0ae360 con 0x7fde0c1552f0 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.089+0000 7fde0259c640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 0x7fde0c152ca0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fddf40012e0 tx=0x7fddf4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.090+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4034840 con 0x7fde0c1552f0 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.090+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddf40349a0 con 0x7fde0c1552f0 2023-08-25T09:40:38.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.090+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf4034d30 con 0x7fde0c1552f0 2023-08-25T09:40:38.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.090+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0c0ae640 con 0x7fde0c1552f0 2023-08-25T09:40:38.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.090+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0c153bd0 con 0x7fde0c1552f0 2023-08-25T09:40:38.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.091+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde0c04e420 con 0x7fde0c1552f0 2023-08-25T09:40:38.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.092+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fddf4031030 con 0x7fde0c1552f0 2023-08-25T09:40:38.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.094+0000 7fde03fff640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fddec06a050 0x7fddec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:38.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.094+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fddf40a9dc0 con 0x7fde0c1552f0 2023-08-25T09:40:38.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.094+0000 7fde0ad9d640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fddec06a050 0x7fddec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:38.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.095+0000 7fde0ad9d640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fddec06a050 0x7fddec06c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:38.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.098+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fddf4075410 con 0x7fde0c1552f0 2023-08-25T09:40:38.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.333+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fde0c0701e0 con 0x7fde0c1552f0 2023-08-25T09:40:38.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.334+0000 7fde03fff640 1 -- 172.21.15.70:0/1908885205 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fddf4046020 con 0x7fde0c1552f0 2023-08-25T09:40:38.337 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:40:38.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fddec06a050 msgr2=0x7fddec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:38.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fddec06a050 0x7fddec06c510 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009870 tx=0x7fddfc008040 comp rx=0 tx=0).stop 2023-08-25T09:40:38.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 msgr2=0x7fde0c152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:38.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 0x7fde0c152ca0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fddf40012e0 tx=0x7fddf4001d80 comp rx=0 tx=0).stop 2023-08-25T09:40:38.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 shutdown_connections 2023-08-25T09:40:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fddec06a050 0x7fddec06c510 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde0c0cd830 0x7fde0c0cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde0c1552f0 0x7fde0c152ca0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 --2- 172.21.15.70:0/1908885205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddf80047c0 0x7fde0c1524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 >> 172.21.15.70:0/1908885205 conn(0x7fde0c069850 msgr2=0x7fde0c15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 shutdown_connections 2023-08-25T09:40:38.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:38.336+0000 7fde110e7640 1 -- 172.21.15.70:0/1908885205 wait complete. 2023-08-25T09:40:38.357 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:40:38.358 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:40:38.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:40:38.359 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:40:38.359 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:38.360 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:40:38.360 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:40:38.361 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:40:38.361 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:40:38.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:38.362 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:40:38.423 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:40:38.423 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:40:38.424 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:40:38.424 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.424 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:40:38.424 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.424 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:40:38.425 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:38.425 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:40:38.425 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:40:38.425 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:38.425 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:40:38.425 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:40:38.426 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:38.426 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:40:38.426 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:38.426 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:40:38.426 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42251sec preferred_lft 42251sec 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.427 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:40:38.428 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.428 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:40:38.428 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:40:38.428 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:40:38.428 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.428 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:40:38.429 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:40:38.429 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:40:38.429 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:40:38.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:38.431 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:40:38.526 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:40:38.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:38.532 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:40:38.590 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:40:38.591 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:40:38.591 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:40:38.591 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:40:38.591 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:40:38.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:38.592 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:40:38.593 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:40:38.593 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:40:38.593 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:40:38.594 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:40:38.594 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:40:38.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:38.779 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:40:38.836 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:40:38.836 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:40:38.836 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:40:38.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:38.837 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:40:38.916 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:40:38.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:40:38.917 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:38.971 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:40:38.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:40:38.972 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:39.023 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:40:39.117 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:40:39.118 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:40:39.205 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:40:39.209 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:40:39.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:39.210 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:40:39.267 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:40:39.267 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:40:39.273 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:40:39.352 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:40:39.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:40:39.426 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:40:39.429 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:40:39.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:39.430 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:40:39.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:40:39.458+0000 7f584f44c480 -1 init, newargv = 0x55eca268cf50 newargc=15 2023-08-25T09:40:39.460 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[87948]: starting ceph client 2023-08-25T09:40:39.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[87948]: starting fuse 2023-08-25T09:40:39.476 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:40:39.476 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:40:39.477 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:40:39.477 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:40:39.477 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:40:39.478 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:40:39.478 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:40:39.478 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:40:39.479 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:40:39.479 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:40:39.479 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:40:39.480 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:40:39.480 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:40:39.480 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:40:39.480 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:40:39.481 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:40:39.481 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:40:39.481 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:40:39.482 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:40:39.482 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:40:39.482 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:40:39.483 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:40:39.483 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:40:39.484 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:40:39.484 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:40:39.484 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:40:39.485 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:40:39.485 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:40:39.485 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:40:39.683 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.87948.asok 2023-08-25T09:40:39.687 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:40:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:39.700 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.87948.asok 2023-08-25T09:40:39.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:39.701 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.87948.asok status 2023-08-25T09:40:39.940 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:40:39.941 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "87948", 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4870, 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:40:39.942 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4870 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2898095474 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:40:39.943 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2898095474 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4870 192.168.0.1:0/2898095474", 2023-08-25T09:40:39.944 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/2898095474", 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 16, 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 37, 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:40:39.945 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:40:39.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:39.956 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:40:39.957 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:40.010 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:40:40.011 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:40.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:40.013 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:40.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.328+0000 7f8e8601f640 1 Processor -- start 2023-08-25T09:40:40.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.328+0000 7f8e8601f640 1 -- start start 2023-08-25T09:40:40.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.330+0000 7f8e8601f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 0x7f8e800ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:40.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.330+0000 7f8e8601f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8e8015aa10 0x7f8e800b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:40.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.330+0000 7f8e8601f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800ceed0 con 0x7f8e800b2680 2023-08-25T09:40:40.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.330+0000 7f8e8601f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e800b2b10 con 0x7f8e8015aa10 2023-08-25T09:40:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.330+0000 7f8e8601f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80159d10 con 0x7f8e800ce5b0 2023-08-25T09:40:40.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.331+0000 7f8e7f7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 0x7f8e800ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.331+0000 7f8e7f7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 0x7f8e800ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:50786/0 (socket says 172.21.15.70:50786) 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.331+0000 7f8e7f7fe640 1 -- 172.21.15.70:0/1644598170 learned_addr learned my addr 172.21.15.70:0/1644598170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.331+0000 7f8e7ffff640 1 --1- 172.21.15.70:0/1644598170 >> v1:172.21.15.155:6789/0 conn(0x7f8e800b2680 0x7f8e8006be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:35296/0 (socket says 172.21.15.70:35296) 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.331+0000 7f8e77fff640 1 --2- 172.21.15.70:0/1644598170 >> v2:172.21.15.155:3300/0 conn(0x7f8e8015aa10 0x7f8e800b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.332+0000 7f8e7f7fe640 1 -- 172.21.15.70:0/1644598170 >> v2:172.21.15.155:3300/0 conn(0x7f8e8015aa10 msgr2=0x7f8e800b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.332+0000 7f8e7f7fe640 1 --2- 172.21.15.70:0/1644598170 >> v2:172.21.15.155:3300/0 conn(0x7f8e8015aa10 0x7f8e800b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.332+0000 7f8e7f7fe640 1 -- 172.21.15.70:0/1644598170 >> v1:172.21.15.155:6789/0 conn(0x7f8e800b2680 legacy=0x7f8e8006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.332+0000 7f8e7f7fe640 1 -- 172.21.15.70:0/1644598170 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8015d3f0 con 0x7f8e800ce5b0 2023-08-25T09:40:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.333+0000 7f8e7f7fe640 1 --2- 172.21.15.70:0/1644598170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 0x7f8e800ce990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009870 tx=0x7f8e7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b8d4f14468f499 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.334+0000 7f8e7effd640 1 -- 172.21.15.70:0/1644598170 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70030840 con 0x7f8e800ce5b0 2023-08-25T09:40:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.334+0000 7f8e7effd640 1 -- 172.21.15.70:0/1644598170 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e700309a0 con 0x7f8e800ce5b0 2023-08-25T09:40:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.334+0000 7f8e7effd640 1 -- 172.21.15.70:0/1644598170 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e70030d30 con 0x7f8e800ce5b0 2023-08-25T09:40:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.334+0000 7f8e8601f640 1 -- 172.21.15.70:0/1644598170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 msgr2=0x7f8e800ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:40.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.334+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1644598170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 0x7f8e800ce990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009870 tx=0x7f8e7002b980 comp rx=0 tx=0).stop 2023-08-25T09:40:40.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.335+0000 7f8e8601f640 1 -- 172.21.15.70:0/1644598170 shutdown_connections 2023-08-25T09:40:40.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.335+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1644598170 >> v2:172.21.15.155:3300/0 conn(0x7f8e8015aa10 0x7f8e800b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.335+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1644598170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e800ce5b0 0x7f8e800ce990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.335+0000 7f8e8601f640 1 -- 172.21.15.70:0/1644598170 >> 172.21.15.70:0/1644598170 conn(0x7f8e80069850 msgr2=0x7f8e80069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:40.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.335+0000 7f8e8601f640 1 -- 172.21.15.70:0/1644598170 shutdown_connections 2023-08-25T09:40:40.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.335+0000 7f8e8601f640 1 -- 172.21.15.70:0/1644598170 wait complete. 2023-08-25T09:40:40.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.336+0000 7f8e8601f640 1 Processor -- start 2023-08-25T09:40:40.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.337+0000 7f8e8601f640 1 -- start start 2023-08-25T09:40:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.337+0000 7f8e8601f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800b2680 0x7f8e8013a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.337+0000 7f8e8601f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 0x7f8e801571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e8601f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8015aa10 0x7f8e80157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:40.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e8601f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80151f80 con 0x7f8e800ce5b0 2023-08-25T09:40:40.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e8601f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80151370 con 0x7f8e800b2680 2023-08-25T09:40:40.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e8601f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e80151650 con 0x7f8e8015aa10 2023-08-25T09:40:40.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 0x7f8e801571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:40.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 0x7f8e801571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54800/0 (socket says 172.21.15.70:54800) 2023-08-25T09:40:40.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 -- 172.21.15.70:0/1744945330 learned_addr learned my addr 172.21.15.70:0/1744945330 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:40.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e7ffff640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8015aa10 0x7f8e80157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:40.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e7f7fe640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800b2680 0x7f8e8013a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:40.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 -- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8015aa10 msgr2=0x7f8e80157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:40.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8015aa10 0x7f8e80157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 -- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800b2680 msgr2=0x7f8e8013a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:40.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800b2680 0x7f8e8013a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e77fff640 1 -- 172.21.15.70:0/1744945330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e700094d0 con 0x7f8e800ce5b0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.338+0000 7f8e7ffff640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8015aa10 0x7f8e80157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.339+0000 7f8e77fff640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 0x7f8e801571c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9b0 tx=0x7f8e6800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.339+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68009670 con 0x7f8e800ce5b0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.339+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6800c9a0 con 0x7f8e800ce5b0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.339+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800cd50 con 0x7f8e800ce5b0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.339+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e80151930 con 0x7f8e800ce5b0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.339+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e80152410 con 0x7f8e800ce5b0 2023-08-25T09:40:40.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.340+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e801520f0 con 0x7f8e800ce5b0 2023-08-25T09:40:40.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.341+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8e68013470 con 0x7f8e800ce5b0 2023-08-25T09:40:40.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.342+0000 7f8e7cff9640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e6006a050 0x7f8e6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:40.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.342+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8e68088910 con 0x7f8e800ce5b0 2023-08-25T09:40:40.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.342+0000 7f8e7f7fe640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e6006a050 0x7f8e6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:40.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.343+0000 7f8e7f7fe640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e6006a050 0x7f8e6006c510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e70036000 tx=0x7f8e7003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:40.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.346+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8e68053f20 con 0x7f8e800ce5b0 2023-08-25T09:40:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.572+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e8004e420 con 0x7f8e800ce5b0 2023-08-25T09:40:40.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.573+0000 7f8e7cff9640 1 -- 172.21.15.70:0/1744945330 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f8e68057540 con 0x7f8e800ce5b0 2023-08-25T09:40:40.576 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:40.576 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e6006a050 msgr2=0x7f8e6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e6006a050 0x7f8e6006c510 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8e70036000 tx=0x7f8e7003b040 comp rx=0 tx=0).stop 2023-08-25T09:40:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 msgr2=0x7f8e801571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 0x7f8e801571c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800e9b0 tx=0x7f8e6800ee70 comp rx=0 tx=0).stop 2023-08-25T09:40:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 shutdown_connections 2023-08-25T09:40:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8e6006a050 0x7f8e6006c510 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e8015aa10 0x7f8e80157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e800ce5b0 0x7f8e801571c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 --2- 172.21.15.70:0/1744945330 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e800b2680 0x7f8e8013a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:40.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 >> 172.21.15.70:0/1744945330 conn(0x7f8e80069850 msgr2=0x7f8e80069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:40.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 shutdown_connections 2023-08-25T09:40:40.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:40.576+0000 7f8e8601f640 1 -- 172.21.15.70:0/1744945330 wait complete. 2023-08-25T09:40:40.596 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-08-25T09:40:40.846 INFO:teuthology.orchestra.run.smithi155.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":5361,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.400618643,"mdsmap_epoch":16,"osdmap_epoch":37,"osdmap_epoch_barrier":35,"uptime":23.188435637000001} 2023-08-25T09:40:40.847 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 5361, "mdsmap_epoch": 16, "osdmap_epoch": 37, "osdmap_epoch_barrier": 35, "rank_uptime": 13.400618643, "state": "up:active", "uptime": 23.188435637, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:40:40.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-08-25T09:40:41.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.070+0000 7fb5295cf640 1 Processor -- start 2023-08-25T09:40:41.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.071+0000 7fb5295cf640 1 -- start start 2023-08-25T09:40:41.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5295cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb52406c020 0x7fb5240ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:41.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5295cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5240ce4c0 con 0x7fb5240cabd0 2023-08-25T09:40:41.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5295cf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb524161520 con 0x7fb5240b13f0 2023-08-25T09:40:41.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5295cf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5240ae6d0 con 0x7fb52406c020 2023-08-25T09:40:41.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5227fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb52406c020 0x7fb5240ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:41.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5227fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb52406c020 0x7fb5240ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54814/0 (socket says 172.21.15.70:54814) 2023-08-25T09:40:41.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.072+0000 7fb5227fc640 1 -- 172.21.15.70:0/2375358720 learned_addr learned my addr 172.21.15.70:0/2375358720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.073+0000 7fb5227fc640 1 -- 172.21.15.70:0/2375358720 >> v1:172.21.15.155:6789/0 conn(0x7fb5240b13f0 legacy=0x7fb5240ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.073+0000 7fb5227fc640 1 -- 172.21.15.70:0/2375358720 >> v1:172.21.15.70:6789/0 conn(0x7fb5240cabd0 legacy=0x7fb5240b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.073+0000 7fb5227fc640 1 -- 172.21.15.70:0/2375358720 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240ae110 con 0x7fb52406c020 2023-08-25T09:40:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.074+0000 7fb5227fc640 1 --2- 172.21.15.70:0/2375358720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52406c020 0x7fb5240ce9a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009870 tx=0x7fb50c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427131702105e520 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.074+0000 7fb521ffb640 1 -- 172.21.15.70:0/2375358720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c030860 con 0x7fb52406c020 2023-08-25T09:40:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.074+0000 7fb521ffb640 1 -- 172.21.15.70:0/2375358720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c0309c0 con 0x7fb52406c020 2023-08-25T09:40:41.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.074+0000 7fb521ffb640 1 -- 172.21.15.70:0/2375358720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c030d50 con 0x7fb52406c020 2023-08-25T09:40:41.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 -- 172.21.15.70:0/2375358720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52406c020 msgr2=0x7fb5240ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:41.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 --2- 172.21.15.70:0/2375358720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52406c020 0x7fb5240ce9a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009870 tx=0x7fb50c02b9a0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 -- 172.21.15.70:0/2375358720 shutdown_connections 2023-08-25T09:40:41.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 --2- 172.21.15.70:0/2375358720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb52406c020 0x7fb5240ce9a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 -- 172.21.15.70:0/2375358720 >> 172.21.15.70:0/2375358720 conn(0x7fb524069a60 msgr2=0x7fb524069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:41.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 -- 172.21.15.70:0/2375358720 shutdown_connections 2023-08-25T09:40:41.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.075+0000 7fb5295cf640 1 -- 172.21.15.70:0/2375358720 wait complete. 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.077+0000 7fb5295cf640 1 Processor -- start 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.077+0000 7fb5295cf640 1 -- start start 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5295cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb52406c020 0x7fb5240c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5295cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 0x7fb524157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5295cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5240cabd0 0x7fb52406f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5295cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524070ce0 con 0x7fb5240b13f0 2023-08-25T09:40:41.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5295cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52406fc40 con 0x7fb5240cabd0 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5295cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52406ff20 con 0x7fb52406c020 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5227fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 0x7fb524157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5227fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 0x7fb524157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54828/0 (socket says 172.21.15.70:54828) 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb5227fc640 1 -- 172.21.15.70:0/1414257728 learned_addr learned my addr 172.21.15.70:0/1414257728 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.078+0000 7fb522ffd640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb52406c020 0x7fb5240c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5237fe640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5240cabd0 0x7fb52406f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:41.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5227fc640 1 -- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb52406c020 msgr2=0x7fb5240c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5227fc640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb52406c020 0x7fb5240c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5227fc640 1 -- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5240cabd0 msgr2=0x7fb52406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5227fc640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5240cabd0 0x7fb52406f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5227fc640 1 -- 172.21.15.70:0/1414257728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0094d0 con 0x7fb5240b13f0 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5227fc640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 0x7fb524157db0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009870 tx=0x7fb50c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c030810 con 0x7fb5240b13f0 2023-08-25T09:40:41.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c039d90 con 0x7fb5240b13f0 2023-08-25T09:40:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c043630 con 0x7fb5240b13f0 2023-08-25T09:40:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb524070200 con 0x7fb5240b13f0 2023-08-25T09:40:41.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.079+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5240707c0 con 0x7fb5240b13f0 2023-08-25T09:40:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.081+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5240668a0 con 0x7fb5240b13f0 2023-08-25T09:40:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.081+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb50c02f3b0 con 0x7fb5240b13f0 2023-08-25T09:40:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.083+0000 7fb51b7fe640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb50406a050 0x7fb50406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.083+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb50c0aa9a0 con 0x7fb5240b13f0 2023-08-25T09:40:41.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.083+0000 7fb522ffd640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb50406a050 0x7fb50406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:41.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.084+0000 7fb522ffd640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb50406a050 0x7fb50406c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb514000f60 tx=0x7fb514009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:41.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.087+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb50c0760e0 con 0x7fb5240b13f0 2023-08-25T09:40:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.348+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fb52404e420 con 0x7fb5240b13f0 2023-08-25T09:40:41.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.349+0000 7fb51b7fe640 1 -- 172.21.15.70:0/1414257728 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1921 (secure 0 0 0) 0x7fb50c0487a0 con 0x7fb5240b13f0 2023-08-25T09:40:41.352 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:41.352 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:41.352 INFO:teuthology.orchestra.run.smithi070.stdout: "stats": { 2023-08-25T09:40:41.352 INFO:teuthology.orchestra.run.smithi070.stdout: "total_bytes": 1600000000, 2023-08-25T09:40:41.352 INFO:teuthology.orchestra.run.smithi070.stdout: "total_avail_bytes": 1597503728, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "total_used_bytes": 2496272, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "total_used_raw_bytes": 2496272, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "total_used_raw_ratio": 0.0015601699706166983, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "num_osds": 8, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "num_per_pool_osds": 0, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "num_per_pool_omap_osds": 0 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:41.353 INFO:teuthology.orchestra.run.smithi070.stdout: "stats_by_class": { 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: "hdd": { 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: "total_bytes": 1600000000, 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: "total_avail_bytes": 1597503728, 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: "total_used_bytes": 2496272, 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: "total_used_raw_bytes": 2496272, 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: "total_used_raw_ratio": 0.0015601699706166983 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:41.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: "name": ".mgr", 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 1, 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: "stats": { 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: "stored": 590368, 2023-08-25T09:40:41.355 INFO:teuthology.orchestra.run.smithi070.stdout: "objects": 2, 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: "kb_used": 577, 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: "bytes_used": 590368, 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: "percent_used": 0.0005296840681694448, 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: "max_avail": 556987968 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:41.356 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "rbd", 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 2, 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "stats": { 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "stored": 0, 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "objects": 0, 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "kb_used": 0, 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "bytes_used": 0, 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "percent_used": 0, 2023-08-25T09:40:41.357 INFO:teuthology.orchestra.run.smithi070.stdout: "max_avail": 556987968 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs_metadata", 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5, 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: "stats": { 2023-08-25T09:40:41.358 INFO:teuthology.orchestra.run.smithi070.stdout: "stored": 2410, 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: "objects": 22, 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: "kb_used": 3, 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: "bytes_used": 2410, 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: "percent_used": 2.1634173208440188e-06, 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: "max_avail": 556987968 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:41.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "cephfs_data", 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6, 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "stats": { 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "stored": 0, 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "objects": 0, 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "kb_used": 0, 2023-08-25T09:40:41.360 INFO:teuthology.orchestra.run.smithi070.stdout: "bytes_used": 0, 2023-08-25T09:40:41.361 INFO:teuthology.orchestra.run.smithi070.stdout: "percent_used": 0, 2023-08-25T09:40:41.361 INFO:teuthology.orchestra.run.smithi070.stdout: "max_avail": 556987968 2023-08-25T09:40:41.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:41.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:41.361 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:40:41.361 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb50406a050 msgr2=0x7fb50406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb50406a050 0x7fb50406c510 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb514000f60 tx=0x7fb514009040 comp rx=0 tx=0).stop 2023-08-25T09:40:41.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 msgr2=0x7fb524157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 0x7fb524157db0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009870 tx=0x7fb50c004730 comp rx=0 tx=0).stop 2023-08-25T09:40:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 shutdown_connections 2023-08-25T09:40:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb50406a050 0x7fb50406c510 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5240cabd0 0x7fb52406f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5240b13f0 0x7fb524157db0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 --2- 172.21.15.70:0/1414257728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb52406c020 0x7fb5240c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 >> 172.21.15.70:0/1414257728 conn(0x7fb524069a60 msgr2=0x7fb5240ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 shutdown_connections 2023-08-25T09:40:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41.352+0000 7fb5295cf640 1 -- 172.21.15.70:0/1414257728 wait complete. 2023-08-25T09:40:41.375 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:40:41.376 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:41.376 DEBUG:teuthology.orchestra.run.smithi070:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-08-25T09:40:41.377 DEBUG:teuthology.orchestra.run.smithi070:> os.fchmod(fd, 0O666) 2023-08-25T09:40:41.377 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(fd) 2023-08-25T09:40:41.377 DEBUG:teuthology.orchestra.run.smithi070:> os.close(fd) 2023-08-25T09:40:41.377 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:41.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:41.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:40:41.545 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:40:41.546 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:40:41.546 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:41.546 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:40:41.547 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:41.547 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:40:41.547 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:40:41.548 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:40:41.548 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:40:41.548 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:41.549 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:40:41.549 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:40:41.549 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:40:41.550 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:40:41.550 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:41.550 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:40:41.551 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:40:41.551 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:40:41.551 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:40:41.551 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:40:41.552 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:40:41.552 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:40:41.552 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:40:41.553 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:41.553 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:40:41.553 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:41.767 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:41.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:41.785 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:41.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:41.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95809.asok status 2023-08-25T09:40:42.028 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:40:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:40:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:40:42.029 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:40:42.030 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:40:42.030 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:40:42.030 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "95809", 2023-08-25T09:40:42.030 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:40:42.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:42.030 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 1, 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5553, 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 5553 2023-08-25T09:40:42.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:42.032 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.5553 192.168.0.1:0/2574353996", 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2574353996", 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 2, 2023-08-25T09:40:42.033 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 16, 2023-08-25T09:40:42.034 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 35, 2023-08-25T09:40:42.034 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 35, 2023-08-25T09:40:42.034 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:40:42.034 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:40:42.034 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:42.042 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-08-25T09:40:42.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.250+0000 7ff35e483640 1 Processor -- start 2023-08-25T09:40:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.250+0000 7ff35e483640 1 -- start start 2023-08-25T09:40:42.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff35e483640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff35815acd0 0x7ff35815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff35e483640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3580ca780 0x7ff3580cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff35e483640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff35e483640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35806c0e0 con 0x7ff35815acd0 2023-08-25T09:40:42.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff35e483640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3580b12b0 con 0x7ff3580ca780 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff35e483640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35806ae20 con 0x7ff3580b3cf0 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.252+0000 7ff357fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff35815acd0 0x7ff35815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff35815acd0 0x7ff35815b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54834/0 (socket says 172.21.15.70:54834) 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 -- 172.21.15.70:0/3944156517 learned_addr learned my addr 172.21.15.70:0/3944156517 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff35ca20640 1 --2- 172.21.15.70:0/3944156517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff3577fe640 1 --2- 172.21.15.70:0/3944156517 >> v2:172.21.15.155:3300/0 conn(0x7ff3580ca780 0x7ff3580cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 -- 172.21.15.70:0/3944156517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 msgr2=0x7ff3580b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:42.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 --2- 172.21.15.70:0/3944156517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 -- 172.21.15.70:0/3944156517 >> v2:172.21.15.155:3300/0 conn(0x7ff3580ca780 msgr2=0x7ff3580cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:42.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 --2- 172.21.15.70:0/3944156517 >> v2:172.21.15.155:3300/0 conn(0x7ff3580ca780 0x7ff3580cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.253+0000 7ff357fff640 1 -- 172.21.15.70:0/3944156517 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35806b910 con 0x7ff35815acd0 2023-08-25T09:40:42.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.255+0000 7ff357fff640 1 --2- 172.21.15.70:0/3944156517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff35815acd0 0x7ff35815b0b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff35806cd90 tx=0x7ff34802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798c1660d1f3dff6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:42.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.255+0000 7ff356ffd640 1 -- 172.21.15.70:0/3944156517 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480317c0 con 0x7ff35815acd0 2023-08-25T09:40:42.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.255+0000 7ff356ffd640 1 -- 172.21.15.70:0/3944156517 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff348031920 con 0x7ff35815acd0 2023-08-25T09:40:42.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.255+0000 7ff356ffd640 1 -- 172.21.15.70:0/3944156517 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348031cb0 con 0x7ff35815acd0 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 -- 172.21.15.70:0/3944156517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff35815acd0 msgr2=0x7ff35815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 --2- 172.21.15.70:0/3944156517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff35815acd0 0x7ff35815b0b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff35806cd90 tx=0x7ff34802b860 comp rx=0 tx=0).stop 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 -- 172.21.15.70:0/3944156517 shutdown_connections 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 --2- 172.21.15.70:0/3944156517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 --2- 172.21.15.70:0/3944156517 >> v2:172.21.15.155:3300/0 conn(0x7ff3580ca780 0x7ff3580cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 --2- 172.21.15.70:0/3944156517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff35815acd0 0x7ff35815b0b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 -- 172.21.15.70:0/3944156517 >> 172.21.15.70:0/3944156517 conn(0x7ff3580ce760 msgr2=0x7ff3580ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:42.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 -- 172.21.15.70:0/3944156517 shutdown_connections 2023-08-25T09:40:42.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.256+0000 7ff35e483640 1 -- 172.21.15.70:0/3944156517 wait complete. 2023-08-25T09:40:42.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.258+0000 7ff35e483640 1 Processor -- start 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.258+0000 7ff35e483640 1 -- start start 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.259+0000 7ff35e483640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.259+0000 7ff35e483640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 0x7ff3581edb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.259+0000 7ff35e483640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35815acd0 0x7ff3581ee0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.259+0000 7ff35e483640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580cb9a0 con 0x7ff3580ca780 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.259+0000 7ff35e483640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580cbc80 con 0x7ff35815acd0 2023-08-25T09:40:42.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.259+0000 7ff35e483640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3580cbf60 con 0x7ff3580b3cf0 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 0x7ff3581edb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 0x7ff3581edb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54838/0 (socket says 172.21.15.70:54838) 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 -- 172.21.15.70:0/3181164633 learned_addr learned my addr 172.21.15.70:0/3181164633 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff35ca20640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35815acd0 0x7ff3581ee0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff357fff640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 -- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 msgr2=0x7ff3580cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:42.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 -- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35815acd0 msgr2=0x7ff3581ee0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:42.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35815acd0 0x7ff3581ee0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:42.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 -- 172.21.15.70:0/3181164633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348009470 con 0x7ff3580ca780 2023-08-25T09:40:42.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff357fff640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:42.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.260+0000 7ff3577fe640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 0x7ff3581edb90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00d800 tx=0x7ff34c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:42.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.261+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c00b6c0 con 0x7ff3580ca780 2023-08-25T09:40:42.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.261+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c00b820 con 0x7ff3580ca780 2023-08-25T09:40:42.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.261+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c00bbb0 con 0x7ff3580ca780 2023-08-25T09:40:42.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.261+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35815bd60 con 0x7ff3580ca780 2023-08-25T09:40:42.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.261+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3580cb5d0 con 0x7ff3580ca780 2023-08-25T09:40:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.262+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35804e420 con 0x7ff3580ca780 2023-08-25T09:40:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.263+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff34c00bd10 con 0x7ff3580ca780 2023-08-25T09:40:42.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.264+0000 7ff354ff9640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff34406a050 0x7ff34406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:42.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.264+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff34c088c00 con 0x7ff3580ca780 2023-08-25T09:40:42.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.264+0000 7ff357fff640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff34406a050 0x7ff34406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:42.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.265+0000 7ff357fff640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff34406a050 0x7ff34406c510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff348036000 tx=0x7ff34803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:42.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.269+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff34c054210 con 0x7ff3580ca780 2023-08-25T09:40:42.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:42.467+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7ff35806bec0 con 0x7ff3580ca780 2023-08-25T09:40:43.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.275+0000 7ff354ff9640 1 -- 172.21.15.70:0/3181164633 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v38) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff34c057830 con 0x7ff3580ca780 2023-08-25T09:40:43.278 INFO:teuthology.orchestra.run.smithi070.stderr:pauserd,pausewr is set 2023-08-25T09:40:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.278+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff34406a050 msgr2=0x7ff34406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.278+0000 7ff35e483640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff34406a050 0x7ff34406c510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff348036000 tx=0x7ff34803d040 comp rx=0 tx=0).stop 2023-08-25T09:40:43.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.278+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 msgr2=0x7ff3581edb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.278+0000 7ff35e483640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 0x7ff3581edb90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff34c00d800 tx=0x7ff34c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.278+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 shutdown_connections 2023-08-25T09:40:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff34406a050 0x7ff34406c510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff35815acd0 0x7ff3581ee0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3580ca780 0x7ff3581edb90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 --2- 172.21.15.70:0/3181164633 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3580b3cf0 0x7ff3580cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 >> 172.21.15.70:0/3181164633 conn(0x7ff3580ce760 msgr2=0x7ff35815a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 shutdown_connections 2023-08-25T09:40:43.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.279+0000 7ff35e483640 1 -- 172.21.15.70:0/3181164633 wait complete. 2023-08-25T09:40:43.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-08-25T09:40:43.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:40:43.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:40:43.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.500+0000 7faae2446640 1 Processor -- start 2023-08-25T09:40:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.500+0000 7faae2446640 1 -- start start 2023-08-25T09:40:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faae2446640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faadc0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faae2446640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faadc0b12b0 con 0x7faadc15acd0 2023-08-25T09:40:43.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faae2446640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faadc06ae20 con 0x7faadc0ca780 2023-08-25T09:40:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faae2446640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0b47e0 con 0x7faadc0b4400 2023-08-25T09:40:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faadbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7faadc15acd0 0x7faadc15b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40538/0 (socket says 172.21.15.70:40538) 2023-08-25T09:40:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faadbfff640 1 -- 172.21.15.70:0/141633146 learned_addr learned my addr 172.21.15.70:0/141633146 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faae09e3640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faadc0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478983725 0 0) 0x7faadc0b12b0 con 0x7faadc15acd0 2023-08-25T09:40:43.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.502+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faac8003520 con 0x7faadc15acd0 2023-08-25T09:40:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377315470 0 0) 0x7faac8003520 con 0x7faadc15acd0 2023-08-25T09:40:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 msgr2=0x7faadc0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faadc0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 >> v1:172.21.15.155:6789/0 conn(0x7faadc0ca780 legacy=0x7faadc0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc15b7c0 con 0x7faadc15acd0 2023-08-25T09:40:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7faacc0017f0 con 0x7faadc15acd0 2023-08-25T09:40:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 >> v1:172.21.15.70:6789/0 conn(0x7faadc15acd0 legacy=0x7faadc15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faac80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faac80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faac8004d00 0x7faac8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc0b47e0 con 0x7faac8003f20 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac800bb10 con 0x7faac8004d00 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac800bcd0 con 0x7faadc0b4400 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faad3fff640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faac80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.503+0000 7faadbfff640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faac80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:43.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faad3fff640 1 -- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 msgr2=0x7faac80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faae09e3640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faac8004d00 0x7faac8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faad3fff640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faac80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faad3fff640 1 -- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faac8004d00 msgr2=0x7faac8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faad3fff640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faac8004d00 0x7faac8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faad3fff640 1 -- 172.21.15.70:0/141633146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac8003520 con 0x7faac8003f20 2023-08-25T09:40:43.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.504+0000 7faadbfff640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc0b4400 0x7faac80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:43.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faad3fff640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faac80047c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faac4001750 tx=0x7faac80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcc6d386cfe3d59 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac40052d0 con 0x7faac8003f20 2023-08-25T09:40:43.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac402e050 con 0x7faac8003f20 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faadb7fe640 1 -- 172.21.15.70:0/141633146 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac40035f0 con 0x7faac8003f20 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faae2446640 1 -- 172.21.15.70:0/141633146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 msgr2=0x7faac80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faae2446640 1 --2- 172.21.15.70:0/141633146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faac80047c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7faac4001750 tx=0x7faac80057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.505+0000 7faae09e3640 1 -- 172.21.15.70:0/141633146 reap_dead start 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.506+0000 7faae2446640 1 -- 172.21.15.70:0/141633146 shutdown_connections 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.506+0000 7faae2446640 1 -- 172.21.15.70:0/141633146 >> 172.21.15.70:0/141633146 conn(0x7faadc0ce760 msgr2=0x7faadc0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:43.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.506+0000 7faae2446640 1 -- 172.21.15.70:0/141633146 shutdown_connections 2023-08-25T09:40:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.506+0000 7faae2446640 1 -- 172.21.15.70:0/141633146 wait complete. 2023-08-25T09:40:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.507+0000 7faae2446640 1 Processor -- start 2023-08-25T09:40:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.507+0000 7faae2446640 1 -- start start 2023-08-25T09:40:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.508+0000 7faae2446640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faadc156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.508+0000 7faae2446640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc14c6c0 0x7faadc148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.508+0000 7faae2446640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faadc14e780 0x7faadc14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.508+0000 7faae2446640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc154860 con 0x7faac8003f20 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.508+0000 7faae2446640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faadc14f140 con 0x7faadc14e780 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.508+0000 7faae2446640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faadc156da0 con 0x7faadc14c6c0 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faadc156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faadc156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54850/0 (socket says 172.21.15.70:54850) 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 -- 172.21.15.70:0/1938600166 learned_addr learned my addr 172.21.15.70:0/1938600166 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:43.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 -- 172.21.15.70:0/1938600166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc14c6c0 msgr2=0x7faadc148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc14c6c0 0x7faadc148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 -- 172.21.15.70:0/1938600166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faadc14e780 msgr2=0x7faadc14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:43.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faadc14e780 0x7faadc14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:43.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc148760 con 0x7faac8003f20 2023-08-25T09:40:43.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.509+0000 7faadbfff640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faadc156860 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faacc00d850 tx=0x7faacc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:43.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.510+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc007480 con 0x7faac8003f20 2023-08-25T09:40:43.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.510+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faacc0075e0 con 0x7faac8003f20 2023-08-25T09:40:43.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.510+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faacc0059a0 con 0x7faac8003f20 2023-08-25T09:40:43.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.510+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faadc0cbe10 con 0x7faac8003f20 2023-08-25T09:40:43.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.510+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faadc1572b0 con 0x7faac8003f20 2023-08-25T09:40:43.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.511+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faadc04e420 con 0x7faac8003f20 2023-08-25T09:40:43.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.512+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faacc005b00 con 0x7faac8003f20 2023-08-25T09:40:43.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.512+0000 7faad97fa640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faabc06a080 0x7faabc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:43.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.513+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faacc088530 con 0x7faac8003f20 2023-08-25T09:40:43.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.513+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=39+}) v3 -- 0x7faabc072290 con 0x7faac8003f20 2023-08-25T09:40:43.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.513+0000 7faad3fff640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faabc06a080 0x7faabc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:43.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.513+0000 7faad3fff640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faabc06a080 0x7faabc06c540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faac4001750 tx=0x7faac402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:43.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.517+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faacc0575d0 con 0x7faac8003f20 2023-08-25T09:40:43.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:43.699+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7faadc148760 con 0x7faac8003f20 2023-08-25T09:40:44.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.280+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(39..39 src has 1..39) v4 ==== 346+0+0 (secure 0 0 0) 0x7faacc0016a0 con 0x7faac8003f20 2023-08-25T09:40:44.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.280+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=40}) v3 -- 0x7faabc072ad0 con 0x7faac8003f20 2023-08-25T09:40:44.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.286+0000 7faad97fa640 1 -- 172.21.15.70:0/1938600166 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7faadc148760 con 0x7faac8003f20 2023-08-25T09:40:44.289 INFO:teuthology.orchestra.run.smithi070.stderr:pauserd,pausewr is unset 2023-08-25T09:40:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faabc06a080 msgr2=0x7faabc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faabc06a080 0x7faabc06c540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faac4001750 tx=0x7faac402f000 comp rx=0 tx=0).stop 2023-08-25T09:40:44.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 msgr2=0x7faadc156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faadc156860 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7faacc00d850 tx=0x7faacc00dd10 comp rx=0 tx=0).stop 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 shutdown_connections 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faabc06a080 0x7faabc06c540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faadc14e780 0x7faadc14ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faadc14c6c0 0x7faadc148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.289+0000 7faae2446640 1 --2- 172.21.15.70:0/1938600166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faac8003f20 0x7faadc156860 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.290+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 >> 172.21.15.70:0/1938600166 conn(0x7faadc0ce760 msgr2=0x7faadc15b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:44.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.290+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 shutdown_connections 2023-08-25T09:40:44.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.290+0000 7faae2446640 1 -- 172.21.15.70:0/1938600166 wait complete. 2023-08-25T09:40:44.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:44.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.568+0000 7f886febf640 1 Processor -- start 2023-08-25T09:40:44.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.568+0000 7f886febf640 1 -- start start 2023-08-25T09:40:44.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.569+0000 7f886febf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 0x7f886806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886febf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f886815ec10 0x7f88680b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886febf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680b1b60 con 0x7f886806b2b0 2023-08-25T09:40:44.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886febf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886806bd80 con 0x7f886815ec10 2023-08-25T09:40:44.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886febf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886806b7c0 con 0x7f886815e830 2023-08-25T09:40:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886d45a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f886806b2b0 0x7f886815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40550/0 (socket says 172.21.15.70:40550) 2023-08-25T09:40:44.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886d45a640 1 -- 172.21.15.70:0/3460124254 learned_addr learned my addr 172.21.15.70:0/3460124254 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:44.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 0x7f886806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:44.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886e45c640 1 --2- 172.21.15.70:0/3460124254 >> v2:172.21.15.155:3300/0 conn(0x7f886815ec10 0x7f88680b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:44.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114431792 0 0) 0x7f88680b1b60 con 0x7f886806b2b0 2023-08-25T09:40:44.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.570+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8850003520 con 0x7f886806b2b0 2023-08-25T09:40:44.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3070510138 0 0) 0x7f8850003520 con 0x7f886806b2b0 2023-08-25T09:40:44.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 msgr2=0x7f886806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 0x7f886806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 >> v2:172.21.15.155:3300/0 conn(0x7f886815ec10 msgr2=0x7f88680b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 --2- 172.21.15.70:0/3460124254 >> v2:172.21.15.155:3300/0 conn(0x7f886815ec10 0x7f88680b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886806baa0 con 0x7f886806b2b0 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f885c001670 con 0x7f886806b2b0 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 >> v1:172.21.15.70:6789/0 conn(0x7f886806b2b0 legacy=0x7f886815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8850003970 0x7f8850003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 0x7f88500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850004d00 0x7f8850005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886806bd80 con 0x7f8850003970 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886806b7c0 con 0x7f8850004d00 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885000ba30 con 0x7f886815e830 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.571+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8850003970 0x7f8850003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:44.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.572+0000 7f886dc5b640 1 -- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 msgr2=0x7f88500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.572+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f886815e830 0x7f88500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.572+0000 7f886dc5b640 1 -- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850004d00 msgr2=0x7f8850005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.572+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850004d00 0x7f8850005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.572+0000 7f886dc5b640 1 -- 172.21.15.70:0/3460124254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8850003520 con 0x7f8850003970 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8850003970 0x7f8850003d90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8858007390 tx=0x7f88500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fa96a309874b24 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885802ccf0 con 0x7f8850003970 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885802ce50 con 0x7f8850003970 2023-08-25T09:40:44.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886cc59640 1 -- 172.21.15.70:0/3460124254 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8858034660 con 0x7f8850003970 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886febf640 1 -- 172.21.15.70:0/3460124254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8850003970 msgr2=0x7f8850003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886febf640 1 --2- 172.21.15.70:0/3460124254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8850003970 0x7f8850003d90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8858007390 tx=0x7f88500057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.573+0000 7f886e45c640 1 -- 172.21.15.70:0/3460124254 reap_dead start 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.574+0000 7f886febf640 1 -- 172.21.15.70:0/3460124254 shutdown_connections 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.574+0000 7f886febf640 1 -- 172.21.15.70:0/3460124254 >> 172.21.15.70:0/3460124254 conn(0x7f8868069850 msgr2=0x7f8868069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.574+0000 7f886febf640 1 -- 172.21.15.70:0/3460124254 shutdown_connections 2023-08-25T09:40:44.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.574+0000 7f886febf640 1 -- 172.21.15.70:0/3460124254 wait complete. 2023-08-25T09:40:44.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.575+0000 7f886febf640 1 Processor -- start 2023-08-25T09:40:44.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.575+0000 7f886febf640 1 -- start start 2023-08-25T09:40:44.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.576+0000 7f886febf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850003970 0x7f8868145e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.576+0000 7f886febf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 0x7f88680ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.576+0000 7f886febf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8868144800 0x7f8868144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.576+0000 7f886febf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88681451c0 con 0x7f8868147fa0 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.576+0000 7f886febf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8868147170 con 0x7f8850003970 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.576+0000 7f886febf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886814b9c0 con 0x7f8868144800 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 0x7f88680ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:44.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 0x7f88680ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54874/0 (socket says 172.21.15.70:54874) 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 -- 172.21.15.70:0/3101909472 learned_addr learned my addr 172.21.15.70:0/3101909472 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 -- 172.21.15.70:0/3101909472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8868144800 msgr2=0x7f8868144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8868144800 0x7f8868144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 -- 172.21.15.70:0/3101909472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850003970 msgr2=0x7f8868145e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850003970 0x7f8868145e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 -- 172.21.15.70:0/3101909472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886815ca20 con 0x7f8868147fa0 2023-08-25T09:40:44.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.577+0000 7f886d45a640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 0x7f88680ccf70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f885c00c700 tx=0x7f885c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.578+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c0074e0 con 0x7f8868147fa0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.578+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c007660 con 0x7f8868147fa0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.578+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c004a20 con 0x7f8868147fa0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.578+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8868148420 con 0x7f8868147fa0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.578+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8868146840 con 0x7f8868147fa0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.579+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886804e420 con 0x7f8868147fa0 2023-08-25T09:40:44.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.580+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f885c004b80 con 0x7f8868147fa0 2023-08-25T09:40:44.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.580+0000 7f88667fc640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f885406a050 0x7f885406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:44.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.581+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f885c087930 con 0x7f8868147fa0 2023-08-25T09:40:44.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.581+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f885406a050 0x7f885406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:44.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.582+0000 7f886dc5b640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f885406a050 0x7f885406c510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8858036000 tx=0x7f885802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:44.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.585+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f885c053070 con 0x7f8868147fa0 2023-08-25T09:40:44.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.764+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f886806c620 con 0x7f8868147fa0 2023-08-25T09:40:44.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.767+0000 7f88667fc640 1 -- 172.21.15.70:0/3101909472 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18401 (secure 0 0 0) 0x7f885c018090 con 0x7f8868147fa0 2023-08-25T09:40:44.770 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:44.771 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:44.272862+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:44.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f885406a050 msgr2=0x7f885406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f885406a050 0x7f885406c510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8858036000 tx=0x7f885802e000 comp rx=0 tx=0).stop 2023-08-25T09:40:44.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 msgr2=0x7f88680ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:44.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 0x7f88680ccf70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f885c00c700 tx=0x7f885c00cbc0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 shutdown_connections 2023-08-25T09:40:44.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f885406a050 0x7f885406c510 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8868144800 0x7f8868144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8868147fa0 0x7f88680ccf70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 --2- 172.21.15.70:0/3101909472 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8850003970 0x7f8868145e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:44.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.769+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 >> 172.21.15.70:0/3101909472 conn(0x7f8868069850 msgr2=0x7f8868159230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:44.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.770+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 shutdown_connections 2023-08-25T09:40:44.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44.770+0000 7f886febf640 1 -- 172.21.15.70:0/3101909472 wait complete. 2023-08-25T09:40:44.790 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:40:44.790 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:44.791 DEBUG:teuthology.orchestra.run.smithi070:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-08-25T09:40:44.792 DEBUG:teuthology.orchestra.run.smithi070:> os.fchmod(fd, 0O666) 2023-08-25T09:40:44.793 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(fd) 2023-08-25T09:40:44.793 DEBUG:teuthology.orchestra.run.smithi070:> os.close(fd) 2023-08-25T09:40:44.794 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:44.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:44.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:40:44.951 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:40:44.951 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:40:44.951 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:44.951 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:40:44.952 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:44.952 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:40:44.953 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:40:44.953 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:40:44.953 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:40:44.954 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:44.954 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:40:44.954 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:40:44.954 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:40:44.955 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:40:44.955 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:44.955 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:40:44.956 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:40:44.956 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:40:44.956 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:40:44.957 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:40:44.957 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:40:44.957 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:40:44.958 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:40:44.958 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:44.958 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:40:44.959 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:45.175 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:45.192 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:45.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:45.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95809.asok status 2023-08-25T09:40:45.428 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:45.428 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:40:45.428 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:40:45.428 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "95809", 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:45.429 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 1, 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5553, 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 5553 2023-08-25T09:40:45.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:45.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.5553 192.168.0.1:0/2574353996", 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2574353996", 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 2, 2023-08-25T09:40:45.432 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 16, 2023-08-25T09:40:45.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 35, 2023-08-25T09:40:45.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 35, 2023-08-25T09:40:45.433 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:40:45.433 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:40:45.433 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:45.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:45.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.655+0000 7ff959c46640 1 Processor -- start 2023-08-25T09:40:45.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.655+0000 7ff959c46640 1 -- start start 2023-08-25T09:40:45.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.656+0000 7ff959c46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff95415e830 0x7ff95406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff959c46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff95406b2b0 0x7ff95415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff959c46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff9540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff959c46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95406bd80 con 0x7ff95415e830 2023-08-25T09:40:45.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff959c46640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9540b1b60 con 0x7ff95406b2b0 2023-08-25T09:40:45.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff959c46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95406b7c0 con 0x7ff95415c6d0 2023-08-25T09:40:45.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff9537fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff95415e830 0x7ff95406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:45.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff9537fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff95415e830 0x7ff95406ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54876/0 (socket says 172.21.15.70:54876) 2023-08-25T09:40:45.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff9537fe640 1 -- 172.21.15.70:0/3507129898 learned_addr learned my addr 172.21.15.70:0/3507129898 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:45.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff952ffd640 1 --2- 172.21.15.70:0/3507129898 >> v2:172.21.15.155:3300/0 conn(0x7ff95406b2b0 0x7ff95415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:45.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff9537fe640 1 -- 172.21.15.70:0/3507129898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 msgr2=0x7ff9540b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:45.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.657+0000 7ff953fff640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff9540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:45.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.658+0000 7ff9537fe640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff9540b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.658+0000 7ff9537fe640 1 -- 172.21.15.70:0/3507129898 >> v2:172.21.15.155:3300/0 conn(0x7ff95406b2b0 msgr2=0x7ff95415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.658+0000 7ff9537fe640 1 --2- 172.21.15.70:0/3507129898 >> v2:172.21.15.155:3300/0 conn(0x7ff95406b2b0 0x7ff95415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.658+0000 7ff9537fe640 1 -- 172.21.15.70:0/3507129898 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95406baa0 con 0x7ff95415e830 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.658+0000 7ff952ffd640 1 --2- 172.21.15.70:0/3507129898 >> v2:172.21.15.155:3300/0 conn(0x7ff95406b2b0 0x7ff95415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.658+0000 7ff953fff640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff9540b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.659+0000 7ff9537fe640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95415e830 0x7ff95406ad70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cf6f0 tx=0x7ff94402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11cb03460788f8a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.659+0000 7ff9527fc640 1 -- 172.21.15.70:0/3507129898 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944031860 con 0x7ff95415e830 2023-08-25T09:40:45.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.659+0000 7ff9527fc640 1 -- 172.21.15.70:0/3507129898 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9440319c0 con 0x7ff95415e830 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.659+0000 7ff9527fc640 1 -- 172.21.15.70:0/3507129898 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944031d50 con 0x7ff95415e830 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 -- 172.21.15.70:0/3507129898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95415e830 msgr2=0x7ff95406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95415e830 0x7ff95406ad70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff9540cf6f0 tx=0x7ff94402b900 comp rx=0 tx=0).stop 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 -- 172.21.15.70:0/3507129898 shutdown_connections 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff9540b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 --2- 172.21.15.70:0/3507129898 >> v2:172.21.15.155:3300/0 conn(0x7ff95406b2b0 0x7ff95415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 --2- 172.21.15.70:0/3507129898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95415e830 0x7ff95406ad70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 -- 172.21.15.70:0/3507129898 >> 172.21.15.70:0/3507129898 conn(0x7ff954069850 msgr2=0x7ff954069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:45.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 -- 172.21.15.70:0/3507129898 shutdown_connections 2023-08-25T09:40:45.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.660+0000 7ff959c46640 1 -- 172.21.15.70:0/3507129898 wait complete. 2023-08-25T09:40:45.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.662+0000 7ff959c46640 1 Processor -- start 2023-08-25T09:40:45.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.662+0000 7ff959c46640 1 -- start start 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff959c46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 0x7ff954155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff959c46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff95414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff959c46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff95415e830 0x7ff95406ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff959c46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff954156940 con 0x7ff95406b2b0 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff959c46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9540c9d30 con 0x7ff95415e830 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff959c46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95414bf90 con 0x7ff95415c6d0 2023-08-25T09:40:45.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff9537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 0x7ff954155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.663+0000 7ff9537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 0x7ff954155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54886/0 (socket says 172.21.15.70:54886) 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 -- 172.21.15.70:0/4251264960 learned_addr learned my addr 172.21.15.70:0/4251264960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 -- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 msgr2=0x7ff95414ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff95414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 -- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff95415e830 msgr2=0x7ff95406ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff953fff640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff95415e830 0x7ff95406ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:45.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff95415e830 0x7ff95406ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 -- 172.21.15.70:0/4251264960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9440094d0 con 0x7ff95406b2b0 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff9537fe640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 0x7ff954155630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff9440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.664+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944031850 con 0x7ff95406b2b0 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.665+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944044400 con 0x7ff95406b2b0 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.665+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440447b0 con 0x7ff95406b2b0 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.665+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff954155d30 con 0x7ff95406b2b0 2023-08-25T09:40:45.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.665+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9541562f0 con 0x7ff95406b2b0 2023-08-25T09:40:45.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.666+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff954156ab0 con 0x7ff95406b2b0 2023-08-25T09:40:45.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.667+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff94404d440 con 0x7ff95406b2b0 2023-08-25T09:40:45.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.668+0000 7ff93bfff640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff94006a050 0x7ff94006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:45.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.668+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff9440aad40 con 0x7ff95406b2b0 2023-08-25T09:40:45.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.668+0000 7ff952ffd640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff94006a050 0x7ff94006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:45.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.669+0000 7ff952ffd640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff94006a050 0x7ff94006c510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9540702e0 tx=0x7ff948008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:45.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.672+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff9440763f0 con 0x7ff95406b2b0 2023-08-25T09:40:45.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.900+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff954156a80 con 0x7ff95406b2b0 2023-08-25T09:40:45.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.901+0000 7ff93bfff640 1 -- 172.21.15.70:0/4251264960 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7ff944035090 con 0x7ff95406b2b0 2023-08-25T09:40:45.902 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:45.903 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:45.903 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:45.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.903+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff94006a050 msgr2=0x7ff94006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.903+0000 7ff959c46640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff94006a050 0x7ff94006c510 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9540702e0 tx=0x7ff948008040 comp rx=0 tx=0).stop 2023-08-25T09:40:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 msgr2=0x7ff954155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:45.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 0x7ff954155630 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff944000c00 tx=0x7ff9440046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 shutdown_connections 2023-08-25T09:40:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff94006a050 0x7ff94006c510 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff95415e830 0x7ff95406ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff95415c6d0 0x7ff95414ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 --2- 172.21.15.70:0/4251264960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff95406b2b0 0x7ff954155630 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:45.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 >> 172.21.15.70:0/4251264960 conn(0x7ff954069850 msgr2=0x7ff9540ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:45.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 shutdown_connections 2023-08-25T09:40:45.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:45.904+0000 7ff959c46640 1 -- 172.21.15.70:0/4251264960 wait complete. 2023-08-25T09:40:45.925 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json osdmap barrier 39 2023-08-25T09:40:46.167 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-08-25T09:40:46.167 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi070:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-08-25T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi070:> os.fchmod(fd, 0O666) 2023-08-25T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(fd) 2023-08-25T09:40:46.169 DEBUG:teuthology.orchestra.run.smithi070:> os.close(fd) 2023-08-25T09:40:46.169 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:46.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:46.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:40:46.376 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:40:46.377 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:40:46.377 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:46.377 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:40:46.378 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.378 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:40:46.379 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:40:46.379 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:40:46.379 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:40:46.380 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.380 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:40:46.381 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:40:46.382 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:40:46.382 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:40:46.383 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.384 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:40:46.384 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:40:46.385 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:40:46.386 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:40:46.386 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:40:46.387 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:40:46.388 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:40:46.389 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:40:46.389 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.390 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:40:46.391 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:46.622 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:46.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:46.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95809.asok status 2023-08-25T09:40:46.863 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:46.863 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:40:46.863 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:40:46.863 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:40:46.863 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "95809", 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 2, 2023-08-25T09:40:46.864 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5553, 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 5553 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:46.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:46.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.5553 192.168.0.1:0/2574353996", 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2574353996", 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 3, 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 16, 2023-08-25T09:40:46.867 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 39, 2023-08-25T09:40:46.868 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 39, 2023-08-25T09:40:46.868 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:40:46.868 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:40:46.868 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:46.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:40:46.878 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:40:46.878 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:40:46.878 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:40:46.879 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:40:46.879 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.879 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:40:46.880 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:40:46.880 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:40:46.880 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:40:46.881 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.881 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:40:46.881 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:40:46.881 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:40:46.882 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:40:46.882 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.882 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:40:46.883 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:40:46.883 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:40:46.883 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:40:46.883 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:40:46.884 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:40:46.884 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:40:46.884 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:40:46.885 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:40:46.885 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:40:46.886 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:40:47.070 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:47.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:40:47.088 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95809.asok 2023-08-25T09:40:47.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:47.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95809.asok status 2023-08-25T09:40:47.332 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:40:47.332 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "95809", 2023-08-25T09:40:47.333 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:40:47.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:47.334 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 2, 2023-08-25T09:40:47.334 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:40:47.334 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5553, 2023-08-25T09:40:47.334 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:40:47.334 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 5553 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:47.335 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:47.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:40:47.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:47.336 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:40:47.336 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:40:47.336 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:40:47.336 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2574353996 2023-08-25T09:40:47.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:40:47.337 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.5553 192.168.0.1:0/2574353996", 2023-08-25T09:40:47.337 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2574353996", 2023-08-25T09:40:47.337 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 3, 2023-08-25T09:40:47.338 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 16, 2023-08-25T09:40:47.338 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 39, 2023-08-25T09:40:47.338 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 39, 2023-08-25T09:40:47.339 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:40:47.339 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:40:47.339 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:40:47.347 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:40:47.347 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:40:47.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:40:47.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:40:47.587 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:40:47.588 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:40:47.588 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:47.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:47.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.870+0000 7fae2af76640 1 Processor -- start 2023-08-25T09:40:47.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.871+0000 7fae2af76640 1 -- start start 2023-08-25T09:40:47.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.872+0000 7fae2af76640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415ec10 0x7fae2415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:47.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.872+0000 7fae2af76640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae2415c6d0 0x7fae240b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:47.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.872+0000 7fae2af76640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240b1b60 con 0x7fae2415e830 2023-08-25T09:40:47.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.872+0000 7fae2af76640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2406bd80 con 0x7fae2415c6d0 2023-08-25T09:40:47.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.872+0000 7fae2af76640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2406b7c0 con 0x7fae2415ec10 2023-08-25T09:40:47.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae28d12640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fae2415e830 0x7fae2406ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40554/0 (socket says 172.21.15.70:40554) 2023-08-25T09:40:47.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae28d12640 1 -- 172.21.15.70:0/1377598224 learned_addr learned my addr 172.21.15.70:0/1377598224 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae29513640 1 --2- 172.21.15.70:0/1377598224 >> v2:172.21.15.70:3300/0 conn(0x7fae2415c6d0 0x7fae240b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae237fe640 1 -- 172.21.15.70:0/1377598224 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188548808 0 0) 0x7fae240b1b60 con 0x7fae2415e830 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae237fe640 1 -- 172.21.15.70:0/1377598224 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae0c003520 con 0x7fae2415e830 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae29513640 1 -- 172.21.15.70:0/1377598224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415ec10 msgr2=0x7fae2415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae29513640 1 --2- 172.21.15.70:0/1377598224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415ec10 0x7fae2415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae29513640 1 -- 172.21.15.70:0/1377598224 >> v1:172.21.15.70:6789/0 conn(0x7fae2415e830 legacy=0x7fae2406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:47.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.873+0000 7fae29513640 1 -- 172.21.15.70:0/1377598224 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2406baa0 con 0x7fae2415c6d0 2023-08-25T09:40:47.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.874+0000 7fae29513640 1 --2- 172.21.15.70:0/1377598224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415c6d0 0x7fae240b1ff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009d90 tx=0x7fae1c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f7d17509d2b930 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:47.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.874+0000 7fae237fe640 1 -- 172.21.15.70:0/1377598224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c02cbc0 con 0x7fae2415c6d0 2023-08-25T09:40:47.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae237fe640 1 -- 172.21.15.70:0/1377598224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c02cd20 con 0x7fae2415c6d0 2023-08-25T09:40:47.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae237fe640 1 -- 172.21.15.70:0/1377598224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c034640 con 0x7fae2415c6d0 2023-08-25T09:40:47.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 -- 172.21.15.70:0/1377598224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415c6d0 msgr2=0x7fae240b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:47.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 --2- 172.21.15.70:0/1377598224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415c6d0 0x7fae240b1ff0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009d90 tx=0x7fae1c02be10 comp rx=0 tx=0).stop 2023-08-25T09:40:47.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 -- 172.21.15.70:0/1377598224 shutdown_connections 2023-08-25T09:40:47.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 --2- 172.21.15.70:0/1377598224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415c6d0 0x7fae240b1ff0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:47.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 --2- 172.21.15.70:0/1377598224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415ec10 0x7fae2415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:47.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 -- 172.21.15.70:0/1377598224 >> 172.21.15.70:0/1377598224 conn(0x7fae24069850 msgr2=0x7fae24069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:47.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 -- 172.21.15.70:0/1377598224 shutdown_connections 2023-08-25T09:40:47.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.875+0000 7fae2af76640 1 -- 172.21.15.70:0/1377598224 wait complete. 2023-08-25T09:40:47.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.877+0000 7fae2af76640 1 Processor -- start 2023-08-25T09:40:47.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.877+0000 7fae2af76640 1 -- start start 2023-08-25T09:40:47.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae2af76640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415c6d0 0x7fae24146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:47.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae2af76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 0x7fae241491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:47.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae2af76640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae2415ec10 0x7fae24149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:47.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae2af76640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2414cfd0 con 0x7fae2415e830 2023-08-25T09:40:47.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae2af76640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae240b0050 con 0x7fae2415ec10 2023-08-25T09:40:47.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae2af76640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae24147820 con 0x7fae2415c6d0 2023-08-25T09:40:47.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.878+0000 7fae23fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 0x7fae241491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:47.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 0x7fae241491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54902/0 (socket says 172.21.15.70:54902) 2023-08-25T09:40:47.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 -- 172.21.15.70:0/2839727254 learned_addr learned my addr 172.21.15.70:0/2839727254 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:47.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae28d12640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415c6d0 0x7fae24146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:47.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae29513640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae2415ec10 0x7fae24149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:47.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 -- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415c6d0 msgr2=0x7fae24146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:47.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415c6d0 0x7fae24146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:47.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 -- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae2415ec10 msgr2=0x7fae24149700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:47.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae2415ec10 0x7fae24149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:47.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 -- 172.21.15.70:0/2839727254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c009a70 con 0x7fae2415e830 2023-08-25T09:40:47.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae28d12640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415c6d0 0x7fae24146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:47.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.879+0000 7fae23fff640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 0x7fae241491c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b4d0 tx=0x7fae1800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:47.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.880+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18010870 con 0x7fae2415e830 2023-08-25T09:40:47.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.880+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae180109d0 con 0x7fae2415e830 2023-08-25T09:40:47.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.880+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18010d80 con 0x7fae2415e830 2023-08-25T09:40:47.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.880+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24149d80 con 0x7fae2415e830 2023-08-25T09:40:47.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.880+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae241471d0 con 0x7fae2415e830 2023-08-25T09:40:47.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.881+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2404e420 con 0x7fae2415e830 2023-08-25T09:40:47.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.882+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae1800f3c0 con 0x7fae2415e830 2023-08-25T09:40:47.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.883+0000 7fae217fa640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae1006a080 0x7fae1006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:47.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.883+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fae18088530 con 0x7fae2415e830 2023-08-25T09:40:47.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.883+0000 7fae28d12640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae1006a080 0x7fae1006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:47.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.884+0000 7fae28d12640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae1006a080 0x7fae1006c540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fae14005880 tx=0x7fae14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:47.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:47.887+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae18053bd0 con 0x7fae2415e830 2023-08-25T09:40:48.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.076+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae24147960 con 0x7fae2415e830 2023-08-25T09:40:48.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.078+0000 7fae217fa640 1 -- 172.21.15.70:0/2839727254 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18401 (secure 0 0 0) 0x7fae180571f0 con 0x7fae2415e830 2023-08-25T09:40:48.080 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:48.081 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:44.272862+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:48.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae1006a080 msgr2=0x7fae1006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae1006a080 0x7fae1006c540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fae14005880 tx=0x7fae14009040 comp rx=0 tx=0).stop 2023-08-25T09:40:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 msgr2=0x7fae241491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 0x7fae241491c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fae1800b4d0 tx=0x7fae1800b990 comp rx=0 tx=0).stop 2023-08-25T09:40:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 shutdown_connections 2023-08-25T09:40:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae1006a080 0x7fae1006c540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae2415ec10 0x7fae24149700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae2415e830 0x7fae241491c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 --2- 172.21.15.70:0/2839727254 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae2415c6d0 0x7fae24146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 >> 172.21.15.70:0/2839727254 conn(0x7fae24069850 msgr2=0x7fae240ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 shutdown_connections 2023-08-25T09:40:48.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.081+0000 7fae2af76640 1 -- 172.21.15.70:0/2839727254 wait complete. 2023-08-25T09:40:48.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:40:25.098622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:40:25.912835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:48.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:48.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.307+0000 7f183675d640 1 Processor -- start 2023-08-25T09:40:48.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.307+0000 7f183675d640 1 -- start start 2023-08-25T09:40:48.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f183675d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f183015e240 0x7f1830070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f183675d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18300cd660 0x7f18300cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f183675d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f183006f8e0 con 0x7f1830159bf0 2023-08-25T09:40:48.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f183675d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18300ce030 con 0x7f18300cd660 2023-08-25T09:40:48.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f183675d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183006fbc0 con 0x7f183015e240 2023-08-25T09:40:48.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f182ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1830159bf0 0x7f183006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40560/0 (socket says 172.21.15.70:40560) 2023-08-25T09:40:48.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.309+0000 7f182ffff640 1 -- 172.21.15.70:0/2237821996 learned_addr learned my addr 172.21.15.70:0/2237821996 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:48.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.310+0000 7f1834cfa640 1 --2- 172.21.15.70:0/2237821996 >> v2:172.21.15.70:3300/0 conn(0x7f18300cd660 0x7f18300cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.310+0000 7f1834cfa640 1 -- 172.21.15.70:0/2237821996 >> v2:172.21.15.155:3300/0 conn(0x7f183015e240 msgr2=0x7f1830070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:48.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.310+0000 7f1834cfa640 1 --2- 172.21.15.70:0/2237821996 >> v2:172.21.15.155:3300/0 conn(0x7f183015e240 0x7f1830070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.310+0000 7f1834cfa640 1 -- 172.21.15.70:0/2237821996 >> v1:172.21.15.70:6789/0 conn(0x7f1830159bf0 legacy=0x7f183006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.310+0000 7f1834cfa640 1 -- 172.21.15.70:0/2237821996 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f183006fea0 con 0x7f18300cd660 2023-08-25T09:40:48.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.310+0000 7f182f7fe640 1 -- 172.21.15.70:0/2237821996 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048952516 0 0) 0x7f183006f8e0 con 0x7f1830159bf0 2023-08-25T09:40:48.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.311+0000 7f1834cfa640 1 --2- 172.21.15.70:0/2237821996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18300cd660 0x7f18300cdaf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f183006af90 tx=0x7f182802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd60323cf78792ab server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:48.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.311+0000 7f182f7fe640 1 -- 172.21.15.70:0/2237821996 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182802cc80 con 0x7f18300cd660 2023-08-25T09:40:48.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f182f7fe640 1 -- 172.21.15.70:0/2237821996 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182802cde0 con 0x7f18300cd660 2023-08-25T09:40:48.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f182f7fe640 1 -- 172.21.15.70:0/2237821996 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1828034660 con 0x7f18300cd660 2023-08-25T09:40:48.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f183675d640 1 -- 172.21.15.70:0/2237821996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18300cd660 msgr2=0x7f18300cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f183675d640 1 --2- 172.21.15.70:0/2237821996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18300cd660 0x7f18300cdaf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f183006af90 tx=0x7f182802bed0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f183675d640 1 -- 172.21.15.70:0/2237821996 shutdown_connections 2023-08-25T09:40:48.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f183675d640 1 --2- 172.21.15.70:0/2237821996 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18300cd660 0x7f18300cdaf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f183675d640 1 --2- 172.21.15.70:0/2237821996 >> v2:172.21.15.155:3300/0 conn(0x7f183015e240 0x7f1830070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.312+0000 7f183675d640 1 -- 172.21.15.70:0/2237821996 >> 172.21.15.70:0/2237821996 conn(0x7f18300cb0d0 msgr2=0x7f18300cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:48.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.313+0000 7f183675d640 1 -- 172.21.15.70:0/2237821996 shutdown_connections 2023-08-25T09:40:48.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.313+0000 7f183675d640 1 -- 172.21.15.70:0/2237821996 wait complete. 2023-08-25T09:40:48.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.314+0000 7f183675d640 1 Processor -- start 2023-08-25T09:40:48.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.314+0000 7f183675d640 1 -- start start 2023-08-25T09:40:48.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.315+0000 7f183675d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18300cd660 0x7f18300c3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.315+0000 7f183675d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 0x7f18301f1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.315+0000 7f183675d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f183015e240 0x7f18300ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.315+0000 7f183675d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300c84f0 con 0x7f1830159bf0 2023-08-25T09:40:48.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.315+0000 7f183675d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300c78e0 con 0x7f18300cd660 2023-08-25T09:40:48.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.315+0000 7f183675d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18300c7bc0 con 0x7f183015e240 2023-08-25T09:40:48.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 0x7f18301f1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 0x7f18301f1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54932/0 (socket says 172.21.15.70:54932) 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 -- 172.21.15.70:0/28937572 learned_addr learned my addr 172.21.15.70:0/28937572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f182ffff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18300cd660 0x7f18300c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 -- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f183015e240 msgr2=0x7f18300ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1834cfa640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f183015e240 0x7f18300ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f183015e240 0x7f18300ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 -- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18300cd660 msgr2=0x7f18300c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18300cd660 0x7f18300c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f1827fff640 1 -- 172.21.15.70:0/28937572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009a70 con 0x7f1830159bf0 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.316+0000 7f182ffff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18300cd660 0x7f18300c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.317+0000 7f1827fff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 0x7f18301f1d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f181800b410 tx=0x7f181800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.317+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818011830 con 0x7f1830159bf0 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.317+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18180119b0 con 0x7f1830159bf0 2023-08-25T09:40:48.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.317+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818011d60 con 0x7f1830159bf0 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.317+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18300c7ea0 con 0x7f1830159bf0 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.317+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18300c75d0 con 0x7f1830159bf0 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.318+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1830066fb0 con 0x7f1830159bf0 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.319+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f181801a400 con 0x7f1830159bf0 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.320+0000 7f182d7fa640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181006a050 0x7f181006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.320+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f18180887d0 con 0x7f1830159bf0 2023-08-25T09:40:48.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.320+0000 7f182ffff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181006a050 0x7f181006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.321+0000 7f182ffff640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181006a050 0x7f181006c510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1820000f60 tx=0x7f1820009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:48.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.324+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1818053de0 con 0x7f1830159bf0 2023-08-25T09:40:48.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.516+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1830159010 con 0x7f1830159bf0 2023-08-25T09:40:48.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.517+0000 7f182d7fa640 1 -- 172.21.15.70:0/28937572 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1818057400 con 0x7f1830159bf0 2023-08-25T09:40:48.518 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:48.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181006a050 msgr2=0x7f181006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181006a050 0x7f181006c510 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1820000f60 tx=0x7f1820009040 comp rx=0 tx=0).stop 2023-08-25T09:40:48.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 msgr2=0x7f18301f1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 0x7f18301f1d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f181800b410 tx=0x7f181800b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 shutdown_connections 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f181006a050 0x7f181006c510 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f183015e240 0x7f18300ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1830159bf0 0x7f18301f1d60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 --2- 172.21.15.70:0/28937572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18300cd660 0x7f18300c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.519+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 >> 172.21.15.70:0/28937572 conn(0x7f18300cb0d0 msgr2=0x7f18300ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.520+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 shutdown_connections 2023-08-25T09:40:48.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.520+0000 7f183675d640 1 -- 172.21.15.70:0/28937572 wait complete. 2023-08-25T09:40:48.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:48.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.755+0000 7fa794a5d640 1 Processor -- start 2023-08-25T09:40:48.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.755+0000 7fa794a5d640 1 -- start start 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa794a5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7900b0320 0x7fa7900c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa794a5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7900adbb0 con 0x7fa7900b1ce0 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa794a5d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7900c90a0 con 0x7fa79015a920 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa794a5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7900c9380 con 0x7fa7900b0320 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa78ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7900b0320 0x7fa7900c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa78f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa7900b1ce0 0x7fa7900ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40568/0 (socket says 172.21.15.70:40568) 2023-08-25T09:40:48.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.757+0000 7fa78f59e640 1 -- 172.21.15.70:0/1664332370 learned_addr learned my addr 172.21.15.70:0/1664332370 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:48.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.758+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1664332370 >> v1:172.21.15.155:6789/0 conn(0x7fa79015a920 legacy=0x7fa7900b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.758+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1664332370 >> v1:172.21.15.70:6789/0 conn(0x7fa7900b1ce0 legacy=0x7fa7900ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.758+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1664332370 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7900c9660 con 0x7fa7900b0320 2023-08-25T09:40:48.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.758+0000 7fa78dd9b640 1 -- 172.21.15.70:0/1664332370 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992502907 0 0) 0x7fa7900adbb0 con 0x7fa7900b1ce0 2023-08-25T09:40:48.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.759+0000 7fa78ed9d640 1 --2- 172.21.15.70:0/1664332370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900c9e60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa780009de0 tx=0x7fa78002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e5eb8f1e02a68f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:48.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.759+0000 7fa78dd9b640 1 -- 172.21.15.70:0/1664332370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78002cd30 con 0x7fa7900b0320 2023-08-25T09:40:48.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.759+0000 7fa78dd9b640 1 -- 172.21.15.70:0/1664332370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78002ce90 con 0x7fa7900b0320 2023-08-25T09:40:48.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.759+0000 7fa78dd9b640 1 -- 172.21.15.70:0/1664332370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780034660 con 0x7fa7900b0320 2023-08-25T09:40:48.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 -- 172.21.15.70:0/1664332370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 msgr2=0x7fa7900c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1664332370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900c9e60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa780009de0 tx=0x7fa78002bf80 comp rx=0 tx=0).stop 2023-08-25T09:40:48.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 -- 172.21.15.70:0/1664332370 shutdown_connections 2023-08-25T09:40:48.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1664332370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900c9e60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 -- 172.21.15.70:0/1664332370 >> 172.21.15.70:0/1664332370 conn(0x7fa7900ce830 msgr2=0x7fa7900cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:48.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 -- 172.21.15.70:0/1664332370 shutdown_connections 2023-08-25T09:40:48.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.760+0000 7fa794a5d640 1 -- 172.21.15.70:0/1664332370 wait complete. 2023-08-25T09:40:48.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.763+0000 7fa794a5d640 1 Processor -- start 2023-08-25T09:40:48.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.763+0000 7fa794a5d640 1 -- start start 2023-08-25T09:40:48.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa794a5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa794a5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7900b1ce0 0x7fa7901f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa794a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79015a920 0x7fa7900cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa794a5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa790154910 con 0x7fa7900b0320 2023-08-25T09:40:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa794a5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900cd760 con 0x7fa7900b1ce0 2023-08-25T09:40:48.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa794a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900cda40 con 0x7fa79015a920 2023-08-25T09:40:48.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa78ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa78ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54944/0 (socket says 172.21.15.70:54944) 2023-08-25T09:40:48.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.764+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1971106382 learned_addr learned my addr 172.21.15.70:0/1971106382 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:48.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1971106382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79015a920 msgr2=0x7fa7900cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa78ed9d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79015a920 0x7fa7900cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1971106382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7900b1ce0 msgr2=0x7fa7901f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa78ed9d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7900b1ce0 0x7fa7901f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa78ed9d640 1 -- 172.21.15.70:0/1971106382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780009a70 con 0x7fa7900b0320 2023-08-25T09:40:48.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa78ed9d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900cd080 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa780009de0 tx=0x7fa780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:48.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78003f420 con 0x7fa7900b0320 2023-08-25T09:40:48.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78003f580 con 0x7fa7900b0320 2023-08-25T09:40:48.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.765+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7900cdd20 con 0x7fa7900b0320 2023-08-25T09:40:48.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.766+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78003f910 con 0x7fa7900b0320 2023-08-25T09:40:48.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.766+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7901f55c0 con 0x7fa7900b0320 2023-08-25T09:40:48.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.767+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7900cbf30 con 0x7fa7900b0320 2023-08-25T09:40:48.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.768+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa78003fa70 con 0x7fa7900b0320 2023-08-25T09:40:48.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.769+0000 7fa7777fe640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa77c06a050 0x7fa77c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:48.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.769+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa7800aafe0 con 0x7fa7900b0320 2023-08-25T09:40:48.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.769+0000 7fa78e59c640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa77c06a050 0x7fa77c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:48.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.770+0000 7fa78e59c640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa77c06a050 0x7fa77c06c510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa784000f60 tx=0x7fa7840079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:48.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.773+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa7900cbf30 con 0x7fa7900b0320 2023-08-25T09:40:48.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.960+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7900cc310 con 0x7fa7900b0320 2023-08-25T09:40:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.961+0000 7fa7777fe640 1 -- 172.21.15.70:0/1971106382 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7900cc310 con 0x7fa7900b0320 2023-08-25T09:40:48.964 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:48.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.963+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa77c06a050 msgr2=0x7fa77c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.963+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa77c06a050 0x7fa77c06c510 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa784000f60 tx=0x7fa7840079b0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.963+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 msgr2=0x7fa7900cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.963+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900cd080 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa780009de0 tx=0x7fa780004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 shutdown_connections 2023-08-25T09:40:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa77c06a050 0x7fa77c06c510 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa79015a920 0x7fa7900cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7900b1ce0 0x7fa7901f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 --2- 172.21.15.70:0/1971106382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7900b0320 0x7fa7900cd080 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 >> 172.21.15.70:0/1971106382 conn(0x7fa7900ce830 msgr2=0x7fa7900cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 shutdown_connections 2023-08-25T09:40:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:48.964+0000 7fa794a5d640 1 -- 172.21.15.70:0/1971106382 wait complete. 2023-08-25T09:40:48.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:49.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.203+0000 7f0611225640 1 Processor -- start 2023-08-25T09:40:49.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.203+0000 7f0611225640 1 -- start start 2023-08-25T09:40:49.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.204+0000 7f0611225640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f060c15a920 0x7f060c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.204+0000 7f0611225640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b1ce0 0x7f060c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.204+0000 7f0611225640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f060c0ad8f0 con 0x7f060c0b0320 2023-08-25T09:40:49.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.204+0000 7f0611225640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f060c0adbb0 con 0x7f060c15a920 2023-08-25T09:40:49.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.204+0000 7f0611225640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0c90a0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f060c15a920 0x7f060c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f060c15a920 0x7f060c0c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48214/0 (socket says 172.21.15.70:48214) 2023-08-25T09:40:49.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 -- 172.21.15.70:0/1570684688 learned_addr learned my addr 172.21.15.70:0/1570684688 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:49.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060bfff640 1 --1- 172.21.15.70:0/1570684688 >> v1:172.21.15.70:6789/0 conn(0x7f060c0b0320 0x7f060c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40930/0 (socket says 172.21.15.70:40930) 2023-08-25T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f0610a24640 1 --2- 172.21.15.70:0/1570684688 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b1ce0 0x7f060c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 -- 172.21.15.70:0/1570684688 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b1ce0 msgr2=0x7f060c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 --2- 172.21.15.70:0/1570684688 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b1ce0 0x7f060c0ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 -- 172.21.15.70:0/1570684688 >> v1:172.21.15.70:6789/0 conn(0x7f060c0b0320 legacy=0x7f060c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060b7fe640 1 -- 172.21.15.70:0/1570684688 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c0c9380 con 0x7f060c15a920 2023-08-25T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.205+0000 7f060affd640 1 -- 172.21.15.70:0/1570684688 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550163784 0 0) 0x7f060c0ad8f0 con 0x7f060c0b0320 2023-08-25T09:40:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.206+0000 7f060b7fe640 1 --2- 172.21.15.70:0/1570684688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c15a920 0x7f060c0c9e60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0600009870 tx=0x7f060002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec06138c55c6505a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f060affd640 1 -- 172.21.15.70:0/1570684688 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600030840 con 0x7f060c15a920 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f060affd640 1 -- 172.21.15.70:0/1570684688 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06000309a0 con 0x7f060c15a920 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f060affd640 1 -- 172.21.15.70:0/1570684688 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600030d30 con 0x7f060c15a920 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 -- 172.21.15.70:0/1570684688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c15a920 msgr2=0x7f060c0c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 --2- 172.21.15.70:0/1570684688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c15a920 0x7f060c0c9e60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0600009870 tx=0x7f060002b980 comp rx=0 tx=0).stop 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 -- 172.21.15.70:0/1570684688 shutdown_connections 2023-08-25T09:40:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 --2- 172.21.15.70:0/1570684688 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c15a920 0x7f060c0c9e60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 --2- 172.21.15.70:0/1570684688 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b1ce0 0x7f060c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 -- 172.21.15.70:0/1570684688 >> 172.21.15.70:0/1570684688 conn(0x7f060c0ce830 msgr2=0x7f060c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.207+0000 7f0611225640 1 -- 172.21.15.70:0/1570684688 shutdown_connections 2023-08-25T09:40:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.208+0000 7f0611225640 1 -- 172.21.15.70:0/1570684688 wait complete. 2023-08-25T09:40:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.209+0000 7f0611225640 1 Processor -- start 2023-08-25T09:40:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.209+0000 7f0611225640 1 -- start start 2023-08-25T09:40:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.210+0000 7f0611225640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b0320 0x7f060c1efbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.210+0000 7f0611225640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 0x7f060c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.210+0000 7f0611225640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c15a920 0x7f060c0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.210+0000 7f0611225640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c15c050 con 0x7f060c0b1ce0 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.210+0000 7f0611225640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0c3b30 con 0x7f060c15a920 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.210+0000 7f0611225640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060c0c3e10 con 0x7f060c0b0320 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 0x7f060c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 0x7f060c1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48220/0 (socket says 172.21.15.70:48220) 2023-08-25T09:40:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 -- 172.21.15.70:0/3170178231 learned_addr learned my addr 172.21.15.70:0/3170178231 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:49.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 -- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b0320 msgr2=0x7f060c1efbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060bfff640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b0320 0x7f060c1efbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f0610a24640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c15a920 0x7f060c0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b0320 0x7f060c1efbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 -- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c15a920 msgr2=0x7f060c0c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c15a920 0x7f060c0c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 -- 172.21.15.70:0/3170178231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06000094d0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.211+0000 7f060b7fe640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 0x7f060c1585f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0600000c00 tx=0x7f0600004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.212+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600039a90 con 0x7f060c0b1ce0 2023-08-25T09:40:49.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.212+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600039bf0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.212+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600043410 con 0x7f060c0b1ce0 2023-08-25T09:40:49.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.212+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060c15c690 con 0x7f060c0b1ce0 2023-08-25T09:40:49.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.212+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060c158b30 con 0x7f060c0b1ce0 2023-08-25T09:40:49.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.213+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060c15cbb0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.214+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f060002f390 con 0x7f060c0b1ce0 2023-08-25T09:40:49.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.215+0000 7f0608ff9640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05f806a050 0x7f05f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.215+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f06000aa880 con 0x7f060c0b1ce0 2023-08-25T09:40:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.215+0000 7f060bfff640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05f806a050 0x7f05f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.216+0000 7f060bfff640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05f806a050 0x7f05f806c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05fc001680 tx=0x7f05fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:49.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.219+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f060c15cbb0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.412+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f060c15cbb0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.412+0000 7f0608ff9640 1 -- 172.21.15.70:0/3170178231 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f060c15cbb0 con 0x7f060c0b1ce0 2023-08-25T09:40:49.414 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05f806a050 msgr2=0x7f05f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05f806a050 0x7f05f806c510 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f05fc001680 tx=0x7f05fc009040 comp rx=0 tx=0).stop 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 msgr2=0x7f060c1585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 0x7f060c1585f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0600000c00 tx=0x7f0600004730 comp rx=0 tx=0).stop 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 shutdown_connections 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05f806a050 0x7f05f806c510 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f060c15a920 0x7f060c0c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f060c0b1ce0 0x7f060c1585f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 --2- 172.21.15.70:0/3170178231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f060c0b0320 0x7f060c1efbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 >> 172.21.15.70:0/3170178231 conn(0x7f060c0ce830 msgr2=0x7f060c0cd590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:49.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 shutdown_connections 2023-08-25T09:40:49.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.415+0000 7f0611225640 1 -- 172.21.15.70:0/3170178231 wait complete. 2023-08-25T09:40:49.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:49.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.651+0000 7f1c252e0640 1 Processor -- start 2023-08-25T09:40:49.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.651+0000 7f1c252e0640 1 -- start start 2023-08-25T09:40:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.652+0000 7f1c252e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 0x7f1c200c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.652+0000 7f1c252e0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c200b2170 con 0x7f1c200b1ce0 2023-08-25T09:40:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.652+0000 7f1c252e0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c200adbb0 con 0x7f1c200b0320 2023-08-25T09:40:49.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.652+0000 7f1c252e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c90a0 con 0x7f1c2015a920 2023-08-25T09:40:49.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c24adf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1c200b1ce0 0x7f1c200ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40934/0 (socket says 172.21.15.70:40934) 2023-08-25T09:40:49.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c24adf640 1 -- 172.21.15.70:0/4205792772 learned_addr learned my addr 172.21.15.70:0/4205792772 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:49.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c177fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 0x7f1c200c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082563537 0 0) 0x7f1c200b2170 con 0x7f1c200b1ce0 2023-08-25T09:40:49.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c0c003520 con 0x7f1c200b1ce0 2023-08-25T09:40:49.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3074263987 0 0) 0x7f1c0c003520 con 0x7f1c200b1ce0 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 msgr2=0x7f1c200c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.653+0000 7f1c1f7fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 0x7f1c200c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 >> v1:172.21.15.155:6789/0 conn(0x7f1c200b0320 legacy=0x7f1c200b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200c9380 con 0x7f1c200b1ce0 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1c180019f0 con 0x7f1c200b1ce0 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 >> v1:172.21.15.70:6789/0 conn(0x7f1c200b1ce0 legacy=0x7f1c200ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 0x7f1c0c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c0c003f20 0x7f1c0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c004d00 0x7f1c0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c90a0 con 0x7f1c0c003f20 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c00ba60 con 0x7f1c0c004d00 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c00bc20 con 0x7f1c2015a920 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c177fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c0c003f20 0x7f1c0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c177fe640 1 -- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 msgr2=0x7f1c0c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:49.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c177fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c2015a920 0x7f1c0c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c177fe640 1 -- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c004d00 msgr2=0x7f1c0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c177fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c004d00 0x7f1c0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.654+0000 7f1c177fe640 1 -- 172.21.15.70:0/4205792772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c003520 con 0x7f1c0c003f20 2023-08-25T09:40:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c177fe640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c0c003f20 0x7f1c0c0047c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009aa0 tx=0x7f1c0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540d24e285adedc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0802c710 con 0x7f1c0c003f20 2023-08-25T09:40:49.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0802c870 con 0x7f1c0c003f20 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c1f7fe640 1 -- 172.21.15.70:0/4205792772 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0802cc20 con 0x7f1c0c003f20 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c252e0640 1 -- 172.21.15.70:0/4205792772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c0c003f20 msgr2=0x7f1c0c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c252e0640 1 --2- 172.21.15.70:0/4205792772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c0c003f20 0x7f1c0c0047c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009aa0 tx=0x7f1c0c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.656+0000 7f1c24adf640 1 -- 172.21.15.70:0/4205792772 reap_dead start 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.657+0000 7f1c252e0640 1 -- 172.21.15.70:0/4205792772 shutdown_connections 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.657+0000 7f1c252e0640 1 -- 172.21.15.70:0/4205792772 >> 172.21.15.70:0/4205792772 conn(0x7f1c200ce830 msgr2=0x7f1c200cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.657+0000 7f1c252e0640 1 -- 172.21.15.70:0/4205792772 shutdown_connections 2023-08-25T09:40:49.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.657+0000 7f1c252e0640 1 -- 172.21.15.70:0/4205792772 wait complete. 2023-08-25T09:40:49.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.658+0000 7f1c252e0640 1 Processor -- start 2023-08-25T09:40:49.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 -- start start 2023-08-25T09:40:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c003f20 0x7f1c20157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 0x7f1c200c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c200cbfc0 0x7f1c200cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200c3f80 con 0x7f1c200c35c0 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c201576d0 con 0x7f1c0c003f20 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.659+0000 7f1c252e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200cccf0 con 0x7f1c200cbfc0 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 0x7f1c200c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 0x7f1c200c3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48240/0 (socket says 172.21.15.70:48240) 2023-08-25T09:40:49.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 -- 172.21.15.70:0/1037055863 learned_addr learned my addr 172.21.15.70:0/1037055863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 -- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c200cbfc0 msgr2=0x7f1c200cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c24adf640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c200cbfc0 0x7f1c200cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c1ffff640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c003f20 0x7f1c20157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c200cbfc0 0x7f1c200cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 -- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c003f20 msgr2=0x7f1c20157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c003f20 0x7f1c20157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c177fe640 1 -- 172.21.15.70:0/1037055863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200ccfd0 con 0x7f1c200c35c0 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.660+0000 7f1c1ffff640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c003f20 0x7f1c20157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.661+0000 7f1c177fe640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 0x7f1c200c3a40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802c2b0 tx=0x7f1c08004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.661+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08034c80 con 0x7f1c200c35c0 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.661+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08034de0 con 0x7f1c200c35c0 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.661+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c200cd2b0 con 0x7f1c200c35c0 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.661+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0803e630 con 0x7f1c200c35c0 2023-08-25T09:40:49.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.661+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c200cd870 con 0x7f1c200c35c0 2023-08-25T09:40:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.662+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c20061460 con 0x7f1c200c35c0 2023-08-25T09:40:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.663+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1c0803b050 con 0x7f1c200c35c0 2023-08-25T09:40:49.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.664+0000 7f1c1d7fa640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c0006a080 0x7f1c0006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.664+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1c080aa560 con 0x7f1c200c35c0 2023-08-25T09:40:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.664+0000 7f1c1ffff640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c0006a080 0x7f1c0006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:49.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.665+0000 7f1c1ffff640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c0006a080 0x7f1c0006c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c10001760 tx=0x7f1c10009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.669+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1c08075d40 con 0x7f1c200c35c0 2023-08-25T09:40:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.856+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c200ca720 con 0x7f1c200c35c0 2023-08-25T09:40:49.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.857+0000 7f1c1d7fa640 1 -- 172.21.15.70:0/1037055863 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c08079360 con 0x7f1c200c35c0 2023-08-25T09:40:49.859 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c0006a080 msgr2=0x7f1c0006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c0006a080 0x7f1c0006c540 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1c10001760 tx=0x7f1c10009040 comp rx=0 tx=0).stop 2023-08-25T09:40:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 msgr2=0x7f1c200c3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 0x7f1c200c3a40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1c0802c2b0 tx=0x7f1c08004730 comp rx=0 tx=0).stop 2023-08-25T09:40:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 shutdown_connections 2023-08-25T09:40:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c0006a080 0x7f1c0006c540 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1c200cbfc0 0x7f1c200cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1c200c35c0 0x7f1c200c3a40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 --2- 172.21.15.70:0/1037055863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1c0c003f20 0x7f1c20157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 >> 172.21.15.70:0/1037055863 conn(0x7f1c200ce830 msgr2=0x7f1c20158a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 shutdown_connections 2023-08-25T09:40:49.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:49.860+0000 7f1c252e0640 1 -- 172.21.15.70:0/1037055863 wait complete. 2023-08-25T09:40:49.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:40:50.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.091+0000 7f10bf2d9640 1 Processor -- start 2023-08-25T09:40:50.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.092+0000 7f10bf2d9640 1 -- start start 2023-08-25T09:40:50.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.093+0000 7f10bf2d9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10b80aee40 0x7f10b8072d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.093+0000 7f10bf2d9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b8073260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.093+0000 7f10bf2d9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b8072820 con 0x7f10b815ccb0 2023-08-25T09:40:50.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.093+0000 7f10bf2d9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b806bdc0 con 0x7f10b80aee40 2023-08-25T09:40:50.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.093+0000 7f10bf2d9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b806c0a0 con 0x7f10b806b8c0 2023-08-25T09:40:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b8073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b8073260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:43720/0 (socket says 172.21.15.70:43720) 2023-08-25T09:40:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 -- 172.21.15.70:0/764863034 learned_addr learned my addr 172.21.15.70:0/764863034 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bc874640 1 --2- 172.21.15.70:0/764863034 >> v2:172.21.15.155:3300/0 conn(0x7f10b80aee40 0x7f10b8072d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 -- 172.21.15.70:0/764863034 >> v2:172.21.15.155:3300/0 conn(0x7f10b80aee40 msgr2=0x7f10b8072d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 --2- 172.21.15.70:0/764863034 >> v2:172.21.15.155:3300/0 conn(0x7f10b80aee40 0x7f10b8072d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 -- 172.21.15.70:0/764863034 >> v1:172.21.15.155:6789/0 conn(0x7f10b815ccb0 legacy=0x7f10b815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10bd876640 1 -- 172.21.15.70:0/764863034 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b806c380 con 0x7f10b806b8c0 2023-08-25T09:40:50.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.094+0000 7f10b7fff640 1 -- 172.21.15.70:0/764863034 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945753957 0 0) 0x7f10b8072820 con 0x7f10b815ccb0 2023-08-25T09:40:50.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.096+0000 7f10bd876640 1 --2- 172.21.15.70:0/764863034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b8073260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009e50 tx=0x7f10b002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a25dc2aae9c6460 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:50.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.096+0000 7f10b7fff640 1 -- 172.21.15.70:0/764863034 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b002cb00 con 0x7f10b806b8c0 2023-08-25T09:40:50.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10b7fff640 1 -- 172.21.15.70:0/764863034 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b002cc60 con 0x7f10b806b8c0 2023-08-25T09:40:50.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10b7fff640 1 -- 172.21.15.70:0/764863034 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b00343f0 con 0x7f10b806b8c0 2023-08-25T09:40:50.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 -- 172.21.15.70:0/764863034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 msgr2=0x7f10b8073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/764863034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b8073260 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009e50 tx=0x7f10b002bf60 comp rx=0 tx=0).stop 2023-08-25T09:40:50.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 -- 172.21.15.70:0/764863034 shutdown_connections 2023-08-25T09:40:50.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/764863034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b8073260 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/764863034 >> v2:172.21.15.155:3300/0 conn(0x7f10b80aee40 0x7f10b8072d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 -- 172.21.15.70:0/764863034 >> 172.21.15.70:0/764863034 conn(0x7f10b80b12c0 msgr2=0x7f10b80b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:50.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 -- 172.21.15.70:0/764863034 shutdown_connections 2023-08-25T09:40:50.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.097+0000 7f10bf2d9640 1 -- 172.21.15.70:0/764863034 wait complete. 2023-08-25T09:40:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.099+0000 7f10bf2d9640 1 Processor -- start 2023-08-25T09:40:50.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.099+0000 7f10bf2d9640 1 -- start start 2023-08-25T09:40:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.100+0000 7f10bf2d9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b81588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.100+0000 7f10bf2d9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b815ccb0 0x7f10b80c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.100+0000 7f10bf2d9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 0x7f10b80add80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.100+0000 7f10bf2d9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80ae2c0 con 0x7f10b80ad8d0 2023-08-25T09:40:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.100+0000 7f10bf2d9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b80c9b70 con 0x7f10b815ccb0 2023-08-25T09:40:50.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.100+0000 7f10bf2d9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b8158fc0 con 0x7f10b806b8c0 2023-08-25T09:40:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 0x7f10b80add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 0x7f10b80add80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48246/0 (socket says 172.21.15.70:48246) 2023-08-25T09:40:50.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 -- 172.21.15.70:0/1994973899 learned_addr learned my addr 172.21.15.70:0/1994973899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:50.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd075640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b81588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bc874640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b815ccb0 0x7f10b80c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 -- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 msgr2=0x7f10b81588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b81588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 -- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b815ccb0 msgr2=0x7f10b80c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b815ccb0 0x7f10b80c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 -- 172.21.15.70:0/1994973899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0009b30 con 0x7f10b80ad8d0 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.101+0000 7f10bd876640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 0x7f10b80add80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f10b0036040 tx=0x7f10b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:50.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.102+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0009510 con 0x7f10b80ad8d0 2023-08-25T09:40:50.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.102+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b003e450 con 0x7f10b80ad8d0 2023-08-25T09:40:50.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.102+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b003e800 con 0x7f10b80ad8d0 2023-08-25T09:40:50.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.102+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b80c7210 con 0x7f10b80ad8d0 2023-08-25T09:40:50.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.102+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b80c77d0 con 0x7f10b80ad8d0 2023-08-25T09:40:50.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.103+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b804e420 con 0x7f10b80ad8d0 2023-08-25T09:40:50.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.104+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f10b003e960 con 0x7f10b80ad8d0 2023-08-25T09:40:50.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.105+0000 7f10b5ffb640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10a406a080 0x7f10a406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.105+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f10b00aafb0 con 0x7f10b80ad8d0 2023-08-25T09:40:50.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.105+0000 7f10bd075640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10a406a080 0x7f10a406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.106+0000 7f10bd075640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10a406a080 0x7f10a406c540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007fd0 tx=0x7f10a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:50.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.109+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10b00dd990 con 0x7f10b80ad8d0 2023-08-25T09:40:50.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.313+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10b8159100 con 0x7f10b80ad8d0 2023-08-25T09:40:50.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.314+0000 7f10b5ffb640 1 -- 172.21.15.70:0/1994973899 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f10b0076660 con 0x7f10b80ad8d0 2023-08-25T09:40:50.315 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:40:50.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10a406a080 msgr2=0x7f10a406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10a406a080 0x7f10a406c540 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f10a8007fd0 tx=0x7f10a8009040 comp rx=0 tx=0).stop 2023-08-25T09:40:50.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 msgr2=0x7f10b80add80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 0x7f10b80add80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f10b0036040 tx=0x7f10b0004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:50.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 shutdown_connections 2023-08-25T09:40:50.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f10a406a080 0x7f10a406c540 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10b80ad8d0 0x7f10b80add80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10b815ccb0 0x7f10b80c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 --2- 172.21.15.70:0/1994973899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10b806b8c0 0x7f10b81588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.316+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 >> 172.21.15.70:0/1994973899 conn(0x7f10b80b12c0 msgr2=0x7f10b806fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:50.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.317+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 shutdown_connections 2023-08-25T09:40:50.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.317+0000 7f10bf2d9640 1 -- 172.21.15.70:0/1994973899 wait complete. 2023-08-25T09:40:50.337 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:40:50.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:50.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:40:50.433 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[95809]: fuse finished with error 0 and tester_r 0 2023-08-25T09:40:50.438 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:40:50.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:50.439 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:40:50.636 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:40:50.637 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:50.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:50.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.917+0000 7f5b57a83640 1 Processor -- start 2023-08-25T09:40:50.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.918+0000 7f5b57a83640 1 -- start start 2023-08-25T09:40:50.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b57a83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b5006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b57a83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b5015ec10 0x7f5b500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b57a83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b500b1b60 con 0x7f5b5006b2b0 2023-08-25T09:40:50.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b57a83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5006bd80 con 0x7f5b5015ec10 2023-08-25T09:40:50.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b57a83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5006b7c0 con 0x7f5b5015e830 2023-08-25T09:40:50.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b56020640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b5015ec10 0x7f5b500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b56020640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b5015ec10 0x7f5b500b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48254/0 (socket says 172.21.15.70:48254) 2023-08-25T09:40:50.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.919+0000 7f5b56020640 1 -- 172.21.15.70:0/3487648469 learned_addr learned my addr 172.21.15.70:0/3487648469 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:50.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.920+0000 7f5b5581f640 1 --2- 172.21.15.70:0/3487648469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b5006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.920+0000 7f5b56020640 1 -- 172.21.15.70:0/3487648469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 msgr2=0x7f5b5006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:50.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.920+0000 7f5b56020640 1 --2- 172.21.15.70:0/3487648469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b5006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.920+0000 7f5b56020640 1 -- 172.21.15.70:0/3487648469 >> v1:172.21.15.155:6789/0 conn(0x7f5b5006b2b0 legacy=0x7f5b5015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:50.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.920+0000 7f5b56020640 1 -- 172.21.15.70:0/3487648469 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5006baa0 con 0x7f5b5015ec10 2023-08-25T09:40:50.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.921+0000 7f5b56020640 1 --2- 172.21.15.70:0/3487648469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b500b1ff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009e50 tx=0x7f5b4802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7ad3db9feea7bd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:50.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.921+0000 7f5b5481d640 1 -- 172.21.15.70:0/3487648469 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4802ca70 con 0x7f5b5015ec10 2023-08-25T09:40:50.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.921+0000 7f5b5481d640 1 -- 172.21.15.70:0/3487648469 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4802cbd0 con 0x7f5b5015ec10 2023-08-25T09:40:50.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.921+0000 7f5b5481d640 1 -- 172.21.15.70:0/3487648469 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b480343f0 con 0x7f5b5015ec10 2023-08-25T09:40:50.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 -- 172.21.15.70:0/3487648469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 msgr2=0x7f5b500b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3487648469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b500b1ff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009e50 tx=0x7f5b4802bed0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 -- 172.21.15.70:0/3487648469 shutdown_connections 2023-08-25T09:40:50.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3487648469 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b500b1ff0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3487648469 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b5006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 -- 172.21.15.70:0/3487648469 >> 172.21.15.70:0/3487648469 conn(0x7f5b50069850 msgr2=0x7f5b50069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:50.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 -- 172.21.15.70:0/3487648469 shutdown_connections 2023-08-25T09:40:50.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.922+0000 7f5b57a83640 1 -- 172.21.15.70:0/3487648469 wait complete. 2023-08-25T09:40:50.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.924+0000 7f5b57a83640 1 Processor -- start 2023-08-25T09:40:50.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.924+0000 7f5b57a83640 1 -- start start 2023-08-25T09:40:50.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b57a83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5006b2b0 0x7f5b50158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b57a83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b50158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b57a83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b50159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b57a83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500ca8c0 con 0x7f5b5015ec10 2023-08-25T09:40:50.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b57a83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b501e4270 con 0x7f5b5006b2b0 2023-08-25T09:40:50.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b57a83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b501e8ad0 con 0x7f5b5015e830 2023-08-25T09:40:50.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b56020640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b50159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b56020640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b50159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48262/0 (socket says 172.21.15.70:48262) 2023-08-25T09:40:50.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b56020640 1 -- 172.21.15.70:0/3839012554 learned_addr learned my addr 172.21.15.70:0/3839012554 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:50.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.925+0000 7f5b5581f640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5006b2b0 0x7f5b50158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b56020640 1 -- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 msgr2=0x7f5b50158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b5501e640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b50158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b56020640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b50158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b56020640 1 -- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5006b2b0 msgr2=0x7f5b50158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:50.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b56020640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5006b2b0 0x7f5b50158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:50.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b56020640 1 -- 172.21.15.70:0/3839012554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48009b30 con 0x7f5b5015ec10 2023-08-25T09:40:50.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b5501e640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b50158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:50.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b56020640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b50159340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009e50 tx=0x7f5b48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:50.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b48009510 con 0x7f5b5015ec10 2023-08-25T09:40:50.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4803e450 con 0x7f5b5015ec10 2023-08-25T09:40:50.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4803e7e0 con 0x7f5b5015ec10 2023-08-25T09:40:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5015ca20 con 0x7f5b5015ec10 2023-08-25T09:40:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.926+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b501e44e0 con 0x7f5b5015ec10 2023-08-25T09:40:50.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.928+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b501e4700 con 0x7f5b5015ec10 2023-08-25T09:40:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.929+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5b4803e940 con 0x7f5b5015ec10 2023-08-25T09:40:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.930+0000 7f5b467fc640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b3406a050 0x7f5b3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.930+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5b480ab6c0 con 0x7f5b5015ec10 2023-08-25T09:40:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.930+0000 7f5b5581f640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b3406a050 0x7f5b3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:50.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.931+0000 7f5b5581f640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b3406a050 0x7f5b3406c510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:50.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:50.935+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5b48076d70 con 0x7f5b5015ec10 2023-08-25T09:40:51.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.127+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b5004e420 con 0x7f5b5015ec10 2023-08-25T09:40:51.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.129+0000 7f5b467fc640 1 -- 172.21.15.70:0/3839012554 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18401 (secure 0 0 0) 0x7f5b4807a390 con 0x7f5b5015ec10 2023-08-25T09:40:51.131 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:51.132 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:44.272862+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:51.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b3406a050 msgr2=0x7f5b3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b3406a050 0x7f5b3406c510 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c006fd0 tx=0x7f5b4c008040 comp rx=0 tx=0).stop 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 msgr2=0x7f5b50159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b50159340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f5b48009e50 tx=0x7f5b48004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 shutdown_connections 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b3406a050 0x7f5b3406c510 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b5015ec10 0x7f5b50159340 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b5015e830 0x7f5b50158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 --2- 172.21.15.70:0/3839012554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b5006b2b0 0x7f5b50158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 >> 172.21.15.70:0/3839012554 conn(0x7f5b50069850 msgr2=0x7f5b5015dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:51.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 shutdown_connections 2023-08-25T09:40:51.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.132+0000 7f5b57a83640 1 -- 172.21.15.70:0/3839012554 wait complete. 2023-08-25T09:40:51.153 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:40:25.098622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:40:25.912835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:51.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.371+0000 7f6d3b36a640 1 Processor -- start 2023-08-25T09:40:51.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.372+0000 7f6d3b36a640 1 -- start start 2023-08-25T09:40:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d3b36a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d34159bf0 0x7f6d3415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d3b36a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6d3406f0e0 0x7f6d34070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d3b36a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3415e780 con 0x7f6d340cd660 2023-08-25T09:40:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d3b36a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3406f8e0 con 0x7f6d34159bf0 2023-08-25T09:40:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d3b36a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3406fbc0 con 0x7f6d3406f0e0 2023-08-25T09:40:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d39106640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d34159bf0 0x7f6d3415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d39106640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6d34159bf0 0x7f6d3415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48264/0 (socket says 172.21.15.70:48264) 2023-08-25T09:40:51.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.373+0000 7f6d39106640 1 -- 172.21.15.70:0/3350157692 learned_addr learned my addr 172.21.15.70:0/3350157692 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.374+0000 7f6d39106640 1 -- 172.21.15.70:0/3350157692 >> v2:172.21.15.155:3300/0 conn(0x7f6d3406f0e0 msgr2=0x7f6d34070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.374+0000 7f6d39106640 1 --2- 172.21.15.70:0/3350157692 >> v2:172.21.15.155:3300/0 conn(0x7f6d3406f0e0 0x7f6d34070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.374+0000 7f6d39106640 1 -- 172.21.15.70:0/3350157692 >> v1:172.21.15.155:6789/0 conn(0x7f6d340cd660 legacy=0x7f6d340c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:51.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.374+0000 7f6d39106640 1 -- 172.21.15.70:0/3350157692 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3406fea0 con 0x7f6d34159bf0 2023-08-25T09:40:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.375+0000 7f6d39106640 1 --2- 172.21.15.70:0/3350157692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d3415e240 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d3406af90 tx=0x7f6d2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75729c63995c232 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.375+0000 7f6d33fff640 1 -- 172.21.15.70:0/3350157692 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24031870 con 0x7f6d34159bf0 2023-08-25T09:40:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.375+0000 7f6d33fff640 1 -- 172.21.15.70:0/3350157692 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d240319d0 con 0x7f6d34159bf0 2023-08-25T09:40:51.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.375+0000 7f6d33fff640 1 -- 172.21.15.70:0/3350157692 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d24031d60 con 0x7f6d34159bf0 2023-08-25T09:40:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3350157692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 msgr2=0x7f6d3415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3350157692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d3415e240 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d3406af90 tx=0x7f6d2402b910 comp rx=0 tx=0).stop 2023-08-25T09:40:51.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3350157692 shutdown_connections 2023-08-25T09:40:51.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3350157692 >> v2:172.21.15.155:3300/0 conn(0x7f6d3406f0e0 0x7f6d34070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3350157692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d3415e240 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3350157692 >> 172.21.15.70:0/3350157692 conn(0x7f6d340cb0d0 msgr2=0x7f6d340cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:51.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3350157692 shutdown_connections 2023-08-25T09:40:51.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.376+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3350157692 wait complete. 2023-08-25T09:40:51.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.378+0000 7f6d3b36a640 1 Processor -- start 2023-08-25T09:40:51.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.378+0000 7f6d3b36a640 1 -- start start 2023-08-25T09:40:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d3b36a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3406f0e0 0x7f6d340c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d3b36a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d340cd660 0x7f6d34156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d3b36a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d340c1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d3b36a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c5bd0 con 0x7f6d34159bf0 2023-08-25T09:40:51.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d3b36a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d34156960 con 0x7f6d3406f0e0 2023-08-25T09:40:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d3b36a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d340c5e70 con 0x7f6d340cd660 2023-08-25T09:40:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d39907640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d340c1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d39907640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d340c1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48276/0 (socket says 172.21.15.70:48276) 2023-08-25T09:40:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.379+0000 7f6d39907640 1 -- 172.21.15.70:0/3724159618 learned_addr learned my addr 172.21.15.70:0/3724159618 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:51.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39106640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3406f0e0 0x7f6d340c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39907640 1 -- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d340cd660 msgr2=0x7f6d34156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d38905640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d340cd660 0x7f6d34156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39907640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d340cd660 0x7f6d34156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39907640 1 -- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3406f0e0 msgr2=0x7f6d340c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39907640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3406f0e0 0x7f6d340c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39907640 1 -- 172.21.15.70:0/3724159618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240094d0 con 0x7f6d34159bf0 2023-08-25T09:40:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39106640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3406f0e0 0x7f6d340c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d38905640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d340cd660 0x7f6d34156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d39907640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d340c1930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c780 tx=0x7f6d2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:51.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c003d80 con 0x7f6d34159bf0 2023-08-25T09:40:51.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c00ce90 con 0x7f6d34159bf0 2023-08-25T09:40:51.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0052b0 con 0x7f6d34159bf0 2023-08-25T09:40:51.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.380+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d340c6150 con 0x7f6d34159bf0 2023-08-25T09:40:51.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.381+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d340c6710 con 0x7f6d34159bf0 2023-08-25T09:40:51.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.382+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d34066fb0 con 0x7f6d34159bf0 2023-08-25T09:40:51.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.383+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6d2c005410 con 0x7f6d34159bf0 2023-08-25T09:40:51.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.384+0000 7f6d31ffb640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d2006a050 0x7f6d2006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.384+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6d2c087af0 con 0x7f6d34159bf0 2023-08-25T09:40:51.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.384+0000 7f6d39106640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d2006a050 0x7f6d2006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.385+0000 7f6d39106640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d2006a050 0x7f6d2006c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6d24036000 tx=0x7f6d2403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:51.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.388+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6d2c053190 con 0x7f6d34159bf0 2023-08-25T09:40:51.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.577+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d340cbf50 con 0x7f6d34159bf0 2023-08-25T09:40:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.578+0000 7f6d31ffb640 1 -- 172.21.15.70:0/3724159618 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d2c0170a0 con 0x7f6d34159bf0 2023-08-25T09:40:51.580 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d2006a050 msgr2=0x7f6d2006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d2006a050 0x7f6d2006c510 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6d24036000 tx=0x7f6d2403c040 comp rx=0 tx=0).stop 2023-08-25T09:40:51.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 msgr2=0x7f6d340c1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d340c1930 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c780 tx=0x7f6d2c00cc40 comp rx=0 tx=0).stop 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 shutdown_connections 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d2006a050 0x7f6d2006c510 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6d34159bf0 0x7f6d340c1930 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6d340cd660 0x7f6d34156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 --2- 172.21.15.70:0/3724159618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6d3406f0e0 0x7f6d340c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 >> 172.21.15.70:0/3724159618 conn(0x7f6d340cb0d0 msgr2=0x7f6d34151e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:51.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 shutdown_connections 2023-08-25T09:40:51.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.582+0000 7f6d3b36a640 1 -- 172.21.15.70:0/3724159618 wait complete. 2023-08-25T09:40:51.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:51.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.817+0000 7f065ffff640 1 Processor -- start 2023-08-25T09:40:51.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.818+0000 7f065ffff640 1 -- start start 2023-08-25T09:40:51.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.819+0000 7f065ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 0x7f06600c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.819+0000 7f065ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06600b1ce0 0x7f06600ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.819+0000 7f065ffff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06600b2170 con 0x7f066015a920 2023-08-25T09:40:51.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.819+0000 7f065ffff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06600ad8f0 con 0x7f06600b1ce0 2023-08-25T09:40:51.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.819+0000 7f065ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06600adbb0 con 0x7f06600b0320 2023-08-25T09:40:51.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.820+0000 7f065effd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 0x7f06600c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.820+0000 7f0657fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f066015a920 0x7f06600b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:35828/0 (socket says 172.21.15.70:35828) 2023-08-25T09:40:51.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.820+0000 7f065effd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 0x7f06600c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:43762/0 (socket says 172.21.15.70:43762) 2023-08-25T09:40:51.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.820+0000 7f0657fff640 1 -- 172.21.15.70:0/1647001301 learned_addr learned my addr 172.21.15.70:0/1647001301 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:51.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.820+0000 7f065f7fe640 1 --2- 172.21.15.70:0/1647001301 >> v2:172.21.15.155:3300/0 conn(0x7f06600b1ce0 0x7f06600ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065e7fc640 1 -- 172.21.15.70:0/1647001301 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069937604 0 0) 0x7f06600b2170 con 0x7f066015a920 2023-08-25T09:40:51.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065e7fc640 1 -- 172.21.15.70:0/1647001301 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f064c003520 con 0x7f066015a920 2023-08-25T09:40:51.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065effd640 1 -- 172.21.15.70:0/1647001301 >> v2:172.21.15.155:3300/0 conn(0x7f06600b1ce0 msgr2=0x7f06600ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:51.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065effd640 1 --2- 172.21.15.70:0/1647001301 >> v2:172.21.15.155:3300/0 conn(0x7f06600b1ce0 0x7f06600ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065effd640 1 -- 172.21.15.70:0/1647001301 >> v1:172.21.15.155:6789/0 conn(0x7f066015a920 legacy=0x7f06600b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065effd640 1 -- 172.21.15.70:0/1647001301 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06600c90a0 con 0x7f06600b0320 2023-08-25T09:40:51.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.821+0000 7f065f7fe640 1 --2- 172.21.15.70:0/1647001301 >> v2:172.21.15.155:3300/0 conn(0x7f06600b1ce0 0x7f06600ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:40:51.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.822+0000 7f065effd640 1 --2- 172.21.15.70:0/1647001301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 0x7f06600c9e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0650009790 tx=0x7f065002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=575e1a8568c75b4e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:51.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.823+0000 7f065e7fc640 1 -- 172.21.15.70:0/1647001301 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065002d040 con 0x7f06600b0320 2023-08-25T09:40:51.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.823+0000 7f065e7fc640 1 -- 172.21.15.70:0/1647001301 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0650031810 con 0x7f06600b0320 2023-08-25T09:40:51.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.823+0000 7f065e7fc640 1 -- 172.21.15.70:0/1647001301 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650031ba0 con 0x7f06600b0320 2023-08-25T09:40:51.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 -- 172.21.15.70:0/1647001301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 msgr2=0x7f06600c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 --2- 172.21.15.70:0/1647001301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 0x7f06600c9e60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0650009790 tx=0x7f065002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 -- 172.21.15.70:0/1647001301 shutdown_connections 2023-08-25T09:40:51.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 --2- 172.21.15.70:0/1647001301 >> v2:172.21.15.155:3300/0 conn(0x7f06600b1ce0 0x7f06600ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 --2- 172.21.15.70:0/1647001301 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06600b0320 0x7f06600c9e60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 -- 172.21.15.70:0/1647001301 >> 172.21.15.70:0/1647001301 conn(0x7f06600ce830 msgr2=0x7f06600cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:51.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 -- 172.21.15.70:0/1647001301 shutdown_connections 2023-08-25T09:40:51.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.824+0000 7f065ffff640 1 -- 172.21.15.70:0/1647001301 wait complete. 2023-08-25T09:40:51.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.826+0000 7f065ffff640 1 Processor -- start 2023-08-25T09:40:51.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.826+0000 7f065ffff640 1 -- start start 2023-08-25T09:40:51.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 0x7f066006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06600b1ce0 0x7f0660154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015a920 0x7f06600aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066006a4d0 con 0x7f06600b0320 2023-08-25T09:40:51.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065ffff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066006ae30 con 0x7f06600b1ce0 2023-08-25T09:40:51.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0660069850 con 0x7f066015a920 2023-08-25T09:40:51.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 0x7f066006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 0x7f066006a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48278/0 (socket says 172.21.15.70:48278) 2023-08-25T09:40:51.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065effd640 1 -- 172.21.15.70:0/2976595644 learned_addr learned my addr 172.21.15.70:0/2976595644 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:51.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f0657fff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06600b1ce0 0x7f0660154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065effd640 1 -- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015a920 msgr2=0x7f06600aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065effd640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015a920 0x7f06600aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.828+0000 7f065effd640 1 -- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06600b1ce0 msgr2=0x7f0660154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:51.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065effd640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06600b1ce0 0x7f0660154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:51.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065effd640 1 -- 172.21.15.70:0/2976595644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0658009a70 con 0x7f06600b0320 2023-08-25T09:40:51.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065effd640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 0x7f066006a8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f065002bec0 tx=0x7f06500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:51.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650031610 con 0x7f06600b0320 2023-08-25T09:40:51.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065003ace0 con 0x7f06600b0320 2023-08-25T09:40:51.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650005bd0 con 0x7f06600b0320 2023-08-25T09:40:51.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06500093a0 con 0x7f06600b0320 2023-08-25T09:40:51.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.829+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0660069e10 con 0x7f06600b0320 2023-08-25T09:40:51.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.831+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f066006a610 con 0x7f06600b0320 2023-08-25T09:40:51.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.831+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f06500423f0 con 0x7f06600b0320 2023-08-25T09:40:51.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.832+0000 7f065cff9640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064006a080 0x7f064006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:51.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.832+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f06500aa300 con 0x7f06600b0320 2023-08-25T09:40:51.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.833+0000 7f0657fff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064006a080 0x7f064006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:51.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.833+0000 7f0657fff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064006a080 0x7f064006c540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06601f5b80 tx=0x7f0648009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:51.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:51.836+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f066006a610 con 0x7f06600b0320 2023-08-25T09:40:52.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.028+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f066006a610 con 0x7f06600b0320 2023-08-25T09:40:52.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.029+0000 7f065cff9640 1 -- 172.21.15.70:0/2976595644 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f066006a610 con 0x7f06600b0320 2023-08-25T09:40:52.031 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:52.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064006a080 msgr2=0x7f064006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064006a080 0x7f064006c540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f06601f5b80 tx=0x7f0648009040 comp rx=0 tx=0).stop 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 msgr2=0x7f066006a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 0x7f066006a8f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f065002bec0 tx=0x7f06500046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 shutdown_connections 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f064006a080 0x7f064006c540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f066015a920 0x7f06600aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06600b1ce0 0x7f0660154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 --2- 172.21.15.70:0/2976595644 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06600b0320 0x7f066006a8f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 >> 172.21.15.70:0/2976595644 conn(0x7f06600ce830 msgr2=0x7f06600654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:52.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 shutdown_connections 2023-08-25T09:40:52.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.032+0000 7f065ffff640 1 -- 172.21.15.70:0/2976595644 wait complete. 2023-08-25T09:40:52.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.271+0000 7fd88cf53640 1 Processor -- start 2023-08-25T09:40:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.271+0000 7fd88cf53640 1 -- start start 2023-08-25T09:40:52.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.273+0000 7fd88cf53640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8880d09f0 0x7fd8880cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.273+0000 7fd88cf53640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88815d0b0 con 0x7fd88815ccd0 2023-08-25T09:40:52.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.273+0000 7fd88cf53640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880b58d0 con 0x7fd8880cf820 2023-08-25T09:40:52.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.273+0000 7fd88cf53640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880cfc90 con 0x7fd8880d09f0 2023-08-25T09:40:52.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.273+0000 7fd8877fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd88815ccd0 0x7fd8880d1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40944/0 (socket says 172.21.15.70:40944) 2023-08-25T09:40:52.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.273+0000 7fd8877fe640 1 -- 172.21.15.70:0/248635543 learned_addr learned my addr 172.21.15.70:0/248635543 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:52.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd886ffd640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8880d09f0 0x7fd8880cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706951546 0 0) 0x7fd88815d0b0 con 0x7fd88815ccd0 2023-08-25T09:40:52.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd870003520 con 0x7fd88815ccd0 2023-08-25T09:40:52.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672247049 0 0) 0x7fd870003520 con 0x7fd88815ccd0 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8880d09f0 msgr2=0x7fd8880cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8880d09f0 0x7fd8880cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 >> v1:172.21.15.155:6789/0 conn(0x7fd8880cf820 legacy=0x7fd8880b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880d25d0 con 0x7fd88815ccd0 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fd878001670 con 0x7fd88815ccd0 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 >> v1:172.21.15.70:6789/0 conn(0x7fd88815ccd0 legacy=0x7fd8880d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700039a0 0x7fd870003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8700047c0 0x7fd870004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd870005140 0x7fd87000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880cfc90 con 0x7fd8700039a0 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87000bec0 con 0x7fd870005140 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.274+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87000c000 con 0x7fd8700047c0 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd8877fe640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700039a0 0x7fd870003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd886ffd640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8700047c0 0x7fd870004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd8877fe640 1 -- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8700047c0 msgr2=0x7fd870004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:52.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd8877fe640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8700047c0 0x7fd870004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd8877fe640 1 -- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd870005140 msgr2=0x7fd87000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:52.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd8877fe640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd870005140 0x7fd87000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.275+0000 7fd8877fe640 1 -- 172.21.15.70:0/248635543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870003520 con 0x7fd8700039a0 2023-08-25T09:40:52.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.276+0000 7fd8877fe640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700039a0 0x7fd870003dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c240 tx=0x7fd8700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ad43ffc5e50f91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:52.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.276+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878005ad0 con 0x7fd8700039a0 2023-08-25T09:40:52.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.276+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878005c30 con 0x7fd8700039a0 2023-08-25T09:40:52.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.276+0000 7fd8867fc640 1 -- 172.21.15.70:0/248635543 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87802c8a0 con 0x7fd8700039a0 2023-08-25T09:40:52.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd88cf53640 1 -- 172.21.15.70:0/248635543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700039a0 msgr2=0x7fd870003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd88cf53640 1 --2- 172.21.15.70:0/248635543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8700039a0 0x7fd870003dc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd87800c240 tx=0x7fd8700057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd887fff640 1 -- 172.21.15.70:0/248635543 reap_dead start 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd88cf53640 1 -- 172.21.15.70:0/248635543 shutdown_connections 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd88cf53640 1 -- 172.21.15.70:0/248635543 >> 172.21.15.70:0/248635543 conn(0x7fd888069850 msgr2=0x7fd888069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd88cf53640 1 -- 172.21.15.70:0/248635543 shutdown_connections 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.277+0000 7fd88cf53640 1 -- 172.21.15.70:0/248635543 wait complete. 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.279+0000 7fd88cf53640 1 Processor -- start 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.279+0000 7fd88cf53640 1 -- start start 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.280+0000 7fd88cf53640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8700039a0 0x7fd88815e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.280+0000 7fd88cf53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 0x7fd8880ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.280+0000 7fd88cf53640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd888152850 0x7fd8880ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.280+0000 7fd88cf53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88815dc70 con 0x7fd888153340 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.280+0000 7fd88cf53640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88815e640 con 0x7fd8700039a0 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.280+0000 7fd88cf53640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881591a0 con 0x7fd888152850 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 0x7fd8880ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 0x7fd8880ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48288/0 (socket says 172.21.15.70:48288) 2023-08-25T09:40:52.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 -- 172.21.15.70:0/2193866762 learned_addr learned my addr 172.21.15.70:0/2193866762 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd887fff640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd888152850 0x7fd8880ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd8877fe640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8700039a0 0x7fd88815e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 -- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd888152850 msgr2=0x7fd8880ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd888152850 0x7fd8880ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 -- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8700039a0 msgr2=0x7fd88815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8700039a0 0x7fd88815e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 -- 172.21.15.70:0/2193866762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd888158ec0 con 0x7fd888153340 2023-08-25T09:40:52.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd887fff640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd888152850 0x7fd8880ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:52.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.281+0000 7fd886ffd640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 0x7fd8880ca380 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd87c011600 tx=0x7fd87c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.282+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00f920 con 0x7fd888153340 2023-08-25T09:40:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.282+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c00fa80 con 0x7fd888153340 2023-08-25T09:40:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.282+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88815d620 con 0x7fd888153340 2023-08-25T09:40:52.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.282+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00fdf0 con 0x7fd888153340 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.282+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888158ae0 con 0x7fd888153340 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.283+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88815de20 con 0x7fd888153340 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.284+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd87c015460 con 0x7fd888153340 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.285+0000 7fd86ffff640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd87406a080 0x7fd87406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.285+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd87c088d90 con 0x7fd888153340 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.285+0000 7fd8877fe640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd87406a080 0x7fd87406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.286+0000 7fd8877fe640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd87406a080 0x7fd87406c540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd878034000 tx=0x7fd878009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:52.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.289+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd88815de20 con 0x7fd888153340 2023-08-25T09:40:52.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.478+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd888061460 con 0x7fd888153340 2023-08-25T09:40:52.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.479+0000 7fd86ffff640 1 -- 172.21.15.70:0/2193866762 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd87c054440 con 0x7fd888153340 2023-08-25T09:40:52.481 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:52.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd87406a080 msgr2=0x7fd87406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd87406a080 0x7fd87406c540 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd878034000 tx=0x7fd878009030 comp rx=0 tx=0).stop 2023-08-25T09:40:52.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 msgr2=0x7fd8880ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 0x7fd8880ca380 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd87c011600 tx=0x7fd87c011ac0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 shutdown_connections 2023-08-25T09:40:52.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd87406a080 0x7fd87406c540 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd888152850 0x7fd8880ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd888153340 0x7fd8880ca380 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 --2- 172.21.15.70:0/2193866762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8700039a0 0x7fd88815e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.483+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 >> 172.21.15.70:0/2193866762 conn(0x7fd888069850 msgr2=0x7fd8880cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:52.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.484+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 shutdown_connections 2023-08-25T09:40:52.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.484+0000 7fd88cf53640 1 -- 172.21.15.70:0/2193866762 wait complete. 2023-08-25T09:40:52.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:52.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.721+0000 7fa925206640 1 Processor -- start 2023-08-25T09:40:52.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.721+0000 7fa925206640 1 -- start start 2023-08-25T09:40:52.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa925206640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa920159bf0 0x7fa92015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa925206640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa92006f0e0 0x7fa920070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa925206640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa92015e780 con 0x7fa9200cd660 2023-08-25T09:40:52.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa925206640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92006f8e0 con 0x7fa920159bf0 2023-08-25T09:40:52.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa925206640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92006fbc0 con 0x7fa92006f0e0 2023-08-25T09:40:52.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa91ed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa920159bf0 0x7fa92015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa91f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa9200cd660 0x7fa9200c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40958/0 (socket says 172.21.15.70:40958) 2023-08-25T09:40:52.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.723+0000 7fa91f59e640 1 -- 172.21.15.70:0/2529105045 learned_addr learned my addr 172.21.15.70:0/2529105045 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:52.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.724+0000 7fa917fff640 1 --2- 172.21.15.70:0/2529105045 >> v2:172.21.15.155:3300/0 conn(0x7fa92006f0e0 0x7fa920070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.724+0000 7fa91ed9d640 1 -- 172.21.15.70:0/2529105045 >> v2:172.21.15.155:3300/0 conn(0x7fa92006f0e0 msgr2=0x7fa920070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:52.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.724+0000 7fa91ed9d640 1 --2- 172.21.15.70:0/2529105045 >> v2:172.21.15.155:3300/0 conn(0x7fa92006f0e0 0x7fa920070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.724+0000 7fa91ed9d640 1 -- 172.21.15.70:0/2529105045 >> v1:172.21.15.70:6789/0 conn(0x7fa9200cd660 legacy=0x7fa9200c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.724+0000 7fa91ed9d640 1 -- 172.21.15.70:0/2529105045 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92006fea0 con 0x7fa920159bf0 2023-08-25T09:40:52.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.725+0000 7fa91ed9d640 1 --2- 172.21.15.70:0/2529105045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa920159bf0 0x7fa92015e240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa92006af90 tx=0x7fa91002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d10718ad0db52a5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:52.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.725+0000 7fa91e59c640 1 -- 172.21.15.70:0/2529105045 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910031830 con 0x7fa920159bf0 2023-08-25T09:40:52.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.725+0000 7fa91e59c640 1 -- 172.21.15.70:0/2529105045 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910031990 con 0x7fa920159bf0 2023-08-25T09:40:52.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.725+0000 7fa91e59c640 1 -- 172.21.15.70:0/2529105045 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910031d20 con 0x7fa920159bf0 2023-08-25T09:40:52.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 -- 172.21.15.70:0/2529105045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa920159bf0 msgr2=0x7fa92015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 --2- 172.21.15.70:0/2529105045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa920159bf0 0x7fa92015e240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa92006af90 tx=0x7fa91002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 -- 172.21.15.70:0/2529105045 shutdown_connections 2023-08-25T09:40:52.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 --2- 172.21.15.70:0/2529105045 >> v2:172.21.15.155:3300/0 conn(0x7fa92006f0e0 0x7fa920070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 --2- 172.21.15.70:0/2529105045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa920159bf0 0x7fa92015e240 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 -- 172.21.15.70:0/2529105045 >> 172.21.15.70:0/2529105045 conn(0x7fa9200cb0d0 msgr2=0x7fa9200cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:52.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 -- 172.21.15.70:0/2529105045 shutdown_connections 2023-08-25T09:40:52.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.726+0000 7fa925206640 1 -- 172.21.15.70:0/2529105045 wait complete. 2023-08-25T09:40:52.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.728+0000 7fa925206640 1 Processor -- start 2023-08-25T09:40:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.728+0000 7fa925206640 1 -- start start 2023-08-25T09:40:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa925206640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 0x7fa920153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa925206640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9200cd660 0x7fa92015c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa925206640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa920159bf0 0x7fa92015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa925206640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92015a8e0 con 0x7fa92006f0e0 2023-08-25T09:40:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa925206640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9201f2460 con 0x7fa9200cd660 2023-08-25T09:40:52.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa925206640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa920159fd0 con 0x7fa920159bf0 2023-08-25T09:40:52.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 0x7fa920153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 0x7fa920153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48292/0 (socket says 172.21.15.70:48292) 2023-08-25T09:40:52.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 -- 172.21.15.70:0/3991478637 learned_addr learned my addr 172.21.15.70:0/3991478637 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:52.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91f59e640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa920159bf0 0x7fa92015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 -- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa920159bf0 msgr2=0x7fa92015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa920159bf0 0x7fa92015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 -- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9200cd660 msgr2=0x7fa92015c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.729+0000 7fa91ed9d640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9200cd660 0x7fa92015c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa91ed9d640 1 -- 172.21.15.70:0/3991478637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009400 con 0x7fa92006f0e0 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa91f59e640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa920159bf0 0x7fa92015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa91ed9d640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 0x7fa920153060 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:52.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91003aa60 con 0x7fa92006f0e0 2023-08-25T09:40:52.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91003abc0 con 0x7fa92006f0e0 2023-08-25T09:40:52.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9100443f0 con 0x7fa92006f0e0 2023-08-25T09:40:52.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9201535a0 con 0x7fa92006f0e0 2023-08-25T09:40:52.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.730+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92015a220 con 0x7fa92006f0e0 2023-08-25T09:40:52.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.732+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9200cf7a0 con 0x7fa92006f0e0 2023-08-25T09:40:52.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.732+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa910044550 con 0x7fa92006f0e0 2023-08-25T09:40:52.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.733+0000 7fa9177fe640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa90006a080 0x7fa90006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:52.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.733+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa9100aa830 con 0x7fa92006f0e0 2023-08-25T09:40:52.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.734+0000 7fa917fff640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa90006a080 0x7fa90006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:52.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.734+0000 7fa917fff640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa90006a080 0x7fa90006c540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9201588c0 tx=0x7fa908008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:52.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.738+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa910075ff0 con 0x7fa92006f0e0 2023-08-25T09:40:52.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.925+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa920066fb0 con 0x7fa92006f0e0 2023-08-25T09:40:52.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.926+0000 7fa9177fe640 1 -- 172.21.15.70:0/3991478637 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa910035020 con 0x7fa92006f0e0 2023-08-25T09:40:52.928 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:52.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.928+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa90006a080 msgr2=0x7fa90006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.928+0000 7fa925206640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa90006a080 0x7fa90006c540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa9201588c0 tx=0x7fa908008040 comp rx=0 tx=0).stop 2023-08-25T09:40:52.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.928+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 msgr2=0x7fa920153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:52.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.928+0000 7fa925206640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 0x7fa920153060 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa9100046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 shutdown_connections 2023-08-25T09:40:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa90006a080 0x7fa90006c540 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa920159bf0 0x7fa92015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa9200cd660 0x7fa92015c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 --2- 172.21.15.70:0/3991478637 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa92006f0e0 0x7fa920153060 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 >> 172.21.15.70:0/3991478637 conn(0x7fa9200cb0d0 msgr2=0x7fa920069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 shutdown_connections 2023-08-25T09:40:52.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:52.929+0000 7fa925206640 1 -- 172.21.15.70:0/3991478637 wait complete. 2023-08-25T09:40:52.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:40:53.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.158+0000 7fb4accec640 1 Processor -- start 2023-08-25T09:40:53.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.158+0000 7fb4accec640 1 -- start start 2023-08-25T09:40:53.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4accec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4a80ca780 0x7fb4a80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:53.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4accec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a80b12b0 con 0x7fb4a815acd0 2023-08-25T09:40:53.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4accec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a806ae20 con 0x7fb4a80b3cf0 2023-08-25T09:40:53.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4accec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a80b47e0 con 0x7fb4a80ca780 2023-08-25T09:40:53.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4a5d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4a80ca780 0x7fb4a80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:53.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4a5d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4a80ca780 0x7fb4a80cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48298/0 (socket says 172.21.15.70:48298) 2023-08-25T09:40:53.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4a5d9b640 1 -- 172.21.15.70:0/2921791778 learned_addr learned my addr 172.21.15.70:0/2921791778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:53.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4a5d9b640 1 -- 172.21.15.70:0/2921791778 >> v1:172.21.15.155:6789/0 conn(0x7fb4a80b3cf0 legacy=0x7fb4a80b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:53.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4a5d9b640 1 -- 172.21.15.70:0/2921791778 >> v1:172.21.15.70:6789/0 conn(0x7fb4a815acd0 legacy=0x7fb4a815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:53.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.160+0000 7fb4a5d9b640 1 -- 172.21.15.70:0/2921791778 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a815b7c0 con 0x7fb4a80ca780 2023-08-25T09:40:53.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4a5d9b640 1 --2- 172.21.15.70:0/2921791778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a80ca780 0x7fb4a80cac30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009870 tx=0x7fb49c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459846a00a29983c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:53.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4a559a640 1 -- 172.21.15.70:0/2921791778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c031870 con 0x7fb4a80ca780 2023-08-25T09:40:53.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4a559a640 1 -- 172.21.15.70:0/2921791778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c0319d0 con 0x7fb4a80ca780 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4a559a640 1 -- 172.21.15.70:0/2921791778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c031d60 con 0x7fb4a80ca780 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4accec640 1 -- 172.21.15.70:0/2921791778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a80ca780 msgr2=0x7fb4a80cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4accec640 1 --2- 172.21.15.70:0/2921791778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a80ca780 0x7fb4a80cac30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb49c009870 tx=0x7fb49c02b910 comp rx=0 tx=0).stop 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4accec640 1 -- 172.21.15.70:0/2921791778 shutdown_connections 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4accec640 1 --2- 172.21.15.70:0/2921791778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a80ca780 0x7fb4a80cac30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.162+0000 7fb4accec640 1 -- 172.21.15.70:0/2921791778 >> 172.21.15.70:0/2921791778 conn(0x7fb4a80ce760 msgr2=0x7fb4a80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:53.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.163+0000 7fb4accec640 1 -- 172.21.15.70:0/2921791778 shutdown_connections 2023-08-25T09:40:53.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.163+0000 7fb4accec640 1 -- 172.21.15.70:0/2921791778 wait complete. 2023-08-25T09:40:53.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.164+0000 7fb4accec640 1 Processor -- start 2023-08-25T09:40:53.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.164+0000 7fb4accec640 1 -- start start 2023-08-25T09:40:53.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4accec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4a80b3cf0 0x7fb4a80ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:53.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4accec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4a80ca780 0x7fb4a8146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:53.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4accec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 0x7fb4a80cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:53.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4accec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8153620 con 0x7fb4a815acd0 2023-08-25T09:40:53.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4accec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80705e0 con 0x7fb4a80ca780 2023-08-25T09:40:53.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4accec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a81538c0 con 0x7fb4a80b3cf0 2023-08-25T09:40:53.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.165+0000 7fb4a6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 0x7fb4a80cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 0x7fb4a80cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48306/0 (socket says 172.21.15.70:48306) 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 -- 172.21.15.70:0/2706577400 learned_addr learned my addr 172.21.15.70:0/2706577400 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a659c640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4a80b3cf0 0x7fb4a80ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 -- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4a80b3cf0 msgr2=0x7fb4a80ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4a80b3cf0 0x7fb4a80ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a5d9b640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4a80ca780 0x7fb4a8146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:53.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 -- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4a80ca780 msgr2=0x7fb4a8146150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4a80ca780 0x7fb4a8146150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 -- 172.21.15.70:0/2706577400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c0094d0 con 0x7fb4a815acd0 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb4a6d9d640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 0x7fb4a80cb2e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000c780 tx=0x7fb4a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.166+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0004b60 con 0x7fb4a815acd0 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.167+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a000ce90 con 0x7fb4a815acd0 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.167+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a000a850 con 0x7fb4a815acd0 2023-08-25T09:40:53.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.167+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a8153ba0 con 0x7fb4a815acd0 2023-08-25T09:40:53.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.167+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a8146690 con 0x7fb4a815acd0 2023-08-25T09:40:53.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.168+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a81540c0 con 0x7fb4a815acd0 2023-08-25T09:40:53.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.169+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb4a000a9b0 con 0x7fb4a815acd0 2023-08-25T09:40:53.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.170+0000 7fb496ffd640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb48806a050 0x7fb48806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:53.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.170+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb4a0088840 con 0x7fb4a815acd0 2023-08-25T09:40:53.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.170+0000 7fb4a659c640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb48806a050 0x7fb48806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:53.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.171+0000 7fb4a659c640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb48806a050 0x7fb48806c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb498000fd0 tx=0x7fb498009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:53.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.174+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb4a81540c0 con 0x7fb4a815acd0 2023-08-25T09:40:53.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.363+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4a81540c0 con 0x7fb4a815acd0 2023-08-25T09:40:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.364+0000 7fb496ffd640 1 -- 172.21.15.70:0/2706577400 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb4a81540c0 con 0x7fb4a815acd0 2023-08-25T09:40:53.365 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:40:53.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb48806a050 msgr2=0x7fb48806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb48806a050 0x7fb48806c510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb498000fd0 tx=0x7fb498009040 comp rx=0 tx=0).stop 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 msgr2=0x7fb4a80cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 0x7fb4a80cb2e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb4a000c780 tx=0x7fb4a000cc40 comp rx=0 tx=0).stop 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 shutdown_connections 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb48806a050 0x7fb48806c510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4a815acd0 0x7fb4a80cb2e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb4a80ca780 0x7fb4a8146150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 --2- 172.21.15.70:0/2706577400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4a80b3cf0 0x7fb4a80ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 >> 172.21.15.70:0/2706577400 conn(0x7fb4a80ce760 msgr2=0x7fb4a80b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 shutdown_connections 2023-08-25T09:40:53.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:53.366+0000 7fb4accec640 1 -- 172.21.15.70:0/2706577400 wait complete. 2023-08-25T09:40:53.386 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:40:53.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:53.387 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:40:53.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[87948]: fuse finished with error 0 and tester_r 0 2023-08-25T09:40:53.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:40:53.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:53.493 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:40:53.671 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:40:53.672 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:53.730 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:40:53.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:40:53.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:53.732 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:40:53.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:40:53.785 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:40:53.786 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:40:53.786 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:40:53.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:40:53.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:40:53.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:40:53.845 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:40:53.897 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:40:53.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:40:53.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:54.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.117+0000 7f8856c21640 1 Processor -- start 2023-08-25T09:40:54.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.117+0000 7f8856c21640 1 -- start start 2023-08-25T09:40:54.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.119+0000 7f8856c21640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88500cea30 0x7f88500b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.119+0000 7f8856c21640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885006f050 con 0x7f88500ac870 2023-08-25T09:40:54.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.119+0000 7f8856c21640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885006f330 con 0x7f885015cb40 2023-08-25T09:40:54.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.119+0000 7f8856c21640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885006f610 con 0x7f88500cea30 2023-08-25T09:40:54.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.120+0000 7f884ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88500cea30 0x7f88500b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.120+0000 7f884ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88500cea30 0x7f88500b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48310/0 (socket says 172.21.15.70:48310) 2023-08-25T09:40:54.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.120+0000 7f884ffff640 1 -- 172.21.15.70:0/739536314 learned_addr learned my addr 172.21.15.70:0/739536314 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:54.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.120+0000 7f884ffff640 1 -- 172.21.15.70:0/739536314 >> v1:172.21.15.155:6789/0 conn(0x7f885015cb40 legacy=0x7f88500ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:54.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.120+0000 7f884ffff640 1 -- 172.21.15.70:0/739536314 >> v1:172.21.15.70:6789/0 conn(0x7f88500ac870 legacy=0x7f88500b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:54.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.120+0000 7f884ffff640 1 -- 172.21.15.70:0/739536314 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885006f8f0 con 0x7f88500cea30 2023-08-25T09:40:54.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.121+0000 7f884ffff640 1 --2- 172.21.15.70:0/739536314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500cea30 0x7f88500b1b30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8840009e50 tx=0x7f884002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a5c56f21f48d8b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.121+0000 7f884f7fe640 1 -- 172.21.15.70:0/739536314 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884002ca80 con 0x7f88500cea30 2023-08-25T09:40:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.121+0000 7f884f7fe640 1 -- 172.21.15.70:0/739536314 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884002cbe0 con 0x7f88500cea30 2023-08-25T09:40:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.121+0000 7f884f7fe640 1 -- 172.21.15.70:0/739536314 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88400343f0 con 0x7f88500cea30 2023-08-25T09:40:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 -- 172.21.15.70:0/739536314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500cea30 msgr2=0x7f88500b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 --2- 172.21.15.70:0/739536314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500cea30 0x7f88500b1b30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8840009e50 tx=0x7f884002bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 -- 172.21.15.70:0/739536314 shutdown_connections 2023-08-25T09:40:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 --2- 172.21.15.70:0/739536314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500cea30 0x7f88500b1b30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 -- 172.21.15.70:0/739536314 >> 172.21.15.70:0/739536314 conn(0x7f8850069850 msgr2=0x7f8850069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 -- 172.21.15.70:0/739536314 shutdown_connections 2023-08-25T09:40:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.122+0000 7f8856c21640 1 -- 172.21.15.70:0/739536314 wait complete. 2023-08-25T09:40:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.124+0000 7f8856c21640 1 Processor -- start 2023-08-25T09:40:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.124+0000 7f8856c21640 1 -- start start 2023-08-25T09:40:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f8856c21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 0x7f88501535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f8856c21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500cea30 0x7f885014c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f8856c21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f885015cb40 0x7f885014d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f8856c21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885014ce30 con 0x7f88500ac870 2023-08-25T09:40:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f8856c21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88501548c0 con 0x7f88500cea30 2023-08-25T09:40:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f8856c21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885014b6b0 con 0x7f885015cb40 2023-08-25T09:40:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f88549bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 0x7f88501535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f88549bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 0x7f88501535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48314/0 (socket says 172.21.15.70:48314) 2023-08-25T09:40:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.125+0000 7f88549bd640 1 -- 172.21.15.70:0/2226662774 learned_addr learned my addr 172.21.15.70:0/2226662774 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f88549bd640 1 -- 172.21.15.70:0/2226662774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f885015cb40 msgr2=0x7f885014d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f88549bd640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f885015cb40 0x7f885014d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f88549bd640 1 -- 172.21.15.70:0/2226662774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500cea30 msgr2=0x7f885014c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f88549bd640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500cea30 0x7f885014c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f88549bd640 1 -- 172.21.15.70:0/2226662774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8840009b30 con 0x7f88500ac870 2023-08-25T09:40:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f88549bd640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 0x7f88501535d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f883800b410 tx=0x7f883800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838011830 con 0x7f88500ac870 2023-08-25T09:40:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8838011990 con 0x7f88500ac870 2023-08-25T09:40:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838011d40 con 0x7f88500ac870 2023-08-25T09:40:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.126+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8850153cb0 con 0x7f88500ac870 2023-08-25T09:40:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.127+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8850154270 con 0x7f88500ac870 2023-08-25T09:40:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.128+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885015a410 con 0x7f88500ac870 2023-08-25T09:40:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.128+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f883801a400 con 0x7f88500ac870 2023-08-25T09:40:54.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.129+0000 7f884d7fa640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f883006a050 0x7f883006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.129+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f88380887f0 con 0x7f88500ac870 2023-08-25T09:40:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.130+0000 7f884ffff640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f883006a050 0x7f883006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.130+0000 7f884ffff640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f883006a050 0x7f883006c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8840035000 tx=0x7f884002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:54.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.133+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8838053e90 con 0x7f88500ac870 2023-08-25T09:40:54.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.360+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f885004e420 con 0x7f88500ac870 2023-08-25T09:40:54.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.361+0000 7f884d7fa640 1 -- 172.21.15.70:0/2226662774 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f88380574b0 con 0x7f88500ac870 2023-08-25T09:40:54.363 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:54.363 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:54.363 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:54.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f883006a050 msgr2=0x7f883006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f883006a050 0x7f883006c510 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8840035000 tx=0x7f884002e000 comp rx=0 tx=0).stop 2023-08-25T09:40:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 msgr2=0x7f88501535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 0x7f88501535d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f883800b410 tx=0x7f883800b8d0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 shutdown_connections 2023-08-25T09:40:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f883006a050 0x7f883006c510 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f885015cb40 0x7f885014d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88500cea30 0x7f885014c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 --2- 172.21.15.70:0/2226662774 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88500ac870 0x7f88501535d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 >> 172.21.15.70:0/2226662774 conn(0x7f8850069850 msgr2=0x7f8850069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 shutdown_connections 2023-08-25T09:40:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.364+0000 7f8856c21640 1 -- 172.21.15.70:0/2226662774 wait complete. 2023-08-25T09:40:54.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:54.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.588+0000 7fde349cb640 1 Processor -- start 2023-08-25T09:40:54.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.589+0000 7fde349cb640 1 -- start start 2023-08-25T09:40:54.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.590+0000 7fde349cb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde300b12f0 0x7fde3015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.590+0000 7fde349cb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde3015f190 0x7fde300cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.590+0000 7fde349cb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3015e7c0 con 0x7fde3015a7c0 2023-08-25T09:40:54.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.590+0000 7fde349cb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde30071b50 con 0x7fde3015f190 2023-08-25T09:40:54.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.590+0000 7fde349cb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3015e490 con 0x7fde300b12f0 2023-08-25T09:40:54.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.591+0000 7fde2ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde3015f190 0x7fde300cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.591+0000 7fde2ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fde3015f190 0x7fde300cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:49994/0 (socket says 172.21.15.70:49994) 2023-08-25T09:40:54.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.591+0000 7fde2ffff640 1 -- 172.21.15.70:0/2196649875 learned_addr learned my addr 172.21.15.70:0/2196649875 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:54.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.591+0000 7fde27fff640 1 --2- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde300b12f0 0x7fde3015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.592+0000 7fde2ffff640 1 -- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde300b12f0 msgr2=0x7fde3015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:54.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.592+0000 7fde2ffff640 1 --2- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde300b12f0 0x7fde3015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.592+0000 7fde2ffff640 1 -- 172.21.15.70:0/2196649875 >> v1:172.21.15.155:6789/0 conn(0x7fde3015a7c0 legacy=0x7fde300b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.592+0000 7fde2ffff640 1 -- 172.21.15.70:0/2196649875 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300b3d60 con 0x7fde3015f190 2023-08-25T09:40:54.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.593+0000 7fde2ffff640 1 --2- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde300cad20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde28009e50 tx=0x7fde2802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fa2b9b22a10c9ac server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:54.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.594+0000 7fde2effd640 1 -- 172.21.15.70:0/2196649875 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2802caf0 con 0x7fde3015f190 2023-08-25T09:40:54.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.594+0000 7fde2effd640 1 -- 172.21.15.70:0/2196649875 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2802cc50 con 0x7fde3015f190 2023-08-25T09:40:54.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.594+0000 7fde2effd640 1 -- 172.21.15.70:0/2196649875 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde280343f0 con 0x7fde3015f190 2023-08-25T09:40:54.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.594+0000 7fde349cb640 1 -- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 msgr2=0x7fde300cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.594+0000 7fde349cb640 1 --2- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde300cad20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde28009e50 tx=0x7fde2802bf50 comp rx=0 tx=0).stop 2023-08-25T09:40:54.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.595+0000 7fde349cb640 1 -- 172.21.15.70:0/2196649875 shutdown_connections 2023-08-25T09:40:54.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.595+0000 7fde349cb640 1 --2- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde300cad20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.595+0000 7fde349cb640 1 --2- 172.21.15.70:0/2196649875 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde300b12f0 0x7fde3015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.595+0000 7fde349cb640 1 -- 172.21.15.70:0/2196649875 >> 172.21.15.70:0/2196649875 conn(0x7fde300affd0 msgr2=0x7fde300b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:54.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.595+0000 7fde349cb640 1 -- 172.21.15.70:0/2196649875 shutdown_connections 2023-08-25T09:40:54.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.595+0000 7fde349cb640 1 -- 172.21.15.70:0/2196649875 wait complete. 2023-08-25T09:40:54.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.596+0000 7fde349cb640 1 Processor -- start 2023-08-25T09:40:54.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.597+0000 7fde349cb640 1 -- start start 2023-08-25T09:40:54.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.597+0000 7fde349cb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 0x7fde300c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde349cb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015a7c0 0x7fde300c34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde349cb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde3006e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde349cb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c30b0 con 0x7fde300b12f0 2023-08-25T09:40:54.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde349cb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c2320 con 0x7fde3015f190 2023-08-25T09:40:54.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde349cb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c8560 con 0x7fde3015a7c0 2023-08-25T09:40:54.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 0x7fde300c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 0x7fde300c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48324/0 (socket says 172.21.15.70:48324) 2023-08-25T09:40:54.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 -- 172.21.15.70:0/2657287518 learned_addr learned my addr 172.21.15.70:0/2657287518 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:54.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2ffff640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde3006e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde27fff640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015a7c0 0x7fde300c34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 -- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015a7c0 msgr2=0x7fde300c34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015a7c0 0x7fde300c34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 -- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 msgr2=0x7fde3006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde3006e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.598+0000 7fde2f7fe640 1 -- 172.21.15.70:0/2657287518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde28009b30 con 0x7fde300b12f0 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.599+0000 7fde2f7fe640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 0x7fde300c5570 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c430 tx=0x7fde2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.599+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde200052d0 con 0x7fde300b12f0 2023-08-25T09:40:54.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.599+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde20005450 con 0x7fde300b12f0 2023-08-25T09:40:54.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.599+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde20007260 con 0x7fde300b12f0 2023-08-25T09:40:54.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.599+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde300c5ab0 con 0x7fde300b12f0 2023-08-25T09:40:54.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.599+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde300c29f0 con 0x7fde300b12f0 2023-08-25T09:40:54.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.600+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde300c31f0 con 0x7fde300b12f0 2023-08-25T09:40:54.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.601+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fde200073c0 con 0x7fde300b12f0 2023-08-25T09:40:54.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.602+0000 7fde2cff9640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1006a050 0x7fde1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:54.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.602+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fde20087a30 con 0x7fde300b12f0 2023-08-25T09:40:54.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.603+0000 7fde27fff640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1006a050 0x7fde1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:54.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.603+0000 7fde27fff640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1006a050 0x7fde1006c510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fde300c3d70 tx=0x7fde18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:54.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.607+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fde300c31f0 con 0x7fde300b12f0 2023-08-25T09:40:54.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.789+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde300c31f0 con 0x7fde300b12f0 2023-08-25T09:40:54.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.791+0000 7fde2cff9640 1 -- 172.21.15.70:0/2657287518 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18401 (secure 0 0 0) 0x7fde300c31f0 con 0x7fde300b12f0 2023-08-25T09:40:54.794 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:54.794 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:44.272862+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1006a050 msgr2=0x7fde1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1006a050 0x7fde1006c510 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fde300c3d70 tx=0x7fde18008040 comp rx=0 tx=0).stop 2023-08-25T09:40:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 msgr2=0x7fde300c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 0x7fde300c5570 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c430 tx=0x7fde2000c8f0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 shutdown_connections 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1006a050 0x7fde1006c510 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3015f190 0x7fde3006e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015a7c0 0x7fde300c34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 --2- 172.21.15.70:0/2657287518 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde300b12f0 0x7fde300c5570 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 >> 172.21.15.70:0/2657287518 conn(0x7fde300affd0 msgr2=0x7fde300c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 shutdown_connections 2023-08-25T09:40:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:54.794+0000 7fde349cb640 1 -- 172.21.15.70:0/2657287518 wait complete. 2023-08-25T09:40:54.815 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:40:25.098622+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:40:25.912835+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:40:54.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:40:55.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.027+0000 7faf707c2640 1 Processor -- start 2023-08-25T09:40:55.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.027+0000 7faf707c2640 1 -- start start 2023-08-25T09:40:55.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf707c2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7faf6815ae50 0x7faf680c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf707c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf680cdc20 0x7faf68069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf707c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf680ce060 0x7faf68069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf707c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf680cc5f0 con 0x7faf680cdc20 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf707c2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf68070650 con 0x7faf6815ae50 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf707c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf68070930 con 0x7faf680ce060 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf6dd5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf680cdc20 0x7faf68069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf6dd5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf680cdc20 0x7faf68069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48340/0 (socket says 172.21.15.70:48340) 2023-08-25T09:40:55.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.029+0000 7faf6dd5d640 1 -- 172.21.15.70:0/2077032055 learned_addr learned my addr 172.21.15.70:0/2077032055 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:55.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6ed5f640 1 --2- 172.21.15.70:0/2077032055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf680ce060 0x7faf68069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6e55e640 1 --2- 172.21.15.70:0/2077032055 >> v2:172.21.15.155:3300/0 conn(0x7faf6815ae50 0x7faf680c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6dd5d640 1 -- 172.21.15.70:0/2077032055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf680ce060 msgr2=0x7faf68069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6dd5d640 1 --2- 172.21.15.70:0/2077032055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf680ce060 0x7faf68069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6dd5d640 1 -- 172.21.15.70:0/2077032055 >> v2:172.21.15.155:3300/0 conn(0x7faf6815ae50 msgr2=0x7faf680c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6dd5d640 1 --2- 172.21.15.70:0/2077032055 >> v2:172.21.15.155:3300/0 conn(0x7faf6815ae50 0x7faf680c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6dd5d640 1 -- 172.21.15.70:0/2077032055 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf68070c10 con 0x7faf680cdc20 2023-08-25T09:40:55.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.030+0000 7faf6e55e640 1 --2- 172.21.15.70:0/2077032055 >> v2:172.21.15.155:3300/0 conn(0x7faf6815ae50 0x7faf680c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:55.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.031+0000 7faf6dd5d640 1 --2- 172.21.15.70:0/2077032055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680cdc20 0x7faf68069850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faf680b1210 tx=0x7faf5802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af510394224285cd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.031+0000 7faf6d55c640 1 -- 172.21.15.70:0/2077032055 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58030850 con 0x7faf680cdc20 2023-08-25T09:40:55.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.031+0000 7faf6d55c640 1 -- 172.21.15.70:0/2077032055 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf580309b0 con 0x7faf680cdc20 2023-08-25T09:40:55.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.031+0000 7faf6d55c640 1 -- 172.21.15.70:0/2077032055 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58030d40 con 0x7faf680cdc20 2023-08-25T09:40:55.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 -- 172.21.15.70:0/2077032055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680cdc20 msgr2=0x7faf68069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 --2- 172.21.15.70:0/2077032055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680cdc20 0x7faf68069850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faf680b1210 tx=0x7faf5802b990 comp rx=0 tx=0).stop 2023-08-25T09:40:55.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 -- 172.21.15.70:0/2077032055 shutdown_connections 2023-08-25T09:40:55.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 --2- 172.21.15.70:0/2077032055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf680ce060 0x7faf68069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 --2- 172.21.15.70:0/2077032055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680cdc20 0x7faf68069850 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 --2- 172.21.15.70:0/2077032055 >> v2:172.21.15.155:3300/0 conn(0x7faf6815ae50 0x7faf680c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 -- 172.21.15.70:0/2077032055 >> 172.21.15.70:0/2077032055 conn(0x7faf680cb0d0 msgr2=0x7faf680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:55.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 -- 172.21.15.70:0/2077032055 shutdown_connections 2023-08-25T09:40:55.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.032+0000 7faf707c2640 1 -- 172.21.15.70:0/2077032055 wait complete. 2023-08-25T09:40:55.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.034+0000 7faf707c2640 1 Processor -- start 2023-08-25T09:40:55.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.034+0000 7faf707c2640 1 -- start start 2023-08-25T09:40:55.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf707c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf680cdc20 0x7faf681f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf707c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 0x7faf68155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf707c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf6815ae50 0x7faf6815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf707c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f64d0 con 0x7faf680ce060 2023-08-25T09:40:55.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf707c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f67b0 con 0x7faf680cdc20 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf707c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf681f6a90 con 0x7faf6815ae50 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf6dd5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 0x7faf68155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf6dd5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 0x7faf68155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48350/0 (socket says 172.21.15.70:48350) 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.035+0000 7faf6dd5d640 1 -- 172.21.15.70:0/3862402265 learned_addr learned my addr 172.21.15.70:0/3862402265 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6e55e640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf680cdc20 0x7faf681f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6ed5f640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf6815ae50 0x7faf6815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6dd5d640 1 -- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf6815ae50 msgr2=0x7faf6815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6dd5d640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf6815ae50 0x7faf6815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6dd5d640 1 -- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf680cdc20 msgr2=0x7faf681f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6dd5d640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf680cdc20 0x7faf681f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6dd5d640 1 -- 172.21.15.70:0/3862402265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580094d0 con 0x7faf680ce060 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf6dd5d640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 0x7faf68155920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faf5802b9d0 tx=0x7faf58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf58039d60 con 0x7faf680ce060 2023-08-25T09:40:55.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf58043400 con 0x7faf680ce060 2023-08-25T09:40:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf580437b0 con 0x7faf680ce060 2023-08-25T09:40:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.036+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6815b3b0 con 0x7faf680ce060 2023-08-25T09:40:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.037+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf681f61e0 con 0x7faf680ce060 2023-08-25T09:40:55.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.038+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6815b8d0 con 0x7faf680ce060 2023-08-25T09:40:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.038+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf5802e070 con 0x7faf680ce060 2023-08-25T09:40:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.039+0000 7faf5effd640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf4c06a050 0x7faf4c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.039+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7faf5803e070 con 0x7faf680ce060 2023-08-25T09:40:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.040+0000 7faf6e55e640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf4c06a050 0x7faf4c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.040+0000 7faf6e55e640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf4c06a050 0x7faf4c06c510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf6815df60 tx=0x7faf64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.044+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf6815b8d0 con 0x7faf680ce060 2023-08-25T09:40:55.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.235+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf6815b8d0 con 0x7faf680ce060 2023-08-25T09:40:55.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.236+0000 7faf5effd640 1 -- 172.21.15.70:0/3862402265 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf6815b8d0 con 0x7faf680ce060 2023-08-25T09:40:55.238 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:40:55.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf4c06a050 msgr2=0x7faf4c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf4c06a050 0x7faf4c06c510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7faf6815df60 tx=0x7faf64008040 comp rx=0 tx=0).stop 2023-08-25T09:40:55.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 msgr2=0x7faf68155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 0x7faf68155920 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faf5802b9d0 tx=0x7faf58004730 comp rx=0 tx=0).stop 2023-08-25T09:40:55.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 shutdown_connections 2023-08-25T09:40:55.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf4c06a050 0x7faf4c06c510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf6815ae50 0x7faf6815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf680ce060 0x7faf68155920 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 --2- 172.21.15.70:0/3862402265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf680cdc20 0x7faf681f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 >> 172.21.15.70:0/3862402265 conn(0x7faf680cb0d0 msgr2=0x7faf68159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:55.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 shutdown_connections 2023-08-25T09:40:55.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.239+0000 7faf707c2640 1 -- 172.21.15.70:0/3862402265 wait complete. 2023-08-25T09:40:55.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:40:55.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.477+0000 7ff7c98e0640 1 Processor -- start 2023-08-25T09:40:55.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.478+0000 7ff7c98e0640 1 -- start start 2023-08-25T09:40:55.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c98e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7c40b0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c98e0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff7c40ab100 0x7ff7c40ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c98e0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7c40c9b10 con 0x7ff7c415cca0 2023-08-25T09:40:55.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c98e0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7c40cd7b0 con 0x7ff7c40ab100 2023-08-25T09:40:55.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c98e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40b0d90 con 0x7ff7c40b0410 2023-08-25T09:40:55.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c88de640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff7c415cca0 0x7ff7c40cbf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40988/0 (socket says 172.21.15.70:40988) 2023-08-25T09:40:55.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.479+0000 7ff7c88de640 1 -- 172.21.15.70:0/4102779280 learned_addr learned my addr 172.21.15.70:0/4102779280 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:55.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796620484 0 0) 0x7ff7c40c9b10 con 0x7ff7c415cca0 2023-08-25T09:40:55.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7b4003520 con 0x7ff7c415cca0 2023-08-25T09:40:55.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c90df640 1 --2- 172.21.15.70:0/4102779280 >> v2:172.21.15.155:3300/0 conn(0x7ff7c40ab100 0x7ff7c40ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c3fff640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7c40b0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175716543 0 0) 0x7ff7b4003520 con 0x7ff7c415cca0 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 msgr2=0x7ff7c40b0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7c40b0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 >> v2:172.21.15.155:3300/0 conn(0x7ff7c40ab100 msgr2=0x7ff7c40ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 --2- 172.21.15.70:0/4102779280 >> v2:172.21.15.155:3300/0 conn(0x7ff7c40ab100 0x7ff7c40ab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c40ce040 con 0x7ff7c415cca0 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff7b80017f0 con 0x7ff7c415cca0 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 >> v1:172.21.15.70:6789/0 conn(0x7ff7c415cca0 legacy=0x7ff7c40cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b40047c0 0x7ff7b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40cd7b0 con 0x7ff7b4003970 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40b0d90 con 0x7ff7b40047c0 2023-08-25T09:40:55.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.480+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b400bac0 con 0x7ff7c40b0410 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c88de640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c3fff640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b40047c0 0x7ff7b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c90df640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c88de640 1 -- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 msgr2=0x7ff7b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c88de640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7b4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c88de640 1 -- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b40047c0 msgr2=0x7ff7b4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c88de640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7b40047c0 0x7ff7b4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c88de640 1 -- 172.21.15.70:0/4102779280 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b4003520 con 0x7ff7b4003970 2023-08-25T09:40:55.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.481+0000 7ff7c90df640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40b0410 0x7ff7b4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:55.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.482+0000 7ff7c88de640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7b4003d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8001eb0 tx=0x7ff7b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e1cee2ee244996 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.482+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8007270 con 0x7ff7b4003970 2023-08-25T09:40:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.482+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b80073f0 con 0x7ff7b4003970 2023-08-25T09:40:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c37fe640 1 -- 172.21.15.70:0/4102779280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b802c870 con 0x7ff7b4003970 2023-08-25T09:40:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c98e0640 1 -- 172.21.15.70:0/4102779280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 msgr2=0x7ff7b4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/4102779280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7b4003d90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8001eb0 tx=0x7ff7b40057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c90df640 1 -- 172.21.15.70:0/4102779280 reap_dead start 2023-08-25T09:40:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c98e0640 1 -- 172.21.15.70:0/4102779280 shutdown_connections 2023-08-25T09:40:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c98e0640 1 -- 172.21.15.70:0/4102779280 >> 172.21.15.70:0/4102779280 conn(0x7ff7c40ce830 msgr2=0x7ff7c40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c98e0640 1 -- 172.21.15.70:0/4102779280 shutdown_connections 2023-08-25T09:40:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.483+0000 7ff7c98e0640 1 -- 172.21.15.70:0/4102779280 wait complete. 2023-08-25T09:40:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.485+0000 7ff7c98e0640 1 Processor -- start 2023-08-25T09:40:55.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.485+0000 7ff7c98e0640 1 -- start start 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c98e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7c415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c98e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40af380 0x7ff7c40af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c98e0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415d080 0x7ff7c415d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c98e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c406b010 con 0x7ff7b4003970 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c98e0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c40afd40 con 0x7ff7c415d080 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c98e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c415da40 con 0x7ff7c40af380 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.486+0000 7ff7c88de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7c415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7c415b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48380/0 (socket says 172.21.15.70:48380) 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 -- 172.21.15.70:0/3682212619 learned_addr learned my addr 172.21.15.70:0/3682212619 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c90df640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415d080 0x7ff7c415d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 -- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40af380 msgr2=0x7ff7c40af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40af380 0x7ff7c40af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 -- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415d080 msgr2=0x7ff7c415d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415d080 0x7ff7c415d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 -- 172.21.15.70:0/3682212619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c415dcd0 con 0x7ff7b4003970 2023-08-25T09:40:55.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c88de640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7c415b190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7b802c410 tx=0x7ff7c40ab050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.487+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b803c040 con 0x7ff7b4003970 2023-08-25T09:40:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.488+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b8007280 con 0x7ff7b4003970 2023-08-25T09:40:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.488+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c415dfb0 con 0x7ff7b4003970 2023-08-25T09:40:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.488+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b80059e0 con 0x7ff7b4003970 2023-08-25T09:40:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.488+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c415e570 con 0x7ff7b4003970 2023-08-25T09:40:55.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.489+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c4061460 con 0x7ff7b4003970 2023-08-25T09:40:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.490+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff7b8032070 con 0x7ff7b4003970 2023-08-25T09:40:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.490+0000 7ff7c17fa640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff7a806a080 0x7ff7a806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.491+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff7b80a96e0 con 0x7ff7b4003970 2023-08-25T09:40:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.491+0000 7ff7c3fff640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff7a806a080 0x7ff7a806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.492+0000 7ff7c3fff640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff7a806a080 0x7ff7a806c540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40ad700 tx=0x7ff7b000b2e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.495+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff7b8074d80 con 0x7ff7b4003970 2023-08-25T09:40:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.682+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff7c40cd130 con 0x7ff7b4003970 2023-08-25T09:40:55.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.683+0000 7ff7c17fa640 1 -- 172.21.15.70:0/3682212619 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff7b80017b0 con 0x7ff7b4003970 2023-08-25T09:40:55.684 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:40:55.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff7a806a080 msgr2=0x7ff7a806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff7a806a080 0x7ff7a806c540 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40ad700 tx=0x7ff7b000b2e0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 msgr2=0x7ff7c415b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7c415b190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff7b802c410 tx=0x7ff7c40ab050 comp rx=0 tx=0).stop 2023-08-25T09:40:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 shutdown_connections 2023-08-25T09:40:55.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff7a806a080 0x7ff7a806c540 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7c415d080 0x7ff7c415d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7c40af380 0x7ff7c40af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 --2- 172.21.15.70:0/3682212619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff7b4003970 0x7ff7c415b190 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 >> 172.21.15.70:0/3682212619 conn(0x7ff7c40ce830 msgr2=0x7ff7c4070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 shutdown_connections 2023-08-25T09:40:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.686+0000 7ff7c98e0640 1 -- 172.21.15.70:0/3682212619 wait complete. 2023-08-25T09:40:55.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:40:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.915+0000 7f7f76e91640 1 Processor -- start 2023-08-25T09:40:55.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.915+0000 7f7f76e91640 1 -- start start 2023-08-25T09:40:55.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f76e91640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f700b2470 0x7f7f70159930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f76e91640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f700c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f76e91640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f70159e70 con 0x7f7f700b2850 2023-08-25T09:40:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f76e91640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7015ad20 con 0x7f7f700b2470 2023-08-25T09:40:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f76e91640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700cd800 con 0x7f7f700c8d20 2023-08-25T09:40:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f7568e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7f700b2850 0x7f7f700cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:40992/0 (socket says 172.21.15.70:40992) 2023-08-25T09:40:55.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.917+0000 7f7f7568e640 1 -- 172.21.15.70:0/1911035732 learned_addr learned my addr 172.21.15.70:0/1911035732 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f76690640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f700c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f75e8f640 1 --2- 172.21.15.70:0/1911035732 >> v2:172.21.15.155:3300/0 conn(0x7f7f700b2470 0x7f7f70159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 442272182 0 0) 0x7f7f70159e70 con 0x7f7f700b2850 2023-08-25T09:40:55.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f5c003420 con 0x7f7f700b2850 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3552102270 0 0) 0x7f7f5c003420 con 0x7f7f700b2850 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 msgr2=0x7f7f700c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f700c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 >> v2:172.21.15.155:3300/0 conn(0x7f7f700b2470 msgr2=0x7f7f70159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 --2- 172.21.15.70:0/1911035732 >> v2:172.21.15.155:3300/0 conn(0x7f7f700b2470 0x7f7f70159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7015a110 con 0x7f7f700b2850 2023-08-25T09:40:55.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7f60001670 con 0x7f7f700b2850 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 >> v1:172.21.15.70:6789/0 conn(0x7f7f700b2850 legacy=0x7f7f700cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c004a50 0x7f7f5c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f5c003660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5c003ba0 0x7f7f5c004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7015ad20 con 0x7f7f5c003ba0 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700cd800 con 0x7f7f5c004a50 2023-08-25T09:40:55.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.918+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c00fd00 con 0x7f7f700c8d20 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f76690640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5c003ba0 0x7f7f5c004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f75e8f640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c004a50 0x7f7f5c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f7568e640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f5c003660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f76690640 1 -- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 msgr2=0x7f7f5c003660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f76690640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f5c003660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f76690640 1 -- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c004a50 msgr2=0x7f7f5c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:55.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f76690640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c004a50 0x7f7f5c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f76690640 1 -- 172.21.15.70:0/1911035732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c003420 con 0x7f7f5c003ba0 2023-08-25T09:40:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f7568e640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f700c8d20 0x7f7f5c003660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:40:55.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.919+0000 7f7f75e8f640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f5c004a50 0x7f7f5c004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:40:55.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.921+0000 7f7f76690640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5c003ba0 0x7f7f5c004020 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009ac0 tx=0x7f7f5c0046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3985f1582b38cb63 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.921+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6802c6b0 con 0x7f7f5c003ba0 2023-08-25T09:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.921+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6802c810 con 0x7f7f5c003ba0 2023-08-25T09:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.921+0000 7f7f74e8d640 1 -- 172.21.15.70:0/1911035732 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6802cba0 con 0x7f7f5c003ba0 2023-08-25T09:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.921+0000 7f7f76e91640 1 -- 172.21.15.70:0/1911035732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5c003ba0 msgr2=0x7f7f5c004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.922+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1911035732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f5c003ba0 0x7f7f5c004020 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009ac0 tx=0x7f7f5c0046e0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.922+0000 7f7f76690640 1 -- 172.21.15.70:0/1911035732 reap_dead start 2023-08-25T09:40:55.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.922+0000 7f7f76e91640 1 -- 172.21.15.70:0/1911035732 shutdown_connections 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.922+0000 7f7f76e91640 1 -- 172.21.15.70:0/1911035732 >> 172.21.15.70:0/1911035732 conn(0x7f7f700ce830 msgr2=0x7f7f700cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.922+0000 7f7f76e91640 1 -- 172.21.15.70:0/1911035732 shutdown_connections 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.922+0000 7f7f76e91640 1 -- 172.21.15.70:0/1911035732 wait complete. 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.924+0000 7f7f76e91640 1 Processor -- start 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.924+0000 7f7f76e91640 1 -- start start 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f76e91640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f5c003ba0 0x7f7f701edb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f76e91640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 0x7f7f70099dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f76e91640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f7009a300 0x7f7f700ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f76e91640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70098a40 con 0x7f7f700c5450 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f76e91640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700c58d0 con 0x7f7f7009a300 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f76e91640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70069ba0 con 0x7f7f5c003ba0 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 0x7f7f70099dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 0x7f7f70099dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48402/0 (socket says 172.21.15.70:48402) 2023-08-25T09:40:55.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 -- 172.21.15.70:0/1164095120 learned_addr learned my addr 172.21.15.70:0/1164095120 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f75e8f640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f5c003ba0 0x7f7f701edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 -- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f5c003ba0 msgr2=0x7f7f701edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f5c003ba0 0x7f7f701edb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 -- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f7009a300 msgr2=0x7f7f700ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f7009a300 0x7f7f700ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.925+0000 7f7f7568e640 1 -- 172.21.15.70:0/1164095120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f70069e80 con 0x7f7f700c5450 2023-08-25T09:40:55.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.926+0000 7f7f7568e640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 0x7f7f70099dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000eaf0 tx=0x7f7f60005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.926+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f600049f0 con 0x7f7f700c5450 2023-08-25T09:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.926+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60004b50 con 0x7f7f700c5450 2023-08-25T09:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.926+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60005050 con 0x7f7f700c5450 2023-08-25T09:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.926+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7006a160 con 0x7f7f700c5450 2023-08-25T09:40:55.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.926+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f70069850 con 0x7f7f700c5450 2023-08-25T09:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.927+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f70061460 con 0x7f7f700c5450 2023-08-25T09:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.928+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7f600051b0 con 0x7f7f700c5450 2023-08-25T09:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.929+0000 7f7f667fc640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f5406a050 0x7f7f5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.929+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7f60088100 con 0x7f7f700c5450 2023-08-25T09:40:55.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.930+0000 7f7f75e8f640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f5406a050 0x7f7f5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:55.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.930+0000 7f7f75e8f640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f5406a050 0x7f7f5406c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009fd0 tx=0x7f7f6c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:55.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:55.933+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f600537c0 con 0x7f7f700c5450 2023-08-25T09:40:56.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.126+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f70071d30 con 0x7f7f700c5450 2023-08-25T09:40:56.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.127+0000 7f7f667fc640 1 -- 172.21.15.70:0/1164095120 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f60056de0 con 0x7f7f700c5450 2023-08-25T09:40:56.129 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:40:56.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f5406a050 msgr2=0x7f7f5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f5406a050 0x7f7f5406c510 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009fd0 tx=0x7f7f6c00b040 comp rx=0 tx=0).stop 2023-08-25T09:40:56.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 msgr2=0x7f7f70099dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 0x7f7f70099dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000eaf0 tx=0x7f7f60005e50 comp rx=0 tx=0).stop 2023-08-25T09:40:56.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 shutdown_connections 2023-08-25T09:40:56.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f5406a050 0x7f7f5406c510 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f7009a300 0x7f7f700ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f700c5450 0x7f7f70099dc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 --2- 172.21.15.70:0/1164095120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f5c003ba0 0x7f7f701edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 >> 172.21.15.70:0/1164095120 conn(0x7f7f700ce830 msgr2=0x7f7f70153510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:56.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.130+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 shutdown_connections 2023-08-25T09:40:56.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.131+0000 7f7f76e91640 1 -- 172.21.15.70:0/1164095120 wait complete. 2023-08-25T09:40:56.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.367+0000 7fc4ec196640 1 Processor -- start 2023-08-25T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.367+0000 7fc4ec196640 1 -- start start 2023-08-25T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4ec196640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4e406a110 0x7fc4e40ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4ec196640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc4e406c510 0x7fc4e406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4ec196640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40cd4e0 con 0x7fc4e4069d30 2023-08-25T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4ec196640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40cc550 con 0x7fc4e406a110 2023-08-25T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4ec196640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40ccb10 con 0x7fc4e406c510 2023-08-25T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4e9731640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4e406a110 0x7fc4e40ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4e9731640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc4e406a110 0x7fc4e40ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48412/0 (socket says 172.21.15.70:48412) 2023-08-25T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.369+0000 7fc4e9731640 1 -- 172.21.15.70:0/2576651349 learned_addr learned my addr 172.21.15.70:0/2576651349 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.370+0000 7fc4e9731640 1 -- 172.21.15.70:0/2576651349 >> v2:172.21.15.155:3300/0 conn(0x7fc4e406c510 msgr2=0x7fc4e406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.370+0000 7fc4e9731640 1 --2- 172.21.15.70:0/2576651349 >> v2:172.21.15.155:3300/0 conn(0x7fc4e406c510 0x7fc4e406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.370+0000 7fc4e9731640 1 -- 172.21.15.70:0/2576651349 >> v1:172.21.15.70:6789/0 conn(0x7fc4e4069d30 legacy=0x7fc4e40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.370+0000 7fc4e9731640 1 -- 172.21.15.70:0/2576651349 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40ade40 con 0x7fc4e406a110 2023-08-25T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.371+0000 7fc4e9731640 1 --2- 172.21.15.70:0/2576651349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e406a110 0x7fc4e40ccfa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009870 tx=0x7fc4d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e46bfecd4dcd84 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4e8f30640 1 -- 172.21.15.70:0/2576651349 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4031870 con 0x7fc4e406a110 2023-08-25T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4e8f30640 1 -- 172.21.15.70:0/2576651349 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d40319d0 con 0x7fc4e406a110 2023-08-25T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4e8f30640 1 -- 172.21.15.70:0/2576651349 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4031d60 con 0x7fc4e406a110 2023-08-25T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 -- 172.21.15.70:0/2576651349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e406a110 msgr2=0x7fc4e40ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 --2- 172.21.15.70:0/2576651349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e406a110 0x7fc4e40ccfa0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009870 tx=0x7fc4d402b910 comp rx=0 tx=0).stop 2023-08-25T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 -- 172.21.15.70:0/2576651349 shutdown_connections 2023-08-25T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 --2- 172.21.15.70:0/2576651349 >> v2:172.21.15.155:3300/0 conn(0x7fc4e406c510 0x7fc4e406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 --2- 172.21.15.70:0/2576651349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e406a110 0x7fc4e40ccfa0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 -- 172.21.15.70:0/2576651349 >> 172.21.15.70:0/2576651349 conn(0x7fc4e40cb0d0 msgr2=0x7fc4e40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 -- 172.21.15.70:0/2576651349 shutdown_connections 2023-08-25T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.372+0000 7fc4ec196640 1 -- 172.21.15.70:0/2576651349 wait complete. 2023-08-25T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.374+0000 7fc4ec196640 1 Processor -- start 2023-08-25T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.374+0000 7fc4ec196640 1 -- start start 2023-08-25T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4ec196640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 0x7fc4e40c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4ec196640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e406a110 0x7fc4e406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4ec196640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e406c510 0x7fc4e4070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4ec196640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c65c0 con 0x7fc4e4069d30 2023-08-25T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4ec196640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c6f80 con 0x7fc4e406c510 2023-08-25T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4ec196640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c7260 con 0x7fc4e406a110 2023-08-25T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.375+0000 7fc4e9f32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 0x7fc4e40c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 0x7fc4e40c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48426/0 (socket says 172.21.15.70:48426) 2023-08-25T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 -- 172.21.15.70:0/883329129 learned_addr learned my addr 172.21.15.70:0/883329129 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 -- 172.21.15.70:0/883329129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e406a110 msgr2=0x7fc4e406fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e406a110 0x7fc4e406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 -- 172.21.15.70:0/883329129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e406c510 msgr2=0x7fc4e4070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e406c510 0x7fc4e4070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 -- 172.21.15.70:0/883329129 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d40094d0 con 0x7fc4e4069d30 2023-08-25T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4e9f32640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 0x7fc4e40c3750 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c730 tx=0x7fc4e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.376+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0003b70 con 0x7fc4e4069d30 2023-08-25T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.377+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e0003cf0 con 0x7fc4e4069d30 2023-08-25T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.377+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0003ea0 con 0x7fc4e4069d30 2023-08-25T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.377+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40c7540 con 0x7fc4e4069d30 2023-08-25T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.377+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e40c7d40 con 0x7fc4e4069d30 2023-08-25T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.378+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e4066fb0 con 0x7fc4e4069d30 2023-08-25T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.379+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc4e000aa90 con 0x7fc4e4069d30 2023-08-25T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.379+0000 7fc4da7fc640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc4c806a050 0x7fc4c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.380+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc4e0088330 con 0x7fc4e4069d30 2023-08-25T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.380+0000 7fc4e9731640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc4c806a050 0x7fc4c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.381+0000 7fc4e9731640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc4c806a050 0x7fc4c806c510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4036000 tx=0x7fc4d4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.384+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc4e0053940 con 0x7fc4e4069d30 2023-08-25T09:40:56.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.574+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4e40cd4e0 con 0x7fc4e4069d30 2023-08-25T09:40:56.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.575+0000 7fc4da7fc640 1 -- 172.21.15.70:0/883329129 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4e40cd4e0 con 0x7fc4e4069d30 2023-08-25T09:40:56.577 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:40:56.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.577+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc4c806a050 msgr2=0x7fc4c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.577+0000 7fc4ec196640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc4c806a050 0x7fc4c806c510 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4036000 tx=0x7fc4d4030000 comp rx=0 tx=0).stop 2023-08-25T09:40:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.577+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 msgr2=0x7fc4e40c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.577+0000 7fc4ec196640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 0x7fc4e40c3750 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000c730 tx=0x7fc4e000cbf0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 shutdown_connections 2023-08-25T09:40:56.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc4c806a050 0x7fc4c806c510 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc4e406c510 0x7fc4e4070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc4e406a110 0x7fc4e406fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 --2- 172.21.15.70:0/883329129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc4e4069d30 0x7fc4e40c3750 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 >> 172.21.15.70:0/883329129 conn(0x7fc4e40cb0d0 msgr2=0x7fc4e40c9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 shutdown_connections 2023-08-25T09:40:56.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.578+0000 7fc4ec196640 1 -- 172.21.15.70:0/883329129 wait complete. 2023-08-25T09:40:56.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:56.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.812+0000 7f826759e640 1 Processor -- start 2023-08-25T09:40:56.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.813+0000 7f826759e640 1 -- start start 2023-08-25T09:40:56.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.814+0000 7f826759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82680b12f0 0x7f826815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.814+0000 7f826759e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f826815f190 0x7f82680cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.814+0000 7f826759e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826815e7c0 con 0x7f826815a7c0 2023-08-25T09:40:56.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.814+0000 7f826759e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8268071b50 con 0x7f826815f190 2023-08-25T09:40:56.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.814+0000 7f826759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826815e490 con 0x7f82680b12f0 2023-08-25T09:40:56.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f826815f190 0x7f82680cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f826815f190 0x7f82680cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48440/0 (socket says 172.21.15.70:48440) 2023-08-25T09:40:56.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 -- 172.21.15.70:0/4150320696 learned_addr learned my addr 172.21.15.70:0/4150320696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:56.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f825ffff640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82680b12f0 0x7f826815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 -- 172.21.15.70:0/4150320696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82680b12f0 msgr2=0x7f826815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:56.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82680b12f0 0x7f826815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 -- 172.21.15.70:0/4150320696 >> v1:172.21.15.155:6789/0 conn(0x7f826815a7c0 legacy=0x7f82680b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:56.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f8266d9d640 1 -- 172.21.15.70:0/4150320696 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82680b3d60 con 0x7f826815f190 2023-08-25T09:40:56.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.815+0000 7f825ffff640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82680b12f0 0x7f826815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:56.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.816+0000 7f8266d9d640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680cad20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8260009e50 tx=0x7f826002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b30f9d3ac0d876 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.816+0000 7f8265d9b640 1 -- 172.21.15.70:0/4150320696 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826002ca80 con 0x7f826815f190 2023-08-25T09:40:56.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f8265d9b640 1 -- 172.21.15.70:0/4150320696 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826002cbe0 con 0x7f826815f190 2023-08-25T09:40:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f8265d9b640 1 -- 172.21.15.70:0/4150320696 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82600343f0 con 0x7f826815f190 2023-08-25T09:40:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 -- 172.21.15.70:0/4150320696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 msgr2=0x7f82680cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680cad20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8260009e50 tx=0x7f826002bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 -- 172.21.15.70:0/4150320696 shutdown_connections 2023-08-25T09:40:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680cad20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 --2- 172.21.15.70:0/4150320696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82680b12f0 0x7f826815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 -- 172.21.15.70:0/4150320696 >> 172.21.15.70:0/4150320696 conn(0x7f82680affd0 msgr2=0x7f82680b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:56.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 -- 172.21.15.70:0/4150320696 shutdown_connections 2023-08-25T09:40:56.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.817+0000 7f826759e640 1 -- 172.21.15.70:0/4150320696 wait complete. 2023-08-25T09:40:56.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.819+0000 7f826759e640 1 Processor -- start 2023-08-25T09:40:56.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.819+0000 7f826759e640 1 -- start start 2023-08-25T09:40:56.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.820+0000 7f826759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82680b12f0 0x7f826806e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.820+0000 7f826759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f826815a7c0 0x7f8268157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.820+0000 7f826759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.820+0000 7f826759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8268158220 con 0x7f826815f190 2023-08-25T09:40:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.820+0000 7f826759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82680c32c0 con 0x7f82680b12f0 2023-08-25T09:40:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.820+0000 7f826759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82680c6b10 con 0x7f826815a7c0 2023-08-25T09:40:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680c29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48450/0 (socket says 172.21.15.70:48450) 2023-08-25T09:40:56.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 -- 172.21.15.70:0/1340264737 learned_addr learned my addr 172.21.15.70:0/1340264737 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 -- 172.21.15.70:0/1340264737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f826815a7c0 msgr2=0x7f8268157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f826815a7c0 0x7f8268157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 -- 172.21.15.70:0/1340264737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82680b12f0 msgr2=0x7f826806e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82680b12f0 0x7f826806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 -- 172.21.15.70:0/1340264737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8260009b30 con 0x7f826815f190 2023-08-25T09:40:56.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.821+0000 7f8266d9d640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680c29e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8260036040 tx=0x7f8260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:56.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.822+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260040670 con 0x7f826815f190 2023-08-25T09:40:56.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.822+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82600407f0 con 0x7f826815f190 2023-08-25T09:40:56.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.822+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260040ba0 con 0x7f826815f190 2023-08-25T09:40:56.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.822+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82680c6df0 con 0x7f826815f190 2023-08-25T09:40:56.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.822+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82680c73b0 con 0x7f826815f190 2023-08-25T09:40:56.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.823+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8268066fb0 con 0x7f826815f190 2023-08-25T09:40:56.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.824+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f826002cd50 con 0x7f826815f190 2023-08-25T09:40:56.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.825+0000 7f825effd640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f824806a050 0x7f824806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.825+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f82600ab580 con 0x7f826815f190 2023-08-25T09:40:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.825+0000 7f826659c640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f824806a050 0x7f824806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:56.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.826+0000 7f826659c640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f824806a050 0x7f824806c510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8258003cc0 tx=0x7f82580079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:56.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:56.829+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8260076b90 con 0x7f826815f190 2023-08-25T09:40:57.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.056+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82680cf5e0 con 0x7f826815f190 2023-08-25T09:40:57.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.058+0000 7f825effd640 1 -- 172.21.15.70:0/1340264737 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f826007a1b0 con 0x7f826815f190 2023-08-25T09:40:57.059 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:57.059 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:57.060 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:57.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f824806a050 msgr2=0x7f824806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f824806a050 0x7f824806c510 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8258003cc0 tx=0x7f82580079b0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 msgr2=0x7f82680c29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680c29e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8260036040 tx=0x7f8260004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 shutdown_connections 2023-08-25T09:40:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f824806a050 0x7f824806c510 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f826815f190 0x7f82680c29e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f826815a7c0 0x7f8268157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 --2- 172.21.15.70:0/1340264737 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82680b12f0 0x7f826806e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.060+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 >> 172.21.15.70:0/1340264737 conn(0x7f82680affd0 msgr2=0x7f82680ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:57.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.061+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 shutdown_connections 2023-08-25T09:40:57.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.061+0000 7f826759e640 1 -- 172.21.15.70:0/1340264737 wait complete. 2023-08-25T09:40:57.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:57.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.297+0000 7f664eec0640 1 Processor -- start 2023-08-25T09:40:57.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.297+0000 7f664eec0640 1 -- start start 2023-08-25T09:40:57.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664eec0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 0x7f66480cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664eec0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f66480acd50 0x7f66480ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664eec0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6648160680 con 0x7f66480c5120 2023-08-25T09:40:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664eec0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6648160960 con 0x7f66480acd50 2023-08-25T09:40:57.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664eec0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6648160c40 con 0x7f664815ca00 2023-08-25T09:40:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664d6bd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f66480c5120 0x7f664806ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41008/0 (socket says 172.21.15.70:41008) 2023-08-25T09:40:57.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664d6bd640 1 -- 172.21.15.70:0/2541634595 learned_addr learned my addr 172.21.15.70:0/2541634595 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2541634595 >> v2:172.21.15.155:3300/0 conn(0x7f66480acd50 0x7f66480ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.299+0000 7f664debe640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 0x7f66480cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193533822 0 0) 0x7f6648160680 con 0x7f66480c5120 2023-08-25T09:40:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6634003520 con 0x7f66480c5120 2023-08-25T09:40:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 262070742 0 0) 0x7f6634003520 con 0x7f66480c5120 2023-08-25T09:40:57.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 msgr2=0x7f66480cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 0x7f66480cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 >> v2:172.21.15.155:3300/0 conn(0x7f66480acd50 msgr2=0x7f66480ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 --2- 172.21.15.70:0/2541634595 >> v2:172.21.15.155:3300/0 conn(0x7f66480acd50 0x7f66480ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6648160f20 con 0x7f66480c5120 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f663c001670 con 0x7f66480c5120 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 >> v1:172.21.15.70:6789/0 conn(0x7f66480c5120 legacy=0x7f664806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634003970 0x7f6634003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 0x7f66340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004d00 0x7f6634005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648160960 con 0x7f6634004d00 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648160c40 con 0x7f6634003970 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663400b980 con 0x7f664815ca00 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.300+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004d00 0x7f6634005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664debe640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634003970 0x7f6634003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664d6bd640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 0x7f66340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664e6bf640 1 -- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 msgr2=0x7f66340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:57.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f664815ca00 0x7f66340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664e6bf640 1 -- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634003970 msgr2=0x7f6634003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:57.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634003970 0x7f6634003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.301+0000 7f664e6bf640 1 -- 172.21.15.70:0/2541634595 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6634003520 con 0x7f6634004d00 2023-08-25T09:40:57.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.302+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004d00 0x7f6634005180 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f66480ad160 tx=0x7f66340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bc1270c7cf9a22 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:57.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.302+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664002cbd0 con 0x7f6634004d00 2023-08-25T09:40:57.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.302+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000f040 con 0x7f6634004d00 2023-08-25T09:40:57.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.302+0000 7f664cebc640 1 -- 172.21.15.70:0/2541634595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640034440 con 0x7f6634004d00 2023-08-25T09:40:57.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664eec0640 1 -- 172.21.15.70:0/2541634595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004d00 msgr2=0x7f6634005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664eec0640 1 --2- 172.21.15.70:0/2541634595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6634004d00 0x7f6634005180 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f66480ad160 tx=0x7f66340057f0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664e6bf640 1 -- 172.21.15.70:0/2541634595 reap_dead start 2023-08-25T09:40:57.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664eec0640 1 -- 172.21.15.70:0/2541634595 shutdown_connections 2023-08-25T09:40:57.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664eec0640 1 -- 172.21.15.70:0/2541634595 >> 172.21.15.70:0/2541634595 conn(0x7f66480affd0 msgr2=0x7f66480b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:57.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664eec0640 1 -- 172.21.15.70:0/2541634595 shutdown_connections 2023-08-25T09:40:57.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.303+0000 7f664eec0640 1 -- 172.21.15.70:0/2541634595 wait complete. 2023-08-25T09:40:57.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.305+0000 7f664eec0640 1 Processor -- start 2023-08-25T09:40:57.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.305+0000 7f664eec0640 1 -- start start 2023-08-25T09:40:57.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664eec0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634004d00 0x7f66480b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664eec0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6648159240 0x7f664815eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664eec0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 0x7f66480b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664eec0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664815f620 con 0x7f664815eff0 2023-08-25T09:40:57.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664eec0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66480b1f10 con 0x7f6634004d00 2023-08-25T09:40:57.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664eec0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66480b21f0 con 0x7f6648159240 2023-08-25T09:40:57.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664e6bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 0x7f66480b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664e6bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 0x7f66480b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48458/0 (socket says 172.21.15.70:48458) 2023-08-25T09:40:57.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.306+0000 7f664e6bf640 1 -- 172.21.15.70:0/2501972573 learned_addr learned my addr 172.21.15.70:0/2501972573 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:57.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664e6bf640 1 -- 172.21.15.70:0/2501972573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6648159240 msgr2=0x7f664815eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6648159240 0x7f664815eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664e6bf640 1 -- 172.21.15.70:0/2501972573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634004d00 msgr2=0x7f66480b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634004d00 0x7f66480b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664e6bf640 1 -- 172.21.15.70:0/2501972573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480b24d0 con 0x7f664815eff0 2023-08-25T09:40:57.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664e6bf640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 0x7f66480b33e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6640037040 tx=0x7f6640009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:57.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640043070 con 0x7f664815eff0 2023-08-25T09:40:57.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6640034b50 con 0x7f664815eff0 2023-08-25T09:40:57.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640003990 con 0x7f664815eff0 2023-08-25T09:40:57.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.307+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6648159a20 con 0x7f664815eff0 2023-08-25T09:40:57.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.308+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66480b1b70 con 0x7f664815eff0 2023-08-25T09:40:57.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.309+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66480b2050 con 0x7f664815eff0 2023-08-25T09:40:57.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.309+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f664003e440 con 0x7f664815eff0 2023-08-25T09:40:57.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.311+0000 7f663a7fc640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f662c06a050 0x7f662c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.311+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f66400a9fe0 con 0x7f664815eff0 2023-08-25T09:40:57.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.311+0000 7f664debe640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f662c06a050 0x7f662c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.311+0000 7f664debe640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f662c06a050 0x7f662c06c510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6644009910 tx=0x7f6644008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:57.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.315+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f66480b2050 con 0x7f664815eff0 2023-08-25T09:40:57.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.505+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66480b2050 con 0x7f664815eff0 2023-08-25T09:40:57.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.506+0000 7f663a7fc640 1 -- 172.21.15.70:0/2501972573 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18401 (secure 0 0 0) 0x7f66480b2050 con 0x7f664815eff0 2023-08-25T09:40:57.508 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:44.272862+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:57.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f662c06a050 msgr2=0x7f662c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f662c06a050 0x7f662c06c510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6644009910 tx=0x7f6644008040 comp rx=0 tx=0).stop 2023-08-25T09:40:57.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 msgr2=0x7f66480b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 0x7f66480b33e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6640037040 tx=0x7f6640009750 comp rx=0 tx=0).stop 2023-08-25T09:40:57.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 shutdown_connections 2023-08-25T09:40:57.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f662c06a050 0x7f662c06c510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f664815eff0 0x7f66480b33e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6648159240 0x7f664815eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 --2- 172.21.15.70:0/2501972573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6634004d00 0x7f66480b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 >> 172.21.15.70:0/2501972573 conn(0x7f66480affd0 msgr2=0x7f66480af450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:57.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 shutdown_connections 2023-08-25T09:40:57.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.509+0000 7f664eec0640 1 -- 172.21.15.70:0/2501972573 wait complete. 2023-08-25T09:40:57.529 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:40:57.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:57.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.741+0000 7f1210a14640 1 Processor -- start 2023-08-25T09:40:57.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.741+0000 7f1210a14640 1 -- start start 2023-08-25T09:40:57.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.743+0000 7f1210a14640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f120c0ce5b0 0x7f120c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.743+0000 7f1210a14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 0x7f120c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.743+0000 7f1210a14640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120c0ceed0 con 0x7f120c15aa10 2023-08-25T09:40:57.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.743+0000 7f1210a14640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f120c159d10 con 0x7f120c0ce5b0 2023-08-25T09:40:57.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.743+0000 7f1210a14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c15d3f0 con 0x7f120c0b2850 2023-08-25T09:40:57.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120ad9d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 0x7f120c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120ad9d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 0x7f120c06f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:43974/0 (socket says 172.21.15.70:43974) 2023-08-25T09:40:57.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120ad9d640 1 -- 172.21.15.70:0/3778375247 learned_addr learned my addr 172.21.15.70:0/3778375247 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:57.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120a59c640 1 --2- 172.21.15.70:0/3778375247 >> v2:172.21.15.155:3300/0 conn(0x7f120c0ce5b0 0x7f120c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120a59c640 1 -- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 msgr2=0x7f120c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:57.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120a59c640 1 --2- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 0x7f120c06f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.744+0000 7f120a59c640 1 -- 172.21.15.70:0/3778375247 >> v1:172.21.15.155:6789/0 conn(0x7f120c15aa10 legacy=0x7f120c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:57.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.745+0000 7f120a59c640 1 -- 172.21.15.70:0/3778375247 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c0b2d20 con 0x7f120c0ce5b0 2023-08-25T09:40:57.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.745+0000 7f120ad9d640 1 --2- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 0x7f120c06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:40:57.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.746+0000 7f120a59c640 1 --2- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0ce5b0 0x7f120c0ce990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f120c06f470 tx=0x7f11fc02bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d3736dc5310d2c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:57.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.746+0000 7f1209d9b640 1 -- 172.21.15.70:0/3778375247 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc02cb00 con 0x7f120c0ce5b0 2023-08-25T09:40:57.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.746+0000 7f1209d9b640 1 -- 172.21.15.70:0/3778375247 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc02cc60 con 0x7f120c0ce5b0 2023-08-25T09:40:57.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.746+0000 7f1209d9b640 1 -- 172.21.15.70:0/3778375247 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc034440 con 0x7f120c0ce5b0 2023-08-25T09:40:57.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 -- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0ce5b0 msgr2=0x7f120c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 --2- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0ce5b0 0x7f120c0ce990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f120c06f470 tx=0x7f11fc02bd50 comp rx=0 tx=0).stop 2023-08-25T09:40:57.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 -- 172.21.15.70:0/3778375247 shutdown_connections 2023-08-25T09:40:57.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 --2- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0b2850 0x7f120c06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 --2- 172.21.15.70:0/3778375247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c0ce5b0 0x7f120c0ce990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 -- 172.21.15.70:0/3778375247 >> 172.21.15.70:0/3778375247 conn(0x7f120c069850 msgr2=0x7f120c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:57.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 -- 172.21.15.70:0/3778375247 shutdown_connections 2023-08-25T09:40:57.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.747+0000 7f1210a14640 1 -- 172.21.15.70:0/3778375247 wait complete. 2023-08-25T09:40:57.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.749+0000 7f1210a14640 1 Processor -- start 2023-08-25T09:40:57.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.749+0000 7f1210a14640 1 -- start start 2023-08-25T09:40:57.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f1210a14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 0x7f120c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f1210a14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0ce5b0 0x7f120c0ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f1210a14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c15aa10 0x7f120c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f1210a14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c157a00 con 0x7f120c0b2850 2023-08-25T09:40:57.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f1210a14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0ace60 con 0x7f120c15aa10 2023-08-25T09:40:57.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f1210a14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f120c0ac250 con 0x7f120c0ce5b0 2023-08-25T09:40:57.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f120a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 0x7f120c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f120a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 0x7f120c1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48462/0 (socket says 172.21.15.70:48462) 2023-08-25T09:40:57.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.750+0000 7f120a59c640 1 -- 172.21.15.70:0/3326652789 learned_addr learned my addr 172.21.15.70:0/3326652789 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:57.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120a59c640 1 -- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0ce5b0 msgr2=0x7f120c0ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120a59c640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0ce5b0 0x7f120c0ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120ad9d640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c15aa10 0x7f120c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120a59c640 1 -- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c15aa10 msgr2=0x7f120c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120a59c640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c15aa10 0x7f120c06f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120a59c640 1 -- 172.21.15.70:0/3326652789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12040094d0 con 0x7f120c0b2850 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f120a59c640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 0x7f120c1574c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f11fc037040 tx=0x7f11fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:57.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.751+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc03e610 con 0x7f120c0b2850 2023-08-25T09:40:57.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.752+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc03e7b0 con 0x7f120c0b2850 2023-08-25T09:40:57.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.752+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc009a70 con 0x7f120c0b2850 2023-08-25T09:40:57.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.752+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc03eb60 con 0x7f120c0b2850 2023-08-25T09:40:57.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.752+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0ac810 con 0x7f120c0b2850 2023-08-25T09:40:57.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.753+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f120c04e420 con 0x7f120c0b2850 2023-08-25T09:40:57.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.754+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f11fc03b050 con 0x7f120c0b2850 2023-08-25T09:40:57.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.755+0000 7f12037fe640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11ec06a050 0x7f11ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:57.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.755+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f11fc0ab220 con 0x7f120c0b2850 2023-08-25T09:40:57.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.755+0000 7f1201d9b640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11ec06a050 0x7f11ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:57.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.755+0000 7f1201d9b640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11ec06a050 0x7f11ec06c510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11f4006fd0 tx=0x7f11f4006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:57.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.759+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f11fc076950 con 0x7f120c0b2850 2023-08-25T09:40:57.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.984+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f120c0cae50 con 0x7f120c0b2850 2023-08-25T09:40:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.985+0000 7f12037fe640 1 -- 172.21.15.70:0/3326652789 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7f11fc039090 con 0x7f120c0b2850 2023-08-25T09:40:58.006 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:58.006 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:58.007 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:58.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11ec06a050 msgr2=0x7f11ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11ec06a050 0x7f11ec06c510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11f4006fd0 tx=0x7f11f4006bc0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 msgr2=0x7f120c1574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 0x7f120c1574c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f11fc037040 tx=0x7f11fc004a40 comp rx=0 tx=0).stop 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 shutdown_connections 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11ec06a050 0x7f11ec06c510 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f120c15aa10 0x7f120c06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f120c0ce5b0 0x7f120c0ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 --2- 172.21.15.70:0/3326652789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f120c0b2850 0x7f120c1574c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.987+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 >> 172.21.15.70:0/3326652789 conn(0x7f120c069850 msgr2=0x7f120c1585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:58.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.988+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 shutdown_connections 2023-08-25T09:40:58.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:57.988+0000 7f1210a14640 1 -- 172.21.15.70:0/3326652789 wait complete. 2023-08-25T09:40:58.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:40:58.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.223+0000 7ff5cc142640 1 Processor -- start 2023-08-25T09:40:58.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.223+0000 7ff5cc142640 1 -- start start 2023-08-25T09:40:58.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5cc142640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40cea30 0x7ff5c40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5cc142640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5c40ac870 0x7ff5c40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5cc142640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c406f050 con 0x7ff5c415cb40 2023-08-25T09:40:58.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5cc142640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c406f330 con 0x7ff5c40ac870 2023-08-25T09:40:58.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5cc142640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c406f610 con 0x7ff5c40cea30 2023-08-25T09:40:58.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5ca6df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5c40ac870 0x7ff5c40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5ca6df640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5c40ac870 0x7ff5c40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48476/0 (socket says 172.21.15.70:48476) 2023-08-25T09:40:58.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.225+0000 7ff5ca6df640 1 -- 172.21.15.70:0/2107024541 learned_addr learned my addr 172.21.15.70:0/2107024541 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:58.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.226+0000 7ff5ca6df640 1 -- 172.21.15.70:0/2107024541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40cea30 msgr2=0x7ff5c40b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:58.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.226+0000 7ff5ca6df640 1 --2- 172.21.15.70:0/2107024541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40cea30 0x7ff5c40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.226+0000 7ff5ca6df640 1 -- 172.21.15.70:0/2107024541 >> v1:172.21.15.155:6789/0 conn(0x7ff5c415cb40 legacy=0x7ff5c40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:58.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.226+0000 7ff5ca6df640 1 -- 172.21.15.70:0/2107024541 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c406f8f0 con 0x7ff5c40ac870 2023-08-25T09:40:58.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.227+0000 7ff5ca6df640 1 --2- 172.21.15.70:0/2107024541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40ac870 0x7ff5c40b2070 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009e50 tx=0x7ff5bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd42ff807e2cc1b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:58.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.227+0000 7ff5c8edc640 1 -- 172.21.15.70:0/2107024541 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc02ca80 con 0x7ff5c40ac870 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.227+0000 7ff5c8edc640 1 -- 172.21.15.70:0/2107024541 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc02cbe0 con 0x7ff5c40ac870 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.227+0000 7ff5c8edc640 1 -- 172.21.15.70:0/2107024541 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc0343f0 con 0x7ff5c40ac870 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 -- 172.21.15.70:0/2107024541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40ac870 msgr2=0x7ff5c40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 --2- 172.21.15.70:0/2107024541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40ac870 0x7ff5c40b2070 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009e50 tx=0x7ff5bc02bee0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 -- 172.21.15.70:0/2107024541 shutdown_connections 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 --2- 172.21.15.70:0/2107024541 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40ac870 0x7ff5c40b2070 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 --2- 172.21.15.70:0/2107024541 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40cea30 0x7ff5c40b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 -- 172.21.15.70:0/2107024541 >> 172.21.15.70:0/2107024541 conn(0x7ff5c4069850 msgr2=0x7ff5c4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:58.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 -- 172.21.15.70:0/2107024541 shutdown_connections 2023-08-25T09:40:58.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.228+0000 7ff5cc142640 1 -- 172.21.15.70:0/2107024541 wait complete. 2023-08-25T09:40:58.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.229+0000 7ff5cc142640 1 Processor -- start 2023-08-25T09:40:58.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.230+0000 7ff5cc142640 1 -- start start 2023-08-25T09:40:58.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5cc142640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40ac870 0x7ff5c40cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5cc142640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 0x7ff5c4153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5cc142640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c415cb40 0x7ff5c415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5cc142640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40cb470 con 0x7ff5c40cea30 2023-08-25T09:40:58.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5cc142640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c415f840 con 0x7ff5c415cb40 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5cc142640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c40cd730 con 0x7ff5c40ac870 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 0x7ff5c4153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 0x7ff5c4153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48478/0 (socket says 172.21.15.70:48478) 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 -- 172.21.15.70:0/1854327336 learned_addr learned my addr 172.21.15.70:0/1854327336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5ca6df640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c415cb40 0x7ff5c415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c9ede640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40ac870 0x7ff5c40cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 -- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40ac870 msgr2=0x7ff5c40cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40ac870 0x7ff5c40cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 -- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c415cb40 msgr2=0x7ff5c415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c415cb40 0x7ff5c415d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5c96dd640 1 -- 172.21.15.70:0/1854327336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009b30 con 0x7ff5c40cea30 2023-08-25T09:40:58.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.231+0000 7ff5ca6df640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c415cb40 0x7ff5c415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:58.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.232+0000 7ff5c96dd640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 0x7ff5c4153b40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b4a0 tx=0x7ff5b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.232+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4010610 con 0x7ff5c40cea30 2023-08-25T09:40:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.232+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b4010770 con 0x7ff5c40cea30 2023-08-25T09:40:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.232+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c415f280 con 0x7ff5c40cea30 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.232+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4010b20 con 0x7ff5c40cea30 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.232+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c40cb6a0 con 0x7ff5c40cea30 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.233+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c404e420 con 0x7ff5c40cea30 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.234+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff5b4010c80 con 0x7ff5c40cea30 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.235+0000 7ff5ba7fc640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff5a806a050 0x7ff5a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.235+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff5b4088220 con 0x7ff5c40cea30 2023-08-25T09:40:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.235+0000 7ff5c9ede640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff5a806a050 0x7ff5a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.236+0000 7ff5c9ede640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff5a806a050 0x7ff5a806c510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000a730 tx=0x7ff5c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:58.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.239+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff5b40538d0 con 0x7ff5c40cea30 2023-08-25T09:40:58.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.475+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5c40afb20 con 0x7ff5c40cea30 2023-08-25T09:40:58.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.475+0000 7ff5ba7fc640 1 -- 172.21.15.70:0/1854327336 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4875 (secure 0 0 0) 0x7ff5b4056ef0 con 0x7ff5c40cea30 2023-08-25T09:40:58.477 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:58.478 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:40:26.917357+0000","modified":"2023-08-25T09:40:30.467270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5361":{"gid":5361,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.155:6835/396571742","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":396571742},{"type":"v1","addr":"172.21.15.155:6835","nonce":396571742}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T09:40:58.478 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 16 2023-08-25T09:40:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff5a806a050 msgr2=0x7ff5a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff5a806a050 0x7ff5a806c510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000a730 tx=0x7ff5c0008040 comp rx=0 tx=0).stop 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 msgr2=0x7ff5c4153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 0x7ff5c4153b40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b4a0 tx=0x7ff5b400b960 comp rx=0 tx=0).stop 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 shutdown_connections 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff5a806a050 0x7ff5a806c510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5c415cb40 0x7ff5c415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5c40cea30 0x7ff5c4153b40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 --2- 172.21.15.70:0/1854327336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5c40ac870 0x7ff5c40cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 >> 172.21.15.70:0/1854327336 conn(0x7ff5c4069850 msgr2=0x7ff5c406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.478+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 shutdown_connections 2023-08-25T09:40:58.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.479+0000 7ff5cc142640 1 -- 172.21.15.70:0/1854327336 wait complete. 2023-08-25T09:40:58.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:40:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.713+0000 7f79d9246640 1 Processor -- start 2023-08-25T09:40:58.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.714+0000 7f79d9246640 1 -- start start 2023-08-25T09:40:58.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.715+0000 7f79d9246640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d40b1ff0 0x7f79d40b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.715+0000 7f79d9246640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40b1b60 con 0x7f79d415ec10 2023-08-25T09:40:58.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.715+0000 7f79d9246640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d406b7c0 con 0x7f79d415e830 2023-08-25T09:40:58.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.715+0000 7f79d9246640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d406bd80 con 0x7f79d40b1ff0 2023-08-25T09:40:58.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79d415ec10 0x7f79d415c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41010/0 (socket says 172.21.15.70:41010) 2023-08-25T09:40:58.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d259c640 1 -- 172.21.15.70:0/1843130982 learned_addr learned my addr 172.21.15.70:0/1843130982 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:40:58.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d359e640 1 --2- 172.21.15.70:0/1843130982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d40b1ff0 0x7f79d40b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285229333 0 0) 0x7f79d40b1b60 con 0x7f79d415ec10 2023-08-25T09:40:58.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c0003520 con 0x7f79d415ec10 2023-08-25T09:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182376802 0 0) 0x7f79d406b7c0 con 0x7f79d415e830 2023-08-25T09:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.716+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d40b1b60 con 0x7f79d415e830 2023-08-25T09:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.717+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358371799 0 0) 0x7f79c0003520 con 0x7f79d415ec10 2023-08-25T09:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.717+0000 7f79d359e640 1 -- 172.21.15.70:0/1843130982 >> v1:172.21.15.155:6789/0 conn(0x7f79d415e830 legacy=0x7f79d406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.717+0000 7f79d359e640 1 -- 172.21.15.70:0/1843130982 >> v1:172.21.15.70:6789/0 conn(0x7f79d415ec10 legacy=0x7f79d415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.717+0000 7f79d359e640 1 -- 172.21.15.70:0/1843130982 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d406baa0 con 0x7f79d40b1ff0 2023-08-25T09:40:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.718+0000 7f79d359e640 1 --2- 172.21.15.70:0/1843130982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d40b1ff0 0x7f79d40b2480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0097d0 tx=0x7f79cc02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e59e32d653ca67f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.718+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc001670 con 0x7f79d40b1ff0 2023-08-25T09:40:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.718+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc02e050 con 0x7f79d40b1ff0 2023-08-25T09:40:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.718+0000 7f79d1d9b640 1 -- 172.21.15.70:0/1843130982 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc030730 con 0x7f79d40b1ff0 2023-08-25T09:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 -- 172.21.15.70:0/1843130982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d40b1ff0 msgr2=0x7f79d40b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 --2- 172.21.15.70:0/1843130982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d40b1ff0 0x7f79d40b2480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0097d0 tx=0x7f79cc02b800 comp rx=0 tx=0).stop 2023-08-25T09:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 -- 172.21.15.70:0/1843130982 shutdown_connections 2023-08-25T09:40:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 --2- 172.21.15.70:0/1843130982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d40b1ff0 0x7f79d40b2480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 -- 172.21.15.70:0/1843130982 >> 172.21.15.70:0/1843130982 conn(0x7f79d4069850 msgr2=0x7f79d4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 -- 172.21.15.70:0/1843130982 shutdown_connections 2023-08-25T09:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.719+0000 7f79d9246640 1 -- 172.21.15.70:0/1843130982 wait complete. 2023-08-25T09:40:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.721+0000 7f79d9246640 1 Processor -- start 2023-08-25T09:40:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.721+0000 7f79d9246640 1 -- start start 2023-08-25T09:40:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.722+0000 7f79d9246640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d40b1ff0 0x7f79d40ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.722+0000 7f79d9246640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 0x7f79d40aa910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.722+0000 7f79d9246640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d415ec10 0x7f79d40c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.722+0000 7f79d9246640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40ad630 con 0x7f79d415e830 2023-08-25T09:40:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.722+0000 7f79d9246640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40ad910 con 0x7f79d40b1ff0 2023-08-25T09:40:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.722+0000 7f79d9246640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40adbf0 con 0x7f79d415ec10 2023-08-25T09:40:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 0x7f79d40aa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 0x7f79d40aa910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:48488/0 (socket says 172.21.15.70:48488) 2023-08-25T09:40:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 -- 172.21.15.70:0/2507569745 learned_addr learned my addr 172.21.15.70:0/2507569745 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d2d9d640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d40b1ff0 0x7f79d40ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d359e640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d415ec10 0x7f79d40c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 -- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d415ec10 msgr2=0x7f79d40c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d415ec10 0x7f79d40c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 -- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d40b1ff0 msgr2=0x7f79d40ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d40b1ff0 0x7f79d40ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 -- 172.21.15.70:0/2507569745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009400 con 0x7f79d415e830 2023-08-25T09:40:58.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.723+0000 7f79d259c640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 0x7f79d40aa910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c750 tx=0x7f79bc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:58.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.724+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc005ad0 con 0x7f79d415e830 2023-08-25T09:40:58.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.724+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc005c30 con 0x7f79d415e830 2023-08-25T09:40:58.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.724+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc007290 con 0x7f79d415e830 2023-08-25T09:40:58.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.724+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d40aded0 con 0x7f79d415e830 2023-08-25T09:40:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.724+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d40ad290 con 0x7f79d415e830 2023-08-25T09:40:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.725+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d404e420 con 0x7f79d415e830 2023-08-25T09:40:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.726+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f79bc0073f0 con 0x7f79d415e830 2023-08-25T09:40:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.727+0000 7f79cb7fe640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79b406a050 0x7f79b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.727+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f79bc087c90 con 0x7f79d415e830 2023-08-25T09:40:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.728+0000 7f79d2d9d640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79b406a050 0x7f79b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.728+0000 7f79d2d9d640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79b406a050 0x7f79b406c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f79c4001460 tx=0x7f79c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:58.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.732+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f79bc053350 con 0x7f79d415e830 2023-08-25T09:40:58.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.920+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79d40ad770 con 0x7f79d415e830 2023-08-25T09:40:58.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.923+0000 7f79cb7fe640 1 -- 172.21.15.70:0/2507569745 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18401 (secure 0 0 0) 0x7f79d40ad770 con 0x7f79d415e830 2023-08-25T09:40:58.925 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:40:58.925 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":39,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:40:44.272862+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:40:25.098622+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T09:40:25.912835+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:40:58.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79b406a050 msgr2=0x7f79b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79b406a050 0x7f79b406c510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f79c4001460 tx=0x7f79c4009040 comp rx=0 tx=0).stop 2023-08-25T09:40:58.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 msgr2=0x7f79d40aa910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:58.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 0x7f79d40aa910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c750 tx=0x7f79bc00cc10 comp rx=0 tx=0).stop 2023-08-25T09:40:58.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 shutdown_connections 2023-08-25T09:40:58.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79b406a050 0x7f79b406c510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79d415ec10 0x7f79d40c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79d415e830 0x7f79d40aa910 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 --2- 172.21.15.70:0/2507569745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79d40b1ff0 0x7f79d40ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:58.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 >> 172.21.15.70:0/2507569745 conn(0x7f79d4069850 msgr2=0x7f79d40b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:58.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 shutdown_connections 2023-08-25T09:40:58.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:58.926+0000 7f79d9246640 1 -- 172.21.15.70:0/2507569745 wait complete. 2023-08-25T09:40:58.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:40:59.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.152+0000 7fc284e2d640 1 Processor -- start 2023-08-25T09:40:59.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.152+0000 7fc284e2d640 1 -- start start 2023-08-25T09:40:59.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.153+0000 7fc284e2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc28015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.153+0000 7fc284e2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2800ca780 0x7fc2800cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.153+0000 7fc284e2d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc2800b3cf0 0x7fc2800b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc284e2d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28006c0e0 con 0x7fc2800ca780 2023-08-25T09:40:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc284e2d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2800b12b0 con 0x7fc2800b3cf0 2023-08-25T09:40:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc284e2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28006ae20 con 0x7fc28015acd0 2023-08-25T09:40:59.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2800ca780 0x7fc2800cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2800ca780 0x7fc2800cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37564/0 (socket says 172.21.15.70:37564) 2023-08-25T09:40:59.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1590166899 learned_addr learned my addr 172.21.15.70:0/1590166899 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:59.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27e59c640 1 --2- 172.21.15.70:0/1590166899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc28015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27ed9d640 1 --2- 172.21.15.70:0/1590166899 >> v2:172.21.15.155:3300/0 conn(0x7fc2800b3cf0 0x7fc2800b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1590166899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 msgr2=0x7fc28015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 --2- 172.21.15.70:0/1590166899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc28015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1590166899 >> v2:172.21.15.155:3300/0 conn(0x7fc2800b3cf0 msgr2=0x7fc2800b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.154+0000 7fc27dd9b640 1 --2- 172.21.15.70:0/1590166899 >> v2:172.21.15.155:3300/0 conn(0x7fc2800b3cf0 0x7fc2800b4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.155+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1590166899 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28006b910 con 0x7fc2800ca780 2023-08-25T09:40:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.156+0000 7fc27dd9b640 1 --2- 172.21.15.70:0/1590166899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc2800cac00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc2800b4990 tx=0x7fc26802b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47405bcb8794fb02 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:59.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.156+0000 7fc27d59a640 1 -- 172.21.15.70:0/1590166899 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2680317f0 con 0x7fc2800ca780 2023-08-25T09:40:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.156+0000 7fc27d59a640 1 -- 172.21.15.70:0/1590166899 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268031950 con 0x7fc2800ca780 2023-08-25T09:40:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.156+0000 7fc27d59a640 1 -- 172.21.15.70:0/1590166899 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268031ce0 con 0x7fc2800ca780 2023-08-25T09:40:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 -- 172.21.15.70:0/1590166899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 msgr2=0x7fc2800cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1590166899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc2800cac00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc2800b4990 tx=0x7fc26802b890 comp rx=0 tx=0).stop 2023-08-25T09:40:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 -- 172.21.15.70:0/1590166899 shutdown_connections 2023-08-25T09:40:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1590166899 >> v2:172.21.15.155:3300/0 conn(0x7fc2800b3cf0 0x7fc2800b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1590166899 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc2800cac00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1590166899 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc28015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 -- 172.21.15.70:0/1590166899 >> 172.21.15.70:0/1590166899 conn(0x7fc2800ce760 msgr2=0x7fc2800ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 -- 172.21.15.70:0/1590166899 shutdown_connections 2023-08-25T09:40:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.157+0000 7fc284e2d640 1 -- 172.21.15.70:0/1590166899 wait complete. 2023-08-25T09:40:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.159+0000 7fc284e2d640 1 Processor -- start 2023-08-25T09:40:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.159+0000 7fc284e2d640 1 -- start start 2023-08-25T09:40:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc284e2d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2800b3cf0 0x7fc28014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc284e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc28006ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc284e2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc280070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc284e2d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2801e7280 con 0x7fc2800ca780 2023-08-25T09:40:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc284e2d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2801e6600 con 0x7fc2800b3cf0 2023-08-25T09:40:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc284e2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2801e68e0 con 0x7fc28015acd0 2023-08-25T09:40:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc28006ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc28006ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37568/0 (socket says 172.21.15.70:37568) 2023-08-25T09:40:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1040020641 learned_addr learned my addr 172.21.15.70:0/1040020641 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27e59c640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2800b3cf0 0x7fc28014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27ed9d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc280070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 msgr2=0x7fc280070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc280070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2800b3cf0 msgr2=0x7fc28014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.160+0000 7fc27dd9b640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2800b3cf0 0x7fc28014c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc27dd9b640 1 -- 172.21.15.70:0/1040020641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2680094d0 con 0x7fc2800ca780 2023-08-25T09:40:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc27ed9d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc280070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:40:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc27dd9b640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc28006ff50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc268000c00 tx=0x7fc2680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:59.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268032890 con 0x7fc2800ca780 2023-08-25T09:40:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc268032a30 con 0x7fc2800ca780 2023-08-25T09:40:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc268032de0 con 0x7fc2800ca780 2023-08-25T09:40:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2801e6bc0 con 0x7fc2800ca780 2023-08-25T09:40:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.161+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28014c930 con 0x7fc2800ca780 2023-08-25T09:40:59.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.162+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2801e73c0 con 0x7fc2800ca780 2023-08-25T09:40:59.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.163+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc26804e450 con 0x7fc2800ca780 2023-08-25T09:40:59.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.164+0000 7fc276ffd640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc26006a050 0x7fc26006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.164+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc2680aa940 con 0x7fc2800ca780 2023-08-25T09:40:59.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.164+0000 7fc27e59c640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc26006a050 0x7fc26006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.165+0000 7fc27e59c640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc26006a050 0x7fc26006c510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc28015d700 tx=0x7fc27000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:59.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.168+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc2801e73c0 con 0x7fc2800ca780 2023-08-25T09:40:59.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.389+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc2801e73c0 con 0x7fc2800ca780 2023-08-25T09:40:59.391 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:40:59.390+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:40:59.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.395+0000 7fc276ffd640 1 -- 172.21.15.70:0/1040020641 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc2801e73c0 con 0x7fc2800ca780 2023-08-25T09:40:59.398 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:40:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc26006a050 msgr2=0x7fc26006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc26006a050 0x7fc26006c510 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc28015d700 tx=0x7fc27000b040 comp rx=0 tx=0).stop 2023-08-25T09:40:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 msgr2=0x7fc28006ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc28006ff50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc268000c00 tx=0x7fc2680046c0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 shutdown_connections 2023-08-25T09:40:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc26006a050 0x7fc26006c510 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc28015acd0 0x7fc280070490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2800ca780 0x7fc28006ff50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 --2- 172.21.15.70:0/1040020641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2800b3cf0 0x7fc28014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.399+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 >> 172.21.15.70:0/1040020641 conn(0x7fc2800ce760 msgr2=0x7fc28006ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:59.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.400+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 shutdown_connections 2023-08-25T09:40:59.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.400+0000 7fc284e2d640 1 -- 172.21.15.70:0/1040020641 wait complete. 2023-08-25T09:40:59.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:40:59.429 INFO:tasks.ceph.mds.b.smithi155.stderr: -3546> 2023-08-25T09:40:33.385+0000 7f22e0334640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:40:59.520 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:40:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.630+0000 7fc69b322640 1 Processor -- start 2023-08-25T09:40:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.630+0000 7fc69b322640 1 -- start start 2023-08-25T09:40:59.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.631+0000 7fc69b322640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69406b610 0x7fc69406ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.631+0000 7fc69b322640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc69415b030 0x7fc69415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.631+0000 7fc69b322640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69406bfb0 con 0x7fc69415cdc0 2023-08-25T09:40:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.631+0000 7fc69b322640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc694159d80 con 0x7fc69415b030 2023-08-25T09:40:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.631+0000 7fc69b322640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69415d1a0 con 0x7fc69406b610 2023-08-25T09:40:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc69415b030 0x7fc69415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc69415b030 0x7fc69415b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37582/0 (socket says 172.21.15.70:37582) 2023-08-25T09:40:59.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 -- 172.21.15.70:0/2056725401 learned_addr learned my addr 172.21.15.70:0/2056725401 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 -- 172.21.15.70:0/2056725401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69406b610 msgr2=0x7fc69406ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:40:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6988bd640 1 --2- 172.21.15.70:0/2056725401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69406b610 0x7fc69406ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 --2- 172.21.15.70:0/2056725401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69406b610 0x7fc69406ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 -- 172.21.15.70:0/2056725401 >> v1:172.21.15.155:6789/0 conn(0x7fc69415cdc0 legacy=0x7fc6940b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.632+0000 7fc6998bf640 1 -- 172.21.15.70:0/2056725401 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6940b3710 con 0x7fc69415b030 2023-08-25T09:40:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.634+0000 7fc6998bf640 1 --2- 172.21.15.70:0/2056725401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69415b030 0x7fc69415b4a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e50 tx=0x7fc68c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f9f3ae95f68c70 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.634+0000 7fc68bfff640 1 -- 172.21.15.70:0/2056725401 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c02cb00 con 0x7fc69415b030 2023-08-25T09:40:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.634+0000 7fc68bfff640 1 -- 172.21.15.70:0/2056725401 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c02cc60 con 0x7fc69415b030 2023-08-25T09:40:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.634+0000 7fc68bfff640 1 -- 172.21.15.70:0/2056725401 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0343f0 con 0x7fc69415b030 2023-08-25T09:40:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.634+0000 7fc69b322640 1 -- 172.21.15.70:0/2056725401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69415b030 msgr2=0x7fc69415b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.634+0000 7fc69b322640 1 --2- 172.21.15.70:0/2056725401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69415b030 0x7fc69415b4a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009e50 tx=0x7fc68c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:40:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.635+0000 7fc69b322640 1 -- 172.21.15.70:0/2056725401 shutdown_connections 2023-08-25T09:40:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.635+0000 7fc69b322640 1 --2- 172.21.15.70:0/2056725401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69415b030 0x7fc69415b4a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.635+0000 7fc69b322640 1 --2- 172.21.15.70:0/2056725401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69406b610 0x7fc69406ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.635+0000 7fc69b322640 1 -- 172.21.15.70:0/2056725401 >> 172.21.15.70:0/2056725401 conn(0x7fc694069850 msgr2=0x7fc694069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:40:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.635+0000 7fc69b322640 1 -- 172.21.15.70:0/2056725401 shutdown_connections 2023-08-25T09:40:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.635+0000 7fc69b322640 1 -- 172.21.15.70:0/2056725401 wait complete. 2023-08-25T09:40:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.636+0000 7fc69b322640 1 Processor -- start 2023-08-25T09:40:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 -- start start 2023-08-25T09:40:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 0x7fc6941f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc69415b030 0x7fc6940cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69415cdc0 0x7fc694155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc694155180 con 0x7fc69406b610 2023-08-25T09:40:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6941f2010 con 0x7fc69415b030 2023-08-25T09:40:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.637+0000 7fc69b322640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc694154570 con 0x7fc69415cdc0 2023-08-25T09:40:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 0x7fc6941f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 0x7fc6941f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37598/0 (socket says 172.21.15.70:37598) 2023-08-25T09:40:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 -- 172.21.15.70:0/48313985 learned_addr learned my addr 172.21.15.70:0/48313985 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:40:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 -- 172.21.15.70:0/48313985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69415cdc0 msgr2=0x7fc694155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69415cdc0 0x7fc694155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 -- 172.21.15.70:0/48313985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc69415b030 msgr2=0x7fc6940cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc69415b030 0x7fc6940cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 -- 172.21.15.70:0/48313985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009b30 con 0x7fc69406b610 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.638+0000 7fc6990be640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 0x7fc6941f1ad0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc69000c450 tx=0x7fc69000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.639+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6900052d0 con 0x7fc69406b610 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.639+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc690005430 con 0x7fc69406b610 2023-08-25T09:40:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.639+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc690012870 con 0x7fc69406b610 2023-08-25T09:40:59.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.639+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc694154850 con 0x7fc69406b610 2023-08-25T09:40:59.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.639+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc694154e10 con 0x7fc69406b610 2023-08-25T09:40:59.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.640+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69404e420 con 0x7fc69406b610 2023-08-25T09:40:59.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.641+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc6900129d0 con 0x7fc69406b610 2023-08-25T09:40:59.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.642+0000 7fc689ffb640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc67806a080 0x7fc67806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:40:59.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.642+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fc690088140 con 0x7fc69406b610 2023-08-25T09:40:59.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.643+0000 7fc6988bd640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc67806a080 0x7fc67806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:40:59.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.643+0000 7fc6988bd640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc67806a080 0x7fc67806c540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc694158d20 tx=0x7fc684008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:40:59.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.647+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc690053790 con 0x7fc69406b610 2023-08-25T09:40:59.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:40:59.855+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc694154fd0 con 0x7fc69406b610 2023-08-25T09:41:00.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.400+0000 7fc689ffb640 1 -- 172.21.15.70:0/48313985 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc690056db0 con 0x7fc69406b610 2023-08-25T09:41:00.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc67806a080 msgr2=0x7fc67806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc67806a080 0x7fc67806c540 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc694158d20 tx=0x7fc684008040 comp rx=0 tx=0).stop 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 msgr2=0x7fc6941f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 0x7fc6941f1ad0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc69000c450 tx=0x7fc69000c910 comp rx=0 tx=0).stop 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 shutdown_connections 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc67806a080 0x7fc67806c540 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc69415cdc0 0x7fc694155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc69415b030 0x7fc6940cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 --2- 172.21.15.70:0/48313985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc69406b610 0x7fc6941f1ad0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 >> 172.21.15.70:0/48313985 conn(0x7fc694069850 msgr2=0x7fc69415b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 shutdown_connections 2023-08-25T09:41:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.404+0000 7fc69b322640 1 -- 172.21.15.70:0/48313985 wait complete. 2023-08-25T09:41:00.426 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:41:00.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.640+0000 7f6da9670640 1 Processor -- start 2023-08-25T09:41:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.640+0000 7f6da9670640 1 -- start start 2023-08-25T09:41:00.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da9670640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6da4155120 0x7f6da40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da9670640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6da40cd890 0x7f6da40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da9670640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da415e6f0 con 0x7f6da40ca820 2023-08-25T09:41:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da9670640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da40cdd40 con 0x7f6da4155120 2023-08-25T09:41:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da9670640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da40caf30 con 0x7f6da40cd890 2023-08-25T09:41:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6da4155120 0x7f6da40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:00.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6da4155120 0x7f6da40cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37612/0 (socket says 172.21.15.70:37612) 2023-08-25T09:41:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.642+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2097840144 learned_addr learned my addr 172.21.15.70:0/2097840144 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:00.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.643+0000 7f6da27fc640 1 --2- 172.21.15.70:0/2097840144 >> v2:172.21.15.155:3300/0 conn(0x7f6da40cd890 0x7f6da40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.643+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2097840144 >> v2:172.21.15.155:3300/0 conn(0x7f6da40cd890 msgr2=0x7f6da40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.643+0000 7f6da2ffd640 1 --2- 172.21.15.70:0/2097840144 >> v2:172.21.15.155:3300/0 conn(0x7f6da40cd890 0x7f6da40ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.643+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2097840144 >> v1:172.21.15.155:6789/0 conn(0x7f6da40ca820 legacy=0x7f6da406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.643+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2097840144 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da415e9d0 con 0x7f6da4155120 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.644+0000 7f6da2ffd640 1 --2- 172.21.15.70:0/2097840144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da4155120 0x7f6da40cd350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6da406ce50 tx=0x7f6d9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ba86fb8f0080d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.644+0000 7f6da1ffb640 1 -- 172.21.15.70:0/2097840144 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94031870 con 0x7f6da4155120 2023-08-25T09:41:00.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.644+0000 7f6da1ffb640 1 -- 172.21.15.70:0/2097840144 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d940319d0 con 0x7f6da4155120 2023-08-25T09:41:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.644+0000 7f6da1ffb640 1 -- 172.21.15.70:0/2097840144 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94031d60 con 0x7f6da4155120 2023-08-25T09:41:00.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 -- 172.21.15.70:0/2097840144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da4155120 msgr2=0x7f6da40cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 --2- 172.21.15.70:0/2097840144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da4155120 0x7f6da40cd350 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6da406ce50 tx=0x7f6d9402b910 comp rx=0 tx=0).stop 2023-08-25T09:41:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 -- 172.21.15.70:0/2097840144 shutdown_connections 2023-08-25T09:41:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 --2- 172.21.15.70:0/2097840144 >> v2:172.21.15.155:3300/0 conn(0x7f6da40cd890 0x7f6da40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 --2- 172.21.15.70:0/2097840144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da4155120 0x7f6da40cd350 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 -- 172.21.15.70:0/2097840144 >> 172.21.15.70:0/2097840144 conn(0x7f6da4069c70 msgr2=0x7f6da406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 -- 172.21.15.70:0/2097840144 shutdown_connections 2023-08-25T09:41:00.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.645+0000 7f6da9670640 1 -- 172.21.15.70:0/2097840144 wait complete. 2023-08-25T09:41:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.647+0000 7f6da9670640 1 Processor -- start 2023-08-25T09:41:00.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.647+0000 7f6da9670640 1 -- start start 2023-08-25T09:41:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da9670640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 0x7f6da40c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da9670640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da40cd890 0x7f6da4145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da9670640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6da4155120 0x7f6da40c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da9670640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40c7080 con 0x7f6da40ca820 2023-08-25T09:41:00.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da9670640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40c8a60 con 0x7f6da40cd890 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da9670640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da40c8d40 con 0x7f6da4155120 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 0x7f6da40c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 0x7f6da40c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37624/0 (socket says 172.21.15.70:37624) 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.648+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2930760442 learned_addr learned my addr 172.21.15.70:0/2930760442 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2930760442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6da4155120 msgr2=0x7f6da40c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da2ffd640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6da4155120 0x7f6da40c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2930760442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da40cd890 msgr2=0x7f6da4145f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:00.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da2ffd640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da40cd890 0x7f6da4145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:00.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da2ffd640 1 -- 172.21.15.70:0/2930760442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d940094d0 con 0x7f6da40ca820 2023-08-25T09:41:00.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da2ffd640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 0x7f6da40c9540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000c00 tx=0x7f6d940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:00.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d94031840 con 0x7f6da40ca820 2023-08-25T09:41:00.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d940443f0 con 0x7f6da40ca820 2023-08-25T09:41:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da40c9020 con 0x7f6da40ca820 2023-08-25T09:41:00.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d940447c0 con 0x7f6da40ca820 2023-08-25T09:41:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.649+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da41464b0 con 0x7f6da40ca820 2023-08-25T09:41:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.650+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da40c88a0 con 0x7f6da40ca820 2023-08-25T09:41:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.651+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6d9404d450 con 0x7f6da40ca820 2023-08-25T09:41:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.653+0000 7f6d9b7fe640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d8406a050 0x7f6d8406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.653+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f6d9402d040 con 0x7f6da40ca820 2023-08-25T09:41:00.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.653+0000 7f6da27fc640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d8406a050 0x7f6d8406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:00.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.654+0000 7f6da27fc640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d8406a050 0x7f6d8406c510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6da41f5630 tx=0x7f6d8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:00.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.657+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6d940764c0 con 0x7f6da40ca820 2023-08-25T09:41:00.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:00.845+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6da40c9230 con 0x7f6da40ca820 2023-08-25T09:41:01.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.404+0000 7f6d9b7fe640 1 -- 172.21.15.70:0/2930760442 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v41) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6d940350d0 con 0x7f6da40ca820 2023-08-25T09:41:01.406 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:41:01.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d8406a050 msgr2=0x7f6d8406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d8406a050 0x7f6d8406c510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6da41f5630 tx=0x7f6d8c008040 comp rx=0 tx=0).stop 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 msgr2=0x7f6da40c9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 0x7f6da40c9540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6d94000c00 tx=0x7f6d940046c0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 shutdown_connections 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6d8406a050 0x7f6d8406c510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6da4155120 0x7f6da40c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6da40cd890 0x7f6da4145f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 --2- 172.21.15.70:0/2930760442 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6da40ca820 0x7f6da40c9540 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 >> 172.21.15.70:0/2930760442 conn(0x7f6da4069c70 msgr2=0x7f6da40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 shutdown_connections 2023-08-25T09:41:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.408+0000 7f6da9670640 1 -- 172.21.15.70:0/2930760442 wait complete. 2023-08-25T09:41:01.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:41:01.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.647+0000 7fd45180d640 1 Processor -- start 2023-08-25T09:41:01.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.648+0000 7fd45180d640 1 -- start start 2023-08-25T09:41:01.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.649+0000 7fd45180d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd44c142740 0x7fd44c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:01.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.649+0000 7fd45180d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c06aa70 0x7fd44c06af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:01.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.649+0000 7fd45180d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c15d320 con 0x7fd44c142c30 2023-08-25T09:41:01.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.649+0000 7fd45180d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c06b440 con 0x7fd44c142740 2023-08-25T09:41:01.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.649+0000 7fd45180d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0ccd40 con 0x7fd44c06aa70 2023-08-25T09:41:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd44c142740 0x7fd44c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd44c142740 0x7fd44c0cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:45226/0 (socket says 172.21.15.70:45226) 2023-08-25T09:41:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 -- 172.21.15.70:0/3484189897 learned_addr learned my addr 172.21.15.70:0/3484189897 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44b7fe640 1 --2- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c06aa70 0x7fd44c06af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:01.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 -- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c06aa70 msgr2=0x7fd44c06af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 --2- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c06aa70 0x7fd44c06af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 -- 172.21.15.70:0/3484189897 >> v1:172.21.15.155:6789/0 conn(0x7fd44c142c30 legacy=0x7fd44c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.650+0000 7fd44affd640 1 -- 172.21.15.70:0/3484189897 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0cd020 con 0x7fd44c142740 2023-08-25T09:41:01.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.652+0000 7fd44affd640 1 --2- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142740 0x7fd44c0cc6b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4340098a0 tx=0x7fd43402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce5eb0c71bd852b2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:01.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd449ffb640 1 -- 172.21.15.70:0/3484189897 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434030840 con 0x7fd44c142740 2023-08-25T09:41:01.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd449ffb640 1 -- 172.21.15.70:0/3484189897 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4340309a0 con 0x7fd44c142740 2023-08-25T09:41:01.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd449ffb640 1 -- 172.21.15.70:0/3484189897 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434030d30 con 0x7fd44c142740 2023-08-25T09:41:01.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd45180d640 1 -- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142740 msgr2=0x7fd44c0cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:01.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd45180d640 1 --2- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142740 0x7fd44c0cc6b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4340098a0 tx=0x7fd43402b980 comp rx=0 tx=0).stop 2023-08-25T09:41:01.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd45180d640 1 -- 172.21.15.70:0/3484189897 shutdown_connections 2023-08-25T09:41:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd45180d640 1 --2- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c06aa70 0x7fd44c06af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd45180d640 1 --2- 172.21.15.70:0/3484189897 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142740 0x7fd44c0cc6b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.653+0000 7fd45180d640 1 -- 172.21.15.70:0/3484189897 >> 172.21.15.70:0/3484189897 conn(0x7fd44c069c70 msgr2=0x7fd44c06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.654+0000 7fd45180d640 1 -- 172.21.15.70:0/3484189897 shutdown_connections 2023-08-25T09:41:01.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.654+0000 7fd45180d640 1 -- 172.21.15.70:0/3484189897 wait complete. 2023-08-25T09:41:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.655+0000 7fd45180d640 1 Processor -- start 2023-08-25T09:41:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.655+0000 7fd45180d640 1 -- start start 2023-08-25T09:41:01.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.656+0000 7fd45180d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 0x7fd44c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:01.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.656+0000 7fd45180d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c142740 0x7fd44c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:01.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.656+0000 7fd45180d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142c30 0x7fd44c06fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:01.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.656+0000 7fd45180d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0b18e0 con 0x7fd44c06aa70 2023-08-25T09:41:01.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.656+0000 7fd45180d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c15a880 con 0x7fd44c142c30 2023-08-25T09:41:01.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.656+0000 7fd45180d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c15bc70 con 0x7fd44c142740 2023-08-25T09:41:01.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 0x7fd44c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 0x7fd44c0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37630/0 (socket says 172.21.15.70:37630) 2023-08-25T09:41:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 -- 172.21.15.70:0/3102144642 learned_addr learned my addr 172.21.15.70:0/3102144642 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 -- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c142740 msgr2=0x7fd44c1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44a7fc640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c142740 0x7fd44c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c142740 0x7fd44c1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 -- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142c30 msgr2=0x7fd44c06fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142c30 0x7fd44c06fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 -- 172.21.15.70:0/3102144642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340094d0 con 0x7fd44c06aa70 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44a7fc640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c142740 0x7fd44c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.657+0000 7fd44affd640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 0x7fd44c0b13a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.658+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434039c90 con 0x7fd44c06aa70 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.658+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd434039df0 con 0x7fd44c06aa70 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.658+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd434043640 con 0x7fd44c06aa70 2023-08-25T09:41:01.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.658+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c15bf50 con 0x7fd44c06aa70 2023-08-25T09:41:01.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.658+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c15c510 con 0x7fd44c06aa70 2023-08-25T09:41:01.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.659+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c1656c0 con 0x7fd44c06aa70 2023-08-25T09:41:01.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.660+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4340437a0 con 0x7fd44c06aa70 2023-08-25T09:41:01.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.661+0000 7fd443fff640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd42c06a050 0x7fd42c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:01.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.661+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fd43403e070 con 0x7fd44c06aa70 2023-08-25T09:41:01.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.661+0000 7fd44a7fc640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd42c06a050 0x7fd42c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:01.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.662+0000 7fd44a7fc640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd42c06a050 0x7fd42c06c510 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd44c164c70 tx=0x7fd43c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:01.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.665+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd44c1656c0 con 0x7fd44c06aa70 2023-08-25T09:41:01.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:01.858+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd44c1656c0 con 0x7fd44c06aa70 2023-08-25T09:41:02.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.407+0000 7fd443fff640 1 -- 172.21.15.70:0/3102144642 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v42) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd44c1656c0 con 0x7fd44c06aa70 2023-08-25T09:41:02.410 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:41:02.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.412+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd42c06a050 msgr2=0x7fd42c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.412+0000 7fd45180d640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd42c06a050 0x7fd42c06c510 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd44c164c70 tx=0x7fd43c009040 comp rx=0 tx=0).stop 2023-08-25T09:41:02.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.412+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 msgr2=0x7fd44c0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.412+0000 7fd45180d640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 0x7fd44c0b13a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd434000c00 tx=0x7fd434004730 comp rx=0 tx=0).stop 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 shutdown_connections 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd42c06a050 0x7fd42c06c510 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd44c142c30 0x7fd44c06fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd44c142740 0x7fd44c1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 --2- 172.21.15.70:0/3102144642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd44c06aa70 0x7fd44c0b13a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 >> 172.21.15.70:0/3102144642 conn(0x7fd44c069c70 msgr2=0x7fd44c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:02.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 shutdown_connections 2023-08-25T09:41:02.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.413+0000 7fd45180d640 1 -- 172.21.15.70:0/3102144642 wait complete. 2023-08-25T09:41:02.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:41:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.649+0000 7f4c02077640 1 Processor -- start 2023-08-25T09:41:02.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.649+0000 7f4c02077640 1 -- start start 2023-08-25T09:41:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4c02077640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bfc06f300 0x7f4bfc06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:02.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4c02077640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc15be60 con 0x7f4bfc15ae90 2023-08-25T09:41:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4c02077640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc06ee70 con 0x7f4bfc0ce410 2023-08-25T09:41:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4c02077640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc15b2f0 con 0x7f4bfc06f300 2023-08-25T09:41:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4bfbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bfc06f300 0x7f4bfc06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4bfbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4bfc06f300 0x7f4bfc06f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37632/0 (socket says 172.21.15.70:37632) 2023-08-25T09:41:02.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.651+0000 7f4bfbfff640 1 -- 172.21.15.70:0/1600722399 learned_addr learned my addr 172.21.15.70:0/1600722399 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.652+0000 7f4bfbfff640 1 -- 172.21.15.70:0/1600722399 >> v1:172.21.15.155:6789/0 conn(0x7f4bfc0ce410 legacy=0x7f4bfc070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.652+0000 7f4bfbfff640 1 -- 172.21.15.70:0/1600722399 >> v1:172.21.15.70:6789/0 conn(0x7f4bfc15ae90 legacy=0x7f4bfc0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.652+0000 7f4bfbfff640 1 -- 172.21.15.70:0/1600722399 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc15b5c0 con 0x7f4bfc06f300 2023-08-25T09:41:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.653+0000 7f4bfbfff640 1 --2- 172.21.15.70:0/1600722399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc06f790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009e50 tx=0x7f4bf402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438762c6ccaf6cbe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.653+0000 7f4bfaffd640 1 -- 172.21.15.70:0/1600722399 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402cb10 con 0x7f4bfc06f300 2023-08-25T09:41:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.653+0000 7f4bfaffd640 1 -- 172.21.15.70:0/1600722399 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf402cc70 con 0x7f4bfc06f300 2023-08-25T09:41:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.653+0000 7f4bfaffd640 1 -- 172.21.15.70:0/1600722399 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf40343f0 con 0x7f4bfc06f300 2023-08-25T09:41:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 -- 172.21.15.70:0/1600722399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 msgr2=0x7f4bfc06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 --2- 172.21.15.70:0/1600722399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc06f790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009e50 tx=0x7f4bf402bf70 comp rx=0 tx=0).stop 2023-08-25T09:41:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 -- 172.21.15.70:0/1600722399 shutdown_connections 2023-08-25T09:41:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 --2- 172.21.15.70:0/1600722399 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc06f790 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 -- 172.21.15.70:0/1600722399 >> 172.21.15.70:0/1600722399 conn(0x7f4bfc069850 msgr2=0x7f4bfc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:02.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 -- 172.21.15.70:0/1600722399 shutdown_connections 2023-08-25T09:41:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.654+0000 7f4c02077640 1 -- 172.21.15.70:0/1600722399 wait complete. 2023-08-25T09:41:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.656+0000 7f4c02077640 1 Processor -- start 2023-08-25T09:41:02.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.656+0000 7f4c02077640 1 -- start start 2023-08-25T09:41:02.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4c02077640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc1555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:02.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4c02077640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0ce410 0x7f4bfc1e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4c02077640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bfc15ae90 0x7f4bfc0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4c02077640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc155b30 con 0x7f4bfc06f300 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4c02077640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc1e5240 con 0x7f4bfc0ce410 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4c02077640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc0b1f00 con 0x7f4bfc15ae90 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4bfb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc1555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4bfb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc1555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37646/0 (socket says 172.21.15.70:37646) 2023-08-25T09:41:02.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.657+0000 7f4bfb7fe640 1 -- 172.21.15.70:0/1089997425 learned_addr learned my addr 172.21.15.70:0/1089997425 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:02.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bf3fff640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0ce410 0x7f4bfc1e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:02.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bfb7fe640 1 -- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bfc15ae90 msgr2=0x7f4bfc0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bfb7fe640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bfc15ae90 0x7f4bfc0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bfb7fe640 1 -- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0ce410 msgr2=0x7f4bfc1e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bfb7fe640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0ce410 0x7f4bfc1e4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bfb7fe640 1 -- 172.21.15.70:0/1089997425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009b30 con 0x7f4bfc06f300 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bf3fff640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0ce410 0x7f4bfc1e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bfb7fe640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc1555f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c430 tx=0x7f4bec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec011790 con 0x7f4bfc06f300 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec0118f0 con 0x7f4bfc06f300 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec011ce0 con 0x7f4bfc06f300 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bfc0b21e0 con 0x7f4bfc06f300 2023-08-25T09:41:02.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.658+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc0b27a0 con 0x7f4bfc06f300 2023-08-25T09:41:02.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.660+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc04e420 con 0x7f4bfc06f300 2023-08-25T09:41:02.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.661+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4bec01c460 con 0x7f4bfc06f300 2023-08-25T09:41:02.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.662+0000 7f4bf8ff9640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4bdc06a050 0x7f4bdc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:02.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.662+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4bec088260 con 0x7f4bfc06f300 2023-08-25T09:41:02.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.662+0000 7f4bf3fff640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4bdc06a050 0x7f4bdc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:02.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.663+0000 7f4bf3fff640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4bdc06a050 0x7f4bdc06c510 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4be4000f60 tx=0x7f4be4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.666+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4bec053bd0 con 0x7f4bfc06f300 2023-08-25T09:41:02.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.859+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4bfc15de10 con 0x7f4bfc06f300 2023-08-25T09:41:02.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.865+0000 7f4bf8ff9640 1 -- 172.21.15.70:0/1089997425 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bec0571f0 con 0x7f4bfc06f300 2023-08-25T09:41:02.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4bdc06a050 msgr2=0x7f4bdc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4bdc06a050 0x7f4bdc06c510 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4be4000f60 tx=0x7f4be4009040 comp rx=0 tx=0).stop 2023-08-25T09:41:02.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 msgr2=0x7f4bfc1555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:02.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc1555f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c430 tx=0x7f4bec00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 shutdown_connections 2023-08-25T09:41:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4bdc06a050 0x7f4bdc06c510 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4bfc15ae90 0x7f4bfc0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4bfc0ce410 0x7f4bfc1e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 --2- 172.21.15.70:0/1089997425 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4bfc06f300 0x7f4bfc1555f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 >> 172.21.15.70:0/1089997425 conn(0x7f4bfc069850 msgr2=0x7f4bfc158240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:02.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.868+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 shutdown_connections 2023-08-25T09:41:02.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:02.869+0000 7f4c02077640 1 -- 172.21.15.70:0/1089997425 wait complete. 2023-08-25T09:41:02.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-08-25T09:41:03.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.100+0000 7fad7c1e1640 1 Processor -- start 2023-08-25T09:41:03.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.100+0000 7fad7c1e1640 1 -- start start 2023-08-25T09:41:03.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7c1e1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fad7415e830 0x7fad7406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7c1e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7406b2b0 0x7fad7415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7c1e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad7415c6d0 0x7fad740b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7c1e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad7406bd80 con 0x7fad7415c6d0 2023-08-25T09:41:03.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7c1e1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740b1b60 con 0x7fad7415e830 2023-08-25T09:41:03.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7c1e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7406b7c0 con 0x7fad7406b2b0 2023-08-25T09:41:03.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7a77e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad7415c6d0 0x7fad740b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7a77e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fad7415c6d0 0x7fad740b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37656/0 (socket says 172.21.15.70:37656) 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad7a77e640 1 -- 172.21.15.70:0/2436012401 learned_addr learned my addr 172.21.15.70:0/2436012401 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.102+0000 7fad79f7d640 1 --2- 172.21.15.70:0/2436012401 >> v2:172.21.15.155:3300/0 conn(0x7fad7415e830 0x7fad7406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.103+0000 7fad7977c640 1 --2- 172.21.15.70:0/2436012401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7406b2b0 0x7fad7415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.103+0000 7fad7a77e640 1 -- 172.21.15.70:0/2436012401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7406b2b0 msgr2=0x7fad7415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.103+0000 7fad7a77e640 1 --2- 172.21.15.70:0/2436012401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7406b2b0 0x7fad7415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.103+0000 7fad7a77e640 1 -- 172.21.15.70:0/2436012401 >> v2:172.21.15.155:3300/0 conn(0x7fad7415e830 msgr2=0x7fad7406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.103+0000 7fad7a77e640 1 --2- 172.21.15.70:0/2436012401 >> v2:172.21.15.155:3300/0 conn(0x7fad7415e830 0x7fad7406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.103+0000 7fad7a77e640 1 -- 172.21.15.70:0/2436012401 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7406baa0 con 0x7fad7415c6d0 2023-08-25T09:41:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.104+0000 7fad7a77e640 1 --2- 172.21.15.70:0/2436012401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad740b1ff0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fad7406bf10 tx=0x7fad6c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afd9f0e95b60b45 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.104+0000 7fad78f7b640 1 -- 172.21.15.70:0/2436012401 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c031870 con 0x7fad7415c6d0 2023-08-25T09:41:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.104+0000 7fad78f7b640 1 -- 172.21.15.70:0/2436012401 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c0319d0 con 0x7fad7415c6d0 2023-08-25T09:41:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.104+0000 7fad78f7b640 1 -- 172.21.15.70:0/2436012401 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c031d60 con 0x7fad7415c6d0 2023-08-25T09:41:03.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 -- 172.21.15.70:0/2436012401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 msgr2=0x7fad740b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/2436012401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad740b1ff0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fad7406bf10 tx=0x7fad6c02b910 comp rx=0 tx=0).stop 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 -- 172.21.15.70:0/2436012401 shutdown_connections 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/2436012401 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad740b1ff0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/2436012401 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7406b2b0 0x7fad7415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/2436012401 >> v2:172.21.15.155:3300/0 conn(0x7fad7415e830 0x7fad7406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 -- 172.21.15.70:0/2436012401 >> 172.21.15.70:0/2436012401 conn(0x7fad74069850 msgr2=0x7fad74069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:03.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 -- 172.21.15.70:0/2436012401 shutdown_connections 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.105+0000 7fad7c1e1640 1 -- 172.21.15.70:0/2436012401 wait complete. 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.107+0000 7fad7c1e1640 1 Processor -- start 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.107+0000 7fad7c1e1640 1 -- start start 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7c1e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad7406b2b0 0x7fad741e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7c1e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad741e81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7c1e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7415e830 0x7fad741e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7c1e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad741e8bb0 con 0x7fad7415c6d0 2023-08-25T09:41:03.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7c1e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad741e8e90 con 0x7fad7406b2b0 2023-08-25T09:41:03.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7c1e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad741e9170 con 0x7fad7415e830 2023-08-25T09:41:03.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad741e81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad741e81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37668/0 (socket says 172.21.15.70:37668) 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 -- 172.21.15.70:0/3998368701 learned_addr learned my addr 172.21.15.70:0/3998368701 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad79f7d640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad7406b2b0 0x7fad741e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7a77e640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7415e830 0x7fad741e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 -- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7415e830 msgr2=0x7fad741e4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7415e830 0x7fad741e4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 -- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad7406b2b0 msgr2=0x7fad741e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad7406b2b0 0x7fad741e69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7977c640 1 -- 172.21.15.70:0/3998368701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0094d0 con 0x7fad7415c6d0 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.108+0000 7fad7a77e640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7415e830 0x7fad741e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.109+0000 7fad7977c640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad741e81b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c7f0 tx=0x7fad6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.109+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68004220 con 0x7fad7415c6d0 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.109+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad68004380 con 0x7fad7415c6d0 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.109+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad68007860 con 0x7fad7415c6d0 2023-08-25T09:41:03.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.109+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad741e4360 con 0x7fad7415c6d0 2023-08-25T09:41:03.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.109+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad741e86f0 con 0x7fad7415c6d0 2023-08-25T09:41:03.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.111+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad741e4880 con 0x7fad7415c6d0 2023-08-25T09:41:03.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.111+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fad68007bd0 con 0x7fad7415c6d0 2023-08-25T09:41:03.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.112+0000 7fad727fc640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad6006a050 0x7fad6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.112+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fad68088240 con 0x7fad7415c6d0 2023-08-25T09:41:03.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.113+0000 7fad79f7d640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad6006a050 0x7fad6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.113+0000 7fad79f7d640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad6006a050 0x7fad6006c510 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad640099c0 tx=0x7fad64003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:03.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.117+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fad68053bb0 con 0x7fad7415c6d0 2023-08-25T09:41:03.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.388+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fad741e8910 con 0x7fad7415c6d0 2023-08-25T09:41:03.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.424+0000 7fad727fc640 1 -- 172.21.15.70:0/3998368701 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v141) v1 ==== 130+0+0 (secure 0 0 0) 0x7fad680571d0 con 0x7fad7415c6d0 2023-08-25T09:41:03.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad6006a050 msgr2=0x7fad6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad6006a050 0x7fad6006c510 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fad640099c0 tx=0x7fad64003990 comp rx=0 tx=0).stop 2023-08-25T09:41:03.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 msgr2=0x7fad741e81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad741e81b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fad6800c7f0 tx=0x7fad6800ccb0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 shutdown_connections 2023-08-25T09:41:03.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad6006a050 0x7fad6006c510 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad7415e830 0x7fad741e4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7415c6d0 0x7fad741e81b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 --2- 172.21.15.70:0/3998368701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad7406b2b0 0x7fad741e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 >> 172.21.15.70:0/3998368701 conn(0x7fad74069850 msgr2=0x7fad740ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 shutdown_connections 2023-08-25T09:41:03.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.427+0000 7fad7c1e1640 1 -- 172.21.15.70:0/3998368701 wait complete. 2023-08-25T09:41:03.448 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-08-25T09:41:03.448 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-08-25T09:41:03.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:41:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.663+0000 7f883ff79640 1 Processor -- start 2023-08-25T09:41:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.663+0000 7f883ff79640 1 -- start start 2023-08-25T09:41:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.665+0000 7f883ff79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815ec10 0x7f883815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.665+0000 7f883ff79640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f883815c6d0 0x7f88380b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.665+0000 7f883ff79640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88380b1b60 con 0x7f883815e830 2023-08-25T09:41:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.665+0000 7f883ff79640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883806bd80 con 0x7f883815c6d0 2023-08-25T09:41:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.665+0000 7f883ff79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883806b7c0 con 0x7f883815ec10 2023-08-25T09:41:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f883815c6d0 0x7f88380b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f883815c6d0 0x7f88380b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:45276/0 (socket says 172.21.15.70:45276) 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 -- 172.21.15.70:0/450106451 learned_addr learned my addr 172.21.15.70:0/450106451 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883d514640 1 --2- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815ec10 0x7f883815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 -- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815ec10 msgr2=0x7f883815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 --2- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815ec10 0x7f883815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 -- 172.21.15.70:0/450106451 >> v1:172.21.15.155:6789/0 conn(0x7f883815e830 legacy=0x7f883806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.666+0000 7f883e516640 1 -- 172.21.15.70:0/450106451 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883806baa0 con 0x7f883815c6d0 2023-08-25T09:41:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.668+0000 7f883e516640 1 --2- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815c6d0 0x7f88380b1ff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8830009e50 tx=0x7f883002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f07570f81ac55879 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.668+0000 7f883cd13640 1 -- 172.21.15.70:0/450106451 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883002ca70 con 0x7f883815c6d0 2023-08-25T09:41:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.668+0000 7f883cd13640 1 -- 172.21.15.70:0/450106451 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883002cbd0 con 0x7f883815c6d0 2023-08-25T09:41:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.668+0000 7f883cd13640 1 -- 172.21.15.70:0/450106451 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300343f0 con 0x7f883815c6d0 2023-08-25T09:41:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 -- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815c6d0 msgr2=0x7f88380b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 --2- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815c6d0 0x7f88380b1ff0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8830009e50 tx=0x7f883002bed0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 -- 172.21.15.70:0/450106451 shutdown_connections 2023-08-25T09:41:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 --2- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815c6d0 0x7f88380b1ff0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 --2- 172.21.15.70:0/450106451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815ec10 0x7f883815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 -- 172.21.15.70:0/450106451 >> 172.21.15.70:0/450106451 conn(0x7f8838069850 msgr2=0x7f8838069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 -- 172.21.15.70:0/450106451 shutdown_connections 2023-08-25T09:41:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.669+0000 7f883ff79640 1 -- 172.21.15.70:0/450106451 wait complete. 2023-08-25T09:41:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.671+0000 7f883ff79640 1 Processor -- start 2023-08-25T09:41:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.671+0000 7f883ff79640 1 -- start start 2023-08-25T09:41:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883ff79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815c6d0 0x7f88381f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883ff79640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 0x7f883814b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883ff79640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815ec10 0x7f883815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883ff79640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381f4470 con 0x7f883815e830 2023-08-25T09:41:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883ff79640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380ad660 con 0x7f883815ec10 2023-08-25T09:41:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883ff79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88380aaf70 con 0x7f883815c6d0 2023-08-25T09:41:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883d514640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 0x7f883814b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883d514640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 0x7f883814b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37682/0 (socket says 172.21.15.70:37682) 2023-08-25T09:41:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883d514640 1 -- 172.21.15.70:0/754719014 learned_addr learned my addr 172.21.15.70:0/754719014 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:03.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.672+0000 7f883e516640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815ec10 0x7f883815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883d514640 1 -- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815c6d0 msgr2=0x7f88381f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883d514640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815c6d0 0x7f88381f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883d514640 1 -- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815ec10 msgr2=0x7f883815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883d514640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815ec10 0x7f883815d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883d514640 1 -- 172.21.15.70:0/754719014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009b30 con 0x7f883815e830 2023-08-25T09:41:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883d514640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 0x7f883814b6a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f882800e8a0 tx=0x7f882800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280094f0 con 0x7f883815e830 2023-08-25T09:41:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882800c940 con 0x7f883815e830 2023-08-25T09:41:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800ccf0 con 0x7f883815e830 2023-08-25T09:41:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88380ab250 con 0x7f883815e830 2023-08-25T09:41:03.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.673+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88380ab810 con 0x7f883815e830 2023-08-25T09:41:03.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.674+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883804e420 con 0x7f883815e830 2023-08-25T09:41:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.675+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8828013470 con 0x7f883815e830 2023-08-25T09:41:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.676+0000 7f882e7fc640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f881c06a050 0x7f881c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:03.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.676+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f88280883a0 con 0x7f883815e830 2023-08-25T09:41:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.677+0000 7f883dd15640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f881c06a050 0x7f881c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.677+0000 7f883dd15640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f881c06a050 0x7f881c06c510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8834000f60 tx=0x7f88340079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:03.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.681+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8828053d10 con 0x7f883815e830 2023-08-25T09:41:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.873+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88380b17c0 con 0x7f883815e830 2023-08-25T09:41:03.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.875+0000 7f882e7fc640 1 -- 172.21.15.70:0/754719014 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14782 (secure 0 0 0) 0x7f8828057330 con 0x7f883815e830 2023-08-25T09:41:03.877 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:03.879 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":42,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:02.399903+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6835/396571742":"2023-08-26T09:40:59.390384+0000","172.21.15.155:6833/396571742":"2023-08-26T09:40:59.390384+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:41:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f881c06a050 msgr2=0x7f881c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f881c06a050 0x7f881c06c510 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8834000f60 tx=0x7f88340079b0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 msgr2=0x7f883814b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 0x7f883814b6a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f882800e8a0 tx=0x7f882800ed60 comp rx=0 tx=0).stop 2023-08-25T09:41:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 shutdown_connections 2023-08-25T09:41:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f881c06a050 0x7f881c06c510 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f883815ec10 0x7f883815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f883815e830 0x7f883814b6a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 --2- 172.21.15.70:0/754719014 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f883815c6d0 0x7f88381f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.877+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 >> 172.21.15.70:0/754719014 conn(0x7f8838069850 msgr2=0x7f88380b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.878+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 shutdown_connections 2023-08-25T09:41:03.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:03.878+0000 7f883ff79640 1 -- 172.21.15.70:0/754719014 wait complete. 2023-08-25T09:41:03.898 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:41:03.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:41:04.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.116+0000 7fc2dcd12640 1 Processor -- start 2023-08-25T09:41:04.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.117+0000 7fc2dcd12640 1 -- start start 2023-08-25T09:41:04.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.118+0000 7fc2dcd12640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2d80b0960 0x7fc2d80b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.118+0000 7fc2dcd12640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d80cb710 con 0x7fc2d806ae60 2023-08-25T09:41:04.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.118+0000 7fc2dcd12640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d80b1280 con 0x7fc2d80ce110 2023-08-25T09:41:04.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.118+0000 7fc2dcd12640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d80cecb0 con 0x7fc2d80b0960 2023-08-25T09:41:04.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.118+0000 7fc2d659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2d80b0960 0x7fc2d80b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.119+0000 7fc2d659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2d80b0960 0x7fc2d80b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37698/0 (socket says 172.21.15.70:37698) 2023-08-25T09:41:04.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.119+0000 7fc2d659c640 1 -- 172.21.15.70:0/229911720 learned_addr learned my addr 172.21.15.70:0/229911720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.119+0000 7fc2d6d9d640 1 --1- 172.21.15.70:0/229911720 >> v1:172.21.15.70:6789/0 conn(0x7fc2d806ae60 0x7fc2d806b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39908/0 (socket says 172.21.15.70:39908) 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.119+0000 7fc2d659c640 1 -- 172.21.15.70:0/229911720 >> v1:172.21.15.155:6789/0 conn(0x7fc2d80ce110 legacy=0x7fc2d80ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.119+0000 7fc2d659c640 1 -- 172.21.15.70:0/229911720 >> v1:172.21.15.70:6789/0 conn(0x7fc2d806ae60 legacy=0x7fc2d806b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.119+0000 7fc2d659c640 1 -- 172.21.15.70:0/229911720 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80ccb50 con 0x7fc2d80b0960 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.120+0000 7fc2d659c640 1 --2- 172.21.15.70:0/229911720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b0d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009e50 tx=0x7fc2c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7be49b75cd716f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.120+0000 7fc2d5d9b640 1 -- 172.21.15.70:0/229911720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c802ca80 con 0x7fc2d80b0960 2023-08-25T09:41:04.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.120+0000 7fc2d5d9b640 1 -- 172.21.15.70:0/229911720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c802cbe0 con 0x7fc2d80b0960 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.120+0000 7fc2d5d9b640 1 -- 172.21.15.70:0/229911720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80343f0 con 0x7fc2d80b0960 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 -- 172.21.15.70:0/229911720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80b0960 msgr2=0x7fc2d80b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/229911720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b0d40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8009e50 tx=0x7fc2c802bee0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 -- 172.21.15.70:0/229911720 shutdown_connections 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/229911720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b0d40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 -- 172.21.15.70:0/229911720 >> 172.21.15.70:0/229911720 conn(0x7fc2d80cb960 msgr2=0x7fc2d80cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 -- 172.21.15.70:0/229911720 shutdown_connections 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.121+0000 7fc2dcd12640 1 -- 172.21.15.70:0/229911720 wait complete. 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.123+0000 7fc2dcd12640 1 Processor -- start 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.123+0000 7fc2dcd12640 1 -- start start 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.124+0000 7fc2dcd12640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d806ae60 0x7fc2d814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.124+0000 7fc2dcd12640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.124+0000 7fc2dcd12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 0x7fc2d8070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.124+0000 7fc2dcd12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d81f5de0 con 0x7fc2d80ce110 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.124+0000 7fc2dcd12640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d81f6960 con 0x7fc2d80b0960 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.124+0000 7fc2dcd12640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d81f6030 con 0x7fc2d806ae60 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 0x7fc2d8070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 0x7fc2d8070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37700/0 (socket says 172.21.15.70:37700) 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 -- 172.21.15.70:0/3245726003 learned_addr learned my addr 172.21.15.70:0/3245726003 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2cdd9b640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 -- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d806ae60 msgr2=0x7fc2d814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d806ae60 0x7fc2d814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 -- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d80b0960 msgr2=0x7fc2d80b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 -- 172.21.15.70:0/3245726003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8009b30 con 0x7fc2d80ce110 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2cdd9b640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.125+0000 7fc2d6d9d640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 0x7fc2d8070b00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c400 tx=0x7fc2d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.126+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00074e0 con 0x7fc2d80ce110 2023-08-25T09:41:04.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.126+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0007640 con 0x7fc2d80ce110 2023-08-25T09:41:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.126+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00049e0 con 0x7fc2d80ce110 2023-08-25T09:41:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.126+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d81f6310 con 0x7fc2d80ce110 2023-08-25T09:41:04.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.126+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d814b750 con 0x7fc2d80ce110 2023-08-25T09:41:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.127+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d8066fb0 con 0x7fc2d80ce110 2023-08-25T09:41:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.128+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc2d0004b40 con 0x7fc2d80ce110 2023-08-25T09:41:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.129+0000 7fc2cf7fe640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc2b806a050 0x7fc2b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.129+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc2d0087c30 con 0x7fc2d80ce110 2023-08-25T09:41:04.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.129+0000 7fc2d659c640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc2b806a050 0x7fc2b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.130+0000 7fc2d659c640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc2b806a050 0x7fc2b806c510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8035000 tx=0x7fc2c802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.133+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc2d00536b0 con 0x7fc2d80ce110 2023-08-25T09:41:04.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.322+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2d81f6ad0 con 0x7fc2d80ce110 2023-08-25T09:41:04.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.323+0000 7fc2cf7fe640 1 -- 172.21.15.70:0/3245726003 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2d0056cd0 con 0x7fc2d80ce110 2023-08-25T09:41:04.325 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:41:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc2b806a050 msgr2=0x7fc2b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc2b806a050 0x7fc2b806c510 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8035000 tx=0x7fc2c802e000 comp rx=0 tx=0).stop 2023-08-25T09:41:04.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 msgr2=0x7fc2d8070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 0x7fc2d8070b00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000c400 tx=0x7fc2d000c8c0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 shutdown_connections 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc2b806a050 0x7fc2b806c510 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2d80ce110 0x7fc2d8070b00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2d80b0960 0x7fc2d80b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 --2- 172.21.15.70:0/3245726003 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2d806ae60 0x7fc2d814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.325+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 >> 172.21.15.70:0/3245726003 conn(0x7fc2d80cb960 msgr2=0x7fc2d80cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:04.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.326+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 shutdown_connections 2023-08-25T09:41:04.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.326+0000 7fc2dcd12640 1 -- 172.21.15.70:0/3245726003 wait complete. 2023-08-25T09:41:04.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:41:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.554+0000 7f8a6b12f640 1 Processor -- start 2023-08-25T09:41:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.554+0000 7f8a6b12f640 1 -- start start 2023-08-25T09:41:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.555+0000 7f8a6b12f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a64159bf0 0x7f8a6415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.555+0000 7f8a6b12f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8a6406f0e0 0x7f8a64070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.555+0000 7f8a6b12f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6415e780 con 0x7f8a640cd660 2023-08-25T09:41:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.555+0000 7f8a6b12f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6406f8e0 con 0x7f8a64159bf0 2023-08-25T09:41:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.555+0000 7f8a6b12f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a6406fbc0 con 0x7f8a6406f0e0 2023-08-25T09:41:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a64159bf0 0x7f8a6415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8a64159bf0 0x7f8a6415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37712/0 (socket says 172.21.15.70:37712) 2023-08-25T09:41:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 -- 172.21.15.70:0/4207511933 learned_addr learned my addr 172.21.15.70:0/4207511933 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 -- 172.21.15.70:0/4207511933 >> v2:172.21.15.155:3300/0 conn(0x7f8a6406f0e0 msgr2=0x7f8a64070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 --2- 172.21.15.70:0/4207511933 >> v2:172.21.15.155:3300/0 conn(0x7f8a6406f0e0 0x7f8a64070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 -- 172.21.15.70:0/4207511933 >> v1:172.21.15.155:6789/0 conn(0x7f8a640cd660 legacy=0x7f8a640c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.556+0000 7f8a68ecb640 1 -- 172.21.15.70:0/4207511933 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6406fea0 con 0x7f8a64159bf0 2023-08-25T09:41:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.557+0000 7f8a68ecb640 1 --2- 172.21.15.70:0/4207511933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a6415e240 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8a6406af90 tx=0x7f8a5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfee88b32aea829 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.557+0000 7f8a637fe640 1 -- 172.21.15.70:0/4207511933 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54031870 con 0x7f8a64159bf0 2023-08-25T09:41:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a637fe640 1 -- 172.21.15.70:0/4207511933 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a540319d0 con 0x7f8a64159bf0 2023-08-25T09:41:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a637fe640 1 -- 172.21.15.70:0/4207511933 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54031d60 con 0x7f8a64159bf0 2023-08-25T09:41:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 -- 172.21.15.70:0/4207511933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 msgr2=0x7f8a6415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/4207511933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a6415e240 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f8a6406af90 tx=0x7f8a5402b910 comp rx=0 tx=0).stop 2023-08-25T09:41:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 -- 172.21.15.70:0/4207511933 shutdown_connections 2023-08-25T09:41:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/4207511933 >> v2:172.21.15.155:3300/0 conn(0x7f8a6406f0e0 0x7f8a64070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/4207511933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a6415e240 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 -- 172.21.15.70:0/4207511933 >> 172.21.15.70:0/4207511933 conn(0x7f8a640cb0d0 msgr2=0x7f8a640cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 -- 172.21.15.70:0/4207511933 shutdown_connections 2023-08-25T09:41:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.558+0000 7f8a6b12f640 1 -- 172.21.15.70:0/4207511933 wait complete. 2023-08-25T09:41:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.560+0000 7f8a6b12f640 1 Processor -- start 2023-08-25T09:41:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.560+0000 7f8a6b12f640 1 -- start start 2023-08-25T09:41:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a6b12f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a6406f0e0 0x7f8a640c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a6b12f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a640cd660 0x7f8a6414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a6b12f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a64148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a6b12f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6414bb80 con 0x7f8a64159bf0 2023-08-25T09:41:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a6b12f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6414be60 con 0x7f8a6406f0e0 2023-08-25T09:41:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a6b12f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6414c140 con 0x7f8a640cd660 2023-08-25T09:41:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a696cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a64148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a696cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a64148a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37728/0 (socket says 172.21.15.70:37728) 2023-08-25T09:41:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.561+0000 7f8a696cc640 1 -- 172.21.15.70:0/3188314872 learned_addr learned my addr 172.21.15.70:0/3188314872 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a696cc640 1 -- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a640cd660 msgr2=0x7f8a6414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a696cc640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a640cd660 0x7f8a6414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a68ecb640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a6406f0e0 0x7f8a640c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a696cc640 1 -- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a6406f0e0 msgr2=0x7f8a640c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a696cc640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a6406f0e0 0x7f8a640c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a696cc640 1 -- 172.21.15.70:0/3188314872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540094d0 con 0x7f8a64159bf0 2023-08-25T09:41:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a696cc640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a64148a00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c780 tx=0x7f8a5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c003d80 con 0x7f8a64159bf0 2023-08-25T09:41:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00ce90 con 0x7f8a64159bf0 2023-08-25T09:41:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.562+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c0052b0 con 0x7f8a64159bf0 2023-08-25T09:41:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.563+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640c7730 con 0x7f8a64159bf0 2023-08-25T09:41:04.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.563+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a64148f40 con 0x7f8a64159bf0 2023-08-25T09:41:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.564+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a640c7920 con 0x7f8a64159bf0 2023-08-25T09:41:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.565+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8a5c005410 con 0x7f8a64159bf0 2023-08-25T09:41:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.565+0000 7f8a617fa640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a4406a050 0x7f8a4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.566+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8a5c087800 con 0x7f8a64159bf0 2023-08-25T09:41:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.566+0000 7f8a68ecb640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a4406a050 0x7f8a4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.566+0000 7f8a68ecb640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a4406a050 0x7f8a4406c510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a54036000 tx=0x7f8a5403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.570+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8a5c053170 con 0x7f8a64159bf0 2023-08-25T09:41:04.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.760+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8a64066fb0 con 0x7f8a64159bf0 2023-08-25T09:41:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.760+0000 7f8a617fa640 1 -- 172.21.15.70:0/3188314872 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8a5c017020 con 0x7f8a64159bf0 2023-08-25T09:41:04.762 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:41:04.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a4406a050 msgr2=0x7f8a4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a4406a050 0x7f8a4406c510 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a54036000 tx=0x7f8a5403c040 comp rx=0 tx=0).stop 2023-08-25T09:41:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 msgr2=0x7f8a64148a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a64148a00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c780 tx=0x7f8a5c00cc40 comp rx=0 tx=0).stop 2023-08-25T09:41:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 shutdown_connections 2023-08-25T09:41:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a4406a050 0x7f8a4406c510 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a64159bf0 0x7f8a64148a00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a640cd660 0x7f8a6414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 --2- 172.21.15.70:0/3188314872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8a6406f0e0 0x7f8a640c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 >> 172.21.15.70:0/3188314872 conn(0x7f8a640cb0d0 msgr2=0x7f8a640cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:04.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 shutdown_connections 2023-08-25T09:41:04.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.763+0000 7f8a6b12f640 1 -- 172.21.15.70:0/3188314872 wait complete. 2023-08-25T09:41:04.782 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-08-25T09:41:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.986+0000 7f704df55640 1 Processor -- start 2023-08-25T09:41:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.986+0000 7f704df55640 1 -- start start 2023-08-25T09:41:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f704df55640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f704806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f704df55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f704815ec10 0x7f70480b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f704df55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480b1b60 con 0x7f704806b2b0 2023-08-25T09:41:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f704df55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704806bd80 con 0x7f704815ec10 2023-08-25T09:41:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f704df55640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704806b7c0 con 0x7f704815e830 2023-08-25T09:41:04.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f7046ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f704806b2b0 0x7f704815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39924/0 (socket says 172.21.15.70:39924) 2023-08-25T09:41:04.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f7046ffd640 1 -- 172.21.15.70:0/3949294839 learned_addr learned my addr 172.21.15.70:0/3949294839 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:04.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f7047fff640 1 --2- 172.21.15.70:0/3949294839 >> v2:172.21.15.70:3300/0 conn(0x7f704815ec10 0x7f70480b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.988+0000 7f70477fe640 1 --2- 172.21.15.70:0/3949294839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f704806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:04.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.989+0000 7f70467fc640 1 -- 172.21.15.70:0/3949294839 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743911042 0 0) 0x7f70480b1b60 con 0x7f704806b2b0 2023-08-25T09:41:04.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.989+0000 7f7047fff640 1 -- 172.21.15.70:0/3949294839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 msgr2=0x7f704806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:04.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.989+0000 7f7047fff640 1 --2- 172.21.15.70:0/3949294839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f704806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.989+0000 7f7047fff640 1 -- 172.21.15.70:0/3949294839 >> v1:172.21.15.70:6789/0 conn(0x7f704806b2b0 legacy=0x7f704815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.989+0000 7f7047fff640 1 -- 172.21.15.70:0/3949294839 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704806baa0 con 0x7f704815ec10 2023-08-25T09:41:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.990+0000 7f7047fff640 1 --2- 172.21.15.70:0/3949294839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704815ec10 0x7f70480b1ff0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7040009d90 tx=0x7f704002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3f6328a612186f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.990+0000 7f70467fc640 1 -- 172.21.15.70:0/3949294839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704002cc50 con 0x7f704815ec10 2023-08-25T09:41:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.990+0000 7f70467fc640 1 -- 172.21.15.70:0/3949294839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704002cdb0 con 0x7f704815ec10 2023-08-25T09:41:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.990+0000 7f70467fc640 1 -- 172.21.15.70:0/3949294839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040034660 con 0x7f704815ec10 2023-08-25T09:41:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 -- 172.21.15.70:0/3949294839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704815ec10 msgr2=0x7f70480b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 --2- 172.21.15.70:0/3949294839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704815ec10 0x7f70480b1ff0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7040009d90 tx=0x7f704002bea0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 -- 172.21.15.70:0/3949294839 shutdown_connections 2023-08-25T09:41:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 --2- 172.21.15.70:0/3949294839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704815ec10 0x7f70480b1ff0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 --2- 172.21.15.70:0/3949294839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f704806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 -- 172.21.15.70:0/3949294839 >> 172.21.15.70:0/3949294839 conn(0x7f7048069850 msgr2=0x7f7048069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 -- 172.21.15.70:0/3949294839 shutdown_connections 2023-08-25T09:41:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.991+0000 7f704df55640 1 -- 172.21.15.70:0/3949294839 wait complete. 2023-08-25T09:41:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.992+0000 7f704df55640 1 Processor -- start 2023-08-25T09:41:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.993+0000 7f704df55640 1 -- start start 2023-08-25T09:41:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f704df55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 0x7f704814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f704df55640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f7048146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f704df55640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704815ec10 0x7f704806f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f704df55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048147c10 con 0x7f704806b2b0 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f704df55640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048146fd0 con 0x7f704815ec10 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f704df55640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704806fe50 con 0x7f704815e830 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 0x7f704814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 0x7f704814ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37746/0 (socket says 172.21.15.70:37746) 2023-08-25T09:41:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 -- 172.21.15.70:0/4196437165 learned_addr learned my addr 172.21.15.70:0/4196437165 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 -- 172.21.15.70:0/4196437165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 msgr2=0x7f7048146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f7048146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 -- 172.21.15.70:0/4196437165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704815ec10 msgr2=0x7f704806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704815ec10 0x7f704806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.994+0000 7f70477fe640 1 -- 172.21.15.70:0/4196437165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009a70 con 0x7f704806b2b0 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.995+0000 7f70477fe640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 0x7f704814ce70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f703800d800 tx=0x7f703800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.995+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703800b660 con 0x7f704806b2b0 2023-08-25T09:41:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.995+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703800b7c0 con 0x7f704806b2b0 2023-08-25T09:41:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.995+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703800bb50 con 0x7f704806b2b0 2023-08-25T09:41:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.995+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704815ca20 con 0x7f704806b2b0 2023-08-25T09:41:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.995+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7048147270 con 0x7f704806b2b0 2023-08-25T09:41:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.996+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704804e420 con 0x7f704806b2b0 2023-08-25T09:41:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.997+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f703800bcb0 con 0x7f704806b2b0 2023-08-25T09:41:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.998+0000 7f702ffff640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f703406a050 0x7f703406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.998+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f70380887b0 con 0x7f704806b2b0 2023-08-25T09:41:05.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.998+0000 7f7046ffd640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f703406a050 0x7f703406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:04.999+0000 7f7046ffd640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f703406a050 0x7f703406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f703c007fd0 tx=0x7f703c007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:05.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.002+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70380541b0 con 0x7f704806b2b0 2023-08-25T09:41:05.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.257+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f704806c620 con 0x7f704806b2b0 2023-08-25T09:41:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.429+0000 7f702ffff640 1 -- 172.21.15.70:0/4196437165 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v143) v1 ==== 145+0+0 (secure 0 0 0) 0x7f70380577d0 con 0x7f704806b2b0 2023-08-25T09:41:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.432+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f703406a050 msgr2=0x7f703406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.432+0000 7f704df55640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f703406a050 0x7f703406c510 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f703c007fd0 tx=0x7f703c007920 comp rx=0 tx=0).stop 2023-08-25T09:41:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.432+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 msgr2=0x7f704814ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.432+0000 7f704df55640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 0x7f704814ce70 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f703800d800 tx=0x7f703800dcc0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.432+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 shutdown_connections 2023-08-25T09:41:05.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.432+0000 7f704df55640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f703406a050 0x7f703406c510 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.433+0000 7f704df55640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f704815ec10 0x7f704806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.433+0000 7f704df55640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f704815e830 0x7f7048146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.433+0000 7f704df55640 1 --2- 172.21.15.70:0/4196437165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f704806b2b0 0x7f704814ce70 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.433+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 >> 172.21.15.70:0/4196437165 conn(0x7f7048069850 msgr2=0x7f70480cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.433+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 shutdown_connections 2023-08-25T09:41:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.433+0000 7f704df55640 1 -- 172.21.15.70:0/4196437165 wait complete. 2023-08-25T09:41:05.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-08-25T09:41:05.679 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_objectstore":"memstore"} 2023-08-25T09:41:05.679 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-08-25T09:41:05.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:41:05.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.885+0000 7f37b4271640 1 Processor -- start 2023-08-25T09:41:05.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.885+0000 7f37b4271640 1 -- start start 2023-08-25T09:41:05.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b4271640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 0x7f37ac070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b4271640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f37ac0cd660 0x7f37ac0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b4271640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0ce000 con 0x7f37ac159bf0 2023-08-25T09:41:05.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b4271640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac06f8e0 con 0x7f37ac0cd660 2023-08-25T09:41:05.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b4271640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac06fbc0 con 0x7f37ac15e240 2023-08-25T09:41:05.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b200d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f37ac159bf0 0x7f37ac06f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39932/0 (socket says 172.21.15.70:39932) 2023-08-25T09:41:05.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.887+0000 7f37b200d640 1 -- 172.21.15.70:0/1305947985 learned_addr learned my addr 172.21.15.70:0/1305947985 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:05.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b280e640 1 --2- 172.21.15.70:0/1305947985 >> v2:172.21.15.155:3300/0 conn(0x7f37ac0cd660 0x7f37ac0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480414851 0 0) 0x7f37ac0ce000 con 0x7f37ac159bf0 2023-08-25T09:41:05.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b180c640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 0x7f37ac070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3798005730 con 0x7f37ac159bf0 2023-08-25T09:41:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3111598828 0 0) 0x7f3798005730 con 0x7f37ac159bf0 2023-08-25T09:41:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 msgr2=0x7f37ac070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 0x7f37ac070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 >> v2:172.21.15.155:3300/0 conn(0x7f37ac0cd660 msgr2=0x7f37ac0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 --2- 172.21.15.70:0/1305947985 >> v2:172.21.15.155:3300/0 conn(0x7f37ac0cd660 0x7f37ac0cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac06fea0 con 0x7f37ac159bf0 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f37a8001670 con 0x7f37ac159bf0 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 >> v1:172.21.15.70:6789/0 conn(0x7f37ac159bf0 legacy=0x7f37ac06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 0x7f3798005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3798006100 0x7f37980069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3798006f10 0x7f3798007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac06f8e0 con 0x7f3798006f10 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac06fbc0 con 0x7f3798006100 2023-08-25T09:41:05.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.888+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379800db90 con 0x7f37ac15e240 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b280e640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3798006f10 0x7f3798007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b200d640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 0x7f3798005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b280e640 1 -- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 msgr2=0x7f3798005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b280e640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f37ac15e240 0x7f3798005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b280e640 1 -- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3798006100 msgr2=0x7f37980069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b280e640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3798006100 0x7f37980069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.889+0000 7f37b280e640 1 -- 172.21.15.70:0/1305947985 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798005730 con 0x7f3798006f10 2023-08-25T09:41:05.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.890+0000 7f37b280e640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3798006f10 0x7f3798007390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fc0 tx=0x7f3798007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980b05f1973cb3cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:05.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.890+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a402cc40 con 0x7f3798006f10 2023-08-25T09:41:05.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.890+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a400f040 con 0x7f3798006f10 2023-08-25T09:41:05.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.890+0000 7f37b100b640 1 -- 172.21.15.70:0/1305947985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4034440 con 0x7f3798006f10 2023-08-25T09:41:05.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b4271640 1 -- 172.21.15.70:0/1305947985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3798006f10 msgr2=0x7f3798007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b4271640 1 --2- 172.21.15.70:0/1305947985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3798006f10 0x7f3798007390 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009fc0 tx=0x7f3798007a00 comp rx=0 tx=0).stop 2023-08-25T09:41:05.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b280e640 1 -- 172.21.15.70:0/1305947985 reap_dead start 2023-08-25T09:41:05.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b4271640 1 -- 172.21.15.70:0/1305947985 shutdown_connections 2023-08-25T09:41:05.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b4271640 1 -- 172.21.15.70:0/1305947985 >> 172.21.15.70:0/1305947985 conn(0x7f37ac0cb0d0 msgr2=0x7f37ac0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:05.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b4271640 1 -- 172.21.15.70:0/1305947985 shutdown_connections 2023-08-25T09:41:05.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.891+0000 7f37b4271640 1 -- 172.21.15.70:0/1305947985 wait complete. 2023-08-25T09:41:05.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.893+0000 7f37b4271640 1 Processor -- start 2023-08-25T09:41:05.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.893+0000 7f37b4271640 1 -- start start 2023-08-25T09:41:05.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.894+0000 7f37b4271640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3798006f10 0x7f37ac1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.894+0000 7f37b4271640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 0x7f37ac0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.894+0000 7f37b4271640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac1f5b20 0x7f37ac1f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.894+0000 7f37b4271640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0b02a0 con 0x7f37ac15dc30 2023-08-25T09:41:05.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.894+0000 7f37b4271640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1f64e0 con 0x7f37ac1f5b20 2023-08-25T09:41:05.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.894+0000 7f37b4271640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0b0b60 con 0x7f3798006f10 2023-08-25T09:41:05.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 0x7f37ac0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 0x7f37ac0afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37756/0 (socket says 172.21.15.70:37756) 2023-08-25T09:41:05.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 -- 172.21.15.70:0/3049053549 learned_addr learned my addr 172.21.15.70:0/3049053549 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b280e640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac1f5b20 0x7f37ac1f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 -- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3798006f10 msgr2=0x7f37ac1f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b200d640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3798006f10 0x7f37ac1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3798006f10 0x7f37ac1f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 -- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac1f5b20 msgr2=0x7f37ac1f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac1f5b20 0x7f37ac1f5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 -- 172.21.15.70:0/3049053549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0b05a0 con 0x7f37ac15dc30 2023-08-25T09:41:05.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.895+0000 7f37b180c640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 0x7f37ac0afa50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e7c0 tx=0x7f379c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.896+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00eed0 con 0x7f37ac15dc30 2023-08-25T09:41:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.896+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00c8c0 con 0x7f37ac15dc30 2023-08-25T09:41:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.896+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00cc70 con 0x7f37ac15dc30 2023-08-25T09:41:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.896+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac0b0880 con 0x7f37ac15dc30 2023-08-25T09:41:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.896+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac0aff90 con 0x7f37ac15dc30 2023-08-25T09:41:05.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.897+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac066fb0 con 0x7f37ac15dc30 2023-08-25T09:41:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.898+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f379c013470 con 0x7f37ac15dc30 2023-08-25T09:41:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.899+0000 7f37a2ffd640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f379006a050 0x7f379006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.899+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f379c0883f0 con 0x7f37ac15dc30 2023-08-25T09:41:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.899+0000 7f37b200d640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f379006a050 0x7f379006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:05.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.900+0000 7f37b200d640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f379006a050 0x7f379006c510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f37a8001050 tx=0x7f37a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:05.903+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f379c053d60 con 0x7f37ac15dc30 2023-08-25T09:41:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.092+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f37ac0cd1b0 con 0x7f37ac15dc30 2023-08-25T09:41:06.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.099+0000 7f37a2ffd640 1 -- 172.21.15.70:0/3049053549 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f379c057380 con 0x7f37ac15dc30 2023-08-25T09:41:06.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f379006a050 msgr2=0x7f379006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:06.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f379006a050 0x7f379006c510 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f37a8001050 tx=0x7f37a8009040 comp rx=0 tx=0).stop 2023-08-25T09:41:06.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 msgr2=0x7f37ac0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:06.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 0x7f37ac0afa50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f379c00e7c0 tx=0x7f379c00ec80 comp rx=0 tx=0).stop 2023-08-25T09:41:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 shutdown_connections 2023-08-25T09:41:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f379006a050 0x7f379006c510 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37ac1f5b20 0x7f37ac1f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37ac15dc30 0x7f37ac0afa50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 --2- 172.21.15.70:0/3049053549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3798006f10 0x7f37ac1f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 >> 172.21.15.70:0/3049053549 conn(0x7f37ac0cb0d0 msgr2=0x7f37ac158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:06.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 shutdown_connections 2023-08-25T09:41:06.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.102+0000 7f37b4271640 1 -- 172.21.15.70:0/3049053549 wait complete. 2023-08-25T09:41:06.122 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:41:06.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:41:06.236 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:41:06.368 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:41:06.369 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:06.369 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:06.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:41:06.430 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:41:06.430 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:06.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:41:06.491 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:06.491 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:06.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:41:06.550 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:41:06.550 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:06.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:41:06.610 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:41:06.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:41:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.831+0000 7f6557e2d640 1 Processor -- start 2023-08-25T09:41:06.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.831+0000 7f6557e2d640 1 -- start start 2023-08-25T09:41:06.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.833+0000 7f6557e2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65500ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.833+0000 7f6557e2d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65500ac870 0x7f65500ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.833+0000 7f6557e2d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f65500b3880 0x7f6550160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.833+0000 7f6557e2d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655006f050 con 0x7f65500ac870 2023-08-25T09:41:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.833+0000 7f6557e2d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655006f330 con 0x7f65500b3880 2023-08-25T09:41:06.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.833+0000 7f6557e2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655006f610 con 0x7f655015cb40 2023-08-25T09:41:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65500ac870 0x7f65500ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f65500ac870 0x7f65500ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37764/0 (socket says 172.21.15.70:37764) 2023-08-25T09:41:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 -- 172.21.15.70:0/3934086059 learned_addr learned my addr 172.21.15.70:0/3934086059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65563ca640 1 --2- 172.21.15.70:0/3934086059 >> v2:172.21.15.155:3300/0 conn(0x7f65500b3880 0x7f6550160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f6555bc9640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65500ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 -- 172.21.15.70:0/3934086059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 msgr2=0x7f65500ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65500ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 -- 172.21.15.70:0/3934086059 >> v2:172.21.15.155:3300/0 conn(0x7f65500b3880 msgr2=0x7f6550160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 --2- 172.21.15.70:0/3934086059 >> v2:172.21.15.155:3300/0 conn(0x7f65500b3880 0x7f6550160790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f65553c8640 1 -- 172.21.15.70:0/3934086059 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655006f8f0 con 0x7f65500ac870 2023-08-25T09:41:06.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.834+0000 7f6555bc9640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65500ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:06.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.836+0000 7f65553c8640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500ac870 0x7f65500ce860 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f65500adbd0 tx=0x7f654002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61928bc931e5ef6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.836+0000 7f6554bc7640 1 -- 172.21.15.70:0/3934086059 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540031880 con 0x7f65500ac870 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.836+0000 7f6554bc7640 1 -- 172.21.15.70:0/3934086059 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65400319e0 con 0x7f65500ac870 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.836+0000 7f6554bc7640 1 -- 172.21.15.70:0/3934086059 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540031d70 con 0x7f65500ac870 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 -- 172.21.15.70:0/3934086059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500ac870 msgr2=0x7f65500ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500ac870 0x7f65500ce860 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f65500adbd0 tx=0x7f654002b920 comp rx=0 tx=0).stop 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 -- 172.21.15.70:0/3934086059 shutdown_connections 2023-08-25T09:41:06.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3934086059 >> v2:172.21.15.155:3300/0 conn(0x7f65500b3880 0x7f6550160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500ac870 0x7f65500ce860 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3934086059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65500ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 -- 172.21.15.70:0/3934086059 >> 172.21.15.70:0/3934086059 conn(0x7f6550069850 msgr2=0x7f6550069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 -- 172.21.15.70:0/3934086059 shutdown_connections 2023-08-25T09:41:06.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.837+0000 7f6557e2d640 1 -- 172.21.15.70:0/3934086059 wait complete. 2023-08-25T09:41:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.839+0000 7f6557e2d640 1 Processor -- start 2023-08-25T09:41:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.840+0000 7f6557e2d640 1 -- start start 2023-08-25T09:41:06.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f6557e2d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65500ac870 0x7f6550156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f6557e2d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 0x7f65501f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f6557e2d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65501f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f6557e2d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655015fa30 con 0x7f65500b3880 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f6557e2d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65501599a0 con 0x7f65500ac870 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f6557e2d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65501570f0 con 0x7f655015cb40 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f65553c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 0x7f65501f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f65553c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 0x7f65501f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37778/0 (socket says 172.21.15.70:37778) 2023-08-25T09:41:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.841+0000 7f65553c8640 1 -- 172.21.15.70:0/3295783012 learned_addr learned my addr 172.21.15.70:0/3295783012 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f6555bc9640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65500ac870 0x7f6550156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65553c8640 1 -- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 msgr2=0x7f65501f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65563ca640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65501f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65553c8640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65501f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65553c8640 1 -- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65500ac870 msgr2=0x7f6550156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65553c8640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65500ac870 0x7f6550156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65553c8640 1 -- 172.21.15.70:0/3295783012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65400094d0 con 0x7f65500b3880 2023-08-25T09:41:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f6555bc9640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65500ac870 0x7f6550156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65553c8640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 0x7f65501f56b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f654002b960 tx=0x7f65400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654003ad30 con 0x7f65500b3880 2023-08-25T09:41:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65400443f0 con 0x7f65500b3880 2023-08-25T09:41:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400447a0 con 0x7f65500b3880 2023-08-25T09:41:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.842+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655015f750 con 0x7f65500b3880 2023-08-25T09:41:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.843+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655015e0e0 con 0x7f65500b3880 2023-08-25T09:41:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.844+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655015dc60 con 0x7f65500b3880 2023-08-25T09:41:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.844+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f654004d450 con 0x7f65500b3880 2023-08-25T09:41:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.846+0000 7f65467fc640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f653406a050 0x7f653406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.846+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f65400aa890 con 0x7f65500b3880 2023-08-25T09:41:06.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.846+0000 7f6555bc9640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f653406a050 0x7f653406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:06.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.847+0000 7f6555bc9640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f653406a050 0x7f653406c510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6550160210 tx=0x7f654c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:06.850+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f655015dc60 con 0x7f65500b3880 2023-08-25T09:41:07.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.074+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f655015dc60 con 0x7f65500b3880 2023-08-25T09:41:07.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.075+0000 7f65467fc640 1 -- 172.21.15.70:0/3295783012 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f655015dc60 con 0x7f65500b3880 2023-08-25T09:41:07.077 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:07.077 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4592,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4194266154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4194266154},{"type":"v1","addr":"172.21.15.70:6836","nonce":4194266154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4774,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4074047155","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4074047155},{"type":"v1","addr":"172.21.15.155:6834","nonce":4074047155}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5355,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/965866040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":965866040},{"type":"v1","addr":"172.21.15.70:6837","nonce":965866040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2284384321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":2284384321},{"type":"v1","addr":"172.21.15.155:6835","nonce":2284384321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[]} 2023-08-25T09:41:07.077 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 18 2023-08-25T09:41:07.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.077+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f653406a050 msgr2=0x7f653406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.077+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f653406a050 0x7f653406c510 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6550160210 tx=0x7f654c008040 comp rx=0 tx=0).stop 2023-08-25T09:41:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.077+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 msgr2=0x7f65501f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:07.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.077+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 0x7f65501f56b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f654002b960 tx=0x7f65400046c0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 shutdown_connections 2023-08-25T09:41:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f653406a050 0x7f653406c510 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f655015cb40 0x7f65501f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65500b3880 0x7f65501f56b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 --2- 172.21.15.70:0/3295783012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65500ac870 0x7f6550156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 >> 172.21.15.70:0/3295783012 conn(0x7f6550069850 msgr2=0x7f65500acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 shutdown_connections 2023-08-25T09:41:07.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.078+0000 7f6557e2d640 1 -- 172.21.15.70:0/3295783012 wait complete. 2023-08-25T09:41:07.099 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:41:07.099 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:41:07.100 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:41:07.100 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:41:07.101 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:41:07.101 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:41:07.101 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:41:07.101 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:41:07.102 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:41:07.102 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:41:07.102 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:41:07.102 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:41:07.102 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:41:07.103 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:41:07.103 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:41:07.103 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:41:07.117 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:41:07.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:41:07.120 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:41:07.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:41:07.124 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:41:07.125 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:41:07.132 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:41:07.132 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:41:07.168 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:41:07.168 DEBUG:teuthology.parallel:result is None 2023-08-25T09:41:07.176 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:41:07.176 DEBUG:teuthology.parallel:result is None 2023-08-25T09:41:07.180 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:41:07.181 DEBUG:teuthology.parallel:result is None 2023-08-25T09:41:07.187 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:41:07.188 DEBUG:teuthology.parallel:result is None 2023-08-25T09:41:07.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:41:07.330 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:41:07.331 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:41:07.352 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:41:07.363 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:41:07.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.386+0000 7f18c3fff640 1 Processor -- start 2023-08-25T09:41:07.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.386+0000 7f18c3fff640 1 -- start start 2023-08-25T09:41:07.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.387+0000 7f18c3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18c40b7df0 0x7f18c4070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:07.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.387+0000 7f18c3fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f18c415cf00 0x7f18c415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:07.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.387+0000 7f18c3fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c40b81d0 con 0x7f18c40cf210 2023-08-25T09:41:07.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.387+0000 7f18c3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c41604f0 con 0x7f18c40b7df0 2023-08-25T09:41:07.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.387+0000 7f18c3fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c415d3c0 con 0x7f18c415cf00 2023-08-25T09:41:07.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.387+0000 7f18c2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18c40b7df0 0x7f18c4070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:07.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18c40b7df0 0x7f18c4070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37816/0 (socket says 172.21.15.70:37816) 2023-08-25T09:41:07.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c2ffd640 1 -- 172.21.15.70:0/411105127 learned_addr learned my addr 172.21.15.70:0/411105127 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:07.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c27fc640 1 --2- 172.21.15.70:0/411105127 >> v2:172.21.15.155:3300/0 conn(0x7f18c415cf00 0x7f18c415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:07.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c2ffd640 1 -- 172.21.15.70:0/411105127 >> v2:172.21.15.155:3300/0 conn(0x7f18c415cf00 msgr2=0x7f18c415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:07.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/411105127 >> v2:172.21.15.155:3300/0 conn(0x7f18c415cf00 0x7f18c415ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c2ffd640 1 -- 172.21.15.70:0/411105127 >> v1:172.21.15.155:6789/0 conn(0x7f18c40cf210 legacy=0x7f18c40cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:07.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.388+0000 7f18c2ffd640 1 -- 172.21.15.70:0/411105127 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c415d6a0 con 0x7f18c40b7df0 2023-08-25T09:41:07.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.389+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/411105127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40b7df0 0x7f18c4070880 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f18c40b22e0 tx=0x7f18b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0383f89d8c90fe7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:07.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.389+0000 7f18c1ffb640 1 -- 172.21.15.70:0/411105127 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4031870 con 0x7f18c40b7df0 2023-08-25T09:41:07.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.389+0000 7f18c1ffb640 1 -- 172.21.15.70:0/411105127 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b40319d0 con 0x7f18c40b7df0 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.389+0000 7f18c1ffb640 1 -- 172.21.15.70:0/411105127 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b4031d60 con 0x7f18c40b7df0 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 -- 172.21.15.70:0/411105127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40b7df0 msgr2=0x7f18c4070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 --2- 172.21.15.70:0/411105127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40b7df0 0x7f18c4070880 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f18c40b22e0 tx=0x7f18b402b910 comp rx=0 tx=0).stop 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 -- 172.21.15.70:0/411105127 shutdown_connections 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 --2- 172.21.15.70:0/411105127 >> v2:172.21.15.155:3300/0 conn(0x7f18c415cf00 0x7f18c415ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 --2- 172.21.15.70:0/411105127 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40b7df0 0x7f18c4070880 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 -- 172.21.15.70:0/411105127 >> 172.21.15.70:0/411105127 conn(0x7f18c40b43a0 msgr2=0x7f18c40b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:07.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 -- 172.21.15.70:0/411105127 shutdown_connections 2023-08-25T09:41:07.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.390+0000 7f18c3fff640 1 -- 172.21.15.70:0/411105127 wait complete. 2023-08-25T09:41:07.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.392+0000 7f18c3fff640 1 Processor -- start 2023-08-25T09:41:07.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.392+0000 7f18c3fff640 1 -- start start 2023-08-25T09:41:07.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.393+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c40b7df0 0x7f18c4154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.393+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 0x7f18c415f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.393+0000 7f18c3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c415cf00 0x7f18c4156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.393+0000 7f18c3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c4155250 con 0x7f18c40cf210 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.393+0000 7f18c3fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c415f980 con 0x7f18c415cf00 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.393+0000 7f18c3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c415e3a0 con 0x7f18c40b7df0 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 0x7f18c415f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:07.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 0x7f18c415f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37824/0 (socket says 172.21.15.70:37824) 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 -- 172.21.15.70:0/926970791 learned_addr learned my addr 172.21.15.70:0/926970791 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c40b7df0 0x7f18c4154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c37fe640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c415cf00 0x7f18c4156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 -- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c40b7df0 msgr2=0x7f18c4154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c40b7df0 0x7f18c4154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 -- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c415cf00 msgr2=0x7f18c4156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c415cf00 0x7f18c4156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:07.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.394+0000 7f18c27fc640 1 -- 172.21.15.70:0/926970791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b40094d0 con 0x7f18c40cf210 2023-08-25T09:41:07.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.395+0000 7f18c27fc640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 0x7f18c415f440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e980 tx=0x7f18b800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:07.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.395+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8009670 con 0x7f18c40cf210 2023-08-25T09:41:07.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.395+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b800ca00 con 0x7f18c40cf210 2023-08-25T09:41:07.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.395+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b800cd90 con 0x7f18c40cf210 2023-08-25T09:41:07.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.395+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c415e680 con 0x7f18c40cf210 2023-08-25T09:41:07.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.395+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c415ec40 con 0x7f18c40cf210 2023-08-25T09:41:07.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.396+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c4066fb0 con 0x7f18c40cf210 2023-08-25T09:41:07.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.397+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18b8013470 con 0x7f18c40cf210 2023-08-25T09:41:07.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.398+0000 7f18abfff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:07.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.398+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f18b8088470 con 0x7f18c40cf210 2023-08-25T09:41:07.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.399+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:07.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.399+0000 7f18c2ffd640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f18b4036000 tx=0x7f18b403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:07.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.403+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18b8053e90 con 0x7f18c40cf210 2023-08-25T09:41:07.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:07.610+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f18c40ccc10 con 0x7f18c40cf210 2023-08-25T09:41:08.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.458+0000 7f18abfff640 1 -- 172.21.15.70:0/926970791 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v47) v1 ==== 97+0+0 (secure 0 0 0) 0x7f18b80574b0 con 0x7f18c40cf210 2023-08-25T09:41:08.461 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:41:08.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 msgr2=0x7f18b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f18b4036000 tx=0x7f18b403c040 comp rx=0 tx=0).stop 2023-08-25T09:41:08.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 msgr2=0x7f18c415f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 0x7f18c415f440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e980 tx=0x7f18b800ee40 comp rx=0 tx=0).stop 2023-08-25T09:41:08.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 shutdown_connections 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18b006a050 0x7f18b006c510 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c415cf00 0x7f18c4156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18c40cf210 0x7f18c415f440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 --2- 172.21.15.70:0/926970791 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c40b7df0 0x7f18c4154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 >> 172.21.15.70:0/926970791 conn(0x7f18c40b43a0 msgr2=0x7f18c4158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 shutdown_connections 2023-08-25T09:41:08.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.462+0000 7f18c3fff640 1 -- 172.21.15.70:0/926970791 wait complete. 2023-08-25T09:41:08.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:41:08.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.687+0000 7f353e425640 1 Processor -- start 2023-08-25T09:41:08.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.687+0000 7f353e425640 1 -- start start 2023-08-25T09:41:08.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.688+0000 7f353e425640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3538159840 0x7f35380b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:08.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.688+0000 7f353e425640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35380b4450 con 0x7f35380cadb0 2023-08-25T09:41:08.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353e425640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f353815d890 con 0x7f353815d3d0 2023-08-25T09:41:08.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353e425640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35380c8b60 con 0x7f3538159840 2023-08-25T09:41:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353d423640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3538159840 0x7f35380b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353d423640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3538159840 0x7f35380b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37826/0 (socket says 172.21.15.70:37826) 2023-08-25T09:41:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353d423640 1 -- 172.21.15.70:0/3108977905 learned_addr learned my addr 172.21.15.70:0/3108977905 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353d423640 1 -- 172.21.15.70:0/3108977905 >> v1:172.21.15.155:6789/0 conn(0x7f353815d3d0 legacy=0x7f35380b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353d423640 1 -- 172.21.15.70:0/3108977905 >> v1:172.21.15.70:6789/0 conn(0x7f35380cadb0 legacy=0x7f35380cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.689+0000 7f353d423640 1 -- 172.21.15.70:0/3108977905 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353815db70 con 0x7f3538159840 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.690+0000 7f353d423640 1 --2- 172.21.15.70:0/3108977905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380b2390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3534009870 tx=0x7f353402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2a4087c13fc520 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f352ffff640 1 -- 172.21.15.70:0/3108977905 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534030860 con 0x7f3538159840 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f352ffff640 1 -- 172.21.15.70:0/3108977905 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35340309c0 con 0x7f3538159840 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f352ffff640 1 -- 172.21.15.70:0/3108977905 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534030d50 con 0x7f3538159840 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 -- 172.21.15.70:0/3108977905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 msgr2=0x7f35380b2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 --2- 172.21.15.70:0/3108977905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380b2390 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f3534009870 tx=0x7f353402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 -- 172.21.15.70:0/3108977905 shutdown_connections 2023-08-25T09:41:08.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 --2- 172.21.15.70:0/3108977905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380b2390 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 -- 172.21.15.70:0/3108977905 >> 172.21.15.70:0/3108977905 conn(0x7f35380affd0 msgr2=0x7f35380b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:08.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 -- 172.21.15.70:0/3108977905 shutdown_connections 2023-08-25T09:41:08.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.691+0000 7f353e425640 1 -- 172.21.15.70:0/3108977905 wait complete. 2023-08-25T09:41:08.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.693+0000 7f353e425640 1 Processor -- start 2023-08-25T09:41:08.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.693+0000 7f353e425640 1 -- start start 2023-08-25T09:41:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353e425640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35380cadb0 0x7f35380af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353e425640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353e425640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f353815d3d0 0x7f35381574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353e425640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353806ea30 con 0x7f3538159840 2023-08-25T09:41:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353e425640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353806ec10 con 0x7f353815d3d0 2023-08-25T09:41:08.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353e425640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f353806eef0 con 0x7f35380cadb0 2023-08-25T09:41:08.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.694+0000 7f353cc22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:08.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37832/0 (socket says 172.21.15.70:37832) 2023-08-25T09:41:08.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 -- 172.21.15.70:0/1106124270 learned_addr learned my addr 172.21.15.70:0/1106124270 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:08.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 -- 172.21.15.70:0/1106124270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35380cadb0 msgr2=0x7f35380af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35380cadb0 0x7f35380af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 -- 172.21.15.70:0/1106124270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f353815d3d0 msgr2=0x7f35381574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f353815d3d0 0x7f35381574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 -- 172.21.15.70:0/1106124270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35340094d0 con 0x7f3538159840 2023-08-25T09:41:08.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353cc22640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380c5710 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f352800b3e0 tx=0x7f352800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:08.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528011830 con 0x7f3538159840 2023-08-25T09:41:08.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35280119b0 con 0x7f3538159840 2023-08-25T09:41:08.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353806f1d0 con 0x7f3538159840 2023-08-25T09:41:08.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.695+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528011d60 con 0x7f3538159840 2023-08-25T09:41:08.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.696+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35380af620 con 0x7f3538159840 2023-08-25T09:41:08.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.697+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353806f9d0 con 0x7f3538159840 2023-08-25T09:41:08.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.697+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f352801a400 con 0x7f3538159840 2023-08-25T09:41:08.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.698+0000 7f352dffb640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f351c06a050 0x7f351c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:08.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.698+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3528088380 con 0x7f3538159840 2023-08-25T09:41:08.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.699+0000 7f353d423640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f351c06a050 0x7f351c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:08.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.699+0000 7f353d423640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f351c06a050 0x7f351c06c510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3534036000 tx=0x7f353403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:08.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.703+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f353806f9d0 con 0x7f3538159840 2023-08-25T09:41:08.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.942+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f353806f9d0 con 0x7f3538159840 2023-08-25T09:41:08.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.943+0000 7f352dffb640 1 -- 172.21.15.70:0/1106124270 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f353806f9d0 con 0x7f3538159840 2023-08-25T09:41:08.945 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:08.945 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.946 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:41:08.947 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:41:08.947 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.947 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.948 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:41:08.948 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:41:08.948 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.948 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:41:08.949 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:41:08.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.950 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.950 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:41:08.951 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:41:08.951 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.951 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.952 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:41:08.952 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:41:08.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.953 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:41:08.954 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:41:08.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.954 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:41:08.955 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:41:08.955 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.955 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.955 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.956 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.956 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:41:08.957 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:41:08.957 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.957 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.957 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.957 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.958 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:41:08.959 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:41:08.960 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.960 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.960 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.960 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.960 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.961 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:41:08.961 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:41:08.961 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.961 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:41:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:41:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:41:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:41:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:41:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:41:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:41:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.967 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.967 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:41:08.968 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:41:08.968 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.968 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.969 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:41:08.969 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:41:08.969 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.970 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:41:08.970 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:41:08.971 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.971 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.971 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:41:08.971 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:41:08.972 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:41:08.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:41:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:41:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.973 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:41:08.974 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:41:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:41:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:41:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:41:08.975 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:41:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:41:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:41:08.976 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:41:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:41:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:41:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.977 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:41:08.978 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:41:08.979 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:08.980 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f351c06a050 msgr2=0x7f351c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f351c06a050 0x7f351c06c510 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3534036000 tx=0x7f353403b040 comp rx=0 tx=0).stop 2023-08-25T09:41:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 msgr2=0x7f35380c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:08.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380c5710 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f352800b3e0 tx=0x7f352800b8a0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 shutdown_connections 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f351c06a050 0x7f351c06c510 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f353815d3d0 0x7f35381574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3538159840 0x7f35380c5710 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 --2- 172.21.15.70:0/1106124270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35380cadb0 0x7f35380af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 >> 172.21.15.70:0/1106124270 conn(0x7f35380affd0 msgr2=0x7f35380cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:08.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 shutdown_connections 2023-08-25T09:41:08.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:08.945+0000 7f353e425640 1 -- 172.21.15.70:0/1106124270 wait complete. 2023-08-25T09:41:08.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:41:09.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.191+0000 7f538ca14640 1 Processor -- start 2023-08-25T09:41:09.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.191+0000 7f538ca14640 1 -- start start 2023-08-25T09:41:09.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f538ca14640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f53880ce5b0 0x7f53880ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f538ca14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880ceed0 con 0x7f53880b2850 2023-08-25T09:41:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f538ca14640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538815d3f0 con 0x7f538815aa10 2023-08-25T09:41:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f538ca14640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5388159d10 con 0x7f53880ce5b0 2023-08-25T09:41:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f5386d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f53880b2850 0x7f538806f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53438/0 (socket says 172.21.15.70:53438) 2023-08-25T09:41:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f5386d9d640 1 -- 172.21.15.70:0/4023400953 learned_addr learned my addr 172.21.15.70:0/4023400953 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:09.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.193+0000 7f538659c640 1 --2- 172.21.15.70:0/4023400953 >> v2:172.21.15.70:3300/0 conn(0x7f53880ce5b0 0x7f53880ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.194+0000 7f538659c640 1 -- 172.21.15.70:0/4023400953 >> v1:172.21.15.155:6789/0 conn(0x7f538815aa10 legacy=0x7f53880b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.194+0000 7f538659c640 1 -- 172.21.15.70:0/4023400953 >> v1:172.21.15.70:6789/0 conn(0x7f53880b2850 legacy=0x7f538806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.194+0000 7f538559a640 1 -- 172.21.15.70:0/4023400953 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010035363 0 0) 0x7f53880ceed0 con 0x7f53880b2850 2023-08-25T09:41:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.194+0000 7f538659c640 1 -- 172.21.15.70:0/4023400953 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880b2d20 con 0x7f53880ce5b0 2023-08-25T09:41:09.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.195+0000 7f538659c640 1 --2- 172.21.15.70:0/4023400953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f53880ce990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f53780097a0 tx=0x7f537802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d36ece4ad5200a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:09.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.195+0000 7f538559a640 1 -- 172.21.15.70:0/4023400953 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537802e070 con 0x7f53880ce5b0 2023-08-25T09:41:09.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.195+0000 7f538559a640 1 -- 172.21.15.70:0/4023400953 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780315a0 con 0x7f53880ce5b0 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.195+0000 7f538559a640 1 -- 172.21.15.70:0/4023400953 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378031930 con 0x7f53880ce5b0 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 -- 172.21.15.70:0/4023400953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 msgr2=0x7f53880ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 --2- 172.21.15.70:0/4023400953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f53880ce990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f53780097a0 tx=0x7f537802b840 comp rx=0 tx=0).stop 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 -- 172.21.15.70:0/4023400953 shutdown_connections 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 --2- 172.21.15.70:0/4023400953 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f53880ce990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 -- 172.21.15.70:0/4023400953 >> 172.21.15.70:0/4023400953 conn(0x7f5388069850 msgr2=0x7f5388069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 -- 172.21.15.70:0/4023400953 shutdown_connections 2023-08-25T09:41:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.196+0000 7f538ca14640 1 -- 172.21.15.70:0/4023400953 wait complete. 2023-08-25T09:41:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.198+0000 7f538ca14640 1 Processor -- start 2023-08-25T09:41:09.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.198+0000 7f538ca14640 1 -- start start 2023-08-25T09:41:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538ca14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53880b2850 0x7f5388155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538ca14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f538813f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538ca14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f538815aa10 0x7f538815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538ca14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388156570 con 0x7f53880ce5b0 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538ca14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388155960 con 0x7f53880b2850 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538ca14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5388155c40 con 0x7f538815aa10 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f538813f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f538813f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49914/0 (socket says 172.21.15.70:49914) 2023-08-25T09:41:09.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 -- 172.21.15.70:0/2934592585 learned_addr learned my addr 172.21.15.70:0/2934592585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:09.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538659c640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53880b2850 0x7f5388155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 -- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f538815aa10 msgr2=0x7f538815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f538815aa10 0x7f538815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 -- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53880b2850 msgr2=0x7f5388155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53880b2850 0x7f5388155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f5385d9b640 1 -- 172.21.15.70:0/2934592585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5378009400 con 0x7f53880ce5b0 2023-08-25T09:41:09.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.199+0000 7f538659c640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53880b2850 0x7f5388155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.200+0000 7f5385d9b640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f538813f000 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f537000ba50 tx=0x7f537000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.200+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537000cbe0 con 0x7f53880ce5b0 2023-08-25T09:41:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.200+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537000cd80 con 0x7f53880ce5b0 2023-08-25T09:41:09.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.200+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5370014680 con 0x7f53880ce5b0 2023-08-25T09:41:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.200+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5388155f20 con 0x7f53880ce5b0 2023-08-25T09:41:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.200+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538813f540 con 0x7f53880ce5b0 2023-08-25T09:41:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.202+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538804e420 con 0x7f53880ce5b0 2023-08-25T09:41:09.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.202+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f53700147e0 con 0x7f53880ce5b0 2023-08-25T09:41:09.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.203+0000 7f537effd640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f536806a080 0x7f536806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.204+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5370088120 con 0x7f53880ce5b0 2023-08-25T09:41:09.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.204+0000 7f538659c640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f536806a080 0x7f536806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.205+0000 7f538659c640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f536806a080 0x7f536806c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5378036000 tx=0x7f537803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:09.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.208+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5370053b70 con 0x7f53880ce5b0 2023-08-25T09:41:09.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.398+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53880cae50 con 0x7f53880ce5b0 2023-08-25T09:41:09.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.401+0000 7f537effd640 1 -- 172.21.15.70:0/2934592585 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+14653 (secure 0 0 0) 0x7f5370057190 con 0x7f53880ce5b0 2023-08-25T09:41:09.404 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:09.404 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":47,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:08.452851+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:41:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.403+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f536806a080 msgr2=0x7f536806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.403+0000 7f538ca14640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f536806a080 0x7f536806c540 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5378036000 tx=0x7f537803d040 comp rx=0 tx=0).stop 2023-08-25T09:41:09.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.403+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 msgr2=0x7f538813f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.403+0000 7f538ca14640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f538813f000 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f537000ba50 tx=0x7f537000bf10 comp rx=0 tx=0).stop 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 shutdown_connections 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f536806a080 0x7f536806c540 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f538815aa10 0x7f538815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f53880ce5b0 0x7f538813f000 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 --2- 172.21.15.70:0/2934592585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f53880b2850 0x7f5388155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 >> 172.21.15.70:0/2934592585 conn(0x7f5388069850 msgr2=0x7f538806f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:09.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 shutdown_connections 2023-08-25T09:41:09.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.404+0000 7f538ca14640 1 -- 172.21.15.70:0/2934592585 wait complete. 2023-08-25T09:41:09.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:41:09.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:41:09.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.633+0000 7f6120abd640 1 Processor -- start 2023-08-25T09:41:09.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.633+0000 7f6120abd640 1 -- start start 2023-08-25T09:41:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.634+0000 7f6120abd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c0b0320 0x7f611c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.634+0000 7f6120abd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f611c15a920 0x7f611c0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.634+0000 7f6120abd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611c0adbb0 con 0x7f611c0b0700 2023-08-25T09:41:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.634+0000 7f6120abd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611c0b2220 con 0x7f611c15a920 2023-08-25T09:41:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.634+0000 7f6120abd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0c90a0 con 0x7f611c0b0320 2023-08-25T09:41:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f611c15a920 0x7f611c0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f611c15a920 0x7f611c0b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49930/0 (socket says 172.21.15.70:49930) 2023-08-25T09:41:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 -- 172.21.15.70:0/119150189 learned_addr learned my addr 172.21.15.70:0/119150189 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f611b7fe640 1 --2- 172.21.15.70:0/119150189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c0b0320 0x7f611c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 -- 172.21.15.70:0/119150189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c0b0320 msgr2=0x7f611c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 --2- 172.21.15.70:0/119150189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c0b0320 0x7f611c0c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 -- 172.21.15.70:0/119150189 >> v1:172.21.15.155:6789/0 conn(0x7f611c0b0700 legacy=0x7f611c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.635+0000 7f6113fff640 1 -- 172.21.15.70:0/119150189 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c0c9380 con 0x7f611c15a920 2023-08-25T09:41:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.636+0000 7f6113fff640 1 --2- 172.21.15.70:0/119150189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c15a920 0x7f611c0b1ce0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6104009870 tx=0x7f610402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f0893dfbc2b8f3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f611affd640 1 -- 172.21.15.70:0/119150189 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104031640 con 0x7f611c15a920 2023-08-25T09:41:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f611affd640 1 -- 172.21.15.70:0/119150189 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61040317a0 con 0x7f611c15a920 2023-08-25T09:41:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f611affd640 1 -- 172.21.15.70:0/119150189 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104031b30 con 0x7f611c15a920 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f6120abd640 1 -- 172.21.15.70:0/119150189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c15a920 msgr2=0x7f611c0b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f6120abd640 1 --2- 172.21.15.70:0/119150189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c15a920 0x7f611c0b1ce0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6104009870 tx=0x7f610402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f6120abd640 1 -- 172.21.15.70:0/119150189 shutdown_connections 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f6120abd640 1 --2- 172.21.15.70:0/119150189 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c15a920 0x7f611c0b1ce0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f6120abd640 1 --2- 172.21.15.70:0/119150189 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c0b0320 0x7f611c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.637+0000 7f6120abd640 1 -- 172.21.15.70:0/119150189 >> 172.21.15.70:0/119150189 conn(0x7f611c0ce830 msgr2=0x7f611c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.638+0000 7f6120abd640 1 -- 172.21.15.70:0/119150189 shutdown_connections 2023-08-25T09:41:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.638+0000 7f6120abd640 1 -- 172.21.15.70:0/119150189 wait complete. 2023-08-25T09:41:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.639+0000 7f6120abd640 1 Processor -- start 2023-08-25T09:41:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.640+0000 7f6120abd640 1 -- start start 2023-08-25T09:41:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.640+0000 7f6120abd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 0x7f611c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.640+0000 7f6120abd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611c0b0700 0x7f611c1573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.640+0000 7f6120abd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c15a920 0x7f611c0c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f6120abd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0c6ea0 con 0x7f611c0b0320 2023-08-25T09:41:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f6120abd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0c7180 con 0x7f611c0b0700 2023-08-25T09:41:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f6120abd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c0c7460 con 0x7f611c15a920 2023-08-25T09:41:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 0x7f611c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 0x7f611c0c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49942/0 (socket says 172.21.15.70:49942) 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 -- 172.21.15.70:0/2943061043 learned_addr learned my addr 172.21.15.70:0/2943061043 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 -- 172.21.15.70:0/2943061043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c15a920 msgr2=0x7f611c0c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c15a920 0x7f611c0c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 -- 172.21.15.70:0/2943061043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611c0b0700 msgr2=0x7f611c1573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611c0b0700 0x7f611c1573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.641+0000 7f611b7fe640 1 -- 172.21.15.70:0/2943061043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61040094d0 con 0x7f611c0b0320 2023-08-25T09:41:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.642+0000 7f611b7fe640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 0x7f611c0c5630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c7f0 tx=0x7f610c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.642+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c007820 con 0x7f611c0b0320 2023-08-25T09:41:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.642+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610c007980 con 0x7f611c0b0320 2023-08-25T09:41:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.642+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f611c0c7740 con 0x7f611c0b0320 2023-08-25T09:41:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.642+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610c007d50 con 0x7f611c0b0320 2023-08-25T09:41:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.642+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611c0c7a20 con 0x7f611c0b0320 2023-08-25T09:41:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.643+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f611c061460 con 0x7f611c0b0320 2023-08-25T09:41:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.644+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f610c0113f0 con 0x7f611c0b0320 2023-08-25T09:41:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.645+0000 7f6118ff9640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60fc06a050 0x7f60fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.645+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f610c088910 con 0x7f611c0b0320 2023-08-25T09:41:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.645+0000 7f6113fff640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60fc06a050 0x7f60fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:09.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.646+0000 7f6113fff640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60fc06a050 0x7f60fc06c510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6104036000 tx=0x7f6104030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:09.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.649+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f610c054340 con 0x7f611c0b0320 2023-08-25T09:41:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.842+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f611c0c1eb0 con 0x7f611c0b0320 2023-08-25T09:41:09.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.843+0000 7f6118ff9640 1 -- 172.21.15.70:0/2943061043 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f610c01f020 con 0x7f611c0b0320 2023-08-25T09:41:09.846 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:41:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60fc06a050 msgr2=0x7f60fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60fc06a050 0x7f60fc06c510 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6104036000 tx=0x7f6104030000 comp rx=0 tx=0).stop 2023-08-25T09:41:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 msgr2=0x7f611c0c5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 0x7f611c0c5630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f610c00c7f0 tx=0x7f610c00ccb0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 shutdown_connections 2023-08-25T09:41:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60fc06a050 0x7f60fc06c510 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f611c15a920 0x7f611c0c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f611c0b0700 0x7f611c1573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 --2- 172.21.15.70:0/2943061043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f611c0b0320 0x7f611c0c5630 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 >> 172.21.15.70:0/2943061043 conn(0x7f611c0ce830 msgr2=0x7f611c0af290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 shutdown_connections 2023-08-25T09:41:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:09.846+0000 7f6120abd640 1 -- 172.21.15.70:0/2943061043 wait complete. 2023-08-25T09:41:09.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:41:10.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.085+0000 7fefbf463640 1 Processor -- start 2023-08-25T09:41:10.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.085+0000 7fefbf463640 1 -- start start 2023-08-25T09:41:10.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.087+0000 7fefbf463640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefb80b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.087+0000 7fefbf463640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb815d0b0 con 0x7fefb80d09f0 2023-08-25T09:41:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.087+0000 7fefbf463640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb80b58d0 con 0x7fefb815ccd0 2023-08-25T09:41:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.087+0000 7fefbf463640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80d25d0 con 0x7fefb80cf9f0 2023-08-25T09:41:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.087+0000 7fefbdc60640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fefb80d09f0 0x7fefb80cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53446/0 (socket says 172.21.15.70:53446) 2023-08-25T09:41:10.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.087+0000 7fefbdc60640 1 -- 172.21.15.70:0/573753561 learned_addr learned my addr 172.21.15.70:0/573753561 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:10.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbec62640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefb80b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843849717 0 0) 0x7fefb815d0b0 con 0x7fefb80d09f0 2023-08-25T09:41:10.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa4003520 con 0x7fefb80d09f0 2023-08-25T09:41:10.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 618135750 0 0) 0x7fefa4003520 con 0x7fefb80d09f0 2023-08-25T09:41:10.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 msgr2=0x7fefb80b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:10.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefb80b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 >> v1:172.21.15.155:6789/0 conn(0x7fefb815ccd0 legacy=0x7fefb80d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb80d0e80 con 0x7fefb80d09f0 2023-08-25T09:41:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fefa8001690 con 0x7fefb80d09f0 2023-08-25T09:41:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbec62640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefb80b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:41:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 >> v1:172.21.15.70:6789/0 conn(0x7fefb80d09f0 legacy=0x7fefb80cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa40039a0 0x7fefa4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40047c0 0x7fefa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80d25d0 con 0x7fefa40039a0 2023-08-25T09:41:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa400bae0 con 0x7fefa40047c0 2023-08-25T09:41:10.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.088+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa400bca0 con 0x7fefb80cf9f0 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbe461640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa40039a0 0x7fefa4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbdc60640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40047c0 0x7fefa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbec62640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbe461640 1 -- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 msgr2=0x7fefa4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbe461640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb80cf9f0 0x7fefa4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbe461640 1 -- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40047c0 msgr2=0x7fefa4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:10.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbe461640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40047c0 0x7fefa4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.089+0000 7fefbe461640 1 -- 172.21.15.70:0/573753561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa4003520 con 0x7fefa40039a0 2023-08-25T09:41:10.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.090+0000 7fefbe461640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa40039a0 0x7fefa4003dc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fefb4001750 tx=0x7fefa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d60044f1ff753dc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.090+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40052d0 con 0x7fefa40039a0 2023-08-25T09:41:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.090+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb402e050 con 0x7fefa40039a0 2023-08-25T09:41:10.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbd45f640 1 -- 172.21.15.70:0/573753561 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb40035f0 con 0x7fefa40039a0 2023-08-25T09:41:10.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbf463640 1 -- 172.21.15.70:0/573753561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa40039a0 msgr2=0x7fefa4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbf463640 1 --2- 172.21.15.70:0/573753561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefa40039a0 0x7fefa4003dc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fefb4001750 tx=0x7fefa40057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbec62640 1 -- 172.21.15.70:0/573753561 reap_dead start 2023-08-25T09:41:10.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbf463640 1 -- 172.21.15.70:0/573753561 shutdown_connections 2023-08-25T09:41:10.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbf463640 1 -- 172.21.15.70:0/573753561 >> 172.21.15.70:0/573753561 conn(0x7fefb8069850 msgr2=0x7fefb8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:10.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbf463640 1 -- 172.21.15.70:0/573753561 shutdown_connections 2023-08-25T09:41:10.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.091+0000 7fefbf463640 1 -- 172.21.15.70:0/573753561 wait complete. 2023-08-25T09:41:10.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.093+0000 7fefbf463640 1 Processor -- start 2023-08-25T09:41:10.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.093+0000 7fefbf463640 1 -- start start 2023-08-25T09:41:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbf463640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40039a0 0x7fefb80cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbf463640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 0x7fefb815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbf463640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb8157ac0 0x7fefb8157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbf463640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb815b120 con 0x7fefb815a780 2023-08-25T09:41:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbf463640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb81590b0 con 0x7fefa40039a0 2023-08-25T09:41:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbf463640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb80cc6f0 con 0x7fefb8157ac0 2023-08-25T09:41:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbdc60640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 0x7fefb815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbdc60640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 0x7fefb815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49960/0 (socket says 172.21.15.70:49960) 2023-08-25T09:41:10.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.094+0000 7fefbdc60640 1 -- 172.21.15.70:0/3836094533 learned_addr learned my addr 172.21.15.70:0/3836094533 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:10.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbdc60640 1 -- 172.21.15.70:0/3836094533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb8157ac0 msgr2=0x7fefb8157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbdc60640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb8157ac0 0x7fefb8157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbdc60640 1 -- 172.21.15.70:0/3836094533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40039a0 msgr2=0x7fefb80cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbdc60640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40039a0 0x7fefb80cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbdc60640 1 -- 172.21.15.70:0/3836094533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8158810 con 0x7fefb815a780 2023-08-25T09:41:10.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbdc60640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 0x7fefb815abe0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fefa8001010 tx=0x7fefa800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80074e0 con 0x7fefb815a780 2023-08-25T09:41:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa8007640 con 0x7fefb815a780 2023-08-25T09:41:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb8158af0 con 0x7fefb815a780 2023-08-25T09:41:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80049e0 con 0x7fefb815a780 2023-08-25T09:41:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.095+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb8141da0 con 0x7fefb815a780 2023-08-25T09:41:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.097+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb8061460 con 0x7fefb815a780 2023-08-25T09:41:10.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.098+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fefa8004b40 con 0x7fefb815a780 2023-08-25T09:41:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.098+0000 7fefaeffd640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef9c06a050 0x7fef9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.099+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fefa80879e0 con 0x7fefb815a780 2023-08-25T09:41:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.099+0000 7fefbe461640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef9c06a050 0x7fef9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.099+0000 7fefbe461640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef9c06a050 0x7fef9c06c510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fefb4033000 tx=0x7fefb402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.103+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fefa8053560 con 0x7fefb815a780 2023-08-25T09:41:10.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.291+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fefb8141f60 con 0x7fefb815a780 2023-08-25T09:41:10.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.291+0000 7fefaeffd640 1 -- 172.21.15.70:0/3836094533 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v47) v1 ==== 92+0+10 (secure 0 0 0) 0x7fefa8019020 con 0x7fefb815a780 2023-08-25T09:41:10.294 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:41:10.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef9c06a050 msgr2=0x7fef9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef9c06a050 0x7fef9c06c510 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fefb4033000 tx=0x7fefb402f000 comp rx=0 tx=0).stop 2023-08-25T09:41:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 msgr2=0x7fefb815abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 0x7fefb815abe0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fefa8001010 tx=0x7fefa800dc70 comp rx=0 tx=0).stop 2023-08-25T09:41:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 shutdown_connections 2023-08-25T09:41:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef9c06a050 0x7fef9c06c510 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fefb8157ac0 0x7fefb8157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fefb815a780 0x7fefb815abe0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 --2- 172.21.15.70:0/3836094533 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fefa40039a0 0x7fefb80cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 >> 172.21.15.70:0/3836094533 conn(0x7fefb8069850 msgr2=0x7fefb80c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:10.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.294+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 shutdown_connections 2023-08-25T09:41:10.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.295+0000 7fefbf463640 1 -- 172.21.15.70:0/3836094533 wait complete. 2023-08-25T09:41:10.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:41:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.529+0000 7f2f2b59e640 1 Processor -- start 2023-08-25T09:41:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.529+0000 7f2f2b59e640 1 -- start start 2023-08-25T09:41:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c96a0 0x7f2f2c0cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2b59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2f2c0b4150 0x7f2f2c0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2b59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c0cd330 con 0x7f2f2c0c92c0 2023-08-25T09:41:10.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2b59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2c1615b0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0cb7c0 con 0x7f2f2c0c96a0 2023-08-25T09:41:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2f2c0b4150 0x7f2f2c0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c96a0 0x7f2f2c0cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2ad9d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2f2c0b4150 0x7f2f2c0cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:54644/0 (socket says 172.21.15.70:54644) 2023-08-25T09:41:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.531+0000 7f2f2ad9d640 1 -- 172.21.15.70:0/447015481 learned_addr learned my addr 172.21.15.70:0/447015481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.532+0000 7f2f2ad9d640 1 -- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c96a0 msgr2=0x7f2f2c0cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.532+0000 7f2f2ad9d640 1 --2- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c96a0 0x7f2f2c0cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.532+0000 7f2f2ad9d640 1 -- 172.21.15.70:0/447015481 >> v1:172.21.15.155:6789/0 conn(0x7f2f2c0c92c0 legacy=0x7f2f2c0b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.532+0000 7f2f2ad9d640 1 -- 172.21.15.70:0/447015481 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c0b0e80 con 0x7f2f2c0b4150 2023-08-25T09:41:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.534+0000 7f2f2ad9d640 1 --2- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c0cb280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009e50 tx=0x7f2f2402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d371cb6859fff76 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.534+0000 7f2f29d9b640 1 -- 172.21.15.70:0/447015481 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2402cb00 con 0x7f2f2c0b4150 2023-08-25T09:41:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.534+0000 7f2f29d9b640 1 -- 172.21.15.70:0/447015481 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2402cc60 con 0x7f2f2c0b4150 2023-08-25T09:41:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.534+0000 7f2f29d9b640 1 -- 172.21.15.70:0/447015481 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f240343f0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 -- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0b4150 msgr2=0x7f2f2c0cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c0cb280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009e50 tx=0x7f2f2402bf60 comp rx=0 tx=0).stop 2023-08-25T09:41:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 -- 172.21.15.70:0/447015481 shutdown_connections 2023-08-25T09:41:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c0cb280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/447015481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c96a0 0x7f2f2c0cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 -- 172.21.15.70:0/447015481 >> 172.21.15.70:0/447015481 conn(0x7f2f2c0affd0 msgr2=0x7f2f2c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 -- 172.21.15.70:0/447015481 shutdown_connections 2023-08-25T09:41:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.535+0000 7f2f2b59e640 1 -- 172.21.15.70:0/447015481 wait complete. 2023-08-25T09:41:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.537+0000 7f2f2b59e640 1 Processor -- start 2023-08-25T09:41:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.537+0000 7f2f2b59e640 1 -- start start 2023-08-25T09:41:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c1ea330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c92c0 0x7f2f2c14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2b59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0c96a0 0x7f2f2c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c14e330 con 0x7f2f2c0b4150 2023-08-25T09:41:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2b59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c14e610 con 0x7f2f2c0c96a0 2023-08-25T09:41:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c14e8f0 con 0x7f2f2c0c92c0 2023-08-25T09:41:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c1ea330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c1ea330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49962/0 (socket says 172.21.15.70:49962) 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2a59c640 1 -- 172.21.15.70:0/77228051 learned_addr learned my addr 172.21.15.70:0/77228051 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f23fff640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c92c0 0x7f2f2c14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.538+0000 7f2f2ad9d640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0c96a0 0x7f2f2c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2a59c640 1 -- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c92c0 msgr2=0x7f2f2c14d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2a59c640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c92c0 0x7f2f2c14d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2a59c640 1 -- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0c96a0 msgr2=0x7f2f2c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2a59c640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0c96a0 0x7f2f2c0c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2a59c640 1 -- 172.21.15.70:0/77228051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009b30 con 0x7f2f2c0b4150 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2a59c640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c1ea330 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c400 tx=0x7f2f1c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0052d0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c005450 con 0x7f2f2c0b4150 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c007260 con 0x7f2f2c0b4150 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c14ebd0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.539+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c1ea870 con 0x7f2f2c0b4150 2023-08-25T09:41:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.540+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c14f0f0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.541+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f1c0073c0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.542+0000 7f2f22ffd640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f0c06a050 0x7f2f0c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.542+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2f1c0876c0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.543+0000 7f2f23fff640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f0c06a050 0x7f2f0c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.543+0000 7f2f23fff640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f0c06a050 0x7f2f0c06c510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0c5b40 tx=0x7f2f14008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.546+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f2c14f0f0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.737+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2f2c066fb0 con 0x7f2f2c0b4150 2023-08-25T09:41:10.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.737+0000 7f2f22ffd640 1 -- 172.21.15.70:0/77228051 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v47) v1 ==== 84+0+126 (secure 0 0 0) 0x7f2f1c056b80 con 0x7f2f2c0b4150 2023-08-25T09:41:10.740 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:10.740 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:41:10.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:41:10.741 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:10.742 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:41:10.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f0c06a050 msgr2=0x7f2f0c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f0c06a050 0x7f2f0c06c510 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0c5b40 tx=0x7f2f14008040 comp rx=0 tx=0).stop 2023-08-25T09:41:10.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 msgr2=0x7f2f2c1ea330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c1ea330 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00c400 tx=0x7f2f1c00c8c0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 shutdown_connections 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f0c06a050 0x7f2f0c06c510 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f2c0c96a0 0x7f2f2c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f2c0c92c0 0x7f2f2c14d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 --2- 172.21.15.70:0/77228051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f2c0b4150 0x7f2f2c1ea330 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 >> 172.21.15.70:0/77228051 conn(0x7f2f2c0affd0 msgr2=0x7f2f2c160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 shutdown_connections 2023-08-25T09:41:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.740+0000 7f2f2b59e640 1 -- 172.21.15.70:0/77228051 wait complete. 2023-08-25T09:41:10.760 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:41:10.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:41:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.984+0000 7fc6e536b640 1 Processor -- start 2023-08-25T09:41:10.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.985+0000 7fc6e536b640 1 -- start start 2023-08-25T09:41:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.986+0000 7fc6e536b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc6e015c950 0x7fc6e00b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.986+0000 7fc6e536b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6e015cd30 0x7fc6e006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.986+0000 7fc6e536b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e00b2300 con 0x7fc6e00b0980 2023-08-25T09:41:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.986+0000 7fc6e536b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e00b16f0 con 0x7fc6e015cd30 2023-08-25T09:41:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.986+0000 7fc6e536b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e00b19d0 con 0x7fc6e015c950 2023-08-25T09:41:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6d67fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6e015cd30 0x7fc6e006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6df7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc6e00b0980 0x7fc6e00b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53460/0 (socket says 172.21.15.70:53460) 2023-08-25T09:41:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6df7fe640 1 -- 172.21.15.70:0/3390432280 learned_addr learned my addr 172.21.15.70:0/3390432280 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6deffd640 1 --2- 172.21.15.70:0/3390432280 >> v2:172.21.15.155:3300/0 conn(0x7fc6e015c950 0x7fc6e00b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6d67fc640 1 -- 172.21.15.70:0/3390432280 >> v2:172.21.15.155:3300/0 conn(0x7fc6e015c950 msgr2=0x7fc6e00b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6d67fc640 1 --2- 172.21.15.70:0/3390432280 >> v2:172.21.15.155:3300/0 conn(0x7fc6e015c950 0x7fc6e00b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6d67fc640 1 -- 172.21.15.70:0/3390432280 >> v1:172.21.15.70:6789/0 conn(0x7fc6e00b0980 legacy=0x7fc6e00b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.987+0000 7fc6d67fc640 1 -- 172.21.15.70:0/3390432280 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e00b1cb0 con 0x7fc6e015cd30 2023-08-25T09:41:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.988+0000 7fc6d67fc640 1 --2- 172.21.15.70:0/3390432280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e006aa70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc6e006cdc0 tx=0x7fc6c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e020c3ab8f2772 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.989+0000 7fc6de7fc640 1 -- 172.21.15.70:0/3390432280 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8031830 con 0x7fc6e015cd30 2023-08-25T09:41:10.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.989+0000 7fc6de7fc640 1 -- 172.21.15.70:0/3390432280 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c8031990 con 0x7fc6e015cd30 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.989+0000 7fc6de7fc640 1 -- 172.21.15.70:0/3390432280 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8031d20 con 0x7fc6e015cd30 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.989+0000 7fc6e536b640 1 -- 172.21.15.70:0/3390432280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 msgr2=0x7fc6e006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.989+0000 7fc6e536b640 1 --2- 172.21.15.70:0/3390432280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e006aa70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc6e006cdc0 tx=0x7fc6c802b8d0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.990+0000 7fc6e536b640 1 -- 172.21.15.70:0/3390432280 shutdown_connections 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.990+0000 7fc6e536b640 1 --2- 172.21.15.70:0/3390432280 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e006aa70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.990+0000 7fc6e536b640 1 --2- 172.21.15.70:0/3390432280 >> v2:172.21.15.155:3300/0 conn(0x7fc6e015c950 0x7fc6e00b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.990+0000 7fc6e536b640 1 -- 172.21.15.70:0/3390432280 >> 172.21.15.70:0/3390432280 conn(0x7fc6e0069c70 msgr2=0x7fc6e006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.990+0000 7fc6e536b640 1 -- 172.21.15.70:0/3390432280 shutdown_connections 2023-08-25T09:41:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.990+0000 7fc6e536b640 1 -- 172.21.15.70:0/3390432280 wait complete. 2023-08-25T09:41:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.991+0000 7fc6e536b640 1 Processor -- start 2023-08-25T09:41:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.991+0000 7fc6e536b640 1 -- start start 2023-08-25T09:41:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.992+0000 7fc6e536b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6e00b0980 0x7fc6e0154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.992+0000 7fc6e536b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6e015c950 0x7fc6e006fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.992+0000 7fc6e536b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e0070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.992+0000 7fc6e536b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e015a730 con 0x7fc6e015cd30 2023-08-25T09:41:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.992+0000 7fc6e536b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e015aa10 con 0x7fc6e00b0980 2023-08-25T09:41:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.992+0000 7fc6e536b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e015acf0 con 0x7fc6e015c950 2023-08-25T09:41:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49976/0 (socket says 172.21.15.70:49976) 2023-08-25T09:41:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 -- 172.21.15.70:0/44246440 learned_addr learned my addr 172.21.15.70:0/44246440 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6d67fc640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6e015c950 0x7fc6e006fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6deffd640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6e00b0980 0x7fc6e0154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 -- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6e015c950 msgr2=0x7fc6e006fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6e015c950 0x7fc6e006fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 -- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6e00b0980 msgr2=0x7fc6e0154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6e00b0980 0x7fc6e0154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.993+0000 7fc6df7fe640 1 -- 172.21.15.70:0/44246440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c8009400 con 0x7fc6e015cd30 2023-08-25T09:41:11.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.994+0000 7fc6df7fe640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e0070350 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c420 tx=0x7fc6d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:11.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.994+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8005ce0 con 0x7fc6e015cd30 2023-08-25T09:41:11.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.994+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8005e40 con 0x7fc6e015cd30 2023-08-25T09:41:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.994+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e015afd0 con 0x7fc6e015cd30 2023-08-25T09:41:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.994+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80072d0 con 0x7fc6e015cd30 2023-08-25T09:41:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.994+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e0155820 con 0x7fc6e015cd30 2023-08-25T09:41:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.995+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e014c790 con 0x7fc6e015cd30 2023-08-25T09:41:11.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.996+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc6d80074f0 con 0x7fc6e015cd30 2023-08-25T09:41:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.997+0000 7fc6d7fff640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6c006a050 0x7fc6c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.997+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc6d80877d0 con 0x7fc6e015cd30 2023-08-25T09:41:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.998+0000 7fc6deffd640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6c006a050 0x7fc6c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:10.998+0000 7fc6deffd640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6c006a050 0x7fc6c006c510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009a60 tx=0x7fc6d0003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:11.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.002+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc6e014c790 con 0x7fc6e015cd30 2023-08-25T09:41:11.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.192+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc6e014c790 con 0x7fc6e015cd30 2023-08-25T09:41:11.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.490+0000 7fc6d7fff640 1 -- 172.21.15.70:0/44246440 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v48) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc6e014c790 con 0x7fc6e015cd30 2023-08-25T09:41:11.493 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:41:11.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.494+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6c006a050 msgr2=0x7fc6c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.494+0000 7fc6e536b640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6c006a050 0x7fc6c006c510 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009a60 tx=0x7fc6d0003990 comp rx=0 tx=0).stop 2023-08-25T09:41:11.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.494+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 msgr2=0x7fc6e0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.494+0000 7fc6e536b640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e0070350 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c420 tx=0x7fc6d800c8e0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 shutdown_connections 2023-08-25T09:41:11.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6c006a050 0x7fc6c006c510 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6e015cd30 0x7fc6e0070350 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6e015c950 0x7fc6e006fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 --2- 172.21.15.70:0/44246440 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6e00b0980 0x7fc6e0154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 >> 172.21.15.70:0/44246440 conn(0x7fc6e0069c70 msgr2=0x7fc6e00cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.495+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 shutdown_connections 2023-08-25T09:41:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.496+0000 7fc6e536b640 1 -- 172.21.15.70:0/44246440 wait complete. 2023-08-25T09:41:11.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:41:11.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.728+0000 7f149fa7c640 1 Processor -- start 2023-08-25T09:41:11.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.728+0000 7f149fa7c640 1 -- start start 2023-08-25T09:41:11.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149fa7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f14980c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149fa7c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14980cdc20 0x7f1498069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149fa7c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980cc5f0 con 0x7f14980ce060 2023-08-25T09:41:11.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149fa7c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1498070650 con 0x7f14980cdc20 2023-08-25T09:41:11.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149fa7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1498070930 con 0x7f149815ae50 2023-08-25T09:41:11.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149d017640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14980cdc20 0x7f1498069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.730+0000 7f149d818640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f14980c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d017640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f14980cdc20 0x7f1498069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:54684/0 (socket says 172.21.15.70:54684) 2023-08-25T09:41:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d818640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f14980c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:42412/0 (socket says 172.21.15.70:42412) 2023-08-25T09:41:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d017640 1 -- 172.21.15.70:0/1387685907 learned_addr learned my addr 172.21.15.70:0/1387685907 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d818640 1 -- 172.21.15.70:0/1387685907 >> v2:172.21.15.155:3300/0 conn(0x7f14980cdc20 msgr2=0x7f1498069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d818640 1 --2- 172.21.15.70:0/1387685907 >> v2:172.21.15.155:3300/0 conn(0x7f14980cdc20 0x7f1498069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d818640 1 -- 172.21.15.70:0/1387685907 >> v1:172.21.15.155:6789/0 conn(0x7f14980ce060 legacy=0x7f14980ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.731+0000 7f149d818640 1 -- 172.21.15.70:0/1387685907 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1498070c10 con 0x7f149815ae50 2023-08-25T09:41:11.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.733+0000 7f149d818640 1 --2- 172.21.15.70:0/1387685907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f14980c2110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14940097a0 tx=0x7f149402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b49545f6960a657 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.733+0000 7f149c816640 1 -- 172.21.15.70:0/1387685907 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494030800 con 0x7f149815ae50 2023-08-25T09:41:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149c816640 1 -- 172.21.15.70:0/1387685907 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1494030960 con 0x7f149815ae50 2023-08-25T09:41:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149c816640 1 -- 172.21.15.70:0/1387685907 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494030cf0 con 0x7f149815ae50 2023-08-25T09:41:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 -- 172.21.15.70:0/1387685907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 msgr2=0x7f14980c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1387685907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f14980c2110 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14940097a0 tx=0x7f149402b940 comp rx=0 tx=0).stop 2023-08-25T09:41:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 -- 172.21.15.70:0/1387685907 shutdown_connections 2023-08-25T09:41:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1387685907 >> v2:172.21.15.155:3300/0 conn(0x7f14980cdc20 0x7f1498069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1387685907 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f14980c2110 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 -- 172.21.15.70:0/1387685907 >> 172.21.15.70:0/1387685907 conn(0x7f14980cb0d0 msgr2=0x7f14980cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:11.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 -- 172.21.15.70:0/1387685907 shutdown_connections 2023-08-25T09:41:11.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.734+0000 7f149fa7c640 1 -- 172.21.15.70:0/1387685907 wait complete. 2023-08-25T09:41:11.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.736+0000 7f149fa7c640 1 Processor -- start 2023-08-25T09:41:11.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.736+0000 7f149fa7c640 1 -- start start 2023-08-25T09:41:11.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.737+0000 7f149fa7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 0x7f14980c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.737+0000 7f149fa7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14980ce060 0x7f149815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.737+0000 7f149fa7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f149814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.737+0000 7f149fa7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149815c300 con 0x7f14980cdc20 2023-08-25T09:41:11.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.737+0000 7f149fa7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14981f3a40 con 0x7f14980ce060 2023-08-25T09:41:11.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.737+0000 7f149fa7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14981f3d20 con 0x7f149815ae50 2023-08-25T09:41:11.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 0x7f14980c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 0x7f14980c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49982/0 (socket says 172.21.15.70:49982) 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 -- 172.21.15.70:0/1282962456 learned_addr learned my addr 172.21.15.70:0/1282962456 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d017640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14980ce060 0x7f149815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149e019640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f149814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 -- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 msgr2=0x7f149814fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f149814fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 -- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14980ce060 msgr2=0x7f149815bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14980ce060 0x7f149815bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 -- 172.21.15.70:0/1282962456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494009400 con 0x7f14980cdc20 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.738+0000 7f149d818640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 0x7f14980c5f70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f149402bf50 tx=0x7f14940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.739+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149403e070 con 0x7f14980cdc20 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.739+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1494039a10 con 0x7f14980cdc20 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.739+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14981f4000 con 0x7f14980cdc20 2023-08-25T09:41:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.739+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1494039da0 con 0x7f14980cdc20 2023-08-25T09:41:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.739+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14981f45c0 con 0x7f14980cdc20 2023-08-25T09:41:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.740+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1498066fb0 con 0x7f14980cdc20 2023-08-25T09:41:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.741+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14940423f0 con 0x7f14980cdc20 2023-08-25T09:41:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.742+0000 7f148e7fc640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f147c06a050 0x7f147c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.743+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f14940aa110 con 0x7f14980cdc20 2023-08-25T09:41:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.743+0000 7f149d017640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f147c06a050 0x7f147c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.743+0000 7f149d017640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f147c06a050 0x7f147c06c510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1498158e00 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.747+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1494075940 con 0x7f14980cdc20 2023-08-25T09:41:11.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:11.936+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f14981f4930 con 0x7f14980cdc20 2023-08-25T09:41:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.500+0000 7f148e7fc640 1 -- 172.21.15.70:0/1282962456 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v49) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1494078f60 con 0x7f14980cdc20 2023-08-25T09:41:12.506 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:41:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.511+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f147c06a050 msgr2=0x7f147c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.511+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f147c06a050 0x7f147c06c510 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1498158e00 tx=0x7f1488008040 comp rx=0 tx=0).stop 2023-08-25T09:41:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.511+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 msgr2=0x7f14980c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.511+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 0x7f14980c5f70 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f149402bf50 tx=0x7f14940046c0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.513+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 shutdown_connections 2023-08-25T09:41:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.513+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f147c06a050 0x7f147c06c510 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.513+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f149815ae50 0x7f149814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.513+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14980ce060 0x7f149815bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.513+0000 7f149fa7c640 1 --2- 172.21.15.70:0/1282962456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14980cdc20 0x7f14980c5f70 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.513+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 >> 172.21.15.70:0/1282962456 conn(0x7f14980cb0d0 msgr2=0x7f14980cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:12.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.514+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 shutdown_connections 2023-08-25T09:41:12.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.514+0000 7f149fa7c640 1 -- 172.21.15.70:0/1282962456 wait complete. 2023-08-25T09:41:12.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:41:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.736+0000 7f83819e3640 1 Processor -- start 2023-08-25T09:41:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.736+0000 7f83819e3640 1 -- start start 2023-08-25T09:41:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f83819e3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f837c069850 0x7f837c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f83819e3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f837c0cca20 0x7f837c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f83819e3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c0ac880 con 0x7f837c0c9620 2023-08-25T09:41:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f83819e3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c0ad400 con 0x7f837c069850 2023-08-25T09:41:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f83819e3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c0ad890 con 0x7f837c0cca20 2023-08-25T09:41:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f837affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f837c069850 0x7f837c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f837affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f837c069850 0x7f837c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49992/0 (socket says 172.21.15.70:49992) 2023-08-25T09:41:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.738+0000 7f837affd640 1 -- 172.21.15.70:0/1166606608 learned_addr learned my addr 172.21.15.70:0/1166606608 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.739+0000 7f837affd640 1 -- 172.21.15.70:0/1166606608 >> v2:172.21.15.155:3300/0 conn(0x7f837c0cca20 msgr2=0x7f837c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.739+0000 7f837b7fe640 1 --2- 172.21.15.70:0/1166606608 >> v2:172.21.15.155:3300/0 conn(0x7f837c0cca20 0x7f837c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.739+0000 7f837affd640 1 --2- 172.21.15.70:0/1166606608 >> v2:172.21.15.155:3300/0 conn(0x7f837c0cca20 0x7f837c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.739+0000 7f837affd640 1 -- 172.21.15.70:0/1166606608 >> v1:172.21.15.155:6789/0 conn(0x7f837c0c9620 legacy=0x7f837c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.739+0000 7f837affd640 1 -- 172.21.15.70:0/1166606608 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c0acad0 con 0x7f837c069850 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.740+0000 7f837affd640 1 --2- 172.21.15.70:0/1166606608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c069850 0x7f837c15e210 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f837c0c5610 tx=0x7f836c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eeacbe53636d936 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.740+0000 7f8379ffb640 1 -- 172.21.15.70:0/1166606608 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c02cb20 con 0x7f837c069850 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.740+0000 7f8379ffb640 1 -- 172.21.15.70:0/1166606608 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f836c02cc80 con 0x7f837c069850 2023-08-25T09:41:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.740+0000 7f8379ffb640 1 -- 172.21.15.70:0/1166606608 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836c0343f0 con 0x7f837c069850 2023-08-25T09:41:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 -- 172.21.15.70:0/1166606608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c069850 msgr2=0x7f837c15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 --2- 172.21.15.70:0/1166606608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c069850 0x7f837c15e210 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f837c0c5610 tx=0x7f836c02bf80 comp rx=0 tx=0).stop 2023-08-25T09:41:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 -- 172.21.15.70:0/1166606608 shutdown_connections 2023-08-25T09:41:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 --2- 172.21.15.70:0/1166606608 >> v2:172.21.15.155:3300/0 conn(0x7f837c0cca20 0x7f837c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 --2- 172.21.15.70:0/1166606608 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c069850 0x7f837c15e210 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 -- 172.21.15.70:0/1166606608 >> 172.21.15.70:0/1166606608 conn(0x7f837c0cb0d0 msgr2=0x7f837c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 -- 172.21.15.70:0/1166606608 shutdown_connections 2023-08-25T09:41:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.741+0000 7f83819e3640 1 -- 172.21.15.70:0/1166606608 wait complete. 2023-08-25T09:41:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.743+0000 7f83819e3640 1 Processor -- start 2023-08-25T09:41:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.743+0000 7f83819e3640 1 -- start start 2023-08-25T09:41:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f83819e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f837c069850 0x7f837c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f83819e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f837c0c9620 0x7f837c0c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f83819e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 0x7f837c14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f83819e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0c7ef0 con 0x7f837c0cca20 2023-08-25T09:41:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f83819e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0c5d80 con 0x7f837c069850 2023-08-25T09:41:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f83819e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0ae210 con 0x7f837c0c9620 2023-08-25T09:41:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f837b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 0x7f837c14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f837b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 0x7f837c14e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49998/0 (socket says 172.21.15.70:49998) 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f837b7fe640 1 -- 172.21.15.70:0/2913365669 learned_addr learned my addr 172.21.15.70:0/2913365669 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f837affd640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f837c069850 0x7f837c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.744+0000 7f837a7fc640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f837c0c9620 0x7f837c0c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f837b7fe640 1 -- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f837c0c9620 msgr2=0x7f837c0c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f837b7fe640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f837c0c9620 0x7f837c0c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f837b7fe640 1 -- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f837c069850 msgr2=0x7f837c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f837b7fe640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f837c069850 0x7f837c0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f837b7fe640 1 -- 172.21.15.70:0/2913365669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836c009b30 con 0x7f837c0cca20 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f837b7fe640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 0x7f837c14e8b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f837400c7f0 tx=0x7f837400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374004220 con 0x7f837c0cca20 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8374004380 con 0x7f837c0cca20 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8374007860 con 0x7f837c0cca20 2023-08-25T09:41:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c0ae4f0 con 0x7f837c0cca20 2023-08-25T09:41:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.745+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c0c5a10 con 0x7f837c0cca20 2023-08-25T09:41:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.746+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c0c76d0 con 0x7f837c0cca20 2023-08-25T09:41:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.748+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8374007bd0 con 0x7f837c0cca20 2023-08-25T09:41:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.748+0000 7f8373fff640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f835c06a050 0x7f835c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.749+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f83740884e0 con 0x7f837c0cca20 2023-08-25T09:41:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.749+0000 7f837affd640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f835c06a050 0x7f835c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.750+0000 7f837affd640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f835c06a050 0x7f835c06c510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f836c035000 tx=0x7f836c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.754+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f837c0c76d0 con 0x7f837c0cca20 2023-08-25T09:41:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:12.982+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f837c0c76d0 con 0x7f837c0cca20 2023-08-25T09:41:13.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.562+0000 7f8373fff640 1 -- 172.21.15.70:0/2913365669 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f837c0c76d0 con 0x7f837c0cca20 2023-08-25T09:41:13.566 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 7 and data pool 8 2023-08-25T09:41:13.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f835c06a050 msgr2=0x7f835c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:13.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f835c06a050 0x7f835c06c510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f836c035000 tx=0x7f836c030000 comp rx=0 tx=0).stop 2023-08-25T09:41:13.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 msgr2=0x7f837c14e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:13.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 0x7f837c14e8b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f837400c7f0 tx=0x7f837400ccb0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 shutdown_connections 2023-08-25T09:41:13.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f835c06a050 0x7f835c06c510 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f837c0cca20 0x7f837c14e8b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f837c0c9620 0x7f837c0c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 --2- 172.21.15.70:0/2913365669 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f837c069850 0x7f837c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.568+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 >> 172.21.15.70:0/2913365669 conn(0x7f837c0cb0d0 msgr2=0x7f837c06c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:13.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.569+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 shutdown_connections 2023-08-25T09:41:13.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.569+0000 7f83819e3640 1 -- 172.21.15.70:0/2913365669 wait complete. 2023-08-25T09:41:13.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:41:13.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:41:13.770 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:41:13.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.783+0000 7f7e75f14640 1 Processor -- start 2023-08-25T09:41:13.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.783+0000 7f7e75f14640 1 -- start start 2023-08-25T09:41:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.784+0000 7f7e75f14640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e7015ce80 0x7f7e700cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.784+0000 7f7e75f14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 0x7f7e700cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.784+0000 7f7e75f14640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700d1470 con 0x7f7e7015caa0 2023-08-25T09:41:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.784+0000 7f7e75f14640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e700d1750 con 0x7f7e7015ce80 2023-08-25T09:41:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.784+0000 7f7e75f14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700d1a30 con 0x7f7e700b8100 2023-08-25T09:41:13.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e75713640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 0x7f7e700cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:13.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e6ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e7015ce80 0x7f7e700cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:13.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e75713640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 0x7f7e700cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:42444/0 (socket says 172.21.15.70:42444) 2023-08-25T09:41:13.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e75713640 1 -- 172.21.15.70:0/2439821166 learned_addr learned my addr 172.21.15.70:0/2439821166 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:13.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e75713640 1 -- 172.21.15.70:0/2439821166 >> v2:172.21.15.155:3300/0 conn(0x7f7e7015ce80 msgr2=0x7f7e700cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:13.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e75713640 1 --2- 172.21.15.70:0/2439821166 >> v2:172.21.15.155:3300/0 conn(0x7f7e7015ce80 0x7f7e700cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.785+0000 7f7e75713640 1 -- 172.21.15.70:0/2439821166 >> v1:172.21.15.155:6789/0 conn(0x7f7e7015caa0 legacy=0x7f7e700d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:13.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.786+0000 7f7e75713640 1 -- 172.21.15.70:0/2439821166 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700d1d10 con 0x7f7e700b8100 2023-08-25T09:41:13.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.787+0000 7f7e75713640 1 --2- 172.21.15.70:0/2439821166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 0x7f7e700cf7d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009e50 tx=0x7f7e6802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24688784039060cd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:13.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.787+0000 7f7e6f7fe640 1 -- 172.21.15.70:0/2439821166 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6802cb90 con 0x7f7e700b8100 2023-08-25T09:41:13.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.787+0000 7f7e6f7fe640 1 -- 172.21.15.70:0/2439821166 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6802ccf0 con 0x7f7e700b8100 2023-08-25T09:41:13.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.787+0000 7f7e6f7fe640 1 -- 172.21.15.70:0/2439821166 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68034610 con 0x7f7e700b8100 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 -- 172.21.15.70:0/2439821166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 msgr2=0x7f7e700cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 --2- 172.21.15.70:0/2439821166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 0x7f7e700cf7d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009e50 tx=0x7f7e6802bff0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 -- 172.21.15.70:0/2439821166 shutdown_connections 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 --2- 172.21.15.70:0/2439821166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e700b8100 0x7f7e700cf7d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 --2- 172.21.15.70:0/2439821166 >> v2:172.21.15.155:3300/0 conn(0x7f7e7015ce80 0x7f7e700cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 -- 172.21.15.70:0/2439821166 >> 172.21.15.70:0/2439821166 conn(0x7f7e7006fe70 msgr2=0x7f7e70070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 -- 172.21.15.70:0/2439821166 shutdown_connections 2023-08-25T09:41:13.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.788+0000 7f7e75f14640 1 -- 172.21.15.70:0/2439821166 wait complete. 2023-08-25T09:41:13.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.790+0000 7f7e75f14640 1 Processor -- start 2023-08-25T09:41:13.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.790+0000 7f7e75f14640 1 -- start start 2023-08-25T09:41:13.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75f14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e700b8100 0x7f7e7014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:13.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75f14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7015caa0 0x7f7e7015b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:13.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75f14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 0x7f7e7015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:13.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75f14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7014c210 con 0x7f7e7015ce80 2023-08-25T09:41:13.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75f14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700c8120 con 0x7f7e700b8100 2023-08-25T09:41:13.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75f14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700c8650 con 0x7f7e7015caa0 2023-08-25T09:41:13.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75713640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 0x7f7e7015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:13.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75713640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 0x7f7e7015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50006/0 (socket says 172.21.15.70:50006) 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e75713640 1 -- 172.21.15.70:0/1980051015 learned_addr learned my addr 172.21.15.70:0/1980051015 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e6ffff640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7015caa0 0x7f7e7015b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.791+0000 7f7e74f12640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e700b8100 0x7f7e7014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.792+0000 7f7e75713640 1 -- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7015caa0 msgr2=0x7f7e7015b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.792+0000 7f7e75713640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7015caa0 0x7f7e7015b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.792+0000 7f7e75713640 1 -- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e700b8100 msgr2=0x7f7e7014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:13.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.792+0000 7f7e75713640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e700b8100 0x7f7e7014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.792+0000 7f7e75713640 1 -- 172.21.15.70:0/1980051015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68009b30 con 0x7f7e7015ce80 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.793+0000 7f7e75713640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 0x7f7e7015b910 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7e68036040 tx=0x7f7e68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.793+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68009510 con 0x7f7e7015ce80 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.793+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6803e450 con 0x7f7e7015ce80 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.793+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6803e7e0 con 0x7f7e7015ce80 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.793+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e700c8370 con 0x7f7e7015ce80 2023-08-25T09:41:13.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.793+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e700c8e30 con 0x7f7e7015ce80 2023-08-25T09:41:13.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.794+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e70066fb0 con 0x7f7e7015ce80 2023-08-25T09:41:13.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.796+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e6803e940 con 0x7f7e7015ce80 2023-08-25T09:41:13.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.797+0000 7f7e6d7fa640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5406a050 0x7f7e5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:13.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.797+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e680ab2a0 con 0x7f7e7015ce80 2023-08-25T09:41:13.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.797+0000 7f7e74f12640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5406a050 0x7f7e5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:13.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.798+0000 7f7e74f12640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5406a050 0x7f7e5406c510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e64000f60 tx=0x7f7e64009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:13.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.802+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e68076a30 con 0x7f7e7015ce80 2023-08-25T09:41:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.992+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e700c8fc0 con 0x7f7e7015ce80 2023-08-25T09:41:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.994+0000 7f7e6d7fa640 1 -- 172.21.15.70:0/1980051015 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18318 (secure 0 0 0) 0x7f7e6802ce60 con 0x7f7e7015ce80 2023-08-25T09:41:13.997 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:13.997 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:12.985250+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:41:13.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5406a050 msgr2=0x7f7e5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5406a050 0x7f7e5406c510 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e64000f60 tx=0x7f7e64009040 comp rx=0 tx=0).stop 2023-08-25T09:41:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 msgr2=0x7f7e7015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 0x7f7e7015b910 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7e68036040 tx=0x7f7e68004a40 comp rx=0 tx=0).stop 2023-08-25T09:41:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 shutdown_connections 2023-08-25T09:41:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5406a050 0x7f7e5406c510 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7015ce80 0x7f7e7015b910 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7015caa0 0x7f7e7015b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 --2- 172.21.15.70:0/1980051015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e700b8100 0x7f7e7014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 >> 172.21.15.70:0/1980051015 conn(0x7f7e7006fe70 msgr2=0x7f7e700cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 shutdown_connections 2023-08-25T09:41:14.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:13.997+0000 7f7e75f14640 1 -- 172.21.15.70:0/1980051015 wait complete. 2023-08-25T09:41:14.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:41:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.226+0000 7f93c2312640 1 Processor -- start 2023-08-25T09:41:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.226+0000 7f93c2312640 1 -- start start 2023-08-25T09:41:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93c2312640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93bc15e830 0x7f93bc06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93c2312640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f93bc06b2b0 0x7f93bc15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93c2312640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93bc0b1b60 con 0x7f93bc15c6d0 2023-08-25T09:41:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93c2312640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc06bd80 con 0x7f93bc15e830 2023-08-25T09:41:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93c2312640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93bc06b7c0 con 0x7f93bc06b2b0 2023-08-25T09:41:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93bc15e830 0x7f93bc06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93bbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f93bc15e830 0x7f93bc06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50014/0 (socket says 172.21.15.70:50014) 2023-08-25T09:41:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93bbfff640 1 -- 172.21.15.70:0/2177625024 learned_addr learned my addr 172.21.15.70:0/2177625024 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93b3fff640 1 --2- 172.21.15.70:0/2177625024 >> v2:172.21.15.155:3300/0 conn(0x7f93bc06b2b0 0x7f93bc15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93bbfff640 1 -- 172.21.15.70:0/2177625024 >> v2:172.21.15.155:3300/0 conn(0x7f93bc06b2b0 msgr2=0x7f93bc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:14.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93bbfff640 1 --2- 172.21.15.70:0/2177625024 >> v2:172.21.15.155:3300/0 conn(0x7f93bc06b2b0 0x7f93bc15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.228+0000 7f93bbfff640 1 -- 172.21.15.70:0/2177625024 >> v1:172.21.15.155:6789/0 conn(0x7f93bc15c6d0 legacy=0x7f93bc0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:14.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.229+0000 7f93bbfff640 1 -- 172.21.15.70:0/2177625024 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc06baa0 con 0x7f93bc15e830 2023-08-25T09:41:14.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.229+0000 7f93b3fff640 1 --2- 172.21.15.70:0/2177625024 >> v2:172.21.15.155:3300/0 conn(0x7f93bc06b2b0 0x7f93bc15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.230+0000 7f93bbfff640 1 --2- 172.21.15.70:0/2177625024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc15e830 0x7f93bc06ad70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0cf6f0 tx=0x7f93ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98ce757bd20b98d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.230+0000 7f93bb7fe640 1 -- 172.21.15.70:0/2177625024 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac031870 con 0x7f93bc15e830 2023-08-25T09:41:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.230+0000 7f93bb7fe640 1 -- 172.21.15.70:0/2177625024 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac0319d0 con 0x7f93bc15e830 2023-08-25T09:41:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.230+0000 7f93bb7fe640 1 -- 172.21.15.70:0/2177625024 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac031d60 con 0x7f93bc15e830 2023-08-25T09:41:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 -- 172.21.15.70:0/2177625024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc15e830 msgr2=0x7f93bc06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 --2- 172.21.15.70:0/2177625024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc15e830 0x7f93bc06ad70 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0cf6f0 tx=0x7f93ac02b910 comp rx=0 tx=0).stop 2023-08-25T09:41:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 -- 172.21.15.70:0/2177625024 shutdown_connections 2023-08-25T09:41:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 --2- 172.21.15.70:0/2177625024 >> v2:172.21.15.155:3300/0 conn(0x7f93bc06b2b0 0x7f93bc15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 --2- 172.21.15.70:0/2177625024 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc15e830 0x7f93bc06ad70 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 -- 172.21.15.70:0/2177625024 >> 172.21.15.70:0/2177625024 conn(0x7f93bc069850 msgr2=0x7f93bc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:14.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 -- 172.21.15.70:0/2177625024 shutdown_connections 2023-08-25T09:41:14.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.231+0000 7f93c2312640 1 -- 172.21.15.70:0/2177625024 wait complete. 2023-08-25T09:41:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.233+0000 7f93c2312640 1 Processor -- start 2023-08-25T09:41:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.233+0000 7f93c2312640 1 -- start start 2023-08-25T09:41:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93c2312640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 0x7f93bc1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93c2312640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93bc15c6d0 0x7f93bc1f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93c2312640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93bc15e830 0x7f93bc0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93c2312640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc150c40 con 0x7f93bc06b2b0 2023-08-25T09:41:14.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93c2312640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1f3ad0 con 0x7f93bc15e830 2023-08-25T09:41:14.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93c2312640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93bc1f3d30 con 0x7f93bc15c6d0 2023-08-25T09:41:14.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 0x7f93bc1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 0x7f93bc1539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50016/0 (socket says 172.21.15.70:50016) 2023-08-25T09:41:14.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 -- 172.21.15.70:0/4207962594 learned_addr learned my addr 172.21.15.70:0/4207962594 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:14.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 -- 172.21.15.70:0/4207962594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93bc15c6d0 msgr2=0x7f93bc1f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93bc15c6d0 0x7f93bc1f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 -- 172.21.15.70:0/4207962594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93bc15e830 msgr2=0x7f93bc0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93bc15e830 0x7f93bc0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.234+0000 7f93bbfff640 1 -- 172.21.15.70:0/4207962594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ac0094d0 con 0x7f93bc06b2b0 2023-08-25T09:41:14.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.235+0000 7f93bbfff640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 0x7f93bc1539d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f93ac000c00 tx=0x7f93ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:14.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.235+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac031840 con 0x7f93bc06b2b0 2023-08-25T09:41:14.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.235+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93ac0443f0 con 0x7f93bc06b2b0 2023-08-25T09:41:14.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.235+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93bc1f4010 con 0x7f93bc06b2b0 2023-08-25T09:41:14.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.235+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93ac0447a0 con 0x7f93bc06b2b0 2023-08-25T09:41:14.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.235+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93bc150e50 con 0x7f93bc06b2b0 2023-08-25T09:41:14.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.236+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93bc04e420 con 0x7f93bc06b2b0 2023-08-25T09:41:14.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.237+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f93ac04d450 con 0x7f93bc06b2b0 2023-08-25T09:41:14.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.238+0000 7f93b97fa640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f939c06a050 0x7f939c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.238+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93ac02d040 con 0x7f93bc06b2b0 2023-08-25T09:41:14.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.238+0000 7f93b3fff640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f939c06a050 0x7f939c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.239+0000 7f93b3fff640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f939c06a050 0x7f939c06c510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006fd0 tx=0x7f93a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:14.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.242+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f93ac076400 con 0x7f93bc06b2b0 2023-08-25T09:41:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.436+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93bc0afd50 con 0x7f93bc06b2b0 2023-08-25T09:41:14.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.439+0000 7f93b97fa640 1 -- 172.21.15.70:0/4207962594 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+18318 (secure 0 0 0) 0x7f93ac035090 con 0x7f93bc06b2b0 2023-08-25T09:41:14.443 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:14.443 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":50,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:12.985250+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:41:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f939c06a050 msgr2=0x7f939c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f939c06a050 0x7f939c06c510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f93a4006fd0 tx=0x7f93a4008040 comp rx=0 tx=0).stop 2023-08-25T09:41:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 msgr2=0x7f93bc1539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 0x7f93bc1539d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f93ac000c00 tx=0x7f93ac0046c0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 shutdown_connections 2023-08-25T09:41:14.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f939c06a050 0x7f939c06c510 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93bc15e830 0x7f93bc0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93bc15c6d0 0x7f93bc1f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 --2- 172.21.15.70:0/4207962594 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93bc06b2b0 0x7f93bc1539d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 >> 172.21.15.70:0/4207962594 conn(0x7f93bc069850 msgr2=0x7f93bc0ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.442+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 shutdown_connections 2023-08-25T09:41:14.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.443+0000 7f93c2312640 1 -- 172.21.15.70:0/4207962594 wait complete. 2023-08-25T09:41:14.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:41:14.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.678+0000 7f18c4afa640 1 Processor -- start 2023-08-25T09:41:14.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.678+0000 7f18c4afa640 1 -- start start 2023-08-25T09:41:14.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18c4afa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18c00c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18c4afa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f18c00cdc20 0x7f18c0069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18c4afa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c00cc5f0 con 0x7f18c00ce060 2023-08-25T09:41:14.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18c4afa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c0070650 con 0x7f18c00cdc20 2023-08-25T09:41:14.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18c4afa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0070930 con 0x7f18c015ae50 2023-08-25T09:41:14.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18bed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f18c00ce060 0x7f18c00ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53462/0 (socket says 172.21.15.70:53462) 2023-08-25T09:41:14.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18bed9d640 1 -- 172.21.15.70:0/2070174843 learned_addr learned my addr 172.21.15.70:0/2070174843 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:14.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18bdd9b640 1 --2- 172.21.15.70:0/2070174843 >> v2:172.21.15.155:3300/0 conn(0x7f18c00cdc20 0x7f18c0069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.680+0000 7f18be59c640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18c00c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216622655 0 0) 0x7f18c00cc5f0 con 0x7f18c00ce060 2023-08-25T09:41:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18a8003520 con 0x7f18c00ce060 2023-08-25T09:41:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2266957879 0 0) 0x7f18a8003520 con 0x7f18c00ce060 2023-08-25T09:41:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 msgr2=0x7f18c00c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18c00c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 >> v2:172.21.15.155:3300/0 conn(0x7f18c00cdc20 msgr2=0x7f18c0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:14.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 --2- 172.21.15.70:0/2070174843 >> v2:172.21.15.155:3300/0 conn(0x7f18c00cdc20 0x7f18c0069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0070c10 con 0x7f18c00ce060 2023-08-25T09:41:14.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f18b80019f0 con 0x7f18c00ce060 2023-08-25T09:41:14.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 >> v1:172.21.15.70:6789/0 conn(0x7f18c00ce060 legacy=0x7f18c00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18a8003970 0x7f18a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0070650 con 0x7f18a8004d00 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c0070930 con 0x7f18a8003970 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.681+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a800b980 con 0x7f18c015ae50 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bed9d640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18be59c640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18a8003970 0x7f18a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bed9d640 1 -- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 msgr2=0x7f18a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bdd9b640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bed9d640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18a80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bed9d640 1 -- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18a8003970 msgr2=0x7f18a8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:14.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bed9d640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18a8003970 0x7f18a8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bed9d640 1 -- 172.21.15.70:0/2070174843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8003520 con 0x7f18a8004d00 2023-08-25T09:41:14.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18bdd9b640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c015ae50 0x7f18a80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:14.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.682+0000 7f18be59c640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18a8003970 0x7f18a8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:41:14.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.683+0000 7f18bed9d640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18a8005180 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f18b800a830 tx=0x7f18a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b697b3a5e8caa878 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:14.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.683+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8034630 con 0x7f18a8004d00 2023-08-25T09:41:14.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.683+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b802f070 con 0x7f18a8004d00 2023-08-25T09:41:14.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.683+0000 7f18bd59a640 1 -- 172.21.15.70:0/2070174843 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80349f0 con 0x7f18a8004d00 2023-08-25T09:41:14.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18c4afa640 1 -- 172.21.15.70:0/2070174843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 msgr2=0x7f18a8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18c4afa640 1 --2- 172.21.15.70:0/2070174843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18a8005180 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f18b800a830 tx=0x7f18a80057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18bed9d640 1 -- 172.21.15.70:0/2070174843 reap_dead start 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18c4afa640 1 -- 172.21.15.70:0/2070174843 shutdown_connections 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18c4afa640 1 -- 172.21.15.70:0/2070174843 >> 172.21.15.70:0/2070174843 conn(0x7f18c00cb0d0 msgr2=0x7f18c00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18c4afa640 1 -- 172.21.15.70:0/2070174843 shutdown_connections 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.684+0000 7f18c4afa640 1 -- 172.21.15.70:0/2070174843 wait complete. 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.686+0000 7f18c4afa640 1 Processor -- start 2023-08-25T09:41:14.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.686+0000 7f18c4afa640 1 -- start start 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18c4afa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18c015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18c4afa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c00c80e0 0x7f18c013dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18c4afa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c01f2da0 0x7f18c00c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18c4afa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00c7120 con 0x7f18a8004d00 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18c4afa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00c8810 con 0x7f18c00c80e0 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18c4afa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c00c8560 con 0x7f18c01f2da0 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.687+0000 7f18be59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18c015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18c015bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50042/0 (socket says 172.21.15.70:50042) 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 -- 172.21.15.70:0/439893844 learned_addr learned my addr 172.21.15.70:0/439893844 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 -- 172.21.15.70:0/439893844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c01f2da0 msgr2=0x7f18c00c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c01f2da0 0x7f18c00c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 -- 172.21.15.70:0/439893844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c00c80e0 msgr2=0x7f18c013dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c00c80e0 0x7f18c013dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 -- 172.21.15.70:0/439893844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c01e37f0 con 0x7f18a8004d00 2023-08-25T09:41:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18be59c640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18c015bc40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f18b0011600 tx=0x7f18b0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.688+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0015870 con 0x7f18a8004d00 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.689+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b00159d0 con 0x7f18a8004d00 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.689+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0015d60 con 0x7f18a8004d00 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.689+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c01e3ad0 con 0x7f18a8004d00 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.689+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c01e4090 con 0x7f18a8004d00 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.690+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c0066fb0 con 0x7f18a8004d00 2023-08-25T09:41:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.691+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18b000f490 con 0x7f18a8004d00 2023-08-25T09:41:14.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.692+0000 7f18aeffd640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:14.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.692+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18b0088d70 con 0x7f18a8004d00 2023-08-25T09:41:14.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.692+0000 7f18bdd9b640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:14.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.693+0000 7f18bdd9b640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f18c006ec30 tx=0x7f18b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:14.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.696+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18b0057ee0 con 0x7f18a8004d00 2023-08-25T09:41:14.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:14.926+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f18c00c4c50 con 0x7f18a8004d00 2023-08-25T09:41:15.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.580+0000 7f18aeffd640 1 -- 172.21.15.70:0/439893844 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f18b00220b0 con 0x7f18a8004d00 2023-08-25T09:41:15.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.583+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 msgr2=0x7f18a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.583+0000 7f18c4afa640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f18c006ec30 tx=0x7f18b4008040 comp rx=0 tx=0).stop 2023-08-25T09:41:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.583+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 msgr2=0x7f18c015bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.583+0000 7f18c4afa640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18c015bc40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f18b0011600 tx=0x7f18b0011ac0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 shutdown_connections 2023-08-25T09:41:15.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18c01f2da0 0x7f18c00c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18c00c80e0 0x7f18c013dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 --2- 172.21.15.70:0/439893844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18a8004d00 0x7f18c015bc40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 >> 172.21.15.70:0/439893844 conn(0x7f18c00cb0d0 msgr2=0x7f18c00c2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 shutdown_connections 2023-08-25T09:41:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.584+0000 7f18c4afa640 1 -- 172.21.15.70:0/439893844 wait complete. 2023-08-25T09:41:15.607 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:41:15.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:41:15.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.826+0000 7f778c694640 1 Processor -- start 2023-08-25T09:41:15.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.827+0000 7f778c694640 1 -- start start 2023-08-25T09:41:15.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.828+0000 7f778c694640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77840b8510 0x7f7784167930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:15.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.828+0000 7f778c694640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840b6cc0 con 0x7f77840cd850 2023-08-25T09:41:15.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.828+0000 7f778c694640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77840d1500 con 0x7f778415cfe0 2023-08-25T09:41:15.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.828+0000 7f778c694640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77840d1310 con 0x7f77840b8510 2023-08-25T09:41:15.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778a430640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f77840cd850 0x7f77840cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53476/0 (socket says 172.21.15.70:53476) 2023-08-25T09:41:15.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778a430640 1 -- 172.21.15.70:0/3997887662 learned_addr learned my addr 172.21.15.70:0/3997887662 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:15.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778ac31640 1 --2- 172.21.15.70:0/3997887662 >> v2:172.21.15.70:3300/0 conn(0x7f77840b8510 0x7f7784167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:15.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778942e640 1 -- 172.21.15.70:0/3997887662 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875268849 0 0) 0x7f7780001110 con 0x7f77840cd850 2023-08-25T09:41:15.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778942e640 1 -- 172.21.15.70:0/3997887662 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7770003520 con 0x7f77840cd850 2023-08-25T09:41:15.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778ac31640 1 -- 172.21.15.70:0/3997887662 >> v1:172.21.15.155:6789/0 conn(0x7f778415cfe0 legacy=0x7f77840d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:15.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778ac31640 1 -- 172.21.15.70:0/3997887662 >> v1:172.21.15.70:6789/0 conn(0x7f77840cd850 legacy=0x7f77840cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:15.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.829+0000 7f778ac31640 1 -- 172.21.15.70:0/3997887662 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77840d17e0 con 0x7f77840b8510 2023-08-25T09:41:15.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.830+0000 7f778ac31640 1 --2- 172.21.15.70:0/3997887662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77840b8510 0x7f7784167930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f777c009d90 tx=0x7f777c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0096f6456de042 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:15.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778942e640 1 -- 172.21.15.70:0/3997887662 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c02cc60 con 0x7f77840b8510 2023-08-25T09:41:15.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778942e640 1 -- 172.21.15.70:0/3997887662 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c02cdc0 con 0x7f77840b8510 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778942e640 1 -- 172.21.15.70:0/3997887662 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c034660 con 0x7f77840b8510 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 -- 172.21.15.70:0/3997887662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77840b8510 msgr2=0x7f7784167930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 --2- 172.21.15.70:0/3997887662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77840b8510 0x7f7784167930 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f777c009d90 tx=0x7f777c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 -- 172.21.15.70:0/3997887662 shutdown_connections 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 --2- 172.21.15.70:0/3997887662 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77840b8510 0x7f7784167930 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 -- 172.21.15.70:0/3997887662 >> 172.21.15.70:0/3997887662 conn(0x7f7784069850 msgr2=0x7f7784069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:15.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 -- 172.21.15.70:0/3997887662 shutdown_connections 2023-08-25T09:41:15.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.831+0000 7f778c694640 1 -- 172.21.15.70:0/3997887662 wait complete. 2023-08-25T09:41:15.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.833+0000 7f778c694640 1 Processor -- start 2023-08-25T09:41:15.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.833+0000 7f778c694640 1 -- start start 2023-08-25T09:41:15.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.834+0000 7f778c694640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77840b8510 0x7f77840c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:15.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.834+0000 7f778c694640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77840cd850 0x7f7784151920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:15.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.834+0000 7f778c694640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 0x7f77841f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:15.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.834+0000 7f778c694640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77841f2040 con 0x7f778415cfe0 2023-08-25T09:41:15.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.834+0000 7f778c694640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77841f6600 con 0x7f77840cd850 2023-08-25T09:41:15.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.834+0000 7f778c694640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77840c8b20 con 0x7f77840b8510 2023-08-25T09:41:15.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 0x7f77841f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:15.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 0x7f77841f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50064/0 (socket says 172.21.15.70:50064) 2023-08-25T09:41:15.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 -- 172.21.15.70:0/4032532413 learned_addr learned my addr 172.21.15.70:0/4032532413 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:15.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f7789c2f640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77840cd850 0x7f7784151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:15.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 -- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77840b8510 msgr2=0x7f77840c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:15.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77840b8510 0x7f77840c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 -- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77840cd850 msgr2=0x7f7784151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:15.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77840cd850 0x7f7784151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:15.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 -- 172.21.15.70:0/4032532413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c009a70 con 0x7f778415cfe0 2023-08-25T09:41:15.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f7789c2f640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77840cd850 0x7f7784151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:15.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.835+0000 7f778ac31640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 0x7f77841f60c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f777c0011e0 tx=0x7f777c004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:15.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.836+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c03e610 con 0x7f778415cfe0 2023-08-25T09:41:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.836+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777c03e770 con 0x7f778415cfe0 2023-08-25T09:41:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.836+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777c03eb20 con 0x7f778415cfe0 2023-08-25T09:41:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.836+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77840b2570 con 0x7f778415cfe0 2023-08-25T09:41:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.836+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77840b2b30 con 0x7f778415cfe0 2023-08-25T09:41:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.837+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7784066fb0 con 0x7f778415cfe0 2023-08-25T09:41:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.838+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f777c03b050 con 0x7f778415cfe0 2023-08-25T09:41:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.839+0000 7f777affd640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f776806a050 0x7f776806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.839+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f777c0aafe0 con 0x7f778415cfe0 2023-08-25T09:41:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.839+0000 7f778a430640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f776806a050 0x7f776806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.840+0000 7f778a430640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f776806a050 0x7f776806c510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7780005880 tx=0x7f7780001150 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:15.843+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f777c076770 con 0x7f778415cfe0 2023-08-25T09:41:16.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.068+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f77840b8010 con 0x7f778415cfe0 2023-08-25T09:41:16.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.583+0000 7f777affd640 1 -- 172.21.15.70:0/4032532413 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f777c04e0c0 con 0x7f778415cfe0 2023-08-25T09:41:16.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f776806a050 msgr2=0x7f776806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f776806a050 0x7f776806c510 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7780005880 tx=0x7f7780001150 comp rx=0 tx=0).stop 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 msgr2=0x7f77841f60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 0x7f77841f60c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f777c0011e0 tx=0x7f777c004a90 comp rx=0 tx=0).stop 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 shutdown_connections 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f776806a050 0x7f776806c510 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f778415cfe0 0x7f77841f60c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77840cd850 0x7f7784151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 --2- 172.21.15.70:0/4032532413 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77840b8510 0x7f77840c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.586+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 >> 172.21.15.70:0/4032532413 conn(0x7f7784069850 msgr2=0x7f778415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.587+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 shutdown_connections 2023-08-25T09:41:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.587+0000 7f778c694640 1 -- 172.21.15.70:0/4032532413 wait complete. 2023-08-25T09:41:16.610 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:41:16.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:41:16.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.829+0000 7fcc72438640 1 Processor -- start 2023-08-25T09:41:16.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.829+0000 7fcc72438640 1 -- start start 2023-08-25T09:41:16.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.831+0000 7fcc72438640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 0x7fcc6c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.831+0000 7fcc72438640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0b1b60 con 0x7fcc6c06b2b0 2023-08-25T09:41:16.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.831+0000 7fcc72438640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c06b7c0 con 0x7fcc6c15ec10 2023-08-25T09:41:16.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.831+0000 7fcc72438640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c06bd80 con 0x7fcc6c15e830 2023-08-25T09:41:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.831+0000 7fcc6b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcc6c06b2b0 0x7fcc6c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53480/0 (socket says 172.21.15.70:53480) 2023-08-25T09:41:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.831+0000 7fcc6b7fe640 1 -- 172.21.15.70:0/3896405467 learned_addr learned my addr 172.21.15.70:0/3896405467 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6bfff640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 0x7fcc6c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457195777 0 0) 0x7fcc6c0b1b60 con 0x7fcc6c06b2b0 2023-08-25T09:41:16.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc54003520 con 0x7fcc6c06b2b0 2023-08-25T09:41:16.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3291194689 0 0) 0x7fcc54003520 con 0x7fcc6c06b2b0 2023-08-25T09:41:16.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 msgr2=0x7fcc6c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 0x7fcc6c06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 >> v1:172.21.15.155:6789/0 conn(0x7fcc6c15ec10 legacy=0x7fcc6c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c06baa0 con 0x7fcc6c06b2b0 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fcc60001670 con 0x7fcc6c06b2b0 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 >> v1:172.21.15.70:6789/0 conn(0x7fcc6c06b2b0 legacy=0x7fcc6c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 0x7fcc540039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc54003f20 0x7fcc540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc54004d00 0x7fcc54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c06bd80 con 0x7fcc54003f20 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5400ba60 con 0x7fcc54004d00 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.832+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5400bc20 con 0x7fcc6c15e830 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc6b7fe640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc54003f20 0x7fcc540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc6b7fe640 1 -- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 msgr2=0x7fcc540039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:16.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc6b7fe640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc6c15e830 0x7fcc540039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc6b7fe640 1 -- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc54004d00 msgr2=0x7fcc54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:16.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc709d5640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc54004d00 0x7fcc54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc6b7fe640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc54004d00 0x7fcc54005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc6b7fe640 1 -- 172.21.15.70:0/3896405467 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54003520 con 0x7fcc54003f20 2023-08-25T09:41:16.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.833+0000 7fcc709d5640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc54004d00 0x7fcc54005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:16.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.834+0000 7fcc6b7fe640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc54003f20 0x7fcc540047c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc60007fd0 tx=0x7fcc540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda4470d21dbe63e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:16.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.834+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60005ad0 con 0x7fcc54003f20 2023-08-25T09:41:16.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.834+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc60005c30 con 0x7fcc54003f20 2023-08-25T09:41:16.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.834+0000 7fcc6affd640 1 -- 172.21.15.70:0/3896405467 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6002cab0 con 0x7fcc54003f20 2023-08-25T09:41:16.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc72438640 1 -- 172.21.15.70:0/3896405467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc54003f20 msgr2=0x7fcc540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc72438640 1 --2- 172.21.15.70:0/3896405467 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc54003f20 0x7fcc540047c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fcc60007fd0 tx=0x7fcc540057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc709d5640 1 -- 172.21.15.70:0/3896405467 reap_dead start 2023-08-25T09:41:16.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc72438640 1 -- 172.21.15.70:0/3896405467 shutdown_connections 2023-08-25T09:41:16.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc72438640 1 -- 172.21.15.70:0/3896405467 >> 172.21.15.70:0/3896405467 conn(0x7fcc6c069850 msgr2=0x7fcc6c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:16.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc72438640 1 -- 172.21.15.70:0/3896405467 shutdown_connections 2023-08-25T09:41:16.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.835+0000 7fcc72438640 1 -- 172.21.15.70:0/3896405467 wait complete. 2023-08-25T09:41:16.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.837+0000 7fcc72438640 1 Processor -- start 2023-08-25T09:41:16.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.837+0000 7fcc72438640 1 -- start start 2023-08-25T09:41:16.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc72438640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc54003f20 0x7fcc6c0c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc72438640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc6c153180 0x7fcc6c145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc72438640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 0x7fcc6c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc72438640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0cd120 con 0x7fcc6c0cc760 2023-08-25T09:41:16.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc72438640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c151c80 con 0x7fcc6c153180 2023-08-25T09:41:16.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc72438640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c146510 con 0x7fcc54003f20 2023-08-25T09:41:16.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc709d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 0x7fcc6c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc709d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 0x7fcc6c0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50090/0 (socket says 172.21.15.70:50090) 2023-08-25T09:41:16.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.838+0000 7fcc709d5640 1 -- 172.21.15.70:0/440961621 learned_addr learned my addr 172.21.15.70:0/440961621 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:16.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc6bfff640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc54003f20 0x7fcc6c0c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc6b7fe640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc6c153180 0x7fcc6c145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc709d5640 1 -- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc54003f20 msgr2=0x7fcc6c0c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc709d5640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc54003f20 0x7fcc6c0c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc709d5640 1 -- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc6c153180 msgr2=0x7fcc6c145fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:16.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc709d5640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc6c153180 0x7fcc6c145fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:16.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc709d5640 1 -- 172.21.15.70:0/440961621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c15ca20 con 0x7fcc6c0cc760 2023-08-25T09:41:16.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc6b7fe640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc6c153180 0x7fcc6c145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:16.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc709d5640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 0x7fcc6c0ccbe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcc64000f80 tx=0x7fcc6400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:16.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64006d50 con 0x7fcc6c0cc760 2023-08-25T09:41:16.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc64003970 con 0x7fcc6c0cc760 2023-08-25T09:41:16.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c150be0 con 0x7fcc6c0cc760 2023-08-25T09:41:16.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.839+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64003d60 con 0x7fcc6c0cc760 2023-08-25T09:41:16.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.840+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c1511a0 con 0x7fcc6c0cc760 2023-08-25T09:41:16.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.841+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c04e420 con 0x7fcc6c0cc760 2023-08-25T09:41:16.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.842+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcc64005070 con 0x7fcc6c0cc760 2023-08-25T09:41:16.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.843+0000 7fcc68ff9640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcc5806a080 0x7fcc5806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:16.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.843+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc64088440 con 0x7fcc6c0cc760 2023-08-25T09:41:16.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.843+0000 7fcc6bfff640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcc5806a080 0x7fcc5806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:16.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.844+0000 7fcc6bfff640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcc5806a080 0x7fcc5806c540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009a50 tx=0x7fcc5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:16.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:16.847+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcc64053a90 con 0x7fcc6c0cc760 2023-08-25T09:41:17.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.065+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc6c06c620 con 0x7fcc6c0cc760 2023-08-25T09:41:17.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.066+0000 7fcc68ff9640 1 -- 172.21.15.70:0/440961621 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7fcc640570b0 con 0x7fcc6c0cc760 2023-08-25T09:41:17.070 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:17.070 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:41:17.070 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:41:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcc5806a080 msgr2=0x7fcc5806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcc5806a080 0x7fcc5806c540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009a50 tx=0x7fcc5c008040 comp rx=0 tx=0).stop 2023-08-25T09:41:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 msgr2=0x7fcc6c0ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 0x7fcc6c0ccbe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcc64000f80 tx=0x7fcc6400dc00 comp rx=0 tx=0).stop 2023-08-25T09:41:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 shutdown_connections 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcc5806a080 0x7fcc5806c540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc6c0cc760 0x7fcc6c0ccbe0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc6c153180 0x7fcc6c145fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 --2- 172.21.15.70:0/440961621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc54003f20 0x7fcc6c0c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 >> 172.21.15.70:0/440961621 conn(0x7fcc6c069850 msgr2=0x7fcc6c15b920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 shutdown_connections 2023-08-25T09:41:17.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.069+0000 7fcc72438640 1 -- 172.21.15.70:0/440961621 wait complete. 2023-08-25T09:41:17.098 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:41:17.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.296+0000 7ff707fff640 1 Processor -- start 2023-08-25T09:41:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.296+0000 7ff707fff640 1 -- start start 2023-08-25T09:41:17.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.297+0000 7ff707fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 0x7ff7080b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.297+0000 7ff707fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 0x7ff7080b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.297+0000 7ff707fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70815d890 con 0x7ff7080cadb0 2023-08-25T09:41:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.297+0000 7ff707fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7080c8b60 con 0x7ff708159840 2023-08-25T09:41:17.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.297+0000 7ff707fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70815db70 con 0x7ff70815d3d0 2023-08-25T09:41:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.298+0000 7ff6fffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 0x7ff7080b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.298+0000 7ff706ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 0x7ff7080b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.298+0000 7ff706ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 0x7ff7080b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:54810/0 (socket says 172.21.15.70:54810) 2023-08-25T09:41:17.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.298+0000 7ff706ffd640 1 -- 172.21.15.70:0/3668903754 learned_addr learned my addr 172.21.15.70:0/3668903754 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:17.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.298+0000 7ff6fffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 0x7ff7080b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:42520/0 (socket says 172.21.15.70:42520) 2023-08-25T09:41:17.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.299+0000 7ff6fffff640 1 -- 172.21.15.70:0/3668903754 >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 msgr2=0x7ff7080b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:17.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.299+0000 7ff6fffff640 1 --2- 172.21.15.70:0/3668903754 >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 0x7ff7080b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.299+0000 7ff6fffff640 1 -- 172.21.15.70:0/3668903754 >> v1:172.21.15.155:6789/0 conn(0x7ff7080cadb0 legacy=0x7ff7080cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:17.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.299+0000 7ff6fffff640 1 -- 172.21.15.70:0/3668903754 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70815de50 con 0x7ff70815d3d0 2023-08-25T09:41:17.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.299+0000 7ff706ffd640 1 --2- 172.21.15.70:0/3668903754 >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 0x7ff7080b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:41:17.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.300+0000 7ff6fffff640 1 --2- 172.21.15.70:0/3668903754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 0x7ff7080b3d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00097a0 tx=0x7ff6f002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7839b6e8497d695 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:17.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.301+0000 7ff7067fc640 1 -- 172.21.15.70:0/3668903754 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0031780 con 0x7ff70815d3d0 2023-08-25T09:41:17.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.301+0000 7ff7067fc640 1 -- 172.21.15.70:0/3668903754 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00318e0 con 0x7ff70815d3d0 2023-08-25T09:41:17.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.301+0000 7ff7067fc640 1 -- 172.21.15.70:0/3668903754 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0031c70 con 0x7ff70815d3d0 2023-08-25T09:41:17.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 -- 172.21.15.70:0/3668903754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 msgr2=0x7ff7080b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 --2- 172.21.15.70:0/3668903754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 0x7ff7080b3d40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00097a0 tx=0x7ff6f002b820 comp rx=0 tx=0).stop 2023-08-25T09:41:17.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 -- 172.21.15.70:0/3668903754 shutdown_connections 2023-08-25T09:41:17.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 --2- 172.21.15.70:0/3668903754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff70815d3d0 0x7ff7080b3d40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 --2- 172.21.15.70:0/3668903754 >> v2:172.21.15.155:3300/0 conn(0x7ff708159840 0x7ff7080b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 -- 172.21.15.70:0/3668903754 >> 172.21.15.70:0/3668903754 conn(0x7ff7080affd0 msgr2=0x7ff7080b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:17.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 -- 172.21.15.70:0/3668903754 shutdown_connections 2023-08-25T09:41:17.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.302+0000 7ff707fff640 1 -- 172.21.15.70:0/3668903754 wait complete. 2023-08-25T09:41:17.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.304+0000 7ff707fff640 1 Processor -- start 2023-08-25T09:41:17.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.304+0000 7ff707fff640 1 -- start start 2023-08-25T09:41:17.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff707fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7080cadb0 0x7ff7081f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff707fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff708159840 0x7ff7080c2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff707fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 0x7ff708152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff707fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c42f0 con 0x7ff70815d3d0 2023-08-25T09:41:17.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff707fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff708154890 con 0x7ff7080cadb0 2023-08-25T09:41:17.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff707fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7080c3420 con 0x7ff708159840 2023-08-25T09:41:17.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 0x7ff708152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 0x7ff708152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50102/0 (socket says 172.21.15.70:50102) 2023-08-25T09:41:17.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 -- 172.21.15.70:0/3982507726 learned_addr learned my addr 172.21.15.70:0/3982507726 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:17.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff6fffff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff708159840 0x7ff7080c2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff706ffd640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7080cadb0 0x7ff7081f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 -- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff708159840 msgr2=0x7ff7080c2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff708159840 0x7ff7080c2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 -- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7080cadb0 msgr2=0x7ff7081f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7080cadb0 0x7ff7081f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff7077fe640 1 -- 172.21.15.70:0/3982507726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f800b770 con 0x7ff70815d3d0 2023-08-25T09:41:17.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.305+0000 7ff706ffd640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7080cadb0 0x7ff7081f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.306+0000 7ff7077fe640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 0x7ff708152d20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff70000b970 tx=0x7ff70000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.306+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70000cb40 con 0x7ff70815d3d0 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.306+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff70000cca0 con 0x7ff70815d3d0 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.306+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff700014680 con 0x7ff70815d3d0 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.306+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f0009400 con 0x7ff70815d3d0 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.306+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7080c39c0 con 0x7ff70815d3d0 2023-08-25T09:41:17.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.307+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff708066fb0 con 0x7ff70815d3d0 2023-08-25T09:41:17.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.309+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff70001d3b0 con 0x7ff70815d3d0 2023-08-25T09:41:17.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.309+0000 7ff6ff7fe640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff6e806a050 0x7ff6e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.309+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff700088390 con 0x7ff70815d3d0 2023-08-25T09:41:17.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.310+0000 7ff706ffd640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff6e806a050 0x7ff6e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.310+0000 7ff706ffd640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff6e806a050 0x7ff6e806c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff70815bca0 tx=0x7ff6f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:17.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.314+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff700053a80 con 0x7ff70815d3d0 2023-08-25T09:41:17.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.492+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff708071b30 con 0x7ff70815d3d0 2023-08-25T09:41:17.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.494+0000 7ff6ff7fe640 1 -- 172.21.15.70:0/3982507726 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+18305 (secure 0 0 0) 0x7ff7000570a0 con 0x7ff70815d3d0 2023-08-25T09:41:17.497 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:17.497 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":51,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:14.522565+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:41:17.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff6e806a050 msgr2=0x7ff6e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff6e806a050 0x7ff6e806c510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff70815bca0 tx=0x7ff6f8008040 comp rx=0 tx=0).stop 2023-08-25T09:41:17.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 msgr2=0x7ff708152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 0x7ff708152d20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff70000b970 tx=0x7ff70000be30 comp rx=0 tx=0).stop 2023-08-25T09:41:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 shutdown_connections 2023-08-25T09:41:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff6e806a050 0x7ff6e806c510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff70815d3d0 0x7ff708152d20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff708159840 0x7ff7080c2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 --2- 172.21.15.70:0/3982507726 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff7080cadb0 0x7ff7081f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 >> 172.21.15.70:0/3982507726 conn(0x7ff7080affd0 msgr2=0x7ff708158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.496+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 shutdown_connections 2023-08-25T09:41:17.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.497+0000 7ff707fff640 1 -- 172.21.15.70:0/3982507726 wait complete. 2023-08-25T09:41:17.517 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:41:17.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:41:17.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.729+0000 7f6981070640 1 Processor -- start 2023-08-25T09:41:17.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.729+0000 7f6981070640 1 -- start start 2023-08-25T09:41:17.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f6981070640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f697c0b2850 0x7f697c164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f6981070640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c15d3f0 con 0x7f697c15aa10 2023-08-25T09:41:17.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f6981070640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c070a00 con 0x7f697c0ce5b0 2023-08-25T09:41:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f6981070640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c159d10 con 0x7f697c0b2850 2023-08-25T09:41:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f697259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f697c15aa10 0x7f697c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53482/0 (socket says 172.21.15.70:53482) 2023-08-25T09:41:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f697259c640 1 -- 172.21.15.70:0/670071175 learned_addr learned my addr 172.21.15.70:0/670071175 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:17.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f697b59e640 1 --2- 172.21.15.70:0/670071175 >> v2:172.21.15.155:3300/0 conn(0x7f697c0b2850 0x7f697c164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069553219 0 0) 0x7f697c15d3f0 con 0x7f697c15aa10 2023-08-25T09:41:17.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.731+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f695c003520 con 0x7f697c15aa10 2023-08-25T09:41:17.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2892105311 0 0) 0x7f695c003520 con 0x7f697c15aa10 2023-08-25T09:41:17.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 >> v2:172.21.15.155:3300/0 conn(0x7f697c0b2850 msgr2=0x7f697c164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:17.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 --2- 172.21.15.70:0/670071175 >> v2:172.21.15.155:3300/0 conn(0x7f697c0b2850 0x7f697c164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 >> v1:172.21.15.155:6789/0 conn(0x7f697c0ce5b0 legacy=0x7f697c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0b2d20 con 0x7f697c15aa10 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6964001670 con 0x7f697c15aa10 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 >> v1:172.21.15.70:6789/0 conn(0x7f697c15aa10 legacy=0x7f697c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f695c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695c0047c0 0x7f695c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695c0052a0 0x7f695c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c159d10 con 0x7f695c003970 2023-08-25T09:41:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c00c1a0 con 0x7f695c0047c0 2023-08-25T09:41:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c00c360 con 0x7f695c0052a0 2023-08-25T09:41:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.732+0000 7f697ad9d640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f695c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697259c640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695c0047c0 0x7f695c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697ad9d640 1 -- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695c0052a0 msgr2=0x7f695c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697ad9d640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f695c0052a0 0x7f695c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697ad9d640 1 -- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695c0047c0 msgr2=0x7f695c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697ad9d640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f695c0047c0 0x7f695c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697b59e640 1 -- 172.21.15.70:0/670071175 reap_dead start 2023-08-25T09:41:17.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.733+0000 7f697ad9d640 1 -- 172.21.15.70:0/670071175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c003520 con 0x7f695c003970 2023-08-25T09:41:17.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.734+0000 7f697ad9d640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f695c003db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f696c001720 tx=0x7f695c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3adb4bdb3ebc69 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.734+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c0052d0 con 0x7f695c003970 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.734+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c02e050 con 0x7f695c003970 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.734+0000 7f697a59c640 1 -- 172.21.15.70:0/670071175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c003830 con 0x7f695c003970 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 -- 172.21.15.70:0/670071175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 msgr2=0x7f695c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f695c003db0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f696c001720 tx=0x7f695c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 -- 172.21.15.70:0/670071175 shutdown_connections 2023-08-25T09:41:17.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 --2- 172.21.15.70:0/670071175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f695c003db0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 -- 172.21.15.70:0/670071175 >> 172.21.15.70:0/670071175 conn(0x7f697c069850 msgr2=0x7f697c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 -- 172.21.15.70:0/670071175 shutdown_connections 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.735+0000 7f6981070640 1 -- 172.21.15.70:0/670071175 wait complete. 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.737+0000 7f6981070640 1 Processor -- start 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.737+0000 7f6981070640 1 -- start start 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.737+0000 7f6981070640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f697c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.737+0000 7f6981070640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c14abe0 0x7f697c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f6981070640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c0af1e0 0x7f697c15b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f6981070640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c14dc60 con 0x7f695c003970 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f6981070640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c14d630 con 0x7f697c0af1e0 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f6981070640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0af7a0 con 0x7f697c14abe0 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f697c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f697c153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50126/0 (socket says 172.21.15.70:50126) 2023-08-25T09:41:17.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 -- 172.21.15.70:0/1220348075 learned_addr learned my addr 172.21.15.70:0/1220348075 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697b59e640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c0af1e0 0x7f697c15b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 -- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c14abe0 msgr2=0x7f697c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c14abe0 0x7f697c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 -- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c0af1e0 msgr2=0x7f697c15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c0af1e0 0x7f697c15b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.738+0000 7f697ad9d640 1 -- 172.21.15.70:0/1220348075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0afa80 con 0x7f695c003970 2023-08-25T09:41:17.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.739+0000 7f697ad9d640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f697c153c60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f696c005040 tx=0x7f696c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.739+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c007a30 con 0x7f695c003970 2023-08-25T09:41:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.739+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c0069e0 con 0x7f695c003970 2023-08-25T09:41:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.739+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c006d70 con 0x7f695c003970 2023-08-25T09:41:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.739+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c14d8f0 con 0x7f695c003970 2023-08-25T09:41:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.739+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1541a0 con 0x7f695c003970 2023-08-25T09:41:17.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.740+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c0afc10 con 0x7f695c003970 2023-08-25T09:41:17.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.741+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f696c0035f0 con 0x7f695c003970 2023-08-25T09:41:17.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.742+0000 7f6973fff640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696006a050 0x7f696006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:17.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.742+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f696c0aa720 con 0x7f695c003970 2023-08-25T09:41:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.742+0000 7f697259c640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696006a050 0x7f696006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.743+0000 7f697259c640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696006a050 0x7f696006c510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f69640017a0 tx=0x7f6964009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:17.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.746+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f696c075e00 con 0x7f695c003970 2023-08-25T09:41:17.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.932+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f697c15d990 con 0x7f696006a050 2023-08-25T09:41:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.948+0000 7f6973fff640 1 -- 172.21.15.70:0/1220348075 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302767 (secure 0 0 0) 0x7f6964001c10 con 0x7f696006a050 2023-08-25T09:41:17.954 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:41:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696006a050 msgr2=0x7f696006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696006a050 0x7f696006c510 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f69640017a0 tx=0x7f6964009040 comp rx=0 tx=0).stop 2023-08-25T09:41:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 msgr2=0x7f697c153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f697c153c60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f696c005040 tx=0x7f696c001120 comp rx=0 tx=0).stop 2023-08-25T09:41:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 shutdown_connections 2023-08-25T09:41:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696006a050 0x7f696006c510 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f697c0af1e0 0x7f697c15b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f697c14abe0 0x7f697c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 --2- 172.21.15.70:0/1220348075 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f695c003970 0x7f697c153c60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 >> 172.21.15.70:0/1220348075 conn(0x7f697c069850 msgr2=0x7f697c15c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 shutdown_connections 2023-08-25T09:41:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:17.954+0000 7f6981070640 1 -- 172.21.15.70:0/1220348075 wait complete. 2023-08-25T09:41:17.976 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2023-08-25T09:41:17.364296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3127,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559372,"statfs":{"total":1600000000,"available":1596797846,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.880560"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557059+0000","last_change":"2023-08-25T09:41:13.556673+0000","last_active":"2023-08-25T09:41:13.557059+0000","last_peered":"2023-08-25T09:41:13.557059+0000","last_clean":"2023-08-25T09:41:13.557059+0000","last_became_active":"2023-08-25T09:41:13.555714+0000","last_became_peered":"2023-08-25T09:41:13.555714+0000","last_unstale":"2023-08-25T09:41:13.557059+0000","last_undegraded":"2023-08-25T09:41:13.557059+0000","last_fullsized":"2023-08-25T09:41:13.557059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:41.177052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.550206+0000","last_change":"2023-08-25T09:41:13.550125+0000","last_active":"2023-08-25T09:41:13.550206+0000","last_peered":"2023-08-25T09:41:13.550206+0000","last_clean":"2023-08-25T09:41:13.550206+0000","last_became_active":"2023-08-25T09:41:13.549751+0000","last_became_peered":"2023-08-25T09:41:13.549751+0000","last_unstale":"2023-08-25T09:41:13.550206+0000","last_undegraded":"2023-08-25T09:41:13.550206+0000","last_fullsized":"2023-08-25T09:41:13.550206+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:17:21.788914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559057+0000","last_change":"2023-08-25T09:41:13.558900+0000","last_active":"2023-08-25T09:41:13.559057+0000","last_peered":"2023-08-25T09:41:13.559057+0000","last_clean":"2023-08-25T09:41:13.559057+0000","last_became_active":"2023-08-25T09:41:13.558569+0000","last_became_peered":"2023-08-25T09:41:13.558569+0000","last_unstale":"2023-08-25T09:41:13.559057+0000","last_undegraded":"2023-08-25T09:41:13.559057+0000","last_fullsized":"2023-08-25T09:41:13.559057+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:26:20.502288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556775+0000","last_change":"2023-08-25T09:41:13.556515+0000","last_active":"2023-08-25T09:41:13.556775+0000","last_peered":"2023-08-25T09:41:13.556775+0000","last_clean":"2023-08-25T09:41:13.556775+0000","last_became_active":"2023-08-25T09:41:13.554947+0000","last_became_peered":"2023-08-25T09:41:13.554947+0000","last_unstale":"2023-08-25T09:41:13.556775+0000","last_undegraded":"2023-08-25T09:41:13.556775+0000","last_fullsized":"2023-08-25T09:41:13.556775+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:52:22.767545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.541443+0000","last_change":"2023-08-25T09:41:13.541056+0000","last_active":"2023-08-25T09:41:13.541443+0000","last_peered":"2023-08-25T09:41:13.541443+0000","last_clean":"2023-08-25T09:41:13.541443+0000","last_became_active":"2023-08-25T09:41:13.537590+0000","last_became_peered":"2023-08-25T09:41:13.537590+0000","last_unstale":"2023-08-25T09:41:13.541443+0000","last_undegraded":"2023-08-25T09:41:13.541443+0000","last_fullsized":"2023-08-25T09:41:13.541443+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:22:20.266956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553906+0000","last_change":"2023-08-25T09:41:13.551465+0000","last_active":"2023-08-25T09:41:13.553906+0000","last_peered":"2023-08-25T09:41:13.553906+0000","last_clean":"2023-08-25T09:41:13.553906+0000","last_became_active":"2023-08-25T09:41:13.550778+0000","last_became_peered":"2023-08-25T09:41:13.550778+0000","last_unstale":"2023-08-25T09:41:13.553906+0000","last_undegraded":"2023-08-25T09:41:13.553906+0000","last_fullsized":"2023-08-25T09:41:13.553906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:50.350958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549212+0000","last_change":"2023-08-25T09:41:13.549071+0000","last_active":"2023-08-25T09:41:13.549212+0000","last_peered":"2023-08-25T09:41:13.549212+0000","last_clean":"2023-08-25T09:41:13.549212+0000","last_became_active":"2023-08-25T09:41:13.547530+0000","last_became_peered":"2023-08-25T09:41:13.547530+0000","last_unstale":"2023-08-25T09:41:13.549212+0000","last_undegraded":"2023-08-25T09:41:13.549212+0000","last_fullsized":"2023-08-25T09:41:13.549212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:09:24.021766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559707+0000","last_change":"2023-08-25T09:41:13.559609+0000","last_active":"2023-08-25T09:41:13.559707+0000","last_peered":"2023-08-25T09:41:13.559707+0000","last_clean":"2023-08-25T09:41:13.559707+0000","last_became_active":"2023-08-25T09:41:13.559163+0000","last_became_peered":"2023-08-25T09:41:13.559163+0000","last_unstale":"2023-08-25T09:41:13.559707+0000","last_undegraded":"2023-08-25T09:41:13.559707+0000","last_fullsized":"2023-08-25T09:41:13.559707+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:45.056101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555792+0000","last_change":"2023-08-25T09:41:13.555515+0000","last_active":"2023-08-25T09:41:13.555792+0000","last_peered":"2023-08-25T09:41:13.555792+0000","last_clean":"2023-08-25T09:41:13.555792+0000","last_became_active":"2023-08-25T09:41:13.555189+0000","last_became_peered":"2023-08-25T09:41:13.555189+0000","last_unstale":"2023-08-25T09:41:13.555792+0000","last_undegraded":"2023-08-25T09:41:13.555792+0000","last_fullsized":"2023-08-25T09:41:13.555792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:27:44.405255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557056+0000","last_change":"2023-08-25T09:41:13.556924+0000","last_active":"2023-08-25T09:41:13.557056+0000","last_peered":"2023-08-25T09:41:13.557056+0000","last_clean":"2023-08-25T09:41:13.557056+0000","last_became_active":"2023-08-25T09:41:13.556667+0000","last_became_peered":"2023-08-25T09:41:13.556667+0000","last_unstale":"2023-08-25T09:41:13.557056+0000","last_undegraded":"2023-08-25T09:41:13.557056+0000","last_fullsized":"2023-08-25T09:41:13.557056+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:06.224484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554954+0000","last_change":"2023-08-25T09:41:13.553702+0000","last_active":"2023-08-25T09:41:13.554954+0000","last_peered":"2023-08-25T09:41:13.554954+0000","last_clean":"2023-08-25T09:41:13.554954+0000","last_became_active":"2023-08-25T09:41:13.553424+0000","last_became_peered":"2023-08-25T09:41:13.553424+0000","last_unstale":"2023-08-25T09:41:13.554954+0000","last_undegraded":"2023-08-25T09:41:13.554954+0000","last_fullsized":"2023-08-25T09:41:13.554954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:18:27.360452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555955+0000","last_change":"2023-08-25T09:41:13.548665+0000","last_active":"2023-08-25T09:41:13.555955+0000","last_peered":"2023-08-25T09:41:13.555955+0000","last_clean":"2023-08-25T09:41:13.555955+0000","last_became_active":"2023-08-25T09:41:13.548083+0000","last_became_peered":"2023-08-25T09:41:13.548083+0000","last_unstale":"2023-08-25T09:41:13.555955+0000","last_undegraded":"2023-08-25T09:41:13.555955+0000","last_fullsized":"2023-08-25T09:41:13.555955+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:41.655105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.541577+0000","last_change":"2023-08-25T09:41:13.539800+0000","last_active":"2023-08-25T09:41:13.541577+0000","last_peered":"2023-08-25T09:41:13.541577+0000","last_clean":"2023-08-25T09:41:13.541577+0000","last_became_active":"2023-08-25T09:41:13.539070+0000","last_became_peered":"2023-08-25T09:41:13.539070+0000","last_unstale":"2023-08-25T09:41:13.541577+0000","last_undegraded":"2023-08-25T09:41:13.541577+0000","last_fullsized":"2023-08-25T09:41:13.541577+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:16.561672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556481+0000","last_change":"2023-08-25T09:41:13.555925+0000","last_active":"2023-08-25T09:41:13.556481+0000","last_peered":"2023-08-25T09:41:13.556481+0000","last_clean":"2023-08-25T09:41:13.556481+0000","last_became_active":"2023-08-25T09:41:13.555603+0000","last_became_peered":"2023-08-25T09:41:13.555603+0000","last_unstale":"2023-08-25T09:41:13.556481+0000","last_undegraded":"2023-08-25T09:41:13.556481+0000","last_fullsized":"2023-08-25T09:41:13.556481+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:48.330370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.546620+0000","last_change":"2023-08-25T09:41:13.546354+0000","last_active":"2023-08-25T09:41:13.546620+0000","last_peered":"2023-08-25T09:41:13.546620+0000","last_clean":"2023-08-25T09:41:13.546620+0000","last_became_active":"2023-08-25T09:41:13.545510+0000","last_became_peered":"2023-08-25T09:41:13.545510+0000","last_unstale":"2023-08-25T09:41:13.546620+0000","last_undegraded":"2023-08-25T09:41:13.546620+0000","last_fullsized":"2023-08-25T09:41:13.546620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:56:03.228283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553891+0000","last_change":"2023-08-25T09:41:13.551194+0000","last_active":"2023-08-25T09:41:13.553891+0000","last_peered":"2023-08-25T09:41:13.553891+0000","last_clean":"2023-08-25T09:41:13.553891+0000","last_became_active":"2023-08-25T09:41:13.550330+0000","last_became_peered":"2023-08-25T09:41:13.550330+0000","last_unstale":"2023-08-25T09:41:13.553891+0000","last_undegraded":"2023-08-25T09:41:13.553891+0000","last_fullsized":"2023-08-25T09:41:13.553891+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:00:04.708379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556501+0000","last_change":"2023-08-25T09:41:13.556438+0000","last_active":"2023-08-25T09:41:13.556501+0000","last_peered":"2023-08-25T09:41:13.556501+0000","last_clean":"2023-08-25T09:41:13.556501+0000","last_became_active":"2023-08-25T09:41:13.555917+0000","last_became_peered":"2023-08-25T09:41:13.555917+0000","last_unstale":"2023-08-25T09:41:13.556501+0000","last_undegraded":"2023-08-25T09:41:13.556501+0000","last_fullsized":"2023-08-25T09:41:13.556501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:06.224484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554646+0000","last_change":"2023-08-25T09:41:13.554458+0000","last_active":"2023-08-25T09:41:13.554646+0000","last_peered":"2023-08-25T09:41:13.554646+0000","last_clean":"2023-08-25T09:41:13.554646+0000","last_became_active":"2023-08-25T09:41:13.554005+0000","last_became_peered":"2023-08-25T09:41:13.554005+0000","last_unstale":"2023-08-25T09:41:13.554646+0000","last_undegraded":"2023-08-25T09:41:13.554646+0000","last_fullsized":"2023-08-25T09:41:13.554646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:48.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553935+0000","last_change":"2023-08-25T09:41:13.553867+0000","last_active":"2023-08-25T09:41:13.553935+0000","last_peered":"2023-08-25T09:41:13.553935+0000","last_clean":"2023-08-25T09:41:13.553935+0000","last_became_active":"2023-08-25T09:41:13.553458+0000","last_became_peered":"2023-08-25T09:41:13.553458+0000","last_unstale":"2023-08-25T09:41:13.553935+0000","last_undegraded":"2023-08-25T09:41:13.553935+0000","last_fullsized":"2023-08-25T09:41:13.553935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:32.662025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555719+0000","last_change":"2023-08-25T09:41:13.555601+0000","last_active":"2023-08-25T09:41:13.555719+0000","last_peered":"2023-08-25T09:41:13.555719+0000","last_clean":"2023-08-25T09:41:13.555719+0000","last_became_active":"2023-08-25T09:41:13.554965+0000","last_became_peered":"2023-08-25T09:41:13.554965+0000","last_unstale":"2023-08-25T09:41:13.555719+0000","last_undegraded":"2023-08-25T09:41:13.555719+0000","last_fullsized":"2023-08-25T09:41:13.555719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:17:09.525913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553987+0000","last_change":"2023-08-25T09:41:13.553869+0000","last_active":"2023-08-25T09:41:13.553987+0000","last_peered":"2023-08-25T09:41:13.553987+0000","last_clean":"2023-08-25T09:41:13.553987+0000","last_became_active":"2023-08-25T09:41:13.553305+0000","last_became_peered":"2023-08-25T09:41:13.553305+0000","last_unstale":"2023-08-25T09:41:13.553987+0000","last_undegraded":"2023-08-25T09:41:13.553987+0000","last_fullsized":"2023-08-25T09:41:13.553987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:24.016454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.552840+0000","last_change":"2023-08-25T09:41:13.552708+0000","last_active":"2023-08-25T09:41:13.552840+0000","last_peered":"2023-08-25T09:41:13.552840+0000","last_clean":"2023-08-25T09:41:13.552840+0000","last_became_active":"2023-08-25T09:41:13.552166+0000","last_became_peered":"2023-08-25T09:41:13.552166+0000","last_unstale":"2023-08-25T09:41:13.552840+0000","last_undegraded":"2023-08-25T09:41:13.552840+0000","last_fullsized":"2023-08-25T09:41:13.552840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:37.360613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.544562+0000","last_change":"2023-08-25T09:41:13.544321+0000","last_active":"2023-08-25T09:41:13.544562+0000","last_peered":"2023-08-25T09:41:13.544562+0000","last_clean":"2023-08-25T09:41:13.544562+0000","last_became_active":"2023-08-25T09:41:13.543707+0000","last_became_peered":"2023-08-25T09:41:13.543707+0000","last_unstale":"2023-08-25T09:41:13.544562+0000","last_undegraded":"2023-08-25T09:41:13.544562+0000","last_fullsized":"2023-08-25T09:41:13.544562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:29.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553918+0000","last_change":"2023-08-25T09:41:13.553706+0000","last_active":"2023-08-25T09:41:13.553918+0000","last_peered":"2023-08-25T09:41:13.553918+0000","last_clean":"2023-08-25T09:41:13.553918+0000","last_became_active":"2023-08-25T09:41:13.552154+0000","last_became_peered":"2023-08-25T09:41:13.552154+0000","last_unstale":"2023-08-25T09:41:13.553918+0000","last_undegraded":"2023-08-25T09:41:13.553918+0000","last_fullsized":"2023-08-25T09:41:13.553918+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:03.273034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556524+0000","last_change":"2023-08-25T09:41:13.555731+0000","last_active":"2023-08-25T09:41:13.556524+0000","last_peered":"2023-08-25T09:41:13.556524+0000","last_clean":"2023-08-25T09:41:13.556524+0000","last_became_active":"2023-08-25T09:41:13.554808+0000","last_became_peered":"2023-08-25T09:41:13.554808+0000","last_unstale":"2023-08-25T09:41:13.556524+0000","last_undegraded":"2023-08-25T09:41:13.556524+0000","last_fullsized":"2023-08-25T09:41:13.556524+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:32.208562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553867+0000","last_change":"2023-08-25T09:41:13.553486+0000","last_active":"2023-08-25T09:41:13.553867+0000","last_peered":"2023-08-25T09:41:13.553867+0000","last_clean":"2023-08-25T09:41:13.553867+0000","last_became_active":"2023-08-25T09:41:13.552843+0000","last_became_peered":"2023-08-25T09:41:13.552843+0000","last_unstale":"2023-08-25T09:41:13.553867+0000","last_undegraded":"2023-08-25T09:41:13.553867+0000","last_fullsized":"2023-08-25T09:41:13.553867+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:05:14.270183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559084+0000","last_change":"2023-08-25T09:41:13.559010+0000","last_active":"2023-08-25T09:41:13.559084+0000","last_peered":"2023-08-25T09:41:13.559084+0000","last_clean":"2023-08-25T09:41:13.559084+0000","last_became_active":"2023-08-25T09:41:13.558720+0000","last_became_peered":"2023-08-25T09:41:13.558720+0000","last_unstale":"2023-08-25T09:41:13.559084+0000","last_undegraded":"2023-08-25T09:41:13.559084+0000","last_fullsized":"2023-08-25T09:41:13.559084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:59:38.698765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549531+0000","last_change":"2023-08-25T09:41:13.547485+0000","last_active":"2023-08-25T09:41:13.549531+0000","last_peered":"2023-08-25T09:41:13.549531+0000","last_clean":"2023-08-25T09:41:13.549531+0000","last_became_active":"2023-08-25T09:41:13.546997+0000","last_became_peered":"2023-08-25T09:41:13.546997+0000","last_unstale":"2023-08-25T09:41:13.549531+0000","last_undegraded":"2023-08-25T09:41:13.549531+0000","last_fullsized":"2023-08-25T09:41:13.549531+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:01.247517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559092+0000","last_change":"2023-08-25T09:41:13.558973+0000","last_active":"2023-08-25T09:41:13.559092+0000","last_peered":"2023-08-25T09:41:13.559092+0000","last_clean":"2023-08-25T09:41:13.559092+0000","last_became_active":"2023-08-25T09:41:13.558717+0000","last_became_peered":"2023-08-25T09:41:13.558717+0000","last_unstale":"2023-08-25T09:41:13.559092+0000","last_undegraded":"2023-08-25T09:41:13.559092+0000","last_fullsized":"2023-08-25T09:41:13.559092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:15.199506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.551228+0000","last_change":"2023-08-25T09:41:13.550468+0000","last_active":"2023-08-25T09:41:13.551228+0000","last_peered":"2023-08-25T09:41:13.551228+0000","last_clean":"2023-08-25T09:41:13.551228+0000","last_became_active":"2023-08-25T09:41:13.550079+0000","last_became_peered":"2023-08-25T09:41:13.550079+0000","last_unstale":"2023-08-25T09:41:13.551228+0000","last_undegraded":"2023-08-25T09:41:13.551228+0000","last_fullsized":"2023-08-25T09:41:13.551228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:22.630319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553902+0000","last_change":"2023-08-25T09:41:13.553656+0000","last_active":"2023-08-25T09:41:13.553902+0000","last_peered":"2023-08-25T09:41:13.553902+0000","last_clean":"2023-08-25T09:41:13.553902+0000","last_became_active":"2023-08-25T09:41:13.553222+0000","last_became_peered":"2023-08-25T09:41:13.553222+0000","last_unstale":"2023-08-25T09:41:13.553902+0000","last_undegraded":"2023-08-25T09:41:13.553902+0000","last_fullsized":"2023-08-25T09:41:13.553902+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:38.489653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554207+0000","last_change":"2023-08-25T09:41:13.553299+0000","last_active":"2023-08-25T09:41:13.554207+0000","last_peered":"2023-08-25T09:41:13.554207+0000","last_clean":"2023-08-25T09:41:13.554207+0000","last_became_active":"2023-08-25T09:41:13.552885+0000","last_became_peered":"2023-08-25T09:41:13.552885+0000","last_unstale":"2023-08-25T09:41:13.554207+0000","last_undegraded":"2023-08-25T09:41:13.554207+0000","last_fullsized":"2023-08-25T09:41:13.554207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:08:09.081236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557591+0000","last_change":"2023-08-25T09:41:13.557462+0000","last_active":"2023-08-25T09:41:13.557591+0000","last_peered":"2023-08-25T09:41:13.557591+0000","last_clean":"2023-08-25T09:41:13.557591+0000","last_became_active":"2023-08-25T09:41:13.557175+0000","last_became_peered":"2023-08-25T09:41:13.557175+0000","last_unstale":"2023-08-25T09:41:13.557591+0000","last_undegraded":"2023-08-25T09:41:13.557591+0000","last_fullsized":"2023-08-25T09:41:13.557591+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:00.337761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574528+0000","last_change":"2023-08-25T09:41:12.513117+0000","last_active":"2023-08-25T09:41:13.574528+0000","last_peered":"2023-08-25T09:41:13.574528+0000","last_clean":"2023-08-25T09:41:13.574528+0000","last_became_active":"2023-08-25T09:41:12.512850+0000","last_became_peered":"2023-08-25T09:41:12.512850+0000","last_unstale":"2023-08-25T09:41:13.574528+0000","last_undegraded":"2023-08-25T09:41:13.574528+0000","last_fullsized":"2023-08-25T09:41:13.574528+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:50.952127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549482+0000","last_change":"2023-08-25T09:41:13.548251+0000","last_active":"2023-08-25T09:41:13.549482+0000","last_peered":"2023-08-25T09:41:13.549482+0000","last_clean":"2023-08-25T09:41:13.549482+0000","last_became_active":"2023-08-25T09:41:13.547754+0000","last_became_peered":"2023-08-25T09:41:13.547754+0000","last_unstale":"2023-08-25T09:41:13.549482+0000","last_undegraded":"2023-08-25T09:41:13.549482+0000","last_fullsized":"2023-08-25T09:41:13.549482+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:53:29.655373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574094+0000","last_change":"2023-08-25T09:41:12.517976+0000","last_active":"2023-08-25T09:41:13.574094+0000","last_peered":"2023-08-25T09:41:13.574094+0000","last_clean":"2023-08-25T09:41:13.574094+0000","last_became_active":"2023-08-25T09:41:12.517578+0000","last_became_peered":"2023-08-25T09:41:12.517578+0000","last_unstale":"2023-08-25T09:41:13.574094+0000","last_undegraded":"2023-08-25T09:41:13.574094+0000","last_fullsized":"2023-08-25T09:41:13.574094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:18:34.026179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553056+0000","last_change":"2023-08-25T09:41:13.552455+0000","last_active":"2023-08-25T09:41:13.553056+0000","last_peered":"2023-08-25T09:41:13.553056+0000","last_clean":"2023-08-25T09:41:13.553056+0000","last_became_active":"2023-08-25T09:41:13.551928+0000","last_became_peered":"2023-08-25T09:41:13.551928+0000","last_unstale":"2023-08-25T09:41:13.553056+0000","last_undegraded":"2023-08-25T09:41:13.553056+0000","last_fullsized":"2023-08-25T09:41:13.553056+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:54.383488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574472+0000","last_change":"2023-08-25T09:41:12.511383+0000","last_active":"2023-08-25T09:41:13.574472+0000","last_peered":"2023-08-25T09:41:13.574472+0000","last_clean":"2023-08-25T09:41:13.574472+0000","last_became_active":"2023-08-25T09:41:12.509068+0000","last_became_peered":"2023-08-25T09:41:12.509068+0000","last_unstale":"2023-08-25T09:41:13.574472+0000","last_undegraded":"2023-08-25T09:41:13.574472+0000","last_fullsized":"2023-08-25T09:41:13.574472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:40.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555409+0000","last_change":"2023-08-25T09:41:13.555330+0000","last_active":"2023-08-25T09:41:13.555409+0000","last_peered":"2023-08-25T09:41:13.555409+0000","last_clean":"2023-08-25T09:41:13.555409+0000","last_became_active":"2023-08-25T09:41:13.555057+0000","last_became_peered":"2023-08-25T09:41:13.555057+0000","last_unstale":"2023-08-25T09:41:13.555409+0000","last_undegraded":"2023-08-25T09:41:13.555409+0000","last_fullsized":"2023-08-25T09:41:13.555409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:06.469710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574001+0000","last_change":"2023-08-25T09:41:12.500623+0000","last_active":"2023-08-25T09:41:13.574001+0000","last_peered":"2023-08-25T09:41:13.574001+0000","last_clean":"2023-08-25T09:41:13.574001+0000","last_became_active":"2023-08-25T09:41:12.499955+0000","last_became_peered":"2023-08-25T09:41:12.499955+0000","last_unstale":"2023-08-25T09:41:13.574001+0000","last_undegraded":"2023-08-25T09:41:13.574001+0000","last_fullsized":"2023-08-25T09:41:13.574001+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:16.593221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554214+0000","last_change":"2023-08-25T09:41:13.553384+0000","last_active":"2023-08-25T09:41:13.554214+0000","last_peered":"2023-08-25T09:41:13.554214+0000","last_clean":"2023-08-25T09:41:13.554214+0000","last_became_active":"2023-08-25T09:41:13.551446+0000","last_became_peered":"2023-08-25T09:41:13.551446+0000","last_unstale":"2023-08-25T09:41:13.554214+0000","last_undegraded":"2023-08-25T09:41:13.554214+0000","last_fullsized":"2023-08-25T09:41:13.554214+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:32.208562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574761+0000","last_change":"2023-08-25T09:41:12.506088+0000","last_active":"2023-08-25T09:41:13.574761+0000","last_peered":"2023-08-25T09:41:13.574761+0000","last_clean":"2023-08-25T09:41:13.574761+0000","last_became_active":"2023-08-25T09:41:12.505415+0000","last_became_peered":"2023-08-25T09:41:12.505415+0000","last_unstale":"2023-08-25T09:41:13.574761+0000","last_undegraded":"2023-08-25T09:41:13.574761+0000","last_fullsized":"2023-08-25T09:41:13.574761+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:22:05.106316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535718+0000","last_change":"2023-08-25T09:41:13.534770+0000","last_active":"2023-08-25T09:41:13.535718+0000","last_peered":"2023-08-25T09:41:13.535718+0000","last_clean":"2023-08-25T09:41:13.535718+0000","last_became_active":"2023-08-25T09:41:13.533829+0000","last_became_peered":"2023-08-25T09:41:13.533829+0000","last_unstale":"2023-08-25T09:41:13.535718+0000","last_undegraded":"2023-08-25T09:41:13.535718+0000","last_fullsized":"2023-08-25T09:41:13.535718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:13:33.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575161+0000","last_change":"2023-08-25T09:41:12.515423+0000","last_active":"2023-08-25T09:41:13.575161+0000","last_peered":"2023-08-25T09:41:13.575161+0000","last_clean":"2023-08-25T09:41:13.575161+0000","last_became_active":"2023-08-25T09:41:12.514991+0000","last_became_peered":"2023-08-25T09:41:12.514991+0000","last_unstale":"2023-08-25T09:41:13.575161+0000","last_undegraded":"2023-08-25T09:41:13.575161+0000","last_fullsized":"2023-08-25T09:41:13.575161+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:05.098179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554737+0000","last_change":"2023-08-25T09:41:13.554679+0000","last_active":"2023-08-25T09:41:13.554737+0000","last_peered":"2023-08-25T09:41:13.554737+0000","last_clean":"2023-08-25T09:41:13.554737+0000","last_became_active":"2023-08-25T09:41:13.554274+0000","last_became_peered":"2023-08-25T09:41:13.554274+0000","last_unstale":"2023-08-25T09:41:13.554737+0000","last_undegraded":"2023-08-25T09:41:13.554737+0000","last_fullsized":"2023-08-25T09:41:13.554737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:18:27.360452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.524605+0000","last_change":"2023-08-25T09:41:12.512877+0000","last_active":"2023-08-25T09:41:13.524605+0000","last_peered":"2023-08-25T09:41:13.524605+0000","last_clean":"2023-08-25T09:41:13.524605+0000","last_became_active":"2023-08-25T09:41:12.512569+0000","last_became_peered":"2023-08-25T09:41:12.512569+0000","last_unstale":"2023-08-25T09:41:13.524605+0000","last_undegraded":"2023-08-25T09:41:13.524605+0000","last_fullsized":"2023-08-25T09:41:13.524605+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:19:52.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.558277+0000","last_change":"2023-08-25T09:41:13.558115+0000","last_active":"2023-08-25T09:41:13.558277+0000","last_peered":"2023-08-25T09:41:13.558277+0000","last_clean":"2023-08-25T09:41:13.558277+0000","last_became_active":"2023-08-25T09:41:13.557800+0000","last_became_peered":"2023-08-25T09:41:13.557800+0000","last_unstale":"2023-08-25T09:41:13.558277+0000","last_undegraded":"2023-08-25T09:41:13.558277+0000","last_fullsized":"2023-08-25T09:41:13.558277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:14:55.454430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.538892+0000","last_change":"2023-08-25T09:41:12.510611+0000","last_active":"2023-08-25T09:41:13.538892+0000","last_peered":"2023-08-25T09:41:13.538892+0000","last_clean":"2023-08-25T09:41:13.538892+0000","last_became_active":"2023-08-25T09:41:12.509916+0000","last_became_peered":"2023-08-25T09:41:12.509916+0000","last_unstale":"2023-08-25T09:41:13.538892+0000","last_undegraded":"2023-08-25T09:41:13.538892+0000","last_fullsized":"2023-08-25T09:41:13.538892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:09:42.385237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556148+0000","last_change":"2023-08-25T09:41:13.556066+0000","last_active":"2023-08-25T09:41:13.556148+0000","last_peered":"2023-08-25T09:41:13.556148+0000","last_clean":"2023-08-25T09:41:13.556148+0000","last_became_active":"2023-08-25T09:41:13.555789+0000","last_became_peered":"2023-08-25T09:41:13.555789+0000","last_unstale":"2023-08-25T09:41:13.556148+0000","last_undegraded":"2023-08-25T09:41:13.556148+0000","last_fullsized":"2023-08-25T09:41:13.556148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:48.330370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545605+0000","last_change":"2023-08-25T09:41:12.515713+0000","last_active":"2023-08-25T09:41:13.545605+0000","last_peered":"2023-08-25T09:41:13.545605+0000","last_clean":"2023-08-25T09:41:13.545605+0000","last_became_active":"2023-08-25T09:41:12.514648+0000","last_became_peered":"2023-08-25T09:41:12.514648+0000","last_unstale":"2023-08-25T09:41:13.545605+0000","last_undegraded":"2023-08-25T09:41:13.545605+0000","last_fullsized":"2023-08-25T09:41:13.545605+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:20:25.833638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557181+0000","last_change":"2023-08-25T09:41:13.556973+0000","last_active":"2023-08-25T09:41:13.557181+0000","last_peered":"2023-08-25T09:41:13.557181+0000","last_clean":"2023-08-25T09:41:13.557181+0000","last_became_active":"2023-08-25T09:41:13.556096+0000","last_became_peered":"2023-08-25T09:41:13.556096+0000","last_unstale":"2023-08-25T09:41:13.557181+0000","last_undegraded":"2023-08-25T09:41:13.557181+0000","last_fullsized":"2023-08-25T09:41:13.557181+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:44.421307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.542963+0000","last_change":"2023-08-25T09:41:12.510440+0000","last_active":"2023-08-25T09:41:13.542963+0000","last_peered":"2023-08-25T09:41:13.542963+0000","last_clean":"2023-08-25T09:41:13.542963+0000","last_became_active":"2023-08-25T09:41:12.510124+0000","last_became_peered":"2023-08-25T09:41:12.510124+0000","last_unstale":"2023-08-25T09:41:13.542963+0000","last_undegraded":"2023-08-25T09:41:13.542963+0000","last_fullsized":"2023-08-25T09:41:13.542963+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:17.191428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.560053+0000","last_change":"2023-08-25T09:41:13.559867+0000","last_active":"2023-08-25T09:41:13.560053+0000","last_peered":"2023-08-25T09:41:13.560053+0000","last_clean":"2023-08-25T09:41:13.560053+0000","last_became_active":"2023-08-25T09:41:13.559094+0000","last_became_peered":"2023-08-25T09:41:13.559094+0000","last_unstale":"2023-08-25T09:41:13.560053+0000","last_undegraded":"2023-08-25T09:41:13.560053+0000","last_fullsized":"2023-08-25T09:41:13.560053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:49.136987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575613+0000","last_change":"2023-08-25T09:41:12.515258+0000","last_active":"2023-08-25T09:41:13.575613+0000","last_peered":"2023-08-25T09:41:13.575613+0000","last_clean":"2023-08-25T09:41:13.575613+0000","last_became_active":"2023-08-25T09:41:12.514451+0000","last_became_peered":"2023-08-25T09:41:12.514451+0000","last_unstale":"2023-08-25T09:41:13.575613+0000","last_undegraded":"2023-08-25T09:41:13.575613+0000","last_fullsized":"2023-08-25T09:41:13.575613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:07:23.660161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549557+0000","last_change":"2023-08-25T09:41:13.549345+0000","last_active":"2023-08-25T09:41:13.549557+0000","last_peered":"2023-08-25T09:41:13.549557+0000","last_clean":"2023-08-25T09:41:13.549557+0000","last_became_active":"2023-08-25T09:41:13.548945+0000","last_became_peered":"2023-08-25T09:41:13.548945+0000","last_unstale":"2023-08-25T09:41:13.549557+0000","last_undegraded":"2023-08-25T09:41:13.549557+0000","last_fullsized":"2023-08-25T09:41:13.549557+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:12:13.225820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.577176+0000","last_change":"2023-08-25T09:41:12.519449+0000","last_active":"2023-08-25T09:41:13.577176+0000","last_peered":"2023-08-25T09:41:13.577176+0000","last_clean":"2023-08-25T09:41:13.577176+0000","last_became_active":"2023-08-25T09:41:12.518508+0000","last_became_peered":"2023-08-25T09:41:12.518508+0000","last_unstale":"2023-08-25T09:41:13.577176+0000","last_undegraded":"2023-08-25T09:41:13.577176+0000","last_fullsized":"2023-08-25T09:41:13.577176+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:08:52.365857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535637+0000","last_change":"2023-08-25T09:41:13.534458+0000","last_active":"2023-08-25T09:41:13.535637+0000","last_peered":"2023-08-25T09:41:13.535637+0000","last_clean":"2023-08-25T09:41:13.535637+0000","last_became_active":"2023-08-25T09:41:13.533826+0000","last_became_peered":"2023-08-25T09:41:13.533826+0000","last_unstale":"2023-08-25T09:41:13.535637+0000","last_undegraded":"2023-08-25T09:41:13.535637+0000","last_fullsized":"2023-08-25T09:41:13.535637+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:12.173559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575077+0000","last_change":"2023-08-25T09:41:12.511311+0000","last_active":"2023-08-25T09:41:13.575077+0000","last_peered":"2023-08-25T09:41:13.575077+0000","last_clean":"2023-08-25T09:41:13.575077+0000","last_became_active":"2023-08-25T09:41:12.510674+0000","last_became_peered":"2023-08-25T09:41:12.510674+0000","last_unstale":"2023-08-25T09:41:13.575077+0000","last_undegraded":"2023-08-25T09:41:13.575077+0000","last_fullsized":"2023-08-25T09:41:13.575077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:55.908363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557040+0000","last_change":"2023-08-25T09:41:13.556659+0000","last_active":"2023-08-25T09:41:13.557040+0000","last_peered":"2023-08-25T09:41:13.557040+0000","last_clean":"2023-08-25T09:41:13.557040+0000","last_became_active":"2023-08-25T09:41:13.555482+0000","last_became_peered":"2023-08-25T09:41:13.555482+0000","last_unstale":"2023-08-25T09:41:13.557040+0000","last_undegraded":"2023-08-25T09:41:13.557040+0000","last_fullsized":"2023-08-25T09:41:13.557040+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:06.469710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545920+0000","last_change":"2023-08-25T09:41:12.515872+0000","last_active":"2023-08-25T09:41:13.545920+0000","last_peered":"2023-08-25T09:41:13.545920+0000","last_clean":"2023-08-25T09:41:13.545920+0000","last_became_active":"2023-08-25T09:41:12.515111+0000","last_became_peered":"2023-08-25T09:41:12.515111+0000","last_unstale":"2023-08-25T09:41:13.545920+0000","last_undegraded":"2023-08-25T09:41:13.545920+0000","last_fullsized":"2023-08-25T09:41:13.545920+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:47:14.103836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553871+0000","last_change":"2023-08-25T09:41:13.550907+0000","last_active":"2023-08-25T09:41:13.553871+0000","last_peered":"2023-08-25T09:41:13.553871+0000","last_clean":"2023-08-25T09:41:13.553871+0000","last_became_active":"2023-08-25T09:41:13.550543+0000","last_became_peered":"2023-08-25T09:41:13.550543+0000","last_unstale":"2023-08-25T09:41:13.553871+0000","last_undegraded":"2023-08-25T09:41:13.553871+0000","last_fullsized":"2023-08-25T09:41:13.553871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:49.136987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.532366+0000","last_change":"2023-08-25T09:41:12.503171+0000","last_active":"2023-08-25T09:41:13.532366+0000","last_peered":"2023-08-25T09:41:13.532366+0000","last_clean":"2023-08-25T09:41:13.532366+0000","last_became_active":"2023-08-25T09:41:12.501972+0000","last_became_peered":"2023-08-25T09:41:12.501972+0000","last_unstale":"2023-08-25T09:41:13.532366+0000","last_undegraded":"2023-08-25T09:41:13.532366+0000","last_fullsized":"2023-08-25T09:41:13.532366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:39:25.984514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553836+0000","last_change":"2023-08-25T09:41:13.553478+0000","last_active":"2023-08-25T09:41:13.553836+0000","last_peered":"2023-08-25T09:41:13.553836+0000","last_clean":"2023-08-25T09:41:13.553836+0000","last_became_active":"2023-08-25T09:41:13.552404+0000","last_became_peered":"2023-08-25T09:41:13.552404+0000","last_unstale":"2023-08-25T09:41:13.553836+0000","last_undegraded":"2023-08-25T09:41:13.553836+0000","last_fullsized":"2023-08-25T09:41:13.553836+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:27.131711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.576766+0000","last_change":"2023-08-25T09:41:12.515577+0000","last_active":"2023-08-25T09:41:13.576766+0000","last_peered":"2023-08-25T09:41:13.576766+0000","last_clean":"2023-08-25T09:41:13.576766+0000","last_became_active":"2023-08-25T09:41:12.515311+0000","last_became_peered":"2023-08-25T09:41:12.515311+0000","last_unstale":"2023-08-25T09:41:13.576766+0000","last_undegraded":"2023-08-25T09:41:13.576766+0000","last_fullsized":"2023-08-25T09:41:13.576766+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:40:00.815532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.546668+0000","last_change":"2023-08-25T09:41:13.545931+0000","last_active":"2023-08-25T09:41:13.546668+0000","last_peered":"2023-08-25T09:41:13.546668+0000","last_clean":"2023-08-25T09:41:13.546668+0000","last_became_active":"2023-08-25T09:41:13.545526+0000","last_became_peered":"2023-08-25T09:41:13.545526+0000","last_unstale":"2023-08-25T09:41:13.546668+0000","last_undegraded":"2023-08-25T09:41:13.546668+0000","last_fullsized":"2023-08-25T09:41:13.546668+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:45.077348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574045+0000","last_change":"2023-08-25T09:41:12.519312+0000","last_active":"2023-08-25T09:41:13.574045+0000","last_peered":"2023-08-25T09:41:13.574045+0000","last_clean":"2023-08-25T09:41:13.574045+0000","last_became_active":"2023-08-25T09:41:12.519017+0000","last_became_peered":"2023-08-25T09:41:12.519017+0000","last_unstale":"2023-08-25T09:41:13.574045+0000","last_undegraded":"2023-08-25T09:41:13.574045+0000","last_fullsized":"2023-08-25T09:41:13.574045+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:59:26.539321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559752+0000","last_change":"2023-08-25T09:41:13.559584+0000","last_active":"2023-08-25T09:41:13.559752+0000","last_peered":"2023-08-25T09:41:13.559752+0000","last_clean":"2023-08-25T09:41:13.559752+0000","last_became_active":"2023-08-25T09:41:13.557845+0000","last_became_peered":"2023-08-25T09:41:13.557845+0000","last_unstale":"2023-08-25T09:41:13.559752+0000","last_undegraded":"2023-08-25T09:41:13.559752+0000","last_fullsized":"2023-08-25T09:41:13.559752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:51.965474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574205+0000","last_change":"2023-08-25T09:41:12.511088+0000","last_active":"2023-08-25T09:41:13.574205+0000","last_peered":"2023-08-25T09:41:13.574205+0000","last_clean":"2023-08-25T09:41:13.574205+0000","last_became_active":"2023-08-25T09:41:12.510155+0000","last_became_peered":"2023-08-25T09:41:12.510155+0000","last_unstale":"2023-08-25T09:41:13.574205+0000","last_undegraded":"2023-08-25T09:41:13.574205+0000","last_fullsized":"2023-08-25T09:41:13.574205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:40.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535027+0000","last_change":"2023-08-25T09:41:13.534418+0000","last_active":"2023-08-25T09:41:13.535027+0000","last_peered":"2023-08-25T09:41:13.535027+0000","last_clean":"2023-08-25T09:41:13.535027+0000","last_became_active":"2023-08-25T09:41:13.533797+0000","last_became_peered":"2023-08-25T09:41:13.533797+0000","last_unstale":"2023-08-25T09:41:13.535027+0000","last_undegraded":"2023-08-25T09:41:13.535027+0000","last_fullsized":"2023-08-25T09:41:13.535027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:08:32.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.573538+0000","last_change":"2023-08-25T09:41:12.516988+0000","last_active":"2023-08-25T09:41:13.573538+0000","last_peered":"2023-08-25T09:41:13.573538+0000","last_clean":"2023-08-25T09:41:13.573538+0000","last_became_active":"2023-08-25T09:41:12.516819+0000","last_became_peered":"2023-08-25T09:41:12.516819+0000","last_unstale":"2023-08-25T09:41:13.573538+0000","last_undegraded":"2023-08-25T09:41:13.573538+0000","last_fullsized":"2023-08-25T09:41:13.573538+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:45.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.558911+0000","last_change":"2023-08-25T09:41:13.558785+0000","last_active":"2023-08-25T09:41:13.558911+0000","last_peered":"2023-08-25T09:41:13.558911+0000","last_clean":"2023-08-25T09:41:13.558911+0000","last_became_active":"2023-08-25T09:41:13.556586+0000","last_became_peered":"2023-08-25T09:41:13.556586+0000","last_unstale":"2023-08-25T09:41:13.558911+0000","last_undegraded":"2023-08-25T09:41:13.558911+0000","last_fullsized":"2023-08-25T09:41:13.558911+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:18:19.023697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.532731+0000","last_change":"2023-08-25T09:41:12.504818+0000","last_active":"2023-08-25T09:41:13.532731+0000","last_peered":"2023-08-25T09:41:13.532731+0000","last_clean":"2023-08-25T09:41:13.532731+0000","last_became_active":"2023-08-25T09:41:12.503878+0000","last_became_peered":"2023-08-25T09:41:12.503878+0000","last_unstale":"2023-08-25T09:41:13.532731+0000","last_undegraded":"2023-08-25T09:41:13.532731+0000","last_fullsized":"2023-08-25T09:41:13.532731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:34.598980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.539618+0000","last_change":"2023-08-25T09:41:13.539194+0000","last_active":"2023-08-25T09:41:13.539618+0000","last_peered":"2023-08-25T09:41:13.539618+0000","last_clean":"2023-08-25T09:41:13.539618+0000","last_became_active":"2023-08-25T09:41:13.536791+0000","last_became_peered":"2023-08-25T09:41:13.536791+0000","last_unstale":"2023-08-25T09:41:13.539618+0000","last_undegraded":"2023-08-25T09:41:13.539618+0000","last_fullsized":"2023-08-25T09:41:13.539618+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:54:05.572860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575634+0000","last_change":"2023-08-25T09:41:12.512519+0000","last_active":"2023-08-25T09:41:13.575634+0000","last_peered":"2023-08-25T09:41:13.575634+0000","last_clean":"2023-08-25T09:41:13.575634+0000","last_became_active":"2023-08-25T09:41:12.511084+0000","last_became_peered":"2023-08-25T09:41:12.511084+0000","last_unstale":"2023-08-25T09:41:13.575634+0000","last_undegraded":"2023-08-25T09:41:13.575634+0000","last_fullsized":"2023-08-25T09:41:13.575634+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:44.042754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.548427+0000","last_change":"2023-08-25T09:41:13.547760+0000","last_active":"2023-08-25T09:41:13.548427+0000","last_peered":"2023-08-25T09:41:13.548427+0000","last_clean":"2023-08-25T09:41:13.548427+0000","last_became_active":"2023-08-25T09:41:13.547174+0000","last_became_peered":"2023-08-25T09:41:13.547174+0000","last_unstale":"2023-08-25T09:41:13.548427+0000","last_undegraded":"2023-08-25T09:41:13.548427+0000","last_fullsized":"2023-08-25T09:41:13.548427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:22:33.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.540717+0000","last_change":"2023-08-25T09:41:12.518243+0000","last_active":"2023-08-25T09:41:13.540717+0000","last_peered":"2023-08-25T09:41:13.540717+0000","last_clean":"2023-08-25T09:41:13.540717+0000","last_became_active":"2023-08-25T09:41:12.517554+0000","last_became_peered":"2023-08-25T09:41:12.517554+0000","last_unstale":"2023-08-25T09:41:13.540717+0000","last_undegraded":"2023-08-25T09:41:13.540717+0000","last_fullsized":"2023-08-25T09:41:13.540717+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:33.632875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559729+0000","last_change":"2023-08-25T09:41:13.559392+0000","last_active":"2023-08-25T09:41:13.559729+0000","last_peered":"2023-08-25T09:41:13.559729+0000","last_clean":"2023-08-25T09:41:13.559729+0000","last_became_active":"2023-08-25T09:41:13.557417+0000","last_became_peered":"2023-08-25T09:41:13.557417+0000","last_unstale":"2023-08-25T09:41:13.559729+0000","last_undegraded":"2023-08-25T09:41:13.559729+0000","last_fullsized":"2023-08-25T09:41:13.559729+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:29:03.964811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.543369+0000","last_change":"2023-08-25T09:41:12.518319+0000","last_active":"2023-08-25T09:41:13.543369+0000","last_peered":"2023-08-25T09:41:13.543369+0000","last_clean":"2023-08-25T09:41:13.543369+0000","last_became_active":"2023-08-25T09:41:12.517592+0000","last_became_peered":"2023-08-25T09:41:12.517592+0000","last_unstale":"2023-08-25T09:41:13.543369+0000","last_undegraded":"2023-08-25T09:41:13.543369+0000","last_fullsized":"2023-08-25T09:41:13.543369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:14.186160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.546197+0000","last_change":"2023-08-25T09:41:13.546067+0000","last_active":"2023-08-25T09:41:13.546197+0000","last_peered":"2023-08-25T09:41:13.546197+0000","last_clean":"2023-08-25T09:41:13.546197+0000","last_became_active":"2023-08-25T09:41:13.545588+0000","last_became_peered":"2023-08-25T09:41:13.545588+0000","last_unstale":"2023-08-25T09:41:13.546197+0000","last_undegraded":"2023-08-25T09:41:13.546197+0000","last_fullsized":"2023-08-25T09:41:13.546197+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:44:25.714216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.537526+0000","last_change":"2023-08-25T09:41:12.516676+0000","last_active":"2023-08-25T09:41:13.537526+0000","last_peered":"2023-08-25T09:41:13.537526+0000","last_clean":"2023-08-25T09:41:13.537526+0000","last_became_active":"2023-08-25T09:41:12.516338+0000","last_became_peered":"2023-08-25T09:41:12.516338+0000","last_unstale":"2023-08-25T09:41:13.537526+0000","last_undegraded":"2023-08-25T09:41:13.537526+0000","last_fullsized":"2023-08-25T09:41:13.537526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:42.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.558441+0000","last_change":"2023-08-25T09:41:13.558356+0000","last_active":"2023-08-25T09:41:13.558441+0000","last_peered":"2023-08-25T09:41:13.558441+0000","last_clean":"2023-08-25T09:41:13.558441+0000","last_became_active":"2023-08-25T09:41:13.558057+0000","last_became_peered":"2023-08-25T09:41:13.558057+0000","last_unstale":"2023-08-25T09:41:13.558441+0000","last_undegraded":"2023-08-25T09:41:13.558441+0000","last_fullsized":"2023-08-25T09:41:13.558441+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:15.199506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.543751+0000","last_change":"2023-08-25T09:41:12.512020+0000","last_active":"2023-08-25T09:41:13.543751+0000","last_peered":"2023-08-25T09:41:13.543751+0000","last_clean":"2023-08-25T09:41:13.543751+0000","last_became_active":"2023-08-25T09:41:12.511549+0000","last_became_peered":"2023-08-25T09:41:12.511549+0000","last_unstale":"2023-08-25T09:41:13.543751+0000","last_undegraded":"2023-08-25T09:41:13.543751+0000","last_fullsized":"2023-08-25T09:41:13.543751+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:47.317023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545693+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:41:13.545693+0000","last_peered":"2023-08-25T09:41:13.545693+0000","last_clean":"2023-08-25T09:41:13.545693+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:41:13.545693+0000","last_undegraded":"2023-08-25T09:41:13.545693+0000","last_fullsized":"2023-08-25T09:41:13.545693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:38:01.222062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.526726+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:41:13.526726+0000","last_peered":"2023-08-25T09:41:13.526726+0000","last_clean":"2023-08-25T09:41:13.526726+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:41:13.526726+0000","last_undegraded":"2023-08-25T09:41:13.526726+0000","last_fullsized":"2023-08-25T09:41:13.526726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:11.566878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.551555+0000","last_change":"2023-08-25T09:41:13.549732+0000","last_active":"2023-08-25T09:41:13.551555+0000","last_peered":"2023-08-25T09:41:13.551555+0000","last_clean":"2023-08-25T09:41:13.551555+0000","last_became_active":"2023-08-25T09:41:13.549414+0000","last_became_peered":"2023-08-25T09:41:13.549414+0000","last_unstale":"2023-08-25T09:41:13.551555+0000","last_undegraded":"2023-08-25T09:41:13.551555+0000","last_fullsized":"2023-08-25T09:41:13.551555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:30:24.865893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.525283+0000","last_change":"2023-08-25T09:41:12.514623+0000","last_active":"2023-08-25T09:41:13.525283+0000","last_peered":"2023-08-25T09:41:13.525283+0000","last_clean":"2023-08-25T09:41:13.525283+0000","last_became_active":"2023-08-25T09:41:12.514316+0000","last_became_peered":"2023-08-25T09:41:12.514316+0000","last_unstale":"2023-08-25T09:41:13.525283+0000","last_undegraded":"2023-08-25T09:41:13.525283+0000","last_fullsized":"2023-08-25T09:41:13.525283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:10:57.965384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.525476+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:41:13.525476+0000","last_peered":"2023-08-25T09:41:13.525476+0000","last_clean":"2023-08-25T09:41:13.525476+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:41:13.525476+0000","last_undegraded":"2023-08-25T09:41:13.525476+0000","last_fullsized":"2023-08-25T09:41:13.525476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:08:39.671664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553886+0000","last_change":"2023-08-25T09:41:13.553625+0000","last_active":"2023-08-25T09:41:13.553886+0000","last_peered":"2023-08-25T09:41:13.553886+0000","last_clean":"2023-08-25T09:41:13.553886+0000","last_became_active":"2023-08-25T09:41:13.553235+0000","last_became_peered":"2023-08-25T09:41:13.553235+0000","last_unstale":"2023-08-25T09:41:13.553886+0000","last_undegraded":"2023-08-25T09:41:13.553886+0000","last_fullsized":"2023-08-25T09:41:13.553886+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:50:21.885432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.539266+0000","last_change":"2023-08-25T09:41:12.510402+0000","last_active":"2023-08-25T09:41:13.539266+0000","last_peered":"2023-08-25T09:41:13.539266+0000","last_clean":"2023-08-25T09:41:13.539266+0000","last_became_active":"2023-08-25T09:41:12.509222+0000","last_became_peered":"2023-08-25T09:41:12.509222+0000","last_unstale":"2023-08-25T09:41:13.539266+0000","last_undegraded":"2023-08-25T09:41:13.539266+0000","last_fullsized":"2023-08-25T09:41:13.539266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:47.279756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.528974+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:41:13.528974+0000","last_peered":"2023-08-25T09:41:13.528974+0000","last_clean":"2023-08-25T09:41:13.528974+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:41:13.528974+0000","last_undegraded":"2023-08-25T09:41:13.528974+0000","last_fullsized":"2023-08-25T09:41:13.528974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:05.710926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555453+0000","last_change":"2023-08-25T09:41:13.555310+0000","last_active":"2023-08-25T09:41:13.555453+0000","last_peered":"2023-08-25T09:41:13.555453+0000","last_clean":"2023-08-25T09:41:13.555453+0000","last_became_active":"2023-08-25T09:41:13.554281+0000","last_became_peered":"2023-08-25T09:41:13.554281+0000","last_unstale":"2023-08-25T09:41:13.555453+0000","last_undegraded":"2023-08-25T09:41:13.555453+0000","last_fullsized":"2023-08-25T09:41:13.555453+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:40:49.031922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575381+0000","last_change":"2023-08-25T09:41:12.513317+0000","last_active":"2023-08-25T09:41:13.575381+0000","last_peered":"2023-08-25T09:41:13.575381+0000","last_clean":"2023-08-25T09:41:13.575381+0000","last_became_active":"2023-08-25T09:41:12.513057+0000","last_became_peered":"2023-08-25T09:41:12.513057+0000","last_unstale":"2023-08-25T09:41:13.575381+0000","last_undegraded":"2023-08-25T09:41:13.575381+0000","last_fullsized":"2023-08-25T09:41:13.575381+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:48:06.788767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.526306+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:41:13.526306+0000","last_peered":"2023-08-25T09:41:13.526306+0000","last_clean":"2023-08-25T09:41:13.526306+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:41:13.526306+0000","last_undegraded":"2023-08-25T09:41:13.526306+0000","last_fullsized":"2023-08-25T09:41:13.526306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:43.216828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556193+0000","last_change":"2023-08-25T09:41:13.556112+0000","last_active":"2023-08-25T09:41:13.556193+0000","last_peered":"2023-08-25T09:41:13.556193+0000","last_clean":"2023-08-25T09:41:13.556193+0000","last_became_active":"2023-08-25T09:41:13.555863+0000","last_became_peered":"2023-08-25T09:41:13.555863+0000","last_unstale":"2023-08-25T09:41:13.556193+0000","last_undegraded":"2023-08-25T09:41:13.556193+0000","last_fullsized":"2023-08-25T09:41:13.556193+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:51.965474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.531117+0000","last_change":"2023-08-25T09:41:12.510171+0000","last_active":"2023-08-25T09:41:13.531117+0000","last_peered":"2023-08-25T09:41:13.531117+0000","last_clean":"2023-08-25T09:41:13.531117+0000","last_became_active":"2023-08-25T09:41:12.509627+0000","last_became_peered":"2023-08-25T09:41:12.509627+0000","last_unstale":"2023-08-25T09:41:13.531117+0000","last_undegraded":"2023-08-25T09:41:13.531117+0000","last_fullsized":"2023-08-25T09:41:13.531117+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:47.804783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.544313+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:41:13.544313+0000","last_peered":"2023-08-25T09:41:13.544313+0000","last_clean":"2023-08-25T09:41:13.544313+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:41:13.544313+0000","last_undegraded":"2023-08-25T09:41:13.544313+0000","last_fullsized":"2023-08-25T09:41:13.544313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:26:37.473166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555977+0000","last_change":"2023-08-25T09:41:13.554744+0000","last_active":"2023-08-25T09:41:13.555977+0000","last_peered":"2023-08-25T09:41:13.555977+0000","last_clean":"2023-08-25T09:41:13.555977+0000","last_became_active":"2023-08-25T09:41:13.554517+0000","last_became_peered":"2023-08-25T09:41:13.554517+0000","last_unstale":"2023-08-25T09:41:13.555977+0000","last_undegraded":"2023-08-25T09:41:13.555977+0000","last_fullsized":"2023-08-25T09:41:13.555977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:11:02.046065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545449+0000","last_change":"2023-08-25T09:41:12.513736+0000","last_active":"2023-08-25T09:41:13.545449+0000","last_peered":"2023-08-25T09:41:13.545449+0000","last_clean":"2023-08-25T09:41:13.545449+0000","last_became_active":"2023-08-25T09:41:12.513456+0000","last_became_peered":"2023-08-25T09:41:12.513456+0000","last_unstale":"2023-08-25T09:41:13.545449+0000","last_undegraded":"2023-08-25T09:41:13.545449+0000","last_fullsized":"2023-08-25T09:41:13.545449+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:44:00.871853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.524828+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:41:13.524828+0000","last_peered":"2023-08-25T09:41:13.524828+0000","last_clean":"2023-08-25T09:41:13.524828+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:41:13.524828+0000","last_undegraded":"2023-08-25T09:41:13.524828+0000","last_fullsized":"2023-08-25T09:41:13.524828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:02.023908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554633+0000","last_change":"2023-08-25T09:41:13.554492+0000","last_active":"2023-08-25T09:41:13.554633+0000","last_peered":"2023-08-25T09:41:13.554633+0000","last_clean":"2023-08-25T09:41:13.554633+0000","last_became_active":"2023-08-25T09:41:13.553919+0000","last_became_peered":"2023-08-25T09:41:13.553919+0000","last_unstale":"2023-08-25T09:41:13.554633+0000","last_undegraded":"2023-08-25T09:41:13.554633+0000","last_fullsized":"2023-08-25T09:41:13.554633+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:32.208562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.538241+0000","last_change":"2023-08-25T09:41:12.510574+0000","last_active":"2023-08-25T09:41:13.538241+0000","last_peered":"2023-08-25T09:41:13.538241+0000","last_clean":"2023-08-25T09:41:13.538241+0000","last_became_active":"2023-08-25T09:41:12.508978+0000","last_became_peered":"2023-08-25T09:41:12.508978+0000","last_unstale":"2023-08-25T09:41:13.538241+0000","last_undegraded":"2023-08-25T09:41:13.538241+0000","last_fullsized":"2023-08-25T09:41:13.538241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:12.710316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.542931+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:41:13.542931+0000","last_peered":"2023-08-25T09:41:13.542931+0000","last_clean":"2023-08-25T09:41:13.542931+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:41:13.542931+0000","last_undegraded":"2023-08-25T09:41:13.542931+0000","last_fullsized":"2023-08-25T09:41:13.542931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:12:54.008847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535072+0000","last_change":"2023-08-25T09:41:13.534896+0000","last_active":"2023-08-25T09:41:13.535072+0000","last_peered":"2023-08-25T09:41:13.535072+0000","last_clean":"2023-08-25T09:41:13.535072+0000","last_became_active":"2023-08-25T09:41:13.534272+0000","last_became_peered":"2023-08-25T09:41:13.534272+0000","last_unstale":"2023-08-25T09:41:13.535072+0000","last_undegraded":"2023-08-25T09:41:13.535072+0000","last_fullsized":"2023-08-25T09:41:13.535072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:36:13.288869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.542447+0000","last_change":"2023-08-25T09:41:12.516335+0000","last_active":"2023-08-25T09:41:13.542447+0000","last_peered":"2023-08-25T09:41:13.542447+0000","last_clean":"2023-08-25T09:41:13.542447+0000","last_became_active":"2023-08-25T09:41:12.516027+0000","last_became_peered":"2023-08-25T09:41:12.516027+0000","last_unstale":"2023-08-25T09:41:13.542447+0000","last_undegraded":"2023-08-25T09:41:13.542447+0000","last_fullsized":"2023-08-25T09:41:13.542447+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:08:25.708535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.525681+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:41:13.525681+0000","last_peered":"2023-08-25T09:41:13.525681+0000","last_clean":"2023-08-25T09:41:13.525681+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:41:13.525681+0000","last_undegraded":"2023-08-25T09:41:13.525681+0000","last_fullsized":"2023-08-25T09:41:13.525681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:38.069632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195064,"statfs":{"total":200000000,"available":199745870,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.038,"15min":1.0149999999999999},"min":{"1min":0.81299999999999994,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.004},"min":{"1min":0.76200000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1080000000000001,"15min":1.103},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5700000000000001,"5min":1.651,"15min":1.651},"last":1.091},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.0880000000000001,"15min":1.0609999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.0509999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.069,"15min":1.05},"min":{"1min":0.90400000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.745},{"interface":"front","average":{"1min":1.254,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.02}]},{"osd":3,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.159,"15min":1.1399999999999999},"min":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.456,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1459999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2609999999999999,"15min":1.2529999999999999},"min":{"1min":0.86499999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.379,"5min":1.2909999999999999,"15min":1.2769999999999999},"min":{"1min":0.96099999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.874}]},{"osd":5,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.3380000000000001,"15min":1.306},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.369,"15min":1.3540000000000001},"min":{"1min":0.83599999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.0179999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.415,"15min":1.401},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0859999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.3919999999999999,"15min":1.3720000000000001},"min":{"1min":1.0149999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9890000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.89900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195063,"statfs":{"total":200000000,"available":199745344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.133,"15min":1.1479999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.508,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.115,"5min":1.121,"15min":1.1220000000000001},"min":{"1min":0.70199999999999996,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8200000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.179,"15min":1.1779999999999999},"min":{"1min":0.76000000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5680000000000001,"5min":1.927,"15min":1.927},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.995}]},{"osd":2,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.085,"15min":1.0780000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5529999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.149,"15min":1.133},"min":{"1min":0.83399999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.262}]},{"osd":3,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.208,"15min":1.1910000000000001},"min":{"1min":1.0429999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.476}]},{"osd":4,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.2490000000000001,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.302,"15min":1.29},"min":{"1min":0.93100000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.512}]},{"osd":5,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.274,"15min":1.262},"min":{"1min":0.91900000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.3540000000000001,"15min":1.3300000000000001},"min":{"1min":0.91300000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.353}]},{"osd":7,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.474,"15min":1.4690000000000001},"min":{"1min":0.96599999999999997,"5min":0.622,"15min":0.622},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.651},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.48,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.726}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":826,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199153856,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.74199999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.998},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.1020000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1870000000000001,"15min":1.2090000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4279999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.081,"15min":1.085},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.4059999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.417}]},{"osd":2,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.4810000000000001,"5min":1.788,"15min":1.788},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1930000000000001,"15min":1.1990000000000001},"min":{"1min":0.79300000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4450000000000001,"5min":1.962,"15min":1.962},"last":1.446}]},{"osd":3,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.244,"15min":1.262},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6599999999999999,"5min":1.921,"15min":1.921},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7949999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.6120000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.29,"15min":1.282},"min":{"1min":0.92000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7150000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.51},{"interface":"front","average":{"1min":1.26,"5min":1.3600000000000001,"15min":1.3759999999999999},"min":{"1min":0.876,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0270000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.3560000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.413,"15min":1.423},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.696,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.85399999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.724,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.679}]},{"osd":7,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4179999999999999,"15min":1.425},"min":{"1min":0.82699999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.508,"5min":1.5609999999999999,"15min":1.5700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.857,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.911}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":241,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195071,"statfs":{"total":200000000,"available":199753054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.143,"15min":1.139},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.22,"5min":1.115,"15min":1.097},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.111},"min":{"1min":0.78200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6000000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.288},{"interface":"front","average":{"1min":1.149,"5min":1.413,"15min":1.4570000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9379999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.026}]},{"osd":2,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2969999999999999,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.482,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.1539999999999999,"15min":1.137},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.224}]},{"osd":3,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.218,"15min":1.206},"min":{"1min":0.82699999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.53},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3680000000000001,"15min":1.3720000000000001},"min":{"1min":1.083,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.4370000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.292,"15min":1.294},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.0640000000000001,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.216,"15min":1.2110000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.6379999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.381}]},{"osd":6,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.7709999999999999,"5min":2.863,"15min":2.863},"last":1.569},{"interface":"front","average":{"1min":1.544,"5min":1.5609999999999999,"15min":1.5640000000000001},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.1339999999999999,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.4910000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.375,"15min":1.359},"min":{"1min":0.93799999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.583,"15min":1.591},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":1.9850000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.5149999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195064,"statfs":{"total":200000000,"available":199745836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3660000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86599999999999999,"15min":0.86899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.874},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.93600000000000005,"15min":0.95399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.292,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.83399999999999996,"15min":0.82799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.081,"15min":1.101},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9710000000000001,"5min":3.399,"15min":3.399},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1659999999999999,"15min":1.173},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.173,"15min":1.179},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5620000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1599999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.286,"15min":1.3},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0139999999999998,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.395},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.84899999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.544,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.198}]},{"osd":6,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.226,"15min":1.2230000000000001},"min":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.3819999999999999,"15min":1.3999999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.9159999999999999,"5min":3.496,"15min":3.496},"last":1.3340000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.329,"15min":1.341},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.921,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2609999999999999,"15min":1.2490000000000001},"min":{"1min":1.0389999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.3660000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":241,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195071,"statfs":{"total":200000000,"available":199753054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82499999999999996,"15min":0.82199999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.46,"5min":2.383,"15min":2.383},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.0569999999999999,"15min":1.1040000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.373,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.92500000000000004}]},{"osd":1,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96999999999999997,"15min":0.98599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2370000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0609999999999999,"15min":1.0649999999999999},"min":{"1min":0.505,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.677,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.0800000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.099,"15min":1.1339999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.343,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.091,"15min":1.105},"min":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4490000000000001,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.5760000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.343,"15min":1.359},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.6930000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1930000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.385,"15min":1.415},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6839999999999999,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.163},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.393,"15min":1.405},"min":{"1min":0.93999999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.635,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.425,"15min":1.4350000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.766,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.4770000000000001,"15min":1.5109999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6419999999999999,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.456,"15min":1.4730000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.881,"5min":2.968,"15min":2.968},"last":1.363},{"interface":"front","average":{"1min":1.321,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.778,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.415}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":824,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194487,"statfs":{"total":200000000,"available":199155488,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.51700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1850000000000001,"5min":1.627,"15min":1.627},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.77000000000000002,"15min":0.76700000000000002},"min":{"1min":0.46999999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2190000000000001,"5min":1.296,"15min":1.296},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.57699999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.282,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.997,"15min":0.99299999999999999},"min":{"1min":0.60299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.6020000000000001,"5min":1.7,"15min":1.7},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.91100000000000003,"15min":0.90800000000000003},"min":{"1min":0.503,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.745}]},{"osd":4,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0620000000000001,"15min":1.056},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.25,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.85999999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6020000000000001,"5min":1.919,"15min":1.919},"last":0.629}]},{"osd":5,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.169,"15min":1.165},"min":{"1min":0.80700000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.6459999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.133,"5min":1.141,"15min":1.143},"min":{"1min":0.83099999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.496,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.26,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5349999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2549999999999999,"15min":1.2589999999999999},"min":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.728,"5min":1.847,"15min":1.847},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.708,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.357,"5min":1.2849999999999999,"15min":1.2729999999999999},"min":{"1min":0.93400000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7350000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.81899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195063,"statfs":{"total":200000000,"available":199745344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.3260000000000001,"5min":1.361,"15min":1.361},"last":1.2},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.92100000000000004,"15min":0.94099999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3779999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0840000000000001,"15min":1.121},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3120000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.99199999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.095,"5min":1.387,"15min":1.387},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.099,"15min":1.1259999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4970000000000001,"5min":1.869,"15min":1.869},"last":1.325}]},{"osd":4,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.21,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4379999999999999,"5min":1.704,"15min":1.704},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3320000000000001,"15min":1.3520000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7090000000000001,"5min":1.948,"15min":1.948},"last":1.45}]},{"osd":5,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.3400000000000001,"15min":1.3859999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.431,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.151},{"interface":"front","average":{"1min":1.105,"5min":1.2809999999999999,"15min":1.3109999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5920000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.1220000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.321,"15min":1.349},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6970000000000001,"5min":2.044,"15min":2.044},"last":1.395},{"interface":"front","average":{"1min":1.202,"5min":1.2809999999999999,"15min":1.294},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.262}]},{"osd":7,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.708,"5min":2.121,"15min":2.121},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3029999999999999,"15min":1.3200000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5469999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.282}]}]}],"pool_statfs":[]}} 2023-08-25T09:41:17.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:41:18.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.232+0000 7f7c90fb1640 1 Processor -- start 2023-08-25T09:41:18.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.232+0000 7f7c90fb1640 1 -- start start 2023-08-25T09:41:18.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.233+0000 7f7c90fb1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7c8c159720 0x7f7c8c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.233+0000 7f7c90fb1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7c8c06f820 0x7f7c8c0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.233+0000 7f7c90fb1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8c0ca2a0 con 0x7f7c8c159b00 2023-08-25T09:41:18.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.233+0000 7f7c90fb1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c0ca580 con 0x7f7c8c06f820 2023-08-25T09:41:18.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.233+0000 7f7c90fb1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8c0cd350 con 0x7f7c8c159720 2023-08-25T09:41:18.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7c8c06f820 0x7f7c8c0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7c8c06f820 0x7f7c8c0b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50138/0 (socket says 172.21.15.70:50138) 2023-08-25T09:41:18.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/2732897696 learned_addr learned my addr 172.21.15.70:0/2732897696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:18.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c89d9b640 1 --1- 172.21.15.70:0/2732897696 >> v1:172.21.15.70:6789/0 conn(0x7f7c8c159b00 0x7f7c8c0c9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53484/0 (socket says 172.21.15.70:53484) 2023-08-25T09:41:18.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8a59c640 1 --2- 172.21.15.70:0/2732897696 >> v2:172.21.15.155:3300/0 conn(0x7f7c8c159720 0x7f7c8c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/2732897696 >> v2:172.21.15.155:3300/0 conn(0x7f7c8c159720 msgr2=0x7f7c8c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:18.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 --2- 172.21.15.70:0/2732897696 >> v2:172.21.15.155:3300/0 conn(0x7f7c8c159720 0x7f7c8c15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/2732897696 >> v1:172.21.15.70:6789/0 conn(0x7f7c8c159b00 legacy=0x7f7c8c0c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/2732897696 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c8c0cd630 con 0x7f7c8c06f820 2023-08-25T09:41:18.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.234+0000 7f7c8a59c640 1 --2- 172.21.15.70:0/2732897696 >> v2:172.21.15.155:3300/0 conn(0x7f7c8c159720 0x7f7c8c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:18.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.235+0000 7f7c8ad9d640 1 --2- 172.21.15.70:0/2732897696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0b0f80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c06f170 tx=0x7f7c8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1ccdd6fba9492f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:18.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.236+0000 7f7c8959a640 1 -- 172.21.15.70:0/2732897696 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84031870 con 0x7f7c8c06f820 2023-08-25T09:41:18.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.236+0000 7f7c8959a640 1 -- 172.21.15.70:0/2732897696 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c840319d0 con 0x7f7c8c06f820 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.236+0000 7f7c8959a640 1 -- 172.21.15.70:0/2732897696 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84031d60 con 0x7f7c8c06f820 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.236+0000 7f7c90fb1640 1 -- 172.21.15.70:0/2732897696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c06f820 msgr2=0x7f7c8c0b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.236+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/2732897696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0b0f80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c06f170 tx=0x7f7c8402b910 comp rx=0 tx=0).stop 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.237+0000 7f7c90fb1640 1 -- 172.21.15.70:0/2732897696 shutdown_connections 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.237+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/2732897696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0b0f80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.237+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/2732897696 >> v2:172.21.15.155:3300/0 conn(0x7f7c8c159720 0x7f7c8c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.237+0000 7f7c90fb1640 1 -- 172.21.15.70:0/2732897696 >> 172.21.15.70:0/2732897696 conn(0x7f7c8c069c70 msgr2=0x7f7c8c06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:18.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.237+0000 7f7c90fb1640 1 -- 172.21.15.70:0/2732897696 shutdown_connections 2023-08-25T09:41:18.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.237+0000 7f7c90fb1640 1 -- 172.21.15.70:0/2732897696 wait complete. 2023-08-25T09:41:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.238+0000 7f7c90fb1640 1 Processor -- start 2023-08-25T09:41:18.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.239+0000 7f7c90fb1640 1 -- start start 2023-08-25T09:41:18.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c90fb1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c90fb1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c8c159720 0x7f7c8c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c90fb1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 0x7f7c8c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c90fb1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0c4450 con 0x7f7c8c159b00 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c90fb1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0cdb80 con 0x7f7c8c06f820 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c90fb1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c0cde60 con 0x7f7c8c159720 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 0x7f7c8c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 0x7f7c8c0b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50152/0 (socket says 172.21.15.70:50152) 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/3293530090 learned_addr learned my addr 172.21.15.70:0/3293530090 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:18.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c8c159720 msgr2=0x7f7c8c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c8c159720 0x7f7c8c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c8c06f820 msgr2=0x7f7c8c0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8a59c640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8ad9d640 1 -- 172.21.15.70:0/3293530090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840094d0 con 0x7f7c8c159b00 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.240+0000 7f7c8a59c640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.241+0000 7f7c8ad9d640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 0x7f7c8c0b35c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7c8402b950 tx=0x7f7c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.241+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84031800 con 0x7f7c8c159b00 2023-08-25T09:41:18.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.241+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c840443f0 con 0x7f7c8c159b00 2023-08-25T09:41:18.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.241+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c840447a0 con 0x7f7c8c159b00 2023-08-25T09:41:18.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.241+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c0ce140 con 0x7f7c8c159b00 2023-08-25T09:41:18.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.241+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c154980 con 0x7f7c8c159b00 2023-08-25T09:41:18.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.242+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c0ce660 con 0x7f7c8c159b00 2023-08-25T09:41:18.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.243+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7c84044900 con 0x7f7c8c159b00 2023-08-25T09:41:18.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.244+0000 7f7c82ffd640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c6c06a050 0x7f7c6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.244+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c8402d040 con 0x7f7c8c159b00 2023-08-25T09:41:18.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.244+0000 7f7c8a59c640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c6c06a050 0x7f7c6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.245+0000 7f7c8a59c640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c6c06a050 0x7f7c6c06c510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00d430 tx=0x7f7c7c007530 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:18.249 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-08-25T09:41:18.247+0000 7f14da781640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:41:18.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.249+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7c8c0ce660 con 0x7f7c8c159b00 2023-08-25T09:41:18.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.434+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c8c0b14c0 con 0x7f7c6c06a050 2023-08-25T09:41:18.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.456+0000 7f7c82ffd640 1 -- 172.21.15.70:0/3293530090 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302767 (secure 0 0 0) 0x7f7c7c018470 con 0x7f7c6c06a050 2023-08-25T09:41:18.464 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:18.466 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:41:18.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c6c06a050 msgr2=0x7f7c6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c6c06a050 0x7f7c6c06c510 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00d430 tx=0x7f7c7c007530 comp rx=0 tx=0).stop 2023-08-25T09:41:18.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 msgr2=0x7f7c8c0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 0x7f7c8c0b35c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7c8402b950 tx=0x7f7c840046c0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 shutdown_connections 2023-08-25T09:41:18.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c6c06a050 0x7f7c6c06c510 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c8c159b00 0x7f7c8c0b35c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.463+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c8c159720 0x7f7c8c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.464+0000 7f7c90fb1640 1 --2- 172.21.15.70:0/3293530090 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c8c06f820 0x7f7c8c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.464+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 >> 172.21.15.70:0/3293530090 conn(0x7f7c8c069c70 msgr2=0x7f7c8c0aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:18.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.464+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 shutdown_connections 2023-08-25T09:41:18.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.464+0000 7f7c90fb1640 1 -- 172.21.15.70:0/3293530090 wait complete. 2023-08-25T09:41:18.486 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2023-08-25T09:41:17.364296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3127,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559372,"statfs":{"total":1600000000,"available":1596797846,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.880560"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557059+0000","last_change":"2023-08-25T09:41:13.556673+0000","last_active":"2023-08-25T09:41:13.557059+0000","last_peered":"2023-08-25T09:41:13.557059+0000","last_clean":"2023-08-25T09:41:13.557059+0000","last_became_active":"2023-08-25T09:41:13.555714+0000","last_became_peered":"2023-08-25T09:41:13.555714+0000","last_unstale":"2023-08-25T09:41:13.557059+0000","last_undegraded":"2023-08-25T09:41:13.557059+0000","last_fullsized":"2023-08-25T09:41:13.557059+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:41.177052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.550206+0000","last_change":"2023-08-25T09:41:13.550125+0000","last_active":"2023-08-25T09:41:13.550206+0000","last_peered":"2023-08-25T09:41:13.550206+0000","last_clean":"2023-08-25T09:41:13.550206+0000","last_became_active":"2023-08-25T09:41:13.549751+0000","last_became_peered":"2023-08-25T09:41:13.549751+0000","last_unstale":"2023-08-25T09:41:13.550206+0000","last_undegraded":"2023-08-25T09:41:13.550206+0000","last_fullsized":"2023-08-25T09:41:13.550206+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:17:21.788914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559057+0000","last_change":"2023-08-25T09:41:13.558900+0000","last_active":"2023-08-25T09:41:13.559057+0000","last_peered":"2023-08-25T09:41:13.559057+0000","last_clean":"2023-08-25T09:41:13.559057+0000","last_became_active":"2023-08-25T09:41:13.558569+0000","last_became_peered":"2023-08-25T09:41:13.558569+0000","last_unstale":"2023-08-25T09:41:13.559057+0000","last_undegraded":"2023-08-25T09:41:13.559057+0000","last_fullsized":"2023-08-25T09:41:13.559057+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:26:20.502288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556775+0000","last_change":"2023-08-25T09:41:13.556515+0000","last_active":"2023-08-25T09:41:13.556775+0000","last_peered":"2023-08-25T09:41:13.556775+0000","last_clean":"2023-08-25T09:41:13.556775+0000","last_became_active":"2023-08-25T09:41:13.554947+0000","last_became_peered":"2023-08-25T09:41:13.554947+0000","last_unstale":"2023-08-25T09:41:13.556775+0000","last_undegraded":"2023-08-25T09:41:13.556775+0000","last_fullsized":"2023-08-25T09:41:13.556775+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:52:22.767545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.541443+0000","last_change":"2023-08-25T09:41:13.541056+0000","last_active":"2023-08-25T09:41:13.541443+0000","last_peered":"2023-08-25T09:41:13.541443+0000","last_clean":"2023-08-25T09:41:13.541443+0000","last_became_active":"2023-08-25T09:41:13.537590+0000","last_became_peered":"2023-08-25T09:41:13.537590+0000","last_unstale":"2023-08-25T09:41:13.541443+0000","last_undegraded":"2023-08-25T09:41:13.541443+0000","last_fullsized":"2023-08-25T09:41:13.541443+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:22:20.266956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553906+0000","last_change":"2023-08-25T09:41:13.551465+0000","last_active":"2023-08-25T09:41:13.553906+0000","last_peered":"2023-08-25T09:41:13.553906+0000","last_clean":"2023-08-25T09:41:13.553906+0000","last_became_active":"2023-08-25T09:41:13.550778+0000","last_became_peered":"2023-08-25T09:41:13.550778+0000","last_unstale":"2023-08-25T09:41:13.553906+0000","last_undegraded":"2023-08-25T09:41:13.553906+0000","last_fullsized":"2023-08-25T09:41:13.553906+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:50.350958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549212+0000","last_change":"2023-08-25T09:41:13.549071+0000","last_active":"2023-08-25T09:41:13.549212+0000","last_peered":"2023-08-25T09:41:13.549212+0000","last_clean":"2023-08-25T09:41:13.549212+0000","last_became_active":"2023-08-25T09:41:13.547530+0000","last_became_peered":"2023-08-25T09:41:13.547530+0000","last_unstale":"2023-08-25T09:41:13.549212+0000","last_undegraded":"2023-08-25T09:41:13.549212+0000","last_fullsized":"2023-08-25T09:41:13.549212+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:09:24.021766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559707+0000","last_change":"2023-08-25T09:41:13.559609+0000","last_active":"2023-08-25T09:41:13.559707+0000","last_peered":"2023-08-25T09:41:13.559707+0000","last_clean":"2023-08-25T09:41:13.559707+0000","last_became_active":"2023-08-25T09:41:13.559163+0000","last_became_peered":"2023-08-25T09:41:13.559163+0000","last_unstale":"2023-08-25T09:41:13.559707+0000","last_undegraded":"2023-08-25T09:41:13.559707+0000","last_fullsized":"2023-08-25T09:41:13.559707+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:45.056101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555792+0000","last_change":"2023-08-25T09:41:13.555515+0000","last_active":"2023-08-25T09:41:13.555792+0000","last_peered":"2023-08-25T09:41:13.555792+0000","last_clean":"2023-08-25T09:41:13.555792+0000","last_became_active":"2023-08-25T09:41:13.555189+0000","last_became_peered":"2023-08-25T09:41:13.555189+0000","last_unstale":"2023-08-25T09:41:13.555792+0000","last_undegraded":"2023-08-25T09:41:13.555792+0000","last_fullsized":"2023-08-25T09:41:13.555792+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:27:44.405255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557056+0000","last_change":"2023-08-25T09:41:13.556924+0000","last_active":"2023-08-25T09:41:13.557056+0000","last_peered":"2023-08-25T09:41:13.557056+0000","last_clean":"2023-08-25T09:41:13.557056+0000","last_became_active":"2023-08-25T09:41:13.556667+0000","last_became_peered":"2023-08-25T09:41:13.556667+0000","last_unstale":"2023-08-25T09:41:13.557056+0000","last_undegraded":"2023-08-25T09:41:13.557056+0000","last_fullsized":"2023-08-25T09:41:13.557056+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:06.224484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554954+0000","last_change":"2023-08-25T09:41:13.553702+0000","last_active":"2023-08-25T09:41:13.554954+0000","last_peered":"2023-08-25T09:41:13.554954+0000","last_clean":"2023-08-25T09:41:13.554954+0000","last_became_active":"2023-08-25T09:41:13.553424+0000","last_became_peered":"2023-08-25T09:41:13.553424+0000","last_unstale":"2023-08-25T09:41:13.554954+0000","last_undegraded":"2023-08-25T09:41:13.554954+0000","last_fullsized":"2023-08-25T09:41:13.554954+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:18:27.360452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555955+0000","last_change":"2023-08-25T09:41:13.548665+0000","last_active":"2023-08-25T09:41:13.555955+0000","last_peered":"2023-08-25T09:41:13.555955+0000","last_clean":"2023-08-25T09:41:13.555955+0000","last_became_active":"2023-08-25T09:41:13.548083+0000","last_became_peered":"2023-08-25T09:41:13.548083+0000","last_unstale":"2023-08-25T09:41:13.555955+0000","last_undegraded":"2023-08-25T09:41:13.555955+0000","last_fullsized":"2023-08-25T09:41:13.555955+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:41.655105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.541577+0000","last_change":"2023-08-25T09:41:13.539800+0000","last_active":"2023-08-25T09:41:13.541577+0000","last_peered":"2023-08-25T09:41:13.541577+0000","last_clean":"2023-08-25T09:41:13.541577+0000","last_became_active":"2023-08-25T09:41:13.539070+0000","last_became_peered":"2023-08-25T09:41:13.539070+0000","last_unstale":"2023-08-25T09:41:13.541577+0000","last_undegraded":"2023-08-25T09:41:13.541577+0000","last_fullsized":"2023-08-25T09:41:13.541577+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:16.561672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556481+0000","last_change":"2023-08-25T09:41:13.555925+0000","last_active":"2023-08-25T09:41:13.556481+0000","last_peered":"2023-08-25T09:41:13.556481+0000","last_clean":"2023-08-25T09:41:13.556481+0000","last_became_active":"2023-08-25T09:41:13.555603+0000","last_became_peered":"2023-08-25T09:41:13.555603+0000","last_unstale":"2023-08-25T09:41:13.556481+0000","last_undegraded":"2023-08-25T09:41:13.556481+0000","last_fullsized":"2023-08-25T09:41:13.556481+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:48.330370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.546620+0000","last_change":"2023-08-25T09:41:13.546354+0000","last_active":"2023-08-25T09:41:13.546620+0000","last_peered":"2023-08-25T09:41:13.546620+0000","last_clean":"2023-08-25T09:41:13.546620+0000","last_became_active":"2023-08-25T09:41:13.545510+0000","last_became_peered":"2023-08-25T09:41:13.545510+0000","last_unstale":"2023-08-25T09:41:13.546620+0000","last_undegraded":"2023-08-25T09:41:13.546620+0000","last_fullsized":"2023-08-25T09:41:13.546620+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:56:03.228283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553891+0000","last_change":"2023-08-25T09:41:13.551194+0000","last_active":"2023-08-25T09:41:13.553891+0000","last_peered":"2023-08-25T09:41:13.553891+0000","last_clean":"2023-08-25T09:41:13.553891+0000","last_became_active":"2023-08-25T09:41:13.550330+0000","last_became_peered":"2023-08-25T09:41:13.550330+0000","last_unstale":"2023-08-25T09:41:13.553891+0000","last_undegraded":"2023-08-25T09:41:13.553891+0000","last_fullsized":"2023-08-25T09:41:13.553891+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:00:04.708379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556501+0000","last_change":"2023-08-25T09:41:13.556438+0000","last_active":"2023-08-25T09:41:13.556501+0000","last_peered":"2023-08-25T09:41:13.556501+0000","last_clean":"2023-08-25T09:41:13.556501+0000","last_became_active":"2023-08-25T09:41:13.555917+0000","last_became_peered":"2023-08-25T09:41:13.555917+0000","last_unstale":"2023-08-25T09:41:13.556501+0000","last_undegraded":"2023-08-25T09:41:13.556501+0000","last_fullsized":"2023-08-25T09:41:13.556501+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:06.224484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554646+0000","last_change":"2023-08-25T09:41:13.554458+0000","last_active":"2023-08-25T09:41:13.554646+0000","last_peered":"2023-08-25T09:41:13.554646+0000","last_clean":"2023-08-25T09:41:13.554646+0000","last_became_active":"2023-08-25T09:41:13.554005+0000","last_became_peered":"2023-08-25T09:41:13.554005+0000","last_unstale":"2023-08-25T09:41:13.554646+0000","last_undegraded":"2023-08-25T09:41:13.554646+0000","last_fullsized":"2023-08-25T09:41:13.554646+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:11:48.764235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553935+0000","last_change":"2023-08-25T09:41:13.553867+0000","last_active":"2023-08-25T09:41:13.553935+0000","last_peered":"2023-08-25T09:41:13.553935+0000","last_clean":"2023-08-25T09:41:13.553935+0000","last_became_active":"2023-08-25T09:41:13.553458+0000","last_became_peered":"2023-08-25T09:41:13.553458+0000","last_unstale":"2023-08-25T09:41:13.553935+0000","last_undegraded":"2023-08-25T09:41:13.553935+0000","last_fullsized":"2023-08-25T09:41:13.553935+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:32.662025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555719+0000","last_change":"2023-08-25T09:41:13.555601+0000","last_active":"2023-08-25T09:41:13.555719+0000","last_peered":"2023-08-25T09:41:13.555719+0000","last_clean":"2023-08-25T09:41:13.555719+0000","last_became_active":"2023-08-25T09:41:13.554965+0000","last_became_peered":"2023-08-25T09:41:13.554965+0000","last_unstale":"2023-08-25T09:41:13.555719+0000","last_undegraded":"2023-08-25T09:41:13.555719+0000","last_fullsized":"2023-08-25T09:41:13.555719+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:17:09.525913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553987+0000","last_change":"2023-08-25T09:41:13.553869+0000","last_active":"2023-08-25T09:41:13.553987+0000","last_peered":"2023-08-25T09:41:13.553987+0000","last_clean":"2023-08-25T09:41:13.553987+0000","last_became_active":"2023-08-25T09:41:13.553305+0000","last_became_peered":"2023-08-25T09:41:13.553305+0000","last_unstale":"2023-08-25T09:41:13.553987+0000","last_undegraded":"2023-08-25T09:41:13.553987+0000","last_fullsized":"2023-08-25T09:41:13.553987+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:24.016454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.552840+0000","last_change":"2023-08-25T09:41:13.552708+0000","last_active":"2023-08-25T09:41:13.552840+0000","last_peered":"2023-08-25T09:41:13.552840+0000","last_clean":"2023-08-25T09:41:13.552840+0000","last_became_active":"2023-08-25T09:41:13.552166+0000","last_became_peered":"2023-08-25T09:41:13.552166+0000","last_unstale":"2023-08-25T09:41:13.552840+0000","last_undegraded":"2023-08-25T09:41:13.552840+0000","last_fullsized":"2023-08-25T09:41:13.552840+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:37.360613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.544562+0000","last_change":"2023-08-25T09:41:13.544321+0000","last_active":"2023-08-25T09:41:13.544562+0000","last_peered":"2023-08-25T09:41:13.544562+0000","last_clean":"2023-08-25T09:41:13.544562+0000","last_became_active":"2023-08-25T09:41:13.543707+0000","last_became_peered":"2023-08-25T09:41:13.543707+0000","last_unstale":"2023-08-25T09:41:13.544562+0000","last_undegraded":"2023-08-25T09:41:13.544562+0000","last_fullsized":"2023-08-25T09:41:13.544562+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:29.117817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553918+0000","last_change":"2023-08-25T09:41:13.553706+0000","last_active":"2023-08-25T09:41:13.553918+0000","last_peered":"2023-08-25T09:41:13.553918+0000","last_clean":"2023-08-25T09:41:13.553918+0000","last_became_active":"2023-08-25T09:41:13.552154+0000","last_became_peered":"2023-08-25T09:41:13.552154+0000","last_unstale":"2023-08-25T09:41:13.553918+0000","last_undegraded":"2023-08-25T09:41:13.553918+0000","last_fullsized":"2023-08-25T09:41:13.553918+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:03.273034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556524+0000","last_change":"2023-08-25T09:41:13.555731+0000","last_active":"2023-08-25T09:41:13.556524+0000","last_peered":"2023-08-25T09:41:13.556524+0000","last_clean":"2023-08-25T09:41:13.556524+0000","last_became_active":"2023-08-25T09:41:13.554808+0000","last_became_peered":"2023-08-25T09:41:13.554808+0000","last_unstale":"2023-08-25T09:41:13.556524+0000","last_undegraded":"2023-08-25T09:41:13.556524+0000","last_fullsized":"2023-08-25T09:41:13.556524+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:32.208562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553867+0000","last_change":"2023-08-25T09:41:13.553486+0000","last_active":"2023-08-25T09:41:13.553867+0000","last_peered":"2023-08-25T09:41:13.553867+0000","last_clean":"2023-08-25T09:41:13.553867+0000","last_became_active":"2023-08-25T09:41:13.552843+0000","last_became_peered":"2023-08-25T09:41:13.552843+0000","last_unstale":"2023-08-25T09:41:13.553867+0000","last_undegraded":"2023-08-25T09:41:13.553867+0000","last_fullsized":"2023-08-25T09:41:13.553867+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:05:14.270183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559084+0000","last_change":"2023-08-25T09:41:13.559010+0000","last_active":"2023-08-25T09:41:13.559084+0000","last_peered":"2023-08-25T09:41:13.559084+0000","last_clean":"2023-08-25T09:41:13.559084+0000","last_became_active":"2023-08-25T09:41:13.558720+0000","last_became_peered":"2023-08-25T09:41:13.558720+0000","last_unstale":"2023-08-25T09:41:13.559084+0000","last_undegraded":"2023-08-25T09:41:13.559084+0000","last_fullsized":"2023-08-25T09:41:13.559084+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:59:38.698765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549531+0000","last_change":"2023-08-25T09:41:13.547485+0000","last_active":"2023-08-25T09:41:13.549531+0000","last_peered":"2023-08-25T09:41:13.549531+0000","last_clean":"2023-08-25T09:41:13.549531+0000","last_became_active":"2023-08-25T09:41:13.546997+0000","last_became_peered":"2023-08-25T09:41:13.546997+0000","last_unstale":"2023-08-25T09:41:13.549531+0000","last_undegraded":"2023-08-25T09:41:13.549531+0000","last_fullsized":"2023-08-25T09:41:13.549531+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:01.247517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559092+0000","last_change":"2023-08-25T09:41:13.558973+0000","last_active":"2023-08-25T09:41:13.559092+0000","last_peered":"2023-08-25T09:41:13.559092+0000","last_clean":"2023-08-25T09:41:13.559092+0000","last_became_active":"2023-08-25T09:41:13.558717+0000","last_became_peered":"2023-08-25T09:41:13.558717+0000","last_unstale":"2023-08-25T09:41:13.559092+0000","last_undegraded":"2023-08-25T09:41:13.559092+0000","last_fullsized":"2023-08-25T09:41:13.559092+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:15.199506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.551228+0000","last_change":"2023-08-25T09:41:13.550468+0000","last_active":"2023-08-25T09:41:13.551228+0000","last_peered":"2023-08-25T09:41:13.551228+0000","last_clean":"2023-08-25T09:41:13.551228+0000","last_became_active":"2023-08-25T09:41:13.550079+0000","last_became_peered":"2023-08-25T09:41:13.550079+0000","last_unstale":"2023-08-25T09:41:13.551228+0000","last_undegraded":"2023-08-25T09:41:13.551228+0000","last_fullsized":"2023-08-25T09:41:13.551228+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:22.630319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553902+0000","last_change":"2023-08-25T09:41:13.553656+0000","last_active":"2023-08-25T09:41:13.553902+0000","last_peered":"2023-08-25T09:41:13.553902+0000","last_clean":"2023-08-25T09:41:13.553902+0000","last_became_active":"2023-08-25T09:41:13.553222+0000","last_became_peered":"2023-08-25T09:41:13.553222+0000","last_unstale":"2023-08-25T09:41:13.553902+0000","last_undegraded":"2023-08-25T09:41:13.553902+0000","last_fullsized":"2023-08-25T09:41:13.553902+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:38.489653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554207+0000","last_change":"2023-08-25T09:41:13.553299+0000","last_active":"2023-08-25T09:41:13.554207+0000","last_peered":"2023-08-25T09:41:13.554207+0000","last_clean":"2023-08-25T09:41:13.554207+0000","last_became_active":"2023-08-25T09:41:13.552885+0000","last_became_peered":"2023-08-25T09:41:13.552885+0000","last_unstale":"2023-08-25T09:41:13.554207+0000","last_undegraded":"2023-08-25T09:41:13.554207+0000","last_fullsized":"2023-08-25T09:41:13.554207+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:08:09.081236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557591+0000","last_change":"2023-08-25T09:41:13.557462+0000","last_active":"2023-08-25T09:41:13.557591+0000","last_peered":"2023-08-25T09:41:13.557591+0000","last_clean":"2023-08-25T09:41:13.557591+0000","last_became_active":"2023-08-25T09:41:13.557175+0000","last_became_peered":"2023-08-25T09:41:13.557175+0000","last_unstale":"2023-08-25T09:41:13.557591+0000","last_undegraded":"2023-08-25T09:41:13.557591+0000","last_fullsized":"2023-08-25T09:41:13.557591+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:00.337761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"50'4","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574528+0000","last_change":"2023-08-25T09:41:12.513117+0000","last_active":"2023-08-25T09:41:13.574528+0000","last_peered":"2023-08-25T09:41:13.574528+0000","last_clean":"2023-08-25T09:41:13.574528+0000","last_became_active":"2023-08-25T09:41:12.512850+0000","last_became_peered":"2023-08-25T09:41:12.512850+0000","last_unstale":"2023-08-25T09:41:13.574528+0000","last_undegraded":"2023-08-25T09:41:13.574528+0000","last_fullsized":"2023-08-25T09:41:13.574528+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:50.952127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549482+0000","last_change":"2023-08-25T09:41:13.548251+0000","last_active":"2023-08-25T09:41:13.549482+0000","last_peered":"2023-08-25T09:41:13.549482+0000","last_clean":"2023-08-25T09:41:13.549482+0000","last_became_active":"2023-08-25T09:41:13.547754+0000","last_became_peered":"2023-08-25T09:41:13.547754+0000","last_unstale":"2023-08-25T09:41:13.549482+0000","last_undegraded":"2023-08-25T09:41:13.549482+0000","last_fullsized":"2023-08-25T09:41:13.549482+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:53:29.655373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574094+0000","last_change":"2023-08-25T09:41:12.517976+0000","last_active":"2023-08-25T09:41:13.574094+0000","last_peered":"2023-08-25T09:41:13.574094+0000","last_clean":"2023-08-25T09:41:13.574094+0000","last_became_active":"2023-08-25T09:41:12.517578+0000","last_became_peered":"2023-08-25T09:41:12.517578+0000","last_unstale":"2023-08-25T09:41:13.574094+0000","last_undegraded":"2023-08-25T09:41:13.574094+0000","last_fullsized":"2023-08-25T09:41:13.574094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:18:34.026179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553056+0000","last_change":"2023-08-25T09:41:13.552455+0000","last_active":"2023-08-25T09:41:13.553056+0000","last_peered":"2023-08-25T09:41:13.553056+0000","last_clean":"2023-08-25T09:41:13.553056+0000","last_became_active":"2023-08-25T09:41:13.551928+0000","last_became_peered":"2023-08-25T09:41:13.551928+0000","last_unstale":"2023-08-25T09:41:13.553056+0000","last_undegraded":"2023-08-25T09:41:13.553056+0000","last_fullsized":"2023-08-25T09:41:13.553056+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:54.383488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574472+0000","last_change":"2023-08-25T09:41:12.511383+0000","last_active":"2023-08-25T09:41:13.574472+0000","last_peered":"2023-08-25T09:41:13.574472+0000","last_clean":"2023-08-25T09:41:13.574472+0000","last_became_active":"2023-08-25T09:41:12.509068+0000","last_became_peered":"2023-08-25T09:41:12.509068+0000","last_unstale":"2023-08-25T09:41:13.574472+0000","last_undegraded":"2023-08-25T09:41:13.574472+0000","last_fullsized":"2023-08-25T09:41:13.574472+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:46:40.163705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555409+0000","last_change":"2023-08-25T09:41:13.555330+0000","last_active":"2023-08-25T09:41:13.555409+0000","last_peered":"2023-08-25T09:41:13.555409+0000","last_clean":"2023-08-25T09:41:13.555409+0000","last_became_active":"2023-08-25T09:41:13.555057+0000","last_became_peered":"2023-08-25T09:41:13.555057+0000","last_unstale":"2023-08-25T09:41:13.555409+0000","last_undegraded":"2023-08-25T09:41:13.555409+0000","last_fullsized":"2023-08-25T09:41:13.555409+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:06.469710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574001+0000","last_change":"2023-08-25T09:41:12.500623+0000","last_active":"2023-08-25T09:41:13.574001+0000","last_peered":"2023-08-25T09:41:13.574001+0000","last_clean":"2023-08-25T09:41:13.574001+0000","last_became_active":"2023-08-25T09:41:12.499955+0000","last_became_peered":"2023-08-25T09:41:12.499955+0000","last_unstale":"2023-08-25T09:41:13.574001+0000","last_undegraded":"2023-08-25T09:41:13.574001+0000","last_fullsized":"2023-08-25T09:41:13.574001+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:16.593221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554214+0000","last_change":"2023-08-25T09:41:13.553384+0000","last_active":"2023-08-25T09:41:13.554214+0000","last_peered":"2023-08-25T09:41:13.554214+0000","last_clean":"2023-08-25T09:41:13.554214+0000","last_became_active":"2023-08-25T09:41:13.551446+0000","last_became_peered":"2023-08-25T09:41:13.551446+0000","last_unstale":"2023-08-25T09:41:13.554214+0000","last_undegraded":"2023-08-25T09:41:13.554214+0000","last_fullsized":"2023-08-25T09:41:13.554214+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:32.208562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574761+0000","last_change":"2023-08-25T09:41:12.506088+0000","last_active":"2023-08-25T09:41:13.574761+0000","last_peered":"2023-08-25T09:41:13.574761+0000","last_clean":"2023-08-25T09:41:13.574761+0000","last_became_active":"2023-08-25T09:41:12.505415+0000","last_became_peered":"2023-08-25T09:41:12.505415+0000","last_unstale":"2023-08-25T09:41:13.574761+0000","last_undegraded":"2023-08-25T09:41:13.574761+0000","last_fullsized":"2023-08-25T09:41:13.574761+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:22:05.106316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535718+0000","last_change":"2023-08-25T09:41:13.534770+0000","last_active":"2023-08-25T09:41:13.535718+0000","last_peered":"2023-08-25T09:41:13.535718+0000","last_clean":"2023-08-25T09:41:13.535718+0000","last_became_active":"2023-08-25T09:41:13.533829+0000","last_became_peered":"2023-08-25T09:41:13.533829+0000","last_unstale":"2023-08-25T09:41:13.535718+0000","last_undegraded":"2023-08-25T09:41:13.535718+0000","last_fullsized":"2023-08-25T09:41:13.535718+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:13:33.365554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575161+0000","last_change":"2023-08-25T09:41:12.515423+0000","last_active":"2023-08-25T09:41:13.575161+0000","last_peered":"2023-08-25T09:41:13.575161+0000","last_clean":"2023-08-25T09:41:13.575161+0000","last_became_active":"2023-08-25T09:41:12.514991+0000","last_became_peered":"2023-08-25T09:41:12.514991+0000","last_unstale":"2023-08-25T09:41:13.575161+0000","last_undegraded":"2023-08-25T09:41:13.575161+0000","last_fullsized":"2023-08-25T09:41:13.575161+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:05.098179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554737+0000","last_change":"2023-08-25T09:41:13.554679+0000","last_active":"2023-08-25T09:41:13.554737+0000","last_peered":"2023-08-25T09:41:13.554737+0000","last_clean":"2023-08-25T09:41:13.554737+0000","last_became_active":"2023-08-25T09:41:13.554274+0000","last_became_peered":"2023-08-25T09:41:13.554274+0000","last_unstale":"2023-08-25T09:41:13.554737+0000","last_undegraded":"2023-08-25T09:41:13.554737+0000","last_fullsized":"2023-08-25T09:41:13.554737+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:18:27.360452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.524605+0000","last_change":"2023-08-25T09:41:12.512877+0000","last_active":"2023-08-25T09:41:13.524605+0000","last_peered":"2023-08-25T09:41:13.524605+0000","last_clean":"2023-08-25T09:41:13.524605+0000","last_became_active":"2023-08-25T09:41:12.512569+0000","last_became_peered":"2023-08-25T09:41:12.512569+0000","last_unstale":"2023-08-25T09:41:13.524605+0000","last_undegraded":"2023-08-25T09:41:13.524605+0000","last_fullsized":"2023-08-25T09:41:13.524605+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:19:52.599603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.558277+0000","last_change":"2023-08-25T09:41:13.558115+0000","last_active":"2023-08-25T09:41:13.558277+0000","last_peered":"2023-08-25T09:41:13.558277+0000","last_clean":"2023-08-25T09:41:13.558277+0000","last_became_active":"2023-08-25T09:41:13.557800+0000","last_became_peered":"2023-08-25T09:41:13.557800+0000","last_unstale":"2023-08-25T09:41:13.558277+0000","last_undegraded":"2023-08-25T09:41:13.558277+0000","last_fullsized":"2023-08-25T09:41:13.558277+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:14:55.454430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.538892+0000","last_change":"2023-08-25T09:41:12.510611+0000","last_active":"2023-08-25T09:41:13.538892+0000","last_peered":"2023-08-25T09:41:13.538892+0000","last_clean":"2023-08-25T09:41:13.538892+0000","last_became_active":"2023-08-25T09:41:12.509916+0000","last_became_peered":"2023-08-25T09:41:12.509916+0000","last_unstale":"2023-08-25T09:41:13.538892+0000","last_undegraded":"2023-08-25T09:41:13.538892+0000","last_fullsized":"2023-08-25T09:41:13.538892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:09:42.385237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556148+0000","last_change":"2023-08-25T09:41:13.556066+0000","last_active":"2023-08-25T09:41:13.556148+0000","last_peered":"2023-08-25T09:41:13.556148+0000","last_clean":"2023-08-25T09:41:13.556148+0000","last_became_active":"2023-08-25T09:41:13.555789+0000","last_became_peered":"2023-08-25T09:41:13.555789+0000","last_unstale":"2023-08-25T09:41:13.556148+0000","last_undegraded":"2023-08-25T09:41:13.556148+0000","last_fullsized":"2023-08-25T09:41:13.556148+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:48.330370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545605+0000","last_change":"2023-08-25T09:41:12.515713+0000","last_active":"2023-08-25T09:41:13.545605+0000","last_peered":"2023-08-25T09:41:13.545605+0000","last_clean":"2023-08-25T09:41:13.545605+0000","last_became_active":"2023-08-25T09:41:12.514648+0000","last_became_peered":"2023-08-25T09:41:12.514648+0000","last_unstale":"2023-08-25T09:41:13.545605+0000","last_undegraded":"2023-08-25T09:41:13.545605+0000","last_fullsized":"2023-08-25T09:41:13.545605+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:20:25.833638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557181+0000","last_change":"2023-08-25T09:41:13.556973+0000","last_active":"2023-08-25T09:41:13.557181+0000","last_peered":"2023-08-25T09:41:13.557181+0000","last_clean":"2023-08-25T09:41:13.557181+0000","last_became_active":"2023-08-25T09:41:13.556096+0000","last_became_peered":"2023-08-25T09:41:13.556096+0000","last_unstale":"2023-08-25T09:41:13.557181+0000","last_undegraded":"2023-08-25T09:41:13.557181+0000","last_fullsized":"2023-08-25T09:41:13.557181+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:44.421307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.542963+0000","last_change":"2023-08-25T09:41:12.510440+0000","last_active":"2023-08-25T09:41:13.542963+0000","last_peered":"2023-08-25T09:41:13.542963+0000","last_clean":"2023-08-25T09:41:13.542963+0000","last_became_active":"2023-08-25T09:41:12.510124+0000","last_became_peered":"2023-08-25T09:41:12.510124+0000","last_unstale":"2023-08-25T09:41:13.542963+0000","last_undegraded":"2023-08-25T09:41:13.542963+0000","last_fullsized":"2023-08-25T09:41:13.542963+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:17.191428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.560053+0000","last_change":"2023-08-25T09:41:13.559867+0000","last_active":"2023-08-25T09:41:13.560053+0000","last_peered":"2023-08-25T09:41:13.560053+0000","last_clean":"2023-08-25T09:41:13.560053+0000","last_became_active":"2023-08-25T09:41:13.559094+0000","last_became_peered":"2023-08-25T09:41:13.559094+0000","last_unstale":"2023-08-25T09:41:13.560053+0000","last_undegraded":"2023-08-25T09:41:13.560053+0000","last_fullsized":"2023-08-25T09:41:13.560053+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:49.136987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"50'3","reported_seq":14,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575613+0000","last_change":"2023-08-25T09:41:12.515258+0000","last_active":"2023-08-25T09:41:13.575613+0000","last_peered":"2023-08-25T09:41:13.575613+0000","last_clean":"2023-08-25T09:41:13.575613+0000","last_became_active":"2023-08-25T09:41:12.514451+0000","last_became_peered":"2023-08-25T09:41:12.514451+0000","last_unstale":"2023-08-25T09:41:13.575613+0000","last_undegraded":"2023-08-25T09:41:13.575613+0000","last_fullsized":"2023-08-25T09:41:13.575613+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:07:23.660161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.549557+0000","last_change":"2023-08-25T09:41:13.549345+0000","last_active":"2023-08-25T09:41:13.549557+0000","last_peered":"2023-08-25T09:41:13.549557+0000","last_clean":"2023-08-25T09:41:13.549557+0000","last_became_active":"2023-08-25T09:41:13.548945+0000","last_became_peered":"2023-08-25T09:41:13.548945+0000","last_unstale":"2023-08-25T09:41:13.549557+0000","last_undegraded":"2023-08-25T09:41:13.549557+0000","last_fullsized":"2023-08-25T09:41:13.549557+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:12:13.225820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"50'6","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.577176+0000","last_change":"2023-08-25T09:41:12.519449+0000","last_active":"2023-08-25T09:41:13.577176+0000","last_peered":"2023-08-25T09:41:13.577176+0000","last_clean":"2023-08-25T09:41:13.577176+0000","last_became_active":"2023-08-25T09:41:12.518508+0000","last_became_peered":"2023-08-25T09:41:12.518508+0000","last_unstale":"2023-08-25T09:41:13.577176+0000","last_undegraded":"2023-08-25T09:41:13.577176+0000","last_fullsized":"2023-08-25T09:41:13.577176+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:08:52.365857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535637+0000","last_change":"2023-08-25T09:41:13.534458+0000","last_active":"2023-08-25T09:41:13.535637+0000","last_peered":"2023-08-25T09:41:13.535637+0000","last_clean":"2023-08-25T09:41:13.535637+0000","last_became_active":"2023-08-25T09:41:13.533826+0000","last_became_peered":"2023-08-25T09:41:13.533826+0000","last_unstale":"2023-08-25T09:41:13.535637+0000","last_undegraded":"2023-08-25T09:41:13.535637+0000","last_fullsized":"2023-08-25T09:41:13.535637+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:12.173559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575077+0000","last_change":"2023-08-25T09:41:12.511311+0000","last_active":"2023-08-25T09:41:13.575077+0000","last_peered":"2023-08-25T09:41:13.575077+0000","last_clean":"2023-08-25T09:41:13.575077+0000","last_became_active":"2023-08-25T09:41:12.510674+0000","last_became_peered":"2023-08-25T09:41:12.510674+0000","last_unstale":"2023-08-25T09:41:13.575077+0000","last_undegraded":"2023-08-25T09:41:13.575077+0000","last_fullsized":"2023-08-25T09:41:13.575077+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:55.908363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.557040+0000","last_change":"2023-08-25T09:41:13.556659+0000","last_active":"2023-08-25T09:41:13.557040+0000","last_peered":"2023-08-25T09:41:13.557040+0000","last_clean":"2023-08-25T09:41:13.557040+0000","last_became_active":"2023-08-25T09:41:13.555482+0000","last_became_peered":"2023-08-25T09:41:13.555482+0000","last_unstale":"2023-08-25T09:41:13.557040+0000","last_undegraded":"2023-08-25T09:41:13.557040+0000","last_fullsized":"2023-08-25T09:41:13.557040+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:06.469710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545920+0000","last_change":"2023-08-25T09:41:12.515872+0000","last_active":"2023-08-25T09:41:13.545920+0000","last_peered":"2023-08-25T09:41:13.545920+0000","last_clean":"2023-08-25T09:41:13.545920+0000","last_became_active":"2023-08-25T09:41:12.515111+0000","last_became_peered":"2023-08-25T09:41:12.515111+0000","last_unstale":"2023-08-25T09:41:13.545920+0000","last_undegraded":"2023-08-25T09:41:13.545920+0000","last_fullsized":"2023-08-25T09:41:13.545920+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:47:14.103836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553871+0000","last_change":"2023-08-25T09:41:13.550907+0000","last_active":"2023-08-25T09:41:13.553871+0000","last_peered":"2023-08-25T09:41:13.553871+0000","last_clean":"2023-08-25T09:41:13.553871+0000","last_became_active":"2023-08-25T09:41:13.550543+0000","last_became_peered":"2023-08-25T09:41:13.550543+0000","last_unstale":"2023-08-25T09:41:13.553871+0000","last_undegraded":"2023-08-25T09:41:13.553871+0000","last_fullsized":"2023-08-25T09:41:13.553871+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:49.136987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.532366+0000","last_change":"2023-08-25T09:41:12.503171+0000","last_active":"2023-08-25T09:41:13.532366+0000","last_peered":"2023-08-25T09:41:13.532366+0000","last_clean":"2023-08-25T09:41:13.532366+0000","last_became_active":"2023-08-25T09:41:12.501972+0000","last_became_peered":"2023-08-25T09:41:12.501972+0000","last_unstale":"2023-08-25T09:41:13.532366+0000","last_undegraded":"2023-08-25T09:41:13.532366+0000","last_fullsized":"2023-08-25T09:41:13.532366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:39:25.984514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553836+0000","last_change":"2023-08-25T09:41:13.553478+0000","last_active":"2023-08-25T09:41:13.553836+0000","last_peered":"2023-08-25T09:41:13.553836+0000","last_clean":"2023-08-25T09:41:13.553836+0000","last_became_active":"2023-08-25T09:41:13.552404+0000","last_became_peered":"2023-08-25T09:41:13.552404+0000","last_unstale":"2023-08-25T09:41:13.553836+0000","last_undegraded":"2023-08-25T09:41:13.553836+0000","last_fullsized":"2023-08-25T09:41:13.553836+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:27.131711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"50'8","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.576766+0000","last_change":"2023-08-25T09:41:12.515577+0000","last_active":"2023-08-25T09:41:13.576766+0000","last_peered":"2023-08-25T09:41:13.576766+0000","last_clean":"2023-08-25T09:41:13.576766+0000","last_became_active":"2023-08-25T09:41:12.515311+0000","last_became_peered":"2023-08-25T09:41:12.515311+0000","last_unstale":"2023-08-25T09:41:13.576766+0000","last_undegraded":"2023-08-25T09:41:13.576766+0000","last_fullsized":"2023-08-25T09:41:13.576766+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:40:00.815532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.546668+0000","last_change":"2023-08-25T09:41:13.545931+0000","last_active":"2023-08-25T09:41:13.546668+0000","last_peered":"2023-08-25T09:41:13.546668+0000","last_clean":"2023-08-25T09:41:13.546668+0000","last_became_active":"2023-08-25T09:41:13.545526+0000","last_became_peered":"2023-08-25T09:41:13.545526+0000","last_unstale":"2023-08-25T09:41:13.546668+0000","last_undegraded":"2023-08-25T09:41:13.546668+0000","last_fullsized":"2023-08-25T09:41:13.546668+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:45.077348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574045+0000","last_change":"2023-08-25T09:41:12.519312+0000","last_active":"2023-08-25T09:41:13.574045+0000","last_peered":"2023-08-25T09:41:13.574045+0000","last_clean":"2023-08-25T09:41:13.574045+0000","last_became_active":"2023-08-25T09:41:12.519017+0000","last_became_peered":"2023-08-25T09:41:12.519017+0000","last_unstale":"2023-08-25T09:41:13.574045+0000","last_undegraded":"2023-08-25T09:41:13.574045+0000","last_fullsized":"2023-08-25T09:41:13.574045+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:59:26.539321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559752+0000","last_change":"2023-08-25T09:41:13.559584+0000","last_active":"2023-08-25T09:41:13.559752+0000","last_peered":"2023-08-25T09:41:13.559752+0000","last_clean":"2023-08-25T09:41:13.559752+0000","last_became_active":"2023-08-25T09:41:13.557845+0000","last_became_peered":"2023-08-25T09:41:13.557845+0000","last_unstale":"2023-08-25T09:41:13.559752+0000","last_undegraded":"2023-08-25T09:41:13.559752+0000","last_fullsized":"2023-08-25T09:41:13.559752+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:51.965474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.574205+0000","last_change":"2023-08-25T09:41:12.511088+0000","last_active":"2023-08-25T09:41:13.574205+0000","last_peered":"2023-08-25T09:41:13.574205+0000","last_clean":"2023-08-25T09:41:13.574205+0000","last_became_active":"2023-08-25T09:41:12.510155+0000","last_became_peered":"2023-08-25T09:41:12.510155+0000","last_unstale":"2023-08-25T09:41:13.574205+0000","last_undegraded":"2023-08-25T09:41:13.574205+0000","last_fullsized":"2023-08-25T09:41:13.574205+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:07:40.030930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535027+0000","last_change":"2023-08-25T09:41:13.534418+0000","last_active":"2023-08-25T09:41:13.535027+0000","last_peered":"2023-08-25T09:41:13.535027+0000","last_clean":"2023-08-25T09:41:13.535027+0000","last_became_active":"2023-08-25T09:41:13.533797+0000","last_became_peered":"2023-08-25T09:41:13.533797+0000","last_unstale":"2023-08-25T09:41:13.535027+0000","last_undegraded":"2023-08-25T09:41:13.535027+0000","last_fullsized":"2023-08-25T09:41:13.535027+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:08:32.239164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"50'1","reported_seq":12,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.573538+0000","last_change":"2023-08-25T09:41:12.516988+0000","last_active":"2023-08-25T09:41:13.573538+0000","last_peered":"2023-08-25T09:41:13.573538+0000","last_clean":"2023-08-25T09:41:13.573538+0000","last_became_active":"2023-08-25T09:41:12.516819+0000","last_became_peered":"2023-08-25T09:41:12.516819+0000","last_unstale":"2023-08-25T09:41:13.573538+0000","last_undegraded":"2023-08-25T09:41:13.573538+0000","last_fullsized":"2023-08-25T09:41:13.573538+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:45.553977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.558911+0000","last_change":"2023-08-25T09:41:13.558785+0000","last_active":"2023-08-25T09:41:13.558911+0000","last_peered":"2023-08-25T09:41:13.558911+0000","last_clean":"2023-08-25T09:41:13.558911+0000","last_became_active":"2023-08-25T09:41:13.556586+0000","last_became_peered":"2023-08-25T09:41:13.556586+0000","last_unstale":"2023-08-25T09:41:13.558911+0000","last_undegraded":"2023-08-25T09:41:13.558911+0000","last_fullsized":"2023-08-25T09:41:13.558911+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:18:19.023697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.532731+0000","last_change":"2023-08-25T09:41:12.504818+0000","last_active":"2023-08-25T09:41:13.532731+0000","last_peered":"2023-08-25T09:41:13.532731+0000","last_clean":"2023-08-25T09:41:13.532731+0000","last_became_active":"2023-08-25T09:41:12.503878+0000","last_became_peered":"2023-08-25T09:41:12.503878+0000","last_unstale":"2023-08-25T09:41:13.532731+0000","last_undegraded":"2023-08-25T09:41:13.532731+0000","last_fullsized":"2023-08-25T09:41:13.532731+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:34.598980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.539618+0000","last_change":"2023-08-25T09:41:13.539194+0000","last_active":"2023-08-25T09:41:13.539618+0000","last_peered":"2023-08-25T09:41:13.539618+0000","last_clean":"2023-08-25T09:41:13.539618+0000","last_became_active":"2023-08-25T09:41:13.536791+0000","last_became_peered":"2023-08-25T09:41:13.536791+0000","last_unstale":"2023-08-25T09:41:13.539618+0000","last_undegraded":"2023-08-25T09:41:13.539618+0000","last_fullsized":"2023-08-25T09:41:13.539618+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:54:05.572860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575634+0000","last_change":"2023-08-25T09:41:12.512519+0000","last_active":"2023-08-25T09:41:13.575634+0000","last_peered":"2023-08-25T09:41:13.575634+0000","last_clean":"2023-08-25T09:41:13.575634+0000","last_became_active":"2023-08-25T09:41:12.511084+0000","last_became_peered":"2023-08-25T09:41:12.511084+0000","last_unstale":"2023-08-25T09:41:13.575634+0000","last_undegraded":"2023-08-25T09:41:13.575634+0000","last_fullsized":"2023-08-25T09:41:13.575634+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:44.042754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.548427+0000","last_change":"2023-08-25T09:41:13.547760+0000","last_active":"2023-08-25T09:41:13.548427+0000","last_peered":"2023-08-25T09:41:13.548427+0000","last_clean":"2023-08-25T09:41:13.548427+0000","last_became_active":"2023-08-25T09:41:13.547174+0000","last_became_peered":"2023-08-25T09:41:13.547174+0000","last_unstale":"2023-08-25T09:41:13.548427+0000","last_undegraded":"2023-08-25T09:41:13.548427+0000","last_fullsized":"2023-08-25T09:41:13.548427+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:22:33.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.540717+0000","last_change":"2023-08-25T09:41:12.518243+0000","last_active":"2023-08-25T09:41:13.540717+0000","last_peered":"2023-08-25T09:41:13.540717+0000","last_clean":"2023-08-25T09:41:13.540717+0000","last_became_active":"2023-08-25T09:41:12.517554+0000","last_became_peered":"2023-08-25T09:41:12.517554+0000","last_unstale":"2023-08-25T09:41:13.540717+0000","last_undegraded":"2023-08-25T09:41:13.540717+0000","last_fullsized":"2023-08-25T09:41:13.540717+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:33.632875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.559729+0000","last_change":"2023-08-25T09:41:13.559392+0000","last_active":"2023-08-25T09:41:13.559729+0000","last_peered":"2023-08-25T09:41:13.559729+0000","last_clean":"2023-08-25T09:41:13.559729+0000","last_became_active":"2023-08-25T09:41:13.557417+0000","last_became_peered":"2023-08-25T09:41:13.557417+0000","last_unstale":"2023-08-25T09:41:13.559729+0000","last_undegraded":"2023-08-25T09:41:13.559729+0000","last_fullsized":"2023-08-25T09:41:13.559729+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:29:03.964811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.543369+0000","last_change":"2023-08-25T09:41:12.518319+0000","last_active":"2023-08-25T09:41:13.543369+0000","last_peered":"2023-08-25T09:41:13.543369+0000","last_clean":"2023-08-25T09:41:13.543369+0000","last_became_active":"2023-08-25T09:41:12.517592+0000","last_became_peered":"2023-08-25T09:41:12.517592+0000","last_unstale":"2023-08-25T09:41:13.543369+0000","last_undegraded":"2023-08-25T09:41:13.543369+0000","last_fullsized":"2023-08-25T09:41:13.543369+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:14.186160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.546197+0000","last_change":"2023-08-25T09:41:13.546067+0000","last_active":"2023-08-25T09:41:13.546197+0000","last_peered":"2023-08-25T09:41:13.546197+0000","last_clean":"2023-08-25T09:41:13.546197+0000","last_became_active":"2023-08-25T09:41:13.545588+0000","last_became_peered":"2023-08-25T09:41:13.545588+0000","last_unstale":"2023-08-25T09:41:13.546197+0000","last_undegraded":"2023-08-25T09:41:13.546197+0000","last_fullsized":"2023-08-25T09:41:13.546197+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:44:25.714216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.537526+0000","last_change":"2023-08-25T09:41:12.516676+0000","last_active":"2023-08-25T09:41:13.537526+0000","last_peered":"2023-08-25T09:41:13.537526+0000","last_clean":"2023-08-25T09:41:13.537526+0000","last_became_active":"2023-08-25T09:41:12.516338+0000","last_became_peered":"2023-08-25T09:41:12.516338+0000","last_unstale":"2023-08-25T09:41:13.537526+0000","last_undegraded":"2023-08-25T09:41:13.537526+0000","last_fullsized":"2023-08-25T09:41:13.537526+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:42.431812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.558441+0000","last_change":"2023-08-25T09:41:13.558356+0000","last_active":"2023-08-25T09:41:13.558441+0000","last_peered":"2023-08-25T09:41:13.558441+0000","last_clean":"2023-08-25T09:41:13.558441+0000","last_became_active":"2023-08-25T09:41:13.558057+0000","last_became_peered":"2023-08-25T09:41:13.558057+0000","last_unstale":"2023-08-25T09:41:13.558441+0000","last_undegraded":"2023-08-25T09:41:13.558441+0000","last_fullsized":"2023-08-25T09:41:13.558441+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:15.199506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.543751+0000","last_change":"2023-08-25T09:41:12.512020+0000","last_active":"2023-08-25T09:41:13.543751+0000","last_peered":"2023-08-25T09:41:13.543751+0000","last_clean":"2023-08-25T09:41:13.543751+0000","last_became_active":"2023-08-25T09:41:12.511549+0000","last_became_peered":"2023-08-25T09:41:12.511549+0000","last_unstale":"2023-08-25T09:41:13.543751+0000","last_undegraded":"2023-08-25T09:41:13.543751+0000","last_fullsized":"2023-08-25T09:41:13.543751+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:47.317023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":214,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545693+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:41:13.545693+0000","last_peered":"2023-08-25T09:41:13.545693+0000","last_clean":"2023-08-25T09:41:13.545693+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:41:13.545693+0000","last_undegraded":"2023-08-25T09:41:13.545693+0000","last_fullsized":"2023-08-25T09:41:13.545693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:38:01.222062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.526726+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:41:13.526726+0000","last_peered":"2023-08-25T09:41:13.526726+0000","last_clean":"2023-08-25T09:41:13.526726+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:41:13.526726+0000","last_undegraded":"2023-08-25T09:41:13.526726+0000","last_fullsized":"2023-08-25T09:41:13.526726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:11.566878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.551555+0000","last_change":"2023-08-25T09:41:13.549732+0000","last_active":"2023-08-25T09:41:13.551555+0000","last_peered":"2023-08-25T09:41:13.551555+0000","last_clean":"2023-08-25T09:41:13.551555+0000","last_became_active":"2023-08-25T09:41:13.549414+0000","last_became_peered":"2023-08-25T09:41:13.549414+0000","last_unstale":"2023-08-25T09:41:13.551555+0000","last_undegraded":"2023-08-25T09:41:13.551555+0000","last_fullsized":"2023-08-25T09:41:13.551555+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:30:24.865893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.525283+0000","last_change":"2023-08-25T09:41:12.514623+0000","last_active":"2023-08-25T09:41:13.525283+0000","last_peered":"2023-08-25T09:41:13.525283+0000","last_clean":"2023-08-25T09:41:13.525283+0000","last_became_active":"2023-08-25T09:41:12.514316+0000","last_became_peered":"2023-08-25T09:41:12.514316+0000","last_unstale":"2023-08-25T09:41:13.525283+0000","last_undegraded":"2023-08-25T09:41:13.525283+0000","last_fullsized":"2023-08-25T09:41:13.525283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:10:57.965384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.525476+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:41:13.525476+0000","last_peered":"2023-08-25T09:41:13.525476+0000","last_clean":"2023-08-25T09:41:13.525476+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:41:13.525476+0000","last_undegraded":"2023-08-25T09:41:13.525476+0000","last_fullsized":"2023-08-25T09:41:13.525476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:08:39.671664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.553886+0000","last_change":"2023-08-25T09:41:13.553625+0000","last_active":"2023-08-25T09:41:13.553886+0000","last_peered":"2023-08-25T09:41:13.553886+0000","last_clean":"2023-08-25T09:41:13.553886+0000","last_became_active":"2023-08-25T09:41:13.553235+0000","last_became_peered":"2023-08-25T09:41:13.553235+0000","last_unstale":"2023-08-25T09:41:13.553886+0000","last_undegraded":"2023-08-25T09:41:13.553886+0000","last_fullsized":"2023-08-25T09:41:13.553886+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:50:21.885432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.539266+0000","last_change":"2023-08-25T09:41:12.510402+0000","last_active":"2023-08-25T09:41:13.539266+0000","last_peered":"2023-08-25T09:41:13.539266+0000","last_clean":"2023-08-25T09:41:13.539266+0000","last_became_active":"2023-08-25T09:41:12.509222+0000","last_became_peered":"2023-08-25T09:41:12.509222+0000","last_unstale":"2023-08-25T09:41:13.539266+0000","last_undegraded":"2023-08-25T09:41:13.539266+0000","last_fullsized":"2023-08-25T09:41:13.539266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:47.279756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":74,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.528974+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:41:13.528974+0000","last_peered":"2023-08-25T09:41:13.528974+0000","last_clean":"2023-08-25T09:41:13.528974+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:41:13.528974+0000","last_undegraded":"2023-08-25T09:41:13.528974+0000","last_fullsized":"2023-08-25T09:41:13.528974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:05.710926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555453+0000","last_change":"2023-08-25T09:41:13.555310+0000","last_active":"2023-08-25T09:41:13.555453+0000","last_peered":"2023-08-25T09:41:13.555453+0000","last_clean":"2023-08-25T09:41:13.555453+0000","last_became_active":"2023-08-25T09:41:13.554281+0000","last_became_peered":"2023-08-25T09:41:13.554281+0000","last_unstale":"2023-08-25T09:41:13.555453+0000","last_undegraded":"2023-08-25T09:41:13.555453+0000","last_fullsized":"2023-08-25T09:41:13.555453+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:40:49.031922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"50'2","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.575381+0000","last_change":"2023-08-25T09:41:12.513317+0000","last_active":"2023-08-25T09:41:13.575381+0000","last_peered":"2023-08-25T09:41:13.575381+0000","last_clean":"2023-08-25T09:41:13.575381+0000","last_became_active":"2023-08-25T09:41:12.513057+0000","last_became_peered":"2023-08-25T09:41:12.513057+0000","last_unstale":"2023-08-25T09:41:13.575381+0000","last_undegraded":"2023-08-25T09:41:13.575381+0000","last_fullsized":"2023-08-25T09:41:13.575381+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:48:06.788767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.526306+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:41:13.526306+0000","last_peered":"2023-08-25T09:41:13.526306+0000","last_clean":"2023-08-25T09:41:13.526306+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:41:13.526306+0000","last_undegraded":"2023-08-25T09:41:13.526306+0000","last_fullsized":"2023-08-25T09:41:13.526306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:43.216828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.556193+0000","last_change":"2023-08-25T09:41:13.556112+0000","last_active":"2023-08-25T09:41:13.556193+0000","last_peered":"2023-08-25T09:41:13.556193+0000","last_clean":"2023-08-25T09:41:13.556193+0000","last_became_active":"2023-08-25T09:41:13.555863+0000","last_became_peered":"2023-08-25T09:41:13.555863+0000","last_unstale":"2023-08-25T09:41:13.556193+0000","last_undegraded":"2023-08-25T09:41:13.556193+0000","last_fullsized":"2023-08-25T09:41:13.556193+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:43:51.965474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.531117+0000","last_change":"2023-08-25T09:41:12.510171+0000","last_active":"2023-08-25T09:41:13.531117+0000","last_peered":"2023-08-25T09:41:13.531117+0000","last_clean":"2023-08-25T09:41:13.531117+0000","last_became_active":"2023-08-25T09:41:12.509627+0000","last_became_peered":"2023-08-25T09:41:12.509627+0000","last_unstale":"2023-08-25T09:41:13.531117+0000","last_undegraded":"2023-08-25T09:41:13.531117+0000","last_fullsized":"2023-08-25T09:41:13.531117+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:47.804783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":62,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.544313+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:41:13.544313+0000","last_peered":"2023-08-25T09:41:13.544313+0000","last_clean":"2023-08-25T09:41:13.544313+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:41:13.544313+0000","last_undegraded":"2023-08-25T09:41:13.544313+0000","last_fullsized":"2023-08-25T09:41:13.544313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:26:37.473166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.555977+0000","last_change":"2023-08-25T09:41:13.554744+0000","last_active":"2023-08-25T09:41:13.555977+0000","last_peered":"2023-08-25T09:41:13.555977+0000","last_clean":"2023-08-25T09:41:13.555977+0000","last_became_active":"2023-08-25T09:41:13.554517+0000","last_became_peered":"2023-08-25T09:41:13.554517+0000","last_unstale":"2023-08-25T09:41:13.555977+0000","last_undegraded":"2023-08-25T09:41:13.555977+0000","last_fullsized":"2023-08-25T09:41:13.555977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:11:02.046065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.545449+0000","last_change":"2023-08-25T09:41:12.513736+0000","last_active":"2023-08-25T09:41:13.545449+0000","last_peered":"2023-08-25T09:41:13.545449+0000","last_clean":"2023-08-25T09:41:13.545449+0000","last_became_active":"2023-08-25T09:41:12.513456+0000","last_became_peered":"2023-08-25T09:41:12.513456+0000","last_unstale":"2023-08-25T09:41:13.545449+0000","last_undegraded":"2023-08-25T09:41:13.545449+0000","last_fullsized":"2023-08-25T09:41:13.545449+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:44:00.871853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.524828+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:41:13.524828+0000","last_peered":"2023-08-25T09:41:13.524828+0000","last_clean":"2023-08-25T09:41:13.524828+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:41:13.524828+0000","last_undegraded":"2023-08-25T09:41:13.524828+0000","last_fullsized":"2023-08-25T09:41:13.524828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:02.023908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.554633+0000","last_change":"2023-08-25T09:41:13.554492+0000","last_active":"2023-08-25T09:41:13.554633+0000","last_peered":"2023-08-25T09:41:13.554633+0000","last_clean":"2023-08-25T09:41:13.554633+0000","last_became_active":"2023-08-25T09:41:13.553919+0000","last_became_peered":"2023-08-25T09:41:13.553919+0000","last_unstale":"2023-08-25T09:41:13.554633+0000","last_undegraded":"2023-08-25T09:41:13.554633+0000","last_fullsized":"2023-08-25T09:41:13.554633+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:32.208562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.538241+0000","last_change":"2023-08-25T09:41:12.510574+0000","last_active":"2023-08-25T09:41:13.538241+0000","last_peered":"2023-08-25T09:41:13.538241+0000","last_clean":"2023-08-25T09:41:13.538241+0000","last_became_active":"2023-08-25T09:41:12.508978+0000","last_became_peered":"2023-08-25T09:41:12.508978+0000","last_unstale":"2023-08-25T09:41:13.538241+0000","last_undegraded":"2023-08-25T09:41:13.538241+0000","last_fullsized":"2023-08-25T09:41:13.538241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:12.710316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":60,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.542931+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:41:13.542931+0000","last_peered":"2023-08-25T09:41:13.542931+0000","last_clean":"2023-08-25T09:41:13.542931+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:41:13.542931+0000","last_undegraded":"2023-08-25T09:41:13.542931+0000","last_fullsized":"2023-08-25T09:41:13.542931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:12:54.008847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.535072+0000","last_change":"2023-08-25T09:41:13.534896+0000","last_active":"2023-08-25T09:41:13.535072+0000","last_peered":"2023-08-25T09:41:13.535072+0000","last_clean":"2023-08-25T09:41:13.535072+0000","last_became_active":"2023-08-25T09:41:13.534272+0000","last_became_peered":"2023-08-25T09:41:13.534272+0000","last_unstale":"2023-08-25T09:41:13.535072+0000","last_undegraded":"2023-08-25T09:41:13.535072+0000","last_fullsized":"2023-08-25T09:41:13.535072+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:12.482232+0000","last_clean_scrub_stamp":"2023-08-25T09:41:12.482232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:36:13.288869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.542447+0000","last_change":"2023-08-25T09:41:12.516335+0000","last_active":"2023-08-25T09:41:13.542447+0000","last_peered":"2023-08-25T09:41:13.542447+0000","last_clean":"2023-08-25T09:41:13.542447+0000","last_became_active":"2023-08-25T09:41:12.516027+0000","last_became_peered":"2023-08-25T09:41:12.516027+0000","last_unstale":"2023-08-25T09:41:13.542447+0000","last_undegraded":"2023-08-25T09:41:13.542447+0000","last_fullsized":"2023-08-25T09:41:13.542447+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:41:11.468886+0000","last_clean_scrub_stamp":"2023-08-25T09:41:11.468886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:08:25.708535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":70,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-08-25T09:41:13.525681+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:41:13.525681+0000","last_peered":"2023-08-25T09:41:13.525681+0000","last_clean":"2023-08-25T09:41:13.525681+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:41:13.525681+0000","last_undegraded":"2023-08-25T09:41:13.525681+0000","last_fullsized":"2023-08-25T09:41:13.525681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:38.069632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195064,"statfs":{"total":200000000,"available":199745870,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.038,"15min":1.0149999999999999},"min":{"1min":0.81299999999999994,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.014,"15min":1.004},"min":{"1min":0.76200000000000001,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.1080000000000001,"15min":1.103},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5700000000000001,"5min":1.651,"15min":1.651},"last":1.091},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.0880000000000001,"15min":1.0609999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.0509999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.069,"15min":1.05},"min":{"1min":0.90400000000000003,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.745},{"interface":"front","average":{"1min":1.254,"5min":1.2030000000000001,"15min":1.1950000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.02}]},{"osd":3,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.159,"15min":1.1399999999999999},"min":{"1min":0.75800000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":1.456,"5min":1.3320000000000001,"15min":1.3120000000000001},"min":{"1min":0.94599999999999995,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1459999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.2609999999999999,"15min":1.2529999999999999},"min":{"1min":0.86499999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.379,"5min":1.2909999999999999,"15min":1.2769999999999999},"min":{"1min":0.96099999999999997,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.874}]},{"osd":5,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.5269999999999999,"5min":1.3380000000000001,"15min":1.306},"min":{"1min":0.78000000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.369,"15min":1.3540000000000001},"min":{"1min":0.83599999999999997,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":2.0179999999999998,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.1200000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.415,"15min":1.401},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0859999999999999,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.3919999999999999,"15min":1.3720000000000001},"min":{"1min":1.0149999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9890000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.89900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195063,"statfs":{"total":200000000,"available":199745344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.133,"15min":1.1479999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.508,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.115,"5min":1.121,"15min":1.1220000000000001},"min":{"1min":0.70199999999999996,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.8200000000000001,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.91700000000000004}]},{"osd":1,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.179,"15min":1.1779999999999999},"min":{"1min":0.76000000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5680000000000001,"5min":1.927,"15min":1.927},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.093,"15min":1.087},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.995}]},{"osd":2,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.085,"15min":1.0780000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5529999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.149,"15min":1.133},"min":{"1min":0.83399999999999996,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.262}]},{"osd":3,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.208,"15min":1.1910000000000001},"min":{"1min":1.0429999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.2609999999999999,"15min":1.252},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.476}]},{"osd":4,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.2490000000000001,"15min":1.2210000000000001},"min":{"1min":0.70899999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.484,"5min":2.484,"15min":2.484},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.302,"15min":1.29},"min":{"1min":0.93100000000000005,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1499999999999999,"5min":2.1499999999999999,"15min":2.1499999999999999},"last":1.512}]},{"osd":5,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.274,"15min":1.262},"min":{"1min":0.91900000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.0049999999999999,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.3540000000000001,"15min":1.3300000000000001},"min":{"1min":0.91300000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.353}]},{"osd":7,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.474,"15min":1.4690000000000001},"min":{"1min":0.96599999999999997,"5min":0.622,"15min":0.622},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.651},{"interface":"front","average":{"1min":1.5149999999999999,"5min":1.48,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.726}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":826,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194486,"statfs":{"total":200000000,"available":199153856,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.1279999999999999,"15min":1.1259999999999999},"min":{"1min":0.74199999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.998},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.974,"5min":1.974,"15min":1.974},"last":1.1020000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.1870000000000001,"15min":1.2090000000000001},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4279999999999999,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.081,"15min":1.085},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.4059999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.417}]},{"osd":2,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1539999999999999,"15min":1.1579999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.4810000000000001,"5min":1.788,"15min":1.788},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1930000000000001,"15min":1.1990000000000001},"min":{"1min":0.79300000000000004,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.4450000000000001,"5min":1.962,"15min":1.962},"last":1.446}]},{"osd":3,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.244,"15min":1.262},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6599999999999999,"5min":1.921,"15min":1.921},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"min":{"1min":0.747,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7949999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.6120000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.29,"15min":1.282},"min":{"1min":0.92000000000000004,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7150000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.51},{"interface":"front","average":{"1min":1.26,"5min":1.3600000000000001,"15min":1.3759999999999999},"min":{"1min":0.876,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0270000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.3560000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.413,"15min":1.423},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.696,"5min":2.2829999999999999,"15min":2.2829999999999999},"last":1.8109999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.3879999999999999,"15min":1.3759999999999999},"min":{"1min":0.85399999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":1.724,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.679}]},{"osd":7,"last update":"Fri Aug 25 09:40:40 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.4179999999999999,"15min":1.425},"min":{"1min":0.82699999999999996,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.508,"5min":1.5609999999999999,"15min":1.5700000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.857,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.911}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":241,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195071,"statfs":{"total":200000000,"available":199753054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.143,"15min":1.139},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.22,"5min":1.115,"15min":1.097},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.89200000000000002}]},{"osd":1,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.103,"15min":1.111},"min":{"1min":0.78200000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.6000000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.288},{"interface":"front","average":{"1min":1.149,"5min":1.413,"15min":1.4570000000000001},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.9379999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.026}]},{"osd":2,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2969999999999999,"15min":1.331},"min":{"1min":0.85199999999999998,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":1.482,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.1539999999999999,"15min":1.137},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.224}]},{"osd":3,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.218,"15min":1.206},"min":{"1min":0.82699999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.53},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3680000000000001,"15min":1.3720000000000001},"min":{"1min":1.083,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":1.911,"5min":1.911,"15min":1.911},"last":1.4370000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.292,"15min":1.294},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.0640000000000001,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.216,"15min":1.2110000000000001},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.6379999999999999,"5min":1.7430000000000001,"15min":1.7430000000000001},"last":1.381}]},{"osd":6,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.425,"15min":1.4359999999999999},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":1.7709999999999999,"5min":2.863,"15min":2.863},"last":1.569},{"interface":"front","average":{"1min":1.544,"5min":1.5609999999999999,"15min":1.5640000000000001},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":2.1339999999999999,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.4910000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:40:42 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.375,"15min":1.359},"min":{"1min":0.93799999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.583,"15min":1.591},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":1.9850000000000001,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.5149999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195064,"statfs":{"total":200000000,"available":199745836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81899999999999995,"15min":0.82699999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.3660000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.86599999999999999,"15min":0.86899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.874},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.93600000000000005,"15min":0.95399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.292,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.83499999999999996}]},{"osd":2,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.83399999999999996,"15min":0.82799999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.081,"15min":1.101},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9710000000000001,"5min":3.399,"15min":3.399},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.1659999999999999,"15min":1.173},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.173,"15min":1.179},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5620000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.1599999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.286,"15min":1.3},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0139999999999998,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.395},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.2450000000000001,"15min":1.256},"min":{"1min":0.84899999999999998,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.544,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.198}]},{"osd":6,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.226,"15min":1.2230000000000001},"min":{"1min":0.74099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.3819999999999999,"15min":1.3999999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.9159999999999999,"5min":3.496,"15min":3.496},"last":1.3340000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:40:41 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.329,"15min":1.341},"min":{"1min":0.871,"5min":0.871,"15min":0.871},"max":{"1min":1.921,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2609999999999999,"15min":1.2490000000000001},"min":{"1min":1.0389999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.3660000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":241,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195071,"statfs":{"total":200000000,"available":199753054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82499999999999996,"15min":0.82199999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.46,"5min":2.383,"15min":2.383},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":1.0569999999999999,"15min":1.1040000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.373,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.92500000000000004}]},{"osd":1,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.96999999999999997,"15min":0.98599999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2370000000000001,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0609999999999999,"15min":1.0649999999999999},"min":{"1min":0.505,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.677,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.0800000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":1.099,"15min":1.1339999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.343,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.091,"15min":1.105},"min":{"1min":0.63100000000000001,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4490000000000001,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":0.98499999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.395,"15min":1.4259999999999999},"min":{"1min":0.87,"5min":0.87,"15min":0.87},"max":{"1min":1.5760000000000001,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.343,"15min":1.359},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.6930000000000001,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.1930000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.385,"15min":1.415},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"max":{"1min":1.6839999999999999,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.163},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.393,"15min":1.405},"min":{"1min":0.93999999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.635,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.425,"15min":1.4350000000000001},"min":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.766,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.4770000000000001,"15min":1.5109999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.6419999999999999,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:40:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.456,"15min":1.4730000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.881,"5min":2.968,"15min":2.968},"last":1.363},{"interface":"front","average":{"1min":1.321,"5min":1.5009999999999999,"15min":1.5309999999999999},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.778,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.415}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":824,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194487,"statfs":{"total":200000000,"available":199155488,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.51700000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.1850000000000001,"5min":1.627,"15min":1.627},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.77000000000000002,"15min":0.76700000000000002},"min":{"1min":0.46999999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2190000000000001,"5min":1.296,"15min":1.296},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.57699999999999996,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.282,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.997,"15min":0.99299999999999999},"min":{"1min":0.60299999999999998,"5min":0.372,"15min":0.372},"max":{"1min":1.6020000000000001,"5min":1.7,"15min":1.7},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.91100000000000003,"15min":0.90800000000000003},"min":{"1min":0.503,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":0.745}]},{"osd":4,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0620000000000001,"15min":1.056},"min":{"1min":0.75,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.25,"5min":1.1870000000000001,"15min":1.177},"min":{"1min":0.85999999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.6020000000000001,"5min":1.919,"15min":1.919},"last":0.629}]},{"osd":5,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.169,"15min":1.165},"min":{"1min":0.80700000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.6459999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.133,"5min":1.141,"15min":1.143},"min":{"1min":0.83099999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.496,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.26,"15min":1.258},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"max":{"1min":1.5349999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2549999999999999,"15min":1.2589999999999999},"min":{"1min":0.77400000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.728,"5min":1.847,"15min":1.847},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Aug 25 09:40:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":1.708,"5min":2.3140000000000001,"15min":2.3140000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.357,"5min":1.2849999999999999,"15min":1.2729999999999999},"min":{"1min":0.93400000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.7350000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.81899999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":248,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195063,"statfs":{"total":200000000,"available":199745344,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.3260000000000001,"5min":1.361,"15min":1.361},"last":1.2},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.92100000000000004,"15min":0.94099999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.3779999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.91800000000000004,"15min":0.93300000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":1.0840000000000001,"15min":1.121},"min":{"1min":0.47799999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3120000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.99199999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.88600000000000001,"15min":0.90000000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.095,"5min":1.387,"15min":1.387},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.099,"15min":1.1259999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4970000000000001,"5min":1.869,"15min":1.869},"last":1.325}]},{"osd":4,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.21,"15min":1.2290000000000001},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.4379999999999999,"5min":1.704,"15min":1.704},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3320000000000001,"15min":1.3520000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7090000000000001,"5min":1.948,"15min":1.948},"last":1.45}]},{"osd":5,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.3400000000000001,"15min":1.3859999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.431,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.151},{"interface":"front","average":{"1min":1.105,"5min":1.2809999999999999,"15min":1.3109999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5920000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.1220000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.321,"15min":1.349},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6970000000000001,"5min":2.044,"15min":2.044},"last":1.395},{"interface":"front","average":{"1min":1.202,"5min":1.2809999999999999,"15min":1.294},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.764,"5min":1.764,"15min":1.764},"last":1.262}]},{"osd":7,"last update":"Fri Aug 25 09:40:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3779999999999999,"15min":1.401},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.708,"5min":2.121,"15min":2.121},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.3029999999999999,"15min":1.3200000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.5469999999999999,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.282}]}]}],"pool_statfs":[]}} 2023-08-25T09:41:18.494 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:41:18.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:41:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.758+0000 7ff68cda4640 1 Processor -- start 2023-08-25T09:41:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.758+0000 7ff68cda4640 1 -- start start 2023-08-25T09:41:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff68cda4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff68815ae50 0x7ff6880c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff68cda4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff6880cdc20 0x7ff688069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff68cda4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880cc5f0 con 0x7ff6880ce060 2023-08-25T09:41:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff68cda4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff688070650 con 0x7ff6880cdc20 2023-08-25T09:41:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff68cda4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff688070930 con 0x7ff68815ae50 2023-08-25T09:41:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff686d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff6880ce060 0x7ff6880ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53498/0 (socket says 172.21.15.70:53498) 2023-08-25T09:41:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff686d9d640 1 -- 172.21.15.70:0/3681563052 learned_addr learned my addr 172.21.15.70:0/3681563052 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.760+0000 7ff685d9b640 1 --2- 172.21.15.70:0/3681563052 >> v2:172.21.15.70:3300/0 conn(0x7ff6880cdc20 0x7ff688069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff68659c640 1 --2- 172.21.15.70:0/3681563052 >> v2:172.21.15.155:3300/0 conn(0x7ff68815ae50 0x7ff6880c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff68559a640 1 -- 172.21.15.70:0/3681563052 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409987556 0 0) 0x7ff6880cc5f0 con 0x7ff6880ce060 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff68559a640 1 -- 172.21.15.70:0/3681563052 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c003520 con 0x7ff6880ce060 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff685d9b640 1 -- 172.21.15.70:0/3681563052 >> v2:172.21.15.155:3300/0 conn(0x7ff68815ae50 msgr2=0x7ff6880c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff685d9b640 1 --2- 172.21.15.70:0/3681563052 >> v2:172.21.15.155:3300/0 conn(0x7ff68815ae50 0x7ff6880c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff685d9b640 1 -- 172.21.15.70:0/3681563052 >> v1:172.21.15.70:6789/0 conn(0x7ff6880ce060 legacy=0x7ff6880ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.761+0000 7ff685d9b640 1 -- 172.21.15.70:0/3681563052 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688070c10 con 0x7ff6880cdc20 2023-08-25T09:41:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.762+0000 7ff685d9b640 1 --2- 172.21.15.70:0/3681563052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6880cdc20 0x7ff688069850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff6880b1210 tx=0x7ff67402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a186bbf2a3290b0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:18.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.762+0000 7ff68559a640 1 -- 172.21.15.70:0/3681563052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674030610 con 0x7ff6880cdc20 2023-08-25T09:41:18.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.762+0000 7ff68559a640 1 -- 172.21.15.70:0/3681563052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674030770 con 0x7ff6880cdc20 2023-08-25T09:41:18.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.762+0000 7ff68559a640 1 -- 172.21.15.70:0/3681563052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674030b00 con 0x7ff6880cdc20 2023-08-25T09:41:18.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 -- 172.21.15.70:0/3681563052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6880cdc20 msgr2=0x7ff688069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 --2- 172.21.15.70:0/3681563052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6880cdc20 0x7ff688069850 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff6880b1210 tx=0x7ff67402b960 comp rx=0 tx=0).stop 2023-08-25T09:41:18.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 -- 172.21.15.70:0/3681563052 shutdown_connections 2023-08-25T09:41:18.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 --2- 172.21.15.70:0/3681563052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6880cdc20 0x7ff688069850 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 --2- 172.21.15.70:0/3681563052 >> v2:172.21.15.155:3300/0 conn(0x7ff68815ae50 0x7ff6880c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 -- 172.21.15.70:0/3681563052 >> 172.21.15.70:0/3681563052 conn(0x7ff6880cb0d0 msgr2=0x7ff6880cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:18.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 -- 172.21.15.70:0/3681563052 shutdown_connections 2023-08-25T09:41:18.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.763+0000 7ff68cda4640 1 -- 172.21.15.70:0/3681563052 wait complete. 2023-08-25T09:41:18.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.765+0000 7ff68cda4640 1 Processor -- start 2023-08-25T09:41:18.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.765+0000 7ff68cda4640 1 -- start start 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.766+0000 7ff68cda4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6880cdc20 0x7ff688158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.766+0000 7ff68cda4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6880ce060 0x7ff6880c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.766+0000 7ff68cda4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 0x7ff6880c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.766+0000 7ff68cda4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880b04b0 con 0x7ff68815ae50 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.766+0000 7ff68cda4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880c6450 con 0x7ff6880ce060 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.766+0000 7ff68cda4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6881593e0 con 0x7ff6880cdc20 2023-08-25T09:41:18.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 0x7ff6880c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 0x7ff6880c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50166/0 (socket says 172.21.15.70:50166) 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 -- 172.21.15.70:0/614688930 learned_addr learned my addr 172.21.15.70:0/614688930 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff68659c640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6880cdc20 0x7ff688158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff685d9b640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6880ce060 0x7ff6880c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 -- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6880cdc20 msgr2=0x7ff688158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6880cdc20 0x7ff688158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 -- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6880ce060 msgr2=0x7ff6880c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:18.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6880ce060 0x7ff6880c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:18.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 -- 172.21.15.70:0/614688930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff674009400 con 0x7ff68815ae50 2023-08-25T09:41:18.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.767+0000 7ff686d9d640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 0x7ff6880c8250 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff68000cb40 tx=0x7ff68000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:18.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.768+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000aca0 con 0x7ff68815ae50 2023-08-25T09:41:18.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.768+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68000ae00 con 0x7ff68815ae50 2023-08-25T09:41:18.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.768+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680012620 con 0x7ff68815ae50 2023-08-25T09:41:18.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.768+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880c66f0 con 0x7ff68815ae50 2023-08-25T09:41:18.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.768+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6880c6cb0 con 0x7ff68815ae50 2023-08-25T09:41:18.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.769+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688066fb0 con 0x7ff68815ae50 2023-08-25T09:41:18.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.770+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff680007070 con 0x7ff68815ae50 2023-08-25T09:41:18.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.771+0000 7ff67effd640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff67006a050 0x7ff67006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:18.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.771+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff6800885a0 con 0x7ff68815ae50 2023-08-25T09:41:18.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.771+0000 7ff68659c640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff67006a050 0x7ff67006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:18.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.772+0000 7ff68659c640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff67006a050 0x7ff67006c510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff678006fd0 tx=0x7ff678008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:18.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:18.775+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff680053c80 con 0x7ff68815ae50 2023-08-25T09:41:19.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.038+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff68815e570 con 0x7ff68815ae50 2023-08-25T09:41:19.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.046+0000 7ff67effd640 1 -- 172.21.15.70:0/614688930 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff6800572a0 con 0x7ff68815ae50 2023-08-25T09:41:19.049 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:41:19.049 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:41:19.049 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:41:19.049 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:41:19.050 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:41:19.050 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:41:19.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff67006a050 msgr2=0x7ff67006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff67006a050 0x7ff67006c510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff678006fd0 tx=0x7ff678008040 comp rx=0 tx=0).stop 2023-08-25T09:41:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 msgr2=0x7ff6880c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 0x7ff6880c8250 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff68000cb40 tx=0x7ff68000a060 comp rx=0 tx=0).stop 2023-08-25T09:41:19.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 shutdown_connections 2023-08-25T09:41:19.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff67006a050 0x7ff67006c510 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff68815ae50 0x7ff6880c8250 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff6880ce060 0x7ff6880c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 --2- 172.21.15.70:0/614688930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6880cdc20 0x7ff688158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 >> 172.21.15.70:0/614688930 conn(0x7ff6880cb0d0 msgr2=0x7ff6880c3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:19.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 shutdown_connections 2023-08-25T09:41:19.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.049+0000 7ff68cda4640 1 -- 172.21.15.70:0/614688930 wait complete. 2023-08-25T09:41:19.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:41:19.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.274+0000 7f18639f3640 1 Processor -- start 2023-08-25T09:41:19.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.275+0000 7f18639f3640 1 -- start start 2023-08-25T09:41:19.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f18639f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f185c15e830 0x7f185c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f18639f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 0x7f185c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f18639f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c0b1b60 con 0x7f185c06b2b0 2023-08-25T09:41:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f18639f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185c06bd80 con 0x7f185c15e830 2023-08-25T09:41:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f18639f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c06b7c0 con 0x7f185c15ec10 2023-08-25T09:41:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f1860f8e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f185c06b2b0 0x7f185c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49564/0 (socket says 172.21.15.70:49564) 2023-08-25T09:41:19.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.276+0000 7f1860f8e640 1 -- 172.21.15.70:0/4245346116 learned_addr learned my addr 172.21.15.70:0/4245346116 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:19.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f1861f90640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 0x7f185c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353243656 0 0) 0x7f185c0b1b60 con 0x7f185c06b2b0 2023-08-25T09:41:19.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f186178f640 1 --2- 172.21.15.70:0/4245346116 >> v2:172.21.15.155:3300/0 conn(0x7f185c15e830 0x7f185c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1844003520 con 0x7f185c06b2b0 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1242057193 0 0) 0x7f1844003520 con 0x7f185c06b2b0 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 msgr2=0x7f185c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 0x7f185c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 >> v2:172.21.15.155:3300/0 conn(0x7f185c15e830 msgr2=0x7f185c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 --2- 172.21.15.70:0/4245346116 >> v2:172.21.15.155:3300/0 conn(0x7f185c15e830 0x7f185c06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c06baa0 con 0x7f185c06b2b0 2023-08-25T09:41:19.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1850001670 con 0x7f185c06b2b0 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 >> v1:172.21.15.70:6789/0 conn(0x7f185c06b2b0 legacy=0x7f185c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1844003970 0x7f1844003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18440047c0 0x7f1844004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 0x7f1844005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c06bd80 con 0x7f18440047c0 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c06b7c0 con 0x7f1844003970 2023-08-25T09:41:19.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.277+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184400ba60 con 0x7f185c15ec10 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f1860f8e640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18440047c0 0x7f1844004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f186178f640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1844003970 0x7f1844003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f1860f8e640 1 -- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 msgr2=0x7f1844005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f1860f8e640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c15ec10 0x7f1844005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f1860f8e640 1 -- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1844003970 msgr2=0x7f1844003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f1860f8e640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1844003970 0x7f1844003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.278+0000 7f1860f8e640 1 -- 172.21.15.70:0/4245346116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1844003520 con 0x7f18440047c0 2023-08-25T09:41:19.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.279+0000 7f1860f8e640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18440047c0 0x7f1844004c40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1850001eb0 tx=0x7f18440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3ed072a4d38841 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:19.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.279+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850005ad0 con 0x7f18440047c0 2023-08-25T09:41:19.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.279+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850005c30 con 0x7f18440047c0 2023-08-25T09:41:19.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.279+0000 7f185bfff640 1 -- 172.21.15.70:0/4245346116 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185002c890 con 0x7f18440047c0 2023-08-25T09:41:19.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f18639f3640 1 -- 172.21.15.70:0/4245346116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18440047c0 msgr2=0x7f1844004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f18639f3640 1 --2- 172.21.15.70:0/4245346116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18440047c0 0x7f1844004c40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1850001eb0 tx=0x7f18440057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f1861f90640 1 -- 172.21.15.70:0/4245346116 reap_dead start 2023-08-25T09:41:19.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f18639f3640 1 -- 172.21.15.70:0/4245346116 shutdown_connections 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f18639f3640 1 -- 172.21.15.70:0/4245346116 >> 172.21.15.70:0/4245346116 conn(0x7f185c069850 msgr2=0x7f185c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f18639f3640 1 -- 172.21.15.70:0/4245346116 shutdown_connections 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.280+0000 7f18639f3640 1 -- 172.21.15.70:0/4245346116 wait complete. 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.282+0000 7f18639f3640 1 Processor -- start 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.282+0000 7f18639f3640 1 -- start start 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f18639f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18440047c0 0x7f185c13f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f18639f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 0x7f185c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f18639f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c1f1ff0 0x7f185c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f18639f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c156540 con 0x7f185c156010 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f18639f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c156820 con 0x7f18440047c0 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f18639f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c156b00 con 0x7f185c1f1ff0 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 0x7f185c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 0x7f185c1575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39164/0 (socket says 172.21.15.70:39164) 2023-08-25T09:41:19.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 -- 172.21.15.70:0/3340468143 learned_addr learned my addr 172.21.15.70:0/3340468143 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 -- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c1f1ff0 msgr2=0x7f185c157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f186178f640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18440047c0 0x7f185c13f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1861f90640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c1f1ff0 0x7f185c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c1f1ff0 0x7f185c157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 -- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18440047c0 msgr2=0x7f185c13f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18440047c0 0x7f185c13f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.283+0000 7f1860f8e640 1 -- 172.21.15.70:0/3340468143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c15ca20 con 0x7f185c156010 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.284+0000 7f1860f8e640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 0x7f185c1575e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f185002c430 tx=0x7f185c0b1ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.284+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850039070 con 0x7f185c156010 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.284+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850001760 con 0x7f185c156010 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.284+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185003c040 con 0x7f185c156010 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.284+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c156de0 con 0x7f185c156010 2023-08-25T09:41:19.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.284+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c13f540 con 0x7f185c156010 2023-08-25T09:41:19.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.285+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c04e420 con 0x7f185c156010 2023-08-25T09:41:19.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.286+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1850032070 con 0x7f185c156010 2023-08-25T09:41:19.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.287+0000 7f1859ffb640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f184806a080 0x7f184806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:19.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.287+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18500a9960 con 0x7f185c156010 2023-08-25T09:41:19.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.287+0000 7f186178f640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f184806a080 0x7f184806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:19.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.288+0000 7f186178f640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f184806a080 0x7f184806c540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f184c006fd0 tx=0x7f184c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:19.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.292+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1850074fb0 con 0x7f185c156010 2023-08-25T09:41:19.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.520+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f185c06c620 con 0x7f185c156010 2023-08-25T09:41:19.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.522+0000 7f1859ffb640 1 -- 172.21.15.70:0/3340468143 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f18500785d0 con 0x7f185c156010 2023-08-25T09:41:19.524 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:19.525 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:41:19.525 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:41:19.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f184806a080 msgr2=0x7f184806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f184806a080 0x7f184806c540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f184c006fd0 tx=0x7f184c008040 comp rx=0 tx=0).stop 2023-08-25T09:41:19.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 msgr2=0x7f185c1575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:19.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 0x7f185c1575e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f185002c430 tx=0x7f185c0b1ad0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 shutdown_connections 2023-08-25T09:41:19.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f184806a080 0x7f184806c540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f185c1f1ff0 0x7f185c157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f185c156010 0x7f185c1575e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 --2- 172.21.15.70:0/3340468143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18440047c0 0x7f185c13f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:19.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 >> 172.21.15.70:0/3340468143 conn(0x7f185c069850 msgr2=0x7f185c06f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:19.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 shutdown_connections 2023-08-25T09:41:19.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:19.524+0000 7f18639f3640 1 -- 172.21.15.70:0/3340468143 wait complete. 2023-08-25T09:41:19.948 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:41:12.986069+0000', 'modified': '2023-08-25T09:41:16.578011+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5835}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5835': {'gid': 5835, 'name': 'a', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6837/3881965592', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6835', 'nonce': 3881965592}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 3881965592}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:41:19.949 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:41:19.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T09:41:20.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.172+0000 7ff8d2a5d640 1 Processor -- start 2023-08-25T09:41:20.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.172+0000 7ff8d2a5d640 1 -- start start 2023-08-25T09:41:20.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d2a5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8cc0b7df0 0x7ff8cc070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d2a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 0x7ff8cc0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d2a5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc0b81d0 con 0x7ff8cc15cf00 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d2a5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc1604f0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d2a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc15b250 con 0x7ff8cc0cf210 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d1a5b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8cc0b7df0 0x7ff8cc070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d1a5b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8cc0b7df0 0x7ff8cc070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39180/0 (socket says 172.21.15.70:39180) 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2695612298 learned_addr learned my addr 172.21.15.70:0/2695612298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.174+0000 7ff8d125a640 1 --1- 172.21.15.70:0/2695612298 >> v1:172.21.15.70:6789/0 conn(0x7ff8cc15cf00 0x7ff8cc15ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49572/0 (socket says 172.21.15.70:49572) 2023-08-25T09:41:20.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.175+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2695612298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 msgr2=0x7ff8cc0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.175+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2695612298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 0x7ff8cc0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.175+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2695612298 >> v1:172.21.15.70:6789/0 conn(0x7ff8cc15cf00 legacy=0x7ff8cc15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.175+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2695612298 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc0cfba0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.176+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2695612298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc070880 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009870 tx=0x7ff8c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4aa59a7e54bf91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.176+0000 7ff8d0a59640 1 -- 172.21.15.70:0/2695612298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8030840 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.176+0000 7ff8d0a59640 1 -- 172.21.15.70:0/2695612298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c80309a0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.176+0000 7ff8d0a59640 1 -- 172.21.15.70:0/2695612298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8030d30 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2695612298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 msgr2=0x7ff8cc070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2695612298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc070880 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009870 tx=0x7ff8c802b980 comp rx=0 tx=0).stop 2023-08-25T09:41:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2695612298 shutdown_connections 2023-08-25T09:41:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2695612298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 0x7ff8cc0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2695612298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc070880 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2695612298 >> 172.21.15.70:0/2695612298 conn(0x7ff8cc0b43a0 msgr2=0x7ff8cc0b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2695612298 shutdown_connections 2023-08-25T09:41:20.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.177+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2695612298 wait complete. 2023-08-25T09:41:20.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.179+0000 7ff8d2a5d640 1 Processor -- start 2023-08-25T09:41:20.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.179+0000 7ff8d2a5d640 1 -- start start 2023-08-25T09:41:20.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d2a5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d2a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 0x7ff8cc14f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d2a5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8cc15cf00 0x7ff8cc15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d2a5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc14f540 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d2a5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1f69a0 con 0x7ff8cc15cf00 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d2a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0ca870 con 0x7ff8cc0cf210 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d1a5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d1a5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39194/0 (socket says 172.21.15.70:39194) 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.180+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2930691159 learned_addr learned my addr 172.21.15.70:0/2930691159 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2930691159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 msgr2=0x7ff8cc14f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 0x7ff8cc14f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2930691159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8cc15cf00 msgr2=0x7ff8cc15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8cc15cf00 0x7ff8cc15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2930691159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80094d0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc0cc300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009870 tx=0x7ff8c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8c27fc640 1 -- 172.21.15.70:0/2930691159 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80307f0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8c27fc640 1 -- 172.21.15.70:0/2930691159 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c8043400 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8c27fc640 1 -- 172.21.15.70:0/2930691159 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c80437b0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.181+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc1f5e20 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.182+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc1f63e0 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.184+0000 7ff8c27fc640 1 -- 172.21.15.70:0/2930691159 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff8c8043910 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.185+0000 7ff8c27fc640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8b006a050 0x7ff8b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.185+0000 7ff8c27fc640 1 -- 172.21.15.70:0/2930691159 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8c803e070 con 0x7ff8cc0b7df0 2023-08-25T09:41:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.185+0000 7ff8d125a640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8b006a050 0x7ff8b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.185+0000 7ff8d6838740 1 Processor -- start 2023-08-25T09:41:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.185+0000 7ff8d125a640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8b006a050 0x7ff8b006c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc007980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.185+0000 7ff8d6838740 1 -- start start 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.186+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f6647720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.186+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f66172c0 0x5617f66445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.186+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6644af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.186+0000 7ff8d6838740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f658a190 con 0x5617f661aec0 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.186+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f6587880 con 0x5617f66172c0 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.186+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f6583170 con 0x5617f6647340 2023-08-25T09:41:20.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6644af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6644af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39204/0 (socket says 172.21.15.70:39204) 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 -- 172.21.15.70:0/1592757724 learned_addr learned my addr 172.21.15.70:0/1592757724 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f6647720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 -- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 msgr2=0x5617f6647720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d125a640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f66172c0 0x5617f66445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f6647720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 -- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f66172c0 msgr2=0x5617f66445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f66172c0 0x5617f66445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 -- 172.21.15.70:0/1592757724 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f65817f0 con 0x5617f661aec0 2023-08-25T09:41:20.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d125a640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f66172c0 0x5617f66445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:20.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f6647720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:20.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.187+0000 7ff8d225c640 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6644af0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009cd0 tx=0x7ff8c400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1fe32e307a93a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8af7fe640 1 -- 172.21.15.70:0/1592757724 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400e070 con 0x5617f661aec0 2023-08-25T09:41:20.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8af7fe640 1 -- 172.21.15.70:0/1592757724 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c400bd40 con 0x5617f661aec0 2023-08-25T09:41:20.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8af7fe640 1 -- 172.21.15.70:0/1592757724 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c4012670 con 0x5617f661aec0 2023-08-25T09:41:20.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 -- 172.21.15.70:0/1592757724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 msgr2=0x5617f6644af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6644af0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009cd0 tx=0x7ff8c400b420 comp rx=0 tx=0).stop 2023-08-25T09:41:20.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 -- 172.21.15.70:0/1592757724 shutdown_connections 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6644af0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f66172c0 0x5617f66445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1592757724 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f6647720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 -- 172.21.15.70:0/1592757724 >> 172.21.15.70:0/1592757724 conn(0x5617f66195d0 msgr2=0x5617f6662150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.188+0000 7ff8d6838740 1 -- 172.21.15.70:0/1592757724 shutdown_connections 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.189+0000 7ff8d6838740 1 -- 172.21.15.70:0/1592757724 wait complete. 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.189+0000 7ff8d6838740 1 -- start start 2023-08-25T09:41:20.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.189+0000 7ff8d6838740 1 Processor -- start 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.190+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6642690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.190+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f664ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.190+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6658be0 0x5617f6659060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.190+0000 7ff8d6838740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f6522d20 con 0x5617f661aec0 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.190+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f65229c0 con 0x5617f6658be0 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.190+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f62e8f20 con 0x5617f6647340 2023-08-25T09:41:20.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d6838740 10 client.0 _ll_register_callbacks cb 0x5617f653a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6642690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6642690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39218/0 (socket says 172.21.15.70:39218) 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/1477502848 learned_addr learned my addr 172.21.15.70:0/1477502848 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d225c640 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6658be0 0x5617f6659060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 msgr2=0x5617f664ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f664ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6658be0 msgr2=0x5617f6659060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6658be0 0x5617f6659060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/1477502848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c4009980 con 0x5617f661aec0 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6642690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8006da0 tx=0x7ff8c807a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c223ca5d6ee96d82 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.191+0000 7ff8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8acff9640 1 -- 172.21.15.70:0/1477502848 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8082970 con 0x5617f661aec0 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8acff9640 1 -- 172.21.15.70:0/1477502848 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8c8043bc0 con 0x5617f661aec0 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617f6557540 con 0x5617f661aec0 2023-08-25T09:41:20.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8acff9640 1 -- 172.21.15.70:0/1477502848 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8c8043d80 con 0x5617f661aec0 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 10 client.5979 fetch_fsmap learned FSMap version 24 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5617f657af30 con 0x5617f661aec0 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8acff9640 1 -- 172.21.15.70:0/1477502848 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5617f6557540 con 0x5617f661aec0 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8acff9640 1 -- 172.21.15.70:0/1477502848 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff8c8076a50 con 0x5617f661aec0 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 10 client.5979 fetch_fsmap finished waiting for FSMap version 24 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 10 client.5979 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T09:41:20.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 20 client.5979 populate_metadata read hostname 'smithi070' 2023-08-25T09:41:20.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6672500 0x5617f66728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 4 client.5979 mds_command: new command op to 5835 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:41:20.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.192+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 --> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f657c150 con 0x5617f6672500 2023-08-25T09:41:20.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.193+0000 7ff8d125a640 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6672500 0x5617f66728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.193+0000 7ff8d125a640 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6672500 0x5617f66728e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3483ae5a90496ed3 server_cookie=8e6564818bbfaf8a in_seq=0 out_seq=0 2023-08-25T09:41:20.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.193+0000 7ff8acff9640 10 client.5979 ms_handle_connect on v2:172.21.15.70:6835/3881965592 2023-08-25T09:41:20.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8acff9640 1 -- 172.21.15.70:0/1477502848 <== mds.0 v2:172.21.15.70:6835/3881965592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5617f657c150 con 0x5617f6672500 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8acff9640 10 client.5979 handle_command_reply: tid=0 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 client.5979 shutdown 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 msgr2=0x5617f6642690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6642690 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8006da0 tx=0x7ff8c807a8d0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 shutdown_connections 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6672500 0x5617f66728e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6658be0 0x5617f6659060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6647340 0x5617f664ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 --2- 172.21.15.70:0/1477502848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f6642690 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8acff9640 0 client.5979 ms_handle_reset on v2:172.21.15.70:6835/3881965592 2023-08-25T09:41:20.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 >> 172.21.15.70:0/1477502848 conn(0x5617f66172c0 msgr2=0x5617f661b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 shutdown_connections 2023-08-25T09:41:20.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.194+0000 7ff8d6838740 1 -- 172.21.15.70:0/1477502848 wait complete. 2023-08-25T09:41:20.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.195+0000 7ff8d6838740 20 client.5979 trim_cache size 0 max 16384 2023-08-25T09:41:20.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.210+0000 7ff8d6838740 1 Processor -- start 2023-08-25T09:41:20.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.210+0000 7ff8d6838740 1 -- start start 2023-08-25T09:41:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.211+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f66cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.211+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6647340 0x5617f6647d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.211+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6648240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.211+0000 7ff8d6838740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f664a4d0 con 0x5617f661aec0 2023-08-25T09:41:20.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.211+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f6582310 con 0x5617f6647340 2023-08-25T09:41:20.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.211+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f66cd650 con 0x5617f6658be0 2023-08-25T09:41:20.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f66cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f66cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39220/0 (socket says 172.21.15.70:39220) 2023-08-25T09:41:20.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2727089055 learned_addr learned my addr 172.21.15.70:0/2727089055 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2727089055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 msgr2=0x5617f6648240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6648240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2727089055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6647340 msgr2=0x5617f6647d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6647340 0x5617f6647d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 -- 172.21.15.70:0/2727089055 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f6521a10 con 0x5617f661aec0 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f66cd110 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff8c802bf90 tx=0x7ff8c804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471bca5546934b1b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8affff640 1 -- 172.21.15.70:0/2727089055 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8076fa0 con 0x5617f661aec0 2023-08-25T09:41:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8affff640 1 -- 172.21.15.70:0/2727089055 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c807aa30 con 0x5617f661aec0 2023-08-25T09:41:20.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.212+0000 7ff8affff640 1 -- 172.21.15.70:0/2727089055 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c807a7a0 con 0x5617f661aec0 2023-08-25T09:41:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 -- 172.21.15.70:0/2727089055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 msgr2=0x5617f66cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f66cd110 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff8c802bf90 tx=0x7ff8c804f000 comp rx=0 tx=0).stop 2023-08-25T09:41:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 -- 172.21.15.70:0/2727089055 shutdown_connections 2023-08-25T09:41:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6648240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f6647340 0x5617f6647d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 --2- 172.21.15.70:0/2727089055 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f661aec0 0x5617f66cd110 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 -- 172.21.15.70:0/2727089055 >> 172.21.15.70:0/2727089055 conn(0x5617f66172c0 msgr2=0x5617f665bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 -- 172.21.15.70:0/2727089055 shutdown_connections 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.213+0000 7ff8d6838740 1 -- 172.21.15.70:0/2727089055 wait complete. 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.214+0000 7ff8d6838740 1 -- start start 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.214+0000 7ff8d6838740 1 Processor -- start 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f661aec0 0x5617f66b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 0x5617f66c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6670ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f6618110 con 0x5617f6647340 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f66ca270 con 0x5617f661aec0 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f66ca410 con 0x5617f6658be0 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d6838740 10 client.0 _ll_register_callbacks cb 0x7ff8c4009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d125a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 0x5617f66c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d125a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 0x5617f66c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39232/0 (socket says 172.21.15.70:39232) 2023-08-25T09:41:20.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.215+0000 7ff8d125a640 1 -- 172.21.15.70:0/3846159811 learned_addr learned my addr 172.21.15.70:0/3846159811 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f661aec0 0x5617f66b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d125a640 1 -- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 msgr2=0x5617f6670ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d225c640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6670ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d125a640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6670ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d125a640 1 -- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f661aec0 msgr2=0x5617f66b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d125a640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f661aec0 0x5617f66b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d125a640 1 -- 172.21.15.70:0/3846159811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f657af30 con 0x5617f6647340 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f661aec0 0x5617f66b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d125a640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 0x5617f66c9d30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007040 tx=0x7ff8bc0076d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfec08c1e2ca20b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8acff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8acff9640 1 -- 172.21.15.70:0/3846159811 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0203d0 con 0x5617f6647340 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8acff9640 1 -- 172.21.15.70:0/3846159811 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8bc020530 con 0x5617f6647340 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8c400bd40 con 0x5617f6647340 2023-08-25T09:41:20.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.216+0000 7ff8acff9640 1 -- 172.21.15.70:0/3846159811 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8bc020920 con 0x5617f6647340 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 10 client.5985 fetch_fsmap learned FSMap version 24 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5617f6546de0 con 0x5617f6647340 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8acff9640 1 -- 172.21.15.70:0/3846159811 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c400bd40 con 0x5617f6647340 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8acff9640 1 -- 172.21.15.70:0/3846159811 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff8bc01f070 con 0x5617f6647340 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 10 client.5985 fetch_fsmap finished waiting for FSMap version 24 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 10 client.5985 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 20 client.5985 populate_metadata read hostname 'smithi070' 2023-08-25T09:41:20.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6659400 0x5617f66597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 4 client.5985 mds_command: new command op to 5835 tid=0[{"prefix": "status"}] 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 --> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5617f62325e0 con 0x5617f6659400 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.217+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6659400 0x5617f66597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d1a5b640 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6659400 0x5617f66597e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac6cd93c8aa37f0b server_cookie=fa26f1593cc6631a in_seq=0 out_seq=0 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8acff9640 10 client.5985 ms_handle_connect on v2:172.21.15.70:6835/3881965592 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8acff9640 1 -- 172.21.15.70:0/3846159811 <== mds.0 v2:172.21.15.70:6835/3881965592 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5617f62325e0 con 0x5617f6659400 2023-08-25T09:41:20.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8acff9640 10 client.5985 handle_command_reply: tid=0 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 client.5985 shutdown 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 msgr2=0x5617f66c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 0x5617f66c9d30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007040 tx=0x7ff8bc0076d0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 shutdown_connections 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:6835/3881965592,v1:172.21.15.70:6837/3881965592] conn(0x5617f6659400 0x5617f66597e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5617f6658be0 0x5617f6670ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5617f6647340 0x5617f66c9d30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 --2- 172.21.15.70:0/3846159811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5617f661aec0 0x5617f66b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 >> 172.21.15.70:0/3846159811 conn(0x5617f66172c0 msgr2=0x5617f661b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8acff9640 0 client.5985 ms_handle_reset on v2:172.21.15.70:6835/3881965592 2023-08-25T09:41:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 shutdown_connections 2023-08-25T09:41:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 1 -- 172.21.15.70:0/3846159811 wait complete. 2023-08-25T09:41:20.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.218+0000 7ff8d6838740 20 client.5985 trim_cache size 0 max 16384 2023-08-25T09:41:20.228 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:20.229 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:20.229 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:41:20.229 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 5835, 2023-08-25T09:41:20.229 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:41:20.229 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:41:20.229 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:41:20.230 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6576924450000003, 2023-08-25T09:41:20.230 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 24, 2023-08-25T09:41:20.230 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 51, 2023-08-25T09:41:20.230 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 51, 2023-08-25T09:41:20.230 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.881977964000001 2023-08-25T09:41:20.230 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8b006a050 msgr2=0x7ff8b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8b006a050 0x7ff8b006c510 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc007fd0 tx=0x7ff8bc007980 comp rx=0 tx=0).stop 2023-08-25T09:41:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 msgr2=0x7ff8cc0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc0cc300 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009870 tx=0x7ff8c8004730 comp rx=0 tx=0).stop 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 shutdown_connections 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8b006a050 0x7ff8b006c510 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8cc15cf00 0x7ff8cc15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8cc0cf210 0x7ff8cc14f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 --2- 172.21.15.70:0/2930691159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8cc0b7df0 0x7ff8cc0cc300 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 >> 172.21.15.70:0/2930691159 conn(0x7ff8cc0b43a0 msgr2=0x7ff8cc1589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 shutdown_connections 2023-08-25T09:41:20.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.219+0000 7ff8d2a5d640 1 -- 172.21.15.70:0/2930691159 wait complete. 2023-08-25T09:41:20.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:41:20.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.417+0000 7f4d7b566640 1 Processor -- start 2023-08-25T09:41:20.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.417+0000 7f4d7b566640 1 -- start start 2023-08-25T09:41:20.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7b566640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4d740c92c0 0x7f4d740b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7b566640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740c96a0 0x7f4d740cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7b566640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d740b4150 0x7f4d740cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7b566640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d741615b0 con 0x7f4d740b4150 2023-08-25T09:41:20.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7b566640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d740cd330 con 0x7f4d740c92c0 2023-08-25T09:41:20.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7b566640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740cb7c0 con 0x7f4d740c96a0 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7ad65640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d740b4150 0x7f4d740cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.419+0000 7f4d7ad65640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d740b4150 0x7f4d740cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39234/0 (socket says 172.21.15.70:39234) 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1878796904 learned_addr learned my addr 172.21.15.70:0/1878796904 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1878796904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740c96a0 msgr2=0x7f4d740cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7a564640 1 --2- 172.21.15.70:0/1878796904 >> v2:172.21.15.155:3300/0 conn(0x7f4d740c92c0 0x7f4d740b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d79d63640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740c96a0 0x7f4d740cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7ad65640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740c96a0 0x7f4d740cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1878796904 >> v2:172.21.15.155:3300/0 conn(0x7f4d740c92c0 msgr2=0x7f4d740b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:20.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7ad65640 1 --2- 172.21.15.70:0/1878796904 >> v2:172.21.15.155:3300/0 conn(0x7f4d740c92c0 0x7f4d740b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1878796904 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740b0e80 con 0x7f4d740b4150 2023-08-25T09:41:20.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d79d63640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740c96a0 0x7f4d740cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:20.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.420+0000 7f4d7a564640 1 --2- 172.21.15.70:0/1878796904 >> v2:172.21.15.155:3300/0 conn(0x7f4d740c92c0 0x7f4d740b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:20.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.421+0000 7f4d7ad65640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740b4150 0x7f4d740cb280 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4d7406c670 tx=0x7f4d6c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67274c3a6d5faa77 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.421+0000 7f4d79562640 1 -- 172.21.15.70:0/1878796904 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c02ca80 con 0x7f4d740b4150 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.421+0000 7f4d79562640 1 -- 172.21.15.70:0/1878796904 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c02cbe0 con 0x7f4d740b4150 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.421+0000 7f4d79562640 1 -- 172.21.15.70:0/1878796904 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0343f0 con 0x7f4d740b4150 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 -- 172.21.15.70:0/1878796904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740b4150 msgr2=0x7f4d740cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740b4150 0x7f4d740cb280 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4d7406c670 tx=0x7f4d6c02bee0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 -- 172.21.15.70:0/1878796904 shutdown_connections 2023-08-25T09:41:20.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740b4150 0x7f4d740cb280 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1878796904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740c96a0 0x7f4d740cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1878796904 >> v2:172.21.15.155:3300/0 conn(0x7f4d740c92c0 0x7f4d740b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 -- 172.21.15.70:0/1878796904 >> 172.21.15.70:0/1878796904 conn(0x7f4d740affd0 msgr2=0x7f4d740b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 -- 172.21.15.70:0/1878796904 shutdown_connections 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.422+0000 7f4d7b566640 1 -- 172.21.15.70:0/1878796904 wait complete. 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.424+0000 7f4d7b566640 1 Processor -- start 2023-08-25T09:41:20.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.424+0000 7f4d7b566640 1 -- start start 2023-08-25T09:41:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.425+0000 7f4d7b566640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740b4150 0x7f4d741eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.425+0000 7f4d7b566640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c92c0 0x7f4d740713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.425+0000 7f4d7b566640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 0x7f4d741603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.425+0000 7f4d7b566640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d74071900 con 0x7f4d740c96a0 2023-08-25T09:41:20.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.425+0000 7f4d7b566640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d74160910 con 0x7f4d740c92c0 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.425+0000 7f4d7b566640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d740c56f0 con 0x7f4d740b4150 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 0x7f4d741603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 0x7f4d741603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39248/0 (socket says 172.21.15.70:39248) 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1399882483 learned_addr learned my addr 172.21.15.70:0/1399882483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7a564640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740b4150 0x7f4d741eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d79d63640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c92c0 0x7f4d740713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740b4150 msgr2=0x7f4d741eb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740b4150 0x7f4d741eb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c92c0 msgr2=0x7f4d740713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c92c0 0x7f4d740713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d7ad65640 1 -- 172.21.15.70:0/1399882483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c009b30 con 0x7f4d740c96a0 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.426+0000 7f4d79d63640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c92c0 0x7f4d740713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.427+0000 7f4d7ad65640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 0x7f4d741603d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c036040 tx=0x7f4d6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.427+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c009510 con 0x7f4d740c96a0 2023-08-25T09:41:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.427+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6c03e450 con 0x7f4d740c96a0 2023-08-25T09:41:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.427+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c03e800 con 0x7f4d740c96a0 2023-08-25T09:41:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.427+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d741ebb10 con 0x7f4d740c96a0 2023-08-25T09:41:20.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.427+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d740c59b0 con 0x7f4d740c96a0 2023-08-25T09:41:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.428+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d74066fb0 con 0x7f4d740c96a0 2023-08-25T09:41:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.429+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4d6c02cd50 con 0x7f4d740c96a0 2023-08-25T09:41:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.430+0000 7f4d6affd640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d5806a050 0x7f4d5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.430+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d6c0ab1d0 con 0x7f4d740c96a0 2023-08-25T09:41:20.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.430+0000 7f4d7a564640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d5806a050 0x7f4d5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:20.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.431+0000 7f4d7a564640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d5806a050 0x7f4d5806c510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006b10 tx=0x7f4d70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:20.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.434+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4d6c076a80 con 0x7f4d740c96a0 2023-08-25T09:41:20.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.659+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4d741ebfd0 con 0x7f4d740c96a0 2023-08-25T09:41:20.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.659+0000 7f4d6affd640 1 -- 172.21.15.70:0/1399882483 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4d6c043060 con 0x7f4d740c96a0 2023-08-25T09:41:20.662 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:41:20.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.661+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d5806a050 msgr2=0x7f4d5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.661+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d5806a050 0x7f4d5806c510 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d70006b10 tx=0x7f4d70008040 comp rx=0 tx=0).stop 2023-08-25T09:41:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.661+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 msgr2=0x7f4d741603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:20.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.661+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 0x7f4d741603d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c036040 tx=0x7f4d6c004a40 comp rx=0 tx=0).stop 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 shutdown_connections 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d5806a050 0x7f4d5806c510 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d740c96a0 0x7f4d741603d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d740c92c0 0x7f4d740713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 --2- 172.21.15.70:0/1399882483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d740b4150 0x7f4d741eb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 >> 172.21.15.70:0/1399882483 conn(0x7f4d740affd0 msgr2=0x7f4d74161a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:20.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 shutdown_connections 2023-08-25T09:41:20.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:20.662+0000 7f4d7b566640 1 -- 172.21.15.70:0/1399882483 wait complete. 2023-08-25T09:41:20.680 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:41:20.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:41:20.681 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:41:20.681 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:41:20.682 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:41:20.682 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:41:20.682 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:41:20.682 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:41:20.683 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:41:20.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:20.683 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:41:20.742 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:41:20.742 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:41:20.742 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:41:20.742 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.742 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:41:20.743 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:41:20.744 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:20.744 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:41:20.744 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:20.744 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:41:20.744 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42195sec preferred_lft 42195sec 2023-08-25T09:41:20.744 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:41:20.745 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:41:20.746 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:41:20.747 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:20.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:20.747 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:41:20.842 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:41:20.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:20.847 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:41:20.903 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:41:20.903 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:41:20.904 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:41:20.904 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:41:20.904 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:41:20.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:20.905 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:41:20.906 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:41:20.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:41:20.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:41:20.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:41:20.908 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:41:21.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:21.100 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:41:21.153 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:41:21.153 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:41:21.154 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:41:21.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:21.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:41:21.245 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:41:21.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:41:21.247 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:41:21.302 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:41:21.303 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:41:21.304 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:41:21.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:41:21.447 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:41:21.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:41:21.542 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:41:21.546 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:41:21.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:21.548 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:41:21.608 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:41:21.609 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:41:21.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:41:21.692 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:41:21.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:41:21.769 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:41:21.772 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:41:21.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:21.773 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:41:21.804 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:41:21.802+0000 7fb429238480 -1 init, newargv = 0x55ef118b1bc0 newargc=15 2023-08-25T09:41:21.805 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[99410]: starting ceph client 2023-08-25T09:41:21.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[99410]: starting fuse 2023-08-25T09:41:21.819 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:41:21.822 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:41:21.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:41:21.823 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:41:21.823 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:41:21.823 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:41:21.824 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:41:21.824 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:41:21.824 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:41:21.825 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:41:21.825 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:41:21.826 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:41:21.826 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:41:21.827 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:41:21.828 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:41:21.829 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:41:21.829 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:41:21.829 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:41:21.830 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:41:21.831 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:41:21.832 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:41:21.832 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:41:21.833 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:41:21.834 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:41:21.835 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:41:21.836 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:41:21.837 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:41:21.838 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:41:21.838 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:41:22.022 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.99410.asok 2023-08-25T09:41:22.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:41:22.039 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99410.asok 2023-08-25T09:41:22.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:22.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99410.asok status 2023-08-25T09:41:22.282 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:41:22.283 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "99410", 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6000, 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:41:22.284 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 6000 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:41:22.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2561359675 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2561359675 2023-08-25T09:41:22.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:22.287 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.6000 192.168.0.1:0/2561359675", 2023-08-25T09:41:22.287 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2561359675", 2023-08-25T09:41:22.287 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:41:22.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 24, 2023-08-25T09:41:22.287 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 51, 2023-08-25T09:41:22.287 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:41:22.288 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:41:22.288 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:41:22.288 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:22.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:22.297 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:41:22.298 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:41:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:41:22.348 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:41:22.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:22.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:41:22.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:41:22.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.656+0000 7f295d5a6640 1 Processor -- start 2023-08-25T09:41:22.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.656+0000 7f295d5a6640 1 -- start start 2023-08-25T09:41:22.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f295d5a6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f295815ec10 0x7f295815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f295d5a6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815c6d0 0x7f29580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f295d5a6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29580b1b60 con 0x7f295815e830 2023-08-25T09:41:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f295d5a6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295806bd80 con 0x7f295815ec10 2023-08-25T09:41:22.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f295d5a6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295806b7c0 con 0x7f295815c6d0 2023-08-25T09:41:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f294ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f295815ec10 0x7f295815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:22.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f29577fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815c6d0 0x7f29580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.658+0000 7f29577fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815c6d0 0x7f29580b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:60350/0 (socket says 172.21.15.70:60350) 2023-08-25T09:41:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.659+0000 7f29577fe640 1 -- 172.21.15.70:0/1176467290 learned_addr learned my addr 172.21.15.70:0/1176467290 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.659+0000 7f294ffff640 1 -- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815c6d0 msgr2=0x7f29580b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.659+0000 7f294ffff640 1 --2- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815c6d0 0x7f29580b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.659+0000 7f294ffff640 1 -- 172.21.15.70:0/1176467290 >> v1:172.21.15.155:6789/0 conn(0x7f295815e830 legacy=0x7f295806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:22.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.659+0000 7f294ffff640 1 -- 172.21.15.70:0/1176467290 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295806baa0 con 0x7f295815ec10 2023-08-25T09:41:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.661+0000 7f294ffff640 1 --2- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815ec10 0x7f295815c190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29400097a0 tx=0x7f294002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573e12fafba94da4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:22.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.661+0000 7f29567fc640 1 -- 172.21.15.70:0/1176467290 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940031830 con 0x7f295815ec10 2023-08-25T09:41:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f29567fc640 1 -- 172.21.15.70:0/1176467290 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940031990 con 0x7f295815ec10 2023-08-25T09:41:22.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f29567fc640 1 -- 172.21.15.70:0/1176467290 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940031d20 con 0x7f295815ec10 2023-08-25T09:41:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 -- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815ec10 msgr2=0x7f295815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 --2- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815ec10 0x7f295815c190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f29400097a0 tx=0x7f294002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 -- 172.21.15.70:0/1176467290 shutdown_connections 2023-08-25T09:41:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 --2- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815c6d0 0x7f29580b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 --2- 172.21.15.70:0/1176467290 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815ec10 0x7f295815c190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 -- 172.21.15.70:0/1176467290 >> 172.21.15.70:0/1176467290 conn(0x7f2958069850 msgr2=0x7f2958069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:22.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 -- 172.21.15.70:0/1176467290 shutdown_connections 2023-08-25T09:41:22.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.662+0000 7f295d5a6640 1 -- 172.21.15.70:0/1176467290 wait complete. 2023-08-25T09:41:22.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.664+0000 7f295d5a6640 1 Processor -- start 2023-08-25T09:41:22.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.664+0000 7f295d5a6640 1 -- start start 2023-08-25T09:41:22.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f295d5a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815c6d0 0x7f2958158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:22.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f295d5a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 0x7f295814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:22.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f295d5a6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815ec10 0x7f29580c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:22.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f295d5a6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958070650 con 0x7f295815e830 2023-08-25T09:41:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f295d5a6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295806f6f0 con 0x7f295815c6d0 2023-08-25T09:41:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f295d5a6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295806f9d0 con 0x7f295815ec10 2023-08-25T09:41:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.665+0000 7f294ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 0x7f295814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 0x7f295814ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39260/0 (socket says 172.21.15.70:39260) 2023-08-25T09:41:22.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 -- 172.21.15.70:0/2004776632 learned_addr learned my addr 172.21.15.70:0/2004776632 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f2956ffd640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815c6d0 0x7f2958158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f29577fe640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815ec10 0x7f29580c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 -- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815ec10 msgr2=0x7f29580c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815ec10 0x7f29580c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 -- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815c6d0 msgr2=0x7f2958158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815c6d0 0x7f2958158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 -- 172.21.15.70:0/2004776632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2940009400 con 0x7f295815e830 2023-08-25T09:41:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f29577fe640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815ec10 0x7f29580c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.666+0000 7f294ffff640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 0x7f295814ff70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2940000c00 tx=0x7f29400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:22.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.667+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294003a830 con 0x7f295815e830 2023-08-25T09:41:22.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.667+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294003a9b0 con 0x7f295815e830 2023-08-25T09:41:22.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.667+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294003ad60 con 0x7f295815e830 2023-08-25T09:41:22.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.667+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295806fcb0 con 0x7f295815e830 2023-08-25T09:41:22.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.667+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2958070270 con 0x7f295815e830 2023-08-25T09:41:22.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.668+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295804e420 con 0x7f295815e830 2023-08-25T09:41:22.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.669+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f294004d450 con 0x7f295815e830 2023-08-25T09:41:22.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.670+0000 7f294f7fe640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f293806a050 0x7f293806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:22.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.670+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29400aa720 con 0x7f295815e830 2023-08-25T09:41:22.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.670+0000 7f2956ffd640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f293806a050 0x7f293806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:22.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.671+0000 7f2956ffd640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f293806a050 0x7f293806c510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2948000f60 tx=0x7f2948009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:22.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.675+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2940075ce0 con 0x7f295815e830 2023-08-25T09:41:22.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.898+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f295806c620 con 0x7f295815e830 2023-08-25T09:41:22.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.899+0000 7f294f7fe640 1 -- 172.21.15.70:0/2004776632 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2940035070 con 0x7f295815e830 2023-08-25T09:41:22.901 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:41:22.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.901+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f293806a050 msgr2=0x7f293806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:22.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.901+0000 7f295d5a6640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f293806a050 0x7f293806c510 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2948000f60 tx=0x7f2948009040 comp rx=0 tx=0).stop 2023-08-25T09:41:22.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.901+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 msgr2=0x7f295814ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.901+0000 7f295d5a6640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 0x7f295814ff70 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2940000c00 tx=0x7f29400046c0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.901+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 shutdown_connections 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.901+0000 7f295d5a6640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f293806a050 0x7f293806c510 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.902+0000 7f295d5a6640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f295815ec10 0x7f29580c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.902+0000 7f295d5a6640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f295815e830 0x7f295814ff70 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.902+0000 7f295d5a6640 1 --2- 172.21.15.70:0/2004776632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f295815c6d0 0x7f2958158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.902+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 >> 172.21.15.70:0/2004776632 conn(0x7f2958069850 msgr2=0x7f295815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:22.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.902+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 shutdown_connections 2023-08-25T09:41:22.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:22.902+0000 7f295d5a6640 1 -- 172.21.15.70:0/2004776632 wait complete. 2023-08-25T09:41:22.921 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:41:22.921 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:41:22.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:41:22.922 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:41:22.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:41:22.922 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:41:22.923 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:41:22.923 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:41:22.923 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:41:22.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:22.924 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:41:22.985 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:41:22.985 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:41:22.985 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:41:22.986 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:22.987 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42206sec preferred_lft 42206sec 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.988 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:41:22.989 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.989 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:41:22.989 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:41:22.989 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:41:22.989 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.989 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:41:22.990 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:41:22.990 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:41:22.990 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:41:22.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:22.991 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:41:23.087 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:41:23.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:23.091 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:41:23.147 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:41:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:41:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:41:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:41:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:41:23.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:23.150 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:41:23.150 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:41:23.151 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:41:23.151 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:41:23.152 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:41:23.152 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:41:23.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:23.343 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:41:23.392 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:41:23.392 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:41:23.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:41:23.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:23.394 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:41:23.535 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:41:23.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:41:23.535 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:41:23.588 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:41:23.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:41:23.589 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:41:23.644 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:41:23.748 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:41:23.750 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:41:23.846 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:41:23.849 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:41:23.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:23.851 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:41:23.903 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:41:23.903 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:41:23.910 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:41:23.986 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:41:23.987 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:41:24.066 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:41:24.070 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:41:24.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:24.071 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:41:24.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:41:24.097+0000 7f081f471480 -1 init, newargv = 0x56541076be50 newargc=15 2023-08-25T09:41:24.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[88654]: starting ceph client 2023-08-25T09:41:24.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[88654]: starting fuse 2023-08-25T09:41:24.117 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:41:24.118 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:41:24.119 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:41:24.119 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:41:24.120 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:41:24.120 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:41:24.121 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:41:24.121 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:41:24.121 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:41:24.122 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:41:24.122 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:41:24.122 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:41:24.123 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:41:24.123 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:41:24.123 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:41:24.124 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:41:24.124 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:41:24.124 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:41:24.125 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:41:24.125 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:41:24.125 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:41:24.125 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:41:24.126 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:41:24.126 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:41:24.126 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:41:24.127 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:41:24.127 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:41:24.127 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:41:24.128 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:41:24.332 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.88654.asok 2023-08-25T09:41:24.336 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:41:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:41:24.349 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.88654.asok 2023-08-25T09:41:24.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:24.350 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.88654.asok status 2023-08-25T09:41:24.582 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:41:24.582 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:41:24.583 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:41:24.583 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:41:24.583 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:41:24.584 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:41:24.585 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:41:24.585 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "88654", 2023-08-25T09:41:24.585 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:41:24.586 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:41:24.586 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:41:24.587 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:41:24.587 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5071, 2023-08-25T09:41:24.588 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:41:24.588 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:41:24.589 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:41:24.589 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5071 2023-08-25T09:41:24.589 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:41:24.589 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:41:24.590 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:41:24.590 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:41:24.590 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2906722472 2023-08-25T09:41:24.590 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:41:24.591 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:41:24.591 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:41:24.591 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:41:24.591 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:41:24.591 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2906722472 2023-08-25T09:41:24.592 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:41:24.592 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5071 192.168.0.1:0/2906722472", 2023-08-25T09:41:24.592 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/2906722472", 2023-08-25T09:41:24.592 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:41:24.593 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 24, 2023-08-25T09:41:24.593 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 51, 2023-08-25T09:41:24.593 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:41:24.594 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:41:24.594 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:41:24.594 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:41:24.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:24.597 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:41:24.598 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:41:24.657 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:41:24.657 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:41:24.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:24.659 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:41:24.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:41:24.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.979+0000 7f88ac724640 1 Processor -- start 2023-08-25T09:41:24.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.979+0000 7f88ac724640 1 -- start start 2023-08-25T09:41:24.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.980+0000 7f88ac724640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88a40b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.980+0000 7f88ac724640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a40b1b60 con 0x7f88a415ec10 2023-08-25T09:41:24.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.980+0000 7f88ac724640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a406b7c0 con 0x7f88a415e830 2023-08-25T09:41:24.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.980+0000 7f88ac724640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a406bd80 con 0x7f88a40b1ff0 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.981+0000 7f88a9cbf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f88a415ec10 0x7f88a415c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49584/0 (socket says 172.21.15.70:49584) 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.981+0000 7f88a9cbf640 1 -- 172.21.15.70:0/23823322 learned_addr learned my addr 172.21.15.70:0/23823322 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.981+0000 7f88aacc1640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88a40b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.981+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716061972 0 0) 0x7f88a40b1b60 con 0x7f88a415ec10 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.981+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8890003520 con 0x7f88a415ec10 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4257456870 0 0) 0x7f8890003520 con 0x7f88a415ec10 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 msgr2=0x7f88a40b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:24.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88a40b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 >> v1:172.21.15.155:6789/0 conn(0x7f88a415e830 legacy=0x7f88a406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a406baa0 con 0x7f88a415ec10 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8894001670 con 0x7f88a415ec10 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 >> v1:172.21.15.70:6789/0 conn(0x7f88a415ec10 legacy=0x7f88a415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88900039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890003f20 0x7f88900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890004d00 0x7f8890005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a406bd80 con 0x7f8890004d00 2023-08-25T09:41:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889000ba60 con 0x7f8890003f20 2023-08-25T09:41:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889000bc20 con 0x7f88a40b1ff0 2023-08-25T09:41:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88aacc1640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890004d00 0x7f8890005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:24.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88a9cbf640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890003f20 0x7f88900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:24.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88aacc1640 1 -- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 msgr2=0x7f88900039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:24.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.982+0000 7f88aa4c0640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88900039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:24.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.983+0000 7f88aacc1640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88900039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:24.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.983+0000 7f88aacc1640 1 -- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890003f20 msgr2=0x7f88900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:24.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.983+0000 7f88aacc1640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890003f20 0x7f88900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:24.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.983+0000 7f88aacc1640 1 -- 172.21.15.70:0/23823322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890003520 con 0x7f8890004d00 2023-08-25T09:41:24.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.983+0000 7f88a9cbf640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890003f20 0x7f88900047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:24.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.983+0000 7f88aa4c0640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a40b1ff0 0x7f88900039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:24.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88aacc1640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890004d00 0x7f8890005180 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f889c009fc0 tx=0x7f88900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afdf6940298c92f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c02cbf0 con 0x7f8890004d00 2023-08-25T09:41:24.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c00f040 con 0x7f8890004d00 2023-08-25T09:41:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88a94be640 1 -- 172.21.15.70:0/23823322 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c034440 con 0x7f8890004d00 2023-08-25T09:41:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88ac724640 1 -- 172.21.15.70:0/23823322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890004d00 msgr2=0x7f8890005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88ac724640 1 --2- 172.21.15.70:0/23823322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8890004d00 0x7f8890005180 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f889c009fc0 tx=0x7f88900057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:24.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.984+0000 7f88aacc1640 1 -- 172.21.15.70:0/23823322 reap_dead start 2023-08-25T09:41:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.985+0000 7f88ac724640 1 -- 172.21.15.70:0/23823322 shutdown_connections 2023-08-25T09:41:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.985+0000 7f88ac724640 1 -- 172.21.15.70:0/23823322 >> 172.21.15.70:0/23823322 conn(0x7f88a4069850 msgr2=0x7f88a4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.985+0000 7f88ac724640 1 -- 172.21.15.70:0/23823322 shutdown_connections 2023-08-25T09:41:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.985+0000 7f88ac724640 1 -- 172.21.15.70:0/23823322 wait complete. 2023-08-25T09:41:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.987+0000 7f88ac724640 1 Processor -- start 2023-08-25T09:41:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.987+0000 7f88ac724640 1 -- start start 2023-08-25T09:41:24.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88ac724640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890004d00 0x7f88a41f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88ac724640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 0x7f88a41efeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88ac724640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a41f03f0 0x7f88a4155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:24.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88ac724640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41f0870 con 0x7f88a414cd20 2023-08-25T09:41:24.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88ac724640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41f44c0 con 0x7f8890004d00 2023-08-25T09:41:25.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88ac724640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a41f0ad0 con 0x7f88a41f03f0 2023-08-25T09:41:25.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 0x7f88a41efeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:25.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 0x7f88a41efeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39290/0 (socket says 172.21.15.70:39290) 2023-08-25T09:41:25.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 -- 172.21.15.70:0/2399897616 learned_addr learned my addr 172.21.15.70:0/2399897616 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88aa4c0640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890004d00 0x7f88a41f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88aacc1640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a41f03f0 0x7f88a4155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:25.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 -- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a41f03f0 msgr2=0x7f88a4155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a41f03f0 0x7f88a4155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 -- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890004d00 msgr2=0x7f88a41f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890004d00 0x7f88a41f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88a9cbf640 1 -- 172.21.15.70:0/2399897616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a41f0db0 con 0x7f88a414cd20 2023-08-25T09:41:25.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88aa4c0640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890004d00 0x7f88a41f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.988+0000 7f88aacc1640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a41f03f0 0x7f88a4155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.989+0000 7f88a9cbf640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 0x7f88a41efeb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f889400d8b0 tx=0x7f889400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:25.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.989+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940074e0 con 0x7f88a414cd20 2023-08-25T09:41:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.989+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8894007640 con 0x7f88a414cd20 2023-08-25T09:41:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.989+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88940049e0 con 0x7f88a414cd20 2023-08-25T09:41:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.989+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a41f3780 con 0x7f88a414cd20 2023-08-25T09:41:25.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.989+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a4152580 con 0x7f88a414cd20 2023-08-25T09:41:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.990+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a41f35e0 con 0x7f88a414cd20 2023-08-25T09:41:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.991+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8894004b40 con 0x7f88a414cd20 2023-08-25T09:41:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.993+0000 7f889affd640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f888806a050 0x7f888806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:25.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.993+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8894087ed0 con 0x7f88a414cd20 2023-08-25T09:41:25.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.993+0000 7f88aa4c0640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f888806a050 0x7f888806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:25.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.994+0000 7f88aa4c0640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f888806a050 0x7f888806c510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c2d0 tx=0x7f88a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:25.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:24.997+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f88a41f35e0 con 0x7f88a414cd20 2023-08-25T09:41:25.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.223+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88a41f35e0 con 0x7f88a414cd20 2023-08-25T09:41:25.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.224+0000 7f889affd640 1 -- 172.21.15.70:0/2399897616 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f88a41f35e0 con 0x7f88a414cd20 2023-08-25T09:41:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:25.227 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:41:25.229 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:41:25.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f888806a050 msgr2=0x7f888806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:25.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f888806a050 0x7f888806c510 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f88a406c2d0 tx=0x7f88a0009040 comp rx=0 tx=0).stop 2023-08-25T09:41:25.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 msgr2=0x7f88a41efeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:25.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 0x7f88a41efeb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f889400d8b0 tx=0x7f889400dd70 comp rx=0 tx=0).stop 2023-08-25T09:41:25.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 shutdown_connections 2023-08-25T09:41:25.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f888806a050 0x7f888806c510 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:25.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f88a41f03f0 0x7f88a4155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:25.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88a414cd20 0x7f88a41efeb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:25.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 --2- 172.21.15.70:0/2399897616 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8890004d00 0x7f88a41f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 >> 172.21.15.70:0/2399897616 conn(0x7f88a4069850 msgr2=0x7f88a415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 shutdown_connections 2023-08-25T09:41:25.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:25.227+0000 7f88ac724640 1 -- 172.21.15.70:0/2399897616 wait complete. 2023-08-25T09:41:25.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:41:25.470 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":5835,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.893404203999999,"mdsmap_epoch":24,"osdmap_epoch":51,"osdmap_epoch_barrier":51,"uptime":18.117704481000001} 2023-08-25T09:41:25.471 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 5835, "mdsmap_epoch": 24, "osdmap_epoch": 51, "osdmap_epoch_barrier": 51, "rank_uptime": 11.893404204, "state": "up:active", "uptime": 18.117704481, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:41:25.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:41:25.701 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:41:25.701 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:41:25.702 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-08-25T09:41:25.702 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:25.705 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:25.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:25.757 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-08-25T09:41:31.568 INFO:teuthology.orchestra.run.smithi070.stderr:265+0 records in 2023-08-25T09:41:31.569 INFO:teuthology.orchestra.run.smithi070.stderr:265+0 records out 2023-08-25T09:41:31.570 INFO:teuthology.orchestra.run.smithi070.stderr:277872640 bytes (278 MB, 265 MiB) copied, 5.70653 s, 48.7 MB/s 2023-08-25T09:41:31.571 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:31.572 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:31.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:31.632 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-08-25T09:41:35.713 INFO:teuthology.orchestra.run.smithi070.stderr:292+0 records in 2023-08-25T09:41:35.713 INFO:teuthology.orchestra.run.smithi070.stderr:292+0 records out 2023-08-25T09:41:35.713 INFO:teuthology.orchestra.run.smithi070.stderr:306184192 bytes (306 MB, 292 MiB) copied, 4.02511 s, 76.1 MB/s 2023-08-25T09:41:35.715 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-08-25T09:41:35.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:35.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.927+0000 7fc9e8ca7640 1 Processor -- start 2023-08-25T09:41:35.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.928+0000 7fc9e8ca7640 1 -- start start 2023-08-25T09:41:35.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.929+0000 7fc9e8ca7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc9e415cc90 0x7fc9e40d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.929+0000 7fc9e8ca7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40d1fd0 con 0x7fc9e415d070 2023-08-25T09:41:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.929+0000 7fc9e8ca7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e40d13c0 con 0x7fc9e40cfa10 2023-08-25T09:41:35.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.929+0000 7fc9e8ca7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e40d28a0 con 0x7fc9e415cc90 2023-08-25T09:41:35.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.929+0000 7fc9e1d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc9e415d070 0x7fc9e40cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53070/0 (socket says 172.21.15.70:53070) 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.929+0000 7fc9e1d9b640 1 -- 172.21.15.70:0/151453712 learned_addr learned my addr 172.21.15.70:0/151453712 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077534975 0 0) 0x7fc9e40d1fd0 con 0x7fc9e415d070 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9cc003520 con 0x7fc9e415d070 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e259c640 1 --2- 172.21.15.70:0/151453712 >> v2:172.21.15.155:3300/0 conn(0x7fc9e415cc90 0x7fc9e40d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1301563311 0 0) 0x7fc9cc003520 con 0x7fc9e415d070 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 >> v2:172.21.15.155:3300/0 conn(0x7fc9e415cc90 msgr2=0x7fc9e40d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:35.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 --2- 172.21.15.70:0/151453712 >> v2:172.21.15.155:3300/0 conn(0x7fc9e415cc90 0x7fc9e40d0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 >> v1:172.21.15.155:6789/0 conn(0x7fc9e40cfa10 legacy=0x7fc9e40b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40d16a0 con 0x7fc9e415d070 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc9d8001670 con 0x7fc9e415d070 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 >> v1:172.21.15.70:6789/0 conn(0x7fc9e415d070 legacy=0x7fc9e40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.930+0000 7fc9e159a640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e159a640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9cc0047c0 0x7fc9cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e159a640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9cc0051d0 0x7fc9cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc00c0e0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc00c250 con 0x7fc9cc0051d0 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc00c410 con 0x7fc9cc0047c0 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e259c640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e259c640 1 -- 172.21.15.70:0/151453712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9cc0047c0 msgr2=0x7fc9cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e259c640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9cc0047c0 0x7fc9cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e259c640 1 -- 172.21.15.70:0/151453712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9cc0051d0 msgr2=0x7fc9cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e259c640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9cc0051d0 0x7fc9cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e259c640 1 -- 172.21.15.70:0/151453712 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc003520 con 0x7fc9cc0039c0 2023-08-25T09:41:35.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.931+0000 7fc9e2d9d640 1 -- 172.21.15.70:0/151453712 reap_dead start 2023-08-25T09:41:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.932+0000 7fc9e259c640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9cc003de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400ba30 tx=0x7fc9cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58423c97686324f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.932+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4032680 con 0x7fc9cc0039c0 2023-08-25T09:41:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.932+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d40327e0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.932+0000 7fc9e159a640 1 -- 172.21.15.70:0/151453712 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4032b70 con 0x7fc9cc0039c0 2023-08-25T09:41:35.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/151453712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 msgr2=0x7fc9cc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9cc003de0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400ba30 tx=0x7fc9cc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/151453712 shutdown_connections 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/151453712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9cc003de0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/151453712 >> 172.21.15.70:0/151453712 conn(0x7fc9e4070260 msgr2=0x7fc9e4070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/151453712 shutdown_connections 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.933+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/151453712 wait complete. 2023-08-25T09:41:35.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.935+0000 7fc9e8ca7640 1 Processor -- start 2023-08-25T09:41:35.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.935+0000 7fc9e8ca7640 1 -- start start 2023-08-25T09:41:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.936+0000 7fc9e8ca7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9e40b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.936+0000 7fc9e8ca7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4159ef0 0x7fc9e40b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.936+0000 7fc9e8ca7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9e40b2ab0 0x7fc9e40b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.936+0000 7fc9e8ca7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e415a9d0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.936+0000 7fc9e8ca7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40b46d0 con 0x7fc9e4159ef0 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.936+0000 7fc9e8ca7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e40c9cf0 con 0x7fc9e40b2ab0 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9e40b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9e40b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56660/0 (socket says 172.21.15.70:56660) 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 -- 172.21.15.70:0/2882316329 learned_addr learned my addr 172.21.15.70:0/2882316329 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 -- 172.21.15.70:0/2882316329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9e40b2ab0 msgr2=0x7fc9e40b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9e40b2ab0 0x7fc9e40b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 -- 172.21.15.70:0/2882316329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4159ef0 msgr2=0x7fc9e40b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:35.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4159ef0 0x7fc9e40b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 -- 172.21.15.70:0/2882316329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40c9fd0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.937+0000 7fc9e259c640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9e40b40f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc9d402bf00 tx=0x7fc9d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.938+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4030a60 con 0x7fc9cc0039c0 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.938+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d4030bc0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.938+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e40ca2b0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.938+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d403b420 con 0x7fc9cc0039c0 2023-08-25T09:41:35.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.938+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e41f5470 con 0x7fc9cc0039c0 2023-08-25T09:41:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.939+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e40c9b50 con 0x7fc9cc0039c0 2023-08-25T09:41:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.940+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc9d404d3f0 con 0x7fc9cc0039c0 2023-08-25T09:41:35.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.941+0000 7fc9d2ffd640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9c406a080 0x7fc9c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:35.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.941+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9d40aab10 con 0x7fc9cc0039c0 2023-08-25T09:41:35.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.941+0000 7fc9e1d9b640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9c406a080 0x7fc9c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:35.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.942+0000 7fc9e1d9b640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9c406a080 0x7fc9c406c540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40b75b0 tx=0x7fc9d8001540 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:35.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:35.946+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9e40c9b50 con 0x7fc9cc0039c0 2023-08-25T09:41:36.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.138+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc9e40c9b50 con 0x7fc9cc0039c0 2023-08-25T09:41:36.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.141+0000 7fc9d2ffd640 1 -- 172.21.15.70:0/2882316329 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) v1 ==== 81+0+36952 (secure 0 0 0) 0x7fc9e40c9b50 con 0x7fc9cc0039c0 2023-08-25T09:41:36.144 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:36.145 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:36.145 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 51, 2023-08-25T09:41:36.145 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:36.146 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:36.146 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:14.522565+0000", 2023-08-25T09:41:36.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:36.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:36.147 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:36.147 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:36.148 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:36.148 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:36.148 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:36.148 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:36.149 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:36.149 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:36.150 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:36.150 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:36.150 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:36.150 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:36.150 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:36.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:36.151 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:36.151 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:36.151 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:41:36.152 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:36.152 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:36.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:36.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:36.153 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:36.153 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:36.153 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:36.154 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:36.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:36.154 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:36.154 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:36.154 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:36.154 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:36.155 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:36.155 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:36.156 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:36.156 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:36.156 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:36.157 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "16", 2023-08-25T09:41:36.160 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:36.160 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:36.160 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:36.160 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:36.161 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:36.162 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:36.163 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:36.164 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:36.165 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:36.165 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:36.165 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:36.165 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:36.165 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:36.165 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:36.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.167 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:36.167 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:36.167 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:36.167 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:36.167 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:36.167 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.168 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:36.169 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:36.169 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:36.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:36.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:36.170 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:36.170 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:36.170 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:36.170 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:36.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:36.172 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "19", 2023-08-25T09:41:36.173 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:36.174 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:36.175 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:36.176 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:36.177 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:36.177 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:36.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.177 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:36.177 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:36.177 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:36.178 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:36.179 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:36.180 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:36.180 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:36.180 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:36.180 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:36.180 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:36.181 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:36.182 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:36.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:36.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:36.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:36.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:36.184 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:36.184 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "50", 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:36.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:36.187 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:36.187 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:36.187 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:36.187 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:36.187 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:36.188 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:36.188 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:36.189 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:36.189 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:36.189 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:36.190 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:36.190 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:36.190 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:36.190 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:36.191 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:36.191 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:36.191 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:36.191 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:36.192 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:36.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.192 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:36.193 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:36.193 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:36.193 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:36.193 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:36.194 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:36.194 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:36.194 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:36.195 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:36.195 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:36.196 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:36.196 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:36.196 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:36.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:36.197 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:36.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.197 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:36.198 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:36.198 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:36.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.199 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:36.200 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:36.200 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:36.200 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:36.200 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:36.200 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:36.201 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:36.201 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:36.201 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:36.201 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.202 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:36.202 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:36.202 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:36.203 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:36.203 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:36.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:36.203 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:36.203 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:36.204 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:36.204 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:36.204 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:36.204 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:36.204 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:36.205 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:36.205 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:36.205 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:36.205 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:36.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:36.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:36.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:36.206 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:36.207 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:36.207 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:36.207 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:36.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:36.208 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:36.208 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:36.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.209 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "51", 2023-08-25T09:41:36.210 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:36.210 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:36.210 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:36.211 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:36.211 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:36.211 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:36.211 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:36.212 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:36.212 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:36.212 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:36.213 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:36.213 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:36.213 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:36.213 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:36.214 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:36.214 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:36.214 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:36.214 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:36.215 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:36.215 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:36.215 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:36.216 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:36.216 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:36.216 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:36.216 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:36.217 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:36.218 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:36.219 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:36.219 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:36.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.219 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:36.220 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:36.220 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:36.220 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.221 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:36.221 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:36.221 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:36.221 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:36.222 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:36.222 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:36.222 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:36.223 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:36.223 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:36.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.224 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:36.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:36.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:36.225 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.225 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.225 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.225 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.226 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.226 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.226 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.226 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.227 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.227 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.227 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.227 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.229 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:36.229 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:36.231 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.232 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.232 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.233 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:36.233 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.235 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:36.235 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.235 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.236 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.236 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.237 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.237 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:36.237 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.238 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.238 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:36.238 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.239 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.239 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.240 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.241 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:36.241 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.241 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:36.242 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.244 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:36.244 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.244 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.244 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.245 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.245 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.246 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.246 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.246 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.246 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.247 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.247 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.247 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.248 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:36.248 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.249 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.249 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:36.250 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.250 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.251 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.251 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.251 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.252 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:36.252 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.253 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.253 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:36.253 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.253 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.253 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.254 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.254 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.255 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:36.255 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.258 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.260 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.261 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.264 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.265 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:36.266 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.267 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:36.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.269 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.269 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:36.269 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.271 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.272 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.273 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:36.274 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.274 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.274 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.275 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.275 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.276 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:36.276 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.277 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:36.277 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.278 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.279 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.279 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.280 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:36.280 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.281 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.281 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:36.281 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.282 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.283 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.283 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.283 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.283 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:36.284 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.285 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:36.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:36.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.285 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.286 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:36.286 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:36.365 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:36.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9c406a080 msgr2=0x7fc9c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:36.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9c406a080 0x7fc9c406c540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40b75b0 tx=0x7fc9d8001540 comp rx=0 tx=0).stop 2023-08-25T09:41:36.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 msgr2=0x7fc9e40b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9e40b40f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc9d402bf00 tx=0x7fc9d4004730 comp rx=0 tx=0).stop 2023-08-25T09:41:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 shutdown_connections 2023-08-25T09:41:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc9c406a080 0x7fc9c406c540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:36.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9e40b2ab0 0x7fc9e40b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9e4159ef0 0x7fc9e40b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 --2- 172.21.15.70:0/2882316329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9cc0039c0 0x7fc9e40b40f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 >> 172.21.15.70:0/2882316329 conn(0x7fc9e4070260 msgr2=0x7fc9e40cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 shutdown_connections 2023-08-25T09:41:36.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:36.144+0000 7fc9e8ca7640 1 -- 172.21.15.70:0/2882316329 wait complete. 2023-08-25T09:41:36.371 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:41:36.446 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:41:36.444+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-08-25T09:41:41.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:41.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.595+0000 7f8d5f4d7640 1 Processor -- start 2023-08-25T09:41:41.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.595+0000 7f8d5f4d7640 1 -- start start 2023-08-25T09:41:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5f4d7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8d580b12f0 0x7f8d5815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5f4d7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d5815f190 0x7f8d580cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5f4d7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5815e7c0 con 0x7f8d5815a7c0 2023-08-25T09:41:41.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5f4d7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d58071b50 con 0x7f8d5815f190 2023-08-25T09:41:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5f4d7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5815e490 con 0x7f8d580b12f0 2023-08-25T09:41:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5ecd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d5815f190 0x7f8d580cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:41.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5ecd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8d5815f190 0x7f8d580cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54696/0 (socket says 172.21.15.70:54696) 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.597+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/2102402821 learned_addr learned my addr 172.21.15.70:0/2102402821 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.598+0000 7f8d5dcd4640 1 --2- 172.21.15.70:0/2102402821 >> v2:172.21.15.155:3300/0 conn(0x7f8d580b12f0 0x7f8d5815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.598+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/2102402821 >> v2:172.21.15.155:3300/0 conn(0x7f8d580b12f0 msgr2=0x7f8d5815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.598+0000 7f8d5ecd6640 1 --2- 172.21.15.70:0/2102402821 >> v2:172.21.15.155:3300/0 conn(0x7f8d580b12f0 0x7f8d5815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.598+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/2102402821 >> v1:172.21.15.155:6789/0 conn(0x7f8d5815a7c0 legacy=0x7f8d580b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.598+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/2102402821 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580b3d60 con 0x7f8d5815f190 2023-08-25T09:41:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.598+0000 7f8d5dcd4640 1 --2- 172.21.15.70:0/2102402821 >> v2:172.21.15.155:3300/0 conn(0x7f8d580b12f0 0x7f8d5815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:41.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.599+0000 7f8d5ecd6640 1 --2- 172.21.15.70:0/2102402821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580cad20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8d580b0c50 tx=0x7f8d5002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e292834278101a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.599+0000 7f8d5d4d3640 1 -- 172.21.15.70:0/2102402821 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5002ca80 con 0x7f8d5815f190 2023-08-25T09:41:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.599+0000 7f8d5d4d3640 1 -- 172.21.15.70:0/2102402821 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5002cbe0 con 0x7f8d5815f190 2023-08-25T09:41:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.599+0000 7f8d5d4d3640 1 -- 172.21.15.70:0/2102402821 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d500343f0 con 0x7f8d5815f190 2023-08-25T09:41:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/2102402821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 msgr2=0x7f8d580cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/2102402821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580cad20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8d580b0c50 tx=0x7f8d5002bee0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/2102402821 shutdown_connections 2023-08-25T09:41:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/2102402821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580cad20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/2102402821 >> v2:172.21.15.155:3300/0 conn(0x7f8d580b12f0 0x7f8d5815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/2102402821 >> 172.21.15.70:0/2102402821 conn(0x7f8d580affd0 msgr2=0x7f8d580b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/2102402821 shutdown_connections 2023-08-25T09:41:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.600+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/2102402821 wait complete. 2023-08-25T09:41:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.602+0000 7f8d5f4d7640 1 Processor -- start 2023-08-25T09:41:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.602+0000 7f8d5f4d7640 1 -- start start 2023-08-25T09:41:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.603+0000 7f8d5f4d7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d580b12f0 0x7f8d5815d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5f4d7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d5815a7c0 0x7f8d580c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5f4d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5f4d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5815c190 con 0x7f8d5815f190 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5f4d7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580c55d0 con 0x7f8d5815a7c0 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5f4d7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5815d9f0 con 0x7f8d580b12f0 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54706/0 (socket says 172.21.15.70:54706) 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/3597288275 learned_addr learned my addr 172.21.15.70:0/3597288275 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5dcd4640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d5815a7c0 0x7f8d580c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5e4d5640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d580b12f0 0x7f8d5815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d580b12f0 msgr2=0x7f8d5815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d580b12f0 0x7f8d5815d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d5815a7c0 msgr2=0x7f8d580c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d5815a7c0 0x7f8d580c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.604+0000 7f8d5ecd6640 1 -- 172.21.15.70:0/3597288275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d50009b30 con 0x7f8d5815f190 2023-08-25T09:41:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.605+0000 7f8d5ecd6640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580c6560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8d50036040 tx=0x7f8d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.605+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d50009510 con 0x7f8d5815f190 2023-08-25T09:41:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.605+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5003e450 con 0x7f8d5815f190 2023-08-25T09:41:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.605+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5003e800 con 0x7f8d5815f190 2023-08-25T09:41:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.605+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5815dcd0 con 0x7f8d5815f190 2023-08-25T09:41:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.605+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d580c5b00 con 0x7f8d5815f190 2023-08-25T09:41:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.607+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d580b25f0 con 0x7f8d5815f190 2023-08-25T09:41:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.608+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8d5002cd50 con 0x7f8d5815f190 2023-08-25T09:41:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.609+0000 7f8d4effd640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d3c06a050 0x7f8d3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.609+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8d500ab0f0 con 0x7f8d5815f190 2023-08-25T09:41:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.609+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=56}) v3 -- 0x7f8d3c072200 con 0x7f8d5815f190 2023-08-25T09:41:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.609+0000 7f8d5e4d5640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d3c06a050 0x7f8d3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.610+0000 7f8d5e4d5640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d3c06a050 0x7f8d3c06c510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d54003d90 tx=0x7f8d54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:41.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.613+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8d50043060 con 0x7f8d5815f190 2023-08-25T09:41:41.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.801+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8d58066fb0 con 0x7f8d5815f190 2023-08-25T09:41:41.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.803+0000 7f8d4effd640 1 -- 172.21.15.70:0/3597288275 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+37346 (secure 0 0 0) 0x7f8d50076900 con 0x7f8d5815f190 2023-08-25T09:41:41.805 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 55, 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:39.489458+0000", 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:41.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:41.807 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:41.808 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:41.809 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:41:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:41.810 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:41.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:41.812 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:41.813 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:41.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:41.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:41.816 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:41.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:41.818 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:41.819 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:41.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:41.821 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:41.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:41.823 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:41.824 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:41.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:41.826 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:41.827 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:41.828 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:41.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:41.830 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:41.831 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:41.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.833 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:41.834 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:41.835 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:41.836 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:41.837 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:41.838 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:41.839 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:41.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:41.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:41.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:41.843 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:41.844 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:41.845 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:41.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:41.847 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:41.848 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:41.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:41.850 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:41.852 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:41.853 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:41.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:41.855 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:41.856 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:41.857 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:41.858 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:41.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:41.860 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:41.861 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:41.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:41.863 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:41.864 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:41.864 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:41.864 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:41.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.864 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:41.865 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:41.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.865 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:41.865 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.866 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 54, 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:41.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.868 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.871 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.872 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.874 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:41.874 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.876 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.876 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.878 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:41.879 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.879 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.879 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.879 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.882 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:41.882 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:41.882 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:41.882 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:41.882 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.883 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:41.883 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.883 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.883 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.884 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.886 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:41.888 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:41.891 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.893 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.894 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.895 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.899 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:41.903 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.904 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.905 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.906 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.907 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:41.908 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.911 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.912 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:41.913 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.915 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.916 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.918 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.919 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:41.923 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.925 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.926 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.927 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.928 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:41.929 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.930 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.930 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.930 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:41.931 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.932 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:41.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.934 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.935 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.936 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:41.937 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.940 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.941 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:41.943 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:41.944 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:41.945 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:41.946 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.947 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.948 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:41.949 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.952 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:41.954 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.955 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.956 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.957 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.958 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:41.959 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.960 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.961 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:41.962 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.964 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:41.965 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:41.966 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.9", 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.967 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:41.968 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:41.970 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:41.971 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:41.971 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:41.971 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:41.971 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:41.971 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:41.971 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d3c06a050 msgr2=0x7f8d3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d3c06a050 0x7f8d3c06c510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8d54003d90 tx=0x7f8d54009040 comp rx=0 tx=0).stop 2023-08-25T09:41:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 msgr2=0x7f8d580c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580c6560 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8d50036040 tx=0x7f8d50004a40 comp rx=0 tx=0).stop 2023-08-25T09:41:41.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 shutdown_connections 2023-08-25T09:41:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8d3c06a050 0x7f8d3c06c510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8d5815f190 0x7f8d580c6560 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8d5815a7c0 0x7f8d580c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 --2- 172.21.15.70:0/3597288275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8d580b12f0 0x7f8d5815d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 >> 172.21.15.70:0/3597288275 conn(0x7f8d580affd0 msgr2=0x7f8d580cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 shutdown_connections 2023-08-25T09:41:41.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:41.806+0000 7f8d5f4d7640 1 -- 172.21.15.70:0/3597288275 wait complete. 2023-08-25T09:41:41.976 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-08-25T09:41:41.976 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:41.976 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:42.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:42.035 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-08-25T09:41:42.239 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:41:42.237+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 5 full osd(s) (OSD_FULL) 2023-08-25T09:41:42.270 INFO:teuthology.orchestra.run.smithi070.stderr:dd: error writing 'large_file_b': No space left on device 2023-08-25T09:41:42.271 INFO:teuthology.orchestra.run.smithi070.stderr:dd: closing output file 'large_file_b': No space left on device 2023-08-25T09:41:42.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:41:42.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.485+0000 7f3432599640 1 Processor -- start 2023-08-25T09:41:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.485+0000 7f3432599640 1 -- start start 2023-08-25T09:41:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.486+0000 7f3432599640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f342c15aa10 0x7f342c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.486+0000 7f3432599640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f342c0b2680 0x7f342c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:42.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3432599640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0b2ad0 con 0x7f342c0ce5b0 2023-08-25T09:41:42.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3432599640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c159d10 con 0x7f342c15aa10 2023-08-25T09:41:42.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3432599640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c15d3f0 con 0x7f342c0b2680 2023-08-25T09:41:42.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f342c15aa10 0x7f342c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:42.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f342c15aa10 0x7f342c0b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54714/0 (socket says 172.21.15.70:54714) 2023-08-25T09:41:42.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 -- 172.21.15.70:0/585531785 learned_addr learned my addr 172.21.15.70:0/585531785 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:42.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 -- 172.21.15.70:0/585531785 >> v2:172.21.15.155:3300/0 conn(0x7f342c0b2680 msgr2=0x7f342c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 --2- 172.21.15.70:0/585531785 >> v2:172.21.15.155:3300/0 conn(0x7f342c0b2680 0x7f342c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 -- 172.21.15.70:0/585531785 >> v1:172.21.15.70:6789/0 conn(0x7f342c0ce5b0 legacy=0x7f342c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f342b7fe640 1 -- 172.21.15.70:0/585531785 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712849033 0 0) 0x7f342c0b2ad0 con 0x7f342c0ce5b0 2023-08-25T09:41:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.487+0000 7f3423fff640 1 -- 172.21.15.70:0/585531785 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0b2d20 con 0x7f342c15aa10 2023-08-25T09:41:42.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.488+0000 7f3423fff640 1 --2- 172.21.15.70:0/585531785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c15aa10 0x7f342c0b2140 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3414009870 tx=0x7f341402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9f7e5099be5467 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f342b7fe640 1 -- 172.21.15.70:0/585531785 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414031870 con 0x7f342c15aa10 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f342b7fe640 1 -- 172.21.15.70:0/585531785 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34140319d0 con 0x7f342c15aa10 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f342b7fe640 1 -- 172.21.15.70:0/585531785 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414031d60 con 0x7f342c15aa10 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f3432599640 1 -- 172.21.15.70:0/585531785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c15aa10 msgr2=0x7f342c0b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f3432599640 1 --2- 172.21.15.70:0/585531785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c15aa10 0x7f342c0b2140 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3414009870 tx=0x7f341402b910 comp rx=0 tx=0).stop 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f3432599640 1 -- 172.21.15.70:0/585531785 shutdown_connections 2023-08-25T09:41:42.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f3432599640 1 --2- 172.21.15.70:0/585531785 >> v2:172.21.15.155:3300/0 conn(0x7f342c0b2680 0x7f342c06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f3432599640 1 --2- 172.21.15.70:0/585531785 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c15aa10 0x7f342c0b2140 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.489+0000 7f3432599640 1 -- 172.21.15.70:0/585531785 >> 172.21.15.70:0/585531785 conn(0x7f342c069850 msgr2=0x7f342c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:42.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.490+0000 7f3432599640 1 -- 172.21.15.70:0/585531785 shutdown_connections 2023-08-25T09:41:42.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.490+0000 7f3432599640 1 -- 172.21.15.70:0/585531785 wait complete. 2023-08-25T09:41:42.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.491+0000 7f3432599640 1 Processor -- start 2023-08-25T09:41:42.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.491+0000 7f3432599640 1 -- start start 2023-08-25T09:41:42.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.492+0000 7f3432599640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 0x7f342c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:42.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.492+0000 7f3432599640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f342c0ce5b0 0x7f342c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:42.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.492+0000 7f3432599640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f342c15aa10 0x7f342c0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:42.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.492+0000 7f3432599640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c154360 con 0x7f342c0b2680 2023-08-25T09:41:42.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.492+0000 7f3432599640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c154e50 con 0x7f342c0ce5b0 2023-08-25T09:41:42.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.492+0000 7f3432599640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c1545b0 con 0x7f342c15aa10 2023-08-25T09:41:42.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 0x7f342c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:42.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 0x7f342c151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54728/0 (socket says 172.21.15.70:54728) 2023-08-25T09:41:42.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 -- 172.21.15.70:0/3105114790 learned_addr learned my addr 172.21.15.70:0/3105114790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:42.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f3430b36640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f342c15aa10 0x7f342c0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:42.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f3423fff640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f342c0ce5b0 0x7f342c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 -- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f342c15aa10 msgr2=0x7f342c0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f342c15aa10 0x7f342c0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 -- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f342c0ce5b0 msgr2=0x7f342c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f342c0ce5b0 0x7f342c157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 -- 172.21.15.70:0/3105114790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34140094d0 con 0x7f342c0b2680 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f3430b36640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f342c15aa10 0x7f342c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.493+0000 7f342bfff640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 0x7f342c151320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c4e0 tx=0x7f341c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:42.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.494+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c005480 con 0x7f342c0b2680 2023-08-25T09:41:42.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.494+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c0035f0 con 0x7f342c0b2680 2023-08-25T09:41:42.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.494+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c0039c0 con 0x7f342c0b2680 2023-08-25T09:41:42.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.494+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c154890 con 0x7f342c0b2680 2023-08-25T09:41:42.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.494+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c1f39c0 con 0x7f342c0b2680 2023-08-25T09:41:42.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.495+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c04e420 con 0x7f342c0b2680 2023-08-25T09:41:42.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.496+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f341c003b20 con 0x7f342c0b2680 2023-08-25T09:41:42.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.497+0000 7f34297fa640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f340c06a080 0x7f340c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:42.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.497+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f341c0879f0 con 0x7f342c0b2680 2023-08-25T09:41:42.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.497+0000 7f3423fff640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f340c06a080 0x7f340c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:42.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.497+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=56}) v3 -- 0x7f340c072230 con 0x7f342c0b2680 2023-08-25T09:41:42.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.498+0000 7f3423fff640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f340c06a080 0x7f340c06c540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3414036000 tx=0x7f341403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:42.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.502+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f341c017020 con 0x7f342c0b2680 2023-08-25T09:41:42.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.691+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f342c0cee50 con 0x7f342c0b2680 2023-08-25T09:41:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.695+0000 7f34297fa640 1 -- 172.21.15.70:0/3105114790 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+37346 (secure 0 0 0) 0x7f341c053160 con 0x7f342c0b2680 2023-08-25T09:41:42.699 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:42.699 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:42.700 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 55, 2023-08-25T09:41:42.700 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:42.701 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:42.701 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:39.489458+0000", 2023-08-25T09:41:42.701 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:42.701 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:42.701 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:42.701 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:42.702 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:42.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:42.702 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:42.702 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:42.702 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:42.702 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:42.703 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.704 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:42.705 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:42.705 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:42.705 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:42.705 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:42.705 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:42.706 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:42.706 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:42.706 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:42.706 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:42.706 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:42.707 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:42.708 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:42.708 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:42.709 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:42.709 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:42.709 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:42.710 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:42.710 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:42.710 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:42.711 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:42.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:42.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:42.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:42.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:42.712 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:42.713 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:42.713 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:42.713 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:42.713 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:42.713 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:42.713 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:42.714 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:42.714 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:42.714 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:42.714 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:42.714 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:42.714 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:42.715 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:42.715 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:42.715 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:42.715 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:42.715 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:42.716 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:42.716 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:42.716 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:42.716 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:42.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:42.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.717 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:42.717 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:42.717 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:42.717 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:42.717 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:42.717 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:42.718 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:42.719 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.721 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:42.721 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:42.721 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:42.721 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:42.721 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:42.722 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:42.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:42.724 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:42.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:42.724 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:42.724 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:42.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:42.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:42.726 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:42.726 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:42.726 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:42.726 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:42.726 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:42.727 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:42.728 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:42.729 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:42.730 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:42.731 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:42.732 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:42.733 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:42.734 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:42.735 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:42.736 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:42.737 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:42.738 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:42.739 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:42.746 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:42.747 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:42.748 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:42.749 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:42.750 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:42.751 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:42.752 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:42.753 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:42.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:42.755 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:42.757 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:42.758 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.759 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 54, 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.760 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.761 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:42.762 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.765 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:42.768 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.769 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.770 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.774 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.775 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:42.777 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:42.778 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.780 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.782 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.784 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.785 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:42.786 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:42.788 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.789 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.790 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.791 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:42.792 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.793 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:42.794 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.799 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.800 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.801 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.802 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.803 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.804 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.807 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:42.810 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.812 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.813 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.814 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.820 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:42.821 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.823 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:42.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:42.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:42.834 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:42.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.840 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.841 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.841 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:42.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:42.843 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:42.844 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:42.844 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:42.844 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:42.844 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:42.844 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:42.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.845 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:42.847 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.848 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:42.849 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.851 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:42.852 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.852 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:42.852 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.852 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.853 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:42.854 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:42.855 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:41:42.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.9", 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.857 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:42.858 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:42.860 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:42.861 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:42.861 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:42.861 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:42.861 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:42.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:42.861 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.697+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f340c06a080 msgr2=0x7f340c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.697+0000 7f3432599640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f340c06a080 0x7f340c06c540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3414036000 tx=0x7f341403c040 comp rx=0 tx=0).stop 2023-08-25T09:41:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.697+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 msgr2=0x7f342c151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.697+0000 7f3432599640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 0x7f342c151320 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f341c00c4e0 tx=0x7f341c00c9a0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 shutdown_connections 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f340c06a080 0x7f340c06c540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f342c15aa10 0x7f342c0ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f342c0ce5b0 0x7f342c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 --2- 172.21.15.70:0/3105114790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f342c0b2680 0x7f342c151320 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 >> 172.21.15.70:0/3105114790 conn(0x7f342c069850 msgr2=0x7f342c070980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 shutdown_connections 2023-08-25T09:41:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:42.698+0000 7f3432599640 1 -- 172.21.15.70:0/3105114790 wait complete. 2023-08-25T09:41:42.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:41:43.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.079+0000 7fedd5b0c640 1 Processor -- start 2023-08-25T09:41:43.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.080+0000 7fedd5b0c640 1 -- start start 2023-08-25T09:41:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.081+0000 7fedd5b0c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fedd015ec10 0x7fedd015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.081+0000 7fedd5b0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.081+0000 7fedd5b0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd00b1b60 con 0x7fedd015e830 2023-08-25T09:41:43.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.081+0000 7fedd5b0c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd006bd80 con 0x7fedd015ec10 2023-08-25T09:41:43.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.081+0000 7fedd5b0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd006b7c0 con 0x7fedd015c6d0 2023-08-25T09:41:43.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd00b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:41410/0 (socket says 172.21.15.70:41410) 2023-08-25T09:41:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 -- 172.21.15.70:0/2001013120 learned_addr learned my addr 172.21.15.70:0/2001013120 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedc6ffd640 1 --2- 172.21.15.70:0/2001013120 >> v2:172.21.15.155:3300/0 conn(0x7fedd015ec10 0x7fedd015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:43.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 -- 172.21.15.70:0/2001013120 >> v2:172.21.15.155:3300/0 conn(0x7fedd015ec10 msgr2=0x7fedd015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 --2- 172.21.15.70:0/2001013120 >> v2:172.21.15.155:3300/0 conn(0x7fedd015ec10 0x7fedd015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 -- 172.21.15.70:0/2001013120 >> v1:172.21.15.155:6789/0 conn(0x7fedd015e830 legacy=0x7fedd006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.082+0000 7fedcffff640 1 -- 172.21.15.70:0/2001013120 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd006baa0 con 0x7fedd015c6d0 2023-08-25T09:41:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.084+0000 7fedcffff640 1 --2- 172.21.15.70:0/2001013120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd00b1ff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009870 tx=0x7fedc802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1eca808a868435d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.084+0000 7fedceffd640 1 -- 172.21.15.70:0/2001013120 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8030840 con 0x7fedd015c6d0 2023-08-25T09:41:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.084+0000 7fedceffd640 1 -- 172.21.15.70:0/2001013120 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc80309a0 con 0x7fedd015c6d0 2023-08-25T09:41:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.084+0000 7fedceffd640 1 -- 172.21.15.70:0/2001013120 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8030d30 con 0x7fedd015c6d0 2023-08-25T09:41:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 -- 172.21.15.70:0/2001013120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 msgr2=0x7fedd00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:43.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/2001013120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd00b1ff0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009870 tx=0x7fedc802b980 comp rx=0 tx=0).stop 2023-08-25T09:41:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 -- 172.21.15.70:0/2001013120 shutdown_connections 2023-08-25T09:41:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/2001013120 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd00b1ff0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/2001013120 >> v2:172.21.15.155:3300/0 conn(0x7fedd015ec10 0x7fedd015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 -- 172.21.15.70:0/2001013120 >> 172.21.15.70:0/2001013120 conn(0x7fedd0069850 msgr2=0x7fedd0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 -- 172.21.15.70:0/2001013120 shutdown_connections 2023-08-25T09:41:43.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.085+0000 7fedd5b0c640 1 -- 172.21.15.70:0/2001013120 wait complete. 2023-08-25T09:41:43.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.087+0000 7fedd5b0c640 1 Processor -- start 2023-08-25T09:41:43.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.087+0000 7fedd5b0c640 1 -- start start 2023-08-25T09:41:43.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedd5b0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd01f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:43.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedd5b0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 0x7fedd01f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedd5b0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd015ec10 0x7fedd01f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedd5b0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd01f66a0 con 0x7fedd015e830 2023-08-25T09:41:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedd5b0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c8460 con 0x7fedd015ec10 2023-08-25T09:41:43.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedd5b0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd00c8740 con 0x7fedd015c6d0 2023-08-25T09:41:43.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedc6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 0x7fedd01f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedc6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 0x7fedd01f2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54738/0 (socket says 172.21.15.70:54738) 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedc6ffd640 1 -- 172.21.15.70:0/1937826941 learned_addr learned my addr 172.21.15.70:0/1937826941 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedcf7fe640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd01f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.088+0000 7fedcffff640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd015ec10 0x7fedd01f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedc6ffd640 1 -- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 msgr2=0x7fedd01f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedc6ffd640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd01f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedc6ffd640 1 -- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd015ec10 msgr2=0x7fedd01f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedc6ffd640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd015ec10 0x7fedd01f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedc6ffd640 1 -- 172.21.15.70:0/1937826941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80094d0 con 0x7fedd015e830 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedcffff640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd015ec10 0x7fedd01f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedc6ffd640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 0x7fedd01f2180 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fedb800efc0 tx=0x7fedb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8009e80 con 0x7fedd015e830 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb800f040 con 0x7fedd015e830 2023-08-25T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8014680 con 0x7fedd015e830 2023-08-25T09:41:43.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd00c8a20 con 0x7fedd015e830 2023-08-25T09:41:43.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.089+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd00c9770 con 0x7fedd015e830 2023-08-25T09:41:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.090+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd00c9930 con 0x7fedd015e830 2023-08-25T09:41:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.091+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fedb80147e0 con 0x7fedd015e830 2023-08-25T09:41:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.092+0000 7fedccff9640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fedb006a050 0x7fedb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:43.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.092+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fedb8089330 con 0x7fedd015e830 2023-08-25T09:41:43.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.093+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=56}) v3 -- 0x7fedb0072200 con 0x7fedd015e830 2023-08-25T09:41:43.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.093+0000 7fedcf7fe640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fedb006a050 0x7fedb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:43.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.093+0000 7fedcf7fe640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fedb006a050 0x7fedb006c510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedc0007fd0 tx=0x7fedc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:43.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.097+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fedb8058500 con 0x7fedd015e830 2023-08-25T09:41:43.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.323+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedd004e420 con 0x7fedd015e830 2023-08-25T09:41:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.324+0000 7fedccff9640 1 -- 172.21.15.70:0/1937826941 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7fedb8058090 con 0x7fedd015e830 2023-08-25T09:41:43.327 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:43.328 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:41:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:41:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fedb006a050 msgr2=0x7fedb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fedb006a050 0x7fedb006c510 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedc0007fd0 tx=0x7fedc0009040 comp rx=0 tx=0).stop 2023-08-25T09:41:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 msgr2=0x7fedd01f2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 0x7fedd01f2180 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fedb800efc0 tx=0x7fedb800c490 comp rx=0 tx=0).stop 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 shutdown_connections 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fedb006a050 0x7fedb006c510 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fedd015ec10 0x7fedd01f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fedd015e830 0x7fedd01f2180 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 --2- 172.21.15.70:0/1937826941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fedd015c6d0 0x7fedd01f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 >> 172.21.15.70:0/1937826941 conn(0x7fedd0069850 msgr2=0x7fedd00ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 shutdown_connections 2023-08-25T09:41:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:43.327+0000 7fedd5b0c640 1 -- 172.21.15.70:0/1937826941 wait complete. 2023-08-25T09:41:43.349 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:41:43.586 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":5835,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":30.008102209,"mdsmap_epoch":24,"osdmap_epoch":55,"osdmap_epoch_barrier":51,"uptime":36.232405507000003} 2023-08-25T09:41:43.587 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 5835, "mdsmap_epoch": 24, "osdmap_epoch": 55, "osdmap_epoch_barrier": 51, "rank_uptime": 30.008102209, "state": "up:active", "uptime": 36.232405507, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:41:43.587 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:41:43.588 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:43.588 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:43.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:43.644 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-08-25T09:41:43.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:41:43.693+0000 7fb40dffb640 -1 client.6000 handle_client_reply no pending request on tid 14 safe is:1 2023-08-25T09:41:43.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:41:43.697 INFO:teuthology.orchestra.run.smithi070.stderr:dd: failed to open 'small_file_1': Permission denied 2023-08-25T09:41:43.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:43.697 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-08-25T09:41:43.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:43.798 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-08-25T09:41:43.863 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:44.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.071+0000 7f8edddfd640 1 Processor -- start 2023-08-25T09:41:44.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.071+0000 7f8edddfd640 1 -- start start 2023-08-25T09:41:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.072+0000 7f8edddfd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8ed815aa10 0x7f8ed80b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.072+0000 7f8edddfd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8ed80b2680 0x7f8ed806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.072+0000 7f8edddfd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed80b2ad0 con 0x7f8ed80ce5b0 2023-08-25T09:41:44.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.072+0000 7f8edddfd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed8159d10 con 0x7f8ed80b2680 2023-08-25T09:41:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.072+0000 7f8edddfd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed815d3f0 con 0x7f8ed815aa10 2023-08-25T09:41:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8ed80ce5b0 0x7f8ed80ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:51616/0 (socket says 172.21.15.70:51616) 2023-08-25T09:41:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed77fe640 1 -- 172.21.15.70:0/916050183 learned_addr learned my addr 172.21.15.70:0/916050183 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:44.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed7fff640 1 --2- 172.21.15.70:0/916050183 >> v2:172.21.15.70:3300/0 conn(0x7f8ed80b2680 0x7f8ed806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:44.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed6ffd640 1 -- 172.21.15.70:0/916050183 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537094962 0 0) 0x7f8ed80b2ad0 con 0x7f8ed80ce5b0 2023-08-25T09:41:44.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ecffff640 1 --2- 172.21.15.70:0/916050183 >> v2:172.21.15.155:3300/0 conn(0x7f8ed815aa10 0x7f8ed80b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:44.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed6ffd640 1 -- 172.21.15.70:0/916050183 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb8005730 con 0x7f8ed80ce5b0 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed7fff640 1 -- 172.21.15.70:0/916050183 >> v2:172.21.15.155:3300/0 conn(0x7f8ed815aa10 msgr2=0x7f8ed80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed7fff640 1 --2- 172.21.15.70:0/916050183 >> v2:172.21.15.155:3300/0 conn(0x7f8ed815aa10 0x7f8ed80b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed7fff640 1 -- 172.21.15.70:0/916050183 >> v1:172.21.15.70:6789/0 conn(0x7f8ed80ce5b0 legacy=0x7f8ed80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ed7fff640 1 -- 172.21.15.70:0/916050183 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80b2d20 con 0x7f8ed80b2680 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.073+0000 7f8ecffff640 1 --2- 172.21.15.70:0/916050183 >> v2:172.21.15.155:3300/0 conn(0x7f8ed815aa10 0x7f8ed80b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.074+0000 7f8ed7fff640 1 --2- 172.21.15.70:0/916050183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed80b2680 0x7f8ed806f9f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009d90 tx=0x7f8ed002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbbbc755fb706c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:44.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8ed6ffd640 1 -- 172.21.15.70:0/916050183 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002cbd0 con 0x7f8ed80b2680 2023-08-25T09:41:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8ed6ffd640 1 -- 172.21.15.70:0/916050183 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed002cd30 con 0x7f8ed80b2680 2023-08-25T09:41:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8ed6ffd640 1 -- 172.21.15.70:0/916050183 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002cea0 con 0x7f8ed80b2680 2023-08-25T09:41:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 -- 172.21.15.70:0/916050183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed80b2680 msgr2=0x7f8ed806f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:44.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 --2- 172.21.15.70:0/916050183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed80b2680 0x7f8ed806f9f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009d90 tx=0x7f8ed002be20 comp rx=0 tx=0).stop 2023-08-25T09:41:44.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 -- 172.21.15.70:0/916050183 shutdown_connections 2023-08-25T09:41:44.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 --2- 172.21.15.70:0/916050183 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed80b2680 0x7f8ed806f9f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 --2- 172.21.15.70:0/916050183 >> v2:172.21.15.155:3300/0 conn(0x7f8ed815aa10 0x7f8ed80b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 -- 172.21.15.70:0/916050183 >> 172.21.15.70:0/916050183 conn(0x7f8ed8069850 msgr2=0x7f8ed8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:44.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 -- 172.21.15.70:0/916050183 shutdown_connections 2023-08-25T09:41:44.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.075+0000 7f8edddfd640 1 -- 172.21.15.70:0/916050183 wait complete. 2023-08-25T09:41:44.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.077+0000 7f8edddfd640 1 Processor -- start 2023-08-25T09:41:44.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.077+0000 7f8edddfd640 1 -- start start 2023-08-25T09:41:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8edddfd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ed80b2680 0x7f8ed814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8edddfd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80ce5b0 0x7f8ed8150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8edddfd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 0x7f8ed80b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8edddfd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81489f0 con 0x7f8ed815aa10 2023-08-25T09:41:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8edddfd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80b0820 con 0x7f8ed80ce5b0 2023-08-25T09:41:44.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8edddfd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80b0b00 con 0x7f8ed80b2680 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.078+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 0x7f8ed80b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 0x7f8ed80b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54758/0 (socket says 172.21.15.70:54758) 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 -- 172.21.15.70:0/346439602 learned_addr learned my addr 172.21.15.70:0/346439602 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 -- 172.21.15.70:0/346439602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ed80b2680 msgr2=0x7f8ed814cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ed80b2680 0x7f8ed814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 -- 172.21.15.70:0/346439602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80ce5b0 msgr2=0x7f8ed8150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80ce5b0 0x7f8ed8150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 -- 172.21.15.70:0/346439602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed0009a70 con 0x7f8ed815aa10 2023-08-25T09:41:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed7fff640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 0x7f8ed80b0120 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009d90 tx=0x7f8ed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.079+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002cea0 con 0x7f8ed815aa10 2023-08-25T09:41:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.080+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed0040650 con 0x7f8ed815aa10 2023-08-25T09:41:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.080+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0040a00 con 0x7f8ed815aa10 2023-08-25T09:41:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.080+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed80b0de0 con 0x7f8ed815aa10 2023-08-25T09:41:44.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.080+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed80b13a0 con 0x7f8ed815aa10 2023-08-25T09:41:44.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.081+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed804e420 con 0x7f8ed815aa10 2023-08-25T09:41:44.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.082+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8ed0040b60 con 0x7f8ed815aa10 2023-08-25T09:41:44.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.083+0000 7f8ed4ff9640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8ebc06a050 0x7f8ebc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.083+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8ed00ab270 con 0x7f8ed815aa10 2023-08-25T09:41:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.083+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=56}) v3 -- 0x7f8ebc072200 con 0x7f8ed815aa10 2023-08-25T09:41:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.083+0000 7f8ed77fe640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8ebc06a050 0x7f8ebc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:44.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.084+0000 7f8ed77fe640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8ebc06a050 0x7f8ebc06c510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001610 tx=0x7f8ec8007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:44.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.087+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8ed00768a0 con 0x7f8ed815aa10 2023-08-25T09:41:44.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:41:44.217 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:41:44.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.277+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8ed80cae50 con 0x7f8ed815aa10 2023-08-25T09:41:44.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.278+0000 7f8ed4ff9640 1 -- 172.21.15.70:0/346439602 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+37346 (secure 0 0 0) 0x7f8ed0039090 con 0x7f8ed815aa10 2023-08-25T09:41:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:44.282 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 55, 2023-08-25T09:41:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:39.489458+0000", 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:41:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 54, 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:44.348 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.348 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.348 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:44.349 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.350 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.351 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.356 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.356 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.356 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.357 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.358 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.358 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:44.359 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:44.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.360 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:44.360 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:44.360 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:44.361 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:44.361 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.361 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.362 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:44.362 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.362 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.362 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:44.363 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.363 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.363 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.363 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.363 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.363 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.364 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.364 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.364 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.364 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.364 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.364 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.366 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.366 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:44.366 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.366 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.367 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.368 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.368 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.368 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:44.369 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.371 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:44.374 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.376 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:44.376 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.376 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.376 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.376 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.377 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.379 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.381 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.382 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.384 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:44.385 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.386 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.387 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.388 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:44.389 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.390 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.392 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.392 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.392 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.394 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:44.394 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.394 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.395 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:44.395 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:44.397 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.398 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.399 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.403 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.404 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:44.406 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:44.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.407 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.407 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:44.407 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.408 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.409 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.410 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.411 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.412 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:44.413 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.414 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.415 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.417 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:44.418 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:44.419 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:44.420 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:44.424 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:44.426 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:44.427 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:44.428 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.429 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:44.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:44.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.430 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:44.430 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.430 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.430 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:44.432 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.433 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.434 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:44.435 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.437 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.438 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.439 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.439 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:44.439 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.439 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:44.440 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.9", 2023-08-25T09:41:44.441 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.442 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:44.443 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:44.444 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:44.444 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:44.444 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:44.444 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:44.444 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:44.444 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:44.445 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:44.445 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:44.445 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:44.445 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:44.445 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:44.446 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:44.446 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:44.446 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:44.446 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:44.446 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:44.447 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:44.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:44.447 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:44.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8ebc06a050 msgr2=0x7f8ebc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:44.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8ebc06a050 0x7f8ebc06c510 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8001610 tx=0x7f8ec8007930 comp rx=0 tx=0).stop 2023-08-25T09:41:44.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 msgr2=0x7f8ed80b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 0x7f8ed80b0120 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009d90 tx=0x7f8ed0004a40 comp rx=0 tx=0).stop 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 shutdown_connections 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8ebc06a050 0x7f8ebc06c510 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ed815aa10 0x7f8ed80b0120 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ed80ce5b0 0x7f8ed8150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 --2- 172.21.15.70:0/346439602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8ed80b2680 0x7f8ed814cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:44.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 >> 172.21.15.70:0/346439602 conn(0x7f8ed8069850 msgr2=0x7f8ed8070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:44.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 shutdown_connections 2023-08-25T09:41:44.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:44.281+0000 7f8edddfd640 1 -- 172.21.15.70:0/346439602 wait complete. 2023-08-25T09:41:44.451 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:41:47.241 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:41:47.239+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-08-25T09:41:49.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.687+0000 7f425ec06640 1 Processor -- start 2023-08-25T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.688+0000 7f425ec06640 1 -- start start 2023-08-25T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.689+0000 7f425ec06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815e830 0x7f425806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.689+0000 7f425ec06640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f425806b2b0 0x7f425815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.689+0000 7f425ec06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f425815c6d0 0x7f42580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.689+0000 7f425ec06640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425806bd80 con 0x7f425815c6d0 2023-08-25T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425ec06640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580b1b60 con 0x7f425806b2b0 2023-08-25T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425ec06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425806b7c0 con 0x7f425815e830 2023-08-25T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f425815c6d0 0x7f42580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f425815c6d0 0x7f42580b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49640/0 (socket says 172.21.15.70:49640) 2023-08-25T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 -- 172.21.15.70:0/1890580818 learned_addr learned my addr 172.21.15.70:0/1890580818 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f4257fff640 1 --2- 172.21.15.70:0/1890580818 >> v2:172.21.15.155:3300/0 conn(0x7f425806b2b0 0x7f425815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425c9a2640 1 --2- 172.21.15.70:0/1890580818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815e830 0x7f425806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 -- 172.21.15.70:0/1890580818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815e830 msgr2=0x7f425806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 --2- 172.21.15.70:0/1890580818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815e830 0x7f425806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 -- 172.21.15.70:0/1890580818 >> v2:172.21.15.155:3300/0 conn(0x7f425806b2b0 msgr2=0x7f425815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 --2- 172.21.15.70:0/1890580818 >> v2:172.21.15.155:3300/0 conn(0x7f425806b2b0 0x7f425815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.690+0000 7f425d1a3640 1 -- 172.21.15.70:0/1890580818 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425806baa0 con 0x7f425815c6d0 2023-08-25T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.691+0000 7f425d1a3640 1 --2- 172.21.15.70:0/1890580818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425815c6d0 0x7f42580b1ff0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4258071120 tx=0x7f425002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765d4923f367171 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f42577fe640 1 -- 172.21.15.70:0/1890580818 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425002ca80 con 0x7f425815c6d0 2023-08-25T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f42577fe640 1 -- 172.21.15.70:0/1890580818 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425002cbe0 con 0x7f425815c6d0 2023-08-25T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f42577fe640 1 -- 172.21.15.70:0/1890580818 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42500343f0 con 0x7f425815c6d0 2023-08-25T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 -- 172.21.15.70:0/1890580818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425815c6d0 msgr2=0x7f42580b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 --2- 172.21.15.70:0/1890580818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425815c6d0 0x7f42580b1ff0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4258071120 tx=0x7f425002bee0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 -- 172.21.15.70:0/1890580818 shutdown_connections 2023-08-25T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 --2- 172.21.15.70:0/1890580818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425815c6d0 0x7f42580b1ff0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 --2- 172.21.15.70:0/1890580818 >> v2:172.21.15.155:3300/0 conn(0x7f425806b2b0 0x7f425815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 --2- 172.21.15.70:0/1890580818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815e830 0x7f425806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.692+0000 7f425ec06640 1 -- 172.21.15.70:0/1890580818 >> 172.21.15.70:0/1890580818 conn(0x7f4258069850 msgr2=0x7f4258069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.693+0000 7f425ec06640 1 -- 172.21.15.70:0/1890580818 shutdown_connections 2023-08-25T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.693+0000 7f425ec06640 1 -- 172.21.15.70:0/1890580818 wait complete. 2023-08-25T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.694+0000 7f425ec06640 1 Processor -- start 2023-08-25T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.694+0000 7f425ec06640 1 -- start start 2023-08-25T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.695+0000 7f425ec06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 0x7f4258146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.695+0000 7f425ec06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815c6d0 0x7f42580ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.695+0000 7f425ec06640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f425815e830 0x7f425806fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.695+0000 7f425ec06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42581476a0 con 0x7f425806b2b0 2023-08-25T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.695+0000 7f425ec06640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258147390 con 0x7f425815e830 2023-08-25T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.695+0000 7f425ec06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4258147940 con 0x7f425815c6d0 2023-08-25T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 0x7f4258146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 0x7f4258146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49656/0 (socket says 172.21.15.70:49656) 2023-08-25T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 -- 172.21.15.70:0/1212439757 learned_addr learned my addr 172.21.15.70:0/1212439757 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 -- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815c6d0 msgr2=0x7f42580ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815c6d0 0x7f42580ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 -- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f425815e830 msgr2=0x7f425806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425d1a3640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f425815e830 0x7f425806fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f425815e830 0x7f425806fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 -- 172.21.15.70:0/1212439757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250009b30 con 0x7f425806b2b0 2023-08-25T09:41:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f425c9a2640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 0x7f4258146e50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c7f0 tx=0x7f424c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:49.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.696+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c004220 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.697+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c004380 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.697+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c007840 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.697+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258147c20 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.697+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42581481e0 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.698+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425804e420 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.699+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f424c007bb0 con 0x7f425806b2b0 2023-08-25T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.700+0000 7f42557fa640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423c06a050 0x7f423c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:49.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.700+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f424c088530 con 0x7f425806b2b0 2023-08-25T09:41:49.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.700+0000 7f4257fff640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423c06a050 0x7f423c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:49.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.700+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=59}) v3 -- 0x7f423c072200 con 0x7f425806b2b0 2023-08-25T09:41:49.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.701+0000 7f4257fff640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423c06a050 0x7f423c06c510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4258156260 tx=0x7f4244008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:49.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.704+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f424c053bf0 con 0x7f425806b2b0 2023-08-25T09:41:49.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.894+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f4258156900 con 0x7f425806b2b0 2023-08-25T09:41:50.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.898+0000 7f42557fa640 1 -- 172.21.15.70:0/1212439757 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) v1 ==== 81+0+37242 (secure 0 0 0) 0x7f424c01e030 con 0x7f425806b2b0 2023-08-25T09:41:50.222 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:50.222 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 58, 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:49.247767+0000", 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:50.223 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:50.224 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:50.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:50.226 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:50.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:50.228 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:50.229 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:50.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:50.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:50.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:50.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:50.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:50.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:50.230 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:50.230 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:50.230 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:50.230 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:50.230 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:50.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:50.232 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:50.233 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:50.234 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:50.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:50.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:50.237 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:50.238 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:50.238 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:50.239 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:50.239 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:50.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:50.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.241 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:50.242 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:50.242 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:50.242 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:50.243 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:50.243 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:50.243 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:50.243 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:50.244 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:50.244 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:50.245 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:50.247 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:50.248 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:50.249 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:50.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.249 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:50.250 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:50.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.250 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:50.250 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:50.252 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:50.252 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:50.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:50.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:50.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:50.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.258 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:50.258 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:50.258 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:50.258 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:50.258 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:50.258 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:50.259 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:50.260 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:50.260 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:50.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:50.260 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:50.260 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:50.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:50.261 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:50.261 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:50.261 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:50.261 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:50.261 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:50.262 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:50.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:50.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.262 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:50.262 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:50.263 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:50.263 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:50.263 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:50.263 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:50.263 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:50.263 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:50.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:50.265 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:50.266 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:50.266 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:50.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.266 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:50.267 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:50.268 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:50.268 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:50.268 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:50.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:50.276 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:50.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:50.278 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:50.279 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:50.280 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:50.281 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:50.282 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:50.284 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:50.285 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:50.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "52", 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:50.289 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:50.290 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:50.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.292 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:50.293 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.294 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:50.295 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.296 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.297 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 54, 2023-08-25T09:41:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.300 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.302 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:50.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:50.308 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.309 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.310 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:50.317 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.318 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.319 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:50.320 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.321 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.321 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.321 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.321 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.322 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.323 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:50.339 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.339 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.339 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull", 2023-08-25T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.9", 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2023-08-25T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423c06a050 msgr2=0x7f423c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423c06a050 0x7f423c06c510 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4258156260 tx=0x7f4244008040 comp rx=0 tx=0).stop 2023-08-25T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 msgr2=0x7f4258146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 0x7f4258146e50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f424c00c7f0 tx=0x7f424c00ccb0 comp rx=0 tx=0).stop 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 shutdown_connections 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423c06a050 0x7f423c06c510 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f425815e830 0x7f425806fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425815c6d0 0x7f42580ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.900+0000 7f425ec06640 1 --2- 172.21.15.70:0/1212439757 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425806b2b0 0x7f4258146e50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.901+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 >> 172.21.15.70:0/1212439757 conn(0x7f4258069850 msgr2=0x7f42580b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.901+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 shutdown_connections 2023-08-25T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:49.901+0000 7f425ec06640 1 -- 172.21.15.70:0/1212439757 wait complete. 2023-08-25T09:41:50.420 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:41:52.243 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:41:52.241+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-08-25T09:41:55.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.646+0000 7f99b8806640 1 Processor -- start 2023-08-25T09:41:55.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.647+0000 7f99b8806640 1 -- start start 2023-08-25T09:41:55.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.648+0000 7f99b8806640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f99b415ec10 0x7f99b415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:55.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.648+0000 7f99b8806640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f99b415c6d0 0x7f99b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:55.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.648+0000 7f99b8806640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99b40b1b60 con 0x7f99b415e830 2023-08-25T09:41:55.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.648+0000 7f99b8806640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b406bd80 con 0x7f99b415c6d0 2023-08-25T09:41:55.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.648+0000 7f99b8806640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99b406b7c0 con 0x7f99b415ec10 2023-08-25T09:41:55.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f99b415e830 0x7f99b406ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55330/0 (socket says 172.21.15.70:55330) 2023-08-25T09:41:55.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b259c640 1 -- 172.21.15.70:0/2018189563 learned_addr learned my addr 172.21.15.70:0/2018189563 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:55.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b2d9d640 1 --2- 172.21.15.70:0/2018189563 >> v2:172.21.15.70:3300/0 conn(0x7f99b415c6d0 0x7f99b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b1d9b640 1 -- 172.21.15.70:0/2018189563 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1785685838 0 0) 0x7f99b40b1b60 con 0x7f99b415e830 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b1d9b640 1 -- 172.21.15.70:0/2018189563 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99a0003520 con 0x7f99b415e830 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99abfff640 1 --2- 172.21.15.70:0/2018189563 >> v2:172.21.15.155:3300/0 conn(0x7f99b415ec10 0x7f99b415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b2d9d640 1 -- 172.21.15.70:0/2018189563 >> v2:172.21.15.155:3300/0 conn(0x7f99b415ec10 msgr2=0x7f99b415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b2d9d640 1 --2- 172.21.15.70:0/2018189563 >> v2:172.21.15.155:3300/0 conn(0x7f99b415ec10 0x7f99b415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b2d9d640 1 -- 172.21.15.70:0/2018189563 >> v1:172.21.15.70:6789/0 conn(0x7f99b415e830 legacy=0x7f99b406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99b2d9d640 1 -- 172.21.15.70:0/2018189563 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b406baa0 con 0x7f99b415c6d0 2023-08-25T09:41:55.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.649+0000 7f99abfff640 1 --2- 172.21.15.70:0/2018189563 >> v2:172.21.15.155:3300/0 conn(0x7f99b415ec10 0x7f99b415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:55.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.650+0000 7f99b2d9d640 1 --2- 172.21.15.70:0/2018189563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415c6d0 0x7f99b40b1ff0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f99b40cf6f0 tx=0x7f99ac02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccdb47f16d4d126 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b1d9b640 1 -- 172.21.15.70:0/2018189563 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac02cbe0 con 0x7f99b415c6d0 2023-08-25T09:41:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b1d9b640 1 -- 172.21.15.70:0/2018189563 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac02cd40 con 0x7f99b415c6d0 2023-08-25T09:41:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b1d9b640 1 -- 172.21.15.70:0/2018189563 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac034640 con 0x7f99b415c6d0 2023-08-25T09:41:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 -- 172.21.15.70:0/2018189563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415c6d0 msgr2=0x7f99b40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 --2- 172.21.15.70:0/2018189563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415c6d0 0x7f99b40b1ff0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f99b40cf6f0 tx=0x7f99ac02be30 comp rx=0 tx=0).stop 2023-08-25T09:41:55.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 -- 172.21.15.70:0/2018189563 shutdown_connections 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 --2- 172.21.15.70:0/2018189563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415c6d0 0x7f99b40b1ff0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 --2- 172.21.15.70:0/2018189563 >> v2:172.21.15.155:3300/0 conn(0x7f99b415ec10 0x7f99b415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 -- 172.21.15.70:0/2018189563 >> 172.21.15.70:0/2018189563 conn(0x7f99b4069850 msgr2=0x7f99b4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 -- 172.21.15.70:0/2018189563 shutdown_connections 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.651+0000 7f99b8806640 1 -- 172.21.15.70:0/2018189563 wait complete. 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.653+0000 7f99b8806640 1 Processor -- start 2023-08-25T09:41:55.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.653+0000 7f99b8806640 1 -- start start 2023-08-25T09:41:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99b8806640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b415c6d0 0x7f99b4155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99b8806640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 0x7f99b41f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99b8806640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b415ec10 0x7f99b40cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:55.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99b8806640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b406f030 con 0x7f99b415e830 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99b8806640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b406ece0 con 0x7f99b415c6d0 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99b8806640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b41f4bc0 con 0x7f99b415ec10 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.654+0000 7f99abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 0x7f99b41f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 0x7f99b41f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49688/0 (socket says 172.21.15.70:49688) 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 -- 172.21.15.70:0/4269301809 learned_addr learned my addr 172.21.15.70:0/4269301809 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 -- 172.21.15.70:0/4269301809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b415ec10 msgr2=0x7f99b40cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b415ec10 0x7f99b40cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 -- 172.21.15.70:0/4269301809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b415c6d0 msgr2=0x7f99b4155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b415c6d0 0x7f99b4155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 -- 172.21.15.70:0/4269301809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac009a70 con 0x7f99b415e830 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99abfff640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 0x7f99b41f4340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c820 tx=0x7f999c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c004230 con 0x7f99b415e830 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999c004390 con 0x7f99b415e830 2023-08-25T09:41:55.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999c007a70 con 0x7f99b415e830 2023-08-25T09:41:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.655+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99b41f4ea0 con 0x7f99b415e830 2023-08-25T09:41:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.656+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b41558c0 con 0x7f99b415e830 2023-08-25T09:41:55.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.657+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99b404e420 con 0x7f99b415e830 2023-08-25T09:41:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.658+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f999c007bd0 con 0x7f99b415e830 2023-08-25T09:41:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.658+0000 7f99aaffd640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f999406a050 0x7f999406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:55.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.659+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f999c088620 con 0x7f99b415e830 2023-08-25T09:41:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.659+0000 7f99b259c640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f999406a050 0x7f999406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:55.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.659+0000 7f99b259c640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f999406a050 0x7f999406c510 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000f60 tx=0x7f99a40078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:55.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.663+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f999c053c50 con 0x7f99b415e830 2023-08-25T09:41:55.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.854+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f99b415e2b0 con 0x7f99b415e830 2023-08-25T09:41:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.858+0000 7f99aaffd640 1 -- 172.21.15.70:0/4269301809 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v61) v1 ==== 81+0+37142 (secure 0 0 0) 0x7f999c057270 con 0x7f99b415e830 2023-08-25T09:41:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:55.861 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 61, 2023-08-25T09:41:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:55.861 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:53.340729+0000", 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:55.862 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:55.863 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:55.864 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:55.865 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:55.866 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:55.867 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:55.867 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:55.867 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:55.867 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:55.867 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:55.867 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:55.868 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:55.868 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:55.868 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:55.868 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:55.868 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:55.868 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:55.869 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:55.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.869 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:55.869 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:55.870 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:55.870 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:55.870 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:55.870 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:55.870 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:55.871 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:55.871 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:55.871 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:55.871 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:55.871 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:55.872 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:55.873 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:55.873 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:55.873 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:55.873 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:55.874 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:55.875 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:55.876 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:55.876 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:55.876 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:55.876 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:55.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.876 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:55.877 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:55.879 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:55.880 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:55.880 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:55.881 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:55.881 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:55.881 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:55.881 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:55.881 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:55.881 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:55.882 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:55.882 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:55.882 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:55.882 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:55.882 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:55.883 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:55.884 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:55.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.884 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:55.884 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:55.884 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:55.884 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:55.885 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:55.885 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:55.885 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:55.885 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:55.885 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:55.885 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:55.886 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:55.887 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:55.888 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:55.888 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:55.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.888 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:55.888 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:55.888 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:55.889 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:55.889 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:55.889 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:55.889 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:55.889 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:55.890 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:55.890 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:55.891 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:55.891 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:55.891 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:55.891 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:55.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.891 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:55.892 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:55.892 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:55.892 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:55.892 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:55.892 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:55.892 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:55.893 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:55.894 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:55.895 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:55.896 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:55.897 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:55.898 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:55.899 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:55.900 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:55.900 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:55.900 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:55.900 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:55.900 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:55.901 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:55.901 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:55.901 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:55.902 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:55.903 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:55.904 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:55.905 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.906 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:55.907 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:55.908 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:55.909 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:55.910 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.911 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:55.912 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:55.913 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:55.914 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:55.915 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:55.916 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:55.917 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:55.918 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.919 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:55.920 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 54, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:55.921 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:55.922 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:55.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.924 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.926 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.927 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:55.928 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.929 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:55.930 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:55.931 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:55.932 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.934 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.935 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:55.936 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.938 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.940 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:55.941 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:55.943 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:55.945 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.946 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.947 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:55.948 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.949 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:55.952 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:55.953 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:55.954 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:55.956 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.957 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.959 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:55.961 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:55.967 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:55.967 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.967 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:55.968 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:55.969 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.972 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.973 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.976 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.977 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.979 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:55.980 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:55.981 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:55.982 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:55.983 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:55.984 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:55.985 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:55.986 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:55.990 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:55.993 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.994 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.995 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:55.996 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:55.997 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:55.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:55.999 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.000 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:56.001 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.002 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.003 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.006 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:56.009 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.010 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:56.011 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:56.012 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.014 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.016 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.018 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:56.018 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.018 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.018 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.018 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.019 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.019 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.019 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.019 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.020 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.020 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.020 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.020 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.021 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.021 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:56.021 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.022 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.022 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:56.022 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.023 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.023 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.023 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.023 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.024 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:56.025 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.025 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.025 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.025 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.025 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.026 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.026 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:56.026 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.027 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:56.027 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.027 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.028 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.028 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.028 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.029 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.029 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:56.029 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.030 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.030 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:56.030 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.030 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.031 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:56.031 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:56.031 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:56.031 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:56.031 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.032 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.032 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.032 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.032 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.032 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.033 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.034 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.034 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.034 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:56.034 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.035 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:56.035 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.035 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.035 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.035 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.036 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.036 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:56.036 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.037 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:56.037 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.037 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.037 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.037 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.038 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.038 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:56.038 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.039 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:56.039 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.039 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.040 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.040 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.040 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.040 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:56.041 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.041 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:56.042 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.042 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.042 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.042 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.042 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.042 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.043 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:56.045 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.045 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.046 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.046 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.046 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.046 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:56.047 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.047 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:56.048 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.048 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.049 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.049 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.049 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.049 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:41:56.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.051 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.9", 2023-08-25T09:41:56.051 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:41:56.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.052 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2023-08-25T09:41:56.052 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:41:56.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.054 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.054 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.055 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:56.055 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:56.055 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:56.055 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:56.055 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:56.056 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:56.056 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:56.056 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:56.056 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:56.056 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:56.056 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:56.057 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:56.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.057 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:56.057 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:56.058 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:56.058 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:56.058 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:56.059 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:56.059 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:56.059 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:56.059 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:56.060 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:56.060 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:56.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.060 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:56.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f999406a050 msgr2=0x7f999406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f999406a050 0x7f999406c510 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000f60 tx=0x7f99a40078f0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 msgr2=0x7f99b41f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 0x7f99b41f4340 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f999c00c820 tx=0x7f999c00cce0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 shutdown_connections 2023-08-25T09:41:56.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f999406a050 0x7f999406c510 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99b415ec10 0x7f99b40cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99b415e830 0x7f99b41f4340 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 --2- 172.21.15.70:0/4269301809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99b415c6d0 0x7f99b4155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 >> 172.21.15.70:0/4269301809 conn(0x7f99b4069850 msgr2=0x7f99b40cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:56.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 shutdown_connections 2023-08-25T09:41:56.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:55.861+0000 7f99b8806640 1 -- 172.21.15.70:0/4269301809 wait complete. 2023-08-25T09:41:56.066 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-08-25T09:41:56.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:41:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.289+0000 7f83ba7c8640 1 Processor -- start 2023-08-25T09:41:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.289+0000 7f83ba7c8640 1 -- start start 2023-08-25T09:41:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.291+0000 7f83ba7c8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f83b415e830 0x7f83b406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.291+0000 7f83ba7c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 0x7f83b415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.291+0000 7f83ba7c8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b40b1b60 con 0x7f83b415c6d0 2023-08-25T09:41:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.291+0000 7f83ba7c8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b406bd80 con 0x7f83b415e830 2023-08-25T09:41:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.291+0000 7f83ba7c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b406b7c0 con 0x7f83b406b2b0 2023-08-25T09:41:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.291+0000 7f83b37fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 0x7f83b415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b3fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f83b415e830 0x7f83b406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b37fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 0x7f83b415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:50356/0 (socket says 172.21.15.70:50356) 2023-08-25T09:41:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b37fe640 1 -- 172.21.15.70:0/2953646689 learned_addr learned my addr 172.21.15.70:0/2953646689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b37fe640 1 -- 172.21.15.70:0/2953646689 >> v2:172.21.15.155:3300/0 conn(0x7f83b415e830 msgr2=0x7f83b406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b37fe640 1 --2- 172.21.15.70:0/2953646689 >> v2:172.21.15.155:3300/0 conn(0x7f83b415e830 0x7f83b406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b37fe640 1 -- 172.21.15.70:0/2953646689 >> v1:172.21.15.155:6789/0 conn(0x7f83b415c6d0 legacy=0x7f83b40b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b37fe640 1 -- 172.21.15.70:0/2953646689 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b406baa0 con 0x7f83b406b2b0 2023-08-25T09:41:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.292+0000 7f83b3fff640 1 --2- 172.21.15.70:0/2953646689 >> v2:172.21.15.155:3300/0 conn(0x7f83b415e830 0x7f83b406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:41:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.294+0000 7f83b37fe640 1 --2- 172.21.15.70:0/2953646689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 0x7f83b415c190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009870 tx=0x7f83a802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67127cba4156ba7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.294+0000 7f83b2ffd640 1 -- 172.21.15.70:0/2953646689 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8030840 con 0x7f83b406b2b0 2023-08-25T09:41:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.294+0000 7f83b2ffd640 1 -- 172.21.15.70:0/2953646689 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a80309a0 con 0x7f83b406b2b0 2023-08-25T09:41:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.294+0000 7f83b2ffd640 1 -- 172.21.15.70:0/2953646689 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a8030d30 con 0x7f83b406b2b0 2023-08-25T09:41:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 -- 172.21.15.70:0/2953646689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 msgr2=0x7f83b415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/2953646689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 0x7f83b415c190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f83a8009870 tx=0x7f83a802b980 comp rx=0 tx=0).stop 2023-08-25T09:41:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 -- 172.21.15.70:0/2953646689 shutdown_connections 2023-08-25T09:41:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/2953646689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b406b2b0 0x7f83b415c190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/2953646689 >> v2:172.21.15.155:3300/0 conn(0x7f83b415e830 0x7f83b406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 -- 172.21.15.70:0/2953646689 >> 172.21.15.70:0/2953646689 conn(0x7f83b4069850 msgr2=0x7f83b4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 -- 172.21.15.70:0/2953646689 shutdown_connections 2023-08-25T09:41:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.295+0000 7f83ba7c8640 1 -- 172.21.15.70:0/2953646689 wait complete. 2023-08-25T09:41:56.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.297+0000 7f83ba7c8640 1 Processor -- start 2023-08-25T09:41:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.297+0000 7f83ba7c8640 1 -- start start 2023-08-25T09:41:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.298+0000 7f83ba7c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 0x7f83b4146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.298+0000 7f83ba7c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b415c6d0 0x7f83b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.298+0000 7f83ba7c8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b415e830 0x7f83b40b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.298+0000 7f83ba7c8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40aee60 con 0x7f83b406b2b0 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.298+0000 7f83ba7c8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40af280 con 0x7f83b415e830 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.298+0000 7f83ba7c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b40ae1e0 con 0x7f83b415c6d0 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 0x7f83b4146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 0x7f83b4146220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49690/0 (socket says 172.21.15.70:49690) 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 -- 172.21.15.70:0/1676067098 learned_addr learned my addr 172.21.15.70:0/1676067098 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b8d65640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b415e830 0x7f83b40b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b37fe640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b415c6d0 0x7f83b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 -- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b415c6d0 msgr2=0x7f83b40b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b415c6d0 0x7f83b40b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 -- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b415e830 msgr2=0x7f83b40b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b415e830 0x7f83b40b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b3fff640 1 -- 172.21.15.70:0/1676067098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009400 con 0x7f83b406b2b0 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.299+0000 7f83b37fe640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b415c6d0 0x7f83b40b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.300+0000 7f83b3fff640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 0x7f83b4146220 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e960 tx=0x7f83a400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:56.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.300+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400ca00 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.300+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a400cb60 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.300+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a4013470 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.300+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a80094d0 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.300+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b40ae7a0 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.301+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b40af0d0 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.302+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f83a40135d0 con 0x7f83b406b2b0 2023-08-25T09:41:56.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.303+0000 7f83b0ff9640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83a006a050 0x7f83a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.303+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f83a4088860 con 0x7f83b406b2b0 2023-08-25T09:41:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.303+0000 7f83b37fe640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83a006a050 0x7f83a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.304+0000 7f83b37fe640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83a006a050 0x7f83a006c510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f83a8036000 tx=0x7f83a803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:56.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.308+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f83a4053f20 con 0x7f83b406b2b0 2023-08-25T09:41:56.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.501+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f83b404e420 con 0x7f83b406b2b0 2023-08-25T09:41:56.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.503+0000 7f83b0ff9640 1 -- 172.21.15.70:0/1676067098 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v61) v1 ==== 81+0+37142 (secure 0 0 0) 0x7f83a4057540 con 0x7f83b406b2b0 2023-08-25T09:41:56.506 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:56.506 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:41:56.506 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 61, 2023-08-25T09:41:56.506 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:41:56.506 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:41:56.506 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:41:53.340729+0000", 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:41:56.507 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:41:56.508 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:41:56.508 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:41:56.508 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.508 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:41:56.508 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:41:56.508 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:41:56.509 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:56.510 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:56.511 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:41:56.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:41:56.513 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:41:56.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:56.513 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.514 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:56.515 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:56.516 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:56.517 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:56.518 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:56.518 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:56.520 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:56.520 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:56.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.521 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:56.521 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:56.524 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:56.524 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:56.524 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:56.524 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:56.524 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:41:56.525 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:41:56.526 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:56.528 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:56.529 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:56.530 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:56.531 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:56.532 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:56.533 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:56.534 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:56.535 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:56.536 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:41:56.537 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:41:56.538 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:41:56.539 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:41:56.539 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:56.539 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:56.539 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:56.539 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2023-08-25T09:41:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:41:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.244136+0000", 2023-08-25T09:41:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:56.540 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:56.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:56.544 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:56.544 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:56.545 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:56.547 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:56.547 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:56.547 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:56.547 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:56.547 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:56.548 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:56.549 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:56.550 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:56.550 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:56.550 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:56.550 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:56.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.550 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:56.551 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:56.551 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:56.551 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:56.551 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:56.551 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:56.552 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:56.552 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:56.553 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:56.553 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:56.553 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:56.553 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:56.553 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:56.553 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:56.554 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:56.554 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:56.554 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:56.554 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:56.554 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:56.554 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:56.555 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:56.555 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:56.555 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:56.555 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:56.555 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:56.555 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:56.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:56.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.556 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:56.556 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:56.556 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:56.557 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:56.557 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:56.557 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:56.557 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:56.557 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:56.557 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:56.558 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:56.558 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:56.558 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:56.559 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:41:56.559 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:41:56.559 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:41:56.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.559 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:56.560 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:56.560 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:41:56.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.560 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:56.560 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:56.561 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:56.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.562 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2023-08-25T09:41:56.562 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:41:56.562 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:41:11.990910+0000", 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:41:56.563 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:41:56.564 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:41:56.565 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:41:56.566 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:41:56.566 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:41:56.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.567 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2023-08-25T09:41:56.567 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:41:56.567 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:41:56.567 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:41:56.567 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:41:56.568 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:41:56.568 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:41:56.568 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:41:56.568 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:41:56.568 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:41:56.569 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:41:56.569 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:41:56.569 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:41:56.569 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:41:56.569 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:41:56.570 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:41:56.571 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:41:56.572 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:41:56.573 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:41:56.574 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:41:56.574 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:41:56.574 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:41:56.574 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:41:56.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.574 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:41:56.575 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:41:56.575 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:41:56.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.576 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:41:56.576 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2023-08-25T09:41:56.576 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2023-08-25T09:41:56.576 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:41:56.576 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2023-08-25T09:41:56.577 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2023-08-25T09:41:56.577 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:41:56.577 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:41:56.577 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:41:56.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.578 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.578 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:41:56.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.578 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:56.578 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.579 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.579 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.580 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.580 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.580 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.580 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 54, 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.582 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.582 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:41:56.582 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.585 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:41:56.586 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.586 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.587 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.587 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.587 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.588 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:41:56.593 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.594 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:41:56.594 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.594 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.594 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.595 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.595 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.595 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.595 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:41:56.596 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.596 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.596 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:41:56.597 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.597 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.597 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.597 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.597 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.598 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:41:56.598 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.599 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.600 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.601 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:56.601 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.601 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.601 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.602 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.602 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.602 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.602 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.604 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.605 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:41:56.606 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.607 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.608 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.609 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.610 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.611 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.612 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.613 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:41:56.614 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.617 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.617 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:41:56.617 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:41:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.619 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.622 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.623 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.624 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.624 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.627 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.628 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.629 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.629 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.629 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.630 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.630 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.631 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:41:56.631 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.632 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.632 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:41:56.632 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.633 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.635 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.635 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.635 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:41:56.636 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.637 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.637 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:41:56.638 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.638 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.639 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.640 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.640 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.641 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.641 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:41:56.641 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.642 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:41:56.642 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.643 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.643 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.643 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.644 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:41:56.645 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:41:56.646 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.646 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.646 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.647 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.650 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:41:56.652 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.652 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.653 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:41:56.653 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.653 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.653 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.655 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.656 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.656 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.656 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:41:56.656 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.671 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.673 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:56.673 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.673 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.673 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.673 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.674 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.679 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:41:56.679 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:41:56.682 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.682 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.684 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.685 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.686 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.687 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:41:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.689 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.693 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:41:56.695 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:41:56.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 49, 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.699 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:41:56.699 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:41:56.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.702 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:41:56.702 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.708 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:41:56.709 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.709 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.709 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.711 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:41:56.711 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.711 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.712 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:41:56.713 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:41:56.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:41:56.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:41:56.716 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:41:56.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.716 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.716 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:41:56.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:41:56.717 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:41:56.717 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:41:56.718 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:41:56.719 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:41:56.719 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:41:56.719 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.719 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.719 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.720 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:41:56.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.720 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:41:56.720 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.720 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.721 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:41:56.722 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.722 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.722 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.722 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.722 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.722 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.723 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.724 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:41:56.727 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.728 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:41:56.728 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.728 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.728 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.729 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.729 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.729 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:41:56.729 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.730 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:41:56.730 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.730 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.730 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.731 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:41:56.732 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.732 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.732 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.732 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.732 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.732 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.733 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.734 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:41:56.735 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.9", 2023-08-25T09:41:56.736 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.737 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:41:56.738 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:41:56.740 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:41:56.740 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:41:56.740 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:41:56.748 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:41:56.748 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:41:56.749 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:41:56.750 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:41:56.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83a006a050 msgr2=0x7f83a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83a006a050 0x7f83a006c510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f83a8036000 tx=0x7f83a803b040 comp rx=0 tx=0).stop 2023-08-25T09:41:56.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 msgr2=0x7f83b4146220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 0x7f83b4146220 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f83a400e960 tx=0x7f83a400ee20 comp rx=0 tx=0).stop 2023-08-25T09:41:56.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 shutdown_connections 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83a006a050 0x7f83a006c510 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83b415e830 0x7f83b40b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83b415c6d0 0x7f83b40b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 --2- 172.21.15.70:0/1676067098 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83b406b2b0 0x7f83b4146220 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 >> 172.21.15.70:0/1676067098 conn(0x7f83b4069850 msgr2=0x7f83b40cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.506+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 shutdown_connections 2023-08-25T09:41:56.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.507+0000 7f83ba7c8640 1 -- 172.21.15.70:0/1676067098 wait complete. 2023-08-25T09:41:56.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:41:56.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.975+0000 7f5af960d640 1 Processor -- start 2023-08-25T09:41:56.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.975+0000 7f5af960d640 1 -- start start 2023-08-25T09:41:56.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af960d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40cea30 0x7f5af40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af960d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5af40ac870 0x7f5af40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af960d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af406f050 con 0x7f5af415cb40 2023-08-25T09:41:56.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af960d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af406f330 con 0x7f5af40ac870 2023-08-25T09:41:56.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af960d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af406f610 con 0x7f5af40cea30 2023-08-25T09:41:56.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5af40ac870 0x7f5af40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5af40ac870 0x7f5af40b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49704/0 (socket says 172.21.15.70:49704) 2023-08-25T09:41:56.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5af37fe640 1 -- 172.21.15.70:0/1225369809 learned_addr learned my addr 172.21.15.70:0/1225369809 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:56.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.977+0000 7f5aebfff640 1 --2- 172.21.15.70:0/1225369809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40cea30 0x7f5af40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.978+0000 7f5af37fe640 1 -- 172.21.15.70:0/1225369809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40cea30 msgr2=0x7f5af40b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:56.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.978+0000 7f5af37fe640 1 --2- 172.21.15.70:0/1225369809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40cea30 0x7f5af40b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.978+0000 7f5af37fe640 1 -- 172.21.15.70:0/1225369809 >> v1:172.21.15.155:6789/0 conn(0x7f5af415cb40 legacy=0x7f5af40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:56.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.978+0000 7f5af37fe640 1 -- 172.21.15.70:0/1225369809 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af406f8f0 con 0x7f5af40ac870 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.979+0000 7f5af37fe640 1 --2- 172.21.15.70:0/1225369809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af40ac870 0x7f5af40b2070 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e50 tx=0x7f5aec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568409a19319cc86 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.979+0000 7f5af27fc640 1 -- 172.21.15.70:0/1225369809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec02ca70 con 0x7f5af40ac870 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.979+0000 7f5af27fc640 1 -- 172.21.15.70:0/1225369809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec02cbd0 con 0x7f5af40ac870 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.979+0000 7f5af27fc640 1 -- 172.21.15.70:0/1225369809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec0343f0 con 0x7f5af40ac870 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 -- 172.21.15.70:0/1225369809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af40ac870 msgr2=0x7f5af40b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 --2- 172.21.15.70:0/1225369809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af40ac870 0x7f5af40b2070 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009e50 tx=0x7f5aec02bed0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 -- 172.21.15.70:0/1225369809 shutdown_connections 2023-08-25T09:41:56.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 --2- 172.21.15.70:0/1225369809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af40ac870 0x7f5af40b2070 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 --2- 172.21.15.70:0/1225369809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40cea30 0x7f5af40b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 -- 172.21.15.70:0/1225369809 >> 172.21.15.70:0/1225369809 conn(0x7f5af4069850 msgr2=0x7f5af4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:56.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 -- 172.21.15.70:0/1225369809 shutdown_connections 2023-08-25T09:41:56.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.980+0000 7f5af960d640 1 -- 172.21.15.70:0/1225369809 wait complete. 2023-08-25T09:41:56.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.982+0000 7f5af960d640 1 Processor -- start 2023-08-25T09:41:56.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.982+0000 7f5af960d640 1 -- start start 2023-08-25T09:41:56.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af960d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40ac870 0x7f5af4158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af960d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5af40cea30 0x7f5af41451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af960d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 0x7f5af4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af960d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4158210 con 0x7f5af415cb40 2023-08-25T09:41:56.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af960d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4157590 con 0x7f5af40cea30 2023-08-25T09:41:56.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af960d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4157870 con 0x7f5af40ac870 2023-08-25T09:41:56.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 0x7f5af4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 0x7f5af4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49712/0 (socket says 172.21.15.70:49712) 2023-08-25T09:41:56.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.983+0000 7f5af37fe640 1 -- 172.21.15.70:0/190627500 learned_addr learned my addr 172.21.15.70:0/190627500 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:56.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af37fe640 1 -- 172.21.15.70:0/190627500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40ac870 msgr2=0x7f5af4158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af37fe640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40ac870 0x7f5af4158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af37fe640 1 -- 172.21.15.70:0/190627500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5af40cea30 msgr2=0x7f5af41451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:56.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af37fe640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5af40cea30 0x7f5af41451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:56.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af37fe640 1 -- 172.21.15.70:0/190627500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec009b30 con 0x7f5af415cb40 2023-08-25T09:41:56.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af37fe640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 0x7f5af4156bb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5aec036040 tx=0x7f5aec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:56.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec009510 con 0x7f5af415cb40 2023-08-25T09:41:56.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aec03e450 con 0x7f5af415cb40 2023-08-25T09:41:56.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af4157b50 con 0x7f5af415cb40 2023-08-25T09:41:56.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.984+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aec03e7e0 con 0x7f5af415cb40 2023-08-25T09:41:56.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.985+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af41570f0 con 0x7f5af415cb40 2023-08-25T09:41:56.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.986+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4157340 con 0x7f5af415cb40 2023-08-25T09:41:56.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.987+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5aec03e940 con 0x7f5af415cb40 2023-08-25T09:41:56.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.988+0000 7f5aeb7fe640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5ad406a050 0x7f5ad406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:56.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.988+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f5aec0ab3a0 con 0x7f5af415cb40 2023-08-25T09:41:56.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.989+0000 7f5af2ffd640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5ad406a050 0x7f5ad406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:56.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.989+0000 7f5af2ffd640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5ad406a050 0x7f5ad406c510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5af4155e80 tx=0x7f5ae40079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:56.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:56.992+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5aec076a60 con 0x7f5af415cb40 2023-08-25T09:41:57.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.217+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5af4158350 con 0x7f5af415cb40 2023-08-25T09:41:57.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.218+0000 7f5aeb7fe640 1 -- 172.21.15.70:0/190627500 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f5af4158350 con 0x7f5af415cb40 2023-08-25T09:41:57.221 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:57.222 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:41:57.222 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:41:57.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5ad406a050 msgr2=0x7f5ad406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:57.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5ad406a050 0x7f5ad406c510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5af4155e80 tx=0x7f5ae40079b0 comp rx=0 tx=0).stop 2023-08-25T09:41:57.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 msgr2=0x7f5af4156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:57.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 0x7f5af4156bb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5aec036040 tx=0x7f5aec004a40 comp rx=0 tx=0).stop 2023-08-25T09:41:57.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 shutdown_connections 2023-08-25T09:41:57.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5ad406a050 0x7f5ad406c510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:57.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5af415cb40 0x7f5af4156bb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:57.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5af40cea30 0x7f5af41451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:57.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 --2- 172.21.15.70:0/190627500 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5af40ac870 0x7f5af4158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:57.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.221+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 >> 172.21.15.70:0/190627500 conn(0x7f5af4069850 msgr2=0x7f5af4154290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:57.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.222+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 shutdown_connections 2023-08-25T09:41:57.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:57.222+0000 7f5af960d640 1 -- 172.21.15.70:0/190627500 wait complete. 2023-08-25T09:41:57.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:41:57.477 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":5835,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":43.898432796999998,"mdsmap_epoch":24,"osdmap_epoch":61,"osdmap_epoch_barrier":55,"uptime":50.122772007000002} 2023-08-25T09:41:57.477 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 5835, "mdsmap_epoch": 24, "osdmap_epoch": 61, "osdmap_epoch_barrier": 55, "rank_uptime": 43.898432797, "state": "up:active", "uptime": 50.122772007, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:41:57.478 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:41:57.478 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:57.478 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:57.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:41:57.533 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-08-25T09:41:58.341 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records in 2023-08-25T09:41:58.341 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records out 2023-08-25T09:41:58.341 INFO:teuthology.orchestra.run.smithi070.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.748988 s, 70.0 MB/s 2023-08-25T09:41:58.343 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:41:58.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:41:58.468 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:41:58.595 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:41:58.596 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:41:58.596 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:41:58.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:41:58.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.863+0000 7fc8269ab640 1 Processor -- start 2023-08-25T09:41:58.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.863+0000 7fc8269ab640 1 -- start start 2023-08-25T09:41:58.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.864+0000 7fc8269ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc82015ca00 0x7fc8200cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:58.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.864+0000 7fc8269ab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc8200acd50 0x7fc8200ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:58.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.864+0000 7fc8269ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc820160680 con 0x7fc8200c5120 2023-08-25T09:41:58.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.864+0000 7fc8269ab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc820160960 con 0x7fc82015ca00 2023-08-25T09:41:58.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.864+0000 7fc8269ab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc820160c40 con 0x7fc8200acd50 2023-08-25T09:41:58.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc82015ca00 0x7fc8200cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:58.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc82015ca00 0x7fc8200cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49722/0 (socket says 172.21.15.70:49722) 2023-08-25T09:41:58.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 -- 172.21.15.70:0/2442307647 learned_addr learned my addr 172.21.15.70:0/2442307647 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:58.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8251a8640 1 --1- 172.21.15.70:0/2442307647 >> v1:172.21.15.70:6789/0 conn(0x7fc8200c5120 0x7fc82006ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:55346/0 (socket says 172.21.15.70:55346) 2023-08-25T09:41:58.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 -- 172.21.15.70:0/2442307647 >> v2:172.21.15.155:3300/0 conn(0x7fc8200acd50 msgr2=0x7fc8200ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:58.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 --2- 172.21.15.70:0/2442307647 >> v2:172.21.15.155:3300/0 conn(0x7fc8200acd50 0x7fc8200ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:58.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 -- 172.21.15.70:0/2442307647 >> v1:172.21.15.70:6789/0 conn(0x7fc8200c5120 legacy=0x7fc82006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:58.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.865+0000 7fc8259a9640 1 -- 172.21.15.70:0/2442307647 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820160f20 con 0x7fc82015ca00 2023-08-25T09:41:58.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.866+0000 7fc8259a9640 1 --2- 172.21.15.70:0/2442307647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200cceb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc8200b4610 tx=0x7fc81c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d41af2451b3e2f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:58.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.866+0000 7fc8249a7640 1 -- 172.21.15.70:0/2442307647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c030860 con 0x7fc82015ca00 2023-08-25T09:41:58.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8249a7640 1 -- 172.21.15.70:0/2442307647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81c0309c0 con 0x7fc82015ca00 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8249a7640 1 -- 172.21.15.70:0/2442307647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c030d50 con 0x7fc82015ca00 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 -- 172.21.15.70:0/2442307647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 msgr2=0x7fc8200cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 --2- 172.21.15.70:0/2442307647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200cceb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc8200b4610 tx=0x7fc81c02b9a0 comp rx=0 tx=0).stop 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 -- 172.21.15.70:0/2442307647 shutdown_connections 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 --2- 172.21.15.70:0/2442307647 >> v2:172.21.15.155:3300/0 conn(0x7fc8200acd50 0x7fc8200ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 --2- 172.21.15.70:0/2442307647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200cceb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:58.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 -- 172.21.15.70:0/2442307647 >> 172.21.15.70:0/2442307647 conn(0x7fc8200affd0 msgr2=0x7fc8200b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:58.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 -- 172.21.15.70:0/2442307647 shutdown_connections 2023-08-25T09:41:58.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.867+0000 7fc8269ab640 1 -- 172.21.15.70:0/2442307647 wait complete. 2023-08-25T09:41:58.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.869+0000 7fc8269ab640 1 Processor -- start 2023-08-25T09:41:58.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.869+0000 7fc8269ab640 1 -- start start 2023-08-25T09:41:58.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8269ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200acd50 0x7fc820159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:58.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8269ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200c5120 0x7fc8200ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:58.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8269ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:58.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8269ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82015a0d0 con 0x7fc82015ca00 2023-08-25T09:41:58.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8269ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82015ea70 con 0x7fc8200acd50 2023-08-25T09:41:58.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8269ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82015ed50 con 0x7fc8200c5120 2023-08-25T09:41:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8261aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8261aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:49736/0 (socket says 172.21.15.70:49736) 2023-08-25T09:41:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.870+0000 7fc8261aa640 1 -- 172.21.15.70:0/4149698548 learned_addr learned my addr 172.21.15.70:0/4149698548 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8261aa640 1 -- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200c5120 msgr2=0x7fc8200ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8261aa640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200c5120 0x7fc8200ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8261aa640 1 -- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200acd50 msgr2=0x7fc820159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8259a9640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200acd50 0x7fc820159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:58.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8261aa640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200acd50 0x7fc820159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8261aa640 1 -- 172.21.15.70:0/4149698548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0094d0 con 0x7fc82015ca00 2023-08-25T09:41:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8261aa640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200ae380 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc81800b9e0 tx=0x7fc81800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:58.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800cb20 con 0x7fc82015ca00 2023-08-25T09:41:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81800cca0 con 0x7fc82015ca00 2023-08-25T09:41:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818014680 con 0x7fc82015ca00 2023-08-25T09:41:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.871+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82015f030 con 0x7fc82015ca00 2023-08-25T09:41:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.872+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc82015f5f0 con 0x7fc82015ca00 2023-08-25T09:41:58.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.873+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc820066fb0 con 0x7fc82015ca00 2023-08-25T09:41:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.874+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc81801d3b0 con 0x7fc82015ca00 2023-08-25T09:41:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.875+0000 7fc8167fc640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.875+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc818088530 con 0x7fc82015ca00 2023-08-25T09:41:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.875+0000 7fc8259a9640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:58.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.876+0000 7fc8259a9640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc81c036000 tx=0x7fc81c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:58.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:58.880+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc818053c00 con 0x7fc82015ca00 2023-08-25T09:41:59.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.068+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8200cd790 con 0x7fc82015ca00 2023-08-25T09:41:59.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.072+0000 7fc8167fc640 1 -- 172.21.15.70:0/4149698548 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18353 (secure 0 0 0) 0x7fc818057220 con 0x7fc82015ca00 2023-08-25T09:41:59.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:41:59.074 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":61,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:53.340729+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.9","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:41:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 msgr2=0x7fc80406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc81c036000 tx=0x7fc81c03b040 comp rx=0 tx=0).stop 2023-08-25T09:41:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 msgr2=0x7fc8200ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200ae380 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc81800b9e0 tx=0x7fc81800bea0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 shutdown_connections 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015ca00 0x7fc8200ae380 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200c5120 0x7fc8200ade40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 --2- 172.21.15.70:0/4149698548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200acd50 0x7fc820159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 >> 172.21.15.70:0/4149698548 conn(0x7fc8200affd0 msgr2=0x7fc8200c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.074+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 shutdown_connections 2023-08-25T09:41:59.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.075+0000 7fc8269ab640 1 -- 172.21.15.70:0/4149698548 wait complete. 2023-08-25T09:41:59.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:41:11.244136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:41:11.990910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:41:59.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:41:59.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.313+0000 7f95048f9640 1 Processor -- start 2023-08-25T09:41:59.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.313+0000 7f95048f9640 1 -- start start 2023-08-25T09:41:59.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.314+0000 7f95048f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 0x7f95000c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f95048f9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95000b2170 con 0x7f950015a920 2023-08-25T09:41:59.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f95048f9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95000adbb0 con 0x7f95000b1ce0 2023-08-25T09:41:59.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f95048f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95000c90a0 con 0x7f95000b0320 2023-08-25T09:41:59.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f94fe59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f950015a920 0x7f95000b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36178/0 (socket says 172.21.15.70:36178) 2023-08-25T09:41:59.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f94fe59c640 1 -- 172.21.15.70:0/1432063854 learned_addr learned my addr 172.21.15.70:0/1432063854 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:41:59.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f94fed9d640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 0x7f95000c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.315+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168656268 0 0) 0x7f95000b2170 con 0x7f950015a920 2023-08-25T09:41:59.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e8003520 con 0x7f950015a920 2023-08-25T09:41:59.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089685838 0 0) 0x7f94e8003520 con 0x7f950015a920 2023-08-25T09:41:59.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 msgr2=0x7f95000c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:59.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 0x7f95000c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 >> v1:172.21.15.155:6789/0 conn(0x7f95000b1ce0 legacy=0x7f95000ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95000c9380 con 0x7f950015a920 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f94f4001670 con 0x7f950015a920 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 >> v1:172.21.15.70:6789/0 conn(0x7f950015a920 legacy=0x7f95000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80039a0 0x7f94e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80047c0 0x7f94e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 0x7f94e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95000c90a0 con 0x7f94e80047c0 2023-08-25T09:41:59.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e800ba30 con 0x7f94e80039a0 2023-08-25T09:41:59.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.316+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e800bbf0 con 0x7f95000b0320 2023-08-25T09:41:59.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fe59c640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80047c0 0x7f94e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fed9d640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80039a0 0x7f94e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fe59c640 1 -- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 msgr2=0x7f94e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:59.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fe59c640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95000b0320 0x7f94e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fe59c640 1 -- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80039a0 msgr2=0x7f94e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:59.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fe59c640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80039a0 0x7f94e8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fe59c640 1 -- 172.21.15.70:0/1432063854 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e8003520 con 0x7f94e80047c0 2023-08-25T09:41:59.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.317+0000 7f94fed9d640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80039a0 0x7f94e8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:41:59.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.318+0000 7f94fe59c640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80047c0 0x7f94e8004c40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c210 tx=0x7f94e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708c8b2873da3f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:59.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.318+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4005ad0 con 0x7f94e80047c0 2023-08-25T09:41:59.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.318+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f4005c30 con 0x7f94e80047c0 2023-08-25T09:41:59.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.318+0000 7f94fdd9b640 1 -- 172.21.15.70:0/1432063854 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f402c870 con 0x7f94e80047c0 2023-08-25T09:41:59.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f95048f9640 1 -- 172.21.15.70:0/1432063854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80047c0 msgr2=0x7f94e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f95048f9640 1 --2- 172.21.15.70:0/1432063854 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80047c0 0x7f94e8004c40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f94f400c210 tx=0x7f94e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f94ff59e640 1 -- 172.21.15.70:0/1432063854 reap_dead start 2023-08-25T09:41:59.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f95048f9640 1 -- 172.21.15.70:0/1432063854 shutdown_connections 2023-08-25T09:41:59.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f95048f9640 1 -- 172.21.15.70:0/1432063854 >> 172.21.15.70:0/1432063854 conn(0x7f95000ce830 msgr2=0x7f95000cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:59.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f95048f9640 1 -- 172.21.15.70:0/1432063854 shutdown_connections 2023-08-25T09:41:59.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.319+0000 7f95048f9640 1 -- 172.21.15.70:0/1432063854 wait complete. 2023-08-25T09:41:59.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.321+0000 7f95048f9640 1 Processor -- start 2023-08-25T09:41:59.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.321+0000 7f95048f9640 1 -- start start 2023-08-25T09:41:59.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f95048f9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80047c0 0x7f950015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f95048f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9500154310 0x7f95001f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f95048f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 0x7f95001f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f95048f9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95001f10e0 con 0x7f9500154790 2023-08-25T09:41:59.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f95048f9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950015de10 con 0x7f94e80047c0 2023-08-25T09:41:59.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f95048f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95001523e0 con 0x7f9500154310 2023-08-25T09:41:59.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 0x7f95001f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 0x7f95001f0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45644/0 (socket says 172.21.15.70:45644) 2023-08-25T09:41:59.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 -- 172.21.15.70:0/2635083796 learned_addr learned my addr 172.21.15.70:0/2635083796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:59.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94fe59c640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9500154310 0x7f95001f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94fed9d640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80047c0 0x7f950015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 -- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9500154310 msgr2=0x7f95001f0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9500154310 0x7f95001f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 -- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80047c0 msgr2=0x7f950015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80047c0 0x7f950015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.322+0000 7f94ff59e640 1 -- 172.21.15.70:0/2635083796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95001517d0 con 0x7f9500154790 2023-08-25T09:41:59.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.323+0000 7f94ff59e640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 0x7f95001f0ba0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f950009c330 tx=0x7f94f800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:59.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.323+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f8005d40 con 0x7f9500154790 2023-08-25T09:41:59.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.323+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f800ced0 con 0x7f9500154790 2023-08-25T09:41:59.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.323+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800a850 con 0x7f9500154790 2023-08-25T09:41:59.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.323+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9500151ab0 con 0x7f9500154790 2023-08-25T09:41:59.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.323+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9500152070 con 0x7f9500154790 2023-08-25T09:41:59.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.325+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9500152570 con 0x7f9500154790 2023-08-25T09:41:59.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.326+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f94f800a9b0 con 0x7f9500154790 2023-08-25T09:41:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.327+0000 7f94e77fe640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94ec06a080 0x7f94ec06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.327+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f94f8087450 con 0x7f9500154790 2023-08-25T09:41:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.327+0000 7f94fed9d640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94ec06a080 0x7f94ec06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.328+0000 7f94fed9d640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94ec06a080 0x7f94ec06c540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94f00099d0 tx=0x7f94f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:59.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.331+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94f8052be0 con 0x7f9500154790 2023-08-25T09:41:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.524+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9500061460 con 0x7f9500154790 2023-08-25T09:41:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.525+0000 7f94e77fe640 1 -- 172.21.15.70:0/2635083796 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94f8056200 con 0x7f9500154790 2023-08-25T09:41:59.527 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:41:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94ec06a080 msgr2=0x7f94ec06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94ec06a080 0x7f94ec06c540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f94f00099d0 tx=0x7f94f0008040 comp rx=0 tx=0).stop 2023-08-25T09:41:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 msgr2=0x7f95001f0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 0x7f95001f0ba0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f950009c330 tx=0x7f94f800cb90 comp rx=0 tx=0).stop 2023-08-25T09:41:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 shutdown_connections 2023-08-25T09:41:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94ec06a080 0x7f94ec06c540 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9500154790 0x7f95001f0ba0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9500154310 0x7f95001f0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 --2- 172.21.15.70:0/2635083796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80047c0 0x7f950015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 >> 172.21.15.70:0/2635083796 conn(0x7f95000ce830 msgr2=0x7f95000b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 shutdown_connections 2023-08-25T09:41:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.528+0000 7f95048f9640 1 -- 172.21.15.70:0/2635083796 wait complete. 2023-08-25T09:41:59.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:41:59.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.759+0000 7fb79b59e640 1 Processor -- start 2023-08-25T09:41:59.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.759+0000 7fb79b59e640 1 -- start start 2023-08-25T09:41:59.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.760+0000 7fb79b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 0x7fb79c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.760+0000 7fb79b59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb79c0c8d20 0x7fb79c0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.760+0000 7fb79b59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb79c0cd800 con 0x7fb79c0b2470 2023-08-25T09:41:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.760+0000 7fb79b59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79c15ad20 con 0x7fb79c0c8d20 2023-08-25T09:41:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.760+0000 7fb79b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c15a110 con 0x7fb79c0b2850 2023-08-25T09:41:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.761+0000 7fb793fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 0x7fb79c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.761+0000 7fb793fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 0x7fb79c0cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46790/0 (socket says 172.21.15.70:46790) 2023-08-25T09:41:59.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.761+0000 7fb793fff640 1 -- 172.21.15.70:0/1392163450 learned_addr learned my addr 172.21.15.70:0/1392163450 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.761+0000 7fb79ad9d640 1 --2- 172.21.15.70:0/1392163450 >> v2:172.21.15.155:3300/0 conn(0x7fb79c0c8d20 0x7fb79c0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.762+0000 7fb793fff640 1 -- 172.21.15.70:0/1392163450 >> v2:172.21.15.155:3300/0 conn(0x7fb79c0c8d20 msgr2=0x7fb79c0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:41:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.762+0000 7fb793fff640 1 --2- 172.21.15.70:0/1392163450 >> v2:172.21.15.155:3300/0 conn(0x7fb79c0c8d20 0x7fb79c0c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.762+0000 7fb793fff640 1 -- 172.21.15.70:0/1392163450 >> v1:172.21.15.155:6789/0 conn(0x7fb79c0b2470 legacy=0x7fb79c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.762+0000 7fb793fff640 1 -- 172.21.15.70:0/1392163450 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c15a3f0 con 0x7fb79c0b2850 2023-08-25T09:41:59.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.764+0000 7fb793fff640 1 --2- 172.21.15.70:0/1392163450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 0x7fb79c0cdb30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb784009870 tx=0x7fb78402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e8bffefb89b208 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:59.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.764+0000 7fb799d9b640 1 -- 172.21.15.70:0/1392163450 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784030840 con 0x7fb79c0b2850 2023-08-25T09:41:59.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.764+0000 7fb799d9b640 1 -- 172.21.15.70:0/1392163450 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7840309a0 con 0x7fb79c0b2850 2023-08-25T09:41:59.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.764+0000 7fb799d9b640 1 -- 172.21.15.70:0/1392163450 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784030d30 con 0x7fb79c0b2850 2023-08-25T09:41:59.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 -- 172.21.15.70:0/1392163450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 msgr2=0x7fb79c0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1392163450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 0x7fb79c0cdb30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb784009870 tx=0x7fb78402b980 comp rx=0 tx=0).stop 2023-08-25T09:41:59.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 -- 172.21.15.70:0/1392163450 shutdown_connections 2023-08-25T09:41:59.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1392163450 >> v2:172.21.15.155:3300/0 conn(0x7fb79c0c8d20 0x7fb79c0c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1392163450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2850 0x7fb79c0cdb30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 -- 172.21.15.70:0/1392163450 >> 172.21.15.70:0/1392163450 conn(0x7fb79c0ce830 msgr2=0x7fb79c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 -- 172.21.15.70:0/1392163450 shutdown_connections 2023-08-25T09:41:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.765+0000 7fb79b59e640 1 -- 172.21.15.70:0/1392163450 wait complete. 2023-08-25T09:41:59.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.767+0000 7fb79b59e640 1 Processor -- start 2023-08-25T09:41:59.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.767+0000 7fb79b59e640 1 -- start start 2023-08-25T09:41:59.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2470 0x7fb79c0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79b59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb79c0b2850 0x7fb79c15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 0x7fb79c09b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c15b590 con 0x7fb79c0c8d20 2023-08-25T09:41:59.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79b59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c15b870 con 0x7fb79c0b2850 2023-08-25T09:41:59.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79c15bb50 con 0x7fb79c0b2470 2023-08-25T09:41:59.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 0x7fb79c09b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 0x7fb79c09b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45654/0 (socket says 172.21.15.70:45654) 2023-08-25T09:41:59.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.768+0000 7fb79ad9d640 1 -- 172.21.15.70:0/1351509835 learned_addr learned my addr 172.21.15.70:0/1351509835 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:41:59.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79a59c640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2470 0x7fb79c0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb793fff640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb79c0b2850 0x7fb79c15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79ad9d640 1 -- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2470 msgr2=0x7fb79c0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79ad9d640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2470 0x7fb79c0b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79ad9d640 1 -- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb79c0b2850 msgr2=0x7fb79c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79ad9d640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb79c0b2850 0x7fb79c15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79ad9d640 1 -- 172.21.15.70:0/1351509835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840094d0 con 0x7fb79c0c8d20 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79ad9d640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 0x7fb79c09b190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb79400efc0 tx=0x7fb79400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794009ba0 con 0x7fb79c0c8d20 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79400f040 con 0x7fb79c0c8d20 2023-08-25T09:41:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940146c0 con 0x7fb79c0c8d20 2023-08-25T09:41:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.769+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79c15be30 con 0x7fb79c0c8d20 2023-08-25T09:41:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.770+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79c15b1b0 con 0x7fb79c0c8d20 2023-08-25T09:41:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.771+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb79c15a990 con 0x7fb79c0c8d20 2023-08-25T09:41:59.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.772+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb794014820 con 0x7fb79c0c8d20 2023-08-25T09:41:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.772+0000 7fb792ffd640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb77c06a080 0x7fb77c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:41:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.773+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb794089140 con 0x7fb79c0c8d20 2023-08-25T09:41:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.773+0000 7fb79a59c640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb77c06a080 0x7fb77c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:41:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.773+0000 7fb79a59c640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb77c06a080 0x7fb77c06c540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0016d0 tx=0x7fb78c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:41:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.777+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb79c15a990 con 0x7fb79c0c8d20 2023-08-25T09:41:59.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.965+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb79c15a990 con 0x7fb79c0c8d20 2023-08-25T09:41:59.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.966+0000 7fb792ffd640 1 -- 172.21.15.70:0/1351509835 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb79c15a990 con 0x7fb79c0c8d20 2023-08-25T09:41:59.968 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:41:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.968+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb77c06a080 msgr2=0x7fb77c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.968+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb77c06a080 0x7fb77c06c540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0016d0 tx=0x7fb78c009040 comp rx=0 tx=0).stop 2023-08-25T09:41:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.968+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 msgr2=0x7fb79c09b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:41:59.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.968+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 0x7fb79c09b190 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb79400efc0 tx=0x7fb79400c490 comp rx=0 tx=0).stop 2023-08-25T09:41:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 shutdown_connections 2023-08-25T09:41:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb77c06a080 0x7fb77c06c540 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb79c0c8d20 0x7fb79c09b190 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb79c0b2850 0x7fb79c15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 --2- 172.21.15.70:0/1351509835 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb79c0b2470 0x7fb79c0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:41:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 >> 172.21.15.70:0/1351509835 conn(0x7fb79c0ce830 msgr2=0x7fb79c14fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:41:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 shutdown_connections 2023-08-25T09:41:59.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:41:59.969+0000 7fb79b59e640 1 -- 172.21.15.70:0/1351509835 wait complete. 2023-08-25T09:41:59.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:42:00.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.206+0000 7fe9cfb4e640 1 Processor -- start 2023-08-25T09:42:00.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.207+0000 7fe9cfb4e640 1 -- start start 2023-08-25T09:42:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.208+0000 7fe9cfb4e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9c8160290 0x7fe9c80cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.208+0000 7fe9cfb4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9c8069850 0x7fe9c80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.208+0000 7fe9cfb4e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c8071c10 con 0x7fe9c8160670 2023-08-25T09:42:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.208+0000 7fe9cfb4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c8071000 con 0x7fe9c8069850 2023-08-25T09:42:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.208+0000 7fe9cfb4e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c80712e0 con 0x7fe9c8160290 2023-08-25T09:42:00.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.208+0000 7fe9ce0eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9c8069850 0x7fe9c80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.209+0000 7fe9ce0eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe9c8069850 0x7fe9c80d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45668/0 (socket says 172.21.15.70:45668) 2023-08-25T09:42:00.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.209+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/1121398360 learned_addr learned my addr 172.21.15.70:0/1121398360 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:00.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.209+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/1121398360 >> v2:172.21.15.155:3300/0 conn(0x7fe9c8160290 msgr2=0x7fe9c80cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:00.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.209+0000 7fe9ce0eb640 1 --2- 172.21.15.70:0/1121398360 >> v2:172.21.15.155:3300/0 conn(0x7fe9c8160290 0x7fe9c80cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.209+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/1121398360 >> v1:172.21.15.155:6789/0 conn(0x7fe9c8160670 legacy=0x7fe9c80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:00.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.209+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/1121398360 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80715c0 con 0x7fe9c8069850 2023-08-25T09:42:00.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.210+0000 7fe9ce0eb640 1 --2- 172.21.15.70:0/1121398360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8069850 0x7fe9c80d0380 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe9c806ce80 tx=0x7fe9c002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad117b1bf946065 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:00.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.210+0000 7fe9cc8e8640 1 -- 172.21.15.70:0/1121398360 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c002ca80 con 0x7fe9c8069850 2023-08-25T09:42:00.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cc8e8640 1 -- 172.21.15.70:0/1121398360 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c002cbe0 con 0x7fe9c8069850 2023-08-25T09:42:00.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cc8e8640 1 -- 172.21.15.70:0/1121398360 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c00343f0 con 0x7fe9c8069850 2023-08-25T09:42:00.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/1121398360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8069850 msgr2=0x7fe9c80d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/1121398360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8069850 0x7fe9c80d0380 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe9c806ce80 tx=0x7fe9c002bee0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/1121398360 shutdown_connections 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/1121398360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8069850 0x7fe9c80d0380 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/1121398360 >> v2:172.21.15.155:3300/0 conn(0x7fe9c8160290 0x7fe9c80cd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/1121398360 >> 172.21.15.70:0/1121398360 conn(0x7fe9c806ba60 msgr2=0x7fe9c806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/1121398360 shutdown_connections 2023-08-25T09:42:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.211+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/1121398360 wait complete. 2023-08-25T09:42:00.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.213+0000 7fe9cfb4e640 1 Processor -- start 2023-08-25T09:42:00.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.213+0000 7fe9cfb4e640 1 -- start start 2023-08-25T09:42:00.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.214+0000 7fe9cfb4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9c8069850 0x7fe9c814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.214+0000 7fe9cfb4e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9c8160290 0x7fe9c81f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.214+0000 7fe9cfb4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 0x7fe9c80c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.214+0000 7fe9cfb4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b17b0 con 0x7fe9c8160670 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.214+0000 7fe9cfb4e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b1a90 con 0x7fe9c8160290 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.214+0000 7fe9cfb4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b1d70 con 0x7fe9c8069850 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 0x7fe9c80c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 0x7fe9c80c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45678/0 (socket says 172.21.15.70:45678) 2023-08-25T09:42:00.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/3828323457 learned_addr learned my addr 172.21.15.70:0/3828323457 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9cd0e9640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9c8160290 0x7fe9c81f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9cd8ea640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9c8069850 0x7fe9c814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9c8069850 msgr2=0x7fe9c814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9c8069850 0x7fe9c814fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9c8160290 msgr2=0x7fe9c81f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9c8160290 0x7fe9c81f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 -- 172.21.15.70:0/3828323457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0009b30 con 0x7fe9c8160670 2023-08-25T09:42:00.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9cd0e9640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9c8160290 0x7fe9c81f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:00.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9cd8ea640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9c8069850 0x7fe9c814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.215+0000 7fe9ce0eb640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 0x7fe9c80c9a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0036040 tx=0x7fe9c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.216+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c003e450 con 0x7fe9c8160670 2023-08-25T09:42:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.216+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c003e5d0 con 0x7fe9c8160670 2023-08-25T09:42:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.216+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c003e980 con 0x7fe9c8160670 2023-08-25T09:42:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.216+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c80b2050 con 0x7fe9c8160670 2023-08-25T09:42:00.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.216+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c81595f0 con 0x7fe9c8160670 2023-08-25T09:42:00.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.217+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c80b2570 con 0x7fe9c8160670 2023-08-25T09:42:00.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.218+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9c002cd50 con 0x7fe9c8160670 2023-08-25T09:42:00.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.219+0000 7fe9be7fc640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9ac06a050 0x7fe9ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.219+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe9c00ab200 con 0x7fe9c8160670 2023-08-25T09:42:00.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.219+0000 7fe9cd8ea640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9ac06a050 0x7fe9ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.220+0000 7fe9cd8ea640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9ac06a050 0x7fe9ac06c510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:00.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.223+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9c80b2570 con 0x7fe9c8160670 2023-08-25T09:42:00.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.412+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9c80b2570 con 0x7fe9c8160670 2023-08-25T09:42:00.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.413+0000 7fe9be7fc640 1 -- 172.21.15.70:0/3828323457 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9c80b2570 con 0x7fe9c8160670 2023-08-25T09:42:00.415 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:42:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9ac06a050 msgr2=0x7fe9ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9ac06a050 0x7fe9ac06c510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4008040 comp rx=0 tx=0).stop 2023-08-25T09:42:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 msgr2=0x7fe9c80c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 0x7fe9c80c9a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0036040 tx=0x7fe9c0004a40 comp rx=0 tx=0).stop 2023-08-25T09:42:00.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 shutdown_connections 2023-08-25T09:42:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9ac06a050 0x7fe9ac06c510 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9c8160670 0x7fe9c80c9a50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9c8160290 0x7fe9c81f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 --2- 172.21.15.70:0/3828323457 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9c8069850 0x7fe9c814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 >> 172.21.15.70:0/3828323457 conn(0x7fe9c806ba60 msgr2=0x7fe9c80ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:00.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 shutdown_connections 2023-08-25T09:42:00.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.416+0000 7fe9cfb4e640 1 -- 172.21.15.70:0/3828323457 wait complete. 2023-08-25T09:42:00.439 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:42:00.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.663+0000 7f54c4584640 1 Processor -- start 2023-08-25T09:42:00.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.663+0000 7f54c4584640 1 -- start start 2023-08-25T09:42:00.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c4584640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f54bc15ae50 0x7f54bc0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c4584640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 0x7f54bc0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c4584640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54bc0cc5f0 con 0x7f54bc0cdc20 2023-08-25T09:42:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c4584640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc070650 con 0x7f54bc15ae50 2023-08-25T09:42:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c4584640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc070930 con 0x7f54bc069f60 2023-08-25T09:42:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c1b1f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f54bc0cdc20 0x7f54bc069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36192/0 (socket says 172.21.15.70:36192) 2023-08-25T09:42:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.665+0000 7f54c1b1f640 1 -- 172.21.15.70:0/1244151848 learned_addr learned my addr 172.21.15.70:0/1244151848 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c2320640 1 --2- 172.21.15.70:0/1244151848 >> v2:172.21.15.155:3300/0 conn(0x7f54bc15ae50 0x7f54bc0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c2b21640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 0x7f54bc0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436952216 0 0) 0x7f54bc0cc5f0 con 0x7f54bc0cdc20 2023-08-25T09:42:00.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54a8003520 con 0x7f54bc0cdc20 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2000493083 0 0) 0x7f54a8003520 con 0x7f54bc0cdc20 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 msgr2=0x7f54bc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 0x7f54bc0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 >> v2:172.21.15.155:3300/0 conn(0x7f54bc15ae50 msgr2=0x7f54bc0c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 --2- 172.21.15.70:0/1244151848 >> v2:172.21.15.155:3300/0 conn(0x7f54bc15ae50 0x7f54bc0c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc070c10 con 0x7f54bc0cdc20 2023-08-25T09:42:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f54ac001670 con 0x7f54bc0cdc20 2023-08-25T09:42:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 >> v1:172.21.15.70:6789/0 conn(0x7f54bc0cdc20 legacy=0x7f54bc069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a8003970 0x7f54a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 0x7f54a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8004d00 0x7f54a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc070650 con 0x7f54a8003970 2023-08-25T09:42:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc070930 con 0x7f54a8004d00 2023-08-25T09:42:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.666+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a800b980 con 0x7f54bc069f60 2023-08-25T09:42:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2320640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a8003970 0x7f54a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c1b1f640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 0x7f54a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2b21640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8004d00 0x7f54a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2320640 1 -- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 msgr2=0x7f54a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2320640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc069f60 0x7f54a80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2320640 1 -- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8004d00 msgr2=0x7f54a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2320640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8004d00 0x7f54a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.667+0000 7f54c2320640 1 -- 172.21.15.70:0/1244151848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8003520 con 0x7f54a8003970 2023-08-25T09:42:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.668+0000 7f54c2320640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a8003970 0x7f54a8003d90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009ac0 tx=0x7f54a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506002e65d3e232c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b802c710 con 0x7f54a8003970 2023-08-25T09:42:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b802c870 con 0x7f54a8003970 2023-08-25T09:42:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c131e640 1 -- 172.21.15.70:0/1244151848 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b802cc00 con 0x7f54a8003970 2023-08-25T09:42:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c4584640 1 -- 172.21.15.70:0/1244151848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a8003970 msgr2=0x7f54a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c4584640 1 --2- 172.21.15.70:0/1244151848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54a8003970 0x7f54a8003d90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009ac0 tx=0x7f54a80057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c2b21640 1 -- 172.21.15.70:0/1244151848 reap_dead start 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c4584640 1 -- 172.21.15.70:0/1244151848 shutdown_connections 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.669+0000 7f54c4584640 1 -- 172.21.15.70:0/1244151848 >> 172.21.15.70:0/1244151848 conn(0x7f54bc0cb0d0 msgr2=0x7f54bc0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.670+0000 7f54c4584640 1 -- 172.21.15.70:0/1244151848 shutdown_connections 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.670+0000 7f54c4584640 1 -- 172.21.15.70:0/1244151848 wait complete. 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.671+0000 7f54c4584640 1 Processor -- start 2023-08-25T09:42:00.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.671+0000 7f54c4584640 1 -- start start 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.672+0000 7f54c4584640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8003970 0x7f54bc1e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.672+0000 7f54c4584640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc1546e0 0x7f54bc1590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.672+0000 7f54c4584640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 0x7f54bc1e0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.672+0000 7f54c4584640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc1e0a80 con 0x7f54bc1e00c0 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.672+0000 7f54c4584640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc1e70a0 con 0x7f54a8003970 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.672+0000 7f54c4584640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc1e5ac0 con 0x7f54bc1546e0 2023-08-25T09:42:00.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 0x7f54bc1e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 0x7f54bc1e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45702/0 (socket says 172.21.15.70:45702) 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 -- 172.21.15.70:0/4111723808 learned_addr learned my addr 172.21.15.70:0/4111723808 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c1b1f640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc1546e0 0x7f54bc1590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2320640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8003970 0x7f54bc1e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 -- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc1546e0 msgr2=0x7f54bc1590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc1546e0 0x7f54bc1590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 -- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8003970 msgr2=0x7f54bc1e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8003970 0x7f54bc1e6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2b21640 1 -- 172.21.15.70:0/4111723808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc1e5da0 con 0x7f54bc1e00c0 2023-08-25T09:42:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.673+0000 7f54c2320640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8003970 0x7f54bc1e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:42:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.674+0000 7f54c2b21640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 0x7f54bc1e0540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0b00f0 tx=0x7f54b40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.674+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400e070 con 0x7f54bc1e00c0 2023-08-25T09:42:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.674+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4011e40 con 0x7f54bc1e00c0 2023-08-25T09:42:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.674+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4003990 con 0x7f54bc1e00c0 2023-08-25T09:42:00.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.674+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc1e6080 con 0x7f54bc1e00c0 2023-08-25T09:42:00.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.674+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc1e6640 con 0x7f54bc1e00c0 2023-08-25T09:42:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.675+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc066fb0 con 0x7f54bc1e00c0 2023-08-25T09:42:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.676+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f54b4003b10 con 0x7f54bc1e00c0 2023-08-25T09:42:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.677+0000 7f54b2ffd640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54a006a080 0x7f54a006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.677+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f54b4088b60 con 0x7f54bc1e00c0 2023-08-25T09:42:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.677+0000 7f54c2320640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54a006a080 0x7f54a006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:00.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.678+0000 7f54c2320640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54a006a080 0x7f54a006c540 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54b8036000 tx=0x7f54b8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:00.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.681+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f54b4054230 con 0x7f54bc1e00c0 2023-08-25T09:42:00.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.869+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54bc1e6800 con 0x7f54bc1e00c0 2023-08-25T09:42:00.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.870+0000 7f54b2ffd640 1 -- 172.21.15.70:0/4111723808 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54b4057850 con 0x7f54bc1e00c0 2023-08-25T09:42:00.872 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:42:00.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54a006a080 msgr2=0x7f54a006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54a006a080 0x7f54a006c540 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54b8036000 tx=0x7f54b8030000 comp rx=0 tx=0).stop 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 msgr2=0x7f54bc1e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 0x7f54bc1e0540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0b00f0 tx=0x7f54b40115a0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 shutdown_connections 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f54a006a080 0x7f54a006c540 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f54bc1e00c0 0x7f54bc1e0540 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f54bc1546e0 0x7f54bc1590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 --2- 172.21.15.70:0/4111723808 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f54a8003970 0x7f54bc1e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:00.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 >> 172.21.15.70:0/4111723808 conn(0x7f54bc0cb0d0 msgr2=0x7f54bc159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:00.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 shutdown_connections 2023-08-25T09:42:00.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:00.874+0000 7f54c4584640 1 -- 172.21.15.70:0/4111723808 wait complete. 2023-08-25T09:42:00.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:42:01.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.101+0000 7fb7a9c83640 1 Processor -- start 2023-08-25T09:42:01.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.101+0000 7fb7a9c83640 1 -- start start 2023-08-25T09:42:01.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.102+0000 7fb7a9c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a41593a0 0x7fb7a40cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.102+0000 7fb7a9c83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb7a406ae20 0x7fb7a406ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.102+0000 7fb7a9c83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7a406b280 0x7fb7a406f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.102+0000 7fb7a9c83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a4159780 con 0x7fb7a406b280 2023-08-25T09:42:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a9c83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a40b21d0 con 0x7fb7a406ae20 2023-08-25T09:42:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a9c83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4070740 con 0x7fb7a41593a0 2023-08-25T09:42:01.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7a406b280 0x7fb7a406f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7a406b280 0x7fb7a406f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45716/0 (socket says 172.21.15.70:45716) 2023-08-25T09:42:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1780446745 learned_addr learned my addr 172.21.15.70:0/1780446745 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a37fe640 1 --2- 172.21.15.70:0/1780446745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a41593a0 0x7fb7a40cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb79bfff640 1 --2- 172.21.15.70:0/1780446745 >> v2:172.21.15.155:3300/0 conn(0x7fb7a406ae20 0x7fb7a406ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1780446745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a41593a0 msgr2=0x7fb7a40cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:01.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/1780446745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a41593a0 0x7fb7a40cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1780446745 >> v2:172.21.15.155:3300/0 conn(0x7fb7a406ae20 msgr2=0x7fb7a406ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/1780446745 >> v2:172.21.15.155:3300/0 conn(0x7fb7a406ae20 0x7fb7a406ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.103+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1780446745 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a406fea0 con 0x7fb7a406b280 2023-08-25T09:42:01.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.104+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/1780446745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a406b280 0x7fb7a406f290 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb7a406e710 tx=0x7fb79c02e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da377a42779ccf0c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:01.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a2ffd640 1 -- 172.21.15.70:0/1780446745 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c02ea00 con 0x7fb7a406b280 2023-08-25T09:42:01.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a2ffd640 1 -- 172.21.15.70:0/1780446745 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c02eb60 con 0x7fb7a406b280 2023-08-25T09:42:01.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a2ffd640 1 -- 172.21.15.70:0/1780446745 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c009600 con 0x7fb7a406b280 2023-08-25T09:42:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1780446745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a406b280 msgr2=0x7fb7a406f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1780446745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a406b280 0x7fb7a406f290 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb7a406e710 tx=0x7fb79c02e060 comp rx=0 tx=0).stop 2023-08-25T09:42:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1780446745 shutdown_connections 2023-08-25T09:42:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1780446745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a406b280 0x7fb7a406f290 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1780446745 >> v2:172.21.15.155:3300/0 conn(0x7fb7a406ae20 0x7fb7a406ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1780446745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a41593a0 0x7fb7a40cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1780446745 >> 172.21.15.70:0/1780446745 conn(0x7fb7a40ce760 msgr2=0x7fb7a40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.105+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1780446745 shutdown_connections 2023-08-25T09:42:01.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.106+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1780446745 wait complete. 2023-08-25T09:42:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.107+0000 7fb7a9c83640 1 Processor -- start 2023-08-25T09:42:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.107+0000 7fb7a9c83640 1 -- start start 2023-08-25T09:42:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.108+0000 7fb7a9c83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a406ae20 0x7fb7a414b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.108+0000 7fb7a9c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a406b280 0x7fb7a414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.108+0000 7fb7a9c83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 0x7fb7a4159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.108+0000 7fb7a9c83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a414b800 con 0x7fb7a41593a0 2023-08-25T09:42:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.108+0000 7fb7a9c83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a414cd40 con 0x7fb7a406ae20 2023-08-25T09:42:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.108+0000 7fb7a9c83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4153b10 con 0x7fb7a406b280 2023-08-25T09:42:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 0x7fb7a4159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 0x7fb7a4159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45728/0 (socket says 172.21.15.70:45728) 2023-08-25T09:42:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1875163904 learned_addr learned my addr 172.21.15.70:0/1875163904 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a406b280 msgr2=0x7fb7a414c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb79bfff640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a406b280 0x7fb7a414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a406b280 0x7fb7a414c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a37fe640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a406ae20 0x7fb7a414b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a406ae20 msgr2=0x7fb7a414b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a406ae20 0x7fb7a414b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 -- 172.21.15.70:0/1875163904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c00bd40 con 0x7fb7a41593a0 2023-08-25T09:42:01.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.109+0000 7fb7a3fff640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 0x7fb7a4159a00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb79c038040 tx=0x7fb79c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.110+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c02ea00 con 0x7fb7a41593a0 2023-08-25T09:42:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.110+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c036820 con 0x7fb7a41593a0 2023-08-25T09:42:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.110+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c036bd0 con 0x7fb7a41593a0 2023-08-25T09:42:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.110+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a414bd20 con 0x7fb7a41593a0 2023-08-25T09:42:01.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.110+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a414c2e0 con 0x7fb7a41593a0 2023-08-25T09:42:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.111+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a414bb80 con 0x7fb7a41593a0 2023-08-25T09:42:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.112+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb79c036d30 con 0x7fb7a41593a0 2023-08-25T09:42:01.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.113+0000 7fb7a0ff9640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb78806a050 0x7fb78806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.113+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb79c0aba70 con 0x7fb7a41593a0 2023-08-25T09:42:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.113+0000 7fb7a37fe640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb78806a050 0x7fb78806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.114+0000 7fb7a37fe640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb78806a050 0x7fb78806c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb7940096d0 tx=0x7fb794008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:01.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.117+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7a414bb80 con 0x7fb7a41593a0 2023-08-25T09:42:01.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.317+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb7a414bb80 con 0x7fb7a41593a0 2023-08-25T09:42:01.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.318+0000 7fb7a0ff9640 1 -- 172.21.15.70:0/1875163904 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb7a414bb80 con 0x7fb7a41593a0 2023-08-25T09:42:01.320 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:42:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.320+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb78806a050 msgr2=0x7fb78806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.320+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb78806a050 0x7fb78806c510 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb7940096d0 tx=0x7fb794008040 comp rx=0 tx=0).stop 2023-08-25T09:42:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.320+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 msgr2=0x7fb7a4159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.320+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 0x7fb7a4159a00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb79c038040 tx=0x7fb79c004a40 comp rx=0 tx=0).stop 2023-08-25T09:42:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 shutdown_connections 2023-08-25T09:42:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb78806a050 0x7fb78806c510 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7a41593a0 0x7fb7a4159a00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7a406b280 0x7fb7a414c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 --2- 172.21.15.70:0/1875163904 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7a406ae20 0x7fb7a414b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 >> 172.21.15.70:0/1875163904 conn(0x7fb7a40ce760 msgr2=0x7fb7a40b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 shutdown_connections 2023-08-25T09:42:01.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.321+0000 7fb7a9c83640 1 -- 172.21.15.70:0/1875163904 wait complete. 2023-08-25T09:42:01.344 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:42:01.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:01.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:01.428 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[99410]: fuse finished with error 0 and tester_r 0 2023-08-25T09:42:01.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:42:01.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:01.457 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:42:01.644 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:42:01.644 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:01.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.921+0000 7f4c97fff640 1 Processor -- start 2023-08-25T09:42:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.921+0000 7f4c97fff640 1 -- start start 2023-08-25T09:42:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.922+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 0x7f4c980cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.922+0000 7f4c97fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9815e7c0 con 0x7f4c980b12f0 2023-08-25T09:42:01.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.922+0000 7f4c97fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c98071b50 con 0x7f4c9815a7c0 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.922+0000 7f4c97fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980cb6f0 con 0x7f4c980cad20 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.923+0000 7f4c8e7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4c980b12f0 0x7f4c9815ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36208/0 (socket says 172.21.15.70:36208) 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.923+0000 7f4c8e7fc640 1 -- 172.21.15.70:0/989655663 learned_addr learned my addr 172.21.15.70:0/989655663 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.923+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094126491 0 0) 0x7f4c9815e7c0 con 0x7f4c980b12f0 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.923+0000 7f4c977fe640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 0x7f4c980cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.923+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c78003520 con 0x7f4c980b12f0 2023-08-25T09:42:01.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2509584944 0 0) 0x7f4c78003520 con 0x7f4c980b12f0 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 msgr2=0x7f4c980cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 0x7f4c980cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 >> v1:172.21.15.155:6789/0 conn(0x7f4c9815a7c0 legacy=0x7f4c980b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9815e490 con 0x7f4c980b12f0 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4c80001670 con 0x7f4c980b12f0 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 >> v1:172.21.15.70:6789/0 conn(0x7f4c980b12f0 legacy=0x7f4c9815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c780047c0 0x7f4c78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 0x7f4c780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980cb6f0 con 0x7f4c780039a0 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800bae0 con 0x7f4c780047c0 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7800bca0 con 0x7f4c980cad20 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c96ffd640 1 -- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 msgr2=0x7f4c780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:01.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980cad20 0x7f4c780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c8e7fc640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c780047c0 0x7f4c78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.924+0000 7f4c96ffd640 1 -- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c780047c0 msgr2=0x7f4c78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.925+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c780047c0 0x7f4c78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.925+0000 7f4c96ffd640 1 -- 172.21.15.70:0/989655663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c78003520 con 0x7f4c780039a0 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.925+0000 7f4c8e7fc640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c780047c0 0x7f4c78004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c78003dc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4c88001750 tx=0x7f4c780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21a8e0f9261c2dc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880052d0 con 0x7f4c780039a0 2023-08-25T09:42:01.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8802e050 con 0x7f4c780039a0 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c967fc640 1 -- 172.21.15.70:0/989655663 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880035f0 con 0x7f4c780039a0 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c97fff640 1 -- 172.21.15.70:0/989655663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 msgr2=0x7f4c78003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c97fff640 1 --2- 172.21.15.70:0/989655663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c78003dc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f4c88001750 tx=0x7f4c780057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.926+0000 7f4c977fe640 1 -- 172.21.15.70:0/989655663 reap_dead start 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.927+0000 7f4c97fff640 1 -- 172.21.15.70:0/989655663 shutdown_connections 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.927+0000 7f4c97fff640 1 -- 172.21.15.70:0/989655663 >> 172.21.15.70:0/989655663 conn(0x7f4c980affd0 msgr2=0x7f4c980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.927+0000 7f4c97fff640 1 -- 172.21.15.70:0/989655663 shutdown_connections 2023-08-25T09:42:01.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.927+0000 7f4c97fff640 1 -- 172.21.15.70:0/989655663 wait complete. 2023-08-25T09:42:01.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.928+0000 7f4c97fff640 1 Processor -- start 2023-08-25T09:42:01.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.929+0000 7f4c97fff640 1 -- start start 2023-08-25T09:42:01.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.929+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c981489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c980cd520 0x7f4c980cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c97fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980c55e0 0x7f4c980c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c97fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980c6310 con 0x7f4c780039a0 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c97fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980cdee0 con 0x7f4c980cd520 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c97fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980c2ab0 con 0x7f4c980c55e0 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c981489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c981489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45756/0 (socket says 172.21.15.70:45756) 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 -- 172.21.15.70:0/3847422430 learned_addr learned my addr 172.21.15.70:0/3847422430 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 -- 172.21.15.70:0/3847422430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980c55e0 msgr2=0x7f4c980c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980c55e0 0x7f4c980c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 -- 172.21.15.70:0/3847422430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c980cd520 msgr2=0x7f4c980cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c980cd520 0x7f4c980cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.930+0000 7f4c96ffd640 1 -- 172.21.15.70:0/3847422430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980c2d90 con 0x7f4c780039a0 2023-08-25T09:42:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.931+0000 7f4c96ffd640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c981489f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4c88008fd0 tx=0x7f4c88001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.931+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880076c0 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.931+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88007a30 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.931+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88006be0 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.931+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c980c3070 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.931+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c980c5fa0 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.932+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c98066fb0 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.933+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4c88006d40 con 0x7f4c780039a0 2023-08-25T09:42:01.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.934+0000 7f4c8ffff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c7c06a050 0x7f4c7c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:01.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.934+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f4c880aa720 con 0x7f4c780039a0 2023-08-25T09:42:01.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.934+0000 7f4c8e7fc640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c7c06a050 0x7f4c7c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:01.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.935+0000 7f4c8e7fc640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c7c06a050 0x7f4c7c06c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c9814c660 tx=0x7f4c80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:01.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:01.938+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4c88075d50 con 0x7f4c780039a0 2023-08-25T09:42:02.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.128+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c980b0d00 con 0x7f4c780039a0 2023-08-25T09:42:02.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.130+0000 7f4c8ffff640 1 -- 172.21.15.70:0/3847422430 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18353 (secure 0 0 0) 0x7f4c88079370 con 0x7f4c780039a0 2023-08-25T09:42:02.132 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:02.133 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":61,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:53.340729+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.9","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c7c06a050 msgr2=0x7f4c7c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c7c06a050 0x7f4c7c06c510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4c9814c660 tx=0x7f4c80009040 comp rx=0 tx=0).stop 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 msgr2=0x7f4c981489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c981489f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4c88008fd0 tx=0x7f4c88001120 comp rx=0 tx=0).stop 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 shutdown_connections 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c7c06a050 0x7f4c7c06c510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c980c55e0 0x7f4c980c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c980cd520 0x7f4c980cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 --2- 172.21.15.70:0/3847422430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c780039a0 0x7f4c981489f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 >> 172.21.15.70:0/3847422430 conn(0x7f4c980affd0 msgr2=0x7f4c980cc070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 shutdown_connections 2023-08-25T09:42:02.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.132+0000 7f4c97fff640 1 -- 172.21.15.70:0/3847422430 wait complete. 2023-08-25T09:42:02.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:41:11.244136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:41:11.990910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:42:02.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:42:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.380+0000 7fe0d687c640 1 Processor -- start 2023-08-25T09:42:02.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.380+0000 7fe0d687c640 1 -- start start 2023-08-25T09:42:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.382+0000 7fe0d687c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0d00c2110 0x7fe0d0069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.382+0000 7fe0d687c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00c2550 con 0x7fe0d015ae50 2023-08-25T09:42:02.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.382+0000 7fe0d687c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00ce330 con 0x7fe0d0069d90 2023-08-25T09:42:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.382+0000 7fe0d687c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d006a200 con 0x7fe0d00c2110 2023-08-25T09:42:02.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.382+0000 7fe0cffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe0d015ae50 0x7fe0d00cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36214/0 (socket says 172.21.15.70:36214) 2023-08-25T09:42:02.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.382+0000 7fe0cffff640 1 -- 172.21.15.70:0/16345783 learned_addr learned my addr 172.21.15.70:0/16345783 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510947339 0 0) 0x7fe0d00c2550 con 0x7fe0d015ae50 2023-08-25T09:42:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b8003520 con 0x7fe0d015ae50 2023-08-25T09:42:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0cf7fe640 1 --2- 172.21.15.70:0/16345783 >> v2:172.21.15.155:3300/0 conn(0x7fe0d00c2110 0x7fe0d0069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3285332448 0 0) 0x7fe0b8003520 con 0x7fe0d015ae50 2023-08-25T09:42:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 >> v2:172.21.15.155:3300/0 conn(0x7fe0d00c2110 msgr2=0x7fe0d0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/16345783 >> v2:172.21.15.155:3300/0 conn(0x7fe0d00c2110 0x7fe0d0069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 >> v1:172.21.15.155:6789/0 conn(0x7fe0d0069d90 legacy=0x7fe0d00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00cc5f0 con 0x7fe0d015ae50 2023-08-25T09:42:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe0c0001670 con 0x7fe0d015ae50 2023-08-25T09:42:02.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 >> v1:172.21.15.70:6789/0 conn(0x7fe0d015ae50 legacy=0x7fe0d00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0b8003970 0x7fe0b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b80047c0 0x7fe0b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0b8005180 0x7fe0b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d006a200 con 0x7fe0b8005180 2023-08-25T09:42:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b800c0b0 con 0x7fe0b8003970 2023-08-25T09:42:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.383+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b800c270 con 0x7fe0b80047c0 2023-08-25T09:42:02.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0b8005180 0x7fe0b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0cffff640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0b8003970 0x7fe0b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0d4e19640 1 -- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b80047c0 msgr2=0x7fe0b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0cf7fe640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b80047c0 0x7fe0b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b80047c0 0x7fe0b8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0d4e19640 1 -- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0b8003970 msgr2=0x7fe0b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0b8003970 0x7fe0b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.384+0000 7fe0d4e19640 1 -- 172.21.15.70:0/16345783 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8003520 con 0x7fe0b8005180 2023-08-25T09:42:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.385+0000 7fe0d4e19640 1 -- 172.21.15.70:0/16345783 reap_dead start 2023-08-25T09:42:02.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.385+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0b8005180 0x7fe0b800b980 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8001e50 tx=0x7fe0b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a55856e0021e0e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:02.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8003b70 con 0x7fe0b8005180 2023-08-25T09:42:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c8003d10 con 0x7fe0b8005180 2023-08-25T09:42:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0ceffd640 1 -- 172.21.15.70:0/16345783 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8005310 con 0x7fe0b8005180 2023-08-25T09:42:02.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 -- 172.21.15.70:0/16345783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0b8005180 msgr2=0x7fe0b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0b8005180 0x7fe0b800b980 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8001e50 tx=0x7fe0b80057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 -- 172.21.15.70:0/16345783 shutdown_connections 2023-08-25T09:42:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 --2- 172.21.15.70:0/16345783 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0b8005180 0x7fe0b800b980 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 -- 172.21.15.70:0/16345783 >> 172.21.15.70:0/16345783 conn(0x7fe0d00cb0d0 msgr2=0x7fe0d00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 -- 172.21.15.70:0/16345783 shutdown_connections 2023-08-25T09:42:02.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.386+0000 7fe0d687c640 1 -- 172.21.15.70:0/16345783 wait complete. 2023-08-25T09:42:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.388+0000 7fe0d687c640 1 Processor -- start 2023-08-25T09:42:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.388+0000 7fe0d687c640 1 -- start start 2023-08-25T09:42:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d687c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b8005180 0x7fe0d0157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d687c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d0157e40 0x7fe0d015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d687c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 0x7fe0d00c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d687c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d015c140 con 0x7fe0d015c740 2023-08-25T09:42:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d687c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0157ba0 con 0x7fe0d0157e40 2023-08-25T09:42:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d687c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d015bb80 con 0x7fe0b8005180 2023-08-25T09:42:02.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.389+0000 7fe0d4e19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 0x7fe0d00c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 0x7fe0d00c3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45762/0 (socket says 172.21.15.70:45762) 2023-08-25T09:42:02.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 -- 172.21.15.70:0/2403108201 learned_addr learned my addr 172.21.15.70:0/2403108201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 -- 172.21.15.70:0/2403108201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b8005180 msgr2=0x7fe0d0157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b8005180 0x7fe0d0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 -- 172.21.15.70:0/2403108201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d0157e40 msgr2=0x7fe0d015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d0157e40 0x7fe0d015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 -- 172.21.15.70:0/2403108201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d015be60 con 0x7fe0d015c740 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.390+0000 7fe0d4e19640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 0x7fe0d00c3750 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8001f70 tx=0x7fe0c80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.391+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8006d50 con 0x7fe0d015c740 2023-08-25T09:42:02.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.391+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c8007030 con 0x7fe0d015c740 2023-08-25T09:42:02.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.391+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80073a0 con 0x7fe0d015c740 2023-08-25T09:42:02.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.391+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d00ab340 con 0x7fe0d015c740 2023-08-25T09:42:02.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.391+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d015b770 con 0x7fe0d015c740 2023-08-25T09:42:02.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.392+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d0066fb0 con 0x7fe0d015c740 2023-08-25T09:42:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.393+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe0c8007500 con 0x7fe0d015c740 2023-08-25T09:42:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.394+0000 7fe0ccff9640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0bc06a050 0x7fe0bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.394+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe0c80aa240 con 0x7fe0d015c740 2023-08-25T09:42:02.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.394+0000 7fe0cffff640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0bc06a050 0x7fe0bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.395+0000 7fe0cffff640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0bc06a050 0x7fe0bc06c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00058b0 tx=0x7fe0c0000ef0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:02.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.398+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0c80757e0 con 0x7fe0d015c740 2023-08-25T09:42:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.589+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0d00cd8e0 con 0x7fe0d015c740 2023-08-25T09:42:02.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.590+0000 7fe0ccff9640 1 -- 172.21.15.70:0/2403108201 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0c8078e00 con 0x7fe0d015c740 2023-08-25T09:42:02.591 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:42:02.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.592+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0bc06a050 msgr2=0x7fe0bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.592+0000 7fe0d687c640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0bc06a050 0x7fe0bc06c510 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe0c00058b0 tx=0x7fe0c0000ef0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.592+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 msgr2=0x7fe0d00c3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.592+0000 7fe0d687c640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 0x7fe0d00c3750 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8001f70 tx=0x7fe0c80014a0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 shutdown_connections 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0bc06a050 0x7fe0bc06c510 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0d015c740 0x7fe0d00c3750 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0d0157e40 0x7fe0d015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 --2- 172.21.15.70:0/2403108201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0b8005180 0x7fe0d0157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 >> 172.21.15.70:0/2403108201 conn(0x7fe0d00cb0d0 msgr2=0x7fe0d00c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:02.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 shutdown_connections 2023-08-25T09:42:02.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.593+0000 7fe0d687c640 1 -- 172.21.15.70:0/2403108201 wait complete. 2023-08-25T09:42:02.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:42:02.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.829+0000 7f6ec9eb4640 1 Processor -- start 2023-08-25T09:42:02.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.829+0000 7f6ec9eb4640 1 -- start start 2023-08-25T09:42:02.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec9eb4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ec4069d30 0x7f6ec40ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec9eb4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6ec406a110 0x7f6ec40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec9eb4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406c510 0x7f6ec406c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec9eb4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40cd4e0 con 0x7f6ec4069d30 2023-08-25T09:42:02.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec9eb4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec40cc550 con 0x7f6ec406a110 2023-08-25T09:42:02.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec9eb4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40ccb10 con 0x7f6ec406c510 2023-08-25T09:42:02.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ec4069d30 0x7f6ec40ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6ec4069d30 0x7f6ec40ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45776/0 (socket says 172.21.15.70:45776) 2023-08-25T09:42:02.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec37fe640 1 -- 172.21.15.70:0/4209329659 learned_addr learned my addr 172.21.15.70:0/4209329659 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:02.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.831+0000 7f6ec37fe640 1 -- 172.21.15.70:0/4209329659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406c510 msgr2=0x7f6ec406c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:02.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.832+0000 7f6ec37fe640 1 --2- 172.21.15.70:0/4209329659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406c510 0x7f6ec406c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.832+0000 7f6ec37fe640 1 -- 172.21.15.70:0/4209329659 >> v2:172.21.15.155:3300/0 conn(0x7f6ec406a110 msgr2=0x7f6ec40ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:02.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.832+0000 7f6ec37fe640 1 --2- 172.21.15.70:0/4209329659 >> v2:172.21.15.155:3300/0 conn(0x7f6ec406a110 0x7f6ec40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.832+0000 7f6ec37fe640 1 -- 172.21.15.70:0/4209329659 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec40ade40 con 0x7f6ec4069d30 2023-08-25T09:42:02.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.833+0000 7f6ec37fe640 1 --2- 172.21.15.70:0/4209329659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec40ccfa0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40c8b40 tx=0x7f6eb402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477a654ecb39c41b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:02.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.833+0000 7f6ec27fc640 1 -- 172.21.15.70:0/4209329659 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40317e0 con 0x7f6ec4069d30 2023-08-25T09:42:02.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.833+0000 7f6ec27fc640 1 -- 172.21.15.70:0/4209329659 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4031940 con 0x7f6ec4069d30 2023-08-25T09:42:02.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.833+0000 7f6ec27fc640 1 -- 172.21.15.70:0/4209329659 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4031cd0 con 0x7f6ec4069d30 2023-08-25T09:42:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/4209329659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 msgr2=0x7f6ec40ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/4209329659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec40ccfa0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40c8b40 tx=0x7f6eb402b880 comp rx=0 tx=0).stop 2023-08-25T09:42:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/4209329659 shutdown_connections 2023-08-25T09:42:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/4209329659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406c510 0x7f6ec406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/4209329659 >> v2:172.21.15.155:3300/0 conn(0x7f6ec406a110 0x7f6ec40ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/4209329659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec40ccfa0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/4209329659 >> 172.21.15.70:0/4209329659 conn(0x7f6ec40cb0d0 msgr2=0x7f6ec40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/4209329659 shutdown_connections 2023-08-25T09:42:02.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.834+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/4209329659 wait complete. 2023-08-25T09:42:02.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.836+0000 7f6ec9eb4640 1 Processor -- start 2023-08-25T09:42:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.836+0000 7f6ec9eb4640 1 -- start start 2023-08-25T09:42:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec9eb4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec415f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec9eb4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406a110 0x7f6ec40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec9eb4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ec406c510 0x7f6ec41fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec9eb4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec415fcd0 con 0x7f6ec4069d30 2023-08-25T09:42:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec9eb4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40c53e0 con 0x7f6ec406c510 2023-08-25T09:42:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec9eb4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec40b11a0 con 0x7f6ec406a110 2023-08-25T09:42:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec415f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec415f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45782/0 (socket says 172.21.15.70:45782) 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.837+0000 7f6ec37fe640 1 -- 172.21.15.70:0/1889194012 learned_addr learned my addr 172.21.15.70:0/1889194012 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec37fe640 1 -- 172.21.15.70:0/1889194012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406a110 msgr2=0x7f6ec40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec37fe640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406a110 0x7f6ec40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec37fe640 1 -- 172.21.15.70:0/1889194012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ec406c510 msgr2=0x7f6ec41fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec37fe640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ec406c510 0x7f6ec41fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec37fe640 1 -- 172.21.15.70:0/1889194012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40094d0 con 0x7f6ec4069d30 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec37fe640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec415f790 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4000c00 tx=0x7f6eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4032940 con 0x7f6ec4069d30 2023-08-25T09:42:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4032aa0 con 0x7f6ec4069d30 2023-08-25T09:42:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec40b13f0 con 0x7f6ec4069d30 2023-08-25T09:42:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.838+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4032e50 con 0x7f6ec4069d30 2023-08-25T09:42:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.839+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec40b19b0 con 0x7f6ec4069d30 2023-08-25T09:42:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.840+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec4066fb0 con 0x7f6ec4069d30 2023-08-25T09:42:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.840+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6eb40453f0 con 0x7f6ec4069d30 2023-08-25T09:42:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.841+0000 7f6eabfff640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6eb006a050 0x7f6eb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.841+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f6eb40aa730 con 0x7f6ec4069d30 2023-08-25T09:42:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.842+0000 7f6ec2ffd640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6eb006a050 0x7f6eb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.842+0000 7f6ec2ffd640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6eb006a050 0x7f6eb006c510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40c8480 tx=0x7f6eb8006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:02.846+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6eb4075fb0 con 0x7f6ec4069d30 2023-08-25T09:42:03.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.033+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6ec40b45d0 con 0x7f6ec4069d30 2023-08-25T09:42:03.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.034+0000 7f6eabfff640 1 -- 172.21.15.70:0/1889194012 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6eb40795d0 con 0x7f6ec4069d30 2023-08-25T09:42:03.036 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:42:03.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6eb006a050 msgr2=0x7f6eb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6eb006a050 0x7f6eb006c510 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40c8480 tx=0x7f6eb8006c90 comp rx=0 tx=0).stop 2023-08-25T09:42:03.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 msgr2=0x7f6ec415f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec415f790 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4000c00 tx=0x7f6eb40046c0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 shutdown_connections 2023-08-25T09:42:03.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6eb006a050 0x7f6eb006c510 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ec406c510 0x7f6ec41fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ec406a110 0x7f6ec40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 --2- 172.21.15.70:0/1889194012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ec4069d30 0x7f6ec415f790 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 >> 172.21.15.70:0/1889194012 conn(0x7f6ec40cb0d0 msgr2=0x7f6ec406fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:03.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 shutdown_connections 2023-08-25T09:42:03.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.037+0000 7f6ec9eb4640 1 -- 172.21.15.70:0/1889194012 wait complete. 2023-08-25T09:42:03.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:42:03.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.265+0000 7fac851bd640 1 Processor -- start 2023-08-25T09:42:03.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.265+0000 7fac851bd640 1 -- start start 2023-08-25T09:42:03.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.266+0000 7fac851bd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fac8015a920 0x7fac800c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.266+0000 7fac851bd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac800b1ce0 0x7fac800ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.266+0000 7fac851bd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac800ad8f0 con 0x7fac800b0320 2023-08-25T09:42:03.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.266+0000 7fac851bd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac800adbb0 con 0x7fac800b1ce0 2023-08-25T09:42:03.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac851bd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac800c90a0 con 0x7fac8015a920 2023-08-25T09:42:03.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac800b1ce0 0x7fac800ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac800b1ce0 0x7fac800ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45794/0 (socket says 172.21.15.70:45794) 2023-08-25T09:42:03.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 -- 172.21.15.70:0/544666101 learned_addr learned my addr 172.21.15.70:0/544666101 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:03.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac777fe640 1 --2- 172.21.15.70:0/544666101 >> v2:172.21.15.155:3300/0 conn(0x7fac8015a920 0x7fac800c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 -- 172.21.15.70:0/544666101 >> v2:172.21.15.155:3300/0 conn(0x7fac8015a920 msgr2=0x7fac800c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 --2- 172.21.15.70:0/544666101 >> v2:172.21.15.155:3300/0 conn(0x7fac8015a920 0x7fac800c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 -- 172.21.15.70:0/544666101 >> v1:172.21.15.155:6789/0 conn(0x7fac800b0320 legacy=0x7fac800b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac849bc640 1 -- 172.21.15.70:0/544666101 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac800c9380 con 0x7fac800b1ce0 2023-08-25T09:42:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.267+0000 7fac777fe640 1 --2- 172.21.15.70:0/544666101 >> v2:172.21.15.155:3300/0 conn(0x7fac8015a920 0x7fac800c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.268+0000 7fac849bc640 1 --2- 172.21.15.70:0/544666101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800ad3b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fac80071e80 tx=0x7fac7802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5236e1ddd99a39b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.269+0000 7fac7f7fe640 1 -- 172.21.15.70:0/544666101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78031890 con 0x7fac800b1ce0 2023-08-25T09:42:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.269+0000 7fac7f7fe640 1 -- 172.21.15.70:0/544666101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac780319f0 con 0x7fac800b1ce0 2023-08-25T09:42:03.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.269+0000 7fac7f7fe640 1 -- 172.21.15.70:0/544666101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac78031d80 con 0x7fac800b1ce0 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.269+0000 7fac851bd640 1 -- 172.21.15.70:0/544666101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 msgr2=0x7fac800ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.269+0000 7fac851bd640 1 --2- 172.21.15.70:0/544666101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800ad3b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fac80071e80 tx=0x7fac7802b930 comp rx=0 tx=0).stop 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.270+0000 7fac851bd640 1 -- 172.21.15.70:0/544666101 shutdown_connections 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.270+0000 7fac851bd640 1 --2- 172.21.15.70:0/544666101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800ad3b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.270+0000 7fac851bd640 1 --2- 172.21.15.70:0/544666101 >> v2:172.21.15.155:3300/0 conn(0x7fac8015a920 0x7fac800c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.270+0000 7fac851bd640 1 -- 172.21.15.70:0/544666101 >> 172.21.15.70:0/544666101 conn(0x7fac800ce830 msgr2=0x7fac800cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:03.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.270+0000 7fac851bd640 1 -- 172.21.15.70:0/544666101 shutdown_connections 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.270+0000 7fac851bd640 1 -- 172.21.15.70:0/544666101 wait complete. 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.271+0000 7fac851bd640 1 Processor -- start 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.272+0000 7fac851bd640 1 -- start start 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.272+0000 7fac851bd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac800b0320 0x7fac800c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.272+0000 7fac851bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac851bd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac8015a920 0x7fac800cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac851bd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800cc280 con 0x7fac800b1ce0 2023-08-25T09:42:03.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac851bd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800c5bd0 con 0x7fac8015a920 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac851bd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac800cdd30 con 0x7fac800b0320 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45796/0 (socket says 172.21.15.70:45796) 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 -- 172.21.15.70:0/2054692811 learned_addr learned my addr 172.21.15.70:0/2054692811 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac7ffff640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac800b0320 0x7fac800c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 -- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac800b0320 msgr2=0x7fac800c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac800b0320 0x7fac800c4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 -- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac8015a920 msgr2=0x7fac800cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac8015a920 0x7fac800cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.273+0000 7fac777fe640 1 -- 172.21.15.70:0/2054692811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac780094d0 con 0x7fac800b1ce0 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.274+0000 7fac777fe640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800c45c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fac6800ce30 tx=0x7fac6800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.274+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6800f680 con 0x7fac800b1ce0 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.274+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6800f7e0 con 0x7fac800b1ce0 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.274+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac800cda50 con 0x7fac800b1ce0 2023-08-25T09:42:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.274+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6800fb70 con 0x7fac800b1ce0 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.274+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac800c5860 con 0x7fac800b1ce0 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.275+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac80061460 con 0x7fac800b1ce0 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.276+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fac6800b040 con 0x7fac800b1ce0 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.278+0000 7fac7d7fa640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac6006a050 0x7fac6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.278+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fac68088fe0 con 0x7fac800b1ce0 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.278+0000 7fac7ffff640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac6006a050 0x7fac6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.279+0000 7fac7ffff640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac6006a050 0x7fac6006c510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fac70003e00 tx=0x7fac70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:03.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.282+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fac680546a0 con 0x7fac800b1ce0 2023-08-25T09:42:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.477+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac800ca720 con 0x7fac800b1ce0 2023-08-25T09:42:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.478+0000 7fac7d7fa640 1 -- 172.21.15.70:0/2054692811 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7fac68057cc0 con 0x7fac800b1ce0 2023-08-25T09:42:03.480 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:42:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac6006a050 msgr2=0x7fac6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac6006a050 0x7fac6006c510 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fac70003e00 tx=0x7fac70009040 comp rx=0 tx=0).stop 2023-08-25T09:42:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 msgr2=0x7fac800c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800c45c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fac6800ce30 tx=0x7fac6800a650 comp rx=0 tx=0).stop 2023-08-25T09:42:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 shutdown_connections 2023-08-25T09:42:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac6006a050 0x7fac6006c510 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac8015a920 0x7fac800cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac800b1ce0 0x7fac800c45c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 --2- 172.21.15.70:0/2054692811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac800b0320 0x7fac800c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 >> 172.21.15.70:0/2054692811 conn(0x7fac800ce830 msgr2=0x7fac80070eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 shutdown_connections 2023-08-25T09:42:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.481+0000 7fac851bd640 1 -- 172.21.15.70:0/2054692811 wait complete. 2023-08-25T09:42:03.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:42:03.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.720+0000 7f595fca6640 1 Processor -- start 2023-08-25T09:42:03.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.721+0000 7f595fca6640 1 -- start start 2023-08-25T09:42:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.722+0000 7f595fca6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5958069850 0x7f595815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.722+0000 7f595fca6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 0x7f59580cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595fca6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580ac880 con 0x7f59580c9620 2023-08-25T09:42:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595fca6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580ad400 con 0x7f5958069850 2023-08-25T09:42:03.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595fca6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59580ad890 con 0x7f59580cca20 2023-08-25T09:42:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5958069850 0x7f595815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5958069850 0x7f595815e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45800/0 (socket says 172.21.15.70:45800) 2023-08-25T09:42:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 -- 172.21.15.70:0/3049281121 learned_addr learned my addr 172.21.15.70:0/3049281121 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 -- 172.21.15.70:0/3049281121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 msgr2=0x7f59580cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 --2- 172.21.15.70:0/3049281121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 0x7f59580cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 -- 172.21.15.70:0/3049281121 >> v1:172.21.15.155:6789/0 conn(0x7f59580c9620 legacy=0x7f5958069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.723+0000 7f595da42640 1 -- 172.21.15.70:0/3049281121 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580acad0 con 0x7f5958069850 2023-08-25T09:42:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.724+0000 7f595da42640 1 --2- 172.21.15.70:0/3049281121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5958069850 0x7f595815e210 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5948009e50 tx=0x7f594802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7750b81a4b8c89cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595ca40640 1 -- 172.21.15.70:0/3049281121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594802cb00 con 0x7f5958069850 2023-08-25T09:42:03.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595ca40640 1 -- 172.21.15.70:0/3049281121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594802cc60 con 0x7f5958069850 2023-08-25T09:42:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595ca40640 1 -- 172.21.15.70:0/3049281121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480343f0 con 0x7f5958069850 2023-08-25T09:42:03.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 -- 172.21.15.70:0/3049281121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5958069850 msgr2=0x7f595815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 --2- 172.21.15.70:0/3049281121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5958069850 0x7f595815e210 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5948009e50 tx=0x7f594802bf60 comp rx=0 tx=0).stop 2023-08-25T09:42:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 -- 172.21.15.70:0/3049281121 shutdown_connections 2023-08-25T09:42:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 --2- 172.21.15.70:0/3049281121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 0x7f59580cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 --2- 172.21.15.70:0/3049281121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5958069850 0x7f595815e210 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 -- 172.21.15.70:0/3049281121 >> 172.21.15.70:0/3049281121 conn(0x7f59580cb0d0 msgr2=0x7f59580cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 -- 172.21.15.70:0/3049281121 shutdown_connections 2023-08-25T09:42:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.725+0000 7f595fca6640 1 -- 172.21.15.70:0/3049281121 wait complete. 2023-08-25T09:42:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.727+0000 7f595fca6640 1 Processor -- start 2023-08-25T09:42:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.727+0000 7f595fca6640 1 -- start start 2023-08-25T09:42:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.728+0000 7f595fca6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5958069850 0x7f595806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.728+0000 7f595fca6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 0x7f59580c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.728+0000 7f595fca6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 0x7f5958156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.728+0000 7f595fca6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580c28e0 con 0x7f59580c9620 2023-08-25T09:42:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.728+0000 7f595fca6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580c20d0 con 0x7f5958069850 2023-08-25T09:42:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.728+0000 7f595fca6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59580c2320 con 0x7f59580cca20 2023-08-25T09:42:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 0x7f59580c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 0x7f59580c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45810/0 (socket says 172.21.15.70:45810) 2023-08-25T09:42:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 -- 172.21.15.70:0/4024418312 learned_addr learned my addr 172.21.15.70:0/4024418312 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 -- 172.21.15.70:0/4024418312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 msgr2=0x7f5958156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 0x7f5958156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 -- 172.21.15.70:0/4024418312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5958069850 msgr2=0x7f595806b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5958069850 0x7f595806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 -- 172.21.15.70:0/4024418312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5948009b30 con 0x7f59580c9620 2023-08-25T09:42:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f595d241640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 0x7f59580c4a60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c410 tx=0x7f594c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:03.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.729+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0035f0 con 0x7f59580c9620 2023-08-25T09:42:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.730+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c003750 con 0x7f59580c9620 2023-08-25T09:42:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.730+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c003b40 con 0x7f59580c9620 2023-08-25T09:42:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.730+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59580c2600 con 0x7f59580c9620 2023-08-25T09:42:03.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.730+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59580af560 con 0x7f59580c9620 2023-08-25T09:42:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.731+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59580af0e0 con 0x7f59580c9620 2023-08-25T09:42:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.732+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f594c003ca0 con 0x7f59580c9620 2023-08-25T09:42:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.733+0000 7f59567fc640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594406a050 0x7f594406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.733+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f594c087860 con 0x7f59580c9620 2023-08-25T09:42:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.733+0000 7f595da42640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594406a050 0x7f594406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.734+0000 7f595da42640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594406a050 0x7f594406c510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5948035000 tx=0x7f594802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.737+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59580af0e0 con 0x7f59580c9620 2023-08-25T09:42:03.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.924+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59580af0e0 con 0x7f59580c9620 2023-08-25T09:42:03.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.925+0000 7f59567fc640 1 -- 172.21.15.70:0/4024418312 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59580af0e0 con 0x7f59580c9620 2023-08-25T09:42:03.927 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:42:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.927+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594406a050 msgr2=0x7f594406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.927+0000 7f595fca6640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594406a050 0x7f594406c510 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5948035000 tx=0x7f594802e000 comp rx=0 tx=0).stop 2023-08-25T09:42:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.927+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 msgr2=0x7f59580c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:03.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.927+0000 7f595fca6640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 0x7f59580c4a60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f594c00c410 tx=0x7f594c00c8d0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 shutdown_connections 2023-08-25T09:42:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594406a050 0x7f594406c510 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59580cca20 0x7f5958156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59580c9620 0x7f59580c4a60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 --2- 172.21.15.70:0/4024418312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5958069850 0x7f595806b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 >> 172.21.15.70:0/4024418312 conn(0x7f59580cb0d0 msgr2=0x7f59580c76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:03.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 shutdown_connections 2023-08-25T09:42:03.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:03.928+0000 7f595fca6640 1 -- 172.21.15.70:0/4024418312 wait complete. 2023-08-25T09:42:03.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:42:04.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.161+0000 7fe961b4e640 1 Processor -- start 2023-08-25T09:42:04.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.161+0000 7fe961b4e640 1 -- start start 2023-08-25T09:42:04.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe961b4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe95c159aa0 0x7fe95c070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:04.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe961b4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c15df10 0x7fe95c070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:04.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe961b4e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c06aec0 con 0x7fe95c06bc80 2023-08-25T09:42:04.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe961b4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c159e80 con 0x7fe95c159aa0 2023-08-25T09:42:04.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe961b4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c06b1a0 con 0x7fe95c15df10 2023-08-25T09:42:04.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe95b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe95c159aa0 0x7fe95c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:04.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe95b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe95c159aa0 0x7fe95c070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45816/0 (socket says 172.21.15.70:45816) 2023-08-25T09:42:04.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe95b7fe640 1 -- 172.21.15.70:0/923168162 learned_addr learned my addr 172.21.15.70:0/923168162 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:04.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.163+0000 7fe95affd640 1 --1- 172.21.15.70:0/923168162 >> v1:172.21.15.70:6789/0 conn(0x7fe95c06bc80 0x7fe95c0c9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36220/0 (socket says 172.21.15.70:36220) 2023-08-25T09:42:04.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95bfff640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c15df10 0x7fe95c070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:04.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95b7fe640 1 -- 172.21.15.70:0/923168162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c15df10 msgr2=0x7fe95c070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:04.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95b7fe640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c15df10 0x7fe95c070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95b7fe640 1 -- 172.21.15.70:0/923168162 >> v1:172.21.15.70:6789/0 conn(0x7fe95c06bc80 legacy=0x7fe95c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:04.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95b7fe640 1 -- 172.21.15.70:0/923168162 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c06b480 con 0x7fe95c159aa0 2023-08-25T09:42:04.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95a7fc640 1 -- 172.21.15.70:0/923168162 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508773001 0 0) 0x7fe95c06aec0 con 0x7fe95c06bc80 2023-08-25T09:42:04.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.164+0000 7fe95bfff640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c15df10 0x7fe95c070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:04.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.165+0000 7fe95b7fe640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c159aa0 0x7fe95c070410 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009850 tx=0x7fe94c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80f605fb2c617ce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:04.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.165+0000 7fe95a7fc640 1 -- 172.21.15.70:0/923168162 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c004230 con 0x7fe95c159aa0 2023-08-25T09:42:04.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.165+0000 7fe95a7fc640 1 -- 172.21.15.70:0/923168162 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c004390 con 0x7fe95c159aa0 2023-08-25T09:42:04.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.165+0000 7fe95a7fc640 1 -- 172.21.15.70:0/923168162 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c003810 con 0x7fe95c159aa0 2023-08-25T09:42:04.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 -- 172.21.15.70:0/923168162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c159aa0 msgr2=0x7fe95c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:04.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c159aa0 0x7fe95c070410 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009850 tx=0x7fe94c02b960 comp rx=0 tx=0).stop 2023-08-25T09:42:04.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 -- 172.21.15.70:0/923168162 shutdown_connections 2023-08-25T09:42:04.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c15df10 0x7fe95c070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 --2- 172.21.15.70:0/923168162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c159aa0 0x7fe95c070410 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 -- 172.21.15.70:0/923168162 >> 172.21.15.70:0/923168162 conn(0x7fe95c0ce760 msgr2=0x7fe95c0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:04.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 -- 172.21.15.70:0/923168162 shutdown_connections 2023-08-25T09:42:04.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.166+0000 7fe961b4e640 1 -- 172.21.15.70:0/923168162 wait complete. 2023-08-25T09:42:04.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.168+0000 7fe961b4e640 1 Processor -- start 2023-08-25T09:42:04.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.168+0000 7fe961b4e640 1 -- start start 2023-08-25T09:42:04.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe961b4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c159aa0 0x7fe95c14adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:04.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe961b4e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe95c14a800 0x7fe95c154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:04.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe961b4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 0x7fe95c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:04.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe961b4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c14ac80 con 0x7fe95c0cd990 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe961b4e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0cc8f0 con 0x7fe95c14a800 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe961b4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0ccbd0 con 0x7fe95c159aa0 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 0x7fe95c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 0x7fe95c15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45818/0 (socket says 172.21.15.70:45818) 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 -- 172.21.15.70:0/4245566557 learned_addr learned my addr 172.21.15.70:0/4245566557 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 -- 172.21.15.70:0/4245566557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c159aa0 msgr2=0x7fe95c14adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:04.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c159aa0 0x7fe95c14adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 -- 172.21.15.70:0/4245566557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe95c14a800 msgr2=0x7fe95c154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe95c14a800 0x7fe95c154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.169+0000 7fe95bfff640 1 -- 172.21.15.70:0/4245566557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009460 con 0x7fe95c0cd990 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.170+0000 7fe95bfff640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 0x7fe95c15b550 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe95400cea0 tx=0x7fe95400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.170+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954003b90 con 0x7fe95c0cd990 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.170+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe954003d10 con 0x7fe95c0cd990 2023-08-25T09:42:04.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.170+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954004260 con 0x7fe95c0cd990 2023-08-25T09:42:04.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.170+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c0cceb0 con 0x7fe95c0cd990 2023-08-25T09:42:04.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.170+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0cd470 con 0x7fe95c0cd990 2023-08-25T09:42:04.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.171+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c04e420 con 0x7fe95c0cd990 2023-08-25T09:42:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.172+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9540043c0 con 0x7fe95c0cd990 2023-08-25T09:42:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.173+0000 7fe943fff640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe94806a050 0x7fe94806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.173+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe954088eb0 con 0x7fe95c0cd990 2023-08-25T09:42:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.173+0000 7fe95b7fe640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe94806a050 0x7fe94806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.174+0000 7fe95b7fe640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe94806a050 0x7fe94806c510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe94c035000 tx=0x7fe94c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:04.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.177+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe954054570 con 0x7fe95c0cd990 2023-08-25T09:42:04.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.381+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe95c0c8b70 con 0x7fe95c0cd990 2023-08-25T09:42:04.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.382+0000 7fe943fff640 1 -- 172.21.15.70:0/4245566557 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v61) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe954057b90 con 0x7fe95c0cd990 2023-08-25T09:42:04.385 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:42:04.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe94806a050 msgr2=0x7fe94806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:04.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe94806a050 0x7fe94806c510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe94c035000 tx=0x7fe94c030000 comp rx=0 tx=0).stop 2023-08-25T09:42:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 msgr2=0x7fe95c15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 0x7fe95c15b550 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe95400cea0 tx=0x7fe95400cf90 comp rx=0 tx=0).stop 2023-08-25T09:42:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 shutdown_connections 2023-08-25T09:42:04.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe94806a050 0x7fe94806c510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe95c0cd990 0x7fe95c15b550 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe95c14a800 0x7fe95c154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 --2- 172.21.15.70:0/4245566557 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe95c159aa0 0x7fe95c14adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 >> 172.21.15.70:0/4245566557 conn(0x7fe95c0ce760 msgr2=0x7fe95c0703c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.385+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 shutdown_connections 2023-08-25T09:42:04.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:04.386+0000 7fe961b4e640 1 -- 172.21.15.70:0/4245566557 wait complete. 2023-08-25T09:42:04.406 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:42:04.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:04.407 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:04.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[88654]: fuse finished with error 0 and tester_r 0 2023-08-25T09:42:04.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:42:04.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:04.508 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:42:04.556 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:42:04.557 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:04.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:42:04.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:42:04.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:04.621 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:42:04.721 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:42:04.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:42:04.723 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:42:04.724 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:04.783 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:42:04.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:42:04.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:04.785 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:42:04.839 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:42:04.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:42:04.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.062+0000 7ff3fbd4f640 1 Processor -- start 2023-08-25T09:42:05.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.062+0000 7ff3fbd4f640 1 -- start start 2023-08-25T09:42:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fbd4f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f415e830 0x7ff3f406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fbd4f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3f415ec10 0x7ff3f40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fbd4f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f40b1b60 con 0x7ff3f406b2b0 2023-08-25T09:42:05.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fbd4f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f406bd80 con 0x7ff3f415ec10 2023-08-25T09:42:05.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fbd4f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f406b7c0 con 0x7ff3f415e830 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fa2ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3f415ec10 0x7ff3f40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fa2ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3f415ec10 0x7ff3f40b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45828/0 (socket says 172.21.15.70:45828) 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.064+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/2492071696 learned_addr learned my addr 172.21.15.70:0/2492071696 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.065+0000 7ff3f9aeb640 1 --2- 172.21.15.70:0/2492071696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f415e830 0x7ff3f406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.065+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/2492071696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f415e830 msgr2=0x7ff3f406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.065+0000 7ff3fa2ec640 1 --2- 172.21.15.70:0/2492071696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f415e830 0x7ff3f406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.065+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/2492071696 >> v1:172.21.15.70:6789/0 conn(0x7ff3f406b2b0 legacy=0x7ff3f415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.065+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/2492071696 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f406baa0 con 0x7ff3f415ec10 2023-08-25T09:42:05.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.065+0000 7ff3f8ae9640 1 -- 172.21.15.70:0/2492071696 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688787907 0 0) 0x7ff3f40b1b60 con 0x7ff3f406b2b0 2023-08-25T09:42:05.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.066+0000 7ff3fa2ec640 1 --2- 172.21.15.70:0/2492071696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f40b1ff0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009e50 tx=0x7ff3ec02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de127b55a69039 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3f8ae9640 1 -- 172.21.15.70:0/2492071696 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec02cb00 con 0x7ff3f415ec10 2023-08-25T09:42:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3f8ae9640 1 -- 172.21.15.70:0/2492071696 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec02cc60 con 0x7ff3f415ec10 2023-08-25T09:42:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3f8ae9640 1 -- 172.21.15.70:0/2492071696 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec0343f0 con 0x7ff3f415ec10 2023-08-25T09:42:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/2492071696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 msgr2=0x7ff3f40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/2492071696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f40b1ff0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009e50 tx=0x7ff3ec02bf60 comp rx=0 tx=0).stop 2023-08-25T09:42:05.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/2492071696 shutdown_connections 2023-08-25T09:42:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/2492071696 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f40b1ff0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/2492071696 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f415e830 0x7ff3f406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/2492071696 >> 172.21.15.70:0/2492071696 conn(0x7ff3f4069850 msgr2=0x7ff3f4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/2492071696 shutdown_connections 2023-08-25T09:42:05.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.067+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/2492071696 wait complete. 2023-08-25T09:42:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.070+0000 7ff3fbd4f640 1 Processor -- start 2023-08-25T09:42:05.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.070+0000 7ff3fbd4f640 1 -- start start 2023-08-25T09:42:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fbd4f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f406b2b0 0x7ff3f40c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fbd4f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3f415e830 0x7ff3f41f0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fbd4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f415ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fbd4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40ca120 con 0x7ff3f415ec10 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fbd4f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f40c4b50 con 0x7ff3f415e830 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fbd4f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f41f0e60 con 0x7ff3f406b2b0 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fa2ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f415ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fa2ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f415ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45836/0 (socket says 172.21.15.70:45836) 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.071+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/830439246 learned_addr learned my addr 172.21.15.70:0/830439246 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3f9aeb640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f406b2b0 0x7ff3f40c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3f92ea640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3f415e830 0x7ff3f41f0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f406b2b0 msgr2=0x7ff3f40c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fa2ec640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f406b2b0 0x7ff3f40c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3f415e830 msgr2=0x7ff3f41f0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fa2ec640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3f415e830 0x7ff3f41f0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fa2ec640 1 -- 172.21.15.70:0/830439246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec009b30 con 0x7ff3f415ec10 2023-08-25T09:42:05.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3f92ea640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3f415e830 0x7ff3f41f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fa2ec640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f415ce20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009e50 tx=0x7ff3ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:05.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec009510 con 0x7ff3f415ec10 2023-08-25T09:42:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec03e450 con 0x7ff3f415ec10 2023-08-25T09:42:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f415ca20 con 0x7ff3f415ec10 2023-08-25T09:42:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.072+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec03e7e0 con 0x7ff3f415ec10 2023-08-25T09:42:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.073+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f41f1b50 con 0x7ff3f415ec10 2023-08-25T09:42:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.074+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f404e420 con 0x7ff3f415ec10 2023-08-25T09:42:05.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.075+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3ec03e940 con 0x7ff3f415ec10 2023-08-25T09:42:05.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.076+0000 7ff3ea7fc640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3d806a050 0x7ff3d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.076+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7ff3ec0ab740 con 0x7ff3f415ec10 2023-08-25T09:42:05.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.076+0000 7ff3f9aeb640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3d806a050 0x7ff3d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.077+0000 7ff3f9aeb640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3d806a050 0x7ff3d806c510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:05.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.080+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3ec076d70 con 0x7ff3f415ec10 2023-08-25T09:42:05.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.304+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3f4156900 con 0x7ff3f415ec10 2023-08-25T09:42:05.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.305+0000 7ff3ea7fc640 1 -- 172.21.15.70:0/830439246 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7ff3ec07a390 con 0x7ff3f415ec10 2023-08-25T09:42:05.316 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:05.316 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:42:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:42:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3d806a050 msgr2=0x7ff3d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3d806a050 0x7ff3d806c510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0008040 comp rx=0 tx=0).stop 2023-08-25T09:42:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 msgr2=0x7ff3f415ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f415ce20 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec009e50 tx=0x7ff3ec004a40 comp rx=0 tx=0).stop 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 shutdown_connections 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3d806a050 0x7ff3d806c510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3f415ec10 0x7ff3f415ce20 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3f415e830 0x7ff3f41f0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 --2- 172.21.15.70:0/830439246 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3f406b2b0 0x7ff3f40c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 >> 172.21.15.70:0/830439246 conn(0x7ff3f4069850 msgr2=0x7ff3f40cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:05.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 shutdown_connections 2023-08-25T09:42:05.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.308+0000 7ff3fbd4f640 1 -- 172.21.15.70:0/830439246 wait complete. 2023-08-25T09:42:05.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:05.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.539+0000 7f79af287640 1 Processor -- start 2023-08-25T09:42:05.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.539+0000 7f79af287640 1 -- start start 2023-08-25T09:42:05.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79af287640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f79a815e830 0x7f79a806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79af287640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 0x7f79a815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79af287640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a80b1b60 con 0x7f79a815c6d0 2023-08-25T09:42:05.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79af287640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a806bd80 con 0x7f79a815e830 2023-08-25T09:42:05.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79af287640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a806b7c0 con 0x7f79a806b2b0 2023-08-25T09:42:05.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79ad824640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f79a815c6d0 0x7f79a80b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36230/0 (socket says 172.21.15.70:36230) 2023-08-25T09:42:05.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.541+0000 7f79ad824640 1 -- 172.21.15.70:0/21432400 learned_addr learned my addr 172.21.15.70:0/21432400 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:05.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f79ac822640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 0x7f79a815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f79ad023640 1 --2- 172.21.15.70:0/21432400 >> v2:172.21.15.155:3300/0 conn(0x7f79a815e830 0x7f79a806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320252480 0 0) 0x7f79a80b1b60 con 0x7f79a815c6d0 2023-08-25T09:42:05.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7994003520 con 0x7f79a815c6d0 2023-08-25T09:42:05.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1093903007 0 0) 0x7f7994003520 con 0x7f79a815c6d0 2023-08-25T09:42:05.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 msgr2=0x7f79a815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:05.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 0x7f79a815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 >> v2:172.21.15.155:3300/0 conn(0x7f79a815e830 msgr2=0x7f79a806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 --2- 172.21.15.70:0/21432400 >> v2:172.21.15.155:3300/0 conn(0x7f79a815e830 0x7f79a806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a806baa0 con 0x7f79a815c6d0 2023-08-25T09:42:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f79a00019f0 con 0x7f79a815c6d0 2023-08-25T09:42:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 >> v1:172.21.15.70:6789/0 conn(0x7f79a815c6d0 legacy=0x7f79a80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7994003970 0x7f7994003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 0x7f79940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7994004d00 0x7f7994005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a806bd80 con 0x7f7994003970 2023-08-25T09:42:05.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a806b7c0 con 0x7f7994004d00 2023-08-25T09:42:05.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.542+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799400b980 con 0x7f79a806b2b0 2023-08-25T09:42:05.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.543+0000 7f79ad023640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7994003970 0x7f7994003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.543+0000 7f79ad023640 1 -- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 msgr2=0x7f79940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:05.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.543+0000 7f79ad023640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79a806b2b0 0x7f79940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.543+0000 7f79ad023640 1 -- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7994004d00 msgr2=0x7f7994005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:05.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.543+0000 7f79ad023640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7994004d00 0x7f7994005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.543+0000 7f79ad023640 1 -- 172.21.15.70:0/21432400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994003520 con 0x7f7994003970 2023-08-25T09:42:05.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.544+0000 7f79ad023640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7994003970 0x7f7994003d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009ac0 tx=0x7f79940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03b20c9d2e1ec6a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:05.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.544+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a402c6f0 con 0x7f7994003970 2023-08-25T09:42:05.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.544+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a402c850 con 0x7f7994003970 2023-08-25T09:42:05.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.544+0000 7f799ffff640 1 -- 172.21.15.70:0/21432400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a402cbe0 con 0x7f7994003970 2023-08-25T09:42:05.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79af287640 1 -- 172.21.15.70:0/21432400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7994003970 msgr2=0x7f7994003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79af287640 1 --2- 172.21.15.70:0/21432400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7994003970 0x7f7994003d90 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009ac0 tx=0x7f79940057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79ad824640 1 -- 172.21.15.70:0/21432400 reap_dead start 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79af287640 1 -- 172.21.15.70:0/21432400 shutdown_connections 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79af287640 1 -- 172.21.15.70:0/21432400 >> 172.21.15.70:0/21432400 conn(0x7f79a8069850 msgr2=0x7f79a8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79af287640 1 -- 172.21.15.70:0/21432400 shutdown_connections 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.545+0000 7f79af287640 1 -- 172.21.15.70:0/21432400 wait complete. 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.547+0000 7f79af287640 1 Processor -- start 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.547+0000 7f79af287640 1 -- start start 2023-08-25T09:42:05.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79af287640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7994003970 0x7f79a81552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79af287640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 0x7f79a81557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79af287640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a8155d20 0x7f79a806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79af287640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8154240 con 0x7f79a8150ae0 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79af287640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a81544e0 con 0x7f79a8155d20 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79af287640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a81547c0 con 0x7f7994003970 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79ac822640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 0x7f79a81557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.548+0000 7f79ac822640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 0x7f79a81557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45868/0 (socket says 172.21.15.70:45868) 2023-08-25T09:42:05.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 -- 172.21.15.70:0/1994879861 learned_addr learned my addr 172.21.15.70:0/1994879861 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:05.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 -- 172.21.15.70:0/1994879861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7994003970 msgr2=0x7f79a81552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7994003970 0x7f79a81552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 -- 172.21.15.70:0/1994879861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a8155d20 msgr2=0x7f79a806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a8155d20 0x7f79a806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 -- 172.21.15.70:0/1994879861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8154aa0 con 0x7f79a8150ae0 2023-08-25T09:42:05.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79ac822640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 0x7f79a81557e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f799800e7f0 tx=0x7f799800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:05.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800eea0 con 0x7f79a8150ae0 2023-08-25T09:42:05.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799800c940 con 0x7f79a8150ae0 2023-08-25T09:42:05.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799800ccd0 con 0x7f79a8150ae0 2023-08-25T09:42:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.549+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a8154d80 con 0x7f79a8150ae0 2023-08-25T09:42:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.550+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a806f220 con 0x7f79a8150ae0 2023-08-25T09:42:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.551+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a804e420 con 0x7f79a8150ae0 2023-08-25T09:42:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.551+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7998013470 con 0x7f79a8150ae0 2023-08-25T09:42:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.552+0000 7f799dffb640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f798c06a080 0x7f798c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.552+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7998088650 con 0x7f79a8150ae0 2023-08-25T09:42:05.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.553+0000 7f79ad023640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f798c06a080 0x7f798c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:05.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.553+0000 7f79ad023640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f798c06a080 0x7f798c06c540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f79a4036000 tx=0x7f79a402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:05.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.557+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7998053d10 con 0x7f79a8150ae0 2023-08-25T09:42:05.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.746+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79a806c620 con 0x7f79a8150ae0 2023-08-25T09:42:05.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.749+0000 7f799dffb640 1 -- 172.21.15.70:0/1994879861 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18353 (secure 0 0 0) 0x7f7998057330 con 0x7f79a8150ae0 2023-08-25T09:42:05.756 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:05.756 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":61,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:53.340729+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.9","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:05.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f798c06a080 msgr2=0x7f798c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f798c06a080 0x7f798c06c540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f79a4036000 tx=0x7f79a402e000 comp rx=0 tx=0).stop 2023-08-25T09:42:05.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 msgr2=0x7f79a81557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:05.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 0x7f79a81557e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f799800e7f0 tx=0x7f799800ecb0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 shutdown_connections 2023-08-25T09:42:05.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f798c06a080 0x7f798c06c540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.752+0000 7f79af287640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79a8155d20 0x7f79a806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.753+0000 7f79af287640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79a8150ae0 0x7f79a81557e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.753+0000 7f79af287640 1 --2- 172.21.15.70:0/1994879861 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7994003970 0x7f79a81552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:05.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.753+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 >> 172.21.15.70:0/1994879861 conn(0x7f79a8069850 msgr2=0x7f79a80cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:05.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.753+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 shutdown_connections 2023-08-25T09:42:05.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.753+0000 7f79af287640 1 -- 172.21.15.70:0/1994879861 wait complete. 2023-08-25T09:42:05.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:41:11.244136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:41:11.990910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:42:05.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:42:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.992+0000 7f9ccec6a640 1 Processor -- start 2023-08-25T09:42:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.993+0000 7f9ccec6a640 1 -- start start 2023-08-25T09:42:05.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.994+0000 7f9ccec6a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 0x7f9cc815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:05.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.994+0000 7f9ccec6a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80ac880 con 0x7f9cc80c9620 2023-08-25T09:42:05.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.994+0000 7f9ccec6a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80ad400 con 0x7f9cc80cca20 2023-08-25T09:42:05.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.994+0000 7f9ccec6a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80ad890 con 0x7f9cc8069850 2023-08-25T09:42:05.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.994+0000 7f9cc7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9cc80c9620 0x7f9cc8069c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36246/0 (socket says 172.21.15.70:36246) 2023-08-25T09:42:05.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.994+0000 7f9cc7fff640 1 -- 172.21.15.70:0/704417885 learned_addr learned my addr 172.21.15.70:0/704417885 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:06.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864456780 0 0) 0x7f9cc80ac880 con 0x7f9cc80c9620 2023-08-25T09:42:06.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb4003520 con 0x7f9cc80c9620 2023-08-25T09:42:06.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9ccca06640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 0x7f9cc815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1144639092 0 0) 0x7f9cb4003520 con 0x7f9cc80c9620 2023-08-25T09:42:06.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 msgr2=0x7f9cc815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:06.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 0x7f9cc815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 >> v1:172.21.15.155:6789/0 conn(0x7f9cc80cca20 legacy=0x7f9cc80cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80acad0 con 0x7f9cc80c9620 2023-08-25T09:42:06.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9cb80019f0 con 0x7f9cc80c9620 2023-08-25T09:42:06.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 >> v1:172.21.15.70:6789/0 conn(0x7f9cc80c9620 legacy=0x7f9cc8069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 0x7f9cb40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80ad890 con 0x7f9cb4003fd0 2023-08-25T09:42:06.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb400bbd0 con 0x7f9cb4004d00 2023-08-25T09:42:06.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.995+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb400bd90 con 0x7f9cc8069850 2023-08-25T09:42:06.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9ccd207640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9ccca06640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 0x7f9cb40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9cc7fff640 1 -- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 msgr2=0x7f9cb40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:06.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc8069850 0x7f9cb40039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9cc7fff640 1 -- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb4004d00 msgr2=0x7f9cb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:06.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9cc7fff640 1 -- 172.21.15.70:0/704417885 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb4003520 con 0x7f9cb4003fd0 2023-08-25T09:42:06.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.996+0000 7f9ccd207640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:06.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.997+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cb40047c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800a830 tx=0x7f9cb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4737292dd036fd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.997+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80072d0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.997+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb802f070 con 0x7f9cb4003fd0 2023-08-25T09:42:06.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.997+0000 7f9cc77fe640 1 -- 172.21.15.70:0/704417885 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb80076f0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccec6a640 1 -- 172.21.15.70:0/704417885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 msgr2=0x7f9cb40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/704417885 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cb40047c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800a830 tx=0x7f9cb40057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccd207640 1 -- 172.21.15.70:0/704417885 reap_dead start 2023-08-25T09:42:06.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccec6a640 1 -- 172.21.15.70:0/704417885 shutdown_connections 2023-08-25T09:42:06.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccec6a640 1 -- 172.21.15.70:0/704417885 >> 172.21.15.70:0/704417885 conn(0x7f9cc80cb0d0 msgr2=0x7f9cc80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:06.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccec6a640 1 -- 172.21.15.70:0/704417885 shutdown_connections 2023-08-25T09:42:06.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:05.998+0000 7f9ccec6a640 1 -- 172.21.15.70:0/704417885 wait complete. 2023-08-25T09:42:06.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.000+0000 7f9ccec6a640 1 Processor -- start 2023-08-25T09:42:06.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.000+0000 7f9ccec6a640 1 -- start start 2023-08-25T09:42:06.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccec6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cc80c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccec6a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc815f630 0x7f9cc814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccec6a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc815ddb0 0x7f9cc806bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccec6a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc814f2b0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccec6a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc815e590 con 0x7f9cc815f630 2023-08-25T09:42:06.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccec6a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc815e870 con 0x7f9cc815ddb0 2023-08-25T09:42:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccca06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cc80c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccca06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cc80c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45884/0 (socket says 172.21.15.70:45884) 2023-08-25T09:42:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.001+0000 7f9ccca06640 1 -- 172.21.15.70:0/2238549587 learned_addr learned my addr 172.21.15.70:0/2238549587 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc815f630 0x7f9cc814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccca06640 1 -- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc815ddb0 msgr2=0x7f9cc806bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccca06640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc815ddb0 0x7f9cc806bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccca06640 1 -- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc815f630 msgr2=0x7f9cc814ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccca06640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc815f630 0x7f9cc814ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccca06640 1 -- 172.21.15.70:0/2238549587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc815eb50 con 0x7f9cb4003fd0 2023-08-25T09:42:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccca06640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cc80c42f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000e7e0 tx=0x7f9cb000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000c6d0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb000c830 con 0x7f9cb4003fd0 2023-08-25T09:42:06.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000cba0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc815ee30 con 0x7f9cb4003fd0 2023-08-25T09:42:06.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.002+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc815e230 con 0x7f9cb4003fd0 2023-08-25T09:42:06.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.003+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc8066fb0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.004+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9cb000cd00 con 0x7f9cb4003fd0 2023-08-25T09:42:06.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.005+0000 7f9cc57fa640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ca806a050 0x7f9ca806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.005+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f9cb0088410 con 0x7f9cb4003fd0 2023-08-25T09:42:06.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.005+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ca806a050 0x7f9ca806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.006+0000 7f9cc7fff640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ca806a050 0x7f9ca806c510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8034000 tx=0x7f9cb8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.009+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9cb0053ae0 con 0x7f9cb4003fd0 2023-08-25T09:42:06.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.183+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cc806a810 con 0x7f9cb4003fd0 2023-08-25T09:42:06.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.184+0000 7f9cc57fa640 1 -- 172.21.15.70:0/2238549587 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cb0057100 con 0x7f9cb4003fd0 2023-08-25T09:42:06.186 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:42:06.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.186+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ca806a050 msgr2=0x7f9ca806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.186+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ca806a050 0x7f9ca806c510 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8034000 tx=0x7f9cb8007000 comp rx=0 tx=0).stop 2023-08-25T09:42:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.186+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 msgr2=0x7f9cc80c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.186+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cc80c42f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000e7e0 tx=0x7f9cb000eca0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 shutdown_connections 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ca806a050 0x7f9ca806c510 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cc815ddb0 0x7f9cc806bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cc815f630 0x7f9cc814ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 --2- 172.21.15.70:0/2238549587 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cb4003fd0 0x7f9cc80c42f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 >> 172.21.15.70:0/2238549587 conn(0x7f9cc80cb0d0 msgr2=0x7f9cc80c5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 shutdown_connections 2023-08-25T09:42:06.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.187+0000 7f9ccec6a640 1 -- 172.21.15.70:0/2238549587 wait complete. 2023-08-25T09:42:06.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.408+0000 7fd6d81f5640 1 Processor -- start 2023-08-25T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.408+0000 7fd6d81f5640 1 -- start start 2023-08-25T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d81f5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd6d0159bf0 0x7fd6d015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d81f5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d006f0e0 0x7fd6d0070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d81f5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d015e780 con 0x7fd6d00cd660 2023-08-25T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d81f5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d006f8e0 con 0x7fd6d0159bf0 2023-08-25T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d81f5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d006fbc0 con 0x7fd6d006f0e0 2023-08-25T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d5f91640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd6d0159bf0 0x7fd6d015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d5f91640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd6d0159bf0 0x7fd6d015e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45894/0 (socket says 172.21.15.70:45894) 2023-08-25T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.409+0000 7fd6d5f91640 1 -- 172.21.15.70:0/2697400454 learned_addr learned my addr 172.21.15.70:0/2697400454 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.410+0000 7fd6d5f91640 1 -- 172.21.15.70:0/2697400454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d006f0e0 msgr2=0x7fd6d0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.410+0000 7fd6d5f91640 1 --2- 172.21.15.70:0/2697400454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d006f0e0 0x7fd6d0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.410+0000 7fd6d5f91640 1 -- 172.21.15.70:0/2697400454 >> v1:172.21.15.155:6789/0 conn(0x7fd6d00cd660 legacy=0x7fd6d00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.410+0000 7fd6d5f91640 1 -- 172.21.15.70:0/2697400454 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d006fea0 con 0x7fd6d0159bf0 2023-08-25T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.411+0000 7fd6d5f91640 1 --2- 172.21.15.70:0/2697400454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d015e240 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00bb10 tx=0x7fd6cc02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17e977b161e7dcf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.411+0000 7fd6d4f8f640 1 -- 172.21.15.70:0/2697400454 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc02ee70 con 0x7fd6d0159bf0 2023-08-25T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.411+0000 7fd6d4f8f640 1 -- 172.21.15.70:0/2697400454 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc036440 con 0x7fd6d0159bf0 2023-08-25T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.411+0000 7fd6d4f8f640 1 -- 172.21.15.70:0/2697400454 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc0367d0 con 0x7fd6d0159bf0 2023-08-25T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 -- 172.21.15.70:0/2697400454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 msgr2=0x7fd6d015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/2697400454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d015e240 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc00bb10 tx=0x7fd6cc02e0c0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 -- 172.21.15.70:0/2697400454 shutdown_connections 2023-08-25T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/2697400454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d006f0e0 0x7fd6d0070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/2697400454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d015e240 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 -- 172.21.15.70:0/2697400454 >> 172.21.15.70:0/2697400454 conn(0x7fd6d00cb0d0 msgr2=0x7fd6d00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 -- 172.21.15.70:0/2697400454 shutdown_connections 2023-08-25T09:42:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.412+0000 7fd6d81f5640 1 -- 172.21.15.70:0/2697400454 wait complete. 2023-08-25T09:42:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.413+0000 7fd6d81f5640 1 Processor -- start 2023-08-25T09:42:06.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.414+0000 7fd6d81f5640 1 -- start start 2023-08-25T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.414+0000 7fd6d81f5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d006f0e0 0x7fd6d0069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.414+0000 7fd6d81f5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d00cd660 0x7fd6d00af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d81f5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d00c0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d81f5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0148630 con 0x7fd6d0159bf0 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d81f5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01f5d30 con 0x7fd6d006f0e0 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d81f5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d01f6010 con 0x7fd6d00cd660 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d00c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d00c0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45904/0 (socket says 172.21.15.70:45904) 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 -- 172.21.15.70:0/1017068299 learned_addr learned my addr 172.21.15.70:0/1017068299 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 -- 172.21.15.70:0/1017068299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d00cd660 msgr2=0x7fd6d00af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d00cd660 0x7fd6d00af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 -- 172.21.15.70:0/1017068299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d006f0e0 msgr2=0x7fd6d0069ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d006f0e0 0x7fd6d0069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 -- 172.21.15.70:0/1017068299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc00b770 con 0x7fd6d0159bf0 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.415+0000 7fd6d6792640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d00c0570 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c780 tx=0x7fd6c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.416+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8003d80 con 0x7fd6d0159bf0 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.416+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c800ce90 con 0x7fd6d0159bf0 2023-08-25T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.416+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80052b0 con 0x7fd6d0159bf0 2023-08-25T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.416+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01f62f0 con 0x7fd6d0159bf0 2023-08-25T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.416+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01f68b0 con 0x7fd6d0159bf0 2023-08-25T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.417+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d00c5690 con 0x7fd6d0159bf0 2023-08-25T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.418+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd6c8005410 con 0x7fd6d0159bf0 2023-08-25T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.419+0000 7fd6c67fc640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd6b406a050 0x7fd6b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.419+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd6c8087b60 con 0x7fd6d0159bf0 2023-08-25T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.419+0000 7fd6d5f91640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd6b406a050 0x7fd6b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.420+0000 7fd6d5f91640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd6b406a050 0x7fd6b406c510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc038000 tx=0x7fd6cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.423+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd6d00c5690 con 0x7fd6d0159bf0 2023-08-25T09:42:06.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.608+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6d00c5690 con 0x7fd6d0159bf0 2023-08-25T09:42:06.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.609+0000 7fd6c67fc640 1 -- 172.21.15.70:0/1017068299 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6d00c5690 con 0x7fd6d0159bf0 2023-08-25T09:42:06.612 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:42:06.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd6b406a050 msgr2=0x7fd6b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd6b406a050 0x7fd6b406c510 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc038000 tx=0x7fd6cc008040 comp rx=0 tx=0).stop 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 msgr2=0x7fd6d00c0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d00c0570 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c780 tx=0x7fd6c800cc40 comp rx=0 tx=0).stop 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 shutdown_connections 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd6b406a050 0x7fd6b406c510 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6d0159bf0 0x7fd6d00c0570 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6d00cd660 0x7fd6d00af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 --2- 172.21.15.70:0/1017068299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd6d006f0e0 0x7fd6d0069ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 >> 172.21.15.70:0/1017068299 conn(0x7fd6d00cb0d0 msgr2=0x7fd6d00ccdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:06.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 shutdown_connections 2023-08-25T09:42:06.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.611+0000 7fd6d81f5640 1 -- 172.21.15.70:0/1017068299 wait complete. 2023-08-25T09:42:06.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:42:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.841+0000 7f8585131640 1 Processor -- start 2023-08-25T09:42:06.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.841+0000 7f8585131640 1 -- start start 2023-08-25T09:42:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.842+0000 7f8585131640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f858015e240 0x7f8580070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.842+0000 7f8585131640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858006f8e0 con 0x7f85800cd660 2023-08-25T09:42:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.842+0000 7f8585131640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858006fbc0 con 0x7f8580159bf0 2023-08-25T09:42:06.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.842+0000 7f8585131640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858006fea0 con 0x7f858015e240 2023-08-25T09:42:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f8577fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f858015e240 0x7f8580070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f8577fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f858015e240 0x7f8580070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45912/0 (socket says 172.21.15.70:45912) 2023-08-25T09:42:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f8577fff640 1 -- 172.21.15.70:0/3792539378 learned_addr learned my addr 172.21.15.70:0/3792539378 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:06.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f8577fff640 1 -- 172.21.15.70:0/3792539378 >> v1:172.21.15.155:6789/0 conn(0x7f8580159bf0 legacy=0x7f858006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f8577fff640 1 -- 172.21.15.70:0/3792539378 >> v1:172.21.15.70:6789/0 conn(0x7f85800cd660 legacy=0x7f85800c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f857e59c640 1 -- 172.21.15.70:0/3792539378 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401212636 0 0) 0x7f858006f8e0 con 0x7f85800cd660 2023-08-25T09:42:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.843+0000 7f8577fff640 1 -- 172.21.15.70:0/3792539378 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580070180 con 0x7f858015e240 2023-08-25T09:42:06.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.845+0000 7f8577fff640 1 --2- 172.21.15.70:0/3792539378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f8580070980 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8568009870 tx=0x7f856802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7019fc5a33a6bb3f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.845+0000 7f857e59c640 1 -- 172.21.15.70:0/3792539378 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568030860 con 0x7f858015e240 2023-08-25T09:42:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.845+0000 7f857e59c640 1 -- 172.21.15.70:0/3792539378 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85680309c0 con 0x7f858015e240 2023-08-25T09:42:06.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.845+0000 7f857e59c640 1 -- 172.21.15.70:0/3792539378 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568030d50 con 0x7f858015e240 2023-08-25T09:42:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 -- 172.21.15.70:0/3792539378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 msgr2=0x7f8580070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 --2- 172.21.15.70:0/3792539378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f8580070980 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8568009870 tx=0x7f856802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 -- 172.21.15.70:0/3792539378 shutdown_connections 2023-08-25T09:42:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 --2- 172.21.15.70:0/3792539378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f8580070980 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 -- 172.21.15.70:0/3792539378 >> 172.21.15.70:0/3792539378 conn(0x7f85800cb0d0 msgr2=0x7f85800cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:06.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 -- 172.21.15.70:0/3792539378 shutdown_connections 2023-08-25T09:42:06.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.846+0000 7f8585131640 1 -- 172.21.15.70:0/3792539378 wait complete. 2023-08-25T09:42:06.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.848+0000 7f8585131640 1 Processor -- start 2023-08-25T09:42:06.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.848+0000 7f8585131640 1 -- start start 2023-08-25T09:42:06.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8585131640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800cd660 0x7f85800c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8585131640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8580159bf0 0x7f8580155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8585131640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f85801558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8585131640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800c85b0 con 0x7f858015e240 2023-08-25T09:42:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8585131640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85800c7510 con 0x7f85800cd660 2023-08-25T09:42:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8585131640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85800c77f0 con 0x7f8580159bf0 2023-08-25T09:42:06.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f857f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f85801558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f857f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f85801558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45916/0 (socket says 172.21.15.70:45916) 2023-08-25T09:42:06.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f857f59e640 1 -- 172.21.15.70:0/2447766863 learned_addr learned my addr 172.21.15.70:0/2447766863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:06.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f857ed9d640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800cd660 0x7f85800c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.849+0000 7f8577fff640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8580159bf0 0x7f8580155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857f59e640 1 -- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8580159bf0 msgr2=0x7f8580155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857f59e640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8580159bf0 0x7f8580155390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857f59e640 1 -- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800cd660 msgr2=0x7f85800c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857f59e640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800cd660 0x7f85800c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857f59e640 1 -- 172.21.15.70:0/2447766863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85680094d0 con 0x7f858015e240 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857ed9d640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800cd660 0x7f85800c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f857f59e640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f85801558d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f857800c760 tx=0x7f857800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578003da0 con 0x7f858015e240 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857800ce70 con 0x7f858015e240 2023-08-25T09:42:06.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85800c7ad0 con 0x7f858015e240 2023-08-25T09:42:06.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8578005290 con 0x7f858015e240 2023-08-25T09:42:06.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.850+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85800c8090 con 0x7f858015e240 2023-08-25T09:42:06.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.852+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8580066fb0 con 0x7f858015e240 2023-08-25T09:42:06.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.852+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8578005410 con 0x7f858015e240 2023-08-25T09:42:06.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.854+0000 7f85777fe640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f856006a050 0x7f856006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:06.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.854+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8578088430 con 0x7f858015e240 2023-08-25T09:42:06.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.854+0000 7f857ed9d640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f856006a050 0x7f856006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:06.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.855+0000 7f857ed9d640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f856006a050 0x7f856006c510 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8570003eb0 tx=0x7f8570009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:06.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:06.858+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8578053af0 con 0x7f858015e240 2023-08-25T09:42:07.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.047+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85800c1a00 con 0x7f858015e240 2023-08-25T09:42:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.048+0000 7f85777fe640 1 -- 172.21.15.70:0/2447766863 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8578057110 con 0x7f858015e240 2023-08-25T09:42:07.049 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:42:07.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f856006a050 msgr2=0x7f856006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f856006a050 0x7f856006c510 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8570003eb0 tx=0x7f8570009040 comp rx=0 tx=0).stop 2023-08-25T09:42:07.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 msgr2=0x7f85801558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f85801558d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f857800c760 tx=0x7f857800cc20 comp rx=0 tx=0).stop 2023-08-25T09:42:07.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 shutdown_connections 2023-08-25T09:42:07.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f856006a050 0x7f856006c510 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f858015e240 0x7f85801558d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8580159bf0 0x7f8580155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 --2- 172.21.15.70:0/2447766863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f85800cd660 0x7f85800c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.050+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 >> 172.21.15.70:0/2447766863 conn(0x7f85800cb0d0 msgr2=0x7f85800ca280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:07.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.051+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 shutdown_connections 2023-08-25T09:42:07.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.051+0000 7f8585131640 1 -- 172.21.15.70:0/2447766863 wait complete. 2023-08-25T09:42:07.072 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:42:07.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.286+0000 7f8687bc4640 1 Processor -- start 2023-08-25T09:42:07.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.286+0000 7f8687bc4640 1 -- start start 2023-08-25T09:42:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.287+0000 7f8687bc4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8680069d30 0x7f86800ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.287+0000 7f8687bc4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f868006c510 0x7f868006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.287+0000 7f8687bc4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86800cd4e0 con 0x7f868006a110 2023-08-25T09:42:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8687bc4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800cc550 con 0x7f868006c510 2023-08-25T09:42:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8687bc4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86800ccb10 con 0x7f8680069d30 2023-08-25T09:42:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8686161640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f868006c510 0x7f868006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f868515f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f868006a110 0x7f86800ca6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36258/0 (socket says 172.21.15.70:36258) 2023-08-25T09:42:07.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f868515f640 1 -- 172.21.15.70:0/2144146277 learned_addr learned my addr 172.21.15.70:0/2144146277 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8685960640 1 --2- 172.21.15.70:0/2144146277 >> v2:172.21.15.155:3300/0 conn(0x7f8680069d30 0x7f86800ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8686161640 1 -- 172.21.15.70:0/2144146277 >> v2:172.21.15.155:3300/0 conn(0x7f8680069d30 msgr2=0x7f86800ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8686161640 1 --2- 172.21.15.70:0/2144146277 >> v2:172.21.15.155:3300/0 conn(0x7f8680069d30 0x7f86800ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8686161640 1 -- 172.21.15.70:0/2144146277 >> v1:172.21.15.70:6789/0 conn(0x7f868006a110 legacy=0x7f86800ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f8686161640 1 -- 172.21.15.70:0/2144146277 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800ade40 con 0x7f868006c510 2023-08-25T09:42:07.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.288+0000 7f868495e640 1 -- 172.21.15.70:0/2144146277 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445675519 0 0) 0x7f8670001110 con 0x7f868006a110 2023-08-25T09:42:07.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.289+0000 7f8686161640 1 --2- 172.21.15.70:0/2144146277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f868006c510 0x7f868006c9a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8678009d90 tx=0x7f867802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9227ce569e8d2d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:07.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f868495e640 1 -- 172.21.15.70:0/2144146277 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867802cbd0 con 0x7f868006c510 2023-08-25T09:42:07.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f868495e640 1 -- 172.21.15.70:0/2144146277 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867802cd30 con 0x7f868006c510 2023-08-25T09:42:07.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f868495e640 1 -- 172.21.15.70:0/2144146277 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867802cea0 con 0x7f868006c510 2023-08-25T09:42:07.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 -- 172.21.15.70:0/2144146277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f868006c510 msgr2=0x7f868006c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 --2- 172.21.15.70:0/2144146277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f868006c510 0x7f868006c9a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8678009d90 tx=0x7f867802be20 comp rx=0 tx=0).stop 2023-08-25T09:42:07.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 -- 172.21.15.70:0/2144146277 shutdown_connections 2023-08-25T09:42:07.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 --2- 172.21.15.70:0/2144146277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f868006c510 0x7f868006c9a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 --2- 172.21.15.70:0/2144146277 >> v2:172.21.15.155:3300/0 conn(0x7f8680069d30 0x7f86800ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 -- 172.21.15.70:0/2144146277 >> 172.21.15.70:0/2144146277 conn(0x7f86800cb0d0 msgr2=0x7f86800cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:07.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 -- 172.21.15.70:0/2144146277 shutdown_connections 2023-08-25T09:42:07.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.290+0000 7f8687bc4640 1 -- 172.21.15.70:0/2144146277 wait complete. 2023-08-25T09:42:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.292+0000 7f8687bc4640 1 Processor -- start 2023-08-25T09:42:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.292+0000 7f8687bc4640 1 -- start start 2023-08-25T09:42:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8687bc4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 0x7f8680157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8687bc4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f868006a110 0x7f868015f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8687bc4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868006c510 0x7f868015f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8687bc4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c7440 con 0x7f8680069d30 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8687bc4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86800c6ac0 con 0x7f868006c510 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8687bc4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86801fac00 con 0x7f868006a110 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8685960640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 0x7f8680157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8685960640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 0x7f8680157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45934/0 (socket says 172.21.15.70:45934) 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.293+0000 7f8685960640 1 -- 172.21.15.70:0/427456619 learned_addr learned my addr 172.21.15.70:0/427456619 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8685960640 1 -- 172.21.15.70:0/427456619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f868006a110 msgr2=0x7f868015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8685960640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f868006a110 0x7f868015f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8685960640 1 -- 172.21.15.70:0/427456619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868006c510 msgr2=0x7f868015f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8685960640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868006c510 0x7f868015f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8685960640 1 -- 172.21.15.70:0/427456619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8678009a70 con 0x7f8680069d30 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8685960640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 0x7f8680157900 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c7f0 tx=0x7f867c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c004220 con 0x7f8680069d30 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c004380 con 0x7f8680069d30 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c007840 con 0x7f8680069d30 2023-08-25T09:42:07.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.294+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86800c6d80 con 0x7f8680069d30 2023-08-25T09:42:07.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.295+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8680157e40 con 0x7f8680069d30 2023-08-25T09:42:07.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.296+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680066fb0 con 0x7f8680069d30 2023-08-25T09:42:07.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.297+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f867c007bb0 con 0x7f8680069d30 2023-08-25T09:42:07.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.298+0000 7f86767fc640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f866406a050 0x7f866406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.298+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f867c088580 con 0x7f8680069d30 2023-08-25T09:42:07.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.298+0000 7f868515f640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f866406a050 0x7f866406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.298+0000 7f868515f640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f866406a050 0x7f866406c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8670007fd0 tx=0x7f8670007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.302+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f867c053c40 con 0x7f8680069d30 2023-08-25T09:42:07.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.479+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86800b45d0 con 0x7f8680069d30 2023-08-25T09:42:07.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.480+0000 7f86767fc640 1 -- 172.21.15.70:0/427456619 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f867c007e60 con 0x7f8680069d30 2023-08-25T09:42:07.482 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:42:07.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f866406a050 msgr2=0x7f866406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f866406a050 0x7f866406c510 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8670007fd0 tx=0x7f8670007960 comp rx=0 tx=0).stop 2023-08-25T09:42:07.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 msgr2=0x7f8680157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 0x7f8680157900 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f867c00c7f0 tx=0x7f867c00ccb0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 shutdown_connections 2023-08-25T09:42:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f866406a050 0x7f866406c510 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f868006c510 0x7f868015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f868006a110 0x7f868015f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 --2- 172.21.15.70:0/427456619 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8680069d30 0x7f8680157900 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.482+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 >> 172.21.15.70:0/427456619 conn(0x7f86800cb0d0 msgr2=0x7f8680070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.483+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 shutdown_connections 2023-08-25T09:42:07.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.483+0000 7f8687bc4640 1 -- 172.21.15.70:0/427456619 wait complete. 2023-08-25T09:42:07.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:07.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.707+0000 7fa87a66a640 1 Processor -- start 2023-08-25T09:42:07.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.707+0000 7fa87a66a640 1 -- start start 2023-08-25T09:42:07.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa87a66a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa874159100 0x7fa87406be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa87a66a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8740709a0 0x7fa8740b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa87a66a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa87406c3b0 0x7fa8740b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa87a66a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87406ba50 con 0x7fa874159100 2023-08-25T09:42:07.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa87a66a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87406ae10 con 0x7fa87406c3b0 2023-08-25T09:42:07.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa87a66a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87406b0b0 con 0x7fa8740709a0 2023-08-25T09:42:07.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa873fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa874159100 0x7fa87406be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa873fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa874159100 0x7fa87406be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45940/0 (socket says 172.21.15.70:45940) 2023-08-25T09:42:07.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.709+0000 7fa873fff640 1 -- 172.21.15.70:0/44242550 learned_addr learned my addr 172.21.15.70:0/44242550 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:07.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.710+0000 7fa873fff640 1 -- 172.21.15.70:0/44242550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8740709a0 msgr2=0x7fa8740b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:07.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.710+0000 7fa873fff640 1 --2- 172.21.15.70:0/44242550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8740709a0 0x7fa8740b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.710+0000 7fa873fff640 1 -- 172.21.15.70:0/44242550 >> v2:172.21.15.155:3300/0 conn(0x7fa87406c3b0 msgr2=0x7fa8740b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:07.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.710+0000 7fa873fff640 1 --2- 172.21.15.70:0/44242550 >> v2:172.21.15.155:3300/0 conn(0x7fa87406c3b0 0x7fa8740b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.710+0000 7fa873fff640 1 -- 172.21.15.70:0/44242550 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87406b390 con 0x7fa874159100 2023-08-25T09:42:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.711+0000 7fa873fff640 1 --2- 172.21.15.70:0/44242550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa874159100 0x7fa87406be70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa8740cefb0 tx=0x7fa86402ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3f4f05edc2efc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:07.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.711+0000 7fa8737fe640 1 -- 172.21.15.70:0/44242550 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86402da60 con 0x7fa874159100 2023-08-25T09:42:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.711+0000 7fa8737fe640 1 -- 172.21.15.70:0/44242550 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86402dbc0 con 0x7fa874159100 2023-08-25T09:42:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.711+0000 7fa8737fe640 1 -- 172.21.15.70:0/44242550 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864035440 con 0x7fa874159100 2023-08-25T09:42:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 -- 172.21.15.70:0/44242550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa874159100 msgr2=0x7fa87406be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 --2- 172.21.15.70:0/44242550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa874159100 0x7fa87406be70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa8740cefb0 tx=0x7fa86402ccb0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 -- 172.21.15.70:0/44242550 shutdown_connections 2023-08-25T09:42:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 --2- 172.21.15.70:0/44242550 >> v2:172.21.15.155:3300/0 conn(0x7fa87406c3b0 0x7fa8740b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 --2- 172.21.15.70:0/44242550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8740709a0 0x7fa8740b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 --2- 172.21.15.70:0/44242550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa874159100 0x7fa87406be70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 -- 172.21.15.70:0/44242550 >> 172.21.15.70:0/44242550 conn(0x7fa874069850 msgr2=0x7fa874069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 -- 172.21.15.70:0/44242550 shutdown_connections 2023-08-25T09:42:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.712+0000 7fa87a66a640 1 -- 172.21.15.70:0/44242550 wait complete. 2023-08-25T09:42:07.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.714+0000 7fa87a66a640 1 Processor -- start 2023-08-25T09:42:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.714+0000 7fa87a66a640 1 -- start start 2023-08-25T09:42:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa87a66a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 0x7fa874152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa87a66a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8740709a0 0x7fa874157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa87a66a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa874159100 0x7fa8741594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa87a66a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874157090 con 0x7fa87406c3b0 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa87a66a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874155d10 con 0x7fa8740709a0 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa87a66a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa874156760 con 0x7fa874159100 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 0x7fa874152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 0x7fa874152f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45952/0 (socket says 172.21.15.70:45952) 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 -- 172.21.15.70:0/1056292544 learned_addr learned my addr 172.21.15.70:0/1056292544 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 -- 172.21.15.70:0/1056292544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa874159100 msgr2=0x7fa8741594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa874159100 0x7fa8741594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 -- 172.21.15.70:0/1056292544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8740709a0 msgr2=0x7fa874157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8740709a0 0x7fa874157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.715+0000 7fa873fff640 1 -- 172.21.15.70:0/1056292544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86400a670 con 0x7fa87406c3b0 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.716+0000 7fa873fff640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 0x7fa874152f70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa86402ccf0 tx=0x7fa864004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.716+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864040860 con 0x7fa87406c3b0 2023-08-25T09:42:07.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.716+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8640409c0 con 0x7fa87406c3b0 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.716+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa864040d70 con 0x7fa87406c3b0 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.716+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa874156a40 con 0x7fa87406c3b0 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.716+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8741534b0 con 0x7fa87406c3b0 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.717+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8740adb20 con 0x7fa87406c3b0 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.718+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa86403c070 con 0x7fa87406c3b0 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.719+0000 7fa8717fa640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85406a050 0x7fa85406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:07.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.719+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa8640ab130 con 0x7fa87406c3b0 2023-08-25T09:42:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.719+0000 7fa86b7fe640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85406a050 0x7fa85406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.720+0000 7fa86b7fe640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85406a050 0x7fa85406c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa874156030 tx=0x7fa85c006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:07.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.723+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa864076760 con 0x7fa87406c3b0 2023-08-25T09:42:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.952+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa87404e420 con 0x7fa87406c3b0 2023-08-25T09:42:07.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.953+0000 7fa8717fa640 1 -- 172.21.15.70:0/1056292544 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7fa864079d80 con 0x7fa87406c3b0 2023-08-25T09:42:07.956 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:07.956 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:42:07.957 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:42:07.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85406a050 msgr2=0x7fa85406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85406a050 0x7fa85406c510 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa874156030 tx=0x7fa85c006bf0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 msgr2=0x7fa874152f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:07.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 0x7fa874152f70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa86402ccf0 tx=0x7fa864004730 comp rx=0 tx=0).stop 2023-08-25T09:42:07.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 shutdown_connections 2023-08-25T09:42:07.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85406a050 0x7fa85406c510 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa874159100 0x7fa8741594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8740709a0 0x7fa874157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 --2- 172.21.15.70:0/1056292544 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87406c3b0 0x7fa874152f70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 >> 172.21.15.70:0/1056292544 conn(0x7fa874069850 msgr2=0x7fa8740ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 shutdown_connections 2023-08-25T09:42:07.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:07.956+0000 7fa87a66a640 1 -- 172.21.15.70:0/1056292544 wait complete. 2023-08-25T09:42:07.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:08.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.192+0000 7ff3c00bf640 1 Processor -- start 2023-08-25T09:42:08.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.193+0000 7ff3c00bf640 1 -- start start 2023-08-25T09:42:08.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.194+0000 7ff3c00bf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3b80b2850 0x7ff3b8164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.194+0000 7ff3c00bf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b815d3f0 con 0x7ff3b815aa10 2023-08-25T09:42:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.194+0000 7ff3c00bf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b8070a00 con 0x7ff3b80ce5b0 2023-08-25T09:42:08.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.194+0000 7ff3c00bf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b8159d10 con 0x7ff3b80b2850 2023-08-25T09:42:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.195+0000 7ff3bd65a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff3b815aa10 0x7ff3b80b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36264/0 (socket says 172.21.15.70:36264) 2023-08-25T09:42:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.195+0000 7ff3bd65a640 1 -- 172.21.15.70:0/1480543126 learned_addr learned my addr 172.21.15.70:0/1480543126 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.195+0000 7ff3be65c640 1 --2- 172.21.15.70:0/1480543126 >> v2:172.21.15.155:3300/0 conn(0x7ff3b80b2850 0x7ff3b8164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.195+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360251407 0 0) 0x7ff3b815d3f0 con 0x7ff3b815aa10 2023-08-25T09:42:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.195+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a4005730 con 0x7ff3b815aa10 2023-08-25T09:42:08.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2975699939 0 0) 0x7ff3a4005730 con 0x7ff3b815aa10 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 >> v2:172.21.15.155:3300/0 conn(0x7ff3b80b2850 msgr2=0x7ff3b8164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 --2- 172.21.15.70:0/1480543126 >> v2:172.21.15.155:3300/0 conn(0x7ff3b80b2850 0x7ff3b8164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 >> v1:172.21.15.155:6789/0 conn(0x7ff3b80ce5b0 legacy=0x7ff3b80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80b2d20 con 0x7ff3b815aa10 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff3a8001670 con 0x7ff3b815aa10 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 >> v1:172.21.15.70:6789/0 conn(0x7ff3b815aa10 legacy=0x7ff3b80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a4005b80 0x7ff3a4005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3a40069d0 0x7ff3a4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a40073e0 0x7ff3a400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8159d10 con 0x7ff3a4005b80 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a400e310 con 0x7ff3a40069d0 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a400e4d0 con 0x7ff3a40073e0 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a4005b80 0x7ff3a4005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bd65a640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3a40069d0 0x7ff3a4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bde5b640 1 -- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a40073e0 msgr2=0x7ff3a400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a40073e0 0x7ff3a400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bde5b640 1 -- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3a40069d0 msgr2=0x7ff3a4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3a40069d0 0x7ff3a4006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3bde5b640 1 -- 172.21.15.70:0/1480543126 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a4005730 con 0x7ff3a4005b80 2023-08-25T09:42:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.196+0000 7ff3be65c640 1 -- 172.21.15.70:0/1480543126 reap_dead start 2023-08-25T09:42:08.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.197+0000 7ff3bd65a640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3a40069d0 0x7ff3a4006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a4005b80 0x7ff3a4005fc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4001750 tx=0x7ff3a4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a016ea2a041e64 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40052d0 con 0x7ff3a4005b80 2023-08-25T09:42:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b402e050 con 0x7ff3a4005b80 2023-08-25T09:42:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3bce59640 1 -- 172.21.15.70:0/1480543126 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4003830 con 0x7ff3a4005b80 2023-08-25T09:42:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3c00bf640 1 -- 172.21.15.70:0/1480543126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a4005b80 msgr2=0x7ff3a4005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a4005b80 0x7ff3a4005fc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4001750 tx=0x7ff3a4007a00 comp rx=0 tx=0).stop 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3c00bf640 1 -- 172.21.15.70:0/1480543126 shutdown_connections 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/1480543126 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3a4005b80 0x7ff3a4005fc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3c00bf640 1 -- 172.21.15.70:0/1480543126 >> 172.21.15.70:0/1480543126 conn(0x7ff3b8069850 msgr2=0x7ff3b8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.198+0000 7ff3c00bf640 1 -- 172.21.15.70:0/1480543126 shutdown_connections 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.199+0000 7ff3c00bf640 1 -- 172.21.15.70:0/1480543126 wait complete. 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.200+0000 7ff3c00bf640 1 Processor -- start 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.200+0000 7ff3c00bf640 1 -- start start 2023-08-25T09:42:08.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.201+0000 7ff3c00bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a4005b80 0x7ff3b814a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.201+0000 7ff3c00bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b8158960 0x7ff3b80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.201+0000 7ff3c00bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 0x7ff3b80ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.201+0000 7ff3c00bf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b81584d0 con 0x7ff3b814ad10 2023-08-25T09:42:08.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.201+0000 7ff3c00bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80aebf0 con 0x7ff3b8158960 2023-08-25T09:42:08.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.201+0000 7ff3c00bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8157e80 con 0x7ff3a4005b80 2023-08-25T09:42:08.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 0x7ff3b80ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 0x7ff3b80ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45972/0 (socket says 172.21.15.70:45972) 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 -- 172.21.15.70:0/392180091 learned_addr learned my addr 172.21.15.70:0/392180091 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a4005b80 0x7ff3b814a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3bd65a640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b8158960 0x7ff3b80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 -- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a4005b80 msgr2=0x7ff3b814a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a4005b80 0x7ff3b814a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 -- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b8158960 msgr2=0x7ff3b80ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b8158960 0x7ff3b80ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 -- 172.21.15.70:0/392180091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b8158160 con 0x7ff3b814ad10 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.202+0000 7ff3be65c640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 0x7ff3b80ac870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80cae20 tx=0x7ff3b000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.203+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0010040 con 0x7ff3b814ad10 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.203+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b000bc80 con 0x7ff3b814ad10 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.203+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0014400 con 0x7ff3b814ad10 2023-08-25T09:42:08.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.203+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b814aa10 con 0x7ff3b814ad10 2023-08-25T09:42:08.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.203+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b80af0b0 con 0x7ff3b814ad10 2023-08-25T09:42:08.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.204+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b8158640 con 0x7ff3b814ad10 2023-08-25T09:42:08.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.205+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3b0014560 con 0x7ff3b814ad10 2023-08-25T09:42:08.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.206+0000 7ff3ae7fc640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff39c06a050 0x7ff39c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.206+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7ff3b0088ca0 con 0x7ff3b814ad10 2023-08-25T09:42:08.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.206+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff39c06a050 0x7ff39c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.207+0000 7ff3bde5b640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff39c06a050 0x7ff39c06c510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4033000 tx=0x7ff3b4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:08.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.210+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3b0054400 con 0x7ff3b814ad10 2023-08-25T09:42:08.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.406+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3b804e420 con 0x7ff3b814ad10 2023-08-25T09:42:08.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.409+0000 7ff3ae7fc640 1 -- 172.21.15.70:0/392180091 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18353 (secure 0 0 0) 0x7ff3b0057a20 con 0x7ff3b814ad10 2023-08-25T09:42:08.411 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:08.411 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":61,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:53.340729+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.9","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:08.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff39c06a050 msgr2=0x7ff39c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff39c06a050 0x7ff39c06c510 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4033000 tx=0x7ff3b4009030 comp rx=0 tx=0).stop 2023-08-25T09:42:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 msgr2=0x7ff3b80ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 0x7ff3b80ac870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff3b80cae20 tx=0x7ff3b000b480 comp rx=0 tx=0).stop 2023-08-25T09:42:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 shutdown_connections 2023-08-25T09:42:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff39c06a050 0x7ff39c06c510 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3b814ad10 0x7ff3b80ac870 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3b8158960 0x7ff3b80ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 --2- 172.21.15.70:0/392180091 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3a4005b80 0x7ff3b814a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.411+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 >> 172.21.15.70:0/392180091 conn(0x7ff3b8069850 msgr2=0x7ff3b81591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.412+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 shutdown_connections 2023-08-25T09:42:08.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.412+0000 7ff3c00bf640 1 -- 172.21.15.70:0/392180091 wait complete. 2023-08-25T09:42:08.433 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:42:08.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:08.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.647+0000 7f12ae929640 1 Processor -- start 2023-08-25T09:42:08.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.647+0000 7f12ae929640 1 -- start start 2023-08-25T09:42:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae929640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 0x7f12a80b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae929640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12a80c96a0 0x7f12a80cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae929640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a80cd330 con 0x7f12a80b4150 2023-08-25T09:42:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae929640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a81615b0 con 0x7f12a80c96a0 2023-08-25T09:42:08.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae929640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80b0e80 con 0x7f12a80c92c0 2023-08-25T09:42:08.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae128640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f12a80b4150 0x7f12a80cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36278/0 (socket says 172.21.15.70:36278) 2023-08-25T09:42:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.649+0000 7f12ae128640 1 -- 172.21.15.70:0/4217679502 learned_addr learned my addr 172.21.15.70:0/4217679502 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ad126640 1 --2- 172.21.15.70:0/4217679502 >> v2:172.21.15.155:3300/0 conn(0x7f12a80c96a0 0x7f12a80cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ad927640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 0x7f12a80b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792071394 0 0) 0x7f12a80cd330 con 0x7f12a80b4150 2023-08-25T09:42:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1294003520 con 0x7f12a80b4150 2023-08-25T09:42:08.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2923919708 0 0) 0x7f1294003520 con 0x7f12a80b4150 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 msgr2=0x7f12a80b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 0x7f12a80b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 >> v2:172.21.15.155:3300/0 conn(0x7f12a80c96a0 msgr2=0x7f12a80cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 --2- 172.21.15.70:0/4217679502 >> v2:172.21.15.155:3300/0 conn(0x7f12a80c96a0 0x7f12a80cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80cc720 con 0x7f12a80b4150 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f12a00019f0 con 0x7f12a80b4150 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 >> v1:172.21.15.70:6789/0 conn(0x7f12a80b4150 legacy=0x7f12a80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1294003970 0x7f1294003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12940047c0 0x7f1294004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 0x7f1294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a81615b0 con 0x7f12940047c0 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80b0e80 con 0x7f1294003970 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.650+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129400b980 con 0x7f12a80c92c0 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad126640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12940047c0 0x7f1294004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad927640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1294003970 0x7f1294003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad126640 1 -- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 msgr2=0x7f1294005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ae128640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 0x7f1294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad126640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c92c0 0x7f1294005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad126640 1 -- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1294003970 msgr2=0x7f1294003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad126640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1294003970 0x7f1294003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad126640 1 -- 172.21.15.70:0/4217679502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294003520 con 0x7f12940047c0 2023-08-25T09:42:08.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.651+0000 7f12ad927640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1294003970 0x7f1294003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.652+0000 7f12ad126640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12940047c0 0x7f1294004c40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1298009ac0 tx=0x7f12940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a4947b8f9071cd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.652+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129802c710 con 0x7f12940047c0 2023-08-25T09:42:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.652+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129802c870 con 0x7f12940047c0 2023-08-25T09:42:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.652+0000 7f12ac925640 1 -- 172.21.15.70:0/4217679502 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129802cc00 con 0x7f12940047c0 2023-08-25T09:42:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae929640 1 -- 172.21.15.70:0/4217679502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12940047c0 msgr2=0x7f1294004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae929640 1 --2- 172.21.15.70:0/4217679502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12940047c0 0x7f1294004c40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f1298009ac0 tx=0x7f12940057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae128640 1 -- 172.21.15.70:0/4217679502 reap_dead start 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae929640 1 -- 172.21.15.70:0/4217679502 shutdown_connections 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae929640 1 -- 172.21.15.70:0/4217679502 >> 172.21.15.70:0/4217679502 conn(0x7f12a80affd0 msgr2=0x7f12a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae929640 1 -- 172.21.15.70:0/4217679502 shutdown_connections 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.653+0000 7f12ae929640 1 -- 172.21.15.70:0/4217679502 wait complete. 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.655+0000 7f12ae929640 1 Processor -- start 2023-08-25T09:42:08.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.655+0000 7f12ae929640 1 -- start start 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ae929640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12940047c0 0x7f12a815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ae929640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 0x7f12a80b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ae929640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c5570 0x7f12a80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ae929640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80b30c0 con 0x7f12a80b2700 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ae929640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80c7380 con 0x7f12940047c0 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ae929640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80c62e0 con 0x7f12a80c5570 2023-08-25T09:42:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ad126640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 0x7f12a80b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ad126640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 0x7f12a80b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45994/0 (socket says 172.21.15.70:45994) 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.656+0000 7f12ad126640 1 -- 172.21.15.70:0/2151082844 learned_addr learned my addr 172.21.15.70:0/2151082844 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad927640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12940047c0 0x7f12a815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad126640 1 -- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c5570 msgr2=0x7f12a80c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad126640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c5570 0x7f12a80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad126640 1 -- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12940047c0 msgr2=0x7f12a815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad126640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12940047c0 0x7f12a815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad126640 1 -- 172.21.15.70:0/2151082844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80c65c0 con 0x7f12a80b2700 2023-08-25T09:42:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f12ad126640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 0x7f12a80b2b80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f129802c2b0 tx=0x7f1298004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.657+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298003810 con 0x7f12a80b2700 2023-08-25T09:42:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.658+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129802de90 con 0x7f12a80b2700 2023-08-25T09:42:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.658+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298003bb0 con 0x7f12a80b2700 2023-08-25T09:42:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.658+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a80c68a0 con 0x7f12a80b2700 2023-08-25T09:42:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.658+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a80c6e60 con 0x7f12a80b2700 2023-08-25T09:42:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.659+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a80c6ff0 con 0x7f12a80b2700 2023-08-25T09:42:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.660+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1298003d10 con 0x7f12a80b2700 2023-08-25T09:42:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.660+0000 7f129e7fc640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f128c06a080 0x7f128c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.661+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f12980aa630 con 0x7f12a80b2700 2023-08-25T09:42:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.661+0000 7f12ad927640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f128c06a080 0x7f128c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.661+0000 7f12ad927640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f128c06a080 0x7f128c06c540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4006ce0 tx=0x7f12a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:08.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.665+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1298075e20 con 0x7f12a80b2700 2023-08-25T09:42:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.892+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12a8066fb0 con 0x7f12a80b2700 2023-08-25T09:42:08.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.893+0000 7f129e7fc640 1 -- 172.21.15.70:0/2151082844 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f1298079440 con 0x7f12a80b2700 2023-08-25T09:42:08.895 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:08.895 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:42:08.895 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:42:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.895+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f128c06a080 msgr2=0x7f128c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.895+0000 7f12ae929640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f128c06a080 0x7f128c06c540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f12a4006ce0 tx=0x7f12a4008040 comp rx=0 tx=0).stop 2023-08-25T09:42:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.895+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 msgr2=0x7f12a80b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.895+0000 7f12ae929640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 0x7f12a80b2b80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f129802c2b0 tx=0x7f1298004730 comp rx=0 tx=0).stop 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 shutdown_connections 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f128c06a080 0x7f128c06c540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f12a80c5570 0x7f12a80c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12a80b2700 0x7f12a80b2b80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 --2- 172.21.15.70:0/2151082844 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12940047c0 0x7f12a815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 >> 172.21.15.70:0/2151082844 conn(0x7f12a80affd0 msgr2=0x7f12a8160c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:08.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 shutdown_connections 2023-08-25T09:42:08.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:08.896+0000 7f12ae929640 1 -- 172.21.15.70:0/2151082844 wait complete. 2023-08-25T09:42:08.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.124+0000 7f895e324640 1 Processor -- start 2023-08-25T09:42:09.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.124+0000 7f895e324640 1 -- start start 2023-08-25T09:42:09.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f895e324640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f895815ec10 0x7f895815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f895e324640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89580b1b60 con 0x7f895815e830 2023-08-25T09:42:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f895e324640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895806b7c0 con 0x7f895815c6d0 2023-08-25T09:42:09.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f895e324640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f895806bd80 con 0x7f895815ec10 2023-08-25T09:42:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f8957fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f895815e830 0x7f895806ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49228/0 (socket says 172.21.15.70:49228) 2023-08-25T09:42:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f8957fff640 1 -- 172.21.15.70:0/54077905 learned_addr learned my addr 172.21.15.70:0/54077905 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.126+0000 7f89577fe640 1 --2- 172.21.15.70:0/54077905 >> v2:172.21.15.155:3300/0 conn(0x7f895815ec10 0x7f895815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533648026 0 0) 0x7f89580b1b60 con 0x7f895815e830 2023-08-25T09:42:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8940003520 con 0x7f895815e830 2023-08-25T09:42:09.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2718967858 0 0) 0x7f8940003520 con 0x7f895815e830 2023-08-25T09:42:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 >> v2:172.21.15.155:3300/0 conn(0x7f895815ec10 msgr2=0x7f895815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 --2- 172.21.15.70:0/54077905 >> v2:172.21.15.155:3300/0 conn(0x7f895815ec10 0x7f895815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 >> v1:172.21.15.155:6789/0 conn(0x7f895815c6d0 legacy=0x7f89580b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895806baa0 con 0x7f895815e830 2023-08-25T09:42:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8948001670 con 0x7f895815e830 2023-08-25T09:42:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f89577fe640 1 --2- 172.21.15.70:0/54077905 >> v2:172.21.15.155:3300/0 conn(0x7f895815ec10 0x7f895815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:42:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 >> v1:172.21.15.70:6789/0 conn(0x7f895815e830 legacy=0x7f895806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8940003970 0x7f8940003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.127+0000 7f8956ffd640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89400047c0 0x7f8940004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f8956ffd640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f894000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895806bd80 con 0x7f8940005180 2023-08-25T09:42:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894000c120 con 0x7f8940003970 2023-08-25T09:42:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894000c2e0 con 0x7f89400047c0 2023-08-25T09:42:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f895c8c1640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f894000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f8957fff640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8940003970 0x7f8940003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f895c8c1640 1 -- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89400047c0 msgr2=0x7f8940004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f895c8c1640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89400047c0 0x7f8940004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f895c8c1640 1 -- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8940003970 msgr2=0x7f8940003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f895c8c1640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8940003970 0x7f8940003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.128+0000 7f895c8c1640 1 -- 172.21.15.70:0/54077905 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8940003520 con 0x7f8940005180 2023-08-25T09:42:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.129+0000 7f895c8c1640 1 -- 172.21.15.70:0/54077905 reap_dead start 2023-08-25T09:42:09.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.129+0000 7f895c8c1640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f894000b980 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8950001e50 tx=0x7f89400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada2e18b8a82541f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950003b70 con 0x7f8940005180 2023-08-25T09:42:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950003d10 con 0x7f8940005180 2023-08-25T09:42:09.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f8956ffd640 1 -- 172.21.15.70:0/54077905 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950005310 con 0x7f8940005180 2023-08-25T09:42:09.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 -- 172.21.15.70:0/54077905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 msgr2=0x7f894000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f894000b980 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8950001e50 tx=0x7f89400057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 -- 172.21.15.70:0/54077905 shutdown_connections 2023-08-25T09:42:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 --2- 172.21.15.70:0/54077905 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f894000b980 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 -- 172.21.15.70:0/54077905 >> 172.21.15.70:0/54077905 conn(0x7f8958069850 msgr2=0x7f8958069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 -- 172.21.15.70:0/54077905 shutdown_connections 2023-08-25T09:42:09.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.130+0000 7f895e324640 1 -- 172.21.15.70:0/54077905 wait complete. 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.132+0000 7f895e324640 1 Processor -- start 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.133+0000 7f895e324640 1 -- start start 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f895e324640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f89580aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f895e324640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89580c9c90 0x7f89581f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f895e324640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8958155610 0x7f89580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f895e324640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89581f33d0 con 0x7f8940005180 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f895e324640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89581f2e10 con 0x7f89580c9c90 2023-08-25T09:42:09.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f895e324640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89581f30f0 con 0x7f8958155610 2023-08-25T09:42:09.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f89580aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f89580aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37392/0 (socket says 172.21.15.70:37392) 2023-08-25T09:42:09.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 -- 172.21.15.70:0/2261581154 learned_addr learned my addr 172.21.15.70:0/2261581154 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:09.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f89577fe640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89580c9c90 0x7f89581f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 -- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8958155610 msgr2=0x7f89580b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8958155610 0x7f89580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 -- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89580c9c90 msgr2=0x7f89581f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89580c9c90 0x7f89581f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.134+0000 7f8957fff640 1 -- 172.21.15.70:0/2261581154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8958154e10 con 0x7f8940005180 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.135+0000 7f8957fff640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f89580aedb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f894800e7f0 tx=0x7f894800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:09.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.135+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894800eea0 con 0x7f8940005180 2023-08-25T09:42:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.135+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894800c870 con 0x7f8940005180 2023-08-25T09:42:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.135+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894800cc00 con 0x7f8940005180 2023-08-25T09:42:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.135+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89581550f0 con 0x7f8940005180 2023-08-25T09:42:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.135+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89581f2b90 con 0x7f8940005180 2023-08-25T09:42:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.136+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8958155330 con 0x7f8940005180 2023-08-25T09:42:09.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.137+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f894801d460 con 0x7f8940005180 2023-08-25T09:42:09.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.138+0000 7f8954ff9640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f894406a050 0x7f894406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.138+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8948088480 con 0x7f8940005180 2023-08-25T09:42:09.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.139+0000 7f89577fe640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f894406a050 0x7f894406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.139+0000 7f89577fe640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f894406a050 0x7f894406c510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f89580c94c0 tx=0x7f894c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:09.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.143+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8958155330 con 0x7f8940005180 2023-08-25T09:42:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.375+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8958155330 con 0x7f8940005180 2023-08-25T09:42:09.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.376+0000 7f8954ff9640 1 -- 172.21.15.70:0/2261581154 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4878 (secure 0 0 0) 0x7f8948057670 con 0x7f8940005180 2023-08-25T09:42:09.378 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:09.379 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:41:12.986069+0000","modified":"2023-08-25T09:41:16.578011+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5835},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5835":{"gid":5835,"name":"a","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/3881965592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":3881965592},{"type":"v1","addr":"172.21.15.70:6837","nonce":3881965592}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-08-25T09:42:09.379 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 24 2023-08-25T09:42:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f894406a050 msgr2=0x7f894406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f894406a050 0x7f894406c510 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f89580c94c0 tx=0x7f894c008040 comp rx=0 tx=0).stop 2023-08-25T09:42:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 msgr2=0x7f89580aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f89580aedb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f894800e7f0 tx=0x7f894800ecb0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 shutdown_connections 2023-08-25T09:42:09.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f894406a050 0x7f894406c510 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8958155610 0x7f89580b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89580c9c90 0x7f89581f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 --2- 172.21.15.70:0/2261581154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8940005180 0x7f89580aedb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 >> 172.21.15.70:0/2261581154 conn(0x7f8958069850 msgr2=0x7f8958070100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 shutdown_connections 2023-08-25T09:42:09.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.379+0000 7f895e324640 1 -- 172.21.15.70:0/2261581154 wait complete. 2023-08-25T09:42:09.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:09.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.622+0000 7f2e45fd6640 1 Processor -- start 2023-08-25T09:42:09.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.622+0000 7f2e45fd6640 1 -- start start 2023-08-25T09:42:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.623+0000 7f2e45fd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e400b35c0 0x7f2e400b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.623+0000 7f2e45fd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015e9e0 0x7f2e4015ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.623+0000 7f2e45fd6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4015e550 con 0x7f2e4015a850 2023-08-25T09:42:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.623+0000 7f2e45fd6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e400b43f0 con 0x7f2e400b35c0 2023-08-25T09:42:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.623+0000 7f2e45fd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4015ac30 con 0x7f2e4015e9e0 2023-08-25T09:42:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e44fd4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2e4015a850 0x7f2e400c8a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49236/0 (socket says 172.21.15.70:49236) 2023-08-25T09:42:09.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e44fd4640 1 -- 172.21.15.70:0/876522816 learned_addr learned my addr 172.21.15.70:0/876522816 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3ffff640 1 --2- 172.21.15.70:0/876522816 >> v2:172.21.15.70:3300/0 conn(0x7f2e400b35c0 0x7f2e400b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e457d5640 1 --2- 172.21.15.70:0/876522816 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015e9e0 0x7f2e4015ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3f7fe640 1 -- 172.21.15.70:0/876522816 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249148645 0 0) 0x7f2e4015e550 con 0x7f2e4015a850 2023-08-25T09:42:09.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3f7fe640 1 -- 172.21.15.70:0/876522816 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e30003520 con 0x7f2e4015a850 2023-08-25T09:42:09.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3ffff640 1 -- 172.21.15.70:0/876522816 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015e9e0 msgr2=0x7f2e4015ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:09.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3ffff640 1 --2- 172.21.15.70:0/876522816 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015e9e0 0x7f2e4015ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3ffff640 1 -- 172.21.15.70:0/876522816 >> v1:172.21.15.70:6789/0 conn(0x7f2e4015a850 legacy=0x7f2e400c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.624+0000 7f2e3ffff640 1 -- 172.21.15.70:0/876522816 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e400b3b50 con 0x7f2e400b35c0 2023-08-25T09:42:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.626+0000 7f2e3ffff640 1 --2- 172.21.15.70:0/876522816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e400b08b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097a0 tx=0x7f2e2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5668820474e7019 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.626+0000 7f2e3f7fe640 1 -- 172.21.15.70:0/876522816 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c031830 con 0x7f2e400b35c0 2023-08-25T09:42:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.626+0000 7f2e3f7fe640 1 -- 172.21.15.70:0/876522816 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c031990 con 0x7f2e400b35c0 2023-08-25T09:42:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.626+0000 7f2e3f7fe640 1 -- 172.21.15.70:0/876522816 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c031d20 con 0x7f2e400b35c0 2023-08-25T09:42:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 -- 172.21.15.70:0/876522816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 msgr2=0x7f2e400b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/876522816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e400b08b0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c0097a0 tx=0x7f2e2c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 -- 172.21.15.70:0/876522816 shutdown_connections 2023-08-25T09:42:09.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/876522816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e400b08b0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/876522816 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015e9e0 0x7f2e4015ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 -- 172.21.15.70:0/876522816 >> 172.21.15.70:0/876522816 conn(0x7f2e40069e80 msgr2=0x7f2e4006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 -- 172.21.15.70:0/876522816 shutdown_connections 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.627+0000 7f2e45fd6640 1 -- 172.21.15.70:0/876522816 wait complete. 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.629+0000 7f2e45fd6640 1 Processor -- start 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.629+0000 7f2e45fd6640 1 -- start start 2023-08-25T09:42:09.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e45fd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e401488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e45fd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015a850 0x7f2e401f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e45fd6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e4015e9e0 0x7f2e400b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e45fd6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401f62c0 con 0x7f2e400b35c0 2023-08-25T09:42:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e45fd6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40148df0 con 0x7f2e4015e9e0 2023-08-25T09:42:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e45fd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e401f5d00 con 0x7f2e4015a850 2023-08-25T09:42:09.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e401488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e401488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37406/0 (socket says 172.21.15.70:37406) 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 -- 172.21.15.70:0/36402461 learned_addr learned my addr 172.21.15.70:0/36402461 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 -- 172.21.15.70:0/36402461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015a850 msgr2=0x7f2e401f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015a850 0x7f2e401f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 -- 172.21.15.70:0/36402461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e4015e9e0 msgr2=0x7f2e400b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e4015e9e0 0x7f2e400b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.630+0000 7f2e44fd4640 1 -- 172.21.15.70:0/36402461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2c009400 con 0x7f2e400b35c0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.631+0000 7f2e44fd4640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e401488b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c500 tx=0x7f2e3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.631+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34005ce0 con 0x7f2e400b35c0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.631+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34005e40 con 0x7f2e400b35c0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.631+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34007330 con 0x7f2e400b35c0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.631+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e401f5fe0 con 0x7f2e400b35c0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.631+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4015c200 con 0x7f2e400b35c0 2023-08-25T09:42:09.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.632+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4015bd80 con 0x7f2e400b35c0 2023-08-25T09:42:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.633+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e34007490 con 0x7f2e400b35c0 2023-08-25T09:42:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.634+0000 7f2e3d7fa640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e2406a050 0x7f2e2406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.634+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f2e34087cb0 con 0x7f2e400b35c0 2023-08-25T09:42:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.634+0000 7f2e3ffff640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e2406a050 0x7f2e2406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.635+0000 7f2e3ffff640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e2406a050 0x7f2e2406c510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c036000 tx=0x7f2e2c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:09.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.638+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2e4015bd80 con 0x7f2e400b35c0 2023-08-25T09:42:09.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.825+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e4015bd80 con 0x7f2e400b35c0 2023-08-25T09:42:09.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.827+0000 7f2e3d7fa640 1 -- 172.21.15.70:0/36402461 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+18353 (secure 0 0 0) 0x7f2e4015bd80 con 0x7f2e400b35c0 2023-08-25T09:42:09.830 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:09.830 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":61,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:41:53.340729+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:41:11.244136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-08-25T09:41:11.990910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.9","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:09.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e2406a050 msgr2=0x7f2e2406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e2406a050 0x7f2e2406c510 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c036000 tx=0x7f2e2c030000 comp rx=0 tx=0).stop 2023-08-25T09:42:09.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 msgr2=0x7f2e401488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:09.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e401488b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400c500 tx=0x7f2e3400c9c0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 shutdown_connections 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e2406a050 0x7f2e2406c510 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e4015e9e0 0x7f2e400b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e4015a850 0x7f2e401f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 --2- 172.21.15.70:0/36402461 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e400b35c0 0x7f2e401488b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 >> 172.21.15.70:0/36402461 conn(0x7f2e40069e80 msgr2=0x7f2e400c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 shutdown_connections 2023-08-25T09:42:09.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:09.830+0000 7f2e45fd6640 1 -- 172.21.15.70:0/36402461 wait complete. 2023-08-25T09:42:09.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:42:10.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.060+0000 7f0d84b83640 1 Processor -- start 2023-08-25T09:42:10.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.061+0000 7f0d84b83640 1 -- start start 2023-08-25T09:42:10.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.062+0000 7f0d84b83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 0x7f0d800cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.062+0000 7f0d84b83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d800b3cf0 0x7f0d800b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.062+0000 7f0d84b83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800b12b0 con 0x7f0d8015acd0 2023-08-25T09:42:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.062+0000 7f0d84b83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8006c0e0 con 0x7f0d800b3cf0 2023-08-25T09:42:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.062+0000 7f0d84b83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8006ae20 con 0x7f0d800ca780 2023-08-25T09:42:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d8015acd0 0x7f0d8015b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49252/0 (socket says 172.21.15.70:49252) 2023-08-25T09:42:10.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7e59c640 1 -- 172.21.15.70:0/2884555292 learned_addr learned my addr 172.21.15.70:0/2884555292 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910597356 0 0) 0x7f0d800b12b0 con 0x7f0d8015acd0 2023-08-25T09:42:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d6c003520 con 0x7f0d8015acd0 2023-08-25T09:42:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7ed9d640 1 --2- 172.21.15.70:0/2884555292 >> v2:172.21.15.155:3300/0 conn(0x7f0d800b3cf0 0x7f0d800b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7dd9b640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 0x7f0d800cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 43111921 0 0) 0x7f0d6c003520 con 0x7f0d8015acd0 2023-08-25T09:42:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 msgr2=0x7f0d800cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 0x7f0d800cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 >> v2:172.21.15.155:3300/0 conn(0x7f0d800b3cf0 msgr2=0x7f0d800b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 --2- 172.21.15.70:0/2884555292 >> v2:172.21.15.155:3300/0 conn(0x7f0d800b3cf0 0x7f0d800b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8015b7c0 con 0x7f0d8015acd0 2023-08-25T09:42:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0d70001b70 con 0x7f0d8015acd0 2023-08-25T09:42:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 >> v1:172.21.15.70:6789/0 conn(0x7f0d8015acd0 legacy=0x7f0d8015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.063+0000 7f0d7d59a640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d6c003970 0x7f0d6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7d59a640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 0x7f0d6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7d59a640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d6c004d00 0x7f0d6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8006c0e0 con 0x7f0d6c003970 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8006ae20 con 0x7f0d6c004d00 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00bb10 con 0x7f0d800ca780 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7e59c640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d6c003970 0x7f0d6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7ed9d640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d6c004d00 0x7f0d6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7dd9b640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 0x7f0d6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7e59c640 1 -- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 msgr2=0x7f0d6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7e59c640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d800ca780 0x7f0d6c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7e59c640 1 -- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d6c004d00 msgr2=0x7f0d6c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7e59c640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d6c004d00 0x7f0d6c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.064+0000 7f0d7e59c640 1 -- 172.21.15.70:0/2884555292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c003520 con 0x7f0d6c003970 2023-08-25T09:42:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.065+0000 7f0d7e59c640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d6c003970 0x7f0d6c003d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c510 tx=0x7f0d6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16aec4b0fd2bc715 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70005b20 con 0x7f0d6c003970 2023-08-25T09:42:10.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70005c80 con 0x7f0d6c003970 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d7d59a640 1 -- 172.21.15.70:0/2884555292 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70007070 con 0x7f0d6c003970 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d84b83640 1 -- 172.21.15.70:0/2884555292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d6c003970 msgr2=0x7f0d6c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d84b83640 1 --2- 172.21.15.70:0/2884555292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d6c003970 0x7f0d6c003d90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000c510 tx=0x7f0d6c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d7ed9d640 1 -- 172.21.15.70:0/2884555292 reap_dead start 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d84b83640 1 -- 172.21.15.70:0/2884555292 shutdown_connections 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d84b83640 1 -- 172.21.15.70:0/2884555292 >> 172.21.15.70:0/2884555292 conn(0x7f0d800ce760 msgr2=0x7f0d800ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d84b83640 1 -- 172.21.15.70:0/2884555292 shutdown_connections 2023-08-25T09:42:10.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.066+0000 7f0d84b83640 1 -- 172.21.15.70:0/2884555292 wait complete. 2023-08-25T09:42:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.068+0000 7f0d84b83640 1 Processor -- start 2023-08-25T09:42:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.068+0000 7f0d84b83640 1 -- start start 2023-08-25T09:42:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d84b83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d6c003970 0x7f0d801ed540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d84b83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 0x7f0d8014f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d84b83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8014fd40 0x7f0d801513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d84b83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d80142740 con 0x7f0d8014ee60 2023-08-25T09:42:10.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d84b83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d801404d0 con 0x7f0d8014fd40 2023-08-25T09:42:10.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d84b83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8014f5d0 con 0x7f0d6c003970 2023-08-25T09:42:10.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.069+0000 7f0d7dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 0x7f0d8014f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 0x7f0d8014f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37430/0 (socket says 172.21.15.70:37430) 2023-08-25T09:42:10.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 -- 172.21.15.70:0/1099105531 learned_addr learned my addr 172.21.15.70:0/1099105531 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7ed9d640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8014fd40 0x7f0d801513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 -- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d6c003970 msgr2=0x7f0d801ed540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d6c003970 0x7f0d801ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 -- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8014fd40 msgr2=0x7f0d801513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8014fd40 0x7f0d801513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 -- 172.21.15.70:0/1099105531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8014f2f0 con 0x7f0d8014ee60 2023-08-25T09:42:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d7dd9b640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 0x7f0d8014f800 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e7e0 tx=0x7f0d6800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.070+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800c6d0 con 0x7f0d8014ee60 2023-08-25T09:42:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.071+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6800c830 con 0x7f0d8014ee60 2023-08-25T09:42:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.071+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8013fe80 con 0x7f0d8014ee60 2023-08-25T09:42:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.071+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6800cba0 con 0x7f0d8014ee60 2023-08-25T09:42:10.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.071+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d801eda80 con 0x7f0d8014ee60 2023-08-25T09:42:10.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.072+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d800c9190 con 0x7f0d8014ee60 2023-08-25T09:42:10.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.073+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0d6800cd00 con 0x7f0d8014ee60 2023-08-25T09:42:10.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.074+0000 7f0d76ffd640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d6006a080 0x7f0d6006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.074+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f0d680883d0 con 0x7f0d8014ee60 2023-08-25T09:42:10.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.074+0000 7f0d7e59c640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d6006a080 0x7f0d6006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.075+0000 7f0d7e59c640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d6006a080 0x7f0d6006c540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d70034000 tx=0x7f0d7002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:10.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.078+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0d68053a90 con 0x7f0d8014ee60 2023-08-25T09:42:10.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.303+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0d8004e420 con 0x7f0d8014ee60 2023-08-25T09:42:10.305 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:42:10.304+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:42:10.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.318+0000 7f0d76ffd640 1 -- 172.21.15.70:0/1099105531 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0d680570b0 con 0x7f0d8014ee60 2023-08-25T09:42:10.320 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:42:10.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.320+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d6006a080 msgr2=0x7f0d6006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.320+0000 7f0d84b83640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d6006a080 0x7f0d6006c540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d70034000 tx=0x7f0d7002e000 comp rx=0 tx=0).stop 2023-08-25T09:42:10.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.320+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 msgr2=0x7f0d8014f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.320+0000 7f0d84b83640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 0x7f0d8014f800 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800e7e0 tx=0x7f0d6800eca0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 shutdown_connections 2023-08-25T09:42:10.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0d6006a080 0x7f0d6006c540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d8014fd40 0x7f0d801513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0d8014ee60 0x7f0d8014f800 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 --2- 172.21.15.70:0/1099105531 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d6c003970 0x7f0d801ed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 >> 172.21.15.70:0/1099105531 conn(0x7f0d800ce760 msgr2=0x7f0d8015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 shutdown_connections 2023-08-25T09:42:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.321+0000 7f0d84b83640 1 -- 172.21.15.70:0/1099105531 wait complete. 2023-08-25T09:42:10.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:42:10.423 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:42:10.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.525+0000 7f477b95d640 1 Processor -- start 2023-08-25T09:42:10.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.525+0000 7f477b95d640 1 -- start start 2023-08-25T09:42:10.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.526+0000 7f477b95d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 0x7f477406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.526+0000 7f477b95d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f47740cec00 0x7f47740b1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.526+0000 7f477b95d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477406c8b0 con 0x7f47741607e0 2023-08-25T09:42:10.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.526+0000 7f477b95d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477406b250 con 0x7f47740cec00 2023-08-25T09:42:10.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.526+0000 7f477b95d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477406b510 con 0x7f477415cbb0 2023-08-25T09:42:10.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.527+0000 7f4778ef8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f47741607e0 0x7f47740ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49276/0 (socket says 172.21.15.70:49276) 2023-08-25T09:42:10.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.527+0000 7f4778ef8640 1 -- 172.21.15.70:0/3613870283 learned_addr learned my addr 172.21.15.70:0/3613870283 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:10.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.527+0000 7f47796f9640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 0x7f477406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.527+0000 7f4779efa640 1 --2- 172.21.15.70:0/3613870283 >> v2:172.21.15.155:3300/0 conn(0x7f47740cec00 0x7f47740b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.527+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893186742 0 0) 0x7f477406c8b0 con 0x7f47741607e0 2023-08-25T09:42:10.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.527+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4760003520 con 0x7f47741607e0 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123143356 0 0) 0x7f4760003520 con 0x7f47741607e0 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 msgr2=0x7f477406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 0x7f477406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 >> v2:172.21.15.155:3300/0 conn(0x7f47740cec00 msgr2=0x7f47740b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 --2- 172.21.15.70:0/3613870283 >> v2:172.21.15.155:3300/0 conn(0x7f47740cec00 0x7f47740b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477406b7f0 con 0x7f47741607e0 2023-08-25T09:42:10.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4764001670 con 0x7f47741607e0 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 >> v1:172.21.15.70:6789/0 conn(0x7f47741607e0 legacy=0x7f47740ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 0x7f4760003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4760003ef0 0x7f47600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760004d00 0x7f4760005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477406b250 con 0x7f4760003ef0 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477406b510 con 0x7f4760004d00 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476000b9b0 con 0x7f477415cbb0 2023-08-25T09:42:10.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f4778ef8640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4760003ef0 0x7f47600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f4778ef8640 1 -- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 msgr2=0x7f4760003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f4778ef8640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cbb0 0x7f4760003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f4778ef8640 1 -- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760004d00 msgr2=0x7f4760005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f4778ef8640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760004d00 0x7f4760005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.528+0000 7f4778ef8640 1 -- 172.21.15.70:0/3613870283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4760003520 con 0x7f4760003ef0 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.529+0000 7f4778ef8640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4760003ef0 0x7f47600047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f476400c240 tx=0x7f47600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d6029aa932793e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476402cb00 con 0x7f4760003ef0 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476402cc60 con 0x7f4760003ef0 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f476bfff640 1 -- 172.21.15.70:0/3613870283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764005af0 con 0x7f4760003ef0 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f477b95d640 1 -- 172.21.15.70:0/3613870283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4760003ef0 msgr2=0x7f47600047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f477b95d640 1 --2- 172.21.15.70:0/3613870283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4760003ef0 0x7f47600047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f476400c240 tx=0x7f47600057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f4779efa640 1 -- 172.21.15.70:0/3613870283 reap_dead start 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f477b95d640 1 -- 172.21.15.70:0/3613870283 shutdown_connections 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f477b95d640 1 -- 172.21.15.70:0/3613870283 >> 172.21.15.70:0/3613870283 conn(0x7f4774069850 msgr2=0x7f4774069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f477b95d640 1 -- 172.21.15.70:0/3613870283 shutdown_connections 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.530+0000 7f477b95d640 1 -- 172.21.15.70:0/3613870283 wait complete. 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.532+0000 7f477b95d640 1 Processor -- start 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.532+0000 7f477b95d640 1 -- start start 2023-08-25T09:42:10.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f477b95d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760003ef0 0x7f4774141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f477b95d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cf90 0x7f477415d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f477b95d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 0x7f47740cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f477b95d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477415fa30 con 0x7f47740cce20 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f477b95d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47740cd7e0 con 0x7f4760003ef0 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f477b95d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4774141f60 con 0x7f477415cf90 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f4779efa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 0x7f47740cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f4779efa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 0x7f47740cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37468/0 (socket says 172.21.15.70:37468) 2023-08-25T09:42:10.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.533+0000 7f4779efa640 1 -- 172.21.15.70:0/3296064474 learned_addr learned my addr 172.21.15.70:0/3296064474 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4779efa640 1 -- 172.21.15.70:0/3296064474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cf90 msgr2=0x7f477415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4779efa640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cf90 0x7f477415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4779efa640 1 -- 172.21.15.70:0/3296064474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760003ef0 msgr2=0x7f4774141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4779efa640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760003ef0 0x7f4774141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4779efa640 1 -- 172.21.15.70:0/3296064474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477415da50 con 0x7f47740cce20 2023-08-25T09:42:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4779efa640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 0x7f47740cd2a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f47740b13d0 tx=0x7f476c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c010040 con 0x7f47740cce20 2023-08-25T09:42:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00ba80 con 0x7f47740cce20 2023-08-25T09:42:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c00be30 con 0x7f47740cce20 2023-08-25T09:42:10.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.534+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477415dd30 con 0x7f47740cce20 2023-08-25T09:42:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.535+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47740ca970 con 0x7f47740cce20 2023-08-25T09:42:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.535+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477404e420 con 0x7f47740cce20 2023-08-25T09:42:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.536+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f476c01e410 con 0x7f47740cce20 2023-08-25T09:42:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.537+0000 7f4769ffb640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f475806a050 0x7f475806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.537+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6177+0+0 (secure 0 0 0) 0x7f476c088940 con 0x7f47740cce20 2023-08-25T09:42:10.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.538+0000 7f47796f9640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f475806a050 0x7f475806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.538+0000 7f47796f9640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f475806a050 0x7f475806c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4770009970 tx=0x7f4770009d70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.541+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f476c053f10 con 0x7f47740cce20 2023-08-25T09:42:10.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:10.772+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f477406baa0 con 0x7f47740cce20 2023-08-25T09:42:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.317+0000 7f4769ffb640 1 -- 172.21.15.70:0/3296064474 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f476c057530 con 0x7f47740cce20 2023-08-25T09:42:11.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f475806a050 msgr2=0x7f475806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f475806a050 0x7f475806c510 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4770009970 tx=0x7f4770009d70 comp rx=0 tx=0).stop 2023-08-25T09:42:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 msgr2=0x7f47740cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 0x7f47740cd2a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f47740b13d0 tx=0x7f476c00b480 comp rx=0 tx=0).stop 2023-08-25T09:42:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 shutdown_connections 2023-08-25T09:42:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f475806a050 0x7f475806c510 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47740cce20 0x7f47740cd2a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f477415cf90 0x7f477415d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 --2- 172.21.15.70:0/3296064474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4760003ef0 0x7f4774141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 >> 172.21.15.70:0/3296064474 conn(0x7f4774069850 msgr2=0x7f47740aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 shutdown_connections 2023-08-25T09:42:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.321+0000 7f477b95d640 1 -- 172.21.15.70:0/3296064474 wait complete. 2023-08-25T09:42:11.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:42:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.567+0000 7fc1a9446640 1 Processor -- start 2023-08-25T09:42:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.567+0000 7fc1a9446640 1 -- start start 2023-08-25T09:42:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a9446640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a4155120 0x7fc1a40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a9446640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40ca9f0 0x7fc1a415af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a9446640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a415e6f0 con 0x7fc1a40cd890 2023-08-25T09:42:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a9446640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a40cdd40 con 0x7fc1a4155120 2023-08-25T09:42:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a9446640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40caf30 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a4155120 0x7fc1a40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a4155120 0x7fc1a40cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37472/0 (socket says 172.21.15.70:37472) 2023-08-25T09:42:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.569+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/4232799178 learned_addr learned my addr 172.21.15.70:0/4232799178 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.570+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/4232799178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40ca9f0 msgr2=0x7fc1a415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:11.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.570+0000 7fc1a2ffd640 1 --2- 172.21.15.70:0/4232799178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40ca9f0 0x7fc1a415af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.570+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/4232799178 >> v1:172.21.15.155:6789/0 conn(0x7fc1a40cd890 legacy=0x7fc1a40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:11.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.570+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/4232799178 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a415e9d0 con 0x7fc1a4155120 2023-08-25T09:42:11.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.571+0000 7fc1a2ffd640 1 --2- 172.21.15.70:0/4232799178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a4155120 0x7fc1a40cd350 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc194009870 tx=0x7fc19402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c40310b83e1216 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:11.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.571+0000 7fc1a27fc640 1 -- 172.21.15.70:0/4232799178 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194031640 con 0x7fc1a4155120 2023-08-25T09:42:11.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.571+0000 7fc1a27fc640 1 -- 172.21.15.70:0/4232799178 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940317a0 con 0x7fc1a4155120 2023-08-25T09:42:11.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.571+0000 7fc1a27fc640 1 -- 172.21.15.70:0/4232799178 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194031b30 con 0x7fc1a4155120 2023-08-25T09:42:11.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 -- 172.21.15.70:0/4232799178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a4155120 msgr2=0x7fc1a40cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:11.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 --2- 172.21.15.70:0/4232799178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a4155120 0x7fc1a40cd350 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc194009870 tx=0x7fc19402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 -- 172.21.15.70:0/4232799178 shutdown_connections 2023-08-25T09:42:11.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 --2- 172.21.15.70:0/4232799178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40ca9f0 0x7fc1a415af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 --2- 172.21.15.70:0/4232799178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a4155120 0x7fc1a40cd350 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 -- 172.21.15.70:0/4232799178 >> 172.21.15.70:0/4232799178 conn(0x7fc1a4069c70 msgr2=0x7fc1a406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:11.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 -- 172.21.15.70:0/4232799178 shutdown_connections 2023-08-25T09:42:11.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.572+0000 7fc1a9446640 1 -- 172.21.15.70:0/4232799178 wait complete. 2023-08-25T09:42:11.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.574+0000 7fc1a9446640 1 Processor -- start 2023-08-25T09:42:11.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.574+0000 7fc1a9446640 1 -- start start 2023-08-25T09:42:11.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a9446640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 0x7fc1a41e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:11.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a9446640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cd890 0x7fc1a4151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:11.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a9446640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4155120 0x7fc1a41f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a9446640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a41e9350 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a9446640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a41e48a0 con 0x7fc1a4155120 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a9446640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a4151ab0 con 0x7fc1a40cd890 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 0x7fc1a41e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 0x7fc1a41e7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37486/0 (socket says 172.21.15.70:37486) 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/1411198515 learned_addr learned my addr 172.21.15.70:0/1411198515 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a37fe640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4155120 0x7fc1a41f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc19bfff640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cd890 0x7fc1a4151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cd890 msgr2=0x7fc1a4151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cd890 0x7fc1a4151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4155120 msgr2=0x7fc1a41f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4155120 0x7fc1a41f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a2ffd640 1 -- 172.21.15.70:0/1411198515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1940094d0 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc19bfff640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cd890 0x7fc1a4151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.575+0000 7fc1a37fe640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4155120 0x7fc1a41f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.576+0000 7fc1a2ffd640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 0x7fc1a41e7350 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc194009870 tx=0x7fc1940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.576+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19403ac60 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.576+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19403adc0 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.576+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194044630 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.576+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a41e8d90 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.576+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a41e4d80 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.577+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a404e420 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.578+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc19402f050 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.579+0000 7fc19b7fe640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:11.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.579+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fc1940aa610 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.579+0000 7fc19bfff640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:11.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.580+0000 7fc19bfff640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007fd0 tx=0x7fc18c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.583+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc194075c70 con 0x7fc1a40ca9f0 2023-08-25T09:42:11.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:11.771+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc1a415ec80 con 0x7fc1a40ca9f0 2023-08-25T09:42:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.487+0000 7fc19b7fe640 1 -- 172.21.15.70:0/1411198515 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v63) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc194031e60 con 0x7fc1a40ca9f0 2023-08-25T09:42:12.489 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:42:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 msgr2=0x7fc18406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc18c007fd0 tx=0x7fc18c009040 comp rx=0 tx=0).stop 2023-08-25T09:42:12.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 msgr2=0x7fc1a41e7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 0x7fc1a41e7350 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc194009870 tx=0x7fc1940046c0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 shutdown_connections 2023-08-25T09:42:12.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a4155120 0x7fc1a41f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cd890 0x7fc1a4151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 --2- 172.21.15.70:0/1411198515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a40ca9f0 0x7fc1a41e7350 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 >> 172.21.15.70:0/1411198515 conn(0x7fc1a4069c70 msgr2=0x7fc1a40cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 shutdown_connections 2023-08-25T09:42:12.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.491+0000 7fc1a9446640 1 -- 172.21.15.70:0/1411198515 wait complete. 2023-08-25T09:42:12.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:42:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.737+0000 7f7ed3fff640 1 Processor -- start 2023-08-25T09:42:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.738+0000 7f7ed3fff640 1 -- start start 2023-08-25T09:42:12.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40b1400 0x7f7ed40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ed40c9650 0x7f7ed40c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed3fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed40c8c00 con 0x7f7ed40b0030 2023-08-25T09:42:12.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed40c91c0 con 0x7f7ed40c9650 2023-08-25T09:42:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40b1e50 con 0x7f7ed40b1400 2023-08-25T09:42:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ed40c9650 0x7f7ed40c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:12.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed2ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7ed40b0030 0x7f7ed40b0410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49290/0 (socket says 172.21.15.70:49290) 2023-08-25T09:42:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ed40c9650 0x7f7ed40c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37490/0 (socket says 172.21.15.70:37490) 2023-08-25T09:42:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.739+0000 7f7ed2ffd640 1 -- 172.21.15.70:0/1839645311 learned_addr learned my addr 172.21.15.70:0/1839645311 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:12.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40b1400 0x7f7ed40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ed37fe640 1 -- 172.21.15.70:0/1839645311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40b1400 msgr2=0x7f7ed40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ed37fe640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40b1400 0x7f7ed40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ed37fe640 1 -- 172.21.15.70:0/1839645311 >> v1:172.21.15.70:6789/0 conn(0x7f7ed40b0030 legacy=0x7f7ed40b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:12.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ed37fe640 1 -- 172.21.15.70:0/1839645311 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed40b2130 con 0x7f7ed40c9650 2023-08-25T09:42:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ed27fc640 1 -- 172.21.15.70:0/1839645311 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386189663 0 0) 0x7f7ed40c8c00 con 0x7f7ed40b0030 2023-08-25T09:42:12.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.740+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40b1400 0x7f7ed40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:12.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.741+0000 7f7ed37fe640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40c9650 0x7f7ed40c9ae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b6b30c4e71efb9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.741+0000 7f7ed27fc640 1 -- 172.21.15.70:0/1839645311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc02cbd0 con 0x7f7ed40c9650 2023-08-25T09:42:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.741+0000 7f7ed27fc640 1 -- 172.21.15.70:0/1839645311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc02cd30 con 0x7f7ed40c9650 2023-08-25T09:42:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.741+0000 7f7ed27fc640 1 -- 172.21.15.70:0/1839645311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc02cea0 con 0x7f7ed40c9650 2023-08-25T09:42:12.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 -- 172.21.15.70:0/1839645311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40c9650 msgr2=0x7f7ed40c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40c9650 0x7f7ed40c9ae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009d90 tx=0x7f7ecc02be20 comp rx=0 tx=0).stop 2023-08-25T09:42:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 -- 172.21.15.70:0/1839645311 shutdown_connections 2023-08-25T09:42:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40c9650 0x7f7ed40c9ae0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/1839645311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40b1400 0x7f7ed40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 -- 172.21.15.70:0/1839645311 >> 172.21.15.70:0/1839645311 conn(0x7f7ed406b560 msgr2=0x7f7ed40cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 -- 172.21.15.70:0/1839645311 shutdown_connections 2023-08-25T09:42:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.742+0000 7f7ed3fff640 1 -- 172.21.15.70:0/1839645311 wait complete. 2023-08-25T09:42:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.744+0000 7f7ed3fff640 1 Processor -- start 2023-08-25T09:42:12.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.744+0000 7f7ed3fff640 1 -- start start 2023-08-25T09:42:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed40b0030 0x7f7ed4158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 0x7f7ed40c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40c9650 0x7f7ed40c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40c6ee0 con 0x7f7ed40b1400 2023-08-25T09:42:12.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40c62d0 con 0x7f7ed40b0030 2023-08-25T09:42:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40c65b0 con 0x7f7ed40c9650 2023-08-25T09:42:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 0x7f7ed40c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 0x7f7ed40c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37496/0 (socket says 172.21.15.70:37496) 2023-08-25T09:42:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 -- 172.21.15.70:0/830397318 learned_addr learned my addr 172.21.15.70:0/830397318 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:12.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 -- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40c9650 msgr2=0x7f7ed40c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40c9650 0x7f7ed40c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 -- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed40b0030 msgr2=0x7f7ed4158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ed2ffd640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed40b0030 0x7f7ed4158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed40b0030 0x7f7ed4158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:12.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.745+0000 7f7ecbfff640 1 -- 172.21.15.70:0/830397318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc009a70 con 0x7f7ed40b1400 2023-08-25T09:42:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.746+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 0x7f7ed40c7970 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c7f0 tx=0x7f7ebc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.746+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc007a30 con 0x7f7ed40b1400 2023-08-25T09:42:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.746+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc007b90 con 0x7f7ed40b1400 2023-08-25T09:42:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.746+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00ce40 con 0x7f7ed40b1400 2023-08-25T09:42:12.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.746+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed40c6890 con 0x7f7ed40b1400 2023-08-25T09:42:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.746+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed4159350 con 0x7f7ed40b1400 2023-08-25T09:42:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.747+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed41f2340 con 0x7f7ed40b1400 2023-08-25T09:42:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.748+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ebc0113f0 con 0x7f7ed40b1400 2023-08-25T09:42:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.750+0000 7f7ed0ff9640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb406a050 0x7f7eb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:12.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.750+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5739+0+0 (secure 0 0 0) 0x7f7ebc088640 con 0x7f7ed40b1400 2023-08-25T09:42:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.750+0000 7f7ed2ffd640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb406a050 0x7f7eb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:12.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.750+0000 7f7ed2ffd640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb406a050 0x7f7eb406c510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4000f60 tx=0x7f7ec4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.754+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ebc053dd0 con 0x7f7ed40b1400 2023-08-25T09:42:12.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:12.953+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7ed41594e0 con 0x7f7ed40b1400 2023-08-25T09:42:13.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.523+0000 7f7ed0ff9640 1 -- 172.21.15.70:0/830397318 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v64) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7ebc0573f0 con 0x7f7ed40b1400 2023-08-25T09:42:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:42:13.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.528+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb406a050 msgr2=0x7f7eb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.528+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb406a050 0x7f7eb406c510 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4000f60 tx=0x7f7ec4009040 comp rx=0 tx=0).stop 2023-08-25T09:42:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.528+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 msgr2=0x7f7ed40c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.528+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 0x7f7ed40c7970 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc00c7f0 tx=0x7f7ebc00cbf0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 shutdown_connections 2023-08-25T09:42:13.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb406a050 0x7f7eb406c510 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ed40c9650 0x7f7ed40c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ed40b1400 0x7f7ed40c7970 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 --2- 172.21.15.70:0/830397318 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ed40b0030 0x7f7ed4158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 >> 172.21.15.70:0/830397318 conn(0x7f7ed406b560 msgr2=0x7f7ed40aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:13.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 shutdown_connections 2023-08-25T09:42:13.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.529+0000 7f7ed3fff640 1 -- 172.21.15.70:0/830397318 wait complete. 2023-08-25T09:42:13.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:42:13.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.772+0000 7f2053d42640 1 Processor -- start 2023-08-25T09:42:13.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.773+0000 7f2053d42640 1 -- start start 2023-08-25T09:42:13.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.774+0000 7f2053d42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f204c15ae90 0x7f204c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:13.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.774+0000 7f2053d42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f204c06f300 0x7f204c06f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:13.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.774+0000 7f2053d42640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c15be60 con 0x7f204c0ce410 2023-08-25T09:42:13.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.774+0000 7f2053d42640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c06ee70 con 0x7f204c06f300 2023-08-25T09:42:13.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.774+0000 7f2053d42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0ced30 con 0x7f204c15ae90 2023-08-25T09:42:13.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f2051ade640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f204c0ce410 0x7f204c070770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49304/0 (socket says 172.21.15.70:49304) 2023-08-25T09:42:13.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f2051ade640 1 -- 172.21.15.70:0/4170832756 learned_addr learned my addr 172.21.15.70:0/4170832756 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20522df640 1 --2- 172.21.15.70:0/4170832756 >> v2:172.21.15.70:3300/0 conn(0x7f204c06f300 0x7f204c06f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f2050adc640 1 -- 172.21.15.70:0/4170832756 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027110998 0 0) 0x7f204c15be60 con 0x7f204c0ce410 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f2050adc640 1 -- 172.21.15.70:0/4170832756 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2038003520 con 0x7f204c0ce410 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20512dd640 1 --2- 172.21.15.70:0/4170832756 >> v2:172.21.15.155:3300/0 conn(0x7f204c15ae90 0x7f204c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20522df640 1 -- 172.21.15.70:0/4170832756 >> v2:172.21.15.155:3300/0 conn(0x7f204c15ae90 msgr2=0x7f204c0ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20522df640 1 --2- 172.21.15.70:0/4170832756 >> v2:172.21.15.155:3300/0 conn(0x7f204c15ae90 0x7f204c0ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20522df640 1 -- 172.21.15.70:0/4170832756 >> v1:172.21.15.70:6789/0 conn(0x7f204c0ce410 legacy=0x7f204c070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20522df640 1 -- 172.21.15.70:0/4170832756 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c15b2f0 con 0x7f204c06f300 2023-08-25T09:42:13.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.775+0000 7f20512dd640 1 --2- 172.21.15.70:0/4170832756 >> v2:172.21.15.155:3300/0 conn(0x7f204c15ae90 0x7f204c0ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:13.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.776+0000 7f20522df640 1 --2- 172.21.15.70:0/4170832756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c06f300 0x7f204c06f790 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f204c0cc000 tx=0x7f204402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175e315c747cf3dc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:13.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.777+0000 7f2050adc640 1 -- 172.21.15.70:0/4170832756 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204402cc80 con 0x7f204c06f300 2023-08-25T09:42:13.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.777+0000 7f2050adc640 1 -- 172.21.15.70:0/4170832756 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204402cde0 con 0x7f204c06f300 2023-08-25T09:42:13.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.777+0000 7f2050adc640 1 -- 172.21.15.70:0/4170832756 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044034660 con 0x7f204c06f300 2023-08-25T09:42:13.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.777+0000 7f2053d42640 1 -- 172.21.15.70:0/4170832756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c06f300 msgr2=0x7f204c06f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.777+0000 7f2053d42640 1 --2- 172.21.15.70:0/4170832756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c06f300 0x7f204c06f790 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f204c0cc000 tx=0x7f204402bed0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.778+0000 7f2053d42640 1 -- 172.21.15.70:0/4170832756 shutdown_connections 2023-08-25T09:42:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.778+0000 7f2053d42640 1 --2- 172.21.15.70:0/4170832756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c06f300 0x7f204c06f790 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.778+0000 7f2053d42640 1 --2- 172.21.15.70:0/4170832756 >> v2:172.21.15.155:3300/0 conn(0x7f204c15ae90 0x7f204c0ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.778+0000 7f2053d42640 1 -- 172.21.15.70:0/4170832756 >> 172.21.15.70:0/4170832756 conn(0x7f204c069850 msgr2=0x7f204c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.778+0000 7f2053d42640 1 -- 172.21.15.70:0/4170832756 shutdown_connections 2023-08-25T09:42:13.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.778+0000 7f2053d42640 1 -- 172.21.15.70:0/4170832756 wait complete. 2023-08-25T09:42:13.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.779+0000 7f2053d42640 1 Processor -- start 2023-08-25T09:42:13.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.779+0000 7f2053d42640 1 -- start start 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.780+0000 7f2053d42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204c06f300 0x7f204c0c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.780+0000 7f2053d42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 0x7f204c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.780+0000 7f2053d42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f204c15ae90 0x7f204c06beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.780+0000 7f2053d42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0c7b30 con 0x7f204c0ce410 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.780+0000 7f2053d42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c1effb0 con 0x7f204c06f300 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.780+0000 7f2053d42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c1f3ab0 con 0x7f204c15ae90 2023-08-25T09:42:13.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 0x7f204c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 0x7f204c1efa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37520/0 (socket says 172.21.15.70:37520) 2023-08-25T09:42:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 -- 172.21.15.70:0/326766353 learned_addr learned my addr 172.21.15.70:0/326766353 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 -- 172.21.15.70:0/326766353 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f204c15ae90 msgr2=0x7f204c06beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f204c15ae90 0x7f204c06beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 -- 172.21.15.70:0/326766353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204c06f300 msgr2=0x7f204c0c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204c06f300 0x7f204c0c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 -- 172.21.15.70:0/326766353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2044009a70 con 0x7f204c0ce410 2023-08-25T09:42:13.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20512dd640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 0x7f204c1efa70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f203c00c7f0 tx=0x7f203c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.781+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c007820 con 0x7f204c0ce410 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.782+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c007980 con 0x7f204c0ce410 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.782+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c007cf0 con 0x7f204c0ce410 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.782+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c1f3d90 con 0x7f204c0ce410 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.782+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c1f4350 con 0x7f204c0ce410 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.783+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0c5420 con 0x7f204c0ce410 2023-08-25T09:42:13.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.784+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f203c0113f0 con 0x7f204c0ce410 2023-08-25T09:42:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.784+0000 7f20427fc640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f203006a050 0x7f203006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.785+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f203c088610 con 0x7f204c0ce410 2023-08-25T09:42:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.785+0000 7f2051ade640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f203006a050 0x7f203006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:13.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.785+0000 7f2051ade640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f203006a050 0x7f203006c510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2048001610 tx=0x7f2048007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:13.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.789+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f204c0c5420 con 0x7f204c0ce410 2023-08-25T09:42:13.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.980+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f204c0c53f0 con 0x7f204c0ce410 2023-08-25T09:42:13.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.988+0000 7f20427fc640 1 -- 172.21.15.70:0/326766353 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f203c053f80 con 0x7f204c0ce410 2023-08-25T09:42:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f203006a050 msgr2=0x7f203006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f203006a050 0x7f203006c510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2048001610 tx=0x7f2048007930 comp rx=0 tx=0).stop 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 msgr2=0x7f204c1efa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 0x7f204c1efa70 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f203c00c7f0 tx=0x7f203c00ccb0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 shutdown_connections 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f203006a050 0x7f203006c510 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f204c15ae90 0x7f204c06beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f204c0ce410 0x7f204c1efa70 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 --2- 172.21.15.70:0/326766353 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f204c06f300 0x7f204c0c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 >> 172.21.15.70:0/326766353 conn(0x7f204c069850 msgr2=0x7f204c157680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 shutdown_connections 2023-08-25T09:42:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:13.991+0000 7f2053d42640 1 -- 172.21.15.70:0/326766353 wait complete. 2023-08-25T09:42:14.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-08-25T09:42:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.223+0000 7fc670e71640 1 Processor -- start 2023-08-25T09:42:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.223+0000 7fc670e71640 1 -- start start 2023-08-25T09:42:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc670e71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc670e71640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc66c15c6d0 0x7fc66c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc670e71640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c0b1b60 con 0x7fc66c15e830 2023-08-25T09:42:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc670e71640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c06bd80 con 0x7fc66c15c6d0 2023-08-25T09:42:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc670e71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c06b7c0 con 0x7fc66c15ec10 2023-08-25T09:42:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc66ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc66c15c6d0 0x7fc66c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc66ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc66c15c6d0 0x7fc66c0b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37536/0 (socket says 172.21.15.70:37536) 2023-08-25T09:42:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc66ad9d640 1 -- 172.21.15.70:0/874306621 learned_addr learned my addr 172.21.15.70:0/874306621 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.225+0000 7fc66a59c640 1 --1- 172.21.15.70:0/874306621 >> v1:172.21.15.70:6789/0 conn(0x7fc66c15e830 0x7fc66c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49320/0 (socket says 172.21.15.70:49320) 2023-08-25T09:42:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc669d9b640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc66ad9d640 1 -- 172.21.15.70:0/874306621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 msgr2=0x7fc66c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc66ad9d640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc66ad9d640 1 -- 172.21.15.70:0/874306621 >> v1:172.21.15.70:6789/0 conn(0x7fc66c15e830 legacy=0x7fc66c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc66ad9d640 1 -- 172.21.15.70:0/874306621 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c06baa0 con 0x7fc66c15c6d0 2023-08-25T09:42:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc66959a640 1 -- 172.21.15.70:0/874306621 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477893713 0 0) 0x7fc66c0b1b60 con 0x7fc66c15e830 2023-08-25T09:42:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.226+0000 7fc669d9b640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.227+0000 7fc66ad9d640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0b1ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc664009e50 tx=0x7fc66402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bcde8dc47d7392 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.227+0000 7fc66959a640 1 -- 172.21.15.70:0/874306621 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66402ca60 con 0x7fc66c15c6d0 2023-08-25T09:42:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.227+0000 7fc66959a640 1 -- 172.21.15.70:0/874306621 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66402cbc0 con 0x7fc66c15c6d0 2023-08-25T09:42:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.227+0000 7fc66959a640 1 -- 172.21.15.70:0/874306621 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6640343f0 con 0x7fc66c15c6d0 2023-08-25T09:42:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 -- 172.21.15.70:0/874306621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15c6d0 msgr2=0x7fc66c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0b1ff0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc664009e50 tx=0x7fc66402bec0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 -- 172.21.15.70:0/874306621 shutdown_connections 2023-08-25T09:42:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0b1ff0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 --2- 172.21.15.70:0/874306621 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 -- 172.21.15.70:0/874306621 >> 172.21.15.70:0/874306621 conn(0x7fc66c069850 msgr2=0x7fc66c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 -- 172.21.15.70:0/874306621 shutdown_connections 2023-08-25T09:42:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.228+0000 7fc670e71640 1 -- 172.21.15.70:0/874306621 wait complete. 2023-08-25T09:42:14.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.229+0000 7fc670e71640 1 Processor -- start 2023-08-25T09:42:14.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.230+0000 7fc670e71640 1 -- start start 2023-08-25T09:42:14.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.230+0000 7fc670e71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.230+0000 7fc670e71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 0x7fc66c1f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc670e71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c1f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc670e71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c4a20 con 0x7fc66c15e830 2023-08-25T09:42:14.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc670e71640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c3e10 con 0x7fc66c15c6d0 2023-08-25T09:42:14.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc670e71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c40f0 con 0x7fc66c15ec10 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 0x7fc66c1f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 0x7fc66c1f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37538/0 (socket says 172.21.15.70:37538) 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 -- 172.21.15.70:0/1554511811 learned_addr learned my addr 172.21.15.70:0/1554511811 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 -- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 msgr2=0x7fc66c1f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c1f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc66a59c640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 -- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc66c15c6d0 msgr2=0x7fc66c0c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc669d9b640 1 -- 172.21.15.70:0/1554511811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc664009b30 con 0x7fc66c15e830 2023-08-25T09:42:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.231+0000 7fc66a59c640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.232+0000 7fc669d9b640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 0x7fc66c1f3230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c7f0 tx=0x7fc66000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.232+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660007820 con 0x7fc66c15e830 2023-08-25T09:42:14.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.232+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660007980 con 0x7fc66c15e830 2023-08-25T09:42:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.232+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660007cf0 con 0x7fc66c15e830 2023-08-25T09:42:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.232+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c0c43d0 con 0x7fc66c15e830 2023-08-25T09:42:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.232+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c144ca0 con 0x7fc66c15e830 2023-08-25T09:42:14.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.233+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c0c4b90 con 0x7fc66c15e830 2023-08-25T09:42:14.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.234+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc6600113f0 con 0x7fc66c15e830 2023-08-25T09:42:14.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.236+0000 7fc65affd640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc64c06a050 0x7fc64c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.236+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc660088670 con 0x7fc66c15e830 2023-08-25T09:42:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.236+0000 7fc66a59c640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc64c06a050 0x7fc64c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.236+0000 7fc66a59c640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc64c06a050 0x7fc64c06c510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc65c000f60 tx=0x7fc65c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:14.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.241+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc660053fe0 con 0x7fc66c15e830 2023-08-25T09:42:14.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:42:14.463 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:42:14.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.500+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7fc66c04e420 con 0x7fc66c15e830 2023-08-25T09:42:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.510+0000 7fc65affd640 1 -- 172.21.15.70:0/1554511811 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v203) v1 ==== 142+0+0 (secure 0 0 0) 0x7fc660057600 con 0x7fc66c15e830 2023-08-25T09:42:14.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc64c06a050 msgr2=0x7fc64c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc64c06a050 0x7fc64c06c510 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc65c000f60 tx=0x7fc65c009040 comp rx=0 tx=0).stop 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 msgr2=0x7fc66c1f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 0x7fc66c1f3230 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c7f0 tx=0x7fc66000ccb0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 shutdown_connections 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc64c06a050 0x7fc64c06c510 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc66c15ec10 0x7fc66c1f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc66c15e830 0x7fc66c1f3230 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 --2- 172.21.15.70:0/1554511811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc66c15c6d0 0x7fc66c0c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 >> 172.21.15.70:0/1554511811 conn(0x7fc66c069850 msgr2=0x7fc66c15da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 shutdown_connections 2023-08-25T09:42:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.513+0000 7fc670e71640 1 -- 172.21.15.70:0/1554511811 wait complete. 2023-08-25T09:42:14.532 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-08-25T09:42:14.532 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-08-25T09:42:14.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:14.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.783+0000 7fae5bfd5640 1 Processor -- start 2023-08-25T09:42:14.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.783+0000 7fae5bfd5640 1 -- start start 2023-08-25T09:42:14.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae5bfd5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae5415e830 0x7fae5406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae5bfd5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fae5406b2b0 0x7fae5415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae5bfd5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae540b1b60 con 0x7fae5415c6d0 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae5bfd5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5406bd80 con 0x7fae5415e830 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae5bfd5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5406b7c0 con 0x7fae5406b2b0 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae59d71640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae5415e830 0x7fae5406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae59d71640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae5415e830 0x7fae5406ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37542/0 (socket says 172.21.15.70:37542) 2023-08-25T09:42:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.785+0000 7fae59d71640 1 -- 172.21.15.70:0/4257172718 learned_addr learned my addr 172.21.15.70:0/4257172718 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:14.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.786+0000 7fae59570640 1 --2- 172.21.15.70:0/4257172718 >> v2:172.21.15.155:3300/0 conn(0x7fae5406b2b0 0x7fae5415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.786+0000 7fae59d71640 1 -- 172.21.15.70:0/4257172718 >> v2:172.21.15.155:3300/0 conn(0x7fae5406b2b0 msgr2=0x7fae5415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.786+0000 7fae59d71640 1 --2- 172.21.15.70:0/4257172718 >> v2:172.21.15.155:3300/0 conn(0x7fae5406b2b0 0x7fae5415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.786+0000 7fae59d71640 1 -- 172.21.15.70:0/4257172718 >> v1:172.21.15.155:6789/0 conn(0x7fae5415c6d0 legacy=0x7fae540b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.786+0000 7fae59d71640 1 -- 172.21.15.70:0/4257172718 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5406baa0 con 0x7fae5415e830 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.787+0000 7fae59d71640 1 --2- 172.21.15.70:0/4257172718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415e830 0x7fae5406ad70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fae540cf6f0 tx=0x7fae5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ae98058a4511a1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.787+0000 7fae58d6f640 1 -- 172.21.15.70:0/4257172718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50031870 con 0x7fae5415e830 2023-08-25T09:42:14.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.787+0000 7fae58d6f640 1 -- 172.21.15.70:0/4257172718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae500319d0 con 0x7fae5415e830 2023-08-25T09:42:14.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.787+0000 7fae58d6f640 1 -- 172.21.15.70:0/4257172718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50031d60 con 0x7fae5415e830 2023-08-25T09:42:14.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 -- 172.21.15.70:0/4257172718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415e830 msgr2=0x7fae5406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/4257172718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415e830 0x7fae5406ad70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fae540cf6f0 tx=0x7fae5002b910 comp rx=0 tx=0).stop 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 -- 172.21.15.70:0/4257172718 shutdown_connections 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/4257172718 >> v2:172.21.15.155:3300/0 conn(0x7fae5406b2b0 0x7fae5415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/4257172718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415e830 0x7fae5406ad70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 -- 172.21.15.70:0/4257172718 >> 172.21.15.70:0/4257172718 conn(0x7fae54069850 msgr2=0x7fae54069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 -- 172.21.15.70:0/4257172718 shutdown_connections 2023-08-25T09:42:14.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.788+0000 7fae5bfd5640 1 -- 172.21.15.70:0/4257172718 wait complete. 2023-08-25T09:42:14.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.790+0000 7fae5bfd5640 1 Processor -- start 2023-08-25T09:42:14.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.790+0000 7fae5bfd5640 1 -- start start 2023-08-25T09:42:14.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.791+0000 7fae5bfd5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae5406b2b0 0x7fae541f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.791+0000 7fae5bfd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 0x7fae540a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae5bfd5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5415e830 0x7fae540c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae5bfd5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae540703e0 con 0x7fae5415c6d0 2023-08-25T09:42:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae5bfd5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5406fe20 con 0x7fae5415e830 2023-08-25T09:42:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae5bfd5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae54070100 con 0x7fae5406b2b0 2023-08-25T09:42:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 0x7fae540a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 0x7fae540a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37554/0 (socket says 172.21.15.70:37554) 2023-08-25T09:42:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 -- 172.21.15.70:0/2436556803 learned_addr learned my addr 172.21.15.70:0/2436556803 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:14.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59d71640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae5406b2b0 0x7fae541f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae5a572640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5415e830 0x7fae540c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 -- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae5406b2b0 msgr2=0x7fae541f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae5406b2b0 0x7fae541f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 -- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5415e830 msgr2=0x7fae540c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5415e830 0x7fae540c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.792+0000 7fae59570640 1 -- 172.21.15.70:0/2436556803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae500094d0 con 0x7fae5415c6d0 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.793+0000 7fae59570640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 0x7fae540a8460 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fae4400c7f0 tx=0x7fae4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.793+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44004220 con 0x7fae5415c6d0 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.793+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae440043a0 con 0x7fae5415c6d0 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.793+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44007860 con 0x7fae5415c6d0 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.793+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae540c9560 con 0x7fae5415c6d0 2023-08-25T09:42:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.793+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae541f1f00 con 0x7fae5415c6d0 2023-08-25T09:42:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.794+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae540c9a80 con 0x7fae5415c6d0 2023-08-25T09:42:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.795+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae44007bd0 con 0x7fae5415c6d0 2023-08-25T09:42:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.796+0000 7fae4a7fc640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae3806a050 0x7fae3806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.796+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fae440882a0 con 0x7fae5415c6d0 2023-08-25T09:42:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.797+0000 7fae59d71640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae3806a050 0x7fae3806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:14.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.797+0000 7fae59d71640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae3806a050 0x7fae3806c510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae50036000 tx=0x7fae5003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:14.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.800+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae540c9a80 con 0x7fae5415c6d0 2023-08-25T09:42:14.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.989+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae540c9a80 con 0x7fae5415c6d0 2023-08-25T09:42:14.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.991+0000 7fae4a7fc640 1 -- 172.21.15.70:0/2436556803 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+14782 (secure 0 0 0) 0x7fae540c9a80 con 0x7fae5415c6d0 2023-08-25T09:42:14.993 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:14.994 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":64,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:13.504434+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6837/3881965592":"2023-08-26T09:42:10.304635+0000","172.21.15.70:6835/3881965592":"2023-08-26T09:42:10.304635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:14.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.994+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae3806a050 msgr2=0x7fae3806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.994+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae3806a050 0x7fae3806c510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fae50036000 tx=0x7fae5003c040 comp rx=0 tx=0).stop 2023-08-25T09:42:14.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.994+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 msgr2=0x7fae540a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.994+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 0x7fae540a8460 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fae4400c7f0 tx=0x7fae4400ccb0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 shutdown_connections 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae3806a050 0x7fae3806c510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae5415e830 0x7fae540c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae5415c6d0 0x7fae540a8460 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 --2- 172.21.15.70:0/2436556803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae5406b2b0 0x7fae541f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:14.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 >> 172.21.15.70:0/2436556803 conn(0x7fae54069850 msgr2=0x7fae540ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:14.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 shutdown_connections 2023-08-25T09:42:14.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:14.995+0000 7fae5bfd5640 1 -- 172.21.15.70:0/2436556803 wait complete. 2023-08-25T09:42:15.015 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:42:15.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:42:15.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.224+0000 7f8f64b96640 1 Processor -- start 2023-08-25T09:42:15.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.224+0000 7f8f64b96640 1 -- start start 2023-08-25T09:42:15.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.225+0000 7f8f64b96640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8f6015ae50 0x7f8f600c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.225+0000 7f8f64b96640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f600cdc20 0x7f8f60069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.225+0000 7f8f64b96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600ce060 0x7f8f60069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.225+0000 7f8f64b96640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f600cc5f0 con 0x7f8f600cdc20 2023-08-25T09:42:15.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.225+0000 7f8f64b96640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f60070650 con 0x7f8f6015ae50 2023-08-25T09:42:15.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.225+0000 7f8f64b96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60070930 con 0x7f8f600ce060 2023-08-25T09:42:15.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f600cdc20 0x7f8f60069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8f600cdc20 0x7f8f60069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37568/0 (socket says 172.21.15.70:37568) 2023-08-25T09:42:15.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 -- 172.21.15.70:0/4136571071 learned_addr learned my addr 172.21.15.70:0/4136571071 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:15.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f5e59c640 1 --2- 172.21.15.70:0/4136571071 >> v2:172.21.15.155:3300/0 conn(0x7f8f6015ae50 0x7f8f600c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f5ed9d640 1 --2- 172.21.15.70:0/4136571071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600ce060 0x7f8f60069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 -- 172.21.15.70:0/4136571071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600ce060 msgr2=0x7f8f60069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 --2- 172.21.15.70:0/4136571071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600ce060 0x7f8f60069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 -- 172.21.15.70:0/4136571071 >> v2:172.21.15.155:3300/0 conn(0x7f8f6015ae50 msgr2=0x7f8f600c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 --2- 172.21.15.70:0/4136571071 >> v2:172.21.15.155:3300/0 conn(0x7f8f6015ae50 0x7f8f600c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.226+0000 7f8f57fff640 1 -- 172.21.15.70:0/4136571071 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60070c10 con 0x7f8f600cdc20 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.227+0000 7f8f57fff640 1 --2- 172.21.15.70:0/4136571071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f600cdc20 0x7f8f60069850 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f600b1210 tx=0x7f8f4802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e403aadac980eeae server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f5dd9b640 1 -- 172.21.15.70:0/4136571071 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48030860 con 0x7f8f600cdc20 2023-08-25T09:42:15.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f5dd9b640 1 -- 172.21.15.70:0/4136571071 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f480309c0 con 0x7f8f600cdc20 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f5dd9b640 1 -- 172.21.15.70:0/4136571071 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48030d50 con 0x7f8f600cdc20 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 -- 172.21.15.70:0/4136571071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f600cdc20 msgr2=0x7f8f60069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 --2- 172.21.15.70:0/4136571071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f600cdc20 0x7f8f60069850 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8f600b1210 tx=0x7f8f4802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 -- 172.21.15.70:0/4136571071 shutdown_connections 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 --2- 172.21.15.70:0/4136571071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600ce060 0x7f8f60069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 --2- 172.21.15.70:0/4136571071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f600cdc20 0x7f8f60069850 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 --2- 172.21.15.70:0/4136571071 >> v2:172.21.15.155:3300/0 conn(0x7f8f6015ae50 0x7f8f600c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.228+0000 7f8f64b96640 1 -- 172.21.15.70:0/4136571071 >> 172.21.15.70:0/4136571071 conn(0x7f8f600cb0d0 msgr2=0x7f8f600cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:15.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.229+0000 7f8f64b96640 1 -- 172.21.15.70:0/4136571071 shutdown_connections 2023-08-25T09:42:15.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.229+0000 7f8f64b96640 1 -- 172.21.15.70:0/4136571071 wait complete. 2023-08-25T09:42:15.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.230+0000 7f8f64b96640 1 Processor -- start 2023-08-25T09:42:15.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.230+0000 7f8f64b96640 1 -- start start 2023-08-25T09:42:15.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.231+0000 7f8f64b96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600cdc20 0x7f8f60150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.231+0000 7f8f64b96640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f600ce060 0x7f8f6015d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.231+0000 7f8f64b96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 0x7f8f600cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.231+0000 7f8f64b96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6015c530 con 0x7f8f6015ae50 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.231+0000 7f8f64b96640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6015b920 con 0x7f8f600ce060 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.231+0000 7f8f64b96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6015bc00 con 0x7f8f600cdc20 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 0x7f8f600cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 0x7f8f600cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37574/0 (socket says 172.21.15.70:37574) 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 -- 172.21.15.70:0/1757382093 learned_addr learned my addr 172.21.15.70:0/1757382093 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f57fff640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f600ce060 0x7f8f6015d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5e59c640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600cdc20 0x7f8f60150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 -- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600cdc20 msgr2=0x7f8f60150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600cdc20 0x7f8f60150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 -- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f600ce060 msgr2=0x7f8f6015d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f600ce060 0x7f8f6015d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 -- 172.21.15.70:0/1757382093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480094d0 con 0x7f8f6015ae50 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5e59c640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600cdc20 0x7f8f60150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:15.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.232+0000 7f8f5ed9d640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 0x7f8f600cc9d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f58007e30 tx=0x7f8f5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:15.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.233+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58014620 con 0x7f8f6015ae50 2023-08-25T09:42:15.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.233+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5800ee90 con 0x7f8f6015ae50 2023-08-25T09:42:15.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.233+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f580149c0 con 0x7f8f6015ae50 2023-08-25T09:42:15.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.233+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6015bee0 con 0x7f8f6015ae50 2023-08-25T09:42:15.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.233+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6015ddc0 con 0x7f8f6015ae50 2023-08-25T09:42:15.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.234+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f60066fb0 con 0x7f8f6015ae50 2023-08-25T09:42:15.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.235+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8f58014b20 con 0x7f8f6015ae50 2023-08-25T09:42:15.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.236+0000 7f8f56ffd640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8f4006a080 0x7f8f4006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.236+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8f58088cd0 con 0x7f8f6015ae50 2023-08-25T09:42:15.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.236+0000 7f8f5e59c640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8f4006a080 0x7f8f4006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.237+0000 7f8f5e59c640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8f4006a080 0x7f8f4006c540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f50006fd0 tx=0x7f8f50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:15.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.240+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8f58054640 con 0x7f8f6015ae50 2023-08-25T09:42:15.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.430+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f601e56d0 con 0x7f8f6015ae50 2023-08-25T09:42:15.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.430+0000 7f8f56ffd640 1 -- 172.21.15.70:0/1757382093 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f58057c60 con 0x7f8f6015ae50 2023-08-25T09:42:15.432 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:42:15.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.433+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8f4006a080 msgr2=0x7f8f4006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.433+0000 7f8f64b96640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8f4006a080 0x7f8f4006c540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8f50006fd0 tx=0x7f8f50008040 comp rx=0 tx=0).stop 2023-08-25T09:42:15.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.433+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 msgr2=0x7f8f600cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.433+0000 7f8f64b96640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 0x7f8f600cc9d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8f58007e30 tx=0x7f8f5800e4b0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 shutdown_connections 2023-08-25T09:42:15.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8f4006a080 0x7f8f4006c540 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8f6015ae50 0x7f8f600cc9d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8f600ce060 0x7f8f6015d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 --2- 172.21.15.70:0/1757382093 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8f600cdc20 0x7f8f60150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 >> 172.21.15.70:0/1757382093 conn(0x7f8f600cb0d0 msgr2=0x7f8f60158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:15.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 shutdown_connections 2023-08-25T09:42:15.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.434+0000 7f8f64b96640 1 -- 172.21.15.70:0/1757382093 wait complete. 2023-08-25T09:42:15.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:42:15.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.674+0000 7f658f414640 1 Processor -- start 2023-08-25T09:42:15.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.674+0000 7f658f414640 1 -- start start 2023-08-25T09:42:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.675+0000 7f658f414640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6588160170 0x7f65880b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f658f414640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65880b7170 con 0x7f65880cdf30 2023-08-25T09:42:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f658f414640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65880b7450 con 0x7f65880b6be0 2023-08-25T09:42:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f658f414640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6588160550 con 0x7f6588160170 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f658d9b1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f65880cdf30 0x7f65880d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49336/0 (socket says 172.21.15.70:49336) 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f658d9b1640 1 -- 172.21.15.70:0/1838198508 learned_addr learned my addr 172.21.15.70:0/1838198508 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f658d1b0640 1 --2- 172.21.15.70:0/1838198508 >> v2:172.21.15.155:3300/0 conn(0x7f6588160170 0x7f65880b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.676+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340164485 0 0) 0x7f65880b7170 con 0x7f65880cdf30 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6574003520 con 0x7f65880cdf30 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083647760 0 0) 0x7f6574003520 con 0x7f65880cdf30 2023-08-25T09:42:15.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 >> v2:172.21.15.155:3300/0 conn(0x7f6588160170 msgr2=0x7f65880b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 --2- 172.21.15.70:0/1838198508 >> v2:172.21.15.155:3300/0 conn(0x7f6588160170 0x7f65880b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 >> v1:172.21.15.155:6789/0 conn(0x7f65880b6be0 legacy=0x7f65880cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880b7730 con 0x7f65880cdf30 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f65800019f0 con 0x7f65880cdf30 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 >> v1:172.21.15.70:6789/0 conn(0x7f65880cdf30 legacy=0x7f65880d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6574003970 0x7f6574003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65740047c0 0x7f6574004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6574005180 0x7f657400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6588160550 con 0x7f65740047c0 2023-08-25T09:42:15.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657400c120 con 0x7f6574005180 2023-08-25T09:42:15.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.677+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657400c2e0 con 0x7f6574003970 2023-08-25T09:42:15.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658c9af640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65740047c0 0x7f6574004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658d1b0640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6574003970 0x7f6574003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658d9b1640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6574005180 0x7f657400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658c9af640 1 -- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6574003970 msgr2=0x7f6574003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658c9af640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6574003970 0x7f6574003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658c9af640 1 -- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6574005180 msgr2=0x7f657400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658c9af640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6574005180 0x7f657400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658c9af640 1 -- 172.21.15.70:0/1838198508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6574003520 con 0x7f65740047c0 2023-08-25T09:42:15.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658d9b1640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6574005180 0x7f657400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.678+0000 7f658d9b1640 1 -- 172.21.15.70:0/1838198508 reap_dead start 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.679+0000 7f658c9af640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65740047c0 0x7f6574004c40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6578001750 tx=0x7f65740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a719845a5c2e78c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.679+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65780052d0 con 0x7f65740047c0 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.679+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657802e050 con 0x7f65740047c0 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.679+0000 7f657ffff640 1 -- 172.21.15.70:0/1838198508 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578003830 con 0x7f65740047c0 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 -- 172.21.15.70:0/1838198508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65740047c0 msgr2=0x7f6574004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65740047c0 0x7f6574004c40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6578001750 tx=0x7f65740057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 -- 172.21.15.70:0/1838198508 shutdown_connections 2023-08-25T09:42:15.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 --2- 172.21.15.70:0/1838198508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65740047c0 0x7f6574004c40 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 -- 172.21.15.70:0/1838198508 >> 172.21.15.70:0/1838198508 conn(0x7f6588070290 msgr2=0x7f65880706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:15.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 -- 172.21.15.70:0/1838198508 shutdown_connections 2023-08-25T09:42:15.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.680+0000 7f658f414640 1 -- 172.21.15.70:0/1838198508 wait complete. 2023-08-25T09:42:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.682+0000 7f658f414640 1 Processor -- start 2023-08-25T09:42:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.682+0000 7f658f414640 1 -- start start 2023-08-25T09:42:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658f414640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65740047c0 0x7f658814bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658f414640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 0x7f65880ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658f414640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65880cbea0 0x7f65880cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658f414640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65880cc860 con 0x7f65880acd50 2023-08-25T09:42:15.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658f414640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65881f69a0 con 0x7f65740047c0 2023-08-25T09:42:15.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658f414640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658814c270 con 0x7f65880cbea0 2023-08-25T09:42:15.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658c9af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 0x7f65880ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658c9af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 0x7f65880ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37594/0 (socket says 172.21.15.70:37594) 2023-08-25T09:42:15.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.683+0000 7f658c9af640 1 -- 172.21.15.70:0/2925167863 learned_addr learned my addr 172.21.15.70:0/2925167863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:15.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658d1b0640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65740047c0 0x7f658814bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658c9af640 1 -- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65880cbea0 msgr2=0x7f65880cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658d9b1640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65880cbea0 0x7f65880cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658c9af640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65880cbea0 0x7f65880cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658c9af640 1 -- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65740047c0 msgr2=0x7f658814bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658c9af640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65740047c0 0x7f658814bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658c9af640 1 -- 172.21.15.70:0/2925167863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65881f5be0 con 0x7f65880acd50 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658d1b0640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65740047c0 0x7f658814bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658c9af640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 0x7f65880ad1d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6578005040 tx=0x7f6578001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578007a30 con 0x7f65880acd50 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65780069e0 con 0x7f65880acd50 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6578006d70 con 0x7f65880acd50 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.684+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65881f5ec0 con 0x7f65880acd50 2023-08-25T09:42:15.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.685+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65881f6480 con 0x7f65880acd50 2023-08-25T09:42:15.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.686+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6588066fb0 con 0x7f65880acd50 2023-08-25T09:42:15.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.686+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f65780035f0 con 0x7f65880acd50 2023-08-25T09:42:15.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.687+0000 7f657dffb640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f656c06a080 0x7f656c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:15.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.687+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f65780aa2b0 con 0x7f65880acd50 2023-08-25T09:42:15.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.688+0000 7f658d1b0640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f656c06a080 0x7f656c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:15.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.688+0000 7f658d1b0640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f656c06a080 0x7f656c06c540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6588152cb0 tx=0x7f6584008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:15.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.692+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6578075d30 con 0x7f65880acd50 2023-08-25T09:42:15.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.880+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65880708e0 con 0x7f65880acd50 2023-08-25T09:42:15.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.881+0000 7f657dffb640 1 -- 172.21.15.70:0/2925167863 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6578079350 con 0x7f65880acd50 2023-08-25T09:42:15.882 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:42:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f656c06a080 msgr2=0x7f656c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f656c06a080 0x7f656c06c540 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6588152cb0 tx=0x7f6584008040 comp rx=0 tx=0).stop 2023-08-25T09:42:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 msgr2=0x7f65880ad1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:15.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 0x7f65880ad1d0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6578005040 tx=0x7f6578001120 comp rx=0 tx=0).stop 2023-08-25T09:42:15.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 shutdown_connections 2023-08-25T09:42:15.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f656c06a080 0x7f656c06c540 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f65880cbea0 0x7f65880cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f65880acd50 0x7f65880ad1d0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 --2- 172.21.15.70:0/2925167863 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65740047c0 0x7f658814bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:15.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 >> 172.21.15.70:0/2925167863 conn(0x7f6588070290 msgr2=0x7f65880d1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:15.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 shutdown_connections 2023-08-25T09:42:15.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:15.884+0000 7f658f414640 1 -- 172.21.15.70:0/2925167863 wait complete. 2023-08-25T09:42:15.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-08-25T09:42:16.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.121+0000 7f0ec9883640 1 Processor -- start 2023-08-25T09:42:16.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.121+0000 7f0ec9883640 1 -- start start 2023-08-25T09:42:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.122+0000 7f0ec9883640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ec4159aa0 0x7f0ec4070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.122+0000 7f0ec9883640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0ec415df10 0x7f0ec4070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.122+0000 7f0ec9883640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec406aec0 con 0x7f0ec406bc80 2023-08-25T09:42:16.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.122+0000 7f0ec9883640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec4159e80 con 0x7f0ec4159aa0 2023-08-25T09:42:16.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.122+0000 7f0ec9883640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec406b1a0 con 0x7f0ec415df10 2023-08-25T09:42:16.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ec4159aa0 0x7f0ec4070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ebbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0ec406bc80 0x7f0ec40c9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49344/0 (socket says 172.21.15.70:49344) 2023-08-25T09:42:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ebbfff640 1 -- 172.21.15.70:0/1643211618 learned_addr learned my addr 172.21.15.70:0/1643211618 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0ec4159aa0 0x7f0ec4070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37602/0 (socket says 172.21.15.70:37602) 2023-08-25T09:42:16.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec37fe640 1 --2- 172.21.15.70:0/1643211618 >> v2:172.21.15.155:3300/0 conn(0x7f0ec415df10 0x7f0ec4070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec2ffd640 1 -- 172.21.15.70:0/1643211618 >> v2:172.21.15.155:3300/0 conn(0x7f0ec415df10 msgr2=0x7f0ec4070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec2ffd640 1 --2- 172.21.15.70:0/1643211618 >> v2:172.21.15.155:3300/0 conn(0x7f0ec415df10 0x7f0ec4070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec2ffd640 1 -- 172.21.15.70:0/1643211618 >> v1:172.21.15.70:6789/0 conn(0x7f0ec406bc80 legacy=0x7f0ec40c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec2ffd640 1 -- 172.21.15.70:0/1643211618 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec406b480 con 0x7f0ec4159aa0 2023-08-25T09:42:16.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec27fc640 1 -- 172.21.15.70:0/1643211618 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3295886074 0 0) 0x7f0ec406aec0 con 0x7f0ec406bc80 2023-08-25T09:42:16.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.123+0000 7f0ec37fe640 1 --2- 172.21.15.70:0/1643211618 >> v2:172.21.15.155:3300/0 conn(0x7f0ec415df10 0x7f0ec4070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.124+0000 7f0ec2ffd640 1 --2- 172.21.15.70:0/1643211618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec4159aa0 0x7f0ec4070410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40b4910 tx=0x7f0eb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25a5ea02c8dedf9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.124+0000 7f0ec27fc640 1 -- 172.21.15.70:0/1643211618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4031830 con 0x7f0ec4159aa0 2023-08-25T09:42:16.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec27fc640 1 -- 172.21.15.70:0/1643211618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb4031990 con 0x7f0ec4159aa0 2023-08-25T09:42:16.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec27fc640 1 -- 172.21.15.70:0/1643211618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4031d20 con 0x7f0ec4159aa0 2023-08-25T09:42:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 -- 172.21.15.70:0/1643211618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec4159aa0 msgr2=0x7f0ec4070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 --2- 172.21.15.70:0/1643211618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec4159aa0 0x7f0ec4070410 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40b4910 tx=0x7f0eb402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 -- 172.21.15.70:0/1643211618 shutdown_connections 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 --2- 172.21.15.70:0/1643211618 >> v2:172.21.15.155:3300/0 conn(0x7f0ec415df10 0x7f0ec4070950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 --2- 172.21.15.70:0/1643211618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec4159aa0 0x7f0ec4070410 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 -- 172.21.15.70:0/1643211618 >> 172.21.15.70:0/1643211618 conn(0x7f0ec40ce760 msgr2=0x7f0ec40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 -- 172.21.15.70:0/1643211618 shutdown_connections 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.125+0000 7f0ec9883640 1 -- 172.21.15.70:0/1643211618 wait complete. 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.127+0000 7f0ec9883640 1 Processor -- start 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.127+0000 7f0ec9883640 1 -- start start 2023-08-25T09:42:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.128+0000 7f0ec9883640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec4159aa0 0x7f0ec41f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.128+0000 7f0ec9883640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c7ce0 0x7f0ec40af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.128+0000 7f0ec9883640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 0x7f0ec40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.128+0000 7f0ec9883640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41565f0 con 0x7f0ec40c8160 2023-08-25T09:42:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.128+0000 7f0ec9883640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4155c70 con 0x7f0ec40c7ce0 2023-08-25T09:42:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.128+0000 7f0ec9883640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4155f30 con 0x7f0ec4159aa0 2023-08-25T09:42:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 0x7f0ec40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 0x7f0ec40aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37616/0 (socket says 172.21.15.70:37616) 2023-08-25T09:42:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 -- 172.21.15.70:0/543450725 learned_addr learned my addr 172.21.15.70:0/543450725 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec2ffd640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec4159aa0 0x7f0ec41f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ebbfff640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c7ce0 0x7f0ec40af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 -- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec4159aa0 msgr2=0x7f0ec41f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec4159aa0 0x7f0ec41f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 -- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c7ce0 msgr2=0x7f0ec40af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c7ce0 0x7f0ec40af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.129+0000 7f0ec37fe640 1 -- 172.21.15.70:0/543450725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40093a0 con 0x7f0ec40c8160 2023-08-25T09:42:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.130+0000 7f0ec37fe640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 0x7f0ec40aff20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00cef0 tx=0x7f0ebc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.130+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00d840 con 0x7f0ec40c8160 2023-08-25T09:42:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.130+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc00d9c0 con 0x7f0ec40c8160 2023-08-25T09:42:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.130+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc00dd70 con 0x7f0ec40c8160 2023-08-25T09:42:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.130+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec4156210 con 0x7f0ec40c8160 2023-08-25T09:42:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.130+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec41f2820 con 0x7f0ec40c8160 2023-08-25T09:42:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.131+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec4156730 con 0x7f0ec40c8160 2023-08-25T09:42:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.132+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0ebc01e3b0 con 0x7f0ec40c8160 2023-08-25T09:42:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.133+0000 7f0ec0ff9640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0ea406a080 0x7f0ea406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.133+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0ebc088d00 con 0x7f0ec40c8160 2023-08-25T09:42:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.133+0000 7f0ec2ffd640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0ea406a080 0x7f0ea406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.134+0000 7f0ec2ffd640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0ea406a080 0x7f0ea406c540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4036000 tx=0x7f0eb403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.138+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0ec4156730 con 0x7f0ec40c8160 2023-08-25T09:42:16.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.384+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f0ec4156730 con 0x7f0ec40c8160 2023-08-25T09:42:16.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.524+0000 7f0ec0ff9640 1 -- 172.21.15.70:0/543450725 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v205) v1 ==== 137+0+0 (secure 0 0 0) 0x7f0ec4156730 con 0x7f0ec40c8160 2023-08-25T09:42:16.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.527+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0ea406a080 msgr2=0x7f0ea406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.527+0000 7f0ec9883640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0ea406a080 0x7f0ea406c540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4036000 tx=0x7f0eb403c040 comp rx=0 tx=0).stop 2023-08-25T09:42:16.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.527+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 msgr2=0x7f0ec40aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.527+0000 7f0ec9883640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 0x7f0ec40aff20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00cef0 tx=0x7f0ebc00a710 comp rx=0 tx=0).stop 2023-08-25T09:42:16.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 shutdown_connections 2023-08-25T09:42:16.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0ea406a080 0x7f0ea406c540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0ec40c8160 0x7f0ec40aff20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ec40c7ce0 0x7f0ec40af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 --2- 172.21.15.70:0/543450725 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ec4159aa0 0x7f0ec41f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 >> 172.21.15.70:0/543450725 conn(0x7f0ec40ce760 msgr2=0x7f0ec4154140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:16.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 shutdown_connections 2023-08-25T09:42:16.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.528+0000 7f0ec9883640 1 -- 172.21.15.70:0/543450725 wait complete. 2023-08-25T09:42:16.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-08-25T09:42:16.779 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_objectstore":"memstore"} 2023-08-25T09:42:16.780 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-08-25T09:42:16.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:42:16.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.987+0000 7fa124555640 1 Processor -- start 2023-08-25T09:42:16.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.987+0000 7fa124555640 1 -- start start 2023-08-25T09:42:16.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa124555640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa11c0c2110 0x7fa11c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa124555640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa11c069d90 0x7fa11c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:16.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa124555640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0c2550 con 0x7fa11c15ae50 2023-08-25T09:42:16.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa124555640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c06a200 con 0x7fa11c0c2110 2023-08-25T09:42:16.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa124555640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0ce330 con 0x7fa11c069d90 2023-08-25T09:42:16.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa121af0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa11c0c2110 0x7fa11c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa121af0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa11c0c2110 0x7fa11c069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37620/0 (socket says 172.21.15.70:37620) 2023-08-25T09:42:16.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa121af0640 1 -- 172.21.15.70:0/2181060771 learned_addr learned my addr 172.21.15.70:0/2181060771 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.989+0000 7fa1222f1640 1 --1- 172.21.15.70:0/2181060771 >> v1:172.21.15.70:6789/0 conn(0x7fa11c15ae50 0x7fa11c0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49352/0 (socket says 172.21.15.70:49352) 2023-08-25T09:42:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa122af2640 1 --2- 172.21.15.70:0/2181060771 >> v2:172.21.15.155:3300/0 conn(0x7fa11c069d90 0x7fa11c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:16.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa121af0640 1 -- 172.21.15.70:0/2181060771 >> v2:172.21.15.155:3300/0 conn(0x7fa11c069d90 msgr2=0x7fa11c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa121af0640 1 --2- 172.21.15.70:0/2181060771 >> v2:172.21.15.155:3300/0 conn(0x7fa11c069d90 0x7fa11c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa121af0640 1 -- 172.21.15.70:0/2181060771 >> v1:172.21.15.70:6789/0 conn(0x7fa11c15ae50 legacy=0x7fa11c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa121af0640 1 -- 172.21.15.70:0/2181060771 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0cc5f0 con 0x7fa11c0c2110 2023-08-25T09:42:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa122af2640 1 --2- 172.21.15.70:0/2181060771 >> v2:172.21.15.155:3300/0 conn(0x7fa11c069d90 0x7fa11c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.990+0000 7fa1212ef640 1 -- 172.21.15.70:0/2181060771 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75074837 0 0) 0x7fa11c0c2550 con 0x7fa11c15ae50 2023-08-25T09:42:17.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.991+0000 7fa121af0640 1 --2- 172.21.15.70:0/2181060771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c0c2110 0x7fa11c069850 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0b00d0 tx=0x7fa10c02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3fe11c24a90765 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:17.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.991+0000 7fa1212ef640 1 -- 172.21.15.70:0/2181060771 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c02f850 con 0x7fa11c0c2110 2023-08-25T09:42:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.991+0000 7fa1212ef640 1 -- 172.21.15.70:0/2181060771 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c02f9b0 con 0x7fa11c0c2110 2023-08-25T09:42:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.991+0000 7fa1212ef640 1 -- 172.21.15.70:0/2181060771 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c02fd40 con 0x7fa11c0c2110 2023-08-25T09:42:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.991+0000 7fa124555640 1 -- 172.21.15.70:0/2181060771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c0c2110 msgr2=0x7fa11c069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.991+0000 7fa124555640 1 --2- 172.21.15.70:0/2181060771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c0c2110 0x7fa11c069850 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0b00d0 tx=0x7fa10c02ba30 comp rx=0 tx=0).stop 2023-08-25T09:42:17.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.992+0000 7fa124555640 1 -- 172.21.15.70:0/2181060771 shutdown_connections 2023-08-25T09:42:17.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.992+0000 7fa124555640 1 --2- 172.21.15.70:0/2181060771 >> v2:172.21.15.155:3300/0 conn(0x7fa11c069d90 0x7fa11c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.992+0000 7fa124555640 1 --2- 172.21.15.70:0/2181060771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c0c2110 0x7fa11c069850 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.992+0000 7fa124555640 1 -- 172.21.15.70:0/2181060771 >> 172.21.15.70:0/2181060771 conn(0x7fa11c0cb0d0 msgr2=0x7fa11c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.992+0000 7fa124555640 1 -- 172.21.15.70:0/2181060771 shutdown_connections 2023-08-25T09:42:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.992+0000 7fa124555640 1 -- 172.21.15.70:0/2181060771 wait complete. 2023-08-25T09:42:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.993+0000 7fa124555640 1 Processor -- start 2023-08-25T09:42:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.993+0000 7fa124555640 1 -- start start 2023-08-25T09:42:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.994+0000 7fa124555640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa11c069d90 0x7fa11c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.994+0000 7fa124555640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa11c0c2110 0x7fa11c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.994+0000 7fa124555640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 0x7fa11c0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.994+0000 7fa124555640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c57c0 con 0x7fa11c15ae50 2023-08-25T09:42:17.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.994+0000 7fa124555640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c5aa0 con 0x7fa11c0c2110 2023-08-25T09:42:17.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.994+0000 7fa124555640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c5d80 con 0x7fa11c069d90 2023-08-25T09:42:17.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 0x7fa11c0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 0x7fa11c0b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37626/0 (socket says 172.21.15.70:37626) 2023-08-25T09:42:17.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 -- 172.21.15.70:0/139093917 learned_addr learned my addr 172.21.15.70:0/139093917 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:17.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 -- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa11c069d90 msgr2=0x7fa11c0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa1222f1640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa11c069d90 0x7fa11c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa121af0640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa11c0c2110 0x7fa11c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa11c069d90 0x7fa11c0ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 -- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa11c0c2110 msgr2=0x7fa11c0acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa11c0c2110 0x7fa11c0acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 -- 172.21.15.70:0/139093917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0094d0 con 0x7fa11c15ae50 2023-08-25T09:42:17.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa1222f1640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa11c069d90 0x7fa11c0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.995+0000 7fa122af2640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 0x7fa11c0b1e30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa11400cef0 tx=0x7fa11400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.996+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400d840 con 0x7fa11c15ae50 2023-08-25T09:42:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.996+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11400d9c0 con 0x7fa11c15ae50 2023-08-25T09:42:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.996+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11400dd70 con 0x7fa11c15ae50 2023-08-25T09:42:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.996+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c0c6060 con 0x7fa11c15ae50 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.996+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c0c6830 con 0x7fa11c15ae50 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.997+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c0cba30 con 0x7fa11c15ae50 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.998+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa11401e3b0 con 0x7fa11c15ae50 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.999+0000 7fa112ffd640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa10006a050 0x7fa10006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.999+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa114088d80 con 0x7fa11c15ae50 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.999+0000 7fa1222f1640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa10006a050 0x7fa10006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:16.999+0000 7fa1222f1640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa10006a050 0x7fa10006c510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa118000f60 tx=0x7fa118009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.003+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa114054840 con 0x7fa11c15ae50 2023-08-25T09:42:17.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.173+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa11c0c6580 con 0x7fa11c15ae50 2023-08-25T09:42:17.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.178+0000 7fa112ffd640 1 -- 172.21.15.70:0/139093917 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa11c0c6580 con 0x7fa11c15ae50 2023-08-25T09:42:17.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.180+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa10006a050 msgr2=0x7fa10006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.180+0000 7fa124555640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa10006a050 0x7fa10006c510 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa118000f60 tx=0x7fa118009040 comp rx=0 tx=0).stop 2023-08-25T09:42:17.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 msgr2=0x7fa11c0b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 0x7fa11c0b1e30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa11400cef0 tx=0x7fa11400a710 comp rx=0 tx=0).stop 2023-08-25T09:42:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 shutdown_connections 2023-08-25T09:42:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa10006a050 0x7fa10006c510 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa11c15ae50 0x7fa11c0b1e30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa11c0c2110 0x7fa11c0acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 --2- 172.21.15.70:0/139093917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa11c069d90 0x7fa11c0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 >> 172.21.15.70:0/139093917 conn(0x7fa11c0cb0d0 msgr2=0x7fa11c0c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 shutdown_connections 2023-08-25T09:42:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.181+0000 7fa124555640 1 -- 172.21.15.70:0/139093917 wait complete. 2023-08-25T09:42:17.201 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:42:17.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:42:17.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:42:17.451 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:42:17.452 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:42:17.452 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:17.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:42:17.516 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:42:17.516 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:17.580 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:42:17.581 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:42:17.582 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:17.643 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:42:17.644 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:42:17.644 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:42:17.704 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:42:17.705 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:42:17.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:17.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.924+0000 7f1675ff3640 1 Processor -- start 2023-08-25T09:42:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.924+0000 7f1675ff3640 1 -- start start 2023-08-25T09:42:17.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f1675ff3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16700cea30 0x7f16700b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f1675ff3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167006f050 con 0x7f167015cb40 2023-08-25T09:42:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f1675ff3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167006f330 con 0x7f16700ac870 2023-08-25T09:42:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f1675ff3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f167006f610 con 0x7f16700cea30 2023-08-25T09:42:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f166effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16700cea30 0x7f16700b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f166effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f16700cea30 0x7f16700b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37640/0 (socket says 172.21.15.70:37640) 2023-08-25T09:42:17.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.926+0000 7f166effd640 1 -- 172.21.15.70:0/3043570955 learned_addr learned my addr 172.21.15.70:0/3043570955 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.927+0000 7f166effd640 1 -- 172.21.15.70:0/3043570955 >> v1:172.21.15.155:6789/0 conn(0x7f16700ac870 legacy=0x7f16700b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.927+0000 7f166effd640 1 -- 172.21.15.70:0/3043570955 >> v1:172.21.15.70:6789/0 conn(0x7f167015cb40 legacy=0x7f16700ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.927+0000 7f166effd640 1 -- 172.21.15.70:0/3043570955 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167006f8f0 con 0x7f16700cea30 2023-08-25T09:42:17.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.928+0000 7f166effd640 1 --2- 172.21.15.70:0/3043570955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700cea30 0x7f16700b1b30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f166402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b87cf92b47c0073 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:17.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.928+0000 7f166e7fc640 1 -- 172.21.15.70:0/3043570955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664030860 con 0x7f16700cea30 2023-08-25T09:42:17.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.928+0000 7f166e7fc640 1 -- 172.21.15.70:0/3043570955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16640309c0 con 0x7f16700cea30 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.928+0000 7f166e7fc640 1 -- 172.21.15.70:0/3043570955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664030d50 con 0x7f16700cea30 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 -- 172.21.15.70:0/3043570955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700cea30 msgr2=0x7f16700b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3043570955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700cea30 0x7f16700b1b30 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f1664009870 tx=0x7f166402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 -- 172.21.15.70:0/3043570955 shutdown_connections 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3043570955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700cea30 0x7f16700b1b30 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 -- 172.21.15.70:0/3043570955 >> 172.21.15.70:0/3043570955 conn(0x7f1670069850 msgr2=0x7f1670069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:17.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 -- 172.21.15.70:0/3043570955 shutdown_connections 2023-08-25T09:42:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.929+0000 7f1675ff3640 1 -- 172.21.15.70:0/3043570955 wait complete. 2023-08-25T09:42:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.931+0000 7f1675ff3640 1 Processor -- start 2023-08-25T09:42:17.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.931+0000 7f1675ff3640 1 -- start start 2023-08-25T09:42:17.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f1675ff3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 0x7f16701f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f1675ff3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16700cea30 0x7f1670151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f1675ff3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f167015cb40 0x7f16701f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f1675ff3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1670144990 con 0x7f16700ac870 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f1675ff3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167014c8d0 con 0x7f16700cea30 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f1675ff3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167014cbb0 con 0x7f167015cb40 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 0x7f16701f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 0x7f16701f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37642/0 (socket says 172.21.15.70:37642) 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 -- 172.21.15.70:0/3789366829 learned_addr learned my addr 172.21.15.70:0/3789366829 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166effd640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16700cea30 0x7f1670151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 -- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f167015cb40 msgr2=0x7f16701f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166ffff640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f167015cb40 0x7f16701f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f167015cb40 0x7f16701f5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 -- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16700cea30 msgr2=0x7f1670151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16700cea30 0x7f1670151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.932+0000 7f166f7fe640 1 -- 172.21.15.70:0/3789366829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16640094d0 con 0x7f16700ac870 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f166effd640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16700cea30 0x7f1670151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:17.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f166f7fe640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 0x7f16701f1f70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f166000c400 tx=0x7f166000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16600052d0 con 0x7f16700ac870 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660005430 con 0x7f16700ac870 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660003830 con 0x7f16700ac870 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167014ce90 con 0x7f16700ac870 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.933+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167014d450 con 0x7f16700ac870 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.934+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167004e420 con 0x7f16700ac870 2023-08-25T09:42:17.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.935+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1660003990 con 0x7f16700ac870 2023-08-25T09:42:17.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.936+0000 7f1657fff640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f165c06a050 0x7f165c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:17.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.936+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f16600878a0 con 0x7f16700ac870 2023-08-25T09:42:17.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.936+0000 7f166effd640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f165c06a050 0x7f165c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:17.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.937+0000 7f166effd640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f165c06a050 0x7f165c06c510 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1664036000 tx=0x7f166403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:17.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:17.941+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1660053210 con 0x7f16700ac870 2023-08-25T09:42:18.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.171+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16700b1510 con 0x7f16700ac870 2023-08-25T09:42:18.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.172+0000 7f1657fff640 1 -- 172.21.15.70:0/3789366829 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f1660017e80 con 0x7f16700ac870 2023-08-25T09:42:18.175 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:18.175 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4742,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/4291194833","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":4291194833},{"type":"v1","addr":"172.21.15.155:6835","nonce":4291194833}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":4993,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/396887680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":396887680},{"type":"v1","addr":"172.21.15.155:6833","nonce":396887680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1528766267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1528766267},{"type":"v1","addr":"172.21.15.70:6836","nonce":1528766267}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":6231,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/17341203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":17341203},{"type":"v1","addr":"172.21.15.70:6837","nonce":17341203}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[]} 2023-08-25T09:42:18.175 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 26 2023-08-25T09:42:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f165c06a050 msgr2=0x7f165c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f165c06a050 0x7f165c06c510 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1664036000 tx=0x7f166403b040 comp rx=0 tx=0).stop 2023-08-25T09:42:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 msgr2=0x7f16701f1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 0x7f16701f1f70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f166000c400 tx=0x7f166000c8c0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 shutdown_connections 2023-08-25T09:42:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f165c06a050 0x7f165c06c510 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f167015cb40 0x7f16701f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16700cea30 0x7f1670151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 --2- 172.21.15.70:0/3789366829 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16700ac870 0x7f16701f1f70 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.175+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 >> 172.21.15.70:0/3789366829 conn(0x7f1670069850 msgr2=0x7f1670158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:18.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.176+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 shutdown_connections 2023-08-25T09:42:18.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.176+0000 7f1675ff3640 1 -- 172.21.15.70:0/3789366829 wait complete. 2023-08-25T09:42:18.201 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:42:18.201 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:42:18.201 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:42:18.202 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:42:18.202 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:42:18.202 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:42:18.202 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:42:18.203 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:42:18.203 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:42:18.203 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:42:18.203 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:42:18.204 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:42:18.204 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:42:18.204 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:42:18.204 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:42:18.205 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:42:18.222 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:42:18.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:42:18.223 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:42:18.224 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:42:18.225 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:42:18.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:42:18.232 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:42:18.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:42:18.246 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:42:18.246 DEBUG:teuthology.parallel:result is None 2023-08-25T09:42:18.251 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:42:18.252 DEBUG:teuthology.parallel:result is None 2023-08-25T09:42:18.291 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:42:18.292 DEBUG:teuthology.parallel:result is None 2023-08-25T09:42:18.299 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:42:18.299 DEBUG:teuthology.parallel:result is None 2023-08-25T09:42:18.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:42:18.400 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:42:18.419 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:42:18.456 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:42:18.457 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:42:18.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.492+0000 7f5eb6057640 1 Processor -- start 2023-08-25T09:42:18.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.492+0000 7f5eb6057640 1 -- start start 2023-08-25T09:42:18.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.493+0000 7f5eb6057640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5eb00ca780 0x7f5eb00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.493+0000 7f5eb6057640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5eb00b3cf0 0x7f5eb00b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.493+0000 7f5eb6057640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb00b12b0 con 0x7f5eb015acd0 2023-08-25T09:42:18.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.493+0000 7f5eb6057640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb006c0e0 con 0x7f5eb00ca780 2023-08-25T09:42:18.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.493+0000 7f5eb6057640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb006ae20 con 0x7f5eb00b3cf0 2023-08-25T09:42:18.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5eaf7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5eb015acd0 0x7f5eb015b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49366/0 (socket says 172.21.15.70:49366) 2023-08-25T09:42:18.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5eaf7fe640 1 -- 172.21.15.70:0/605762468 learned_addr learned my addr 172.21.15.70:0/605762468 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:18.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5ea6ffd640 1 --2- 172.21.15.70:0/605762468 >> v2:172.21.15.70:3300/0 conn(0x7f5eb00ca780 0x7f5eb00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:18.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5eaffff640 1 --2- 172.21.15.70:0/605762468 >> v2:172.21.15.155:3300/0 conn(0x7f5eb00b3cf0 0x7f5eb00b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:18.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/605762468 >> v2:172.21.15.155:3300/0 conn(0x7f5eb00b3cf0 msgr2=0x7f5eb00b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:18.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5ea6ffd640 1 --2- 172.21.15.70:0/605762468 >> v2:172.21.15.155:3300/0 conn(0x7f5eb00b3cf0 0x7f5eb00b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/605762468 >> v1:172.21.15.70:6789/0 conn(0x7f5eb015acd0 legacy=0x7f5eb015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:18.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/605762468 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb015b7c0 con 0x7f5eb00ca780 2023-08-25T09:42:18.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.494+0000 7f5eaeffd640 1 -- 172.21.15.70:0/605762468 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091248342 0 0) 0x7f5eb00b12b0 con 0x7f5eb015acd0 2023-08-25T09:42:18.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.495+0000 7f5ea6ffd640 1 --2- 172.21.15.70:0/605762468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cac30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00c8d30 tx=0x7f5e9802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e79fe0d7c451bc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:18.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eaeffd640 1 -- 172.21.15.70:0/605762468 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9802e070 con 0x7f5eb00ca780 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eaeffd640 1 -- 172.21.15.70:0/605762468 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e980315a0 con 0x7f5eb00ca780 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eaeffd640 1 -- 172.21.15.70:0/605762468 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98031930 con 0x7f5eb00ca780 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eb6057640 1 -- 172.21.15.70:0/605762468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 msgr2=0x7f5eb00cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eb6057640 1 --2- 172.21.15.70:0/605762468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cac30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00c8d30 tx=0x7f5e9802b840 comp rx=0 tx=0).stop 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eb6057640 1 -- 172.21.15.70:0/605762468 shutdown_connections 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eb6057640 1 --2- 172.21.15.70:0/605762468 >> v2:172.21.15.155:3300/0 conn(0x7f5eb00b3cf0 0x7f5eb00b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eb6057640 1 --2- 172.21.15.70:0/605762468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cac30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.496+0000 7f5eb6057640 1 -- 172.21.15.70:0/605762468 >> 172.21.15.70:0/605762468 conn(0x7f5eb00ce760 msgr2=0x7f5eb00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.497+0000 7f5eb6057640 1 -- 172.21.15.70:0/605762468 shutdown_connections 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.497+0000 7f5eb6057640 1 -- 172.21.15.70:0/605762468 wait complete. 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.498+0000 7f5eb6057640 1 Processor -- start 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.498+0000 7f5eb6057640 1 -- start start 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.499+0000 7f5eb6057640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb00b3cf0 0x7f5eb0142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.499+0000 7f5eb6057640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:18.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.499+0000 7f5eb6057640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb015acd0 0x7f5eb00c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.499+0000 7f5eb6057640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cbf80 con 0x7f5eb00ca780 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.499+0000 7f5eb6057640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cb110 con 0x7f5eb015acd0 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.499+0000 7f5eb6057640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00cb370 con 0x7f5eb00b3cf0 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37678/0 (socket says 172.21.15.70:37678) 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/1456774473 learned_addr learned my addr 172.21.15.70:0/1456774473 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:18.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5eaf7fe640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb00b3cf0 0x7f5eb0142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5eaffff640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb015acd0 0x7f5eb00c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb00b3cf0 msgr2=0x7f5eb0142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb00b3cf0 0x7f5eb0142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb015acd0 msgr2=0x7f5eb00c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb015acd0 0x7f5eb00c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 -- 172.21.15.70:0/1456774473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009400 con 0x7f5eb00ca780 2023-08-25T09:42:18.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5eaf7fe640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb00b3cf0 0x7f5eb0142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5eaffff640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb015acd0 0x7f5eb00c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.500+0000 7f5ea6ffd640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cabd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5e98000c00 tx=0x7f5e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.501+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98032840 con 0x7f5eb00ca780 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.501+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e980329c0 con 0x7f5eb00ca780 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.501+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb00cb650 con 0x7f5eb00ca780 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.501+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98032d70 con 0x7f5eb00ca780 2023-08-25T09:42:18.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.501+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb00cbc10 con 0x7f5eb00ca780 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.502+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb004e420 con 0x7f5eb00ca780 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.503+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e9804d3f0 con 0x7f5eb00ca780 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.504+0000 7f5eacff9640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e9006a050 0x7f5e9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.504+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5e980a9de0 con 0x7f5eb00ca780 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.504+0000 7f5eaf7fe640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e9006a050 0x7f5e9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.505+0000 7f5eaf7fe640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e9006a050 0x7f5e9006c510 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00016b0 tx=0x7f5ea0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:18.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.508+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e980757d0 con 0x7f5eb00ca780 2023-08-25T09:42:18.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:18.716+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5eb006bec0 con 0x7f5eb00ca780 2023-08-25T09:42:19.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.565+0000 7f5eacff9640 1 -- 172.21.15.70:0/1456774473 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5e98078df0 con 0x7f5eb00ca780 2023-08-25T09:42:19.568 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:42:19.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e9006a050 msgr2=0x7f5e9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:19.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e9006a050 0x7f5e9006c510 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00016b0 tx=0x7f5ea0009040 comp rx=0 tx=0).stop 2023-08-25T09:42:19.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 msgr2=0x7f5eb00cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:19.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cabd0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5e98000c00 tx=0x7f5e980046c0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 shutdown_connections 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e9006a050 0x7f5e9006c510 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5eb015acd0 0x7f5eb00c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5eb00ca780 0x7f5eb00cabd0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 --2- 172.21.15.70:0/1456774473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5eb00b3cf0 0x7f5eb0142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.568+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 >> 172.21.15.70:0/1456774473 conn(0x7f5eb00ce760 msgr2=0x7f5eb00c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.569+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 shutdown_connections 2023-08-25T09:42:19.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.569+0000 7f5eb6057640 1 -- 172.21.15.70:0/1456774473 wait complete. 2023-08-25T09:42:19.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:42:19.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.813+0000 7facc68f3640 1 Processor -- start 2023-08-25T09:42:19.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.813+0000 7facc68f3640 1 -- start start 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc68f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7facc00b12f0 0x7facc015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc68f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc015f190 0x7facc00cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc68f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc015e7c0 con 0x7facc015a7c0 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc68f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc0071b50 con 0x7facc015f190 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc68f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc015e490 con 0x7facc00b12f0 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc60f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc015f190 0x7facc00cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:19.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc60f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7facc015f190 0x7facc00cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50394/0 (socket says 172.21.15.70:50394) 2023-08-25T09:42:19.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.815+0000 7facc60f2640 1 -- 172.21.15.70:0/2652938497 learned_addr learned my addr 172.21.15.70:0/2652938497 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:19.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.816+0000 7facc50f0640 1 --2- 172.21.15.70:0/2652938497 >> v2:172.21.15.155:3300/0 conn(0x7facc00b12f0 0x7facc015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:19.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.816+0000 7facc60f2640 1 -- 172.21.15.70:0/2652938497 >> v2:172.21.15.155:3300/0 conn(0x7facc00b12f0 msgr2=0x7facc015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:19.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.816+0000 7facc60f2640 1 --2- 172.21.15.70:0/2652938497 >> v2:172.21.15.155:3300/0 conn(0x7facc00b12f0 0x7facc015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.816+0000 7facc60f2640 1 -- 172.21.15.70:0/2652938497 >> v1:172.21.15.155:6789/0 conn(0x7facc015a7c0 legacy=0x7facc00b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:19.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.816+0000 7facc60f2640 1 -- 172.21.15.70:0/2652938497 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc00b3d60 con 0x7facc015f190 2023-08-25T09:42:19.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.817+0000 7facc60f2640 1 --2- 172.21.15.70:0/2652938497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc00cad20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7facc00b0c50 tx=0x7facb802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a989461c4091532f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:19.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc48ef640 1 -- 172.21.15.70:0/2652938497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb802ca80 con 0x7facc015f190 2023-08-25T09:42:19.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc48ef640 1 -- 172.21.15.70:0/2652938497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb802cbe0 con 0x7facc015f190 2023-08-25T09:42:19.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc48ef640 1 -- 172.21.15.70:0/2652938497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80343f0 con 0x7facc015f190 2023-08-25T09:42:19.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 -- 172.21.15.70:0/2652938497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 msgr2=0x7facc00cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:19.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 --2- 172.21.15.70:0/2652938497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc00cad20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7facc00b0c50 tx=0x7facb802bee0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 -- 172.21.15.70:0/2652938497 shutdown_connections 2023-08-25T09:42:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 --2- 172.21.15.70:0/2652938497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc00cad20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 --2- 172.21.15.70:0/2652938497 >> v2:172.21.15.155:3300/0 conn(0x7facc00b12f0 0x7facc015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 -- 172.21.15.70:0/2652938497 >> 172.21.15.70:0/2652938497 conn(0x7facc00affd0 msgr2=0x7facc00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 -- 172.21.15.70:0/2652938497 shutdown_connections 2023-08-25T09:42:19.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.818+0000 7facc68f3640 1 -- 172.21.15.70:0/2652938497 wait complete. 2023-08-25T09:42:19.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.820+0000 7facc68f3640 1 Processor -- start 2023-08-25T09:42:19.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.820+0000 7facc68f3640 1 -- start start 2023-08-25T09:42:19.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc68f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc00b12f0 0x7facc00703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:19.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc68f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc015a7c0 0x7facc00c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:19.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc68f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc01f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:19.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc68f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00c5630 con 0x7facc015f190 2023-08-25T09:42:19.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc68f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc00709b0 con 0x7facc015a7c0 2023-08-25T09:42:19.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc68f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc0070c90 con 0x7facc00b12f0 2023-08-25T09:42:19.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.821+0000 7facc60f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc01f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:19.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc01f38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50408/0 (socket says 172.21.15.70:50408) 2023-08-25T09:42:19.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 -- 172.21.15.70:0/3173194096 learned_addr learned my addr 172.21.15.70:0/3173194096 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:19.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc50f0640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc015a7c0 0x7facc00c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc58f1640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc00b12f0 0x7facc00703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 -- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc00b12f0 msgr2=0x7facc00703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc00b12f0 0x7facc00703d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 -- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc015a7c0 msgr2=0x7facc00c2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc015a7c0 0x7facc00c2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 -- 172.21.15.70:0/3173194096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8009b30 con 0x7facc015f190 2023-08-25T09:42:19.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc50f0640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc015a7c0 0x7facc00c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.822+0000 7facc60f2640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc01f38b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7facb8036040 tx=0x7facb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.823+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8009510 con 0x7facc015f190 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.823+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb803e450 con 0x7facc015f190 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.823+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb803e7e0 con 0x7facc015f190 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.823+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc0070f70 con 0x7facc015f190 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.823+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc015dd00 con 0x7facc015f190 2023-08-25T09:42:19.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.824+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc00c57e0 con 0x7facc015f190 2023-08-25T09:42:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.825+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7facb802cd50 con 0x7facc015f190 2023-08-25T09:42:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.826+0000 7facb67fc640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faca406a050 0x7faca406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.826+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5210+0+0 (secure 0 0 0) 0x7facb80aad50 con 0x7facc015f190 2023-08-25T09:42:19.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.826+0000 7facc58f1640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faca406a050 0x7faca406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:19.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.827+0000 7facc58f1640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faca406a050 0x7faca406c510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7facbc0015d0 tx=0x7facbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:19.830+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7facc00c57e0 con 0x7facc015f190 2023-08-25T09:42:20.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.081+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7facc00b42f0 con 0x7facc015f190 2023-08-25T09:42:20.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.082+0000 7facb67fc640 1 -- 172.21.15.70:0/3173194096 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7facb8043020 con 0x7facc015f190 2023-08-25T09:42:20.085 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:20.085 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:42:20.086 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:42:20.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.086 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:42:20.086 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:42:20.087 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.087 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.087 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:42:20.088 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:42:20.088 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.088 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:42:20.090 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:42:20.091 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.092 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.092 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:42:20.092 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:42:20.092 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.093 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.094 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:42:20.094 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:42:20.095 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.095 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.095 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.096 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.097 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:42:20.097 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:42:20.097 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.098 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.098 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.098 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.099 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.099 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:42:20.100 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:42:20.101 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.101 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.101 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.101 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.102 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:42:20.103 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:42:20.103 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.103 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.104 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.104 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.104 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.105 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.106 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.107 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.108 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:42:20.108 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:42:20.108 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.109 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.109 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.109 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.109 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.110 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:42:20.110 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:42:20.111 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.111 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:42:20.111 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:42:20.112 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:42:20.113 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.114 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:42:20.114 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:42:20.114 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.116 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.116 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:42:20.116 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:42:20.116 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:42:20.116 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.117 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:42:20.118 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:42:20.119 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.120 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:42:20.120 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:42:20.120 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:42:20.120 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:42:20.120 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.121 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:42:20.121 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:42:20.121 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.122 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:42:20.122 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:42:20.122 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:42:20.123 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.123 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.124 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:42:20.124 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:42:20.124 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.125 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:42:20.125 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:42:20.125 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:20.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:20.127 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:42:20.127 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:42:20.127 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:42:20.127 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:42:20.128 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:42:20.128 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:42:20.128 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.129 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:20.129 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:42:20.129 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:42:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faca406a050 msgr2=0x7faca406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faca406a050 0x7faca406c510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7facbc0015d0 tx=0x7facbc009040 comp rx=0 tx=0).stop 2023-08-25T09:42:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 msgr2=0x7facc01f38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc01f38b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7facb8036040 tx=0x7facb8004a40 comp rx=0 tx=0).stop 2023-08-25T09:42:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 shutdown_connections 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faca406a050 0x7faca406c510 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facc015f190 0x7facc01f38b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facc015a7c0 0x7facc00c2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 --2- 172.21.15.70:0/3173194096 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facc00b12f0 0x7facc00703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 >> 172.21.15.70:0/3173194096 conn(0x7facc00affd0 msgr2=0x7facc00ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 shutdown_connections 2023-08-25T09:42:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.085+0000 7facc68f3640 1 -- 172.21.15.70:0/3173194096 wait complete. 2023-08-25T09:42:20.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.354+0000 7f08f1808640 1 Processor -- start 2023-08-25T09:42:20.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.354+0000 7f08f1808640 1 -- start start 2023-08-25T09:42:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.355+0000 7f08f1808640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 0x7f08ec0c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.355+0000 7f08f1808640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f08ec0cd5c0 0x7f08ec0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.355+0000 7f08f1808640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec160680 con 0x7f08ec15ca00 2023-08-25T09:42:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.355+0000 7f08f1808640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec160960 con 0x7f08ec0cd5c0 2023-08-25T09:42:20.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.355+0000 7f08f1808640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec160c40 con 0x7f08ec0c5120 2023-08-25T09:42:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08f0806640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f08ec15ca00 0x7f08ec0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39650/0 (socket says 172.21.15.70:39650) 2023-08-25T09:42:20.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08f0806640 1 -- 172.21.15.70:0/3585177063 learned_addr learned my addr 172.21.15.70:0/3585177063 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08f1007640 1 --2- 172.21.15.70:0/3585177063 >> v2:172.21.15.155:3300/0 conn(0x7f08ec0cd5c0 0x7f08ec0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478016872 0 0) 0x7f08ec160680 con 0x7f08ec15ca00 2023-08-25T09:42:20.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08dc003520 con 0x7f08ec15ca00 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08ebfff640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 0x7f08ec0c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.356+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2733761420 0 0) 0x7f08dc003520 con 0x7f08ec15ca00 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 msgr2=0x7f08ec0c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 0x7f08ec0c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 >> v2:172.21.15.155:3300/0 conn(0x7f08ec0cd5c0 msgr2=0x7f08ec0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 --2- 172.21.15.70:0/3585177063 >> v2:172.21.15.155:3300/0 conn(0x7f08ec0cd5c0 0x7f08ec0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec160f20 con 0x7f08ec15ca00 2023-08-25T09:42:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f08e0001670 con 0x7f08ec15ca00 2023-08-25T09:42:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 >> v1:172.21.15.70:6789/0 conn(0x7f08ec15ca00 legacy=0x7f08ec0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 0x7f08dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec160960 con 0x7f08dc004d00 2023-08-25T09:42:20.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec160c40 con 0x7f08dc003970 2023-08-25T09:42:20.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc00ba30 con 0x7f08ec0c5120 2023-08-25T09:42:20.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f1007640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08ebfff640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 0x7f08dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f0806640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f1007640 1 -- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 msgr2=0x7f08dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f1007640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c5120 0x7f08dc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f1007640 1 -- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc003970 msgr2=0x7f08dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f1007640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08dc003970 0x7f08dc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.357+0000 7f08f1007640 1 -- 172.21.15.70:0/3585177063 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc003520 con 0x7f08dc004d00 2023-08-25T09:42:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.358+0000 7f08f1007640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f08e4007360 tx=0x7f08dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4559562842eaad4b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e402ccb0 con 0x7f08dc004d00 2023-08-25T09:42:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e402ce10 con 0x7f08dc004d00 2023-08-25T09:42:20.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08eb7fe640 1 -- 172.21.15.70:0/3585177063 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4034660 con 0x7f08dc004d00 2023-08-25T09:42:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1808640 1 -- 172.21.15.70:0/3585177063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 msgr2=0x7f08dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1808640 1 --2- 172.21.15.70:0/3585177063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08dc005180 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f08e4007360 tx=0x7f08dc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1007640 1 -- 172.21.15.70:0/3585177063 reap_dead start 2023-08-25T09:42:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1808640 1 -- 172.21.15.70:0/3585177063 shutdown_connections 2023-08-25T09:42:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1808640 1 -- 172.21.15.70:0/3585177063 >> 172.21.15.70:0/3585177063 conn(0x7f08ec0affd0 msgr2=0x7f08ec0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1808640 1 -- 172.21.15.70:0/3585177063 shutdown_connections 2023-08-25T09:42:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.359+0000 7f08f1808640 1 -- 172.21.15.70:0/3585177063 wait complete. 2023-08-25T09:42:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.361+0000 7f08f1808640 1 Processor -- start 2023-08-25T09:42:20.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.361+0000 7f08f1808640 1 -- start start 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f1808640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08ec0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f1808640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ec0c55e0 0x7f08ec15fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f1808640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c1fd0 0x7f08ec0c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f1808640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0c69a0 con 0x7f08dc004d00 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f1808640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec1f5230 con 0x7f08ec0c55e0 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f1808640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15fff0 con 0x7f08ec0c1fd0 2023-08-25T09:42:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f0806640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08ec0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f0806640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08ec0c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50432/0 (socket says 172.21.15.70:50432) 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.362+0000 7f08f0806640 1 -- 172.21.15.70:0/3751855954 learned_addr learned my addr 172.21.15.70:0/3751855954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f0806640 1 -- 172.21.15.70:0/3751855954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c1fd0 msgr2=0x7f08ec0c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f0806640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c1fd0 0x7f08ec0c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f0806640 1 -- 172.21.15.70:0/3751855954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ec0c55e0 msgr2=0x7f08ec15fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f0806640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ec0c55e0 0x7f08ec15fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f0806640 1 -- 172.21.15.70:0/3751855954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec15ea10 con 0x7f08dc004d00 2023-08-25T09:42:20.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f0806640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08ec0c6460 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f08e000d8b0 tx=0x7f08e000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e00074e0 con 0x7f08dc004d00 2023-08-25T09:42:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e0007640 con 0x7f08dc004d00 2023-08-25T09:42:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e0004a00 con 0x7f08dc004d00 2023-08-25T09:42:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.363+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec15ecf0 con 0x7f08dc004d00 2023-08-25T09:42:20.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.364+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec15f2b0 con 0x7f08dc004d00 2023-08-25T09:42:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.365+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec066fb0 con 0x7f08dc004d00 2023-08-25T09:42:20.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.366+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08e0004b60 con 0x7f08dc004d00 2023-08-25T09:42:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.366+0000 7f08e97fa640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08d006a080 0x7f08d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.367+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f08e0087c60 con 0x7f08dc004d00 2023-08-25T09:42:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.367+0000 7f08ebfff640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08d006a080 0x7f08d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.367+0000 7f08ebfff640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08d006a080 0x7f08d006c540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006a90 tx=0x7f08d8006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:20.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.371+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08e0057150 con 0x7f08dc004d00 2023-08-25T09:42:20.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.562+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08ec0cb220 con 0x7f08dc004d00 2023-08-25T09:42:20.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.563+0000 7f08e97fa640 1 -- 172.21.15.70:0/3751855954 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+14653 (secure 0 0 0) 0x7f08e0019090 con 0x7f08dc004d00 2023-08-25T09:42:20.567 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:20.567 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":69,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:19.558417+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:20.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08d006a080 msgr2=0x7f08d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08d006a080 0x7f08d006c540 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f08d8006a90 tx=0x7f08d8006a20 comp rx=0 tx=0).stop 2023-08-25T09:42:20.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 msgr2=0x7f08ec0c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08ec0c6460 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f08e000d8b0 tx=0x7f08e000dd70 comp rx=0 tx=0).stop 2023-08-25T09:42:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 shutdown_connections 2023-08-25T09:42:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08d006a080 0x7f08d006c540 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ec0c1fd0 0x7f08ec0c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ec0c55e0 0x7f08ec15fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 --2- 172.21.15.70:0/3751855954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08dc004d00 0x7f08ec0c6460 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 >> 172.21.15.70:0/3751855954 conn(0x7f08ec0affd0 msgr2=0x7f08ec0aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:20.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.566+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 shutdown_connections 2023-08-25T09:42:20.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.567+0000 7f08f1808640 1 -- 172.21.15.70:0/3751855954 wait complete. 2023-08-25T09:42:20.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:42:20.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:42:20.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.792+0000 7f9cf11b5640 1 Processor -- start 2023-08-25T09:42:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.792+0000 7f9cf11b5640 1 -- start start 2023-08-25T09:42:20.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cf11b5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9cec143330 0x7f9cec0b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cf11b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 0x7f9cec0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cf11b5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec070a10 con 0x7f9cec142f50 2023-08-25T09:42:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cf11b5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec070ea0 con 0x7f9cec143330 2023-08-25T09:42:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cf11b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0b3ea0 con 0x7f9cec0b3a80 2023-08-25T09:42:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cead9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9cec142f50 0x7f9cec15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39656/0 (socket says 172.21.15.70:39656) 2023-08-25T09:42:20.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.793+0000 7f9cead9d640 1 -- 172.21.15.70:0/836372586 learned_addr learned my addr 172.21.15.70:0/836372586 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:20.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737583451 0 0) 0x7f9cec070a10 con 0x7f9cec142f50 2023-08-25T09:42:20.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd8003520 con 0x7f9cec142f50 2023-08-25T09:42:20.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9ceb59e640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 0x7f9cec0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9ce3fff640 1 --2- 172.21.15.70:0/836372586 >> v2:172.21.15.155:3300/0 conn(0x7f9cec143330 0x7f9cec0b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464026088 0 0) 0x7f9cd8003520 con 0x7f9cec142f50 2023-08-25T09:42:20.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 msgr2=0x7f9cec0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 0x7f9cec0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 >> v2:172.21.15.155:3300/0 conn(0x7f9cec143330 msgr2=0x7f9cec0b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 --2- 172.21.15.70:0/836372586 >> v2:172.21.15.155:3300/0 conn(0x7f9cec143330 0x7f9cec0b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec06fe40 con 0x7f9cec142f50 2023-08-25T09:42:20.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9cdc001670 con 0x7f9cec142f50 2023-08-25T09:42:20.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 >> v1:172.21.15.70:6789/0 conn(0x7f9cec142f50 legacy=0x7f9cec15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 0x7f9cd8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd8003fa0 0x7f9cd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cd8004d00 0x7f9cd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec070ea0 con 0x7f9cd8004d00 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0b3ea0 con 0x7f9cd8003fa0 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.794+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd800dc90 con 0x7f9cec0b3a80 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ceb59e640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cd8004d00 0x7f9cd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9cead9d640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 0x7f9cd8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ce3fff640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd8003fa0 0x7f9cd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ceb59e640 1 -- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 msgr2=0x7f9cd8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ceb59e640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cec0b3a80 0x7f9cd8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ceb59e640 1 -- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd8003fa0 msgr2=0x7f9cd80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ceb59e640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cd8003fa0 0x7f9cd80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.795+0000 7f9ceb59e640 1 -- 172.21.15.70:0/836372586 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd8003520 con 0x7f9cd8004d00 2023-08-25T09:42:20.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.796+0000 7f9ceb59e640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cd8004d00 0x7f9cd8005180 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007360 tx=0x7f9cd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2180c030953cdf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:20.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.796+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce402ccd0 con 0x7f9cd8004d00 2023-08-25T09:42:20.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.796+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce402ce30 con 0x7f9cd8004d00 2023-08-25T09:42:20.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.796+0000 7f9cea59c640 1 -- 172.21.15.70:0/836372586 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4034660 con 0x7f9cd8004d00 2023-08-25T09:42:20.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9cf11b5640 1 -- 172.21.15.70:0/836372586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cd8004d00 msgr2=0x7f9cd8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/836372586 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cd8004d00 0x7f9cd8005180 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007360 tx=0x7f9cd80057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9ceb59e640 1 -- 172.21.15.70:0/836372586 reap_dead start 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9cf11b5640 1 -- 172.21.15.70:0/836372586 shutdown_connections 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9cf11b5640 1 -- 172.21.15.70:0/836372586 >> 172.21.15.70:0/836372586 conn(0x7f9cec069a30 msgr2=0x7f9cec069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9cf11b5640 1 -- 172.21.15.70:0/836372586 shutdown_connections 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.797+0000 7f9cf11b5640 1 -- 172.21.15.70:0/836372586 wait complete. 2023-08-25T09:42:20.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.799+0000 7f9cf11b5640 1 Processor -- start 2023-08-25T09:42:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.799+0000 7f9cf11b5640 1 -- start start 2023-08-25T09:42:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9cf11b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cd8004d00 0x7f9cec1f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9cf11b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 0x7f9cec0cd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9cf11b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cec0cd540 0x7f9cec138310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9cf11b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0cd9c0 con 0x7f9cec1f4d50 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9cf11b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0c9f50 con 0x7f9cec0cd540 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9cf11b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec138fb0 con 0x7f9cd8004d00 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 0x7f9cec0cd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9ce3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 0x7f9cec0cd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50446/0 (socket says 172.21.15.70:50446) 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.800+0000 7f9ce3fff640 1 -- 172.21.15.70:0/2116438135 learned_addr learned my addr 172.21.15.70:0/2116438135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce3fff640 1 -- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cd8004d00 msgr2=0x7f9cec1f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce3fff640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cd8004d00 0x7f9cec1f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ceb59e640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cec0cd540 0x7f9cec138310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce3fff640 1 -- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cec0cd540 msgr2=0x7f9cec138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce3fff640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cec0cd540 0x7f9cec138310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce3fff640 1 -- 172.21.15.70:0/2116438135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec139290 con 0x7f9cec1f4d50 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce3fff640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 0x7f9cec0cd000 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400f600 tx=0x7f9cd400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4014870 con 0x7f9cec1f4d50 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd40149d0 con 0x7f9cec1f4d50 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.801+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4014d40 con 0x7f9cec1f4d50 2023-08-25T09:42:20.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.802+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec0c9900 con 0x7f9cec1f4d50 2023-08-25T09:42:20.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.802+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec1f51d0 con 0x7f9cec1f4d50 2023-08-25T09:42:20.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.803+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec1f5390 con 0x7f9cec1f4d50 2023-08-25T09:42:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.804+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9cd4015410 con 0x7f9cec1f4d50 2023-08-25T09:42:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.805+0000 7f9ce37fe640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ccc06a080 0x7f9ccc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.805+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9cd4088cd0 con 0x7f9cec1f4d50 2023-08-25T09:42:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.805+0000 7f9cead9d640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ccc06a080 0x7f9ccc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.806+0000 7f9cead9d640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ccc06a080 0x7f9ccc06c540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0059f0 tx=0x7f9cdc001d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:20.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.809+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9cd4058100 con 0x7f9cec1f4d50 2023-08-25T09:42:21.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.999+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cec066fb0 con 0x7f9cec1f4d50 2023-08-25T09:42:21.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:20.999+0000 7f9ce37fe640 1 -- 172.21.15.70:0/2116438135 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cd4057c90 con 0x7f9cec1f4d50 2023-08-25T09:42:21.002 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:42:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ccc06a080 msgr2=0x7f9ccc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ccc06a080 0x7f9ccc06c540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0059f0 tx=0x7f9cdc001d90 comp rx=0 tx=0).stop 2023-08-25T09:42:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 msgr2=0x7f9cec0cd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 0x7f9cec0cd000 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400f600 tx=0x7f9cd400fac0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 shutdown_connections 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9ccc06a080 0x7f9ccc06c540 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9cec0cd540 0x7f9cec138310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9cec1f4d50 0x7f9cec0cd000 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 --2- 172.21.15.70:0/2116438135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9cd8004d00 0x7f9cec1f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 >> 172.21.15.70:0/2116438135 conn(0x7f9cec069a30 msgr2=0x7f9cec1579f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 shutdown_connections 2023-08-25T09:42:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.002+0000 7f9cf11b5640 1 -- 172.21.15.70:0/2116438135 wait complete. 2023-08-25T09:42:21.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:42:21.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.235+0000 7f968a3d6640 1 Processor -- start 2023-08-25T09:42:21.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.235+0000 7f968a3d6640 1 -- start start 2023-08-25T09:42:21.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.237+0000 7f968a3d6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f96840c2110 0x7f9684069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.237+0000 7f968a3d6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9684069d90 0x7f96840ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.237+0000 7f968a3d6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840c2550 con 0x7f968415ae50 2023-08-25T09:42:21.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.237+0000 7f968a3d6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968406a220 con 0x7f96840c2110 2023-08-25T09:42:21.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.237+0000 7f968a3d6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840ce330 con 0x7f9684069d90 2023-08-25T09:42:21.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.237+0000 7f96837fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f96840c2110 0x7f9684069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f9688973640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9684069d90 0x7f96840ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f96837fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f96840c2110 0x7f9684069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:38304/0 (socket says 172.21.15.70:38304) 2023-08-25T09:42:21.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f96837fe640 1 -- 172.21.15.70:0/3983757293 learned_addr learned my addr 172.21.15.70:0/3983757293 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:21.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f9688973640 1 -- 172.21.15.70:0/3983757293 >> v2:172.21.15.155:3300/0 conn(0x7f96840c2110 msgr2=0x7f9684069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:21.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f9688973640 1 --2- 172.21.15.70:0/3983757293 >> v2:172.21.15.155:3300/0 conn(0x7f96840c2110 0x7f9684069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f9682ffd640 1 -- 172.21.15.70:0/3983757293 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886806408 0 0) 0x7f96840c2550 con 0x7f968415ae50 2023-08-25T09:42:21.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f9688973640 1 -- 172.21.15.70:0/3983757293 >> v1:172.21.15.155:6789/0 conn(0x7f968415ae50 legacy=0x7f96840cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.238+0000 7f9688973640 1 -- 172.21.15.70:0/3983757293 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840cc5f0 con 0x7f9684069d90 2023-08-25T09:42:21.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.240+0000 7f9688973640 1 --2- 172.21.15.70:0/3983757293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9684069d90 0x7f96840ccad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f967c009d90 tx=0x7f967c02bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea23d4f004561fae server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:21.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.240+0000 7f9682ffd640 1 -- 172.21.15.70:0/3983757293 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c02cce0 con 0x7f9684069d90 2023-08-25T09:42:21.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.240+0000 7f9682ffd640 1 -- 172.21.15.70:0/3983757293 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c02ce40 con 0x7f9684069d90 2023-08-25T09:42:21.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.240+0000 7f9682ffd640 1 -- 172.21.15.70:0/3983757293 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c034660 con 0x7f9684069d90 2023-08-25T09:42:21.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 -- 172.21.15.70:0/3983757293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9684069d90 msgr2=0x7f96840ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 --2- 172.21.15.70:0/3983757293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9684069d90 0x7f96840ccad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f967c009d90 tx=0x7f967c02bf30 comp rx=0 tx=0).stop 2023-08-25T09:42:21.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 -- 172.21.15.70:0/3983757293 shutdown_connections 2023-08-25T09:42:21.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 --2- 172.21.15.70:0/3983757293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9684069d90 0x7f96840ccad0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 --2- 172.21.15.70:0/3983757293 >> v2:172.21.15.155:3300/0 conn(0x7f96840c2110 0x7f9684069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 -- 172.21.15.70:0/3983757293 >> 172.21.15.70:0/3983757293 conn(0x7f96840cb0d0 msgr2=0x7f96840cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:21.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 -- 172.21.15.70:0/3983757293 shutdown_connections 2023-08-25T09:42:21.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.241+0000 7f968a3d6640 1 -- 172.21.15.70:0/3983757293 wait complete. 2023-08-25T09:42:21.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.243+0000 7f968a3d6640 1 Processor -- start 2023-08-25T09:42:21.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.243+0000 7f968a3d6640 1 -- start start 2023-08-25T09:42:21.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f968a3d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9684069d90 0x7f96840c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f968a3d6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96840c2110 0x7f968414ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f968a3d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 0x7f96841498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f968a3d6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c7df0 con 0x7f968415ae50 2023-08-25T09:42:21.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f968a3d6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968414d230 con 0x7f9684069d90 2023-08-25T09:42:21.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f968a3d6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c5d50 con 0x7f96840c2110 2023-08-25T09:42:21.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f9688973640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 0x7f96841498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f9688973640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 0x7f96841498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50460/0 (socket says 172.21.15.70:50460) 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f9688973640 1 -- 172.21.15.70:0/909146618 learned_addr learned my addr 172.21.15.70:0/909146618 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.244+0000 7f96837fe640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96840c2110 0x7f968414ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9683fff640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9684069d90 0x7f96840c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9688973640 1 -- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96840c2110 msgr2=0x7f968414ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9688973640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96840c2110 0x7f968414ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9688973640 1 -- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9684069d90 msgr2=0x7f96840c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9688973640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9684069d90 0x7f96840c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9688973640 1 -- 172.21.15.70:0/909146618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967c009a70 con 0x7f968415ae50 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9688973640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 0x7f96841498f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f967c009eb0 tx=0x7f967c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c03d600 con 0x7f968415ae50 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c03d780 con 0x7f968415ae50 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c03db30 con 0x7f968415ae50 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.245+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96840c7730 con 0x7f968415ae50 2023-08-25T09:42:21.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.246+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9684149e30 con 0x7f968415ae50 2023-08-25T09:42:21.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.247+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96840c7f30 con 0x7f968415ae50 2023-08-25T09:42:21.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.248+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f967c03b040 con 0x7f968415ae50 2023-08-25T09:42:21.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.249+0000 7f9680ff9640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f967006a050 0x7f967006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.249+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f967c0aa740 con 0x7f968415ae50 2023-08-25T09:42:21.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.249+0000 7f9683fff640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f967006a050 0x7f967006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.250+0000 7f9683fff640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f967006a050 0x7f967006c510 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9674007fd0 tx=0x7f9674009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:21.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.253+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f967c076210 con 0x7f968415ae50 2023-08-25T09:42:21.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.442+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9684066fb0 con 0x7f968415ae50 2023-08-25T09:42:21.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.443+0000 7f9680ff9640 1 -- 172.21.15.70:0/909146618 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v69) v1 ==== 92+0+10 (secure 0 0 0) 0x7f967c039020 con 0x7f968415ae50 2023-08-25T09:42:21.446 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:42:21.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.445+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f967006a050 msgr2=0x7f967006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.445+0000 7f968a3d6640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f967006a050 0x7f967006c510 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9674007fd0 tx=0x7f9674009040 comp rx=0 tx=0).stop 2023-08-25T09:42:21.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.445+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 msgr2=0x7f96841498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.445+0000 7f968a3d6640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 0x7f96841498f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f967c009eb0 tx=0x7f967c004a40 comp rx=0 tx=0).stop 2023-08-25T09:42:21.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 shutdown_connections 2023-08-25T09:42:21.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f967006a050 0x7f967006c510 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f968415ae50 0x7f96841498f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96840c2110 0x7f968414ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 --2- 172.21.15.70:0/909146618 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9684069d90 0x7f96840c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 >> 172.21.15.70:0/909146618 conn(0x7f96840cb0d0 msgr2=0x7f96840cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:21.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 shutdown_connections 2023-08-25T09:42:21.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.446+0000 7f968a3d6640 1 -- 172.21.15.70:0/909146618 wait complete. 2023-08-25T09:42:21.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:42:21.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.673+0000 7fe5fda70640 1 Processor -- start 2023-08-25T09:42:21.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.673+0000 7fe5fda70640 1 -- start start 2023-08-25T09:42:21.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.674+0000 7fe5fda70640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe5f815caa0 0x7fe5f80d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.674+0000 7fe5fda70640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80d11d0 con 0x7fe5f80cf9a0 2023-08-25T09:42:21.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.674+0000 7fe5fda70640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80d1470 con 0x7fe5f815ce80 2023-08-25T09:42:21.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.674+0000 7fe5fda70640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80d1750 con 0x7fe5f815caa0 2023-08-25T09:42:21.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5fd26f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe5f80cf9a0 0x7fe5f80b5880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39672/0 (socket says 172.21.15.70:39672) 2023-08-25T09:42:21.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5fd26f640 1 -- 172.21.15.70:0/3401563558 learned_addr learned my addr 172.21.15.70:0/3401563558 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:21.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/3401563558 >> v2:172.21.15.155:3300/0 conn(0x7fe5f815caa0 0x7fe5f80d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782192515 0 0) 0x7fe5f80d11d0 con 0x7fe5f80cf9a0 2023-08-25T09:42:21.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e8003520 con 0x7fe5f80cf9a0 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2125789854 0 0) 0x7fe5e8003520 con 0x7fe5f80cf9a0 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 >> v2:172.21.15.155:3300/0 conn(0x7fe5f815caa0 msgr2=0x7fe5f80d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5f77fe640 1 --2- 172.21.15.70:0/3401563558 >> v2:172.21.15.155:3300/0 conn(0x7fe5f815caa0 0x7fe5f80d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.675+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 >> v1:172.21.15.155:6789/0 conn(0x7fe5f815ce80 legacy=0x7fe5f80cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80d1a30 con 0x7fe5f80cf9a0 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe5f00019f0 con 0x7fe5f80cf9a0 2023-08-25T09:42:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 >> v1:172.21.15.70:6789/0 conn(0x7fe5f80cf9a0 legacy=0x7fe5f80b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e8003970 0x7fe5e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e80047c0 0x7fe5e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8005180 0x7fe5e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80d1750 con 0x7fe5e8003970 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800c030 con 0x7fe5e80047c0 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800c1f0 con 0x7fe5e8005180 2023-08-25T09:42:21.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e8003970 0x7fe5e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fca6e640 1 -- 172.21.15.70:0/3401563558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8005180 msgr2=0x7fe5e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:21.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8005180 0x7fe5e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fca6e640 1 -- 172.21.15.70:0/3401563558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e80047c0 msgr2=0x7fe5e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:21.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5e80047c0 0x7fe5e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fca6e640 1 -- 172.21.15.70:0/3401563558 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8003520 con 0x7fe5e8003970 2023-08-25T09:42:21.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.676+0000 7fe5fd26f640 1 -- 172.21.15.70:0/3401563558 reap_dead start 2023-08-25T09:42:21.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.677+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e8003970 0x7fe5e8003d90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009ac0 tx=0x7fe5e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7f6b3330b21f56 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec02c6b0 con 0x7fe5e8003970 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec02c810 con 0x7fe5e8003970 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5f77fe640 1 -- 172.21.15.70:0/3401563558 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec02cba0 con 0x7fe5e8003970 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 -- 172.21.15.70:0/3401563558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e8003970 msgr2=0x7fe5e8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e8003970 0x7fe5e8003d90 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009ac0 tx=0x7fe5e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 -- 172.21.15.70:0/3401563558 shutdown_connections 2023-08-25T09:42:21.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 --2- 172.21.15.70:0/3401563558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5e8003970 0x7fe5e8003d90 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 -- 172.21.15.70:0/3401563558 >> 172.21.15.70:0/3401563558 conn(0x7fe5f806fe70 msgr2=0x7fe5f8070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 -- 172.21.15.70:0/3401563558 shutdown_connections 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.678+0000 7fe5fda70640 1 -- 172.21.15.70:0/3401563558 wait complete. 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.680+0000 7fe5fda70640 1 Processor -- start 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.680+0000 7fe5fda70640 1 -- start start 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.681+0000 7fe5fda70640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8003970 0x7fe5f815f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.681+0000 7fe5fda70640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 0x7fe5f814dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.681+0000 7fe5fda70640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f80c6320 0x7fe5f80c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.681+0000 7fe5fda70640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f80c6ce0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.681+0000 7fe5fda70640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815fda0 con 0x7fe5f80c6320 2023-08-25T09:42:21.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.681+0000 7fe5fda70640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815dda0 con 0x7fe5e8003970 2023-08-25T09:42:21.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 0x7fe5f814dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 0x7fe5f814dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50464/0 (socket says 172.21.15.70:50464) 2023-08-25T09:42:21.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 -- 172.21.15.70:0/4075526789 learned_addr learned my addr 172.21.15.70:0/4075526789 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:21.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 -- 172.21.15.70:0/4075526789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8003970 msgr2=0x7fe5f815f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8003970 0x7fe5f815f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 -- 172.21.15.70:0/4075526789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f80c6320 msgr2=0x7fe5f80c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f80c6320 0x7fe5f80c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 -- 172.21.15.70:0/4075526789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f815e080 con 0x7fe5f815f2b0 2023-08-25T09:42:21.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f7fff640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 0x7fe5f814dfb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c500 tx=0x7fe5e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:21.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.682+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40052d0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.683+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4005430 con 0x7fe5f815f2b0 2023-08-25T09:42:21.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.683+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4012870 con 0x7fe5f815f2b0 2023-08-25T09:42:21.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.683+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f815e360 con 0x7fe5f815f2b0 2023-08-25T09:42:21.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.683+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f815e920 con 0x7fe5f815f2b0 2023-08-25T09:42:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.684+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f815eae0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.685+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe5e40129d0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.686+0000 7fe5f57fa640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe5dc06a050 0x7fe5dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:21.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.686+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe5e4087ef0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.686+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe5dc06a050 0x7fe5dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:21.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.687+0000 7fe5fca6e640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe5dc06a050 0x7fe5dc06c510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec036000 tx=0x7fe5ec02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:21.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.690+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe5e40573b0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.881+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe5f8066fb0 con 0x7fe5f815f2b0 2023-08-25T09:42:21.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.882+0000 7fe5f57fa640 1 -- 172.21.15.70:0/4075526789 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe5e4056f40 con 0x7fe5f815f2b0 2023-08-25T09:42:21.886 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:21.886 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:42:21.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:21.886 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:42:21.887 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:42:21.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:21.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:42:21.887 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:42:21.888 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:42:21.888 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:21.888 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:42:21.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe5dc06a050 msgr2=0x7fe5dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe5dc06a050 0x7fe5dc06c510 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec036000 tx=0x7fe5ec02e000 comp rx=0 tx=0).stop 2023-08-25T09:42:21.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 msgr2=0x7fe5f814dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:21.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 0x7fe5f814dfb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c500 tx=0x7fe5e400c9c0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 shutdown_connections 2023-08-25T09:42:21.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe5dc06a050 0x7fe5dc06c510 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe5f80c6320 0x7fe5f80c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5f815f2b0 0x7fe5f814dfb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 --2- 172.21.15.70:0/4075526789 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5e8003970 0x7fe5f815f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:21.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.884+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 >> 172.21.15.70:0/4075526789 conn(0x7fe5f806fe70 msgr2=0x7fe5f8070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:21.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.885+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 shutdown_connections 2023-08-25T09:42:21.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:21.885+0000 7fe5fda70640 1 -- 172.21.15.70:0/4075526789 wait complete. 2023-08-25T09:42:21.906 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:42:21.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:42:22.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.127+0000 7fb39e70c640 1 Processor -- start 2023-08-25T09:42:22.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.127+0000 7fb39e70c640 1 -- start start 2023-08-25T09:42:22.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb39e70c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb398142c30 0x7fb3980cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:22.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb39e70c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb39806aa70 0x7fb39806af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:22.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb39e70c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39815d320 con 0x7fb398142740 2023-08-25T09:42:22.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb39e70c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb39806b440 con 0x7fb398142c30 2023-08-25T09:42:22.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb39e70c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980ccd40 con 0x7fb39806aa70 2023-08-25T09:42:22.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb3977fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb398142c30 0x7fb3980cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:22.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb3977fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb398142c30 0x7fb3980cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50466/0 (socket says 172.21.15.70:50466) 2023-08-25T09:42:22.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.129+0000 7fb3977fe640 1 -- 172.21.15.70:0/3583431304 learned_addr learned my addr 172.21.15.70:0/3583431304 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:22.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.130+0000 7fb3977fe640 1 -- 172.21.15.70:0/3583431304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb39806aa70 msgr2=0x7fb39806af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:22.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.130+0000 7fb3977fe640 1 --2- 172.21.15.70:0/3583431304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb39806aa70 0x7fb39806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:22.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.130+0000 7fb3977fe640 1 -- 172.21.15.70:0/3583431304 >> v1:172.21.15.155:6789/0 conn(0x7fb398142740 legacy=0x7fb39815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:22.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.130+0000 7fb3977fe640 1 -- 172.21.15.70:0/3583431304 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3980cd020 con 0x7fb398142c30 2023-08-25T09:42:22.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.131+0000 7fb3977fe640 1 --2- 172.21.15.70:0/3583431304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398142c30 0x7fb3980cc6b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00bb40 tx=0x7fb38c02e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9497a6a40e1e7b5e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.131+0000 7fb396ffd640 1 -- 172.21.15.70:0/3583431304 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c02ee70 con 0x7fb398142c30 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.131+0000 7fb396ffd640 1 -- 172.21.15.70:0/3583431304 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c036440 con 0x7fb398142c30 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.131+0000 7fb396ffd640 1 -- 172.21.15.70:0/3583431304 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0367d0 con 0x7fb398142c30 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 -- 172.21.15.70:0/3583431304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398142c30 msgr2=0x7fb3980cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 --2- 172.21.15.70:0/3583431304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398142c30 0x7fb3980cc6b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00bb40 tx=0x7fb38c02e0c0 comp rx=0 tx=0).stop 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 -- 172.21.15.70:0/3583431304 shutdown_connections 2023-08-25T09:42:22.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 --2- 172.21.15.70:0/3583431304 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb39806aa70 0x7fb39806af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 --2- 172.21.15.70:0/3583431304 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb398142c30 0x7fb3980cc6b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 -- 172.21.15.70:0/3583431304 >> 172.21.15.70:0/3583431304 conn(0x7fb398069c70 msgr2=0x7fb39806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 -- 172.21.15.70:0/3583431304 shutdown_connections 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.132+0000 7fb39e70c640 1 -- 172.21.15.70:0/3583431304 wait complete. 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.134+0000 7fb39e70c640 1 Processor -- start 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.134+0000 7fb39e70c640 1 -- start start 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39e70c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 0x7fb3981584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:22.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39e70c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb398142740 0x7fb39813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39e70c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb398142c30 0x7fb39813eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39e70c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39813dd60 con 0x7fb39806aa70 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39e70c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398158a20 con 0x7fb398142c30 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39e70c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39813d0e0 con 0x7fb398142740 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 0x7fb3981584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 0x7fb3981584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50482/0 (socket says 172.21.15.70:50482) 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 -- 172.21.15.70:0/4102047647 learned_addr learned my addr 172.21.15.70:0/4102047647 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:22.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 -- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb398142740 msgr2=0x7fb39813c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb39cca9640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb398142c30 0x7fb39813eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb3977fe640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb398142740 0x7fb39813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb398142740 0x7fb39813c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 -- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb398142c30 msgr2=0x7fb39813eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb398142c30 0x7fb39813eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.135+0000 7fb397fff640 1 -- 172.21.15.70:0/4102047647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c00b770 con 0x7fb39806aa70 2023-08-25T09:42:22.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.136+0000 7fb397fff640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 0x7fb3981584e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c430 tx=0x7fb38800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.136+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388011790 con 0x7fb39806aa70 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.136+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3880118f0 con 0x7fb39806aa70 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.136+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388011ca0 con 0x7fb39806aa70 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.136+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39813d3c0 con 0x7fb39806aa70 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.136+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39813d980 con 0x7fb39806aa70 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.137+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39804e420 con 0x7fb39806aa70 2023-08-25T09:42:22.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.138+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb38801c460 con 0x7fb39806aa70 2023-08-25T09:42:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.139+0000 7fb394ff9640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb38406a050 0x7fb38406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.139+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb388088230 con 0x7fb39806aa70 2023-08-25T09:42:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.139+0000 7fb3977fe640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb38406a050 0x7fb38406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.140+0000 7fb3977fe640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb38406a050 0x7fb38406c510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb38c038000 tx=0x7fb38c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:22.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.143+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb388053bd0 con 0x7fb39806aa70 2023-08-25T09:42:22.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:22.334+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb3980b3910 con 0x7fb39806aa70 2023-08-25T09:42:23.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.268+0000 7fb394ff9640 1 -- 172.21.15.70:0/4102047647 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb3880571f0 con 0x7fb39806aa70 2023-08-25T09:42:23.273 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:42:23.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb38406a050 msgr2=0x7fb38406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb38406a050 0x7fb38406c510 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb38c038000 tx=0x7fb38c008040 comp rx=0 tx=0).stop 2023-08-25T09:42:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 msgr2=0x7fb3981584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 0x7fb3981584e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c430 tx=0x7fb38800c8f0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 shutdown_connections 2023-08-25T09:42:23.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb38406a050 0x7fb38406c510 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb398142c30 0x7fb39813eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb398142740 0x7fb39813c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 --2- 172.21.15.70:0/4102047647 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb39806aa70 0x7fb3981584e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 >> 172.21.15.70:0/4102047647 conn(0x7fb398069c70 msgr2=0x7fb398140fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:23.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 shutdown_connections 2023-08-25T09:42:23.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.275+0000 7fb39e70c640 1 -- 172.21.15.70:0/4102047647 wait complete. 2023-08-25T09:42:23.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:42:23.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.520+0000 7f30d6742640 1 Processor -- start 2023-08-25T09:42:23.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.520+0000 7f30d6742640 1 -- start start 2023-08-25T09:42:23.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.522+0000 7f30d6742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30d0164470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.522+0000 7f30d6742640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d0157730 con 0x7f30d0157350 2023-08-25T09:42:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.522+0000 7f30d6742640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d0159fc0 con 0x7f30d00b4380 2023-08-25T09:42:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.522+0000 7f30d6742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015ac40 con 0x7f30d006b390 2023-08-25T09:42:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.522+0000 7f30d5740640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f30d0157350 0x7f30d015dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39680/0 (socket says 172.21.15.70:39680) 2023-08-25T09:42:23.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.522+0000 7f30d5740640 1 -- 172.21.15.70:0/4232644462 learned_addr learned my addr 172.21.15.70:0/4232644462 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30d5f41640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30d0164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354465893 0 0) 0x7f30d0157730 con 0x7f30d0157350 2023-08-25T09:42:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003520 con 0x7f30d0157350 2023-08-25T09:42:23.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756881592 0 0) 0x7f30bc003520 con 0x7f30d0157350 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 msgr2=0x7f30d0164470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30d0164470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 >> v1:172.21.15.155:6789/0 conn(0x7f30d00b4380 legacy=0x7f30d006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d015a2a0 con 0x7f30d0157350 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f30cc001690 con 0x7f30d0157350 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 >> v1:172.21.15.70:6789/0 conn(0x7f30d0157350 legacy=0x7f30d015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015ac40 con 0x7f30bc004d00 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc00ba60 con 0x7f30bc0039a0 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.523+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc00de30 con 0x7f30d006b390 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5f41640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5740640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:23.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d4f3f640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5f41640 1 -- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 msgr2=0x7f30bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5f41640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5f41640 1 -- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30bc0039a0 msgr2=0x7f30bc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5f41640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30bc0039a0 0x7f30bc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d5f41640 1 -- 172.21.15.70:0/4232644462 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc003520 con 0x7f30bc004d00 2023-08-25T09:42:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.524+0000 7f30d4f3f640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30d006b390 0x7f30bc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.525+0000 7f30d5f41640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009f90 tx=0x7f30bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d0daaf5c4a7c53 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.525+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c802cbd0 con 0x7f30bc004d00 2023-08-25T09:42:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.525+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c800f040 con 0x7f30bc004d00 2023-08-25T09:42:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.525+0000 7f30c7fff640 1 -- 172.21.15.70:0/4232644462 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8034440 con 0x7f30bc004d00 2023-08-25T09:42:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d6742640 1 -- 172.21.15.70:0/4232644462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc004d00 msgr2=0x7f30bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d6742640 1 --2- 172.21.15.70:0/4232644462 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30bc004d00 0x7f30bc005180 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009f90 tx=0x7f30bc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d5f41640 1 -- 172.21.15.70:0/4232644462 reap_dead start 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d6742640 1 -- 172.21.15.70:0/4232644462 shutdown_connections 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d6742640 1 -- 172.21.15.70:0/4232644462 >> 172.21.15.70:0/4232644462 conn(0x7f30d00ce530 msgr2=0x7f30d00ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d6742640 1 -- 172.21.15.70:0/4232644462 shutdown_connections 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.526+0000 7f30d6742640 1 -- 172.21.15.70:0/4232644462 wait complete. 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.528+0000 7f30d6742640 1 Processor -- start 2023-08-25T09:42:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.528+0000 7f30d6742640 1 -- start start 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d6742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30bc004d00 0x7f30d0153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d6742640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 0x7f30d015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d6742640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30d0158d80 0x7f30d0159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d6742640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0159740 con 0x7f30d00af850 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d6742640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d015c6d0 con 0x7f30d0158d80 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d6742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00afcb0 con 0x7f30bc004d00 2023-08-25T09:42:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.529+0000 7f30d4f3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 0x7f30d015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 0x7f30d015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50498/0 (socket says 172.21.15.70:50498) 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 -- 172.21.15.70:0/337790201 learned_addr learned my addr 172.21.15.70:0/337790201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 -- 172.21.15.70:0/337790201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30bc004d00 msgr2=0x7f30d0153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30bc004d00 0x7f30d0153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 -- 172.21.15.70:0/337790201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30d0158d80 msgr2=0x7f30d0159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30d0158d80 0x7f30d0159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 -- 172.21.15.70:0/337790201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d015c910 con 0x7f30d00af850 2023-08-25T09:42:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d4f3f640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 0x7f30d015c190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f30c000d7a0 tx=0x7f30c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0017070 con 0x7f30d00af850 2023-08-25T09:42:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c0005070 con 0x7f30d00af850 2023-08-25T09:42:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.530+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d015cbf0 con 0x7f30d00af850 2023-08-25T09:42:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.531+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0005420 con 0x7f30d00af850 2023-08-25T09:42:23.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.531+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d015d1b0 con 0x7f30d00af850 2023-08-25T09:42:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.532+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d0066fb0 con 0x7f30d00af850 2023-08-25T09:42:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.533+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f30c0007030 con 0x7f30d00af850 2023-08-25T09:42:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.534+0000 7f30c5ffb640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f30b406a050 0x7f30b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.534+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f30c0087c30 con 0x7f30d00af850 2023-08-25T09:42:23.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.534+0000 7f30d5740640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f30b406a050 0x7f30b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:23.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.535+0000 7f30d5740640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f30b406a050 0x7f30b406c510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f30cc001220 tx=0x7f30cc001150 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:23.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.539+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f30c0053460 con 0x7f30d00af850 2023-08-25T09:42:23.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:23.728+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f30d00b1750 con 0x7f30d00af850 2023-08-25T09:42:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.289+0000 7f30c5ffb640 1 -- 172.21.15.70:0/337790201 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v71) v1 ==== 170+0+0 (secure 0 0 0) 0x7f30c0056a80 con 0x7f30d00af850 2023-08-25T09:42:24.297 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:42:24.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f30b406a050 msgr2=0x7f30b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:24.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f30b406a050 0x7f30b406c510 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f30cc001220 tx=0x7f30cc001150 comp rx=0 tx=0).stop 2023-08-25T09:42:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 msgr2=0x7f30d015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 0x7f30d015c190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f30c000d7a0 tx=0x7f30c000dc60 comp rx=0 tx=0).stop 2023-08-25T09:42:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 shutdown_connections 2023-08-25T09:42:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f30b406a050 0x7f30b406c510 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30d0158d80 0x7f30d0159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30d00af850 0x7f30d015c190 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 --2- 172.21.15.70:0/337790201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30bc004d00 0x7f30d0153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.306+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 >> 172.21.15.70:0/337790201 conn(0x7f30d00ce530 msgr2=0x7f30d0154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.307+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 shutdown_connections 2023-08-25T09:42:24.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.307+0000 7f30d6742640 1 -- 172.21.15.70:0/337790201 wait complete. 2023-08-25T09:42:24.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:42:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.523+0000 7f2cbee63640 1 Processor -- start 2023-08-25T09:42:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.523+0000 7f2cbee63640 1 -- start start 2023-08-25T09:42:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbee63640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2cb80c2110 0x7f2cb8069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbee63640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2cb8069d90 0x7f2cb80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbee63640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80c2550 con 0x7f2cb815ae50 2023-08-25T09:42:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbee63640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb806a200 con 0x7f2cb8069d90 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbee63640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80ce330 con 0x7f2cb80c2110 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbd400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2cb8069d90 0x7f2cb80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbd400640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2cb8069d90 0x7f2cb80ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50506/0 (socket says 172.21.15.70:50506) 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.525+0000 7f2cbd400640 1 -- 172.21.15.70:0/3127277083 learned_addr learned my addr 172.21.15.70:0/3127277083 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.526+0000 7f2cb7fff640 1 --2- 172.21.15.70:0/3127277083 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80c2110 0x7f2cb8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.526+0000 7f2cbd400640 1 -- 172.21.15.70:0/3127277083 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80c2110 msgr2=0x7f2cb8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.526+0000 7f2cbd400640 1 --2- 172.21.15.70:0/3127277083 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80c2110 0x7f2cb8069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.526+0000 7f2cbd400640 1 -- 172.21.15.70:0/3127277083 >> v1:172.21.15.155:6789/0 conn(0x7f2cb815ae50 legacy=0x7f2cb80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.526+0000 7f2cbd400640 1 -- 172.21.15.70:0/3127277083 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80cc5f0 con 0x7f2cb8069d90 2023-08-25T09:42:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.526+0000 7f2cb7fff640 1 --2- 172.21.15.70:0/3127277083 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80c2110 0x7f2cb8069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.527+0000 7f2cbd400640 1 --2- 172.21.15.70:0/3127277083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80ccad0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80cb030 tx=0x7f2cb002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98cbcdb4761501 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.527+0000 7f2cb77fe640 1 -- 172.21.15.70:0/3127277083 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb002cb20 con 0x7f2cb8069d90 2023-08-25T09:42:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.527+0000 7f2cb77fe640 1 -- 172.21.15.70:0/3127277083 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb002cc80 con 0x7f2cb8069d90 2023-08-25T09:42:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.527+0000 7f2cb77fe640 1 -- 172.21.15.70:0/3127277083 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00343f0 con 0x7f2cb8069d90 2023-08-25T09:42:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 -- 172.21.15.70:0/3127277083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 msgr2=0x7f2cb80ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 --2- 172.21.15.70:0/3127277083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80ccad0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80cb030 tx=0x7f2cb002bf80 comp rx=0 tx=0).stop 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 -- 172.21.15.70:0/3127277083 shutdown_connections 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 --2- 172.21.15.70:0/3127277083 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80ccad0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 --2- 172.21.15.70:0/3127277083 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80c2110 0x7f2cb8069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 -- 172.21.15.70:0/3127277083 >> 172.21.15.70:0/3127277083 conn(0x7f2cb80cb0d0 msgr2=0x7f2cb80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 -- 172.21.15.70:0/3127277083 shutdown_connections 2023-08-25T09:42:24.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.528+0000 7f2cbee63640 1 -- 172.21.15.70:0/3127277083 wait complete. 2023-08-25T09:42:24.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.530+0000 7f2cbee63640 1 Processor -- start 2023-08-25T09:42:24.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.530+0000 7f2cbee63640 1 -- start start 2023-08-25T09:42:24.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbee63640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbee63640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80c2110 0x7f2cb80c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbee63640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ae50 0x7f2cb8151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbee63640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80c5bd0 con 0x7f2cb8069d90 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbee63640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80c5e00 con 0x7f2cb815ae50 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbee63640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80c60e0 con 0x7f2cb80c2110 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbcbff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbcbff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50510/0 (socket says 172.21.15.70:50510) 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbcbff640 1 -- 172.21.15.70:0/2731293007 learned_addr learned my addr 172.21.15.70:0/2731293007 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cbd400640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ae50 0x7f2cb8151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbcbff640 1 -- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80c2110 msgr2=0x7f2cb80c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.531+0000 7f2cb7fff640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80c2110 0x7f2cb80c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbcbff640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80c2110 0x7f2cb80c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbcbff640 1 -- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ae50 msgr2=0x7f2cb8151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbcbff640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ae50 0x7f2cb8151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbcbff640 1 -- 172.21.15.70:0/2731293007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0009b30 con 0x7f2cb8069d90 2023-08-25T09:42:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbd400640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ae50 0x7f2cb8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbcbff640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80c8170 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00b3e0 tx=0x7f2cac00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac011830 con 0x7f2cb8069d90 2023-08-25T09:42:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac0119b0 con 0x7f2cb8069d90 2023-08-25T09:42:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac011d60 con 0x7f2cb8069d90 2023-08-25T09:42:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb80c63c0 con 0x7f2cb8069d90 2023-08-25T09:42:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.532+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb80c6980 con 0x7f2cb8069d90 2023-08-25T09:42:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.533+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb8066fb0 con 0x7f2cb8069d90 2023-08-25T09:42:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.535+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2cac01a400 con 0x7f2cb8069d90 2023-08-25T09:42:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.535+0000 7f2cb57fa640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c9c06a080 0x7f2c9c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.536+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f2cac088600 con 0x7f2cb8069d90 2023-08-25T09:42:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.536+0000 7f2cb7fff640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c9c06a080 0x7f2c9c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.536+0000 7f2cb7fff640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c9c06a080 0x7f2c9c06c540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ad3c0 tx=0x7f2ca4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.540+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2cac053de0 con 0x7f2cb8069d90 2023-08-25T09:42:24.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:24.768+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2cb80c1ef0 con 0x7f2cb8069d90 2023-08-25T09:42:25.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.296+0000 7f2cb57fa640 1 -- 172.21.15.70:0/2731293007 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v28) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2cac057400 con 0x7f2cb8069d90 2023-08-25T09:42:25.308 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 9 and data pool 10 2023-08-25T09:42:25.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.312+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c9c06a080 msgr2=0x7f2c9c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.312+0000 7f2cbee63640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c9c06a080 0x7f2c9c06c540 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ad3c0 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2023-08-25T09:42:25.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.312+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 msgr2=0x7f2cb80c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.312+0000 7f2cbee63640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80c8170 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00b3e0 tx=0x7f2cac00b8a0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.313+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 shutdown_connections 2023-08-25T09:42:25.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.313+0000 7f2cbee63640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c9c06a080 0x7f2c9c06c540 secure :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ad3c0 tx=0x7f2ca4008040 comp rx=0 tx=0).stop 2023-08-25T09:42:25.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.313+0000 7f2cbee63640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ae50 0x7f2cb8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.313+0000 7f2cbee63640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80c2110 0x7f2cb80c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.313+0000 7f2cbee63640 1 --2- 172.21.15.70:0/2731293007 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb8069d90 0x7f2cb80c8170 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.313+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 >> 172.21.15.70:0/2731293007 conn(0x7f2cb80cb0d0 msgr2=0x7f2cb80ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.314+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 shutdown_connections 2023-08-25T09:42:25.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.315+0000 7f2cbee63640 1 -- 172.21.15.70:0/2731293007 wait complete. 2023-08-25T09:42:25.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:25.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.519+0000 7f58d3d79640 1 Processor -- start 2023-08-25T09:42:25.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.519+0000 7f58d3d79640 1 -- start start 2023-08-25T09:42:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.520+0000 7f58d3d79640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58cc0ac870 0x7f58cc0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.520+0000 7f58d3d79640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58cc06f050 con 0x7f58cc15cb40 2023-08-25T09:42:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.520+0000 7f58d3d79640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58cc06f330 con 0x7f58cc0cea30 2023-08-25T09:42:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.520+0000 7f58d3d79640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58cc06f610 con 0x7f58cc0ac870 2023-08-25T09:42:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.521+0000 7f58d1b15640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f58cc15cb40 0x7f58cc0ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39696/0 (socket says 172.21.15.70:39696) 2023-08-25T09:42:25.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.521+0000 7f58d1b15640 1 -- 172.21.15.70:0/1620898948 learned_addr learned my addr 172.21.15.70:0/1620898948 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.521+0000 7f58d2316640 1 --2- 172.21.15.70:0/1620898948 >> v2:172.21.15.155:3300/0 conn(0x7f58cc0ac870 0x7f58cc0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.521+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552894325 0 0) 0x7f58cc06f050 con 0x7f58cc15cb40 2023-08-25T09:42:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.521+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58bc003520 con 0x7f58cc15cb40 2023-08-25T09:42:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35831048 0 0) 0x7f58bc003520 con 0x7f58cc15cb40 2023-08-25T09:42:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 >> v2:172.21.15.155:3300/0 conn(0x7f58cc0ac870 msgr2=0x7f58cc0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 --2- 172.21.15.70:0/1620898948 >> v2:172.21.15.155:3300/0 conn(0x7f58cc0ac870 0x7f58cc0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 >> v1:172.21.15.155:6789/0 conn(0x7f58cc0cea30 legacy=0x7f58cc0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc06f8f0 con 0x7f58cc15cb40 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f58c0001670 con 0x7f58cc15cb40 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 >> v1:172.21.15.70:6789/0 conn(0x7f58cc15cb40 legacy=0x7f58cc0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc003970 0x7f58bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58bc005180 0x7f58bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc06f610 con 0x7f58bc0047c0 2023-08-25T09:42:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc00c020 con 0x7f58bc003970 2023-08-25T09:42:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc00c1e0 con 0x7f58bc005180 2023-08-25T09:42:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1314640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1314640 1 -- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58bc005180 msgr2=0x7f58bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1314640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58bc005180 0x7f58bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1b15640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc003970 0x7f58bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1314640 1 -- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc003970 msgr2=0x7f58bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:25.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1314640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58bc003970 0x7f58bc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d1314640 1 -- 172.21.15.70:0/1620898948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc003520 con 0x7f58bc0047c0 2023-08-25T09:42:25.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.522+0000 7f58d2316640 1 -- 172.21.15.70:0/1620898948 reap_dead start 2023-08-25T09:42:25.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.523+0000 7f58d1314640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58bc004c40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f58b8001e50 tx=0x7f58bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14e321d6442aa27 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:25.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8003b70 con 0x7f58bc0047c0 2023-08-25T09:42:25.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b8003cd0 con 0x7f58bc0047c0 2023-08-25T09:42:25.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d0b13640 1 -- 172.21.15.70:0/1620898948 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8005310 con 0x7f58bc0047c0 2023-08-25T09:42:25.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 -- 172.21.15.70:0/1620898948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 msgr2=0x7f58bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58bc004c40 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f58b8001e50 tx=0x7f58bc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 -- 172.21.15.70:0/1620898948 shutdown_connections 2023-08-25T09:42:25.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 --2- 172.21.15.70:0/1620898948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58bc004c40 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 -- 172.21.15.70:0/1620898948 >> 172.21.15.70:0/1620898948 conn(0x7f58cc069850 msgr2=0x7f58cc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:25.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 -- 172.21.15.70:0/1620898948 shutdown_connections 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.524+0000 7f58d3d79640 1 -- 172.21.15.70:0/1620898948 wait complete. 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.526+0000 7f58d3d79640 1 Processor -- start 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.526+0000 7f58d3d79640 1 -- start start 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d3d79640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58cc153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d3d79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58cc15a130 0x7f58cc15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d3d79640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc15a5b0 0x7f58cc15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d3d79640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc15d630 con 0x7f58bc0047c0 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d3d79640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc153880 con 0x7f58cc15a5b0 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d3d79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc15ab80 con 0x7f58cc15a130 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d1b15640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58cc153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d1b15640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58cc153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50536/0 (socket says 172.21.15.70:50536) 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.527+0000 7f58d1b15640 1 -- 172.21.15.70:0/3403153542 learned_addr learned my addr 172.21.15.70:0/3403153542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d1b15640 1 -- 172.21.15.70:0/3403153542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58cc15a130 msgr2=0x7f58cc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d1b15640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58cc15a130 0x7f58cc15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d1b15640 1 -- 172.21.15.70:0/3403153542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc15a5b0 msgr2=0x7f58cc15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d1b15640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc15a5b0 0x7f58cc15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d1b15640 1 -- 172.21.15.70:0/3403153542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc15ae60 con 0x7f58bc0047c0 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d1b15640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58cc153340 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000ff0 tx=0x7f58c000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c00074e0 con 0x7f58bc0047c0 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c0007640 con 0x7f58bc0047c0 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c0004a00 con 0x7f58bc0047c0 2023-08-25T09:42:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc15d2c0 con 0x7f58bc0047c0 2023-08-25T09:42:25.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.528+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc0cdca0 con 0x7f58bc0047c0 2023-08-25T09:42:25.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.529+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc04e420 con 0x7f58bc0047c0 2023-08-25T09:42:25.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.530+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f58c0004b60 con 0x7f58bc0047c0 2023-08-25T09:42:25.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.531+0000 7f58ca7fc640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58b006a050 0x7f58b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.531+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58c0088070 con 0x7f58bc0047c0 2023-08-25T09:42:25.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.532+0000 7f58d1314640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58b006a050 0x7f58b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.532+0000 7f58d1314640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58b006a050 0x7f58b006c510 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58b8034000 tx=0x7f58b8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:25.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.536+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58c00ba990 con 0x7f58bc0047c0 2023-08-25T09:42:25.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.713+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58cc158de0 con 0x7f58bc0047c0 2023-08-25T09:42:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.716+0000 7f58ca7fc640 1 -- 172.21.15.70:0/3403153542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+18368 (secure 0 0 0) 0x7f58c0019090 con 0x7f58bc0047c0 2023-08-25T09:42:25.719 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:25.720 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":72,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:24.770574+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58b006a050 msgr2=0x7f58b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58b006a050 0x7f58b006c510 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58b8034000 tx=0x7f58b8030000 comp rx=0 tx=0).stop 2023-08-25T09:42:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 msgr2=0x7f58cc153340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58cc153340 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f58c0000ff0 tx=0x7f58c000dc00 comp rx=0 tx=0).stop 2023-08-25T09:42:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 shutdown_connections 2023-08-25T09:42:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58b006a050 0x7f58b006c510 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58cc15a5b0 0x7f58cc15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58cc15a130 0x7f58cc15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 --2- 172.21.15.70:0/3403153542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58bc0047c0 0x7f58cc153340 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 >> 172.21.15.70:0/3403153542 conn(0x7f58cc069850 msgr2=0x7f58cc154160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 shutdown_connections 2023-08-25T09:42:25.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.719+0000 7f58d3d79640 1 -- 172.21.15.70:0/3403153542 wait complete. 2023-08-25T09:42:25.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:25.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.946+0000 7f840f112640 1 Processor -- start 2023-08-25T09:42:25.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.946+0000 7f840f112640 1 -- start start 2023-08-25T09:42:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f840f112640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 0x7f840806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f840f112640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84080b1b60 con 0x7f840806b2b0 2023-08-25T09:42:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f840f112640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840806b7c0 con 0x7f840815ec10 2023-08-25T09:42:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f840f112640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840806bd80 con 0x7f840815e830 2023-08-25T09:42:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f8407fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f840806b2b0 0x7f840815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39700/0 (socket says 172.21.15.70:39700) 2023-08-25T09:42:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f8407fff640 1 -- 172.21.15.70:0/2083521431 learned_addr learned my addr 172.21.15.70:0/2083521431 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.948+0000 7f840ceae640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 0x7f840806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842208684 0 0) 0x7f84080b1b60 con 0x7f840806b2b0 2023-08-25T09:42:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f8003520 con 0x7f840806b2b0 2023-08-25T09:42:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2988299226 0 0) 0x7f83f8003520 con 0x7f840806b2b0 2023-08-25T09:42:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 msgr2=0x7f840806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 0x7f840806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 >> v1:172.21.15.155:6789/0 conn(0x7f840815ec10 legacy=0x7f84080b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840806baa0 con 0x7f840806b2b0 2023-08-25T09:42:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f83f4001670 con 0x7f840806b2b0 2023-08-25T09:42:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 >> v1:172.21.15.70:6789/0 conn(0x7f840806b2b0 legacy=0x7f840815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 0x7f83f80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83f8003f20 0x7f83f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840806bd80 con 0x7f83f8004d00 2023-08-25T09:42:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f800ba60 con 0x7f83f8003f20 2023-08-25T09:42:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.949+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f800bc20 con 0x7f840815e830 2023-08-25T09:42:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840d6af640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840ceae640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 0x7f83f80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f8407fff640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83f8003f20 0x7f83f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840d6af640 1 -- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 msgr2=0x7f83f80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840d6af640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f840815e830 0x7f83f80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840d6af640 1 -- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83f8003f20 msgr2=0x7f83f80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840d6af640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83f8003f20 0x7f83f80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.950+0000 7f840d6af640 1 -- 172.21.15.70:0/2083521431 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8003520 con 0x7f83f8004d00 2023-08-25T09:42:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.951+0000 7f840d6af640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8400001e50 tx=0x7f83f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9e140d3ff5c6eb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:25.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.951+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400003b70 con 0x7f83f8004d00 2023-08-25T09:42:25.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.951+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8400003cd0 con 0x7f83f8004d00 2023-08-25T09:42:25.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.951+0000 7f84077fe640 1 -- 172.21.15.70:0/2083521431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400005310 con 0x7f83f8004d00 2023-08-25T09:42:25.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840f112640 1 -- 172.21.15.70:0/2083521431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 msgr2=0x7f83f8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840f112640 1 --2- 172.21.15.70:0/2083521431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f83f8005180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8400001e50 tx=0x7f83f80057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840d6af640 1 -- 172.21.15.70:0/2083521431 reap_dead start 2023-08-25T09:42:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840f112640 1 -- 172.21.15.70:0/2083521431 shutdown_connections 2023-08-25T09:42:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840f112640 1 -- 172.21.15.70:0/2083521431 >> 172.21.15.70:0/2083521431 conn(0x7f8408069850 msgr2=0x7f8408069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840f112640 1 -- 172.21.15.70:0/2083521431 shutdown_connections 2023-08-25T09:42:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.952+0000 7f840f112640 1 -- 172.21.15.70:0/2083521431 wait complete. 2023-08-25T09:42:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.954+0000 7f840f112640 1 Processor -- start 2023-08-25T09:42:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.954+0000 7f840f112640 1 -- start start 2023-08-25T09:42:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840f112640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f84081f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840f112640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080708a0 0x7f84081553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840f112640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84081505a0 0x7f8408153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840f112640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84081558e0 con 0x7f83f8004d00 2023-08-25T09:42:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840f112640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408070d00 con 0x7f84081505a0 2023-08-25T09:42:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840f112640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84081f2570 con 0x7f84080708a0 2023-08-25T09:42:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f84081f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f84081f3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50554/0 (socket says 172.21.15.70:50554) 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 -- 172.21.15.70:0/3564395607 learned_addr learned my addr 172.21.15.70:0/3564395607 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 -- 172.21.15.70:0/3564395607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080708a0 msgr2=0x7f84081553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080708a0 0x7f84081553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 -- 172.21.15.70:0/3564395607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84081505a0 msgr2=0x7f8408153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.955+0000 7f840ceae640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84081505a0 0x7f8408153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f840ceae640 1 -- 172.21.15.70:0/3564395607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840815ca20 con 0x7f83f8004d00 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f840ceae640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f84081f3030 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f83fc010860 tx=0x7f83fc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc009610 con 0x7f83f8004d00 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc00eb60 con 0x7f83f8004d00 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc01c400 con 0x7f83f8004d00 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8408155ad0 con 0x7f83f8004d00 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.956+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84081f2830 con 0x7f83f8004d00 2023-08-25T09:42:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.957+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84080cef50 con 0x7f83f8004d00 2023-08-25T09:42:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.958+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f83fc01c560 con 0x7f83f8004d00 2023-08-25T09:42:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.959+0000 7f84057fa640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83ec06a080 0x7f83ec06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.959+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83fc088a50 con 0x7f83f8004d00 2023-08-25T09:42:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.960+0000 7f8407fff640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83ec06a080 0x7f83ec06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.960+0000 7f8407fff640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83ec06a080 0x7f83ec06c540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f83f4001eb0 tx=0x7f83f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:25.963+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f83fc0541e0 con 0x7f83f8004d00 2023-08-25T09:42:26.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.151+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f840804e420 con 0x7f83f8004d00 2023-08-25T09:42:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.152+0000 7f84057fa640 1 -- 172.21.15.70:0/3564395607 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+18368 (secure 0 0 0) 0x7f83fc057800 con 0x7f83f8004d00 2023-08-25T09:42:26.156 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:26.156 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":72,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:24.770574+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:26.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83ec06a080 msgr2=0x7f83ec06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:26.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83ec06a080 0x7f83ec06c540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f83f4001eb0 tx=0x7f83f4009040 comp rx=0 tx=0).stop 2023-08-25T09:42:26.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 msgr2=0x7f84081f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:26.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f84081f3030 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f83fc010860 tx=0x7f83fc010d20 comp rx=0 tx=0).stop 2023-08-25T09:42:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 shutdown_connections 2023-08-25T09:42:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83ec06a080 0x7f83ec06c540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f84081505a0 0x7f8408153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84080708a0 0x7f84081553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 --2- 172.21.15.70:0/3564395607 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83f8004d00 0x7f84081f3030 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 >> 172.21.15.70:0/3564395607 conn(0x7f8408069850 msgr2=0x7f840815cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:26.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 shutdown_connections 2023-08-25T09:42:26.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.155+0000 7f840f112640 1 -- 172.21.15.70:0/3564395607 wait complete. 2023-08-25T09:42:26.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:42:26.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.393+0000 7fbc295fd640 1 Processor -- start 2023-08-25T09:42:26.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.394+0000 7fbc295fd640 1 -- start start 2023-08-25T09:42:26.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.395+0000 7fbc295fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc2415ae50 0x7fbc240c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:26.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.395+0000 7fbc295fd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240cc5f0 con 0x7fbc24069f60 2023-08-25T09:42:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.395+0000 7fbc295fd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc24070650 con 0x7fbc240cdc20 2023-08-25T09:42:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.395+0000 7fbc295fd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc24070930 con 0x7fbc2415ae50 2023-08-25T09:42:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.395+0000 7fbc22ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc2415ae50 0x7fbc240c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.396+0000 7fbc22ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc2415ae50 0x7fbc240c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50556/0 (socket says 172.21.15.70:50556) 2023-08-25T09:42:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.396+0000 7fbc22ffd640 1 -- 172.21.15.70:0/294131403 learned_addr learned my addr 172.21.15.70:0/294131403 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.396+0000 7fbc22ffd640 1 -- 172.21.15.70:0/294131403 >> v1:172.21.15.155:6789/0 conn(0x7fbc240cdc20 legacy=0x7fbc24069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.396+0000 7fbc22ffd640 1 -- 172.21.15.70:0/294131403 >> v1:172.21.15.70:6789/0 conn(0x7fbc24069f60 legacy=0x7fbc240ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.396+0000 7fbc22ffd640 1 -- 172.21.15.70:0/294131403 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24070c10 con 0x7fbc2415ae50 2023-08-25T09:42:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.397+0000 7fbc22ffd640 1 --2- 172.21.15.70:0/294131403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc2415ae50 0x7fbc240c2110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009870 tx=0x7fbc1402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32406c457e08587 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.397+0000 7fbc227fc640 1 -- 172.21.15.70:0/294131403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1402f850 con 0x7fbc2415ae50 2023-08-25T09:42:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.397+0000 7fbc227fc640 1 -- 172.21.15.70:0/294131403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1402f9b0 con 0x7fbc2415ae50 2023-08-25T09:42:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.397+0000 7fbc227fc640 1 -- 172.21.15.70:0/294131403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1402fd40 con 0x7fbc2415ae50 2023-08-25T09:42:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 -- 172.21.15.70:0/294131403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc2415ae50 msgr2=0x7fbc240c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 --2- 172.21.15.70:0/294131403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc2415ae50 0x7fbc240c2110 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009870 tx=0x7fbc1402ba30 comp rx=0 tx=0).stop 2023-08-25T09:42:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 -- 172.21.15.70:0/294131403 shutdown_connections 2023-08-25T09:42:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 --2- 172.21.15.70:0/294131403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc2415ae50 0x7fbc240c2110 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 -- 172.21.15.70:0/294131403 >> 172.21.15.70:0/294131403 conn(0x7fbc240cb0d0 msgr2=0x7fbc240cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 -- 172.21.15.70:0/294131403 shutdown_connections 2023-08-25T09:42:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.398+0000 7fbc295fd640 1 -- 172.21.15.70:0/294131403 wait complete. 2023-08-25T09:42:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.400+0000 7fbc295fd640 1 Processor -- start 2023-08-25T09:42:26.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.400+0000 7fbc295fd640 1 -- start start 2023-08-25T09:42:26.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc295fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 0x7fbc241481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:26.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc295fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc240cdc20 0x7fbc240c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc295fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2415ae50 0x7fbc240ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc295fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c82b0 con 0x7fbc24069f60 2023-08-25T09:42:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc295fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c7630 con 0x7fbc240cdc20 2023-08-25T09:42:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc295fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240c7910 con 0x7fbc2415ae50 2023-08-25T09:42:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc22ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 0x7fbc241481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc22ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 0x7fbc241481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50566/0 (socket says 172.21.15.70:50566) 2023-08-25T09:42:26.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc22ffd640 1 -- 172.21.15.70:0/3626016825 learned_addr learned my addr 172.21.15.70:0/3626016825 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:26.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc22ffd640 1 -- 172.21.15.70:0/3626016825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2415ae50 msgr2=0x7fbc240ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:26.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc22ffd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2415ae50 0x7fbc240ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.401+0000 7fbc22ffd640 1 -- 172.21.15.70:0/3626016825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc240cdc20 msgr2=0x7fbc240c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc22ffd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc240cdc20 0x7fbc240c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc22ffd640 1 -- 172.21.15.70:0/3626016825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc140094d0 con 0x7fbc24069f60 2023-08-25T09:42:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc22ffd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 0x7fbc241481c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009870 tx=0x7fbc14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:26.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14037d10 con 0x7fbc24069f60 2023-08-25T09:42:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14037e70 con 0x7fbc24069f60 2023-08-25T09:42:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14041620 con 0x7fbc24069f60 2023-08-25T09:42:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc240c7bf0 con 0x7fbc24069f60 2023-08-25T09:42:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.402+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24148700 con 0x7fbc24069f60 2023-08-25T09:42:26.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.403+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc24066fb0 con 0x7fbc24069f60 2023-08-25T09:42:26.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.404+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbc14041780 con 0x7fbc24069f60 2023-08-25T09:42:26.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.405+0000 7fbc1b7fe640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc0406a050 0x7fbc0406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:26.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.405+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc1403c070 con 0x7fbc24069f60 2023-08-25T09:42:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.406+0000 7fbc1bfff640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc0406a050 0x7fbc0406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:26.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.406+0000 7fbc1bfff640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc0406a050 0x7fbc0406c510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc240c4230 tx=0x7fbc0c0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:26.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.410+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbc14076ac0 con 0x7fbc24069f60 2023-08-25T09:42:26.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:26.619+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbc240cb850 con 0x7fbc24069f60 2023-08-25T09:42:27.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.338+0000 7fbc1b7fe640 1 -- 172.21.15.70:0/3626016825 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbc1407a0e0 con 0x7fbc24069f60 2023-08-25T09:42:27.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc0406a050 msgr2=0x7fbc0406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:27.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc0406a050 0x7fbc0406c510 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbc240c4230 tx=0x7fbc0c0079b0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 msgr2=0x7fbc241481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:27.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 0x7fbc241481c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009870 tx=0x7fbc14004730 comp rx=0 tx=0).stop 2023-08-25T09:42:27.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 shutdown_connections 2023-08-25T09:42:27.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc0406a050 0x7fbc0406c510 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc2415ae50 0x7fbc240ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc240cdc20 0x7fbc240c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 --2- 172.21.15.70:0/3626016825 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc24069f60 0x7fbc241481c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 >> 172.21.15.70:0/3626016825 conn(0x7fbc240cb0d0 msgr2=0x7fbc2406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:27.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 shutdown_connections 2023-08-25T09:42:27.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.342+0000 7fbc295fd640 1 -- 172.21.15.70:0/3626016825 wait complete. 2023-08-25T09:42:27.364 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:42:27.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:42:27.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.569+0000 7f1b9324b640 1 Processor -- start 2023-08-25T09:42:27.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.569+0000 7f1b9324b640 1 -- start start 2023-08-25T09:42:27.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b9324b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 0x7f1b8c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b9324b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1b8c0ad340 0x7f1b8c06e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b9324b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c15ac10 con 0x7f1b8c0ca450 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b9324b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8c0adb40 con 0x7f1b8c0ad340 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b9324b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0cac80 con 0x7f1b8c0b0300 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b91a48640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1b8c0ca450 0x7f1b8c0b06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39714/0 (socket says 172.21.15.70:39714) 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b91a48640 1 -- 172.21.15.70:0/4202699814 learned_addr learned my addr 172.21.15.70:0/4202699814 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:27.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b92a4a640 1 --2- 172.21.15.70:0/4202699814 >> v2:172.21.15.155:3300/0 conn(0x7f1b8c0ad340 0x7f1b8c06e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b92249640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 0x7f1b8c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481223603 0 0) 0x7f1b8c15ac10 con 0x7f1b8c0ca450 2023-08-25T09:42:27.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.571+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b78003520 con 0x7f1b8c0ca450 2023-08-25T09:42:27.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 523947451 0 0) 0x7f1b78003520 con 0x7f1b8c0ca450 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 msgr2=0x7f1b8c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 0x7f1b8c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 >> v2:172.21.15.155:3300/0 conn(0x7f1b8c0ad340 msgr2=0x7f1b8c06e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 --2- 172.21.15.70:0/4202699814 >> v2:172.21.15.155:3300/0 conn(0x7f1b8c0ad340 0x7f1b8c06e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0c9110 con 0x7f1b8c0ca450 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1b7c001670 con 0x7f1b8c0ca450 2023-08-25T09:42:27.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 >> v1:172.21.15.70:6789/0 conn(0x7f1b8c0ca450 legacy=0x7f1b8c0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 0x7f1b78003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b78004d00 0x7f1b78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0adb40 con 0x7f1b78003fa0 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0cac80 con 0x7f1b78004d00 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7800bb40 con 0x7f1b8c0b0300 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91a48640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b92249640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 0x7f1b78003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91a48640 1 -- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 msgr2=0x7f1b78003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91a48640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c0b0300 0x7f1b78003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91a48640 1 -- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b78004d00 msgr2=0x7f1b78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.572+0000 7f1b91a48640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b78004d00 0x7f1b78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.573+0000 7f1b91a48640 1 -- 172.21.15.70:0/4202699814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78003520 con 0x7f1b78003fa0 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.573+0000 7f1b91a48640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b780047c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00c290 tx=0x7f1b780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d7eb3237c64430 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c005ad0 con 0x7f1b78003fa0 2023-08-25T09:42:27.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7c005c30 con 0x7f1b78003fa0 2023-08-25T09:42:27.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b91247640 1 -- 172.21.15.70:0/4202699814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c02c8d0 con 0x7f1b78003fa0 2023-08-25T09:42:27.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b9324b640 1 -- 172.21.15.70:0/4202699814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 msgr2=0x7f1b780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:27.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b9324b640 1 --2- 172.21.15.70:0/4202699814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b780047c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00c290 tx=0x7f1b780057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b92a4a640 1 -- 172.21.15.70:0/4202699814 reap_dead start 2023-08-25T09:42:27.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b9324b640 1 -- 172.21.15.70:0/4202699814 shutdown_connections 2023-08-25T09:42:27.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b9324b640 1 -- 172.21.15.70:0/4202699814 >> 172.21.15.70:0/4202699814 conn(0x7f1b8c0ce810 msgr2=0x7f1b8c0cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:27.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b9324b640 1 -- 172.21.15.70:0/4202699814 shutdown_connections 2023-08-25T09:42:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.574+0000 7f1b9324b640 1 -- 172.21.15.70:0/4202699814 wait complete. 2023-08-25T09:42:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.576+0000 7f1b9324b640 1 Processor -- start 2023-08-25T09:42:27.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.576+0000 7f1b9324b640 1 -- start start 2023-08-25T09:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b9324b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b8c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b9324b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c15a1d0 0x7f1b8c152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b9324b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c153e40 0x7f1b8c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b9324b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c151320 con 0x7f1b78003fa0 2023-08-25T09:42:27.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b9324b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c1507a0 con 0x7f1b8c153e40 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b9324b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c150a80 con 0x7f1b8c15a1d0 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b92249640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b8c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b92249640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b8c0cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50574/0 (socket says 172.21.15.70:50574) 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b92249640 1 -- 172.21.15.70:0/2102842522 learned_addr learned my addr 172.21.15.70:0/2102842522 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.577+0000 7f1b92a4a640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c153e40 0x7f1b8c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b92249640 1 -- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c15a1d0 msgr2=0x7f1b8c152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:27.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b92249640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c15a1d0 0x7f1b8c152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b92249640 1 -- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c153e40 msgr2=0x7f1b8c152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:27.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b92249640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c153e40 0x7f1b8c152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:27.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b92249640 1 -- 172.21.15.70:0/2102842522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c150d60 con 0x7f1b78003fa0 2023-08-25T09:42:27.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b92249640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b8c0cd8e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e810 tx=0x7f1b8800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800c6d0 con 0x7f1b78003fa0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8800c830 con 0x7f1b78003fa0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8800cbc0 con 0x7f1b78003fa0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c151040 con 0x7f1b78003fa0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.578+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c0cde20 con 0x7f1b78003fa0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.579+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8c15c110 con 0x7f1b78003fa0 2023-08-25T09:42:27.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.580+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1b8800cd20 con 0x7f1b78003fa0 2023-08-25T09:42:27.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.581+0000 7f1b82ffd640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1b7006a050 0x7f1b7006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:27.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.581+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b88088280 con 0x7f1b78003fa0 2023-08-25T09:42:27.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.581+0000 7f1b91a48640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1b7006a050 0x7f1b7006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:27.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.582+0000 7f1b91a48640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1b7006a050 0x7f1b7006c510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c034000 tx=0x7f1b7c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:27.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.585+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1b88053960 con 0x7f1b78003fa0 2023-08-25T09:42:27.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:27.793+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1b8c061460 con 0x7f1b78003fa0 2023-08-25T09:42:28.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.342+0000 7f1b82ffd640 1 -- 172.21.15.70:0/2102842522 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1b880093a0 con 0x7f1b78003fa0 2023-08-25T09:42:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1b7006a050 msgr2=0x7f1b7006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1b7006a050 0x7f1b7006c510 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c034000 tx=0x7f1b7c02e000 comp rx=0 tx=0).stop 2023-08-25T09:42:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 msgr2=0x7f1b8c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b8c0cd8e0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800e810 tx=0x7f1b8800ecd0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 shutdown_connections 2023-08-25T09:42:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1b7006a050 0x7f1b7006c510 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b8c153e40 0x7f1b8c152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b8c15a1d0 0x7f1b8c152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 --2- 172.21.15.70:0/2102842522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b78003fa0 0x7f1b8c0cd8e0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.346+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 >> 172.21.15.70:0/2102842522 conn(0x7f1b8c0ce810 msgr2=0x7f1b8c0cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.347+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 shutdown_connections 2023-08-25T09:42:28.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.347+0000 7f1b9324b640 1 -- 172.21.15.70:0/2102842522 wait complete. 2023-08-25T09:42:28.369 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:42:28.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.582+0000 7f5bef1ab640 1 Processor -- start 2023-08-25T09:42:28.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.582+0000 7f5bef1ab640 1 -- start start 2023-08-25T09:42:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5bef1ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5be80b2850 0x7f5be8164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:28.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5bef1ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be815d3f0 con 0x7f5be815aa10 2023-08-25T09:42:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5bef1ab640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be8070a00 con 0x7f5be80ce5b0 2023-08-25T09:42:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5bef1ab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5be8159d10 con 0x7f5be80b2850 2023-08-25T09:42:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5bed748640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5be80b2850 0x7f5be8164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5be7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5be815aa10 0x7f5be80b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39726/0 (socket says 172.21.15.70:39726) 2023-08-25T09:42:28.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.584+0000 7f5be7fff640 1 -- 172.21.15.70:0/930765466 learned_addr learned my addr 172.21.15.70:0/930765466 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:28.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.585+0000 7f5bed748640 1 -- 172.21.15.70:0/930765466 >> v1:172.21.15.155:6789/0 conn(0x7f5be80ce5b0 legacy=0x7f5be80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.585+0000 7f5bed748640 1 -- 172.21.15.70:0/930765466 >> v1:172.21.15.70:6789/0 conn(0x7f5be815aa10 legacy=0x7f5be80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.585+0000 7f5bed748640 1 -- 172.21.15.70:0/930765466 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be80b2d20 con 0x7f5be80b2850 2023-08-25T09:42:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.585+0000 7f5be77fe640 1 -- 172.21.15.70:0/930765466 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176721306 0 0) 0x7f5be815d3f0 con 0x7f5be815aa10 2023-08-25T09:42:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.586+0000 7f5bed748640 1 --2- 172.21.15.70:0/930765466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80b2850 0x7f5be8164780 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5be815d5e0 tx=0x7f5be002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17e98a4f6e423 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.586+0000 7f5be77fe640 1 -- 172.21.15.70:0/930765466 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be002cbb0 con 0x7f5be80b2850 2023-08-25T09:42:28.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.586+0000 7f5be77fe640 1 -- 172.21.15.70:0/930765466 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be002cd10 con 0x7f5be80b2850 2023-08-25T09:42:28.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.586+0000 7f5be77fe640 1 -- 172.21.15.70:0/930765466 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0034660 con 0x7f5be80b2850 2023-08-25T09:42:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 -- 172.21.15.70:0/930765466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80b2850 msgr2=0x7f5be8164780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/930765466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80b2850 0x7f5be8164780 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f5be815d5e0 tx=0x7f5be002be00 comp rx=0 tx=0).stop 2023-08-25T09:42:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 -- 172.21.15.70:0/930765466 shutdown_connections 2023-08-25T09:42:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/930765466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80b2850 0x7f5be8164780 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 -- 172.21.15.70:0/930765466 >> 172.21.15.70:0/930765466 conn(0x7f5be8069850 msgr2=0x7f5be8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:28.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 -- 172.21.15.70:0/930765466 shutdown_connections 2023-08-25T09:42:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.587+0000 7f5bef1ab640 1 -- 172.21.15.70:0/930765466 wait complete. 2023-08-25T09:42:28.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.589+0000 7f5bef1ab640 1 Processor -- start 2023-08-25T09:42:28.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.589+0000 7f5bef1ab640 1 -- start start 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5bef1ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80b2850 0x7f5be8152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5bef1ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 0x7f5be814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5bef1ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815aa10 0x7f5be814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5bef1ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be814eef0 con 0x7f5be80ce5b0 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5bef1ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be81e3250 con 0x7f5be80b2850 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5bef1ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be81e3530 con 0x7f5be815aa10 2023-08-25T09:42:28.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 0x7f5be814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 0x7f5be814df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50596/0 (socket says 172.21.15.70:50596) 2023-08-25T09:42:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 -- 172.21.15.70:0/3297897070 learned_addr learned my addr 172.21.15.70:0/3297897070 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 -- 172.21.15.70:0/3297897070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815aa10 msgr2=0x7f5be814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815aa10 0x7f5be814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 -- 172.21.15.70:0/3297897070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80b2850 msgr2=0x7f5be8152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80b2850 0x7f5be8152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.590+0000 7f5be7fff640 1 -- 172.21.15.70:0/3297897070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0009a70 con 0x7f5be80ce5b0 2023-08-25T09:42:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.591+0000 7f5be7fff640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 0x7f5be814df80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400c4d0 tx=0x7f5bd400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.591+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd40074e0 con 0x7f5be80ce5b0 2023-08-25T09:42:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.591+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd4007660 con 0x7f5be80ce5b0 2023-08-25T09:42:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.591+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4004a20 con 0x7f5be80ce5b0 2023-08-25T09:42:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.591+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be81e3810 con 0x7f5be80ce5b0 2023-08-25T09:42:28.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.591+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be814eb30 con 0x7f5be80ce5b0 2023-08-25T09:42:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.592+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be81e3d30 con 0x7f5be80ce5b0 2023-08-25T09:42:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.593+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5bd4004b80 con 0x7f5be80ce5b0 2023-08-25T09:42:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.594+0000 7f5be57fa640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bcc06a050 0x7f5bcc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.594+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5bd4087730 con 0x7f5be80ce5b0 2023-08-25T09:42:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.594+0000 7f5becf47640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bcc06a050 0x7f5bcc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:28.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.595+0000 7f5becf47640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bcc06a050 0x7f5bcc06c510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5be80b3110 tx=0x7f5bdc0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:28.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.599+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5be81e3d30 con 0x7f5be80ce5b0 2023-08-25T09:42:28.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.821+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5be81e3d30 con 0x7f5be80ce5b0 2023-08-25T09:42:28.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.822+0000 7f5be57fa640 1 -- 172.21.15.70:0/3297897070 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f5be81e3d30 con 0x7f5be80ce5b0 2023-08-25T09:42:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:28.824 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:42:28.825 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:42:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bcc06a050 msgr2=0x7f5bcc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bcc06a050 0x7f5bcc06c510 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5be80b3110 tx=0x7f5bdc0079b0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 msgr2=0x7f5be814df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 0x7f5be814df80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400c4d0 tx=0x7f5bd400c990 comp rx=0 tx=0).stop 2023-08-25T09:42:28.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 shutdown_connections 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bcc06a050 0x7f5bcc06c510 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815aa10 0x7f5be814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be80ce5b0 0x7f5be814df80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 --2- 172.21.15.70:0/3297897070 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80b2850 0x7f5be8152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.824+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 >> 172.21.15.70:0/3297897070 conn(0x7f5be8069850 msgr2=0x7f5be806bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.825+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 shutdown_connections 2023-08-25T09:42:28.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:28.825+0000 7f5bef1ab640 1 -- 172.21.15.70:0/3297897070 wait complete. 2023-08-25T09:42:28.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:42:29.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.049+0000 7fccfc48e640 1 Processor -- start 2023-08-25T09:42:29.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.049+0000 7fccfc48e640 1 -- start start 2023-08-25T09:42:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfc48e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccf4160ff0 0x7fccf406b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfc48e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fccf4161ae0 0x7fccf40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfc48e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf41620b0 con 0x7fccf40ce5f0 2023-08-25T09:42:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfc48e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf40cefe0 con 0x7fccf4160ff0 2023-08-25T09:42:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfc48e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf406ad70 con 0x7fccf4161ae0 2023-08-25T09:42:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfa22a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccf4160ff0 0x7fccf406b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfa22a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fccf4160ff0 0x7fccf406b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58680/0 (socket says 172.21.15.70:58680) 2023-08-25T09:42:29.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.051+0000 7fccfa22a640 1 -- 172.21.15.70:0/2489874350 learned_addr learned my addr 172.21.15.70:0/2489874350 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:29.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.052+0000 7fccfa22a640 1 -- 172.21.15.70:0/2489874350 >> v2:172.21.15.155:3300/0 conn(0x7fccf4161ae0 msgr2=0x7fccf40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:29.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.052+0000 7fccfa22a640 1 --2- 172.21.15.70:0/2489874350 >> v2:172.21.15.155:3300/0 conn(0x7fccf4161ae0 0x7fccf40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.052+0000 7fccfa22a640 1 -- 172.21.15.70:0/2489874350 >> v1:172.21.15.155:6789/0 conn(0x7fccf40ce5f0 legacy=0x7fccf41613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:29.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.052+0000 7fccfa22a640 1 -- 172.21.15.70:0/2489874350 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4162390 con 0x7fccf4160ff0 2023-08-25T09:42:29.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.053+0000 7fccfa22a640 1 --2- 172.21.15.70:0/2489874350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf406b780 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fccf40b2a30 tx=0x7fccf002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84bf54084779ff7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.053+0000 7fccf9228640 1 -- 172.21.15.70:0/2489874350 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0031870 con 0x7fccf4160ff0 2023-08-25T09:42:29.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.053+0000 7fccf9228640 1 -- 172.21.15.70:0/2489874350 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf00319d0 con 0x7fccf4160ff0 2023-08-25T09:42:29.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.053+0000 7fccf9228640 1 -- 172.21.15.70:0/2489874350 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0031d60 con 0x7fccf4160ff0 2023-08-25T09:42:29.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 -- 172.21.15.70:0/2489874350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 msgr2=0x7fccf406b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 --2- 172.21.15.70:0/2489874350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf406b780 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fccf40b2a30 tx=0x7fccf002b910 comp rx=0 tx=0).stop 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 -- 172.21.15.70:0/2489874350 shutdown_connections 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 --2- 172.21.15.70:0/2489874350 >> v2:172.21.15.155:3300/0 conn(0x7fccf4161ae0 0x7fccf40ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 --2- 172.21.15.70:0/2489874350 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf406b780 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 -- 172.21.15.70:0/2489874350 >> 172.21.15.70:0/2489874350 conn(0x7fccf4069850 msgr2=0x7fccf4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 -- 172.21.15.70:0/2489874350 shutdown_connections 2023-08-25T09:42:29.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.054+0000 7fccfc48e640 1 -- 172.21.15.70:0/2489874350 wait complete. 2023-08-25T09:42:29.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.056+0000 7fccfc48e640 1 Processor -- start 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.056+0000 7fccfc48e640 1 -- start start 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccfc48e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccf40ce5f0 0x7fccf40cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccfc48e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccfc48e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf4161ae0 0x7fccf40af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccfc48e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40cbf90 con 0x7fccf4160ff0 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccfc48e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40cd2e0 con 0x7fccf4161ae0 2023-08-25T09:42:29.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccfc48e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40cc240 con 0x7fccf40ce5f0 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf414f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58682/0 (socket says 172.21.15.70:58682) 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 -- 172.21.15.70:0/198601248 learned_addr learned my addr 172.21.15.70:0/198601248 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 -- 172.21.15.70:0/198601248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccf40ce5f0 msgr2=0x7fccf40cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccf40ce5f0 0x7fccf40cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 -- 172.21.15.70:0/198601248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf4161ae0 msgr2=0x7fccf40af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf4161ae0 0x7fccf40af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.057+0000 7fccf9a29640 1 -- 172.21.15.70:0/198601248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00094d0 con 0x7fccf4160ff0 2023-08-25T09:42:29.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.058+0000 7fccf9a29640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf414f890 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c410 tx=0x7fcce400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.058+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce40035f0 con 0x7fccf4160ff0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.058+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce4003750 con 0x7fccf4160ff0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.058+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf40cc520 con 0x7fccf4160ff0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.058+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce4003b40 con 0x7fccf4160ff0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.058+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf40ccae0 con 0x7fccf4160ff0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.059+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf404e420 con 0x7fccf4160ff0 2023-08-25T09:42:29.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.060+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcce4003ca0 con 0x7fccf4160ff0 2023-08-25T09:42:29.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.061+0000 7fcceaffd640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fccd806a050 0x7fccd806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.061+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcce4087890 con 0x7fccf4160ff0 2023-08-25T09:42:29.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.061+0000 7fccfa22a640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fccd806a050 0x7fccd806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.062+0000 7fccfa22a640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fccd806a050 0x7fccd806c510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fccf0036000 tx=0x7fccf0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.065+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcce4052ee0 con 0x7fccf4160ff0 2023-08-25T09:42:29.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.243+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccf406c8b0 con 0x7fccf4160ff0 2023-08-25T09:42:29.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.245+0000 7fcceaffd640 1 -- 172.21.15.70:0/198601248 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+18355 (secure 0 0 0) 0x7fcce4056500 con 0x7fccf4160ff0 2023-08-25T09:42:29.248 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:29.248 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":73,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:26.282031+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:42:29.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fccd806a050 msgr2=0x7fccd806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fccd806a050 0x7fccd806c510 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fccf0036000 tx=0x7fccf0030000 comp rx=0 tx=0).stop 2023-08-25T09:42:29.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 msgr2=0x7fccf414f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf414f890 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c410 tx=0x7fcce400c8d0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 shutdown_connections 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fccd806a050 0x7fccd806c510 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fccf4161ae0 0x7fccf40af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fccf4160ff0 0x7fccf414f890 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 --2- 172.21.15.70:0/198601248 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fccf40ce5f0 0x7fccf40cba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 >> 172.21.15.70:0/198601248 conn(0x7fccf4069850 msgr2=0x7fccf4070330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 shutdown_connections 2023-08-25T09:42:29.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.248+0000 7fccfc48e640 1 -- 172.21.15.70:0/198601248 wait complete. 2023-08-25T09:42:29.267 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:42:29.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:42:29.278 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-08-25T09:42:29.277+0000 7fd58a2aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:42:29.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.472+0000 7f9f2850c640 1 Processor -- start 2023-08-25T09:42:29.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.472+0000 7f9f2850c640 1 -- start start 2023-08-25T09:42:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f2850c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f200b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f2850c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2006f050 con 0x7f9f200ac870 2023-08-25T09:42:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f2850c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2006f330 con 0x7f9f2015cb40 2023-08-25T09:42:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f2850c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2006f610 con 0x7f9f200cea30 2023-08-25T09:42:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f26aa9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9f200ac870 0x7f9f200b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45328/0 (socket says 172.21.15.70:45328) 2023-08-25T09:42:29.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f26aa9640 1 -- 172.21.15.70:0/4052067473 learned_addr learned my addr 172.21.15.70:0/4052067473 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:29.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f25aa7640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f200b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.474+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143396945 0 0) 0x7f9f2006f050 con 0x7f9f200ac870 2023-08-25T09:42:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f08003520 con 0x7f9f200ac870 2023-08-25T09:42:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683317231 0 0) 0x7f9f08003520 con 0x7f9f200ac870 2023-08-25T09:42:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 msgr2=0x7f9f200b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f200b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 >> v1:172.21.15.155:6789/0 conn(0x7f9f2015cb40 legacy=0x7f9f200ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2006f8f0 con 0x7f9f200ac870 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9f18001670 con 0x7f9f200ac870 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 >> v1:172.21.15.70:6789/0 conn(0x7f9f200ac870 legacy=0x7f9f200b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080039a0 0x7f9f08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f080047c0 0x7f9f08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2006f610 con 0x7f9f080039a0 2023-08-25T09:42:29.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0800ba30 con 0x7f9f080047c0 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0800bbf0 con 0x7f9f200cea30 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.475+0000 7f9f262a8640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080039a0 0x7f9f08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f25aa7640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f080047c0 0x7f9f08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f26aa9640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f262a8640 1 -- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 msgr2=0x7f9f08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f262a8640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f08005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f262a8640 1 -- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f080047c0 msgr2=0x7f9f08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f262a8640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f080047c0 0x7f9f08004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f262a8640 1 -- 172.21.15.70:0/4052067473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f08003520 con 0x7f9f080039a0 2023-08-25T09:42:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f25aa7640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f080047c0 0x7f9f08004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:29.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.476+0000 7f9f26aa9640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f200cea30 0x7f9f08005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.477+0000 7f9f262a8640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080039a0 0x7f9f08003dc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f10001e20 tx=0x7f9f080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147b1c18eaf740e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.477+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10003b70 con 0x7f9f080039a0 2023-08-25T09:42:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.477+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f10003cd0 con 0x7f9f080039a0 2023-08-25T09:42:29.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.477+0000 7f9f252a6640 1 -- 172.21.15.70:0/4052067473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f10005310 con 0x7f9f080039a0 2023-08-25T09:42:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f2850c640 1 -- 172.21.15.70:0/4052067473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080039a0 msgr2=0x7f9f08003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f2850c640 1 --2- 172.21.15.70:0/4052067473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f080039a0 0x7f9f08003dc0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f10001e20 tx=0x7f9f080057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f26aa9640 1 -- 172.21.15.70:0/4052067473 reap_dead start 2023-08-25T09:42:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f2850c640 1 -- 172.21.15.70:0/4052067473 shutdown_connections 2023-08-25T09:42:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f2850c640 1 -- 172.21.15.70:0/4052067473 >> 172.21.15.70:0/4052067473 conn(0x7f9f20069850 msgr2=0x7f9f20069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:29.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f2850c640 1 -- 172.21.15.70:0/4052067473 shutdown_connections 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.478+0000 7f9f2850c640 1 -- 172.21.15.70:0/4052067473 wait complete. 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.479+0000 7f9f2850c640 1 Processor -- start 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.480+0000 7f9f2850c640 1 -- start start 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.480+0000 7f9f2850c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f080039a0 0x7f9f2014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.480+0000 7f9f2850c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 0x7f9f200cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f2850c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f200cd690 0x7f9f2015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f2850c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2014b560 con 0x7f9f2014bb90 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f2850c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f200cdb50 con 0x7f9f200cd690 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f2850c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f20158180 con 0x7f9f080039a0 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 0x7f9f200cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 0x7f9f200cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58704/0 (socket says 172.21.15.70:58704) 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 -- 172.21.15.70:0/483963657 learned_addr learned my addr 172.21.15.70:0/483963657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 -- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f080039a0 msgr2=0x7f9f2014b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f262a8640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f080039a0 0x7f9f2014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f080039a0 0x7f9f2014b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 -- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f200cd690 msgr2=0x7f9f2015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f200cd690 0x7f9f2015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 -- 172.21.15.70:0/483963657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f20158460 con 0x7f9f2014bb90 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.481+0000 7f9f25aa7640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 0x7f9f200cd150 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f140108b0 tx=0x7f9f14010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.482+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400eb60 con 0x7f9f2014bb90 2023-08-25T09:42:29.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.482+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1400ecc0 con 0x7f9f2014bb90 2023-08-25T09:42:29.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.482+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1401c400 con 0x7f9f2014bb90 2023-08-25T09:42:29.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.482+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f20158740 con 0x7f9f2014bb90 2023-08-25T09:42:29.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.482+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2014e930 con 0x7f9f2014bb90 2023-08-25T09:42:29.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.483+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2014ebb0 con 0x7f9f2014bb90 2023-08-25T09:42:29.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.484+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f1401c560 con 0x7f9f2014bb90 2023-08-25T09:42:29.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.485+0000 7f9f1effd640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f0c06a050 0x7f9f0c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.486+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f14088b30 con 0x7f9f2014bb90 2023-08-25T09:42:29.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.486+0000 7f9f262a8640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f0c06a050 0x7f9f0c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.486+0000 7f9f262a8640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f0c06a050 0x7f9f0c06c510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f10034000 tx=0x7f9f10039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.490+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9f140542c0 con 0x7f9f2014bb90 2023-08-25T09:42:29.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.677+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f2006ece0 con 0x7f9f0c06a050 2023-08-25T09:42:29.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.692+0000 7f9f1effd640 1 -- 172.21.15.70:0/483963657 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303174 (secure 0 0 0) 0x7f9f10003af0 con 0x7f9f0c06a050 2023-08-25T09:42:29.695 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:29.700 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:42:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f0c06a050 msgr2=0x7f9f0c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f0c06a050 0x7f9f0c06c510 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9f10034000 tx=0x7f9f10039040 comp rx=0 tx=0).stop 2023-08-25T09:42:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 msgr2=0x7f9f200cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 0x7f9f200cd150 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f140108b0 tx=0x7f9f14010d70 comp rx=0 tx=0).stop 2023-08-25T09:42:29.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 shutdown_connections 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f0c06a050 0x7f9f0c06c510 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f200cd690 0x7f9f2015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f2014bb90 0x7f9f200cd150 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 --2- 172.21.15.70:0/483963657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f080039a0 0x7f9f2014b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 >> 172.21.15.70:0/483963657 conn(0x7f9f20069850 msgr2=0x7f9f200ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 shutdown_connections 2023-08-25T09:42:29.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.699+0000 7f9f2850c640 1 -- 172.21.15.70:0/483963657 wait complete. 2023-08-25T09:42:29.721 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-08-25T09:42:29.389158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4281,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558218,"statfs":{"total":1600000000,"available":1595615700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001588"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293733+0000","last_change":"2023-08-25T09:42:25.313741+0000","last_active":"2023-08-25T09:42:26.293733+0000","last_peered":"2023-08-25T09:42:26.293733+0000","last_clean":"2023-08-25T09:42:26.293733+0000","last_became_active":"2023-08-25T09:42:25.313118+0000","last_became_peered":"2023-08-25T09:42:25.313118+0000","last_unstale":"2023-08-25T09:42:26.293733+0000","last_undegraded":"2023-08-25T09:42:26.293733+0000","last_fullsized":"2023-08-25T09:42:26.293733+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:14.994192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959148+0000","last_change":"2023-08-25T09:42:25.310740+0000","last_active":"2023-08-25T09:42:26.959148+0000","last_peered":"2023-08-25T09:42:26.959148+0000","last_clean":"2023-08-25T09:42:26.959148+0000","last_became_active":"2023-08-25T09:42:25.310443+0000","last_became_peered":"2023-08-25T09:42:25.310443+0000","last_unstale":"2023-08-25T09:42:26.959148+0000","last_undegraded":"2023-08-25T09:42:26.959148+0000","last_fullsized":"2023-08-25T09:42:26.959148+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:41:10.686668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287278+0000","last_change":"2023-08-25T09:42:25.311910+0000","last_active":"2023-08-25T09:42:26.287278+0000","last_peered":"2023-08-25T09:42:26.287278+0000","last_clean":"2023-08-25T09:42:26.287278+0000","last_became_active":"2023-08-25T09:42:25.311596+0000","last_became_peered":"2023-08-25T09:42:25.311596+0000","last_unstale":"2023-08-25T09:42:26.287278+0000","last_undegraded":"2023-08-25T09:42:26.287278+0000","last_fullsized":"2023-08-25T09:42:26.287278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:58:48.112882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835224+0000","last_change":"2023-08-25T09:42:25.303222+0000","last_active":"2023-08-25T09:42:26.835224+0000","last_peered":"2023-08-25T09:42:26.835224+0000","last_clean":"2023-08-25T09:42:26.835224+0000","last_became_active":"2023-08-25T09:42:25.302620+0000","last_became_peered":"2023-08-25T09:42:25.302620+0000","last_unstale":"2023-08-25T09:42:26.835224+0000","last_undegraded":"2023-08-25T09:42:26.835224+0000","last_fullsized":"2023-08-25T09:42:26.835224+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:58.439459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287593+0000","last_change":"2023-08-25T09:42:25.294257+0000","last_active":"2023-08-25T09:42:26.287593+0000","last_peered":"2023-08-25T09:42:26.287593+0000","last_clean":"2023-08-25T09:42:26.287593+0000","last_became_active":"2023-08-25T09:42:25.293209+0000","last_became_peered":"2023-08-25T09:42:25.293209+0000","last_unstale":"2023-08-25T09:42:26.287593+0000","last_undegraded":"2023-08-25T09:42:26.287593+0000","last_fullsized":"2023-08-25T09:42:26.287593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:38.507153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959537+0000","last_change":"2023-08-25T09:42:25.309747+0000","last_active":"2023-08-25T09:42:26.959537+0000","last_peered":"2023-08-25T09:42:26.959537+0000","last_clean":"2023-08-25T09:42:26.959537+0000","last_became_active":"2023-08-25T09:42:25.308811+0000","last_became_peered":"2023-08-25T09:42:25.308811+0000","last_unstale":"2023-08-25T09:42:26.959537+0000","last_undegraded":"2023-08-25T09:42:26.959537+0000","last_fullsized":"2023-08-25T09:42:26.959537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:23:07.802680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291554+0000","last_change":"2023-08-25T09:42:25.318991+0000","last_active":"2023-08-25T09:42:26.291554+0000","last_peered":"2023-08-25T09:42:26.291554+0000","last_clean":"2023-08-25T09:42:26.291554+0000","last_became_active":"2023-08-25T09:42:25.318693+0000","last_became_peered":"2023-08-25T09:42:25.318693+0000","last_unstale":"2023-08-25T09:42:26.291554+0000","last_undegraded":"2023-08-25T09:42:26.291554+0000","last_fullsized":"2023-08-25T09:42:26.291554+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:07.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287704+0000","last_change":"2023-08-25T09:42:25.294147+0000","last_active":"2023-08-25T09:42:26.287704+0000","last_peered":"2023-08-25T09:42:26.287704+0000","last_clean":"2023-08-25T09:42:26.287704+0000","last_became_active":"2023-08-25T09:42:25.293879+0000","last_became_peered":"2023-08-25T09:42:25.293879+0000","last_unstale":"2023-08-25T09:42:26.287704+0000","last_undegraded":"2023-08-25T09:42:26.287704+0000","last_fullsized":"2023-08-25T09:42:26.287704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:20.804220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836350+0000","last_change":"2023-08-25T09:42:25.296158+0000","last_active":"2023-08-25T09:42:26.836350+0000","last_peered":"2023-08-25T09:42:26.836350+0000","last_clean":"2023-08-25T09:42:26.836350+0000","last_became_active":"2023-08-25T09:42:25.295931+0000","last_became_peered":"2023-08-25T09:42:25.295931+0000","last_unstale":"2023-08-25T09:42:26.836350+0000","last_undegraded":"2023-08-25T09:42:26.836350+0000","last_fullsized":"2023-08-25T09:42:26.836350+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:21:02.657884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958409+0000","last_change":"2023-08-25T09:42:25.308724+0000","last_active":"2023-08-25T09:42:26.958409+0000","last_peered":"2023-08-25T09:42:26.958409+0000","last_clean":"2023-08-25T09:42:26.958409+0000","last_became_active":"2023-08-25T09:42:25.307914+0000","last_became_peered":"2023-08-25T09:42:25.307914+0000","last_unstale":"2023-08-25T09:42:26.958409+0000","last_undegraded":"2023-08-25T09:42:26.958409+0000","last_fullsized":"2023-08-25T09:42:26.958409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:54.097673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835532+0000","last_change":"2023-08-25T09:42:25.307697+0000","last_active":"2023-08-25T09:42:26.835532+0000","last_peered":"2023-08-25T09:42:26.835532+0000","last_clean":"2023-08-25T09:42:26.835532+0000","last_became_active":"2023-08-25T09:42:25.307150+0000","last_became_peered":"2023-08-25T09:42:25.307150+0000","last_unstale":"2023-08-25T09:42:26.835532+0000","last_undegraded":"2023-08-25T09:42:26.835532+0000","last_fullsized":"2023-08-25T09:42:26.835532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:37:49.817626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835932+0000","last_change":"2023-08-25T09:42:25.299224+0000","last_active":"2023-08-25T09:42:26.835932+0000","last_peered":"2023-08-25T09:42:26.835932+0000","last_clean":"2023-08-25T09:42:26.835932+0000","last_became_active":"2023-08-25T09:42:25.297844+0000","last_became_peered":"2023-08-25T09:42:25.297844+0000","last_unstale":"2023-08-25T09:42:26.835932+0000","last_undegraded":"2023-08-25T09:42:26.835932+0000","last_fullsized":"2023-08-25T09:42:26.835932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:02:27.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.614869+0000","last_change":"2023-08-25T09:42:25.307456+0000","last_active":"2023-08-25T09:42:26.614869+0000","last_peered":"2023-08-25T09:42:26.614869+0000","last_clean":"2023-08-25T09:42:26.614869+0000","last_became_active":"2023-08-25T09:42:25.307041+0000","last_became_peered":"2023-08-25T09:42:25.307041+0000","last_unstale":"2023-08-25T09:42:26.614869+0000","last_undegraded":"2023-08-25T09:42:26.614869+0000","last_fullsized":"2023-08-25T09:42:26.614869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:37.616311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615264+0000","last_change":"2023-08-25T09:42:25.310305+0000","last_active":"2023-08-25T09:42:26.615264+0000","last_peered":"2023-08-25T09:42:26.615264+0000","last_clean":"2023-08-25T09:42:26.615264+0000","last_became_active":"2023-08-25T09:42:25.310049+0000","last_became_peered":"2023-08-25T09:42:25.310049+0000","last_unstale":"2023-08-25T09:42:26.615264+0000","last_undegraded":"2023-08-25T09:42:26.615264+0000","last_fullsized":"2023-08-25T09:42:26.615264+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:18:39.592685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958581+0000","last_change":"2023-08-25T09:42:25.308876+0000","last_active":"2023-08-25T09:42:26.958581+0000","last_peered":"2023-08-25T09:42:26.958581+0000","last_clean":"2023-08-25T09:42:26.958581+0000","last_became_active":"2023-08-25T09:42:25.308300+0000","last_became_peered":"2023-08-25T09:42:25.308300+0000","last_unstale":"2023-08-25T09:42:26.958581+0000","last_undegraded":"2023-08-25T09:42:26.958581+0000","last_fullsized":"2023-08-25T09:42:26.958581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:38:35.707512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293322+0000","last_change":"2023-08-25T09:42:25.314067+0000","last_active":"2023-08-25T09:42:26.293322+0000","last_peered":"2023-08-25T09:42:26.293322+0000","last_clean":"2023-08-25T09:42:26.293322+0000","last_became_active":"2023-08-25T09:42:25.313642+0000","last_became_peered":"2023-08-25T09:42:25.313642+0000","last_unstale":"2023-08-25T09:42:26.293322+0000","last_undegraded":"2023-08-25T09:42:26.293322+0000","last_fullsized":"2023-08-25T09:42:26.293322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:21.405708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959680+0000","last_change":"2023-08-25T09:42:25.318667+0000","last_active":"2023-08-25T09:42:26.959680+0000","last_peered":"2023-08-25T09:42:26.959680+0000","last_clean":"2023-08-25T09:42:26.959680+0000","last_became_active":"2023-08-25T09:42:25.316927+0000","last_became_peered":"2023-08-25T09:42:25.316927+0000","last_unstale":"2023-08-25T09:42:26.959680+0000","last_undegraded":"2023-08-25T09:42:26.959680+0000","last_fullsized":"2023-08-25T09:42:26.959680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:11:04.711050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836504+0000","last_change":"2023-08-25T09:42:25.311991+0000","last_active":"2023-08-25T09:42:26.836504+0000","last_peered":"2023-08-25T09:42:26.836504+0000","last_clean":"2023-08-25T09:42:26.836504+0000","last_became_active":"2023-08-25T09:42:25.311712+0000","last_became_peered":"2023-08-25T09:42:25.311712+0000","last_unstale":"2023-08-25T09:42:26.836504+0000","last_undegraded":"2023-08-25T09:42:26.836504+0000","last_fullsized":"2023-08-25T09:42:26.836504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:14.864127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835654+0000","last_change":"2023-08-25T09:42:25.309543+0000","last_active":"2023-08-25T09:42:26.835654+0000","last_peered":"2023-08-25T09:42:26.835654+0000","last_clean":"2023-08-25T09:42:26.835654+0000","last_became_active":"2023-08-25T09:42:25.309186+0000","last_became_peered":"2023-08-25T09:42:25.309186+0000","last_unstale":"2023-08-25T09:42:26.835654+0000","last_undegraded":"2023-08-25T09:42:26.835654+0000","last_fullsized":"2023-08-25T09:42:26.835654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:15:20.503161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959720+0000","last_change":"2023-08-25T09:42:25.306746+0000","last_active":"2023-08-25T09:42:26.959720+0000","last_peered":"2023-08-25T09:42:26.959720+0000","last_clean":"2023-08-25T09:42:26.959720+0000","last_became_active":"2023-08-25T09:42:25.304143+0000","last_became_peered":"2023-08-25T09:42:25.304143+0000","last_unstale":"2023-08-25T09:42:26.959720+0000","last_undegraded":"2023-08-25T09:42:26.959720+0000","last_fullsized":"2023-08-25T09:42:26.959720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:16:44.203926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615322+0000","last_change":"2023-08-25T09:42:25.302887+0000","last_active":"2023-08-25T09:42:26.615322+0000","last_peered":"2023-08-25T09:42:26.615322+0000","last_clean":"2023-08-25T09:42:26.615322+0000","last_became_active":"2023-08-25T09:42:25.302336+0000","last_became_peered":"2023-08-25T09:42:25.302336+0000","last_unstale":"2023-08-25T09:42:26.615322+0000","last_undegraded":"2023-08-25T09:42:26.615322+0000","last_fullsized":"2023-08-25T09:42:26.615322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:29.208977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959523+0000","last_change":"2023-08-25T09:42:25.309100+0000","last_active":"2023-08-25T09:42:26.959523+0000","last_peered":"2023-08-25T09:42:26.959523+0000","last_clean":"2023-08-25T09:42:26.959523+0000","last_became_active":"2023-08-25T09:42:25.308130+0000","last_became_peered":"2023-08-25T09:42:25.308130+0000","last_unstale":"2023-08-25T09:42:26.959523+0000","last_undegraded":"2023-08-25T09:42:26.959523+0000","last_fullsized":"2023-08-25T09:42:26.959523+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:38:59.737305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291510+0000","last_change":"2023-08-25T09:42:25.312306+0000","last_active":"2023-08-25T09:42:26.291510+0000","last_peered":"2023-08-25T09:42:26.291510+0000","last_clean":"2023-08-25T09:42:26.291510+0000","last_became_active":"2023-08-25T09:42:25.311957+0000","last_became_peered":"2023-08-25T09:42:25.311957+0000","last_unstale":"2023-08-25T09:42:26.291510+0000","last_undegraded":"2023-08-25T09:42:26.291510+0000","last_fullsized":"2023-08-25T09:42:26.291510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:16.474501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287095+0000","last_change":"2023-08-25T09:42:25.310727+0000","last_active":"2023-08-25T09:42:26.287095+0000","last_peered":"2023-08-25T09:42:26.287095+0000","last_clean":"2023-08-25T09:42:26.287095+0000","last_became_active":"2023-08-25T09:42:25.310457+0000","last_became_peered":"2023-08-25T09:42:25.310457+0000","last_unstale":"2023-08-25T09:42:26.287095+0000","last_undegraded":"2023-08-25T09:42:26.287095+0000","last_fullsized":"2023-08-25T09:42:26.287095+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:31.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615441+0000","last_change":"2023-08-25T09:42:25.306018+0000","last_active":"2023-08-25T09:42:26.615441+0000","last_peered":"2023-08-25T09:42:26.615441+0000","last_clean":"2023-08-25T09:42:26.615441+0000","last_became_active":"2023-08-25T09:42:25.305638+0000","last_became_peered":"2023-08-25T09:42:25.305638+0000","last_unstale":"2023-08-25T09:42:26.615441+0000","last_undegraded":"2023-08-25T09:42:26.615441+0000","last_fullsized":"2023-08-25T09:42:26.615441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:28:44.356442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287586+0000","last_change":"2023-08-25T09:42:25.306734+0000","last_active":"2023-08-25T09:42:26.287586+0000","last_peered":"2023-08-25T09:42:26.287586+0000","last_clean":"2023-08-25T09:42:26.287586+0000","last_became_active":"2023-08-25T09:42:25.306325+0000","last_became_peered":"2023-08-25T09:42:25.306325+0000","last_unstale":"2023-08-25T09:42:26.287586+0000","last_undegraded":"2023-08-25T09:42:26.287586+0000","last_fullsized":"2023-08-25T09:42:26.287586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:48.184593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615412+0000","last_change":"2023-08-25T09:42:25.314653+0000","last_active":"2023-08-25T09:42:26.615412+0000","last_peered":"2023-08-25T09:42:26.615412+0000","last_clean":"2023-08-25T09:42:26.615412+0000","last_became_active":"2023-08-25T09:42:25.314372+0000","last_became_peered":"2023-08-25T09:42:25.314372+0000","last_unstale":"2023-08-25T09:42:26.615412+0000","last_undegraded":"2023-08-25T09:42:26.615412+0000","last_fullsized":"2023-08-25T09:42:26.615412+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:00.467750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292201+0000","last_change":"2023-08-25T09:42:25.311064+0000","last_active":"2023-08-25T09:42:26.292201+0000","last_peered":"2023-08-25T09:42:26.292201+0000","last_clean":"2023-08-25T09:42:26.292201+0000","last_became_active":"2023-08-25T09:42:25.309531+0000","last_became_peered":"2023-08-25T09:42:25.309531+0000","last_unstale":"2023-08-25T09:42:26.292201+0000","last_undegraded":"2023-08-25T09:42:26.292201+0000","last_fullsized":"2023-08-25T09:42:26.292201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:00:19.936765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835431+0000","last_change":"2023-08-25T09:42:25.300679+0000","last_active":"2023-08-25T09:42:26.835431+0000","last_peered":"2023-08-25T09:42:26.835431+0000","last_clean":"2023-08-25T09:42:26.835431+0000","last_became_active":"2023-08-25T09:42:25.300224+0000","last_became_peered":"2023-08-25T09:42:25.300224+0000","last_unstale":"2023-08-25T09:42:26.835431+0000","last_undegraded":"2023-08-25T09:42:26.835431+0000","last_fullsized":"2023-08-25T09:42:26.835431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:26:57.958880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836311+0000","last_change":"2023-08-25T09:42:25.302735+0000","last_active":"2023-08-25T09:42:26.836311+0000","last_peered":"2023-08-25T09:42:26.836311+0000","last_clean":"2023-08-25T09:42:26.836311+0000","last_became_active":"2023-08-25T09:42:25.302467+0000","last_became_peered":"2023-08-25T09:42:25.302467+0000","last_unstale":"2023-08-25T09:42:26.836311+0000","last_undegraded":"2023-08-25T09:42:26.836311+0000","last_fullsized":"2023-08-25T09:42:26.836311+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:16.735134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288245+0000","last_change":"2023-08-25T09:42:25.312535+0000","last_active":"2023-08-25T09:42:26.288245+0000","last_peered":"2023-08-25T09:42:26.288245+0000","last_clean":"2023-08-25T09:42:26.288245+0000","last_became_active":"2023-08-25T09:42:25.312129+0000","last_became_peered":"2023-08-25T09:42:25.312129+0000","last_unstale":"2023-08-25T09:42:26.288245+0000","last_undegraded":"2023-08-25T09:42:26.288245+0000","last_fullsized":"2023-08-25T09:42:26.288245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:49:00.167189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292696+0000","last_change":"2023-08-25T09:42:25.311364+0000","last_active":"2023-08-25T09:42:26.292696+0000","last_peered":"2023-08-25T09:42:26.292696+0000","last_clean":"2023-08-25T09:42:26.292696+0000","last_became_active":"2023-08-25T09:42:25.311111+0000","last_became_peered":"2023-08-25T09:42:25.311111+0000","last_unstale":"2023-08-25T09:42:26.292696+0000","last_undegraded":"2023-08-25T09:42:26.292696+0000","last_fullsized":"2023-08-25T09:42:26.292696+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:26:57.958880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836115+0000","last_change":"2023-08-25T09:42:25.307472+0000","last_active":"2023-08-25T09:42:26.836115+0000","last_peered":"2023-08-25T09:42:26.836115+0000","last_clean":"2023-08-25T09:42:26.836115+0000","last_became_active":"2023-08-25T09:42:25.304647+0000","last_became_peered":"2023-08-25T09:42:25.304647+0000","last_unstale":"2023-08-25T09:42:26.836115+0000","last_undegraded":"2023-08-25T09:42:26.836115+0000","last_fullsized":"2023-08-25T09:42:26.836115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:01:16.816662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"72'4","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292521+0000","last_change":"2023-08-25T09:42:24.304286+0000","last_active":"2023-08-25T09:42:26.292521+0000","last_peered":"2023-08-25T09:42:26.292521+0000","last_clean":"2023-08-25T09:42:26.292521+0000","last_became_active":"2023-08-25T09:42:24.304050+0000","last_became_peered":"2023-08-25T09:42:24.304050+0000","last_unstale":"2023-08-25T09:42:26.292521+0000","last_undegraded":"2023-08-25T09:42:26.292521+0000","last_fullsized":"2023-08-25T09:42:26.292521+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:57.433491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287792+0000","last_change":"2023-08-25T09:42:25.294485+0000","last_active":"2023-08-25T09:42:26.287792+0000","last_peered":"2023-08-25T09:42:26.287792+0000","last_clean":"2023-08-25T09:42:26.287792+0000","last_became_active":"2023-08-25T09:42:25.293864+0000","last_became_peered":"2023-08-25T09:42:25.293864+0000","last_unstale":"2023-08-25T09:42:26.287792+0000","last_undegraded":"2023-08-25T09:42:26.287792+0000","last_fullsized":"2023-08-25T09:42:26.287792+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:15:56.713821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292389+0000","last_change":"2023-08-25T09:42:24.291550+0000","last_active":"2023-08-25T09:42:26.292389+0000","last_peered":"2023-08-25T09:42:26.292389+0000","last_clean":"2023-08-25T09:42:26.292389+0000","last_became_active":"2023-08-25T09:42:24.290087+0000","last_became_peered":"2023-08-25T09:42:24.290087+0000","last_unstale":"2023-08-25T09:42:26.292389+0000","last_undegraded":"2023-08-25T09:42:26.292389+0000","last_fullsized":"2023-08-25T09:42:26.292389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:23:02.401341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959587+0000","last_change":"2023-08-25T09:42:25.311690+0000","last_active":"2023-08-25T09:42:26.959587+0000","last_peered":"2023-08-25T09:42:26.959587+0000","last_clean":"2023-08-25T09:42:26.959587+0000","last_became_active":"2023-08-25T09:42:25.311192+0000","last_became_peered":"2023-08-25T09:42:25.311192+0000","last_unstale":"2023-08-25T09:42:26.959587+0000","last_undegraded":"2023-08-25T09:42:26.959587+0000","last_fullsized":"2023-08-25T09:42:26.959587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:39.023744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293667+0000","last_change":"2023-08-25T09:42:24.305143+0000","last_active":"2023-08-25T09:42:26.293667+0000","last_peered":"2023-08-25T09:42:26.293667+0000","last_clean":"2023-08-25T09:42:26.293667+0000","last_became_active":"2023-08-25T09:42:24.304889+0000","last_became_peered":"2023-08-25T09:42:24.304889+0000","last_unstale":"2023-08-25T09:42:26.293667+0000","last_undegraded":"2023-08-25T09:42:26.293667+0000","last_fullsized":"2023-08-25T09:42:26.293667+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:02:26.502294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836829+0000","last_change":"2023-08-25T09:42:25.306837+0000","last_active":"2023-08-25T09:42:26.836829+0000","last_peered":"2023-08-25T09:42:26.836829+0000","last_clean":"2023-08-25T09:42:26.836829+0000","last_became_active":"2023-08-25T09:42:25.305194+0000","last_became_peered":"2023-08-25T09:42:25.305194+0000","last_unstale":"2023-08-25T09:42:26.836829+0000","last_undegraded":"2023-08-25T09:42:26.836829+0000","last_fullsized":"2023-08-25T09:42:26.836829+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:15:44.243511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960437+0000","last_change":"2023-08-25T09:42:24.307070+0000","last_active":"2023-08-25T09:42:26.960437+0000","last_peered":"2023-08-25T09:42:26.960437+0000","last_clean":"2023-08-25T09:42:26.960437+0000","last_became_active":"2023-08-25T09:42:24.305606+0000","last_became_peered":"2023-08-25T09:42:24.305606+0000","last_unstale":"2023-08-25T09:42:26.960437+0000","last_undegraded":"2023-08-25T09:42:26.960437+0000","last_fullsized":"2023-08-25T09:42:26.960437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:16.112292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960090+0000","last_change":"2023-08-25T09:42:25.317917+0000","last_active":"2023-08-25T09:42:26.960090+0000","last_peered":"2023-08-25T09:42:26.960090+0000","last_clean":"2023-08-25T09:42:26.960090+0000","last_became_active":"2023-08-25T09:42:25.308995+0000","last_became_peered":"2023-08-25T09:42:25.308995+0000","last_unstale":"2023-08-25T09:42:26.960090+0000","last_undegraded":"2023-08-25T09:42:26.960090+0000","last_fullsized":"2023-08-25T09:42:26.960090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:15.399316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960349+0000","last_change":"2023-08-25T09:42:24.305408+0000","last_active":"2023-08-25T09:42:26.960349+0000","last_peered":"2023-08-25T09:42:26.960349+0000","last_clean":"2023-08-25T09:42:26.960349+0000","last_became_active":"2023-08-25T09:42:24.304874+0000","last_became_peered":"2023-08-25T09:42:24.304874+0000","last_unstale":"2023-08-25T09:42:26.960349+0000","last_undegraded":"2023-08-25T09:42:26.960349+0000","last_fullsized":"2023-08-25T09:42:26.960349+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:53:02.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291635+0000","last_change":"2023-08-25T09:42:25.306618+0000","last_active":"2023-08-25T09:42:26.291635+0000","last_peered":"2023-08-25T09:42:26.291635+0000","last_clean":"2023-08-25T09:42:26.291635+0000","last_became_active":"2023-08-25T09:42:25.306231+0000","last_became_peered":"2023-08-25T09:42:25.306231+0000","last_unstale":"2023-08-25T09:42:26.291635+0000","last_undegraded":"2023-08-25T09:42:26.291635+0000","last_fullsized":"2023-08-25T09:42:26.291635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:10:41.738975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959375+0000","last_change":"2023-08-25T09:42:24.306085+0000","last_active":"2023-08-25T09:42:26.959375+0000","last_peered":"2023-08-25T09:42:26.959375+0000","last_clean":"2023-08-25T09:42:26.959375+0000","last_became_active":"2023-08-25T09:42:24.305829+0000","last_became_peered":"2023-08-25T09:42:24.305829+0000","last_unstale":"2023-08-25T09:42:26.959375+0000","last_undegraded":"2023-08-25T09:42:26.959375+0000","last_fullsized":"2023-08-25T09:42:26.959375+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:38:02.470521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957904+0000","last_change":"2023-08-25T09:42:25.301456+0000","last_active":"2023-08-25T09:42:26.957904+0000","last_peered":"2023-08-25T09:42:26.957904+0000","last_clean":"2023-08-25T09:42:26.957904+0000","last_became_active":"2023-08-25T09:42:25.298185+0000","last_became_peered":"2023-08-25T09:42:25.298185+0000","last_unstale":"2023-08-25T09:42:26.957904+0000","last_undegraded":"2023-08-25T09:42:26.957904+0000","last_fullsized":"2023-08-25T09:42:26.957904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:17.613490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293161+0000","last_change":"2023-08-25T09:42:24.292740+0000","last_active":"2023-08-25T09:42:26.293161+0000","last_peered":"2023-08-25T09:42:26.293161+0000","last_clean":"2023-08-25T09:42:26.293161+0000","last_became_active":"2023-08-25T09:42:24.292159+0000","last_became_peered":"2023-08-25T09:42:24.292159+0000","last_unstale":"2023-08-25T09:42:26.293161+0000","last_undegraded":"2023-08-25T09:42:26.293161+0000","last_fullsized":"2023-08-25T09:42:26.293161+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:15:19.497193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959323+0000","last_change":"2023-08-25T09:42:25.318093+0000","last_active":"2023-08-25T09:42:26.959323+0000","last_peered":"2023-08-25T09:42:26.959323+0000","last_clean":"2023-08-25T09:42:26.959323+0000","last_became_active":"2023-08-25T09:42:25.316206+0000","last_became_peered":"2023-08-25T09:42:25.316206+0000","last_unstale":"2023-08-25T09:42:26.959323+0000","last_undegraded":"2023-08-25T09:42:26.959323+0000","last_fullsized":"2023-08-25T09:42:26.959323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:08.384644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836628+0000","last_change":"2023-08-25T09:42:24.307057+0000","last_active":"2023-08-25T09:42:26.836628+0000","last_peered":"2023-08-25T09:42:26.836628+0000","last_clean":"2023-08-25T09:42:26.836628+0000","last_became_active":"2023-08-25T09:42:24.306810+0000","last_became_peered":"2023-08-25T09:42:24.306810+0000","last_unstale":"2023-08-25T09:42:26.836628+0000","last_undegraded":"2023-08-25T09:42:26.836628+0000","last_fullsized":"2023-08-25T09:42:26.836628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:21.502328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292020+0000","last_change":"2023-08-25T09:42:25.312120+0000","last_active":"2023-08-25T09:42:26.292020+0000","last_peered":"2023-08-25T09:42:26.292020+0000","last_clean":"2023-08-25T09:42:26.292020+0000","last_became_active":"2023-08-25T09:42:25.311822+0000","last_became_peered":"2023-08-25T09:42:25.311822+0000","last_unstale":"2023-08-25T09:42:26.292020+0000","last_undegraded":"2023-08-25T09:42:26.292020+0000","last_fullsized":"2023-08-25T09:42:26.292020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:12:54.337914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836465+0000","last_change":"2023-08-25T09:42:24.307403+0000","last_active":"2023-08-25T09:42:26.836465+0000","last_peered":"2023-08-25T09:42:26.836465+0000","last_clean":"2023-08-25T09:42:26.836465+0000","last_became_active":"2023-08-25T09:42:24.307148+0000","last_became_peered":"2023-08-25T09:42:24.307148+0000","last_unstale":"2023-08-25T09:42:26.836465+0000","last_undegraded":"2023-08-25T09:42:26.836465+0000","last_fullsized":"2023-08-25T09:42:26.836465+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:52:45.387058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960380+0000","last_change":"2023-08-25T09:42:25.309429+0000","last_active":"2023-08-25T09:42:26.960380+0000","last_peered":"2023-08-25T09:42:26.960380+0000","last_clean":"2023-08-25T09:42:26.960380+0000","last_became_active":"2023-08-25T09:42:25.309094+0000","last_became_peered":"2023-08-25T09:42:25.309094+0000","last_unstale":"2023-08-25T09:42:26.960380+0000","last_undegraded":"2023-08-25T09:42:26.960380+0000","last_fullsized":"2023-08-25T09:42:26.960380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:47.499893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615885+0000","last_change":"2023-08-25T09:42:24.302558+0000","last_active":"2023-08-25T09:42:26.615885+0000","last_peered":"2023-08-25T09:42:26.615885+0000","last_clean":"2023-08-25T09:42:26.615885+0000","last_became_active":"2023-08-25T09:42:24.302055+0000","last_became_peered":"2023-08-25T09:42:24.302055+0000","last_unstale":"2023-08-25T09:42:26.615885+0000","last_undegraded":"2023-08-25T09:42:26.615885+0000","last_fullsized":"2023-08-25T09:42:26.615885+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:19.070872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615617+0000","last_change":"2023-08-25T09:42:25.304157+0000","last_active":"2023-08-25T09:42:26.615617+0000","last_peered":"2023-08-25T09:42:26.615617+0000","last_clean":"2023-08-25T09:42:26.615617+0000","last_became_active":"2023-08-25T09:42:25.303877+0000","last_became_peered":"2023-08-25T09:42:25.303877+0000","last_unstale":"2023-08-25T09:42:26.615617+0000","last_undegraded":"2023-08-25T09:42:26.615617+0000","last_fullsized":"2023-08-25T09:42:26.615617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:34:59.383014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"72'3","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288534+0000","last_change":"2023-08-25T09:42:24.307353+0000","last_active":"2023-08-25T09:42:26.288534+0000","last_peered":"2023-08-25T09:42:26.288534+0000","last_clean":"2023-08-25T09:42:26.288534+0000","last_became_active":"2023-08-25T09:42:24.305922+0000","last_became_peered":"2023-08-25T09:42:24.305922+0000","last_unstale":"2023-08-25T09:42:26.288534+0000","last_undegraded":"2023-08-25T09:42:26.288534+0000","last_fullsized":"2023-08-25T09:42:26.288534+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:23.839075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287949+0000","last_change":"2023-08-25T09:42:25.312360+0000","last_active":"2023-08-25T09:42:26.287949+0000","last_peered":"2023-08-25T09:42:26.287949+0000","last_clean":"2023-08-25T09:42:26.287949+0000","last_became_active":"2023-08-25T09:42:25.311723+0000","last_became_peered":"2023-08-25T09:42:25.311723+0000","last_unstale":"2023-08-25T09:42:26.287949+0000","last_undegraded":"2023-08-25T09:42:26.287949+0000","last_fullsized":"2023-08-25T09:42:26.287949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:23.674251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"72'6","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960141+0000","last_change":"2023-08-25T09:42:24.304814+0000","last_active":"2023-08-25T09:42:26.960141+0000","last_peered":"2023-08-25T09:42:26.960141+0000","last_clean":"2023-08-25T09:42:26.960141+0000","last_became_active":"2023-08-25T09:42:24.302203+0000","last_became_peered":"2023-08-25T09:42:24.302203+0000","last_unstale":"2023-08-25T09:42:26.960141+0000","last_undegraded":"2023-08-25T09:42:26.960141+0000","last_fullsized":"2023-08-25T09:42:26.960141+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:44:31.715159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958165+0000","last_change":"2023-08-25T09:42:25.302156+0000","last_active":"2023-08-25T09:42:26.958165+0000","last_peered":"2023-08-25T09:42:26.958165+0000","last_clean":"2023-08-25T09:42:26.958165+0000","last_became_active":"2023-08-25T09:42:25.301767+0000","last_became_peered":"2023-08-25T09:42:25.301767+0000","last_unstale":"2023-08-25T09:42:26.958165+0000","last_undegraded":"2023-08-25T09:42:26.958165+0000","last_fullsized":"2023-08-25T09:42:26.958165+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:22:11.515926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836961+0000","last_change":"2023-08-25T09:42:24.302995+0000","last_active":"2023-08-25T09:42:26.836961+0000","last_peered":"2023-08-25T09:42:26.836961+0000","last_clean":"2023-08-25T09:42:26.836961+0000","last_became_active":"2023-08-25T09:42:24.302602+0000","last_became_peered":"2023-08-25T09:42:24.302602+0000","last_unstale":"2023-08-25T09:42:26.836961+0000","last_undegraded":"2023-08-25T09:42:26.836961+0000","last_fullsized":"2023-08-25T09:42:26.836961+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:44.114078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957872+0000","last_change":"2023-08-25T09:42:25.317621+0000","last_active":"2023-08-25T09:42:26.957872+0000","last_peered":"2023-08-25T09:42:26.957872+0000","last_clean":"2023-08-25T09:42:26.957872+0000","last_became_active":"2023-08-25T09:42:25.317373+0000","last_became_peered":"2023-08-25T09:42:25.317373+0000","last_unstale":"2023-08-25T09:42:26.957872+0000","last_undegraded":"2023-08-25T09:42:26.957872+0000","last_fullsized":"2023-08-25T09:42:26.957872+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:21.405708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958205+0000","last_change":"2023-08-25T09:42:24.302248+0000","last_active":"2023-08-25T09:42:26.958205+0000","last_peered":"2023-08-25T09:42:26.958205+0000","last_clean":"2023-08-25T09:42:26.958205+0000","last_became_active":"2023-08-25T09:42:24.301884+0000","last_became_peered":"2023-08-25T09:42:24.301884+0000","last_unstale":"2023-08-25T09:42:26.958205+0000","last_undegraded":"2023-08-25T09:42:26.958205+0000","last_fullsized":"2023-08-25T09:42:26.958205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:14.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957867+0000","last_change":"2023-08-25T09:42:25.312890+0000","last_active":"2023-08-25T09:42:26.957867+0000","last_peered":"2023-08-25T09:42:26.957867+0000","last_clean":"2023-08-25T09:42:26.957867+0000","last_became_active":"2023-08-25T09:42:25.312634+0000","last_became_peered":"2023-08-25T09:42:25.312634+0000","last_unstale":"2023-08-25T09:42:26.957867+0000","last_undegraded":"2023-08-25T09:42:26.957867+0000","last_fullsized":"2023-08-25T09:42:26.957867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:11.469930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287905+0000","last_change":"2023-08-25T09:42:24.306160+0000","last_active":"2023-08-25T09:42:26.287905+0000","last_peered":"2023-08-25T09:42:26.287905+0000","last_clean":"2023-08-25T09:42:26.287905+0000","last_became_active":"2023-08-25T09:42:24.305749+0000","last_became_peered":"2023-08-25T09:42:24.305749+0000","last_unstale":"2023-08-25T09:42:26.287905+0000","last_undegraded":"2023-08-25T09:42:26.287905+0000","last_fullsized":"2023-08-25T09:42:26.287905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:16.138562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293479+0000","last_change":"2023-08-25T09:42:25.311281+0000","last_active":"2023-08-25T09:42:26.293479+0000","last_peered":"2023-08-25T09:42:26.293479+0000","last_clean":"2023-08-25T09:42:26.293479+0000","last_became_active":"2023-08-25T09:42:25.310911+0000","last_became_peered":"2023-08-25T09:42:25.310911+0000","last_unstale":"2023-08-25T09:42:26.293479+0000","last_undegraded":"2023-08-25T09:42:26.293479+0000","last_fullsized":"2023-08-25T09:42:26.293479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:05.008842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"72'8","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958600+0000","last_change":"2023-08-25T09:42:24.293445+0000","last_active":"2023-08-25T09:42:26.958600+0000","last_peered":"2023-08-25T09:42:26.958600+0000","last_clean":"2023-08-25T09:42:26.958600+0000","last_became_active":"2023-08-25T09:42:24.293158+0000","last_became_peered":"2023-08-25T09:42:24.293158+0000","last_unstale":"2023-08-25T09:42:26.958600+0000","last_undegraded":"2023-08-25T09:42:26.958600+0000","last_fullsized":"2023-08-25T09:42:26.958600+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:10.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960442+0000","last_change":"2023-08-25T09:42:25.318282+0000","last_active":"2023-08-25T09:42:26.960442+0000","last_peered":"2023-08-25T09:42:26.960442+0000","last_clean":"2023-08-25T09:42:26.960442+0000","last_became_active":"2023-08-25T09:42:25.309358+0000","last_became_peered":"2023-08-25T09:42:25.309358+0000","last_unstale":"2023-08-25T09:42:26.960442+0000","last_undegraded":"2023-08-25T09:42:26.960442+0000","last_fullsized":"2023-08-25T09:42:26.960442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:26:43.451903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836317+0000","last_change":"2023-08-25T09:42:24.305003+0000","last_active":"2023-08-25T09:42:26.836317+0000","last_peered":"2023-08-25T09:42:26.836317+0000","last_clean":"2023-08-25T09:42:26.836317+0000","last_became_active":"2023-08-25T09:42:24.304721+0000","last_became_peered":"2023-08-25T09:42:24.304721+0000","last_unstale":"2023-08-25T09:42:26.836317+0000","last_undegraded":"2023-08-25T09:42:26.836317+0000","last_fullsized":"2023-08-25T09:42:26.836317+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:32:25.305128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960670+0000","last_change":"2023-08-25T09:42:25.309064+0000","last_active":"2023-08-25T09:42:26.960670+0000","last_peered":"2023-08-25T09:42:26.960670+0000","last_clean":"2023-08-25T09:42:26.960670+0000","last_became_active":"2023-08-25T09:42:25.308362+0000","last_became_peered":"2023-08-25T09:42:25.308362+0000","last_unstale":"2023-08-25T09:42:26.960670+0000","last_undegraded":"2023-08-25T09:42:26.960670+0000","last_fullsized":"2023-08-25T09:42:26.960670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:10.299581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"73'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.837025+0000","last_change":"2023-08-25T09:42:24.305708+0000","last_active":"2023-08-25T09:42:26.837025+0000","last_peered":"2023-08-25T09:42:26.837025+0000","last_clean":"2023-08-25T09:42:26.837025+0000","last_became_active":"2023-08-25T09:42:24.305437+0000","last_became_peered":"2023-08-25T09:42:24.305437+0000","last_unstale":"2023-08-25T09:42:26.837025+0000","last_undegraded":"2023-08-25T09:42:26.837025+0000","last_fullsized":"2023-08-25T09:42:26.837025+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:02:25.683034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288036+0000","last_change":"2023-08-25T09:42:25.304424+0000","last_active":"2023-08-25T09:42:26.288036+0000","last_peered":"2023-08-25T09:42:26.288036+0000","last_clean":"2023-08-25T09:42:26.288036+0000","last_became_active":"2023-08-25T09:42:25.304153+0000","last_became_peered":"2023-08-25T09:42:25.304153+0000","last_unstale":"2023-08-25T09:42:26.288036+0000","last_undegraded":"2023-08-25T09:42:26.288036+0000","last_fullsized":"2023-08-25T09:42:26.288036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:09.952773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960468+0000","last_change":"2023-08-25T09:42:24.305243+0000","last_active":"2023-08-25T09:42:26.960468+0000","last_peered":"2023-08-25T09:42:26.960468+0000","last_clean":"2023-08-25T09:42:26.960468+0000","last_became_active":"2023-08-25T09:42:24.304555+0000","last_became_peered":"2023-08-25T09:42:24.304555+0000","last_unstale":"2023-08-25T09:42:26.960468+0000","last_undegraded":"2023-08-25T09:42:26.960468+0000","last_fullsized":"2023-08-25T09:42:26.960468+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:14:59.771777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288529+0000","last_change":"2023-08-25T09:42:25.311088+0000","last_active":"2023-08-25T09:42:26.288529+0000","last_peered":"2023-08-25T09:42:26.288529+0000","last_clean":"2023-08-25T09:42:26.288529+0000","last_became_active":"2023-08-25T09:42:25.310554+0000","last_became_peered":"2023-08-25T09:42:25.310554+0000","last_unstale":"2023-08-25T09:42:26.288529+0000","last_undegraded":"2023-08-25T09:42:26.288529+0000","last_fullsized":"2023-08-25T09:42:26.288529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:06.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292571+0000","last_change":"2023-08-25T09:42:24.301764+0000","last_active":"2023-08-25T09:42:26.292571+0000","last_peered":"2023-08-25T09:42:26.292571+0000","last_clean":"2023-08-25T09:42:26.292571+0000","last_became_active":"2023-08-25T09:42:24.301475+0000","last_became_peered":"2023-08-25T09:42:24.301475+0000","last_unstale":"2023-08-25T09:42:26.292571+0000","last_undegraded":"2023-08-25T09:42:26.292571+0000","last_fullsized":"2023-08-25T09:42:26.292571+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:10.463961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836059+0000","last_change":"2023-08-25T09:42:25.300480+0000","last_active":"2023-08-25T09:42:26.836059+0000","last_peered":"2023-08-25T09:42:26.836059+0000","last_clean":"2023-08-25T09:42:26.836059+0000","last_became_active":"2023-08-25T09:42:25.299879+0000","last_became_peered":"2023-08-25T09:42:25.299879+0000","last_unstale":"2023-08-25T09:42:26.836059+0000","last_undegraded":"2023-08-25T09:42:26.836059+0000","last_fullsized":"2023-08-25T09:42:26.836059+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:14.994192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615578+0000","last_change":"2023-08-25T09:42:24.302869+0000","last_active":"2023-08-25T09:42:26.615578+0000","last_peered":"2023-08-25T09:42:26.615578+0000","last_clean":"2023-08-25T09:42:26.615578+0000","last_became_active":"2023-08-25T09:42:24.302379+0000","last_became_peered":"2023-08-25T09:42:24.302379+0000","last_unstale":"2023-08-25T09:42:26.615578+0000","last_undegraded":"2023-08-25T09:42:26.615578+0000","last_fullsized":"2023-08-25T09:42:26.615578+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:05:22.358354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615119+0000","last_change":"2023-08-25T09:42:25.307304+0000","last_active":"2023-08-25T09:42:26.615119+0000","last_peered":"2023-08-25T09:42:26.615119+0000","last_clean":"2023-08-25T09:42:26.615119+0000","last_became_active":"2023-08-25T09:42:25.306702+0000","last_became_peered":"2023-08-25T09:42:25.306702+0000","last_unstale":"2023-08-25T09:42:26.615119+0000","last_undegraded":"2023-08-25T09:42:26.615119+0000","last_fullsized":"2023-08-25T09:42:26.615119+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:55.792001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293059+0000","last_change":"2023-08-25T09:42:24.301955+0000","last_active":"2023-08-25T09:42:26.293059+0000","last_peered":"2023-08-25T09:42:26.293059+0000","last_clean":"2023-08-25T09:42:26.293059+0000","last_became_active":"2023-08-25T09:42:24.301280+0000","last_became_peered":"2023-08-25T09:42:24.301280+0000","last_unstale":"2023-08-25T09:42:26.293059+0000","last_undegraded":"2023-08-25T09:42:26.293059+0000","last_fullsized":"2023-08-25T09:42:26.293059+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:37:48.811658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959912+0000","last_change":"2023-08-25T09:42:25.305417+0000","last_active":"2023-08-25T09:42:26.959912+0000","last_peered":"2023-08-25T09:42:26.959912+0000","last_clean":"2023-08-25T09:42:26.959912+0000","last_became_active":"2023-08-25T09:42:25.304933+0000","last_became_peered":"2023-08-25T09:42:25.304933+0000","last_unstale":"2023-08-25T09:42:26.959912+0000","last_undegraded":"2023-08-25T09:42:26.959912+0000","last_fullsized":"2023-08-25T09:42:26.959912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:28.006718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291833+0000","last_change":"2023-08-25T09:42:24.306699+0000","last_active":"2023-08-25T09:42:26.291833+0000","last_peered":"2023-08-25T09:42:26.291833+0000","last_clean":"2023-08-25T09:42:26.291833+0000","last_became_active":"2023-08-25T09:42:24.306437+0000","last_became_peered":"2023-08-25T09:42:24.306437+0000","last_unstale":"2023-08-25T09:42:26.291833+0000","last_undegraded":"2023-08-25T09:42:26.291833+0000","last_fullsized":"2023-08-25T09:42:26.291833+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:33.951474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.961019+0000","last_change":"2023-08-25T09:42:25.303448+0000","last_active":"2023-08-25T09:42:26.961019+0000","last_peered":"2023-08-25T09:42:26.961019+0000","last_clean":"2023-08-25T09:42:26.961019+0000","last_became_active":"2023-08-25T09:42:25.302770+0000","last_became_peered":"2023-08-25T09:42:25.302770+0000","last_unstale":"2023-08-25T09:42:26.961019+0000","last_undegraded":"2023-08-25T09:42:26.961019+0000","last_fullsized":"2023-08-25T09:42:26.961019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:37.982860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960756+0000","last_change":"2023-08-25T09:42:24.305948+0000","last_active":"2023-08-25T09:42:26.960756+0000","last_peered":"2023-08-25T09:42:26.960756+0000","last_clean":"2023-08-25T09:42:26.960756+0000","last_became_active":"2023-08-25T09:42:24.305521+0000","last_became_peered":"2023-08-25T09:42:24.305521+0000","last_unstale":"2023-08-25T09:42:26.960756+0000","last_undegraded":"2023-08-25T09:42:26.960756+0000","last_fullsized":"2023-08-25T09:42:26.960756+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:35:38.184245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958338+0000","last_change":"2023-08-25T09:42:25.310738+0000","last_active":"2023-08-25T09:42:26.958338+0000","last_peered":"2023-08-25T09:42:26.958338+0000","last_clean":"2023-08-25T09:42:26.958338+0000","last_became_active":"2023-08-25T09:42:25.309165+0000","last_became_peered":"2023-08-25T09:42:25.309165+0000","last_unstale":"2023-08-25T09:42:26.958338+0000","last_undegraded":"2023-08-25T09:42:26.958338+0000","last_fullsized":"2023-08-25T09:42:26.958338+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:44.760882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960755+0000","last_change":"2023-08-25T09:42:24.306848+0000","last_active":"2023-08-25T09:42:26.960755+0000","last_peered":"2023-08-25T09:42:26.960755+0000","last_clean":"2023-08-25T09:42:26.960755+0000","last_became_active":"2023-08-25T09:42:24.305161+0000","last_became_peered":"2023-08-25T09:42:24.305161+0000","last_unstale":"2023-08-25T09:42:26.960755+0000","last_undegraded":"2023-08-25T09:42:26.960755+0000","last_fullsized":"2023-08-25T09:42:26.960755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:06.503919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958025+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:42:26.958025+0000","last_peered":"2023-08-25T09:42:26.958025+0000","last_clean":"2023-08-25T09:42:26.958025+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:42:26.958025+0000","last_undegraded":"2023-08-25T09:42:26.958025+0000","last_fullsized":"2023-08-25T09:42:26.958025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:46:24.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288864+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:42:26.288864+0000","last_peered":"2023-08-25T09:42:26.288864+0000","last_clean":"2023-08-25T09:42:26.288864+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:42:26.288864+0000","last_undegraded":"2023-08-25T09:42:26.288864+0000","last_fullsized":"2023-08-25T09:42:26.288864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:15:25.745588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959927+0000","last_change":"2023-08-25T09:42:25.309959+0000","last_active":"2023-08-25T09:42:26.959927+0000","last_peered":"2023-08-25T09:42:26.959927+0000","last_clean":"2023-08-25T09:42:26.959927+0000","last_became_active":"2023-08-25T09:42:25.309177+0000","last_became_peered":"2023-08-25T09:42:25.309177+0000","last_unstale":"2023-08-25T09:42:26.959927+0000","last_undegraded":"2023-08-25T09:42:26.959927+0000","last_fullsized":"2023-08-25T09:42:26.959927+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:50.719507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960016+0000","last_change":"2023-08-25T09:42:24.301077+0000","last_active":"2023-08-25T09:42:26.960016+0000","last_peered":"2023-08-25T09:42:26.960016+0000","last_clean":"2023-08-25T09:42:26.960016+0000","last_became_active":"2023-08-25T09:42:24.298490+0000","last_became_peered":"2023-08-25T09:42:24.298490+0000","last_unstale":"2023-08-25T09:42:26.960016+0000","last_undegraded":"2023-08-25T09:42:26.960016+0000","last_fullsized":"2023-08-25T09:42:26.960016+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:05:10.157081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287633+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:42:26.287633+0000","last_peered":"2023-08-25T09:42:26.287633+0000","last_clean":"2023-08-25T09:42:26.287633+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:42:26.287633+0000","last_undegraded":"2023-08-25T09:42:26.287633+0000","last_fullsized":"2023-08-25T09:42:26.287633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:57:58.707505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615903+0000","last_change":"2023-08-25T09:42:25.305873+0000","last_active":"2023-08-25T09:42:26.615903+0000","last_peered":"2023-08-25T09:42:26.615903+0000","last_clean":"2023-08-25T09:42:26.615903+0000","last_became_active":"2023-08-25T09:42:25.305296+0000","last_became_peered":"2023-08-25T09:42:25.305296+0000","last_unstale":"2023-08-25T09:42:26.615903+0000","last_undegraded":"2023-08-25T09:42:26.615903+0000","last_fullsized":"2023-08-25T09:42:26.615903+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:24:08.503228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957806+0000","last_change":"2023-08-25T09:42:24.295421+0000","last_active":"2023-08-25T09:42:26.957806+0000","last_peered":"2023-08-25T09:42:26.957806+0000","last_clean":"2023-08-25T09:42:26.957806+0000","last_became_active":"2023-08-25T09:42:24.294838+0000","last_became_peered":"2023-08-25T09:42:24.294838+0000","last_unstale":"2023-08-25T09:42:26.957806+0000","last_undegraded":"2023-08-25T09:42:26.957806+0000","last_fullsized":"2023-08-25T09:42:26.957806+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:48:20.672723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292151+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:42:26.292151+0000","last_peered":"2023-08-25T09:42:26.292151+0000","last_clean":"2023-08-25T09:42:26.292151+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:42:26.292151+0000","last_undegraded":"2023-08-25T09:42:26.292151+0000","last_fullsized":"2023-08-25T09:42:26.292151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:44:43.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835878+0000","last_change":"2023-08-25T09:42:25.303375+0000","last_active":"2023-08-25T09:42:26.835878+0000","last_peered":"2023-08-25T09:42:26.835878+0000","last_clean":"2023-08-25T09:42:26.835878+0000","last_became_active":"2023-08-25T09:42:25.302988+0000","last_became_peered":"2023-08-25T09:42:25.302988+0000","last_unstale":"2023-08-25T09:42:26.835878+0000","last_undegraded":"2023-08-25T09:42:26.835878+0000","last_fullsized":"2023-08-25T09:42:26.835878+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:05.008842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959022+0000","last_change":"2023-08-25T09:42:24.303402+0000","last_active":"2023-08-25T09:42:26.959022+0000","last_peered":"2023-08-25T09:42:26.959022+0000","last_clean":"2023-08-25T09:42:26.959022+0000","last_became_active":"2023-08-25T09:42:24.303131+0000","last_became_peered":"2023-08-25T09:42:24.303131+0000","last_unstale":"2023-08-25T09:42:26.959022+0000","last_undegraded":"2023-08-25T09:42:26.959022+0000","last_fullsized":"2023-08-25T09:42:26.959022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:41.614060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288840+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:42:26.288840+0000","last_peered":"2023-08-25T09:42:26.288840+0000","last_clean":"2023-08-25T09:42:26.288840+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:42:26.288840+0000","last_undegraded":"2023-08-25T09:42:26.288840+0000","last_fullsized":"2023-08-25T09:42:26.288840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:37:02.531152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836133+0000","last_change":"2023-08-25T09:42:25.299354+0000","last_active":"2023-08-25T09:42:26.836133+0000","last_peered":"2023-08-25T09:42:26.836133+0000","last_clean":"2023-08-25T09:42:26.836133+0000","last_became_active":"2023-08-25T09:42:25.298183+0000","last_became_peered":"2023-08-25T09:42:25.298183+0000","last_unstale":"2023-08-25T09:42:26.836133+0000","last_undegraded":"2023-08-25T09:42:26.836133+0000","last_fullsized":"2023-08-25T09:42:26.836133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:19:45.199434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836611+0000","last_change":"2023-08-25T09:42:24.304030+0000","last_active":"2023-08-25T09:42:26.836611+0000","last_peered":"2023-08-25T09:42:26.836611+0000","last_clean":"2023-08-25T09:42:26.836611+0000","last_became_active":"2023-08-25T09:42:24.303752+0000","last_became_peered":"2023-08-25T09:42:24.303752+0000","last_unstale":"2023-08-25T09:42:26.836611+0000","last_undegraded":"2023-08-25T09:42:26.836611+0000","last_fullsized":"2023-08-25T09:42:26.836611+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:56.778336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":104,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959351+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:42:26.959351+0000","last_peered":"2023-08-25T09:42:26.959351+0000","last_clean":"2023-08-25T09:42:26.959351+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:42:26.959351+0000","last_undegraded":"2023-08-25T09:42:26.959351+0000","last_fullsized":"2023-08-25T09:42:26.959351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:04.518492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615458+0000","last_change":"2023-08-25T09:42:25.302644+0000","last_active":"2023-08-25T09:42:26.615458+0000","last_peered":"2023-08-25T09:42:26.615458+0000","last_clean":"2023-08-25T09:42:26.615458+0000","last_became_active":"2023-08-25T09:42:25.301907+0000","last_became_peered":"2023-08-25T09:42:25.301907+0000","last_unstale":"2023-08-25T09:42:26.615458+0000","last_undegraded":"2023-08-25T09:42:26.615458+0000","last_fullsized":"2023-08-25T09:42:26.615458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:31:47.458467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.837273+0000","last_change":"2023-08-25T09:42:24.304596+0000","last_active":"2023-08-25T09:42:26.837273+0000","last_peered":"2023-08-25T09:42:26.837273+0000","last_clean":"2023-08-25T09:42:26.837273+0000","last_became_active":"2023-08-25T09:42:24.304317+0000","last_became_peered":"2023-08-25T09:42:24.304317+0000","last_unstale":"2023-08-25T09:42:26.837273+0000","last_undegraded":"2023-08-25T09:42:26.837273+0000","last_fullsized":"2023-08-25T09:42:26.837273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:12.308233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287497+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:42:26.287497+0000","last_peered":"2023-08-25T09:42:26.287497+0000","last_clean":"2023-08-25T09:42:26.287497+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:42:26.287497+0000","last_undegraded":"2023-08-25T09:42:26.287497+0000","last_fullsized":"2023-08-25T09:42:26.287497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:14.456424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615734+0000","last_change":"2023-08-25T09:42:25.302876+0000","last_active":"2023-08-25T09:42:26.615734+0000","last_peered":"2023-08-25T09:42:26.615734+0000","last_clean":"2023-08-25T09:42:26.615734+0000","last_became_active":"2023-08-25T09:42:25.302604+0000","last_became_peered":"2023-08-25T09:42:25.302604+0000","last_unstale":"2023-08-25T09:42:26.615734+0000","last_undegraded":"2023-08-25T09:42:26.615734+0000","last_fullsized":"2023-08-25T09:42:26.615734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:45:52.202179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288554+0000","last_change":"2023-08-25T09:42:24.292895+0000","last_active":"2023-08-25T09:42:26.288554+0000","last_peered":"2023-08-25T09:42:26.288554+0000","last_clean":"2023-08-25T09:42:26.288554+0000","last_became_active":"2023-08-25T09:42:24.292375+0000","last_became_peered":"2023-08-25T09:42:24.292375+0000","last_unstale":"2023-08-25T09:42:26.288554+0000","last_undegraded":"2023-08-25T09:42:26.288554+0000","last_fullsized":"2023-08-25T09:42:26.288554+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:45.682019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.961020+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:42:26.961020+0000","last_peered":"2023-08-25T09:42:26.961020+0000","last_clean":"2023-08-25T09:42:26.961020+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:42:26.961020+0000","last_undegraded":"2023-08-25T09:42:26.961020+0000","last_fullsized":"2023-08-25T09:42:26.961020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:37.515711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287271+0000","last_change":"2023-08-25T09:42:25.303188+0000","last_active":"2023-08-25T09:42:26.287271+0000","last_peered":"2023-08-25T09:42:26.287271+0000","last_clean":"2023-08-25T09:42:26.287271+0000","last_became_active":"2023-08-25T09:42:25.302930+0000","last_became_peered":"2023-08-25T09:42:25.302930+0000","last_unstale":"2023-08-25T09:42:26.287271+0000","last_undegraded":"2023-08-25T09:42:26.287271+0000","last_fullsized":"2023-08-25T09:42:26.287271+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:41:30.441978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292111+0000","last_change":"2023-08-25T09:42:24.301838+0000","last_active":"2023-08-25T09:42:26.292111+0000","last_peered":"2023-08-25T09:42:26.292111+0000","last_clean":"2023-08-25T09:42:26.292111+0000","last_became_active":"2023-08-25T09:42:24.300672+0000","last_became_peered":"2023-08-25T09:42:24.300672+0000","last_unstale":"2023-08-25T09:42:26.292111+0000","last_undegraded":"2023-08-25T09:42:26.292111+0000","last_fullsized":"2023-08-25T09:42:26.292111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:22.842620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288202+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:42:26.288202+0000","last_peered":"2023-08-25T09:42:26.288202+0000","last_clean":"2023-08-25T09:42:26.288202+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:42:26.288202+0000","last_undegraded":"2023-08-25T09:42:26.288202+0000","last_fullsized":"2023-08-25T09:42:26.288202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:01:39.204663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194922,"statfs":{"total":200000000,"available":199600157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.1499999999999999,"15min":1.052},"min":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":10.747,"5min":10.747,"15min":10.747},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0249999999999999,"15min":1.008},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.395,"5min":1.599,"15min":1.599},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1319999999999999,"15min":1.111},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.125,"15min":1.073},"min":{"1min":0.748,"5min":0.745,"15min":0.745},"max":{"1min":1.6679999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0569999999999999},"min":{"1min":0.745,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.48,"5min":1.512,"15min":1.512},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2190000000000001,"15min":1.2},"min":{"1min":0.80500000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.167,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.417,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.341,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":1.0109999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":7.1639999999999997,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.874,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.571,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.365,"15min":1.3149999999999999},"min":{"1min":1.1519999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7490000000000001,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.46,"5min":1.391,"15min":1.361},"min":{"1min":1.1200000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.784,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.444,"15min":1.411},"min":{"1min":1.1699999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8440000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4059999999999999,"15min":1.3759999999999999},"min":{"1min":0.89900000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9590000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.71199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":392,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194919,"statfs":{"total":200000000,"available":199597893,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1240000000000001,"15min":1.145},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.173,"15min":1.139},"min":{"1min":0.79900000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.5410000000000004,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.099}]},{"osd":1,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.23,"15min":1.1950000000000001},"min":{"1min":0.81599999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.1240000000000001,"15min":1.0980000000000001},"min":{"1min":0.93899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.1180000000000001,"15min":1.089},"min":{"1min":0.76700000000000002,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.857,"5min":1.296,"15min":1.1819999999999999},"min":{"1min":0.93600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":13.193,"5min":13.193,"15min":13.193},"last":1.357}]},{"osd":3,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.2709999999999999,"15min":1.212},"min":{"1min":0.999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.609},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.262,"15min":1.252},"min":{"1min":0.85899999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.7150000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2070000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2809999999999999,"15min":1.232},"min":{"1min":0.79000000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.768,"5min":2.484,"15min":2.484},"last":1.169},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.3700000000000001,"15min":1.3120000000000001},"min":{"1min":0.88500000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.4699999999999998,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.72}]},{"osd":5,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.2889999999999999,"15min":1.2669999999999999},"min":{"1min":0.93799999999999994,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6160000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.379},{"interface":"front","average":{"1min":1.538,"5min":1.3979999999999999,"15min":1.345},"min":{"1min":1.2929999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8300000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4059999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.54,"15min":1.4910000000000001},"min":{"1min":1.1850000000000001,"5min":0.622,"15min":0.622},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.819},{"interface":"front","average":{"1min":1.508,"5min":1.4870000000000001,"15min":1.4770000000000001},"min":{"1min":1.1579999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8280000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.3200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":968,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194343,"statfs":{"total":200000000,"available":199008065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1499999999999999,"15min":1.133},"min":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.667,"15min":1.667},"last":1.786},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1890000000000001,"15min":1.1970000000000001},"min":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6259999999999999,"5min":1.974,"15min":1.974},"last":1.216}]},{"osd":1,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1850000000000001,"15min":1.208},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7250000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.125,"15min":1.1000000000000001},"min":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.282}]},{"osd":2,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.1910000000000001,"15min":1.1699999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.655},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.212,"15min":1.206},"min":{"1min":0.96899999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8049999999999999,"5min":1.962,"15min":1.962},"last":1.0609999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":2.0590000000000002,"5min":1.401,"15min":1.3140000000000001},"min":{"1min":0.747,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":17.356999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.355,"5min":1.2829999999999999,"15min":1.2709999999999999},"min":{"1min":0.95999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7350000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.3939999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3169999999999999,"15min":1.2909999999999999},"min":{"1min":1.0469999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3420000000000001,"15min":1.3700000000000001},"min":{"1min":0.93100000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8620000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.345}]},{"osd":6,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4330000000000001,"15min":1.4299999999999999},"min":{"1min":1.0129999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.4419999999999999,"15min":1.3939999999999999},"min":{"1min":1.385,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.911}]},{"osd":7,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4239999999999999,"15min":1.427},"min":{"1min":1.2,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.5660000000000001,"15min":1.5720000000000001},"min":{"1min":1.2629999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.911,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.72}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":391,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194921,"statfs":{"total":200000000,"available":199599575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.153,"15min":1.1419999999999999},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.2050000000000001,"15min":1.127},"min":{"1min":0.76400000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":7.367,"5min":7.367,"15min":7.367},"last":1.276}]},{"osd":1,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.103,"15min":1.111},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.579,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.3540000000000001,"15min":1.4379999999999999},"min":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.613,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.111}]},{"osd":2,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.256,"15min":1.3180000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5600000000000001,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1619999999999999,"15min":1.1399999999999999},"min":{"1min":0.80000000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.629,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.627}]},{"osd":3,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.29,"15min":1.23},"min":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.246,"5min":1.343,"15min":1.3640000000000001},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.548,"5min":1.911,"15min":1.911},"last":1.397}]},{"osd":5,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.294,"15min":1.294},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7090000000000001,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.49},{"interface":"front","average":{"1min":1.633,"5min":1.3009999999999999,"15min":1.2390000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":8.2409999999999997,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.4350000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.407,"15min":1.4299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8859999999999999,"5min":2.863,"15min":2.863},"last":1.671},{"interface":"front","average":{"1min":1.48,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.1070000000000002,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.7090000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.466,"15min":1.3899999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":8.1760000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.5649999999999999,"15min":1.585},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.976,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.754}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194921,"statfs":{"total":200000000,"available":199599997,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4650000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85999999999999999,"15min":0.86699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3580000000000001,"5min":1.55,"15min":1.55},"last":1.1180000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80600000000000005,"15min":0.77600000000000002},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.85799999999999998,"15min":0.83599999999999997},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.47699999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8280000000000001,"5min":3.399,"15min":3.399},"last":1.1919999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.76100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5649999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.1930000000000001,"15min":1.1859999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.2949999999999999,"15min":1.3029999999999999},"min":{"1min":0.92600000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7569999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.411},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.256,"15min":1.26},"min":{"1min":0.81499999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.335}]},{"osd":6,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.236,"15min":1.226},"min":{"1min":0.78200000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.24},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3799999999999999,"15min":1.3999999999999999},"min":{"1min":0.91300000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.984,"5min":3.496,"15min":3.496},"last":1.3080000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.349,"15min":1.347},"min":{"1min":1.0209999999999999,"5min":0.871,"15min":0.871},"max":{"1min":2.044,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.385},{"interface":"front","average":{"1min":1.45,"5min":1.302,"15min":1.2629999999999999},"min":{"1min":0.88100000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3520000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194921,"statfs":{"total":200000000,"available":199600101,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81999999999999995},"min":{"1min":0.48299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.589,"5min":2.383,"15min":2.383},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.995,"15min":1.083},"min":{"1min":0.502,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.212,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1}]},{"osd":1,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.53500000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2829999999999999,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.036,"15min":1.0569999999999999},"min":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.518,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.042,"15min":1.115},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5209999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.744,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.24}]},{"osd":4,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.353,"15min":1.4119999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0099999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2969999999999999,"15min":1.3440000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7909999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.5640000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.335,"15min":1.3979999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.639,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.339},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0249999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.3029999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.704,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4330000000000001,"15min":1.4970000000000001},"min":{"1min":0.997,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.7749999999999999,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.506}]},{"osd":7,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.4219999999999999,"15min":1.462},"min":{"1min":0.85699999999999998,"5min":0.746,"15min":0.746},"max":{"1min":2.0640000000000001,"5min":2.968,"15min":2.968},"last":1.206},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.522},"min":{"1min":0.95199999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.841,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.401}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":967,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194345,"statfs":{"total":200000000,"available":199009755,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.355,"5min":1.627,"15min":1.627},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.79700000000000004,"15min":0.75700000000000001},"min":{"1min":0.50600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.117,"5min":0.91700000000000004,"15min":0.88300000000000001},"min":{"1min":0.41899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.78700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":1.663,"5min":1.7,"15min":1.7},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.92100000000000004},"min":{"1min":0.48799999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.53,"5min":1.764,"15min":1.764},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.087,"15min":1.0640000000000001},"min":{"1min":0.77400000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.236,"15min":1.1930000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.5249999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.208,"15min":1.1779999999999999},"min":{"1min":0.89700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.823,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.1699999999999999,"15min":1.1519999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.198}]},{"osd":6,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.2989999999999999,"15min":1.2709999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.75,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.284,"15min":1.2689999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.671}]},{"osd":7,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.3759999999999999,"15min":1.3360000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.665},{"interface":"front","average":{"1min":1.472,"5min":1.3260000000000001,"15min":1.2869999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194922,"statfs":{"total":200000000,"available":199600157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":0.91300000000000003,"15min":0.85499999999999998},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":11.257,"5min":11.257,"15min":11.257},"last":0.752},{"interface":"front","average":{"1min":1.252,"5min":0.98099999999999998,"15min":0.96099999999999997},"min":{"1min":0.47099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.86599999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.21},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0569999999999999,"15min":1.1120000000000001},"min":{"1min":0.55300000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2210000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.89700000000000002,"15min":0.90400000000000003},"min":{"1min":0.49199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3420000000000001,"5min":1.387,"15min":1.387},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0980000000000001,"15min":1.1259999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.869,"15min":1.869},"last":1.548}]},{"osd":4,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2170000000000001,"15min":1.2310000000000001},"min":{"1min":0.94199999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.5649999999999999,"5min":1.704,"15min":1.704},"last":1.155},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.339,"15min":1.355},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.6870000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.319,"15min":1.379},"min":{"1min":0.83899999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5680000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.2769999999999999,"15min":1.3089999999999999},"min":{"1min":0.97299999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8100000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6419999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3160000000000001,"15min":1.3480000000000001},"min":{"1min":0.82799999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.282,"15min":1.294},"min":{"1min":0.93500000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6100000000000001,"5min":1.764,"15min":1.764},"last":1.595}]},{"osd":7,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3819999999999999,"15min":1.4019999999999999},"min":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9139999999999999,"5min":2.121,"15min":2.121},"last":1.514},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3149999999999999,"15min":1.3240000000000001},"min":{"1min":0.94999999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.4370000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:42:29.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:42:29.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.981+0000 7f15fa960640 1 Processor -- start 2023-08-25T09:42:29.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.981+0000 7f15fa960640 1 -- start start 2023-08-25T09:42:29.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15fa960640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15f415a7c0 0x7f15f40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15fa960640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15fa960640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f415e7c0 con 0x7f15f40b12f0 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15fa960640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f40cb6f0 con 0x7f15f415a7c0 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15fa960640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f4071b50 con 0x7f15f40cad20 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15f995e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15f415a7c0 0x7f15f40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15f915d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f15f40b12f0 0x7f15f415ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45334/0 (socket says 172.21.15.70:45334) 2023-08-25T09:42:29.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15f995e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15f415a7c0 0x7f15f40b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58718/0 (socket says 172.21.15.70:58718) 2023-08-25T09:42:29.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.983+0000 7f15f915d640 1 -- 172.21.15.70:0/262772657 learned_addr learned my addr 172.21.15.70:0/262772657 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:29.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.984+0000 7f15fa15f640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.984+0000 7f15f995e640 1 -- 172.21.15.70:0/262772657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 msgr2=0x7f15f40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.984+0000 7f15f995e640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.984+0000 7f15f995e640 1 -- 172.21.15.70:0/262772657 >> v1:172.21.15.70:6789/0 conn(0x7f15f40b12f0 legacy=0x7f15f415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.984+0000 7f15f995e640 1 -- 172.21.15.70:0/262772657 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f415e490 con 0x7f15f415a7c0 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.984+0000 7f15fa15f640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.985+0000 7f15f995e640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f40b3560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f15f00097a0 tx=0x7f15f002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e31af7d472b6a68 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.985+0000 7f15f895c640 1 -- 172.21.15.70:0/262772657 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f002d040 con 0x7f15f415a7c0 2023-08-25T09:42:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.985+0000 7f15f895c640 1 -- 172.21.15.70:0/262772657 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f0031810 con 0x7f15f415a7c0 2023-08-25T09:42:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.985+0000 7f15f895c640 1 -- 172.21.15.70:0/262772657 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0031ba0 con 0x7f15f415a7c0 2023-08-25T09:42:29.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 -- 172.21.15.70:0/262772657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 msgr2=0x7f15f40b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f40b3560 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f15f00097a0 tx=0x7f15f002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 -- 172.21.15.70:0/262772657 shutdown_connections 2023-08-25T09:42:29.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 --2- 172.21.15.70:0/262772657 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f40b3560 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 -- 172.21.15.70:0/262772657 >> 172.21.15.70:0/262772657 conn(0x7f15f40affd0 msgr2=0x7f15f40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:29.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 -- 172.21.15.70:0/262772657 shutdown_connections 2023-08-25T09:42:29.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.986+0000 7f15fa960640 1 -- 172.21.15.70:0/262772657 wait complete. 2023-08-25T09:42:29.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.988+0000 7f15fa960640 1 Processor -- start 2023-08-25T09:42:29.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.988+0000 7f15fa960640 1 -- start start 2023-08-25T09:42:29.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa960640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40b12f0 0x7f15f4156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa960640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa960640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f4157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa960640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f415c9d0 con 0x7f15f415a7c0 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa960640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f40c4ef0 con 0x7f15f40b12f0 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa960640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f41574c0 con 0x7f15f40cad20 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa15f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f4157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa15f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f4157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58728/0 (socket says 172.21.15.70:58728) 2023-08-25T09:42:29.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15fa15f640 1 -- 172.21.15.70:0/3388848260 learned_addr learned my addr 172.21.15.70:0/3388848260 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.989+0000 7f15f995e640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40b12f0 0x7f15f4156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa15f640 1 -- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 msgr2=0x7f15f40c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa15f640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa15f640 1 -- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40b12f0 msgr2=0x7f15f4156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa15f640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40b12f0 0x7f15f4156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa15f640 1 -- 172.21.15.70:0/3388848260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009400 con 0x7f15f415a7c0 2023-08-25T09:42:29.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa15f640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f4157710 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00cef0 tx=0x7f15ec00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00d840 con 0x7f15f415a7c0 2023-08-25T09:42:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec00d9c0 con 0x7f15f415a7c0 2023-08-25T09:42:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f415bd50 con 0x7f15f415a7c0 2023-08-25T09:42:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00dd70 con 0x7f15f415a7c0 2023-08-25T09:42:29.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.990+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f415c310 con 0x7f15f415a7c0 2023-08-25T09:42:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.991+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f415cb10 con 0x7f15f415a7c0 2023-08-25T09:42:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.992+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f15ec01e3b0 con 0x7f15f415a7c0 2023-08-25T09:42:29.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.994+0000 7f15ea7fc640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15d806a050 0x7f15d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.994+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15ec0890a0 con 0x7f15f415a7c0 2023-08-25T09:42:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.994+0000 7f15f995e640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15d806a050 0x7f15d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:29.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.995+0000 7f15f995e640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15d806a050 0x7f15d806c510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15f0036000 tx=0x7f15f003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:29.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:29.998+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f15f415cb10 con 0x7f15f415a7c0 2023-08-25T09:42:30.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.182+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15f40b3d60 con 0x7f15d806a050 2023-08-25T09:42:30.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.191+0000 7f15ea7fc640 1 -- 172.21.15.70:0/3388848260 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303174 (secure 0 0 0) 0x7f15f0044400 con 0x7f15d806a050 2023-08-25T09:42:30.199 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:30.200 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:42:30.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15d806a050 msgr2=0x7f15d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15d806a050 0x7f15d806c510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15f0036000 tx=0x7f15f003c040 comp rx=0 tx=0).stop 2023-08-25T09:42:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 msgr2=0x7f15f4157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f4157710 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f15ec00cef0 tx=0x7f15ec00a710 comp rx=0 tx=0).stop 2023-08-25T09:42:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 shutdown_connections 2023-08-25T09:42:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15d806a050 0x7f15d806c510 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15f415a7c0 0x7f15f4157710 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15f40cad20 0x7f15f40c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 --2- 172.21.15.70:0/3388848260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15f40b12f0 0x7f15f4156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.198+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 >> 172.21.15.70:0/3388848260 conn(0x7f15f40affd0 msgr2=0x7f15f4070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.199+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 shutdown_connections 2023-08-25T09:42:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.199+0000 7f15fa960640 1 -- 172.21.15.70:0/3388848260 wait complete. 2023-08-25T09:42:30.219 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-08-25T09:42:29.389158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4281,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558218,"statfs":{"total":1600000000,"available":1595615700,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001588"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293733+0000","last_change":"2023-08-25T09:42:25.313741+0000","last_active":"2023-08-25T09:42:26.293733+0000","last_peered":"2023-08-25T09:42:26.293733+0000","last_clean":"2023-08-25T09:42:26.293733+0000","last_became_active":"2023-08-25T09:42:25.313118+0000","last_became_peered":"2023-08-25T09:42:25.313118+0000","last_unstale":"2023-08-25T09:42:26.293733+0000","last_undegraded":"2023-08-25T09:42:26.293733+0000","last_fullsized":"2023-08-25T09:42:26.293733+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:14.994192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959148+0000","last_change":"2023-08-25T09:42:25.310740+0000","last_active":"2023-08-25T09:42:26.959148+0000","last_peered":"2023-08-25T09:42:26.959148+0000","last_clean":"2023-08-25T09:42:26.959148+0000","last_became_active":"2023-08-25T09:42:25.310443+0000","last_became_peered":"2023-08-25T09:42:25.310443+0000","last_unstale":"2023-08-25T09:42:26.959148+0000","last_undegraded":"2023-08-25T09:42:26.959148+0000","last_fullsized":"2023-08-25T09:42:26.959148+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:41:10.686668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287278+0000","last_change":"2023-08-25T09:42:25.311910+0000","last_active":"2023-08-25T09:42:26.287278+0000","last_peered":"2023-08-25T09:42:26.287278+0000","last_clean":"2023-08-25T09:42:26.287278+0000","last_became_active":"2023-08-25T09:42:25.311596+0000","last_became_peered":"2023-08-25T09:42:25.311596+0000","last_unstale":"2023-08-25T09:42:26.287278+0000","last_undegraded":"2023-08-25T09:42:26.287278+0000","last_fullsized":"2023-08-25T09:42:26.287278+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:58:48.112882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835224+0000","last_change":"2023-08-25T09:42:25.303222+0000","last_active":"2023-08-25T09:42:26.835224+0000","last_peered":"2023-08-25T09:42:26.835224+0000","last_clean":"2023-08-25T09:42:26.835224+0000","last_became_active":"2023-08-25T09:42:25.302620+0000","last_became_peered":"2023-08-25T09:42:25.302620+0000","last_unstale":"2023-08-25T09:42:26.835224+0000","last_undegraded":"2023-08-25T09:42:26.835224+0000","last_fullsized":"2023-08-25T09:42:26.835224+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:58.439459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287593+0000","last_change":"2023-08-25T09:42:25.294257+0000","last_active":"2023-08-25T09:42:26.287593+0000","last_peered":"2023-08-25T09:42:26.287593+0000","last_clean":"2023-08-25T09:42:26.287593+0000","last_became_active":"2023-08-25T09:42:25.293209+0000","last_became_peered":"2023-08-25T09:42:25.293209+0000","last_unstale":"2023-08-25T09:42:26.287593+0000","last_undegraded":"2023-08-25T09:42:26.287593+0000","last_fullsized":"2023-08-25T09:42:26.287593+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:38.507153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959537+0000","last_change":"2023-08-25T09:42:25.309747+0000","last_active":"2023-08-25T09:42:26.959537+0000","last_peered":"2023-08-25T09:42:26.959537+0000","last_clean":"2023-08-25T09:42:26.959537+0000","last_became_active":"2023-08-25T09:42:25.308811+0000","last_became_peered":"2023-08-25T09:42:25.308811+0000","last_unstale":"2023-08-25T09:42:26.959537+0000","last_undegraded":"2023-08-25T09:42:26.959537+0000","last_fullsized":"2023-08-25T09:42:26.959537+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:23:07.802680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291554+0000","last_change":"2023-08-25T09:42:25.318991+0000","last_active":"2023-08-25T09:42:26.291554+0000","last_peered":"2023-08-25T09:42:26.291554+0000","last_clean":"2023-08-25T09:42:26.291554+0000","last_became_active":"2023-08-25T09:42:25.318693+0000","last_became_peered":"2023-08-25T09:42:25.318693+0000","last_unstale":"2023-08-25T09:42:26.291554+0000","last_undegraded":"2023-08-25T09:42:26.291554+0000","last_fullsized":"2023-08-25T09:42:26.291554+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:07.745510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287704+0000","last_change":"2023-08-25T09:42:25.294147+0000","last_active":"2023-08-25T09:42:26.287704+0000","last_peered":"2023-08-25T09:42:26.287704+0000","last_clean":"2023-08-25T09:42:26.287704+0000","last_became_active":"2023-08-25T09:42:25.293879+0000","last_became_peered":"2023-08-25T09:42:25.293879+0000","last_unstale":"2023-08-25T09:42:26.287704+0000","last_undegraded":"2023-08-25T09:42:26.287704+0000","last_fullsized":"2023-08-25T09:42:26.287704+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:20.804220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836350+0000","last_change":"2023-08-25T09:42:25.296158+0000","last_active":"2023-08-25T09:42:26.836350+0000","last_peered":"2023-08-25T09:42:26.836350+0000","last_clean":"2023-08-25T09:42:26.836350+0000","last_became_active":"2023-08-25T09:42:25.295931+0000","last_became_peered":"2023-08-25T09:42:25.295931+0000","last_unstale":"2023-08-25T09:42:26.836350+0000","last_undegraded":"2023-08-25T09:42:26.836350+0000","last_fullsized":"2023-08-25T09:42:26.836350+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:21:02.657884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958409+0000","last_change":"2023-08-25T09:42:25.308724+0000","last_active":"2023-08-25T09:42:26.958409+0000","last_peered":"2023-08-25T09:42:26.958409+0000","last_clean":"2023-08-25T09:42:26.958409+0000","last_became_active":"2023-08-25T09:42:25.307914+0000","last_became_peered":"2023-08-25T09:42:25.307914+0000","last_unstale":"2023-08-25T09:42:26.958409+0000","last_undegraded":"2023-08-25T09:42:26.958409+0000","last_fullsized":"2023-08-25T09:42:26.958409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:54.097673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835532+0000","last_change":"2023-08-25T09:42:25.307697+0000","last_active":"2023-08-25T09:42:26.835532+0000","last_peered":"2023-08-25T09:42:26.835532+0000","last_clean":"2023-08-25T09:42:26.835532+0000","last_became_active":"2023-08-25T09:42:25.307150+0000","last_became_peered":"2023-08-25T09:42:25.307150+0000","last_unstale":"2023-08-25T09:42:26.835532+0000","last_undegraded":"2023-08-25T09:42:26.835532+0000","last_fullsized":"2023-08-25T09:42:26.835532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:37:49.817626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835932+0000","last_change":"2023-08-25T09:42:25.299224+0000","last_active":"2023-08-25T09:42:26.835932+0000","last_peered":"2023-08-25T09:42:26.835932+0000","last_clean":"2023-08-25T09:42:26.835932+0000","last_became_active":"2023-08-25T09:42:25.297844+0000","last_became_peered":"2023-08-25T09:42:25.297844+0000","last_unstale":"2023-08-25T09:42:26.835932+0000","last_undegraded":"2023-08-25T09:42:26.835932+0000","last_fullsized":"2023-08-25T09:42:26.835932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:02:27.508262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.614869+0000","last_change":"2023-08-25T09:42:25.307456+0000","last_active":"2023-08-25T09:42:26.614869+0000","last_peered":"2023-08-25T09:42:26.614869+0000","last_clean":"2023-08-25T09:42:26.614869+0000","last_became_active":"2023-08-25T09:42:25.307041+0000","last_became_peered":"2023-08-25T09:42:25.307041+0000","last_unstale":"2023-08-25T09:42:26.614869+0000","last_undegraded":"2023-08-25T09:42:26.614869+0000","last_fullsized":"2023-08-25T09:42:26.614869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:37.616311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615264+0000","last_change":"2023-08-25T09:42:25.310305+0000","last_active":"2023-08-25T09:42:26.615264+0000","last_peered":"2023-08-25T09:42:26.615264+0000","last_clean":"2023-08-25T09:42:26.615264+0000","last_became_active":"2023-08-25T09:42:25.310049+0000","last_became_peered":"2023-08-25T09:42:25.310049+0000","last_unstale":"2023-08-25T09:42:26.615264+0000","last_undegraded":"2023-08-25T09:42:26.615264+0000","last_fullsized":"2023-08-25T09:42:26.615264+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:18:39.592685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958581+0000","last_change":"2023-08-25T09:42:25.308876+0000","last_active":"2023-08-25T09:42:26.958581+0000","last_peered":"2023-08-25T09:42:26.958581+0000","last_clean":"2023-08-25T09:42:26.958581+0000","last_became_active":"2023-08-25T09:42:25.308300+0000","last_became_peered":"2023-08-25T09:42:25.308300+0000","last_unstale":"2023-08-25T09:42:26.958581+0000","last_undegraded":"2023-08-25T09:42:26.958581+0000","last_fullsized":"2023-08-25T09:42:26.958581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:38:35.707512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293322+0000","last_change":"2023-08-25T09:42:25.314067+0000","last_active":"2023-08-25T09:42:26.293322+0000","last_peered":"2023-08-25T09:42:26.293322+0000","last_clean":"2023-08-25T09:42:26.293322+0000","last_became_active":"2023-08-25T09:42:25.313642+0000","last_became_peered":"2023-08-25T09:42:25.313642+0000","last_unstale":"2023-08-25T09:42:26.293322+0000","last_undegraded":"2023-08-25T09:42:26.293322+0000","last_fullsized":"2023-08-25T09:42:26.293322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:21.405708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959680+0000","last_change":"2023-08-25T09:42:25.318667+0000","last_active":"2023-08-25T09:42:26.959680+0000","last_peered":"2023-08-25T09:42:26.959680+0000","last_clean":"2023-08-25T09:42:26.959680+0000","last_became_active":"2023-08-25T09:42:25.316927+0000","last_became_peered":"2023-08-25T09:42:25.316927+0000","last_unstale":"2023-08-25T09:42:26.959680+0000","last_undegraded":"2023-08-25T09:42:26.959680+0000","last_fullsized":"2023-08-25T09:42:26.959680+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:11:04.711050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836504+0000","last_change":"2023-08-25T09:42:25.311991+0000","last_active":"2023-08-25T09:42:26.836504+0000","last_peered":"2023-08-25T09:42:26.836504+0000","last_clean":"2023-08-25T09:42:26.836504+0000","last_became_active":"2023-08-25T09:42:25.311712+0000","last_became_peered":"2023-08-25T09:42:25.311712+0000","last_unstale":"2023-08-25T09:42:26.836504+0000","last_undegraded":"2023-08-25T09:42:26.836504+0000","last_fullsized":"2023-08-25T09:42:26.836504+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:14.864127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835654+0000","last_change":"2023-08-25T09:42:25.309543+0000","last_active":"2023-08-25T09:42:26.835654+0000","last_peered":"2023-08-25T09:42:26.835654+0000","last_clean":"2023-08-25T09:42:26.835654+0000","last_became_active":"2023-08-25T09:42:25.309186+0000","last_became_peered":"2023-08-25T09:42:25.309186+0000","last_unstale":"2023-08-25T09:42:26.835654+0000","last_undegraded":"2023-08-25T09:42:26.835654+0000","last_fullsized":"2023-08-25T09:42:26.835654+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:15:20.503161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959720+0000","last_change":"2023-08-25T09:42:25.306746+0000","last_active":"2023-08-25T09:42:26.959720+0000","last_peered":"2023-08-25T09:42:26.959720+0000","last_clean":"2023-08-25T09:42:26.959720+0000","last_became_active":"2023-08-25T09:42:25.304143+0000","last_became_peered":"2023-08-25T09:42:25.304143+0000","last_unstale":"2023-08-25T09:42:26.959720+0000","last_undegraded":"2023-08-25T09:42:26.959720+0000","last_fullsized":"2023-08-25T09:42:26.959720+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:16:44.203926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615322+0000","last_change":"2023-08-25T09:42:25.302887+0000","last_active":"2023-08-25T09:42:26.615322+0000","last_peered":"2023-08-25T09:42:26.615322+0000","last_clean":"2023-08-25T09:42:26.615322+0000","last_became_active":"2023-08-25T09:42:25.302336+0000","last_became_peered":"2023-08-25T09:42:25.302336+0000","last_unstale":"2023-08-25T09:42:26.615322+0000","last_undegraded":"2023-08-25T09:42:26.615322+0000","last_fullsized":"2023-08-25T09:42:26.615322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:29.208977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959523+0000","last_change":"2023-08-25T09:42:25.309100+0000","last_active":"2023-08-25T09:42:26.959523+0000","last_peered":"2023-08-25T09:42:26.959523+0000","last_clean":"2023-08-25T09:42:26.959523+0000","last_became_active":"2023-08-25T09:42:25.308130+0000","last_became_peered":"2023-08-25T09:42:25.308130+0000","last_unstale":"2023-08-25T09:42:26.959523+0000","last_undegraded":"2023-08-25T09:42:26.959523+0000","last_fullsized":"2023-08-25T09:42:26.959523+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:38:59.737305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291510+0000","last_change":"2023-08-25T09:42:25.312306+0000","last_active":"2023-08-25T09:42:26.291510+0000","last_peered":"2023-08-25T09:42:26.291510+0000","last_clean":"2023-08-25T09:42:26.291510+0000","last_became_active":"2023-08-25T09:42:25.311957+0000","last_became_peered":"2023-08-25T09:42:25.311957+0000","last_unstale":"2023-08-25T09:42:26.291510+0000","last_undegraded":"2023-08-25T09:42:26.291510+0000","last_fullsized":"2023-08-25T09:42:26.291510+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:16.474501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287095+0000","last_change":"2023-08-25T09:42:25.310727+0000","last_active":"2023-08-25T09:42:26.287095+0000","last_peered":"2023-08-25T09:42:26.287095+0000","last_clean":"2023-08-25T09:42:26.287095+0000","last_became_active":"2023-08-25T09:42:25.310457+0000","last_became_peered":"2023-08-25T09:42:25.310457+0000","last_unstale":"2023-08-25T09:42:26.287095+0000","last_undegraded":"2023-08-25T09:42:26.287095+0000","last_fullsized":"2023-08-25T09:42:26.287095+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:31.034087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615441+0000","last_change":"2023-08-25T09:42:25.306018+0000","last_active":"2023-08-25T09:42:26.615441+0000","last_peered":"2023-08-25T09:42:26.615441+0000","last_clean":"2023-08-25T09:42:26.615441+0000","last_became_active":"2023-08-25T09:42:25.305638+0000","last_became_peered":"2023-08-25T09:42:25.305638+0000","last_unstale":"2023-08-25T09:42:26.615441+0000","last_undegraded":"2023-08-25T09:42:26.615441+0000","last_fullsized":"2023-08-25T09:42:26.615441+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:28:44.356442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287586+0000","last_change":"2023-08-25T09:42:25.306734+0000","last_active":"2023-08-25T09:42:26.287586+0000","last_peered":"2023-08-25T09:42:26.287586+0000","last_clean":"2023-08-25T09:42:26.287586+0000","last_became_active":"2023-08-25T09:42:25.306325+0000","last_became_peered":"2023-08-25T09:42:25.306325+0000","last_unstale":"2023-08-25T09:42:26.287586+0000","last_undegraded":"2023-08-25T09:42:26.287586+0000","last_fullsized":"2023-08-25T09:42:26.287586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:48.184593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615412+0000","last_change":"2023-08-25T09:42:25.314653+0000","last_active":"2023-08-25T09:42:26.615412+0000","last_peered":"2023-08-25T09:42:26.615412+0000","last_clean":"2023-08-25T09:42:26.615412+0000","last_became_active":"2023-08-25T09:42:25.314372+0000","last_became_peered":"2023-08-25T09:42:25.314372+0000","last_unstale":"2023-08-25T09:42:26.615412+0000","last_undegraded":"2023-08-25T09:42:26.615412+0000","last_fullsized":"2023-08-25T09:42:26.615412+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:00.467750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292201+0000","last_change":"2023-08-25T09:42:25.311064+0000","last_active":"2023-08-25T09:42:26.292201+0000","last_peered":"2023-08-25T09:42:26.292201+0000","last_clean":"2023-08-25T09:42:26.292201+0000","last_became_active":"2023-08-25T09:42:25.309531+0000","last_became_peered":"2023-08-25T09:42:25.309531+0000","last_unstale":"2023-08-25T09:42:26.292201+0000","last_undegraded":"2023-08-25T09:42:26.292201+0000","last_fullsized":"2023-08-25T09:42:26.292201+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:00:19.936765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835431+0000","last_change":"2023-08-25T09:42:25.300679+0000","last_active":"2023-08-25T09:42:26.835431+0000","last_peered":"2023-08-25T09:42:26.835431+0000","last_clean":"2023-08-25T09:42:26.835431+0000","last_became_active":"2023-08-25T09:42:25.300224+0000","last_became_peered":"2023-08-25T09:42:25.300224+0000","last_unstale":"2023-08-25T09:42:26.835431+0000","last_undegraded":"2023-08-25T09:42:26.835431+0000","last_fullsized":"2023-08-25T09:42:26.835431+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:26:57.958880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836311+0000","last_change":"2023-08-25T09:42:25.302735+0000","last_active":"2023-08-25T09:42:26.836311+0000","last_peered":"2023-08-25T09:42:26.836311+0000","last_clean":"2023-08-25T09:42:26.836311+0000","last_became_active":"2023-08-25T09:42:25.302467+0000","last_became_peered":"2023-08-25T09:42:25.302467+0000","last_unstale":"2023-08-25T09:42:26.836311+0000","last_undegraded":"2023-08-25T09:42:26.836311+0000","last_fullsized":"2023-08-25T09:42:26.836311+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:16.735134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288245+0000","last_change":"2023-08-25T09:42:25.312535+0000","last_active":"2023-08-25T09:42:26.288245+0000","last_peered":"2023-08-25T09:42:26.288245+0000","last_clean":"2023-08-25T09:42:26.288245+0000","last_became_active":"2023-08-25T09:42:25.312129+0000","last_became_peered":"2023-08-25T09:42:25.312129+0000","last_unstale":"2023-08-25T09:42:26.288245+0000","last_undegraded":"2023-08-25T09:42:26.288245+0000","last_fullsized":"2023-08-25T09:42:26.288245+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:49:00.167189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292696+0000","last_change":"2023-08-25T09:42:25.311364+0000","last_active":"2023-08-25T09:42:26.292696+0000","last_peered":"2023-08-25T09:42:26.292696+0000","last_clean":"2023-08-25T09:42:26.292696+0000","last_became_active":"2023-08-25T09:42:25.311111+0000","last_became_peered":"2023-08-25T09:42:25.311111+0000","last_unstale":"2023-08-25T09:42:26.292696+0000","last_undegraded":"2023-08-25T09:42:26.292696+0000","last_fullsized":"2023-08-25T09:42:26.292696+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:26:57.958880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836115+0000","last_change":"2023-08-25T09:42:25.307472+0000","last_active":"2023-08-25T09:42:26.836115+0000","last_peered":"2023-08-25T09:42:26.836115+0000","last_clean":"2023-08-25T09:42:26.836115+0000","last_became_active":"2023-08-25T09:42:25.304647+0000","last_became_peered":"2023-08-25T09:42:25.304647+0000","last_unstale":"2023-08-25T09:42:26.836115+0000","last_undegraded":"2023-08-25T09:42:26.836115+0000","last_fullsized":"2023-08-25T09:42:26.836115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:01:16.816662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"72'4","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292521+0000","last_change":"2023-08-25T09:42:24.304286+0000","last_active":"2023-08-25T09:42:26.292521+0000","last_peered":"2023-08-25T09:42:26.292521+0000","last_clean":"2023-08-25T09:42:26.292521+0000","last_became_active":"2023-08-25T09:42:24.304050+0000","last_became_peered":"2023-08-25T09:42:24.304050+0000","last_unstale":"2023-08-25T09:42:26.292521+0000","last_undegraded":"2023-08-25T09:42:26.292521+0000","last_fullsized":"2023-08-25T09:42:26.292521+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:57.433491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287792+0000","last_change":"2023-08-25T09:42:25.294485+0000","last_active":"2023-08-25T09:42:26.287792+0000","last_peered":"2023-08-25T09:42:26.287792+0000","last_clean":"2023-08-25T09:42:26.287792+0000","last_became_active":"2023-08-25T09:42:25.293864+0000","last_became_peered":"2023-08-25T09:42:25.293864+0000","last_unstale":"2023-08-25T09:42:26.287792+0000","last_undegraded":"2023-08-25T09:42:26.287792+0000","last_fullsized":"2023-08-25T09:42:26.287792+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:15:56.713821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292389+0000","last_change":"2023-08-25T09:42:24.291550+0000","last_active":"2023-08-25T09:42:26.292389+0000","last_peered":"2023-08-25T09:42:26.292389+0000","last_clean":"2023-08-25T09:42:26.292389+0000","last_became_active":"2023-08-25T09:42:24.290087+0000","last_became_peered":"2023-08-25T09:42:24.290087+0000","last_unstale":"2023-08-25T09:42:26.292389+0000","last_undegraded":"2023-08-25T09:42:26.292389+0000","last_fullsized":"2023-08-25T09:42:26.292389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:23:02.401341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959587+0000","last_change":"2023-08-25T09:42:25.311690+0000","last_active":"2023-08-25T09:42:26.959587+0000","last_peered":"2023-08-25T09:42:26.959587+0000","last_clean":"2023-08-25T09:42:26.959587+0000","last_became_active":"2023-08-25T09:42:25.311192+0000","last_became_peered":"2023-08-25T09:42:25.311192+0000","last_unstale":"2023-08-25T09:42:26.959587+0000","last_undegraded":"2023-08-25T09:42:26.959587+0000","last_fullsized":"2023-08-25T09:42:26.959587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:39.023744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293667+0000","last_change":"2023-08-25T09:42:24.305143+0000","last_active":"2023-08-25T09:42:26.293667+0000","last_peered":"2023-08-25T09:42:26.293667+0000","last_clean":"2023-08-25T09:42:26.293667+0000","last_became_active":"2023-08-25T09:42:24.304889+0000","last_became_peered":"2023-08-25T09:42:24.304889+0000","last_unstale":"2023-08-25T09:42:26.293667+0000","last_undegraded":"2023-08-25T09:42:26.293667+0000","last_fullsized":"2023-08-25T09:42:26.293667+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:02:26.502294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836829+0000","last_change":"2023-08-25T09:42:25.306837+0000","last_active":"2023-08-25T09:42:26.836829+0000","last_peered":"2023-08-25T09:42:26.836829+0000","last_clean":"2023-08-25T09:42:26.836829+0000","last_became_active":"2023-08-25T09:42:25.305194+0000","last_became_peered":"2023-08-25T09:42:25.305194+0000","last_unstale":"2023-08-25T09:42:26.836829+0000","last_undegraded":"2023-08-25T09:42:26.836829+0000","last_fullsized":"2023-08-25T09:42:26.836829+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:15:44.243511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960437+0000","last_change":"2023-08-25T09:42:24.307070+0000","last_active":"2023-08-25T09:42:26.960437+0000","last_peered":"2023-08-25T09:42:26.960437+0000","last_clean":"2023-08-25T09:42:26.960437+0000","last_became_active":"2023-08-25T09:42:24.305606+0000","last_became_peered":"2023-08-25T09:42:24.305606+0000","last_unstale":"2023-08-25T09:42:26.960437+0000","last_undegraded":"2023-08-25T09:42:26.960437+0000","last_fullsized":"2023-08-25T09:42:26.960437+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:16.112292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960090+0000","last_change":"2023-08-25T09:42:25.317917+0000","last_active":"2023-08-25T09:42:26.960090+0000","last_peered":"2023-08-25T09:42:26.960090+0000","last_clean":"2023-08-25T09:42:26.960090+0000","last_became_active":"2023-08-25T09:42:25.308995+0000","last_became_peered":"2023-08-25T09:42:25.308995+0000","last_unstale":"2023-08-25T09:42:26.960090+0000","last_undegraded":"2023-08-25T09:42:26.960090+0000","last_fullsized":"2023-08-25T09:42:26.960090+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:15.399316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960349+0000","last_change":"2023-08-25T09:42:24.305408+0000","last_active":"2023-08-25T09:42:26.960349+0000","last_peered":"2023-08-25T09:42:26.960349+0000","last_clean":"2023-08-25T09:42:26.960349+0000","last_became_active":"2023-08-25T09:42:24.304874+0000","last_became_peered":"2023-08-25T09:42:24.304874+0000","last_unstale":"2023-08-25T09:42:26.960349+0000","last_undegraded":"2023-08-25T09:42:26.960349+0000","last_fullsized":"2023-08-25T09:42:26.960349+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:53:02.842270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291635+0000","last_change":"2023-08-25T09:42:25.306618+0000","last_active":"2023-08-25T09:42:26.291635+0000","last_peered":"2023-08-25T09:42:26.291635+0000","last_clean":"2023-08-25T09:42:26.291635+0000","last_became_active":"2023-08-25T09:42:25.306231+0000","last_became_peered":"2023-08-25T09:42:25.306231+0000","last_unstale":"2023-08-25T09:42:26.291635+0000","last_undegraded":"2023-08-25T09:42:26.291635+0000","last_fullsized":"2023-08-25T09:42:26.291635+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:10:41.738975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959375+0000","last_change":"2023-08-25T09:42:24.306085+0000","last_active":"2023-08-25T09:42:26.959375+0000","last_peered":"2023-08-25T09:42:26.959375+0000","last_clean":"2023-08-25T09:42:26.959375+0000","last_became_active":"2023-08-25T09:42:24.305829+0000","last_became_peered":"2023-08-25T09:42:24.305829+0000","last_unstale":"2023-08-25T09:42:26.959375+0000","last_undegraded":"2023-08-25T09:42:26.959375+0000","last_fullsized":"2023-08-25T09:42:26.959375+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:38:02.470521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957904+0000","last_change":"2023-08-25T09:42:25.301456+0000","last_active":"2023-08-25T09:42:26.957904+0000","last_peered":"2023-08-25T09:42:26.957904+0000","last_clean":"2023-08-25T09:42:26.957904+0000","last_became_active":"2023-08-25T09:42:25.298185+0000","last_became_peered":"2023-08-25T09:42:25.298185+0000","last_unstale":"2023-08-25T09:42:26.957904+0000","last_undegraded":"2023-08-25T09:42:26.957904+0000","last_fullsized":"2023-08-25T09:42:26.957904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:17.613490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293161+0000","last_change":"2023-08-25T09:42:24.292740+0000","last_active":"2023-08-25T09:42:26.293161+0000","last_peered":"2023-08-25T09:42:26.293161+0000","last_clean":"2023-08-25T09:42:26.293161+0000","last_became_active":"2023-08-25T09:42:24.292159+0000","last_became_peered":"2023-08-25T09:42:24.292159+0000","last_unstale":"2023-08-25T09:42:26.293161+0000","last_undegraded":"2023-08-25T09:42:26.293161+0000","last_fullsized":"2023-08-25T09:42:26.293161+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:15:19.497193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959323+0000","last_change":"2023-08-25T09:42:25.318093+0000","last_active":"2023-08-25T09:42:26.959323+0000","last_peered":"2023-08-25T09:42:26.959323+0000","last_clean":"2023-08-25T09:42:26.959323+0000","last_became_active":"2023-08-25T09:42:25.316206+0000","last_became_peered":"2023-08-25T09:42:25.316206+0000","last_unstale":"2023-08-25T09:42:26.959323+0000","last_undegraded":"2023-08-25T09:42:26.959323+0000","last_fullsized":"2023-08-25T09:42:26.959323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:08.384644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836628+0000","last_change":"2023-08-25T09:42:24.307057+0000","last_active":"2023-08-25T09:42:26.836628+0000","last_peered":"2023-08-25T09:42:26.836628+0000","last_clean":"2023-08-25T09:42:26.836628+0000","last_became_active":"2023-08-25T09:42:24.306810+0000","last_became_peered":"2023-08-25T09:42:24.306810+0000","last_unstale":"2023-08-25T09:42:26.836628+0000","last_undegraded":"2023-08-25T09:42:26.836628+0000","last_fullsized":"2023-08-25T09:42:26.836628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:21.502328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292020+0000","last_change":"2023-08-25T09:42:25.312120+0000","last_active":"2023-08-25T09:42:26.292020+0000","last_peered":"2023-08-25T09:42:26.292020+0000","last_clean":"2023-08-25T09:42:26.292020+0000","last_became_active":"2023-08-25T09:42:25.311822+0000","last_became_peered":"2023-08-25T09:42:25.311822+0000","last_unstale":"2023-08-25T09:42:26.292020+0000","last_undegraded":"2023-08-25T09:42:26.292020+0000","last_fullsized":"2023-08-25T09:42:26.292020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:12:54.337914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836465+0000","last_change":"2023-08-25T09:42:24.307403+0000","last_active":"2023-08-25T09:42:26.836465+0000","last_peered":"2023-08-25T09:42:26.836465+0000","last_clean":"2023-08-25T09:42:26.836465+0000","last_became_active":"2023-08-25T09:42:24.307148+0000","last_became_peered":"2023-08-25T09:42:24.307148+0000","last_unstale":"2023-08-25T09:42:26.836465+0000","last_undegraded":"2023-08-25T09:42:26.836465+0000","last_fullsized":"2023-08-25T09:42:26.836465+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:52:45.387058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960380+0000","last_change":"2023-08-25T09:42:25.309429+0000","last_active":"2023-08-25T09:42:26.960380+0000","last_peered":"2023-08-25T09:42:26.960380+0000","last_clean":"2023-08-25T09:42:26.960380+0000","last_became_active":"2023-08-25T09:42:25.309094+0000","last_became_peered":"2023-08-25T09:42:25.309094+0000","last_unstale":"2023-08-25T09:42:26.960380+0000","last_undegraded":"2023-08-25T09:42:26.960380+0000","last_fullsized":"2023-08-25T09:42:26.960380+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:47.499893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615885+0000","last_change":"2023-08-25T09:42:24.302558+0000","last_active":"2023-08-25T09:42:26.615885+0000","last_peered":"2023-08-25T09:42:26.615885+0000","last_clean":"2023-08-25T09:42:26.615885+0000","last_became_active":"2023-08-25T09:42:24.302055+0000","last_became_peered":"2023-08-25T09:42:24.302055+0000","last_unstale":"2023-08-25T09:42:26.615885+0000","last_undegraded":"2023-08-25T09:42:26.615885+0000","last_fullsized":"2023-08-25T09:42:26.615885+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:19.070872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615617+0000","last_change":"2023-08-25T09:42:25.304157+0000","last_active":"2023-08-25T09:42:26.615617+0000","last_peered":"2023-08-25T09:42:26.615617+0000","last_clean":"2023-08-25T09:42:26.615617+0000","last_became_active":"2023-08-25T09:42:25.303877+0000","last_became_peered":"2023-08-25T09:42:25.303877+0000","last_unstale":"2023-08-25T09:42:26.615617+0000","last_undegraded":"2023-08-25T09:42:26.615617+0000","last_fullsized":"2023-08-25T09:42:26.615617+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:34:59.383014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"72'3","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288534+0000","last_change":"2023-08-25T09:42:24.307353+0000","last_active":"2023-08-25T09:42:26.288534+0000","last_peered":"2023-08-25T09:42:26.288534+0000","last_clean":"2023-08-25T09:42:26.288534+0000","last_became_active":"2023-08-25T09:42:24.305922+0000","last_became_peered":"2023-08-25T09:42:24.305922+0000","last_unstale":"2023-08-25T09:42:26.288534+0000","last_undegraded":"2023-08-25T09:42:26.288534+0000","last_fullsized":"2023-08-25T09:42:26.288534+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:23.839075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287949+0000","last_change":"2023-08-25T09:42:25.312360+0000","last_active":"2023-08-25T09:42:26.287949+0000","last_peered":"2023-08-25T09:42:26.287949+0000","last_clean":"2023-08-25T09:42:26.287949+0000","last_became_active":"2023-08-25T09:42:25.311723+0000","last_became_peered":"2023-08-25T09:42:25.311723+0000","last_unstale":"2023-08-25T09:42:26.287949+0000","last_undegraded":"2023-08-25T09:42:26.287949+0000","last_fullsized":"2023-08-25T09:42:26.287949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:27:23.674251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"72'6","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960141+0000","last_change":"2023-08-25T09:42:24.304814+0000","last_active":"2023-08-25T09:42:26.960141+0000","last_peered":"2023-08-25T09:42:26.960141+0000","last_clean":"2023-08-25T09:42:26.960141+0000","last_became_active":"2023-08-25T09:42:24.302203+0000","last_became_peered":"2023-08-25T09:42:24.302203+0000","last_unstale":"2023-08-25T09:42:26.960141+0000","last_undegraded":"2023-08-25T09:42:26.960141+0000","last_fullsized":"2023-08-25T09:42:26.960141+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:44:31.715159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958165+0000","last_change":"2023-08-25T09:42:25.302156+0000","last_active":"2023-08-25T09:42:26.958165+0000","last_peered":"2023-08-25T09:42:26.958165+0000","last_clean":"2023-08-25T09:42:26.958165+0000","last_became_active":"2023-08-25T09:42:25.301767+0000","last_became_peered":"2023-08-25T09:42:25.301767+0000","last_unstale":"2023-08-25T09:42:26.958165+0000","last_undegraded":"2023-08-25T09:42:26.958165+0000","last_fullsized":"2023-08-25T09:42:26.958165+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:22:11.515926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836961+0000","last_change":"2023-08-25T09:42:24.302995+0000","last_active":"2023-08-25T09:42:26.836961+0000","last_peered":"2023-08-25T09:42:26.836961+0000","last_clean":"2023-08-25T09:42:26.836961+0000","last_became_active":"2023-08-25T09:42:24.302602+0000","last_became_peered":"2023-08-25T09:42:24.302602+0000","last_unstale":"2023-08-25T09:42:26.836961+0000","last_undegraded":"2023-08-25T09:42:26.836961+0000","last_fullsized":"2023-08-25T09:42:26.836961+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:44.114078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957872+0000","last_change":"2023-08-25T09:42:25.317621+0000","last_active":"2023-08-25T09:42:26.957872+0000","last_peered":"2023-08-25T09:42:26.957872+0000","last_clean":"2023-08-25T09:42:26.957872+0000","last_became_active":"2023-08-25T09:42:25.317373+0000","last_became_peered":"2023-08-25T09:42:25.317373+0000","last_unstale":"2023-08-25T09:42:26.957872+0000","last_undegraded":"2023-08-25T09:42:26.957872+0000","last_fullsized":"2023-08-25T09:42:26.957872+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:21.405708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958205+0000","last_change":"2023-08-25T09:42:24.302248+0000","last_active":"2023-08-25T09:42:26.958205+0000","last_peered":"2023-08-25T09:42:26.958205+0000","last_clean":"2023-08-25T09:42:26.958205+0000","last_became_active":"2023-08-25T09:42:24.301884+0000","last_became_peered":"2023-08-25T09:42:24.301884+0000","last_unstale":"2023-08-25T09:42:26.958205+0000","last_undegraded":"2023-08-25T09:42:26.958205+0000","last_fullsized":"2023-08-25T09:42:26.958205+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:14.589792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957867+0000","last_change":"2023-08-25T09:42:25.312890+0000","last_active":"2023-08-25T09:42:26.957867+0000","last_peered":"2023-08-25T09:42:26.957867+0000","last_clean":"2023-08-25T09:42:26.957867+0000","last_became_active":"2023-08-25T09:42:25.312634+0000","last_became_peered":"2023-08-25T09:42:25.312634+0000","last_unstale":"2023-08-25T09:42:26.957867+0000","last_undegraded":"2023-08-25T09:42:26.957867+0000","last_fullsized":"2023-08-25T09:42:26.957867+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:11.469930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287905+0000","last_change":"2023-08-25T09:42:24.306160+0000","last_active":"2023-08-25T09:42:26.287905+0000","last_peered":"2023-08-25T09:42:26.287905+0000","last_clean":"2023-08-25T09:42:26.287905+0000","last_became_active":"2023-08-25T09:42:24.305749+0000","last_became_peered":"2023-08-25T09:42:24.305749+0000","last_unstale":"2023-08-25T09:42:26.287905+0000","last_undegraded":"2023-08-25T09:42:26.287905+0000","last_fullsized":"2023-08-25T09:42:26.287905+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:16.138562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293479+0000","last_change":"2023-08-25T09:42:25.311281+0000","last_active":"2023-08-25T09:42:26.293479+0000","last_peered":"2023-08-25T09:42:26.293479+0000","last_clean":"2023-08-25T09:42:26.293479+0000","last_became_active":"2023-08-25T09:42:25.310911+0000","last_became_peered":"2023-08-25T09:42:25.310911+0000","last_unstale":"2023-08-25T09:42:26.293479+0000","last_undegraded":"2023-08-25T09:42:26.293479+0000","last_fullsized":"2023-08-25T09:42:26.293479+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:05.008842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"72'8","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958600+0000","last_change":"2023-08-25T09:42:24.293445+0000","last_active":"2023-08-25T09:42:26.958600+0000","last_peered":"2023-08-25T09:42:26.958600+0000","last_clean":"2023-08-25T09:42:26.958600+0000","last_became_active":"2023-08-25T09:42:24.293158+0000","last_became_peered":"2023-08-25T09:42:24.293158+0000","last_unstale":"2023-08-25T09:42:26.958600+0000","last_undegraded":"2023-08-25T09:42:26.958600+0000","last_fullsized":"2023-08-25T09:42:26.958600+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:10.198092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960442+0000","last_change":"2023-08-25T09:42:25.318282+0000","last_active":"2023-08-25T09:42:26.960442+0000","last_peered":"2023-08-25T09:42:26.960442+0000","last_clean":"2023-08-25T09:42:26.960442+0000","last_became_active":"2023-08-25T09:42:25.309358+0000","last_became_peered":"2023-08-25T09:42:25.309358+0000","last_unstale":"2023-08-25T09:42:26.960442+0000","last_undegraded":"2023-08-25T09:42:26.960442+0000","last_fullsized":"2023-08-25T09:42:26.960442+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:26:43.451903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836317+0000","last_change":"2023-08-25T09:42:24.305003+0000","last_active":"2023-08-25T09:42:26.836317+0000","last_peered":"2023-08-25T09:42:26.836317+0000","last_clean":"2023-08-25T09:42:26.836317+0000","last_became_active":"2023-08-25T09:42:24.304721+0000","last_became_peered":"2023-08-25T09:42:24.304721+0000","last_unstale":"2023-08-25T09:42:26.836317+0000","last_undegraded":"2023-08-25T09:42:26.836317+0000","last_fullsized":"2023-08-25T09:42:26.836317+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:32:25.305128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960670+0000","last_change":"2023-08-25T09:42:25.309064+0000","last_active":"2023-08-25T09:42:26.960670+0000","last_peered":"2023-08-25T09:42:26.960670+0000","last_clean":"2023-08-25T09:42:26.960670+0000","last_became_active":"2023-08-25T09:42:25.308362+0000","last_became_peered":"2023-08-25T09:42:25.308362+0000","last_unstale":"2023-08-25T09:42:26.960670+0000","last_undegraded":"2023-08-25T09:42:26.960670+0000","last_fullsized":"2023-08-25T09:42:26.960670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:10.299581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"73'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.837025+0000","last_change":"2023-08-25T09:42:24.305708+0000","last_active":"2023-08-25T09:42:26.837025+0000","last_peered":"2023-08-25T09:42:26.837025+0000","last_clean":"2023-08-25T09:42:26.837025+0000","last_became_active":"2023-08-25T09:42:24.305437+0000","last_became_peered":"2023-08-25T09:42:24.305437+0000","last_unstale":"2023-08-25T09:42:26.837025+0000","last_undegraded":"2023-08-25T09:42:26.837025+0000","last_fullsized":"2023-08-25T09:42:26.837025+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:02:25.683034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288036+0000","last_change":"2023-08-25T09:42:25.304424+0000","last_active":"2023-08-25T09:42:26.288036+0000","last_peered":"2023-08-25T09:42:26.288036+0000","last_clean":"2023-08-25T09:42:26.288036+0000","last_became_active":"2023-08-25T09:42:25.304153+0000","last_became_peered":"2023-08-25T09:42:25.304153+0000","last_unstale":"2023-08-25T09:42:26.288036+0000","last_undegraded":"2023-08-25T09:42:26.288036+0000","last_fullsized":"2023-08-25T09:42:26.288036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:09.952773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960468+0000","last_change":"2023-08-25T09:42:24.305243+0000","last_active":"2023-08-25T09:42:26.960468+0000","last_peered":"2023-08-25T09:42:26.960468+0000","last_clean":"2023-08-25T09:42:26.960468+0000","last_became_active":"2023-08-25T09:42:24.304555+0000","last_became_peered":"2023-08-25T09:42:24.304555+0000","last_unstale":"2023-08-25T09:42:26.960468+0000","last_undegraded":"2023-08-25T09:42:26.960468+0000","last_fullsized":"2023-08-25T09:42:26.960468+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:14:59.771777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288529+0000","last_change":"2023-08-25T09:42:25.311088+0000","last_active":"2023-08-25T09:42:26.288529+0000","last_peered":"2023-08-25T09:42:26.288529+0000","last_clean":"2023-08-25T09:42:26.288529+0000","last_became_active":"2023-08-25T09:42:25.310554+0000","last_became_peered":"2023-08-25T09:42:25.310554+0000","last_unstale":"2023-08-25T09:42:26.288529+0000","last_undegraded":"2023-08-25T09:42:26.288529+0000","last_fullsized":"2023-08-25T09:42:26.288529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:06.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292571+0000","last_change":"2023-08-25T09:42:24.301764+0000","last_active":"2023-08-25T09:42:26.292571+0000","last_peered":"2023-08-25T09:42:26.292571+0000","last_clean":"2023-08-25T09:42:26.292571+0000","last_became_active":"2023-08-25T09:42:24.301475+0000","last_became_peered":"2023-08-25T09:42:24.301475+0000","last_unstale":"2023-08-25T09:42:26.292571+0000","last_undegraded":"2023-08-25T09:42:26.292571+0000","last_fullsized":"2023-08-25T09:42:26.292571+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:10.463961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836059+0000","last_change":"2023-08-25T09:42:25.300480+0000","last_active":"2023-08-25T09:42:26.836059+0000","last_peered":"2023-08-25T09:42:26.836059+0000","last_clean":"2023-08-25T09:42:26.836059+0000","last_became_active":"2023-08-25T09:42:25.299879+0000","last_became_peered":"2023-08-25T09:42:25.299879+0000","last_unstale":"2023-08-25T09:42:26.836059+0000","last_undegraded":"2023-08-25T09:42:26.836059+0000","last_fullsized":"2023-08-25T09:42:26.836059+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:14.994192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615578+0000","last_change":"2023-08-25T09:42:24.302869+0000","last_active":"2023-08-25T09:42:26.615578+0000","last_peered":"2023-08-25T09:42:26.615578+0000","last_clean":"2023-08-25T09:42:26.615578+0000","last_became_active":"2023-08-25T09:42:24.302379+0000","last_became_peered":"2023-08-25T09:42:24.302379+0000","last_unstale":"2023-08-25T09:42:26.615578+0000","last_undegraded":"2023-08-25T09:42:26.615578+0000","last_fullsized":"2023-08-25T09:42:26.615578+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:05:22.358354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615119+0000","last_change":"2023-08-25T09:42:25.307304+0000","last_active":"2023-08-25T09:42:26.615119+0000","last_peered":"2023-08-25T09:42:26.615119+0000","last_clean":"2023-08-25T09:42:26.615119+0000","last_became_active":"2023-08-25T09:42:25.306702+0000","last_became_peered":"2023-08-25T09:42:25.306702+0000","last_unstale":"2023-08-25T09:42:26.615119+0000","last_undegraded":"2023-08-25T09:42:26.615119+0000","last_fullsized":"2023-08-25T09:42:26.615119+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:55.792001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.293059+0000","last_change":"2023-08-25T09:42:24.301955+0000","last_active":"2023-08-25T09:42:26.293059+0000","last_peered":"2023-08-25T09:42:26.293059+0000","last_clean":"2023-08-25T09:42:26.293059+0000","last_became_active":"2023-08-25T09:42:24.301280+0000","last_became_peered":"2023-08-25T09:42:24.301280+0000","last_unstale":"2023-08-25T09:42:26.293059+0000","last_undegraded":"2023-08-25T09:42:26.293059+0000","last_fullsized":"2023-08-25T09:42:26.293059+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:37:48.811658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959912+0000","last_change":"2023-08-25T09:42:25.305417+0000","last_active":"2023-08-25T09:42:26.959912+0000","last_peered":"2023-08-25T09:42:26.959912+0000","last_clean":"2023-08-25T09:42:26.959912+0000","last_became_active":"2023-08-25T09:42:25.304933+0000","last_became_peered":"2023-08-25T09:42:25.304933+0000","last_unstale":"2023-08-25T09:42:26.959912+0000","last_undegraded":"2023-08-25T09:42:26.959912+0000","last_fullsized":"2023-08-25T09:42:26.959912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:28.006718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.291833+0000","last_change":"2023-08-25T09:42:24.306699+0000","last_active":"2023-08-25T09:42:26.291833+0000","last_peered":"2023-08-25T09:42:26.291833+0000","last_clean":"2023-08-25T09:42:26.291833+0000","last_became_active":"2023-08-25T09:42:24.306437+0000","last_became_peered":"2023-08-25T09:42:24.306437+0000","last_unstale":"2023-08-25T09:42:26.291833+0000","last_undegraded":"2023-08-25T09:42:26.291833+0000","last_fullsized":"2023-08-25T09:42:26.291833+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:53:33.951474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.961019+0000","last_change":"2023-08-25T09:42:25.303448+0000","last_active":"2023-08-25T09:42:26.961019+0000","last_peered":"2023-08-25T09:42:26.961019+0000","last_clean":"2023-08-25T09:42:26.961019+0000","last_became_active":"2023-08-25T09:42:25.302770+0000","last_became_peered":"2023-08-25T09:42:25.302770+0000","last_unstale":"2023-08-25T09:42:26.961019+0000","last_undegraded":"2023-08-25T09:42:26.961019+0000","last_fullsized":"2023-08-25T09:42:26.961019+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:35:37.982860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960756+0000","last_change":"2023-08-25T09:42:24.305948+0000","last_active":"2023-08-25T09:42:26.960756+0000","last_peered":"2023-08-25T09:42:26.960756+0000","last_clean":"2023-08-25T09:42:26.960756+0000","last_became_active":"2023-08-25T09:42:24.305521+0000","last_became_peered":"2023-08-25T09:42:24.305521+0000","last_unstale":"2023-08-25T09:42:26.960756+0000","last_undegraded":"2023-08-25T09:42:26.960756+0000","last_fullsized":"2023-08-25T09:42:26.960756+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:35:38.184245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958338+0000","last_change":"2023-08-25T09:42:25.310738+0000","last_active":"2023-08-25T09:42:26.958338+0000","last_peered":"2023-08-25T09:42:26.958338+0000","last_clean":"2023-08-25T09:42:26.958338+0000","last_became_active":"2023-08-25T09:42:25.309165+0000","last_became_peered":"2023-08-25T09:42:25.309165+0000","last_unstale":"2023-08-25T09:42:26.958338+0000","last_undegraded":"2023-08-25T09:42:26.958338+0000","last_fullsized":"2023-08-25T09:42:26.958338+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:44.760882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960755+0000","last_change":"2023-08-25T09:42:24.306848+0000","last_active":"2023-08-25T09:42:26.960755+0000","last_peered":"2023-08-25T09:42:26.960755+0000","last_clean":"2023-08-25T09:42:26.960755+0000","last_became_active":"2023-08-25T09:42:24.305161+0000","last_became_peered":"2023-08-25T09:42:24.305161+0000","last_unstale":"2023-08-25T09:42:26.960755+0000","last_undegraded":"2023-08-25T09:42:26.960755+0000","last_fullsized":"2023-08-25T09:42:26.960755+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:06.503919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":256,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.958025+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:42:26.958025+0000","last_peered":"2023-08-25T09:42:26.958025+0000","last_clean":"2023-08-25T09:42:26.958025+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:42:26.958025+0000","last_undegraded":"2023-08-25T09:42:26.958025+0000","last_fullsized":"2023-08-25T09:42:26.958025+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:46:24.591022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288864+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:42:26.288864+0000","last_peered":"2023-08-25T09:42:26.288864+0000","last_clean":"2023-08-25T09:42:26.288864+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:42:26.288864+0000","last_undegraded":"2023-08-25T09:42:26.288864+0000","last_fullsized":"2023-08-25T09:42:26.288864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:15:25.745588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959927+0000","last_change":"2023-08-25T09:42:25.309959+0000","last_active":"2023-08-25T09:42:26.959927+0000","last_peered":"2023-08-25T09:42:26.959927+0000","last_clean":"2023-08-25T09:42:26.959927+0000","last_became_active":"2023-08-25T09:42:25.309177+0000","last_became_peered":"2023-08-25T09:42:25.309177+0000","last_unstale":"2023-08-25T09:42:26.959927+0000","last_undegraded":"2023-08-25T09:42:26.959927+0000","last_fullsized":"2023-08-25T09:42:26.959927+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:50.719507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.960016+0000","last_change":"2023-08-25T09:42:24.301077+0000","last_active":"2023-08-25T09:42:26.960016+0000","last_peered":"2023-08-25T09:42:26.960016+0000","last_clean":"2023-08-25T09:42:26.960016+0000","last_became_active":"2023-08-25T09:42:24.298490+0000","last_became_peered":"2023-08-25T09:42:24.298490+0000","last_unstale":"2023-08-25T09:42:26.960016+0000","last_undegraded":"2023-08-25T09:42:26.960016+0000","last_fullsized":"2023-08-25T09:42:26.960016+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:05:10.157081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287633+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:42:26.287633+0000","last_peered":"2023-08-25T09:42:26.287633+0000","last_clean":"2023-08-25T09:42:26.287633+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:42:26.287633+0000","last_undegraded":"2023-08-25T09:42:26.287633+0000","last_fullsized":"2023-08-25T09:42:26.287633+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:57:58.707505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615903+0000","last_change":"2023-08-25T09:42:25.305873+0000","last_active":"2023-08-25T09:42:26.615903+0000","last_peered":"2023-08-25T09:42:26.615903+0000","last_clean":"2023-08-25T09:42:26.615903+0000","last_became_active":"2023-08-25T09:42:25.305296+0000","last_became_peered":"2023-08-25T09:42:25.305296+0000","last_unstale":"2023-08-25T09:42:26.615903+0000","last_undegraded":"2023-08-25T09:42:26.615903+0000","last_fullsized":"2023-08-25T09:42:26.615903+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:24:08.503228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.957806+0000","last_change":"2023-08-25T09:42:24.295421+0000","last_active":"2023-08-25T09:42:26.957806+0000","last_peered":"2023-08-25T09:42:26.957806+0000","last_clean":"2023-08-25T09:42:26.957806+0000","last_became_active":"2023-08-25T09:42:24.294838+0000","last_became_peered":"2023-08-25T09:42:24.294838+0000","last_unstale":"2023-08-25T09:42:26.957806+0000","last_undegraded":"2023-08-25T09:42:26.957806+0000","last_fullsized":"2023-08-25T09:42:26.957806+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:48:20.672723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292151+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:42:26.292151+0000","last_peered":"2023-08-25T09:42:26.292151+0000","last_clean":"2023-08-25T09:42:26.292151+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:42:26.292151+0000","last_undegraded":"2023-08-25T09:42:26.292151+0000","last_fullsized":"2023-08-25T09:42:26.292151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:44:43.405694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.835878+0000","last_change":"2023-08-25T09:42:25.303375+0000","last_active":"2023-08-25T09:42:26.835878+0000","last_peered":"2023-08-25T09:42:26.835878+0000","last_clean":"2023-08-25T09:42:26.835878+0000","last_became_active":"2023-08-25T09:42:25.302988+0000","last_became_peered":"2023-08-25T09:42:25.302988+0000","last_unstale":"2023-08-25T09:42:26.835878+0000","last_undegraded":"2023-08-25T09:42:26.835878+0000","last_fullsized":"2023-08-25T09:42:26.835878+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:34:05.008842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959022+0000","last_change":"2023-08-25T09:42:24.303402+0000","last_active":"2023-08-25T09:42:26.959022+0000","last_peered":"2023-08-25T09:42:26.959022+0000","last_clean":"2023-08-25T09:42:26.959022+0000","last_became_active":"2023-08-25T09:42:24.303131+0000","last_became_peered":"2023-08-25T09:42:24.303131+0000","last_unstale":"2023-08-25T09:42:26.959022+0000","last_undegraded":"2023-08-25T09:42:26.959022+0000","last_fullsized":"2023-08-25T09:42:26.959022+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:41.614060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288840+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:42:26.288840+0000","last_peered":"2023-08-25T09:42:26.288840+0000","last_clean":"2023-08-25T09:42:26.288840+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:42:26.288840+0000","last_undegraded":"2023-08-25T09:42:26.288840+0000","last_fullsized":"2023-08-25T09:42:26.288840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:37:02.531152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836133+0000","last_change":"2023-08-25T09:42:25.299354+0000","last_active":"2023-08-25T09:42:26.836133+0000","last_peered":"2023-08-25T09:42:26.836133+0000","last_clean":"2023-08-25T09:42:26.836133+0000","last_became_active":"2023-08-25T09:42:25.298183+0000","last_became_peered":"2023-08-25T09:42:25.298183+0000","last_unstale":"2023-08-25T09:42:26.836133+0000","last_undegraded":"2023-08-25T09:42:26.836133+0000","last_fullsized":"2023-08-25T09:42:26.836133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:19:45.199434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.836611+0000","last_change":"2023-08-25T09:42:24.304030+0000","last_active":"2023-08-25T09:42:26.836611+0000","last_peered":"2023-08-25T09:42:26.836611+0000","last_clean":"2023-08-25T09:42:26.836611+0000","last_became_active":"2023-08-25T09:42:24.303752+0000","last_became_peered":"2023-08-25T09:42:24.303752+0000","last_unstale":"2023-08-25T09:42:26.836611+0000","last_undegraded":"2023-08-25T09:42:26.836611+0000","last_fullsized":"2023-08-25T09:42:26.836611+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:56.778336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":104,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.959351+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:42:26.959351+0000","last_peered":"2023-08-25T09:42:26.959351+0000","last_clean":"2023-08-25T09:42:26.959351+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:42:26.959351+0000","last_undegraded":"2023-08-25T09:42:26.959351+0000","last_fullsized":"2023-08-25T09:42:26.959351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:04.518492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615458+0000","last_change":"2023-08-25T09:42:25.302644+0000","last_active":"2023-08-25T09:42:26.615458+0000","last_peered":"2023-08-25T09:42:26.615458+0000","last_clean":"2023-08-25T09:42:26.615458+0000","last_became_active":"2023-08-25T09:42:25.301907+0000","last_became_peered":"2023-08-25T09:42:25.301907+0000","last_unstale":"2023-08-25T09:42:26.615458+0000","last_undegraded":"2023-08-25T09:42:26.615458+0000","last_fullsized":"2023-08-25T09:42:26.615458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:31:47.458467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"72'2","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.837273+0000","last_change":"2023-08-25T09:42:24.304596+0000","last_active":"2023-08-25T09:42:26.837273+0000","last_peered":"2023-08-25T09:42:26.837273+0000","last_clean":"2023-08-25T09:42:26.837273+0000","last_became_active":"2023-08-25T09:42:24.304317+0000","last_became_peered":"2023-08-25T09:42:24.304317+0000","last_unstale":"2023-08-25T09:42:26.837273+0000","last_undegraded":"2023-08-25T09:42:26.837273+0000","last_fullsized":"2023-08-25T09:42:26.837273+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:12.308233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287497+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:42:26.287497+0000","last_peered":"2023-08-25T09:42:26.287497+0000","last_clean":"2023-08-25T09:42:26.287497+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:42:26.287497+0000","last_undegraded":"2023-08-25T09:42:26.287497+0000","last_fullsized":"2023-08-25T09:42:26.287497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:14.456424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.615734+0000","last_change":"2023-08-25T09:42:25.302876+0000","last_active":"2023-08-25T09:42:26.615734+0000","last_peered":"2023-08-25T09:42:26.615734+0000","last_clean":"2023-08-25T09:42:26.615734+0000","last_became_active":"2023-08-25T09:42:25.302604+0000","last_became_peered":"2023-08-25T09:42:25.302604+0000","last_unstale":"2023-08-25T09:42:26.615734+0000","last_undegraded":"2023-08-25T09:42:26.615734+0000","last_fullsized":"2023-08-25T09:42:26.615734+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:45:52.202179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"72'1","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288554+0000","last_change":"2023-08-25T09:42:24.292895+0000","last_active":"2023-08-25T09:42:26.288554+0000","last_peered":"2023-08-25T09:42:26.288554+0000","last_clean":"2023-08-25T09:42:26.288554+0000","last_became_active":"2023-08-25T09:42:24.292375+0000","last_became_peered":"2023-08-25T09:42:24.292375+0000","last_unstale":"2023-08-25T09:42:26.288554+0000","last_undegraded":"2023-08-25T09:42:26.288554+0000","last_fullsized":"2023-08-25T09:42:26.288554+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:45.682019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":100,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.961020+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:42:26.961020+0000","last_peered":"2023-08-25T09:42:26.961020+0000","last_clean":"2023-08-25T09:42:26.961020+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:42:26.961020+0000","last_undegraded":"2023-08-25T09:42:26.961020+0000","last_fullsized":"2023-08-25T09:42:26.961020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:37.515711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":11,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.287271+0000","last_change":"2023-08-25T09:42:25.303188+0000","last_active":"2023-08-25T09:42:26.287271+0000","last_peered":"2023-08-25T09:42:26.287271+0000","last_clean":"2023-08-25T09:42:26.287271+0000","last_became_active":"2023-08-25T09:42:25.302930+0000","last_became_peered":"2023-08-25T09:42:25.302930+0000","last_unstale":"2023-08-25T09:42:26.287271+0000","last_undegraded":"2023-08-25T09:42:26.287271+0000","last_fullsized":"2023-08-25T09:42:26.287271+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:24.263584+0000","last_clean_scrub_stamp":"2023-08-25T09:42:24.263584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:41:30.441978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.292111+0000","last_change":"2023-08-25T09:42:24.301838+0000","last_active":"2023-08-25T09:42:26.292111+0000","last_peered":"2023-08-25T09:42:26.292111+0000","last_clean":"2023-08-25T09:42:26.292111+0000","last_became_active":"2023-08-25T09:42:24.300672+0000","last_became_peered":"2023-08-25T09:42:24.300672+0000","last_unstale":"2023-08-25T09:42:26.292111+0000","last_undegraded":"2023-08-25T09:42:26.292111+0000","last_fullsized":"2023-08-25T09:42:26.292111+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:42:23.257616+0000","last_clean_scrub_stamp":"2023-08-25T09:42:23.257616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:22.842620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-08-25T09:42:26.288202+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:42:26.288202+0000","last_peered":"2023-08-25T09:42:26.288202+0000","last_clean":"2023-08-25T09:42:26.288202+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:42:26.288202+0000","last_undegraded":"2023-08-25T09:42:26.288202+0000","last_fullsized":"2023-08-25T09:42:26.288202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:01:39.204663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194922,"statfs":{"total":200000000,"available":199600157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5629999999999999,"5min":1.1499999999999999,"15min":1.052},"min":{"1min":0.68600000000000005,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":10.747,"5min":10.747,"15min":10.747},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0249999999999999,"15min":1.008},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.395,"5min":1.599,"15min":1.599},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.1319999999999999,"15min":1.111},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.61299999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.125,"15min":1.073},"min":{"1min":0.748,"5min":0.745,"15min":0.745},"max":{"1min":1.6679999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.091,"15min":1.0569999999999999},"min":{"1min":0.745,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.48,"5min":1.512,"15min":1.512},"last":0.50600000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2190000000000001,"15min":1.2},"min":{"1min":0.80500000000000005,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.167,"15min":1.143},"min":{"1min":0.84199999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.417,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":1.341,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":1.0109999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.80500000000000005,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":7.1639999999999997,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.2929999999999999,"15min":1.2769999999999999},"min":{"1min":0.874,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.571,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.365,"15min":1.3149999999999999},"min":{"1min":1.1519999999999999,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.7490000000000001,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.46,"5min":1.391,"15min":1.361},"min":{"1min":1.1200000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.784,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":0.64100000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.444,"15min":1.411},"min":{"1min":1.1699999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8440000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4059999999999999,"15min":1.3759999999999999},"min":{"1min":0.89900000000000002,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.9590000000000001,"5min":2.0139999999999998,"15min":2.0139999999999998},"last":0.71199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":392,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194919,"statfs":{"total":200000000,"available":199597893,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.1240000000000001,"15min":1.145},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.173,"15min":1.139},"min":{"1min":0.79900000000000004,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":5.5410000000000004,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.099}]},{"osd":1,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.23,"15min":1.1950000000000001},"min":{"1min":0.81599999999999995,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":5.6799999999999997,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.1240000000000001,"15min":1.0980000000000001},"min":{"1min":0.93899999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.677,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.1180000000000001,"15min":1.089},"min":{"1min":0.76700000000000002,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.708,"5min":1.708,"15min":1.708},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.857,"5min":1.296,"15min":1.1819999999999999},"min":{"1min":0.93600000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":13.193,"5min":13.193,"15min":13.193},"last":1.357}]},{"osd":3,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.2709999999999999,"15min":1.212},"min":{"1min":0.999,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":5.6239999999999997,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.609},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.262,"15min":1.252},"min":{"1min":0.85899999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.7150000000000001,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2070000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2809999999999999,"15min":1.232},"min":{"1min":0.79000000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.768,"5min":2.484,"15min":2.484},"last":1.169},{"interface":"front","average":{"1min":1.6220000000000001,"5min":1.3700000000000001,"15min":1.3120000000000001},"min":{"1min":0.88500000000000001,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":5.4699999999999998,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.72}]},{"osd":5,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.2889999999999999,"15min":1.2669999999999999},"min":{"1min":0.93799999999999994,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6160000000000001,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.379},{"interface":"front","average":{"1min":1.538,"5min":1.3979999999999999,"15min":1.345},"min":{"1min":1.2929999999999999,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.8300000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4059999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:41:45 2023","interfaces":[{"interface":"back","average":{"1min":1.794,"5min":1.54,"15min":1.4910000000000001},"min":{"1min":1.1850000000000001,"5min":0.622,"15min":0.622},"max":{"1min":6.7709999999999999,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.819},{"interface":"front","average":{"1min":1.508,"5min":1.4870000000000001,"15min":1.4770000000000001},"min":{"1min":1.1579999999999999,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.8280000000000001,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.3200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":968,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194343,"statfs":{"total":200000000,"available":199008065,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1499999999999999,"15min":1.133},"min":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.5649999999999999,"5min":1.667,"15min":1.667},"last":1.786},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1890000000000001,"15min":1.1970000000000001},"min":{"1min":0.72499999999999998,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.6259999999999999,"5min":1.974,"15min":1.974},"last":1.216}]},{"osd":1,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.1850000000000001,"15min":1.208},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.7250000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.125,"15min":1.1000000000000001},"min":{"1min":0.80800000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":1.282}]},{"osd":2,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.1910000000000001,"15min":1.1699999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":1.655},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.212,"15min":1.206},"min":{"1min":0.96899999999999997,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.8049999999999999,"5min":1.962,"15min":1.962},"last":1.0609999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":2.0590000000000002,"5min":1.401,"15min":1.3140000000000001},"min":{"1min":0.747,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":17.356999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.355,"5min":1.2829999999999999,"15min":1.2709999999999999},"min":{"1min":0.95999999999999996,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.7350000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.3939999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":1.3169999999999999,"15min":1.2909999999999999},"min":{"1min":1.0469999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3420000000000001,"15min":1.3700000000000001},"min":{"1min":0.93100000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.8620000000000001,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.345}]},{"osd":6,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4330000000000001,"15min":1.4299999999999999},"min":{"1min":1.0129999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":2.4820000000000002,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.6379999999999999,"5min":1.4419999999999999,"15min":1.3939999999999999},"min":{"1min":1.385,"5min":0.77800000000000002,"15min":0.77800000000000002},"max":{"1min":2.5539999999999998,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.911}]},{"osd":7,"last update":"Fri Aug 25 09:41:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.4239999999999999,"15min":1.427},"min":{"1min":1.2,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.601,"5min":1.5660000000000001,"15min":1.5720000000000001},"min":{"1min":1.2629999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.911,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.72}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":391,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194921,"statfs":{"total":200000000,"available":199599575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.153,"15min":1.1419999999999999},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.2050000000000001,"15min":1.127},"min":{"1min":0.76400000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":7.367,"5min":7.367,"15min":7.367},"last":1.276}]},{"osd":1,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.103,"15min":1.111},"min":{"1min":0.72199999999999998,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.579,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.3540000000000001,"15min":1.4379999999999999},"min":{"1min":0.78100000000000003,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.613,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.111}]},{"osd":2,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.256,"15min":1.3180000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.5600000000000001,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1619999999999999,"15min":1.1399999999999999},"min":{"1min":0.80000000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.629,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.627}]},{"osd":3,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.29,"15min":1.23},"min":{"1min":0.86499999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":7.2560000000000002,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":1.246,"5min":1.343,"15min":1.3640000000000001},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.548,"5min":1.911,"15min":1.911},"last":1.397}]},{"osd":5,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.294,"15min":1.294},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7090000000000001,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.49},{"interface":"front","average":{"1min":1.633,"5min":1.3009999999999999,"15min":1.2390000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":8.2409999999999997,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.4350000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.407,"15min":1.4299999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8859999999999999,"5min":2.863,"15min":2.863},"last":1.671},{"interface":"front","average":{"1min":1.48,"5min":1.544,"15min":1.5580000000000001},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":2.1070000000000002,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.7090000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.466,"15min":1.3899999999999999},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":8.1760000000000002,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.5649999999999999,"15min":1.585},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.976,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.754}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194921,"statfs":{"total":200000000,"available":199599997,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82799999999999996,"15min":0.82999999999999996},"min":{"1min":0.432,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.4650000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85999999999999999,"15min":0.86699999999999999},"min":{"1min":0.50600000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.3580000000000001,"5min":1.55,"15min":1.55},"last":1.1180000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.80600000000000005,"15min":0.77600000000000002},"min":{"1min":0.39700000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.0840000000000001,"5min":0.95999999999999996,"15min":0.96199999999999997},"min":{"1min":0.52400000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.722,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.85799999999999998,"15min":0.83599999999999997},"min":{"1min":0.45000000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0820000000000001,"15min":1.101},"min":{"1min":0.47699999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8280000000000001,"5min":3.399,"15min":3.399},"last":1.1919999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1699999999999999,"15min":1.1739999999999999},"min":{"1min":0.76100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5649999999999999,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.1930000000000001,"15min":1.1859999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.1840000000000002,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.2949999999999999,"15min":1.3029999999999999},"min":{"1min":0.92600000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.7569999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.411},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.256,"15min":1.26},"min":{"1min":0.81499999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":1.335}]},{"osd":6,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.236,"15min":1.226},"min":{"1min":0.78200000000000003,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.24},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3799999999999999,"15min":1.3999999999999999},"min":{"1min":0.91300000000000003,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.984,"5min":3.496,"15min":3.496},"last":1.3080000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:41:43 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.349,"15min":1.347},"min":{"1min":1.0209999999999999,"5min":0.871,"15min":0.871},"max":{"1min":2.044,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.385},{"interface":"front","average":{"1min":1.45,"5min":1.302,"15min":1.2629999999999999},"min":{"1min":0.88100000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.3520000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":36,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194921,"statfs":{"total":200000000,"available":199600101,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81799999999999995,"15min":0.81999999999999995},"min":{"1min":0.48299999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.589,"5min":2.383,"15min":2.383},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.995,"15min":1.083},"min":{"1min":0.502,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.212,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":1}]},{"osd":1,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.96199999999999997,"15min":0.98399999999999999},"min":{"1min":0.53500000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.2829999999999999,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.036,"15min":1.0569999999999999},"min":{"1min":0.49299999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.518,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":0.83899999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":1.042,"15min":1.115},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.5209999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.1000000000000001},"min":{"1min":0.61899999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.744,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.24}]},{"osd":4,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.353,"15min":1.4119999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.0099999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.2969999999999999,"15min":1.3440000000000001},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7909999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.5640000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.335,"15min":1.3979999999999999},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.639,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.339},{"interface":"front","average":{"1min":1.369,"5min":1.3839999999999999,"15min":1.4019999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0249999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.3029999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3959999999999999,"15min":1.425},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.704,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4330000000000001,"15min":1.4970000000000001},"min":{"1min":0.997,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.7749999999999999,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.506}]},{"osd":7,"last update":"Fri Aug 25 09:41:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.4219999999999999,"15min":1.462},"min":{"1min":0.85699999999999998,"5min":0.746,"15min":0.746},"max":{"1min":2.0640000000000001,"5min":2.968,"15min":2.968},"last":1.206},{"interface":"front","average":{"1min":1.399,"5min":1.472,"15min":1.522},"min":{"1min":0.95199999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.841,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.401}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":967,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194345,"statfs":{"total":200000000,"available":199009755,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.75900000000000001,"15min":0.753},"min":{"1min":0.36299999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.355,"5min":1.627,"15min":1.627},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.79700000000000004,"15min":0.75700000000000001},"min":{"1min":0.50600000000000001,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.79600000000000004,"15min":0.77600000000000002},"min":{"1min":0.47899999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.117,"5min":0.91700000000000004,"15min":0.88300000000000001},"min":{"1min":0.41899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.0449999999999999,"15min":1.0089999999999999},"min":{"1min":0.78700000000000003,"5min":0.372,"15min":0.372},"max":{"1min":1.663,"5min":1.7,"15min":1.7},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.94899999999999995,"15min":0.92100000000000004},"min":{"1min":0.48799999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.53,"5min":1.764,"15min":1.764},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.087,"15min":1.0640000000000001},"min":{"1min":0.77400000000000002,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.236,"15min":1.1930000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.5249999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.208,"15min":1.1779999999999999},"min":{"1min":0.89700000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.823,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.1699999999999999,"15min":1.1519999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.198}]},{"osd":6,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4530000000000001,"5min":1.2989999999999999,"15min":1.2709999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.75,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.284,"15min":1.2689999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.671}]},{"osd":7,"last update":"Fri Aug 25 09:41:46 2023","interfaces":[{"interface":"back","average":{"1min":1.611,"5min":1.3759999999999999,"15min":1.3360000000000001},"min":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":2.9529999999999998,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.665},{"interface":"front","average":{"1min":1.472,"5min":1.3260000000000001,"15min":1.2869999999999999},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9299999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.5629999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":390,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194922,"statfs":{"total":200000000,"available":199600157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":0.91300000000000003,"15min":0.85499999999999998},"min":{"1min":0.34899999999999998,"5min":0.313,"15min":0.313},"max":{"1min":11.257,"5min":11.257,"15min":11.257},"last":0.752},{"interface":"front","average":{"1min":1.252,"5min":0.98099999999999998,"15min":0.96099999999999997},"min":{"1min":0.47099999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":6.4260000000000002,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.86599999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.92600000000000005,"15min":0.93600000000000005},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.21},{"interface":"front","average":{"1min":1.0069999999999999,"5min":1.0569999999999999,"15min":1.1120000000000001},"min":{"1min":0.55300000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2210000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.89700000000000002,"15min":0.90400000000000003},"min":{"1min":0.49199999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.3420000000000001,"5min":1.387,"15min":1.387},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0980000000000001,"15min":1.1259999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.7110000000000001,"5min":1.869,"15min":1.869},"last":1.548}]},{"osd":4,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2170000000000001,"15min":1.2310000000000001},"min":{"1min":0.94199999999999995,"5min":0.625,"15min":0.625},"max":{"1min":1.5649999999999999,"5min":1.704,"15min":1.704},"last":1.155},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.339,"15min":1.355},"min":{"1min":0.877,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.982,"5min":1.982,"15min":1.982},"last":1.6870000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.319,"15min":1.379},"min":{"1min":0.83899999999999997,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.5680000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.2769999999999999,"15min":1.3089999999999999},"min":{"1min":0.97299999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8100000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6419999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3160000000000001,"15min":1.3480000000000001},"min":{"1min":0.82799999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.8380000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.282,"15min":1.294},"min":{"1min":0.93500000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6100000000000001,"5min":1.764,"15min":1.764},"last":1.595}]},{"osd":7,"last update":"Fri Aug 25 09:41:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3819999999999999,"15min":1.4019999999999999},"min":{"1min":0.89100000000000001,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.9139999999999999,"5min":2.121,"15min":2.121},"last":1.514},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3149999999999999,"15min":1.3240000000000001},"min":{"1min":0.94999999999999996,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.4370000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:42:30.228 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:42:30.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:42:30.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.487+0000 7f4e128d2640 1 Processor -- start 2023-08-25T09:42:30.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.488+0000 7f4e128d2640 1 -- start start 2023-08-25T09:42:30.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e128d2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e0c15ae50 0x7f4e0c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:30.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e128d2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c0cc5f0 con 0x7f4e0c069f60 2023-08-25T09:42:30.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e128d2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c070650 con 0x7f4e0c0cdc20 2023-08-25T09:42:30.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e128d2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c070930 con 0x7f4e0c15ae50 2023-08-25T09:42:30.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e0bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e0c15ae50 0x7f4e0c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:30.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e0bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4e0c15ae50 0x7f4e0c0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58740/0 (socket says 172.21.15.70:58740) 2023-08-25T09:42:30.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.489+0000 7f4e0bfff640 1 -- 172.21.15.70:0/768672476 learned_addr learned my addr 172.21.15.70:0/768672476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:30.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.490+0000 7f4e0bfff640 1 -- 172.21.15.70:0/768672476 >> v1:172.21.15.155:6789/0 conn(0x7f4e0c0cdc20 legacy=0x7f4e0c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:30.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.490+0000 7f4e0bfff640 1 -- 172.21.15.70:0/768672476 >> v1:172.21.15.70:6789/0 conn(0x7f4e0c069f60 legacy=0x7f4e0c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.490+0000 7f4e0bfff640 1 -- 172.21.15.70:0/768672476 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c070c10 con 0x7f4e0c15ae50 2023-08-25T09:42:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.491+0000 7f4e0bfff640 1 --2- 172.21.15.70:0/768672476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c0c2110 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009870 tx=0x7f4dfc02ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc2ac632c7fe337 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.491+0000 7f4e0b7fe640 1 -- 172.21.15.70:0/768672476 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc02f850 con 0x7f4e0c15ae50 2023-08-25T09:42:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.491+0000 7f4e0b7fe640 1 -- 172.21.15.70:0/768672476 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc02f9b0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.491+0000 7f4e0b7fe640 1 -- 172.21.15.70:0/768672476 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc02fd40 con 0x7f4e0c15ae50 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 -- 172.21.15.70:0/768672476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 msgr2=0x7f4e0c0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 --2- 172.21.15.70:0/768672476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c0c2110 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009870 tx=0x7f4dfc02ba30 comp rx=0 tx=0).stop 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 -- 172.21.15.70:0/768672476 shutdown_connections 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 --2- 172.21.15.70:0/768672476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c0c2110 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 -- 172.21.15.70:0/768672476 >> 172.21.15.70:0/768672476 conn(0x7f4e0c0cb0d0 msgr2=0x7f4e0c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 -- 172.21.15.70:0/768672476 shutdown_connections 2023-08-25T09:42:30.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.492+0000 7f4e128d2640 1 -- 172.21.15.70:0/768672476 wait complete. 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.495+0000 7f4e128d2640 1 Processor -- start 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.495+0000 7f4e128d2640 1 -- start start 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e128d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e0c069f60 0x7f4e0c156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e128d2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c0cdc20 0x7f4e0c15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e128d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c1f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e128d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c1f3d30 con 0x7f4e0c15ae50 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e128d2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c15b8c0 con 0x7f4e0c0cdc20 2023-08-25T09:42:30.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e128d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c1f3770 con 0x7f4e0c069f60 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e10e6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c1f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e10e6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c1f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58756/0 (socket says 172.21.15.70:58756) 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.496+0000 7f4e10e6f640 1 -- 172.21.15.70:0/680083437 learned_addr learned my addr 172.21.15.70:0/680083437 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e10e6f640 1 -- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e0c069f60 msgr2=0x7f4e0c156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e03fff640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c0cdc20 0x7f4e0c15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e10e6f640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e0c069f60 0x7f4e0c156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e10e6f640 1 -- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c0cdc20 msgr2=0x7f4e0c15bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e10e6f640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c0cdc20 0x7f4e0c15bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e10e6f640 1 -- 172.21.15.70:0/680083437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0094d0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e03fff640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c0cdc20 0x7f4e0c15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e10e6f640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c1f2cd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400c730 tx=0x7f4e0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04003b70 con 0x7f4e0c15ae50 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e04003cd0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.497+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c1f3a50 con 0x7f4e0c15ae50 2023-08-25T09:42:30.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.498+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400aa90 con 0x7f4e0c15ae50 2023-08-25T09:42:30.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.498+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c0c86d0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.499+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c066fb0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.500+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4e0400abf0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.501+0000 7f4e097fa640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4dec06a080 0x7f4dec06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.501+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e040883e0 con 0x7f4e0c15ae50 2023-08-25T09:42:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.501+0000 7f4e0bfff640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4dec06a080 0x7f4dec06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.502+0000 7f4e0bfff640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4dec06a080 0x7f4dec06c540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc036000 tx=0x7f4dfc039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:30.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.505+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e04053a30 con 0x7f4e0c15ae50 2023-08-25T09:42:30.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.762+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4e0c15ba00 con 0x7f4e0c15ae50 2023-08-25T09:42:30.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.769+0000 7f4e097fa640 1 -- 172.21.15.70:0/680083437 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4e04057050 con 0x7f4e0c15ae50 2023-08-25T09:42:30.772 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:42:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:42:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:42:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:42:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:42:30.781 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:42:30.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4dec06a080 msgr2=0x7f4dec06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4dec06a080 0x7f4dec06c540 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc036000 tx=0x7f4dfc039040 comp rx=0 tx=0).stop 2023-08-25T09:42:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 msgr2=0x7f4e0c1f2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c1f2cd0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f4e0400c730 tx=0x7f4e0400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 shutdown_connections 2023-08-25T09:42:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4dec06a080 0x7f4dec06c540 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e0c15ae50 0x7f4e0c1f2cd0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e0c0cdc20 0x7f4e0c15bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 --2- 172.21.15.70:0/680083437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e0c069f60 0x7f4e0c156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.772+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 >> 172.21.15.70:0/680083437 conn(0x7f4e0c0cb0d0 msgr2=0x7f4e0c0c8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.773+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 shutdown_connections 2023-08-25T09:42:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:30.773+0000 7f4e128d2640 1 -- 172.21.15.70:0/680083437 wait complete. 2023-08-25T09:42:30.794 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:31.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.006+0000 7f776704b640 1 Processor -- start 2023-08-25T09:42:31.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.006+0000 7f776704b640 1 -- start start 2023-08-25T09:42:31.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.007+0000 7f776704b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 0x7f77600cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.007+0000 7f776704b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f77600c5120 0x7f77600cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f776704b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7760160680 con 0x7f77600acd50 2023-08-25T09:42:31.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f776704b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7760160960 con 0x7f77600c5120 2023-08-25T09:42:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f776704b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7760160c40 con 0x7f776015ca00 2023-08-25T09:42:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f776684a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f77600acd50 0x7f776006ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45358/0 (socket says 172.21.15.70:45358) 2023-08-25T09:42:31.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f776684a640 1 -- 172.21.15.70:0/3381934732 learned_addr learned my addr 172.21.15.70:0/3381934732 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f7766049640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 0x7f77600cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f7765848640 1 --2- 172.21.15.70:0/3381934732 >> v2:172.21.15.155:3300/0 conn(0x7f77600c5120 0x7f77600cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685979587 0 0) 0x7f7760160680 con 0x7f77600acd50 2023-08-25T09:42:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.008+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f774c003520 con 0x7f77600acd50 2023-08-25T09:42:31.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928006495 0 0) 0x7f774c003520 con 0x7f77600acd50 2023-08-25T09:42:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 msgr2=0x7f77600cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 0x7f77600cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 >> v2:172.21.15.155:3300/0 conn(0x7f77600c5120 msgr2=0x7f77600cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 --2- 172.21.15.70:0/3381934732 >> v2:172.21.15.155:3300/0 conn(0x7f77600c5120 0x7f77600cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7760160f20 con 0x7f77600acd50 2023-08-25T09:42:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f77580019f0 con 0x7f77600acd50 2023-08-25T09:42:31.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 >> v1:172.21.15.70:6789/0 conn(0x7f77600acd50 legacy=0x7f776006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 0x7f774c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f774c003ef0 0x7f774c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f774c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760160960 con 0x7f774c004d00 2023-08-25T09:42:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760160c40 con 0x7f774c003ef0 2023-08-25T09:42:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774c00b980 con 0x7f776015ca00 2023-08-25T09:42:31.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.009+0000 7f776684a640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f774c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.010+0000 7f7766049640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 0x7f774c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.010+0000 7f776684a640 1 -- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 msgr2=0x7f774c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.010+0000 7f776684a640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ca00 0x7f774c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.010+0000 7f776684a640 1 -- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f774c003ef0 msgr2=0x7f774c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.010+0000 7f776684a640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f774c003ef0 0x7f774c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.010+0000 7f776684a640 1 -- 172.21.15.70:0/3381934732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c003520 con 0x7f774c004d00 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.011+0000 7f776684a640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f774c005180 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f77600ad160 tx=0x7f774c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec70e9d5a04f9ce1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.011+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758034630 con 0x7f774c004d00 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.011+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775802f070 con 0x7f774c004d00 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.011+0000 7f7765047640 1 -- 172.21.15.70:0/3381934732 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758034a30 con 0x7f774c004d00 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776704b640 1 -- 172.21.15.70:0/3381934732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 msgr2=0x7f774c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776704b640 1 --2- 172.21.15.70:0/3381934732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f774c005180 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f77600ad160 tx=0x7f774c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776684a640 1 -- 172.21.15.70:0/3381934732 reap_dead start 2023-08-25T09:42:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776704b640 1 -- 172.21.15.70:0/3381934732 shutdown_connections 2023-08-25T09:42:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776704b640 1 -- 172.21.15.70:0/3381934732 >> 172.21.15.70:0/3381934732 conn(0x7f77600affd0 msgr2=0x7f77600b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776704b640 1 -- 172.21.15.70:0/3381934732 shutdown_connections 2023-08-25T09:42:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.012+0000 7f776704b640 1 -- 172.21.15.70:0/3381934732 wait complete. 2023-08-25T09:42:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.014+0000 7f776704b640 1 Processor -- start 2023-08-25T09:42:31.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.014+0000 7f776704b640 1 -- start start 2023-08-25T09:42:31.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f776704b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f776015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f776704b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f776015d0e0 0x7f77601590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f776704b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776006f0e0 0x7f776006f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f776704b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776015e5e0 con 0x7f774c004d00 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f776704b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776015e8c0 con 0x7f776015d0e0 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f776704b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776015eba0 con 0x7f776006f0e0 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f776015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f776015dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58774/0 (socket says 172.21.15.70:58774) 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 -- 172.21.15.70:0/3325630514 learned_addr learned my addr 172.21.15.70:0/3325630514 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 -- 172.21.15.70:0/3325630514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776006f0e0 msgr2=0x7f776006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776006f0e0 0x7f776006f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 -- 172.21.15.70:0/3325630514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f776015d0e0 msgr2=0x7f77601590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f776015d0e0 0x7f77601590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.015+0000 7f7766049640 1 -- 172.21.15.70:0/3325630514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776015ee80 con 0x7f774c004d00 2023-08-25T09:42:31.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.016+0000 7f7766049640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f776015dfd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f775c010860 tx=0x7f775c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.016+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c009610 con 0x7f774c004d00 2023-08-25T09:42:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.016+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c00eae0 con 0x7f774c004d00 2023-08-25T09:42:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.016+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7760159610 con 0x7f774c004d00 2023-08-25T09:42:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.016+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c00ee70 con 0x7f774c004d00 2023-08-25T09:42:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.016+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776015fdd0 con 0x7f774c004d00 2023-08-25T09:42:31.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.017+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7760066fb0 con 0x7f774c004d00 2023-08-25T09:42:31.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.018+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f775c01c400 con 0x7f774c004d00 2023-08-25T09:42:31.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.019+0000 7f7756ffd640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774406a050 0x7f774406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.019+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f775c088c00 con 0x7f774c004d00 2023-08-25T09:42:31.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.019+0000 7f7765848640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774406a050 0x7f774406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.020+0000 7f7765848640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774406a050 0x7f774406c510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7750006a90 tx=0x7f7750006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.023+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f775c054270 con 0x7f774c004d00 2023-08-25T09:42:31.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.248+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f776015f010 con 0x7f774c004d00 2023-08-25T09:42:31.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.249+0000 7f7756ffd640 1 -- 172.21.15.70:0/3325630514 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f775c057890 con 0x7f774c004d00 2023-08-25T09:42:31.251 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:31.251 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:42:31.252 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:42:31.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774406a050 msgr2=0x7f774406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774406a050 0x7f774406c510 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7750006a90 tx=0x7f7750006a20 comp rx=0 tx=0).stop 2023-08-25T09:42:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 msgr2=0x7f776015dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f776015dfd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f775c010860 tx=0x7f775c010d20 comp rx=0 tx=0).stop 2023-08-25T09:42:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 shutdown_connections 2023-08-25T09:42:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774406a050 0x7f774406c510 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776006f0e0 0x7f776006f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f776015d0e0 0x7f77601590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 --2- 172.21.15.70:0/3325630514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f774c004d00 0x7f776015dfd0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.252+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 >> 172.21.15.70:0/3325630514 conn(0x7f77600affd0 msgr2=0x7f77600ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.253+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 shutdown_connections 2023-08-25T09:42:31.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.253+0000 7f776704b640 1 -- 172.21.15.70:0/3325630514 wait complete. 2023-08-25T09:42:31.680 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:42:24.770908+0000', 'modified': '2023-08-25T09:42:28.336751+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6324}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6324': {'gid': 6324, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6837/1838015952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 1838015952}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 1838015952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:42:31.681 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:42:31.682 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-25T09:42:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.905+0000 7f681ffff640 1 Processor -- start 2023-08-25T09:42:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.906+0000 7f681ffff640 1 -- start start 2023-08-25T09:42:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.907+0000 7f681ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f68200cd130 0x7f68200cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.907+0000 7f681ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f68200cfc00 0x7f68200d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.907+0000 7f681ffff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68200d0050 con 0x7f682015cce0 2023-08-25T09:42:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.907+0000 7f681ffff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68200b8160 con 0x7f68200cfc00 2023-08-25T09:42:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.907+0000 7f681ffff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682015d0c0 con 0x7f68200cd130 2023-08-25T09:42:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f68200cfc00 0x7f68200d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f68200cfc00 0x7f68200d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58776/0 (socket says 172.21.15.70:58776) 2023-08-25T09:42:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 -- 172.21.15.70:0/3729226981 learned_addr learned my addr 172.21.15.70:0/3729226981 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681e7fc640 1 --2- 172.21.15.70:0/3729226981 >> v2:172.21.15.155:3300/0 conn(0x7f68200cd130 0x7f68200cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 -- 172.21.15.70:0/3729226981 >> v2:172.21.15.155:3300/0 conn(0x7f68200cd130 msgr2=0x7f68200cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 --2- 172.21.15.70:0/3729226981 >> v2:172.21.15.155:3300/0 conn(0x7f68200cd130 0x7f68200cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 -- 172.21.15.70:0/3729226981 >> v1:172.21.15.155:6789/0 conn(0x7f682015cce0 legacy=0x7f68200cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.908+0000 7f681f7fe640 1 -- 172.21.15.70:0/3729226981 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68200ce110 con 0x7f68200cfc00 2023-08-25T09:42:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.909+0000 7f681f7fe640 1 --2- 172.21.15.70:0/3729226981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cfc00 0x7f68200d0f50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f68180098a0 tx=0x7f681802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfb38fb24c89026 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.909+0000 7f681dffb640 1 -- 172.21.15.70:0/3729226981 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818031740 con 0x7f68200cfc00 2023-08-25T09:42:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.909+0000 7f681dffb640 1 -- 172.21.15.70:0/3729226981 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68180318a0 con 0x7f68200cfc00 2023-08-25T09:42:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.909+0000 7f681dffb640 1 -- 172.21.15.70:0/3729226981 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818031c30 con 0x7f68200cfc00 2023-08-25T09:42:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 -- 172.21.15.70:0/3729226981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cfc00 msgr2=0x7f68200d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 --2- 172.21.15.70:0/3729226981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cfc00 0x7f68200d0f50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f68180098a0 tx=0x7f681802b7e0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 -- 172.21.15.70:0/3729226981 shutdown_connections 2023-08-25T09:42:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 --2- 172.21.15.70:0/3729226981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cfc00 0x7f68200d0f50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 --2- 172.21.15.70:0/3729226981 >> v2:172.21.15.155:3300/0 conn(0x7f68200cd130 0x7f68200cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 -- 172.21.15.70:0/3729226981 >> 172.21.15.70:0/3729226981 conn(0x7f6820070050 msgr2=0x7f6820070460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 -- 172.21.15.70:0/3729226981 shutdown_connections 2023-08-25T09:42:31.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.910+0000 7f681ffff640 1 -- 172.21.15.70:0/3729226981 wait complete. 2023-08-25T09:42:31.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.912+0000 7f681ffff640 1 Processor -- start 2023-08-25T09:42:31.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.912+0000 7f681ffff640 1 -- start start 2023-08-25T09:42:31.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 0x7f682014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68200cfc00 0x7f68200ca240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f682015cce0 0x7f68201f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68200c9e20 con 0x7f68200cd130 2023-08-25T09:42:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681ffff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68201f2d90 con 0x7f682015cce0 2023-08-25T09:42:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68201f3070 con 0x7f68200cfc00 2023-08-25T09:42:31.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 0x7f682014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 0x7f682014fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58780/0 (socket says 172.21.15.70:58780) 2023-08-25T09:42:31.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.913+0000 7f681effd640 1 -- 172.21.15.70:0/3921344400 learned_addr learned my addr 172.21.15.70:0/3921344400 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681e7fc640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68200cfc00 0x7f68200ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681f7fe640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f682015cce0 0x7f68201f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681effd640 1 -- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68200cfc00 msgr2=0x7f68200ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681effd640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68200cfc00 0x7f68200ca240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681effd640 1 -- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f682015cce0 msgr2=0x7f68201f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681effd640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f682015cce0 0x7f68201f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681effd640 1 -- 172.21.15.70:0/3921344400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180094d0 con 0x7f68200cd130 2023-08-25T09:42:31.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681effd640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 0x7f682014fbf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f681000c7a0 tx=0x7f681000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f6817fff640 1 -- 172.21.15.70:0/3921344400 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810003d80 con 0x7f68200cd130 2023-08-25T09:42:31.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f6817fff640 1 -- 172.21.15.70:0/3921344400 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810005070 con 0x7f68200cd130 2023-08-25T09:42:31.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68201f3350 con 0x7f68200cd130 2023-08-25T09:42:31.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f6817fff640 1 -- 172.21.15.70:0/3921344400 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810005460 con 0x7f68200cd130 2023-08-25T09:42:31.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.914+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6820150130 con 0x7f68200cd130 2023-08-25T09:42:31.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.917+0000 7f6817fff640 1 -- 172.21.15.70:0/3921344400 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f68100055c0 con 0x7f68200cd130 2023-08-25T09:42:31.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.918+0000 7f6817fff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f680006a050 0x7f680006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.918+0000 7f6817fff640 1 -- 172.21.15.70:0/3921344400 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6810087ea0 con 0x7f68200cd130 2023-08-25T09:42:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.919+0000 7f681e7fc640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f680006a050 0x7f680006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.919+0000 7f68285c8740 1 Processor -- start 2023-08-25T09:42:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.919+0000 7f68285c8740 1 -- start start 2023-08-25T09:42:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.919+0000 7f681e7fc640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f680006a050 0x7f680006c510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6808006fd0 tx=0x7f6808008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65c2c0 0x560c9a6895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f68285c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a689af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f68285c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a5cf190 con 0x560c9a65fec0 2023-08-25T09:42:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a5cc880 con 0x560c9a68c340 2023-08-25T09:42:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9a5c8170 con 0x560c9a65c2c0 2023-08-25T09:42:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f681f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a689af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f681f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a689af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58786/0 (socket says 172.21.15.70:58786) 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f681f7fe640 1 -- 172.21.15.70:0/1356592878 learned_addr learned my addr 172.21.15.70:0/1356592878 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f681effd640 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.920+0000 7f681e7fc640 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65c2c0 0x560c9a6895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681f7fe640 1 -- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65c2c0 msgr2=0x560c9a6895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681f7fe640 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65c2c0 0x560c9a6895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681f7fe640 1 -- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 msgr2=0x560c9a68c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681f7fe640 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681f7fe640 1 -- 172.21.15.70:0/1356592878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9a5c67f0 con 0x560c9a65fec0 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681e7fc640 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65c2c0 0x560c9a6895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f681f7fe640 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a689af0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f681802b820 tx=0x7f6818030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7855f07ac3a84b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f68157fa640 1 -- 172.21.15.70:0/1356592878 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818031a10 con 0x560c9a65fec0 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f68157fa640 1 -- 172.21.15.70:0/1356592878 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6818031b70 con 0x560c9a65fec0 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.921+0000 7f68157fa640 1 -- 172.21.15.70:0/1356592878 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6818031f00 con 0x560c9a65fec0 2023-08-25T09:42:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 -- 172.21.15.70:0/1356592878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 msgr2=0x560c9a689af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a689af0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f681802b820 tx=0x7f6818030000 comp rx=0 tx=0).stop 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 -- 172.21.15.70:0/1356592878 shutdown_connections 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a689af0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65c2c0 0x560c9a6895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 --2- 172.21.15.70:0/1356592878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 -- 172.21.15.70:0/1356592878 >> 172.21.15.70:0/1356592878 conn(0x560c9a65e5d0 msgr2=0x560c9a6a7150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 -- 172.21.15.70:0/1356592878 shutdown_connections 2023-08-25T09:42:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.922+0000 7f68285c8740 1 -- 172.21.15.70:0/1356592878 wait complete. 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.923+0000 7f68285c8740 1 -- start start 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.923+0000 7f68285c8740 1 Processor -- start 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a67c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 0x560c9a693330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a567d20 con 0x560c9a65fec0 2023-08-25T09:42:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a5679c0 con 0x560c9a68c340 2023-08-25T09:42:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9a32df20 con 0x560c9a692eb0 2023-08-25T09:42:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f68285c8740 10 client.0 _ll_register_callbacks cb 0x560c9a57f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:42:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a67c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a67c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58790/0 (socket says 172.21.15.70:58790) 2023-08-25T09:42:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 -- 172.21.15.70:0/290007944 learned_addr learned my addr 172.21.15.70:0/290007944 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 -- 172.21.15.70:0/290007944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 msgr2=0x560c9a693330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 0x560c9a693330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 -- 172.21.15.70:0/290007944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 msgr2=0x560c9a68d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.924+0000 7f681effd640 1 -- 172.21.15.70:0/290007944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68200ce110 con 0x560c9a65fec0 2023-08-25T09:42:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f681effd640 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a67c5e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f681000a430 tx=0x7f681005ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f01b5dff6a630 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f67fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f67fe7fc640 1 -- 172.21.15.70:0/290007944 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681006e3c0 con 0x560c9a65fec0 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f67fe7fc640 1 -- 172.21.15.70:0/290007944 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f681005eac0 con 0x560c9a65fec0 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c9a59c540 con 0x560c9a65fec0 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f67fe7fc640 1 -- 172.21.15.70:0/290007944 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f681005ec80 con 0x560c9a65fec0 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f68285c8740 10 client.6471 fetch_fsmap learned FSMap version 32 2023-08-25T09:42:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x560c9a5bff30 con 0x560c9a65fec0 2023-08-25T09:42:31.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f67fe7fc640 1 -- 172.21.15.70:0/290007944 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c9a59c540 con 0x560c9a65fec0 2023-08-25T09:42:31.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.925+0000 7f67fe7fc640 1 -- 172.21.15.70:0/290007944 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6810053030 con 0x560c9a65fec0 2023-08-25T09:42:31.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f68285c8740 10 client.6471 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:42:31.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f68285c8740 10 client.6471 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:42:31.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f68285c8740 20 client.6471 populate_metadata read hostname 'smithi070' 2023-08-25T09:42:31.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f68285c8740 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6914d0 0x560c9a6918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f68285c8740 4 client.6471 mds_command: new command op to 6324 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:42:31.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 --> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c9a5c1150 con 0x560c9a6914d0 2023-08-25T09:42:31.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f681e7fc640 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6914d0 0x560c9a6918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f681e7fc640 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6914d0 0x560c9a6918d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562d235702231b24 server_cookie=252b81ad81efee14 in_seq=0 out_seq=0 2023-08-25T09:42:31.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.926+0000 7f67fe7fc640 10 client.6471 ms_handle_connect on v2:172.21.15.70:6836/1838015952 2023-08-25T09:42:31.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f67fe7fc640 1 -- 172.21.15.70:0/290007944 <== mds.0 v2:172.21.15.70:6836/1838015952 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c9a5c1150 con 0x560c9a6914d0 2023-08-25T09:42:31.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f67fe7fc640 10 client.6471 handle_command_reply: tid=0 2023-08-25T09:42:31.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 client.6471 shutdown 2023-08-25T09:42:31.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 msgr2=0x560c9a67c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a67c5e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f681000a430 tx=0x7f681005ef50 comp rx=0 tx=0).stop 2023-08-25T09:42:31.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 shutdown_connections 2023-08-25T09:42:31.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 0x560c9a693330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6914d0 0x560c9a6918d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a68d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 --2- 172.21.15.70:0/290007944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a67c5e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f67fe7fc640 0 client.6471 ms_handle_reset on v2:172.21.15.70:6836/1838015952 2023-08-25T09:42:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.927+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 >> 172.21.15.70:0/290007944 conn(0x560c9a65c2c0 msgr2=0x560c9a690070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.928+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 shutdown_connections 2023-08-25T09:42:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.928+0000 7f68285c8740 1 -- 172.21.15.70:0/290007944 wait complete. 2023-08-25T09:42:31.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.928+0000 7f68285c8740 20 client.6471 trim_cache size 0 max 16384 2023-08-25T09:42:31.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.944+0000 7f68285c8740 1 Processor -- start 2023-08-25T09:42:31.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.944+0000 7f68285c8740 1 -- start start 2023-08-25T09:42:31.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f68285c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a6a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a6918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 0x560c9a691df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f68285c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a5c7310 con 0x560c9a65fec0 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a68eec0 con 0x560c9a68c340 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9a65f260 con 0x560c9a692eb0 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f681effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a6a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.945+0000 7f681effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a6a9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58804/0 (socket says 172.21.15.70:58804) 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 -- 172.21.15.70:0/3074353101 learned_addr learned my addr 172.21.15.70:0/3074353101 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 -- 172.21.15.70:0/3074353101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 msgr2=0x560c9a691df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 0x560c9a691df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 -- 172.21.15.70:0/3074353101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 msgr2=0x560c9a6918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a6918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 -- 172.21.15.70:0/3074353101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9a566a10 con 0x560c9a65fec0 2023-08-25T09:42:31.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f681effd640 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a6a9b80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f681005ea20 tx=0x7f6810059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a2e50c044c17e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f6815ffb640 1 -- 172.21.15.70:0/3074353101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810053580 con 0x560c9a65fec0 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f6815ffb640 1 -- 172.21.15.70:0/3074353101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681006e1c0 con 0x560c9a65fec0 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.946+0000 7f6815ffb640 1 -- 172.21.15.70:0/3074353101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681005ece0 con 0x560c9a65fec0 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 -- 172.21.15.70:0/3074353101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 msgr2=0x560c9a6a9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a6a9b80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f681005ea20 tx=0x7f6810059000 comp rx=0 tx=0).stop 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 -- 172.21.15.70:0/3074353101 shutdown_connections 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a692eb0 0x560c9a691df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a68c340 0x560c9a6918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 --2- 172.21.15.70:0/3074353101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a65fec0 0x560c9a6a9b80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 -- 172.21.15.70:0/3074353101 >> 172.21.15.70:0/3074353101 conn(0x560c9a65c2c0 msgr2=0x560c9a65b210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 -- 172.21.15.70:0/3074353101 shutdown_connections 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.947+0000 7f68285c8740 1 -- 172.21.15.70:0/3074353101 wait complete. 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.948+0000 7f68285c8740 1 -- start start 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.948+0000 7f68285c8740 1 Processor -- start 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65fec0 0x560c9a6f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 0x560c9a6a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a692eb0 0x560c9a6a7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a6a8710 con 0x560c9a68c340 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9a6bb510 con 0x560c9a692eb0 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9a6bb7f0 con 0x560c9a65fec0 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f68285c8740 10 client.0 _ll_register_callbacks cb 0x7f6818036070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f681e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 0x560c9a6a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f681e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 0x560c9a6a8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58816/0 (socket says 172.21.15.70:58816) 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f681e7fc640 1 -- 172.21.15.70:0/285974548 learned_addr learned my addr 172.21.15.70:0/285974548 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f681e7fc640 1 -- 172.21.15.70:0/285974548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65fec0 msgr2=0x560c9a6f6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.949+0000 7f681e7fc640 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65fec0 0x560c9a6f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f681e7fc640 1 -- 172.21.15.70:0/285974548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a692eb0 msgr2=0x560c9a6a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f681e7fc640 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a692eb0 0x560c9a6a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f681e7fc640 1 -- 172.21.15.70:0/285974548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c9a5bff30 con 0x560c9a68c340 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f681e7fc640 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 0x560c9a6a8ba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f680800e650 tx=0x7f6808006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a651e4a77bb086b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:31.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f67fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f67fe7fc640 1 -- 172.21.15.70:0/285974548 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68080245b0 con 0x560c9a68c340 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f67fe7fc640 1 -- 172.21.15.70:0/285974548 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f680800e7e0 con 0x560c9a68c340 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6818031b70 con 0x560c9a68c340 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f67fe7fc640 1 -- 172.21.15.70:0/285974548 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f680800e9a0 con 0x560c9a68c340 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f68285c8740 10 client.6477 fetch_fsmap learned FSMap version 32 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.950+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x560c9a58bde0 con 0x560c9a68c340 2023-08-25T09:42:31.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f67fe7fc640 1 -- 172.21.15.70:0/285974548 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6818031b70 con 0x560c9a68c340 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f67fe7fc640 1 -- 172.21.15.70:0/285974548 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f680800eb00 con 0x560c9a68c340 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f68285c8740 10 client.6477 fetch_fsmap finished waiting for FSMap version 32 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f68285c8740 10 client.6477 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f68285c8740 20 client.6477 populate_metadata read hostname 'smithi070' 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f68285c8740 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6b5c80 0x560c9a6b6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f68285c8740 4 client.6477 mds_command: new command op to 6324 tid=0[{"prefix": "status"}] 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 --> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c9a2775e0 con 0x560c9a6b5c80 2023-08-25T09:42:31.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f681effd640 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6b5c80 0x560c9a6b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f681effd640 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6b5c80 0x560c9a6b6060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60a6884327df57c0 server_cookie=fcd5a28ae7f1f2a9 in_seq=0 out_seq=0 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.951+0000 7f67fe7fc640 10 client.6477 ms_handle_connect on v2:172.21.15.70:6836/1838015952 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f67fe7fc640 1 -- 172.21.15.70:0/285974548 <== mds.0 v2:172.21.15.70:6836/1838015952 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560c9a2775e0 con 0x560c9a6b5c80 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f67fe7fc640 10 client.6477 handle_command_reply: tid=0 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 client.6477 shutdown 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 msgr2=0x560c9a6a8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 0x560c9a6a8ba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f680800e650 tx=0x7f6808006ca0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 shutdown_connections 2023-08-25T09:42:31.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:6836/1838015952,v1:172.21.15.70:6837/1838015952] conn(0x560c9a6b5c80 0x560c9a6b6060 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560c9a692eb0 0x560c9a6a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560c9a68c340 0x560c9a6a8ba0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 --2- 172.21.15.70:0/285974548 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560c9a65fec0 0x560c9a6f6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f67fe7fc640 0 client.6477 ms_handle_reset on v2:172.21.15.70:6836/1838015952 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 >> 172.21.15.70:0/285974548 conn(0x560c9a65c2c0 msgr2=0x560c9a686630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 shutdown_connections 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 1 -- 172.21.15.70:0/285974548 wait complete. 2023-08-25T09:42:31.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.952+0000 7f68285c8740 20 client.6477 trim_cache size 0 max 16384 2023-08-25T09:42:31.967 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:42:31.967 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:42:31.967 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:42:31.967 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6324, 2023-08-25T09:42:31.967 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6685440710000004, 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 32, 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 73, 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 73, 2023-08-25T09:42:31.968 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.533644642000001 2023-08-25T09:42:31.969 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:42:31.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f680006a050 msgr2=0x7f680006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f680006a050 0x7f680006c510 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6808006fd0 tx=0x7f6808008040 comp rx=0 tx=0).stop 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 msgr2=0x7f682014fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 0x7f682014fbf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f681000c7a0 tx=0x7f681000cc60 comp rx=0 tx=0).stop 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 shutdown_connections 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f680006a050 0x7f680006c510 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f682015cce0 0x7f68201f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68200cfc00 0x7f68200ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 --2- 172.21.15.70:0/3921344400 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f68200cd130 0x7f682014fbf0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:31.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 >> 172.21.15.70:0/3921344400 conn(0x7f6820070050 msgr2=0x7f68200d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:31.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 shutdown_connections 2023-08-25T09:42:31.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:31.953+0000 7f681ffff640 1 -- 172.21.15.70:0/3921344400 wait complete. 2023-08-25T09:42:31.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:42:32.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.186+0000 7fbef8687640 1 Processor -- start 2023-08-25T09:42:32.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.187+0000 7fbef8687640 1 -- start start 2023-08-25T09:42:32.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef8687640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbef006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef8687640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbef006b2b0 0x7fbef015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef8687640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef00b1b60 con 0x7fbef015c6d0 2023-08-25T09:42:32.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef8687640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef006bd80 con 0x7fbef006b2b0 2023-08-25T09:42:32.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef8687640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef006b7c0 con 0x7fbef015e830 2023-08-25T09:42:32.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef6c24640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbef015c6d0 0x7fbef00b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45366/0 (socket says 172.21.15.70:45366) 2023-08-25T09:42:32.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.188+0000 7fbef6c24640 1 -- 172.21.15.70:0/2143058065 learned_addr learned my addr 172.21.15.70:0/2143058065 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5c22640 1 --2- 172.21.15.70:0/2143058065 >> v2:172.21.15.155:3300/0 conn(0x7fbef006b2b0 0x7fbef015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef6423640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbef006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 41408506 0 0) 0x7fbef00b1b60 con 0x7fbef015c6d0 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbedc003520 con 0x7fbef015c6d0 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3871563402 0 0) 0x7fbedc003520 con 0x7fbef015c6d0 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 msgr2=0x7fbef006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:32.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbef006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 >> v2:172.21.15.155:3300/0 conn(0x7fbef006b2b0 msgr2=0x7fbef015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 --2- 172.21.15.70:0/2143058065 >> v2:172.21.15.155:3300/0 conn(0x7fbef006b2b0 0x7fbef015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef006baa0 con 0x7fbef015c6d0 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fbee80019f0 con 0x7fbef015c6d0 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 >> v1:172.21.15.70:6789/0 conn(0x7fbef015c6d0 legacy=0x7fbef00b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbedc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc003ef0 0x7fbedc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc004d00 0x7fbedc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef006bd80 con 0x7fbedc003ef0 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef006b7c0 con 0x7fbedc004d00 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.189+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc00b980 con 0x7fbef015e830 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef5c22640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc003ef0 0x7fbedc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef5c22640 1 -- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 msgr2=0x7fbedc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef6423640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbedc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef5c22640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbedc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef6c24640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc004d00 0x7fbedc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef5c22640 1 -- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc004d00 msgr2=0x7fbedc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:32.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef5c22640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc004d00 0x7fbedc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef5c22640 1 -- 172.21.15.70:0/2143058065 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc003520 con 0x7fbedc003ef0 2023-08-25T09:42:32.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.190+0000 7fbef6423640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbef015e830 0x7fbedc003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:32.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.191+0000 7fbef5c22640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc003ef0 0x7fbedc0047c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009a90 tx=0x7fbedc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e1496dd9b28ce9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:32.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee002c6b0 con 0x7fbedc003ef0 2023-08-25T09:42:32.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee002c810 con 0x7fbedc003ef0 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef5421640 1 -- 172.21.15.70:0/2143058065 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee002cba0 con 0x7fbedc003ef0 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef8687640 1 -- 172.21.15.70:0/2143058065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc003ef0 msgr2=0x7fbedc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef8687640 1 --2- 172.21.15.70:0/2143058065 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc003ef0 0x7fbedc0047c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009a90 tx=0x7fbedc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef6c24640 1 -- 172.21.15.70:0/2143058065 reap_dead start 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef8687640 1 -- 172.21.15.70:0/2143058065 shutdown_connections 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef8687640 1 -- 172.21.15.70:0/2143058065 >> 172.21.15.70:0/2143058065 conn(0x7fbef0069850 msgr2=0x7fbef0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef8687640 1 -- 172.21.15.70:0/2143058065 shutdown_connections 2023-08-25T09:42:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.192+0000 7fbef8687640 1 -- 172.21.15.70:0/2143058065 wait complete. 2023-08-25T09:42:32.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.194+0000 7fbef8687640 1 Processor -- start 2023-08-25T09:42:32.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.194+0000 7fbef8687640 1 -- start start 2023-08-25T09:42:32.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.195+0000 7fbef8687640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc003ef0 0x7fbef01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.195+0000 7fbef8687640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef01f1e40 0x7fbef00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.195+0000 7fbef8687640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 0x7fbef00b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.195+0000 7fbef8687640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01e45f0 con 0x7fbef00cecf0 2023-08-25T09:42:32.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.195+0000 7fbef8687640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01e48d0 con 0x7fbef01f1e40 2023-08-25T09:42:32.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.195+0000 7fbef8687640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef01e4bb0 con 0x7fbedc003ef0 2023-08-25T09:42:32.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 0x7fbef00b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 0x7fbef00b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58842/0 (socket says 172.21.15.70:58842) 2023-08-25T09:42:32.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 -- 172.21.15.70:0/1855365642 learned_addr learned my addr 172.21.15.70:0/1855365642 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6423640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc003ef0 0x7fbef01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef5c22640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef01f1e40 0x7fbef00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 -- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc003ef0 msgr2=0x7fbef01548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc003ef0 0x7fbef01548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 -- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef01f1e40 msgr2=0x7fbef00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef01f1e40 0x7fbef00ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 -- 172.21.15.70:0/1855365642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef01e4e90 con 0x7fbef00cecf0 2023-08-25T09:42:32.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6423640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc003ef0 0x7fbef01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef5c22640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef01f1e40 0x7fbef00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.196+0000 7fbef6c24640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 0x7fbef00b1ef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbee8000f80 tx=0x7fbee800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.197+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004d60 con 0x7fbef00cecf0 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.197+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee800bb80 con 0x7fbef00cecf0 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.197+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8007110 con 0x7fbef00cecf0 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.197+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef01e5170 con 0x7fbef00cecf0 2023-08-25T09:42:32.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.197+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef01e42c0 con 0x7fbef00cecf0 2023-08-25T09:42:32.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.198+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef004e420 con 0x7fbef00cecf0 2023-08-25T09:42:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.199+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbee8007270 con 0x7fbef00cecf0 2023-08-25T09:42:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.200+0000 7fbee6ffd640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbed406a080 0x7fbed406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.200+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbee8088950 con 0x7fbef00cecf0 2023-08-25T09:42:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.200+0000 7fbef6423640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbed406a080 0x7fbed406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.201+0000 7fbef6423640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbed406a080 0x7fbed406c540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0099c0 tx=0x7fbeec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:32.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.204+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbee80540d0 con 0x7fbef00cecf0 2023-08-25T09:42:32.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.435+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbef006c620 con 0x7fbef00cecf0 2023-08-25T09:42:32.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.436+0000 7fbee6ffd640 1 -- 172.21.15.70:0/1855365642 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbee80576f0 con 0x7fbef00cecf0 2023-08-25T09:42:32.438 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:42:32.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.438+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbed406a080 msgr2=0x7fbed406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:32.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.438+0000 7fbef8687640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbed406a080 0x7fbed406c540 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0099c0 tx=0x7fbeec008040 comp rx=0 tx=0).stop 2023-08-25T09:42:32.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.438+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 msgr2=0x7fbef00b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:32.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.438+0000 7fbef8687640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 0x7fbef00b1ef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fbee8000f80 tx=0x7fbee800b4a0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 shutdown_connections 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbed406a080 0x7fbed406c540 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbef00cecf0 0x7fbef00b1ef0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbef01f1e40 0x7fbef00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 --2- 172.21.15.70:0/1855365642 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc003ef0 0x7fbef01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 >> 172.21.15.70:0/1855365642 conn(0x7fbef0069850 msgr2=0x7fbef00700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 shutdown_connections 2023-08-25T09:42:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:32.439+0000 7fbef8687640 1 -- 172.21.15.70:0/1855365642 wait complete. 2023-08-25T09:42:32.460 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:42:32.461 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:42:32.461 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:42:32.461 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:42:32.462 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:32.462 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:42:32.462 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:42:32.463 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:42:32.463 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:42:32.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:32.464 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:42:32.516 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:42:32.516 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:42:32.516 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:42:32.516 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:42:32.517 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42124sec preferred_lft 42124sec 2023-08-25T09:42:32.518 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:42:32.519 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:42:32.520 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:32.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:32.522 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:42:32.616 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:42:32.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:32.620 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:42:32.674 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:42:32.674 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:42:32.674 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:42:32.675 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:42:32.675 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:42:32.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:32.676 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:42:32.676 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:42:32.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:42:32.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:42:32.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:42:32.678 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:42:32.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:32.857 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:42:32.906 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:42:32.906 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:42:32.906 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:42:32.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:32.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:42:32.994 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:42:32.994 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:42:32.995 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:33.049 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:42:33.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:42:33.051 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:33.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:42:33.195 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:42:33.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:42:33.289 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:42:33.293 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:42:33.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:33.294 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:42:33.353 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:42:33.353 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:42:33.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:42:33.436 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:42:33.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:42:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:42:33.517 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:42:33.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:33.519 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:42:33.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:42:33.547+0000 7f2ba3645480 -1 init, newargv = 0x562dda396010 newargc=15 2023-08-25T09:42:33.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[103229]: starting ceph client 2023-08-25T09:42:33.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[103229]: starting fuse 2023-08-25T09:42:33.562 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:42:33.562 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:42:33.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:42:33.563 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:42:33.564 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:42:33.564 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:42:33.564 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:42:33.568 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:33.569 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:42:33.570 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:42:33.570 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:42:33.571 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:42:33.571 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:33.571 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:42:33.572 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:42:33.572 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:42:33.572 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:42:33.573 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:33.573 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:42:33.574 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:42:33.574 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:42:33.574 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:42:33.575 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:42:33.576 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:42:33.576 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:42:33.577 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:42:33.577 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:33.577 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:42:33.578 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:42:33.767 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.103229.asok 2023-08-25T09:42:33.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:42:33.785 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103229.asok 2023-08-25T09:42:33.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:33.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103229.asok status 2023-08-25T09:42:34.018 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:42:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:42:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:42:34.055 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "103229", 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:34.056 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6492, 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 6492 2023-08-25T09:42:34.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3148041320 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:42:34.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:42:34.059 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3148041320 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.6492 192.168.0.1:0/3148041320", 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3148041320", 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 32, 2023-08-25T09:42:34.060 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 73, 2023-08-25T09:42:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:42:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:42:34.061 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:42:34.061 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:42:34.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:34.062 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:42:34.063 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:34.124 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:42:34.125 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:34.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:34.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:42:34.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:42:34.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.440+0000 7fd6907c9640 1 Processor -- start 2023-08-25T09:42:34.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.440+0000 7fd6907c9640 1 -- start start 2023-08-25T09:42:34.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd6907c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd68815e830 0x7fd68806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:34.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd6907c9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd68806b2b0 0x7fd68815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:34.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd6907c9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6880b1b60 con 0x7fd68815c6d0 2023-08-25T09:42:34.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd6907c9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68806bd80 con 0x7fd68815e830 2023-08-25T09:42:34.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd6907c9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd68806b7c0 con 0x7fd68806b2b0 2023-08-25T09:42:34.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68e565640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd68815e830 0x7fd68806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:34.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68ed66640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd68815c6d0 0x7fd6880b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45376/0 (socket says 172.21.15.70:45376) 2023-08-25T09:42:34.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68e565640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd68815e830 0x7fd68806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58844/0 (socket says 172.21.15.70:58844) 2023-08-25T09:42:34.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68ed66640 1 -- 172.21.15.70:0/495712852 learned_addr learned my addr 172.21.15.70:0/495712852 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68e565640 1 -- 172.21.15.70:0/495712852 >> v2:172.21.15.155:3300/0 conn(0x7fd68806b2b0 msgr2=0x7fd68815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:34.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68dd64640 1 --2- 172.21.15.70:0/495712852 >> v2:172.21.15.155:3300/0 conn(0x7fd68806b2b0 0x7fd68815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68e565640 1 --2- 172.21.15.70:0/495712852 >> v2:172.21.15.155:3300/0 conn(0x7fd68806b2b0 0x7fd68815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.442+0000 7fd68e565640 1 -- 172.21.15.70:0/495712852 >> v1:172.21.15.70:6789/0 conn(0x7fd68815c6d0 legacy=0x7fd6880b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.443+0000 7fd68d563640 1 -- 172.21.15.70:0/495712852 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613077116 0 0) 0x7fd6880b1b60 con 0x7fd68815c6d0 2023-08-25T09:42:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.443+0000 7fd68e565640 1 -- 172.21.15.70:0/495712852 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68806baa0 con 0x7fd68815e830 2023-08-25T09:42:34.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.444+0000 7fd68e565640 1 --2- 172.21.15.70:0/495712852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd68806ad70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd6880cf6f0 tx=0x7fd68402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e289bf47fae730 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:34.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.444+0000 7fd68d563640 1 -- 172.21.15.70:0/495712852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684031830 con 0x7fd68815e830 2023-08-25T09:42:34.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.444+0000 7fd68d563640 1 -- 172.21.15.70:0/495712852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd684031990 con 0x7fd68815e830 2023-08-25T09:42:34.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.444+0000 7fd68d563640 1 -- 172.21.15.70:0/495712852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684031d20 con 0x7fd68815e830 2023-08-25T09:42:34.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.444+0000 7fd6907c9640 1 -- 172.21.15.70:0/495712852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 msgr2=0x7fd68806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.444+0000 7fd6907c9640 1 --2- 172.21.15.70:0/495712852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd68806ad70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd6880cf6f0 tx=0x7fd68402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.445+0000 7fd6907c9640 1 -- 172.21.15.70:0/495712852 shutdown_connections 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.445+0000 7fd6907c9640 1 --2- 172.21.15.70:0/495712852 >> v2:172.21.15.155:3300/0 conn(0x7fd68806b2b0 0x7fd68815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.445+0000 7fd6907c9640 1 --2- 172.21.15.70:0/495712852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd68806ad70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.445+0000 7fd6907c9640 1 -- 172.21.15.70:0/495712852 >> 172.21.15.70:0/495712852 conn(0x7fd688069850 msgr2=0x7fd688069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.445+0000 7fd6907c9640 1 -- 172.21.15.70:0/495712852 shutdown_connections 2023-08-25T09:42:34.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.445+0000 7fd6907c9640 1 -- 172.21.15.70:0/495712852 wait complete. 2023-08-25T09:42:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.446+0000 7fd6907c9640 1 Processor -- start 2023-08-25T09:42:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.447+0000 7fd6907c9640 1 -- start start 2023-08-25T09:42:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.447+0000 7fd6907c9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68806b2b0 0x7fd6881f1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:34.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.447+0000 7fd6907c9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd68815c6d0 0x7fd6880c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.447+0000 7fd6907c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd6880c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.447+0000 7fd6907c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6880c36f0 con 0x7fd68815e830 2023-08-25T09:42:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd6907c9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6880c3990 con 0x7fd68806b2b0 2023-08-25T09:42:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd6907c9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6880c3c70 con 0x7fd68815c6d0 2023-08-25T09:42:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd6880c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:34.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd6880c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58858/0 (socket says 172.21.15.70:58858) 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 -- 172.21.15.70:0/1065309741 learned_addr learned my addr 172.21.15.70:0/1065309741 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68e565640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68806b2b0 0x7fd6881f1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68dd64640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd68815c6d0 0x7fd6880c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 -- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd68815c6d0 msgr2=0x7fd6880c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd68815c6d0 0x7fd6880c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 -- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68806b2b0 msgr2=0x7fd6881f1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:34.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68806b2b0 0x7fd6881f1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.448+0000 7fd68ed66640 1 -- 172.21.15.70:0/1065309741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd684009400 con 0x7fd68815e830 2023-08-25T09:42:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.449+0000 7fd68ed66640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd6880c4c90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c7a0 tx=0x7fd68000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.449+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68000a640 con 0x7fd68815e830 2023-08-25T09:42:34.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.449+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68000a7a0 con 0x7fd68815e830 2023-08-25T09:42:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.449+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68000abd0 con 0x7fd68815e830 2023-08-25T09:42:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.449+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6880c3f50 con 0x7fd68815e830 2023-08-25T09:42:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.449+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6881f18c0 con 0x7fd68815e830 2023-08-25T09:42:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.450+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6880c4470 con 0x7fd68815e830 2023-08-25T09:42:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.451+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd68000ad30 con 0x7fd68815e830 2023-08-25T09:42:34.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.452+0000 7fd67effd640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66c06a050 0x7fd66c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.452+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd680087b60 con 0x7fd68815e830 2023-08-25T09:42:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.452+0000 7fd68e565640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66c06a050 0x7fd66c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.453+0000 7fd68e565640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66c06a050 0x7fd66c06c510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd684036000 tx=0x7fd684008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:34.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.457+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd6880c4470 con 0x7fd68815e830 2023-08-25T09:42:34.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.683+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd6880c4470 con 0x7fd68815e830 2023-08-25T09:42:34.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.684+0000 7fd67effd640 1 -- 172.21.15.70:0/1065309741 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd6880c4470 con 0x7fd68815e830 2023-08-25T09:42:34.686 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:42:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66c06a050 msgr2=0x7fd66c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66c06a050 0x7fd66c06c510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd684036000 tx=0x7fd684008030 comp rx=0 tx=0).stop 2023-08-25T09:42:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 msgr2=0x7fd6880c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd6880c4c90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd68000c7a0 tx=0x7fd68000cc60 comp rx=0 tx=0).stop 2023-08-25T09:42:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 shutdown_connections 2023-08-25T09:42:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66c06a050 0x7fd66c06c510 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd68815e830 0x7fd6880c4c90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd68815c6d0 0x7fd6880c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 --2- 172.21.15.70:0/1065309741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd68806b2b0 0x7fd6881f1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.686+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 >> 172.21.15.70:0/1065309741 conn(0x7fd688069850 msgr2=0x7fd68815b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.687+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 shutdown_connections 2023-08-25T09:42:34.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:34.687+0000 7fd6907c9640 1 -- 172.21.15.70:0/1065309741 wait complete. 2023-08-25T09:42:34.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:42:34.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:42:34.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:42:34.708 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:42:34.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:34.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:42:34.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:42:34.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:42:34.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:42:34.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:34.710 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:42:34.771 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:42:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:42:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:42:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.771 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:34.772 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42134sec preferred_lft 42134sec 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:42:34.773 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:42:34.774 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:42:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:42:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.775 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:42:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:42:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:42:34.775 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:42:34.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:34.777 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:42:34.870 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:42:34.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:34.874 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:42:34.931 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:42:34.932 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:42:34.932 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:42:34.932 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:42:34.932 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:42:34.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:34.933 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:42:34.933 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:42:34.934 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:42:34.934 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:42:34.934 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:42:34.934 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:42:35.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:35.123 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:42:35.173 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:42:35.173 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:42:35.174 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:42:35.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:35.175 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:42:35.274 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:42:35.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:42:35.274 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:35.331 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:42:35.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:42:35.332 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:35.385 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:42:35.483 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:42:35.484 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:42:35.587 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:42:35.591 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:42:35.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:35.593 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:42:35.652 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:42:35.653 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:42:35.662 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:42:35.745 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:42:35.746 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:42:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:42:35.824 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:42:35.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:35.825 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:42:35.860 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:42:35.858+0000 7f9f55a45480 -1 init, newargv = 0x564a6d199d20 newargc=15 2023-08-25T09:42:35.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[89302]: starting ceph client 2023-08-25T09:42:35.869 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[89302]: starting fuse 2023-08-25T09:42:35.873 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:42:35.874 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:42:35.875 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:42:35.875 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:42:35.875 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:42:35.876 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:42:35.877 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:42:35.878 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:42:35.878 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:42:35.878 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:42:35.879 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:42:35.879 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:42:35.879 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:42:35.880 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:42:35.880 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:42:35.880 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:42:35.881 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:42:35.881 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:42:35.881 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:42:35.882 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:42:35.882 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:42:35.882 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:42:35.883 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:42:35.883 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:42:35.884 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:42:35.884 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:42:35.884 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:42:35.885 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:42:35.886 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:42:36.083 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.89302.asok 2023-08-25T09:42:36.086 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:42:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:42:36.101 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.89302.asok 2023-08-25T09:42:36.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:36.102 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.89302.asok status 2023-08-25T09:42:36.337 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:42:36.337 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:42:36.337 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:42:36.337 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:42:36.337 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:42:36.337 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "89302", 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:42:36.338 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 4931, 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 4931 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:42:36.339 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:42:36.340 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1260488936 2023-08-25T09:42:36.340 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:42:36.340 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:42:36.340 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:42:36.340 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:42:36.340 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1260488936 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.4931 192.168.0.1:0/1260488936", 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/1260488936", 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 32, 2023-08-25T09:42:36.341 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 73, 2023-08-25T09:42:36.342 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:42:36.342 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:42:36.342 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:42:36.342 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:42:36.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:36.352 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:42:36.353 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:36.408 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:42:36.409 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:36.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:42:36.410 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:42:36.507 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:42:36.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.729+0000 7f702759e640 1 Processor -- start 2023-08-25T09:42:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.729+0000 7f702759e640 1 -- start start 2023-08-25T09:42:36.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.730+0000 7f702759e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f70280b12f0 0x7f702815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.730+0000 7f702759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f702815e7c0 con 0x7f702815f190 2023-08-25T09:42:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.730+0000 7f702759e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7028071b50 con 0x7f702815a7c0 2023-08-25T09:42:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.730+0000 7f702759e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f702815e490 con 0x7f70280b12f0 2023-08-25T09:42:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.731+0000 7f7026d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f702815f190 0x7f70280cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45386/0 (socket says 172.21.15.70:45386) 2023-08-25T09:42:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.731+0000 7f7026d9d640 1 -- 172.21.15.70:0/3715184604 learned_addr learned my addr 172.21.15.70:0/3715184604 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:42:36.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.731+0000 7f701ffff640 1 --2- 172.21.15.70:0/3715184604 >> v2:172.21.15.155:3300/0 conn(0x7f70280b12f0 0x7f702815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.731+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245903436 0 0) 0x7f702815e7c0 con 0x7f702815f190 2023-08-25T09:42:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.731+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7014003520 con 0x7f702815f190 2023-08-25T09:42:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.731+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2387651082 0 0) 0x7f7014003520 con 0x7f702815f190 2023-08-25T09:42:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 >> v2:172.21.15.155:3300/0 conn(0x7f70280b12f0 msgr2=0x7f702815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:36.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 --2- 172.21.15.70:0/3715184604 >> v2:172.21.15.155:3300/0 conn(0x7f70280b12f0 0x7f702815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 >> v1:172.21.15.155:6789/0 conn(0x7f702815a7c0 legacy=0x7f70280b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70280b3d60 con 0x7f702815f190 2023-08-25T09:42:36.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f70200019f0 con 0x7f702815f190 2023-08-25T09:42:36.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 >> v1:172.21.15.70:6789/0 conn(0x7f702815f190 legacy=0x7f70280cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7014003970 0x7f7014003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70140047c0 0x7f7014004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7014005180 0x7f701400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702815e490 con 0x7f7014005180 2023-08-25T09:42:36.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701400c020 con 0x7f7014003970 2023-08-25T09:42:36.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701400c1e0 con 0x7f70140047c0 2023-08-25T09:42:36.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f7026d9d640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7014005180 0x7f701400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f702659c640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7014003970 0x7f7014003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.732+0000 7f701ffff640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70140047c0 0x7f7014004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.733+0000 7f7026d9d640 1 -- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70140047c0 msgr2=0x7f7014004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:36.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.733+0000 7f7026d9d640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70140047c0 0x7f7014004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.733+0000 7f7026d9d640 1 -- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7014003970 msgr2=0x7f7014003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:42:36.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.733+0000 7f7026d9d640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7014003970 0x7f7014003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.733+0000 7f7026d9d640 1 -- 172.21.15.70:0/3715184604 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7014003520 con 0x7f7014005180 2023-08-25T09:42:36.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.733+0000 7f701ffff640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70140047c0 0x7f7014004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:42:36.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.734+0000 7f7026d9d640 1 -- 172.21.15.70:0/3715184604 reap_dead start 2023-08-25T09:42:36.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.734+0000 7f7026d9d640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7014005180 0x7f701400b980 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f702000a7f0 tx=0x7f70140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e0c7d4773c3e91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:36.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.734+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020034630 con 0x7f7014005180 2023-08-25T09:42:36.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.734+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702002f070 con 0x7f7014005180 2023-08-25T09:42:36.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.734+0000 7f7025d9b640 1 -- 172.21.15.70:0/3715184604 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200349f0 con 0x7f7014005180 2023-08-25T09:42:36.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 -- 172.21.15.70:0/3715184604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7014005180 msgr2=0x7f701400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7014005180 0x7f701400b980 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f702000a7f0 tx=0x7f70140057f0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 -- 172.21.15.70:0/3715184604 shutdown_connections 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 --2- 172.21.15.70:0/3715184604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7014005180 0x7f701400b980 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 -- 172.21.15.70:0/3715184604 >> 172.21.15.70:0/3715184604 conn(0x7f70280affd0 msgr2=0x7f70280b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 -- 172.21.15.70:0/3715184604 shutdown_connections 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.735+0000 7f702759e640 1 -- 172.21.15.70:0/3715184604 wait complete. 2023-08-25T09:42:36.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.737+0000 7f702759e640 1 Processor -- start 2023-08-25T09:42:36.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.737+0000 7f702759e640 1 -- start start 2023-08-25T09:42:36.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f702759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7014005180 0x7f7028149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f702759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 0x7f70280c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f702759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70280cd750 0x7f7028158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f702759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70280cd2c0 con 0x7f70280cc1d0 2023-08-25T09:42:36.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f702759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702814a3c0 con 0x7f70280cd750 2023-08-25T09:42:36.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f702759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7028149eb0 con 0x7f7014005180 2023-08-25T09:42:36.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 0x7f70280c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 0x7f70280c73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58872/0 (socket says 172.21.15.70:58872) 2023-08-25T09:42:36.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 -- 172.21.15.70:0/3023202546 learned_addr learned my addr 172.21.15.70:0/3023202546 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:42:36.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 -- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7014005180 msgr2=0x7f7028149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7014005180 0x7f7028149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 -- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70280cd750 msgr2=0x7f7028158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f7026d9d640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70280cd750 0x7f7028158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70280cd750 0x7f7028158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.738+0000 7f701ffff640 1 -- 172.21.15.70:0/3023202546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70280cc6b0 con 0x7f70280cc1d0 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.739+0000 7f701ffff640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 0x7f70280c73e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7010009c00 tx=0x7f701000b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:36.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.739+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010017070 con 0x7f70280cc1d0 2023-08-25T09:42:36.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.739+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701000be30 con 0x7f70280cc1d0 2023-08-25T09:42:36.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.739+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010012270 con 0x7f70280cc1d0 2023-08-25T09:42:36.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.739+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70280cc990 con 0x7f70280cc1d0 2023-08-25T09:42:36.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.739+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70280ccf50 con 0x7f70280cc1d0 2023-08-25T09:42:36.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.740+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70280cd110 con 0x7f70280cc1d0 2023-08-25T09:42:36.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.741+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7010010560 con 0x7f70280cc1d0 2023-08-25T09:42:36.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.742+0000 7f701effd640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f700806a080 0x7f700806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:42:36.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.742+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f701008c450 con 0x7f70280cc1d0 2023-08-25T09:42:36.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.742+0000 7f702659c640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f700806a080 0x7f700806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:42:36.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.743+0000 7f702659c640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f700806a080 0x7f700806c540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7018007fd0 tx=0x7f7018009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:42:36.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.747+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f701005b530 con 0x7f70280cc1d0 2023-08-25T09:42:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.974+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7028066fb0 con 0x7f70280cc1d0 2023-08-25T09:42:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.975+0000 7f701effd640 1 -- 172.21.15.70:0/3023202546 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f70100123d0 con 0x7f70280cc1d0 2023-08-25T09:42:36.977 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:42:36.978 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:42:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:42:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f700806a080 msgr2=0x7f700806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f700806a080 0x7f700806c540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f7018007fd0 tx=0x7f7018009040 comp rx=0 tx=0).stop 2023-08-25T09:42:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 msgr2=0x7f70280c73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:42:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 0x7f70280c73e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7010009c00 tx=0x7f701000b510 comp rx=0 tx=0).stop 2023-08-25T09:42:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 shutdown_connections 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f700806a080 0x7f700806c540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70280cd750 0x7f7028158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70280cc1d0 0x7f70280c73e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 --2- 172.21.15.70:0/3023202546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7014005180 0x7f7028149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 >> 172.21.15.70:0/3023202546 conn(0x7f70280affd0 msgr2=0x7f7028159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 shutdown_connections 2023-08-25T09:42:36.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:42:36.978+0000 7f702759e640 1 -- 172.21.15.70:0/3023202546 wait complete. 2023-08-25T09:42:36.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-08-25T09:42:37.232 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":6324,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.931485143,"mdsmap_epoch":32,"osdmap_epoch":73,"osdmap_epoch_barrier":73,"uptime":18.796599766} 2023-08-25T09:42:37.233 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 6324, "mdsmap_epoch": 32, "osdmap_epoch": 73, "osdmap_epoch_barrier": 73, "rank_uptime": 11.931485143, "state": "up:active", "uptime": 18.796599766, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:42:37.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:42:37.456 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:42:37.456 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:42:37.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-08-25T09:42:37.667 INFO:teuthology.orchestra.run.smithi070.stdout:{"mon_tick_interval":"5"} 2023-08-25T09:42:37.667 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-08-25T09:42:37.668 INFO:tasks.cephfs.test_full:pool capacity 556987968, 531MB should be enough to fill it 2023-08-25T09:42:37.672 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:42:37.676 DEBUG:teuthology.orchestra.run.smithi070:> import time 2023-08-25T09:42:37.677 DEBUG:teuthology.orchestra.run.smithi070:> import datetime 2023-08-25T09:42:37.677 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:42:37.678 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:42:37.679 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.679 DEBUG:teuthology.orchestra.run.smithi070:> # Write some buffered data through before going full, all should be well 2023-08-25T09:42:37.679 DEBUG:teuthology.orchestra.run.smithi070:> print("writing some data through which we expect to succeed") 2023-08-25T09:42:37.680 DEBUG:teuthology.orchestra.run.smithi070:> bytes = 0 2023-08-25T09:42:37.681 DEBUG:teuthology.orchestra.run.smithi070:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-08-25T09:42:37.681 DEBUG:teuthology.orchestra.run.smithi070:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-08-25T09:42:37.682 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-08-25T09:42:37.682 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:42:37.683 DEBUG:teuthology.orchestra.run.smithi070:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-08-25T09:42:37.683 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.683 DEBUG:teuthology.orchestra.run.smithi070:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-08-25T09:42:37.684 DEBUG:teuthology.orchestra.run.smithi070:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-08-25T09:42:37.684 DEBUG:teuthology.orchestra.run.smithi070:> # buffered IO, the error should always only appear from fsync and not 2023-08-25T09:42:37.685 DEBUG:teuthology.orchestra.run.smithi070:> # from write 2023-08-25T09:42:37.686 DEBUG:teuthology.orchestra.run.smithi070:> full = False 2023-08-25T09:42:37.686 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.687 DEBUG:teuthology.orchestra.run.smithi070:> for n in range(0, int(531 * 0.9)): 2023-08-25T09:42:37.688 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-08-25T09:42:37.688 DEBUG:teuthology.orchestra.run.smithi070:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-08-25T09:42:37.689 DEBUG:teuthology.orchestra.run.smithi070:> print("done writing {0} bytes".format(bytes)) 2023-08-25T09:42:37.689 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.691 DEBUG:teuthology.orchestra.run.smithi070:> # OK, now we should sneak in under the full condition 2023-08-25T09:42:37.691 DEBUG:teuthology.orchestra.run.smithi070:> # due to the time it takes the OSDs to report to the 2023-08-25T09:42:37.691 DEBUG:teuthology.orchestra.run.smithi070:> # mons, and get a successful fsync on our full-making data 2023-08-25T09:42:37.692 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:42:37.692 DEBUG:teuthology.orchestra.run.smithi070:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-08-25T09:42:37.692 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.693 DEBUG:teuthology.orchestra.run.smithi070:> # buffered write, add more dirty data to the buffer 2023-08-25T09:42:37.693 DEBUG:teuthology.orchestra.run.smithi070:> print("starting buffered write") 2023-08-25T09:42:37.694 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:42:37.694 DEBUG:teuthology.orchestra.run.smithi070:> for n in range(0, int(531 * 0.2)): 2023-08-25T09:42:37.694 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-08-25T09:42:37.694 DEBUG:teuthology.orchestra.run.smithi070:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-08-25T09:42:37.696 DEBUG:teuthology.orchestra.run.smithi070:> time.sleep(15.0) 2023-08-25T09:42:37.696 DEBUG:teuthology.orchestra.run.smithi070:> except OSError: 2023-08-25T09:42:37.697 DEBUG:teuthology.orchestra.run.smithi070:> pass; 2023-08-25T09:42:37.698 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.699 DEBUG:teuthology.orchestra.run.smithi070:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-08-25T09:42:37.700 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.700 DEBUG:teuthology.orchestra.run.smithi070:> # Wait long enough for a background flush that should fail 2023-08-25T09:42:37.701 DEBUG:teuthology.orchestra.run.smithi070:> time.sleep(30) 2023-08-25T09:42:37.702 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.703 DEBUG:teuthology.orchestra.run.smithi070:> if True: 2023-08-25T09:42:37.703 DEBUG:teuthology.orchestra.run.smithi070:> # ...and check that the failed background flush is reflected in fclose 2023-08-25T09:42:37.704 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:42:37.704 DEBUG:teuthology.orchestra.run.smithi070:> os.close(f) 2023-08-25T09:42:37.705 DEBUG:teuthology.orchestra.run.smithi070:> except OSError: 2023-08-25T09:42:37.705 DEBUG:teuthology.orchestra.run.smithi070:> print("close() returned an error as expected") 2023-08-25T09:42:37.706 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:42:37.706 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("close() failed to raise error") 2023-08-25T09:42:37.707 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:42:37.708 DEBUG:teuthology.orchestra.run.smithi070:> # The kernel cephfs client does not raise errors on fclose 2023-08-25T09:42:37.708 DEBUG:teuthology.orchestra.run.smithi070:> os.close(f) 2023-08-25T09:42:37.709 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:42:37.710 DEBUG:teuthology.orchestra.run.smithi070:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-08-25T09:42:37.711 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:42:40.127 INFO:teuthology.orchestra.run.smithi070.stdout:writing some data through which we expect to succeed 2023-08-25T09:42:40.127 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-08-25T09:42:40.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-08-25T09:42:40.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-08-25T09:42:40.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-08-25T09:42:40.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-08-25T09:42:40.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-08-25T09:42:40.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-08-25T09:42:40.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-08-25T09:42:40.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-08-25T09:42:40.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-08-25T09:42:40.137 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-08-25T09:42:40.137 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-08-25T09:42:40.137 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-08-25T09:42:40.137 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-08-25T09:42:40.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-08-25T09:42:40.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-08-25T09:42:40.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-08-25T09:42:40.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-08-25T09:42:40.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-08-25T09:42:40.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-08-25T09:42:40.140 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-08-25T09:42:40.140 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-08-25T09:42:40.140 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-08-25T09:42:40.140 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-08-25T09:42:40.140 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-08-25T09:42:40.140 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-08-25T09:42:40.141 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-08-25T09:42:40.142 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-08-25T09:42:40.143 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-08-25T09:42:40.144 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-08-25T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-08-25T09:42:40.146 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-08-25T09:42:40.147 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-08-25T09:42:40.147 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-08-25T09:42:40.147 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-08-25T09:42:40.147 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-08-25T09:42:40.148 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-08-25T09:42:40.148 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-08-25T09:42:40.148 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-08-25T09:42:40.148 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-08-25T09:42:40.148 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-08-25T09:42:40.148 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-08-25T09:42:40.149 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-08-25T09:42:41.436 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-08-25T09:42:41.437 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-08-25T09:42:41.438 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-08-25T09:42:41.438 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-08-25T09:42:41.438 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-08-25T09:42:41.438 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-08-25T09:42:41.438 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-08-25T09:42:41.438 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-08-25T09:42:41.439 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-08-25T09:42:41.439 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-08-25T09:42:41.439 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-08-25T09:42:41.439 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-08-25T09:42:41.439 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-08-25T09:42:41.440 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-08-25T09:42:41.440 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-08-25T09:42:41.440 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-08-25T09:42:41.440 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-08-25T09:42:41.440 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-08-25T09:42:41.441 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-08-25T09:42:41.441 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-08-25T09:42:41.441 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-08-25T09:42:41.441 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-08-25T09:42:41.441 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-08-25T09:42:41.442 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-08-25T09:42:41.442 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-08-25T09:42:41.442 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-08-25T09:42:41.442 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-08-25T09:42:41.442 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-08-25T09:42:41.442 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-08-25T09:42:41.443 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-08-25T09:42:41.444 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-08-25T09:42:41.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-08-25T09:42:41.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-08-25T09:42:41.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-08-25T09:42:41.448 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-08-25T09:42:41.448 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-08-25T09:42:41.448 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-08-25T09:42:41.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-08-25T09:42:41.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-08-25T09:42:41.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-08-25T09:42:41.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-08-25T09:42:41.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-08-25T09:42:41.450 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-08-25T09:42:41.450 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-08-25T09:42:41.450 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-08-25T09:42:41.450 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-08-25T09:42:41.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-08-25T09:42:41.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-08-25T09:42:41.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-08-25T09:42:41.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-08-25T09:42:41.452 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-08-25T09:42:41.452 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-08-25T09:42:41.452 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-08-25T09:42:41.452 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-08-25T09:42:41.453 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-08-25T09:42:41.453 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-08-25T09:42:41.453 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-08-25T09:42:41.453 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-08-25T09:42:41.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-08-25T09:42:41.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-08-25T09:42:41.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-08-25T09:42:41.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-08-25T09:42:41.455 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-08-25T09:42:41.455 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-08-25T09:42:41.455 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-08-25T09:42:41.455 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-08-25T09:42:41.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-08-25T09:42:41.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-08-25T09:42:41.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-08-25T09:42:41.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-08-25T09:42:41.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-08-25T09:42:41.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-08-25T09:42:41.457 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-08-25T09:42:41.457 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-08-25T09:42:41.457 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-08-25T09:42:41.458 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-08-25T09:42:41.458 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-08-25T09:42:41.461 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-08-25T09:42:41.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-08-25T09:42:41.463 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-08-25T09:42:41.463 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-08-25T09:42:41.463 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-08-25T09:42:41.463 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-08-25T09:42:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-08-25T09:42:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-08-25T09:42:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-08-25T09:42:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-08-25T09:42:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-08-25T09:42:41.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-08-25T09:42:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-08-25T09:42:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-08-25T09:42:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-08-25T09:42:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-08-25T09:42:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-08-25T09:42:41.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-08-25T09:42:41.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-08-25T09:42:41.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-08-25T09:42:41.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-08-25T09:42:41.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-08-25T09:42:41.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-08-25T09:42:41.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-08-25T09:42:41.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-08-25T09:42:41.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-08-25T09:42:41.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-08-25T09:42:41.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-08-25T09:42:41.468 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-08-25T09:42:41.468 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-08-25T09:42:41.468 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-08-25T09:42:41.468 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-08-25T09:42:42.572 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-08-25T09:42:42.572 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-08-25T09:42:42.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-08-25T09:42:42.574 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-08-25T09:42:42.574 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-08-25T09:42:42.574 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-08-25T09:42:42.574 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-08-25T09:42:42.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-08-25T09:42:42.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-08-25T09:42:42.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-08-25T09:42:42.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-08-25T09:42:42.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-08-25T09:42:42.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-08-25T09:42:42.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-08-25T09:42:42.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-08-25T09:42:42.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-08-25T09:42:42.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-08-25T09:42:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-08-25T09:42:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-08-25T09:42:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-08-25T09:42:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-08-25T09:42:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-08-25T09:42:42.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-08-25T09:42:42.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-08-25T09:42:42.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-08-25T09:42:42.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-08-25T09:42:42.585 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-08-25T09:42:42.585 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-08-25T09:42:42.585 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-08-25T09:42:42.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-08-25T09:42:42.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-08-25T09:42:42.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-08-25T09:42:42.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-08-25T09:42:42.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-08-25T09:42:42.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-08-25T09:42:42.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-08-25T09:42:42.589 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-08-25T09:42:42.590 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-08-25T09:42:42.591 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-08-25T09:42:42.592 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-08-25T09:42:42.593 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-08-25T09:42:42.594 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-08-25T09:42:43.458 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:42:43.457+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-08-25T09:42:44.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:42:44.822 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:42:52.267 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:42:52.265+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-08-25T09:43:14.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:43:14.972 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:43:27.838 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-08-25T09:43:27.838 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-08-25T09:43:27.838 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-08-25T09:43:27.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-08-25T09:43:27.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-08-25T09:43:27.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-08-25T09:43:27.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-08-25T09:43:27.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-08-25T09:43:27.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-08-25T09:43:27.840 INFO:teuthology.orchestra.run.smithi070.stdout:done writing 500695040 bytes 2023-08-25T09:43:27.841 INFO:teuthology.orchestra.run.smithi070.stdout:successfully fsync'ed prior to getting full state reported 2023-08-25T09:43:27.841 INFO:teuthology.orchestra.run.smithi070.stdout:starting buffered write 2023-08-25T09:43:27.841 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:43:27.841 INFO:teuthology.orchestra.run.smithi070.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-08-25T09:43:27.841 INFO:teuthology.orchestra.run.smithi070.stdout:close() returned an error as expected 2023-08-25T09:43:27.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:43:27.866 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:43:27.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:43:27.993 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:43:28.133 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:43:28.133 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:43:28.134 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:28.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:28.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.410+0000 7f9c696fa640 1 Processor -- start 2023-08-25T09:43:28.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.410+0000 7f9c696fa640 1 -- start start 2023-08-25T09:43:28.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c696fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c6406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c696fa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6415ec10 0x7f9c640b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c696fa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c640b1b60 con 0x7f9c6406b2b0 2023-08-25T09:43:28.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c696fa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6406bd80 con 0x7f9c6415ec10 2023-08-25T09:43:28.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c696fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6406b7c0 con 0x7f9c6415e830 2023-08-25T09:43:28.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6415ec10 0x7f9c640b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6415ec10 0x7f9c640b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58760/0 (socket says 172.21.15.70:58760) 2023-08-25T09:43:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.412+0000 7f9c637fe640 1 -- 172.21.15.70:0/1631154184 learned_addr learned my addr 172.21.15.70:0/1631154184 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c637fe640 1 -- 172.21.15.70:0/1631154184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 msgr2=0x7f9c6406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:28.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c6406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c637fe640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c6406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c637fe640 1 -- 172.21.15.70:0/1631154184 >> v1:172.21.15.70:6789/0 conn(0x7f9c6406b2b0 legacy=0x7f9c6415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c637fe640 1 -- 172.21.15.70:0/1631154184 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6406baa0 con 0x7f9c6415ec10 2023-08-25T09:43:28.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c627fc640 1 -- 172.21.15.70:0/1631154184 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117506399 0 0) 0x7f9c640b1b60 con 0x7f9c6406b2b0 2023-08-25T09:43:28.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.413+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c6406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:28.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.414+0000 7f9c637fe640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b1ff0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009e50 tx=0x7f9c5c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67a5bcd7c5c0dfb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:28.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.414+0000 7f9c627fc640 1 -- 172.21.15.70:0/1631154184 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c02cb00 con 0x7f9c6415ec10 2023-08-25T09:43:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.414+0000 7f9c627fc640 1 -- 172.21.15.70:0/1631154184 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c02cc60 con 0x7f9c6415ec10 2023-08-25T09:43:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.414+0000 7f9c627fc640 1 -- 172.21.15.70:0/1631154184 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0343f0 con 0x7f9c6415ec10 2023-08-25T09:43:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 -- 172.21.15.70:0/1631154184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 msgr2=0x7f9c640b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b1ff0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009e50 tx=0x7f9c5c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:43:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 -- 172.21.15.70:0/1631154184 shutdown_connections 2023-08-25T09:43:28.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b1ff0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 --2- 172.21.15.70:0/1631154184 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c6406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 -- 172.21.15.70:0/1631154184 >> 172.21.15.70:0/1631154184 conn(0x7f9c64069850 msgr2=0x7f9c64069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:28.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 -- 172.21.15.70:0/1631154184 shutdown_connections 2023-08-25T09:43:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.415+0000 7f9c696fa640 1 -- 172.21.15.70:0/1631154184 wait complete. 2023-08-25T09:43:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.417+0000 7f9c696fa640 1 Processor -- start 2023-08-25T09:43:28.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.417+0000 7f9c696fa640 1 -- start start 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c696fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6406b2b0 0x7f9c64152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c696fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c640c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c696fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c696fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c640c9460 con 0x7f9c6415ec10 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c696fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64153090 con 0x7f9c6406b2b0 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c696fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c64154550 con 0x7f9c6415e830 2023-08-25T09:43:28.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58770/0 (socket says 172.21.15.70:58770) 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 -- 172.21.15.70:0/724661242 learned_addr learned my addr 172.21.15.70:0/724661242 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6406b2b0 0x7f9c64152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 -- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 msgr2=0x7f9c640c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c640c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 -- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6406b2b0 msgr2=0x7f9c64152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6406b2b0 0x7f9c64152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.418+0000 7f9c637fe640 1 -- 172.21.15.70:0/724661242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009b30 con 0x7f9c6415ec10 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.419+0000 7f9c637fe640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b09a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009e50 tx=0x7f9c5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.419+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c03e450 con 0x7f9c6415ec10 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.419+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c03e5d0 con 0x7f9c6415ec10 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.419+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c03e980 con 0x7f9c6415ec10 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.419+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6415ca20 con 0x7f9c6415ec10 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.419+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640c8da0 con 0x7f9c6415ec10 2023-08-25T09:43:28.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.420+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c640c95a0 con 0x7f9c6415ec10 2023-08-25T09:43:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.421+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9c5c03eae0 con 0x7f9c6415ec10 2023-08-25T09:43:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.422+0000 7f9c5bfff640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.422+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f9c5c0ab8a0 con 0x7f9c6415ec10 2023-08-25T09:43:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.422+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.422+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7f9c44072480 con 0x7f9c6415ec10 2023-08-25T09:43:28.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.423+0000 7f9c62ffd640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c54008850 tx=0x7f9c54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:28.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.427+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9c640c95a0 con 0x7f9c6415ec10 2023-08-25T09:43:28.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.618+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c640c95a0 con 0x7f9c6415ec10 2023-08-25T09:43:28.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.619+0000 7f9c5bfff640 1 -- 172.21.15.70:0/724661242 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f9c640c95a0 con 0x7f9c6415ec10 2023-08-25T09:43:28.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:28.622 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":79,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:46.536583+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","full","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":1}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.38","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:28.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.621+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 msgr2=0x7f9c4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.621+0000 7f9c696fa640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c54008850 tx=0x7f9c54008040 comp rx=0 tx=0).stop 2023-08-25T09:43:28.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.621+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 msgr2=0x7f9c640b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.621+0000 7f9c696fa640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b09a0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009e50 tx=0x7f9c5c004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 shutdown_connections 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6415ec10 0x7f9c640b09a0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c6415e830 0x7f9c640c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 --2- 172.21.15.70:0/724661242 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6406b2b0 0x7f9c64152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 >> 172.21.15.70:0/724661242 conn(0x7f9c64069850 msgr2=0x7f9c640b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:28.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 shutdown_connections 2023-08-25T09:43:28.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.622+0000 7f9c696fa640 1 -- 172.21.15.70:0/724661242 wait complete. 2023-08-25T09:43:28.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:42:22.389960+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:42:23.780164+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:43:28.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:43:28.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.858+0000 7fc98d742640 1 Processor -- start 2023-08-25T09:43:28.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.859+0000 7fc98d742640 1 -- start start 2023-08-25T09:43:28.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.860+0000 7fc98d742640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc9880c2110 0x7fc988069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.860+0000 7fc98d742640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc988069d90 0x7fc9880ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.860+0000 7fc98d742640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9880c2550 con 0x7fc98815ae50 2023-08-25T09:43:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.860+0000 7fc98d742640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98806a200 con 0x7fc9880c2110 2023-08-25T09:43:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.860+0000 7fc98d742640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9880ce330 con 0x7fc988069d90 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.860+0000 7fc9867fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc9880c2110 0x7fc988069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9867fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc9880c2110 0x7fc988069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58788/0 (socket says 172.21.15.70:58788) 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9867fc640 1 -- 172.21.15.70:0/437055370 learned_addr learned my addr 172.21.15.70:0/437055370 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc986ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc98815ae50 0x7fc9880cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36682/0 (socket says 172.21.15.70:36682) 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9877fe640 1 --2- 172.21.15.70:0/437055370 >> v2:172.21.15.155:3300/0 conn(0x7fc988069d90 0x7fc9880ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9867fc640 1 -- 172.21.15.70:0/437055370 >> v2:172.21.15.155:3300/0 conn(0x7fc988069d90 msgr2=0x7fc9880ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9867fc640 1 --2- 172.21.15.70:0/437055370 >> v2:172.21.15.155:3300/0 conn(0x7fc988069d90 0x7fc9880ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9867fc640 1 -- 172.21.15.70:0/437055370 >> v1:172.21.15.70:6789/0 conn(0x7fc98815ae50 legacy=0x7fc9880cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.861+0000 7fc9867fc640 1 -- 172.21.15.70:0/437055370 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9880cc5f0 con 0x7fc9880c2110 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.862+0000 7fc9867fc640 1 --2- 172.21.15.70:0/437055370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9880c2110 0x7fc988069850 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc9880b00d0 tx=0x7fc97002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c226b37bef6b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc985ffb640 1 -- 172.21.15.70:0/437055370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97002cbb0 con 0x7fc9880c2110 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc985ffb640 1 -- 172.21.15.70:0/437055370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97002cd10 con 0x7fc9880c2110 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc985ffb640 1 -- 172.21.15.70:0/437055370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970034610 con 0x7fc9880c2110 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 -- 172.21.15.70:0/437055370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9880c2110 msgr2=0x7fc988069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 --2- 172.21.15.70:0/437055370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9880c2110 0x7fc988069850 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc9880b00d0 tx=0x7fc97002c010 comp rx=0 tx=0).stop 2023-08-25T09:43:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 -- 172.21.15.70:0/437055370 shutdown_connections 2023-08-25T09:43:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 --2- 172.21.15.70:0/437055370 >> v2:172.21.15.155:3300/0 conn(0x7fc988069d90 0x7fc9880ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 --2- 172.21.15.70:0/437055370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9880c2110 0x7fc988069850 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 -- 172.21.15.70:0/437055370 >> 172.21.15.70:0/437055370 conn(0x7fc9880cb0d0 msgr2=0x7fc9880cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:28.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.863+0000 7fc98d742640 1 -- 172.21.15.70:0/437055370 shutdown_connections 2023-08-25T09:43:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.864+0000 7fc98d742640 1 -- 172.21.15.70:0/437055370 wait complete. 2023-08-25T09:43:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.865+0000 7fc98d742640 1 Processor -- start 2023-08-25T09:43:28.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.865+0000 7fc98d742640 1 -- start start 2023-08-25T09:43:28.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.866+0000 7fc98d742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc988069d90 0x7fc988158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.866+0000 7fc98d742640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880c2110 0x7fc9880a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.866+0000 7fc98d742640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 0x7fc9880af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.866+0000 7fc98d742640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc988158560 con 0x7fc98815ae50 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.866+0000 7fc98d742640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c7390 con 0x7fc9880c2110 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.866+0000 7fc98d742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880c7670 con 0x7fc988069d90 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 0x7fc9880af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 0x7fc9880af1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58800/0 (socket says 172.21.15.70:58800) 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 -- 172.21.15.70:0/235935962 learned_addr learned my addr 172.21.15.70:0/235935962 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9867fc640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880c2110 0x7fc9880a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc986ffd640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc988069d90 0x7fc988158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 -- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc988069d90 msgr2=0x7fc988158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc988069d90 0x7fc988158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 -- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880c2110 msgr2=0x7fc9880a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880c2110 0x7fc9880a8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 -- 172.21.15.70:0/235935962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc970009b30 con 0x7fc98815ae50 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.867+0000 7fc9877fe640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 0x7fc9880af1b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d800 tx=0x7fc98000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.868+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98000b6c0 con 0x7fc98815ae50 2023-08-25T09:43:28.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.868+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98000b840 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.868+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98000bbf0 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.868+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9880c7950 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.868+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9880c7f10 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.869+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc988066fb0 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.870+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc98000bd50 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.871+0000 7fc97f7fe640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96806a050 0x7fc96806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.871+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fc980088c30 con 0x7fc98815ae50 2023-08-25T09:43:28.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.871+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7fc968072500 con 0x7fc98815ae50 2023-08-25T09:43:28.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.871+0000 7fc986ffd640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96806a050 0x7fc96806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:28.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.872+0000 7fc986ffd640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96806a050 0x7fc96806c510 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc978007fd0 tx=0x7fc978009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:28.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:28.875+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9800541b0 con 0x7fc98815ae50 2023-08-25T09:43:29.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.062+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc98815e360 con 0x7fc98815ae50 2023-08-25T09:43:29.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.063+0000 7fc97f7fe640 1 -- 172.21.15.70:0/235935962 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc9800577d0 con 0x7fc98815ae50 2023-08-25T09:43:29.066 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:43:29.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.065+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96806a050 msgr2=0x7fc96806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.065+0000 7fc98d742640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96806a050 0x7fc96806c510 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc978007fd0 tx=0x7fc978009040 comp rx=0 tx=0).stop 2023-08-25T09:43:29.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.065+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 msgr2=0x7fc9880af1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.065+0000 7fc98d742640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 0x7fc9880af1b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc98000d800 tx=0x7fc98000dcc0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 shutdown_connections 2023-08-25T09:43:29.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96806a050 0x7fc96806c510 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc98815ae50 0x7fc9880af1b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9880c2110 0x7fc9880a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 --2- 172.21.15.70:0/235935962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc988069d90 0x7fc988158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 >> 172.21.15.70:0/235935962 conn(0x7fc9880cb0d0 msgr2=0x7fc9880ae110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:29.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 shutdown_connections 2023-08-25T09:43:29.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.066+0000 7fc98d742640 1 -- 172.21.15.70:0/235935962 wait complete. 2023-08-25T09:43:29.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:43:29.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.299+0000 7f08b462d640 1 Processor -- start 2023-08-25T09:43:29.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.300+0000 7f08b462d640 1 -- start start 2023-08-25T09:43:29.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b462d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f08ac15e240 0x7f08ac070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b462d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08ac0cd660 0x7f08ac0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b462d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac06f8e0 con 0x7f08ac159bf0 2023-08-25T09:43:29.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b462d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0ce030 con 0x7f08ac0cd660 2023-08-25T09:43:29.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b462d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac06fbc0 con 0x7f08ac15e240 2023-08-25T09:43:29.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b2bca640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08ac0cd660 0x7f08ac0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b23c9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f08ac159bf0 0x7f08ac06f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48020/0 (socket says 172.21.15.70:48020) 2023-08-25T09:43:29.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.301+0000 7f08b23c9640 1 -- 172.21.15.70:0/3337420711 learned_addr learned my addr 172.21.15.70:0/3337420711 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:29.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.302+0000 7f08b2bca640 1 -- 172.21.15.70:0/3337420711 >> v2:172.21.15.155:3300/0 conn(0x7f08ac15e240 msgr2=0x7f08ac070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:29.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.302+0000 7f08b2bca640 1 --2- 172.21.15.70:0/3337420711 >> v2:172.21.15.155:3300/0 conn(0x7f08ac15e240 0x7f08ac070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.302+0000 7f08b2bca640 1 -- 172.21.15.70:0/3337420711 >> v1:172.21.15.70:6789/0 conn(0x7f08ac159bf0 legacy=0x7f08ac06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.302+0000 7f08b2bca640 1 -- 172.21.15.70:0/3337420711 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac06fea0 con 0x7f08ac0cd660 2023-08-25T09:43:29.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.302+0000 7f08b13c7640 1 -- 172.21.15.70:0/3337420711 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205640751 0 0) 0x7f08ac06f8e0 con 0x7f08ac159bf0 2023-08-25T09:43:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.303+0000 7f08b2bca640 1 --2- 172.21.15.70:0/3337420711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0cd660 0x7f08ac0cdaf0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f08ac06af90 tx=0x7f08a402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b571ad55a00224 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.303+0000 7f08b13c7640 1 -- 172.21.15.70:0/3337420711 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a402cc80 con 0x7f08ac0cd660 2023-08-25T09:43:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.303+0000 7f08b13c7640 1 -- 172.21.15.70:0/3337420711 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a402cde0 con 0x7f08ac0cd660 2023-08-25T09:43:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.303+0000 7f08b13c7640 1 -- 172.21.15.70:0/3337420711 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4034660 con 0x7f08ac0cd660 2023-08-25T09:43:29.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 -- 172.21.15.70:0/3337420711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0cd660 msgr2=0x7f08ac0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 --2- 172.21.15.70:0/3337420711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0cd660 0x7f08ac0cdaf0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f08ac06af90 tx=0x7f08a402bed0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 -- 172.21.15.70:0/3337420711 shutdown_connections 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 --2- 172.21.15.70:0/3337420711 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac0cd660 0x7f08ac0cdaf0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 --2- 172.21.15.70:0/3337420711 >> v2:172.21.15.155:3300/0 conn(0x7f08ac15e240 0x7f08ac070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 -- 172.21.15.70:0/3337420711 >> 172.21.15.70:0/3337420711 conn(0x7f08ac0cb0d0 msgr2=0x7f08ac0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 -- 172.21.15.70:0/3337420711 shutdown_connections 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.304+0000 7f08b462d640 1 -- 172.21.15.70:0/3337420711 wait complete. 2023-08-25T09:43:29.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.306+0000 7f08b462d640 1 Processor -- start 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.306+0000 7f08b462d640 1 -- start start 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b462d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ac0cd660 0x7f08ac14b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b462d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac159bf0 0x7f08ac15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b462d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 0x7f08ac0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b462d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1567d0 con 0x7f08ac15e240 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b462d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac157630 con 0x7f08ac159bf0 2023-08-25T09:43:29.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b462d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac14bb00 con 0x7f08ac0cd660 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.307+0000 7f08b2bca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 0x7f08ac0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 0x7f08ac0ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60338/0 (socket says 172.21.15.70:60338) 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 -- 172.21.15.70:0/647363333 learned_addr learned my addr 172.21.15.70:0/647363333 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b23c9640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ac0cd660 0x7f08ac14b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b1bc8640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac159bf0 0x7f08ac15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 -- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ac0cd660 msgr2=0x7f08ac14b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ac0cd660 0x7f08ac14b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 -- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac159bf0 msgr2=0x7f08ac15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac159bf0 0x7f08ac15cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 -- 172.21.15.70:0/647363333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009a70 con 0x7f08ac15e240 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.308+0000 7f08b2bca640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 0x7f08ac0ac880 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.309+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4003990 con 0x7f08ac15e240 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.309+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a4003af0 con 0x7f08ac15e240 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.309+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac156a20 con 0x7f08ac15e240 2023-08-25T09:43:29.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.309+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4003e80 con 0x7f08ac15e240 2023-08-25T09:43:29.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.309+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac156fe0 con 0x7f08ac15e240 2023-08-25T09:43:29.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.310+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac065d00 con 0x7f08ac15e240 2023-08-25T09:43:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.311+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08a4004000 con 0x7f08ac15e240 2023-08-25T09:43:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.312+0000 7f08a2ffd640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f089006a050 0x7f089006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.312+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f08a40ab370 con 0x7f08ac15e240 2023-08-25T09:43:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.312+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7f0890072480 con 0x7f08ac15e240 2023-08-25T09:43:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.312+0000 7f08b23c9640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f089006a050 0x7f089006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.313+0000 7f08b23c9640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f089006a050 0x7f089006c510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007fd0 tx=0x7f08a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:29.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.317+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08a40768f0 con 0x7f08ac15e240 2023-08-25T09:43:29.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.503+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f08ac1577c0 con 0x7f08ac15e240 2023-08-25T09:43:29.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.504+0000 7f08a2ffd640 1 -- 172.21.15.70:0/647363333 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v79) v1 ==== 92+0+10 (secure 0 0 0) 0x7f08a404d0a0 con 0x7f08ac15e240 2023-08-25T09:43:29.507 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:43:29.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f089006a050 msgr2=0x7f089006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f089006a050 0x7f089006c510 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007fd0 tx=0x7f08a8009040 comp rx=0 tx=0).stop 2023-08-25T09:43:29.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 msgr2=0x7f08ac0ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 0x7f08ac0ac880 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a4004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:29.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 shutdown_connections 2023-08-25T09:43:29.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f089006a050 0x7f089006c510 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08ac15e240 0x7f08ac0ac880 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08ac159bf0 0x7f08ac15cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 --2- 172.21.15.70:0/647363333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08ac0cd660 0x7f08ac14b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 >> 172.21.15.70:0/647363333 conn(0x7f08ac0cb0d0 msgr2=0x7f08ac0654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:29.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 shutdown_connections 2023-08-25T09:43:29.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.507+0000 7f08b462d640 1 -- 172.21.15.70:0/647363333 wait complete. 2023-08-25T09:43:29.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:43:29.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.750+0000 7f96efa1f640 1 Processor -- start 2023-08-25T09:43:29.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.750+0000 7f96efa1f640 1 -- start start 2023-08-25T09:43:29.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.751+0000 7f96efa1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 0x7f96e80b8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.751+0000 7f96efa1f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f96e80b6a60 0x7f96e80cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.751+0000 7f96efa1f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e80b6ef0 con 0x7f96e80cf440 2023-08-25T09:43:29.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.751+0000 7f96efa1f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e80cfdc0 con 0x7f96e80b6a60 2023-08-25T09:43:29.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.751+0000 7f96efa1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80b7140 con 0x7f96e815ccd0 2023-08-25T09:43:29.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96ecfba640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f96e80cf440 0x7f96e80d0b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48024/0 (socket says 172.21.15.70:48024) 2023-08-25T09:43:29.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96ecfba640 1 -- 172.21.15.70:0/3022212793 learned_addr learned my addr 172.21.15.70:0/3022212793 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:29.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96edfbc640 1 --2- 172.21.15.70:0/3022212793 >> v2:172.21.15.155:3300/0 conn(0x7f96e80b6a60 0x7f96e80cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96ed7bb640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 0x7f96e80b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109604276 0 0) 0x7f96e80b6ef0 con 0x7f96e80cf440 2023-08-25T09:43:29.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d4003520 con 0x7f96e80cf440 2023-08-25T09:43:29.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.752+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3666657980 0 0) 0x7f96d4003520 con 0x7f96e80cf440 2023-08-25T09:43:29.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 msgr2=0x7f96e80b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:29.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 0x7f96e80b8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 >> v2:172.21.15.155:3300/0 conn(0x7f96e80b6a60 msgr2=0x7f96e80cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:29.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 --2- 172.21.15.70:0/3022212793 >> v2:172.21.15.155:3300/0 conn(0x7f96e80b6a60 0x7f96e80cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80b7420 con 0x7f96e80cf440 2023-08-25T09:43:29.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f96d8001670 con 0x7f96e80cf440 2023-08-25T09:43:29.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 >> v1:172.21.15.70:6789/0 conn(0x7f96e80cf440 legacy=0x7f96e80d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d4003970 0x7f96d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 0x7f96d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80cfdc0 con 0x7f96d40047c0 2023-08-25T09:43:29.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80b7140 con 0x7f96d4003970 2023-08-25T09:43:29.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d400b980 con 0x7f96e815ccd0 2023-08-25T09:43:29.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96ecfba640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96ecfba640 1 -- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 msgr2=0x7f96d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:29.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96ecfba640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e815ccd0 0x7f96d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96ecfba640 1 -- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d4003970 msgr2=0x7f96d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:29.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96ecfba640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96d4003970 0x7f96d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.753+0000 7f96ecfba640 1 -- 172.21.15.70:0/3022212793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4003520 con 0x7f96d40047c0 2023-08-25T09:43:29.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.754+0000 7f96ecfba640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96d4004c40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c240 tx=0x7f96d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63aaa553a1ef331d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:29.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d802cb00 con 0x7f96d40047c0 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d802cc60 con 0x7f96d40047c0 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96dffff640 1 -- 172.21.15.70:0/3022212793 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8005af0 con 0x7f96d40047c0 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96efa1f640 1 -- 172.21.15.70:0/3022212793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 msgr2=0x7f96d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96efa1f640 1 --2- 172.21.15.70:0/3022212793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96d4004c40 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f96d800c240 tx=0x7f96d40057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96edfbc640 1 -- 172.21.15.70:0/3022212793 reap_dead start 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96efa1f640 1 -- 172.21.15.70:0/3022212793 shutdown_connections 2023-08-25T09:43:29.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96efa1f640 1 -- 172.21.15.70:0/3022212793 >> 172.21.15.70:0/3022212793 conn(0x7f96e8070260 msgr2=0x7f96e8070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96efa1f640 1 -- 172.21.15.70:0/3022212793 shutdown_connections 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.755+0000 7f96efa1f640 1 -- 172.21.15.70:0/3022212793 wait complete. 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.757+0000 7f96efa1f640 1 Processor -- start 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.757+0000 7f96efa1f640 1 -- start start 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96efa1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96e8143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96efa1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96e80cc7b0 0x7f96e80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96efa1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e80cb1d0 0x7f96e80d2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96efa1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80cc390 con 0x7f96d40047c0 2023-08-25T09:43:29.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96efa1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80cb750 con 0x7f96e80cc7b0 2023-08-25T09:43:29.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96efa1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80cb9f0 con 0x7f96e80cb1d0 2023-08-25T09:43:29.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.758+0000 7f96ed7bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96e8143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96e8143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60356/0 (socket says 172.21.15.70:60356) 2023-08-25T09:43:29.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 -- 172.21.15.70:0/1948796333 learned_addr learned my addr 172.21.15.70:0/1948796333 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:29.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 -- 172.21.15.70:0/1948796333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e80cb1d0 msgr2=0x7f96e80d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e80cb1d0 0x7f96e80d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 -- 172.21.15.70:0/1948796333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96e80cc7b0 msgr2=0x7f96e80d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96e80cc7b0 0x7f96e80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 -- 172.21.15.70:0/1948796333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e80cbcd0 con 0x7f96d40047c0 2023-08-25T09:43:29.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ed7bb640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96e8143bf0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f96e400e7a0 tx=0x7f96e400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:29.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.759+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400eeb0 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.760+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e400c8c0 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.760+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e400cc70 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.760+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e80cbfb0 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.760+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e8144130 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.761+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e8144320 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.762+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f96e4013470 con 0x7f96d40047c0 2023-08-25T09:43:29.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.763+0000 7f96ddffb640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96cc06a080 0x7f96cc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:29.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.764+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f96e40885e0 con 0x7f96d40047c0 2023-08-25T09:43:29.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.764+0000 7f96ecfba640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96cc06a080 0x7f96cc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:29.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.764+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7f96cc0724b0 con 0x7f96d40047c0 2023-08-25T09:43:29.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.764+0000 7f96ecfba640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96cc06a080 0x7f96cc06c540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96d8034000 tx=0x7f96d802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:29.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.768+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f96e4053bf0 con 0x7f96d40047c0 2023-08-25T09:43:29.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.956+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96e8066fb0 con 0x7f96d40047c0 2023-08-25T09:43:29.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.957+0000 7f96ddffb640 1 -- 172.21.15.70:0/1948796333 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96e4057210 con 0x7f96d40047c0 2023-08-25T09:43:29.960 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:43:29.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96cc06a080 msgr2=0x7f96cc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96cc06a080 0x7f96cc06c540 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f96d8034000 tx=0x7f96d802e000 comp rx=0 tx=0).stop 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 msgr2=0x7f96e8143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96e8143bf0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f96e400e7a0 tx=0x7f96e400ec60 comp rx=0 tx=0).stop 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 shutdown_connections 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96cc06a080 0x7f96cc06c540 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96e80cb1d0 0x7f96e80d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96e80cc7b0 0x7f96e80d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 --2- 172.21.15.70:0/1948796333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f96d40047c0 0x7f96e8143bf0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:29.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 >> 172.21.15.70:0/1948796333 conn(0x7f96e8070260 msgr2=0x7f96e815a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:29.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 shutdown_connections 2023-08-25T09:43:29.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:29.960+0000 7f96efa1f640 1 -- 172.21.15.70:0/1948796333 wait complete. 2023-08-25T09:43:29.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:43:30.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.195+0000 7f3ae590c640 1 Processor -- start 2023-08-25T09:43:30.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.196+0000 7f3ae590c640 1 -- start start 2023-08-25T09:43:30.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.197+0000 7f3ae590c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3ae0069f60 0x7f3ae00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.197+0000 7f3ae590c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00c2550 con 0x7f3ae015ae50 2023-08-25T09:43:30.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.197+0000 7f3ae590c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00ce330 con 0x7f3ae00c2110 2023-08-25T09:43:30.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.197+0000 7f3ae590c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae00cc5f0 con 0x7f3ae0069f60 2023-08-25T09:43:30.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.197+0000 7f3adeffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ae015ae50 0x7f3ae00cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48026/0 (socket says 172.21.15.70:48026) 2023-08-25T09:43:30.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.197+0000 7f3adeffd640 1 -- 172.21.15.70:0/1155767090 learned_addr learned my addr 172.21.15.70:0/1155767090 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:30.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.198+0000 7f3adf7fe640 1 --2- 172.21.15.70:0/1155767090 >> v2:172.21.15.70:3300/0 conn(0x7f3ae0069f60 0x7f3ae00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.198+0000 7f3ade7fc640 1 -- 172.21.15.70:0/1155767090 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391656519 0 0) 0x7f3ae00c2550 con 0x7f3ae015ae50 2023-08-25T09:43:30.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.198+0000 7f3ade7fc640 1 -- 172.21.15.70:0/1155767090 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3acc003520 con 0x7f3ae015ae50 2023-08-25T09:43:30.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.198+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1155767090 >> v1:172.21.15.155:6789/0 conn(0x7f3ae00c2110 legacy=0x7f3ae0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:30.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.198+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1155767090 >> v1:172.21.15.70:6789/0 conn(0x7f3ae015ae50 legacy=0x7f3ae00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.198+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1155767090 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0070650 con 0x7f3ae0069f60 2023-08-25T09:43:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.199+0000 7f3adf7fe640 1 --2- 172.21.15.70:0/1155767090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae0069f60 0x7f3ae00ccad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009d90 tx=0x7f3ad802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f58e6b0cdea14d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ade7fc640 1 -- 172.21.15.70:0/1155767090 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad802cd80 con 0x7f3ae0069f60 2023-08-25T09:43:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ade7fc640 1 -- 172.21.15.70:0/1155767090 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8034440 con 0x7f3ae0069f60 2023-08-25T09:43:30.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ade7fc640 1 -- 172.21.15.70:0/1155767090 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80347d0 con 0x7f3ae0069f60 2023-08-25T09:43:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 -- 172.21.15.70:0/1155767090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae0069f60 msgr2=0x7f3ae00ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1155767090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae0069f60 0x7f3ae00ccad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009d90 tx=0x7f3ad802bfd0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 -- 172.21.15.70:0/1155767090 shutdown_connections 2023-08-25T09:43:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1155767090 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae0069f60 0x7f3ae00ccad0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 -- 172.21.15.70:0/1155767090 >> 172.21.15.70:0/1155767090 conn(0x7f3ae00cb0d0 msgr2=0x7f3ae00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 -- 172.21.15.70:0/1155767090 shutdown_connections 2023-08-25T09:43:30.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.200+0000 7f3ae590c640 1 -- 172.21.15.70:0/1155767090 wait complete. 2023-08-25T09:43:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.202+0000 7f3ae590c640 1 Processor -- start 2023-08-25T09:43:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.202+0000 7f3ae590c640 1 -- start start 2023-08-25T09:43:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.203+0000 7f3ae590c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae0069f60 0x7f3ae00b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.203+0000 7f3ae590c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ae00c2110 0x7f3ae00ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.203+0000 7f3ae590c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 0x7f3ae01e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.203+0000 7f3ae590c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00b1090 con 0x7f3ae015ae50 2023-08-25T09:43:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.203+0000 7f3ae590c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01e2460 con 0x7f3ae0069f60 2023-08-25T09:43:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.203+0000 7f3ae590c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01f3d90 con 0x7f3ae00c2110 2023-08-25T09:43:30.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 0x7f3ae01e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 0x7f3ae01e49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60376/0 (socket says 172.21.15.70:60376) 2023-08-25T09:43:30.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1811169358 learned_addr learned my addr 172.21.15.70:0/1811169358 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:30.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adeffd640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae0069f60 0x7f3ae00b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3ad67fc640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ae00c2110 0x7f3ae00ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ae00c2110 msgr2=0x7f3ae00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ae00c2110 0x7f3ae00ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae0069f60 msgr2=0x7f3ae00b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae0069f60 0x7f3ae00b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 -- 172.21.15.70:0/1811169358 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad8009a70 con 0x7f3ae015ae50 2023-08-25T09:43:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adeffd640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae0069f60 0x7f3ae00b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3ad67fc640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ae00c2110 0x7f3ae00ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.204+0000 7f3adf7fe640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 0x7f3ae01e49b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009d90 tx=0x7f3ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.205+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8040650 con 0x7f3ae015ae50 2023-08-25T09:43:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.205+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad80407b0 con 0x7f3ae015ae50 2023-08-25T09:43:30.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.205+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8040b60 con 0x7f3ae015ae50 2023-08-25T09:43:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.205+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae01f4070 con 0x7f3ae015ae50 2023-08-25T09:43:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.205+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae01f4630 con 0x7f3ae015ae50 2023-08-25T09:43:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.206+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae01f4820 con 0x7f3ae015ae50 2023-08-25T09:43:30.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.207+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3ad8040cc0 con 0x7f3ae015ae50 2023-08-25T09:43:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.208+0000 7f3adcff9640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ac006a050 0x7f3ac006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.209+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f3ad80ab4e0 con 0x7f3ae015ae50 2023-08-25T09:43:30.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.209+0000 7f3adeffd640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ac006a050 0x7f3ac006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.209+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7f3ac0072480 con 0x7f3ae015ae50 2023-08-25T09:43:30.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.209+0000 7f3adeffd640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ac006a050 0x7f3ac006c510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00011e0 tx=0x7f3ad0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:30.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.213+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ae01f4820 con 0x7f3ae015ae50 2023-08-25T09:43:30.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.411+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ae01e25d0 con 0x7f3ae015ae50 2023-08-25T09:43:30.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.415+0000 7f3adcff9640 1 -- 172.21.15.70:0/1811169358 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ae01e25d0 con 0x7f3ae015ae50 2023-08-25T09:43:30.418 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:43:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.419+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ac006a050 msgr2=0x7f3ac006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.419+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ac006a050 0x7f3ac006c510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00011e0 tx=0x7f3ad0009040 comp rx=0 tx=0).stop 2023-08-25T09:43:30.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.419+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 msgr2=0x7f3ae01e49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.419+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 0x7f3ae01e49b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8009d90 tx=0x7f3ad8004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:30.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 shutdown_connections 2023-08-25T09:43:30.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ac006a050 0x7f3ac006c510 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ae015ae50 0x7f3ae01e49b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ae00c2110 0x7f3ae00ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 --2- 172.21.15.70:0/1811169358 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ae0069f60 0x7f3ae00b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 >> 172.21.15.70:0/1811169358 conn(0x7f3ae00cb0d0 msgr2=0x7f3ae00adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:30.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 shutdown_connections 2023-08-25T09:43:30.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.421+0000 7f3ae590c640 1 -- 172.21.15.70:0/1811169358 wait complete. 2023-08-25T09:43:30.450 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:43:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.663+0000 7f2fe3fff640 1 Processor -- start 2023-08-25T09:43:30.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.663+0000 7f2fe3fff640 1 -- start start 2023-08-25T09:43:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fe415a7d0 0x7f2fe40b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe3fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2fe415abb0 0x7f2fe40b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4071850 0x7f2fe406ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe40c53b0 con 0x7f2fe415a7d0 2023-08-25T09:43:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe3fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe415e4a0 con 0x7f2fe415abb0 2023-08-25T09:43:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe415f020 con 0x7f2fe4071850 2023-08-25T09:43:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fe415a7d0 0x7f2fe40b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fe415a7d0 0x7f2fe40b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60382/0 (socket says 172.21.15.70:60382) 2023-08-25T09:43:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/1469570372 learned_addr learned my addr 172.21.15.70:0/1469570372 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:30.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/1469570372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4071850 msgr2=0x7f2fe406ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:30.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/1469570372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4071850 0x7f2fe406ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/1469570372 >> v2:172.21.15.155:3300/0 conn(0x7f2fe415abb0 msgr2=0x7f2fe40b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.665+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/1469570372 >> v2:172.21.15.155:3300/0 conn(0x7f2fe415abb0 0x7f2fe40b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.666+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/1469570372 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe415e6f0 con 0x7f2fe415a7d0 2023-08-25T09:43:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.667+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/1469570372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe415a7d0 0x7f2fe40b3620 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40c1f50 tx=0x7f2fd402b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c9b3da2b582f5c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.667+0000 7f2fe1ffb640 1 -- 172.21.15.70:0/1469570372 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4031760 con 0x7f2fe415a7d0 2023-08-25T09:43:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.667+0000 7f2fe1ffb640 1 -- 172.21.15.70:0/1469570372 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd40318c0 con 0x7f2fe415a7d0 2023-08-25T09:43:30.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.667+0000 7f2fe1ffb640 1 -- 172.21.15.70:0/1469570372 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4031c50 con 0x7f2fe415a7d0 2023-08-25T09:43:30.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.667+0000 7f2fe3fff640 1 -- 172.21.15.70:0/1469570372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe415a7d0 msgr2=0x7f2fe40b3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.667+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/1469570372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe415a7d0 0x7f2fe40b3620 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40c1f50 tx=0x7f2fd402b800 comp rx=0 tx=0).stop 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 -- 172.21.15.70:0/1469570372 shutdown_connections 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/1469570372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe4071850 0x7f2fe406ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/1469570372 >> v2:172.21.15.155:3300/0 conn(0x7f2fe415abb0 0x7f2fe40b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/1469570372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe415a7d0 0x7f2fe40b3620 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 -- 172.21.15.70:0/1469570372 >> 172.21.15.70:0/1469570372 conn(0x7f2fe40aff60 msgr2=0x7f2fe40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 -- 172.21.15.70:0/1469570372 shutdown_connections 2023-08-25T09:43:30.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.668+0000 7f2fe3fff640 1 -- 172.21.15.70:0/1469570372 wait complete. 2023-08-25T09:43:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.669+0000 7f2fe3fff640 1 Processor -- start 2023-08-25T09:43:30.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.670+0000 7f2fe3fff640 1 -- start start 2023-08-25T09:43:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 0x7f2fe41557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe415a7d0 0x7f2fe40cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070980 0x7f2fe4070e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe41f67a0 con 0x7f2fe4071850 2023-08-25T09:43:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe3fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4071350 con 0x7f2fe4070980 2023-08-25T09:43:30.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe40cb8b0 con 0x7f2fe415a7d0 2023-08-25T09:43:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 0x7f2fe41557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 0x7f2fe41557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60396/0 (socket says 172.21.15.70:60396) 2023-08-25T09:43:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/3436973812 learned_addr learned my addr 172.21.15.70:0/3436973812 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe415a7d0 msgr2=0x7f2fe40cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe415a7d0 0x7f2fe40cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe37fe640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070980 0x7f2fe4070e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe415a7d0 0x7f2fe40cb370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070980 msgr2=0x7f2fe4070e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070980 0x7f2fe4070e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.671+0000 7f2fe2ffd640 1 -- 172.21.15.70:0/3436973812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40094d0 con 0x7f2fe4071850 2023-08-25T09:43:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.672+0000 7f2fe2ffd640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 0x7f2fe41557c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2fd402b840 tx=0x7f2fd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:30.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.672+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd403b800 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.672+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd403b9a0 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.672+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd403bd50 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.672+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe4154a00 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.672+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe4154fc0 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.673+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4066fb0 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.674+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2fd404e450 con 0x7f2fe4071850 2023-08-25T09:43:30.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.675+0000 7f2fcb7fe640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.675+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f2fd402f050 con 0x7f2fe4071850 2023-08-25T09:43:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.675+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7f2fd0072480 con 0x7f2fe4071850 2023-08-25T09:43:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.675+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:30.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.676+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2fe41f5710 tx=0x7f2fd8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:30.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.679+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2fd4075c20 con 0x7f2fe4071850 2023-08-25T09:43:30.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.880+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2fe40b0d60 con 0x7f2fe4071850 2023-08-25T09:43:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.881+0000 7f2fcb7fe640 1 -- 172.21.15.70:0/3436973812 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2fd4079240 con 0x7f2fe4071850 2023-08-25T09:43:30.883 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:43:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 msgr2=0x7f2fd006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2fe41f5710 tx=0x7f2fd8008040 comp rx=0 tx=0).stop 2023-08-25T09:43:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 msgr2=0x7f2fe41557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 0x7f2fe41557c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f2fd402b840 tx=0x7f2fd40046c0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 shutdown_connections 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fe4070980 0x7f2fe4070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fe415a7d0 0x7f2fe40cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.883+0000 7f2fe3fff640 1 --2- 172.21.15.70:0/3436973812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fe4071850 0x7f2fe41557c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.884+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 >> 172.21.15.70:0/3436973812 conn(0x7f2fe40aff60 msgr2=0x7f2fe40ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.884+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 shutdown_connections 2023-08-25T09:43:30.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:30.884+0000 7f2fe3fff640 1 -- 172.21.15.70:0/3436973812 wait complete. 2023-08-25T09:43:30.904 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:43:30.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:43:30.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:43:30.998 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:43:30.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:43:30.999 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:43:31.009 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[103229]: fuse finished with error 0 and tester_r 0 2023-08-25T09:43:31.178 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:43:31.179 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:31.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.462+0000 7f3485c21640 1 Processor -- start 2023-08-25T09:43:31.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.462+0000 7f3485c21640 1 -- start start 2023-08-25T09:43:31.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.463+0000 7f3485c21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f34800b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.463+0000 7f3485c21640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f34800cad20 0x7f34800cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.463+0000 7f3485c21640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34800cb6c0 con 0x7f34800b12f0 2023-08-25T09:43:31.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.463+0000 7f3485c21640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348015e7c0 con 0x7f34800cad20 2023-08-25T09:43:31.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.463+0000 7f3485c21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3480071b50 con 0x7f348015a7c0 2023-08-25T09:43:31.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f34800b12f0 0x7f348015ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48040/0 (socket says 172.21.15.70:48040) 2023-08-25T09:43:31.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347ffff640 1 -- 172.21.15.70:0/184658260 learned_addr learned my addr 172.21.15.70:0/184658260 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:31.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f3484c1f640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f34800b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f3485420640 1 --2- 172.21.15.70:0/184658260 >> v2:172.21.15.155:3300/0 conn(0x7f34800cad20 0x7f34800cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726766716 0 0) 0x7f34800cb6c0 con 0x7f34800b12f0 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003520 con 0x7f34800b12f0 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388586968 0 0) 0x7f346c003520 con 0x7f34800b12f0 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 msgr2=0x7f34800b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f34800b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 >> v2:172.21.15.155:3300/0 conn(0x7f34800cad20 msgr2=0x7f34800cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 --2- 172.21.15.70:0/184658260 >> v2:172.21.15.155:3300/0 conn(0x7f34800cad20 0x7f34800cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.464+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348015e490 con 0x7f34800b12f0 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3468001670 con 0x7f34800b12f0 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 >> v1:172.21.15.70:6789/0 conn(0x7f34800b12f0 legacy=0x7f348015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c003970 0x7f346c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0047c0 0x7f346c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f346c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348015e7c0 con 0x7f346c0047c0 2023-08-25T09:43:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480071b50 con 0x7f346c003970 2023-08-25T09:43:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00b980 con 0x7f348015a7c0 2023-08-25T09:43:31.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347ffff640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0047c0 0x7f346c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f3484c1f640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c003970 0x7f346c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f3485420640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f346c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347ffff640 1 -- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 msgr2=0x7f346c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347ffff640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f346c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347ffff640 1 -- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c003970 msgr2=0x7f346c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347ffff640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c003970 0x7f346c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f347ffff640 1 -- 172.21.15.70:0/184658260 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c003520 con 0x7f346c0047c0 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.465+0000 7f3485420640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f348015a7c0 0x7f346c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.466+0000 7f347ffff640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0047c0 0x7f346c004c40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f346800c210 tx=0x7f346c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f52eddffdb2c488 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346802cae0 con 0x7f346c0047c0 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346802cc40 con 0x7f346c0047c0 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f347f7fe640 1 -- 172.21.15.70:0/184658260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680058d0 con 0x7f346c0047c0 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f3485c21640 1 -- 172.21.15.70:0/184658260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0047c0 msgr2=0x7f346c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f3485c21640 1 --2- 172.21.15.70:0/184658260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0047c0 0x7f346c004c40 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f346800c210 tx=0x7f346c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f3485420640 1 -- 172.21.15.70:0/184658260 reap_dead start 2023-08-25T09:43:31.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f3485c21640 1 -- 172.21.15.70:0/184658260 shutdown_connections 2023-08-25T09:43:31.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.467+0000 7f3485c21640 1 -- 172.21.15.70:0/184658260 >> 172.21.15.70:0/184658260 conn(0x7f34800affd0 msgr2=0x7f34800b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:31.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.468+0000 7f3485c21640 1 -- 172.21.15.70:0/184658260 shutdown_connections 2023-08-25T09:43:31.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.468+0000 7f3485c21640 1 -- 172.21.15.70:0/184658260 wait complete. 2023-08-25T09:43:31.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.469+0000 7f3485c21640 1 Processor -- start 2023-08-25T09:43:31.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.469+0000 7f3485c21640 1 -- start start 2023-08-25T09:43:31.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.470+0000 7f3485c21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f346c0047c0 0x7f34800c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.470+0000 7f3485c21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 0x7f34801476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.470+0000 7f3485c21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480147be0 0x7f34801456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.470+0000 7f3485c21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480070000 con 0x7f3480147220 2023-08-25T09:43:31.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.470+0000 7f3485c21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34800c5ab0 con 0x7f3480147be0 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.470+0000 7f3485c21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34801453f0 con 0x7f346c0047c0 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 0x7f34801476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 0x7f34801476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60418/0 (socket says 172.21.15.70:60418) 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 -- 172.21.15.70:0/3730848511 learned_addr learned my addr 172.21.15.70:0/3730848511 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 -- 172.21.15.70:0/3730848511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f346c0047c0 msgr2=0x7f34800c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f346c0047c0 0x7f34800c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 -- 172.21.15.70:0/3730848511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480147be0 msgr2=0x7f34801456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480147be0 0x7f34801456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 -- 172.21.15.70:0/3730848511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480070420 con 0x7f3480147220 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.471+0000 7f347ffff640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 0x7f34801476a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f346800c210 tx=0x7f348015e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.472+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468039070 con 0x7f3480147220 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.472+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3468004bf0 con 0x7f3480147220 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.472+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346803c040 con 0x7f3480147220 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.472+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3480148060 con 0x7f3480147220 2023-08-25T09:43:31.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.472+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348006f660 con 0x7f3480147220 2023-08-25T09:43:31.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.473+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3480070270 con 0x7f3480147220 2023-08-25T09:43:31.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.474+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3468032070 con 0x7f3480147220 2023-08-25T09:43:31.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.475+0000 7f347d7fa640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.475+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f34680a9bb0 con 0x7f3480147220 2023-08-25T09:43:31.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.475+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7f3460072480 con 0x7f3480147220 2023-08-25T09:43:31.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.475+0000 7f3484c1f640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.475+0000 7f3484c1f640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3470009f00 tx=0x7f3470009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:31.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.479+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f34680b0080 con 0x7f3480147220 2023-08-25T09:43:31.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.669+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3480066fb0 con 0x7f3480147220 2023-08-25T09:43:31.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.673+0000 7f347d7fa640 1 -- 172.21.15.70:0/3730848511 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18757 (secure 0 0 0) 0x7f3468075340 con 0x7f3480147220 2023-08-25T09:43:31.677 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:31.677 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":79,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:42:46.536583+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","full","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":1}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.38","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:31.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.675+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 msgr2=0x7f346006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.675+0000 7f3485c21640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3470009f00 tx=0x7f3470009750 comp rx=0 tx=0).stop 2023-08-25T09:43:31.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.675+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 msgr2=0x7f34801476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.675+0000 7f3485c21640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 0x7f34801476a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f346800c210 tx=0x7f348015e730 comp rx=0 tx=0).stop 2023-08-25T09:43:31.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 shutdown_connections 2023-08-25T09:43:31.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480147be0 0x7f34801456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3480147220 0x7f34801476a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 --2- 172.21.15.70:0/3730848511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f346c0047c0 0x7f34800c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 >> 172.21.15.70:0/3730848511 conn(0x7f34800affd0 msgr2=0x7f34800ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:31.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 shutdown_connections 2023-08-25T09:43:31.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.676+0000 7f3485c21640 1 -- 172.21.15.70:0/3730848511 wait complete. 2023-08-25T09:43:31.699 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:42:22.389960+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:42:23.780164+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:43:31.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:43:31.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.925+0000 7ff76de83640 1 Processor -- start 2023-08-25T09:43:31.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.925+0000 7ff76de83640 1 -- start start 2023-08-25T09:43:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.926+0000 7ff76de83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff768069d30 0x7ff7680ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff76de83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680cd4e0 con 0x7ff76806c510 2023-08-25T09:43:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff76de83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7680cc550 con 0x7ff76806a110 2023-08-25T09:43:31.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff76de83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7680ccb10 con 0x7ff768069d30 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff767fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff76806c510 0x7ff76806c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48054/0 (socket says 172.21.15.70:48054) 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff767fff640 1 -- 172.21.15.70:0/1419909883 learned_addr learned my addr 172.21.15.70:0/1419909883 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff7677fe640 1 --2- 172.21.15.70:0/1419909883 >> v2:172.21.15.155:3300/0 conn(0x7ff768069d30 0x7ff7680ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574496357 0 0) 0x7ff760001490 con 0x7ff76806c510 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.927+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff754003520 con 0x7ff76806c510 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 512348887 0 0) 0x7ff754003520 con 0x7ff76806c510 2023-08-25T09:43:31.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 >> v2:172.21.15.155:3300/0 conn(0x7ff768069d30 msgr2=0x7ff7680ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 --2- 172.21.15.70:0/1419909883 >> v2:172.21.15.155:3300/0 conn(0x7ff768069d30 0x7ff7680ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 >> v1:172.21.15.155:6789/0 conn(0x7ff76806a110 legacy=0x7ff7680ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680ade40 con 0x7ff76806c510 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff760001b80 con 0x7ff76806c510 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 >> v1:172.21.15.70:6789/0 conn(0x7ff76806c510 legacy=0x7ff76806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff754003970 0x7ff754003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7540047c0 0x7ff754004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff75400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680ccb10 con 0x7ff754005180 2023-08-25T09:43:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75400c120 con 0x7ff754003970 2023-08-25T09:43:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75400c2e0 con 0x7ff7540047c0 2023-08-25T09:43:31.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.928+0000 7ff767fff640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff75400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.929+0000 7ff7677fe640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff754003970 0x7ff754003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.929+0000 7ff767fff640 1 -- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7540047c0 msgr2=0x7ff754004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.929+0000 7ff767fff640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7540047c0 0x7ff754004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.929+0000 7ff767fff640 1 -- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff754003970 msgr2=0x7ff754003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.929+0000 7ff767fff640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff754003970 0x7ff754003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.929+0000 7ff767fff640 1 -- 172.21.15.70:0/1419909883 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754003520 con 0x7ff754005180 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.930+0000 7ff767fff640 1 -- 172.21.15.70:0/1419909883 reap_dead start 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.930+0000 7ff767fff640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff75400b980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff76000a810 tx=0x7ff7540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240ccb5677c0b2c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:31.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.930+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760034630 con 0x7ff754005180 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.930+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76002f070 con 0x7ff754005180 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.930+0000 7ff766ffd640 1 -- 172.21.15.70:0/1419909883 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7600349f0 con 0x7ff754005180 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 -- 172.21.15.70:0/1419909883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 msgr2=0x7ff75400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff75400b980 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff76000a810 tx=0x7ff7540057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 -- 172.21.15.70:0/1419909883 shutdown_connections 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 --2- 172.21.15.70:0/1419909883 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff75400b980 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 -- 172.21.15.70:0/1419909883 >> 172.21.15.70:0/1419909883 conn(0x7ff7680cb0d0 msgr2=0x7ff7680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 -- 172.21.15.70:0/1419909883 shutdown_connections 2023-08-25T09:43:31.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.931+0000 7ff76de83640 1 -- 172.21.15.70:0/1419909883 wait complete. 2023-08-25T09:43:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.933+0000 7ff76de83640 1 Processor -- start 2023-08-25T09:43:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.933+0000 7ff76de83640 1 -- start start 2023-08-25T09:43:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff76de83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff768070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff76de83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff768150760 0x7ff7680c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff76de83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7680b3290 0x7ff7680b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff76de83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680c3bf0 con 0x7ff754005180 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff76de83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680b2e00 con 0x7ff768150760 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff76de83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7680c43a0 con 0x7ff7680b3290 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff7677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff768070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff7677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff768070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60438/0 (socket says 172.21.15.70:60438) 2023-08-25T09:43:31.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.934+0000 7ff7677fe640 1 -- 172.21.15.70:0/2359000054 learned_addr learned my addr 172.21.15.70:0/2359000054 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff7677fe640 1 -- 172.21.15.70:0/2359000054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7680b3290 msgr2=0x7ff7680b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff7677fe640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7680b3290 0x7ff7680b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff7677fe640 1 -- 172.21.15.70:0/2359000054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff768150760 msgr2=0x7ff7680c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff7677fe640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff768150760 0x7ff7680c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff7677fe640 1 -- 172.21.15.70:0/2359000054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7680c3de0 con 0x7ff754005180 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff7677fe640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff768070020 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff758010860 tx=0x7ff758010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:31.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758009610 con 0x7ff754005180 2023-08-25T09:43:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75800eb00 con 0x7ff754005180 2023-08-25T09:43:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75800ee90 con 0x7ff754005180 2023-08-25T09:43:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7680c40c0 con 0x7ff754005180 2023-08-25T09:43:31.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.935+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff768070560 con 0x7ff754005180 2023-08-25T09:43:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.937+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7680b28f0 con 0x7ff754005180 2023-08-25T09:43:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.937+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff75801c400 con 0x7ff754005180 2023-08-25T09:43:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.938+0000 7ff764ff9640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff74806a080 0x7ff74806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.938+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ff758088c60 con 0x7ff754005180 2023-08-25T09:43:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.938+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=80}) v3 -- 0x7ff7480724b0 con 0x7ff754005180 2023-08-25T09:43:31.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.939+0000 7ff75ffff640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff74806a080 0x7ff74806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:31.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.939+0000 7ff75ffff640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff74806a080 0x7ff74806c540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff768069be0 tx=0x7ff7500079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:31.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:31.942+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff7680b28f0 con 0x7ff754005180 2023-08-25T09:43:32.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.130+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7680b28f0 con 0x7ff754005180 2023-08-25T09:43:32.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.131+0000 7ff764ff9640 1 -- 172.21.15.70:0/2359000054 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7680b28f0 con 0x7ff754005180 2023-08-25T09:43:32.134 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:43:32.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff74806a080 msgr2=0x7ff74806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff74806a080 0x7ff74806c540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff768069be0 tx=0x7ff7500079b0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 msgr2=0x7ff768070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff768070020 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff758010860 tx=0x7ff758010d20 comp rx=0 tx=0).stop 2023-08-25T09:43:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 shutdown_connections 2023-08-25T09:43:32.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff74806a080 0x7ff74806c540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff7680b3290 0x7ff7680b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff768150760 0x7ff7680c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 --2- 172.21.15.70:0/2359000054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff754005180 0x7ff768070020 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.133+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 >> 172.21.15.70:0/2359000054 conn(0x7ff7680cb0d0 msgr2=0x7ff76806ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.134+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 shutdown_connections 2023-08-25T09:43:32.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.134+0000 7ff76de83640 1 -- 172.21.15.70:0/2359000054 wait complete. 2023-08-25T09:43:32.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:43:32.309 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:43:32.306+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-08-25T09:43:32.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.374+0000 7f596b6fa640 1 Processor -- start 2023-08-25T09:43:32.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.374+0000 7f596b6fa640 1 -- start start 2023-08-25T09:43:32.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.375+0000 7f596b6fa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f596415e210 0x7f5964069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f596b6fa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640ac880 con 0x7f5964069850 2023-08-25T09:43:32.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f596b6fa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59640ad400 con 0x7f59640cca20 2023-08-25T09:43:32.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f596b6fa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596415e650 con 0x7f596415e210 2023-08-25T09:43:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f5969496640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5964069850 0x7f59640c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48064/0 (socket says 172.21.15.70:48064) 2023-08-25T09:43:32.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f5969496640 1 -- 172.21.15.70:0/3279595121 learned_addr learned my addr 172.21.15.70:0/3279595121 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f5968c95640 1 --2- 172.21.15.70:0/3279595121 >> v2:172.21.15.155:3300/0 conn(0x7f596415e210 0x7f5964069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140551735 0 0) 0x7f59640ac880 con 0x7f5964069850 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.376+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5950003520 con 0x7f5964069850 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1936774953 0 0) 0x7f5950003520 con 0x7f5964069850 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 >> v2:172.21.15.155:3300/0 conn(0x7f596415e210 msgr2=0x7f5964069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 --2- 172.21.15.70:0/3279595121 >> v2:172.21.15.155:3300/0 conn(0x7f596415e210 0x7f5964069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 >> v1:172.21.15.155:6789/0 conn(0x7f59640cca20 legacy=0x7f59640cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640ad890 con 0x7f5964069850 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5960001670 con 0x7f5964069850 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 >> v1:172.21.15.70:6789/0 conn(0x7f5964069850 legacy=0x7f59640c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59500039c0 0x7f5950003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59500047c0 0x7f5950004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500051d0 0x7f595000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596415e650 con 0x7f59500047c0 2023-08-25T09:43:32.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595000c0e0 con 0x7f59500051d0 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595000c2a0 con 0x7f59500039c0 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f5968c95640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59500047c0 0x7f5950004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.377+0000 7f5969496640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59500039c0 0x7f5950003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5969c97640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500051d0 0x7f595000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5968c95640 1 -- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59500039c0 msgr2=0x7f5950003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5968c95640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59500039c0 0x7f5950003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5968c95640 1 -- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500051d0 msgr2=0x7f595000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5968c95640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500051d0 0x7f595000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5968c95640 1 -- 172.21.15.70:0/3279595121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950003520 con 0x7f59500047c0 2023-08-25T09:43:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5969c97640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500051d0 0x7f595000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:32.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.378+0000 7f5969c97640 1 -- 172.21.15.70:0/3279595121 reap_dead start 2023-08-25T09:43:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.379+0000 7f5968c95640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59500047c0 0x7f5950004c40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f595400ba30 tx=0x7f59500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c938f68250c07cd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.379+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954032640 con 0x7f59500047c0 2023-08-25T09:43:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.379+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540327a0 con 0x7f59500047c0 2023-08-25T09:43:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.379+0000 7f595bfff640 1 -- 172.21.15.70:0/3279595121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954032b30 con 0x7f59500047c0 2023-08-25T09:43:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 -- 172.21.15.70:0/3279595121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59500047c0 msgr2=0x7f5950004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59500047c0 0x7f5950004c40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f595400ba30 tx=0x7f59500057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 -- 172.21.15.70:0/3279595121 shutdown_connections 2023-08-25T09:43:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 --2- 172.21.15.70:0/3279595121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59500047c0 0x7f5950004c40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 -- 172.21.15.70:0/3279595121 >> 172.21.15.70:0/3279595121 conn(0x7f59640cb0d0 msgr2=0x7f59640cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 -- 172.21.15.70:0/3279595121 shutdown_connections 2023-08-25T09:43:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.380+0000 7f596b6fa640 1 -- 172.21.15.70:0/3279595121 wait complete. 2023-08-25T09:43:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.382+0000 7f596b6fa640 1 Processor -- start 2023-08-25T09:43:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.382+0000 7f596b6fa640 1 -- start start 2023-08-25T09:43:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f596b6fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500047c0 0x7f59641492f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f596b6fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59640c6cf0 0x7f59640b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f596b6fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 0x7f59641f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f596b6fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5964149830 con 0x7f59640c5690 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f596b6fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640c5c50 con 0x7f59500047c0 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f596b6fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640c5f30 con 0x7f59640c6cf0 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 0x7f59641f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 0x7f59641f2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60464/0 (socket says 172.21.15.70:60464) 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 -- 172.21.15.70:0/11167178 learned_addr learned my addr 172.21.15.70:0/11167178 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5968c95640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59640c6cf0 0x7f59640b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969496640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500047c0 0x7f59641492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 -- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59640c6cf0 msgr2=0x7f59640b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59640c6cf0 0x7f59640b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 -- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500047c0 msgr2=0x7f59641492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500047c0 0x7f59641492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.383+0000 7f5969c97640 1 -- 172.21.15.70:0/11167178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640c6210 con 0x7f59640c5690 2023-08-25T09:43:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f5969496640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500047c0 0x7f59641492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f5969c97640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 0x7f59641f2b30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f59640b1240 tx=0x7f595c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c006d50 con 0x7f59640c5690 2023-08-25T09:43:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c004950 con 0x7f59640c5690 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c004d00 con 0x7f59640c5690 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640c64f0 con 0x7f59640c5690 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.384+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59640b3f20 con 0x7f59640c5690 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.385+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59640c6a10 con 0x7f59640c5690 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.386+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f595c005030 con 0x7f59640c5690 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.387+0000 7f5959ffb640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594806a050 0x7f594806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.387+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f595c087900 con 0x7f59640c5690 2023-08-25T09:43:32.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.387+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=81}) v3 -- 0x7f5948072480 con 0x7f59640c5690 2023-08-25T09:43:32.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.388+0000 7f5969496640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594806a050 0x7f594806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.388+0000 7f5969496640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594806a050 0x7f594806c510 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5960001050 tx=0x7f5960001190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:32.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.392+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59640c6a10 con 0x7f59640c5690 2023-08-25T09:43:32.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.579+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f59640c6a10 con 0x7f59640c5690 2023-08-25T09:43:32.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.580+0000 7f5959ffb640 1 -- 172.21.15.70:0/11167178 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f59640c6a10 con 0x7f59640c5690 2023-08-25T09:43:32.583 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:43:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594806a050 msgr2=0x7f594806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594806a050 0x7f594806c510 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5960001050 tx=0x7f5960001190 comp rx=0 tx=0).stop 2023-08-25T09:43:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 msgr2=0x7f59641f2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 0x7f59641f2b30 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f59640b1240 tx=0x7f595c00d9d0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 shutdown_connections 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f594806a050 0x7f594806c510 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59640c5690 0x7f59641f2b30 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59640c6cf0 0x7f59640b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 --2- 172.21.15.70:0/11167178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59500047c0 0x7f59641492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 >> 172.21.15.70:0/11167178 conn(0x7f59640cb0d0 msgr2=0x7f596415e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 shutdown_connections 2023-08-25T09:43:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.583+0000 7f596b6fa640 1 -- 172.21.15.70:0/11167178 wait complete. 2023-08-25T09:43:32.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:43:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.819+0000 7f7a14522640 1 Processor -- start 2023-08-25T09:43:32.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.819+0000 7f7a14522640 1 -- start start 2023-08-25T09:43:32.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a14522640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 0x7f7a0c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a14522640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7a0c160670 0x7f7a0c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a14522640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c071c10 con 0x7f7a0c069850 2023-08-25T09:43:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a14522640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c071000 con 0x7f7a0c160670 2023-08-25T09:43:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a14522640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0712e0 con 0x7f7a0c160290 2023-08-25T09:43:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a12abf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7a0c069850 0x7f7a0c0cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48080/0 (socket says 172.21.15.70:48080) 2023-08-25T09:43:32.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.821+0000 7f7a12abf640 1 -- 172.21.15.70:0/2774089815 learned_addr learned my addr 172.21.15.70:0/2774089815 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:32.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a122be640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 0x7f7a0c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775826196 0 0) 0x7f7a0c071c10 con 0x7f7a0c069850 2023-08-25T09:43:32.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79f4003520 con 0x7f7a0c069850 2023-08-25T09:43:32.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a11abd640 1 --2- 172.21.15.70:0/2774089815 >> v2:172.21.15.155:3300/0 conn(0x7f7a0c160670 0x7f7a0c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189988624 0 0) 0x7f79f4003520 con 0x7f7a0c069850 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 msgr2=0x7f7a0c0cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 0x7f7a0c0cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 >> v2:172.21.15.155:3300/0 conn(0x7f7a0c160670 msgr2=0x7f7a0c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 --2- 172.21.15.70:0/2774089815 >> v2:172.21.15.155:3300/0 conn(0x7f7a0c160670 0x7f7a0c0cd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0715c0 con 0x7f7a0c069850 2023-08-25T09:43:32.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7a040019f0 con 0x7f7a0c069850 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 >> v1:172.21.15.70:6789/0 conn(0x7f7a0c069850 legacy=0x7f7a0c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4003970 0x7f79f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f40047c0 0x7f79f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 0x7f79f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c071000 con 0x7f79f40047c0 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0712e0 con 0x7f79f4003970 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.822+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f400b980 con 0x7f7a0c160290 2023-08-25T09:43:32.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a11abd640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f40047c0 0x7f79f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a122be640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4003970 0x7f79f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a11abd640 1 -- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 msgr2=0x7f79f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a11abd640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7a0c160290 0x7f79f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a11abd640 1 -- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4003970 msgr2=0x7f79f4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a11abd640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f79f4003970 0x7f79f4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.823+0000 7f7a11abd640 1 -- 172.21.15.70:0/2774089815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f4003520 con 0x7f79f40047c0 2023-08-25T09:43:32.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.824+0000 7f7a11abd640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f40047c0 0x7f79f4004c40 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000ba30 tx=0x7f79f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccc017937baff68 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:32.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.824+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00032680 con 0x7f79f40047c0 2023-08-25T09:43:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.824+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a000327e0 con 0x7f79f40047c0 2023-08-25T09:43:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.824+0000 7f7a112bc640 1 -- 172.21.15.70:0/2774089815 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00032b70 con 0x7f79f40047c0 2023-08-25T09:43:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a14522640 1 -- 172.21.15.70:0/2774089815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f40047c0 msgr2=0x7f79f4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a14522640 1 --2- 172.21.15.70:0/2774089815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f79f40047c0 0x7f79f4004c40 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000ba30 tx=0x7f79f40057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a12abf640 1 -- 172.21.15.70:0/2774089815 reap_dead start 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a14522640 1 -- 172.21.15.70:0/2774089815 shutdown_connections 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a14522640 1 -- 172.21.15.70:0/2774089815 >> 172.21.15.70:0/2774089815 conn(0x7f7a0c06ba60 msgr2=0x7f7a0c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a14522640 1 -- 172.21.15.70:0/2774089815 shutdown_connections 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.825+0000 7f7a14522640 1 -- 172.21.15.70:0/2774089815 wait complete. 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.827+0000 7f7a14522640 1 Processor -- start 2023-08-25T09:43:32.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.827+0000 7f7a14522640 1 -- start start 2023-08-25T09:43:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a14522640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79f40047c0 0x7f7a0c148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a14522640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 0x7f7a0c1f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a14522640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a0c0ae540 0x7f7a0c1494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a14522640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c149260 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a14522640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1f2e40 con 0x7f7a0c0ae540 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a14522640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1f3120 con 0x7f79f40047c0 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.828+0000 7f7a11abd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 0x7f7a0c1f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 0x7f7a0c1f24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60488/0 (socket says 172.21.15.70:60488) 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 -- 172.21.15.70:0/845617336 learned_addr learned my addr 172.21.15.70:0/845617336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a122be640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79f40047c0 0x7f7a0c148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 -- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79f40047c0 msgr2=0x7f7a0c148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79f40047c0 0x7f7a0c148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 -- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a0c0ae540 msgr2=0x7f7a0c1494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a0c0ae540 0x7f7a0c1494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:32.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 -- 172.21.15.70:0/845617336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c1f3400 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.829+0000 7f7a11abd640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 0x7f7a0c1f24d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7a00035040 tx=0x7f7a00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.830+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00030810 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.830+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a00030970 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.830+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a00030ce0 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.830+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c0add40 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.830+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c1f2a10 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.831+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c1f2ba0 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.832+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7a00043070 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.833+0000 7f7a0affd640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79f806a080 0x7f79f806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:32.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.833+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f7a000aaa70 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.833+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=81}) v3 -- 0x7f79f80724b0 con 0x7f7a0c14c1b0 2023-08-25T09:43:32.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.833+0000 7f7a122be640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79f806a080 0x7f79f806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:32.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.834+0000 7f7a122be640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79f806a080 0x7f79f806c540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009830 tx=0x7f79fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:32.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:32.837+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7a0004a060 con 0x7f7a0c14c1b0 2023-08-25T09:43:33.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.026+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a0c066fb0 con 0x7f7a0c14c1b0 2023-08-25T09:43:33.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.027+0000 7f7a0affd640 1 -- 172.21.15.70:0/845617336 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a00036020 con 0x7f7a0c14c1b0 2023-08-25T09:43:33.030 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:43:33.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79f806a080 msgr2=0x7f79f806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79f806a080 0x7f79f806c540 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009830 tx=0x7f79fc008040 comp rx=0 tx=0).stop 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 msgr2=0x7f7a0c1f24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 0x7f7a0c1f24d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7a00035040 tx=0x7f7a00004730 comp rx=0 tx=0).stop 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 shutdown_connections 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f79f806a080 0x7f79f806c540 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7a0c0ae540 0x7f7a0c1494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7a0c14c1b0 0x7f7a0c1f24d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 --2- 172.21.15.70:0/845617336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f79f40047c0 0x7f7a0c148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 >> 172.21.15.70:0/845617336 conn(0x7f7a0c06ba60 msgr2=0x7f7a0c0ad070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:33.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 shutdown_connections 2023-08-25T09:43:33.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.030+0000 7f7a14522640 1 -- 172.21.15.70:0/845617336 wait complete. 2023-08-25T09:43:33.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:43:33.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.259+0000 7ff930ebf640 1 Processor -- start 2023-08-25T09:43:33.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.259+0000 7ff930ebf640 1 -- start start 2023-08-25T09:43:33.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff930ebf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff92c160290 0x7ff92c0cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff930ebf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160670 0x7ff92c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff930ebf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff92c069850 0x7ff92c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff930ebf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c071c10 con 0x7ff92c160290 2023-08-25T09:43:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff930ebf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c071000 con 0x7ff92c069850 2023-08-25T09:43:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff930ebf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0712e0 con 0x7ff92c160670 2023-08-25T09:43:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff92a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff92c160290 0x7ff92c0cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff92a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff92c160290 0x7ff92c0cd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60496/0 (socket says 172.21.15.70:60496) 2023-08-25T09:43:33.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.261+0000 7ff92a59c640 1 -- 172.21.15.70:0/301530034 learned_addr learned my addr 172.21.15.70:0/301530034 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.262+0000 7ff92a59c640 1 -- 172.21.15.70:0/301530034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160670 msgr2=0x7ff92c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.262+0000 7ff92a59c640 1 --2- 172.21.15.70:0/301530034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160670 0x7ff92c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.262+0000 7ff92a59c640 1 -- 172.21.15.70:0/301530034 >> v2:172.21.15.155:3300/0 conn(0x7ff92c069850 msgr2=0x7ff92c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.262+0000 7ff92a59c640 1 --2- 172.21.15.70:0/301530034 >> v2:172.21.15.155:3300/0 conn(0x7ff92c069850 0x7ff92c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.262+0000 7ff92a59c640 1 -- 172.21.15.70:0/301530034 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c0715c0 con 0x7ff92c160290 2023-08-25T09:43:33.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.263+0000 7ff92a59c640 1 --2- 172.21.15.70:0/301530034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c160290 0x7ff92c0cd360 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0afcd0 tx=0x7ff91c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2728b1b5a4b3ea6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:33.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.263+0000 7ff92959a640 1 -- 172.21.15.70:0/301530034 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c031860 con 0x7ff92c160290 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.263+0000 7ff92959a640 1 -- 172.21.15.70:0/301530034 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c0319c0 con 0x7ff92c160290 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.263+0000 7ff92959a640 1 -- 172.21.15.70:0/301530034 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c031d50 con 0x7ff92c160290 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 -- 172.21.15.70:0/301530034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c160290 msgr2=0x7ff92c0cd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 --2- 172.21.15.70:0/301530034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c160290 0x7ff92c0cd360 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0afcd0 tx=0x7ff91c02b900 comp rx=0 tx=0).stop 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 -- 172.21.15.70:0/301530034 shutdown_connections 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 --2- 172.21.15.70:0/301530034 >> v2:172.21.15.155:3300/0 conn(0x7ff92c069850 0x7ff92c0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 --2- 172.21.15.70:0/301530034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160670 0x7ff92c0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 --2- 172.21.15.70:0/301530034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c160290 0x7ff92c0cd360 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 -- 172.21.15.70:0/301530034 >> 172.21.15.70:0/301530034 conn(0x7ff92c06ba60 msgr2=0x7ff92c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 -- 172.21.15.70:0/301530034 shutdown_connections 2023-08-25T09:43:33.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.264+0000 7ff930ebf640 1 -- 172.21.15.70:0/301530034 wait complete. 2023-08-25T09:43:33.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.266+0000 7ff930ebf640 1 Processor -- start 2023-08-25T09:43:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.266+0000 7ff930ebf640 1 -- start start 2023-08-25T09:43:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff930ebf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 0x7ff92c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff930ebf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160290 0x7ff92c14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff930ebf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c160670 0x7ff92c0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff930ebf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c1f56f0 con 0x7ff92c069850 2023-08-25T09:43:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff930ebf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c14bec0 con 0x7ff92c160670 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff930ebf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c14b2b0 con 0x7ff92c160290 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff92a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 0x7ff92c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff92a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 0x7ff92c1f51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60506/0 (socket says 172.21.15.70:60506) 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.267+0000 7ff92a59c640 1 -- 172.21.15.70:0/2106335031 learned_addr learned my addr 172.21.15.70:0/2106335031 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff92a59c640 1 -- 172.21.15.70:0/2106335031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160290 msgr2=0x7ff92c14a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff92a59c640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160290 0x7ff92c14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff92a59c640 1 -- 172.21.15.70:0/2106335031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c160670 msgr2=0x7ff92c0c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff92a59c640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c160670 0x7ff92c0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff92a59c640 1 -- 172.21.15.70:0/2106335031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0094d0 con 0x7ff92c069850 2023-08-25T09:43:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff92a59c640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 0x7ff92c1f51b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff91c000c00 tx=0x7ff91c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c03ad10 con 0x7ff92c069850 2023-08-25T09:43:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c03ae70 con 0x7ff92c069850 2023-08-25T09:43:33.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c044610 con 0x7ff92c069850 2023-08-25T09:43:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c14b590 con 0x7ff92c069850 2023-08-25T09:43:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.268+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c14bb50 con 0x7ff92c069850 2023-08-25T09:43:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.269+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c066fb0 con 0x7ff92c069850 2023-08-25T09:43:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.271+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff91c044770 con 0x7ff92c069850 2023-08-25T09:43:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.272+0000 7ff91affd640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff90c06a050 0x7ff90c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.272+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ff91c0aae40 con 0x7ff92c069850 2023-08-25T09:43:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.272+0000 7ff929d9b640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff90c06a050 0x7ff90c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.272+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=81}) v3 -- 0x7ff90c072480 con 0x7ff92c069850 2023-08-25T09:43:33.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.273+0000 7ff929d9b640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff90c06a050 0x7ff90c06c510 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff92c15cd70 tx=0x7ff920006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.276+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff91c0764e0 con 0x7ff92c069850 2023-08-25T09:43:33.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.473+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff92c0b68f0 con 0x7ff92c069850 2023-08-25T09:43:33.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.474+0000 7ff91affd640 1 -- 172.21.15.70:0/2106335031 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff91c079b00 con 0x7ff92c069850 2023-08-25T09:43:33.477 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:43:33.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.477+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff90c06a050 msgr2=0x7ff90c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.477+0000 7ff930ebf640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff90c06a050 0x7ff90c06c510 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff92c15cd70 tx=0x7ff920006bf0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.477+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 msgr2=0x7ff92c1f51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.477+0000 7ff930ebf640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 0x7ff92c1f51b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff91c000c00 tx=0x7ff91c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 shutdown_connections 2023-08-25T09:43:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff90c06a050 0x7ff90c06c510 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff92c160670 0x7ff92c0c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff92c160290 0x7ff92c14a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 --2- 172.21.15.70:0/2106335031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff92c069850 0x7ff92c1f51b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 >> 172.21.15.70:0/2106335031 conn(0x7ff92c06ba60 msgr2=0x7ff92c0cc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 shutdown_connections 2023-08-25T09:43:33.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.478+0000 7ff930ebf640 1 -- 172.21.15.70:0/2106335031 wait complete. 2023-08-25T09:43:33.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:43:33.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.711+0000 7f29c6c4d640 1 Processor -- start 2023-08-25T09:43:33.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.711+0000 7f29c6c4d640 1 -- start start 2023-08-25T09:43:33.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.712+0000 7f29c6c4d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f29c015acd0 0x7f29c015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.712+0000 7f29c6c4d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29c00b4400 0x7f29c00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.712+0000 7f29c6c4d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c00b12b0 con 0x7f29c00ca780 2023-08-25T09:43:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.712+0000 7f29c6c4d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00b47e0 con 0x7f29c00b4400 2023-08-25T09:43:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.712+0000 7f29c6c4d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c006c0e0 con 0x7f29c015acd0 2023-08-25T09:43:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29c51ea640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29c00b4400 0x7f29c00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29bffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f29c00ca780 0x7f29c00b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48086/0 (socket says 172.21.15.70:48086) 2023-08-25T09:43:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29bffff640 1 -- 172.21.15.70:0/2493573317 learned_addr learned my addr 172.21.15.70:0/2493573317 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29c49e9640 1 --2- 172.21.15.70:0/2493573317 >> v2:172.21.15.155:3300/0 conn(0x7f29c015acd0 0x7f29c015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29c51ea640 1 -- 172.21.15.70:0/2493573317 >> v2:172.21.15.155:3300/0 conn(0x7f29c015acd0 msgr2=0x7f29c015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:33.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29c51ea640 1 --2- 172.21.15.70:0/2493573317 >> v2:172.21.15.155:3300/0 conn(0x7f29c015acd0 0x7f29c015b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29c51ea640 1 -- 172.21.15.70:0/2493573317 >> v1:172.21.15.70:6789/0 conn(0x7f29c00ca780 legacy=0x7f29c00b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.713+0000 7f29c51ea640 1 -- 172.21.15.70:0/2493573317 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c006ae20 con 0x7f29c00b4400 2023-08-25T09:43:33.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.714+0000 7f29c51ea640 1 --2- 172.21.15.70:0/2493573317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00b4400 0x7f29c00cac30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f29c00714b0 tx=0x7f29b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2e0f78cd3f89e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:33.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.715+0000 7f29bf7fe640 1 -- 172.21.15.70:0/2493573317 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b002e070 con 0x7f29c00b4400 2023-08-25T09:43:33.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.715+0000 7f29bf7fe640 1 -- 172.21.15.70:0/2493573317 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00315a0 con 0x7f29c00b4400 2023-08-25T09:43:33.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.715+0000 7f29bf7fe640 1 -- 172.21.15.70:0/2493573317 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0031930 con 0x7f29c00b4400 2023-08-25T09:43:33.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.715+0000 7f29c6c4d640 1 -- 172.21.15.70:0/2493573317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00b4400 msgr2=0x7f29c00cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.715+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/2493573317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00b4400 0x7f29c00cac30 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f29c00714b0 tx=0x7f29b002b840 comp rx=0 tx=0).stop 2023-08-25T09:43:33.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.716+0000 7f29c6c4d640 1 -- 172.21.15.70:0/2493573317 shutdown_connections 2023-08-25T09:43:33.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.716+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/2493573317 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00b4400 0x7f29c00cac30 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.716+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/2493573317 >> v2:172.21.15.155:3300/0 conn(0x7f29c015acd0 0x7f29c015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.716+0000 7f29c6c4d640 1 -- 172.21.15.70:0/2493573317 >> 172.21.15.70:0/2493573317 conn(0x7f29c00ce760 msgr2=0x7f29c00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:33.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.716+0000 7f29c6c4d640 1 -- 172.21.15.70:0/2493573317 shutdown_connections 2023-08-25T09:43:33.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.716+0000 7f29c6c4d640 1 -- 172.21.15.70:0/2493573317 wait complete. 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.717+0000 7f29c6c4d640 1 Processor -- start 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.717+0000 7f29c6c4d640 1 -- start start 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.718+0000 7f29c6c4d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c00b4400 0x7f29c0143d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.718+0000 7f29c6c4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 0x7f29c00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.718+0000 7f29c6c4d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c015acd0 0x7f29c014e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.718+0000 7f29c6c4d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00cda90 con 0x7f29c00ca780 2023-08-25T09:43:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.718+0000 7f29c6c4d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01442d0 con 0x7f29c00b4400 2023-08-25T09:43:33.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.718+0000 7f29c6c4d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00cd1f0 con 0x7f29c015acd0 2023-08-25T09:43:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 0x7f29c00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 0x7f29c00cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60522/0 (socket says 172.21.15.70:60522) 2023-08-25T09:43:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 -- 172.21.15.70:0/734239175 learned_addr learned my addr 172.21.15.70:0/734239175 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29c49e9640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c00b4400 0x7f29c0143d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29c51ea640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c015acd0 0x7f29c014e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 -- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c015acd0 msgr2=0x7f29c014e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c015acd0 0x7f29c014e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 -- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c00b4400 msgr2=0x7f29c0143d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c00b4400 0x7f29c0143d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 -- 172.21.15.70:0/734239175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0009400 con 0x7f29c00ca780 2023-08-25T09:43:33.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.719+0000 7f29bffff640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 0x7f29c00cc7c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c4a0 tx=0x7f29a800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:33.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.720+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8005ce0 con 0x7f29c00ca780 2023-08-25T09:43:33.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.720+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8005e40 con 0x7f29c00ca780 2023-08-25T09:43:33.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.720+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8007270 con 0x7f29c00ca780 2023-08-25T09:43:33.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.720+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00cd4d0 con 0x7f29c00ca780 2023-08-25T09:43:33.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.720+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00cad40 con 0x7f29c00ca780 2023-08-25T09:43:33.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.721+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c004e420 con 0x7f29c00ca780 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.722+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29a8007490 con 0x7f29c00ca780 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.723+0000 7f29bd7fa640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29a006a050 0x7f29a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.723+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f29a8087b40 con 0x7f29c00ca780 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.723+0000 7f29c49e9640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29a006a050 0x7f29a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.723+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=81}) v3 -- 0x7f29a0072480 con 0x7f29c00ca780 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.724+0000 7f29c49e9640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29a006a050 0x7f29a006c510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f29b8008ea0 tx=0x7f29b80083f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.727+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29a8017060 con 0x7f29c00ca780 2023-08-25T09:43:33.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.928+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f29c006b910 con 0x7f29c00ca780 2023-08-25T09:43:33.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.929+0000 7f29bd7fa640 1 -- 172.21.15.70:0/734239175 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) v1 ==== 80+0+0 (secure 0 0 0) 0x7f29a8053200 con 0x7f29c00ca780 2023-08-25T09:43:33.932 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:43:33.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29a006a050 msgr2=0x7f29a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29a006a050 0x7f29a006c510 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f29b8008ea0 tx=0x7f29b80083f0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 msgr2=0x7f29c00cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 0x7f29c00cc7c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f29a800c4a0 tx=0x7f29a800c960 comp rx=0 tx=0).stop 2023-08-25T09:43:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 shutdown_connections 2023-08-25T09:43:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29a006a050 0x7f29a006c510 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c015acd0 0x7f29c014e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c00ca780 0x7f29c00cc7c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 --2- 172.21.15.70:0/734239175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c00b4400 0x7f29c0143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:33.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.932+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 >> 172.21.15.70:0/734239175 conn(0x7f29c00ce760 msgr2=0x7f29c0152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:33.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.933+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 shutdown_connections 2023-08-25T09:43:33.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:33.933+0000 7f29c6c4d640 1 -- 172.21.15.70:0/734239175 wait complete. 2023-08-25T09:43:33.954 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:43:33.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:43:33.955 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:43:34.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[89302]: fuse finished with error 0 and tester_r 0 2023-08-25T09:43:34.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:43:34.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:43:34.061 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:43:34.127 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:43:34.128 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:34.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:43:34.188 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:43:34.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:43:34.188 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:43:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:43:34.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:43:34.241 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:43:34.242 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:34.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:43:34.301 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:43:34.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:43:34.302 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:43:34.357 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:43:34.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:43:34.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:43:34.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.573+0000 7f425996b640 1 Processor -- start 2023-08-25T09:43:34.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.573+0000 7f425996b640 1 -- start start 2023-08-25T09:43:34.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.574+0000 7f425996b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415e830 0x7f425406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:34.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.574+0000 7f425996b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f425415ec10 0x7f42540b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:34.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.574+0000 7f425996b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42540b1b60 con 0x7f425406b2b0 2023-08-25T09:43:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.575+0000 7f425996b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425406bd80 con 0x7f425415ec10 2023-08-25T09:43:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.575+0000 7f425996b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425406b7c0 con 0x7f425415e830 2023-08-25T09:43:34.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.575+0000 7f42537fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f425415ec10 0x7f42540b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.575+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415e830 0x7f425406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.575+0000 7f42537fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f425415ec10 0x7f42540b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:34332/0 (socket says 172.21.15.70:34332) 2023-08-25T09:43:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.575+0000 7f42537fe640 1 -- 172.21.15.70:0/311119242 learned_addr learned my addr 172.21.15.70:0/311119242 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.576+0000 7f4252ffd640 1 -- 172.21.15.70:0/311119242 >> v2:172.21.15.155:3300/0 conn(0x7f425415ec10 msgr2=0x7f42540b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:34.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.576+0000 7f4252ffd640 1 --2- 172.21.15.70:0/311119242 >> v2:172.21.15.155:3300/0 conn(0x7f425415ec10 0x7f42540b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.576+0000 7f4252ffd640 1 -- 172.21.15.70:0/311119242 >> v1:172.21.15.155:6789/0 conn(0x7f425406b2b0 legacy=0x7f425415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.576+0000 7f4252ffd640 1 -- 172.21.15.70:0/311119242 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425406baa0 con 0x7f425415e830 2023-08-25T09:43:34.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.578+0000 7f4252ffd640 1 --2- 172.21.15.70:0/311119242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415e830 0x7f425406ad70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f42440097a0 tx=0x7f424402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bce23d4a5ecc928 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.578+0000 7f42527fc640 1 -- 172.21.15.70:0/311119242 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244031830 con 0x7f425415e830 2023-08-25T09:43:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.578+0000 7f42527fc640 1 -- 172.21.15.70:0/311119242 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4244031990 con 0x7f425415e830 2023-08-25T09:43:34.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.578+0000 7f42527fc640 1 -- 172.21.15.70:0/311119242 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244031d20 con 0x7f425415e830 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 -- 172.21.15.70:0/311119242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415e830 msgr2=0x7f425406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 --2- 172.21.15.70:0/311119242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415e830 0x7f425406ad70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f42440097a0 tx=0x7f424402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 -- 172.21.15.70:0/311119242 shutdown_connections 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 --2- 172.21.15.70:0/311119242 >> v2:172.21.15.155:3300/0 conn(0x7f425415ec10 0x7f42540b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 --2- 172.21.15.70:0/311119242 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415e830 0x7f425406ad70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 -- 172.21.15.70:0/311119242 >> 172.21.15.70:0/311119242 conn(0x7f4254069850 msgr2=0x7f4254069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:34.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 -- 172.21.15.70:0/311119242 shutdown_connections 2023-08-25T09:43:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.579+0000 7f425996b640 1 -- 172.21.15.70:0/311119242 wait complete. 2023-08-25T09:43:34.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.581+0000 7f425996b640 1 Processor -- start 2023-08-25T09:43:34.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.581+0000 7f425996b640 1 -- start start 2023-08-25T09:43:34.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f425996b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 0x7f425415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:34.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f425996b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415ec10 0x7f4254159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:34.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f425996b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4254159670 0x7f4254154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:34.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f425996b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42540af280 con 0x7f425406b2b0 2023-08-25T09:43:34.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f425996b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4254159b00 con 0x7f4254159670 2023-08-25T09:43:34.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f425996b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425415d5e0 con 0x7f425415ec10 2023-08-25T09:43:34.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 0x7f425415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:34.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f4252ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 0x7f425415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60534/0 (socket says 172.21.15.70:60534) 2023-08-25T09:43:34.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f4252ffd640 1 -- 172.21.15.70:0/441669788 learned_addr learned my addr 172.21.15.70:0/441669788 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:34.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f424bfff640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415ec10 0x7f4254159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:34.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.583+0000 7f42537fe640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4254159670 0x7f4254154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:34.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4252ffd640 1 -- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415ec10 msgr2=0x7f4254159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:34.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4252ffd640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415ec10 0x7f4254159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4252ffd640 1 -- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4254159670 msgr2=0x7f4254154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4252ffd640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4254159670 0x7f4254154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4252ffd640 1 -- 172.21.15.70:0/441669788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244009400 con 0x7f425406b2b0 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4252ffd640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 0x7f425415d0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f424402bee0 tx=0x7f42440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424403a630 con 0x7f425406b2b0 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424403a790 con 0x7f425406b2b0 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424403ab40 con 0x7f425406b2b0 2023-08-25T09:43:34.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425415ca20 con 0x7f425406b2b0 2023-08-25T09:43:34.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.584+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42540ae8e0 con 0x7f425406b2b0 2023-08-25T09:43:34.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.586+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425404e420 con 0x7f425406b2b0 2023-08-25T09:43:34.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.587+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f424403aca0 con 0x7f425406b2b0 2023-08-25T09:43:34.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.587+0000 7f4250ff9640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423406a050 0x7f423406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:34.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.588+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f42440a9f00 con 0x7f425406b2b0 2023-08-25T09:43:34.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.588+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f4234072480 con 0x7f425406b2b0 2023-08-25T09:43:34.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.588+0000 7f424bfff640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423406a050 0x7f423406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:34.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.588+0000 7f424bfff640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423406a050 0x7f423406c510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f425415dfb0 tx=0x7f423c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:34.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.592+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4244048060 con 0x7f425406b2b0 2023-08-25T09:43:34.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.816+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f425406c620 con 0x7f425406b2b0 2023-08-25T09:43:34.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.817+0000 7f4250ff9640 1 -- 172.21.15.70:0/441669788 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f4244048030 con 0x7f425406b2b0 2023-08-25T09:43:34.820 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:34.820 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:43:34.821 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:43:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423406a050 msgr2=0x7f423406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423406a050 0x7f423406c510 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f425415dfb0 tx=0x7f423c009040 comp rx=0 tx=0).stop 2023-08-25T09:43:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 msgr2=0x7f425415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:34.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 0x7f425415d0a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f424402bee0 tx=0x7f42440046c0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 shutdown_connections 2023-08-25T09:43:34.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f423406a050 0x7f423406c510 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4254159670 0x7f4254154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f425415ec10 0x7f4254159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 --2- 172.21.15.70:0/441669788 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f425406b2b0 0x7f425415d0a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:34.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 >> 172.21.15.70:0/441669788 conn(0x7f4254069850 msgr2=0x7f42540cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:34.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 shutdown_connections 2023-08-25T09:43:34.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:34.820+0000 7f425996b640 1 -- 172.21.15.70:0/441669788 wait complete. 2023-08-25T09:43:34.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.051+0000 7f8925b47640 1 Processor -- start 2023-08-25T09:43:35.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.052+0000 7f8925b47640 1 -- start start 2023-08-25T09:43:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.053+0000 7f8925b47640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89200b1ff0 0x7f89200b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.053+0000 7f8925b47640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89200b1b60 con 0x7f892015ec10 2023-08-25T09:43:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.053+0000 7f8925b47640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892006b7c0 con 0x7f892015e830 2023-08-25T09:43:35.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.053+0000 7f8925b47640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892006bd80 con 0x7f89200b1ff0 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89200b1ff0 0x7f89200b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f89200b1ff0 0x7f89200b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60544/0 (socket says 172.21.15.70:60544) 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891ffff640 1 -- 172.21.15.70:0/2783378843 learned_addr learned my addr 172.21.15.70:0/2783378843 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891effd640 1 --1- 172.21.15.70:0/2783378843 >> v1:172.21.15.70:6789/0 conn(0x7f892015ec10 0x7f892015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48096/0 (socket says 172.21.15.70:48096) 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891ffff640 1 -- 172.21.15.70:0/2783378843 >> v1:172.21.15.155:6789/0 conn(0x7f892015e830 legacy=0x7f892006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891ffff640 1 -- 172.21.15.70:0/2783378843 >> v1:172.21.15.70:6789/0 conn(0x7f892015ec10 legacy=0x7f892015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.054+0000 7f891ffff640 1 -- 172.21.15.70:0/2783378843 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892006baa0 con 0x7f89200b1ff0 2023-08-25T09:43:35.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.055+0000 7f891ffff640 1 --2- 172.21.15.70:0/2783378843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89200b2440 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8918009870 tx=0x7f891802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0e15f89d51d5e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.055+0000 7f891e7fc640 1 -- 172.21.15.70:0/2783378843 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918030860 con 0x7f89200b1ff0 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.055+0000 7f891e7fc640 1 -- 172.21.15.70:0/2783378843 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89180309c0 con 0x7f89200b1ff0 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.055+0000 7f891e7fc640 1 -- 172.21.15.70:0/2783378843 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8918030d50 con 0x7f89200b1ff0 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 -- 172.21.15.70:0/2783378843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 msgr2=0x7f89200b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 --2- 172.21.15.70:0/2783378843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89200b2440 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8918009870 tx=0x7f891802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 -- 172.21.15.70:0/2783378843 shutdown_connections 2023-08-25T09:43:35.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 --2- 172.21.15.70:0/2783378843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89200b2440 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 -- 172.21.15.70:0/2783378843 >> 172.21.15.70:0/2783378843 conn(0x7f8920069850 msgr2=0x7f8920069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 -- 172.21.15.70:0/2783378843 shutdown_connections 2023-08-25T09:43:35.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.056+0000 7f8925b47640 1 -- 172.21.15.70:0/2783378843 wait complete. 2023-08-25T09:43:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.058+0000 7f8925b47640 1 Processor -- start 2023-08-25T09:43:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.058+0000 7f8925b47640 1 -- start start 2023-08-25T09:43:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f8925b47640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89201f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f8925b47640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892015e830 0x7f89201532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f8925b47640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892015ec10 0x7f89201e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f8925b47640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89201537f0 con 0x7f89200b1ff0 2023-08-25T09:43:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f8925b47640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892006ece0 con 0x7f892015e830 2023-08-25T09:43:35.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f8925b47640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892006ef80 con 0x7f892015ec10 2023-08-25T09:43:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.059+0000 7f891f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89201f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89201f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60556/0 (socket says 172.21.15.70:60556) 2023-08-25T09:43:35.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 -- 172.21.15.70:0/2933824839 learned_addr learned my addr 172.21.15.70:0/2933824839 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 -- 172.21.15.70:0/2933824839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892015ec10 msgr2=0x7f89201e4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892015ec10 0x7f89201e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 -- 172.21.15.70:0/2933824839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892015e830 msgr2=0x7f89201532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892015e830 0x7f89201532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 -- 172.21.15.70:0/2933824839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89180094d0 con 0x7f89200b1ff0 2023-08-25T09:43:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f891f7fe640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89201f3470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f891000c430 tx=0x7f891000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910011790 con 0x7f89200b1ff0 2023-08-25T09:43:35.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89100118f0 con 0x7f89200b1ff0 2023-08-25T09:43:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892006f260 con 0x7f89200b1ff0 2023-08-25T09:43:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.060+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910011ce0 con 0x7f89200b1ff0 2023-08-25T09:43:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.061+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892006f820 con 0x7f89200b1ff0 2023-08-25T09:43:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.062+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892004e420 con 0x7f89200b1ff0 2023-08-25T09:43:35.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.063+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f891001c460 con 0x7f89200b1ff0 2023-08-25T09:43:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.064+0000 7f8907fff640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f890c06a050 0x7f890c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.064+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f8910088680 con 0x7f89200b1ff0 2023-08-25T09:43:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.064+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f890c072480 con 0x7f89200b1ff0 2023-08-25T09:43:35.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.064+0000 7f891effd640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f890c06a050 0x7f890c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.065+0000 7f891effd640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f890c06a050 0x7f890c06c510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8914007fd0 tx=0x7f8914007d80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.068+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8910053c00 con 0x7f89200b1ff0 2023-08-25T09:43:35.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.261+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f892006c620 con 0x7f89200b1ff0 2023-08-25T09:43:35.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.264+0000 7f8907fff640 1 -- 172.21.15.70:0/2933824839 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18735 (secure 0 0 0) 0x7f8910057220 con 0x7f89200b1ff0 2023-08-25T09:43:35.267 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:35.268 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":81,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:34.320892+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":1}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.38","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:35.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f890c06a050 msgr2=0x7f890c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f890c06a050 0x7f890c06c510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8914007fd0 tx=0x7f8914007d80 comp rx=0 tx=0).stop 2023-08-25T09:43:35.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 msgr2=0x7f89201f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89201f3470 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f891000c430 tx=0x7f891000c8f0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 shutdown_connections 2023-08-25T09:43:35.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f890c06a050 0x7f890c06c510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892015ec10 0x7f89201e4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f892015e830 0x7f89201532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 --2- 172.21.15.70:0/2933824839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89200b1ff0 0x7f89201f3470 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 >> 172.21.15.70:0/2933824839 conn(0x7f8920069850 msgr2=0x7f89200ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:35.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 shutdown_connections 2023-08-25T09:43:35.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.267+0000 7f8925b47640 1 -- 172.21.15.70:0/2933824839 wait complete. 2023-08-25T09:43:35.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:42:22.389960+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:42:23.780164+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:43:35.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:43:35.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.508+0000 7f7ed0ff5640 1 Processor -- start 2023-08-25T09:43:35.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.509+0000 7f7ed0ff5640 1 -- start start 2023-08-25T09:43:35.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.510+0000 7f7ed0ff5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0320 0x7f7ecc0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.510+0000 7f7ed0ff5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ecc15a920 0x7f7ecc0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.510+0000 7f7ed0ff5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ecc0b0700 0x7f7ecc0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.510+0000 7f7ed0ff5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc0b2220 con 0x7f7ecc15a920 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.510+0000 7f7ed0ff5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ecc0ad8f0 con 0x7f7ecc0b0700 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.510+0000 7f7ed0ff5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc0adbb0 con 0x7f7ecc0b0320 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ecc15a920 0x7f7ecc0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ecc15a920 0x7f7ecc0b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60560/0 (socket says 172.21.15.70:60560) 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 -- 172.21.15.70:0/10142962 learned_addr learned my addr 172.21.15.70:0/10142962 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecb7fe640 1 --2- 172.21.15.70:0/10142962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0320 0x7f7ecc0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/10142962 >> v2:172.21.15.155:3300/0 conn(0x7f7ecc0b0700 0x7f7ecc0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 -- 172.21.15.70:0/10142962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0320 msgr2=0x7f7ecc0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 --2- 172.21.15.70:0/10142962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0320 0x7f7ecc0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 -- 172.21.15.70:0/10142962 >> v2:172.21.15.155:3300/0 conn(0x7f7ecc0b0700 msgr2=0x7f7ecc0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 --2- 172.21.15.70:0/10142962 >> v2:172.21.15.155:3300/0 conn(0x7f7ecc0b0700 0x7f7ecc0ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.511+0000 7f7ecaffd640 1 -- 172.21.15.70:0/10142962 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0c90a0 con 0x7f7ecc15a920 2023-08-25T09:43:35.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.512+0000 7f7ecaffd640 1 --2- 172.21.15.70:0/10142962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0b1ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0b2fd0 tx=0x7f7ebc02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2caa6ace1e28030 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.513+0000 7f7eca7fc640 1 -- 172.21.15.70:0/10142962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc031880 con 0x7f7ecc15a920 2023-08-25T09:43:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.513+0000 7f7eca7fc640 1 -- 172.21.15.70:0/10142962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0319e0 con 0x7f7ecc15a920 2023-08-25T09:43:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.513+0000 7f7eca7fc640 1 -- 172.21.15.70:0/10142962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc031d70 con 0x7f7ecc15a920 2023-08-25T09:43:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.513+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/10142962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 msgr2=0x7f7ecc0b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.513+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/10142962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0b1ce0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0b2fd0 tx=0x7f7ebc02b920 comp rx=0 tx=0).stop 2023-08-25T09:43:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/10142962 shutdown_connections 2023-08-25T09:43:35.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/10142962 >> v2:172.21.15.155:3300/0 conn(0x7f7ecc0b0700 0x7f7ecc0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/10142962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0b1ce0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/10142962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0320 0x7f7ecc0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/10142962 >> 172.21.15.70:0/10142962 conn(0x7f7ecc0ce830 msgr2=0x7f7ecc0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/10142962 shutdown_connections 2023-08-25T09:43:35.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.514+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/10142962 wait complete. 2023-08-25T09:43:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.515+0000 7f7ed0ff5640 1 Processor -- start 2023-08-25T09:43:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 -- start start 2023-08-25T09:43:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ecc0b0320 0x7f7ecc0ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0700 0x7f7ecc0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc15c8f0 con 0x7f7ecc15a920 2023-08-25T09:43:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc069850 con 0x7f7ecc0b0320 2023-08-25T09:43:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.516+0000 7f7ed0ff5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ecc15bfc0 con 0x7f7ecc0b0700 2023-08-25T09:43:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60562/0 (socket says 172.21.15.70:60562) 2023-08-25T09:43:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 -- 172.21.15.70:0/578905931 learned_addr learned my addr 172.21.15.70:0/578905931 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecaffd640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0700 0x7f7ecc0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecb7fe640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ecc0b0320 0x7f7ecc0ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 -- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0700 msgr2=0x7f7ecc0c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0700 0x7f7ecc0c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 -- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ecc0b0320 msgr2=0x7f7ecc0ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ecc0b0320 0x7f7ecc0ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.517+0000 7f7ecbfff640 1 -- 172.21.15.70:0/578905931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0094d0 con 0x7f7ecc15a920 2023-08-25T09:43:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.518+0000 7f7ecbfff640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0c48f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400cda0 tx=0x7f7ec400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.518+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4003b90 con 0x7f7ecc15a920 2023-08-25T09:43:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.518+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec4003cf0 con 0x7f7ecc15a920 2023-08-25T09:43:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.518+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ecc15c2a0 con 0x7f7ecc15a920 2023-08-25T09:43:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.518+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4004240 con 0x7f7ecc15a920 2023-08-25T09:43:35.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.518+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ecc1572e0 con 0x7f7ecc15a920 2023-08-25T09:43:35.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.519+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc15caa0 con 0x7f7ecc15a920 2023-08-25T09:43:35.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.520+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ec40043a0 con 0x7f7ecc15a920 2023-08-25T09:43:35.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.522+0000 7f7eb3fff640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb806a080 0x7f7eb806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.522+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f7ec4012070 con 0x7f7ecc15a920 2023-08-25T09:43:35.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.522+0000 7f7ecb7fe640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb806a080 0x7f7eb806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.523+0000 7f7ecb7fe640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb806a080 0x7f7eb806c540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009770 tx=0x7f7ec0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.526+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ecc15caa0 con 0x7f7ecc15a920 2023-08-25T09:43:35.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.716+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ecc15caa0 con 0x7f7ecc15a920 2023-08-25T09:43:35.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.717+0000 7f7eb3fff640 1 -- 172.21.15.70:0/578905931 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ecc15caa0 con 0x7f7ecc15a920 2023-08-25T09:43:35.720 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:43:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb806a080 msgr2=0x7f7eb806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb806a080 0x7f7eb806c540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009770 tx=0x7f7ec0008040 comp rx=0 tx=0).stop 2023-08-25T09:43:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 msgr2=0x7f7ecc0c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0c48f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400cda0 tx=0x7f7ec400a660 comp rx=0 tx=0).stop 2023-08-25T09:43:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 shutdown_connections 2023-08-25T09:43:35.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7eb806a080 0x7f7eb806c540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ecc15a920 0x7f7ecc0c48f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ecc0b0700 0x7f7ecc0c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 --2- 172.21.15.70:0/578905931 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ecc0b0320 0x7f7ecc0ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.719+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 >> 172.21.15.70:0/578905931 conn(0x7f7ecc0ce830 msgr2=0x7f7ecc070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:35.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.720+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 shutdown_connections 2023-08-25T09:43:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.720+0000 7f7ed0ff5640 1 -- 172.21.15.70:0/578905931 wait complete. 2023-08-25T09:43:35.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:43:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.960+0000 7fa1e25ab640 1 Processor -- start 2023-08-25T09:43:35.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.960+0000 7fa1e25ab640 1 -- start start 2023-08-25T09:43:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.961+0000 7fa1e25ab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa1dc0b2470 0x7fa1dc159930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.961+0000 7fa1e25ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.961+0000 7fa1e25ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1dc15d2b0 0x7fa1dc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.961+0000 7fa1e25ab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc15ad20 con 0x7fa1dc15d2b0 2023-08-25T09:43:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.961+0000 7fa1e25ab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc159e70 con 0x7fa1dc0b2470 2023-08-25T09:43:35.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.961+0000 7fa1e25ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cd800 con 0x7fa1dc0b2850 2023-08-25T09:43:35.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1dc15d2b0 0x7fa1dc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa1dc15d2b0 0x7fa1dc15d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60574/0 (socket says 172.21.15.70:60574) 2023-08-25T09:43:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 -- 172.21.15.70:0/3928778215 learned_addr learned my addr 172.21.15.70:0/3928778215 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e15a9640 1 --2- 172.21.15.70:0/3928778215 >> v2:172.21.15.155:3300/0 conn(0x7fa1dc0b2470 0x7fa1dc159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e0da8640 1 --2- 172.21.15.70:0/3928778215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 -- 172.21.15.70:0/3928778215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 msgr2=0x7fa1dc0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:35.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 --2- 172.21.15.70:0/3928778215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 -- 172.21.15.70:0/3928778215 >> v2:172.21.15.155:3300/0 conn(0x7fa1dc0b2470 msgr2=0x7fa1dc159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 --2- 172.21.15.70:0/3928778215 >> v2:172.21.15.155:3300/0 conn(0x7fa1dc0b2470 0x7fa1dc159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.962+0000 7fa1e1daa640 1 -- 172.21.15.70:0/3928778215 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc15a110 con 0x7fa1dc15d2b0 2023-08-25T09:43:35.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.963+0000 7fa1e1daa640 1 --2- 172.21.15.70:0/3928778215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc15d2b0 0x7fa1dc15d740 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0b0f90 tx=0x7fa1d402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ab7c941c5588cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.964+0000 7fa1d3fff640 1 -- 172.21.15.70:0/3928778215 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d402ea80 con 0x7fa1dc15d2b0 2023-08-25T09:43:35.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.964+0000 7fa1d3fff640 1 -- 172.21.15.70:0/3928778215 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d402ebe0 con 0x7fa1dc15d2b0 2023-08-25T09:43:35.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.964+0000 7fa1d3fff640 1 -- 172.21.15.70:0/3928778215 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4009600 con 0x7fa1dc15d2b0 2023-08-25T09:43:35.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.964+0000 7fa1e25ab640 1 -- 172.21.15.70:0/3928778215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc15d2b0 msgr2=0x7fa1dc15d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.964+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/3928778215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc15d2b0 0x7fa1dc15d740 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0b0f90 tx=0x7fa1d402e0e0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 -- 172.21.15.70:0/3928778215 shutdown_connections 2023-08-25T09:43:35.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/3928778215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc15d2b0 0x7fa1dc15d740 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/3928778215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/3928778215 >> v2:172.21.15.155:3300/0 conn(0x7fa1dc0b2470 0x7fa1dc159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 -- 172.21.15.70:0/3928778215 >> 172.21.15.70:0/3928778215 conn(0x7fa1dc0ce830 msgr2=0x7fa1dc0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:35.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 -- 172.21.15.70:0/3928778215 shutdown_connections 2023-08-25T09:43:35.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.965+0000 7fa1e25ab640 1 -- 172.21.15.70:0/3928778215 wait complete. 2023-08-25T09:43:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.966+0000 7fa1e25ab640 1 Processor -- start 2023-08-25T09:43:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.967+0000 7fa1e25ab640 1 -- start start 2023-08-25T09:43:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.967+0000 7fa1e25ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 0x7fa1dc1f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.967+0000 7fa1e25ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.967+0000 7fa1e25ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1dc15d2b0 0x7fa1dc153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e25ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc069850 con 0x7fa1dc0b2470 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e25ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc1f11c0 con 0x7fa1dc15d2b0 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e25ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc069aa0 con 0x7fa1dc0b2850 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 0x7fa1dc1f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 0x7fa1dc1f0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60584/0 (socket says 172.21.15.70:60584) 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 -- 172.21.15.70:0/1907432285 learned_addr learned my addr 172.21.15.70:0/1907432285 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 -- 172.21.15.70:0/1907432285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 msgr2=0x7fa1dc153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 -- 172.21.15.70:0/1907432285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1dc15d2b0 msgr2=0x7fa1dc153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1dc15d2b0 0x7fa1dc153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 -- 172.21.15.70:0/1907432285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d400bd40 con 0x7fa1dc0b2470 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.968+0000 7fa1e15a9640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 0x7fa1dc1f0c80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d800 tx=0x7fa1d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.969+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80096a0 con 0x7fa1dc0b2470 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.969+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8009800 con 0x7fa1dc0b2470 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.969+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc069d80 con 0x7fa1dc0b2470 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.969+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8009b90 con 0x7fa1dc0b2470 2023-08-25T09:43:35.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.969+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc06a340 con 0x7fa1dc0b2470 2023-08-25T09:43:35.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.970+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc061460 con 0x7fa1dc0b2470 2023-08-25T09:43:35.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.971+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa1d8009cf0 con 0x7fa1dc0b2470 2023-08-25T09:43:35.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.972+0000 7fa1d1ffb640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1c006a050 0x7fa1c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:35.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.972+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa1d8088bb0 con 0x7fa1dc0b2470 2023-08-25T09:43:35.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.972+0000 7fa1e0da8640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1c006a050 0x7fa1c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:35.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.973+0000 7fa1e0da8640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1c006a050 0x7fa1c006c510 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006fd0 tx=0x7fa1cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:35.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:35.977+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa1d8054130 con 0x7fa1dc0b2470 2023-08-25T09:43:36.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.161+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1dc071d30 con 0x7fa1dc0b2470 2023-08-25T09:43:36.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.162+0000 7fa1d1ffb640 1 -- 172.21.15.70:0/1907432285 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1d8057750 con 0x7fa1dc0b2470 2023-08-25T09:43:36.165 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:43:36.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1c006a050 msgr2=0x7fa1c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1c006a050 0x7fa1c006c510 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc006fd0 tx=0x7fa1cc008040 comp rx=0 tx=0).stop 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 msgr2=0x7fa1dc1f0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 0x7fa1dc1f0c80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d800 tx=0x7fa1d800dcc0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 shutdown_connections 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa1c006a050 0x7fa1c006c510 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa1dc15d2b0 0x7fa1dc153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa1dc0b2850 0x7fa1dc153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 --2- 172.21.15.70:0/1907432285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa1dc0b2470 0x7fa1dc1f0c80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 >> 172.21.15.70:0/1907432285 conn(0x7fa1dc0ce830 msgr2=0x7fa1dc0c96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:36.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 shutdown_connections 2023-08-25T09:43:36.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.165+0000 7fa1e25ab640 1 -- 172.21.15.70:0/1907432285 wait complete. 2023-08-25T09:43:36.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:43:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.409+0000 7fe1cdbc9640 1 Processor -- start 2023-08-25T09:43:36.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.409+0000 7fe1cdbc9640 1 -- start start 2023-08-25T09:43:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.411+0000 7fe1cdbc9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 0x7fe1c81682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.411+0000 7fe1cdbc9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80ac880 con 0x7fe1c8069850 2023-08-25T09:43:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.411+0000 7fe1cdbc9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80ad400 con 0x7fe1c815e210 2023-08-25T09:43:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.411+0000 7fe1cdbc9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815e650 con 0x7fe1c8165f00 2023-08-25T09:43:36.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.411+0000 7fe1c77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe1c8069850 0x7fe1c80c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48102/0 (socket says 172.21.15.70:48102) 2023-08-25T09:43:36.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.411+0000 7fe1c77fe640 1 -- 172.21.15.70:0/4105142805 learned_addr learned my addr 172.21.15.70:0/4105142805 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399451911 0 0) 0x7fe1c80ac880 con 0x7fe1c8069850 2023-08-25T09:43:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b4003520 con 0x7fe1c8069850 2023-08-25T09:43:36.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c7fff640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 0x7fe1c81682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 356504326 0 0) 0x7fe1b4003520 con 0x7fe1c8069850 2023-08-25T09:43:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 msgr2=0x7fe1c81682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 0x7fe1c81682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 >> v1:172.21.15.155:6789/0 conn(0x7fe1c815e210 legacy=0x7fe1c8069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80ad890 con 0x7fe1c8069850 2023-08-25T09:43:36.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe1b8001670 con 0x7fe1c8069850 2023-08-25T09:43:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 >> v1:172.21.15.70:6789/0 conn(0x7fe1c8069850 legacy=0x7fe1c80c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40039a0 0x7fe1b4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 0x7fe1b40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815e650 con 0x7fe1b40047c0 2023-08-25T09:43:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400bb50 con 0x7fe1b40039a0 2023-08-25T09:43:36.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.412+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b400bd10 con 0x7fe1c8165f00 2023-08-25T09:43:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1bffff640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1c77fe640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40039a0 0x7fe1b4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1c7fff640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 0x7fe1b40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1bffff640 1 -- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 msgr2=0x7fe1b40051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1bffff640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c8165f00 0x7fe1b40051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1bffff640 1 -- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40039a0 msgr2=0x7fe1b4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1bffff640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40039a0 0x7fe1b4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1bffff640 1 -- 172.21.15.70:0/4105142805 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b4003520 con 0x7fe1b40047c0 2023-08-25T09:43:36.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.413+0000 7fe1c77fe640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40039a0 0x7fe1b4003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:36.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.414+0000 7fe1bffff640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0003db0 tx=0x7fe1b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b1d6359c080217 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:36.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.414+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0032600 con 0x7fe1b40047c0 2023-08-25T09:43:36.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.414+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b002e050 con 0x7fe1b40047c0 2023-08-25T09:43:36.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.414+0000 7fe1c6ffd640 1 -- 172.21.15.70:0/4105142805 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00329a0 con 0x7fe1b40047c0 2023-08-25T09:43:36.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/4105142805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1b40047c0 msgr2=0x7fe1b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/4105142805 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1b40047c0 0x7fe1b4004c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0003db0 tx=0x7fe1b40057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1c7fff640 1 -- 172.21.15.70:0/4105142805 reap_dead start 2023-08-25T09:43:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/4105142805 shutdown_connections 2023-08-25T09:43:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/4105142805 >> 172.21.15.70:0/4105142805 conn(0x7fe1c80cb0d0 msgr2=0x7fe1c80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/4105142805 shutdown_connections 2023-08-25T09:43:36.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.415+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/4105142805 wait complete. 2023-08-25T09:43:36.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.417+0000 7fe1cdbc9640 1 Processor -- start 2023-08-25T09:43:36.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.417+0000 7fe1cdbc9640 1 -- start start 2023-08-25T09:43:36.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1cdbc9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40047c0 0x7fe1c81f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1cdbc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 0x7fe1c806ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1cdbc9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c80c5b10 0x7fe1c8151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1cdbc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8149360 con 0x7fe1c80c5690 2023-08-25T09:43:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1cdbc9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8145740 con 0x7fe1b40047c0 2023-08-25T09:43:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1cdbc9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c806fc20 con 0x7fe1c80c5b10 2023-08-25T09:43:36.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 0x7fe1c806ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 0x7fe1c806ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60596/0 (socket says 172.21.15.70:60596) 2023-08-25T09:43:36.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.418+0000 7fe1bffff640 1 -- 172.21.15.70:0/3311105574 learned_addr learned my addr 172.21.15.70:0/3311105574 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:36.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1bffff640 1 -- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c80c5b10 msgr2=0x7fe1c8151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1bffff640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c80c5b10 0x7fe1c8151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1bffff640 1 -- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40047c0 msgr2=0x7fe1c81f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1c77fe640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40047c0 0x7fe1c81f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1bffff640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40047c0 0x7fe1c81f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1bffff640 1 -- 172.21.15.70:0/3311105574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c806f660 con 0x7fe1c80c5690 2023-08-25T09:43:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1bffff640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 0x7fe1c806ebe0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe1b002bb30 tx=0x7fe1b00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.419+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b003d420 con 0x7fe1c80c5690 2023-08-25T09:43:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.420+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b003d580 con 0x7fe1c80c5690 2023-08-25T09:43:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.420+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b003d930 con 0x7fe1c80c5690 2023-08-25T09:43:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.420+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c806f940 con 0x7fe1c80c5690 2023-08-25T09:43:36.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.420+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c80702b0 con 0x7fe1c80c5690 2023-08-25T09:43:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.421+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c81494a0 con 0x7fe1c80c5690 2023-08-25T09:43:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.422+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe1b003da90 con 0x7fe1c80c5690 2023-08-25T09:43:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.423+0000 7fe1c4ff9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1a806a080 0x7fe1a806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.423+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fe1b00aa1c0 con 0x7fe1c80c5690 2023-08-25T09:43:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.423+0000 7fe1c77fe640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1a806a080 0x7fe1a806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.424+0000 7fe1c77fe640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1a806a080 0x7fe1a806c540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80017a0 tx=0x7fe1b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:36.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.427+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe1c81494a0 con 0x7fe1c80c5690 2023-08-25T09:43:36.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.618+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1c81494a0 con 0x7fe1c80c5690 2023-08-25T09:43:36.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.619+0000 7fe1c4ff9640 1 -- 172.21.15.70:0/3311105574 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1c81494a0 con 0x7fe1c80c5690 2023-08-25T09:43:36.621 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:43:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1a806a080 msgr2=0x7fe1a806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1a806a080 0x7fe1a806c540 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80017a0 tx=0x7fe1b8009040 comp rx=0 tx=0).stop 2023-08-25T09:43:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 msgr2=0x7fe1c806ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 0x7fe1c806ebe0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe1b002bb30 tx=0x7fe1b00036f0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 shutdown_connections 2023-08-25T09:43:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1a806a080 0x7fe1a806c540 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1c80c5b10 0x7fe1c8151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1c80c5690 0x7fe1c806ebe0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 --2- 172.21.15.70:0/3311105574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1b40047c0 0x7fe1c81f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 >> 172.21.15.70:0/3311105574 conn(0x7fe1c80cb0d0 msgr2=0x7fe1c815d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 shutdown_connections 2023-08-25T09:43:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.622+0000 7fe1cdbc9640 1 -- 172.21.15.70:0/3311105574 wait complete. 2023-08-25T09:43:36.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:43:36.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.846+0000 7ff402e8e640 1 Processor -- start 2023-08-25T09:43:36.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.846+0000 7ff402e8e640 1 -- start start 2023-08-25T09:43:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.847+0000 7ff402e8e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff3fc15ae50 0x7ff3fc0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.847+0000 7ff402e8e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3fc0cc5f0 con 0x7ff3fc069f60 2023-08-25T09:43:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff402e8e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3fc070650 con 0x7ff3fc0cdc20 2023-08-25T09:43:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff402e8e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc070930 con 0x7ff3fc15ae50 2023-08-25T09:43:36.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff40142b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff3fc069f60 0x7ff3fc0ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48114/0 (socket says 172.21.15.70:48114) 2023-08-25T09:43:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff40142b640 1 -- 172.21.15.70:0/2038501277 learned_addr learned my addr 172.21.15.70:0/2038501277 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff400c2a640 1 --2- 172.21.15.70:0/2038501277 >> v2:172.21.15.155:3300/0 conn(0x7ff3fc15ae50 0x7ff3fc0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956860095 0 0) 0x7ff3fc0cc5f0 con 0x7ff3fc069f60 2023-08-25T09:43:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.848+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec003520 con 0x7ff3fc069f60 2023-08-25T09:43:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3187960191 0 0) 0x7ff3ec003520 con 0x7ff3fc069f60 2023-08-25T09:43:36.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 >> v2:172.21.15.155:3300/0 conn(0x7ff3fc15ae50 msgr2=0x7ff3fc0c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 --2- 172.21.15.70:0/2038501277 >> v2:172.21.15.155:3300/0 conn(0x7ff3fc15ae50 0x7ff3fc0c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 >> v1:172.21.15.155:6789/0 conn(0x7ff3fc0cdc20 legacy=0x7ff3fc069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc070c10 con 0x7ff3fc069f60 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff3f40019f0 con 0x7ff3fc069f60 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 >> v1:172.21.15.70:6789/0 conn(0x7ff3fc069f60 legacy=0x7ff3fc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0047c0 0x7ff3ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec005180 0x7ff3ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc070930 con 0x7ff3ec003970 2023-08-25T09:43:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00c030 con 0x7ff3ec005180 2023-08-25T09:43:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec00c1f0 con 0x7ff3ec0047c0 2023-08-25T09:43:36.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.849+0000 7ff400c2a640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff3fbfff640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0047c0 0x7ff3ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff40142b640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec005180 0x7ff3ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff400c2a640 1 -- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0047c0 msgr2=0x7ff3ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff400c2a640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0047c0 0x7ff3ec004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff400c2a640 1 -- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec005180 msgr2=0x7ff3ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff400c2a640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec005180 0x7ff3ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff400c2a640 1 -- 172.21.15.70:0/2038501277 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec003520 con 0x7ff3ec003970 2023-08-25T09:43:36.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff40142b640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec005180 0x7ff3ec00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.850+0000 7ff40142b640 1 -- 172.21.15.70:0/2038501277 reap_dead start 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.851+0000 7ff400c2a640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3ec003d90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ac0 tx=0x7ff3ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5a3f1eb466f300 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.851+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f002c6b0 con 0x7ff3ec003970 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.851+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f002c810 con 0x7ff3ec003970 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.851+0000 7ff3fb7fe640 1 -- 172.21.15.70:0/2038501277 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f002cba0 con 0x7ff3ec003970 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 -- 172.21.15.70:0/2038501277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 msgr2=0x7ff3ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3ec003d90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ac0 tx=0x7ff3ec0057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 -- 172.21.15.70:0/2038501277 shutdown_connections 2023-08-25T09:43:36.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 --2- 172.21.15.70:0/2038501277 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3ec003d90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 -- 172.21.15.70:0/2038501277 >> 172.21.15.70:0/2038501277 conn(0x7ff3fc0cb0d0 msgr2=0x7ff3fc0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:36.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 -- 172.21.15.70:0/2038501277 shutdown_connections 2023-08-25T09:43:36.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.852+0000 7ff402e8e640 1 -- 172.21.15.70:0/2038501277 wait complete. 2023-08-25T09:43:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.854+0000 7ff402e8e640 1 Processor -- start 2023-08-25T09:43:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.855+0000 7ff402e8e640 1 -- start start 2023-08-25T09:43:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff402e8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3fc0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff402e8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3fc15dbd0 0x7ff3fc0c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff402e8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3fc156e80 0x7ff3fc0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff402e8e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c5cb0 con 0x7ff3ec003970 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff402e8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c5f90 con 0x7ff3fc15dbd0 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff402e8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c6270 con 0x7ff3fc156e80 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff400c2a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3fc0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff400c2a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3fc0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60618/0 (socket says 172.21.15.70:60618) 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.856+0000 7ff400c2a640 1 -- 172.21.15.70:0/3487722113 learned_addr learned my addr 172.21.15.70:0/3487722113 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff3fbfff640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3fc15dbd0 0x7ff3fc0c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff40142b640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3fc156e80 0x7ff3fc0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff400c2a640 1 -- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3fc156e80 msgr2=0x7ff3fc0c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff400c2a640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3fc156e80 0x7ff3fc0c7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff400c2a640 1 -- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3fc15dbd0 msgr2=0x7ff3fc0c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff400c2a640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3fc15dbd0 0x7ff3fc0c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff400c2a640 1 -- 172.21.15.70:0/3487722113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc0c6550 con 0x7ff3ec003970 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff3fbfff640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3fc15dbd0 0x7ff3fc0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:36.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff400c2a640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3fc0c5690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ac0 tx=0x7ff3f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003810 con 0x7ff3ec003970 2023-08-25T09:43:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0003970 con 0x7ff3ec003970 2023-08-25T09:43:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc0c6830 con 0x7ff3ec003970 2023-08-25T09:43:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.857+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003d00 con 0x7ff3ec003970 2023-08-25T09:43:36.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.858+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc0c7290 con 0x7ff3ec003970 2023-08-25T09:43:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.859+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc066fb0 con 0x7ff3ec003970 2023-08-25T09:43:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.860+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3f0040450 con 0x7ff3ec003970 2023-08-25T09:43:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.860+0000 7ff3f97fa640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3e006a080 0x7ff3e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.861+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7ff3f00aa580 con 0x7ff3ec003970 2023-08-25T09:43:36.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.861+0000 7ff3fbfff640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3e006a080 0x7ff3e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:36.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.862+0000 7ff3fbfff640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3e006a080 0x7ff3e006c540 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8007fd0 tx=0x7ff3e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:36.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:36.865+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3f0075b10 con 0x7ff3ec003970 2023-08-25T09:43:37.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.050+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff3fc0cb8c0 con 0x7ff3ec003970 2023-08-25T09:43:37.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.052+0000 7ff3f97fa640 1 -- 172.21.15.70:0/3487722113 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff3f0003e60 con 0x7ff3ec003970 2023-08-25T09:43:37.054 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:43:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3e006a080 msgr2=0x7ff3e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3e006a080 0x7ff3e006c540 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff3e8007fd0 tx=0x7ff3e8009040 comp rx=0 tx=0).stop 2023-08-25T09:43:37.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 msgr2=0x7ff3fc0c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3fc0c5690 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ac0 tx=0x7ff3f0004730 comp rx=0 tx=0).stop 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 shutdown_connections 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3e006a080 0x7ff3e006c540 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3fc156e80 0x7ff3fc0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3fc15dbd0 0x7ff3fc0c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 --2- 172.21.15.70:0/3487722113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec003970 0x7ff3fc0c5690 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 >> 172.21.15.70:0/3487722113 conn(0x7ff3fc0cb0d0 msgr2=0x7ff3fc0cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 shutdown_connections 2023-08-25T09:43:37.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.054+0000 7ff402e8e640 1 -- 172.21.15.70:0/3487722113 wait complete. 2023-08-25T09:43:37.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:43:37.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.281+0000 7fb1d8739640 1 Processor -- start 2023-08-25T09:43:37.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.281+0000 7fb1d8739640 1 -- start start 2023-08-25T09:43:37.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d8739640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb1d015ec10 0x7fb1d015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d8739640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb1d015c6d0 0x7fb1d00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d8739640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d00b1b60 con 0x7fb1d015e830 2023-08-25T09:43:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d8739640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d006bd80 con 0x7fb1d015ec10 2023-08-25T09:43:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d8739640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d006b7c0 con 0x7fb1d015c6d0 2023-08-25T09:43:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d64d5640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb1d015e830 0x7fb1d006ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48118/0 (socket says 172.21.15.70:48118) 2023-08-25T09:43:37.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d64d5640 1 -- 172.21.15.70:0/3233880188 learned_addr learned my addr 172.21.15.70:0/3233880188 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.283+0000 7fb1d5cd4640 1 --2- 172.21.15.70:0/3233880188 >> v2:172.21.15.70:3300/0 conn(0x7fb1d015ec10 0x7fb1d015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.284+0000 7fb1d6cd6640 1 --2- 172.21.15.70:0/3233880188 >> v2:172.21.15.155:3300/0 conn(0x7fb1d015c6d0 0x7fb1d00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.284+0000 7fb1d5cd4640 1 -- 172.21.15.70:0/3233880188 >> v2:172.21.15.155:3300/0 conn(0x7fb1d015c6d0 msgr2=0x7fb1d00b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:37.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.284+0000 7fb1d5cd4640 1 --2- 172.21.15.70:0/3233880188 >> v2:172.21.15.155:3300/0 conn(0x7fb1d015c6d0 0x7fb1d00b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.284+0000 7fb1d5cd4640 1 -- 172.21.15.70:0/3233880188 >> v1:172.21.15.70:6789/0 conn(0x7fb1d015e830 legacy=0x7fb1d006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.284+0000 7fb1d5cd4640 1 -- 172.21.15.70:0/3233880188 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d006baa0 con 0x7fb1d015ec10 2023-08-25T09:43:37.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.286+0000 7fb1d5cd4640 1 --2- 172.21.15.70:0/3233880188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d015c190 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00cf460 tx=0x7fb1c402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8aad2782b4a42a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:37.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.286+0000 7fb1d54d3640 1 -- 172.21.15.70:0/3233880188 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c402cbf0 con 0x7fb1d015ec10 2023-08-25T09:43:37.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.286+0000 7fb1d54d3640 1 -- 172.21.15.70:0/3233880188 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c402cd50 con 0x7fb1d015ec10 2023-08-25T09:43:37.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.286+0000 7fb1d54d3640 1 -- 172.21.15.70:0/3233880188 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4034660 con 0x7fb1d015ec10 2023-08-25T09:43:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 -- 172.21.15.70:0/3233880188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 msgr2=0x7fb1d015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 --2- 172.21.15.70:0/3233880188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d015c190 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00cf460 tx=0x7fb1c402be40 comp rx=0 tx=0).stop 2023-08-25T09:43:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 -- 172.21.15.70:0/3233880188 shutdown_connections 2023-08-25T09:43:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 --2- 172.21.15.70:0/3233880188 >> v2:172.21.15.155:3300/0 conn(0x7fb1d015c6d0 0x7fb1d00b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 --2- 172.21.15.70:0/3233880188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d015c190 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 -- 172.21.15.70:0/3233880188 >> 172.21.15.70:0/3233880188 conn(0x7fb1d0069850 msgr2=0x7fb1d0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 -- 172.21.15.70:0/3233880188 shutdown_connections 2023-08-25T09:43:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.287+0000 7fb1d8739640 1 -- 172.21.15.70:0/3233880188 wait complete. 2023-08-25T09:43:37.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.289+0000 7fb1d8739640 1 Processor -- start 2023-08-25T09:43:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 -- start start 2023-08-25T09:43:37.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d015c6d0 0x7fb1d014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d015e830 0x7fb1d0146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d0148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00c9430 con 0x7fb1d015ec10 2023-08-25T09:43:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00c9710 con 0x7fb1d015e830 2023-08-25T09:43:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d8739640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00c99f0 con 0x7fb1d015c6d0 2023-08-25T09:43:37.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d6cd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d0148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d6cd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d0148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60638/0 (socket says 172.21.15.70:60638) 2023-08-25T09:43:37.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d6cd6640 1 -- 172.21.15.70:0/128408352 learned_addr learned my addr 172.21.15.70:0/128408352 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:37.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d64d5640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d015c6d0 0x7fb1d014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.291+0000 7fb1d5cd4640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d015e830 0x7fb1d0146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d6cd6640 1 -- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d015c6d0 msgr2=0x7fb1d014cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d6cd6640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d015c6d0 0x7fb1d014cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d6cd6640 1 -- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d015e830 msgr2=0x7fb1d0146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d6cd6640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d015e830 0x7fb1d0146200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d6cd6640 1 -- 172.21.15.70:0/128408352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4009a70 con 0x7fb1d015ec10 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d64d5640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d015c6d0 0x7fb1d014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d6cd6640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d0148be0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800e7f0 tx=0x7fb1c800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8009510 con 0x7fb1d015ec10 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c800ee40 con 0x7fb1d015ec10 2023-08-25T09:43:37.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c800cb80 con 0x7fb1d015ec10 2023-08-25T09:43:37.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d00c9cd0 con 0x7fb1d015ec10 2023-08-25T09:43:37.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.292+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d00ca290 con 0x7fb1d015ec10 2023-08-25T09:43:37.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.293+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d0146b50 con 0x7fb1d015ec10 2023-08-25T09:43:37.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.294+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1c800cce0 con 0x7fb1d015ec10 2023-08-25T09:43:37.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.296+0000 7fb1c2ffd640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.296+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fb1c80886c0 con 0x7fb1d015ec10 2023-08-25T09:43:37.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.296+0000 7fb1d64d5640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.297+0000 7fb1d64d5640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc005880 tx=0x7fb1cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:37.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.301+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb1d0146b50 con 0x7fb1d015ec10 2023-08-25T09:43:37.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.533+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d0146b50 con 0x7fb1d015ec10 2023-08-25T09:43:37.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.534+0000 7fb1c2ffd640 1 -- 172.21.15.70:0/128408352 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fb1d0146b50 con 0x7fb1d015ec10 2023-08-25T09:43:37.537 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:37.537 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:43:37.537 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:43:37.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 msgr2=0x7fb1b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc005880 tx=0x7fb1cc009040 comp rx=0 tx=0).stop 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 msgr2=0x7fb1d0148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d0148be0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb1c800e7f0 tx=0x7fb1c800ebf0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 shutdown_connections 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d015ec10 0x7fb1d0148be0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d015e830 0x7fb1d0146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 --2- 172.21.15.70:0/128408352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d015c6d0 0x7fb1d014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 >> 172.21.15.70:0/128408352 conn(0x7fb1d0069850 msgr2=0x7fb1d00b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:37.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 shutdown_connections 2023-08-25T09:43:37.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.537+0000 7fb1d8739640 1 -- 172.21.15.70:0/128408352 wait complete. 2023-08-25T09:43:37.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:37.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 Processor -- start 2023-08-25T09:43:37.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 -- start start 2023-08-25T09:43:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 0x7fa6a006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6a015ec10 0x7fa6a00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00b1b60 con 0x7fa6a006b2b0 2023-08-25T09:43:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a006bd80 con 0x7fa6a015ec10 2023-08-25T09:43:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.770+0000 7fa6a87cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006b7c0 con 0x7fa6a015e830 2023-08-25T09:43:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.771+0000 7fa6a5d6a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa6a006b2b0 0x7fa6a015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48134/0 (socket says 172.21.15.70:48134) 2023-08-25T09:43:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.771+0000 7fa6a5d6a640 1 -- 172.21.15.70:0/262034699 learned_addr learned my addr 172.21.15.70:0/262034699 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.771+0000 7fa6a656b640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 0x7fa6a006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.771+0000 7fa6a6d6c640 1 --2- 172.21.15.70:0/262034699 >> v2:172.21.15.155:3300/0 conn(0x7fa6a015ec10 0x7fa6a00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.771+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 690157427 0 0) 0x7fa6a00b1b60 con 0x7fa6a006b2b0 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.771+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa68c003520 con 0x7fa6a006b2b0 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683112844 0 0) 0x7fa68c003520 con 0x7fa6a006b2b0 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 msgr2=0x7fa6a006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 0x7fa6a006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 >> v2:172.21.15.155:3300/0 conn(0x7fa6a015ec10 msgr2=0x7fa6a00b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 --2- 172.21.15.70:0/262034699 >> v2:172.21.15.155:3300/0 conn(0x7fa6a015ec10 0x7fa6a00b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a006baa0 con 0x7fa6a006b2b0 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa690001670 con 0x7fa6a006b2b0 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 >> v1:172.21.15.70:6789/0 conn(0x7fa6a006b2b0 legacy=0x7fa6a015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa68c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68c0047c0 0x7fa68c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 0x7fa68c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006bd80 con 0x7fa68c003970 2023-08-25T09:43:37.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006b7c0 con 0x7fa68c0047c0 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c00b980 con 0x7fa6a015e830 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a656b640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa68c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a656b640 1 -- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 msgr2=0x7fa68c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a656b640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a015e830 0x7fa68c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a656b640 1 -- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68c0047c0 msgr2=0x7fa68c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.773+0000 7fa6a5d6a640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68c0047c0 0x7fa68c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.774+0000 7fa6a656b640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa68c0047c0 0x7fa68c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.774+0000 7fa6a656b640 1 -- 172.21.15.70:0/262034699 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c003520 con 0x7fa68c003970 2023-08-25T09:43:37.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.775+0000 7fa6a656b640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa68c003d90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009ac0 tx=0x7fa68c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c273bf337065403 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:37.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.775+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c02c710 con 0x7fa68c003970 2023-08-25T09:43:37.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.775+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c02c870 con 0x7fa68c003970 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.775+0000 7fa6a5569640 1 -- 172.21.15.70:0/262034699 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c02cc00 con 0x7fa68c003970 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.775+0000 7fa6a87cf640 1 -- 172.21.15.70:0/262034699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 msgr2=0x7fa68c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.775+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/262034699 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa68c003d90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009ac0 tx=0x7fa68c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.776+0000 7fa6a6d6c640 1 -- 172.21.15.70:0/262034699 reap_dead start 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.776+0000 7fa6a87cf640 1 -- 172.21.15.70:0/262034699 shutdown_connections 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.776+0000 7fa6a87cf640 1 -- 172.21.15.70:0/262034699 >> 172.21.15.70:0/262034699 conn(0x7fa6a0069850 msgr2=0x7fa6a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:37.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.776+0000 7fa6a87cf640 1 -- 172.21.15.70:0/262034699 shutdown_connections 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.776+0000 7fa6a87cf640 1 -- 172.21.15.70:0/262034699 wait complete. 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.778+0000 7fa6a87cf640 1 Processor -- start 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.778+0000 7fa6a87cf640 1 -- start start 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a87cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa6a01f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a87cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a01533e0 0x7fa6a01f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a87cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a01ede80 0x7fa6a00b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a87cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0153860 con 0x7fa68c003970 2023-08-25T09:43:37.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a87cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a01588e0 con 0x7fa6a01533e0 2023-08-25T09:43:37.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a87cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a0158fa0 con 0x7fa6a01ede80 2023-08-25T09:43:37.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa6a01f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa6a01f1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60652/0 (socket says 172.21.15.70:60652) 2023-08-25T09:43:37.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 -- 172.21.15.70:0/2649174436 learned_addr learned my addr 172.21.15.70:0/2649174436 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:37.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 -- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a01ede80 msgr2=0x7fa6a00b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a01ede80 0x7fa6a00b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 -- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a01533e0 msgr2=0x7fa6a01f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a5d6a640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a01533e0 0x7fa6a01f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a01533e0 0x7fa6a01f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.779+0000 7fa6a656b640 1 -- 172.21.15.70:0/2649174436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a015ca20 con 0x7fa68c003970 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.780+0000 7fa6a656b640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa6a01f1410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa69c02c2b0 tx=0x7fa69c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.780+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c02de90 con 0x7fa68c003970 2023-08-25T09:43:37.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.780+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c003610 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.780+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0039c0 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.780+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a01f11b0 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.780+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01f1cd0 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.781+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a004e420 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.782+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa69c003b20 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.783+0000 7fa696ffd640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa68406a050 0x7fa68406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.783+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7fa69c0aa8c0 con 0x7fa68c003970 2023-08-25T09:43:37.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.783+0000 7fa6a5d6a640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa68406a050 0x7fa68406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:37.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.784+0000 7fa6a5d6a640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa68406a050 0x7fa68406c510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa690001eb0 tx=0x7fa690009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:37.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.787+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa69c075ed0 con 0x7fa68c003970 2023-08-25T09:43:37.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.976+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6a01548e0 con 0x7fa68c003970 2023-08-25T09:43:37.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.978+0000 7fa696ffd640 1 -- 172.21.15.70:0/2649174436 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18693 (secure 0 0 0) 0x7fa69c0794f0 con 0x7fa68c003970 2023-08-25T09:43:37.980 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:37.980 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:35.328919+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":1}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.38","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:37.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa68406a050 msgr2=0x7fa68406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa68406a050 0x7fa68406c510 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa690001eb0 tx=0x7fa690009040 comp rx=0 tx=0).stop 2023-08-25T09:43:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 msgr2=0x7fa6a01f1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa6a01f1410 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fa69c02c2b0 tx=0x7fa69c004730 comp rx=0 tx=0).stop 2023-08-25T09:43:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 shutdown_connections 2023-08-25T09:43:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa68406a050 0x7fa68406c510 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6a01ede80 0x7fa6a00b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa6a01533e0 0x7fa6a01f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 --2- 172.21.15.70:0/2649174436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa68c003970 0x7fa6a01f1410 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 >> 172.21.15.70:0/2649174436 conn(0x7fa6a0069850 msgr2=0x7fa6a00b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 shutdown_connections 2023-08-25T09:43:37.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:37.981+0000 7fa6a87cf640 1 -- 172.21.15.70:0/2649174436 wait complete. 2023-08-25T09:43:38.002 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:43:38.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:43:38.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.214+0000 7f60435fd640 1 Processor -- start 2023-08-25T09:43:38.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.214+0000 7f60435fd640 1 -- start start 2023-08-25T09:43:38.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f60435fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f603c0b1ff0 0x7f603c0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f60435fd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c0b1b60 con 0x7f603c15ec10 2023-08-25T09:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f60435fd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c06b7c0 con 0x7f603c15e830 2023-08-25T09:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f60435fd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c06bd80 con 0x7f603c0b1ff0 2023-08-25T09:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f6040b98640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f603c15ec10 0x7f603c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:48144/0 (socket says 172.21.15.70:48144) 2023-08-25T09:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f6040b98640 1 -- 172.21.15.70:0/1647090664 learned_addr learned my addr 172.21.15.70:0/1647090664 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:38.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.216+0000 7f6041b9a640 1 --2- 172.21.15.70:0/1647090664 >> v2:172.21.15.70:3300/0 conn(0x7f603c0b1ff0 0x7f603c0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.217+0000 7f6041b9a640 1 -- 172.21.15.70:0/1647090664 >> v1:172.21.15.155:6789/0 conn(0x7f603c15e830 legacy=0x7f603c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.217+0000 7f6041b9a640 1 -- 172.21.15.70:0/1647090664 >> v1:172.21.15.70:6789/0 conn(0x7f603c15ec10 legacy=0x7f603c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.217+0000 7f6033fff640 1 -- 172.21.15.70:0/1647090664 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905996956 0 0) 0x7f603c0b1b60 con 0x7f603c15ec10 2023-08-25T09:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.217+0000 7f6041b9a640 1 -- 172.21.15.70:0/1647090664 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c06baa0 con 0x7f603c0b1ff0 2023-08-25T09:43:38.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.218+0000 7f6041b9a640 1 --2- 172.21.15.70:0/1647090664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b2440 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6034009d90 tx=0x7f603402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfe4116f303f0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.218+0000 7f6033fff640 1 -- 172.21.15.70:0/1647090664 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f603402cd00 con 0x7f603c0b1ff0 2023-08-25T09:43:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.218+0000 7f6033fff640 1 -- 172.21.15.70:0/1647090664 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f603402ce60 con 0x7f603c0b1ff0 2023-08-25T09:43:38.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.218+0000 7f6033fff640 1 -- 172.21.15.70:0/1647090664 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034034660 con 0x7f603c0b1ff0 2023-08-25T09:43:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 -- 172.21.15.70:0/1647090664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 msgr2=0x7f603c0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 --2- 172.21.15.70:0/1647090664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b2440 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6034009d90 tx=0x7f603402bf50 comp rx=0 tx=0).stop 2023-08-25T09:43:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 -- 172.21.15.70:0/1647090664 shutdown_connections 2023-08-25T09:43:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 --2- 172.21.15.70:0/1647090664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b2440 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 -- 172.21.15.70:0/1647090664 >> 172.21.15.70:0/1647090664 conn(0x7f603c069850 msgr2=0x7f603c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 -- 172.21.15.70:0/1647090664 shutdown_connections 2023-08-25T09:43:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.219+0000 7f60435fd640 1 -- 172.21.15.70:0/1647090664 wait complete. 2023-08-25T09:43:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.221+0000 7f60435fd640 1 Processor -- start 2023-08-25T09:43:38.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.221+0000 7f60435fd640 1 -- start start 2023-08-25T09:43:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f60435fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f60435fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c15e830 0x7f603c1f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f60435fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c15ec10 0x7f603c1f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f60435fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0ad770 con 0x7f603c0b1ff0 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f60435fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0af080 con 0x7f603c15ec10 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f60435fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0b15f0 con 0x7f603c15e830 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60668/0 (socket says 172.21.15.70:60668) 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 -- 172.21.15.70:0/3781978483 learned_addr learned my addr 172.21.15.70:0/3781978483 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 -- 172.21.15.70:0/3781978483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c15e830 msgr2=0x7f603c1f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c15e830 0x7f603c1f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 -- 172.21.15.70:0/3781978483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c15ec10 msgr2=0x7f603c1f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c15ec10 0x7f603c1f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.222+0000 7f6041399640 1 -- 172.21.15.70:0/3781978483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034009a70 con 0x7f603c0b1ff0 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.223+0000 7f6041399640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b10b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f603800c450 tx=0x7f603800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.223+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60380052d0 con 0x7f603c0b1ff0 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.223+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6038005430 con 0x7f603c0b1ff0 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.223+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038012870 con 0x7f603c0b1ff0 2023-08-25T09:43:38.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.223+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c0af320 con 0x7f603c0b1ff0 2023-08-25T09:43:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.223+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c0ad400 con 0x7f603c0b1ff0 2023-08-25T09:43:38.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.224+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c0ad250 con 0x7f603c0b1ff0 2023-08-25T09:43:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.225+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f60380129d0 con 0x7f603c0b1ff0 2023-08-25T09:43:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.226+0000 7f6031ffb640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f602006a050 0x7f602006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.226+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f6038088280 con 0x7f603c0b1ff0 2023-08-25T09:43:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.226+0000 7f6040b98640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f602006a050 0x7f602006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.227+0000 7f6040b98640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f602006a050 0x7f602006c510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c0079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:38.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.230+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6038053800 con 0x7f603c0b1ff0 2023-08-25T09:43:38.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.457+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f603c0af840 con 0x7f603c0b1ff0 2023-08-25T09:43:38.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.459+0000 7f6031ffb640 1 -- 172.21.15.70:0/3781978483 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f6038056e20 con 0x7f603c0b1ff0 2023-08-25T09:43:38.462 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:38.462 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:43:38.463 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:43:38.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f602006a050 msgr2=0x7f602006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f602006a050 0x7f602006c510 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f602c000c00 tx=0x7f602c0079c0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 msgr2=0x7f603c0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b10b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f603800c450 tx=0x7f603800c910 comp rx=0 tx=0).stop 2023-08-25T09:43:38.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 shutdown_connections 2023-08-25T09:43:38.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f602006a050 0x7f602006c510 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f603c15ec10 0x7f603c1f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f603c15e830 0x7f603c1f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 --2- 172.21.15.70:0/3781978483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f603c0b1ff0 0x7f603c0b10b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 >> 172.21.15.70:0/3781978483 conn(0x7f603c069850 msgr2=0x7f603c15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:38.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.461+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 shutdown_connections 2023-08-25T09:43:38.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.462+0000 7f60435fd640 1 -- 172.21.15.70:0/3781978483 wait complete. 2023-08-25T09:43:38.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:43:38.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.684+0000 7f4551ad7640 1 Processor -- start 2023-08-25T09:43:38.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.684+0000 7f4551ad7640 1 -- start start 2023-08-25T09:43:38.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.685+0000 7f4551ad7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f454c15a8e0 0x7f454c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.685+0000 7f4551ad7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f454c15acc0 0x7f454c06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f4551ad7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f4551ad7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c06fdd0 con 0x7f454c15acc0 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f4551ad7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c0cb850 con 0x7f454c15a8e0 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f4551ad7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0cbb30 con 0x7f454c070500 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f454c15acc0 0x7f454c06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f454c15acc0 0x7f454c06aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60680/0 (socket says 172.21.15.70:60680) 2023-08-25T09:43:38.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 -- 172.21.15.70:0/922914924 learned_addr learned my addr 172.21.15.70:0/922914924 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454bfff640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 -- 172.21.15.70:0/922914924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 msgr2=0x7f454c072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 -- 172.21.15.70:0/922914924 >> v2:172.21.15.155:3300/0 conn(0x7f454c15a8e0 msgr2=0x7f454c0cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:38.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 --2- 172.21.15.70:0/922914924 >> v2:172.21.15.155:3300/0 conn(0x7f454c15a8e0 0x7f454c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454affd640 1 -- 172.21.15.70:0/922914924 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0cbe10 con 0x7f454c15acc0 2023-08-25T09:43:38.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.686+0000 7f454bfff640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:38.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.688+0000 7f454affd640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c06aa70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f454c0738e0 tx=0x7f453402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9cf883f48b9612d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:38.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.688+0000 7f454a7fc640 1 -- 172.21.15.70:0/922914924 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534031880 con 0x7f454c15acc0 2023-08-25T09:43:38.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.688+0000 7f454a7fc640 1 -- 172.21.15.70:0/922914924 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45340319e0 con 0x7f454c15acc0 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.688+0000 7f454a7fc640 1 -- 172.21.15.70:0/922914924 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534031d70 con 0x7f454c15acc0 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.688+0000 7f4551ad7640 1 -- 172.21.15.70:0/922914924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 msgr2=0x7f454c06aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.688+0000 7f4551ad7640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c06aa70 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f454c0738e0 tx=0x7f453402b920 comp rx=0 tx=0).stop 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 -- 172.21.15.70:0/922914924 shutdown_connections 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 --2- 172.21.15.70:0/922914924 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c06aa70 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 --2- 172.21.15.70:0/922914924 >> v2:172.21.15.155:3300/0 conn(0x7f454c15a8e0 0x7f454c0cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 -- 172.21.15.70:0/922914924 >> 172.21.15.70:0/922914924 conn(0x7f454c0cf9c0 msgr2=0x7f454c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 -- 172.21.15.70:0/922914924 shutdown_connections 2023-08-25T09:43:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.689+0000 7f4551ad7640 1 -- 172.21.15.70:0/922914924 wait complete. 2023-08-25T09:43:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.691+0000 7f4551ad7640 1 Processor -- start 2023-08-25T09:43:38.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.691+0000 7f4551ad7640 1 -- start start 2023-08-25T09:43:38.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f4551ad7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f4551ad7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f454c15a8e0 0x7f454c0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f4551ad7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f4551ad7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b12b0 con 0x7f454c15acc0 2023-08-25T09:43:38.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f4551ad7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b1590 con 0x7f454c15a8e0 2023-08-25T09:43:38.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f4551ad7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c0b1870 con 0x7f454c070500 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60692/0 (socket says 172.21.15.70:60692) 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 -- 172.21.15.70:0/4224531528 learned_addr learned my addr 172.21.15.70:0/4224531528 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454b7fe640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454affd640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f454c15a8e0 0x7f454c0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 -- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 msgr2=0x7f454c0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 -- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f454c15a8e0 msgr2=0x7f454c0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.692+0000 7f454bfff640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f454c15a8e0 0x7f454c0c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f454bfff640 1 -- 172.21.15.70:0/4224531528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340094d0 con 0x7f454c15acc0 2023-08-25T09:43:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f454bfff640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c0ad200 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f454400cca0 tx=0x7f454400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400f680 con 0x7f454c15acc0 2023-08-25T09:43:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400f800 con 0x7f454c15acc0 2023-08-25T09:43:38.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400fbb0 con 0x7f454c15acc0 2023-08-25T09:43:38.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c0b1b50 con 0x7f454c15acc0 2023-08-25T09:43:38.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.693+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c0c7800 con 0x7f454c15acc0 2023-08-25T09:43:38.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.694+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c0b2070 con 0x7f454c15acc0 2023-08-25T09:43:38.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.695+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f454400b040 con 0x7f454c15acc0 2023-08-25T09:43:38.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.696+0000 7f4543fff640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f452c06a080 0x7f452c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:38.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.696+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f4544088fd0 con 0x7f454c15acc0 2023-08-25T09:43:38.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.696+0000 7f454b7fe640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f452c06a080 0x7f452c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:38.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.697+0000 7f454b7fe640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f452c06a080 0x7f452c06c540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b3b10 tx=0x7f453c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:38.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.701+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f454c0b2070 con 0x7f454c15acc0 2023-08-25T09:43:38.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.930+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f454c0b2070 con 0x7f454c15acc0 2023-08-25T09:43:38.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.932+0000 7f4543fff640 1 -- 172.21.15.70:0/4224531528 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f454c0b2070 con 0x7f454c15acc0 2023-08-25T09:43:38.933 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:38.933 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:42:24.770908+0000","modified":"2023-08-25T09:42:28.336751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6324},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6324":{"gid":6324,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1838015952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1838015952},{"type":"v1","addr":"172.21.15.70:6837","nonce":1838015952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-08-25T09:43:38.934 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 32 2023-08-25T09:43:38.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f452c06a080 msgr2=0x7f452c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f452c06a080 0x7f452c06c540 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f454c0b3b10 tx=0x7f453c008040 comp rx=0 tx=0).stop 2023-08-25T09:43:38.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 msgr2=0x7f454c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:38.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c0ad200 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f454400cca0 tx=0x7f454400a650 comp rx=0 tx=0).stop 2023-08-25T09:43:38.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 shutdown_connections 2023-08-25T09:43:38.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f452c06a080 0x7f452c06c540 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f454c15acc0 0x7f454c0ad200 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f454c15a8e0 0x7f454c0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 --2- 172.21.15.70:0/4224531528 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f454c070500 0x7f454c0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:38.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 >> 172.21.15.70:0/4224531528 conn(0x7f454c0cf9c0 msgr2=0x7f454c06b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:38.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 shutdown_connections 2023-08-25T09:43:38.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:38.934+0000 7f4551ad7640 1 -- 172.21.15.70:0/4224531528 wait complete. 2023-08-25T09:43:38.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:39.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.167+0000 7f02756f3640 1 Processor -- start 2023-08-25T09:43:39.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.167+0000 7f02756f3640 1 -- start start 2023-08-25T09:43:39.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f02756f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f02756f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f027015ec10 0x7f02700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f02756f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02700b1b60 con 0x7f027006b2b0 2023-08-25T09:43:39.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f02756f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027006bd80 con 0x7f027015ec10 2023-08-25T09:43:39.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f02756f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027006b7c0 con 0x7f027015e830 2023-08-25T09:43:39.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f027015ec10 0x7f02700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f027015ec10 0x7f02700b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37192/0 (socket says 172.21.15.70:37192) 2023-08-25T09:43:39.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 -- 172.21.15.70:0/666848787 learned_addr learned my addr 172.21.15.70:0/666848787 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:39.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 -- 172.21.15.70:0/666848787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 msgr2=0x7f027006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:39.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026effd640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 -- 172.21.15.70:0/666848787 >> v1:172.21.15.155:6789/0 conn(0x7f027006b2b0 legacy=0x7f027015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:39.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026f7fe640 1 -- 172.21.15.70:0/666848787 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027006baa0 con 0x7f027015ec10 2023-08-25T09:43:39.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.169+0000 7f026effd640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:39.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.171+0000 7f026f7fe640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027015ec10 0x7f02700b1ff0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0268009e50 tx=0x7f026802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168c0e89afed2aac server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:39.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.171+0000 7f026dffb640 1 -- 172.21.15.70:0/666848787 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026802ca70 con 0x7f027015ec10 2023-08-25T09:43:39.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.171+0000 7f026dffb640 1 -- 172.21.15.70:0/666848787 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026802cbd0 con 0x7f027015ec10 2023-08-25T09:43:39.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.171+0000 7f026dffb640 1 -- 172.21.15.70:0/666848787 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02680343f0 con 0x7f027015ec10 2023-08-25T09:43:39.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 -- 172.21.15.70:0/666848787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027015ec10 msgr2=0x7f02700b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027015ec10 0x7f02700b1ff0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f0268009e50 tx=0x7f026802bed0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 -- 172.21.15.70:0/666848787 shutdown_connections 2023-08-25T09:43:39.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027015ec10 0x7f02700b1ff0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 --2- 172.21.15.70:0/666848787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 -- 172.21.15.70:0/666848787 >> 172.21.15.70:0/666848787 conn(0x7f0270069850 msgr2=0x7f0270069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 -- 172.21.15.70:0/666848787 shutdown_connections 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.172+0000 7f02756f3640 1 -- 172.21.15.70:0/666848787 wait complete. 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.174+0000 7f02756f3640 1 Processor -- start 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.174+0000 7f02756f3640 1 -- start start 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f02756f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 0x7f02700ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f02756f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f02756f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f027015ec10 0x7f027015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f02756f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700add50 con 0x7f027006b2b0 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f02756f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027006f290 con 0x7f027015ec10 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f02756f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027006f570 con 0x7f027015e830 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 0x7f02700ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 0x7f02700ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37202/0 (socket says 172.21.15.70:37202) 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 -- 172.21.15.70:0/3637780809 learned_addr learned my addr 172.21.15.70:0/3637780809 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 -- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 msgr2=0x7f027015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026e7fc640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026f7fe640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f027015ec10 0x7f027015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027015cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 -- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f027015ec10 msgr2=0x7f027015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f027015ec10 0x7f027015d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.175+0000 7f026effd640 1 -- 172.21.15.70:0/3637780809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0268009b30 con 0x7f027006b2b0 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.176+0000 7f026effd640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 0x7f02700ad370 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f026000c7f0 tx=0x7f026000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:39.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.176+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260007a30 con 0x7f027006b2b0 2023-08-25T09:43:39.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.176+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0260007b90 con 0x7f027006b2b0 2023-08-25T09:43:39.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.176+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026000ce40 con 0x7f027006b2b0 2023-08-25T09:43:39.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.176+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027015ca20 con 0x7f027006b2b0 2023-08-25T09:43:39.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.176+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02700ae030 con 0x7f027006b2b0 2023-08-25T09:43:39.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.177+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f027004e420 con 0x7f027006b2b0 2023-08-25T09:43:39.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.178+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f02600113f0 con 0x7f027006b2b0 2023-08-25T09:43:39.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.179+0000 7f02577fe640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f025c06a050 0x7f025c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.179+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7f0260088850 con 0x7f027006b2b0 2023-08-25T09:43:39.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.179+0000 7f026e7fc640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f025c06a050 0x7f025c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.180+0000 7f026e7fc640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f025c06a050 0x7f025c06c510 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f027015d990 tx=0x7f0264009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:39.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.183+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0260053dd0 con 0x7f027006b2b0 2023-08-25T09:43:39.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.372+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f027006c620 con 0x7f027006b2b0 2023-08-25T09:43:39.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.376+0000 7f02577fe640 1 -- 172.21.15.70:0/3637780809 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18693 (secure 0 0 0) 0x7f02600573f0 con 0x7f027006b2b0 2023-08-25T09:43:39.380 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:39.381 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:35.328919+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:42:22.389960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-08-25T09:42:23.780164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":5}]},{"pgid":"10.1c","mappings":[{"from":2,"to":7}]},{"pgid":"10.23","mappings":[{"from":2,"to":1}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.38","mappings":[{"from":2,"to":7}]},{"pgid":"10.3c","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:39.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f025c06a050 msgr2=0x7f025c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f025c06a050 0x7f025c06c510 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f027015d990 tx=0x7f0264009040 comp rx=0 tx=0).stop 2023-08-25T09:43:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 msgr2=0x7f02700ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 0x7f02700ad370 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f026000c7f0 tx=0x7f026000cbf0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 shutdown_connections 2023-08-25T09:43:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f025c06a050 0x7f025c06c510 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f027015ec10 0x7f027015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f027015e830 0x7f027015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 --2- 172.21.15.70:0/3637780809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f027006b2b0 0x7f02700ad370 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 >> 172.21.15.70:0/3637780809 conn(0x7f0270069850 msgr2=0x7f0270159070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:39.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 shutdown_connections 2023-08-25T09:43:39.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.379+0000 7f02756f3640 1 -- 172.21.15.70:0/3637780809 wait complete. 2023-08-25T09:43:39.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:43:39.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.613+0000 7feaf4b98640 1 Processor -- start 2023-08-25T09:43:39.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.613+0000 7feaf4b98640 1 -- start start 2023-08-25T09:43:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feaf4b98640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7feaf015acd0 0x7feaf015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feaf4b98640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feaf00ca780 0x7feaf00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feaf4b98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00b3cf0 0x7feaf00b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feaf4b98640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf006c0e0 con 0x7feaf00ca780 2023-08-25T09:43:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feaf4b98640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf00b12b0 con 0x7feaf015acd0 2023-08-25T09:43:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feaf4b98640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf006ae20 con 0x7feaf00b3cf0 2023-08-25T09:43:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feae7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feaf00ca780 0x7feaf00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feae7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feaf00ca780 0x7feaf00cac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37214/0 (socket says 172.21.15.70:37214) 2023-08-25T09:43:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.615+0000 7feae7fff640 1 -- 172.21.15.70:0/2741890505 learned_addr learned my addr 172.21.15.70:0/2741890505 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:39.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feaee59c640 1 --2- 172.21.15.70:0/2741890505 >> v2:172.21.15.155:3300/0 conn(0x7feaf015acd0 0x7feaf015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feaeed9d640 1 --2- 172.21.15.70:0/2741890505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00b3cf0 0x7feaf00b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feae7fff640 1 -- 172.21.15.70:0/2741890505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00b3cf0 msgr2=0x7feaf00b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feae7fff640 1 --2- 172.21.15.70:0/2741890505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00b3cf0 0x7feaf00b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feae7fff640 1 -- 172.21.15.70:0/2741890505 >> v2:172.21.15.155:3300/0 conn(0x7feaf015acd0 msgr2=0x7feaf015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feae7fff640 1 --2- 172.21.15.70:0/2741890505 >> v2:172.21.15.155:3300/0 conn(0x7feaf015acd0 0x7feaf015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.616+0000 7feae7fff640 1 -- 172.21.15.70:0/2741890505 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf006b910 con 0x7feaf00ca780 2023-08-25T09:43:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.617+0000 7feae7fff640 1 --2- 172.21.15.70:0/2741890505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00ca780 0x7feaf00cac30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7feaf00714b0 tx=0x7fead802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe71d3314c29952 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.617+0000 7feaedd9b640 1 -- 172.21.15.70:0/2741890505 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80317d0 con 0x7feaf00ca780 2023-08-25T09:43:39.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.617+0000 7feaedd9b640 1 -- 172.21.15.70:0/2741890505 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead8031930 con 0x7feaf00ca780 2023-08-25T09:43:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.617+0000 7feaedd9b640 1 -- 172.21.15.70:0/2741890505 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8031cc0 con 0x7feaf00ca780 2023-08-25T09:43:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 -- 172.21.15.70:0/2741890505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00ca780 msgr2=0x7feaf00cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 --2- 172.21.15.70:0/2741890505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00ca780 0x7feaf00cac30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7feaf00714b0 tx=0x7fead802b870 comp rx=0 tx=0).stop 2023-08-25T09:43:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 -- 172.21.15.70:0/2741890505 shutdown_connections 2023-08-25T09:43:39.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 --2- 172.21.15.70:0/2741890505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00b3cf0 0x7feaf00b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 --2- 172.21.15.70:0/2741890505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00ca780 0x7feaf00cac30 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 --2- 172.21.15.70:0/2741890505 >> v2:172.21.15.155:3300/0 conn(0x7feaf015acd0 0x7feaf015b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 -- 172.21.15.70:0/2741890505 >> 172.21.15.70:0/2741890505 conn(0x7feaf00ce760 msgr2=0x7feaf00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:39.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 -- 172.21.15.70:0/2741890505 shutdown_connections 2023-08-25T09:43:39.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.618+0000 7feaf4b98640 1 -- 172.21.15.70:0/2741890505 wait complete. 2023-08-25T09:43:39.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.620+0000 7feaf4b98640 1 Processor -- start 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.620+0000 7feaf4b98640 1 -- start start 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaf4b98640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 0x7feaf0158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaf4b98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00ca780 0x7feaf013d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaf4b98640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf015acd0 0x7feaf01568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaf4b98640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0155630 con 0x7feaf00b3cf0 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaf4b98640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0155070 con 0x7feaf015acd0 2023-08-25T09:43:39.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaf4b98640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf0155350 con 0x7feaf00ca780 2023-08-25T09:43:39.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 0x7feaf0158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 0x7feaf0158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37222/0 (socket says 172.21.15.70:37222) 2023-08-25T09:43:39.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 -- 172.21.15.70:0/3811929393 learned_addr learned my addr 172.21.15.70:0/3811929393 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:39.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaeed9d640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf015acd0 0x7feaf01568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feae7fff640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00ca780 0x7feaf013d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 -- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00ca780 msgr2=0x7feaf013d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00ca780 0x7feaf013d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 -- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf015acd0 msgr2=0x7feaf01568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.621+0000 7feaee59c640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf015acd0 0x7feaf01568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feaee59c640 1 -- 172.21.15.70:0/3811929393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80094d0 con 0x7feaf00b3cf0 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feaee59c640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 0x7feaf0158240 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7feae000d7d0 tx=0x7feae000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:39.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae00096c0 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0009820 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0009bd0 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf013e480 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.622+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf013ddd0 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.624+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf004e420 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.624+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feae0009d30 con 0x7feaf00b3cf0 2023-08-25T09:43:39.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.626+0000 7feae6ffd640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fead006a050 0x7fead006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.626+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6265+0+0 (secure 0 0 0) 0x7feae0088ed0 con 0x7feaf00b3cf0 2023-08-25T09:43:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.626+0000 7feae7fff640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fead006a050 0x7fead006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:39.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.627+0000 7feae7fff640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fead006a050 0x7fead006c510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fead8036000 tx=0x7fead803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:39.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.630+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feae0057f70 con 0x7feaf00b3cf0 2023-08-25T09:43:39.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.854+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7feaf006bec0 con 0x7feaf00b3cf0 2023-08-25T09:43:39.856 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:43:39.855+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:43:39.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.867+0000 7feae6ffd640 1 -- 172.21.15.70:0/3811929393 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7feae0057b00 con 0x7feaf00b3cf0 2023-08-25T09:43:39.869 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:43:39.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fead006a050 msgr2=0x7fead006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fead006a050 0x7fead006c510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fead8036000 tx=0x7fead803d040 comp rx=0 tx=0).stop 2023-08-25T09:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 msgr2=0x7feaf0158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 0x7feaf0158240 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7feae000d7d0 tx=0x7feae000dc90 comp rx=0 tx=0).stop 2023-08-25T09:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 shutdown_connections 2023-08-25T09:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fead006a050 0x7fead006c510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feaf015acd0 0x7feaf01568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feaf00ca780 0x7feaf013d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 --2- 172.21.15.70:0/3811929393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feaf00b3cf0 0x7feaf0158240 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 >> 172.21.15.70:0/3811929393 conn(0x7feaf00ce760 msgr2=0x7feaf00cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 shutdown_connections 2023-08-25T09:43:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:39.871+0000 7feaf4b98640 1 -- 172.21.15.70:0/3811929393 wait complete. 2023-08-25T09:43:39.885 INFO:tasks.ceph.mds.c.smithi070.stderr: -7655> 2023-08-25T09:42:29.277+0000 7fd58a2aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:43:39.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:43:39.942 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:43:40.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.093+0000 7f2c8b184640 1 Processor -- start 2023-08-25T09:43:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.093+0000 7f2c8b184640 1 -- start start 2023-08-25T09:43:40.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c8b184640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2c8415cbb0 0x7f2c8406ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c8b184640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c841607e0 0x7f2c840ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c8b184640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c840cea30 0x7f2c840ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c8b184640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c840ceec0 con 0x7f2c840cea30 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c8b184640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8406c8b0 con 0x7f2c8415cbb0 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c8b184640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8406b250 con 0x7f2c841607e0 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c89721640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c840cea30 0x7f2c840ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:40.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c89721640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c840cea30 0x7f2c840ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37242/0 (socket says 172.21.15.70:37242) 2023-08-25T09:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.095+0000 7f2c89721640 1 -- 172.21.15.70:0/2481472762 learned_addr learned my addr 172.21.15.70:0/2481472762 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:40.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c83fff640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c841607e0 0x7f2c840ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c88f20640 1 --2- 172.21.15.70:0/2481472762 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415cbb0 0x7f2c8406ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c89721640 1 -- 172.21.15.70:0/2481472762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c841607e0 msgr2=0x7f2c840ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c89721640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c841607e0 0x7f2c840ce4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c89721640 1 -- 172.21.15.70:0/2481472762 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415cbb0 msgr2=0x7f2c8406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c89721640 1 --2- 172.21.15.70:0/2481472762 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415cbb0 0x7f2c8406ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c89721640 1 -- 172.21.15.70:0/2481472762 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8406b510 con 0x7f2c840cea30 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c83fff640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c841607e0 0x7f2c840ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.096+0000 7f2c88f20640 1 --2- 172.21.15.70:0/2481472762 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415cbb0 0x7f2c8406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.097+0000 7f2c89721640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c840cea30 0x7f2c840ac870 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2c840cf460 tx=0x7f2c7c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6fe03ca2160e368 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.097+0000 7f2c837fe640 1 -- 172.21.15.70:0/2481472762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c02cb00 con 0x7f2c840cea30 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.097+0000 7f2c837fe640 1 -- 172.21.15.70:0/2481472762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c02cc60 con 0x7f2c840cea30 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.097+0000 7f2c837fe640 1 -- 172.21.15.70:0/2481472762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c0343f0 con 0x7f2c840cea30 2023-08-25T09:43:40.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.097+0000 7f2c8b184640 1 -- 172.21.15.70:0/2481472762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c840cea30 msgr2=0x7f2c840ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:40.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.097+0000 7f2c8b184640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c840cea30 0x7f2c840ac870 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2c840cf460 tx=0x7f2c7c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:43:40.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 -- 172.21.15.70:0/2481472762 shutdown_connections 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c840cea30 0x7f2c840ac870 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 --2- 172.21.15.70:0/2481472762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c841607e0 0x7f2c840ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 --2- 172.21.15.70:0/2481472762 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415cbb0 0x7f2c8406ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 -- 172.21.15.70:0/2481472762 >> 172.21.15.70:0/2481472762 conn(0x7f2c84069850 msgr2=0x7f2c84069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 -- 172.21.15.70:0/2481472762 shutdown_connections 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.098+0000 7f2c8b184640 1 -- 172.21.15.70:0/2481472762 wait complete. 2023-08-25T09:43:40.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.099+0000 7f2c8b184640 1 Processor -- start 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.099+0000 7f2c8b184640 1 -- start start 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.100+0000 7f2c8b184640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c840cea30 0x7f2c84152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.100+0000 7f2c8b184640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415cbb0 0x7f2c84155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.100+0000 7f2c8b184640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 0x7f2c841f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.100+0000 7f2c8b184640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84155a30 con 0x7f2c841607e0 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.100+0000 7f2c8b184640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84152ba0 con 0x7f2c840cea30 2023-08-25T09:43:40.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.100+0000 7f2c8b184640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c841590e0 con 0x7f2c8415cbb0 2023-08-25T09:43:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 0x7f2c841f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 0x7f2c841f1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37248/0 (socket says 172.21.15.70:37248) 2023-08-25T09:43:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 -- 172.21.15.70:0/972215620 learned_addr learned my addr 172.21.15.70:0/972215620 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 -- 172.21.15.70:0/972215620 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415cbb0 msgr2=0x7f2c84155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:40.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415cbb0 0x7f2c84155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 -- 172.21.15.70:0/972215620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c840cea30 msgr2=0x7f2c84152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:40.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c840cea30 0x7f2c84152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 -- 172.21.15.70:0/972215620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c009b30 con 0x7f2c841607e0 2023-08-25T09:43:40.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.101+0000 7f2c89721640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 0x7f2c841f1cc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c036040 tx=0x7f2c7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:40.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.102+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c009510 con 0x7f2c841607e0 2023-08-25T09:43:40.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.102+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c03e450 con 0x7f2c841607e0 2023-08-25T09:43:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.102+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c03e800 con 0x7f2c841607e0 2023-08-25T09:43:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.102+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c841593c0 con 0x7f2c841607e0 2023-08-25T09:43:40.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.102+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c84159a10 con 0x7f2c841607e0 2023-08-25T09:43:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.103+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8404e420 con 0x7f2c841607e0 2023-08-25T09:43:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.104+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2c7c03e960 con 0x7f2c841607e0 2023-08-25T09:43:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.105+0000 7f2c817fa640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a080 0x7f2c6806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:40.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.105+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6351+0+0 (secure 0 0 0) 0x7f2c7c0ab570 con 0x7f2c841607e0 2023-08-25T09:43:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.105+0000 7f2c88f20640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a080 0x7f2c6806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.105+0000 7f2c88f20640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a080 0x7f2c6806c540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006fd0 tx=0x7f2c78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:40.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.109+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c7c076d00 con 0x7f2c841607e0 2023-08-25T09:43:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.335+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2c840cba30 con 0x7f2c841607e0 2023-08-25T09:43:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.866+0000 7f2c817fa640 1 -- 172.21.15.70:0/972215620 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2c840cba30 con 0x7f2c841607e0 2023-08-25T09:43:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a080 msgr2=0x7f2c6806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a080 0x7f2c6806c540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c78006fd0 tx=0x7f2c78008040 comp rx=0 tx=0).stop 2023-08-25T09:43:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 msgr2=0x7f2c841f1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 0x7f2c841f1cc0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c036040 tx=0x7f2c7c004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:40.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 shutdown_connections 2023-08-25T09:43:40.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a080 0x7f2c6806c540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c841607e0 0x7f2c841f1cc0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415cbb0 0x7f2c84155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 --2- 172.21.15.70:0/972215620 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c840cea30 0x7f2c84152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 >> 172.21.15.70:0/972215620 conn(0x7f2c84069850 msgr2=0x7f2c84154200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 shutdown_connections 2023-08-25T09:43:40.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:40.870+0000 7f2c8b184640 1 -- 172.21.15.70:0/972215620 wait complete. 2023-08-25T09:43:40.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:43:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.158+0000 7fe467083640 1 Processor -- start 2023-08-25T09:43:41.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.158+0000 7fe467083640 1 -- start start 2023-08-25T09:43:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe467083640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe460155120 0x7fe4600cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe467083640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe4600ca9f0 0x7fe46015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe467083640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe46015e6f0 con 0x7fe4600cd890 2023-08-25T09:43:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe467083640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4600cdd40 con 0x7fe460155120 2023-08-25T09:43:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe467083640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4600caf30 con 0x7fe4600ca9f0 2023-08-25T09:43:41.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe464e1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe460155120 0x7fe4600cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:41.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe464e1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe460155120 0x7fe4600cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37262/0 (socket says 172.21.15.70:37262) 2023-08-25T09:43:41.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe45ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe4600cd890 0x7fe4600ca2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33164/0 (socket says 172.21.15.70:33164) 2023-08-25T09:43:41.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.160+0000 7fe464e1f640 1 -- 172.21.15.70:0/4245653129 learned_addr learned my addr 172.21.15.70:0/4245653129 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.161+0000 7fe464e1f640 1 -- 172.21.15.70:0/4245653129 >> v2:172.21.15.155:3300/0 conn(0x7fe4600ca9f0 msgr2=0x7fe46015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.161+0000 7fe464e1f640 1 --2- 172.21.15.70:0/4245653129 >> v2:172.21.15.155:3300/0 conn(0x7fe4600ca9f0 0x7fe46015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.161+0000 7fe464e1f640 1 -- 172.21.15.70:0/4245653129 >> v1:172.21.15.70:6789/0 conn(0x7fe4600cd890 legacy=0x7fe4600ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.161+0000 7fe464e1f640 1 -- 172.21.15.70:0/4245653129 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46015e9d0 con 0x7fe460155120 2023-08-25T09:43:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.161+0000 7fe45f7fe640 1 -- 172.21.15.70:0/4245653129 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861204168 0 0) 0x7fe46015e6f0 con 0x7fe4600cd890 2023-08-25T09:43:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.162+0000 7fe464e1f640 1 --2- 172.21.15.70:0/4245653129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe460155120 0x7fe4600cd350 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe46006ebb0 tx=0x7fe45402b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79c2de2dea3eaae server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.162+0000 7fe45f7fe640 1 -- 172.21.15.70:0/4245653129 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454030880 con 0x7fe460155120 2023-08-25T09:43:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.162+0000 7fe45f7fe640 1 -- 172.21.15.70:0/4245653129 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4540309e0 con 0x7fe460155120 2023-08-25T09:43:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.162+0000 7fe45f7fe640 1 -- 172.21.15.70:0/4245653129 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454030d70 con 0x7fe460155120 2023-08-25T09:43:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 -- 172.21.15.70:0/4245653129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe460155120 msgr2=0x7fe4600cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 --2- 172.21.15.70:0/4245653129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe460155120 0x7fe4600cd350 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fe46006ebb0 tx=0x7fe45402b9c0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 -- 172.21.15.70:0/4245653129 shutdown_connections 2023-08-25T09:43:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 --2- 172.21.15.70:0/4245653129 >> v2:172.21.15.155:3300/0 conn(0x7fe4600ca9f0 0x7fe46015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 --2- 172.21.15.70:0/4245653129 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe460155120 0x7fe4600cd350 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 -- 172.21.15.70:0/4245653129 >> 172.21.15.70:0/4245653129 conn(0x7fe460069c70 msgr2=0x7fe46006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 -- 172.21.15.70:0/4245653129 shutdown_connections 2023-08-25T09:43:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.163+0000 7fe467083640 1 -- 172.21.15.70:0/4245653129 wait complete. 2023-08-25T09:43:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.165+0000 7fe467083640 1 Processor -- start 2023-08-25T09:43:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.165+0000 7fe467083640 1 -- start start 2023-08-25T09:43:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe467083640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 0x7fe460142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe467083640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4600cd890 0x7fe4601456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe467083640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe460155120 0x7fe4601f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe467083640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe460144ca0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe467083640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4601429c0 con 0x7fe4600cd890 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe467083640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe460145bf0 con 0x7fe460155120 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe464e1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 0x7fe460142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe464e1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 0x7fe460142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37264/0 (socket says 172.21.15.70:37264) 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.166+0000 7fe464e1f640 1 -- 172.21.15.70:0/953428372 learned_addr learned my addr 172.21.15.70:0/953428372 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe464e1f640 1 -- 172.21.15.70:0/953428372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe460155120 msgr2=0x7fe4601f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe464e1f640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe460155120 0x7fe4601f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe464e1f640 1 -- 172.21.15.70:0/953428372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4600cd890 msgr2=0x7fe4601456b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe464e1f640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4600cd890 0x7fe4601456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe464e1f640 1 -- 172.21.15.70:0/953428372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540094d0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe464e1f640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 0x7fe460142480 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe454000c00 tx=0x7fe454004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454039d60 con 0x7fe4600ca9f0 2023-08-25T09:43:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe454043400 con 0x7fe4600ca9f0 2023-08-25T09:43:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4540437b0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.167+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4601f5cc0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.168+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe460144eb0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.168+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4601f61e0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.169+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe454043910 con 0x7fe4600ca9f0 2023-08-25T09:43:41.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.170+0000 7fe45d7fa640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe44406a080 0x7fe44406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:41.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.170+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6351+0+0 (secure 0 0 0) 0x7fe45403e070 con 0x7fe4600ca9f0 2023-08-25T09:43:41.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.171+0000 7fe45ffff640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe44406a080 0x7fe44406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:41.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.171+0000 7fe45ffff640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe44406a080 0x7fe44406c540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe44c007fd0 tx=0x7fe44c007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:41.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.175+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe4601f61e0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.365+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe4601f61e0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.873+0000 7fe45d7fa640 1 -- 172.21.15.70:0/953428372 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe4601f61e0 con 0x7fe4600ca9f0 2023-08-25T09:43:41.877 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:43:41.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe44406a080 msgr2=0x7fe44406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe44406a080 0x7fe44406c540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe44c007fd0 tx=0x7fe44c007920 comp rx=0 tx=0).stop 2023-08-25T09:43:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 msgr2=0x7fe460142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:41.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 0x7fe460142480 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe454000c00 tx=0x7fe454004730 comp rx=0 tx=0).stop 2023-08-25T09:43:41.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 shutdown_connections 2023-08-25T09:43:41.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe44406a080 0x7fe44406c540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe460155120 0x7fe4601f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4600cd890 0x7fe4601456b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 --2- 172.21.15.70:0/953428372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4600ca9f0 0x7fe460142480 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:41.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.879+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 >> 172.21.15.70:0/953428372 conn(0x7fe460069c70 msgr2=0x7fe4600654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:41.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.880+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 shutdown_connections 2023-08-25T09:43:41.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:41.880+0000 7fe467083640 1 -- 172.21.15.70:0/953428372 wait complete. 2023-08-25T09:43:41.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:43:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.112+0000 7f5923534640 1 Processor -- start 2023-08-25T09:43:42.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.113+0000 7f5923534640 1 -- start start 2023-08-25T09:43:42.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.114+0000 7f5923534640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f591c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.114+0000 7f5923534640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c15d190 con 0x7f591c15cdb0 2023-08-25T09:43:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.114+0000 7f5923534640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c0709b0 con 0x7f591c0b6f00 2023-08-25T09:43:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.114+0000 7f5923534640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c070c90 con 0x7f591c0cd8e0 2023-08-25T09:43:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.114+0000 7f5922532640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f591c15cdb0 0x7f591c0cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33172/0 (socket says 172.21.15.70:33172) 2023-08-25T09:43:42.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.114+0000 7f5922532640 1 -- 172.21.15.70:0/977470274 learned_addr learned my addr 172.21.15.70:0/977470274 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350517153 0 0) 0x7f590c001110 con 0x7f591c15cdb0 2023-08-25T09:43:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921d31640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f591c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:42.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5904003520 con 0x7f591c15cdb0 2023-08-25T09:43:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3083990856 0 0) 0x7f5904003520 con 0x7f591c15cdb0 2023-08-25T09:43:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 msgr2=0x7f591c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f591c0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:42.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 >> v1:172.21.15.155:6789/0 conn(0x7f591c0b6f00 legacy=0x7f591c0d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c070f70 con 0x7f591c15cdb0 2023-08-25T09:43:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921d31640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f591c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:43:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f590c001800 con 0x7f591c15cdb0 2023-08-25T09:43:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.115+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 >> v1:172.21.15.70:6789/0 conn(0x7f591c15cdb0 legacy=0x7f591c0cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:42.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921530640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f59040039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921530640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5904003f90 0x7f59040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921530640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f5904005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c070c90 con 0x7f5904004d00 2023-08-25T09:43:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590400bae0 con 0x7f5904003f90 2023-08-25T09:43:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590400bca0 con 0x7f591c0cd8e0 2023-08-25T09:43:42.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922d33640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f5904005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:42.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5921d31640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5904003f90 0x7f59040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:42.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922532640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f59040039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:42.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922d33640 1 -- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 msgr2=0x7f59040039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:42.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922d33640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0cd8e0 0x7f59040039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:42.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922d33640 1 -- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5904003f90 msgr2=0x7f59040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:42.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922d33640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5904003f90 0x7f59040047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:42.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.116+0000 7f5922d33640 1 -- 172.21.15.70:0/977470274 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5904003520 con 0x7f5904004d00 2023-08-25T09:43:42.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.117+0000 7f5922d33640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f5904005180 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5914001750 tx=0x7f59040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b2ed90974c5159 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:42.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59140052d0 con 0x7f5904004d00 2023-08-25T09:43:42.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591402e050 con 0x7f5904004d00 2023-08-25T09:43:42.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5921530640 1 -- 172.21.15.70:0/977470274 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914003830 con 0x7f5904004d00 2023-08-25T09:43:42.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5923534640 1 -- 172.21.15.70:0/977470274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 msgr2=0x7f5904005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:42.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5923534640 1 --2- 172.21.15.70:0/977470274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f5904005180 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5914001750 tx=0x7f59040057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:42.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5922d33640 1 -- 172.21.15.70:0/977470274 reap_dead start 2023-08-25T09:43:42.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5923534640 1 -- 172.21.15.70:0/977470274 shutdown_connections 2023-08-25T09:43:42.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5923534640 1 -- 172.21.15.70:0/977470274 >> 172.21.15.70:0/977470274 conn(0x7f591c0d28e0 msgr2=0x7f591c0d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:42.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5923534640 1 -- 172.21.15.70:0/977470274 shutdown_connections 2023-08-25T09:43:42.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.118+0000 7f5923534640 1 -- 172.21.15.70:0/977470274 wait complete. 2023-08-25T09:43:42.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.120+0000 7f5923534640 1 Processor -- start 2023-08-25T09:43:42.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.120+0000 7f5923534640 1 -- start start 2023-08-25T09:43:42.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5923534640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f591c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5923534640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0b1ed0 0x7f591c0b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5923534640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f591c0b2350 0x7f591c0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5923534640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c15a730 con 0x7f5904004d00 2023-08-25T09:43:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5923534640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0b2890 con 0x7f591c0b2350 2023-08-25T09:43:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5923534640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0b2b70 con 0x7f591c0b1ed0 2023-08-25T09:43:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.121+0000 7f5922532640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f591c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f591c15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37270/0 (socket says 172.21.15.70:37270) 2023-08-25T09:43:42.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 -- 172.21.15.70:0/1000644982 learned_addr learned my addr 172.21.15.70:0/1000644982 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 -- 172.21.15.70:0/1000644982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0b1ed0 msgr2=0x7f591c0b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0b1ed0 0x7f591c0b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 -- 172.21.15.70:0/1000644982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f591c0b2350 msgr2=0x7f591c0b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f591c0b2350 0x7f591c0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:42.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 -- 172.21.15.70:0/1000644982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0b2e50 con 0x7f5904004d00 2023-08-25T09:43:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5922532640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f591c15a1f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f590c00d810 tx=0x7f590c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.122+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0074e0 con 0x7f5904004d00 2023-08-25T09:43:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.123+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c007640 con 0x7f5904004d00 2023-08-25T09:43:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.123+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c0049e0 con 0x7f5904004d00 2023-08-25T09:43:42.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.123+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c0b3130 con 0x7f5904004d00 2023-08-25T09:43:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.123+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c1f5aa0 con 0x7f5904004d00 2023-08-25T09:43:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.124+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c067a50 con 0x7f5904004d00 2023-08-25T09:43:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.125+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f590c004b40 con 0x7f5904004d00 2023-08-25T09:43:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.126+0000 7f5912ffd640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f590806a050 0x7f590806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:42.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.126+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5913+0+0 (secure 0 0 0) 0x7f590c087b80 con 0x7f5904004d00 2023-08-25T09:43:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.126+0000 7f5921d31640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f590806a050 0x7f590806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:42.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.127+0000 7f5921d31640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f590806a050 0x7f590806c510 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5918003fd0 tx=0x7f5918009500 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:42.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.130+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f590c0533a0 con 0x7f5904004d00 2023-08-25T09:43:42.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:42.318+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f591c15e310 con 0x7f5904004d00 2023-08-25T09:43:43.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.298+0000 7f5912ffd640 1 -- 172.21.15.70:0/1000644982 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) v1 ==== 169+0+0 (secure 0 0 0) 0x7f591c15e310 con 0x7f5904004d00 2023-08-25T09:43:43.302 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:43:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f590806a050 msgr2=0x7f590806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f590806a050 0x7f590806c510 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5918003fd0 tx=0x7f5918009500 comp rx=0 tx=0).stop 2023-08-25T09:43:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 msgr2=0x7f591c15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f591c15a1f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f590c00d810 tx=0x7f590c00dcd0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 shutdown_connections 2023-08-25T09:43:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f590806a050 0x7f590806c510 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f591c0b2350 0x7f591c0b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591c0b1ed0 0x7f591c0b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 --2- 172.21.15.70:0/1000644982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5904004d00 0x7f591c15a1f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.305+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 >> 172.21.15.70:0/1000644982 conn(0x7f591c0d28e0 msgr2=0x7f591c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:43.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.306+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 shutdown_connections 2023-08-25T09:43:43.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.306+0000 7f5923534640 1 -- 172.21.15.70:0/1000644982 wait complete. 2023-08-25T09:43:43.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:43:43.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.530+0000 7fe9ea28e640 1 Processor -- start 2023-08-25T09:43:43.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.530+0000 7fe9ea28e640 1 -- start start 2023-08-25T09:43:43.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9ea28e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9e415ae90 0x7fe9e40ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9ea28e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e415be60 con 0x7fe9e40ce410 2023-08-25T09:43:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9ea28e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9e406ee70 con 0x7fe9e406f300 2023-08-25T09:43:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9ea28e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9e40ced30 con 0x7fe9e415ae90 2023-08-25T09:43:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9e3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe9e40ce410 0x7fe9e4070770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33174/0 (socket says 172.21.15.70:33174) 2023-08-25T09:43:43.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9e3fff640 1 -- 172.21.15.70:0/1773014054 learned_addr learned my addr 172.21.15.70:0/1773014054 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:43.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.532+0000 7fe9dbfff640 1 --2- 172.21.15.70:0/1773014054 >> v2:172.21.15.155:3300/0 conn(0x7fe9e415ae90 0x7fe9e40ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202129129 0 0) 0x7fe9e415be60 con 0x7fe9e40ce410 2023-08-25T09:43:43.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d0003520 con 0x7fe9e40ce410 2023-08-25T09:43:43.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916695810 0 0) 0x7fe9d0003520 con 0x7fe9e40ce410 2023-08-25T09:43:43.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 >> v2:172.21.15.155:3300/0 conn(0x7fe9e415ae90 msgr2=0x7fe9e40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:43.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 --2- 172.21.15.70:0/1773014054 >> v2:172.21.15.155:3300/0 conn(0x7fe9e415ae90 0x7fe9e40ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 >> v1:172.21.15.155:6789/0 conn(0x7fe9e406f300 legacy=0x7fe9e406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e415b2f0 con 0x7fe9e40ce410 2023-08-25T09:43:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe9d4001670 con 0x7fe9e40ce410 2023-08-25T09:43:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 >> v1:172.21.15.70:6789/0 conn(0x7fe9e40ce410 legacy=0x7fe9e4070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0003970 0x7fe9d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9d00047c0 0x7fe9d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d0005180 0x7fe9d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e40ced30 con 0x7fe9d0005180 2023-08-25T09:43:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d000c020 con 0x7fe9d0003970 2023-08-25T09:43:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d000c1e0 con 0x7fe9d00047c0 2023-08-25T09:43:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.533+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d0005180 0x7fe9d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9e3fff640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0003970 0x7fe9d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9dbfff640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9d00047c0 0x7fe9d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9e882b640 1 -- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9d00047c0 msgr2=0x7fe9d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9d00047c0 0x7fe9d0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9e882b640 1 -- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0003970 msgr2=0x7fe9d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0003970 0x7fe9d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.534+0000 7fe9e882b640 1 -- 172.21.15.70:0/1773014054 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0003520 con 0x7fe9d0005180 2023-08-25T09:43:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.535+0000 7fe9e882b640 1 -- 172.21.15.70:0/1773014054 reap_dead start 2023-08-25T09:43:43.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.535+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d0005180 0x7fe9d000b980 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc001e50 tx=0x7fe9d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f065f75bae8bfb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.535+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc003b70 con 0x7fe9d0005180 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.535+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc003d10 con 0x7fe9d0005180 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.535+0000 7fe9e37fe640 1 -- 172.21.15.70:0/1773014054 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc005310 con 0x7fe9d0005180 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1773014054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d0005180 msgr2=0x7fe9d000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d0005180 0x7fe9d000b980 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc001e50 tx=0x7fe9d00057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1773014054 shutdown_connections 2023-08-25T09:43:43.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1773014054 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9d0005180 0x7fe9d000b980 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1773014054 >> 172.21.15.70:0/1773014054 conn(0x7fe9e4069850 msgr2=0x7fe9e4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1773014054 shutdown_connections 2023-08-25T09:43:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.536+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1773014054 wait complete. 2023-08-25T09:43:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.538+0000 7fe9ea28e640 1 Processor -- start 2023-08-25T09:43:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.538+0000 7fe9ea28e640 1 -- start start 2023-08-25T09:43:43.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9ea28e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0005180 0x7fe9e40ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9ea28e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e40ae2b0 0x7fe9e41f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9ea28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 0x7fe9e4156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9ea28e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e40ad7c0 con 0x7fe9e4156180 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9ea28e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e4156b40 con 0x7fe9d0005180 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9ea28e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e40ad9b0 con 0x7fe9e40ae2b0 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 0x7fe9e4156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 0x7fe9e4156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37278/0 (socket says 172.21.15.70:37278) 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 -- 172.21.15.70:0/1812967773 learned_addr learned my addr 172.21.15.70:0/1812967773 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e3fff640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0005180 0x7fe9e40ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 -- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e40ae2b0 msgr2=0x7fe9e41f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e40ae2b0 0x7fe9e41f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 -- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0005180 msgr2=0x7fe9e40ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0005180 0x7fe9e40ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.539+0000 7fe9e882b640 1 -- 172.21.15.70:0/1812967773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4145f00 con 0x7fe9e4156180 2023-08-25T09:43:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.540+0000 7fe9e882b640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 0x7fe9e4156600 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc005040 tx=0x7fe9dc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.540+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc006d50 con 0x7fe9e4156180 2023-08-25T09:43:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.540+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9dc039480 con 0x7fe9e4156180 2023-08-25T09:43:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.540+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0397f0 con 0x7fe9e4156180 2023-08-25T09:43:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.540+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e41461e0 con 0x7fe9e4156180 2023-08-25T09:43:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.540+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e41467a0 con 0x7fe9e4156180 2023-08-25T09:43:43.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.541+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9e404e420 con 0x7fe9e4156180 2023-08-25T09:43:43.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.542+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9dc039950 con 0x7fe9e4156180 2023-08-25T09:43:43.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.543+0000 7fe9e17fa640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9c406a080 0x7fe9c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.543+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe9dc0a9f00 con 0x7fe9e4156180 2023-08-25T09:43:43.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.543+0000 7fe9e3fff640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9c406a080 0x7fe9c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.544+0000 7fe9e3fff640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9c406a080 0x7fe9c406c540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40016f0 tx=0x7fe9d4000f10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:43.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.548+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9dc0759b0 con 0x7fe9e4156180 2023-08-25T09:43:43.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.737+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe9e4070f00 con 0x7fe9e4156180 2023-08-25T09:43:43.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.745+0000 7fe9e17fa640 1 -- 172.21.15.70:0/1812967773 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9dc078fd0 con 0x7fe9e4156180 2023-08-25T09:43:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9c406a080 msgr2=0x7fe9c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9c406a080 0x7fe9c406c540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe9d40016f0 tx=0x7fe9d4000f10 comp rx=0 tx=0).stop 2023-08-25T09:43:43.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 msgr2=0x7fe9e4156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 0x7fe9e4156600 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc005040 tx=0x7fe9dc0014a0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 shutdown_connections 2023-08-25T09:43:43.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9c406a080 0x7fe9c406c540 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e4156180 0x7fe9e4156600 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e40ae2b0 0x7fe9e41f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 --2- 172.21.15.70:0/1812967773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9d0005180 0x7fe9e40ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.748+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 >> 172.21.15.70:0/1812967773 conn(0x7fe9e4069850 msgr2=0x7fe9e4157ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:43.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.749+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 shutdown_connections 2023-08-25T09:43:43.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.749+0000 7fe9ea28e640 1 -- 172.21.15.70:0/1812967773 wait complete. 2023-08-25T09:43:43.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-08-25T09:43:43.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.985+0000 7f60debe1640 1 Processor -- start 2023-08-25T09:43:43.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.985+0000 7f60debe1640 1 -- start start 2023-08-25T09:43:43.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60debe1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60d81621f0 0x7f60d806ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60debe1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d8162640 con 0x7f60d81644d0 2023-08-25T09:43:43.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60debe1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d806b360 con 0x7f60d806f460 2023-08-25T09:43:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60debe1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d806c1d0 con 0x7f60d81621f0 2023-08-25T09:43:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60dc97d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f60d81644d0 0x7f60d806ed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33178/0 (socket says 172.21.15.70:33178) 2023-08-25T09:43:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60dc97d640 1 -- 172.21.15.70:0/1009219346 learned_addr learned my addr 172.21.15.70:0/1009219346 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60dd17e640 1 --2- 172.21.15.70:0/1009219346 >> v2:172.21.15.70:3300/0 conn(0x7f60d81621f0 0x7f60d806ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60d77fe640 1 -- 172.21.15.70:0/1009219346 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 489690456 0 0) 0x7f60d8162640 con 0x7f60d81644d0 2023-08-25T09:43:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60d77fe640 1 -- 172.21.15.70:0/1009219346 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60c8003520 con 0x7f60d81644d0 2023-08-25T09:43:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60dd17e640 1 -- 172.21.15.70:0/1009219346 >> v1:172.21.15.155:6789/0 conn(0x7f60d806f460 legacy=0x7f60d8162b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.987+0000 7f60dd17e640 1 -- 172.21.15.70:0/1009219346 >> v1:172.21.15.70:6789/0 conn(0x7f60d81644d0 legacy=0x7f60d806ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.988+0000 7f60dd17e640 1 -- 172.21.15.70:0/1009219346 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d806b5c0 con 0x7f60d81621f0 2023-08-25T09:43:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60dd17e640 1 --2- 172.21.15.70:0/1009219346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81621f0 0x7f60d806ae20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009d90 tx=0x7f60d002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6603dc97f7013cd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60d77fe640 1 -- 172.21.15.70:0/1009219346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d002cc60 con 0x7f60d81621f0 2023-08-25T09:43:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60d77fe640 1 -- 172.21.15.70:0/1009219346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d002cdc0 con 0x7f60d81621f0 2023-08-25T09:43:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60d77fe640 1 -- 172.21.15.70:0/1009219346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0034640 con 0x7f60d81621f0 2023-08-25T09:43:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60debe1640 1 -- 172.21.15.70:0/1009219346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81621f0 msgr2=0x7f60d806ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60debe1640 1 --2- 172.21.15.70:0/1009219346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81621f0 0x7f60d806ae20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f60d0009d90 tx=0x7f60d002beb0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60debe1640 1 -- 172.21.15.70:0/1009219346 shutdown_connections 2023-08-25T09:43:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60debe1640 1 --2- 172.21.15.70:0/1009219346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81621f0 0x7f60d806ae20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.989+0000 7f60debe1640 1 -- 172.21.15.70:0/1009219346 >> 172.21.15.70:0/1009219346 conn(0x7f60d80ce760 msgr2=0x7f60d80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.990+0000 7f60debe1640 1 -- 172.21.15.70:0/1009219346 shutdown_connections 2023-08-25T09:43:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.990+0000 7f60debe1640 1 -- 172.21.15.70:0/1009219346 wait complete. 2023-08-25T09:43:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.991+0000 7f60debe1640 1 Processor -- start 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.991+0000 7f60debe1640 1 -- start start 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.992+0000 7f60debe1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60d806f460 0x7f60d814a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.992+0000 7f60debe1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d81621f0 0x7f60d80cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.992+0000 7f60debe1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 0x7f60d80cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.992+0000 7f60debe1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80b2e20 con 0x7f60d81644d0 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.992+0000 7f60debe1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d81faa00 con 0x7f60d81621f0 2023-08-25T09:43:43.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.992+0000 7f60debe1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80ccfd0 con 0x7f60d806f460 2023-08-25T09:43:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 0x7f60d80cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 0x7f60d80cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37294/0 (socket says 172.21.15.70:37294) 2023-08-25T09:43:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 -- 172.21.15.70:0/2838466812 learned_addr learned my addr 172.21.15.70:0/2838466812 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dc97d640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60d806f460 0x7f60d814a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 -- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60d806f460 msgr2=0x7f60d814a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60d806f460 0x7f60d814a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60d7fff640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d81621f0 0x7f60d80cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 -- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d81621f0 msgr2=0x7f60d80cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d81621f0 0x7f60d80cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 -- 172.21.15.70:0/2838466812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d0009a70 con 0x7f60d81644d0 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60dd17e640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 0x7f60d80cca90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f60d002c4c0 tx=0x7f60d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.993+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004250 con 0x7f60d81644d0 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.994+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d00043b0 con 0x7f60d81644d0 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.994+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0004760 con 0x7f60d81644d0 2023-08-25T09:43:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.994+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d80b24f0 con 0x7f60d81644d0 2023-08-25T09:43:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.994+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d80b2ab0 con 0x7f60d81644d0 2023-08-25T09:43:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.995+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d80b2fd0 con 0x7f60d81644d0 2023-08-25T09:43:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.996+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f60d003b070 con 0x7f60d81644d0 2023-08-25T09:43:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.997+0000 7f60d57fa640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60bc06a050 0x7f60bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.997+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f60d00aa820 con 0x7f60d81644d0 2023-08-25T09:43:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.997+0000 7f60dc97d640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60bc06a050 0x7f60bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:43.998+0000 7f60dc97d640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60bc06a050 0x7f60bc06c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f60cc001360 tx=0x7f60cc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:44.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.001+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f60d80b2fd0 con 0x7f60d81644d0 2023-08-25T09:43:44.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.256+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f60d80b2fd0 con 0x7f60d81644d0 2023-08-25T09:43:44.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.305+0000 7f60d57fa640 1 -- 172.21.15.70:0/2838466812 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v270) v1 ==== 134+0+0 (secure 0 0 0) 0x7f60d80b2fd0 con 0x7f60d81644d0 2023-08-25T09:43:44.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60bc06a050 msgr2=0x7f60bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60bc06a050 0x7f60bc06c510 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f60cc001360 tx=0x7f60cc009040 comp rx=0 tx=0).stop 2023-08-25T09:43:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 msgr2=0x7f60d80cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 0x7f60d80cca90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f60d002c4c0 tx=0x7f60d0004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 shutdown_connections 2023-08-25T09:43:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f60bc06a050 0x7f60bc06c510 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60d81644d0 0x7f60d80cca90 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60d81621f0 0x7f60d80cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 --2- 172.21.15.70:0/2838466812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60d806f460 0x7f60d814a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 >> 172.21.15.70:0/2838466812 conn(0x7f60d80ce760 msgr2=0x7f60d8162b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 shutdown_connections 2023-08-25T09:43:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.308+0000 7f60debe1640 1 -- 172.21.15.70:0/2838466812 wait complete. 2023-08-25T09:43:44.329 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-08-25T09:43:44.330 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-08-25T09:43:44.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:44.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.539+0000 7f6167034640 1 Processor -- start 2023-08-25T09:43:44.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.539+0000 7f6167034640 1 -- start start 2023-08-25T09:43:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6167034640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f616015e830 0x7f616006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6167034640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015ec10 0x7f61600b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6167034640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61600b1b60 con 0x7f616006b2b0 2023-08-25T09:43:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6167034640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616006bd80 con 0x7f616015e830 2023-08-25T09:43:44.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6167034640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616006b7c0 con 0x7f616015ec10 2023-08-25T09:43:44.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6164dd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f616015e830 0x7f616006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6164dd0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f616015e830 0x7f616006ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37302/0 (socket says 172.21.15.70:37302) 2023-08-25T09:43:44.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6164dd0640 1 -- 172.21.15.70:0/2359245821 learned_addr learned my addr 172.21.15.70:0/2359245821 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:44.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f61655d1640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015ec10 0x7f61600b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.541+0000 7f6164dd0640 1 -- 172.21.15.70:0/2359245821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015ec10 msgr2=0x7f61600b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:44.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.542+0000 7f6164dd0640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015ec10 0x7f61600b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.542+0000 7f6164dd0640 1 -- 172.21.15.70:0/2359245821 >> v1:172.21.15.70:6789/0 conn(0x7f616006b2b0 legacy=0x7f616015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.542+0000 7f6164dd0640 1 -- 172.21.15.70:0/2359245821 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616006baa0 con 0x7f616015e830 2023-08-25T09:43:44.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.542+0000 7f61577fe640 1 -- 172.21.15.70:0/2359245821 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623999077 0 0) 0x7f61600b1b60 con 0x7f616006b2b0 2023-08-25T09:43:44.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.542+0000 7f61655d1640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015ec10 0x7f61600b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:44.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.543+0000 7f6164dd0640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616015e830 0x7f616006ad70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f615c009e50 tx=0x7f615c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5964e225f6e43347 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:44.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.543+0000 7f61577fe640 1 -- 172.21.15.70:0/2359245821 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c02cb00 con 0x7f616015e830 2023-08-25T09:43:44.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.543+0000 7f61577fe640 1 -- 172.21.15.70:0/2359245821 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c02cc60 con 0x7f616015e830 2023-08-25T09:43:44.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.543+0000 7f61577fe640 1 -- 172.21.15.70:0/2359245821 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0343f0 con 0x7f616015e830 2023-08-25T09:43:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 -- 172.21.15.70:0/2359245821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616015e830 msgr2=0x7f616006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616015e830 0x7f616006ad70 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f615c009e50 tx=0x7f615c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:43:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 -- 172.21.15.70:0/2359245821 shutdown_connections 2023-08-25T09:43:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015ec10 0x7f61600b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 --2- 172.21.15.70:0/2359245821 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616015e830 0x7f616006ad70 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 -- 172.21.15.70:0/2359245821 >> 172.21.15.70:0/2359245821 conn(0x7f6160069850 msgr2=0x7f6160069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 -- 172.21.15.70:0/2359245821 shutdown_connections 2023-08-25T09:43:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.544+0000 7f6167034640 1 -- 172.21.15.70:0/2359245821 wait complete. 2023-08-25T09:43:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.546+0000 7f6167034640 1 Processor -- start 2023-08-25T09:43:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.546+0000 7f6167034640 1 -- start start 2023-08-25T09:43:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6167034640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 0x7f61601490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6167034640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015e830 0x7f6160155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6167034640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616015ec10 0x7f61601551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6167034640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6160154d10 con 0x7f616006b2b0 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6167034640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61601495e0 con 0x7f616015ec10 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6167034640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61601560d0 con 0x7f616015e830 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 0x7f61601490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 0x7f61601490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37310/0 (socket says 172.21.15.70:37310) 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 -- 172.21.15.70:0/2486651787 learned_addr learned my addr 172.21.15.70:0/2486651787 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 -- 172.21.15.70:0/2486651787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015e830 msgr2=0x7f6160155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015e830 0x7f6160155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 -- 172.21.15.70:0/2486651787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616015ec10 msgr2=0x7f61601551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616015ec10 0x7f61601551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.547+0000 7f6164dd0640 1 -- 172.21.15.70:0/2486651787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c009b30 con 0x7f616006b2b0 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.548+0000 7f6164dd0640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 0x7f61601490a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f615c009e50 tx=0x7f615c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.548+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c03e450 con 0x7f616006b2b0 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.548+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c03e5b0 con 0x7f616006b2b0 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.548+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c03e720 con 0x7f616006b2b0 2023-08-25T09:43:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.548+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616015ca20 con 0x7f616006b2b0 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.548+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61601543e0 con 0x7f616006b2b0 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.549+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616004e420 con 0x7f616006b2b0 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.550+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f615c03e880 con 0x7f616006b2b0 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.551+0000 7f61557fa640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f614c06a080 0x7f614c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.551+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f615c0ab1e0 con 0x7f616006b2b0 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.551+0000 7f6157fff640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f614c06a080 0x7f614c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.552+0000 7f6157fff640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f614c06a080 0x7f614c06c540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61601f3ae0 tx=0x7f6150007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.555+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f615c076b60 con 0x7f616006b2b0 2023-08-25T09:43:44.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.744+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f616006c620 con 0x7f616006b2b0 2023-08-25T09:43:44.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.745+0000 7f61557fa640 1 -- 172.21.15.70:0/2486651787 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+14783 (secure 0 0 0) 0x7f615c033e80 con 0x7f616006b2b0 2023-08-25T09:43:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:44.748 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":85,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:43.288312+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6837/1838015952":"2023-08-26T09:43:39.855652+0000","172.21.15.70:6836/1838015952":"2023-08-26T09:43:39.855652+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:44.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f614c06a080 msgr2=0x7f614c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f614c06a080 0x7f614c06c540 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f61601f3ae0 tx=0x7f6150007910 comp rx=0 tx=0).stop 2023-08-25T09:43:44.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 msgr2=0x7f61601490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 0x7f61601490a0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f615c009e50 tx=0x7f615c004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 shutdown_connections 2023-08-25T09:43:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f614c06a080 0x7f614c06c540 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f616015ec10 0x7f61601551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f616015e830 0x7f6160155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 --2- 172.21.15.70:0/2486651787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f616006b2b0 0x7f61601490a0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 >> 172.21.15.70:0/2486651787 conn(0x7f6160069850 msgr2=0x7f616015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:44.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 shutdown_connections 2023-08-25T09:43:44.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.748+0000 7f6167034640 1 -- 172.21.15.70:0/2486651787 wait complete. 2023-08-25T09:43:44.769 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:43:44.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:43:44.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.982+0000 7f9c09e75640 1 Processor -- start 2023-08-25T09:43:44.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.982+0000 7f9c09e75640 1 -- start start 2023-08-25T09:43:44.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09e75640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9c040b0320 0x7f9c040c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09e75640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c0415a920 0x7f9c040b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09e75640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c040adbb0 con 0x7f9c040b0700 2023-08-25T09:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09e75640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c040b2220 con 0x7f9c040b0320 2023-08-25T09:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09e75640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040c90a0 con 0x7f9c0415a920 2023-08-25T09:43:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09674640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c040b0700 0x7f9c040ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33198/0 (socket says 172.21.15.70:33198) 2023-08-25T09:43:44.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.984+0000 7f9c09674640 1 -- 172.21.15.70:0/1708872370 learned_addr learned my addr 172.21.15.70:0/1708872370 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:44.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.985+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1708872370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c0415a920 0x7f9c040b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.985+0000 7f9c08e73640 1 --2- 172.21.15.70:0/1708872370 >> v2:172.21.15.155:3300/0 conn(0x7f9c040b0320 0x7f9c040c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.985+0000 7f9c037fe640 1 -- 172.21.15.70:0/1708872370 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910897035 0 0) 0x7f9c040adbb0 con 0x7f9c040b0700 2023-08-25T09:43:44.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.985+0000 7f9c037fe640 1 -- 172.21.15.70:0/1708872370 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bf4003520 con 0x7f9c040b0700 2023-08-25T09:43:44.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.986+0000 7f9c03fff640 1 -- 172.21.15.70:0/1708872370 >> v2:172.21.15.155:3300/0 conn(0x7f9c040b0320 msgr2=0x7f9c040c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:44.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.986+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1708872370 >> v2:172.21.15.155:3300/0 conn(0x7f9c040b0320 0x7f9c040c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.986+0000 7f9c03fff640 1 -- 172.21.15.70:0/1708872370 >> v1:172.21.15.70:6789/0 conn(0x7f9c040b0700 legacy=0x7f9c040ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.986+0000 7f9c03fff640 1 -- 172.21.15.70:0/1708872370 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c040c9380 con 0x7f9c0415a920 2023-08-25T09:43:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.988+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1708872370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c0415a920 0x7f9c040b1ce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097a0 tx=0x7f9bf002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d10e9854e64999f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.988+0000 7f9c037fe640 1 -- 172.21.15.70:0/1708872370 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf002d040 con 0x7f9c0415a920 2023-08-25T09:43:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.988+0000 7f9c037fe640 1 -- 172.21.15.70:0/1708872370 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf0031810 con 0x7f9c0415a920 2023-08-25T09:43:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c037fe640 1 -- 172.21.15.70:0/1708872370 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0031ba0 con 0x7f9c0415a920 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c09e75640 1 -- 172.21.15.70:0/1708872370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c0415a920 msgr2=0x7f9c040b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1708872370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c0415a920 0x7f9c040b1ce0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00097a0 tx=0x7f9bf002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c09e75640 1 -- 172.21.15.70:0/1708872370 shutdown_connections 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1708872370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c0415a920 0x7f9c040b1ce0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1708872370 >> v2:172.21.15.155:3300/0 conn(0x7f9c040b0320 0x7f9c040c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.989+0000 7f9c09e75640 1 -- 172.21.15.70:0/1708872370 >> 172.21.15.70:0/1708872370 conn(0x7f9c040ce830 msgr2=0x7f9c040cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.990+0000 7f9c09e75640 1 -- 172.21.15.70:0/1708872370 shutdown_connections 2023-08-25T09:43:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.990+0000 7f9c09e75640 1 -- 172.21.15.70:0/1708872370 wait complete. 2023-08-25T09:43:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.992+0000 7f9c09e75640 1 Processor -- start 2023-08-25T09:43:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.992+0000 7f9c09e75640 1 -- start start 2023-08-25T09:43:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c09e75640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c040b0320 0x7f9c040be4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c09e75640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 0x7f9c040af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c09e75640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c0415a920 0x7f9c040bba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c09e75640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415c040 con 0x7f9c040b0700 2023-08-25T09:43:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c09e75640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c040be9f0 con 0x7f9c0415a920 2023-08-25T09:43:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c09e75640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0415c2e0 con 0x7f9c040b0320 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 0x7f9c040af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 0x7f9c040af3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37326/0 (socket says 172.21.15.70:37326) 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.993+0000 7f9c03fff640 1 -- 172.21.15.70:0/1983399006 learned_addr learned my addr 172.21.15.70:0/1983399006 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c03fff640 1 -- 172.21.15.70:0/1983399006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c040b0320 msgr2=0x7f9c040be4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c040b0320 0x7f9c040be4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c03fff640 1 -- 172.21.15.70:0/1983399006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c0415a920 msgr2=0x7f9c040bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c0415a920 0x7f9c040bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c03fff640 1 -- 172.21.15.70:0/1983399006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf0009400 con 0x7f9c040b0700 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c03fff640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 0x7f9c040af3d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00098c0 tx=0x7f9bf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0006e10 con 0x7f9c040b0700 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bf003ad60 con 0x7f9c040b0700 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0044610 con 0x7f9c040b0700 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0415c5c0 con 0x7f9c040b0700 2023-08-25T09:43:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.994+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0415cb80 con 0x7f9c040b0700 2023-08-25T09:43:45.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.995+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c04061460 con 0x7f9c040b0700 2023-08-25T09:43:45.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.996+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9bf004d450 con 0x7f9c040b0700 2023-08-25T09:43:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.997+0000 7f9c017fa640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9be806a050 0x7f9be806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.997+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9bf00aa560 con 0x7f9c040b0700 2023-08-25T09:43:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.998+0000 7f9c08e73640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9be806a050 0x7f9be806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:44.998+0000 7f9c08e73640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9be806a050 0x7f9be806c510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8006e20 tx=0x7f9bf8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.002+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9bf0075f60 con 0x7f9c040b0700 2023-08-25T09:43:45.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:43:45.126 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:43:45.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.187+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9c040ca720 con 0x7f9c040b0700 2023-08-25T09:43:45.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.187+0000 7f9c017fa640 1 -- 172.21.15.70:0/1983399006 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bf0079580 con 0x7f9c040b0700 2023-08-25T09:43:45.189 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:43:45.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.189+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9be806a050 msgr2=0x7f9be806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.189+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9be806a050 0x7f9be806c510 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8006e20 tx=0x7f9bf8008040 comp rx=0 tx=0).stop 2023-08-25T09:43:45.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 msgr2=0x7f9c040af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 0x7f9c040af3d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9bf00098c0 tx=0x7f9bf00046c0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 shutdown_connections 2023-08-25T09:43:45.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9be806a050 0x7f9be806c510 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c0415a920 0x7f9c040bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c040b0700 0x7f9c040af3d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 --2- 172.21.15.70:0/1983399006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c040b0320 0x7f9c040be4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 >> 172.21.15.70:0/1983399006 conn(0x7f9c040ce830 msgr2=0x7f9c04159930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:45.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 shutdown_connections 2023-08-25T09:43:45.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.190+0000 7f9c09e75640 1 -- 172.21.15.70:0/1983399006 wait complete. 2023-08-25T09:43:45.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:43:45.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.411+0000 7f316e776640 1 Processor -- start 2023-08-25T09:43:45.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.411+0000 7f316e776640 1 -- start start 2023-08-25T09:43:45.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.412+0000 7f316e776640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31680c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.412+0000 7f316e776640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31680cdc20 0x7f3168069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.412+0000 7f316e776640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31680cc5f0 con 0x7f31680ce060 2023-08-25T09:43:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.412+0000 7f316e776640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3168070650 con 0x7f31680cdc20 2023-08-25T09:43:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.412+0000 7f316e776640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3168070930 con 0x7f316815ae50 2023-08-25T09:43:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31680cdc20 0x7f3168069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31680cdc20 0x7f3168069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37334/0 (socket says 172.21.15.70:37334) 2023-08-25T09:43:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 -- 172.21.15.70:0/1431085143 learned_addr learned my addr 172.21.15.70:0/1431085143 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 -- 172.21.15.70:0/1431085143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 msgr2=0x7f31680c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 --2- 172.21.15.70:0/1431085143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31680c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 -- 172.21.15.70:0/1431085143 >> v1:172.21.15.155:6789/0 conn(0x7f31680ce060 legacy=0x7f31680ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.413+0000 7f31677fe640 1 -- 172.21.15.70:0/1431085143 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3168070c10 con 0x7f31680cdc20 2023-08-25T09:43:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.415+0000 7f31677fe640 1 --2- 172.21.15.70:0/1431085143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680cdc20 0x7f3168069850 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f315c009870 tx=0x7f315c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c752792e51dcee3d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.415+0000 7f3166ffd640 1 -- 172.21.15.70:0/1431085143 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c030840 con 0x7f31680cdc20 2023-08-25T09:43:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.415+0000 7f3166ffd640 1 -- 172.21.15.70:0/1431085143 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c0309a0 con 0x7f31680cdc20 2023-08-25T09:43:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.415+0000 7f3166ffd640 1 -- 172.21.15.70:0/1431085143 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c030d30 con 0x7f31680cdc20 2023-08-25T09:43:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 -- 172.21.15.70:0/1431085143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680cdc20 msgr2=0x7f3168069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 --2- 172.21.15.70:0/1431085143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680cdc20 0x7f3168069850 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f315c009870 tx=0x7f315c02b980 comp rx=0 tx=0).stop 2023-08-25T09:43:45.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 -- 172.21.15.70:0/1431085143 shutdown_connections 2023-08-25T09:43:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 --2- 172.21.15.70:0/1431085143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680cdc20 0x7f3168069850 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 --2- 172.21.15.70:0/1431085143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31680c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 -- 172.21.15.70:0/1431085143 >> 172.21.15.70:0/1431085143 conn(0x7f31680cb0d0 msgr2=0x7f31680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:45.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 -- 172.21.15.70:0/1431085143 shutdown_connections 2023-08-25T09:43:45.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.416+0000 7f316e776640 1 -- 172.21.15.70:0/1431085143 wait complete. 2023-08-25T09:43:45.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.418+0000 7f316e776640 1 Processor -- start 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.418+0000 7f316e776640 1 -- start start 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316e776640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31680cdc20 0x7f31680c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316e776640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 0x7f31680c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316e776640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31681eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316e776640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31681eeb00 con 0x7f31680ce060 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316e776640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c84e0 con 0x7f31680cdc20 2023-08-25T09:43:45.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316e776640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31680c7f20 con 0x7f316815ae50 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 0x7f31680c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 0x7f31680c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37350/0 (socket says 172.21.15.70:37350) 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 -- 172.21.15.70:0/1593800111 learned_addr learned my addr 172.21.15.70:0/1593800111 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f316cd13640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31681eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f3167fff640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31680cdc20 0x7f31680c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 -- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 msgr2=0x7f31681eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31681eef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 -- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31680cdc20 msgr2=0x7f31680c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.419+0000 7f31677fe640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31680cdc20 0x7f31680c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f31677fe640 1 -- 172.21.15.70:0/1593800111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c0094d0 con 0x7f31680ce060 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f3167fff640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31680cdc20 0x7f31680c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f31677fe640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 0x7f31680c78b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f315c009870 tx=0x7f315c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c039d60 con 0x7f31680ce060 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c043400 con 0x7f31680ce060 2023-08-25T09:43:45.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31680c8200 con 0x7f31680ce060 2023-08-25T09:43:45.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c0437b0 con 0x7f31680ce060 2023-08-25T09:43:45.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.420+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31681ef4d0 con 0x7f31680ce060 2023-08-25T09:43:45.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.421+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3168066fb0 con 0x7f31680ce060 2023-08-25T09:43:45.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.422+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f315c043910 con 0x7f31680ce060 2023-08-25T09:43:45.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.423+0000 7f3164ff9640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f315406a050 0x7f315406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.423+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f315c03e070 con 0x7f31680ce060 2023-08-25T09:43:45.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.423+0000 7f3167fff640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f315406a050 0x7f315406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.424+0000 7f3167fff640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f315406a050 0x7f315406c510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f315800a870 tx=0x7f3158008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.427+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f315c076e50 con 0x7f31680ce060 2023-08-25T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.617+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f31680c46f0 con 0x7f31680ce060 2023-08-25T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.618+0000 7f3164ff9640 1 -- 172.21.15.70:0/1593800111 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v85) v1 ==== 92+0+10 (secure 0 0 0) 0x7f315c07a470 con 0x7f31680ce060 2023-08-25T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:43:45.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f315406a050 msgr2=0x7f315406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f315406a050 0x7f315406c510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f315800a870 tx=0x7f3158008040 comp rx=0 tx=0).stop 2023-08-25T09:43:45.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 msgr2=0x7f31680c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 0x7f31680c78b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f315c009870 tx=0x7f315c004730 comp rx=0 tx=0).stop 2023-08-25T09:43:45.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 shutdown_connections 2023-08-25T09:43:45.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f315406a050 0x7f315406c510 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f316815ae50 0x7f31681eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31680ce060 0x7f31680c78b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 --2- 172.21.15.70:0/1593800111 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31680cdc20 0x7f31680c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 >> 172.21.15.70:0/1593800111 conn(0x7f31680cb0d0 msgr2=0x7f31680af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:45.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 shutdown_connections 2023-08-25T09:43:45.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.621+0000 7f316e776640 1 -- 172.21.15.70:0/1593800111 wait complete. 2023-08-25T09:43:45.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-08-25T09:43:45.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.846+0000 7f66b0742640 1 Processor -- start 2023-08-25T09:43:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.846+0000 7f66b0742640 1 -- start start 2023-08-25T09:43:45.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.847+0000 7f66b0742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a80ca780 0x7f66a80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.847+0000 7f66b0742640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f66a80b3cf0 0x7f66a80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.847+0000 7f66b0742640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a80b12b0 con 0x7f66a815acd0 2023-08-25T09:43:45.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.847+0000 7f66b0742640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a806c0e0 con 0x7f66a80b3cf0 2023-08-25T09:43:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.847+0000 7f66b0742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a806ae20 con 0x7f66a80ca780 2023-08-25T09:43:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66ae4de640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f66a815acd0 0x7f66a815b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33200/0 (socket says 172.21.15.70:33200) 2023-08-25T09:43:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66ae4de640 1 -- 172.21.15.70:0/2884494148 learned_addr learned my addr 172.21.15.70:0/2884494148 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:45.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66aecdf640 1 --2- 172.21.15.70:0/2884494148 >> v2:172.21.15.70:3300/0 conn(0x7f66a80b3cf0 0x7f66a80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66adcdd640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a80ca780 0x7f66a80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66ad4dc640 1 -- 172.21.15.70:0/2884494148 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143887175 0 0) 0x7f66a80b12b0 con 0x7f66a815acd0 2023-08-25T09:43:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66ad4dc640 1 -- 172.21.15.70:0/2884494148 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6694003520 con 0x7f66a815acd0 2023-08-25T09:43:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66aecdf640 1 -- 172.21.15.70:0/2884494148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a80ca780 msgr2=0x7f66a80cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66aecdf640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a80ca780 0x7f66a80cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66aecdf640 1 -- 172.21.15.70:0/2884494148 >> v1:172.21.15.70:6789/0 conn(0x7f66a815acd0 legacy=0x7f66a815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66aecdf640 1 -- 172.21.15.70:0/2884494148 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a815b7c0 con 0x7f66a80b3cf0 2023-08-25T09:43:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.848+0000 7f66adcdd640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a80ca780 0x7f66a80cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.849+0000 7f66aecdf640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80b3cf0 0x7f66a80b4140 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009d90 tx=0x7f66a002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777358738d9c6ea8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66ad4dc640 1 -- 172.21.15.70:0/2884494148 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a002cbc0 con 0x7f66a80b3cf0 2023-08-25T09:43:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66ad4dc640 1 -- 172.21.15.70:0/2884494148 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a002cd20 con 0x7f66a80b3cf0 2023-08-25T09:43:45.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66ad4dc640 1 -- 172.21.15.70:0/2884494148 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0034640 con 0x7f66a80b3cf0 2023-08-25T09:43:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 -- 172.21.15.70:0/2884494148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80b3cf0 msgr2=0x7f66a80b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80b3cf0 0x7f66a80b4140 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009d90 tx=0x7f66a002be10 comp rx=0 tx=0).stop 2023-08-25T09:43:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 -- 172.21.15.70:0/2884494148 shutdown_connections 2023-08-25T09:43:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80b3cf0 0x7f66a80b4140 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 --2- 172.21.15.70:0/2884494148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a80ca780 0x7f66a80cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 -- 172.21.15.70:0/2884494148 >> 172.21.15.70:0/2884494148 conn(0x7f66a80ce760 msgr2=0x7f66a80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 -- 172.21.15.70:0/2884494148 shutdown_connections 2023-08-25T09:43:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.850+0000 7f66b0742640 1 -- 172.21.15.70:0/2884494148 wait complete. 2023-08-25T09:43:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.852+0000 7f66b0742640 1 Processor -- start 2023-08-25T09:43:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.852+0000 7f66b0742640 1 -- start start 2023-08-25T09:43:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66b0742640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66a80b3cf0 0x7f66a81f1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66b0742640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 0x7f66a8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66b0742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a815acd0 0x7f66a8154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66b0742640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a806fac0 con 0x7f66a80ca780 2023-08-25T09:43:45.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66b0742640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8070920 con 0x7f66a80b3cf0 2023-08-25T09:43:45.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66b0742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8154c90 con 0x7f66a815acd0 2023-08-25T09:43:45.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66adcdd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 0x7f66a8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66adcdd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 0x7f66a8156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37372/0 (socket says 172.21.15.70:37372) 2023-08-25T09:43:45.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.853+0000 7f66adcdd640 1 -- 172.21.15.70:0/1102034151 learned_addr learned my addr 172.21.15.70:0/1102034151 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:45.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66adcdd640 1 -- 172.21.15.70:0/1102034151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a815acd0 msgr2=0x7f66a8154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66adcdd640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a815acd0 0x7f66a8154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66adcdd640 1 -- 172.21.15.70:0/1102034151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66a80b3cf0 msgr2=0x7f66a81f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66adcdd640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66a80b3cf0 0x7f66a81f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66adcdd640 1 -- 172.21.15.70:0/1102034151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0009a70 con 0x7f66a80ca780 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66adcdd640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 0x7f66a8156f20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f669800c7f0 tx=0x7f669800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698007a30 con 0x7f66a80ca780 2023-08-25T09:43:45.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6698007b90 con 0x7f66a80ca780 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669800ce40 con 0x7f66a80ca780 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a806fd10 con 0x7f66a80ca780 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.854+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a80702d0 con 0x7f66a80ca780 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.856+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a804e420 con 0x7f66a80ca780 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.856+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f66980113f0 con 0x7f66a80ca780 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.857+0000 7f669effd640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f668c06a080 0x7f668c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.857+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f66980884c0 con 0x7f66a80ca780 2023-08-25T09:43:45.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.858+0000 7f66ae4de640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f668c06a080 0x7f668c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:45.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.858+0000 7f66ae4de640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f668c06a080 0x7f668c06c540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66a4007fd0 tx=0x7f66a40077c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:45.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:45.861+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6698053e30 con 0x7f66a80ca780 2023-08-25T09:43:46.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.119+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f66a806bec0 con 0x7f66a80ca780 2023-08-25T09:43:46.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.317+0000 7f669effd640 1 -- 172.21.15.70:0/1102034151 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v272) v1 ==== 136+0+0 (secure 0 0 0) 0x7f6698057450 con 0x7f66a80ca780 2023-08-25T09:43:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f668c06a080 msgr2=0x7f668c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f668c06a080 0x7f668c06c540 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f66a4007fd0 tx=0x7f66a40077c0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 msgr2=0x7f66a8156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:46.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 0x7f66a8156f20 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f669800c7f0 tx=0x7f669800cbf0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 shutdown_connections 2023-08-25T09:43:46.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f668c06a080 0x7f668c06c540 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66a815acd0 0x7f66a8154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66a80ca780 0x7f66a8156f20 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 --2- 172.21.15.70:0/1102034151 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66a80b3cf0 0x7f66a81f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.320+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 >> 172.21.15.70:0/1102034151 conn(0x7f66a80ce760 msgr2=0x7f66a815cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:46.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.321+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 shutdown_connections 2023-08-25T09:43:46.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.321+0000 7f66b0742640 1 -- 172.21.15.70:0/1102034151 wait complete. 2023-08-25T09:43:46.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-08-25T09:43:46.613 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_objectstore":"memstore"} 2023-08-25T09:43:46.613 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-08-25T09:43:46.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:43:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.820+0000 7fe4a0f0c640 1 Processor -- start 2023-08-25T09:43:46.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.821+0000 7fe4a0f0c640 1 -- start start 2023-08-25T09:43:46.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.822+0000 7fe4a0f0c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe49c069850 0x7fe49c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:46.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.822+0000 7fe4a0f0c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe49c0c9620 0x7fe49c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:46.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.822+0000 7fe4a0f0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c0ac880 con 0x7fe49c160be0 2023-08-25T09:43:46.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.822+0000 7fe4a0f0c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c0ad400 con 0x7fe49c069850 2023-08-25T09:43:46.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.822+0000 7fe4a0f0c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c0ad890 con 0x7fe49c0c9620 2023-08-25T09:43:46.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe49c069850 0x7fe49c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:46.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe49c069850 0x7fe49c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37384/0 (socket says 172.21.15.70:37384) 2023-08-25T09:43:46.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 -- 172.21.15.70:0/3948648002 learned_addr learned my addr 172.21.15.70:0/3948648002 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:46.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe493fff640 1 --2- 172.21.15.70:0/3948648002 >> v2:172.21.15.155:3300/0 conn(0x7fe49c0c9620 0x7fe49c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:46.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 -- 172.21.15.70:0/3948648002 >> v2:172.21.15.155:3300/0 conn(0x7fe49c0c9620 msgr2=0x7fe49c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:46.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 --2- 172.21.15.70:0/3948648002 >> v2:172.21.15.155:3300/0 conn(0x7fe49c0c9620 0x7fe49c069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 -- 172.21.15.70:0/3948648002 >> v1:172.21.15.155:6789/0 conn(0x7fe49c160be0 legacy=0x7fe49c161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:46.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe49a59c640 1 -- 172.21.15.70:0/3948648002 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c0acad0 con 0x7fe49c069850 2023-08-25T09:43:46.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.823+0000 7fe493fff640 1 --2- 172.21.15.70:0/3948648002 >> v2:172.21.15.155:3300/0 conn(0x7fe49c0c9620 0x7fe49c069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:46.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.824+0000 7fe49a59c640 1 --2- 172.21.15.70:0/3948648002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c069850 0x7fe49c15e210 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe49c06d120 tx=0x7fe48c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4324b98b33910ea6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:46.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.824+0000 7fe499d9b640 1 -- 172.21.15.70:0/3948648002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c030860 con 0x7fe49c069850 2023-08-25T09:43:46.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.824+0000 7fe499d9b640 1 -- 172.21.15.70:0/3948648002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48c0309c0 con 0x7fe49c069850 2023-08-25T09:43:46.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe499d9b640 1 -- 172.21.15.70:0/3948648002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48c030d50 con 0x7fe49c069850 2023-08-25T09:43:46.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/3948648002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c069850 msgr2=0x7fe49c15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:46.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/3948648002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c069850 0x7fe49c15e210 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe49c06d120 tx=0x7fe48c02b9a0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/3948648002 shutdown_connections 2023-08-25T09:43:46.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/3948648002 >> v2:172.21.15.155:3300/0 conn(0x7fe49c0c9620 0x7fe49c069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/3948648002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c069850 0x7fe49c15e210 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/3948648002 >> 172.21.15.70:0/3948648002 conn(0x7fe49c0cb0d0 msgr2=0x7fe49c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/3948648002 shutdown_connections 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.825+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/3948648002 wait complete. 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.827+0000 7fe4a0f0c640 1 Processor -- start 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.827+0000 7fe4a0f0c640 1 -- start start 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe4a0f0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe49c069850 0x7fe49c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:46.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe4a0f0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 0x7fe49c1f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe4a0f0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe49c160be0 0x7fe49c0c4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe4a0f0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c79a0 con 0x7fe49c0c9620 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe4a0f0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c7e30 con 0x7fe49c160be0 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe4a0f0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c6d90 con 0x7fe49c069850 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.828+0000 7fe493fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 0x7fe49c1f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 0x7fe49c1f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37394/0 (socket says 172.21.15.70:37394) 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 -- 172.21.15.70:0/2019321678 learned_addr learned my addr 172.21.15.70:0/2019321678 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:46.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe49a59c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe49c069850 0x7fe49c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 -- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe49c069850 msgr2=0x7fe49c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe49c069850 0x7fe49c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 -- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe49c160be0 msgr2=0x7fe49c0c4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe49ad9d640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe49c160be0 0x7fe49c0c4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe49c160be0 0x7fe49c0c4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 -- 172.21.15.70:0/2019321678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c0094d0 con 0x7fe49c0c9620 2023-08-25T09:43:46.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe49a59c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe49c069850 0x7fe49c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:46.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe49ad9d640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe49c160be0 0x7fe49c0c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:46.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.829+0000 7fe493fff640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 0x7fe49c1f6bc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe48400c7f0 tx=0x7fe48400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:46.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.830+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484007a30 con 0x7fe49c0c9620 2023-08-25T09:43:46.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.830+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe484007b90 con 0x7fe49c0c9620 2023-08-25T09:43:46.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.830+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48400ce40 con 0x7fe49c0c9620 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.830+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c0c7070 con 0x7fe49c0c9620 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.830+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c0c7630 con 0x7fe49c0c9620 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.831+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c0c2e00 con 0x7fe49c0c9620 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.832+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe4840113f0 con 0x7fe49c0c9620 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.833+0000 7fe492ffd640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe47c06a050 0x7fe47c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.833+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe4840883f0 con 0x7fe49c0c9620 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.833+0000 7fe49a59c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe47c06a050 0x7fe47c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:46.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.834+0000 7fe49a59c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe47c06a050 0x7fe47c06c510 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe48c036000 tx=0x7fe48c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:46.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:46.837+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe49c0c2e00 con 0x7fe49c0c9620 2023-08-25T09:43:47.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.025+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe49c0c2e00 con 0x7fe49c0c9620 2023-08-25T09:43:47.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.031+0000 7fe492ffd640 1 -- 172.21.15.70:0/2019321678 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe49c0c2e00 con 0x7fe49c0c9620 2023-08-25T09:43:47.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe47c06a050 msgr2=0x7fe47c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe47c06a050 0x7fe47c06c510 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe48c036000 tx=0x7fe48c03b040 comp rx=0 tx=0).stop 2023-08-25T09:43:47.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 msgr2=0x7fe49c1f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 0x7fe49c1f6bc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe48400c7f0 tx=0x7fe48400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 shutdown_connections 2023-08-25T09:43:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe47c06a050 0x7fe47c06c510 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe49c160be0 0x7fe49c0c4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe49c0c9620 0x7fe49c1f6bc0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 --2- 172.21.15.70:0/2019321678 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe49c069850 0x7fe49c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 >> 172.21.15.70:0/2019321678 conn(0x7fe49c0cb0d0 msgr2=0x7fe49c15c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 shutdown_connections 2023-08-25T09:43:47.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.034+0000 7fe4a0f0c640 1 -- 172.21.15.70:0/2019321678 wait complete. 2023-08-25T09:43:47.056 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:43:47.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:43:47.174 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:43:47.301 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:43:47.302 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:43:47.303 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:47.364 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:43:47.365 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:43:47.365 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:47.424 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:43:47.425 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:43:47.425 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:47.484 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:43:47.485 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:43:47.485 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:43:47.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:43:47.543 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:43:47.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:43:47.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.770+0000 7f5d58871640 1 Processor -- start 2023-08-25T09:43:47.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.770+0000 7f5d58871640 1 -- start start 2023-08-25T09:43:47.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.771+0000 7f5d58871640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d5415a7c0 0x7f5d540b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:47.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d58871640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540cad20 0x7f5d540cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:47.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d58871640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5415e7c0 con 0x7f5d540b12f0 2023-08-25T09:43:47.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d58871640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540cb6f0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d58871640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54071b50 con 0x7f5d540cad20 2023-08-25T09:43:47.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d5415a7c0 0x7f5d540b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:47.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5d5415a7c0 0x7f5d540b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37396/0 (socket says 172.21.15.70:37396) 2023-08-25T09:43:47.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 -- 172.21.15.70:0/3316966877 learned_addr learned my addr 172.21.15.70:0/3316966877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:47.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d52ffd640 1 --1- 172.21.15.70:0/3316966877 >> v1:172.21.15.70:6789/0 conn(0x7f5d540b12f0 0x7f5d5415ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33206/0 (socket says 172.21.15.70:33206) 2023-08-25T09:43:47.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 -- 172.21.15.70:0/3316966877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540cad20 msgr2=0x7f5d540cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:47.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 --2- 172.21.15.70:0/3316966877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540cad20 0x7f5d540cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 -- 172.21.15.70:0/3316966877 >> v1:172.21.15.70:6789/0 conn(0x7f5d540b12f0 legacy=0x7f5d5415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:47.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.772+0000 7f5d537fe640 1 -- 172.21.15.70:0/3316966877 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5415e490 con 0x7f5d5415a7c0 2023-08-25T09:43:47.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.774+0000 7f5d537fe640 1 --2- 172.21.15.70:0/3316966877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540b3560 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009870 tx=0x7f5d4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f955f0fd0dd2dd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:47.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.774+0000 7f5d527fc640 1 -- 172.21.15.70:0/3316966877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44030840 con 0x7f5d5415a7c0 2023-08-25T09:43:47.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.774+0000 7f5d527fc640 1 -- 172.21.15.70:0/3316966877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440309a0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.774+0000 7f5d527fc640 1 -- 172.21.15.70:0/3316966877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44030d30 con 0x7f5d5415a7c0 2023-08-25T09:43:47.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 -- 172.21.15.70:0/3316966877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 msgr2=0x7f5d540b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 --2- 172.21.15.70:0/3316966877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540b3560 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009870 tx=0x7f5d4402b980 comp rx=0 tx=0).stop 2023-08-25T09:43:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 -- 172.21.15.70:0/3316966877 shutdown_connections 2023-08-25T09:43:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 --2- 172.21.15.70:0/3316966877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540cad20 0x7f5d540cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 --2- 172.21.15.70:0/3316966877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540b3560 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 -- 172.21.15.70:0/3316966877 >> 172.21.15.70:0/3316966877 conn(0x7f5d540affd0 msgr2=0x7f5d540b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 -- 172.21.15.70:0/3316966877 shutdown_connections 2023-08-25T09:43:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.775+0000 7f5d58871640 1 -- 172.21.15.70:0/3316966877 wait complete. 2023-08-25T09:43:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.776+0000 7f5d58871640 1 Processor -- start 2023-08-25T09:43:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.777+0000 7f5d58871640 1 -- start start 2023-08-25T09:43:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.777+0000 7f5d58871640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540b12f0 0x7f5d5414bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d58871640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d540cad20 0x7f5d540c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d58871640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d58871640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5414c060 con 0x7f5d5415a7c0 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d58871640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c2d10 con 0x7f5d540cad20 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d58871640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c2100 con 0x7f5d540b12f0 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37412/0 (socket says 172.21.15.70:37412) 2023-08-25T09:43:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 -- 172.21.15.70:0/2973883894 learned_addr learned my addr 172.21.15.70:0/2973883894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d537fe640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540b12f0 0x7f5d5414bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d52ffd640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d540cad20 0x7f5d540c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 -- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540b12f0 msgr2=0x7f5d5414bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540b12f0 0x7f5d5414bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 -- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d540cad20 msgr2=0x7f5d540c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d540cad20 0x7f5d540c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d53fff640 1 -- 172.21.15.70:0/2973883894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440094d0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.778+0000 7f5d52ffd640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d540cad20 0x7f5d540c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.779+0000 7f5d53fff640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540c9a20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b970 tx=0x7f5d4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.779+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c00cb40 con 0x7f5d5415a7c0 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.779+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c00ccc0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.779+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c014680 con 0x7f5d5415a7c0 2023-08-25T09:43:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.779+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540c23e0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.779+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540c29a0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.780+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d540c2ec0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.781+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d4c01e440 con 0x7f5d5415a7c0 2023-08-25T09:43:47.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.782+0000 7f5d3bfff640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d4006a050 0x7f5d4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:47.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.782+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5d4c0881b0 con 0x7f5d5415a7c0 2023-08-25T09:43:47.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.782+0000 7f5d537fe640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d4006a050 0x7f5d4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:47.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.783+0000 7f5d537fe640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d4006a050 0x7f5d4006c510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d44036000 tx=0x7f5d4403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:47.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:47.786+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d540c2ec0 con 0x7f5d5415a7c0 2023-08-25T09:43:48.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.010+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d540c2e80 con 0x7f5d5415a7c0 2023-08-25T09:43:48.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.011+0000 7f5d3bfff640 1 -- 172.21.15.70:0/2973883894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f5d4c057640 con 0x7f5d5415a7c0 2023-08-25T09:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:48.014 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/3963511862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3963511862},{"type":"v1","addr":"172.21.15.155:6834","nonce":3963511862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":4892,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3003042985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3003042985},{"type":"v1","addr":"172.21.15.155:6835","nonce":3003042985}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6318,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3193304425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3193304425},{"type":"v1","addr":"172.21.15.70:6835","nonce":3193304425}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6684,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/144533415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":144533415},{"type":"v1","addr":"172.21.15.70:6837","nonce":144533415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-08-25T09:43:48.014 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 34 2023-08-25T09:43:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d4006a050 msgr2=0x7f5d4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:48.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d4006a050 0x7f5d4006c510 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d44036000 tx=0x7f5d4403b040 comp rx=0 tx=0).stop 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 msgr2=0x7f5d540c9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540c9a20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c00b970 tx=0x7f5d4c00be30 comp rx=0 tx=0).stop 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 shutdown_connections 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5d4006a050 0x7f5d4006c510 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5d5415a7c0 0x7f5d540c9a20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5d540cad20 0x7f5d540c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 --2- 172.21.15.70:0/2973883894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5d540b12f0 0x7f5d5414bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 >> 172.21.15.70:0/2973883894 conn(0x7f5d540affd0 msgr2=0x7f5d541e3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:48.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 shutdown_connections 2023-08-25T09:43:48.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.014+0000 7f5d58871640 1 -- 172.21.15.70:0/2973883894 wait complete. 2023-08-25T09:43:48.036 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:43:48.036 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:43:48.036 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:43:48.036 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:43:48.037 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:43:48.037 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:43:48.037 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:43:48.037 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:43:48.038 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:43:48.038 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:43:48.038 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:43:48.039 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:43:48.040 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:43:48.040 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:43:48.040 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:43:48.041 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:43:48.053 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:43:48.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:43:48.057 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:43:48.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:43:48.066 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:43:48.066 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:43:48.072 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:43:48.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:43:48.105 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:43:48.106 DEBUG:teuthology.parallel:result is None 2023-08-25T09:43:48.113 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:43:48.113 DEBUG:teuthology.parallel:result is None 2023-08-25T09:43:48.119 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:43:48.120 DEBUG:teuthology.parallel:result is None 2023-08-25T09:43:48.125 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:43:48.126 DEBUG:teuthology.parallel:result is None 2023-08-25T09:43:48.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:43:48.266 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:43:48.273 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:43:48.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-08-25T09:43:48.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-08-25T09:43:48.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-08-25T09:43:48.285 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:43:48.285 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:43:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.325+0000 7fc3a759e640 1 Processor -- start 2023-08-25T09:43:48.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.325+0000 7fc3a759e640 1 -- start start 2023-08-25T09:43:48.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.327+0000 7fc3a759e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3a80b7df0 0x7fc3a8070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.327+0000 7fc3a759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.327+0000 7fc3a759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a80b81d0 con 0x7fc3a80cf210 2023-08-25T09:43:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.327+0000 7fc3a759e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a81604f0 con 0x7fc3a80b7df0 2023-08-25T09:43:48.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.327+0000 7fc3a759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a815d3c0 con 0x7fc3a815cf00 2023-08-25T09:43:48.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.327+0000 7fc3a659c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3a80b7df0 0x7fc3a8070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a6d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc3a80cf210 0x7fc3a80cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:33210/0 (socket says 172.21.15.70:33210) 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a6d9d640 1 -- 172.21.15.70:0/2875165959 learned_addr learned my addr 172.21.15.70:0/2875165959 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a5d9b640 1 --2- 172.21.15.70:0/2875165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a659c640 1 -- 172.21.15.70:0/2875165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 msgr2=0x7fc3a815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a659c640 1 --2- 172.21.15.70:0/2875165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a659c640 1 -- 172.21.15.70:0/2875165959 >> v1:172.21.15.70:6789/0 conn(0x7fc3a80cf210 legacy=0x7fc3a80cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:48.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.328+0000 7fc3a659c640 1 -- 172.21.15.70:0/2875165959 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a815d6a0 con 0x7fc3a80b7df0 2023-08-25T09:43:48.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.329+0000 7fc3a659c640 1 --2- 172.21.15.70:0/2875165959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80b7df0 0x7fc3a8070880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc3980097a0 tx=0x7fc39802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed8f8faf222d809 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:48.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.330+0000 7fc3a559a640 1 -- 172.21.15.70:0/2875165959 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39802d040 con 0x7fc3a80b7df0 2023-08-25T09:43:48.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.330+0000 7fc3a559a640 1 -- 172.21.15.70:0/2875165959 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc398031810 con 0x7fc3a80b7df0 2023-08-25T09:43:48.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.330+0000 7fc3a559a640 1 -- 172.21.15.70:0/2875165959 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398031ba0 con 0x7fc3a80b7df0 2023-08-25T09:43:48.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.330+0000 7fc3a759e640 1 -- 172.21.15.70:0/2875165959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80b7df0 msgr2=0x7fc3a8070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.330+0000 7fc3a759e640 1 --2- 172.21.15.70:0/2875165959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80b7df0 0x7fc3a8070880 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc3980097a0 tx=0x7fc39802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.331+0000 7fc3a759e640 1 -- 172.21.15.70:0/2875165959 shutdown_connections 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.331+0000 7fc3a759e640 1 --2- 172.21.15.70:0/2875165959 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.331+0000 7fc3a759e640 1 --2- 172.21.15.70:0/2875165959 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80b7df0 0x7fc3a8070880 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.331+0000 7fc3a759e640 1 -- 172.21.15.70:0/2875165959 >> 172.21.15.70:0/2875165959 conn(0x7fc3a80b43a0 msgr2=0x7fc3a80b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.331+0000 7fc3a759e640 1 -- 172.21.15.70:0/2875165959 shutdown_connections 2023-08-25T09:43:48.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.331+0000 7fc3a759e640 1 -- 172.21.15.70:0/2875165959 wait complete. 2023-08-25T09:43:48.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.333+0000 7fc3a759e640 1 Processor -- start 2023-08-25T09:43:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.333+0000 7fc3a759e640 1 -- start start 2023-08-25T09:43:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a80b7df0 0x7fc3a81550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 0x7fc3a80c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a8156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:48.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b5960 con 0x7fc3a80cf210 2023-08-25T09:43:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b5c40 con 0x7fc3a80b7df0 2023-08-25T09:43:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a80b5f20 con 0x7fc3a815cf00 2023-08-25T09:43:48.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 0x7fc3a80c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 0x7fc3a80c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37450/0 (socket says 172.21.15.70:37450) 2023-08-25T09:43:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a5d9b640 1 -- 172.21.15.70:0/602916271 learned_addr learned my addr 172.21.15.70:0/602916271 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:48.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a659c640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a80b7df0 0x7fc3a81550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:48.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a6d9d640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:48.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.334+0000 7fc3a5d9b640 1 -- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 msgr2=0x7fc3a8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a5d9b640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a8156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a5d9b640 1 -- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a80b7df0 msgr2=0x7fc3a81550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:48.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a5d9b640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a80b7df0 0x7fc3a81550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a5d9b640 1 -- 172.21.15.70:0/602916271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398009400 con 0x7fc3a80cf210 2023-08-25T09:43:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a659c640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a80b7df0 0x7fc3a81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:43:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a6d9d640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.335+0000 7fc3a5d9b640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 0x7fc3a80c9f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00e9b0 tx=0x7fc39c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.337+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c009670 con 0x7fc3a80cf210 2023-08-25T09:43:48.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.337+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39c00c9a0 con 0x7fc3a80cf210 2023-08-25T09:43:48.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.337+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a80b6200 con 0x7fc3a80cf210 2023-08-25T09:43:48.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.337+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00cd50 con 0x7fc3a80cf210 2023-08-25T09:43:48.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.337+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a8155630 con 0x7fc3a80cf210 2023-08-25T09:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.338+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a80b1860 con 0x7fc3a80cf210 2023-08-25T09:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.339+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc39c013470 con 0x7fc3a80cf210 2023-08-25T09:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.340+0000 7fc396ffd640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc38806a050 0x7fc38806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.340+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc39c088520 con 0x7fc3a80cf210 2023-08-25T09:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.340+0000 7fc3a659c640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc38806a050 0x7fc38806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:48.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.341+0000 7fc3a659c640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc38806a050 0x7fc38806c510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc398036000 tx=0x7fc39803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:48.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.345+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc39c053f20 con 0x7fc3a80cf210 2023-08-25T09:43:48.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:48.547+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc3a8066fb0 con 0x7fc3a80cf210 2023-08-25T09:43:49.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.346+0000 7fc396ffd640 1 -- 172.21.15.70:0/602916271 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc39c057540 con 0x7fc3a80cf210 2023-08-25T09:43:49.348 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:43:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.350+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc38806a050 msgr2=0x7fc38806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.350+0000 7fc3a759e640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc38806a050 0x7fc38806c510 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc398036000 tx=0x7fc39803c040 comp rx=0 tx=0).stop 2023-08-25T09:43:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 msgr2=0x7fc3a80c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 0x7fc3a80c9f20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc39c00e9b0 tx=0x7fc39c00ee70 comp rx=0 tx=0).stop 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 shutdown_connections 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc38806a050 0x7fc38806c510 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3a815cf00 0x7fc3a8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3a80cf210 0x7fc3a80c9f20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 --2- 172.21.15.70:0/602916271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3a80b7df0 0x7fc3a81550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.351+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 >> 172.21.15.70:0/602916271 conn(0x7fc3a80b43a0 msgr2=0x7fc3a806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:49.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.352+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 shutdown_connections 2023-08-25T09:43:49.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.352+0000 7fc3a759e640 1 -- 172.21.15.70:0/602916271 wait complete. 2023-08-25T09:43:49.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:43:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.586+0000 7f57acd76640 1 Processor -- start 2023-08-25T09:43:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.587+0000 7f57acd76640 1 -- start start 2023-08-25T09:43:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.588+0000 7f57acd76640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f57a815ec10 0x7f57a815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.588+0000 7f57acd76640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80b1b60 con 0x7f57a815e830 2023-08-25T09:43:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.588+0000 7f57acd76640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a806b7c0 con 0x7f57a815c6d0 2023-08-25T09:43:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.588+0000 7f57acd76640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a806bd80 con 0x7f57a815ec10 2023-08-25T09:43:49.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f57a815e830 0x7f57a806ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49000/0 (socket says 172.21.15.70:49000) 2023-08-25T09:43:49.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a659c640 1 -- 172.21.15.70:0/465026858 learned_addr learned my addr 172.21.15.70:0/465026858 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927982574 0 0) 0x7f57a80b1b60 con 0x7f57a815e830 2023-08-25T09:43:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5794003520 con 0x7f57a815e830 2023-08-25T09:43:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f579ffff640 1 --2- 172.21.15.70:0/465026858 >> v2:172.21.15.155:3300/0 conn(0x7f57a815ec10 0x7f57a815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1502179976 0 0) 0x7f5794003520 con 0x7f57a815e830 2023-08-25T09:43:49.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 >> v2:172.21.15.155:3300/0 conn(0x7f57a815ec10 msgr2=0x7f57a815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:49.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 --2- 172.21.15.70:0/465026858 >> v2:172.21.15.155:3300/0 conn(0x7f57a815ec10 0x7f57a815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 >> v1:172.21.15.155:6789/0 conn(0x7f57a815c6d0 legacy=0x7f57a80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:49.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a806baa0 con 0x7f57a815e830 2023-08-25T09:43:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.589+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5798001670 con 0x7f57a815e830 2023-08-25T09:43:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 >> v1:172.21.15.70:6789/0 conn(0x7f57a815e830 legacy=0x7f57a806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5794003970 0x7f5794003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57940047c0 0x7f5794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57940052a0 0x7f579400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a806bd80 con 0x7f5794003970 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579400c150 con 0x7f57940047c0 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579400c310 con 0x7f57940052a0 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a659c640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5794003970 0x7f5794003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a6d9d640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57940052a0 0x7f579400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a659c640 1 -- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57940052a0 msgr2=0x7f579400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:49.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a659c640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57940052a0 0x7f579400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a659c640 1 -- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57940047c0 msgr2=0x7f5794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:49.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a659c640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57940047c0 0x7f5794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a659c640 1 -- 172.21.15.70:0/465026858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794003520 con 0x7f5794003970 2023-08-25T09:43:49.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a6d9d640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57940052a0 0x7f579400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:49.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.590+0000 7f57a6d9d640 1 -- 172.21.15.70:0/465026858 reap_dead start 2023-08-25T09:43:49.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.591+0000 7f57a659c640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5794003970 0x7f5794003d70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f579800c210 tx=0x7f57940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932b4c69b2a0757f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798005ad0 con 0x7f5794003970 2023-08-25T09:43:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798005c30 con 0x7f5794003970 2023-08-25T09:43:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57a5d9b640 1 -- 172.21.15.70:0/465026858 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579802c870 con 0x7f5794003970 2023-08-25T09:43:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 -- 172.21.15.70:0/465026858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5794003970 msgr2=0x7f5794003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5794003970 0x7f5794003d70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f579800c210 tx=0x7f57940057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 -- 172.21.15.70:0/465026858 shutdown_connections 2023-08-25T09:43:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 --2- 172.21.15.70:0/465026858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5794003970 0x7f5794003d70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 -- 172.21.15.70:0/465026858 >> 172.21.15.70:0/465026858 conn(0x7f57a8069850 msgr2=0x7f57a8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 -- 172.21.15.70:0/465026858 shutdown_connections 2023-08-25T09:43:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.592+0000 7f57acd76640 1 -- 172.21.15.70:0/465026858 wait complete. 2023-08-25T09:43:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.594+0000 7f57acd76640 1 Processor -- start 2023-08-25T09:43:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.594+0000 7f57acd76640 1 -- start start 2023-08-25T09:43:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.595+0000 7f57acd76640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5794003970 0x7f57a8147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.595+0000 7f57acd76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 0x7f57a80a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.595+0000 7f57acd76640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57a8147bf0 0x7f57a8158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.595+0000 7f57acd76640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81f23c0 con 0x7f57a80708c0 2023-08-25T09:43:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.595+0000 7f57acd76640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81f31e0 con 0x7f57a8147bf0 2023-08-25T09:43:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.595+0000 7f57acd76640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81f25d0 con 0x7f5794003970 2023-08-25T09:43:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 0x7f57a80a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 0x7f57a80a8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41972/0 (socket says 172.21.15.70:41972) 2023-08-25T09:43:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 -- 172.21.15.70:0/1707861360 learned_addr learned my addr 172.21.15.70:0/1707861360 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f57a659c640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5794003970 0x7f57a8147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f57a6d9d640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57a8147bf0 0x7f57a8158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 -- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5794003970 msgr2=0x7f57a8147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5794003970 0x7f57a8147330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 -- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57a8147bf0 msgr2=0x7f57a8158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57a8147bf0 0x7f57a8158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 -- 172.21.15.70:0/1707861360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a81f28b0 con 0x7f57a80708c0 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.596+0000 7f579ffff640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 0x7f57a80a8690 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f579000d7d0 tx=0x7f579000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.597+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000b6c0 con 0x7f57a80708c0 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.597+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579000b860 con 0x7f57a80708c0 2023-08-25T09:43:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.597+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579000bc10 con 0x7f57a80708c0 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.597+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a81f2b90 con 0x7f57a80708c0 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.597+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a8147870 con 0x7f57a80708c0 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.598+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a81f3350 con 0x7f57a80708c0 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.599+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f579001e460 con 0x7f57a80708c0 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.600+0000 7f579effd640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f578806a080 0x7f578806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.600+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5790088660 con 0x7f57a80708c0 2023-08-25T09:43:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.600+0000 7f57a659c640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f578806a080 0x7f578806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:49.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.601+0000 7f57a659c640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f578806a080 0x7f578806c540 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5798034000 tx=0x7f5798009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:49.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.604+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5790054090 con 0x7f57a80708c0 2023-08-25T09:43:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.860+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f57a804e420 con 0x7f57a80708c0 2023-08-25T09:43:49.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.861+0000 7f579effd640 1 -- 172.21.15.70:0/1707861360 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f57900073e0 con 0x7f57a80708c0 2023-08-25T09:43:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:49.863 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:43:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:43:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:43:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:43:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:43:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:43:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:43:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:43:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:43:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:43:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:43:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:43:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:43:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:43:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:43:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:43:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:43:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:43:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:43:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:43:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:43:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:43:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:43:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:43:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:43:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:43:49.894 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f578806a080 msgr2=0x7f578806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f578806a080 0x7f578806c540 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5798034000 tx=0x7f5798009030 comp rx=0 tx=0).stop 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 msgr2=0x7f57a80a8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 0x7f57a80a8690 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f579000d7d0 tx=0x7f579000dc90 comp rx=0 tx=0).stop 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 shutdown_connections 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f578806a080 0x7f578806c540 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57a8147bf0 0x7f57a8158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57a80708c0 0x7f57a80a8690 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 --2- 172.21.15.70:0/1707861360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5794003970 0x7f57a8147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:49.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.863+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 >> 172.21.15.70:0/1707861360 conn(0x7f57a8069850 msgr2=0x7f57a815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:49.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.864+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 shutdown_connections 2023-08-25T09:43:49.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:49.864+0000 7f57acd76640 1 -- 172.21.15.70:0/1707861360 wait complete. 2023-08-25T09:43:49.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:50.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.113+0000 7f95ab883640 1 Processor -- start 2023-08-25T09:43:50.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.113+0000 7f95ab883640 1 -- start start 2023-08-25T09:43:50.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.115+0000 7f95ab883640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 0x7f95a40b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.115+0000 7f95ab883640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f95a406f0a0 0x7f95a40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.115+0000 7f95ab883640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a406ae10 con 0x7f95a4159100 2023-08-25T09:43:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.115+0000 7f95ab883640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a406ba50 con 0x7f95a406f0a0 2023-08-25T09:43:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.115+0000 7f95ab883640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a406b0b0 con 0x7f95a40709a0 2023-08-25T09:43:50.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.115+0000 7f95a8e1e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 0x7f95a40b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a8e1e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 0x7f95a40b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:57056/0 (socket says 172.21.15.70:57056) 2023-08-25T09:43:50.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a8e1e640 1 -- 172.21.15.70:0/3998869650 learned_addr learned my addr 172.21.15.70:0/3998869650 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:50.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a9e20640 1 --2- 172.21.15.70:0/3998869650 >> v2:172.21.15.155:3300/0 conn(0x7f95a406f0a0 0x7f95a40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a8e1e640 1 -- 172.21.15.70:0/3998869650 >> v2:172.21.15.155:3300/0 conn(0x7f95a406f0a0 msgr2=0x7f95a40b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a8e1e640 1 --2- 172.21.15.70:0/3998869650 >> v2:172.21.15.155:3300/0 conn(0x7f95a406f0a0 0x7f95a40b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a8e1e640 1 -- 172.21.15.70:0/3998869650 >> v1:172.21.15.155:6789/0 conn(0x7f95a4159100 legacy=0x7f95a406be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.116+0000 7f95a8e1e640 1 -- 172.21.15.70:0/3998869650 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a406b390 con 0x7f95a40709a0 2023-08-25T09:43:50.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.118+0000 7f95a8e1e640 1 --2- 172.21.15.70:0/3998869650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 0x7f95a40b1960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f959400aa10 tx=0x7f959402cdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=672e81bc1fd6666d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.118+0000 7f959bfff640 1 -- 172.21.15.70:0/3998869650 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959402db80 con 0x7f95a40709a0 2023-08-25T09:43:50.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.118+0000 7f959bfff640 1 -- 172.21.15.70:0/3998869650 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959402dce0 con 0x7f95a40709a0 2023-08-25T09:43:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.118+0000 7f959bfff640 1 -- 172.21.15.70:0/3998869650 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9594035660 con 0x7f95a40709a0 2023-08-25T09:43:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 -- 172.21.15.70:0/3998869650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 msgr2=0x7f95a40b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 --2- 172.21.15.70:0/3998869650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 0x7f95a40b1960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f959400aa10 tx=0x7f959402cdd0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 -- 172.21.15.70:0/3998869650 shutdown_connections 2023-08-25T09:43:50.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 --2- 172.21.15.70:0/3998869650 >> v2:172.21.15.155:3300/0 conn(0x7f95a406f0a0 0x7f95a40b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 --2- 172.21.15.70:0/3998869650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a40709a0 0x7f95a40b1960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 -- 172.21.15.70:0/3998869650 >> 172.21.15.70:0/3998869650 conn(0x7f95a4069850 msgr2=0x7f95a4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:50.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 -- 172.21.15.70:0/3998869650 shutdown_connections 2023-08-25T09:43:50.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.119+0000 7f95ab883640 1 -- 172.21.15.70:0/3998869650 wait complete. 2023-08-25T09:43:50.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.122+0000 7f95ab883640 1 Processor -- start 2023-08-25T09:43:50.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.122+0000 7f95ab883640 1 -- start start 2023-08-25T09:43:50.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95ab883640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a406f0a0 0x7f95a4153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95ab883640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95a40709a0 0x7f95a4156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95ab883640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 0x7f95a41eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95ab883640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a41e2950 con 0x7f95a4159100 2023-08-25T09:43:50.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95ab883640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a41e2c30 con 0x7f95a40709a0 2023-08-25T09:43:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95ab883640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a41e2f10 con 0x7f95a406f0a0 2023-08-25T09:43:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95a9e20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 0x7f95a41eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95a9e20640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 0x7f95a41eff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41982/0 (socket says 172.21.15.70:41982) 2023-08-25T09:43:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.123+0000 7f95a9e20640 1 -- 172.21.15.70:0/2608128777 learned_addr learned my addr 172.21.15.70:0/2608128777 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a961f640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a406f0a0 0x7f95a4153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a8e1e640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95a40709a0 0x7f95a4156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a9e20640 1 -- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a406f0a0 msgr2=0x7f95a4153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a9e20640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a406f0a0 0x7f95a4153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a9e20640 1 -- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95a40709a0 msgr2=0x7f95a4156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a9e20640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95a40709a0 0x7f95a4156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a9e20640 1 -- 172.21.15.70:0/2608128777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959400a670 con 0x7f95a4159100 2023-08-25T09:43:50.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a8e1e640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95a40709a0 0x7f95a4156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.124+0000 7f95a9e20640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 0x7f95a41eff50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e930 tx=0x7f959c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.125+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c009690 con 0x7f95a4159100 2023-08-25T09:43:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.125+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00c980 con 0x7f95a4159100 2023-08-25T09:43:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.125+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a41e31f0 con 0x7f95a4159100 2023-08-25T09:43:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.125+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00cd30 con 0x7f95a4159100 2023-08-25T09:43:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.125+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a4153960 con 0x7f95a4159100 2023-08-25T09:43:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.126+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a41e3710 con 0x7f95a4159100 2023-08-25T09:43:50.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.127+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f959c013470 con 0x7f95a4159100 2023-08-25T09:43:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.128+0000 7f9599ffb640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f958806a050 0x7f958806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.128+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f959c088510 con 0x7f95a4159100 2023-08-25T09:43:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.128+0000 7f95a961f640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f958806a050 0x7f958806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.129+0000 7f95a961f640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f958806a050 0x7f958806c510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f95a41e6c70 tx=0x7f95a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.132+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f95a41e3710 con 0x7f95a4159100 2023-08-25T09:43:50.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.327+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95a41e3710 con 0x7f95a4159100 2023-08-25T09:43:50.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.328+0000 7f9599ffb640 1 -- 172.21.15.70:0/2608128777 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+14654 (secure 0 0 0) 0x7f95a41e3710 con 0x7f95a4159100 2023-08-25T09:43:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:50.332 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":90,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:49.338922+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:50.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.331+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f958806a050 msgr2=0x7f958806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.331+0000 7f95ab883640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f958806a050 0x7f958806c510 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f95a41e6c70 tx=0x7f95a0009040 comp rx=0 tx=0).stop 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 msgr2=0x7f95a41eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 0x7f95a41eff50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e930 tx=0x7f959c00edf0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 shutdown_connections 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f958806a050 0x7f958806c510 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f95a4159100 0x7f95a41eff50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f95a40709a0 0x7f95a4156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 --2- 172.21.15.70:0/2608128777 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f95a406f0a0 0x7f95a4153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 >> 172.21.15.70:0/2608128777 conn(0x7f95a4069850 msgr2=0x7f95a415d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:50.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 shutdown_connections 2023-08-25T09:43:50.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.332+0000 7f95ab883640 1 -- 172.21.15.70:0/2608128777 wait complete. 2023-08-25T09:43:50.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:43:50.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:43:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.561+0000 7fc826b0f640 1 Processor -- start 2023-08-25T09:43:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.561+0000 7fc826b0f640 1 -- start start 2023-08-25T09:43:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.563+0000 7fc826b0f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc820072f30 0x7fc8200acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.563+0000 7fc826b0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 0x7fc8200ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.563+0000 7fc826b0f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82015d340 con 0x7fc820153d70 2023-08-25T09:43:50.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.563+0000 7fc826b0f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc820154150 con 0x7fc820072f30 2023-08-25T09:43:50.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.563+0000 7fc826b0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82015d620 con 0x7fc8200b2870 2023-08-25T09:43:50.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.564+0000 7fc8250ac640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 0x7fc8200ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.564+0000 7fc8248ab640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc820153d70 0x7fc820072820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:46918/0 (socket says 172.21.15.70:46918) 2023-08-25T09:43:50.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.564+0000 7fc8250ac640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 0x7fc8200ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:57062/0 (socket says 172.21.15.70:57062) 2023-08-25T09:43:50.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.564+0000 7fc8248ab640 1 -- 172.21.15.70:0/2633880603 learned_addr learned my addr 172.21.15.70:0/2633880603 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:50.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.564+0000 7fc81ffff640 1 --2- 172.21.15.70:0/2633880603 >> v2:172.21.15.155:3300/0 conn(0x7fc820072f30 0x7fc8200acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.565+0000 7fc8250ac640 1 -- 172.21.15.70:0/2633880603 >> v2:172.21.15.155:3300/0 conn(0x7fc820072f30 msgr2=0x7fc8200acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:50.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.565+0000 7fc8250ac640 1 --2- 172.21.15.70:0/2633880603 >> v2:172.21.15.155:3300/0 conn(0x7fc820072f30 0x7fc8200acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.565+0000 7fc8250ac640 1 -- 172.21.15.70:0/2633880603 >> v1:172.21.15.155:6789/0 conn(0x7fc820153d70 legacy=0x7fc820072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.565+0000 7fc81f7fe640 1 -- 172.21.15.70:0/2633880603 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1479649803 0 0) 0x7fc82015d340 con 0x7fc820153d70 2023-08-25T09:43:50.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.565+0000 7fc8250ac640 1 -- 172.21.15.70:0/2633880603 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82015d900 con 0x7fc8200b2870 2023-08-25T09:43:50.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.566+0000 7fc8250ac640 1 --2- 172.21.15.70:0/2633880603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 0x7fc8200ad1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d90 tx=0x7fc81802be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e8e50d308881d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:50.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.566+0000 7fc81f7fe640 1 -- 172.21.15.70:0/2633880603 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81802cbd0 con 0x7fc8200b2870 2023-08-25T09:43:50.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.567+0000 7fc81f7fe640 1 -- 172.21.15.70:0/2633880603 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81802cd30 con 0x7fc8200b2870 2023-08-25T09:43:50.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.567+0000 7fc81f7fe640 1 -- 172.21.15.70:0/2633880603 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81802cea0 con 0x7fc8200b2870 2023-08-25T09:43:50.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.567+0000 7fc826b0f640 1 -- 172.21.15.70:0/2633880603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 msgr2=0x7fc8200ad1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.567+0000 7fc826b0f640 1 --2- 172.21.15.70:0/2633880603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 0x7fc8200ad1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d90 tx=0x7fc81802be20 comp rx=0 tx=0).stop 2023-08-25T09:43:50.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.568+0000 7fc826b0f640 1 -- 172.21.15.70:0/2633880603 shutdown_connections 2023-08-25T09:43:50.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.568+0000 7fc826b0f640 1 --2- 172.21.15.70:0/2633880603 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200b2870 0x7fc8200ad1b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.568+0000 7fc826b0f640 1 --2- 172.21.15.70:0/2633880603 >> v2:172.21.15.155:3300/0 conn(0x7fc820072f30 0x7fc8200acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.568+0000 7fc826b0f640 1 -- 172.21.15.70:0/2633880603 >> 172.21.15.70:0/2633880603 conn(0x7fc8200cfc70 msgr2=0x7fc8200d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:50.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.568+0000 7fc826b0f640 1 -- 172.21.15.70:0/2633880603 shutdown_connections 2023-08-25T09:43:50.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.568+0000 7fc826b0f640 1 -- 172.21.15.70:0/2633880603 wait complete. 2023-08-25T09:43:50.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.569+0000 7fc826b0f640 1 Processor -- start 2023-08-25T09:43:50.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 -- start start 2023-08-25T09:43:50.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc820072f30 0x7fc8201542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200b2870 0x7fc820154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 0x7fc820154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82006cab0 con 0x7fc820153d70 2023-08-25T09:43:50.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82006cd90 con 0x7fc8200b2870 2023-08-25T09:43:50.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.570+0000 7fc826b0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82006d070 con 0x7fc820072f30 2023-08-25T09:43:50.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 0x7fc820154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 0x7fc820154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41998/0 (socket says 172.21.15.70:41998) 2023-08-25T09:43:50.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 -- 172.21.15.70:0/3453627921 learned_addr learned my addr 172.21.15.70:0/3453627921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:50.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8248ab640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc820072f30 0x7fc8201542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc81ffff640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200b2870 0x7fc820154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 -- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc820072f30 msgr2=0x7fc8201542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc820072f30 0x7fc8201542c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 -- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200b2870 msgr2=0x7fc820154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200b2870 0x7fc820154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.571+0000 7fc8250ac640 1 -- 172.21.15.70:0/3453627921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818009a70 con 0x7fc820153d70 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.572+0000 7fc8250ac640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 0x7fc820154d40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d90 tx=0x7fc818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:50.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.572+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81802cea0 con 0x7fc820153d70 2023-08-25T09:43:50.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.572+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc818040650 con 0x7fc820153d70 2023-08-25T09:43:50.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.572+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818040a00 con 0x7fc820153d70 2023-08-25T09:43:50.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.572+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc82006d350 con 0x7fc820153d70 2023-08-25T09:43:50.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.572+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc820150890 con 0x7fc820153d70 2023-08-25T09:43:50.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.573+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8200738b0 con 0x7fc820153d70 2023-08-25T09:43:50.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.574+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc818040b60 con 0x7fc820153d70 2023-08-25T09:43:50.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.575+0000 7fc81d7fa640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:50.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.575+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc8180ab030 con 0x7fc820153d70 2023-08-25T09:43:50.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.575+0000 7fc8248ab640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:50.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.576+0000 7fc8248ab640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc814000f60 tx=0x7fc814007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:50.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.579+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc8180dd990 con 0x7fc820153d70 2023-08-25T09:43:50.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.766+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8200cd0f0 con 0x7fc820153d70 2023-08-25T09:43:50.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.767+0000 7fc81d7fa640 1 -- 172.21.15.70:0/3453627921 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc818076af0 con 0x7fc820153d70 2023-08-25T09:43:50.770 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:43:50.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.770+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 msgr2=0x7fc80406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.770+0000 7fc826b0f640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc814000f60 tx=0x7fc814007c00 comp rx=0 tx=0).stop 2023-08-25T09:43:50.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.770+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 msgr2=0x7fc820154d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:50.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.770+0000 7fc826b0f640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 0x7fc820154d40 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fc818009d90 tx=0x7fc818004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 shutdown_connections 2023-08-25T09:43:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80406a050 0x7fc80406c510 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc820153d70 0x7fc820154d40 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8200b2870 0x7fc820154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 --2- 172.21.15.70:0/3453627921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc820072f30 0x7fc8201542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:50.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 >> 172.21.15.70:0/3453627921 conn(0x7fc8200cfc70 msgr2=0x7fc8200acfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 shutdown_connections 2023-08-25T09:43:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:50.771+0000 7fc826b0f640 1 -- 172.21.15.70:0/3453627921 wait complete. 2023-08-25T09:43:50.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:43:51.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.016+0000 7f5f61fc8640 1 Processor -- start 2023-08-25T09:43:51.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.017+0000 7f5f61fc8640 1 -- start start 2023-08-25T09:43:51.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.018+0000 7f5f61fc8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5f5c15ae50 0x7f5f5c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.018+0000 7f5f61fc8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c0cc5f0 con 0x7f5f5c069f60 2023-08-25T09:43:51.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.018+0000 7f5f61fc8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5c070650 con 0x7f5f5c0cdc20 2023-08-25T09:43:51.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.018+0000 7f5f61fc8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5c070930 con 0x7f5f5c15ae50 2023-08-25T09:43:51.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5f5c069f60 0x7f5f5c0ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49010/0 (socket says 172.21.15.70:49010) 2023-08-25T09:43:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5bfff640 1 -- 172.21.15.70:0/1119190432 learned_addr learned my addr 172.21.15.70:0/1119190432 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990556095 0 0) 0x7f5f5c0cc5f0 con 0x7f5f5c069f60 2023-08-25T09:43:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5b7fe640 1 --2- 172.21.15.70:0/1119190432 >> v2:172.21.15.155:3300/0 conn(0x7f5f5c15ae50 0x7f5f5c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f48003520 con 0x7f5f5c069f60 2023-08-25T09:43:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184363588 0 0) 0x7f5f48003520 con 0x7f5f5c069f60 2023-08-25T09:43:51.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 >> v2:172.21.15.155:3300/0 conn(0x7f5f5c15ae50 msgr2=0x7f5f5c0c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 --2- 172.21.15.70:0/1119190432 >> v2:172.21.15.155:3300/0 conn(0x7f5f5c15ae50 0x7f5f5c0c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 >> v1:172.21.15.155:6789/0 conn(0x7f5f5c0cdc20 legacy=0x7f5f5c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c070c10 con 0x7f5f5c069f60 2023-08-25T09:43:51.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5f540019f0 con 0x7f5f5c069f60 2023-08-25T09:43:51.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.019+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 >> v1:172.21.15.70:6789/0 conn(0x7f5f5c069f60 legacy=0x7f5f5c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5affd640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f48003970 0x7f5f48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5affd640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f480047c0 0x7f5f48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5affd640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f48005180 0x7f5f4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c070930 con 0x7f5f480047c0 2023-08-25T09:43:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4800c0b0 con 0x7f5f48005180 2023-08-25T09:43:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4800c270 con 0x7f5f48003970 2023-08-25T09:43:51.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f53fff640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f480047c0 0x7f5f48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5bfff640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f48005180 0x7f5f4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5b7fe640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f48003970 0x7f5f48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f53fff640 1 -- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f48003970 msgr2=0x7f5f48003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f53fff640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f48003970 0x7f5f48003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f53fff640 1 -- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f48005180 msgr2=0x7f5f4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f53fff640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f48005180 0x7f5f4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f53fff640 1 -- 172.21.15.70:0/1119190432 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48003520 con 0x7f5f480047c0 2023-08-25T09:43:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5bfff640 1 -- 172.21.15.70:0/1119190432 reap_dead start 2023-08-25T09:43:51.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.020+0000 7f5f5b7fe640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f48003970 0x7f5f48003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f53fff640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f480047c0 0x7f5f48004c40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5f44001750 tx=0x7f5f480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7c75b2e058947 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440052d0 con 0x7f5f480047c0 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4402e050 con 0x7f5f480047c0 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f5affd640 1 -- 172.21.15.70:0/1119190432 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44003830 con 0x7f5f480047c0 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f61fc8640 1 -- 172.21.15.70:0/1119190432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f480047c0 msgr2=0x7f5f48004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f480047c0 0x7f5f48004c40 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5f44001750 tx=0x7f5f480057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f61fc8640 1 -- 172.21.15.70:0/1119190432 shutdown_connections 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/1119190432 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f480047c0 0x7f5f48004c40 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.022+0000 7f5f61fc8640 1 -- 172.21.15.70:0/1119190432 >> 172.21.15.70:0/1119190432 conn(0x7f5f5c0cb0d0 msgr2=0x7f5f5c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.023+0000 7f5f61fc8640 1 -- 172.21.15.70:0/1119190432 shutdown_connections 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.023+0000 7f5f61fc8640 1 -- 172.21.15.70:0/1119190432 wait complete. 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.024+0000 7f5f61fc8640 1 Processor -- start 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.024+0000 7f5f61fc8640 1 -- start start 2023-08-25T09:43:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.025+0000 7f5f61fc8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f480047c0 0x7f5f5c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.025+0000 7f5f61fc8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 0x7f5f5c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.025+0000 7f5f61fc8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f5c154d10 0x7f5f5c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.025+0000 7f5f61fc8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0b1c40 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.025+0000 7f5f61fc8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0c5690 con 0x7f5f5c154d10 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.025+0000 7f5f61fc8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c0ae920 con 0x7f5f480047c0 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 0x7f5f5c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 0x7f5f5c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42012/0 (socket says 172.21.15.70:42012) 2023-08-25T09:43:51.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 -- 172.21.15.70:0/838557831 learned_addr learned my addr 172.21.15.70:0/838557831 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f5b7fe640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f480047c0 0x7f5f5c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 -- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f480047c0 msgr2=0x7f5f5c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f5bfff640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f5c154d10 0x7f5f5c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f480047c0 0x7f5f5c0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 -- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f5c154d10 msgr2=0x7f5f5c1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f5c154d10 0x7f5f5c1551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 -- 172.21.15.70:0/838557831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c1556e0 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.026+0000 7f5f53fff640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 0x7f5f5c0b2160 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5f4402bc00 tx=0x7f5f44001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.027+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f440076c0 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.027+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f44007a30 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.027+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5c0aeb70 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.027+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f44006be0 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.027+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5c0af130 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.028+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f5c0701c0 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.029+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5f44006d40 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.030+0000 7f5f58ff9640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3c06a050 0x7f5f3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.030+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5f440a9f90 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.031+0000 7f5f5b7fe640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3c06a050 0x7f5f3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.032+0000 7f5f5b7fe640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3c06a050 0x7f5f3c06c510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c069270 tx=0x7f5f4c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.034+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5f440793c0 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.222+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5f5c066fb0 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.223+0000 7f5f58ff9640 1 -- 172.21.15.70:0/838557831 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5f44078f50 con 0x7f5f5c0c97a0 2023-08-25T09:43:51.228 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:43:51.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3c06a050 msgr2=0x7f5f3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3c06a050 0x7f5f3c06c510 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c069270 tx=0x7f5f4c008040 comp rx=0 tx=0).stop 2023-08-25T09:43:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 msgr2=0x7f5f5c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 0x7f5f5c0b2160 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5f4402bc00 tx=0x7f5f44001120 comp rx=0 tx=0).stop 2023-08-25T09:43:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 shutdown_connections 2023-08-25T09:43:51.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3c06a050 0x7f5f3c06c510 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f5c154d10 0x7f5f5c1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f5c0c97a0 0x7f5f5c0b2160 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 --2- 172.21.15.70:0/838557831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f480047c0 0x7f5f5c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 >> 172.21.15.70:0/838557831 conn(0x7f5f5c0cb0d0 msgr2=0x7f5f5c0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 shutdown_connections 2023-08-25T09:43:51.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.227+0000 7f5f61fc8640 1 -- 172.21.15.70:0/838557831 wait complete. 2023-08-25T09:43:51.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:43:51.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.447+0000 7f029af24640 1 Processor -- start 2023-08-25T09:43:51.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.448+0000 7f029af24640 1 -- start start 2023-08-25T09:43:51.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.449+0000 7f029af24640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940cced0 0x7f029415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.449+0000 7f029af24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02940b2a10 0x7f029415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.449+0000 7f029af24640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02940d03c0 con 0x7f02941614c0 2023-08-25T09:43:51.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.449+0000 7f029af24640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02940b3240 con 0x7f02940b2a10 2023-08-25T09:43:51.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.449+0000 7f029af24640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940cde20 con 0x7f02940cced0 2023-08-25T09:43:51.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f02994c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f02940b2a10 0x7f029415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f0298cc0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f02941614c0 0x7f02940b2300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49014/0 (socket says 172.21.15.70:49014) 2023-08-25T09:43:51.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f0298cc0640 1 -- 172.21.15.70:0/3530953351 learned_addr learned my addr 172.21.15.70:0/3530953351 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:51.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f028bfff640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940cced0 0x7f029415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f02994c1640 1 -- 172.21.15.70:0/3530953351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940cced0 msgr2=0x7f029415c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f02994c1640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940cced0 0x7f029415c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f02994c1640 1 -- 172.21.15.70:0/3530953351 >> v1:172.21.15.70:6789/0 conn(0x7f02941614c0 legacy=0x7f02940b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f02994c1640 1 -- 172.21.15.70:0/3530953351 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02940ce100 con 0x7f02940b2a10 2023-08-25T09:43:51.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f028b7fe640 1 -- 172.21.15.70:0/3530953351 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367138445 0 0) 0x7f02940d03c0 con 0x7f02941614c0 2023-08-25T09:43:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.450+0000 7f028bfff640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940cced0 0x7f029415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.451+0000 7f02994c1640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02940b2a10 0x7f029415ca30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f028c0097a0 tx=0x7f028c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5687a67c6da15de server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f028b7fe640 1 -- 172.21.15.70:0/3530953351 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c030800 con 0x7f02940b2a10 2023-08-25T09:43:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f028b7fe640 1 -- 172.21.15.70:0/3530953351 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c030960 con 0x7f02940b2a10 2023-08-25T09:43:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f028b7fe640 1 -- 172.21.15.70:0/3530953351 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c030cf0 con 0x7f02940b2a10 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 -- 172.21.15.70:0/3530953351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02940b2a10 msgr2=0x7f029415ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02940b2a10 0x7f029415ca30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f028c0097a0 tx=0x7f028c02b940 comp rx=0 tx=0).stop 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 -- 172.21.15.70:0/3530953351 shutdown_connections 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02940b2a10 0x7f029415ca30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 --2- 172.21.15.70:0/3530953351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940cced0 0x7f029415c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 -- 172.21.15.70:0/3530953351 >> 172.21.15.70:0/3530953351 conn(0x7f02940cd300 msgr2=0x7f02940cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 -- 172.21.15.70:0/3530953351 shutdown_connections 2023-08-25T09:43:51.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.452+0000 7f029af24640 1 -- 172.21.15.70:0/3530953351 wait complete. 2023-08-25T09:43:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.454+0000 7f029af24640 1 Processor -- start 2023-08-25T09:43:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.454+0000 7f029af24640 1 -- start start 2023-08-25T09:43:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.455+0000 7f029af24640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940b2a10 0x7f02940b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.455+0000 7f029af24640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02940cced0 0x7f02940c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.455+0000 7f029af24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 0x7f02940ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.455+0000 7f029af24640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940c6370 con 0x7f02941614c0 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.455+0000 7f029af24640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02940c6650 con 0x7f02940cced0 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.455+0000 7f029af24640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02940c6930 con 0x7f02940b2a10 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 0x7f02940ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 0x7f02940ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42024/0 (socket says 172.21.15.70:42024) 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 -- 172.21.15.70:0/2418772490 learned_addr learned my addr 172.21.15.70:0/2418772490 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 -- 172.21.15.70:0/2418772490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940b2a10 msgr2=0x7f02940b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940b2a10 0x7f02940b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 -- 172.21.15.70:0/2418772490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02940cced0 msgr2=0x7f02940c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02940cced0 0x7f02940c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 -- 172.21.15.70:0/2418772490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c009400 con 0x7f02941614c0 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.456+0000 7f02994c1640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 0x7f02940ad930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f028c0098c0 tx=0x7f028c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.457+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c03e070 con 0x7f02941614c0 2023-08-25T09:43:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.457+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c0397d0 con 0x7f02941614c0 2023-08-25T09:43:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.457+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c039b60 con 0x7f02941614c0 2023-08-25T09:43:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.457+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02940c6c10 con 0x7f02941614c0 2023-08-25T09:43:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.457+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02940acab0 con 0x7f02941614c0 2023-08-25T09:43:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.458+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02940c7130 con 0x7f02941614c0 2023-08-25T09:43:51.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.459+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f028c042070 con 0x7f02941614c0 2023-08-25T09:43:51.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.460+0000 7f02897fa640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f028006a050 0x7f028006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.460+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f028c0aa350 con 0x7f02941614c0 2023-08-25T09:43:51.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.460+0000 7f0298cc0640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f028006a050 0x7f028006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.461+0000 7f0298cc0640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f028006a050 0x7f028006c510 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0290001560 tx=0x7f0290007ca0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.464+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f02940c7130 con 0x7f02941614c0 2023-08-25T09:43:51.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.656+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f02940c7130 con 0x7f02941614c0 2023-08-25T09:43:51.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.657+0000 7f02897fa640 1 -- 172.21.15.70:0/2418772490 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) v1 ==== 84+0+126 (secure 0 0 0) 0x7f02940c7130 con 0x7f02941614c0 2023-08-25T09:43:51.658 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:51.659 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:43:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:43:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:43:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:43:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:43:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:43:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:43:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:43:51.660 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:43:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.659+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f028006a050 msgr2=0x7f028006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.659+0000 7f029af24640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f028006a050 0x7f028006c510 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0290001560 tx=0x7f0290007ca0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.659+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 msgr2=0x7f02940ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.659+0000 7f029af24640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 0x7f02940ad930 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f028c0098c0 tx=0x7f028c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 shutdown_connections 2023-08-25T09:43:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f028006a050 0x7f028006c510 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f02941614c0 0x7f02940ad930 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f02940cced0 0x7f02940c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 --2- 172.21.15.70:0/2418772490 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f02940b2a10 0x7f02940b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 >> 172.21.15.70:0/2418772490 conn(0x7f02940cd300 msgr2=0x7f0294071640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 shutdown_connections 2023-08-25T09:43:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.660+0000 7f029af24640 1 -- 172.21.15.70:0/2418772490 wait complete. 2023-08-25T09:43:51.680 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:43:51.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:43:51.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.893+0000 7f636f339640 1 Processor -- start 2023-08-25T09:43:51.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.893+0000 7f636f339640 1 -- start start 2023-08-25T09:43:51.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.895+0000 7f636f339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 0x7f63680cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.895+0000 7f636f339640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636815e6f0 con 0x7f63680ca9f0 2023-08-25T09:43:51.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.895+0000 7f636f339640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636815e9d0 con 0x7f63680cd890 2023-08-25T09:43:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.895+0000 7f636f339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680caf30 con 0x7f6368155120 2023-08-25T09:43:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.895+0000 7f636d8d6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63680ca9f0 0x7f636815af60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49016/0 (socket says 172.21.15.70:49016) 2023-08-25T09:43:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.895+0000 7f636d8d6640 1 -- 172.21.15.70:0/267035275 learned_addr learned my addr 172.21.15.70:0/267035275 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229139647 0 0) 0x7f636815e6f0 con 0x7f63680ca9f0 2023-08-25T09:43:51.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f636d0d5640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 0x7f63680cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6350003520 con 0x7f63680ca9f0 2023-08-25T09:43:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635908406 0 0) 0x7f6350003520 con 0x7f63680ca9f0 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 msgr2=0x7f63680cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 0x7f63680cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 >> v1:172.21.15.155:6789/0 conn(0x7f63680cd890 legacy=0x7f63680ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636815ecb0 con 0x7f63680ca9f0 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6360001670 con 0x7f63680ca9f0 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 >> v1:172.21.15.70:6789/0 conn(0x7f63680ca9f0 legacy=0x7f636815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 0x7f63500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6350003f20 0x7f63500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6350004d00 0x7f6350005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.896+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635000ba30 con 0x7f6350003f20 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635000bba0 con 0x7f6350004d00 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635000bd60 con 0x7f6368155120 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636c8d4640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6350003f20 0x7f63500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636d8d6640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6350004d00 0x7f6350005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636c8d4640 1 -- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 msgr2=0x7f63500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636c8d4640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6368155120 0x7f63500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636c8d4640 1 -- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6350004d00 msgr2=0x7f6350005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636c8d4640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6350004d00 0x7f6350005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.897+0000 7f636c8d4640 1 -- 172.21.15.70:0/267035275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350003520 con 0x7f6350003f20 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.898+0000 7f636c8d4640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6350003f20 0x7f63500047c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6364001e50 tx=0x7f63500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246204b22cbd1a12 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.898+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364003b70 con 0x7f6350003f20 2023-08-25T09:43:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6364003cd0 con 0x7f6350003f20 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f635ffff640 1 -- 172.21.15.70:0/267035275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364005310 con 0x7f6350003f20 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636f339640 1 -- 172.21.15.70:0/267035275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6350003f20 msgr2=0x7f63500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636f339640 1 --2- 172.21.15.70:0/267035275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6350003f20 0x7f63500047c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6364001e50 tx=0x7f63500057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636d8d6640 1 -- 172.21.15.70:0/267035275 reap_dead start 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636f339640 1 -- 172.21.15.70:0/267035275 shutdown_connections 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636f339640 1 -- 172.21.15.70:0/267035275 >> 172.21.15.70:0/267035275 conn(0x7f6368069c70 msgr2=0x7f636806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636f339640 1 -- 172.21.15.70:0/267035275 shutdown_connections 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.899+0000 7f636f339640 1 -- 172.21.15.70:0/267035275 wait complete. 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.901+0000 7f636f339640 1 Processor -- start 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.901+0000 7f636f339640 1 -- start start 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636f339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6350003f20 0x7f63681f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636f339640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6368149eb0 0x7f63681574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636f339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 0x7f6368152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636f339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368155cd0 con 0x7f636814a330 2023-08-25T09:43:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636f339640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368157ef0 con 0x7f6368149eb0 2023-08-25T09:43:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636f339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6368155610 con 0x7f6350003f20 2023-08-25T09:43:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636d8d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 0x7f6368152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636d8d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 0x7f6368152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42032/0 (socket says 172.21.15.70:42032) 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.902+0000 7f636d8d6640 1 -- 172.21.15.70:0/533024738 learned_addr learned my addr 172.21.15.70:0/533024738 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636c8d4640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6368149eb0 0x7f63681574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d0d5640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6350003f20 0x7f63681f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d8d6640 1 -- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6350003f20 msgr2=0x7f63681f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d8d6640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6350003f20 0x7f63681f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d8d6640 1 -- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6368149eb0 msgr2=0x7f63681574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d8d6640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6368149eb0 0x7f63681574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d8d6640 1 -- 172.21.15.70:0/533024738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63681558f0 con 0x7f636814a330 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d0d5640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6350003f20 0x7f63681f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f636d8d6640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 0x7f6368152c20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f636806bd50 tx=0x7f636000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.903+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000eed0 con 0x7f636814a330 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.904+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636000c690 con 0x7f636814a330 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.904+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000ca20 con 0x7f636814a330 2023-08-25T09:43:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.904+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63681f2d90 con 0x7f636814a330 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.904+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636806ffd0 con 0x7f636814a330 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.905+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6368155e10 con 0x7f636814a330 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.906+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f636000cb80 con 0x7f636814a330 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.906+0000 7f635dffb640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f635406a050 0x7f635406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.907+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6360087f00 con 0x7f636814a330 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.907+0000 7f636d0d5640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f635406a050 0x7f635406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.908+0000 7f636d0d5640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f635406a050 0x7f635406c510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6368152690 tx=0x7f6358008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:51.911+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6368155e10 con 0x7f636814a330 2023-08-25T09:43:52.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.100+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6368155e10 con 0x7f636814a330 2023-08-25T09:43:52.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.365+0000 7f635dffb640 1 -- 172.21.15.70:0/533024738 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v91) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6368155e10 con 0x7f636814a330 2023-08-25T09:43:52.367 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:43:52.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.371+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f635406a050 msgr2=0x7f635406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:52.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.371+0000 7f636f339640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f635406a050 0x7f635406c510 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6368152690 tx=0x7f6358008040 comp rx=0 tx=0).stop 2023-08-25T09:43:52.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.371+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 msgr2=0x7f6368152c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:52.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.371+0000 7f636f339640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 0x7f6368152c20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f636806bd50 tx=0x7f636000eb90 comp rx=0 tx=0).stop 2023-08-25T09:43:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.372+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 shutdown_connections 2023-08-25T09:43:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.372+0000 7f636f339640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f635406a050 0x7f635406c510 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.372+0000 7f636f339640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f636814a330 0x7f6368152c20 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.372+0000 7f636f339640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6368149eb0 0x7f63681574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.372+0000 7f636f339640 1 --2- 172.21.15.70:0/533024738 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6350003f20 0x7f63681f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.372+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 >> 172.21.15.70:0/533024738 conn(0x7f6368069c70 msgr2=0x7f63680b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:52.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.373+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 shutdown_connections 2023-08-25T09:43:52.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.373+0000 7f636f339640 1 -- 172.21.15.70:0/533024738 wait complete. 2023-08-25T09:43:52.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:43:52.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.614+0000 7fc2a8339640 1 Processor -- start 2023-08-25T09:43:52.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.615+0000 7fc2a8339640 1 -- start start 2023-08-25T09:43:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.616+0000 7fc2a8339640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2a0069f60 0x7fc2a00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:52.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.616+0000 7fc2a8339640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a00c2550 con 0x7fc2a015ae50 2023-08-25T09:43:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.616+0000 7fc2a8339640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a00ce330 con 0x7fc2a00c2110 2023-08-25T09:43:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.616+0000 7fc2a8339640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a00cc5f0 con 0x7fc2a0069f60 2023-08-25T09:43:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a68d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2a0069f60 0x7fc2a00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:52.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a60d5640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc2a015ae50 0x7fc2a00cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49024/0 (socket says 172.21.15.70:49024) 2023-08-25T09:43:52.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a68d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc2a0069f60 0x7fc2a00ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42042/0 (socket says 172.21.15.70:42042) 2023-08-25T09:43:52.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a60d5640 1 -- 172.21.15.70:0/3590454841 learned_addr learned my addr 172.21.15.70:0/3590454841 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a68d6640 1 -- 172.21.15.70:0/3590454841 >> v1:172.21.15.155:6789/0 conn(0x7fc2a00c2110 legacy=0x7fc2a0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:52.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a68d6640 1 -- 172.21.15.70:0/3590454841 >> v1:172.21.15.70:6789/0 conn(0x7fc2a015ae50 legacy=0x7fc2a00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.617+0000 7fc2a68d6640 1 -- 172.21.15.70:0/3590454841 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0070650 con 0x7fc2a0069f60 2023-08-25T09:43:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.618+0000 7fc2a68d6640 1 --2- 172.21.15.70:0/3590454841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0069f60 0x7fc2a00ccad0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc298009d90 tx=0x7fc29802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4025e1c86d7de435 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:52.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.618+0000 7fc2a50d3640 1 -- 172.21.15.70:0/3590454841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29802ccf0 con 0x7fc2a0069f60 2023-08-25T09:43:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a50d3640 1 -- 172.21.15.70:0/3590454841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29802ce50 con 0x7fc2a0069f60 2023-08-25T09:43:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a50d3640 1 -- 172.21.15.70:0/3590454841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298034660 con 0x7fc2a0069f60 2023-08-25T09:43:52.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a8339640 1 -- 172.21.15.70:0/3590454841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0069f60 msgr2=0x7fc2a00ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a8339640 1 --2- 172.21.15.70:0/3590454841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0069f60 0x7fc2a00ccad0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fc298009d90 tx=0x7fc29802bf40 comp rx=0 tx=0).stop 2023-08-25T09:43:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a8339640 1 -- 172.21.15.70:0/3590454841 shutdown_connections 2023-08-25T09:43:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a8339640 1 --2- 172.21.15.70:0/3590454841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a0069f60 0x7fc2a00ccad0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a8339640 1 -- 172.21.15.70:0/3590454841 >> 172.21.15.70:0/3590454841 conn(0x7fc2a00cb0d0 msgr2=0x7fc2a00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.619+0000 7fc2a8339640 1 -- 172.21.15.70:0/3590454841 shutdown_connections 2023-08-25T09:43:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.620+0000 7fc2a8339640 1 -- 172.21.15.70:0/3590454841 wait complete. 2023-08-25T09:43:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.622+0000 7fc2a8339640 1 Processor -- start 2023-08-25T09:43:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.622+0000 7fc2a8339640 1 -- start start 2023-08-25T09:43:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a8339640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2a0069f60 0x7fc2a0147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a8339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 0x7fc2a00c19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a8339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2a015ae50 0x7fc2a00ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a8339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0145350 con 0x7fc2a00c2110 2023-08-25T09:43:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a8339640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0144740 con 0x7fc2a0069f60 2023-08-25T09:43:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a8339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0144a20 con 0x7fc2a015ae50 2023-08-25T09:43:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a58d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 0x7fc2a00c19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a58d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 0x7fc2a00c19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42056/0 (socket says 172.21.15.70:42056) 2023-08-25T09:43:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.623+0000 7fc2a58d4640 1 -- 172.21.15.70:0/964888366 learned_addr learned my addr 172.21.15.70:0/964888366 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a68d6640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2a015ae50 0x7fc2a00ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a60d5640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2a0069f60 0x7fc2a0147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a58d4640 1 -- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2a015ae50 msgr2=0x7fc2a00ac1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a58d4640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2a015ae50 0x7fc2a00ac1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a58d4640 1 -- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2a0069f60 msgr2=0x7fc2a0147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a58d4640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2a0069f60 0x7fc2a0147b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a58d4640 1 -- 172.21.15.70:0/964888366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc298009a70 con 0x7fc2a00c2110 2023-08-25T09:43:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a58d4640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 0x7fc2a00c19e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc29000c430 tx=0x7fc29000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290011790 con 0x7fc2a00c2110 2023-08-25T09:43:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2900118f0 con 0x7fc2a00c2110 2023-08-25T09:43:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a0144d00 con 0x7fc2a00c2110 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290011ce0 con 0x7fc2a00c2110 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.624+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a014a860 con 0x7fc2a00c2110 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.625+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a0066fb0 con 0x7fc2a00c2110 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.627+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc29001c460 con 0x7fc2a00c2110 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.628+0000 7fc296ffd640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc28406a050 0x7fc28406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.628+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fc290088e50 con 0x7fc2a00c2110 2023-08-25T09:43:52.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.628+0000 7fc2a60d5640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc28406a050 0x7fc28406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.629+0000 7fc2a60d5640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc28406a050 0x7fc28406c510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005880 tx=0x7fc29c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.632+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc290054700 con 0x7fc2a00c2110 2023-08-25T09:43:52.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:52.820+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc2a006ebe0 con 0x7fc2a00c2110 2023-08-25T09:43:53.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.381+0000 7fc296ffd640 1 -- 172.21.15.70:0/964888366 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v92) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc290057d20 con 0x7fc2a00c2110 2023-08-25T09:43:53.384 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:43:53.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.393+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc28406a050 msgr2=0x7fc28406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.393+0000 7fc2a8339640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc28406a050 0x7fc28406c510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005880 tx=0x7fc29c009040 comp rx=0 tx=0).stop 2023-08-25T09:43:53.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.393+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 msgr2=0x7fc2a00c19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.393+0000 7fc2a8339640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 0x7fc2a00c19e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fc29000c430 tx=0x7fc29000c8f0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.394+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 shutdown_connections 2023-08-25T09:43:53.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.394+0000 7fc2a8339640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc28406a050 0x7fc28406c510 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.394+0000 7fc2a8339640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc2a015ae50 0x7fc2a00ac1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.394+0000 7fc2a8339640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc2a00c2110 0x7fc2a00c19e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.394+0000 7fc2a8339640 1 --2- 172.21.15.70:0/964888366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc2a0069f60 0x7fc2a0147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.394+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 >> 172.21.15.70:0/964888366 conn(0x7fc2a00cb0d0 msgr2=0x7fc2a00c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:53.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.398+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 shutdown_connections 2023-08-25T09:43:53.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.398+0000 7fc2a8339640 1 -- 172.21.15.70:0/964888366 wait complete. 2023-08-25T09:43:53.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:43:53.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.618+0000 7f5a23c3f640 1 Processor -- start 2023-08-25T09:43:53.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.619+0000 7f5a23c3f640 1 -- start start 2023-08-25T09:43:53.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.620+0000 7f5a23c3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 0x7f5a1c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.620+0000 7f5a23c3f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c15e780 con 0x7f5a1c06f0e0 2023-08-25T09:43:53.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.620+0000 7f5a23c3f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c06f8e0 con 0x7f5a1c0cd660 2023-08-25T09:43:53.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.620+0000 7f5a23c3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c06fbc0 con 0x7f5a1c159bf0 2023-08-25T09:43:53.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.621+0000 7f5a211da640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5a1c06f0e0 0x7f5a1c070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49040/0 (socket says 172.21.15.70:49040) 2023-08-25T09:43:53.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.621+0000 7f5a211da640 1 -- 172.21.15.70:0/1488816255 learned_addr learned my addr 172.21.15.70:0/1488816255 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:53.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.621+0000 7f5a219db640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 0x7f5a1c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.621+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314770022 0 0) 0x7f5a1c15e780 con 0x7f5a1c06f0e0 2023-08-25T09:43:53.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.621+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a08005730 con 0x7f5a1c06f0e0 2023-08-25T09:43:53.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2549717067 0 0) 0x7f5a08005730 con 0x7f5a1c06f0e0 2023-08-25T09:43:53.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 msgr2=0x7f5a1c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 0x7f5a1c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 >> v1:172.21.15.155:6789/0 conn(0x7f5a1c0cd660 legacy=0x7f5a1c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c06fea0 con 0x7f5a1c06f0e0 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5a0c001670 con 0x7f5a1c06f0e0 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 >> v1:172.21.15.70:6789/0 conn(0x7f5a1c06f0e0 legacy=0x7f5a1c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a08005bb0 0x7f5a08005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a080069d0 0x7f5a08006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 0x7f5a08007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c06fbc0 con 0x7f5a080069d0 2023-08-25T09:43:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0800dc40 con 0x7f5a08005bb0 2023-08-25T09:43:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0800de00 con 0x7f5a1c159bf0 2023-08-25T09:43:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a211da640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a080069d0 0x7f5a08006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a219db640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a08005bb0 0x7f5a08005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a211da640 1 -- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 msgr2=0x7f5a08007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:53.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.622+0000 7f5a211da640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a1c159bf0 0x7f5a08007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.623+0000 7f5a211da640 1 -- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a08005bb0 msgr2=0x7f5a08005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:53.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.623+0000 7f5a211da640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a08005bb0 0x7f5a08005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.623+0000 7f5a211da640 1 -- 172.21.15.70:0/1488816255 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08005730 con 0x7f5a080069d0 2023-08-25T09:43:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a211da640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a080069d0 0x7f5a08006e50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c210 tx=0x7f5a08007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e06c891e6cd81d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:53.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c005ad0 con 0x7f5a080069d0 2023-08-25T09:43:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c005c30 con 0x7f5a080069d0 2023-08-25T09:43:53.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a209d9640 1 -- 172.21.15.70:0/1488816255 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c02c870 con 0x7f5a080069d0 2023-08-25T09:43:53.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1488816255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a080069d0 msgr2=0x7f5a08006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1488816255 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a080069d0 0x7f5a08006e50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00c210 tx=0x7f5a08007a00 comp rx=0 tx=0).stop 2023-08-25T09:43:53.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.624+0000 7f5a221dc640 1 -- 172.21.15.70:0/1488816255 reap_dead start 2023-08-25T09:43:53.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.625+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1488816255 shutdown_connections 2023-08-25T09:43:53.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.625+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1488816255 >> 172.21.15.70:0/1488816255 conn(0x7f5a1c0cb0d0 msgr2=0x7f5a1c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:53.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.625+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1488816255 shutdown_connections 2023-08-25T09:43:53.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.625+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1488816255 wait complete. 2023-08-25T09:43:53.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.626+0000 7f5a23c3f640 1 Processor -- start 2023-08-25T09:43:53.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.627+0000 7f5a23c3f640 1 -- start start 2023-08-25T09:43:53.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.627+0000 7f5a23c3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a080069d0 0x7f5a1c15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a23c3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c0af320 0x7f5a1c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a23c3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 0x7f5a1c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a23c3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c15a690 con 0x7f5a1c15aab0 2023-08-25T09:43:53.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a23c3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0c8580 con 0x7f5a1c0af320 2023-08-25T09:43:53.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a23c3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c15ba00 con 0x7f5a080069d0 2023-08-25T09:43:53.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 0x7f5a1c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 0x7f5a1c157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42070/0 (socket says 172.21.15.70:42070) 2023-08-25T09:43:53.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 -- 172.21.15.70:0/1105622980 learned_addr learned my addr 172.21.15.70:0/1105622980 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:53.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a219db640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a080069d0 0x7f5a1c15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 -- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a080069d0 msgr2=0x7f5a1c15b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a080069d0 0x7f5a1c15b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a211da640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c0af320 0x7f5a1c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 -- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c0af320 msgr2=0x7f5a1c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:53.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c0af320 0x7f5a1c156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:53.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a221dc640 1 -- 172.21.15.70:0/1105622980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c15bce0 con 0x7f5a1c15aab0 2023-08-25T09:43:53.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.628+0000 7f5a219db640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a080069d0 0x7f5a1c15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:53.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.629+0000 7f5a221dc640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 0x7f5a1c157840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0c8d90 tx=0x7f5a1400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:53.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.629+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14005d40 con 0x7f5a1c15aab0 2023-08-25T09:43:53.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.629+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1400ced0 con 0x7f5a1c15aab0 2023-08-25T09:43:53.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.629+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400a850 con 0x7f5a1c15aab0 2023-08-25T09:43:53.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.629+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c159fd0 con 0x7f5a1c15aab0 2023-08-25T09:43:53.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.629+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c157170 con 0x7f5a1c15aab0 2023-08-25T09:43:53.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.630+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c15a7d0 con 0x7f5a1c15aab0 2023-08-25T09:43:53.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.631+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5a1400a9b0 con 0x7f5a1c15aab0 2023-08-25T09:43:53.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.633+0000 7f5a127fc640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0006a050 0x7f5a0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:53.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.633+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f5a14087710 con 0x7f5a1c15aab0 2023-08-25T09:43:53.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.634+0000 7f5a219db640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0006a050 0x7f5a0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:53.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.634+0000 7f5a219db640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0006a050 0x7f5a0006c510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009a20 tx=0x7f5a18008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:53.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.639+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5a1c15a7d0 con 0x7f5a1c15aab0 2023-08-25T09:43:53.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:53.867+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5a1c15a7d0 con 0x7f5a1c15aab0 2023-08-25T09:43:54.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.388+0000 7f5a127fc640 1 -- 172.21.15.70:0/1105622980 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v37) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5a1c15a7d0 con 0x7f5a1c15aab0 2023-08-25T09:43:54.390 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 11 and data pool 12 2023-08-25T09:43:54.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.394+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0006a050 msgr2=0x7f5a0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.394+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0006a050 0x7f5a0006c510 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009a20 tx=0x7f5a18008040 comp rx=0 tx=0).stop 2023-08-25T09:43:54.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.394+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 msgr2=0x7f5a1c157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.394+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 0x7f5a1c157840 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c0c8d90 tx=0x7f5a1400cb90 comp rx=0 tx=0).stop 2023-08-25T09:43:54.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.398+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 shutdown_connections 2023-08-25T09:43:54.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.398+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5a0006a050 0x7f5a0006c510 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.398+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5a1c15aab0 0x7f5a1c157840 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.398+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5a1c0af320 0x7f5a1c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.398+0000 7f5a23c3f640 1 --2- 172.21.15.70:0/1105622980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5a080069d0 0x7f5a1c15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.398+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 >> 172.21.15.70:0/1105622980 conn(0x7f5a1c0cb0d0 msgr2=0x7f5a1c0c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:54.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.402+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 shutdown_connections 2023-08-25T09:43:54.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.403+0000 7f5a23c3f640 1 -- 172.21.15.70:0/1105622980 wait complete. 2023-08-25T09:43:54.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:54.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.616+0000 7f8682c46640 1 Processor -- start 2023-08-25T09:43:54.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.617+0000 7f8682c46640 1 -- start start 2023-08-25T09:43:54.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.618+0000 7f8682c46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f867c15cb40 0x7f867c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.618+0000 7f8682c46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f867c0ac870 0x7f867c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.618+0000 7f8682c46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 0x7f867c160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.618+0000 7f8682c46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c06f050 con 0x7f867c0ac870 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.618+0000 7f8682c46640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c06f330 con 0x7f867c15cb40 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.618+0000 7f8682c46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c06f610 con 0x7f867c0b3880 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f867c0ac870 0x7f867c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:54.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f867c0ac870 0x7f867c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42074/0 (socket says 172.21.15.70:42074) 2023-08-25T09:43:54.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 -- 172.21.15.70:0/211482980 learned_addr learned my addr 172.21.15.70:0/211482980 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:54.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 -- 172.21.15.70:0/211482980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 msgr2=0x7f867c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:54.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 --2- 172.21.15.70:0/211482980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 0x7f867c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 -- 172.21.15.70:0/211482980 >> v2:172.21.15.155:3300/0 conn(0x7f867c15cb40 msgr2=0x7f867c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:54.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 --2- 172.21.15.70:0/211482980 >> v2:172.21.15.155:3300/0 conn(0x7f867c15cb40 0x7f867c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.619+0000 7f867bfff640 1 -- 172.21.15.70:0/211482980 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c06f8f0 con 0x7f867c0ac870 2023-08-25T09:43:54.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.620+0000 7f867bfff640 1 --2- 172.21.15.70:0/211482980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c0ac870 0x7f867c0ce860 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f867c06a7b0 tx=0x7f866c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c760b4da1e88d65 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:54.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.620+0000 7f867b7fe640 1 -- 172.21.15.70:0/211482980 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c031870 con 0x7f867c0ac870 2023-08-25T09:43:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f867b7fe640 1 -- 172.21.15.70:0/211482980 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c0319d0 con 0x7f867c0ac870 2023-08-25T09:43:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f867b7fe640 1 -- 172.21.15.70:0/211482980 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c031d60 con 0x7f867c0ac870 2023-08-25T09:43:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 -- 172.21.15.70:0/211482980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c0ac870 msgr2=0x7f867c0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 --2- 172.21.15.70:0/211482980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c0ac870 0x7f867c0ce860 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f867c06a7b0 tx=0x7f866c02b910 comp rx=0 tx=0).stop 2023-08-25T09:43:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 -- 172.21.15.70:0/211482980 shutdown_connections 2023-08-25T09:43:54.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 --2- 172.21.15.70:0/211482980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 0x7f867c160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 --2- 172.21.15.70:0/211482980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c0ac870 0x7f867c0ce860 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 --2- 172.21.15.70:0/211482980 >> v2:172.21.15.155:3300/0 conn(0x7f867c15cb40 0x7f867c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 -- 172.21.15.70:0/211482980 >> 172.21.15.70:0/211482980 conn(0x7f867c069850 msgr2=0x7f867c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:54.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 -- 172.21.15.70:0/211482980 shutdown_connections 2023-08-25T09:43:54.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.621+0000 7f8682c46640 1 -- 172.21.15.70:0/211482980 wait complete. 2023-08-25T09:43:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.623+0000 7f8682c46640 1 Processor -- start 2023-08-25T09:43:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.623+0000 7f8682c46640 1 -- start start 2023-08-25T09:43:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f8682c46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f867c0ac870 0x7f867c14dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f8682c46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 0x7f867c0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f8682c46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 0x7f867c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f8682c46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c151f40 con 0x7f867c15cb40 2023-08-25T09:43:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f8682c46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15d4e0 con 0x7f867c0ac870 2023-08-25T09:43:54.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f8682c46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c1f2240 con 0x7f867c0b3880 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f86811e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 0x7f867c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f86811e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 0x7f867c0afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42086/0 (socket says 172.21.15.70:42086) 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.624+0000 7f86811e3640 1 -- 172.21.15.70:0/995657521 learned_addr learned my addr 172.21.15.70:0/995657521 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86811e3640 1 -- 172.21.15.70:0/995657521 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 msgr2=0x7f867c0af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86811e3640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 0x7f867c0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86811e3640 1 -- 172.21.15.70:0/995657521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f867c0ac870 msgr2=0x7f867c14dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86811e3640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f867c0ac870 0x7f867c14dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86811e3640 1 -- 172.21.15.70:0/995657521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0094d0 con 0x7f867c15cb40 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86811e3640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 0x7f867c0afb90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f867400b970 tx=0x7f867400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867400cb40 con 0x7f867c15cb40 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867400ccc0 con 0x7f867c15cb40 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.625+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674014680 con 0x7f867c15cb40 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.626+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c152150 con 0x7f867c15cb40 2023-08-25T09:43:54.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.626+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c152710 con 0x7f867c15cb40 2023-08-25T09:43:54.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.627+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c04e420 con 0x7f867c15cb40 2023-08-25T09:43:54.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.628+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f867401d3b0 con 0x7f867c15cb40 2023-08-25T09:43:54.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.629+0000 7f86797fa640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f865c06a050 0x7f865c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:54.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.629+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86740885c0 con 0x7f867c15cb40 2023-08-25T09:43:54.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.629+0000 7f86809e2640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f865c06a050 0x7f865c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:54.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.630+0000 7f86809e2640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f865c06a050 0x7f865c06c510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8664009670 tx=0x7f8664006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:54.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.633+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8674053c10 con 0x7f867c15cb40 2023-08-25T09:43:54.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.823+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f867c070c40 con 0x7f867c15cb40 2023-08-25T09:43:54.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.825+0000 7f86797fa640 1 -- 172.21.15.70:0/995657521 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+18321 (secure 0 0 0) 0x7f8674057230 con 0x7f867c15cb40 2023-08-25T09:43:54.826 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:54.827 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":93,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:53.868354+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:54.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.827+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f865c06a050 msgr2=0x7f865c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.827+0000 7f8682c46640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f865c06a050 0x7f865c06c510 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8664009670 tx=0x7f8664006bf0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.827+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 msgr2=0x7f867c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:54.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.827+0000 7f8682c46640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 0x7f867c0afb90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f867400b970 tx=0x7f867400be30 comp rx=0 tx=0).stop 2023-08-25T09:43:54.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 shutdown_connections 2023-08-25T09:43:54.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f865c06a050 0x7f865c06c510 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f867c15cb40 0x7f867c0afb90 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f867c0b3880 0x7f867c0af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 --2- 172.21.15.70:0/995657521 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f867c0ac870 0x7f867c14dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:54.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 >> 172.21.15.70:0/995657521 conn(0x7f867c069850 msgr2=0x7f867c0b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:54.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 shutdown_connections 2023-08-25T09:43:54.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:54.828+0000 7f8682c46640 1 -- 172.21.15.70:0/995657521 wait complete. 2023-08-25T09:43:54.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:54.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-08-25T09:43:54.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-08-25T09:43:54.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-08-25T09:43:55.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.062+0000 7fdc1faf3640 1 Processor -- start 2023-08-25T09:43:55.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.062+0000 7fdc1faf3640 1 -- start start 2023-08-25T09:43:55.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1faf3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdc180ce5b0 0x7fdc180ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1faf3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 0x7fdc1806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1faf3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc180ceed0 con 0x7fdc1815aa10 2023-08-25T09:43:55.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1faf3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc18159d10 con 0x7fdc180ce5b0 2023-08-25T09:43:55.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1faf3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815d3f0 con 0x7fdc180b2850 2023-08-25T09:43:55.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1d08e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdc1815aa10 0x7fdc180b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49046/0 (socket says 172.21.15.70:49046) 2023-08-25T09:43:55.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.064+0000 7fdc1d08e640 1 -- 172.21.15.70:0/3181395314 learned_addr learned my addr 172.21.15.70:0/3181395314 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:55.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2095891285 0 0) 0x7fdc180ceed0 con 0x7fdc1815aa10 2023-08-25T09:43:55.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc04005730 con 0x7fdc1815aa10 2023-08-25T09:43:55.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1e090640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 0x7fdc1806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1d88f640 1 --2- 172.21.15.70:0/3181395314 >> v2:172.21.15.155:3300/0 conn(0x7fdc180ce5b0 0x7fdc180ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 772488015 0 0) 0x7fdc04005730 con 0x7fdc1815aa10 2023-08-25T09:43:55.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 msgr2=0x7fdc1806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 0x7fdc1806f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 >> v2:172.21.15.155:3300/0 conn(0x7fdc180ce5b0 msgr2=0x7fdc180ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 --2- 172.21.15.70:0/3181395314 >> v2:172.21.15.155:3300/0 conn(0x7fdc180ce5b0 0x7fdc180ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180b2d20 con 0x7fdc1815aa10 2023-08-25T09:43:55.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fdc08001690 con 0x7fdc1815aa10 2023-08-25T09:43:55.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 >> v1:172.21.15.70:6789/0 conn(0x7fdc1815aa10 legacy=0x7fdc180b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 0x7fdc04005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.065+0000 7fdc1c88d640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc040061b0 0x7fdc040069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1c88d640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc04006f10 0x7fdc04007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc18159d10 con 0x7fdc040061b0 2023-08-25T09:43:55.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1815d3f0 con 0x7fdc04006f10 2023-08-25T09:43:55.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0400dd50 con 0x7fdc180b2850 2023-08-25T09:43:55.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d08e640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc040061b0 0x7fdc040069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d88f640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 0x7fdc04005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d08e640 1 -- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 msgr2=0x7fdc04005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d08e640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc180b2850 0x7fdc04005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d08e640 1 -- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc04006f10 msgr2=0x7fdc04007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d08e640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc04006f10 0x7fdc04007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.066+0000 7fdc1d08e640 1 -- 172.21.15.70:0/3181395314 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc04005730 con 0x7fdc040061b0 2023-08-25T09:43:55.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.067+0000 7fdc1d08e640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc040061b0 0x7fdc040069d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdc08007fd0 tx=0x7fdc04007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c380adc15c3c0e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:55.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08005ad0 con 0x7fdc040061b0 2023-08-25T09:43:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08005c50 con 0x7fdc040061b0 2023-08-25T09:43:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1c88d640 1 -- 172.21.15.70:0/3181395314 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0802c8d0 con 0x7fdc040061b0 2023-08-25T09:43:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1faf3640 1 -- 172.21.15.70:0/3181395314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc040061b0 msgr2=0x7fdc040069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/3181395314 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc040061b0 0x7fdc040069d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdc08007fd0 tx=0x7fdc04007a00 comp rx=0 tx=0).stop 2023-08-25T09:43:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1e090640 1 -- 172.21.15.70:0/3181395314 reap_dead start 2023-08-25T09:43:55.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1faf3640 1 -- 172.21.15.70:0/3181395314 shutdown_connections 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.068+0000 7fdc1faf3640 1 -- 172.21.15.70:0/3181395314 >> 172.21.15.70:0/3181395314 conn(0x7fdc18069850 msgr2=0x7fdc18069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.069+0000 7fdc1faf3640 1 -- 172.21.15.70:0/3181395314 shutdown_connections 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.069+0000 7fdc1faf3640 1 -- 172.21.15.70:0/3181395314 wait complete. 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.071+0000 7fdc1faf3640 1 Processor -- start 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.071+0000 7fdc1faf3640 1 -- start start 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1faf3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc040061b0 0x7fdc180aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1faf3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc18148fe0 0x7fdc180ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1faf3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 0x7fdc180acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1faf3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180cd120 con 0x7fdc18149460 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1faf3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180aee80 con 0x7fdc18148fe0 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1faf3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc180af160 con 0x7fdc040061b0 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1e090640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 0x7fdc180acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1e090640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 0x7fdc180acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42096/0 (socket says 172.21.15.70:42096) 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.072+0000 7fdc1e090640 1 -- 172.21.15.70:0/1618189627 learned_addr learned my addr 172.21.15.70:0/1618189627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:55.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1e090640 1 -- 172.21.15.70:0/1618189627 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc040061b0 msgr2=0x7fdc180aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1e090640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc040061b0 0x7fdc180aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1e090640 1 -- 172.21.15.70:0/1618189627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc18148fe0 msgr2=0x7fdc180ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1e090640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc18148fe0 0x7fdc180ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1e090640 1 -- 172.21.15.70:0/1618189627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc180af440 con 0x7fdc18149460 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1e090640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 0x7fdc180acdb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdc180cae20 tx=0x7fdc1000d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10004030 con 0x7fdc18149460 2023-08-25T09:43:55.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc100041b0 con 0x7fdc18149460 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10003610 con 0x7fdc18149460 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.073+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc180af720 con 0x7fdc18149460 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.074+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc180ae2a0 con 0x7fdc18149460 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.075+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc180afc40 con 0x7fdc18149460 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.076+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdc10003770 con 0x7fdc18149460 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.076+0000 7fdc0e7fc640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdbfc06a050 0x7fdbfc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.077+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc10087ce0 con 0x7fdc18149460 2023-08-25T09:43:55.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.077+0000 7fdc1d88f640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdbfc06a050 0x7fdbfc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.077+0000 7fdc1d88f640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdbfc06a050 0x7fdbfc06c510 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc140094c0 tx=0x7fdc1400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:55.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.081+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdc180afc40 con 0x7fdc18149460 2023-08-25T09:43:55.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.269+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdc180afc40 con 0x7fdc18149460 2023-08-25T09:43:55.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.272+0000 7fdc0e7fc640 1 -- 172.21.15.70:0/1618189627 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+18321 (secure 0 0 0) 0x7fdc180afc40 con 0x7fdc18149460 2023-08-25T09:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:55.274 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":93,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:53.868354+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:55.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdbfc06a050 msgr2=0x7fdbfc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdbfc06a050 0x7fdbfc06c510 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc140094c0 tx=0x7fdc1400b040 comp rx=0 tx=0).stop 2023-08-25T09:43:55.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 msgr2=0x7fdc180acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 0x7fdc180acdb0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fdc180cae20 tx=0x7fdc1000d820 comp rx=0 tx=0).stop 2023-08-25T09:43:55.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 shutdown_connections 2023-08-25T09:43:55.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdbfc06a050 0x7fdbfc06c510 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc18149460 0x7fdc180acdb0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc18148fe0 0x7fdc180ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 --2- 172.21.15.70:0/1618189627 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc040061b0 0x7fdc180aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 >> 172.21.15.70:0/1618189627 conn(0x7fdc18069850 msgr2=0x7fdc181591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:55.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 shutdown_connections 2023-08-25T09:43:55.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.275+0000 7fdc1faf3640 1 -- 172.21.15.70:0/1618189627 wait complete. 2023-08-25T09:43:55.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:43:55.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.515+0000 7f57fed5d640 1 Processor -- start 2023-08-25T09:43:55.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.516+0000 7f57fed5d640 1 -- start start 2023-08-25T09:43:55.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.517+0000 7f57fed5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 0x7f57f80b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.517+0000 7f57fed5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80cb710 con 0x7f57f80ce110 2023-08-25T09:43:55.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57fed5d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80b1280 con 0x7f57f806ae60 2023-08-25T09:43:55.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57fed5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80cec80 con 0x7f57f80b0960 2023-08-25T09:43:55.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57f7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f57f80ce110 0x7f57f80ce570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:49060/0 (socket says 172.21.15.70:49060) 2023-08-25T09:43:55.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57f7fff640 1 -- 172.21.15.70:0/2133897809 learned_addr learned my addr 172.21.15.70:0/2133897809 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:55.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 0x7f57f80b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107959140 0 0) 0x7f57f80cb710 con 0x7f57f80ce110 2023-08-25T09:43:55.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.518+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e4003520 con 0x7f57f80ce110 2023-08-25T09:43:55.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2043423124 0 0) 0x7f57e4003520 con 0x7f57f80ce110 2023-08-25T09:43:55.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 msgr2=0x7f57f80b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 0x7f57f80b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 >> v1:172.21.15.155:6789/0 conn(0x7f57f806ae60 legacy=0x7f57f806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80ccb50 con 0x7f57f80ce110 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f57e00019f0 con 0x7f57f80ce110 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 >> v1:172.21.15.70:6789/0 conn(0x7f57f80ce110 legacy=0x7f57f80ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57e40039a0 0x7f57e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40047c0 0x7f57e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 0x7f57e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80cec80 con 0x7f57e40039a0 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e40054e0 con 0x7f57e40047c0 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e400bec0 con 0x7f57f80b0960 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57e40039a0 0x7f57e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57fcaf9640 1 -- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 msgr2=0x7f57e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b0960 0x7f57e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57fcaf9640 1 -- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40047c0 msgr2=0x7f57e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:55.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40047c0 0x7f57e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.519+0000 7f57fcaf9640 1 -- 172.21.15.70:0/2133897809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e4003520 con 0x7f57e40039a0 2023-08-25T09:43:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57e40039a0 0x7f57e4003dc0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ba30 tx=0x7f57e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b188b2f24524a26 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8032680 con 0x7f57e40039a0 2023-08-25T09:43:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e80327e0 con 0x7f57e40039a0 2023-08-25T09:43:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57f77fe640 1 -- 172.21.15.70:0/2133897809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8032b70 con 0x7f57e40039a0 2023-08-25T09:43:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57fed5d640 1 -- 172.21.15.70:0/2133897809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57e40039a0 msgr2=0x7f57e4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57fed5d640 1 --2- 172.21.15.70:0/2133897809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57e40039a0 0x7f57e4003dc0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ba30 tx=0x7f57e40057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57fd2fa640 1 -- 172.21.15.70:0/2133897809 reap_dead start 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57fed5d640 1 -- 172.21.15.70:0/2133897809 shutdown_connections 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.521+0000 7f57fed5d640 1 -- 172.21.15.70:0/2133897809 >> 172.21.15.70:0/2133897809 conn(0x7f57f80cb960 msgr2=0x7f57f80cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.522+0000 7f57fed5d640 1 -- 172.21.15.70:0/2133897809 shutdown_connections 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.522+0000 7f57fed5d640 1 -- 172.21.15.70:0/2133897809 wait complete. 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.523+0000 7f57fed5d640 1 Processor -- start 2023-08-25T09:43:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.523+0000 7f57fed5d640 1 -- start start 2023-08-25T09:43:55.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.524+0000 7f57fed5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40039a0 0x7f57f8147670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.524+0000 7f57fed5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 0x7f57f80b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.524+0000 7f57fed5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b2000 0x7f57f80b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.524+0000 7f57fed5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b2480 con 0x7f57f814a910 2023-08-25T09:43:55.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.524+0000 7f57fed5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8147bb0 con 0x7f57e40039a0 2023-08-25T09:43:55.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.524+0000 7f57fed5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c7e20 con 0x7f57f80b2000 2023-08-25T09:43:55.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 0x7f57f80b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 0x7f57f80b1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42116/0 (socket says 172.21.15.70:42116) 2023-08-25T09:43:55.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 -- 172.21.15.70:0/1631375857 learned_addr learned my addr 172.21.15.70:0/1631375857 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:55.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 -- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b2000 msgr2=0x7f57f80b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b2000 0x7f57f80b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40039a0 0x7f57f8147670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 -- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40039a0 msgr2=0x7f57f8147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40039a0 0x7f57f8147670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.525+0000 7f57f7fff640 1 -- 172.21.15.70:0/1631375857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80c8100 con 0x7f57f814a910 2023-08-25T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.526+0000 7f57f7fff640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 0x7f57f80b1ac0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f57e0002230 tx=0x7f57e000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.526+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000a7c0 con 0x7f57f814a910 2023-08-25T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.526+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e000a920 con 0x7f57f814a910 2023-08-25T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.526+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000ad30 con 0x7f57f814a910 2023-08-25T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.526+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f80c83e0 con 0x7f57f814a910 2023-08-25T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.526+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f80c89a0 con 0x7f57f814a910 2023-08-25T09:43:55.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.528+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f8066fb0 con 0x7f57f814a910 2023-08-25T09:43:55.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.529+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f57e00070d0 con 0x7f57f814a910 2023-08-25T09:43:55.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.530+0000 7f57f57fa640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57d806a050 0x7f57d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:55.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.530+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57e0087d90 con 0x7f57f814a910 2023-08-25T09:43:55.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.530+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57d806a050 0x7f57d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:55.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.531+0000 7f57fcaf9640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57d806a050 0x7f57d806c510 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ba00 tx=0x7f57e8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:55.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.534+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f57e0053490 con 0x7f57f814a910 2023-08-25T09:43:55.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:55.764+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57f80cc200 con 0x7f57f814a910 2023-08-25T09:43:56.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.428+0000 7f57f57fa640 1 -- 172.21.15.70:0/1631375857 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f57e0018070 con 0x7f57f814a910 2023-08-25T09:43:56.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57d806a050 msgr2=0x7f57d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:56.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57d806a050 0x7f57d806c510 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f57e800ba00 tx=0x7f57e8030000 comp rx=0 tx=0).stop 2023-08-25T09:43:56.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 msgr2=0x7f57f80b1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:56.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 0x7f57f80b1ac0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f57e0002230 tx=0x7f57e000cf00 comp rx=0 tx=0).stop 2023-08-25T09:43:56.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 shutdown_connections 2023-08-25T09:43:56.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57d806a050 0x7f57d806c510 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57f80b2000 0x7f57f80b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57f814a910 0x7f57f80b1ac0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 --2- 172.21.15.70:0/1631375857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57e40039a0 0x7f57f8147670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.431+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 >> 172.21.15.70:0/1631375857 conn(0x7f57f80cb960 msgr2=0x7f57f80ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:56.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.432+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 shutdown_connections 2023-08-25T09:43:56.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.432+0000 7f57fed5d640 1 -- 172.21.15.70:0/1631375857 wait complete. 2023-08-25T09:43:56.452 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:43:56.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:43:56.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.672+0000 7f4a6a1f3640 1 Processor -- start 2023-08-25T09:43:56.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.672+0000 7f4a6a1f3640 1 -- start start 2023-08-25T09:43:56.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a6a1f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a64159bf0 0x7f4a6415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a6a1f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4a6406f0e0 0x7f4a64070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a6a1f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4a640cd660 0x7f4a640cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a6a1f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a6415e780 con 0x7f4a6406f0e0 2023-08-25T09:43:56.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a6a1f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a640ce000 con 0x7f4a640cd660 2023-08-25T09:43:56.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a6a1f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6406f8e0 con 0x7f4a64159bf0 2023-08-25T09:43:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a5bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4a6406f0e0 0x7f4a64070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a5bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4a6406f0e0 0x7f4a64070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42124/0 (socket says 172.21.15.70:42124) 2023-08-25T09:43:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.674+0000 7f4a5bfff640 1 -- 172.21.15.70:0/3336076411 learned_addr learned my addr 172.21.15.70:0/3336076411 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:56.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a637fe640 1 --2- 172.21.15.70:0/3336076411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a64159bf0 0x7f4a6415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a63fff640 1 --2- 172.21.15.70:0/3336076411 >> v2:172.21.15.155:3300/0 conn(0x7f4a640cd660 0x7f4a640cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a5bfff640 1 -- 172.21.15.70:0/3336076411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a64159bf0 msgr2=0x7f4a6415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:56.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a5bfff640 1 --2- 172.21.15.70:0/3336076411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a64159bf0 0x7f4a6415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a5bfff640 1 -- 172.21.15.70:0/3336076411 >> v2:172.21.15.155:3300/0 conn(0x7f4a640cd660 msgr2=0x7f4a640cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a5bfff640 1 --2- 172.21.15.70:0/3336076411 >> v2:172.21.15.155:3300/0 conn(0x7f4a640cd660 0x7f4a640cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.675+0000 7f4a5bfff640 1 -- 172.21.15.70:0/3336076411 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6406fbc0 con 0x7f4a6406f0e0 2023-08-25T09:43:56.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.676+0000 7f4a5bfff640 1 --2- 172.21.15.70:0/3336076411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6406f0e0 0x7f4a64070980 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4a64071cd0 tx=0x7f4a4c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aad93edc8e152e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:56.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.676+0000 7f4a62ffd640 1 -- 172.21.15.70:0/3336076411 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c02ee90 con 0x7f4a6406f0e0 2023-08-25T09:43:56.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.676+0000 7f4a62ffd640 1 -- 172.21.15.70:0/3336076411 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4c036440 con 0x7f4a6406f0e0 2023-08-25T09:43:56.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.676+0000 7f4a62ffd640 1 -- 172.21.15.70:0/3336076411 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c0367d0 con 0x7f4a6406f0e0 2023-08-25T09:43:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3336076411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6406f0e0 msgr2=0x7f4a64070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3336076411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6406f0e0 0x7f4a64070980 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f4a64071cd0 tx=0x7f4a4c02e0e0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3336076411 shutdown_connections 2023-08-25T09:43:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3336076411 >> v2:172.21.15.155:3300/0 conn(0x7f4a640cd660 0x7f4a640cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3336076411 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a6406f0e0 0x7f4a64070980 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3336076411 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a64159bf0 0x7f4a6415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3336076411 >> 172.21.15.70:0/3336076411 conn(0x7f4a640cb0d0 msgr2=0x7f4a640cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:56.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3336076411 shutdown_connections 2023-08-25T09:43:56.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.677+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3336076411 wait complete. 2023-08-25T09:43:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.679+0000 7f4a6a1f3640 1 Processor -- start 2023-08-25T09:43:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.679+0000 7f4a6a1f3640 1 -- start start 2023-08-25T09:43:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a6a1f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a6406f0e0 0x7f4a640c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a6a1f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a640cd660 0x7f4a640c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a6a1f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 0x7f4a6414a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a6a1f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640c3ad0 con 0x7f4a64159bf0 2023-08-25T09:43:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a6a1f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640c3db0 con 0x7f4a640cd660 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a6a1f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a640c4090 con 0x7f4a6406f0e0 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a63fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 0x7f4a6414a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a63fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 0x7f4a6414a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42140/0 (socket says 172.21.15.70:42140) 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.680+0000 7f4a63fff640 1 -- 172.21.15.70:0/3779783016 learned_addr learned my addr 172.21.15.70:0/3779783016 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a637fe640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a6406f0e0 0x7f4a640c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a63fff640 1 -- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a6406f0e0 msgr2=0x7f4a640c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a5bfff640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a640cd660 0x7f4a640c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a63fff640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a6406f0e0 0x7f4a640c4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a63fff640 1 -- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a640cd660 msgr2=0x7f4a640c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a63fff640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a640cd660 0x7f4a640c6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a63fff640 1 -- 172.21.15.70:0/3779783016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c00b770 con 0x7f4a64159bf0 2023-08-25T09:43:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a5bfff640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a640cd660 0x7f4a640c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a63fff640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 0x7f4a6414a890 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00cd70 tx=0x7f4a5c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.681+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c003b90 con 0x7f4a64159bf0 2023-08-25T09:43:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.682+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5c003cf0 con 0x7f4a64159bf0 2023-08-25T09:43:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.682+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a640c4370 con 0x7f4a64159bf0 2023-08-25T09:43:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.682+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c004240 con 0x7f4a64159bf0 2023-08-25T09:43:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.682+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a640c36d0 con 0x7f4a64159bf0 2023-08-25T09:43:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.683+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a64066fb0 con 0x7f4a64159bf0 2023-08-25T09:43:56.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.684+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4a5c0043a0 con 0x7f4a64159bf0 2023-08-25T09:43:56.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.684+0000 7f4a60ff9640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4a4406a050 0x7f4a4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:56.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.685+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4a5c012070 con 0x7f4a64159bf0 2023-08-25T09:43:56.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.685+0000 7f4a637fe640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4a4406a050 0x7f4a4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:56.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.685+0000 7f4a637fe640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4a4406a050 0x7f4a4406c510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b740 tx=0x7f4a54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:56.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.689+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4a5c054700 con 0x7f4a64159bf0 2023-08-25T09:43:56.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:56.912+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4a640cbf50 con 0x7f4a64159bf0 2023-08-25T09:43:57.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.429+0000 7f4a60ff9640 1 -- 172.21.15.70:0/3779783016 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4a5c057d20 con 0x7f4a64159bf0 2023-08-25T09:43:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4a4406a050 msgr2=0x7f4a4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4a4406a050 0x7f4a4406c510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b740 tx=0x7f4a54008040 comp rx=0 tx=0).stop 2023-08-25T09:43:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 msgr2=0x7f4a6414a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 0x7f4a6414a890 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00cd70 tx=0x7f4a5c00a660 comp rx=0 tx=0).stop 2023-08-25T09:43:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 shutdown_connections 2023-08-25T09:43:57.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4a4406a050 0x7f4a4406c510 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4a64159bf0 0x7f4a6414a890 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4a640cd660 0x7f4a640c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 --2- 172.21.15.70:0/3779783016 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4a6406f0e0 0x7f4a640c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 >> 172.21.15.70:0/3779783016 conn(0x7f4a640cb0d0 msgr2=0x7f4a64150260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:57.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 shutdown_connections 2023-08-25T09:43:57.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.433+0000 7f4a6a1f3640 1 -- 172.21.15.70:0/3779783016 wait complete. 2023-08-25T09:43:57.455 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:43:57.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:43:57.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.670+0000 7f35c9621640 1 Processor -- start 2023-08-25T09:43:57.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.671+0000 7f35c9621640 1 -- start start 2023-08-25T09:43:57.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.672+0000 7f35c9621640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35c415ec10 0x7f35c415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.672+0000 7f35c9621640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415c6d0 0x7f35c40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.672+0000 7f35c9621640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c40b1b60 con 0x7f35c415e830 2023-08-25T09:43:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.672+0000 7f35c9621640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c406bd80 con 0x7f35c415ec10 2023-08-25T09:43:57.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.672+0000 7f35c9621640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c406b7c0 con 0x7f35c415c6d0 2023-08-25T09:43:57.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.672+0000 7f35c27fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35c415ec10 0x7f35c415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:57.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.673+0000 7f35c27fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35c415ec10 0x7f35c415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42146/0 (socket says 172.21.15.70:42146) 2023-08-25T09:43:57.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.673+0000 7f35c27fc640 1 -- 172.21.15.70:0/3636958632 learned_addr learned my addr 172.21.15.70:0/3636958632 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:57.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.673+0000 7f35c27fc640 1 -- 172.21.15.70:0/3636958632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415c6d0 msgr2=0x7f35c40b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.673+0000 7f35c27fc640 1 --2- 172.21.15.70:0/3636958632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415c6d0 0x7f35c40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.673+0000 7f35c27fc640 1 -- 172.21.15.70:0/3636958632 >> v1:172.21.15.155:6789/0 conn(0x7f35c415e830 legacy=0x7f35c406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.673+0000 7f35c27fc640 1 -- 172.21.15.70:0/3636958632 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c406baa0 con 0x7f35c415ec10 2023-08-25T09:43:57.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.674+0000 7f35c27fc640 1 --2- 172.21.15.70:0/3636958632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415ec10 0x7f35c415c190 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009870 tx=0x7f35b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af94342f97a7057 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.674+0000 7f35c1ffb640 1 -- 172.21.15.70:0/3636958632 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8031640 con 0x7f35c415ec10 2023-08-25T09:43:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.674+0000 7f35c1ffb640 1 -- 172.21.15.70:0/3636958632 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b80317a0 con 0x7f35c415ec10 2023-08-25T09:43:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.674+0000 7f35c1ffb640 1 -- 172.21.15.70:0/3636958632 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8031b30 con 0x7f35c415ec10 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 -- 172.21.15.70:0/3636958632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415ec10 msgr2=0x7f35c415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 --2- 172.21.15.70:0/3636958632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415ec10 0x7f35c415c190 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009870 tx=0x7f35b802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 -- 172.21.15.70:0/3636958632 shutdown_connections 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 --2- 172.21.15.70:0/3636958632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415c6d0 0x7f35c40b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 --2- 172.21.15.70:0/3636958632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415ec10 0x7f35c415c190 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 -- 172.21.15.70:0/3636958632 >> 172.21.15.70:0/3636958632 conn(0x7f35c4069850 msgr2=0x7f35c4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 -- 172.21.15.70:0/3636958632 shutdown_connections 2023-08-25T09:43:57.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.675+0000 7f35c9621640 1 -- 172.21.15.70:0/3636958632 wait complete. 2023-08-25T09:43:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.677+0000 7f35c9621640 1 Processor -- start 2023-08-25T09:43:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.677+0000 7f35c9621640 1 -- start start 2023-08-25T09:43:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c9621640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 0x7f35c41587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:57.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c9621640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35c415e830 0x7f35c41e5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c9621640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415ec10 0x7f35c414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c9621640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c414b670 con 0x7f35c415c6d0 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c9621640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c41e47a0 con 0x7f35c415e830 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c9621640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c41e4a80 con 0x7f35c415ec10 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 0x7f35c41587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 0x7f35c41587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42158/0 (socket says 172.21.15.70:42158) 2023-08-25T09:43:57.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 -- 172.21.15.70:0/1306287366 learned_addr learned my addr 172.21.15.70:0/1306287366 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 -- 172.21.15.70:0/1306287366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415ec10 msgr2=0x7f35c414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415ec10 0x7f35c414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 -- 172.21.15.70:0/1306287366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35c415e830 msgr2=0x7f35c41e5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35c415e830 0x7f35c41e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.678+0000 7f35c2ffd640 1 -- 172.21.15.70:0/1306287366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80094d0 con 0x7f35c415c6d0 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.679+0000 7f35c2ffd640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 0x7f35c41587a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f35b400c410 tx=0x7f35b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.679+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40035f0 con 0x7f35c415c6d0 2023-08-25T09:43:57.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.679+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b4003750 con 0x7f35c415c6d0 2023-08-25T09:43:57.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.679+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4003b40 con 0x7f35c415c6d0 2023-08-25T09:43:57.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.679+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c41e4d60 con 0x7f35c415c6d0 2023-08-25T09:43:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.679+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c41e5320 con 0x7f35c415c6d0 2023-08-25T09:43:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.680+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c404e420 con 0x7f35c415c6d0 2023-08-25T09:43:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.681+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f35b4003ca0 con 0x7f35c415c6d0 2023-08-25T09:43:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.682+0000 7f35ab7fe640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b006a050 0x7f35b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.682+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35b40878a0 con 0x7f35c415c6d0 2023-08-25T09:43:57.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.682+0000 7f35c27fc640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b006a050 0x7f35b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:57.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.683+0000 7f35c27fc640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b006a050 0x7f35b006c510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f35b8036000 tx=0x7f35b8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:57.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.687+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35b4052ef0 con 0x7f35c415c6d0 2023-08-25T09:43:57.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.899+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35c406c620 con 0x7f35c415c6d0 2023-08-25T09:43:57.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.901+0000 7f35ab7fe640 1 -- 172.21.15.70:0/1306287366 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7f35b4056510 con 0x7f35c415c6d0 2023-08-25T09:43:57.902 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:57.903 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:43:57.903 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:43:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b006a050 msgr2=0x7f35b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b006a050 0x7f35b006c510 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f35b8036000 tx=0x7f35b8030000 comp rx=0 tx=0).stop 2023-08-25T09:43:57.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 msgr2=0x7f35c41587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 0x7f35c41587a0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f35b400c410 tx=0x7f35b400c8d0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 shutdown_connections 2023-08-25T09:43:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b006a050 0x7f35b006c510 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35c415ec10 0x7f35c414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35c415e830 0x7f35c41e5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 --2- 172.21.15.70:0/1306287366 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35c415c6d0 0x7f35c41587a0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:57.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 >> 172.21.15.70:0/1306287366 conn(0x7f35c4069850 msgr2=0x7f35c415dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 shutdown_connections 2023-08-25T09:43:57.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:57.903+0000 7f35c9621640 1 -- 172.21.15.70:0/1306287366 wait complete. 2023-08-25T09:43:57.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:43:58.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.131+0000 7fc1adc06640 1 Processor -- start 2023-08-25T09:43:58.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.131+0000 7fc1adc06640 1 -- start start 2023-08-25T09:43:58.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.132+0000 7fc1adc06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a80709a0 0x7fc1a80ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.132+0000 7fc1adc06640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1a806f0a0 0x7fc1a806be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.132+0000 7fc1adc06640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a806ae10 con 0x7fc1a806c3b0 2023-08-25T09:43:58.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.132+0000 7fc1adc06640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a806ba50 con 0x7fc1a80709a0 2023-08-25T09:43:58.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.132+0000 7fc1adc06640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a806b0b0 con 0x7fc1a806f0a0 2023-08-25T09:43:58.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a80709a0 0x7fc1a80ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a80709a0 0x7fc1a80ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42164/0 (socket says 172.21.15.70:42164) 2023-08-25T09:43:58.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 -- 172.21.15.70:0/3037671044 learned_addr learned my addr 172.21.15.70:0/3037671044 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:58.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 -- 172.21.15.70:0/3037671044 >> v2:172.21.15.155:3300/0 conn(0x7fc1a806f0a0 msgr2=0x7fc1a806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:58.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 --2- 172.21.15.70:0/3037671044 >> v2:172.21.15.155:3300/0 conn(0x7fc1a806f0a0 0x7fc1a806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 -- 172.21.15.70:0/3037671044 >> v1:172.21.15.155:6789/0 conn(0x7fc1a806c3b0 legacy=0x7fc1a80b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:58.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.133+0000 7fc1a77fe640 1 -- 172.21.15.70:0/3037671044 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a806b390 con 0x7fc1a80709a0 2023-08-25T09:43:58.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.134+0000 7fc1a77fe640 1 --2- 172.21.15.70:0/3037671044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80ce600 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc1a806eac0 tx=0x7fc19802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02bc1e090df526a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:58.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.135+0000 7fc1a6ffd640 1 -- 172.21.15.70:0/3037671044 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1980317e0 con 0x7fc1a80709a0 2023-08-25T09:43:58.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.135+0000 7fc1a6ffd640 1 -- 172.21.15.70:0/3037671044 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc198031940 con 0x7fc1a80709a0 2023-08-25T09:43:58.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.135+0000 7fc1a6ffd640 1 -- 172.21.15.70:0/3037671044 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198031cd0 con 0x7fc1a80709a0 2023-08-25T09:43:58.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.135+0000 7fc1adc06640 1 -- 172.21.15.70:0/3037671044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 msgr2=0x7fc1a80ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.135+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3037671044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80ce600 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc1a806eac0 tx=0x7fc19802b880 comp rx=0 tx=0).stop 2023-08-25T09:43:58.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.136+0000 7fc1adc06640 1 -- 172.21.15.70:0/3037671044 shutdown_connections 2023-08-25T09:43:58.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.136+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3037671044 >> v2:172.21.15.155:3300/0 conn(0x7fc1a806f0a0 0x7fc1a806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.136+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3037671044 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80ce600 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.136+0000 7fc1adc06640 1 -- 172.21.15.70:0/3037671044 >> 172.21.15.70:0/3037671044 conn(0x7fc1a8069850 msgr2=0x7fc1a8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:58.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.136+0000 7fc1adc06640 1 -- 172.21.15.70:0/3037671044 shutdown_connections 2023-08-25T09:43:58.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.136+0000 7fc1adc06640 1 -- 172.21.15.70:0/3037671044 wait complete. 2023-08-25T09:43:58.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.137+0000 7fc1adc06640 1 Processor -- start 2023-08-25T09:43:58.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.137+0000 7fc1adc06640 1 -- start start 2023-08-25T09:43:58.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1adc06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a806c3b0 0x7fc1a8146200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1adc06640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a806f0a0 0x7fc1a80a9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1adc06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1adc06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8159750 con 0x7fc1a80709a0 2023-08-25T09:43:58.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1adc06640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a8146740 con 0x7fc1a806f0a0 2023-08-25T09:43:58.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1adc06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a80a9d60 con 0x7fc1a806c3b0 2023-08-25T09:43:58.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.138+0000 7fc1a7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80a9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42178/0 (socket says 172.21.15.70:42178) 2023-08-25T09:43:58.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 -- 172.21.15.70:0/3414634568 learned_addr learned my addr 172.21.15.70:0/3414634568 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:58.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 -- 172.21.15.70:0/3414634568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a806c3b0 msgr2=0x7fc1a8146200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a806c3b0 0x7fc1a8146200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 -- 172.21.15.70:0/3414634568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a806f0a0 msgr2=0x7fc1a80a9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a806f0a0 0x7fc1a80a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 -- 172.21.15.70:0/3414634568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1980094d0 con 0x7fc1a80709a0 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a7fff640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80a9540 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c780 tx=0x7fc1a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0003d80 con 0x7fc1a80709a0 2023-08-25T09:43:58.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a000ce90 con 0x7fc1a80709a0 2023-08-25T09:43:58.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a00052b0 con 0x7fc1a80709a0 2023-08-25T09:43:58.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.139+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a80aa040 con 0x7fc1a80709a0 2023-08-25T09:43:58.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.140+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a81593e0 con 0x7fc1a80709a0 2023-08-25T09:43:58.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.141+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a804e420 con 0x7fc1a80709a0 2023-08-25T09:43:58.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.142+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc1a0005410 con 0x7fc1a80709a0 2023-08-25T09:43:58.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.142+0000 7fc1a4ff9640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18806a050 0x7fc18806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.143+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc1a0087b00 con 0x7fc1a80709a0 2023-08-25T09:43:58.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.143+0000 7fc1a77fe640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18806a050 0x7fc18806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.144+0000 7fc1a77fe640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18806a050 0x7fc18806c510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc198036000 tx=0x7fc198030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:58.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.147+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc1a0089e60 con 0x7fc1a80709a0 2023-08-25T09:43:58.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.335+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1a80a9a80 con 0x7fc1a80709a0 2023-08-25T09:43:58.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.339+0000 7fc1a4ff9640 1 -- 172.21.15.70:0/3414634568 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+18308 (secure 0 0 0) 0x7fc1a0053150 con 0x7fc1a80709a0 2023-08-25T09:43:58.341 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:58.341 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":94,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:43:55.379594+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:43:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18806a050 msgr2=0x7fc18806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18806a050 0x7fc18806c510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc198036000 tx=0x7fc198030000 comp rx=0 tx=0).stop 2023-08-25T09:43:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 msgr2=0x7fc1a80a9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80a9540 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000c780 tx=0x7fc1a000cc40 comp rx=0 tx=0).stop 2023-08-25T09:43:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 shutdown_connections 2023-08-25T09:43:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18806a050 0x7fc18806c510 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a80709a0 0x7fc1a80a9540 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a806f0a0 0x7fc1a80a9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 --2- 172.21.15.70:0/3414634568 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a806c3b0 0x7fc1a8146200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.341+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 >> 172.21.15.70:0/3414634568 conn(0x7fc1a8069850 msgr2=0x7fc1a80cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:58.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.342+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 shutdown_connections 2023-08-25T09:43:58.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.342+0000 7fc1adc06640 1 -- 172.21.15.70:0/3414634568 wait complete. 2023-08-25T09:43:58.362 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:43:58.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:43:58.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.573+0000 7f552eafa640 1 Processor -- start 2023-08-25T09:43:58.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.573+0000 7f552eafa640 1 -- start start 2023-08-25T09:43:58.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552eafa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f552815ad30 0x7f55280b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552eafa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552eafa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55280ccff0 con 0x7f55280b36b0 2023-08-25T09:43:58.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552eafa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552815b110 con 0x7f552815ad30 2023-08-25T09:43:58.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552eafa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280d0720 con 0x7f55280d0290 2023-08-25T09:43:58.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552c896640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f552815ad30 0x7f55280b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552c896640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f552815ad30 0x7f55280b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42192/0 (socket says 172.21.15.70:42192) 2023-08-25T09:43:58.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.574+0000 7f552c896640 1 -- 172.21.15.70:0/269171168 learned_addr learned my addr 172.21.15.70:0/269171168 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:58.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.575+0000 7f552c896640 1 -- 172.21.15.70:0/269171168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 msgr2=0x7f55280b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:58.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.575+0000 7f552c896640 1 --2- 172.21.15.70:0/269171168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.575+0000 7f552c896640 1 -- 172.21.15.70:0/269171168 >> v1:172.21.15.155:6789/0 conn(0x7f55280b36b0 legacy=0x7f55280cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:58.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.575+0000 7f552c896640 1 -- 172.21.15.70:0/269171168 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280cc450 con 0x7f552815ad30 2023-08-25T09:43:58.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.576+0000 7f552c896640 1 --2- 172.21.15.70:0/269171168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552815ad30 0x7f55280b2300 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e80 tx=0x7f551802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fc840f10959a75 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:58.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.576+0000 7f55277fe640 1 -- 172.21.15.70:0/269171168 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551802ca50 con 0x7f552815ad30 2023-08-25T09:43:58.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.576+0000 7f55277fe640 1 -- 172.21.15.70:0/269171168 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551802cbb0 con 0x7f552815ad30 2023-08-25T09:43:58.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.576+0000 7f55277fe640 1 -- 172.21.15.70:0/269171168 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180343f0 con 0x7f552815ad30 2023-08-25T09:43:58.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 -- 172.21.15.70:0/269171168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552815ad30 msgr2=0x7f55280b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 --2- 172.21.15.70:0/269171168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552815ad30 0x7f55280b2300 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5518009e80 tx=0x7f551802beb0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 -- 172.21.15.70:0/269171168 shutdown_connections 2023-08-25T09:43:58.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 --2- 172.21.15.70:0/269171168 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 --2- 172.21.15.70:0/269171168 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f552815ad30 0x7f55280b2300 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 -- 172.21.15.70:0/269171168 >> 172.21.15.70:0/269171168 conn(0x7f55280cd300 msgr2=0x7f55280cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:58.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 -- 172.21.15.70:0/269171168 shutdown_connections 2023-08-25T09:43:58.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.577+0000 7f552eafa640 1 -- 172.21.15.70:0/269171168 wait complete. 2023-08-25T09:43:58.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.578+0000 7f552eafa640 1 Processor -- start 2023-08-25T09:43:58.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.579+0000 7f552eafa640 1 -- start start 2023-08-25T09:43:58.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.579+0000 7f552eafa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 0x7f552814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552eafa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552eafa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f552815ad30 0x7f55280aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552eafa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5528069c30 con 0x7f55280b36b0 2023-08-25T09:43:58.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552eafa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5528150390 con 0x7f552815ad30 2023-08-25T09:43:58.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552eafa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280ad800 con 0x7f55280d0290 2023-08-25T09:43:58.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 0x7f552814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 0x7f552814fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42198/0 (socket says 172.21.15.70:42198) 2023-08-25T09:43:58.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 -- 172.21.15.70:0/3315239968 learned_addr learned my addr 172.21.15.70:0/3315239968 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:58.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 -- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 msgr2=0x7f55280ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f5527fff640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280ae850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 -- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f552815ad30 msgr2=0x7f55280aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552d097640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f552815ad30 0x7f55280aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f552815ad30 0x7f55280aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f552c896640 1 -- 172.21.15.70:0/3315239968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518009b30 con 0x7f55280b36b0 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.580+0000 7f5527fff640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.581+0000 7f552c896640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 0x7f552814fe50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f55180093e0 tx=0x7f5518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.581+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551803e450 con 0x7f55280b36b0 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.581+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551803e5b0 con 0x7f55280b36b0 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.581+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551803e960 con 0x7f55280b36b0 2023-08-25T09:43:58.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.581+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55280adae0 con 0x7f55280b36b0 2023-08-25T09:43:58.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.581+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5528069850 con 0x7f55280b36b0 2023-08-25T09:43:58.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.582+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552804e420 con 0x7f55280b36b0 2023-08-25T09:43:58.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.583+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f551803eac0 con 0x7f55280b36b0 2023-08-25T09:43:58.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.584+0000 7f55257fa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f550806a050 0x7f550806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:58.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.584+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55180ab490 con 0x7f55280b36b0 2023-08-25T09:43:58.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.584+0000 7f5527fff640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f550806a050 0x7f550806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:58.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.585+0000 7f5527fff640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f550806a050 0x7f550806c510 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55280ae5d0 tx=0x7f5510008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:58.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.589+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5518076c90 con 0x7f55280b36b0 2023-08-25T09:43:58.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.772+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55280cc9a0 con 0x7f550806a050 2023-08-25T09:43:58.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.787+0000 7f55257fa640 1 -- 172.21.15.70:0/3315239968 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297739 (secure 0 0 0) 0x7f551000ad60 con 0x7f550806a050 2023-08-25T09:43:58.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:58.794 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:43:58.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f550806a050 msgr2=0x7f550806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f550806a050 0x7f550806c510 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f55280ae5d0 tx=0x7f5510008040 comp rx=0 tx=0).stop 2023-08-25T09:43:58.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 msgr2=0x7f552814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:58.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 0x7f552814fe50 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f55180093e0 tx=0x7f5518004a40 comp rx=0 tx=0).stop 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 shutdown_connections 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f550806a050 0x7f550806c510 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f552815ad30 0x7f55280aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55280d0290 0x7f55280ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 --2- 172.21.15.70:0/3315239968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f55280b36b0 0x7f552814fe50 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 >> 172.21.15.70:0/3315239968 conn(0x7f55280cd300 msgr2=0x7f55280cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:58.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 shutdown_connections 2023-08-25T09:43:58.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:58.794+0000 7f552eafa640 1 -- 172.21.15.70:0/3315239968 wait complete. 2023-08-25T09:43:58.815 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-08-25T09:43:57.420426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5238,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557261,"statfs":{"total":1600000000,"available":1594636224,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060816"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394706+0000","last_change":"2023-08-25T09:43:53.394706+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394706+0000","last_undegraded":"2023-08-25T09:43:53.394706+0000","last_fullsized":"2023-08-25T09:43:53.394706+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388146+0000","last_change":"2023-08-25T09:43:53.388146+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388146+0000","last_undegraded":"2023-08-25T09:43:53.388146+0000","last_fullsized":"2023-08-25T09:43:53.388146+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390842+0000","last_change":"2023-08-25T09:43:53.390842+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390842+0000","last_undegraded":"2023-08-25T09:43:53.390842+0000","last_fullsized":"2023-08-25T09:43:53.390842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388186+0000","last_change":"2023-08-25T09:43:53.388186+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388186+0000","last_undegraded":"2023-08-25T09:43:53.388186+0000","last_fullsized":"2023-08-25T09:43:53.388186+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.391681+0000","last_change":"2023-08-25T09:43:53.391681+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.391681+0000","last_undegraded":"2023-08-25T09:43:53.391681+0000","last_fullsized":"2023-08-25T09:43:53.391681+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393690+0000","last_change":"2023-08-25T09:43:53.393690+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393690+0000","last_undegraded":"2023-08-25T09:43:53.393690+0000","last_fullsized":"2023-08-25T09:43:53.393690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388452+0000","last_change":"2023-08-25T09:43:53.388452+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388452+0000","last_undegraded":"2023-08-25T09:43:53.388452+0000","last_fullsized":"2023-08-25T09:43:53.388452+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.398637+0000","last_change":"2023-08-25T09:43:53.398637+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.398637+0000","last_undegraded":"2023-08-25T09:43:53.398637+0000","last_fullsized":"2023-08-25T09:43:53.398637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388519+0000","last_change":"2023-08-25T09:43:53.388519+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388519+0000","last_undegraded":"2023-08-25T09:43:53.388519+0000","last_fullsized":"2023-08-25T09:43:53.388519+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392240+0000","last_change":"2023-08-25T09:43:53.392240+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392240+0000","last_undegraded":"2023-08-25T09:43:53.392240+0000","last_fullsized":"2023-08-25T09:43:53.392240+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390689+0000","last_change":"2023-08-25T09:43:53.390689+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390689+0000","last_undegraded":"2023-08-25T09:43:53.390689+0000","last_fullsized":"2023-08-25T09:43:53.390689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.384369+0000","last_change":"2023-08-25T09:43:53.384369+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.384369+0000","last_undegraded":"2023-08-25T09:43:53.384369+0000","last_fullsized":"2023-08-25T09:43:53.384369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390224+0000","last_change":"2023-08-25T09:43:53.390224+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390224+0000","last_undegraded":"2023-08-25T09:43:53.390224+0000","last_fullsized":"2023-08-25T09:43:53.390224+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.395201+0000","last_change":"2023-08-25T09:43:53.395201+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.395201+0000","last_undegraded":"2023-08-25T09:43:53.395201+0000","last_fullsized":"2023-08-25T09:43:53.395201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.391206+0000","last_change":"2023-08-25T09:43:53.391206+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.391206+0000","last_undegraded":"2023-08-25T09:43:53.391206+0000","last_fullsized":"2023-08-25T09:43:53.391206+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386780+0000","last_change":"2023-08-25T09:43:53.386780+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386780+0000","last_undegraded":"2023-08-25T09:43:53.386780+0000","last_fullsized":"2023-08-25T09:43:53.386780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392750+0000","last_change":"2023-08-25T09:43:53.392750+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392750+0000","last_undegraded":"2023-08-25T09:43:53.392750+0000","last_fullsized":"2023-08-25T09:43:53.392750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389618+0000","last_change":"2023-08-25T09:43:53.389618+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389618+0000","last_undegraded":"2023-08-25T09:43:53.389618+0000","last_fullsized":"2023-08-25T09:43:53.389618+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394736+0000","last_change":"2023-08-25T09:43:53.394736+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394736+0000","last_undegraded":"2023-08-25T09:43:53.394736+0000","last_fullsized":"2023-08-25T09:43:53.394736+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393381+0000","last_change":"2023-08-25T09:43:53.393381+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393381+0000","last_undegraded":"2023-08-25T09:43:53.393381+0000","last_fullsized":"2023-08-25T09:43:53.393381+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382607+0000","last_change":"2023-08-25T09:43:53.382607+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382607+0000","last_undegraded":"2023-08-25T09:43:53.382607+0000","last_fullsized":"2023-08-25T09:43:53.382607+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387309+0000","last_change":"2023-08-25T09:43:53.387309+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387309+0000","last_undegraded":"2023-08-25T09:43:53.387309+0000","last_fullsized":"2023-08-25T09:43:53.387309+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392937+0000","last_change":"2023-08-25T09:43:53.392937+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392937+0000","last_undegraded":"2023-08-25T09:43:53.392937+0000","last_fullsized":"2023-08-25T09:43:53.392937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385729+0000","last_change":"2023-08-25T09:43:53.385729+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385729+0000","last_undegraded":"2023-08-25T09:43:53.385729+0000","last_fullsized":"2023-08-25T09:43:53.385729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388736+0000","last_change":"2023-08-25T09:43:53.388736+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388736+0000","last_undegraded":"2023-08-25T09:43:53.388736+0000","last_fullsized":"2023-08-25T09:43:53.388736+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389631+0000","last_change":"2023-08-25T09:43:53.389631+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389631+0000","last_undegraded":"2023-08-25T09:43:53.389631+0000","last_fullsized":"2023-08-25T09:43:53.389631+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387236+0000","last_change":"2023-08-25T09:43:53.387236+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387236+0000","last_undegraded":"2023-08-25T09:43:53.387236+0000","last_fullsized":"2023-08-25T09:43:53.387236+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393788+0000","last_change":"2023-08-25T09:43:53.393788+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393788+0000","last_undegraded":"2023-08-25T09:43:53.393788+0000","last_fullsized":"2023-08-25T09:43:53.393788+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393225+0000","last_change":"2023-08-25T09:43:53.393225+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393225+0000","last_undegraded":"2023-08-25T09:43:53.393225+0000","last_fullsized":"2023-08-25T09:43:53.393225+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392835+0000","last_change":"2023-08-25T09:43:53.392835+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392835+0000","last_undegraded":"2023-08-25T09:43:53.392835+0000","last_fullsized":"2023-08-25T09:43:53.392835+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390133+0000","last_change":"2023-08-25T09:43:53.390133+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390133+0000","last_undegraded":"2023-08-25T09:43:53.390133+0000","last_fullsized":"2023-08-25T09:43:53.390133+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386447+0000","last_change":"2023-08-25T09:43:53.386447+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386447+0000","last_undegraded":"2023-08-25T09:43:53.386447+0000","last_fullsized":"2023-08-25T09:43:53.386447+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390654+0000","last_change":"2023-08-25T09:43:53.390654+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390654+0000","last_undegraded":"2023-08-25T09:43:53.390654+0000","last_fullsized":"2023-08-25T09:43:53.390654+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.398618+0000","last_change":"2023-08-25T09:43:53.397210+0000","last_active":"2023-08-25T09:43:53.398618+0000","last_peered":"2023-08-25T09:43:53.398618+0000","last_clean":"2023-08-25T09:43:53.398618+0000","last_became_active":"2023-08-25T09:43:53.396946+0000","last_became_peered":"2023-08-25T09:43:53.396946+0000","last_unstale":"2023-08-25T09:43:53.398618+0000","last_undegraded":"2023-08-25T09:43:53.398618+0000","last_fullsized":"2023-08-25T09:43:53.398618+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:30.659547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382695+0000","last_change":"2023-08-25T09:43:53.382695+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382695+0000","last_undegraded":"2023-08-25T09:43:53.382695+0000","last_fullsized":"2023-08-25T09:43:53.382695+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.393543+0000","last_change":"2023-08-25T09:43:53.393077+0000","last_active":"2023-08-25T09:43:53.393543+0000","last_peered":"2023-08-25T09:43:53.393543+0000","last_clean":"2023-08-25T09:43:53.393543+0000","last_became_active":"2023-08-25T09:43:53.392575+0000","last_became_peered":"2023-08-25T09:43:53.392575+0000","last_unstale":"2023-08-25T09:43:53.393543+0000","last_undegraded":"2023-08-25T09:43:53.393543+0000","last_fullsized":"2023-08-25T09:43:53.393543+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:28.378444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382523+0000","last_change":"2023-08-25T09:43:53.382523+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382523+0000","last_undegraded":"2023-08-25T09:43:53.382523+0000","last_fullsized":"2023-08-25T09:43:53.382523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399026+0000","last_change":"2023-08-25T09:43:53.398957+0000","last_active":"2023-08-25T09:43:53.399026+0000","last_peered":"2023-08-25T09:43:53.399026+0000","last_clean":"2023-08-25T09:43:53.399026+0000","last_became_active":"2023-08-25T09:43:53.398717+0000","last_became_peered":"2023-08-25T09:43:53.398717+0000","last_unstale":"2023-08-25T09:43:53.399026+0000","last_undegraded":"2023-08-25T09:43:53.399026+0000","last_fullsized":"2023-08-25T09:43:53.399026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:01:47.062976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387825+0000","last_change":"2023-08-25T09:43:53.387825+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387825+0000","last_undegraded":"2023-08-25T09:43:53.387825+0000","last_fullsized":"2023-08-25T09:43:53.387825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.389650+0000","last_change":"2023-08-25T09:43:53.389534+0000","last_active":"2023-08-25T09:43:53.389650+0000","last_peered":"2023-08-25T09:43:53.389650+0000","last_clean":"2023-08-25T09:43:53.389650+0000","last_became_active":"2023-08-25T09:43:53.389145+0000","last_became_peered":"2023-08-25T09:43:53.389145+0000","last_unstale":"2023-08-25T09:43:53.389650+0000","last_undegraded":"2023-08-25T09:43:53.389650+0000","last_fullsized":"2023-08-25T09:43:53.389650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:37:26.224049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389464+0000","last_change":"2023-08-25T09:43:53.389464+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389464+0000","last_undegraded":"2023-08-25T09:43:53.389464+0000","last_fullsized":"2023-08-25T09:43:53.389464+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.385290+0000","last_change":"2023-08-25T09:43:53.385163+0000","last_active":"2023-08-25T09:43:53.385290+0000","last_peered":"2023-08-25T09:43:53.385290+0000","last_clean":"2023-08-25T09:43:53.385290+0000","last_became_active":"2023-08-25T09:43:53.384839+0000","last_became_peered":"2023-08-25T09:43:53.384839+0000","last_unstale":"2023-08-25T09:43:53.385290+0000","last_undegraded":"2023-08-25T09:43:53.385290+0000","last_fullsized":"2023-08-25T09:43:53.385290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:29:38.869803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386780+0000","last_change":"2023-08-25T09:43:53.386780+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386780+0000","last_undegraded":"2023-08-25T09:43:53.386780+0000","last_fullsized":"2023-08-25T09:43:53.386780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401260+0000","last_change":"2023-08-25T09:43:53.401154+0000","last_active":"2023-08-25T09:43:53.401260+0000","last_peered":"2023-08-25T09:43:53.401260+0000","last_clean":"2023-08-25T09:43:53.401260+0000","last_became_active":"2023-08-25T09:43:53.400830+0000","last_became_peered":"2023-08-25T09:43:53.400830+0000","last_unstale":"2023-08-25T09:43:53.401260+0000","last_undegraded":"2023-08-25T09:43:53.401260+0000","last_fullsized":"2023-08-25T09:43:53.401260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:43.181749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382181+0000","last_change":"2023-08-25T09:43:53.382181+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382181+0000","last_undegraded":"2023-08-25T09:43:53.382181+0000","last_fullsized":"2023-08-25T09:43:53.382181+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401536+0000","last_change":"2023-08-25T09:43:53.401459+0000","last_active":"2023-08-25T09:43:53.401536+0000","last_peered":"2023-08-25T09:43:53.401536+0000","last_clean":"2023-08-25T09:43:53.401536+0000","last_became_active":"2023-08-25T09:43:53.400825+0000","last_became_peered":"2023-08-25T09:43:53.400825+0000","last_unstale":"2023-08-25T09:43:53.401536+0000","last_undegraded":"2023-08-25T09:43:53.401536+0000","last_fullsized":"2023-08-25T09:43:53.401536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:03:32.834869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385987+0000","last_change":"2023-08-25T09:43:53.385987+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385987+0000","last_undegraded":"2023-08-25T09:43:53.385987+0000","last_fullsized":"2023-08-25T09:43:53.385987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399517+0000","last_change":"2023-08-25T09:43:53.399348+0000","last_active":"2023-08-25T09:43:53.399517+0000","last_peered":"2023-08-25T09:43:53.399517+0000","last_clean":"2023-08-25T09:43:53.399517+0000","last_became_active":"2023-08-25T09:43:53.399099+0000","last_became_peered":"2023-08-25T09:43:53.399099+0000","last_unstale":"2023-08-25T09:43:53.399517+0000","last_undegraded":"2023-08-25T09:43:53.399517+0000","last_fullsized":"2023-08-25T09:43:53.399517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:28:28.010065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389030+0000","last_change":"2023-08-25T09:43:53.389030+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389030+0000","last_undegraded":"2023-08-25T09:43:53.389030+0000","last_fullsized":"2023-08-25T09:43:53.389030+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.398019+0000","last_change":"2023-08-25T09:43:53.397375+0000","last_active":"2023-08-25T09:43:53.398019+0000","last_peered":"2023-08-25T09:43:53.398019+0000","last_clean":"2023-08-25T09:43:53.398019+0000","last_became_active":"2023-08-25T09:43:53.397119+0000","last_became_peered":"2023-08-25T09:43:53.397119+0000","last_unstale":"2023-08-25T09:43:53.398019+0000","last_undegraded":"2023-08-25T09:43:53.398019+0000","last_fullsized":"2023-08-25T09:43:53.398019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:12:32.796450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.383399+0000","last_change":"2023-08-25T09:43:53.383399+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.383399+0000","last_undegraded":"2023-08-25T09:43:53.383399+0000","last_fullsized":"2023-08-25T09:43:53.383399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399347+0000","last_change":"2023-08-25T09:43:53.399279+0000","last_active":"2023-08-25T09:43:53.399347+0000","last_peered":"2023-08-25T09:43:53.399347+0000","last_clean":"2023-08-25T09:43:53.399347+0000","last_became_active":"2023-08-25T09:43:53.399023+0000","last_became_peered":"2023-08-25T09:43:53.399023+0000","last_unstale":"2023-08-25T09:43:53.399347+0000","last_undegraded":"2023-08-25T09:43:53.399347+0000","last_fullsized":"2023-08-25T09:43:53.399347+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:56:44.165537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.396739+0000","last_change":"2023-08-25T09:43:53.396739+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.396739+0000","last_undegraded":"2023-08-25T09:43:53.396739+0000","last_fullsized":"2023-08-25T09:43:53.396739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401575+0000","last_change":"2023-08-25T09:43:53.400335+0000","last_active":"2023-08-25T09:43:53.401575+0000","last_peered":"2023-08-25T09:43:53.401575+0000","last_clean":"2023-08-25T09:43:53.401575+0000","last_became_active":"2023-08-25T09:43:53.397372+0000","last_became_peered":"2023-08-25T09:43:53.397372+0000","last_unstale":"2023-08-25T09:43:53.401575+0000","last_undegraded":"2023-08-25T09:43:53.401575+0000","last_fullsized":"2023-08-25T09:43:53.401575+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:14.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387953+0000","last_change":"2023-08-25T09:43:53.387953+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387953+0000","last_undegraded":"2023-08-25T09:43:53.387953+0000","last_fullsized":"2023-08-25T09:43:53.387953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399436+0000","last_change":"2023-08-25T09:43:53.399357+0000","last_active":"2023-08-25T09:43:53.399436+0000","last_peered":"2023-08-25T09:43:53.399436+0000","last_clean":"2023-08-25T09:43:53.399436+0000","last_became_active":"2023-08-25T09:43:53.398803+0000","last_became_peered":"2023-08-25T09:43:53.398803+0000","last_unstale":"2023-08-25T09:43:53.399436+0000","last_undegraded":"2023-08-25T09:43:53.399436+0000","last_fullsized":"2023-08-25T09:43:53.399436+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:42.746759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386020+0000","last_change":"2023-08-25T09:43:53.386020+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386020+0000","last_undegraded":"2023-08-25T09:43:53.386020+0000","last_fullsized":"2023-08-25T09:43:53.386020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.400300+0000","last_change":"2023-08-25T09:43:53.400186+0000","last_active":"2023-08-25T09:43:53.400300+0000","last_peered":"2023-08-25T09:43:53.400300+0000","last_clean":"2023-08-25T09:43:53.400300+0000","last_became_active":"2023-08-25T09:43:53.399735+0000","last_became_peered":"2023-08-25T09:43:53.399735+0000","last_unstale":"2023-08-25T09:43:53.400300+0000","last_undegraded":"2023-08-25T09:43:53.400300+0000","last_fullsized":"2023-08-25T09:43:53.400300+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:26:44.320129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392484+0000","last_change":"2023-08-25T09:43:53.392484+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392484+0000","last_undegraded":"2023-08-25T09:43:53.392484+0000","last_fullsized":"2023-08-25T09:43:53.392484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.394286+0000","last_change":"2023-08-25T09:43:53.394175+0000","last_active":"2023-08-25T09:43:53.394286+0000","last_peered":"2023-08-25T09:43:53.394286+0000","last_clean":"2023-08-25T09:43:53.394286+0000","last_became_active":"2023-08-25T09:43:53.393635+0000","last_became_peered":"2023-08-25T09:43:53.393635+0000","last_unstale":"2023-08-25T09:43:53.394286+0000","last_undegraded":"2023-08-25T09:43:53.394286+0000","last_fullsized":"2023-08-25T09:43:53.394286+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:42:31.522547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.383906+0000","last_change":"2023-08-25T09:43:53.383906+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.383906+0000","last_undegraded":"2023-08-25T09:43:53.383906+0000","last_fullsized":"2023-08-25T09:43:53.383906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403254+0000","last_change":"2023-08-25T09:43:53.403154+0000","last_active":"2023-08-25T09:43:53.403254+0000","last_peered":"2023-08-25T09:43:53.403254+0000","last_clean":"2023-08-25T09:43:53.403254+0000","last_became_active":"2023-08-25T09:43:53.399755+0000","last_became_peered":"2023-08-25T09:43:53.399755+0000","last_unstale":"2023-08-25T09:43:53.403254+0000","last_undegraded":"2023-08-25T09:43:53.403254+0000","last_fullsized":"2023-08-25T09:43:53.403254+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:43:09.183840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394273+0000","last_change":"2023-08-25T09:43:53.394273+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394273+0000","last_undegraded":"2023-08-25T09:43:53.394273+0000","last_fullsized":"2023-08-25T09:43:53.394273+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403433+0000","last_change":"2023-08-25T09:43:53.403310+0000","last_active":"2023-08-25T09:43:53.403433+0000","last_peered":"2023-08-25T09:43:53.403433+0000","last_clean":"2023-08-25T09:43:53.403433+0000","last_became_active":"2023-08-25T09:43:53.400976+0000","last_became_peered":"2023-08-25T09:43:53.400976+0000","last_unstale":"2023-08-25T09:43:53.403433+0000","last_undegraded":"2023-08-25T09:43:53.403433+0000","last_fullsized":"2023-08-25T09:43:53.403433+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:42:08.889550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387526+0000","last_change":"2023-08-25T09:43:53.387526+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387526+0000","last_undegraded":"2023-08-25T09:43:53.387526+0000","last_fullsized":"2023-08-25T09:43:53.387526+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.400319+0000","last_change":"2023-08-25T09:43:53.400229+0000","last_active":"2023-08-25T09:43:53.400319+0000","last_peered":"2023-08-25T09:43:53.400319+0000","last_clean":"2023-08-25T09:43:53.400319+0000","last_became_active":"2023-08-25T09:43:53.400007+0000","last_became_peered":"2023-08-25T09:43:53.400007+0000","last_unstale":"2023-08-25T09:43:53.400319+0000","last_undegraded":"2023-08-25T09:43:53.400319+0000","last_fullsized":"2023-08-25T09:43:53.400319+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:28.772786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.391936+0000","last_change":"2023-08-25T09:43:53.391936+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.391936+0000","last_undegraded":"2023-08-25T09:43:53.391936+0000","last_fullsized":"2023-08-25T09:43:53.391936+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401594+0000","last_change":"2023-08-25T09:43:53.400677+0000","last_active":"2023-08-25T09:43:53.401594+0000","last_peered":"2023-08-25T09:43:53.401594+0000","last_clean":"2023-08-25T09:43:53.401594+0000","last_became_active":"2023-08-25T09:43:53.397908+0000","last_became_peered":"2023-08-25T09:43:53.397908+0000","last_unstale":"2023-08-25T09:43:53.401594+0000","last_undegraded":"2023-08-25T09:43:53.401594+0000","last_fullsized":"2023-08-25T09:43:53.401594+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:32.103986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.380835+0000","last_change":"2023-08-25T09:43:53.380835+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.380835+0000","last_undegraded":"2023-08-25T09:43:53.380835+0000","last_fullsized":"2023-08-25T09:43:53.380835+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.397632+0000","last_change":"2023-08-25T09:43:53.397556+0000","last_active":"2023-08-25T09:43:53.397632+0000","last_peered":"2023-08-25T09:43:53.397632+0000","last_clean":"2023-08-25T09:43:53.397632+0000","last_became_active":"2023-08-25T09:43:53.397131+0000","last_became_peered":"2023-08-25T09:43:53.397131+0000","last_unstale":"2023-08-25T09:43:53.397632+0000","last_undegraded":"2023-08-25T09:43:53.397632+0000","last_fullsized":"2023-08-25T09:43:53.397632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:06:19.290929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.396298+0000","last_change":"2023-08-25T09:43:53.396298+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.396298+0000","last_undegraded":"2023-08-25T09:43:53.396298+0000","last_fullsized":"2023-08-25T09:43:53.396298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.390553+0000","last_change":"2023-08-25T09:43:53.390461+0000","last_active":"2023-08-25T09:43:53.390553+0000","last_peered":"2023-08-25T09:43:53.390553+0000","last_clean":"2023-08-25T09:43:53.390553+0000","last_became_active":"2023-08-25T09:43:53.390304+0000","last_became_peered":"2023-08-25T09:43:53.390304+0000","last_unstale":"2023-08-25T09:43:53.390553+0000","last_undegraded":"2023-08-25T09:43:53.390553+0000","last_fullsized":"2023-08-25T09:43:53.390553+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:12.289326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.380931+0000","last_change":"2023-08-25T09:43:53.380931+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.380931+0000","last_undegraded":"2023-08-25T09:43:53.380931+0000","last_fullsized":"2023-08-25T09:43:53.380931+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.395273+0000","last_change":"2023-08-25T09:43:53.394719+0000","last_active":"2023-08-25T09:43:53.395273+0000","last_peered":"2023-08-25T09:43:53.395273+0000","last_clean":"2023-08-25T09:43:53.395273+0000","last_became_active":"2023-08-25T09:43:53.394231+0000","last_became_peered":"2023-08-25T09:43:53.394231+0000","last_unstale":"2023-08-25T09:43:53.395273+0000","last_undegraded":"2023-08-25T09:43:53.395273+0000","last_fullsized":"2023-08-25T09:43:53.395273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:10:59.580451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387859+0000","last_change":"2023-08-25T09:43:53.387859+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387859+0000","last_undegraded":"2023-08-25T09:43:53.387859+0000","last_fullsized":"2023-08-25T09:43:53.387859+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401611+0000","last_change":"2023-08-25T09:43:53.401366+0000","last_active":"2023-08-25T09:43:53.401611+0000","last_peered":"2023-08-25T09:43:53.401611+0000","last_clean":"2023-08-25T09:43:53.401611+0000","last_became_active":"2023-08-25T09:43:53.400954+0000","last_became_peered":"2023-08-25T09:43:53.400954+0000","last_unstale":"2023-08-25T09:43:53.401611+0000","last_undegraded":"2023-08-25T09:43:53.401611+0000","last_fullsized":"2023-08-25T09:43:53.401611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:29.214752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388434+0000","last_change":"2023-08-25T09:43:53.388434+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388434+0000","last_undegraded":"2023-08-25T09:43:53.388434+0000","last_fullsized":"2023-08-25T09:43:53.388434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.397194+0000","last_change":"2023-08-25T09:43:53.396976+0000","last_active":"2023-08-25T09:43:53.397194+0000","last_peered":"2023-08-25T09:43:53.397194+0000","last_clean":"2023-08-25T09:43:53.397194+0000","last_became_active":"2023-08-25T09:43:53.396705+0000","last_became_peered":"2023-08-25T09:43:53.396705+0000","last_unstale":"2023-08-25T09:43:53.397194+0000","last_undegraded":"2023-08-25T09:43:53.397194+0000","last_fullsized":"2023-08-25T09:43:53.397194+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:54:31.933639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382822+0000","last_change":"2023-08-25T09:43:53.382822+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382822+0000","last_undegraded":"2023-08-25T09:43:53.382822+0000","last_fullsized":"2023-08-25T09:43:53.382822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399494+0000","last_change":"2023-08-25T09:43:53.398900+0000","last_active":"2023-08-25T09:43:53.399494+0000","last_peered":"2023-08-25T09:43:53.399494+0000","last_clean":"2023-08-25T09:43:53.399494+0000","last_became_active":"2023-08-25T09:43:53.398452+0000","last_became_peered":"2023-08-25T09:43:53.398452+0000","last_unstale":"2023-08-25T09:43:53.399494+0000","last_undegraded":"2023-08-25T09:43:53.399494+0000","last_fullsized":"2023-08-25T09:43:53.399494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:42.580732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386971+0000","last_change":"2023-08-25T09:43:53.386971+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386971+0000","last_undegraded":"2023-08-25T09:43:53.386971+0000","last_fullsized":"2023-08-25T09:43:53.386971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.396058+0000","last_change":"2023-08-25T09:43:53.395918+0000","last_active":"2023-08-25T09:43:53.396058+0000","last_peered":"2023-08-25T09:43:53.396058+0000","last_clean":"2023-08-25T09:43:53.396058+0000","last_became_active":"2023-08-25T09:43:53.395232+0000","last_became_peered":"2023-08-25T09:43:53.395232+0000","last_unstale":"2023-08-25T09:43:53.396058+0000","last_undegraded":"2023-08-25T09:43:53.396058+0000","last_fullsized":"2023-08-25T09:43:53.396058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:50.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":289,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.392570+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:43:53.392570+0000","last_peered":"2023-08-25T09:43:53.392570+0000","last_clean":"2023-08-25T09:43:53.392570+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:43:53.392570+0000","last_undegraded":"2023-08-25T09:43:53.392570+0000","last_fullsized":"2023-08-25T09:43:53.392570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:14.142319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.378409+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:43:53.378409+0000","last_peered":"2023-08-25T09:43:53.378409+0000","last_clean":"2023-08-25T09:43:53.378409+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:43:53.378409+0000","last_undegraded":"2023-08-25T09:43:53.378409+0000","last_fullsized":"2023-08-25T09:43:53.378409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:04:54.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.381251+0000","last_change":"2023-08-25T09:43:53.381251+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.381251+0000","last_undegraded":"2023-08-25T09:43:53.381251+0000","last_fullsized":"2023-08-25T09:43:53.381251+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.402510+0000","last_change":"2023-08-25T09:43:53.402444+0000","last_active":"2023-08-25T09:43:53.402510+0000","last_peered":"2023-08-25T09:43:53.402510+0000","last_clean":"2023-08-25T09:43:53.402510+0000","last_became_active":"2023-08-25T09:43:53.402171+0000","last_became_peered":"2023-08-25T09:43:53.402171+0000","last_unstale":"2023-08-25T09:43:53.402510+0000","last_undegraded":"2023-08-25T09:43:53.402510+0000","last_fullsized":"2023-08-25T09:43:53.402510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:29.499043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.379405+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:43:53.379405+0000","last_peered":"2023-08-25T09:43:53.379405+0000","last_clean":"2023-08-25T09:43:53.379405+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:43:53.379405+0000","last_undegraded":"2023-08-25T09:43:53.379405+0000","last_fullsized":"2023-08-25T09:43:53.379405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:07:53.248356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385747+0000","last_change":"2023-08-25T09:43:53.385747+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385747+0000","last_undegraded":"2023-08-25T09:43:53.385747+0000","last_fullsized":"2023-08-25T09:43:53.385747+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403994+0000","last_change":"2023-08-25T09:43:53.403913+0000","last_active":"2023-08-25T09:43:53.403994+0000","last_peered":"2023-08-25T09:43:53.403994+0000","last_clean":"2023-08-25T09:43:53.403994+0000","last_became_active":"2023-08-25T09:43:53.403680+0000","last_became_peered":"2023-08-25T09:43:53.403680+0000","last_unstale":"2023-08-25T09:43:53.403994+0000","last_undegraded":"2023-08-25T09:43:53.403994+0000","last_fullsized":"2023-08-25T09:43:53.403994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:45.510377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.379835+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:43:53.379835+0000","last_peered":"2023-08-25T09:43:53.379835+0000","last_clean":"2023-08-25T09:43:53.379835+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:43:53.379835+0000","last_undegraded":"2023-08-25T09:43:53.379835+0000","last_fullsized":"2023-08-25T09:43:53.379835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:20.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.383791+0000","last_change":"2023-08-25T09:43:53.383791+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.383791+0000","last_undegraded":"2023-08-25T09:43:53.383791+0000","last_fullsized":"2023-08-25T09:43:53.383791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403168+0000","last_change":"2023-08-25T09:43:53.403077+0000","last_active":"2023-08-25T09:43:53.403168+0000","last_peered":"2023-08-25T09:43:53.403168+0000","last_clean":"2023-08-25T09:43:53.403168+0000","last_became_active":"2023-08-25T09:43:53.402353+0000","last_became_peered":"2023-08-25T09:43:53.402353+0000","last_unstale":"2023-08-25T09:43:53.403168+0000","last_undegraded":"2023-08-25T09:43:53.403168+0000","last_fullsized":"2023-08-25T09:43:53.403168+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:46:28.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.377620+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:43:53.377620+0000","last_peered":"2023-08-25T09:43:53.377620+0000","last_clean":"2023-08-25T09:43:53.377620+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:43:53.377620+0000","last_undegraded":"2023-08-25T09:43:53.377620+0000","last_fullsized":"2023-08-25T09:43:53.377620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:28:04.621813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394222+0000","last_change":"2023-08-25T09:43:53.394222+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394222+0000","last_undegraded":"2023-08-25T09:43:53.394222+0000","last_fullsized":"2023-08-25T09:43:53.394222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.388818+0000","last_change":"2023-08-25T09:43:53.388577+0000","last_active":"2023-08-25T09:43:53.388818+0000","last_peered":"2023-08-25T09:43:53.388818+0000","last_clean":"2023-08-25T09:43:53.388818+0000","last_became_active":"2023-08-25T09:43:53.387937+0000","last_became_peered":"2023-08-25T09:43:53.387937+0000","last_unstale":"2023-08-25T09:43:53.388818+0000","last_undegraded":"2023-08-25T09:43:53.388818+0000","last_fullsized":"2023-08-25T09:43:53.388818+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:26.108801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.390792+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:43:53.390792+0000","last_peered":"2023-08-25T09:43:53.390792+0000","last_clean":"2023-08-25T09:43:53.390792+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:43:53.390792+0000","last_undegraded":"2023-08-25T09:43:53.390792+0000","last_fullsized":"2023-08-25T09:43:53.390792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:57.748770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.381192+0000","last_change":"2023-08-25T09:43:53.381192+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.381192+0000","last_undegraded":"2023-08-25T09:43:53.381192+0000","last_fullsized":"2023-08-25T09:43:53.381192+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.404023+0000","last_change":"2023-08-25T09:43:53.403927+0000","last_active":"2023-08-25T09:43:53.404023+0000","last_peered":"2023-08-25T09:43:53.404023+0000","last_clean":"2023-08-25T09:43:53.404023+0000","last_became_active":"2023-08-25T09:43:53.402901+0000","last_became_peered":"2023-08-25T09:43:53.402901+0000","last_unstale":"2023-08-25T09:43:53.404023+0000","last_undegraded":"2023-08-25T09:43:53.404023+0000","last_fullsized":"2023-08-25T09:43:53.404023+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:23.214744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.379139+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:43:53.379139+0000","last_peered":"2023-08-25T09:43:53.379139+0000","last_clean":"2023-08-25T09:43:53.379139+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:43:53.379139+0000","last_undegraded":"2023-08-25T09:43:53.379139+0000","last_fullsized":"2023-08-25T09:43:53.379139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:07.210547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.381290+0000","last_change":"2023-08-25T09:43:53.381290+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.381290+0000","last_undegraded":"2023-08-25T09:43:53.381290+0000","last_fullsized":"2023-08-25T09:43:53.381290+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.402063+0000","last_change":"2023-08-25T09:43:53.401971+0000","last_active":"2023-08-25T09:43:53.402063+0000","last_peered":"2023-08-25T09:43:53.402063+0000","last_clean":"2023-08-25T09:43:53.402063+0000","last_became_active":"2023-08-25T09:43:53.401743+0000","last_became_peered":"2023-08-25T09:43:53.401743+0000","last_unstale":"2023-08-25T09:43:53.402063+0000","last_undegraded":"2023-08-25T09:43:53.402063+0000","last_fullsized":"2023-08-25T09:43:53.402063+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:36.470044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.386317+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:43:53.386317+0000","last_peered":"2023-08-25T09:43:53.386317+0000","last_clean":"2023-08-25T09:43:53.386317+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:43:53.386317+0000","last_undegraded":"2023-08-25T09:43:53.386317+0000","last_fullsized":"2023-08-25T09:43:53.386317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:35:09.047256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385995+0000","last_change":"2023-08-25T09:43:53.385995+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385995+0000","last_undegraded":"2023-08-25T09:43:53.385995+0000","last_fullsized":"2023-08-25T09:43:53.385995+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.400870+0000","last_change":"2023-08-25T09:43:53.400796+0000","last_active":"2023-08-25T09:43:53.400870+0000","last_peered":"2023-08-25T09:43:53.400870+0000","last_clean":"2023-08-25T09:43:53.400870+0000","last_became_active":"2023-08-25T09:43:53.400551+0000","last_became_peered":"2023-08-25T09:43:53.400551+0000","last_unstale":"2023-08-25T09:43:53.400870+0000","last_undegraded":"2023-08-25T09:43:53.400870+0000","last_fullsized":"2023-08-25T09:43:53.400870+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:36:43.484716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.377413+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:43:53.377413+0000","last_peered":"2023-08-25T09:43:53.377413+0000","last_clean":"2023-08-25T09:43:53.377413+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:43:53.377413+0000","last_undegraded":"2023-08-25T09:43:53.377413+0000","last_fullsized":"2023-08-25T09:43:53.377413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:19:37.873496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1910000000000001,"15min":1.0660000000000001},"min":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.54,"5min":10.747,"15min":10.747},"last":0.75},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.86899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1859999999999999,"15min":1.129},"min":{"1min":0.88,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1799999999999999,"15min":1.0920000000000001},"min":{"1min":0.76500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5820000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1559999999999999,"15min":1.079},"min":{"1min":0.75600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.24},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.27,"15min":1.2170000000000001},"min":{"1min":1.0620000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.129}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.81000000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.399},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3839999999999999,"15min":1.329},"min":{"1min":1.0629999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.466}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7689999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.286},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.286},"min":{"1min":1.05,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7609999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.5649999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.429,"15min":1.3740000000000001},"min":{"1min":1.04,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9830000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5209999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5069999999999999,"15min":1.4319999999999999},"min":{"1min":1.1439999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8740000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.472,"15min":1.3979999999999999},"min":{"1min":1.276,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.6890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.1399999999999999,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1910000000000001,"15min":1.145},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7649999999999999,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.3620000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2290000000000001,"15min":1.1950000000000001},"min":{"1min":0.79300000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.7050000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.1830000000000001,"15min":1.117},"min":{"1min":0.94799999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1659999999999999,"15min":1.105},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.482,"5min":1.708,"15min":1.708},"last":1.276},{"interface":"front","average":{"1min":1.409,"5min":1.3620000000000001,"15min":1.204},"min":{"1min":0.93400000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.768,"5min":13.193,"15min":13.193},"last":1.083}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.302,"15min":1.222},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.121,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.272,"15min":1.256},"min":{"1min":0.98099999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.548,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.5209999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.24},"min":{"1min":0.95999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.607,"5min":2.484,"15min":2.484},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.385,"15min":1.3169999999999999},"min":{"1min":0.89200000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.02,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.1299999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.2749999999999999},"min":{"1min":1.034,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.837,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.411},{"interface":"front","average":{"1min":1.532,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":1.1990000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7210000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.621}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.538,"15min":1.49},"min":{"1min":0.89600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.2240000000000002,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.472},"min":{"1min":1.244,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4710000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1087,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194225,"statfs":{"total":200000000,"available":198886752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1930000000000001,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.577,"5min":2.081,"15min":2.081},"last":1.139},{"interface":"front","average":{"1min":1.27,"5min":1.22,"15min":1.208},"min":{"1min":0.76200000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5469999999999999,"5min":1.974,"15min":1.974},"last":1.407}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.145,"15min":1.1950000000000001},"min":{"1min":0.746,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.577,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1819999999999999,"15min":1.119},"min":{"1min":0.997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5649999999999999,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.2150000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.238,"15min":1.1859999999999999},"min":{"1min":0.94899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.665,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":1.048,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6259999999999999,"5min":1.962,"15min":1.962},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4059999999999999,"15min":1.3160000000000001},"min":{"1min":1.0549999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3439999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":1.024,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.552,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.3500000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.355,"15min":1.3029999999999999},"min":{"1min":0.96199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7490000000000001,"5min":2.625,"15min":2.625},"last":1.044},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.3720000000000001},"min":{"1min":0.94499999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.857,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.5}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.4419999999999999,"15min":1.4330000000000001},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.77,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.512,"15min":1.417},"min":{"1min":1.177,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1059999999999999,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.579}]},{"osd":7,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.4770000000000001,"15min":1.4450000000000001},"min":{"1min":1.2869999999999999,"5min":0.63,"15min":0.63},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.587,"15min":1.579},"min":{"1min":1.4319999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.46}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1759999999999999,"15min":1.1499999999999999},"min":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5329999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.1459999999999999},"min":{"1min":1.026,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5289999999999999,"5min":7.367,"15min":7.367},"last":1.5449999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1359999999999999,"15min":1.1220000000000001},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.359},{"interface":"front","average":{"1min":1.173,"5min":1.244,"15min":1.401},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5249999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.492}]},{"osd":2,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.208,"15min":1.302},"min":{"1min":0.93200000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.244,"15min":1.167},"min":{"1min":0.95899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.631,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.5820000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.292,"15min":1.23},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7110000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.371},"min":{"1min":1.1439999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.778,"5min":1.911,"15min":1.911},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.86599999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.095},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3280000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.829,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.385}]},{"osd":6,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.423,"15min":1.4350000000000001},"min":{"1min":0.81100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0219999999999998,"5min":2.863,"15min":2.863},"last":1.444},{"interface":"front","average":{"1min":1.542,"5min":1.548,"15min":1.5600000000000001},"min":{"1min":0.97799999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9730000000000001,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.196}]},{"osd":7,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.55,"15min":1.4179999999999999},"min":{"1min":1.1910000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.593,"15min":1.595},"min":{"1min":1.3320000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1230000000000002,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.625}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.86699999999999999,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.95399999999999996,"15min":0.89800000000000002},"min":{"1min":0.65900000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5349999999999999,"5min":1.55,"15min":1.55},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.871,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.153,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.004,"15min":0.97699999999999998},"min":{"1min":0.58699999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4910000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.96199999999999997,"15min":0.87},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":1.573,"5min":1.919,"15min":1.919},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1020000000000001,"15min":1.1080000000000001},"min":{"1min":0.81000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6140000000000001,"5min":3.399,"15min":3.399},"last":1.3640000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.214,"15min":1.1890000000000001},"min":{"1min":0.83499999999999996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5800000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.196},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.405,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.089}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3120000000000001},"min":{"1min":0.98899999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6699999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.294},{"interface":"front","average":{"1min":1.355,"5min":1.2849999999999999,"15min":1.2689999999999999},"min":{"1min":0.92000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5980000000000001,"5min":1.788,"15min":1.788},"last":1.327}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.282,"15min":1.242},"min":{"1min":1.0389999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.917,"15min":1.917},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.397,"15min":1.405},"min":{"1min":1.2230000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.734,"5min":3.496,"15min":3.496},"last":1.24}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.085,"5min":0.871,"15min":0.871},"max":{"1min":1.881,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.446},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":0.98899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.669,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.2569999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.82499999999999996},"min":{"1min":0.53300000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.4199999999999999,"5min":2.383,"15min":2.383},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.425,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.411,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.238},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5649999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.284}]},{"osd":3,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5429999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.081,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.486,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.3220000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3080000000000001,"15min":1.397},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.1309999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.266,"15min":1.375},"min":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.369,"15min":1.397},"min":{"1min":1.081,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9890000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.444}]},{"osd":6,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.381,"15min":1.4199999999999999},"min":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.875,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.425},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.482},"min":{"1min":0.995,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.097,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.7070000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.383,"15min":1.4490000000000001},"min":{"1min":0.89600000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7829999999999999,"5min":2.968,"15min":2.968},"last":1.478},{"interface":"front","average":{"1min":1.409,"5min":1.4199999999999999,"15min":1.504},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.056,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.8200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1087,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194225,"statfs":{"total":200000000,"available":198886752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.77200000000000002},"min":{"1min":0.45900000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2250000000000001,"5min":1.627,"15min":1.627},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.876,"15min":0.78400000000000003},"min":{"1min":0.56799999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6850000000000001,"5min":1.895,"15min":1.895},"last":1.2849999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.79900000000000004},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.45,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.048,"5min":0.97499999999999998,"15min":0.90300000000000002},"min":{"1min":0.51500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4239999999999999,"5min":1.746,"15min":1.746},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1339999999999999,"15min":1.038},"min":{"1min":0.61199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.639,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.085,"5min":1.008,"15min":0.93999999999999995},"min":{"1min":0.67900000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4750000000000001,"5min":1.764,"15min":1.764},"last":1.2050000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.155,"15min":1.087},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.042},{"interface":"front","average":{"1min":1.369,"5min":1.3100000000000001,"15min":1.218},"min":{"1min":0.876,"5min":0.629,"15min":0.629},"max":{"1min":1.641,"5min":2.008,"15min":2.008},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1890000000000001},"min":{"1min":0.80000000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7909999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.077},{"interface":"front","average":{"1min":1.286,"5min":1.2170000000000001,"15min":1.1679999999999999},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.738,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3799999999999999,"15min":1.298},"min":{"1min":1.042,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9930000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.395},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.82199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8380000000000001,"5min":2.004,"15min":2.004},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":1.145,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7030000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.3899999999999999,"15min":1.3080000000000001},"min":{"1min":0.93500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9039999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.3480000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90700000000000003,"15min":0.85299999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.355,"5min":11.257,"15min":11.257},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.97099999999999997,"15min":0.95799999999999996},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7110000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.38800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.433},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0029999999999999,"15min":1.0940000000000001},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5640000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5609999999999999,"5min":1.869,"15min":1.869},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3340000000000001,"15min":1.353},"min":{"1min":0.83299999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7390000000000001,"5min":1.982,"15min":1.982},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.256,"15min":1.3580000000000001},"min":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8340000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.268,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.81100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6579999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.006},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.296},"min":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.752}]},{"osd":7,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3660000000000001,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0779999999999998,"5min":2.121,"15min":2.121},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.319},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.69399999999999995}]}]}],"pool_statfs":[]}} 2023-08-25T09:43:58.823 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:43:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.076+0000 7fe5720a7640 1 Processor -- start 2023-08-25T09:43:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.077+0000 7fe5720a7640 1 -- start start 2023-08-25T09:43:59.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.078+0000 7fe5720a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe56c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.078+0000 7fe5720a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe56c06b2b0 0x7fe56c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.078+0000 7fe5720a7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56c0b1b60 con 0x7fe56c15c6d0 2023-08-25T09:43:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.078+0000 7fe5720a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56c06bd80 con 0x7fe56c06b2b0 2023-08-25T09:43:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.078+0000 7fe5720a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c06b7c0 con 0x7fe56c15e830 2023-08-25T09:43:59.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe56c15c6d0 0x7fe56c0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39148/0 (socket says 172.21.15.70:39148) 2023-08-25T09:43:59.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56bfff640 1 -- 172.21.15.70:0/3271973612 learned_addr learned my addr 172.21.15.70:0/3271973612 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:43:59.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56b7fe640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe56c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026448733 0 0) 0x7fe56c0b1b60 con 0x7fe56c15c6d0 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe563fff640 1 --2- 172.21.15.70:0/3271973612 >> v2:172.21.15.155:3300/0 conn(0x7fe56c06b2b0 0x7fe56c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe558003520 con 0x7fe56c15c6d0 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2255298746 0 0) 0x7fe558003520 con 0x7fe56c15c6d0 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 msgr2=0x7fe56c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe56c06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 >> v2:172.21.15.155:3300/0 conn(0x7fe56c06b2b0 msgr2=0x7fe56c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 --2- 172.21.15.70:0/3271973612 >> v2:172.21.15.155:3300/0 conn(0x7fe56c06b2b0 0x7fe56c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c06baa0 con 0x7fe56c15c6d0 2023-08-25T09:43:59.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe5640019f0 con 0x7fe56c15c6d0 2023-08-25T09:43:59.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56b7fe640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe56c06ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:43:59.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.079+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 >> v1:172.21.15.70:6789/0 conn(0x7fe56c15c6d0 legacy=0x7fe56c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56affd640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558003970 0x7fe558003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56affd640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5580047c0 0x7fe558004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56affd640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe558005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c06bd80 con 0x7fe5580047c0 2023-08-25T09:43:59.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c06b7c0 con 0x7fe558003970 2023-08-25T09:43:59.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55800ba00 con 0x7fe56c15e830 2023-08-25T09:43:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 --2- 172.21.15.70:0/3271973612 >> v2:172.21.15.155:3300/0 conn(0x7fe56c06b2b0 0x7fe56c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:43:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5580047c0 0x7fe558004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56bfff640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe558005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 -- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 msgr2=0x7fe558005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe558005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 -- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558003970 msgr2=0x7fe558003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe558003970 0x7fe558003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe563fff640 1 -- 172.21.15.70:0/3271973612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe558003520 con 0x7fe5580047c0 2023-08-25T09:43:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.080+0000 7fe56bfff640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56c15e830 0x7fe558005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:43:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.081+0000 7fe563fff640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5580047c0 0x7fe558004c40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe56c06b6f0 tx=0x7fe5580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5835eeb918c10d8e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554030660 con 0x7fe5580047c0 2023-08-25T09:43:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5540307c0 con 0x7fe5580047c0 2023-08-25T09:43:59.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe56affd640 1 -- 172.21.15.70:0/3271973612 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554030b30 con 0x7fe5580047c0 2023-08-25T09:43:59.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe5720a7640 1 -- 172.21.15.70:0/3271973612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5580047c0 msgr2=0x7fe558004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe5720a7640 1 --2- 172.21.15.70:0/3271973612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5580047c0 0x7fe558004c40 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe56c06b6f0 tx=0x7fe5580057f0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe56bfff640 1 -- 172.21.15.70:0/3271973612 reap_dead start 2023-08-25T09:43:59.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe5720a7640 1 -- 172.21.15.70:0/3271973612 shutdown_connections 2023-08-25T09:43:59.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe5720a7640 1 -- 172.21.15.70:0/3271973612 >> 172.21.15.70:0/3271973612 conn(0x7fe56c069850 msgr2=0x7fe56c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:59.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.082+0000 7fe5720a7640 1 -- 172.21.15.70:0/3271973612 shutdown_connections 2023-08-25T09:43:59.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.083+0000 7fe5720a7640 1 -- 172.21.15.70:0/3271973612 wait complete. 2023-08-25T09:43:59.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.084+0000 7fe5720a7640 1 Processor -- start 2023-08-25T09:43:59.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.085+0000 7fe5720a7640 1 -- start start 2023-08-25T09:43:59.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.085+0000 7fe5720a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5580047c0 0x7fe56c1f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.085+0000 7fe5720a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 0x7fe56c153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe5720a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56c15d640 0x7fe56c15dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe5720a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c15e000 con 0x7fe56c1516a0 2023-08-25T09:43:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe5720a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c1527e0 con 0x7fe56c15d640 2023-08-25T09:43:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe5720a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56c1f3740 con 0x7fe5580047c0 2023-08-25T09:43:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 0x7fe56c153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 0x7fe56c153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45278/0 (socket says 172.21.15.70:45278) 2023-08-25T09:43:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 -- 172.21.15.70:0/90389035 learned_addr learned my addr 172.21.15.70:0/90389035 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe56b7fe640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5580047c0 0x7fe56c1f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe56bfff640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56c15d640 0x7fe56c15dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 -- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5580047c0 msgr2=0x7fe56c1f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5580047c0 0x7fe56c1f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 -- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56c15d640 msgr2=0x7fe56c15dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56c15d640 0x7fe56c15dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.086+0000 7fe563fff640 1 -- 172.21.15.70:0/90389035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c151bd0 con 0x7fe56c1516a0 2023-08-25T09:43:59.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.087+0000 7fe563fff640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 0x7fe56c153800 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe554009960 tx=0x7fe554004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:59.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.087+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540399a0 con 0x7fe56c1516a0 2023-08-25T09:43:59.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.087+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55403f040 con 0x7fe56c1516a0 2023-08-25T09:43:59.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.087+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56c151eb0 con 0x7fe56c1516a0 2023-08-25T09:43:59.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.087+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554039d60 con 0x7fe56c1516a0 2023-08-25T09:43:59.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.087+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56c152470 con 0x7fe56c1516a0 2023-08-25T09:43:59.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.089+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56c152630 con 0x7fe56c1516a0 2023-08-25T09:43:59.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.089+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe55404d460 con 0x7fe56c1516a0 2023-08-25T09:43:59.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.090+0000 7fe568ff9640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54c06a080 0x7fe54c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.090+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5540aae40 con 0x7fe56c1516a0 2023-08-25T09:43:59.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.090+0000 7fe56b7fe640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54c06a080 0x7fe54c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.091+0000 7fe56b7fe640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54c06a080 0x7fe54c06c540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006fd0 tx=0x7fe55c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:59.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.094+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe554076530 con 0x7fe56c1516a0 2023-08-25T09:43:59.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.279+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe56c15e000 con 0x7fe54c06a080 2023-08-25T09:43:59.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.302+0000 7fe568ff9640 1 -- 172.21.15.70:0/90389035 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297739 (secure 0 0 0) 0x7fe56c15e000 con 0x7fe54c06a080 2023-08-25T09:43:59.305 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:59.311 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:43:59.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54c06a080 msgr2=0x7fe54c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54c06a080 0x7fe54c06c540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006fd0 tx=0x7fe55c008040 comp rx=0 tx=0).stop 2023-08-25T09:43:59.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 msgr2=0x7fe56c153800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 0x7fe56c153800 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe554009960 tx=0x7fe554004730 comp rx=0 tx=0).stop 2023-08-25T09:43:59.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 shutdown_connections 2023-08-25T09:43:59.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54c06a080 0x7fe54c06c540 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56c15d640 0x7fe56c15dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe56c1516a0 0x7fe56c153800 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 --2- 172.21.15.70:0/90389035 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5580047c0 0x7fe56c1f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 >> 172.21.15.70:0/90389035 conn(0x7fe56c069850 msgr2=0x7fe56c0cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:59.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 shutdown_connections 2023-08-25T09:43:59.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.309+0000 7fe5720a7640 1 -- 172.21.15.70:0/90389035 wait complete. 2023-08-25T09:43:59.331 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2023-08-25T09:43:57.420426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5238,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557261,"statfs":{"total":1600000000,"available":1594636224,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060816"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394706+0000","last_change":"2023-08-25T09:43:53.394706+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394706+0000","last_undegraded":"2023-08-25T09:43:53.394706+0000","last_fullsized":"2023-08-25T09:43:53.394706+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388146+0000","last_change":"2023-08-25T09:43:53.388146+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388146+0000","last_undegraded":"2023-08-25T09:43:53.388146+0000","last_fullsized":"2023-08-25T09:43:53.388146+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390842+0000","last_change":"2023-08-25T09:43:53.390842+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390842+0000","last_undegraded":"2023-08-25T09:43:53.390842+0000","last_fullsized":"2023-08-25T09:43:53.390842+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388186+0000","last_change":"2023-08-25T09:43:53.388186+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388186+0000","last_undegraded":"2023-08-25T09:43:53.388186+0000","last_fullsized":"2023-08-25T09:43:53.388186+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.391681+0000","last_change":"2023-08-25T09:43:53.391681+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.391681+0000","last_undegraded":"2023-08-25T09:43:53.391681+0000","last_fullsized":"2023-08-25T09:43:53.391681+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393690+0000","last_change":"2023-08-25T09:43:53.393690+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393690+0000","last_undegraded":"2023-08-25T09:43:53.393690+0000","last_fullsized":"2023-08-25T09:43:53.393690+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388452+0000","last_change":"2023-08-25T09:43:53.388452+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388452+0000","last_undegraded":"2023-08-25T09:43:53.388452+0000","last_fullsized":"2023-08-25T09:43:53.388452+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.398637+0000","last_change":"2023-08-25T09:43:53.398637+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.398637+0000","last_undegraded":"2023-08-25T09:43:53.398637+0000","last_fullsized":"2023-08-25T09:43:53.398637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388519+0000","last_change":"2023-08-25T09:43:53.388519+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388519+0000","last_undegraded":"2023-08-25T09:43:53.388519+0000","last_fullsized":"2023-08-25T09:43:53.388519+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392240+0000","last_change":"2023-08-25T09:43:53.392240+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392240+0000","last_undegraded":"2023-08-25T09:43:53.392240+0000","last_fullsized":"2023-08-25T09:43:53.392240+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390689+0000","last_change":"2023-08-25T09:43:53.390689+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390689+0000","last_undegraded":"2023-08-25T09:43:53.390689+0000","last_fullsized":"2023-08-25T09:43:53.390689+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.384369+0000","last_change":"2023-08-25T09:43:53.384369+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.384369+0000","last_undegraded":"2023-08-25T09:43:53.384369+0000","last_fullsized":"2023-08-25T09:43:53.384369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390224+0000","last_change":"2023-08-25T09:43:53.390224+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390224+0000","last_undegraded":"2023-08-25T09:43:53.390224+0000","last_fullsized":"2023-08-25T09:43:53.390224+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.395201+0000","last_change":"2023-08-25T09:43:53.395201+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.395201+0000","last_undegraded":"2023-08-25T09:43:53.395201+0000","last_fullsized":"2023-08-25T09:43:53.395201+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.391206+0000","last_change":"2023-08-25T09:43:53.391206+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.391206+0000","last_undegraded":"2023-08-25T09:43:53.391206+0000","last_fullsized":"2023-08-25T09:43:53.391206+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386780+0000","last_change":"2023-08-25T09:43:53.386780+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386780+0000","last_undegraded":"2023-08-25T09:43:53.386780+0000","last_fullsized":"2023-08-25T09:43:53.386780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392750+0000","last_change":"2023-08-25T09:43:53.392750+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392750+0000","last_undegraded":"2023-08-25T09:43:53.392750+0000","last_fullsized":"2023-08-25T09:43:53.392750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389618+0000","last_change":"2023-08-25T09:43:53.389618+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389618+0000","last_undegraded":"2023-08-25T09:43:53.389618+0000","last_fullsized":"2023-08-25T09:43:53.389618+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394736+0000","last_change":"2023-08-25T09:43:53.394736+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394736+0000","last_undegraded":"2023-08-25T09:43:53.394736+0000","last_fullsized":"2023-08-25T09:43:53.394736+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393381+0000","last_change":"2023-08-25T09:43:53.393381+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393381+0000","last_undegraded":"2023-08-25T09:43:53.393381+0000","last_fullsized":"2023-08-25T09:43:53.393381+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382607+0000","last_change":"2023-08-25T09:43:53.382607+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382607+0000","last_undegraded":"2023-08-25T09:43:53.382607+0000","last_fullsized":"2023-08-25T09:43:53.382607+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387309+0000","last_change":"2023-08-25T09:43:53.387309+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387309+0000","last_undegraded":"2023-08-25T09:43:53.387309+0000","last_fullsized":"2023-08-25T09:43:53.387309+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392937+0000","last_change":"2023-08-25T09:43:53.392937+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392937+0000","last_undegraded":"2023-08-25T09:43:53.392937+0000","last_fullsized":"2023-08-25T09:43:53.392937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385729+0000","last_change":"2023-08-25T09:43:53.385729+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385729+0000","last_undegraded":"2023-08-25T09:43:53.385729+0000","last_fullsized":"2023-08-25T09:43:53.385729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388736+0000","last_change":"2023-08-25T09:43:53.388736+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388736+0000","last_undegraded":"2023-08-25T09:43:53.388736+0000","last_fullsized":"2023-08-25T09:43:53.388736+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389631+0000","last_change":"2023-08-25T09:43:53.389631+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389631+0000","last_undegraded":"2023-08-25T09:43:53.389631+0000","last_fullsized":"2023-08-25T09:43:53.389631+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387236+0000","last_change":"2023-08-25T09:43:53.387236+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387236+0000","last_undegraded":"2023-08-25T09:43:53.387236+0000","last_fullsized":"2023-08-25T09:43:53.387236+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393788+0000","last_change":"2023-08-25T09:43:53.393788+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393788+0000","last_undegraded":"2023-08-25T09:43:53.393788+0000","last_fullsized":"2023-08-25T09:43:53.393788+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.393225+0000","last_change":"2023-08-25T09:43:53.393225+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.393225+0000","last_undegraded":"2023-08-25T09:43:53.393225+0000","last_fullsized":"2023-08-25T09:43:53.393225+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392835+0000","last_change":"2023-08-25T09:43:53.392835+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392835+0000","last_undegraded":"2023-08-25T09:43:53.392835+0000","last_fullsized":"2023-08-25T09:43:53.392835+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390133+0000","last_change":"2023-08-25T09:43:53.390133+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390133+0000","last_undegraded":"2023-08-25T09:43:53.390133+0000","last_fullsized":"2023-08-25T09:43:53.390133+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386447+0000","last_change":"2023-08-25T09:43:53.386447+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386447+0000","last_undegraded":"2023-08-25T09:43:53.386447+0000","last_fullsized":"2023-08-25T09:43:53.386447+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.390654+0000","last_change":"2023-08-25T09:43:53.390654+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.390654+0000","last_undegraded":"2023-08-25T09:43:53.390654+0000","last_fullsized":"2023-08-25T09:43:53.390654+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.398618+0000","last_change":"2023-08-25T09:43:53.397210+0000","last_active":"2023-08-25T09:43:53.398618+0000","last_peered":"2023-08-25T09:43:53.398618+0000","last_clean":"2023-08-25T09:43:53.398618+0000","last_became_active":"2023-08-25T09:43:53.396946+0000","last_became_peered":"2023-08-25T09:43:53.396946+0000","last_unstale":"2023-08-25T09:43:53.398618+0000","last_undegraded":"2023-08-25T09:43:53.398618+0000","last_fullsized":"2023-08-25T09:43:53.398618+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:30.659547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382695+0000","last_change":"2023-08-25T09:43:53.382695+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382695+0000","last_undegraded":"2023-08-25T09:43:53.382695+0000","last_fullsized":"2023-08-25T09:43:53.382695+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.393543+0000","last_change":"2023-08-25T09:43:53.393077+0000","last_active":"2023-08-25T09:43:53.393543+0000","last_peered":"2023-08-25T09:43:53.393543+0000","last_clean":"2023-08-25T09:43:53.393543+0000","last_became_active":"2023-08-25T09:43:53.392575+0000","last_became_peered":"2023-08-25T09:43:53.392575+0000","last_unstale":"2023-08-25T09:43:53.393543+0000","last_undegraded":"2023-08-25T09:43:53.393543+0000","last_fullsized":"2023-08-25T09:43:53.393543+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:28.378444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382523+0000","last_change":"2023-08-25T09:43:53.382523+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382523+0000","last_undegraded":"2023-08-25T09:43:53.382523+0000","last_fullsized":"2023-08-25T09:43:53.382523+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399026+0000","last_change":"2023-08-25T09:43:53.398957+0000","last_active":"2023-08-25T09:43:53.399026+0000","last_peered":"2023-08-25T09:43:53.399026+0000","last_clean":"2023-08-25T09:43:53.399026+0000","last_became_active":"2023-08-25T09:43:53.398717+0000","last_became_peered":"2023-08-25T09:43:53.398717+0000","last_unstale":"2023-08-25T09:43:53.399026+0000","last_undegraded":"2023-08-25T09:43:53.399026+0000","last_fullsized":"2023-08-25T09:43:53.399026+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:01:47.062976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387825+0000","last_change":"2023-08-25T09:43:53.387825+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387825+0000","last_undegraded":"2023-08-25T09:43:53.387825+0000","last_fullsized":"2023-08-25T09:43:53.387825+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.389650+0000","last_change":"2023-08-25T09:43:53.389534+0000","last_active":"2023-08-25T09:43:53.389650+0000","last_peered":"2023-08-25T09:43:53.389650+0000","last_clean":"2023-08-25T09:43:53.389650+0000","last_became_active":"2023-08-25T09:43:53.389145+0000","last_became_peered":"2023-08-25T09:43:53.389145+0000","last_unstale":"2023-08-25T09:43:53.389650+0000","last_undegraded":"2023-08-25T09:43:53.389650+0000","last_fullsized":"2023-08-25T09:43:53.389650+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:37:26.224049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389464+0000","last_change":"2023-08-25T09:43:53.389464+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389464+0000","last_undegraded":"2023-08-25T09:43:53.389464+0000","last_fullsized":"2023-08-25T09:43:53.389464+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.385290+0000","last_change":"2023-08-25T09:43:53.385163+0000","last_active":"2023-08-25T09:43:53.385290+0000","last_peered":"2023-08-25T09:43:53.385290+0000","last_clean":"2023-08-25T09:43:53.385290+0000","last_became_active":"2023-08-25T09:43:53.384839+0000","last_became_peered":"2023-08-25T09:43:53.384839+0000","last_unstale":"2023-08-25T09:43:53.385290+0000","last_undegraded":"2023-08-25T09:43:53.385290+0000","last_fullsized":"2023-08-25T09:43:53.385290+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:29:38.869803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386780+0000","last_change":"2023-08-25T09:43:53.386780+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386780+0000","last_undegraded":"2023-08-25T09:43:53.386780+0000","last_fullsized":"2023-08-25T09:43:53.386780+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401260+0000","last_change":"2023-08-25T09:43:53.401154+0000","last_active":"2023-08-25T09:43:53.401260+0000","last_peered":"2023-08-25T09:43:53.401260+0000","last_clean":"2023-08-25T09:43:53.401260+0000","last_became_active":"2023-08-25T09:43:53.400830+0000","last_became_peered":"2023-08-25T09:43:53.400830+0000","last_unstale":"2023-08-25T09:43:53.401260+0000","last_undegraded":"2023-08-25T09:43:53.401260+0000","last_fullsized":"2023-08-25T09:43:53.401260+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:43.181749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382181+0000","last_change":"2023-08-25T09:43:53.382181+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382181+0000","last_undegraded":"2023-08-25T09:43:53.382181+0000","last_fullsized":"2023-08-25T09:43:53.382181+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401536+0000","last_change":"2023-08-25T09:43:53.401459+0000","last_active":"2023-08-25T09:43:53.401536+0000","last_peered":"2023-08-25T09:43:53.401536+0000","last_clean":"2023-08-25T09:43:53.401536+0000","last_became_active":"2023-08-25T09:43:53.400825+0000","last_became_peered":"2023-08-25T09:43:53.400825+0000","last_unstale":"2023-08-25T09:43:53.401536+0000","last_undegraded":"2023-08-25T09:43:53.401536+0000","last_fullsized":"2023-08-25T09:43:53.401536+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:03:32.834869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385987+0000","last_change":"2023-08-25T09:43:53.385987+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385987+0000","last_undegraded":"2023-08-25T09:43:53.385987+0000","last_fullsized":"2023-08-25T09:43:53.385987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399517+0000","last_change":"2023-08-25T09:43:53.399348+0000","last_active":"2023-08-25T09:43:53.399517+0000","last_peered":"2023-08-25T09:43:53.399517+0000","last_clean":"2023-08-25T09:43:53.399517+0000","last_became_active":"2023-08-25T09:43:53.399099+0000","last_became_peered":"2023-08-25T09:43:53.399099+0000","last_unstale":"2023-08-25T09:43:53.399517+0000","last_undegraded":"2023-08-25T09:43:53.399517+0000","last_fullsized":"2023-08-25T09:43:53.399517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:28:28.010065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.389030+0000","last_change":"2023-08-25T09:43:53.389030+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.389030+0000","last_undegraded":"2023-08-25T09:43:53.389030+0000","last_fullsized":"2023-08-25T09:43:53.389030+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.398019+0000","last_change":"2023-08-25T09:43:53.397375+0000","last_active":"2023-08-25T09:43:53.398019+0000","last_peered":"2023-08-25T09:43:53.398019+0000","last_clean":"2023-08-25T09:43:53.398019+0000","last_became_active":"2023-08-25T09:43:53.397119+0000","last_became_peered":"2023-08-25T09:43:53.397119+0000","last_unstale":"2023-08-25T09:43:53.398019+0000","last_undegraded":"2023-08-25T09:43:53.398019+0000","last_fullsized":"2023-08-25T09:43:53.398019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:12:32.796450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.383399+0000","last_change":"2023-08-25T09:43:53.383399+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.383399+0000","last_undegraded":"2023-08-25T09:43:53.383399+0000","last_fullsized":"2023-08-25T09:43:53.383399+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399347+0000","last_change":"2023-08-25T09:43:53.399279+0000","last_active":"2023-08-25T09:43:53.399347+0000","last_peered":"2023-08-25T09:43:53.399347+0000","last_clean":"2023-08-25T09:43:53.399347+0000","last_became_active":"2023-08-25T09:43:53.399023+0000","last_became_peered":"2023-08-25T09:43:53.399023+0000","last_unstale":"2023-08-25T09:43:53.399347+0000","last_undegraded":"2023-08-25T09:43:53.399347+0000","last_fullsized":"2023-08-25T09:43:53.399347+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:56:44.165537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.396739+0000","last_change":"2023-08-25T09:43:53.396739+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.396739+0000","last_undegraded":"2023-08-25T09:43:53.396739+0000","last_fullsized":"2023-08-25T09:43:53.396739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401575+0000","last_change":"2023-08-25T09:43:53.400335+0000","last_active":"2023-08-25T09:43:53.401575+0000","last_peered":"2023-08-25T09:43:53.401575+0000","last_clean":"2023-08-25T09:43:53.401575+0000","last_became_active":"2023-08-25T09:43:53.397372+0000","last_became_peered":"2023-08-25T09:43:53.397372+0000","last_unstale":"2023-08-25T09:43:53.401575+0000","last_undegraded":"2023-08-25T09:43:53.401575+0000","last_fullsized":"2023-08-25T09:43:53.401575+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:14.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387953+0000","last_change":"2023-08-25T09:43:53.387953+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387953+0000","last_undegraded":"2023-08-25T09:43:53.387953+0000","last_fullsized":"2023-08-25T09:43:53.387953+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399436+0000","last_change":"2023-08-25T09:43:53.399357+0000","last_active":"2023-08-25T09:43:53.399436+0000","last_peered":"2023-08-25T09:43:53.399436+0000","last_clean":"2023-08-25T09:43:53.399436+0000","last_became_active":"2023-08-25T09:43:53.398803+0000","last_became_peered":"2023-08-25T09:43:53.398803+0000","last_unstale":"2023-08-25T09:43:53.399436+0000","last_undegraded":"2023-08-25T09:43:53.399436+0000","last_fullsized":"2023-08-25T09:43:53.399436+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:42.746759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386020+0000","last_change":"2023-08-25T09:43:53.386020+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386020+0000","last_undegraded":"2023-08-25T09:43:53.386020+0000","last_fullsized":"2023-08-25T09:43:53.386020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.400300+0000","last_change":"2023-08-25T09:43:53.400186+0000","last_active":"2023-08-25T09:43:53.400300+0000","last_peered":"2023-08-25T09:43:53.400300+0000","last_clean":"2023-08-25T09:43:53.400300+0000","last_became_active":"2023-08-25T09:43:53.399735+0000","last_became_peered":"2023-08-25T09:43:53.399735+0000","last_unstale":"2023-08-25T09:43:53.400300+0000","last_undegraded":"2023-08-25T09:43:53.400300+0000","last_fullsized":"2023-08-25T09:43:53.400300+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:26:44.320129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.392484+0000","last_change":"2023-08-25T09:43:53.392484+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.392484+0000","last_undegraded":"2023-08-25T09:43:53.392484+0000","last_fullsized":"2023-08-25T09:43:53.392484+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.394286+0000","last_change":"2023-08-25T09:43:53.394175+0000","last_active":"2023-08-25T09:43:53.394286+0000","last_peered":"2023-08-25T09:43:53.394286+0000","last_clean":"2023-08-25T09:43:53.394286+0000","last_became_active":"2023-08-25T09:43:53.393635+0000","last_became_peered":"2023-08-25T09:43:53.393635+0000","last_unstale":"2023-08-25T09:43:53.394286+0000","last_undegraded":"2023-08-25T09:43:53.394286+0000","last_fullsized":"2023-08-25T09:43:53.394286+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:42:31.522547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.383906+0000","last_change":"2023-08-25T09:43:53.383906+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.383906+0000","last_undegraded":"2023-08-25T09:43:53.383906+0000","last_fullsized":"2023-08-25T09:43:53.383906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403254+0000","last_change":"2023-08-25T09:43:53.403154+0000","last_active":"2023-08-25T09:43:53.403254+0000","last_peered":"2023-08-25T09:43:53.403254+0000","last_clean":"2023-08-25T09:43:53.403254+0000","last_became_active":"2023-08-25T09:43:53.399755+0000","last_became_peered":"2023-08-25T09:43:53.399755+0000","last_unstale":"2023-08-25T09:43:53.403254+0000","last_undegraded":"2023-08-25T09:43:53.403254+0000","last_fullsized":"2023-08-25T09:43:53.403254+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:43:09.183840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394273+0000","last_change":"2023-08-25T09:43:53.394273+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394273+0000","last_undegraded":"2023-08-25T09:43:53.394273+0000","last_fullsized":"2023-08-25T09:43:53.394273+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403433+0000","last_change":"2023-08-25T09:43:53.403310+0000","last_active":"2023-08-25T09:43:53.403433+0000","last_peered":"2023-08-25T09:43:53.403433+0000","last_clean":"2023-08-25T09:43:53.403433+0000","last_became_active":"2023-08-25T09:43:53.400976+0000","last_became_peered":"2023-08-25T09:43:53.400976+0000","last_unstale":"2023-08-25T09:43:53.403433+0000","last_undegraded":"2023-08-25T09:43:53.403433+0000","last_fullsized":"2023-08-25T09:43:53.403433+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:42:08.889550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387526+0000","last_change":"2023-08-25T09:43:53.387526+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387526+0000","last_undegraded":"2023-08-25T09:43:53.387526+0000","last_fullsized":"2023-08-25T09:43:53.387526+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.400319+0000","last_change":"2023-08-25T09:43:53.400229+0000","last_active":"2023-08-25T09:43:53.400319+0000","last_peered":"2023-08-25T09:43:53.400319+0000","last_clean":"2023-08-25T09:43:53.400319+0000","last_became_active":"2023-08-25T09:43:53.400007+0000","last_became_peered":"2023-08-25T09:43:53.400007+0000","last_unstale":"2023-08-25T09:43:53.400319+0000","last_undegraded":"2023-08-25T09:43:53.400319+0000","last_fullsized":"2023-08-25T09:43:53.400319+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:28.772786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.391936+0000","last_change":"2023-08-25T09:43:53.391936+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.391936+0000","last_undegraded":"2023-08-25T09:43:53.391936+0000","last_fullsized":"2023-08-25T09:43:53.391936+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401594+0000","last_change":"2023-08-25T09:43:53.400677+0000","last_active":"2023-08-25T09:43:53.401594+0000","last_peered":"2023-08-25T09:43:53.401594+0000","last_clean":"2023-08-25T09:43:53.401594+0000","last_became_active":"2023-08-25T09:43:53.397908+0000","last_became_peered":"2023-08-25T09:43:53.397908+0000","last_unstale":"2023-08-25T09:43:53.401594+0000","last_undegraded":"2023-08-25T09:43:53.401594+0000","last_fullsized":"2023-08-25T09:43:53.401594+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:32.103986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.380835+0000","last_change":"2023-08-25T09:43:53.380835+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.380835+0000","last_undegraded":"2023-08-25T09:43:53.380835+0000","last_fullsized":"2023-08-25T09:43:53.380835+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.397632+0000","last_change":"2023-08-25T09:43:53.397556+0000","last_active":"2023-08-25T09:43:53.397632+0000","last_peered":"2023-08-25T09:43:53.397632+0000","last_clean":"2023-08-25T09:43:53.397632+0000","last_became_active":"2023-08-25T09:43:53.397131+0000","last_became_peered":"2023-08-25T09:43:53.397131+0000","last_unstale":"2023-08-25T09:43:53.397632+0000","last_undegraded":"2023-08-25T09:43:53.397632+0000","last_fullsized":"2023-08-25T09:43:53.397632+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:06:19.290929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.396298+0000","last_change":"2023-08-25T09:43:53.396298+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.396298+0000","last_undegraded":"2023-08-25T09:43:53.396298+0000","last_fullsized":"2023-08-25T09:43:53.396298+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.390553+0000","last_change":"2023-08-25T09:43:53.390461+0000","last_active":"2023-08-25T09:43:53.390553+0000","last_peered":"2023-08-25T09:43:53.390553+0000","last_clean":"2023-08-25T09:43:53.390553+0000","last_became_active":"2023-08-25T09:43:53.390304+0000","last_became_peered":"2023-08-25T09:43:53.390304+0000","last_unstale":"2023-08-25T09:43:53.390553+0000","last_undegraded":"2023-08-25T09:43:53.390553+0000","last_fullsized":"2023-08-25T09:43:53.390553+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:12.289326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.380931+0000","last_change":"2023-08-25T09:43:53.380931+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.380931+0000","last_undegraded":"2023-08-25T09:43:53.380931+0000","last_fullsized":"2023-08-25T09:43:53.380931+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.395273+0000","last_change":"2023-08-25T09:43:53.394719+0000","last_active":"2023-08-25T09:43:53.395273+0000","last_peered":"2023-08-25T09:43:53.395273+0000","last_clean":"2023-08-25T09:43:53.395273+0000","last_became_active":"2023-08-25T09:43:53.394231+0000","last_became_peered":"2023-08-25T09:43:53.394231+0000","last_unstale":"2023-08-25T09:43:53.395273+0000","last_undegraded":"2023-08-25T09:43:53.395273+0000","last_fullsized":"2023-08-25T09:43:53.395273+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:10:59.580451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.387859+0000","last_change":"2023-08-25T09:43:53.387859+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.387859+0000","last_undegraded":"2023-08-25T09:43:53.387859+0000","last_fullsized":"2023-08-25T09:43:53.387859+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.401611+0000","last_change":"2023-08-25T09:43:53.401366+0000","last_active":"2023-08-25T09:43:53.401611+0000","last_peered":"2023-08-25T09:43:53.401611+0000","last_clean":"2023-08-25T09:43:53.401611+0000","last_became_active":"2023-08-25T09:43:53.400954+0000","last_became_peered":"2023-08-25T09:43:53.400954+0000","last_unstale":"2023-08-25T09:43:53.401611+0000","last_undegraded":"2023-08-25T09:43:53.401611+0000","last_fullsized":"2023-08-25T09:43:53.401611+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:29.214752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.388434+0000","last_change":"2023-08-25T09:43:53.388434+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.388434+0000","last_undegraded":"2023-08-25T09:43:53.388434+0000","last_fullsized":"2023-08-25T09:43:53.388434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.397194+0000","last_change":"2023-08-25T09:43:53.396976+0000","last_active":"2023-08-25T09:43:53.397194+0000","last_peered":"2023-08-25T09:43:53.397194+0000","last_clean":"2023-08-25T09:43:53.397194+0000","last_became_active":"2023-08-25T09:43:53.396705+0000","last_became_peered":"2023-08-25T09:43:53.396705+0000","last_unstale":"2023-08-25T09:43:53.397194+0000","last_undegraded":"2023-08-25T09:43:53.397194+0000","last_fullsized":"2023-08-25T09:43:53.397194+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:54:31.933639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.382822+0000","last_change":"2023-08-25T09:43:53.382822+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.382822+0000","last_undegraded":"2023-08-25T09:43:53.382822+0000","last_fullsized":"2023-08-25T09:43:53.382822+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.399494+0000","last_change":"2023-08-25T09:43:53.398900+0000","last_active":"2023-08-25T09:43:53.399494+0000","last_peered":"2023-08-25T09:43:53.399494+0000","last_clean":"2023-08-25T09:43:53.399494+0000","last_became_active":"2023-08-25T09:43:53.398452+0000","last_became_peered":"2023-08-25T09:43:53.398452+0000","last_unstale":"2023-08-25T09:43:53.399494+0000","last_undegraded":"2023-08-25T09:43:53.399494+0000","last_fullsized":"2023-08-25T09:43:53.399494+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:42.580732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.386971+0000","last_change":"2023-08-25T09:43:53.386971+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.386971+0000","last_undegraded":"2023-08-25T09:43:53.386971+0000","last_fullsized":"2023-08-25T09:43:53.386971+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.396058+0000","last_change":"2023-08-25T09:43:53.395918+0000","last_active":"2023-08-25T09:43:53.396058+0000","last_peered":"2023-08-25T09:43:53.396058+0000","last_clean":"2023-08-25T09:43:53.396058+0000","last_became_active":"2023-08-25T09:43:53.395232+0000","last_became_peered":"2023-08-25T09:43:53.395232+0000","last_unstale":"2023-08-25T09:43:53.396058+0000","last_undegraded":"2023-08-25T09:43:53.396058+0000","last_fullsized":"2023-08-25T09:43:53.396058+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:50.827642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":289,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.392570+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:43:53.392570+0000","last_peered":"2023-08-25T09:43:53.392570+0000","last_clean":"2023-08-25T09:43:53.392570+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:43:53.392570+0000","last_undegraded":"2023-08-25T09:43:53.392570+0000","last_fullsized":"2023-08-25T09:43:53.392570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:14.142319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.378409+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:43:53.378409+0000","last_peered":"2023-08-25T09:43:53.378409+0000","last_clean":"2023-08-25T09:43:53.378409+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:43:53.378409+0000","last_undegraded":"2023-08-25T09:43:53.378409+0000","last_fullsized":"2023-08-25T09:43:53.378409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:04:54.199079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.381251+0000","last_change":"2023-08-25T09:43:53.381251+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.381251+0000","last_undegraded":"2023-08-25T09:43:53.381251+0000","last_fullsized":"2023-08-25T09:43:53.381251+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.402510+0000","last_change":"2023-08-25T09:43:53.402444+0000","last_active":"2023-08-25T09:43:53.402510+0000","last_peered":"2023-08-25T09:43:53.402510+0000","last_clean":"2023-08-25T09:43:53.402510+0000","last_became_active":"2023-08-25T09:43:53.402171+0000","last_became_peered":"2023-08-25T09:43:53.402171+0000","last_unstale":"2023-08-25T09:43:53.402510+0000","last_undegraded":"2023-08-25T09:43:53.402510+0000","last_fullsized":"2023-08-25T09:43:53.402510+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:29.499043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.379405+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:43:53.379405+0000","last_peered":"2023-08-25T09:43:53.379405+0000","last_clean":"2023-08-25T09:43:53.379405+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:43:53.379405+0000","last_undegraded":"2023-08-25T09:43:53.379405+0000","last_fullsized":"2023-08-25T09:43:53.379405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:07:53.248356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385747+0000","last_change":"2023-08-25T09:43:53.385747+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385747+0000","last_undegraded":"2023-08-25T09:43:53.385747+0000","last_fullsized":"2023-08-25T09:43:53.385747+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403994+0000","last_change":"2023-08-25T09:43:53.403913+0000","last_active":"2023-08-25T09:43:53.403994+0000","last_peered":"2023-08-25T09:43:53.403994+0000","last_clean":"2023-08-25T09:43:53.403994+0000","last_became_active":"2023-08-25T09:43:53.403680+0000","last_became_peered":"2023-08-25T09:43:53.403680+0000","last_unstale":"2023-08-25T09:43:53.403994+0000","last_undegraded":"2023-08-25T09:43:53.403994+0000","last_fullsized":"2023-08-25T09:43:53.403994+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:45.510377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":157,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.379835+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:43:53.379835+0000","last_peered":"2023-08-25T09:43:53.379835+0000","last_clean":"2023-08-25T09:43:53.379835+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:43:53.379835+0000","last_undegraded":"2023-08-25T09:43:53.379835+0000","last_fullsized":"2023-08-25T09:43:53.379835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:54:20.222500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.383791+0000","last_change":"2023-08-25T09:43:53.383791+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.383791+0000","last_undegraded":"2023-08-25T09:43:53.383791+0000","last_fullsized":"2023-08-25T09:43:53.383791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.403168+0000","last_change":"2023-08-25T09:43:53.403077+0000","last_active":"2023-08-25T09:43:53.403168+0000","last_peered":"2023-08-25T09:43:53.403168+0000","last_clean":"2023-08-25T09:43:53.403168+0000","last_became_active":"2023-08-25T09:43:53.402353+0000","last_became_peered":"2023-08-25T09:43:53.402353+0000","last_unstale":"2023-08-25T09:43:53.403168+0000","last_undegraded":"2023-08-25T09:43:53.403168+0000","last_fullsized":"2023-08-25T09:43:53.403168+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:46:28.753997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.377620+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:43:53.377620+0000","last_peered":"2023-08-25T09:43:53.377620+0000","last_clean":"2023-08-25T09:43:53.377620+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:43:53.377620+0000","last_undegraded":"2023-08-25T09:43:53.377620+0000","last_fullsized":"2023-08-25T09:43:53.377620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:28:04.621813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.394222+0000","last_change":"2023-08-25T09:43:53.394222+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.394222+0000","last_undegraded":"2023-08-25T09:43:53.394222+0000","last_fullsized":"2023-08-25T09:43:53.394222+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.388818+0000","last_change":"2023-08-25T09:43:53.388577+0000","last_active":"2023-08-25T09:43:53.388818+0000","last_peered":"2023-08-25T09:43:53.388818+0000","last_clean":"2023-08-25T09:43:53.388818+0000","last_became_active":"2023-08-25T09:43:53.387937+0000","last_became_peered":"2023-08-25T09:43:53.387937+0000","last_unstale":"2023-08-25T09:43:53.388818+0000","last_undegraded":"2023-08-25T09:43:53.388818+0000","last_fullsized":"2023-08-25T09:43:53.388818+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:26.108801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":137,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.390792+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:43:53.390792+0000","last_peered":"2023-08-25T09:43:53.390792+0000","last_clean":"2023-08-25T09:43:53.390792+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:43:53.390792+0000","last_undegraded":"2023-08-25T09:43:53.390792+0000","last_fullsized":"2023-08-25T09:43:53.390792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:57.748770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.381192+0000","last_change":"2023-08-25T09:43:53.381192+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.381192+0000","last_undegraded":"2023-08-25T09:43:53.381192+0000","last_fullsized":"2023-08-25T09:43:53.381192+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.404023+0000","last_change":"2023-08-25T09:43:53.403927+0000","last_active":"2023-08-25T09:43:53.404023+0000","last_peered":"2023-08-25T09:43:53.404023+0000","last_clean":"2023-08-25T09:43:53.404023+0000","last_became_active":"2023-08-25T09:43:53.402901+0000","last_became_peered":"2023-08-25T09:43:53.402901+0000","last_unstale":"2023-08-25T09:43:53.404023+0000","last_undegraded":"2023-08-25T09:43:53.404023+0000","last_fullsized":"2023-08-25T09:43:53.404023+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:23.214744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.379139+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:43:53.379139+0000","last_peered":"2023-08-25T09:43:53.379139+0000","last_clean":"2023-08-25T09:43:53.379139+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:43:53.379139+0000","last_undegraded":"2023-08-25T09:43:53.379139+0000","last_fullsized":"2023-08-25T09:43:53.379139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:07.210547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.381290+0000","last_change":"2023-08-25T09:43:53.381290+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.381290+0000","last_undegraded":"2023-08-25T09:43:53.381290+0000","last_fullsized":"2023-08-25T09:43:53.381290+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.402063+0000","last_change":"2023-08-25T09:43:53.401971+0000","last_active":"2023-08-25T09:43:53.402063+0000","last_peered":"2023-08-25T09:43:53.402063+0000","last_clean":"2023-08-25T09:43:53.402063+0000","last_became_active":"2023-08-25T09:43:53.401743+0000","last_became_peered":"2023-08-25T09:43:53.401743+0000","last_unstale":"2023-08-25T09:43:53.402063+0000","last_undegraded":"2023-08-25T09:43:53.402063+0000","last_fullsized":"2023-08-25T09:43:53.402063+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:36.470044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.386317+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:43:53.386317+0000","last_peered":"2023-08-25T09:43:53.386317+0000","last_clean":"2023-08-25T09:43:53.386317+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:43:53.386317+0000","last_undegraded":"2023-08-25T09:43:53.386317+0000","last_fullsized":"2023-08-25T09:43:53.386317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:35:09.047256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":92,"state":"creating+peering","last_fresh":"2023-08-25T09:43:53.385995+0000","last_change":"2023-08-25T09:43:53.385995+0000","last_active":"2023-08-25T09:43:53.359358+0000","last_peered":"2023-08-25T09:43:53.359358+0000","last_clean":"2023-08-25T09:43:53.359358+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:43:53.385995+0000","last_undegraded":"2023-08-25T09:43:53.385995+0000","last_fullsized":"2023-08-25T09:43:53.385995+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.400870+0000","last_change":"2023-08-25T09:43:53.400796+0000","last_active":"2023-08-25T09:43:53.400870+0000","last_peered":"2023-08-25T09:43:53.400870+0000","last_clean":"2023-08-25T09:43:53.400870+0000","last_became_active":"2023-08-25T09:43:53.400551+0000","last_became_peered":"2023-08-25T09:43:53.400551+0000","last_unstale":"2023-08-25T09:43:53.400870+0000","last_undegraded":"2023-08-25T09:43:53.400870+0000","last_fullsized":"2023-08-25T09:43:53.400870+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:36:43.484716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-08-25T09:43:53.377413+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:43:53.377413+0000","last_peered":"2023-08-25T09:43:53.377413+0000","last_clean":"2023-08-25T09:43:53.377413+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:43:53.377413+0000","last_undegraded":"2023-08-25T09:43:53.377413+0000","last_fullsized":"2023-08-25T09:43:53.377413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:19:37.873496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1910000000000001,"15min":1.0660000000000001},"min":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.54,"5min":10.747,"15min":10.747},"last":0.75},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.86899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1859999999999999,"15min":1.129},"min":{"1min":0.88,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1799999999999999,"15min":1.0920000000000001},"min":{"1min":0.76500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5820000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1559999999999999,"15min":1.079},"min":{"1min":0.75600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.24},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.27,"15min":1.2170000000000001},"min":{"1min":1.0620000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.129}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.81000000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.399},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3839999999999999,"15min":1.329},"min":{"1min":1.0629999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.466}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7689999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.286},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.286},"min":{"1min":1.05,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7609999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.5649999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.429,"15min":1.3740000000000001},"min":{"1min":1.04,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9830000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.5209999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5069999999999999,"15min":1.4319999999999999},"min":{"1min":1.1439999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8740000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.472,"15min":1.3979999999999999},"min":{"1min":1.276,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.6890000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.1399999999999999,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1910000000000001,"15min":1.145},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7649999999999999,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.3620000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2290000000000001,"15min":1.1950000000000001},"min":{"1min":0.79300000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.7050000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.1830000000000001,"15min":1.117},"min":{"1min":0.94799999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1659999999999999,"15min":1.105},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.482,"5min":1.708,"15min":1.708},"last":1.276},{"interface":"front","average":{"1min":1.409,"5min":1.3620000000000001,"15min":1.204},"min":{"1min":0.93400000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.768,"5min":13.193,"15min":13.193},"last":1.083}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.302,"15min":1.222},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.121,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.272,"15min":1.256},"min":{"1min":0.98099999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.548,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.5209999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.24},"min":{"1min":0.95999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.607,"5min":2.484,"15min":2.484},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.385,"15min":1.3169999999999999},"min":{"1min":0.89200000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.02,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.1299999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.2749999999999999},"min":{"1min":1.034,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.837,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.411},{"interface":"front","average":{"1min":1.532,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":1.1990000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7210000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.621}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.538,"15min":1.49},"min":{"1min":0.89600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.2240000000000002,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.472},"min":{"1min":1.244,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4710000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1087,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194225,"statfs":{"total":200000000,"available":198886752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1930000000000001,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.577,"5min":2.081,"15min":2.081},"last":1.139},{"interface":"front","average":{"1min":1.27,"5min":1.22,"15min":1.208},"min":{"1min":0.76200000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5469999999999999,"5min":1.974,"15min":1.974},"last":1.407}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.145,"15min":1.1950000000000001},"min":{"1min":0.746,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.577,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1819999999999999,"15min":1.119},"min":{"1min":0.997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5649999999999999,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.2150000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.238,"15min":1.1859999999999999},"min":{"1min":0.94899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.665,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":1.048,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6259999999999999,"5min":1.962,"15min":1.962},"last":1.1679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4059999999999999,"15min":1.3160000000000001},"min":{"1min":1.0549999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3439999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":1.024,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.552,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.3500000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.355,"15min":1.3029999999999999},"min":{"1min":0.96199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7490000000000001,"5min":2.625,"15min":2.625},"last":1.044},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.3720000000000001},"min":{"1min":0.94499999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.857,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.5}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.4419999999999999,"15min":1.4330000000000001},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.77,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.512,"15min":1.417},"min":{"1min":1.177,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1059999999999999,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.579}]},{"osd":7,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.4770000000000001,"15min":1.4450000000000001},"min":{"1min":1.2869999999999999,"5min":0.63,"15min":0.63},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.587,"15min":1.579},"min":{"1min":1.4319999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.46}]}]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1759999999999999,"15min":1.1499999999999999},"min":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5329999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.1459999999999999},"min":{"1min":1.026,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5289999999999999,"5min":7.367,"15min":7.367},"last":1.5449999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1359999999999999,"15min":1.1220000000000001},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.359},{"interface":"front","average":{"1min":1.173,"5min":1.244,"15min":1.401},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5249999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.492}]},{"osd":2,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.208,"15min":1.302},"min":{"1min":0.93200000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.244,"15min":1.167},"min":{"1min":0.95899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.631,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.5820000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.292,"15min":1.23},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7110000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.371},"min":{"1min":1.1439999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.778,"5min":1.911,"15min":1.911},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.86599999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.095},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3280000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.829,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.385}]},{"osd":6,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.423,"15min":1.4350000000000001},"min":{"1min":0.81100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0219999999999998,"5min":2.863,"15min":2.863},"last":1.444},{"interface":"front","average":{"1min":1.542,"5min":1.548,"15min":1.5600000000000001},"min":{"1min":0.97799999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9730000000000001,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.196}]},{"osd":7,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.55,"15min":1.4179999999999999},"min":{"1min":1.1910000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.593,"15min":1.595},"min":{"1min":1.3320000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1230000000000002,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.625}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.86699999999999999,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.95399999999999996,"15min":0.89800000000000002},"min":{"1min":0.65900000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5349999999999999,"5min":1.55,"15min":1.55},"last":0.94099999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.871,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.153,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.004,"15min":0.97699999999999998},"min":{"1min":0.58699999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4910000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.96199999999999997,"15min":0.87},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":1.573,"5min":1.919,"15min":1.919},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1020000000000001,"15min":1.1080000000000001},"min":{"1min":0.81000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6140000000000001,"5min":3.399,"15min":3.399},"last":1.3640000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.214,"15min":1.1890000000000001},"min":{"1min":0.83499999999999996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5800000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.196},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.405,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.089}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3120000000000001},"min":{"1min":0.98899999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6699999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.294},{"interface":"front","average":{"1min":1.355,"5min":1.2849999999999999,"15min":1.2689999999999999},"min":{"1min":0.92000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5980000000000001,"5min":1.788,"15min":1.788},"last":1.327}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.282,"15min":1.242},"min":{"1min":1.0389999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.917,"15min":1.917},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.45,"5min":1.397,"15min":1.405},"min":{"1min":1.2230000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.734,"5min":3.496,"15min":3.496},"last":1.24}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.085,"5min":0.871,"15min":0.871},"max":{"1min":1.881,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.446},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":0.98899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.669,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.2569999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.82499999999999996},"min":{"1min":0.53300000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.4199999999999999,"5min":2.383,"15min":2.383},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.425,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.411,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.238},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5649999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.284}]},{"osd":3,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5429999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.081,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.486,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.3220000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3080000000000001,"15min":1.397},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.1309999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.266,"15min":1.375},"min":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.369,"15min":1.397},"min":{"1min":1.081,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9890000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.444}]},{"osd":6,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.381,"15min":1.4199999999999999},"min":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.875,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.425},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.482},"min":{"1min":0.995,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.097,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.7070000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.383,"15min":1.4490000000000001},"min":{"1min":0.89600000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7829999999999999,"5min":2.968,"15min":2.968},"last":1.478},{"interface":"front","average":{"1min":1.409,"5min":1.4199999999999999,"15min":1.504},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.056,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.8200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1087,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194225,"statfs":{"total":200000000,"available":198886752,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.77200000000000002},"min":{"1min":0.45900000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2250000000000001,"5min":1.627,"15min":1.627},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.876,"15min":0.78400000000000003},"min":{"1min":0.56799999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6850000000000001,"5min":1.895,"15min":1.895},"last":1.2849999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.79900000000000004},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.45,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.048,"5min":0.97499999999999998,"15min":0.90300000000000002},"min":{"1min":0.51500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4239999999999999,"5min":1.746,"15min":1.746},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1339999999999999,"15min":1.038},"min":{"1min":0.61199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.639,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.085,"5min":1.008,"15min":0.93999999999999995},"min":{"1min":0.67900000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4750000000000001,"5min":1.764,"15min":1.764},"last":1.2050000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.155,"15min":1.087},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.042},{"interface":"front","average":{"1min":1.369,"5min":1.3100000000000001,"15min":1.218},"min":{"1min":0.876,"5min":0.629,"15min":0.629},"max":{"1min":1.641,"5min":2.008,"15min":2.008},"last":1.1399999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1890000000000001},"min":{"1min":0.80000000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7909999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.077},{"interface":"front","average":{"1min":1.286,"5min":1.2170000000000001,"15min":1.1679999999999999},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.738,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3799999999999999,"15min":1.298},"min":{"1min":1.042,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9930000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.395},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.82199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8380000000000001,"5min":2.004,"15min":2.004},"last":1.3169999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":1.145,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7030000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.3899999999999999,"15min":1.3080000000000001},"min":{"1min":0.93500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9039999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.3480000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477120,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90700000000000003,"15min":0.85299999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.355,"5min":11.257,"15min":11.257},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.97099999999999997,"15min":0.95799999999999996},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7110000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.34799999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.38800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.433},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0029999999999999,"15min":1.0940000000000001},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5640000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":0.47399999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.012,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5609999999999999,"5min":1.869,"15min":1.869},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3340000000000001,"15min":1.353},"min":{"1min":0.83299999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7390000000000001,"5min":1.982,"15min":1.982},"last":0.83299999999999996}]},{"osd":5,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.256,"15min":1.3580000000000001},"min":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8340000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.268,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.81100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6579999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.006},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.296},"min":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.752}]},{"osd":7,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3660000000000001,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0779999999999998,"5min":2.121,"15min":2.121},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.319},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":0.69399999999999995}]}]}],"pool_statfs":[]}} 2023-08-25T09:43:59.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:43:59.356 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-08-25T09:43:59.354+0000 7f2d05cfc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:43:59.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.575+0000 7efbf759e640 1 Processor -- start 2023-08-25T09:43:59.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.575+0000 7efbf759e640 1 -- start start 2023-08-25T09:43:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.577+0000 7efbf759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 0x7efbf80b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.577+0000 7efbf759e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efbf80b12f0 0x7efbf815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.577+0000 7efbf759e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf815e7c0 con 0x7efbf815f190 2023-08-25T09:43:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.577+0000 7efbf759e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf8071b50 con 0x7efbf80b12f0 2023-08-25T09:43:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.577+0000 7efbf759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf815e490 con 0x7efbf815a7c0 2023-08-25T09:43:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 0x7efbf80b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 0x7efbf80b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:47174/0 (socket says 172.21.15.70:47174) 2023-08-25T09:43:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 -- 172.21.15.70:0/2405461192 learned_addr learned my addr 172.21.15.70:0/2405461192 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf5d9b640 1 --2- 172.21.15.70:0/2405461192 >> v2:172.21.15.155:3300/0 conn(0x7efbf80b12f0 0x7efbf815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 -- 172.21.15.70:0/2405461192 >> v2:172.21.15.155:3300/0 conn(0x7efbf80b12f0 msgr2=0x7efbf815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:43:59.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 --2- 172.21.15.70:0/2405461192 >> v2:172.21.15.155:3300/0 conn(0x7efbf80b12f0 0x7efbf815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 -- 172.21.15.70:0/2405461192 >> v1:172.21.15.155:6789/0 conn(0x7efbf815f190 legacy=0x7efbf80cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.578+0000 7efbf659c640 1 -- 172.21.15.70:0/2405461192 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80b3aa0 con 0x7efbf815a7c0 2023-08-25T09:43:59.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.580+0000 7efbf659c640 1 --2- 172.21.15.70:0/2405461192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 0x7efbf80b3560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009870 tx=0x7efbe802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc942cf6f62cd7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.580+0000 7efbf559a640 1 -- 172.21.15.70:0/2405461192 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8030840 con 0x7efbf815a7c0 2023-08-25T09:43:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.580+0000 7efbf559a640 1 -- 172.21.15.70:0/2405461192 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe80309a0 con 0x7efbf815a7c0 2023-08-25T09:43:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.580+0000 7efbf559a640 1 -- 172.21.15.70:0/2405461192 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe8030d30 con 0x7efbf815a7c0 2023-08-25T09:43:59.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 -- 172.21.15.70:0/2405461192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 msgr2=0x7efbf80b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 --2- 172.21.15.70:0/2405461192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 0x7efbf80b3560 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009870 tx=0x7efbe802b980 comp rx=0 tx=0).stop 2023-08-25T09:43:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 -- 172.21.15.70:0/2405461192 shutdown_connections 2023-08-25T09:43:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 --2- 172.21.15.70:0/2405461192 >> v2:172.21.15.155:3300/0 conn(0x7efbf80b12f0 0x7efbf815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 --2- 172.21.15.70:0/2405461192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815a7c0 0x7efbf80b3560 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 -- 172.21.15.70:0/2405461192 >> 172.21.15.70:0/2405461192 conn(0x7efbf80affd0 msgr2=0x7efbf80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 -- 172.21.15.70:0/2405461192 shutdown_connections 2023-08-25T09:43:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.581+0000 7efbf759e640 1 -- 172.21.15.70:0/2405461192 wait complete. 2023-08-25T09:43:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.583+0000 7efbf759e640 1 Processor -- start 2023-08-25T09:43:59.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.583+0000 7efbf759e640 1 -- start start 2023-08-25T09:43:59.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf80b12f0 0x7efbf80cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 0x7efbf806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815f190 0x7efbf81f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8071150 con 0x7efbf815a7c0 2023-08-25T09:43:59.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80c2270 con 0x7efbf80b12f0 2023-08-25T09:43:59.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf80708b0 con 0x7efbf815f190 2023-08-25T09:43:59.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 0x7efbf806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf5d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 0x7efbf806f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45280/0 (socket says 172.21.15.70:45280) 2023-08-25T09:43:59.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf5d9b640 1 -- 172.21.15.70:0/1593417235 learned_addr learned my addr 172.21.15.70:0/1593417235 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:43:59.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf659c640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf80b12f0 0x7efbf80cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.584+0000 7efbf6d9d640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815f190 0x7efbf81f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf5d9b640 1 -- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815f190 msgr2=0x7efbf81f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf5d9b640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815f190 0x7efbf81f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf5d9b640 1 -- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf80b12f0 msgr2=0x7efbf80cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf5d9b640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf80b12f0 0x7efbf80cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf5d9b640 1 -- 172.21.15.70:0/1593417235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe80094d0 con 0x7efbf815a7c0 2023-08-25T09:43:59.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf5d9b640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 0x7efbf806f590 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efbe400e9b0 tx=0x7efbe400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4009670 con 0x7efbf815a7c0 2023-08-25T09:43:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe400c9a0 con 0x7efbf815a7c0 2023-08-25T09:43:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8070b90 con 0x7efbf815a7c0 2023-08-25T09:43:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe400cd50 con 0x7efbf815a7c0 2023-08-25T09:43:59.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.585+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf806fad0 con 0x7efbf815a7c0 2023-08-25T09:43:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.587+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf80c1fd0 con 0x7efbf815a7c0 2023-08-25T09:43:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.588+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efbe4013470 con 0x7efbf815a7c0 2023-08-25T09:43:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.589+0000 7efbeeffd640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efbe006a050 0x7efbe006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:43:59.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.589+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efbe4088840 con 0x7efbf815a7c0 2023-08-25T09:43:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.589+0000 7efbf659c640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efbe006a050 0x7efbe006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:43:59.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.590+0000 7efbf659c640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efbe006a050 0x7efbe006c510 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efbe8036000 tx=0x7efbe8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:43:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.593+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efbf80c1fd0 con 0x7efbf815a7c0 2023-08-25T09:43:59.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.781+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efbf80b4040 con 0x7efbe006a050 2023-08-25T09:43:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.793+0000 7efbeeffd640 1 -- 172.21.15.70:0/1593417235 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302945 (secure 0 0 0) 0x7efbe8044470 con 0x7efbe006a050 2023-08-25T09:43:59.797 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:43:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:43:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efbe006a050 msgr2=0x7efbe006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efbe006a050 0x7efbe006c510 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efbe8036000 tx=0x7efbe8008030 comp rx=0 tx=0).stop 2023-08-25T09:43:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 msgr2=0x7efbf806f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:43:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 0x7efbf806f590 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7efbe400e9b0 tx=0x7efbe400ee70 comp rx=0 tx=0).stop 2023-08-25T09:43:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 shutdown_connections 2023-08-25T09:43:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efbe006a050 0x7efbe006c510 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efbf815f190 0x7efbf81f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efbf815a7c0 0x7efbf806f590 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 --2- 172.21.15.70:0/1593417235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efbf80b12f0 0x7efbf80cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:43:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 >> 172.21.15.70:0/1593417235 conn(0x7efbf80affd0 msgr2=0x7efbf80ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:43:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 shutdown_connections 2023-08-25T09:43:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:43:59.800+0000 7efbf759e640 1 -- 172.21.15.70:0/1593417235 wait complete. 2023-08-25T09:43:59.820 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2023-08-25T09:43:59.420989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5350,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557149,"statfs":{"total":1600000000,"available":1594521156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.044672"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529053+0000","last_change":"2023-08-25T09:43:54.404594+0000","last_active":"2023-08-25T09:43:55.529053+0000","last_peered":"2023-08-25T09:43:55.529053+0000","last_clean":"2023-08-25T09:43:55.529053+0000","last_became_active":"2023-08-25T09:43:54.404347+0000","last_became_peered":"2023-08-25T09:43:54.404347+0000","last_unstale":"2023-08-25T09:43:55.529053+0000","last_undegraded":"2023-08-25T09:43:55.529053+0000","last_fullsized":"2023-08-25T09:43:55.529053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:33.114359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.386937+0000","last_change":"2023-08-25T09:43:54.408980+0000","last_active":"2023-08-25T09:43:55.386937+0000","last_peered":"2023-08-25T09:43:55.386937+0000","last_clean":"2023-08-25T09:43:55.386937+0000","last_became_active":"2023-08-25T09:43:54.408724+0000","last_became_peered":"2023-08-25T09:43:54.408724+0000","last_unstale":"2023-08-25T09:43:55.386937+0000","last_undegraded":"2023-08-25T09:43:55.386937+0000","last_fullsized":"2023-08-25T09:43:55.386937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:27.591271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388658+0000","last_change":"2023-08-25T09:43:54.405905+0000","last_active":"2023-08-25T09:43:55.388658+0000","last_peered":"2023-08-25T09:43:55.388658+0000","last_clean":"2023-08-25T09:43:55.388658+0000","last_became_active":"2023-08-25T09:43:54.398170+0000","last_became_peered":"2023-08-25T09:43:54.398170+0000","last_unstale":"2023-08-25T09:43:55.388658+0000","last_undegraded":"2023-08-25T09:43:55.388658+0000","last_fullsized":"2023-08-25T09:43:55.388658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:17:42.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676440+0000","last_change":"2023-08-25T09:43:54.401703+0000","last_active":"2023-08-25T09:43:55.676440+0000","last_peered":"2023-08-25T09:43:55.676440+0000","last_clean":"2023-08-25T09:43:55.676440+0000","last_became_active":"2023-08-25T09:43:54.401389+0000","last_became_peered":"2023-08-25T09:43:54.401389+0000","last_unstale":"2023-08-25T09:43:55.676440+0000","last_undegraded":"2023-08-25T09:43:55.676440+0000","last_fullsized":"2023-08-25T09:43:55.676440+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:55.776311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389283+0000","last_change":"2023-08-25T09:43:54.416278+0000","last_active":"2023-08-25T09:43:55.389283+0000","last_peered":"2023-08-25T09:43:55.389283+0000","last_clean":"2023-08-25T09:43:55.389283+0000","last_became_active":"2023-08-25T09:43:54.415330+0000","last_became_peered":"2023-08-25T09:43:54.415330+0000","last_unstale":"2023-08-25T09:43:55.389283+0000","last_undegraded":"2023-08-25T09:43:55.389283+0000","last_fullsized":"2023-08-25T09:43:55.389283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:25.392124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675633+0000","last_change":"2023-08-25T09:43:54.413935+0000","last_active":"2023-08-25T09:43:55.675633+0000","last_peered":"2023-08-25T09:43:55.675633+0000","last_clean":"2023-08-25T09:43:55.675633+0000","last_became_active":"2023-08-25T09:43:54.413685+0000","last_became_peered":"2023-08-25T09:43:54.413685+0000","last_unstale":"2023-08-25T09:43:55.675633+0000","last_undegraded":"2023-08-25T09:43:55.675633+0000","last_fullsized":"2023-08-25T09:43:55.675633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:09.188639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677337+0000","last_change":"2023-08-25T09:43:54.409395+0000","last_active":"2023-08-25T09:43:55.677337+0000","last_peered":"2023-08-25T09:43:55.677337+0000","last_clean":"2023-08-25T09:43:55.677337+0000","last_became_active":"2023-08-25T09:43:54.409083+0000","last_became_peered":"2023-08-25T09:43:54.409083+0000","last_unstale":"2023-08-25T09:43:55.677337+0000","last_undegraded":"2023-08-25T09:43:55.677337+0000","last_fullsized":"2023-08-25T09:43:55.677337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:27.799378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530267+0000","last_change":"2023-08-25T09:43:54.407151+0000","last_active":"2023-08-25T09:43:55.530267+0000","last_peered":"2023-08-25T09:43:55.530267+0000","last_clean":"2023-08-25T09:43:55.530267+0000","last_became_active":"2023-08-25T09:43:54.406266+0000","last_became_peered":"2023-08-25T09:43:54.406266+0000","last_unstale":"2023-08-25T09:43:55.530267+0000","last_undegraded":"2023-08-25T09:43:55.530267+0000","last_fullsized":"2023-08-25T09:43:55.530267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:46:18.101385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676097+0000","last_change":"2023-08-25T09:43:54.408942+0000","last_active":"2023-08-25T09:43:55.676097+0000","last_peered":"2023-08-25T09:43:55.676097+0000","last_clean":"2023-08-25T09:43:55.676097+0000","last_became_active":"2023-08-25T09:43:54.408439+0000","last_became_peered":"2023-08-25T09:43:54.408439+0000","last_unstale":"2023-08-25T09:43:55.676097+0000","last_undegraded":"2023-08-25T09:43:55.676097+0000","last_fullsized":"2023-08-25T09:43:55.676097+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:53.991517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529111+0000","last_change":"2023-08-25T09:43:54.419056+0000","last_active":"2023-08-25T09:43:55.529111+0000","last_peered":"2023-08-25T09:43:55.529111+0000","last_clean":"2023-08-25T09:43:55.529111+0000","last_became_active":"2023-08-25T09:43:54.418667+0000","last_became_peered":"2023-08-25T09:43:54.418667+0000","last_unstale":"2023-08-25T09:43:55.529111+0000","last_undegraded":"2023-08-25T09:43:55.529111+0000","last_fullsized":"2023-08-25T09:43:55.529111+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:30.225126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387796+0000","last_change":"2023-08-25T09:43:54.415935+0000","last_active":"2023-08-25T09:43:55.387796+0000","last_peered":"2023-08-25T09:43:55.387796+0000","last_clean":"2023-08-25T09:43:55.387796+0000","last_became_active":"2023-08-25T09:43:54.414902+0000","last_became_peered":"2023-08-25T09:43:54.414902+0000","last_unstale":"2023-08-25T09:43:55.387796+0000","last_undegraded":"2023-08-25T09:43:55.387796+0000","last_fullsized":"2023-08-25T09:43:55.387796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:05:26.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676861+0000","last_change":"2023-08-25T09:43:54.404468+0000","last_active":"2023-08-25T09:43:55.676861+0000","last_peered":"2023-08-25T09:43:55.676861+0000","last_clean":"2023-08-25T09:43:55.676861+0000","last_became_active":"2023-08-25T09:43:54.403839+0000","last_became_peered":"2023-08-25T09:43:54.403839+0000","last_unstale":"2023-08-25T09:43:55.676861+0000","last_undegraded":"2023-08-25T09:43:55.676861+0000","last_fullsized":"2023-08-25T09:43:55.676861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:25.809595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387622+0000","last_change":"2023-08-25T09:43:54.408393+0000","last_active":"2023-08-25T09:43:55.387622+0000","last_peered":"2023-08-25T09:43:55.387622+0000","last_clean":"2023-08-25T09:43:55.387622+0000","last_became_active":"2023-08-25T09:43:54.407528+0000","last_became_peered":"2023-08-25T09:43:54.407528+0000","last_unstale":"2023-08-25T09:43:55.387622+0000","last_undegraded":"2023-08-25T09:43:55.387622+0000","last_fullsized":"2023-08-25T09:43:55.387622+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:46.864971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530141+0000","last_change":"2023-08-25T09:43:54.409118+0000","last_active":"2023-08-25T09:43:55.530141+0000","last_peered":"2023-08-25T09:43:55.530141+0000","last_clean":"2023-08-25T09:43:55.530141+0000","last_became_active":"2023-08-25T09:43:54.408893+0000","last_became_peered":"2023-08-25T09:43:54.408893+0000","last_unstale":"2023-08-25T09:43:55.530141+0000","last_undegraded":"2023-08-25T09:43:55.530141+0000","last_fullsized":"2023-08-25T09:43:55.530141+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:12.464768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389643+0000","last_change":"2023-08-25T09:43:54.417738+0000","last_active":"2023-08-25T09:43:55.389643+0000","last_peered":"2023-08-25T09:43:55.389643+0000","last_clean":"2023-08-25T09:43:55.389643+0000","last_became_active":"2023-08-25T09:43:54.415654+0000","last_became_peered":"2023-08-25T09:43:54.415654+0000","last_unstale":"2023-08-25T09:43:55.389643+0000","last_undegraded":"2023-08-25T09:43:55.389643+0000","last_fullsized":"2023-08-25T09:43:55.389643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:35.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677379+0000","last_change":"2023-08-25T09:43:54.394213+0000","last_active":"2023-08-25T09:43:55.677379+0000","last_peered":"2023-08-25T09:43:55.677379+0000","last_clean":"2023-08-25T09:43:55.677379+0000","last_became_active":"2023-08-25T09:43:54.393647+0000","last_became_peered":"2023-08-25T09:43:54.393647+0000","last_unstale":"2023-08-25T09:43:55.677379+0000","last_undegraded":"2023-08-25T09:43:55.677379+0000","last_fullsized":"2023-08-25T09:43:55.677379+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:39.470885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530362+0000","last_change":"2023-08-25T09:43:54.408147+0000","last_active":"2023-08-25T09:43:55.530362+0000","last_peered":"2023-08-25T09:43:55.530362+0000","last_clean":"2023-08-25T09:43:55.530362+0000","last_became_active":"2023-08-25T09:43:54.407786+0000","last_became_peered":"2023-08-25T09:43:54.407786+0000","last_unstale":"2023-08-25T09:43:55.530362+0000","last_undegraded":"2023-08-25T09:43:55.530362+0000","last_fullsized":"2023-08-25T09:43:55.530362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:24.779031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387994+0000","last_change":"2023-08-25T09:43:54.407208+0000","last_active":"2023-08-25T09:43:55.387994+0000","last_peered":"2023-08-25T09:43:55.387994+0000","last_clean":"2023-08-25T09:43:55.387994+0000","last_became_active":"2023-08-25T09:43:54.406324+0000","last_became_peered":"2023-08-25T09:43:54.406324+0000","last_unstale":"2023-08-25T09:43:55.387994+0000","last_undegraded":"2023-08-25T09:43:55.387994+0000","last_fullsized":"2023-08-25T09:43:55.387994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:00.393632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.531168+0000","last_change":"2023-08-25T09:43:54.415832+0000","last_active":"2023-08-25T09:43:55.531168+0000","last_peered":"2023-08-25T09:43:55.531168+0000","last_clean":"2023-08-25T09:43:55.531168+0000","last_became_active":"2023-08-25T09:43:54.415552+0000","last_became_peered":"2023-08-25T09:43:54.415552+0000","last_unstale":"2023-08-25T09:43:55.531168+0000","last_undegraded":"2023-08-25T09:43:55.531168+0000","last_fullsized":"2023-08-25T09:43:55.531168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:29.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676471+0000","last_change":"2023-08-25T09:43:54.402938+0000","last_active":"2023-08-25T09:43:55.676471+0000","last_peered":"2023-08-25T09:43:55.676471+0000","last_clean":"2023-08-25T09:43:55.676471+0000","last_became_active":"2023-08-25T09:43:54.400897+0000","last_became_peered":"2023-08-25T09:43:54.400897+0000","last_unstale":"2023-08-25T09:43:55.676471+0000","last_undegraded":"2023-08-25T09:43:55.676471+0000","last_fullsized":"2023-08-25T09:43:55.676471+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:44.889724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676124+0000","last_change":"2023-08-25T09:43:54.412294+0000","last_active":"2023-08-25T09:43:55.676124+0000","last_peered":"2023-08-25T09:43:55.676124+0000","last_clean":"2023-08-25T09:43:55.676124+0000","last_became_active":"2023-08-25T09:43:54.412114+0000","last_became_peered":"2023-08-25T09:43:54.412114+0000","last_unstale":"2023-08-25T09:43:55.676124+0000","last_undegraded":"2023-08-25T09:43:55.676124+0000","last_fullsized":"2023-08-25T09:43:55.676124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:42:21.825196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387480+0000","last_change":"2023-08-25T09:43:54.408210+0000","last_active":"2023-08-25T09:43:55.387480+0000","last_peered":"2023-08-25T09:43:55.387480+0000","last_clean":"2023-08-25T09:43:55.387480+0000","last_became_active":"2023-08-25T09:43:54.407960+0000","last_became_peered":"2023-08-25T09:43:54.407960+0000","last_unstale":"2023-08-25T09:43:55.387480+0000","last_undegraded":"2023-08-25T09:43:55.387480+0000","last_fullsized":"2023-08-25T09:43:55.387480+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:13.932322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.528897+0000","last_change":"2023-08-25T09:43:54.402889+0000","last_active":"2023-08-25T09:43:55.528897+0000","last_peered":"2023-08-25T09:43:55.528897+0000","last_clean":"2023-08-25T09:43:55.528897+0000","last_became_active":"2023-08-25T09:43:54.402611+0000","last_became_peered":"2023-08-25T09:43:54.402611+0000","last_unstale":"2023-08-25T09:43:55.528897+0000","last_undegraded":"2023-08-25T09:43:55.528897+0000","last_fullsized":"2023-08-25T09:43:55.528897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:15.783761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.386759+0000","last_change":"2023-08-25T09:43:54.407433+0000","last_active":"2023-08-25T09:43:55.386759+0000","last_peered":"2023-08-25T09:43:55.386759+0000","last_clean":"2023-08-25T09:43:55.386759+0000","last_became_active":"2023-08-25T09:43:54.407190+0000","last_became_peered":"2023-08-25T09:43:54.407190+0000","last_unstale":"2023-08-25T09:43:55.386759+0000","last_undegraded":"2023-08-25T09:43:55.386759+0000","last_fullsized":"2023-08-25T09:43:55.386759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:27.119175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675738+0000","last_change":"2023-08-25T09:43:54.414015+0000","last_active":"2023-08-25T09:43:55.675738+0000","last_peered":"2023-08-25T09:43:55.675738+0000","last_clean":"2023-08-25T09:43:55.675738+0000","last_became_active":"2023-08-25T09:43:54.412759+0000","last_became_peered":"2023-08-25T09:43:54.412759+0000","last_unstale":"2023-08-25T09:43:55.675738+0000","last_undegraded":"2023-08-25T09:43:55.675738+0000","last_fullsized":"2023-08-25T09:43:55.675738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:18.576641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677241+0000","last_change":"2023-08-25T09:43:54.412030+0000","last_active":"2023-08-25T09:43:55.677241+0000","last_peered":"2023-08-25T09:43:55.677241+0000","last_clean":"2023-08-25T09:43:55.677241+0000","last_became_active":"2023-08-25T09:43:54.411737+0000","last_became_peered":"2023-08-25T09:43:54.411737+0000","last_unstale":"2023-08-25T09:43:55.677241+0000","last_undegraded":"2023-08-25T09:43:55.677241+0000","last_fullsized":"2023-08-25T09:43:55.677241+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:51.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678060+0000","last_change":"2023-08-25T09:43:54.392602+0000","last_active":"2023-08-25T09:43:55.678060+0000","last_peered":"2023-08-25T09:43:55.678060+0000","last_clean":"2023-08-25T09:43:55.678060+0000","last_became_active":"2023-08-25T09:43:54.392310+0000","last_became_peered":"2023-08-25T09:43:54.392310+0000","last_unstale":"2023-08-25T09:43:55.678060+0000","last_undegraded":"2023-08-25T09:43:55.678060+0000","last_fullsized":"2023-08-25T09:43:55.678060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:38.123428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388355+0000","last_change":"2023-08-25T09:43:54.411259+0000","last_active":"2023-08-25T09:43:55.388355+0000","last_peered":"2023-08-25T09:43:55.388355+0000","last_clean":"2023-08-25T09:43:55.388355+0000","last_became_active":"2023-08-25T09:43:54.411008+0000","last_became_peered":"2023-08-25T09:43:54.411008+0000","last_unstale":"2023-08-25T09:43:55.388355+0000","last_undegraded":"2023-08-25T09:43:55.388355+0000","last_fullsized":"2023-08-25T09:43:55.388355+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:29:35.594192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675937+0000","last_change":"2023-08-25T09:43:54.411396+0000","last_active":"2023-08-25T09:43:55.675937+0000","last_peered":"2023-08-25T09:43:55.675937+0000","last_clean":"2023-08-25T09:43:55.675937+0000","last_became_active":"2023-08-25T09:43:54.410849+0000","last_became_peered":"2023-08-25T09:43:54.410849+0000","last_unstale":"2023-08-25T09:43:55.675937+0000","last_undegraded":"2023-08-25T09:43:55.675937+0000","last_fullsized":"2023-08-25T09:43:55.675937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:51.094231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.528980+0000","last_change":"2023-08-25T09:43:54.418899+0000","last_active":"2023-08-25T09:43:55.528980+0000","last_peered":"2023-08-25T09:43:55.528980+0000","last_clean":"2023-08-25T09:43:55.528980+0000","last_became_active":"2023-08-25T09:43:54.417852+0000","last_became_peered":"2023-08-25T09:43:54.417852+0000","last_unstale":"2023-08-25T09:43:55.528980+0000","last_undegraded":"2023-08-25T09:43:55.528980+0000","last_fullsized":"2023-08-25T09:43:55.528980+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:42:09.899923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388185+0000","last_change":"2023-08-25T09:43:54.407528+0000","last_active":"2023-08-25T09:43:55.388185+0000","last_peered":"2023-08-25T09:43:55.388185+0000","last_clean":"2023-08-25T09:43:55.388185+0000","last_became_active":"2023-08-25T09:43:54.407006+0000","last_became_peered":"2023-08-25T09:43:54.407006+0000","last_unstale":"2023-08-25T09:43:55.388185+0000","last_undegraded":"2023-08-25T09:43:55.388185+0000","last_fullsized":"2023-08-25T09:43:55.388185+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:03.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678374+0000","last_change":"2023-08-25T09:43:54.407447+0000","last_active":"2023-08-25T09:43:55.678374+0000","last_peered":"2023-08-25T09:43:55.678374+0000","last_clean":"2023-08-25T09:43:55.678374+0000","last_became_active":"2023-08-25T09:43:54.407225+0000","last_became_peered":"2023-08-25T09:43:54.407225+0000","last_unstale":"2023-08-25T09:43:55.678374+0000","last_undegraded":"2023-08-25T09:43:55.678374+0000","last_fullsized":"2023-08-25T09:43:55.678374+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:18:36.366812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.390000+0000","last_change":"2023-08-25T09:43:54.417973+0000","last_active":"2023-08-25T09:43:55.390000+0000","last_peered":"2023-08-25T09:43:55.390000+0000","last_clean":"2023-08-25T09:43:55.390000+0000","last_became_active":"2023-08-25T09:43:54.416003+0000","last_became_peered":"2023-08-25T09:43:54.416003+0000","last_unstale":"2023-08-25T09:43:55.390000+0000","last_undegraded":"2023-08-25T09:43:55.390000+0000","last_fullsized":"2023-08-25T09:43:55.390000+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:22.319316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"93'4","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389004+0000","last_change":"2023-08-25T09:43:53.397210+0000","last_active":"2023-08-25T09:43:55.389004+0000","last_peered":"2023-08-25T09:43:55.389004+0000","last_clean":"2023-08-25T09:43:55.389004+0000","last_became_active":"2023-08-25T09:43:53.396946+0000","last_became_peered":"2023-08-25T09:43:53.396946+0000","last_unstale":"2023-08-25T09:43:55.389004+0000","last_undegraded":"2023-08-25T09:43:55.389004+0000","last_fullsized":"2023-08-25T09:43:55.389004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:22.103597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387784+0000","last_change":"2023-08-25T09:43:54.413793+0000","last_active":"2023-08-25T09:43:55.387784+0000","last_peered":"2023-08-25T09:43:55.387784+0000","last_clean":"2023-08-25T09:43:55.387784+0000","last_became_active":"2023-08-25T09:43:54.413378+0000","last_became_peered":"2023-08-25T09:43:54.413378+0000","last_unstale":"2023-08-25T09:43:55.387784+0000","last_undegraded":"2023-08-25T09:43:55.387784+0000","last_fullsized":"2023-08-25T09:43:55.387784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:28:34.617151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384615+0000","last_change":"2023-08-25T09:43:53.393077+0000","last_active":"2023-08-25T09:43:55.384615+0000","last_peered":"2023-08-25T09:43:55.384615+0000","last_clean":"2023-08-25T09:43:55.384615+0000","last_became_active":"2023-08-25T09:43:53.392575+0000","last_became_peered":"2023-08-25T09:43:53.392575+0000","last_unstale":"2023-08-25T09:43:55.384615+0000","last_undegraded":"2023-08-25T09:43:55.384615+0000","last_fullsized":"2023-08-25T09:43:55.384615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:04.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677126+0000","last_change":"2023-08-25T09:43:54.409806+0000","last_active":"2023-08-25T09:43:55.677126+0000","last_peered":"2023-08-25T09:43:55.677126+0000","last_clean":"2023-08-25T09:43:55.677126+0000","last_became_active":"2023-08-25T09:43:54.409518+0000","last_became_peered":"2023-08-25T09:43:54.409518+0000","last_unstale":"2023-08-25T09:43:55.677126+0000","last_undegraded":"2023-08-25T09:43:55.677126+0000","last_fullsized":"2023-08-25T09:43:55.677126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:46.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529263+0000","last_change":"2023-08-25T09:43:53.398957+0000","last_active":"2023-08-25T09:43:55.529263+0000","last_peered":"2023-08-25T09:43:55.529263+0000","last_clean":"2023-08-25T09:43:55.529263+0000","last_became_active":"2023-08-25T09:43:53.398717+0000","last_became_peered":"2023-08-25T09:43:53.398717+0000","last_unstale":"2023-08-25T09:43:55.529263+0000","last_undegraded":"2023-08-25T09:43:55.529263+0000","last_fullsized":"2023-08-25T09:43:55.529263+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:49.791339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677734+0000","last_change":"2023-08-25T09:43:54.401591+0000","last_active":"2023-08-25T09:43:55.677734+0000","last_peered":"2023-08-25T09:43:55.677734+0000","last_clean":"2023-08-25T09:43:55.677734+0000","last_became_active":"2023-08-25T09:43:54.401293+0000","last_became_peered":"2023-08-25T09:43:54.401293+0000","last_unstale":"2023-08-25T09:43:55.677734+0000","last_undegraded":"2023-08-25T09:43:55.677734+0000","last_fullsized":"2023-08-25T09:43:55.677734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:46:18.935110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676265+0000","last_change":"2023-08-25T09:43:53.389534+0000","last_active":"2023-08-25T09:43:55.676265+0000","last_peered":"2023-08-25T09:43:55.676265+0000","last_clean":"2023-08-25T09:43:55.676265+0000","last_became_active":"2023-08-25T09:43:53.389145+0000","last_became_peered":"2023-08-25T09:43:53.389145+0000","last_unstale":"2023-08-25T09:43:55.676265+0000","last_undegraded":"2023-08-25T09:43:55.676265+0000","last_fullsized":"2023-08-25T09:43:55.676265+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:06.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676066+0000","last_change":"2023-08-25T09:43:54.401991+0000","last_active":"2023-08-25T09:43:55.676066+0000","last_peered":"2023-08-25T09:43:55.676066+0000","last_clean":"2023-08-25T09:43:55.676066+0000","last_became_active":"2023-08-25T09:43:54.401631+0000","last_became_peered":"2023-08-25T09:43:54.401631+0000","last_unstale":"2023-08-25T09:43:55.676066+0000","last_undegraded":"2023-08-25T09:43:55.676066+0000","last_fullsized":"2023-08-25T09:43:55.676066+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:04.265830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678143+0000","last_change":"2023-08-25T09:43:53.385163+0000","last_active":"2023-08-25T09:43:55.678143+0000","last_peered":"2023-08-25T09:43:55.678143+0000","last_clean":"2023-08-25T09:43:55.678143+0000","last_became_active":"2023-08-25T09:43:53.384839+0000","last_became_peered":"2023-08-25T09:43:53.384839+0000","last_unstale":"2023-08-25T09:43:55.678143+0000","last_undegraded":"2023-08-25T09:43:55.678143+0000","last_fullsized":"2023-08-25T09:43:55.678143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:50:28.252589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387331+0000","last_change":"2023-08-25T09:43:54.402776+0000","last_active":"2023-08-25T09:43:55.387331+0000","last_peered":"2023-08-25T09:43:55.387331+0000","last_clean":"2023-08-25T09:43:55.387331+0000","last_became_active":"2023-08-25T09:43:54.401343+0000","last_became_peered":"2023-08-25T09:43:54.401343+0000","last_unstale":"2023-08-25T09:43:55.387331+0000","last_undegraded":"2023-08-25T09:43:55.387331+0000","last_fullsized":"2023-08-25T09:43:55.387331+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:26.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385440+0000","last_change":"2023-08-25T09:43:53.401154+0000","last_active":"2023-08-25T09:43:55.385440+0000","last_peered":"2023-08-25T09:43:55.385440+0000","last_clean":"2023-08-25T09:43:55.385440+0000","last_became_active":"2023-08-25T09:43:53.400830+0000","last_became_peered":"2023-08-25T09:43:53.400830+0000","last_unstale":"2023-08-25T09:43:55.385440+0000","last_undegraded":"2023-08-25T09:43:55.385440+0000","last_fullsized":"2023-08-25T09:43:55.385440+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:56:52.629469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676387+0000","last_change":"2023-08-25T09:43:54.413882+0000","last_active":"2023-08-25T09:43:55.676387+0000","last_peered":"2023-08-25T09:43:55.676387+0000","last_clean":"2023-08-25T09:43:55.676387+0000","last_became_active":"2023-08-25T09:43:54.413721+0000","last_became_peered":"2023-08-25T09:43:54.413721+0000","last_unstale":"2023-08-25T09:43:55.676387+0000","last_undegraded":"2023-08-25T09:43:55.676387+0000","last_fullsized":"2023-08-25T09:43:55.676387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:43:17.280367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530798+0000","last_change":"2023-08-25T09:43:53.401459+0000","last_active":"2023-08-25T09:43:55.530798+0000","last_peered":"2023-08-25T09:43:55.530798+0000","last_clean":"2023-08-25T09:43:55.530798+0000","last_became_active":"2023-08-25T09:43:53.400825+0000","last_became_peered":"2023-08-25T09:43:53.400825+0000","last_unstale":"2023-08-25T09:43:55.530798+0000","last_undegraded":"2023-08-25T09:43:55.530798+0000","last_fullsized":"2023-08-25T09:43:55.530798+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:14.912189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385020+0000","last_change":"2023-08-25T09:43:54.398080+0000","last_active":"2023-08-25T09:43:55.385020+0000","last_peered":"2023-08-25T09:43:55.385020+0000","last_clean":"2023-08-25T09:43:55.385020+0000","last_became_active":"2023-08-25T09:43:54.397273+0000","last_became_peered":"2023-08-25T09:43:54.397273+0000","last_unstale":"2023-08-25T09:43:55.385020+0000","last_undegraded":"2023-08-25T09:43:55.385020+0000","last_fullsized":"2023-08-25T09:43:55.385020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:01.756935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387956+0000","last_change":"2023-08-25T09:43:53.399348+0000","last_active":"2023-08-25T09:43:55.387956+0000","last_peered":"2023-08-25T09:43:55.387956+0000","last_clean":"2023-08-25T09:43:55.387956+0000","last_became_active":"2023-08-25T09:43:53.399099+0000","last_became_peered":"2023-08-25T09:43:53.399099+0000","last_unstale":"2023-08-25T09:43:55.387956+0000","last_undegraded":"2023-08-25T09:43:55.387956+0000","last_fullsized":"2023-08-25T09:43:55.387956+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:06:36.086345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388332+0000","last_change":"2023-08-25T09:43:54.406818+0000","last_active":"2023-08-25T09:43:55.388332+0000","last_peered":"2023-08-25T09:43:55.388332+0000","last_clean":"2023-08-25T09:43:55.388332+0000","last_became_active":"2023-08-25T09:43:54.405375+0000","last_became_peered":"2023-08-25T09:43:54.405375+0000","last_unstale":"2023-08-25T09:43:55.388332+0000","last_undegraded":"2023-08-25T09:43:55.388332+0000","last_fullsized":"2023-08-25T09:43:55.388332+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:30.258111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677565+0000","last_change":"2023-08-25T09:43:53.397375+0000","last_active":"2023-08-25T09:43:55.677565+0000","last_peered":"2023-08-25T09:43:55.677565+0000","last_clean":"2023-08-25T09:43:55.677565+0000","last_became_active":"2023-08-25T09:43:53.397119+0000","last_became_peered":"2023-08-25T09:43:53.397119+0000","last_unstale":"2023-08-25T09:43:55.677565+0000","last_undegraded":"2023-08-25T09:43:55.677565+0000","last_fullsized":"2023-08-25T09:43:55.677565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:20:30.983138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677836+0000","last_change":"2023-08-25T09:43:54.400636+0000","last_active":"2023-08-25T09:43:55.677836+0000","last_peered":"2023-08-25T09:43:55.677836+0000","last_clean":"2023-08-25T09:43:55.677836+0000","last_became_active":"2023-08-25T09:43:54.400265+0000","last_became_peered":"2023-08-25T09:43:54.400265+0000","last_unstale":"2023-08-25T09:43:55.677836+0000","last_undegraded":"2023-08-25T09:43:55.677836+0000","last_fullsized":"2023-08-25T09:43:55.677836+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:06:46.573434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529113+0000","last_change":"2023-08-25T09:43:53.399279+0000","last_active":"2023-08-25T09:43:55.529113+0000","last_peered":"2023-08-25T09:43:55.529113+0000","last_clean":"2023-08-25T09:43:55.529113+0000","last_became_active":"2023-08-25T09:43:53.399023+0000","last_became_peered":"2023-08-25T09:43:53.399023+0000","last_unstale":"2023-08-25T09:43:55.529113+0000","last_undegraded":"2023-08-25T09:43:55.529113+0000","last_fullsized":"2023-08-25T09:43:55.529113+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:14:04.107168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530821+0000","last_change":"2023-08-25T09:43:54.411304+0000","last_active":"2023-08-25T09:43:55.530821+0000","last_peered":"2023-08-25T09:43:55.530821+0000","last_clean":"2023-08-25T09:43:55.530821+0000","last_became_active":"2023-08-25T09:43:54.410775+0000","last_became_peered":"2023-08-25T09:43:54.410775+0000","last_unstale":"2023-08-25T09:43:55.530821+0000","last_undegraded":"2023-08-25T09:43:55.530821+0000","last_fullsized":"2023-08-25T09:43:55.530821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:00.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"93'3","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675501+0000","last_change":"2023-08-25T09:43:53.400335+0000","last_active":"2023-08-25T09:43:55.675501+0000","last_peered":"2023-08-25T09:43:55.675501+0000","last_clean":"2023-08-25T09:43:55.675501+0000","last_became_active":"2023-08-25T09:43:53.397372+0000","last_became_peered":"2023-08-25T09:43:53.397372+0000","last_unstale":"2023-08-25T09:43:55.675501+0000","last_undegraded":"2023-08-25T09:43:55.675501+0000","last_fullsized":"2023-08-25T09:43:55.675501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:57.202891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676283+0000","last_change":"2023-08-25T09:43:54.412139+0000","last_active":"2023-08-25T09:43:55.676283+0000","last_peered":"2023-08-25T09:43:55.676283+0000","last_clean":"2023-08-25T09:43:55.676283+0000","last_became_active":"2023-08-25T09:43:54.402590+0000","last_became_peered":"2023-08-25T09:43:54.402590+0000","last_unstale":"2023-08-25T09:43:55.676283+0000","last_undegraded":"2023-08-25T09:43:55.676283+0000","last_fullsized":"2023-08-25T09:43:55.676283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:44.502910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"93'6","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384787+0000","last_change":"2023-08-25T09:43:53.399357+0000","last_active":"2023-08-25T09:43:55.384787+0000","last_peered":"2023-08-25T09:43:55.384787+0000","last_clean":"2023-08-25T09:43:55.384787+0000","last_became_active":"2023-08-25T09:43:53.398803+0000","last_became_peered":"2023-08-25T09:43:53.398803+0000","last_unstale":"2023-08-25T09:43:55.384787+0000","last_undegraded":"2023-08-25T09:43:55.384787+0000","last_fullsized":"2023-08-25T09:43:55.384787+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:41.985823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677987+0000","last_change":"2023-08-25T09:43:54.408743+0000","last_active":"2023-08-25T09:43:55.677987+0000","last_peered":"2023-08-25T09:43:55.677987+0000","last_clean":"2023-08-25T09:43:55.677987+0000","last_became_active":"2023-08-25T09:43:54.407981+0000","last_became_peered":"2023-08-25T09:43:54.407981+0000","last_unstale":"2023-08-25T09:43:55.677987+0000","last_undegraded":"2023-08-25T09:43:55.677987+0000","last_fullsized":"2023-08-25T09:43:55.677987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:22.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529338+0000","last_change":"2023-08-25T09:43:53.400186+0000","last_active":"2023-08-25T09:43:55.529338+0000","last_peered":"2023-08-25T09:43:55.529338+0000","last_clean":"2023-08-25T09:43:55.529338+0000","last_became_active":"2023-08-25T09:43:53.399735+0000","last_became_peered":"2023-08-25T09:43:53.399735+0000","last_unstale":"2023-08-25T09:43:55.529338+0000","last_undegraded":"2023-08-25T09:43:55.529338+0000","last_fullsized":"2023-08-25T09:43:55.529338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:58.045564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529095+0000","last_change":"2023-08-25T09:43:54.408566+0000","last_active":"2023-08-25T09:43:55.529095+0000","last_peered":"2023-08-25T09:43:55.529095+0000","last_clean":"2023-08-25T09:43:55.529095+0000","last_became_active":"2023-08-25T09:43:54.408126+0000","last_became_peered":"2023-08-25T09:43:54.408126+0000","last_unstale":"2023-08-25T09:43:55.529095+0000","last_undegraded":"2023-08-25T09:43:55.529095+0000","last_fullsized":"2023-08-25T09:43:55.529095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:51.838016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676385+0000","last_change":"2023-08-25T09:43:53.394175+0000","last_active":"2023-08-25T09:43:55.676385+0000","last_peered":"2023-08-25T09:43:55.676385+0000","last_clean":"2023-08-25T09:43:55.676385+0000","last_became_active":"2023-08-25T09:43:53.393635+0000","last_became_peered":"2023-08-25T09:43:53.393635+0000","last_unstale":"2023-08-25T09:43:55.676385+0000","last_undegraded":"2023-08-25T09:43:55.676385+0000","last_fullsized":"2023-08-25T09:43:55.676385+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:05.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677189+0000","last_change":"2023-08-25T09:43:54.397561+0000","last_active":"2023-08-25T09:43:55.677189+0000","last_peered":"2023-08-25T09:43:55.677189+0000","last_clean":"2023-08-25T09:43:55.677189+0000","last_became_active":"2023-08-25T09:43:54.397258+0000","last_became_peered":"2023-08-25T09:43:54.397258+0000","last_unstale":"2023-08-25T09:43:55.677189+0000","last_undegraded":"2023-08-25T09:43:55.677189+0000","last_fullsized":"2023-08-25T09:43:55.677189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:52.770025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389288+0000","last_change":"2023-08-25T09:43:53.403154+0000","last_active":"2023-08-25T09:43:55.389288+0000","last_peered":"2023-08-25T09:43:55.389288+0000","last_clean":"2023-08-25T09:43:55.389288+0000","last_became_active":"2023-08-25T09:43:53.399755+0000","last_became_peered":"2023-08-25T09:43:53.399755+0000","last_unstale":"2023-08-25T09:43:55.389288+0000","last_undegraded":"2023-08-25T09:43:55.389288+0000","last_fullsized":"2023-08-25T09:43:55.389288+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:14.044339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529191+0000","last_change":"2023-08-25T09:43:54.406065+0000","last_active":"2023-08-25T09:43:55.529191+0000","last_peered":"2023-08-25T09:43:55.529191+0000","last_clean":"2023-08-25T09:43:55.529191+0000","last_became_active":"2023-08-25T09:43:54.405822+0000","last_became_peered":"2023-08-25T09:43:54.405822+0000","last_unstale":"2023-08-25T09:43:55.529191+0000","last_undegraded":"2023-08-25T09:43:55.529191+0000","last_fullsized":"2023-08-25T09:43:55.529191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:21.088250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"93'8","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675963+0000","last_change":"2023-08-25T09:43:53.403310+0000","last_active":"2023-08-25T09:43:55.675963+0000","last_peered":"2023-08-25T09:43:55.675963+0000","last_clean":"2023-08-25T09:43:55.675963+0000","last_became_active":"2023-08-25T09:43:53.400976+0000","last_became_peered":"2023-08-25T09:43:53.400976+0000","last_unstale":"2023-08-25T09:43:55.675963+0000","last_undegraded":"2023-08-25T09:43:55.675963+0000","last_fullsized":"2023-08-25T09:43:55.675963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:05:07.588952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677139+0000","last_change":"2023-08-25T09:43:54.394896+0000","last_active":"2023-08-25T09:43:55.677139+0000","last_peered":"2023-08-25T09:43:55.677139+0000","last_clean":"2023-08-25T09:43:55.677139+0000","last_became_active":"2023-08-25T09:43:54.394161+0000","last_became_peered":"2023-08-25T09:43:54.394161+0000","last_unstale":"2023-08-25T09:43:55.677139+0000","last_undegraded":"2023-08-25T09:43:55.677139+0000","last_fullsized":"2023-08-25T09:43:55.677139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:03:43.678318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529257+0000","last_change":"2023-08-25T09:43:53.400229+0000","last_active":"2023-08-25T09:43:55.529257+0000","last_peered":"2023-08-25T09:43:55.529257+0000","last_clean":"2023-08-25T09:43:55.529257+0000","last_became_active":"2023-08-25T09:43:53.400007+0000","last_became_peered":"2023-08-25T09:43:53.400007+0000","last_unstale":"2023-08-25T09:43:55.529257+0000","last_undegraded":"2023-08-25T09:43:55.529257+0000","last_fullsized":"2023-08-25T09:43:55.529257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:23.839606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529294+0000","last_change":"2023-08-25T09:43:54.408421+0000","last_active":"2023-08-25T09:43:55.529294+0000","last_peered":"2023-08-25T09:43:55.529294+0000","last_clean":"2023-08-25T09:43:55.529294+0000","last_became_active":"2023-08-25T09:43:54.407614+0000","last_became_peered":"2023-08-25T09:43:54.407614+0000","last_unstale":"2023-08-25T09:43:55.529294+0000","last_undegraded":"2023-08-25T09:43:55.529294+0000","last_fullsized":"2023-08-25T09:43:55.529294+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:15.802242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"94'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676384+0000","last_change":"2023-08-25T09:43:53.400677+0000","last_active":"2023-08-25T09:43:55.676384+0000","last_peered":"2023-08-25T09:43:55.676384+0000","last_clean":"2023-08-25T09:43:55.676384+0000","last_became_active":"2023-08-25T09:43:53.397908+0000","last_became_peered":"2023-08-25T09:43:53.397908+0000","last_unstale":"2023-08-25T09:43:55.676384+0000","last_undegraded":"2023-08-25T09:43:55.676384+0000","last_fullsized":"2023-08-25T09:43:55.676384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:45.920977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677501+0000","last_change":"2023-08-25T09:43:54.408623+0000","last_active":"2023-08-25T09:43:55.677501+0000","last_peered":"2023-08-25T09:43:55.677501+0000","last_clean":"2023-08-25T09:43:55.677501+0000","last_became_active":"2023-08-25T09:43:54.408346+0000","last_became_peered":"2023-08-25T09:43:54.408346+0000","last_unstale":"2023-08-25T09:43:55.677501+0000","last_undegraded":"2023-08-25T09:43:55.677501+0000","last_fullsized":"2023-08-25T09:43:55.677501+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:39.048547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529155+0000","last_change":"2023-08-25T09:43:53.397556+0000","last_active":"2023-08-25T09:43:55.529155+0000","last_peered":"2023-08-25T09:43:55.529155+0000","last_clean":"2023-08-25T09:43:55.529155+0000","last_became_active":"2023-08-25T09:43:53.397131+0000","last_became_peered":"2023-08-25T09:43:53.397131+0000","last_unstale":"2023-08-25T09:43:55.529155+0000","last_undegraded":"2023-08-25T09:43:55.529155+0000","last_fullsized":"2023-08-25T09:43:55.529155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:34:31.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.531351+0000","last_change":"2023-08-25T09:43:54.411155+0000","last_active":"2023-08-25T09:43:55.531351+0000","last_peered":"2023-08-25T09:43:55.531351+0000","last_clean":"2023-08-25T09:43:55.531351+0000","last_became_active":"2023-08-25T09:43:54.410442+0000","last_became_peered":"2023-08-25T09:43:54.410442+0000","last_unstale":"2023-08-25T09:43:55.531351+0000","last_undegraded":"2023-08-25T09:43:55.531351+0000","last_fullsized":"2023-08-25T09:43:55.531351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:06:27.055748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677890+0000","last_change":"2023-08-25T09:43:53.390461+0000","last_active":"2023-08-25T09:43:55.677890+0000","last_peered":"2023-08-25T09:43:55.677890+0000","last_clean":"2023-08-25T09:43:55.677890+0000","last_became_active":"2023-08-25T09:43:53.390304+0000","last_became_peered":"2023-08-25T09:43:53.390304+0000","last_unstale":"2023-08-25T09:43:55.677890+0000","last_undegraded":"2023-08-25T09:43:55.677890+0000","last_fullsized":"2023-08-25T09:43:55.677890+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:10.567771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676883+0000","last_change":"2023-08-25T09:43:54.416330+0000","last_active":"2023-08-25T09:43:55.676883+0000","last_peered":"2023-08-25T09:43:55.676883+0000","last_clean":"2023-08-25T09:43:55.676883+0000","last_became_active":"2023-08-25T09:43:54.416029+0000","last_became_peered":"2023-08-25T09:43:54.416029+0000","last_unstale":"2023-08-25T09:43:55.676883+0000","last_undegraded":"2023-08-25T09:43:55.676883+0000","last_fullsized":"2023-08-25T09:43:55.676883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:49.899994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385379+0000","last_change":"2023-08-25T09:43:53.394719+0000","last_active":"2023-08-25T09:43:55.385379+0000","last_peered":"2023-08-25T09:43:55.385379+0000","last_clean":"2023-08-25T09:43:55.385379+0000","last_became_active":"2023-08-25T09:43:53.394231+0000","last_became_peered":"2023-08-25T09:43:53.394231+0000","last_unstale":"2023-08-25T09:43:55.385379+0000","last_undegraded":"2023-08-25T09:43:55.385379+0000","last_fullsized":"2023-08-25T09:43:55.385379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:53.753394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676795+0000","last_change":"2023-08-25T09:43:54.395059+0000","last_active":"2023-08-25T09:43:55.676795+0000","last_peered":"2023-08-25T09:43:55.676795+0000","last_clean":"2023-08-25T09:43:55.676795+0000","last_became_active":"2023-08-25T09:43:54.394487+0000","last_became_peered":"2023-08-25T09:43:54.394487+0000","last_unstale":"2023-08-25T09:43:55.676795+0000","last_undegraded":"2023-08-25T09:43:55.676795+0000","last_fullsized":"2023-08-25T09:43:55.676795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:12.153733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676095+0000","last_change":"2023-08-25T09:43:53.401366+0000","last_active":"2023-08-25T09:43:55.676095+0000","last_peered":"2023-08-25T09:43:55.676095+0000","last_clean":"2023-08-25T09:43:55.676095+0000","last_became_active":"2023-08-25T09:43:53.400954+0000","last_became_peered":"2023-08-25T09:43:53.400954+0000","last_unstale":"2023-08-25T09:43:55.676095+0000","last_undegraded":"2023-08-25T09:43:55.676095+0000","last_fullsized":"2023-08-25T09:43:55.676095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:43.297912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389004+0000","last_change":"2023-08-25T09:43:54.408774+0000","last_active":"2023-08-25T09:43:55.389004+0000","last_peered":"2023-08-25T09:43:55.389004+0000","last_clean":"2023-08-25T09:43:55.389004+0000","last_became_active":"2023-08-25T09:43:54.408056+0000","last_became_peered":"2023-08-25T09:43:54.408056+0000","last_unstale":"2023-08-25T09:43:55.389004+0000","last_undegraded":"2023-08-25T09:43:55.389004+0000","last_fullsized":"2023-08-25T09:43:55.389004+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:39.809497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676019+0000","last_change":"2023-08-25T09:43:53.396976+0000","last_active":"2023-08-25T09:43:55.676019+0000","last_peered":"2023-08-25T09:43:55.676019+0000","last_clean":"2023-08-25T09:43:55.676019+0000","last_became_active":"2023-08-25T09:43:53.396705+0000","last_became_peered":"2023-08-25T09:43:53.396705+0000","last_unstale":"2023-08-25T09:43:55.676019+0000","last_undegraded":"2023-08-25T09:43:55.676019+0000","last_fullsized":"2023-08-25T09:43:55.676019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:12.675960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388510+0000","last_change":"2023-08-25T09:43:54.410392+0000","last_active":"2023-08-25T09:43:55.388510+0000","last_peered":"2023-08-25T09:43:55.388510+0000","last_clean":"2023-08-25T09:43:55.388510+0000","last_became_active":"2023-08-25T09:43:54.410129+0000","last_became_peered":"2023-08-25T09:43:54.410129+0000","last_unstale":"2023-08-25T09:43:55.388510+0000","last_undegraded":"2023-08-25T09:43:55.388510+0000","last_fullsized":"2023-08-25T09:43:55.388510+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:35.069148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387143+0000","last_change":"2023-08-25T09:43:53.398900+0000","last_active":"2023-08-25T09:43:55.387143+0000","last_peered":"2023-08-25T09:43:55.387143+0000","last_clean":"2023-08-25T09:43:55.387143+0000","last_became_active":"2023-08-25T09:43:53.398452+0000","last_became_peered":"2023-08-25T09:43:53.398452+0000","last_unstale":"2023-08-25T09:43:55.387143+0000","last_undegraded":"2023-08-25T09:43:55.387143+0000","last_fullsized":"2023-08-25T09:43:55.387143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:23.214744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385717+0000","last_change":"2023-08-25T09:43:54.413745+0000","last_active":"2023-08-25T09:43:55.385717+0000","last_peered":"2023-08-25T09:43:55.385717+0000","last_clean":"2023-08-25T09:43:55.385717+0000","last_became_active":"2023-08-25T09:43:54.413429+0000","last_became_peered":"2023-08-25T09:43:54.413429+0000","last_unstale":"2023-08-25T09:43:55.385717+0000","last_undegraded":"2023-08-25T09:43:55.385717+0000","last_fullsized":"2023-08-25T09:43:55.385717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:45.596511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676451+0000","last_change":"2023-08-25T09:43:53.395918+0000","last_active":"2023-08-25T09:43:55.676451+0000","last_peered":"2023-08-25T09:43:55.676451+0000","last_clean":"2023-08-25T09:43:55.676451+0000","last_became_active":"2023-08-25T09:43:53.395232+0000","last_became_peered":"2023-08-25T09:43:53.395232+0000","last_unstale":"2023-08-25T09:43:55.676451+0000","last_undegraded":"2023-08-25T09:43:55.676451+0000","last_fullsized":"2023-08-25T09:43:55.676451+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:05.916939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":293,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529317+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:43:55.529317+0000","last_peered":"2023-08-25T09:43:55.529317+0000","last_clean":"2023-08-25T09:43:55.529317+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:43:55.529317+0000","last_undegraded":"2023-08-25T09:43:55.529317+0000","last_fullsized":"2023-08-25T09:43:55.529317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:04.320619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385863+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:43:55.385863+0000","last_peered":"2023-08-25T09:43:55.385863+0000","last_clean":"2023-08-25T09:43:55.385863+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:43:55.385863+0000","last_undegraded":"2023-08-25T09:43:55.385863+0000","last_fullsized":"2023-08-25T09:43:55.385863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:22.300510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.390505+0000","last_change":"2023-08-25T09:43:54.418281+0000","last_active":"2023-08-25T09:43:55.390505+0000","last_peered":"2023-08-25T09:43:55.390505+0000","last_clean":"2023-08-25T09:43:55.390505+0000","last_became_active":"2023-08-25T09:43:54.416607+0000","last_became_peered":"2023-08-25T09:43:54.416607+0000","last_unstale":"2023-08-25T09:43:55.390505+0000","last_undegraded":"2023-08-25T09:43:55.390505+0000","last_fullsized":"2023-08-25T09:43:55.390505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:56.229389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530785+0000","last_change":"2023-08-25T09:43:53.402444+0000","last_active":"2023-08-25T09:43:55.530785+0000","last_peered":"2023-08-25T09:43:55.530785+0000","last_clean":"2023-08-25T09:43:55.530785+0000","last_became_active":"2023-08-25T09:43:53.402171+0000","last_became_peered":"2023-08-25T09:43:53.402171+0000","last_unstale":"2023-08-25T09:43:55.530785+0000","last_undegraded":"2023-08-25T09:43:55.530785+0000","last_fullsized":"2023-08-25T09:43:55.530785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:15.932838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385043+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:43:55.385043+0000","last_peered":"2023-08-25T09:43:55.385043+0000","last_clean":"2023-08-25T09:43:55.385043+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:43:55.385043+0000","last_undegraded":"2023-08-25T09:43:55.385043+0000","last_fullsized":"2023-08-25T09:43:55.385043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:20:22.560940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384463+0000","last_change":"2023-08-25T09:43:54.392619+0000","last_active":"2023-08-25T09:43:55.384463+0000","last_peered":"2023-08-25T09:43:55.384463+0000","last_clean":"2023-08-25T09:43:55.384463+0000","last_became_active":"2023-08-25T09:43:54.392340+0000","last_became_peered":"2023-08-25T09:43:54.392340+0000","last_unstale":"2023-08-25T09:43:55.384463+0000","last_undegraded":"2023-08-25T09:43:55.384463+0000","last_fullsized":"2023-08-25T09:43:55.384463+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:33.252763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388517+0000","last_change":"2023-08-25T09:43:53.403913+0000","last_active":"2023-08-25T09:43:55.388517+0000","last_peered":"2023-08-25T09:43:55.388517+0000","last_clean":"2023-08-25T09:43:55.388517+0000","last_became_active":"2023-08-25T09:43:53.403680+0000","last_became_peered":"2023-08-25T09:43:53.403680+0000","last_unstale":"2023-08-25T09:43:55.388517+0000","last_undegraded":"2023-08-25T09:43:55.388517+0000","last_fullsized":"2023-08-25T09:43:55.388517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:27:24.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389312+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:43:55.389312+0000","last_peered":"2023-08-25T09:43:55.389312+0000","last_clean":"2023-08-25T09:43:55.389312+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:43:55.389312+0000","last_undegraded":"2023-08-25T09:43:55.389312+0000","last_fullsized":"2023-08-25T09:43:55.389312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:54.055742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388940+0000","last_change":"2023-08-25T09:43:54.412066+0000","last_active":"2023-08-25T09:43:55.388940+0000","last_peered":"2023-08-25T09:43:55.388940+0000","last_clean":"2023-08-25T09:43:55.388940+0000","last_became_active":"2023-08-25T09:43:54.411815+0000","last_became_peered":"2023-08-25T09:43:54.411815+0000","last_unstale":"2023-08-25T09:43:55.388940+0000","last_undegraded":"2023-08-25T09:43:55.388940+0000","last_fullsized":"2023-08-25T09:43:55.388940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:25.923883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389578+0000","last_change":"2023-08-25T09:43:53.403077+0000","last_active":"2023-08-25T09:43:55.389578+0000","last_peered":"2023-08-25T09:43:55.389578+0000","last_clean":"2023-08-25T09:43:55.389578+0000","last_became_active":"2023-08-25T09:43:53.402353+0000","last_became_peered":"2023-08-25T09:43:53.402353+0000","last_unstale":"2023-08-25T09:43:55.389578+0000","last_undegraded":"2023-08-25T09:43:55.389578+0000","last_fullsized":"2023-08-25T09:43:55.389578+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:45.178645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385518+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:43:55.385518+0000","last_peered":"2023-08-25T09:43:55.385518+0000","last_clean":"2023-08-25T09:43:55.385518+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:43:55.385518+0000","last_undegraded":"2023-08-25T09:43:55.385518+0000","last_fullsized":"2023-08-25T09:43:55.385518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:22:06.254532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530488+0000","last_change":"2023-08-25T09:43:54.412739+0000","last_active":"2023-08-25T09:43:55.530488+0000","last_peered":"2023-08-25T09:43:55.530488+0000","last_clean":"2023-08-25T09:43:55.530488+0000","last_became_active":"2023-08-25T09:43:54.412482+0000","last_became_peered":"2023-08-25T09:43:54.412482+0000","last_unstale":"2023-08-25T09:43:55.530488+0000","last_undegraded":"2023-08-25T09:43:55.530488+0000","last_fullsized":"2023-08-25T09:43:55.530488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:23.371240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389665+0000","last_change":"2023-08-25T09:43:53.388577+0000","last_active":"2023-08-25T09:43:55.389665+0000","last_peered":"2023-08-25T09:43:55.389665+0000","last_clean":"2023-08-25T09:43:55.389665+0000","last_became_active":"2023-08-25T09:43:53.387937+0000","last_became_peered":"2023-08-25T09:43:53.387937+0000","last_unstale":"2023-08-25T09:43:55.389665+0000","last_undegraded":"2023-08-25T09:43:55.389665+0000","last_fullsized":"2023-08-25T09:43:55.389665+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:12.082324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676632+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:43:55.676632+0000","last_peered":"2023-08-25T09:43:55.676632+0000","last_clean":"2023-08-25T09:43:55.676632+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:43:55.676632+0000","last_undegraded":"2023-08-25T09:43:55.676632+0000","last_fullsized":"2023-08-25T09:43:55.676632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:56.103124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388697+0000","last_change":"2023-08-25T09:43:54.416364+0000","last_active":"2023-08-25T09:43:55.388697+0000","last_peered":"2023-08-25T09:43:55.388697+0000","last_clean":"2023-08-25T09:43:55.388697+0000","last_became_active":"2023-08-25T09:43:54.415180+0000","last_became_peered":"2023-08-25T09:43:54.415180+0000","last_unstale":"2023-08-25T09:43:55.388697+0000","last_undegraded":"2023-08-25T09:43:55.388697+0000","last_fullsized":"2023-08-25T09:43:55.388697+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:18.233987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389600+0000","last_change":"2023-08-25T09:43:53.403927+0000","last_active":"2023-08-25T09:43:55.389600+0000","last_peered":"2023-08-25T09:43:55.389600+0000","last_clean":"2023-08-25T09:43:55.389600+0000","last_became_active":"2023-08-25T09:43:53.402901+0000","last_became_peered":"2023-08-25T09:43:53.402901+0000","last_unstale":"2023-08-25T09:43:55.389600+0000","last_undegraded":"2023-08-25T09:43:55.389600+0000","last_fullsized":"2023-08-25T09:43:55.389600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:47.804156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384922+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:43:55.384922+0000","last_peered":"2023-08-25T09:43:55.384922+0000","last_clean":"2023-08-25T09:43:55.384922+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:43:55.384922+0000","last_undegraded":"2023-08-25T09:43:55.384922+0000","last_fullsized":"2023-08-25T09:43:55.384922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:08:27.873205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389307+0000","last_change":"2023-08-25T09:43:54.418893+0000","last_active":"2023-08-25T09:43:55.389307+0000","last_peered":"2023-08-25T09:43:55.389307+0000","last_clean":"2023-08-25T09:43:55.389307+0000","last_became_active":"2023-08-25T09:43:54.418633+0000","last_became_peered":"2023-08-25T09:43:54.418633+0000","last_unstale":"2023-08-25T09:43:55.389307+0000","last_undegraded":"2023-08-25T09:43:55.389307+0000","last_fullsized":"2023-08-25T09:43:55.389307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:46:42.226958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678185+0000","last_change":"2023-08-25T09:43:53.401971+0000","last_active":"2023-08-25T09:43:55.678185+0000","last_peered":"2023-08-25T09:43:55.678185+0000","last_clean":"2023-08-25T09:43:55.678185+0000","last_became_active":"2023-08-25T09:43:53.401743+0000","last_became_peered":"2023-08-25T09:43:53.401743+0000","last_unstale":"2023-08-25T09:43:55.678185+0000","last_undegraded":"2023-08-25T09:43:55.678185+0000","last_fullsized":"2023-08-25T09:43:55.678185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:56.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530611+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:43:55.530611+0000","last_peered":"2023-08-25T09:43:55.530611+0000","last_clean":"2023-08-25T09:43:55.530611+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:43:55.530611+0000","last_undegraded":"2023-08-25T09:43:55.530611+0000","last_fullsized":"2023-08-25T09:43:55.530611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:01.681521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387661+0000","last_change":"2023-08-25T09:43:54.399751+0000","last_active":"2023-08-25T09:43:55.387661+0000","last_peered":"2023-08-25T09:43:55.387661+0000","last_clean":"2023-08-25T09:43:55.387661+0000","last_became_active":"2023-08-25T09:43:54.399246+0000","last_became_peered":"2023-08-25T09:43:54.399246+0000","last_unstale":"2023-08-25T09:43:55.387661+0000","last_undegraded":"2023-08-25T09:43:55.387661+0000","last_fullsized":"2023-08-25T09:43:55.387661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:24.732531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677545+0000","last_change":"2023-08-25T09:43:53.400796+0000","last_active":"2023-08-25T09:43:55.677545+0000","last_peered":"2023-08-25T09:43:55.677545+0000","last_clean":"2023-08-25T09:43:55.677545+0000","last_became_active":"2023-08-25T09:43:53.400551+0000","last_became_peered":"2023-08-25T09:43:53.400551+0000","last_unstale":"2023-08-25T09:43:55.677545+0000","last_undegraded":"2023-08-25T09:43:55.677545+0000","last_fullsized":"2023-08-25T09:43:55.677545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:51:58.400833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385228+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:43:55.385228+0000","last_peered":"2023-08-25T09:43:55.385228+0000","last_clean":"2023-08-25T09:43:55.385228+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:43:55.385228+0000","last_undegraded":"2023-08-25T09:43:55.385228+0000","last_fullsized":"2023-08-25T09:43:55.385228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:47.623306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194787,"statfs":{"total":200000000,"available":199462687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1910000000000001,"15min":1.0660000000000001},"min":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.54,"5min":10.747,"15min":10.747},"last":0.752},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.86899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0409999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1859999999999999,"15min":1.129},"min":{"1min":0.88,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1799999999999999,"15min":1.0920000000000001},"min":{"1min":0.76500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5820000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1559999999999999,"15min":1.079},"min":{"1min":0.75600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.27,"15min":1.2170000000000001},"min":{"1min":1.0620000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.708}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.81000000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3839999999999999,"15min":1.329},"min":{"1min":1.0629999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.639}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7689999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.286},"min":{"1min":1.05,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7609999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.429,"15min":1.3740000000000001},"min":{"1min":1.04,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9830000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.462}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5069999999999999,"15min":1.4319999999999999},"min":{"1min":1.1439999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8740000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.579},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.472,"15min":1.3979999999999999},"min":{"1min":1.276,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.6160000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194787,"statfs":{"total":200000000,"available":199462667,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.1399999999999999,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.224},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1910000000000001,"15min":1.145},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7649999999999999,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":0.752}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2290000000000001,"15min":1.1950000000000001},"min":{"1min":0.79300000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.7050000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.1830000000000001,"15min":1.117},"min":{"1min":0.94799999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.337}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1659999999999999,"15min":1.105},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.482,"5min":1.708,"15min":1.708},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.409,"5min":1.3620000000000001,"15min":1.204},"min":{"1min":0.93400000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.768,"5min":13.193,"15min":13.193},"last":1.4159999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.302,"15min":1.222},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.121,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.272,"15min":1.256},"min":{"1min":0.98099999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.548,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.24},"min":{"1min":0.95999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.607,"5min":2.484,"15min":2.484},"last":1.528},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.385,"15min":1.3169999999999999},"min":{"1min":0.89200000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.02,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.177}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.2749999999999999},"min":{"1min":1.034,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.837,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.272},{"interface":"front","average":{"1min":1.532,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":1.1990000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7210000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4750000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.538,"15min":1.49},"min":{"1min":0.89600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.2240000000000002,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.472},"min":{"1min":1.244,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.3879999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1100,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194211,"statfs":{"total":200000000,"available":198872881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1930000000000001,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.577,"5min":2.081,"15min":2.081},"last":1.738},{"interface":"front","average":{"1min":1.27,"5min":1.22,"15min":1.208},"min":{"1min":0.76200000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5469999999999999,"5min":1.974,"15min":1.974},"last":1.585}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.145,"15min":1.1950000000000001},"min":{"1min":0.746,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.577,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.24},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1819999999999999,"15min":1.119},"min":{"1min":0.997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5649999999999999,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.8640000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.238,"15min":1.1859999999999999},"min":{"1min":0.94899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.665,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.012},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":1.048,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6259999999999999,"5min":1.962,"15min":1.962},"last":1.778}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4059999999999999,"15min":1.3160000000000001},"min":{"1min":1.0549999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3439999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.177},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":1.024,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.552,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.367}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.355,"15min":1.3029999999999999},"min":{"1min":0.96199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7490000000000001,"5min":2.625,"15min":2.625},"last":1.9530000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.3720000000000001},"min":{"1min":0.94499999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.857,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.4419999999999999,"15min":1.4330000000000001},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.77,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.512,"15min":1.417},"min":{"1min":1.177,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1059999999999999,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.478}]},{"osd":7,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.4770000000000001,"15min":1.4450000000000001},"min":{"1min":1.2869999999999999,"5min":0.63,"15min":0.63},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.587,"15min":1.579},"min":{"1min":1.4319999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.7050000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1759999999999999,"15min":1.1499999999999999},"min":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5329999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.1459999999999999},"min":{"1min":1.026,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5289999999999999,"5min":7.367,"15min":7.367},"last":1.1719999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1359999999999999,"15min":1.1220000000000001},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.996},{"interface":"front","average":{"1min":1.173,"5min":1.244,"15min":1.401},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5249999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.4650000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.208,"15min":1.302},"min":{"1min":0.93200000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.244,"15min":1.167},"min":{"1min":0.95899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.631,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.121}]},{"osd":3,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.292,"15min":1.23},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7110000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.371},"min":{"1min":1.1439999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.778,"5min":1.911,"15min":1.911},"last":1.2090000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.86599999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.276},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3280000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.829,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.423,"15min":1.4350000000000001},"min":{"1min":0.81100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0219999999999998,"5min":2.863,"15min":2.863},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.548,"15min":1.5600000000000001},"min":{"1min":0.97799999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9730000000000001,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.5429999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.55,"15min":1.4179999999999999},"min":{"1min":1.1910000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.355},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.593,"15min":1.595},"min":{"1min":1.3320000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1230000000000002,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.587}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":525,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194786,"statfs":{"total":200000000,"available":199461623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.86699999999999999,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.256},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.95399999999999996,"15min":0.89800000000000002},"min":{"1min":0.65900000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5349999999999999,"5min":1.55,"15min":1.55},"last":0.88900000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.871,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.153,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.004,"15min":0.97699999999999998},"min":{"1min":0.58699999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4910000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.139}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.96199999999999997,"15min":0.87},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":1.573,"5min":1.919,"15min":1.919},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1020000000000001,"15min":1.1080000000000001},"min":{"1min":0.81000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6140000000000001,"5min":3.399,"15min":3.399},"last":1.345}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.214,"15min":1.1890000000000001},"min":{"1min":0.83499999999999996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5800000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.105},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.196},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.405,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3120000000000001},"min":{"1min":0.98899999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6699999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.2849999999999999,"15min":1.2689999999999999},"min":{"1min":0.92000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5980000000000001,"5min":1.788,"15min":1.788},"last":1.3069999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.282,"15min":1.242},"min":{"1min":1.0389999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.917,"15min":1.917},"last":1.048},{"interface":"front","average":{"1min":1.45,"5min":1.397,"15min":1.405},"min":{"1min":1.2230000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.734,"5min":3.496,"15min":3.496},"last":1.448}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.085,"5min":0.871,"15min":0.871},"max":{"1min":1.881,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":0.98899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.669,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.1890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463339,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.82499999999999996},"min":{"1min":0.53300000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.4199999999999999,"5min":2.383,"15min":2.383},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.425,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.411,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.238},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5649999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.284}]},{"osd":3,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5429999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.081,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.486,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.3220000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3080000000000001,"15min":1.397},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.1309999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.266,"15min":1.375},"min":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.369,"15min":1.397},"min":{"1min":1.081,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9890000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.444}]},{"osd":6,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.381,"15min":1.4199999999999999},"min":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.875,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.425},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.482},"min":{"1min":0.995,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.097,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.7070000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.383,"15min":1.4490000000000001},"min":{"1min":0.89600000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7829999999999999,"5min":2.968,"15min":2.968},"last":1.478},{"interface":"front","average":{"1min":1.409,"5min":1.4199999999999999,"15min":1.504},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.056,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.8200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1102,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194210,"statfs":{"total":200000000,"available":198871325,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.77200000000000002},"min":{"1min":0.45900000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2250000000000001,"5min":1.627,"15min":1.627},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.876,"15min":0.78400000000000003},"min":{"1min":0.56799999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6850000000000001,"5min":1.895,"15min":1.895},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.79900000000000004},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.45,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.048,"5min":0.97499999999999998,"15min":0.90300000000000002},"min":{"1min":0.51500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4239999999999999,"5min":1.746,"15min":1.746},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1339999999999999,"15min":1.038},"min":{"1min":0.61199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.639,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.008,"15min":0.93999999999999995},"min":{"1min":0.67900000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4750000000000001,"5min":1.764,"15min":1.764},"last":1.034}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.155,"15min":1.087},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.117},{"interface":"front","average":{"1min":1.369,"5min":1.3100000000000001,"15min":1.218},"min":{"1min":0.876,"5min":0.629,"15min":0.629},"max":{"1min":1.641,"5min":2.008,"15min":2.008},"last":1.2809999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1890000000000001},"min":{"1min":0.80000000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7909999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2170000000000001,"15min":1.1679999999999999},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.738,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.401}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3799999999999999,"15min":1.298},"min":{"1min":1.042,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9930000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.82199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8380000000000001,"5min":2.004,"15min":2.004},"last":1.341}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":1.145,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7030000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.3899999999999999,"15min":1.3080000000000001},"min":{"1min":0.93500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9039999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.2969999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90700000000000003,"15min":0.85299999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.355,"5min":11.257,"15min":11.257},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.97099999999999997,"15min":0.95799999999999996},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7110000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.88}]},{"osd":2,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.38800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0029999999999999,"15min":1.0940000000000001},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5640000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":1.085}]},{"osd":3,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.002},{"interface":"front","average":{"1min":1.012,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5609999999999999,"5min":1.869,"15min":1.869},"last":1.3700000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.327},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3340000000000001,"15min":1.353},"min":{"1min":0.83299999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7390000000000001,"5min":1.982,"15min":1.982},"last":1.4910000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.256,"15min":1.3580000000000001},"min":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8340000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.81100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.246}]},{"osd":6,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6579999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.296},"min":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3660000000000001,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0779999999999998,"5min":2.121,"15min":2.121},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.319},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.1870000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:43:59.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-25T09:44:00.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.075+0000 7fc8348ec640 1 Processor -- start 2023-08-25T09:44:00.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.075+0000 7fc8348ec640 1 -- start start 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc8348ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8301644d0 0x7fc8301621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc8348ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc83006ae20 0x7fc83006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc8348ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83006f6c0 con 0x7fc83006f290 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc8348ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83006c1d0 con 0x7fc8301644d0 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc8348ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83006b5c0 con 0x7fc83006ae20 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc82e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8301644d0 0x7fc8301621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc82e59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8301644d0 0x7fc8301621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45294/0 (socket says 172.21.15.70:45294) 2023-08-25T09:44:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.077+0000 7fc82e59c640 1 -- 172.21.15.70:0/329944718 learned_addr learned my addr 172.21.15.70:0/329944718 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.078+0000 7fc82e59c640 1 -- 172.21.15.70:0/329944718 >> v2:172.21.15.155:3300/0 conn(0x7fc83006ae20 msgr2=0x7fc83006ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.078+0000 7fc82e59c640 1 --2- 172.21.15.70:0/329944718 >> v2:172.21.15.155:3300/0 conn(0x7fc83006ae20 0x7fc83006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.078+0000 7fc82e59c640 1 -- 172.21.15.70:0/329944718 >> v1:172.21.15.155:6789/0 conn(0x7fc83006f290 legacy=0x7fc830164c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:00.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.078+0000 7fc82e59c640 1 -- 172.21.15.70:0/329944718 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83006b8a0 con 0x7fc8301644d0 2023-08-25T09:44:00.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.079+0000 7fc82e59c640 1 --2- 172.21.15.70:0/329944718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc8301621f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc8300b1150 tx=0x7fc82002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65c5ed53b3e1696 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.079+0000 7fc82d59a640 1 -- 172.21.15.70:0/329944718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820004230 con 0x7fc8301644d0 2023-08-25T09:44:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc82d59a640 1 -- 172.21.15.70:0/329944718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc820004390 con 0x7fc8301644d0 2023-08-25T09:44:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc82d59a640 1 -- 172.21.15.70:0/329944718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820003810 con 0x7fc8301644d0 2023-08-25T09:44:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 -- 172.21.15.70:0/329944718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 msgr2=0x7fc8301621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 --2- 172.21.15.70:0/329944718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc8301621f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fc8300b1150 tx=0x7fc82002b8f0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 -- 172.21.15.70:0/329944718 shutdown_connections 2023-08-25T09:44:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 --2- 172.21.15.70:0/329944718 >> v2:172.21.15.155:3300/0 conn(0x7fc83006ae20 0x7fc83006ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 --2- 172.21.15.70:0/329944718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc8301621f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 -- 172.21.15.70:0/329944718 >> 172.21.15.70:0/329944718 conn(0x7fc8300ce760 msgr2=0x7fc8300ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 -- 172.21.15.70:0/329944718 shutdown_connections 2023-08-25T09:44:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.080+0000 7fc8348ec640 1 -- 172.21.15.70:0/329944718 wait complete. 2023-08-25T09:44:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.082+0000 7fc8348ec640 1 Processor -- start 2023-08-25T09:44:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.082+0000 7fc8348ec640 1 -- start start 2023-08-25T09:44:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.083+0000 7fc8348ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83006ae20 0x7fc83013f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.083+0000 7fc8348ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc83006f290 0x7fc830141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.083+0000 7fc8348ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc83015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.083+0000 7fc8348ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83013ffb0 con 0x7fc8301644d0 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.083+0000 7fc8348ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc830140290 con 0x7fc83006ae20 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.083+0000 7fc8348ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc830140570 con 0x7fc83006f290 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc83015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc83015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45306/0 (socket says 172.21.15.70:45306) 2023-08-25T09:44:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 -- 172.21.15.70:0/1324695798 learned_addr learned my addr 172.21.15.70:0/1324695798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82dd9b640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc83006f290 0x7fc830141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82e59c640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83006ae20 0x7fc83013f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 -- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc83006f290 msgr2=0x7fc830141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc83006f290 0x7fc830141050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 -- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83006ae20 msgr2=0x7fc83013f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83006ae20 0x7fc83013f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82ed9d640 1 -- 172.21.15.70:0/1324695798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820009460 con 0x7fc8301644d0 2023-08-25T09:44:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.084+0000 7fc82e59c640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83006ae20 0x7fc83013f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.085+0000 7fc82ed9d640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc83015d5d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c780 tx=0x7fc82800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.085+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828004b60 con 0x7fc8301644d0 2023-08-25T09:44:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.085+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800ce90 con 0x7fc8301644d0 2023-08-25T09:44:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.085+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800a850 con 0x7fc8301644d0 2023-08-25T09:44:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.085+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc830140850 con 0x7fc8301644d0 2023-08-25T09:44:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.085+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83013f860 con 0x7fc8301644d0 2023-08-25T09:44:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.086+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83004e420 con 0x7fc8301644d0 2023-08-25T09:44:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.087+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc82800a9b0 con 0x7fc8301644d0 2023-08-25T09:44:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.088+0000 7fc81effd640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc81006a050 0x7fc81006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.088+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc828088770 con 0x7fc8301644d0 2023-08-25T09:44:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.088+0000 7fc82e59c640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc81006a050 0x7fc81006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.089+0000 7fc82e59c640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc81006a050 0x7fc81006c510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc820035000 tx=0x7fc820008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.092+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc828053e50 con 0x7fc8301644d0 2023-08-25T09:44:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.351+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc8300cf670 con 0x7fc8301644d0 2023-08-25T09:44:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.351+0000 7fc81effd640 1 -- 172.21.15.70:0/1324695798 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1008 (secure 0 0 0) 0x7fc828057470 con 0x7fc8301644d0 2023-08-25T09:44:00.353 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:00.354 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":327,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":94,"num_osds":8,"num_up_osds":8,"osd_up_since":1692956318,"num_in_osds":8,"osd_in_since":1692956315,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592778,"bytes_used":5478844,"bytes_avail":1594521156,"bytes_total":1600000000,"write_bytes_sec":2371,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":41,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6780}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-08-25T09:43:29.413036+0000","services":{}},"progress_events":{}} 2023-08-25T09:44:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc81006a050 msgr2=0x7fc81006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc81006a050 0x7fc81006c510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc820035000 tx=0x7fc820008030 comp rx=0 tx=0).stop 2023-08-25T09:44:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 msgr2=0x7fc83015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc83015d5d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c780 tx=0x7fc82800cc40 comp rx=0 tx=0).stop 2023-08-25T09:44:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 shutdown_connections 2023-08-25T09:44:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc81006a050 0x7fc81006c510 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8301644d0 0x7fc83015d5d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc83006f290 0x7fc830141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 --2- 172.21.15.70:0/1324695798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc83006ae20 0x7fc83013f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 >> 172.21.15.70:0/1324695798 conn(0x7fc8300ce760 msgr2=0x7fc830145c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.354+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 shutdown_connections 2023-08-25T09:44:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.355+0000 7fc8348ec640 1 -- 172.21.15.70:0/1324695798 wait complete. 2023-08-25T09:44:00.373 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-25T09:44:00.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:44:00.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.581+0000 7fc872779640 1 Processor -- start 2023-08-25T09:44:00.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.582+0000 7fc872779640 1 -- start start 2023-08-25T09:44:00.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.583+0000 7fc872779640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc86c15e830 0x7fc86c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.583+0000 7fc872779640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 0x7fc86c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.583+0000 7fc872779640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0b1b60 con 0x7fc86c15c6d0 2023-08-25T09:44:00.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.583+0000 7fc872779640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c06bd80 con 0x7fc86c15e830 2023-08-25T09:44:00.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.583+0000 7fc872779640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c06b7c0 con 0x7fc86c06b2b0 2023-08-25T09:44:00.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc86c15e830 0x7fc86c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc86c15e830 0x7fc86c06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45310/0 (socket says 172.21.15.70:45310) 2023-08-25T09:44:00.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 -- 172.21.15.70:0/927589830 learned_addr learned my addr 172.21.15.70:0/927589830 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:00.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 -- 172.21.15.70:0/927589830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 msgr2=0x7fc86c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:00.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 --2- 172.21.15.70:0/927589830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 0x7fc86c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 -- 172.21.15.70:0/927589830 >> v1:172.21.15.155:6789/0 conn(0x7fc86c15c6d0 legacy=0x7fc86c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:00.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.584+0000 7fc86bfff640 1 -- 172.21.15.70:0/927589830 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c06baa0 con 0x7fc86c15e830 2023-08-25T09:44:00.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.585+0000 7fc86bfff640 1 --2- 172.21.15.70:0/927589830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c06ad70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009870 tx=0x7fc85c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a7c620b0e48fa5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:00.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc86b7fe640 1 -- 172.21.15.70:0/927589830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c031640 con 0x7fc86c15e830 2023-08-25T09:44:00.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc86b7fe640 1 -- 172.21.15.70:0/927589830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0317a0 con 0x7fc86c15e830 2023-08-25T09:44:00.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc86b7fe640 1 -- 172.21.15.70:0/927589830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c031b30 con 0x7fc86c15e830 2023-08-25T09:44:00.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc872779640 1 -- 172.21.15.70:0/927589830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 msgr2=0x7fc86c06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc872779640 1 --2- 172.21.15.70:0/927589830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c06ad70 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009870 tx=0x7fc85c02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc872779640 1 -- 172.21.15.70:0/927589830 shutdown_connections 2023-08-25T09:44:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc872779640 1 --2- 172.21.15.70:0/927589830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 0x7fc86c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc872779640 1 --2- 172.21.15.70:0/927589830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c06ad70 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.586+0000 7fc872779640 1 -- 172.21.15.70:0/927589830 >> 172.21.15.70:0/927589830 conn(0x7fc86c069850 msgr2=0x7fc86c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.587+0000 7fc872779640 1 -- 172.21.15.70:0/927589830 shutdown_connections 2023-08-25T09:44:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.587+0000 7fc872779640 1 -- 172.21.15.70:0/927589830 wait complete. 2023-08-25T09:44:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.589+0000 7fc872779640 1 Processor -- start 2023-08-25T09:44:00.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.589+0000 7fc872779640 1 -- start start 2023-08-25T09:44:00.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc872779640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 0x7fc86c150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc872779640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c15c6d0 0x7fc86c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc872779640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc872779640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1505b0 con 0x7fc86c15e830 2023-08-25T09:44:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc872779640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c06fa60 con 0x7fc86c15c6d0 2023-08-25T09:44:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc872779640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c06fd40 con 0x7fc86c06b2b0 2023-08-25T09:44:00.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc870d16640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc870d16640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c0ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45322/0 (socket says 172.21.15.70:45322) 2023-08-25T09:44:00.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.590+0000 7fc870d16640 1 -- 172.21.15.70:0/3353377756 learned_addr learned my addr 172.21.15.70:0/3353377756 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc870d16640 1 -- 172.21.15.70:0/3353377756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 msgr2=0x7fc86c150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc870d16640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 0x7fc86c150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc870d16640 1 -- 172.21.15.70:0/3353377756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c15c6d0 msgr2=0x7fc86c0ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc870d16640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c15c6d0 0x7fc86c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc870d16640 1 -- 172.21.15.70:0/3353377756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c0094d0 con 0x7fc86c15e830 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc870d16640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c0ae3b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c780 tx=0x7fc86400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:00.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864003d80 con 0x7fc86c15e830 2023-08-25T09:44:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400ce90 con 0x7fc86c15e830 2023-08-25T09:44:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640052b0 con 0x7fc86c15e830 2023-08-25T09:44:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.591+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c070020 con 0x7fc86c15e830 2023-08-25T09:44:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.592+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c0705e0 con 0x7fc86c15e830 2023-08-25T09:44:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.593+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c0707f0 con 0x7fc86c15e830 2023-08-25T09:44:00.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.594+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc864005410 con 0x7fc86c15e830 2023-08-25T09:44:00.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.595+0000 7fc8697fa640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc84c06a080 0x7fc84c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:00.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.595+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc864087930 con 0x7fc86c15e830 2023-08-25T09:44:00.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.595+0000 7fc86bfff640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc84c06a080 0x7fc84c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:00.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.596+0000 7fc86bfff640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc84c06a080 0x7fc84c06c540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc85c036000 tx=0x7fc85c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.599+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc864053010 con 0x7fc86c15e830 2023-08-25T09:44:00.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.783+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc86c0af210 con 0x7fc84c06a080 2023-08-25T09:44:00.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.792+0000 7fc8697fa640 1 -- 172.21.15.70:0/3353377756 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302945 (secure 0 0 0) 0x7fc86c0af210 con 0x7fc84c06a080 2023-08-25T09:44:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:44:00.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc84c06a080 msgr2=0x7fc84c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc84c06a080 0x7fc84c06c540 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc85c036000 tx=0x7fc85c030000 comp rx=0 tx=0).stop 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 msgr2=0x7fc86c0ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c0ae3b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc86400c780 tx=0x7fc86400cc40 comp rx=0 tx=0).stop 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 shutdown_connections 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc84c06a080 0x7fc84c06c540 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c15e830 0x7fc86c0ae3b0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c15c6d0 0x7fc86c0ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 --2- 172.21.15.70:0/3353377756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c06b2b0 0x7fc86c150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 >> 172.21.15.70:0/3353377756 conn(0x7fc86c069850 msgr2=0x7fc86c0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 shutdown_connections 2023-08-25T09:44:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:00.797+0000 7fc872779640 1 -- 172.21.15.70:0/3353377756 wait complete. 2023-08-25T09:44:00.819 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2023-08-25T09:43:59.420989+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5350,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557149,"statfs":{"total":1600000000,"available":1594521156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.044672"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529053+0000","last_change":"2023-08-25T09:43:54.404594+0000","last_active":"2023-08-25T09:43:55.529053+0000","last_peered":"2023-08-25T09:43:55.529053+0000","last_clean":"2023-08-25T09:43:55.529053+0000","last_became_active":"2023-08-25T09:43:54.404347+0000","last_became_peered":"2023-08-25T09:43:54.404347+0000","last_unstale":"2023-08-25T09:43:55.529053+0000","last_undegraded":"2023-08-25T09:43:55.529053+0000","last_fullsized":"2023-08-25T09:43:55.529053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:33.114359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.386937+0000","last_change":"2023-08-25T09:43:54.408980+0000","last_active":"2023-08-25T09:43:55.386937+0000","last_peered":"2023-08-25T09:43:55.386937+0000","last_clean":"2023-08-25T09:43:55.386937+0000","last_became_active":"2023-08-25T09:43:54.408724+0000","last_became_peered":"2023-08-25T09:43:54.408724+0000","last_unstale":"2023-08-25T09:43:55.386937+0000","last_undegraded":"2023-08-25T09:43:55.386937+0000","last_fullsized":"2023-08-25T09:43:55.386937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:27.591271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388658+0000","last_change":"2023-08-25T09:43:54.405905+0000","last_active":"2023-08-25T09:43:55.388658+0000","last_peered":"2023-08-25T09:43:55.388658+0000","last_clean":"2023-08-25T09:43:55.388658+0000","last_became_active":"2023-08-25T09:43:54.398170+0000","last_became_peered":"2023-08-25T09:43:54.398170+0000","last_unstale":"2023-08-25T09:43:55.388658+0000","last_undegraded":"2023-08-25T09:43:55.388658+0000","last_fullsized":"2023-08-25T09:43:55.388658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:17:42.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676440+0000","last_change":"2023-08-25T09:43:54.401703+0000","last_active":"2023-08-25T09:43:55.676440+0000","last_peered":"2023-08-25T09:43:55.676440+0000","last_clean":"2023-08-25T09:43:55.676440+0000","last_became_active":"2023-08-25T09:43:54.401389+0000","last_became_peered":"2023-08-25T09:43:54.401389+0000","last_unstale":"2023-08-25T09:43:55.676440+0000","last_undegraded":"2023-08-25T09:43:55.676440+0000","last_fullsized":"2023-08-25T09:43:55.676440+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:55.776311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389283+0000","last_change":"2023-08-25T09:43:54.416278+0000","last_active":"2023-08-25T09:43:55.389283+0000","last_peered":"2023-08-25T09:43:55.389283+0000","last_clean":"2023-08-25T09:43:55.389283+0000","last_became_active":"2023-08-25T09:43:54.415330+0000","last_became_peered":"2023-08-25T09:43:54.415330+0000","last_unstale":"2023-08-25T09:43:55.389283+0000","last_undegraded":"2023-08-25T09:43:55.389283+0000","last_fullsized":"2023-08-25T09:43:55.389283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:25.392124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675633+0000","last_change":"2023-08-25T09:43:54.413935+0000","last_active":"2023-08-25T09:43:55.675633+0000","last_peered":"2023-08-25T09:43:55.675633+0000","last_clean":"2023-08-25T09:43:55.675633+0000","last_became_active":"2023-08-25T09:43:54.413685+0000","last_became_peered":"2023-08-25T09:43:54.413685+0000","last_unstale":"2023-08-25T09:43:55.675633+0000","last_undegraded":"2023-08-25T09:43:55.675633+0000","last_fullsized":"2023-08-25T09:43:55.675633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:09.188639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677337+0000","last_change":"2023-08-25T09:43:54.409395+0000","last_active":"2023-08-25T09:43:55.677337+0000","last_peered":"2023-08-25T09:43:55.677337+0000","last_clean":"2023-08-25T09:43:55.677337+0000","last_became_active":"2023-08-25T09:43:54.409083+0000","last_became_peered":"2023-08-25T09:43:54.409083+0000","last_unstale":"2023-08-25T09:43:55.677337+0000","last_undegraded":"2023-08-25T09:43:55.677337+0000","last_fullsized":"2023-08-25T09:43:55.677337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:27.799378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530267+0000","last_change":"2023-08-25T09:43:54.407151+0000","last_active":"2023-08-25T09:43:55.530267+0000","last_peered":"2023-08-25T09:43:55.530267+0000","last_clean":"2023-08-25T09:43:55.530267+0000","last_became_active":"2023-08-25T09:43:54.406266+0000","last_became_peered":"2023-08-25T09:43:54.406266+0000","last_unstale":"2023-08-25T09:43:55.530267+0000","last_undegraded":"2023-08-25T09:43:55.530267+0000","last_fullsized":"2023-08-25T09:43:55.530267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:46:18.101385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676097+0000","last_change":"2023-08-25T09:43:54.408942+0000","last_active":"2023-08-25T09:43:55.676097+0000","last_peered":"2023-08-25T09:43:55.676097+0000","last_clean":"2023-08-25T09:43:55.676097+0000","last_became_active":"2023-08-25T09:43:54.408439+0000","last_became_peered":"2023-08-25T09:43:54.408439+0000","last_unstale":"2023-08-25T09:43:55.676097+0000","last_undegraded":"2023-08-25T09:43:55.676097+0000","last_fullsized":"2023-08-25T09:43:55.676097+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:53.991517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529111+0000","last_change":"2023-08-25T09:43:54.419056+0000","last_active":"2023-08-25T09:43:55.529111+0000","last_peered":"2023-08-25T09:43:55.529111+0000","last_clean":"2023-08-25T09:43:55.529111+0000","last_became_active":"2023-08-25T09:43:54.418667+0000","last_became_peered":"2023-08-25T09:43:54.418667+0000","last_unstale":"2023-08-25T09:43:55.529111+0000","last_undegraded":"2023-08-25T09:43:55.529111+0000","last_fullsized":"2023-08-25T09:43:55.529111+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:30.225126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387796+0000","last_change":"2023-08-25T09:43:54.415935+0000","last_active":"2023-08-25T09:43:55.387796+0000","last_peered":"2023-08-25T09:43:55.387796+0000","last_clean":"2023-08-25T09:43:55.387796+0000","last_became_active":"2023-08-25T09:43:54.414902+0000","last_became_peered":"2023-08-25T09:43:54.414902+0000","last_unstale":"2023-08-25T09:43:55.387796+0000","last_undegraded":"2023-08-25T09:43:55.387796+0000","last_fullsized":"2023-08-25T09:43:55.387796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:05:26.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676861+0000","last_change":"2023-08-25T09:43:54.404468+0000","last_active":"2023-08-25T09:43:55.676861+0000","last_peered":"2023-08-25T09:43:55.676861+0000","last_clean":"2023-08-25T09:43:55.676861+0000","last_became_active":"2023-08-25T09:43:54.403839+0000","last_became_peered":"2023-08-25T09:43:54.403839+0000","last_unstale":"2023-08-25T09:43:55.676861+0000","last_undegraded":"2023-08-25T09:43:55.676861+0000","last_fullsized":"2023-08-25T09:43:55.676861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:25.809595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387622+0000","last_change":"2023-08-25T09:43:54.408393+0000","last_active":"2023-08-25T09:43:55.387622+0000","last_peered":"2023-08-25T09:43:55.387622+0000","last_clean":"2023-08-25T09:43:55.387622+0000","last_became_active":"2023-08-25T09:43:54.407528+0000","last_became_peered":"2023-08-25T09:43:54.407528+0000","last_unstale":"2023-08-25T09:43:55.387622+0000","last_undegraded":"2023-08-25T09:43:55.387622+0000","last_fullsized":"2023-08-25T09:43:55.387622+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:46.864971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530141+0000","last_change":"2023-08-25T09:43:54.409118+0000","last_active":"2023-08-25T09:43:55.530141+0000","last_peered":"2023-08-25T09:43:55.530141+0000","last_clean":"2023-08-25T09:43:55.530141+0000","last_became_active":"2023-08-25T09:43:54.408893+0000","last_became_peered":"2023-08-25T09:43:54.408893+0000","last_unstale":"2023-08-25T09:43:55.530141+0000","last_undegraded":"2023-08-25T09:43:55.530141+0000","last_fullsized":"2023-08-25T09:43:55.530141+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:12.464768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389643+0000","last_change":"2023-08-25T09:43:54.417738+0000","last_active":"2023-08-25T09:43:55.389643+0000","last_peered":"2023-08-25T09:43:55.389643+0000","last_clean":"2023-08-25T09:43:55.389643+0000","last_became_active":"2023-08-25T09:43:54.415654+0000","last_became_peered":"2023-08-25T09:43:54.415654+0000","last_unstale":"2023-08-25T09:43:55.389643+0000","last_undegraded":"2023-08-25T09:43:55.389643+0000","last_fullsized":"2023-08-25T09:43:55.389643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:35.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677379+0000","last_change":"2023-08-25T09:43:54.394213+0000","last_active":"2023-08-25T09:43:55.677379+0000","last_peered":"2023-08-25T09:43:55.677379+0000","last_clean":"2023-08-25T09:43:55.677379+0000","last_became_active":"2023-08-25T09:43:54.393647+0000","last_became_peered":"2023-08-25T09:43:54.393647+0000","last_unstale":"2023-08-25T09:43:55.677379+0000","last_undegraded":"2023-08-25T09:43:55.677379+0000","last_fullsized":"2023-08-25T09:43:55.677379+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:39.470885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530362+0000","last_change":"2023-08-25T09:43:54.408147+0000","last_active":"2023-08-25T09:43:55.530362+0000","last_peered":"2023-08-25T09:43:55.530362+0000","last_clean":"2023-08-25T09:43:55.530362+0000","last_became_active":"2023-08-25T09:43:54.407786+0000","last_became_peered":"2023-08-25T09:43:54.407786+0000","last_unstale":"2023-08-25T09:43:55.530362+0000","last_undegraded":"2023-08-25T09:43:55.530362+0000","last_fullsized":"2023-08-25T09:43:55.530362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:24.779031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387994+0000","last_change":"2023-08-25T09:43:54.407208+0000","last_active":"2023-08-25T09:43:55.387994+0000","last_peered":"2023-08-25T09:43:55.387994+0000","last_clean":"2023-08-25T09:43:55.387994+0000","last_became_active":"2023-08-25T09:43:54.406324+0000","last_became_peered":"2023-08-25T09:43:54.406324+0000","last_unstale":"2023-08-25T09:43:55.387994+0000","last_undegraded":"2023-08-25T09:43:55.387994+0000","last_fullsized":"2023-08-25T09:43:55.387994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:00.393632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.531168+0000","last_change":"2023-08-25T09:43:54.415832+0000","last_active":"2023-08-25T09:43:55.531168+0000","last_peered":"2023-08-25T09:43:55.531168+0000","last_clean":"2023-08-25T09:43:55.531168+0000","last_became_active":"2023-08-25T09:43:54.415552+0000","last_became_peered":"2023-08-25T09:43:54.415552+0000","last_unstale":"2023-08-25T09:43:55.531168+0000","last_undegraded":"2023-08-25T09:43:55.531168+0000","last_fullsized":"2023-08-25T09:43:55.531168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:29.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676471+0000","last_change":"2023-08-25T09:43:54.402938+0000","last_active":"2023-08-25T09:43:55.676471+0000","last_peered":"2023-08-25T09:43:55.676471+0000","last_clean":"2023-08-25T09:43:55.676471+0000","last_became_active":"2023-08-25T09:43:54.400897+0000","last_became_peered":"2023-08-25T09:43:54.400897+0000","last_unstale":"2023-08-25T09:43:55.676471+0000","last_undegraded":"2023-08-25T09:43:55.676471+0000","last_fullsized":"2023-08-25T09:43:55.676471+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:44.889724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676124+0000","last_change":"2023-08-25T09:43:54.412294+0000","last_active":"2023-08-25T09:43:55.676124+0000","last_peered":"2023-08-25T09:43:55.676124+0000","last_clean":"2023-08-25T09:43:55.676124+0000","last_became_active":"2023-08-25T09:43:54.412114+0000","last_became_peered":"2023-08-25T09:43:54.412114+0000","last_unstale":"2023-08-25T09:43:55.676124+0000","last_undegraded":"2023-08-25T09:43:55.676124+0000","last_fullsized":"2023-08-25T09:43:55.676124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:42:21.825196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387480+0000","last_change":"2023-08-25T09:43:54.408210+0000","last_active":"2023-08-25T09:43:55.387480+0000","last_peered":"2023-08-25T09:43:55.387480+0000","last_clean":"2023-08-25T09:43:55.387480+0000","last_became_active":"2023-08-25T09:43:54.407960+0000","last_became_peered":"2023-08-25T09:43:54.407960+0000","last_unstale":"2023-08-25T09:43:55.387480+0000","last_undegraded":"2023-08-25T09:43:55.387480+0000","last_fullsized":"2023-08-25T09:43:55.387480+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:13.932322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.528897+0000","last_change":"2023-08-25T09:43:54.402889+0000","last_active":"2023-08-25T09:43:55.528897+0000","last_peered":"2023-08-25T09:43:55.528897+0000","last_clean":"2023-08-25T09:43:55.528897+0000","last_became_active":"2023-08-25T09:43:54.402611+0000","last_became_peered":"2023-08-25T09:43:54.402611+0000","last_unstale":"2023-08-25T09:43:55.528897+0000","last_undegraded":"2023-08-25T09:43:55.528897+0000","last_fullsized":"2023-08-25T09:43:55.528897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:15.783761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.386759+0000","last_change":"2023-08-25T09:43:54.407433+0000","last_active":"2023-08-25T09:43:55.386759+0000","last_peered":"2023-08-25T09:43:55.386759+0000","last_clean":"2023-08-25T09:43:55.386759+0000","last_became_active":"2023-08-25T09:43:54.407190+0000","last_became_peered":"2023-08-25T09:43:54.407190+0000","last_unstale":"2023-08-25T09:43:55.386759+0000","last_undegraded":"2023-08-25T09:43:55.386759+0000","last_fullsized":"2023-08-25T09:43:55.386759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:27.119175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675738+0000","last_change":"2023-08-25T09:43:54.414015+0000","last_active":"2023-08-25T09:43:55.675738+0000","last_peered":"2023-08-25T09:43:55.675738+0000","last_clean":"2023-08-25T09:43:55.675738+0000","last_became_active":"2023-08-25T09:43:54.412759+0000","last_became_peered":"2023-08-25T09:43:54.412759+0000","last_unstale":"2023-08-25T09:43:55.675738+0000","last_undegraded":"2023-08-25T09:43:55.675738+0000","last_fullsized":"2023-08-25T09:43:55.675738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:18.576641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677241+0000","last_change":"2023-08-25T09:43:54.412030+0000","last_active":"2023-08-25T09:43:55.677241+0000","last_peered":"2023-08-25T09:43:55.677241+0000","last_clean":"2023-08-25T09:43:55.677241+0000","last_became_active":"2023-08-25T09:43:54.411737+0000","last_became_peered":"2023-08-25T09:43:54.411737+0000","last_unstale":"2023-08-25T09:43:55.677241+0000","last_undegraded":"2023-08-25T09:43:55.677241+0000","last_fullsized":"2023-08-25T09:43:55.677241+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:51.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678060+0000","last_change":"2023-08-25T09:43:54.392602+0000","last_active":"2023-08-25T09:43:55.678060+0000","last_peered":"2023-08-25T09:43:55.678060+0000","last_clean":"2023-08-25T09:43:55.678060+0000","last_became_active":"2023-08-25T09:43:54.392310+0000","last_became_peered":"2023-08-25T09:43:54.392310+0000","last_unstale":"2023-08-25T09:43:55.678060+0000","last_undegraded":"2023-08-25T09:43:55.678060+0000","last_fullsized":"2023-08-25T09:43:55.678060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:38.123428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388355+0000","last_change":"2023-08-25T09:43:54.411259+0000","last_active":"2023-08-25T09:43:55.388355+0000","last_peered":"2023-08-25T09:43:55.388355+0000","last_clean":"2023-08-25T09:43:55.388355+0000","last_became_active":"2023-08-25T09:43:54.411008+0000","last_became_peered":"2023-08-25T09:43:54.411008+0000","last_unstale":"2023-08-25T09:43:55.388355+0000","last_undegraded":"2023-08-25T09:43:55.388355+0000","last_fullsized":"2023-08-25T09:43:55.388355+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:29:35.594192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675937+0000","last_change":"2023-08-25T09:43:54.411396+0000","last_active":"2023-08-25T09:43:55.675937+0000","last_peered":"2023-08-25T09:43:55.675937+0000","last_clean":"2023-08-25T09:43:55.675937+0000","last_became_active":"2023-08-25T09:43:54.410849+0000","last_became_peered":"2023-08-25T09:43:54.410849+0000","last_unstale":"2023-08-25T09:43:55.675937+0000","last_undegraded":"2023-08-25T09:43:55.675937+0000","last_fullsized":"2023-08-25T09:43:55.675937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:51.094231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.528980+0000","last_change":"2023-08-25T09:43:54.418899+0000","last_active":"2023-08-25T09:43:55.528980+0000","last_peered":"2023-08-25T09:43:55.528980+0000","last_clean":"2023-08-25T09:43:55.528980+0000","last_became_active":"2023-08-25T09:43:54.417852+0000","last_became_peered":"2023-08-25T09:43:54.417852+0000","last_unstale":"2023-08-25T09:43:55.528980+0000","last_undegraded":"2023-08-25T09:43:55.528980+0000","last_fullsized":"2023-08-25T09:43:55.528980+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:42:09.899923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388185+0000","last_change":"2023-08-25T09:43:54.407528+0000","last_active":"2023-08-25T09:43:55.388185+0000","last_peered":"2023-08-25T09:43:55.388185+0000","last_clean":"2023-08-25T09:43:55.388185+0000","last_became_active":"2023-08-25T09:43:54.407006+0000","last_became_peered":"2023-08-25T09:43:54.407006+0000","last_unstale":"2023-08-25T09:43:55.388185+0000","last_undegraded":"2023-08-25T09:43:55.388185+0000","last_fullsized":"2023-08-25T09:43:55.388185+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:03.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678374+0000","last_change":"2023-08-25T09:43:54.407447+0000","last_active":"2023-08-25T09:43:55.678374+0000","last_peered":"2023-08-25T09:43:55.678374+0000","last_clean":"2023-08-25T09:43:55.678374+0000","last_became_active":"2023-08-25T09:43:54.407225+0000","last_became_peered":"2023-08-25T09:43:54.407225+0000","last_unstale":"2023-08-25T09:43:55.678374+0000","last_undegraded":"2023-08-25T09:43:55.678374+0000","last_fullsized":"2023-08-25T09:43:55.678374+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:18:36.366812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.390000+0000","last_change":"2023-08-25T09:43:54.417973+0000","last_active":"2023-08-25T09:43:55.390000+0000","last_peered":"2023-08-25T09:43:55.390000+0000","last_clean":"2023-08-25T09:43:55.390000+0000","last_became_active":"2023-08-25T09:43:54.416003+0000","last_became_peered":"2023-08-25T09:43:54.416003+0000","last_unstale":"2023-08-25T09:43:55.390000+0000","last_undegraded":"2023-08-25T09:43:55.390000+0000","last_fullsized":"2023-08-25T09:43:55.390000+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:22.319316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"93'4","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389004+0000","last_change":"2023-08-25T09:43:53.397210+0000","last_active":"2023-08-25T09:43:55.389004+0000","last_peered":"2023-08-25T09:43:55.389004+0000","last_clean":"2023-08-25T09:43:55.389004+0000","last_became_active":"2023-08-25T09:43:53.396946+0000","last_became_peered":"2023-08-25T09:43:53.396946+0000","last_unstale":"2023-08-25T09:43:55.389004+0000","last_undegraded":"2023-08-25T09:43:55.389004+0000","last_fullsized":"2023-08-25T09:43:55.389004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:22.103597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387784+0000","last_change":"2023-08-25T09:43:54.413793+0000","last_active":"2023-08-25T09:43:55.387784+0000","last_peered":"2023-08-25T09:43:55.387784+0000","last_clean":"2023-08-25T09:43:55.387784+0000","last_became_active":"2023-08-25T09:43:54.413378+0000","last_became_peered":"2023-08-25T09:43:54.413378+0000","last_unstale":"2023-08-25T09:43:55.387784+0000","last_undegraded":"2023-08-25T09:43:55.387784+0000","last_fullsized":"2023-08-25T09:43:55.387784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:28:34.617151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384615+0000","last_change":"2023-08-25T09:43:53.393077+0000","last_active":"2023-08-25T09:43:55.384615+0000","last_peered":"2023-08-25T09:43:55.384615+0000","last_clean":"2023-08-25T09:43:55.384615+0000","last_became_active":"2023-08-25T09:43:53.392575+0000","last_became_peered":"2023-08-25T09:43:53.392575+0000","last_unstale":"2023-08-25T09:43:55.384615+0000","last_undegraded":"2023-08-25T09:43:55.384615+0000","last_fullsized":"2023-08-25T09:43:55.384615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:04.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677126+0000","last_change":"2023-08-25T09:43:54.409806+0000","last_active":"2023-08-25T09:43:55.677126+0000","last_peered":"2023-08-25T09:43:55.677126+0000","last_clean":"2023-08-25T09:43:55.677126+0000","last_became_active":"2023-08-25T09:43:54.409518+0000","last_became_peered":"2023-08-25T09:43:54.409518+0000","last_unstale":"2023-08-25T09:43:55.677126+0000","last_undegraded":"2023-08-25T09:43:55.677126+0000","last_fullsized":"2023-08-25T09:43:55.677126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:46.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529263+0000","last_change":"2023-08-25T09:43:53.398957+0000","last_active":"2023-08-25T09:43:55.529263+0000","last_peered":"2023-08-25T09:43:55.529263+0000","last_clean":"2023-08-25T09:43:55.529263+0000","last_became_active":"2023-08-25T09:43:53.398717+0000","last_became_peered":"2023-08-25T09:43:53.398717+0000","last_unstale":"2023-08-25T09:43:55.529263+0000","last_undegraded":"2023-08-25T09:43:55.529263+0000","last_fullsized":"2023-08-25T09:43:55.529263+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:49.791339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677734+0000","last_change":"2023-08-25T09:43:54.401591+0000","last_active":"2023-08-25T09:43:55.677734+0000","last_peered":"2023-08-25T09:43:55.677734+0000","last_clean":"2023-08-25T09:43:55.677734+0000","last_became_active":"2023-08-25T09:43:54.401293+0000","last_became_peered":"2023-08-25T09:43:54.401293+0000","last_unstale":"2023-08-25T09:43:55.677734+0000","last_undegraded":"2023-08-25T09:43:55.677734+0000","last_fullsized":"2023-08-25T09:43:55.677734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:46:18.935110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676265+0000","last_change":"2023-08-25T09:43:53.389534+0000","last_active":"2023-08-25T09:43:55.676265+0000","last_peered":"2023-08-25T09:43:55.676265+0000","last_clean":"2023-08-25T09:43:55.676265+0000","last_became_active":"2023-08-25T09:43:53.389145+0000","last_became_peered":"2023-08-25T09:43:53.389145+0000","last_unstale":"2023-08-25T09:43:55.676265+0000","last_undegraded":"2023-08-25T09:43:55.676265+0000","last_fullsized":"2023-08-25T09:43:55.676265+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:06.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676066+0000","last_change":"2023-08-25T09:43:54.401991+0000","last_active":"2023-08-25T09:43:55.676066+0000","last_peered":"2023-08-25T09:43:55.676066+0000","last_clean":"2023-08-25T09:43:55.676066+0000","last_became_active":"2023-08-25T09:43:54.401631+0000","last_became_peered":"2023-08-25T09:43:54.401631+0000","last_unstale":"2023-08-25T09:43:55.676066+0000","last_undegraded":"2023-08-25T09:43:55.676066+0000","last_fullsized":"2023-08-25T09:43:55.676066+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:04.265830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678143+0000","last_change":"2023-08-25T09:43:53.385163+0000","last_active":"2023-08-25T09:43:55.678143+0000","last_peered":"2023-08-25T09:43:55.678143+0000","last_clean":"2023-08-25T09:43:55.678143+0000","last_became_active":"2023-08-25T09:43:53.384839+0000","last_became_peered":"2023-08-25T09:43:53.384839+0000","last_unstale":"2023-08-25T09:43:55.678143+0000","last_undegraded":"2023-08-25T09:43:55.678143+0000","last_fullsized":"2023-08-25T09:43:55.678143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:50:28.252589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387331+0000","last_change":"2023-08-25T09:43:54.402776+0000","last_active":"2023-08-25T09:43:55.387331+0000","last_peered":"2023-08-25T09:43:55.387331+0000","last_clean":"2023-08-25T09:43:55.387331+0000","last_became_active":"2023-08-25T09:43:54.401343+0000","last_became_peered":"2023-08-25T09:43:54.401343+0000","last_unstale":"2023-08-25T09:43:55.387331+0000","last_undegraded":"2023-08-25T09:43:55.387331+0000","last_fullsized":"2023-08-25T09:43:55.387331+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:26.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385440+0000","last_change":"2023-08-25T09:43:53.401154+0000","last_active":"2023-08-25T09:43:55.385440+0000","last_peered":"2023-08-25T09:43:55.385440+0000","last_clean":"2023-08-25T09:43:55.385440+0000","last_became_active":"2023-08-25T09:43:53.400830+0000","last_became_peered":"2023-08-25T09:43:53.400830+0000","last_unstale":"2023-08-25T09:43:55.385440+0000","last_undegraded":"2023-08-25T09:43:55.385440+0000","last_fullsized":"2023-08-25T09:43:55.385440+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:56:52.629469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676387+0000","last_change":"2023-08-25T09:43:54.413882+0000","last_active":"2023-08-25T09:43:55.676387+0000","last_peered":"2023-08-25T09:43:55.676387+0000","last_clean":"2023-08-25T09:43:55.676387+0000","last_became_active":"2023-08-25T09:43:54.413721+0000","last_became_peered":"2023-08-25T09:43:54.413721+0000","last_unstale":"2023-08-25T09:43:55.676387+0000","last_undegraded":"2023-08-25T09:43:55.676387+0000","last_fullsized":"2023-08-25T09:43:55.676387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:43:17.280367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530798+0000","last_change":"2023-08-25T09:43:53.401459+0000","last_active":"2023-08-25T09:43:55.530798+0000","last_peered":"2023-08-25T09:43:55.530798+0000","last_clean":"2023-08-25T09:43:55.530798+0000","last_became_active":"2023-08-25T09:43:53.400825+0000","last_became_peered":"2023-08-25T09:43:53.400825+0000","last_unstale":"2023-08-25T09:43:55.530798+0000","last_undegraded":"2023-08-25T09:43:55.530798+0000","last_fullsized":"2023-08-25T09:43:55.530798+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:14.912189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385020+0000","last_change":"2023-08-25T09:43:54.398080+0000","last_active":"2023-08-25T09:43:55.385020+0000","last_peered":"2023-08-25T09:43:55.385020+0000","last_clean":"2023-08-25T09:43:55.385020+0000","last_became_active":"2023-08-25T09:43:54.397273+0000","last_became_peered":"2023-08-25T09:43:54.397273+0000","last_unstale":"2023-08-25T09:43:55.385020+0000","last_undegraded":"2023-08-25T09:43:55.385020+0000","last_fullsized":"2023-08-25T09:43:55.385020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:01.756935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387956+0000","last_change":"2023-08-25T09:43:53.399348+0000","last_active":"2023-08-25T09:43:55.387956+0000","last_peered":"2023-08-25T09:43:55.387956+0000","last_clean":"2023-08-25T09:43:55.387956+0000","last_became_active":"2023-08-25T09:43:53.399099+0000","last_became_peered":"2023-08-25T09:43:53.399099+0000","last_unstale":"2023-08-25T09:43:55.387956+0000","last_undegraded":"2023-08-25T09:43:55.387956+0000","last_fullsized":"2023-08-25T09:43:55.387956+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:06:36.086345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388332+0000","last_change":"2023-08-25T09:43:54.406818+0000","last_active":"2023-08-25T09:43:55.388332+0000","last_peered":"2023-08-25T09:43:55.388332+0000","last_clean":"2023-08-25T09:43:55.388332+0000","last_became_active":"2023-08-25T09:43:54.405375+0000","last_became_peered":"2023-08-25T09:43:54.405375+0000","last_unstale":"2023-08-25T09:43:55.388332+0000","last_undegraded":"2023-08-25T09:43:55.388332+0000","last_fullsized":"2023-08-25T09:43:55.388332+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:30.258111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677565+0000","last_change":"2023-08-25T09:43:53.397375+0000","last_active":"2023-08-25T09:43:55.677565+0000","last_peered":"2023-08-25T09:43:55.677565+0000","last_clean":"2023-08-25T09:43:55.677565+0000","last_became_active":"2023-08-25T09:43:53.397119+0000","last_became_peered":"2023-08-25T09:43:53.397119+0000","last_unstale":"2023-08-25T09:43:55.677565+0000","last_undegraded":"2023-08-25T09:43:55.677565+0000","last_fullsized":"2023-08-25T09:43:55.677565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:20:30.983138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677836+0000","last_change":"2023-08-25T09:43:54.400636+0000","last_active":"2023-08-25T09:43:55.677836+0000","last_peered":"2023-08-25T09:43:55.677836+0000","last_clean":"2023-08-25T09:43:55.677836+0000","last_became_active":"2023-08-25T09:43:54.400265+0000","last_became_peered":"2023-08-25T09:43:54.400265+0000","last_unstale":"2023-08-25T09:43:55.677836+0000","last_undegraded":"2023-08-25T09:43:55.677836+0000","last_fullsized":"2023-08-25T09:43:55.677836+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:06:46.573434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529113+0000","last_change":"2023-08-25T09:43:53.399279+0000","last_active":"2023-08-25T09:43:55.529113+0000","last_peered":"2023-08-25T09:43:55.529113+0000","last_clean":"2023-08-25T09:43:55.529113+0000","last_became_active":"2023-08-25T09:43:53.399023+0000","last_became_peered":"2023-08-25T09:43:53.399023+0000","last_unstale":"2023-08-25T09:43:55.529113+0000","last_undegraded":"2023-08-25T09:43:55.529113+0000","last_fullsized":"2023-08-25T09:43:55.529113+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:14:04.107168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530821+0000","last_change":"2023-08-25T09:43:54.411304+0000","last_active":"2023-08-25T09:43:55.530821+0000","last_peered":"2023-08-25T09:43:55.530821+0000","last_clean":"2023-08-25T09:43:55.530821+0000","last_became_active":"2023-08-25T09:43:54.410775+0000","last_became_peered":"2023-08-25T09:43:54.410775+0000","last_unstale":"2023-08-25T09:43:55.530821+0000","last_undegraded":"2023-08-25T09:43:55.530821+0000","last_fullsized":"2023-08-25T09:43:55.530821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:00.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"93'3","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675501+0000","last_change":"2023-08-25T09:43:53.400335+0000","last_active":"2023-08-25T09:43:55.675501+0000","last_peered":"2023-08-25T09:43:55.675501+0000","last_clean":"2023-08-25T09:43:55.675501+0000","last_became_active":"2023-08-25T09:43:53.397372+0000","last_became_peered":"2023-08-25T09:43:53.397372+0000","last_unstale":"2023-08-25T09:43:55.675501+0000","last_undegraded":"2023-08-25T09:43:55.675501+0000","last_fullsized":"2023-08-25T09:43:55.675501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:57.202891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676283+0000","last_change":"2023-08-25T09:43:54.412139+0000","last_active":"2023-08-25T09:43:55.676283+0000","last_peered":"2023-08-25T09:43:55.676283+0000","last_clean":"2023-08-25T09:43:55.676283+0000","last_became_active":"2023-08-25T09:43:54.402590+0000","last_became_peered":"2023-08-25T09:43:54.402590+0000","last_unstale":"2023-08-25T09:43:55.676283+0000","last_undegraded":"2023-08-25T09:43:55.676283+0000","last_fullsized":"2023-08-25T09:43:55.676283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:44.502910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"93'6","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384787+0000","last_change":"2023-08-25T09:43:53.399357+0000","last_active":"2023-08-25T09:43:55.384787+0000","last_peered":"2023-08-25T09:43:55.384787+0000","last_clean":"2023-08-25T09:43:55.384787+0000","last_became_active":"2023-08-25T09:43:53.398803+0000","last_became_peered":"2023-08-25T09:43:53.398803+0000","last_unstale":"2023-08-25T09:43:55.384787+0000","last_undegraded":"2023-08-25T09:43:55.384787+0000","last_fullsized":"2023-08-25T09:43:55.384787+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:41.985823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677987+0000","last_change":"2023-08-25T09:43:54.408743+0000","last_active":"2023-08-25T09:43:55.677987+0000","last_peered":"2023-08-25T09:43:55.677987+0000","last_clean":"2023-08-25T09:43:55.677987+0000","last_became_active":"2023-08-25T09:43:54.407981+0000","last_became_peered":"2023-08-25T09:43:54.407981+0000","last_unstale":"2023-08-25T09:43:55.677987+0000","last_undegraded":"2023-08-25T09:43:55.677987+0000","last_fullsized":"2023-08-25T09:43:55.677987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:22.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529338+0000","last_change":"2023-08-25T09:43:53.400186+0000","last_active":"2023-08-25T09:43:55.529338+0000","last_peered":"2023-08-25T09:43:55.529338+0000","last_clean":"2023-08-25T09:43:55.529338+0000","last_became_active":"2023-08-25T09:43:53.399735+0000","last_became_peered":"2023-08-25T09:43:53.399735+0000","last_unstale":"2023-08-25T09:43:55.529338+0000","last_undegraded":"2023-08-25T09:43:55.529338+0000","last_fullsized":"2023-08-25T09:43:55.529338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:58.045564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529095+0000","last_change":"2023-08-25T09:43:54.408566+0000","last_active":"2023-08-25T09:43:55.529095+0000","last_peered":"2023-08-25T09:43:55.529095+0000","last_clean":"2023-08-25T09:43:55.529095+0000","last_became_active":"2023-08-25T09:43:54.408126+0000","last_became_peered":"2023-08-25T09:43:54.408126+0000","last_unstale":"2023-08-25T09:43:55.529095+0000","last_undegraded":"2023-08-25T09:43:55.529095+0000","last_fullsized":"2023-08-25T09:43:55.529095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:51.838016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676385+0000","last_change":"2023-08-25T09:43:53.394175+0000","last_active":"2023-08-25T09:43:55.676385+0000","last_peered":"2023-08-25T09:43:55.676385+0000","last_clean":"2023-08-25T09:43:55.676385+0000","last_became_active":"2023-08-25T09:43:53.393635+0000","last_became_peered":"2023-08-25T09:43:53.393635+0000","last_unstale":"2023-08-25T09:43:55.676385+0000","last_undegraded":"2023-08-25T09:43:55.676385+0000","last_fullsized":"2023-08-25T09:43:55.676385+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:05.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677189+0000","last_change":"2023-08-25T09:43:54.397561+0000","last_active":"2023-08-25T09:43:55.677189+0000","last_peered":"2023-08-25T09:43:55.677189+0000","last_clean":"2023-08-25T09:43:55.677189+0000","last_became_active":"2023-08-25T09:43:54.397258+0000","last_became_peered":"2023-08-25T09:43:54.397258+0000","last_unstale":"2023-08-25T09:43:55.677189+0000","last_undegraded":"2023-08-25T09:43:55.677189+0000","last_fullsized":"2023-08-25T09:43:55.677189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:52.770025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389288+0000","last_change":"2023-08-25T09:43:53.403154+0000","last_active":"2023-08-25T09:43:55.389288+0000","last_peered":"2023-08-25T09:43:55.389288+0000","last_clean":"2023-08-25T09:43:55.389288+0000","last_became_active":"2023-08-25T09:43:53.399755+0000","last_became_peered":"2023-08-25T09:43:53.399755+0000","last_unstale":"2023-08-25T09:43:55.389288+0000","last_undegraded":"2023-08-25T09:43:55.389288+0000","last_fullsized":"2023-08-25T09:43:55.389288+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:14.044339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529191+0000","last_change":"2023-08-25T09:43:54.406065+0000","last_active":"2023-08-25T09:43:55.529191+0000","last_peered":"2023-08-25T09:43:55.529191+0000","last_clean":"2023-08-25T09:43:55.529191+0000","last_became_active":"2023-08-25T09:43:54.405822+0000","last_became_peered":"2023-08-25T09:43:54.405822+0000","last_unstale":"2023-08-25T09:43:55.529191+0000","last_undegraded":"2023-08-25T09:43:55.529191+0000","last_fullsized":"2023-08-25T09:43:55.529191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:21.088250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"93'8","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675963+0000","last_change":"2023-08-25T09:43:53.403310+0000","last_active":"2023-08-25T09:43:55.675963+0000","last_peered":"2023-08-25T09:43:55.675963+0000","last_clean":"2023-08-25T09:43:55.675963+0000","last_became_active":"2023-08-25T09:43:53.400976+0000","last_became_peered":"2023-08-25T09:43:53.400976+0000","last_unstale":"2023-08-25T09:43:55.675963+0000","last_undegraded":"2023-08-25T09:43:55.675963+0000","last_fullsized":"2023-08-25T09:43:55.675963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:05:07.588952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677139+0000","last_change":"2023-08-25T09:43:54.394896+0000","last_active":"2023-08-25T09:43:55.677139+0000","last_peered":"2023-08-25T09:43:55.677139+0000","last_clean":"2023-08-25T09:43:55.677139+0000","last_became_active":"2023-08-25T09:43:54.394161+0000","last_became_peered":"2023-08-25T09:43:54.394161+0000","last_unstale":"2023-08-25T09:43:55.677139+0000","last_undegraded":"2023-08-25T09:43:55.677139+0000","last_fullsized":"2023-08-25T09:43:55.677139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:03:43.678318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529257+0000","last_change":"2023-08-25T09:43:53.400229+0000","last_active":"2023-08-25T09:43:55.529257+0000","last_peered":"2023-08-25T09:43:55.529257+0000","last_clean":"2023-08-25T09:43:55.529257+0000","last_became_active":"2023-08-25T09:43:53.400007+0000","last_became_peered":"2023-08-25T09:43:53.400007+0000","last_unstale":"2023-08-25T09:43:55.529257+0000","last_undegraded":"2023-08-25T09:43:55.529257+0000","last_fullsized":"2023-08-25T09:43:55.529257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:23.839606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529294+0000","last_change":"2023-08-25T09:43:54.408421+0000","last_active":"2023-08-25T09:43:55.529294+0000","last_peered":"2023-08-25T09:43:55.529294+0000","last_clean":"2023-08-25T09:43:55.529294+0000","last_became_active":"2023-08-25T09:43:54.407614+0000","last_became_peered":"2023-08-25T09:43:54.407614+0000","last_unstale":"2023-08-25T09:43:55.529294+0000","last_undegraded":"2023-08-25T09:43:55.529294+0000","last_fullsized":"2023-08-25T09:43:55.529294+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:15.802242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"94'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676384+0000","last_change":"2023-08-25T09:43:53.400677+0000","last_active":"2023-08-25T09:43:55.676384+0000","last_peered":"2023-08-25T09:43:55.676384+0000","last_clean":"2023-08-25T09:43:55.676384+0000","last_became_active":"2023-08-25T09:43:53.397908+0000","last_became_peered":"2023-08-25T09:43:53.397908+0000","last_unstale":"2023-08-25T09:43:55.676384+0000","last_undegraded":"2023-08-25T09:43:55.676384+0000","last_fullsized":"2023-08-25T09:43:55.676384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:45.920977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677501+0000","last_change":"2023-08-25T09:43:54.408623+0000","last_active":"2023-08-25T09:43:55.677501+0000","last_peered":"2023-08-25T09:43:55.677501+0000","last_clean":"2023-08-25T09:43:55.677501+0000","last_became_active":"2023-08-25T09:43:54.408346+0000","last_became_peered":"2023-08-25T09:43:54.408346+0000","last_unstale":"2023-08-25T09:43:55.677501+0000","last_undegraded":"2023-08-25T09:43:55.677501+0000","last_fullsized":"2023-08-25T09:43:55.677501+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:39.048547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529155+0000","last_change":"2023-08-25T09:43:53.397556+0000","last_active":"2023-08-25T09:43:55.529155+0000","last_peered":"2023-08-25T09:43:55.529155+0000","last_clean":"2023-08-25T09:43:55.529155+0000","last_became_active":"2023-08-25T09:43:53.397131+0000","last_became_peered":"2023-08-25T09:43:53.397131+0000","last_unstale":"2023-08-25T09:43:55.529155+0000","last_undegraded":"2023-08-25T09:43:55.529155+0000","last_fullsized":"2023-08-25T09:43:55.529155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:34:31.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.531351+0000","last_change":"2023-08-25T09:43:54.411155+0000","last_active":"2023-08-25T09:43:55.531351+0000","last_peered":"2023-08-25T09:43:55.531351+0000","last_clean":"2023-08-25T09:43:55.531351+0000","last_became_active":"2023-08-25T09:43:54.410442+0000","last_became_peered":"2023-08-25T09:43:54.410442+0000","last_unstale":"2023-08-25T09:43:55.531351+0000","last_undegraded":"2023-08-25T09:43:55.531351+0000","last_fullsized":"2023-08-25T09:43:55.531351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:06:27.055748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677890+0000","last_change":"2023-08-25T09:43:53.390461+0000","last_active":"2023-08-25T09:43:55.677890+0000","last_peered":"2023-08-25T09:43:55.677890+0000","last_clean":"2023-08-25T09:43:55.677890+0000","last_became_active":"2023-08-25T09:43:53.390304+0000","last_became_peered":"2023-08-25T09:43:53.390304+0000","last_unstale":"2023-08-25T09:43:55.677890+0000","last_undegraded":"2023-08-25T09:43:55.677890+0000","last_fullsized":"2023-08-25T09:43:55.677890+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:10.567771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676883+0000","last_change":"2023-08-25T09:43:54.416330+0000","last_active":"2023-08-25T09:43:55.676883+0000","last_peered":"2023-08-25T09:43:55.676883+0000","last_clean":"2023-08-25T09:43:55.676883+0000","last_became_active":"2023-08-25T09:43:54.416029+0000","last_became_peered":"2023-08-25T09:43:54.416029+0000","last_unstale":"2023-08-25T09:43:55.676883+0000","last_undegraded":"2023-08-25T09:43:55.676883+0000","last_fullsized":"2023-08-25T09:43:55.676883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:49.899994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385379+0000","last_change":"2023-08-25T09:43:53.394719+0000","last_active":"2023-08-25T09:43:55.385379+0000","last_peered":"2023-08-25T09:43:55.385379+0000","last_clean":"2023-08-25T09:43:55.385379+0000","last_became_active":"2023-08-25T09:43:53.394231+0000","last_became_peered":"2023-08-25T09:43:53.394231+0000","last_unstale":"2023-08-25T09:43:55.385379+0000","last_undegraded":"2023-08-25T09:43:55.385379+0000","last_fullsized":"2023-08-25T09:43:55.385379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:53.753394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676795+0000","last_change":"2023-08-25T09:43:54.395059+0000","last_active":"2023-08-25T09:43:55.676795+0000","last_peered":"2023-08-25T09:43:55.676795+0000","last_clean":"2023-08-25T09:43:55.676795+0000","last_became_active":"2023-08-25T09:43:54.394487+0000","last_became_peered":"2023-08-25T09:43:54.394487+0000","last_unstale":"2023-08-25T09:43:55.676795+0000","last_undegraded":"2023-08-25T09:43:55.676795+0000","last_fullsized":"2023-08-25T09:43:55.676795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:12.153733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676095+0000","last_change":"2023-08-25T09:43:53.401366+0000","last_active":"2023-08-25T09:43:55.676095+0000","last_peered":"2023-08-25T09:43:55.676095+0000","last_clean":"2023-08-25T09:43:55.676095+0000","last_became_active":"2023-08-25T09:43:53.400954+0000","last_became_peered":"2023-08-25T09:43:53.400954+0000","last_unstale":"2023-08-25T09:43:55.676095+0000","last_undegraded":"2023-08-25T09:43:55.676095+0000","last_fullsized":"2023-08-25T09:43:55.676095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:43.297912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389004+0000","last_change":"2023-08-25T09:43:54.408774+0000","last_active":"2023-08-25T09:43:55.389004+0000","last_peered":"2023-08-25T09:43:55.389004+0000","last_clean":"2023-08-25T09:43:55.389004+0000","last_became_active":"2023-08-25T09:43:54.408056+0000","last_became_peered":"2023-08-25T09:43:54.408056+0000","last_unstale":"2023-08-25T09:43:55.389004+0000","last_undegraded":"2023-08-25T09:43:55.389004+0000","last_fullsized":"2023-08-25T09:43:55.389004+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:39.809497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676019+0000","last_change":"2023-08-25T09:43:53.396976+0000","last_active":"2023-08-25T09:43:55.676019+0000","last_peered":"2023-08-25T09:43:55.676019+0000","last_clean":"2023-08-25T09:43:55.676019+0000","last_became_active":"2023-08-25T09:43:53.396705+0000","last_became_peered":"2023-08-25T09:43:53.396705+0000","last_unstale":"2023-08-25T09:43:55.676019+0000","last_undegraded":"2023-08-25T09:43:55.676019+0000","last_fullsized":"2023-08-25T09:43:55.676019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:12.675960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388510+0000","last_change":"2023-08-25T09:43:54.410392+0000","last_active":"2023-08-25T09:43:55.388510+0000","last_peered":"2023-08-25T09:43:55.388510+0000","last_clean":"2023-08-25T09:43:55.388510+0000","last_became_active":"2023-08-25T09:43:54.410129+0000","last_became_peered":"2023-08-25T09:43:54.410129+0000","last_unstale":"2023-08-25T09:43:55.388510+0000","last_undegraded":"2023-08-25T09:43:55.388510+0000","last_fullsized":"2023-08-25T09:43:55.388510+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:35.069148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387143+0000","last_change":"2023-08-25T09:43:53.398900+0000","last_active":"2023-08-25T09:43:55.387143+0000","last_peered":"2023-08-25T09:43:55.387143+0000","last_clean":"2023-08-25T09:43:55.387143+0000","last_became_active":"2023-08-25T09:43:53.398452+0000","last_became_peered":"2023-08-25T09:43:53.398452+0000","last_unstale":"2023-08-25T09:43:55.387143+0000","last_undegraded":"2023-08-25T09:43:55.387143+0000","last_fullsized":"2023-08-25T09:43:55.387143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:23.214744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385717+0000","last_change":"2023-08-25T09:43:54.413745+0000","last_active":"2023-08-25T09:43:55.385717+0000","last_peered":"2023-08-25T09:43:55.385717+0000","last_clean":"2023-08-25T09:43:55.385717+0000","last_became_active":"2023-08-25T09:43:54.413429+0000","last_became_peered":"2023-08-25T09:43:54.413429+0000","last_unstale":"2023-08-25T09:43:55.385717+0000","last_undegraded":"2023-08-25T09:43:55.385717+0000","last_fullsized":"2023-08-25T09:43:55.385717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:45.596511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676451+0000","last_change":"2023-08-25T09:43:53.395918+0000","last_active":"2023-08-25T09:43:55.676451+0000","last_peered":"2023-08-25T09:43:55.676451+0000","last_clean":"2023-08-25T09:43:55.676451+0000","last_became_active":"2023-08-25T09:43:53.395232+0000","last_became_peered":"2023-08-25T09:43:53.395232+0000","last_unstale":"2023-08-25T09:43:55.676451+0000","last_undegraded":"2023-08-25T09:43:55.676451+0000","last_fullsized":"2023-08-25T09:43:55.676451+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:05.916939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":293,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529317+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:43:55.529317+0000","last_peered":"2023-08-25T09:43:55.529317+0000","last_clean":"2023-08-25T09:43:55.529317+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:43:55.529317+0000","last_undegraded":"2023-08-25T09:43:55.529317+0000","last_fullsized":"2023-08-25T09:43:55.529317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:04.320619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385863+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:43:55.385863+0000","last_peered":"2023-08-25T09:43:55.385863+0000","last_clean":"2023-08-25T09:43:55.385863+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:43:55.385863+0000","last_undegraded":"2023-08-25T09:43:55.385863+0000","last_fullsized":"2023-08-25T09:43:55.385863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:22.300510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.390505+0000","last_change":"2023-08-25T09:43:54.418281+0000","last_active":"2023-08-25T09:43:55.390505+0000","last_peered":"2023-08-25T09:43:55.390505+0000","last_clean":"2023-08-25T09:43:55.390505+0000","last_became_active":"2023-08-25T09:43:54.416607+0000","last_became_peered":"2023-08-25T09:43:54.416607+0000","last_unstale":"2023-08-25T09:43:55.390505+0000","last_undegraded":"2023-08-25T09:43:55.390505+0000","last_fullsized":"2023-08-25T09:43:55.390505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:56.229389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530785+0000","last_change":"2023-08-25T09:43:53.402444+0000","last_active":"2023-08-25T09:43:55.530785+0000","last_peered":"2023-08-25T09:43:55.530785+0000","last_clean":"2023-08-25T09:43:55.530785+0000","last_became_active":"2023-08-25T09:43:53.402171+0000","last_became_peered":"2023-08-25T09:43:53.402171+0000","last_unstale":"2023-08-25T09:43:55.530785+0000","last_undegraded":"2023-08-25T09:43:55.530785+0000","last_fullsized":"2023-08-25T09:43:55.530785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:15.932838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385043+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:43:55.385043+0000","last_peered":"2023-08-25T09:43:55.385043+0000","last_clean":"2023-08-25T09:43:55.385043+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:43:55.385043+0000","last_undegraded":"2023-08-25T09:43:55.385043+0000","last_fullsized":"2023-08-25T09:43:55.385043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:20:22.560940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384463+0000","last_change":"2023-08-25T09:43:54.392619+0000","last_active":"2023-08-25T09:43:55.384463+0000","last_peered":"2023-08-25T09:43:55.384463+0000","last_clean":"2023-08-25T09:43:55.384463+0000","last_became_active":"2023-08-25T09:43:54.392340+0000","last_became_peered":"2023-08-25T09:43:54.392340+0000","last_unstale":"2023-08-25T09:43:55.384463+0000","last_undegraded":"2023-08-25T09:43:55.384463+0000","last_fullsized":"2023-08-25T09:43:55.384463+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:33.252763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388517+0000","last_change":"2023-08-25T09:43:53.403913+0000","last_active":"2023-08-25T09:43:55.388517+0000","last_peered":"2023-08-25T09:43:55.388517+0000","last_clean":"2023-08-25T09:43:55.388517+0000","last_became_active":"2023-08-25T09:43:53.403680+0000","last_became_peered":"2023-08-25T09:43:53.403680+0000","last_unstale":"2023-08-25T09:43:55.388517+0000","last_undegraded":"2023-08-25T09:43:55.388517+0000","last_fullsized":"2023-08-25T09:43:55.388517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:27:24.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389312+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:43:55.389312+0000","last_peered":"2023-08-25T09:43:55.389312+0000","last_clean":"2023-08-25T09:43:55.389312+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:43:55.389312+0000","last_undegraded":"2023-08-25T09:43:55.389312+0000","last_fullsized":"2023-08-25T09:43:55.389312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:54.055742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388940+0000","last_change":"2023-08-25T09:43:54.412066+0000","last_active":"2023-08-25T09:43:55.388940+0000","last_peered":"2023-08-25T09:43:55.388940+0000","last_clean":"2023-08-25T09:43:55.388940+0000","last_became_active":"2023-08-25T09:43:54.411815+0000","last_became_peered":"2023-08-25T09:43:54.411815+0000","last_unstale":"2023-08-25T09:43:55.388940+0000","last_undegraded":"2023-08-25T09:43:55.388940+0000","last_fullsized":"2023-08-25T09:43:55.388940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:25.923883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389578+0000","last_change":"2023-08-25T09:43:53.403077+0000","last_active":"2023-08-25T09:43:55.389578+0000","last_peered":"2023-08-25T09:43:55.389578+0000","last_clean":"2023-08-25T09:43:55.389578+0000","last_became_active":"2023-08-25T09:43:53.402353+0000","last_became_peered":"2023-08-25T09:43:53.402353+0000","last_unstale":"2023-08-25T09:43:55.389578+0000","last_undegraded":"2023-08-25T09:43:55.389578+0000","last_fullsized":"2023-08-25T09:43:55.389578+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:45.178645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385518+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:43:55.385518+0000","last_peered":"2023-08-25T09:43:55.385518+0000","last_clean":"2023-08-25T09:43:55.385518+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:43:55.385518+0000","last_undegraded":"2023-08-25T09:43:55.385518+0000","last_fullsized":"2023-08-25T09:43:55.385518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:22:06.254532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530488+0000","last_change":"2023-08-25T09:43:54.412739+0000","last_active":"2023-08-25T09:43:55.530488+0000","last_peered":"2023-08-25T09:43:55.530488+0000","last_clean":"2023-08-25T09:43:55.530488+0000","last_became_active":"2023-08-25T09:43:54.412482+0000","last_became_peered":"2023-08-25T09:43:54.412482+0000","last_unstale":"2023-08-25T09:43:55.530488+0000","last_undegraded":"2023-08-25T09:43:55.530488+0000","last_fullsized":"2023-08-25T09:43:55.530488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:23.371240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389665+0000","last_change":"2023-08-25T09:43:53.388577+0000","last_active":"2023-08-25T09:43:55.389665+0000","last_peered":"2023-08-25T09:43:55.389665+0000","last_clean":"2023-08-25T09:43:55.389665+0000","last_became_active":"2023-08-25T09:43:53.387937+0000","last_became_peered":"2023-08-25T09:43:53.387937+0000","last_unstale":"2023-08-25T09:43:55.389665+0000","last_undegraded":"2023-08-25T09:43:55.389665+0000","last_fullsized":"2023-08-25T09:43:55.389665+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:12.082324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676632+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:43:55.676632+0000","last_peered":"2023-08-25T09:43:55.676632+0000","last_clean":"2023-08-25T09:43:55.676632+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:43:55.676632+0000","last_undegraded":"2023-08-25T09:43:55.676632+0000","last_fullsized":"2023-08-25T09:43:55.676632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:56.103124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388697+0000","last_change":"2023-08-25T09:43:54.416364+0000","last_active":"2023-08-25T09:43:55.388697+0000","last_peered":"2023-08-25T09:43:55.388697+0000","last_clean":"2023-08-25T09:43:55.388697+0000","last_became_active":"2023-08-25T09:43:54.415180+0000","last_became_peered":"2023-08-25T09:43:54.415180+0000","last_unstale":"2023-08-25T09:43:55.388697+0000","last_undegraded":"2023-08-25T09:43:55.388697+0000","last_fullsized":"2023-08-25T09:43:55.388697+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:18.233987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389600+0000","last_change":"2023-08-25T09:43:53.403927+0000","last_active":"2023-08-25T09:43:55.389600+0000","last_peered":"2023-08-25T09:43:55.389600+0000","last_clean":"2023-08-25T09:43:55.389600+0000","last_became_active":"2023-08-25T09:43:53.402901+0000","last_became_peered":"2023-08-25T09:43:53.402901+0000","last_unstale":"2023-08-25T09:43:55.389600+0000","last_undegraded":"2023-08-25T09:43:55.389600+0000","last_fullsized":"2023-08-25T09:43:55.389600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:47.804156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384922+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:43:55.384922+0000","last_peered":"2023-08-25T09:43:55.384922+0000","last_clean":"2023-08-25T09:43:55.384922+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:43:55.384922+0000","last_undegraded":"2023-08-25T09:43:55.384922+0000","last_fullsized":"2023-08-25T09:43:55.384922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:08:27.873205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389307+0000","last_change":"2023-08-25T09:43:54.418893+0000","last_active":"2023-08-25T09:43:55.389307+0000","last_peered":"2023-08-25T09:43:55.389307+0000","last_clean":"2023-08-25T09:43:55.389307+0000","last_became_active":"2023-08-25T09:43:54.418633+0000","last_became_peered":"2023-08-25T09:43:54.418633+0000","last_unstale":"2023-08-25T09:43:55.389307+0000","last_undegraded":"2023-08-25T09:43:55.389307+0000","last_fullsized":"2023-08-25T09:43:55.389307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:46:42.226958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678185+0000","last_change":"2023-08-25T09:43:53.401971+0000","last_active":"2023-08-25T09:43:55.678185+0000","last_peered":"2023-08-25T09:43:55.678185+0000","last_clean":"2023-08-25T09:43:55.678185+0000","last_became_active":"2023-08-25T09:43:53.401743+0000","last_became_peered":"2023-08-25T09:43:53.401743+0000","last_unstale":"2023-08-25T09:43:55.678185+0000","last_undegraded":"2023-08-25T09:43:55.678185+0000","last_fullsized":"2023-08-25T09:43:55.678185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:56.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530611+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:43:55.530611+0000","last_peered":"2023-08-25T09:43:55.530611+0000","last_clean":"2023-08-25T09:43:55.530611+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:43:55.530611+0000","last_undegraded":"2023-08-25T09:43:55.530611+0000","last_fullsized":"2023-08-25T09:43:55.530611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:01.681521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387661+0000","last_change":"2023-08-25T09:43:54.399751+0000","last_active":"2023-08-25T09:43:55.387661+0000","last_peered":"2023-08-25T09:43:55.387661+0000","last_clean":"2023-08-25T09:43:55.387661+0000","last_became_active":"2023-08-25T09:43:54.399246+0000","last_became_peered":"2023-08-25T09:43:54.399246+0000","last_unstale":"2023-08-25T09:43:55.387661+0000","last_undegraded":"2023-08-25T09:43:55.387661+0000","last_fullsized":"2023-08-25T09:43:55.387661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:24.732531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677545+0000","last_change":"2023-08-25T09:43:53.400796+0000","last_active":"2023-08-25T09:43:55.677545+0000","last_peered":"2023-08-25T09:43:55.677545+0000","last_clean":"2023-08-25T09:43:55.677545+0000","last_became_active":"2023-08-25T09:43:53.400551+0000","last_became_peered":"2023-08-25T09:43:53.400551+0000","last_unstale":"2023-08-25T09:43:55.677545+0000","last_undegraded":"2023-08-25T09:43:55.677545+0000","last_fullsized":"2023-08-25T09:43:55.677545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:51:58.400833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385228+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:43:55.385228+0000","last_peered":"2023-08-25T09:43:55.385228+0000","last_clean":"2023-08-25T09:43:55.385228+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:43:55.385228+0000","last_undegraded":"2023-08-25T09:43:55.385228+0000","last_fullsized":"2023-08-25T09:43:55.385228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:47.623306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194787,"statfs":{"total":200000000,"available":199462687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1910000000000001,"15min":1.0660000000000001},"min":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.54,"5min":10.747,"15min":10.747},"last":0.752},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.86899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0409999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1859999999999999,"15min":1.129},"min":{"1min":0.88,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1799999999999999,"15min":1.0920000000000001},"min":{"1min":0.76500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5820000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1559999999999999,"15min":1.079},"min":{"1min":0.75600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.27,"15min":1.2170000000000001},"min":{"1min":1.0620000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.708}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.81000000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3839999999999999,"15min":1.329},"min":{"1min":1.0629999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.639}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7689999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.286},"min":{"1min":1.05,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7609999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.429,"15min":1.3740000000000001},"min":{"1min":1.04,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9830000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.462}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5069999999999999,"15min":1.4319999999999999},"min":{"1min":1.1439999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8740000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.579},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.472,"15min":1.3979999999999999},"min":{"1min":1.276,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.6160000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194787,"statfs":{"total":200000000,"available":199462667,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.1399999999999999,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.224},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1910000000000001,"15min":1.145},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7649999999999999,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":0.752}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2290000000000001,"15min":1.1950000000000001},"min":{"1min":0.79300000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.7050000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.1830000000000001,"15min":1.117},"min":{"1min":0.94799999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.337}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1659999999999999,"15min":1.105},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.482,"5min":1.708,"15min":1.708},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.409,"5min":1.3620000000000001,"15min":1.204},"min":{"1min":0.93400000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.768,"5min":13.193,"15min":13.193},"last":1.4159999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.302,"15min":1.222},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.121,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.272,"15min":1.256},"min":{"1min":0.98099999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.548,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.24},"min":{"1min":0.95999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.607,"5min":2.484,"15min":2.484},"last":1.528},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.385,"15min":1.3169999999999999},"min":{"1min":0.89200000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.02,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.177}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.2749999999999999},"min":{"1min":1.034,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.837,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.272},{"interface":"front","average":{"1min":1.532,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":1.1990000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7210000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4750000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.538,"15min":1.49},"min":{"1min":0.89600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.2240000000000002,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.472},"min":{"1min":1.244,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.3879999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1100,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194211,"statfs":{"total":200000000,"available":198872881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1930000000000001,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.577,"5min":2.081,"15min":2.081},"last":1.738},{"interface":"front","average":{"1min":1.27,"5min":1.22,"15min":1.208},"min":{"1min":0.76200000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5469999999999999,"5min":1.974,"15min":1.974},"last":1.585}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.145,"15min":1.1950000000000001},"min":{"1min":0.746,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.577,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.24},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1819999999999999,"15min":1.119},"min":{"1min":0.997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5649999999999999,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.8640000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.238,"15min":1.1859999999999999},"min":{"1min":0.94899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.665,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.012},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":1.048,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6259999999999999,"5min":1.962,"15min":1.962},"last":1.778}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4059999999999999,"15min":1.3160000000000001},"min":{"1min":1.0549999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3439999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.177},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":1.024,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.552,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.367}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.355,"15min":1.3029999999999999},"min":{"1min":0.96199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7490000000000001,"5min":2.625,"15min":2.625},"last":1.9530000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.3720000000000001},"min":{"1min":0.94499999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.857,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.4419999999999999,"15min":1.4330000000000001},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.77,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.512,"15min":1.417},"min":{"1min":1.177,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1059999999999999,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.478}]},{"osd":7,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.4770000000000001,"15min":1.4450000000000001},"min":{"1min":1.2869999999999999,"5min":0.63,"15min":0.63},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.587,"15min":1.579},"min":{"1min":1.4319999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.7050000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1759999999999999,"15min":1.1499999999999999},"min":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5329999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.1459999999999999},"min":{"1min":1.026,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5289999999999999,"5min":7.367,"15min":7.367},"last":1.1719999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1359999999999999,"15min":1.1220000000000001},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.996},{"interface":"front","average":{"1min":1.173,"5min":1.244,"15min":1.401},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5249999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.4650000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.208,"15min":1.302},"min":{"1min":0.93200000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.244,"15min":1.167},"min":{"1min":0.95899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.631,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.121}]},{"osd":3,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.292,"15min":1.23},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7110000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.371},"min":{"1min":1.1439999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.778,"5min":1.911,"15min":1.911},"last":1.2090000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.86599999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.276},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3280000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.829,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.423,"15min":1.4350000000000001},"min":{"1min":0.81100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0219999999999998,"5min":2.863,"15min":2.863},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.548,"15min":1.5600000000000001},"min":{"1min":0.97799999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9730000000000001,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.5429999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.55,"15min":1.4179999999999999},"min":{"1min":1.1910000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.355},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.593,"15min":1.595},"min":{"1min":1.3320000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1230000000000002,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.587}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":525,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194786,"statfs":{"total":200000000,"available":199461623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.86699999999999999,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.256},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.95399999999999996,"15min":0.89800000000000002},"min":{"1min":0.65900000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5349999999999999,"5min":1.55,"15min":1.55},"last":0.88900000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.871,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.153,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.004,"15min":0.97699999999999998},"min":{"1min":0.58699999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4910000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.139}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.96199999999999997,"15min":0.87},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":1.573,"5min":1.919,"15min":1.919},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1020000000000001,"15min":1.1080000000000001},"min":{"1min":0.81000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6140000000000001,"5min":3.399,"15min":3.399},"last":1.345}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.214,"15min":1.1890000000000001},"min":{"1min":0.83499999999999996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5800000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.105},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.196},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.405,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3120000000000001},"min":{"1min":0.98899999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6699999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.2849999999999999,"15min":1.2689999999999999},"min":{"1min":0.92000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5980000000000001,"5min":1.788,"15min":1.788},"last":1.3069999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.282,"15min":1.242},"min":{"1min":1.0389999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.917,"15min":1.917},"last":1.048},{"interface":"front","average":{"1min":1.45,"5min":1.397,"15min":1.405},"min":{"1min":1.2230000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.734,"5min":3.496,"15min":3.496},"last":1.448}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.085,"5min":0.871,"15min":0.871},"max":{"1min":1.881,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":0.98899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.669,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.1890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463339,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.82499999999999996},"min":{"1min":0.53300000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.4199999999999999,"5min":2.383,"15min":2.383},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.425,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.411,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.238},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5649999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.284}]},{"osd":3,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5429999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.081,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.486,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.3220000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3080000000000001,"15min":1.397},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.1309999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.266,"15min":1.375},"min":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.369,"15min":1.397},"min":{"1min":1.081,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9890000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.444}]},{"osd":6,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.381,"15min":1.4199999999999999},"min":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.875,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.425},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.482},"min":{"1min":0.995,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.097,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.7070000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.383,"15min":1.4490000000000001},"min":{"1min":0.89600000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7829999999999999,"5min":2.968,"15min":2.968},"last":1.478},{"interface":"front","average":{"1min":1.409,"5min":1.4199999999999999,"15min":1.504},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.056,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.8200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1102,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194210,"statfs":{"total":200000000,"available":198871325,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.77200000000000002},"min":{"1min":0.45900000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2250000000000001,"5min":1.627,"15min":1.627},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.876,"15min":0.78400000000000003},"min":{"1min":0.56799999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6850000000000001,"5min":1.895,"15min":1.895},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.79900000000000004},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.45,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.048,"5min":0.97499999999999998,"15min":0.90300000000000002},"min":{"1min":0.51500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4239999999999999,"5min":1.746,"15min":1.746},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1339999999999999,"15min":1.038},"min":{"1min":0.61199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.639,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.008,"15min":0.93999999999999995},"min":{"1min":0.67900000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4750000000000001,"5min":1.764,"15min":1.764},"last":1.034}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.155,"15min":1.087},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.117},{"interface":"front","average":{"1min":1.369,"5min":1.3100000000000001,"15min":1.218},"min":{"1min":0.876,"5min":0.629,"15min":0.629},"max":{"1min":1.641,"5min":2.008,"15min":2.008},"last":1.2809999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1890000000000001},"min":{"1min":0.80000000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7909999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2170000000000001,"15min":1.1679999999999999},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.738,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.401}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3799999999999999,"15min":1.298},"min":{"1min":1.042,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9930000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.82199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8380000000000001,"5min":2.004,"15min":2.004},"last":1.341}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":1.145,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7030000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.3899999999999999,"15min":1.3080000000000001},"min":{"1min":0.93500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9039999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.2969999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90700000000000003,"15min":0.85299999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.355,"5min":11.257,"15min":11.257},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.97099999999999997,"15min":0.95799999999999996},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7110000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.88}]},{"osd":2,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.38800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0029999999999999,"15min":1.0940000000000001},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5640000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":1.085}]},{"osd":3,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.002},{"interface":"front","average":{"1min":1.012,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5609999999999999,"5min":1.869,"15min":1.869},"last":1.3700000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.327},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3340000000000001,"15min":1.353},"min":{"1min":0.83299999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7390000000000001,"5min":1.982,"15min":1.982},"last":1.4910000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.256,"15min":1.3580000000000001},"min":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8340000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.81100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.246}]},{"osd":6,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6579999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.296},"min":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3660000000000001,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0779999999999998,"5min":2.121,"15min":2.121},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.319},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.1870000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:44:03.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:44:04.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.052+0000 7f337eff3640 1 Processor -- start 2023-08-25T09:44:04.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.052+0000 7f337eff3640 1 -- start start 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.053+0000 7f337eff3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815ec10 0x7f337815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.053+0000 7f337eff3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f337815c6d0 0x7f33780b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.053+0000 7f337eff3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33780b1b60 con 0x7f337815e830 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.053+0000 7f337eff3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337806bd80 con 0x7f337815c6d0 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.053+0000 7f337eff3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f337806b7c0 con 0x7f337815ec10 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337cd8f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f337815e830 0x7f337806ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39160/0 (socket says 172.21.15.70:39160) 2023-08-25T09:44:04.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337cd8f640 1 -- 172.21.15.70:0/2411543170 learned_addr learned my addr 172.21.15.70:0/2411543170 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:04.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337d590640 1 --2- 172.21.15.70:0/2411543170 >> v2:172.21.15.70:3300/0 conn(0x7f337815c6d0 0x7f33780b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f336ffff640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815ec10 0x7f337815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f336f7fe640 1 -- 172.21.15.70:0/2411543170 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923593880 0 0) 0x7f33780b1b60 con 0x7f337815e830 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f336f7fe640 1 -- 172.21.15.70:0/2411543170 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3360003520 con 0x7f337815e830 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337d590640 1 -- 172.21.15.70:0/2411543170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815ec10 msgr2=0x7f337815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337d590640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815ec10 0x7f337815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337d590640 1 -- 172.21.15.70:0/2411543170 >> v1:172.21.15.70:6789/0 conn(0x7f337815e830 legacy=0x7f337806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f337d590640 1 -- 172.21.15.70:0/2411543170 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337806baa0 con 0x7f337815c6d0 2023-08-25T09:44:04.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.054+0000 7f336ffff640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815ec10 0x7f337815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:04.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.055+0000 7f337d590640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815c6d0 0x7f33780b1ff0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3370009d90 tx=0x7f337002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f706257a8cf8546e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:04.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f336f7fe640 1 -- 172.21.15.70:0/2411543170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002cbd0 con 0x7f337815c6d0 2023-08-25T09:44:04.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f336f7fe640 1 -- 172.21.15.70:0/2411543170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337002cd30 con 0x7f337815c6d0 2023-08-25T09:44:04.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f336f7fe640 1 -- 172.21.15.70:0/2411543170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002cea0 con 0x7f337815c6d0 2023-08-25T09:44:04.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 -- 172.21.15.70:0/2411543170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815c6d0 msgr2=0x7f33780b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815c6d0 0x7f33780b1ff0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f3370009d90 tx=0x7f337002be20 comp rx=0 tx=0).stop 2023-08-25T09:44:04.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 -- 172.21.15.70:0/2411543170 shutdown_connections 2023-08-25T09:44:04.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815c6d0 0x7f33780b1ff0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 --2- 172.21.15.70:0/2411543170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815ec10 0x7f337815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 -- 172.21.15.70:0/2411543170 >> 172.21.15.70:0/2411543170 conn(0x7f3378069850 msgr2=0x7f3378069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:04.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 -- 172.21.15.70:0/2411543170 shutdown_connections 2023-08-25T09:44:04.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.056+0000 7f337eff3640 1 -- 172.21.15.70:0/2411543170 wait complete. 2023-08-25T09:44:04.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.058+0000 7f337eff3640 1 Processor -- start 2023-08-25T09:44:04.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.058+0000 7f337eff3640 1 -- start start 2023-08-25T09:44:04.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.059+0000 7f337eff3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815c6d0 0x7f337814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.059+0000 7f337eff3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f337815e830 0x7f33780ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.059+0000 7f337eff3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 0x7f33780a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.059+0000 7f337eff3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c8580 con 0x7f337815ec10 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.059+0000 7f337eff3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c8860 con 0x7f337815e830 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.059+0000 7f337eff3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33780c8b40 con 0x7f337815c6d0 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 0x7f33780a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 0x7f33780a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45344/0 (socket says 172.21.15.70:45344) 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 -- 172.21.15.70:0/1858425130 learned_addr learned my addr 172.21.15.70:0/1858425130 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337cd8f640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815c6d0 0x7f337814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f336ffff640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f337815e830 0x7f33780ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 -- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815c6d0 msgr2=0x7f337814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815c6d0 0x7f337814f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 -- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f337815e830 msgr2=0x7f33780ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f337815e830 0x7f33780ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 -- 172.21.15.70:0/1858425130 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3370009a70 con 0x7f337815ec10 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.060+0000 7f337d590640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 0x7f33780a9e80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3370037040 tx=0x7f3370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:04.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.061+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337002cea0 con 0x7f337815ec10 2023-08-25T09:44:04.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.061+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370004820 con 0x7f337815ec10 2023-08-25T09:44:04.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.061+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370034680 con 0x7f337815ec10 2023-08-25T09:44:04.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.061+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33780c8e20 con 0x7f337815ec10 2023-08-25T09:44:04.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.061+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33780a9700 con 0x7f337815ec10 2023-08-25T09:44:04.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.062+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33780c9340 con 0x7f337815ec10 2023-08-25T09:44:04.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.063+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33700347e0 con 0x7f337815ec10 2023-08-25T09:44:04.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.064+0000 7f336d7fa640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a050 0x7f336406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.064+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33700aa940 con 0x7f337815ec10 2023-08-25T09:44:04.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.064+0000 7f337cd8f640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a050 0x7f336406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.065+0000 7f337cd8f640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a050 0x7f336406c510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f33740011e0 tx=0x7f3374009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:04.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.068+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f33780c9340 con 0x7f337815ec10 2023-08-25T09:44:04.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.255+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f337806b480 con 0x7f336406a050 2023-08-25T09:44:04.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.265+0000 7f336d7fa640 1 -- 172.21.15.70:0/1858425130 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302945 (secure 0 0 0) 0x7f3374001c40 con 0x7f336406a050 2023-08-25T09:44:04.271 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:04.272 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:44:04.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a050 msgr2=0x7f336406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a050 0x7f336406c510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f33740011e0 tx=0x7f3374009040 comp rx=0 tx=0).stop 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 msgr2=0x7f33780a9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 0x7f33780a9e80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f3370037040 tx=0x7f3370004a40 comp rx=0 tx=0).stop 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 shutdown_connections 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f336406a050 0x7f336406c510 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f337815ec10 0x7f33780a9e80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f337815e830 0x7f33780ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 --2- 172.21.15.70:0/1858425130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f337815c6d0 0x7f337814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 >> 172.21.15.70:0/1858425130 conn(0x7f3378069850 msgr2=0x7f337806ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 shutdown_connections 2023-08-25T09:44:04.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.270+0000 7f337eff3640 1 -- 172.21.15.70:0/1858425130 wait complete. 2023-08-25T09:44:04.290 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-08-25T09:44:03.422332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":201,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5350,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557149,"statfs":{"total":1600000000,"available":1594521156,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.040021"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529053+0000","last_change":"2023-08-25T09:43:54.404594+0000","last_active":"2023-08-25T09:43:55.529053+0000","last_peered":"2023-08-25T09:43:55.529053+0000","last_clean":"2023-08-25T09:43:55.529053+0000","last_became_active":"2023-08-25T09:43:54.404347+0000","last_became_peered":"2023-08-25T09:43:54.404347+0000","last_unstale":"2023-08-25T09:43:55.529053+0000","last_undegraded":"2023-08-25T09:43:55.529053+0000","last_fullsized":"2023-08-25T09:43:55.529053+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:33.114359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.386937+0000","last_change":"2023-08-25T09:43:54.408980+0000","last_active":"2023-08-25T09:43:55.386937+0000","last_peered":"2023-08-25T09:43:55.386937+0000","last_clean":"2023-08-25T09:43:55.386937+0000","last_became_active":"2023-08-25T09:43:54.408724+0000","last_became_peered":"2023-08-25T09:43:54.408724+0000","last_unstale":"2023-08-25T09:43:55.386937+0000","last_undegraded":"2023-08-25T09:43:55.386937+0000","last_fullsized":"2023-08-25T09:43:55.386937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:27.591271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388658+0000","last_change":"2023-08-25T09:43:54.405905+0000","last_active":"2023-08-25T09:43:55.388658+0000","last_peered":"2023-08-25T09:43:55.388658+0000","last_clean":"2023-08-25T09:43:55.388658+0000","last_became_active":"2023-08-25T09:43:54.398170+0000","last_became_peered":"2023-08-25T09:43:54.398170+0000","last_unstale":"2023-08-25T09:43:55.388658+0000","last_undegraded":"2023-08-25T09:43:55.388658+0000","last_fullsized":"2023-08-25T09:43:55.388658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:17:42.892816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676440+0000","last_change":"2023-08-25T09:43:54.401703+0000","last_active":"2023-08-25T09:43:55.676440+0000","last_peered":"2023-08-25T09:43:55.676440+0000","last_clean":"2023-08-25T09:43:55.676440+0000","last_became_active":"2023-08-25T09:43:54.401389+0000","last_became_peered":"2023-08-25T09:43:54.401389+0000","last_unstale":"2023-08-25T09:43:55.676440+0000","last_undegraded":"2023-08-25T09:43:55.676440+0000","last_fullsized":"2023-08-25T09:43:55.676440+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:55.776311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389283+0000","last_change":"2023-08-25T09:43:54.416278+0000","last_active":"2023-08-25T09:43:55.389283+0000","last_peered":"2023-08-25T09:43:55.389283+0000","last_clean":"2023-08-25T09:43:55.389283+0000","last_became_active":"2023-08-25T09:43:54.415330+0000","last_became_peered":"2023-08-25T09:43:54.415330+0000","last_unstale":"2023-08-25T09:43:55.389283+0000","last_undegraded":"2023-08-25T09:43:55.389283+0000","last_fullsized":"2023-08-25T09:43:55.389283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:25.392124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675633+0000","last_change":"2023-08-25T09:43:54.413935+0000","last_active":"2023-08-25T09:43:55.675633+0000","last_peered":"2023-08-25T09:43:55.675633+0000","last_clean":"2023-08-25T09:43:55.675633+0000","last_became_active":"2023-08-25T09:43:54.413685+0000","last_became_peered":"2023-08-25T09:43:54.413685+0000","last_unstale":"2023-08-25T09:43:55.675633+0000","last_undegraded":"2023-08-25T09:43:55.675633+0000","last_fullsized":"2023-08-25T09:43:55.675633+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:09.188639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677337+0000","last_change":"2023-08-25T09:43:54.409395+0000","last_active":"2023-08-25T09:43:55.677337+0000","last_peered":"2023-08-25T09:43:55.677337+0000","last_clean":"2023-08-25T09:43:55.677337+0000","last_became_active":"2023-08-25T09:43:54.409083+0000","last_became_peered":"2023-08-25T09:43:54.409083+0000","last_unstale":"2023-08-25T09:43:55.677337+0000","last_undegraded":"2023-08-25T09:43:55.677337+0000","last_fullsized":"2023-08-25T09:43:55.677337+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:27.799378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530267+0000","last_change":"2023-08-25T09:43:54.407151+0000","last_active":"2023-08-25T09:43:55.530267+0000","last_peered":"2023-08-25T09:43:55.530267+0000","last_clean":"2023-08-25T09:43:55.530267+0000","last_became_active":"2023-08-25T09:43:54.406266+0000","last_became_peered":"2023-08-25T09:43:54.406266+0000","last_unstale":"2023-08-25T09:43:55.530267+0000","last_undegraded":"2023-08-25T09:43:55.530267+0000","last_fullsized":"2023-08-25T09:43:55.530267+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:46:18.101385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676097+0000","last_change":"2023-08-25T09:43:54.408942+0000","last_active":"2023-08-25T09:43:55.676097+0000","last_peered":"2023-08-25T09:43:55.676097+0000","last_clean":"2023-08-25T09:43:55.676097+0000","last_became_active":"2023-08-25T09:43:54.408439+0000","last_became_peered":"2023-08-25T09:43:54.408439+0000","last_unstale":"2023-08-25T09:43:55.676097+0000","last_undegraded":"2023-08-25T09:43:55.676097+0000","last_fullsized":"2023-08-25T09:43:55.676097+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:53.991517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529111+0000","last_change":"2023-08-25T09:43:54.419056+0000","last_active":"2023-08-25T09:43:55.529111+0000","last_peered":"2023-08-25T09:43:55.529111+0000","last_clean":"2023-08-25T09:43:55.529111+0000","last_became_active":"2023-08-25T09:43:54.418667+0000","last_became_peered":"2023-08-25T09:43:54.418667+0000","last_unstale":"2023-08-25T09:43:55.529111+0000","last_undegraded":"2023-08-25T09:43:55.529111+0000","last_fullsized":"2023-08-25T09:43:55.529111+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:30.225126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387796+0000","last_change":"2023-08-25T09:43:54.415935+0000","last_active":"2023-08-25T09:43:55.387796+0000","last_peered":"2023-08-25T09:43:55.387796+0000","last_clean":"2023-08-25T09:43:55.387796+0000","last_became_active":"2023-08-25T09:43:54.414902+0000","last_became_peered":"2023-08-25T09:43:54.414902+0000","last_unstale":"2023-08-25T09:43:55.387796+0000","last_undegraded":"2023-08-25T09:43:55.387796+0000","last_fullsized":"2023-08-25T09:43:55.387796+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:05:26.982246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676861+0000","last_change":"2023-08-25T09:43:54.404468+0000","last_active":"2023-08-25T09:43:55.676861+0000","last_peered":"2023-08-25T09:43:55.676861+0000","last_clean":"2023-08-25T09:43:55.676861+0000","last_became_active":"2023-08-25T09:43:54.403839+0000","last_became_peered":"2023-08-25T09:43:54.403839+0000","last_unstale":"2023-08-25T09:43:55.676861+0000","last_undegraded":"2023-08-25T09:43:55.676861+0000","last_fullsized":"2023-08-25T09:43:55.676861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:17:25.809595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387622+0000","last_change":"2023-08-25T09:43:54.408393+0000","last_active":"2023-08-25T09:43:55.387622+0000","last_peered":"2023-08-25T09:43:55.387622+0000","last_clean":"2023-08-25T09:43:55.387622+0000","last_became_active":"2023-08-25T09:43:54.407528+0000","last_became_peered":"2023-08-25T09:43:54.407528+0000","last_unstale":"2023-08-25T09:43:55.387622+0000","last_undegraded":"2023-08-25T09:43:55.387622+0000","last_fullsized":"2023-08-25T09:43:55.387622+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:46.864971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530141+0000","last_change":"2023-08-25T09:43:54.409118+0000","last_active":"2023-08-25T09:43:55.530141+0000","last_peered":"2023-08-25T09:43:55.530141+0000","last_clean":"2023-08-25T09:43:55.530141+0000","last_became_active":"2023-08-25T09:43:54.408893+0000","last_became_peered":"2023-08-25T09:43:54.408893+0000","last_unstale":"2023-08-25T09:43:55.530141+0000","last_undegraded":"2023-08-25T09:43:55.530141+0000","last_fullsized":"2023-08-25T09:43:55.530141+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:28:12.464768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389643+0000","last_change":"2023-08-25T09:43:54.417738+0000","last_active":"2023-08-25T09:43:55.389643+0000","last_peered":"2023-08-25T09:43:55.389643+0000","last_clean":"2023-08-25T09:43:55.389643+0000","last_became_active":"2023-08-25T09:43:54.415654+0000","last_became_peered":"2023-08-25T09:43:54.415654+0000","last_unstale":"2023-08-25T09:43:55.389643+0000","last_undegraded":"2023-08-25T09:43:55.389643+0000","last_fullsized":"2023-08-25T09:43:55.389643+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:35.656446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677379+0000","last_change":"2023-08-25T09:43:54.394213+0000","last_active":"2023-08-25T09:43:55.677379+0000","last_peered":"2023-08-25T09:43:55.677379+0000","last_clean":"2023-08-25T09:43:55.677379+0000","last_became_active":"2023-08-25T09:43:54.393647+0000","last_became_peered":"2023-08-25T09:43:54.393647+0000","last_unstale":"2023-08-25T09:43:55.677379+0000","last_undegraded":"2023-08-25T09:43:55.677379+0000","last_fullsized":"2023-08-25T09:43:55.677379+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:17:39.470885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530362+0000","last_change":"2023-08-25T09:43:54.408147+0000","last_active":"2023-08-25T09:43:55.530362+0000","last_peered":"2023-08-25T09:43:55.530362+0000","last_clean":"2023-08-25T09:43:55.530362+0000","last_became_active":"2023-08-25T09:43:54.407786+0000","last_became_peered":"2023-08-25T09:43:54.407786+0000","last_unstale":"2023-08-25T09:43:55.530362+0000","last_undegraded":"2023-08-25T09:43:55.530362+0000","last_fullsized":"2023-08-25T09:43:55.530362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:24.779031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387994+0000","last_change":"2023-08-25T09:43:54.407208+0000","last_active":"2023-08-25T09:43:55.387994+0000","last_peered":"2023-08-25T09:43:55.387994+0000","last_clean":"2023-08-25T09:43:55.387994+0000","last_became_active":"2023-08-25T09:43:54.406324+0000","last_became_peered":"2023-08-25T09:43:54.406324+0000","last_unstale":"2023-08-25T09:43:55.387994+0000","last_undegraded":"2023-08-25T09:43:55.387994+0000","last_fullsized":"2023-08-25T09:43:55.387994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:41:00.393632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.531168+0000","last_change":"2023-08-25T09:43:54.415832+0000","last_active":"2023-08-25T09:43:55.531168+0000","last_peered":"2023-08-25T09:43:55.531168+0000","last_clean":"2023-08-25T09:43:55.531168+0000","last_became_active":"2023-08-25T09:43:54.415552+0000","last_became_peered":"2023-08-25T09:43:54.415552+0000","last_unstale":"2023-08-25T09:43:55.531168+0000","last_undegraded":"2023-08-25T09:43:55.531168+0000","last_fullsized":"2023-08-25T09:43:55.531168+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:09:29.490496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676471+0000","last_change":"2023-08-25T09:43:54.402938+0000","last_active":"2023-08-25T09:43:55.676471+0000","last_peered":"2023-08-25T09:43:55.676471+0000","last_clean":"2023-08-25T09:43:55.676471+0000","last_became_active":"2023-08-25T09:43:54.400897+0000","last_became_peered":"2023-08-25T09:43:54.400897+0000","last_unstale":"2023-08-25T09:43:55.676471+0000","last_undegraded":"2023-08-25T09:43:55.676471+0000","last_fullsized":"2023-08-25T09:43:55.676471+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:48:44.889724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676124+0000","last_change":"2023-08-25T09:43:54.412294+0000","last_active":"2023-08-25T09:43:55.676124+0000","last_peered":"2023-08-25T09:43:55.676124+0000","last_clean":"2023-08-25T09:43:55.676124+0000","last_became_active":"2023-08-25T09:43:54.412114+0000","last_became_peered":"2023-08-25T09:43:54.412114+0000","last_unstale":"2023-08-25T09:43:55.676124+0000","last_undegraded":"2023-08-25T09:43:55.676124+0000","last_fullsized":"2023-08-25T09:43:55.676124+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:42:21.825196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387480+0000","last_change":"2023-08-25T09:43:54.408210+0000","last_active":"2023-08-25T09:43:55.387480+0000","last_peered":"2023-08-25T09:43:55.387480+0000","last_clean":"2023-08-25T09:43:55.387480+0000","last_became_active":"2023-08-25T09:43:54.407960+0000","last_became_peered":"2023-08-25T09:43:54.407960+0000","last_unstale":"2023-08-25T09:43:55.387480+0000","last_undegraded":"2023-08-25T09:43:55.387480+0000","last_fullsized":"2023-08-25T09:43:55.387480+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:13.932322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.528897+0000","last_change":"2023-08-25T09:43:54.402889+0000","last_active":"2023-08-25T09:43:55.528897+0000","last_peered":"2023-08-25T09:43:55.528897+0000","last_clean":"2023-08-25T09:43:55.528897+0000","last_became_active":"2023-08-25T09:43:54.402611+0000","last_became_peered":"2023-08-25T09:43:54.402611+0000","last_unstale":"2023-08-25T09:43:55.528897+0000","last_undegraded":"2023-08-25T09:43:55.528897+0000","last_fullsized":"2023-08-25T09:43:55.528897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:15.783761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.386759+0000","last_change":"2023-08-25T09:43:54.407433+0000","last_active":"2023-08-25T09:43:55.386759+0000","last_peered":"2023-08-25T09:43:55.386759+0000","last_clean":"2023-08-25T09:43:55.386759+0000","last_became_active":"2023-08-25T09:43:54.407190+0000","last_became_peered":"2023-08-25T09:43:54.407190+0000","last_unstale":"2023-08-25T09:43:55.386759+0000","last_undegraded":"2023-08-25T09:43:55.386759+0000","last_fullsized":"2023-08-25T09:43:55.386759+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:27.119175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675738+0000","last_change":"2023-08-25T09:43:54.414015+0000","last_active":"2023-08-25T09:43:55.675738+0000","last_peered":"2023-08-25T09:43:55.675738+0000","last_clean":"2023-08-25T09:43:55.675738+0000","last_became_active":"2023-08-25T09:43:54.412759+0000","last_became_peered":"2023-08-25T09:43:54.412759+0000","last_unstale":"2023-08-25T09:43:55.675738+0000","last_undegraded":"2023-08-25T09:43:55.675738+0000","last_fullsized":"2023-08-25T09:43:55.675738+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:36:18.576641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677241+0000","last_change":"2023-08-25T09:43:54.412030+0000","last_active":"2023-08-25T09:43:55.677241+0000","last_peered":"2023-08-25T09:43:55.677241+0000","last_clean":"2023-08-25T09:43:55.677241+0000","last_became_active":"2023-08-25T09:43:54.411737+0000","last_became_peered":"2023-08-25T09:43:54.411737+0000","last_unstale":"2023-08-25T09:43:55.677241+0000","last_undegraded":"2023-08-25T09:43:55.677241+0000","last_fullsized":"2023-08-25T09:43:55.677241+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:51.546066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678060+0000","last_change":"2023-08-25T09:43:54.392602+0000","last_active":"2023-08-25T09:43:55.678060+0000","last_peered":"2023-08-25T09:43:55.678060+0000","last_clean":"2023-08-25T09:43:55.678060+0000","last_became_active":"2023-08-25T09:43:54.392310+0000","last_became_peered":"2023-08-25T09:43:54.392310+0000","last_unstale":"2023-08-25T09:43:55.678060+0000","last_undegraded":"2023-08-25T09:43:55.678060+0000","last_fullsized":"2023-08-25T09:43:55.678060+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:38.123428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388355+0000","last_change":"2023-08-25T09:43:54.411259+0000","last_active":"2023-08-25T09:43:55.388355+0000","last_peered":"2023-08-25T09:43:55.388355+0000","last_clean":"2023-08-25T09:43:55.388355+0000","last_became_active":"2023-08-25T09:43:54.411008+0000","last_became_peered":"2023-08-25T09:43:54.411008+0000","last_unstale":"2023-08-25T09:43:55.388355+0000","last_undegraded":"2023-08-25T09:43:55.388355+0000","last_fullsized":"2023-08-25T09:43:55.388355+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:29:35.594192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675937+0000","last_change":"2023-08-25T09:43:54.411396+0000","last_active":"2023-08-25T09:43:55.675937+0000","last_peered":"2023-08-25T09:43:55.675937+0000","last_clean":"2023-08-25T09:43:55.675937+0000","last_became_active":"2023-08-25T09:43:54.410849+0000","last_became_peered":"2023-08-25T09:43:54.410849+0000","last_unstale":"2023-08-25T09:43:55.675937+0000","last_undegraded":"2023-08-25T09:43:55.675937+0000","last_fullsized":"2023-08-25T09:43:55.675937+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:51.094231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.528980+0000","last_change":"2023-08-25T09:43:54.418899+0000","last_active":"2023-08-25T09:43:55.528980+0000","last_peered":"2023-08-25T09:43:55.528980+0000","last_clean":"2023-08-25T09:43:55.528980+0000","last_became_active":"2023-08-25T09:43:54.417852+0000","last_became_peered":"2023-08-25T09:43:54.417852+0000","last_unstale":"2023-08-25T09:43:55.528980+0000","last_undegraded":"2023-08-25T09:43:55.528980+0000","last_fullsized":"2023-08-25T09:43:55.528980+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:42:09.899923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388185+0000","last_change":"2023-08-25T09:43:54.407528+0000","last_active":"2023-08-25T09:43:55.388185+0000","last_peered":"2023-08-25T09:43:55.388185+0000","last_clean":"2023-08-25T09:43:55.388185+0000","last_became_active":"2023-08-25T09:43:54.407006+0000","last_became_peered":"2023-08-25T09:43:54.407006+0000","last_unstale":"2023-08-25T09:43:55.388185+0000","last_undegraded":"2023-08-25T09:43:55.388185+0000","last_fullsized":"2023-08-25T09:43:55.388185+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:03.387238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678374+0000","last_change":"2023-08-25T09:43:54.407447+0000","last_active":"2023-08-25T09:43:55.678374+0000","last_peered":"2023-08-25T09:43:55.678374+0000","last_clean":"2023-08-25T09:43:55.678374+0000","last_became_active":"2023-08-25T09:43:54.407225+0000","last_became_peered":"2023-08-25T09:43:54.407225+0000","last_unstale":"2023-08-25T09:43:55.678374+0000","last_undegraded":"2023-08-25T09:43:55.678374+0000","last_fullsized":"2023-08-25T09:43:55.678374+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:18:36.366812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.390000+0000","last_change":"2023-08-25T09:43:54.417973+0000","last_active":"2023-08-25T09:43:55.390000+0000","last_peered":"2023-08-25T09:43:55.390000+0000","last_clean":"2023-08-25T09:43:55.390000+0000","last_became_active":"2023-08-25T09:43:54.416003+0000","last_became_peered":"2023-08-25T09:43:54.416003+0000","last_unstale":"2023-08-25T09:43:55.390000+0000","last_undegraded":"2023-08-25T09:43:55.390000+0000","last_fullsized":"2023-08-25T09:43:55.390000+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:22.319316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"93'4","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389004+0000","last_change":"2023-08-25T09:43:53.397210+0000","last_active":"2023-08-25T09:43:55.389004+0000","last_peered":"2023-08-25T09:43:55.389004+0000","last_clean":"2023-08-25T09:43:55.389004+0000","last_became_active":"2023-08-25T09:43:53.396946+0000","last_became_peered":"2023-08-25T09:43:53.396946+0000","last_unstale":"2023-08-25T09:43:55.389004+0000","last_undegraded":"2023-08-25T09:43:55.389004+0000","last_fullsized":"2023-08-25T09:43:55.389004+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:22.103597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387784+0000","last_change":"2023-08-25T09:43:54.413793+0000","last_active":"2023-08-25T09:43:55.387784+0000","last_peered":"2023-08-25T09:43:55.387784+0000","last_clean":"2023-08-25T09:43:55.387784+0000","last_became_active":"2023-08-25T09:43:54.413378+0000","last_became_peered":"2023-08-25T09:43:54.413378+0000","last_unstale":"2023-08-25T09:43:55.387784+0000","last_undegraded":"2023-08-25T09:43:55.387784+0000","last_fullsized":"2023-08-25T09:43:55.387784+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:28:34.617151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384615+0000","last_change":"2023-08-25T09:43:53.393077+0000","last_active":"2023-08-25T09:43:55.384615+0000","last_peered":"2023-08-25T09:43:55.384615+0000","last_clean":"2023-08-25T09:43:55.384615+0000","last_became_active":"2023-08-25T09:43:53.392575+0000","last_became_peered":"2023-08-25T09:43:53.392575+0000","last_unstale":"2023-08-25T09:43:55.384615+0000","last_undegraded":"2023-08-25T09:43:55.384615+0000","last_fullsized":"2023-08-25T09:43:55.384615+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:04.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677126+0000","last_change":"2023-08-25T09:43:54.409806+0000","last_active":"2023-08-25T09:43:55.677126+0000","last_peered":"2023-08-25T09:43:55.677126+0000","last_clean":"2023-08-25T09:43:55.677126+0000","last_became_active":"2023-08-25T09:43:54.409518+0000","last_became_peered":"2023-08-25T09:43:54.409518+0000","last_unstale":"2023-08-25T09:43:55.677126+0000","last_undegraded":"2023-08-25T09:43:55.677126+0000","last_fullsized":"2023-08-25T09:43:55.677126+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:46.240304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529263+0000","last_change":"2023-08-25T09:43:53.398957+0000","last_active":"2023-08-25T09:43:55.529263+0000","last_peered":"2023-08-25T09:43:55.529263+0000","last_clean":"2023-08-25T09:43:55.529263+0000","last_became_active":"2023-08-25T09:43:53.398717+0000","last_became_peered":"2023-08-25T09:43:53.398717+0000","last_unstale":"2023-08-25T09:43:55.529263+0000","last_undegraded":"2023-08-25T09:43:55.529263+0000","last_fullsized":"2023-08-25T09:43:55.529263+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:49.791339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677734+0000","last_change":"2023-08-25T09:43:54.401591+0000","last_active":"2023-08-25T09:43:55.677734+0000","last_peered":"2023-08-25T09:43:55.677734+0000","last_clean":"2023-08-25T09:43:55.677734+0000","last_became_active":"2023-08-25T09:43:54.401293+0000","last_became_peered":"2023-08-25T09:43:54.401293+0000","last_unstale":"2023-08-25T09:43:55.677734+0000","last_undegraded":"2023-08-25T09:43:55.677734+0000","last_fullsized":"2023-08-25T09:43:55.677734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:46:18.935110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676265+0000","last_change":"2023-08-25T09:43:53.389534+0000","last_active":"2023-08-25T09:43:55.676265+0000","last_peered":"2023-08-25T09:43:55.676265+0000","last_clean":"2023-08-25T09:43:55.676265+0000","last_became_active":"2023-08-25T09:43:53.389145+0000","last_became_peered":"2023-08-25T09:43:53.389145+0000","last_unstale":"2023-08-25T09:43:55.676265+0000","last_undegraded":"2023-08-25T09:43:55.676265+0000","last_fullsized":"2023-08-25T09:43:55.676265+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:06.592553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676066+0000","last_change":"2023-08-25T09:43:54.401991+0000","last_active":"2023-08-25T09:43:55.676066+0000","last_peered":"2023-08-25T09:43:55.676066+0000","last_clean":"2023-08-25T09:43:55.676066+0000","last_became_active":"2023-08-25T09:43:54.401631+0000","last_became_peered":"2023-08-25T09:43:54.401631+0000","last_unstale":"2023-08-25T09:43:55.676066+0000","last_undegraded":"2023-08-25T09:43:55.676066+0000","last_fullsized":"2023-08-25T09:43:55.676066+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:14:04.265830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678143+0000","last_change":"2023-08-25T09:43:53.385163+0000","last_active":"2023-08-25T09:43:55.678143+0000","last_peered":"2023-08-25T09:43:55.678143+0000","last_clean":"2023-08-25T09:43:55.678143+0000","last_became_active":"2023-08-25T09:43:53.384839+0000","last_became_peered":"2023-08-25T09:43:53.384839+0000","last_unstale":"2023-08-25T09:43:55.678143+0000","last_undegraded":"2023-08-25T09:43:55.678143+0000","last_fullsized":"2023-08-25T09:43:55.678143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:50:28.252589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387331+0000","last_change":"2023-08-25T09:43:54.402776+0000","last_active":"2023-08-25T09:43:55.387331+0000","last_peered":"2023-08-25T09:43:55.387331+0000","last_clean":"2023-08-25T09:43:55.387331+0000","last_became_active":"2023-08-25T09:43:54.401343+0000","last_became_peered":"2023-08-25T09:43:54.401343+0000","last_unstale":"2023-08-25T09:43:55.387331+0000","last_undegraded":"2023-08-25T09:43:55.387331+0000","last_fullsized":"2023-08-25T09:43:55.387331+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:48:26.864145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385440+0000","last_change":"2023-08-25T09:43:53.401154+0000","last_active":"2023-08-25T09:43:55.385440+0000","last_peered":"2023-08-25T09:43:55.385440+0000","last_clean":"2023-08-25T09:43:55.385440+0000","last_became_active":"2023-08-25T09:43:53.400830+0000","last_became_peered":"2023-08-25T09:43:53.400830+0000","last_unstale":"2023-08-25T09:43:55.385440+0000","last_undegraded":"2023-08-25T09:43:55.385440+0000","last_fullsized":"2023-08-25T09:43:55.385440+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:56:52.629469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676387+0000","last_change":"2023-08-25T09:43:54.413882+0000","last_active":"2023-08-25T09:43:55.676387+0000","last_peered":"2023-08-25T09:43:55.676387+0000","last_clean":"2023-08-25T09:43:55.676387+0000","last_became_active":"2023-08-25T09:43:54.413721+0000","last_became_peered":"2023-08-25T09:43:54.413721+0000","last_unstale":"2023-08-25T09:43:55.676387+0000","last_undegraded":"2023-08-25T09:43:55.676387+0000","last_fullsized":"2023-08-25T09:43:55.676387+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:43:17.280367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530798+0000","last_change":"2023-08-25T09:43:53.401459+0000","last_active":"2023-08-25T09:43:55.530798+0000","last_peered":"2023-08-25T09:43:55.530798+0000","last_clean":"2023-08-25T09:43:55.530798+0000","last_became_active":"2023-08-25T09:43:53.400825+0000","last_became_peered":"2023-08-25T09:43:53.400825+0000","last_unstale":"2023-08-25T09:43:55.530798+0000","last_undegraded":"2023-08-25T09:43:55.530798+0000","last_fullsized":"2023-08-25T09:43:55.530798+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:14.912189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385020+0000","last_change":"2023-08-25T09:43:54.398080+0000","last_active":"2023-08-25T09:43:55.385020+0000","last_peered":"2023-08-25T09:43:55.385020+0000","last_clean":"2023-08-25T09:43:55.385020+0000","last_became_active":"2023-08-25T09:43:54.397273+0000","last_became_peered":"2023-08-25T09:43:54.397273+0000","last_unstale":"2023-08-25T09:43:55.385020+0000","last_undegraded":"2023-08-25T09:43:55.385020+0000","last_fullsized":"2023-08-25T09:43:55.385020+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:16:01.756935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387956+0000","last_change":"2023-08-25T09:43:53.399348+0000","last_active":"2023-08-25T09:43:55.387956+0000","last_peered":"2023-08-25T09:43:55.387956+0000","last_clean":"2023-08-25T09:43:55.387956+0000","last_became_active":"2023-08-25T09:43:53.399099+0000","last_became_peered":"2023-08-25T09:43:53.399099+0000","last_unstale":"2023-08-25T09:43:55.387956+0000","last_undegraded":"2023-08-25T09:43:55.387956+0000","last_fullsized":"2023-08-25T09:43:55.387956+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:06:36.086345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388332+0000","last_change":"2023-08-25T09:43:54.406818+0000","last_active":"2023-08-25T09:43:55.388332+0000","last_peered":"2023-08-25T09:43:55.388332+0000","last_clean":"2023-08-25T09:43:55.388332+0000","last_became_active":"2023-08-25T09:43:54.405375+0000","last_became_peered":"2023-08-25T09:43:54.405375+0000","last_unstale":"2023-08-25T09:43:55.388332+0000","last_undegraded":"2023-08-25T09:43:55.388332+0000","last_fullsized":"2023-08-25T09:43:55.388332+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:30.258111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677565+0000","last_change":"2023-08-25T09:43:53.397375+0000","last_active":"2023-08-25T09:43:55.677565+0000","last_peered":"2023-08-25T09:43:55.677565+0000","last_clean":"2023-08-25T09:43:55.677565+0000","last_became_active":"2023-08-25T09:43:53.397119+0000","last_became_peered":"2023-08-25T09:43:53.397119+0000","last_unstale":"2023-08-25T09:43:55.677565+0000","last_undegraded":"2023-08-25T09:43:55.677565+0000","last_fullsized":"2023-08-25T09:43:55.677565+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:20:30.983138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677836+0000","last_change":"2023-08-25T09:43:54.400636+0000","last_active":"2023-08-25T09:43:55.677836+0000","last_peered":"2023-08-25T09:43:55.677836+0000","last_clean":"2023-08-25T09:43:55.677836+0000","last_became_active":"2023-08-25T09:43:54.400265+0000","last_became_peered":"2023-08-25T09:43:54.400265+0000","last_unstale":"2023-08-25T09:43:55.677836+0000","last_undegraded":"2023-08-25T09:43:55.677836+0000","last_fullsized":"2023-08-25T09:43:55.677836+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:06:46.573434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529113+0000","last_change":"2023-08-25T09:43:53.399279+0000","last_active":"2023-08-25T09:43:55.529113+0000","last_peered":"2023-08-25T09:43:55.529113+0000","last_clean":"2023-08-25T09:43:55.529113+0000","last_became_active":"2023-08-25T09:43:53.399023+0000","last_became_peered":"2023-08-25T09:43:53.399023+0000","last_unstale":"2023-08-25T09:43:55.529113+0000","last_undegraded":"2023-08-25T09:43:55.529113+0000","last_fullsized":"2023-08-25T09:43:55.529113+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:14:04.107168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530821+0000","last_change":"2023-08-25T09:43:54.411304+0000","last_active":"2023-08-25T09:43:55.530821+0000","last_peered":"2023-08-25T09:43:55.530821+0000","last_clean":"2023-08-25T09:43:55.530821+0000","last_became_active":"2023-08-25T09:43:54.410775+0000","last_became_peered":"2023-08-25T09:43:54.410775+0000","last_unstale":"2023-08-25T09:43:55.530821+0000","last_undegraded":"2023-08-25T09:43:55.530821+0000","last_fullsized":"2023-08-25T09:43:55.530821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:00.580658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"93'3","reported_seq":16,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675501+0000","last_change":"2023-08-25T09:43:53.400335+0000","last_active":"2023-08-25T09:43:55.675501+0000","last_peered":"2023-08-25T09:43:55.675501+0000","last_clean":"2023-08-25T09:43:55.675501+0000","last_became_active":"2023-08-25T09:43:53.397372+0000","last_became_peered":"2023-08-25T09:43:53.397372+0000","last_unstale":"2023-08-25T09:43:55.675501+0000","last_undegraded":"2023-08-25T09:43:55.675501+0000","last_fullsized":"2023-08-25T09:43:55.675501+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:57.202891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676283+0000","last_change":"2023-08-25T09:43:54.412139+0000","last_active":"2023-08-25T09:43:55.676283+0000","last_peered":"2023-08-25T09:43:55.676283+0000","last_clean":"2023-08-25T09:43:55.676283+0000","last_became_active":"2023-08-25T09:43:54.402590+0000","last_became_peered":"2023-08-25T09:43:54.402590+0000","last_unstale":"2023-08-25T09:43:55.676283+0000","last_undegraded":"2023-08-25T09:43:55.676283+0000","last_fullsized":"2023-08-25T09:43:55.676283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:44.502910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"93'6","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384787+0000","last_change":"2023-08-25T09:43:53.399357+0000","last_active":"2023-08-25T09:43:55.384787+0000","last_peered":"2023-08-25T09:43:55.384787+0000","last_clean":"2023-08-25T09:43:55.384787+0000","last_became_active":"2023-08-25T09:43:53.398803+0000","last_became_peered":"2023-08-25T09:43:53.398803+0000","last_unstale":"2023-08-25T09:43:55.384787+0000","last_undegraded":"2023-08-25T09:43:55.384787+0000","last_fullsized":"2023-08-25T09:43:55.384787+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:41.985823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677987+0000","last_change":"2023-08-25T09:43:54.408743+0000","last_active":"2023-08-25T09:43:55.677987+0000","last_peered":"2023-08-25T09:43:55.677987+0000","last_clean":"2023-08-25T09:43:55.677987+0000","last_became_active":"2023-08-25T09:43:54.407981+0000","last_became_peered":"2023-08-25T09:43:54.407981+0000","last_unstale":"2023-08-25T09:43:55.677987+0000","last_undegraded":"2023-08-25T09:43:55.677987+0000","last_fullsized":"2023-08-25T09:43:55.677987+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:25:22.647516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529338+0000","last_change":"2023-08-25T09:43:53.400186+0000","last_active":"2023-08-25T09:43:55.529338+0000","last_peered":"2023-08-25T09:43:55.529338+0000","last_clean":"2023-08-25T09:43:55.529338+0000","last_became_active":"2023-08-25T09:43:53.399735+0000","last_became_peered":"2023-08-25T09:43:53.399735+0000","last_unstale":"2023-08-25T09:43:55.529338+0000","last_undegraded":"2023-08-25T09:43:55.529338+0000","last_fullsized":"2023-08-25T09:43:55.529338+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:58.045564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529095+0000","last_change":"2023-08-25T09:43:54.408566+0000","last_active":"2023-08-25T09:43:55.529095+0000","last_peered":"2023-08-25T09:43:55.529095+0000","last_clean":"2023-08-25T09:43:55.529095+0000","last_became_active":"2023-08-25T09:43:54.408126+0000","last_became_peered":"2023-08-25T09:43:54.408126+0000","last_unstale":"2023-08-25T09:43:55.529095+0000","last_undegraded":"2023-08-25T09:43:55.529095+0000","last_fullsized":"2023-08-25T09:43:55.529095+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:55:51.838016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676385+0000","last_change":"2023-08-25T09:43:53.394175+0000","last_active":"2023-08-25T09:43:55.676385+0000","last_peered":"2023-08-25T09:43:55.676385+0000","last_clean":"2023-08-25T09:43:55.676385+0000","last_became_active":"2023-08-25T09:43:53.393635+0000","last_became_peered":"2023-08-25T09:43:53.393635+0000","last_unstale":"2023-08-25T09:43:55.676385+0000","last_undegraded":"2023-08-25T09:43:55.676385+0000","last_fullsized":"2023-08-25T09:43:55.676385+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:05.065902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677189+0000","last_change":"2023-08-25T09:43:54.397561+0000","last_active":"2023-08-25T09:43:55.677189+0000","last_peered":"2023-08-25T09:43:55.677189+0000","last_clean":"2023-08-25T09:43:55.677189+0000","last_became_active":"2023-08-25T09:43:54.397258+0000","last_became_peered":"2023-08-25T09:43:54.397258+0000","last_unstale":"2023-08-25T09:43:55.677189+0000","last_undegraded":"2023-08-25T09:43:55.677189+0000","last_fullsized":"2023-08-25T09:43:55.677189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:52.770025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389288+0000","last_change":"2023-08-25T09:43:53.403154+0000","last_active":"2023-08-25T09:43:55.389288+0000","last_peered":"2023-08-25T09:43:55.389288+0000","last_clean":"2023-08-25T09:43:55.389288+0000","last_became_active":"2023-08-25T09:43:53.399755+0000","last_became_peered":"2023-08-25T09:43:53.399755+0000","last_unstale":"2023-08-25T09:43:55.389288+0000","last_undegraded":"2023-08-25T09:43:55.389288+0000","last_fullsized":"2023-08-25T09:43:55.389288+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:14.044339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529191+0000","last_change":"2023-08-25T09:43:54.406065+0000","last_active":"2023-08-25T09:43:55.529191+0000","last_peered":"2023-08-25T09:43:55.529191+0000","last_clean":"2023-08-25T09:43:55.529191+0000","last_became_active":"2023-08-25T09:43:54.405822+0000","last_became_peered":"2023-08-25T09:43:54.405822+0000","last_unstale":"2023-08-25T09:43:55.529191+0000","last_undegraded":"2023-08-25T09:43:55.529191+0000","last_fullsized":"2023-08-25T09:43:55.529191+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:21.088250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"93'8","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.675963+0000","last_change":"2023-08-25T09:43:53.403310+0000","last_active":"2023-08-25T09:43:55.675963+0000","last_peered":"2023-08-25T09:43:55.675963+0000","last_clean":"2023-08-25T09:43:55.675963+0000","last_became_active":"2023-08-25T09:43:53.400976+0000","last_became_peered":"2023-08-25T09:43:53.400976+0000","last_unstale":"2023-08-25T09:43:55.675963+0000","last_undegraded":"2023-08-25T09:43:55.675963+0000","last_fullsized":"2023-08-25T09:43:55.675963+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:05:07.588952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677139+0000","last_change":"2023-08-25T09:43:54.394896+0000","last_active":"2023-08-25T09:43:55.677139+0000","last_peered":"2023-08-25T09:43:55.677139+0000","last_clean":"2023-08-25T09:43:55.677139+0000","last_became_active":"2023-08-25T09:43:54.394161+0000","last_became_peered":"2023-08-25T09:43:54.394161+0000","last_unstale":"2023-08-25T09:43:55.677139+0000","last_undegraded":"2023-08-25T09:43:55.677139+0000","last_fullsized":"2023-08-25T09:43:55.677139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:03:43.678318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529257+0000","last_change":"2023-08-25T09:43:53.400229+0000","last_active":"2023-08-25T09:43:55.529257+0000","last_peered":"2023-08-25T09:43:55.529257+0000","last_clean":"2023-08-25T09:43:55.529257+0000","last_became_active":"2023-08-25T09:43:53.400007+0000","last_became_peered":"2023-08-25T09:43:53.400007+0000","last_unstale":"2023-08-25T09:43:55.529257+0000","last_undegraded":"2023-08-25T09:43:55.529257+0000","last_fullsized":"2023-08-25T09:43:55.529257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:17:23.839606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529294+0000","last_change":"2023-08-25T09:43:54.408421+0000","last_active":"2023-08-25T09:43:55.529294+0000","last_peered":"2023-08-25T09:43:55.529294+0000","last_clean":"2023-08-25T09:43:55.529294+0000","last_became_active":"2023-08-25T09:43:54.407614+0000","last_became_peered":"2023-08-25T09:43:54.407614+0000","last_unstale":"2023-08-25T09:43:55.529294+0000","last_undegraded":"2023-08-25T09:43:55.529294+0000","last_fullsized":"2023-08-25T09:43:55.529294+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:15.802242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"94'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676384+0000","last_change":"2023-08-25T09:43:53.400677+0000","last_active":"2023-08-25T09:43:55.676384+0000","last_peered":"2023-08-25T09:43:55.676384+0000","last_clean":"2023-08-25T09:43:55.676384+0000","last_became_active":"2023-08-25T09:43:53.397908+0000","last_became_peered":"2023-08-25T09:43:53.397908+0000","last_unstale":"2023-08-25T09:43:55.676384+0000","last_undegraded":"2023-08-25T09:43:55.676384+0000","last_fullsized":"2023-08-25T09:43:55.676384+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:45.920977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677501+0000","last_change":"2023-08-25T09:43:54.408623+0000","last_active":"2023-08-25T09:43:55.677501+0000","last_peered":"2023-08-25T09:43:55.677501+0000","last_clean":"2023-08-25T09:43:55.677501+0000","last_became_active":"2023-08-25T09:43:54.408346+0000","last_became_peered":"2023-08-25T09:43:54.408346+0000","last_unstale":"2023-08-25T09:43:55.677501+0000","last_undegraded":"2023-08-25T09:43:55.677501+0000","last_fullsized":"2023-08-25T09:43:55.677501+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:39.048547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529155+0000","last_change":"2023-08-25T09:43:53.397556+0000","last_active":"2023-08-25T09:43:55.529155+0000","last_peered":"2023-08-25T09:43:55.529155+0000","last_clean":"2023-08-25T09:43:55.529155+0000","last_became_active":"2023-08-25T09:43:53.397131+0000","last_became_peered":"2023-08-25T09:43:53.397131+0000","last_unstale":"2023-08-25T09:43:55.529155+0000","last_undegraded":"2023-08-25T09:43:55.529155+0000","last_fullsized":"2023-08-25T09:43:55.529155+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:34:31.868462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.531351+0000","last_change":"2023-08-25T09:43:54.411155+0000","last_active":"2023-08-25T09:43:55.531351+0000","last_peered":"2023-08-25T09:43:55.531351+0000","last_clean":"2023-08-25T09:43:55.531351+0000","last_became_active":"2023-08-25T09:43:54.410442+0000","last_became_peered":"2023-08-25T09:43:54.410442+0000","last_unstale":"2023-08-25T09:43:55.531351+0000","last_undegraded":"2023-08-25T09:43:55.531351+0000","last_fullsized":"2023-08-25T09:43:55.531351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:06:27.055748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677890+0000","last_change":"2023-08-25T09:43:53.390461+0000","last_active":"2023-08-25T09:43:55.677890+0000","last_peered":"2023-08-25T09:43:55.677890+0000","last_clean":"2023-08-25T09:43:55.677890+0000","last_became_active":"2023-08-25T09:43:53.390304+0000","last_became_peered":"2023-08-25T09:43:53.390304+0000","last_unstale":"2023-08-25T09:43:55.677890+0000","last_undegraded":"2023-08-25T09:43:55.677890+0000","last_fullsized":"2023-08-25T09:43:55.677890+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:10.567771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676883+0000","last_change":"2023-08-25T09:43:54.416330+0000","last_active":"2023-08-25T09:43:55.676883+0000","last_peered":"2023-08-25T09:43:55.676883+0000","last_clean":"2023-08-25T09:43:55.676883+0000","last_became_active":"2023-08-25T09:43:54.416029+0000","last_became_peered":"2023-08-25T09:43:54.416029+0000","last_unstale":"2023-08-25T09:43:55.676883+0000","last_undegraded":"2023-08-25T09:43:55.676883+0000","last_fullsized":"2023-08-25T09:43:55.676883+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:49.899994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385379+0000","last_change":"2023-08-25T09:43:53.394719+0000","last_active":"2023-08-25T09:43:55.385379+0000","last_peered":"2023-08-25T09:43:55.385379+0000","last_clean":"2023-08-25T09:43:55.385379+0000","last_became_active":"2023-08-25T09:43:53.394231+0000","last_became_peered":"2023-08-25T09:43:53.394231+0000","last_unstale":"2023-08-25T09:43:55.385379+0000","last_undegraded":"2023-08-25T09:43:55.385379+0000","last_fullsized":"2023-08-25T09:43:55.385379+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:53.753394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676795+0000","last_change":"2023-08-25T09:43:54.395059+0000","last_active":"2023-08-25T09:43:55.676795+0000","last_peered":"2023-08-25T09:43:55.676795+0000","last_clean":"2023-08-25T09:43:55.676795+0000","last_became_active":"2023-08-25T09:43:54.394487+0000","last_became_peered":"2023-08-25T09:43:54.394487+0000","last_unstale":"2023-08-25T09:43:55.676795+0000","last_undegraded":"2023-08-25T09:43:55.676795+0000","last_fullsized":"2023-08-25T09:43:55.676795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:12.153733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676095+0000","last_change":"2023-08-25T09:43:53.401366+0000","last_active":"2023-08-25T09:43:55.676095+0000","last_peered":"2023-08-25T09:43:55.676095+0000","last_clean":"2023-08-25T09:43:55.676095+0000","last_became_active":"2023-08-25T09:43:53.400954+0000","last_became_peered":"2023-08-25T09:43:53.400954+0000","last_unstale":"2023-08-25T09:43:55.676095+0000","last_undegraded":"2023-08-25T09:43:55.676095+0000","last_fullsized":"2023-08-25T09:43:55.676095+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:43.297912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389004+0000","last_change":"2023-08-25T09:43:54.408774+0000","last_active":"2023-08-25T09:43:55.389004+0000","last_peered":"2023-08-25T09:43:55.389004+0000","last_clean":"2023-08-25T09:43:55.389004+0000","last_became_active":"2023-08-25T09:43:54.408056+0000","last_became_peered":"2023-08-25T09:43:54.408056+0000","last_unstale":"2023-08-25T09:43:55.389004+0000","last_undegraded":"2023-08-25T09:43:55.389004+0000","last_fullsized":"2023-08-25T09:43:55.389004+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:39.809497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676019+0000","last_change":"2023-08-25T09:43:53.396976+0000","last_active":"2023-08-25T09:43:55.676019+0000","last_peered":"2023-08-25T09:43:55.676019+0000","last_clean":"2023-08-25T09:43:55.676019+0000","last_became_active":"2023-08-25T09:43:53.396705+0000","last_became_peered":"2023-08-25T09:43:53.396705+0000","last_unstale":"2023-08-25T09:43:55.676019+0000","last_undegraded":"2023-08-25T09:43:55.676019+0000","last_fullsized":"2023-08-25T09:43:55.676019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:12.675960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388510+0000","last_change":"2023-08-25T09:43:54.410392+0000","last_active":"2023-08-25T09:43:55.388510+0000","last_peered":"2023-08-25T09:43:55.388510+0000","last_clean":"2023-08-25T09:43:55.388510+0000","last_became_active":"2023-08-25T09:43:54.410129+0000","last_became_peered":"2023-08-25T09:43:54.410129+0000","last_unstale":"2023-08-25T09:43:55.388510+0000","last_undegraded":"2023-08-25T09:43:55.388510+0000","last_fullsized":"2023-08-25T09:43:55.388510+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:35.069148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387143+0000","last_change":"2023-08-25T09:43:53.398900+0000","last_active":"2023-08-25T09:43:55.387143+0000","last_peered":"2023-08-25T09:43:55.387143+0000","last_clean":"2023-08-25T09:43:55.387143+0000","last_became_active":"2023-08-25T09:43:53.398452+0000","last_became_peered":"2023-08-25T09:43:53.398452+0000","last_unstale":"2023-08-25T09:43:55.387143+0000","last_undegraded":"2023-08-25T09:43:55.387143+0000","last_fullsized":"2023-08-25T09:43:55.387143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:23.214744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385717+0000","last_change":"2023-08-25T09:43:54.413745+0000","last_active":"2023-08-25T09:43:55.385717+0000","last_peered":"2023-08-25T09:43:55.385717+0000","last_clean":"2023-08-25T09:43:55.385717+0000","last_became_active":"2023-08-25T09:43:54.413429+0000","last_became_peered":"2023-08-25T09:43:54.413429+0000","last_unstale":"2023-08-25T09:43:55.385717+0000","last_undegraded":"2023-08-25T09:43:55.385717+0000","last_fullsized":"2023-08-25T09:43:55.385717+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:45.596511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676451+0000","last_change":"2023-08-25T09:43:53.395918+0000","last_active":"2023-08-25T09:43:55.676451+0000","last_peered":"2023-08-25T09:43:55.676451+0000","last_clean":"2023-08-25T09:43:55.676451+0000","last_became_active":"2023-08-25T09:43:53.395232+0000","last_became_peered":"2023-08-25T09:43:53.395232+0000","last_unstale":"2023-08-25T09:43:55.676451+0000","last_undegraded":"2023-08-25T09:43:55.676451+0000","last_fullsized":"2023-08-25T09:43:55.676451+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:04:05.916939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":293,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.529317+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:43:55.529317+0000","last_peered":"2023-08-25T09:43:55.529317+0000","last_clean":"2023-08-25T09:43:55.529317+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:43:55.529317+0000","last_undegraded":"2023-08-25T09:43:55.529317+0000","last_fullsized":"2023-08-25T09:43:55.529317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:04.320619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385863+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:43:55.385863+0000","last_peered":"2023-08-25T09:43:55.385863+0000","last_clean":"2023-08-25T09:43:55.385863+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:43:55.385863+0000","last_undegraded":"2023-08-25T09:43:55.385863+0000","last_fullsized":"2023-08-25T09:43:55.385863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:49:22.300510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.390505+0000","last_change":"2023-08-25T09:43:54.418281+0000","last_active":"2023-08-25T09:43:55.390505+0000","last_peered":"2023-08-25T09:43:55.390505+0000","last_clean":"2023-08-25T09:43:55.390505+0000","last_became_active":"2023-08-25T09:43:54.416607+0000","last_became_peered":"2023-08-25T09:43:54.416607+0000","last_unstale":"2023-08-25T09:43:55.390505+0000","last_undegraded":"2023-08-25T09:43:55.390505+0000","last_fullsized":"2023-08-25T09:43:55.390505+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:56.229389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530785+0000","last_change":"2023-08-25T09:43:53.402444+0000","last_active":"2023-08-25T09:43:55.530785+0000","last_peered":"2023-08-25T09:43:55.530785+0000","last_clean":"2023-08-25T09:43:55.530785+0000","last_became_active":"2023-08-25T09:43:53.402171+0000","last_became_peered":"2023-08-25T09:43:53.402171+0000","last_unstale":"2023-08-25T09:43:55.530785+0000","last_undegraded":"2023-08-25T09:43:55.530785+0000","last_fullsized":"2023-08-25T09:43:55.530785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:15.932838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385043+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:43:55.385043+0000","last_peered":"2023-08-25T09:43:55.385043+0000","last_clean":"2023-08-25T09:43:55.385043+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:43:55.385043+0000","last_undegraded":"2023-08-25T09:43:55.385043+0000","last_fullsized":"2023-08-25T09:43:55.385043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:20:22.560940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384463+0000","last_change":"2023-08-25T09:43:54.392619+0000","last_active":"2023-08-25T09:43:55.384463+0000","last_peered":"2023-08-25T09:43:55.384463+0000","last_clean":"2023-08-25T09:43:55.384463+0000","last_became_active":"2023-08-25T09:43:54.392340+0000","last_became_peered":"2023-08-25T09:43:54.392340+0000","last_unstale":"2023-08-25T09:43:55.384463+0000","last_undegraded":"2023-08-25T09:43:55.384463+0000","last_fullsized":"2023-08-25T09:43:55.384463+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:33.252763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388517+0000","last_change":"2023-08-25T09:43:53.403913+0000","last_active":"2023-08-25T09:43:55.388517+0000","last_peered":"2023-08-25T09:43:55.388517+0000","last_clean":"2023-08-25T09:43:55.388517+0000","last_became_active":"2023-08-25T09:43:53.403680+0000","last_became_peered":"2023-08-25T09:43:53.403680+0000","last_unstale":"2023-08-25T09:43:55.388517+0000","last_undegraded":"2023-08-25T09:43:55.388517+0000","last_fullsized":"2023-08-25T09:43:55.388517+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:27:24.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389312+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:43:55.389312+0000","last_peered":"2023-08-25T09:43:55.389312+0000","last_clean":"2023-08-25T09:43:55.389312+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:43:55.389312+0000","last_undegraded":"2023-08-25T09:43:55.389312+0000","last_fullsized":"2023-08-25T09:43:55.389312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:54.055742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388940+0000","last_change":"2023-08-25T09:43:54.412066+0000","last_active":"2023-08-25T09:43:55.388940+0000","last_peered":"2023-08-25T09:43:55.388940+0000","last_clean":"2023-08-25T09:43:55.388940+0000","last_became_active":"2023-08-25T09:43:54.411815+0000","last_became_peered":"2023-08-25T09:43:54.411815+0000","last_unstale":"2023-08-25T09:43:55.388940+0000","last_undegraded":"2023-08-25T09:43:55.388940+0000","last_fullsized":"2023-08-25T09:43:55.388940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:25.923883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389578+0000","last_change":"2023-08-25T09:43:53.403077+0000","last_active":"2023-08-25T09:43:55.389578+0000","last_peered":"2023-08-25T09:43:55.389578+0000","last_clean":"2023-08-25T09:43:55.389578+0000","last_became_active":"2023-08-25T09:43:53.402353+0000","last_became_peered":"2023-08-25T09:43:53.402353+0000","last_unstale":"2023-08-25T09:43:55.389578+0000","last_undegraded":"2023-08-25T09:43:55.389578+0000","last_fullsized":"2023-08-25T09:43:55.389578+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:45.178645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385518+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:43:55.385518+0000","last_peered":"2023-08-25T09:43:55.385518+0000","last_clean":"2023-08-25T09:43:55.385518+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:43:55.385518+0000","last_undegraded":"2023-08-25T09:43:55.385518+0000","last_fullsized":"2023-08-25T09:43:55.385518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:22:06.254532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530488+0000","last_change":"2023-08-25T09:43:54.412739+0000","last_active":"2023-08-25T09:43:55.530488+0000","last_peered":"2023-08-25T09:43:55.530488+0000","last_clean":"2023-08-25T09:43:55.530488+0000","last_became_active":"2023-08-25T09:43:54.412482+0000","last_became_peered":"2023-08-25T09:43:54.412482+0000","last_unstale":"2023-08-25T09:43:55.530488+0000","last_undegraded":"2023-08-25T09:43:55.530488+0000","last_fullsized":"2023-08-25T09:43:55.530488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:23.371240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389665+0000","last_change":"2023-08-25T09:43:53.388577+0000","last_active":"2023-08-25T09:43:55.389665+0000","last_peered":"2023-08-25T09:43:55.389665+0000","last_clean":"2023-08-25T09:43:55.389665+0000","last_became_active":"2023-08-25T09:43:53.387937+0000","last_became_peered":"2023-08-25T09:43:53.387937+0000","last_unstale":"2023-08-25T09:43:55.389665+0000","last_undegraded":"2023-08-25T09:43:55.389665+0000","last_fullsized":"2023-08-25T09:43:55.389665+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:12.082324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":141,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.676632+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:43:55.676632+0000","last_peered":"2023-08-25T09:43:55.676632+0000","last_clean":"2023-08-25T09:43:55.676632+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:43:55.676632+0000","last_undegraded":"2023-08-25T09:43:55.676632+0000","last_fullsized":"2023-08-25T09:43:55.676632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:56.103124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.388697+0000","last_change":"2023-08-25T09:43:54.416364+0000","last_active":"2023-08-25T09:43:55.388697+0000","last_peered":"2023-08-25T09:43:55.388697+0000","last_clean":"2023-08-25T09:43:55.388697+0000","last_became_active":"2023-08-25T09:43:54.415180+0000","last_became_peered":"2023-08-25T09:43:54.415180+0000","last_unstale":"2023-08-25T09:43:55.388697+0000","last_undegraded":"2023-08-25T09:43:55.388697+0000","last_fullsized":"2023-08-25T09:43:55.388697+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:18.233987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389600+0000","last_change":"2023-08-25T09:43:53.403927+0000","last_active":"2023-08-25T09:43:55.389600+0000","last_peered":"2023-08-25T09:43:55.389600+0000","last_clean":"2023-08-25T09:43:55.389600+0000","last_became_active":"2023-08-25T09:43:53.402901+0000","last_became_peered":"2023-08-25T09:43:53.402901+0000","last_unstale":"2023-08-25T09:43:55.389600+0000","last_undegraded":"2023-08-25T09:43:55.389600+0000","last_fullsized":"2023-08-25T09:43:55.389600+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:47.804156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.384922+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:43:55.384922+0000","last_peered":"2023-08-25T09:43:55.384922+0000","last_clean":"2023-08-25T09:43:55.384922+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:43:55.384922+0000","last_undegraded":"2023-08-25T09:43:55.384922+0000","last_fullsized":"2023-08-25T09:43:55.384922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:08:27.873205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.389307+0000","last_change":"2023-08-25T09:43:54.418893+0000","last_active":"2023-08-25T09:43:55.389307+0000","last_peered":"2023-08-25T09:43:55.389307+0000","last_clean":"2023-08-25T09:43:55.389307+0000","last_became_active":"2023-08-25T09:43:54.418633+0000","last_became_peered":"2023-08-25T09:43:54.418633+0000","last_unstale":"2023-08-25T09:43:55.389307+0000","last_undegraded":"2023-08-25T09:43:55.389307+0000","last_fullsized":"2023-08-25T09:43:55.389307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:46:42.226958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"93'1","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.678185+0000","last_change":"2023-08-25T09:43:53.401971+0000","last_active":"2023-08-25T09:43:55.678185+0000","last_peered":"2023-08-25T09:43:55.678185+0000","last_clean":"2023-08-25T09:43:55.678185+0000","last_became_active":"2023-08-25T09:43:53.401743+0000","last_became_peered":"2023-08-25T09:43:53.401743+0000","last_unstale":"2023-08-25T09:43:55.678185+0000","last_undegraded":"2023-08-25T09:43:55.678185+0000","last_fullsized":"2023-08-25T09:43:55.678185+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:56.603768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.530611+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:43:55.530611+0000","last_peered":"2023-08-25T09:43:55.530611+0000","last_clean":"2023-08-25T09:43:55.530611+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:43:55.530611+0000","last_undegraded":"2023-08-25T09:43:55.530611+0000","last_fullsized":"2023-08-25T09:43:55.530611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:08:01.681521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.387661+0000","last_change":"2023-08-25T09:43:54.399751+0000","last_active":"2023-08-25T09:43:55.387661+0000","last_peered":"2023-08-25T09:43:55.387661+0000","last_clean":"2023-08-25T09:43:55.387661+0000","last_became_active":"2023-08-25T09:43:54.399246+0000","last_became_peered":"2023-08-25T09:43:54.399246+0000","last_unstale":"2023-08-25T09:43:55.387661+0000","last_undegraded":"2023-08-25T09:43:55.387661+0000","last_fullsized":"2023-08-25T09:43:55.387661+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:53.359358+0000","last_clean_scrub_stamp":"2023-08-25T09:43:53.359358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:56:24.732531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"93'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.677545+0000","last_change":"2023-08-25T09:43:53.400796+0000","last_active":"2023-08-25T09:43:55.677545+0000","last_peered":"2023-08-25T09:43:55.677545+0000","last_clean":"2023-08-25T09:43:55.677545+0000","last_became_active":"2023-08-25T09:43:53.400551+0000","last_became_peered":"2023-08-25T09:43:53.400551+0000","last_unstale":"2023-08-25T09:43:55.677545+0000","last_undegraded":"2023-08-25T09:43:55.677545+0000","last_fullsized":"2023-08-25T09:43:55.677545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:43:52.348985+0000","last_clean_scrub_stamp":"2023-08-25T09:43:52.348985+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:51:58.400833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-08-25T09:43:55.385228+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:43:55.385228+0000","last_peered":"2023-08-25T09:43:55.385228+0000","last_clean":"2023-08-25T09:43:55.385228+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:43:55.385228+0000","last_undegraded":"2023-08-25T09:43:55.385228+0000","last_fullsized":"2023-08-25T09:43:55.385228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:32:47.623306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194787,"statfs":{"total":200000000,"available":199462687,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1910000000000001,"15min":1.0660000000000001},"min":{"1min":0.64900000000000002,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.54,"5min":10.747,"15min":10.747},"last":0.752},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.0800000000000001,"15min":1.026},"min":{"1min":0.86899999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0409999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.1859999999999999,"15min":1.129},"min":{"1min":0.88,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8500000000000001,"5min":1.8500000000000001,"15min":1.8500000000000001},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1799999999999999,"15min":1.0920000000000001},"min":{"1min":0.76500000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5820000000000001,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1559999999999999,"15min":1.079},"min":{"1min":0.75600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.27,"15min":1.2170000000000001},"min":{"1min":1.0620000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":1.708}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2050000000000001,"15min":1.155},"min":{"1min":0.81000000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.615,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.3839999999999999,"15min":1.329},"min":{"1min":1.0629999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9219999999999999,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.639}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.3120000000000001,"15min":1.27},"min":{"1min":0.77400000000000002,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7689999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.319,"15min":1.286},"min":{"1min":1.05,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.7609999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.3899999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4259999999999999,"15min":1.3360000000000001},"min":{"1min":0.81999999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.028,"5min":2.028,"15min":2.028},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.429,"15min":1.3740000000000001},"min":{"1min":1.04,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9830000000000001,"5min":2.0739999999999998,"15min":2.0739999999999998},"last":1.462}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.5069999999999999,"15min":1.4319999999999999},"min":{"1min":1.1439999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.8740000000000001,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.579},{"interface":"front","average":{"1min":1.5489999999999999,"5min":1.472,"15min":1.3979999999999999},"min":{"1min":1.276,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":2.0710000000000002,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.6160000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194787,"statfs":{"total":200000000,"available":199462667,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.1399999999999999,"15min":1.1499999999999999},"min":{"1min":0.69899999999999995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":1.224},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.1910000000000001,"15min":1.145},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.7649999999999999,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":0.752}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.2290000000000001,"15min":1.1950000000000001},"min":{"1min":0.79300000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.7050000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.1830000000000001,"15min":1.117},"min":{"1min":0.94799999999999995,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.337}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.1659999999999999,"15min":1.105},"min":{"1min":0.85599999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.482,"5min":1.708,"15min":1.708},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.409,"5min":1.3620000000000001,"15min":1.204},"min":{"1min":0.93400000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.768,"5min":13.193,"15min":13.193},"last":1.4159999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.302,"15min":1.222},"min":{"1min":0.79300000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.121,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.272,"15min":1.256},"min":{"1min":0.98099999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.548,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":0.99299999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.304,"15min":1.24},"min":{"1min":0.95999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.607,"5min":2.484,"15min":2.484},"last":1.528},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.385,"15min":1.3169999999999999},"min":{"1min":0.89200000000000002,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.02,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.177}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3129999999999999,"15min":1.2749999999999999},"min":{"1min":1.034,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.837,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.272},{"interface":"front","average":{"1min":1.532,"5min":1.454,"15min":1.3640000000000001},"min":{"1min":1.1990000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7210000000000001,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.4750000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.538,"15min":1.49},"min":{"1min":0.89600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.2240000000000002,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.472},"min":{"1min":1.244,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.3879999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1100,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194211,"statfs":{"total":200000000,"available":198872881,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.1930000000000001,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.577,"5min":2.081,"15min":2.081},"last":1.738},{"interface":"front","average":{"1min":1.27,"5min":1.22,"15min":1.208},"min":{"1min":0.76200000000000001,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.5469999999999999,"5min":1.974,"15min":1.974},"last":1.585}]},{"osd":1,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.145,"15min":1.1950000000000001},"min":{"1min":0.746,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.577,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.24},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1819999999999999,"15min":1.119},"min":{"1min":0.997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5649999999999999,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.8640000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.238,"15min":1.1859999999999999},"min":{"1min":0.94899999999999995,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.665,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.012},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":1.048,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6259999999999999,"5min":1.962,"15min":1.962},"last":1.778}]},{"osd":3,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.4059999999999999,"15min":1.3160000000000001},"min":{"1min":1.0549999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.3439999999999999,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.177},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3069999999999999,"15min":1.2789999999999999},"min":{"1min":1.024,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.552,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.367}]},{"osd":4,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.355,"15min":1.3029999999999999},"min":{"1min":0.96199999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7490000000000001,"5min":2.625,"15min":2.625},"last":1.9530000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.3480000000000001,"15min":1.3720000000000001},"min":{"1min":0.94499999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.857,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.397}]},{"osd":6,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.4419999999999999,"15min":1.4330000000000001},"min":{"1min":1.1399999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.77,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":1.512,"15min":1.417},"min":{"1min":1.177,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1059999999999999,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.478}]},{"osd":7,"last update":"Fri Aug 25 09:43:46 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.4770000000000001,"15min":1.4450000000000001},"min":{"1min":1.2869999999999999,"5min":0.63,"15min":0.63},"max":{"1min":2.2149999999999999,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.71,"5min":1.587,"15min":1.579},"min":{"1min":1.4319999999999999,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.0790000000000002,"5min":2.3700000000000001,"15min":2.3700000000000001},"last":1.7050000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1759999999999999,"15min":1.1499999999999999},"min":{"1min":0.70499999999999996,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.5329999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2609999999999999,"15min":1.1459999999999999},"min":{"1min":1.026,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5289999999999999,"5min":7.367,"15min":7.367},"last":1.1719999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.1359999999999999,"15min":1.1220000000000001},"min":{"1min":0.76900000000000002,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5800000000000001,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.996},{"interface":"front","average":{"1min":1.173,"5min":1.244,"15min":1.401},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.5249999999999999,"5min":8.5850000000000009,"15min":8.5850000000000009},"last":1.4650000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.208,"15min":1.302},"min":{"1min":0.93200000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":4.1669999999999998,"15min":4.1669999999999998},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.244,"15min":1.167},"min":{"1min":0.95899999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.631,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.121}]},{"osd":3,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.292,"15min":1.23},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.7110000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.486,"5min":1.3640000000000001,"15min":1.371},"min":{"1min":1.1439999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.778,"5min":1.911,"15min":1.911},"last":1.2090000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.288,"15min":1.292},"min":{"1min":0.86599999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.276},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.3280000000000001,"15min":1.248},"min":{"1min":0.91700000000000004,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.829,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.0649999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.423,"15min":1.4350000000000001},"min":{"1min":0.81100000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.0219999999999998,"5min":2.863,"15min":2.863},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.548,"15min":1.5600000000000001},"min":{"1min":0.97799999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9730000000000001,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":1.5429999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:43:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5800000000000001,"5min":1.55,"15min":1.4179999999999999},"min":{"1min":1.1910000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.056,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.355},{"interface":"front","average":{"1min":1.6890000000000001,"5min":1.593,"15min":1.595},"min":{"1min":1.3320000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1230000000000002,"5min":2.7930000000000001,"15min":2.7930000000000001},"last":1.587}]}]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":525,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194786,"statfs":{"total":200000000,"available":199461623,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.86699999999999999,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.302,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.256},{"interface":"front","average":{"1min":1.0980000000000001,"5min":0.95399999999999996,"15min":0.89800000000000002},"min":{"1min":0.65900000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.5349999999999999,"5min":1.55,"15min":1.55},"last":0.88900000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.871,"15min":0.79800000000000004},"min":{"1min":0.54200000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.153,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.004,"15min":0.97699999999999998},"min":{"1min":0.58699999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.4910000000000001,"5min":2.3399999999999999,"15min":2.3399999999999999},"last":1.139}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":0.96199999999999997,"15min":0.87},"min":{"1min":0.625,"5min":0.378,"15min":0.378},"max":{"1min":1.573,"5min":1.919,"15min":1.919},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1020000000000001,"15min":1.1080000000000001},"min":{"1min":0.81000000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6140000000000001,"5min":3.399,"15min":3.399},"last":1.345}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.214,"15min":1.1890000000000001},"min":{"1min":0.83499999999999996,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5800000000000001,"5min":2.1949999999999998,"15min":2.1949999999999998},"last":1.105},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.224,"15min":1.196},"min":{"1min":0.89800000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.405,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.323,"15min":1.3120000000000001},"min":{"1min":0.98899999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6699999999999999,"5min":2.4060000000000001,"15min":2.4060000000000001},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.2849999999999999,"15min":1.2689999999999999},"min":{"1min":0.92000000000000004,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5980000000000001,"5min":1.788,"15min":1.788},"last":1.3069999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.282,"15min":1.242},"min":{"1min":1.0389999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.75,"5min":1.917,"15min":1.917},"last":1.048},{"interface":"front","average":{"1min":1.45,"5min":1.397,"15min":1.405},"min":{"1min":1.2230000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.734,"5min":3.496,"15min":3.496},"last":1.448}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.399,"15min":1.3640000000000001},"min":{"1min":1.085,"5min":0.871,"15min":0.871},"max":{"1min":1.881,"5min":2.2989999999999999,"15min":2.2989999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.387,"15min":1.2909999999999999},"min":{"1min":0.98899999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.669,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.1890000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463339,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83399999999999996,"15min":0.82499999999999996},"min":{"1min":0.53300000000000003,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.4199999999999999,"5min":2.383,"15min":2.383},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.94699999999999995,"15min":1.0669999999999999},"min":{"1min":0.64800000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.425,"5min":2.7309999999999999,"15min":2.7309999999999999},"last":0.86499999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":0.97699999999999998,"15min":0.98899999999999999},"min":{"1min":0.60399999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.411,"5min":2.3180000000000001,"15min":2.3180000000000001},"last":1.238},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.0429999999999999,"15min":1.0589999999999999},"min":{"1min":0.53100000000000003,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.5649999999999999,"5min":2.8599999999999999,"15min":2.8599999999999999},"last":1.284}]},{"osd":3,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.97199999999999998,"15min":1.0920000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5429999999999999,"5min":4.9569999999999999,"15min":4.9569999999999999},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.081,"15min":1.101},"min":{"1min":0.73599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.486,"5min":2.2040000000000002,"15min":2.2040000000000002},"last":1.3220000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.3080000000000001,"15min":1.397},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.1309999999999998,"5min":3.0150000000000001,"15min":3.0150000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.81299999999999994,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.6919999999999999,"5min":2.7639999999999998,"15min":2.7639999999999998},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.266,"15min":1.375},"min":{"1min":0.83799999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.369,"15min":1.397},"min":{"1min":1.081,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.9890000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.444}]},{"osd":6,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.381,"15min":1.4199999999999999},"min":{"1min":0.755,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.875,"5min":2.6709999999999998,"15min":2.6709999999999998},"last":1.425},{"interface":"front","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.482},"min":{"1min":0.995,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.097,"5min":5.0599999999999996,"15min":5.0599999999999996},"last":1.7070000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.383,"15min":1.4490000000000001},"min":{"1min":0.89600000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7829999999999999,"5min":2.968,"15min":2.968},"last":1.478},{"interface":"front","average":{"1min":1.409,"5min":1.4199999999999999,"15min":1.504},"min":{"1min":1.0309999999999999,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.056,"5min":5.2990000000000004,"15min":5.2990000000000004},"last":1.8200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1102,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194210,"statfs":{"total":200000000,"available":198871325,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.81699999999999995,"15min":0.77200000000000002},"min":{"1min":0.45900000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2250000000000001,"5min":1.627,"15min":1.627},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.876,"15min":0.78400000000000003},"min":{"1min":0.56799999999999995,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.6850000000000001,"5min":1.895,"15min":1.895},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.86499999999999999,"15min":0.79900000000000004},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.45,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.048,"5min":0.97499999999999998,"15min":0.90300000000000002},"min":{"1min":0.51500000000000001,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.4239999999999999,"5min":1.746,"15min":1.746},"last":0.83599999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.1339999999999999,"15min":1.038},"min":{"1min":0.61199999999999999,"5min":0.372,"15min":0.372},"max":{"1min":1.639,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.008,"15min":0.93999999999999995},"min":{"1min":0.67900000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.4750000000000001,"5min":1.764,"15min":1.764},"last":1.034}]},{"osd":4,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.155,"15min":1.087},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.117},{"interface":"front","average":{"1min":1.369,"5min":1.3100000000000001,"15min":1.218},"min":{"1min":0.876,"5min":0.629,"15min":0.629},"max":{"1min":1.641,"5min":2.008,"15min":2.008},"last":1.2809999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2410000000000001,"15min":1.1890000000000001},"min":{"1min":0.80000000000000004,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7909999999999999,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2170000000000001,"15min":1.1679999999999999},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.738,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.401}]},{"osd":6,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.3799999999999999,"15min":1.298},"min":{"1min":1.042,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.9930000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.3460000000000001,"15min":1.29},"min":{"1min":0.82199999999999995,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8380000000000001,"5min":2.004,"15min":2.004},"last":1.341}]},{"osd":7,"last update":"Fri Aug 25 09:43:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.423,"15min":1.3520000000000001},"min":{"1min":1.145,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.7030000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.3899999999999999,"15min":1.3080000000000001},"min":{"1min":0.93500000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9039999999999999,"5min":1.9299999999999999,"15min":1.9299999999999999},"last":1.2969999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":524,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194788,"statfs":{"total":200000000,"available":199463317,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.90700000000000003,"15min":0.85299999999999998},"min":{"1min":0.33900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":1.355,"5min":11.257,"15min":11.257},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.97099999999999997,"15min":0.95799999999999996},"min":{"1min":0.34799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.7110000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.88}]},{"osd":2,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.38800000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0029999999999999,"15min":1.0940000000000001},"min":{"1min":0.47399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.5640000000000001,"5min":3.4889999999999999,"15min":3.4889999999999999},"last":1.085}]},{"osd":3,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.88400000000000001,"15min":0.90000000000000002},"min":{"1min":0.45800000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.534,"5min":1.534,"15min":1.534},"last":1.002},{"interface":"front","average":{"1min":1.012,"5min":1.0629999999999999,"15min":1.1140000000000001},"min":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.5609999999999999,"5min":1.869,"15min":1.869},"last":1.3700000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2230000000000001,"15min":1.2330000000000001},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.327},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.3340000000000001,"15min":1.353},"min":{"1min":0.83299999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.7390000000000001,"5min":1.982,"15min":1.982},"last":1.4910000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.256,"15min":1.3580000000000001},"min":{"1min":0.71099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.8340000000000001,"5min":3.3769999999999998,"15min":3.3769999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.81100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6659999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.246}]},{"osd":6,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2909999999999999,"15min":1.339},"min":{"1min":0.73699999999999999,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.6579999999999999,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.286,"15min":1.296},"min":{"1min":0.752,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.5760000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:43:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4339999999999999,"5min":1.3660000000000001,"15min":1.397},"min":{"1min":0.79000000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.0779999999999998,"5min":2.121,"15min":2.121},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3,"15min":1.319},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7969999999999999,"5min":1.8009999999999999,"15min":1.8009999999999999},"last":1.1870000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:44:04.297 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:44:04.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:44:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.554+0000 7fbf16c26640 1 Processor -- start 2023-08-25T09:44:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.554+0000 7fbf16c26640 1 -- start start 2023-08-25T09:44:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf16c26640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf1015ae50 0x7fbf100c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf16c26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 0x7fbf10069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf16c26640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf100cc5f0 con 0x7fbf100ce060 2023-08-25T09:44:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf16c26640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf10070650 con 0x7fbf1015ae50 2023-08-25T09:44:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf16c26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10070930 con 0x7fbf100cdc20 2023-08-25T09:44:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf149c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf1015ae50 0x7fbf100c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf149c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf1015ae50 0x7fbf100c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45356/0 (socket says 172.21.15.70:45356) 2023-08-25T09:44:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.556+0000 7fbf149c2640 1 -- 172.21.15.70:0/1316966542 learned_addr learned my addr 172.21.15.70:0/1316966542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.557+0000 7fbf149c2640 1 -- 172.21.15.70:0/1316966542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 msgr2=0x7fbf10069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.557+0000 7fbf149c2640 1 --2- 172.21.15.70:0/1316966542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 0x7fbf10069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.557+0000 7fbf149c2640 1 -- 172.21.15.70:0/1316966542 >> v1:172.21.15.155:6789/0 conn(0x7fbf100ce060 legacy=0x7fbf100ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.557+0000 7fbf149c2640 1 -- 172.21.15.70:0/1316966542 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10070c10 con 0x7fbf1015ae50 2023-08-25T09:44:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.558+0000 7fbf149c2640 1 --2- 172.21.15.70:0/1316966542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf1015ae50 0x7fbf100c2110 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009e50 tx=0x7fbf0002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ed579b56704bf4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.558+0000 7fbf0f7fe640 1 -- 172.21.15.70:0/1316966542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0002cb00 con 0x7fbf1015ae50 2023-08-25T09:44:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.558+0000 7fbf0f7fe640 1 -- 172.21.15.70:0/1316966542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0002cc60 con 0x7fbf1015ae50 2023-08-25T09:44:04.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.558+0000 7fbf0f7fe640 1 -- 172.21.15.70:0/1316966542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000343f0 con 0x7fbf1015ae50 2023-08-25T09:44:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 -- 172.21.15.70:0/1316966542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf1015ae50 msgr2=0x7fbf100c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1316966542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf1015ae50 0x7fbf100c2110 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009e50 tx=0x7fbf0002bf60 comp rx=0 tx=0).stop 2023-08-25T09:44:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 -- 172.21.15.70:0/1316966542 shutdown_connections 2023-08-25T09:44:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1316966542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 0x7fbf10069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1316966542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf1015ae50 0x7fbf100c2110 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 -- 172.21.15.70:0/1316966542 >> 172.21.15.70:0/1316966542 conn(0x7fbf100cb0d0 msgr2=0x7fbf100cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:04.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 -- 172.21.15.70:0/1316966542 shutdown_connections 2023-08-25T09:44:04.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.559+0000 7fbf16c26640 1 -- 172.21.15.70:0/1316966542 wait complete. 2023-08-25T09:44:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.561+0000 7fbf16c26640 1 Processor -- start 2023-08-25T09:44:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.561+0000 7fbf16c26640 1 -- start start 2023-08-25T09:44:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf16c26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 0x7fbf100c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf16c26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 0x7fbf100c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf16c26640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf1015ae50 0x7fbf101424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf16c26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c8a10 con 0x7fbf100ce060 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf16c26640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100c5f80 con 0x7fbf1015ae50 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf16c26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1015ceb0 con 0x7fbf100cdc20 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 0x7fbf100c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 0x7fbf100c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45360/0 (socket says 172.21.15.70:45360) 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 -- 172.21.15.70:0/1745539768 learned_addr learned my addr 172.21.15.70:0/1745539768 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 -- 172.21.15.70:0/1745539768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 msgr2=0x7fbf100c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 0x7fbf100c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 -- 172.21.15.70:0/1745539768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf1015ae50 msgr2=0x7fbf101424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf1015ae50 0x7fbf101424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 -- 172.21.15.70:0/1745539768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00009b30 con 0x7fbf100ce060 2023-08-25T09:44:04.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.562+0000 7fbf0ffff640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 0x7fbf100c84d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b410 tx=0x7fbf0400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.563+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011830 con 0x7fbf100ce060 2023-08-25T09:44:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.563+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf040119b0 con 0x7fbf100ce060 2023-08-25T09:44:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.563+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf04011d60 con 0x7fbf100ce060 2023-08-25T09:44:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.563+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1015d190 con 0x7fbf100ce060 2023-08-25T09:44:04.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.563+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1015d750 con 0x7fbf100ce060 2023-08-25T09:44:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.564+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf10066fb0 con 0x7fbf100ce060 2023-08-25T09:44:04.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.565+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbf0401a400 con 0x7fbf100ce060 2023-08-25T09:44:04.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.566+0000 7fbf0d7fa640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbefc06a050 0x7fbefc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:04.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.566+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbf04088790 con 0x7fbf100ce060 2023-08-25T09:44:04.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.566+0000 7fbf149c2640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbefc06a050 0x7fbefc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:04.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.567+0000 7fbf149c2640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbefc06a050 0x7fbefc06c510 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbf00035000 tx=0x7fbf0002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:04.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.570+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf04053de0 con 0x7fbf100ce060 2023-08-25T09:44:04.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.811+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbf100c4dd0 con 0x7fbf100ce060 2023-08-25T09:44:04.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.820+0000 7fbf0d7fa640 1 -- 172.21.15.70:0/1745539768 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbf04057400 con 0x7fbf100ce060 2023-08-25T09:44:04.823 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:44:04.823 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:44:04.823 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:44:04.823 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:44:04.824 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:44:04.825 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:44:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbefc06a050 msgr2=0x7fbefc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbefc06a050 0x7fbefc06c510 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbf00035000 tx=0x7fbf0002e000 comp rx=0 tx=0).stop 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 msgr2=0x7fbf100c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 0x7fbf100c84d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400b410 tx=0x7fbf0400b8d0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 shutdown_connections 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbefc06a050 0x7fbefc06c510 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf1015ae50 0x7fbf101424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf100ce060 0x7fbf100c84d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 --2- 172.21.15.70:0/1745539768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf100cdc20 0x7fbf100c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 >> 172.21.15.70:0/1745539768 conn(0x7fbf100cb0d0 msgr2=0x7fbf100c3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 shutdown_connections 2023-08-25T09:44:04.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:04.823+0000 7fbf16c26640 1 -- 172.21.15.70:0/1745539768 wait complete. 2023-08-25T09:44:04.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:44:05.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.040+0000 7f3485bc9640 1 Processor -- start 2023-08-25T09:44:05.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.040+0000 7f3485bc9640 1 -- start start 2023-08-25T09:44:05.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f3485bc9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 0x7f3480164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f3485bc9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348015d3f0 con 0x7f34800ce5b0 2023-08-25T09:44:05.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f3485bc9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3480070a00 con 0x7f348015aa10 2023-08-25T09:44:05.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f3485bc9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3480159d10 con 0x7f34800b2850 2023-08-25T09:44:05.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f347f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f34800ce5b0 0x7f34800ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39164/0 (socket says 172.21.15.70:39164) 2023-08-25T09:44:05.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f347f7fe640 1 -- 172.21.15.70:0/3985575573 learned_addr learned my addr 172.21.15.70:0/3985575573 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:05.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f347ffff640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 0x7f3480164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55371094 0 0) 0x7f348015d3f0 con 0x7f34800ce5b0 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.042+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003520 con 0x7f34800ce5b0 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1871676512 0 0) 0x7f346c003520 con 0x7f34800ce5b0 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 msgr2=0x7f3480164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 0x7f3480164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 >> v1:172.21.15.155:6789/0 conn(0x7f348015aa10 legacy=0x7f34800b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800b2d20 con 0x7f34800ce5b0 2023-08-25T09:44:05.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f34700019f0 con 0x7f34800ce5b0 2023-08-25T09:44:05.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 >> v1:172.21.15.70:6789/0 conn(0x7f34800ce5b0 legacy=0x7f34800ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f346c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 0x7f346c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c004d00 0x7f346c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480159d10 con 0x7f346c0039a0 2023-08-25T09:44:05.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00bb60 con 0x7f346c004d00 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00bd20 con 0x7f34800b2850 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.043+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f346c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347ffff640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c004d00 0x7f346c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347f7fe640 1 -- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 msgr2=0x7f346c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34800b2850 0x7f346c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347f7fe640 1 -- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c004d00 msgr2=0x7f346c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c004d00 0x7f346c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347f7fe640 1 -- 172.21.15.70:0/3985575573 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c003520 con 0x7f346c0039a0 2023-08-25T09:44:05.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.044+0000 7f347ffff640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f346c004d00 0x7f346c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:05.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f346c003dc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f347000a830 tx=0x7f346c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6d6b4313a438 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700072d0 con 0x7f346c0039a0 2023-08-25T09:44:05.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347002f070 con 0x7f346c0039a0 2023-08-25T09:44:05.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f347effd640 1 -- 172.21.15.70:0/3985575573 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700076b0 con 0x7f346c0039a0 2023-08-25T09:44:05.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f3485bc9640 1 -- 172.21.15.70:0/3985575573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 msgr2=0x7f346c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3985575573 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f346c003dc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f347000a830 tx=0x7f346c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.045+0000 7f347ffff640 1 -- 172.21.15.70:0/3985575573 reap_dead start 2023-08-25T09:44:05.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.046+0000 7f3485bc9640 1 -- 172.21.15.70:0/3985575573 shutdown_connections 2023-08-25T09:44:05.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.046+0000 7f3485bc9640 1 -- 172.21.15.70:0/3985575573 >> 172.21.15.70:0/3985575573 conn(0x7f3480069850 msgr2=0x7f3480069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.046+0000 7f3485bc9640 1 -- 172.21.15.70:0/3985575573 shutdown_connections 2023-08-25T09:44:05.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.046+0000 7f3485bc9640 1 -- 172.21.15.70:0/3985575573 wait complete. 2023-08-25T09:44:05.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.047+0000 7f3485bc9640 1 Processor -- start 2023-08-25T09:44:05.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.048+0000 7f3485bc9640 1 -- start start 2023-08-25T09:44:05.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.048+0000 7f3485bc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f34800cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.048+0000 7f3485bc9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480152b90 0x7f34801f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.048+0000 7f3485bc9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34801f64a0 0x7f34801f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f3485bc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480153540 con 0x7f346c0039a0 2023-08-25T09:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f3485bc9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480153da0 con 0x7f3480152b90 2023-08-25T09:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f3485bc9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3480153090 con 0x7f34801f64a0 2023-08-25T09:44:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f34800cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f34800cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45384/0 (socket says 172.21.15.70:45384) 2023-08-25T09:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 -- 172.21.15.70:0/3656097803 learned_addr learned my addr 172.21.15.70:0/3656097803 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 -- 172.21.15.70:0/3656097803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34801f64a0 msgr2=0x7f34801f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34801f64a0 0x7f34801f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 -- 172.21.15.70:0/3656097803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480152b90 msgr2=0x7f34801f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480152b90 0x7f34801f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 -- 172.21.15.70:0/3656097803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480153750 con 0x7f346c0039a0 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.049+0000 7f347f7fe640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f34800cdaa0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f347000a830 tx=0x7f3470001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.050+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470044070 con 0x7f346c0039a0 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.050+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470004d70 con 0x7f346c0039a0 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.050+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3480153a30 con 0x7f346c0039a0 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.050+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34700070d0 con 0x7f346c0039a0 2023-08-25T09:44:05.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.050+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34800ce1c0 con 0x7f346c0039a0 2023-08-25T09:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.051+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348006cb30 con 0x7f346c0039a0 2023-08-25T09:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.052+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3470031030 con 0x7f346c0039a0 2023-08-25T09:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.053+0000 7f347cff9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.053+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34700a9cd0 con 0x7f346c0039a0 2023-08-25T09:44:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.053+0000 7f3476ffd640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.054+0000 7f3476ffd640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f34800b2300 tx=0x7f3468007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.057+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3470075450 con 0x7f346c0039a0 2023-08-25T09:44:05.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.265+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34800b35a0 con 0x7f346c0039a0 2023-08-25T09:44:05.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.266+0000 7f347cff9640 1 -- 172.21.15.70:0/3656097803 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7f3470078a70 con 0x7f346c0039a0 2023-08-25T09:44:05.270 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:05.270 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:44:05.272 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:44:05.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 msgr2=0x7f346006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f34800b2300 tx=0x7f3468007990 comp rx=0 tx=0).stop 2023-08-25T09:44:05.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 msgr2=0x7f34800cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f34800cdaa0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f347000a830 tx=0x7f3470001d80 comp rx=0 tx=0).stop 2023-08-25T09:44:05.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 shutdown_connections 2023-08-25T09:44:05.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f346006a050 0x7f346006c510 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f34801f64a0 0x7f34801f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3480152b90 0x7f34801f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 --2- 172.21.15.70:0/3656097803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f346c0039a0 0x7f34800cdaa0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.268+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 >> 172.21.15.70:0/3656097803 conn(0x7f3480069850 msgr2=0x7f3480069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.269+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 shutdown_connections 2023-08-25T09:44:05.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.269+0000 7f3485bc9640 1 -- 172.21.15.70:0/3656097803 wait complete. 2023-08-25T09:44:05.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:43:53.868790+0000', 'modified': '2023-08-25T09:43:57.423437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6780}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6780': {'gid': 6780, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6837/1195200165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6836', 'nonce': 1195200165}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 1195200165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:44:05.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:44:05.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-25T09:44:05.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.910+0000 7fdc65b86640 1 Processor -- start 2023-08-25T09:44:05.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.910+0000 7fdc65b86640 1 -- start start 2023-08-25T09:44:05.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.912+0000 7fdc65b86640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdc600b4400 0x7fdc600cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.912+0000 7fdc65b86640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600b12b0 con 0x7fdc600ca780 2023-08-25T09:44:05.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.912+0000 7fdc65b86640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6006ae20 con 0x7fdc6015acd0 2023-08-25T09:44:05.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.912+0000 7fdc65b86640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc600b47e0 con 0x7fdc600b4400 2023-08-25T09:44:05.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.912+0000 7fdc56ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdc600ca780 0x7fdc600b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39166/0 (socket says 172.21.15.70:39166) 2023-08-25T09:44:05.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.912+0000 7fdc56ffd640 1 -- 172.21.15.70:0/1642720341 learned_addr learned my addr 172.21.15.70:0/1642720341 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:05.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061261335 0 0) 0x7fdc600b12b0 con 0x7fdc600ca780 2023-08-25T09:44:05.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc4c003520 con 0x7fdc600ca780 2023-08-25T09:44:05.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/1642720341 >> v2:172.21.15.155:3300/0 conn(0x7fdc600b4400 0x7fdc600cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4012531614 0 0) 0x7fdc4c003520 con 0x7fdc600ca780 2023-08-25T09:44:05.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 >> v2:172.21.15.155:3300/0 conn(0x7fdc600b4400 msgr2=0x7fdc600cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 --2- 172.21.15.70:0/1642720341 >> v2:172.21.15.155:3300/0 conn(0x7fdc600b4400 0x7fdc600cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 >> v1:172.21.15.155:6789/0 conn(0x7fdc6015acd0 legacy=0x7fdc6015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6015b7c0 con 0x7fdc600ca780 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fdc48001670 con 0x7fdc600ca780 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 >> v1:172.21.15.70:6789/0 conn(0x7fdc600ca780 legacy=0x7fdc600b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc4c003970 0x7fdc4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc4c0047c0 0x7fdc4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0052a0 0x7fdc4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600b47e0 con 0x7fdc4c0047c0 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c00c170 con 0x7fdc4c003970 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.913+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c00c330 con 0x7fdc4c0052a0 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc4c0047c0 0x7fdc4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc4c003970 0x7fdc4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0052a0 0x7fdc4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc56ffd640 1 -- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0052a0 msgr2=0x7fdc4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0052a0 0x7fdc4c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc56ffd640 1 -- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc4c003970 msgr2=0x7fdc4c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc4c003970 0x7fdc4c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc56ffd640 1 -- 172.21.15.70:0/1642720341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c003520 con 0x7fdc4c0047c0 2023-08-25T09:44:05.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.914+0000 7fdc5ffff640 1 -- 172.21.15.70:0/1642720341 reap_dead start 2023-08-25T09:44:05.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.915+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc4c0047c0 0x7fdc4c004c40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdc48007fd0 tx=0x7fdc4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d3fb5d7ce93446 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.915+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48005ad0 con 0x7fdc4c0047c0 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.915+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48005c30 con 0x7fdc4c0047c0 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.915+0000 7fdc5effd640 1 -- 172.21.15.70:0/1642720341 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4802ca50 con 0x7fdc4c0047c0 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 -- 172.21.15.70:0/1642720341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc4c0047c0 msgr2=0x7fdc4c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc4c0047c0 0x7fdc4c004c40 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdc48007fd0 tx=0x7fdc4c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 -- 172.21.15.70:0/1642720341 shutdown_connections 2023-08-25T09:44:05.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 --2- 172.21.15.70:0/1642720341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc4c0047c0 0x7fdc4c004c40 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 -- 172.21.15.70:0/1642720341 >> 172.21.15.70:0/1642720341 conn(0x7fdc600ce760 msgr2=0x7fdc600ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 -- 172.21.15.70:0/1642720341 shutdown_connections 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.916+0000 7fdc65b86640 1 -- 172.21.15.70:0/1642720341 wait complete. 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.918+0000 7fdc65b86640 1 Processor -- start 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.918+0000 7fdc65b86640 1 -- start start 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc65b86640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0047c0 0x7fdc60156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc65b86640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 0x7fdc600c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc65b86640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc600c4570 0x7fdc601f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc65b86640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60156bd0 con 0x7fdc60158f20 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc65b86640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc60156de0 con 0x7fdc600c4570 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc65b86640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc601570c0 con 0x7fdc4c0047c0 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 0x7fdc600c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 0x7fdc600c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45398/0 (socket says 172.21.15.70:45398) 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 -- 172.21.15.70:0/3970570414 learned_addr learned my addr 172.21.15.70:0/3970570414 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:05.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 -- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0047c0 msgr2=0x7fdc60156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0047c0 0x7fdc60156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 -- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc600c4570 msgr2=0x7fdc601f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc600c4570 0x7fdc601f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc600c4570 0x7fdc601f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.919+0000 7fdc56ffd640 1 -- 172.21.15.70:0/3970570414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc601573a0 con 0x7fdc60158f20 2023-08-25T09:44:05.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.920+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 0x7fdc600c4030 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009000 tx=0x7fdc4802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.920+0000 7fdc5cff9640 1 -- 172.21.15.70:0/3970570414 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4803c040 con 0x7fdc60158f20 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.920+0000 7fdc5cff9640 1 -- 172.21.15.70:0/3970570414 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4802cd60 con 0x7fdc60158f20 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.920+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc60157680 con 0x7fdc60158f20 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.920+0000 7fdc5cff9640 1 -- 172.21.15.70:0/3970570414 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480072b0 con 0x7fdc60158f20 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.920+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc601f24a0 con 0x7fdc60158f20 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.922+0000 7fdc5cff9640 1 -- 172.21.15.70:0/3970570414 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdc48032070 con 0x7fdc60158f20 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.923+0000 7fdc5cff9640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdc4006a050 0x7fdc4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.923+0000 7fdc5cff9640 1 -- 172.21.15.70:0/3970570414 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc480a9b40 con 0x7fdc60158f20 2023-08-25T09:44:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.923+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdc4006a050 0x7fdc4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.924+0000 7fdc687d3740 1 Processor -- start 2023-08-25T09:44:05.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.924+0000 7fdc687d3740 1 -- start start 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.924+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdc4006a050 0x7fdc4006c510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdc50007fd0 tx=0x7fdc50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db985530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db955130 0x5591db9823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591db982900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc687d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db8c8190 con 0x5591db958cd0 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db8c5880 con 0x5591db955130 2023-08-25T09:44:05.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591db8c1170 con 0x5591db985150 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591db982900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591db982900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45404/0 (socket says 172.21.15.70:45404) 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2694854245 learned_addr learned my addr 172.21.15.70:0/2694854245 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2694854245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 msgr2=0x5591db985530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db985530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2694854245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db955130 msgr2=0x5591db9823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db955130 0x5591db9823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.925+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2694854245 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591db8bf7f0 con 0x5591db958cd0 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.926+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591db982900 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fdc600cf590 tx=0x7fdc58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d434f87472867df server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.926+0000 7fdc557fa640 1 -- 172.21.15.70:0/2694854245 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004850 con 0x5591db958cd0 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.926+0000 7fdc557fa640 1 -- 172.21.15.70:0/2694854245 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800f050 con 0x5591db958cd0 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.926+0000 7fdc557fa640 1 -- 172.21.15.70:0/2694854245 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58014650 con 0x5591db958cd0 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.926+0000 7fdc687d3740 1 -- 172.21.15.70:0/2694854245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 msgr2=0x5591db982900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.926+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591db982900 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fdc600cf590 tx=0x7fdc58004600 comp rx=0 tx=0).stop 2023-08-25T09:44:05.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 -- 172.21.15.70:0/2694854245 shutdown_connections 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591db982900 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db955130 0x5591db9823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2694854245 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db985530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 -- 172.21.15.70:0/2694854245 >> 172.21.15.70:0/2694854245 conn(0x5591db9573e0 msgr2=0x5591db99ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 -- 172.21.15.70:0/2694854245 shutdown_connections 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 -- 172.21.15.70:0/2694854245 wait complete. 2023-08-25T09:44:05.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.927+0000 7fdc687d3740 1 -- start start 2023-08-25T09:44:05.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.928+0000 7fdc687d3740 1 Processor -- start 2023-08-25T09:44:05.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db958cd0 0x5591db96c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db97af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 0x5591db9b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db860d20 con 0x5591db97b460 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db8609c0 con 0x5591db958cd0 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591db626f20 con 0x5591db985150 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc687d3740 10 client.0 _ll_register_callbacks cb 0x5591db878630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 0x5591db9b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 0x5591db9b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45414/0 (socket says 172.21.15.70:45414) 2023-08-25T09:44:05.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 -- 172.21.15.70:0/3806717048 learned_addr learned my addr 172.21.15.70:0/3806717048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db958cd0 0x5591db96c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 -- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 msgr2=0x5591db97af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db97af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 -- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db958cd0 msgr2=0x5591db96c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db958cd0 0x5591db96c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.929+0000 7fdc5ffff640 1 -- 172.21.15.70:0/3806717048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc58003de0 con 0x5591db97b460 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 0x5591db9b3430 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdc58014af0 tx=0x7fdc58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9871a9a03aaa91cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/3806717048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58028620 con 0x5591db97b460 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591db895540 con 0x5591db97b460 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/3806717048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc58004850 con 0x5591db97b460 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/3806717048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc58028780 con 0x5591db97b460 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 10 client.6936 fetch_fsmap learned FSMap version 41 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5591db8b8f30 con 0x5591db97b460 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/3806717048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5591db895540 con 0x5591db97b460 2023-08-25T09:44:05.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/3806717048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdc580288e0 con 0x5591db97b460 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 10 client.6936 fetch_fsmap finished waiting for FSMap version 41 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 10 client.6936 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 20 client.6936 populate_metadata read hostname 'smithi070' 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db97e970 0x5591db97ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 4 client.6936 mds_command: new command op to 6780 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.930+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 --> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591db8ba150 con 0x5591db97e970 2023-08-25T09:44:05.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.931+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db97e970 0x5591db97ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.931+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db97e970 0x5591db97ed50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b94fbf516018e8a server_cookie=9585ed8dc148b9f7 in_seq=0 out_seq=0 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.931+0000 7fdc3e7fc640 10 client.6936 ms_handle_connect on v2:172.21.15.70:6836/1195200165 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/3806717048 <== mds.0 v2:172.21.15.70:6836/1195200165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5591db8ba150 con 0x5591db97e970 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc3e7fc640 10 client.6936 handle_command_reply: tid=0 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 client.6936 shutdown 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 msgr2=0x5591db9b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 0x5591db9b3430 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fdc58014af0 tx=0x7fdc58004a40 comp rx=0 tx=0).stop 2023-08-25T09:44:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 shutdown_connections 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db97b460 0x5591db9b3430 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db97af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db97e970 0x5591db97ed50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 --2- 172.21.15.70:0/3806717048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db958cd0 0x5591db96c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 >> 172.21.15.70:0/3806717048 conn(0x5591db955130 msgr2=0x5591db955e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.932+0000 7fdc3e7fc640 0 client.6936 ms_handle_reset on v2:172.21.15.70:6836/1195200165 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.933+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 shutdown_connections 2023-08-25T09:44:05.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.933+0000 7fdc687d3740 1 -- 172.21.15.70:0/3806717048 wait complete. 2023-08-25T09:44:05.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.933+0000 7fdc687d3740 20 client.6936 trim_cache size 0 max 16384 2023-08-25T09:44:05.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.949+0000 7fdc687d3740 1 Processor -- start 2023-08-25T09:44:05.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.949+0000 7fdc687d3740 1 -- start start 2023-08-25T09:44:05.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db958cd0 0x5591db9b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 0x5591db968a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 0x5591db97ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc687d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db8c30f0 con 0x5591db985150 2023-08-25T09:44:05.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db968f50 con 0x5591db97b460 2023-08-25T09:44:05.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591db985d70 con 0x5591db958cd0 2023-08-25T09:44:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 0x5591db97ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 0x5591db97ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45418/0 (socket says 172.21.15.70:45418) 2023-08-25T09:44:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2751483648 learned_addr learned my addr 172.21.15.70:0/2751483648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2751483648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db958cd0 msgr2=0x5591db9b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db958cd0 0x5591db9b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2751483648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 msgr2=0x5591db968a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 0x5591db968a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.950+0000 7fdc5ffff640 1 -- 172.21.15.70:0/2751483648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591db85fa10 con 0x5591db985150 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.951+0000 7fdc5ffff640 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 0x5591db97ed50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdc58014800 tx=0x7fdc58021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7b371e7cd33d4b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.951+0000 7fdc55ffb640 1 -- 172.21.15.70:0/2751483648 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58028d50 con 0x5591db985150 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.951+0000 7fdc55ffb640 1 -- 172.21.15.70:0/2751483648 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc58028420 con 0x5591db985150 2023-08-25T09:44:05.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.951+0000 7fdc55ffb640 1 -- 172.21.15.70:0/2751483648 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc580287b0 con 0x5591db985150 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.951+0000 7fdc687d3740 1 -- 172.21.15.70:0/2751483648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 msgr2=0x5591db97ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 0x5591db97ed50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdc58014800 tx=0x7fdc58021070 comp rx=0 tx=0).stop 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 -- 172.21.15.70:0/2751483648 shutdown_connections 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db985150 0x5591db97ed50 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 0x5591db968a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 --2- 172.21.15.70:0/2751483648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db958cd0 0x5591db9b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 -- 172.21.15.70:0/2751483648 >> 172.21.15.70:0/2751483648 conn(0x5591db955130 msgr2=0x5591db9c28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 -- 172.21.15.70:0/2751483648 shutdown_connections 2023-08-25T09:44:05.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.952+0000 7fdc687d3740 1 -- 172.21.15.70:0/2751483648 wait complete. 2023-08-25T09:44:05.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.953+0000 7fdc687d3740 1 -- start start 2023-08-25T09:44:05.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.953+0000 7fdc687d3740 1 Processor -- start 2023-08-25T09:44:05.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591dba080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 0x5591db97ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db97f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db9802f0 con 0x5591db958cd0 2023-08-25T09:44:05.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591db959590 con 0x5591db97b460 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5591dba08920 con 0x5591db985150 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc687d3740 10 client.0 _ll_register_callbacks cb 0x7fdc58003ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591dba080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591dba080c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45430/0 (socket says 172.21.15.70:45430) 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 -- 172.21.15.70:0/1729480627 learned_addr learned my addr 172.21.15.70:0/1729480627 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 -- 172.21.15.70:0/1729480627 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 msgr2=0x5591db97f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db97f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 -- 172.21.15.70:0/1729480627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 msgr2=0x5591db97ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 0x5591db97ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.954+0000 7fdc5f7fe640 1 -- 172.21.15.70:0/1729480627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591db8b8f30 con 0x5591db958cd0 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc5f7fe640 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591dba080c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdc600c7f50 tx=0x7fdc50014b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5987cb6061d8ea server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc3e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/1729480627 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500232e0 con 0x5591db958cd0 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/1729480627 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc50023440 con 0x5591db958cd0 2023-08-25T09:44:05.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc5800f050 con 0x5591db958cd0 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/1729480627 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc50020890 con 0x5591db958cd0 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc687d3740 10 client.6942 fetch_fsmap learned FSMap version 41 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.955+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x5591db884de0 con 0x5591db958cd0 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/1729480627 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800f050 con 0x5591db958cd0 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/1729480627 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdc500209f0 con 0x5591db958cd0 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc687d3740 10 client.6942 fetch_fsmap finished waiting for FSMap version 41 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc687d3740 10 client.6942 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:44:05.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc687d3740 20 client.6942 populate_metadata read hostname 'smithi070' 2023-08-25T09:44:05.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc687d3740 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db9810f0 0x5591db99f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:05.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc687d3740 4 client.6942 mds_command: new command op to 6780 tid=0[{"prefix": "status"}] 2023-08-25T09:44:05.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 --> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5591db5705e0 con 0x5591db9810f0 2023-08-25T09:44:05.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db9810f0 0x5591db99f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:05.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc3e7fc640 10 client.6942 ms_handle_connect on v2:172.21.15.70:6836/1195200165 2023-08-25T09:44:05.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.956+0000 7fdc56ffd640 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db9810f0 0x5591db99f940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b85ff7b067a35a0 server_cookie=8e21a3e1cc9862aa in_seq=0 out_seq=0 2023-08-25T09:44:05.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc3e7fc640 1 -- 172.21.15.70:0/1729480627 <== mds.0 v2:172.21.15.70:6836/1195200165 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x5591db5705e0 con 0x5591db9810f0 2023-08-25T09:44:05.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc3e7fc640 10 client.6942 handle_command_reply: tid=0 2023-08-25T09:44:05.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 client.6942 shutdown 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 msgr2=0x5591dba080c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591dba080c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdc600c7f50 tx=0x7fdc50014b90 comp rx=0 tx=0).stop 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 shutdown_connections 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5591db985150 0x5591db97f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:6836/1195200165,v1:172.21.15.70:6837/1195200165] conn(0x5591db9810f0 0x5591db99f940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5591db97b460 0x5591db97ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 --2- 172.21.15.70:0/1729480627 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5591db958cd0 0x5591dba080c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc3e7fc640 0 client.6942 ms_handle_reset on v2:172.21.15.70:6836/1195200165 2023-08-25T09:44:05.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 >> 172.21.15.70:0/1729480627 conn(0x5591db955130 msgr2=0x5591db950730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 shutdown_connections 2023-08-25T09:44:05.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 1 -- 172.21.15.70:0/1729480627 wait complete. 2023-08-25T09:44:05.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.957+0000 7fdc687d3740 20 client.6942 trim_cache size 0 max 16384 2023-08-25T09:44:05.966 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6780, 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 11.576403687000001, 2023-08-25T09:44:05.967 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 41, 2023-08-25T09:44:05.968 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 94, 2023-08-25T09:44:05.968 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 94, 2023-08-25T09:44:05.968 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 17.67917228 2023-08-25T09:44:05.968 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdc4006a050 msgr2=0x7fdc4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdc4006a050 0x7fdc4006c510 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdc50007fd0 tx=0x7fdc50009040 comp rx=0 tx=0).stop 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 msgr2=0x7fdc600c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 0x7fdc600c4030 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009000 tx=0x7fdc4802e070 comp rx=0 tx=0).stop 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 shutdown_connections 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdc4006a050 0x7fdc4006c510 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdc600c4570 0x7fdc601f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdc60158f20 0x7fdc600c4030 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 --2- 172.21.15.70:0/3970570414 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdc4c0047c0 0x7fdc60156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:05.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 >> 172.21.15.70:0/3970570414 conn(0x7fdc600ce760 msgr2=0x7fdc600cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:05.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 shutdown_connections 2023-08-25T09:44:05.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:05.958+0000 7fdc65b86640 1 -- 172.21.15.70:0/3970570414 wait complete. 2023-08-25T09:44:05.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:44:06.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.158+0000 7f06b9f79640 1 Processor -- start 2023-08-25T09:44:06.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.158+0000 7f06b9f79640 1 -- start start 2023-08-25T09:44:06.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b9f79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 0x7f06b406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b9f79640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06b406b2b0 0x7f06b415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b9f79640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b40b1b60 con 0x7f06b415c6d0 2023-08-25T09:44:06.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b9f79640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b406bd80 con 0x7f06b406b2b0 2023-08-25T09:44:06.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b9f79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b406b7c0 con 0x7f06b415e830 2023-08-25T09:44:06.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f06b415c6d0 0x7f06b40b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39178/0 (socket says 172.21.15.70:39178) 2023-08-25T09:44:06.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.160+0000 7f06b3fff640 1 -- 172.21.15.70:0/3448826776 learned_addr learned my addr 172.21.15.70:0/3448826776 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:06.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06abfff640 1 --2- 172.21.15.70:0/3448826776 >> v2:172.21.15.155:3300/0 conn(0x7f06b406b2b0 0x7f06b415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b37fe640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 0x7f06b406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172547361 0 0) 0x7f06b40b1b60 con 0x7f06b415c6d0 2023-08-25T09:44:06.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06a0003520 con 0x7f06b415c6d0 2023-08-25T09:44:06.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 579724836 0 0) 0x7f06a0003520 con 0x7f06b415c6d0 2023-08-25T09:44:06.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 msgr2=0x7f06b406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:06.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 0x7f06b406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 >> v2:172.21.15.155:3300/0 conn(0x7f06b406b2b0 msgr2=0x7f06b415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:06.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 --2- 172.21.15.70:0/3448826776 >> v2:172.21.15.155:3300/0 conn(0x7f06b406b2b0 0x7f06b415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b406baa0 con 0x7f06b415c6d0 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f06ac0019f0 con 0x7f06b415c6d0 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06abfff640 1 --2- 172.21.15.70:0/3448826776 >> v2:172.21.15.155:3300/0 conn(0x7f06b406b2b0 0x7f06b415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 >> v1:172.21.15.70:6789/0 conn(0x7f06b415c6d0 legacy=0x7f06b40b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 0x7f06a0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a0003fa0 0x7f06a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a0004d00 0x7f06a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b406bd80 con 0x7f06a0003fa0 2023-08-25T09:44:06.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b406b7c0 con 0x7f06a0004d00 2023-08-25T09:44:06.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.161+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a000fe90 con 0x7f06b415e830 2023-08-25T09:44:06.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06abfff640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a0003fa0 0x7f06a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06b37fe640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 0x7f06a0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06abfff640 1 -- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 msgr2=0x7f06a0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:06.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06abfff640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06b415e830 0x7f06a0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06b3fff640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a0004d00 0x7f06a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06abfff640 1 -- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a0004d00 msgr2=0x7f06a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:06.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06abfff640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a0004d00 0x7f06a0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06abfff640 1 -- 172.21.15.70:0/3448826776 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a0003520 con 0x7f06a0003fa0 2023-08-25T09:44:06.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.162+0000 7f06b3fff640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06a0004d00 0x7f06a0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:06.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.163+0000 7f06abfff640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a0003fa0 0x7f06a00047c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f06b406b6f0 tx=0x7f06a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2763dc427c945e6d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:06.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.163+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c004230 con 0x7f06a0003fa0 2023-08-25T09:44:06.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.163+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c004390 con 0x7f06a0003fa0 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.163+0000 7f06b2ffd640 1 -- 172.21.15.70:0/3448826776 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c003830 con 0x7f06a0003fa0 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b9f79640 1 -- 172.21.15.70:0/3448826776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a0003fa0 msgr2=0x7f06a00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b9f79640 1 --2- 172.21.15.70:0/3448826776 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06a0003fa0 0x7f06a00047c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f06b406b6f0 tx=0x7f06a00057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b3fff640 1 -- 172.21.15.70:0/3448826776 reap_dead start 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b9f79640 1 -- 172.21.15.70:0/3448826776 shutdown_connections 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b9f79640 1 -- 172.21.15.70:0/3448826776 >> 172.21.15.70:0/3448826776 conn(0x7f06b4069850 msgr2=0x7f06b4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:06.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b9f79640 1 -- 172.21.15.70:0/3448826776 shutdown_connections 2023-08-25T09:44:06.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.164+0000 7f06b9f79640 1 -- 172.21.15.70:0/3448826776 wait complete. 2023-08-25T09:44:06.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.166+0000 7f06b9f79640 1 Processor -- start 2023-08-25T09:44:06.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.166+0000 7f06b9f79640 1 -- start start 2023-08-25T09:44:06.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06b9f79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a0003fa0 0x7f06b40ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06b9f79640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 0x7f06b41eddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06b9f79640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06b41ee330 0x7f06b40ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06b9f79640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40ce1b0 con 0x7f06b40a8d80 2023-08-25T09:44:06.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06b9f79640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40cd5a0 con 0x7f06b41ee330 2023-08-25T09:44:06.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06b9f79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b40cd880 con 0x7f06a0003fa0 2023-08-25T09:44:06.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 0x7f06b41eddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06abfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 0x7f06b41eddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:45456/0 (socket says 172.21.15.70:45456) 2023-08-25T09:44:06.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.167+0000 7f06abfff640 1 -- 172.21.15.70:0/2115487072 learned_addr learned my addr 172.21.15.70:0/2115487072 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:06.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06abfff640 1 -- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a0003fa0 msgr2=0x7f06b40ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:06.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06abfff640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a0003fa0 0x7f06b40ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06b3fff640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06b41ee330 0x7f06b40ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06abfff640 1 -- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06b41ee330 msgr2=0x7f06b40ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:06.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06abfff640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06b41ee330 0x7f06b40ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06abfff640 1 -- 172.21.15.70:0/2115487072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b40cdb60 con 0x7f06b40a8d80 2023-08-25T09:44:06.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06abfff640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 0x7f06b41eddf0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f069c009990 tx=0x7f069c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:06.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c030660 con 0x7f06b40a8d80 2023-08-25T09:44:06.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069c0307c0 con 0x7f06b40a8d80 2023-08-25T09:44:06.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c030b70 con 0x7f06b40a8d80 2023-08-25T09:44:06.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.168+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b40cde40 con 0x7f06b40a8d80 2023-08-25T09:44:06.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.169+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b40a9200 con 0x7f06b40a8d80 2023-08-25T09:44:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.170+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b40a9420 con 0x7f06b40a8d80 2023-08-25T09:44:06.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.171+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f069c030cd0 con 0x7f06b40a8d80 2023-08-25T09:44:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.172+0000 7f06b0ff9640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f069406a080 0x7f069406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.172+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f069c0aa650 con 0x7f06b40a8d80 2023-08-25T09:44:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.172+0000 7f06b37fe640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f069406a080 0x7f069406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.172+0000 7f06b37fe640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f069406a080 0x7f069406c540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f06a4006f00 tx=0x7f06a4006e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.176+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f06b40a9420 con 0x7f06b40a8d80 2023-08-25T09:44:06.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.385+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06b40a9420 con 0x7f06b40a8d80 2023-08-25T09:44:06.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.386+0000 7f06b0ff9640 1 -- 172.21.15.70:0/2115487072 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06b40a9420 con 0x7f06b40a8d80 2023-08-25T09:44:06.389 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:44:06.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f069406a080 msgr2=0x7f069406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:06.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f069406a080 0x7f069406c540 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f06a4006f00 tx=0x7f06a4006e10 comp rx=0 tx=0).stop 2023-08-25T09:44:06.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 msgr2=0x7f06b41eddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:06.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 0x7f06b41eddf0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f069c009990 tx=0x7f069c004730 comp rx=0 tx=0).stop 2023-08-25T09:44:06.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 shutdown_connections 2023-08-25T09:44:06.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f069406a080 0x7f069406c540 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06b41ee330 0x7f06b40ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06b40a8d80 0x7f06b41eddf0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 --2- 172.21.15.70:0/2115487072 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06a0003fa0 0x7f06b40ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:06.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 >> 172.21.15.70:0/2115487072 conn(0x7f06b4069850 msgr2=0x7f06b4070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:06.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 shutdown_connections 2023-08-25T09:44:06.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:06.389+0000 7f06b9f79640 1 -- 172.21.15.70:0/2115487072 wait complete. 2023-08-25T09:44:06.409 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:44:06.409 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:44:06.410 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:44:06.410 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:44:06.410 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:06.410 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:44:06.411 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:44:06.411 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:44:06.411 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:44:06.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:06.412 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:44:06.461 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:44:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:06.463 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 42030sec preferred_lft 42030sec 2023-08-25T09:44:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.465 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:44:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:44:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:06.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:06.468 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:44:06.603 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:44:06.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:06.608 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:44:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:44:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:44:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:44:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:44:06.660 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:44:06.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:06.662 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:44:06.662 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:44:06.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:44:06.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:44:06.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:44:06.663 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:44:06.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:06.831 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:44:06.883 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:44:06.883 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:44:06.884 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:44:06.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:06.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:44:06.987 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:44:06.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:44:06.988 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:07.045 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:44:07.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:44:07.046 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:07.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:44:07.203 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:44:07.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:44:07.355 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:44:07.901 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:44:07.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:07.903 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:44:07.965 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:44:07.965 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:44:07.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:44:08.051 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:44:08.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:44:08.123 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:44:08.126 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:44:08.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:08.127 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:44:08.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:44:08.161+0000 7f829d034480 -1 init, newargv = 0x560740295340 newargc=15 2023-08-25T09:44:08.163 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[106653]: starting ceph client 2023-08-25T09:44:08.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[106653]: starting fuse 2023-08-25T09:44:09.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:44:09.378 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:44:09.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:44:09.472 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:44:09.475 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:44:09.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:09.476 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:44:09.535 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:44:09.535 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:44:09.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:44:09.536 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:44:09.536 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:44:09.537 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:44:09.537 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:44:09.537 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:09.537 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:44:09.538 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:44:09.538 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:44:09.538 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:44:09.539 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:09.539 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:44:09.539 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:44:09.540 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:44:09.540 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:44:09.540 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:09.540 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:44:09.541 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:44:09.541 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:44:09.541 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:44:09.542 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:44:09.542 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:44:09.542 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:44:09.542 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:44:09.543 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:09.543 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:44:09.543 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:44:09.720 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.106653.asok 2023-08-25T09:44:09.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:44:09.737 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106653.asok 2023-08-25T09:44:09.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:09.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106653.asok status 2023-08-25T09:44:09.977 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:44:09.977 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:44:09.977 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:44:09.978 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:44:09.978 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:44:09.978 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:44:09.978 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:44:09.979 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "106653", 2023-08-25T09:44:09.979 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:44:09.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:44:09.979 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:44:09.979 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:44:09.980 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 6957, 2023-08-25T09:44:09.980 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:44:09.980 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:44:09.980 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:44:09.980 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 6957 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3980176671 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:44:09.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:44:09.982 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:44:09.982 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:44:09.982 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:44:09.982 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3980176671 2023-08-25T09:44:09.982 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:44:09.982 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.6957 192.168.0.1:0/3980176671", 2023-08-25T09:44:09.983 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3980176671", 2023-08-25T09:44:09.983 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:44:09.983 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 41, 2023-08-25T09:44:09.983 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 94, 2023-08-25T09:44:09.983 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:44:09.983 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:44:09.984 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:44:09.984 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:44:09.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:09.992 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:44:09.993 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:10.048 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:44:10.048 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:10.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:10.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:10.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:44:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.363+0000 7f18c3aa7640 1 Processor -- start 2023-08-25T09:44:10.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.363+0000 7f18c3aa7640 1 -- start start 2023-08-25T09:44:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c3aa7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18bc0cea30 0x7f18bc0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c3aa7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18bc06f050 con 0x7f18bc0ac870 2023-08-25T09:44:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c3aa7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18bc06f330 con 0x7f18bc15cb40 2023-08-25T09:44:10.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c3aa7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18bc06f610 con 0x7f18bc0cea30 2023-08-25T09:44:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c1042640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18bc0cea30 0x7f18bc0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c1042640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f18bc0cea30 0x7f18bc0b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60648/0 (socket says 172.21.15.70:60648) 2023-08-25T09:44:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c1042640 1 -- 172.21.15.70:0/535656860 learned_addr learned my addr 172.21.15.70:0/535656860 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:10.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c2044640 1 --1- 172.21.15.70:0/535656860 >> v1:172.21.15.70:6789/0 conn(0x7f18bc0ac870 0x7f18bc0b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44024/0 (socket says 172.21.15.70:44024) 2023-08-25T09:44:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c1042640 1 -- 172.21.15.70:0/535656860 >> v1:172.21.15.155:6789/0 conn(0x7f18bc15cb40 legacy=0x7f18bc0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c1042640 1 -- 172.21.15.70:0/535656860 >> v1:172.21.15.70:6789/0 conn(0x7f18bc0ac870 legacy=0x7f18bc0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:10.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.365+0000 7f18c1042640 1 -- 172.21.15.70:0/535656860 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18bc06f8f0 con 0x7f18bc0cea30 2023-08-25T09:44:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.366+0000 7f18c1042640 1 --2- 172.21.15.70:0/535656860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0cea30 0x7f18bc0b1b30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009870 tx=0x7f18ac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca059f2b4626ccb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.367+0000 7f18c0841640 1 -- 172.21.15.70:0/535656860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac030860 con 0x7f18bc0cea30 2023-08-25T09:44:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.367+0000 7f18c0841640 1 -- 172.21.15.70:0/535656860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac0309c0 con 0x7f18bc0cea30 2023-08-25T09:44:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.367+0000 7f18c0841640 1 -- 172.21.15.70:0/535656860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac030d50 con 0x7f18bc0cea30 2023-08-25T09:44:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.367+0000 7f18c3aa7640 1 -- 172.21.15.70:0/535656860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0cea30 msgr2=0x7f18bc0b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:10.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.367+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/535656860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0cea30 0x7f18bc0b1b30 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009870 tx=0x7f18ac02b9a0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.368+0000 7f18c3aa7640 1 -- 172.21.15.70:0/535656860 shutdown_connections 2023-08-25T09:44:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.368+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/535656860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0cea30 0x7f18bc0b1b30 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.368+0000 7f18c3aa7640 1 -- 172.21.15.70:0/535656860 >> 172.21.15.70:0/535656860 conn(0x7f18bc069850 msgr2=0x7f18bc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.368+0000 7f18c3aa7640 1 -- 172.21.15.70:0/535656860 shutdown_connections 2023-08-25T09:44:10.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.368+0000 7f18c3aa7640 1 -- 172.21.15.70:0/535656860 wait complete. 2023-08-25T09:44:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.370+0000 7f18c3aa7640 1 Processor -- start 2023-08-25T09:44:10.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.370+0000 7f18c3aa7640 1 -- start start 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c3aa7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 0x7f18bc0ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c3aa7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc0cea30 0x7f18bc146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c3aa7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18bc15cb40 0x7f18bc0cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c3aa7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0cdcb0 con 0x7f18bc0ac870 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c3aa7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0ccc10 con 0x7f18bc0cea30 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c3aa7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18bc0ccef0 con 0x7f18bc15cb40 2023-08-25T09:44:10.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c1843640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 0x7f18bc0ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c1843640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 0x7f18bc0ce4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60664/0 (socket says 172.21.15.70:60664) 2023-08-25T09:44:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.371+0000 7f18c1843640 1 -- 172.21.15.70:0/3715633298 learned_addr learned my addr 172.21.15.70:0/3715633298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:10.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18c1843640 1 -- 172.21.15.70:0/3715633298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18bc15cb40 msgr2=0x7f18bc0cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18c1843640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18bc15cb40 0x7f18bc0cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18c1843640 1 -- 172.21.15.70:0/3715633298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc0cea30 msgr2=0x7f18bc146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18c1843640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc0cea30 0x7f18bc146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18c1843640 1 -- 172.21.15.70:0/3715633298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ac0094d0 con 0x7f18bc0ac870 2023-08-25T09:44:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18c1843640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 0x7f18bc0ce4d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b3f0 tx=0x7f18b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:10.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8011830 con 0x7f18bc0ac870 2023-08-25T09:44:10.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.372+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8011990 con 0x7f18bc0ac870 2023-08-25T09:44:10.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.373+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8011d40 con 0x7f18bc0ac870 2023-08-25T09:44:10.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.373+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18bc0cd1d0 con 0x7f18bc0ac870 2023-08-25T09:44:10.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.373+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18bc0cd790 con 0x7f18bc0ac870 2023-08-25T09:44:10.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.374+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18bc04e420 con 0x7f18bc0ac870 2023-08-25T09:44:10.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.375+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18b801a400 con 0x7f18bc0ac870 2023-08-25T09:44:10.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.376+0000 7f18b27fc640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:10.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.376+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18b80884d0 con 0x7f18bc0ac870 2023-08-25T09:44:10.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.376+0000 7f18c1042640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:10.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.377+0000 7f18c1042640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18ac036000 tx=0x7f18ac03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:10.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.381+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18b8053bd0 con 0x7f18bc0ac870 2023-08-25T09:44:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.612+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f18bc0cc900 con 0x7f18bc0ac870 2023-08-25T09:44:10.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.612+0000 7f18b27fc640 1 -- 172.21.15.70:0/3715633298 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f18b80571f0 con 0x7f18bc0ac870 2023-08-25T09:44:10.617 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:44:10.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 msgr2=0x7f18a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:10.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f18ac036000 tx=0x7f18ac03b040 comp rx=0 tx=0).stop 2023-08-25T09:44:10.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 msgr2=0x7f18bc0ce4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:10.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 0x7f18bc0ce4d0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f18b800b3f0 tx=0x7f18b800b8b0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 shutdown_connections 2023-08-25T09:44:10.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18a006a050 0x7f18a006c510 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18bc15cb40 0x7f18bc0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18bc0cea30 0x7f18bc146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 --2- 172.21.15.70:0/3715633298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18bc0ac870 0x7f18bc0ce4d0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:10.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 >> 172.21.15.70:0/3715633298 conn(0x7f18bc069850 msgr2=0x7f18bc06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:10.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 shutdown_connections 2023-08-25T09:44:10.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:10.615+0000 7f18c3aa7640 1 -- 172.21.15.70:0/3715633298 wait complete. 2023-08-25T09:44:10.637 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:44:10.638 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:44:10.638 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:44:10.638 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:44:10.638 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:10.639 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:44:10.639 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:44:10.639 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:44:10.639 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:44:10.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:10.640 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:44:10.702 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:44:10.702 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:44:10.702 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:44:10.702 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.702 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:10.703 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 42038sec preferred_lft 42038sec 2023-08-25T09:44:10.704 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:44:10.705 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.705 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:44:10.705 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:44:10.705 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:44:10.705 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.705 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:44:10.706 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:44:10.707 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:44:10.707 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:44:10.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:10.708 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:44:10.800 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:44:10.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:10.805 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:44:10.861 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:44:10.861 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:44:10.861 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:44:10.861 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:44:10.862 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:44:10.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:10.862 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:44:10.863 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:44:10.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:44:10.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:44:10.864 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:44:10.864 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:44:11.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:11.048 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:44:11.099 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:44:11.100 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:44:11.100 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:44:11.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:11.103 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:44:11.196 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:44:11.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:44:11.197 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:11.263 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:44:11.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:44:11.264 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:11.324 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:44:11.423 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:44:11.424 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:44:11.528 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:44:11.531 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:44:11.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:11.533 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:44:11.584 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:44:11.584 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:44:11.590 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:44:11.672 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:44:11.673 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:44:11.745 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:44:11.748 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:44:11.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:11.749 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:44:11.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:44:11.772+0000 7f58bcd6d480 -1 init, newargv = 0x5644f08a0180 newargc=15 2023-08-25T09:44:11.775 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[89968]: starting ceph client 2023-08-25T09:44:11.785 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[89968]: starting fuse 2023-08-25T09:44:11.793 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:44:11.794 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:44:11.795 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:44:11.795 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:44:11.796 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:44:11.797 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:44:11.797 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:44:11.798 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:44:11.798 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:44:11.798 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:44:11.799 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:44:11.800 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:44:11.800 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:44:11.801 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:44:11.801 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:44:11.801 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:44:11.802 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:44:11.802 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:44:11.802 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:44:11.803 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:44:11.803 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:44:11.803 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:44:11.803 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:44:11.804 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:44:11.804 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:44:11.805 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:44:11.805 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:44:11.805 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:44:11.806 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:44:12.006 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.89968.asok 2023-08-25T09:44:12.011 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:44:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:44:12.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.89968.asok 2023-08-25T09:44:12.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:12.026 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.89968.asok status 2023-08-25T09:44:12.272 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:44:12.273 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "89968", 2023-08-25T09:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:44:12.274 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5054, 2023-08-25T09:44:12.275 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:44:12.276 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5054 2023-08-25T09:44:12.278 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:44:12.279 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3201453961 2023-08-25T09:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:44:12.280 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:44:12.281 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3201453961 2023-08-25T09:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:44:12.282 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5054 192.168.0.1:0/3201453961", 2023-08-25T09:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/3201453961", 2023-08-25T09:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:44:12.283 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 41, 2023-08-25T09:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 94, 2023-08-25T09:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:44:12.284 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:44:12.285 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:44:12.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:12.287 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:44:12.288 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:12.353 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:44:12.354 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:12.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:12.355 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:12.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:44:12.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.678+0000 7f1e83d6b640 1 Processor -- start 2023-08-25T09:44:12.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.678+0000 7f1e83d6b640 1 -- start start 2023-08-25T09:44:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e83d6b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15ec10 0x7f1e7c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e83d6b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e7c15c6d0 0x7f1e7c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:12.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e83d6b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e7c0b1b60 con 0x7f1e7c15e830 2023-08-25T09:44:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e83d6b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e7c06bd80 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e83d6b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c06b7c0 con 0x7f1e7c15ec10 2023-08-25T09:44:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e82308640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e7c15c6d0 0x7f1e7c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e82308640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1e7c15c6d0 0x7f1e7c0b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60670/0 (socket says 172.21.15.70:60670) 2023-08-25T09:44:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e82308640 1 -- 172.21.15.70:0/42628002 learned_addr learned my addr 172.21.15.70:0/42628002 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:12.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e81306640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15ec10 0x7f1e7c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:12.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.680+0000 7f1e82308640 1 -- 172.21.15.70:0/42628002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15ec10 msgr2=0x7f1e7c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:12.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.681+0000 7f1e82308640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15ec10 0x7f1e7c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.681+0000 7f1e82308640 1 -- 172.21.15.70:0/42628002 >> v1:172.21.15.70:6789/0 conn(0x7f1e7c15e830 legacy=0x7f1e7c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:12.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.681+0000 7f1e82308640 1 -- 172.21.15.70:0/42628002 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c06baa0 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.681+0000 7f1e81306640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15ec10 0x7f1e7c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:12.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.682+0000 7f1e82308640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c0b1ff0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009e50 tx=0x7f1e7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4a875a6ed247a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:12.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.682+0000 7f1e80b05640 1 -- 172.21.15.70:0/42628002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7402ca80 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.682+0000 7f1e80b05640 1 -- 172.21.15.70:0/42628002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7402cbe0 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.682+0000 7f1e80b05640 1 -- 172.21.15.70:0/42628002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e740343f0 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.682+0000 7f1e83d6b640 1 -- 172.21.15.70:0/42628002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 msgr2=0x7f1e7c0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:12.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.682+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c0b1ff0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009e50 tx=0x7f1e7402bee0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.683+0000 7f1e83d6b640 1 -- 172.21.15.70:0/42628002 shutdown_connections 2023-08-25T09:44:12.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.683+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c0b1ff0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.683+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/42628002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15ec10 0x7f1e7c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.683+0000 7f1e83d6b640 1 -- 172.21.15.70:0/42628002 >> 172.21.15.70:0/42628002 conn(0x7f1e7c069850 msgr2=0x7f1e7c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:12.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.683+0000 7f1e83d6b640 1 -- 172.21.15.70:0/42628002 shutdown_connections 2023-08-25T09:44:12.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.683+0000 7f1e83d6b640 1 -- 172.21.15.70:0/42628002 wait complete. 2023-08-25T09:44:12.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.684+0000 7f1e83d6b640 1 Processor -- start 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.684+0000 7f1e83d6b640 1 -- start start 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.685+0000 7f1e83d6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.685+0000 7f1e83d6b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15e830 0x7f1e7c0a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.685+0000 7f1e83d6b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e7c15ec10 0x7f1e7c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.685+0000 7f1e83d6b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c0ab780 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.685+0000 7f1e83d6b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c14bb50 con 0x7f1e7c15ec10 2023-08-25T09:44:12.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.685+0000 7f1e83d6b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7c148b70 con 0x7f1e7c15e830 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c14b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60676/0 (socket says 172.21.15.70:60676) 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 -- 172.21.15.70:0/2974893257 learned_addr learned my addr 172.21.15.70:0/2974893257 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 -- 172.21.15.70:0/2974893257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15e830 msgr2=0x7f1e7c0a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15e830 0x7f1e7c0a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 -- 172.21.15.70:0/2974893257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e7c15ec10 msgr2=0x7f1e7c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:12.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e7c15ec10 0x7f1e7c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 -- 172.21.15.70:0/2974893257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74009b30 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.686+0000 7f1e81b07640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c14b610 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c450 tx=0x7f1e7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.687+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780052d0 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.687+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e78005430 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.687+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78012870 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.687+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7c0aa890 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.687+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7c0aae50 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.688+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7c04e420 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.689+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e780129d0 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.689+0000 7f1e727fc640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a050 0x7f1e6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:12.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.690+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e780881b0 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.690+0000 7f1e81306640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a050 0x7f1e6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:12.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.690+0000 7f1e81306640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a050 0x7f1e6006c510 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c006fd0 tx=0x7f1e6c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:12.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.694+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e78053800 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.919+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e7c06c620 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.921+0000 7f1e727fc640 1 -- 172.21.15.70:0/2974893257 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7f1e78056e20 con 0x7f1e7c15c6d0 2023-08-25T09:44:12.923 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:12.923 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:44:12.924 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:44:12.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a050 msgr2=0x7f1e6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:12.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a050 0x7f1e6006c510 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c006fd0 tx=0x7f1e6c008040 comp rx=0 tx=0).stop 2023-08-25T09:44:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 msgr2=0x7f1e7c14b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c14b610 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800c450 tx=0x7f1e7800c910 comp rx=0 tx=0).stop 2023-08-25T09:44:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 shutdown_connections 2023-08-25T09:44:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e6006a050 0x7f1e6006c510 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e7c15ec10 0x7f1e7c15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7c15e830 0x7f1e7c0a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 --2- 172.21.15.70:0/2974893257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7c15c6d0 0x7f1e7c14b610 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:12.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 >> 172.21.15.70:0/2974893257 conn(0x7f1e7c069850 msgr2=0x7f1e7c0b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:12.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 shutdown_connections 2023-08-25T09:44:12.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:12.923+0000 7f1e83d6b640 1 -- 172.21.15.70:0/2974893257 wait complete. 2023-08-25T09:44:12.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-08-25T09:44:13.162 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":6780,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.763548839999999,"mdsmap_epoch":41,"osdmap_epoch":94,"osdmap_epoch_barrier":94,"uptime":24.866339795999998} 2023-08-25T09:44:13.162 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 6780, "mdsmap_epoch": 41, "osdmap_epoch": 94, "osdmap_epoch_barrier": 94, "rank_uptime": 18.76354884, "state": "up:active", "uptime": 24.866339796, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:44:13.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:44:13.377 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:44:13.377 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:44:13.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-08-25T09:44:13.602 INFO:teuthology.orchestra.run.smithi070.stdout:{"mon_tick_interval":"5"} 2023-08-25T09:44:13.602 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-08-25T09:44:13.602 INFO:tasks.cephfs.test_full:pool capacity 556987968, 531MB should be enough to fill it 2023-08-25T09:44:13.603 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:44:13.603 DEBUG:teuthology.orchestra.run.smithi070:> import time 2023-08-25T09:44:13.603 DEBUG:teuthology.orchestra.run.smithi070:> import datetime 2023-08-25T09:44:13.604 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:44:13.604 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:44:13.604 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.605 DEBUG:teuthology.orchestra.run.smithi070:> # Write some buffered data through before going full, all should be well 2023-08-25T09:44:13.605 DEBUG:teuthology.orchestra.run.smithi070:> print("writing some data through which we expect to succeed") 2023-08-25T09:44:13.605 DEBUG:teuthology.orchestra.run.smithi070:> bytes = 0 2023-08-25T09:44:13.605 DEBUG:teuthology.orchestra.run.smithi070:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-08-25T09:44:13.606 DEBUG:teuthology.orchestra.run.smithi070:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-08-25T09:44:13.606 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-08-25T09:44:13.606 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:44:13.606 DEBUG:teuthology.orchestra.run.smithi070:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-08-25T09:44:13.607 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.607 DEBUG:teuthology.orchestra.run.smithi070:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-08-25T09:44:13.607 DEBUG:teuthology.orchestra.run.smithi070:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-08-25T09:44:13.608 DEBUG:teuthology.orchestra.run.smithi070:> # buffered IO, the error should always only appear from fsync and not 2023-08-25T09:44:13.608 DEBUG:teuthology.orchestra.run.smithi070:> # from write 2023-08-25T09:44:13.608 DEBUG:teuthology.orchestra.run.smithi070:> full = False 2023-08-25T09:44:13.608 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.609 DEBUG:teuthology.orchestra.run.smithi070:> for n in range(0, int(531 * 1.1)): 2023-08-25T09:44:13.609 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:44:13.609 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-08-25T09:44:13.609 DEBUG:teuthology.orchestra.run.smithi070:> print("wrote bytes via buffered write, moving on to fsync") 2023-08-25T09:44:13.610 DEBUG:teuthology.orchestra.run.smithi070:> except OSError as e: 2023-08-25T09:44:13.610 DEBUG:teuthology.orchestra.run.smithi070:> if True: 2023-08-25T09:44:13.611 DEBUG:teuthology.orchestra.run.smithi070:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-08-25T09:44:13.611 DEBUG:teuthology.orchestra.run.smithi070:> raise 2023-08-25T09:44:13.612 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:44:13.613 DEBUG:teuthology.orchestra.run.smithi070:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-08-25T09:44:13.613 DEBUG:teuthology.orchestra.run.smithi070:> full = True 2023-08-25T09:44:13.613 DEBUG:teuthology.orchestra.run.smithi070:> break 2023-08-25T09:44:13.614 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.614 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:44:13.614 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:44:13.615 DEBUG:teuthology.orchestra.run.smithi070:> print("fsync'"'"'ed successfully") 2023-08-25T09:44:13.615 DEBUG:teuthology.orchestra.run.smithi070:> except OSError as e: 2023-08-25T09:44:13.615 DEBUG:teuthology.orchestra.run.smithi070:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-08-25T09:44:13.616 DEBUG:teuthology.orchestra.run.smithi070:> full = True 2023-08-25T09:44:13.616 DEBUG:teuthology.orchestra.run.smithi070:> break 2023-08-25T09:44:13.617 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:44:13.617 DEBUG:teuthology.orchestra.run.smithi070:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-08-25T09:44:13.617 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.617 DEBUG:teuthology.orchestra.run.smithi070:> if n > 531 * 0.9: 2023-08-25T09:44:13.618 DEBUG:teuthology.orchestra.run.smithi070:> # Be cautious in the last region where we expect to hit 2023-08-25T09:44:13.618 DEBUG:teuthology.orchestra.run.smithi070:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-08-25T09:44:13.618 DEBUG:teuthology.orchestra.run.smithi070:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-08-25T09:44:13.619 DEBUG:teuthology.orchestra.run.smithi070:> time.sleep(15.0) 2023-08-25T09:44:13.619 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.619 DEBUG:teuthology.orchestra.run.smithi070:> if not full: 2023-08-25T09:44:13.619 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-08-25T09:44:13.620 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.620 DEBUG:teuthology.orchestra.run.smithi070:> # close() should not raise an error because we already caught it in 2023-08-25T09:44:13.620 DEBUG:teuthology.orchestra.run.smithi070:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-08-25T09:44:13.620 DEBUG:teuthology.orchestra.run.smithi070:> # since then because all IOs got cancelled on the full flag. 2023-08-25T09:44:13.621 DEBUG:teuthology.orchestra.run.smithi070:> print("calling close") 2023-08-25T09:44:13.621 DEBUG:teuthology.orchestra.run.smithi070:> os.close(f) 2023-08-25T09:44:13.621 DEBUG:teuthology.orchestra.run.smithi070:> print("close() did not raise error") 2023-08-25T09:44:13.622 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:44:13.622 DEBUG:teuthology.orchestra.run.smithi070:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-08-25T09:44:13.622 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:44:15.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:44:15.274 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:44:19.712 INFO:teuthology.orchestra.run.smithi070.stdout:writing some data through which we expect to succeed 2023-08-25T09:44:19.713 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-08-25T09:44:19.713 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.713 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.713 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 1.00 MB 2023-08-25T09:44:19.713 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.713 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 2.00 MB 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 3.00 MB 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.714 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 4.00 MB 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 5.00 MB 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 6.00 MB 2023-08-25T09:44:19.715 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.716 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.716 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 7.00 MB 2023-08-25T09:44:19.716 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.716 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.716 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 8.00 MB 2023-08-25T09:44:19.717 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.717 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.717 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 9.00 MB 2023-08-25T09:44:19.718 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.718 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.718 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 10.00 MB 2023-08-25T09:44:19.718 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.719 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.719 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 11.00 MB 2023-08-25T09:44:19.719 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.719 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.720 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 12.00 MB 2023-08-25T09:44:19.720 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.720 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.721 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 13.00 MB 2023-08-25T09:44:19.721 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.721 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.721 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 14.00 MB 2023-08-25T09:44:19.722 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.722 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.722 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 15.00 MB 2023-08-25T09:44:19.722 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.723 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.723 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 16.00 MB 2023-08-25T09:44:19.723 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.723 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.723 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 17.00 MB 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 18.00 MB 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 19.00 MB 2023-08-25T09:44:19.724 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 20.00 MB 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 21.00 MB 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.725 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 22.00 MB 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 23.00 MB 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 24.00 MB 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.726 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 25.00 MB 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 26.00 MB 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.727 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 27.00 MB 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 28.00 MB 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 29.00 MB 2023-08-25T09:44:19.728 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 30.00 MB 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 31.00 MB 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.729 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 32.00 MB 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 33.00 MB 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 34.00 MB 2023-08-25T09:44:19.730 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 35.00 MB 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 36.00 MB 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.731 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.732 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 37.00 MB 2023-08-25T09:44:19.732 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.732 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.732 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 38.00 MB 2023-08-25T09:44:19.732 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.732 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 39.00 MB 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 40.00 MB 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.733 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 41.00 MB 2023-08-25T09:44:19.734 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.734 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.734 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 42.00 MB 2023-08-25T09:44:19.734 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.734 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.734 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 43.00 MB 2023-08-25T09:44:19.735 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.735 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.735 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 44.00 MB 2023-08-25T09:44:19.735 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.736 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.736 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 45.00 MB 2023-08-25T09:44:19.736 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.737 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.737 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 46.00 MB 2023-08-25T09:44:19.737 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.738 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.738 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 47.00 MB 2023-08-25T09:44:19.738 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.738 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.739 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 48.00 MB 2023-08-25T09:44:19.739 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.739 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.739 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 49.00 MB 2023-08-25T09:44:19.740 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.740 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.740 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 50.00 MB 2023-08-25T09:44:19.741 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.741 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.741 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 51.00 MB 2023-08-25T09:44:19.741 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.742 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.742 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 52.00 MB 2023-08-25T09:44:19.742 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.743 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.743 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 53.00 MB 2023-08-25T09:44:19.743 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.743 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 54.00 MB 2023-08-25T09:44:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.744 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 55.00 MB 2023-08-25T09:44:19.745 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.745 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.746 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 56.00 MB 2023-08-25T09:44:19.746 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.746 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.746 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 57.00 MB 2023-08-25T09:44:19.746 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.746 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 58.00 MB 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 59.00 MB 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.747 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 60.00 MB 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 61.00 MB 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 62.00 MB 2023-08-25T09:44:19.748 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 63.00 MB 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 64.00 MB 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.749 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 65.00 MB 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 66.00 MB 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.750 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 67.00 MB 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 68.00 MB 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 69.00 MB 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.751 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 70.00 MB 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 71.00 MB 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.752 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 72.00 MB 2023-08-25T09:44:19.753 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.753 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.753 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 73.00 MB 2023-08-25T09:44:19.754 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.754 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.754 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 74.00 MB 2023-08-25T09:44:19.755 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.755 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.755 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 75.00 MB 2023-08-25T09:44:19.756 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.756 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.756 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 76.00 MB 2023-08-25T09:44:19.757 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.757 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.757 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 77.00 MB 2023-08-25T09:44:19.757 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.758 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.758 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 78.00 MB 2023-08-25T09:44:19.758 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.759 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.759 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 79.00 MB 2023-08-25T09:44:19.759 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:19.759 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:19.760 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 80.00 MB 2023-08-25T09:44:21.104 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.104 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.105 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 81.00 MB 2023-08-25T09:44:21.105 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.105 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.105 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 82.00 MB 2023-08-25T09:44:21.105 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 83.00 MB 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 84.00 MB 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 85.00 MB 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 86.00 MB 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 87.00 MB 2023-08-25T09:44:21.107 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.108 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.108 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 88.00 MB 2023-08-25T09:44:21.110 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.110 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.110 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 89.00 MB 2023-08-25T09:44:21.110 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.110 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 90.00 MB 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 91.00 MB 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.111 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 92.00 MB 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 93.00 MB 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 94.00 MB 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.112 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 95.00 MB 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 96.00 MB 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.113 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 97.00 MB 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 98.00 MB 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 99.00 MB 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.114 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 100.00 MB 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 101.00 MB 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 102.00 MB 2023-08-25T09:44:21.115 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 103.00 MB 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 104.00 MB 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 105.00 MB 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 106.00 MB 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 107.00 MB 2023-08-25T09:44:21.117 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 108.00 MB 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 109.00 MB 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.118 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 110.00 MB 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 111.00 MB 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 112.00 MB 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.119 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 113.00 MB 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 114.00 MB 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.120 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 115.00 MB 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 116.00 MB 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 117.00 MB 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.121 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 118.00 MB 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 119.00 MB 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 120.00 MB 2023-08-25T09:44:21.122 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 121.00 MB 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 122.00 MB 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.123 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 123.00 MB 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 124.00 MB 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 125.00 MB 2023-08-25T09:44:21.124 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 126.00 MB 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 127.00 MB 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.125 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 128.00 MB 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 129.00 MB 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 130.00 MB 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.126 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 131.00 MB 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 132.00 MB 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.127 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 133.00 MB 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 134.00 MB 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 135.00 MB 2023-08-25T09:44:21.128 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 136.00 MB 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 137.00 MB 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.129 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 138.00 MB 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 139.00 MB 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 140.00 MB 2023-08-25T09:44:21.130 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 141.00 MB 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 142.00 MB 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.131 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 143.00 MB 2023-08-25T09:44:21.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.132 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.132 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 144.00 MB 2023-08-25T09:44:21.132 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.132 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.133 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 145.00 MB 2023-08-25T09:44:21.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.133 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.133 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 146.00 MB 2023-08-25T09:44:21.133 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.133 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 147.00 MB 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 148.00 MB 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.134 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 149.00 MB 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 150.00 MB 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 151.00 MB 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.135 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.136 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 152.00 MB 2023-08-25T09:44:21.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.136 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.136 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 153.00 MB 2023-08-25T09:44:21.136 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 154.00 MB 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 155.00 MB 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.137 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 156.00 MB 2023-08-25T09:44:21.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.138 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.138 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 157.00 MB 2023-08-25T09:44:21.138 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.138 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.138 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 158.00 MB 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 159.00 MB 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 160.00 MB 2023-08-25T09:44:21.139 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.328 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 161.00 MB 2023-08-25T09:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 162.00 MB 2023-08-25T09:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.329 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.330 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 163.00 MB 2023-08-25T09:44:22.330 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.330 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.330 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 164.00 MB 2023-08-25T09:44:22.330 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.330 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 165.00 MB 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 166.00 MB 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.331 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 167.00 MB 2023-08-25T09:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 168.00 MB 2023-08-25T09:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.332 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 169.00 MB 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 170.00 MB 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 171.00 MB 2023-08-25T09:44:22.333 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.334 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.334 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 172.00 MB 2023-08-25T09:44:22.334 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 173.00 MB 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 174.00 MB 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.335 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.336 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 175.00 MB 2023-08-25T09:44:22.336 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.336 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.336 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 176.00 MB 2023-08-25T09:44:22.336 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 177.00 MB 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 178.00 MB 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.337 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 179.00 MB 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 180.00 MB 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.338 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 181.00 MB 2023-08-25T09:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 182.00 MB 2023-08-25T09:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.339 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 183.00 MB 2023-08-25T09:44:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 184.00 MB 2023-08-25T09:44:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.340 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 185.00 MB 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 186.00 MB 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 187.00 MB 2023-08-25T09:44:22.341 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 188.00 MB 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 189.00 MB 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.342 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 190.00 MB 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 191.00 MB 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 192.00 MB 2023-08-25T09:44:22.343 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.344 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.344 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 193.00 MB 2023-08-25T09:44:22.344 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.344 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.344 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 194.00 MB 2023-08-25T09:44:22.344 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 195.00 MB 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 196.00 MB 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.345 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.346 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 197.00 MB 2023-08-25T09:44:22.346 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.346 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 198.00 MB 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 199.00 MB 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.347 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 200.00 MB 2023-08-25T09:44:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 201.00 MB 2023-08-25T09:44:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.348 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 202.00 MB 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 203.00 MB 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.349 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 204.00 MB 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 205.00 MB 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 206.00 MB 2023-08-25T09:44:22.350 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 207.00 MB 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 208.00 MB 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.351 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 209.00 MB 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 210.00 MB 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.352 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 211.00 MB 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 212.00 MB 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 213.00 MB 2023-08-25T09:44:22.353 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.354 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.354 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 214.00 MB 2023-08-25T09:44:22.354 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.354 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.354 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 215.00 MB 2023-08-25T09:44:22.355 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.355 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.355 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 216.00 MB 2023-08-25T09:44:22.355 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.355 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 217.00 MB 2023-08-25T09:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 218.00 MB 2023-08-25T09:44:22.356 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.357 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.357 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 219.00 MB 2023-08-25T09:44:22.357 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.357 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.357 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 220.00 MB 2023-08-25T09:44:22.358 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.358 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.358 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 221.00 MB 2023-08-25T09:44:22.358 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.358 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.358 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 222.00 MB 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 223.00 MB 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 224.00 MB 2023-08-25T09:44:22.359 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 225.00 MB 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 226.00 MB 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.360 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 227.00 MB 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 228.00 MB 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 229.00 MB 2023-08-25T09:44:22.361 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 230.00 MB 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 231.00 MB 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.362 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 232.00 MB 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 233.00 MB 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 234.00 MB 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.363 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 235.00 MB 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 236.00 MB 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.364 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 237.00 MB 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 238.00 MB 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 239.00 MB 2023-08-25T09:44:22.365 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.366 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:22.366 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 240.00 MB 2023-08-25T09:44:22.366 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:22.366 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.444 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 241.00 MB 2023-08-25T09:44:23.445 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.446 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.446 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 242.00 MB 2023-08-25T09:44:23.446 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.446 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.446 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 243.00 MB 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 244.00 MB 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 245.00 MB 2023-08-25T09:44:23.447 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.448 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.448 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 246.00 MB 2023-08-25T09:44:23.448 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.448 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.448 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 247.00 MB 2023-08-25T09:44:23.448 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 248.00 MB 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 249.00 MB 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.449 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 250.00 MB 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 251.00 MB 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.450 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 252.00 MB 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 253.00 MB 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 254.00 MB 2023-08-25T09:44:23.451 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 255.00 MB 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 256.00 MB 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 257.00 MB 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 258.00 MB 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.453 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 259.00 MB 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 260.00 MB 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 261.00 MB 2023-08-25T09:44:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 262.00 MB 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 263.00 MB 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.455 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 264.00 MB 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 265.00 MB 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.456 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 266.00 MB 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 267.00 MB 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 268.00 MB 2023-08-25T09:44:23.457 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 269.00 MB 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 270.00 MB 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.458 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 271.00 MB 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 272.00 MB 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 273.00 MB 2023-08-25T09:44:23.459 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 274.00 MB 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 275.00 MB 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.460 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 276.00 MB 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 277.00 MB 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.461 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 278.00 MB 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 279.00 MB 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 280.00 MB 2023-08-25T09:44:23.462 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 281.00 MB 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 282.00 MB 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.463 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 283.00 MB 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 284.00 MB 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.464 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 285.00 MB 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 286.00 MB 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 287.00 MB 2023-08-25T09:44:23.465 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 288.00 MB 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 289.00 MB 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.466 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 290.00 MB 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 291.00 MB 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 292.00 MB 2023-08-25T09:44:23.467 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 293.00 MB 2023-08-25T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 294.00 MB 2023-08-25T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 295.00 MB 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 296.00 MB 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.469 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 297.00 MB 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 298.00 MB 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 299.00 MB 2023-08-25T09:44:23.470 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 300.00 MB 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 301.00 MB 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.471 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 302.00 MB 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 303.00 MB 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 304.00 MB 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 305.00 MB 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 306.00 MB 2023-08-25T09:44:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 307.00 MB 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 308.00 MB 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 309.00 MB 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 310.00 MB 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 311.00 MB 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 312.00 MB 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 313.00 MB 2023-08-25T09:44:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 314.00 MB 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 315.00 MB 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.477 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 316.00 MB 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 317.00 MB 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 318.00 MB 2023-08-25T09:44:23.478 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 319.00 MB 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 320.00 MB 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:23.479 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:23.480 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 321.00 MB 2023-08-25T09:44:24.550 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.550 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 322.00 MB 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 323.00 MB 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.551 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 324.00 MB 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 325.00 MB 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 326.00 MB 2023-08-25T09:44:24.552 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 327.00 MB 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 328.00 MB 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.553 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 329.00 MB 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 330.00 MB 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.554 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 331.00 MB 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 332.00 MB 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 333.00 MB 2023-08-25T09:44:24.555 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.556 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.556 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 334.00 MB 2023-08-25T09:44:24.556 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.556 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.557 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 335.00 MB 2023-08-25T09:44:24.557 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.557 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.557 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 336.00 MB 2023-08-25T09:44:24.558 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.558 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.558 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 337.00 MB 2023-08-25T09:44:24.558 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.558 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 338.00 MB 2023-08-25T09:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 339.00 MB 2023-08-25T09:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.559 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 340.00 MB 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 341.00 MB 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.560 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 342.00 MB 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 343.00 MB 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 344.00 MB 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.561 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.562 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 345.00 MB 2023-08-25T09:44:24.562 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.562 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.562 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 346.00 MB 2023-08-25T09:44:24.562 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.562 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 347.00 MB 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 348.00 MB 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.563 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 349.00 MB 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 350.00 MB 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 351.00 MB 2023-08-25T09:44:24.564 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 352.00 MB 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 353.00 MB 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.565 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 354.00 MB 2023-08-25T09:44:24.566 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.566 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.566 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 355.00 MB 2023-08-25T09:44:24.566 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.566 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.566 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 356.00 MB 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 357.00 MB 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 358.00 MB 2023-08-25T09:44:24.567 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 359.00 MB 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 360.00 MB 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.568 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 361.00 MB 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 362.00 MB 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 363.00 MB 2023-08-25T09:44:24.569 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 364.00 MB 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 365.00 MB 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.570 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 366.00 MB 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 367.00 MB 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 368.00 MB 2023-08-25T09:44:24.571 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.572 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.572 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 369.00 MB 2023-08-25T09:44:24.572 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.572 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.572 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 370.00 MB 2023-08-25T09:44:24.572 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.573 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.573 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 371.00 MB 2023-08-25T09:44:24.573 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.574 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.574 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 372.00 MB 2023-08-25T09:44:24.574 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.575 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.575 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 373.00 MB 2023-08-25T09:44:24.575 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.575 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.576 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 374.00 MB 2023-08-25T09:44:24.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.576 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.576 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 375.00 MB 2023-08-25T09:44:24.576 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.577 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.577 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 376.00 MB 2023-08-25T09:44:24.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.577 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.577 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 377.00 MB 2023-08-25T09:44:24.577 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.578 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.578 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 378.00 MB 2023-08-25T09:44:24.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.578 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.578 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 379.00 MB 2023-08-25T09:44:24.578 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.579 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.579 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 380.00 MB 2023-08-25T09:44:24.579 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.579 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.579 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 381.00 MB 2023-08-25T09:44:24.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.580 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.580 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 382.00 MB 2023-08-25T09:44:24.580 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.581 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.581 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 383.00 MB 2023-08-25T09:44:24.581 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.581 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.582 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 384.00 MB 2023-08-25T09:44:24.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.582 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.582 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 385.00 MB 2023-08-25T09:44:24.582 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 386.00 MB 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 387.00 MB 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.583 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 388.00 MB 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 389.00 MB 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 390.00 MB 2023-08-25T09:44:24.584 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 391.00 MB 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 392.00 MB 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.585 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 393.00 MB 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 394.00 MB 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 395.00 MB 2023-08-25T09:44:24.586 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 396.00 MB 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 397.00 MB 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.587 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 398.00 MB 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 399.00 MB 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 400.00 MB 2023-08-25T09:44:24.588 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:24.589 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:24.589 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 401.00 MB 2023-08-25T09:44:26.506 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:44:26.504+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-08-25T09:44:40.838 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.838 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 402.00 MB 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 403.00 MB 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.839 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 404.00 MB 2023-08-25T09:44:40.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.840 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.840 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 405.00 MB 2023-08-25T09:44:40.840 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.840 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.840 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 406.00 MB 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 407.00 MB 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 408.00 MB 2023-08-25T09:44:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.842 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.842 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 409.00 MB 2023-08-25T09:44:40.842 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.842 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.842 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 410.00 MB 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 411.00 MB 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 412.00 MB 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.843 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.844 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 413.00 MB 2023-08-25T09:44:40.844 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.844 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.844 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 414.00 MB 2023-08-25T09:44:40.844 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.844 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 415.00 MB 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 416.00 MB 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 417.00 MB 2023-08-25T09:44:40.845 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 418.00 MB 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 419.00 MB 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.846 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 420.00 MB 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 421.00 MB 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.847 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 422.00 MB 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 423.00 MB 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 424.00 MB 2023-08-25T09:44:40.848 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.849 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.849 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 425.00 MB 2023-08-25T09:44:40.849 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.849 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.849 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 426.00 MB 2023-08-25T09:44:40.849 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.850 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.850 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 427.00 MB 2023-08-25T09:44:40.850 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.850 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.851 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 428.00 MB 2023-08-25T09:44:40.851 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.851 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.851 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 429.00 MB 2023-08-25T09:44:40.851 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.852 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.852 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 430.00 MB 2023-08-25T09:44:40.852 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.852 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.852 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 431.00 MB 2023-08-25T09:44:40.853 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.853 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.853 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 432.00 MB 2023-08-25T09:44:40.853 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.853 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.853 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 433.00 MB 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 434.00 MB 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 435.00 MB 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.854 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 436.00 MB 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 437.00 MB 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.855 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 438.00 MB 2023-08-25T09:44:40.856 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.856 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.856 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 439.00 MB 2023-08-25T09:44:40.857 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.858 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.858 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 440.00 MB 2023-08-25T09:44:40.858 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.858 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.858 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 441.00 MB 2023-08-25T09:44:40.858 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.859 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.859 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 442.00 MB 2023-08-25T09:44:40.859 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.859 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.859 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 443.00 MB 2023-08-25T09:44:40.859 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 444.00 MB 2023-08-25T09:44:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 445.00 MB 2023-08-25T09:44:40.860 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.861 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.861 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 446.00 MB 2023-08-25T09:44:40.861 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.861 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.861 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 447.00 MB 2023-08-25T09:44:40.862 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.862 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.862 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 448.00 MB 2023-08-25T09:44:40.862 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.862 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.863 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 449.00 MB 2023-08-25T09:44:40.863 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.863 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.863 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 450.00 MB 2023-08-25T09:44:40.863 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.863 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 451.00 MB 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 452.00 MB 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.864 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 453.00 MB 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 454.00 MB 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 455.00 MB 2023-08-25T09:44:40.865 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 456.00 MB 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 457.00 MB 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.866 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 458.00 MB 2023-08-25T09:44:40.867 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.867 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.867 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 459.00 MB 2023-08-25T09:44:40.867 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.867 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.867 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 460.00 MB 2023-08-25T09:44:40.868 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.868 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.868 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 461.00 MB 2023-08-25T09:44:40.868 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.868 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.868 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 462.00 MB 2023-08-25T09:44:40.869 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.869 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.869 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 463.00 MB 2023-08-25T09:44:40.869 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.870 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.870 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 464.00 MB 2023-08-25T09:44:40.871 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.871 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.871 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 465.00 MB 2023-08-25T09:44:40.871 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.871 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.871 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 466.00 MB 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 467.00 MB 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 468.00 MB 2023-08-25T09:44:40.872 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.873 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.873 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 469.00 MB 2023-08-25T09:44:40.873 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.873 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 470.00 MB 2023-08-25T09:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 471.00 MB 2023-08-25T09:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.874 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.875 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 472.00 MB 2023-08-25T09:44:40.875 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.875 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.875 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 473.00 MB 2023-08-25T09:44:40.875 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.875 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.880 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 474.00 MB 2023-08-25T09:44:40.880 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 475.00 MB 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 476.00 MB 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.881 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 477.00 MB 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 478.00 MB 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:44:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 479.00 MB 2023-08-25T09:44:40.883 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:44:40.883 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:44:40.883 INFO:teuthology.orchestra.run.smithi070.stdout:Reached fullness after 480.00 MB 2023-08-25T09:44:40.883 INFO:teuthology.orchestra.run.smithi070.stdout:calling close 2023-08-25T09:44:40.883 INFO:teuthology.orchestra.run.smithi070.stdout:close() did not raise error 2023-08-25T09:44:40.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:44:40.886 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:44:40.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:44:41.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:44:41.132 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:44:41.133 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:44:41.133 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:44:41.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:44:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.413+0000 7fc875596640 1 Processor -- start 2023-08-25T09:44:41.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.414+0000 7fc875596640 1 -- start start 2023-08-25T09:44:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.415+0000 7fc875596640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc87006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.415+0000 7fc875596640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc87006b2b0 0x7fc87015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.415+0000 7fc875596640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8700b1b60 con 0x7fc87015c6d0 2023-08-25T09:44:41.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.415+0000 7fc875596640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87006bd80 con 0x7fc87006b2b0 2023-08-25T09:44:41.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.415+0000 7fc875596640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87006b7c0 con 0x7fc87015e830 2023-08-25T09:44:41.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc87006b2b0 0x7fc87015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc87006b2b0 0x7fc87015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50132/0 (socket says 172.21.15.70:50132) 2023-08-25T09:44:41.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 -- 172.21.15.70:0/3910935721 learned_addr learned my addr 172.21.15.70:0/3910935721 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:41.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86effd640 1 --2- 172.21.15.70:0/3910935721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc87006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 -- 172.21.15.70:0/3910935721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 msgr2=0x7fc87006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:41.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 --2- 172.21.15.70:0/3910935721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc87006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 -- 172.21.15.70:0/3910935721 >> v1:172.21.15.155:6789/0 conn(0x7fc87015c6d0 legacy=0x7fc8700b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.416+0000 7fc86e7fc640 1 -- 172.21.15.70:0/3910935721 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87006baa0 con 0x7fc87006b2b0 2023-08-25T09:44:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.417+0000 7fc86e7fc640 1 --2- 172.21.15.70:0/3910935721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc87015c190 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc858009870 tx=0x7fc85802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed54d25302602d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc86dffb640 1 -- 172.21.15.70:0/3910935721 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858030840 con 0x7fc87006b2b0 2023-08-25T09:44:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc86dffb640 1 -- 172.21.15.70:0/3910935721 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8580309a0 con 0x7fc87006b2b0 2023-08-25T09:44:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc86dffb640 1 -- 172.21.15.70:0/3910935721 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858030d30 con 0x7fc87006b2b0 2023-08-25T09:44:41.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc875596640 1 -- 172.21.15.70:0/3910935721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 msgr2=0x7fc87015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc875596640 1 --2- 172.21.15.70:0/3910935721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc87015c190 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc858009870 tx=0x7fc85802b980 comp rx=0 tx=0).stop 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc875596640 1 -- 172.21.15.70:0/3910935721 shutdown_connections 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc875596640 1 --2- 172.21.15.70:0/3910935721 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc87015c190 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc875596640 1 --2- 172.21.15.70:0/3910935721 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc87006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.418+0000 7fc875596640 1 -- 172.21.15.70:0/3910935721 >> 172.21.15.70:0/3910935721 conn(0x7fc870069850 msgr2=0x7fc870069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.419+0000 7fc875596640 1 -- 172.21.15.70:0/3910935721 shutdown_connections 2023-08-25T09:44:41.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.419+0000 7fc875596640 1 -- 172.21.15.70:0/3910935721 wait complete. 2023-08-25T09:44:41.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.420+0000 7fc875596640 1 Processor -- start 2023-08-25T09:44:41.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.420+0000 7fc875596640 1 -- start start 2023-08-25T09:44:41.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.421+0000 7fc875596640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc8700c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.421+0000 7fc875596640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc87015c6d0 0x7fc870150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.421+0000 7fc875596640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc870151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.421+0000 7fc875596640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc870155b60 con 0x7fc87006b2b0 2023-08-25T09:44:41.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.421+0000 7fc875596640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8701eff40 con 0x7fc87015c6d0 2023-08-25T09:44:41.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.421+0000 7fc875596640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8700ca450 con 0x7fc87015e830 2023-08-25T09:44:41.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc8700c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc8700c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50134/0 (socket says 172.21.15.70:50134) 2023-08-25T09:44:41.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 -- 172.21.15.70:0/376751308 learned_addr learned my addr 172.21.15.70:0/376751308 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 -- 172.21.15.70:0/376751308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 msgr2=0x7fc870151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc870151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 -- 172.21.15.70:0/376751308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc87015c6d0 msgr2=0x7fc870150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc87015c6d0 0x7fc870150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 -- 172.21.15.70:0/376751308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580094d0 con 0x7fc87006b2b0 2023-08-25T09:44:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.422+0000 7fc86effd640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc8700c9f10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc860007e40 tx=0x7fc86000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:41.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.423+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860014620 con 0x7fc87006b2b0 2023-08-25T09:44:41.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.423+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86000ee90 con 0x7fc87006b2b0 2023-08-25T09:44:41.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.423+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8600149c0 con 0x7fc87006b2b0 2023-08-25T09:44:41.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.423+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc870154fe0 con 0x7fc87006b2b0 2023-08-25T09:44:41.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.423+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8701555a0 con 0x7fc87006b2b0 2023-08-25T09:44:41.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.424+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87004e420 con 0x7fc87006b2b0 2023-08-25T09:44:41.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.425+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc860014b20 con 0x7fc87006b2b0 2023-08-25T09:44:41.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.426+0000 7fc8677fe640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85006a050 0x7fc85006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.426+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc860088fe0 con 0x7fc87006b2b0 2023-08-25T09:44:41.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.426+0000 7fc86e7fc640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85006a050 0x7fc85006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.426+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7fc8500721a0 con 0x7fc87006b2b0 2023-08-25T09:44:41.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.427+0000 7fc86e7fc640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85006a050 0x7fc85006c510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc858036000 tx=0x7fc85803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:41.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.430+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc860054630 con 0x7fc87006b2b0 2023-08-25T09:44:41.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.621+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87006c620 con 0x7fc87006b2b0 2023-08-25T09:44:41.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.624+0000 7fc8677fe640 1 -- 172.21.15.70:0/376751308 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+18410 (secure 0 0 0) 0x7fc860057c50 con 0x7fc87006b2b0 2023-08-25T09:44:41.627 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:41.627 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:44:31.554429+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["backfillfull","exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:44:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85006a050 msgr2=0x7fc85006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85006a050 0x7fc85006c510 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc858036000 tx=0x7fc85803b040 comp rx=0 tx=0).stop 2023-08-25T09:44:41.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 msgr2=0x7fc8700c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc8700c9f10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc860007e40 tx=0x7fc86000e4b0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 shutdown_connections 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85006a050 0x7fc85006c510 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc87015e830 0x7fc870151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc87015c6d0 0x7fc870150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 --2- 172.21.15.70:0/376751308 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc87006b2b0 0x7fc8700c9f10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 >> 172.21.15.70:0/376751308 conn(0x7fc870069850 msgr2=0x7fc87015b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:41.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.626+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 shutdown_connections 2023-08-25T09:44:41.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.627+0000 7fc875596640 1 -- 172.21.15.70:0/376751308 wait complete. 2023-08-25T09:44:41.648 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:43:52.155883+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:43:52.872267+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:44:41.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:44:41.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.859+0000 7f43933d5640 1 Processor -- start 2023-08-25T09:44:41.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.859+0000 7f43933d5640 1 -- start start 2023-08-25T09:44:41.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.860+0000 7f43933d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f438c15ae50 0x7f438c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.860+0000 7f43933d5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0cdc20 0x7f438c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.860+0000 7f43933d5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f438c0ce060 0x7f438c069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.860+0000 7f43933d5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438c0cc5f0 con 0x7f438c15ae50 2023-08-25T09:44:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.860+0000 7f43933d5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438c070650 con 0x7f438c0ce060 2023-08-25T09:44:41.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.860+0000 7f43933d5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c070930 con 0x7f438c0cdc20 2023-08-25T09:44:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f438c15ae50 0x7f438c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f438c15ae50 0x7f438c0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50144/0 (socket says 172.21.15.70:50144) 2023-08-25T09:44:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 -- 172.21.15.70:0/331088378 learned_addr learned my addr 172.21.15.70:0/331088378 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:41.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4390970640 1 --2- 172.21.15.70:0/331088378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0cdc20 0x7f438c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391972640 1 --2- 172.21.15.70:0/331088378 >> v2:172.21.15.155:3300/0 conn(0x7f438c0ce060 0x7f438c069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 -- 172.21.15.70:0/331088378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0cdc20 msgr2=0x7f438c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:41.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 --2- 172.21.15.70:0/331088378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0cdc20 0x7f438c069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 -- 172.21.15.70:0/331088378 >> v2:172.21.15.155:3300/0 conn(0x7f438c0ce060 msgr2=0x7f438c069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 --2- 172.21.15.70:0/331088378 >> v2:172.21.15.155:3300/0 conn(0x7f438c0ce060 0x7f438c069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.861+0000 7f4391171640 1 -- 172.21.15.70:0/331088378 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c070c10 con 0x7f438c15ae50 2023-08-25T09:44:41.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.862+0000 7f4391171640 1 --2- 172.21.15.70:0/331088378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c15ae50 0x7f438c0c2110 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f438c0cc030 tx=0x7f438802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6467e60430fb826 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.862+0000 7f4383fff640 1 -- 172.21.15.70:0/331088378 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388030860 con 0x7f438c15ae50 2023-08-25T09:44:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f4383fff640 1 -- 172.21.15.70:0/331088378 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43880309c0 con 0x7f438c15ae50 2023-08-25T09:44:41.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f4383fff640 1 -- 172.21.15.70:0/331088378 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388030d50 con 0x7f438c15ae50 2023-08-25T09:44:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 -- 172.21.15.70:0/331088378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c15ae50 msgr2=0x7f438c0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 --2- 172.21.15.70:0/331088378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c15ae50 0x7f438c0c2110 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f438c0cc030 tx=0x7f438802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 -- 172.21.15.70:0/331088378 shutdown_connections 2023-08-25T09:44:41.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 --2- 172.21.15.70:0/331088378 >> v2:172.21.15.155:3300/0 conn(0x7f438c0ce060 0x7f438c069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 --2- 172.21.15.70:0/331088378 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c0cdc20 0x7f438c069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 --2- 172.21.15.70:0/331088378 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c15ae50 0x7f438c0c2110 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 -- 172.21.15.70:0/331088378 >> 172.21.15.70:0/331088378 conn(0x7f438c0cb0d0 msgr2=0x7f438c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:41.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 -- 172.21.15.70:0/331088378 shutdown_connections 2023-08-25T09:44:41.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.863+0000 7f43933d5640 1 -- 172.21.15.70:0/331088378 wait complete. 2023-08-25T09:44:41.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.865+0000 7f43933d5640 1 Processor -- start 2023-08-25T09:44:41.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.865+0000 7f43933d5640 1 -- start start 2023-08-25T09:44:41.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f43933d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 0x7f438c1e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f43933d5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c0ce060 0x7f438c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f43933d5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c15ae50 0x7f438c1526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f43933d5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c0c8650 con 0x7f438c0cdc20 2023-08-25T09:44:41.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f43933d5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f438c1e46b0 con 0x7f438c0ce060 2023-08-25T09:44:41.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f43933d5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f438c1e3aa0 con 0x7f438c15ae50 2023-08-25T09:44:41.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.866+0000 7f4391171640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 0x7f438c1e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 0x7f438c1e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50152/0 (socket says 172.21.15.70:50152) 2023-08-25T09:44:41.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 -- 172.21.15.70:0/3674026190 learned_addr learned my addr 172.21.15.70:0/3674026190 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:41.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 -- 172.21.15.70:0/3674026190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c15ae50 msgr2=0x7f438c1526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c15ae50 0x7f438c1526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 -- 172.21.15.70:0/3674026190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c0ce060 msgr2=0x7f438c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:41.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c0ce060 0x7f438c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:41.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 -- 172.21.15.70:0/3674026190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880094d0 con 0x7f438c0cdc20 2023-08-25T09:44:41.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.867+0000 7f4391171640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 0x7f438c1e4b40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f438802b9e0 tx=0x7f4388004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:41.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.868+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388030850 con 0x7f438c0cdc20 2023-08-25T09:44:41.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.868+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4388039d90 con 0x7f438c0cdc20 2023-08-25T09:44:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.868+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4388043650 con 0x7f438c0cdc20 2023-08-25T09:44:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.868+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f438c1e3d80 con 0x7f438c0cdc20 2023-08-25T09:44:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.868+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438c1e4340 con 0x7f438c0cdc20 2023-08-25T09:44:41.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.869+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438c066fb0 con 0x7f438c0cdc20 2023-08-25T09:44:41.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.870+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f438804c3b0 con 0x7f438c0cdc20 2023-08-25T09:44:41.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.871+0000 7f4381ffb640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f437006a050 0x7f437006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:41.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.871+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43880aae80 con 0x7f438c0cdc20 2023-08-25T09:44:41.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.871+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f43700721a0 con 0x7f438c0cdc20 2023-08-25T09:44:41.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.871+0000 7f4390970640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f437006a050 0x7f437006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:41.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.872+0000 7f4390970640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f437006a050 0x7f437006c510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f438c1e5320 tx=0x7f437c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:41.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:41.875+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4388048020 con 0x7f438c0cdc20 2023-08-25T09:44:42.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.063+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f438c152c00 con 0x7f438c0cdc20 2023-08-25T09:44:42.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.064+0000 7f4381ffb640 1 -- 172.21.15.70:0/3674026190 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4388034070 con 0x7f438c0cdc20 2023-08-25T09:44:42.067 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:44:42.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.066+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f437006a050 msgr2=0x7f437006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.066+0000 7f43933d5640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f437006a050 0x7f437006c510 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f438c1e5320 tx=0x7f437c008040 comp rx=0 tx=0).stop 2023-08-25T09:44:42.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.066+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 msgr2=0x7f438c1e4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.066+0000 7f43933d5640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 0x7f438c1e4b40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f438802b9e0 tx=0x7f4388004730 comp rx=0 tx=0).stop 2023-08-25T09:44:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 shutdown_connections 2023-08-25T09:44:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f437006a050 0x7f437006c510 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f438c15ae50 0x7f438c1526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f438c0ce060 0x7f438c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 --2- 172.21.15.70:0/3674026190 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f438c0cdc20 0x7f438c1e4b40 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 >> 172.21.15.70:0/3674026190 conn(0x7f438c0cb0d0 msgr2=0x7f438c153580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:42.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 shutdown_connections 2023-08-25T09:44:42.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.067+0000 7f43933d5640 1 -- 172.21.15.70:0/3674026190 wait complete. 2023-08-25T09:44:42.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:44:42.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.299+0000 7f7d1e192640 1 Processor -- start 2023-08-25T09:44:42.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.299+0000 7f7d1e192640 1 -- start start 2023-08-25T09:44:42.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.300+0000 7f7d1e192640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d1815ae50 0x7f7d180c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.300+0000 7f7d1e192640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180cdc20 0x7f7d18069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.300+0000 7f7d1e192640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7d180ce060 0x7f7d18069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.300+0000 7f7d1e192640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d180cc5f0 con 0x7f7d1815ae50 2023-08-25T09:44:42.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.300+0000 7f7d1e192640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d18070650 con 0x7f7d180ce060 2023-08-25T09:44:42.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.300+0000 7f7d1e192640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18070930 con 0x7f7d180cdc20 2023-08-25T09:44:42.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d1815ae50 0x7f7d180c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7d1815ae50 0x7f7d180c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50164/0 (socket says 172.21.15.70:50164) 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 -- 172.21.15.70:0/262429233 learned_addr learned my addr 172.21.15.70:0/262429233 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d16ffd640 1 --2- 172.21.15.70:0/262429233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180cdc20 0x7f7d18069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d17fff640 1 --2- 172.21.15.70:0/262429233 >> v2:172.21.15.155:3300/0 conn(0x7f7d180ce060 0x7f7d18069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 -- 172.21.15.70:0/262429233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180cdc20 msgr2=0x7f7d18069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 --2- 172.21.15.70:0/262429233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180cdc20 0x7f7d18069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 -- 172.21.15.70:0/262429233 >> v2:172.21.15.155:3300/0 conn(0x7f7d180ce060 msgr2=0x7f7d18069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:42.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 --2- 172.21.15.70:0/262429233 >> v2:172.21.15.155:3300/0 conn(0x7f7d180ce060 0x7f7d18069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.301+0000 7f7d177fe640 1 -- 172.21.15.70:0/262429233 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18070c10 con 0x7f7d1815ae50 2023-08-25T09:44:42.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.302+0000 7f7d177fe640 1 --2- 172.21.15.70:0/262429233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d180c2110 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7d180cc030 tx=0x7f7d0002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93c5d80cf4f8a05 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:42.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.303+0000 7f7d167fc640 1 -- 172.21.15.70:0/262429233 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00030860 con 0x7f7d1815ae50 2023-08-25T09:44:42.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.303+0000 7f7d167fc640 1 -- 172.21.15.70:0/262429233 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d000309c0 con 0x7f7d1815ae50 2023-08-25T09:44:42.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.303+0000 7f7d167fc640 1 -- 172.21.15.70:0/262429233 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d00030d50 con 0x7f7d1815ae50 2023-08-25T09:44:42.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 -- 172.21.15.70:0/262429233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 msgr2=0x7f7d180c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 --2- 172.21.15.70:0/262429233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d180c2110 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f7d180cc030 tx=0x7f7d0002b9a0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 -- 172.21.15.70:0/262429233 shutdown_connections 2023-08-25T09:44:42.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 --2- 172.21.15.70:0/262429233 >> v2:172.21.15.155:3300/0 conn(0x7f7d180ce060 0x7f7d18069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 --2- 172.21.15.70:0/262429233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180cdc20 0x7f7d18069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 --2- 172.21.15.70:0/262429233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d180c2110 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 -- 172.21.15.70:0/262429233 >> 172.21.15.70:0/262429233 conn(0x7f7d180cb0d0 msgr2=0x7f7d180cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:42.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 -- 172.21.15.70:0/262429233 shutdown_connections 2023-08-25T09:44:42.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.304+0000 7f7d1e192640 1 -- 172.21.15.70:0/262429233 wait complete. 2023-08-25T09:44:42.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.306+0000 7f7d1e192640 1 Processor -- start 2023-08-25T09:44:42.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.306+0000 7f7d1e192640 1 -- start start 2023-08-25T09:44:42.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d1e192640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d180cdc20 0x7f7d18147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d1e192640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180ce060 0x7f7d180c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d1e192640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d181f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d1e192640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180b2750 con 0x7f7d1815ae50 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d1e192640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180b2a30 con 0x7f7d180cdc20 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d1e192640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180b2d10 con 0x7f7d180ce060 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d181f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d17fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d181f2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50174/0 (socket says 172.21.15.70:50174) 2023-08-25T09:44:42.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d17fff640 1 -- 172.21.15.70:0/1333004342 learned_addr learned my addr 172.21.15.70:0/1333004342 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:42.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d16ffd640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180ce060 0x7f7d180c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.307+0000 7f7d177fe640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d180cdc20 0x7f7d18147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d17fff640 1 -- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180ce060 msgr2=0x7f7d180c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d17fff640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180ce060 0x7f7d180c4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d17fff640 1 -- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d180cdc20 msgr2=0x7f7d18147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d17fff640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d180cdc20 0x7f7d18147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d17fff640 1 -- 172.21.15.70:0/1333004342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000094d0 con 0x7f7d1815ae50 2023-08-25T09:44:42.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d177fe640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d180cdc20 0x7f7d18147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:44:42.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.308+0000 7f7d17fff640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d181f2640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d5d0 tx=0x7f7d1000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:42.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.309+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d100125f0 con 0x7f7d1815ae50 2023-08-25T09:44:42.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.309+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d10012750 con 0x7f7d1815ae50 2023-08-25T09:44:42.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.309+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180b2ff0 con 0x7f7d1815ae50 2023-08-25T09:44:42.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.309+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10012b00 con 0x7f7d1815ae50 2023-08-25T09:44:42.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.309+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d181481d0 con 0x7f7d1815ae50 2023-08-25T09:44:42.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.310+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d180b3510 con 0x7f7d1815ae50 2023-08-25T09:44:42.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.311+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d10012c60 con 0x7f7d1815ae50 2023-08-25T09:44:42.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.312+0000 7f7d0ffff640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7cf806a050 0x7f7cf806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.312+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d10088a20 con 0x7f7d1815ae50 2023-08-25T09:44:42.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.313+0000 7f7d177fe640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7cf806a050 0x7f7cf806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.313+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f7cf80721a0 con 0x7f7d1815ae50 2023-08-25T09:44:42.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.313+0000 7f7d177fe640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7cf806a050 0x7f7cf806c510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d00036000 tx=0x7f7d0003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:42.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.318+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7d10088030 con 0x7f7d1815ae50 2023-08-25T09:44:42.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.510+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7d18066fb0 con 0x7f7d1815ae50 2023-08-25T09:44:42.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.513+0000 7f7d0ffff640 1 -- 172.21.15.70:0/1333004342 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7d10088030 con 0x7f7d1815ae50 2023-08-25T09:44:42.516 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:44:42.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7cf806a050 msgr2=0x7f7cf806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7cf806a050 0x7f7cf806c510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d00036000 tx=0x7f7d0003b040 comp rx=0 tx=0).stop 2023-08-25T09:44:42.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 msgr2=0x7f7d181f2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d181f2640 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000d5d0 tx=0x7f7d1000da90 comp rx=0 tx=0).stop 2023-08-25T09:44:42.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 shutdown_connections 2023-08-25T09:44:42.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7cf806a050 0x7f7cf806c510 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7d1815ae50 0x7f7d181f2640 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7d180ce060 0x7f7d180c4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 --2- 172.21.15.70:0/1333004342 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7d180cdc20 0x7f7d18147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 >> 172.21.15.70:0/1333004342 conn(0x7f7d180cb0d0 msgr2=0x7f7d180aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:42.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 shutdown_connections 2023-08-25T09:44:42.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.516+0000 7f7d1e192640 1 -- 172.21.15.70:0/1333004342 wait complete. 2023-08-25T09:44:42.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:44:42.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.742+0000 7f7116be8640 1 Processor -- start 2023-08-25T09:44:42.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.742+0000 7f7116be8640 1 -- start start 2023-08-25T09:44:42.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.744+0000 7f7116be8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f71100d1ba0 0x7f71100cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.744+0000 7f7116be8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71100b7140 con 0x7f711015ccd0 2023-08-25T09:44:42.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.744+0000 7f7116be8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71100b7420 con 0x7f71100cf980 2023-08-25T09:44:42.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.744+0000 7f7116be8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71100cfe10 con 0x7f71100d1ba0 2023-08-25T09:44:42.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.744+0000 7f7114984640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f711015ccd0 0x7f71100b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52308/0 (socket says 172.21.15.70:52308) 2023-08-25T09:44:42.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.744+0000 7f7114984640 1 -- 172.21.15.70:0/912261430 learned_addr learned my addr 172.21.15.70:0/912261430 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:42.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338147178 0 0) 0x7f71100b7140 con 0x7f711015ccd0 2023-08-25T09:44:42.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7100003520 con 0x7f711015ccd0 2023-08-25T09:44:42.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710ffff640 1 --2- 172.21.15.70:0/912261430 >> v2:172.21.15.155:3300/0 conn(0x7f71100d1ba0 0x7f71100cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668907086 0 0) 0x7f7100003520 con 0x7f711015ccd0 2023-08-25T09:44:42.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 >> v2:172.21.15.155:3300/0 conn(0x7f71100d1ba0 msgr2=0x7f71100cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:42.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 --2- 172.21.15.70:0/912261430 >> v2:172.21.15.155:3300/0 conn(0x7f71100d1ba0 0x7f71100cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 >> v1:172.21.15.155:6789/0 conn(0x7f71100cf980 legacy=0x7f71100b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71100b7700 con 0x7f711015ccd0 2023-08-25T09:44:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7104001670 con 0x7f711015ccd0 2023-08-25T09:44:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 >> v1:172.21.15.70:6789/0 conn(0x7f711015ccd0 legacy=0x7f71100b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7100003970 0x7f7100003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71000047c0 0x7f7100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100005180 0x7f710000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100cfe10 con 0x7f71000047c0 2023-08-25T09:44:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710000c090 con 0x7f7100005180 2023-08-25T09:44:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.745+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710000c250 con 0x7f7100003970 2023-08-25T09:44:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f710ffff640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71000047c0 0x7f7100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f7114984640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7100003970 0x7f7100003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f710ffff640 1 -- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7100003970 msgr2=0x7f7100003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f710ffff640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7100003970 0x7f7100003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f710ffff640 1 -- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100005180 msgr2=0x7f710000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f7115185640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100005180 0x7f710000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f710ffff640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7100005180 0x7f710000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f7115185640 1 -- 172.21.15.70:0/912261430 reap_dead start 2023-08-25T09:44:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.746+0000 7f710ffff640 1 -- 172.21.15.70:0/912261430 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7100003520 con 0x7f71000047c0 2023-08-25T09:44:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.747+0000 7f710ffff640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71000047c0 0x7f7100004c40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ba30 tx=0x7f71000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d13a9f5f7a86296 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.747+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc032640 con 0x7f71000047c0 2023-08-25T09:44:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.747+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc0327a0 con 0x7f71000047c0 2023-08-25T09:44:42.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.747+0000 7f710f7fe640 1 -- 172.21.15.70:0/912261430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc032b30 con 0x7f71000047c0 2023-08-25T09:44:42.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 -- 172.21.15.70:0/912261430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71000047c0 msgr2=0x7f7100004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71000047c0 0x7f7100004c40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f70fc00ba30 tx=0x7f71000057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 -- 172.21.15.70:0/912261430 shutdown_connections 2023-08-25T09:44:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 --2- 172.21.15.70:0/912261430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71000047c0 0x7f7100004c40 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 -- 172.21.15.70:0/912261430 >> 172.21.15.70:0/912261430 conn(0x7f7110070260 msgr2=0x7f7110070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 -- 172.21.15.70:0/912261430 shutdown_connections 2023-08-25T09:44:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.748+0000 7f7116be8640 1 -- 172.21.15.70:0/912261430 wait complete. 2023-08-25T09:44:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.750+0000 7f7116be8640 1 Processor -- start 2023-08-25T09:44:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.750+0000 7f7116be8640 1 -- start start 2023-08-25T09:44:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7116be8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71000047c0 0x7f711014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7116be8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71100c7b90 0x7f71101f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7116be8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 0x7f71100c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7116be8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71100c8c40 con 0x7f71100c82a0 2023-08-25T09:44:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7116be8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7110150440 con 0x7f71100c7b90 2023-08-25T09:44:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7116be8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7110150720 con 0x7f71000047c0 2023-08-25T09:44:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7115185640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 0x7f71100c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7115185640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 0x7f71100c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50192/0 (socket says 172.21.15.70:50192) 2023-08-25T09:44:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7115185640 1 -- 172.21.15.70:0/1703471042 learned_addr learned my addr 172.21.15.70:0/1703471042 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7114984640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71000047c0 0x7f711014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.751+0000 7f7115185640 1 -- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71000047c0 msgr2=0x7f711014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7115185640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71000047c0 0x7f711014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7115185640 1 -- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71100c7b90 msgr2=0x7f71101f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7115185640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71100c7b90 0x7f71101f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7115185640 1 -- 172.21.15.70:0/1703471042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110150a00 con 0x7f71100c82a0 2023-08-25T09:44:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7114984640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71000047c0 0x7f711014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7115185640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 0x7f71100c8700 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f71100d0590 tx=0x7f710800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108012800 con 0x7f71100c82a0 2023-08-25T09:44:42.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108012960 con 0x7f71100c82a0 2023-08-25T09:44:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108012d10 con 0x7f71100c82a0 2023-08-25T09:44:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7110150ce0 con 0x7f71100c82a0 2023-08-25T09:44:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.752+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711015de10 con 0x7f71100c82a0 2023-08-25T09:44:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.753+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711015e0c0 con 0x7f71100c82a0 2023-08-25T09:44:42.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.754+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f710801e460 con 0x7f71100c82a0 2023-08-25T09:44:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.755+0000 7f710d7fa640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70f406a050 0x7f70f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.755+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71080883a0 con 0x7f71100c82a0 2023-08-25T09:44:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.755+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f70f40721a0 con 0x7f71100c82a0 2023-08-25T09:44:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.755+0000 7f7114984640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70f406a050 0x7f70f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.756+0000 7f7114984640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70f406a050 0x7f70f406c510 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7104001eb0 tx=0x7f7104009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.759+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f711015e0c0 con 0x7f71100c82a0 2023-08-25T09:44:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.945+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f711015e0c0 con 0x7f71100c82a0 2023-08-25T09:44:42.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.946+0000 7f710d7fa640 1 -- 172.21.15.70:0/1703471042 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f711015e0c0 con 0x7f71100c82a0 2023-08-25T09:44:42.949 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:44:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70f406a050 msgr2=0x7f70f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70f406a050 0x7f70f406c510 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7104001eb0 tx=0x7f7104009040 comp rx=0 tx=0).stop 2023-08-25T09:44:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 msgr2=0x7f71100c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 0x7f71100c8700 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f71100d0590 tx=0x7f710800d960 comp rx=0 tx=0).stop 2023-08-25T09:44:42.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 shutdown_connections 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70f406a050 0x7f70f406c510 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71100c82a0 0x7f71100c8700 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71100c7b90 0x7f71101f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 --2- 172.21.15.70:0/1703471042 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71000047c0 0x7f711014fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.948+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 >> 172.21.15.70:0/1703471042 conn(0x7f7110070260 msgr2=0x7f711015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.949+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 shutdown_connections 2023-08-25T09:44:42.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:42.949+0000 7f7116be8640 1 -- 172.21.15.70:0/1703471042 wait complete. 2023-08-25T09:44:42.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:44:43.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.172+0000 7f61fb96a640 1 Processor -- start 2023-08-25T09:44:43.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.173+0000 7f61fb96a640 1 -- start start 2023-08-25T09:44:43.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.174+0000 7f61fb96a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 0x7f61f40c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.174+0000 7f61fb96a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40cc5f0 con 0x7f61f40cdc20 2023-08-25T09:44:43.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.174+0000 7f61fb96a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f4070650 con 0x7f61f4069f60 2023-08-25T09:44:43.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.174+0000 7f61fb96a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4070930 con 0x7f61f415ae50 2023-08-25T09:44:43.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61f8f05640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f61f40cdc20 0x7f61f4069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52310/0 (socket says 172.21.15.70:52310) 2023-08-25T09:44:43.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61f8f05640 1 -- 172.21.15.70:0/2035249036 learned_addr learned my addr 172.21.15.70:0/2035249036 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61f9706640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 0x7f61f40c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285677041 0 0) 0x7f61f40cc5f0 con 0x7f61f40cdc20 2023-08-25T09:44:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e0003520 con 0x7f61f40cdc20 2023-08-25T09:44:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3832177756 0 0) 0x7f61e0003520 con 0x7f61f40cdc20 2023-08-25T09:44:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 msgr2=0x7f61f40c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.175+0000 7f61ebfff640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 0x7f61f40c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 >> v1:172.21.15.155:6789/0 conn(0x7f61f4069f60 legacy=0x7f61f40ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4070c10 con 0x7f61f40cdc20 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f61e4001670 con 0x7f61f40cdc20 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 >> v1:172.21.15.70:6789/0 conn(0x7f61f40cdc20 legacy=0x7f61f4069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61e00039a0 0x7f61e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00047c0 0x7f61e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 0x7f61e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4070930 con 0x7f61e00039a0 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e000bac0 con 0x7f61e00047c0 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e000bc50 con 0x7f61f415ae50 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61f9706640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61e00039a0 0x7f61e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61f9706640 1 -- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 msgr2=0x7f61e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61f9706640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61f415ae50 0x7f61e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61f9706640 1 -- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00047c0 msgr2=0x7f61e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61f9706640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61e00047c0 0x7f61e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.176+0000 7f61f9706640 1 -- 172.21.15.70:0/2035249036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e0003520 con 0x7f61e00039a0 2023-08-25T09:44:43.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.177+0000 7f61f9706640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61e00039a0 0x7f61e0003da0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009ac0 tx=0x7f61e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7f5573c389717e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:43.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f002c680 con 0x7f61e00039a0 2023-08-25T09:44:43.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f002c7e0 con 0x7f61e00039a0 2023-08-25T09:44:43.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61ebfff640 1 -- 172.21.15.70:0/2035249036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f002cb70 con 0x7f61e00039a0 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61fb96a640 1 -- 172.21.15.70:0/2035249036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61e00039a0 msgr2=0x7f61e0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61fb96a640 1 --2- 172.21.15.70:0/2035249036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61e00039a0 0x7f61e0003da0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009ac0 tx=0x7f61e00057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61f9f07640 1 -- 172.21.15.70:0/2035249036 reap_dead start 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61fb96a640 1 -- 172.21.15.70:0/2035249036 shutdown_connections 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61fb96a640 1 -- 172.21.15.70:0/2035249036 >> 172.21.15.70:0/2035249036 conn(0x7f61f40cb0d0 msgr2=0x7f61f40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61fb96a640 1 -- 172.21.15.70:0/2035249036 shutdown_connections 2023-08-25T09:44:43.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.178+0000 7f61fb96a640 1 -- 172.21.15.70:0/2035249036 wait complete. 2023-08-25T09:44:43.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.180+0000 7f61fb96a640 1 Processor -- start 2023-08-25T09:44:43.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.180+0000 7f61fb96a640 1 -- start start 2023-08-25T09:44:43.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61fb96a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00039a0 0x7f61f40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61fb96a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61f415b880 0x7f61f4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61fb96a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 0x7f61f415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61fb96a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415cdc0 con 0x7f61f4146ea0 2023-08-25T09:44:43.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61fb96a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415bd20 con 0x7f61f415b880 2023-08-25T09:44:43.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61fb96a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f415c000 con 0x7f61e00039a0 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61f9f07640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 0x7f61f415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61f9f07640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 0x7f61f415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50210/0 (socket says 172.21.15.70:50210) 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.181+0000 7f61f9f07640 1 -- 172.21.15.70:0/1752097921 learned_addr learned my addr 172.21.15.70:0/1752097921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61f9f07640 1 -- 172.21.15.70:0/1752097921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00039a0 msgr2=0x7f61f40ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61f9f07640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00039a0 0x7f61f40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61f9f07640 1 -- 172.21.15.70:0/1752097921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61f415b880 msgr2=0x7f61f4159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61f9f07640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61f415b880 0x7f61f4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61f9f07640 1 -- 172.21.15.70:0/1752097921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f415c2e0 con 0x7f61f4146ea0 2023-08-25T09:44:43.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61f9f07640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 0x7f61f415a070 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f61f40c51a0 tx=0x7f61ec00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:43.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec007070 con 0x7f61f4146ea0 2023-08-25T09:44:43.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec0071f0 con 0x7f61f4146ea0 2023-08-25T09:44:43.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec0075a0 con 0x7f61f4146ea0 2023-08-25T09:44:43.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.182+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f415c5c0 con 0x7f61f4146ea0 2023-08-25T09:44:43.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.183+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f40ca6a0 con 0x7f61f4146ea0 2023-08-25T09:44:43.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.184+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f415cae0 con 0x7f61f4146ea0 2023-08-25T09:44:43.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.185+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f61ec007700 con 0x7f61f4146ea0 2023-08-25T09:44:43.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.186+0000 7f61e9ffb640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f61d806a050 0x7f61d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.186+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61ec087ac0 con 0x7f61f4146ea0 2023-08-25T09:44:43.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.186+0000 7f61f9706640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f61d806a050 0x7f61d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.186+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f61d80721a0 con 0x7f61f4146ea0 2023-08-25T09:44:43.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.187+0000 7f61f9706640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f61d806a050 0x7f61d806c510 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f61f0036000 tx=0x7f61f002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:43.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.190+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f61ec017060 con 0x7f61f4146ea0 2023-08-25T09:44:43.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.377+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61f40701c0 con 0x7f61f4146ea0 2023-08-25T09:44:43.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.378+0000 7f61e9ffb640 1 -- 172.21.15.70:0/1752097921 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61f415cae0 con 0x7f61f4146ea0 2023-08-25T09:44:43.381 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:44:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.380+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f61d806a050 msgr2=0x7f61d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.380+0000 7f61fb96a640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f61d806a050 0x7f61d806c510 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f61f0036000 tx=0x7f61f002e000 comp rx=0 tx=0).stop 2023-08-25T09:44:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.380+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 msgr2=0x7f61f415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.380+0000 7f61fb96a640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 0x7f61f415a070 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f61f40c51a0 tx=0x7f61ec00c980 comp rx=0 tx=0).stop 2023-08-25T09:44:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 shutdown_connections 2023-08-25T09:44:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f61d806a050 0x7f61d806c510 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f61f4146ea0 0x7f61f415a070 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f61f415b880 0x7f61f4159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 --2- 172.21.15.70:0/1752097921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61e00039a0 0x7f61f40ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 >> 172.21.15.70:0/1752097921 conn(0x7f61f40cb0d0 msgr2=0x7f61f40af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:43.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 shutdown_connections 2023-08-25T09:44:43.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.381+0000 7f61fb96a640 1 -- 172.21.15.70:0/1752097921 wait complete. 2023-08-25T09:44:43.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:44:43.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.613+0000 7ff98d942640 1 Processor -- start 2023-08-25T09:44:43.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.613+0000 7ff98d942640 1 -- start start 2023-08-25T09:44:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.615+0000 7ff98d942640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9880b5060 0x7ff9880cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.615+0000 7ff98d942640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880d0fe0 con 0x7ff9880d0b00 2023-08-25T09:44:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.615+0000 7ff98d942640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9880d12c0 con 0x7ff98815cb60 2023-08-25T09:44:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.615+0000 7ff98d942640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9880d15a0 con 0x7ff9880b5060 2023-08-25T09:44:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.615+0000 7ff9877fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff9880d0b00 0x7ff9880b5ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52314/0 (socket says 172.21.15.70:52314) 2023-08-25T09:44:43.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.615+0000 7ff9877fe640 1 -- 172.21.15.70:0/3023673157 learned_addr learned my addr 172.21.15.70:0/3023673157 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:43.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098139814 0 0) 0x7ff9880d0fe0 con 0x7ff9880d0b00 2023-08-25T09:44:43.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3023673157 >> v2:172.21.15.155:3300/0 conn(0x7ff9880b5060 0x7ff9880cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff974003520 con 0x7ff9880d0b00 2023-08-25T09:44:43.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 75556435 0 0) 0x7ff974003520 con 0x7ff9880d0b00 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 >> v2:172.21.15.155:3300/0 conn(0x7ff9880b5060 msgr2=0x7ff9880cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 --2- 172.21.15.70:0/3023673157 >> v2:172.21.15.155:3300/0 conn(0x7ff9880b5060 0x7ff9880cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 >> v1:172.21.15.155:6789/0 conn(0x7ff98815cb60 legacy=0x7ff9880b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9880d1880 con 0x7ff9880d0b00 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff9800019f0 con 0x7ff9880d0b00 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 >> v1:172.21.15.70:6789/0 conn(0x7ff9880d0b00 legacy=0x7ff9880b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9740039a0 0x7ff974003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740051d0 0x7ff97400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880d15a0 con 0x7ff9740047c0 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97400c100 con 0x7ff9740051d0 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.616+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97400c2c0 con 0x7ff9740039a0 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff97ffff640 1 -- 172.21.15.70:0/3023673157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9740039a0 msgr2=0x7ff974003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9740039a0 0x7ff974003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff97ffff640 1 -- 172.21.15.70:0/3023673157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740051d0 msgr2=0x7ff97400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:43.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9740051d0 0x7ff97400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff97ffff640 1 -- 172.21.15.70:0/3023673157 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff974003520 con 0x7ff9740047c0 2023-08-25T09:44:43.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.617+0000 7ff9877fe640 1 -- 172.21.15.70:0/3023673157 reap_dead start 2023-08-25T09:44:43.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.618+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff97000ba00 tx=0x7ff9740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf34071264ab542a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:43.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970032620 con 0x7ff9740047c0 2023-08-25T09:44:43.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff970032780 con 0x7ff9740047c0 2023-08-25T09:44:43.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff9867fc640 1 -- 172.21.15.70:0/3023673157 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970032b10 con 0x7ff9740047c0 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 -- 172.21.15.70:0/3023673157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 msgr2=0x7ff974004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff97000ba00 tx=0x7ff9740057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 -- 172.21.15.70:0/3023673157 shutdown_connections 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 --2- 172.21.15.70:0/3023673157 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff974004c40 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 -- 172.21.15.70:0/3023673157 >> 172.21.15.70:0/3023673157 conn(0x7ff988070080 msgr2=0x7ff988070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 -- 172.21.15.70:0/3023673157 shutdown_connections 2023-08-25T09:44:43.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.619+0000 7ff98d942640 1 -- 172.21.15.70:0/3023673157 wait complete. 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.621+0000 7ff98d942640 1 Processor -- start 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.621+0000 7ff98d942640 1 -- start start 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff98d942640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff98814b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff98d942640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9880a9490 0x7ff9880a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff98d942640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9881f5900 0x7ff9881f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff98d942640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98814bac0 con 0x7ff9740047c0 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff98d942640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9880a9e50 con 0x7ff9881f5900 2023-08-25T09:44:43.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff98d942640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98814bca0 con 0x7ff9880a9490 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff98814b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff986ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff98814b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50218/0 (socket says 172.21.15.70:50218) 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.622+0000 7ff986ffd640 1 -- 172.21.15.70:0/3689851580 learned_addr learned my addr 172.21.15.70:0/3689851580 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff986ffd640 1 -- 172.21.15.70:0/3689851580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9880a9490 msgr2=0x7ff9880a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff986ffd640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9880a9490 0x7ff9880a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff986ffd640 1 -- 172.21.15.70:0/3689851580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9881f5900 msgr2=0x7ff9881f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff986ffd640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9881f5900 0x7ff9881f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff986ffd640 1 -- 172.21.15.70:0/3689851580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98814bf80 con 0x7ff9740047c0 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff986ffd640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff98814b580 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c430 tx=0x7ff97800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9780052d0 con 0x7ff9740047c0 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff978005450 con 0x7ff9740047c0 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978003830 con 0x7ff9740047c0 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.623+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98814c260 con 0x7ff9740047c0 2023-08-25T09:44:43.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.624+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98814ca60 con 0x7ff9740047c0 2023-08-25T09:44:43.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.625+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98814bde0 con 0x7ff9740047c0 2023-08-25T09:44:43.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.626+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff978003990 con 0x7ff9740047c0 2023-08-25T09:44:43.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.626+0000 7ff984ff9640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff96806a080 0x7ff96806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:43.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.627+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff9780875d0 con 0x7ff9740047c0 2023-08-25T09:44:43.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.627+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff96806a080 0x7ff96806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:43.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.627+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7ff9680721d0 con 0x7ff9740047c0 2023-08-25T09:44:43.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.628+0000 7ff97ffff640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff96806a080 0x7ff96806c540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff9740057a0 tx=0x7ff970032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:43.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.631+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff98814bde0 con 0x7ff9740047c0 2023-08-25T09:44:43.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.833+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff98814bde0 con 0x7ff9740047c0 2023-08-25T09:44:43.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.834+0000 7ff984ff9640 1 -- 172.21.15.70:0/3689851580 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v99) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff98814bde0 con 0x7ff9740047c0 2023-08-25T09:44:43.836 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:44:43.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff96806a080 msgr2=0x7ff96806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff96806a080 0x7ff96806c540 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff9740057a0 tx=0x7ff970032000 comp rx=0 tx=0).stop 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 msgr2=0x7ff98814b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff98814b580 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff97800c430 tx=0x7ff97800c8f0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 shutdown_connections 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff96806a080 0x7ff96806c540 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9881f5900 0x7ff9881f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9880a9490 0x7ff9880a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 --2- 172.21.15.70:0/3689851580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9740047c0 0x7ff98814b580 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:43.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.836+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 >> 172.21.15.70:0/3689851580 conn(0x7ff988070080 msgr2=0x7ff98815c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:43.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.837+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 shutdown_connections 2023-08-25T09:44:43.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:43.837+0000 7ff98d942640 1 -- 172.21.15.70:0/3689851580 wait complete. 2023-08-25T09:44:43.858 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:44:43.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:43.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:44:43.950 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[106653]: fuse finished with error 0 and tester_r 0 2023-08-25T09:44:43.958 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:44:43.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:43.959 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:44:44.140 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:44:44.140 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:44:44.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:44:44.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.423+0000 7f3856d42640 1 Processor -- start 2023-08-25T09:44:44.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.423+0000 7f3856d42640 1 -- start start 2023-08-25T09:44:44.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f3856d42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f385015e830 0x7f385006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f3856d42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f385006b2b0 0x7f385015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f3856d42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 0x7f38500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f3856d42640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385006bd80 con 0x7f385006b2b0 2023-08-25T09:44:44.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f3856d42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38500b1b60 con 0x7f385015e830 2023-08-25T09:44:44.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f3856d42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385006b7c0 con 0x7f385015c6d0 2023-08-25T09:44:44.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f384ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f385006b2b0 0x7f385015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f384ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f385006b2b0 0x7f385015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50222/0 (socket says 172.21.15.70:50222) 2023-08-25T09:44:44.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.425+0000 7f384ffff640 1 -- 172.21.15.70:0/2627096550 learned_addr learned my addr 172.21.15.70:0/2627096550 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:44.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.426+0000 7f384ffff640 1 -- 172.21.15.70:0/2627096550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 msgr2=0x7f38500b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:44.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.426+0000 7f384ffff640 1 --2- 172.21.15.70:0/2627096550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 0x7f38500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.426+0000 7f384ffff640 1 -- 172.21.15.70:0/2627096550 >> v2:172.21.15.155:3300/0 conn(0x7f385015e830 msgr2=0x7f385006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:44.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.426+0000 7f3854ade640 1 --2- 172.21.15.70:0/2627096550 >> v2:172.21.15.155:3300/0 conn(0x7f385015e830 0x7f385006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.426+0000 7f384ffff640 1 --2- 172.21.15.70:0/2627096550 >> v2:172.21.15.155:3300/0 conn(0x7f385015e830 0x7f385006ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.426+0000 7f384ffff640 1 -- 172.21.15.70:0/2627096550 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385006baa0 con 0x7f385006b2b0 2023-08-25T09:44:44.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.427+0000 7f384ffff640 1 --2- 172.21.15.70:0/2627096550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f385015c190 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f385006bf10 tx=0x7f383c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca300b7b07dc2cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:44.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.427+0000 7f384f7fe640 1 -- 172.21.15.70:0/2627096550 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c031870 con 0x7f385006b2b0 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.427+0000 7f384f7fe640 1 -- 172.21.15.70:0/2627096550 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f383c0319d0 con 0x7f385006b2b0 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.427+0000 7f384f7fe640 1 -- 172.21.15.70:0/2627096550 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f383c031d60 con 0x7f385006b2b0 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 -- 172.21.15.70:0/2627096550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 msgr2=0x7f385015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 --2- 172.21.15.70:0/2627096550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f385015c190 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f385006bf10 tx=0x7f383c02b910 comp rx=0 tx=0).stop 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 -- 172.21.15.70:0/2627096550 shutdown_connections 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 --2- 172.21.15.70:0/2627096550 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 0x7f38500b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 --2- 172.21.15.70:0/2627096550 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f385015c190 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 --2- 172.21.15.70:0/2627096550 >> v2:172.21.15.155:3300/0 conn(0x7f385015e830 0x7f385006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 -- 172.21.15.70:0/2627096550 >> 172.21.15.70:0/2627096550 conn(0x7f3850069850 msgr2=0x7f3850069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:44.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 -- 172.21.15.70:0/2627096550 shutdown_connections 2023-08-25T09:44:44.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.428+0000 7f3856d42640 1 -- 172.21.15.70:0/2627096550 wait complete. 2023-08-25T09:44:44.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.429+0000 7f3856d42640 1 Processor -- start 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.430+0000 7f3856d42640 1 -- start start 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3856d42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f38500ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3856d42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 0x7f38501f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3856d42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f385015e830 0x7f38501f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3856d42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500ad690 con 0x7f385006b2b0 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3856d42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385014a020 con 0x7f385015e830 2023-08-25T09:44:44.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3856d42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385014a300 con 0x7f385015c6d0 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f38500ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f38500ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50234/0 (socket says 172.21.15.70:50234) 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 -- 172.21.15.70:0/3939053865 learned_addr learned my addr 172.21.15.70:0/3939053865 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 -- 172.21.15.70:0/3939053865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 msgr2=0x7f38501f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 0x7f38501f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 -- 172.21.15.70:0/3939053865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f385015e830 msgr2=0x7f38501f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f385015e830 0x7f38501f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.431+0000 7f3854ade640 1 -- 172.21.15.70:0/3939053865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0094d0 con 0x7f385006b2b0 2023-08-25T09:44:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.432+0000 7f3854ade640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f38500ad150 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f384400c7f0 tx=0x7f384400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.432+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844004220 con 0x7f385006b2b0 2023-08-25T09:44:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.432+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844004380 con 0x7f385006b2b0 2023-08-25T09:44:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.432+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844007840 con 0x7f385006b2b0 2023-08-25T09:44:44.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.432+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385014a5e0 con 0x7f385006b2b0 2023-08-25T09:44:44.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.432+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385014aba0 con 0x7f385006b2b0 2023-08-25T09:44:44.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.433+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385014ad90 con 0x7f385006b2b0 2023-08-25T09:44:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.434+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3844007bb0 con 0x7f385006b2b0 2023-08-25T09:44:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.435+0000 7f384d7fa640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f383406a050 0x7f383406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.435+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38440884d0 con 0x7f385006b2b0 2023-08-25T09:44:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.435+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f38340721a0 con 0x7f385006b2b0 2023-08-25T09:44:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.435+0000 7f384ffff640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f383406a050 0x7f383406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.436+0000 7f384ffff640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f383406a050 0x7f383406c510 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f383c036000 tx=0x7f383c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:44.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.440+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3844053bb0 con 0x7f385006b2b0 2023-08-25T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.680+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f385004e420 con 0x7f385006b2b0 2023-08-25T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.683+0000 7f384d7fa640 1 -- 172.21.15.70:0/3939053865 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+18410 (secure 0 0 0) 0x7f384401e030 con 0x7f385006b2b0 2023-08-25T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":99,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:44:31.554429+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["backfillfull","exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.685+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f383406a050 msgr2=0x7f383406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.685+0000 7f3856d42640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f383406a050 0x7f383406c510 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f383c036000 tx=0x7f383c030000 comp rx=0 tx=0).stop 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.685+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 msgr2=0x7f38500ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.685+0000 7f3856d42640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f38500ad150 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f384400c7f0 tx=0x7f384400ccb0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 shutdown_connections 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f383406a050 0x7f383406c510 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f385015e830 0x7f38501f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f385015c6d0 0x7f38501f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 --2- 172.21.15.70:0/3939053865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f385006b2b0 0x7f38500ad150 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 >> 172.21.15.70:0/3939053865 conn(0x7f3850069850 msgr2=0x7f38500b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 shutdown_connections 2023-08-25T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.686+0000 7f3856d42640 1 -- 172.21.15.70:0/3939053865 wait complete. 2023-08-25T09:44:44.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:43:52.155883+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:43:52.872267+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:44:44.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:44:44.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.926+0000 7ff2fc992640 1 Processor -- start 2023-08-25T09:44:44.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.926+0000 7ff2fc992640 1 -- start start 2023-08-25T09:44:44.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2fc992640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 0x7ff2f8069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2fc992640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff2f8069d90 0x7ff2f80ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2fc992640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f80c2550 con 0x7ff2f815ae50 2023-08-25T09:44:44.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2fc992640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f806a220 con 0x7ff2f8069d90 2023-08-25T09:44:44.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2fc992640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f80ce330 con 0x7ff2f80c2110 2023-08-25T09:44:44.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2f659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff2f815ae50 0x7ff2f80cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52316/0 (socket says 172.21.15.70:52316) 2023-08-25T09:44:44.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.928+0000 7ff2f659c640 1 -- 172.21.15.70:0/502186021 learned_addr learned my addr 172.21.15.70:0/502186021 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:44.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2effff640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 0x7ff2f8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f6d9d640 1 --2- 172.21.15.70:0/502186021 >> v2:172.21.15.155:3300/0 conn(0x7ff2f8069d90 0x7ff2f80ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101152805 0 0) 0x7ff2f80c2550 con 0x7ff2f815ae50 2023-08-25T09:44:44.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e4003520 con 0x7ff2f815ae50 2023-08-25T09:44:44.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1611899285 0 0) 0x7ff2e4003520 con 0x7ff2f815ae50 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 msgr2=0x7ff2f8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 0x7ff2f8069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 >> v2:172.21.15.155:3300/0 conn(0x7ff2f8069d90 msgr2=0x7ff2f80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 --2- 172.21.15.70:0/502186021 >> v2:172.21.15.155:3300/0 conn(0x7ff2f8069d90 0x7ff2f80ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f80cc5f0 con 0x7ff2f815ae50 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff2e8001670 con 0x7ff2f815ae50 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 >> v1:172.21.15.70:6789/0 conn(0x7ff2f815ae50 legacy=0x7ff2f80cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2e4003970 0x7ff2e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 0x7ff2e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e4004d00 0x7ff2e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f806a220 con 0x7ff2e4003970 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f80ce330 con 0x7ff2e4004d00 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.929+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e400b980 con 0x7ff2f80c2110 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2f659c640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2e4003970 0x7ff2e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2effff640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 0x7ff2e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2f659c640 1 -- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 msgr2=0x7ff2e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:44.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2f659c640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2f80c2110 0x7ff2e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2f659c640 1 -- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e4004d00 msgr2=0x7ff2e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:44.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2f659c640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2e4004d00 0x7ff2e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.930+0000 7ff2f659c640 1 -- 172.21.15.70:0/502186021 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e4003520 con 0x7ff2e4003970 2023-08-25T09:44:44.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.931+0000 7ff2f659c640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2e4003970 0x7ff2e4003d90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800c240 tx=0x7ff2e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c6f0db7ac4b685 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.931+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e802cb00 con 0x7ff2e4003970 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e802cc60 con 0x7ff2e4003970 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2f5d9b640 1 -- 172.21.15.70:0/502186021 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8005af0 con 0x7ff2e4003970 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2fc992640 1 -- 172.21.15.70:0/502186021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2e4003970 msgr2=0x7ff2e4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2fc992640 1 --2- 172.21.15.70:0/502186021 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2e4003970 0x7ff2e4003d90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800c240 tx=0x7ff2e40057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2f6d9d640 1 -- 172.21.15.70:0/502186021 reap_dead start 2023-08-25T09:44:44.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2fc992640 1 -- 172.21.15.70:0/502186021 shutdown_connections 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2fc992640 1 -- 172.21.15.70:0/502186021 >> 172.21.15.70:0/502186021 conn(0x7ff2f80cb0d0 msgr2=0x7ff2f80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2fc992640 1 -- 172.21.15.70:0/502186021 shutdown_connections 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.932+0000 7ff2fc992640 1 -- 172.21.15.70:0/502186021 wait complete. 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.934+0000 7ff2fc992640 1 Processor -- start 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.934+0000 7ff2fc992640 1 -- start start 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.935+0000 7ff2fc992640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e4003970 0x7ff2f81534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.935+0000 7ff2fc992640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2f8156900 0x7ff2f80c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.935+0000 7ff2fc992640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 0x7ff2f8141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.935+0000 7ff2fc992640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8142380 con 0x7ff2f81419b0 2023-08-25T09:44:44.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.935+0000 7ff2fc992640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8153a00 con 0x7ff2f8156900 2023-08-25T09:44:44.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.935+0000 7ff2fc992640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8156db0 con 0x7ff2e4003970 2023-08-25T09:44:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 0x7ff2f8141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 0x7ff2f8141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50248/0 (socket says 172.21.15.70:50248) 2023-08-25T09:44:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 -- 172.21.15.70:0/1381891775 learned_addr learned my addr 172.21.15.70:0/1381891775 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:44.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f659c640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e4003970 0x7ff2f81534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2effff640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2f8156900 0x7ff2f80c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 -- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e4003970 msgr2=0x7ff2f81534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e4003970 0x7ff2f81534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 -- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2f8156900 msgr2=0x7ff2f80c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:44.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2f8156900 0x7ff2f80c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:44.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 -- 172.21.15.70:0/1381891775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f813f9c0 con 0x7ff2f81419b0 2023-08-25T09:44:44.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f659c640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e4003970 0x7ff2f81534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.936+0000 7ff2f6d9d640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 0x7ff2f8141e40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff2f806d1f0 tx=0x7ff2f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.937+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0010040 con 0x7ff2f81419b0 2023-08-25T09:44:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.937+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f000bce0 con 0x7ff2f81419b0 2023-08-25T09:44:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.937+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0014640 con 0x7ff2f81419b0 2023-08-25T09:44:44.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.937+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f813fca0 con 0x7ff2f81419b0 2023-08-25T09:44:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.937+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f8157090 con 0x7ff2f81419b0 2023-08-25T09:44:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.938+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f81401c0 con 0x7ff2f81419b0 2023-08-25T09:44:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.939+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff2f00147a0 con 0x7ff2f81419b0 2023-08-25T09:44:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.940+0000 7ff2eeffd640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2d806a080 0x7ff2d806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.940+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2f0088910 con 0x7ff2f81419b0 2023-08-25T09:44:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.940+0000 7ff2f659c640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2d806a080 0x7ff2d806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:44.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.940+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7ff2d8072250 con 0x7ff2f81419b0 2023-08-25T09:44:44.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.941+0000 7ff2f659c640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2d806a080 0x7ff2d806c540 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8034000 tx=0x7ff2e8007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:44.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:44.944+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff2f81401c0 con 0x7ff2f81419b0 2023-08-25T09:44:45.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.136+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2f81401c0 con 0x7ff2f81419b0 2023-08-25T09:44:45.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.136+0000 7ff2eeffd640 1 -- 172.21.15.70:0/1381891775 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2f81401c0 con 0x7ff2f81419b0 2023-08-25T09:44:45.139 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:44:45.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2d806a080 msgr2=0x7ff2d806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2d806a080 0x7ff2d806c540 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8034000 tx=0x7ff2e8007000 comp rx=0 tx=0).stop 2023-08-25T09:44:45.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 msgr2=0x7ff2f8141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 0x7ff2f8141e40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7ff2f806d1f0 tx=0x7ff2f000b480 comp rx=0 tx=0).stop 2023-08-25T09:44:45.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 shutdown_connections 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2d806a080 0x7ff2d806c540 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2f81419b0 0x7ff2f8141e40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2f8156900 0x7ff2f80c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 --2- 172.21.15.70:0/1381891775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2e4003970 0x7ff2f81534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 >> 172.21.15.70:0/1381891775 conn(0x7ff2f80cb0d0 msgr2=0x7ff2f80ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 shutdown_connections 2023-08-25T09:44:45.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.139+0000 7ff2fc992640 1 -- 172.21.15.70:0/1381891775 wait complete. 2023-08-25T09:44:45.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:44:45.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.379+0000 7f17f4396640 1 Processor -- start 2023-08-25T09:44:45.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.380+0000 7f17f4396640 1 -- start start 2023-08-25T09:44:45.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.381+0000 7f17f4396640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f17ec0b0960 0x7f17ec0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.381+0000 7f17f4396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec06b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.381+0000 7f17f4396640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17ec0cb710 con 0x7f17ec0ce110 2023-08-25T09:44:45.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.381+0000 7f17f4396640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17ec0cecb0 con 0x7f17ec0b0960 2023-08-25T09:44:45.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.381+0000 7f17f4396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0b1280 con 0x7f17ec06ae60 2023-08-25T09:44:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f1931640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f17ec0ce110 0x7f17ec0ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52328/0 (socket says 172.21.15.70:52328) 2023-08-25T09:44:45.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f1931640 1 -- 172.21.15.70:0/1727641046 learned_addr learned my addr 172.21.15.70:0/1727641046 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f2132640 1 --2- 172.21.15.70:0/1727641046 >> v2:172.21.15.70:3300/0 conn(0x7f17ec0b0960 0x7f17ec0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f2933640 1 --2- 172.21.15.70:0/1727641046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec06b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f2132640 1 -- 172.21.15.70:0/1727641046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 msgr2=0x7f17ec06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f2132640 1 --2- 172.21.15.70:0/1727641046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec06b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f2132640 1 -- 172.21.15.70:0/1727641046 >> v1:172.21.15.70:6789/0 conn(0x7f17ec0ce110 legacy=0x7f17ec0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f2132640 1 -- 172.21.15.70:0/1727641046 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0ccb50 con 0x7f17ec0b0960 2023-08-25T09:44:45.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.382+0000 7f17f1130640 1 -- 172.21.15.70:0/1727641046 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093904676 0 0) 0x7f17ec0cb710 con 0x7f17ec0ce110 2023-08-25T09:44:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.383+0000 7f17f2132640 1 --2- 172.21.15.70:0/1727641046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0b0960 0x7f17ec0b0d40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009d90 tx=0x7f17dc02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b57f31cc8d52f9e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f1130640 1 -- 172.21.15.70:0/1727641046 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc02cbc0 con 0x7f17ec0b0960 2023-08-25T09:44:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f1130640 1 -- 172.21.15.70:0/1727641046 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17dc02cd20 con 0x7f17ec0b0960 2023-08-25T09:44:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f1130640 1 -- 172.21.15.70:0/1727641046 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17dc034660 con 0x7f17ec0b0960 2023-08-25T09:44:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 -- 172.21.15.70:0/1727641046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0b0960 msgr2=0x7f17ec0b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 --2- 172.21.15.70:0/1727641046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0b0960 0x7f17ec0b0d40 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009d90 tx=0x7f17dc02be10 comp rx=0 tx=0).stop 2023-08-25T09:44:45.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 -- 172.21.15.70:0/1727641046 shutdown_connections 2023-08-25T09:44:45.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 --2- 172.21.15.70:0/1727641046 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec06b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 --2- 172.21.15.70:0/1727641046 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0b0960 0x7f17ec0b0d40 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 -- 172.21.15.70:0/1727641046 >> 172.21.15.70:0/1727641046 conn(0x7f17ec0cb960 msgr2=0x7f17ec0cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:45.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 -- 172.21.15.70:0/1727641046 shutdown_connections 2023-08-25T09:44:45.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.384+0000 7f17f4396640 1 -- 172.21.15.70:0/1727641046 wait complete. 2023-08-25T09:44:45.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.386+0000 7f17f4396640 1 Processor -- start 2023-08-25T09:44:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.386+0000 7f17f4396640 1 -- start start 2023-08-25T09:44:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f4396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec0c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f4396640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ec0b0960 0x7f17ec14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f4396640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 0x7f17ec09c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f4396640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec09bbc0 con 0x7f17ec0ce110 2023-08-25T09:44:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f4396640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec0c5740 con 0x7f17ec0b0960 2023-08-25T09:44:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f4396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17ec09b290 con 0x7f17ec06ae60 2023-08-25T09:44:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.387+0000 7f17f2933640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 0x7f17ec09c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 0x7f17ec09c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50266/0 (socket says 172.21.15.70:50266) 2023-08-25T09:44:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 -- 172.21.15.70:0/1467151542 learned_addr learned my addr 172.21.15.70:0/1467151542 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 -- 172.21.15.70:0/1467151542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 msgr2=0x7f17ec0c2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec0c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 -- 172.21.15.70:0/1467151542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ec0b0960 msgr2=0x7f17ec14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ec0b0960 0x7f17ec14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 -- 172.21.15.70:0/1467151542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc009a70 con 0x7f17ec0ce110 2023-08-25T09:44:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f2933640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 0x7f17ec09c050 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f17e400d800 tx=0x7f17e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e400b6c0 con 0x7f17ec0ce110 2023-08-25T09:44:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e400b820 con 0x7f17ec0ce110 2023-08-25T09:44:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e400bbd0 con 0x7f17ec0ce110 2023-08-25T09:44:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.388+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ec09b570 con 0x7f17ec0ce110 2023-08-25T09:44:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.389+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17ec14c7e0 con 0x7f17ec0ce110 2023-08-25T09:44:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.390+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17ec0c9a20 con 0x7f17ec0ce110 2023-08-25T09:44:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.391+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17e400bd30 con 0x7f17ec0ce110 2023-08-25T09:44:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.392+0000 7f17eaffd640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17d806a050 0x7f17d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.392+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17e4088b20 con 0x7f17ec0ce110 2023-08-25T09:44:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.392+0000 7f17f2132640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17d806a050 0x7f17d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.393+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f17d80721a0 con 0x7f17ec0ce110 2023-08-25T09:44:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.393+0000 7f17f2132640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17d806a050 0x7f17d806c510 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f17dc036000 tx=0x7f17dc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.396+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17e4054200 con 0x7f17ec0ce110 2023-08-25T09:44:45.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:44:45.439 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:44:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.584+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f17ec066fb0 con 0x7f17ec0ce110 2023-08-25T09:44:45.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.585+0000 7f17eaffd640 1 -- 172.21.15.70:0/1467151542 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v99) v1 ==== 92+0+10 (secure 0 0 0) 0x7f17e4058e60 con 0x7f17ec0ce110 2023-08-25T09:44:45.588 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:44:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17d806a050 msgr2=0x7f17d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17d806a050 0x7f17d806c510 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f17dc036000 tx=0x7f17dc02e000 comp rx=0 tx=0).stop 2023-08-25T09:44:45.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 msgr2=0x7f17ec09c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 0x7f17ec09c050 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f17e400d800 tx=0x7f17e400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 shutdown_connections 2023-08-25T09:44:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17d806a050 0x7f17d806c510 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17ec0ce110 0x7f17ec09c050 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17ec0b0960 0x7f17ec14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 --2- 172.21.15.70:0/1467151542 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17ec06ae60 0x7f17ec0c2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 >> 172.21.15.70:0/1467151542 conn(0x7f17ec0cb960 msgr2=0x7f17ec0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:45.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.587+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 shutdown_connections 2023-08-25T09:44:45.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.588+0000 7f17f4396640 1 -- 172.21.15.70:0/1467151542 wait complete. 2023-08-25T09:44:45.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:44:45.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.863+0000 7f515bd4e640 1 Processor -- start 2023-08-25T09:44:45.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.863+0000 7f515bd4e640 1 -- start start 2023-08-25T09:44:45.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.864+0000 7f515bd4e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f51540c2110 0x7f5154069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.864+0000 7f515bd4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5154069d90 0x7f51540ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.864+0000 7f515bd4e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540c2550 con 0x7f515415ae50 2023-08-25T09:44:45.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.864+0000 7f515bd4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515406a200 con 0x7f5154069d90 2023-08-25T09:44:45.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f515bd4e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51540ce330 con 0x7f51540c2110 2023-08-25T09:44:45.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f515a2eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5154069d90 0x7f51540ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f5159aea640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f515415ae50 0x7f51540cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52338/0 (socket says 172.21.15.70:52338) 2023-08-25T09:44:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f5159aea640 1 -- 172.21.15.70:0/2354419800 learned_addr learned my addr 172.21.15.70:0/2354419800 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f51592e9640 1 --2- 172.21.15.70:0/2354419800 >> v2:172.21.15.155:3300/0 conn(0x7f51540c2110 0x7f5154069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f515a2eb640 1 -- 172.21.15.70:0/2354419800 >> v2:172.21.15.155:3300/0 conn(0x7f51540c2110 msgr2=0x7f5154069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f515a2eb640 1 --2- 172.21.15.70:0/2354419800 >> v2:172.21.15.155:3300/0 conn(0x7f51540c2110 0x7f5154069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f5158ae8640 1 -- 172.21.15.70:0/2354419800 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500603977 0 0) 0x7f51540c2550 con 0x7f515415ae50 2023-08-25T09:44:45.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f515a2eb640 1 -- 172.21.15.70:0/2354419800 >> v1:172.21.15.70:6789/0 conn(0x7f515415ae50 legacy=0x7f51540cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f515a2eb640 1 -- 172.21.15.70:0/2354419800 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540cc5f0 con 0x7f5154069d90 2023-08-25T09:44:45.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.865+0000 7f51592e9640 1 --2- 172.21.15.70:0/2354419800 >> v2:172.21.15.155:3300/0 conn(0x7f51540c2110 0x7f5154069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.866+0000 7f515a2eb640 1 --2- 172.21.15.70:0/2354419800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154069d90 0x7f51540ccad0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f51540cb030 tx=0x7f514c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c05552ad449474 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.867+0000 7f5158ae8640 1 -- 172.21.15.70:0/2354419800 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c02cd00 con 0x7f5154069d90 2023-08-25T09:44:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.867+0000 7f5158ae8640 1 -- 172.21.15.70:0/2354419800 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c02ce60 con 0x7f5154069d90 2023-08-25T09:44:45.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.867+0000 7f5158ae8640 1 -- 172.21.15.70:0/2354419800 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c034660 con 0x7f5154069d90 2023-08-25T09:44:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.867+0000 7f515bd4e640 1 -- 172.21.15.70:0/2354419800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154069d90 msgr2=0x7f51540ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.867+0000 7f515bd4e640 1 --2- 172.21.15.70:0/2354419800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154069d90 0x7f51540ccad0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f51540cb030 tx=0x7f514c02bf50 comp rx=0 tx=0).stop 2023-08-25T09:44:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.868+0000 7f515bd4e640 1 -- 172.21.15.70:0/2354419800 shutdown_connections 2023-08-25T09:44:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.868+0000 7f515bd4e640 1 --2- 172.21.15.70:0/2354419800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5154069d90 0x7f51540ccad0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.868+0000 7f515bd4e640 1 --2- 172.21.15.70:0/2354419800 >> v2:172.21.15.155:3300/0 conn(0x7f51540c2110 0x7f5154069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.868+0000 7f515bd4e640 1 -- 172.21.15.70:0/2354419800 >> 172.21.15.70:0/2354419800 conn(0x7f51540cb0d0 msgr2=0x7f51540cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:45.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.868+0000 7f515bd4e640 1 -- 172.21.15.70:0/2354419800 shutdown_connections 2023-08-25T09:44:45.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.868+0000 7f515bd4e640 1 -- 172.21.15.70:0/2354419800 wait complete. 2023-08-25T09:44:45.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.869+0000 7f515bd4e640 1 Processor -- start 2023-08-25T09:44:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.869+0000 7f515bd4e640 1 -- start start 2023-08-25T09:44:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.870+0000 7f515bd4e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5154069d90 0x7f5154151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.870+0000 7f515bd4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51540c2110 0x7f51540adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.870+0000 7f515bd4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 0x7f51540b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.870+0000 7f515bd4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5154152370 con 0x7f515415ae50 2023-08-25T09:44:45.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.870+0000 7f515bd4e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540af500 con 0x7f5154069d90 2023-08-25T09:44:45.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.870+0000 7f515bd4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540ae8f0 con 0x7f51540c2110 2023-08-25T09:44:45.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 0x7f51540b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 0x7f51540b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50274/0 (socket says 172.21.15.70:50274) 2023-08-25T09:44:45.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 -- 172.21.15.70:0/205665915 learned_addr learned my addr 172.21.15.70:0/205665915 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:45.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 -- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51540c2110 msgr2=0x7f51540adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51540c2110 0x7f51540adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f5159aea640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5154069d90 0x7f5154151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 -- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5154069d90 msgr2=0x7f5154151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5154069d90 0x7f5154151e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 -- 172.21.15.70:0/205665915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c009a70 con 0x7f515415ae50 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.871+0000 7f515a2eb640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 0x7f51540b0200 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f514c009e80 tx=0x7f514c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.872+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c03d600 con 0x7f515415ae50 2023-08-25T09:44:45.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.872+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c03d780 con 0x7f515415ae50 2023-08-25T09:44:45.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.872+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51540aebd0 con 0x7f515415ae50 2023-08-25T09:44:45.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.872+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c03db30 con 0x7f515415ae50 2023-08-25T09:44:45.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.872+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51540af190 con 0x7f515415ae50 2023-08-25T09:44:45.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.873+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51540c4e70 con 0x7f515415ae50 2023-08-25T09:44:45.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.874+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f514c03b040 con 0x7f515415ae50 2023-08-25T09:44:45.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.875+0000 7f514a7fc640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f513806a080 0x7f513806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:45.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.875+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f514c0aad60 con 0x7f515415ae50 2023-08-25T09:44:45.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.875+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f51380721d0 con 0x7f515415ae50 2023-08-25T09:44:45.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.876+0000 7f5159aea640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f513806a080 0x7f513806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:45.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.876+0000 7f5159aea640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f513806a080 0x7f513806c540 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5150000f60 tx=0x7f5150009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:45.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:45.880+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f514c04e060 con 0x7f515415ae50 2023-08-25T09:44:46.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.066+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51540af670 con 0x7f515415ae50 2023-08-25T09:44:46.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.066+0000 7f514a7fc640 1 -- 172.21.15.70:0/205665915 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f514c049e60 con 0x7f515415ae50 2023-08-25T09:44:46.069 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:44:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f513806a080 msgr2=0x7f513806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f513806a080 0x7f513806c540 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5150000f60 tx=0x7f5150009040 comp rx=0 tx=0).stop 2023-08-25T09:44:46.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 msgr2=0x7f51540b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 0x7f51540b0200 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f514c009e80 tx=0x7f514c004a40 comp rx=0 tx=0).stop 2023-08-25T09:44:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 shutdown_connections 2023-08-25T09:44:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f513806a080 0x7f513806c540 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f515415ae50 0x7f51540b0200 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51540c2110 0x7f51540adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 --2- 172.21.15.70:0/205665915 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5154069d90 0x7f5154151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 >> 172.21.15.70:0/205665915 conn(0x7f51540cb0d0 msgr2=0x7f51540cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 shutdown_connections 2023-08-25T09:44:46.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.069+0000 7f515bd4e640 1 -- 172.21.15.70:0/205665915 wait complete. 2023-08-25T09:44:46.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:44:46.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.295+0000 7f87498a7640 1 Processor -- start 2023-08-25T09:44:46.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.295+0000 7f87498a7640 1 -- start start 2023-08-25T09:44:46.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f87498a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f87440ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f87498a7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87440c2550 con 0x7f874415ae50 2023-08-25T09:44:46.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f87498a7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87440ce330 con 0x7f87440c2110 2023-08-25T09:44:46.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f87498a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87440cc5f0 con 0x7f8744069f60 2023-08-25T09:44:46.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f8742ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f874415ae50 0x7f87440cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52352/0 (socket says 172.21.15.70:52352) 2023-08-25T09:44:46.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f8742ffd640 1 -- 172.21.15.70:0/3360676867 learned_addr learned my addr 172.21.15.70:0/3360676867 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:46.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381393328 0 0) 0x7f87440c2550 con 0x7f874415ae50 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f87437fe640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f87440ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.297+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f872c003520 con 0x7f874415ae50 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301928564 0 0) 0x7f872c003520 con 0x7f874415ae50 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 msgr2=0x7f87440ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f87440ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 >> v1:172.21.15.155:6789/0 conn(0x7f87440c2110 legacy=0x7f8744069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744070650 con 0x7f874415ae50 2023-08-25T09:44:46.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8734001670 con 0x7f874415ae50 2023-08-25T09:44:46.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 >> v1:172.21.15.70:6789/0 conn(0x7f874415ae50 legacy=0x7f87440cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f872c0039a0 0x7f872c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f872c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f872c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87440cc5f0 con 0x7f872c004d00 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872c00bba0 con 0x7f872c0039a0 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c00bd60 con 0x7f8744069f60 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.298+0000 7f87437fe640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f872c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f8742ffd640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f872c0039a0 0x7f872c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87437fe640 1 -- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 msgr2=0x7f872c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87427fc640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f872c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87437fe640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f872c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87437fe640 1 -- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f872c0039a0 msgr2=0x7f872c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87437fe640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f872c0039a0 0x7f872c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87437fe640 1 -- 172.21.15.70:0/3360676867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c003520 con 0x7f872c004d00 2023-08-25T09:44:46.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f8742ffd640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f872c0039a0 0x7f872c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:46.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.299+0000 7f87427fc640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8744069f60 0x7f872c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:46.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.300+0000 7f87437fe640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f872c005180 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f873c007330 tx=0x7f872c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b618995eb99aba70 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:46.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.300+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c02cca0 con 0x7f872c004d00 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.300+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873c02ce00 con 0x7f872c004d00 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.300+0000 7f8741ffb640 1 -- 172.21.15.70:0/3360676867 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c034660 con 0x7f872c004d00 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87498a7640 1 -- 172.21.15.70:0/3360676867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 msgr2=0x7f872c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87498a7640 1 --2- 172.21.15.70:0/3360676867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f872c005180 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f873c007330 tx=0x7f872c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87437fe640 1 -- 172.21.15.70:0/3360676867 reap_dead start 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87498a7640 1 -- 172.21.15.70:0/3360676867 shutdown_connections 2023-08-25T09:44:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87498a7640 1 -- 172.21.15.70:0/3360676867 >> 172.21.15.70:0/3360676867 conn(0x7f87440cb0d0 msgr2=0x7f87440cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:46.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87498a7640 1 -- 172.21.15.70:0/3360676867 shutdown_connections 2023-08-25T09:44:46.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.301+0000 7f87498a7640 1 -- 172.21.15.70:0/3360676867 wait complete. 2023-08-25T09:44:46.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.303+0000 7f87498a7640 1 Processor -- start 2023-08-25T09:44:46.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.303+0000 7f87498a7640 1 -- start start 2023-08-25T09:44:46.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f87498a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f874414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f87498a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87441f6320 0x7f87440c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f87498a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87441f5620 0x7f87441f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f87498a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87441f6010 con 0x7f872c004d00 2023-08-25T09:44:46.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f87498a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874415c280 con 0x7f87441f5620 2023-08-25T09:44:46.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f87498a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87440c9b20 con 0x7f87441f6320 2023-08-25T09:44:46.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f874414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f874414fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50284/0 (socket says 172.21.15.70:50284) 2023-08-25T09:44:46.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 -- 172.21.15.70:0/2801296478 learned_addr learned my addr 172.21.15.70:0/2801296478 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:46.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 -- 172.21.15.70:0/2801296478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87441f6320 msgr2=0x7f87440c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87441f6320 0x7f87440c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 -- 172.21.15.70:0/2801296478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87441f5620 msgr2=0x7f87441f5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87441f5620 0x7f87441f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 -- 172.21.15.70:0/2801296478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87440c9e00 con 0x7f872c004d00 2023-08-25T09:44:46.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.305+0000 7f8742ffd640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f874414fea0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f873400d8b0 tx=0x7f873400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:46.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.306+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340074e0 con 0x7f872c004d00 2023-08-25T09:44:46.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.306+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8734007640 con 0x7f872c004d00 2023-08-25T09:44:46.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.306+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340049e0 con 0x7f872c004d00 2023-08-25T09:44:46.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.306+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87440ca0e0 con 0x7f872c004d00 2023-08-25T09:44:46.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.306+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87441503e0 con 0x7f872c004d00 2023-08-25T09:44:46.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.307+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8744065d00 con 0x7f872c004d00 2023-08-25T09:44:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.308+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8734004b40 con 0x7f872c004d00 2023-08-25T09:44:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.309+0000 7f872bfff640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f873006a080 0x7f873006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.309+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8734087ff0 con 0x7f872c004d00 2023-08-25T09:44:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.309+0000 7f87427fc640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f873006a080 0x7f873006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.309+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=100}) v3 -- 0x7f87300721d0 con 0x7f872c004d00 2023-08-25T09:44:46.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.310+0000 7f87427fc640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f873006a080 0x7f873006c540 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f874415dfa0 tx=0x7f8738009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:46.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.313+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f87340ba990 con 0x7f872c004d00 2023-08-25T09:44:46.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.501+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f874415c3f0 con 0x7f872c004d00 2023-08-25T09:44:46.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.503+0000 7f872bfff640 1 -- 172.21.15.70:0/2801296478 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f874415c3f0 con 0x7f872c004d00 2023-08-25T09:44:46.505 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:44:46.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f873006a080 msgr2=0x7f873006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f873006a080 0x7f873006c540 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f874415dfa0 tx=0x7f8738009040 comp rx=0 tx=0).stop 2023-08-25T09:44:46.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 msgr2=0x7f874414fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f874414fea0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f873400d8b0 tx=0x7f873400dd70 comp rx=0 tx=0).stop 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 shutdown_connections 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f873006a080 0x7f873006c540 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87441f5620 0x7f87441f5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87441f6320 0x7f87440c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 --2- 172.21.15.70:0/2801296478 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f872c004d00 0x7f874414fea0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.505+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 >> 172.21.15.70:0/2801296478 conn(0x7f87440cb0d0 msgr2=0x7f8744069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:46.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.506+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 shutdown_connections 2023-08-25T09:44:46.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.506+0000 7f87498a7640 1 -- 172.21.15.70:0/2801296478 wait complete. 2023-08-25T09:44:46.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:44:46.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.728+0000 7f69e7596640 1 Processor -- start 2023-08-25T09:44:46.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.728+0000 7f69e7596640 1 -- start start 2023-08-25T09:44:46.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e7596640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f69e00ca780 0x7f69e00cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e7596640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 0x7f69e00b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e7596640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e00b12b0 con 0x7f69e015acd0 2023-08-25T09:44:46.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e7596640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e006c0e0 con 0x7f69e00ca780 2023-08-25T09:44:46.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e7596640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e006ae20 con 0x7f69e00b3cf0 2023-08-25T09:44:46.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e5332640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f69e015acd0 0x7f69e015b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52360/0 (socket says 172.21.15.70:52360) 2023-08-25T09:44:46.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.730+0000 7f69e5332640 1 -- 172.21.15.70:0/4032539839 learned_addr learned my addr 172.21.15.70:0/4032539839 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:46.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096683224 0 0) 0x7f69e00b12b0 con 0x7f69e015acd0 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69cc003520 con 0x7f69e015acd0 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69e4b31640 1 --2- 172.21.15.70:0/4032539839 >> v2:172.21.15.155:3300/0 conn(0x7f69e00ca780 0x7f69e00cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69e5b33640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 0x7f69e00b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 674649056 0 0) 0x7f69cc003520 con 0x7f69e015acd0 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 msgr2=0x7f69e00b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 0x7f69e00b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 >> v2:172.21.15.155:3300/0 conn(0x7f69e00ca780 msgr2=0x7f69e00cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 --2- 172.21.15.70:0/4032539839 >> v2:172.21.15.155:3300/0 conn(0x7f69e00ca780 0x7f69e00cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e015b7c0 con 0x7f69e015acd0 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f69dc0017f0 con 0x7f69e015acd0 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 >> v1:172.21.15.70:6789/0 conn(0x7f69e015acd0 legacy=0x7f69e015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 0x7f69cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc004d00 0x7f69cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e006c0e0 con 0x7f69cc003970 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e006ae20 con 0x7f69cc004d00 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69cc00bb10 con 0x7f69e00b3cf0 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.731+0000 7f69e5332640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.732+0000 7f69e5332640 1 -- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 msgr2=0x7f69cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.732+0000 7f69e5332640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00b3cf0 0x7f69cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.732+0000 7f69e5332640 1 -- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc004d00 msgr2=0x7f69cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:46.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.732+0000 7f69e5332640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69cc004d00 0x7f69cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.732+0000 7f69e5332640 1 -- 172.21.15.70:0/4032539839 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69cc003520 con 0x7f69cc003970 2023-08-25T09:44:46.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.733+0000 7f69e5332640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69cc003d90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c240 tx=0x7f69cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab343144324ec675 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:46.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.733+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc007270 con 0x7f69cc003970 2023-08-25T09:44:46.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.733+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc0073d0 con 0x7f69cc003970 2023-08-25T09:44:46.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.733+0000 7f69d7fff640 1 -- 172.21.15.70:0/4032539839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc02c8b0 con 0x7f69cc003970 2023-08-25T09:44:46.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e7596640 1 -- 172.21.15.70:0/4032539839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 msgr2=0x7f69cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e7596640 1 --2- 172.21.15.70:0/4032539839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69cc003d90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c240 tx=0x7f69cc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e5b33640 1 -- 172.21.15.70:0/4032539839 reap_dead start 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e7596640 1 -- 172.21.15.70:0/4032539839 shutdown_connections 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e7596640 1 -- 172.21.15.70:0/4032539839 >> 172.21.15.70:0/4032539839 conn(0x7f69e00ce760 msgr2=0x7f69e00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e7596640 1 -- 172.21.15.70:0/4032539839 shutdown_connections 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.734+0000 7f69e7596640 1 -- 172.21.15.70:0/4032539839 wait complete. 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.736+0000 7f69e7596640 1 Processor -- start 2023-08-25T09:44:46.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.736+0000 7f69e7596640 1 -- start start 2023-08-25T09:44:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e7596640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69e0158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e7596640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69e00cb5e0 0x7f69e00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e7596640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00cba60 0x7f69e00cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e7596640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00cab90 con 0x7f69cc003970 2023-08-25T09:44:46.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e7596640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e01f0ae0 con 0x7f69e00cb5e0 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e7596640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e01f0dc0 con 0x7f69e00cba60 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e5332640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69e0158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e5332640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69e0158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50304/0 (socket says 172.21.15.70:50304) 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.737+0000 7f69e5332640 1 -- 172.21.15.70:0/2148007362 learned_addr learned my addr 172.21.15.70:0/2148007362 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e5332640 1 -- 172.21.15.70:0/2148007362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00cba60 msgr2=0x7f69e00cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e5332640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00cba60 0x7f69e00cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e5332640 1 -- 172.21.15.70:0/2148007362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69e00cb5e0 msgr2=0x7f69e00cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e5332640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69e00cb5e0 0x7f69e00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e5332640 1 -- 172.21.15.70:0/2148007362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00cade0 con 0x7f69cc003970 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e5332640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69e0158df0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f69dc02c450 tx=0x7f69e015b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc03c040 con 0x7f69cc003970 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc007750 con 0x7f69cc003970 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0059a0 con 0x7f69cc003970 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e00cb0c0 con 0x7f69cc003970 2023-08-25T09:44:46.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.738+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e0159330 con 0x7f69cc003970 2023-08-25T09:44:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.740+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e004e420 con 0x7f69cc003970 2023-08-25T09:44:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.740+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f69dc032070 con 0x7f69cc003970 2023-08-25T09:44:46.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.742+0000 7f69d5ffb640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f69c406a080 0x7f69c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.742+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69dc0a9b80 con 0x7f69cc003970 2023-08-25T09:44:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.742+0000 7f69e4b31640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f69c406a080 0x7f69c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.742+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=101}) v3 -- 0x7f69c40721d0 con 0x7f69cc003970 2023-08-25T09:44:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.744+0000 7f69e4b31640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f69c406a080 0x7f69c406c540 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f69d0006e80 tx=0x7f69d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:46.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.747+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f69dc0751d0 con 0x7f69cc003970 2023-08-25T09:44:46.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.949+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f69e006bec0 con 0x7f69cc003970 2023-08-25T09:44:46.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.950+0000 7f69d5ffb640 1 -- 172.21.15.70:0/2148007362 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7f69dc0a9e60 con 0x7f69cc003970 2023-08-25T09:44:46.951 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:44:46.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f69c406a080 msgr2=0x7f69c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f69c406a080 0x7f69c406c540 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f69d0006e80 tx=0x7f69d0008040 comp rx=0 tx=0).stop 2023-08-25T09:44:46.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 msgr2=0x7f69e0158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:46.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69e0158df0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f69dc02c450 tx=0x7f69e015b950 comp rx=0 tx=0).stop 2023-08-25T09:44:46.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 shutdown_connections 2023-08-25T09:44:46.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f69c406a080 0x7f69c406c540 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69e00cba60 0x7f69e00cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69e00cb5e0 0x7f69e00cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 --2- 172.21.15.70:0/2148007362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69cc003970 0x7f69e0158df0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:46.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.952+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 >> 172.21.15.70:0/2148007362 conn(0x7f69e00ce760 msgr2=0x7f69e00b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:46.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.953+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 shutdown_connections 2023-08-25T09:44:46.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:46.953+0000 7f69e7596640 1 -- 172.21.15.70:0/2148007362 wait complete. 2023-08-25T09:44:46.973 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:44:46.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:46.973 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:44:47.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[89968]: fuse finished with error 0 and tester_r 0 2023-08-25T09:44:47.073 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:44:47.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:47.074 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:44:47.225 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:44:47.226 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:44:47.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:44:47.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:44:47.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:47.287 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:44:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:44:47.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:44:47.342 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:44:47.342 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:44:47.403 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:44:47.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:44:47.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:44:47.404 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:44:47.461 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:44:47.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:44:47.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:44:47.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.685+0000 7f1a74238640 1 Processor -- start 2023-08-25T09:44:47.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.685+0000 7f1a74238640 1 -- start start 2023-08-25T09:44:47.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.687+0000 7f1a74238640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:47.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.687+0000 7f1a74238640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1a6c06b2b0 0x7f1a6c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:47.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.687+0000 7f1a74238640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0b1b60 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.687+0000 7f1a74238640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c06bd80 con 0x7f1a6c06b2b0 2023-08-25T09:44:47.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.687+0000 7f1a74238640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c06b7c0 con 0x7f1a6c15e830 2023-08-25T09:44:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.687+0000 7f1a71fd4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a71fd4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:53258/0 (socket says 172.21.15.70:53258) 2023-08-25T09:44:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a71fd4640 1 -- 172.21.15.70:0/4146978776 learned_addr learned my addr 172.21.15.70:0/4146978776 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a717d3640 1 --2- 172.21.15.70:0/4146978776 >> v2:172.21.15.155:3300/0 conn(0x7f1a6c06b2b0 0x7f1a6c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a71fd4640 1 -- 172.21.15.70:0/4146978776 >> v2:172.21.15.155:3300/0 conn(0x7f1a6c06b2b0 msgr2=0x7f1a6c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:47.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a71fd4640 1 --2- 172.21.15.70:0/4146978776 >> v2:172.21.15.155:3300/0 conn(0x7f1a6c06b2b0 0x7f1a6c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a71fd4640 1 -- 172.21.15.70:0/4146978776 >> v1:172.21.15.155:6789/0 conn(0x7f1a6c15c6d0 legacy=0x7f1a6c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.688+0000 7f1a71fd4640 1 -- 172.21.15.70:0/4146978776 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c06baa0 con 0x7f1a6c15e830 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.689+0000 7f1a71fd4640 1 --2- 172.21.15.70:0/4146978776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c06ad70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009e50 tx=0x7f1a6802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c4d626b27c3f8a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.690+0000 7f1a70fd2640 1 -- 172.21.15.70:0/4146978776 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6802cb00 con 0x7f1a6c15e830 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.690+0000 7f1a70fd2640 1 -- 172.21.15.70:0/4146978776 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6802cc60 con 0x7f1a6c15e830 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.690+0000 7f1a70fd2640 1 -- 172.21.15.70:0/4146978776 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680343f0 con 0x7f1a6c15e830 2023-08-25T09:44:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.690+0000 7f1a74238640 1 -- 172.21.15.70:0/4146978776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 msgr2=0x7f1a6c06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.690+0000 7f1a74238640 1 --2- 172.21.15.70:0/4146978776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c06ad70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009e50 tx=0x7f1a6802bf60 comp rx=0 tx=0).stop 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.691+0000 7f1a74238640 1 -- 172.21.15.70:0/4146978776 shutdown_connections 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.691+0000 7f1a74238640 1 --2- 172.21.15.70:0/4146978776 >> v2:172.21.15.155:3300/0 conn(0x7f1a6c06b2b0 0x7f1a6c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.691+0000 7f1a74238640 1 --2- 172.21.15.70:0/4146978776 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c06ad70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.691+0000 7f1a74238640 1 -- 172.21.15.70:0/4146978776 >> 172.21.15.70:0/4146978776 conn(0x7f1a6c069850 msgr2=0x7f1a6c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.691+0000 7f1a74238640 1 -- 172.21.15.70:0/4146978776 shutdown_connections 2023-08-25T09:44:47.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.691+0000 7f1a74238640 1 -- 172.21.15.70:0/4146978776 wait complete. 2023-08-25T09:44:47.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.692+0000 7f1a74238640 1 Processor -- start 2023-08-25T09:44:47.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 -- start start 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6c06b2b0 0x7f1a6c1eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 0x7f1a6c155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0a9320 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c15aa90 con 0x7f1a6c06b2b0 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.693+0000 7f1a74238640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c06fa40 con 0x7f1a6c15e830 2023-08-25T09:44:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 0x7f1a6c155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:47.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 0x7f1a6c155ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50314/0 (socket says 172.21.15.70:50314) 2023-08-25T09:44:47.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 -- 172.21.15.70:0/280465553 learned_addr learned my addr 172.21.15.70:0/280465553 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:47.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a71fd4640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6c06b2b0 0x7f1a6c1eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:47.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a727d5640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:47.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 -- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 msgr2=0x7f1a6c1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:47.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c1560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 -- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6c06b2b0 msgr2=0x7f1a6c1eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6c06b2b0 0x7f1a6c1eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.694+0000 7f1a717d3640 1 -- 172.21.15.70:0/280465553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68009b30 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.695+0000 7f1a717d3640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 0x7f1a6c155ba0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00e980 tx=0x7f1a5c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.695+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c009670 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.695+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c00c9a0 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.695+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00cd30 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.695+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0a9500 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.695+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c1edad0 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.696+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c04e420 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.697+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1a5c013470 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.698+0000 7f1a627fc640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a5006a050 0x7f1a5006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.698+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a5c088810 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.698+0000 7f1a71fd4640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a5006a050 0x7f1a5006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.699+0000 7f1a71fd4640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a5006a050 0x7f1a5006c510 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1a68035000 tx=0x7f1a68030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:47.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.702+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a5c053e70 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.912+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a6c0caca0 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.913+0000 7f1a627fc640 1 -- 172.21.15.70:0/280465553 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7f1a5c057490 con 0x7f1a6c15c6d0 2023-08-25T09:44:47.915 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:47.916 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:44:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:44:47.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a5006a050 msgr2=0x7f1a5006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:47.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a5006a050 0x7f1a5006c510 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1a68035000 tx=0x7f1a68030000 comp rx=0 tx=0).stop 2023-08-25T09:44:47.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 msgr2=0x7f1a6c155ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:47.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 0x7f1a6c155ba0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00e980 tx=0x7f1a5c00ee40 comp rx=0 tx=0).stop 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 shutdown_connections 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a5006a050 0x7f1a5006c510 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a6c15e830 0x7f1a6c1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a6c15c6d0 0x7f1a6c155ba0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 --2- 172.21.15.70:0/280465553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a6c06b2b0 0x7f1a6c1eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 >> 172.21.15.70:0/280465553 conn(0x7f1a6c069850 msgr2=0x7f1a6c06ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 shutdown_connections 2023-08-25T09:44:47.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:47.915+0000 7f1a74238640 1 -- 172.21.15.70:0/280465553 wait complete. 2023-08-25T09:44:47.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:44:48.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.141+0000 7f33fa257640 1 Processor -- start 2023-08-25T09:44:48.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.141+0000 7f33fa257640 1 -- start start 2023-08-25T09:44:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.142+0000 7f33fa257640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33f40ce5b0 0x7f33f40ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33fa257640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f40ceed0 con 0x7f33f40b2850 2023-08-25T09:44:48.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33fa257640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f415d3f0 con 0x7f33f415aa10 2023-08-25T09:44:48.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33fa257640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f4159d10 con 0x7f33f40ce5b0 2023-08-25T09:44:48.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33f37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f33f40ce5b0 0x7f33f40ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33f3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f33f40b2850 0x7f33f406f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52372/0 (socket says 172.21.15.70:52372) 2023-08-25T09:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33f3fff640 1 -- 172.21.15.70:0/2823825913 learned_addr learned my addr 172.21.15.70:0/2823825913 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:48.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33f37fe640 1 -- 172.21.15.70:0/2823825913 >> v1:172.21.15.155:6789/0 conn(0x7f33f415aa10 legacy=0x7f33f40b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33f37fe640 1 -- 172.21.15.70:0/2823825913 >> v1:172.21.15.70:6789/0 conn(0x7f33f40b2850 legacy=0x7f33f406f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.143+0000 7f33f37fe640 1 -- 172.21.15.70:0/2823825913 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f40b2d20 con 0x7f33f40ce5b0 2023-08-25T09:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.145+0000 7f33f37fe640 1 --2- 172.21.15.70:0/2823825913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f40ce990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f33e40097a0 tx=0x7f33e402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52ed9ff591b8965 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.145+0000 7f33f27fc640 1 -- 172.21.15.70:0/2823825913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e402e070 con 0x7f33f40ce5b0 2023-08-25T09:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.145+0000 7f33f27fc640 1 -- 172.21.15.70:0/2823825913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e40315a0 con 0x7f33f40ce5b0 2023-08-25T09:44:48.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.145+0000 7f33f27fc640 1 -- 172.21.15.70:0/2823825913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4031930 con 0x7f33f40ce5b0 2023-08-25T09:44:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.145+0000 7f33fa257640 1 -- 172.21.15.70:0/2823825913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 msgr2=0x7f33f40ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.145+0000 7f33fa257640 1 --2- 172.21.15.70:0/2823825913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f40ce990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f33e40097a0 tx=0x7f33e402b840 comp rx=0 tx=0).stop 2023-08-25T09:44:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.146+0000 7f33fa257640 1 -- 172.21.15.70:0/2823825913 shutdown_connections 2023-08-25T09:44:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.146+0000 7f33fa257640 1 --2- 172.21.15.70:0/2823825913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f40ce990 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.146+0000 7f33fa257640 1 -- 172.21.15.70:0/2823825913 >> 172.21.15.70:0/2823825913 conn(0x7f33f4069850 msgr2=0x7f33f4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.146+0000 7f33fa257640 1 -- 172.21.15.70:0/2823825913 shutdown_connections 2023-08-25T09:44:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.146+0000 7f33fa257640 1 -- 172.21.15.70:0/2823825913 wait complete. 2023-08-25T09:44:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.148+0000 7f33fa257640 1 Processor -- start 2023-08-25T09:44:48.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.148+0000 7f33fa257640 1 -- start start 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.149+0000 7f33fa257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33f40b2850 0x7f33f41f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.149+0000 7f33fa257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f413fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.149+0000 7f33fa257640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33f415aa10 0x7f33f40aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.149+0000 7f33fa257640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40ae2c0 con 0x7f33f40ce5b0 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.149+0000 7f33fa257640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f40ae5a0 con 0x7f33f415aa10 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.149+0000 7f33fa257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f4140340 con 0x7f33f40b2850 2023-08-25T09:44:48.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f413fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f413fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50338/0 (socket says 172.21.15.70:50338) 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 -- 172.21.15.70:0/3180356796 learned_addr learned my addr 172.21.15.70:0/3180356796 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f3fff640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33f415aa10 0x7f33f40aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f37fe640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33f40b2850 0x7f33f41f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 -- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33f40b2850 msgr2=0x7f33f41f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33f40b2850 0x7f33f41f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 -- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33f415aa10 msgr2=0x7f33f40aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33f415aa10 0x7f33f40aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 -- 172.21.15.70:0/3180356796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e4009400 con 0x7f33f40ce5b0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.150+0000 7f33f2ffd640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f413fe00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f33e800b3f0 tx=0x7f33e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.151+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8011830 con 0x7f33f40ce5b0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.151+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e8011990 con 0x7f33f40ce5b0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.151+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8011d40 con 0x7f33f40ce5b0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.151+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f4140610 con 0x7f33f40ce5b0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.151+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f4140bd0 con 0x7f33f40ce5b0 2023-08-25T09:44:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.152+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f40cac90 con 0x7f33f40ce5b0 2023-08-25T09:44:48.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.153+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33e801a400 con 0x7f33f40ce5b0 2023-08-25T09:44:48.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.154+0000 7f33dbfff640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f33e006a050 0x7f33e006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.154+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33e8088500 con 0x7f33f40ce5b0 2023-08-25T09:44:48.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.154+0000 7f33f37fe640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f33e006a050 0x7f33e006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.155+0000 7f33f37fe640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f33e006a050 0x7f33e006c510 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:48.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.158+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f33e8053b50 con 0x7f33f40ce5b0 2023-08-25T09:44:48.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.349+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33f40ae710 con 0x7f33f40ce5b0 2023-08-25T09:44:48.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.352+0000 7f33dbfff640 1 -- 172.21.15.70:0/3180356796 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+18407 (secure 0 0 0) 0x7f33e8057170 con 0x7f33f40ce5b0 2023-08-25T09:44:48.354 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:48.355 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":101,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:44:47.553870+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["backfillfull","exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:44:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f33e006a050 msgr2=0x7f33e006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f33e006a050 0x7f33e006c510 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e403d040 comp rx=0 tx=0).stop 2023-08-25T09:44:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 msgr2=0x7f33f413fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f413fe00 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f33e800b3f0 tx=0x7f33e800b8b0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 shutdown_connections 2023-08-25T09:44:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f33e006a050 0x7f33e006c510 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33f415aa10 0x7f33f40aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33f40ce5b0 0x7f33f413fe00 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 --2- 172.21.15.70:0/3180356796 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33f40b2850 0x7f33f41f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 >> 172.21.15.70:0/3180356796 conn(0x7f33f4069850 msgr2=0x7f33f41575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 shutdown_connections 2023-08-25T09:44:48.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.355+0000 7f33fa257640 1 -- 172.21.15.70:0/3180356796 wait complete. 2023-08-25T09:44:48.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:43:52.155883+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:43:52.872267+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:44:48.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:44:48.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.602+0000 7f9707b22640 1 Processor -- start 2023-08-25T09:44:48.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.603+0000 7f9707b22640 1 -- start start 2023-08-25T09:44:48.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.604+0000 7f9707b22640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9700069d30 0x7f97000ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.604+0000 7f9707b22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006a110 0x7f97000ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f9707b22640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f970006c510 0x7f970006c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f9707b22640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000cd4e0 con 0x7f970006c510 2023-08-25T09:44:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f9707b22640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97000cc550 con 0x7f9700069d30 2023-08-25T09:44:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f9707b22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ccb10 con 0x7f970006a110 2023-08-25T09:44:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f970006c510 0x7f970006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f970006c510 0x7f970006c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50340/0 (socket says 172.21.15.70:50340) 2023-08-25T09:44:48.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 -- 172.21.15.70:0/3072766119 learned_addr learned my addr 172.21.15.70:0/3072766119 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:48.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97058be640 1 --2- 172.21.15.70:0/3072766119 >> v2:172.21.15.155:3300/0 conn(0x7f9700069d30 0x7f97000ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97050bd640 1 --2- 172.21.15.70:0/3072766119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006a110 0x7f97000ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 -- 172.21.15.70:0/3072766119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006a110 msgr2=0x7f97000ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 --2- 172.21.15.70:0/3072766119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006a110 0x7f97000ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 -- 172.21.15.70:0/3072766119 >> v2:172.21.15.155:3300/0 conn(0x7f9700069d30 msgr2=0x7f97000ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.605+0000 7f97060bf640 1 --2- 172.21.15.70:0/3072766119 >> v2:172.21.15.155:3300/0 conn(0x7f9700069d30 0x7f97000ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.606+0000 7f97060bf640 1 -- 172.21.15.70:0/3072766119 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000ade40 con 0x7f970006c510 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.607+0000 7f97060bf640 1 --2- 172.21.15.70:0/3072766119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006c510 0x7f970006c970 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f97000cd680 tx=0x7f96f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831574a9dee8c77f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.607+0000 7f97048bc640 1 -- 172.21.15.70:0/3072766119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80317e0 con 0x7f970006c510 2023-08-25T09:44:48.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.607+0000 7f97048bc640 1 -- 172.21.15.70:0/3072766119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8031940 con 0x7f970006c510 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.607+0000 7f97048bc640 1 -- 172.21.15.70:0/3072766119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8031cd0 con 0x7f970006c510 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.607+0000 7f9707b22640 1 -- 172.21.15.70:0/3072766119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006c510 msgr2=0x7f970006c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.607+0000 7f9707b22640 1 --2- 172.21.15.70:0/3072766119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006c510 0x7f970006c970 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f97000cd680 tx=0x7f96f802b880 comp rx=0 tx=0).stop 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 -- 172.21.15.70:0/3072766119 shutdown_connections 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 --2- 172.21.15.70:0/3072766119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006c510 0x7f970006c970 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 --2- 172.21.15.70:0/3072766119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006a110 0x7f97000ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 --2- 172.21.15.70:0/3072766119 >> v2:172.21.15.155:3300/0 conn(0x7f9700069d30 0x7f97000ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 -- 172.21.15.70:0/3072766119 >> 172.21.15.70:0/3072766119 conn(0x7f97000cb0d0 msgr2=0x7f97000cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:48.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 -- 172.21.15.70:0/3072766119 shutdown_connections 2023-08-25T09:44:48.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.608+0000 7f9707b22640 1 -- 172.21.15.70:0/3072766119 wait complete. 2023-08-25T09:44:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.609+0000 7f9707b22640 1 Processor -- start 2023-08-25T09:44:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.610+0000 7f9707b22640 1 -- start start 2023-08-25T09:44:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.610+0000 7f9707b22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9700069d30 0x7f9700156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f9707b22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 0x7f97000c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f9707b22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006c510 0x7f97000af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f9707b22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970015a0b0 con 0x7f970006a110 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f9707b22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9700157cb0 con 0x7f9700069d30 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f9707b22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9700157400 con 0x7f970006c510 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 0x7f97000c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 0x7f97000c28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50342/0 (socket says 172.21.15.70:50342) 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 -- 172.21.15.70:0/3659252347 learned_addr learned my addr 172.21.15.70:0/3659252347 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97058be640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9700069d30 0x7f9700156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 -- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006c510 msgr2=0x7f97000af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006c510 0x7f97000af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 -- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9700069d30 msgr2=0x7f9700156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9700069d30 0x7f9700156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.611+0000 7f97050bd640 1 -- 172.21.15.70:0/3659252347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f80094d0 con 0x7f970006a110 2023-08-25T09:44:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.612+0000 7f97050bd640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 0x7f97000c28e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f96f000d5f0 tx=0x7f96f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.612+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f00125f0 con 0x7f970006a110 2023-08-25T09:44:48.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.612+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0012750 con 0x7f970006a110 2023-08-25T09:44:48.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.612+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0012b00 con 0x7f970006a110 2023-08-25T09:44:48.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.612+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9700157960 con 0x7f970006a110 2023-08-25T09:44:48.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.612+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97000c4ba0 con 0x7f970006a110 2023-08-25T09:44:48.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.613+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9700157e40 con 0x7f970006a110 2023-08-25T09:44:48.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.614+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f96f0012c60 con 0x7f970006a110 2023-08-25T09:44:48.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.615+0000 7f96f67fc640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96e406a050 0x7f96e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:48.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.615+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96f0088a60 con 0x7f970006a110 2023-08-25T09:44:48.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.615+0000 7f97058be640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96e406a050 0x7f96e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:48.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.616+0000 7f97058be640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96e406a050 0x7f96e406c510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009e10 tx=0x7f96fc00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:48.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.620+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f96f0054140 con 0x7f970006a110 2023-08-25T09:44:48.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.808+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9700066fb0 con 0x7f970006a110 2023-08-25T09:44:48.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.809+0000 7f96f67fc640 1 -- 172.21.15.70:0/3659252347 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96f0057760 con 0x7f970006a110 2023-08-25T09:44:48.811 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:44:48.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.811+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96e406a050 msgr2=0x7f96e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.811+0000 7f9707b22640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96e406a050 0x7f96e406c510 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f96fc009e10 tx=0x7f96fc00b040 comp rx=0 tx=0).stop 2023-08-25T09:44:48.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 msgr2=0x7f97000c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 0x7f97000c28e0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f96f000d5f0 tx=0x7f96f000dab0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 shutdown_connections 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f96e406a050 0x7f96e406c510 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f970006c510 0x7f97000af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f970006a110 0x7f97000c28e0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 --2- 172.21.15.70:0/3659252347 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9700069d30 0x7f9700156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 >> 172.21.15.70:0/3659252347 conn(0x7f97000cb0d0 msgr2=0x7f9700163b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 shutdown_connections 2023-08-25T09:44:48.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:48.812+0000 7f9707b22640 1 -- 172.21.15.70:0/3659252347 wait complete. 2023-08-25T09:44:48.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:44:49.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.051+0000 7f41c21e3640 1 Processor -- start 2023-08-25T09:44:49.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.052+0000 7f41c21e3640 1 -- start start 2023-08-25T09:44:49.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c21e3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f41bc0b2850 0x7f41bc0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c21e3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f41bc0c8d20 0x7f41bc0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c21e3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc0cd800 con 0x7f41bc0b2470 2023-08-25T09:44:49.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c21e3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc15ad20 con 0x7f41bc0b2850 2023-08-25T09:44:49.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c21e3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc15a110 con 0x7f41bc0c8d20 2023-08-25T09:44:49.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c11e1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f41bc0b2470 0x7f41bc159930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44860/0 (socket says 172.21.15.70:44860) 2023-08-25T09:44:49.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c11e1640 1 -- 172.21.15.70:0/4252331972 learned_addr learned my addr 172.21.15.70:0/4252331972 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:49.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c09e0640 1 --2- 172.21.15.70:0/4252331972 >> v2:172.21.15.70:3300/0 conn(0x7f41bc0b2850 0x7f41bc0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41c19e2640 1 --2- 172.21.15.70:0/4252331972 >> v2:172.21.15.155:3300/0 conn(0x7f41bc0c8d20 0x7f41bc0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41b3fff640 1 -- 172.21.15.70:0/4252331972 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974760716 0 0) 0x7f41bc0cd800 con 0x7f41bc0b2470 2023-08-25T09:44:49.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.053+0000 7f41b3fff640 1 -- 172.21.15.70:0/4252331972 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a8003520 con 0x7f41bc0b2470 2023-08-25T09:44:49.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.054+0000 7f41c09e0640 1 -- 172.21.15.70:0/4252331972 >> v2:172.21.15.155:3300/0 conn(0x7f41bc0c8d20 msgr2=0x7f41bc0c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:49.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.054+0000 7f41c09e0640 1 --2- 172.21.15.70:0/4252331972 >> v2:172.21.15.155:3300/0 conn(0x7f41bc0c8d20 0x7f41bc0c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.054+0000 7f41c09e0640 1 -- 172.21.15.70:0/4252331972 >> v1:172.21.15.70:6789/0 conn(0x7f41bc0b2470 legacy=0x7f41bc159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.054+0000 7f41c09e0640 1 -- 172.21.15.70:0/4252331972 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc15a3f0 con 0x7f41bc0b2850 2023-08-25T09:44:49.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.055+0000 7f41c09e0640 1 --2- 172.21.15.70:0/4252331972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc0cdb30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f41bc071a20 tx=0x7f41ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e839e1b6bf489df server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.055+0000 7f41b3fff640 1 -- 172.21.15.70:0/4252331972 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac031830 con 0x7f41bc0b2850 2023-08-25T09:44:49.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.055+0000 7f41b3fff640 1 -- 172.21.15.70:0/4252331972 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac031990 con 0x7f41bc0b2850 2023-08-25T09:44:49.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.055+0000 7f41b3fff640 1 -- 172.21.15.70:0/4252331972 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac031d20 con 0x7f41bc0b2850 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.055+0000 7f41c21e3640 1 -- 172.21.15.70:0/4252331972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 msgr2=0x7f41bc0cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.055+0000 7f41c21e3640 1 --2- 172.21.15.70:0/4252331972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc0cdb30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f41bc071a20 tx=0x7f41ac02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.056+0000 7f41c21e3640 1 -- 172.21.15.70:0/4252331972 shutdown_connections 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.056+0000 7f41c21e3640 1 --2- 172.21.15.70:0/4252331972 >> v2:172.21.15.155:3300/0 conn(0x7f41bc0c8d20 0x7f41bc0c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.056+0000 7f41c21e3640 1 --2- 172.21.15.70:0/4252331972 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc0cdb30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.056+0000 7f41c21e3640 1 -- 172.21.15.70:0/4252331972 >> 172.21.15.70:0/4252331972 conn(0x7f41bc0ce830 msgr2=0x7f41bc0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:49.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.056+0000 7f41c21e3640 1 -- 172.21.15.70:0/4252331972 shutdown_connections 2023-08-25T09:44:49.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.056+0000 7f41c21e3640 1 -- 172.21.15.70:0/4252331972 wait complete. 2023-08-25T09:44:49.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.057+0000 7f41c21e3640 1 Processor -- start 2023-08-25T09:44:49.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.058+0000 7f41c21e3640 1 -- start start 2023-08-25T09:44:49.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c21e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41bc0b2470 0x7f41bc09bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c21e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc071180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c21e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41bc0c8d20 0x7f41bc15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c21e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc09c1e0 con 0x7f41bc0b2850 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c21e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc1f3aa0 con 0x7f41bc0c8d20 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c21e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0c4020 con 0x7f41bc0b2470 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc071180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37976/0 (socket says 172.21.15.70:37976) 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 -- 172.21.15.70:0/1012844499 learned_addr learned my addr 172.21.15.70:0/1012844499 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 -- 172.21.15.70:0/1012844499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41bc0b2470 msgr2=0x7f41bc09bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41bc0b2470 0x7f41bc09bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 -- 172.21.15.70:0/1012844499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41bc0c8d20 msgr2=0x7f41bc15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41bc0c8d20 0x7f41bc15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.059+0000 7f41c09e0640 1 -- 172.21.15.70:0/1012844499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac009400 con 0x7f41bc0b2850 2023-08-25T09:44:49.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.060+0000 7f41c09e0640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc071180 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f41ac02bee0 tx=0x7f41ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.060+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac03a6f0 con 0x7f41bc0b2850 2023-08-25T09:44:49.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.060+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac03a870 con 0x7f41bc0b2850 2023-08-25T09:44:49.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.060+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac03ac20 con 0x7f41bc0b2850 2023-08-25T09:44:49.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.060+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc0c4300 con 0x7f41bc0b2850 2023-08-25T09:44:49.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.060+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc1f3730 con 0x7f41bc0b2850 2023-08-25T09:44:49.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.061+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc061460 con 0x7f41bc0b2850 2023-08-25T09:44:49.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.062+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f41ac043410 con 0x7f41bc0b2850 2023-08-25T09:44:49.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.063+0000 7f41b1ffb640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f41a006a050 0x7f41a006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.063+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f41ac0aa0f0 con 0x7f41bc0b2850 2023-08-25T09:44:49.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.063+0000 7f41c11e1640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f41a006a050 0x7f41a006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.064+0000 7f41c11e1640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f41a006a050 0x7f41a006c510 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f41b8001360 tx=0x7f41b80077c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.067+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f41ac0757d0 con 0x7f41bc0b2850 2023-08-25T09:44:49.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.257+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f41bc1f3be0 con 0x7f41bc0b2850 2023-08-25T09:44:49.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.258+0000 7f41b1ffb640 1 -- 172.21.15.70:0/1012844499 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v101) v1 ==== 92+0+10 (secure 0 0 0) 0x7f41bc1f3be0 con 0x7f41bc0b2850 2023-08-25T09:44:49.260 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:44:49.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f41a006a050 msgr2=0x7f41a006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f41a006a050 0x7f41a006c510 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f41b8001360 tx=0x7f41b80077c0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 msgr2=0x7f41bc071180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc071180 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f41ac02bee0 tx=0x7f41ac0046c0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 shutdown_connections 2023-08-25T09:44:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f41a006a050 0x7f41a006c510 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f41bc0c8d20 0x7f41bc15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41bc0b2850 0x7f41bc071180 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 --2- 172.21.15.70:0/1012844499 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f41bc0b2470 0x7f41bc09bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.260+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 >> 172.21.15.70:0/1012844499 conn(0x7f41bc0ce830 msgr2=0x7f41bc0b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.261+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 shutdown_connections 2023-08-25T09:44:49.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.261+0000 7f41c21e3640 1 -- 172.21.15.70:0/1012844499 wait complete. 2023-08-25T09:44:49.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:44:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.490+0000 7f3e5f4a7640 1 Processor -- start 2023-08-25T09:44:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.490+0000 7f3e5f4a7640 1 -- start start 2023-08-25T09:44:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5f4a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 0x7f3e5815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5f4a7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5815e780 con 0x7f3e580cd660 2023-08-25T09:44:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5f4a7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e5806f8e0 con 0x7f3e5806f0e0 2023-08-25T09:44:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5f4a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5806fbc0 con 0x7f3e58159bf0 2023-08-25T09:44:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5da44640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e580cd660 0x7f3e580c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44874/0 (socket says 172.21.15.70:44874) 2023-08-25T09:44:49.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5da44640 1 -- 172.21.15.70:0/3738945433 learned_addr learned my addr 172.21.15.70:0/3738945433 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.492+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 0x7f3e5815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880809941 0 0) 0x7f3e5815e780 con 0x7f3e580cd660 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e44005730 con 0x7f3e580cd660 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3086643397 0 0) 0x7f3e44005730 con 0x7f3e580cd660 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 msgr2=0x7f3e5815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 0x7f3e5815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 >> v1:172.21.15.155:6789/0 conn(0x7f3e5806f0e0 legacy=0x7f3e58070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5806fea0 con 0x7f3e580cd660 2023-08-25T09:44:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3e50001670 con 0x7f3e580cd660 2023-08-25T09:44:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 >> v1:172.21.15.70:6789/0 conn(0x7f3e580cd660 legacy=0x7f3e580c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e44005bb0 0x7f3e44005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e440069d0 0x7f3e44006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 0x7f3e44007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5806fbc0 con 0x7f3e44005bb0 2023-08-25T09:44:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4400dc40 con 0x7f3e440069d0 2023-08-25T09:44:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4400de00 con 0x7f3e58159bf0 2023-08-25T09:44:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.493+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e44005bb0 0x7f3e44005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5ca42640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e440069d0 0x7f3e44006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5d243640 1 -- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 msgr2=0x7f3e44007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e58159bf0 0x7f3e44007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5d243640 1 -- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e440069d0 msgr2=0x7f3e44006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:49.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e440069d0 0x7f3e44006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5d243640 1 -- 172.21.15.70:0/3738945433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44005730 con 0x7f3e44005bb0 2023-08-25T09:44:49.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.494+0000 7f3e5ca42640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e440069d0 0x7f3e44006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.495+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e44005bb0 0x7f3e44005fd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009f90 tx=0x7f3e44007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585d4e4a4a0a2246 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.495+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5402cbf0 con 0x7f3e44005bb0 2023-08-25T09:44:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.495+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5400f040 con 0x7f3e44005bb0 2023-08-25T09:44:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.495+0000 7f3e4ffff640 1 -- 172.21.15.70:0/3738945433 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54034440 con 0x7f3e44005bb0 2023-08-25T09:44:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3738945433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e44005bb0 msgr2=0x7f3e44005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3738945433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e44005bb0 0x7f3e44005fd0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009f90 tx=0x7f3e44007a00 comp rx=0 tx=0).stop 2023-08-25T09:44:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5da44640 1 -- 172.21.15.70:0/3738945433 reap_dead start 2023-08-25T09:44:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3738945433 shutdown_connections 2023-08-25T09:44:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3738945433 >> 172.21.15.70:0/3738945433 conn(0x7f3e580cb0d0 msgr2=0x7f3e580cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3738945433 shutdown_connections 2023-08-25T09:44:49.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.496+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3738945433 wait complete. 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.498+0000 7f3e5f4a7640 1 Processor -- start 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.498+0000 7f3e5f4a7640 1 -- start start 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5f4a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e44005bb0 0x7f3e5814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5f4a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 0x7f3e580c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5f4a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e580c41a0 0x7f3e580c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5f4a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c3120 con 0x7f3e580c64c0 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5f4a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c33e0 con 0x7f3e580c41a0 2023-08-25T09:44:49.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5f4a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e580c36c0 con 0x7f3e44005bb0 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5ca42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 0x7f3e580c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5ca42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 0x7f3e580c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38004/0 (socket says 172.21.15.70:38004) 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5ca42640 1 -- 172.21.15.70:0/3102869459 learned_addr learned my addr 172.21.15.70:0/3102869459 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.499+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e44005bb0 0x7f3e5814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5da44640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e580c41a0 0x7f3e580c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5ca42640 1 -- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e44005bb0 msgr2=0x7f3e5814e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5ca42640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e44005bb0 0x7f3e5814e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5ca42640 1 -- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e580c41a0 msgr2=0x7f3e580c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5ca42640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e580c41a0 0x7f3e580c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5ca42640 1 -- 172.21.15.70:0/3102869459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580c39a0 con 0x7f3e580c64c0 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5ca42640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 0x7f3e580c6940 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800d7a0 tx=0x7f3e4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48017070 con 0x7f3e580c64c0 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48005070 con 0x7f3e580c64c0 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48005420 con 0x7f3e580c64c0 2023-08-25T09:44:49.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e580c3c80 con 0x7f3e580c64c0 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.500+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5815c830 con 0x7f3e580c64c0 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.502+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e58066fb0 con 0x7f3e580c64c0 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.502+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e48007030 con 0x7f3e580c64c0 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.503+0000 7f3e4dffb640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e3c06a080 0x7f3e3c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.503+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e48087ee0 con 0x7f3e580c64c0 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.504+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e3c06a080 0x7f3e3c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.504+0000 7f3e5d243640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e3c06a080 0x7f3e3c06c540 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3e54036000 tx=0x7f3e5402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.508+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3e48053530 con 0x7f3e580c64c0 2023-08-25T09:44:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.699+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e58151c10 con 0x7f3e580c64c0 2023-08-25T09:44:49.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.700+0000 7f3e4dffb640 1 -- 172.21.15.70:0/3102869459 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e48056b50 con 0x7f3e580c64c0 2023-08-25T09:44:49.702 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:44:49.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e3c06a080 msgr2=0x7f3e3c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e3c06a080 0x7f3e3c06c540 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3e54036000 tx=0x7f3e5402e000 comp rx=0 tx=0).stop 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 msgr2=0x7f3e580c6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 0x7f3e580c6940 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800d7a0 tx=0x7f3e4800dc60 comp rx=0 tx=0).stop 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 shutdown_connections 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e3c06a080 0x7f3e3c06c540 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e580c41a0 0x7f3e580c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e580c64c0 0x7f3e580c6940 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 --2- 172.21.15.70:0/3102869459 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e44005bb0 0x7f3e5814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 >> 172.21.15.70:0/3102869459 conn(0x7f3e580cb0d0 msgr2=0x7f3e58158020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 shutdown_connections 2023-08-25T09:44:49.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.703+0000 7f3e5f4a7640 1 -- 172.21.15.70:0/3102869459 wait complete. 2023-08-25T09:44:49.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:44:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.927+0000 7fb727457640 1 Processor -- start 2023-08-25T09:44:49.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.928+0000 7fb727457640 1 -- start start 2023-08-25T09:44:49.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.929+0000 7fb727457640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb7200cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.929+0000 7fb727457640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb72015cfe0 0x7fb7200d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.929+0000 7fb727457640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7200b6cc0 con 0x7fb7200b8510 2023-08-25T09:44:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.929+0000 7fb727457640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7200ce170 con 0x7fb72015cfe0 2023-08-25T09:44:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.929+0000 7fb727457640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200d1500 con 0x7fb7200cd850 2023-08-25T09:44:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7249f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb72015cfe0 0x7fb7200d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7259f4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb7200b8510 0x7fb7200b4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44888/0 (socket says 172.21.15.70:44888) 2023-08-25T09:44:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7259f4640 1 -- 172.21.15.70:0/604610914 learned_addr learned my addr 172.21.15.70:0/604610914 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7251f3640 1 --2- 172.21.15.70:0/604610914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb7200cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7249f2640 1 -- 172.21.15.70:0/604610914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 msgr2=0x7fb7200cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7249f2640 1 --2- 172.21.15.70:0/604610914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb7200cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7249f2640 1 -- 172.21.15.70:0/604610914 >> v1:172.21.15.70:6789/0 conn(0x7fb7200b8510 legacy=0x7fb7200b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb7249f2640 1 -- 172.21.15.70:0/604610914 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200d17e0 con 0x7fb72015cfe0 2023-08-25T09:44:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.930+0000 7fb717fff640 1 -- 172.21.15.70:0/604610914 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217673873 0 0) 0x7fb7180014e0 con 0x7fb7200b8510 2023-08-25T09:44:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.931+0000 7fb7249f2640 1 --2- 172.21.15.70:0/604610914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb7200d0c00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb7100097a0 tx=0x7fb71002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b08bada482536a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.931+0000 7fb717fff640 1 -- 172.21.15.70:0/604610914 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71002d040 con 0x7fb72015cfe0 2023-08-25T09:44:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.931+0000 7fb717fff640 1 -- 172.21.15.70:0/604610914 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb710031810 con 0x7fb72015cfe0 2023-08-25T09:44:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.931+0000 7fb717fff640 1 -- 172.21.15.70:0/604610914 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710031ba0 con 0x7fb72015cfe0 2023-08-25T09:44:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 -- 172.21.15.70:0/604610914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 msgr2=0x7fb7200d0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 --2- 172.21.15.70:0/604610914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb7200d0c00 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb7100097a0 tx=0x7fb71002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 -- 172.21.15.70:0/604610914 shutdown_connections 2023-08-25T09:44:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 --2- 172.21.15.70:0/604610914 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb7200d0c00 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 --2- 172.21.15.70:0/604610914 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb7200cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 -- 172.21.15.70:0/604610914 >> 172.21.15.70:0/604610914 conn(0x7fb720069850 msgr2=0x7fb720069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 -- 172.21.15.70:0/604610914 shutdown_connections 2023-08-25T09:44:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.932+0000 7fb727457640 1 -- 172.21.15.70:0/604610914 wait complete. 2023-08-25T09:44:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.934+0000 7fb727457640 1 Processor -- start 2023-08-25T09:44:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.934+0000 7fb727457640 1 -- start start 2023-08-25T09:44:49.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb727457640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7200b8510 0x7fb7201f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb727457640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb72015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb727457640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb72015f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb727457640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200ccba0 con 0x7fb72015cfe0 2023-08-25T09:44:49.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb727457640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201f5820 con 0x7fb7200b8510 2023-08-25T09:44:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb727457640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201f5a70 con 0x7fb7200cd850 2023-08-25T09:44:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.935+0000 7fb7259f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb72015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb72015f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38020/0 (socket says 172.21.15.70:38020) 2023-08-25T09:44:49.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 -- 172.21.15.70:0/4122202890 learned_addr learned my addr 172.21.15.70:0/4122202890 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 -- 172.21.15.70:0/4122202890 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 msgr2=0x7fb72015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb72015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 -- 172.21.15.70:0/4122202890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7200b8510 msgr2=0x7fb7201f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7200b8510 0x7fb7201f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 -- 172.21.15.70:0/4122202890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb710009400 con 0x7fb72015cfe0 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb7259f4640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb72015f910 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb718001740 tx=0x7fb71800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71800cea0 con 0x7fb72015cfe0 2023-08-25T09:44:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71800a640 con 0x7fb72015cfe0 2023-08-25T09:44:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71800a9f0 con 0x7fb72015cfe0 2023-08-25T09:44:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.936+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7201f5d50 con 0x7fb72015cfe0 2023-08-25T09:44:49.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.937+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7201f6310 con 0x7fb72015cfe0 2023-08-25T09:44:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.938+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72006a700 con 0x7fb72015cfe0 2023-08-25T09:44:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.938+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb71800ab50 con 0x7fb72015cfe0 2023-08-25T09:44:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.939+0000 7fb715ffb640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb70406a080 0x7fb70406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:49.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.939+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb718087890 con 0x7fb72015cfe0 2023-08-25T09:44:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.939+0000 7fb7251f3640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb70406a080 0x7fb70406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.940+0000 7fb7251f3640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb70406a080 0x7fb70406c540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006fd0 tx=0x7fb71c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:49.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:49.943+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb718052f70 con 0x7fb72015cfe0 2023-08-25T09:44:50.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.132+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb7200b6cc0 con 0x7fb72015cfe0 2023-08-25T09:44:50.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.133+0000 7fb715ffb640 1 -- 172.21.15.70:0/4122202890 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb718018020 con 0x7fb72015cfe0 2023-08-25T09:44:50.134 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:44:50.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb70406a080 msgr2=0x7fb70406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb70406a080 0x7fb70406c540 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006fd0 tx=0x7fb71c008040 comp rx=0 tx=0).stop 2023-08-25T09:44:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 msgr2=0x7fb72015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb72015f910 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb718001740 tx=0x7fb71800cc50 comp rx=0 tx=0).stop 2023-08-25T09:44:50.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 shutdown_connections 2023-08-25T09:44:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb70406a080 0x7fb70406c540 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb72015cfe0 0x7fb72015f910 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7200cd850 0x7fb72015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 --2- 172.21.15.70:0/4122202890 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7200b8510 0x7fb7201f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.135+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 >> 172.21.15.70:0/4122202890 conn(0x7fb720069850 msgr2=0x7fb7200d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.136+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 shutdown_connections 2023-08-25T09:44:50.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.136+0000 7fb727457640 1 -- 172.21.15.70:0/4122202890 wait complete. 2023-08-25T09:44:50.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:44:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.361+0000 7fe9fef42640 1 Processor -- start 2023-08-25T09:44:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.361+0000 7fe9fef42640 1 -- start start 2023-08-25T09:44:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.362+0000 7fe9fef42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe9f815ec10 0x7fe9f815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9fef42640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f80b1b60 con 0x7fe9f815e830 2023-08-25T09:44:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9fef42640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f806b7c0 con 0x7fe9f815c6d0 2023-08-25T09:44:50.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9fef42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f806bd80 con 0x7fe9f815ec10 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9fccde640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe9f815e830 0x7fe9f806ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44890/0 (socket says 172.21.15.70:44890) 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9fccde640 1 -- 172.21.15.70:0/1509420849 learned_addr learned my addr 172.21.15.70:0/1509420849 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571118321 0 0) 0x7fe9f80b1b60 con 0x7fe9f815e830 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e8003520 con 0x7fe9f815e830 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.363+0000 7fe9f7fff640 1 --2- 172.21.15.70:0/1509420849 >> v2:172.21.15.155:3300/0 conn(0x7fe9f815ec10 0x7fe9f815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087321616 0 0) 0x7fe9e8003520 con 0x7fe9f815e830 2023-08-25T09:44:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 >> v2:172.21.15.155:3300/0 conn(0x7fe9f815ec10 msgr2=0x7fe9f815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 --2- 172.21.15.70:0/1509420849 >> v2:172.21.15.155:3300/0 conn(0x7fe9f815ec10 0x7fe9f815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 >> v1:172.21.15.155:6789/0 conn(0x7fe9f815c6d0 legacy=0x7fe9f80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f806baa0 con 0x7fe9f815e830 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe9ec001670 con 0x7fe9f815e830 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 >> v1:172.21.15.70:6789/0 conn(0x7fe9f815e830 legacy=0x7fe9f806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e8003970 0x7fe9e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e80047c0 0x7fe9e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e80052a0 0x7fe9e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f806bd80 con 0x7fe9e80052a0 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800c170 con 0x7fe9e80047c0 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800c330 con 0x7fe9e8003970 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9fd4df640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e80052a0 0x7fe9e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9fccde640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e8003970 0x7fe9e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.364+0000 7fe9f7fff640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e80047c0 0x7fe9e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.365+0000 7fe9fd4df640 1 -- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e8003970 msgr2=0x7fe9e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.365+0000 7fe9fd4df640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e8003970 0x7fe9e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.365+0000 7fe9fd4df640 1 -- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e80047c0 msgr2=0x7fe9e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.365+0000 7fe9fd4df640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9e80047c0 0x7fe9e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.365+0000 7fe9fd4df640 1 -- 172.21.15.70:0/1509420849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8003520 con 0x7fe9e80052a0 2023-08-25T09:44:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.365+0000 7fe9fd4df640 1 -- 172.21.15.70:0/1509420849 reap_dead start 2023-08-25T09:44:50.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9fd4df640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e80052a0 0x7fe9e800b980 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0001e20 tx=0x7fe9e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f39068e5513863 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0003b70 con 0x7fe9e80052a0 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f0003cd0 con 0x7fe9e80052a0 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9f77fe640 1 -- 172.21.15.70:0/1509420849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0005310 con 0x7fe9e80052a0 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9fef42640 1 -- 172.21.15.70:0/1509420849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e80052a0 msgr2=0x7fe9e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9fef42640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e80052a0 0x7fe9e800b980 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0001e20 tx=0x7fe9e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9fef42640 1 -- 172.21.15.70:0/1509420849 shutdown_connections 2023-08-25T09:44:50.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9fef42640 1 --2- 172.21.15.70:0/1509420849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9e80052a0 0x7fe9e800b980 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.366+0000 7fe9fef42640 1 -- 172.21.15.70:0/1509420849 >> 172.21.15.70:0/1509420849 conn(0x7fe9f8069850 msgr2=0x7fe9f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.367+0000 7fe9fef42640 1 -- 172.21.15.70:0/1509420849 shutdown_connections 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.367+0000 7fe9fef42640 1 -- 172.21.15.70:0/1509420849 wait complete. 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.368+0000 7fe9fef42640 1 Processor -- start 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.368+0000 7fe9fef42640 1 -- start start 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9fef42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e80052a0 0x7fe9f814b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9fef42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 0x7fe9f81f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9fef42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f81f5ec0 0x7fe9f806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9fef42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f81f6340 con 0x7fe9f80ca020 2023-08-25T09:44:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9fef42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80c79d0 con 0x7fe9f81f5ec0 2023-08-25T09:44:50.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9fef42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f80c8360 con 0x7fe9e80052a0 2023-08-25T09:44:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 0x7fe9f81f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9f7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 0x7fe9f81f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38038/0 (socket says 172.21.15.70:38038) 2023-08-25T09:44:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.369+0000 7fe9f7fff640 1 -- 172.21.15.70:0/3060883514 learned_addr learned my addr 172.21.15.70:0/3060883514 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f7fff640 1 -- 172.21.15.70:0/3060883514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e80052a0 msgr2=0x7fe9f814b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f7fff640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e80052a0 0x7fe9f814b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f7fff640 1 -- 172.21.15.70:0/3060883514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f81f5ec0 msgr2=0x7fe9f806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f7fff640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f81f5ec0 0x7fe9f806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f7fff640 1 -- 172.21.15.70:0/3060883514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f80c8920 con 0x7fe9f80ca020 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f7fff640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 0x7fe9f81f5980 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400f510 tx=0x7fe9e400f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4014870 con 0x7fe9f80ca020 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e40149d0 con 0x7fe9f80ca020 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4014d40 con 0x7fe9f80ca020 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.370+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f80c7c20 con 0x7fe9f80ca020 2023-08-25T09:44:50.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.371+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f80c85b0 con 0x7fe9f80ca020 2023-08-25T09:44:50.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.372+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f80c8770 con 0x7fe9f80ca020 2023-08-25T09:44:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.373+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9e4015410 con 0x7fe9f80ca020 2023-08-25T09:44:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.373+0000 7fe9f57fa640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9dc06a080 0x7fe9dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.374+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9e4088e60 con 0x7fe9f80ca020 2023-08-25T09:44:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.374+0000 7fe9fccde640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9dc06a080 0x7fe9dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.374+0000 7fe9fccde640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9dc06a080 0x7fe9dc06c540 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0058d0 tx=0x7fe9ec000f30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.378+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9e4057fe0 con 0x7fe9f80ca020 2023-08-25T09:44:50.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.595+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9f804e420 con 0x7fe9f80ca020 2023-08-25T09:44:50.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.597+0000 7fe9f57fa640 1 -- 172.21.15.70:0/3060883514 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7fe9e4057b70 con 0x7fe9f80ca020 2023-08-25T09:44:50.598 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:50.599 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:44:50.599 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:44:50.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9dc06a080 msgr2=0x7fe9dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9dc06a080 0x7fe9dc06c540 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec0058d0 tx=0x7fe9ec000f30 comp rx=0 tx=0).stop 2023-08-25T09:44:50.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 msgr2=0x7fe9f81f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 0x7fe9f81f5980 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe9e400f510 tx=0x7fe9e400f9d0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 shutdown_connections 2023-08-25T09:44:50.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe9dc06a080 0x7fe9dc06c540 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe9f81f5ec0 0x7fe9f806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe9f80ca020 0x7fe9f81f5980 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 --2- 172.21.15.70:0/3060883514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe9e80052a0 0x7fe9f814b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 >> 172.21.15.70:0/3060883514 conn(0x7fe9f8069850 msgr2=0x7fe9f80ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:50.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.599+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 shutdown_connections 2023-08-25T09:44:50.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.600+0000 7fe9fef42640 1 -- 172.21.15.70:0/3060883514 wait complete. 2023-08-25T09:44:50.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:44:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.825+0000 7f0c53fd6640 1 Processor -- start 2023-08-25T09:44:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.825+0000 7f0c53fd6640 1 -- start start 2023-08-25T09:44:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.826+0000 7f0c53fd6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c4c15e830 0x7f0c4c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.826+0000 7f0c53fd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c53fd6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0b1b60 con 0x7f0c4c15c6d0 2023-08-25T09:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c53fd6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c06bd80 con 0x7f0c4c15e830 2023-08-25T09:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c53fd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c06b7c0 con 0x7f0c4c06b2b0 2023-08-25T09:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c4c15e830 0x7f0c4c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c4c15e830 0x7f0c4c06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38040/0 (socket says 172.21.15.70:38040) 2023-08-25T09:44:50.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 -- 172.21.15.70:0/3316601535 learned_addr learned my addr 172.21.15.70:0/3316601535 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:50.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51571640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 -- 172.21.15.70:0/3316601535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 msgr2=0x7f0c4c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 -- 172.21.15.70:0/3316601535 >> v1:172.21.15.155:6789/0 conn(0x7f0c4c15c6d0 legacy=0x7f0c4c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51d72640 1 -- 172.21.15.70:0/3316601535 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c06baa0 con 0x7f0c4c15e830 2023-08-25T09:44:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.827+0000 7f0c51571640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.829+0000 7f0c51d72640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c06ad70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009870 tx=0x7f0c4802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db67345bbba3a823 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:50.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.829+0000 7f0c50d70640 1 -- 172.21.15.70:0/3316601535 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48031640 con 0x7f0c4c15e830 2023-08-25T09:44:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.829+0000 7f0c50d70640 1 -- 172.21.15.70:0/3316601535 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c480317a0 con 0x7f0c4c15e830 2023-08-25T09:44:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.829+0000 7f0c50d70640 1 -- 172.21.15.70:0/3316601535 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48031b30 con 0x7f0c4c15e830 2023-08-25T09:44:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 -- 172.21.15.70:0/3316601535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 msgr2=0x7f0c4c06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c06ad70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009870 tx=0x7f0c4802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 -- 172.21.15.70:0/3316601535 shutdown_connections 2023-08-25T09:44:50.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/3316601535 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c06ad70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 -- 172.21.15.70:0/3316601535 >> 172.21.15.70:0/3316601535 conn(0x7f0c4c069850 msgr2=0x7f0c4c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 -- 172.21.15.70:0/3316601535 shutdown_connections 2023-08-25T09:44:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.830+0000 7f0c53fd6640 1 -- 172.21.15.70:0/3316601535 wait complete. 2023-08-25T09:44:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.831+0000 7f0c53fd6640 1 Processor -- start 2023-08-25T09:44:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.832+0000 7f0c53fd6640 1 -- start start 2023-08-25T09:44:50.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c53fd6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c53fd6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c4c15c6d0 0x7f0c4c06ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c53fd6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c0c9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c53fd6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c06b0b0 con 0x7f0c4c15e830 2023-08-25T09:44:50.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c53fd6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c1f31d0 con 0x7f0c4c15c6d0 2023-08-25T09:44:50.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c53fd6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c06f220 con 0x7f0c4c06b2b0 2023-08-25T09:44:50.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c0c9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c0c9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38044/0 (socket says 172.21.15.70:38044) 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 -- 172.21.15.70:0/2436445249 learned_addr learned my addr 172.21.15.70:0/2436445249 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c51d72640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 -- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 msgr2=0x7f0c4c14b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c14b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c51571640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c4c15c6d0 0x7f0c4c06ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 -- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c4c15c6d0 msgr2=0x7f0c4c06ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c4c15c6d0 0x7f0c4c06ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.833+0000 7f0c52573640 1 -- 172.21.15.70:0/2436445249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c480094d0 con 0x7f0c4c15e830 2023-08-25T09:44:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.834+0000 7f0c52573640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c0c9c60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c780 tx=0x7f0c4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.834+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44004b60 con 0x7f0c4c15e830 2023-08-25T09:44:50.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.834+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4400ce90 con 0x7f0c4c15e830 2023-08-25T09:44:50.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.834+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400a850 con 0x7f0c4c15e830 2023-08-25T09:44:50.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.834+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c1f2650 con 0x7f0c4c15e830 2023-08-25T09:44:50.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.834+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c1f2c10 con 0x7f0c4c15e830 2023-08-25T09:44:50.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.835+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c04e420 con 0x7f0c4c15e830 2023-08-25T09:44:50.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.836+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0c4400a9b0 con 0x7f0c4c15e830 2023-08-25T09:44:50.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.837+0000 7f0c427fc640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c3006a050 0x7f0c3006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:50.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.837+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c44088770 con 0x7f0c4c15e830 2023-08-25T09:44:50.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.837+0000 7f0c51d72640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c3006a050 0x7f0c3006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:50.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.838+0000 7f0c51d72640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c3006a050 0x7f0c3006c510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0c48036000 tx=0x7f0c4803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:50.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:50.841+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0c44053e50 con 0x7f0c4c15e830 2023-08-25T09:44:51.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.030+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c4c1f5880 con 0x7f0c4c15e830 2023-08-25T09:44:51.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.032+0000 7f0c427fc640 1 -- 172.21.15.70:0/2436445249 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+18392 (secure 0 0 0) 0x7f0c44057470 con 0x7f0c4c15e830 2023-08-25T09:44:51.034 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:51.034 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":102,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:44:49.559347+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["backfillfull","exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:44:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c3006a050 msgr2=0x7f0c3006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c3006a050 0x7f0c3006c510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f0c48036000 tx=0x7f0c4803c040 comp rx=0 tx=0).stop 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 msgr2=0x7f0c4c0c9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c0c9c60 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c780 tx=0x7f0c4400cc40 comp rx=0 tx=0).stop 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 shutdown_connections 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c3006a050 0x7f0c3006c510 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c4c15e830 0x7f0c4c0c9c60 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c4c15c6d0 0x7f0c4c06ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 --2- 172.21.15.70:0/2436445249 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c4c06b2b0 0x7f0c4c14b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 >> 172.21.15.70:0/2436445249 conn(0x7f0c4c069850 msgr2=0x7f0c4c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:51.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 shutdown_connections 2023-08-25T09:44:51.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.035+0000 7f0c53fd6640 1 -- 172.21.15.70:0/2436445249 wait complete. 2023-08-25T09:44:51.054 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:44:51.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:44:51.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.250+0000 7f7e50290640 1 Processor -- start 2023-08-25T09:44:51.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.250+0000 7f7e50290640 1 -- start start 2023-08-25T09:44:51.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e50290640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7e480ac870 0x7f7e480b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e50290640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4806f050 con 0x7f7e4815cb40 2023-08-25T09:44:51.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e50290640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e4806f330 con 0x7f7e480cea30 2023-08-25T09:44:51.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e50290640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e4806f610 con 0x7f7e480ac870 2023-08-25T09:44:51.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e4e02c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7e4815cb40 0x7f7e480ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44898/0 (socket says 172.21.15.70:44898) 2023-08-25T09:44:51.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e4e02c640 1 -- 172.21.15.70:0/4151633182 learned_addr learned my addr 172.21.15.70:0/4151633182 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:51.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1491886194 0 0) 0x7f7e4806f050 con 0x7f7e4815cb40 2023-08-25T09:44:51.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e34003520 con 0x7f7e4815cb40 2023-08-25T09:44:51.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.252+0000 7f7e4e82d640 1 --2- 172.21.15.70:0/4151633182 >> v2:172.21.15.155:3300/0 conn(0x7f7e480ac870 0x7f7e480b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1597437920 0 0) 0x7f7e34003520 con 0x7f7e4815cb40 2023-08-25T09:44:51.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 >> v2:172.21.15.155:3300/0 conn(0x7f7e480ac870 msgr2=0x7f7e480b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 --2- 172.21.15.70:0/4151633182 >> v2:172.21.15.155:3300/0 conn(0x7f7e480ac870 0x7f7e480b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 >> v1:172.21.15.155:6789/0 conn(0x7f7e480cea30 legacy=0x7f7e480b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4806f8f0 con 0x7f7e4815cb40 2023-08-25T09:44:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7e44001670 con 0x7f7e4815cb40 2023-08-25T09:44:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 >> v1:172.21.15.70:6789/0 conn(0x7f7e4815cb40 legacy=0x7f7e480ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e34003970 0x7f7e34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e340047c0 0x7f7e34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e34005180 0x7f7e3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e4806f610 con 0x7f7e34003970 2023-08-25T09:44:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3400c0d0 con 0x7f7e340047c0 2023-08-25T09:44:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3400c290 con 0x7f7e34005180 2023-08-25T09:44:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4e02c640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e34003970 0x7f7e34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4e82d640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e34005180 0x7f7e3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4e02c640 1 -- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e34005180 msgr2=0x7f7e3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4e02c640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e34005180 0x7f7e3400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4e02c640 1 -- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e340047c0 msgr2=0x7f7e34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.253+0000 7f7e4e02c640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e340047c0 0x7f7e34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.254+0000 7f7e4e02c640 1 -- 172.21.15.70:0/4151633182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34003520 con 0x7f7e34003970 2023-08-25T09:44:51.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.254+0000 7f7e4e82d640 1 -- 172.21.15.70:0/4151633182 reap_dead start 2023-08-25T09:44:51.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e4e02c640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e34003970 0x7f7e34003d90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400c240 tx=0x7f7e340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ae06397980a8e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e44005ad0 con 0x7f7e34003970 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e44005c30 con 0x7f7e34003970 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e4d02a640 1 -- 172.21.15.70:0/4151633182 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4402cab0 con 0x7f7e34003970 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 -- 172.21.15.70:0/4151633182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e34003970 msgr2=0x7f7e34003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e34003970 0x7f7e34003d90 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400c240 tx=0x7f7e340057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 -- 172.21.15.70:0/4151633182 shutdown_connections 2023-08-25T09:44:51.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 --2- 172.21.15.70:0/4151633182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e34003970 0x7f7e34003d90 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 -- 172.21.15.70:0/4151633182 >> 172.21.15.70:0/4151633182 conn(0x7f7e48069850 msgr2=0x7f7e48069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 -- 172.21.15.70:0/4151633182 shutdown_connections 2023-08-25T09:44:51.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.255+0000 7f7e50290640 1 -- 172.21.15.70:0/4151633182 wait complete. 2023-08-25T09:44:51.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.257+0000 7f7e50290640 1 Processor -- start 2023-08-25T09:44:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.257+0000 7f7e50290640 1 -- start start 2023-08-25T09:44:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e50290640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e34003970 0x7f7e481538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e50290640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 0x7f7e48150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e50290640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e48151ea0 0x7f7e48152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e50290640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48151a10 con 0x7f7e4814df30 2023-08-25T09:44:51.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e50290640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48150a90 con 0x7f7e34003970 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e50290640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e48150d70 con 0x7f7e48151ea0 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e4d82b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 0x7f7e48150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e4d82b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 0x7f7e48150120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38056/0 (socket says 172.21.15.70:38056) 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.258+0000 7f7e4d82b640 1 -- 172.21.15.70:0/2262239079 learned_addr learned my addr 172.21.15.70:0/2262239079 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e4d82b640 1 -- 172.21.15.70:0/2262239079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e48151ea0 msgr2=0x7f7e48152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e4d82b640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e48151ea0 0x7f7e48152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e4d82b640 1 -- 172.21.15.70:0/2262239079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e34003970 msgr2=0x7f7e481538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e4d82b640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e34003970 0x7f7e481538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e4d82b640 1 -- 172.21.15.70:0/2262239079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48151050 con 0x7f7e4814df30 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e4d82b640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 0x7f7e48150120 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800c500 tx=0x7f7e3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380052d0 con 0x7f7e4814df30 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38005430 con 0x7f7e4814df30 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38012870 con 0x7f7e4814df30 2023-08-25T09:44:51.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e481516a0 con 0x7f7e4814df30 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.259+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e48150660 con 0x7f7e4814df30 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.261+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e4804e420 con 0x7f7e4814df30 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.261+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e380129d0 con 0x7f7e4814df30 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.262+0000 7f7e3effd640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e2c06a080 0x7f7e2c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.262+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e38088150 con 0x7f7e4814df30 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.263+0000 7f7e4e02c640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e2c06a080 0x7f7e2c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.263+0000 7f7e4e02c640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e2c06a080 0x7f7e2c06c540 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e44034000 tx=0x7f7e44039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:51.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.267+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e38057360 con 0x7f7e4814df30 2023-08-25T09:44:51.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.476+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e48151b50 con 0x7f7e4814df30 2023-08-25T09:44:51.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.477+0000 7f7e3effd640 1 -- 172.21.15.70:0/2262239079 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7f7e380011b0 con 0x7f7e4814df30 2023-08-25T09:44:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:51.479 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:44:51.480 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:44:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e2c06a080 msgr2=0x7f7e2c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e2c06a080 0x7f7e2c06c540 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7e44034000 tx=0x7f7e44039040 comp rx=0 tx=0).stop 2023-08-25T09:44:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 msgr2=0x7f7e48150120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 0x7f7e48150120 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800c500 tx=0x7f7e3800c9c0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 shutdown_connections 2023-08-25T09:44:51.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e2c06a080 0x7f7e2c06c540 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e48151ea0 0x7f7e48152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e4814df30 0x7f7e48150120 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 --2- 172.21.15.70:0/2262239079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e34003970 0x7f7e481538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 >> 172.21.15.70:0/2262239079 conn(0x7f7e48069850 msgr2=0x7f7e480afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 shutdown_connections 2023-08-25T09:44:51.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.480+0000 7f7e50290640 1 -- 172.21.15.70:0/2262239079 wait complete. 2023-08-25T09:44:51.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:44:51.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.701+0000 7fa7509c1640 1 Processor -- start 2023-08-25T09:44:51.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.701+0000 7fa7509c1640 1 -- start start 2023-08-25T09:44:51.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa7509c1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa74c0d1d30 0x7fa74c0d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa7509c1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74c0d26d0 con 0x7fa74c15cc60 2023-08-25T09:44:51.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa7509c1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74c0d04a0 con 0x7fa74c0b5820 2023-08-25T09:44:51.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa7509c1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c15d040 con 0x7fa74c0d1d30 2023-08-25T09:44:51.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa74ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa74c15cc60 0x7fa74c0cf400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44900/0 (socket says 172.21.15.70:44900) 2023-08-25T09:44:51.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa74ad9d640 1 -- 172.21.15.70:0/1874858519 learned_addr learned my addr 172.21.15.70:0/1874858519 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:51.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467203167 0 0) 0x7fa74c0d26d0 con 0x7fa74c15cc60 2023-08-25T09:44:51.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.703+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa734003520 con 0x7fa74c15cc60 2023-08-25T09:44:51.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1874858519 >> v2:172.21.15.155:3300/0 conn(0x7fa74c0d1d30 0x7fa74c0d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 555686139 0 0) 0x7fa734003520 con 0x7fa74c15cc60 2023-08-25T09:44:51.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 >> v2:172.21.15.155:3300/0 conn(0x7fa74c0d1d30 msgr2=0x7fa74c0d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 --2- 172.21.15.70:0/1874858519 >> v2:172.21.15.155:3300/0 conn(0x7fa74c0d1d30 0x7fa74c0d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 >> v1:172.21.15.155:6789/0 conn(0x7fa74c0b5820 legacy=0x7fa74c0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c0d0780 con 0x7fa74c15cc60 2023-08-25T09:44:51.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa73c001670 con 0x7fa74c15cc60 2023-08-25T09:44:51.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 >> v1:172.21.15.70:6789/0 conn(0x7fa74c15cc60 legacy=0x7fa74c0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa734003970 0x7fa734003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa734004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa734005180 0x7fa73400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c15d040 con 0x7fa7340047c0 2023-08-25T09:44:51.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73400c0b0 con 0x7fa734003970 2023-08-25T09:44:51.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73400c270 con 0x7fa734005180 2023-08-25T09:44:51.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.704+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa734004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74ad9d640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa734003970 0x7fa734003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74a59c640 1 -- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa734005180 msgr2=0x7fa73400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa734005180 0x7fa73400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74a59c640 1 -- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa734003970 msgr2=0x7fa734003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:51.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa734003970 0x7fa734003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74a59c640 1 -- 172.21.15.70:0/1874858519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa734003520 con 0x7fa7340047c0 2023-08-25T09:44:51.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.705+0000 7fa74b59e640 1 -- 172.21.15.70:0/1874858519 reap_dead start 2023-08-25T09:44:51.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.706+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa734004c40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa74000b9e0 tx=0x7fa7340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf85aaab62fc7d6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:51.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.706+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740005b90 con 0x7fa7340047c0 2023-08-25T09:44:51.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.706+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740005cf0 con 0x7fa7340047c0 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.706+0000 7fa749d9b640 1 -- 172.21.15.70:0/1874858519 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740006270 con 0x7fa7340047c0 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 -- 172.21.15.70:0/1874858519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 msgr2=0x7fa734004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa734004c40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fa74000b9e0 tx=0x7fa7340057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 -- 172.21.15.70:0/1874858519 shutdown_connections 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1874858519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa734004c40 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 -- 172.21.15.70:0/1874858519 >> 172.21.15.70:0/1874858519 conn(0x7fa74c069850 msgr2=0x7fa74c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:51.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 -- 172.21.15.70:0/1874858519 shutdown_connections 2023-08-25T09:44:51.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.707+0000 7fa7509c1640 1 -- 172.21.15.70:0/1874858519 wait complete. 2023-08-25T09:44:51.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.709+0000 7fa7509c1640 1 Processor -- start 2023-08-25T09:44:51.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.709+0000 7fa7509c1640 1 -- start start 2023-08-25T09:44:51.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa7509c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa74c15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa7509c1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c144f30 0x7fa74c0b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa7509c1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c0c99a0 0x7fa74c0c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa7509c1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0ca360 con 0x7fa7340047c0 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa7509c1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c15e3a0 con 0x7fa74c144f30 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa7509c1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c15e960 con 0x7fa74c0c99a0 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa74c15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa74c15da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38074/0 (socket says 172.21.15.70:38074) 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 -- 172.21.15.70:0/1638625823 learned_addr learned my addr 172.21.15.70:0/1638625823 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 -- 172.21.15.70:0/1638625823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c0c99a0 msgr2=0x7fa74c0c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c0c99a0 0x7fa74c0c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 -- 172.21.15.70:0/1638625823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c144f30 msgr2=0x7fa74c0b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c144f30 0x7fa74c0b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.710+0000 7fa74ad9d640 1 -- 172.21.15.70:0/1638625823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c15df50 con 0x7fa7340047c0 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.711+0000 7fa74ad9d640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa74c15da10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00e7b0 tx=0x7fa73c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.711+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00ee60 con 0x7fa7340047c0 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.711+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c00c870 con 0x7fa7340047c0 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.711+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00cc20 con 0x7fa7340047c0 2023-08-25T09:44:51.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.711+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0cc580 con 0x7fa7340047c0 2023-08-25T09:44:51.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.711+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c0ccb40 con 0x7fa7340047c0 2023-08-25T09:44:51.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.712+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c0ccd00 con 0x7fa7340047c0 2023-08-25T09:44:51.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.713+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa73c01d460 con 0x7fa7340047c0 2023-08-25T09:44:51.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.714+0000 7fa7337fe640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa73806a050 0x7fa73806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:51.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.714+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa73c0883d0 con 0x7fa7340047c0 2023-08-25T09:44:51.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.714+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa73806a050 0x7fa73806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:51.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.715+0000 7fa74a59c640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa73806a050 0x7fa73806c510 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa740036000 tx=0x7fa740031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:51.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.718+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa73c053b50 con 0x7fa7340047c0 2023-08-25T09:44:51.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.947+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa74c065fe0 con 0x7fa7340047c0 2023-08-25T09:44:51.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.949+0000 7fa7337fe640 1 -- 172.21.15.70:0/1638625823 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4874 (secure 0 0 0) 0x7fa73c057170 con 0x7fa7340047c0 2023-08-25T09:44:51.951 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:51.951 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:43:53.868790+0000","modified":"2023-08-25T09:43:57.423437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1195200165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":1195200165},{"type":"v1","addr":"172.21.15.70:6837","nonce":1195200165}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-08-25T09:44:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 41 2023-08-25T09:44:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa73806a050 msgr2=0x7fa73806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa73806a050 0x7fa73806c510 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa740036000 tx=0x7fa740031000 comp rx=0 tx=0).stop 2023-08-25T09:44:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 msgr2=0x7fa74c15da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:51.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa74c15da10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00e7b0 tx=0x7fa73c00ec70 comp rx=0 tx=0).stop 2023-08-25T09:44:51.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 shutdown_connections 2023-08-25T09:44:51.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa73806a050 0x7fa73806c510 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa74c0c99a0 0x7fa74c0c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa74c144f30 0x7fa74c0b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 --2- 172.21.15.70:0/1638625823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7340047c0 0x7fa74c15da10 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:51.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 >> 172.21.15.70:0/1638625823 conn(0x7fa74c069850 msgr2=0x7fa74c1551e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:51.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 shutdown_connections 2023-08-25T09:44:51.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:51.952+0000 7fa7509c1640 1 -- 172.21.15.70:0/1638625823 wait complete. 2023-08-25T09:44:51.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:44:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.181+0000 7f75bf0c2640 1 Processor -- start 2023-08-25T09:44:52.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.182+0000 7f75bf0c2640 1 -- start start 2023-08-25T09:44:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.183+0000 7f75bf0c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75b80cad20 0x7f75b80cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.183+0000 7f75bf0c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b815e7c0 con 0x7f75b80b12f0 2023-08-25T09:44:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.183+0000 7f75bf0c2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b8071b50 con 0x7f75b815a7c0 2023-08-25T09:44:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.183+0000 7f75bf0c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b80cb6f0 con 0x7f75b80cad20 2023-08-25T09:44:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.183+0000 7f75bd8bf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f75b80b12f0 0x7f75b815ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44910/0 (socket says 172.21.15.70:44910) 2023-08-25T09:44:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.183+0000 7f75bd8bf640 1 -- 172.21.15.70:0/2814614926 learned_addr learned my addr 172.21.15.70:0/2814614926 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75be8c1640 1 --2- 172.21.15.70:0/2814614926 >> v2:172.21.15.70:3300/0 conn(0x7f75b80cad20 0x7f75b80cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75bd0be640 1 -- 172.21.15.70:0/2814614926 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971490820 0 0) 0x7f75b815e7c0 con 0x7f75b80b12f0 2023-08-25T09:44:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75bd0be640 1 -- 172.21.15.70:0/2814614926 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a4003520 con 0x7f75b80b12f0 2023-08-25T09:44:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75be8c1640 1 -- 172.21.15.70:0/2814614926 >> v1:172.21.15.155:6789/0 conn(0x7f75b815a7c0 legacy=0x7f75b80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75be8c1640 1 -- 172.21.15.70:0/2814614926 >> v1:172.21.15.70:6789/0 conn(0x7f75b80b12f0 legacy=0x7f75b815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75bd0be640 1 -- 172.21.15.70:0/2814614926 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4170719087 0 0) 0x7f75a4003520 con 0x7f75b80b12f0 2023-08-25T09:44:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.184+0000 7f75be8c1640 1 -- 172.21.15.70:0/2814614926 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b815e490 con 0x7f75b80cad20 2023-08-25T09:44:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.186+0000 7f75be8c1640 1 --2- 172.21.15.70:0/2814614926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b80cb1b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009d90 tx=0x7f75b002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f17b5f1fef9d98 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.186+0000 7f75bd0be640 1 -- 172.21.15.70:0/2814614926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8001670 con 0x7f75b80cad20 2023-08-25T09:44:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.186+0000 7f75bd0be640 1 -- 172.21.15.70:0/2814614926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b002c940 con 0x7f75b80cad20 2023-08-25T09:44:52.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.186+0000 7f75bd0be640 1 -- 172.21.15.70:0/2814614926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b002ccd0 con 0x7f75b80cad20 2023-08-25T09:44:52.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 -- 172.21.15.70:0/2814614926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 msgr2=0x7f75b80cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/2814614926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b80cb1b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009d90 tx=0x7f75b002be10 comp rx=0 tx=0).stop 2023-08-25T09:44:52.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 -- 172.21.15.70:0/2814614926 shutdown_connections 2023-08-25T09:44:52.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/2814614926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b80cb1b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 -- 172.21.15.70:0/2814614926 >> 172.21.15.70:0/2814614926 conn(0x7f75b80affd0 msgr2=0x7f75b80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:52.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 -- 172.21.15.70:0/2814614926 shutdown_connections 2023-08-25T09:44:52.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.187+0000 7f75bf0c2640 1 -- 172.21.15.70:0/2814614926 wait complete. 2023-08-25T09:44:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.189+0000 7f75bf0c2640 1 Processor -- start 2023-08-25T09:44:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.189+0000 7f75bf0c2640 1 -- start start 2023-08-25T09:44:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bf0c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b80b12f0 0x7f75b8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bf0c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b815c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bf0c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b815a7c0 0x7f75b815cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bf0c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b815b890 con 0x7f75b80cad20 2023-08-25T09:44:52.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bf0c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b815bb70 con 0x7f75b80b12f0 2023-08-25T09:44:52.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bf0c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b815be50 con 0x7f75b815a7c0 2023-08-25T09:44:52.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bd8bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bd8bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b815c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38100/0 (socket says 172.21.15.70:38100) 2023-08-25T09:44:52.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bd8bf640 1 -- 172.21.15.70:0/3610134271 learned_addr learned my addr 172.21.15.70:0/3610134271 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:52.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75be0c0640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b80b12f0 0x7f75b8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75bd8bf640 1 -- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b815a7c0 msgr2=0x7f75b815cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.190+0000 7f75be8c1640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b815a7c0 0x7f75b815cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bd8bf640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b815a7c0 0x7f75b815cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bd8bf640 1 -- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b80b12f0 msgr2=0x7f75b8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bd8bf640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b80b12f0 0x7f75b8159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bd8bf640 1 -- 172.21.15.70:0/3610134271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0009a70 con 0x7f75b80cad20 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bd8bf640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b815c980 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c780 tx=0x7f75a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8005ad0 con 0x7f75b80cad20 2023-08-25T09:44:52.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a8005c30 con 0x7f75b80cad20 2023-08-25T09:44:52.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b815c130 con 0x7f75b80cad20 2023-08-25T09:44:52.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a8007070 con 0x7f75b80cad20 2023-08-25T09:44:52.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.191+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b815b580 con 0x7f75b80cad20 2023-08-25T09:44:52.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.192+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b81f1490 con 0x7f75b80cad20 2023-08-25T09:44:52.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.193+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f75a80071d0 con 0x7f75b80cad20 2023-08-25T09:44:52.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.194+0000 7f75aeffd640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f759c06a050 0x7f759c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.194+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f75a8088420 con 0x7f75b80cad20 2023-08-25T09:44:52.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.195+0000 7f75be0c0640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f759c06a050 0x7f759c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.195+0000 7f75be0c0640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f759c06a050 0x7f759c06c510 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000f60 tx=0x7f75b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:52.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.198+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f75a80539e0 con 0x7f75b80cad20 2023-08-25T09:44:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.388+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75b8066fb0 con 0x7f75b80cad20 2023-08-25T09:44:52.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.391+0000 7f75aeffd640 1 -- 172.21.15.70:0/3610134271 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f75a8057000 con 0x7f75b80cad20 2023-08-25T09:44:52.393 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:52.393 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":103,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:44:51.574183+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:43:52.155883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-08-25T09:43:52.872267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:44:52.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f759c06a050 msgr2=0x7f759c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f759c06a050 0x7f759c06c510 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f75b4000f60 tx=0x7f75b4009040 comp rx=0 tx=0).stop 2023-08-25T09:44:52.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 msgr2=0x7f75b815c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b815c980 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f75a800c780 tx=0x7f75a800cc40 comp rx=0 tx=0).stop 2023-08-25T09:44:52.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 shutdown_connections 2023-08-25T09:44:52.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f759c06a050 0x7f759c06c510 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75b815a7c0 0x7f75b815cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75b80cad20 0x7f75b815c980 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 --2- 172.21.15.70:0/3610134271 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75b80b12f0 0x7f75b8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 >> 172.21.15.70:0/3610134271 conn(0x7f75b80affd0 msgr2=0x7f75b80ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:52.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 shutdown_connections 2023-08-25T09:44:52.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.394+0000 7f75bf0c2640 1 -- 172.21.15.70:0/3610134271 wait complete. 2023-08-25T09:44:52.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:44:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.625+0000 7f968eb60640 1 Processor -- start 2023-08-25T09:44:52.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.625+0000 7f968eb60640 1 -- start start 2023-08-25T09:44:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968eb60640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 0x7f96881657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968eb60640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968815e4a0 con 0x7f968815abb0 2023-08-25T09:44:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968eb60640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968815f020 con 0x7f968815a7d0 2023-08-25T09:44:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968eb60640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c53b0 con 0x7f968806fa80 2023-08-25T09:44:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968d35d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f968815abb0 0x7f96880b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44912/0 (socket says 172.21.15.70:44912) 2023-08-25T09:44:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968d35d640 1 -- 172.21.15.70:0/2350966433 learned_addr learned my addr 172.21.15.70:0/2350966433 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.627+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749852258 0 0) 0x7f968815e4a0 con 0x7f968815abb0 2023-08-25T09:44:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9674003520 con 0x7f968815abb0 2023-08-25T09:44:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968e35f640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 0x7f96881657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4264741030 0 0) 0x7f9674003520 con 0x7f968815abb0 2023-08-25T09:44:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 msgr2=0x7f96881657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 0x7f96881657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 >> v1:172.21.15.155:6789/0 conn(0x7f968815a7d0 legacy=0x7f96880b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968815e6f0 con 0x7f968815abb0 2023-08-25T09:44:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9678001670 con 0x7f968815abb0 2023-08-25T09:44:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 >> v1:172.21.15.70:6789/0 conn(0x7f968815abb0 legacy=0x7f96880b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 0x7f9674003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9674004020 0x7f96740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f967400fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9674005610 con 0x7f9674004e20 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96740102d0 con 0x7f9674004020 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9674010410 con 0x7f968806fa80 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.628+0000 7f968e35f640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f967400fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.629+0000 7f968e35f640 1 -- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 msgr2=0x7f9674003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.629+0000 7f968e35f640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f968806fa80 0x7f9674003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.629+0000 7f968e35f640 1 -- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9674004020 msgr2=0x7f96740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.629+0000 7f968e35f640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9674004020 0x7f96740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.629+0000 7f968e35f640 1 -- 172.21.15.70:0/2350966433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674003520 con 0x7f9674004e20 2023-08-25T09:44:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.630+0000 7f968e35f640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f967400fd90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9680007330 tx=0x7f96740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4198db6631e4fb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.630+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968002ccd0 con 0x7f9674004e20 2023-08-25T09:44:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.630+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f968002ce30 con 0x7f9674004e20 2023-08-25T09:44:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.630+0000 7f968cb5c640 1 -- 172.21.15.70:0/2350966433 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680034680 con 0x7f9674004e20 2023-08-25T09:44:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968eb60640 1 -- 172.21.15.70:0/2350966433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 msgr2=0x7f967400fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968eb60640 1 --2- 172.21.15.70:0/2350966433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f967400fd90 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9680007330 tx=0x7f96740057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968e35f640 1 -- 172.21.15.70:0/2350966433 reap_dead start 2023-08-25T09:44:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968eb60640 1 -- 172.21.15.70:0/2350966433 shutdown_connections 2023-08-25T09:44:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968eb60640 1 -- 172.21.15.70:0/2350966433 >> 172.21.15.70:0/2350966433 conn(0x7f96880aff60 msgr2=0x7f96880b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968eb60640 1 -- 172.21.15.70:0/2350966433 shutdown_connections 2023-08-25T09:44:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.631+0000 7f968eb60640 1 -- 172.21.15.70:0/2350966433 wait complete. 2023-08-25T09:44:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.633+0000 7f968eb60640 1 Processor -- start 2023-08-25T09:44:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.633+0000 7f968eb60640 1 -- start start 2023-08-25T09:44:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968eb60640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f96881512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968eb60640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96880c7c80 0x7f9688156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968eb60640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96880c8100 0x7f96881565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968eb60640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c6be0 con 0x7f9674004e20 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968eb60640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c6ec0 con 0x7f96880c7c80 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968eb60640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880c71a0 con 0x7f96880c8100 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968db5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f96881512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968db5e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f96881512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38128/0 (socket says 172.21.15.70:38128) 2023-08-25T09:44:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.634+0000 7f968db5e640 1 -- 172.21.15.70:0/3886638383 learned_addr learned my addr 172.21.15.70:0/3886638383 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968d35d640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96880c7c80 0x7f9688156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968e35f640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96880c8100 0x7f96881565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968db5e640 1 -- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96880c8100 msgr2=0x7f96881565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968db5e640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96880c8100 0x7f96881565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968db5e640 1 -- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96880c7c80 msgr2=0x7f9688156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968db5e640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96880c7c80 0x7f9688156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968db5e640 1 -- 172.21.15.70:0/3886638383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880c7480 con 0x7f9674004e20 2023-08-25T09:44:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968db5e640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f96881512a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f968400c410 tx=0x7f968400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684011790 con 0x7f9674004e20 2023-08-25T09:44:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96840118f0 con 0x7f9674004e20 2023-08-25T09:44:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96880c7760 con 0x7f9674004e20 2023-08-25T09:44:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9684011ce0 con 0x7f9674004e20 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.635+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968815afa0 con 0x7f9674004e20 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.637+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96880c7950 con 0x7f9674004e20 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.637+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f968401c460 con 0x7f9674004e20 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.638+0000 7f967e7fc640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f966c06a080 0x7f966c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.638+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96840882e0 con 0x7f9674004e20 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.639+0000 7f968d35d640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f966c06a080 0x7f966c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:52.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.639+0000 7f968d35d640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f966c06a080 0x7f966c06c540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9688156d40 tx=0x7f96780015a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:52.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.643+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9684053930 con 0x7f9674004e20 2023-08-25T09:44:52.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.868+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9688066fb0 con 0x7f9674004e20 2023-08-25T09:44:52.870 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:44:52.869+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:44:52.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.874+0000 7f967e7fc640 1 -- 172.21.15.70:0/3886638383 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9684056f50 con 0x7f9674004e20 2023-08-25T09:44:52.876 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:44:52.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f966c06a080 msgr2=0x7f966c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f966c06a080 0x7f966c06c540 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9688156d40 tx=0x7f96780015a0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 msgr2=0x7f96881512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:52.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f96881512a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f968400c410 tx=0x7f968400c8d0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 shutdown_connections 2023-08-25T09:44:52.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f966c06a080 0x7f966c06c540 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f96880c8100 0x7f96881565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f96880c7c80 0x7f9688156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 --2- 172.21.15.70:0/3886638383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9674004e20 0x7f96881512a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:52.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 >> 172.21.15.70:0/3886638383 conn(0x7f96880aff60 msgr2=0x7f96881521d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:52.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 shutdown_connections 2023-08-25T09:44:52.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:52.877+0000 7f968eb60640 1 -- 172.21.15.70:0/3886638383 wait complete. 2023-08-25T09:44:52.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:44:52.987 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:44:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.095+0000 7fb7c9b5b640 1 Processor -- start 2023-08-25T09:44:53.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.095+0000 7fb7c9b5b640 1 -- start start 2023-08-25T09:44:53.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.097+0000 7fb7c9b5b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7c406ec40 0x7fb7c4165540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.097+0000 7fb7c9b5b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb7c41611f0 0x7fb7c4161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.097+0000 7fb7c9b5b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c406c8f0 con 0x7fb7c40b3660 2023-08-25T09:44:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.097+0000 7fb7c9b5b640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c406c6e0 con 0x7fb7c406ec40 2023-08-25T09:44:53.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.097+0000 7fb7c9b5b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c4165a80 con 0x7fb7c41611f0 2023-08-25T09:44:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7c406ec40 0x7fb7c4165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7bbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb7c40b3660 0x7fb7c406f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44928/0 (socket says 172.21.15.70:44928) 2023-08-25T09:44:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7bbfff640 1 -- 172.21.15.70:0/2991862606 learned_addr learned my addr 172.21.15.70:0/2991862606 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:53.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c3fff640 1 --2- 172.21.15.70:0/2991862606 >> v2:172.21.15.155:3300/0 conn(0x7fb7c41611f0 0x7fb7c4161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c37fe640 1 -- 172.21.15.70:0/2991862606 >> v2:172.21.15.155:3300/0 conn(0x7fb7c41611f0 msgr2=0x7fb7c4161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c37fe640 1 --2- 172.21.15.70:0/2991862606 >> v2:172.21.15.155:3300/0 conn(0x7fb7c41611f0 0x7fb7c4161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c37fe640 1 -- 172.21.15.70:0/2991862606 >> v1:172.21.15.70:6789/0 conn(0x7fb7c40b3660 legacy=0x7fb7c406f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c2ffd640 1 -- 172.21.15.70:0/2991862606 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441993610 0 0) 0x7fb7c406c8f0 con 0x7fb7c40b3660 2023-08-25T09:44:53.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.098+0000 7fb7c37fe640 1 -- 172.21.15.70:0/2991862606 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c406ac10 con 0x7fb7c406ec40 2023-08-25T09:44:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.099+0000 7fb7c37fe640 1 --2- 172.21.15.70:0/2991862606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c406ec40 0x7fb7c4165540 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40b3c60 tx=0x7fb7b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a890d178d52953a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c2ffd640 1 -- 172.21.15.70:0/2991862606 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4031830 con 0x7fb7c406ec40 2023-08-25T09:44:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c2ffd640 1 -- 172.21.15.70:0/2991862606 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b4031990 con 0x7fb7c406ec40 2023-08-25T09:44:53.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c2ffd640 1 -- 172.21.15.70:0/2991862606 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4031d20 con 0x7fb7c406ec40 2023-08-25T09:44:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/2991862606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c406ec40 msgr2=0x7fb7c4165540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/2991862606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c406ec40 0x7fb7c4165540 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40b3c60 tx=0x7fb7b402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/2991862606 shutdown_connections 2023-08-25T09:44:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/2991862606 >> v2:172.21.15.155:3300/0 conn(0x7fb7c41611f0 0x7fb7c4161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/2991862606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c406ec40 0x7fb7c4165540 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.100+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/2991862606 >> 172.21.15.70:0/2991862606 conn(0x7fb7c4069850 msgr2=0x7fb7c4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.101+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/2991862606 shutdown_connections 2023-08-25T09:44:53.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.101+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/2991862606 wait complete. 2023-08-25T09:44:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.102+0000 7fb7c9b5b640 1 Processor -- start 2023-08-25T09:44:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.102+0000 7fb7c9b5b640 1 -- start start 2023-08-25T09:44:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.103+0000 7fb7c9b5b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7c406ec40 0x7fb7c41579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.103+0000 7fb7c9b5b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7c40b3660 0x7fb7c415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:53.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.103+0000 7fb7c9b5b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 0x7fb7c415b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c9b5b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415e570 con 0x7fb7c41611f0 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c9b5b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415e850 con 0x7fb7c406ec40 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c9b5b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c415eb30 con 0x7fb7c40b3660 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 0x7fb7c415b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 0x7fb7c415b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38144/0 (socket says 172.21.15.70:38144) 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 -- 172.21.15.70:0/698059275 learned_addr learned my addr 172.21.15.70:0/698059275 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:53.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c37fe640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7c406ec40 0x7fb7c41579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 -- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7c40b3660 msgr2=0x7fb7c415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7bbfff640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7c40b3660 0x7fb7c415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7c40b3660 0x7fb7c415f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 -- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7c406ec40 msgr2=0x7fb7c41579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7c406ec40 0x7fb7c41579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c3fff640 1 -- 172.21.15.70:0/698059275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4009400 con 0x7fb7c41611f0 2023-08-25T09:44:53.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.104+0000 7fb7c37fe640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7c406ec40 0x7fb7c41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.105+0000 7fb7c3fff640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 0x7fb7c415b470 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007e60 tx=0x7fb7bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.105+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc014420 con 0x7fb7c41611f0 2023-08-25T09:44:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.105+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc0145a0 con 0x7fb7c41611f0 2023-08-25T09:44:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.105+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c415ee10 con 0x7fb7c41611f0 2023-08-25T09:44:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.105+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc014950 con 0x7fb7c41611f0 2023-08-25T09:44:53.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.105+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c4157ef0 con 0x7fb7c41611f0 2023-08-25T09:44:53.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.106+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c404e420 con 0x7fb7c41611f0 2023-08-25T09:44:53.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.107+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb7bc014ab0 con 0x7fb7c41611f0 2023-08-25T09:44:53.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.108+0000 7fb7c0ff9640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7a406a050 0x7fb7a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.108+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fb7bc088f60 con 0x7fb7c41611f0 2023-08-25T09:44:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.108+0000 7fb7c37fe640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7a406a050 0x7fb7a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:53.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.109+0000 7fb7c37fe640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7a406a050 0x7fb7a406c510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4036000 tx=0x7fb7b4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:53.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.112+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7bc058070 con 0x7fb7c41611f0 2023-08-25T09:44:53.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.337+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb7c4161f80 con 0x7fb7c41611f0 2023-08-25T09:44:53.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.879+0000 7fb7c0ff9640 1 -- 172.21.15.70:0/698059275 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7bc057c00 con 0x7fb7c41611f0 2023-08-25T09:44:53.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.882+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7a406a050 msgr2=0x7fb7a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.882+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7a406a050 0x7fb7a406c510 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4036000 tx=0x7fb7b4008030 comp rx=0 tx=0).stop 2023-08-25T09:44:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.882+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 msgr2=0x7fb7c415b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.882+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 0x7fb7c415b470 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc007e60 tx=0x7fb7bc00e4b0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 shutdown_connections 2023-08-25T09:44:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7a406a050 0x7fb7a406c510 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7c41611f0 0x7fb7c415b470 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7c40b3660 0x7fb7c415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 --2- 172.21.15.70:0/698059275 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7c406ec40 0x7fb7c41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 >> 172.21.15.70:0/698059275 conn(0x7fb7c4069850 msgr2=0x7fb7c415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 shutdown_connections 2023-08-25T09:44:53.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:53.883+0000 7fb7c9b5b640 1 -- 172.21.15.70:0/698059275 wait complete. 2023-08-25T09:44:53.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:44:54.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.122+0000 7f10829c2640 1 Processor -- start 2023-08-25T09:44:54.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.122+0000 7f10829c2640 1 -- start start 2023-08-25T09:44:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f10829c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f107c155120 0x7f107c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f10829c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0cd890 0x7f107c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f10829c2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c0cacb0 con 0x7f107c0ca820 2023-08-25T09:44:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f10829c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107c0caf30 con 0x7f107c155120 2023-08-25T09:44:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f10829c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0cdd10 con 0x7f107c0cd890 2023-08-25T09:44:54.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f107bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f107c155120 0x7f107c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f107bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f107c155120 0x7f107c0cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38150/0 (socket says 172.21.15.70:38150) 2023-08-25T09:44:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f107bfff640 1 -- 172.21.15.70:0/404714674 learned_addr learned my addr 172.21.15.70:0/404714674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:54.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.124+0000 7f107b7fe640 1 --2- 172.21.15.70:0/404714674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0cd890 0x7f107c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.125+0000 7f107bfff640 1 -- 172.21.15.70:0/404714674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0cd890 msgr2=0x7f107c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.125+0000 7f107bfff640 1 --2- 172.21.15.70:0/404714674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0cd890 0x7f107c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.125+0000 7f107bfff640 1 -- 172.21.15.70:0/404714674 >> v1:172.21.15.155:6789/0 conn(0x7f107c0ca820 legacy=0x7f107c06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.125+0000 7f107bfff640 1 -- 172.21.15.70:0/404714674 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c15e6f0 con 0x7f107c155120 2023-08-25T09:44:54.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.126+0000 7f107bfff640 1 --2- 172.21.15.70:0/404714674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c155120 0x7f107c0cd350 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f106c009870 tx=0x7f106c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed32893d6cd9e9b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.126+0000 7f107affd640 1 -- 172.21.15.70:0/404714674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c030840 con 0x7f107c155120 2023-08-25T09:44:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.126+0000 7f107affd640 1 -- 172.21.15.70:0/404714674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c0309a0 con 0x7f107c155120 2023-08-25T09:44:54.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.126+0000 7f107affd640 1 -- 172.21.15.70:0/404714674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c030d30 con 0x7f107c155120 2023-08-25T09:44:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 -- 172.21.15.70:0/404714674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c155120 msgr2=0x7f107c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 --2- 172.21.15.70:0/404714674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c155120 0x7f107c0cd350 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f106c009870 tx=0x7f106c02b980 comp rx=0 tx=0).stop 2023-08-25T09:44:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 -- 172.21.15.70:0/404714674 shutdown_connections 2023-08-25T09:44:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 --2- 172.21.15.70:0/404714674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0cd890 0x7f107c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 --2- 172.21.15.70:0/404714674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c155120 0x7f107c0cd350 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 -- 172.21.15.70:0/404714674 >> 172.21.15.70:0/404714674 conn(0x7f107c069c70 msgr2=0x7f107c06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 -- 172.21.15.70:0/404714674 shutdown_connections 2023-08-25T09:44:54.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.127+0000 7f10829c2640 1 -- 172.21.15.70:0/404714674 wait complete. 2023-08-25T09:44:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.129+0000 7f10829c2640 1 Processor -- start 2023-08-25T09:44:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.129+0000 7f10829c2640 1 -- start start 2023-08-25T09:44:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f10829c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0ca820 0x7f107c1515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:54.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f10829c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 0x7f107c1f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f10829c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f107c155120 0x7f107c0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f10829c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c158e60 con 0x7f107c0cd890 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f10829c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c1f3290 con 0x7f107c155120 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f10829c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0cb520 con 0x7f107c0ca820 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f107b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 0x7f107c1f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f107b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 0x7f107c1f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38164/0 (socket says 172.21.15.70:38164) 2023-08-25T09:44:54.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f107b7fe640 1 -- 172.21.15.70:0/1909590811 learned_addr learned my addr 172.21.15.70:0/1909590811 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f107bfff640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0ca820 0x7f107c1515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f107b7fe640 1 -- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0ca820 msgr2=0x7f107c1515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.130+0000 7f107b7fe640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0ca820 0x7f107c1515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f107b7fe640 1 -- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f107c155120 msgr2=0x7f107c0c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f107b7fe640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f107c155120 0x7f107c0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f107b7fe640 1 -- 172.21.15.70:0/1909590811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c0094d0 con 0x7f107c0cd890 2023-08-25T09:44:54.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f107b7fe640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 0x7f107c1f2d50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f107000d800 tx=0x7f107000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:54.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107000b6c0 con 0x7f107c0cd890 2023-08-25T09:44:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107000b820 con 0x7f107c0cd890 2023-08-25T09:44:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107000bbb0 con 0x7f107c0cd890 2023-08-25T09:44:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c0cb800 con 0x7f107c0cd890 2023-08-25T09:44:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.131+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107c0cbdc0 con 0x7f107c0cd890 2023-08-25T09:44:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.132+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107c04e420 con 0x7f107c0cd890 2023-08-25T09:44:54.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.133+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f107000bd10 con 0x7f107c0cd890 2023-08-25T09:44:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.134+0000 7f1078ff9640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f106806a050 0x7f106806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.134+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f1070088b30 con 0x7f107c0cd890 2023-08-25T09:44:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.134+0000 7f107bfff640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f106806a050 0x7f106806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:54.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.135+0000 7f107bfff640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f106806a050 0x7f106806c510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f106c036000 tx=0x7f106c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:54.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.138+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10700541b0 con 0x7f107c0cd890 2023-08-25T09:44:54.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.327+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f107c15a910 con 0x7f107c0cd890 2023-08-25T09:44:54.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.899+0000 7f1078ff9640 1 -- 172.21.15.70:0/1909590811 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) v1 ==== 181+0+0 (secure 0 0 0) 0x7f10700577d0 con 0x7f107c0cd890 2023-08-25T09:44:54.902 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:44:54.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f106806a050 msgr2=0x7f106806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:54.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f106806a050 0x7f106806c510 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f106c036000 tx=0x7f106c03b040 comp rx=0 tx=0).stop 2023-08-25T09:44:54.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 msgr2=0x7f107c1f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:54.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 0x7f107c1f2d50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f107000d800 tx=0x7f107000dcc0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 shutdown_connections 2023-08-25T09:44:54.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f106806a050 0x7f106806c510 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f107c155120 0x7f107c0c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f107c0cd890 0x7f107c1f2d50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 --2- 172.21.15.70:0/1909590811 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f107c0ca820 0x7f107c1515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:54.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.905+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 >> 172.21.15.70:0/1909590811 conn(0x7f107c069c70 msgr2=0x7f107c0ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:54.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.906+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 shutdown_connections 2023-08-25T09:44:54.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:54.906+0000 7f10829c2640 1 -- 172.21.15.70:0/1909590811 wait complete. 2023-08-25T09:44:54.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:44:55.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.141+0000 7fa2ad3f3640 1 Processor -- start 2023-08-25T09:44:55.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.141+0000 7fa2ad3f3640 1 -- start start 2023-08-25T09:44:55.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.142+0000 7fa2ad3f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa2a8142c30 0x7fa2a80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.142+0000 7fa2ad3f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 0x7fa2a806af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.142+0000 7fa2ad3f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a815d320 con 0x7fa2a8142740 2023-08-25T09:44:55.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.142+0000 7fa2ad3f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a806b440 con 0x7fa2a8142c30 2023-08-25T09:44:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.142+0000 7fa2ad3f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80ccd40 con 0x7fa2a806aa70 2023-08-25T09:44:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.143+0000 7fa2a6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa2a8142740 0x7fa2a815db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44932/0 (socket says 172.21.15.70:44932) 2023-08-25T09:44:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.143+0000 7fa2a6ffd640 1 -- 172.21.15.70:0/3177697830 learned_addr learned my addr 172.21.15.70:0/3177697830 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.143+0000 7fa29ffff640 1 --2- 172.21.15.70:0/3177697830 >> v2:172.21.15.155:3300/0 conn(0x7fa2a8142c30 0x7fa2a80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.143+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 0x7fa2a806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.143+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940203070 0 0) 0x7fa2a815d320 con 0x7fa2a8142740 2023-08-25T09:44:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.143+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa294003520 con 0x7fa2a8142740 2023-08-25T09:44:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1352666856 0 0) 0x7fa294003520 con 0x7fa2a8142740 2023-08-25T09:44:55.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 msgr2=0x7fa2a806af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 0x7fa2a806af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 >> v2:172.21.15.155:3300/0 conn(0x7fa2a8142c30 msgr2=0x7fa2a80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 --2- 172.21.15.70:0/3177697830 >> v2:172.21.15.155:3300/0 conn(0x7fa2a8142c30 0x7fa2a80cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a80cd020 con 0x7fa2a8142740 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa298001670 con 0x7fa2a8142740 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 >> v1:172.21.15.70:6789/0 conn(0x7fa2a8142740 legacy=0x7fa2a815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 0x7fa294003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa294003ef0 0x7fa2940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa294004d00 0x7fa294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a806b440 con 0x7fa294004d00 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80ccd40 con 0x7fa294003ef0 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa29400db90 con 0x7fa2a806aa70 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa294004d00 0x7fa294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa29ffff640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa294003ef0 0x7fa2940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.144+0000 7fa2a6ffd640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 0x7fa294003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.145+0000 7fa2a77fe640 1 -- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 msgr2=0x7fa294003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.145+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa2a806aa70 0x7fa294003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.145+0000 7fa2a77fe640 1 -- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa294003ef0 msgr2=0x7fa2940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.145+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa294003ef0 0x7fa2940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.145+0000 7fa2a77fe640 1 -- 172.21.15.70:0/3177697830 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294003520 con 0x7fa294004d00 2023-08-25T09:44:55.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa294004d00 0x7fa294005180 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fc0 tx=0x7fa2940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be11beea05f7977 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:55.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a002cc00 con 0x7fa294004d00 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a000f040 con 0x7fa294004d00 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2a67fc640 1 -- 172.21.15.70:0/3177697830 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0034440 con 0x7fa294004d00 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/3177697830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa294004d00 msgr2=0x7fa294005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/3177697830 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa294004d00 0x7fa294005180 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009fc0 tx=0x7fa2940057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.146+0000 7fa2a77fe640 1 -- 172.21.15.70:0/3177697830 reap_dead start 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.147+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/3177697830 shutdown_connections 2023-08-25T09:44:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.147+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/3177697830 >> 172.21.15.70:0/3177697830 conn(0x7fa2a8069c70 msgr2=0x7fa2a806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.147+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/3177697830 shutdown_connections 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.147+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/3177697830 wait complete. 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.148+0000 7fa2ad3f3640 1 Processor -- start 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.149+0000 7fa2ad3f3640 1 -- start start 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.149+0000 7fa2ad3f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa294004d00 0x7fa2a8158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2ad3f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8136a10 0x7fa2a8136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2ad3f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 0x7fa2a80b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2ad3f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a815b5e0 con 0x7fa2a80b28c0 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2ad3f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80b3280 con 0x7fa2a8136a10 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2ad3f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a815ba00 con 0x7fa294004d00 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 0x7fa2a80b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 0x7fa2a80b2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38182/0 (socket says 172.21.15.70:38182) 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2929058989 learned_addr learned my addr 172.21.15.70:0/2929058989 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a6ffd640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa294004d00 0x7fa2a8158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa29ffff640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8136a10 0x7fa2a8136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa294004d00 msgr2=0x7fa2a8158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa294004d00 0x7fa2a8158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8136a10 msgr2=0x7fa2a8136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8136a10 0x7fa2a8136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa2a77fe640 1 -- 172.21.15.70:0/2929058989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a815ac40 con 0x7fa2a80b28c0 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.150+0000 7fa29ffff640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8136a10 0x7fa2a8136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:55.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.151+0000 7fa2a77fe640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 0x7fa2a80b2d40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0037040 tx=0x7fa2a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:55.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.151+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0034ad0 con 0x7fa2a80b28c0 2023-08-25T09:44:55.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.151+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a0034c30 con 0x7fa2a80b28c0 2023-08-25T09:44:55.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.151+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0003990 con 0x7fa2a80b28c0 2023-08-25T09:44:55.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.151+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a815af20 con 0x7fa2a80b28c0 2023-08-25T09:44:55.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.151+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a8138ab0 con 0x7fa2a80b28c0 2023-08-25T09:44:55.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.152+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a815b720 con 0x7fa2a80b28c0 2023-08-25T09:44:55.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.153+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa2a0047460 con 0x7fa2a80b28c0 2023-08-25T09:44:55.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.154+0000 7fa29f7fe640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa28806a080 0x7fa28806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:55.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.154+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fa2a00aa2b0 con 0x7fa2a80b28c0 2023-08-25T09:44:55.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.154+0000 7fa2a6ffd640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa28806a080 0x7fa28806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:55.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.155+0000 7fa2a6ffd640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa28806a080 0x7fa28806c540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa298001eb0 tx=0x7fa298001dc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:55.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.158+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa2a815b720 con 0x7fa2a80b28c0 2023-08-25T09:44:55.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.346+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa2a815b720 con 0x7fa2a80b28c0 2023-08-25T09:44:55.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.904+0000 7fa29f7fe640 1 -- 172.21.15.70:0/2929058989 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa2a815b720 con 0x7fa2a80b28c0 2023-08-25T09:44:55.911 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:44:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa28806a080 msgr2=0x7fa28806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa28806a080 0x7fa28806c540 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa298001eb0 tx=0x7fa298001dc0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 msgr2=0x7fa2a80b2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 0x7fa2a80b2d40 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0037040 tx=0x7fa2a0009750 comp rx=0 tx=0).stop 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 shutdown_connections 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa28806a080 0x7fa28806c540 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2a80b28c0 0x7fa2a80b2d40 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2a8136a10 0x7fa2a8136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 --2- 172.21.15.70:0/2929058989 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa294004d00 0x7fa2a8158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 >> 172.21.15.70:0/2929058989 conn(0x7fa2a8069c70 msgr2=0x7fa2a813d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.912+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 shutdown_connections 2023-08-25T09:44:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:55.913+0000 7fa2ad3f3640 1 -- 172.21.15.70:0/2929058989 wait complete. 2023-08-25T09:44:55.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:44:56.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.149+0000 7f854440d640 1 Processor -- start 2023-08-25T09:44:56.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.149+0000 7f854440d640 1 -- start start 2023-08-25T09:44:56.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.150+0000 7f854440d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f853c15ae90 0x7f853c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.150+0000 7f854440d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c15be60 con 0x7f853c06f300 2023-08-25T09:44:56.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.150+0000 7f854440d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853c06ee70 con 0x7f853c0ce410 2023-08-25T09:44:56.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.150+0000 7f854440d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853c0ced30 con 0x7f853c15ae90 2023-08-25T09:44:56.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85419a8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f853c15ae90 0x7f853c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85429aa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f853c06f300 0x7f853c06aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44946/0 (socket says 172.21.15.70:44946) 2023-08-25T09:44:56.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85429aa640 1 -- 172.21.15.70:0/3745561466 learned_addr learned my addr 172.21.15.70:0/3745561466 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:56.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85419a8640 1 -- 172.21.15.70:0/3745561466 >> v1:172.21.15.155:6789/0 conn(0x7f853c0ce410 legacy=0x7f853c070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:56.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85419a8640 1 -- 172.21.15.70:0/3745561466 >> v1:172.21.15.70:6789/0 conn(0x7f853c06f300 legacy=0x7f853c06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85419a8640 1 -- 172.21.15.70:0/3745561466 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c15b2f0 con 0x7f853c15ae90 2023-08-25T09:44:56.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.151+0000 7f85411a7640 1 -- 172.21.15.70:0/3745561466 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350238181 0 0) 0x7f853c15be60 con 0x7f853c06f300 2023-08-25T09:44:56.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.152+0000 7f85419a8640 1 --2- 172.21.15.70:0/3745561466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c0ce7f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f852c0097a0 tx=0x7f852c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c30726df090c2f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f85411a7640 1 -- 172.21.15.70:0/3745561466 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c031830 con 0x7f853c15ae90 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f85411a7640 1 -- 172.21.15.70:0/3745561466 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c031990 con 0x7f853c15ae90 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f85411a7640 1 -- 172.21.15.70:0/3745561466 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c031d20 con 0x7f853c15ae90 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f854440d640 1 -- 172.21.15.70:0/3745561466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 msgr2=0x7f853c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f854440d640 1 --2- 172.21.15.70:0/3745561466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c0ce7f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f852c0097a0 tx=0x7f852c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f854440d640 1 -- 172.21.15.70:0/3745561466 shutdown_connections 2023-08-25T09:44:56.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f854440d640 1 --2- 172.21.15.70:0/3745561466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c0ce7f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.153+0000 7f854440d640 1 -- 172.21.15.70:0/3745561466 >> 172.21.15.70:0/3745561466 conn(0x7f853c069850 msgr2=0x7f853c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.154+0000 7f854440d640 1 -- 172.21.15.70:0/3745561466 shutdown_connections 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.154+0000 7f854440d640 1 -- 172.21.15.70:0/3745561466 wait complete. 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.155+0000 7f854440d640 1 Processor -- start 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.155+0000 7f854440d640 1 -- start start 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.156+0000 7f854440d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f853c06f300 0x7f853c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.156+0000 7f854440d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c0ce410 0x7f853c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.156+0000 7f854440d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c14cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.156+0000 7f854440d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c14d2f0 con 0x7f853c15ae90 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.156+0000 7f854440d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c14d590 con 0x7f853c0ce410 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.156+0000 7f854440d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c14d870 con 0x7f853c06f300 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c14cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c14cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38214/0 (socket says 172.21.15.70:38214) 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 -- 172.21.15.70:0/490798463 learned_addr learned my addr 172.21.15.70:0/490798463 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:56.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 -- 172.21.15.70:0/490798463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f853c06f300 msgr2=0x7f853c14f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f853c06f300 0x7f853c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 -- 172.21.15.70:0/490798463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c0ce410 msgr2=0x7f853c0ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c0ce410 0x7f853c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 -- 172.21.15.70:0/490798463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c009400 con 0x7f853c15ae90 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.157+0000 7f85429aa640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c14cdb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f853400c7e0 tx=0x7f853400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.158+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400a640 con 0x7f853c15ae90 2023-08-25T09:44:56.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.158+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853400a7c0 con 0x7f853c15ae90 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.158+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400ab70 con 0x7f853c15ae90 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.158+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c14db50 con 0x7f853c15ae90 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.158+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c14e110 con 0x7f853c15ae90 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.159+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c14e300 con 0x7f853c15ae90 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.160+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f853400acd0 con 0x7f853c15ae90 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.161+0000 7f8532ffd640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f852006a050 0x7f852006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.161+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f85340878f0 con 0x7f853c15ae90 2023-08-25T09:44:56.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.161+0000 7f85421a9640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f852006a050 0x7f852006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.162+0000 7f85421a9640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f852006a050 0x7f852006c510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8538003cc0 tx=0x7f85380079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:56.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.165+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8534053260 con 0x7f853c15ae90 2023-08-25T09:44:56.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.353+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f853c04e420 con 0x7f853c15ae90 2023-08-25T09:44:56.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.360+0000 7f8532ffd640 1 -- 172.21.15.70:0/490798463 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f85340180a0 con 0x7f853c15ae90 2023-08-25T09:44:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f852006a050 msgr2=0x7f852006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f852006a050 0x7f852006c510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8538003cc0 tx=0x7f85380079b0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 msgr2=0x7f853c14cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c14cdb0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f853400c7e0 tx=0x7f853400cca0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 shutdown_connections 2023-08-25T09:44:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f852006a050 0x7f852006c510 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f853c15ae90 0x7f853c14cdb0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f853c0ce410 0x7f853c0ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.362+0000 7f854440d640 1 --2- 172.21.15.70:0/490798463 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f853c06f300 0x7f853c14f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.363+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 >> 172.21.15.70:0/490798463 conn(0x7f853c069850 msgr2=0x7f853c15e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:56.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.363+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 shutdown_connections 2023-08-25T09:44:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.363+0000 7f854440d640 1 -- 172.21.15.70:0/490798463 wait complete. 2023-08-25T09:44:56.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-08-25T09:44:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.595+0000 7fd2b0814640 1 Processor -- start 2023-08-25T09:44:56.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.596+0000 7fd2b0814640 1 -- start start 2023-08-25T09:44:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.597+0000 7fd2b0814640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2ac15cb40 0x7fd2ac0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.597+0000 7fd2b0814640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac0ac870 0x7fd2ac0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.597+0000 7fd2b0814640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac06f050 con 0x7fd2ac0b3880 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.597+0000 7fd2b0814640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac06f330 con 0x7fd2ac15cb40 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.597+0000 7fd2b0814640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac06f610 con 0x7fd2ac0ac870 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2ac15cb40 0x7fd2ac0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd2ac15cb40 0x7fd2ac0ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38230/0 (socket says 172.21.15.70:38230) 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 -- 172.21.15.70:0/1285375656 learned_addr learned my addr 172.21.15.70:0/1285375656 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 -- 172.21.15.70:0/1285375656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac0ac870 msgr2=0x7fd2ac0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:56.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2a9d9b640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac0ac870 0x7fd2ac0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac0ac870 0x7fd2ac0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 -- 172.21.15.70:0/1285375656 >> v1:172.21.15.155:6789/0 conn(0x7fd2ac0b3880 legacy=0x7fd2ac160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:56.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2aa59c640 1 -- 172.21.15.70:0/1285375656 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac06f8f0 con 0x7fd2ac15cb40 2023-08-25T09:44:56.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.598+0000 7fd2a9d9b640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac0ac870 0x7fd2ac0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:56.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.599+0000 7fd2aa59c640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac15cb40 0x7fd2ac0ce320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009e50 tx=0x7fd29c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb16bf2a9869c510 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:56.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2a959a640 1 -- 172.21.15.70:0/1285375656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c02ca80 con 0x7fd2ac15cb40 2023-08-25T09:44:56.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2a959a640 1 -- 172.21.15.70:0/1285375656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c02cbe0 con 0x7fd2ac15cb40 2023-08-25T09:44:56.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2a959a640 1 -- 172.21.15.70:0/1285375656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0343f0 con 0x7fd2ac15cb40 2023-08-25T09:44:56.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2b0814640 1 -- 172.21.15.70:0/1285375656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac15cb40 msgr2=0x7fd2ac0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2b0814640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac15cb40 0x7fd2ac0ce320 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009e50 tx=0x7fd29c02bee0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2b0814640 1 -- 172.21.15.70:0/1285375656 shutdown_connections 2023-08-25T09:44:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2b0814640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac0ac870 0x7fd2ac0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2b0814640 1 --2- 172.21.15.70:0/1285375656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac15cb40 0x7fd2ac0ce320 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.600+0000 7fd2b0814640 1 -- 172.21.15.70:0/1285375656 >> 172.21.15.70:0/1285375656 conn(0x7fd2ac069850 msgr2=0x7fd2ac069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:56.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.601+0000 7fd2b0814640 1 -- 172.21.15.70:0/1285375656 shutdown_connections 2023-08-25T09:44:56.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.601+0000 7fd2b0814640 1 -- 172.21.15.70:0/1285375656 wait complete. 2023-08-25T09:44:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.603+0000 7fd2b0814640 1 Processor -- start 2023-08-25T09:44:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.603+0000 7fd2b0814640 1 -- start start 2023-08-25T09:44:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2b0814640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 0x7fd2ac150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2b0814640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2ac0b3880 0x7fd2ac15d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2b0814640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac15cb40 0x7fd2ac151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2b0814640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac15e7e0 con 0x7fd2ac0ac870 2023-08-25T09:44:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2b0814640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac15db60 con 0x7fd2ac0b3880 2023-08-25T09:44:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2b0814640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac15de40 con 0x7fd2ac15cb40 2023-08-25T09:44:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2aa59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 0x7fd2ac150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2aa59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 0x7fd2ac150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38238/0 (socket says 172.21.15.70:38238) 2023-08-25T09:44:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.604+0000 7fd2aa59c640 1 -- 172.21.15.70:0/343383206 learned_addr learned my addr 172.21.15.70:0/343383206 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2aa59c640 1 -- 172.21.15.70:0/343383206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac15cb40 msgr2=0x7fd2ac151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2aa59c640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac15cb40 0x7fd2ac151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2aa59c640 1 -- 172.21.15.70:0/343383206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2ac0b3880 msgr2=0x7fd2ac15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2aa59c640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2ac0b3880 0x7fd2ac15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2aa59c640 1 -- 172.21.15.70:0/343383206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009b30 con 0x7fd2ac0ac870 2023-08-25T09:44:56.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2aa59c640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 0x7fd2ac150230 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009e50 tx=0x7fd29c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:56.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c03e450 con 0x7fd2ac0ac870 2023-08-25T09:44:56.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c03e5f0 con 0x7fd2ac0ac870 2023-08-25T09:44:56.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac15e120 con 0x7fd2ac0ac870 2023-08-25T09:44:56.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.605+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c03e9a0 con 0x7fd2ac0ac870 2023-08-25T09:44:56.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.606+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac150770 con 0x7fd2ac0ac870 2023-08-25T09:44:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.607+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac04e420 con 0x7fd2ac0ac870 2023-08-25T09:44:56.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.608+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd29c02cd50 con 0x7fd2ac0ac870 2023-08-25T09:44:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.609+0000 7fd2a2ffd640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd28c06a050 0x7fd28c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.609+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd29c0aaea0 con 0x7fd2ac0ac870 2023-08-25T09:44:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.609+0000 7fd2a9d9b640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd28c06a050 0x7fd28c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:56.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.610+0000 7fd2a9d9b640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd28c06a050 0x7fd28c06c510 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd294006fd0 tx=0x7fd294008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:56.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.613+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd29c0769c0 con 0x7fd2ac0ac870 2023-08-25T09:44:56.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.884+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7fd2ac15df80 con 0x7fd2ac0ac870 2023-08-25T09:44:56.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.901+0000 7fd2a2ffd640 1 -- 172.21.15.70:0/343383206 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v327) v1 ==== 133+0+0 (secure 0 0 0) 0x7fd2ac15df80 con 0x7fd2ac0ac870 2023-08-25T09:44:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd28c06a050 msgr2=0x7fd28c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd28c06a050 0x7fd28c06c510 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd294006fd0 tx=0x7fd294008040 comp rx=0 tx=0).stop 2023-08-25T09:44:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 msgr2=0x7fd2ac150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:56.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 0x7fd2ac150230 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009e50 tx=0x7fd29c004a40 comp rx=0 tx=0).stop 2023-08-25T09:44:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 shutdown_connections 2023-08-25T09:44:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd28c06a050 0x7fd28c06c510 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd2ac15cb40 0x7fd2ac151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd2ac0b3880 0x7fd2ac15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 --2- 172.21.15.70:0/343383206 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd2ac0ac870 0x7fd2ac150230 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:56.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 >> 172.21.15.70:0/343383206 conn(0x7fd2ac069850 msgr2=0x7fd2ac160720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 shutdown_connections 2023-08-25T09:44:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:56.904+0000 7fd2b0814640 1 -- 172.21.15.70:0/343383206 wait complete. 2023-08-25T09:44:56.924 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-08-25T09:44:56.925 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-08-25T09:44:56.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:44:57.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.132+0000 7f07b9962640 1 Processor -- start 2023-08-25T09:44:57.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.132+0000 7f07b9962640 1 -- start start 2023-08-25T09:44:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b9962640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b415e830 0x7f07b406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b9962640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07b415ec10 0x7f07b40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b9962640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b40b1b60 con 0x7f07b406b2b0 2023-08-25T09:44:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b9962640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b406bd80 con 0x7f07b415ec10 2023-08-25T09:44:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b9962640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b406b7c0 con 0x7f07b415e830 2023-08-25T09:44:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07b415ec10 0x7f07b40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07b415ec10 0x7f07b40b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38248/0 (socket says 172.21.15.70:38248) 2023-08-25T09:44:57.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.134+0000 7f07b37fe640 1 -- 172.21.15.70:0/2427302448 learned_addr learned my addr 172.21.15.70:0/2427302448 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:57.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b2ffd640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b415e830 0x7f07b406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b37fe640 1 -- 172.21.15.70:0/2427302448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b415e830 msgr2=0x7f07b406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b37fe640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b415e830 0x7f07b406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b37fe640 1 -- 172.21.15.70:0/2427302448 >> v1:172.21.15.70:6789/0 conn(0x7f07b406b2b0 legacy=0x7f07b415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b1ffb640 1 -- 172.21.15.70:0/2427302448 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185850673 0 0) 0x7f07b40b1b60 con 0x7f07b406b2b0 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b37fe640 1 -- 172.21.15.70:0/2427302448 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b406baa0 con 0x7f07b415ec10 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.135+0000 7f07b2ffd640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b415e830 0x7f07b406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:57.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.136+0000 7f07b37fe640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415ec10 0x7f07b40b1ff0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009870 tx=0x7f07ac02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0967b6c2cab4741 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:57.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.136+0000 7f07b1ffb640 1 -- 172.21.15.70:0/2427302448 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac030840 con 0x7f07b415ec10 2023-08-25T09:44:57.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.136+0000 7f07b1ffb640 1 -- 172.21.15.70:0/2427302448 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac0309a0 con 0x7f07b415ec10 2023-08-25T09:44:57.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.136+0000 7f07b1ffb640 1 -- 172.21.15.70:0/2427302448 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac030d30 con 0x7f07b415ec10 2023-08-25T09:44:57.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 -- 172.21.15.70:0/2427302448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415ec10 msgr2=0x7f07b40b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415ec10 0x7f07b40b1ff0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009870 tx=0x7f07ac02b980 comp rx=0 tx=0).stop 2023-08-25T09:44:57.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 -- 172.21.15.70:0/2427302448 shutdown_connections 2023-08-25T09:44:57.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415ec10 0x7f07b40b1ff0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 --2- 172.21.15.70:0/2427302448 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b415e830 0x7f07b406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 -- 172.21.15.70:0/2427302448 >> 172.21.15.70:0/2427302448 conn(0x7f07b4069850 msgr2=0x7f07b4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:57.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 -- 172.21.15.70:0/2427302448 shutdown_connections 2023-08-25T09:44:57.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.137+0000 7f07b9962640 1 -- 172.21.15.70:0/2427302448 wait complete. 2023-08-25T09:44:57.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.139+0000 7f07b9962640 1 Processor -- start 2023-08-25T09:44:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.139+0000 7f07b9962640 1 -- start start 2023-08-25T09:44:57.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b9962640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b406b2b0 0x7f07b41f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b9962640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 0x7f07b4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b9962640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b415ec10 0x7f07b415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b9962640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40a87f0 con 0x7f07b415e830 2023-08-25T09:44:57.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b9962640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b41f2040 con 0x7f07b415ec10 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b9962640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40c98c0 con 0x7f07b406b2b0 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 0x7f07b4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 0x7f07b4070720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38250/0 (socket says 172.21.15.70:38250) 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.140+0000 7f07b27fc640 1 -- 172.21.15.70:0/40838536 learned_addr learned my addr 172.21.15.70:0/40838536 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b27fc640 1 -- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b406b2b0 msgr2=0x7f07b41f1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b2ffd640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b406b2b0 0x7f07b41f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b37fe640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b415ec10 0x7f07b415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b27fc640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b406b2b0 0x7f07b41f1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b27fc640 1 -- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b415ec10 msgr2=0x7f07b415cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b27fc640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b415ec10 0x7f07b415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b27fc640 1 -- 172.21.15.70:0/40838536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac0094d0 con 0x7f07b415e830 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b2ffd640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b406b2b0 0x7f07b41f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b27fc640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 0x7f07b4070720 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f079c00c8c0 tx=0x7f079c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:57.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c003da0 con 0x7f07b415e830 2023-08-25T09:44:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c005070 con 0x7f07b415e830 2023-08-25T09:44:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c005420 con 0x7f07b415e830 2023-08-25T09:44:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b415ca20 con 0x7f07b415e830 2023-08-25T09:44:57.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.141+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b40c9e80 con 0x7f07b415e830 2023-08-25T09:44:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.142+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b404e420 con 0x7f07b415e830 2023-08-25T09:44:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.143+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f079c005580 con 0x7f07b415e830 2023-08-25T09:44:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.144+0000 7f07abfff640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f079406a050 0x7f079406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.145+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f079c087b90 con 0x7f07b415e830 2023-08-25T09:44:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.145+0000 7f07b2ffd640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f079406a050 0x7f079406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.145+0000 7f07b2ffd640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f079406a050 0x7f079406c510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f07a4006fd0 tx=0x7f07a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:57.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.148+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f079c053590 con 0x7f07b415e830 2023-08-25T09:44:57.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.339+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07b4156900 con 0x7f07b415e830 2023-08-25T09:44:57.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.341+0000 7f07abfff640 1 -- 172.21.15.70:0/40838536 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+14786 (secure 0 0 0) 0x7f079c017070 con 0x7f07b415e830 2023-08-25T09:44:57.343 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:44:57.343 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":106,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:44:55.887227+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6837/1195200165":"2023-08-26T09:44:52.869385+0000","172.21.15.70:6836/1195200165":"2023-08-26T09:44:52.869385+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:44:57.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f079406a050 msgr2=0x7f079406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f079406a050 0x7f079406c510 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f07a4006fd0 tx=0x7f07a4008040 comp rx=0 tx=0).stop 2023-08-25T09:44:57.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 msgr2=0x7f07b4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 0x7f07b4070720 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f079c00c8c0 tx=0x7f079c00cd80 comp rx=0 tx=0).stop 2023-08-25T09:44:57.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 shutdown_connections 2023-08-25T09:44:57.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f079406a050 0x7f079406c510 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07b415ec10 0x7f07b415cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07b415e830 0x7f07b4070720 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 --2- 172.21.15.70:0/40838536 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07b406b2b0 0x7f07b41f1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 >> 172.21.15.70:0/40838536 conn(0x7f07b4069850 msgr2=0x7f07b40cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:57.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 shutdown_connections 2023-08-25T09:44:57.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.344+0000 7f07b9962640 1 -- 172.21.15.70:0/40838536 wait complete. 2023-08-25T09:44:57.374 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:44:57.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:44:57.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.603+0000 7fb58b50f640 1 Processor -- start 2023-08-25T09:44:57.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.603+0000 7fb58b50f640 1 -- start start 2023-08-25T09:44:57.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.605+0000 7fb58b50f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb58415ae50 0x7fb5840c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.605+0000 7fb58b50f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840cc5f0 con 0x7fb584069f60 2023-08-25T09:44:57.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.605+0000 7fb58b50f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb584070650 con 0x7fb5840cdc20 2023-08-25T09:44:57.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.605+0000 7fb58b50f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb584070930 con 0x7fb58415ae50 2023-08-25T09:44:57.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.605+0000 7fb589aac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb584069f60 0x7fb5840ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44954/0 (socket says 172.21.15.70:44954) 2023-08-25T09:44:57.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.605+0000 7fb589aac640 1 -- 172.21.15.70:0/114645293 learned_addr learned my addr 172.21.15.70:0/114645293 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:57.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb5892ab640 1 --2- 172.21.15.70:0/114645293 >> v2:172.21.15.155:3300/0 conn(0x7fb58415ae50 0x7fb5840c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551161371 0 0) 0x7fb5840cc5f0 con 0x7fb584069f60 2023-08-25T09:44:57.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb570003520 con 0x7fb584069f60 2023-08-25T09:44:57.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2847302854 0 0) 0x7fb570003520 con 0x7fb584069f60 2023-08-25T09:44:57.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 >> v2:172.21.15.155:3300/0 conn(0x7fb58415ae50 msgr2=0x7fb5840c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:57.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 --2- 172.21.15.70:0/114645293 >> v2:172.21.15.155:3300/0 conn(0x7fb58415ae50 0x7fb5840c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 >> v1:172.21.15.155:6789/0 conn(0x7fb5840cdc20 legacy=0x7fb584069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584070c10 con 0x7fb584069f60 2023-08-25T09:44:57.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb57c0019f0 con 0x7fb584069f60 2023-08-25T09:44:57.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 >> v1:172.21.15.70:6789/0 conn(0x7fb584069f60 legacy=0x7fb5840ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb570003970 0x7fb570003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5700047c0 0x7fb570004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb570005180 0x7fb57000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb584070930 con 0x7fb5700047c0 2023-08-25T09:44:57.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57000c030 con 0x7fb570003970 2023-08-25T09:44:57.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.606+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57000c1f0 con 0x7fb570005180 2023-08-25T09:44:57.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb588aaa640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5700047c0 0x7fb570004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb5892ab640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb570003970 0x7fb570003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb588aaa640 1 -- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb570005180 msgr2=0x7fb57000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:57.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb588aaa640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb570005180 0x7fb57000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb588aaa640 1 -- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb570003970 msgr2=0x7fb570003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:57.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb588aaa640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb570003970 0x7fb570003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb588aaa640 1 -- 172.21.15.70:0/114645293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570003520 con 0x7fb5700047c0 2023-08-25T09:44:57.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.607+0000 7fb589aac640 1 -- 172.21.15.70:0/114645293 reap_dead start 2023-08-25T09:44:57.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.608+0000 7fb588aaa640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5700047c0 0x7fb570004c40 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb574001750 tx=0x7fb5700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701591486f95d3ac server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:57.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.608+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5740052d0 con 0x7fb5700047c0 2023-08-25T09:44:57.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.608+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57402e050 con 0x7fb5700047c0 2023-08-25T09:44:57.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.608+0000 7fb57bfff640 1 -- 172.21.15.70:0/114645293 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb574003830 con 0x7fb5700047c0 2023-08-25T09:44:57.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 -- 172.21.15.70:0/114645293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5700047c0 msgr2=0x7fb570004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5700047c0 0x7fb570004c40 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb574001750 tx=0x7fb5700057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 -- 172.21.15.70:0/114645293 shutdown_connections 2023-08-25T09:44:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 --2- 172.21.15.70:0/114645293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb5700047c0 0x7fb570004c40 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 -- 172.21.15.70:0/114645293 >> 172.21.15.70:0/114645293 conn(0x7fb5840cb0d0 msgr2=0x7fb5840cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:57.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 -- 172.21.15.70:0/114645293 shutdown_connections 2023-08-25T09:44:57.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.609+0000 7fb58b50f640 1 -- 172.21.15.70:0/114645293 wait complete. 2023-08-25T09:44:57.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.611+0000 7fb58b50f640 1 Processor -- start 2023-08-25T09:44:57.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.611+0000 7fb58b50f640 1 -- start start 2023-08-25T09:44:57.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb58b50f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5700047c0 0x7fb58415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb58b50f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5840c70a0 0x7fb58415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb58b50f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 0x7fb58415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb58b50f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5841eff60 con 0x7fb58415dba0 2023-08-25T09:44:57.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb58b50f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840c7670 con 0x7fb5840c70a0 2023-08-25T09:44:57.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb58b50f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840c7950 con 0x7fb5700047c0 2023-08-25T09:44:57.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb589aac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 0x7fb58415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb589aac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 0x7fb58415b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38266/0 (socket says 172.21.15.70:38266) 2023-08-25T09:44:57.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.612+0000 7fb589aac640 1 -- 172.21.15.70:0/4227072954 learned_addr learned my addr 172.21.15.70:0/4227072954 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:57.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb589aac640 1 -- 172.21.15.70:0/4227072954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5700047c0 msgr2=0x7fb58415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb589aac640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5700047c0 0x7fb58415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb589aac640 1 -- 172.21.15.70:0/4227072954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5840c70a0 msgr2=0x7fb58415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb589aac640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5840c70a0 0x7fb58415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb589aac640 1 -- 172.21.15.70:0/4227072954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840c7c30 con 0x7fb58415dba0 2023-08-25T09:44:57.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb589aac640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 0x7fb58415b9e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001560 tx=0x7fb57c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:57.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00cea0 con 0x7fb58415dba0 2023-08-25T09:44:57.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00a610 con 0x7fb58415dba0 2023-08-25T09:44:57.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00a9c0 con 0x7fb58415dba0 2023-08-25T09:44:57.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5840c7f10 con 0x7fb58415dba0 2023-08-25T09:44:57.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.613+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5840acf40 con 0x7fb58415dba0 2023-08-25T09:44:57.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.615+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5840701c0 con 0x7fb58415dba0 2023-08-25T09:44:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.615+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb57c00ab20 con 0x7fb58415dba0 2023-08-25T09:44:57.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.617+0000 7fb579ffb640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb56806a050 0x7fb56806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.617+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb57c087ad0 con 0x7fb58415dba0 2023-08-25T09:44:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.617+0000 7fb5892ab640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb56806a050 0x7fb56806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.618+0000 7fb5892ab640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb56806a050 0x7fb56806c510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5840a92a0 tx=0x7fb580008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:57.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.621+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb57c053550 con 0x7fb58415dba0 2023-08-25T09:44:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.810+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5841f00b0 con 0x7fb58415dba0 2023-08-25T09:44:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.811+0000 7fb579ffb640 1 -- 172.21.15.70:0/4227072954 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5841f00b0 con 0x7fb58415dba0 2023-08-25T09:44:57.813 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:44:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb56806a050 msgr2=0x7fb56806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb56806a050 0x7fb56806c510 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb5840a92a0 tx=0x7fb580008040 comp rx=0 tx=0).stop 2023-08-25T09:44:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 msgr2=0x7fb58415b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 0x7fb58415b9e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb57c001560 tx=0x7fb57c00cc50 comp rx=0 tx=0).stop 2023-08-25T09:44:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 shutdown_connections 2023-08-25T09:44:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb56806a050 0x7fb56806c510 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb58415dba0 0x7fb58415b9e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5840c70a0 0x7fb58415d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.813+0000 7fb58b50f640 1 --2- 172.21.15.70:0/4227072954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb5700047c0 0x7fb58415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.814+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 >> 172.21.15.70:0/4227072954 conn(0x7fb5840cb0d0 msgr2=0x7fb5840cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.814+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 shutdown_connections 2023-08-25T09:44:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:57.814+0000 7fb58b50f640 1 -- 172.21.15.70:0/4227072954 wait complete. 2023-08-25T09:44:57.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:44:58.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.056+0000 7f8982d79640 1 Processor -- start 2023-08-25T09:44:58.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.056+0000 7f8982d79640 1 -- start start 2023-08-25T09:44:58.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.057+0000 7f8982d79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c069850 0x7f897c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.057+0000 7f8982d79640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f897c0c9620 0x7f897c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.057+0000 7f8982d79640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f897c0ac880 con 0x7f897c160be0 2023-08-25T09:44:58.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.057+0000 7f8982d79640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897c0ad400 con 0x7f897c0c9620 2023-08-25T09:44:58.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.057+0000 7f8982d79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0ad890 con 0x7f897c069850 2023-08-25T09:44:58.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f897c0c9620 0x7f897c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f897c0c9620 0x7f897c069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38274/0 (socket says 172.21.15.70:38274) 2023-08-25T09:44:58.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 -- 172.21.15.70:0/2051773269 learned_addr learned my addr 172.21.15.70:0/2051773269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:58.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f8980b15640 1 --2- 172.21.15.70:0/2051773269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c069850 0x7f897c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 -- 172.21.15.70:0/2051773269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c069850 msgr2=0x7f897c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:58.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 --2- 172.21.15.70:0/2051773269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c069850 0x7f897c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 -- 172.21.15.70:0/2051773269 >> v1:172.21.15.155:6789/0 conn(0x7f897c160be0 legacy=0x7f897c161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:58.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.058+0000 7f897bfff640 1 -- 172.21.15.70:0/2051773269 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0acad0 con 0x7f897c0c9620 2023-08-25T09:44:58.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.059+0000 7f897bfff640 1 --2- 172.21.15.70:0/2051773269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0c9620 0x7f897c069c30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8968009870 tx=0x7f896802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60bd1b58bbe3d12 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:58.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f897b7fe640 1 -- 172.21.15.70:0/2051773269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968030840 con 0x7f897c0c9620 2023-08-25T09:44:58.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f897b7fe640 1 -- 172.21.15.70:0/2051773269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89680309a0 con 0x7f897c0c9620 2023-08-25T09:44:58.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f897b7fe640 1 -- 172.21.15.70:0/2051773269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968030d30 con 0x7f897c0c9620 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f8982d79640 1 -- 172.21.15.70:0/2051773269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0c9620 msgr2=0x7f897c069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f8982d79640 1 --2- 172.21.15.70:0/2051773269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0c9620 0x7f897c069c30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8968009870 tx=0x7f896802b980 comp rx=0 tx=0).stop 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f8982d79640 1 -- 172.21.15.70:0/2051773269 shutdown_connections 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f8982d79640 1 --2- 172.21.15.70:0/2051773269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c0c9620 0x7f897c069c30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f8982d79640 1 --2- 172.21.15.70:0/2051773269 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c069850 0x7f897c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.060+0000 7f8982d79640 1 -- 172.21.15.70:0/2051773269 >> 172.21.15.70:0/2051773269 conn(0x7f897c0cb0d0 msgr2=0x7f897c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:58.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.061+0000 7f8982d79640 1 -- 172.21.15.70:0/2051773269 shutdown_connections 2023-08-25T09:44:58.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.061+0000 7f8982d79640 1 -- 172.21.15.70:0/2051773269 wait complete. 2023-08-25T09:44:58.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.062+0000 7f8982d79640 1 Processor -- start 2023-08-25T09:44:58.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.062+0000 7f8982d79640 1 -- start start 2023-08-25T09:44:58.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.063+0000 7f8982d79640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c069850 0x7f897c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.063+0000 7f8982d79640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c0c9620 0x7f897c1f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.063+0000 7f8982d79640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 0x7f897c1f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.063+0000 7f8982d79640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c5320 con 0x7f897c160be0 2023-08-25T09:44:58.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.063+0000 7f8982d79640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c5690 con 0x7f897c069850 2023-08-25T09:44:58.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.063+0000 7f8982d79640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897c0c4230 con 0x7f897c0c9620 2023-08-25T09:44:58.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 0x7f897c1f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 0x7f897c1f66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38290/0 (socket says 172.21.15.70:38290) 2023-08-25T09:44:58.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 -- 172.21.15.70:0/549178299 learned_addr learned my addr 172.21.15.70:0/549178299 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f897bfff640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c0c9620 0x7f897c1f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8980b15640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c069850 0x7f897c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 -- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c0c9620 msgr2=0x7f897c1f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c0c9620 0x7f897c1f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 -- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c069850 msgr2=0x7f897c14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c069850 0x7f897c14c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 -- 172.21.15.70:0/549178299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89680094d0 con 0x7f897c160be0 2023-08-25T09:44:58.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8980b15640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c069850 0x7f897c14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:58.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.064+0000 7f8981316640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 0x7f897c1f66e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f897400b9a0 tx=0x7f897400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:58.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.065+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897400cba0 con 0x7f897c160be0 2023-08-25T09:44:58.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.065+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897400cd00 con 0x7f897c160be0 2023-08-25T09:44:58.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.065+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8974014680 con 0x7f897c160be0 2023-08-25T09:44:58.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.065+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897c0c4510 con 0x7f897c160be0 2023-08-25T09:44:58.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.065+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897c14c930 con 0x7f897c160be0 2023-08-25T09:44:58.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.066+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897c0c4000 con 0x7f897c160be0 2023-08-25T09:44:58.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.067+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f897401d3b0 con 0x7f897c160be0 2023-08-25T09:44:58.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.068+0000 7f89797fa640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.068+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8974088130 con 0x7f897c160be0 2023-08-25T09:44:58.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.068+0000 7f8980b15640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.069+0000 7f8980b15640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f897c0ce4c0 tx=0x7f8970008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:58.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.072+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8974053aa0 con 0x7f897c160be0 2023-08-25T09:44:58.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.260+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f897c066fb0 con 0x7f897c160be0 2023-08-25T09:44:58.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.261+0000 7f89797fa640 1 -- 172.21.15.70:0/549178299 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v106) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89740570c0 con 0x7f897c160be0 2023-08-25T09:44:58.264 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:44:58.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 msgr2=0x7f896006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f897c0ce4c0 tx=0x7f8970008040 comp rx=0 tx=0).stop 2023-08-25T09:44:58.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 msgr2=0x7f897c1f66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 0x7f897c1f66e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f897400b9a0 tx=0x7f897400be60 comp rx=0 tx=0).stop 2023-08-25T09:44:58.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 shutdown_connections 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896006a050 0x7f896006c510 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f897c160be0 0x7f897c1f66e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f897c0c9620 0x7f897c1f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 --2- 172.21.15.70:0/549178299 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f897c069850 0x7f897c14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 >> 172.21.15.70:0/549178299 conn(0x7f897c0cb0d0 msgr2=0x7f897c06ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 shutdown_connections 2023-08-25T09:44:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.264+0000 7f8982d79640 1 -- 172.21.15.70:0/549178299 wait complete. 2023-08-25T09:44:58.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-08-25T09:44:58.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.507+0000 7ff568d4f640 1 Processor -- start 2023-08-25T09:44:58.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.507+0000 7ff568d4f640 1 -- start start 2023-08-25T09:44:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff568d4f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 0x7ff56406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff568d4f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff56415ec10 0x7ff5640b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff568d4f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5640b1b60 con 0x7ff56406b2b0 2023-08-25T09:44:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff568d4f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56406bd80 con 0x7ff56415ec10 2023-08-25T09:44:58.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff568d4f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406b7c0 con 0x7ff56415e830 2023-08-25T09:44:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff559d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff56406b2b0 0x7ff56415c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44958/0 (socket says 172.21.15.70:44958) 2023-08-25T09:44:58.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff559d9b640 1 -- 172.21.15.70:0/1625436954 learned_addr learned my addr 172.21.15.70:0/1625436954 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:44:58.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff562d9d640 1 --2- 172.21.15.70:0/1625436954 >> v2:172.21.15.155:3300/0 conn(0x7ff56415ec10 0x7ff5640b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.509+0000 7ff56259c640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 0x7ff56406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219660608 0 0) 0x7ff5640b1b60 con 0x7ff56406b2b0 2023-08-25T09:44:58.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff550003520 con 0x7ff56406b2b0 2023-08-25T09:44:58.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1415864423 0 0) 0x7ff550003520 con 0x7ff56406b2b0 2023-08-25T09:44:58.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 msgr2=0x7ff56406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:58.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 0x7ff56406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 >> v2:172.21.15.155:3300/0 conn(0x7ff56415ec10 msgr2=0x7ff5640b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:58.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 --2- 172.21.15.70:0/1625436954 >> v2:172.21.15.155:3300/0 conn(0x7ff56415ec10 0x7ff5640b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56406baa0 con 0x7ff56406b2b0 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff54c0019f0 con 0x7ff56406b2b0 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 >> v1:172.21.15.70:6789/0 conn(0x7ff56406b2b0 legacy=0x7ff56415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff550003970 0x7ff550003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 0x7ff5500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff550004d00 0x7ff550005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406bd80 con 0x7ff550003970 2023-08-25T09:44:58.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56406b7c0 con 0x7ff550004d00 2023-08-25T09:44:58.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55000b980 con 0x7ff56415e830 2023-08-25T09:44:58.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.510+0000 7ff56259c640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff550003970 0x7ff550003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff56259c640 1 -- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 msgr2=0x7ff5500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:58.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff56259c640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff56415e830 0x7ff5500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff562d9d640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff550004d00 0x7ff550005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff56259c640 1 -- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff550004d00 msgr2=0x7ff550005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff56259c640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff550004d00 0x7ff550005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff56259c640 1 -- 172.21.15.70:0/1625436954 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550003520 con 0x7ff550003970 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.511+0000 7ff562d9d640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff550004d00 0x7ff550005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.512+0000 7ff56259c640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff550003970 0x7ff550003d90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff554009ac0 tx=0x7ff5500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6211ff8b0ef9ac05 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.512+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55402c710 con 0x7ff550003970 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.512+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55402c870 con 0x7ff550003970 2023-08-25T09:44:58.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.512+0000 7ff561d9b640 1 -- 172.21.15.70:0/1625436954 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55402cc00 con 0x7ff550003970 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff568d4f640 1 -- 172.21.15.70:0/1625436954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff550003970 msgr2=0x7ff550003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff568d4f640 1 --2- 172.21.15.70:0/1625436954 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff550003970 0x7ff550003d90 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7ff554009ac0 tx=0x7ff5500057f0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff562d9d640 1 -- 172.21.15.70:0/1625436954 reap_dead start 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff568d4f640 1 -- 172.21.15.70:0/1625436954 shutdown_connections 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff568d4f640 1 -- 172.21.15.70:0/1625436954 >> 172.21.15.70:0/1625436954 conn(0x7ff564069850 msgr2=0x7ff564069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff568d4f640 1 -- 172.21.15.70:0/1625436954 shutdown_connections 2023-08-25T09:44:58.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.513+0000 7ff568d4f640 1 -- 172.21.15.70:0/1625436954 wait complete. 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.515+0000 7ff568d4f640 1 Processor -- start 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.515+0000 7ff568d4f640 1 -- start start 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff568d4f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff550003970 0x7ff5640af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff568d4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 0x7ff5641f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff568d4f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5640c35c0 0x7ff5640c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff568d4f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640c3f90 con 0x7ff5640c41a0 2023-08-25T09:44:58.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff568d4f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff564157780 con 0x7ff5640c35c0 2023-08-25T09:44:58.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff568d4f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff564151a80 con 0x7ff550003970 2023-08-25T09:44:58.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff559d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 0x7ff5641f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff559d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 0x7ff5641f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38310/0 (socket says 172.21.15.70:38310) 2023-08-25T09:44:58.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff559d9b640 1 -- 172.21.15.70:0/146173708 learned_addr learned my addr 172.21.15.70:0/146173708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:58.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff56259c640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff550003970 0x7ff5640af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.516+0000 7ff562d9d640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5640c35c0 0x7ff5640c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff559d9b640 1 -- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff550003970 msgr2=0x7ff5640af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff559d9b640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff550003970 0x7ff5640af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff559d9b640 1 -- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5640c35c0 msgr2=0x7ff5640c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff559d9b640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5640c35c0 0x7ff5640c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff559d9b640 1 -- 172.21.15.70:0/146173708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56415ca20 con 0x7ff5640c41a0 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff562d9d640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5640c35c0 0x7ff5640c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:44:58.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff559d9b640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 0x7ff5641f1740 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00caa0 tx=0x7ff54c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:58.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00a7f0 con 0x7ff5640c41a0 2023-08-25T09:44:58.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c00a950 con 0x7ff5640c41a0 2023-08-25T09:44:58.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00ad00 con 0x7ff5640c41a0 2023-08-25T09:44:58.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5641579d0 con 0x7ff5640c41a0 2023-08-25T09:44:58.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.517+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff564158270 con 0x7ff5640c41a0 2023-08-25T09:44:58.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.518+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff564157dc0 con 0x7ff5640c41a0 2023-08-25T09:44:58.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.519+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff54c0070d0 con 0x7ff5640c41a0 2023-08-25T09:44:58.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.520+0000 7ff55b7fe640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff54406a050 0x7ff54406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:58.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.520+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff54c087ba0 con 0x7ff5640c41a0 2023-08-25T09:44:58.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.520+0000 7ff56259c640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff54406a050 0x7ff54406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:58.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.521+0000 7ff56259c640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff54406a050 0x7ff54406c510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff554036000 tx=0x7ff554030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:58.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.524+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff54c053510 con 0x7ff5640c41a0 2023-08-25T09:44:58.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.781+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7ff56404e420 con 0x7ff5640c41a0 2023-08-25T09:44:58.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.910+0000 7ff55b7fe640 1 -- 172.21.15.70:0/146173708 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v329) v1 ==== 140+0+0 (secure 0 0 0) 0x7ff54c018070 con 0x7ff5640c41a0 2023-08-25T09:44:58.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff54406a050 msgr2=0x7ff54406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff54406a050 0x7ff54406c510 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff554036000 tx=0x7ff554030000 comp rx=0 tx=0).stop 2023-08-25T09:44:58.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 msgr2=0x7ff5641f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:58.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 0x7ff5641f1740 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00caa0 tx=0x7ff54c00cf60 comp rx=0 tx=0).stop 2023-08-25T09:44:58.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 shutdown_connections 2023-08-25T09:44:58.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff54406a050 0x7ff54406c510 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5640c35c0 0x7ff5640c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5640c41a0 0x7ff5641f1740 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 --2- 172.21.15.70:0/146173708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff550003970 0x7ff5640af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:58.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.913+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 >> 172.21.15.70:0/146173708 conn(0x7ff564069850 msgr2=0x7ff5640b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:58.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.914+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 shutdown_connections 2023-08-25T09:44:58.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:58.914+0000 7ff568d4f640 1 -- 172.21.15.70:0/146173708 wait complete. 2023-08-25T09:44:58.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-08-25T09:44:59.168 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_objectstore":"memstore"} 2023-08-25T09:44:59.169 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-08-25T09:44:59.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:44:59.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.386+0000 7f8485de8640 1 Processor -- start 2023-08-25T09:44:59.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.386+0000 7f8485de8640 1 -- start start 2023-08-25T09:44:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f8485de8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f8485de8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8480069f60 0x7f84800ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f8485de8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800cc5f0 con 0x7f84800cdc20 2023-08-25T09:44:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f8485de8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8480070650 con 0x7f848015ae50 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f8485de8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8480070930 con 0x7f8480069f60 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f847f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f847f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55046/0 (socket says 172.21.15.70:55046) 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.388+0000 7f847f7fe640 1 -- 172.21.15.70:0/2472372107 learned_addr learned my addr 172.21.15.70:0/2472372107 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.389+0000 7f847ffff640 1 --2- 172.21.15.70:0/2472372107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8480069f60 0x7f84800ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.389+0000 7f847f7fe640 1 -- 172.21.15.70:0/2472372107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8480069f60 msgr2=0x7f84800ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:59.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.389+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2472372107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8480069f60 0x7f84800ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.389+0000 7f847f7fe640 1 -- 172.21.15.70:0/2472372107 >> v1:172.21.15.155:6789/0 conn(0x7f84800cdc20 legacy=0x7f8480069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.389+0000 7f847f7fe640 1 -- 172.21.15.70:0/2472372107 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480070c10 con 0x7f848015ae50 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.390+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2472372107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f84800c2110 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8470009870 tx=0x7f847002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4745f7e2e8142053 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.390+0000 7f847e7fc640 1 -- 172.21.15.70:0/2472372107 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470030840 con 0x7f848015ae50 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.390+0000 7f847e7fc640 1 -- 172.21.15.70:0/2472372107 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700309a0 con 0x7f848015ae50 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.390+0000 7f847e7fc640 1 -- 172.21.15.70:0/2472372107 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470030d30 con 0x7f848015ae50 2023-08-25T09:44:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 -- 172.21.15.70:0/2472372107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 msgr2=0x7f84800c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 --2- 172.21.15.70:0/2472372107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f84800c2110 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f8470009870 tx=0x7f847002b980 comp rx=0 tx=0).stop 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 -- 172.21.15.70:0/2472372107 shutdown_connections 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 --2- 172.21.15.70:0/2472372107 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8480069f60 0x7f84800ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 --2- 172.21.15.70:0/2472372107 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f84800c2110 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 -- 172.21.15.70:0/2472372107 >> 172.21.15.70:0/2472372107 conn(0x7f84800cb0d0 msgr2=0x7f84800cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 -- 172.21.15.70:0/2472372107 shutdown_connections 2023-08-25T09:44:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.391+0000 7f8485de8640 1 -- 172.21.15.70:0/2472372107 wait complete. 2023-08-25T09:44:59.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.393+0000 7f8485de8640 1 Processor -- start 2023-08-25T09:44:59.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.393+0000 7f8485de8640 1 -- start start 2023-08-25T09:44:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.394+0000 7f8485de8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8480069f60 0x7f84800aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.394+0000 7f8485de8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84800cdc20 0x7f84800c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.394+0000 7f8485de8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f848014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.394+0000 7f8485de8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848015c810 con 0x7f848015ae50 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.394+0000 7f8485de8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84801f0290 con 0x7f8480069f60 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.394+0000 7f8485de8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800aab30 con 0x7f84800cdc20 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f848014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f848014e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55060/0 (socket says 172.21.15.70:55060) 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 -- 172.21.15.70:0/2590962451 learned_addr learned my addr 172.21.15.70:0/2590962451 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847effd640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84800cdc20 0x7f84800c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8480069f60 0x7f84800aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 -- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84800cdc20 msgr2=0x7f84800c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84800cdc20 0x7f84800c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 -- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8480069f60 msgr2=0x7f84800aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8480069f60 0x7f84800aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 -- 172.21.15.70:0/2590962451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700094d0 con 0x7f848015ae50 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f847ffff640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f848014e560 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8478007e30 tx=0x7f847800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:59.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478014620 con 0x7f848015ae50 2023-08-25T09:44:59.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.395+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847800ee90 con 0x7f848015ae50 2023-08-25T09:44:59.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.396+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800aae10 con 0x7f848015ae50 2023-08-25T09:44:59.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.396+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84780149c0 con 0x7f848015ae50 2023-08-25T09:44:59.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.396+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800ab0f0 con 0x7f848015ae50 2023-08-25T09:44:59.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.397+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848015c9c0 con 0x7f848015ae50 2023-08-25T09:44:59.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.398+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8478014b20 con 0x7f848015ae50 2023-08-25T09:44:59.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.399+0000 7f8467fff640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a080 0x7f846c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:44:59.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.399+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8478088bd0 con 0x7f848015ae50 2023-08-25T09:44:59.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.399+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a080 0x7f846c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:44:59.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.400+0000 7f847f7fe640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a080 0x7f846c06c540 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8470036000 tx=0x7f847003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:44:59.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.403+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f848015c9c0 con 0x7f848015ae50 2023-08-25T09:44:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.592+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8480066fb0 con 0x7f848015ae50 2023-08-25T09:44:59.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.601+0000 7f8467fff640 1 -- 172.21.15.70:0/2590962451 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84780545c0 con 0x7f848015ae50 2023-08-25T09:44:59.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a080 msgr2=0x7f846c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:59.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a080 0x7f846c06c540 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8470036000 tx=0x7f847003b040 comp rx=0 tx=0).stop 2023-08-25T09:44:59.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 msgr2=0x7f848014e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:44:59.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f848014e560 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f8478007e30 tx=0x7f847800e4b0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 shutdown_connections 2023-08-25T09:44:59.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f846c06a080 0x7f846c06c540 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f848015ae50 0x7f848014e560 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f84800cdc20 0x7f84800c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 --2- 172.21.15.70:0/2590962451 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8480069f60 0x7f84800aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:44:59.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 >> 172.21.15.70:0/2590962451 conn(0x7f84800cb0d0 msgr2=0x7f848006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:44:59.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 shutdown_connections 2023-08-25T09:44:59.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:44:59.604+0000 7f8485de8640 1 -- 172.21.15.70:0/2590962451 wait complete. 2023-08-25T09:44:59.625 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:44:59.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:44:59.761 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:44:59.884 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:44:59.885 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:44:59.885 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:44:59.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:44:59.945 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:44:59.946 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:00.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:45:00.007 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:00.007 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:00.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:45:00.068 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:45:00.069 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:00.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:45:00.130 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:45:00.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.352+0000 7fa7d9246640 1 Processor -- start 2023-08-25T09:45:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.353+0000 7fa7d9246640 1 -- start start 2023-08-25T09:45:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.354+0000 7fa7d9246640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7d4160ff0 0x7fa7d406b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.354+0000 7fa7d9246640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa7d4161ae0 0x7fa7d40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.354+0000 7fa7d9246640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d41620b0 con 0x7fa7d40ce5f0 2023-08-25T09:45:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.354+0000 7fa7d9246640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d40cefe0 con 0x7fa7d4160ff0 2023-08-25T09:45:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.354+0000 7fa7d9246640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d406ad70 con 0x7fa7d4161ae0 2023-08-25T09:45:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.354+0000 7fa7d2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7d4160ff0 0x7fa7d406b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7d2d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa7d4160ff0 0x7fa7d406b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55070/0 (socket says 172.21.15.70:55070) 2023-08-25T09:45:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7d2d9d640 1 -- 172.21.15.70:0/2778492336 learned_addr learned my addr 172.21.15.70:0/2778492336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7ca59c640 1 --1- 172.21.15.70:0/2778492336 >> v1:172.21.15.70:6789/0 conn(0x7fa7d40ce5f0 0x7fa7d41613d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43630/0 (socket says 172.21.15.70:43630) 2023-08-25T09:45:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7d2d9d640 1 -- 172.21.15.70:0/2778492336 >> v2:172.21.15.155:3300/0 conn(0x7fa7d4161ae0 msgr2=0x7fa7d40ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7d2d9d640 1 --2- 172.21.15.70:0/2778492336 >> v2:172.21.15.155:3300/0 conn(0x7fa7d4161ae0 0x7fa7d40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7d2d9d640 1 -- 172.21.15.70:0/2778492336 >> v1:172.21.15.70:6789/0 conn(0x7fa7d40ce5f0 legacy=0x7fa7d41613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.355+0000 7fa7d2d9d640 1 -- 172.21.15.70:0/2778492336 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4162390 con 0x7fa7d4160ff0 2023-08-25T09:45:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.356+0000 7fa7d2d9d640 1 --2- 172.21.15.70:0/2778492336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d406b780 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40b2a30 tx=0x7fa7c402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccc880ebcc1d030 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d259c640 1 -- 172.21.15.70:0/2778492336 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4030870 con 0x7fa7d4160ff0 2023-08-25T09:45:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d259c640 1 -- 172.21.15.70:0/2778492336 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c40309d0 con 0x7fa7d4160ff0 2023-08-25T09:45:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d259c640 1 -- 172.21.15.70:0/2778492336 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4030d60 con 0x7fa7d4160ff0 2023-08-25T09:45:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 -- 172.21.15.70:0/2778492336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 msgr2=0x7fa7d406b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 --2- 172.21.15.70:0/2778492336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d406b780 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40b2a30 tx=0x7fa7c402b9b0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 -- 172.21.15.70:0/2778492336 shutdown_connections 2023-08-25T09:45:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 --2- 172.21.15.70:0/2778492336 >> v2:172.21.15.155:3300/0 conn(0x7fa7d4161ae0 0x7fa7d40ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 --2- 172.21.15.70:0/2778492336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d406b780 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 -- 172.21.15.70:0/2778492336 >> 172.21.15.70:0/2778492336 conn(0x7fa7d4069850 msgr2=0x7fa7d4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 -- 172.21.15.70:0/2778492336 shutdown_connections 2023-08-25T09:45:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.357+0000 7fa7d9246640 1 -- 172.21.15.70:0/2778492336 wait complete. 2023-08-25T09:45:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.359+0000 7fa7d9246640 1 Processor -- start 2023-08-25T09:45:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.359+0000 7fa7d9246640 1 -- start start 2023-08-25T09:45:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7d9246640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d40ce5f0 0x7fa7d4143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7d9246640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d415b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7d9246640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d4161ae0 0x7fa7d4163470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7d9246640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4162740 con 0x7fa7d4160ff0 2023-08-25T09:45:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7d9246640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41439b0 con 0x7fa7d4161ae0 2023-08-25T09:45:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7d9246640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d4160670 con 0x7fa7d40ce5f0 2023-08-25T09:45:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.360+0000 7fa7ca59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d415b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d415b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55072/0 (socket says 172.21.15.70:55072) 2023-08-25T09:45:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 -- 172.21.15.70:0/3205966645 learned_addr learned my addr 172.21.15.70:0/3205966645 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 -- 172.21.15.70:0/3205966645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d40ce5f0 msgr2=0x7fa7d4143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d40ce5f0 0x7fa7d4143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 -- 172.21.15.70:0/3205966645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d4161ae0 msgr2=0x7fa7d4163470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d4161ae0 0x7fa7d4163470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 -- 172.21.15.70:0/3205966645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40094d0 con 0x7fa7d4160ff0 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7ca59c640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d415b360 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c500 tx=0x7fa7bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc007500 con 0x7fa7d4160ff0 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc007660 con 0x7fa7d4160ff0 2023-08-25T09:45:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0049e0 con 0x7fa7d4160ff0 2023-08-25T09:45:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.361+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d4162990 con 0x7fa7d4160ff0 2023-08-25T09:45:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.362+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d4162f50 con 0x7fa7d4160ff0 2023-08-25T09:45:00.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.363+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d4163170 con 0x7fa7d4160ff0 2023-08-25T09:45:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.364+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa7bc004c00 con 0x7fa7d4160ff0 2023-08-25T09:45:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.364+0000 7fa7cbfff640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa7b806a050 0x7fa7b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.364+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa7bc0873b0 con 0x7fa7d4160ff0 2023-08-25T09:45:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.365+0000 7fa7d2d9d640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa7b806a050 0x7fa7b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.365+0000 7fa7d2d9d640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa7b806a050 0x7fa7b806c510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4036000 tx=0x7fa7c403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:00.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.369+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa7bc052e60 con 0x7fa7d4160ff0 2023-08-25T09:45:00.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.594+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7d404e420 con 0x7fa7d4160ff0 2023-08-25T09:45:00.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.595+0000 7fa7cbfff640 1 -- 172.21.15.70:0/3205966645 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3693 (secure 0 0 0) 0x7fa7bc056480 con 0x7fa7d4160ff0 2023-08-25T09:45:00.597 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:00.597 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5365,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2847952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2847952},{"type":"v1","addr":"172.21.15.155:6834","nonce":2847952}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":5368,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/3921032099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":3921032099},{"type":"v1","addr":"172.21.15.155:6835","nonce":3921032099}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6768,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/2652751671","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":2652751671},{"type":"v1","addr":"172.21.15.70:6835","nonce":2652751671}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":7140,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/665133093","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":665133093},{"type":"v1","addr":"172.21.15.70:6837","nonce":665133093}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2023-08-25T09:45:00.598 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 43 2023-08-25T09:45:00.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa7b806a050 msgr2=0x7fa7b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa7b806a050 0x7fa7b806c510 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4036000 tx=0x7fa7c403b040 comp rx=0 tx=0).stop 2023-08-25T09:45:00.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 msgr2=0x7fa7d415b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d415b360 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc00c500 tx=0x7fa7bc00c9c0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 shutdown_connections 2023-08-25T09:45:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa7b806a050 0x7fa7b806c510 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa7d4161ae0 0x7fa7d4163470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa7d4160ff0 0x7fa7d415b360 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 --2- 172.21.15.70:0/3205966645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa7d40ce5f0 0x7fa7d4143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 >> 172.21.15.70:0/3205966645 conn(0x7fa7d4069850 msgr2=0x7fa7d40ada00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 shutdown_connections 2023-08-25T09:45:00.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.597+0000 7fa7d9246640 1 -- 172.21.15.70:0/3205966645 wait complete. 2023-08-25T09:45:00.617 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:45:00.617 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:45:00.617 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:45:00.618 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:45:00.618 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:45:00.618 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:45:00.619 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:45:00.619 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:45:00.619 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:45:00.619 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:45:00.620 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:45:00.620 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:45:00.621 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:45:00.621 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:45:00.621 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:45:00.621 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:45:00.637 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:45:00.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:45:00.645 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:45:00.645 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:45:00.647 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:45:00.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:45:00.656 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:45:00.656 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:45:00.695 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:45:00.696 DEBUG:teuthology.parallel:result is None 2023-08-25T09:45:00.703 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:45:00.704 DEBUG:teuthology.parallel:result is None 2023-08-25T09:45:00.705 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:45:00.706 DEBUG:teuthology.parallel:result is None 2023-08-25T09:45:00.713 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:45:00.713 DEBUG:teuthology.parallel:result is None 2023-08-25T09:45:00.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:45:00.859 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:45:00.860 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:45:00.876 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:45:00.883 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:45:00.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.904+0000 7f33bd9e3640 1 Processor -- start 2023-08-25T09:45:00.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.904+0000 7f33bd9e3640 1 -- start start 2023-08-25T09:45:00.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.905+0000 7f33bd9e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80af810 0x7f33b8071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.905+0000 7f33bd9e3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b8158b40 con 0x7f33b80afbf0 2023-08-25T09:45:00.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.905+0000 7f33bd9e3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b815e960 con 0x7f33b80d02f0 2023-08-25T09:45:00.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.905+0000 7f33bd9e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b815e750 con 0x7f33b80af810 2023-08-25T09:45:00.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.905+0000 7f33ae7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f33b80afbf0 0x7f33b80cfbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43656/0 (socket says 172.21.15.70:43656) 2023-08-25T09:45:00.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.905+0000 7f33ae7fc640 1 -- 172.21.15.70:0/756358271 learned_addr learned my addr 172.21.15.70:0/756358271 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:00.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b6ffd640 1 --2- 172.21.15.70:0/756358271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80af810 0x7f33b8071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b67fc640 1 -- 172.21.15.70:0/756358271 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445391955 0 0) 0x7f33b8158b40 con 0x7f33b80afbf0 2023-08-25T09:45:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b67fc640 1 -- 172.21.15.70:0/756358271 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a4003520 con 0x7f33b80afbf0 2023-08-25T09:45:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b6ffd640 1 -- 172.21.15.70:0/756358271 >> v1:172.21.15.155:6789/0 conn(0x7f33b80d02f0 legacy=0x7f33b80b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b67fc640 1 -- 172.21.15.70:0/756358271 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143310621 0 0) 0x7f33b815e960 con 0x7f33b80d02f0 2023-08-25T09:45:00.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b6ffd640 1 -- 172.21.15.70:0/756358271 >> v1:172.21.15.70:6789/0 conn(0x7f33b80afbf0 legacy=0x7f33b80cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.906+0000 7f33b6ffd640 1 -- 172.21.15.70:0/756358271 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b815ec40 con 0x7f33b80af810 2023-08-25T09:45:00.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.907+0000 7f33b6ffd640 1 --2- 172.21.15.70:0/756358271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80af810 0x7f33b8071650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097d0 tx=0x7f33a802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccdb5815757dd956 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:00.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33b67fc640 1 -- 172.21.15.70:0/756358271 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8031800 con 0x7f33b80af810 2023-08-25T09:45:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33b67fc640 1 -- 172.21.15.70:0/756358271 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a8031960 con 0x7f33b80af810 2023-08-25T09:45:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33b67fc640 1 -- 172.21.15.70:0/756358271 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8031cf0 con 0x7f33b80af810 2023-08-25T09:45:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 -- 172.21.15.70:0/756358271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80af810 msgr2=0x7f33b8071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/756358271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80af810 0x7f33b8071650 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097d0 tx=0x7f33a802b8a0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 -- 172.21.15.70:0/756358271 shutdown_connections 2023-08-25T09:45:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/756358271 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80af810 0x7f33b8071650 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 -- 172.21.15.70:0/756358271 >> 172.21.15.70:0/756358271 conn(0x7f33b806cf60 msgr2=0x7f33b806d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 -- 172.21.15.70:0/756358271 shutdown_connections 2023-08-25T09:45:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.908+0000 7f33bd9e3640 1 -- 172.21.15.70:0/756358271 wait complete. 2023-08-25T09:45:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.910+0000 7f33bd9e3640 1 Processor -- start 2023-08-25T09:45:00.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.910+0000 7f33bd9e3640 1 -- start start 2023-08-25T09:45:00.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.911+0000 7f33bd9e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 0x7f33b8155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.911+0000 7f33bd9e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80afbf0 0x7f33b80b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.911+0000 7f33bd9e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b80d02f0 0x7f33b80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.911+0000 7f33bd9e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b815bad0 con 0x7f33b80af810 2023-08-25T09:45:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.911+0000 7f33bd9e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81561c0 con 0x7f33b80d02f0 2023-08-25T09:45:00.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.911+0000 7f33bd9e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80b2760 con 0x7f33b80afbf0 2023-08-25T09:45:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 0x7f33b8155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 0x7f33b8155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55090/0 (socket says 172.21.15.70:55090) 2023-08-25T09:45:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 -- 172.21.15.70:0/2483448333 learned_addr learned my addr 172.21.15.70:0/2483448333 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:00.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33ae7fc640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80afbf0 0x7f33b80b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b77fe640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b80d02f0 0x7f33b80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 -- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80afbf0 msgr2=0x7f33b80b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80afbf0 0x7f33b80b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 -- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b80d02f0 msgr2=0x7f33b80c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b80d02f0 0x7f33b80c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 -- 172.21.15.70:0/2483448333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009400 con 0x7f33b80af810 2023-08-25T09:45:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.912+0000 7f33b6ffd640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 0x7f33b8155c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f33a8034040 tx=0x7f33a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:00.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.913+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a803aa50 con 0x7f33b80af810 2023-08-25T09:45:00.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.913+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a803abb0 con 0x7f33b80af810 2023-08-25T09:45:00.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.913+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8044420 con 0x7f33b80af810 2023-08-25T09:45:00.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.913+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b80b2a40 con 0x7f33b80af810 2023-08-25T09:45:00.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.913+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b815bfc0 con 0x7f33b80af810 2023-08-25T09:45:00.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.914+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b804e420 con 0x7f33b80af810 2023-08-25T09:45:00.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.915+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33a8042070 con 0x7f33b80af810 2023-08-25T09:45:00.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.916+0000 7f33b4ff9640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f339806a050 0x7f339806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:00.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.916+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f33a802e070 con 0x7f33b80af810 2023-08-25T09:45:00.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.916+0000 7f33ae7fc640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f339806a050 0x7f339806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.917+0000 7f33ae7fc640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f339806a050 0x7f339806c510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f33b815e380 tx=0x7f33a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:00.919+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f33a8075f50 con 0x7f33b80af810 2023-08-25T09:45:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.125+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f33b80b3500 con 0x7f33b80af810 2023-08-25T09:45:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.959+0000 7f33b4ff9640 1 -- 172.21.15.70:0/2483448333 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v111) v1 ==== 97+0+0 (secure 0 0 0) 0x7f33a8049080 con 0x7f33b80af810 2023-08-25T09:45:01.961 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:45:01.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f339806a050 msgr2=0x7f339806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:01.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f339806a050 0x7f339806c510 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f33b815e380 tx=0x7f33a0009040 comp rx=0 tx=0).stop 2023-08-25T09:45:01.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 msgr2=0x7f33b8155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:01.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 0x7f33b8155c80 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f33a8034040 tx=0x7f33a80046c0 comp rx=0 tx=0).stop 2023-08-25T09:45:01.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 shutdown_connections 2023-08-25T09:45:01.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f339806a050 0x7f339806c510 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:01.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.962+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f33b80d02f0 0x7f33b80c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:01.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.963+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f33b80afbf0 0x7f33b80b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:01.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.963+0000 7f33bd9e3640 1 --2- 172.21.15.70:0/2483448333 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f33b80af810 0x7f33b8155c80 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:01.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.963+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 >> 172.21.15.70:0/2483448333 conn(0x7f33b806cf60 msgr2=0x7f33b8158d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:01.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.963+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 shutdown_connections 2023-08-25T09:45:01.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:01.963+0000 7f33bd9e3640 1 -- 172.21.15.70:0/2483448333 wait complete. 2023-08-25T09:45:01.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:45:02.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.199+0000 7ff3069bf640 1 Processor -- start 2023-08-25T09:45:02.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.200+0000 7ff3069bf640 1 -- start start 2023-08-25T09:45:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.201+0000 7ff3069bf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff30015d3d0 0x7ff3000b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.201+0000 7ff3069bf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3000b2aa0 con 0x7ff300159840 2023-08-25T09:45:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.201+0000 7ff3069bf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff30015d890 con 0x7ff3000cadb0 2023-08-25T09:45:02.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.201+0000 7ff3069bf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3000c8b60 con 0x7ff30015d3d0 2023-08-25T09:45:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.201+0000 7ff3059bd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff300159840 0x7ff3000b2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43670/0 (socket says 172.21.15.70:43670) 2023-08-25T09:45:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.202+0000 7ff3051bc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff30015d3d0 0x7ff3000b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.202+0000 7ff3059bd640 1 -- 172.21.15.70:0/1916332977 learned_addr learned my addr 172.21.15.70:0/1916332977 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:02.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.202+0000 7ff3051bc640 1 -- 172.21.15.70:0/1916332977 >> v1:172.21.15.155:6789/0 conn(0x7ff3000cadb0 legacy=0x7ff3000cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.202+0000 7ff3051bc640 1 -- 172.21.15.70:0/1916332977 >> v1:172.21.15.70:6789/0 conn(0x7ff300159840 legacy=0x7ff3000b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.202+0000 7ff3051bc640 1 -- 172.21.15.70:0/1916332977 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30015db70 con 0x7ff30015d3d0 2023-08-25T09:45:02.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.202+0000 7ff3049bb640 1 -- 172.21.15.70:0/1916332977 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101669776 0 0) 0x7ff3000b2aa0 con 0x7ff300159840 2023-08-25T09:45:02.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.203+0000 7ff3051bc640 1 --2- 172.21.15.70:0/1916332977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000b3d40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00097a0 tx=0x7ff2f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56c628947c34d79 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.203+0000 7ff3049bb640 1 -- 172.21.15.70:0/1916332977 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0031830 con 0x7ff30015d3d0 2023-08-25T09:45:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.203+0000 7ff3049bb640 1 -- 172.21.15.70:0/1916332977 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0031990 con 0x7ff30015d3d0 2023-08-25T09:45:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.203+0000 7ff3049bb640 1 -- 172.21.15.70:0/1916332977 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0031d20 con 0x7ff30015d3d0 2023-08-25T09:45:02.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 -- 172.21.15.70:0/1916332977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 msgr2=0x7ff3000b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 --2- 172.21.15.70:0/1916332977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000b3d40 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00097a0 tx=0x7ff2f002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 -- 172.21.15.70:0/1916332977 shutdown_connections 2023-08-25T09:45:02.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 --2- 172.21.15.70:0/1916332977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000b3d40 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 -- 172.21.15.70:0/1916332977 >> 172.21.15.70:0/1916332977 conn(0x7ff3000affd0 msgr2=0x7ff3000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 -- 172.21.15.70:0/1916332977 shutdown_connections 2023-08-25T09:45:02.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.204+0000 7ff3069bf640 1 -- 172.21.15.70:0/1916332977 wait complete. 2023-08-25T09:45:02.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.206+0000 7ff3069bf640 1 Processor -- start 2023-08-25T09:45:02.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.206+0000 7ff3069bf640 1 -- start start 2023-08-25T09:45:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3069bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3000cadb0 0x7ff30006ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3069bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff300159840 0x7ff3000c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3069bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3069bf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000c8480 con 0x7ff30015d3d0 2023-08-25T09:45:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3069bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000c6140 con 0x7ff300159840 2023-08-25T09:45:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3069bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3000c7560 con 0x7ff3000cadb0 2023-08-25T09:45:02.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3061be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3061be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55108/0 (socket says 172.21.15.70:55108) 2023-08-25T09:45:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.207+0000 7ff3061be640 1 -- 172.21.15.70:0/2116173916 learned_addr learned my addr 172.21.15.70:0/2116173916 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3051bc640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff300159840 0x7ff3000c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3061be640 1 -- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3000cadb0 msgr2=0x7ff30006ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3061be640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3000cadb0 0x7ff30006ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3061be640 1 -- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff300159840 msgr2=0x7ff3000c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3061be640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff300159840 0x7ff3000c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3061be640 1 -- 172.21.15.70:0/2116173916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0009400 con 0x7ff30015d3d0 2023-08-25T09:45:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3061be640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000c5c00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800b9c0 tx=0x7ff2f800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:02.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800cb40 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f800cca0 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8014680 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.208+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3000c7800 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.209+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3000c7dc0 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.210+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff300066fb0 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.211+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff2f801d3b0 con 0x7ff30015d3d0 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.212+0000 7ff2f67fc640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2e406a050 0x7ff2e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.212+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff2f80881a0 con 0x7ff30015d3d0 2023-08-25T09:45:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.212+0000 7ff3059bd640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2e406a050 0x7ff2e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.213+0000 7ff3059bd640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2e406a050 0x7ff2e406c510 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc007fd0 tx=0x7ff2fc007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:02.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.216+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff2f8053bd0 con 0x7ff30015d3d0 2023-08-25T09:45:02.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.479+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff300071b30 con 0x7ff30015d3d0 2023-08-25T09:45:02.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.480+0000 7ff2f67fc640 1 -- 172.21.15.70:0/2116173916 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff2f80571f0 con 0x7ff30015d3d0 2023-08-25T09:45:02.482 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:02.482 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:02.483 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:45:02.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.483 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:45:02.483 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:45:02.484 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.484 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.484 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:45:02.484 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:45:02.484 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.485 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:45:02.485 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:45:02.485 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.486 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.486 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:45:02.487 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:45:02.487 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.488 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:45:02.488 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:45:02.489 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.490 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.490 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:45:02.490 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:45:02.491 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.492 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:45:02.492 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:45:02.492 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.493 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.493 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:45:02.494 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:45:02.494 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.495 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:45:02.495 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:45:02.495 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.495 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.496 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.496 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.497 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:45:02.497 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:45:02.498 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.498 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.498 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.498 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.499 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.499 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:45:02.499 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:45:02.500 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.500 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.500 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.500 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.500 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.501 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.502 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:45:02.503 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.505 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:45:02.505 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:45:02.505 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.505 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.505 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.506 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.506 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:45:02.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.509 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.510 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:45:02.510 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:45:02.510 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:45:02.510 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.511 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:45:02.512 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:45:02.512 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:45:02.512 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.512 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:45:02.513 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:45:02.513 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.513 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:45:02.513 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:45:02.513 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:45:02.514 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.515 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:02.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:02.517 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:45:02.517 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:45:02.517 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:45:02.518 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:45:02.518 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:45:02.518 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:45:02.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:02.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:02.519 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:02.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2e406a050 msgr2=0x7ff2e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2e406a050 0x7ff2e406c510 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc007fd0 tx=0x7ff2fc007920 comp rx=0 tx=0).stop 2023-08-25T09:45:02.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 msgr2=0x7ff3000c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000c5c00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800b9c0 tx=0x7ff2f800be80 comp rx=0 tx=0).stop 2023-08-25T09:45:02.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 shutdown_connections 2023-08-25T09:45:02.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff2e406a050 0x7ff2e406c510 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff30015d3d0 0x7ff3000c5c00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff300159840 0x7ff3000c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 --2- 172.21.15.70:0/2116173916 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3000cadb0 0x7ff30006ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 >> 172.21.15.70:0/2116173916 conn(0x7ff3000affd0 msgr2=0x7ff30015bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:02.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 shutdown_connections 2023-08-25T09:45:02.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.483+0000 7ff3069bf640 1 -- 172.21.15.70:0/2116173916 wait complete. 2023-08-25T09:45:02.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.741+0000 7fcaa30a7640 1 Processor -- start 2023-08-25T09:45:02.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.741+0000 7fcaa30a7640 1 -- start start 2023-08-25T09:45:02.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa30a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 0x7fca9c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa30a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fca9c15ec10 0x7fca9c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa30a7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c0b1b60 con 0x7fca9c06b2b0 2023-08-25T09:45:02.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa30a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c06bd80 con 0x7fca9c15ec10 2023-08-25T09:45:02.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa30a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c06b7c0 con 0x7fca9c15e830 2023-08-25T09:45:02.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fca9bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fca9c06b2b0 0x7fca9c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43676/0 (socket says 172.21.15.70:43676) 2023-08-25T09:45:02.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fca9bfff640 1 -- 172.21.15.70:0/3914851980 learned_addr learned my addr 172.21.15.70:0/3914851980 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:02.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211914390 0 0) 0x7fca9c0b1b60 con 0x7fca9c06b2b0 2023-08-25T09:45:02.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 0x7fca9c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c003520 con 0x7fca9c06b2b0 2023-08-25T09:45:02.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.743+0000 7fcaa1644640 1 --2- 172.21.15.70:0/3914851980 >> v2:172.21.15.155:3300/0 conn(0x7fca9c15ec10 0x7fca9c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2514672288 0 0) 0x7fca8c003520 con 0x7fca9c06b2b0 2023-08-25T09:45:02.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 msgr2=0x7fca9c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:02.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 0x7fca9c06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 >> v2:172.21.15.155:3300/0 conn(0x7fca9c15ec10 msgr2=0x7fca9c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 --2- 172.21.15.70:0/3914851980 >> v2:172.21.15.155:3300/0 conn(0x7fca9c15ec10 0x7fca9c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c06baa0 con 0x7fca9c06b2b0 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fca88001670 con 0x7fca9c06b2b0 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 >> v1:172.21.15.70:6789/0 conn(0x7fca9c06b2b0 legacy=0x7fca9c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 0x7fca8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c06bd80 con 0x7fca8c003970 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c06b7c0 con 0x7fca8c0047c0 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c00b980 con 0x7fca9c15e830 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.744+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.745+0000 7fca9bfff640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.745+0000 7fcaa0e43640 1 -- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 msgr2=0x7fca8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.745+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca9c15e830 0x7fca8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.745+0000 7fcaa0e43640 1 -- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c0047c0 msgr2=0x7fca8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.745+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.745+0000 7fcaa0e43640 1 -- 172.21.15.70:0/3914851980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c003520 con 0x7fca8c003970 2023-08-25T09:45:02.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.746+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fca90009ac0 tx=0x7fca8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1da20f7e7eee140 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:02.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.746+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9002c710 con 0x7fca8c003970 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.746+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9002c870 con 0x7fca8c003970 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.746+0000 7fca9b7fe640 1 -- 172.21.15.70:0/3914851980 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9002cc00 con 0x7fca8c003970 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa30a7640 1 -- 172.21.15.70:0/3914851980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c003970 msgr2=0x7fca8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/3914851980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca8c003970 0x7fca8c003d90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fca90009ac0 tx=0x7fca8c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa1644640 1 -- 172.21.15.70:0/3914851980 reap_dead start 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa30a7640 1 -- 172.21.15.70:0/3914851980 shutdown_connections 2023-08-25T09:45:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa30a7640 1 -- 172.21.15.70:0/3914851980 >> 172.21.15.70:0/3914851980 conn(0x7fca9c069850 msgr2=0x7fca9c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa30a7640 1 -- 172.21.15.70:0/3914851980 shutdown_connections 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.747+0000 7fcaa30a7640 1 -- 172.21.15.70:0/3914851980 wait complete. 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.749+0000 7fcaa30a7640 1 Processor -- start 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.749+0000 7fcaa30a7640 1 -- start start 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fcaa30a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c003970 0x7fca9c157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fcaa30a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 0x7fca9c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fcaa30a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0aa200 0x7fca9c0aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fcaa30a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0b28b0 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fcaa30a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c8a20 con 0x7fca9c0aa200 2023-08-25T09:45:02.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fcaa30a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0c9880 con 0x7fca8c003970 2023-08-25T09:45:02.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 0x7fca9c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 0x7fca9c0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55138/0 (socket says 172.21.15.70:55138) 2023-08-25T09:45:02.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 -- 172.21.15.70:0/1345398321 learned_addr learned my addr 172.21.15.70:0/1345398321 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:02.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 -- 172.21.15.70:0/1345398321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c003970 msgr2=0x7fca9c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c003970 0x7fca9c157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 -- 172.21.15.70:0/1345398321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0aa200 msgr2=0x7fca9c0aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0aa200 0x7fca9c0aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.750+0000 7fca9bfff640 1 -- 172.21.15.70:0/1345398321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c15ca20 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.751+0000 7fca9bfff640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 0x7fca9c0b2370 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e7f0 tx=0x7fca8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.751+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800c850 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.751+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8800c9b0 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.751+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8800cd40 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.751+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c0aabc0 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.751+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c0c8f50 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.752+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c04e420 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.753+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fca8801d460 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.755+0000 7fca997fa640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8006a080 0x7fca8006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:02.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.755+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fca880881f0 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.755+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8006a080 0x7fca8006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:02.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.756+0000 7fcaa0e43640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8006a080 0x7fca8006c540 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fca90036000 tx=0x7fca9002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:02.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.760+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fca88053b90 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.953+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca9c0c99c0 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.955+0000 7fca997fa640 1 -- 172.21.15.70:0/1345398321 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+14657 (secure 0 0 0) 0x7fca880571b0 con 0x7fca9c0b1ef0 2023-08-25T09:45:02.957 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:02.957 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":111,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:01.946083+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8006a080 msgr2=0x7fca8006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8006a080 0x7fca8006c540 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fca90036000 tx=0x7fca9002e000 comp rx=0 tx=0).stop 2023-08-25T09:45:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 msgr2=0x7fca9c0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 0x7fca9c0b2370 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fca8800e7f0 tx=0x7fca8800ecb0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 shutdown_connections 2023-08-25T09:45:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8006a080 0x7fca8006c540 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fca9c0aa200 0x7fca9c0aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fca9c0b1ef0 0x7fca9c0b2370 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 --2- 172.21.15.70:0/1345398321 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fca8c003970 0x7fca9c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.957+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 >> 172.21.15.70:0/1345398321 conn(0x7fca9c069850 msgr2=0x7fca9c070230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.958+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 shutdown_connections 2023-08-25T09:45:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:02.958+0000 7fcaa30a7640 1 -- 172.21.15.70:0/1345398321 wait complete. 2023-08-25T09:45:02.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:45:02.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:45:03.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.195+0000 7ff194004640 1 Processor -- start 2023-08-25T09:45:03.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.196+0000 7ff194004640 1 -- start start 2023-08-25T09:45:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff194004640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff18c15e210 0x7ff18c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff194004640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 0x7ff18c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff194004640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18c0ac880 con 0x7ff18c069850 2023-08-25T09:45:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff194004640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff18c15e650 con 0x7ff18c15e210 2023-08-25T09:45:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff194004640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0ad400 con 0x7ff18c0cca20 2023-08-25T09:45:03.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff191da0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff18c069850 0x7ff18c0c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43688/0 (socket says 172.21.15.70:43688) 2023-08-25T09:45:03.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.197+0000 7ff191da0640 1 -- 172.21.15.70:0/2465464925 learned_addr learned my addr 172.21.15.70:0/2465464925 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912937877 0 0) 0x7ff18c0ac880 con 0x7ff18c069850 2023-08-25T09:45:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff19159f640 1 --2- 172.21.15.70:0/2465464925 >> v2:172.21.15.155:3300/0 conn(0x7ff18c15e210 0x7ff18c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff174003520 con 0x7ff18c069850 2023-08-25T09:45:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff1925a1640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 0x7ff18c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 830823949 0 0) 0x7ff174003520 con 0x7ff18c069850 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 msgr2=0x7ff18c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 0x7ff18c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 >> v2:172.21.15.155:3300/0 conn(0x7ff18c15e210 msgr2=0x7ff18c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 --2- 172.21.15.70:0/2465464925 >> v2:172.21.15.155:3300/0 conn(0x7ff18c15e210 0x7ff18c069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.198+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0ad890 con 0x7ff18c069850 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff17c0019f0 con 0x7ff18c069850 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 >> v1:172.21.15.70:6789/0 conn(0x7ff18c069850 legacy=0x7ff18c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 0x7ff174003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff174003ef0 0x7ff1740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff174004d00 0x7ff174005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c15e650 con 0x7ff174003ef0 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0ad400 con 0x7ff174004d00 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17400ba60 con 0x7ff18c0cca20 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff19159f640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff174003ef0 0x7ff1740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff191da0640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 0x7ff174003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.199+0000 7ff1925a1640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff174004d00 0x7ff174005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.200+0000 7ff19159f640 1 -- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 msgr2=0x7ff174003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:03.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.200+0000 7ff19159f640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0cca20 0x7ff174003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.200+0000 7ff19159f640 1 -- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff174004d00 msgr2=0x7ff174005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.200+0000 7ff19159f640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff174004d00 0x7ff174005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.200+0000 7ff19159f640 1 -- 172.21.15.70:0/2465464925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff174003520 con 0x7ff174003ef0 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.202+0000 7ff19159f640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff174003ef0 0x7ff1740047c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff18000ba30 tx=0x7ff1740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b11c5d46b717f7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.203+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180032680 con 0x7ff174003ef0 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.203+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1800327e0 con 0x7ff174003ef0 2023-08-25T09:45:03.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.203+0000 7ff190d9e640 1 -- 172.21.15.70:0/2465464925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff180032b70 con 0x7ff174003ef0 2023-08-25T09:45:03.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff194004640 1 -- 172.21.15.70:0/2465464925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff174003ef0 msgr2=0x7ff1740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff194004640 1 --2- 172.21.15.70:0/2465464925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff174003ef0 0x7ff1740047c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ff18000ba30 tx=0x7ff1740057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff1925a1640 1 -- 172.21.15.70:0/2465464925 reap_dead start 2023-08-25T09:45:03.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff194004640 1 -- 172.21.15.70:0/2465464925 shutdown_connections 2023-08-25T09:45:03.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff194004640 1 -- 172.21.15.70:0/2465464925 >> 172.21.15.70:0/2465464925 conn(0x7ff18c0cb0d0 msgr2=0x7ff18c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:03.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff194004640 1 -- 172.21.15.70:0/2465464925 shutdown_connections 2023-08-25T09:45:03.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.204+0000 7ff194004640 1 -- 172.21.15.70:0/2465464925 wait complete. 2023-08-25T09:45:03.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.206+0000 7ff194004640 1 Processor -- start 2023-08-25T09:45:03.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.206+0000 7ff194004640 1 -- start start 2023-08-25T09:45:03.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff194004640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 0x7ff18c0c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff194004640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0c5090 0x7ff18c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff194004640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c0b2310 0x7ff18c1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff194004640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0c5bd0 con 0x7ff18c14df60 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff194004640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0b2790 con 0x7ff18c0b2310 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff194004640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0b41f0 con 0x7ff18c0c5090 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff191da0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 0x7ff18c0c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff191da0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 0x7ff18c0c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55160/0 (socket says 172.21.15.70:55160) 2023-08-25T09:45:03.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.207+0000 7ff191da0640 1 -- 172.21.15.70:0/1607096159 learned_addr learned my addr 172.21.15.70:0/1607096159 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:03.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff191da0640 1 -- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0c5090 msgr2=0x7ff18c0b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff191da0640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0c5090 0x7ff18c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff1925a1640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c0b2310 0x7ff18c1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff191da0640 1 -- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c0b2310 msgr2=0x7ff18c1f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff191da0640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c0b2310 0x7ff18c1f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff191da0640 1 -- 172.21.15.70:0/1607096159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0c6680 con 0x7ff18c14df60 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff191da0640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 0x7ff18c0c5690 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00a3e0 tx=0x7ff17c00a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c013600 con 0x7ff18c14df60 2023-08-25T09:45:03.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c013760 con 0x7ff18c14df60 2023-08-25T09:45:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c0c5de0 con 0x7ff18c14df60 2023-08-25T09:45:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c013b10 con 0x7ff18c14df60 2023-08-25T09:45:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.208+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c0c63a0 con 0x7ff18c14df60 2023-08-25T09:45:03.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.209+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c066fb0 con 0x7ff18c14df60 2023-08-25T09:45:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.210+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff17c013c70 con 0x7ff18c14df60 2023-08-25T09:45:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.211+0000 7ff18a7fc640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff17806a080 0x7ff17806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.212+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff17c089350 con 0x7ff18c14df60 2023-08-25T09:45:03.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.212+0000 7ff19159f640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff17806a080 0x7ff17806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.213+0000 7ff19159f640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff17806a080 0x7ff17806c540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff18000ba00 tx=0x7ff180032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:03.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.216+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff17c054e00 con 0x7ff18c14df60 2023-08-25T09:45:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.415+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff18c06a810 con 0x7ff18c14df60 2023-08-25T09:45:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.416+0000 7ff18a7fc640 1 -- 172.21.15.70:0/1607096159 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff17c017020 con 0x7ff18c14df60 2023-08-25T09:45:03.417 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:45:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.418+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff17806a080 msgr2=0x7ff17806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.418+0000 7ff194004640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff17806a080 0x7ff17806c540 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff18000ba00 tx=0x7ff180032000 comp rx=0 tx=0).stop 2023-08-25T09:45:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.418+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 msgr2=0x7ff18c0c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.418+0000 7ff194004640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 0x7ff18c0c5690 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00a3e0 tx=0x7ff17c00a8a0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.418+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 shutdown_connections 2023-08-25T09:45:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff17806a080 0x7ff17806c540 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff18c0b2310 0x7ff18c1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff18c0c5090 0x7ff18c0b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 --2- 172.21.15.70:0/1607096159 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff18c14df60 0x7ff18c0c5690 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 >> 172.21.15.70:0/1607096159 conn(0x7ff18c0cb0d0 msgr2=0x7ff18c0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 shutdown_connections 2023-08-25T09:45:03.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.419+0000 7ff194004640 1 -- 172.21.15.70:0/1607096159 wait complete. 2023-08-25T09:45:03.439 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:45:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.659+0000 7f1e7b8ec640 1 Processor -- start 2023-08-25T09:45:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.659+0000 7f1e7b8ec640 1 -- start start 2023-08-25T09:45:03.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.661+0000 7f1e7b8ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7415ae50 0x7f1e740c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.661+0000 7f1e7b8ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e74069f60 0x7f1e740ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.661+0000 7f1e7b8ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740cc5f0 con 0x7f1e740cdc20 2023-08-25T09:45:03.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.661+0000 7f1e7b8ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e74070650 con 0x7f1e74069f60 2023-08-25T09:45:03.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.661+0000 7f1e7b8ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74070930 con 0x7f1e7415ae50 2023-08-25T09:45:03.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.662+0000 7f1e79e89640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e74069f60 0x7f1e740ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.662+0000 7f1e79e89640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e74069f60 0x7f1e740ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:55732/0 (socket says 172.21.15.70:55732) 2023-08-25T09:45:03.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.662+0000 7f1e79e89640 1 -- 172.21.15.70:0/698149080 learned_addr learned my addr 172.21.15.70:0/698149080 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.662+0000 7f1e79688640 1 --2- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7415ae50 0x7f1e740c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.663+0000 7f1e79e89640 1 -- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7415ae50 msgr2=0x7f1e740c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.663+0000 7f1e79e89640 1 --2- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7415ae50 0x7f1e740c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.663+0000 7f1e79e89640 1 -- 172.21.15.70:0/698149080 >> v1:172.21.15.155:6789/0 conn(0x7f1e740cdc20 legacy=0x7f1e74069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.663+0000 7f1e79e89640 1 -- 172.21.15.70:0/698149080 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74070c10 con 0x7f1e74069f60 2023-08-25T09:45:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.664+0000 7f1e79e89640 1 --2- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e740ccad0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009e50 tx=0x7f1e6c02bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b70b58b437aecaa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.665+0000 7f1e6bfff640 1 -- 172.21.15.70:0/698149080 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c02cb90 con 0x7f1e74069f60 2023-08-25T09:45:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.665+0000 7f1e6bfff640 1 -- 172.21.15.70:0/698149080 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c02ccf0 con 0x7f1e74069f60 2023-08-25T09:45:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.665+0000 7f1e6bfff640 1 -- 172.21.15.70:0/698149080 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c034610 con 0x7f1e74069f60 2023-08-25T09:45:03.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.665+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 msgr2=0x7f1e740ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.665+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e740ccad0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009e50 tx=0x7f1e6c02bff0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.666+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/698149080 shutdown_connections 2023-08-25T09:45:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.666+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e740ccad0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.666+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/698149080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e7415ae50 0x7f1e740c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.666+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/698149080 >> 172.21.15.70:0/698149080 conn(0x7f1e740cb0d0 msgr2=0x7f1e740cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.666+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/698149080 shutdown_connections 2023-08-25T09:45:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.666+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/698149080 wait complete. 2023-08-25T09:45:03.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.668+0000 7f1e7b8ec640 1 Processor -- start 2023-08-25T09:45:03.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.668+0000 7f1e7b8ec640 1 -- start start 2023-08-25T09:45:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e7b8ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e7415dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e7b8ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cdc20 0x7f1e740b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e7b8ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 0x7f1e74157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e7b8ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7415b3b0 con 0x7f1e7415ae50 2023-08-25T09:45:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e7b8ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7415e1c0 con 0x7f1e74069f60 2023-08-25T09:45:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e7b8ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7415b600 con 0x7f1e740cdc20 2023-08-25T09:45:03.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 0x7f1e74157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 0x7f1e74157880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55162/0 (socket says 172.21.15.70:55162) 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 -- 172.21.15.70:0/2720325057 learned_addr learned my addr 172.21.15.70:0/2720325057 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79688640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e7415dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e78e87640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cdc20 0x7f1e740b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 -- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cdc20 msgr2=0x7f1e740b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cdc20 0x7f1e740b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 -- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 msgr2=0x7f1e7415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e7415dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.669+0000 7f1e79e89640 1 -- 172.21.15.70:0/2720325057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c009b30 con 0x7f1e7415ae50 2023-08-25T09:45:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e78e87640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cdc20 0x7f1e740b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e79e89640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 0x7f1e74157880 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009e50 tx=0x7f1e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c03e450 con 0x7f1e7415ae50 2023-08-25T09:45:03.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c03e5b0 con 0x7f1e7415ae50 2023-08-25T09:45:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7415b8e0 con 0x7f1e7415ae50 2023-08-25T09:45:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c03e960 con 0x7f1e7415ae50 2023-08-25T09:45:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.670+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7415bea0 con 0x7f1e7415ae50 2023-08-25T09:45:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.671+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7415c060 con 0x7f1e7415ae50 2023-08-25T09:45:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.672+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e6c03eac0 con 0x7f1e7415ae50 2023-08-25T09:45:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.673+0000 7f1e69ffb640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5806a050 0x7f1e5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.673+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1e6c0aadb0 con 0x7f1e7415ae50 2023-08-25T09:45:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.674+0000 7f1e79688640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5806a050 0x7f1e5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:03.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.674+0000 7f1e79688640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5806a050 0x7f1e5806c510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009a30 tx=0x7f1e70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:03.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.678+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e6c0af070 con 0x7f1e7415ae50 2023-08-25T09:45:03.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.869+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e74066fb0 con 0x7f1e7415ae50 2023-08-25T09:45:03.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.870+0000 7f1e69ffb640 1 -- 172.21.15.70:0/2720325057 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v111) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e6c076750 con 0x7f1e7415ae50 2023-08-25T09:45:03.872 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:45:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.872+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5806a050 msgr2=0x7f1e5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.872+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5806a050 0x7f1e5806c510 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009a30 tx=0x7f1e70008040 comp rx=0 tx=0).stop 2023-08-25T09:45:03.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.872+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 msgr2=0x7f1e74157880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.872+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 0x7f1e74157880 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c009e50 tx=0x7f1e6c004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 shutdown_connections 2023-08-25T09:45:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e5806a050 0x7f1e5806c510 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e7415ae50 0x7f1e74157880 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e740cdc20 0x7f1e740b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 --2- 172.21.15.70:0/2720325057 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e74069f60 0x7f1e7415dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:03.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 >> 172.21.15.70:0/2720325057 conn(0x7f1e740cb0d0 msgr2=0x7f1e740af900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:03.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 shutdown_connections 2023-08-25T09:45:03.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:03.873+0000 7f1e7b8ec640 1 -- 172.21.15.70:0/2720325057 wait complete. 2023-08-25T09:45:03.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:45:04.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.100+0000 7fd7ee057640 1 Processor -- start 2023-08-25T09:45:04.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.100+0000 7fd7ee057640 1 -- start start 2023-08-25T09:45:04.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.101+0000 7fd7ee057640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd7e815e830 0x7fd7e806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.101+0000 7fd7ee057640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7e806b2b0 0x7fd7e815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.101+0000 7fd7ee057640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e80b1b60 con 0x7fd7e815c6d0 2023-08-25T09:45:04.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.101+0000 7fd7ee057640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e806bd80 con 0x7fd7e806b2b0 2023-08-25T09:45:04.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.101+0000 7fd7ee057640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e806b7c0 con 0x7fd7e815e830 2023-08-25T09:45:04.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7e7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd7e815c6d0 0x7fd7e80b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43698/0 (socket says 172.21.15.70:43698) 2023-08-25T09:45:04.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7e7fff640 1 -- 172.21.15.70:0/2919509827 learned_addr learned my addr 172.21.15.70:0/2919509827 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:04.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7dffff640 1 --2- 172.21.15.70:0/2919509827 >> v2:172.21.15.70:3300/0 conn(0x7fd7e806b2b0 0x7fd7e815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7e77fe640 1 --2- 172.21.15.70:0/2919509827 >> v2:172.21.15.155:3300/0 conn(0x7fd7e815e830 0x7fd7e806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7dffff640 1 -- 172.21.15.70:0/2919509827 >> v2:172.21.15.155:3300/0 conn(0x7fd7e815e830 msgr2=0x7fd7e806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7e6ffd640 1 -- 172.21.15.70:0/2919509827 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223089377 0 0) 0x7fd7e80b1b60 con 0x7fd7e815c6d0 2023-08-25T09:45:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7dffff640 1 --2- 172.21.15.70:0/2919509827 >> v2:172.21.15.155:3300/0 conn(0x7fd7e815e830 0x7fd7e806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7dffff640 1 -- 172.21.15.70:0/2919509827 >> v1:172.21.15.70:6789/0 conn(0x7fd7e815c6d0 legacy=0x7fd7e80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7dffff640 1 -- 172.21.15.70:0/2919509827 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e806baa0 con 0x7fd7e806b2b0 2023-08-25T09:45:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.102+0000 7fd7e77fe640 1 --2- 172.21.15.70:0/2919509827 >> v2:172.21.15.155:3300/0 conn(0x7fd7e815e830 0x7fd7e806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.103+0000 7fd7dffff640 1 --2- 172.21.15.70:0/2919509827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e806b2b0 0x7fd7e815c190 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806bf10 tx=0x7fd7d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951787eaa35cadc2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7e6ffd640 1 -- 172.21.15.70:0/2919509827 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0031830 con 0x7fd7e806b2b0 2023-08-25T09:45:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7e6ffd640 1 -- 172.21.15.70:0/2919509827 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d0031990 con 0x7fd7e806b2b0 2023-08-25T09:45:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7e6ffd640 1 -- 172.21.15.70:0/2919509827 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0031d20 con 0x7fd7e806b2b0 2023-08-25T09:45:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 -- 172.21.15.70:0/2919509827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e806b2b0 msgr2=0x7fd7e815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 --2- 172.21.15.70:0/2919509827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e806b2b0 0x7fd7e815c190 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd7e806bf10 tx=0x7fd7d002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 -- 172.21.15.70:0/2919509827 shutdown_connections 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 --2- 172.21.15.70:0/2919509827 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e806b2b0 0x7fd7e815c190 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 --2- 172.21.15.70:0/2919509827 >> v2:172.21.15.155:3300/0 conn(0x7fd7e815e830 0x7fd7e806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 -- 172.21.15.70:0/2919509827 >> 172.21.15.70:0/2919509827 conn(0x7fd7e8069850 msgr2=0x7fd7e8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 -- 172.21.15.70:0/2919509827 shutdown_connections 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.104+0000 7fd7ee057640 1 -- 172.21.15.70:0/2919509827 wait complete. 2023-08-25T09:45:04.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.106+0000 7fd7ee057640 1 Processor -- start 2023-08-25T09:45:04.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.106+0000 7fd7ee057640 1 -- start start 2023-08-25T09:45:04.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.107+0000 7fd7ee057640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e806b2b0 0x7fd7e8152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.107+0000 7fd7ee057640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e815c6d0 0x7fd7e80aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.107+0000 7fd7ee057640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 0x7fd7e81548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.107+0000 7fd7ee057640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e80af2f0 con 0x7fd7e815e830 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.107+0000 7fd7ee057640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e81eff30 con 0x7fd7e806b2b0 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.107+0000 7fd7ee057640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e8152fe0 con 0x7fd7e815c6d0 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 0x7fd7e81548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 0x7fd7e81548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55186/0 (socket says 172.21.15.70:55186) 2023-08-25T09:45:04.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 -- 172.21.15.70:0/986724565 learned_addr learned my addr 172.21.15.70:0/986724565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 -- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e815c6d0 msgr2=0x7fd7e80aedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e815c6d0 0x7fd7e80aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e77fe640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e806b2b0 0x7fd7e8152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 -- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e806b2b0 msgr2=0x7fd7e8152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e806b2b0 0x7fd7e8152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 -- 172.21.15.70:0/986724565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0009400 con 0x7fd7e815e830 2023-08-25T09:45:04.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.108+0000 7fd7e7fff640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 0x7fd7e81548f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c7e0 tx=0x7fd7e000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:04.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.109+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000a640 con 0x7fd7e815e830 2023-08-25T09:45:04.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.109+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e000a7c0 con 0x7fd7e815e830 2023-08-25T09:45:04.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.109+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e81532c0 con 0x7fd7e815e830 2023-08-25T09:45:04.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.109+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e000ab70 con 0x7fd7e815e830 2023-08-25T09:45:04.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.109+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e8153880 con 0x7fd7e815e830 2023-08-25T09:45:04.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.110+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e81f0100 con 0x7fd7e815e830 2023-08-25T09:45:04.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.111+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd7e000acd0 con 0x7fd7e815e830 2023-08-25T09:45:04.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.112+0000 7fd7e4ff9640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7c806a050 0x7fd7c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.112+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd7e0087b50 con 0x7fd7e815e830 2023-08-25T09:45:04.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.112+0000 7fd7e77fe640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7c806a050 0x7fd7c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.113+0000 7fd7e77fe640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7c806a050 0x7fd7c806c510 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800b840 tx=0x7fd7d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:04.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.116+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd7e00534f0 con 0x7fd7e815e830 2023-08-25T09:45:04.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.305+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd7e8153a70 con 0x7fd7e815e830 2023-08-25T09:45:04.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.306+0000 7fd7e4ff9640 1 -- 172.21.15.70:0/986724565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v111) v1 ==== 84+0+126 (secure 0 0 0) 0x7fd7e0018020 con 0x7fd7e815e830 2023-08-25T09:45:04.308 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:04.308 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:45:04.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:04.308 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:45:04.308 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:45:04.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:04.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:04.309 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:45:04.309 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:45:04.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:04.309 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:45:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.308+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7c806a050 msgr2=0x7fd7c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.308+0000 7fd7ee057640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7c806a050 0x7fd7c806c510 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800b840 tx=0x7fd7d8008040 comp rx=0 tx=0).stop 2023-08-25T09:45:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.308+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 msgr2=0x7fd7e81548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.308+0000 7fd7ee057640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 0x7fd7e81548f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c7e0 tx=0x7fd7e000cca0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 shutdown_connections 2023-08-25T09:45:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7c806a050 0x7fd7c806c510 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7e815e830 0x7fd7e81548f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7e815c6d0 0x7fd7e80aedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 --2- 172.21.15.70:0/986724565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7e806b2b0 0x7fd7e8152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 >> 172.21.15.70:0/986724565 conn(0x7fd7e8069850 msgr2=0x7fd7e815a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:04.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 shutdown_connections 2023-08-25T09:45:04.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.309+0000 7fd7ee057640 1 -- 172.21.15.70:0/986724565 wait complete. 2023-08-25T09:45:04.327 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:45:04.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:45:04.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.541+0000 7ff0d896a640 1 Processor -- start 2023-08-25T09:45:04.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.541+0000 7ff0d896a640 1 -- start start 2023-08-25T09:45:04.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d896a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff0d4155120 0x7ff0d40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d896a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff0d40cd890 0x7ff0d40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d896a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d415e6f0 con 0x7ff0d40ca820 2023-08-25T09:45:04.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d896a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40cdd40 con 0x7ff0d40cd890 2023-08-25T09:45:04.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d896a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d40caf30 con 0x7ff0d4155120 2023-08-25T09:45:04.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff0d40cd890 0x7ff0d40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff0d40cd890 0x7ff0d40ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55188/0 (socket says 172.21.15.70:55188) 2023-08-25T09:45:04.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 -- 172.21.15.70:0/1498038137 learned_addr learned my addr 172.21.15.70:0/1498038137 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 -- 172.21.15.70:0/1498038137 >> v2:172.21.15.155:3300/0 conn(0x7ff0d4155120 msgr2=0x7ff0d40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 --2- 172.21.15.70:0/1498038137 >> v2:172.21.15.155:3300/0 conn(0x7ff0d4155120 0x7ff0d40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 -- 172.21.15.70:0/1498038137 >> v1:172.21.15.155:6789/0 conn(0x7ff0d40ca820 legacy=0x7ff0d406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.543+0000 7ff0d1d9b640 1 -- 172.21.15.70:0/1498038137 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d415e9d0 con 0x7ff0d40cd890 2023-08-25T09:45:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.545+0000 7ff0d1d9b640 1 --2- 172.21.15.70:0/1498038137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d40cd890 0x7ff0d40ca2e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40ce610 tx=0x7ff0c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a04a577e606a637 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.545+0000 7ff0d159a640 1 -- 172.21.15.70:0/1498038137 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8031870 con 0x7ff0d40cd890 2023-08-25T09:45:04.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.545+0000 7ff0d159a640 1 -- 172.21.15.70:0/1498038137 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c80319d0 con 0x7ff0d40cd890 2023-08-25T09:45:04.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.545+0000 7ff0d159a640 1 -- 172.21.15.70:0/1498038137 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8031d60 con 0x7ff0d40cd890 2023-08-25T09:45:04.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 -- 172.21.15.70:0/1498038137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d40cd890 msgr2=0x7ff0d40ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 --2- 172.21.15.70:0/1498038137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d40cd890 0x7ff0d40ca2e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40ce610 tx=0x7ff0c802b910 comp rx=0 tx=0).stop 2023-08-25T09:45:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 -- 172.21.15.70:0/1498038137 shutdown_connections 2023-08-25T09:45:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 --2- 172.21.15.70:0/1498038137 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d40cd890 0x7ff0d40ca2e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 --2- 172.21.15.70:0/1498038137 >> v2:172.21.15.155:3300/0 conn(0x7ff0d4155120 0x7ff0d40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 -- 172.21.15.70:0/1498038137 >> 172.21.15.70:0/1498038137 conn(0x7ff0d4069c70 msgr2=0x7ff0d406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:04.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 -- 172.21.15.70:0/1498038137 shutdown_connections 2023-08-25T09:45:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.546+0000 7ff0d896a640 1 -- 172.21.15.70:0/1498038137 wait complete. 2023-08-25T09:45:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.548+0000 7ff0d896a640 1 Processor -- start 2023-08-25T09:45:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.548+0000 7ff0d896a640 1 -- start start 2023-08-25T09:45:04.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d896a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0d40ca820 0x7ff0d415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d896a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d40cd890 0x7ff0d4159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d896a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 0x7ff0d4146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d896a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d415d5d0 con 0x7ff0d4155120 2023-08-25T09:45:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d896a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d415d8b0 con 0x7ff0d40cd890 2023-08-25T09:45:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d896a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d415db90 con 0x7ff0d40ca820 2023-08-25T09:45:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 0x7ff0d4146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 0x7ff0d4146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55198/0 (socket says 172.21.15.70:55198) 2023-08-25T09:45:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 -- 172.21.15.70:0/4035325951 learned_addr learned my addr 172.21.15.70:0/4035325951 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d259c640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0d40ca820 0x7ff0d415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d1d9b640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d40cd890 0x7ff0d4159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 -- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0d40ca820 msgr2=0x7ff0d415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0d40ca820 0x7ff0d415cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 -- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d40cd890 msgr2=0x7ff0d4159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d40cd890 0x7ff0d4159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.549+0000 7ff0d2d9d640 1 -- 172.21.15.70:0/4035325951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c80094d0 con 0x7ff0d4155120 2023-08-25T09:45:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.550+0000 7ff0d2d9d640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 0x7ff0d4146010 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c780 tx=0x7ff0cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.550+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc003d80 con 0x7ff0d4155120 2023-08-25T09:45:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.550+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc00ce90 con 0x7ff0d4155120 2023-08-25T09:45:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.550+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0052b0 con 0x7ff0d4155120 2023-08-25T09:45:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.550+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d415ae90 con 0x7ff0d4155120 2023-08-25T09:45:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.550+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d415a2f0 con 0x7ff0d4155120 2023-08-25T09:45:04.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.551+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d415b3b0 con 0x7ff0d4155120 2023-08-25T09:45:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.552+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff0cc005410 con 0x7ff0d4155120 2023-08-25T09:45:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.553+0000 7ff0c2ffd640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0b406a050 0x7ff0b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.553+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff0cc0877f0 con 0x7ff0d4155120 2023-08-25T09:45:04.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.553+0000 7ff0d259c640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0b406a050 0x7ff0b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.554+0000 7ff0d259c640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0b406a050 0x7ff0b406c510 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400a770 tx=0x7ff0c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:04.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.557+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff0d415b3b0 con 0x7ff0d4155120 2023-08-25T09:45:04.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.744+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff0d415b3b0 con 0x7ff0d4155120 2023-08-25T09:45:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.976+0000 7ff0c2ffd640 1 -- 172.21.15.70:0/4035325951 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v112) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff0d415b3b0 con 0x7ff0d4155120 2023-08-25T09:45:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:45:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.984+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0b406a050 msgr2=0x7ff0b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.984+0000 7ff0d896a640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0b406a050 0x7ff0b406c510 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400a770 tx=0x7ff0c4008040 comp rx=0 tx=0).stop 2023-08-25T09:45:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.984+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 msgr2=0x7ff0d4146010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.984+0000 7ff0d896a640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 0x7ff0d4146010 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c780 tx=0x7ff0cc00cc40 comp rx=0 tx=0).stop 2023-08-25T09:45:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.985+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 shutdown_connections 2023-08-25T09:45:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.985+0000 7ff0d896a640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff0b406a050 0x7ff0b406c510 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.985+0000 7ff0d896a640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff0d4155120 0x7ff0d4146010 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.985+0000 7ff0d896a640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff0d40cd890 0x7ff0d4159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.985+0000 7ff0d896a640 1 --2- 172.21.15.70:0/4035325951 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff0d40ca820 0x7ff0d415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.986+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 >> 172.21.15.70:0/4035325951 conn(0x7ff0d4069c70 msgr2=0x7ff0d40cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.986+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 shutdown_connections 2023-08-25T09:45:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:04.986+0000 7ff0d896a640 1 -- 172.21.15.70:0/4035325951 wait complete. 2023-08-25T09:45:05.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:45:05.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.212+0000 7fac5916b640 1 Processor -- start 2023-08-25T09:45:05.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.212+0000 7fac5916b640 1 -- start start 2023-08-25T09:45:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.213+0000 7fac5916b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fac540cd660 0x7fac540cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.213+0000 7fac5916b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5406f8e0 con 0x7fac54159bf0 2023-08-25T09:45:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.213+0000 7fac5916b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5406fbc0 con 0x7fac5415e240 2023-08-25T09:45:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.213+0000 7fac5916b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac540ce030 con 0x7fac540cd660 2023-08-25T09:45:05.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac52d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fac54159bf0 0x7fac5406f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43700/0 (socket says 172.21.15.70:43700) 2023-08-25T09:45:05.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac52d9d640 1 -- 172.21.15.70:0/1570978454 learned_addr learned my addr 172.21.15.70:0/1570978454 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:05.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181902134 0 0) 0x7fac5406f8e0 con 0x7fac54159bf0 2023-08-25T09:45:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac40005730 con 0x7fac54159bf0 2023-08-25T09:45:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5359e640 1 --2- 172.21.15.70:0/1570978454 >> v2:172.21.15.155:3300/0 conn(0x7fac540cd660 0x7fac540cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1604041763 0 0) 0x7fac40005730 con 0x7fac54159bf0 2023-08-25T09:45:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 >> v2:172.21.15.155:3300/0 conn(0x7fac540cd660 msgr2=0x7fac540cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:05.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 --2- 172.21.15.70:0/1570978454 >> v2:172.21.15.155:3300/0 conn(0x7fac540cd660 0x7fac540cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 >> v1:172.21.15.155:6789/0 conn(0x7fac5415e240 legacy=0x7fac54070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5406fea0 con 0x7fac54159bf0 2023-08-25T09:45:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fac44001670 con 0x7fac54159bf0 2023-08-25T09:45:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 >> v1:172.21.15.70:6789/0 conn(0x7fac54159bf0 legacy=0x7fac5406f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:05.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.214+0000 7fac5259c640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac40005b80 0x7fac40005f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5259c640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac400069d0 0x7fac40006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5259c640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac40007390 0x7fac4000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac540ce030 con 0x7fac400069d0 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4000e2c0 con 0x7fac40007390 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4000e480 con 0x7fac40005b80 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac400069d0 0x7fac40006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac4bfff640 1 -- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac40005b80 msgr2=0x7fac40005f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac40005b80 0x7fac40005f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac4bfff640 1 -- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac40007390 msgr2=0x7fac4000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5359e640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac40007390 0x7fac4000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac40007390 0x7fac4000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:05.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac4bfff640 1 -- 172.21.15.70:0/1570978454 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac40005730 con 0x7fac400069d0 2023-08-25T09:45:05.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.215+0000 7fac5359e640 1 -- 172.21.15.70:0/1570978454 reap_dead start 2023-08-25T09:45:05.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.216+0000 7fac4bfff640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac400069d0 0x7fac40006e50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fac3c001620 tx=0x7fac40007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcf4e3966f5c40e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:05.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.217+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c005070 con 0x7fac400069d0 2023-08-25T09:45:05.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.217+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c02e070 con 0x7fac400069d0 2023-08-25T09:45:05.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.217+0000 7fac5259c640 1 -- 172.21.15.70:0/1570978454 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c005430 con 0x7fac400069d0 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.217+0000 7fac5916b640 1 -- 172.21.15.70:0/1570978454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac400069d0 msgr2=0x7fac40006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.217+0000 7fac5916b640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac400069d0 0x7fac40006e50 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fac3c001620 tx=0x7fac40007a00 comp rx=0 tx=0).stop 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.218+0000 7fac5916b640 1 -- 172.21.15.70:0/1570978454 shutdown_connections 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.218+0000 7fac5916b640 1 --2- 172.21.15.70:0/1570978454 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac400069d0 0x7fac40006e50 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.218+0000 7fac5916b640 1 -- 172.21.15.70:0/1570978454 >> 172.21.15.70:0/1570978454 conn(0x7fac540cb0d0 msgr2=0x7fac540cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.218+0000 7fac5916b640 1 -- 172.21.15.70:0/1570978454 shutdown_connections 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.218+0000 7fac5916b640 1 -- 172.21.15.70:0/1570978454 wait complete. 2023-08-25T09:45:05.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.220+0000 7fac5916b640 1 Processor -- start 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.220+0000 7fac5916b640 1 -- start start 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac5916b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac400069d0 0x7fac54153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac5916b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 0x7fac540c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac5916b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac540b1270 0x7fac540c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac5916b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac540b38f0 con 0x7fac54154300 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac5916b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac540c3bc0 con 0x7fac400069d0 2023-08-25T09:45:05.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac5916b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac540b40c0 con 0x7fac540b1270 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 0x7fac540c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 0x7fac540c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55222/0 (socket says 172.21.15.70:55222) 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.221+0000 7fac4bfff640 1 -- 172.21.15.70:0/2974081341 learned_addr learned my addr 172.21.15.70:0/2974081341 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac52d9d640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac400069d0 0x7fac54153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4bfff640 1 -- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac540b1270 msgr2=0x7fac540c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac5359e640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac540b1270 0x7fac540c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4bfff640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac540b1270 0x7fac540c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4bfff640 1 -- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac400069d0 msgr2=0x7fac54153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4bfff640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac400069d0 0x7fac54153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4bfff640 1 -- 172.21.15.70:0/2974081341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac54153f60 con 0x7fac54154300 2023-08-25T09:45:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac52d9d640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac400069d0 0x7fac54153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4bfff640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 0x7fac540c20d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fac3c001740 tx=0x7fac3c000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c042070 con 0x7fac54154300 2023-08-25T09:45:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c0054a0 con 0x7fac54154300 2023-08-25T09:45:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c007240 con 0x7fac54154300 2023-08-25T09:45:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.222+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac540b3b00 con 0x7fac54154300 2023-08-25T09:45:05.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.223+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac540c3df0 con 0x7fac54154300 2023-08-25T09:45:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.224+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac540c4310 con 0x7fac54154300 2023-08-25T09:45:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.225+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fac3c030030 con 0x7fac54154300 2023-08-25T09:45:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.226+0000 7fac4b7fe640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac3406a080 0x7fac3406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.226+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fac3c0a8940 con 0x7fac54154300 2023-08-25T09:45:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.226+0000 7fac52d9d640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac3406a080 0x7fac3406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:05.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.227+0000 7fac52d9d640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac3406a080 0x7fac3406c540 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac440017a0 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:05.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.231+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fac540c4310 con 0x7fac54154300 2023-08-25T09:45:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.424+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fac540c4310 con 0x7fac54154300 2023-08-25T09:45:05.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:05.997+0000 7fac4b7fe640 1 -- 172.21.15.70:0/2974081341 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v113) v1 ==== 170+0+0 (secure 0 0 0) 0x7fac540c4310 con 0x7fac54154300 2023-08-25T09:45:06.004 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:45:06.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.011+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac3406a080 msgr2=0x7fac3406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.011+0000 7fac5916b640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac3406a080 0x7fac3406c540 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fac440017a0 tx=0x7fac44009040 comp rx=0 tx=0).stop 2023-08-25T09:45:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.011+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 msgr2=0x7fac540c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.011+0000 7fac5916b640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 0x7fac540c20d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fac3c001740 tx=0x7fac3c000fb0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.012+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 shutdown_connections 2023-08-25T09:45:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.012+0000 7fac5916b640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac3406a080 0x7fac3406c540 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.012+0000 7fac5916b640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac540b1270 0x7fac540c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.012+0000 7fac5916b640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac54154300 0x7fac540c20d0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.012+0000 7fac5916b640 1 --2- 172.21.15.70:0/2974081341 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac400069d0 0x7fac54153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.012+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 >> 172.21.15.70:0/2974081341 conn(0x7fac540cb0d0 msgr2=0x7fac540ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.016+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 shutdown_connections 2023-08-25T09:45:06.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.016+0000 7fac5916b640 1 -- 172.21.15.70:0/2974081341 wait complete. 2023-08-25T09:45:06.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:45:06.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.233+0000 7f0e3c03f640 1 Processor -- start 2023-08-25T09:45:06.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.233+0000 7f0e3c03f640 1 -- start start 2023-08-25T09:45:06.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.235+0000 7f0e3c03f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 0x7f0e3415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.235+0000 7f0e3c03f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3406b110 con 0x7f0e34072a90 2023-08-25T09:45:06.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.235+0000 7f0e3c03f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e340cb7e0 con 0x7f0e34072e70 2023-08-25T09:45:06.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.235+0000 7f0e3c03f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340cc080 con 0x7f0e340cb250 2023-08-25T09:45:06.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.235+0000 7f0e39ddb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0e34072a90 0x7f0e3406aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43716/0 (socket says 172.21.15.70:43716) 2023-08-25T09:45:06.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.235+0000 7f0e39ddb640 1 -- 172.21.15.70:0/3597989080 learned_addr learned my addr 172.21.15.70:0/3597989080 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:06.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e3a5dc640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 0x7f0e3415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196879368 0 0) 0x7f0e3406b110 con 0x7f0e34072a90 2023-08-25T09:45:06.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e20003520 con 0x7f0e34072a90 2023-08-25T09:45:06.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 697206760 0 0) 0x7f0e20003520 con 0x7f0e34072a90 2023-08-25T09:45:06.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 msgr2=0x7f0e3415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:06.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 0x7f0e3415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 >> v1:172.21.15.155:6789/0 conn(0x7f0e34072e70 legacy=0x7f0e34070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340cbac0 con 0x7f0e34072a90 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0e30001670 con 0x7f0e34072a90 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 >> v1:172.21.15.70:6789/0 conn(0x7f0e34072a90 legacy=0x7f0e3406aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 0x7f0e200039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e20003f20 0x7f0e200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e20004d00 0x7f0e20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340cc080 con 0x7f0e20004d00 2023-08-25T09:45:06.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000ba60 con 0x7f0e20003f20 2023-08-25T09:45:06.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.236+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2000bc20 con 0x7f0e340cb250 2023-08-25T09:45:06.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e3a5dc640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e20004d00 0x7f0e20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e39ddb640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 0x7f0e200039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e395da640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e20003f20 0x7f0e200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e3a5dc640 1 -- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 msgr2=0x7f0e200039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:06.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e3a5dc640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e340cb250 0x7f0e200039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e3a5dc640 1 -- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e20003f20 msgr2=0x7f0e200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:06.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e3a5dc640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e20003f20 0x7f0e200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e3a5dc640 1 -- 172.21.15.70:0/3597989080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20003520 con 0x7f0e20004d00 2023-08-25T09:45:06.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.237+0000 7f0e395da640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e20003f20 0x7f0e200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:06.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.238+0000 7f0e3a5dc640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e20004d00 0x7f0e20005180 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009fc0 tx=0x7f0e200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e569655f3f7d468 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:06.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.238+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c02cc20 con 0x7f0e20004d00 2023-08-25T09:45:06.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.238+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00f040 con 0x7f0e20004d00 2023-08-25T09:45:06.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.238+0000 7f0e38dd9640 1 -- 172.21.15.70:0/3597989080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c034440 con 0x7f0e20004d00 2023-08-25T09:45:06.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3c03f640 1 -- 172.21.15.70:0/3597989080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e20004d00 msgr2=0x7f0e20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/3597989080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e20004d00 0x7f0e20005180 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009fc0 tx=0x7f0e200057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3a5dc640 1 -- 172.21.15.70:0/3597989080 reap_dead start 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3c03f640 1 -- 172.21.15.70:0/3597989080 shutdown_connections 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3c03f640 1 -- 172.21.15.70:0/3597989080 >> 172.21.15.70:0/3597989080 conn(0x7f0e340cf9c0 msgr2=0x7f0e340cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3c03f640 1 -- 172.21.15.70:0/3597989080 shutdown_connections 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.239+0000 7f0e3c03f640 1 -- 172.21.15.70:0/3597989080 wait complete. 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.241+0000 7f0e3c03f640 1 Processor -- start 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.241+0000 7f0e3c03f640 1 -- start start 2023-08-25T09:45:06.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e3c03f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e20004d00 0x7f0e3414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e3c03f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 0x7f0e340af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e3c03f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e34150950 0x7f0e34150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e3c03f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e34151310 con 0x7f0e34154440 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e3c03f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340acc80 con 0x7f0e34150950 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e3c03f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340b15c0 con 0x7f0e20004d00 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e395da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 0x7f0e340af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e395da640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 0x7f0e340af880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55246/0 (socket says 172.21.15.70:55246) 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.242+0000 7f0e395da640 1 -- 172.21.15.70:0/1517933311 learned_addr learned my addr 172.21.15.70:0/1517933311 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e3a5dc640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e34150950 0x7f0e34150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e39ddb640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e20004d00 0x7f0e3414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e395da640 1 -- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e20004d00 msgr2=0x7f0e3414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e395da640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e20004d00 0x7f0e3414e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e395da640 1 -- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e34150950 msgr2=0x7f0e34150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e395da640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e34150950 0x7f0e34150dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:06.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e395da640 1 -- 172.21.15.70:0/1517933311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340b18a0 con 0x7f0e34154440 2023-08-25T09:45:06.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e395da640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 0x7f0e340af880 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400d7a0 tx=0x7f0e2400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:06.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24017070 con 0x7f0e34154440 2023-08-25T09:45:06.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e24005070 con 0x7f0e34154440 2023-08-25T09:45:06.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e341548c0 con 0x7f0e34154440 2023-08-25T09:45:06.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.243+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24005420 con 0x7f0e34154440 2023-08-25T09:45:06.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.244+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e340afdc0 con 0x7f0e34154440 2023-08-25T09:45:06.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.245+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e34073ee0 con 0x7f0e34154440 2023-08-25T09:45:06.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.246+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0e24007030 con 0x7f0e34154440 2023-08-25T09:45:06.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.246+0000 7f0e2a7fc640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e1806a080 0x7f0e1806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:06.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.247+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0e24087e80 con 0x7f0e34154440 2023-08-25T09:45:06.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.247+0000 7f0e39ddb640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e1806a080 0x7f0e1806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:06.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.247+0000 7f0e39ddb640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e1806a080 0x7f0e1806c540 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e30001050 tx=0x7f0e30001d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:06.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.251+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0e24053530 con 0x7f0e34154440 2023-08-25T09:45:06.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.475+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0e34066fb0 con 0x7f0e34154440 2023-08-25T09:45:07.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:06.999+0000 7f0e2a7fc640 1 -- 172.21.15.70:0/1517933311 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v46) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0e24056b50 con 0x7f0e34154440 2023-08-25T09:45:07.001 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 13 and data pool 14 2023-08-25T09:45:07.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.011+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e1806a080 msgr2=0x7f0e1806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.011+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e1806a080 0x7f0e1806c540 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e30001050 tx=0x7f0e30001d40 comp rx=0 tx=0).stop 2023-08-25T09:45:07.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.011+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 msgr2=0x7f0e340af880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.012+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 0x7f0e340af880 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0e2400d7a0 tx=0x7f0e2400dc60 comp rx=0 tx=0).stop 2023-08-25T09:45:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 shutdown_connections 2023-08-25T09:45:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e1806a080 0x7f0e1806c540 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e34150950 0x7f0e34150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e34154440 0x7f0e340af880 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 --2- 172.21.15.70:0/1517933311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e20004d00 0x7f0e3414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 >> 172.21.15.70:0/1517933311 conn(0x7f0e340cf9c0 msgr2=0x7f0e340b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:07.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.014+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 shutdown_connections 2023-08-25T09:45:07.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.017+0000 7f0e3c03f640 1 -- 172.21.15.70:0/1517933311 wait complete. 2023-08-25T09:45:07.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:07.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.225+0000 7fbc8fbe8640 1 Processor -- start 2023-08-25T09:45:07.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.225+0000 7fbc8fbe8640 1 -- start start 2023-08-25T09:45:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.226+0000 7fbc8fbe8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc8815e830 0x7fbc8806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.226+0000 7fbc8fbe8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815ec10 0x7fbc880b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8fbe8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880b1b60 con 0x7fbc8806b2b0 2023-08-25T09:45:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8fbe8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8806bd80 con 0x7fbc8815e830 2023-08-25T09:45:07.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8fbe8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806b7c0 con 0x7fbc8815ec10 2023-08-25T09:45:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc8815e830 0x7fbc8806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbc8815e830 0x7fbc8806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55258/0 (socket says 172.21.15.70:55258) 2023-08-25T09:45:07.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 -- 172.21.15.70:0/1229539086 learned_addr learned my addr 172.21.15.70:0/1229539086 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:07.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8e185640 1 --2- 172.21.15.70:0/1229539086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815ec10 0x7fbc880b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 -- 172.21.15.70:0/1229539086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815ec10 msgr2=0x7fbc880b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:07.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 --2- 172.21.15.70:0/1229539086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815ec10 0x7fbc880b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 -- 172.21.15.70:0/1229539086 >> v1:172.21.15.70:6789/0 conn(0x7fbc8806b2b0 legacy=0x7fbc8815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8d984640 1 -- 172.21.15.70:0/1229539086 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8806baa0 con 0x7fbc8815e830 2023-08-25T09:45:07.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.227+0000 7fbc8c982640 1 -- 172.21.15.70:0/1229539086 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2627354508 0 0) 0x7fbc880b1b60 con 0x7fbc8806b2b0 2023-08-25T09:45:07.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.229+0000 7fbc8d984640 1 --2- 172.21.15.70:0/1229539086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815e830 0x7fbc8806ad70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009870 tx=0x7fbc8402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b803b8ff1a17df server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:07.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.229+0000 7fbc8c982640 1 -- 172.21.15.70:0/1229539086 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84030840 con 0x7fbc8815e830 2023-08-25T09:45:07.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.229+0000 7fbc8c982640 1 -- 172.21.15.70:0/1229539086 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc840309a0 con 0x7fbc8815e830 2023-08-25T09:45:07.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.229+0000 7fbc8c982640 1 -- 172.21.15.70:0/1229539086 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84030d30 con 0x7fbc8815e830 2023-08-25T09:45:07.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.229+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/1229539086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815e830 msgr2=0x7fbc8806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.229+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/1229539086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815e830 0x7fbc8806ad70 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009870 tx=0x7fbc8402b980 comp rx=0 tx=0).stop 2023-08-25T09:45:07.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.230+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/1229539086 shutdown_connections 2023-08-25T09:45:07.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.230+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/1229539086 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815ec10 0x7fbc880b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.230+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/1229539086 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815e830 0x7fbc8806ad70 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.230+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/1229539086 >> 172.21.15.70:0/1229539086 conn(0x7fbc88069850 msgr2=0x7fbc88069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:07.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.230+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/1229539086 shutdown_connections 2023-08-25T09:45:07.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.230+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/1229539086 wait complete. 2023-08-25T09:45:07.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.231+0000 7fbc8fbe8640 1 Processor -- start 2023-08-25T09:45:07.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.232+0000 7fbc8fbe8640 1 -- start start 2023-08-25T09:45:07.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.232+0000 7fbc8fbe8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc8806b2b0 0x7fbc88144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.232+0000 7fbc8fbe8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815e830 0x7fbc880c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8fbe8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 0x7fbc880c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8fbe8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880708c0 con 0x7fbc8815ec10 2023-08-25T09:45:07.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8fbe8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88144fb0 con 0x7fbc8806b2b0 2023-08-25T09:45:07.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8fbe8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806f820 con 0x7fbc8815e830 2023-08-25T09:45:07.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 0x7fbc880c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 0x7fbc880c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55274/0 (socket says 172.21.15.70:55274) 2023-08-25T09:45:07.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 -- 172.21.15.70:0/3543223803 learned_addr learned my addr 172.21.15.70:0/3543223803 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:07.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 -- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815e830 msgr2=0x7fbc880c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815e830 0x7fbc880c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8d984640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc8806b2b0 0x7fbc88144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 -- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc8806b2b0 msgr2=0x7fbc88144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc8806b2b0 0x7fbc88144a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8e185640 1 -- 172.21.15.70:0/3543223803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840094d0 con 0x7fbc8815ec10 2023-08-25T09:45:07.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.233+0000 7fbc8d984640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc8806b2b0 0x7fbc88144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:07.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.234+0000 7fbc8e185640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 0x7fbc880c7c90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000cc60 tx=0x7fbc8000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:07.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.234+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8000ce70 con 0x7fbc8815ec10 2023-08-25T09:45:07.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.234+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc80003990 con 0x7fbc8815ec10 2023-08-25T09:45:07.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.234+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc80003d20 con 0x7fbc8815ec10 2023-08-25T09:45:07.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.234+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8815ca20 con 0x7fbc8815ec10 2023-08-25T09:45:07.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.234+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8806fde0 con 0x7fbc8815ec10 2023-08-25T09:45:07.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.235+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8804e420 con 0x7fbc8815ec10 2023-08-25T09:45:07.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.236+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbc80003e80 con 0x7fbc8815ec10 2023-08-25T09:45:07.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.237+0000 7fbc7e7fc640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc6c06a050 0x7fbc6c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.237+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbc800889e0 con 0x7fbc8815ec10 2023-08-25T09:45:07.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.237+0000 7fbc8d984640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc6c06a050 0x7fbc6c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.238+0000 7fbc8d984640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc6c06a050 0x7fbc6c06c510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc84036000 tx=0x7fbc8403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:07.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.241+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbc800540c0 con 0x7fbc8815ec10 2023-08-25T09:45:07.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.432+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc88156890 con 0x7fbc8815ec10 2023-08-25T09:45:07.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.434+0000 7fbc7e7fc640 1 -- 172.21.15.70:0/3543223803 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18382 (secure 0 0 0) 0x7fbc8001d020 con 0x7fbc8815ec10 2023-08-25T09:45:07.436 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:07.437 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:06.476998+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.436+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc6c06a050 msgr2=0x7fbc6c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.436+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc6c06a050 0x7fbc6c06c510 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbc84036000 tx=0x7fbc8403b040 comp rx=0 tx=0).stop 2023-08-25T09:45:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.436+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 msgr2=0x7fbc880c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.436+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 0x7fbc880c7c90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbc8000cc60 tx=0x7fbc8000a5c0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 shutdown_connections 2023-08-25T09:45:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbc6c06a050 0x7fbc6c06c510 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbc8815ec10 0x7fbc880c7c90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbc8815e830 0x7fbc880c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 --2- 172.21.15.70:0/3543223803 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbc8806b2b0 0x7fbc88144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 >> 172.21.15.70:0/3543223803 conn(0x7fbc88069850 msgr2=0x7fbc8815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 shutdown_connections 2023-08-25T09:45:07.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.437+0000 7fbc8fbe8640 1 -- 172.21.15.70:0/3543223803 wait complete. 2023-08-25T09:45:07.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.670+0000 7f7c291e1640 1 Processor -- start 2023-08-25T09:45:07.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.670+0000 7f7c291e1640 1 -- start start 2023-08-25T09:45:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c291e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415e830 0x7f7c2406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c291e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7c2415ec10 0x7f7c240b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c291e1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240b1b60 con 0x7f7c2406b2b0 2023-08-25T09:45:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c291e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2406bd80 con 0x7f7c2415ec10 2023-08-25T09:45:07.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c291e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2406b7c0 con 0x7f7c2415e830 2023-08-25T09:45:07.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c2259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7c2406b2b0 0x7f7c2415c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43740/0 (socket says 172.21.15.70:43740) 2023-08-25T09:45:07.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c2259c640 1 -- 172.21.15.70:0/2376181891 learned_addr learned my addr 172.21.15.70:0/2376181891 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:07.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.672+0000 7f7c2359e640 1 --2- 172.21.15.70:0/2376181891 >> v2:172.21.15.70:3300/0 conn(0x7f7c2415ec10 0x7f7c240b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.673+0000 7f7c22d9d640 1 --2- 172.21.15.70:0/2376181891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415e830 0x7f7c2406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.673+0000 7f7c21d9b640 1 -- 172.21.15.70:0/2376181891 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136022662 0 0) 0x7f7c240b1b60 con 0x7f7c2406b2b0 2023-08-25T09:45:07.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.673+0000 7f7c2359e640 1 -- 172.21.15.70:0/2376181891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415e830 msgr2=0x7f7c2406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:07.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.673+0000 7f7c2359e640 1 --2- 172.21.15.70:0/2376181891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415e830 0x7f7c2406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.673+0000 7f7c2359e640 1 -- 172.21.15.70:0/2376181891 >> v1:172.21.15.70:6789/0 conn(0x7f7c2406b2b0 legacy=0x7f7c2415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.673+0000 7f7c2359e640 1 -- 172.21.15.70:0/2376181891 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2406baa0 con 0x7f7c2415ec10 2023-08-25T09:45:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.674+0000 7f7c2359e640 1 --2- 172.21.15.70:0/2376181891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2415ec10 0x7f7c240b1ff0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009d90 tx=0x7f7c1c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5211e725e2769dc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.674+0000 7f7c21d9b640 1 -- 172.21.15.70:0/2376181891 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c02cc50 con 0x7f7c2415ec10 2023-08-25T09:45:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.674+0000 7f7c21d9b640 1 -- 172.21.15.70:0/2376181891 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c02cdb0 con 0x7f7c2415ec10 2023-08-25T09:45:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.674+0000 7f7c21d9b640 1 -- 172.21.15.70:0/2376181891 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c034660 con 0x7f7c2415ec10 2023-08-25T09:45:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 -- 172.21.15.70:0/2376181891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2415ec10 msgr2=0x7f7c240b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2376181891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2415ec10 0x7f7c240b1ff0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009d90 tx=0x7f7c1c02bea0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 -- 172.21.15.70:0/2376181891 shutdown_connections 2023-08-25T09:45:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2376181891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2415ec10 0x7f7c240b1ff0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2376181891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415e830 0x7f7c2406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 -- 172.21.15.70:0/2376181891 >> 172.21.15.70:0/2376181891 conn(0x7f7c24069850 msgr2=0x7f7c24069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:07.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 -- 172.21.15.70:0/2376181891 shutdown_connections 2023-08-25T09:45:07.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.675+0000 7f7c291e1640 1 -- 172.21.15.70:0/2376181891 wait complete. 2023-08-25T09:45:07.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.677+0000 7f7c291e1640 1 Processor -- start 2023-08-25T09:45:07.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.677+0000 7f7c291e1640 1 -- start start 2023-08-25T09:45:07.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c291e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 0x7f7c240ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c291e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c2415e830 0x7f7c24148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c291e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415ec10 0x7f7c24149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c291e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2406f6b0 con 0x7f7c2406b2b0 2023-08-25T09:45:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c291e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2414c8f0 con 0x7f7c2415e830 2023-08-25T09:45:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c291e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2406ed80 con 0x7f7c2415ec10 2023-08-25T09:45:07.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c22d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 0x7f7c240ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c22d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 0x7f7c240ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55294/0 (socket says 172.21.15.70:55294) 2023-08-25T09:45:07.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.678+0000 7f7c22d9d640 1 -- 172.21.15.70:0/2311212870 learned_addr learned my addr 172.21.15.70:0/2311212870 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:07.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c2259c640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c2415e830 0x7f7c24148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c22d9d640 1 -- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415ec10 msgr2=0x7f7c24149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c22d9d640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415ec10 0x7f7c24149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c22d9d640 1 -- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c2415e830 msgr2=0x7f7c24148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c22d9d640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c2415e830 0x7f7c24148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c22d9d640 1 -- 172.21.15.70:0/2311212870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c009a70 con 0x7f7c2406b2b0 2023-08-25T09:45:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c2259c640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c2415e830 0x7f7c24148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:07.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c22d9d640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 0x7f7c240ce300 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400d800 tx=0x7f7c1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400b6c0 con 0x7f7c2406b2b0 2023-08-25T09:45:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1400b820 con 0x7f7c2406b2b0 2023-08-25T09:45:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1400bbb0 con 0x7f7c2406b2b0 2023-08-25T09:45:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2415ca20 con 0x7f7c2406b2b0 2023-08-25T09:45:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.679+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2406f340 con 0x7f7c2406b2b0 2023-08-25T09:45:07.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.680+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2404e420 con 0x7f7c2406b2b0 2023-08-25T09:45:07.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.681+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7c1400bd10 con 0x7f7c2406b2b0 2023-08-25T09:45:07.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.682+0000 7f7c0b7fe640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c1006a050 0x7f7c1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:07.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.682+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7c14088bc0 con 0x7f7c2406b2b0 2023-08-25T09:45:07.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.683+0000 7f7c2259c640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c1006a050 0x7f7c1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:07.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.683+0000 7f7c2259c640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c1006a050 0x7f7c1006c510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7c2414ffb0 tx=0x7f7c18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:07.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.687+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7c14054210 con 0x7f7c2406b2b0 2023-08-25T09:45:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.877+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c240caca0 con 0x7f7c2406b2b0 2023-08-25T09:45:07.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.880+0000 7f7c0b7fe640 1 -- 172.21.15.70:0/2311212870 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18382 (secure 0 0 0) 0x7f7c14057830 con 0x7f7c2406b2b0 2023-08-25T09:45:07.882 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:07.883 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":114,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:06.476998+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c1006a050 msgr2=0x7f7c1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c1006a050 0x7f7c1006c510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7c2414ffb0 tx=0x7f7c18009040 comp rx=0 tx=0).stop 2023-08-25T09:45:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 msgr2=0x7f7c240ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 0x7f7c240ce300 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400d800 tx=0x7f7c1400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 shutdown_connections 2023-08-25T09:45:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7c1006a050 0x7f7c1006c510 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7c2415ec10 0x7f7c24149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7c2415e830 0x7f7c24148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 --2- 172.21.15.70:0/2311212870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7c2406b2b0 0x7f7c240ce300 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 >> 172.21.15.70:0/2311212870 conn(0x7f7c24069850 msgr2=0x7f7c2415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 shutdown_connections 2023-08-25T09:45:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:07.883+0000 7f7c291e1640 1 -- 172.21.15.70:0/2311212870 wait complete. 2023-08-25T09:45:07.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:45:08.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.121+0000 7fce409cf640 1 Processor -- start 2023-08-25T09:45:08.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.122+0000 7fce409cf640 1 -- start start 2023-08-25T09:45:08.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce409cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 0x7fce3c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce409cf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fce3c0c9c10 0x7fce3c0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce409cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c1597d0 con 0x7fce3c06ad30 2023-08-25T09:45:08.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce409cf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c15a5b0 con 0x7fce3c0c9c10 2023-08-25T09:45:08.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce409cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15a890 con 0x7fce3c0c9790 2023-08-25T09:45:08.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce3a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fce3c06ad30 0x7fce3c06b110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43742/0 (socket says 172.21.15.70:43742) 2023-08-25T09:45:08.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.123+0000 7fce3a59c640 1 -- 172.21.15.70:0/2497561121 learned_addr learned my addr 172.21.15.70:0/2497561121 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:08.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637530825 0 0) 0x7fce3c1597d0 con 0x7fce3c06ad30 2023-08-25T09:45:08.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce24003520 con 0x7fce3c06ad30 2023-08-25T09:45:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3ad9d640 1 --2- 172.21.15.70:0/2497561121 >> v2:172.21.15.155:3300/0 conn(0x7fce3c0c9c10 0x7fce3c0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce39d9b640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 0x7fce3c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919284566 0 0) 0x7fce24003520 con 0x7fce3c06ad30 2023-08-25T09:45:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 msgr2=0x7fce3c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 0x7fce3c0ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 >> v2:172.21.15.155:3300/0 conn(0x7fce3c0c9c10 msgr2=0x7fce3c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 --2- 172.21.15.70:0/2497561121 >> v2:172.21.15.155:3300/0 conn(0x7fce3c0c9c10 0x7fce3c0ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0c8cb0 con 0x7fce3c06ad30 2023-08-25T09:45:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fce2c001670 con 0x7fce3c06ad30 2023-08-25T09:45:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 >> v1:172.21.15.70:6789/0 conn(0x7fce3c06ad30 legacy=0x7fce3c06b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:08.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 0x7fce24003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24003fa0 0x7fce240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15a5b0 con 0x7fce24004d00 2023-08-25T09:45:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c15a890 con 0x7fce24003fa0 2023-08-25T09:45:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.124+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2400bb90 con 0x7fce3c0c9790 2023-08-25T09:45:08.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce3ad9d640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:08.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce39d9b640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24003fa0 0x7fce240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:08.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce3ad9d640 1 -- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 msgr2=0x7fce24003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:08.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce3ad9d640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c9790 0x7fce24003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce3ad9d640 1 -- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24003fa0 msgr2=0x7fce240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:08.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce3ad9d640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce24003fa0 0x7fce240047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.125+0000 7fce3ad9d640 1 -- 172.21.15.70:0/2497561121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce24003520 con 0x7fce24004d00 2023-08-25T09:45:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.126+0000 7fce3ad9d640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce24005180 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fce3c06aae0 tx=0x7fce240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d16b92cc9714a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.126+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3402cca0 con 0x7fce24004d00 2023-08-25T09:45:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.126+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3402ce00 con 0x7fce24004d00 2023-08-25T09:45:08.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.126+0000 7fce3959a640 1 -- 172.21.15.70:0/2497561121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34034680 con 0x7fce24004d00 2023-08-25T09:45:08.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce409cf640 1 -- 172.21.15.70:0/2497561121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 msgr2=0x7fce24005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:08.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce409cf640 1 --2- 172.21.15.70:0/2497561121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce24005180 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fce3c06aae0 tx=0x7fce240057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce3ad9d640 1 -- 172.21.15.70:0/2497561121 reap_dead start 2023-08-25T09:45:08.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce409cf640 1 -- 172.21.15.70:0/2497561121 shutdown_connections 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce409cf640 1 -- 172.21.15.70:0/2497561121 >> 172.21.15.70:0/2497561121 conn(0x7fce3c0cb0d0 msgr2=0x7fce3c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce409cf640 1 -- 172.21.15.70:0/2497561121 shutdown_connections 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.127+0000 7fce409cf640 1 -- 172.21.15.70:0/2497561121 wait complete. 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.129+0000 7fce409cf640 1 Processor -- start 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.129+0000 7fce409cf640 1 -- start start 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce409cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce3c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce409cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c0cd3e0 0x7fce3c0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce409cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c7b70 0x7fce3c0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce409cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0c4620 con 0x7fce24004d00 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce409cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0cdda0 con 0x7fce3c0cd3e0 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce409cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c0c6320 con 0x7fce3c0c7b70 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce3c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce3c14d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55310/0 (socket says 172.21.15.70:55310) 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 -- 172.21.15.70:0/2923668 learned_addr learned my addr 172.21.15.70:0/2923668 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:08.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 -- 172.21.15.70:0/2923668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c7b70 msgr2=0x7fce3c0c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c7b70 0x7fce3c0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 -- 172.21.15.70:0/2923668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c0cd3e0 msgr2=0x7fce3c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c0cd3e0 0x7fce3c0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.130+0000 7fce3a59c640 1 -- 172.21.15.70:0/2923668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0c57a0 con 0x7fce24004d00 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.131+0000 7fce3a59c640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce3c14d8a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c7a0 tx=0x7fce2c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.131+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0074e0 con 0x7fce24004d00 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.131+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c007640 con 0x7fce24004d00 2023-08-25T09:45:08.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.131+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c004a00 con 0x7fce24004d00 2023-08-25T09:45:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.131+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c0c5a80 con 0x7fce24004d00 2023-08-25T09:45:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.131+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c0c6040 con 0x7fce24004d00 2023-08-25T09:45:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.132+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c0c2560 con 0x7fce24004d00 2023-08-25T09:45:08.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.133+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fce2c004b60 con 0x7fce24004d00 2023-08-25T09:45:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.134+0000 7fce2affd640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fce1c06a080 0x7fce1c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.134+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce2c0879a0 con 0x7fce24004d00 2023-08-25T09:45:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.134+0000 7fce39d9b640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fce1c06a080 0x7fce1c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:08.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.135+0000 7fce39d9b640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fce1c06a080 0x7fce1c06c540 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce30006b10 tx=0x7fce30006c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:08.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.138+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fce3c0c2560 con 0x7fce24004d00 2023-08-25T09:45:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:08.363+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fce3c0c2560 con 0x7fce24004d00 2023-08-25T09:45:09.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.345+0000 7fce2affd640 1 -- 172.21.15.70:0/2923668 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) v1 ==== 118+0+0 (secure 0 0 0) 0x7fce3c0c2560 con 0x7fce24004d00 2023-08-25T09:45:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.348+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fce1c06a080 msgr2=0x7fce1c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.348+0000 7fce409cf640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fce1c06a080 0x7fce1c06c540 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce30006b10 tx=0x7fce30006c80 comp rx=0 tx=0).stop 2023-08-25T09:45:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.348+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 msgr2=0x7fce3c14d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.348+0000 7fce409cf640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce3c14d8a0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c7a0 tx=0x7fce2c00cc60 comp rx=0 tx=0).stop 2023-08-25T09:45:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.348+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 shutdown_connections 2023-08-25T09:45:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.348+0000 7fce409cf640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fce1c06a080 0x7fce1c06c540 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.349+0000 7fce409cf640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fce3c0c7b70 0x7fce3c0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.349+0000 7fce409cf640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fce3c0cd3e0 0x7fce3c0cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.349+0000 7fce409cf640 1 --2- 172.21.15.70:0/2923668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fce24004d00 0x7fce3c14d8a0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.349+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 >> 172.21.15.70:0/2923668 conn(0x7fce3c0cb0d0 msgr2=0x7fce3c15cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.349+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 shutdown_connections 2023-08-25T09:45:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.349+0000 7fce409cf640 1 -- 172.21.15.70:0/2923668 wait complete. 2023-08-25T09:45:09.369 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:45:09.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:45:09.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.590+0000 7fd9b6339640 1 Processor -- start 2023-08-25T09:45:09.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.590+0000 7fd9b6339640 1 -- start start 2023-08-25T09:45:09.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.591+0000 7fd9b6339640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd9b00cd660 0x7fd9b00cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.591+0000 7fd9b6339640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b006f8e0 con 0x7fd9b0159bf0 2023-08-25T09:45:09.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.591+0000 7fd9b6339640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b006fbc0 con 0x7fd9b015e240 2023-08-25T09:45:09.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.591+0000 7fd9b6339640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b00ce030 con 0x7fd9b00cd660 2023-08-25T09:45:09.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.592+0000 7fd9affff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd9b0159bf0 0x7fd9b006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37620/0 (socket says 172.21.15.70:37620) 2023-08-25T09:45:09.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.592+0000 7fd9affff640 1 -- 172.21.15.70:0/1198214234 learned_addr learned my addr 172.21.15.70:0/1198214234 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.592+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613772666 0 0) 0x7fd9b006f8e0 con 0x7fd9b0159bf0 2023-08-25T09:45:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.592+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd998003520 con 0x7fd9b0159bf0 2023-08-25T09:45:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.592+0000 7fd9b48d6640 1 --2- 172.21.15.70:0/1198214234 >> v2:172.21.15.155:3300/0 conn(0x7fd9b00cd660 0x7fd9b00cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 144924682 0 0) 0x7fd998003520 con 0x7fd9b0159bf0 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 >> v2:172.21.15.155:3300/0 conn(0x7fd9b00cd660 msgr2=0x7fd9b00cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 --2- 172.21.15.70:0/1198214234 >> v2:172.21.15.155:3300/0 conn(0x7fd9b00cd660 0x7fd9b00cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 >> v1:172.21.15.155:6789/0 conn(0x7fd9b015e240 legacy=0x7fd9b0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b006fea0 con 0x7fd9b0159bf0 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fd9a0001670 con 0x7fd9b0159bf0 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 >> v1:172.21.15.70:6789/0 conn(0x7fd9b0159bf0 legacy=0x7fd9b006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd998003970 0x7fd998003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd998004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd998005180 0x7fd99800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00ce030 con 0x7fd9980047c0 2023-08-25T09:45:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99800e2e0 con 0x7fd998003970 2023-08-25T09:45:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99800e4a0 con 0x7fd998005180 2023-08-25T09:45:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9af7fe640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd998004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:09.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9affff640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd998003970 0x7fd998003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.593+0000 7fd9af7fe640 1 -- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd998005180 msgr2=0x7fd99800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:09.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.594+0000 7fd9af7fe640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd998005180 0x7fd99800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.594+0000 7fd9af7fe640 1 -- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd998003970 msgr2=0x7fd998003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.594+0000 7fd9af7fe640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd998003970 0x7fd998003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.594+0000 7fd9af7fe640 1 -- 172.21.15.70:0/1198214234 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd998003520 con 0x7fd9980047c0 2023-08-25T09:45:09.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.594+0000 7fd9affff640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd998003970 0x7fd998003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.594+0000 7fd9b48d6640 1 -- 172.21.15.70:0/1198214234 reap_dead start 2023-08-25T09:45:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.595+0000 7fd9af7fe640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd998004c40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4001750 tx=0x7fd9980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4de3a85e03cb6a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.595+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40052d0 con 0x7fd9980047c0 2023-08-25T09:45:09.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.595+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a402e050 con 0x7fd9980047c0 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.595+0000 7fd9aeffd640 1 -- 172.21.15.70:0/1198214234 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4003830 con 0x7fd9980047c0 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.595+0000 7fd9b6339640 1 -- 172.21.15.70:0/1198214234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 msgr2=0x7fd998004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.595+0000 7fd9b6339640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd998004c40 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4001750 tx=0x7fd9980057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.596+0000 7fd9b6339640 1 -- 172.21.15.70:0/1198214234 shutdown_connections 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.596+0000 7fd9b6339640 1 --2- 172.21.15.70:0/1198214234 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd998004c40 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.596+0000 7fd9b6339640 1 -- 172.21.15.70:0/1198214234 >> 172.21.15.70:0/1198214234 conn(0x7fd9b00cb0d0 msgr2=0x7fd9b00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:09.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.596+0000 7fd9b6339640 1 -- 172.21.15.70:0/1198214234 shutdown_connections 2023-08-25T09:45:09.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.596+0000 7fd9b6339640 1 -- 172.21.15.70:0/1198214234 wait complete. 2023-08-25T09:45:09.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.597+0000 7fd9b6339640 1 Processor -- start 2023-08-25T09:45:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 -- start start 2023-08-25T09:45:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd9b01530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9b00c3d50 0x7fd9b00c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9b01549a0 0x7fd9b0154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0155360 con 0x7fd9980047c0 2023-08-25T09:45:09.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0153630 con 0x7fd9b00c3d50 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.598+0000 7fd9b6339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00c67d0 con 0x7fd9b01549a0 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd9b01530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd9b01530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52882/0 (socket says 172.21.15.70:52882) 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 -- 172.21.15.70:0/263367069 learned_addr learned my addr 172.21.15.70:0/263367069 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 -- 172.21.15.70:0/263367069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9b01549a0 msgr2=0x7fd9b0154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9b01549a0 0x7fd9b0154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 -- 172.21.15.70:0/263367069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9b00c3d50 msgr2=0x7fd9b00c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9b00c3d50 0x7fd9b00c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 -- 172.21.15.70:0/263367069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00c2920 con 0x7fd9980047c0 2023-08-25T09:45:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.599+0000 7fd9affff640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd9b01530f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000d7a0 tx=0x7fd9a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.600+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00074e0 con 0x7fd9980047c0 2023-08-25T09:45:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.600+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a0007640 con 0x7fd9980047c0 2023-08-25T09:45:09.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.600+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a0004a20 con 0x7fd9980047c0 2023-08-25T09:45:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.600+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b00c2c00 con 0x7fd9980047c0 2023-08-25T09:45:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.600+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b00c31c0 con 0x7fd9980047c0 2023-08-25T09:45:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.601+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b00c3380 con 0x7fd9980047c0 2023-08-25T09:45:09.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.602+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd9a0004c40 con 0x7fd9980047c0 2023-08-25T09:45:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.603+0000 7fd9acff9640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd99c06a050 0x7fd99c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:09.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.603+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9a0087e70 con 0x7fd9980047c0 2023-08-25T09:45:09.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.603+0000 7fd9af7fe640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd99c06a050 0x7fd99c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:09.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.604+0000 7fd9af7fe640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd99c06a050 0x7fd99c06c510 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4033000 tx=0x7fd9a402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:09.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.607+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd9a0053570 con 0x7fd9980047c0 2023-08-25T09:45:09.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:09.831+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd9b0066fb0 con 0x7fd9980047c0 2023-08-25T09:45:10.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.356+0000 7fd9acff9640 1 -- 172.21.15.70:0/263367069 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9a00190c0 con 0x7fd9980047c0 2023-08-25T09:45:10.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd99c06a050 msgr2=0x7fd99c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd99c06a050 0x7fd99c06c510 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4033000 tx=0x7fd9a402f000 comp rx=0 tx=0).stop 2023-08-25T09:45:10.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 msgr2=0x7fd9b01530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd9b01530f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd9a000d7a0 tx=0x7fd9a000dc60 comp rx=0 tx=0).stop 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 shutdown_connections 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd99c06a050 0x7fd99c06c510 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd9b01549a0 0x7fd9b0154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd9b00c3d50 0x7fd9b00c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 --2- 172.21.15.70:0/263367069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd9980047c0 0x7fd9b01530f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.359+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 >> 172.21.15.70:0/263367069 conn(0x7fd9b00cb0d0 msgr2=0x7fd9b006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.360+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 shutdown_connections 2023-08-25T09:45:10.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.360+0000 7fd9b6339640 1 -- 172.21.15.70:0/263367069 wait complete. 2023-08-25T09:45:10.383 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:45:10.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:10.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.598+0000 7ff4aa0b0640 1 Processor -- start 2023-08-25T09:45:10.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.598+0000 7ff4aa0b0640 1 -- start start 2023-08-25T09:45:10.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4aa0b0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4a4159840 0x7ff4a40b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:10.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4aa0b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4a40cadb0 0x7ff4a40cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:10.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4aa0b0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a40b4450 con 0x7ff4a415d3d0 2023-08-25T09:45:10.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4aa0b0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a40c8b60 con 0x7ff4a40cadb0 2023-08-25T09:45:10.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4aa0b0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a415d890 con 0x7ff4a4159840 2023-08-25T09:45:10.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4a98af640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4a40cadb0 0x7ff4a40cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:10.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4a98af640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff4a40cadb0 0x7ff4a40cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52894/0 (socket says 172.21.15.70:52894) 2023-08-25T09:45:10.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4a98af640 1 -- 172.21.15.70:0/2805978093 learned_addr learned my addr 172.21.15.70:0/2805978093 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:10.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4a88ad640 1 --1- 172.21.15.70:0/2805978093 >> v1:172.21.15.70:6789/0 conn(0x7ff4a415d3d0 0x7ff4a40b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37634/0 (socket says 172.21.15.70:37634) 2023-08-25T09:45:10.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4a90ae640 1 --2- 172.21.15.70:0/2805978093 >> v2:172.21.15.155:3300/0 conn(0x7ff4a4159840 0x7ff4a40b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:10.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.600+0000 7ff4a98af640 1 -- 172.21.15.70:0/2805978093 >> v2:172.21.15.155:3300/0 conn(0x7ff4a4159840 msgr2=0x7ff4a40b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:10.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.601+0000 7ff4a98af640 1 --2- 172.21.15.70:0/2805978093 >> v2:172.21.15.155:3300/0 conn(0x7ff4a4159840 0x7ff4a40b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.601+0000 7ff4a98af640 1 -- 172.21.15.70:0/2805978093 >> v1:172.21.15.70:6789/0 conn(0x7ff4a415d3d0 legacy=0x7ff4a40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.601+0000 7ff4a98af640 1 -- 172.21.15.70:0/2805978093 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a415db70 con 0x7ff4a40cadb0 2023-08-25T09:45:10.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.602+0000 7ff4a98af640 1 --2- 172.21.15.70:0/2805978093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a40cadb0 0x7ff4a40cb1d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff4a406bec0 tx=0x7ff49c02e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6054c0fd3b0a77 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:10.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.602+0000 7ff49bfff640 1 -- 172.21.15.70:0/2805978093 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00b0d0 con 0x7ff4a40cadb0 2023-08-25T09:45:10.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.602+0000 7ff49bfff640 1 -- 172.21.15.70:0/2805978093 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c00b230 con 0x7ff4a40cadb0 2023-08-25T09:45:10.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.602+0000 7ff49bfff640 1 -- 172.21.15.70:0/2805978093 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c009600 con 0x7ff4a40cadb0 2023-08-25T09:45:10.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.602+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/2805978093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a40cadb0 msgr2=0x7ff4a40cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/2805978093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a40cadb0 0x7ff4a40cb1d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7ff4a406bec0 tx=0x7ff49c02e180 comp rx=0 tx=0).stop 2023-08-25T09:45:10.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/2805978093 shutdown_connections 2023-08-25T09:45:10.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/2805978093 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a40cadb0 0x7ff4a40cb1d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/2805978093 >> v2:172.21.15.155:3300/0 conn(0x7ff4a4159840 0x7ff4a40b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/2805978093 >> 172.21.15.70:0/2805978093 conn(0x7ff4a40affd0 msgr2=0x7ff4a40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:10.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/2805978093 shutdown_connections 2023-08-25T09:45:10.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.603+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/2805978093 wait complete. 2023-08-25T09:45:10.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.604+0000 7ff4aa0b0640 1 Processor -- start 2023-08-25T09:45:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 -- start start 2023-08-25T09:45:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40cadb0 0x7ff4a414cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 0x7ff4a40ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:10.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a415d3d0 0x7ff4a40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a414d170 con 0x7ff4a4159840 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40af200 con 0x7ff4a40cadb0 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.605+0000 7ff4aa0b0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a40af4e0 con 0x7ff4a415d3d0 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 0x7ff4a40ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 0x7ff4a40ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52904/0 (socket says 172.21.15.70:52904) 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 -- 172.21.15.70:0/4222395339 learned_addr learned my addr 172.21.15.70:0/4222395339 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:10.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a90ae640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40cadb0 0x7ff4a414cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 -- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a415d3d0 msgr2=0x7ff4a40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a415d3d0 0x7ff4a40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 -- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40cadb0 msgr2=0x7ff4a414cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40cadb0 0x7ff4a414cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.606+0000 7ff4a88ad640 1 -- 172.21.15.70:0/4222395339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c00bd40 con 0x7ff4a4159840 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.607+0000 7ff4a88ad640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 0x7ff4a40ae840 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c3e0 tx=0x7ff49400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.607+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940074e0 con 0x7ff4a4159840 2023-08-25T09:45:10.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.607+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494007640 con 0x7ff4a4159840 2023-08-25T09:45:10.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.607+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a40af7c0 con 0x7ff4a4159840 2023-08-25T09:45:10.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.607+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4940049e0 con 0x7ff4a4159840 2023-08-25T09:45:10.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.607+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a40cbe80 con 0x7ff4a4159840 2023-08-25T09:45:10.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.608+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4066fb0 con 0x7ff4a4159840 2023-08-25T09:45:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.609+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff494004b40 con 0x7ff4a4159840 2023-08-25T09:45:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.610+0000 7ff499ffb640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff48806a080 0x7ff48806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.610+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff494087f40 con 0x7ff4a4159840 2023-08-25T09:45:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.610+0000 7ff4a90ae640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff48806a080 0x7ff48806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.611+0000 7ff4a90ae640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff48806a080 0x7ff48806c540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0006fd0 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:10.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.614+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff494053630 con 0x7ff4a4159840 2023-08-25T09:45:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.838+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4a4071b30 con 0x7ff4a4159840 2023-08-25T09:45:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.840+0000 7ff499ffb640 1 -- 172.21.15.70:0/4222395339 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7ff494056c50 con 0x7ff4a4159840 2023-08-25T09:45:10.842 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:10.843 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff48806a080 msgr2=0x7ff48806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff48806a080 0x7ff48806c540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0006fd0 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2023-08-25T09:45:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 msgr2=0x7ff4a40ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 0x7ff4a40ae840 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c3e0 tx=0x7ff49400c8a0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 shutdown_connections 2023-08-25T09:45:10.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff48806a080 0x7ff48806c540 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4a415d3d0 0x7ff4a40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4159840 0x7ff4a40ae840 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 --2- 172.21.15.70:0/4222395339 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a40cadb0 0x7ff4a414cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 >> 172.21.15.70:0/4222395339 conn(0x7ff4a40affd0 msgr2=0x7ff4a40c8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.842+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 shutdown_connections 2023-08-25T09:45:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:10.843+0000 7ff4aa0b0640 1 -- 172.21.15.70:0/4222395339 wait complete. 2023-08-25T09:45:10.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:11.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.052+0000 7f82fdc8e640 1 Processor -- start 2023-08-25T09:45:11.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.053+0000 7f82fdc8e640 1 -- start start 2023-08-25T09:45:11.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.054+0000 7f82fdc8e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82f80ac870 0x7f82f80b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.054+0000 7f82fdc8e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f806f050 con 0x7f82f80cea30 2023-08-25T09:45:11.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.054+0000 7f82fdc8e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f806f330 con 0x7f82f815cb40 2023-08-25T09:45:11.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.054+0000 7f82fdc8e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f806f610 con 0x7f82f80ac870 2023-08-25T09:45:11.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82effff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f82f80cea30 0x7f82f80b1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37644/0 (socket says 172.21.15.70:37644) 2023-08-25T09:45:11.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82effff640 1 -- 172.21.15.70:0/2502635340 learned_addr learned my addr 172.21.15.70:0/2502635340 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:11.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82f7fff640 1 --2- 172.21.15.70:0/2502635340 >> v2:172.21.15.70:3300/0 conn(0x7f82f80ac870 0x7f82f80b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82f6ffd640 1 -- 172.21.15.70:0/2502635340 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778909869 0 0) 0x7f82f806f050 con 0x7f82f80cea30 2023-08-25T09:45:11.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82f6ffd640 1 -- 172.21.15.70:0/2502635340 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e4003520 con 0x7f82f80cea30 2023-08-25T09:45:11.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82f7fff640 1 -- 172.21.15.70:0/2502635340 >> v1:172.21.15.155:6789/0 conn(0x7f82f815cb40 legacy=0x7f82f80ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:11.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82f7fff640 1 -- 172.21.15.70:0/2502635340 >> v1:172.21.15.70:6789/0 conn(0x7f82f80cea30 legacy=0x7f82f80b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.055+0000 7f82f7fff640 1 -- 172.21.15.70:0/2502635340 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f806f8f0 con 0x7f82f80ac870 2023-08-25T09:45:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.056+0000 7f82f7fff640 1 --2- 172.21.15.70:0/2502635340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f80b2240 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009d90 tx=0x7f82f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5722e5970b9402a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82f6ffd640 1 -- 172.21.15.70:0/2502635340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f002cc60 con 0x7f82f80ac870 2023-08-25T09:45:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82f6ffd640 1 -- 172.21.15.70:0/2502635340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f002cdc0 con 0x7f82f80ac870 2023-08-25T09:45:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82f6ffd640 1 -- 172.21.15.70:0/2502635340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0034660 con 0x7f82f80ac870 2023-08-25T09:45:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82fdc8e640 1 -- 172.21.15.70:0/2502635340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 msgr2=0x7f82f80b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/2502635340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f80b2240 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f82f0009d90 tx=0x7f82f002beb0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82fdc8e640 1 -- 172.21.15.70:0/2502635340 shutdown_connections 2023-08-25T09:45:11.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/2502635340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f80b2240 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82fdc8e640 1 -- 172.21.15.70:0/2502635340 >> 172.21.15.70:0/2502635340 conn(0x7f82f8069850 msgr2=0x7f82f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:11.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.057+0000 7f82fdc8e640 1 -- 172.21.15.70:0/2502635340 shutdown_connections 2023-08-25T09:45:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.058+0000 7f82fdc8e640 1 -- 172.21.15.70:0/2502635340 wait complete. 2023-08-25T09:45:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.059+0000 7f82fdc8e640 1 Processor -- start 2023-08-25T09:45:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.059+0000 7f82fdc8e640 1 -- start start 2023-08-25T09:45:11.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.060+0000 7f82fdc8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f814b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.060+0000 7f82fdc8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80cea30 0x7f82f8151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.060+0000 7f82fdc8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f815cb40 0x7f82f81f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.060+0000 7f82fdc8e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f814d070 con 0x7f82f80ac870 2023-08-25T09:45:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.060+0000 7f82fdc8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f8150800 con 0x7f82f815cb40 2023-08-25T09:45:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.060+0000 7f82fdc8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f814c6d0 con 0x7f82f80cea30 2023-08-25T09:45:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f814b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f814b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52928/0 (socket says 172.21.15.70:52928) 2023-08-25T09:45:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 -- 172.21.15.70:0/3949012011 learned_addr learned my addr 172.21.15.70:0/3949012011 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 -- 172.21.15.70:0/3949012011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80cea30 msgr2=0x7f82f8151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80cea30 0x7f82f8151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 -- 172.21.15.70:0/3949012011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f815cb40 msgr2=0x7f82f81f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f815cb40 0x7f82f81f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 -- 172.21.15.70:0/3949012011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0009a70 con 0x7f82f80ac870 2023-08-25T09:45:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.061+0000 7f82f77fe640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f814b230 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f82e800b3f0 tx=0x7f82e800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.062+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011830 con 0x7f82f80ac870 2023-08-25T09:45:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.062+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8011990 con 0x7f82f80ac870 2023-08-25T09:45:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.062+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011d40 con 0x7f82f80ac870 2023-08-25T09:45:11.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.062+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f814c9b0 con 0x7f82f80ac870 2023-08-25T09:45:11.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.062+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f81521e0 con 0x7f82f80ac870 2023-08-25T09:45:11.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.063+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f804e420 con 0x7f82f80ac870 2023-08-25T09:45:11.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.064+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f82e801a400 con 0x7f82f80ac870 2023-08-25T09:45:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.065+0000 7f82f4ff9640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.065+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82e80883b0 con 0x7f82f80ac870 2023-08-25T09:45:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.065+0000 7f82effff640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.066+0000 7f82effff640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f82e0001610 tx=0x7f82e0007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.069+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f82e8053aa0 con 0x7f82f80ac870 2023-08-25T09:45:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.249+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82f80cde90 con 0x7f82f80ac870 2023-08-25T09:45:11.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.252+0000 7f82f4ff9640 1 -- 172.21.15.70:0/3949012011 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+18369 (secure 0 0 0) 0x7f82e80570c0 con 0x7f82f80ac870 2023-08-25T09:45:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:11.256 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":115,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:07.990577+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:11.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 msgr2=0x7f82d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f82e0001610 tx=0x7f82e0007930 comp rx=0 tx=0).stop 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 msgr2=0x7f82f814b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f814b230 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f82e800b3f0 tx=0x7f82e800b8b0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 shutdown_connections 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f82d806a050 0x7f82d806c510 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82f815cb40 0x7f82f81f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82f80cea30 0x7f82f8151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 --2- 172.21.15.70:0/3949012011 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82f80ac870 0x7f82f814b230 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 >> 172.21.15.70:0/3949012011 conn(0x7f82f8069850 msgr2=0x7f82f80af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:11.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 shutdown_connections 2023-08-25T09:45:11.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.255+0000 7f82fdc8e640 1 -- 172.21.15.70:0/3949012011 wait complete. 2023-08-25T09:45:11.275 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:45:11.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:45:11.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.473+0000 7fa4b4d55640 1 Processor -- start 2023-08-25T09:45:11.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.474+0000 7fa4b4d55640 1 -- start start 2023-08-25T09:45:11.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.475+0000 7fa4b4d55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4b00b1ff0 0x7fa4b00b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.475+0000 7fa4b4d55640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b00b1b60 con 0x7fa4b015ec10 2023-08-25T09:45:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.475+0000 7fa4b4d55640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b006b7c0 con 0x7fa4b015e830 2023-08-25T09:45:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.475+0000 7fa4b4d55640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b006bd80 con 0x7fa4b00b1ff0 2023-08-25T09:45:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4aed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4b00b1ff0 0x7fa4b00b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4aed9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4b00b1ff0 0x7fa4b00b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52936/0 (socket says 172.21.15.70:52936) 2023-08-25T09:45:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4aed9d640 1 -- 172.21.15.70:0/2254169173 learned_addr learned my addr 172.21.15.70:0/2254169173 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4a7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa4b015ec10 0x7fa4b015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37648/0 (socket says 172.21.15.70:37648) 2023-08-25T09:45:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4aed9d640 1 -- 172.21.15.70:0/2254169173 >> v1:172.21.15.155:6789/0 conn(0x7fa4b015e830 legacy=0x7fa4b006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4aed9d640 1 -- 172.21.15.70:0/2254169173 >> v1:172.21.15.70:6789/0 conn(0x7fa4b015ec10 legacy=0x7fa4b015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.476+0000 7fa4aed9d640 1 -- 172.21.15.70:0/2254169173 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b006baa0 con 0x7fa4b00b1ff0 2023-08-25T09:45:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.477+0000 7fa4aed9d640 1 --2- 172.21.15.70:0/2254169173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b00b1ff0 0x7fa4b00b2440 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009e50 tx=0x7fa4a802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf7d37ff0e07206 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.477+0000 7fa4add9b640 1 -- 172.21.15.70:0/2254169173 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a802cb10 con 0x7fa4b00b1ff0 2023-08-25T09:45:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4add9b640 1 -- 172.21.15.70:0/2254169173 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a802cc70 con 0x7fa4b00b1ff0 2023-08-25T09:45:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4add9b640 1 -- 172.21.15.70:0/2254169173 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a80343f0 con 0x7fa4b00b1ff0 2023-08-25T09:45:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 -- 172.21.15.70:0/2254169173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b00b1ff0 msgr2=0x7fa4b00b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/2254169173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b00b1ff0 0x7fa4b00b2440 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009e50 tx=0x7fa4a802bf70 comp rx=0 tx=0).stop 2023-08-25T09:45:11.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 -- 172.21.15.70:0/2254169173 shutdown_connections 2023-08-25T09:45:11.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/2254169173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b00b1ff0 0x7fa4b00b2440 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 -- 172.21.15.70:0/2254169173 >> 172.21.15.70:0/2254169173 conn(0x7fa4b0069850 msgr2=0x7fa4b0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:11.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 -- 172.21.15.70:0/2254169173 shutdown_connections 2023-08-25T09:45:11.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.478+0000 7fa4b4d55640 1 -- 172.21.15.70:0/2254169173 wait complete. 2023-08-25T09:45:11.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.480+0000 7fa4b4d55640 1 Processor -- start 2023-08-25T09:45:11.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.480+0000 7fa4b4d55640 1 -- start start 2023-08-25T09:45:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4b4d55640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00b1ff0 0x7fa4b015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4b4d55640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b015e830 0x7fa4b01e5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4b4d55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 0x7fa4b01425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4b4d55640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01436b0 con 0x7fa4b015ec10 2023-08-25T09:45:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4b4d55640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01e6230 con 0x7fa4b00b1ff0 2023-08-25T09:45:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4b4d55640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01e5c70 con 0x7fa4b015e830 2023-08-25T09:45:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.481+0000 7fa4aed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 0x7fa4b01425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 0x7fa4b01425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52950/0 (socket says 172.21.15.70:52950) 2023-08-25T09:45:11.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 -- 172.21.15.70:0/3727099511 learned_addr learned my addr 172.21.15.70:0/3727099511 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:11.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 -- 172.21.15.70:0/3727099511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b015e830 msgr2=0x7fa4b01e5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b015e830 0x7fa4b01e5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 -- 172.21.15.70:0/3727099511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00b1ff0 msgr2=0x7fa4b015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00b1ff0 0x7fa4b015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 -- 172.21.15.70:0/3727099511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a8009b30 con 0x7fa4b015ec10 2023-08-25T09:45:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4aed9d640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 0x7fa4b01425a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8036040 tx=0x7fa4a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a803e450 con 0x7fa4b015ec10 2023-08-25T09:45:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a803e5d0 con 0x7fa4b015ec10 2023-08-25T09:45:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a803e980 con 0x7fa4b015ec10 2023-08-25T09:45:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.482+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b01e5f50 con 0x7fa4b015ec10 2023-08-25T09:45:11.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.483+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b0156a40 con 0x7fa4b015ec10 2023-08-25T09:45:11.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.484+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b0156bd0 con 0x7fa4b015ec10 2023-08-25T09:45:11.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.484+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa4a803eae0 con 0x7fa4b015ec10 2023-08-25T09:45:11.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.485+0000 7fa4a6ffd640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa49006a050 0x7fa49006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.485+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa4a80ab4f0 con 0x7fa4b015ec10 2023-08-25T09:45:11.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.485+0000 7fa4ae59c640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa49006a050 0x7fa49006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.486+0000 7fa4ae59c640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa49006a050 0x7fa49006c510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0003cc0 tx=0x7fa4a00079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.489+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa4a8076bd0 con 0x7fa4b015ec10 2023-08-25T09:45:11.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.672+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4b006b480 con 0x7fa49006a050 2023-08-25T09:45:11.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.696+0000 7fa4a6ffd640 1 -- 172.21.15.70:0/3727099511 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303470 (secure 0 0 0) 0x7fa4a0014400 con 0x7fa49006a050 2023-08-25T09:45:11.702 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:11.706 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:45:11.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.703+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa49006a050 msgr2=0x7fa49006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.703+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa49006a050 0x7fa49006c510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0003cc0 tx=0x7fa4a00079b0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.703+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 msgr2=0x7fa4b01425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.703+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 0x7fa4b01425a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8036040 tx=0x7fa4a8004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 shutdown_connections 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa49006a050 0x7fa49006c510 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4b015ec10 0x7fa4b01425a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4b015e830 0x7fa4b01e5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 --2- 172.21.15.70:0/3727099511 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4b00b1ff0 0x7fa4b015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 >> 172.21.15.70:0/3727099511 conn(0x7fa4b0069850 msgr2=0x7fa4b0069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:11.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 shutdown_connections 2023-08-25T09:45:11.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.704+0000 7fa4b4d55640 1 -- 172.21.15.70:0/3727099511 wait complete. 2023-08-25T09:45:11.726 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-08-25T09:45:11.446088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6395,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556104,"statfs":{"total":1600000000,"available":1593451276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001701"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457060+0000","last_change":"2023-08-25T09:45:07.018788+0000","last_active":"2023-08-25T09:45:08.457060+0000","last_peered":"2023-08-25T09:45:08.457060+0000","last_clean":"2023-08-25T09:45:08.457060+0000","last_became_active":"2023-08-25T09:45:07.018327+0000","last_became_peered":"2023-08-25T09:45:07.018327+0000","last_unstale":"2023-08-25T09:45:08.457060+0000","last_undegraded":"2023-08-25T09:45:08.457060+0000","last_fullsized":"2023-08-25T09:45:08.457060+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:40:01.723200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458171+0000","last_change":"2023-08-25T09:45:07.021377+0000","last_active":"2023-08-25T09:45:08.458171+0000","last_peered":"2023-08-25T09:45:08.458171+0000","last_clean":"2023-08-25T09:45:08.458171+0000","last_became_active":"2023-08-25T09:45:07.021097+0000","last_became_peered":"2023-08-25T09:45:07.021097+0000","last_unstale":"2023-08-25T09:45:08.458171+0000","last_undegraded":"2023-08-25T09:45:08.458171+0000","last_fullsized":"2023-08-25T09:45:08.458171+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:40:24.636956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998468+0000","last_change":"2023-08-25T09:45:07.016113+0000","last_active":"2023-08-25T09:45:07.998468+0000","last_peered":"2023-08-25T09:45:07.998468+0000","last_clean":"2023-08-25T09:45:07.998468+0000","last_became_active":"2023-08-25T09:45:07.014579+0000","last_became_peered":"2023-08-25T09:45:07.014579+0000","last_unstale":"2023-08-25T09:45:07.998468+0000","last_undegraded":"2023-08-25T09:45:07.998468+0000","last_fullsized":"2023-08-25T09:45:07.998468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:30:30.717613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999709+0000","last_change":"2023-08-25T09:45:07.003314+0000","last_active":"2023-08-25T09:45:07.999709+0000","last_peered":"2023-08-25T09:45:07.999709+0000","last_clean":"2023-08-25T09:45:07.999709+0000","last_became_active":"2023-08-25T09:45:07.002658+0000","last_became_peered":"2023-08-25T09:45:07.002658+0000","last_unstale":"2023-08-25T09:45:07.999709+0000","last_undegraded":"2023-08-25T09:45:07.999709+0000","last_fullsized":"2023-08-25T09:45:07.999709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:39:45.616304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457890+0000","last_change":"2023-08-25T09:45:07.009812+0000","last_active":"2023-08-25T09:45:08.457890+0000","last_peered":"2023-08-25T09:45:08.457890+0000","last_clean":"2023-08-25T09:45:08.457890+0000","last_became_active":"2023-08-25T09:45:07.009054+0000","last_became_peered":"2023-08-25T09:45:07.009054+0000","last_unstale":"2023-08-25T09:45:08.457890+0000","last_undegraded":"2023-08-25T09:45:08.457890+0000","last_fullsized":"2023-08-25T09:45:08.457890+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:34.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997998+0000","last_change":"2023-08-25T09:45:07.029036+0000","last_active":"2023-08-25T09:45:07.997998+0000","last_peered":"2023-08-25T09:45:07.997998+0000","last_clean":"2023-08-25T09:45:07.997998+0000","last_became_active":"2023-08-25T09:45:07.028340+0000","last_became_peered":"2023-08-25T09:45:07.028340+0000","last_unstale":"2023-08-25T09:45:07.997998+0000","last_undegraded":"2023-08-25T09:45:07.997998+0000","last_fullsized":"2023-08-25T09:45:07.997998+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:11:51.691747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455840+0000","last_change":"2023-08-25T09:45:07.002767+0000","last_active":"2023-08-25T09:45:08.455840+0000","last_peered":"2023-08-25T09:45:08.455840+0000","last_clean":"2023-08-25T09:45:08.455840+0000","last_became_active":"2023-08-25T09:45:07.002481+0000","last_became_peered":"2023-08-25T09:45:07.002481+0000","last_unstale":"2023-08-25T09:45:08.455840+0000","last_undegraded":"2023-08-25T09:45:08.455840+0000","last_fullsized":"2023-08-25T09:45:08.455840+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:58.975507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997590+0000","last_change":"2023-08-25T09:45:07.019170+0000","last_active":"2023-08-25T09:45:07.997590+0000","last_peered":"2023-08-25T09:45:07.997590+0000","last_clean":"2023-08-25T09:45:07.997590+0000","last_became_active":"2023-08-25T09:45:07.018672+0000","last_became_peered":"2023-08-25T09:45:07.018672+0000","last_unstale":"2023-08-25T09:45:07.997590+0000","last_undegraded":"2023-08-25T09:45:07.997590+0000","last_fullsized":"2023-08-25T09:45:07.997590+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:56.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455799+0000","last_change":"2023-08-25T09:45:07.014038+0000","last_active":"2023-08-25T09:45:08.455799+0000","last_peered":"2023-08-25T09:45:08.455799+0000","last_clean":"2023-08-25T09:45:08.455799+0000","last_became_active":"2023-08-25T09:45:07.013696+0000","last_became_peered":"2023-08-25T09:45:07.013696+0000","last_unstale":"2023-08-25T09:45:08.455799+0000","last_undegraded":"2023-08-25T09:45:08.455799+0000","last_fullsized":"2023-08-25T09:45:08.455799+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:22:20.943333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998464+0000","last_change":"2023-08-25T09:45:07.019850+0000","last_active":"2023-08-25T09:45:07.998464+0000","last_peered":"2023-08-25T09:45:07.998464+0000","last_clean":"2023-08-25T09:45:07.998464+0000","last_became_active":"2023-08-25T09:45:07.019357+0000","last_became_peered":"2023-08-25T09:45:07.019357+0000","last_unstale":"2023-08-25T09:45:07.998464+0000","last_undegraded":"2023-08-25T09:45:07.998464+0000","last_fullsized":"2023-08-25T09:45:07.998464+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:05:33.042533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456105+0000","last_change":"2023-08-25T09:45:07.002899+0000","last_active":"2023-08-25T09:45:08.456105+0000","last_peered":"2023-08-25T09:45:08.456105+0000","last_clean":"2023-08-25T09:45:08.456105+0000","last_became_active":"2023-08-25T09:45:07.002623+0000","last_became_peered":"2023-08-25T09:45:07.002623+0000","last_unstale":"2023-08-25T09:45:08.456105+0000","last_undegraded":"2023-08-25T09:45:08.456105+0000","last_fullsized":"2023-08-25T09:45:08.456105+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:45.338295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998932+0000","last_change":"2023-08-25T09:45:07.015896+0000","last_active":"2023-08-25T09:45:07.998932+0000","last_peered":"2023-08-25T09:45:07.998932+0000","last_clean":"2023-08-25T09:45:07.998932+0000","last_became_active":"2023-08-25T09:45:07.014480+0000","last_became_peered":"2023-08-25T09:45:07.014480+0000","last_unstale":"2023-08-25T09:45:07.998932+0000","last_undegraded":"2023-08-25T09:45:07.998932+0000","last_fullsized":"2023-08-25T09:45:07.998932+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:06:14.961119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456288+0000","last_change":"2023-08-25T09:45:07.012279+0000","last_active":"2023-08-25T09:45:08.456288+0000","last_peered":"2023-08-25T09:45:08.456288+0000","last_clean":"2023-08-25T09:45:08.456288+0000","last_became_active":"2023-08-25T09:45:07.010810+0000","last_became_peered":"2023-08-25T09:45:07.010810+0000","last_unstale":"2023-08-25T09:45:08.456288+0000","last_undegraded":"2023-08-25T09:45:08.456288+0000","last_fullsized":"2023-08-25T09:45:08.456288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:48.782532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.454597+0000","last_change":"2023-08-25T09:45:07.023031+0000","last_active":"2023-08-25T09:45:08.454597+0000","last_peered":"2023-08-25T09:45:08.454597+0000","last_clean":"2023-08-25T09:45:08.454597+0000","last_became_active":"2023-08-25T09:45:07.022767+0000","last_became_peered":"2023-08-25T09:45:07.022767+0000","last_unstale":"2023-08-25T09:45:08.454597+0000","last_undegraded":"2023-08-25T09:45:08.454597+0000","last_fullsized":"2023-08-25T09:45:08.454597+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:18:23.532566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998832+0000","last_change":"2023-08-25T09:45:07.018678+0000","last_active":"2023-08-25T09:45:07.998832+0000","last_peered":"2023-08-25T09:45:07.998832+0000","last_clean":"2023-08-25T09:45:07.998832+0000","last_became_active":"2023-08-25T09:45:07.018448+0000","last_became_peered":"2023-08-25T09:45:07.018448+0000","last_unstale":"2023-08-25T09:45:07.998832+0000","last_undegraded":"2023-08-25T09:45:07.998832+0000","last_fullsized":"2023-08-25T09:45:07.998832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:11:38.915413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997027+0000","last_change":"2023-08-25T09:45:07.029557+0000","last_active":"2023-08-25T09:45:07.997027+0000","last_peered":"2023-08-25T09:45:07.997027+0000","last_clean":"2023-08-25T09:45:07.997027+0000","last_became_active":"2023-08-25T09:45:07.029308+0000","last_became_peered":"2023-08-25T09:45:07.029308+0000","last_unstale":"2023-08-25T09:45:07.997027+0000","last_undegraded":"2023-08-25T09:45:07.997027+0000","last_fullsized":"2023-08-25T09:45:07.997027+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:14.571213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997010+0000","last_change":"2023-08-25T09:45:07.027976+0000","last_active":"2023-08-25T09:45:07.997010+0000","last_peered":"2023-08-25T09:45:07.997010+0000","last_clean":"2023-08-25T09:45:07.997010+0000","last_became_active":"2023-08-25T09:45:07.027378+0000","last_became_peered":"2023-08-25T09:45:07.027378+0000","last_unstale":"2023-08-25T09:45:07.997010+0000","last_undegraded":"2023-08-25T09:45:07.997010+0000","last_fullsized":"2023-08-25T09:45:07.997010+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:51:25.679286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998835+0000","last_change":"2023-08-25T09:45:07.016300+0000","last_active":"2023-08-25T09:45:07.998835+0000","last_peered":"2023-08-25T09:45:07.998835+0000","last_clean":"2023-08-25T09:45:07.998835+0000","last_became_active":"2023-08-25T09:45:07.015302+0000","last_became_peered":"2023-08-25T09:45:07.015302+0000","last_unstale":"2023-08-25T09:45:07.998835+0000","last_undegraded":"2023-08-25T09:45:07.998835+0000","last_fullsized":"2023-08-25T09:45:07.998835+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:10:58.357774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997335+0000","last_change":"2023-08-25T09:45:07.024094+0000","last_active":"2023-08-25T09:45:07.997335+0000","last_peered":"2023-08-25T09:45:07.997335+0000","last_clean":"2023-08-25T09:45:07.997335+0000","last_became_active":"2023-08-25T09:45:07.023831+0000","last_became_peered":"2023-08-25T09:45:07.023831+0000","last_unstale":"2023-08-25T09:45:07.997335+0000","last_undegraded":"2023-08-25T09:45:07.997335+0000","last_fullsized":"2023-08-25T09:45:07.997335+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:50.215800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997284+0000","last_change":"2023-08-25T09:45:07.026627+0000","last_active":"2023-08-25T09:45:07.997284+0000","last_peered":"2023-08-25T09:45:07.997284+0000","last_clean":"2023-08-25T09:45:07.997284+0000","last_became_active":"2023-08-25T09:45:07.026265+0000","last_became_peered":"2023-08-25T09:45:07.026265+0000","last_unstale":"2023-08-25T09:45:07.997284+0000","last_undegraded":"2023-08-25T09:45:07.997284+0000","last_fullsized":"2023-08-25T09:45:07.997284+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:31.523445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457288+0000","last_change":"2023-08-25T09:45:07.008397+0000","last_active":"2023-08-25T09:45:08.457288+0000","last_peered":"2023-08-25T09:45:08.457288+0000","last_clean":"2023-08-25T09:45:08.457288+0000","last_became_active":"2023-08-25T09:45:07.008149+0000","last_became_peered":"2023-08-25T09:45:07.008149+0000","last_unstale":"2023-08-25T09:45:08.457288+0000","last_undegraded":"2023-08-25T09:45:08.457288+0000","last_fullsized":"2023-08-25T09:45:08.457288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:25.169720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997906+0000","last_change":"2023-08-25T09:45:07.015630+0000","last_active":"2023-08-25T09:45:07.997906+0000","last_peered":"2023-08-25T09:45:07.997906+0000","last_clean":"2023-08-25T09:45:07.997906+0000","last_became_active":"2023-08-25T09:45:07.015196+0000","last_became_peered":"2023-08-25T09:45:07.015196+0000","last_unstale":"2023-08-25T09:45:07.997906+0000","last_undegraded":"2023-08-25T09:45:07.997906+0000","last_fullsized":"2023-08-25T09:45:07.997906+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:57:32.829430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457180+0000","last_change":"2023-08-25T09:45:07.015871+0000","last_active":"2023-08-25T09:45:08.457180+0000","last_peered":"2023-08-25T09:45:08.457180+0000","last_clean":"2023-08-25T09:45:08.457180+0000","last_became_active":"2023-08-25T09:45:07.014839+0000","last_became_peered":"2023-08-25T09:45:07.014839+0000","last_unstale":"2023-08-25T09:45:08.457180+0000","last_undegraded":"2023-08-25T09:45:08.457180+0000","last_fullsized":"2023-08-25T09:45:08.457180+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:31.523445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455575+0000","last_change":"2023-08-25T09:45:07.017579+0000","last_active":"2023-08-25T09:45:08.455575+0000","last_peered":"2023-08-25T09:45:08.455575+0000","last_clean":"2023-08-25T09:45:08.455575+0000","last_became_active":"2023-08-25T09:45:07.012913+0000","last_became_peered":"2023-08-25T09:45:07.012913+0000","last_unstale":"2023-08-25T09:45:08.455575+0000","last_undegraded":"2023-08-25T09:45:08.455575+0000","last_fullsized":"2023-08-25T09:45:08.455575+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:37.605603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458242+0000","last_change":"2023-08-25T09:45:07.009670+0000","last_active":"2023-08-25T09:45:08.458242+0000","last_peered":"2023-08-25T09:45:08.458242+0000","last_clean":"2023-08-25T09:45:08.458242+0000","last_became_active":"2023-08-25T09:45:07.008750+0000","last_became_peered":"2023-08-25T09:45:07.008750+0000","last_unstale":"2023-08-25T09:45:08.458242+0000","last_undegraded":"2023-08-25T09:45:08.458242+0000","last_fullsized":"2023-08-25T09:45:08.458242+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:31.335736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457638+0000","last_change":"2023-08-25T09:45:07.021311+0000","last_active":"2023-08-25T09:45:08.457638+0000","last_peered":"2023-08-25T09:45:08.457638+0000","last_clean":"2023-08-25T09:45:08.457638+0000","last_became_active":"2023-08-25T09:45:07.021066+0000","last_became_peered":"2023-08-25T09:45:07.021066+0000","last_unstale":"2023-08-25T09:45:08.457638+0000","last_undegraded":"2023-08-25T09:45:08.457638+0000","last_fullsized":"2023-08-25T09:45:08.457638+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:58.112859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999290+0000","last_change":"2023-08-25T09:45:07.015711+0000","last_active":"2023-08-25T09:45:07.999290+0000","last_peered":"2023-08-25T09:45:07.999290+0000","last_clean":"2023-08-25T09:45:07.999290+0000","last_became_active":"2023-08-25T09:45:07.012945+0000","last_became_peered":"2023-08-25T09:45:07.012945+0000","last_unstale":"2023-08-25T09:45:07.999290+0000","last_undegraded":"2023-08-25T09:45:07.999290+0000","last_fullsized":"2023-08-25T09:45:07.999290+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:56.582338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457405+0000","last_change":"2023-08-25T09:45:07.028772+0000","last_active":"2023-08-25T09:45:08.457405+0000","last_peered":"2023-08-25T09:45:08.457405+0000","last_clean":"2023-08-25T09:45:08.457405+0000","last_became_active":"2023-08-25T09:45:07.020336+0000","last_became_peered":"2023-08-25T09:45:07.020336+0000","last_unstale":"2023-08-25T09:45:08.457405+0000","last_undegraded":"2023-08-25T09:45:08.457405+0000","last_fullsized":"2023-08-25T09:45:08.457405+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:31:31.074453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456138+0000","last_change":"2023-08-25T09:45:07.011738+0000","last_active":"2023-08-25T09:45:08.456138+0000","last_peered":"2023-08-25T09:45:08.456138+0000","last_clean":"2023-08-25T09:45:08.456138+0000","last_became_active":"2023-08-25T09:45:07.011483+0000","last_became_peered":"2023-08-25T09:45:07.011483+0000","last_unstale":"2023-08-25T09:45:08.456138+0000","last_undegraded":"2023-08-25T09:45:08.456138+0000","last_fullsized":"2023-08-25T09:45:08.456138+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:37.713289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458581+0000","last_change":"2023-08-25T09:45:07.010864+0000","last_active":"2023-08-25T09:45:08.458581+0000","last_peered":"2023-08-25T09:45:08.458581+0000","last_clean":"2023-08-25T09:45:08.458581+0000","last_became_active":"2023-08-25T09:45:07.010456+0000","last_became_peered":"2023-08-25T09:45:07.010456+0000","last_unstale":"2023-08-25T09:45:08.458581+0000","last_undegraded":"2023-08-25T09:45:08.458581+0000","last_fullsized":"2023-08-25T09:45:08.458581+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:22:11.323869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997912+0000","last_change":"2023-08-25T09:45:07.014242+0000","last_active":"2023-08-25T09:45:07.997912+0000","last_peered":"2023-08-25T09:45:07.997912+0000","last_clean":"2023-08-25T09:45:07.997912+0000","last_became_active":"2023-08-25T09:45:07.013874+0000","last_became_peered":"2023-08-25T09:45:07.013874+0000","last_unstale":"2023-08-25T09:45:07.997912+0000","last_undegraded":"2023-08-25T09:45:07.997912+0000","last_fullsized":"2023-08-25T09:45:07.997912+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:19.523502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997380+0000","last_change":"2023-08-25T09:45:07.028140+0000","last_active":"2023-08-25T09:45:07.997380+0000","last_peered":"2023-08-25T09:45:07.997380+0000","last_clean":"2023-08-25T09:45:07.997380+0000","last_became_active":"2023-08-25T09:45:07.027737+0000","last_became_peered":"2023-08-25T09:45:07.027737+0000","last_unstale":"2023-08-25T09:45:07.997380+0000","last_undegraded":"2023-08-25T09:45:07.997380+0000","last_fullsized":"2023-08-25T09:45:07.997380+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:41.641452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457703+0000","last_change":"2023-08-25T09:45:07.032688+0000","last_active":"2023-08-25T09:45:08.457703+0000","last_peered":"2023-08-25T09:45:08.457703+0000","last_clean":"2023-08-25T09:45:08.457703+0000","last_became_active":"2023-08-25T09:45:07.030877+0000","last_became_peered":"2023-08-25T09:45:07.030877+0000","last_unstale":"2023-08-25T09:45:08.457703+0000","last_undegraded":"2023-08-25T09:45:08.457703+0000","last_fullsized":"2023-08-25T09:45:08.457703+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:37:24.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"114'4","reported_seq":17,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457498+0000","last_change":"2023-08-25T09:45:06.008832+0000","last_active":"2023-08-25T09:45:08.457498+0000","last_peered":"2023-08-25T09:45:08.457498+0000","last_clean":"2023-08-25T09:45:08.457498+0000","last_became_active":"2023-08-25T09:45:06.008561+0000","last_became_peered":"2023-08-25T09:45:06.008561+0000","last_unstale":"2023-08-25T09:45:08.457498+0000","last_undegraded":"2023-08-25T09:45:08.457498+0000","last_fullsized":"2023-08-25T09:45:08.457498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:55.247252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997661+0000","last_change":"2023-08-25T09:45:07.026484+0000","last_active":"2023-08-25T09:45:07.997661+0000","last_peered":"2023-08-25T09:45:07.997661+0000","last_clean":"2023-08-25T09:45:07.997661+0000","last_became_active":"2023-08-25T09:45:07.025954+0000","last_became_peered":"2023-08-25T09:45:07.025954+0000","last_unstale":"2023-08-25T09:45:07.997661+0000","last_undegraded":"2023-08-25T09:45:07.997661+0000","last_fullsized":"2023-08-25T09:45:07.997661+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:58.112859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.454999+0000","last_change":"2023-08-25T09:45:06.007006+0000","last_active":"2023-08-25T09:45:08.454999+0000","last_peered":"2023-08-25T09:45:08.454999+0000","last_clean":"2023-08-25T09:45:08.454999+0000","last_became_active":"2023-08-25T09:45:06.006496+0000","last_became_peered":"2023-08-25T09:45:06.006496+0000","last_unstale":"2023-08-25T09:45:08.454999+0000","last_undegraded":"2023-08-25T09:45:08.454999+0000","last_fullsized":"2023-08-25T09:45:08.454999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:17.054478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456121+0000","last_change":"2023-08-25T09:45:07.017615+0000","last_active":"2023-08-25T09:45:08.456121+0000","last_peered":"2023-08-25T09:45:08.456121+0000","last_clean":"2023-08-25T09:45:08.456121+0000","last_became_active":"2023-08-25T09:45:07.016721+0000","last_became_peered":"2023-08-25T09:45:07.016721+0000","last_unstale":"2023-08-25T09:45:08.456121+0000","last_undegraded":"2023-08-25T09:45:08.456121+0000","last_fullsized":"2023-08-25T09:45:08.456121+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:36.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459405+0000","last_change":"2023-08-25T09:45:06.014579+0000","last_active":"2023-08-25T09:45:08.459405+0000","last_peered":"2023-08-25T09:45:08.459405+0000","last_clean":"2023-08-25T09:45:08.459405+0000","last_became_active":"2023-08-25T09:45:06.013805+0000","last_became_peered":"2023-08-25T09:45:06.013805+0000","last_unstale":"2023-08-25T09:45:08.459405+0000","last_undegraded":"2023-08-25T09:45:08.459405+0000","last_fullsized":"2023-08-25T09:45:08.459405+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:36:47.741454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999218+0000","last_change":"2023-08-25T09:45:07.031808+0000","last_active":"2023-08-25T09:45:07.999218+0000","last_peered":"2023-08-25T09:45:07.999218+0000","last_clean":"2023-08-25T09:45:07.999218+0000","last_became_active":"2023-08-25T09:45:07.030388+0000","last_became_peered":"2023-08-25T09:45:07.030388+0000","last_unstale":"2023-08-25T09:45:07.999218+0000","last_undegraded":"2023-08-25T09:45:07.999218+0000","last_fullsized":"2023-08-25T09:45:07.999218+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:38:59.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455315+0000","last_change":"2023-08-25T09:45:06.015040+0000","last_active":"2023-08-25T09:45:08.455315+0000","last_peered":"2023-08-25T09:45:08.455315+0000","last_clean":"2023-08-25T09:45:08.455315+0000","last_became_active":"2023-08-25T09:45:06.014290+0000","last_became_peered":"2023-08-25T09:45:06.014290+0000","last_unstale":"2023-08-25T09:45:08.455315+0000","last_undegraded":"2023-08-25T09:45:08.455315+0000","last_fullsized":"2023-08-25T09:45:08.455315+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:29:47.318166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456623+0000","last_change":"2023-08-25T09:45:07.015641+0000","last_active":"2023-08-25T09:45:08.456623+0000","last_peered":"2023-08-25T09:45:08.456623+0000","last_clean":"2023-08-25T09:45:08.456623+0000","last_became_active":"2023-08-25T09:45:07.012560+0000","last_became_peered":"2023-08-25T09:45:07.012560+0000","last_unstale":"2023-08-25T09:45:08.456623+0000","last_undegraded":"2023-08-25T09:45:08.456623+0000","last_fullsized":"2023-08-25T09:45:08.456623+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:25.238443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998717+0000","last_change":"2023-08-25T09:45:06.005724+0000","last_active":"2023-08-25T09:45:07.998717+0000","last_peered":"2023-08-25T09:45:07.998717+0000","last_clean":"2023-08-25T09:45:07.998717+0000","last_became_active":"2023-08-25T09:45:06.004851+0000","last_became_peered":"2023-08-25T09:45:06.004851+0000","last_unstale":"2023-08-25T09:45:07.998717+0000","last_undegraded":"2023-08-25T09:45:07.998717+0000","last_fullsized":"2023-08-25T09:45:07.998717+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:09:24.138960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456493+0000","last_change":"2023-08-25T09:45:07.004212+0000","last_active":"2023-08-25T09:45:08.456493+0000","last_peered":"2023-08-25T09:45:08.456493+0000","last_clean":"2023-08-25T09:45:08.456493+0000","last_became_active":"2023-08-25T09:45:07.003746+0000","last_became_peered":"2023-08-25T09:45:07.003746+0000","last_unstale":"2023-08-25T09:45:08.456493+0000","last_undegraded":"2023-08-25T09:45:08.456493+0000","last_fullsized":"2023-08-25T09:45:08.456493+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:22:08.206567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457034+0000","last_change":"2023-08-25T09:45:06.012843+0000","last_active":"2023-08-25T09:45:08.457034+0000","last_peered":"2023-08-25T09:45:08.457034+0000","last_clean":"2023-08-25T09:45:08.457034+0000","last_became_active":"2023-08-25T09:45:06.012483+0000","last_became_peered":"2023-08-25T09:45:06.012483+0000","last_unstale":"2023-08-25T09:45:08.457034+0000","last_undegraded":"2023-08-25T09:45:08.457034+0000","last_fullsized":"2023-08-25T09:45:08.457034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:36.691777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997935+0000","last_change":"2023-08-25T09:45:07.027023+0000","last_active":"2023-08-25T09:45:07.997935+0000","last_peered":"2023-08-25T09:45:07.997935+0000","last_clean":"2023-08-25T09:45:07.997935+0000","last_became_active":"2023-08-25T09:45:07.026782+0000","last_became_peered":"2023-08-25T09:45:07.026782+0000","last_unstale":"2023-08-25T09:45:07.997935+0000","last_undegraded":"2023-08-25T09:45:07.997935+0000","last_fullsized":"2023-08-25T09:45:07.997935+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:05.257568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999933+0000","last_change":"2023-08-25T09:45:06.011016+0000","last_active":"2023-08-25T09:45:07.999933+0000","last_peered":"2023-08-25T09:45:07.999933+0000","last_clean":"2023-08-25T09:45:07.999933+0000","last_became_active":"2023-08-25T09:45:06.010625+0000","last_became_peered":"2023-08-25T09:45:06.010625+0000","last_unstale":"2023-08-25T09:45:07.999933+0000","last_undegraded":"2023-08-25T09:45:07.999933+0000","last_fullsized":"2023-08-25T09:45:07.999933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:16.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455589+0000","last_change":"2023-08-25T09:45:07.018964+0000","last_active":"2023-08-25T09:45:08.455589+0000","last_peered":"2023-08-25T09:45:08.455589+0000","last_clean":"2023-08-25T09:45:08.455589+0000","last_became_active":"2023-08-25T09:45:07.018539+0000","last_became_peered":"2023-08-25T09:45:07.018539+0000","last_unstale":"2023-08-25T09:45:08.455589+0000","last_undegraded":"2023-08-25T09:45:08.455589+0000","last_fullsized":"2023-08-25T09:45:08.455589+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:14:27.519003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459559+0000","last_change":"2023-08-25T09:45:06.014462+0000","last_active":"2023-08-25T09:45:08.459559+0000","last_peered":"2023-08-25T09:45:08.459559+0000","last_clean":"2023-08-25T09:45:08.459559+0000","last_became_active":"2023-08-25T09:45:06.013461+0000","last_became_peered":"2023-08-25T09:45:06.013461+0000","last_unstale":"2023-08-25T09:45:08.459559+0000","last_undegraded":"2023-08-25T09:45:08.459559+0000","last_fullsized":"2023-08-25T09:45:08.459559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:42:48.543047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456465+0000","last_change":"2023-08-25T09:45:07.017168+0000","last_active":"2023-08-25T09:45:08.456465+0000","last_peered":"2023-08-25T09:45:08.456465+0000","last_clean":"2023-08-25T09:45:08.456465+0000","last_became_active":"2023-08-25T09:45:07.016927+0000","last_became_peered":"2023-08-25T09:45:07.016927+0000","last_unstale":"2023-08-25T09:45:08.456465+0000","last_undegraded":"2023-08-25T09:45:08.456465+0000","last_fullsized":"2023-08-25T09:45:08.456465+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:31:31.074453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458200+0000","last_change":"2023-08-25T09:45:06.017079+0000","last_active":"2023-08-25T09:45:08.458200+0000","last_peered":"2023-08-25T09:45:08.458200+0000","last_clean":"2023-08-25T09:45:08.458200+0000","last_became_active":"2023-08-25T09:45:06.016691+0000","last_became_peered":"2023-08-25T09:45:06.016691+0000","last_unstale":"2023-08-25T09:45:08.458200+0000","last_undegraded":"2023-08-25T09:45:08.458200+0000","last_fullsized":"2023-08-25T09:45:08.458200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:10:02.255328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458231+0000","last_change":"2023-08-25T09:45:07.006905+0000","last_active":"2023-08-25T09:45:08.458231+0000","last_peered":"2023-08-25T09:45:08.458231+0000","last_clean":"2023-08-25T09:45:08.458231+0000","last_became_active":"2023-08-25T09:45:07.006654+0000","last_became_peered":"2023-08-25T09:45:07.006654+0000","last_unstale":"2023-08-25T09:45:08.458231+0000","last_undegraded":"2023-08-25T09:45:08.458231+0000","last_fullsized":"2023-08-25T09:45:08.458231+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:12:55.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.000024+0000","last_change":"2023-08-25T09:45:06.012825+0000","last_active":"2023-08-25T09:45:08.000024+0000","last_peered":"2023-08-25T09:45:08.000024+0000","last_clean":"2023-08-25T09:45:08.000024+0000","last_became_active":"2023-08-25T09:45:06.012494+0000","last_became_peered":"2023-08-25T09:45:06.012494+0000","last_unstale":"2023-08-25T09:45:08.000024+0000","last_undegraded":"2023-08-25T09:45:08.000024+0000","last_fullsized":"2023-08-25T09:45:08.000024+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:44.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455938+0000","last_change":"2023-08-25T09:45:07.029628+0000","last_active":"2023-08-25T09:45:08.455938+0000","last_peered":"2023-08-25T09:45:08.455938+0000","last_clean":"2023-08-25T09:45:08.455938+0000","last_became_active":"2023-08-25T09:45:07.029397+0000","last_became_peered":"2023-08-25T09:45:07.029397+0000","last_unstale":"2023-08-25T09:45:08.455938+0000","last_undegraded":"2023-08-25T09:45:08.455938+0000","last_fullsized":"2023-08-25T09:45:08.455938+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:26:52.553474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"114'3","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458933+0000","last_change":"2023-08-25T09:45:06.015147+0000","last_active":"2023-08-25T09:45:08.458933+0000","last_peered":"2023-08-25T09:45:08.458933+0000","last_clean":"2023-08-25T09:45:08.458933+0000","last_became_active":"2023-08-25T09:45:06.014444+0000","last_became_peered":"2023-08-25T09:45:06.014444+0000","last_unstale":"2023-08-25T09:45:08.458933+0000","last_undegraded":"2023-08-25T09:45:08.458933+0000","last_fullsized":"2023-08-25T09:45:08.458933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:24.480215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997351+0000","last_change":"2023-08-25T09:45:07.028837+0000","last_active":"2023-08-25T09:45:07.997351+0000","last_peered":"2023-08-25T09:45:07.997351+0000","last_clean":"2023-08-25T09:45:07.997351+0000","last_became_active":"2023-08-25T09:45:07.027502+0000","last_became_peered":"2023-08-25T09:45:07.027502+0000","last_unstale":"2023-08-25T09:45:07.997351+0000","last_undegraded":"2023-08-25T09:45:07.997351+0000","last_fullsized":"2023-08-25T09:45:07.997351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:25.169720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"114'6","reported_seq":19,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455939+0000","last_change":"2023-08-25T09:45:06.011490+0000","last_active":"2023-08-25T09:45:08.455939+0000","last_peered":"2023-08-25T09:45:08.455939+0000","last_clean":"2023-08-25T09:45:08.455939+0000","last_became_active":"2023-08-25T09:45:06.011205+0000","last_became_peered":"2023-08-25T09:45:06.011205+0000","last_unstale":"2023-08-25T09:45:08.455939+0000","last_undegraded":"2023-08-25T09:45:08.455939+0000","last_fullsized":"2023-08-25T09:45:08.455939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:01.175314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456459+0000","last_change":"2023-08-25T09:45:07.021129+0000","last_active":"2023-08-25T09:45:08.456459+0000","last_peered":"2023-08-25T09:45:08.456459+0000","last_clean":"2023-08-25T09:45:08.456459+0000","last_became_active":"2023-08-25T09:45:07.009281+0000","last_became_peered":"2023-08-25T09:45:07.009281+0000","last_unstale":"2023-08-25T09:45:08.456459+0000","last_undegraded":"2023-08-25T09:45:08.456459+0000","last_fullsized":"2023-08-25T09:45:08.456459+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:05:16.912498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459734+0000","last_change":"2023-08-25T09:45:06.014337+0000","last_active":"2023-08-25T09:45:08.459734+0000","last_peered":"2023-08-25T09:45:08.459734+0000","last_clean":"2023-08-25T09:45:08.459734+0000","last_became_active":"2023-08-25T09:45:06.013135+0000","last_became_peered":"2023-08-25T09:45:06.013135+0000","last_unstale":"2023-08-25T09:45:08.459734+0000","last_undegraded":"2023-08-25T09:45:08.459734+0000","last_fullsized":"2023-08-25T09:45:08.459734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:03.026776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457934+0000","last_change":"2023-08-25T09:45:07.013029+0000","last_active":"2023-08-25T09:45:08.457934+0000","last_peered":"2023-08-25T09:45:08.457934+0000","last_clean":"2023-08-25T09:45:08.457934+0000","last_became_active":"2023-08-25T09:45:07.012794+0000","last_became_peered":"2023-08-25T09:45:07.012794+0000","last_unstale":"2023-08-25T09:45:08.457934+0000","last_undegraded":"2023-08-25T09:45:08.457934+0000","last_fullsized":"2023-08-25T09:45:08.457934+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:05.257568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457891+0000","last_change":"2023-08-25T09:45:06.010519+0000","last_active":"2023-08-25T09:45:08.457891+0000","last_peered":"2023-08-25T09:45:08.457891+0000","last_clean":"2023-08-25T09:45:08.457891+0000","last_became_active":"2023-08-25T09:45:06.010124+0000","last_became_peered":"2023-08-25T09:45:06.010124+0000","last_unstale":"2023-08-25T09:45:08.457891+0000","last_undegraded":"2023-08-25T09:45:08.457891+0000","last_fullsized":"2023-08-25T09:45:08.457891+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:58:46.096650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457402+0000","last_change":"2023-08-25T09:45:07.010841+0000","last_active":"2023-08-25T09:45:08.457402+0000","last_peered":"2023-08-25T09:45:08.457402+0000","last_clean":"2023-08-25T09:45:08.457402+0000","last_became_active":"2023-08-25T09:45:07.009239+0000","last_became_peered":"2023-08-25T09:45:07.009239+0000","last_unstale":"2023-08-25T09:45:08.457402+0000","last_undegraded":"2023-08-25T09:45:08.457402+0000","last_fullsized":"2023-08-25T09:45:08.457402+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:06:18.447160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456260+0000","last_change":"2023-08-25T09:45:06.016643+0000","last_active":"2023-08-25T09:45:08.456260+0000","last_peered":"2023-08-25T09:45:08.456260+0000","last_clean":"2023-08-25T09:45:08.456260+0000","last_became_active":"2023-08-25T09:45:06.016388+0000","last_became_peered":"2023-08-25T09:45:06.016388+0000","last_unstale":"2023-08-25T09:45:08.456260+0000","last_undegraded":"2023-08-25T09:45:08.456260+0000","last_fullsized":"2023-08-25T09:45:08.456260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:31:15.463974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457000+0000","last_change":"2023-08-25T09:45:07.017011+0000","last_active":"2023-08-25T09:45:08.457000+0000","last_peered":"2023-08-25T09:45:08.457000+0000","last_clean":"2023-08-25T09:45:08.457000+0000","last_became_active":"2023-08-25T09:45:07.013095+0000","last_became_peered":"2023-08-25T09:45:07.013095+0000","last_unstale":"2023-08-25T09:45:08.457000+0000","last_undegraded":"2023-08-25T09:45:08.457000+0000","last_fullsized":"2023-08-25T09:45:08.457000+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:37:44.533417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"114'8","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999913+0000","last_change":"2023-08-25T09:45:06.013865+0000","last_active":"2023-08-25T09:45:07.999913+0000","last_peered":"2023-08-25T09:45:07.999913+0000","last_clean":"2023-08-25T09:45:07.999913+0000","last_became_active":"2023-08-25T09:45:06.012819+0000","last_became_peered":"2023-08-25T09:45:06.012819+0000","last_unstale":"2023-08-25T09:45:07.999913+0000","last_undegraded":"2023-08-25T09:45:07.999913+0000","last_fullsized":"2023-08-25T09:45:07.999913+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:04:22.417962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997333+0000","last_change":"2023-08-25T09:45:07.012196+0000","last_active":"2023-08-25T09:45:07.997333+0000","last_peered":"2023-08-25T09:45:07.997333+0000","last_clean":"2023-08-25T09:45:07.997333+0000","last_became_active":"2023-08-25T09:45:07.011966+0000","last_became_peered":"2023-08-25T09:45:07.011966+0000","last_unstale":"2023-08-25T09:45:07.997333+0000","last_undegraded":"2023-08-25T09:45:07.997333+0000","last_fullsized":"2023-08-25T09:45:07.997333+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:12:26.608084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456157+0000","last_change":"2023-08-25T09:45:06.010923+0000","last_active":"2023-08-25T09:45:08.456157+0000","last_peered":"2023-08-25T09:45:08.456157+0000","last_clean":"2023-08-25T09:45:08.456157+0000","last_became_active":"2023-08-25T09:45:06.009176+0000","last_became_peered":"2023-08-25T09:45:06.009176+0000","last_unstale":"2023-08-25T09:45:08.456157+0000","last_undegraded":"2023-08-25T09:45:08.456157+0000","last_fullsized":"2023-08-25T09:45:08.456157+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:54:20.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455578+0000","last_change":"2023-08-25T09:45:07.017770+0000","last_active":"2023-08-25T09:45:08.455578+0000","last_peered":"2023-08-25T09:45:08.455578+0000","last_clean":"2023-08-25T09:45:08.455578+0000","last_became_active":"2023-08-25T09:45:07.017283+0000","last_became_peered":"2023-08-25T09:45:07.017283+0000","last_unstale":"2023-08-25T09:45:08.455578+0000","last_undegraded":"2023-08-25T09:45:08.455578+0000","last_fullsized":"2023-08-25T09:45:08.455578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:22:37.842328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459253+0000","last_change":"2023-08-25T09:45:06.014822+0000","last_active":"2023-08-25T09:45:08.459253+0000","last_peered":"2023-08-25T09:45:08.459253+0000","last_clean":"2023-08-25T09:45:08.459253+0000","last_became_active":"2023-08-25T09:45:06.011281+0000","last_became_peered":"2023-08-25T09:45:06.011281+0000","last_unstale":"2023-08-25T09:45:08.459253+0000","last_undegraded":"2023-08-25T09:45:08.459253+0000","last_fullsized":"2023-08-25T09:45:08.459253+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:36:44.768678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456638+0000","last_change":"2023-08-25T09:45:07.030669+0000","last_active":"2023-08-25T09:45:08.456638+0000","last_peered":"2023-08-25T09:45:08.456638+0000","last_clean":"2023-08-25T09:45:08.456638+0000","last_became_active":"2023-08-25T09:45:07.020707+0000","last_became_peered":"2023-08-25T09:45:07.020707+0000","last_unstale":"2023-08-25T09:45:08.456638+0000","last_undegraded":"2023-08-25T09:45:08.456638+0000","last_fullsized":"2023-08-25T09:45:08.456638+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:13.015807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997728+0000","last_change":"2023-08-25T09:45:06.013441+0000","last_active":"2023-08-25T09:45:07.997728+0000","last_peered":"2023-08-25T09:45:07.997728+0000","last_clean":"2023-08-25T09:45:07.997728+0000","last_became_active":"2023-08-25T09:45:06.012972+0000","last_became_peered":"2023-08-25T09:45:06.012972+0000","last_unstale":"2023-08-25T09:45:07.997728+0000","last_undegraded":"2023-08-25T09:45:07.997728+0000","last_fullsized":"2023-08-25T09:45:07.997728+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:45:51.015883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458336+0000","last_change":"2023-08-25T09:45:07.011038+0000","last_active":"2023-08-25T09:45:08.458336+0000","last_peered":"2023-08-25T09:45:08.458336+0000","last_clean":"2023-08-25T09:45:08.458336+0000","last_became_active":"2023-08-25T09:45:07.010757+0000","last_became_peered":"2023-08-25T09:45:07.010757+0000","last_unstale":"2023-08-25T09:45:08.458336+0000","last_undegraded":"2023-08-25T09:45:08.458336+0000","last_fullsized":"2023-08-25T09:45:08.458336+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:59:42.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457222+0000","last_change":"2023-08-25T09:45:06.016203+0000","last_active":"2023-08-25T09:45:08.457222+0000","last_peered":"2023-08-25T09:45:08.457222+0000","last_clean":"2023-08-25T09:45:08.457222+0000","last_became_active":"2023-08-25T09:45:06.015680+0000","last_became_peered":"2023-08-25T09:45:06.015680+0000","last_unstale":"2023-08-25T09:45:08.457222+0000","last_undegraded":"2023-08-25T09:45:08.457222+0000","last_fullsized":"2023-08-25T09:45:08.457222+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:03:35.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.996369+0000","last_change":"2023-08-25T09:45:07.018321+0000","last_active":"2023-08-25T09:45:07.996369+0000","last_peered":"2023-08-25T09:45:07.996369+0000","last_clean":"2023-08-25T09:45:07.996369+0000","last_became_active":"2023-08-25T09:45:07.018012+0000","last_became_peered":"2023-08-25T09:45:07.018012+0000","last_unstale":"2023-08-25T09:45:07.996369+0000","last_undegraded":"2023-08-25T09:45:07.996369+0000","last_fullsized":"2023-08-25T09:45:07.996369+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:37.713289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458047+0000","last_change":"2023-08-25T09:45:06.015000+0000","last_active":"2023-08-25T09:45:08.458047+0000","last_peered":"2023-08-25T09:45:08.458047+0000","last_clean":"2023-08-25T09:45:08.458047+0000","last_became_active":"2023-08-25T09:45:06.014711+0000","last_became_peered":"2023-08-25T09:45:06.014711+0000","last_unstale":"2023-08-25T09:45:08.458047+0000","last_undegraded":"2023-08-25T09:45:08.458047+0000","last_fullsized":"2023-08-25T09:45:08.458047+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:38:41.808705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999549+0000","last_change":"2023-08-25T09:45:07.031211+0000","last_active":"2023-08-25T09:45:07.999549+0000","last_peered":"2023-08-25T09:45:07.999549+0000","last_clean":"2023-08-25T09:45:07.999549+0000","last_became_active":"2023-08-25T09:45:07.029888+0000","last_became_peered":"2023-08-25T09:45:07.029888+0000","last_unstale":"2023-08-25T09:45:07.999549+0000","last_undegraded":"2023-08-25T09:45:07.999549+0000","last_fullsized":"2023-08-25T09:45:07.999549+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:03.991566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458555+0000","last_change":"2023-08-25T09:45:06.012784+0000","last_active":"2023-08-25T09:45:08.458555+0000","last_peered":"2023-08-25T09:45:08.458555+0000","last_clean":"2023-08-25T09:45:08.458555+0000","last_became_active":"2023-08-25T09:45:06.009560+0000","last_became_peered":"2023-08-25T09:45:06.009560+0000","last_unstale":"2023-08-25T09:45:08.458555+0000","last_undegraded":"2023-08-25T09:45:08.458555+0000","last_fullsized":"2023-08-25T09:45:08.458555+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:09:13.825828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998217+0000","last_change":"2023-08-25T09:45:07.019869+0000","last_active":"2023-08-25T09:45:07.998217+0000","last_peered":"2023-08-25T09:45:07.998217+0000","last_clean":"2023-08-25T09:45:07.998217+0000","last_became_active":"2023-08-25T09:45:07.019473+0000","last_became_peered":"2023-08-25T09:45:07.019473+0000","last_unstale":"2023-08-25T09:45:07.998217+0000","last_undegraded":"2023-08-25T09:45:07.998217+0000","last_fullsized":"2023-08-25T09:45:07.998217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:06:06.316438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457980+0000","last_change":"2023-08-25T09:45:06.007418+0000","last_active":"2023-08-25T09:45:08.457980+0000","last_peered":"2023-08-25T09:45:08.457980+0000","last_clean":"2023-08-25T09:45:08.457980+0000","last_became_active":"2023-08-25T09:45:06.007192+0000","last_became_peered":"2023-08-25T09:45:06.007192+0000","last_unstale":"2023-08-25T09:45:08.457980+0000","last_undegraded":"2023-08-25T09:45:08.457980+0000","last_fullsized":"2023-08-25T09:45:08.457980+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:11:50.670235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998297+0000","last_change":"2023-08-25T09:45:07.025834+0000","last_active":"2023-08-25T09:45:07.998297+0000","last_peered":"2023-08-25T09:45:07.998297+0000","last_clean":"2023-08-25T09:45:07.998297+0000","last_became_active":"2023-08-25T09:45:07.025294+0000","last_became_peered":"2023-08-25T09:45:07.025294+0000","last_unstale":"2023-08-25T09:45:07.998297+0000","last_undegraded":"2023-08-25T09:45:07.998297+0000","last_fullsized":"2023-08-25T09:45:07.998297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:40:24.636956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456778+0000","last_change":"2023-08-25T09:45:06.008788+0000","last_active":"2023-08-25T09:45:08.456778+0000","last_peered":"2023-08-25T09:45:08.456778+0000","last_clean":"2023-08-25T09:45:08.456778+0000","last_became_active":"2023-08-25T09:45:06.008551+0000","last_became_peered":"2023-08-25T09:45:06.008551+0000","last_unstale":"2023-08-25T09:45:08.456778+0000","last_undegraded":"2023-08-25T09:45:08.456778+0000","last_fullsized":"2023-08-25T09:45:08.456778+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:51:24.657773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457078+0000","last_change":"2023-08-25T09:45:07.015935+0000","last_active":"2023-08-25T09:45:08.457078+0000","last_peered":"2023-08-25T09:45:08.457078+0000","last_clean":"2023-08-25T09:45:08.457078+0000","last_became_active":"2023-08-25T09:45:07.015655+0000","last_became_peered":"2023-08-25T09:45:07.015655+0000","last_unstale":"2023-08-25T09:45:08.457078+0000","last_undegraded":"2023-08-25T09:45:08.457078+0000","last_fullsized":"2023-08-25T09:45:08.457078+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:14.571213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456596+0000","last_change":"2023-08-25T09:45:06.011461+0000","last_active":"2023-08-25T09:45:08.456596+0000","last_peered":"2023-08-25T09:45:08.456596+0000","last_clean":"2023-08-25T09:45:08.456596+0000","last_became_active":"2023-08-25T09:45:06.011148+0000","last_became_peered":"2023-08-25T09:45:06.011148+0000","last_unstale":"2023-08-25T09:45:08.456596+0000","last_undegraded":"2023-08-25T09:45:08.456596+0000","last_fullsized":"2023-08-25T09:45:08.456596+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:22.443464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":330,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457294+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:45:08.457294+0000","last_peered":"2023-08-25T09:45:08.457294+0000","last_clean":"2023-08-25T09:45:08.457294+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:45:08.457294+0000","last_undegraded":"2023-08-25T09:45:08.457294+0000","last_fullsized":"2023-08-25T09:45:08.457294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:15.605498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456698+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:45:08.456698+0000","last_peered":"2023-08-25T09:45:08.456698+0000","last_clean":"2023-08-25T09:45:08.456698+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:45:08.456698+0000","last_undegraded":"2023-08-25T09:45:08.456698+0000","last_fullsized":"2023-08-25T09:45:08.456698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:30.563884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455743+0000","last_change":"2023-08-25T09:45:07.016220+0000","last_active":"2023-08-25T09:45:08.455743+0000","last_peered":"2023-08-25T09:45:08.455743+0000","last_clean":"2023-08-25T09:45:08.455743+0000","last_became_active":"2023-08-25T09:45:07.015601+0000","last_became_peered":"2023-08-25T09:45:07.015601+0000","last_unstale":"2023-08-25T09:45:08.455743+0000","last_undegraded":"2023-08-25T09:45:08.455743+0000","last_fullsized":"2023-08-25T09:45:08.455743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:44:07.745761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997611+0000","last_change":"2023-08-25T09:45:06.006735+0000","last_active":"2023-08-25T09:45:07.997611+0000","last_peered":"2023-08-25T09:45:07.997611+0000","last_clean":"2023-08-25T09:45:07.997611+0000","last_became_active":"2023-08-25T09:45:06.006166+0000","last_became_peered":"2023-08-25T09:45:06.006166+0000","last_unstale":"2023-08-25T09:45:07.997611+0000","last_undegraded":"2023-08-25T09:45:07.997611+0000","last_fullsized":"2023-08-25T09:45:07.997611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:53.981249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457478+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:45:08.457478+0000","last_peered":"2023-08-25T09:45:08.457478+0000","last_clean":"2023-08-25T09:45:08.457478+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:45:08.457478+0000","last_undegraded":"2023-08-25T09:45:08.457478+0000","last_fullsized":"2023-08-25T09:45:08.457478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:37.642813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456459+0000","last_change":"2023-08-25T09:45:07.012508+0000","last_active":"2023-08-25T09:45:08.456459+0000","last_peered":"2023-08-25T09:45:08.456459+0000","last_clean":"2023-08-25T09:45:08.456459+0000","last_became_active":"2023-08-25T09:45:07.012059+0000","last_became_peered":"2023-08-25T09:45:07.012059+0000","last_unstale":"2023-08-25T09:45:08.456459+0000","last_undegraded":"2023-08-25T09:45:08.456459+0000","last_fullsized":"2023-08-25T09:45:08.456459+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:44:24.456723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455769+0000","last_change":"2023-08-25T09:45:06.016416+0000","last_active":"2023-08-25T09:45:08.455769+0000","last_peered":"2023-08-25T09:45:08.455769+0000","last_clean":"2023-08-25T09:45:08.455769+0000","last_became_active":"2023-08-25T09:45:06.016184+0000","last_became_peered":"2023-08-25T09:45:06.016184+0000","last_unstale":"2023-08-25T09:45:08.455769+0000","last_undegraded":"2023-08-25T09:45:08.455769+0000","last_fullsized":"2023-08-25T09:45:08.455769+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:54.539445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":204,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.000122+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:45:08.000122+0000","last_peered":"2023-08-25T09:45:08.000122+0000","last_clean":"2023-08-25T09:45:08.000122+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:45:08.000122+0000","last_undegraded":"2023-08-25T09:45:08.000122+0000","last_fullsized":"2023-08-25T09:45:08.000122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:22:22.645926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458933+0000","last_change":"2023-08-25T09:45:07.010712+0000","last_active":"2023-08-25T09:45:08.458933+0000","last_peered":"2023-08-25T09:45:08.458933+0000","last_clean":"2023-08-25T09:45:08.458933+0000","last_became_active":"2023-08-25T09:45:07.010065+0000","last_became_peered":"2023-08-25T09:45:07.010065+0000","last_unstale":"2023-08-25T09:45:08.458933+0000","last_undegraded":"2023-08-25T09:45:08.458933+0000","last_fullsized":"2023-08-25T09:45:08.458933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:18.759455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455638+0000","last_change":"2023-08-25T09:45:06.015186+0000","last_active":"2023-08-25T09:45:08.455638+0000","last_peered":"2023-08-25T09:45:08.455638+0000","last_clean":"2023-08-25T09:45:08.455638+0000","last_became_active":"2023-08-25T09:45:06.014799+0000","last_became_peered":"2023-08-25T09:45:06.014799+0000","last_unstale":"2023-08-25T09:45:08.455638+0000","last_undegraded":"2023-08-25T09:45:08.455638+0000","last_fullsized":"2023-08-25T09:45:08.455638+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:40.638351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456639+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:45:08.456639+0000","last_peered":"2023-08-25T09:45:08.456639+0000","last_clean":"2023-08-25T09:45:08.456639+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:45:08.456639+0000","last_undegraded":"2023-08-25T09:45:08.456639+0000","last_fullsized":"2023-08-25T09:45:08.456639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:21:47.636025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997674+0000","last_change":"2023-08-25T09:45:07.026005+0000","last_active":"2023-08-25T09:45:07.997674+0000","last_peered":"2023-08-25T09:45:07.997674+0000","last_clean":"2023-08-25T09:45:07.997674+0000","last_became_active":"2023-08-25T09:45:07.025556+0000","last_became_peered":"2023-08-25T09:45:07.025556+0000","last_unstale":"2023-08-25T09:45:07.997674+0000","last_undegraded":"2023-08-25T09:45:07.997674+0000","last_fullsized":"2023-08-25T09:45:07.997674+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:34.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458569+0000","last_change":"2023-08-25T09:45:06.014987+0000","last_active":"2023-08-25T09:45:08.458569+0000","last_peered":"2023-08-25T09:45:08.458569+0000","last_clean":"2023-08-25T09:45:08.458569+0000","last_became_active":"2023-08-25T09:45:06.013951+0000","last_became_peered":"2023-08-25T09:45:06.013951+0000","last_unstale":"2023-08-25T09:45:08.458569+0000","last_undegraded":"2023-08-25T09:45:08.458569+0000","last_fullsized":"2023-08-25T09:45:08.458569+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:30.852980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":178,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457386+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:45:08.457386+0000","last_peered":"2023-08-25T09:45:08.457386+0000","last_clean":"2023-08-25T09:45:08.457386+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:45:08.457386+0000","last_undegraded":"2023-08-25T09:45:08.457386+0000","last_fullsized":"2023-08-25T09:45:08.457386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:05.697579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455394+0000","last_change":"2023-08-25T09:45:07.020753+0000","last_active":"2023-08-25T09:45:08.455394+0000","last_peered":"2023-08-25T09:45:08.455394+0000","last_clean":"2023-08-25T09:45:08.455394+0000","last_became_active":"2023-08-25T09:45:07.020515+0000","last_became_peered":"2023-08-25T09:45:07.020515+0000","last_unstale":"2023-08-25T09:45:08.455394+0000","last_undegraded":"2023-08-25T09:45:08.455394+0000","last_fullsized":"2023-08-25T09:45:08.455394+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:46.095770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.996707+0000","last_change":"2023-08-25T09:45:06.013140+0000","last_active":"2023-08-25T09:45:07.996707+0000","last_peered":"2023-08-25T09:45:07.996707+0000","last_clean":"2023-08-25T09:45:07.996707+0000","last_became_active":"2023-08-25T09:45:06.012902+0000","last_became_peered":"2023-08-25T09:45:06.012902+0000","last_unstale":"2023-08-25T09:45:07.996707+0000","last_undegraded":"2023-08-25T09:45:07.996707+0000","last_fullsized":"2023-08-25T09:45:07.996707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:31:30.052940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457039+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:45:08.457039+0000","last_peered":"2023-08-25T09:45:08.457039+0000","last_clean":"2023-08-25T09:45:08.457039+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:45:08.457039+0000","last_undegraded":"2023-08-25T09:45:08.457039+0000","last_fullsized":"2023-08-25T09:45:08.457039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:08.303316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997478+0000","last_change":"2023-08-25T09:45:07.023970+0000","last_active":"2023-08-25T09:45:07.997478+0000","last_peered":"2023-08-25T09:45:07.997478+0000","last_clean":"2023-08-25T09:45:07.997478+0000","last_became_active":"2023-08-25T09:45:07.023533+0000","last_became_peered":"2023-08-25T09:45:07.023533+0000","last_unstale":"2023-08-25T09:45:07.997478+0000","last_undegraded":"2023-08-25T09:45:07.997478+0000","last_fullsized":"2023-08-25T09:45:07.997478+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:04.196878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456076+0000","last_change":"2023-08-25T09:45:06.004354+0000","last_active":"2023-08-25T09:45:08.456076+0000","last_peered":"2023-08-25T09:45:08.456076+0000","last_clean":"2023-08-25T09:45:08.456076+0000","last_became_active":"2023-08-25T09:45:06.003654+0000","last_became_peered":"2023-08-25T09:45:06.003654+0000","last_unstale":"2023-08-25T09:45:08.456076+0000","last_undegraded":"2023-08-25T09:45:08.456076+0000","last_fullsized":"2023-08-25T09:45:08.456076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:49:21.205415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":173,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998435+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:45:07.998435+0000","last_peered":"2023-08-25T09:45:07.998435+0000","last_clean":"2023-08-25T09:45:07.998435+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:45:07.998435+0000","last_undegraded":"2023-08-25T09:45:07.998435+0000","last_fullsized":"2023-08-25T09:45:07.998435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:43.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456824+0000","last_change":"2023-08-25T09:45:07.024647+0000","last_active":"2023-08-25T09:45:08.456824+0000","last_peered":"2023-08-25T09:45:08.456824+0000","last_clean":"2023-08-25T09:45:08.456824+0000","last_became_active":"2023-08-25T09:45:07.024203+0000","last_became_peered":"2023-08-25T09:45:07.024203+0000","last_unstale":"2023-08-25T09:45:08.456824+0000","last_undegraded":"2023-08-25T09:45:08.456824+0000","last_fullsized":"2023-08-25T09:45:08.456824+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:03:02.715234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456152+0000","last_change":"2023-08-25T09:45:06.011087+0000","last_active":"2023-08-25T09:45:08.456152+0000","last_peered":"2023-08-25T09:45:08.456152+0000","last_clean":"2023-08-25T09:45:08.456152+0000","last_became_active":"2023-08-25T09:45:06.010455+0000","last_became_peered":"2023-08-25T09:45:06.010455+0000","last_unstale":"2023-08-25T09:45:08.456152+0000","last_undegraded":"2023-08-25T09:45:08.456152+0000","last_fullsized":"2023-08-25T09:45:08.456152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:15.251374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456352+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:45:08.456352+0000","last_peered":"2023-08-25T09:45:08.456352+0000","last_clean":"2023-08-25T09:45:08.456352+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:45:08.456352+0000","last_undegraded":"2023-08-25T09:45:08.456352+0000","last_fullsized":"2023-08-25T09:45:08.456352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:58.356461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.206,"15min":1.071},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":10.747,"15min":10.747},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1319999999999999,"15min":1.044},"min":{"1min":0.76800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.133},"min":{"1min":0.81200000000000006,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.198,"15min":1.0980000000000001},"min":{"1min":0.66500000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.649,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.173,"15min":1.085},"min":{"1min":0.83499999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5249999999999999,"5min":1.619,"15min":1.619},"last":1.097},{"interface":"front","average":{"1min":1.391,"5min":1.3100000000000001,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2370000000000001,"15min":1.1659999999999999},"min":{"1min":0.78500000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6699999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.403,"15min":1.335},"min":{"1min":1.083,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.89200000000000002}]},{"osd":4,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.325,"15min":1.274},"min":{"1min":0.85099999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7869999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3340000000000001,"15min":1.2909999999999999},"min":{"1min":0.61399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6579999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.4530000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4690000000000001,"15min":1.3500000000000001},"min":{"1min":0.95599999999999996,"5min":0.67700000000000005,"15min":0.52700000000000002},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":0.92500000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.909,"5min":2.0179999999999998,"15min":2.0739999999999998},"last":1.1719999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.5489999999999999,"15min":1.446},"min":{"1min":1.23,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9610000000000001,"5min":2.0859999999999999,"15min":2.1720000000000002},"last":1.335},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.498,"15min":1.407},"min":{"1min":0.753,"5min":0.71199999999999997,"15min":0.61399999999999999},"max":{"1min":1.8620000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.522}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":656,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194656,"statfs":{"total":200000000,"available":199327868,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1499999999999999,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.597,"5min":1.804,"15min":1.804},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.202,"15min":1.149},"min":{"1min":0.71699999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.885,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.0940000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2350000000000001,"15min":1.1970000000000001},"min":{"1min":0.73099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":1.014},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.222,"15min":1.1299999999999999},"min":{"1min":0.79300000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.641}]},{"osd":2,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1830000000000001,"15min":1.111},"min":{"1min":0.78200000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.573,"5min":1.708,"15min":1.708},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.409,"15min":1.22},"min":{"1min":1.0429999999999999,"5min":0.73299999999999998,"15min":0.66200000000000003},"max":{"1min":1.7410000000000001,"5min":13.193,"15min":13.193},"last":1.1739999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3380000000000001,"15min":1.234},"min":{"1min":0.85499999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8240000000000001,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.2769999999999999,"15min":1.2569999999999999},"min":{"1min":0.94099999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6619999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.379}]},{"osd":4,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3480000000000001,"15min":1.254},"min":{"1min":0.80400000000000005,"5min":0.68500000000000005,"15min":0.55300000000000005},"max":{"1min":1.881,"5min":2.484,"15min":2.484},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.411,"15min":1.3260000000000001},"min":{"1min":0.97599999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.016,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.335,"15min":1.282},"min":{"1min":0.997,"5min":0.624,"15min":0.59599999999999997},"max":{"1min":1.742,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.054},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.506,"15min":1.381},"min":{"1min":1.0980000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.978,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.556}]},{"osd":7,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.5589999999999999,"15min":1.4970000000000001},"min":{"1min":1.169,"5min":0.78200000000000003,"15min":0.622},"max":{"1min":1.9590000000000001,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.29},{"interface":"front","average":{"1min":1.526,"5min":1.484,"15min":1.476},"min":{"1min":1.159,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4510000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1231,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194081,"statfs":{"total":200000000,"available":198739236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1950000000000001,"15min":1.1479999999999999},"min":{"1min":0.624,"5min":0.497,"15min":0.497},"max":{"1min":1.738,"5min":2.081,"15min":2.081},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2170000000000001,"15min":1.2070000000000001},"min":{"1min":0.754,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.694,"5min":1.974,"15min":1.974},"last":1.365}]},{"osd":1,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1299999999999999,"15min":1.1899999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.851,"5min":1.851,"15min":2.1779999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.194,"15min":1.123},"min":{"1min":0.90500000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8640000000000001,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.321}]},{"osd":2,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.25,"15min":1.1899999999999999},"min":{"1min":0.92500000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6850000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.484},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.258,"15min":1.2210000000000001},"min":{"1min":0.97799999999999998,"5min":0.79300000000000004,"15min":0.72599999999999998},"max":{"1min":1.778,"5min":1.8049999999999999,"15min":1.962},"last":1.6890000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.395,"15min":1.3120000000000001},"min":{"1min":0.85499999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6080000000000001,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.337,"15min":1.2889999999999999},"min":{"1min":1.159,"5min":0.747,"15min":0.73499999999999999},"max":{"1min":1.7549999999999999,"5min":1.7949999999999999,"15min":1.9650000000000001},"last":1.7769999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.88,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9530000000000001,"5min":2.625,"15min":2.625},"last":1.157},{"interface":"front","average":{"1min":1.423,"5min":1.355,"15min":1.375},"min":{"1min":1.036,"5min":0.72299999999999998,"15min":0.58799999999999997},"max":{"1min":1.7609999999999999,"5min":2.0270000000000001,"15min":2.2789999999999999},"last":0.998}]},{"osd":6,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4550000000000001,"15min":1.4370000000000001},"min":{"1min":1.0069999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8180000000000001,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.5429999999999999,"15min":1.427},"min":{"1min":1.2090000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.671,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4850000000000001,"15min":1.4470000000000001},"min":{"1min":1.1819999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.819,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.103},{"interface":"front","average":{"1min":1.635,"5min":1.599,"15min":1.583},"min":{"1min":1.4490000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.913,"5min":2.0790000000000002,"15min":2.3700000000000001},"last":1.899}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.214,"15min":1.1619999999999999},"min":{"1min":0.96199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.569,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3200000000000001,"15min":1.1659999999999999},"min":{"1min":0.95399999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.976,"5min":7.367,"15min":7.367},"last":1.1870000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"max":{"1min":1.5549999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.554},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.383},"min":{"1min":0.82899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.744,"5min":1.9379999999999999,"15min":8.5850000000000009},"last":1.355}]},{"osd":2,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.292},"min":{"1min":0.80400000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":4.1669999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.274,"15min":1.177},"min":{"1min":0.85799999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.649,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.92600000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2949999999999999,"15min":1.2310000000000001},"min":{"1min":0.91700000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5880000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.8160000000000001,"5min":1.911,"15min":1.911},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2729999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7390000000000001,"5min":2.0640000000000001,"15min":2.4420000000000002},"last":1.488},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.357,"15min":1.258},"min":{"1min":0.94899999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.8049999999999999,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.605}]},{"osd":6,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4239999999999999,"15min":1.4359999999999999},"min":{"1min":1.026,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8560000000000001,"5min":2.0219999999999998,"15min":2.863},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.524,"5min":1.5389999999999999,"15min":1.5569999999999999},"min":{"1min":1.0780000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.913,"5min":2.1339999999999999,"15min":4.0229999999999997},"last":1.764}]},{"osd":7,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5780000000000001,"15min":1.427},"min":{"1min":0.98499999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0649999999999999,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.597,"15min":1.5960000000000001},"min":{"1min":1.2849999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8680000000000001,"5min":2.1230000000000002,"15min":2.7930000000000001},"last":1.9319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":656,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194656,"statfs":{"total":200000000,"available":199327880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.89700000000000002,"15min":0.85299999999999998},"min":{"1min":0.54100000000000004,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.6830000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2869999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.745},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.022,"15min":0.98299999999999998},"min":{"1min":0.54000000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.421,"5min":1.722,"15min":2.3399999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0009999999999999,"15min":0.88300000000000001},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.6839999999999999,"5min":1.919,"15min":1.919},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.21,"5min":1.1220000000000001,"15min":1.1140000000000001},"min":{"1min":0.61199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.708,"5min":1.9710000000000001,"15min":3.399},"last":1.302}]},{"osd":4,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2370000000000001,"15min":1.1970000000000001},"min":{"1min":0.79400000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.238,"15min":1.2010000000000001},"min":{"1min":0.85199999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6060000000000001,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.86099999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.327,"15min":1.3129999999999999},"min":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.077,"5min":2.077,"15min":2.4060000000000001},"last":1.159},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.306,"15min":1.276},"min":{"1min":0.88,"5min":0.81499999999999995,"15min":0.64500000000000002},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.115}]},{"osd":6,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.302,"15min":1.248},"min":{"1min":0.91000000000000003,"5min":0.74099999999999999,"15min":0.73799999999999999},"max":{"1min":1.6890000000000001,"5min":1.917,"15min":1.917},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.407,"15min":1.409},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":3.496},"last":1.2490000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.4159999999999999,"15min":1.3700000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.899,"5min":2.044,"15min":2.2989999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4299999999999999,"15min":1.306},"min":{"1min":1.1200000000000001,"5min":0.88100000000000001,"15min":0.68400000000000005},"max":{"1min":2.0179999999999998,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.83599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.85399999999999998,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.395,"5min":1.7210000000000001,"15min":2.383},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.92500000000000004,"15min":1.0600000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":1.4430000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.98299999999999998,"15min":0.99099999999999999},"min":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5029999999999999,"5min":1.9350000000000001,"15min":2.3180000000000001},"last":1.228},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.0580000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.6399999999999999,"5min":1.677,"15min":2.8599999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94799999999999995,"15min":1.0840000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.72,"5min":2.1190000000000002,"15min":4.9569999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.497,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.145,"5min":2.145,"15min":2.2040000000000002},"last":1.056}]},{"osd":4,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.2989999999999999,"15min":1.3939999999999999},"min":{"1min":0.83599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":1.508},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.72199999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.7639999999999998},"last":1.115}]},{"osd":5,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.236,"15min":1.365},"min":{"1min":0.76100000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6439999999999999,"5min":2.004,"15min":5.1269999999999998},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.373,"15min":1.3979999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.395}]},{"osd":6,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.379,"15min":1.419},"min":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9059999999999999,"5min":2.0800000000000001,"15min":2.6709999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.379,"15min":1.4790000000000001},"min":{"1min":1.014,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.2389999999999999,"5min":2.6749999999999998,"15min":5.0599999999999996},"last":1.28}]},{"osd":7,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3720000000000001,"15min":1.4450000000000001},"min":{"1min":0.74199999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.181,"5min":2.181,"15min":2.968},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.4119999999999999,"15min":1.502},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2309999999999999,"5min":2.7829999999999999,"15min":5.2990000000000004},"last":1.3620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1231,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194080,"statfs":{"total":200000000,"available":198738550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83599999999999997,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":1.484,"5min":1.5700000000000001,"15min":1.627},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":0.79500000000000004},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"max":{"1min":1.331,"5min":1.895,"15min":1.895},"last":1.198}]},{"osd":2,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89300000000000002,"15min":0.80800000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0009999999999999,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.41899999999999998,"15min":0.33600000000000002},"max":{"1min":1.6220000000000001,"5min":1.746,"15min":1.746},"last":1.075}]},{"osd":3,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.161,"15min":1.0469999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.372},"max":{"1min":1.819,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.55800000000000005,"5min":0.48799999999999999,"15min":0.45000000000000001},"max":{"1min":1.5249999999999999,"5min":1.764,"15min":1.764},"last":1.5229999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.177,"15min":1.0940000000000001},"min":{"1min":0.78200000000000003,"5min":0.75,"15min":0.72499999999999998},"max":{"1min":1.4530000000000001,"5min":1.694,"15min":1.694},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.333,"15min":1.2250000000000001},"min":{"1min":0.77700000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.7210000000000001,"5min":2.008,"15min":2.008},"last":1.482}]},{"osd":5,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.256,"15min":1.194},"min":{"1min":0.751,"5min":0.751,"15min":0.68600000000000005},"max":{"1min":1.7589999999999999,"5min":1.823,"15min":1.8340000000000001},"last":1.458},{"interface":"front","average":{"1min":1.286,"5min":1.2450000000000001,"15min":1.177},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.4299999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.419,"15min":1.3109999999999999},"min":{"1min":0.88200000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.008,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.3799999999999999,"15min":1.3009999999999999},"min":{"1min":1.024,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8700000000000001,"5min":2.004,"15min":2.004},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4430000000000001,"15min":1.359},"min":{"1min":0.85499999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.968,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4259999999999999,"15min":1.3200000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.716}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.85999999999999999},"min":{"1min":0.50900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":1.9350000000000001,"5min":11.257,"15min":11.257},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.97899999999999998,"15min":0.96099999999999997},"min":{"1min":0.47399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5580000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.80300000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.63500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.98099999999999998,"15min":1.087},"min":{"1min":0.59999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.89400000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.625},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.073,"15min":1.117},"min":{"1min":0.57499999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.5860000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.238},"min":{"1min":0.72899999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3420000000000001,"15min":1.3560000000000001},"min":{"1min":0.95199999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.659}]},{"osd":5,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2350000000000001,"15min":1.351},"min":{"1min":0.754,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.798},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.875,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.3049999999999999,"15min":1.3440000000000001},"min":{"1min":0.81999999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.278},{"interface":"front","average":{"1min":1.359,"5min":1.2969999999999999,"15min":1.2989999999999999},"min":{"1min":0.79800000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":1.698}]},{"osd":7,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.379,"15min":1.401},"min":{"1min":0.91500000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3120000000000001,"15min":1.323},"min":{"1min":0.81599999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.7410000000000001,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":2.1469999999999998}]}]}],"pool_statfs":[]}} 2023-08-25T09:45:11.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:45:11.956 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-08-25T09:45:11.953+0000 7ff80cdb3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:45:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.985+0000 7fad05fc4640 1 Processor -- start 2023-08-25T09:45:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.985+0000 7fad05fc4640 1 -- start start 2023-08-25T09:45:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.986+0000 7fad05fc4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fad0015a7c0 0x7fad000b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.986+0000 7fad05fc4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad0015e7c0 con 0x7fad000cad20 2023-08-25T09:45:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.986+0000 7fad05fc4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad00071b50 con 0x7fad000b12f0 2023-08-25T09:45:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.986+0000 7fad05fc4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0015e490 con 0x7fad0015a7c0 2023-08-25T09:45:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.987+0000 7fad057c3640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fad000cad20 0x7fad000c8af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37658/0 (socket says 172.21.15.70:37658) 2023-08-25T09:45:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.987+0000 7fad057c3640 1 -- 172.21.15.70:0/821693119 learned_addr learned my addr 172.21.15.70:0/821693119 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.987+0000 7fad04fc2640 1 --2- 172.21.15.70:0/821693119 >> v2:172.21.15.155:3300/0 conn(0x7fad0015a7c0 0x7fad000b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.987+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409830717 0 0) 0x7fad0015e7c0 con 0x7fad000cad20 2023-08-25T09:45:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.987+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facf0003520 con 0x7fad000cad20 2023-08-25T09:45:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2535835649 0 0) 0x7facf0003520 con 0x7fad000cad20 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 >> v2:172.21.15.155:3300/0 conn(0x7fad0015a7c0 msgr2=0x7fad000b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 --2- 172.21.15.70:0/821693119 >> v2:172.21.15.155:3300/0 conn(0x7fad0015a7c0 0x7fad000b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 >> v1:172.21.15.155:6789/0 conn(0x7fad000b12f0 legacy=0x7fad0015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000b3aa0 con 0x7fad000cad20 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7facf80019f0 con 0x7fad000cad20 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 >> v1:172.21.15.70:6789/0 conn(0x7fad000cad20 legacy=0x7fad000c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0003970 0x7facf0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf0005180 0x7facf000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0015e490 con 0x7facf00047c0 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf000c020 con 0x7facf0003970 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf000c1e0 con 0x7facf0005180 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facfffff640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facfffff640 1 -- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf0005180 msgr2=0x7facf000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facfffff640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7facf0005180 0x7facf000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7fad04fc2640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0003970 0x7facf0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.988+0000 7facfffff640 1 -- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0003970 msgr2=0x7facf0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.989+0000 7facfffff640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0003970 0x7facf0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.989+0000 7facfffff640 1 -- 172.21.15.70:0/821693119 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf0003520 con 0x7facf00047c0 2023-08-25T09:45:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.989+0000 7fad057c3640 1 -- 172.21.15.70:0/821693119 reap_dead start 2023-08-25T09:45:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.989+0000 7fad04fc2640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf0003970 0x7facf0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:11.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.990+0000 7facfffff640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facf00047c0 0x7facf0004c40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7facec001750 tx=0x7facf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948801fa885625cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.990+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec0052d0 con 0x7facf00047c0 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.990+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec02e050 con 0x7facf00047c0 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.990+0000 7facff7fe640 1 -- 172.21.15.70:0/821693119 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec003830 con 0x7facf00047c0 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.990+0000 7fad05fc4640 1 -- 172.21.15.70:0/821693119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facf00047c0 msgr2=0x7facf0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.990+0000 7fad05fc4640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facf00047c0 0x7facf0004c40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7facec001750 tx=0x7facf00057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.991+0000 7fad05fc4640 1 -- 172.21.15.70:0/821693119 shutdown_connections 2023-08-25T09:45:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.991+0000 7fad05fc4640 1 --2- 172.21.15.70:0/821693119 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7facf00047c0 0x7facf0004c40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.991+0000 7fad05fc4640 1 -- 172.21.15.70:0/821693119 >> 172.21.15.70:0/821693119 conn(0x7fad000affd0 msgr2=0x7fad000b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.991+0000 7fad05fc4640 1 -- 172.21.15.70:0/821693119 shutdown_connections 2023-08-25T09:45:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.991+0000 7fad05fc4640 1 -- 172.21.15.70:0/821693119 wait complete. 2023-08-25T09:45:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.993+0000 7fad05fc4640 1 Processor -- start 2023-08-25T09:45:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.993+0000 7fad05fc4640 1 -- start start 2023-08-25T09:45:11.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad05fc4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf00047c0 0x7fad000c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad05fc4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 0x7fad001549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad05fc4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad000ce410 0x7fad000ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad05fc4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000c6760 con 0x7fad001585e0 2023-08-25T09:45:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad05fc4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad000cedd0 con 0x7facf00047c0 2023-08-25T09:45:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad05fc4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad000c39e0 con 0x7fad000ce410 2023-08-25T09:45:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 0x7fad001549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 0x7fad001549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52970/0 (socket says 172.21.15.70:52970) 2023-08-25T09:45:12.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 -- 172.21.15.70:0/369005257 learned_addr learned my addr 172.21.15.70:0/369005257 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:12.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad04fc2640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf00047c0 0x7fad000c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad057c3640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad000ce410 0x7fad000ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 -- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad000ce410 msgr2=0x7fad000ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad000ce410 0x7fad000ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 -- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf00047c0 msgr2=0x7fad000c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf00047c0 0x7fad000c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7facfffff640 1 -- 172.21.15.70:0/369005257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000c6480 con 0x7fad001585e0 2023-08-25T09:45:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.994+0000 7fad04fc2640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf00047c0 0x7fad000c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.995+0000 7facfffff640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 0x7fad001549f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7facec005040 tx=0x7facec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.995+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec007a30 con 0x7fad001585e0 2023-08-25T09:45:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.995+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facec0069e0 con 0x7fad001585e0 2023-08-25T09:45:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.995+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facec006d70 con 0x7fad001585e0 2023-08-25T09:45:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.995+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad000c3390 con 0x7fad001585e0 2023-08-25T09:45:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.995+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad00155460 con 0x7fad001585e0 2023-08-25T09:45:12.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.996+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad000c31f0 con 0x7fad001585e0 2023-08-25T09:45:12.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.997+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7facec0035f0 con 0x7fad001585e0 2023-08-25T09:45:12.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.999+0000 7facfd7fa640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7face406a050 0x7face406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.999+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7facec0aa690 con 0x7fad001585e0 2023-08-25T09:45:12.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:11.999+0000 7fad04fc2640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7face406a050 0x7face406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.000+0000 7fad04fc2640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7face406a050 0x7face406c510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fad001f5920 tx=0x7facf4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:12.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.003+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fad000c31f0 con 0x7fad001585e0 2023-08-25T09:45:12.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.192+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad000b4040 con 0x7face406a050 2023-08-25T09:45:12.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.211+0000 7facfd7fa640 1 -- 172.21.15.70:0/369005257 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303470 (secure 0 0 0) 0x7facf40118f0 con 0x7face406a050 2023-08-25T09:45:12.215 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:12.218 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:45:12.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7face406a050 msgr2=0x7face406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7face406a050 0x7face406c510 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fad001f5920 tx=0x7facf4008040 comp rx=0 tx=0).stop 2023-08-25T09:45:12.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 msgr2=0x7fad001549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 0x7fad001549f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7facec005040 tx=0x7facec001120 comp rx=0 tx=0).stop 2023-08-25T09:45:12.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 shutdown_connections 2023-08-25T09:45:12.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7face406a050 0x7face406c510 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad000ce410 0x7fad000ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad001585e0 0x7fad001549f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 --2- 172.21.15.70:0/369005257 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7facf00047c0 0x7fad000c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 >> 172.21.15.70:0/369005257 conn(0x7fad000affd0 msgr2=0x7fad000af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:12.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 shutdown_connections 2023-08-25T09:45:12.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.218+0000 7fad05fc4640 1 -- 172.21.15.70:0/369005257 wait complete. 2023-08-25T09:45:12.241 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-08-25T09:45:11.446088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6395,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556104,"statfs":{"total":1600000000,"available":1593451276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001701"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457060+0000","last_change":"2023-08-25T09:45:07.018788+0000","last_active":"2023-08-25T09:45:08.457060+0000","last_peered":"2023-08-25T09:45:08.457060+0000","last_clean":"2023-08-25T09:45:08.457060+0000","last_became_active":"2023-08-25T09:45:07.018327+0000","last_became_peered":"2023-08-25T09:45:07.018327+0000","last_unstale":"2023-08-25T09:45:08.457060+0000","last_undegraded":"2023-08-25T09:45:08.457060+0000","last_fullsized":"2023-08-25T09:45:08.457060+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:40:01.723200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458171+0000","last_change":"2023-08-25T09:45:07.021377+0000","last_active":"2023-08-25T09:45:08.458171+0000","last_peered":"2023-08-25T09:45:08.458171+0000","last_clean":"2023-08-25T09:45:08.458171+0000","last_became_active":"2023-08-25T09:45:07.021097+0000","last_became_peered":"2023-08-25T09:45:07.021097+0000","last_unstale":"2023-08-25T09:45:08.458171+0000","last_undegraded":"2023-08-25T09:45:08.458171+0000","last_fullsized":"2023-08-25T09:45:08.458171+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:40:24.636956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998468+0000","last_change":"2023-08-25T09:45:07.016113+0000","last_active":"2023-08-25T09:45:07.998468+0000","last_peered":"2023-08-25T09:45:07.998468+0000","last_clean":"2023-08-25T09:45:07.998468+0000","last_became_active":"2023-08-25T09:45:07.014579+0000","last_became_peered":"2023-08-25T09:45:07.014579+0000","last_unstale":"2023-08-25T09:45:07.998468+0000","last_undegraded":"2023-08-25T09:45:07.998468+0000","last_fullsized":"2023-08-25T09:45:07.998468+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:30:30.717613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999709+0000","last_change":"2023-08-25T09:45:07.003314+0000","last_active":"2023-08-25T09:45:07.999709+0000","last_peered":"2023-08-25T09:45:07.999709+0000","last_clean":"2023-08-25T09:45:07.999709+0000","last_became_active":"2023-08-25T09:45:07.002658+0000","last_became_peered":"2023-08-25T09:45:07.002658+0000","last_unstale":"2023-08-25T09:45:07.999709+0000","last_undegraded":"2023-08-25T09:45:07.999709+0000","last_fullsized":"2023-08-25T09:45:07.999709+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:39:45.616304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457890+0000","last_change":"2023-08-25T09:45:07.009812+0000","last_active":"2023-08-25T09:45:08.457890+0000","last_peered":"2023-08-25T09:45:08.457890+0000","last_clean":"2023-08-25T09:45:08.457890+0000","last_became_active":"2023-08-25T09:45:07.009054+0000","last_became_peered":"2023-08-25T09:45:07.009054+0000","last_unstale":"2023-08-25T09:45:08.457890+0000","last_undegraded":"2023-08-25T09:45:08.457890+0000","last_fullsized":"2023-08-25T09:45:08.457890+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:34.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997998+0000","last_change":"2023-08-25T09:45:07.029036+0000","last_active":"2023-08-25T09:45:07.997998+0000","last_peered":"2023-08-25T09:45:07.997998+0000","last_clean":"2023-08-25T09:45:07.997998+0000","last_became_active":"2023-08-25T09:45:07.028340+0000","last_became_peered":"2023-08-25T09:45:07.028340+0000","last_unstale":"2023-08-25T09:45:07.997998+0000","last_undegraded":"2023-08-25T09:45:07.997998+0000","last_fullsized":"2023-08-25T09:45:07.997998+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:11:51.691747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455840+0000","last_change":"2023-08-25T09:45:07.002767+0000","last_active":"2023-08-25T09:45:08.455840+0000","last_peered":"2023-08-25T09:45:08.455840+0000","last_clean":"2023-08-25T09:45:08.455840+0000","last_became_active":"2023-08-25T09:45:07.002481+0000","last_became_peered":"2023-08-25T09:45:07.002481+0000","last_unstale":"2023-08-25T09:45:08.455840+0000","last_undegraded":"2023-08-25T09:45:08.455840+0000","last_fullsized":"2023-08-25T09:45:08.455840+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:44:58.975507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997590+0000","last_change":"2023-08-25T09:45:07.019170+0000","last_active":"2023-08-25T09:45:07.997590+0000","last_peered":"2023-08-25T09:45:07.997590+0000","last_clean":"2023-08-25T09:45:07.997590+0000","last_became_active":"2023-08-25T09:45:07.018672+0000","last_became_peered":"2023-08-25T09:45:07.018672+0000","last_unstale":"2023-08-25T09:45:07.997590+0000","last_undegraded":"2023-08-25T09:45:07.997590+0000","last_fullsized":"2023-08-25T09:45:07.997590+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:56.268764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455799+0000","last_change":"2023-08-25T09:45:07.014038+0000","last_active":"2023-08-25T09:45:08.455799+0000","last_peered":"2023-08-25T09:45:08.455799+0000","last_clean":"2023-08-25T09:45:08.455799+0000","last_became_active":"2023-08-25T09:45:07.013696+0000","last_became_peered":"2023-08-25T09:45:07.013696+0000","last_unstale":"2023-08-25T09:45:08.455799+0000","last_undegraded":"2023-08-25T09:45:08.455799+0000","last_fullsized":"2023-08-25T09:45:08.455799+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:22:20.943333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998464+0000","last_change":"2023-08-25T09:45:07.019850+0000","last_active":"2023-08-25T09:45:07.998464+0000","last_peered":"2023-08-25T09:45:07.998464+0000","last_clean":"2023-08-25T09:45:07.998464+0000","last_became_active":"2023-08-25T09:45:07.019357+0000","last_became_peered":"2023-08-25T09:45:07.019357+0000","last_unstale":"2023-08-25T09:45:07.998464+0000","last_undegraded":"2023-08-25T09:45:07.998464+0000","last_fullsized":"2023-08-25T09:45:07.998464+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:05:33.042533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456105+0000","last_change":"2023-08-25T09:45:07.002899+0000","last_active":"2023-08-25T09:45:08.456105+0000","last_peered":"2023-08-25T09:45:08.456105+0000","last_clean":"2023-08-25T09:45:08.456105+0000","last_became_active":"2023-08-25T09:45:07.002623+0000","last_became_peered":"2023-08-25T09:45:07.002623+0000","last_unstale":"2023-08-25T09:45:08.456105+0000","last_undegraded":"2023-08-25T09:45:08.456105+0000","last_fullsized":"2023-08-25T09:45:08.456105+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:35:45.338295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998932+0000","last_change":"2023-08-25T09:45:07.015896+0000","last_active":"2023-08-25T09:45:07.998932+0000","last_peered":"2023-08-25T09:45:07.998932+0000","last_clean":"2023-08-25T09:45:07.998932+0000","last_became_active":"2023-08-25T09:45:07.014480+0000","last_became_peered":"2023-08-25T09:45:07.014480+0000","last_unstale":"2023-08-25T09:45:07.998932+0000","last_undegraded":"2023-08-25T09:45:07.998932+0000","last_fullsized":"2023-08-25T09:45:07.998932+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:06:14.961119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456288+0000","last_change":"2023-08-25T09:45:07.012279+0000","last_active":"2023-08-25T09:45:08.456288+0000","last_peered":"2023-08-25T09:45:08.456288+0000","last_clean":"2023-08-25T09:45:08.456288+0000","last_became_active":"2023-08-25T09:45:07.010810+0000","last_became_peered":"2023-08-25T09:45:07.010810+0000","last_unstale":"2023-08-25T09:45:08.456288+0000","last_undegraded":"2023-08-25T09:45:08.456288+0000","last_fullsized":"2023-08-25T09:45:08.456288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:48.782532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.454597+0000","last_change":"2023-08-25T09:45:07.023031+0000","last_active":"2023-08-25T09:45:08.454597+0000","last_peered":"2023-08-25T09:45:08.454597+0000","last_clean":"2023-08-25T09:45:08.454597+0000","last_became_active":"2023-08-25T09:45:07.022767+0000","last_became_peered":"2023-08-25T09:45:07.022767+0000","last_unstale":"2023-08-25T09:45:08.454597+0000","last_undegraded":"2023-08-25T09:45:08.454597+0000","last_fullsized":"2023-08-25T09:45:08.454597+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:18:23.532566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998832+0000","last_change":"2023-08-25T09:45:07.018678+0000","last_active":"2023-08-25T09:45:07.998832+0000","last_peered":"2023-08-25T09:45:07.998832+0000","last_clean":"2023-08-25T09:45:07.998832+0000","last_became_active":"2023-08-25T09:45:07.018448+0000","last_became_peered":"2023-08-25T09:45:07.018448+0000","last_unstale":"2023-08-25T09:45:07.998832+0000","last_undegraded":"2023-08-25T09:45:07.998832+0000","last_fullsized":"2023-08-25T09:45:07.998832+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:11:38.915413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997027+0000","last_change":"2023-08-25T09:45:07.029557+0000","last_active":"2023-08-25T09:45:07.997027+0000","last_peered":"2023-08-25T09:45:07.997027+0000","last_clean":"2023-08-25T09:45:07.997027+0000","last_became_active":"2023-08-25T09:45:07.029308+0000","last_became_peered":"2023-08-25T09:45:07.029308+0000","last_unstale":"2023-08-25T09:45:07.997027+0000","last_undegraded":"2023-08-25T09:45:07.997027+0000","last_fullsized":"2023-08-25T09:45:07.997027+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:14.571213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997010+0000","last_change":"2023-08-25T09:45:07.027976+0000","last_active":"2023-08-25T09:45:07.997010+0000","last_peered":"2023-08-25T09:45:07.997010+0000","last_clean":"2023-08-25T09:45:07.997010+0000","last_became_active":"2023-08-25T09:45:07.027378+0000","last_became_peered":"2023-08-25T09:45:07.027378+0000","last_unstale":"2023-08-25T09:45:07.997010+0000","last_undegraded":"2023-08-25T09:45:07.997010+0000","last_fullsized":"2023-08-25T09:45:07.997010+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:51:25.679286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998835+0000","last_change":"2023-08-25T09:45:07.016300+0000","last_active":"2023-08-25T09:45:07.998835+0000","last_peered":"2023-08-25T09:45:07.998835+0000","last_clean":"2023-08-25T09:45:07.998835+0000","last_became_active":"2023-08-25T09:45:07.015302+0000","last_became_peered":"2023-08-25T09:45:07.015302+0000","last_unstale":"2023-08-25T09:45:07.998835+0000","last_undegraded":"2023-08-25T09:45:07.998835+0000","last_fullsized":"2023-08-25T09:45:07.998835+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:10:58.357774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997335+0000","last_change":"2023-08-25T09:45:07.024094+0000","last_active":"2023-08-25T09:45:07.997335+0000","last_peered":"2023-08-25T09:45:07.997335+0000","last_clean":"2023-08-25T09:45:07.997335+0000","last_became_active":"2023-08-25T09:45:07.023831+0000","last_became_peered":"2023-08-25T09:45:07.023831+0000","last_unstale":"2023-08-25T09:45:07.997335+0000","last_undegraded":"2023-08-25T09:45:07.997335+0000","last_fullsized":"2023-08-25T09:45:07.997335+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:50.215800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997284+0000","last_change":"2023-08-25T09:45:07.026627+0000","last_active":"2023-08-25T09:45:07.997284+0000","last_peered":"2023-08-25T09:45:07.997284+0000","last_clean":"2023-08-25T09:45:07.997284+0000","last_became_active":"2023-08-25T09:45:07.026265+0000","last_became_peered":"2023-08-25T09:45:07.026265+0000","last_unstale":"2023-08-25T09:45:07.997284+0000","last_undegraded":"2023-08-25T09:45:07.997284+0000","last_fullsized":"2023-08-25T09:45:07.997284+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:31.523445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457288+0000","last_change":"2023-08-25T09:45:07.008397+0000","last_active":"2023-08-25T09:45:08.457288+0000","last_peered":"2023-08-25T09:45:08.457288+0000","last_clean":"2023-08-25T09:45:08.457288+0000","last_became_active":"2023-08-25T09:45:07.008149+0000","last_became_peered":"2023-08-25T09:45:07.008149+0000","last_unstale":"2023-08-25T09:45:08.457288+0000","last_undegraded":"2023-08-25T09:45:08.457288+0000","last_fullsized":"2023-08-25T09:45:08.457288+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:25.169720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997906+0000","last_change":"2023-08-25T09:45:07.015630+0000","last_active":"2023-08-25T09:45:07.997906+0000","last_peered":"2023-08-25T09:45:07.997906+0000","last_clean":"2023-08-25T09:45:07.997906+0000","last_became_active":"2023-08-25T09:45:07.015196+0000","last_became_peered":"2023-08-25T09:45:07.015196+0000","last_unstale":"2023-08-25T09:45:07.997906+0000","last_undegraded":"2023-08-25T09:45:07.997906+0000","last_fullsized":"2023-08-25T09:45:07.997906+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:57:32.829430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457180+0000","last_change":"2023-08-25T09:45:07.015871+0000","last_active":"2023-08-25T09:45:08.457180+0000","last_peered":"2023-08-25T09:45:08.457180+0000","last_clean":"2023-08-25T09:45:08.457180+0000","last_became_active":"2023-08-25T09:45:07.014839+0000","last_became_peered":"2023-08-25T09:45:07.014839+0000","last_unstale":"2023-08-25T09:45:08.457180+0000","last_undegraded":"2023-08-25T09:45:08.457180+0000","last_fullsized":"2023-08-25T09:45:08.457180+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:31.523445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455575+0000","last_change":"2023-08-25T09:45:07.017579+0000","last_active":"2023-08-25T09:45:08.455575+0000","last_peered":"2023-08-25T09:45:08.455575+0000","last_clean":"2023-08-25T09:45:08.455575+0000","last_became_active":"2023-08-25T09:45:07.012913+0000","last_became_peered":"2023-08-25T09:45:07.012913+0000","last_unstale":"2023-08-25T09:45:08.455575+0000","last_undegraded":"2023-08-25T09:45:08.455575+0000","last_fullsized":"2023-08-25T09:45:08.455575+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:41:37.605603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458242+0000","last_change":"2023-08-25T09:45:07.009670+0000","last_active":"2023-08-25T09:45:08.458242+0000","last_peered":"2023-08-25T09:45:08.458242+0000","last_clean":"2023-08-25T09:45:08.458242+0000","last_became_active":"2023-08-25T09:45:07.008750+0000","last_became_peered":"2023-08-25T09:45:07.008750+0000","last_unstale":"2023-08-25T09:45:08.458242+0000","last_undegraded":"2023-08-25T09:45:08.458242+0000","last_fullsized":"2023-08-25T09:45:08.458242+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:06:31.335736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457638+0000","last_change":"2023-08-25T09:45:07.021311+0000","last_active":"2023-08-25T09:45:08.457638+0000","last_peered":"2023-08-25T09:45:08.457638+0000","last_clean":"2023-08-25T09:45:08.457638+0000","last_became_active":"2023-08-25T09:45:07.021066+0000","last_became_peered":"2023-08-25T09:45:07.021066+0000","last_unstale":"2023-08-25T09:45:08.457638+0000","last_undegraded":"2023-08-25T09:45:08.457638+0000","last_fullsized":"2023-08-25T09:45:08.457638+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:58.112859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999290+0000","last_change":"2023-08-25T09:45:07.015711+0000","last_active":"2023-08-25T09:45:07.999290+0000","last_peered":"2023-08-25T09:45:07.999290+0000","last_clean":"2023-08-25T09:45:07.999290+0000","last_became_active":"2023-08-25T09:45:07.012945+0000","last_became_peered":"2023-08-25T09:45:07.012945+0000","last_unstale":"2023-08-25T09:45:07.999290+0000","last_undegraded":"2023-08-25T09:45:07.999290+0000","last_fullsized":"2023-08-25T09:45:07.999290+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:56.582338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457405+0000","last_change":"2023-08-25T09:45:07.028772+0000","last_active":"2023-08-25T09:45:08.457405+0000","last_peered":"2023-08-25T09:45:08.457405+0000","last_clean":"2023-08-25T09:45:08.457405+0000","last_became_active":"2023-08-25T09:45:07.020336+0000","last_became_peered":"2023-08-25T09:45:07.020336+0000","last_unstale":"2023-08-25T09:45:08.457405+0000","last_undegraded":"2023-08-25T09:45:08.457405+0000","last_fullsized":"2023-08-25T09:45:08.457405+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:31:31.074453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456138+0000","last_change":"2023-08-25T09:45:07.011738+0000","last_active":"2023-08-25T09:45:08.456138+0000","last_peered":"2023-08-25T09:45:08.456138+0000","last_clean":"2023-08-25T09:45:08.456138+0000","last_became_active":"2023-08-25T09:45:07.011483+0000","last_became_peered":"2023-08-25T09:45:07.011483+0000","last_unstale":"2023-08-25T09:45:08.456138+0000","last_undegraded":"2023-08-25T09:45:08.456138+0000","last_fullsized":"2023-08-25T09:45:08.456138+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:37.713289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458581+0000","last_change":"2023-08-25T09:45:07.010864+0000","last_active":"2023-08-25T09:45:08.458581+0000","last_peered":"2023-08-25T09:45:08.458581+0000","last_clean":"2023-08-25T09:45:08.458581+0000","last_became_active":"2023-08-25T09:45:07.010456+0000","last_became_peered":"2023-08-25T09:45:07.010456+0000","last_unstale":"2023-08-25T09:45:08.458581+0000","last_undegraded":"2023-08-25T09:45:08.458581+0000","last_fullsized":"2023-08-25T09:45:08.458581+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:22:11.323869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997912+0000","last_change":"2023-08-25T09:45:07.014242+0000","last_active":"2023-08-25T09:45:07.997912+0000","last_peered":"2023-08-25T09:45:07.997912+0000","last_clean":"2023-08-25T09:45:07.997912+0000","last_became_active":"2023-08-25T09:45:07.013874+0000","last_became_peered":"2023-08-25T09:45:07.013874+0000","last_unstale":"2023-08-25T09:45:07.997912+0000","last_undegraded":"2023-08-25T09:45:07.997912+0000","last_fullsized":"2023-08-25T09:45:07.997912+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:07:19.523502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997380+0000","last_change":"2023-08-25T09:45:07.028140+0000","last_active":"2023-08-25T09:45:07.997380+0000","last_peered":"2023-08-25T09:45:07.997380+0000","last_clean":"2023-08-25T09:45:07.997380+0000","last_became_active":"2023-08-25T09:45:07.027737+0000","last_became_peered":"2023-08-25T09:45:07.027737+0000","last_unstale":"2023-08-25T09:45:07.997380+0000","last_undegraded":"2023-08-25T09:45:07.997380+0000","last_fullsized":"2023-08-25T09:45:07.997380+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:41.641452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457703+0000","last_change":"2023-08-25T09:45:07.032688+0000","last_active":"2023-08-25T09:45:08.457703+0000","last_peered":"2023-08-25T09:45:08.457703+0000","last_clean":"2023-08-25T09:45:08.457703+0000","last_became_active":"2023-08-25T09:45:07.030877+0000","last_became_peered":"2023-08-25T09:45:07.030877+0000","last_unstale":"2023-08-25T09:45:08.457703+0000","last_undegraded":"2023-08-25T09:45:08.457703+0000","last_fullsized":"2023-08-25T09:45:08.457703+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:37:24.030940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"114'4","reported_seq":17,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457498+0000","last_change":"2023-08-25T09:45:06.008832+0000","last_active":"2023-08-25T09:45:08.457498+0000","last_peered":"2023-08-25T09:45:08.457498+0000","last_clean":"2023-08-25T09:45:08.457498+0000","last_became_active":"2023-08-25T09:45:06.008561+0000","last_became_peered":"2023-08-25T09:45:06.008561+0000","last_unstale":"2023-08-25T09:45:08.457498+0000","last_undegraded":"2023-08-25T09:45:08.457498+0000","last_fullsized":"2023-08-25T09:45:08.457498+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:55.247252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997661+0000","last_change":"2023-08-25T09:45:07.026484+0000","last_active":"2023-08-25T09:45:07.997661+0000","last_peered":"2023-08-25T09:45:07.997661+0000","last_clean":"2023-08-25T09:45:07.997661+0000","last_became_active":"2023-08-25T09:45:07.025954+0000","last_became_peered":"2023-08-25T09:45:07.025954+0000","last_unstale":"2023-08-25T09:45:07.997661+0000","last_undegraded":"2023-08-25T09:45:07.997661+0000","last_fullsized":"2023-08-25T09:45:07.997661+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:58.112859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.454999+0000","last_change":"2023-08-25T09:45:06.007006+0000","last_active":"2023-08-25T09:45:08.454999+0000","last_peered":"2023-08-25T09:45:08.454999+0000","last_clean":"2023-08-25T09:45:08.454999+0000","last_became_active":"2023-08-25T09:45:06.006496+0000","last_became_peered":"2023-08-25T09:45:06.006496+0000","last_unstale":"2023-08-25T09:45:08.454999+0000","last_undegraded":"2023-08-25T09:45:08.454999+0000","last_fullsized":"2023-08-25T09:45:08.454999+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:19:17.054478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456121+0000","last_change":"2023-08-25T09:45:07.017615+0000","last_active":"2023-08-25T09:45:08.456121+0000","last_peered":"2023-08-25T09:45:08.456121+0000","last_clean":"2023-08-25T09:45:08.456121+0000","last_became_active":"2023-08-25T09:45:07.016721+0000","last_became_peered":"2023-08-25T09:45:07.016721+0000","last_unstale":"2023-08-25T09:45:08.456121+0000","last_undegraded":"2023-08-25T09:45:08.456121+0000","last_fullsized":"2023-08-25T09:45:08.456121+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:36.251218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459405+0000","last_change":"2023-08-25T09:45:06.014579+0000","last_active":"2023-08-25T09:45:08.459405+0000","last_peered":"2023-08-25T09:45:08.459405+0000","last_clean":"2023-08-25T09:45:08.459405+0000","last_became_active":"2023-08-25T09:45:06.013805+0000","last_became_peered":"2023-08-25T09:45:06.013805+0000","last_unstale":"2023-08-25T09:45:08.459405+0000","last_undegraded":"2023-08-25T09:45:08.459405+0000","last_fullsized":"2023-08-25T09:45:08.459405+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:36:47.741454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999218+0000","last_change":"2023-08-25T09:45:07.031808+0000","last_active":"2023-08-25T09:45:07.999218+0000","last_peered":"2023-08-25T09:45:07.999218+0000","last_clean":"2023-08-25T09:45:07.999218+0000","last_became_active":"2023-08-25T09:45:07.030388+0000","last_became_peered":"2023-08-25T09:45:07.030388+0000","last_unstale":"2023-08-25T09:45:07.999218+0000","last_undegraded":"2023-08-25T09:45:07.999218+0000","last_fullsized":"2023-08-25T09:45:07.999218+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:38:59.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455315+0000","last_change":"2023-08-25T09:45:06.015040+0000","last_active":"2023-08-25T09:45:08.455315+0000","last_peered":"2023-08-25T09:45:08.455315+0000","last_clean":"2023-08-25T09:45:08.455315+0000","last_became_active":"2023-08-25T09:45:06.014290+0000","last_became_peered":"2023-08-25T09:45:06.014290+0000","last_unstale":"2023-08-25T09:45:08.455315+0000","last_undegraded":"2023-08-25T09:45:08.455315+0000","last_fullsized":"2023-08-25T09:45:08.455315+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:29:47.318166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456623+0000","last_change":"2023-08-25T09:45:07.015641+0000","last_active":"2023-08-25T09:45:08.456623+0000","last_peered":"2023-08-25T09:45:08.456623+0000","last_clean":"2023-08-25T09:45:08.456623+0000","last_became_active":"2023-08-25T09:45:07.012560+0000","last_became_peered":"2023-08-25T09:45:07.012560+0000","last_unstale":"2023-08-25T09:45:08.456623+0000","last_undegraded":"2023-08-25T09:45:08.456623+0000","last_fullsized":"2023-08-25T09:45:08.456623+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:25.238443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998717+0000","last_change":"2023-08-25T09:45:06.005724+0000","last_active":"2023-08-25T09:45:07.998717+0000","last_peered":"2023-08-25T09:45:07.998717+0000","last_clean":"2023-08-25T09:45:07.998717+0000","last_became_active":"2023-08-25T09:45:06.004851+0000","last_became_peered":"2023-08-25T09:45:06.004851+0000","last_unstale":"2023-08-25T09:45:07.998717+0000","last_undegraded":"2023-08-25T09:45:07.998717+0000","last_fullsized":"2023-08-25T09:45:07.998717+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:09:24.138960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456493+0000","last_change":"2023-08-25T09:45:07.004212+0000","last_active":"2023-08-25T09:45:08.456493+0000","last_peered":"2023-08-25T09:45:08.456493+0000","last_clean":"2023-08-25T09:45:08.456493+0000","last_became_active":"2023-08-25T09:45:07.003746+0000","last_became_peered":"2023-08-25T09:45:07.003746+0000","last_unstale":"2023-08-25T09:45:08.456493+0000","last_undegraded":"2023-08-25T09:45:08.456493+0000","last_fullsized":"2023-08-25T09:45:08.456493+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:22:08.206567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457034+0000","last_change":"2023-08-25T09:45:06.012843+0000","last_active":"2023-08-25T09:45:08.457034+0000","last_peered":"2023-08-25T09:45:08.457034+0000","last_clean":"2023-08-25T09:45:08.457034+0000","last_became_active":"2023-08-25T09:45:06.012483+0000","last_became_peered":"2023-08-25T09:45:06.012483+0000","last_unstale":"2023-08-25T09:45:08.457034+0000","last_undegraded":"2023-08-25T09:45:08.457034+0000","last_fullsized":"2023-08-25T09:45:08.457034+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:36.691777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997935+0000","last_change":"2023-08-25T09:45:07.027023+0000","last_active":"2023-08-25T09:45:07.997935+0000","last_peered":"2023-08-25T09:45:07.997935+0000","last_clean":"2023-08-25T09:45:07.997935+0000","last_became_active":"2023-08-25T09:45:07.026782+0000","last_became_peered":"2023-08-25T09:45:07.026782+0000","last_unstale":"2023-08-25T09:45:07.997935+0000","last_undegraded":"2023-08-25T09:45:07.997935+0000","last_fullsized":"2023-08-25T09:45:07.997935+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:05.257568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999933+0000","last_change":"2023-08-25T09:45:06.011016+0000","last_active":"2023-08-25T09:45:07.999933+0000","last_peered":"2023-08-25T09:45:07.999933+0000","last_clean":"2023-08-25T09:45:07.999933+0000","last_became_active":"2023-08-25T09:45:06.010625+0000","last_became_peered":"2023-08-25T09:45:06.010625+0000","last_unstale":"2023-08-25T09:45:07.999933+0000","last_undegraded":"2023-08-25T09:45:07.999933+0000","last_fullsized":"2023-08-25T09:45:07.999933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:44:16.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455589+0000","last_change":"2023-08-25T09:45:07.018964+0000","last_active":"2023-08-25T09:45:08.455589+0000","last_peered":"2023-08-25T09:45:08.455589+0000","last_clean":"2023-08-25T09:45:08.455589+0000","last_became_active":"2023-08-25T09:45:07.018539+0000","last_became_peered":"2023-08-25T09:45:07.018539+0000","last_unstale":"2023-08-25T09:45:08.455589+0000","last_undegraded":"2023-08-25T09:45:08.455589+0000","last_fullsized":"2023-08-25T09:45:08.455589+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:14:27.519003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459559+0000","last_change":"2023-08-25T09:45:06.014462+0000","last_active":"2023-08-25T09:45:08.459559+0000","last_peered":"2023-08-25T09:45:08.459559+0000","last_clean":"2023-08-25T09:45:08.459559+0000","last_became_active":"2023-08-25T09:45:06.013461+0000","last_became_peered":"2023-08-25T09:45:06.013461+0000","last_unstale":"2023-08-25T09:45:08.459559+0000","last_undegraded":"2023-08-25T09:45:08.459559+0000","last_fullsized":"2023-08-25T09:45:08.459559+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:42:48.543047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456465+0000","last_change":"2023-08-25T09:45:07.017168+0000","last_active":"2023-08-25T09:45:08.456465+0000","last_peered":"2023-08-25T09:45:08.456465+0000","last_clean":"2023-08-25T09:45:08.456465+0000","last_became_active":"2023-08-25T09:45:07.016927+0000","last_became_peered":"2023-08-25T09:45:07.016927+0000","last_unstale":"2023-08-25T09:45:08.456465+0000","last_undegraded":"2023-08-25T09:45:08.456465+0000","last_fullsized":"2023-08-25T09:45:08.456465+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:31:31.074453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458200+0000","last_change":"2023-08-25T09:45:06.017079+0000","last_active":"2023-08-25T09:45:08.458200+0000","last_peered":"2023-08-25T09:45:08.458200+0000","last_clean":"2023-08-25T09:45:08.458200+0000","last_became_active":"2023-08-25T09:45:06.016691+0000","last_became_peered":"2023-08-25T09:45:06.016691+0000","last_unstale":"2023-08-25T09:45:08.458200+0000","last_undegraded":"2023-08-25T09:45:08.458200+0000","last_fullsized":"2023-08-25T09:45:08.458200+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:10:02.255328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458231+0000","last_change":"2023-08-25T09:45:07.006905+0000","last_active":"2023-08-25T09:45:08.458231+0000","last_peered":"2023-08-25T09:45:08.458231+0000","last_clean":"2023-08-25T09:45:08.458231+0000","last_became_active":"2023-08-25T09:45:07.006654+0000","last_became_peered":"2023-08-25T09:45:07.006654+0000","last_unstale":"2023-08-25T09:45:08.458231+0000","last_undegraded":"2023-08-25T09:45:08.458231+0000","last_fullsized":"2023-08-25T09:45:08.458231+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:12:55.020365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.000024+0000","last_change":"2023-08-25T09:45:06.012825+0000","last_active":"2023-08-25T09:45:08.000024+0000","last_peered":"2023-08-25T09:45:08.000024+0000","last_clean":"2023-08-25T09:45:08.000024+0000","last_became_active":"2023-08-25T09:45:06.012494+0000","last_became_peered":"2023-08-25T09:45:06.012494+0000","last_unstale":"2023-08-25T09:45:08.000024+0000","last_undegraded":"2023-08-25T09:45:08.000024+0000","last_fullsized":"2023-08-25T09:45:08.000024+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:00:44.938914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455938+0000","last_change":"2023-08-25T09:45:07.029628+0000","last_active":"2023-08-25T09:45:08.455938+0000","last_peered":"2023-08-25T09:45:08.455938+0000","last_clean":"2023-08-25T09:45:08.455938+0000","last_became_active":"2023-08-25T09:45:07.029397+0000","last_became_peered":"2023-08-25T09:45:07.029397+0000","last_unstale":"2023-08-25T09:45:08.455938+0000","last_undegraded":"2023-08-25T09:45:08.455938+0000","last_fullsized":"2023-08-25T09:45:08.455938+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:26:52.553474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"114'3","reported_seq":16,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458933+0000","last_change":"2023-08-25T09:45:06.015147+0000","last_active":"2023-08-25T09:45:08.458933+0000","last_peered":"2023-08-25T09:45:08.458933+0000","last_clean":"2023-08-25T09:45:08.458933+0000","last_became_active":"2023-08-25T09:45:06.014444+0000","last_became_peered":"2023-08-25T09:45:06.014444+0000","last_unstale":"2023-08-25T09:45:08.458933+0000","last_undegraded":"2023-08-25T09:45:08.458933+0000","last_fullsized":"2023-08-25T09:45:08.458933+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:24.480215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997351+0000","last_change":"2023-08-25T09:45:07.028837+0000","last_active":"2023-08-25T09:45:07.997351+0000","last_peered":"2023-08-25T09:45:07.997351+0000","last_clean":"2023-08-25T09:45:07.997351+0000","last_became_active":"2023-08-25T09:45:07.027502+0000","last_became_peered":"2023-08-25T09:45:07.027502+0000","last_unstale":"2023-08-25T09:45:07.997351+0000","last_undegraded":"2023-08-25T09:45:07.997351+0000","last_fullsized":"2023-08-25T09:45:07.997351+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:25.169720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"114'6","reported_seq":19,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455939+0000","last_change":"2023-08-25T09:45:06.011490+0000","last_active":"2023-08-25T09:45:08.455939+0000","last_peered":"2023-08-25T09:45:08.455939+0000","last_clean":"2023-08-25T09:45:08.455939+0000","last_became_active":"2023-08-25T09:45:06.011205+0000","last_became_peered":"2023-08-25T09:45:06.011205+0000","last_unstale":"2023-08-25T09:45:08.455939+0000","last_undegraded":"2023-08-25T09:45:08.455939+0000","last_fullsized":"2023-08-25T09:45:08.455939+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:20:01.175314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456459+0000","last_change":"2023-08-25T09:45:07.021129+0000","last_active":"2023-08-25T09:45:08.456459+0000","last_peered":"2023-08-25T09:45:08.456459+0000","last_clean":"2023-08-25T09:45:08.456459+0000","last_became_active":"2023-08-25T09:45:07.009281+0000","last_became_peered":"2023-08-25T09:45:07.009281+0000","last_unstale":"2023-08-25T09:45:08.456459+0000","last_undegraded":"2023-08-25T09:45:08.456459+0000","last_fullsized":"2023-08-25T09:45:08.456459+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:05:16.912498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459734+0000","last_change":"2023-08-25T09:45:06.014337+0000","last_active":"2023-08-25T09:45:08.459734+0000","last_peered":"2023-08-25T09:45:08.459734+0000","last_clean":"2023-08-25T09:45:08.459734+0000","last_became_active":"2023-08-25T09:45:06.013135+0000","last_became_peered":"2023-08-25T09:45:06.013135+0000","last_unstale":"2023-08-25T09:45:08.459734+0000","last_undegraded":"2023-08-25T09:45:08.459734+0000","last_fullsized":"2023-08-25T09:45:08.459734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:03.026776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457934+0000","last_change":"2023-08-25T09:45:07.013029+0000","last_active":"2023-08-25T09:45:08.457934+0000","last_peered":"2023-08-25T09:45:08.457934+0000","last_clean":"2023-08-25T09:45:08.457934+0000","last_became_active":"2023-08-25T09:45:07.012794+0000","last_became_peered":"2023-08-25T09:45:07.012794+0000","last_unstale":"2023-08-25T09:45:08.457934+0000","last_undegraded":"2023-08-25T09:45:08.457934+0000","last_fullsized":"2023-08-25T09:45:08.457934+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:05.257568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457891+0000","last_change":"2023-08-25T09:45:06.010519+0000","last_active":"2023-08-25T09:45:08.457891+0000","last_peered":"2023-08-25T09:45:08.457891+0000","last_clean":"2023-08-25T09:45:08.457891+0000","last_became_active":"2023-08-25T09:45:06.010124+0000","last_became_peered":"2023-08-25T09:45:06.010124+0000","last_unstale":"2023-08-25T09:45:08.457891+0000","last_undegraded":"2023-08-25T09:45:08.457891+0000","last_fullsized":"2023-08-25T09:45:08.457891+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:58:46.096650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457402+0000","last_change":"2023-08-25T09:45:07.010841+0000","last_active":"2023-08-25T09:45:08.457402+0000","last_peered":"2023-08-25T09:45:08.457402+0000","last_clean":"2023-08-25T09:45:08.457402+0000","last_became_active":"2023-08-25T09:45:07.009239+0000","last_became_peered":"2023-08-25T09:45:07.009239+0000","last_unstale":"2023-08-25T09:45:08.457402+0000","last_undegraded":"2023-08-25T09:45:08.457402+0000","last_fullsized":"2023-08-25T09:45:08.457402+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:06:18.447160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456260+0000","last_change":"2023-08-25T09:45:06.016643+0000","last_active":"2023-08-25T09:45:08.456260+0000","last_peered":"2023-08-25T09:45:08.456260+0000","last_clean":"2023-08-25T09:45:08.456260+0000","last_became_active":"2023-08-25T09:45:06.016388+0000","last_became_peered":"2023-08-25T09:45:06.016388+0000","last_unstale":"2023-08-25T09:45:08.456260+0000","last_undegraded":"2023-08-25T09:45:08.456260+0000","last_fullsized":"2023-08-25T09:45:08.456260+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:31:15.463974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457000+0000","last_change":"2023-08-25T09:45:07.017011+0000","last_active":"2023-08-25T09:45:08.457000+0000","last_peered":"2023-08-25T09:45:08.457000+0000","last_clean":"2023-08-25T09:45:08.457000+0000","last_became_active":"2023-08-25T09:45:07.013095+0000","last_became_peered":"2023-08-25T09:45:07.013095+0000","last_unstale":"2023-08-25T09:45:08.457000+0000","last_undegraded":"2023-08-25T09:45:08.457000+0000","last_fullsized":"2023-08-25T09:45:08.457000+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:37:44.533417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"114'8","reported_seq":21,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999913+0000","last_change":"2023-08-25T09:45:06.013865+0000","last_active":"2023-08-25T09:45:07.999913+0000","last_peered":"2023-08-25T09:45:07.999913+0000","last_clean":"2023-08-25T09:45:07.999913+0000","last_became_active":"2023-08-25T09:45:06.012819+0000","last_became_peered":"2023-08-25T09:45:06.012819+0000","last_unstale":"2023-08-25T09:45:07.999913+0000","last_undegraded":"2023-08-25T09:45:07.999913+0000","last_fullsized":"2023-08-25T09:45:07.999913+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:04:22.417962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997333+0000","last_change":"2023-08-25T09:45:07.012196+0000","last_active":"2023-08-25T09:45:07.997333+0000","last_peered":"2023-08-25T09:45:07.997333+0000","last_clean":"2023-08-25T09:45:07.997333+0000","last_became_active":"2023-08-25T09:45:07.011966+0000","last_became_peered":"2023-08-25T09:45:07.011966+0000","last_unstale":"2023-08-25T09:45:07.997333+0000","last_undegraded":"2023-08-25T09:45:07.997333+0000","last_fullsized":"2023-08-25T09:45:07.997333+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:12:26.608084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456157+0000","last_change":"2023-08-25T09:45:06.010923+0000","last_active":"2023-08-25T09:45:08.456157+0000","last_peered":"2023-08-25T09:45:08.456157+0000","last_clean":"2023-08-25T09:45:08.456157+0000","last_became_active":"2023-08-25T09:45:06.009176+0000","last_became_peered":"2023-08-25T09:45:06.009176+0000","last_unstale":"2023-08-25T09:45:08.456157+0000","last_undegraded":"2023-08-25T09:45:08.456157+0000","last_fullsized":"2023-08-25T09:45:08.456157+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:54:20.606839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455578+0000","last_change":"2023-08-25T09:45:07.017770+0000","last_active":"2023-08-25T09:45:08.455578+0000","last_peered":"2023-08-25T09:45:08.455578+0000","last_clean":"2023-08-25T09:45:08.455578+0000","last_became_active":"2023-08-25T09:45:07.017283+0000","last_became_peered":"2023-08-25T09:45:07.017283+0000","last_unstale":"2023-08-25T09:45:08.455578+0000","last_undegraded":"2023-08-25T09:45:08.455578+0000","last_fullsized":"2023-08-25T09:45:08.455578+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:22:37.842328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.459253+0000","last_change":"2023-08-25T09:45:06.014822+0000","last_active":"2023-08-25T09:45:08.459253+0000","last_peered":"2023-08-25T09:45:08.459253+0000","last_clean":"2023-08-25T09:45:08.459253+0000","last_became_active":"2023-08-25T09:45:06.011281+0000","last_became_peered":"2023-08-25T09:45:06.011281+0000","last_unstale":"2023-08-25T09:45:08.459253+0000","last_undegraded":"2023-08-25T09:45:08.459253+0000","last_fullsized":"2023-08-25T09:45:08.459253+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:36:44.768678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456638+0000","last_change":"2023-08-25T09:45:07.030669+0000","last_active":"2023-08-25T09:45:08.456638+0000","last_peered":"2023-08-25T09:45:08.456638+0000","last_clean":"2023-08-25T09:45:08.456638+0000","last_became_active":"2023-08-25T09:45:07.020707+0000","last_became_peered":"2023-08-25T09:45:07.020707+0000","last_unstale":"2023-08-25T09:45:08.456638+0000","last_undegraded":"2023-08-25T09:45:08.456638+0000","last_fullsized":"2023-08-25T09:45:08.456638+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:13.015807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997728+0000","last_change":"2023-08-25T09:45:06.013441+0000","last_active":"2023-08-25T09:45:07.997728+0000","last_peered":"2023-08-25T09:45:07.997728+0000","last_clean":"2023-08-25T09:45:07.997728+0000","last_became_active":"2023-08-25T09:45:06.012972+0000","last_became_peered":"2023-08-25T09:45:06.012972+0000","last_unstale":"2023-08-25T09:45:07.997728+0000","last_undegraded":"2023-08-25T09:45:07.997728+0000","last_fullsized":"2023-08-25T09:45:07.997728+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:45:51.015883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458336+0000","last_change":"2023-08-25T09:45:07.011038+0000","last_active":"2023-08-25T09:45:08.458336+0000","last_peered":"2023-08-25T09:45:08.458336+0000","last_clean":"2023-08-25T09:45:08.458336+0000","last_became_active":"2023-08-25T09:45:07.010757+0000","last_became_peered":"2023-08-25T09:45:07.010757+0000","last_unstale":"2023-08-25T09:45:08.458336+0000","last_undegraded":"2023-08-25T09:45:08.458336+0000","last_fullsized":"2023-08-25T09:45:08.458336+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:59:42.104116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457222+0000","last_change":"2023-08-25T09:45:06.016203+0000","last_active":"2023-08-25T09:45:08.457222+0000","last_peered":"2023-08-25T09:45:08.457222+0000","last_clean":"2023-08-25T09:45:08.457222+0000","last_became_active":"2023-08-25T09:45:06.015680+0000","last_became_peered":"2023-08-25T09:45:06.015680+0000","last_unstale":"2023-08-25T09:45:08.457222+0000","last_undegraded":"2023-08-25T09:45:08.457222+0000","last_fullsized":"2023-08-25T09:45:08.457222+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:03:35.393847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.996369+0000","last_change":"2023-08-25T09:45:07.018321+0000","last_active":"2023-08-25T09:45:07.996369+0000","last_peered":"2023-08-25T09:45:07.996369+0000","last_clean":"2023-08-25T09:45:07.996369+0000","last_became_active":"2023-08-25T09:45:07.018012+0000","last_became_peered":"2023-08-25T09:45:07.018012+0000","last_unstale":"2023-08-25T09:45:07.996369+0000","last_undegraded":"2023-08-25T09:45:07.996369+0000","last_fullsized":"2023-08-25T09:45:07.996369+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:53:37.713289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458047+0000","last_change":"2023-08-25T09:45:06.015000+0000","last_active":"2023-08-25T09:45:08.458047+0000","last_peered":"2023-08-25T09:45:08.458047+0000","last_clean":"2023-08-25T09:45:08.458047+0000","last_became_active":"2023-08-25T09:45:06.014711+0000","last_became_peered":"2023-08-25T09:45:06.014711+0000","last_unstale":"2023-08-25T09:45:08.458047+0000","last_undegraded":"2023-08-25T09:45:08.458047+0000","last_fullsized":"2023-08-25T09:45:08.458047+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:38:41.808705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.999549+0000","last_change":"2023-08-25T09:45:07.031211+0000","last_active":"2023-08-25T09:45:07.999549+0000","last_peered":"2023-08-25T09:45:07.999549+0000","last_clean":"2023-08-25T09:45:07.999549+0000","last_became_active":"2023-08-25T09:45:07.029888+0000","last_became_peered":"2023-08-25T09:45:07.029888+0000","last_unstale":"2023-08-25T09:45:07.999549+0000","last_undegraded":"2023-08-25T09:45:07.999549+0000","last_fullsized":"2023-08-25T09:45:07.999549+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:03.991566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458555+0000","last_change":"2023-08-25T09:45:06.012784+0000","last_active":"2023-08-25T09:45:08.458555+0000","last_peered":"2023-08-25T09:45:08.458555+0000","last_clean":"2023-08-25T09:45:08.458555+0000","last_became_active":"2023-08-25T09:45:06.009560+0000","last_became_peered":"2023-08-25T09:45:06.009560+0000","last_unstale":"2023-08-25T09:45:08.458555+0000","last_undegraded":"2023-08-25T09:45:08.458555+0000","last_fullsized":"2023-08-25T09:45:08.458555+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:09:13.825828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998217+0000","last_change":"2023-08-25T09:45:07.019869+0000","last_active":"2023-08-25T09:45:07.998217+0000","last_peered":"2023-08-25T09:45:07.998217+0000","last_clean":"2023-08-25T09:45:07.998217+0000","last_became_active":"2023-08-25T09:45:07.019473+0000","last_became_peered":"2023-08-25T09:45:07.019473+0000","last_unstale":"2023-08-25T09:45:07.998217+0000","last_undegraded":"2023-08-25T09:45:07.998217+0000","last_fullsized":"2023-08-25T09:45:07.998217+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:06:06.316438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457980+0000","last_change":"2023-08-25T09:45:06.007418+0000","last_active":"2023-08-25T09:45:08.457980+0000","last_peered":"2023-08-25T09:45:08.457980+0000","last_clean":"2023-08-25T09:45:08.457980+0000","last_became_active":"2023-08-25T09:45:06.007192+0000","last_became_peered":"2023-08-25T09:45:06.007192+0000","last_unstale":"2023-08-25T09:45:08.457980+0000","last_undegraded":"2023-08-25T09:45:08.457980+0000","last_fullsized":"2023-08-25T09:45:08.457980+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:11:50.670235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998297+0000","last_change":"2023-08-25T09:45:07.025834+0000","last_active":"2023-08-25T09:45:07.998297+0000","last_peered":"2023-08-25T09:45:07.998297+0000","last_clean":"2023-08-25T09:45:07.998297+0000","last_became_active":"2023-08-25T09:45:07.025294+0000","last_became_peered":"2023-08-25T09:45:07.025294+0000","last_unstale":"2023-08-25T09:45:07.998297+0000","last_undegraded":"2023-08-25T09:45:07.998297+0000","last_fullsized":"2023-08-25T09:45:07.998297+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:40:24.636956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456778+0000","last_change":"2023-08-25T09:45:06.008788+0000","last_active":"2023-08-25T09:45:08.456778+0000","last_peered":"2023-08-25T09:45:08.456778+0000","last_clean":"2023-08-25T09:45:08.456778+0000","last_became_active":"2023-08-25T09:45:06.008551+0000","last_became_peered":"2023-08-25T09:45:06.008551+0000","last_unstale":"2023-08-25T09:45:08.456778+0000","last_undegraded":"2023-08-25T09:45:08.456778+0000","last_fullsized":"2023-08-25T09:45:08.456778+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:51:24.657773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457078+0000","last_change":"2023-08-25T09:45:07.015935+0000","last_active":"2023-08-25T09:45:08.457078+0000","last_peered":"2023-08-25T09:45:08.457078+0000","last_clean":"2023-08-25T09:45:08.457078+0000","last_became_active":"2023-08-25T09:45:07.015655+0000","last_became_peered":"2023-08-25T09:45:07.015655+0000","last_unstale":"2023-08-25T09:45:08.457078+0000","last_undegraded":"2023-08-25T09:45:08.457078+0000","last_fullsized":"2023-08-25T09:45:08.457078+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:47:14.571213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456596+0000","last_change":"2023-08-25T09:45:06.011461+0000","last_active":"2023-08-25T09:45:08.456596+0000","last_peered":"2023-08-25T09:45:08.456596+0000","last_clean":"2023-08-25T09:45:08.456596+0000","last_became_active":"2023-08-25T09:45:06.011148+0000","last_became_peered":"2023-08-25T09:45:06.011148+0000","last_unstale":"2023-08-25T09:45:08.456596+0000","last_undegraded":"2023-08-25T09:45:08.456596+0000","last_fullsized":"2023-08-25T09:45:08.456596+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:22.443464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":330,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457294+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:45:08.457294+0000","last_peered":"2023-08-25T09:45:08.457294+0000","last_clean":"2023-08-25T09:45:08.457294+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:45:08.457294+0000","last_undegraded":"2023-08-25T09:45:08.457294+0000","last_fullsized":"2023-08-25T09:45:08.457294+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:15.605498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456698+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:45:08.456698+0000","last_peered":"2023-08-25T09:45:08.456698+0000","last_clean":"2023-08-25T09:45:08.456698+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:45:08.456698+0000","last_undegraded":"2023-08-25T09:45:08.456698+0000","last_fullsized":"2023-08-25T09:45:08.456698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:30.563884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455743+0000","last_change":"2023-08-25T09:45:07.016220+0000","last_active":"2023-08-25T09:45:08.455743+0000","last_peered":"2023-08-25T09:45:08.455743+0000","last_clean":"2023-08-25T09:45:08.455743+0000","last_became_active":"2023-08-25T09:45:07.015601+0000","last_became_peered":"2023-08-25T09:45:07.015601+0000","last_unstale":"2023-08-25T09:45:08.455743+0000","last_undegraded":"2023-08-25T09:45:08.455743+0000","last_fullsized":"2023-08-25T09:45:08.455743+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:44:07.745761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997611+0000","last_change":"2023-08-25T09:45:06.006735+0000","last_active":"2023-08-25T09:45:07.997611+0000","last_peered":"2023-08-25T09:45:07.997611+0000","last_clean":"2023-08-25T09:45:07.997611+0000","last_became_active":"2023-08-25T09:45:06.006166+0000","last_became_peered":"2023-08-25T09:45:06.006166+0000","last_unstale":"2023-08-25T09:45:07.997611+0000","last_undegraded":"2023-08-25T09:45:07.997611+0000","last_fullsized":"2023-08-25T09:45:07.997611+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:53.981249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457478+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:45:08.457478+0000","last_peered":"2023-08-25T09:45:08.457478+0000","last_clean":"2023-08-25T09:45:08.457478+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:45:08.457478+0000","last_undegraded":"2023-08-25T09:45:08.457478+0000","last_fullsized":"2023-08-25T09:45:08.457478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:37.642813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456459+0000","last_change":"2023-08-25T09:45:07.012508+0000","last_active":"2023-08-25T09:45:08.456459+0000","last_peered":"2023-08-25T09:45:08.456459+0000","last_clean":"2023-08-25T09:45:08.456459+0000","last_became_active":"2023-08-25T09:45:07.012059+0000","last_became_peered":"2023-08-25T09:45:07.012059+0000","last_unstale":"2023-08-25T09:45:08.456459+0000","last_undegraded":"2023-08-25T09:45:08.456459+0000","last_fullsized":"2023-08-25T09:45:08.456459+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:44:24.456723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455769+0000","last_change":"2023-08-25T09:45:06.016416+0000","last_active":"2023-08-25T09:45:08.455769+0000","last_peered":"2023-08-25T09:45:08.455769+0000","last_clean":"2023-08-25T09:45:08.455769+0000","last_became_active":"2023-08-25T09:45:06.016184+0000","last_became_peered":"2023-08-25T09:45:06.016184+0000","last_unstale":"2023-08-25T09:45:08.455769+0000","last_undegraded":"2023-08-25T09:45:08.455769+0000","last_fullsized":"2023-08-25T09:45:08.455769+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:54.539445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":204,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.000122+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:45:08.000122+0000","last_peered":"2023-08-25T09:45:08.000122+0000","last_clean":"2023-08-25T09:45:08.000122+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:45:08.000122+0000","last_undegraded":"2023-08-25T09:45:08.000122+0000","last_fullsized":"2023-08-25T09:45:08.000122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:22:22.645926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458933+0000","last_change":"2023-08-25T09:45:07.010712+0000","last_active":"2023-08-25T09:45:08.458933+0000","last_peered":"2023-08-25T09:45:08.458933+0000","last_clean":"2023-08-25T09:45:08.458933+0000","last_became_active":"2023-08-25T09:45:07.010065+0000","last_became_peered":"2023-08-25T09:45:07.010065+0000","last_unstale":"2023-08-25T09:45:08.458933+0000","last_undegraded":"2023-08-25T09:45:08.458933+0000","last_fullsized":"2023-08-25T09:45:08.458933+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:43:18.759455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"114'1","reported_seq":14,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455638+0000","last_change":"2023-08-25T09:45:06.015186+0000","last_active":"2023-08-25T09:45:08.455638+0000","last_peered":"2023-08-25T09:45:08.455638+0000","last_clean":"2023-08-25T09:45:08.455638+0000","last_became_active":"2023-08-25T09:45:06.014799+0000","last_became_peered":"2023-08-25T09:45:06.014799+0000","last_unstale":"2023-08-25T09:45:08.455638+0000","last_undegraded":"2023-08-25T09:45:08.455638+0000","last_fullsized":"2023-08-25T09:45:08.455638+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:40.638351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456639+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:45:08.456639+0000","last_peered":"2023-08-25T09:45:08.456639+0000","last_clean":"2023-08-25T09:45:08.456639+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:45:08.456639+0000","last_undegraded":"2023-08-25T09:45:08.456639+0000","last_fullsized":"2023-08-25T09:45:08.456639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:21:47.636025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997674+0000","last_change":"2023-08-25T09:45:07.026005+0000","last_active":"2023-08-25T09:45:07.997674+0000","last_peered":"2023-08-25T09:45:07.997674+0000","last_clean":"2023-08-25T09:45:07.997674+0000","last_became_active":"2023-08-25T09:45:07.025556+0000","last_became_peered":"2023-08-25T09:45:07.025556+0000","last_unstale":"2023-08-25T09:45:07.997674+0000","last_undegraded":"2023-08-25T09:45:07.997674+0000","last_fullsized":"2023-08-25T09:45:07.997674+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:33:34.026912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"114'2","reported_seq":15,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.458569+0000","last_change":"2023-08-25T09:45:06.014987+0000","last_active":"2023-08-25T09:45:08.458569+0000","last_peered":"2023-08-25T09:45:08.458569+0000","last_clean":"2023-08-25T09:45:08.458569+0000","last_became_active":"2023-08-25T09:45:06.013951+0000","last_became_peered":"2023-08-25T09:45:06.013951+0000","last_unstale":"2023-08-25T09:45:08.458569+0000","last_undegraded":"2023-08-25T09:45:08.458569+0000","last_fullsized":"2023-08-25T09:45:08.458569+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:30.852980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":178,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457386+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:45:08.457386+0000","last_peered":"2023-08-25T09:45:08.457386+0000","last_clean":"2023-08-25T09:45:08.457386+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:45:08.457386+0000","last_undegraded":"2023-08-25T09:45:08.457386+0000","last_fullsized":"2023-08-25T09:45:08.457386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:45:05.697579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.455394+0000","last_change":"2023-08-25T09:45:07.020753+0000","last_active":"2023-08-25T09:45:08.455394+0000","last_peered":"2023-08-25T09:45:08.455394+0000","last_clean":"2023-08-25T09:45:08.455394+0000","last_became_active":"2023-08-25T09:45:07.020515+0000","last_became_peered":"2023-08-25T09:45:07.020515+0000","last_unstale":"2023-08-25T09:45:08.455394+0000","last_undegraded":"2023-08-25T09:45:08.455394+0000","last_fullsized":"2023-08-25T09:45:08.455394+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:46.095770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.996707+0000","last_change":"2023-08-25T09:45:06.013140+0000","last_active":"2023-08-25T09:45:07.996707+0000","last_peered":"2023-08-25T09:45:07.996707+0000","last_clean":"2023-08-25T09:45:07.996707+0000","last_became_active":"2023-08-25T09:45:06.012902+0000","last_became_peered":"2023-08-25T09:45:06.012902+0000","last_unstale":"2023-08-25T09:45:07.996707+0000","last_undegraded":"2023-08-25T09:45:07.996707+0000","last_fullsized":"2023-08-25T09:45:07.996707+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:31:30.052940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":198,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.457039+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:45:08.457039+0000","last_peered":"2023-08-25T09:45:08.457039+0000","last_clean":"2023-08-25T09:45:08.457039+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:45:08.457039+0000","last_undegraded":"2023-08-25T09:45:08.457039+0000","last_fullsized":"2023-08-25T09:45:08.457039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:08.303316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.997478+0000","last_change":"2023-08-25T09:45:07.023970+0000","last_active":"2023-08-25T09:45:07.997478+0000","last_peered":"2023-08-25T09:45:07.997478+0000","last_clean":"2023-08-25T09:45:07.997478+0000","last_became_active":"2023-08-25T09:45:07.023533+0000","last_became_peered":"2023-08-25T09:45:07.023533+0000","last_unstale":"2023-08-25T09:45:07.997478+0000","last_undegraded":"2023-08-25T09:45:07.997478+0000","last_fullsized":"2023-08-25T09:45:07.997478+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:26:04.196878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456076+0000","last_change":"2023-08-25T09:45:06.004354+0000","last_active":"2023-08-25T09:45:08.456076+0000","last_peered":"2023-08-25T09:45:08.456076+0000","last_clean":"2023-08-25T09:45:08.456076+0000","last_became_active":"2023-08-25T09:45:06.003654+0000","last_became_peered":"2023-08-25T09:45:06.003654+0000","last_unstale":"2023-08-25T09:45:08.456076+0000","last_undegraded":"2023-08-25T09:45:08.456076+0000","last_fullsized":"2023-08-25T09:45:08.456076+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:49:21.205415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":173,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:07.998435+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:45:07.998435+0000","last_peered":"2023-08-25T09:45:07.998435+0000","last_clean":"2023-08-25T09:45:07.998435+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:45:07.998435+0000","last_undegraded":"2023-08-25T09:45:07.998435+0000","last_fullsized":"2023-08-25T09:45:07.998435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:03:43.295133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":11,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456824+0000","last_change":"2023-08-25T09:45:07.024647+0000","last_active":"2023-08-25T09:45:08.456824+0000","last_peered":"2023-08-25T09:45:08.456824+0000","last_clean":"2023-08-25T09:45:08.456824+0000","last_became_active":"2023-08-25T09:45:07.024203+0000","last_became_peered":"2023-08-25T09:45:07.024203+0000","last_unstale":"2023-08-25T09:45:08.456824+0000","last_undegraded":"2023-08-25T09:45:08.456824+0000","last_fullsized":"2023-08-25T09:45:08.456824+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:05.972294+0000","last_clean_scrub_stamp":"2023-08-25T09:45:05.972294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:03:02.715234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456152+0000","last_change":"2023-08-25T09:45:06.011087+0000","last_active":"2023-08-25T09:45:08.456152+0000","last_peered":"2023-08-25T09:45:08.456152+0000","last_clean":"2023-08-25T09:45:08.456152+0000","last_became_active":"2023-08-25T09:45:06.010455+0000","last_became_peered":"2023-08-25T09:45:06.010455+0000","last_unstale":"2023-08-25T09:45:08.456152+0000","last_undegraded":"2023-08-25T09:45:08.456152+0000","last_fullsized":"2023-08-25T09:45:08.456152+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:45:04.950782+0000","last_clean_scrub_stamp":"2023-08-25T09:45:04.950782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:15.251374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-08-25T09:45:08.456352+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:45:08.456352+0000","last_peered":"2023-08-25T09:45:08.456352+0000","last_clean":"2023-08-25T09:45:08.456352+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:45:08.456352+0000","last_undegraded":"2023-08-25T09:45:08.456352+0000","last_fullsized":"2023-08-25T09:45:08.456352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:58.356461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574928,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.206,"15min":1.071},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":10.747,"15min":10.747},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1319999999999999,"15min":1.044},"min":{"1min":0.76800000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1970000000000001,"15min":1.133},"min":{"1min":0.81200000000000006,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.3359999999999999,"5min":2.3359999999999999,"15min":2.3359999999999999},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.198,"15min":1.0980000000000001},"min":{"1min":0.66500000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.649,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.97299999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.173,"15min":1.085},"min":{"1min":0.83499999999999996,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.5249999999999999,"5min":1.619,"15min":1.619},"last":1.097},{"interface":"front","average":{"1min":1.391,"5min":1.3100000000000001,"15min":1.23},"min":{"1min":1.0289999999999999,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2370000000000001,"15min":1.1659999999999999},"min":{"1min":0.78500000000000003,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6699999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.403,"15min":1.335},"min":{"1min":1.083,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.1030000000000002,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":0.89200000000000002}]},{"osd":4,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.325,"15min":1.274},"min":{"1min":0.85099999999999998,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7869999999999999,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.3340000000000001,"15min":1.2909999999999999},"min":{"1min":0.61399999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6579999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.4530000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.4690000000000001,"15min":1.3500000000000001},"min":{"1min":0.95599999999999996,"5min":0.67700000000000005,"15min":0.52700000000000002},"max":{"1min":2.2389999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.4490000000000001,"15min":1.381},"min":{"1min":0.92500000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.909,"5min":2.0179999999999998,"15min":2.0739999999999998},"last":1.1719999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.605,"5min":1.5489999999999999,"15min":1.446},"min":{"1min":1.23,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.9610000000000001,"5min":2.0859999999999999,"15min":2.1720000000000002},"last":1.335},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.498,"15min":1.407},"min":{"1min":0.753,"5min":0.71199999999999997,"15min":0.61399999999999999},"max":{"1min":1.8620000000000001,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.522}]}]},{"osd":6,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":656,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194656,"statfs":{"total":200000000,"available":199327868,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.1499999999999999,"15min":1.1539999999999999},"min":{"1min":0.78000000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.597,"5min":1.804,"15min":1.804},"last":1.3100000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.202,"15min":1.149},"min":{"1min":0.71699999999999997,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.885,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.0940000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2350000000000001,"15min":1.1970000000000001},"min":{"1min":0.73099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":1.014},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.222,"15min":1.1299999999999999},"min":{"1min":0.79300000000000004,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.641}]},{"osd":2,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1830000000000001,"15min":1.111},"min":{"1min":0.78200000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.573,"5min":1.708,"15min":1.708},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.409,"15min":1.22},"min":{"1min":1.0429999999999999,"5min":0.73299999999999998,"15min":0.66200000000000003},"max":{"1min":1.7410000000000001,"5min":13.193,"15min":13.193},"last":1.1739999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.3380000000000001,"15min":1.234},"min":{"1min":0.85499999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.8240000000000001,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.2769999999999999,"15min":1.2569999999999999},"min":{"1min":0.94099999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.6619999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.379}]},{"osd":4,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3480000000000001,"15min":1.254},"min":{"1min":0.80400000000000005,"5min":0.68500000000000005,"15min":0.55300000000000005},"max":{"1min":1.881,"5min":2.484,"15min":2.484},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.411,"15min":1.3260000000000001},"min":{"1min":0.97599999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.016,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.1539999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.335,"15min":1.282},"min":{"1min":0.997,"5min":0.624,"15min":0.59599999999999997},"max":{"1min":1.742,"5min":2.0049999999999999,"15min":2.0049999999999999},"last":1.054},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.506,"15min":1.381},"min":{"1min":1.0980000000000001,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.978,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.556}]},{"osd":7,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.5589999999999999,"15min":1.4970000000000001},"min":{"1min":1.169,"5min":0.78200000000000003,"15min":0.622},"max":{"1min":1.9590000000000001,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.29},{"interface":"front","average":{"1min":1.526,"5min":1.484,"15min":1.476},"min":{"1min":1.159,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":2.1890000000000001,"15min":2.1890000000000001},"last":1.4510000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1231,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194081,"statfs":{"total":200000000,"available":198739236,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1950000000000001,"15min":1.1479999999999999},"min":{"1min":0.624,"5min":0.497,"15min":0.497},"max":{"1min":1.738,"5min":2.081,"15min":2.081},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.2170000000000001,"15min":1.2070000000000001},"min":{"1min":0.754,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.694,"5min":1.974,"15min":1.974},"last":1.365}]},{"osd":1,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1299999999999999,"15min":1.1899999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.851,"5min":1.851,"15min":2.1779999999999999},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.194,"15min":1.123},"min":{"1min":0.90500000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8640000000000001,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.321}]},{"osd":2,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.25,"15min":1.1899999999999999},"min":{"1min":0.92500000000000004,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.6850000000000001,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.484},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.258,"15min":1.2210000000000001},"min":{"1min":0.97799999999999998,"5min":0.79300000000000004,"15min":0.72599999999999998},"max":{"1min":1.778,"5min":1.8049999999999999,"15min":1.962},"last":1.6890000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.395,"15min":1.3120000000000001},"min":{"1min":0.85499999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6080000000000001,"5min":17.356999999999999,"15min":17.356999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.337,"15min":1.2889999999999999},"min":{"1min":1.159,"5min":0.747,"15min":0.73499999999999999},"max":{"1min":1.7549999999999999,"5min":1.7949999999999999,"15min":1.9650000000000001},"last":1.7769999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3660000000000001,"15min":1.3069999999999999},"min":{"1min":0.88,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.9530000000000001,"5min":2.625,"15min":2.625},"last":1.157},{"interface":"front","average":{"1min":1.423,"5min":1.355,"15min":1.375},"min":{"1min":1.036,"5min":0.72299999999999998,"15min":0.58799999999999997},"max":{"1min":1.7609999999999999,"5min":2.0270000000000001,"15min":2.2789999999999999},"last":0.998}]},{"osd":6,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.494,"5min":1.4550000000000001,"15min":1.4370000000000001},"min":{"1min":1.0069999999999999,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.8180000000000001,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.528,"5min":1.5429999999999999,"15min":1.427},"min":{"1min":1.2090000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.671,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.2569999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:44:48 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4850000000000001,"15min":1.4470000000000001},"min":{"1min":1.1819999999999999,"5min":0.63,"15min":0.63},"max":{"1min":1.819,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.103},{"interface":"front","average":{"1min":1.635,"5min":1.599,"15min":1.583},"min":{"1min":1.4490000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.913,"5min":2.0790000000000002,"15min":2.3700000000000001},"last":1.899}]}]},{"osd":4,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329604,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.214,"15min":1.1619999999999999},"min":{"1min":0.96199999999999997,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.569,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3200000000000001,"15min":1.1659999999999999},"min":{"1min":0.95399999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.976,"5min":7.367,"15min":7.367},"last":1.1870000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.147,"15min":1.1259999999999999},"min":{"1min":0.73799999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"max":{"1min":1.5549999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.554},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.383},"min":{"1min":0.82899999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.744,"5min":1.9379999999999999,"15min":8.5850000000000009},"last":1.355}]},{"osd":2,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.179,"15min":1.292},"min":{"1min":0.80400000000000005,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":4.1669999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.274,"15min":1.177},"min":{"1min":0.85799999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.649,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.92600000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2949999999999999,"15min":1.2310000000000001},"min":{"1min":0.91700000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.5880000000000001,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.367,"15min":1.3720000000000001},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":1.8160000000000001,"5min":1.911,"15min":1.911},"last":1.2689999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2729999999999999,"15min":1.2869999999999999},"min":{"1min":0.88,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.7390000000000001,"5min":2.0640000000000001,"15min":2.4420000000000002},"last":1.488},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.357,"15min":1.258},"min":{"1min":0.94899999999999995,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.8049999999999999,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.605}]},{"osd":6,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4239999999999999,"15min":1.4359999999999999},"min":{"1min":1.026,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.8560000000000001,"5min":2.0219999999999998,"15min":2.863},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.524,"5min":1.5389999999999999,"15min":1.5569999999999999},"min":{"1min":1.0780000000000001,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.913,"5min":2.1339999999999999,"15min":4.0229999999999997},"last":1.764}]},{"osd":7,"last update":"Fri Aug 25 09:44:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.5780000000000001,"15min":1.427},"min":{"1min":0.98499999999999999,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0649999999999999,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.617,"5min":1.597,"15min":1.5960000000000001},"min":{"1min":1.2849999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.8680000000000001,"5min":2.1230000000000002,"15min":2.7930000000000001},"last":1.9319999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":656,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194656,"statfs":{"total":200000000,"available":199327880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.89700000000000002,"15min":0.85299999999999998},"min":{"1min":0.54100000000000004,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.51,"5min":1.51,"15min":1.6830000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.99299999999999999,"15min":0.91100000000000003},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.80900000000000005},"min":{"1min":0.58099999999999996,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.2869999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.745},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.022,"15min":0.98299999999999998},"min":{"1min":0.54000000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.421,"5min":1.722,"15min":2.3399999999999999},"last":0.96599999999999997}]},{"osd":2,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.0009999999999999,"15min":0.88300000000000001},"min":{"1min":0.51600000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.6839999999999999,"5min":1.919,"15min":1.919},"last":0.57999999999999996},{"interface":"front","average":{"1min":1.21,"5min":1.1220000000000001,"15min":1.1140000000000001},"min":{"1min":0.61199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.708,"5min":1.9710000000000001,"15min":3.399},"last":1.302}]},{"osd":4,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.2370000000000001,"15min":1.1970000000000001},"min":{"1min":0.79400000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.238,"15min":1.2010000000000001},"min":{"1min":0.85199999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6060000000000001,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":0.86099999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.327,"15min":1.3129999999999999},"min":{"1min":0.73599999999999999,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.077,"5min":2.077,"15min":2.4060000000000001},"last":1.159},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.306,"15min":1.276},"min":{"1min":0.88,"5min":0.81499999999999995,"15min":0.64500000000000002},"max":{"1min":1.8100000000000001,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.115}]},{"osd":6,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.302,"15min":1.248},"min":{"1min":0.91000000000000003,"5min":0.74099999999999999,"15min":0.73799999999999999},"max":{"1min":1.6890000000000001,"5min":1.917,"15min":1.917},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.407,"15min":1.409},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":3.496},"last":1.2490000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:44:52 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.4159999999999999,"15min":1.3700000000000001},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.899,"5min":2.044,"15min":2.2989999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4299999999999999,"15min":1.306},"min":{"1min":1.1200000000000001,"5min":0.88100000000000001,"15min":0.68400000000000005},"max":{"1min":2.0179999999999998,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":0.83599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574928,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":655,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329000,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.85399999999999998,"15min":0.83199999999999996},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.395,"5min":1.7210000000000001,"15min":2.383},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.0189999999999999,"5min":0.92500000000000004,"15min":1.0600000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":1.4430000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.98299999999999998,"15min":0.99099999999999999},"min":{"1min":0.46200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.5029999999999999,"5min":1.9350000000000001,"15min":2.3180000000000001},"last":1.228},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.04,"15min":1.0580000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.6399999999999999,"5min":1.677,"15min":2.8599999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.94799999999999995,"15min":1.0840000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.72,"5min":2.1190000000000002,"15min":4.9569999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.1080000000000001,"15min":1.1100000000000001},"min":{"1min":0.497,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.145,"5min":2.145,"15min":2.2040000000000002},"last":1.056}]},{"osd":4,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.2989999999999999,"15min":1.3939999999999999},"min":{"1min":0.83599999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":2.2919999999999998,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":1.508},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.2829999999999999,"15min":1.339},"min":{"1min":0.72199999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.7639999999999998},"last":1.115}]},{"osd":5,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.236,"15min":1.365},"min":{"1min":0.76100000000000001,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6439999999999999,"5min":2.004,"15min":5.1269999999999998},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.373,"15min":1.3979999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.395}]},{"osd":6,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.379,"15min":1.419},"min":{"1min":0.76800000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9059999999999999,"5min":2.0800000000000001,"15min":2.6709999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.379,"15min":1.4790000000000001},"min":{"1min":1.014,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.2389999999999999,"5min":2.6749999999999998,"15min":5.0599999999999996},"last":1.28}]},{"osd":7,"last update":"Fri Aug 25 09:44:51 2023","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.3720000000000001,"15min":1.4450000000000001},"min":{"1min":0.74199999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.181,"5min":2.181,"15min":2.968},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.508,"5min":1.4119999999999999,"15min":1.502},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.2309999999999999,"5min":2.7829999999999999,"15min":5.2990000000000004},"last":1.3620000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":34,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1231,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194080,"statfs":{"total":200000000,"available":198738550,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83599999999999997,"15min":0.77800000000000002},"min":{"1min":0.36399999999999999,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":1.484,"5min":1.5700000000000001,"15min":1.627},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.91000000000000003,"15min":0.79500000000000004},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.39300000000000002},"max":{"1min":1.331,"5min":1.895,"15min":1.895},"last":1.198}]},{"osd":2,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.89300000000000002,"15min":0.80800000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0009999999999999,"15min":0.91100000000000003},"min":{"1min":0.47799999999999998,"5min":0.41899999999999998,"15min":0.33600000000000002},"max":{"1min":1.6220000000000001,"5min":1.746,"15min":1.746},"last":1.075}]},{"osd":3,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.161,"15min":1.0469999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.372},"max":{"1min":1.819,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.042,"5min":1.0349999999999999,"15min":0.94899999999999995},"min":{"1min":0.55800000000000005,"5min":0.48799999999999999,"15min":0.45000000000000001},"max":{"1min":1.5249999999999999,"5min":1.764,"15min":1.764},"last":1.5229999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.177,"15min":1.0940000000000001},"min":{"1min":0.78200000000000003,"5min":0.75,"15min":0.72499999999999998},"max":{"1min":1.4530000000000001,"5min":1.694,"15min":1.694},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.333,"15min":1.2250000000000001},"min":{"1min":0.77700000000000002,"5min":0.629,"15min":0.629},"max":{"1min":1.7210000000000001,"5min":2.008,"15min":2.008},"last":1.482}]},{"osd":5,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.256,"15min":1.194},"min":{"1min":0.751,"5min":0.751,"15min":0.68600000000000005},"max":{"1min":1.7589999999999999,"5min":1.823,"15min":1.8340000000000001},"last":1.458},{"interface":"front","average":{"1min":1.286,"5min":1.2450000000000001,"15min":1.177},"min":{"1min":0.871,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.76,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.4299999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.419,"15min":1.3109999999999999},"min":{"1min":0.88200000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.008,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.3799999999999999,"15min":1.3009999999999999},"min":{"1min":1.024,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.8700000000000001,"5min":2.004,"15min":2.004},"last":1.6060000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:44:54 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4430000000000001,"15min":1.359},"min":{"1min":0.85499999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"max":{"1min":1.968,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.4259999999999999,"15min":1.3200000000000001},"min":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.716}]}]},{"osd":0,"up_from":13,"seq":55834574929,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194657,"statfs":{"total":200000000,"available":199329534,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.85999999999999999},"min":{"1min":0.50900000000000001,"5min":0.313,"15min":0.313},"max":{"1min":1.9350000000000001,"5min":11.257,"15min":11.257},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.97899999999999998,"15min":0.96099999999999997},"min":{"1min":0.47399999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.5580000000000001,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.80300000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.92400000000000004,"15min":0.93500000000000005},"min":{"1min":0.63500000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.724,"5min":1.724,"15min":1.724},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.0289999999999999,"5min":0.98099999999999998,"15min":1.087},"min":{"1min":0.59999999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.89400000000000002,"15min":0.90300000000000002},"min":{"1min":0.46200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.625},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.073,"15min":1.117},"min":{"1min":0.57499999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.5860000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2370000000000001,"15min":1.238},"min":{"1min":0.72899999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.7669999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3420000000000001,"15min":1.3560000000000001},"min":{"1min":0.95199999999999996,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.659}]},{"osd":5,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.2350000000000001,"15min":1.351},"min":{"1min":0.754,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.798},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.2470000000000001,"15min":1.2989999999999999},"min":{"1min":0.875,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.3109999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.3049999999999999,"15min":1.3440000000000001},"min":{"1min":0.81999999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.278},{"interface":"front","average":{"1min":1.359,"5min":1.2969999999999999,"15min":1.2989999999999999},"min":{"1min":0.79800000000000004,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":1.698}]},{"osd":7,"last update":"Fri Aug 25 09:44:57 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.379,"15min":1.401},"min":{"1min":0.91500000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.5149999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.3120000000000001,"15min":1.323},"min":{"1min":0.81599999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.7410000000000001,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":2.1469999999999998}]}]}],"pool_statfs":[]}} 2023-08-25T09:45:12.256 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:45:12.257 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:45:12.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.510+0000 7f090707c640 1 Processor -- start 2023-08-25T09:45:12.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.510+0000 7f090707c640 1 -- start start 2023-08-25T09:45:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f090707c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f090707c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09000cdc20 0x7f0900069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f090707c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09000ce060 0x7f0900069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f090707c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09000cc5f0 con 0x7f09000ce060 2023-08-25T09:45:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f090707c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0900070650 con 0x7f09000cdc20 2023-08-25T09:45:12.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f090707c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0900070930 con 0x7f090015ae50 2023-08-25T09:45:12.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f0905619640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09000ce060 0x7f0900069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f0905619640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f09000ce060 0x7f0900069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52982/0 (socket says 172.21.15.70:52982) 2023-08-25T09:45:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f0905619640 1 -- 172.21.15.70:0/3962823960 learned_addr learned my addr 172.21.15.70:0/3962823960 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:12.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f08fffff640 1 --2- 172.21.15.70:0/3962823960 >> v2:172.21.15.155:3300/0 conn(0x7f09000cdc20 0x7f0900069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.512+0000 7f0904e18640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.513+0000 7f0905619640 1 -- 172.21.15.70:0/3962823960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 msgr2=0x7f09000c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:12.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.513+0000 7f0905619640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.513+0000 7f0905619640 1 -- 172.21.15.70:0/3962823960 >> v2:172.21.15.155:3300/0 conn(0x7f09000cdc20 msgr2=0x7f0900069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.513+0000 7f0905619640 1 --2- 172.21.15.70:0/3962823960 >> v2:172.21.15.155:3300/0 conn(0x7f09000cdc20 0x7f0900069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.513+0000 7f0905619640 1 -- 172.21.15.70:0/3962823960 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0900070c10 con 0x7f09000ce060 2023-08-25T09:45:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.513+0000 7f0904e18640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.514+0000 7f0905619640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000ce060 0x7f0900069d90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0900071d50 tx=0x7f08f802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dff1cca378d4c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.514+0000 7f08ff7fe640 1 -- 172.21.15.70:0/3962823960 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f802cb00 con 0x7f09000ce060 2023-08-25T09:45:12.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.514+0000 7f08ff7fe640 1 -- 172.21.15.70:0/3962823960 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f802cc60 con 0x7f09000ce060 2023-08-25T09:45:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.514+0000 7f08ff7fe640 1 -- 172.21.15.70:0/3962823960 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80343f0 con 0x7f09000ce060 2023-08-25T09:45:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 -- 172.21.15.70:0/3962823960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000ce060 msgr2=0x7f0900069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000ce060 0x7f0900069d90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f0900071d50 tx=0x7f08f802bf60 comp rx=0 tx=0).stop 2023-08-25T09:45:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 -- 172.21.15.70:0/3962823960 shutdown_connections 2023-08-25T09:45:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000ce060 0x7f0900069d90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 --2- 172.21.15.70:0/3962823960 >> v2:172.21.15.155:3300/0 conn(0x7f09000cdc20 0x7f0900069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 --2- 172.21.15.70:0/3962823960 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 -- 172.21.15.70:0/3962823960 >> 172.21.15.70:0/3962823960 conn(0x7f09000cb0d0 msgr2=0x7f09000cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 -- 172.21.15.70:0/3962823960 shutdown_connections 2023-08-25T09:45:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.515+0000 7f090707c640 1 -- 172.21.15.70:0/3962823960 wait complete. 2023-08-25T09:45:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.517+0000 7f090707c640 1 Processor -- start 2023-08-25T09:45:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.517+0000 7f090707c640 1 -- start start 2023-08-25T09:45:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f090707c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 0x7f090014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f090707c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09000ce060 0x7f09000ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f090707c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f090707c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c4d30 con 0x7f09000cdc20 2023-08-25T09:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f090707c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c3dd0 con 0x7f09000ce060 2023-08-25T09:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f090707c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c40b0 con 0x7f090015ae50 2023-08-25T09:45:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f0904e18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 0x7f090014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f0904e18640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 0x7f090014fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52984/0 (socket says 172.21.15.70:52984) 2023-08-25T09:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f0904e18640 1 -- 172.21.15.70:0/1260115112 learned_addr learned my addr 172.21.15.70:0/1260115112 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f0904e18640 1 -- 172.21.15.70:0/1260115112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 msgr2=0x7f09000ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f0904e18640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.518+0000 7f0904e18640 1 -- 172.21.15.70:0/1260115112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09000ce060 msgr2=0x7f09000ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f0904e18640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09000ce060 0x7f09000ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f0904e18640 1 -- 172.21.15.70:0/1260115112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8009b30 con 0x7f09000cdc20 2023-08-25T09:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f0904e18640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 0x7f090014fdf0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c7f0 tx=0x7f08f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4007a30 con 0x7f09000cdc20 2023-08-25T09:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f4007b90 con 0x7f09000cdc20 2023-08-25T09:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f400ce40 con 0x7f09000cdc20 2023-08-25T09:45:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09000c4390 con 0x7f09000cdc20 2023-08-25T09:45:12.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.519+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09000c4950 con 0x7f09000cdc20 2023-08-25T09:45:12.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.520+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0900066fb0 con 0x7f09000cdc20 2023-08-25T09:45:12.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.521+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08f40113f0 con 0x7f09000cdc20 2023-08-25T09:45:12.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.523+0000 7f08fd7fa640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a080 0x7f08e406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.523+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08f40885c0 con 0x7f09000cdc20 2023-08-25T09:45:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.523+0000 7f08fffff640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a080 0x7f08e406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.524+0000 7f08fffff640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a080 0x7f08e406c540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006fd0 tx=0x7f08ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.528+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08f4053d40 con 0x7f09000cdc20 2023-08-25T09:45:12.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.785+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f09000ae530 con 0x7f09000cdc20 2023-08-25T09:45:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.792+0000 7f08fd7fa640 1 -- 172.21.15.70:0/1260115112 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f08f4057360 con 0x7f09000cdc20 2023-08-25T09:45:12.795 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:45:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:45:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:45:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:45:12.795 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:45:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:45:12.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.794+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a080 msgr2=0x7f08e406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.794+0000 7f090707c640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a080 0x7f08e406c540 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006fd0 tx=0x7f08ec008040 comp rx=0 tx=0).stop 2023-08-25T09:45:12.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.794+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 msgr2=0x7f090014fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:12.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.794+0000 7f090707c640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 0x7f090014fdf0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c7f0 tx=0x7f08f400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.794+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 shutdown_connections 2023-08-25T09:45:12.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.794+0000 7f090707c640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a080 0x7f08e406c540 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.795+0000 7f090707c640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f090015ae50 0x7f09000ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.795+0000 7f090707c640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09000ce060 0x7f09000ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.795+0000 7f090707c640 1 --2- 172.21.15.70:0/1260115112 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09000cdc20 0x7f090014fdf0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:12.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.795+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 >> 172.21.15.70:0/1260115112 conn(0x7f09000cb0d0 msgr2=0x7f09000af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:12.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.795+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 shutdown_connections 2023-08-25T09:45:12.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:12.795+0000 7f090707c640 1 -- 172.21.15.70:0/1260115112 wait complete. 2023-08-25T09:45:12.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.024+0000 7f591d2ee640 1 Processor -- start 2023-08-25T09:45:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.025+0000 7f591d2ee640 1 -- start start 2023-08-25T09:45:13.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.026+0000 7f591d2ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f59180ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.026+0000 7f591d2ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59180ac870 0x7f59180ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.026+0000 7f591d2ee640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591806f050 con 0x7f59180b3880 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.026+0000 7f591d2ee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591806f330 con 0x7f59180ac870 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.026+0000 7f591d2ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591806f610 con 0x7f591815cb40 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59180ac870 0x7f59180ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59180ac870 0x7f59180ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52990/0 (socket says 172.21.15.70:52990) 2023-08-25T09:45:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 -- 172.21.15.70:0/291064511 learned_addr learned my addr 172.21.15.70:0/291064511 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f59177fe640 1 --1- 172.21.15.70:0/291064511 >> v1:172.21.15.70:6789/0 conn(0x7f59180b3880 0x7f5918160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37670/0 (socket says 172.21.15.70:37670) 2023-08-25T09:45:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f5916ffd640 1 --2- 172.21.15.70:0/291064511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f59180ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 -- 172.21.15.70:0/291064511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 msgr2=0x7f59180ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 --2- 172.21.15.70:0/291064511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f59180ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 -- 172.21.15.70:0/291064511 >> v1:172.21.15.70:6789/0 conn(0x7f59180b3880 legacy=0x7f5918160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.027+0000 7f590ffff640 1 -- 172.21.15.70:0/291064511 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591806f8f0 con 0x7f59180ac870 2023-08-25T09:45:13.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.028+0000 7f590ffff640 1 --2- 172.21.15.70:0/291064511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f59180ce860 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5900009870 tx=0x7f590002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf2768b90c0583a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.029+0000 7f59167fc640 1 -- 172.21.15.70:0/291064511 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900030840 con 0x7f59180ac870 2023-08-25T09:45:13.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.029+0000 7f59167fc640 1 -- 172.21.15.70:0/291064511 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59000309a0 con 0x7f59180ac870 2023-08-25T09:45:13.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.029+0000 7f59167fc640 1 -- 172.21.15.70:0/291064511 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5900030d30 con 0x7f59180ac870 2023-08-25T09:45:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.029+0000 7f591d2ee640 1 -- 172.21.15.70:0/291064511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 msgr2=0x7f59180ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.029+0000 7f591d2ee640 1 --2- 172.21.15.70:0/291064511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f59180ce860 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f5900009870 tx=0x7f590002b980 comp rx=0 tx=0).stop 2023-08-25T09:45:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.030+0000 7f591d2ee640 1 -- 172.21.15.70:0/291064511 shutdown_connections 2023-08-25T09:45:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.030+0000 7f591d2ee640 1 --2- 172.21.15.70:0/291064511 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f59180ce860 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.030+0000 7f591d2ee640 1 --2- 172.21.15.70:0/291064511 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f59180ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.030+0000 7f591d2ee640 1 -- 172.21.15.70:0/291064511 >> 172.21.15.70:0/291064511 conn(0x7f5918069850 msgr2=0x7f5918069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.030+0000 7f591d2ee640 1 -- 172.21.15.70:0/291064511 shutdown_connections 2023-08-25T09:45:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.030+0000 7f591d2ee640 1 -- 172.21.15.70:0/291064511 wait complete. 2023-08-25T09:45:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.031+0000 7f591d2ee640 1 Processor -- start 2023-08-25T09:45:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.031+0000 7f591d2ee640 1 -- start start 2023-08-25T09:45:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.032+0000 7f591d2ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f591815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.032+0000 7f591d2ee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59180b3880 0x7f5918146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.032+0000 7f591d2ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f5918147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.032+0000 7f591d2ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59181520b0 con 0x7f59180ac870 2023-08-25T09:45:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.032+0000 7f591d2ee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918154150 con 0x7f59180b3880 2023-08-25T09:45:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.032+0000 7f591d2ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59181522c0 con 0x7f591815cb40 2023-08-25T09:45:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f591815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f591815b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52996/0 (socket says 172.21.15.70:52996) 2023-08-25T09:45:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 -- 172.21.15.70:0/2319417344 learned_addr learned my addr 172.21.15.70:0/2319417344 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 -- 172.21.15.70:0/2319417344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 msgr2=0x7f5918147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f5918147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 -- 172.21.15.70:0/2319417344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59180b3880 msgr2=0x7f5918146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59180b3880 0x7f5918146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 -- 172.21.15.70:0/2319417344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000094d0 con 0x7f59180ac870 2023-08-25T09:45:13.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f5916ffd640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f591815b8e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f590800d800 tx=0x7f590800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.033+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800b6c0 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.034+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590800b820 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.034+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590800bbb0 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.034+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59181525a0 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.034+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591815be20 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.035+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591815bfb0 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.036+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f590800bd10 con 0x7f59180ac870 2023-08-25T09:45:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.037+0000 7f590f7fe640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58f806a050 0x7f58f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.037+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5908088bc0 con 0x7f59180ac870 2023-08-25T09:45:13.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.037+0000 7f590ffff640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58f806a050 0x7f58f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.038+0000 7f590ffff640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58f806a050 0x7f58f806c510 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5900036000 tx=0x7f590003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.042+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5908054210 con 0x7f59180ac870 2023-08-25T09:45:13.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.268+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f591804e420 con 0x7f59180ac870 2023-08-25T09:45:13.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.269+0000 7f590f7fe640 1 -- 172.21.15.70:0/2319417344 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f5908057830 con 0x7f59180ac870 2023-08-25T09:45:13.272 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:13.272 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:13.272 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:13.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58f806a050 msgr2=0x7f58f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58f806a050 0x7f58f806c510 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f5900036000 tx=0x7f590003b040 comp rx=0 tx=0).stop 2023-08-25T09:45:13.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 msgr2=0x7f591815b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f591815b8e0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f590800d800 tx=0x7f590800dcc0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 shutdown_connections 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f58f806a050 0x7f58f806c510 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f591815cb40 0x7f5918147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59180b3880 0x7f5918146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 --2- 172.21.15.70:0/2319417344 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59180ac870 0x7f591815b8e0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 >> 172.21.15.70:0/2319417344 conn(0x7f5918069850 msgr2=0x7f5918160b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 shutdown_connections 2023-08-25T09:45:13.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.272+0000 7f591d2ee640 1 -- 172.21.15.70:0/2319417344 wait complete. 2023-08-25T09:45:13.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:45:06.477747+0000', 'modified': '2023-08-25T09:45:10.342540+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7227}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7227': {'gid': 7227, 'name': 'c', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6837/2920920514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6835', 'nonce': 2920920514}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 2920920514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:45:13.695 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:45:13.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-25T09:45:13.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.920+0000 7f3e8b6b8640 1 Processor -- start 2023-08-25T09:45:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.920+0000 7f3e8b6b8640 1 -- start start 2023-08-25T09:45:13.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.922+0000 7f3e8b6b8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e840b4400 0x7f3e840cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.922+0000 7f3e8b6b8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e840b12b0 con 0x7f3e8415acd0 2023-08-25T09:45:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.922+0000 7f3e8b6b8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8406ae20 con 0x7f3e840ca780 2023-08-25T09:45:13.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.922+0000 7f3e8b6b8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e840b47e0 con 0x7f3e840b4400 2023-08-25T09:45:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e89c55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3e840b4400 0x7f3e840cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e89454640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3e8415acd0 0x7f3e8415b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37682/0 (socket says 172.21.15.70:37682) 2023-08-25T09:45:13.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e89454640 1 -- 172.21.15.70:0/495974463 learned_addr learned my addr 172.21.15.70:0/495974463 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:13.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e89c55640 1 -- 172.21.15.70:0/495974463 >> v1:172.21.15.155:6789/0 conn(0x7f3e840ca780 legacy=0x7f3e840b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e89c55640 1 -- 172.21.15.70:0/495974463 >> v1:172.21.15.70:6789/0 conn(0x7f3e8415acd0 legacy=0x7f3e8415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e89c55640 1 -- 172.21.15.70:0/495974463 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8415b7c0 con 0x7f3e840b4400 2023-08-25T09:45:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.923+0000 7f3e7bfff640 1 -- 172.21.15.70:0/495974463 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496978476 0 0) 0x7f3e840b12b0 con 0x7f3e8415acd0 2023-08-25T09:45:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.924+0000 7f3e89c55640 1 --2- 172.21.15.70:0/495974463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840b4400 0x7f3e840cac30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009d90 tx=0x7f3e7c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee85bb3361773e30 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.924+0000 7f3e7bfff640 1 -- 172.21.15.70:0/495974463 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02cbd0 con 0x7f3e840b4400 2023-08-25T09:45:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.924+0000 7f3e7bfff640 1 -- 172.21.15.70:0/495974463 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c02cd30 con 0x7f3e840b4400 2023-08-25T09:45:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.924+0000 7f3e7bfff640 1 -- 172.21.15.70:0/495974463 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02cea0 con 0x7f3e840b4400 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/495974463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840b4400 msgr2=0x7f3e840cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/495974463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840b4400 0x7f3e840cac30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009d90 tx=0x7f3e7c02be20 comp rx=0 tx=0).stop 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/495974463 shutdown_connections 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/495974463 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840b4400 0x7f3e840cac30 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/495974463 >> 172.21.15.70:0/495974463 conn(0x7f3e840ce760 msgr2=0x7f3e840ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/495974463 shutdown_connections 2023-08-25T09:45:13.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.925+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/495974463 wait complete. 2023-08-25T09:45:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.927+0000 7f3e8b6b8640 1 Processor -- start 2023-08-25T09:45:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.927+0000 7f3e8b6b8640 1 -- start start 2023-08-25T09:45:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.928+0000 7f3e8b6b8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e840b4400 0x7f3e84146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.928+0000 7f3e8b6b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 0x7f3e840c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.928+0000 7f3e8b6b8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8415acd0 0x7f3e840c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.928+0000 7f3e8b6b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e84146f40 con 0x7f3e840ca780 2023-08-25T09:45:13.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.928+0000 7f3e8b6b8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840cd670 con 0x7f3e8415acd0 2023-08-25T09:45:13.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.928+0000 7f3e8b6b8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840cca60 con 0x7f3e840b4400 2023-08-25T09:45:13.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 0x7f3e840c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 0x7f3e840c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53002/0 (socket says 172.21.15.70:53002) 2023-08-25T09:45:13.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 -- 172.21.15.70:0/326034762 learned_addr learned my addr 172.21.15.70:0/326034762 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e89c55640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8415acd0 0x7f3e840c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 -- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e840b4400 msgr2=0x7f3e84146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e840b4400 0x7f3e84146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 -- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8415acd0 msgr2=0x7f3e840c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8415acd0 0x7f3e840c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 -- 172.21.15.70:0/326034762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c009a70 con 0x7f3e840ca780 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.929+0000 7f3e88c53640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 0x7f3e840c5ef0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c460 tx=0x7f3e7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.930+0000 7f3e79ffb640 1 -- 172.21.15.70:0/326034762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74012830 con 0x7f3e840ca780 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.930+0000 7f3e79ffb640 1 -- 172.21.15.70:0/326034762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e740129d0 con 0x7f3e840ca780 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.930+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e840ccd40 con 0x7f3e840ca780 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.930+0000 7f3e79ffb640 1 -- 172.21.15.70:0/326034762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74012d80 con 0x7f3e840ca780 2023-08-25T09:45:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.930+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e840cd300 con 0x7f3e840ca780 2023-08-25T09:45:13.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.932+0000 7f3e79ffb640 1 -- 172.21.15.70:0/326034762 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e7401a3b0 con 0x7f3e840ca780 2023-08-25T09:45:13.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.933+0000 7f3e79ffb640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e6806a050 0x7f3e6806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.933+0000 7f3e79ffb640 1 -- 172.21.15.70:0/326034762 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e74088370 con 0x7f3e840ca780 2023-08-25T09:45:13.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.933+0000 7f3e89454640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e6806a050 0x7f3e6806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.934+0000 7f3e8e232740 1 Processor -- start 2023-08-25T09:45:13.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.934+0000 7f3e8e232740 1 -- start start 2023-08-25T09:45:13.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.934+0000 7f3e89454640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e6806a050 0x7f3e6806c510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001570 tx=0x7f3e80007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340ad150 0x5621340ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 0x5621340aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562134080cd0 0x5621340aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e8e232740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133ff0190 con 0x56213407d130 2023-08-25T09:45:13.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133fed880 con 0x562134080cd0 2023-08-25T09:45:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x562133fe9170 con 0x5621340ad150 2023-08-25T09:45:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e88c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 0x5621340aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e88c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 0x5621340aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53018/0 (socket says 172.21.15.70:53018) 2023-08-25T09:45:13.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.935+0000 7f3e88c53640 1 -- 172.21.15.70:0/3132301929 learned_addr learned my addr 172.21.15.70:0/3132301929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e89454640 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340ad150 0x5621340ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e88c53640 1 -- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340ad150 msgr2=0x5621340ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e88c53640 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340ad150 0x5621340ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e88c53640 1 -- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562134080cd0 msgr2=0x5621340aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e88c53640 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562134080cd0 0x5621340aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e88c53640 1 -- 172.21.15.70:0/3132301929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562133fe77f0 con 0x56213407d130 2023-08-25T09:45:13.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e88c53640 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 0x5621340aa3c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3e7405ee60 tx=0x7f3e7405b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4477b72b0f483b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e6effd640 1 -- 172.21.15.70:0/3132301929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74055990 con 0x56213407d130 2023-08-25T09:45:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e6effd640 1 -- 172.21.15.70:0/3132301929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7405b0d0 con 0x56213407d130 2023-08-25T09:45:13.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.936+0000 7f3e6effd640 1 -- 172.21.15.70:0/3132301929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74053d00 con 0x56213407d130 2023-08-25T09:45:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 -- 172.21.15.70:0/3132301929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 msgr2=0x5621340aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 0x5621340aa3c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3e7405ee60 tx=0x7f3e7405b3b0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 -- 172.21.15.70:0/3132301929 shutdown_connections 2023-08-25T09:45:13.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x562134080cd0 0x5621340aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56213407d130 0x5621340aa3c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3132301929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340ad150 0x5621340ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 -- 172.21.15.70:0/3132301929 >> 172.21.15.70:0/3132301929 conn(0x56213407f3e0 msgr2=0x5621340c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 -- 172.21.15.70:0/3132301929 shutdown_connections 2023-08-25T09:45:13.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.937+0000 7f3e8e232740 1 -- 172.21.15.70:0/3132301929 wait complete. 2023-08-25T09:45:13.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.938+0000 7f3e8e232740 1 -- start start 2023-08-25T09:45:13.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.938+0000 7f3e8e232740 1 Processor -- start 2023-08-25T09:45:13.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x5621340b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133f88d20 con 0x5621340a5280 2023-08-25T09:45:13.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133f889c0 con 0x5621340ad150 2023-08-25T09:45:13.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x562133d4ef20 con 0x562134080cd0 2023-08-25T09:45:13.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e8e232740 10 client.0 _ll_register_callbacks cb 0x562133fa0630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:45:13.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e89c55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e89c55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340936e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53026/0 (socket says 172.21.15.70:53026) 2023-08-25T09:45:13.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.939+0000 7f3e89c55640 1 -- 172.21.15.70:0/2278333634 learned_addr learned my addr 172.21.15.70:0/2278333634 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e88c53640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89454640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x5621340b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89c55640 1 -- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 msgr2=0x5621340b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89c55640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x5621340b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89c55640 1 -- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 msgr2=0x5621340a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89c55640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89c55640 1 -- 172.21.15.70:0/2278333634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e840cd300 con 0x5621340a5280 2023-08-25T09:45:13.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e89c55640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340936e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c02c430 tx=0x7f3e7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78d56bba2716d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e63fff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:45:13.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e63fff640 1 -- 172.21.15.70:0/2278333634 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c02cea0 con 0x5621340a5280 2023-08-25T09:45:13.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562133fbd540 con 0x5621340a5280 2023-08-25T09:45:13.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.940+0000 7f3e63fff640 1 -- 172.21.15.70:0/2278333634 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e7c034c80 con 0x5621340a5280 2023-08-25T09:45:13.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e63fff640 1 -- 172.21.15.70:0/2278333634 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e7c034e40 con 0x5621340a5280 2023-08-25T09:45:13.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 10 client.7389 fetch_fsmap learned FSMap version 49 2023-08-25T09:45:13.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x562133fe0f30 con 0x5621340a5280 2023-08-25T09:45:13.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e63fff640 1 -- 172.21.15.70:0/2278333634 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562133fbd540 con 0x5621340a5280 2023-08-25T09:45:13.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e63fff640 1 -- 172.21.15.70:0/2278333634 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e7c03f480 con 0x5621340a5280 2023-08-25T09:45:13.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 10 client.7389 fetch_fsmap finished waiting for FSMap version 49 2023-08-25T09:45:13.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 10 client.7389 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:45:13.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 20 client.7389 populate_metadata read hostname 'smithi070' 2023-08-25T09:45:13.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621340a6af0 0x5621340a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 4 client.7389 mds_command: new command op to 7227 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:45:13.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.941+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 --> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562133fe2150 con 0x5621340a6af0 2023-08-25T09:45:13.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.942+0000 7f3e89454640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621340a6af0 0x5621340a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.942+0000 7f3e89454640 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621340a6af0 0x5621340a6ed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b8efe218362f4e1 server_cookie=6db9f03dddb0a32 in_seq=0 out_seq=0 2023-08-25T09:45:13.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.942+0000 7f3e63fff640 10 client.7389 ms_handle_connect on v2:172.21.15.70:6835/2920920514 2023-08-25T09:45:13.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e63fff640 1 -- 172.21.15.70:0/2278333634 <== mds.0 v2:172.21.15.70:6835/2920920514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562133fe2150 con 0x5621340a6af0 2023-08-25T09:45:13.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e63fff640 10 client.7389 handle_command_reply: tid=0 2023-08-25T09:45:13.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 client.7389 shutdown 2023-08-25T09:45:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 msgr2=0x5621340936e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340936e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c02c430 tx=0x7f3e7c004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 shutdown_connections 2023-08-25T09:45:13.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340936e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621340a6af0 0x5621340a6ed0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 --2- 172.21.15.70:0/2278333634 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x5621340b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 >> 172.21.15.70:0/2278333634 conn(0x56213407d130 msgr2=0x5621340b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e63fff640 0 client.7389 ms_handle_reset on v2:172.21.15.70:6835/2920920514 2023-08-25T09:45:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 shutdown_connections 2023-08-25T09:45:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.943+0000 7f3e8e232740 1 -- 172.21.15.70:0/2278333634 wait complete. 2023-08-25T09:45:13.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.944+0000 7f3e8e232740 20 client.7389 trim_cache size 0 max 16384 2023-08-25T09:45:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.960+0000 7f3e8e232740 1 Processor -- start 2023-08-25T09:45:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.960+0000 7f3e8e232740 1 -- start start 2023-08-25T09:45:13.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 0x5621340a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340a5280 0x5621340cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e8e232740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562134093790 con 0x562134080cd0 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621340913a0 con 0x5621340ad150 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621340792e0 con 0x5621340a5280 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e89454640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 0x5621340a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e89454640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 0x5621340a6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53036/0 (socket says 172.21.15.70:53036) 2023-08-25T09:45:13.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.961+0000 7f3e89454640 1 -- 172.21.15.70:0/65063798 learned_addr learned my addr 172.21.15.70:0/65063798 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e89454640 1 -- 172.21.15.70:0/65063798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340a5280 msgr2=0x5621340cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e89454640 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340a5280 0x5621340cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e89454640 1 -- 172.21.15.70:0/65063798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 msgr2=0x5621340cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e89454640 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e89454640 1 -- 172.21.15.70:0/65063798 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562133f87a10 con 0x562134080cd0 2023-08-25T09:45:13.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e89454640 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 0x5621340a6ed0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x5621340db860 tx=0x7f3e80010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ecfb8ab9631ff server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e6f7fe640 1 -- 172.21.15.70:0/65063798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e800057a0 con 0x562134080cd0 2023-08-25T09:45:13.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e6f7fe640 1 -- 172.21.15.70:0/65063798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8001a040 con 0x562134080cd0 2023-08-25T09:45:13.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.962+0000 7f3e6f7fe640 1 -- 172.21.15.70:0/65063798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80005b60 con 0x562134080cd0 2023-08-25T09:45:13.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 -- 172.21.15.70:0/65063798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 msgr2=0x5621340a6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 0x5621340a6ed0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x5621340db860 tx=0x7f3e80010000 comp rx=0 tx=0).stop 2023-08-25T09:45:13.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 -- 172.21.15.70:0/65063798 shutdown_connections 2023-08-25T09:45:13.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5621340a5280 0x5621340cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 --2- 172.21.15.70:0/65063798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x562134080cd0 0x5621340a6ed0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 -- 172.21.15.70:0/65063798 >> 172.21.15.70:0/65063798 conn(0x56213407d130 msgr2=0x56213407d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 -- 172.21.15.70:0/65063798 shutdown_connections 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.963+0000 7f3e8e232740 1 -- 172.21.15.70:0/65063798 wait complete. 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.964+0000 7f3e8e232740 1 -- start start 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.964+0000 7f3e8e232740 1 Processor -- start 2023-08-25T09:45:13.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x562134117250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340a8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x562134132100 con 0x5621340a5280 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621340a8e40 con 0x5621340ad150 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621340a8b80 con 0x562134080cd0 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e8e232740 10 client.0 _ll_register_callbacks cb 0x7f3e740573c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:45:13.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e88c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.965+0000 7f3e88c53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340a8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53038/0 (socket says 172.21.15.70:53038) 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 -- 172.21.15.70:0/3210422207 learned_addr learned my addr 172.21.15.70:0/3210422207 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 -- 172.21.15.70:0/3210422207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 msgr2=0x562134117250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x562134117250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 -- 172.21.15.70:0/3210422207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 msgr2=0x5621340cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 -- 172.21.15.70:0/3210422207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e800070d0 con 0x5621340a5280 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e88c53640 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340a8640 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3e74057e30 tx=0x7f3e74086570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccb8162f60b7bfe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e63fff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e63fff640 1 -- 172.21.15.70:0/3210422207 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74053930 con 0x5621340a5280 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e63fff640 1 -- 172.21.15.70:0/3210422207 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e740862e0 con 0x5621340a5280 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e7405b0d0 con 0x5621340a5280 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.966+0000 7f3e63fff640 1 -- 172.21.15.70:0/3210422207 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e740880b0 con 0x5621340a5280 2023-08-25T09:45:13.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 10 client.7395 fetch_fsmap learned FSMap version 49 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x562133facde0 con 0x5621340a5280 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e63fff640 1 -- 172.21.15.70:0/3210422207 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7405b0d0 con 0x5621340a5280 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e63fff640 1 -- 172.21.15.70:0/3210422207 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3e74053d00 con 0x5621340a5280 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 10 client.7395 fetch_fsmap finished waiting for FSMap version 49 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 10 client.7395 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 20 client.7395 populate_metadata read hostname 'smithi070' 2023-08-25T09:45:13.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621341313e0 0x5621341317c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 4 client.7395 mds_command: new command op to 7227 tid=0[{"prefix": "status"}] 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 --> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562133c985e0 con 0x5621341313e0 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.967+0000 7f3e89454640 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621341313e0 0x5621341317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e89454640 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621341313e0 0x5621341317c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9101866a323b78ec server_cookie=900b08e248bed845 in_seq=0 out_seq=0 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e63fff640 10 client.7395 ms_handle_connect on v2:172.21.15.70:6835/2920920514 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e63fff640 1 -- 172.21.15.70:0/3210422207 <== mds.0 v2:172.21.15.70:6835/2920920514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x562133c985e0 con 0x5621341313e0 2023-08-25T09:45:13.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e63fff640 10 client.7395 handle_command_reply: tid=0 2023-08-25T09:45:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e8e232740 1 client.7395 shutdown 2023-08-25T09:45:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 msgr2=0x5621340a8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.968+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340a8640 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3e74057e30 tx=0x7f3e74086570 comp rx=0 tx=0).stop 2023-08-25T09:45:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 shutdown_connections 2023-08-25T09:45:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:6835/2920920514,v1:172.21.15.70:6837/2920920514] conn(0x5621341313e0 0x5621341317c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5621340ad150 0x5621340cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5621340a5280 0x5621340a8640 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 --2- 172.21.15.70:0/3210422207 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x562134080cd0 0x562134117250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e63fff640 0 client.7395 ms_handle_reset on v2:172.21.15.70:6835/2920920514 2023-08-25T09:45:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 >> 172.21.15.70:0/3210422207 conn(0x56213407d130 msgr2=0x562134082fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:13.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 shutdown_connections 2023-08-25T09:45:13.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 1 -- 172.21.15.70:0/3210422207 wait complete. 2023-08-25T09:45:13.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8e232740 20 client.7395 trim_cache size 0 max 16384 2023-08-25T09:45:13.999 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:13.999 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:13.999 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7227, 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.9779850659999996, 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 49, 2023-08-25T09:45:14.000 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 115, 2023-08-25T09:45:14.001 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 115, 2023-08-25T09:45:14.001 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.107263574999999 2023-08-25T09:45:14.001 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:14.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e6806a050 msgr2=0x7f3e6806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e6806a050 0x7f3e6806c510 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3e80001570 tx=0x7f3e80007910 comp rx=0 tx=0).stop 2023-08-25T09:45:14.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 msgr2=0x7f3e840c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.969+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 0x7f3e840c5ef0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c460 tx=0x7f3e7400c920 comp rx=0 tx=0).stop 2023-08-25T09:45:14.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 shutdown_connections 2023-08-25T09:45:14.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e6806a050 0x7f3e6806c510 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e8415acd0 0x7f3e840c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e840ca780 0x7f3e840c5ef0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 --2- 172.21.15.70:0/326034762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e840b4400 0x7f3e84146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 >> 172.21.15.70:0/326034762 conn(0x7f3e840ce760 msgr2=0x7f3e8415b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 shutdown_connections 2023-08-25T09:45:14.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:13.970+0000 7f3e8b6b8640 1 -- 172.21.15.70:0/326034762 wait complete. 2023-08-25T09:45:14.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:45:14.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.217+0000 7f1a47322640 1 Processor -- start 2023-08-25T09:45:14.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.217+0000 7f1a47322640 1 -- start start 2023-08-25T09:45:14.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a47322640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a400ce8e0 0x7f1a400cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:14.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a47322640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400cba70 con 0x7f1a400cdc80 2023-08-25T09:45:14.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a47322640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400cbd50 con 0x7f1a4006e090 2023-08-25T09:45:14.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a47322640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a400cc030 con 0x7f1a400ce8e0 2023-08-25T09:45:14.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a450be640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a400ce8e0 0x7f1a400cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:14.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a448bd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1a400cdc80 0x7f1a400cecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37698/0 (socket says 172.21.15.70:37698) 2023-08-25T09:45:14.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a448bd640 1 -- 172.21.15.70:0/2759694681 learned_addr learned my addr 172.21.15.70:0/2759694681 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:14.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.219+0000 7f1a450be640 1 -- 172.21.15.70:0/2759694681 >> v1:172.21.15.155:6789/0 conn(0x7f1a4006e090 legacy=0x7f1a40164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:14.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.220+0000 7f1a450be640 1 -- 172.21.15.70:0/2759694681 >> v1:172.21.15.70:6789/0 conn(0x7f1a400cdc80 legacy=0x7f1a400cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.220+0000 7f1a450be640 1 -- 172.21.15.70:0/2759694681 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400cc310 con 0x7f1a400ce8e0 2023-08-25T09:45:14.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.220+0000 7f1a37fff640 1 -- 172.21.15.70:0/2759694681 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491335949 0 0) 0x7f1a400cba70 con 0x7f1a400cdc80 2023-08-25T09:45:14.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.221+0000 7f1a450be640 1 --2- 172.21.15.70:0/2759694681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a400ce8e0 0x7f1a400cb470 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009790 tx=0x7f1a3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e65f249547893e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:14.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.221+0000 7f1a37fff640 1 -- 172.21.15.70:0/2759694681 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c031830 con 0x7f1a400ce8e0 2023-08-25T09:45:14.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.221+0000 7f1a37fff640 1 -- 172.21.15.70:0/2759694681 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c031990 con 0x7f1a400ce8e0 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.221+0000 7f1a37fff640 1 -- 172.21.15.70:0/2759694681 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c031d20 con 0x7f1a400ce8e0 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 -- 172.21.15.70:0/2759694681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a400ce8e0 msgr2=0x7f1a400cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 --2- 172.21.15.70:0/2759694681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a400ce8e0 0x7f1a400cb470 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009790 tx=0x7f1a3c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 -- 172.21.15.70:0/2759694681 shutdown_connections 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 --2- 172.21.15.70:0/2759694681 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a400ce8e0 0x7f1a400cb470 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 -- 172.21.15.70:0/2759694681 >> 172.21.15.70:0/2759694681 conn(0x7f1a400d0860 msgr2=0x7f1a400d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:14.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 -- 172.21.15.70:0/2759694681 shutdown_connections 2023-08-25T09:45:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.222+0000 7f1a47322640 1 -- 172.21.15.70:0/2759694681 wait complete. 2023-08-25T09:45:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.224+0000 7f1a47322640 1 Processor -- start 2023-08-25T09:45:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.224+0000 7f1a47322640 1 -- start start 2023-08-25T09:45:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a47322640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 0x7f1a400c8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:14.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a47322640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a400cdc80 0x7f1a400c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a47322640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a400ce8e0 0x7f1a400b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a47322640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400ca9b0 con 0x7f1a4006e090 2023-08-25T09:45:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a47322640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400c9910 con 0x7f1a400ce8e0 2023-08-25T09:45:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a47322640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400c9bf0 con 0x7f1a400cdc80 2023-08-25T09:45:14.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a450be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 0x7f1a400c8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a450be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 0x7f1a400c8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53050/0 (socket says 172.21.15.70:53050) 2023-08-25T09:45:14.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a450be640 1 -- 172.21.15.70:0/660753208 learned_addr learned my addr 172.21.15.70:0/660753208 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a450be640 1 -- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a400cdc80 msgr2=0x7f1a400c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a458bf640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a400ce8e0 0x7f1a400b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a450be640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a400cdc80 0x7f1a400c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.225+0000 7f1a450be640 1 -- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a400ce8e0 msgr2=0x7f1a400b09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a450be640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a400ce8e0 0x7f1a400b09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a450be640 1 -- 172.21.15.70:0/660753208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c0093a0 con 0x7f1a4006e090 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a458bf640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a400ce8e0 0x7f1a400b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a450be640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 0x7f1a400c8de0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000c00 tx=0x7f1a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c004020 con 0x7f1a4006e090 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c004180 con 0x7f1a4006e090 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c005990 con 0x7f1a4006e090 2023-08-25T09:45:14.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a400c9ed0 con 0x7f1a4006e090 2023-08-25T09:45:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.226+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a400ca490 con 0x7f1a4006e090 2023-08-25T09:45:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.227+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a400cb820 con 0x7f1a4006e090 2023-08-25T09:45:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.228+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1a3c043070 con 0x7f1a4006e090 2023-08-25T09:45:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.229+0000 7f1a35ffb640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a2406a050 0x7f1a2406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.229+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1a3c0aa400 con 0x7f1a4006e090 2023-08-25T09:45:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.230+0000 7f1a448bd640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a2406a050 0x7f1a2406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:14.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.230+0000 7f1a448bd640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a2406a050 0x7f1a2406c510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1a30007fd0 tx=0x7f1a30007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:14.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.233+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a3c075af0 con 0x7f1a4006e090 2023-08-25T09:45:14.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.461+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1a400cb820 con 0x7f1a4006e090 2023-08-25T09:45:14.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.461+0000 7f1a35ffb640 1 -- 172.21.15.70:0/660753208 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1a3c079110 con 0x7f1a4006e090 2023-08-25T09:45:14.464 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:45:14.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a2406a050 msgr2=0x7f1a2406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a2406a050 0x7f1a2406c510 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1a30007fd0 tx=0x7f1a30007920 comp rx=0 tx=0).stop 2023-08-25T09:45:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 msgr2=0x7f1a400c8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 0x7f1a400c8de0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c000c00 tx=0x7f1a3c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 shutdown_connections 2023-08-25T09:45:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a2406a050 0x7f1a2406c510 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a400ce8e0 0x7f1a400b09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a400cdc80 0x7f1a400c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 --2- 172.21.15.70:0/660753208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a4006e090 0x7f1a400c8de0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 >> 172.21.15.70:0/660753208 conn(0x7f1a400d0860 msgr2=0x7f1a400ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 shutdown_connections 2023-08-25T09:45:14.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:14.464+0000 7f1a47322640 1 -- 172.21.15.70:0/660753208 wait complete. 2023-08-25T09:45:14.488 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:45:14.488 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:45:14.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:45:14.490 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:45:14.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:14.491 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:45:14.491 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:45:14.492 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:45:14.492 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:45:14.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:14.493 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:45:14.542 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:45:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:45:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:45:14.542 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:45:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.543 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:45:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:14.543 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:45:14.544 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:45:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:45:14.544 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:45:14.544 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:14.544 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:45:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:45:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41962sec preferred_lft 41962sec 2023-08-25T09:45:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:45:14.545 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.546 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:45:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:45:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.546 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:45:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.547 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:45:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:45:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:45:14.547 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.548 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:45:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:45:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:45:14.548 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:14.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:14.550 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:45:14.634 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:45:14.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:14.639 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:45:14.687 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:45:14.688 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:45:14.688 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:45:14.688 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:45:14.688 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:45:14.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:14.689 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:45:14.689 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:45:14.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:45:14.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:45:14.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:45:14.690 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:45:14.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:14.866 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:45:14.920 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:45:14.921 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:45:14.921 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:45:14.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:14.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:45:15.010 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:45:15.010 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:45:15.011 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:15.065 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:45:15.065 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:45:15.066 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:15.118 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:45:15.214 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:45:15.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:45:15.307 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:45:15.311 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:45:15.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:15.312 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:45:15.371 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:45:15.372 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:45:15.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:45:15.463 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:45:15.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:45:15.536 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:45:15.539 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:45:15.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:15.541 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:45:15.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:45:15.581+0000 7f10f6bfc480 -1 init, newargv = 0x56233daecff0 newargc=15 2023-08-25T09:45:15.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[109945]: starting ceph client 2023-08-25T09:45:15.591 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[109945]: starting fuse 2023-08-25T09:45:15.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:45:15.625 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:45:16.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:45:16.788 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:45:16.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:45:16.878 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:45:16.881 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:45:16.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:16.883 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:45:16.937 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:45:16.938 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:45:16.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:45:16.940 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:45:16.940 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:45:16.941 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:45:16.942 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:45:16.942 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:45:16.943 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:45:16.943 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:45:16.943 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:45:16.944 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:45:16.944 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:45:16.945 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:45:16.945 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:45:16.946 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:45:16.946 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:45:16.946 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:45:16.947 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:45:16.947 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:45:16.947 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:45:16.947 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:45:16.948 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:45:16.948 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:45:16.949 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:45:16.949 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:45:16.950 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:45:16.950 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:45:16.951 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:45:17.134 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.109945.asok 2023-08-25T09:45:17.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:45:17.151 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109945.asok 2023-08-25T09:45:17.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:17.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109945.asok status 2023-08-25T09:45:17.394 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:17.394 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:45:17.394 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:45:17.395 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:45:17.395 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:45:17.395 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:45:17.395 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:45:17.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "109945", 2023-08-25T09:45:17.396 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:45:17.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:17.396 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:45:17.396 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:45:17.397 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7407, 2023-08-25T09:45:17.397 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:45:17.397 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:45:17.397 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:45:17.398 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 7407 2023-08-25T09:45:17.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:17.398 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:45:17.398 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:45:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2078528701 2023-08-25T09:45:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:17.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:17.400 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:45:17.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:17.400 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:45:17.401 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2078528701 2023-08-25T09:45:17.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:17.401 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.7407 192.168.0.1:0/2078528701", 2023-08-25T09:45:17.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2078528701", 2023-08-25T09:45:17.402 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:45:17.402 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 49, 2023-08-25T09:45:17.402 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 115, 2023-08-25T09:45:17.402 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:45:17.403 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:45:17.403 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:45:17.403 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:17.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:17.409 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:45:17.410 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:17.466 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:45:17.466 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:17.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:17.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:17.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:45:17.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.779+0000 7f2589a06640 1 Processor -- start 2023-08-25T09:45:17.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.779+0000 7f2589a06640 1 -- start start 2023-08-25T09:45:17.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.780+0000 7f2589a06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25840ce5b0 0x7f25840ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:17.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.780+0000 7f2589a06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 0x7f25840b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:17.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.780+0000 7f2589a06640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840ceed0 con 0x7f25840b2680 2023-08-25T09:45:17.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.780+0000 7f2589a06640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840b2b10 con 0x7f25840ce5b0 2023-08-25T09:45:17.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.780+0000 7f2589a06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2584159d10 con 0x7f258415aa10 2023-08-25T09:45:17.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25840ce5b0 0x7f25840ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:17.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f25840ce5b0 0x7f25840ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53058/0 (socket says 172.21.15.70:53058) 2023-08-25T09:45:17.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 -- 172.21.15.70:0/2840734966 learned_addr learned my addr 172.21.15.70:0/2840734966 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:17.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 -- 172.21.15.70:0/2840734966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 msgr2=0x7f25840b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:17.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 --2- 172.21.15.70:0/2840734966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 0x7f25840b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:17.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 -- 172.21.15.70:0/2840734966 >> v1:172.21.15.155:6789/0 conn(0x7f25840b2680 legacy=0x7f258406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:17.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.781+0000 7f2582ffd640 1 -- 172.21.15.70:0/2840734966 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258415d3f0 con 0x7f25840ce5b0 2023-08-25T09:45:17.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.782+0000 7f2582ffd640 1 --2- 172.21.15.70:0/2840734966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f25840ce990 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2574009e50 tx=0x7f257402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85f06af6be6878a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:17.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2581ffb640 1 -- 172.21.15.70:0/2840734966 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257402ca80 con 0x7f25840ce5b0 2023-08-25T09:45:17.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2581ffb640 1 -- 172.21.15.70:0/2840734966 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257402cbe0 con 0x7f25840ce5b0 2023-08-25T09:45:17.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2581ffb640 1 -- 172.21.15.70:0/2840734966 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25740343f0 con 0x7f25840ce5b0 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2589a06640 1 -- 172.21.15.70:0/2840734966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 msgr2=0x7f25840ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2589a06640 1 --2- 172.21.15.70:0/2840734966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f25840ce990 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2574009e50 tx=0x7f257402bee0 comp rx=0 tx=0).stop 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2589a06640 1 -- 172.21.15.70:0/2840734966 shutdown_connections 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2589a06640 1 --2- 172.21.15.70:0/2840734966 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 0x7f25840b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.783+0000 7f2589a06640 1 --2- 172.21.15.70:0/2840734966 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f25840ce990 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.784+0000 7f2589a06640 1 -- 172.21.15.70:0/2840734966 >> 172.21.15.70:0/2840734966 conn(0x7f2584069850 msgr2=0x7f2584069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:17.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.784+0000 7f2589a06640 1 -- 172.21.15.70:0/2840734966 shutdown_connections 2023-08-25T09:45:17.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.784+0000 7f2589a06640 1 -- 172.21.15.70:0/2840734966 wait complete. 2023-08-25T09:45:17.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.785+0000 7f2589a06640 1 Processor -- start 2023-08-25T09:45:17.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.785+0000 7f2589a06640 1 -- start start 2023-08-25T09:45:17.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.786+0000 7f2589a06640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25840b2680 0x7f25841567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:17.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.786+0000 7f2589a06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f258414dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.786+0000 7f2589a06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 0x7f25840ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.786+0000 7f2589a06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841e6a20 con 0x7f25840ce5b0 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.786+0000 7f2589a06640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25841e5d50 con 0x7f25840b2680 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.786+0000 7f2589a06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840b0ed0 con 0x7f258415aa10 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f258414dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f258414dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:53074/0 (socket says 172.21.15.70:53074) 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 -- 172.21.15.70:0/2285918815 learned_addr learned my addr 172.21.15.70:0/2285918815 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:17.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f2582ffd640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25840b2680 0x7f25841567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:17.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 -- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 msgr2=0x7f25840ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:17.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 0x7f25840ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:17.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 -- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25840b2680 msgr2=0x7f25841567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:17.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25840b2680 0x7f25841567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:17.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 -- 172.21.15.70:0/2285918815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2574009b30 con 0x7f25840ce5b0 2023-08-25T09:45:17.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.787+0000 7f25827fc640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f258414dd10 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f256c00b4a0 tx=0x7f256c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:17.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.788+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c010610 con 0x7f25840ce5b0 2023-08-25T09:45:17.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.788+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256c010770 con 0x7f25840ce5b0 2023-08-25T09:45:17.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.788+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256c010b20 con 0x7f25840ce5b0 2023-08-25T09:45:17.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.788+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25841e6590 con 0x7f25840ce5b0 2023-08-25T09:45:17.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.788+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841e6220 con 0x7f25840ce5b0 2023-08-25T09:45:17.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.789+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25841e6720 con 0x7f25840ce5b0 2023-08-25T09:45:17.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.790+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f256c010c80 con 0x7f25840ce5b0 2023-08-25T09:45:17.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.792+0000 7f257bfff640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256406a080 0x7f256406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:17.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.792+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f256c088230 con 0x7f25840ce5b0 2023-08-25T09:45:17.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.792+0000 7f2582ffd640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256406a080 0x7f256406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:17.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.793+0000 7f2582ffd640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256406a080 0x7f256406c540 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2574035000 tx=0x7f2574030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:17.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:17.796+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f256c053880 con 0x7f25840ce5b0 2023-08-25T09:45:18.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.022+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f258404e420 con 0x7f25840ce5b0 2023-08-25T09:45:18.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.023+0000 7f257bfff640 1 -- 172.21.15.70:0/2285918815 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f256c056ea0 con 0x7f25840ce5b0 2023-08-25T09:45:18.026 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:45:18.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256406a080 msgr2=0x7f256406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:18.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256406a080 0x7f256406c540 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2574035000 tx=0x7f2574030000 comp rx=0 tx=0).stop 2023-08-25T09:45:18.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 msgr2=0x7f258414dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:18.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f258414dd10 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f256c00b4a0 tx=0x7f256c00b960 comp rx=0 tx=0).stop 2023-08-25T09:45:18.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 shutdown_connections 2023-08-25T09:45:18.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f256406a080 0x7f256406c540 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f258415aa10 0x7f25840ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25840ce5b0 0x7f258414dd10 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 --2- 172.21.15.70:0/2285918815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25840b2680 0x7f25841567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 >> 172.21.15.70:0/2285918815 conn(0x7f2584069850 msgr2=0x7f258406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 shutdown_connections 2023-08-25T09:45:18.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:18.026+0000 7f2589a06640 1 -- 172.21.15.70:0/2285918815 wait complete. 2023-08-25T09:45:18.046 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:45:18.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:45:18.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:45:18.047 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:45:18.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:18.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:45:18.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:45:18.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:45:18.049 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:45:18.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.049 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:45:18.111 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:45:18.111 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:45:18.111 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:45:18.111 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.111 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:45:18.112 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41971sec preferred_lft 41971sec 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:45:18.113 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:45:18.114 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:45:18.115 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:45:18.115 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.115 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:45:18.115 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:45:18.115 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:45:18.115 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:45:18.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.117 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:45:18.208 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:45:18.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.212 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:45:18.262 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:45:18.262 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:45:18.262 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:45:18.262 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:45:18.262 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:45:18.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.263 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:45:18.264 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:45:18.264 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:45:18.264 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:45:18.265 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:45:18.265 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:45:18.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.440 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:45:18.491 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:45:18.491 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:45:18.492 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:45:18.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.493 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:45:18.589 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:45:18.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:45:18.590 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:18.645 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:45:18.645 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:45:18.646 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:18.694 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:45:18.790 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:45:18.791 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:45:18.883 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:45:18.886 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:45:18.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:18.887 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:45:18.941 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:45:18.941 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:45:18.947 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:45:19.026 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:45:19.027 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:45:19.101 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:45:19.103 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:45:19.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:19.104 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:45:19.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:45:19.142+0000 7f2b9fe46480 -1 init, newargv = 0x55c6972d8e80 newargc=15 2023-08-25T09:45:19.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[90636]: starting ceph client 2023-08-25T09:45:19.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[90636]: starting fuse 2023-08-25T09:45:20.247 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:45:20.351 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:45:20.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:45:20.448 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:45:20.452 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:45:20.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:20.453 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:45:20.508 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:45:20.509 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:45:20.509 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:45:20.510 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:45:20.511 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:45:20.511 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:45:20.512 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:45:20.512 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:45:20.513 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:45:20.513 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:45:20.514 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:45:20.514 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:45:20.515 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:45:20.516 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:45:20.516 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:45:20.517 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:45:20.517 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:45:20.518 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:45:20.518 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:45:20.519 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:45:20.519 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:45:20.520 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:45:20.520 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:45:20.521 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:45:20.521 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:45:20.522 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:45:20.522 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:45:20.523 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:45:20.523 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:45:20.704 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.90636.asok 2023-08-25T09:45:20.708 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:45:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:45:20.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.90636.asok 2023-08-25T09:45:20.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:20.721 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.90636.asok status 2023-08-25T09:45:20.961 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:45:20.961 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:45:20.961 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "90636", 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:45:20.962 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5165, 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:45:20.963 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5165 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 687730459 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:45:20.964 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:45:20.965 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:45:20.965 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:45:20.965 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:45:20.965 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 687730459 2023-08-25T09:45:20.965 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:45:20.965 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5165 192.168.0.1:0/687730459", 2023-08-25T09:45:20.966 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/687730459", 2023-08-25T09:45:20.966 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:45:20.966 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 49, 2023-08-25T09:45:20.966 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 115, 2023-08-25T09:45:20.967 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:45:20.967 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:45:20.967 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:45:20.967 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:45:20.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:20.977 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:45:20.978 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:21.030 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:45:21.030 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:21.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:21.031 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:21.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:21.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.345+0000 7fb0b8c8a640 1 Processor -- start 2023-08-25T09:45:21.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.345+0000 7fb0b8c8a640 1 -- start start 2023-08-25T09:45:21.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.346+0000 7fb0b8c8a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0b415ca00 0x7fb0b40cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.346+0000 7fb0b8c8a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b406ba40 con 0x7fb0b40c5120 2023-08-25T09:45:21.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.346+0000 7fb0b8c8a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b4160680 con 0x7fb0b40acd50 2023-08-25T09:45:21.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.346+0000 7fb0b8c8a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b4160960 con 0x7fb0b415ca00 2023-08-25T09:45:21.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.347+0000 7fb0abfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb0b40c5120 0x7fb0b406ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:37008/0 (socket says 172.21.15.70:37008) 2023-08-25T09:45:21.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.347+0000 7fb0abfff640 1 -- 172.21.15.70:0/2057687163 learned_addr learned my addr 172.21.15.70:0/2057687163 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:21.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.347+0000 7fb0b37fe640 1 --2- 172.21.15.70:0/2057687163 >> v2:172.21.15.155:3300/0 conn(0x7fb0b415ca00 0x7fb0b40cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.347+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532626777 0 0) 0x7fb0b406ba40 con 0x7fb0b40c5120 2023-08-25T09:45:21.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a0003520 con 0x7fb0b40c5120 2023-08-25T09:45:21.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2713045382 0 0) 0x7fb0a0003520 con 0x7fb0b40c5120 2023-08-25T09:45:21.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 >> v2:172.21.15.155:3300/0 conn(0x7fb0b415ca00 msgr2=0x7fb0b40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:21.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 --2- 172.21.15.70:0/2057687163 >> v2:172.21.15.155:3300/0 conn(0x7fb0b415ca00 0x7fb0b40cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 >> v1:172.21.15.155:6789/0 conn(0x7fb0b40acd50 legacy=0x7fb0b406b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4160c40 con 0x7fb0b40c5120 2023-08-25T09:45:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb09c001670 con 0x7fb0b40c5120 2023-08-25T09:45:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 >> v1:172.21.15.70:6789/0 conn(0x7fb0b40c5120 legacy=0x7fb0b406ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0003970 0x7fb0a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a00047c0 0x7fb0a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a0005180 0x7fb0a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b4160960 con 0x7fb0a00047c0 2023-08-25T09:45:21.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a000c020 con 0x7fb0a0003970 2023-08-25T09:45:21.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a000c1e0 con 0x7fb0a0005180 2023-08-25T09:45:21.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.348+0000 7fb0abfff640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a00047c0 0x7fb0a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0b37fe640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0003970 0x7fb0a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0b3fff640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a0005180 0x7fb0a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0abfff640 1 -- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a0005180 msgr2=0x7fb0a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0abfff640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a0005180 0x7fb0a000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0abfff640 1 -- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0003970 msgr2=0x7fb0a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0abfff640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0003970 0x7fb0a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0abfff640 1 -- 172.21.15.70:0/2057687163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0003520 con 0x7fb0a00047c0 2023-08-25T09:45:21.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.349+0000 7fb0b3fff640 1 -- 172.21.15.70:0/2057687163 reap_dead start 2023-08-25T09:45:21.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.350+0000 7fb0abfff640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a00047c0 0x7fb0a0004c40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c240 tx=0x7fb0a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abd503a5ac8e68e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:21.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.350+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c005ad0 con 0x7fb0a00047c0 2023-08-25T09:45:21.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.350+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c005c30 con 0x7fb0a00047c0 2023-08-25T09:45:21.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.350+0000 7fb0b2ffd640 1 -- 172.21.15.70:0/2057687163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c02caf0 con 0x7fb0a00047c0 2023-08-25T09:45:21.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/2057687163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a00047c0 msgr2=0x7fb0a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a00047c0 0x7fb0a0004c40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00c240 tx=0x7fb0a00057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/2057687163 shutdown_connections 2023-08-25T09:45:21.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/2057687163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a00047c0 0x7fb0a0004c40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/2057687163 >> 172.21.15.70:0/2057687163 conn(0x7fb0b40affd0 msgr2=0x7fb0b40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:21.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/2057687163 shutdown_connections 2023-08-25T09:45:21.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.351+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/2057687163 wait complete. 2023-08-25T09:45:21.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.353+0000 7fb0b8c8a640 1 Processor -- start 2023-08-25T09:45:21.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.353+0000 7fb0b8c8a640 1 -- start start 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b8c8a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00047c0 0x7fb0b406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b8c8a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b40b2e40 0x7fb0b40c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b8c8a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 0x7fb0b4070180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b8c8a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b406eeb0 con 0x7fb0b40c5f30 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b8c8a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b406fcf0 con 0x7fb0b40b2e40 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b8c8a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40b3fd0 con 0x7fb0a00047c0 2023-08-25T09:45:21.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 0x7fb0b4070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 0x7fb0b4070180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:55364/0 (socket says 172.21.15.70:55364) 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.354+0000 7fb0b3fff640 1 -- 172.21.15.70:0/1091743947 learned_addr learned my addr 172.21.15.70:0/1091743947 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b37fe640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00047c0 0x7fb0b406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b3fff640 1 -- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00047c0 msgr2=0x7fb0b406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b3fff640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00047c0 0x7fb0b406e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b3fff640 1 -- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b40b2e40 msgr2=0x7fb0b40c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b3fff640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b40b2e40 0x7fb0b40c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b3fff640 1 -- 172.21.15.70:0/1091743947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b406f0e0 con 0x7fb0b40c5f30 2023-08-25T09:45:21.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b37fe640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00047c0 0x7fb0b406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:21.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b3fff640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 0x7fb0b4070180 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb0b406c710 tx=0x7fb0ac00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:21.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac003d80 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac0050f0 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac005460 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.355+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b406f3c0 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.356+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b406f980 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.357+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b4066fb0 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.358+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0ac0055c0 con 0x7fb0b40c5f30 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.359+0000 7fb0b0ff9640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09406a050 0x7fb09406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:21.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.359+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0ac087e00 con 0x7fb0b40c5f30 2023-08-25T09:45:21.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.359+0000 7fb0b37fe640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09406a050 0x7fb09406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:21.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.360+0000 7fb0b37fe640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09406a050 0x7fb09406c510 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009a80 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:21.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.363+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0ac053590 con 0x7fb0b40c5f30 2023-08-25T09:45:21.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.590+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0b40ae530 con 0x7fb0b40c5f30 2023-08-25T09:45:21.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.591+0000 7fb0b0ff9640 1 -- 172.21.15.70:0/1091743947 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fb0ac018070 con 0x7fb0b40c5f30 2023-08-25T09:45:21.592 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:21.593 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:21.593 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:21.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09406a050 msgr2=0x7fb09406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09406a050 0x7fb09406c510 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009a80 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2023-08-25T09:45:21.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 msgr2=0x7fb0b4070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:21.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 0x7fb0b4070180 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb0b406c710 tx=0x7fb0ac00d9f0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 shutdown_connections 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09406a050 0x7fb09406c510 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0b40c5f30 0x7fb0b4070180 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0b40b2e40 0x7fb0b40c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 --2- 172.21.15.70:0/1091743947 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00047c0 0x7fb0b406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 >> 172.21.15.70:0/1091743947 conn(0x7fb0b40affd0 msgr2=0x7fb0b415ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.593+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 shutdown_connections 2023-08-25T09:45:21.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:21.594+0000 7fb0b8c8a640 1 -- 172.21.15.70:0/1091743947 wait complete. 2023-08-25T09:45:21.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-08-25T09:45:21.838 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":7227,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.830429083,"mdsmap_epoch":49,"osdmap_epoch":115,"osdmap_epoch_barrier":115,"uptime":20.95971943} 2023-08-25T09:45:21.838 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 7227, "mdsmap_epoch": 49, "osdmap_epoch": 115, "osdmap_epoch_barrier": 115, "rank_uptime": 14.830429083, "state": "up:active", "uptime": 20.95971943, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:45:21.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:45:22.058 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:45:22.059 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:45:22.059 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-08-25T09:45:22.060 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:22.060 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:22.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:22.115 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-08-25T09:45:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:265+0 records in 2023-08-25T09:45:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:265+0 records out 2023-08-25T09:45:27.827 INFO:teuthology.orchestra.run.smithi070.stderr:277872640 bytes (278 MB, 265 MiB) copied, 5.65183 s, 49.2 MB/s 2023-08-25T09:45:27.828 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:27.829 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:27.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:27.889 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-08-25T09:45:31.515 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:45:31.513+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2023-08-25T09:45:31.553 INFO:teuthology.orchestra.run.smithi070.stderr:dd: error writing 'large_file_b': No space left on device 2023-08-25T09:45:31.553 INFO:teuthology.orchestra.run.smithi070.stderr:dd: closing output file 'large_file_b': No space left on device 2023-08-25T09:45:31.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:45:31.555 INFO:tasks.cephfs.test_full:Writing file B failed (full status happened already) 2023-08-25T09:45:31.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:45:31.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.770+0000 7fa0039d6640 1 Processor -- start 2023-08-25T09:45:31.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.770+0000 7fa0039d6640 1 -- start start 2023-08-25T09:45:31.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa0039d6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9ffc0cffb0 0x7f9ffc0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa0039d6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc0d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa0039d6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ffc15cfb0 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa0039d6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cd4e0 con 0x7f9ffc0cffb0 2023-08-25T09:45:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa0039d6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc160f10 con 0x7f9ffc0cdeb0 2023-08-25T09:45:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa001772640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9ffc15cbd0 0x7f9ffc1606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43370/0 (socket says 172.21.15.70:43370) 2023-08-25T09:45:31.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.772+0000 7fa001772640 1 -- 172.21.15.70:0/858501538 learned_addr learned my addr 172.21.15.70:0/858501538 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7fa000f71640 1 --2- 172.21.15.70:0/858501538 >> v2:172.21.15.70:3300/0 conn(0x7f9ffc0cffb0 0x7f9ffc0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7fa001f73640 1 --2- 172.21.15.70:0/858501538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc0d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7f9ffbfff640 1 -- 172.21.15.70:0/858501538 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612965562 0 0) 0x7f9ffc15cfb0 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7f9ffbfff640 1 -- 172.21.15.70:0/858501538 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fe4003520 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7fa000f71640 1 -- 172.21.15.70:0/858501538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 msgr2=0x7f9ffc0d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7fa000f71640 1 --2- 172.21.15.70:0/858501538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc0d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7fa000f71640 1 -- 172.21.15.70:0/858501538 >> v1:172.21.15.70:6789/0 conn(0x7f9ffc15cbd0 legacy=0x7f9ffc1606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.773+0000 7fa000f71640 1 -- 172.21.15.70:0/858501538 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc1611f0 con 0x7f9ffc0cffb0 2023-08-25T09:45:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.774+0000 7fa000f71640 1 --2- 172.21.15.70:0/858501538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc0cffb0 0x7f9ffc0cd970 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00097a0 tx=0x7f9ff002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd65e28be26910f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.774+0000 7f9ffbfff640 1 -- 172.21.15.70:0/858501538 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff002d040 con 0x7f9ffc0cffb0 2023-08-25T09:45:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.774+0000 7f9ffbfff640 1 -- 172.21.15.70:0/858501538 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff0031810 con 0x7f9ffc0cffb0 2023-08-25T09:45:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.774+0000 7f9ffbfff640 1 -- 172.21.15.70:0/858501538 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff0031ba0 con 0x7f9ffc0cffb0 2023-08-25T09:45:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 -- 172.21.15.70:0/858501538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc0cffb0 msgr2=0x7f9ffc0cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 --2- 172.21.15.70:0/858501538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc0cffb0 0x7f9ffc0cd970 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00097a0 tx=0x7f9ff002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:45:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 -- 172.21.15.70:0/858501538 shutdown_connections 2023-08-25T09:45:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 --2- 172.21.15.70:0/858501538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc0cffb0 0x7f9ffc0cd970 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 --2- 172.21.15.70:0/858501538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc0d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 -- 172.21.15.70:0/858501538 >> 172.21.15.70:0/858501538 conn(0x7f9ffc070260 msgr2=0x7f9ffc070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:31.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 -- 172.21.15.70:0/858501538 shutdown_connections 2023-08-25T09:45:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.775+0000 7fa0039d6640 1 -- 172.21.15.70:0/858501538 wait complete. 2023-08-25T09:45:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.777+0000 7fa0039d6640 1 Processor -- start 2023-08-25T09:45:31.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.777+0000 7fa0039d6640 1 -- start start 2023-08-25T09:45:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa0039d6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc1f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:31.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa0039d6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ffc0cffb0 0x7f9ffc1f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa0039d6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 0x7f9ffc0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa0039d6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc15d370 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa0039d6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc1f5ce0 con 0x7f9ffc0cffb0 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa0039d6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cb3a0 con 0x7f9ffc0cdeb0 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa001f73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 0x7f9ffc0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa001f73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 0x7f9ffc0cae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41166/0 (socket says 172.21.15.70:41166) 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.778+0000 7fa001f73640 1 -- 172.21.15.70:0/3681239919 learned_addr learned my addr 172.21.15.70:0/3681239919 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:31.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa001f73640 1 -- 172.21.15.70:0/3681239919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 msgr2=0x7f9ffc1f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa001f73640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc1f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa001f73640 1 -- 172.21.15.70:0/3681239919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ffc0cffb0 msgr2=0x7f9ffc1f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa001f73640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ffc0cffb0 0x7f9ffc1f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa001f73640 1 -- 172.21.15.70:0/3681239919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009400 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa001f73640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 0x7f9ffc0cae60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4007e40 tx=0x7f9ff400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4014620 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff400ee90 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff40149e0 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.779+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc1f5f40 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.780+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc1f6500 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.781+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc04e420 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.782+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9ff4014b40 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.783+0000 7f9ff9ffb640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9fe806a050 0x7f9fe806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.783+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ff4088fa0 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.783+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=117}) v3 -- 0x7f9fe80721a0 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.783+0000 7fa001772640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9fe806a050 0x7f9fe806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:31.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.784+0000 7fa001772640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9fe806a050 0x7f9fe806c510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fec001570 tx=0x7f9fec007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:31.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.787+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9ff4054680 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.975+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9ffc0c9250 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.979+0000 7f9ff9ffb640 1 -- 172.21.15.70:0/3681239919 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v116) v1 ==== 81+0+37084 (secure 0 0 0) 0x7f9ff4057ca0 con 0x7f9ffc15cbd0 2023-08-25T09:45:31.981 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 116, 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:45:31.513905+0000", 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:45:31.982 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:45:31.983 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 14, 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:45:31.984 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:45:31.985 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:31.986 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:45:31.987 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:31.988 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:31.989 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:31.990 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:31.991 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:31.992 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:31.993 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:31.994 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:31.995 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:45:31.996 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:31.997 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:45:31.998 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:31.999 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.000 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.001 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.001 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.001 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.001 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.001 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.002 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.003 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.004 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.005 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:45:32.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.006 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.006 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:45:32.006 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:45:32.006 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:45:32.007 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:45:32.007 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.008 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 13, 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:04.798879+0000", 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:32.009 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:32.010 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:45:32.011 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:45:32.011 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:45:32.011 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:45:32.011 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:45:32.011 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:32.012 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:32.012 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:32.012 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:32.012 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.012 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.012 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.013 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.013 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.013 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.013 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.013 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.014 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.015 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.015 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.015 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.015 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.016 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.017 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.018 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:32.019 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:32.020 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 14, 2023-08-25T09:45:32.021 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:45:32.022 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:05.478134+0000", 2023-08-25T09:45:32.022 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:32.022 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:32.022 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:32.023 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:45:32.024 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.025 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.026 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.027 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.028 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.029 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.030 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.032 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.032 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6299999952316284, 2023-08-25T09:45:32.032 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6299999952316284, 2023-08-25T09:45:32.032 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.6299999952316284, 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.6299999952316284, 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.034 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.035 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.037 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.038 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.039 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.041 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.042 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.043 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.044 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.045 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.047 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.048 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.049 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:45:32.051 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:32.053 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.054 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.055 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.057 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:45:32.061 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.064 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.065 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.066 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.068 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.070 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:45:32.071 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.073 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.074 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.075 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.076 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:45:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.084 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:45:32.086 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.087 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:45:32.089 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:45:32.092 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:45:32.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.098 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:45:32.102 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.104 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.105 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.106 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:45:32.107 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.111 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:45:32.112 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.113 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.114 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.115 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.116 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.117 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.119 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.120 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.121 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.122 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:45:32.123 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:45:32.124 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:45:32.125 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:45:32.126 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.127 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9fe806a050 msgr2=0x7f9fe806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9fe806a050 0x7f9fe806c510 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9fec001570 tx=0x7f9fec007930 comp rx=0 tx=0).stop 2023-08-25T09:45:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 msgr2=0x7f9ffc0cae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 0x7f9ffc0cae60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4007e40 tx=0x7f9ff400e4b0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 shutdown_connections 2023-08-25T09:45:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9fe806a050 0x7f9fe806c510 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9ffc15cbd0 0x7f9ffc0cae60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9ffc0cffb0 0x7f9ffc1f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.981+0000 7fa0039d6640 1 --2- 172.21.15.70:0/3681239919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9ffc0cdeb0 0x7f9ffc1f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.982+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 >> 172.21.15.70:0/3681239919 conn(0x7f9ffc070260 msgr2=0x7f9ffc0b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:32.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.982+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 shutdown_connections 2023-08-25T09:45:32.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:31.982+0000 7fa0039d6640 1 -- 172.21.15.70:0/3681239919 wait complete. 2023-08-25T09:45:32.131 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:32.131 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:32.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:32.194 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-08-25T09:45:32.349 INFO:teuthology.orchestra.run.smithi070.stderr:dd: error writing 'large_file_b': No space left on device 2023-08-25T09:45:32.349 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2023-08-25T09:45:32.350 INFO:teuthology.orchestra.run.smithi070.stderr:0+0 records out 2023-08-25T09:45:32.350 INFO:teuthology.orchestra.run.smithi070.stderr:0 bytes copied, 0.00656205 s, 0.0 kB/s 2023-08-25T09:45:32.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:45:32.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:45:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.567+0000 7f77da28e640 1 Processor -- start 2023-08-25T09:45:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.568+0000 7f77da28e640 1 -- start start 2023-08-25T09:45:32.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.569+0000 7f77da28e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77d415ec10 0x7f77d415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.569+0000 7f77da28e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415c6d0 0x7f77d40b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.569+0000 7f77da28e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d40b1b60 con 0x7f77d415e830 2023-08-25T09:45:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.569+0000 7f77da28e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d406bd80 con 0x7f77d415ec10 2023-08-25T09:45:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.569+0000 7f77da28e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d406b7c0 con 0x7f77d415c6d0 2023-08-25T09:45:32.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77d415ec10 0x7f77d415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f77d415ec10 0x7f77d415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41178/0 (socket says 172.21.15.70:41178) 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 -- 172.21.15.70:0/3180581851 learned_addr learned my addr 172.21.15.70:0/3180581851 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 -- 172.21.15.70:0/3180581851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415c6d0 msgr2=0x7f77d40b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d882b640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415c6d0 0x7f77d40b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415c6d0 0x7f77d40b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 -- 172.21.15.70:0/3180581851 >> v1:172.21.15.70:6789/0 conn(0x7f77d415e830 legacy=0x7f77d406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d37fe640 1 -- 172.21.15.70:0/3180581851 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d406baa0 con 0x7f77d415ec10 2023-08-25T09:45:32.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.570+0000 7f77d882b640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415c6d0 0x7f77d40b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:32.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.572+0000 7f77d37fe640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415ec10 0x7f77d415c190 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009870 tx=0x7f77c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb69ba7b2710c41 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:32.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.572+0000 7f77d2ffd640 1 -- 172.21.15.70:0/3180581851 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030840 con 0x7f77d415ec10 2023-08-25T09:45:32.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.572+0000 7f77d2ffd640 1 -- 172.21.15.70:0/3180581851 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c80309a0 con 0x7f77d415ec10 2023-08-25T09:45:32.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.572+0000 7f77d2ffd640 1 -- 172.21.15.70:0/3180581851 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8030d30 con 0x7f77d415ec10 2023-08-25T09:45:32.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 -- 172.21.15.70:0/3180581851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415ec10 msgr2=0x7f77d415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415ec10 0x7f77d415c190 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009870 tx=0x7f77c802b980 comp rx=0 tx=0).stop 2023-08-25T09:45:32.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 -- 172.21.15.70:0/3180581851 shutdown_connections 2023-08-25T09:45:32.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415ec10 0x7f77d415c190 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 --2- 172.21.15.70:0/3180581851 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415c6d0 0x7f77d40b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 -- 172.21.15.70:0/3180581851 >> 172.21.15.70:0/3180581851 conn(0x7f77d4069850 msgr2=0x7f77d4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 -- 172.21.15.70:0/3180581851 shutdown_connections 2023-08-25T09:45:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.573+0000 7f77da28e640 1 -- 172.21.15.70:0/3180581851 wait complete. 2023-08-25T09:45:32.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.575+0000 7f77da28e640 1 Processor -- start 2023-08-25T09:45:32.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.575+0000 7f77da28e640 1 -- start start 2023-08-25T09:45:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77da28e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 0x7f77d40c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77da28e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d415e830 0x7f77d415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77da28e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415ec10 0x7f77d41e7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:32.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77da28e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40ca660 con 0x7f77d415c6d0 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77da28e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d415bcb0 con 0x7f77d415e830 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77da28e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40c9a50 con 0x7f77d415ec10 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 0x7f77d40c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 0x7f77d40c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41192/0 (socket says 172.21.15.70:41192) 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77d3fff640 1 -- 172.21.15.70:0/4253889423 learned_addr learned my addr 172.21.15.70:0/4253889423 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.576+0000 7f77d37fe640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d415e830 0x7f77d415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d882b640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415ec10 0x7f77d41e7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:32.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d3fff640 1 -- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415ec10 msgr2=0x7f77d41e7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d3fff640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415ec10 0x7f77d41e7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d3fff640 1 -- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d415e830 msgr2=0x7f77d415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d3fff640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d415e830 0x7f77d415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d3fff640 1 -- 172.21.15.70:0/4253889423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80094d0 con 0x7f77d415c6d0 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d3fff640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 0x7f77d40c9400 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c450 tx=0x7f77c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40052d0 con 0x7f77d415c6d0 2023-08-25T09:45:32.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4005430 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4012870 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.577+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d40c9d30 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.578+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d40ca2f0 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.579+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d404e420 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.579+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77c40129d0 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.580+0000 7f77d0ff9640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77c006a080 0x7f77c006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.580+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77c4088140 con 0x7f77d415c6d0 2023-08-25T09:45:32.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.581+0000 7f77d37fe640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77c006a080 0x7f77c006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.581+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=117}) v3 -- 0x7f77c00721d0 con 0x7f77d415c6d0 2023-08-25T09:45:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.581+0000 7f77d37fe640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77c006a080 0x7f77c006c540 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f77c8036000 tx=0x7f77c803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:32.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.585+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77c4053790 con 0x7f77d415c6d0 2023-08-25T09:45:32.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.775+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f77d406c620 con 0x7f77d415c6d0 2023-08-25T09:45:32.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.779+0000 7f77d0ff9640 1 -- 172.21.15.70:0/4253889423 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v116) v1 ==== 81+0+37084 (secure 0 0 0) 0x7f77c4056db0 con 0x7f77d415c6d0 2023-08-25T09:45:32.781 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:32.782 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:32.782 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 116, 2023-08-25T09:45:32.782 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:32.783 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:45:32.783 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:45:31.513905+0000", 2023-08-25T09:45:32.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:45:32.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:45:32.784 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:45:32.784 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:45:32.784 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:45:32.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:45:32.785 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:45:32.785 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:45:32.785 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:45:32.786 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.786 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:45:32.786 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:45:32.786 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:45:32.787 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:45:32.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:45:32.787 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 14, 2023-08-25T09:45:32.787 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:45:32.787 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:45:32.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:45:32.788 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:45:32.789 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:32.789 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:32.789 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:32.789 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:32.789 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:32.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:32.791 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.793 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.795 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.796 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.797 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.797 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.797 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.797 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.797 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.797 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.798 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.798 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:32.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:45:32.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:45:32.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.798 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:32.799 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.800 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:32.801 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:32.802 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:32.803 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.804 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.805 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.806 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.807 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.808 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.808 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.808 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.809 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:45:32.810 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.810 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:45:32.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.810 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.810 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:45:32.810 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:45:32.811 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:45:32.811 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:45:32.811 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:45:32.811 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.812 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.812 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 13, 2023-08-25T09:45:32.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:45:32.814 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:04.798879+0000", 2023-08-25T09:45:32.814 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:32.814 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:32.814 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:32.815 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:32.815 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:32.815 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:32.815 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:32.816 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:32.816 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:32.816 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:32.816 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:32.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:32.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:45:32.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:45:32.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:45:32.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:45:32.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:45:32.818 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:32.818 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:32.818 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:32.819 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:32.819 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.819 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.820 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.821 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.821 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.821 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.821 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.822 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.822 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.822 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.822 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.823 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.823 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.823 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.823 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.824 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.824 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.824 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.825 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.825 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.825 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.825 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.826 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.827 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.827 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.828 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.828 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.828 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.828 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.828 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.829 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.829 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.829 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.829 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.829 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.829 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:32.830 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:32.831 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.832 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 14, 2023-08-25T09:45:32.833 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:45:32.833 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:05.478134+0000", 2023-08-25T09:45:32.833 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:32.833 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:32.833 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:32.833 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:32.834 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:32.834 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:32.834 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:32.834 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:32.834 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:45:32.835 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:32.836 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:32.837 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:32.838 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:32.839 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:32.839 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:32.839 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:32.839 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:32.840 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:32.840 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:32.840 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:32.840 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:32.841 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:32.841 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:32.841 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:32.842 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:32.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:32.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.842 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:32.843 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:32.843 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:32.843 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:32.843 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:32.844 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:32.844 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:32.844 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:32.844 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:32.845 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:32.845 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:32.845 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:32.846 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:45:32.846 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:45:32.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.847 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:32.847 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:32.847 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:45:32.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.848 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:32.848 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6299999952316284, 2023-08-25T09:45:32.849 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6299999952316284, 2023-08-25T09:45:32.849 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:32.849 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.6299999952316284, 2023-08-25T09:45:32.849 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.6299999952316284, 2023-08-25T09:45:32.850 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:32.850 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:32.850 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:32.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.851 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.851 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:45:32.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.852 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:32.852 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.852 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.852 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.852 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.853 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.856 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:45:32.858 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:32.860 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.861 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.861 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:45:32.861 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:45:32.861 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:45:32.862 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:45:32.862 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.862 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.862 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.863 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.864 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.864 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.864 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.864 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.864 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.864 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:45:32.865 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.866 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.869 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:45:32.869 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.870 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.871 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.872 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:45:32.872 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:45:32.873 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:32.874 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.874 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.875 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:45:32.875 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:45:32.875 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:45:32.875 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:45:32.876 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.876 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.876 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:32.876 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.877 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.878 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:32.878 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.878 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.878 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.879 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.879 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.879 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.879 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.879 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.880 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.880 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.880 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.880 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:45:32.881 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.882 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:45:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.884 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.885 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:45:32.885 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.886 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:45:32.886 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.886 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.887 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.887 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.887 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.888 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.888 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:45:32.888 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.889 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:45:32.890 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.890 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.890 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.891 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.891 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.892 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:45:32.892 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.893 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.893 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:45:32.893 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:32.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.894 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.894 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:45:32.895 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:45:32.895 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:45:32.895 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:45:32.895 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.896 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.896 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.896 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.897 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:32.897 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.897 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.897 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.898 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.898 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.898 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.898 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.898 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.898 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.899 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:45:32.901 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.902 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.906 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:45:32.907 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:45:32.907 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:45:32.907 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:45:32.907 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.908 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.908 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.908 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.909 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.910 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.911 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.912 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.913 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.915 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:45:32.917 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.919 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:45:32.921 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.923 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:45:32.927 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.928 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.929 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.930 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.934 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.935 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.936 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.937 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.938 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:32.939 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:32.940 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:32.942 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.943 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.944 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:45:32.945 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:32.946 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.947 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:32.948 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:32.949 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:45:32.951 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.952 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.953 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.954 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:45:32.955 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.956 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.957 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:32.958 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:32.959 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:45:32.960 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:45:32.961 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:45:32.962 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:32.963 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77c006a080 msgr2=0x7f77c006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77c006a080 0x7f77c006c540 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f77c8036000 tx=0x7f77c803b040 comp rx=0 tx=0).stop 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 msgr2=0x7f77d40c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 0x7f77d40c9400 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f77c400c450 tx=0x7f77c400c910 comp rx=0 tx=0).stop 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 shutdown_connections 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f77c006a080 0x7f77c006c540 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77d415ec10 0x7f77d41e7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77d415e830 0x7f77d415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 --2- 172.21.15.70:0/4253889423 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f77d415c6d0 0x7f77d40c9400 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 >> 172.21.15.70:0/4253889423 conn(0x7f77d4069850 msgr2=0x7f77d4070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 shutdown_connections 2023-08-25T09:45:32.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:32.782+0000 7f77da28e640 1 -- 172.21.15.70:0/4253889423 wait complete. 2023-08-25T09:45:32.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.189+0000 7f1668034640 1 Processor -- start 2023-08-25T09:45:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.189+0000 7f1668034640 1 -- start start 2023-08-25T09:45:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f1668034640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f166015ec10 0x7f166015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:33.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f1668034640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f166015c6d0 0x7f16600b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:33.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f1668034640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16600b1b60 con 0x7f166015e830 2023-08-25T09:45:33.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f1668034640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166006bd80 con 0x7f166015ec10 2023-08-25T09:45:33.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f1668034640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f166006b7c0 con 0x7f166015c6d0 2023-08-25T09:45:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f16655cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f166015ec10 0x7f166015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f16655cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f166015ec10 0x7f166015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41194/0 (socket says 172.21.15.70:41194) 2023-08-25T09:45:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.191+0000 7f16655cf640 1 -- 172.21.15.70:0/3334348680 learned_addr learned my addr 172.21.15.70:0/3334348680 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:33.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.192+0000 7f16665d1640 1 --2- 172.21.15.70:0/3334348680 >> v2:172.21.15.155:3300/0 conn(0x7f166015c6d0 0x7f16600b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.192+0000 7f16655cf640 1 -- 172.21.15.70:0/3334348680 >> v2:172.21.15.155:3300/0 conn(0x7f166015c6d0 msgr2=0x7f16600b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.192+0000 7f16655cf640 1 --2- 172.21.15.70:0/3334348680 >> v2:172.21.15.155:3300/0 conn(0x7f166015c6d0 0x7f16600b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.192+0000 7f16655cf640 1 -- 172.21.15.70:0/3334348680 >> v1:172.21.15.155:6789/0 conn(0x7f166015e830 legacy=0x7f166006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:33.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.192+0000 7f16655cf640 1 -- 172.21.15.70:0/3334348680 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166006baa0 con 0x7f166015ec10 2023-08-25T09:45:33.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.193+0000 7f16655cf640 1 --2- 172.21.15.70:0/3334348680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015ec10 0x7f166015c190 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f16600cf460 tx=0x7f165002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a65f8ce4ff2ace server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:33.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.193+0000 7f1664dce640 1 -- 172.21.15.70:0/3334348680 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16500317e0 con 0x7f166015ec10 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.193+0000 7f1664dce640 1 -- 172.21.15.70:0/3334348680 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1650031940 con 0x7f166015ec10 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.193+0000 7f1664dce640 1 -- 172.21.15.70:0/3334348680 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1650031cd0 con 0x7f166015ec10 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 -- 172.21.15.70:0/3334348680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015ec10 msgr2=0x7f166015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 --2- 172.21.15.70:0/3334348680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015ec10 0x7f166015c190 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f16600cf460 tx=0x7f165002b880 comp rx=0 tx=0).stop 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 -- 172.21.15.70:0/3334348680 shutdown_connections 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 --2- 172.21.15.70:0/3334348680 >> v2:172.21.15.155:3300/0 conn(0x7f166015c6d0 0x7f16600b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 --2- 172.21.15.70:0/3334348680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015ec10 0x7f166015c190 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 -- 172.21.15.70:0/3334348680 >> 172.21.15.70:0/3334348680 conn(0x7f1660069850 msgr2=0x7f1660069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 -- 172.21.15.70:0/3334348680 shutdown_connections 2023-08-25T09:45:33.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.194+0000 7f1668034640 1 -- 172.21.15.70:0/3334348680 wait complete. 2023-08-25T09:45:33.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.196+0000 7f1668034640 1 Processor -- start 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.196+0000 7f1668034640 1 -- start start 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1668034640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 0x7f16601f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1668034640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166015e830 0x7f16600c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1668034640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166015ec10 0x7f166015cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1668034640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660144020 con 0x7f166015c6d0 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1668034640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1660144300 con 0x7f166015e830 2023-08-25T09:45:33.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1668034640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16601445e0 con 0x7f166015ec10 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1665dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 0x7f16601f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1665dd0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 0x7f16601f1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41210/0 (socket says 172.21.15.70:41210) 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1665dd0640 1 -- 172.21.15.70:0/2996099990 learned_addr learned my addr 172.21.15.70:0/2996099990 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1665dd0640 1 -- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166015ec10 msgr2=0x7f166015cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1665dd0640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166015ec10 0x7f166015cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f16655cf640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166015e830 0x7f16600c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:33.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.197+0000 7f1665dd0640 1 -- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166015e830 msgr2=0x7f16600c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f1665dd0640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166015e830 0x7f16600c9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f1665dd0640 1 -- 172.21.15.70:0/2996099990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16500094d0 con 0x7f166015c6d0 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f1665dd0640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 0x7f16601f1b00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c430 tx=0x7f165c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c011790 con 0x7f166015c6d0 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165c0118f0 con 0x7f166015c6d0 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16601448c0 con 0x7f166015c6d0 2023-08-25T09:45:33.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165c011ce0 con 0x7f166015c6d0 2023-08-25T09:45:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.198+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1660143b90 con 0x7f166015c6d0 2023-08-25T09:45:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.199+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1660141b00 con 0x7f166015c6d0 2023-08-25T09:45:33.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.200+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f165c01c460 con 0x7f166015c6d0 2023-08-25T09:45:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.201+0000 7f16567fc640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164406a080 0x7f164406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.201+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f165c088540 con 0x7f166015c6d0 2023-08-25T09:45:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.202+0000 7f16655cf640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164406a080 0x7f164406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.202+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=117}) v3 -- 0x7f16440721d0 con 0x7f166015c6d0 2023-08-25T09:45:33.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.202+0000 7f16655cf640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164406a080 0x7f164406c540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1650036000 tx=0x7f165003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:33.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.206+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f165c053c20 con 0x7f166015c6d0 2023-08-25T09:45:33.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.433+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1660143d80 con 0x7f166015c6d0 2023-08-25T09:45:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.434+0000 7f16567fc640 1 -- 172.21.15.70:0/2996099990 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f165c057240 con 0x7f166015c6d0 2023-08-25T09:45:33.436 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:33.436 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:33.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.436+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164406a080 msgr2=0x7f164406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.436+0000 7f1668034640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164406a080 0x7f164406c540 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1650036000 tx=0x7f165003d040 comp rx=0 tx=0).stop 2023-08-25T09:45:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.436+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 msgr2=0x7f16601f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:33.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.436+0000 7f1668034640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 0x7f16601f1b00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f165c00c430 tx=0x7f165c00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.436+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 shutdown_connections 2023-08-25T09:45:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.436+0000 7f1668034640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f164406a080 0x7f164406c540 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.437+0000 7f1668034640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f166015ec10 0x7f166015cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.437+0000 7f1668034640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f166015e830 0x7f16600c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.437+0000 7f1668034640 1 --2- 172.21.15.70:0/2996099990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f166015c6d0 0x7f16601f1b00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:33.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.437+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 >> 172.21.15.70:0/2996099990 conn(0x7f1660069850 msgr2=0x7f1660069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:33.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.437+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 shutdown_connections 2023-08-25T09:45:33.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:33.437+0000 7f1668034640 1 -- 172.21.15.70:0/2996099990 wait complete. 2023-08-25T09:45:33.458 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-08-25T09:45:33.694 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":7227,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":26.685500161,"mdsmap_epoch":49,"osdmap_epoch":116,"osdmap_epoch_barrier":116,"uptime":32.814789738000002} 2023-08-25T09:45:33.695 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 7227, "mdsmap_epoch": 49, "osdmap_epoch": 116, "osdmap_epoch_barrier": 116, "rank_uptime": 26.685500161, "state": "up:active", "uptime": 32.814789738, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:45:33.696 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:45:33.697 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:33.697 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:33.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:33.756 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-08-25T09:45:33.807 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:45:33.805+0000 7f10dbfff640 -1 client.7407 handle_client_reply no pending request on tid 13 safe is:1 2023-08-25T09:45:33.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:45:33.810 INFO:teuthology.orchestra.run.smithi070.stderr:dd: failed to open 'small_file_1': Permission denied 2023-08-25T09:45:33.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:33.811 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-08-25T09:45:33.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:33.863 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-08-25T09:45:33.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:45:34.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.185+0000 7f00f4324640 1 Processor -- start 2023-08-25T09:45:34.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.185+0000 7f00f4324640 1 -- start start 2023-08-25T09:45:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f4324640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 0x7f00ec06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f4324640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f00ec15ec10 0x7f00ec0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f4324640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ec0b1b60 con 0x7f00ec06b2b0 2023-08-25T09:45:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f4324640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ec06bd80 con 0x7f00ec15ec10 2023-08-25T09:45:34.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f4324640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec06b7c0 con 0x7f00ec15e830 2023-08-25T09:45:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f18bf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f00ec06b2b0 0x7f00ec15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43374/0 (socket says 172.21.15.70:43374) 2023-08-25T09:45:34.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f18bf640 1 -- 172.21.15.70:0/1860491079 learned_addr learned my addr 172.21.15.70:0/1860491079 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f20c0640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 0x7f00ec06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.187+0000 7f00f28c1640 1 --2- 172.21.15.70:0/1860491079 >> v2:172.21.15.155:3300/0 conn(0x7f00ec15ec10 0x7f00ec0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723317389 0 0) 0x7f00ec0b1b60 con 0x7f00ec06b2b0 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d8003520 con 0x7f00ec06b2b0 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 163076079 0 0) 0x7f00d8003520 con 0x7f00ec06b2b0 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 msgr2=0x7f00ec06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:34.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 0x7f00ec06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 >> v2:172.21.15.155:3300/0 conn(0x7f00ec15ec10 msgr2=0x7f00ec0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 --2- 172.21.15.70:0/1860491079 >> v2:172.21.15.155:3300/0 conn(0x7f00ec15ec10 0x7f00ec0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec06baa0 con 0x7f00ec06b2b0 2023-08-25T09:45:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f00dc001670 con 0x7f00ec06b2b0 2023-08-25T09:45:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 >> v1:172.21.15.70:6789/0 conn(0x7f00ec06b2b0 legacy=0x7f00ec15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00d8003970 0x7f00d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 0x7f00d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec06bd80 con 0x7f00d8004d00 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec06b7c0 con 0x7f00d8003970 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d800b980 con 0x7f00ec15e830 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.188+0000 7f00f28c1640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f20c0640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00d8003970 0x7f00d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f18bf640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 0x7f00d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f28c1640 1 -- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 msgr2=0x7f00d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f28c1640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec15e830 0x7f00d80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f28c1640 1 -- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00d8003970 msgr2=0x7f00d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f28c1640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00d8003970 0x7f00d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.189+0000 7f00f28c1640 1 -- 172.21.15.70:0/1860491079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8003520 con 0x7f00d8004d00 2023-08-25T09:45:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.190+0000 7f00f28c1640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00d8005180 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009fc0 tx=0x7f00d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c199ee443b6f02bb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.190+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e402cc60 con 0x7f00d8004d00 2023-08-25T09:45:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.190+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e400f040 con 0x7f00d8004d00 2023-08-25T09:45:34.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.190+0000 7f00f10be640 1 -- 172.21.15.70:0/1860491079 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4034440 con 0x7f00d8004d00 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f4324640 1 -- 172.21.15.70:0/1860491079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 msgr2=0x7f00d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f4324640 1 --2- 172.21.15.70:0/1860491079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00d8005180 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009fc0 tx=0x7f00d80057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f28c1640 1 -- 172.21.15.70:0/1860491079 reap_dead start 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f4324640 1 -- 172.21.15.70:0/1860491079 shutdown_connections 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f4324640 1 -- 172.21.15.70:0/1860491079 >> 172.21.15.70:0/1860491079 conn(0x7f00ec069850 msgr2=0x7f00ec069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f4324640 1 -- 172.21.15.70:0/1860491079 shutdown_connections 2023-08-25T09:45:34.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.191+0000 7f00f4324640 1 -- 172.21.15.70:0/1860491079 wait complete. 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.193+0000 7f00f4324640 1 Processor -- start 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.193+0000 7f00f4324640 1 -- start start 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f4324640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00ec15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f4324640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00ec0c79d0 0x7f00ec14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f4324640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec0a8460 0x7f00ec06fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f4324640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec14b700 con 0x7f00d8004d00 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f4324640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec1f3f00 con 0x7f00ec0c79d0 2023-08-25T09:45:34.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f4324640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec1f4150 con 0x7f00ec0a8460 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f20c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00ec15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f20c0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00ec15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:41222/0 (socket says 172.21.15.70:41222) 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.194+0000 7f00f20c0640 1 -- 172.21.15.70:0/694241539 learned_addr learned my addr 172.21.15.70:0/694241539 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f18bf640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00ec0c79d0 0x7f00ec14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f20c0640 1 -- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec0a8460 msgr2=0x7f00ec06fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f20c0640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec0a8460 0x7f00ec06fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f20c0640 1 -- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00ec0c79d0 msgr2=0x7f00ec14b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:34.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f20c0640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00ec0c79d0 0x7f00ec14b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f20c0640 1 -- 172.21.15.70:0/694241539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec15ca20 con 0x7f00d8004d00 2023-08-25T09:45:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f18bf640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00ec0c79d0 0x7f00ec14b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:34.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f20c0640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00ec15dec0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f00e8011580 tx=0x7f00e8011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8015870 con 0x7f00d8004d00 2023-08-25T09:45:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e80159d0 con 0x7f00d8004d00 2023-08-25T09:45:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8015d40 con 0x7f00d8004d00 2023-08-25T09:45:34.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ec1f4430 con 0x7f00d8004d00 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.195+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ec1f49f0 con 0x7f00d8004d00 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.196+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ec1f4bb0 con 0x7f00d8004d00 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.197+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f00e800f460 con 0x7f00d8004d00 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.199+0000 7f00e2ffd640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f00d006a080 0x7f00d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.199+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f00e80889a0 con 0x7f00d8004d00 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.199+0000 7f00f18bf640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f00d006a080 0x7f00d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:34.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.199+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=117}) v3 -- 0x7f00d00721d0 con 0x7f00d8004d00 2023-08-25T09:45:34.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.200+0000 7f00f18bf640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f00d006a080 0x7f00d006c540 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0c7df0 tx=0x7f00dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:34.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.203+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f00e80540a0 con 0x7f00d8004d00 2023-08-25T09:45:34.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.392+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f00ec04e420 con 0x7f00d8004d00 2023-08-25T09:45:34.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.396+0000 7f00e2ffd640 1 -- 172.21.15.70:0/694241539 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v116) v1 ==== 81+0+37084 (secure 0 0 0) 0x7f00e80576c0 con 0x7f00d8004d00 2023-08-25T09:45:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:34.399 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 116, 2023-08-25T09:45:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:45:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:45:31.513905+0000", 2023-08-25T09:45:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:45:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:45:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 14, 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:45:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:45:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 13, 2023-08-25T09:45:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:45:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:04.798879+0000", 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:45:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:45:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 14, 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:05.478134+0000", 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:45:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6299999952316284, 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6299999952316284, 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.6299999952316284, 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.6299999952316284, 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:45:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:45:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:45:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:45:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:45:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:45:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:45:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:45:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:45:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:45:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:45:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:45:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:45:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:45:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:45:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:45:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:34.545 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:34.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f00d006a080 msgr2=0x7f00d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f00d006a080 0x7f00d006c540 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0c7df0 tx=0x7f00dc009040 comp rx=0 tx=0).stop 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 msgr2=0x7f00ec15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00ec15dec0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f00e8011580 tx=0x7f00e8011a40 comp rx=0 tx=0).stop 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 shutdown_connections 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f00d006a080 0x7f00d006c540 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f00ec0a8460 0x7f00ec06fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f00ec0c79d0 0x7f00ec14b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 --2- 172.21.15.70:0/694241539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f00d8004d00 0x7f00ec15dec0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:34.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 >> 172.21.15.70:0/694241539 conn(0x7f00ec069850 msgr2=0x7f00ec0ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:34.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 shutdown_connections 2023-08-25T09:45:34.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:34.399+0000 7f00f4324640 1 -- 172.21.15.70:0/694241539 wait complete. 2023-08-25T09:45:34.549 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:45:37.332 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:45:37.331+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 5 full osd(s) (OSD_FULL) 2023-08-25T09:45:39.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:45:39.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.775+0000 7f6e4ce21640 1 Processor -- start 2023-08-25T09:45:39.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.775+0000 7f6e4ce21640 1 -- start start 2023-08-25T09:45:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.776+0000 7f6e4ce21640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6e4815ca00 0x7f6e480cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.776+0000 7f6e4ce21640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4806ba40 con 0x7f6e480acd50 2023-08-25T09:45:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e4ce21640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e48160680 con 0x7f6e480c5120 2023-08-25T09:45:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e4ce21640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e48160960 con 0x7f6e4815ca00 2023-08-25T09:45:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e477fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6e4815ca00 0x7f6e480cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e477fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6e4815ca00 0x7f6e480cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40236/0 (socket says 172.21.15.70:40236) 2023-08-25T09:45:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e47fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6e480acd50 0x7f6e4806b330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57142/0 (socket says 172.21.15.70:57142) 2023-08-25T09:45:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e477fe640 1 -- 172.21.15.70:0/173953421 learned_addr learned my addr 172.21.15.70:0/173953421 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e477fe640 1 -- 172.21.15.70:0/173953421 >> v1:172.21.15.155:6789/0 conn(0x7f6e480c5120 legacy=0x7f6e4806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e477fe640 1 -- 172.21.15.70:0/173953421 >> v1:172.21.15.70:6789/0 conn(0x7f6e480acd50 legacy=0x7f6e4806b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.777+0000 7f6e477fe640 1 -- 172.21.15.70:0/173953421 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48160c40 con 0x7f6e4815ca00 2023-08-25T09:45:39.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.778+0000 7f6e477fe640 1 --2- 172.21.15.70:0/173953421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e480cceb0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009870 tx=0x7f6e3802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d925d2371f876bf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e467fc640 1 -- 172.21.15.70:0/173953421 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38030860 con 0x7f6e4815ca00 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e467fc640 1 -- 172.21.15.70:0/173953421 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e380309c0 con 0x7f6e4815ca00 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e467fc640 1 -- 172.21.15.70:0/173953421 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38030d50 con 0x7f6e4815ca00 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e4ce21640 1 -- 172.21.15.70:0/173953421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 msgr2=0x7f6e480cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/173953421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e480cceb0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009870 tx=0x7f6e3802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e4ce21640 1 -- 172.21.15.70:0/173953421 shutdown_connections 2023-08-25T09:45:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/173953421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e480cceb0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.779+0000 7f6e4ce21640 1 -- 172.21.15.70:0/173953421 >> 172.21.15.70:0/173953421 conn(0x7f6e480affd0 msgr2=0x7f6e480b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.780+0000 7f6e4ce21640 1 -- 172.21.15.70:0/173953421 shutdown_connections 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.780+0000 7f6e4ce21640 1 -- 172.21.15.70:0/173953421 wait complete. 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.781+0000 7f6e4ce21640 1 Processor -- start 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.781+0000 7f6e4ce21640 1 -- start start 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.782+0000 7f6e4ce21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e480acd50 0x7f6e48155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:39.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.782+0000 7f6e4ce21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e480c5120 0x7f6e481556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.782+0000 7f6e4ce21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e481f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.782+0000 7f6e4ce21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e481f7220 con 0x7f6e4815ca00 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.782+0000 7f6e4ce21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48155c20 con 0x7f6e480c5120 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.782+0000 7f6e4ce21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4815dc30 con 0x7f6e480acd50 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e481f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e481f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40246/0 (socket says 172.21.15.70:40246) 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 -- 172.21.15.70:0/2454914734 learned_addr learned my addr 172.21.15.70:0/2454914734 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e477fe640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e480acd50 0x7f6e48155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:39.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 -- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e480acd50 msgr2=0x7f6e48155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:39.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e46ffd640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e480c5120 0x7f6e481556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:39.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e480acd50 0x7f6e48155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:39.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 -- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e480c5120 msgr2=0x7f6e481556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:39.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e480c5120 0x7f6e481556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:39.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 -- 172.21.15.70:0/2454914734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380094d0 con 0x7f6e4815ca00 2023-08-25T09:45:39.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.783+0000 7f6e47fff640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e481f6260 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c6c0 tx=0x7f6e4000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:39.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.784+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40004d60 con 0x7f6e4815ca00 2023-08-25T09:45:39.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.784+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4000a640 con 0x7f6e4815ca00 2023-08-25T09:45:39.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.784+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000a9b0 con 0x7f6e4815ca00 2023-08-25T09:45:39.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.784+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4815df10 con 0x7f6e4815ca00 2023-08-25T09:45:39.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.784+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4815e4d0 con 0x7f6e4815ca00 2023-08-25T09:45:39.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.785+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e480c3670 con 0x7f6e4815ca00 2023-08-25T09:45:39.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.786+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6e4000ab10 con 0x7f6e4815ca00 2023-08-25T09:45:39.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.787+0000 7f6e2ffff640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6e3406a080 0x7f6e3406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:39.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.787+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e400878c0 con 0x7f6e4815ca00 2023-08-25T09:45:39.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.788+0000 7f6e477fe640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6e3406a080 0x7f6e3406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:39.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.788+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=119}) v3 -- 0x7f6e340721d0 con 0x7f6e4815ca00 2023-08-25T09:45:39.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.788+0000 7f6e477fe640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6e3406a080 0x7f6e3406c540 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036000 tx=0x7f6e3803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:39.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.792+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6e40053050 con 0x7f6e4815ca00 2023-08-25T09:45:39.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.983+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6e48066fb0 con 0x7f6e4815ca00 2023-08-25T09:45:39.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.985+0000 7f6e2ffff640 1 -- 172.21.15.70:0/2454914734 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v118) v1 ==== 81+0+37108 (secure 0 0 0) 0x7f6e40018020 con 0x7f6e4815ca00 2023-08-25T09:45:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:39.987 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 118, 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:45:39.396242+0000", 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:45:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:45:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:45:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:45:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:45:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:45:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 14, 2023-08-25T09:45:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:45:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:45:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:45:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:45:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:45:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:45:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 13, 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:04.798879+0000", 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:45:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:45:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 14, 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:05.478134+0000", 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 3, 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full", 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "116", 2023-08-25T09:45:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:45:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6299999952316284, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6299999952316284, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.6299999952316284, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.6299999952316284, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:45:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:45:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:45:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:45:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:45:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:45:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:45:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:45:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:45:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:45:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:45:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:45:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:45:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:45:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.073 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:45:40.073 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.073 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:45:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:45:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:45:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:45:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:45:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:45:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:45:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:45:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:45:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:45:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:45:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:45:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:45:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:45:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:45:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:45:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:45:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:45:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:45:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:45:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:45:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:45:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:45:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:45:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:45:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:45:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:45:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:45:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:45:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:45:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:45:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "full", 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:45:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:45:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:45:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:45:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:45:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:40.152 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:40.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6e3406a080 msgr2=0x7f6e3406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:40.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6e3406a080 0x7f6e3406c540 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036000 tx=0x7f6e3803b040 comp rx=0 tx=0).stop 2023-08-25T09:45:40.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 msgr2=0x7f6e481f6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:40.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e481f6260 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000c6c0 tx=0x7f6e4000cb80 comp rx=0 tx=0).stop 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 shutdown_connections 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6e3406a080 0x7f6e3406c540 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6e4815ca00 0x7f6e481f6260 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6e480c5120 0x7f6e481556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 --2- 172.21.15.70:0/2454914734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6e480acd50 0x7f6e48155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 >> 172.21.15.70:0/2454914734 conn(0x7f6e480affd0 msgr2=0x7f6e480ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 shutdown_connections 2023-08-25T09:45:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:39.988+0000 7f6e4ce21640 1 -- 172.21.15.70:0/2454914734 wait complete. 2023-08-25T09:45:40.156 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:45:42.334 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:45:42.333+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-08-25T09:45:45.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:45:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.385+0000 7f2e224aa640 1 Processor -- start 2023-08-25T09:45:45.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.386+0000 7f2e224aa640 1 -- start start 2023-08-25T09:45:45.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.387+0000 7f2e224aa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c159840 0x7f2e1c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.387+0000 7f2e224aa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e1c15d3d0 0x7f2e1c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.387+0000 7f2e224aa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c15d890 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.387+0000 7f2e224aa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c0c8b60 con 0x7f2e1c15d3d0 2023-08-25T09:45:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.387+0000 7f2e224aa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c15db70 con 0x7f2e1c159840 2023-08-25T09:45:45.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e1c15d3d0 0x7f2e1c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:45.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e1c15d3d0 0x7f2e1c0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40248/0 (socket says 172.21.15.70:40248) 2023-08-25T09:45:45.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 -- 172.21.15.70:0/1769930067 learned_addr learned my addr 172.21.15.70:0/1769930067 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e21ca9640 1 --1- 172.21.15.70:0/1769930067 >> v1:172.21.15.70:6789/0 conn(0x7f2e1c0cadb0 0x7f2e1c0cb1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57158/0 (socket says 172.21.15.70:57158) 2023-08-25T09:45:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 -- 172.21.15.70:0/1769930067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c159840 msgr2=0x7f2e1c0b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e214a8640 1 --2- 172.21.15.70:0/1769930067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c159840 0x7f2e1c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 --2- 172.21.15.70:0/1769930067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c159840 0x7f2e1c0b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 -- 172.21.15.70:0/1769930067 >> v1:172.21.15.70:6789/0 conn(0x7f2e1c0cadb0 legacy=0x7f2e1c0cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e20ca7640 1 -- 172.21.15.70:0/1769930067 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c15de50 con 0x7f2e1c15d3d0 2023-08-25T09:45:45.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.388+0000 7f2e13fff640 1 -- 172.21.15.70:0/1769930067 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410812981 0 0) 0x7f2e1c15d890 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.389+0000 7f2e20ca7640 1 --2- 172.21.15.70:0/1769930067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c15d3d0 0x7f2e1c0b3d40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009870 tx=0x7f2e0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b8527ee3813bdb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.389+0000 7f2e13fff640 1 -- 172.21.15.70:0/1769930067 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c030840 con 0x7f2e1c15d3d0 2023-08-25T09:45:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e13fff640 1 -- 172.21.15.70:0/1769930067 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c0309a0 con 0x7f2e1c15d3d0 2023-08-25T09:45:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e13fff640 1 -- 172.21.15.70:0/1769930067 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c030d30 con 0x7f2e1c15d3d0 2023-08-25T09:45:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 -- 172.21.15.70:0/1769930067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c15d3d0 msgr2=0x7f2e1c0b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:45.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 --2- 172.21.15.70:0/1769930067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c15d3d0 0x7f2e1c0b3d40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c009870 tx=0x7f2e0c02b980 comp rx=0 tx=0).stop 2023-08-25T09:45:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 -- 172.21.15.70:0/1769930067 shutdown_connections 2023-08-25T09:45:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 --2- 172.21.15.70:0/1769930067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c15d3d0 0x7f2e1c0b3d40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 --2- 172.21.15.70:0/1769930067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c159840 0x7f2e1c0b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 -- 172.21.15.70:0/1769930067 >> 172.21.15.70:0/1769930067 conn(0x7f2e1c0affd0 msgr2=0x7f2e1c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 -- 172.21.15.70:0/1769930067 shutdown_connections 2023-08-25T09:45:45.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.390+0000 7f2e224aa640 1 -- 172.21.15.70:0/1769930067 wait complete. 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.392+0000 7f2e224aa640 1 Processor -- start 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.392+0000 7f2e224aa640 1 -- start start 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e224aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 0x7f2e1c06e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e224aa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e1c159840 0x7f2e1c0aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e224aa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c15d3d0 0x7f2e1c0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e224aa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c155590 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e224aa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c153170 con 0x7f2e1c159840 2023-08-25T09:45:45.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e224aa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c153450 con 0x7f2e1c15d3d0 2023-08-25T09:45:45.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.393+0000 7f2e214a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 0x7f2e1c06e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:45.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 0x7f2e1c06e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40250/0 (socket says 172.21.15.70:40250) 2023-08-25T09:45:45.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 -- 172.21.15.70:0/2061020026 learned_addr learned my addr 172.21.15.70:0/2061020026 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:45.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e21ca9640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c15d3d0 0x7f2e1c0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e20ca7640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e1c159840 0x7f2e1c0aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 -- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c15d3d0 msgr2=0x7f2e1c0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c15d3d0 0x7f2e1c0af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 -- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e1c159840 msgr2=0x7f2e1c0aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e1c159840 0x7f2e1c0aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 -- 172.21.15.70:0/2061020026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c0094d0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e20ca7640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e1c159840 0x7f2e1c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:45.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.394+0000 7f2e214a8640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 0x7f2e1c06e970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800c7f0 tx=0x7f2e1800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:45.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.395+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18004220 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.395+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e180043a0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.395+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18007860 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.395+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c153730 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.395+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c155220 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.396+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c152fd0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.397+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e18007bd0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.398+0000 7f2e11ffb640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e0006a050 0x7f2e0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:45.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.398+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f2e180886c0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.398+0000 7f2e20ca7640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e0006a050 0x7f2e0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:45.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.399+0000 7f2e20ca7640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e0006a050 0x7f2e0006c510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c036000 tx=0x7f2e0c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:45.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.402+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2e1c152fd0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.465 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:45:45.463+0000 7fc7619ea640 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 1 pg recovery_toofull (PG_RECOVERY_FULL) 2023-08-25T09:45:45.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.603+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2e1c152fd0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.607+0000 7f2e11ffb640 1 -- 172.21.15.70:0/2061020026 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v121) v1 ==== 81+0+38072 (secure 0 0 0) 0x7f2e1c152fd0 con 0x7f2e1c0cadb0 2023-08-25T09:45:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:45.611 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 121, 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:45:42.437094+0000", 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:45:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:45:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 14, 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:45:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:45:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:45:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:45.621 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:45.622 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:45.623 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:45.624 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:45.625 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:45.626 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:45.627 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:45.628 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:45.629 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.630 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:45.631 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:45.632 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:45.633 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:45.634 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:45.635 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:45:45.636 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.637 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 13, 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:04.798879+0000", 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:45.638 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:45.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:45:45.640 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:45:45.641 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:45:45.641 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:45:45.641 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:45:45.641 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:45.642 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:45.642 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:45.642 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:45.643 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:45.643 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:45.643 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:45.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.644 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:45.644 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:45.644 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:45.645 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:45.645 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:45.645 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:45.645 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:45.646 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:45.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:45.646 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:45.646 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:45.646 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:45.647 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:45.647 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:45.647 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:45.648 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:45.648 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:45.648 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:45.648 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:45.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:45.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:45.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:45.650 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:45.650 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:45.650 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:45.651 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:45.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:45.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.652 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:45.652 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:45.652 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:45.652 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:45.653 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:45.653 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:45.653 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:45.654 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:45.654 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:45.654 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:45.654 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:45.655 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:45.655 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:45:45.655 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:45:45.656 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:45:45.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.656 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:45.656 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:45.657 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:45:45.657 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.657 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:45.658 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:45:45.658 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:45:45.658 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:45.658 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:45.659 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:45.659 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:45.659 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:45.660 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:45.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.661 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 14, 2023-08-25T09:45:45.661 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:45:45.662 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:05.478134+0000", 2023-08-25T09:45:45.662 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:45.662 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:45.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:45.663 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:45.663 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:45.663 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:45.663 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:45.664 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:45.664 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:45.664 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:45.665 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:45.665 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:45.665 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:45:45.665 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:45:45.666 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:45:45.666 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:45:45.666 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:45:45.667 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:45.667 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:45.667 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:45.667 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:45.668 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:45.668 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:45.668 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:45.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:45.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:45.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:45.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:45.670 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:45.670 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:45.670 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:45.671 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:45.671 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:45.671 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:45.671 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:45.672 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:45.672 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:45.672 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:45.673 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:45.673 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:45.673 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:45.673 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:45.674 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:45.674 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:45.674 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:45.675 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:45.675 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:45.675 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:45.676 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:45.676 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:45.676 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:45.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:45.677 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:45:45.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:45.679 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:45.680 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.681 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.682 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:45:45.683 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.684 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.686 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.687 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:45:45.690 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.691 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.692 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.694 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:45:45.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.698 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.699 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:45:45.700 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.701 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.702 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.703 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:45:45.704 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.706 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:45:45.707 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:45:45.710 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:45:45.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.716 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:45:45.717 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:45:45.720 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.721 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.722 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.723 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:45:45.724 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.725 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.726 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.728 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:45:45.730 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.732 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:45:45.735 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.736 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.738 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:45:45.740 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.741 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.742 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.744 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:45:45.745 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.748 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.749 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:45:45.750 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:45.751 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:45.752 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.753 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.754 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.756 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:45.758 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:45:45.760 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.761 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.762 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.763 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:45.764 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:45:45.766 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.767 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.768 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.769 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:45.771 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.772 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "13.b", 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 6, 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 4 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.12", 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.19", 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.775 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.23", 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:45.776 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.2d", 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2023-08-25T09:45:45.777 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.3b", 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:45.778 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:45:45.779 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:45:45.780 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:45:45.781 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:45.782 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:45.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:45:45.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e0006a050 msgr2=0x7f2e0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e0006a050 0x7f2e0006c510 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c036000 tx=0x7f2e0c03b040 comp rx=0 tx=0).stop 2023-08-25T09:45:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 msgr2=0x7f2e1c06e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 0x7f2e1c06e970 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800c7f0 tx=0x7f2e1800ccb0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 shutdown_connections 2023-08-25T09:45:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e0006a050 0x7f2e0006c510 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e1c15d3d0 0x7f2e1c0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e1c159840 0x7f2e1c0aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 --2- 172.21.15.70:0/2061020026 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e1c0cadb0 0x7f2e1c06e970 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:45.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 >> 172.21.15.70:0/2061020026 conn(0x7f2e1c0affd0 msgr2=0x7f2e1c151490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:45.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 shutdown_connections 2023-08-25T09:45:45.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:45.610+0000 7f2e224aa640 1 -- 172.21.15.70:0/2061020026 wait complete. 2023-08-25T09:45:45.787 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-08-25T09:45:45.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:45:45.834 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:45:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.033+0000 7f75fd071640 1 Processor -- start 2023-08-25T09:45:46.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.033+0000 7f75fd071640 1 -- start start 2023-08-25T09:45:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75fd071640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75f80ce5b0 0x7f75f80ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75fd071640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f80b2850 0x7f75f806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75fd071640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f80ceed0 con 0x7f75f815aa10 2023-08-25T09:45:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75fd071640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f8159d10 con 0x7f75f80ce5b0 2023-08-25T09:45:46.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75fd071640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f815d3f0 con 0x7f75f80b2850 2023-08-25T09:45:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75f6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75f80ce5b0 0x7f75f80ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75f6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f75f80ce5b0 0x7f75f80ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40266/0 (socket says 172.21.15.70:40266) 2023-08-25T09:45:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.035+0000 7f75f6d9d640 1 -- 172.21.15.70:0/1415659407 learned_addr learned my addr 172.21.15.70:0/1415659407 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:46.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.036+0000 7f75f6d9d640 1 -- 172.21.15.70:0/1415659407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f80b2850 msgr2=0x7f75f806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.036+0000 7f75f6d9d640 1 --2- 172.21.15.70:0/1415659407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f80b2850 0x7f75f806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.036+0000 7f75f6d9d640 1 -- 172.21.15.70:0/1415659407 >> v1:172.21.15.70:6789/0 conn(0x7f75f815aa10 legacy=0x7f75f80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.036+0000 7f75f6d9d640 1 -- 172.21.15.70:0/1415659407 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80b2d20 con 0x7f75f80ce5b0 2023-08-25T09:45:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.036+0000 7f75f659c640 1 -- 172.21.15.70:0/1415659407 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599919440 0 0) 0x7f75f80ceed0 con 0x7f75f815aa10 2023-08-25T09:45:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.037+0000 7f75f6d9d640 1 --2- 172.21.15.70:0/1415659407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80ce5b0 0x7f75f80ce990 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f75e800d150 tx=0x7f75e802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ba7196b0077fc7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.037+0000 7f75f659c640 1 -- 172.21.15.70:0/1415659407 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e802ea80 con 0x7f75f80ce5b0 2023-08-25T09:45:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.037+0000 7f75f659c640 1 -- 172.21.15.70:0/1415659407 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e802ebe0 con 0x7f75f80ce5b0 2023-08-25T09:45:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.037+0000 7f75f659c640 1 -- 172.21.15.70:0/1415659407 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8009600 con 0x7f75f80ce5b0 2023-08-25T09:45:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 -- 172.21.15.70:0/1415659407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80ce5b0 msgr2=0x7f75f80ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 --2- 172.21.15.70:0/1415659407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80ce5b0 0x7f75f80ce990 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f75e800d150 tx=0x7f75e802e0e0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 -- 172.21.15.70:0/1415659407 shutdown_connections 2023-08-25T09:45:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 --2- 172.21.15.70:0/1415659407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f80b2850 0x7f75f806f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 --2- 172.21.15.70:0/1415659407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80ce5b0 0x7f75f80ce990 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 -- 172.21.15.70:0/1415659407 >> 172.21.15.70:0/1415659407 conn(0x7f75f8069850 msgr2=0x7f75f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:46.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 -- 172.21.15.70:0/1415659407 shutdown_connections 2023-08-25T09:45:46.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.038+0000 7f75fd071640 1 -- 172.21.15.70:0/1415659407 wait complete. 2023-08-25T09:45:46.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.040+0000 7f75fd071640 1 Processor -- start 2023-08-25T09:45:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.040+0000 7f75fd071640 1 -- start start 2023-08-25T09:45:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75fd071640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 0x7f75f814a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75fd071640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75f80ce5b0 0x7f75f80aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75fd071640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f815aa10 0x7f75f80af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75fd071640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f814a910 con 0x7f75f80b2850 2023-08-25T09:45:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75fd071640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80b04f0 con 0x7f75f80ce5b0 2023-08-25T09:45:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75fd071640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f81f31a0 con 0x7f75f815aa10 2023-08-25T09:45:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75f6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 0x7f75f814a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75f6d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 0x7f75f814a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40276/0 (socket says 172.21.15.70:40276) 2023-08-25T09:45:46.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.041+0000 7f75f6d9d640 1 -- 172.21.15.70:0/2549785228 learned_addr learned my addr 172.21.15.70:0/2549785228 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75f6d9d640 1 -- 172.21.15.70:0/2549785228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f815aa10 msgr2=0x7f75f80af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75f6d9d640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f815aa10 0x7f75f80af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75f6d9d640 1 -- 172.21.15.70:0/2549785228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75f80ce5b0 msgr2=0x7f75f80aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75f6d9d640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75f80ce5b0 0x7f75f80aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75f6d9d640 1 -- 172.21.15.70:0/2549785228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e800bd40 con 0x7f75f80b2850 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75f6d9d640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 0x7f75f814a3d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f75e8038040 tx=0x7f75e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e802ea80 con 0x7f75f80b2850 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8036820 con 0x7f75f80b2850 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8036bd0 con 0x7f75f80b2850 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f80af8e0 con 0x7f75f80b2850 2023-08-25T09:45:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.042+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f80afea0 con 0x7f75f80b2850 2023-08-25T09:45:46.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.044+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f804e420 con 0x7f75f80b2850 2023-08-25T09:45:46.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.044+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f75e802ed50 con 0x7f75f80b2850 2023-08-25T09:45:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.045+0000 7f75effff640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f75d806a050 0x7f75d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.045+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f75e80ab7b0 con 0x7f75f80b2850 2023-08-25T09:45:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.046+0000 7f75ee59c640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f75d806a050 0x7f75d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.046+0000 7f75ee59c640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f75d806a050 0x7f75d806c510 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f75f80cee10 tx=0x7f75e0007950 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:46.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.050+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f75e8076d60 con 0x7f75f80b2850 2023-08-25T09:45:46.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.239+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f75f80cae50 con 0x7f75f80b2850 2023-08-25T09:45:46.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.243+0000 7f75effff640 1 -- 172.21.15.70:0/2549785228 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v121) v1 ==== 81+0+38072 (secure 0 0 0) 0x7f75e8035e80 con 0x7f75f80b2850 2023-08-25T09:45:46.246 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:46.246 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 121, 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:45:42.437094+0000", 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:45:46.247 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:45:46.248 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:46.249 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:45:46.249 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:45:46.249 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:45:46.249 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 14, 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:45:46.250 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:45:46.251 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:45:46.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.251 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:45:46.251 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:45:46.251 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:45:46.251 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:46.252 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:45:46.253 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:46.254 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:46.255 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:46.256 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:46.257 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:46.258 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:46.259 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:46.260 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:46.261 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:45:46.262 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:46.263 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:45:46.264 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:46.265 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:46.266 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:46.267 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:46.268 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.269 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:46.270 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:46.271 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:46.272 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 13, 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:04.798879+0000", 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:46.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:46.274 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:46.275 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.276 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:46.277 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:46.278 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:46.279 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:46.280 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:46.281 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:45:46.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:45:46.283 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 14, 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:45:46.284 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:45:05.478134+0000", 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:45:46.285 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:45:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:45:46.287 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:45:46.288 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:45:46.289 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:45:46.290 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:45:46.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:45:46.292 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.293 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2023-08-25T09:45:46.294 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:45:46.295 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2023-08-25T09:45:46.295 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2023-08-25T09:45:46.295 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:45:46.295 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:46.296 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.297 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.299 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.301 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:45:46.305 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:46.306 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.307 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.308 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:45:46.310 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:45:46.315 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.316 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.318 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:45:46.320 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.321 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.323 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:45:46.325 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.327 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.329 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:45:46.330 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.333 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:45:46.335 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.338 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.339 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.341 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:45:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.347 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:45:46.349 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.350 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.351 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:45:46.352 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:45:46.355 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.357 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 119, 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:45:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.360 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.361 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:45:46.362 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:45:46.365 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:45:46.367 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 113, 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:45:46.369 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.371 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:45:46.375 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.377 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.378 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:45:46.380 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.381 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.382 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:45:46.383 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:45:46.386 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2023-08-25T09:45:46.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "13.b", 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 6, 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 4 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.388 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.12", 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:46.389 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.19", 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2023-08-25T09:45:46.390 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.23", 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.391 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.2d", 2023-08-25T09:45:46.392 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.393 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "14.3b", 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 3 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.394 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:45:46.395 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:45:46.396 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:45:46.397 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:45:46.398 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:45:46.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:45:46.398 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:45:46.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.245+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f75d806a050 msgr2=0x7f75d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.245+0000 7f75fd071640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f75d806a050 0x7f75d806c510 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f75f80cee10 tx=0x7f75e0007950 comp rx=0 tx=0).stop 2023-08-25T09:45:46.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.245+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 msgr2=0x7f75f814a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.245+0000 7f75fd071640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 0x7f75f814a3d0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f75e8038040 tx=0x7f75e8004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 shutdown_connections 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f75d806a050 0x7f75d806c510 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f75f815aa10 0x7f75f80af2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f75f80ce5b0 0x7f75f80aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 --2- 172.21.15.70:0/2549785228 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f75f80b2850 0x7f75f814a3d0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 >> 172.21.15.70:0/2549785228 conn(0x7f75f8069850 msgr2=0x7f75f806ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:46.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 shutdown_connections 2023-08-25T09:45:46.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.246+0000 7f75fd071640 1 -- 172.21.15.70:0/2549785228 wait complete. 2023-08-25T09:45:46.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:46.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.621+0000 7f12020cc640 1 Processor -- start 2023-08-25T09:45:46.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.622+0000 7f12020cc640 1 -- start start 2023-08-25T09:45:46.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.623+0000 7f12020cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15ec10 0x7f11fc15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.623+0000 7f12020cc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f11fc15c6d0 0x7f11fc0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.623+0000 7f12020cc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0b1b60 con 0x7f11fc15e830 2023-08-25T09:45:46.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.623+0000 7f12020cc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc06bd80 con 0x7f11fc15c6d0 2023-08-25T09:45:46.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.623+0000 7f12020cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc06b7c0 con 0x7f11fc15ec10 2023-08-25T09:45:46.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11fc15e830 0x7f11fc06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57180/0 (socket says 172.21.15.70:57180) 2023-08-25T09:45:46.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fb7fe640 1 -- 172.21.15.70:0/648039412 learned_addr learned my addr 172.21.15.70:0/648039412 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:46.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fbfff640 1 --2- 172.21.15.70:0/648039412 >> v2:172.21.15.70:3300/0 conn(0x7f11fc15c6d0 0x7f11fc0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11faffd640 1 -- 172.21.15.70:0/648039412 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389400841 0 0) 0x7f11fc0b1b60 con 0x7f11fc15e830 2023-08-25T09:45:46.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11faffd640 1 -- 172.21.15.70:0/648039412 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e8003520 con 0x7f11fc15e830 2023-08-25T09:45:46.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11f3fff640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15ec10 0x7f11fc15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fbfff640 1 -- 172.21.15.70:0/648039412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15ec10 msgr2=0x7f11fc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:46.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fbfff640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15ec10 0x7f11fc15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fbfff640 1 -- 172.21.15.70:0/648039412 >> v1:172.21.15.70:6789/0 conn(0x7f11fc15e830 legacy=0x7f11fc06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11fbfff640 1 -- 172.21.15.70:0/648039412 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc06baa0 con 0x7f11fc15c6d0 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.624+0000 7f11f3fff640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15ec10 0x7f11fc15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.625+0000 7f11fbfff640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15c6d0 0x7f11fc0b1ff0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009d90 tx=0x7f11f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572de1e22ddb92ff server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f11faffd640 1 -- 172.21.15.70:0/648039412 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f402cbd0 con 0x7f11fc15c6d0 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f11faffd640 1 -- 172.21.15.70:0/648039412 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f402cd30 con 0x7f11fc15c6d0 2023-08-25T09:45:46.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f11faffd640 1 -- 172.21.15.70:0/648039412 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f402cea0 con 0x7f11fc15c6d0 2023-08-25T09:45:46.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 -- 172.21.15.70:0/648039412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15c6d0 msgr2=0x7f11fc0b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15c6d0 0x7f11fc0b1ff0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009d90 tx=0x7f11f402be20 comp rx=0 tx=0).stop 2023-08-25T09:45:46.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 -- 172.21.15.70:0/648039412 shutdown_connections 2023-08-25T09:45:46.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15c6d0 0x7f11fc0b1ff0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 --2- 172.21.15.70:0/648039412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15ec10 0x7f11fc15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 -- 172.21.15.70:0/648039412 >> 172.21.15.70:0/648039412 conn(0x7f11fc069850 msgr2=0x7f11fc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:46.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 -- 172.21.15.70:0/648039412 shutdown_connections 2023-08-25T09:45:46.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.626+0000 7f12020cc640 1 -- 172.21.15.70:0/648039412 wait complete. 2023-08-25T09:45:46.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.628+0000 7f12020cc640 1 Processor -- start 2023-08-25T09:45:46.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.628+0000 7f12020cc640 1 -- start start 2023-08-25T09:45:46.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.629+0000 7f12020cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15c6d0 0x7f11fc155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.629+0000 7f12020cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11fc15e830 0x7f11fc1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.629+0000 7f12020cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 0x7f11fc0ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.629+0000 7f12020cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0ad1a0 con 0x7f11fc15ec10 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.629+0000 7f12020cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc155eb0 con 0x7f11fc15e830 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.629+0000 7f12020cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc152170 con 0x7f11fc15c6d0 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 0x7f11fc0ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 0x7f11fc0ad630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40304/0 (socket says 172.21.15.70:40304) 2023-08-25T09:45:46.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 -- 172.21.15.70:0/2503338772 learned_addr learned my addr 172.21.15.70:0/2503338772 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11f3fff640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11fc15e830 0x7f11fc1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 -- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15c6d0 msgr2=0x7f11fc155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15c6d0 0x7f11fc155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 -- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11fc15e830 msgr2=0x7f11fc1518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11fc15e830 0x7f11fc1518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 -- 172.21.15.70:0/2503338772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f4009a70 con 0x7f11fc15ec10 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.630+0000 7f11fbfff640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 0x7f11fc0ad630 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009d90 tx=0x7f11f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:46.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.631+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f402cea0 con 0x7f11fc15ec10 2023-08-25T09:45:46.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.631+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f4040650 con 0x7f11fc15ec10 2023-08-25T09:45:46.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.631+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f4040a00 con 0x7f11fc15ec10 2023-08-25T09:45:46.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.631+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0ac870 con 0x7f11fc15ec10 2023-08-25T09:45:46.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.631+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0ace30 con 0x7f11fc15ec10 2023-08-25T09:45:46.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.632+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc04e420 con 0x7f11fc15ec10 2023-08-25T09:45:46.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.633+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f11f4040b60 con 0x7f11fc15ec10 2023-08-25T09:45:46.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.634+0000 7f11f8ff9640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11dc06a050 0x7f11dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:46.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.634+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f11f40ab3a0 con 0x7f11fc15ec10 2023-08-25T09:45:46.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.634+0000 7f11fb7fe640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11dc06a050 0x7f11dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:46.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.635+0000 7f11fb7fe640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11dc06a050 0x7f11dc06c510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f11ec001610 tx=0x7f11ec007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:46.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.638+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f11f4076af0 con 0x7f11fc15ec10 2023-08-25T09:45:46.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.865+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11fc06b450 con 0x7f11fc15ec10 2023-08-25T09:45:46.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.866+0000 7f11f8ff9640 1 -- 172.21.15.70:0/2503338772 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f11f4009450 con 0x7f11fc15ec10 2023-08-25T09:45:46.870 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:46.870 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:46.872 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:46.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11dc06a050 msgr2=0x7f11dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11dc06a050 0x7f11dc06c510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f11ec001610 tx=0x7f11ec007930 comp rx=0 tx=0).stop 2023-08-25T09:45:46.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 msgr2=0x7f11fc0ad630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:46.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 0x7f11fc0ad630 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009d90 tx=0x7f11f4004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:46.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 shutdown_connections 2023-08-25T09:45:46.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11dc06a050 0x7f11dc06c510 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc15ec10 0x7f11fc0ad630 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11fc15e830 0x7f11fc1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 --2- 172.21.15.70:0/2503338772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc15c6d0 0x7f11fc155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:46.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 >> 172.21.15.70:0/2503338772 conn(0x7f11fc069850 msgr2=0x7f11fc0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:46.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.870+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 shutdown_connections 2023-08-25T09:45:46.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:46.871+0000 7f12020cc640 1 -- 172.21.15.70:0/2503338772 wait complete. 2023-08-25T09:45:46.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-08-25T09:45:47.125 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":7227,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":40.116211597000003,"mdsmap_epoch":49,"osdmap_epoch":121,"osdmap_epoch_barrier":116,"uptime":46.245502221000002} 2023-08-25T09:45:47.125 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 7227, "mdsmap_epoch": 49, "osdmap_epoch": 121, "osdmap_epoch_barrier": 116, "rank_uptime": 40.116211597, "state": "up:active", "uptime": 46.245502221, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:45:47.126 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:45:47.127 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:47.127 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:47.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:47.181 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-08-25T09:45:47.730 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records in 2023-08-25T09:45:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records out 2023-08-25T09:45:47.731 INFO:teuthology.orchestra.run.smithi070.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.496612 s, 106 MB/s 2023-08-25T09:45:47.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:45:47.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:45:47.860 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:45:47.986 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:45:47.986 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:47.986 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:48.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:48.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.264+0000 7f278a9ab640 1 Processor -- start 2023-08-25T09:45:48.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.264+0000 7f278a9ab640 1 -- start start 2023-08-25T09:45:48.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f278a9ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f278415cbd0 0x7f27840cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f278a9ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840d2a00 0x7f27840d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f278a9ab640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278415cfb0 con 0x7f27841606c0 2023-08-25T09:45:48.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f278a9ab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840d33d0 con 0x7f278415cbd0 2023-08-25T09:45:48.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f278a9ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cd4e0 con 0x7f27840d2a00 2023-08-25T09:45:48.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f2783fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f278415cbd0 0x7f27840cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:48.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f2783fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f278415cbd0 0x7f27840cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40318/0 (socket says 172.21.15.70:40318) 2023-08-25T09:45:48.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.265+0000 7f2783fff640 1 -- 172.21.15.70:0/4139171818 learned_addr learned my addr 172.21.15.70:0/4139171818 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:48.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.266+0000 7f2783fff640 1 -- 172.21.15.70:0/4139171818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840d2a00 msgr2=0x7f27840d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:48.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.266+0000 7f2783fff640 1 --2- 172.21.15.70:0/4139171818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840d2a00 0x7f27840d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.266+0000 7f2783fff640 1 -- 172.21.15.70:0/4139171818 >> v1:172.21.15.155:6789/0 conn(0x7f27841606c0 legacy=0x7f27840cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:48.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.266+0000 7f2783fff640 1 -- 172.21.15.70:0/4139171818 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784160f10 con 0x7f278415cbd0 2023-08-25T09:45:48.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.267+0000 7f2783fff640 1 --2- 172.21.15.70:0/4139171818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278415cbd0 0x7f27840cffb0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2774009e50 tx=0x7f277402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79166519f6374bf7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:48.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.267+0000 7f2782ffd640 1 -- 172.21.15.70:0/4139171818 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277402ca70 con 0x7f278415cbd0 2023-08-25T09:45:48.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.267+0000 7f2782ffd640 1 -- 172.21.15.70:0/4139171818 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277402cbd0 con 0x7f278415cbd0 2023-08-25T09:45:48.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.267+0000 7f2782ffd640 1 -- 172.21.15.70:0/4139171818 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740343f0 con 0x7f278415cbd0 2023-08-25T09:45:48.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 -- 172.21.15.70:0/4139171818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278415cbd0 msgr2=0x7f27840cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 --2- 172.21.15.70:0/4139171818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278415cbd0 0x7f27840cffb0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f2774009e50 tx=0x7f277402bed0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 -- 172.21.15.70:0/4139171818 shutdown_connections 2023-08-25T09:45:48.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 --2- 172.21.15.70:0/4139171818 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27840d2a00 0x7f27840d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 --2- 172.21.15.70:0/4139171818 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f278415cbd0 0x7f27840cffb0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 -- 172.21.15.70:0/4139171818 >> 172.21.15.70:0/4139171818 conn(0x7f2784070260 msgr2=0x7f2784070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:48.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 -- 172.21.15.70:0/4139171818 shutdown_connections 2023-08-25T09:45:48.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.268+0000 7f278a9ab640 1 -- 172.21.15.70:0/4139171818 wait complete. 2023-08-25T09:45:48.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.270+0000 7f278a9ab640 1 Processor -- start 2023-08-25T09:45:48.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.270+0000 7f278a9ab640 1 -- start start 2023-08-25T09:45:48.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f278a9ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 0x7f27840cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f278a9ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f278415cbd0 0x7f278415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f278a9ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27841606c0 0x7f27840b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f278a9ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cbc50 con 0x7f27840d2a00 2023-08-25T09:45:48.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f278a9ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27841f6dc0 con 0x7f27841606c0 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f278a9ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cb240 con 0x7f278415cbd0 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f2783fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 0x7f27840cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f2783fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 0x7f27840cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40326/0 (socket says 172.21.15.70:40326) 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f2783fff640 1 -- 172.21.15.70:0/2258861553 learned_addr learned my addr 172.21.15.70:0/2258861553 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f2783fff640 1 -- 172.21.15.70:0/2258861553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f278415cbd0 msgr2=0x7f278415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.271+0000 7f2783fff640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f278415cbd0 0x7f278415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2783fff640 1 -- 172.21.15.70:0/2258861553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27841606c0 msgr2=0x7f27840b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2783fff640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27841606c0 0x7f27840b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2783fff640 1 -- 172.21.15.70:0/2258861553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2774009b30 con 0x7f27840d2a00 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2783fff640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 0x7f27840cb710 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2774009e50 tx=0x7f2774004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277403e450 con 0x7f27840d2a00 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277403e5b0 con 0x7f27840d2a00 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27841f62a0 con 0x7f27840d2a00 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277403e960 con 0x7f27840d2a00 2023-08-25T09:45:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.272+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27841f6ae0 con 0x7f27840d2a00 2023-08-25T09:45:48.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.273+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278415c410 con 0x7f27840d2a00 2023-08-25T09:45:48.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.274+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f277403eac0 con 0x7f27840d2a00 2023-08-25T09:45:48.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.275+0000 7f2780ff9640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276406a050 0x7f276406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.275+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f27740ab590 con 0x7f27840d2a00 2023-08-25T09:45:48.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.276+0000 7f27837fe640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276406a050 0x7f276406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:48.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.276+0000 7f27837fe640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276406a050 0x7f276406c510 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f27840b7ca0 tx=0x7f276c006ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:48.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.280+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2774076ce0 con 0x7f27840d2a00 2023-08-25T09:45:48.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.470+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27841f2900 con 0x7f27840d2a00 2023-08-25T09:45:48.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.474+0000 7f2780ff9640 1 -- 172.21.15.70:0/2258861553 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f277407a300 con 0x7f27840d2a00 2023-08-25T09:45:48.477 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:48.477 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:42.437094+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.b","mappings":[{"from":6,"to":4}]},{"pgid":"14.12","mappings":[{"from":7,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.23","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:48.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276406a050 msgr2=0x7f276406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276406a050 0x7f276406c510 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f27840b7ca0 tx=0x7f276c006ba0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 msgr2=0x7f27840cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 0x7f27840cb710 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f2774009e50 tx=0x7f2774004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 shutdown_connections 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276406a050 0x7f276406c510 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27841606c0 0x7f27840b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f278415cbd0 0x7f278415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.476+0000 7f278a9ab640 1 --2- 172.21.15.70:0/2258861553 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f27840d2a00 0x7f27840cb710 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.477+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 >> 172.21.15.70:0/2258861553 conn(0x7f2784070260 msgr2=0x7f27840d0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:48.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.477+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 shutdown_connections 2023-08-25T09:45:48.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.477+0000 7f278a9ab640 1 -- 172.21.15.70:0/2258861553 wait complete. 2023-08-25T09:45:48.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:45:04.798879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:45:05.478134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:45:48.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:45:48.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.709+0000 7f14ce542640 1 Processor -- start 2023-08-25T09:45:48.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.710+0000 7f14ce542640 1 -- start start 2023-08-25T09:45:48.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.711+0000 7f14ce542640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14c806cdd0 0x7f14c80ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.711+0000 7f14ce542640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c80cd4a0 con 0x7f14c80ad8f0 2023-08-25T09:45:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.711+0000 7f14ce542640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c815d970 con 0x7f14c8159eb0 2023-08-25T09:45:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.711+0000 7f14ce542640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c80c94b0 con 0x7f14c806cdd0 2023-08-25T09:45:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.711+0000 7f14c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14c806cdd0 0x7f14c80ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.712+0000 7f14c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14c806cdd0 0x7f14c80ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40336/0 (socket says 172.21.15.70:40336) 2023-08-25T09:45:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.712+0000 7f14c7fff640 1 -- 172.21.15.70:0/3854247253 learned_addr learned my addr 172.21.15.70:0/3854247253 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.712+0000 7f14c7fff640 1 -- 172.21.15.70:0/3854247253 >> v1:172.21.15.155:6789/0 conn(0x7f14c8159eb0 legacy=0x7f14c815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.712+0000 7f14c7fff640 1 -- 172.21.15.70:0/3854247253 >> v1:172.21.15.70:6789/0 conn(0x7f14c80ad8f0 legacy=0x7f14c815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.712+0000 7f14c7fff640 1 -- 172.21.15.70:0/3854247253 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c80c8c10 con 0x7f14c806cdd0 2023-08-25T09:45:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.712+0000 7f14c6ffd640 1 -- 172.21.15.70:0/3854247253 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729484663 0 0) 0x7f14c80cd4a0 con 0x7f14c80ad8f0 2023-08-25T09:45:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.713+0000 7f14c7fff640 1 --2- 172.21.15.70:0/3854247253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c80ccf60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009e50 tx=0x7f14b802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cb02aa050f7ff6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.713+0000 7f14c6ffd640 1 -- 172.21.15.70:0/3854247253 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b802ca80 con 0x7f14c806cdd0 2023-08-25T09:45:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.713+0000 7f14c6ffd640 1 -- 172.21.15.70:0/3854247253 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b802cbe0 con 0x7f14c806cdd0 2023-08-25T09:45:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.713+0000 7f14c6ffd640 1 -- 172.21.15.70:0/3854247253 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b80343f0 con 0x7f14c806cdd0 2023-08-25T09:45:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 -- 172.21.15.70:0/3854247253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 msgr2=0x7f14c80ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 --2- 172.21.15.70:0/3854247253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c80ccf60 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009e50 tx=0x7f14b802bee0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 -- 172.21.15.70:0/3854247253 shutdown_connections 2023-08-25T09:45:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 --2- 172.21.15.70:0/3854247253 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c80ccf60 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 -- 172.21.15.70:0/3854247253 >> 172.21.15.70:0/3854247253 conn(0x7f14c80cb0d0 msgr2=0x7f14c80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 -- 172.21.15.70:0/3854247253 shutdown_connections 2023-08-25T09:45:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.714+0000 7f14ce542640 1 -- 172.21.15.70:0/3854247253 wait complete. 2023-08-25T09:45:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.716+0000 7f14ce542640 1 Processor -- start 2023-08-25T09:45:48.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.716+0000 7f14ce542640 1 -- start start 2023-08-25T09:45:48.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14ce542640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c8155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14ce542640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80ad8f0 0x7f14c80b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14ce542640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c8159eb0 0x7f14c8156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14ce542640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c80b4040 con 0x7f14c806cdd0 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14ce542640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c81563b0 con 0x7f14c8159eb0 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14ce542640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c81f61c0 con 0x7f14c80ad8f0 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c8155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c8155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40338/0 (socket says 172.21.15.70:40338) 2023-08-25T09:45:48.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.717+0000 7f14c7fff640 1 -- 172.21.15.70:0/1908586441 learned_addr learned my addr 172.21.15.70:0/1908586441 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c7fff640 1 -- 172.21.15.70:0/1908586441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80ad8f0 msgr2=0x7f14c80b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c7fff640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80ad8f0 0x7f14c80b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c7fff640 1 -- 172.21.15.70:0/1908586441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c8159eb0 msgr2=0x7f14c8156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c7fff640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c8159eb0 0x7f14c8156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c7fff640 1 -- 172.21.15.70:0/1908586441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8009b30 con 0x7f14c806cdd0 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c7fff640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c8155e70 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009e50 tx=0x7f14b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:48.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b803e450 con 0x7f14c806cdd0 2023-08-25T09:45:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b803e5d0 con 0x7f14c806cdd0 2023-08-25T09:45:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b803e980 con 0x7f14c806cdd0 2023-08-25T09:45:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.718+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c81f64a0 con 0x7f14c806cdd0 2023-08-25T09:45:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.719+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c81f6a60 con 0x7f14c806cdd0 2023-08-25T09:45:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.720+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c80c8a60 con 0x7f14c806cdd0 2023-08-25T09:45:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.721+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14b802cd50 con 0x7f14c806cdd0 2023-08-25T09:45:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.721+0000 7f14c4ff9640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14a806a050 0x7f14a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.722+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f14b80ab2c0 con 0x7f14c806cdd0 2023-08-25T09:45:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.722+0000 7f14c77fe640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14a806a050 0x7f14a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.723+0000 7f14c77fe640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14a806a050 0x7f14a806c510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f14c8157b30 tx=0x7f14b0007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:48.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.726+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f14b80768f0 con 0x7f14c806cdd0 2023-08-25T09:45:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.912+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14c81f6c50 con 0x7f14c806cdd0 2023-08-25T09:45:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.913+0000 7f14c4ff9640 1 -- 172.21.15.70:0/1908586441 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14b8079f10 con 0x7f14c806cdd0 2023-08-25T09:45:48.916 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:45:48.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.915+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14a806a050 msgr2=0x7f14a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.915+0000 7f14ce542640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14a806a050 0x7f14a806c510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f14c8157b30 tx=0x7f14b0007930 comp rx=0 tx=0).stop 2023-08-25T09:45:48.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.915+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 msgr2=0x7f14c8155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.915+0000 7f14ce542640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c8155e70 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009e50 tx=0x7f14b8004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 shutdown_connections 2023-08-25T09:45:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14a806a050 0x7f14a806c510 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14c8159eb0 0x7f14c8156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14c80ad8f0 0x7f14c80b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 --2- 172.21.15.70:0/1908586441 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14c806cdd0 0x7f14c8155e70 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 >> 172.21.15.70:0/1908586441 conn(0x7f14c80cb0d0 msgr2=0x7f14c8069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 shutdown_connections 2023-08-25T09:45:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:48.916+0000 7f14ce542640 1 -- 172.21.15.70:0/1908586441 wait complete. 2023-08-25T09:45:48.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:45:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.148+0000 7fc8e8a25640 1 Processor -- start 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.148+0000 7fc8e8a25640 1 -- start start 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.149+0000 7fc8e8a25640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e415a920 0x7fc8e40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e8a25640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b1ce0 0x7fc8e40ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e8a25640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40b2170 con 0x7fc8e40b0320 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e8a25640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40ad8f0 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e8a25640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40adbb0 con 0x7fc8e415a920 2023-08-25T09:45:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc8e40b0320 0x7fc8e40b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52888/0 (socket says 172.21.15.70:52888) 2023-08-25T09:45:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e2d9d640 1 -- 172.21.15.70:0/2275465563 learned_addr learned my addr 172.21.15.70:0/2275465563 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e359e640 1 --2- 172.21.15.70:0/2275465563 >> v2:172.21.15.70:3300/0 conn(0x7fc8e40b1ce0 0x7fc8e40ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8dbfff640 1 --2- 172.21.15.70:0/2275465563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e415a920 0x7fc8e40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.150+0000 7fc8e259c640 1 -- 172.21.15.70:0/2275465563 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052948549 0 0) 0x7fc8e40b2170 con 0x7fc8e40b0320 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.151+0000 7fc8e259c640 1 -- 172.21.15.70:0/2275465563 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8c4003520 con 0x7fc8e40b0320 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.151+0000 7fc8e359e640 1 -- 172.21.15.70:0/2275465563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e415a920 msgr2=0x7fc8e40c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.151+0000 7fc8e359e640 1 --2- 172.21.15.70:0/2275465563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e415a920 0x7fc8e40c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.151+0000 7fc8e359e640 1 -- 172.21.15.70:0/2275465563 >> v1:172.21.15.70:6789/0 conn(0x7fc8e40b0320 legacy=0x7fc8e40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.151+0000 7fc8e359e640 1 -- 172.21.15.70:0/2275465563 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40c90a0 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.152+0000 7fc8e359e640 1 --2- 172.21.15.70:0/2275465563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40ad3b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009d90 tx=0x7fc8dc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e74770566ca806 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.152+0000 7fc8e259c640 1 -- 172.21.15.70:0/2275465563 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc02cc50 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.152+0000 7fc8e259c640 1 -- 172.21.15.70:0/2275465563 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc02cdb0 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.152+0000 7fc8e259c640 1 -- 172.21.15.70:0/2275465563 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc034660 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 -- 172.21.15.70:0/2275465563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 msgr2=0x7fc8e40ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/2275465563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40ad3b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009d90 tx=0x7fc8dc02bea0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 -- 172.21.15.70:0/2275465563 shutdown_connections 2023-08-25T09:45:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/2275465563 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40ad3b0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/2275465563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e415a920 0x7fc8e40c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 -- 172.21.15.70:0/2275465563 >> 172.21.15.70:0/2275465563 conn(0x7fc8e40ce830 msgr2=0x7fc8e40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 -- 172.21.15.70:0/2275465563 shutdown_connections 2023-08-25T09:45:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.153+0000 7fc8e8a25640 1 -- 172.21.15.70:0/2275465563 wait complete. 2023-08-25T09:45:49.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.155+0000 7fc8e8a25640 1 Processor -- start 2023-08-25T09:45:49.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.155+0000 7fc8e8a25640 1 -- start start 2023-08-25T09:45:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8e8a25640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e40b0320 0x7fc8e41f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8e8a25640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8e8a25640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8e415a920 0x7fc8e41f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8e8a25640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e4155b70 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8e8a25640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c32a0 con 0x7fc8e415a920 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8e8a25640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40c23f0 con 0x7fc8e40b0320 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38302/0 (socket says 172.21.15.70:38302) 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8dbfff640 1 -- 172.21.15.70:0/1304600002 learned_addr learned my addr 172.21.15.70:0/1304600002 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8dbfff640 1 -- 172.21.15.70:0/1304600002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e40b0320 msgr2=0x7fc8e41f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8dbfff640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e40b0320 0x7fc8e41f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.156+0000 7fc8dbfff640 1 -- 172.21.15.70:0/1304600002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8e415a920 msgr2=0x7fc8e41f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8dbfff640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8e415a920 0x7fc8e41f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8dbfff640 1 -- 172.21.15.70:0/1304600002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc009a70 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8dbfff640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40c1eb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00d620 tx=0x7fc8cc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00b620 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc00b780 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00bb30 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e41f11f0 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.157+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e40c2970 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.158+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e4061460 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.159+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc8cc00bc90 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.160+0000 7fc8db7fe640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8c806a050 0x7fc8c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.160+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fc8cc088b00 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.161+0000 7fc8e2d9d640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8c806a050 0x7fc8c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.161+0000 7fc8e2d9d640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8c806a050 0x7fc8c806c510 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4001360 tx=0x7fc8d40077c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:49.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.165+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc8cc08ae60 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.354+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc8e40c33e0 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.355+0000 7fc8db7fe640 1 -- 172.21.15.70:0/1304600002 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc8cc054130 con 0x7fc8e40b1ce0 2023-08-25T09:45:49.357 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:45:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8c806a050 msgr2=0x7fc8c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8c806a050 0x7fc8c806c510 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4001360 tx=0x7fc8d40077c0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 msgr2=0x7fc8e40c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40c1eb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00d620 tx=0x7fc8cc00dae0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 shutdown_connections 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc8c806a050 0x7fc8c806c510 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc8e415a920 0x7fc8e41f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc8e40b1ce0 0x7fc8e40c1eb0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 --2- 172.21.15.70:0/1304600002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8e40b0320 0x7fc8e41f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 >> 172.21.15.70:0/1304600002 conn(0x7fc8e40ce830 msgr2=0x7fc8e4070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:49.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 shutdown_connections 2023-08-25T09:45:49.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.357+0000 7fc8e8a25640 1 -- 172.21.15.70:0/1304600002 wait complete. 2023-08-25T09:45:49.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:45:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.598+0000 7f4e2da4d640 1 Processor -- start 2023-08-25T09:45:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.598+0000 7f4e2da4d640 1 -- start start 2023-08-25T09:45:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.600+0000 7f4e2da4d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4e28069f60 0x7f4e280ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.600+0000 7f4e2da4d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e280c2550 con 0x7f4e280c2110 2023-08-25T09:45:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.600+0000 7f4e2da4d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e280ce330 con 0x7f4e2815ae50 2023-08-25T09:45:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.600+0000 7f4e2da4d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e280cc5f0 con 0x7f4e28069f60 2023-08-25T09:45:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.600+0000 7f4e267fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4e280c2110 0x7f4e28069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52892/0 (socket says 172.21.15.70:52892) 2023-08-25T09:45:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.600+0000 7f4e267fc640 1 -- 172.21.15.70:0/1360737002 learned_addr learned my addr 172.21.15.70:0/1360737002 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e277fe640 1 --2- 172.21.15.70:0/1360737002 >> v2:172.21.15.155:3300/0 conn(0x7f4e28069f60 0x7f4e280ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64819603 0 0) 0x7f4e280c2550 con 0x7f4e280c2110 2023-08-25T09:45:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e14003520 con 0x7f4e280c2110 2023-08-25T09:45:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 718130199 0 0) 0x7f4e14003520 con 0x7f4e280c2110 2023-08-25T09:45:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 >> v2:172.21.15.155:3300/0 conn(0x7f4e28069f60 msgr2=0x7f4e280ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 --2- 172.21.15.70:0/1360737002 >> v2:172.21.15.155:3300/0 conn(0x7f4e28069f60 0x7f4e280ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 >> v1:172.21.15.155:6789/0 conn(0x7f4e2815ae50 legacy=0x7f4e280cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e28070650 con 0x7f4e280c2110 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4e10001670 con 0x7f4e280c2110 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 >> v1:172.21.15.70:6789/0 conn(0x7f4e280c2110 legacy=0x7f4e28069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e14003970 0x7f4e14003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e140047c0 0x7f4e14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e14005180 0x7f4e1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280cc5f0 con 0x7f4e140047c0 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1400c030 con 0x7f4e14005180 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.601+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1400c1f0 con 0x7f4e14003970 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e267fc640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e140047c0 0x7f4e14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e26ffd640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e14003970 0x7f4e14003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e267fc640 1 -- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e14003970 msgr2=0x7f4e14003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e267fc640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e14003970 0x7f4e14003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e267fc640 1 -- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e14005180 msgr2=0x7f4e1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e267fc640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e14005180 0x7f4e1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e277fe640 1 -- 172.21.15.70:0/1360737002 reap_dead start 2023-08-25T09:45:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e267fc640 1 -- 172.21.15.70:0/1360737002 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e14003520 con 0x7f4e140047c0 2023-08-25T09:45:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.602+0000 7f4e26ffd640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e14003970 0x7f4e14003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:49.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.603+0000 7f4e267fc640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e140047c0 0x7f4e14004c40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c240 tx=0x7f4e140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3618a164bdb610fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:49.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.603+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10005ad0 con 0x7f4e140047c0 2023-08-25T09:45:49.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e10005c30 con 0x7f4e140047c0 2023-08-25T09:45:49.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e25ffb640 1 -- 172.21.15.70:0/1360737002 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1002caf0 con 0x7f4e140047c0 2023-08-25T09:45:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 -- 172.21.15.70:0/1360737002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e140047c0 msgr2=0x7f4e14004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e140047c0 0x7f4e14004c40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c240 tx=0x7f4e140057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 -- 172.21.15.70:0/1360737002 shutdown_connections 2023-08-25T09:45:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/1360737002 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e140047c0 0x7f4e14004c40 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 -- 172.21.15.70:0/1360737002 >> 172.21.15.70:0/1360737002 conn(0x7f4e280cb0d0 msgr2=0x7f4e280cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 -- 172.21.15.70:0/1360737002 shutdown_connections 2023-08-25T09:45:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.604+0000 7f4e2da4d640 1 -- 172.21.15.70:0/1360737002 wait complete. 2023-08-25T09:45:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.606+0000 7f4e2da4d640 1 Processor -- start 2023-08-25T09:45:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.606+0000 7f4e2da4d640 1 -- start start 2023-08-25T09:45:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e2da4d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e140047c0 0x7f4e280af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e2da4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 0x7f4e281f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e2da4d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e281f0920 0x7f4e281f0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e2da4d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280b3d20 con 0x7f4e280b2910 2023-08-25T09:45:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e2da4d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e281f12e0 con 0x7f4e140047c0 2023-08-25T09:45:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e2da4d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e280afeb0 con 0x7f4e281f0920 2023-08-25T09:45:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e267fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 0x7f4e281f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e267fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 0x7f4e281f03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38308/0 (socket says 172.21.15.70:38308) 2023-08-25T09:45:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.607+0000 7f4e267fc640 1 -- 172.21.15.70:0/564488311 learned_addr learned my addr 172.21.15.70:0/564488311 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e26ffd640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e140047c0 0x7f4e280af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e267fc640 1 -- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e281f0920 msgr2=0x7f4e281f0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e267fc640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e281f0920 0x7f4e281f0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e267fc640 1 -- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e140047c0 msgr2=0x7f4e280af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e267fc640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e140047c0 0x7f4e280af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e267fc640 1 -- 172.21.15.70:0/564488311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280b30a0 con 0x7f4e280b2910 2023-08-25T09:45:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.608+0000 7f4e267fc640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 0x7f4e281f03e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c240 tx=0x7f4e1002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.609+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e1003c040 con 0x7f4e280b2910 2023-08-25T09:45:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.609+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1002ccc0 con 0x7f4e280b2910 2023-08-25T09:45:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.609+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e10007290 con 0x7f4e280b2910 2023-08-25T09:45:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.609+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e280b3380 con 0x7f4e280b2910 2023-08-25T09:45:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.609+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e280b3940 con 0x7f4e280b2910 2023-08-25T09:45:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.610+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e28066fb0 con 0x7f4e280b2910 2023-08-25T09:45:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.611+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4e10032070 con 0x7f4e280b2910 2023-08-25T09:45:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.612+0000 7f4e1ffff640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4e0806a050 0x7f4e0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.612+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f4e100a9a90 con 0x7f4e280b2910 2023-08-25T09:45:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.612+0000 7f4e26ffd640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4e0806a050 0x7f4e0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.613+0000 7f4e26ffd640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4e0806a050 0x7f4e0806c510 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000c00 tx=0x7f4e18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.616+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e100750c0 con 0x7f4e280b2910 2023-08-25T09:45:49.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.804+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e28070ec0 con 0x7f4e280b2910 2023-08-25T09:45:49.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.805+0000 7f4e1ffff640 1 -- 172.21.15.70:0/564488311 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e100a9030 con 0x7f4e280b2910 2023-08-25T09:45:49.808 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:45:49.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4e0806a050 msgr2=0x7f4e0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4e0806a050 0x7f4e0806c510 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000c00 tx=0x7f4e18009040 comp rx=0 tx=0).stop 2023-08-25T09:45:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 msgr2=0x7f4e281f03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 0x7f4e281f03e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000c240 tx=0x7f4e1002e070 comp rx=0 tx=0).stop 2023-08-25T09:45:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 shutdown_connections 2023-08-25T09:45:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4e0806a050 0x7f4e0806c510 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4e281f0920 0x7f4e281f0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4e280b2910 0x7f4e281f03e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 --2- 172.21.15.70:0/564488311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4e140047c0 0x7f4e280af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:49.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 >> 172.21.15.70:0/564488311 conn(0x7f4e280cb0d0 msgr2=0x7f4e280cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:49.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 shutdown_connections 2023-08-25T09:45:49.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:49.808+0000 7f4e2da4d640 1 -- 172.21.15.70:0/564488311 wait complete. 2023-08-25T09:45:49.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:45:50.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.038+0000 7f04196a7640 1 Processor -- start 2023-08-25T09:45:50.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.039+0000 7f04196a7640 1 -- start start 2023-08-25T09:45:50.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.040+0000 7f04196a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 0x7f0414071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.040+0000 7f04196a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f04140cef50 0x7f04140cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.040+0000 7f04196a7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04140cf9a0 con 0x7f04140b37c0 2023-08-25T09:45:50.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.040+0000 7f04196a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041415b200 con 0x7f04140cef50 2023-08-25T09:45:50.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.040+0000 7f04196a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cc9f0 con 0x7f041415ae20 2023-08-25T09:45:50.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.040+0000 7f04137fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f04140b37c0 0x7f04140b23f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52908/0 (socket says 172.21.15.70:52908) 2023-08-25T09:45:50.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f04137fe640 1 -- 172.21.15.70:0/4154722289 learned_addr learned my addr 172.21.15.70:0/4154722289 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:50.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0412ffd640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 0x7f0414071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f04127fc640 1 --2- 172.21.15.70:0/4154722289 >> v2:172.21.15.155:3300/0 conn(0x7f04140cef50 0x7f04140cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127090157 0 0) 0x7f04140cf9a0 con 0x7f04140b37c0 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0400003520 con 0x7f04140b37c0 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367713198 0 0) 0x7f0400003520 con 0x7f04140b37c0 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 msgr2=0x7f0414071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 0x7f0414071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 >> v2:172.21.15.155:3300/0 conn(0x7f04140cef50 msgr2=0x7f04140cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 --2- 172.21.15.70:0/4154722289 >> v2:172.21.15.155:3300/0 conn(0x7f04140cef50 0x7f04140cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140cc3e0 con 0x7f04140b37c0 2023-08-25T09:45:50.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f040c001670 con 0x7f04140b37c0 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 >> v1:172.21.15.70:6789/0 conn(0x7f04140b37c0 legacy=0x7f04140b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.041+0000 7f0411ffb640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 0x7f0400003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f0411ffb640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0400003ef0 0x7f04000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f0411ffb640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004d00 0x7f0400005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140cc9f0 con 0x7f0400003ef0 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040000b980 con 0x7f0400004d00 2023-08-25T09:45:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040000bac0 con 0x7f041415ae20 2023-08-25T09:45:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f04127fc640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0400003ef0 0x7f04000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f04127fc640 1 -- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 msgr2=0x7f0400003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f04127fc640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f041415ae20 0x7f0400003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f04127fc640 1 -- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004d00 msgr2=0x7f0400005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f04127fc640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0400004d00 0x7f0400005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.042+0000 7f04127fc640 1 -- 172.21.15.70:0/4154722289 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0400003520 con 0x7f0400003ef0 2023-08-25T09:45:50.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.043+0000 7f04127fc640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0400003ef0 0x7f04000047c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009ac0 tx=0x7f04000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f6a5e3a43100bc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:50.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc02c6d0 con 0x7f0400003ef0 2023-08-25T09:45:50.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc02c830 con 0x7f0400003ef0 2023-08-25T09:45:50.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f0411ffb640 1 -- 172.21.15.70:0/4154722289 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc02cbc0 con 0x7f0400003ef0 2023-08-25T09:45:50.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04196a7640 1 -- 172.21.15.70:0/4154722289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0400003ef0 msgr2=0x7f04000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04196a7640 1 --2- 172.21.15.70:0/4154722289 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0400003ef0 0x7f04000047c0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009ac0 tx=0x7f04000057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04137fe640 1 -- 172.21.15.70:0/4154722289 reap_dead start 2023-08-25T09:45:50.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04196a7640 1 -- 172.21.15.70:0/4154722289 shutdown_connections 2023-08-25T09:45:50.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04196a7640 1 -- 172.21.15.70:0/4154722289 >> 172.21.15.70:0/4154722289 conn(0x7f041406ba60 msgr2=0x7f041406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04196a7640 1 -- 172.21.15.70:0/4154722289 shutdown_connections 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.044+0000 7f04196a7640 1 -- 172.21.15.70:0/4154722289 wait complete. 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.046+0000 7f04196a7640 1 Processor -- start 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.046+0000 7f04196a7640 1 -- start start 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.047+0000 7f04196a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003ef0 0x7f04141f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.047+0000 7f04196a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f041415d7f0 0x7f04140ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.047+0000 7f04196a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 0x7f04140aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.047+0000 7f04196a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140c9a80 con 0x7f04140ca110 2023-08-25T09:45:50.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.047+0000 7f04196a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04140c50d0 con 0x7f041415d7f0 2023-08-25T09:45:50.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.047+0000 7f04196a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04140c8f00 con 0x7f0400003ef0 2023-08-25T09:45:50.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 0x7f04140aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 0x7f04140aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38320/0 (socket says 172.21.15.70:38320) 2023-08-25T09:45:50.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 -- 172.21.15.70:0/1119899197 learned_addr learned my addr 172.21.15.70:0/1119899197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f0412ffd640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003ef0 0x7f04141f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 -- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003ef0 msgr2=0x7f04141f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04127fc640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f041415d7f0 0x7f04140ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003ef0 0x7f04141f1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 -- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f041415d7f0 msgr2=0x7f04140ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f041415d7f0 0x7f04140ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 -- 172.21.15.70:0/1119899197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04140c91e0 con 0x7f04140ca110 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04137fe640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 0x7f04140aedd0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f040c0011e0 tx=0x7f040c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c007440 con 0x7f04140ca110 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c0075a0 con 0x7f04140ca110 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c004a00 con 0x7f04140ca110 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.048+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04140c94c0 con 0x7f04140ca110 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.049+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04141f2270 con 0x7f04140ca110 2023-08-25T09:45:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.050+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04141f2400 con 0x7f04140ca110 2023-08-25T09:45:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.051+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f040c004b60 con 0x7f04140ca110 2023-08-25T09:45:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.052+0000 7f040b7fe640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f03f406a050 0x7f03f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.052+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f040c087910 con 0x7f04140ca110 2023-08-25T09:45:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.052+0000 7f0412ffd640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f03f406a050 0x7f03f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.053+0000 7f0412ffd640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f03f406a050 0x7f03f406c510 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0404004b30 tx=0x7f040400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:50.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.056+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f040c052eb0 con 0x7f04140ca110 2023-08-25T09:45:50.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.231+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0414066fb0 con 0x7f04140ca110 2023-08-25T09:45:50.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.232+0000 7f040b7fe640 1 -- 172.21.15.70:0/1119899197 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f040c0564d0 con 0x7f04140ca110 2023-08-25T09:45:50.235 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:45:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f03f406a050 msgr2=0x7f03f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f03f406a050 0x7f03f406c510 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0404004b30 tx=0x7f040400b040 comp rx=0 tx=0).stop 2023-08-25T09:45:50.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 msgr2=0x7f04140aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 0x7f04140aedd0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f040c0011e0 tx=0x7f040c00c940 comp rx=0 tx=0).stop 2023-08-25T09:45:50.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 shutdown_connections 2023-08-25T09:45:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f03f406a050 0x7f03f406c510 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04140ca110 0x7f04140aedd0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.234+0000 7f04196a7640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f041415d7f0 0x7f04140ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.235+0000 7f04196a7640 1 --2- 172.21.15.70:0/1119899197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0400003ef0 0x7f04141f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.235+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 >> 172.21.15.70:0/1119899197 conn(0x7f041406ba60 msgr2=0x7f04140714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.235+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 shutdown_connections 2023-08-25T09:45:50.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.235+0000 7f04196a7640 1 -- 172.21.15.70:0/1119899197 wait complete. 2023-08-25T09:45:50.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:45:50.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.466+0000 7f824855d640 1 Processor -- start 2023-08-25T09:45:50.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.466+0000 7f824855d640 1 -- start start 2023-08-25T09:45:50.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.468+0000 7f824855d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f8240070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.468+0000 7f824855d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82400cace0 con 0x7f824015df10 2023-08-25T09:45:50.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.468+0000 7f824855d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824006aec0 con 0x7f824006bc80 2023-08-25T09:45:50.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.468+0000 7f824855d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8240159e80 con 0x7f8240159aa0 2023-08-25T09:45:50.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.468+0000 7f8246afa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f824015df10 0x7f82400ca5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52918/0 (socket says 172.21.15.70:52918) 2023-08-25T09:45:50.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.468+0000 7f8246afa640 1 -- 172.21.15.70:0/3656712362 learned_addr learned my addr 172.21.15.70:0/3656712362 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:50.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784237548 0 0) 0x7f82400cace0 con 0x7f824015df10 2023-08-25T09:45:50.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f822c003520 con 0x7f824015df10 2023-08-25T09:45:50.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82462f9640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f8240070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 870454794 0 0) 0x7f822c003520 con 0x7f824015df10 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 msgr2=0x7f8240070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f8240070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 >> v1:172.21.15.155:6789/0 conn(0x7f824006bc80 legacy=0x7f82400c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824006b1a0 con 0x7f824015df10 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8238001670 con 0x7f824015df10 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 >> v1:172.21.15.70:6789/0 conn(0x7f824015df10 legacy=0x7f82400ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f822c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f822c003fd0 0x7f822c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f822c004d00 0x7f822c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240159e80 con 0x7f822c004d00 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c00bb40 con 0x7f822c003fd0 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.469+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c00bd00 con 0x7f8240159aa0 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8246afa640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f822c004d00 0x7f822c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8245af8640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f822c003fd0 0x7f822c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f82462f9640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f822c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8246afa640 1 -- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 msgr2=0x7f822c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8246afa640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f822c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8246afa640 1 -- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f822c003fd0 msgr2=0x7f822c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8246afa640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f822c003fd0 0x7f822c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f8246afa640 1 -- 172.21.15.70:0/3656712362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c003520 con 0x7f822c004d00 2023-08-25T09:45:50.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.470+0000 7f82462f9640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8240159aa0 0x7f822c0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.471+0000 7f8246afa640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f822c004d00 0x7f822c005180 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f823800c210 tx=0x7f822c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838b5df0575d49d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238005ad0 con 0x7f822c004d00 2023-08-25T09:45:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8238005c30 con 0x7f822c004d00 2023-08-25T09:45:50.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f82452f7640 1 -- 172.21.15.70:0/3656712362 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823802cad0 con 0x7f822c004d00 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f824855d640 1 -- 172.21.15.70:0/3656712362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f822c004d00 msgr2=0x7f822c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f824855d640 1 --2- 172.21.15.70:0/3656712362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f822c004d00 0x7f822c005180 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f823800c210 tx=0x7f822c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f8246afa640 1 -- 172.21.15.70:0/3656712362 reap_dead start 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f824855d640 1 -- 172.21.15.70:0/3656712362 shutdown_connections 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f824855d640 1 -- 172.21.15.70:0/3656712362 >> 172.21.15.70:0/3656712362 conn(0x7f82400ce760 msgr2=0x7f82400ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f824855d640 1 -- 172.21.15.70:0/3656712362 shutdown_connections 2023-08-25T09:45:50.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.472+0000 7f824855d640 1 -- 172.21.15.70:0/3656712362 wait complete. 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.474+0000 7f824855d640 1 Processor -- start 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.474+0000 7f824855d640 1 -- start start 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f824855d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f822c004d00 0x7f824015c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f824855d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 0x7f82400b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f824855d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8240157680 0x7f8240157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f824855d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240158370 con 0x7f82400b2ff0 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f824855d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82400b3990 con 0x7f8240157680 2023-08-25T09:45:50.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f824855d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824015a1a0 con 0x7f822c004d00 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f8245af8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 0x7f82400b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f8245af8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 0x7f82400b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38342/0 (socket says 172.21.15.70:38342) 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.475+0000 7f8245af8640 1 -- 172.21.15.70:0/947361751 learned_addr learned my addr 172.21.15.70:0/947361751 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8245af8640 1 -- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f822c004d00 msgr2=0x7f824015c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8245af8640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f822c004d00 0x7f824015c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8246afa640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8240157680 0x7f8240157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8245af8640 1 -- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8240157680 msgr2=0x7f8240157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8245af8640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8240157680 0x7f8240157b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8245af8640 1 -- 172.21.15.70:0/947361751 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824015a480 con 0x7f82400b2ff0 2023-08-25T09:45:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8245af8640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 0x7f82400b3450 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f823000e500 tx=0x7f823000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.476+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82300053a0 con 0x7f82400b2ff0 2023-08-25T09:45:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.477+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82300035f0 con 0x7f82400b2ff0 2023-08-25T09:45:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.477+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82300039a0 con 0x7f82400b2ff0 2023-08-25T09:45:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.477+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824015a760 con 0x7f82400b2ff0 2023-08-25T09:45:50.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.477+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824015ce10 con 0x7f82400b2ff0 2023-08-25T09:45:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.478+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824004e420 con 0x7f82400b2ff0 2023-08-25T09:45:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.479+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8230003b00 con 0x7f82400b2ff0 2023-08-25T09:45:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.479+0000 7f8236ffd640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f822406a050 0x7f822406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.480+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f8230088460 con 0x7f82400b2ff0 2023-08-25T09:45:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.480+0000 7f82462f9640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f822406a050 0x7f822406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:50.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.480+0000 7f82462f9640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f822406a050 0x7f822406c510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f823c009540 tx=0x7f823c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:50.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.484+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8230053a10 con 0x7f82400b2ff0 2023-08-25T09:45:50.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.686+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f82400c8b70 con 0x7f82400b2ff0 2023-08-25T09:45:50.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.687+0000 7f8236ffd640 1 -- 172.21.15.70:0/947361751 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8230003db0 con 0x7f82400b2ff0 2023-08-25T09:45:50.689 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:45:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f822406a050 msgr2=0x7f822406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f822406a050 0x7f822406c510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f823c009540 tx=0x7f823c00b040 comp rx=0 tx=0).stop 2023-08-25T09:45:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 msgr2=0x7f82400b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 0x7f82400b3450 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f823000e500 tx=0x7f823000e9c0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 shutdown_connections 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f822406a050 0x7f822406c510 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8240157680 0x7f8240157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82400b2ff0 0x7f82400b3450 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 --2- 172.21.15.70:0/947361751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f822c004d00 0x7f824015c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.689+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 >> 172.21.15.70:0/947361751 conn(0x7f82400ce760 msgr2=0x7f82400ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.690+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 shutdown_connections 2023-08-25T09:45:50.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:50.690+0000 7f824855d640 1 -- 172.21.15.70:0/947361751 wait complete. 2023-08-25T09:45:50.711 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:45:50.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:50.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:45:50.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[109945]: fuse finished with error 0 and tester_r 0 2023-08-25T09:45:50.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:45:50.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:50.827 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:45:51.015 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:45:51.016 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:51.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:51.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.289+0000 7fcaa7596640 1 Processor -- start 2023-08-25T09:45:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.289+0000 7fcaa7596640 1 -- start start 2023-08-25T09:45:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa7596640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcaa015e830 0x7fcaa006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa7596640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa006b2b0 0x7fcaa015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa7596640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa00b1b60 con 0x7fcaa015c6d0 2023-08-25T09:45:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa7596640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa006bd80 con 0x7fcaa015e830 2023-08-25T09:45:51.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa7596640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa006b7c0 con 0x7fcaa006b2b0 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5332640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcaa015e830 0x7fcaa006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5b33640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcaa015c6d0 0x7fcaa00b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52930/0 (socket says 172.21.15.70:52930) 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5b33640 1 -- 172.21.15.70:0/3673170689 learned_addr learned my addr 172.21.15.70:0/3673170689 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5332640 1 -- 172.21.15.70:0/3673170689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa006b2b0 msgr2=0x7fcaa015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5332640 1 --2- 172.21.15.70:0/3673170689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa006b2b0 0x7fcaa015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5332640 1 -- 172.21.15.70:0/3673170689 >> v1:172.21.15.70:6789/0 conn(0x7fcaa015c6d0 legacy=0x7fcaa00b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.291+0000 7fcaa5332640 1 -- 172.21.15.70:0/3673170689 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa006baa0 con 0x7fcaa015e830 2023-08-25T09:45:51.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.293+0000 7fcaa5332640 1 --2- 172.21.15.70:0/3673170689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015e830 0x7fcaa006ad70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0097a0 tx=0x7fca9c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f139d5a0555975 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.293+0000 7fca97fff640 1 -- 172.21.15.70:0/3673170689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c02d040 con 0x7fcaa015e830 2023-08-25T09:45:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.293+0000 7fca97fff640 1 -- 172.21.15.70:0/3673170689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c031810 con 0x7fcaa015e830 2023-08-25T09:45:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.293+0000 7fca97fff640 1 -- 172.21.15.70:0/3673170689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c031ba0 con 0x7fcaa015e830 2023-08-25T09:45:51.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.293+0000 7fcaa7596640 1 -- 172.21.15.70:0/3673170689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015e830 msgr2=0x7fcaa006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.293+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3673170689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015e830 0x7fcaa006ad70 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0097a0 tx=0x7fca9c02b8b0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.294+0000 7fcaa7596640 1 -- 172.21.15.70:0/3673170689 shutdown_connections 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.294+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3673170689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa006b2b0 0x7fcaa015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.294+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3673170689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015e830 0x7fcaa006ad70 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.294+0000 7fcaa7596640 1 -- 172.21.15.70:0/3673170689 >> 172.21.15.70:0/3673170689 conn(0x7fcaa0069850 msgr2=0x7fcaa0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.294+0000 7fcaa7596640 1 -- 172.21.15.70:0/3673170689 shutdown_connections 2023-08-25T09:45:51.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.294+0000 7fcaa7596640 1 -- 172.21.15.70:0/3673170689 wait complete. 2023-08-25T09:45:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.295+0000 7fcaa7596640 1 Processor -- start 2023-08-25T09:45:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.295+0000 7fcaa7596640 1 -- start start 2023-08-25T09:45:51.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.296+0000 7fcaa7596640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa006b2b0 0x7fcaa01efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.296+0000 7fcaa7596640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 0x7fcaa0154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.296+0000 7fcaa7596640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa015e830 0x7fcaa01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.296+0000 7fcaa7596640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa00a8890 con 0x7fcaa015c6d0 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.296+0000 7fcaa7596640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0140260 con 0x7fcaa006b2b0 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.296+0000 7fcaa7596640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa0153920 con 0x7fcaa015e830 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 0x7fcaa0154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 0x7fcaa0154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38354/0 (socket says 172.21.15.70:38354) 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 -- 172.21.15.70:0/3446963291 learned_addr learned my addr 172.21.15.70:0/3446963291 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 -- 172.21.15.70:0/3446963291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa015e830 msgr2=0x7fcaa01546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa015e830 0x7fcaa01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 -- 172.21.15.70:0/3446963291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa006b2b0 msgr2=0x7fcaa01efb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa006b2b0 0x7fcaa01efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 -- 172.21.15.70:0/3446963291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c009400 con 0x7fcaa015c6d0 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.297+0000 7fcaa4b31640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 0x7fcaa0154160 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b450 tx=0x7fca9000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.298+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011890 con 0x7fcaa015c6d0 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.298+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca900119f0 con 0x7fcaa015c6d0 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.298+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011da0 con 0x7fcaa015c6d0 2023-08-25T09:45:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.298+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa0153bd0 con 0x7fcaa015c6d0 2023-08-25T09:45:51.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.298+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa013fef0 con 0x7fcaa015c6d0 2023-08-25T09:45:51.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.299+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa004e420 con 0x7fcaa015c6d0 2023-08-25T09:45:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.300+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fca9001a400 con 0x7fcaa015c6d0 2023-08-25T09:45:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.301+0000 7fca95ffb640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8406a080 0x7fca8406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.302+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fca90088a50 con 0x7fcaa015c6d0 2023-08-25T09:45:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.302+0000 7fcaa5332640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8406a080 0x7fca8406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:51.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.302+0000 7fcaa5332640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8406a080 0x7fca8406c540 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fca9c036000 tx=0x7fca9c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.306+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fca90053ff0 con 0x7fcaa015c6d0 2023-08-25T09:45:51.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.495+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaa01403a0 con 0x7fcaa015c6d0 2023-08-25T09:45:51.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.496+0000 7fca95ffb640 1 -- 172.21.15.70:0/3446963291 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+18598 (secure 0 0 0) 0x7fca90015090 con 0x7fcaa015c6d0 2023-08-25T09:45:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:51.501 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:42.437094+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.b","mappings":[{"from":6,"to":4}]},{"pgid":"14.12","mappings":[{"from":7,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.23","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:51.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8406a080 msgr2=0x7fca8406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8406a080 0x7fca8406c540 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fca9c036000 tx=0x7fca9c030000 comp rx=0 tx=0).stop 2023-08-25T09:45:51.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 msgr2=0x7fcaa0154160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 0x7fcaa0154160 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fca9000b450 tx=0x7fca9000b910 comp rx=0 tx=0).stop 2023-08-25T09:45:51.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 shutdown_connections 2023-08-25T09:45:51.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fca8406a080 0x7fca8406c540 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcaa015e830 0x7fcaa01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcaa015c6d0 0x7fcaa0154160 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 --2- 172.21.15.70:0/3446963291 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcaa006b2b0 0x7fcaa01efb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 >> 172.21.15.70:0/3446963291 conn(0x7fcaa0069850 msgr2=0x7fcaa00cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:51.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.499+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 shutdown_connections 2023-08-25T09:45:51.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.500+0000 7fcaa7596640 1 -- 172.21.15.70:0/3446963291 wait complete. 2023-08-25T09:45:51.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:45:04.798879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:45:05.478134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:45:51.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:45:51.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.739+0000 7f61918f9640 1 Processor -- start 2023-08-25T09:45:51.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.740+0000 7f61918f9640 1 -- start start 2023-08-25T09:45:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.741+0000 7f61918f9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f618c06a9e0 0x7f618c1610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.741+0000 7f61918f9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c0699f0 con 0x7f618c160200 2023-08-25T09:45:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.741+0000 7f61918f9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c06af30 con 0x7f618c0cd360 2023-08-25T09:45:51.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.741+0000 7f61918f9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c06b210 con 0x7f618c06a9e0 2023-08-25T09:45:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f618c160200 0x7f618c0ceb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52946/0 (socket says 172.21.15.70:52946) 2023-08-25T09:45:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618affd640 1 -- 172.21.15.70:0/1933813471 learned_addr learned my addr 172.21.15.70:0/1933813471 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618b7fe640 1 --2- 172.21.15.70:0/1933813471 >> v2:172.21.15.70:3300/0 conn(0x7f618c06a9e0 0x7f618c1610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:51.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618a7fc640 1 -- 172.21.15.70:0/1933813471 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402164905 0 0) 0x7f618c0699f0 con 0x7f618c160200 2023-08-25T09:45:51.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618a7fc640 1 -- 172.21.15.70:0/1933813471 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6178003520 con 0x7f618c160200 2023-08-25T09:45:51.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618b7fe640 1 -- 172.21.15.70:0/1933813471 >> v1:172.21.15.155:6789/0 conn(0x7f618c0cd360 legacy=0x7f618c1609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:51.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618b7fe640 1 -- 172.21.15.70:0/1933813471 >> v1:172.21.15.70:6789/0 conn(0x7f618c160200 legacy=0x7f618c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.742+0000 7f618b7fe640 1 -- 172.21.15.70:0/1933813471 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c069c90 con 0x7f618c06a9e0 2023-08-25T09:45:51.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.743+0000 7f618b7fe640 1 --2- 172.21.15.70:0/1933813471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c1610f0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6184009d90 tx=0x7f618402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bccdc685e039b10 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:51.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f618a7fc640 1 -- 172.21.15.70:0/1933813471 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618402cc60 con 0x7f618c06a9e0 2023-08-25T09:45:51.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f618a7fc640 1 -- 172.21.15.70:0/1933813471 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618402cdc0 con 0x7f618c06a9e0 2023-08-25T09:45:51.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f618a7fc640 1 -- 172.21.15.70:0/1933813471 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6184034640 con 0x7f618c06a9e0 2023-08-25T09:45:51.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f61918f9640 1 -- 172.21.15.70:0/1933813471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 msgr2=0x7f618c1610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f61918f9640 1 --2- 172.21.15.70:0/1933813471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c1610f0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f6184009d90 tx=0x7f618402beb0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f61918f9640 1 -- 172.21.15.70:0/1933813471 shutdown_connections 2023-08-25T09:45:51.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f61918f9640 1 --2- 172.21.15.70:0/1933813471 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c1610f0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.744+0000 7f61918f9640 1 -- 172.21.15.70:0/1933813471 >> 172.21.15.70:0/1933813471 conn(0x7f618c0cc2a0 msgr2=0x7f618c0cc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:51.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.745+0000 7f61918f9640 1 -- 172.21.15.70:0/1933813471 shutdown_connections 2023-08-25T09:45:51.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.745+0000 7f61918f9640 1 -- 172.21.15.70:0/1933813471 wait complete. 2023-08-25T09:45:51.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.746+0000 7f61918f9640 1 Processor -- start 2023-08-25T09:45:51.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.746+0000 7f61918f9640 1 -- start start 2023-08-25T09:45:51.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.747+0000 7f61918f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c0b2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.747+0000 7f61918f9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f618c0cd360 0x7f618c158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.747+0000 7f61918f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f618c160200 0x7f618c1f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.747+0000 7f61918f9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c142230 con 0x7f618c06a9e0 2023-08-25T09:45:51.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.747+0000 7f61918f9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c142510 con 0x7f618c0cd360 2023-08-25T09:45:51.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.747+0000 7f61918f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c1427f0 con 0x7f618c160200 2023-08-25T09:45:51.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c0b2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:51.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c0b2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38374/0 (socket says 172.21.15.70:38374) 2023-08-25T09:45:51.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 -- 172.21.15.70:0/2762562944 learned_addr learned my addr 172.21.15.70:0/2762562944 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:51.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 -- 172.21.15.70:0/2762562944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f618c160200 msgr2=0x7f618c1f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f618c160200 0x7f618c1f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 -- 172.21.15.70:0/2762562944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f618c0cd360 msgr2=0x7f618c158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f618c0cd360 0x7f618c158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 -- 172.21.15.70:0/2762562944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009a70 con 0x7f618c06a9e0 2023-08-25T09:45:51.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.748+0000 7f618affd640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c0b2a50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c390 tx=0x7f617c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:51.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.749+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c005ce0 con 0x7f618c06a9e0 2023-08-25T09:45:51.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.749+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c005e40 con 0x7f618c06a9e0 2023-08-25T09:45:51.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.749+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c0072d0 con 0x7f618c06a9e0 2023-08-25T09:45:51.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.749+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c142ad0 con 0x7f618c06a9e0 2023-08-25T09:45:51.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.749+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c158c90 con 0x7f618c06a9e0 2023-08-25T09:45:51.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.750+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c0cc0a0 con 0x7f618c06a9e0 2023-08-25T09:45:51.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.751+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f617c007430 con 0x7f618c06a9e0 2023-08-25T09:45:51.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.752+0000 7f6188ff9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f616c06a050 0x7f616c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:51.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.752+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f617c087be0 con 0x7f618c06a9e0 2023-08-25T09:45:51.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.752+0000 7f6183fff640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f616c06a050 0x7f616c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:51.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.753+0000 7f6183fff640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f616c06a050 0x7f616c06c510 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b1ca0 tx=0x7f61740079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:51.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.756+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f617c053230 con 0x7f618c06a9e0 2023-08-25T09:45:51.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.943+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f618c066fb0 con 0x7f618c06a9e0 2023-08-25T09:45:51.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.944+0000 7f6188ff9640 1 -- 172.21.15.70:0/2762562944 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f617c0170a0 con 0x7f618c06a9e0 2023-08-25T09:45:51.947 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:45:51.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f616c06a050 msgr2=0x7f616c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f616c06a050 0x7f616c06c510 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b1ca0 tx=0x7f61740079b0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 msgr2=0x7f618c0b2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:51.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c0b2a50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c390 tx=0x7f617c00c850 comp rx=0 tx=0).stop 2023-08-25T09:45:51.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 shutdown_connections 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f616c06a050 0x7f616c06c510 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f618c160200 0x7f618c1f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f618c0cd360 0x7f618c158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 --2- 172.21.15.70:0/2762562944 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f618c06a9e0 0x7f618c0b2a50 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 >> 172.21.15.70:0/2762562944 conn(0x7f618c0cc2a0 msgr2=0x7f618c0c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 shutdown_connections 2023-08-25T09:45:51.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:51.947+0000 7f61918f9640 1 -- 172.21.15.70:0/2762562944 wait complete. 2023-08-25T09:45:51.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:45:52.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.171+0000 7f3abf2ec640 1 Processor -- start 2023-08-25T09:45:52.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.171+0000 7f3abf2ec640 1 -- start start 2023-08-25T09:45:52.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.173+0000 7f3abf2ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.173+0000 7f3abf2ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80cd4e0 con 0x7f3ab806c510 2023-08-25T09:45:52.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.173+0000 7f3abf2ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80cc550 con 0x7f3ab806a110 2023-08-25T09:45:52.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.173+0000 7f3abf2ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80ccb10 con 0x7f3ab8069d30 2023-08-25T09:45:52.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3abd889640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ab806c510 0x7f3ab806c970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52948/0 (socket says 172.21.15.70:52948) 2023-08-25T09:45:52.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3abd889640 1 -- 172.21.15.70:0/3340461205 learned_addr learned my addr 172.21.15.70:0/3340461205 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:52.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3abd088640 1 --2- 172.21.15.70:0/3340461205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3ab7fff640 1 -- 172.21.15.70:0/3340461205 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366592467 0 0) 0x7f3ab0001110 con 0x7f3ab806c510 2023-08-25T09:45:52.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3ab7fff640 1 -- 172.21.15.70:0/3340461205 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa0003520 con 0x7f3ab806c510 2023-08-25T09:45:52.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3abd088640 1 -- 172.21.15.70:0/3340461205 >> v1:172.21.15.155:6789/0 conn(0x7f3ab806a110 legacy=0x7f3ab80ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3abd088640 1 -- 172.21.15.70:0/3340461205 >> v1:172.21.15.70:6789/0 conn(0x7f3ab806c510 legacy=0x7f3ab806c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.174+0000 7f3abd088640 1 -- 172.21.15.70:0/3340461205 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80ade40 con 0x7f3ab8069d30 2023-08-25T09:45:52.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.176+0000 7f3abd088640 1 --2- 172.21.15.70:0/3340461205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80ccfa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009dc0 tx=0x7f3aa802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c30936d617bf44f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:52.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.176+0000 7f3ab7fff640 1 -- 172.21.15.70:0/3340461205 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa802cc40 con 0x7f3ab8069d30 2023-08-25T09:45:52.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.176+0000 7f3ab7fff640 1 -- 172.21.15.70:0/3340461205 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa802cda0 con 0x7f3ab8069d30 2023-08-25T09:45:52.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.176+0000 7f3ab7fff640 1 -- 172.21.15.70:0/3340461205 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8034660 con 0x7f3ab8069d30 2023-08-25T09:45:52.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 -- 172.21.15.70:0/3340461205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 msgr2=0x7f3ab80ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/3340461205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80ccfa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009dc0 tx=0x7f3aa802be90 comp rx=0 tx=0).stop 2023-08-25T09:45:52.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 -- 172.21.15.70:0/3340461205 shutdown_connections 2023-08-25T09:45:52.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/3340461205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80ccfa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 -- 172.21.15.70:0/3340461205 >> 172.21.15.70:0/3340461205 conn(0x7f3ab80cb0d0 msgr2=0x7f3ab80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:52.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 -- 172.21.15.70:0/3340461205 shutdown_connections 2023-08-25T09:45:52.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.177+0000 7f3abf2ec640 1 -- 172.21.15.70:0/3340461205 wait complete. 2023-08-25T09:45:52.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.179+0000 7f3abf2ec640 1 Processor -- start 2023-08-25T09:45:52.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.179+0000 7f3abf2ec640 1 -- start start 2023-08-25T09:45:52.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abf2ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abf2ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ab806a110 0x7f3ab80c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abf2ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 0x7f3ab80b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abf2ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80c3ba0 con 0x7f3ab806c510 2023-08-25T09:45:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abf2ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80c3e80 con 0x7f3ab806a110 2023-08-25T09:45:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abf2ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80c4160 con 0x7f3ab8069d30 2023-08-25T09:45:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 0x7f3ab80b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 0x7f3ab80b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38386/0 (socket says 172.21.15.70:38386) 2023-08-25T09:45:52.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 -- 172.21.15.70:0/21337980 learned_addr learned my addr 172.21.15.70:0/21337980 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:52.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd088640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abc887640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ab806a110 0x7f3ab80c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 -- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 msgr2=0x7f3ab80c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 -- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ab806a110 msgr2=0x7f3ab80c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ab806a110 0x7f3ab80c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd889640 1 -- 172.21.15.70:0/21337980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa8009a70 con 0x7f3ab806c510 2023-08-25T09:45:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.180+0000 7f3abd088640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.181+0000 7f3abd889640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 0x7f3ab80b2cd0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c700 tx=0x7f3ab000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.181+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0005ab0 con 0x7f3ab806c510 2023-08-25T09:45:52.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.181+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab0005c10 con 0x7f3ab806c510 2023-08-25T09:45:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.181+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80c4440 con 0x7f3ab806c510 2023-08-25T09:45:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.181+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0001d50 con 0x7f3ab806c510 2023-08-25T09:45:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.181+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80b3210 con 0x7f3ab806c510 2023-08-25T09:45:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.182+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab80c4960 con 0x7f3ab806c510 2023-08-25T09:45:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.183+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3ab0007070 con 0x7f3ab806c510 2023-08-25T09:45:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.184+0000 7f3ab5ffb640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a050 0x7f3aa406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.184+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f3ab0087a80 con 0x7f3ab806c510 2023-08-25T09:45:52.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.185+0000 7f3abd088640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a050 0x7f3aa406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.185+0000 7f3abd088640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a050 0x7f3aa406c510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8036000 tx=0x7f3aa8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:52.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.188+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ab80c4960 con 0x7f3ab806c510 2023-08-25T09:45:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.382+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ab80cd4e0 con 0x7f3ab806c510 2023-08-25T09:45:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.382+0000 7f3ab5ffb640 1 -- 172.21.15.70:0/21337980 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3ab80cd4e0 con 0x7f3ab806c510 2023-08-25T09:45:52.385 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:45:52.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a050 msgr2=0x7f3aa406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a050 0x7f3aa406c510 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8036000 tx=0x7f3aa8030000 comp rx=0 tx=0).stop 2023-08-25T09:45:52.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 msgr2=0x7f3ab80b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 0x7f3ab80b2cd0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f3ab000c700 tx=0x7f3ab000cbc0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 shutdown_connections 2023-08-25T09:45:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a050 0x7f3aa406c510 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab806c510 0x7f3ab80b2cd0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ab806a110 0x7f3ab80c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 --2- 172.21.15.70:0/21337980 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab8069d30 0x7f3ab80c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 >> 172.21.15.70:0/21337980 conn(0x7f3ab80cb0d0 msgr2=0x7f3ab806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 shutdown_connections 2023-08-25T09:45:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.385+0000 7f3abf2ec640 1 -- 172.21.15.70:0/21337980 wait complete. 2023-08-25T09:45:52.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:45:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.623+0000 7f91e04e7640 1 Processor -- start 2023-08-25T09:45:52.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.624+0000 7f91e04e7640 1 -- start start 2023-08-25T09:45:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91e04e7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d815e240 0x7f91d8070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91e04e7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f91d80cd660 0x7f91d80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91e04e7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d806f8e0 con 0x7f91d8159bf0 2023-08-25T09:45:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91e04e7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80ce030 con 0x7f91d815e240 2023-08-25T09:45:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91e04e7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d806fbc0 con 0x7f91d80cd660 2023-08-25T09:45:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91dda82640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d815e240 0x7f91d8070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91dda82640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91d815e240 0x7f91d8070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38394/0 (socket says 172.21.15.70:38394) 2023-08-25T09:45:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.625+0000 7f91dda82640 1 -- 172.21.15.70:0/1497585869 learned_addr learned my addr 172.21.15.70:0/1497585869 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:52.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.626+0000 7f91dea84640 1 --2- 172.21.15.70:0/1497585869 >> v2:172.21.15.155:3300/0 conn(0x7f91d80cd660 0x7f91d80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.626+0000 7f91dda82640 1 -- 172.21.15.70:0/1497585869 >> v2:172.21.15.155:3300/0 conn(0x7f91d80cd660 msgr2=0x7f91d80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.626+0000 7f91dda82640 1 --2- 172.21.15.70:0/1497585869 >> v2:172.21.15.155:3300/0 conn(0x7f91d80cd660 0x7f91d80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.626+0000 7f91dda82640 1 -- 172.21.15.70:0/1497585869 >> v1:172.21.15.155:6789/0 conn(0x7f91d8159bf0 legacy=0x7f91d806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.626+0000 7f91dda82640 1 -- 172.21.15.70:0/1497585869 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d806fea0 con 0x7f91d815e240 2023-08-25T09:45:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.627+0000 7f91dda82640 1 --2- 172.21.15.70:0/1497585869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d8070980 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f91d80cbef0 tx=0x7f91c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45d28be4ecb0f5c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.627+0000 7f91dd281640 1 -- 172.21.15.70:0/1497585869 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8031870 con 0x7f91d815e240 2023-08-25T09:45:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.627+0000 7f91dd281640 1 -- 172.21.15.70:0/1497585869 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c80319d0 con 0x7f91d815e240 2023-08-25T09:45:52.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.627+0000 7f91dd281640 1 -- 172.21.15.70:0/1497585869 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8031d60 con 0x7f91d815e240 2023-08-25T09:45:52.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.628+0000 7f91e04e7640 1 -- 172.21.15.70:0/1497585869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 msgr2=0x7f91d8070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.628+0000 7f91e04e7640 1 --2- 172.21.15.70:0/1497585869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d8070980 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f91d80cbef0 tx=0x7f91c802b910 comp rx=0 tx=0).stop 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.628+0000 7f91e04e7640 1 -- 172.21.15.70:0/1497585869 shutdown_connections 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.628+0000 7f91e04e7640 1 --2- 172.21.15.70:0/1497585869 >> v2:172.21.15.155:3300/0 conn(0x7f91d80cd660 0x7f91d80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.628+0000 7f91e04e7640 1 --2- 172.21.15.70:0/1497585869 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d8070980 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.628+0000 7f91e04e7640 1 -- 172.21.15.70:0/1497585869 >> 172.21.15.70:0/1497585869 conn(0x7f91d80cb0d0 msgr2=0x7f91d80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.629+0000 7f91e04e7640 1 -- 172.21.15.70:0/1497585869 shutdown_connections 2023-08-25T09:45:52.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.629+0000 7f91e04e7640 1 -- 172.21.15.70:0/1497585869 wait complete. 2023-08-25T09:45:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.630+0000 7f91e04e7640 1 Processor -- start 2023-08-25T09:45:52.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.630+0000 7f91e04e7640 1 -- start start 2023-08-25T09:45:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.631+0000 7f91e04e7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d80cd660 0x7f91d814e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.631+0000 7f91e04e7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d8159bf0 0x7f91d8149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.631+0000 7f91e04e7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d81517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.631+0000 7f91e04e7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d814a0c0 con 0x7f91d815e240 2023-08-25T09:45:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.631+0000 7f91e04e7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80c8620 con 0x7f91d80cd660 2023-08-25T09:45:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.631+0000 7f91e04e7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80c6700 con 0x7f91d8159bf0 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d81517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d81517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38406/0 (socket says 172.21.15.70:38406) 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 -- 172.21.15.70:0/674758371 learned_addr learned my addr 172.21.15.70:0/674758371 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dda82640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d8159bf0 0x7f91d8149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 -- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d8159bf0 msgr2=0x7f91d8149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d8159bf0 0x7f91d8149b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 -- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d80cd660 msgr2=0x7f91d814e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d80cd660 0x7f91d814e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 -- 172.21.15.70:0/674758371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c80094d0 con 0x7f91d815e240 2023-08-25T09:45:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.632+0000 7f91dea84640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d81517d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f91d0007e40 tx=0x7f91d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.633+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d0014620 con 0x7f91d815e240 2023-08-25T09:45:52.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.633+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000ee90 con 0x7f91d815e240 2023-08-25T09:45:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.633+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80c5af0 con 0x7f91d815e240 2023-08-25T09:45:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.633+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00149e0 con 0x7f91d815e240 2023-08-25T09:45:52.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.633+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80c60b0 con 0x7f91d815e240 2023-08-25T09:45:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.634+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d8066fb0 con 0x7f91d815e240 2023-08-25T09:45:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.635+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f91d0014b40 con 0x7f91d815e240 2023-08-25T09:45:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.636+0000 7f91ceffd640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91bc06a050 0x7f91bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:52.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.636+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f91d0088ff0 con 0x7f91d815e240 2023-08-25T09:45:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.636+0000 7f91de283640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91bc06a050 0x7f91bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:52.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.637+0000 7f91de283640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91bc06a050 0x7f91bc06c510 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f91d4003cc0 tx=0x7f91d40079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:52.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.640+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91d0054680 con 0x7f91d815e240 2023-08-25T09:45:52.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.833+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91d80cbf50 con 0x7f91d815e240 2023-08-25T09:45:52.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.834+0000 7f91ceffd640 1 -- 172.21.15.70:0/674758371 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91d0057ca0 con 0x7f91d815e240 2023-08-25T09:45:52.836 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:45:52.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91bc06a050 msgr2=0x7f91bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91bc06a050 0x7f91bc06c510 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f91d4003cc0 tx=0x7f91d40079b0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 msgr2=0x7f91d81517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d81517d0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f91d0007e40 tx=0x7f91d000e4b0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 shutdown_connections 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f91bc06a050 0x7f91bc06c510 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91d815e240 0x7f91d81517d0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91d8159bf0 0x7f91d8149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 --2- 172.21.15.70:0/674758371 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91d80cd660 0x7f91d814e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:52.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 >> 172.21.15.70:0/674758371 conn(0x7f91d80cb0d0 msgr2=0x7f91d80c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:52.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 shutdown_connections 2023-08-25T09:45:52.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:52.836+0000 7f91e04e7640 1 -- 172.21.15.70:0/674758371 wait complete. 2023-08-25T09:45:52.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:45:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.072+0000 7f931540d640 1 Processor -- start 2023-08-25T09:45:53.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.073+0000 7f931540d640 1 -- start start 2023-08-25T09:45:53.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.074+0000 7f931540d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9310159bf0 0x7f931015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.074+0000 7f931540d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f931006f0e0 0x7f9310070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.074+0000 7f931540d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931015e780 con 0x7f93100cd660 2023-08-25T09:45:53.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.074+0000 7f931540d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931006f8e0 con 0x7f9310159bf0 2023-08-25T09:45:53.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.074+0000 7f931540d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931006fbc0 con 0x7f931006f0e0 2023-08-25T09:45:53.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.074+0000 7f930effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9310159bf0 0x7f931015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f93100cd660 0x7f93100c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52956/0 (socket says 172.21.15.70:52956) 2023-08-25T09:45:53.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9310159bf0 0x7f931015e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38412/0 (socket says 172.21.15.70:38412) 2023-08-25T09:45:53.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930f7fe640 1 -- 172.21.15.70:0/1684416408 learned_addr learned my addr 172.21.15.70:0/1684416408 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f9307fff640 1 --2- 172.21.15.70:0/1684416408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f931006f0e0 0x7f9310070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930effd640 1 -- 172.21.15.70:0/1684416408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f931006f0e0 msgr2=0x7f9310070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930effd640 1 --2- 172.21.15.70:0/1684416408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f931006f0e0 0x7f9310070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930effd640 1 -- 172.21.15.70:0/1684416408 >> v1:172.21.15.70:6789/0 conn(0x7f93100cd660 legacy=0x7f93100c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930effd640 1 -- 172.21.15.70:0/1684416408 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931006fea0 con 0x7f9310159bf0 2023-08-25T09:45:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.075+0000 7f930e7fc640 1 -- 172.21.15.70:0/1684416408 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2099852917 0 0) 0x7f931015e780 con 0x7f93100cd660 2023-08-25T09:45:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.076+0000 7f930effd640 1 --2- 172.21.15.70:0/1684416408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9310159bf0 0x7f931015e240 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f93000097a0 tx=0x7f930002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2331c02e21b9beb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f930e7fc640 1 -- 172.21.15.70:0/1684416408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300031830 con 0x7f9310159bf0 2023-08-25T09:45:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f930e7fc640 1 -- 172.21.15.70:0/1684416408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9300031990 con 0x7f9310159bf0 2023-08-25T09:45:53.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f930e7fc640 1 -- 172.21.15.70:0/1684416408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300031d20 con 0x7f9310159bf0 2023-08-25T09:45:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f931540d640 1 -- 172.21.15.70:0/1684416408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9310159bf0 msgr2=0x7f931015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f931540d640 1 --2- 172.21.15.70:0/1684416408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9310159bf0 0x7f931015e240 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f93000097a0 tx=0x7f930002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f931540d640 1 -- 172.21.15.70:0/1684416408 shutdown_connections 2023-08-25T09:45:53.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f931540d640 1 --2- 172.21.15.70:0/1684416408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f931006f0e0 0x7f9310070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f931540d640 1 --2- 172.21.15.70:0/1684416408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9310159bf0 0x7f931015e240 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.077+0000 7f931540d640 1 -- 172.21.15.70:0/1684416408 >> 172.21.15.70:0/1684416408 conn(0x7f93100cb0d0 msgr2=0x7f93100cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.078+0000 7f931540d640 1 -- 172.21.15.70:0/1684416408 shutdown_connections 2023-08-25T09:45:53.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.078+0000 7f931540d640 1 -- 172.21.15.70:0/1684416408 wait complete. 2023-08-25T09:45:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.079+0000 7f931540d640 1 Processor -- start 2023-08-25T09:45:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.079+0000 7f931540d640 1 -- start start 2023-08-25T09:45:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.080+0000 7f931540d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931006f0e0 0x7f93100c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.080+0000 7f931540d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 0x7f93100acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.080+0000 7f931540d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9310159bf0 0x7f931014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.080+0000 7f931540d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931014ea60 con 0x7f93100cd660 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.080+0000 7f931540d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931014ed40 con 0x7f931006f0e0 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.080+0000 7f931540d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931014f020 con 0x7f9310159bf0 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 0x7f93100acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 0x7f93100acaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38420/0 (socket says 172.21.15.70:38420) 2023-08-25T09:45:53.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 -- 172.21.15.70:0/308826650 learned_addr learned my addr 172.21.15.70:0/308826650 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f930f7fe640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9310159bf0 0x7f931014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f930effd640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931006f0e0 0x7f93100c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 -- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9310159bf0 msgr2=0x7f931014d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9310159bf0 0x7f931014d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 -- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931006f0e0 msgr2=0x7f93100c7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931006f0e0 0x7f93100c7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f9307fff640 1 -- 172.21.15.70:0/308826650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9300009400 con 0x7f93100cd660 2023-08-25T09:45:53.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f930f7fe640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9310159bf0 0x7f931014d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.081+0000 7f930effd640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931006f0e0 0x7f93100c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.082+0000 7f9307fff640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 0x7f93100acaf0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f92f800d800 tx=0x7f92f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.082+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800b6c0 con 0x7f93100cd660 2023-08-25T09:45:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.082+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f800b820 con 0x7f93100cd660 2023-08-25T09:45:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.082+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f800bbb0 con 0x7f93100cd660 2023-08-25T09:45:53.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.082+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931014f300 con 0x7f93100cd660 2023-08-25T09:45:53.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.082+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93100c79a0 con 0x7f93100cd660 2023-08-25T09:45:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.083+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9310066fb0 con 0x7f93100cd660 2023-08-25T09:45:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.084+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f92f800bd10 con 0x7f93100cd660 2023-08-25T09:45:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.085+0000 7f93077fe640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f92f006a050 0x7f92f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.086+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f92f8088ca0 con 0x7f93100cd660 2023-08-25T09:45:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.086+0000 7f930effd640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f92f006a050 0x7f92f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.086+0000 7f930effd640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f92f006a050 0x7f92f006c510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9300036000 tx=0x7f9300008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:53.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.090+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f92f80541b0 con 0x7f93100cd660 2023-08-25T09:45:53.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.280+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9310152530 con 0x7f93100cd660 2023-08-25T09:45:53.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.281+0000 7f93077fe640 1 -- 172.21.15.70:0/308826650 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92f80577d0 con 0x7f93100cd660 2023-08-25T09:45:53.284 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:45:53.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f92f006a050 msgr2=0x7f92f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f92f006a050 0x7f92f006c510 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9300036000 tx=0x7f9300008030 comp rx=0 tx=0).stop 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 msgr2=0x7f93100acaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 0x7f93100acaf0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f92f800d800 tx=0x7f92f800dcc0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 shutdown_connections 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f92f006a050 0x7f92f006c510 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9310159bf0 0x7f931014d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93100cd660 0x7f93100acaf0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 --2- 172.21.15.70:0/308826650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f931006f0e0 0x7f93100c7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 >> 172.21.15.70:0/308826650 conn(0x7f93100cb0d0 msgr2=0x7f93100c20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:53.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 shutdown_connections 2023-08-25T09:45:53.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.284+0000 7f931540d640 1 -- 172.21.15.70:0/308826650 wait complete. 2023-08-25T09:45:53.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:45:53.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.519+0000 7f8846bc4640 1 Processor -- start 2023-08-25T09:45:53.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.519+0000 7f8846bc4640 1 -- start start 2023-08-25T09:45:53.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8846bc4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f88400b3c80 0x7f88400caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8846bc4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f88400cb030 0x7f88400cc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8846bc4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8840159d10 con 0x7f8840159930 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8846bc4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400b2d10 con 0x7f88400cb030 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8846bc4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88400b2ff0 con 0x7f88400b3c80 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8845bc2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8840159930 0x7f88400b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52966/0 (socket says 172.21.15.70:52966) 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f8845bc2640 1 -- 172.21.15.70:0/151850575 learned_addr learned my addr 172.21.15.70:0/151850575 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.521+0000 7f88463c3640 1 --2- 172.21.15.70:0/151850575 >> v2:172.21.15.70:3300/0 conn(0x7f88400cb030 0x7f88400cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f88453c1640 1 --2- 172.21.15.70:0/151850575 >> v2:172.21.15.155:3300/0 conn(0x7f88400b3c80 0x7f88400caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f88463c3640 1 -- 172.21.15.70:0/151850575 >> v2:172.21.15.155:3300/0 conn(0x7f88400b3c80 msgr2=0x7f88400caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:53.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f8844bc0640 1 -- 172.21.15.70:0/151850575 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 143790326 0 0) 0x7f8840159d10 con 0x7f8840159930 2023-08-25T09:45:53.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f88463c3640 1 --2- 172.21.15.70:0/151850575 >> v2:172.21.15.155:3300/0 conn(0x7f88400b3c80 0x7f88400caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f88463c3640 1 -- 172.21.15.70:0/151850575 >> v1:172.21.15.70:6789/0 conn(0x7f8840159930 legacy=0x7f88400b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f88463c3640 1 -- 172.21.15.70:0/151850575 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88400b32d0 con 0x7f88400cb030 2023-08-25T09:45:53.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.522+0000 7f88453c1640 1 --2- 172.21.15.70:0/151850575 >> v2:172.21.15.155:3300/0 conn(0x7f88400b3c80 0x7f88400caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:53.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.523+0000 7f88463c3640 1 --2- 172.21.15.70:0/151850575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f88400cc390 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f88400b1040 tx=0x7f883802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f99bc0c4dad305e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:53.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.523+0000 7f8844bc0640 1 -- 172.21.15.70:0/151850575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883802cbe0 con 0x7f88400cb030 2023-08-25T09:45:53.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.523+0000 7f8844bc0640 1 -- 172.21.15.70:0/151850575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883802cd40 con 0x7f88400cb030 2023-08-25T09:45:53.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.523+0000 7f8844bc0640 1 -- 172.21.15.70:0/151850575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8838034660 con 0x7f88400cb030 2023-08-25T09:45:53.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 -- 172.21.15.70:0/151850575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 msgr2=0x7f88400cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 --2- 172.21.15.70:0/151850575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f88400cc390 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f88400b1040 tx=0x7f883802be30 comp rx=0 tx=0).stop 2023-08-25T09:45:53.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 -- 172.21.15.70:0/151850575 shutdown_connections 2023-08-25T09:45:53.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 --2- 172.21.15.70:0/151850575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f88400cc390 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 --2- 172.21.15.70:0/151850575 >> v2:172.21.15.155:3300/0 conn(0x7f88400b3c80 0x7f88400caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 -- 172.21.15.70:0/151850575 >> 172.21.15.70:0/151850575 conn(0x7f88400aff60 msgr2=0x7f88400b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:53.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 -- 172.21.15.70:0/151850575 shutdown_connections 2023-08-25T09:45:53.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.524+0000 7f8846bc4640 1 -- 172.21.15.70:0/151850575 wait complete. 2023-08-25T09:45:53.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.526+0000 7f8846bc4640 1 Processor -- start 2023-08-25T09:45:53.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.526+0000 7f8846bc4640 1 -- start start 2023-08-25T09:45:53.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8846bc4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88400b3c80 0x7f884014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8846bc4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f884006fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8846bc4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8840159930 0x7f8840070a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8846bc4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884006f770 con 0x7f88400cb030 2023-08-25T09:45:53.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8846bc4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88400700d0 con 0x7f88400b3c80 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8846bc4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f884006eaf0 con 0x7f8840159930 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f884006fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f884006fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38436/0 (socket says 172.21.15.70:38436) 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 -- 172.21.15.70:0/2442044132 learned_addr learned my addr 172.21.15.70:0/2442044132 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8845bc2640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88400b3c80 0x7f884014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 -- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8840159930 msgr2=0x7f8840070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8840159930 0x7f8840070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 -- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88400b3c80 msgr2=0x7f884014ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88400b3c80 0x7f884014ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f88453c1640 1 -- 172.21.15.70:0/2442044132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8838009a70 con 0x7f88400cb030 2023-08-25T09:45:53.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.527+0000 7f8845bc2640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88400b3c80 0x7f884014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:53.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.528+0000 7f88453c1640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f884006fb90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f883000c7f0 tx=0x7f883000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:53.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.528+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830007820 con 0x7f88400cb030 2023-08-25T09:45:53.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.528+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8830007980 con 0x7f88400cb030 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.528+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884006edd0 con 0x7f88400cb030 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.528+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830007cf0 con 0x7f88400cb030 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.528+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884006f390 con 0x7f88400cb030 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.529+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8840066fb0 con 0x7f88400cb030 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.530+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f88300113f0 con 0x7f88400cb030 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.531+0000 7f88367fc640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f882406a050 0x7f882406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:53.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.531+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f88300888b0 con 0x7f88400cb030 2023-08-25T09:45:53.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.531+0000 7f8845bc2640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f882406a050 0x7f882406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:53.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.532+0000 7f8845bc2640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f882406a050 0x7f882406c510 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f883c000f60 tx=0x7f883c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:53.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.535+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8830053f80 con 0x7f88400cb030 2023-08-25T09:45:53.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.737+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8840158b50 con 0x7f88400cb030 2023-08-25T09:45:53.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.738+0000 7f88367fc640 1 -- 172.21.15.70:0/2442044132 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f88300575a0 con 0x7f88400cb030 2023-08-25T09:45:53.740 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:45:53.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f882406a050 msgr2=0x7f882406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f882406a050 0x7f882406c510 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f883c000f60 tx=0x7f883c009040 comp rx=0 tx=0).stop 2023-08-25T09:45:53.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 msgr2=0x7f884006fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:53.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f884006fb90 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f883000c7f0 tx=0x7f883000ccb0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 shutdown_connections 2023-08-25T09:45:53.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f882406a050 0x7f882406c510 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8840159930 0x7f8840070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f88400cb030 0x7f884006fb90 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 --2- 172.21.15.70:0/2442044132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f88400b3c80 0x7f884014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:53.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 >> 172.21.15.70:0/2442044132 conn(0x7f88400aff60 msgr2=0x7f88400cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:53.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.740+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 shutdown_connections 2023-08-25T09:45:53.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:53.741+0000 7f8846bc4640 1 -- 172.21.15.70:0/2442044132 wait complete. 2023-08-25T09:45:53.763 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:45:53.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:53.764 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:45:53.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[90636]: fuse finished with error 0 and tester_r 0 2023-08-25T09:45:53.867 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:45:53.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:53.868 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:45:53.990 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:45:53.990 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:54.051 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:45:54.052 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:45:54.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:54.054 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:45:54.103 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:45:54.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:45:54.105 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:45:54.106 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:45:54.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:45:54.168 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:45:54.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:45:54.168 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:45:54.221 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:45:54.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:45:54.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:54.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.444+0000 7f2ff3e57640 1 Processor -- start 2023-08-25T09:45:54.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.445+0000 7f2ff3e57640 1 -- start start 2023-08-25T09:45:54.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.446+0000 7f2ff3e57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec15e830 0x7f2fec06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.446+0000 7f2ff3e57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fec06b2b0 0x7f2fec15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.446+0000 7f2ff3e57640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fec0b1b60 con 0x7f2fec15c6d0 2023-08-25T09:45:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.446+0000 7f2ff3e57640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fec06bd80 con 0x7f2fec06b2b0 2023-08-25T09:45:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff3e57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec06b7c0 con 0x7f2fec15e830 2023-08-25T09:45:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fec06b2b0 0x7f2fec15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2fec06b2b0 0x7f2fec15c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38440/0 (socket says 172.21.15.70:38440) 2023-08-25T09:45:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 -- 172.21.15.70:0/3110906913 learned_addr learned my addr 172.21.15.70:0/3110906913 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff23f4640 1 --1- 172.21.15.70:0/3110906913 >> v1:172.21.15.70:6789/0 conn(0x7f2fec15c6d0 0x7f2fec0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52982/0 (socket says 172.21.15.70:52982) 2023-08-25T09:45:54.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff1bf3640 1 --2- 172.21.15.70:0/3110906913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec15e830 0x7f2fec06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 -- 172.21.15.70:0/3110906913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec15e830 msgr2=0x7f2fec06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:54.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 --2- 172.21.15.70:0/3110906913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec15e830 0x7f2fec06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 -- 172.21.15.70:0/3110906913 >> v1:172.21.15.70:6789/0 conn(0x7f2fec15c6d0 legacy=0x7f2fec0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.447+0000 7f2ff13f2640 1 -- 172.21.15.70:0/3110906913 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec06baa0 con 0x7f2fec06b2b0 2023-08-25T09:45:54.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.448+0000 7f2ff13f2640 1 --2- 172.21.15.70:0/3110906913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec06b2b0 0x7f2fec15c190 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009870 tx=0x7f2fdc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce080e2fadd4024 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:54.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff0bf1640 1 -- 172.21.15.70:0/3110906913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc030840 con 0x7f2fec06b2b0 2023-08-25T09:45:54.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff0bf1640 1 -- 172.21.15.70:0/3110906913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc0309a0 con 0x7f2fec06b2b0 2023-08-25T09:45:54.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff0bf1640 1 -- 172.21.15.70:0/3110906913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc030d30 con 0x7f2fec06b2b0 2023-08-25T09:45:54.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 -- 172.21.15.70:0/3110906913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec06b2b0 msgr2=0x7f2fec15c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/3110906913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec06b2b0 0x7f2fec15c190 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009870 tx=0x7f2fdc02b980 comp rx=0 tx=0).stop 2023-08-25T09:45:54.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 -- 172.21.15.70:0/3110906913 shutdown_connections 2023-08-25T09:45:54.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/3110906913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec06b2b0 0x7f2fec15c190 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/3110906913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec15e830 0x7f2fec06ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 -- 172.21.15.70:0/3110906913 >> 172.21.15.70:0/3110906913 conn(0x7f2fec069850 msgr2=0x7f2fec069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:54.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 -- 172.21.15.70:0/3110906913 shutdown_connections 2023-08-25T09:45:54.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.449+0000 7f2ff3e57640 1 -- 172.21.15.70:0/3110906913 wait complete. 2023-08-25T09:45:54.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.451+0000 7f2ff3e57640 1 Processor -- start 2023-08-25T09:45:54.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.451+0000 7f2ff3e57640 1 -- start start 2023-08-25T09:45:54.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff3e57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec06b2b0 0x7f2fec158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff3e57640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fec15c6d0 0x7f2fec1f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff3e57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 0x7f2fec0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff3e57640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec1e42c0 con 0x7f2fec15e830 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff3e57640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0ce810 con 0x7f2fec15c6d0 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff3e57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec0ceaf0 con 0x7f2fec06b2b0 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff23f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 0x7f2fec0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.452+0000 7f2ff23f4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 0x7f2fec0ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38456/0 (socket says 172.21.15.70:38456) 2023-08-25T09:45:54.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 -- 172.21.15.70:0/2633751362 learned_addr learned my addr 172.21.15.70:0/2633751362 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:54.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 -- 172.21.15.70:0/2633751362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec06b2b0 msgr2=0x7f2fec158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec06b2b0 0x7f2fec158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 -- 172.21.15.70:0/2633751362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fec15c6d0 msgr2=0x7f2fec1f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fec15c6d0 0x7f2fec1f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 -- 172.21.15.70:0/2633751362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc0094d0 con 0x7f2fec15e830 2023-08-25T09:45:54.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2ff23f4640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 0x7f2fec0ca0d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c780 tx=0x7f2fe400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:54.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400ce90 con 0x7f2fec15e830 2023-08-25T09:45:54.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe400a640 con 0x7f2fec15e830 2023-08-25T09:45:54.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.453+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400a9f0 con 0x7f2fec15e830 2023-08-25T09:45:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.454+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fec0cedd0 con 0x7f2fec15e830 2023-08-25T09:45:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.454+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fec1f2450 con 0x7f2fec15e830 2023-08-25T09:45:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.455+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fec1e4750 con 0x7f2fec15e830 2023-08-25T09:45:54.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.456+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2fe400ab50 con 0x7f2fec15e830 2023-08-25T09:45:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.456+0000 7f2fe27fc640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.457+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f2fe4087e30 con 0x7f2fec15e830 2023-08-25T09:45:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.457+0000 7f2ff1bf3640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.457+0000 7f2ff1bf3640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800a810 tx=0x7f2fe8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.462+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2fec1e4750 con 0x7f2fec15e830 2023-08-25T09:45:54.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.691+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fec1e4750 con 0x7f2fec15e830 2023-08-25T09:45:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.693+0000 7f2fe27fc640 1 -- 172.21.15.70:0/2633751362 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f2fec1e4750 con 0x7f2fec15e830 2023-08-25T09:45:54.696 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:54.696 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 msgr2=0x7f2fd006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800a810 tx=0x7f2fe8008040 comp rx=0 tx=0).stop 2023-08-25T09:45:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 msgr2=0x7f2fec0ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 0x7f2fec0ca0d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400c780 tx=0x7f2fe400cc40 comp rx=0 tx=0).stop 2023-08-25T09:45:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 shutdown_connections 2023-08-25T09:45:54.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fd006a050 0x7f2fd006c510 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2fec15e830 0x7f2fec0ca0d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2fec15c6d0 0x7f2fec1f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 --2- 172.21.15.70:0/2633751362 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2fec06b2b0 0x7f2fec158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 >> 172.21.15.70:0/2633751362 conn(0x7f2fec069850 msgr2=0x7f2fec0b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 shutdown_connections 2023-08-25T09:45:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.696+0000 7f2ff3e57640 1 -- 172.21.15.70:0/2633751362 wait complete. 2023-08-25T09:45:54.718 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:54.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.923+0000 7f3775283640 1 Processor -- start 2023-08-25T09:45:54.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.924+0000 7f3775283640 1 -- start start 2023-08-25T09:45:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.925+0000 7f3775283640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f377015ec10 0x7f377015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.925+0000 7f3775283640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37700b1b60 con 0x7f377015e830 2023-08-25T09:45:54.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.925+0000 7f3775283640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377006b7c0 con 0x7f377015c6d0 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.925+0000 7f3775283640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377006bd80 con 0x7f377015ec10 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f377015e830 0x7f377006ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52998/0 (socket says 172.21.15.70:52998) 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376effd640 1 -- 172.21.15.70:0/2064394443 learned_addr learned my addr 172.21.15.70:0/2064394443 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939199254 0 0) 0x7f37700b1b60 con 0x7f377015e830 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f375c003520 con 0x7f377015e830 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f3767fff640 1 --2- 172.21.15.70:0/2064394443 >> v2:172.21.15.155:3300/0 conn(0x7f377015ec10 0x7f377015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164595745 0 0) 0x7f375c003520 con 0x7f377015e830 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 >> v2:172.21.15.155:3300/0 conn(0x7f377015ec10 msgr2=0x7f377015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 --2- 172.21.15.70:0/2064394443 >> v2:172.21.15.155:3300/0 conn(0x7f377015ec10 0x7f377015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 >> v1:172.21.15.155:6789/0 conn(0x7f377015c6d0 legacy=0x7f37700b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377006baa0 con 0x7f377015e830 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.926+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3760001670 con 0x7f377015e830 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 >> v1:172.21.15.70:6789/0 conn(0x7f377015e830 legacy=0x7f377006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f375c003970 0x7f375c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f375c0047c0 0x7f375c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c0052a0 0x7f375c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377006bd80 con 0x7f375c003970 2023-08-25T09:45:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c00c150 con 0x7f375c0052a0 2023-08-25T09:45:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c00c310 con 0x7f375c0047c0 2023-08-25T09:45:54.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376effd640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f375c003970 0x7f375c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376effd640 1 -- 172.21.15.70:0/2064394443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f375c0047c0 msgr2=0x7f375c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:54.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376effd640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f375c0047c0 0x7f375c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376effd640 1 -- 172.21.15.70:0/2064394443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c0052a0 msgr2=0x7f375c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:54.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376effd640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c0052a0 0x7f375c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376effd640 1 -- 172.21.15.70:0/2064394443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c003520 con 0x7f375c003970 2023-08-25T09:45:54.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.927+0000 7f376f7fe640 1 -- 172.21.15.70:0/2064394443 reap_dead start 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.928+0000 7f376effd640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f375c003970 0x7f375c003d70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f376000c210 tx=0x7f375c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f0215cee7ea033 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760005ad0 con 0x7f375c003970 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760005c30 con 0x7f375c003970 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f376e7fc640 1 -- 172.21.15.70:0/2064394443 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376002c890 con 0x7f375c003970 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 -- 172.21.15.70:0/2064394443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f375c003970 msgr2=0x7f375c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f375c003970 0x7f375c003d70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f376000c210 tx=0x7f375c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 -- 172.21.15.70:0/2064394443 shutdown_connections 2023-08-25T09:45:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 --2- 172.21.15.70:0/2064394443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f375c003970 0x7f375c003d70 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 -- 172.21.15.70:0/2064394443 >> 172.21.15.70:0/2064394443 conn(0x7f3770069850 msgr2=0x7f3770069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 -- 172.21.15.70:0/2064394443 shutdown_connections 2023-08-25T09:45:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.929+0000 7f3775283640 1 -- 172.21.15.70:0/2064394443 wait complete. 2023-08-25T09:45:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.931+0000 7f3775283640 1 Processor -- start 2023-08-25T09:45:54.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.931+0000 7f3775283640 1 -- start start 2023-08-25T09:45:54.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.932+0000 7f3775283640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c003970 0x7f37701588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.932+0000 7f3775283640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 0x7f377006ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.932+0000 7f3775283640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377006f220 0x7f37700a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.932+0000 7f3775283640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770070740 con 0x7f37701f22d0 2023-08-25T09:45:54.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.932+0000 7f3775283640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37700702b0 con 0x7f375c003970 2023-08-25T09:45:54.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.932+0000 7f3775283640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377006f6a0 con 0x7f377006f220 2023-08-25T09:45:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 0x7f377006ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 0x7f377006ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38468/0 (socket says 172.21.15.70:38468) 2023-08-25T09:45:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 -- 172.21.15.70:0/1994996453 learned_addr learned my addr 172.21.15.70:0/1994996453 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f376effd640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c003970 0x7f37701588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 -- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377006f220 msgr2=0x7f37700a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377006f220 0x7f37700a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 -- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c003970 msgr2=0x7f37701588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c003970 0x7f37701588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f3767fff640 1 -- 172.21.15.70:0/1994996453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377006f980 con 0x7f37701f22d0 2023-08-25T09:45:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.933+0000 7f376effd640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c003970 0x7f37701588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.934+0000 7f3767fff640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 0x7f377006ece0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f375800d550 tx=0x7f375800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.934+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580125f0 con 0x7f37701f22d0 2023-08-25T09:45:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.934+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3758012750 con 0x7f37701f22d0 2023-08-25T09:45:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.934+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758012b00 con 0x7f37701f22d0 2023-08-25T09:45:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.934+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377006fc60 con 0x7f37701f22d0 2023-08-25T09:45:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.934+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37700a89a0 con 0x7f37701f22d0 2023-08-25T09:45:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.935+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377004e420 con 0x7f37701f22d0 2023-08-25T09:45:54.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.936+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3758012c60 con 0x7f37701f22d0 2023-08-25T09:45:54.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.937+0000 7f37677fe640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f375006a080 0x7f375006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:54.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.937+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f37580889a0 con 0x7f37701f22d0 2023-08-25T09:45:54.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.937+0000 7f376effd640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f375006a080 0x7f375006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:54.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.938+0000 7f376effd640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f375006a080 0x7f375006c540 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3760034000 tx=0x7f3760009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:54.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:54.942+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3758053f40 con 0x7f37701f22d0 2023-08-25T09:45:55.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.139+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f377006c620 con 0x7f37701f22d0 2023-08-25T09:45:55.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.143+0000 7f37677fe640 1 -- 172.21.15.70:0/1994996453 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f3758057560 con 0x7f37701f22d0 2023-08-25T09:45:55.146 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:55.146 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:42.437094+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.b","mappings":[{"from":6,"to":4}]},{"pgid":"14.12","mappings":[{"from":7,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.23","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.145+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f375006a080 msgr2=0x7f375006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.145+0000 7f3775283640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f375006a080 0x7f375006c540 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3760034000 tx=0x7f3760009030 comp rx=0 tx=0).stop 2023-08-25T09:45:55.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.145+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 msgr2=0x7f377006ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.145+0000 7f3775283640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 0x7f377006ece0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f375800d550 tx=0x7f375800da10 comp rx=0 tx=0).stop 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 shutdown_connections 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f375006a080 0x7f375006c540 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f377006f220 0x7f37700a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37701f22d0 0x7f377006ece0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 --2- 172.21.15.70:0/1994996453 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f375c003970 0x7f37701588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 >> 172.21.15.70:0/1994996453 conn(0x7f3770069850 msgr2=0x7f37700b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 shutdown_connections 2023-08-25T09:45:55.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.146+0000 7f3775283640 1 -- 172.21.15.70:0/1994996453 wait complete. 2023-08-25T09:45:55.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:45:04.798879+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:45:05.478134+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:45:55.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:45:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.389+0000 7ff648a5d640 1 Processor -- start 2023-08-25T09:45:55.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.389+0000 7ff648a5d640 1 -- start start 2023-08-25T09:45:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.389+0000 7ff648a5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff644162630 0x7ff644164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.389+0000 7ff648a5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6440cd800 con 0x7ff6440b2470 2023-08-25T09:45:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.389+0000 7ff648a5d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64415a110 con 0x7ff6440b2850 2023-08-25T09:45:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.389+0000 7ff648a5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64415ad20 con 0x7ff644162630 2023-08-25T09:45:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff6437fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff6440b2470 0x7ff644159930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53012/0 (socket says 172.21.15.70:53012) 2023-08-25T09:45:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff6437fe640 1 -- 172.21.15.70:0/725183832 learned_addr learned my addr 172.21.15.70:0/725183832 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff643fff640 1 --2- 172.21.15.70:0/725183832 >> v2:172.21.15.70:3300/0 conn(0x7ff644162630 0x7ff644164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff643fff640 1 -- 172.21.15.70:0/725183832 >> v1:172.21.15.155:6789/0 conn(0x7ff6440b2850 legacy=0x7ff6440cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff643fff640 1 -- 172.21.15.70:0/725183832 >> v1:172.21.15.70:6789/0 conn(0x7ff6440b2470 legacy=0x7ff644159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff643fff640 1 -- 172.21.15.70:0/725183832 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64415a3f0 con 0x7ff644162630 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.390+0000 7ff642ffd640 1 -- 172.21.15.70:0/725183832 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585472353 0 0) 0x7ff6440cd800 con 0x7ff6440b2470 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.391+0000 7ff643fff640 1 --2- 172.21.15.70:0/725183832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff644162630 0x7ff644164a20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009d90 tx=0x7ff63c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd81b99d699816f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff642ffd640 1 -- 172.21.15.70:0/725183832 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c02cc60 con 0x7ff644162630 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff642ffd640 1 -- 172.21.15.70:0/725183832 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c02cdc0 con 0x7ff644162630 2023-08-25T09:45:55.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff642ffd640 1 -- 172.21.15.70:0/725183832 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c034660 con 0x7ff644162630 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 -- 172.21.15.70:0/725183832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff644162630 msgr2=0x7ff644164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 --2- 172.21.15.70:0/725183832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff644162630 0x7ff644164a20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009d90 tx=0x7ff63c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 -- 172.21.15.70:0/725183832 shutdown_connections 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 --2- 172.21.15.70:0/725183832 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff644162630 0x7ff644164a20 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 -- 172.21.15.70:0/725183832 >> 172.21.15.70:0/725183832 conn(0x7ff6440ce830 msgr2=0x7ff6440cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 -- 172.21.15.70:0/725183832 shutdown_connections 2023-08-25T09:45:55.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.392+0000 7ff648a5d640 1 -- 172.21.15.70:0/725183832 wait complete. 2023-08-25T09:45:55.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.394+0000 7ff648a5d640 1 Processor -- start 2023-08-25T09:45:55.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.394+0000 7ff648a5d640 1 -- start start 2023-08-25T09:45:55.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff648a5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 0x7ff644070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff648a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6440b2850 0x7ff6441f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff648a5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff644162630 0x7ff6440c26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff648a5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440700b0 con 0x7ff6440b2470 2023-08-25T09:45:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff648a5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441f5590 con 0x7ff644162630 2023-08-25T09:45:55.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff648a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6440c24c0 con 0x7ff6440b2850 2023-08-25T09:45:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff6437fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 0x7ff644070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff6437fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 0x7ff644070540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38482/0 (socket says 172.21.15.70:38482) 2023-08-25T09:45:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.395+0000 7ff6437fe640 1 -- 172.21.15.70:0/490787948 learned_addr learned my addr 172.21.15.70:0/490787948 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff6437fe640 1 -- 172.21.15.70:0/490787948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6440b2850 msgr2=0x7ff6441f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff6437fe640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6440b2850 0x7ff6441f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff6437fe640 1 -- 172.21.15.70:0/490787948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff644162630 msgr2=0x7ff6440c26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff6437fe640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff644162630 0x7ff6440c26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff6437fe640 1 -- 172.21.15.70:0/490787948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c009a70 con 0x7ff6440b2470 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff6437fe640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 0x7ff644070540 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c520 tx=0x7ff63400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634007480 con 0x7ff6440b2470 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6340075e0 con 0x7ff6440b2470 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6340059c0 con 0x7ff6440b2470 2023-08-25T09:45:55.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.396+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64406faa0 con 0x7ff6440b2470 2023-08-25T09:45:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.397+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64406fd40 con 0x7ff6440b2470 2023-08-25T09:45:55.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.398+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6440b2cb0 con 0x7ff6440b2470 2023-08-25T09:45:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.399+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff634005b20 con 0x7ff6440b2470 2023-08-25T09:45:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.399+0000 7ff640ff9640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff62806a050 0x7ff62806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.400+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7ff634087ca0 con 0x7ff6440b2470 2023-08-25T09:45:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.400+0000 7ff63bfff640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff62806a050 0x7ff62806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.401+0000 7ff63bfff640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff62806a050 0x7ff62806c510 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff6440c1b70 tx=0x7ff62c0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:55.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.405+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff6440b2cb0 con 0x7ff6440b2470 2023-08-25T09:45:55.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.594+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6440b2cb0 con 0x7ff6440b2470 2023-08-25T09:45:55.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.595+0000 7ff640ff9640 1 -- 172.21.15.70:0/490787948 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6440b2cb0 con 0x7ff6440b2470 2023-08-25T09:45:55.597 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:45:55.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff62806a050 msgr2=0x7ff62806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff62806a050 0x7ff62806c510 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff6440c1b70 tx=0x7ff62c0079b0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 msgr2=0x7ff644070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 0x7ff644070540 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c520 tx=0x7ff63400c9e0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 shutdown_connections 2023-08-25T09:45:55.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff62806a050 0x7ff62806c510 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff644162630 0x7ff6440c26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff6440b2850 0x7ff6441f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 --2- 172.21.15.70:0/490787948 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff6440b2470 0x7ff644070540 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 >> 172.21.15.70:0/490787948 conn(0x7ff6440ce830 msgr2=0x7ff6440c8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:55.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 shutdown_connections 2023-08-25T09:45:55.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.598+0000 7ff648a5d640 1 -- 172.21.15.70:0/490787948 wait complete. 2023-08-25T09:45:55.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:45:55.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.837+0000 7f065ec3f640 1 Processor -- start 2023-08-25T09:45:55.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.837+0000 7f065ec3f640 1 -- start start 2023-08-25T09:45:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.838+0000 7f065ec3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 0x7f0658070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.838+0000 7f065ec3f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06580cd660 0x7f06580cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.838+0000 7f065ec3f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06580ce000 con 0x7f0658159bf0 2023-08-25T09:45:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.838+0000 7f065ec3f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065806f8e0 con 0x7f06580cd660 2023-08-25T09:45:55.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.838+0000 7f065ec3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f065806fbc0 con 0x7f065815e240 2023-08-25T09:45:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065c9db640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0658159bf0 0x7f065806f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53022/0 (socket says 172.21.15.70:53022) 2023-08-25T09:45:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065c9db640 1 -- 172.21.15.70:0/1918873221 learned_addr learned my addr 172.21.15.70:0/1918873221 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065d1dc640 1 --2- 172.21.15.70:0/1918873221 >> v2:172.21.15.70:3300/0 conn(0x7f06580cd660 0x7f06580cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065d1dc640 1 -- 172.21.15.70:0/1918873221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 msgr2=0x7f0658070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:55.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065d1dc640 1 --2- 172.21.15.70:0/1918873221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 0x7f0658070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065d1dc640 1 -- 172.21.15.70:0/1918873221 >> v1:172.21.15.70:6789/0 conn(0x7f0658159bf0 legacy=0x7f065806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f065d1dc640 1 -- 172.21.15.70:0/1918873221 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065806fea0 con 0x7f06580cd660 2023-08-25T09:45:55.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.839+0000 7f06577fe640 1 -- 172.21.15.70:0/1918873221 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681134896 0 0) 0x7f06580ce000 con 0x7f0658159bf0 2023-08-25T09:45:55.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.840+0000 7f065d1dc640 1 --2- 172.21.15.70:0/1918873221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580cdac0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0650009d90 tx=0x7f065002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deccdbf83bcb345c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:55.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f06577fe640 1 -- 172.21.15.70:0/1918873221 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065002cc50 con 0x7f06580cd660 2023-08-25T09:45:55.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f06577fe640 1 -- 172.21.15.70:0/1918873221 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065002cdb0 con 0x7f06580cd660 2023-08-25T09:45:55.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f06577fe640 1 -- 172.21.15.70:0/1918873221 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650034660 con 0x7f06580cd660 2023-08-25T09:45:55.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 -- 172.21.15.70:0/1918873221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 msgr2=0x7f06580cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1918873221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580cdac0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0650009d90 tx=0x7f065002bea0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 -- 172.21.15.70:0/1918873221 shutdown_connections 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1918873221 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580cdac0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1918873221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 0x7f0658070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 -- 172.21.15.70:0/1918873221 >> 172.21.15.70:0/1918873221 conn(0x7f06580cb0d0 msgr2=0x7f06580cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 -- 172.21.15.70:0/1918873221 shutdown_connections 2023-08-25T09:45:55.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.841+0000 7f065ec3f640 1 -- 172.21.15.70:0/1918873221 wait complete. 2023-08-25T09:45:55.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.843+0000 7f065ec3f640 1 Processor -- start 2023-08-25T09:45:55.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.843+0000 7f065ec3f640 1 -- start start 2023-08-25T09:45:55.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065ec3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580c1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065ec3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0658159bf0 0x7f06581574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065ec3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 0x7f0658157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065ec3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06581503b0 con 0x7f06580cd660 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065ec3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658150690 con 0x7f0658159bf0 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065ec3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0658150970 con 0x7f065815e240 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.844+0000 7f065c9db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580c1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38500/0 (socket says 172.21.15.70:38500) 2023-08-25T09:45:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 -- 172.21.15.70:0/1919669037 learned_addr learned my addr 172.21.15.70:0/1919669037 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 -- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 msgr2=0x7f0658157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 0x7f0658157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 -- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0658159bf0 msgr2=0x7f06581574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f0657fff640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0658159bf0 0x7f06581574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0658159bf0 0x7f06581574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 -- 172.21.15.70:0/1919669037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650009a70 con 0x7f06580cd660 2023-08-25T09:45:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.845+0000 7f065c9db640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580c1070 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c4d0 tx=0x7f064c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.846+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c0074e0 con 0x7f06580cd660 2023-08-25T09:45:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.846+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c007660 con 0x7f06580cd660 2023-08-25T09:45:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.846+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c004a20 con 0x7f06580cd660 2023-08-25T09:45:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.846+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0658150c50 con 0x7f06580cd660 2023-08-25T09:45:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.846+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06580c15b0 con 0x7f06580cd660 2023-08-25T09:45:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.847+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0658151170 con 0x7f06580cd660 2023-08-25T09:45:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.848+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f064c004b80 con 0x7f06580cd660 2023-08-25T09:45:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.849+0000 7f06557fa640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f063c06a050 0x7f063c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.849+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f064c087860 con 0x7f06580cd660 2023-08-25T09:45:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.849+0000 7f0657fff640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f063c06a050 0x7f063c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.849+0000 7f0657fff640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f063c06a050 0x7f063c06c510 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f06580c4950 tx=0x7f0644008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:55.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:55.853+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0658151170 con 0x7f06580cd660 2023-08-25T09:45:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.040+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0658151170 con 0x7f06580cd660 2023-08-25T09:45:56.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.041+0000 7f06557fa640 1 -- 172.21.15.70:0/1919669037 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v121) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0658151170 con 0x7f06580cd660 2023-08-25T09:45:56.043 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:45:56.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f063c06a050 msgr2=0x7f063c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f063c06a050 0x7f063c06c510 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f06580c4950 tx=0x7f0644008040 comp rx=0 tx=0).stop 2023-08-25T09:45:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 msgr2=0x7f06580c1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580c1070 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f064c00c4d0 tx=0x7f064c00c990 comp rx=0 tx=0).stop 2023-08-25T09:45:56.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 shutdown_connections 2023-08-25T09:45:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f063c06a050 0x7f063c06c510 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f065815e240 0x7f0658157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0658159bf0 0x7f06581574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 --2- 172.21.15.70:0/1919669037 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06580cd660 0x7f06580c1070 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.043+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 >> 172.21.15.70:0/1919669037 conn(0x7f06580cb0d0 msgr2=0x7f065815d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.044+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 shutdown_connections 2023-08-25T09:45:56.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.044+0000 7f065ec3f640 1 -- 172.21.15.70:0/1919669037 wait complete. 2023-08-25T09:45:56.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:45:56.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.283+0000 7f10ae84d640 1 Processor -- start 2023-08-25T09:45:56.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.283+0000 7f10ae84d640 1 -- start start 2023-08-25T09:45:56.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.284+0000 7f10ae84d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a8069850 0x7f10a815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.284+0000 7f10ae84d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10a80c9620 0x7f10a8069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10ae84d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a80ac880 con 0x7f10a8160be0 2023-08-25T09:45:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10ae84d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a80ad400 con 0x7f10a8069850 2023-08-25T09:45:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10ae84d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a80ad890 con 0x7f10a80c9620 2023-08-25T09:45:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a8069850 0x7f10a815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a8069850 0x7f10a815e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38512/0 (socket says 172.21.15.70:38512) 2023-08-25T09:45:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a7fff640 1 -- 172.21.15.70:0/3142037598 learned_addr learned my addr 172.21.15.70:0/3142037598 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a77fe640 1 --2- 172.21.15.70:0/3142037598 >> v2:172.21.15.155:3300/0 conn(0x7f10a80c9620 0x7f10a8069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a7fff640 1 -- 172.21.15.70:0/3142037598 >> v2:172.21.15.155:3300/0 conn(0x7f10a80c9620 msgr2=0x7f10a8069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a7fff640 1 --2- 172.21.15.70:0/3142037598 >> v2:172.21.15.155:3300/0 conn(0x7f10a80c9620 0x7f10a8069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.285+0000 7f10a7fff640 1 -- 172.21.15.70:0/3142037598 >> v1:172.21.15.155:6789/0 conn(0x7f10a8160be0 legacy=0x7f10a8161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:56.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.286+0000 7f10a7fff640 1 -- 172.21.15.70:0/3142037598 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a80acad0 con 0x7f10a8069850 2023-08-25T09:45:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.287+0000 7f10a7fff640 1 --2- 172.21.15.70:0/3142037598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a8069850 0x7f10a815e210 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f10a806d120 tx=0x7f109802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4a0f7edf59d7b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.287+0000 7f10a6ffd640 1 -- 172.21.15.70:0/3142037598 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098030860 con 0x7f10a8069850 2023-08-25T09:45:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.287+0000 7f10a6ffd640 1 -- 172.21.15.70:0/3142037598 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10980309c0 con 0x7f10a8069850 2023-08-25T09:45:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.287+0000 7f10a6ffd640 1 -- 172.21.15.70:0/3142037598 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098030d50 con 0x7f10a8069850 2023-08-25T09:45:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 -- 172.21.15.70:0/3142037598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a8069850 msgr2=0x7f10a815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3142037598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a8069850 0x7f10a815e210 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f10a806d120 tx=0x7f109802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 -- 172.21.15.70:0/3142037598 shutdown_connections 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3142037598 >> v2:172.21.15.155:3300/0 conn(0x7f10a80c9620 0x7f10a8069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3142037598 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a8069850 0x7f10a815e210 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 -- 172.21.15.70:0/3142037598 >> 172.21.15.70:0/3142037598 conn(0x7f10a80cb0d0 msgr2=0x7f10a80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 -- 172.21.15.70:0/3142037598 shutdown_connections 2023-08-25T09:45:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.288+0000 7f10ae84d640 1 -- 172.21.15.70:0/3142037598 wait complete. 2023-08-25T09:45:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.290+0000 7f10ae84d640 1 Processor -- start 2023-08-25T09:45:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.290+0000 7f10ae84d640 1 -- start start 2023-08-25T09:45:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10ae84d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a8069850 0x7f10a80c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10ae84d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 0x7f10a80c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10ae84d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a8160be0 0x7f10a80c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10ae84d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80c2710 con 0x7f10a80c9620 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10ae84d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80c2fb0 con 0x7f10a8160be0 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10ae84d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80b3f00 con 0x7f10a8069850 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 0x7f10a80c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 0x7f10a80c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38518/0 (socket says 172.21.15.70:38518) 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.291+0000 7f10a77fe640 1 -- 172.21.15.70:0/3308962449 learned_addr learned my addr 172.21.15.70:0/3308962449 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a77fe640 1 -- 172.21.15.70:0/3308962449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a8069850 msgr2=0x7f10a80c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a77fe640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a8069850 0x7f10a80c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a77fe640 1 -- 172.21.15.70:0/3308962449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a8160be0 msgr2=0x7f10a80c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a77fe640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a8160be0 0x7f10a80c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a77fe640 1 -- 172.21.15.70:0/3308962449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10980094d0 con 0x7f10a80c9620 2023-08-25T09:45:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a77fe640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 0x7f10a80c4700 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f109c00d830 tx=0x7f109c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00b6c0 con 0x7f10a80c9620 2023-08-25T09:45:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00b820 con 0x7f10a80c9620 2023-08-25T09:45:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a80c2960 con 0x7f10a80c9620 2023-08-25T09:45:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.292+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00bbb0 con 0x7f10a80c9620 2023-08-25T09:45:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.293+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a81f6ee0 con 0x7f10a80c9620 2023-08-25T09:45:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.294+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a8066fb0 con 0x7f10a80c9620 2023-08-25T09:45:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.295+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f109c01e460 con 0x7f10a80c9620 2023-08-25T09:45:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.296+0000 7f10a4ff9640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f109406a050 0x7f109406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.296+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f109c088de0 con 0x7f10a80c9620 2023-08-25T09:45:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.296+0000 7f10a7fff640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f109406a050 0x7f109406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.297+0000 7f10a7fff640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f109406a050 0x7f109406c510 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1098036000 tx=0x7f109803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.300+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f109c054410 con 0x7f10a80c9620 2023-08-25T09:45:56.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.489+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10a80c30f0 con 0x7f10a80c9620 2023-08-25T09:45:56.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.490+0000 7f10a4ff9640 1 -- 172.21.15.70:0/3308962449 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f109c057a30 con 0x7f10a80c9620 2023-08-25T09:45:56.492 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:45:56.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f109406a050 msgr2=0x7f109406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f109406a050 0x7f109406c510 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1098036000 tx=0x7f109803b040 comp rx=0 tx=0).stop 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 msgr2=0x7f10a80c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 0x7f10a80c4700 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f109c00d830 tx=0x7f109c00dcf0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 shutdown_connections 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f109406a050 0x7f109406c510 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a8160be0 0x7f10a80c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a80c9620 0x7f10a80c4700 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 --2- 172.21.15.70:0/3308962449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a8069850 0x7f10a80c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 >> 172.21.15.70:0/3308962449 conn(0x7f10a80cb0d0 msgr2=0x7f10a815f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:56.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 shutdown_connections 2023-08-25T09:45:56.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.493+0000 7f10ae84d640 1 -- 172.21.15.70:0/3308962449 wait complete. 2023-08-25T09:45:56.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:45:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.729+0000 7f0feeac2640 1 Processor -- start 2023-08-25T09:45:56.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.730+0000 7f0feeac2640 1 -- start start 2023-08-25T09:45:56.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0feeac2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe815a920 0x7f0fe80c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0feeac2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fe80b1ce0 0x7f0fe80ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0feeac2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe80b2170 con 0x7f0fe80b0320 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0feeac2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe80ad8f0 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0feeac2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80adbb0 con 0x7f0fe815a920 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0fee2c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fe80b1ce0 0x7f0fe80ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0fee2c1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fe80b1ce0 0x7f0fe80ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38522/0 (socket says 172.21.15.70:38522) 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.731+0000 7f0fee2c1640 1 -- 172.21.15.70:0/3857528558 learned_addr learned my addr 172.21.15.70:0/3857528558 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.732+0000 7f0fed2bf640 1 --2- 172.21.15.70:0/3857528558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe815a920 0x7f0fe80c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.732+0000 7f0fee2c1640 1 -- 172.21.15.70:0/3857528558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe815a920 msgr2=0x7f0fe80c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:56.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.732+0000 7f0fee2c1640 1 --2- 172.21.15.70:0/3857528558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe815a920 0x7f0fe80c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.732+0000 7f0fee2c1640 1 -- 172.21.15.70:0/3857528558 >> v1:172.21.15.155:6789/0 conn(0x7f0fe80b0320 legacy=0x7f0fe80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:56.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.732+0000 7f0fee2c1640 1 -- 172.21.15.70:0/3857528558 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe80c90a0 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.733+0000 7f0fee2c1640 1 --2- 172.21.15.70:0/3857528558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80ad3b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009e50 tx=0x7f0fe002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c23160f766fa3ad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:56.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.733+0000 7f0fecabe640 1 -- 172.21.15.70:0/3857528558 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe002ca70 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0fecabe640 1 -- 172.21.15.70:0/3857528558 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe002cbd0 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0fecabe640 1 -- 172.21.15.70:0/3857528558 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe00343f0 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 -- 172.21.15.70:0/3857528558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 msgr2=0x7f0fe80ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 --2- 172.21.15.70:0/3857528558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80ad3b0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009e50 tx=0x7f0fe002bed0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 -- 172.21.15.70:0/3857528558 shutdown_connections 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 --2- 172.21.15.70:0/3857528558 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80ad3b0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 --2- 172.21.15.70:0/3857528558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe815a920 0x7f0fe80c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 -- 172.21.15.70:0/3857528558 >> 172.21.15.70:0/3857528558 conn(0x7f0fe80ce830 msgr2=0x7f0fe80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 -- 172.21.15.70:0/3857528558 shutdown_connections 2023-08-25T09:45:56.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.734+0000 7f0feeac2640 1 -- 172.21.15.70:0/3857528558 wait complete. 2023-08-25T09:45:56.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.737+0000 7f0feeac2640 1 Processor -- start 2023-08-25T09:45:56.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.737+0000 7f0feeac2640 1 -- start start 2023-08-25T09:45:56.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0feeac2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe80b0320 0x7f0fe81f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0feeac2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0feeac2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe815a920 0x7f0fe81573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0feeac2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe81f6970 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0feeac2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe80c4500 con 0x7f0fe815a920 2023-08-25T09:45:56.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0feeac2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe8157900 con 0x7f0fe80b0320 2023-08-25T09:45:56.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0fed2bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0fed2bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38532/0 (socket says 172.21.15.70:38532) 2023-08-25T09:45:56.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.738+0000 7f0fed2bf640 1 -- 172.21.15.70:0/881542015 learned_addr learned my addr 172.21.15.70:0/881542015 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:56.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fedac0640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe80b0320 0x7f0fe81f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fed2bf640 1 -- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe80b0320 msgr2=0x7f0fe81f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fed2bf640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe80b0320 0x7f0fe81f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fed2bf640 1 -- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe815a920 msgr2=0x7f0fe81573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fed2bf640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe815a920 0x7f0fe81573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fed2bf640 1 -- 172.21.15.70:0/881542015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009b30 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fed2bf640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80c3fc0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c820 tx=0x7f0fd800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:56.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8004220 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd80043a0 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8007a10 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.739+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe806a220 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.740+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe80698f0 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.741+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe80c9920 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.742+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0fd8007b70 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.743+0000 7f0fde7fc640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fcc06a050 0x7f0fcc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:56.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.743+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f0fd8088440 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.743+0000 7f0fedac0640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fcc06a050 0x7f0fcc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:56.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.744+0000 7f0fedac0640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fcc06a050 0x7f0fcc06c510 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400a790 tx=0x7f0fe4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:56.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.747+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0fe80c9920 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.939+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fe80c9920 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.940+0000 7f0fde7fc640 1 -- 172.21.15.70:0/881542015 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fe80c9920 con 0x7f0fe80b1ce0 2023-08-25T09:45:56.942 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:45:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fcc06a050 msgr2=0x7f0fcc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fcc06a050 0x7f0fcc06c510 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400a790 tx=0x7f0fe4008040 comp rx=0 tx=0).stop 2023-08-25T09:45:56.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 msgr2=0x7f0fe80c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:56.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80c3fc0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800c820 tx=0x7f0fd800cce0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 shutdown_connections 2023-08-25T09:45:56.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fcc06a050 0x7f0fcc06c510 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe815a920 0x7f0fe81573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe80b1ce0 0x7f0fe80c3fc0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 --2- 172.21.15.70:0/881542015 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe80b0320 0x7f0fe81f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 >> 172.21.15.70:0/881542015 conn(0x7f0fe80ce830 msgr2=0x7f0fe80ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 shutdown_connections 2023-08-25T09:45:56.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:56.944+0000 7f0feeac2640 1 -- 172.21.15.70:0/881542015 wait complete. 2023-08-25T09:45:56.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.174+0000 7fdf61a63640 1 Processor -- start 2023-08-25T09:45:57.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.175+0000 7fdf61a63640 1 -- start start 2023-08-25T09:45:57.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.176+0000 7fdf61a63640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdf5c0cf9a0 0x7fdf5c1677e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.176+0000 7fdf61a63640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0d1470 con 0x7fdf5c15caa0 2023-08-25T09:45:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.176+0000 7fdf61a63640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5c0d1750 con 0x7fdf5c15ce80 2023-08-25T09:45:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.176+0000 7fdf61a63640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5c0d1a30 con 0x7fdf5c0cf9a0 2023-08-25T09:45:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.176+0000 7fdf60a61640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdf5c15caa0 0x7fdf5c0d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53038/0 (socket says 172.21.15.70:53038) 2023-08-25T09:45:57.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.176+0000 7fdf60a61640 1 -- 172.21.15.70:0/1619491283 learned_addr learned my addr 172.21.15.70:0/1619491283 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:57.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997279189 0 0) 0x7fdf5c0d1470 con 0x7fdf5c15caa0 2023-08-25T09:45:57.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf4c003520 con 0x7fdf5c15caa0 2023-08-25T09:45:57.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf61262640 1 --2- 172.21.15.70:0/1619491283 >> v2:172.21.15.155:3300/0 conn(0x7fdf5c0cf9a0 0x7fdf5c1677e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833130927 0 0) 0x7fdf4c003520 con 0x7fdf5c15caa0 2023-08-25T09:45:57.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 >> v2:172.21.15.155:3300/0 conn(0x7fdf5c0cf9a0 msgr2=0x7fdf5c1677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 --2- 172.21.15.70:0/1619491283 >> v2:172.21.15.155:3300/0 conn(0x7fdf5c0cf9a0 0x7fdf5c1677e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 >> v1:172.21.15.155:6789/0 conn(0x7fdf5c15ce80 legacy=0x7fdf5c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c0d1d10 con 0x7fdf5c15caa0 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fdf50001670 con 0x7fdf5c15caa0 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 >> v1:172.21.15.70:6789/0 conn(0x7fdf5c15caa0 legacy=0x7fdf5c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf4c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.177+0000 7fdf5b7fe640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c0047c0 0x7fdf4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf5b7fe640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf4c005180 0x7fdf4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0d1a30 con 0x7fdf4c003970 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c00c0b0 con 0x7fdf4c0047c0 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c00c270 con 0x7fdf4c005180 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf60a61640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf4c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf60a61640 1 -- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf4c005180 msgr2=0x7fdf4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf60a61640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf4c005180 0x7fdf4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf60a61640 1 -- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c0047c0 msgr2=0x7fdf4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf5bfff640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c0047c0 0x7fdf4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf60a61640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c0047c0 0x7fdf4c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf60a61640 1 -- 172.21.15.70:0/1619491283 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c003520 con 0x7fdf4c003970 2023-08-25T09:45:57.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf61262640 1 -- 172.21.15.70:0/1619491283 reap_dead start 2023-08-25T09:45:57.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.178+0000 7fdf5bfff640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf4c0047c0 0x7fdf4c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:57.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.179+0000 7fdf60a61640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf4c003d70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fdf50001eb0 tx=0x7fdf4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf50e8d5b4bff1a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:57.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50005ad0 con 0x7fdf4c003970 2023-08-25T09:45:57.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf50005c50 con 0x7fdf4c003970 2023-08-25T09:45:57.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf5b7fe640 1 -- 172.21.15.70:0/1619491283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5002c890 con 0x7fdf4c003970 2023-08-25T09:45:57.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 -- 172.21.15.70:0/1619491283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 msgr2=0x7fdf4c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf4c003d70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fdf50001eb0 tx=0x7fdf4c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 -- 172.21.15.70:0/1619491283 shutdown_connections 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 --2- 172.21.15.70:0/1619491283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf4c003d70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 -- 172.21.15.70:0/1619491283 >> 172.21.15.70:0/1619491283 conn(0x7fdf5c06fe70 msgr2=0x7fdf5c070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 -- 172.21.15.70:0/1619491283 shutdown_connections 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.180+0000 7fdf61a63640 1 -- 172.21.15.70:0/1619491283 wait complete. 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.182+0000 7fdf61a63640 1 Processor -- start 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.182+0000 7fdf61a63640 1 -- start start 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf61a63640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf5c14b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf61a63640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0cb150 0x7fdf5c15eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf61a63640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0cb5d0 0x7fdf5c0cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf61a63640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0ca480 con 0x7fdf4c003970 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf61a63640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0cacc0 con 0x7fdf5c0cb150 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf61a63640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c0ca6b0 con 0x7fdf5c0cb5d0 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf60a61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf5c14b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf60a61640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf5c14b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38556/0 (socket says 172.21.15.70:38556) 2023-08-25T09:45:57.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.183+0000 7fdf60a61640 1 -- 172.21.15.70:0/225031012 learned_addr learned my addr 172.21.15.70:0/225031012 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:57.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf60a61640 1 -- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0cb5d0 msgr2=0x7fdf5c0cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf61262640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0cb5d0 0x7fdf5c0cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf60a61640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0cb5d0 0x7fdf5c0cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf60a61640 1 -- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0cb150 msgr2=0x7fdf5c15eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf60a61640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0cb150 0x7fdf5c15eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf60a61640 1 -- 172.21.15.70:0/225031012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5c15f610 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf60a61640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf5c14b210 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fdf50001eb0 tx=0x7fdf5002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5003c040 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.184+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5002cca0 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.185+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c0ca950 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.185+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf50007270 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.185+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c15f8f0 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.186+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c066fb0 con 0x7fdf4c003970 2023-08-25T09:45:57.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.186+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdf50032070 con 0x7fdf4c003970 2023-08-25T09:45:57.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.187+0000 7fdf597fa640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf4006a050 0x7fdf4006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.188+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fdf500a9b10 con 0x7fdf4c003970 2023-08-25T09:45:57.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.188+0000 7fdf5bfff640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf4006a050 0x7fdf4006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.188+0000 7fdf5bfff640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf4006a050 0x7fdf4006c510 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c15e520 tx=0x7fdf48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:57.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.192+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdf500751d0 con 0x7fdf4c003970 2023-08-25T09:45:57.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.419+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf5c0ce430 con 0x7fdf4c003970 2023-08-25T09:45:57.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.421+0000 7fdf597fa640 1 -- 172.21.15.70:0/225031012 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fdf500017c0 con 0x7fdf4c003970 2023-08-25T09:45:57.422 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:57.423 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:57.423 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:57.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf4006a050 msgr2=0x7fdf4006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf4006a050 0x7fdf4006c510 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c15e520 tx=0x7fdf48009040 comp rx=0 tx=0).stop 2023-08-25T09:45:57.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 msgr2=0x7fdf5c14b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf5c14b210 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fdf50001eb0 tx=0x7fdf5002e070 comp rx=0 tx=0).stop 2023-08-25T09:45:57.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 shutdown_connections 2023-08-25T09:45:57.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf4006a050 0x7fdf4006c510 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf5c0cb5d0 0x7fdf5c0cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf5c0cb150 0x7fdf5c15eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 --2- 172.21.15.70:0/225031012 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf4c003970 0x7fdf5c14b210 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.423+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 >> 172.21.15.70:0/225031012 conn(0x7fdf5c06fe70 msgr2=0x7fdf5c0d2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:57.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.424+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 shutdown_connections 2023-08-25T09:45:57.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.424+0000 7fdf61a63640 1 -- 172.21.15.70:0/225031012 wait complete. 2023-08-25T09:45:57.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:57.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.657+0000 7ff8fe8b8640 1 Processor -- start 2023-08-25T09:45:57.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.657+0000 7ff8fe8b8640 1 -- start start 2023-08-25T09:45:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8fe8b8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 0x7ff8f80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8fe8b8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f81615b0 con 0x7ff8f80ac870 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8fe8b8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f806f050 con 0x7ff8f80b2b40 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8fe8b8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f806f330 con 0x7ff8f815cb40 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8effff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff8f80ac870 0x7ff8f8160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53046/0 (socket says 172.21.15.70:53046) 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8effff640 1 -- 172.21.15.70:0/3526206769 learned_addr learned my addr 172.21.15.70:0/3526206769 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.659+0000 7ff8f7fff640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 0x7ff8f80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045991360 0 0) 0x7ff8f81615b0 con 0x7ff8f80ac870 2023-08-25T09:45:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e4003520 con 0x7ff8f80ac870 2023-08-25T09:45:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3005883593 0 0) 0x7ff8e4003520 con 0x7ff8f80ac870 2023-08-25T09:45:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 msgr2=0x7ff8f80ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 0x7ff8f80ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 >> v1:172.21.15.155:6789/0 conn(0x7ff8f80b2b40 legacy=0x7ff8f8160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f806f610 con 0x7ff8f80ac870 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff8e0001670 con 0x7ff8f80ac870 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 >> v1:172.21.15.70:6789/0 conn(0x7ff8f80ac870 legacy=0x7ff8f8160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e40039a0 0x7ff8e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 0x7ff8e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8e4004d00 0x7ff8e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f806f330 con 0x7ff8e4004d00 2023-08-25T09:45:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e400ba30 con 0x7ff8e40039a0 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e400bbf0 con 0x7ff8f815cb40 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.660+0000 7ff8fce55640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8e4004d00 0x7ff8e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8f7fff640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e40039a0 0x7ff8e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8effff640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 0x7ff8e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8fce55640 1 -- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 msgr2=0x7ff8e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8fce55640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f815cb40 0x7ff8e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8fce55640 1 -- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e40039a0 msgr2=0x7ff8e4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8fce55640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8e40039a0 0x7ff8e4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.661+0000 7ff8fce55640 1 -- 172.21.15.70:0/3526206769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4003520 con 0x7ff8e4004d00 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.662+0000 7ff8fce55640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8e4004d00 0x7ff8e4005180 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80b3830 tx=0x7ff8e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9ef4db3b3fde82 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0003b70 con 0x7ff8e4004d00 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f0003cd0 con 0x7ff8e4004d00 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8f77fe640 1 -- 172.21.15.70:0/3526206769 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0005310 con 0x7ff8e4004d00 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/3526206769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8e4004d00 msgr2=0x7ff8e4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/3526206769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8e4004d00 0x7ff8e4005180 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80b3830 tx=0x7ff8e40057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8fce55640 1 -- 172.21.15.70:0/3526206769 reap_dead start 2023-08-25T09:45:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/3526206769 shutdown_connections 2023-08-25T09:45:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/3526206769 >> 172.21.15.70:0/3526206769 conn(0x7ff8f8069850 msgr2=0x7ff8f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.663+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/3526206769 shutdown_connections 2023-08-25T09:45:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.664+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/3526206769 wait complete. 2023-08-25T09:45:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.665+0000 7ff8fe8b8640 1 Processor -- start 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.665+0000 7ff8fe8b8640 1 -- start start 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.666+0000 7ff8fe8b8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8f814d170 0x7ff8f8158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.666+0000 7ff8fe8b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 0x7ff8f815ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.666+0000 7ff8fe8b8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f814d550 0x7ff8f815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.666+0000 7ff8fe8b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8153230 con 0x7ff8f8158ca0 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.666+0000 7ff8fe8b8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8153510 con 0x7ff8f814d170 2023-08-25T09:45:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.666+0000 7ff8fe8b8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f81537f0 con 0x7ff8f814d550 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 0x7ff8f815ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 0x7ff8f815ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38572/0 (socket says 172.21.15.70:38572) 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 -- 172.21.15.70:0/1036101313 learned_addr learned my addr 172.21.15.70:0/1036101313 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8f7fff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8f814d170 0x7ff8f8158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 -- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f814d550 msgr2=0x7ff8f815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f814d550 0x7ff8f815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 -- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8f814d170 msgr2=0x7ff8f8158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8f814d170 0x7ff8f8158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 -- 172.21.15.70:0/1036101313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f8153ad0 con 0x7ff8f8158ca0 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8f7fff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8f814d170 0x7ff8f8158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.667+0000 7ff8effff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 0x7ff8f815ac20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0001200 tx=0x7ff8e000ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.668+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000c850 con 0x7ff8f8158ca0 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.668+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e000c9b0 con 0x7ff8f8158ca0 2023-08-25T09:45:57.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.668+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cd60 con 0x7ff8f8158ca0 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.668+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f8152ee0 con 0x7ff8f8158ca0 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.668+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f81541d0 con 0x7ff8f8158ca0 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.669+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f804e420 con 0x7ff8f8158ca0 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.670+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff8e001d460 con 0x7ff8f8158ca0 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.671+0000 7ff8f57fa640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8d806a050 0x7ff8d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.671+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7ff8e0088700 con 0x7ff8f8158ca0 2023-08-25T09:45:57.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.671+0000 7ff8f7fff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8d806a050 0x7ff8d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.672+0000 7ff8f7fff640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8d806a050 0x7ff8d806c510 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009a00 tx=0x7ff8e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:57.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.676+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff8e0053ca0 con 0x7ff8f8158ca0 2023-08-25T09:45:57.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.864+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8f8153370 con 0x7ff8f8158ca0 2023-08-25T09:45:57.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.867+0000 7ff8f57fa640 1 -- 172.21.15.70:0/1036101313 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+18598 (secure 0 0 0) 0x7ff8f8153370 con 0x7ff8f8158ca0 2023-08-25T09:45:57.869 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:57.869 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:42.437094+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.b","mappings":[{"from":6,"to":4}]},{"pgid":"14.12","mappings":[{"from":7,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.23","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:57.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.869+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8d806a050 msgr2=0x7ff8d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.869+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8d806a050 0x7ff8d806c510 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009a00 tx=0x7ff8e8008040 comp rx=0 tx=0).stop 2023-08-25T09:45:57.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.869+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 msgr2=0x7ff8f815ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:57.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.869+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 0x7ff8f815ac20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7ff8e0001200 tx=0x7ff8e000ef80 comp rx=0 tx=0).stop 2023-08-25T09:45:57.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 shutdown_connections 2023-08-25T09:45:57.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff8d806a050 0x7ff8d806c510 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8f814d550 0x7ff8f815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8f8158ca0 0x7ff8f815ac20 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 --2- 172.21.15.70:0/1036101313 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8f814d170 0x7ff8f8158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:57.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 >> 172.21.15.70:0/1036101313 conn(0x7ff8f8069850 msgr2=0x7ff8f806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:57.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 shutdown_connections 2023-08-25T09:45:57.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:57.870+0000 7ff8fe8b8640 1 -- 172.21.15.70:0/1036101313 wait complete. 2023-08-25T09:45:57.889 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:45:57.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:58.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.096+0000 7fe314dab640 1 Processor -- start 2023-08-25T09:45:58.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.096+0000 7fe314dab640 1 -- start start 2023-08-25T09:45:58.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe314dab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe31015a7c0 0x7fe3100b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe314dab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe3100cad20 0x7fe3100cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe314dab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe31015e7c0 con 0x7fe3100b12f0 2023-08-25T09:45:58.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe314dab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100cb6f0 con 0x7fe3100cad20 2023-08-25T09:45:58.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe314dab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe310071b50 con 0x7fe31015a7c0 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe30ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe3100cad20 0x7fe3100cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe306ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe3100b12f0 0x7fe31015ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:53058/0 (socket says 172.21.15.70:53058) 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.098+0000 7fe306ffd640 1 -- 172.21.15.70:0/679999744 learned_addr learned my addr 172.21.15.70:0/679999744 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30f7fe640 1 --2- 172.21.15.70:0/679999744 >> v2:172.21.15.155:3300/0 conn(0x7fe31015a7c0 0x7fe3100b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30ffff640 1 -- 172.21.15.70:0/679999744 >> v2:172.21.15.155:3300/0 conn(0x7fe31015a7c0 msgr2=0x7fe3100b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30ffff640 1 --2- 172.21.15.70:0/679999744 >> v2:172.21.15.155:3300/0 conn(0x7fe31015a7c0 0x7fe3100b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30ffff640 1 -- 172.21.15.70:0/679999744 >> v1:172.21.15.70:6789/0 conn(0x7fe3100b12f0 legacy=0x7fe31015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30ffff640 1 -- 172.21.15.70:0/679999744 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31015e490 con 0x7fe3100cad20 2023-08-25T09:45:58.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30f7fe640 1 --2- 172.21.15.70:0/679999744 >> v2:172.21.15.155:3300/0 conn(0x7fe31015a7c0 0x7fe3100b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:58.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.099+0000 7fe30effd640 1 -- 172.21.15.70:0/679999744 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608393123 0 0) 0x7fe31015e7c0 con 0x7fe3100b12f0 2023-08-25T09:45:58.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.100+0000 7fe30ffff640 1 --2- 172.21.15.70:0/679999744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3100cad20 0x7fe3100cb1b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe3100ce280 tx=0x7fe30802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a69d1e148fd6c6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.100+0000 7fe30effd640 1 -- 172.21.15.70:0/679999744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30802cc80 con 0x7fe3100cad20 2023-08-25T09:45:58.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.100+0000 7fe30effd640 1 -- 172.21.15.70:0/679999744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30802cde0 con 0x7fe3100cad20 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.100+0000 7fe30effd640 1 -- 172.21.15.70:0/679999744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308034660 con 0x7fe3100cad20 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 -- 172.21.15.70:0/679999744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3100cad20 msgr2=0x7fe3100cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 --2- 172.21.15.70:0/679999744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3100cad20 0x7fe3100cb1b0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe3100ce280 tx=0x7fe30802bed0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 -- 172.21.15.70:0/679999744 shutdown_connections 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 --2- 172.21.15.70:0/679999744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe3100cad20 0x7fe3100cb1b0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 --2- 172.21.15.70:0/679999744 >> v2:172.21.15.155:3300/0 conn(0x7fe31015a7c0 0x7fe3100b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 -- 172.21.15.70:0/679999744 >> 172.21.15.70:0/679999744 conn(0x7fe3100affd0 msgr2=0x7fe3100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:58.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 -- 172.21.15.70:0/679999744 shutdown_connections 2023-08-25T09:45:58.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.101+0000 7fe314dab640 1 -- 172.21.15.70:0/679999744 wait complete. 2023-08-25T09:45:58.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.103+0000 7fe314dab640 1 Processor -- start 2023-08-25T09:45:58.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.103+0000 7fe314dab640 1 -- start start 2023-08-25T09:45:58.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe314dab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3100b12f0 0x7fe3101592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe314dab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3100cad20 0x7fe3100c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe314dab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 0x7fe3100ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe314dab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100c3800 con 0x7fe31015a7c0 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe314dab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe310159e80 con 0x7fe3100cad20 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe314dab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3100c2b80 con 0x7fe3100b12f0 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 0x7fe3100ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 0x7fe3100ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38600/0 (socket says 172.21.15.70:38600) 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30ffff640 1 -- 172.21.15.70:0/1937147925 learned_addr learned my addr 172.21.15.70:0/1937147925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:58.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30f7fe640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3100b12f0 0x7fe3101592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe306ffd640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3100cad20 0x7fe3100c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30ffff640 1 -- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3100b12f0 msgr2=0x7fe3101592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30ffff640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3100b12f0 0x7fe3101592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.104+0000 7fe30ffff640 1 -- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3100cad20 msgr2=0x7fe3100c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe30ffff640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3100cad20 0x7fe3100c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe30ffff640 1 -- 172.21.15.70:0/1937147925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe308009a70 con 0x7fe31015a7c0 2023-08-25T09:45:58.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe30ffff640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 0x7fe3100ac880 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe308009e80 tx=0x7fe308004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308040650 con 0x7fe31015a7c0 2023-08-25T09:45:58.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3080407b0 con 0x7fe31015a7c0 2023-08-25T09:45:58.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3100c2e60 con 0x7fe31015a7c0 2023-08-25T09:45:58.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe308040b60 con 0x7fe31015a7c0 2023-08-25T09:45:58.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.105+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3100c3420 con 0x7fe31015a7c0 2023-08-25T09:45:58.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.106+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3100c3940 con 0x7fe31015a7c0 2023-08-25T09:45:58.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.107+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe308040cc0 con 0x7fe31015a7c0 2023-08-25T09:45:58.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.109+0000 7fe30cff9640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe2f006a050 0x7fe2f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.109+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fe3080ab500 con 0x7fe31015a7c0 2023-08-25T09:45:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.109+0000 7fe30f7fe640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe2f006a050 0x7fe2f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.110+0000 7fe30f7fe640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe2f006a050 0x7fe2f006c510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe300008850 tx=0x7fe300008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.113+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3100c3940 con 0x7fe31015a7c0 2023-08-25T09:45:58.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.322+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3100c35e0 con 0x7fe31015a7c0 2023-08-25T09:45:58.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.323+0000 7fe30cff9640 1 -- 172.21.15.70:0/1937147925 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fe308076b60 con 0x7fe31015a7c0 2023-08-25T09:45:58.325 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:58.325 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:58.325 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.325+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe2f006a050 msgr2=0x7fe2f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.325+0000 7fe314dab640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe2f006a050 0x7fe2f006c510 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe300008850 tx=0x7fe300008040 comp rx=0 tx=0).stop 2023-08-25T09:45:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.325+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 msgr2=0x7fe3100ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.325+0000 7fe314dab640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 0x7fe3100ac880 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe308009e80 tx=0x7fe308004a40 comp rx=0 tx=0).stop 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 shutdown_connections 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe2f006a050 0x7fe2f006c510 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe31015a7c0 0x7fe3100ac880 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3100cad20 0x7fe3100c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 --2- 172.21.15.70:0/1937147925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3100b12f0 0x7fe3101592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 >> 172.21.15.70:0/1937147925 conn(0x7fe3100affd0 msgr2=0x7fe3100af840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 shutdown_connections 2023-08-25T09:45:58.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.326+0000 7fe314dab640 1 -- 172.21.15.70:0/1937147925 wait complete. 2023-08-25T09:45:58.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:45:58.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.547+0000 7f18ecc34640 1 Processor -- start 2023-08-25T09:45:58.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.547+0000 7f18ecc34640 1 -- start start 2023-08-25T09:45:58.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.549+0000 7f18ecc34640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 0x7f18e815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.549+0000 7f18ecc34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815c6d0 0x7f18e80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.549+0000 7f18ecc34640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e80b1b60 con 0x7f18e815e830 2023-08-25T09:45:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.549+0000 7f18ecc34640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e806bd80 con 0x7f18e815ec10 2023-08-25T09:45:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.549+0000 7f18ecc34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e806b7c0 con 0x7f18e815c6d0 2023-08-25T09:45:58.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.550+0000 7f18ddd9b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 0x7f18e815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.550+0000 7f18e6d9d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815c6d0 0x7f18e80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.550+0000 7f18ddd9b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 0x7f18e815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:46360/0 (socket says 172.21.15.70:46360) 2023-08-25T09:45:58.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.550+0000 7f18ddd9b640 1 -- 172.21.15.70:0/4071401025 learned_addr learned my addr 172.21.15.70:0/4071401025 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:58.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.551+0000 7f18e6d9d640 1 -- 172.21.15.70:0/4071401025 >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 msgr2=0x7f18e815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:58.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.551+0000 7f18e6d9d640 1 --2- 172.21.15.70:0/4071401025 >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 0x7f18e815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.551+0000 7f18e6d9d640 1 -- 172.21.15.70:0/4071401025 >> v1:172.21.15.155:6789/0 conn(0x7f18e815e830 legacy=0x7f18e806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.551+0000 7f18e6d9d640 1 -- 172.21.15.70:0/4071401025 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e806baa0 con 0x7f18e815c6d0 2023-08-25T09:45:58.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.551+0000 7f18ddd9b640 1 --2- 172.21.15.70:0/4071401025 >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 0x7f18e815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:45:58.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.552+0000 7f18e6d9d640 1 --2- 172.21.15.70:0/4071401025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815c6d0 0x7f18e80b1ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009d90 tx=0x7f18e002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca443650df868661 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18e5d9b640 1 -- 172.21.15.70:0/4071401025 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e002cc60 con 0x7f18e815c6d0 2023-08-25T09:45:58.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18e5d9b640 1 -- 172.21.15.70:0/4071401025 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e002cdc0 con 0x7f18e815c6d0 2023-08-25T09:45:58.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18e5d9b640 1 -- 172.21.15.70:0/4071401025 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e0034660 con 0x7f18e815c6d0 2023-08-25T09:45:58.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 -- 172.21.15.70:0/4071401025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815c6d0 msgr2=0x7f18e80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 --2- 172.21.15.70:0/4071401025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815c6d0 0x7f18e80b1ff0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009d90 tx=0x7f18e002beb0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 -- 172.21.15.70:0/4071401025 shutdown_connections 2023-08-25T09:45:58.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 --2- 172.21.15.70:0/4071401025 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815c6d0 0x7f18e80b1ff0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 --2- 172.21.15.70:0/4071401025 >> v2:172.21.15.155:3300/0 conn(0x7f18e815ec10 0x7f18e815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 -- 172.21.15.70:0/4071401025 >> 172.21.15.70:0/4071401025 conn(0x7f18e8069850 msgr2=0x7f18e8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:58.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 -- 172.21.15.70:0/4071401025 shutdown_connections 2023-08-25T09:45:58.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.553+0000 7f18ecc34640 1 -- 172.21.15.70:0/4071401025 wait complete. 2023-08-25T09:45:58.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.555+0000 7f18ecc34640 1 Processor -- start 2023-08-25T09:45:58.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.555+0000 7f18ecc34640 1 -- start start 2023-08-25T09:45:58.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.556+0000 7f18ecc34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 0x7f18e806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.556+0000 7f18ecc34640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e815e830 0x7f18e81f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.556+0000 7f18ecc34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815ec10 0x7f18e81f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.556+0000 7f18ecc34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e81540d0 con 0x7f18e815c6d0 2023-08-25T09:45:58.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.556+0000 7f18ecc34640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8070170 con 0x7f18e815e830 2023-08-25T09:45:58.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.556+0000 7f18ecc34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e8070450 con 0x7f18e815ec10 2023-08-25T09:45:58.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 0x7f18e806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 0x7f18e806f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38614/0 (socket says 172.21.15.70:38614) 2023-08-25T09:45:58.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 -- 172.21.15.70:0/409371322 learned_addr learned my addr 172.21.15.70:0/409371322 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:58.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18ddd9b640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e815e830 0x7f18e81f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e6d9d640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815ec10 0x7f18e81f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 -- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815ec10 msgr2=0x7f18e81f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815ec10 0x7f18e81f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 -- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e815e830 msgr2=0x7f18e81f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e815e830 0x7f18e81f2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 -- 172.21.15.70:0/409371322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00094d0 con 0x7f18e815c6d0 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.557+0000 7f18e659c640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 0x7f18e806f6f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c450 tx=0x7f18d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.558+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d80052d0 con 0x7f18e815c6d0 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.558+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d8005430 con 0x7f18e815c6d0 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.558+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e0009a70 con 0x7f18e815c6d0 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.558+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d8012870 con 0x7f18e815c6d0 2023-08-25T09:45:58.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.558+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e8070a10 con 0x7f18e815c6d0 2023-08-25T09:45:58.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.559+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e804e420 con 0x7f18e815c6d0 2023-08-25T09:45:58.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.560+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f18d80129d0 con 0x7f18e815c6d0 2023-08-25T09:45:58.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.560+0000 7f18df7fe640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18c806a050 0x7f18c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.561+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f18d8088220 con 0x7f18e815c6d0 2023-08-25T09:45:58.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.561+0000 7f18ddd9b640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18c806a050 0x7f18c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.561+0000 7f18ddd9b640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18c806a050 0x7f18c806c510 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f18e80afe30 tx=0x7f18d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.565+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f18d80537d0 con 0x7f18e815c6d0 2023-08-25T09:45:58.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.778+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18e806c620 con 0x7f18e815c6d0 2023-08-25T09:45:58.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.779+0000 7f18df7fe640 1 -- 172.21.15.70:0/409371322 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f18d8056df0 con 0x7f18e815c6d0 2023-08-25T09:45:58.780 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:58.780 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:45:06.477747+0000","modified":"2023-08-25T09:45:10.342540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7227},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7227":{"gid":7227,"name":"c","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/2920920514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":2920920514},{"type":"v1","addr":"172.21.15.70:6837","nonce":2920920514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-08-25T09:45:58.781 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 49 2023-08-25T09:45:58.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18c806a050 msgr2=0x7f18c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18c806a050 0x7f18c806c510 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f18e80afe30 tx=0x7f18d0008040 comp rx=0 tx=0).stop 2023-08-25T09:45:58.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 msgr2=0x7f18e806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 0x7f18e806f6f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f18d800c450 tx=0x7f18d800c910 comp rx=0 tx=0).stop 2023-08-25T09:45:58.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 shutdown_connections 2023-08-25T09:45:58.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f18c806a050 0x7f18c806c510 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f18e815ec10 0x7f18e81f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f18e815e830 0x7f18e81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 --2- 172.21.15.70:0/409371322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f18e815c6d0 0x7f18e806f6f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 >> 172.21.15.70:0/409371322 conn(0x7f18e8069850 msgr2=0x7f18e80b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:58.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 shutdown_connections 2023-08-25T09:45:58.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.781+0000 7f18ecc34640 1 -- 172.21.15.70:0/409371322 wait complete. 2023-08-25T09:45:58.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:45:58.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.983+0000 7f40dd54e640 1 Processor -- start 2023-08-25T09:45:58.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.983+0000 7f40dd54e640 1 -- start start 2023-08-25T09:45:58.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.985+0000 7f40dd54e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f40d8160ff0 0x7f40d806b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.985+0000 7f40dd54e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40d80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.985+0000 7f40dd54e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d81620b0 con 0x7f40d80ce5f0 2023-08-25T09:45:58.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.985+0000 7f40dd54e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d80cefe0 con 0x7f40d8160ff0 2023-08-25T09:45:58.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.985+0000 7f40dd54e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d806ad70 con 0x7f40d8161ae0 2023-08-25T09:45:58.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.985+0000 7f40cffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f40d80ce5f0 0x7f40d81613d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41280/0 (socket says 172.21.15.70:41280) 2023-08-25T09:45:58.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40cffff640 1 -- 172.21.15.70:0/2575015474 learned_addr learned my addr 172.21.15.70:0/2575015474 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d6ffd640 1 --2- 172.21.15.70:0/2575015474 >> v2:172.21.15.155:3300/0 conn(0x7f40d8160ff0 0x7f40d806b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174397463 0 0) 0x7f40d81620b0 con 0x7f40d80ce5f0 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d77fe640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40d80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40c4003520 con 0x7f40d80ce5f0 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390805389 0 0) 0x7f40c4003520 con 0x7f40d80ce5f0 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 msgr2=0x7f40d80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:58.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40d80ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 >> v2:172.21.15.155:3300/0 conn(0x7f40d8160ff0 msgr2=0x7f40d806b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 --2- 172.21.15.70:0/2575015474 >> v2:172.21.15.155:3300/0 conn(0x7f40d8160ff0 0x7f40d806b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8162390 con 0x7f40d80ce5f0 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f40c0001670 con 0x7f40d80ce5f0 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 >> v1:172.21.15.70:6789/0 conn(0x7f40d80ce5f0 legacy=0x7f40d81613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40c4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c4003ef0 0x7f40c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.986+0000 7f40d67fc640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4004d00 0x7f40c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80cefe0 con 0x7f40c4003ef0 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d806ad70 con 0x7f40c4004d00 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c400b9d0 con 0x7f40d8161ae0 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40cffff640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c4003ef0 0x7f40c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40d77fe640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4004d00 0x7f40c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40d6ffd640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40c4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40cffff640 1 -- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 msgr2=0x7f40c4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40cffff640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40c4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40cffff640 1 -- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4004d00 msgr2=0x7f40c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40cffff640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4004d00 0x7f40c4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40cffff640 1 -- 172.21.15.70:0/2575015474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c4003520 con 0x7f40c4003ef0 2023-08-25T09:45:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.987+0000 7f40d6ffd640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d8161ae0 0x7f40c4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:58.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.988+0000 7f40cffff640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c4003ef0 0x7f40c40047c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c240 tx=0x7f40c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f335b4a0c8f86526 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:58.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c002cb00 con 0x7f40c4003ef0 2023-08-25T09:45:58.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c002cc60 con 0x7f40c4003ef0 2023-08-25T09:45:58.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40d67fc640 1 -- 172.21.15.70:0/2575015474 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0005af0 con 0x7f40c4003ef0 2023-08-25T09:45:58.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40dd54e640 1 -- 172.21.15.70:0/2575015474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c4003ef0 msgr2=0x7f40c40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40dd54e640 1 --2- 172.21.15.70:0/2575015474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40c4003ef0 0x7f40c40047c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c240 tx=0x7f40c40057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40d77fe640 1 -- 172.21.15.70:0/2575015474 reap_dead start 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40dd54e640 1 -- 172.21.15.70:0/2575015474 shutdown_connections 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40dd54e640 1 -- 172.21.15.70:0/2575015474 >> 172.21.15.70:0/2575015474 conn(0x7f40d8069850 msgr2=0x7f40d8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.989+0000 7f40dd54e640 1 -- 172.21.15.70:0/2575015474 shutdown_connections 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.990+0000 7f40dd54e640 1 -- 172.21.15.70:0/2575015474 wait complete. 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.991+0000 7f40dd54e640 1 Processor -- start 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.991+0000 7f40dd54e640 1 -- start start 2023-08-25T09:45:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.992+0000 7f40dd54e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4003ef0 0x7f40d81eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.992+0000 7f40dd54e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 0x7f40d814ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.992+0000 7f40dd54e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d81ea6c0 0x7f40d81eab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.992+0000 7f40dd54e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81ea230 con 0x7f40d814c5f0 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.992+0000 7f40dd54e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81e9620 con 0x7f40c4003ef0 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.992+0000 7f40dd54e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81e9900 con 0x7f40d81ea6c0 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 0x7f40d814ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 0x7f40d814ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36868/0 (socket says 172.21.15.70:36868) 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 -- 172.21.15.70:0/811223894 learned_addr learned my addr 172.21.15.70:0/811223894 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40d6ffd640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4003ef0 0x7f40d81eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 -- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d81ea6c0 msgr2=0x7f40d81eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d81ea6c0 0x7f40d81eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 -- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4003ef0 msgr2=0x7f40d81eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4003ef0 0x7f40d81eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 -- 172.21.15.70:0/811223894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d81e9be0 con 0x7f40d814c5f0 2023-08-25T09:45:58.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.993+0000 7f40cffff640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 0x7f40d814ca70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f40c002c4a0 tx=0x7f40c0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.994+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0039070 con 0x7f40d814c5f0 2023-08-25T09:45:59.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.994+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c0005d00 con 0x7f40d814c5f0 2023-08-25T09:45:59.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.994+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d81e9ec0 con 0x7f40d814c5f0 2023-08-25T09:45:59.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.994+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c003c040 con 0x7f40d814c5f0 2023-08-25T09:45:59.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.994+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d81ebcb0 con 0x7f40d814c5f0 2023-08-25T09:45:59.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.995+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d81ea3e0 con 0x7f40d814c5f0 2023-08-25T09:45:59.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.996+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f40c0032070 con 0x7f40d814c5f0 2023-08-25T09:45:59.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.997+0000 7f40cf7fe640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f40b806a050 0x7f40b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.997+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f40c00a98f0 con 0x7f40d814c5f0 2023-08-25T09:45:59.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.997+0000 7f40d6ffd640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f40b806a050 0x7f40b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:58.998+0000 7f40d6ffd640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f40b806a050 0x7f40b806c510 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009a40 tx=0x7f40c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.001+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f40d81ea3e0 con 0x7f40d814c5f0 2023-08-25T09:45:59.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.191+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d81ebcb0 con 0x7f40d814c5f0 2023-08-25T09:45:59.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.194+0000 7f40cf7fe640 1 -- 172.21.15.70:0/811223894 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+18598 (secure 0 0 0) 0x7f40d81ebcb0 con 0x7f40d814c5f0 2023-08-25T09:45:59.197 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:45:59.198 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":121,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:45:42.437094+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:45:04.798879+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-08-25T09:45:05.478134+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.b","mappings":[{"from":6,"to":4}]},{"pgid":"14.12","mappings":[{"from":7,"to":0}]},{"pgid":"14.19","mappings":[{"from":7,"to":0}]},{"pgid":"14.23","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3b","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:45:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.196+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f40b806a050 msgr2=0x7f40b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.196+0000 7f40dd54e640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f40b806a050 0x7f40b806c510 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009a40 tx=0x7f40c8008040 comp rx=0 tx=0).stop 2023-08-25T09:45:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.196+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 msgr2=0x7f40d814ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.196+0000 7f40dd54e640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 0x7f40d814ca70 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f40c002c4a0 tx=0x7f40c0001a00 comp rx=0 tx=0).stop 2023-08-25T09:45:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 shutdown_connections 2023-08-25T09:45:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f40b806a050 0x7f40b806c510 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40d81ea6c0 0x7f40d81eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f40d814c5f0 0x7f40d814ca70 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 --2- 172.21.15.70:0/811223894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f40c4003ef0 0x7f40d81eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 >> 172.21.15.70:0/811223894 conn(0x7f40d8069850 msgr2=0x7f40d8070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 shutdown_connections 2023-08-25T09:45:59.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.197+0000 7f40dd54e640 1 -- 172.21.15.70:0/811223894 wait complete. 2023-08-25T09:45:59.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:45:59.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.431+0000 7f9d8721f640 1 Processor -- start 2023-08-25T09:45:59.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.432+0000 7f9d8721f640 1 -- start start 2023-08-25T09:45:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.433+0000 7f9d8721f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9d8015ccf0 0x7f9d800b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.433+0000 7f9d8721f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 0x7f9d800b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.433+0000 7f9d8721f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d80161600 con 0x7f9d800ca420 2023-08-25T09:45:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.433+0000 7f9d8721f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d80071000 con 0x7f9d8015ccf0 2023-08-25T09:45:59.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.433+0000 7f9d8721f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015e0e0 con 0x7f9d800c7a70 2023-08-25T09:45:59.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.433+0000 7f9d84fbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9d8015ccf0 0x7f9d800b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.434+0000 7f9d84fbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9d8015ccf0 0x7f9d800b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36870/0 (socket says 172.21.15.70:36870) 2023-08-25T09:45:59.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.434+0000 7f9d84fbb640 1 -- 172.21.15.70:0/1826057865 learned_addr learned my addr 172.21.15.70:0/1826057865 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.434+0000 7f9d84fbb640 1 -- 172.21.15.70:0/1826057865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 msgr2=0x7f9d800b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.434+0000 7f9d84fbb640 1 --2- 172.21.15.70:0/1826057865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 0x7f9d800b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.434+0000 7f9d84fbb640 1 -- 172.21.15.70:0/1826057865 >> v1:172.21.15.70:6789/0 conn(0x7f9d800ca420 legacy=0x7f9d8006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.434+0000 7f9d84fbb640 1 -- 172.21.15.70:0/1826057865 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d801609f0 con 0x7f9d8015ccf0 2023-08-25T09:45:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.435+0000 7f9d84fbb640 1 --2- 172.21.15.70:0/1826057865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8015ccf0 0x7f9d800b0a10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009850 tx=0x7f9d7002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95511b8daac39e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.435+0000 7f9d7f7fe640 1 -- 172.21.15.70:0/1826057865 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70004230 con 0x7f9d8015ccf0 2023-08-25T09:45:59.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.435+0000 7f9d7f7fe640 1 -- 172.21.15.70:0/1826057865 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d70004390 con 0x7f9d8015ccf0 2023-08-25T09:45:59.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.435+0000 7f9d7f7fe640 1 -- 172.21.15.70:0/1826057865 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70003810 con 0x7f9d8015ccf0 2023-08-25T09:45:59.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 -- 172.21.15.70:0/1826057865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8015ccf0 msgr2=0x7f9d800b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 --2- 172.21.15.70:0/1826057865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8015ccf0 0x7f9d800b0a10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009850 tx=0x7f9d7002b960 comp rx=0 tx=0).stop 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 -- 172.21.15.70:0/1826057865 shutdown_connections 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 --2- 172.21.15.70:0/1826057865 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 0x7f9d800b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 --2- 172.21.15.70:0/1826057865 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d8015ccf0 0x7f9d800b0a10 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 -- 172.21.15.70:0/1826057865 >> 172.21.15.70:0/1826057865 conn(0x7f9d800ce760 msgr2=0x7f9d800ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 -- 172.21.15.70:0/1826057865 shutdown_connections 2023-08-25T09:45:59.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.436+0000 7f9d8721f640 1 -- 172.21.15.70:0/1826057865 wait complete. 2023-08-25T09:45:59.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.438+0000 7f9d8721f640 1 Processor -- start 2023-08-25T09:45:59.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.438+0000 7f9d8721f640 1 -- start start 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d8721f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 0x7f9d80157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d8721f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 0x7f9d80159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d8721f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d8015ccf0 0x7f9d8014b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d8721f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80154480 con 0x7f9d800ca420 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d8721f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80154760 con 0x7f9d8015ccf0 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d8721f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d80154a40 con 0x7f9d800c7a70 2023-08-25T09:45:59.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 0x7f9d80159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d7ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 0x7f9d80159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36884/0 (socket says 172.21.15.70:36884) 2023-08-25T09:45:59.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.439+0000 7f9d7ffff640 1 -- 172.21.15.70:0/2401833648 learned_addr learned my addr 172.21.15.70:0/2401833648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7ffff640 1 -- 172.21.15.70:0/2401833648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 msgr2=0x7f9d80157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7ffff640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 0x7f9d80157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7ffff640 1 -- 172.21.15.70:0/2401833648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d8015ccf0 msgr2=0x7f9d8014b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7ffff640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d8015ccf0 0x7f9d8014b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7ffff640 1 -- 172.21.15.70:0/2401833648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70009460 con 0x7f9d800ca420 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7ffff640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 0x7f9d80159ec0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800b410 tx=0x7f9d6800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68011830 con 0x7f9d800ca420 2023-08-25T09:45:59.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d680119b0 con 0x7f9d800ca420 2023-08-25T09:45:59.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68011d60 con 0x7f9d800ca420 2023-08-25T09:45:59.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d80154d20 con 0x7f9d800ca420 2023-08-25T09:45:59.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.440+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d801581f0 con 0x7f9d800ca420 2023-08-25T09:45:59.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.441+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d80155240 con 0x7f9d800ca420 2023-08-25T09:45:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.442+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d6801a400 con 0x7f9d800ca420 2023-08-25T09:45:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.443+0000 7f9d7d7fa640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d6006a050 0x7f9d6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.443+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6236+0+0 (secure 0 0 0) 0x7f9d68088740 con 0x7f9d800ca420 2023-08-25T09:45:59.445 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.443+0000 7f9d84fbb640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d6006a050 0x7f9d6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.444+0000 7f9d84fbb640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d6006a050 0x7f9d6006c510 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9d70035000 tx=0x7f9d70030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.447+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d80155240 con 0x7f9d800ca420 2023-08-25T09:45:59.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.675+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9d80155240 con 0x7f9d800ca420 2023-08-25T09:45:59.678 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:45:59.677+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:45:59.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.690+0000 7f9d7d7fa640 1 -- 172.21.15.70:0/2401833648 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9d80155240 con 0x7f9d800ca420 2023-08-25T09:45:59.692 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:45:59.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d6006a050 msgr2=0x7f9d6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d6006a050 0x7f9d6006c510 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9d70035000 tx=0x7f9d70030000 comp rx=0 tx=0).stop 2023-08-25T09:45:59.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 msgr2=0x7f9d80159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 0x7f9d80159ec0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800b410 tx=0x7f9d6800b8d0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 shutdown_connections 2023-08-25T09:45:59.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d6006a050 0x7f9d6006c510 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d8015ccf0 0x7f9d8014b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d800ca420 0x7f9d80159ec0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 --2- 172.21.15.70:0/2401833648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d800c7a70 0x7f9d80157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.694+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 >> 172.21.15.70:0/2401833648 conn(0x7f9d800ce760 msgr2=0x7f9d8015e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:59.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.695+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 shutdown_connections 2023-08-25T09:45:59.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.695+0000 7f9d8721f640 1 -- 172.21.15.70:0/2401833648 wait complete. 2023-08-25T09:45:59.708 INFO:tasks.ceph.mds.c.smithi070.stderr: -9923> 2023-08-25T09:45:11.953+0000 7ff80cdb3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:45:59.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:45:59.772 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:45:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.912+0000 7fb0fb59e640 1 Processor -- start 2023-08-25T09:45:59.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.912+0000 7fb0fb59e640 1 -- start start 2023-08-25T09:45:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.913+0000 7fb0fb59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0fc06fa80 0x7fb0fc1657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.913+0000 7fb0fb59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc15e4a0 con 0x7fb0fc15abb0 2023-08-25T09:45:59.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.913+0000 7fb0fb59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc15f020 con 0x7fb0fc15a7d0 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.913+0000 7fb0fb59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c53b0 con 0x7fb0fc06fa80 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb0fc15abb0 0x7fb0fc0b3d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41298/0 (socket says 172.21.15.70:41298) 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f3fff640 1 -- 172.21.15.70:0/2359384517 learned_addr learned my addr 172.21.15.70:0/2359384517 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0fad9d640 1 --2- 172.21.15.70:0/2359384517 >> v2:172.21.15.155:3300/0 conn(0x7fb0fc06fa80 0x7fb0fc1657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455885154 0 0) 0x7fb0fc15e4a0 con 0x7fb0fc15abb0 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e8003520 con 0x7fb0fc15abb0 2023-08-25T09:45:59.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978589234 0 0) 0x7fb0e8003520 con 0x7fb0fc15abb0 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 >> v2:172.21.15.155:3300/0 conn(0x7fb0fc06fa80 msgr2=0x7fb0fc1657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 --2- 172.21.15.70:0/2359384517 >> v2:172.21.15.155:3300/0 conn(0x7fb0fc06fa80 0x7fb0fc1657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 >> v1:172.21.15.155:6789/0 conn(0x7fb0fc15a7d0 legacy=0x7fb0fc0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc15e6f0 con 0x7fb0fc15abb0 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb0e4001670 con 0x7fb0fc15abb0 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.914+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 >> v1:172.21.15.70:6789/0 conn(0x7fb0fc15abb0 legacy=0x7fb0fc0b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f9d9b640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e8003970 0x7fb0e8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f9d9b640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e80047c0 0x7fb0e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f9d9b640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0e80051f0 0x7fb0e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e800c120 con 0x7fb0e80047c0 2023-08-25T09:45:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e800c290 con 0x7fb0e8003970 2023-08-25T09:45:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e800c450 con 0x7fb0e80051f0 2023-08-25T09:45:59.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f3fff640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e80047c0 0x7fb0e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f3fff640 1 -- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0e80051f0 msgr2=0x7fb0e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f3fff640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0e80051f0 0x7fb0e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f3fff640 1 -- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e8003970 msgr2=0x7fb0e8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:45:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0fa59c640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e8003970 0x7fb0e8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.915+0000 7fb0f3fff640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e8003970 0x7fb0e8003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.916+0000 7fb0fad9d640 1 -- 172.21.15.70:0/2359384517 reap_dead start 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.916+0000 7fb0f3fff640 1 -- 172.21.15.70:0/2359384517 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8003520 con 0x7fb0e80047c0 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.916+0000 7fb0fa59c640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e8003970 0x7fb0e8003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0f3fff640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e80047c0 0x7fb0e8004c40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c210 tx=0x7fb0e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3501ad472ea33d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4005ad0 con 0x7fb0e80047c0 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e4005c50 con 0x7fb0e80047c0 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0f9d9b640 1 -- 172.21.15.70:0/2359384517 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e402c880 con 0x7fb0e80047c0 2023-08-25T09:45:59.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0fb59e640 1 -- 172.21.15.70:0/2359384517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e80047c0 msgr2=0x7fb0e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e80047c0 0x7fb0e8004c40 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c210 tx=0x7fb0e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0fb59e640 1 -- 172.21.15.70:0/2359384517 shutdown_connections 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/2359384517 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0e80047c0 0x7fb0e8004c40 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.917+0000 7fb0fb59e640 1 -- 172.21.15.70:0/2359384517 >> 172.21.15.70:0/2359384517 conn(0x7fb0fc0aff60 msgr2=0x7fb0fc0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.918+0000 7fb0fb59e640 1 -- 172.21.15.70:0/2359384517 shutdown_connections 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.918+0000 7fb0fb59e640 1 -- 172.21.15.70:0/2359384517 wait complete. 2023-08-25T09:45:59.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.919+0000 7fb0fb59e640 1 Processor -- start 2023-08-25T09:45:59.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.919+0000 7fb0fb59e640 1 -- start start 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.920+0000 7fb0fb59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e80047c0 0x7fb0fc150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.920+0000 7fb0fb59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0fc0cdbf0 0x7fb0fc1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.920+0000 7fb0fb59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 0x7fb0fc0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.920+0000 7fb0fb59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c6710 con 0x7fb0fc151df0 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.920+0000 7fb0fb59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc152270 con 0x7fb0e80047c0 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.920+0000 7fb0fb59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc150ca0 con 0x7fb0fc0cdbf0 2023-08-25T09:45:59.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 0x7fb0fc0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 0x7fb0fc0cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36918/0 (socket says 172.21.15.70:36918) 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 -- 172.21.15.70:0/3995437565 learned_addr learned my addr 172.21.15.70:0/3995437565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 -- 172.21.15.70:0/3995437565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0fc0cdbf0 msgr2=0x7fb0fc1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0fc0cdbf0 0x7fb0fc1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 -- 172.21.15.70:0/3995437565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e80047c0 msgr2=0x7fb0fc150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e80047c0 0x7fb0fc150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:45:59.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 -- 172.21.15.70:0/3995437565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc150f80 con 0x7fb0fc151df0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.921+0000 7fb0fad9d640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 0x7fb0fc0cc000 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0addf0 tx=0x7fb0f400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.922+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4010040 con 0x7fb0fc151df0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.922+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f400bb70 con 0x7fb0fc151df0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.922+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4014400 con 0x7fb0fc151df0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.922+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc151260 con 0x7fb0fc151df0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.922+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc150990 con 0x7fb0fc151df0 2023-08-25T09:45:59.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.923+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc0c6880 con 0x7fb0fc151df0 2023-08-25T09:45:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.924+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0f4014560 con 0x7fb0fc151df0 2023-08-25T09:45:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.925+0000 7fb0f2ffd640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0dc06a080 0x7fb0dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:45:59.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.925+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fb0f4088a50 con 0x7fb0fc151df0 2023-08-25T09:45:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.925+0000 7fb0fa59c640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0dc06a080 0x7fb0dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:45:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.925+0000 7fb0fa59c640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0dc06a080 0x7fb0dc06c540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc063520 tx=0x7fb0ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:45:59.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:45:59.929+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0fc0c6880 con 0x7fb0fc151df0 2023-08-25T09:46:00.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.156+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb0fc066fb0 con 0x7fb0fc151df0 2023-08-25T09:46:00.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.686+0000 7fb0f2ffd640 1 -- 172.21.15.70:0/3995437565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb0f4053f90 con 0x7fb0fc151df0 2023-08-25T09:46:00.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.690+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0dc06a080 msgr2=0x7fb0dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.690+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0dc06a080 0x7fb0dc06c540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc063520 tx=0x7fb0ec009040 comp rx=0 tx=0).stop 2023-08-25T09:46:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.690+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 msgr2=0x7fb0fc0cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.690+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 0x7fb0fc0cc000 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc0addf0 tx=0x7fb0f400ef30 comp rx=0 tx=0).stop 2023-08-25T09:46:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 shutdown_connections 2023-08-25T09:46:00.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb0dc06a080 0x7fb0dc06c540 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0fc151df0 0x7fb0fc0cc000 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0fc0cdbf0 0x7fb0fc1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 --2- 172.21.15.70:0/3995437565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0e80047c0 0x7fb0fc150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 >> 172.21.15.70:0/3995437565 conn(0x7fb0fc0aff60 msgr2=0x7fb0fc1581f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:00.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 shutdown_connections 2023-08-25T09:46:00.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.691+0000 7fb0fb59e640 1 -- 172.21.15.70:0/3995437565 wait complete. 2023-08-25T09:46:00.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:46:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.931+0000 7f10aa887640 1 Processor -- start 2023-08-25T09:46:00.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.931+0000 7f10aa887640 1 -- start start 2023-08-25T09:46:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.933+0000 7f10aa887640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f10a4155120 0x7f10a40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.933+0000 7f10aa887640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40cd890 0x7f10a40ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.933+0000 7f10aa887640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a40ca820 0x7f10a406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.933+0000 7f10aa887640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40caf30 con 0x7f10a40ca820 2023-08-25T09:46:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.933+0000 7f10aa887640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40cdd10 con 0x7f10a4155120 2023-08-25T09:46:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.933+0000 7f10aa887640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40cacb0 con 0x7f10a40cd890 2023-08-25T09:46:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a40ca820 0x7f10a406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a40ca820 0x7f10a406aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36926/0 (socket says 172.21.15.70:36926) 2023-08-25T09:46:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 -- 172.21.15.70:0/2426839706 learned_addr learned my addr 172.21.15.70:0/2426839706 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 -- 172.21.15.70:0/2426839706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40cd890 msgr2=0x7f10a40ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 --2- 172.21.15.70:0/2426839706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40cd890 0x7f10a40ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 -- 172.21.15.70:0/2426839706 >> v2:172.21.15.155:3300/0 conn(0x7f10a4155120 msgr2=0x7f10a40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 --2- 172.21.15.70:0/2426839706 >> v2:172.21.15.155:3300/0 conn(0x7f10a4155120 0x7f10a40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.934+0000 7f10a8e24640 1 -- 172.21.15.70:0/2426839706 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a415e6f0 con 0x7f10a40ca820 2023-08-25T09:46:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.935+0000 7f10a8e24640 1 --2- 172.21.15.70:0/2426839706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a40ca820 0x7f10a406aa70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f10a40ce610 tx=0x7f109c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413b87c251aecd96 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.935+0000 7f10a2ffd640 1 -- 172.21.15.70:0/2426839706 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c031860 con 0x7f10a40ca820 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.935+0000 7f10a2ffd640 1 -- 172.21.15.70:0/2426839706 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c0319c0 con 0x7f10a40ca820 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.935+0000 7f10a2ffd640 1 -- 172.21.15.70:0/2426839706 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c031d50 con 0x7f10a40ca820 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 -- 172.21.15.70:0/2426839706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a40ca820 msgr2=0x7f10a406aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 --2- 172.21.15.70:0/2426839706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a40ca820 0x7f10a406aa70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f10a40ce610 tx=0x7f109c02b900 comp rx=0 tx=0).stop 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 -- 172.21.15.70:0/2426839706 shutdown_connections 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 --2- 172.21.15.70:0/2426839706 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a40ca820 0x7f10a406aa70 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 --2- 172.21.15.70:0/2426839706 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40cd890 0x7f10a40ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 --2- 172.21.15.70:0/2426839706 >> v2:172.21.15.155:3300/0 conn(0x7f10a4155120 0x7f10a40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 -- 172.21.15.70:0/2426839706 >> 172.21.15.70:0/2426839706 conn(0x7f10a4069c70 msgr2=0x7f10a406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.936+0000 7f10aa887640 1 -- 172.21.15.70:0/2426839706 shutdown_connections 2023-08-25T09:46:00.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.937+0000 7f10aa887640 1 -- 172.21.15.70:0/2426839706 wait complete. 2023-08-25T09:46:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.938+0000 7f10aa887640 1 Processor -- start 2023-08-25T09:46:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.938+0000 7f10aa887640 1 -- start start 2023-08-25T09:46:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.939+0000 7f10aa887640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40ca820 0x7f10a40ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.939+0000 7f10aa887640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a40cd890 0x7f10a4159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.939+0000 7f10aa887640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 0x7f10a4159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.939+0000 7f10aa887640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c9a30 con 0x7f10a4155120 2023-08-25T09:46:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.939+0000 7f10aa887640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c95a0 con 0x7f10a40cd890 2023-08-25T09:46:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.939+0000 7f10aa887640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40c8990 con 0x7f10a40ca820 2023-08-25T09:46:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 0x7f10a4159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:00.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 0x7f10a4159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36942/0 (socket says 172.21.15.70:36942) 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 -- 172.21.15.70:0/3508451163 learned_addr learned my addr 172.21.15.70:0/3508451163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a3fff640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40ca820 0x7f10a40ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 -- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40ca820 msgr2=0x7f10a40ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40ca820 0x7f10a40ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 -- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a40cd890 msgr2=0x7f10a4159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a37fe640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a40cd890 0x7f10a4159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:00.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a40cd890 0x7f10a4159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a8e24640 1 -- 172.21.15.70:0/3508451163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0094d0 con 0x7f10a4155120 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a3fff640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40ca820 0x7f10a40ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.940+0000 7f10a37fe640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a40cd890 0x7f10a4159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.941+0000 7f10a8e24640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 0x7f10a4159e50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f109c0090b0 tx=0x7f109c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.941+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c03adf0 con 0x7f10a4155120 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.941+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c0443f0 con 0x7f10a4155120 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.941+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40c8c70 con 0x7f10a4155120 2023-08-25T09:46:00.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.941+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c0447a0 con 0x7f10a4155120 2023-08-25T09:46:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.941+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a40c9230 con 0x7f10a4155120 2023-08-25T09:46:00.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.942+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a404e420 con 0x7f10a4155120 2023-08-25T09:46:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.943+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f109c04d450 con 0x7f10a4155120 2023-08-25T09:46:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.944+0000 7f10a0ff9640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a080 0x7f108406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.944+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f109c0aad00 con 0x7f10a4155120 2023-08-25T09:46:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.944+0000 7f10a3fff640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a080 0x7f108406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:00.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.945+0000 7f10a3fff640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a080 0x7f108406c540 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1094007eb0 tx=0x7f10940073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:00.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:00.948+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f109c076240 con 0x7f10a4155120 2023-08-25T09:46:01.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.136+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10a40715c0 con 0x7f10a4155120 2023-08-25T09:46:01.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.704+0000 7f10a0ff9640 1 -- 172.21.15.70:0/3508451163 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 181+0+0 (secure 0 0 0) 0x7f109c079860 con 0x7f10a4155120 2023-08-25T09:46:01.707 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:46:01.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a080 msgr2=0x7f108406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:01.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a080 0x7f108406c540 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1094007eb0 tx=0x7f10940073c0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 msgr2=0x7f10a4159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:01.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 0x7f10a4159e50 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f109c0090b0 tx=0x7f109c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 shutdown_connections 2023-08-25T09:46:01.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a080 0x7f108406c540 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a4155120 0x7f10a4159e50 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a40cd890 0x7f10a4159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 --2- 172.21.15.70:0/3508451163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a40ca820 0x7f10a40ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 >> 172.21.15.70:0/3508451163 conn(0x7f10a4069c70 msgr2=0x7f10a40b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 shutdown_connections 2023-08-25T09:46:01.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.710+0000 7f10aa887640 1 -- 172.21.15.70:0/3508451163 wait complete. 2023-08-25T09:46:01.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:46:01.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.947+0000 7f3dd22fa640 1 Processor -- start 2023-08-25T09:46:01.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.947+0000 7f3dd22fa640 1 -- start start 2023-08-25T09:46:01.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.948+0000 7f3dd22fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc0b8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.948+0000 7f3dd22fa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3dcc0cf430 0x7f3dcc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.948+0000 7f3dd22fa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3dcc0b5ba0 0x7f3dcc0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd22fa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc0ce270 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd22fa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dcc0d0340 con 0x7f3dcc0cf430 2023-08-25T09:46:01.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd22fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc15d1b0 con 0x7f3dcc15cdd0 2023-08-25T09:46:01.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3dcc0b5ba0 0x7f3dcc0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:01.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3dcc0b5ba0 0x7f3dcc0b5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36958/0 (socket says 172.21.15.70:36958) 2023-08-25T09:46:01.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 -- 172.21.15.70:0/2864937997 learned_addr learned my addr 172.21.15.70:0/2864937997 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:01.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dcb7fe640 1 --2- 172.21.15.70:0/2864937997 >> v2:172.21.15.155:3300/0 conn(0x7f3dcc0cf430 0x7f3dcc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:01.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 -- 172.21.15.70:0/2864937997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 msgr2=0x7f3dcc0b8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:01.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 --2- 172.21.15.70:0/2864937997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc0b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 -- 172.21.15.70:0/2864937997 >> v2:172.21.15.155:3300/0 conn(0x7f3dcc0cf430 msgr2=0x7f3dcc0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:01.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 --2- 172.21.15.70:0/2864937997 >> v2:172.21.15.155:3300/0 conn(0x7f3dcc0cf430 0x7f3dcc0cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.949+0000 7f3dd0897640 1 -- 172.21.15.70:0/2864937997 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc0cfdd0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.950+0000 7f3dd0897640 1 --2- 172.21.15.70:0/2864937997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc0b5fd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0d0550 tx=0x7f3dc402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a925796f15d2d2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.951+0000 7f3dcaffd640 1 -- 172.21.15.70:0/2864937997 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4031640 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.951+0000 7f3dcaffd640 1 -- 172.21.15.70:0/2864937997 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc40317a0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.951+0000 7f3dcaffd640 1 -- 172.21.15.70:0/2864937997 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4031b30 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.951+0000 7f3dd22fa640 1 -- 172.21.15.70:0/2864937997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 msgr2=0x7f3dcc0b5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.951+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/2864937997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc0b5fd0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc0d0550 tx=0x7f3dc402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 -- 172.21.15.70:0/2864937997 shutdown_connections 2023-08-25T09:46:01.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/2864937997 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc0b5fd0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/2864937997 >> v2:172.21.15.155:3300/0 conn(0x7f3dcc0cf430 0x7f3dcc0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/2864937997 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc0b8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 -- 172.21.15.70:0/2864937997 >> 172.21.15.70:0/2864937997 conn(0x7f3dcc070290 msgr2=0x7f3dcc0706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:01.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 -- 172.21.15.70:0/2864937997 shutdown_connections 2023-08-25T09:46:01.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.952+0000 7f3dd22fa640 1 -- 172.21.15.70:0/2864937997 wait complete. 2023-08-25T09:46:01.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.953+0000 7f3dd22fa640 1 Processor -- start 2023-08-25T09:46:01.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.954+0000 7f3dd22fa640 1 -- start start 2023-08-25T09:46:01.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd22fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc15dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd22fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dcc0cf430 0x7f3dcc0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd22fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd22fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc0ca3c0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd22fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc15e340 con 0x7f3dcc0cf430 2023-08-25T09:46:01.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd22fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dcc15e620 con 0x7f3dcc15cdd0 2023-08-25T09:46:01.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc15dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc15dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36962/0 (socket says 172.21.15.70:36962) 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 -- 172.21.15.70:0/578868930 learned_addr learned my addr 172.21.15.70:0/578868930 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 -- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 msgr2=0x7f3dcc156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dd0897640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcb7fe640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dcc0cf430 0x7f3dcc0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 -- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dcc0cf430 msgr2=0x7f3dcc0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dcc0cf430 0x7f3dcc0cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.955+0000 7f3dcbfff640 1 -- 172.21.15.70:0/578868930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40094d0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.956+0000 7f3dcbfff640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc15dd60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00ba50 tx=0x7f3dbc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.956+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc00cbe0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.956+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc00cd60 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.956+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dcc15e900 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.956+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc014680 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.956+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dcc15f4a0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.957+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dcc04e420 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.958+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3dbc01d3b0 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.959+0000 7f3dc8ff9640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dac06a050 0x7f3dac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:01.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.959+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5855+0+0 (secure 0 0 0) 0x7f3dbc088540 con 0x7f3dcc0b5ba0 2023-08-25T09:46:01.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.959+0000 7f3dcb7fe640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dac06a050 0x7f3dac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:01.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.960+0000 7f3dcb7fe640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dac06a050 0x7f3dac06c510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3db4006fd0 tx=0x7f3db4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:01.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:01.963+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3dbc053c60 con 0x7f3dcc0b5ba0 2023-08-25T09:46:02.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.156+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3dcc154390 con 0x7f3dcc0b5ba0 2023-08-25T09:46:02.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.706+0000 7f3dc8ff9640 1 -- 172.21.15.70:0/578868930 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3dbc057280 con 0x7f3dcc0b5ba0 2023-08-25T09:46:02.709 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:46:02.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dac06a050 msgr2=0x7f3dac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:02.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dac06a050 0x7f3dac06c510 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3db4006fd0 tx=0x7f3db4008040 comp rx=0 tx=0).stop 2023-08-25T09:46:02.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 msgr2=0x7f3dcc15dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc15dd60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc00ba50 tx=0x7f3dbc00bf10 comp rx=0 tx=0).stop 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 shutdown_connections 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dac06a050 0x7f3dac06c510 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3dcc15cdd0 0x7f3dcc156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3dcc0cf430 0x7f3dcc0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 --2- 172.21.15.70:0/578868930 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3dcc0b5ba0 0x7f3dcc15dd60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.711+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 >> 172.21.15.70:0/578868930 conn(0x7f3dcc070290 msgr2=0x7f3dcc1586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.712+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 shutdown_connections 2023-08-25T09:46:02.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.712+0000 7f3dd22fa640 1 -- 172.21.15.70:0/578868930 wait complete. 2023-08-25T09:46:02.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:46:02.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.948+0000 7fa8c1747640 1 Processor -- start 2023-08-25T09:46:02.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.948+0000 7fa8c1747640 1 -- start start 2023-08-25T09:46:02.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.950+0000 7fa8c1747640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8bc1607e0 0x7fa8bc0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.950+0000 7fa8c1747640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8bc06c8b0 con 0x7fa8bc0cea30 2023-08-25T09:46:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.950+0000 7fa8c1747640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8bc06b510 con 0x7fa8bc15cbb0 2023-08-25T09:46:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.950+0000 7fa8c1747640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8bc0ceec0 con 0x7fa8bc1607e0 2023-08-25T09:46:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.950+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8bc1607e0 0x7fa8bc0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.951+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8bc1607e0 0x7fa8bc0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36964/0 (socket says 172.21.15.70:36964) 2023-08-25T09:46:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.951+0000 7fa8b3fff640 1 -- 172.21.15.70:0/3279222880 learned_addr learned my addr 172.21.15.70:0/3279222880 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.951+0000 7fa8b3fff640 1 -- 172.21.15.70:0/3279222880 >> v1:172.21.15.155:6789/0 conn(0x7fa8bc15cbb0 legacy=0x7fa8bc06ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.951+0000 7fa8b3fff640 1 -- 172.21.15.70:0/3279222880 >> v1:172.21.15.70:6789/0 conn(0x7fa8bc0cea30 legacy=0x7fa8bc0b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:02.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.951+0000 7fa8b3fff640 1 -- 172.21.15.70:0/3279222880 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc06b7f0 con 0x7fa8bc1607e0 2023-08-25T09:46:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.952+0000 7fa8b3fff640 1 --2- 172.21.15.70:0/3279222880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc1607e0 0x7fa8bc0ce4f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009870 tx=0x7fa8a402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921f03b11bdd0597 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.952+0000 7fa8ba7fc640 1 -- 172.21.15.70:0/3279222880 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a402f850 con 0x7fa8bc1607e0 2023-08-25T09:46:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8ba7fc640 1 -- 172.21.15.70:0/3279222880 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a402f9b0 con 0x7fa8bc1607e0 2023-08-25T09:46:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8ba7fc640 1 -- 172.21.15.70:0/3279222880 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a402fd40 con 0x7fa8bc1607e0 2023-08-25T09:46:02.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 -- 172.21.15.70:0/3279222880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc1607e0 msgr2=0x7fa8bc0ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 --2- 172.21.15.70:0/3279222880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc1607e0 0x7fa8bc0ce4f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4009870 tx=0x7fa8a402ba30 comp rx=0 tx=0).stop 2023-08-25T09:46:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 -- 172.21.15.70:0/3279222880 shutdown_connections 2023-08-25T09:46:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 --2- 172.21.15.70:0/3279222880 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc1607e0 0x7fa8bc0ce4f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 -- 172.21.15.70:0/3279222880 >> 172.21.15.70:0/3279222880 conn(0x7fa8bc069850 msgr2=0x7fa8bc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 -- 172.21.15.70:0/3279222880 shutdown_connections 2023-08-25T09:46:02.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.953+0000 7fa8c1747640 1 -- 172.21.15.70:0/3279222880 wait complete. 2023-08-25T09:46:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.955+0000 7fa8c1747640 1 Processor -- start 2023-08-25T09:46:02.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.955+0000 7fa8c1747640 1 -- start start 2023-08-25T09:46:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.956+0000 7fa8c1747640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 0x7fa8bc14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.956+0000 7fa8c1747640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8bc15cbb0 0x7fa8bc15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.956+0000 7fa8c1747640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8bc1607e0 0x7fa8bc150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:02.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.956+0000 7fa8c1747640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc15d640 con 0x7fa8bc0cea30 2023-08-25T09:46:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.956+0000 7fa8c1747640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc15d920 con 0x7fa8bc1607e0 2023-08-25T09:46:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.956+0000 7fa8c1747640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8bc15dc00 con 0x7fa8bc15cbb0 2023-08-25T09:46:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 0x7fa8bc14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 0x7fa8bc14bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36978/0 (socket says 172.21.15.70:36978) 2023-08-25T09:46:02.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 -- 172.21.15.70:0/1384845081 learned_addr learned my addr 172.21.15.70:0/1384845081 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 -- 172.21.15.70:0/1384845081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8bc15cbb0 msgr2=0x7fa8bc15fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8bc15cbb0 0x7fa8bc15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 -- 172.21.15.70:0/1384845081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8bc1607e0 msgr2=0x7fa8bc150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8bc1607e0 0x7fa8bc150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 -- 172.21.15.70:0/1384845081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a40094d0 con 0x7fa8bc0cea30 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.957+0000 7fa8baffd640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 0x7fa8bc14bac0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00c430 tx=0x7fa8ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.958+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac011790 con 0x7fa8bc0cea30 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.958+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0118f0 con 0x7fa8bc0cea30 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.958+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac011ce0 con 0x7fa8bc0cea30 2023-08-25T09:46:02.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.958+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8bc15dee0 con 0x7fa8bc0cea30 2023-08-25T09:46:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.958+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8bc14c000 con 0x7fa8bc0cea30 2023-08-25T09:46:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.959+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8bc04e420 con 0x7fa8bc0cea30 2023-08-25T09:46:02.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.960+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8ac01c460 con 0x7fa8bc0cea30 2023-08-25T09:46:02.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.961+0000 7fa8b37fe640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa89c06a050 0x7fa89c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:02.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.961+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa8ac0882b0 con 0x7fa8bc0cea30 2023-08-25T09:46:02.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.961+0000 7fa8b3fff640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa89c06a050 0x7fa89c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:02.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.962+0000 7fa8b3fff640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa89c06a050 0x7fa89c06c510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4036000 tx=0x7fa8a4039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:02.966+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa8ac053c20 con 0x7fa8bc0cea30 2023-08-25T09:46:03.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.158+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa8bc06f140 con 0x7fa8bc0cea30 2023-08-25T09:46:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.166+0000 7fa8b37fe640 1 -- 172.21.15.70:0/1384845081 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8ac057240 con 0x7fa8bc0cea30 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa89c06a050 msgr2=0x7fa89c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa89c06a050 0x7fa89c06c510 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4036000 tx=0x7fa8a4039040 comp rx=0 tx=0).stop 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 msgr2=0x7fa8bc14bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 0x7fa8bc14bac0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00c430 tx=0x7fa8ac00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 shutdown_connections 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa89c06a050 0x7fa89c06c510 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8bc1607e0 0x7fa8bc150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8bc15cbb0 0x7fa8bc15fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 --2- 172.21.15.70:0/1384845081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8bc0cea30 0x7fa8bc14bac0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 >> 172.21.15.70:0/1384845081 conn(0x7fa8bc069850 msgr2=0x7fa8bc155a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 shutdown_connections 2023-08-25T09:46:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.169+0000 7fa8c1747640 1 -- 172.21.15.70:0/1384845081 wait complete. 2023-08-25T09:46:03.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-08-25T09:46:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.397+0000 7f3de7d99640 1 Processor -- start 2023-08-25T09:46:03.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.398+0000 7f3de7d99640 1 -- start start 2023-08-25T09:46:03.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.399+0000 7f3de7d99640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3de006ad20 0x7f3de006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.399+0000 7f3de7d99640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de00caf30 0x7f3de00cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.399+0000 7f3de7d99640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de00cdd60 con 0x7f3de015cd90 2023-08-25T09:46:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.399+0000 7f3de7d99640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de00cc9e0 con 0x7f3de006ad20 2023-08-25T09:46:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.399+0000 7f3de7d99640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00cccc0 con 0x7f3de00caf30 2023-08-25T09:46:03.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.400+0000 7f3de5334640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3de006ad20 0x7f3de006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.400+0000 7f3de5334640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3de006ad20 0x7f3de006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:50794/0 (socket says 172.21.15.70:50794) 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.400+0000 7f3de5334640 1 -- 172.21.15.70:0/3792772452 learned_addr learned my addr 172.21.15.70:0/3792772452 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.400+0000 7f3de5b35640 1 --1- 172.21.15.70:0/3792772452 >> v1:172.21.15.155:6789/0 conn(0x7f3de015cd90 0x7f3de00718e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:35236/0 (socket says 172.21.15.70:35236) 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.400+0000 7f3de6336640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de00caf30 0x7f3de00cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.400+0000 7f3de4b33640 1 -- 172.21.15.70:0/3792772452 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407472459 0 0) 0x7f3de00cdd60 con 0x7f3de015cd90 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.401+0000 7f3de4b33640 1 -- 172.21.15.70:0/3792772452 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dcc003520 con 0x7f3de015cd90 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.401+0000 7f3de5334640 1 -- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de00caf30 msgr2=0x7f3de00cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:03.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.401+0000 7f3de5334640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de00caf30 0x7f3de00cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.401+0000 7f3de5334640 1 -- 172.21.15.70:0/3792772452 >> v1:172.21.15.155:6789/0 conn(0x7f3de015cd90 legacy=0x7f3de00718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.401+0000 7f3de5334640 1 -- 172.21.15.70:0/3792772452 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00ccfa0 con 0x7f3de006ad20 2023-08-25T09:46:03.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.401+0000 7f3de6336640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de00caf30 0x7f3de00cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:46:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.402+0000 7f3de5334640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de006ad20 0x7f3de006b1b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=296eeab7bc2c6d57 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.402+0000 7f3de4b33640 1 -- 172.21.15.70:0/3792772452 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0030840 con 0x7f3de006ad20 2023-08-25T09:46:03.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de4b33640 1 -- 172.21.15.70:0/3792772452 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd00309a0 con 0x7f3de006ad20 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de4b33640 1 -- 172.21.15.70:0/3792772452 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0030d10 con 0x7f3de006ad20 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 -- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de006ad20 msgr2=0x7f3de006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de006ad20 0x7f3de006b1b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0009870 tx=0x7f3dd002b980 comp rx=0 tx=0).stop 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 -- 172.21.15.70:0/3792772452 shutdown_connections 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de00caf30 0x7f3de00cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 --2- 172.21.15.70:0/3792772452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de006ad20 0x7f3de006b1b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 -- 172.21.15.70:0/3792772452 >> 172.21.15.70:0/3792772452 conn(0x7f3de006cf60 msgr2=0x7f3de006d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 -- 172.21.15.70:0/3792772452 shutdown_connections 2023-08-25T09:46:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.403+0000 7f3de7d99640 1 -- 172.21.15.70:0/3792772452 wait complete. 2023-08-25T09:46:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.405+0000 7f3de7d99640 1 Processor -- start 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.405+0000 7f3de7d99640 1 -- start start 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de7d99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 0x7f3de00d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de7d99640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de00caf30 0x7f3de00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de7d99640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de015cd90 0x7f3de00b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de7d99640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00d15f0 con 0x7f3de006ad20 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de7d99640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00ca4e0 con 0x7f3de00caf30 2023-08-25T09:46:03.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de7d99640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de00c8b10 con 0x7f3de015cd90 2023-08-25T09:46:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de5b35640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 0x7f3de00d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de5b35640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 0x7f3de00d10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36990/0 (socket says 172.21.15.70:36990) 2023-08-25T09:46:03.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.406+0000 7f3de5b35640 1 -- 172.21.15.70:0/2827525027 learned_addr learned my addr 172.21.15.70:0/2827525027 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de5b35640 1 -- 172.21.15.70:0/2827525027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de015cd90 msgr2=0x7f3de00b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de5b35640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de015cd90 0x7f3de00b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de5b35640 1 -- 172.21.15.70:0/2827525027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de00caf30 msgr2=0x7f3de00c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de5b35640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de00caf30 0x7f3de00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de5b35640 1 -- 172.21.15.70:0/2827525027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8009a70 con 0x7f3de006ad20 2023-08-25T09:46:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de5b35640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 0x7f3de00d10b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c390 tx=0x7f3ddc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0074e0 con 0x7f3de006ad20 2023-08-25T09:46:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc007660 con 0x7f3de006ad20 2023-08-25T09:46:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc004a20 con 0x7f3de006ad20 2023-08-25T09:46:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd00094d0 con 0x7f3de006ad20 2023-08-25T09:46:03.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.407+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de00d02f0 con 0x7f3de006ad20 2023-08-25T09:46:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.408+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de004e420 con 0x7f3de006ad20 2023-08-25T09:46:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.409+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3ddc004b80 con 0x7f3de006ad20 2023-08-25T09:46:03.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.411+0000 7f3dd67fc640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dc406a050 0x7f3dc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.411+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3ddc087ba0 con 0x7f3de006ad20 2023-08-25T09:46:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.411+0000 7f3de5334640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dc406a050 0x7f3dc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.412+0000 7f3de5334640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dc406a050 0x7f3dc406c510 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0036000 tx=0x7f3dd003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:03.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.415+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ddc0536e0 con 0x7f3de006ad20 2023-08-25T09:46:03.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.672+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f3de00b31c0 con 0x7f3de006ad20 2023-08-25T09:46:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.726+0000 7f3dd67fc640 1 -- 172.21.15.70:0/2827525027 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v384) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3ddc056d00 con 0x7f3de006ad20 2023-08-25T09:46:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.728+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dc406a050 msgr2=0x7f3dc406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.728+0000 7f3de7d99640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dc406a050 0x7f3dc406c510 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3dd0036000 tx=0x7f3dd003b040 comp rx=0 tx=0).stop 2023-08-25T09:46:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.728+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 msgr2=0x7f3de00d10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.728+0000 7f3de7d99640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 0x7f3de00d10b0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00c390 tx=0x7f3ddc00c850 comp rx=0 tx=0).stop 2023-08-25T09:46:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 shutdown_connections 2023-08-25T09:46:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3dc406a050 0x7f3dc406c510 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3de015cd90 0x7f3de00b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3de00caf30 0x7f3de00c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 --2- 172.21.15.70:0/2827525027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3de006ad20 0x7f3de00d10b0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 >> 172.21.15.70:0/2827525027 conn(0x7f3de006cf60 msgr2=0x7f3de0160600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 shutdown_connections 2023-08-25T09:46:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.729+0000 7f3de7d99640 1 -- 172.21.15.70:0/2827525027 wait complete. 2023-08-25T09:46:03.748 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-08-25T09:46:03.749 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-08-25T09:46:03.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:03.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.948+0000 7f6a4162d640 1 Processor -- start 2023-08-25T09:46:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.949+0000 7f6a4162d640 1 -- start start 2023-08-25T09:46:03.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.950+0000 7f6a4162d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 0x7f6a3c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.950+0000 7f6a4162d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c070120 con 0x7f6a3c0cbc10 2023-08-25T09:46:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.950+0000 7f6a4162d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c06c470 con 0x7f6a3c06d510 2023-08-25T09:46:03.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.950+0000 7f6a4162d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c06c750 con 0x7f6a3c06bfb0 2023-08-25T09:46:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a3affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6a3c0cbc10 0x7f6a3c06a210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41330/0 (socket says 172.21.15.70:41330) 2023-08-25T09:46:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a3affd640 1 -- 172.21.15.70:0/2247147393 learned_addr learned my addr 172.21.15.70:0/2247147393 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:03.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 0x7f6a3c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747516925 0 0) 0x7f6a3c070120 con 0x7f6a3c0cbc10 2023-08-25T09:46:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a28005730 con 0x7f6a3c0cbc10 2023-08-25T09:46:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2657066489 0 0) 0x7f6a28005730 con 0x7f6a3c0cbc10 2023-08-25T09:46:03.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 msgr2=0x7f6a3c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a39ffb640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 0x7f6a3c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.951+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 >> v1:172.21.15.155:6789/0 conn(0x7f6a3c06d510 legacy=0x7f6a3c0ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c06ca30 con 0x7f6a3c0cbc10 2023-08-25T09:46:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6a2c0019f0 con 0x7f6a3c0cbc10 2023-08-25T09:46:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 >> v1:172.21.15.70:6789/0 conn(0x7f6a3c0cbc10 legacy=0x7f6a3c06a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 0x7f6a28005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a280069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a28006f10 0x7f6a28007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c06c750 con 0x7f6a28006150 2023-08-25T09:46:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2800dca0 con 0x7f6a28006f10 2023-08-25T09:46:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2800de60 con 0x7f6a3c06bfb0 2023-08-25T09:46:03.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a280069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a3affd640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 0x7f6a28005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a3a7fc640 1 -- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 msgr2=0x7f6a28005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c06bfb0 0x7f6a28005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a3a7fc640 1 -- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a28006f10 msgr2=0x7f6a28007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.952+0000 7f6a3b7fe640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a28006f10 0x7f6a28007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.953+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a28006f10 0x7f6a28007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.953+0000 7f6a3a7fc640 1 -- 172.21.15.70:0/2247147393 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a28005730 con 0x7f6a28006150 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.953+0000 7f6a3b7fe640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a28006f10 0x7f6a28007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:03.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a280069d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009a90 tx=0x7f6a28007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e7bf5115e71cee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2402c680 con 0x7f6a28006150 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2402c7e0 con 0x7f6a28006150 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a39ffb640 1 -- 172.21.15.70:0/2247147393 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2402cb70 con 0x7f6a28006150 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a4162d640 1 -- 172.21.15.70:0/2247147393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 msgr2=0x7f6a280069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a4162d640 1 --2- 172.21.15.70:0/2247147393 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a280069d0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009a90 tx=0x7f6a28007a00 comp rx=0 tx=0).stop 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.954+0000 7f6a3b7fe640 1 -- 172.21.15.70:0/2247147393 reap_dead start 2023-08-25T09:46:03.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.955+0000 7f6a4162d640 1 -- 172.21.15.70:0/2247147393 shutdown_connections 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.955+0000 7f6a4162d640 1 -- 172.21.15.70:0/2247147393 >> 172.21.15.70:0/2247147393 conn(0x7f6a3c0cf9c0 msgr2=0x7f6a3c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.955+0000 7f6a4162d640 1 -- 172.21.15.70:0/2247147393 shutdown_connections 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.955+0000 7f6a4162d640 1 -- 172.21.15.70:0/2247147393 wait complete. 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.956+0000 7f6a4162d640 1 Processor -- start 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.957+0000 7f6a4162d640 1 -- start start 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.957+0000 7f6a4162d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a3c15df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.957+0000 7f6a4162d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c0a8470 0x7f6a3c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.957+0000 7f6a4162d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a3c0b2370 0x7f6a3c14a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.957+0000 7f6a4162d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c15e470 con 0x7f6a28006150 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a4162d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c0a8a10 con 0x7f6a3c0b2370 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a4162d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1efb30 con 0x7f6a3c0a8470 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a3c15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a3c15df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37016/0 (socket says 172.21.15.70:37016) 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 -- 172.21.15.70:0/3475179804 learned_addr learned my addr 172.21.15.70:0/3475179804 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:03.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 -- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c0a8470 msgr2=0x7f6a3c0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c0a8470 0x7f6a3c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3b7fe640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a3c0b2370 0x7f6a3c14a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c0a8470 0x7f6a3c0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 -- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a3c0b2370 msgr2=0x7f6a3c14a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a3c0b2370 0x7f6a3c14a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.958+0000 7f6a3affd640 1 -- 172.21.15.70:0/3475179804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c1efe10 con 0x7f6a28006150 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.959+0000 7f6a3affd640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a3c15df30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c007b00 tx=0x7f6a2c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:03.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.959+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c013420 con 0x7f6a28006150 2023-08-25T09:46:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.959+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0135a0 con 0x7f6a28006150 2023-08-25T09:46:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.959+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c013950 con 0x7f6a28006150 2023-08-25T09:46:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.959+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c1f00f0 con 0x7f6a28006150 2023-08-25T09:46:03.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.959+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c0c9070 con 0x7f6a28006150 2023-08-25T09:46:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.960+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c1f0610 con 0x7f6a28006150 2023-08-25T09:46:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.961+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6a2c013ab0 con 0x7f6a28006150 2023-08-25T09:46:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.962+0000 7f6a337fe640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a1c06a050 0x7f6a1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:03.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.962+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6a2c088b90 con 0x7f6a28006150 2023-08-25T09:46:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.962+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a1c06a050 0x7f6a1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.963+0000 7f6a3a7fc640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a1c06a050 0x7f6a1c06c510 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a24036000 tx=0x7f6a24030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:03.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:03.966+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6a3c1f0610 con 0x7f6a28006150 2023-08-25T09:46:04.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.153+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a3c1f0610 con 0x7f6a28006150 2023-08-25T09:46:04.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.156+0000 7f6a337fe640 1 -- 172.21.15.70:0/3475179804 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+14794 (secure 0 0 0) 0x7f6a3c1f0610 con 0x7f6a28006150 2023-08-25T09:46:04.158 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:04.158 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":124,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:02.697714+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6837/2920920514":"2023-08-26T09:45:59.677123+0000","172.21.15.70:6835/2920920514":"2023-08-26T09:45:59.677123+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:04.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.158+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a1c06a050 msgr2=0x7f6a1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.158+0000 7f6a4162d640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a1c06a050 0x7f6a1c06c510 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a24036000 tx=0x7f6a24030000 comp rx=0 tx=0).stop 2023-08-25T09:46:04.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.158+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 msgr2=0x7f6a3c15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.158+0000 7f6a4162d640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a3c15df30 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c007b00 tx=0x7f6a2c00e530 comp rx=0 tx=0).stop 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 shutdown_connections 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6a1c06a050 0x7f6a1c06c510 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6a3c0b2370 0x7f6a3c14a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6a3c0a8470 0x7f6a3c0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 --2- 172.21.15.70:0/3475179804 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6a28006150 0x7f6a3c15df30 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 >> 172.21.15.70:0/3475179804 conn(0x7f6a3c0cf9c0 msgr2=0x7f6a3c0730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:04.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 shutdown_connections 2023-08-25T09:46:04.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.159+0000 7f6a4162d640 1 -- 172.21.15.70:0/3475179804 wait complete. 2023-08-25T09:46:04.180 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:04.181 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:04.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.407+0000 7f937f10c640 1 Processor -- start 2023-08-25T09:46:04.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.407+0000 7f937f10c640 1 -- start start 2023-08-25T09:46:04.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.408+0000 7f937f10c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9378159bf0 0x7f937815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.408+0000 7f937f10c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937815e780 con 0x7f93780cd660 2023-08-25T09:46:04.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.408+0000 7f937f10c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f937806f8e0 con 0x7f937806f0e0 2023-08-25T09:46:04.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.408+0000 7f937f10c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f937806fbc0 con 0x7f9378159bf0 2023-08-25T09:46:04.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.409+0000 7f937d6a9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f93780cd660 0x7f93780c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41344/0 (socket says 172.21.15.70:41344) 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.409+0000 7f937d6a9640 1 -- 172.21.15.70:0/1863266512 learned_addr learned my addr 172.21.15.70:0/1863266512 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.409+0000 7f937cea8640 1 --2- 172.21.15.70:0/1863266512 >> v2:172.21.15.155:3300/0 conn(0x7f9378159bf0 0x7f937815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.409+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246947338 0 0) 0x7f937815e780 con 0x7f93780cd660 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.409+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9368005730 con 0x7f93780cd660 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1310959453 0 0) 0x7f9368005730 con 0x7f93780cd660 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 >> v2:172.21.15.155:3300/0 conn(0x7f9378159bf0 msgr2=0x7f937815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:04.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 --2- 172.21.15.70:0/1863266512 >> v2:172.21.15.155:3300/0 conn(0x7f9378159bf0 0x7f937815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 >> v1:172.21.15.155:6789/0 conn(0x7f937806f0e0 legacy=0x7f9378070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937806fea0 con 0x7f93780cd660 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f93700019f0 con 0x7f93780cd660 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 >> v1:172.21.15.70:6789/0 conn(0x7f93780cd660 legacy=0x7f93780c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9368005b80 0x7f9368005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680069d0 0x7f9368006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9368007390 0x7f936800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937806fbc0 con 0x7f9368007390 2023-08-25T09:46:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936800e230 con 0x7f93680069d0 2023-08-25T09:46:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936800e3f0 con 0x7f9368005b80 2023-08-25T09:46:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f937d6a9640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9368007390 0x7f936800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f937cea8640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9368005b80 0x7f9368005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.410+0000 7f9377fff640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680069d0 0x7f9368006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.411+0000 7f937d6a9640 1 -- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9368005b80 msgr2=0x7f9368005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:04.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.411+0000 7f937d6a9640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9368005b80 0x7f9368005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.411+0000 7f937d6a9640 1 -- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680069d0 msgr2=0x7f9368006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.411+0000 7f937d6a9640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93680069d0 0x7f9368006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.411+0000 7f937d6a9640 1 -- 172.21.15.70:0/1863266512 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368005730 con 0x7f9368007390 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.411+0000 7f937d6a9640 1 -- 172.21.15.70:0/1863266512 reap_dead start 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.412+0000 7f937d6a9640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9368007390 0x7f936800db90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f937000a7f0 tx=0x7f9368007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cf3dab6879fcc0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.412+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370034630 con 0x7f9368007390 2023-08-25T09:46:04.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.412+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937002f070 con 0x7f9368007390 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.412+0000 7f93777fe640 1 -- 172.21.15.70:0/1863266512 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700349f0 con 0x7f9368007390 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.412+0000 7f937f10c640 1 -- 172.21.15.70:0/1863266512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9368007390 msgr2=0x7f936800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.412+0000 7f937f10c640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9368007390 0x7f936800db90 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f937000a7f0 tx=0x7f9368007a00 comp rx=0 tx=0).stop 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.413+0000 7f937f10c640 1 -- 172.21.15.70:0/1863266512 shutdown_connections 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.413+0000 7f937f10c640 1 --2- 172.21.15.70:0/1863266512 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9368007390 0x7f936800db90 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.413+0000 7f937f10c640 1 -- 172.21.15.70:0/1863266512 >> 172.21.15.70:0/1863266512 conn(0x7f93780cb0d0 msgr2=0x7f93780cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:04.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.413+0000 7f937f10c640 1 -- 172.21.15.70:0/1863266512 shutdown_connections 2023-08-25T09:46:04.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.413+0000 7f937f10c640 1 -- 172.21.15.70:0/1863266512 wait complete. 2023-08-25T09:46:04.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.415+0000 7f937f10c640 1 Processor -- start 2023-08-25T09:46:04.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.415+0000 7f937f10c640 1 -- start start 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937f10c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9368007390 0x7f93781f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937f10c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 0x7f93780c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937f10c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f937814ff70 0x7f93780c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937f10c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93781e6310 con 0x7f93780c4c10 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937f10c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93781e5a20 con 0x7f9368007390 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937f10c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93781e5cc0 con 0x7f937814ff70 2023-08-25T09:46:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 0x7f93780c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 0x7f93780c2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37032/0 (socket says 172.21.15.70:37032) 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 -- 172.21.15.70:0/1339525363 learned_addr learned my addr 172.21.15.70:0/1339525363 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937cea8640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9368007390 0x7f93781f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937d6a9640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f937814ff70 0x7f93780c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 -- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f937814ff70 msgr2=0x7f93780c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f937814ff70 0x7f93780c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 -- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9368007390 msgr2=0x7f93781f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9368007390 0x7f93781f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f9377fff640 1 -- 172.21.15.70:0/1339525363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93781e5fa0 con 0x7f93780c4c10 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.416+0000 7f937cea8640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9368007390 0x7f93781f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.417+0000 7f9377fff640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 0x7f93780c2ae0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f936400e480 tx=0x7f936400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.417+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93640052e0 con 0x7f93780c4c10 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.417+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364005440 con 0x7f93780c4c10 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.417+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364003830 con 0x7f93780c4c10 2023-08-25T09:46:04.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.417+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93780c3020 con 0x7f93780c4c10 2023-08-25T09:46:04.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.417+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93781503f0 con 0x7f93780c4c10 2023-08-25T09:46:04.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.418+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93781e6130 con 0x7f93780c4c10 2023-08-25T09:46:04.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.419+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9364003990 con 0x7f93780c4c10 2023-08-25T09:46:04.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.420+0000 7f93757fa640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f935c06a050 0x7f935c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.420+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9364087fd0 con 0x7f93780c4c10 2023-08-25T09:46:04.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.420+0000 7f937cea8640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f935c06a050 0x7f935c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.421+0000 7f937cea8640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f935c06a050 0x7f935c06c510 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f936c009980 tx=0x7f936c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:04.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.424+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9364053a50 con 0x7f93780c4c10 2023-08-25T09:46:04.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.611+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9378150610 con 0x7f93780c4c10 2023-08-25T09:46:04.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.612+0000 7f93757fa640 1 -- 172.21.15.70:0/1339525363 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9364057070 con 0x7f93780c4c10 2023-08-25T09:46:04.614 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:04.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f935c06a050 msgr2=0x7f935c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f935c06a050 0x7f935c06c510 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f936c009980 tx=0x7f936c008040 comp rx=0 tx=0).stop 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 msgr2=0x7f93780c2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 0x7f93780c2ae0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f936400e480 tx=0x7f936400e940 comp rx=0 tx=0).stop 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 shutdown_connections 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f935c06a050 0x7f935c06c510 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f937814ff70 0x7f93780c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93780c4c10 0x7f93780c2ae0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 --2- 172.21.15.70:0/1339525363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9368007390 0x7f93781f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 >> 172.21.15.70:0/1339525363 conn(0x7f93780cb0d0 msgr2=0x7f93780cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:04.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 shutdown_connections 2023-08-25T09:46:04.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.615+0000 7f937f10c640 1 -- 172.21.15.70:0/1339525363 wait complete. 2023-08-25T09:46:04.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.848+0000 7fa606714640 1 Processor -- start 2023-08-25T09:46:04.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.848+0000 7fa606714640 1 -- start start 2023-08-25T09:46:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.849+0000 7fa606714640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa6000b0320 0x7fa6000c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.849+0000 7fa606714640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa6000b1ce0 0x7fa6000ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.849+0000 7fa606714640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6000ad8f0 con 0x7fa60015a920 2023-08-25T09:46:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.849+0000 7fa606714640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6000adbb0 con 0x7fa6000b1ce0 2023-08-25T09:46:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.849+0000 7fa606714640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6000c90a0 con 0x7fa6000b0320 2023-08-25T09:46:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa604f11640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa60015a920 0x7fa6000b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41352/0 (socket says 172.21.15.70:41352) 2023-08-25T09:46:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa604f11640 1 -- 172.21.15.70:0/1403270016 learned_addr learned my addr 172.21.15.70:0/1403270016 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa605f13640 1 --2- 172.21.15.70:0/1403270016 >> v2:172.21.15.70:3300/0 conn(0x7fa6000b1ce0 0x7fa6000ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa605712640 1 --2- 172.21.15.70:0/1403270016 >> v2:172.21.15.155:3300/0 conn(0x7fa6000b0320 0x7fa6000c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa5f7fff640 1 -- 172.21.15.70:0/1403270016 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950897896 0 0) 0x7fa6000ad8f0 con 0x7fa60015a920 2023-08-25T09:46:04.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa5f7fff640 1 -- 172.21.15.70:0/1403270016 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5ec003520 con 0x7fa60015a920 2023-08-25T09:46:04.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa605f13640 1 -- 172.21.15.70:0/1403270016 >> v2:172.21.15.155:3300/0 conn(0x7fa6000b0320 msgr2=0x7fa6000c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:04.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa605f13640 1 --2- 172.21.15.70:0/1403270016 >> v2:172.21.15.155:3300/0 conn(0x7fa6000b0320 0x7fa6000c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa605f13640 1 -- 172.21.15.70:0/1403270016 >> v1:172.21.15.70:6789/0 conn(0x7fa60015a920 legacy=0x7fa6000b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.850+0000 7fa605f13640 1 -- 172.21.15.70:0/1403270016 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000c9380 con 0x7fa6000b1ce0 2023-08-25T09:46:04.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.851+0000 7fa605f13640 1 --2- 172.21.15.70:0/1403270016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b1ce0 0x7fa6000ad3b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa6000b29f0 tx=0x7fa5f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d2539fb197719f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa5f7fff640 1 -- 172.21.15.70:0/1403270016 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8031830 con 0x7fa6000b1ce0 2023-08-25T09:46:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa5f7fff640 1 -- 172.21.15.70:0/1403270016 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f8031990 con 0x7fa6000b1ce0 2023-08-25T09:46:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa5f7fff640 1 -- 172.21.15.70:0/1403270016 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8031d20 con 0x7fa6000b1ce0 2023-08-25T09:46:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 -- 172.21.15.70:0/1403270016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b1ce0 msgr2=0x7fa6000ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 --2- 172.21.15.70:0/1403270016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b1ce0 0x7fa6000ad3b0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa6000b29f0 tx=0x7fa5f802b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 -- 172.21.15.70:0/1403270016 shutdown_connections 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 --2- 172.21.15.70:0/1403270016 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b1ce0 0x7fa6000ad3b0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 --2- 172.21.15.70:0/1403270016 >> v2:172.21.15.155:3300/0 conn(0x7fa6000b0320 0x7fa6000c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 -- 172.21.15.70:0/1403270016 >> 172.21.15.70:0/1403270016 conn(0x7fa6000ce830 msgr2=0x7fa6000cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.852+0000 7fa606714640 1 -- 172.21.15.70:0/1403270016 shutdown_connections 2023-08-25T09:46:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.853+0000 7fa606714640 1 -- 172.21.15.70:0/1403270016 wait complete. 2023-08-25T09:46:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.854+0000 7fa606714640 1 Processor -- start 2023-08-25T09:46:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.854+0000 7fa606714640 1 -- start start 2023-08-25T09:46:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.855+0000 7fa606714640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 0x7fa6001f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.855+0000 7fa606714640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6000b1ce0 0x7fa6001ef200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.855+0000 7fa606714640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa60015a920 0x7fa6000adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.855+0000 7fa606714640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001f2ed0 con 0x7fa6000b0320 2023-08-25T09:46:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.855+0000 7fa606714640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000c2cc0 con 0x7fa60015a920 2023-08-25T09:46:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.855+0000 7fa606714640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6000c2fa0 con 0x7fa6000b1ce0 2023-08-25T09:46:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 0x7fa6001f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 0x7fa6001f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37064/0 (socket says 172.21.15.70:37064) 2023-08-25T09:46:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 -- 172.21.15.70:0/118323758 learned_addr learned my addr 172.21.15.70:0/118323758 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 -- 172.21.15.70:0/118323758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6000b1ce0 msgr2=0x7fa6001ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6000b1ce0 0x7fa6001ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 -- 172.21.15.70:0/118323758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa60015a920 msgr2=0x7fa6000adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa60015a920 0x7fa6000adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 -- 172.21.15.70:0/118323758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009400 con 0x7fa6000b0320 2023-08-25T09:46:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa605712640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 0x7fa6001f2990 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009fc0 tx=0x7fa5fc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.856+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc014620 con 0x7fa6000b0320 2023-08-25T09:46:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.857+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc014780 con 0x7fa6000b0320 2023-08-25T09:46:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.857+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc014b30 con 0x7fa6000b0320 2023-08-25T09:46:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.857+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6000c3280 con 0x7fa6000b0320 2023-08-25T09:46:04.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.857+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6000c3840 con 0x7fa6000b0320 2023-08-25T09:46:04.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.858+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa600061460 con 0x7fa6000b0320 2023-08-25T09:46:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.859+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa5fc014c90 con 0x7fa6000b0320 2023-08-25T09:46:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.859+0000 7fa5f5ffb640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5e406a080 0x7fa5e406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.860+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa5fc088d40 con 0x7fa6000b0320 2023-08-25T09:46:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.860+0000 7fa604f11640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5e406a080 0x7fa5e406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.861+0000 7fa604f11640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5e406a080 0x7fa5e406c540 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa6000c2520 tx=0x7fa5f0007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:04.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:04.864+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa5fc0546b0 con 0x7fa6000b0320 2023-08-25T09:46:05.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.053+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6000cdf40 con 0x7fa6000b0320 2023-08-25T09:46:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.054+0000 7fa5f5ffb640 1 -- 172.21.15.70:0/118323758 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v124) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5fc057cd0 con 0x7fa6000b0320 2023-08-25T09:46:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:05.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5e406a080 msgr2=0x7fa5e406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5e406a080 0x7fa5e406c540 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa6000c2520 tx=0x7fa5f0007930 comp rx=0 tx=0).stop 2023-08-25T09:46:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 msgr2=0x7fa6001f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 0x7fa6001f2990 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009fc0 tx=0x7fa5fc00e570 comp rx=0 tx=0).stop 2023-08-25T09:46:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 shutdown_connections 2023-08-25T09:46:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa5e406a080 0x7fa5e406c540 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa60015a920 0x7fa6000adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa6000b1ce0 0x7fa6001ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 --2- 172.21.15.70:0/118323758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa6000b0320 0x7fa6001f2990 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 >> 172.21.15.70:0/118323758 conn(0x7fa6000ce830 msgr2=0x7fa600157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 shutdown_connections 2023-08-25T09:46:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.057+0000 7fa606714640 1 -- 172.21.15.70:0/118323758 wait complete. 2023-08-25T09:46:05.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-08-25T09:46:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.291+0000 7f8fc9d42640 1 Processor -- start 2023-08-25T09:46:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.291+0000 7f8fc9d42640 1 -- start start 2023-08-25T09:46:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc9d42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b15d0 0x7f8fc40c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc9d42640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40ce8f0 con 0x7f8fc415a630 2023-08-25T09:46:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc9d42640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc40cebd0 con 0x7f8fc40ccb20 2023-08-25T09:46:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc9d42640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc40cb7f0 con 0x7f8fc40b15d0 2023-08-25T09:46:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b15d0 0x7f8fc40c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8fc40b15d0 0x7f8fc40c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37070/0 (socket says 172.21.15.70:37070) 2023-08-25T09:46:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc3fff640 1 -- 172.21.15.70:0/2657965476 learned_addr learned my addr 172.21.15.70:0/2657965476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.293+0000 7f8fc37fe640 1 --1- 172.21.15.70:0/2657965476 >> v1:172.21.15.70:6789/0 conn(0x7f8fc415a630 0x7f8fc40cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41364/0 (socket says 172.21.15.70:41364) 2023-08-25T09:46:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.294+0000 7f8fc3fff640 1 -- 172.21.15.70:0/2657965476 >> v1:172.21.15.155:6789/0 conn(0x7f8fc40ccb20 legacy=0x7f8fc40b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.294+0000 7f8fc3fff640 1 -- 172.21.15.70:0/2657965476 >> v1:172.21.15.70:6789/0 conn(0x7f8fc415a630 legacy=0x7f8fc40cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.294+0000 7f8fc3fff640 1 -- 172.21.15.70:0/2657965476 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40cbad0 con 0x7f8fc40b15d0 2023-08-25T09:46:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.294+0000 7f8fc27fc640 1 -- 172.21.15.70:0/2657965476 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745604247 0 0) 0x7f8fc40ce8f0 con 0x7f8fc415a630 2023-08-25T09:46:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.295+0000 7f8fc3fff640 1 --2- 172.21.15.70:0/2657965476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b15d0 0x7f8fc40c95a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009870 tx=0x7f8fbc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132ff7d197ae2c64 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:05.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.295+0000 7f8fc27fc640 1 -- 172.21.15.70:0/2657965476 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc031860 con 0x7f8fc40b15d0 2023-08-25T09:46:05.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.295+0000 7f8fc27fc640 1 -- 172.21.15.70:0/2657965476 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0319c0 con 0x7f8fc40b15d0 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.295+0000 7f8fc27fc640 1 -- 172.21.15.70:0/2657965476 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc031d50 con 0x7f8fc40b15d0 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 -- 172.21.15.70:0/2657965476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b15d0 msgr2=0x7f8fc40c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/2657965476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b15d0 0x7f8fc40c95a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009870 tx=0x7f8fbc02b900 comp rx=0 tx=0).stop 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 -- 172.21.15.70:0/2657965476 shutdown_connections 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/2657965476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc40b15d0 0x7f8fc40c95a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 -- 172.21.15.70:0/2657965476 >> 172.21.15.70:0/2657965476 conn(0x7f8fc4069c70 msgr2=0x7f8fc406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:05.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 -- 172.21.15.70:0/2657965476 shutdown_connections 2023-08-25T09:46:05.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.296+0000 7f8fc9d42640 1 -- 172.21.15.70:0/2657965476 wait complete. 2023-08-25T09:46:05.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.298+0000 7f8fc9d42640 1 Processor -- start 2023-08-25T09:46:05.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.298+0000 7f8fc9d42640 1 -- start start 2023-08-25T09:46:05.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc9d42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc40b15d0 0x7f8fc40c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc9d42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc40ccb20 0x7f8fc4156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc9d42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 0x7f8fc415f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc9d42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c7e50 con 0x7f8fc415a630 2023-08-25T09:46:05.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc9d42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4157810 con 0x7f8fc40ccb20 2023-08-25T09:46:05.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc9d42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40c2fc0 con 0x7f8fc40b15d0 2023-08-25T09:46:05.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 0x7f8fc415f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 0x7f8fc415f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37084/0 (socket says 172.21.15.70:37084) 2023-08-25T09:46:05.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 -- 172.21.15.70:0/639478436 learned_addr learned my addr 172.21.15.70:0/639478436 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:05.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc37fe640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc40b15d0 0x7f8fc40c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc2ffd640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc40ccb20 0x7f8fc4156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 -- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc40b15d0 msgr2=0x7f8fc40c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc40b15d0 0x7f8fc40c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 -- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc40ccb20 msgr2=0x7f8fc4156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc40ccb20 0x7f8fc4156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc3fff640 1 -- 172.21.15.70:0/639478436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0094d0 con 0x7f8fc415a630 2023-08-25T09:46:05.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.299+0000 7f8fc2ffd640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc40ccb20 0x7f8fc4156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:05.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.300+0000 7f8fc3fff640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 0x7f8fc415f6c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02b940 tx=0x7f8fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:05.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.300+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc03adf0 con 0x7f8fc415a630 2023-08-25T09:46:05.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.300+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0443f0 con 0x7f8fc415a630 2023-08-25T09:46:05.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.300+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0447a0 con 0x7f8fc415a630 2023-08-25T09:46:05.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.300+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40c71d0 con 0x7f8fc415a630 2023-08-25T09:46:05.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.300+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc40c7790 con 0x7f8fc415a630 2023-08-25T09:46:05.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.301+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc40c7f90 con 0x7f8fc415a630 2023-08-25T09:46:05.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.302+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8fbc04d450 con 0x7f8fc415a630 2023-08-25T09:46:05.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.304+0000 7f8fbbfff640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fa406a050 0x7f8fa406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.304+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8fbc0aa900 con 0x7f8fc415a630 2023-08-25T09:46:05.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.304+0000 7f8fc37fe640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fa406a050 0x7f8fa406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.305+0000 7f8fc37fe640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fa406a050 0x7f8fa406c510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007ed0 tx=0x7f8fb4007e60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:05.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.308+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fc40c7f90 con 0x7f8fc415a630 2023-08-25T09:46:05.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.568+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f8fc40c7f90 con 0x7f8fc415a630 2023-08-25T09:46:05.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.737+0000 7f8fbbfff640 1 -- 172.21.15.70:0/639478436 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v386) v1 ==== 131+0+0 (secure 0 0 0) 0x7f8fc40c7f90 con 0x7f8fc415a630 2023-08-25T09:46:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fa406a050 msgr2=0x7f8fa406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fa406a050 0x7f8fa406c510 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007ed0 tx=0x7f8fb4007e60 comp rx=0 tx=0).stop 2023-08-25T09:46:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 msgr2=0x7f8fc415f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 0x7f8fc415f6c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02b940 tx=0x7f8fbc0046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 shutdown_connections 2023-08-25T09:46:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8fa406a050 0x7f8fa406c510 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8fc415a630 0x7f8fc415f6c0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8fc40ccb20 0x7f8fc4156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 --2- 172.21.15.70:0/639478436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8fc40b15d0 0x7f8fc40c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.740+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 >> 172.21.15.70:0/639478436 conn(0x7f8fc4069c70 msgr2=0x7f8fc4070ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.741+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 shutdown_connections 2023-08-25T09:46:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.741+0000 7f8fc9d42640 1 -- 172.21.15.70:0/639478436 wait complete. 2023-08-25T09:46:05.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:46:05.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.973+0000 7f7f916ec640 1 Processor -- start 2023-08-25T09:46:05.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.974+0000 7f7f916ec640 1 -- start start 2023-08-25T09:46:05.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.975+0000 7f7f916ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7f8c15ae50 0x7f7f8c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.975+0000 7f7f916ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c0cc5f0 con 0x7f7f8c0cdc20 2023-08-25T09:46:05.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.975+0000 7f7f916ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c070650 con 0x7f7f8c069f60 2023-08-25T09:46:05.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.975+0000 7f7f916ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c070930 con 0x7f7f8c15ae50 2023-08-25T09:46:05.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f827fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7f8c0cdc20 0x7f7f8c069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41378/0 (socket says 172.21.15.70:41378) 2023-08-25T09:46:05.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f827fc640 1 -- 172.21.15.70:0/4080506000 learned_addr learned my addr 172.21.15.70:0/4080506000 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:05.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f8affd640 1 --2- 172.21.15.70:0/4080506000 >> v2:172.21.15.155:3300/0 conn(0x7f7f8c15ae50 0x7f7f8c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976202759 0 0) 0x7f7f8c0cc5f0 con 0x7f7f8c0cdc20 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f78003520 con 0x7f7f8c0cdc20 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588269456 0 0) 0x7f7f78003520 con 0x7f7f8c0cdc20 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 >> v2:172.21.15.155:3300/0 conn(0x7f7f8c15ae50 msgr2=0x7f7f8c0c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.976+0000 7f7f8a7fc640 1 --2- 172.21.15.70:0/4080506000 >> v2:172.21.15.155:3300/0 conn(0x7f7f8c15ae50 0x7f7f8c0c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 >> v1:172.21.15.155:6789/0 conn(0x7f7f8c069f60 legacy=0x7f7f8c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c070c10 con 0x7f7f8c0cdc20 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7f74001670 con 0x7f7f8c0cdc20 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 >> v1:172.21.15.70:6789/0 conn(0x7f7f8c0cdc20 legacy=0x7f7f8c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f78003970 0x7f7f78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f780047c0 0x7f7f78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f78005180 0x7f7f7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c070930 con 0x7f7f780047c0 2023-08-25T09:46:05.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7800c030 con 0x7f7f78005180 2023-08-25T09:46:05.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7800c1f0 con 0x7f7f78003970 2023-08-25T09:46:05.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f827fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f780047c0 0x7f7f78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8b7fe640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f78005180 0x7f7f7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8affd640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f78003970 0x7f7f78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f827fc640 1 -- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f78003970 msgr2=0x7f7f78003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:05.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f827fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f78003970 0x7f7f78003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f827fc640 1 -- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f78005180 msgr2=0x7f7f7800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:05.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f827fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f78005180 0x7f7f7800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f827fc640 1 -- 172.21.15.70:0/4080506000 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f78003520 con 0x7f7f780047c0 2023-08-25T09:46:05.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8b7fe640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f78005180 0x7f7f7800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:05.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.977+0000 7f7f8b7fe640 1 -- 172.21.15.70:0/4080506000 reap_dead start 2023-08-25T09:46:05.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.979+0000 7f7f827fc640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f780047c0 0x7f7f78004c40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c240 tx=0x7f7f780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bf59900027ece7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:05.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.979+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74005ad0 con 0x7f7f780047c0 2023-08-25T09:46:05.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.979+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f74005c30 con 0x7f7f780047c0 2023-08-25T09:46:05.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.979+0000 7f7f8a7fc640 1 -- 172.21.15.70:0/4080506000 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7402caf0 con 0x7f7f780047c0 2023-08-25T09:46:05.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.979+0000 7f7f916ec640 1 -- 172.21.15.70:0/4080506000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f780047c0 msgr2=0x7f7f78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.979+0000 7f7f916ec640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f780047c0 0x7f7f78004c40 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f7f7400c240 tx=0x7f7f780057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.980+0000 7f7f916ec640 1 -- 172.21.15.70:0/4080506000 shutdown_connections 2023-08-25T09:46:05.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.980+0000 7f7f916ec640 1 --2- 172.21.15.70:0/4080506000 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f780047c0 0x7f7f78004c40 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.980+0000 7f7f916ec640 1 -- 172.21.15.70:0/4080506000 >> 172.21.15.70:0/4080506000 conn(0x7f7f8c0cb0d0 msgr2=0x7f7f8c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:05.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.980+0000 7f7f916ec640 1 -- 172.21.15.70:0/4080506000 shutdown_connections 2023-08-25T09:46:05.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.980+0000 7f7f916ec640 1 -- 172.21.15.70:0/4080506000 wait complete. 2023-08-25T09:46:05.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.981+0000 7f7f916ec640 1 Processor -- start 2023-08-25T09:46:05.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.981+0000 7f7f916ec640 1 -- start start 2023-08-25T09:46:05.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.982+0000 7f7f916ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f780047c0 0x7f7f8c0acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.982+0000 7f7f916ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 0x7f7f8c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.982+0000 7f7f916ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f8c0c8c80 0x7f7f8c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.982+0000 7f7f916ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0ca670 con 0x7f7f8c15cb30 2023-08-25T09:46:05.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.982+0000 7f7f916ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0caa90 con 0x7f7f780047c0 2023-08-25T09:46:05.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.982+0000 7f7f916ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0c99f0 con 0x7f7f8c0c8c80 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 0x7f7f8c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 0x7f7f8c15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37096/0 (socket says 172.21.15.70:37096) 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 -- 172.21.15.70:0/3922045949 learned_addr learned my addr 172.21.15.70:0/3922045949 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 -- 172.21.15.70:0/3922045949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f8c0c8c80 msgr2=0x7f7f8c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f8c0c8c80 0x7f7f8c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 -- 172.21.15.70:0/3922045949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f780047c0 msgr2=0x7f7f8c0acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:05.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f780047c0 0x7f7f8c0acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 -- 172.21.15.70:0/3922045949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0c9cd0 con 0x7f7f8c15cb30 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f827fc640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 0x7f7f8c15cfb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7f74001930 tx=0x7f7f7402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.983+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7403c040 con 0x7f7f8c15cb30 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.984+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7402cd80 con 0x7f7f8c15cb30 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.984+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74007290 con 0x7f7f8c15cb30 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.984+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c0c9fb0 con 0x7f7f8c15cb30 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.984+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c15d4f0 con 0x7f7f8c15cb30 2023-08-25T09:46:05.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.985+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c066fb0 con 0x7f7f8c15cb30 2023-08-25T09:46:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.986+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7f74032070 con 0x7f7f8c15cb30 2023-08-25T09:46:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.987+0000 7f7f83fff640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f6c06a080 0x7f7f6c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.987+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7f740a97f0 con 0x7f7f8c15cb30 2023-08-25T09:46:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.987+0000 7f7f8affd640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f6c06a080 0x7f7f6c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.988+0000 7f7f8affd640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f6c06a080 0x7f7f6c06c540 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0099d0 tx=0x7f7f7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:05.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:05.991+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f74075310 con 0x7f7f8c15cb30 2023-08-25T09:46:06.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.177+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7f8c070ec0 con 0x7f7f8c15cb30 2023-08-25T09:46:06.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.187+0000 7f7f83fff640 1 -- 172.21.15.70:0/3922045949 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7f74078930 con 0x7f7f8c15cb30 2023-08-25T09:46:06.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f6c06a080 msgr2=0x7f7f6c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:06.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f6c06a080 0x7f7f6c06c540 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0099d0 tx=0x7f7f7c008040 comp rx=0 tx=0).stop 2023-08-25T09:46:06.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 msgr2=0x7f7f8c15cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:06.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 0x7f7f8c15cfb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f7f74001930 tx=0x7f7f7402e070 comp rx=0 tx=0).stop 2023-08-25T09:46:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 shutdown_connections 2023-08-25T09:46:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7f6c06a080 0x7f7f6c06c540 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f8c0c8c80 0x7f7f8c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f8c15cb30 0x7f7f8c15cfb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 --2- 172.21.15.70:0/3922045949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f780047c0 0x7f7f8c0acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 >> 172.21.15.70:0/3922045949 conn(0x7f7f8c0cb0d0 msgr2=0x7f7f8c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 shutdown_connections 2023-08-25T09:46:06.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.189+0000 7f7f916ec640 1 -- 172.21.15.70:0/3922045949 wait complete. 2023-08-25T09:46:06.209 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:46:06.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:46:06.314 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:46:06.435 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:46:06.436 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:46:06.436 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:06.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:46:06.496 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:46:06.496 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:06.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:46:06.560 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:46:06.560 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:06.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:46:06.622 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:46:06.622 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:06.690 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:46:06.690 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:46:06.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:06.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.908+0000 7ffbd5a3f640 1 Processor -- start 2023-08-25T09:46:06.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.908+0000 7ffbd5a3f640 1 -- start start 2023-08-25T09:46:06.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.909+0000 7ffbd5a3f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbd015e830 0x7ffbd006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.909+0000 7ffbd5a3f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd00b1b60 con 0x7ffbd006b2b0 2023-08-25T09:46:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.909+0000 7ffbd5a3f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd006b7c0 con 0x7ffbd015ec10 2023-08-25T09:46:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.909+0000 7ffbd5a3f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd006bd80 con 0x7ffbd015e830 2023-08-25T09:46:06.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbceffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbd015e830 0x7ffbd006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbceffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffbd015e830 0x7ffbd006ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37102/0 (socket says 172.21.15.70:37102) 2023-08-25T09:46:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbceffd640 1 -- 172.21.15.70:0/17098313 learned_addr learned my addr 172.21.15.70:0/17098313 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbceffd640 1 -- 172.21.15.70:0/17098313 >> v1:172.21.15.155:6789/0 conn(0x7ffbd015ec10 legacy=0x7ffbd00b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbceffd640 1 -- 172.21.15.70:0/17098313 >> v1:172.21.15.70:6789/0 conn(0x7ffbd006b2b0 legacy=0x7ffbd015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbceffd640 1 -- 172.21.15.70:0/17098313 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd006baa0 con 0x7ffbd015e830 2023-08-25T09:46:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.910+0000 7ffbce7fc640 1 -- 172.21.15.70:0/17098313 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289227628 0 0) 0x7ffbd00b1b60 con 0x7ffbd006b2b0 2023-08-25T09:46:06.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.911+0000 7ffbceffd640 1 --2- 172.21.15.70:0/17098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015e830 0x7ffbd006ad70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009870 tx=0x7ffbc002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bb3b9b15a24612 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbce7fc640 1 -- 172.21.15.70:0/17098313 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0030850 con 0x7ffbd015e830 2023-08-25T09:46:06.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbce7fc640 1 -- 172.21.15.70:0/17098313 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00309b0 con 0x7ffbd015e830 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbce7fc640 1 -- 172.21.15.70:0/17098313 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0030d40 con 0x7ffbd015e830 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/17098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015e830 msgr2=0x7ffbd006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/17098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015e830 0x7ffbd006ad70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009870 tx=0x7ffbc002b990 comp rx=0 tx=0).stop 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/17098313 shutdown_connections 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/17098313 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015e830 0x7ffbd006ad70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/17098313 >> 172.21.15.70:0/17098313 conn(0x7ffbd0069850 msgr2=0x7ffbd0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/17098313 shutdown_connections 2023-08-25T09:46:06.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.912+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/17098313 wait complete. 2023-08-25T09:46:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.914+0000 7ffbd5a3f640 1 Processor -- start 2023-08-25T09:46:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.914+0000 7ffbd5a3f640 1 -- start start 2023-08-25T09:46:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.915+0000 7ffbd5a3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbd006b2b0 0x7ffbd015d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.915+0000 7ffbd5a3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd015e830 0x7ffbd015dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:06.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.915+0000 7ffbd5a3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 0x7ffbd00aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.915+0000 7ffbd5a3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01f26d0 con 0x7ffbd015ec10 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.915+0000 7ffbd5a3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01f3530 con 0x7ffbd015e830 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.915+0000 7ffbd5a3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01f28b0 con 0x7ffbd006b2b0 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 0x7ffbd00aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 0x7ffbd00aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37108/0 (socket says 172.21.15.70:37108) 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 -- 172.21.15.70:0/3800454632 learned_addr learned my addr 172.21.15.70:0/3800454632 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 -- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbd006b2b0 msgr2=0x7ffbd015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd015e830 0x7ffbd015dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbceffd640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbd006b2b0 0x7ffbd015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbd006b2b0 0x7ffbd015d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 -- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd015e830 msgr2=0x7ffbd015dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd015e830 0x7ffbd015dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 -- 172.21.15.70:0/3800454632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00094d0 con 0x7ffbd015ec10 2023-08-25T09:46:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbc7fff640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd015e830 0x7ffbd015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.916+0000 7ffbcf7fe640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 0x7ffbd00aca30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c780 tx=0x7ffbc800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.917+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8004b60 con 0x7ffbd015ec10 2023-08-25T09:46:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.917+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc800ce90 con 0x7ffbd015ec10 2023-08-25T09:46:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.917+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc800a850 con 0x7ffbd015ec10 2023-08-25T09:46:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.917+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd015ca20 con 0x7ffbd015ec10 2023-08-25T09:46:06.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.917+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd01f2e70 con 0x7ffbd015ec10 2023-08-25T09:46:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.918+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd01f3670 con 0x7ffbd015ec10 2023-08-25T09:46:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.919+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffbc800a9b0 con 0x7ffbd015ec10 2023-08-25T09:46:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.920+0000 7ffbccff9640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb006a050 0x7ffbb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.920+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffbc8088460 con 0x7ffbd015ec10 2023-08-25T09:46:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.920+0000 7ffbceffd640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb006a050 0x7ffbb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.921+0000 7ffbceffd640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb006a050 0x7ffbb006c510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0036000 tx=0x7ffbc003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:06.924+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffbd01f3670 con 0x7ffbd015ec10 2023-08-25T09:46:07.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.152+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbd01f3670 con 0x7ffbd015ec10 2023-08-25T09:46:07.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.153+0000 7ffbccff9640 1 -- 172.21.15.70:0/3800454632 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ffbd01f3670 con 0x7ffbd015ec10 2023-08-25T09:46:07.155 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:07.155 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5132,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/2867157748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":2867157748},{"type":"v1","addr":"172.21.15.155:6835","nonce":2867157748}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/2532140862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2532140862},{"type":"v1","addr":"172.21.15.155:6833","nonce":2532140862}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7224,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/333385773","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":333385773},{"type":"v1","addr":"172.21.15.70:6836","nonce":333385773}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44},{"gid":7662,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/166644152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":166644152},{"type":"v1","addr":"172.21.15.70:6837","nonce":166644152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[]} 2023-08-25T09:46:07.156 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 51 2023-08-25T09:46:07.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb006a050 msgr2=0x7ffbb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:07.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb006a050 0x7ffbb006c510 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0036000 tx=0x7ffbc003b040 comp rx=0 tx=0).stop 2023-08-25T09:46:07.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 msgr2=0x7ffbd00aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:07.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 0x7ffbd00aca30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c780 tx=0x7ffbc800cc40 comp rx=0 tx=0).stop 2023-08-25T09:46:07.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 shutdown_connections 2023-08-25T09:46:07.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffbb006a050 0x7ffbb006c510 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffbd015ec10 0x7ffbd00aca30 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffbd015e830 0x7ffbd015dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 --2- 172.21.15.70:0/3800454632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffbd006b2b0 0x7ffbd015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 >> 172.21.15.70:0/3800454632 conn(0x7ffbd0069850 msgr2=0x7ffbd015b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:07.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 shutdown_connections 2023-08-25T09:46:07.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.156+0000 7ffbd5a3f640 1 -- 172.21.15.70:0/3800454632 wait complete. 2023-08-25T09:46:07.176 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:46:07.176 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:46:07.177 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:46:07.177 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:07.177 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:46:07.177 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:46:07.178 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:46:07.178 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:07.178 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:46:07.178 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:46:07.179 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:46:07.179 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:07.179 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:46:07.179 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:46:07.180 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:46:07.180 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:07.193 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:46:07.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:46:07.198 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:46:07.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:46:07.202 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:46:07.202 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:46:07.208 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:46:07.208 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:46:07.246 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:46:07.247 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:07.255 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:46:07.255 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:07.256 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:46:07.256 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:07.264 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:46:07.264 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:07.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:46:07.409 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:46:07.410 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:46:07.423 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:46:07.425 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:46:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.453+0000 7fab0ab0c640 1 Processor -- start 2023-08-25T09:46:07.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.453+0000 7fab0ab0c640 1 -- start start 2023-08-25T09:46:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab0ab0c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fab04159aa0 0x7fab04070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab0ab0c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab0406bc80 0x7fab04070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab0ab0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0406aec0 con 0x7fab0415df10 2023-08-25T09:46:07.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab0ab0c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab04159e80 con 0x7fab0406bc80 2023-08-25T09:46:07.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab0ab0c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0406b1a0 con 0x7fab04159aa0 2023-08-25T09:46:07.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab0406bc80 0x7fab04070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:07.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab0406bc80 0x7fab04070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37132/0 (socket says 172.21.15.70:37132) 2023-08-25T09:46:07.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 -- 172.21.15.70:0/1196884823 learned_addr learned my addr 172.21.15.70:0/1196884823 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 -- 172.21.15.70:0/1196884823 >> v2:172.21.15.155:3300/0 conn(0x7fab04159aa0 msgr2=0x7fab04070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 --2- 172.21.15.70:0/1196884823 >> v2:172.21.15.155:3300/0 conn(0x7fab04159aa0 0x7fab04070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 -- 172.21.15.70:0/1196884823 >> v1:172.21.15.155:6789/0 conn(0x7fab0415df10 legacy=0x7fab040c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.455+0000 7fab03fff640 1 -- 172.21.15.70:0/1196884823 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0406b480 con 0x7fab0406bc80 2023-08-25T09:46:07.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.456+0000 7fab03fff640 1 --2- 172.21.15.70:0/1196884823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0406bc80 0x7fab04070950 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fab0406e6e0 tx=0x7faaf002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ba3672c9b2a535 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:07.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab037fe640 1 -- 172.21.15.70:0/1196884823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0031870 con 0x7fab0406bc80 2023-08-25T09:46:07.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab037fe640 1 -- 172.21.15.70:0/1196884823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf00319d0 con 0x7fab0406bc80 2023-08-25T09:46:07.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab037fe640 1 -- 172.21.15.70:0/1196884823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf0031d60 con 0x7fab0406bc80 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1196884823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0406bc80 msgr2=0x7fab04070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1196884823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0406bc80 0x7fab04070950 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fab0406e6e0 tx=0x7faaf002b910 comp rx=0 tx=0).stop 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1196884823 shutdown_connections 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1196884823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab0406bc80 0x7fab04070950 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1196884823 >> v2:172.21.15.155:3300/0 conn(0x7fab04159aa0 0x7fab04070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1196884823 >> 172.21.15.70:0/1196884823 conn(0x7fab040ce760 msgr2=0x7fab040ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:07.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1196884823 shutdown_connections 2023-08-25T09:46:07.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.457+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1196884823 wait complete. 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.459+0000 7fab0ab0c640 1 Processor -- start 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.459+0000 7fab0ab0c640 1 -- start start 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab0ab0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab04159aa0 0x7fab04157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab0ab0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 0x7fab04147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab0ab0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab04148400 0x7fab040c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab0ab0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04157cd0 con 0x7fab04158d70 2023-08-25T09:46:07.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab0ab0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab04157fb0 con 0x7fab04159aa0 2023-08-25T09:46:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab0ab0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab04158290 con 0x7fab04148400 2023-08-25T09:46:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 0x7fab04147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab03fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 0x7fab04147db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37148/0 (socket says 172.21.15.70:37148) 2023-08-25T09:46:07.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.460+0000 7fab03fff640 1 -- 172.21.15.70:0/1555398895 learned_addr learned my addr 172.21.15.70:0/1555398895 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab090a9640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab04148400 0x7fab040c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:07.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab088a8640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab04159aa0 0x7fab04157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab03fff640 1 -- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab04148400 msgr2=0x7fab040c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab03fff640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab04148400 0x7fab040c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab03fff640 1 -- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab04159aa0 msgr2=0x7fab04157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab03fff640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab04159aa0 0x7fab04157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:07.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab03fff640 1 -- 172.21.15.70:0/1555398895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf00094d0 con 0x7fab04158d70 2023-08-25T09:46:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab090a9640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab04148400 0x7fab040c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab088a8640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab04159aa0 0x7fab04157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.461+0000 7fab03fff640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 0x7fab04147db0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7faaf0000c00 tx=0x7faaf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.462+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf003ad30 con 0x7fab04158d70 2023-08-25T09:46:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.462+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf00443f0 con 0x7fab04158d70 2023-08-25T09:46:07.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.462+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab04158570 con 0x7fab04158d70 2023-08-25T09:46:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.462+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf00447a0 con 0x7fab04158d70 2023-08-25T09:46:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.462+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0415b7a0 con 0x7fab04158d70 2023-08-25T09:46:07.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.463+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0404e420 con 0x7fab04158d70 2023-08-25T09:46:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.464+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faaf004d450 con 0x7fab04158d70 2023-08-25T09:46:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.465+0000 7fab017fa640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faae806a080 0x7faae806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.465+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5296+0+0 (secure 0 0 0) 0x7faaf002d040 con 0x7fab04158d70 2023-08-25T09:46:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.466+0000 7fab088a8640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faae806a080 0x7faae806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:07.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.466+0000 7fab088a8640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faae806a080 0x7faae806c540 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faaf800a850 tx=0x7faaf8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:07.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.470+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faaf0076310 con 0x7fab04158d70 2023-08-25T09:46:07.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:07.669+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fab040cf670 con 0x7fab04158d70 2023-08-25T09:46:08.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.778+0000 7fab017fa640 1 -- 172.21.15.70:0/1555398895 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v129) v1 ==== 97+0+0 (secure 0 0 0) 0x7faaf0079930 con 0x7fab04158d70 2023-08-25T09:46:08.780 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:46:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faae806a080 msgr2=0x7faae806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faae806a080 0x7faae806c540 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faaf800a850 tx=0x7faaf8008040 comp rx=0 tx=0).stop 2023-08-25T09:46:08.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 msgr2=0x7fab04147db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 0x7fab04147db0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7faaf0000c00 tx=0x7faaf00046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:08.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 shutdown_connections 2023-08-25T09:46:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faae806a080 0x7faae806c540 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab04148400 0x7fab040c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab04158d70 0x7fab04147db0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 --2- 172.21.15.70:0/1555398895 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab04159aa0 0x7fab04157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:08.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 >> 172.21.15.70:0/1555398895 conn(0x7fab040ce760 msgr2=0x7fab040b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:08.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 shutdown_connections 2023-08-25T09:46:08.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:08.782+0000 7fab0ab0c640 1 -- 172.21.15.70:0/1555398895 wait complete. 2023-08-25T09:46:08.804 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:46:09.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.008+0000 7f7bcdcfa640 1 Processor -- start 2023-08-25T09:46:09.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.008+0000 7f7bcdcfa640 1 -- start start 2023-08-25T09:46:09.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.010+0000 7f7bcdcfa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815ec10 0x7f7bc815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.010+0000 7f7bcdcfa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bc815c6d0 0x7f7bc80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.010+0000 7f7bcdcfa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc80b1b60 con 0x7f7bc815e830 2023-08-25T09:46:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.010+0000 7f7bcdcfa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc806bd80 con 0x7f7bc815c6d0 2023-08-25T09:46:09.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.010+0000 7f7bcdcfa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc806b7c0 con 0x7f7bc815ec10 2023-08-25T09:46:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.010+0000 7f7bc7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bc815c6d0 0x7f7bc80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7bc815c6d0 0x7f7bc80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51142/0 (socket says 172.21.15.70:51142) 2023-08-25T09:46:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc7fff640 1 -- 172.21.15.70:0/1589355595 learned_addr learned my addr 172.21.15.70:0/1589355595 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bbffff640 1 --2- 172.21.15.70:0/1589355595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815ec10 0x7f7bc815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc7fff640 1 -- 172.21.15.70:0/1589355595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815ec10 msgr2=0x7f7bc815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc7fff640 1 --2- 172.21.15.70:0/1589355595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815ec10 0x7f7bc815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc7fff640 1 -- 172.21.15.70:0/1589355595 >> v1:172.21.15.70:6789/0 conn(0x7f7bc815e830 legacy=0x7f7bc806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc7fff640 1 -- 172.21.15.70:0/1589355595 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc806baa0 con 0x7f7bc815c6d0 2023-08-25T09:46:09.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.011+0000 7f7bc6ffd640 1 -- 172.21.15.70:0/1589355595 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539946995 0 0) 0x7f7bc80b1b60 con 0x7f7bc815e830 2023-08-25T09:46:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.012+0000 7f7bc7fff640 1 --2- 172.21.15.70:0/1589355595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815c6d0 0x7f7bc80b1ff0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009e50 tx=0x7f7bc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8305164479dfcf3d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.012+0000 7f7bc6ffd640 1 -- 172.21.15.70:0/1589355595 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc002ca70 con 0x7f7bc815c6d0 2023-08-25T09:46:09.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.012+0000 7f7bc6ffd640 1 -- 172.21.15.70:0/1589355595 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc002cbd0 con 0x7f7bc815c6d0 2023-08-25T09:46:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.012+0000 7f7bc6ffd640 1 -- 172.21.15.70:0/1589355595 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc00343f0 con 0x7f7bc815c6d0 2023-08-25T09:46:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/1589355595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815c6d0 msgr2=0x7f7bc80b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/1589355595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815c6d0 0x7f7bc80b1ff0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009e50 tx=0x7f7bc002bed0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/1589355595 shutdown_connections 2023-08-25T09:46:09.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/1589355595 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815c6d0 0x7f7bc80b1ff0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/1589355595 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815ec10 0x7f7bc815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/1589355595 >> 172.21.15.70:0/1589355595 conn(0x7f7bc8069850 msgr2=0x7f7bc8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:09.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/1589355595 shutdown_connections 2023-08-25T09:46:09.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.013+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/1589355595 wait complete. 2023-08-25T09:46:09.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.015+0000 7f7bcdcfa640 1 Processor -- start 2023-08-25T09:46:09.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.015+0000 7f7bcdcfa640 1 -- start start 2023-08-25T09:46:09.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bcdcfa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815c6d0 0x7f7bc8154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bcdcfa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bc815e830 0x7f7bc8159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bcdcfa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 0x7f7bc80aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bcdcfa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8155500 con 0x7f7bc815ec10 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bcdcfa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8159dc0 con 0x7f7bc815e830 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bcdcfa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc806fb40 con 0x7f7bc815c6d0 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 0x7f7bc80aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 0x7f7bc80aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51146/0 (socket says 172.21.15.70:51146) 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 -- 172.21.15.70:0/4002864922 learned_addr learned my addr 172.21.15.70:0/4002864922 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 -- 172.21.15.70:0/4002864922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815c6d0 msgr2=0x7f7bc8154fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815c6d0 0x7f7bc8154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 -- 172.21.15.70:0/4002864922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bc815e830 msgr2=0x7f7bc8159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bc815e830 0x7f7bc8159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.016+0000 7f7bc7fff640 1 -- 172.21.15.70:0/4002864922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc0009b30 con 0x7f7bc815ec10 2023-08-25T09:46:09.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.017+0000 7f7bc7fff640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 0x7f7bc80aea50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0036040 tx=0x7f7bc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.017+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc003e450 con 0x7f7bc815ec10 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.017+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc003e5d0 con 0x7f7bc815ec10 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.017+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc003e780 con 0x7f7bc815ec10 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.017+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc806fe20 con 0x7f7bc815ec10 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.017+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc80703e0 con 0x7f7bc815ec10 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.018+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc804e420 con 0x7f7bc815ec10 2023-08-25T09:46:09.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.019+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7bc003e8e0 con 0x7f7bc815ec10 2023-08-25T09:46:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.020+0000 7f7bc4ff9640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ba806a050 0x7f7ba806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.020+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7bc00aae70 con 0x7f7bc815ec10 2023-08-25T09:46:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.020+0000 7f7bc77fe640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ba806a050 0x7f7ba806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.021+0000 7f7bc77fe640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ba806a050 0x7f7ba806c510 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8001600 tx=0x7f7bb8007910 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.024+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7bc0076810 con 0x7f7bc815ec10 2023-08-25T09:46:09.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.278+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7bc80ac870 con 0x7f7bc815ec10 2023-08-25T09:46:09.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.279+0000 7f7bc4ff9640 1 -- 172.21.15.70:0/4002864922 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7bc0033e80 con 0x7f7bc815ec10 2023-08-25T09:46:09.280 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:09.280 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:09.281 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:46:09.282 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:09.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:09.283 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.284 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:09.285 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:09.285 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:09.286 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:46:09.288 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.289 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.291 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:46:09.291 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:46:09.291 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.291 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.293 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.293 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.293 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:46:09.294 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:46:09.294 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.294 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.294 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.294 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.295 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:46:09.295 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:46:09.295 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.296 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:09.297 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.298 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:46:09.299 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:46:09.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.299 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:46:09.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:46:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:46:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:46:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:46:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:46:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:46:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:46:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:46:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:46:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:46:09.309 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:09.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ba806a050 msgr2=0x7f7ba806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ba806a050 0x7f7ba806c510 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8001600 tx=0x7f7bb8007910 comp rx=0 tx=0).stop 2023-08-25T09:46:09.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 msgr2=0x7f7bc80aea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 0x7f7bc80aea50 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0036040 tx=0x7f7bc0004a40 comp rx=0 tx=0).stop 2023-08-25T09:46:09.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 shutdown_connections 2023-08-25T09:46:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7ba806a050 0x7f7ba806c510 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bc815ec10 0x7f7bc80aea50 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bc815e830 0x7f7bc8159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 --2- 172.21.15.70:0/4002864922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bc815c6d0 0x7f7bc8154fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 >> 172.21.15.70:0/4002864922 conn(0x7f7bc8069850 msgr2=0x7f7bc815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 shutdown_connections 2023-08-25T09:46:09.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.281+0000 7f7bcdcfa640 1 -- 172.21.15.70:0/4002864922 wait complete. 2023-08-25T09:46:09.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.534+0000 7fb2e8b96640 1 Processor -- start 2023-08-25T09:46:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.534+0000 7fb2e8b96640 1 -- start start 2023-08-25T09:46:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e8b96640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb2e415e830 0x7fb2e406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e8b96640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40b1b60 con 0x7fb2e415ec10 2023-08-25T09:46:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e8b96640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e406b7c0 con 0x7fb2e406b2b0 2023-08-25T09:46:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e8b96640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e406bd80 con 0x7fb2e415e830 2023-08-25T09:46:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e2d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb2e415ec10 0x7fb2e40b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59276/0 (socket says 172.21.15.70:59276) 2023-08-25T09:46:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e2d9d640 1 -- 172.21.15.70:0/2393518907 learned_addr learned my addr 172.21.15.70:0/2393518907 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.536+0000 7fb2e259c640 1 --2- 172.21.15.70:0/2393518907 >> v2:172.21.15.70:3300/0 conn(0x7fb2e415e830 0x7fb2e406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.537+0000 7fb2e1d9b640 1 -- 172.21.15.70:0/2393518907 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280882169 0 0) 0x7fb2e40b1b60 con 0x7fb2e415ec10 2023-08-25T09:46:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.537+0000 7fb2e259c640 1 -- 172.21.15.70:0/2393518907 >> v1:172.21.15.155:6789/0 conn(0x7fb2e406b2b0 legacy=0x7fb2e415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.537+0000 7fb2e259c640 1 -- 172.21.15.70:0/2393518907 >> v1:172.21.15.70:6789/0 conn(0x7fb2e415ec10 legacy=0x7fb2e40b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.537+0000 7fb2e259c640 1 -- 172.21.15.70:0/2393518907 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e406baa0 con 0x7fb2e415e830 2023-08-25T09:46:09.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.538+0000 7fb2e259c640 1 --2- 172.21.15.70:0/2393518907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e415e830 0x7fb2e406ad70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40097a0 tx=0x7fb2d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ef900843f670af server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.538+0000 7fb2e1d9b640 1 -- 172.21.15.70:0/2393518907 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4031830 con 0x7fb2e415e830 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.538+0000 7fb2e1d9b640 1 -- 172.21.15.70:0/2393518907 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d4031990 con 0x7fb2e415e830 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.538+0000 7fb2e1d9b640 1 -- 172.21.15.70:0/2393518907 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4031d20 con 0x7fb2e415e830 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 -- 172.21.15.70:0/2393518907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e415e830 msgr2=0x7fb2e406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/2393518907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e415e830 0x7fb2e406ad70 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40097a0 tx=0x7fb2d402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 -- 172.21.15.70:0/2393518907 shutdown_connections 2023-08-25T09:46:09.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/2393518907 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e415e830 0x7fb2e406ad70 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 -- 172.21.15.70:0/2393518907 >> 172.21.15.70:0/2393518907 conn(0x7fb2e4069850 msgr2=0x7fb2e4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:09.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 -- 172.21.15.70:0/2393518907 shutdown_connections 2023-08-25T09:46:09.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.539+0000 7fb2e8b96640 1 -- 172.21.15.70:0/2393518907 wait complete. 2023-08-25T09:46:09.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.541+0000 7fb2e8b96640 1 Processor -- start 2023-08-25T09:46:09.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.541+0000 7fb2e8b96640 1 -- start start 2023-08-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e8b96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 0x7fb2e40c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e8b96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e415e830 0x7fb2e40b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e8b96640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e415ec10 0x7fb2e40b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e8b96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c97d0 con 0x7fb2e406b2b0 2023-08-25T09:46:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e8b96640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c9aa0 con 0x7fb2e415ec10 2023-08-25T09:46:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e8b96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40c9d80 con 0x7fb2e415e830 2023-08-25T09:46:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 0x7fb2e40c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 0x7fb2e40c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51166/0 (socket says 172.21.15.70:51166) 2023-08-25T09:46:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 -- 172.21.15.70:0/842732994 learned_addr learned my addr 172.21.15.70:0/842732994 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e2d9d640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e415ec10 0x7fb2e40b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2dbfff640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e415e830 0x7fb2e40b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 -- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e415e830 msgr2=0x7fb2e40b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e415e830 0x7fb2e40b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 -- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e415ec10 msgr2=0x7fb2e40b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e415ec10 0x7fb2e40b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e259c640 1 -- 172.21.15.70:0/842732994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4009400 con 0x7fb2e406b2b0 2023-08-25T09:46:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.542+0000 7fb2e2d9d640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e415ec10 0x7fb2e40b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:09.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.543+0000 7fb2e259c640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 0x7fb2e40c9290 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40097a0 tx=0x7fb2d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.543+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d403ca80 con 0x7fb2e406b2b0 2023-08-25T09:46:09.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.543+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d403cbe0 con 0x7fb2e406b2b0 2023-08-25T09:46:09.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.543+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4004030 con 0x7fb2e406b2b0 2023-08-25T09:46:09.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.543+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e415ca20 con 0x7fb2e406b2b0 2023-08-25T09:46:09.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.543+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40ca340 con 0x7fb2e406b2b0 2023-08-25T09:46:09.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.544+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e404e420 con 0x7fb2e406b2b0 2023-08-25T09:46:09.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.545+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb2d4004190 con 0x7fb2e406b2b0 2023-08-25T09:46:09.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.546+0000 7fb2daffd640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb2c406a050 0x7fb2c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.546+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb2d40aaa30 con 0x7fb2e406b2b0 2023-08-25T09:46:09.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.546+0000 7fb2dbfff640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb2c406a050 0x7fb2c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.547+0000 7fb2dbfff640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb2c406a050 0x7fb2c406c510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb2e406c550 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.550+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb2d4076460 con 0x7fb2e406b2b0 2023-08-25T09:46:09.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.739+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2e404e420 con 0x7fb2e406b2b0 2023-08-25T09:46:09.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.741+0000 7fb2daffd640 1 -- 172.21.15.70:0/842732994 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+14665 (secure 0 0 0) 0x7fb2d4079a80 con 0x7fb2e406b2b0 2023-08-25T09:46:09.742 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:09.743 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":129,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:08.771507+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":113,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:09.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.743+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb2c406a050 msgr2=0x7fb2c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb2c406a050 0x7fb2c406c510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb2e406c550 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-08-25T09:46:09.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 msgr2=0x7fb2e40c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 0x7fb2e40c9290 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40097a0 tx=0x7fb2d40046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 shutdown_connections 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb2c406a050 0x7fb2c406c510 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb2e415ec10 0x7fb2e40b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb2e415e830 0x7fb2e40b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 --2- 172.21.15.70:0/842732994 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb2e406b2b0 0x7fb2e40c9290 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 >> 172.21.15.70:0/842732994 conn(0x7fb2e4069850 msgr2=0x7fb2e4070230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:09.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 shutdown_connections 2023-08-25T09:46:09.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.744+0000 7fb2e8b96640 1 -- 172.21.15.70:0/842732994 wait complete. 2023-08-25T09:46:09.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:09.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.976+0000 7fa28c204640 1 Processor -- start 2023-08-25T09:46:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.976+0000 7fa28c204640 1 -- start start 2023-08-25T09:46:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.977+0000 7fa28c204640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2840c2110 0x7fa284069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.978+0000 7fa28c204640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840c2550 con 0x7fa28415ae50 2023-08-25T09:46:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.978+0000 7fa28c204640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2840ce330 con 0x7fa284069d90 2023-08-25T09:46:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.978+0000 7fa28c204640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28406a200 con 0x7fa2840c2110 2023-08-25T09:46:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.978+0000 7fa289fa0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa28415ae50 0x7fa2840cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59280/0 (socket says 172.21.15.70:59280) 2023-08-25T09:46:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.978+0000 7fa28979f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa2840c2110 0x7fa284069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.978+0000 7fa289fa0640 1 -- 172.21.15.70:0/245191839 learned_addr learned my addr 172.21.15.70:0/245191839 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.979+0000 7fa28979f640 1 -- 172.21.15.70:0/245191839 >> v1:172.21.15.155:6789/0 conn(0x7fa284069d90 legacy=0x7fa2840ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.979+0000 7fa28979f640 1 -- 172.21.15.70:0/245191839 >> v1:172.21.15.70:6789/0 conn(0x7fa28415ae50 legacy=0x7fa2840cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.979+0000 7fa28979f640 1 -- 172.21.15.70:0/245191839 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2840cc5f0 con 0x7fa2840c2110 2023-08-25T09:46:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.979+0000 7fa288f9e640 1 -- 172.21.15.70:0/245191839 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826233137 0 0) 0x7fa2840c2550 con 0x7fa28415ae50 2023-08-25T09:46:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.980+0000 7fa28979f640 1 --2- 172.21.15.70:0/245191839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2840c2110 0x7fa284069850 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa2740097a0 tx=0x7fa27402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166c1b954d5dd25e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.980+0000 7fa288f9e640 1 -- 172.21.15.70:0/245191839 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274030610 con 0x7fa2840c2110 2023-08-25T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.980+0000 7fa288f9e640 1 -- 172.21.15.70:0/245191839 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa274030770 con 0x7fa2840c2110 2023-08-25T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.980+0000 7fa288f9e640 1 -- 172.21.15.70:0/245191839 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274030b00 con 0x7fa2840c2110 2023-08-25T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 -- 172.21.15.70:0/245191839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2840c2110 msgr2=0x7fa284069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 --2- 172.21.15.70:0/245191839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2840c2110 0x7fa284069850 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa2740097a0 tx=0x7fa27402b960 comp rx=0 tx=0).stop 2023-08-25T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 -- 172.21.15.70:0/245191839 shutdown_connections 2023-08-25T09:46:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 --2- 172.21.15.70:0/245191839 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa2840c2110 0x7fa284069850 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 -- 172.21.15.70:0/245191839 >> 172.21.15.70:0/245191839 conn(0x7fa2840cb0d0 msgr2=0x7fa2840cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 -- 172.21.15.70:0/245191839 shutdown_connections 2023-08-25T09:46:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.981+0000 7fa28c204640 1 -- 172.21.15.70:0/245191839 wait complete. 2023-08-25T09:46:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.982+0000 7fa28c204640 1 Processor -- start 2023-08-25T09:46:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.983+0000 7fa28c204640 1 -- start start 2023-08-25T09:46:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28c204640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa284069d90 0x7fa28414ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28c204640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2840c2110 0x7fa2840ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28c204640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 0x7fa2840acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28c204640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa284158380 con 0x7fa28415ae50 2023-08-25T09:46:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28c204640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840ad420 con 0x7fa2840c2110 2023-08-25T09:46:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28c204640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840b36c0 con 0x7fa284069d90 2023-08-25T09:46:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 0x7fa2840acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 0x7fa2840acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51176/0 (socket says 172.21.15.70:51176) 2023-08-25T09:46:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 -- 172.21.15.70:0/1943760976 learned_addr learned my addr 172.21.15.70:0/1943760976 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28979f640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2840c2110 0x7fa2840ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 -- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa284069d90 msgr2=0x7fa28414ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa284069d90 0x7fa28414ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 -- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2840c2110 msgr2=0x7fa2840ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2840c2110 0x7fa2840ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.984+0000 7fa28a7a1640 1 -- 172.21.15.70:0/1943760976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa274009400 con 0x7fa28415ae50 2023-08-25T09:46:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.985+0000 7fa28a7a1640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 0x7fa2840acdc0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c780 tx=0x7fa27c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.985+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c004b60 con 0x7fa28415ae50 2023-08-25T09:46:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.985+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00ce90 con 0x7fa28415ae50 2023-08-25T09:46:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.985+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00a850 con 0x7fa28415ae50 2023-08-25T09:46:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.985+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2840b39a0 con 0x7fa28415ae50 2023-08-25T09:46:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.985+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28414d340 con 0x7fa28415ae50 2023-08-25T09:46:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.986+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2840b3ec0 con 0x7fa28415ae50 2023-08-25T09:46:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.987+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa27c00a9b0 con 0x7fa28415ae50 2023-08-25T09:46:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.988+0000 7fa27a7fc640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa26806a050 0x7fa26806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.988+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa27c088410 con 0x7fa28415ae50 2023-08-25T09:46:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.989+0000 7fa289fa0640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa26806a050 0x7fa26806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.989+0000 7fa289fa0640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa26806a050 0x7fa26806c510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa280007fd0 tx=0x7fa280007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:09.993+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa2840b3ec0 con 0x7fa28415ae50 2023-08-25T09:46:10.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.179+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2840b3ec0 con 0x7fa28415ae50 2023-08-25T09:46:10.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.180+0000 7fa27a7fc640 1 -- 172.21.15.70:0/1943760976 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2840b3ec0 con 0x7fa28415ae50 2023-08-25T09:46:10.182 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.183+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa26806a050 msgr2=0x7fa26806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.183+0000 7fa28c204640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa26806a050 0x7fa26806c510 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa280007fd0 tx=0x7fa280007960 comp rx=0 tx=0).stop 2023-08-25T09:46:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.183+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 msgr2=0x7fa2840acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.183+0000 7fa28c204640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 0x7fa2840acdc0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00c780 tx=0x7fa27c00cc40 comp rx=0 tx=0).stop 2023-08-25T09:46:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 shutdown_connections 2023-08-25T09:46:10.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa26806a050 0x7fa26806c510 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa28415ae50 0x7fa2840acdc0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa2840c2110 0x7fa2840ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 --2- 172.21.15.70:0/1943760976 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa284069d90 0x7fa28414ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 >> 172.21.15.70:0/1943760976 conn(0x7fa2840cb0d0 msgr2=0x7fa2840ae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:10.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 shutdown_connections 2023-08-25T09:46:10.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.184+0000 7fa28c204640 1 -- 172.21.15.70:0/1943760976 wait complete. 2023-08-25T09:46:10.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:10.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.406+0000 7fc873e0d640 1 Processor -- start 2023-08-25T09:46:10.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.406+0000 7fc873e0d640 1 -- start start 2023-08-25T09:46:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc873e0d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc86c069850 0x7fc86c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc873e0d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c0c9620 0x7fc86c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc873e0d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0ac880 con 0x7fc86c160be0 2023-08-25T09:46:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc873e0d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c0ad400 con 0x7fc86c069850 2023-08-25T09:46:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc873e0d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0ad890 con 0x7fc86c0c9620 2023-08-25T09:46:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc871ba9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc86c069850 0x7fc86c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc8723aa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc86c160be0 0x7fc86c161040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59294/0 (socket says 172.21.15.70:59294) 2023-08-25T09:46:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc8723aa640 1 -- 172.21.15.70:0/418306755 learned_addr learned my addr 172.21.15.70:0/418306755 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.408+0000 7fc8713a8640 1 --2- 172.21.15.70:0/418306755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c0c9620 0x7fc86c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.409+0000 7fc871ba9640 1 -- 172.21.15.70:0/418306755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c0c9620 msgr2=0x7fc86c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.409+0000 7fc871ba9640 1 --2- 172.21.15.70:0/418306755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c0c9620 0x7fc86c069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.409+0000 7fc871ba9640 1 -- 172.21.15.70:0/418306755 >> v1:172.21.15.70:6789/0 conn(0x7fc86c160be0 legacy=0x7fc86c161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.409+0000 7fc871ba9640 1 -- 172.21.15.70:0/418306755 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c0acad0 con 0x7fc86c069850 2023-08-25T09:46:10.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.409+0000 7fc870ba7640 1 -- 172.21.15.70:0/418306755 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133301616 0 0) 0x7fc86c0ac880 con 0x7fc86c160be0 2023-08-25T09:46:10.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.410+0000 7fc871ba9640 1 --2- 172.21.15.70:0/418306755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c069850 0x7fc86c15e210 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc864009d90 tx=0x7fc86402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfb22064d01dbd8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.410+0000 7fc870ba7640 1 -- 172.21.15.70:0/418306755 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86402cce0 con 0x7fc86c069850 2023-08-25T09:46:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.410+0000 7fc870ba7640 1 -- 172.21.15.70:0/418306755 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86402ce40 con 0x7fc86c069850 2023-08-25T09:46:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.410+0000 7fc870ba7640 1 -- 172.21.15.70:0/418306755 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864034660 con 0x7fc86c069850 2023-08-25T09:46:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 -- 172.21.15.70:0/418306755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c069850 msgr2=0x7fc86c15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 --2- 172.21.15.70:0/418306755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c069850 0x7fc86c15e210 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fc864009d90 tx=0x7fc86402bf30 comp rx=0 tx=0).stop 2023-08-25T09:46:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 -- 172.21.15.70:0/418306755 shutdown_connections 2023-08-25T09:46:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 --2- 172.21.15.70:0/418306755 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c0c9620 0x7fc86c069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 --2- 172.21.15.70:0/418306755 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c069850 0x7fc86c15e210 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 -- 172.21.15.70:0/418306755 >> 172.21.15.70:0/418306755 conn(0x7fc86c0cb0d0 msgr2=0x7fc86c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 -- 172.21.15.70:0/418306755 shutdown_connections 2023-08-25T09:46:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.411+0000 7fc873e0d640 1 -- 172.21.15.70:0/418306755 wait complete. 2023-08-25T09:46:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.413+0000 7fc873e0d640 1 Processor -- start 2023-08-25T09:46:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.413+0000 7fc873e0d640 1 -- start start 2023-08-25T09:46:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc873e0d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c069850 0x7fc86c0ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc873e0d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 0x7fc86c1f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc873e0d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c160be0 0x7fc86c1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc873e0d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0705e0 con 0x7fc86c0c9620 2023-08-25T09:46:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc873e0d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1f2bc0 con 0x7fc86c160be0 2023-08-25T09:46:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc873e0d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c06fa60 con 0x7fc86c069850 2023-08-25T09:46:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc8713a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 0x7fc86c1f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc8713a8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 0x7fc86c1f62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51196/0 (socket says 172.21.15.70:51196) 2023-08-25T09:46:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc8713a8640 1 -- 172.21.15.70:0/1883178208 learned_addr learned my addr 172.21.15.70:0/1883178208 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc871ba9640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c069850 0x7fc86c0ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc8713a8640 1 -- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c069850 msgr2=0x7fc86c0ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.414+0000 7fc8723aa640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c160be0 0x7fc86c1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc8713a8640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c069850 0x7fc86c0ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc8713a8640 1 -- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c160be0 msgr2=0x7fc86c1f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc8713a8640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c160be0 0x7fc86c1f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc8713a8640 1 -- 172.21.15.70:0/1883178208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009a70 con 0x7fc86c0c9620 2023-08-25T09:46:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc8723aa640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c160be0 0x7fc86c1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc8713a8640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 0x7fc86c1f62d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc86000d800 tx=0x7fc86000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000b6c0 con 0x7fc86c0c9620 2023-08-25T09:46:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86000b820 con 0x7fc86c0c9620 2023-08-25T09:46:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000bbb0 con 0x7fc86c0c9620 2023-08-25T09:46:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c06fd40 con 0x7fc86c0c9620 2023-08-25T09:46:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.415+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c070300 con 0x7fc86c0c9620 2023-08-25T09:46:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.416+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c066fb0 con 0x7fc86c0c9620 2023-08-25T09:46:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.417+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc86000bd10 con 0x7fc86c0c9620 2023-08-25T09:46:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.418+0000 7fc86a7fc640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85806a050 0x7fc85806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.418+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc860088870 con 0x7fc86c0c9620 2023-08-25T09:46:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.418+0000 7fc871ba9640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85806a050 0x7fc85806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.419+0000 7fc871ba9640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85806a050 0x7fc85806c510 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc864036000 tx=0x7fc864030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.423+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc860054210 con 0x7fc86c0c9620 2023-08-25T09:46:10.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.620+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc86c0c2370 con 0x7fc86c0c9620 2023-08-25T09:46:10.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.620+0000 7fc86a7fc640 1 -- 172.21.15.70:0/1883178208 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc860057830 con 0x7fc86c0c9620 2023-08-25T09:46:10.622 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:10.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85806a050 msgr2=0x7fc85806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85806a050 0x7fc85806c510 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc864036000 tx=0x7fc864030000 comp rx=0 tx=0).stop 2023-08-25T09:46:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 msgr2=0x7fc86c1f62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 0x7fc86c1f62d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fc86000d800 tx=0x7fc86000dcc0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 shutdown_connections 2023-08-25T09:46:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc85806a050 0x7fc85806c510 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc86c160be0 0x7fc86c1f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc86c0c9620 0x7fc86c1f62d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 --2- 172.21.15.70:0/1883178208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc86c069850 0x7fc86c0ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 >> 172.21.15.70:0/1883178208 conn(0x7fc86c0cb0d0 msgr2=0x7fc86c06ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:10.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 shutdown_connections 2023-08-25T09:46:10.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.623+0000 7fc873e0d640 1 -- 172.21.15.70:0/1883178208 wait complete. 2023-08-25T09:46:10.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:46:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.840+0000 7fc98084c640 1 Processor -- start 2023-08-25T09:46:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.840+0000 7fc98084c640 1 -- start start 2023-08-25T09:46:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.841+0000 7fc98084c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 0x7fc97c0d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.841+0000 7fc98084c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc97c0cf9a0 0x7fc97c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.841+0000 7fc98084c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c0d11d0 con 0x7fc97c15ce80 2023-08-25T09:46:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.841+0000 7fc98084c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0d1470 con 0x7fc97c0cf9a0 2023-08-25T09:46:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.841+0000 7fc98084c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0d1750 con 0x7fc97c15caa0 2023-08-25T09:46:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc973fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc97c15ce80 0x7fc97c0cf290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59308/0 (socket says 172.21.15.70:59308) 2023-08-25T09:46:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc973fff640 1 -- 172.21.15.70:0/3152771125 learned_addr learned my addr 172.21.15.70:0/3152771125 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:10.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106694308 0 0) 0x7fc97c0d11d0 con 0x7fc97c15ce80 2023-08-25T09:46:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc95c003520 con 0x7fc97c15ce80 2023-08-25T09:46:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97b59e640 1 --2- 172.21.15.70:0/3152771125 >> v2:172.21.15.155:3300/0 conn(0x7fc97c0cf9a0 0x7fc97c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97ad9d640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 0x7fc97c0d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1881500458 0 0) 0x7fc95c003520 con 0x7fc97c15ce80 2023-08-25T09:46:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 msgr2=0x7fc97c0d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 0x7fc97c0d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 >> v2:172.21.15.155:3300/0 conn(0x7fc97c0cf9a0 msgr2=0x7fc97c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 --2- 172.21.15.70:0/3152771125 >> v2:172.21.15.155:3300/0 conn(0x7fc97c0cf9a0 0x7fc97c0b5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0d1a30 con 0x7fc97c15ce80 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.842+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc964001670 con 0x7fc97c15ce80 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 >> v1:172.21.15.70:6789/0 conn(0x7fc97c15ce80 legacy=0x7fc97c0cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 0x7fc95c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc95c003ef0 0x7fc95c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c004d00 0x7fc95c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0d1470 con 0x7fc95c003ef0 2023-08-25T09:46:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0d1750 con 0x7fc95c004d00 2023-08-25T09:46:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95c00b9d0 con 0x7fc97c15caa0 2023-08-25T09:46:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc973fff640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc95c003ef0 0x7fc95c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97ad9d640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 0x7fc95c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc973fff640 1 -- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 msgr2=0x7fc95c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc973fff640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c15caa0 0x7fc95c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc973fff640 1 -- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c004d00 msgr2=0x7fc95c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc97b59e640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c004d00 0x7fc95c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc973fff640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c004d00 0x7fc95c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.843+0000 7fc973fff640 1 -- 172.21.15.70:0/3152771125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c003520 con 0x7fc95c003ef0 2023-08-25T09:46:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.844+0000 7fc973fff640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc95c003ef0 0x7fc95c0047c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc96400c210 tx=0x7fc95c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e93d92c5f11e51 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96402cae0 con 0x7fc95c003ef0 2023-08-25T09:46:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96402cc40 con 0x7fc95c003ef0 2023-08-25T09:46:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc97a59c640 1 -- 172.21.15.70:0/3152771125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964005af0 con 0x7fc95c003ef0 2023-08-25T09:46:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc98084c640 1 -- 172.21.15.70:0/3152771125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc95c003ef0 msgr2=0x7fc95c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc98084c640 1 --2- 172.21.15.70:0/3152771125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc95c003ef0 0x7fc95c0047c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fc96400c210 tx=0x7fc95c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc97b59e640 1 -- 172.21.15.70:0/3152771125 reap_dead start 2023-08-25T09:46:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc98084c640 1 -- 172.21.15.70:0/3152771125 shutdown_connections 2023-08-25T09:46:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc98084c640 1 -- 172.21.15.70:0/3152771125 >> 172.21.15.70:0/3152771125 conn(0x7fc97c06fe70 msgr2=0x7fc97c070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc98084c640 1 -- 172.21.15.70:0/3152771125 shutdown_connections 2023-08-25T09:46:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.845+0000 7fc98084c640 1 -- 172.21.15.70:0/3152771125 wait complete. 2023-08-25T09:46:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.847+0000 7fc98084c640 1 Processor -- start 2023-08-25T09:46:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.847+0000 7fc98084c640 1 -- start start 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc98084c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c003ef0 0x7fc97c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc98084c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 0x7fc97c14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc98084c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c14bb50 0x7fc97c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc98084c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15fbd0 con 0x7fc97c14bf80 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc98084c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15f330 con 0x7fc95c003ef0 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc98084c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15f610 con 0x7fc97c14bb50 2023-08-25T09:46:10.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc973fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 0x7fc97c14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc973fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 0x7fc97c14b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51216/0 (socket says 172.21.15.70:51216) 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.848+0000 7fc973fff640 1 -- 172.21.15.70:0/3008976429 learned_addr learned my addr 172.21.15.70:0/3008976429 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc973fff640 1 -- 172.21.15.70:0/3008976429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c14bb50 msgr2=0x7fc97c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc973fff640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c14bb50 0x7fc97c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc973fff640 1 -- 172.21.15.70:0/3008976429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c003ef0 msgr2=0x7fc97c15eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc973fff640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c003ef0 0x7fc97c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc973fff640 1 -- 172.21.15.70:0/3008976429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c15f8f0 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc973fff640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 0x7fc97c14b610 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc96402c480 tx=0x7fc97c0d1360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964039030 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964004bf0 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96403c040 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c0c68b0 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.849+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c0c9e40 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.850+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c0d1cf0 con 0x7fc97c14bf80 2023-08-25T09:46:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.851+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc964032070 con 0x7fc97c14bf80 2023-08-25T09:46:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.852+0000 7fc9737fe640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96006a050 0x7fc96006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.852+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc9640a96b0 con 0x7fc97c14bf80 2023-08-25T09:46:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.852+0000 7fc97ad9d640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96006a050 0x7fc96006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.853+0000 7fc97ad9d640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96006a050 0x7fc96006c510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006f00 tx=0x7fc96c006e10 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:10.856+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc97c0d1cf0 con 0x7fc97c14bf80 2023-08-25T09:46:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.047+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc97c0d1cf0 con 0x7fc97c14bf80 2023-08-25T09:46:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.048+0000 7fc9737fe640 1 -- 172.21.15.70:0/3008976429 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v129) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc97c0d1cf0 con 0x7fc97c14bf80 2023-08-25T09:46:11.049 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:11.050 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:46:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:46:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:46:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:11.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:11.052 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:46:11.052 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:46:11.052 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:11.052 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:46:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96006a050 msgr2=0x7fc96006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96006a050 0x7fc96006c510 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006f00 tx=0x7fc96c006e10 comp rx=0 tx=0).stop 2023-08-25T09:46:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 msgr2=0x7fc97c14b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 0x7fc97c14b610 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fc96402c480 tx=0x7fc97c0d1360 comp rx=0 tx=0).stop 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 shutdown_connections 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc96006a050 0x7fc96006c510 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc97c14bb50 0x7fc97c0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc97c14bf80 0x7fc97c14b610 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 --2- 172.21.15.70:0/3008976429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc95c003ef0 0x7fc97c15eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 >> 172.21.15.70:0/3008976429 conn(0x7fc97c06fe70 msgr2=0x7fc97c159640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:11.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 shutdown_connections 2023-08-25T09:46:11.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.050+0000 7fc98084c640 1 -- 172.21.15.70:0/3008976429 wait complete. 2023-08-25T09:46:11.072 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:46:11.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:46:11.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.294+0000 7f44aa006640 1 Processor -- start 2023-08-25T09:46:11.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.295+0000 7f44aa006640 1 -- start start 2023-08-25T09:46:11.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.296+0000 7f44aa006640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44a4142740 0x7f44a40cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:11.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.296+0000 7f44aa006640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a406aa70 0x7f44a406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:11.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44aa006640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a415d320 con 0x7f44a4142c30 2023-08-25T09:46:11.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44aa006640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a406b440 con 0x7f44a4142740 2023-08-25T09:46:11.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44aa006640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40ccd40 con 0x7f44a406aa70 2023-08-25T09:46:11.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44a4142740 0x7f44a40cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:11.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f44a4142740 0x7f44a40cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51220/0 (socket says 172.21.15.70:51220) 2023-08-25T09:46:11.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 -- 172.21.15.70:0/2980502741 learned_addr learned my addr 172.21.15.70:0/2980502741 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 -- 172.21.15.70:0/2980502741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a406aa70 msgr2=0x7f44a406af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a3fff640 1 --2- 172.21.15.70:0/2980502741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a406aa70 0x7f44a406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 --2- 172.21.15.70:0/2980502741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a406aa70 0x7f44a406af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 -- 172.21.15.70:0/2980502741 >> v1:172.21.15.155:6789/0 conn(0x7f44a4142c30 legacy=0x7f44a415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.297+0000 7f44a37fe640 1 -- 172.21.15.70:0/2980502741 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40cd020 con 0x7f44a4142740 2023-08-25T09:46:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.298+0000 7f44a37fe640 1 --2- 172.21.15.70:0/2980502741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a4142740 0x7f44a40cc6b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f44940098a0 tx=0x7f449402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544ad97f2af038c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.299+0000 7f44a27fc640 1 -- 172.21.15.70:0/2980502741 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494031640 con 0x7f44a4142740 2023-08-25T09:46:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.299+0000 7f44a27fc640 1 -- 172.21.15.70:0/2980502741 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44940317a0 con 0x7f44a4142740 2023-08-25T09:46:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.299+0000 7f44a27fc640 1 -- 172.21.15.70:0/2980502741 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494031b30 con 0x7f44a4142740 2023-08-25T09:46:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.299+0000 7f44aa006640 1 -- 172.21.15.70:0/2980502741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a4142740 msgr2=0x7f44a40cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.299+0000 7f44aa006640 1 --2- 172.21.15.70:0/2980502741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a4142740 0x7f44a40cc6b0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f44940098a0 tx=0x7f449402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.300+0000 7f44aa006640 1 -- 172.21.15.70:0/2980502741 shutdown_connections 2023-08-25T09:46:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.300+0000 7f44aa006640 1 --2- 172.21.15.70:0/2980502741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a406aa70 0x7f44a406af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.300+0000 7f44aa006640 1 --2- 172.21.15.70:0/2980502741 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a4142740 0x7f44a40cc6b0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.300+0000 7f44aa006640 1 -- 172.21.15.70:0/2980502741 >> 172.21.15.70:0/2980502741 conn(0x7f44a4069c70 msgr2=0x7f44a406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.300+0000 7f44aa006640 1 -- 172.21.15.70:0/2980502741 shutdown_connections 2023-08-25T09:46:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.300+0000 7f44aa006640 1 -- 172.21.15.70:0/2980502741 wait complete. 2023-08-25T09:46:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.301+0000 7f44aa006640 1 Processor -- start 2023-08-25T09:46:11.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.302+0000 7f44aa006640 1 -- start start 2023-08-25T09:46:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.302+0000 7f44aa006640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 0x7f44a4158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44aa006640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a4142740 0x7f44a40b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44aa006640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44a4142c30 0x7f44a40b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44aa006640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41592e0 con 0x7f44a406aa70 2023-08-25T09:46:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44aa006640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40b2c20 con 0x7f44a4142c30 2023-08-25T09:46:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44aa006640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a40ac870 con 0x7f44a4142740 2023-08-25T09:46:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 0x7f44a4158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 0x7f44a4158da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51222/0 (socket says 172.21.15.70:51222) 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 -- 172.21.15.70:0/2873354940 learned_addr learned my addr 172.21.15.70:0/2873354940 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a3fff640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44a4142c30 0x7f44a40b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a2ffd640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a4142740 0x7f44a40b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 -- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a4142740 msgr2=0x7f44a40b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a4142740 0x7f44a40b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 -- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44a4142c30 msgr2=0x7f44a40b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44a4142c30 0x7f44a40b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a37fe640 1 -- 172.21.15.70:0/2873354940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44940094d0 con 0x7f44a406aa70 2023-08-25T09:46:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.303+0000 7f44a3fff640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44a4142c30 0x7f44a40b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.304+0000 7f44a37fe640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 0x7f44a4158da0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f44940098a0 tx=0x7f44940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.304+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449403ac60 con 0x7f44a406aa70 2023-08-25T09:46:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.304+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449403ade0 con 0x7f44a406aa70 2023-08-25T09:46:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.304+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494044630 con 0x7f44a406aa70 2023-08-25T09:46:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.304+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a40ad6d0 con 0x7f44a406aa70 2023-08-25T09:46:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.304+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a40acda0 con 0x7f44a406aa70 2023-08-25T09:46:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.305+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a404e420 con 0x7f44a406aa70 2023-08-25T09:46:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.306+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f449402f050 con 0x7f44a406aa70 2023-08-25T09:46:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.307+0000 7f448bfff640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f449006a050 0x7f449006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.307+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f44940aa3d0 con 0x7f44a406aa70 2023-08-25T09:46:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.307+0000 7f44a2ffd640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f449006a050 0x7f449006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.308+0000 7f44a2ffd640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f449006a050 0x7f449006c510 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f44a40b38a0 tx=0x7f4498009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:11.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.311+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4494075e00 con 0x7f44a406aa70 2023-08-25T09:46:11.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.500+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f44a40b3910 con 0x7f44a406aa70 2023-08-25T09:46:11.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.808+0000 7f448bfff640 1 -- 172.21.15.70:0/2873354940 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v130) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4494079420 con 0x7f44a406aa70 2023-08-25T09:46:11.810 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:46:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.814+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f449006a050 msgr2=0x7f449006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.814+0000 7f44aa006640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f449006a050 0x7f449006c510 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f44a40b38a0 tx=0x7f4498009040 comp rx=0 tx=0).stop 2023-08-25T09:46:11.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.814+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 msgr2=0x7f44a4158da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:11.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.814+0000 7f44aa006640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 0x7f44a4158da0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f44940098a0 tx=0x7f44940046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.816+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 shutdown_connections 2023-08-25T09:46:11.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.816+0000 7f44aa006640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f449006a050 0x7f449006c510 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.816+0000 7f44aa006640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f44a4142c30 0x7f44a40b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.816+0000 7f44aa006640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f44a4142740 0x7f44a40b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.816+0000 7f44aa006640 1 --2- 172.21.15.70:0/2873354940 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f44a406aa70 0x7f44a4158da0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:11.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.816+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 >> 172.21.15.70:0/2873354940 conn(0x7f44a4069c70 msgr2=0x7f44a413e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.818+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 shutdown_connections 2023-08-25T09:46:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:11.818+0000 7f44aa006640 1 -- 172.21.15.70:0/2873354940 wait complete. 2023-08-25T09:46:11.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:46:12.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.057+0000 7f277a1f3640 1 Processor -- start 2023-08-25T09:46:12.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.057+0000 7f277a1f3640 1 -- start start 2023-08-25T09:46:12.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f277a1f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27740c2110 0x7f2774069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f277a1f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 0x7f27740ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f277a1f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27740c2550 con 0x7f277415ae50 2023-08-25T09:46:12.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f277a1f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277406a220 con 0x7f27740c2110 2023-08-25T09:46:12.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f277a1f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740ce330 con 0x7f2774069d90 2023-08-25T09:46:12.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f27737fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f277415ae50 0x7f27740cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59318/0 (socket says 172.21.15.70:59318) 2023-08-25T09:46:12.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f27737fe640 1 -- 172.21.15.70:0/1408693941 learned_addr learned my addr 172.21.15.70:0/1408693941 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:12.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f2773fff640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 0x7f27740ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f2772ffd640 1 --2- 172.21.15.70:0/1408693941 >> v2:172.21.15.155:3300/0 conn(0x7f27740c2110 0x7f2774069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.059+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783524976 0 0) 0x7f27740c2550 con 0x7f277415ae50 2023-08-25T09:46:12.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f275c003520 con 0x7f277415ae50 2023-08-25T09:46:12.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680147074 0 0) 0x7f275c003520 con 0x7f277415ae50 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 msgr2=0x7f27740ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 0x7f27740ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 >> v2:172.21.15.155:3300/0 conn(0x7f27740c2110 msgr2=0x7f2774069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 --2- 172.21.15.70:0/1408693941 >> v2:172.21.15.155:3300/0 conn(0x7f27740c2110 0x7f2774069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740cc5f0 con 0x7f277415ae50 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f2764001670 con 0x7f277415ae50 2023-08-25T09:46:12.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 >> v1:172.21.15.70:6789/0 conn(0x7f277415ae50 legacy=0x7f27740cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 0x7f275c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f275c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f275c004d00 0x7f275c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277406a220 con 0x7f275c003ef0 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27740ce330 con 0x7f275c004d00 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c00fdc0 con 0x7f2774069d90 2023-08-25T09:46:12.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.060+0000 7f2772ffd640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f275c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2773fff640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f275c004d00 0x7f275c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f27737fe640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 0x7f275c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2772ffd640 1 -- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 msgr2=0x7f275c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2772ffd640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2774069d90 0x7f275c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2772ffd640 1 -- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f275c004d00 msgr2=0x7f275c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2772ffd640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f275c004d00 0x7f275c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2772ffd640 1 -- 172.21.15.70:0/1408693941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c003520 con 0x7f275c003ef0 2023-08-25T09:46:12.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.061+0000 7f2773fff640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f275c004d00 0x7f275c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:12.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.062+0000 7f2772ffd640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f275c0047c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2768009ac0 tx=0x7f275c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabb718c854d188e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.062+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276802c710 con 0x7f275c003ef0 2023-08-25T09:46:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.062+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276802c870 con 0x7f275c003ef0 2023-08-25T09:46:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.062+0000 7f27727fc640 1 -- 172.21.15.70:0/1408693941 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276802cc00 con 0x7f275c003ef0 2023-08-25T09:46:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f277a1f3640 1 -- 172.21.15.70:0/1408693941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 msgr2=0x7f275c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f277a1f3640 1 --2- 172.21.15.70:0/1408693941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f275c0047c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f2768009ac0 tx=0x7f275c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f2773fff640 1 -- 172.21.15.70:0/1408693941 reap_dead start 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f277a1f3640 1 -- 172.21.15.70:0/1408693941 shutdown_connections 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f277a1f3640 1 -- 172.21.15.70:0/1408693941 >> 172.21.15.70:0/1408693941 conn(0x7f27740cb0d0 msgr2=0x7f27740cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f277a1f3640 1 -- 172.21.15.70:0/1408693941 shutdown_connections 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.063+0000 7f277a1f3640 1 -- 172.21.15.70:0/1408693941 wait complete. 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.065+0000 7f277a1f3640 1 Processor -- start 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.065+0000 7f277a1f3640 1 -- start start 2023-08-25T09:46:12.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f277a1f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f277414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f277a1f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f277415bc80 0x7f2774154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f277a1f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f277415c100 0x7f27740c3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f277a1f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277415b230 con 0x7f275c003ef0 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f277a1f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277415b7f0 con 0x7f277415c100 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f277a1f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2774155a60 con 0x7f277415bc80 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.066+0000 7f27737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f277414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f277414de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51232/0 (socket says 172.21.15.70:51232) 2023-08-25T09:46:12.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 -- 172.21.15.70:0/2100580309 learned_addr learned my addr 172.21.15.70:0/2100580309 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:12.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f2773fff640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f277415c100 0x7f27740c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f2772ffd640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f277415bc80 0x7f2774154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 -- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f277415bc80 msgr2=0x7f2774154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:12.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f277415bc80 0x7f2774154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 -- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f277415c100 msgr2=0x7f27740c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:12.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f277415c100 0x7f27740c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 -- 172.21.15.70:0/2100580309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2774155d40 con 0x7f275c003ef0 2023-08-25T09:46:12.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.067+0000 7f27737fe640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f277414de80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f276400d850 tx=0x7f276400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:12.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.068+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27640074e0 con 0x7f275c003ef0 2023-08-25T09:46:12.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.068+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764007640 con 0x7f275c003ef0 2023-08-25T09:46:12.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.068+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764004a20 con 0x7f275c003ef0 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.068+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f277415b480 con 0x7f275c003ef0 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.068+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27741554e0 con 0x7f275c003ef0 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.069+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2774155f80 con 0x7f275c003ef0 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.070+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2764004b80 con 0x7f275c003ef0 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.071+0000 7f275bfff640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276006a050 0x7f276006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.071+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f2764087ba0 con 0x7f275c003ef0 2023-08-25T09:46:12.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.071+0000 7f2772ffd640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276006a050 0x7f276006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:12.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.072+0000 7f2772ffd640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276006a050 0x7f276006c510 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2768036000 tx=0x7f2768030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:12.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.076+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2764053460 con 0x7f275c003ef0 2023-08-25T09:46:12.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.268+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2774066fb0 con 0x7f275c003ef0 2023-08-25T09:46:12.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.825+0000 7f275bfff640 1 -- 172.21.15.70:0/2100580309 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v131) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2764019020 con 0x7f275c003ef0 2023-08-25T09:46:12.830 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:46:12.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.835+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276006a050 msgr2=0x7f276006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:12.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.835+0000 7f277a1f3640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276006a050 0x7f276006c510 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2768036000 tx=0x7f2768030000 comp rx=0 tx=0).stop 2023-08-25T09:46:12.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.835+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 msgr2=0x7f277414de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:12.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.836+0000 7f277a1f3640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f277414de80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f276400d850 tx=0x7f276400dd10 comp rx=0 tx=0).stop 2023-08-25T09:46:12.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.838+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 shutdown_connections 2023-08-25T09:46:12.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.838+0000 7f277a1f3640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f276006a050 0x7f276006c510 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.838+0000 7f277a1f3640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f277415c100 0x7f27740c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.838+0000 7f277a1f3640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f277415bc80 0x7f2774154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.838+0000 7f277a1f3640 1 --2- 172.21.15.70:0/2100580309 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f275c003ef0 0x7f277414de80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:12.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.838+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 >> 172.21.15.70:0/2100580309 conn(0x7f27740cb0d0 msgr2=0x7f27740c18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:12.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.841+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 shutdown_connections 2023-08-25T09:46:12.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:12.842+0000 7f277a1f3640 1 -- 172.21.15.70:0/2100580309 wait complete. 2023-08-25T09:46:12.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:46:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.062+0000 7fea2d0fa640 1 Processor -- start 2023-08-25T09:46:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.063+0000 7fea2d0fa640 1 -- start start 2023-08-25T09:46:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.064+0000 7fea2d0fa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea280cd660 0x7fea280cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.064+0000 7fea2d0fa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2806f8e0 con 0x7fea2815e240 2023-08-25T09:46:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.064+0000 7fea2d0fa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2806fbc0 con 0x7fea28159bf0 2023-08-25T09:46:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.064+0000 7fea2d0fa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea280ce030 con 0x7fea280cd660 2023-08-25T09:46:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea1e59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fea2815e240 0x7fea28070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59322/0 (socket says 172.21.15.70:59322) 2023-08-25T09:46:13.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea1e59c640 1 -- 172.21.15.70:0/2725108279 learned_addr learned my addr 172.21.15.70:0/2725108279 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2759e640 1 --2- 172.21.15.70:0/2725108279 >> v2:172.21.15.155:3300/0 conn(0x7fea280cd660 0x7fea280cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1092389797 0 0) 0x7fea2806f8e0 con 0x7fea2815e240 2023-08-25T09:46:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea14005730 con 0x7fea2815e240 2023-08-25T09:46:13.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1740484431 0 0) 0x7fea14005730 con 0x7fea2815e240 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 >> v2:172.21.15.155:3300/0 conn(0x7fea280cd660 msgr2=0x7fea280cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 --2- 172.21.15.70:0/2725108279 >> v2:172.21.15.155:3300/0 conn(0x7fea280cd660 0x7fea280cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 >> v1:172.21.15.155:6789/0 conn(0x7fea28159bf0 legacy=0x7fea2806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2806fea0 con 0x7fea2815e240 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.065+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fea10001670 con 0x7fea2815e240 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 >> v1:172.21.15.70:6789/0 conn(0x7fea2815e240 legacy=0x7fea28070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea14005b80 0x7fea14005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea140069d0 0x7fea14006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea14007390 0x7fea1400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea280ce030 con 0x7fea140069d0 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1400e230 con 0x7fea14005b80 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1400e3f0 con 0x7fea14007390 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea1e59c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea140069d0 0x7fea14006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea26d9d640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea14005b80 0x7fea14005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea2759e640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea14007390 0x7fea1400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea1e59c640 1 -- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea14007390 msgr2=0x7fea1400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea1e59c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea14007390 0x7fea1400db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea1e59c640 1 -- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea14005b80 msgr2=0x7fea14005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea1e59c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea14005b80 0x7fea14005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.066+0000 7fea1e59c640 1 -- 172.21.15.70:0/2725108279 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea14005730 con 0x7fea140069d0 2023-08-25T09:46:13.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.067+0000 7fea2759e640 1 -- 172.21.15.70:0/2725108279 reap_dead start 2023-08-25T09:46:13.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.068+0000 7fea1e59c640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea140069d0 0x7fea14006e50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fea1000c240 tx=0x7fea14007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97116ee216ca19d4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.068+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10005ad0 con 0x7fea140069d0 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.068+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea10005c50 con 0x7fea140069d0 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.068+0000 7fea2659c640 1 -- 172.21.15.70:0/2725108279 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1002c830 con 0x7fea140069d0 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 -- 172.21.15.70:0/2725108279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea140069d0 msgr2=0x7fea14006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea140069d0 0x7fea14006e50 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fea1000c240 tx=0x7fea14007a00 comp rx=0 tx=0).stop 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 -- 172.21.15.70:0/2725108279 shutdown_connections 2023-08-25T09:46:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/2725108279 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea140069d0 0x7fea14006e50 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 -- 172.21.15.70:0/2725108279 >> 172.21.15.70:0/2725108279 conn(0x7fea280cb0d0 msgr2=0x7fea280cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:13.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 -- 172.21.15.70:0/2725108279 shutdown_connections 2023-08-25T09:46:13.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.069+0000 7fea2d0fa640 1 -- 172.21.15.70:0/2725108279 wait complete. 2023-08-25T09:46:13.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.071+0000 7fea2d0fa640 1 Processor -- start 2023-08-25T09:46:13.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.071+0000 7fea2d0fa640 1 -- start start 2023-08-25T09:46:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2d0fa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea140069d0 0x7fea28153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2d0fa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea280ac880 0x7fea280acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2d0fa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 0x7fea281f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2d0fa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea281f3fc0 con 0x7fea281f2620 2023-08-25T09:46:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2d0fa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea280ad240 con 0x7fea140069d0 2023-08-25T09:46:13.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2d0fa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2815cc80 con 0x7fea280ac880 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 0x7fea281f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 0x7fea281f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51244/0 (socket says 172.21.15.70:51244) 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea2759e640 1 -- 172.21.15.70:0/4087373346 learned_addr learned my addr 172.21.15.70:0/4087373346 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea26d9d640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea140069d0 0x7fea28153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.072+0000 7fea1e59c640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea280ac880 0x7fea280acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2759e640 1 -- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea280ac880 msgr2=0x7fea280acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2759e640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea280ac880 0x7fea280acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2759e640 1 -- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea140069d0 msgr2=0x7fea28153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2759e640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea140069d0 0x7fea28153f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2759e640 1 -- 172.21.15.70:0/4087373346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea281f32f0 con 0x7fea281f2620 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea1e59c640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea280ac880 0x7fea280acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2759e640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 0x7fea281f2aa0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fea2806f7f0 tx=0x7fea20010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000e8d0 con 0x7fea281f2620 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2000ea30 con 0x7fea281f2620 2023-08-25T09:46:13.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea281f35d0 con 0x7fea281f2620 2023-08-25T09:46:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2000ede0 con 0x7fea281f2620 2023-08-25T09:46:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.073+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea281f3be0 con 0x7fea281f2620 2023-08-25T09:46:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.074+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea281f4100 con 0x7fea281f2620 2023-08-25T09:46:13.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.076+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fea20020410 con 0x7fea281f2620 2023-08-25T09:46:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.077+0000 7fea1ffff640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea0806a050 0x7fea0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.077+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fea20088700 con 0x7fea281f2620 2023-08-25T09:46:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.077+0000 7fea26d9d640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea0806a050 0x7fea0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:13.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.078+0000 7fea26d9d640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea0806a050 0x7fea0806c510 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fea18007fd0 tx=0x7fea18009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:13.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.081+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fea20053e30 con 0x7fea281f2620 2023-08-25T09:46:13.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.308+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fea281f4100 con 0x7fea281f2620 2023-08-25T09:46:13.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.835+0000 7fea1ffff640 1 -- 172.21.15.70:0/4087373346 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7fea281f4100 con 0x7fea281f2620 2023-08-25T09:46:13.843 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 15 and data pool 16 2023-08-25T09:46:13.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.844+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea0806a050 msgr2=0x7fea0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.844+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea0806a050 0x7fea0806c510 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fea18007fd0 tx=0x7fea18009040 comp rx=0 tx=0).stop 2023-08-25T09:46:13.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.844+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 msgr2=0x7fea281f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:13.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.844+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 0x7fea281f2aa0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fea2806f7f0 tx=0x7fea20010c20 comp rx=0 tx=0).stop 2023-08-25T09:46:13.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.844+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 shutdown_connections 2023-08-25T09:46:13.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.845+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea0806a050 0x7fea0806c510 secure :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fea18007fd0 tx=0x7fea18009040 comp rx=0 tx=0).stop 2023-08-25T09:46:13.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.845+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea281f2620 0x7fea281f2aa0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.845+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea280ac880 0x7fea280acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.845+0000 7fea2d0fa640 1 --2- 172.21.15.70:0/4087373346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea140069d0 0x7fea28153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:13.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.845+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 >> 172.21.15.70:0/4087373346 conn(0x7fea280cb0d0 msgr2=0x7fea280c9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:13.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.849+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 shutdown_connections 2023-08-25T09:46:13.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:13.856+0000 7fea2d0fa640 1 -- 172.21.15.70:0/4087373346 wait complete. 2023-08-25T09:46:13.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:14.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.059+0000 7fef71e38640 1 Processor -- start 2023-08-25T09:46:14.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.060+0000 7fef71e38640 1 -- start start 2023-08-25T09:46:14.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.061+0000 7fef71e38640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fef6c0b2850 0x7fef6c164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.061+0000 7fef71e38640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c15d3f0 con 0x7fef6c15aa10 2023-08-25T09:46:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.061+0000 7fef71e38640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c070a00 con 0x7fef6c0ce5b0 2023-08-25T09:46:14.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.061+0000 7fef71e38640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c159d10 con 0x7fef6c0b2850 2023-08-25T09:46:14.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.062+0000 7fef6affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fef6c15aa10 0x7fef6c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59328/0 (socket says 172.21.15.70:59328) 2023-08-25T09:46:14.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.062+0000 7fef6affd640 1 -- 172.21.15.70:0/174285801 learned_addr learned my addr 172.21.15.70:0/174285801 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:14.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.062+0000 7fef6bfff640 1 --2- 172.21.15.70:0/174285801 >> v2:172.21.15.155:3300/0 conn(0x7fef6c0b2850 0x7fef6c164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.062+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473759051 0 0) 0x7fef6c15d3f0 con 0x7fef6c15aa10 2023-08-25T09:46:14.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.062+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef58005730 con 0x7fef6c15aa10 2023-08-25T09:46:14.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3045917619 0 0) 0x7fef58005730 con 0x7fef6c15aa10 2023-08-25T09:46:14.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 >> v2:172.21.15.155:3300/0 conn(0x7fef6c0b2850 msgr2=0x7fef6c164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 --2- 172.21.15.70:0/174285801 >> v2:172.21.15.155:3300/0 conn(0x7fef6c0b2850 0x7fef6c164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 >> v1:172.21.15.155:6789/0 conn(0x7fef6c0ce5b0 legacy=0x7fef6c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c0b2d20 con 0x7fef6c15aa10 2023-08-25T09:46:14.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fef5c0019f0 con 0x7fef6c15aa10 2023-08-25T09:46:14.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 >> v1:172.21.15.70:6789/0 conn(0x7fef6c15aa10 legacy=0x7fef6c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef58005b80 0x7fef58005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef58006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef580073e0 0x7fef5800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c159d10 con 0x7fef580069d0 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5800e310 con 0x7fef580073e0 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5800e4d0 con 0x7fef58005b80 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6affd640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef58006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6b7fe640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef58005b80 0x7fef58005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6affd640 1 -- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef58005b80 msgr2=0x7fef58005fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6bfff640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef580073e0 0x7fef5800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6affd640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef58005b80 0x7fef58005fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6affd640 1 -- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef580073e0 msgr2=0x7fef5800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6affd640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef580073e0 0x7fef5800db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6affd640 1 -- 172.21.15.70:0/174285801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef58005730 con 0x7fef580069d0 2023-08-25T09:46:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6b7fe640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef58005b80 0x7fef58005fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6bfff640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef580073e0 0x7fef5800db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.063+0000 7fef6bfff640 1 -- 172.21.15.70:0/174285801 reap_dead start 2023-08-25T09:46:14.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.065+0000 7fef6affd640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef58006e50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a830 tx=0x7fef58007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd89cedaceead4f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.065+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0072d0 con 0x7fef580069d0 2023-08-25T09:46:14.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.065+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c02f070 con 0x7fef580069d0 2023-08-25T09:46:14.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.065+0000 7fef6a7fc640 1 -- 172.21.15.70:0/174285801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0076f0 con 0x7fef580069d0 2023-08-25T09:46:14.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.065+0000 7fef71e38640 1 -- 172.21.15.70:0/174285801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 msgr2=0x7fef58006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.065+0000 7fef71e38640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef58006e50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00a830 tx=0x7fef58007a00 comp rx=0 tx=0).stop 2023-08-25T09:46:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.066+0000 7fef71e38640 1 -- 172.21.15.70:0/174285801 shutdown_connections 2023-08-25T09:46:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.066+0000 7fef71e38640 1 --2- 172.21.15.70:0/174285801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef58006e50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.066+0000 7fef71e38640 1 -- 172.21.15.70:0/174285801 >> 172.21.15.70:0/174285801 conn(0x7fef6c069850 msgr2=0x7fef6c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.066+0000 7fef71e38640 1 -- 172.21.15.70:0/174285801 shutdown_connections 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.066+0000 7fef71e38640 1 -- 172.21.15.70:0/174285801 wait complete. 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.068+0000 7fef71e38640 1 Processor -- start 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.068+0000 7fef71e38640 1 -- start start 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef71e38640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef6c14cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef71e38640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef6c14d3c0 0x7fef6c14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef71e38640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef6c156c30 0x7fef6c1570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef71e38640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0ae960 con 0x7fef580069d0 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef71e38640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c157600 con 0x7fef6c14d3c0 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef71e38640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c150720 con 0x7fef6c156c30 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef6c14cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef6c14cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51262/0 (socket says 172.21.15.70:51262) 2023-08-25T09:46:14.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 -- 172.21.15.70:0/135332543 learned_addr learned my addr 172.21.15.70:0/135332543 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6affd640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef6c14d3c0 0x7fef6c14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 -- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef6c156c30 msgr2=0x7fef6c1570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef6c156c30 0x7fef6c1570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 -- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef6c14d3c0 msgr2=0x7fef6c14c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef6c14d3c0 0x7fef6c14c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.069+0000 7fef6b7fe640 1 -- 172.21.15.70:0/135332543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c150a00 con 0x7fef580069d0 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.070+0000 7fef6b7fe640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef6c14cd90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fef5400d420 tx=0x7fef5400d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.070+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5400b070 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.070+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef54007030 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.070+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c0ae2a0 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.070+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540073c0 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.070+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c14ca50 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.071+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c04e420 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.072+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fef54007520 con 0x7fef580069d0 2023-08-25T09:46:14.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.073+0000 7fef63fff640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef4c06a050 0x7fef4c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.073+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef540882a0 con 0x7fef580069d0 2023-08-25T09:46:14.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.073+0000 7fef6affd640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef4c06a050 0x7fef4c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.074+0000 7fef6affd640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef4c06a050 0x7fef4c06c510 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fef5c034000 tx=0x7fef5c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.078+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fef54053980 con 0x7fef580069d0 2023-08-25T09:46:14.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.266+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef6c06f510 con 0x7fef580069d0 2023-08-25T09:46:14.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.269+0000 7fef63fff640 1 -- 172.21.15.70:0/135332543 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+18338 (secure 0 0 0) 0x7fef54056fa0 con 0x7fef580069d0 2023-08-25T09:46:14.271 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:14.271 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":132,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:13.310076+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:14.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.271+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef4c06a050 msgr2=0x7fef4c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.271+0000 7fef71e38640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef4c06a050 0x7fef4c06c510 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fef5c034000 tx=0x7fef5c007000 comp rx=0 tx=0).stop 2023-08-25T09:46:14.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.271+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 msgr2=0x7fef6c14cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.271+0000 7fef71e38640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef6c14cd90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fef5400d420 tx=0x7fef5400d8e0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 shutdown_connections 2023-08-25T09:46:14.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef4c06a050 0x7fef4c06c510 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef6c156c30 0x7fef6c1570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef6c14d3c0 0x7fef6c14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 --2- 172.21.15.70:0/135332543 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef580069d0 0x7fef6c14cd90 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 >> 172.21.15.70:0/135332543 conn(0x7fef6c069850 msgr2=0x7fef6c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:14.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 shutdown_connections 2023-08-25T09:46:14.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.272+0000 7fef71e38640 1 -- 172.21.15.70:0/135332543 wait complete. 2023-08-25T09:46:14.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.505+0000 7f29f4f76640 1 Processor -- start 2023-08-25T09:46:14.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.505+0000 7f29f4f76640 1 -- start start 2023-08-25T09:46:14.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.506+0000 7f29f4f76640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f29f00b2850 0x7f29f0164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.506+0000 7f29f4f76640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f015d3f0 con 0x7f29f00ce5b0 2023-08-25T09:46:14.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.506+0000 7f29f4f76640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f0070a00 con 0x7f29f015aa10 2023-08-25T09:46:14.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.506+0000 7f29f4f76640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f0159d10 con 0x7f29f00b2850 2023-08-25T09:46:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.507+0000 7f29ee59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f29f00ce5b0 0x7f29f00ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59332/0 (socket says 172.21.15.70:59332) 2023-08-25T09:46:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.507+0000 7f29ee59c640 1 -- 172.21.15.70:0/436502992 learned_addr learned my addr 172.21.15.70:0/436502992 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.507+0000 7f29eed9d640 1 --2- 172.21.15.70:0/436502992 >> v2:172.21.15.155:3300/0 conn(0x7f29f00b2850 0x7f29f0164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.507+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162194594 0 0) 0x7f29f015d3f0 con 0x7f29f00ce5b0 2023-08-25T09:46:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.507+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29dc003520 con 0x7f29f00ce5b0 2023-08-25T09:46:14.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.507+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2873210410 0 0) 0x7f29dc003520 con 0x7f29f00ce5b0 2023-08-25T09:46:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 >> v2:172.21.15.155:3300/0 conn(0x7f29f00b2850 msgr2=0x7f29f0164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 --2- 172.21.15.70:0/436502992 >> v2:172.21.15.155:3300/0 conn(0x7f29f00b2850 0x7f29f0164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 >> v1:172.21.15.155:6789/0 conn(0x7f29f015aa10 legacy=0x7f29f00b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f00b2d20 con 0x7f29f00ce5b0 2023-08-25T09:46:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f29e0001670 con 0x7f29f00ce5b0 2023-08-25T09:46:14.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 >> v1:172.21.15.70:6789/0 conn(0x7f29f00ce5b0 legacy=0x7f29f00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29dc003970 0x7f29dc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc0047c0 0x7f29dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc0051d0 0x7f29dc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0159d10 con 0x7f29dc003970 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc00e310 con 0x7f29dc0051d0 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc00e4d0 con 0x7f29dc0047c0 2023-08-25T09:46:14.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29ee59c640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29dc003970 0x7f29dc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29ee59c640 1 -- 172.21.15.70:0/436502992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc0047c0 msgr2=0x7f29dc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29ee59c640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc0047c0 0x7f29dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29ee59c640 1 -- 172.21.15.70:0/436502992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc0051d0 msgr2=0x7f29dc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29ee59c640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29dc0051d0 0x7f29dc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29ee59c640 1 -- 172.21.15.70:0/436502992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc003520 con 0x7f29dc003970 2023-08-25T09:46:14.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.508+0000 7f29eed9d640 1 -- 172.21.15.70:0/436502992 reap_dead start 2023-08-25T09:46:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.509+0000 7f29ee59c640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29dc003970 0x7f29dc003db0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f29e000c240 tx=0x7f29dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b6df889142e1a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e0005ad0 con 0x7f29dc003970 2023-08-25T09:46:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e0005c30 con 0x7f29dc003970 2023-08-25T09:46:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29edd9b640 1 -- 172.21.15.70:0/436502992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e002caf0 con 0x7f29dc003970 2023-08-25T09:46:14.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 -- 172.21.15.70:0/436502992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29dc003970 msgr2=0x7f29dc003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29dc003970 0x7f29dc003db0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f29e000c240 tx=0x7f29dc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 -- 172.21.15.70:0/436502992 shutdown_connections 2023-08-25T09:46:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 --2- 172.21.15.70:0/436502992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29dc003970 0x7f29dc003db0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 -- 172.21.15.70:0/436502992 >> 172.21.15.70:0/436502992 conn(0x7f29f0069850 msgr2=0x7f29f0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 -- 172.21.15.70:0/436502992 shutdown_connections 2023-08-25T09:46:14.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.510+0000 7f29f4f76640 1 -- 172.21.15.70:0/436502992 wait complete. 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.512+0000 7f29f4f76640 1 Processor -- start 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.512+0000 7f29f4f76640 1 -- start start 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29f4f76640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc003970 0x7f29f0156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29f4f76640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 0x7f29f0157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29f4f76640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f013d800 0x7f29f013dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29f4f76640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f014f830 con 0x7f29f014d060 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29f4f76640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f013bf80 con 0x7f29f013d800 2023-08-25T09:46:14.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29f4f76640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f013e840 con 0x7f29dc003970 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 0x7f29f0157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 0x7f29f0157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51268/0 (socket says 172.21.15.70:51268) 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.513+0000 7f29e7fff640 1 -- 172.21.15.70:0/806646579 learned_addr learned my addr 172.21.15.70:0/806646579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29ee59c640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc003970 0x7f29f0156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29eed9d640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f013d800 0x7f29f013dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e7fff640 1 -- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc003970 msgr2=0x7f29f0156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e7fff640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc003970 0x7f29f0156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e7fff640 1 -- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f013d800 msgr2=0x7f29f013dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e7fff640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f013d800 0x7f29f013dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e7fff640 1 -- 172.21.15.70:0/806646579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f013eb20 con 0x7f29f014d060 2023-08-25T09:46:14.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29ee59c640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc003970 0x7f29f0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:14.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e7fff640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 0x7f29f0157270 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c390 tx=0x7f29d800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d80052d0 con 0x7f29f014d060 2023-08-25T09:46:14.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8005450 con 0x7f29f014d060 2023-08-25T09:46:14.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8007260 con 0x7f29f014d060 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f013ee00 con 0x7f29f014d060 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.514+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f013e1c0 con 0x7f29f014d060 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.516+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f004e420 con 0x7f29f014d060 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.517+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29d80073c0 con 0x7f29f014d060 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.517+0000 7f29e6ffd640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a050 0x7f29d006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.518+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29d8087970 con 0x7f29f014d060 2023-08-25T09:46:14.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.518+0000 7f29ee59c640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a050 0x7f29d006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.518+0000 7f29ee59c640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a050 0x7f29d006c510 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29e0034000 tx=0x7f29e0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.523+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29d8053060 con 0x7f29f014d060 2023-08-25T09:46:14.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.697+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29f00cae50 con 0x7f29f014d060 2023-08-25T09:46:14.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.699+0000 7f29e6ffd640 1 -- 172.21.15.70:0/806646579 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+18338 (secure 0 0 0) 0x7f29d8056680 con 0x7f29f014d060 2023-08-25T09:46:14.701 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:14.701 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":132,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:13.310076+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:14.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a050 msgr2=0x7f29d006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a050 0x7f29d006c510 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29e0034000 tx=0x7f29e0009030 comp rx=0 tx=0).stop 2023-08-25T09:46:14.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 msgr2=0x7f29f0157270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 0x7f29f0157270 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c390 tx=0x7f29d800c850 comp rx=0 tx=0).stop 2023-08-25T09:46:14.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 shutdown_connections 2023-08-25T09:46:14.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29d006a050 0x7f29d006c510 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29f013d800 0x7f29f013dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29f014d060 0x7f29f0157270 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 --2- 172.21.15.70:0/806646579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29dc003970 0x7f29f0156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 >> 172.21.15.70:0/806646579 conn(0x7f29f0069850 msgr2=0x7f29f006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:14.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 shutdown_connections 2023-08-25T09:46:14.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.701+0000 7f29f4f76640 1 -- 172.21.15.70:0/806646579 wait complete. 2023-08-25T09:46:14.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:46:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.936+0000 7ff5aca46640 1 Processor -- start 2023-08-25T09:46:14.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.936+0000 7ff5aca46640 1 -- start start 2023-08-25T09:46:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.937+0000 7ff5aca46640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff5a8069850 0x7ff5a815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.937+0000 7ff5aca46640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5a80c9620 0x7ff5a8069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.937+0000 7ff5aca46640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80ac880 con 0x7ff5a8160be0 2023-08-25T09:46:14.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.937+0000 7ff5aca46640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80ad400 con 0x7ff5a80c9620 2023-08-25T09:46:14.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.937+0000 7ff5aca46640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80ad890 con 0x7ff5a8069850 2023-08-25T09:46:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff5a6d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff5a8160be0 0x7ff5a8161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59348/0 (socket says 172.21.15.70:59348) 2023-08-25T09:46:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff59ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff5a80c9620 0x7ff5a8069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff5a6d9d640 1 -- 172.21.15.70:0/3746011324 learned_addr learned my addr 172.21.15.70:0/3746011324 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff5a659c640 1 --2- 172.21.15.70:0/3746011324 >> v2:172.21.15.155:3300/0 conn(0x7ff5a8069850 0x7ff5a815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff59ffff640 1 -- 172.21.15.70:0/3746011324 >> v2:172.21.15.155:3300/0 conn(0x7ff5a8069850 msgr2=0x7ff5a815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff59ffff640 1 --2- 172.21.15.70:0/3746011324 >> v2:172.21.15.155:3300/0 conn(0x7ff5a8069850 0x7ff5a815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff59ffff640 1 -- 172.21.15.70:0/3746011324 >> v1:172.21.15.70:6789/0 conn(0x7ff5a8160be0 legacy=0x7ff5a8161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff59ffff640 1 -- 172.21.15.70:0/3746011324 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80acad0 con 0x7ff5a80c9620 2023-08-25T09:46:14.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.938+0000 7ff5a5d9b640 1 -- 172.21.15.70:0/3746011324 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991964023 0 0) 0x7ff5a80ac880 con 0x7ff5a8160be0 2023-08-25T09:46:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.939+0000 7ff59ffff640 1 --2- 172.21.15.70:0/3746011324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a8069c30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80cc0d0 tx=0x7ff59002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4bd097b9abe8de server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5a5d9b640 1 -- 172.21.15.70:0/3746011324 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590030610 con 0x7ff5a80c9620 2023-08-25T09:46:14.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5a5d9b640 1 -- 172.21.15.70:0/3746011324 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff590030770 con 0x7ff5a80c9620 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5a5d9b640 1 -- 172.21.15.70:0/3746011324 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590030b00 con 0x7ff5a80c9620 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 -- 172.21.15.70:0/3746011324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 msgr2=0x7ff5a8069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 --2- 172.21.15.70:0/3746011324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a8069c30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80cc0d0 tx=0x7ff59002b960 comp rx=0 tx=0).stop 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 -- 172.21.15.70:0/3746011324 shutdown_connections 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 --2- 172.21.15.70:0/3746011324 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a8069c30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 --2- 172.21.15.70:0/3746011324 >> v2:172.21.15.155:3300/0 conn(0x7ff5a8069850 0x7ff5a815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 -- 172.21.15.70:0/3746011324 >> 172.21.15.70:0/3746011324 conn(0x7ff5a80cb0d0 msgr2=0x7ff5a80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:14.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 -- 172.21.15.70:0/3746011324 shutdown_connections 2023-08-25T09:46:14.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.940+0000 7ff5aca46640 1 -- 172.21.15.70:0/3746011324 wait complete. 2023-08-25T09:46:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.942+0000 7ff5aca46640 1 Processor -- start 2023-08-25T09:46:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.942+0000 7ff5aca46640 1 -- start start 2023-08-25T09:46:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff5aca46640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a8069850 0x7ff5a814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff5aca46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a80c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff5aca46640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a8160be0 0x7ff5a80c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff5aca46640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c74a0 con 0x7ff5a80c9620 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff5aca46640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c8480 con 0x7ff5a8069850 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff5aca46640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b3590 con 0x7ff5a8160be0 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff59ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a80c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff59ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a80c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51288/0 (socket says 172.21.15.70:51288) 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.943+0000 7ff59ffff640 1 -- 172.21.15.70:0/4032773629 learned_addr learned my addr 172.21.15.70:0/4032773629 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:14.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59ffff640 1 -- 172.21.15.70:0/4032773629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a8160be0 msgr2=0x7ff5a80c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59ffff640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a8160be0 0x7ff5a80c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59ffff640 1 -- 172.21.15.70:0/4032773629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a8069850 msgr2=0x7ff5a814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59ffff640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a8069850 0x7ff5a814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59ffff640 1 -- 172.21.15.70:0/4032773629 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590009400 con 0x7ff5a80c9620 2023-08-25T09:46:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59ffff640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a80c7910 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff590009890 tx=0x7ff590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59003e040 con 0x7ff5a80c9620 2023-08-25T09:46:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff590039cb0 con 0x7ff5a80c9620 2023-08-25T09:46:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff590044630 con 0x7ff5a80c9620 2023-08-25T09:46:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a80b3870 con 0x7ff5a80c9620 2023-08-25T09:46:14.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.944+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a80b3e30 con 0x7ff5a80c9620 2023-08-25T09:46:14.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.945+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a806a150 con 0x7ff5a80c9620 2023-08-25T09:46:14.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.946+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff59004d3b0 con 0x7ff5a80c9620 2023-08-25T09:46:14.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.947+0000 7ff59effd640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff58806a050 0x7ff58806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:14.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.947+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5900aad60 con 0x7ff5a80c9620 2023-08-25T09:46:14.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.947+0000 7ff5a659c640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff58806a050 0x7ff58806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:14.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.948+0000 7ff5a659c640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff58806a050 0x7ff58806c510 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff5980098f0 tx=0x7ff598008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:14.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:14.952+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff5a806a150 con 0x7ff5a80c9620 2023-08-25T09:46:15.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.177+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff5a806a150 con 0x7ff5a80c9620 2023-08-25T09:46:15.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.889+0000 7ff59effd640 1 -- 172.21.15.70:0/4032773629 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff5a806a150 con 0x7ff5a80c9620 2023-08-25T09:46:15.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff58806a050 msgr2=0x7ff58806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:15.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff58806a050 0x7ff58806c510 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff5980098f0 tx=0x7ff598008040 comp rx=0 tx=0).stop 2023-08-25T09:46:15.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 msgr2=0x7ff5a80c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:15.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a80c7910 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff590009890 tx=0x7ff590004730 comp rx=0 tx=0).stop 2023-08-25T09:46:15.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 shutdown_connections 2023-08-25T09:46:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff58806a050 0x7ff58806c510 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff5a8160be0 0x7ff5a80c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff5a80c9620 0x7ff5a80c7910 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 --2- 172.21.15.70:0/4032773629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff5a8069850 0x7ff5a814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 >> 172.21.15.70:0/4032773629 conn(0x7ff5a80cb0d0 msgr2=0x7ff5a806c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:15.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 shutdown_connections 2023-08-25T09:46:15.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:15.893+0000 7ff5aca46640 1 -- 172.21.15.70:0/4032773629 wait complete. 2023-08-25T09:46:15.917 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:46:15.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:46:15.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:46:15.992 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:46:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.132+0000 7f321b59e640 1 Processor -- start 2023-08-25T09:46:16.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.133+0000 7f321b59e640 1 -- start start 2023-08-25T09:46:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f321c0cfba0 0x7f321c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:16.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321b59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0d09d0 con 0x7f321c15cc00 2023-08-25T09:46:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321b59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0d1270 con 0x7f321c0cf710 2023-08-25T09:46:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c15cfe0 con 0x7f321c0cfba0 2023-08-25T09:46:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f321c15cc00 0x7f321c0cd160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59354/0 (socket says 172.21.15.70:59354) 2023-08-25T09:46:16.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321a59c640 1 -- 172.21.15.70:0/3418278171 learned_addr learned my addr 172.21.15.70:0/3418278171 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.134+0000 7f321ad9d640 1 --2- 172.21.15.70:0/3418278171 >> v2:172.21.15.70:3300/0 conn(0x7f321c0cfba0 0x7f321c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.135+0000 7f321959a640 1 -- 172.21.15.70:0/3418278171 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765980326 0 0) 0x7f321c0d09d0 con 0x7f321c15cc00 2023-08-25T09:46:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.135+0000 7f321959a640 1 -- 172.21.15.70:0/3418278171 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3208003520 con 0x7f321c15cc00 2023-08-25T09:46:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.135+0000 7f321ad9d640 1 -- 172.21.15.70:0/3418278171 >> v1:172.21.15.155:6789/0 conn(0x7f321c0cf710 legacy=0x7f321c0b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:16.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.135+0000 7f321ad9d640 1 -- 172.21.15.70:0/3418278171 >> v1:172.21.15.70:6789/0 conn(0x7f321c15cc00 legacy=0x7f321c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:16.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.135+0000 7f321ad9d640 1 -- 172.21.15.70:0/3418278171 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0cdc20 con 0x7f321c0cfba0 2023-08-25T09:46:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.136+0000 7f321ad9d640 1 --2- 172.21.15.70:0/3418278171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0cfba0 0x7f321c0b5f60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f32140097a0 tx=0x7f321402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac63ddbcef8f6e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.136+0000 7f321959a640 1 -- 172.21.15.70:0/3418278171 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214031830 con 0x7f321c0cfba0 2023-08-25T09:46:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.136+0000 7f321959a640 1 -- 172.21.15.70:0/3418278171 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214031990 con 0x7f321c0cfba0 2023-08-25T09:46:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.136+0000 7f321959a640 1 -- 172.21.15.70:0/3418278171 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214031d20 con 0x7f321c0cfba0 2023-08-25T09:46:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 -- 172.21.15.70:0/3418278171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0cfba0 msgr2=0x7f321c0b5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 --2- 172.21.15.70:0/3418278171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0cfba0 0x7f321c0b5f60 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f32140097a0 tx=0x7f321402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 -- 172.21.15.70:0/3418278171 shutdown_connections 2023-08-25T09:46:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 --2- 172.21.15.70:0/3418278171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c0cfba0 0x7f321c0b5f60 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 -- 172.21.15.70:0/3418278171 >> 172.21.15.70:0/3418278171 conn(0x7f321c070050 msgr2=0x7f321c070430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 -- 172.21.15.70:0/3418278171 shutdown_connections 2023-08-25T09:46:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.137+0000 7f321b59e640 1 -- 172.21.15.70:0/3418278171 wait complete. 2023-08-25T09:46:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.139+0000 7f321b59e640 1 Processor -- start 2023-08-25T09:46:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.139+0000 7f321b59e640 1 -- start start 2023-08-25T09:46:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf710 0x7f321c156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321b59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cfba0 0x7f321c0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 0x7f321c0c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c157a90 con 0x7f321c15cc00 2023-08-25T09:46:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321b59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0cbd60 con 0x7f321c0cfba0 2023-08-25T09:46:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c0cc040 con 0x7f321c0cf710 2023-08-25T09:46:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 0x7f321c0c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 0x7f321c0c99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51318/0 (socket says 172.21.15.70:51318) 2023-08-25T09:46:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 -- 172.21.15.70:0/3090326352 learned_addr learned my addr 172.21.15.70:0/3090326352 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321a59c640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf710 0x7f321c156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f3219d9b640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cfba0 0x7f321c0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 -- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf710 msgr2=0x7f321c156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf710 0x7f321c156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 -- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cfba0 msgr2=0x7f321c0b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cfba0 0x7f321c0b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f321ad9d640 1 -- 172.21.15.70:0/3090326352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214009400 con 0x7f321c15cc00 2023-08-25T09:46:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.140+0000 7f3219d9b640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cfba0 0x7f321c0b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.141+0000 7f321ad9d640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 0x7f321c0c99d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f32140098c0 tx=0x7f32140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:16.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.141+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321403a630 con 0x7f321c15cc00 2023-08-25T09:46:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.141+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321403a790 con 0x7f321c15cc00 2023-08-25T09:46:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.141+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321403ab20 con 0x7f321c15cc00 2023-08-25T09:46:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.141+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c0cc320 con 0x7f321c15cc00 2023-08-25T09:46:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.141+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c0cc8e0 con 0x7f321c15cc00 2023-08-25T09:46:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.142+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c0ccb20 con 0x7f321c15cc00 2023-08-25T09:46:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.143+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f321403ac80 con 0x7f321c15cc00 2023-08-25T09:46:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.144+0000 7f3212ffd640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31fc06a050 0x7f31fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.144+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32140aa250 con 0x7f321c15cc00 2023-08-25T09:46:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.144+0000 7f321a59c640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31fc06a050 0x7f31fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.145+0000 7f321a59c640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31fc06a050 0x7f31fc06c510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f320c001560 tx=0x7f320c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.149+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f321c0ccb20 con 0x7f321c15cc00 2023-08-25T09:46:16.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.375+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f321c0ccb20 con 0x7f321c15cc00 2023-08-25T09:46:16.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.893+0000 7f3212ffd640 1 -- 172.21.15.70:0/3090326352 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f321c0ccb20 con 0x7f321c15cc00 2023-08-25T09:46:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31fc06a050 msgr2=0x7f31fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:16.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31fc06a050 0x7f31fc06c510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f320c001560 tx=0x7f320c009040 comp rx=0 tx=0).stop 2023-08-25T09:46:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 msgr2=0x7f321c0c99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 0x7f321c0c99d0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f32140098c0 tx=0x7f32140046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 shutdown_connections 2023-08-25T09:46:16.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31fc06a050 0x7f31fc06c510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f321c15cc00 0x7f321c0c99d0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f321c0cfba0 0x7f321c0b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 --2- 172.21.15.70:0/3090326352 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f321c0cf710 0x7f321c156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:16.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 >> 172.21.15.70:0/3090326352 conn(0x7f321c070050 msgr2=0x7f321c0b67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:16.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 shutdown_connections 2023-08-25T09:46:16.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:16.897+0000 7f321b59e640 1 -- 172.21.15.70:0/3090326352 wait complete. 2023-08-25T09:46:16.919 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:46:16.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:17.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.178+0000 7f12647e8640 1 Processor -- start 2023-08-25T09:46:17.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.178+0000 7f12647e8640 1 -- start start 2023-08-25T09:46:17.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f12647e8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 0x7f125c0b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f12647e8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f125c06f0a0 0x7f125c0b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f12647e8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f125c06ae10 con 0x7f125c159100 2023-08-25T09:46:17.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f12647e8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f125c06ba50 con 0x7f125c06f0a0 2023-08-25T09:46:17.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f12647e8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125c06b0b0 con 0x7f125c0709a0 2023-08-25T09:46:17.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f1262584640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f125c159100 0x7f125c06be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59360/0 (socket says 172.21.15.70:59360) 2023-08-25T09:46:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.180+0000 7f1262584640 1 -- 172.21.15.70:0/2038351941 learned_addr learned my addr 172.21.15.70:0/2038351941 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261d83640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 0x7f125c0b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1262d85640 1 --2- 172.21.15.70:0/2038351941 >> v2:172.21.15.155:3300/0 conn(0x7f125c06f0a0 0x7f125c0b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419040024 0 0) 0x7f125c06ae10 con 0x7f125c159100 2023-08-25T09:46:17.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1248003520 con 0x7f125c159100 2023-08-25T09:46:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3289011452 0 0) 0x7f1248003520 con 0x7f125c159100 2023-08-25T09:46:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 msgr2=0x7f125c0b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 0x7f125c0b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 >> v2:172.21.15.155:3300/0 conn(0x7f125c06f0a0 msgr2=0x7f125c0b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 --2- 172.21.15.70:0/2038351941 >> v2:172.21.15.155:3300/0 conn(0x7f125c06f0a0 0x7f125c0b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c06b390 con 0x7f125c159100 2023-08-25T09:46:17.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1258001670 con 0x7f125c159100 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1262d85640 1 --2- 172.21.15.70:0/2038351941 >> v2:172.21.15.155:3300/0 conn(0x7f125c06f0a0 0x7f125c0b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 >> v1:172.21.15.70:6789/0 conn(0x7f125c159100 legacy=0x7f125c06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f1248003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12480047c0 0x7f1248004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 0x7f1248005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c06ba50 con 0x7f1248003970 2023-08-25T09:46:17.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c06b0b0 con 0x7f12480047c0 2023-08-25T09:46:17.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124800bae0 con 0x7f125c0709a0 2023-08-25T09:46:17.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.181+0000 7f1262584640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f1248003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.182+0000 7f1261d83640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12480047c0 0x7f1248004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.182+0000 7f1262584640 1 -- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 msgr2=0x7f1248005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.182+0000 7f1262584640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0709a0 0x7f1248005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.182+0000 7f1262584640 1 -- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12480047c0 msgr2=0x7f1248004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.182+0000 7f1262584640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12480047c0 0x7f1248004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.182+0000 7f1262584640 1 -- 172.21.15.70:0/2038351941 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1248003520 con 0x7f1248003970 2023-08-25T09:46:17.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.183+0000 7f1262584640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f1248003d90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f125800c210 tx=0x7f12480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1528ee1e0fd6459 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:17.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.183+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258005ad0 con 0x7f1248003970 2023-08-25T09:46:17.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.183+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1258005c30 con 0x7f1248003970 2023-08-25T09:46:17.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.183+0000 7f1261582640 1 -- 172.21.15.70:0/2038351941 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125802c8b0 con 0x7f1248003970 2023-08-25T09:46:17.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f12647e8640 1 -- 172.21.15.70:0/2038351941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 msgr2=0x7f1248003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f12647e8640 1 --2- 172.21.15.70:0/2038351941 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f1248003d90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f125800c210 tx=0x7f12480057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f1262d85640 1 -- 172.21.15.70:0/2038351941 reap_dead start 2023-08-25T09:46:17.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f12647e8640 1 -- 172.21.15.70:0/2038351941 shutdown_connections 2023-08-25T09:46:17.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f12647e8640 1 -- 172.21.15.70:0/2038351941 >> 172.21.15.70:0/2038351941 conn(0x7f125c069850 msgr2=0x7f125c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:17.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f12647e8640 1 -- 172.21.15.70:0/2038351941 shutdown_connections 2023-08-25T09:46:17.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.184+0000 7f12647e8640 1 -- 172.21.15.70:0/2038351941 wait complete. 2023-08-25T09:46:17.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.186+0000 7f12647e8640 1 Processor -- start 2023-08-25T09:46:17.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.186+0000 7f12647e8640 1 -- start start 2023-08-25T09:46:17.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f12647e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f125c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f12647e8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0cc770 0x7f125c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f12647e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f125c14cc60 0x7f125c14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f12647e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c14d620 con 0x7f1248003970 2023-08-25T09:46:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f12647e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c15c800 con 0x7f125c14cc60 2023-08-25T09:46:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f12647e8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f125c151b50 con 0x7f125c0cc770 2023-08-25T09:46:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f125c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f125c1572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51338/0 (socket says 172.21.15.70:51338) 2023-08-25T09:46:17.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 -- 172.21.15.70:0/3896647343 learned_addr learned my addr 172.21.15.70:0/3896647343 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:17.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1261d83640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0cc770 0x7f125c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262d85640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f125c14cc60 0x7f125c14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 -- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0cc770 msgr2=0x7f125c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0cc770 0x7f125c15bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 -- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f125c14cc60 msgr2=0x7f125c14d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.187+0000 7f1262584640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f125c14cc60 0x7f125c14d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f1262584640 1 -- 172.21.15.70:0/3896647343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c151df0 con 0x7f1248003970 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f1262d85640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f125c14cc60 0x7f125c14d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:17.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f1262584640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f125c1572f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f125800c210 tx=0x7f1258001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258039030 con 0x7f1248003970 2023-08-25T09:46:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125802ccc0 con 0x7f1248003970 2023-08-25T09:46:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125803c040 con 0x7f1248003970 2023-08-25T09:46:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c1520d0 con 0x7f1248003970 2023-08-25T09:46:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.188+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c1528d0 con 0x7f1248003970 2023-08-25T09:46:17.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.190+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125c04e420 con 0x7f1248003970 2023-08-25T09:46:17.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.191+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1258032070 con 0x7f1248003970 2023-08-25T09:46:17.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.191+0000 7f1252ffd640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f124006a080 0x7f124006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.192+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12580a99c0 con 0x7f1248003970 2023-08-25T09:46:17.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.192+0000 7f1261d83640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f124006a080 0x7f124006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.193+0000 7f1261d83640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f124006a080 0x7f124006c540 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f125c15ba50 tx=0x7f124c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:17.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.196+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f12580750b0 con 0x7f1248003970 2023-08-25T09:46:17.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.423+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f125c06b640 con 0x7f1248003970 2023-08-25T09:46:17.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.424+0000 7f1252ffd640 1 -- 172.21.15.70:0/3896647343 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7f12580786d0 con 0x7f1248003970 2023-08-25T09:46:17.426 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:17.427 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:17.427 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:17.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f124006a080 msgr2=0x7f124006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f124006a080 0x7f124006c540 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f125c15ba50 tx=0x7f124c008040 comp rx=0 tx=0).stop 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 msgr2=0x7f125c1572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f125c1572f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f125800c210 tx=0x7f1258001a00 comp rx=0 tx=0).stop 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 shutdown_connections 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f124006a080 0x7f124006c540 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f125c14cc60 0x7f125c14d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f125c0cc770 0x7f125c15bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 --2- 172.21.15.70:0/3896647343 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1248003970 0x7f125c1572f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 >> 172.21.15.70:0/3896647343 conn(0x7f125c069850 msgr2=0x7f125c0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 shutdown_connections 2023-08-25T09:46:17.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.427+0000 7f12647e8640 1 -- 172.21.15.70:0/3896647343 wait complete. 2023-08-25T09:46:17.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:17.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.663+0000 7f67f7dab640 1 Processor -- start 2023-08-25T09:46:17.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.664+0000 7f67f7dab640 1 -- start start 2023-08-25T09:46:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.665+0000 7f67f7dab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67f0160ff0 0x7f67f006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f7dab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f01620b0 con 0x7f67f0161ae0 2023-08-25T09:46:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f7dab640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f0162390 con 0x7f67f00ce5f0 2023-08-25T09:46:17.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f7dab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f006ad70 con 0x7f67f0160ff0 2023-08-25T09:46:17.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f6348640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f67f0161ae0 0x7f67f0162970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59364/0 (socket says 172.21.15.70:59364) 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f6348640 1 -- 172.21.15.70:0/6730636 learned_addr learned my addr 172.21.15.70:0/6730636 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f5b47640 1 --2- 172.21.15.70:0/6730636 >> v2:172.21.15.155:3300/0 conn(0x7f67f0160ff0 0x7f67f006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.666+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985872474 0 0) 0x7f67f01620b0 con 0x7f67f0161ae0 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67dc003520 con 0x7f67f0161ae0 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2510225041 0 0) 0x7f67dc003520 con 0x7f67f0161ae0 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 >> v2:172.21.15.155:3300/0 conn(0x7f67f0160ff0 msgr2=0x7f67f006b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 --2- 172.21.15.70:0/6730636 >> v2:172.21.15.155:3300/0 conn(0x7f67f0160ff0 0x7f67f006b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 >> v1:172.21.15.155:6789/0 conn(0x7f67f00ce5f0 legacy=0x7f67f01613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f006af80 con 0x7f67f0161ae0 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f67e80019f0 con 0x7f67f0161ae0 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 >> v1:172.21.15.70:6789/0 conn(0x7f67f0161ae0 legacy=0x7f67f0162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc003970 0x7f67dc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc0047c0 0x7f67dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67dc005180 0x7f67dc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f006ad70 con 0x7f67dc0047c0 2023-08-25T09:46:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00c020 con 0x7f67dc003970 2023-08-25T09:46:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc00c1e0 con 0x7f67dc005180 2023-08-25T09:46:17.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.667+0000 7f67f5346640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc0047c0 0x7f67dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f5b47640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc003970 0x7f67dc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f6348640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67dc005180 0x7f67dc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f5346640 1 -- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67dc005180 msgr2=0x7f67dc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f5346640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67dc005180 0x7f67dc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f5346640 1 -- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc003970 msgr2=0x7f67dc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f5346640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc003970 0x7f67dc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f5346640 1 -- 172.21.15.70:0/6730636 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc003520 con 0x7f67dc0047c0 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.668+0000 7f67f6348640 1 -- 172.21.15.70:0/6730636 reap_dead start 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.669+0000 7f67f5346640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc0047c0 0x7f67dc004c40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f67e0001750 tx=0x7f67dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe88fef7ae3fbc5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:17.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.669+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00052d0 con 0x7f67dc0047c0 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.669+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e002e050 con 0x7f67dc0047c0 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.669+0000 7f67f4b45640 1 -- 172.21.15.70:0/6730636 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003830 con 0x7f67dc0047c0 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 -- 172.21.15.70:0/6730636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc0047c0 msgr2=0x7f67dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc0047c0 0x7f67dc004c40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f67e0001750 tx=0x7f67dc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 -- 172.21.15.70:0/6730636 shutdown_connections 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 --2- 172.21.15.70:0/6730636 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67dc0047c0 0x7f67dc004c40 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 -- 172.21.15.70:0/6730636 >> 172.21.15.70:0/6730636 conn(0x7f67f0069850 msgr2=0x7f67f0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:17.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 -- 172.21.15.70:0/6730636 shutdown_connections 2023-08-25T09:46:17.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.670+0000 7f67f7dab640 1 -- 172.21.15.70:0/6730636 wait complete. 2023-08-25T09:46:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.672+0000 7f67f7dab640 1 Processor -- start 2023-08-25T09:46:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.672+0000 7f67f7dab640 1 -- start start 2023-08-25T09:46:17.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f7dab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc0047c0 0x7f67f014d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f7dab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 0x7f67f015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f7dab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67f015d7e0 0x7f67f015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f7dab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f015c790 con 0x7f67f00ad190 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f7dab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f014e850 con 0x7f67dc0047c0 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f7dab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f014dc40 con 0x7f67f015d7e0 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f5346640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 0x7f67f015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f5346640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 0x7f67f015d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51348/0 (socket says 172.21.15.70:51348) 2023-08-25T09:46:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.673+0000 7f67f5346640 1 -- 172.21.15.70:0/3875956689 learned_addr learned my addr 172.21.15.70:0/3875956689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5b47640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc0047c0 0x7f67f014d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5346640 1 -- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67f015d7e0 msgr2=0x7f67f015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5346640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67f015d7e0 0x7f67f015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5346640 1 -- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc0047c0 msgr2=0x7f67f014d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5346640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc0047c0 0x7f67f014d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5346640 1 -- 172.21.15.70:0/3875956689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f014df20 con 0x7f67f00ad190 2023-08-25T09:46:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f5346640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 0x7f67f015d2a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005040 tx=0x7f67e0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00076c0 con 0x7f67f00ad190 2023-08-25T09:46:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e0007a30 con 0x7f67f00ad190 2023-08-25T09:46:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0006be0 con 0x7f67f00ad190 2023-08-25T09:46:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.674+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f014e200 con 0x7f67f00ad190 2023-08-25T09:46:17.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.675+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f014d880 con 0x7f67f00ad190 2023-08-25T09:46:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.676+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f014da10 con 0x7f67f00ad190 2023-08-25T09:46:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.677+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67e0006d40 con 0x7f67f00ad190 2023-08-25T09:46:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.678+0000 7f67e67fc640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67d406a050 0x7f67d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.678+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67e00aa680 con 0x7f67f00ad190 2023-08-25T09:46:17.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.678+0000 7f67f5b47640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67d406a050 0x7f67d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:17.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.679+0000 7f67f5b47640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67d406a050 0x7f67d406c510 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f67f00aca30 tx=0x7f67ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:17.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.683+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f67e0075cd0 con 0x7f67f00ad190 2023-08-25T09:46:17.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.873+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67f004e420 con 0x7f67f00ad190 2023-08-25T09:46:17.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.876+0000 7f67e67fc640 1 -- 172.21.15.70:0/3875956689 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+18325 (secure 0 0 0) 0x7f67e00792f0 con 0x7f67f00ad190 2023-08-25T09:46:17.878 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:17.878 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":133,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:14.823213+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"133","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:17.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67d406a050 msgr2=0x7f67d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67d406a050 0x7f67d406c510 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f67f00aca30 tx=0x7f67ec008040 comp rx=0 tx=0).stop 2023-08-25T09:46:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 msgr2=0x7f67f015d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 0x7f67f015d2a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005040 tx=0x7f67e0001120 comp rx=0 tx=0).stop 2023-08-25T09:46:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 shutdown_connections 2023-08-25T09:46:17.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f67d406a050 0x7f67d406c510 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67f015d7e0 0x7f67f015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67f00ad190 0x7f67f015d2a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 --2- 172.21.15.70:0/3875956689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67dc0047c0 0x7f67f014d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 >> 172.21.15.70:0/3875956689 conn(0x7f67f0069850 msgr2=0x7f67f00cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 shutdown_connections 2023-08-25T09:46:17.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:17.879+0000 7f67f7dab640 1 -- 172.21.15.70:0/3875956689 wait complete. 2023-08-25T09:46:17.899 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:46:17.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:46:18.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.117+0000 7f4ed2ca6640 1 Processor -- start 2023-08-25T09:46:18.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.118+0000 7f4ed2ca6640 1 -- start start 2023-08-25T09:46:18.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed2ca6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ecc0ce8e0 0x7f4ecc0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed2ca6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ecc06e090 0x7f4ecc164520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed2ca6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0cba70 con 0x7f4ecc0cdc80 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed2ca6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cbd50 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed2ca6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0cc030 con 0x7f4ecc06e090 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed0a42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ecc0ce8e0 0x7f4ecc0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed0a42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ecc0ce8e0 0x7f4ecc0cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51364/0 (socket says 172.21.15.70:51364) 2023-08-25T09:46:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.119+0000 7f4ed0a42640 1 -- 172.21.15.70:0/3433301310 learned_addr learned my addr 172.21.15.70:0/3433301310 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:18.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.120+0000 7f4ed0a42640 1 -- 172.21.15.70:0/3433301310 >> v2:172.21.15.155:3300/0 conn(0x7f4ecc06e090 msgr2=0x7f4ecc164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:18.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.120+0000 7f4ed0a42640 1 --2- 172.21.15.70:0/3433301310 >> v2:172.21.15.155:3300/0 conn(0x7f4ecc06e090 0x7f4ecc164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.120+0000 7f4ed0a42640 1 -- 172.21.15.70:0/3433301310 >> v1:172.21.15.155:6789/0 conn(0x7f4ecc0cdc80 legacy=0x7f4ecc0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:18.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.120+0000 7f4ed0a42640 1 -- 172.21.15.70:0/3433301310 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0cc310 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.121+0000 7f4ed0a42640 1 --2- 172.21.15.70:0/3433301310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc0cb470 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc070cb0 tx=0x7f4ebc02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a54359cf3c89c1b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:18.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.121+0000 7f4ecb7fe640 1 -- 172.21.15.70:0/3433301310 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc02cad0 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.121+0000 7f4ecb7fe640 1 -- 172.21.15.70:0/3433301310 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc02cc30 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.121+0000 7f4ecb7fe640 1 -- 172.21.15.70:0/3433301310 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc0343f0 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/3433301310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 msgr2=0x7f4ecc0cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/3433301310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc0cb470 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc070cb0 tx=0x7f4ebc02bf30 comp rx=0 tx=0).stop 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/3433301310 shutdown_connections 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/3433301310 >> v2:172.21.15.155:3300/0 conn(0x7f4ecc06e090 0x7f4ecc164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/3433301310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc0cb470 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/3433301310 >> 172.21.15.70:0/3433301310 conn(0x7f4ecc0d0860 msgr2=0x7f4ecc0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/3433301310 shutdown_connections 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.122+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/3433301310 wait complete. 2023-08-25T09:46:18.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.124+0000 7f4ed2ca6640 1 Processor -- start 2023-08-25T09:46:18.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.124+0000 7f4ed2ca6640 1 -- start start 2023-08-25T09:46:18.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.125+0000 7f4ed2ca6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ecc06e090 0x7f4ecc0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.125+0000 7f4ed2ca6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ecc0cdc80 0x7f4ecc1f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.125+0000 7f4ed2ca6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc1e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.125+0000 7f4ed2ca6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0aea10 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.125+0000 7f4ed2ca6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0b62d0 con 0x7f4ecc06e090 2023-08-25T09:46:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.125+0000 7f4ed2ca6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0ca5d0 con 0x7f4ecc0cdc80 2023-08-25T09:46:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc1e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc1e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51370/0 (socket says 172.21.15.70:51370) 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 -- 172.21.15.70:0/1131686407 learned_addr learned my addr 172.21.15.70:0/1131686407 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed0a42640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ecc06e090 0x7f4ecc0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ecbfff640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ecc0cdc80 0x7f4ecc1f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 -- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ecc0cdc80 msgr2=0x7f4ecc1f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ecc0cdc80 0x7f4ecc1f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 -- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ecc06e090 msgr2=0x7f4ecc0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ecc06e090 0x7f4ecc0b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 -- 172.21.15.70:0/1131686407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ebc009b30 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed0a42640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ecc06e090 0x7f4ecc0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ecbfff640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ecc0cdc80 0x7f4ecc1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.126+0000 7f4ed1243640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc1e43e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b3f0 tx=0x7f4ec400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.127+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4011830 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.127+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec40119b0 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.127+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc1f6f10 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.127+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4011d60 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.127+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc0ae070 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.128+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc04e420 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.129+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4ec401a400 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.130+0000 7f4ec97fa640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4eb806a050 0x7f4eb806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.130+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ec4088720 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.130+0000 7f4ed0a42640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4eb806a050 0x7f4eb806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.131+0000 7f4ed0a42640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4eb806a050 0x7f4eb806c510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc035000 tx=0x7f4ebc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:18.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.134+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ec4053e10 con 0x7f4ecc0ce8e0 2023-08-25T09:46:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.319+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ecc0ce0f0 con 0x7f4eb806a050 2023-08-25T09:46:18.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.330+0000 7f4ec97fa640 1 -- 172.21.15.70:0/1131686407 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303433 (secure 0 0 0) 0x7f4ebc0343f0 con 0x7f4eb806a050 2023-08-25T09:46:18.335 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:18.336 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:46:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4eb806a050 msgr2=0x7f4eb806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4eb806a050 0x7f4eb806c510 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc035000 tx=0x7f4ebc030000 comp rx=0 tx=0).stop 2023-08-25T09:46:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 msgr2=0x7f4ecc1e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc1e43e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4ec400b3f0 tx=0x7f4ec400b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 shutdown_connections 2023-08-25T09:46:18.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4eb806a050 0x7f4eb806c510 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ecc0ce8e0 0x7f4ecc1e43e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ecc0cdc80 0x7f4ecc1f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 --2- 172.21.15.70:0/1131686407 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ecc06e090 0x7f4ecc0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.335+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 >> 172.21.15.70:0/1131686407 conn(0x7f4ecc0d0860 msgr2=0x7f4ecc0af510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.336+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 shutdown_connections 2023-08-25T09:46:18.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.336+0000 7f4ed2ca6640 1 -- 172.21.15.70:0/1131686407 wait complete. 2023-08-25T09:46:18.356 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-08-25T09:46:17.468304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555285,"statfs":{"total":1600000000,"available":1592612402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.662923"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852515+0000","last_change":"2023-08-25T09:46:13.851390+0000","last_active":"2023-08-25T09:46:13.852515+0000","last_peered":"2023-08-25T09:46:13.852515+0000","last_clean":"2023-08-25T09:46:13.852515+0000","last_became_active":"2023-08-25T09:46:13.849187+0000","last_became_peered":"2023-08-25T09:46:13.849187+0000","last_unstale":"2023-08-25T09:46:13.852515+0000","last_undegraded":"2023-08-25T09:46:13.852515+0000","last_fullsized":"2023-08-25T09:46:13.852515+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:26.295460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854795+0000","last_change":"2023-08-25T09:46:13.854719+0000","last_active":"2023-08-25T09:46:13.854795+0000","last_peered":"2023-08-25T09:46:13.854795+0000","last_clean":"2023-08-25T09:46:13.854795+0000","last_became_active":"2023-08-25T09:46:13.854454+0000","last_became_peered":"2023-08-25T09:46:13.854454+0000","last_unstale":"2023-08-25T09:46:13.854795+0000","last_undegraded":"2023-08-25T09:46:13.854795+0000","last_fullsized":"2023-08-25T09:46:13.854795+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:49.662888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852531+0000","last_change":"2023-08-25T09:46:13.852310+0000","last_active":"2023-08-25T09:46:13.852531+0000","last_peered":"2023-08-25T09:46:13.852531+0000","last_clean":"2023-08-25T09:46:13.852531+0000","last_became_active":"2023-08-25T09:46:13.851416+0000","last_became_peered":"2023-08-25T09:46:13.851416+0000","last_unstale":"2023-08-25T09:46:13.852531+0000","last_undegraded":"2023-08-25T09:46:13.852531+0000","last_fullsized":"2023-08-25T09:46:13.852531+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:21.909471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837138+0000","last_change":"2023-08-25T09:46:13.836516+0000","last_active":"2023-08-25T09:46:13.837138+0000","last_peered":"2023-08-25T09:46:13.837138+0000","last_clean":"2023-08-25T09:46:13.837138+0000","last_became_active":"2023-08-25T09:46:13.835894+0000","last_became_peered":"2023-08-25T09:46:13.835894+0000","last_unstale":"2023-08-25T09:46:13.837138+0000","last_undegraded":"2023-08-25T09:46:13.837138+0000","last_fullsized":"2023-08-25T09:46:13.837138+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:35.225952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837083+0000","last_change":"2023-08-25T09:46:13.835218+0000","last_active":"2023-08-25T09:46:13.837083+0000","last_peered":"2023-08-25T09:46:13.837083+0000","last_clean":"2023-08-25T09:46:13.837083+0000","last_became_active":"2023-08-25T09:46:13.834503+0000","last_became_peered":"2023-08-25T09:46:13.834503+0000","last_unstale":"2023-08-25T09:46:13.837083+0000","last_undegraded":"2023-08-25T09:46:13.837083+0000","last_fullsized":"2023-08-25T09:46:13.837083+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:02.087906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858257+0000","last_change":"2023-08-25T09:46:13.858094+0000","last_active":"2023-08-25T09:46:13.858257+0000","last_peered":"2023-08-25T09:46:13.858257+0000","last_clean":"2023-08-25T09:46:13.858257+0000","last_became_active":"2023-08-25T09:46:13.857822+0000","last_became_peered":"2023-08-25T09:46:13.857822+0000","last_unstale":"2023-08-25T09:46:13.858257+0000","last_undegraded":"2023-08-25T09:46:13.858257+0000","last_fullsized":"2023-08-25T09:46:13.858257+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:14.209373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861577+0000","last_change":"2023-08-25T09:46:13.861529+0000","last_active":"2023-08-25T09:46:13.861577+0000","last_peered":"2023-08-25T09:46:13.861577+0000","last_clean":"2023-08-25T09:46:13.861577+0000","last_became_active":"2023-08-25T09:46:13.861015+0000","last_became_peered":"2023-08-25T09:46:13.861015+0000","last_unstale":"2023-08-25T09:46:13.861577+0000","last_undegraded":"2023-08-25T09:46:13.861577+0000","last_fullsized":"2023-08-25T09:46:13.861577+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:45:39.512662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860687+0000","last_change":"2023-08-25T09:46:13.860520+0000","last_active":"2023-08-25T09:46:13.860687+0000","last_peered":"2023-08-25T09:46:13.860687+0000","last_clean":"2023-08-25T09:46:13.860687+0000","last_became_active":"2023-08-25T09:46:13.859561+0000","last_became_peered":"2023-08-25T09:46:13.859561+0000","last_unstale":"2023-08-25T09:46:13.860687+0000","last_undegraded":"2023-08-25T09:46:13.860687+0000","last_fullsized":"2023-08-25T09:46:13.860687+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:11.935170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836135+0000","last_change":"2023-08-25T09:46:13.836031+0000","last_active":"2023-08-25T09:46:13.836135+0000","last_peered":"2023-08-25T09:46:13.836135+0000","last_clean":"2023-08-25T09:46:13.836135+0000","last_became_active":"2023-08-25T09:46:13.835741+0000","last_became_peered":"2023-08-25T09:46:13.835741+0000","last_unstale":"2023-08-25T09:46:13.836135+0000","last_undegraded":"2023-08-25T09:46:13.836135+0000","last_fullsized":"2023-08-25T09:46:13.836135+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:31:37.550284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.841682+0000","last_change":"2023-08-25T09:46:13.841414+0000","last_active":"2023-08-25T09:46:13.841682+0000","last_peered":"2023-08-25T09:46:13.841682+0000","last_clean":"2023-08-25T09:46:13.841682+0000","last_became_active":"2023-08-25T09:46:13.840612+0000","last_became_peered":"2023-08-25T09:46:13.840612+0000","last_unstale":"2023-08-25T09:46:13.841682+0000","last_undegraded":"2023-08-25T09:46:13.841682+0000","last_fullsized":"2023-08-25T09:46:13.841682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:47.969760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837422+0000","last_change":"2023-08-25T09:46:13.837329+0000","last_active":"2023-08-25T09:46:13.837422+0000","last_peered":"2023-08-25T09:46:13.837422+0000","last_clean":"2023-08-25T09:46:13.837422+0000","last_became_active":"2023-08-25T09:46:13.837038+0000","last_became_peered":"2023-08-25T09:46:13.837038+0000","last_unstale":"2023-08-25T09:46:13.837422+0000","last_undegraded":"2023-08-25T09:46:13.837422+0000","last_fullsized":"2023-08-25T09:46:13.837422+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:16.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.855844+0000","last_change":"2023-08-25T09:46:13.855298+0000","last_active":"2023-08-25T09:46:13.855844+0000","last_peered":"2023-08-25T09:46:13.855844+0000","last_clean":"2023-08-25T09:46:13.855844+0000","last_became_active":"2023-08-25T09:46:13.855025+0000","last_became_peered":"2023-08-25T09:46:13.855025+0000","last_unstale":"2023-08-25T09:46:13.855844+0000","last_undegraded":"2023-08-25T09:46:13.855844+0000","last_fullsized":"2023-08-25T09:46:13.855844+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:13.250629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853311+0000","last_change":"2023-08-25T09:46:13.852919+0000","last_active":"2023-08-25T09:46:13.853311+0000","last_peered":"2023-08-25T09:46:13.853311+0000","last_clean":"2023-08-25T09:46:13.853311+0000","last_became_active":"2023-08-25T09:46:13.852191+0000","last_became_peered":"2023-08-25T09:46:13.852191+0000","last_unstale":"2023-08-25T09:46:13.853311+0000","last_undegraded":"2023-08-25T09:46:13.853311+0000","last_fullsized":"2023-08-25T09:46:13.853311+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:46:33.084456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861004+0000","last_change":"2023-08-25T09:46:13.860886+0000","last_active":"2023-08-25T09:46:13.861004+0000","last_peered":"2023-08-25T09:46:13.861004+0000","last_clean":"2023-08-25T09:46:13.861004+0000","last_became_active":"2023-08-25T09:46:13.856276+0000","last_became_peered":"2023-08-25T09:46:13.856276+0000","last_unstale":"2023-08-25T09:46:13.861004+0000","last_undegraded":"2023-08-25T09:46:13.861004+0000","last_fullsized":"2023-08-25T09:46:13.861004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:35:05.042117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842504+0000","last_change":"2023-08-25T09:46:13.841857+0000","last_active":"2023-08-25T09:46:13.842504+0000","last_peered":"2023-08-25T09:46:13.842504+0000","last_clean":"2023-08-25T09:46:13.842504+0000","last_became_active":"2023-08-25T09:46:13.841316+0000","last_became_peered":"2023-08-25T09:46:13.841316+0000","last_unstale":"2023-08-25T09:46:13.842504+0000","last_undegraded":"2023-08-25T09:46:13.842504+0000","last_fullsized":"2023-08-25T09:46:13.842504+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:46:12.295419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.856792+0000","last_change":"2023-08-25T09:46:13.856705+0000","last_active":"2023-08-25T09:46:13.856792+0000","last_peered":"2023-08-25T09:46:13.856792+0000","last_clean":"2023-08-25T09:46:13.856792+0000","last_became_active":"2023-08-25T09:46:13.856195+0000","last_became_peered":"2023-08-25T09:46:13.856195+0000","last_unstale":"2023-08-25T09:46:13.856792+0000","last_undegraded":"2023-08-25T09:46:13.856792+0000","last_fullsized":"2023-08-25T09:46:13.856792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:10.404156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854392+0000","last_change":"2023-08-25T09:46:13.853493+0000","last_active":"2023-08-25T09:46:13.854392+0000","last_peered":"2023-08-25T09:46:13.854392+0000","last_clean":"2023-08-25T09:46:13.854392+0000","last_became_active":"2023-08-25T09:46:13.853149+0000","last_became_peered":"2023-08-25T09:46:13.853149+0000","last_unstale":"2023-08-25T09:46:13.854392+0000","last_undegraded":"2023-08-25T09:46:13.854392+0000","last_fullsized":"2023-08-25T09:46:13.854392+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:43.082659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836016+0000","last_change":"2023-08-25T09:46:13.835272+0000","last_active":"2023-08-25T09:46:13.836016+0000","last_peered":"2023-08-25T09:46:13.836016+0000","last_clean":"2023-08-25T09:46:13.836016+0000","last_became_active":"2023-08-25T09:46:13.834977+0000","last_became_peered":"2023-08-25T09:46:13.834977+0000","last_unstale":"2023-08-25T09:46:13.836016+0000","last_undegraded":"2023-08-25T09:46:13.836016+0000","last_fullsized":"2023-08-25T09:46:13.836016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:59:53.950833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862636+0000","last_change":"2023-08-25T09:46:13.862068+0000","last_active":"2023-08-25T09:46:13.862636+0000","last_peered":"2023-08-25T09:46:13.862636+0000","last_clean":"2023-08-25T09:46:13.862636+0000","last_became_active":"2023-08-25T09:46:13.859588+0000","last_became_peered":"2023-08-25T09:46:13.859588+0000","last_unstale":"2023-08-25T09:46:13.862636+0000","last_undegraded":"2023-08-25T09:46:13.862636+0000","last_fullsized":"2023-08-25T09:46:13.862636+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:06:02.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845157+0000","last_change":"2023-08-25T09:46:13.844903+0000","last_active":"2023-08-25T09:46:13.845157+0000","last_peered":"2023-08-25T09:46:13.845157+0000","last_clean":"2023-08-25T09:46:13.845157+0000","last_became_active":"2023-08-25T09:46:13.844627+0000","last_became_peered":"2023-08-25T09:46:13.844627+0000","last_unstale":"2023-08-25T09:46:13.845157+0000","last_undegraded":"2023-08-25T09:46:13.845157+0000","last_fullsized":"2023-08-25T09:46:13.845157+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:17:15.261803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845669+0000","last_change":"2023-08-25T09:46:13.845521+0000","last_active":"2023-08-25T09:46:13.845669+0000","last_peered":"2023-08-25T09:46:13.845669+0000","last_clean":"2023-08-25T09:46:13.845669+0000","last_became_active":"2023-08-25T09:46:13.845045+0000","last_became_peered":"2023-08-25T09:46:13.845045+0000","last_unstale":"2023-08-25T09:46:13.845669+0000","last_undegraded":"2023-08-25T09:46:13.845669+0000","last_fullsized":"2023-08-25T09:46:13.845669+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:18.897684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860826+0000","last_change":"2023-08-25T09:46:13.860719+0000","last_active":"2023-08-25T09:46:13.860826+0000","last_peered":"2023-08-25T09:46:13.860826+0000","last_clean":"2023-08-25T09:46:13.860826+0000","last_became_active":"2023-08-25T09:46:13.853489+0000","last_became_peered":"2023-08-25T09:46:13.853489+0000","last_unstale":"2023-08-25T09:46:13.860826+0000","last_undegraded":"2023-08-25T09:46:13.860826+0000","last_fullsized":"2023-08-25T09:46:13.860826+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:15:54.817525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845590+0000","last_change":"2023-08-25T09:46:13.845316+0000","last_active":"2023-08-25T09:46:13.845590+0000","last_peered":"2023-08-25T09:46:13.845590+0000","last_clean":"2023-08-25T09:46:13.845590+0000","last_became_active":"2023-08-25T09:46:13.843482+0000","last_became_peered":"2023-08-25T09:46:13.843482+0000","last_unstale":"2023-08-25T09:46:13.845590+0000","last_undegraded":"2023-08-25T09:46:13.845590+0000","last_fullsized":"2023-08-25T09:46:13.845590+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:26.953645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842416+0000","last_change":"2023-08-25T09:46:13.842254+0000","last_active":"2023-08-25T09:46:13.842416+0000","last_peered":"2023-08-25T09:46:13.842416+0000","last_clean":"2023-08-25T09:46:13.842416+0000","last_became_active":"2023-08-25T09:46:13.841795+0000","last_became_peered":"2023-08-25T09:46:13.841795+0000","last_unstale":"2023-08-25T09:46:13.842416+0000","last_undegraded":"2023-08-25T09:46:13.842416+0000","last_fullsized":"2023-08-25T09:46:13.842416+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:33.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.841784+0000","last_change":"2023-08-25T09:46:13.841555+0000","last_active":"2023-08-25T09:46:13.841784+0000","last_peered":"2023-08-25T09:46:13.841784+0000","last_clean":"2023-08-25T09:46:13.841784+0000","last_became_active":"2023-08-25T09:46:13.841262+0000","last_became_peered":"2023-08-25T09:46:13.841262+0000","last_unstale":"2023-08-25T09:46:13.841784+0000","last_undegraded":"2023-08-25T09:46:13.841784+0000","last_fullsized":"2023-08-25T09:46:13.841784+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:54:34.882076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.844071+0000","last_change":"2023-08-25T09:46:13.843898+0000","last_active":"2023-08-25T09:46:13.844071+0000","last_peered":"2023-08-25T09:46:13.844071+0000","last_clean":"2023-08-25T09:46:13.844071+0000","last_became_active":"2023-08-25T09:46:13.843646+0000","last_became_peered":"2023-08-25T09:46:13.843646+0000","last_unstale":"2023-08-25T09:46:13.844071+0000","last_undegraded":"2023-08-25T09:46:13.844071+0000","last_fullsized":"2023-08-25T09:46:13.844071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:24:31.987367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.841967+0000","last_change":"2023-08-25T09:46:13.841873+0000","last_active":"2023-08-25T09:46:13.841967+0000","last_peered":"2023-08-25T09:46:13.841967+0000","last_clean":"2023-08-25T09:46:13.841967+0000","last_became_active":"2023-08-25T09:46:13.841298+0000","last_became_peered":"2023-08-25T09:46:13.841298+0000","last_unstale":"2023-08-25T09:46:13.841967+0000","last_undegraded":"2023-08-25T09:46:13.841967+0000","last_fullsized":"2023-08-25T09:46:13.841967+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:56.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853597+0000","last_change":"2023-08-25T09:46:13.853496+0000","last_active":"2023-08-25T09:46:13.853597+0000","last_peered":"2023-08-25T09:46:13.853597+0000","last_clean":"2023-08-25T09:46:13.853597+0000","last_became_active":"2023-08-25T09:46:13.853245+0000","last_became_peered":"2023-08-25T09:46:13.853245+0000","last_unstale":"2023-08-25T09:46:13.853597+0000","last_undegraded":"2023-08-25T09:46:13.853597+0000","last_fullsized":"2023-08-25T09:46:13.853597+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:33.068910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854702+0000","last_change":"2023-08-25T09:46:13.854531+0000","last_active":"2023-08-25T09:46:13.854702+0000","last_peered":"2023-08-25T09:46:13.854702+0000","last_clean":"2023-08-25T09:46:13.854702+0000","last_became_active":"2023-08-25T09:46:13.853351+0000","last_became_peered":"2023-08-25T09:46:13.853351+0000","last_unstale":"2023-08-25T09:46:13.854702+0000","last_undegraded":"2023-08-25T09:46:13.854702+0000","last_fullsized":"2023-08-25T09:46:13.854702+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:10.427818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.840288+0000","last_change":"2023-08-25T09:46:13.840029+0000","last_active":"2023-08-25T09:46:13.840288+0000","last_peered":"2023-08-25T09:46:13.840288+0000","last_clean":"2023-08-25T09:46:13.840288+0000","last_became_active":"2023-08-25T09:46:13.839139+0000","last_became_peered":"2023-08-25T09:46:13.839139+0000","last_unstale":"2023-08-25T09:46:13.840288+0000","last_undegraded":"2023-08-25T09:46:13.840288+0000","last_fullsized":"2023-08-25T09:46:13.840288+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:14.126703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854468+0000","last_change":"2023-08-25T09:46:13.854407+0000","last_active":"2023-08-25T09:46:13.854468+0000","last_peered":"2023-08-25T09:46:13.854468+0000","last_clean":"2023-08-25T09:46:13.854468+0000","last_became_active":"2023-08-25T09:46:13.854123+0000","last_became_peered":"2023-08-25T09:46:13.854123+0000","last_unstale":"2023-08-25T09:46:13.854468+0000","last_undegraded":"2023-08-25T09:46:13.854468+0000","last_fullsized":"2023-08-25T09:46:13.854468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:14.437683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.848739+0000","last_change":"2023-08-25T09:46:13.848242+0000","last_active":"2023-08-25T09:46:13.848739+0000","last_peered":"2023-08-25T09:46:13.848739+0000","last_clean":"2023-08-25T09:46:13.848739+0000","last_became_active":"2023-08-25T09:46:13.847826+0000","last_became_peered":"2023-08-25T09:46:13.847826+0000","last_unstale":"2023-08-25T09:46:13.848739+0000","last_undegraded":"2023-08-25T09:46:13.848739+0000","last_fullsized":"2023-08-25T09:46:13.848739+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:37:08.773278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854415+0000","last_change":"2023-08-25T09:46:13.854048+0000","last_active":"2023-08-25T09:46:13.854415+0000","last_peered":"2023-08-25T09:46:13.854415+0000","last_clean":"2023-08-25T09:46:13.854415+0000","last_became_active":"2023-08-25T09:46:13.853308+0000","last_became_peered":"2023-08-25T09:46:13.853308+0000","last_unstale":"2023-08-25T09:46:13.854415+0000","last_undegraded":"2023-08-25T09:46:13.854415+0000","last_fullsized":"2023-08-25T09:46:13.854415+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:06.285281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"132'4","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858589+0000","last_change":"2023-08-25T09:46:12.837594+0000","last_active":"2023-08-25T09:46:13.858589+0000","last_peered":"2023-08-25T09:46:13.858589+0000","last_clean":"2023-08-25T09:46:13.858589+0000","last_became_active":"2023-08-25T09:46:12.837159+0000","last_became_peered":"2023-08-25T09:46:12.837159+0000","last_unstale":"2023-08-25T09:46:13.858589+0000","last_undegraded":"2023-08-25T09:46:13.858589+0000","last_fullsized":"2023-08-25T09:46:13.858589+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:35.672374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.846063+0000","last_change":"2023-08-25T09:46:13.845952+0000","last_active":"2023-08-25T09:46:13.846063+0000","last_peered":"2023-08-25T09:46:13.846063+0000","last_clean":"2023-08-25T09:46:13.846063+0000","last_became_active":"2023-08-25T09:46:13.844786+0000","last_became_peered":"2023-08-25T09:46:13.844786+0000","last_unstale":"2023-08-25T09:46:13.846063+0000","last_undegraded":"2023-08-25T09:46:13.846063+0000","last_fullsized":"2023-08-25T09:46:13.846063+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:00:59.929787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862339+0000","last_change":"2023-08-25T09:46:12.844400+0000","last_active":"2023-08-25T09:46:13.862339+0000","last_peered":"2023-08-25T09:46:13.862339+0000","last_clean":"2023-08-25T09:46:13.862339+0000","last_became_active":"2023-08-25T09:46:12.842665+0000","last_became_peered":"2023-08-25T09:46:12.842665+0000","last_unstale":"2023-08-25T09:46:13.862339+0000","last_undegraded":"2023-08-25T09:46:13.862339+0000","last_fullsized":"2023-08-25T09:46:13.862339+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:25.432915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.839926+0000","last_change":"2023-08-25T09:46:13.839824+0000","last_active":"2023-08-25T09:46:13.839926+0000","last_peered":"2023-08-25T09:46:13.839926+0000","last_clean":"2023-08-25T09:46:13.839926+0000","last_became_active":"2023-08-25T09:46:13.839537+0000","last_became_peered":"2023-08-25T09:46:13.839537+0000","last_unstale":"2023-08-25T09:46:13.839926+0000","last_undegraded":"2023-08-25T09:46:13.839926+0000","last_fullsized":"2023-08-25T09:46:13.839926+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:21.793789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860568+0000","last_change":"2023-08-25T09:46:12.842539+0000","last_active":"2023-08-25T09:46:13.860568+0000","last_peered":"2023-08-25T09:46:13.860568+0000","last_clean":"2023-08-25T09:46:13.860568+0000","last_became_active":"2023-08-25T09:46:12.842300+0000","last_became_peered":"2023-08-25T09:46:12.842300+0000","last_unstale":"2023-08-25T09:46:13.860568+0000","last_undegraded":"2023-08-25T09:46:13.860568+0000","last_fullsized":"2023-08-25T09:46:13.860568+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:12:44.732933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853424+0000","last_change":"2023-08-25T09:46:13.853308+0000","last_active":"2023-08-25T09:46:13.853424+0000","last_peered":"2023-08-25T09:46:13.853424+0000","last_clean":"2023-08-25T09:46:13.853424+0000","last_became_active":"2023-08-25T09:46:13.853058+0000","last_became_peered":"2023-08-25T09:46:13.853058+0000","last_unstale":"2023-08-25T09:46:13.853424+0000","last_undegraded":"2023-08-25T09:46:13.853424+0000","last_fullsized":"2023-08-25T09:46:13.853424+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:28:52.961461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861676+0000","last_change":"2023-08-25T09:46:12.845899+0000","last_active":"2023-08-25T09:46:13.861676+0000","last_peered":"2023-08-25T09:46:13.861676+0000","last_clean":"2023-08-25T09:46:13.861676+0000","last_became_active":"2023-08-25T09:46:12.844324+0000","last_became_peered":"2023-08-25T09:46:12.844324+0000","last_unstale":"2023-08-25T09:46:13.861676+0000","last_undegraded":"2023-08-25T09:46:13.861676+0000","last_fullsized":"2023-08-25T09:46:13.861676+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:51.985346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861509+0000","last_change":"2023-08-25T09:46:13.860977+0000","last_active":"2023-08-25T09:46:13.861509+0000","last_peered":"2023-08-25T09:46:13.861509+0000","last_clean":"2023-08-25T09:46:13.861509+0000","last_became_active":"2023-08-25T09:46:13.859736+0000","last_became_peered":"2023-08-25T09:46:13.859736+0000","last_unstale":"2023-08-25T09:46:13.861509+0000","last_undegraded":"2023-08-25T09:46:13.861509+0000","last_fullsized":"2023-08-25T09:46:13.861509+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:27:09.256131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.857628+0000","last_change":"2023-08-25T09:46:12.840203+0000","last_active":"2023-08-25T09:46:13.857628+0000","last_peered":"2023-08-25T09:46:13.857628+0000","last_clean":"2023-08-25T09:46:13.857628+0000","last_became_active":"2023-08-25T09:46:12.839865+0000","last_became_peered":"2023-08-25T09:46:12.839865+0000","last_unstale":"2023-08-25T09:46:13.857628+0000","last_undegraded":"2023-08-25T09:46:13.857628+0000","last_fullsized":"2023-08-25T09:46:13.857628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:49:30.054349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860451+0000","last_change":"2023-08-25T09:46:13.860372+0000","last_active":"2023-08-25T09:46:13.860451+0000","last_peered":"2023-08-25T09:46:13.860451+0000","last_clean":"2023-08-25T09:46:13.860451+0000","last_became_active":"2023-08-25T09:46:13.856515+0000","last_became_peered":"2023-08-25T09:46:13.856515+0000","last_unstale":"2023-08-25T09:46:13.860451+0000","last_undegraded":"2023-08-25T09:46:13.860451+0000","last_fullsized":"2023-08-25T09:46:13.860451+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:58.562158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858853+0000","last_change":"2023-08-25T09:46:12.842406+0000","last_active":"2023-08-25T09:46:13.858853+0000","last_peered":"2023-08-25T09:46:13.858853+0000","last_clean":"2023-08-25T09:46:13.858853+0000","last_became_active":"2023-08-25T09:46:12.842102+0000","last_became_peered":"2023-08-25T09:46:12.842102+0000","last_unstale":"2023-08-25T09:46:13.858853+0000","last_undegraded":"2023-08-25T09:46:13.858853+0000","last_fullsized":"2023-08-25T09:46:13.858853+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:27:26.027427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842391+0000","last_change":"2023-08-25T09:46:13.841967+0000","last_active":"2023-08-25T09:46:13.842391+0000","last_peered":"2023-08-25T09:46:13.842391+0000","last_clean":"2023-08-25T09:46:13.842391+0000","last_became_active":"2023-08-25T09:46:13.841735+0000","last_became_peered":"2023-08-25T09:46:13.841735+0000","last_unstale":"2023-08-25T09:46:13.842391+0000","last_undegraded":"2023-08-25T09:46:13.842391+0000","last_fullsized":"2023-08-25T09:46:13.842391+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:28.200630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.838120+0000","last_change":"2023-08-25T09:46:12.844774+0000","last_active":"2023-08-25T09:46:13.838120+0000","last_peered":"2023-08-25T09:46:13.838120+0000","last_clean":"2023-08-25T09:46:13.838120+0000","last_became_active":"2023-08-25T09:46:12.844487+0000","last_became_peered":"2023-08-25T09:46:12.844487+0000","last_unstale":"2023-08-25T09:46:13.838120+0000","last_undegraded":"2023-08-25T09:46:13.838120+0000","last_fullsized":"2023-08-25T09:46:13.838120+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:54:35.314750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852488+0000","last_change":"2023-08-25T09:46:13.850782+0000","last_active":"2023-08-25T09:46:13.852488+0000","last_peered":"2023-08-25T09:46:13.852488+0000","last_clean":"2023-08-25T09:46:13.852488+0000","last_became_active":"2023-08-25T09:46:13.850179+0000","last_became_peered":"2023-08-25T09:46:13.850179+0000","last_unstale":"2023-08-25T09:46:13.852488+0000","last_undegraded":"2023-08-25T09:46:13.852488+0000","last_fullsized":"2023-08-25T09:46:13.852488+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:00:32.388159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.846809+0000","last_change":"2023-08-25T09:46:12.848337+0000","last_active":"2023-08-25T09:46:13.846809+0000","last_peered":"2023-08-25T09:46:13.846809+0000","last_clean":"2023-08-25T09:46:13.846809+0000","last_became_active":"2023-08-25T09:46:12.847749+0000","last_became_peered":"2023-08-25T09:46:12.847749+0000","last_unstale":"2023-08-25T09:46:13.846809+0000","last_undegraded":"2023-08-25T09:46:13.846809+0000","last_fullsized":"2023-08-25T09:46:13.846809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:56.362629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852539+0000","last_change":"2023-08-25T09:46:13.851570+0000","last_active":"2023-08-25T09:46:13.852539+0000","last_peered":"2023-08-25T09:46:13.852539+0000","last_clean":"2023-08-25T09:46:13.852539+0000","last_became_active":"2023-08-25T09:46:13.850670+0000","last_became_peered":"2023-08-25T09:46:13.850670+0000","last_unstale":"2023-08-25T09:46:13.852539+0000","last_undegraded":"2023-08-25T09:46:13.852539+0000","last_fullsized":"2023-08-25T09:46:13.852539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:01:50.987872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.847838+0000","last_change":"2023-08-25T09:46:12.837637+0000","last_active":"2023-08-25T09:46:13.847838+0000","last_peered":"2023-08-25T09:46:13.847838+0000","last_clean":"2023-08-25T09:46:13.847838+0000","last_became_active":"2023-08-25T09:46:12.837142+0000","last_became_peered":"2023-08-25T09:46:12.837142+0000","last_unstale":"2023-08-25T09:46:13.847838+0000","last_undegraded":"2023-08-25T09:46:13.847838+0000","last_fullsized":"2023-08-25T09:46:13.847838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:48.099737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860332+0000","last_change":"2023-08-25T09:46:13.860023+0000","last_active":"2023-08-25T09:46:13.860332+0000","last_peered":"2023-08-25T09:46:13.860332+0000","last_clean":"2023-08-25T09:46:13.860332+0000","last_became_active":"2023-08-25T09:46:13.849300+0000","last_became_peered":"2023-08-25T09:46:13.849300+0000","last_unstale":"2023-08-25T09:46:13.860332+0000","last_undegraded":"2023-08-25T09:46:13.860332+0000","last_fullsized":"2023-08-25T09:46:13.860332+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:04.821820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.835958+0000","last_change":"2023-08-25T09:46:12.847429+0000","last_active":"2023-08-25T09:46:13.835958+0000","last_peered":"2023-08-25T09:46:13.835958+0000","last_clean":"2023-08-25T09:46:13.835958+0000","last_became_active":"2023-08-25T09:46:12.846940+0000","last_became_peered":"2023-08-25T09:46:12.846940+0000","last_unstale":"2023-08-25T09:46:13.835958+0000","last_undegraded":"2023-08-25T09:46:13.835958+0000","last_fullsized":"2023-08-25T09:46:13.835958+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:26.779363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860666+0000","last_change":"2023-08-25T09:46:13.860500+0000","last_active":"2023-08-25T09:46:13.860666+0000","last_peered":"2023-08-25T09:46:13.860666+0000","last_clean":"2023-08-25T09:46:13.860666+0000","last_became_active":"2023-08-25T09:46:13.860258+0000","last_became_peered":"2023-08-25T09:46:13.860258+0000","last_unstale":"2023-08-25T09:46:13.860666+0000","last_undegraded":"2023-08-25T09:46:13.860666+0000","last_fullsized":"2023-08-25T09:46:13.860666+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:52:34.877190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"132'3","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858266+0000","last_change":"2023-08-25T09:46:12.828008+0000","last_active":"2023-08-25T09:46:13.858266+0000","last_peered":"2023-08-25T09:46:13.858266+0000","last_clean":"2023-08-25T09:46:13.858266+0000","last_became_active":"2023-08-25T09:46:12.827587+0000","last_became_peered":"2023-08-25T09:46:12.827587+0000","last_unstale":"2023-08-25T09:46:13.858266+0000","last_undegraded":"2023-08-25T09:46:13.858266+0000","last_fullsized":"2023-08-25T09:46:13.858266+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:37:00.505165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854933+0000","last_change":"2023-08-25T09:46:13.854489+0000","last_active":"2023-08-25T09:46:13.854933+0000","last_peered":"2023-08-25T09:46:13.854933+0000","last_clean":"2023-08-25T09:46:13.854933+0000","last_became_active":"2023-08-25T09:46:13.854221+0000","last_became_peered":"2023-08-25T09:46:13.854221+0000","last_unstale":"2023-08-25T09:46:13.854933+0000","last_undegraded":"2023-08-25T09:46:13.854933+0000","last_fullsized":"2023-08-25T09:46:13.854933+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:31:53.955946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"132'6","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862688+0000","last_change":"2023-08-25T09:46:12.845025+0000","last_active":"2023-08-25T09:46:13.862688+0000","last_peered":"2023-08-25T09:46:13.862688+0000","last_clean":"2023-08-25T09:46:13.862688+0000","last_became_active":"2023-08-25T09:46:12.844573+0000","last_became_peered":"2023-08-25T09:46:12.844573+0000","last_unstale":"2023-08-25T09:46:13.862688+0000","last_undegraded":"2023-08-25T09:46:13.862688+0000","last_fullsized":"2023-08-25T09:46:13.862688+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:29:34.685465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836091+0000","last_change":"2023-08-25T09:46:13.835776+0000","last_active":"2023-08-25T09:46:13.836091+0000","last_peered":"2023-08-25T09:46:13.836091+0000","last_clean":"2023-08-25T09:46:13.836091+0000","last_became_active":"2023-08-25T09:46:13.835514+0000","last_became_peered":"2023-08-25T09:46:13.835514+0000","last_unstale":"2023-08-25T09:46:13.836091+0000","last_undegraded":"2023-08-25T09:46:13.836091+0000","last_fullsized":"2023-08-25T09:46:13.836091+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:05.190445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.865675+0000","last_change":"2023-08-25T09:46:12.850998+0000","last_active":"2023-08-25T09:46:13.865675+0000","last_peered":"2023-08-25T09:46:13.865675+0000","last_clean":"2023-08-25T09:46:13.865675+0000","last_became_active":"2023-08-25T09:46:12.848344+0000","last_became_peered":"2023-08-25T09:46:12.848344+0000","last_unstale":"2023-08-25T09:46:13.865675+0000","last_undegraded":"2023-08-25T09:46:13.865675+0000","last_fullsized":"2023-08-25T09:46:13.865675+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:13:33.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.843751+0000","last_change":"2023-08-25T09:46:13.843518+0000","last_active":"2023-08-25T09:46:13.843751+0000","last_peered":"2023-08-25T09:46:13.843751+0000","last_clean":"2023-08-25T09:46:13.843751+0000","last_became_active":"2023-08-25T09:46:13.843246+0000","last_became_peered":"2023-08-25T09:46:13.843246+0000","last_unstale":"2023-08-25T09:46:13.843751+0000","last_undegraded":"2023-08-25T09:46:13.843751+0000","last_fullsized":"2023-08-25T09:46:13.843751+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:39.875204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836274+0000","last_change":"2023-08-25T09:46:12.846154+0000","last_active":"2023-08-25T09:46:13.836274+0000","last_peered":"2023-08-25T09:46:13.836274+0000","last_clean":"2023-08-25T09:46:13.836274+0000","last_became_active":"2023-08-25T09:46:12.844908+0000","last_became_peered":"2023-08-25T09:46:12.844908+0000","last_unstale":"2023-08-25T09:46:13.836274+0000","last_undegraded":"2023-08-25T09:46:13.836274+0000","last_fullsized":"2023-08-25T09:46:13.836274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:21:31.715559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860203+0000","last_change":"2023-08-25T09:46:13.859633+0000","last_active":"2023-08-25T09:46:13.860203+0000","last_peered":"2023-08-25T09:46:13.860203+0000","last_clean":"2023-08-25T09:46:13.860203+0000","last_became_active":"2023-08-25T09:46:13.859231+0000","last_became_peered":"2023-08-25T09:46:13.859231+0000","last_unstale":"2023-08-25T09:46:13.860203+0000","last_undegraded":"2023-08-25T09:46:13.860203+0000","last_fullsized":"2023-08-25T09:46:13.860203+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:43.082659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.827940+0000","last_change":"2023-08-25T09:46:12.846602+0000","last_active":"2023-08-25T09:46:13.827940+0000","last_peered":"2023-08-25T09:46:13.827940+0000","last_clean":"2023-08-25T09:46:13.827940+0000","last_became_active":"2023-08-25T09:46:12.846000+0000","last_became_peered":"2023-08-25T09:46:12.846000+0000","last_unstale":"2023-08-25T09:46:13.827940+0000","last_undegraded":"2023-08-25T09:46:13.827940+0000","last_fullsized":"2023-08-25T09:46:13.827940+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:50:05.747460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854435+0000","last_change":"2023-08-25T09:46:13.854222+0000","last_active":"2023-08-25T09:46:13.854435+0000","last_peered":"2023-08-25T09:46:13.854435+0000","last_clean":"2023-08-25T09:46:13.854435+0000","last_became_active":"2023-08-25T09:46:13.853725+0000","last_became_peered":"2023-08-25T09:46:13.853725+0000","last_unstale":"2023-08-25T09:46:13.854435+0000","last_undegraded":"2023-08-25T09:46:13.854435+0000","last_fullsized":"2023-08-25T09:46:13.854435+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:45:06.643619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"132'8","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.855899+0000","last_change":"2023-08-25T09:46:12.841852+0000","last_active":"2023-08-25T09:46:13.855899+0000","last_peered":"2023-08-25T09:46:13.855899+0000","last_clean":"2023-08-25T09:46:13.855899+0000","last_became_active":"2023-08-25T09:46:12.841528+0000","last_became_peered":"2023-08-25T09:46:12.841528+0000","last_unstale":"2023-08-25T09:46:13.855899+0000","last_undegraded":"2023-08-25T09:46:13.855899+0000","last_fullsized":"2023-08-25T09:46:13.855899+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:10:00.933841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.851100+0000","last_change":"2023-08-25T09:46:13.850722+0000","last_active":"2023-08-25T09:46:13.851100+0000","last_peered":"2023-08-25T09:46:13.851100+0000","last_clean":"2023-08-25T09:46:13.851100+0000","last_became_active":"2023-08-25T09:46:13.850323+0000","last_became_peered":"2023-08-25T09:46:13.850323+0000","last_unstale":"2023-08-25T09:46:13.851100+0000","last_undegraded":"2023-08-25T09:46:13.851100+0000","last_fullsized":"2023-08-25T09:46:13.851100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:21:42.100149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861243+0000","last_change":"2023-08-25T09:46:12.845881+0000","last_active":"2023-08-25T09:46:13.861243+0000","last_peered":"2023-08-25T09:46:13.861243+0000","last_clean":"2023-08-25T09:46:13.861243+0000","last_became_active":"2023-08-25T09:46:12.845604+0000","last_became_peered":"2023-08-25T09:46:12.845604+0000","last_unstale":"2023-08-25T09:46:13.861243+0000","last_undegraded":"2023-08-25T09:46:13.861243+0000","last_fullsized":"2023-08-25T09:46:13.861243+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:56.362629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845957+0000","last_change":"2023-08-25T09:46:13.845840+0000","last_active":"2023-08-25T09:46:13.845957+0000","last_peered":"2023-08-25T09:46:13.845957+0000","last_clean":"2023-08-25T09:46:13.845957+0000","last_became_active":"2023-08-25T09:46:13.845608+0000","last_became_peered":"2023-08-25T09:46:13.845608+0000","last_unstale":"2023-08-25T09:46:13.845957+0000","last_undegraded":"2023-08-25T09:46:13.845957+0000","last_fullsized":"2023-08-25T09:46:13.845957+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:29:09.902573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.834911+0000","last_change":"2023-08-25T09:46:12.841006+0000","last_active":"2023-08-25T09:46:13.834911+0000","last_peered":"2023-08-25T09:46:13.834911+0000","last_clean":"2023-08-25T09:46:13.834911+0000","last_became_active":"2023-08-25T09:46:12.833094+0000","last_became_peered":"2023-08-25T09:46:12.833094+0000","last_unstale":"2023-08-25T09:46:13.834911+0000","last_undegraded":"2023-08-25T09:46:13.834911+0000","last_fullsized":"2023-08-25T09:46:13.834911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:35.672374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860862+0000","last_change":"2023-08-25T09:46:13.860715+0000","last_active":"2023-08-25T09:46:13.860862+0000","last_peered":"2023-08-25T09:46:13.860862+0000","last_clean":"2023-08-25T09:46:13.860862+0000","last_became_active":"2023-08-25T09:46:13.860124+0000","last_became_peered":"2023-08-25T09:46:13.860124+0000","last_unstale":"2023-08-25T09:46:13.860862+0000","last_undegraded":"2023-08-25T09:46:13.860862+0000","last_fullsized":"2023-08-25T09:46:13.860862+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:48.476261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833317+0000","last_change":"2023-08-25T09:46:12.841401+0000","last_active":"2023-08-25T09:46:13.833317+0000","last_peered":"2023-08-25T09:46:13.833317+0000","last_clean":"2023-08-25T09:46:13.833317+0000","last_became_active":"2023-08-25T09:46:12.840778+0000","last_became_peered":"2023-08-25T09:46:12.840778+0000","last_unstale":"2023-08-25T09:46:13.833317+0000","last_undegraded":"2023-08-25T09:46:13.833317+0000","last_fullsized":"2023-08-25T09:46:13.833317+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:48.141548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842250+0000","last_change":"2023-08-25T09:46:13.842044+0000","last_active":"2023-08-25T09:46:13.842250+0000","last_peered":"2023-08-25T09:46:13.842250+0000","last_clean":"2023-08-25T09:46:13.842250+0000","last_became_active":"2023-08-25T09:46:13.841629+0000","last_became_peered":"2023-08-25T09:46:13.841629+0000","last_unstale":"2023-08-25T09:46:13.842250+0000","last_undegraded":"2023-08-25T09:46:13.842250+0000","last_fullsized":"2023-08-25T09:46:13.842250+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:40.787728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833774+0000","last_change":"2023-08-25T09:46:12.851464+0000","last_active":"2023-08-25T09:46:13.833774+0000","last_peered":"2023-08-25T09:46:13.833774+0000","last_clean":"2023-08-25T09:46:13.833774+0000","last_became_active":"2023-08-25T09:46:12.850409+0000","last_became_peered":"2023-08-25T09:46:12.850409+0000","last_unstale":"2023-08-25T09:46:13.833774+0000","last_undegraded":"2023-08-25T09:46:13.833774+0000","last_fullsized":"2023-08-25T09:46:13.833774+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:59:08.643583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.855752+0000","last_change":"2023-08-25T09:46:13.855661+0000","last_active":"2023-08-25T09:46:13.855752+0000","last_peered":"2023-08-25T09:46:13.855752+0000","last_clean":"2023-08-25T09:46:13.855752+0000","last_became_active":"2023-08-25T09:46:13.855350+0000","last_became_peered":"2023-08-25T09:46:13.855350+0000","last_unstale":"2023-08-25T09:46:13.855752+0000","last_undegraded":"2023-08-25T09:46:13.855752+0000","last_fullsized":"2023-08-25T09:46:13.855752+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:47.738941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.834084+0000","last_change":"2023-08-25T09:46:12.849108+0000","last_active":"2023-08-25T09:46:13.834084+0000","last_peered":"2023-08-25T09:46:13.834084+0000","last_clean":"2023-08-25T09:46:13.834084+0000","last_became_active":"2023-08-25T09:46:12.848826+0000","last_became_peered":"2023-08-25T09:46:12.848826+0000","last_unstale":"2023-08-25T09:46:13.834084+0000","last_undegraded":"2023-08-25T09:46:13.834084+0000","last_fullsized":"2023-08-25T09:46:13.834084+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:53:28.856588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862527+0000","last_change":"2023-08-25T09:46:13.862062+0000","last_active":"2023-08-25T09:46:13.862527+0000","last_peered":"2023-08-25T09:46:13.862527+0000","last_clean":"2023-08-25T09:46:13.862527+0000","last_became_active":"2023-08-25T09:46:13.860672+0000","last_became_peered":"2023-08-25T09:46:13.860672+0000","last_unstale":"2023-08-25T09:46:13.862527+0000","last_undegraded":"2023-08-25T09:46:13.862527+0000","last_fullsized":"2023-08-25T09:46:13.862527+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:35.329934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837155+0000","last_change":"2023-08-25T09:46:12.839898+0000","last_active":"2023-08-25T09:46:13.837155+0000","last_peered":"2023-08-25T09:46:13.837155+0000","last_clean":"2023-08-25T09:46:13.837155+0000","last_became_active":"2023-08-25T09:46:12.834307+0000","last_became_peered":"2023-08-25T09:46:12.834307+0000","last_unstale":"2023-08-25T09:46:13.837155+0000","last_undegraded":"2023-08-25T09:46:13.837155+0000","last_fullsized":"2023-08-25T09:46:13.837155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:39:02.892983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852078+0000","last_change":"2023-08-25T09:46:13.851428+0000","last_active":"2023-08-25T09:46:13.852078+0000","last_peered":"2023-08-25T09:46:13.852078+0000","last_clean":"2023-08-25T09:46:13.852078+0000","last_became_active":"2023-08-25T09:46:13.850982+0000","last_became_peered":"2023-08-25T09:46:13.850982+0000","last_unstale":"2023-08-25T09:46:13.852078+0000","last_undegraded":"2023-08-25T09:46:13.852078+0000","last_fullsized":"2023-08-25T09:46:13.852078+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:13.085449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.839768+0000","last_change":"2023-08-25T09:46:12.848152+0000","last_active":"2023-08-25T09:46:13.839768+0000","last_peered":"2023-08-25T09:46:13.839768+0000","last_clean":"2023-08-25T09:46:13.839768+0000","last_became_active":"2023-08-25T09:46:12.847883+0000","last_became_peered":"2023-08-25T09:46:12.847883+0000","last_unstale":"2023-08-25T09:46:13.839768+0000","last_undegraded":"2023-08-25T09:46:13.839768+0000","last_fullsized":"2023-08-25T09:46:13.839768+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:14:39.383791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853263+0000","last_change":"2023-08-25T09:46:13.853177+0000","last_active":"2023-08-25T09:46:13.853263+0000","last_peered":"2023-08-25T09:46:13.853263+0000","last_clean":"2023-08-25T09:46:13.853263+0000","last_became_active":"2023-08-25T09:46:13.852940+0000","last_became_peered":"2023-08-25T09:46:13.852940+0000","last_unstale":"2023-08-25T09:46:13.853263+0000","last_undegraded":"2023-08-25T09:46:13.853263+0000","last_fullsized":"2023-08-25T09:46:13.853263+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:41:13.253957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.838303+0000","last_change":"2023-08-25T09:46:12.840878+0000","last_active":"2023-08-25T09:46:13.838303+0000","last_peered":"2023-08-25T09:46:13.838303+0000","last_clean":"2023-08-25T09:46:13.838303+0000","last_became_active":"2023-08-25T09:46:12.833247+0000","last_became_peered":"2023-08-25T09:46:12.833247+0000","last_unstale":"2023-08-25T09:46:13.838303+0000","last_undegraded":"2023-08-25T09:46:13.838303+0000","last_fullsized":"2023-08-25T09:46:13.838303+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:00:40.793263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861425+0000","last_change":"2023-08-25T09:46:13.861304+0000","last_active":"2023-08-25T09:46:13.861425+0000","last_peered":"2023-08-25T09:46:13.861425+0000","last_clean":"2023-08-25T09:46:13.861425+0000","last_became_active":"2023-08-25T09:46:13.856296+0000","last_became_peered":"2023-08-25T09:46:13.856296+0000","last_unstale":"2023-08-25T09:46:13.861425+0000","last_undegraded":"2023-08-25T09:46:13.861425+0000","last_fullsized":"2023-08-25T09:46:13.861425+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:40.300356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.864417+0000","last_change":"2023-08-25T09:46:12.851284+0000","last_active":"2023-08-25T09:46:13.864417+0000","last_peered":"2023-08-25T09:46:13.864417+0000","last_clean":"2023-08-25T09:46:13.864417+0000","last_became_active":"2023-08-25T09:46:12.850203+0000","last_became_peered":"2023-08-25T09:46:12.850203+0000","last_unstale":"2023-08-25T09:46:13.864417+0000","last_undegraded":"2023-08-25T09:46:13.864417+0000","last_fullsized":"2023-08-25T09:46:13.864417+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:43.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.838223+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:46:13.838223+0000","last_peered":"2023-08-25T09:46:13.838223+0000","last_clean":"2023-08-25T09:46:13.838223+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:46:13.838223+0000","last_undegraded":"2023-08-25T09:46:13.838223+0000","last_fullsized":"2023-08-25T09:46:13.838223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:39:02.299585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.834187+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:46:13.834187+0000","last_peered":"2023-08-25T09:46:13.834187+0000","last_clean":"2023-08-25T09:46:13.834187+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:46:13.834187+0000","last_undegraded":"2023-08-25T09:46:13.834187+0000","last_fullsized":"2023-08-25T09:46:13.834187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:39:48.229465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.856775+0000","last_change":"2023-08-25T09:46:13.856507+0000","last_active":"2023-08-25T09:46:13.856775+0000","last_peered":"2023-08-25T09:46:13.856775+0000","last_clean":"2023-08-25T09:46:13.856775+0000","last_became_active":"2023-08-25T09:46:13.855894+0000","last_became_peered":"2023-08-25T09:46:13.855894+0000","last_unstale":"2023-08-25T09:46:13.856775+0000","last_undegraded":"2023-08-25T09:46:13.856775+0000","last_fullsized":"2023-08-25T09:46:13.856775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:38:40.979862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860616+0000","last_change":"2023-08-25T09:46:12.842641+0000","last_active":"2023-08-25T09:46:13.860616+0000","last_peered":"2023-08-25T09:46:13.860616+0000","last_clean":"2023-08-25T09:46:13.860616+0000","last_became_active":"2023-08-25T09:46:12.842361+0000","last_became_peered":"2023-08-25T09:46:12.842361+0000","last_unstale":"2023-08-25T09:46:13.860616+0000","last_undegraded":"2023-08-25T09:46:13.860616+0000","last_fullsized":"2023-08-25T09:46:13.860616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:42:28.503309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833594+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:46:13.833594+0000","last_peered":"2023-08-25T09:46:13.833594+0000","last_clean":"2023-08-25T09:46:13.833594+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:46:13.833594+0000","last_undegraded":"2023-08-25T09:46:13.833594+0000","last_fullsized":"2023-08-25T09:46:13.833594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:36:23.556533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.859548+0000","last_change":"2023-08-25T09:46:13.859466+0000","last_active":"2023-08-25T09:46:13.859548+0000","last_peered":"2023-08-25T09:46:13.859548+0000","last_clean":"2023-08-25T09:46:13.859548+0000","last_became_active":"2023-08-25T09:46:13.858905+0000","last_became_peered":"2023-08-25T09:46:13.858905+0000","last_unstale":"2023-08-25T09:46:13.859548+0000","last_undegraded":"2023-08-25T09:46:13.859548+0000","last_fullsized":"2023-08-25T09:46:13.859548+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:25.787736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.851088+0000","last_change":"2023-08-25T09:46:12.841819+0000","last_active":"2023-08-25T09:46:13.851088+0000","last_peered":"2023-08-25T09:46:13.851088+0000","last_clean":"2023-08-25T09:46:13.851088+0000","last_became_active":"2023-08-25T09:46:12.840186+0000","last_became_peered":"2023-08-25T09:46:12.840186+0000","last_unstale":"2023-08-25T09:46:13.851088+0000","last_undegraded":"2023-08-25T09:46:13.851088+0000","last_fullsized":"2023-08-25T09:46:13.851088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:05.625951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.848275+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:46:13.848275+0000","last_peered":"2023-08-25T09:46:13.848275+0000","last_clean":"2023-08-25T09:46:13.848275+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:46:13.848275+0000","last_undegraded":"2023-08-25T09:46:13.848275+0000","last_fullsized":"2023-08-25T09:46:13.848275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:01:39.470904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845726+0000","last_change":"2023-08-25T09:46:13.845634+0000","last_active":"2023-08-25T09:46:13.845726+0000","last_peered":"2023-08-25T09:46:13.845726+0000","last_clean":"2023-08-25T09:46:13.845726+0000","last_became_active":"2023-08-25T09:46:13.843965+0000","last_became_peered":"2023-08-25T09:46:13.843965+0000","last_unstale":"2023-08-25T09:46:13.845726+0000","last_undegraded":"2023-08-25T09:46:13.845726+0000","last_fullsized":"2023-08-25T09:46:13.845726+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:17:08.706354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854548+0000","last_change":"2023-08-25T09:46:12.841273+0000","last_active":"2023-08-25T09:46:13.854548+0000","last_peered":"2023-08-25T09:46:13.854548+0000","last_clean":"2023-08-25T09:46:13.854548+0000","last_became_active":"2023-08-25T09:46:12.841026+0000","last_became_peered":"2023-08-25T09:46:12.841026+0000","last_unstale":"2023-08-25T09:46:13.854548+0000","last_undegraded":"2023-08-25T09:46:13.854548+0000","last_fullsized":"2023-08-25T09:46:13.854548+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:19.493227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833072+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:46:13.833072+0000","last_peered":"2023-08-25T09:46:13.833072+0000","last_clean":"2023-08-25T09:46:13.833072+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:46:13.833072+0000","last_undegraded":"2023-08-25T09:46:13.833072+0000","last_fullsized":"2023-08-25T09:46:13.833072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:33.664260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.857145+0000","last_change":"2023-08-25T09:46:13.857037+0000","last_active":"2023-08-25T09:46:13.857145+0000","last_peered":"2023-08-25T09:46:13.857145+0000","last_clean":"2023-08-25T09:46:13.857145+0000","last_became_active":"2023-08-25T09:46:13.853078+0000","last_became_peered":"2023-08-25T09:46:13.853078+0000","last_unstale":"2023-08-25T09:46:13.857145+0000","last_undegraded":"2023-08-25T09:46:13.857145+0000","last_fullsized":"2023-08-25T09:46:13.857145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:01:15.965690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.846440+0000","last_change":"2023-08-25T09:46:12.850794+0000","last_active":"2023-08-25T09:46:13.846440+0000","last_peered":"2023-08-25T09:46:13.846440+0000","last_clean":"2023-08-25T09:46:13.846440+0000","last_became_active":"2023-08-25T09:46:12.850570+0000","last_became_peered":"2023-08-25T09:46:12.850570+0000","last_unstale":"2023-08-25T09:46:13.846440+0000","last_undegraded":"2023-08-25T09:46:13.846440+0000","last_fullsized":"2023-08-25T09:46:13.846440+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:18:20.187390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836951+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:46:13.836951+0000","last_peered":"2023-08-25T09:46:13.836951+0000","last_clean":"2023-08-25T09:46:13.836951+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:46:13.836951+0000","last_undegraded":"2023-08-25T09:46:13.836951+0000","last_fullsized":"2023-08-25T09:46:13.836951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:42:30.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852561+0000","last_change":"2023-08-25T09:46:13.851749+0000","last_active":"2023-08-25T09:46:13.852561+0000","last_peered":"2023-08-25T09:46:13.852561+0000","last_clean":"2023-08-25T09:46:13.852561+0000","last_became_active":"2023-08-25T09:46:13.851078+0000","last_became_peered":"2023-08-25T09:46:13.851078+0000","last_unstale":"2023-08-25T09:46:13.852561+0000","last_undegraded":"2023-08-25T09:46:13.852561+0000","last_fullsized":"2023-08-25T09:46:13.852561+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:45:48.631058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836698+0000","last_change":"2023-08-25T09:46:12.846366+0000","last_active":"2023-08-25T09:46:13.836698+0000","last_peered":"2023-08-25T09:46:13.836698+0000","last_clean":"2023-08-25T09:46:13.836698+0000","last_became_active":"2023-08-25T09:46:12.845484+0000","last_became_peered":"2023-08-25T09:46:12.845484+0000","last_unstale":"2023-08-25T09:46:13.836698+0000","last_undegraded":"2023-08-25T09:46:13.836698+0000","last_fullsized":"2023-08-25T09:46:13.836698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:46:32.336383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833406+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:46:13.833406+0000","last_peered":"2023-08-25T09:46:13.833406+0000","last_clean":"2023-08-25T09:46:13.833406+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:46:13.833406+0000","last_undegraded":"2023-08-25T09:46:13.833406+0000","last_fullsized":"2023-08-25T09:46:13.833406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:10:54.659715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861620+0000","last_change":"2023-08-25T09:46:13.861119+0000","last_active":"2023-08-25T09:46:13.861620+0000","last_peered":"2023-08-25T09:46:13.861620+0000","last_clean":"2023-08-25T09:46:13.861620+0000","last_became_active":"2023-08-25T09:46:13.858979+0000","last_became_peered":"2023-08-25T09:46:13.858979+0000","last_unstale":"2023-08-25T09:46:13.861620+0000","last_undegraded":"2023-08-25T09:46:13.861620+0000","last_fullsized":"2023-08-25T09:46:13.861620+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:59.027427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858819+0000","last_change":"2023-08-25T09:46:12.848200+0000","last_active":"2023-08-25T09:46:13.858819+0000","last_peered":"2023-08-25T09:46:13.858819+0000","last_clean":"2023-08-25T09:46:13.858819+0000","last_became_active":"2023-08-25T09:46:12.847931+0000","last_became_peered":"2023-08-25T09:46:12.847931+0000","last_unstale":"2023-08-25T09:46:13.858819+0000","last_undegraded":"2023-08-25T09:46:13.858819+0000","last_fullsized":"2023-08-25T09:46:13.858819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:31:52.940795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.827729+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:46:13.827729+0000","last_peered":"2023-08-25T09:46:13.827729+0000","last_clean":"2023-08-25T09:46:13.827729+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:46:13.827729+0000","last_undegraded":"2023-08-25T09:46:13.827729+0000","last_fullsized":"2023-08-25T09:46:13.827729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:54.344637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.856822+0000","last_change":"2023-08-25T09:46:13.856436+0000","last_active":"2023-08-25T09:46:13.856822+0000","last_peered":"2023-08-25T09:46:13.856822+0000","last_clean":"2023-08-25T09:46:13.856822+0000","last_became_active":"2023-08-25T09:46:13.856203+0000","last_became_peered":"2023-08-25T09:46:13.856203+0000","last_unstale":"2023-08-25T09:46:13.856822+0000","last_undegraded":"2023-08-25T09:46:13.856822+0000","last_fullsized":"2023-08-25T09:46:13.856822+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:21.909471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837957+0000","last_change":"2023-08-25T09:46:12.840595+0000","last_active":"2023-08-25T09:46:13.837957+0000","last_peered":"2023-08-25T09:46:13.837957+0000","last_clean":"2023-08-25T09:46:13.837957+0000","last_became_active":"2023-08-25T09:46:12.832232+0000","last_became_peered":"2023-08-25T09:46:12.832232+0000","last_unstale":"2023-08-25T09:46:13.837957+0000","last_undegraded":"2023-08-25T09:46:13.837957+0000","last_fullsized":"2023-08-25T09:46:13.837957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:10.133619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.832931+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:46:13.832931+0000","last_peered":"2023-08-25T09:46:13.832931+0000","last_clean":"2023-08-25T09:46:13.832931+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:46:13.832931+0000","last_undegraded":"2023-08-25T09:46:13.832931+0000","last_fullsized":"2023-08-25T09:46:13.832931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:22:30.690053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220366,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":1.1599999999999999,"15min":1.0669999999999999},"min":{"1min":0.67500000000000004,"5min":0.59699999999999998,"15min":0.58899999999999997},"max":{"1min":1.3049999999999999,"5min":10.747,"15min":10.747},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.125,"15min":1.046},"min":{"1min":0.83999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3819999999999999,"5min":1.855,"15min":1.855},"last":1.0620000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.323,"15min":1.177},"min":{"1min":0.66900000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":11.010999999999999,"5min":11.010999999999999,"15min":11.010999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.127,"5min":1.173,"15min":1.103},"min":{"1min":0.93100000000000005,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4019999999999999,"5min":1.6679999999999999,"15min":1.7769999999999999},"last":0.93999999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1599999999999999,"15min":1.0900000000000001},"min":{"1min":0.748,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.403,"5min":1.619,"15min":1.619},"last":1.111},{"interface":"front","average":{"1min":1.224,"5min":1.304,"15min":1.232},"min":{"1min":0.75,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6859999999999999,"5min":1.98,"15min":1.98},"last":1.335}]},{"osd":3,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2410000000000001,"15min":1.177},"min":{"1min":0.93300000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5349999999999999,"5min":1.6699999999999999,"15min":1.7110000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.371,"15min":1.335},"min":{"1min":0.89200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6719999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0149999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3320000000000001,"15min":1.2809999999999999},"min":{"1min":0.99299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7110000000000001,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.1830000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.292}]},{"osd":5,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4339999999999999,"15min":1.3540000000000001},"min":{"1min":0.94299999999999995,"5min":0.67700000000000005,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.448,"15min":1.3879999999999999},"min":{"1min":1.1000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7330000000000001,"5min":1.9830000000000001,"15min":2.0739999999999998},"last":1.228}]},{"osd":6,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.5569999999999999,"15min":1.456},"min":{"1min":1.274,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.819,"5min":2.032,"15min":2.1720000000000002},"last":1.381},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4890000000000001,"15min":1.4139999999999999},"min":{"1min":1.1859999999999999,"5min":0.71199999999999997,"15min":0.61399999999999999},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.2}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220346,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1499999999999999,"15min":1.1459999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.615,"5min":1.804,"15min":1.804},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.1990000000000001,"15min":1.1479999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.734,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.3620000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2330000000000001,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.726,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.202,"5min":1.236,"15min":1.1379999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7669999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.492}]},{"osd":2,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1819999999999999,"15min":1.1140000000000001},"min":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5269999999999999,"5min":1.708,"15min":1.708},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.425,"15min":1.2330000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7070000000000001,"5min":13.193,"15min":13.193},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.2410000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6479999999999999,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.21},{"interface":"front","average":{"1min":1.278,"5min":1.27,"15min":1.2589999999999999},"min":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.76,"5min":1.76,"15min":1.9239999999999999},"last":1.7430000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3400000000000001,"15min":1.266},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8089999999999999,"5min":1.881,"15min":2.484},"last":1.702},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.405,"15min":1.3300000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7789999999999999,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3140000000000001,"15min":1.2809999999999999},"min":{"1min":0.67700000000000005,"5min":0.624,"15min":0.59599999999999997},"max":{"1min":1.663,"5min":1.837,"15min":2.0049999999999999},"last":1.859},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.502,"15min":1.3919999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8129999999999999,"5min":1.978,"15min":2.3450000000000002},"last":1.792}]},{"osd":7,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5620000000000001,"15min":1.5009999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.622},"max":{"1min":1.9490000000000001,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.669},{"interface":"front","average":{"1min":1.446,"5min":1.47,"15min":1.474},"min":{"1min":0.74299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.97,"5min":1.97,"15min":2.1890000000000001},"last":1.946}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193982,"statfs":{"total":200000000,"available":198637778,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2070000000000001,"15min":1.153},"min":{"1min":0.63,"5min":0.497,"15min":0.497},"max":{"1min":1.6060000000000001,"5min":2.081,"15min":2.081},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.4410000000000001,"5min":1.8839999999999999,"15min":1.974},"last":1.4039999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1539999999999999,"15min":1.1870000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6830000000000001,"5min":1.851,"15min":2.1779999999999999},"last":1.208},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.222,"15min":1.1299999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.583,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.3660000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2629999999999999,"15min":1.1919999999999999},"min":{"1min":0.73899999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.5669999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.224},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.6890000000000001,"5min":1.8049999999999999,"15min":1.962},"last":1.0209999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.3759999999999999,"15min":1.2969999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.536,"5min":17.356999999999999,"15min":17.356999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.3400000000000001,"15min":1.29},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.9650000000000001},"last":1.5880000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.3340000000000001,"15min":1.3},"min":{"1min":0.77600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7410000000000001,"5min":2.625,"15min":2.625},"last":1.129},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.343,"15min":1.3620000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.58799999999999997},"max":{"1min":1.845,"5min":1.8620000000000001,"15min":2.2789999999999999},"last":1.0760000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.448,"15min":1.4299999999999999},"min":{"1min":0.85499999999999998,"5min":0.752,"15min":0.58899999999999997},"max":{"1min":1.9219999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.522,"15min":1.4259999999999999},"min":{"1min":0.88700000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.028,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.244}]},{"osd":7,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.472,"15min":1.4399999999999999},"min":{"1min":0.80800000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.9630000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.272},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.591,"15min":1.575},"min":{"1min":0.93100000000000005,"5min":0.876,"15min":0.63900000000000001},"max":{"1min":1.994,"5min":2.0790000000000002,"15min":2.3700000000000001},"last":1.3140000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":753,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199228146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.234,"15min":1.171},"min":{"1min":0.80100000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6519999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3340000000000001,"15min":1.179},"min":{"1min":0.95899999999999996,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":1.635,"5min":7.367,"15min":7.367},"last":1.347}]},{"osd":1,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1799999999999999,"15min":1.133},"min":{"1min":0.89000000000000001,"5min":0.72199999999999998,"15min":0.71299999999999997},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.228,"15min":1.373},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.954,"5min":1.954,"15min":8.5850000000000009},"last":1.115}]},{"osd":2,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.206,"15min":1.284},"min":{"1min":0.85299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6240000000000001,"5min":1.6899999999999999,"15min":4.1669999999999998},"last":1.232},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2889999999999999,"15min":1.1899999999999999},"min":{"1min":0.85299999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6739999999999999,"5min":1.6799999999999999,"15min":1.8520000000000001},"last":1.5129999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.274,"15min":1.2310000000000001},"min":{"1min":0.77100000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.764,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.46},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.887,"5min":1.887,"15min":1.911},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.286,"15min":1.2909999999999999},"min":{"1min":0.77600000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.4420000000000002},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3660000000000001,"15min":1.264},"min":{"1min":0.84199999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9299999999999999,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.7569999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.456,"15min":1.4410000000000001},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.5880000000000001,"5min":3.5880000000000001,"15min":3.5880000000000001},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.746,"5min":1.5800000000000001,"15min":1.569},"min":{"1min":1.244,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5270000000000001,"5min":3.5270000000000001,"15min":4.0229999999999997},"last":1.629}]},{"osd":7,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5840000000000001,"15min":1.4370000000000001},"min":{"1min":0.93999999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.976,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.651},{"interface":"front","average":{"1min":1.653,"5min":1.621,"15min":1.6000000000000001},"min":{"1min":0.80900000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.7930000000000001},"last":1.806}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":762,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194550,"statfs":{"total":200000000,"available":199219406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.94099999999999995,"15min":0.86399999999999999},"min":{"1min":0.58199999999999996,"5min":0.432,"15min":0.29299999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6830000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.0129999999999999,"15min":0.91600000000000004},"min":{"1min":0.54200000000000004,"5min":0.503,"15min":0.41699999999999998},"max":{"1min":1.292,"5min":1.631,"15min":1.631},"last":1.0089999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.90800000000000003,"15min":0.81399999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.35699999999999998},"max":{"1min":1.3580000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.044,"15min":0.98099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":0.94799999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.018,"15min":0.89200000000000002},"min":{"1min":0.56200000000000006,"5min":0.45000000000000001,"15min":0.378},"max":{"1min":1.3939999999999999,"5min":1.919,"15min":1.919},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.167,"15min":1.119},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.44900000000000001},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":3.399},"last":1.2869999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2529999999999999,"15min":1.198},"min":{"1min":0.82199999999999995,"5min":0.76100000000000001,"15min":0.66800000000000004},"max":{"1min":1.6519999999999999,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":1.194},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2569999999999999,"15min":1.204},"min":{"1min":0.73299999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.984,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.23}]},{"osd":5,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3640000000000001,"15min":1.319},"min":{"1min":0.86899999999999999,"5min":0.73599999999999999,"15min":0.71899999999999997},"max":{"1min":2.407,"5min":2.407,"15min":2.407},"last":1.47},{"interface":"front","average":{"1min":1.264,"5min":1.3220000000000001,"15min":1.276},"min":{"1min":0.83599999999999997,"5min":0.81499999999999995,"15min":0.64500000000000002},"max":{"1min":1.7949999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.335}]},{"osd":6,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3100000000000001,"15min":1.252},"min":{"1min":0.85199999999999998,"5min":0.78200000000000003,"15min":0.73799999999999999},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4370000000000001,"15min":1.409},"min":{"1min":0.95399999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.298,"5min":2.6699999999999999,"15min":3.496},"last":1.405}]},{"osd":7,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4279999999999999,"15min":1.3680000000000001},"min":{"1min":0.995,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.659,"5min":2.044,"15min":2.2989999999999999},"last":1.5},{"interface":"front","average":{"1min":1.385,"5min":1.4410000000000001,"15min":1.3149999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.3560000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":753,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199228146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85899999999999999,"15min":0.83499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.383},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95099999999999996,"15min":1.0449999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.546,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.026,"15min":0.997},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.45900000000000002},"max":{"1min":1.698,"5min":1.9350000000000001,"15min":2.3180000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.0389999999999999,"15min":1.056},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.891,"5min":1.1479999999999999,"15min":1.133},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":18.821000000000002,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.502},{"interface":"front","average":{"1min":1.089,"5min":1.1240000000000001,"15min":1.109},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.738,"5min":2.145,"15min":2.2040000000000002},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3240000000000001,"15min":1.387},"min":{"1min":0.751,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.833,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.284,"15min":1.331},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":1.036}]},{"osd":5,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.246,"15min":1.353},"min":{"1min":0.78300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.798,"5min":2.798,"15min":5.1269999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.381,"15min":1.395},"min":{"1min":0.85199999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1179999999999999,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.121}]},{"osd":6,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3859999999999999,"15min":1.417},"min":{"1min":0.77700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.6709999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.395,"5min":1.4039999999999999,"15min":1.47},"min":{"1min":0.874,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.903,"5min":2.903,"15min":5.0599999999999996},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.77000000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.331,"5min":2.331,"15min":2.968},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4299999999999999,"15min":1.492},"min":{"1min":0.81599999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.0169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193982,"statfs":{"total":200000000,"available":198637778,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86399999999999999,"15min":0.78700000000000003},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":1.5049999999999999,"5min":1.5700000000000001,"15min":1.627},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94399999999999995,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39300000000000002},"max":{"1min":1.776,"5min":1.895,"15min":1.895},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93700000000000006,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.038,"15min":0.92400000000000004},"min":{"1min":0.53300000000000003,"5min":0.41899999999999998,"15min":0.33600000000000002},"max":{"1min":1.506,"5min":1.746,"15min":1.746},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1919999999999999,"15min":1.0600000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.372},"max":{"1min":1.8680000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.077,"15min":0.96499999999999997},"min":{"1min":0.56999999999999995,"5min":0.48799999999999999,"15min":0.45000000000000001},"max":{"1min":1.6240000000000001,"5min":1.6439999999999999,"15min":1.764},"last":0.629}]},{"osd":4,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1919999999999999,"15min":1.1020000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5029999999999999,"5min":1.694,"15min":1.694},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3480000000000001,"15min":1.236},"min":{"1min":0.998,"5min":0.629,"15min":0.629},"max":{"1min":1.722,"5min":2.008,"15min":2.008},"last":0.95199999999999996}]},{"osd":5,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2809999999999999,"15min":1.204},"min":{"1min":0.82199999999999995,"5min":0.751,"15min":0.68600000000000005},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.323,"5min":1.2829999999999999,"15min":1.1890000000000001},"min":{"1min":0.83199999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4510000000000001,"15min":1.3220000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1339999999999999,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.002},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3140000000000001},"min":{"1min":0.92400000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9890000000000001,"5min":2.004,"15min":2.004},"last":1.0369999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.472,"15min":1.369},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.9630000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.53,"5min":1.4610000000000001,"15min":1.3380000000000001},"min":{"1min":0.91600000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.0600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":0.86399999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.313},"max":{"1min":1.3939999999999999,"5min":11.257,"15min":11.257},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0149999999999999,"15min":0.96199999999999997},"min":{"1min":0.58099999999999996,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":1.474,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.85499999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.96899999999999997,"15min":0.94199999999999995},"min":{"1min":0.59999999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.724,"15min":1.724},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.016,"15min":1.0800000000000001},"min":{"1min":0.498,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4259999999999999,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.94699999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92100000000000004,"15min":0.90500000000000003},"min":{"1min":0.437,"5min":0.42699999999999999,"15min":0.40699999999999997},"max":{"1min":1.625,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.121},"min":{"1min":0.55500000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6339999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.367}]},{"osd":4,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.272,"15min":1.24},"min":{"1min":0.82799999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.78,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.379,"5min":1.3759999999999999,"15min":1.357},"min":{"1min":0.95299999999999996,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.482}]},{"osd":5,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.2909999999999999,"15min":1.347},"min":{"1min":0.82499999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.9530000000000001,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.294},"min":{"1min":0.89900000000000002,"5min":0.73699999999999999,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.347,"15min":1.3440000000000001},"min":{"1min":0.86399999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"max":{"1min":1.821,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.343,"15min":1.3080000000000001},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.72699999999999998},"max":{"1min":2.2090000000000001,"5min":2.331,"15min":2.331},"last":1.0700000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4139999999999999,"15min":1.4019999999999999},"min":{"1min":0.94999999999999996,"5min":0.79000000000000004,"15min":0.71099999999999997},"max":{"1min":2.085,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.337,"15min":1.323},"min":{"1min":0.86899999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.1469999999999998,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":1.2410000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:46:18.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:46:18.369 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-08-25T09:46:18.368+0000 7f1880fa5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:46:18.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.578+0000 7fdb3e2c2640 1 Processor -- start 2023-08-25T09:46:18.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.578+0000 7fdb3e2c2640 1 -- start start 2023-08-25T09:46:18.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.579+0000 7fdb3e2c2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdb380c92c0 0x7fdb380b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.579+0000 7fdb3e2c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdb380c96a0 0x7fdb380cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.579+0000 7fdb3e2c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb380cd330 con 0x7fdb380b4150 2023-08-25T09:46:18.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.579+0000 7fdb3e2c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb381615b0 con 0x7fdb380c96a0 2023-08-25T09:46:18.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.579+0000 7fdb3e2c2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb380b0e80 con 0x7fdb380c92c0 2023-08-25T09:46:18.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3cabf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdb380c96a0 0x7fdb380cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3dac1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdb380b4150 0x7fdb380cd7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59366/0 (socket says 172.21.15.70:59366) 2023-08-25T09:46:18.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3dac1640 1 -- 172.21.15.70:0/2880813145 learned_addr learned my addr 172.21.15.70:0/2880813145 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:18.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3d2c0640 1 --2- 172.21.15.70:0/2880813145 >> v2:172.21.15.155:3300/0 conn(0x7fdb380c92c0 0x7fdb380b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3cabf640 1 -- 172.21.15.70:0/2880813145 >> v2:172.21.15.155:3300/0 conn(0x7fdb380c92c0 msgr2=0x7fdb380b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:18.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3cabf640 1 --2- 172.21.15.70:0/2880813145 >> v2:172.21.15.155:3300/0 conn(0x7fdb380c92c0 0x7fdb380b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3cabf640 1 -- 172.21.15.70:0/2880813145 >> v1:172.21.15.70:6789/0 conn(0x7fdb380b4150 legacy=0x7fdb380cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3cabf640 1 -- 172.21.15.70:0/2880813145 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380cc720 con 0x7fdb380c96a0 2023-08-25T09:46:18.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb2ffff640 1 -- 172.21.15.70:0/2880813145 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273732234 0 0) 0x7fdb380cd330 con 0x7fdb380b4150 2023-08-25T09:46:18.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.580+0000 7fdb3d2c0640 1 --2- 172.21.15.70:0/2880813145 >> v2:172.21.15.155:3300/0 conn(0x7fdb380c92c0 0x7fdb380b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:18.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.581+0000 7fdb3cabf640 1 --2- 172.21.15.70:0/2880813145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c96a0 0x7fdb380cad40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806c670 tx=0x7fdb2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9b1c61e25eb92d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:18.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb2ffff640 1 -- 172.21.15.70:0/2880813145 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28031830 con 0x7fdb380c96a0 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb2ffff640 1 -- 172.21.15.70:0/2880813145 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28031990 con 0x7fdb380c96a0 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb2ffff640 1 -- 172.21.15.70:0/2880813145 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28031d20 con 0x7fdb380c96a0 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/2880813145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c96a0 msgr2=0x7fdb380cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/2880813145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c96a0 0x7fdb380cad40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdb3806c670 tx=0x7fdb2802b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/2880813145 shutdown_connections 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/2880813145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c96a0 0x7fdb380cad40 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/2880813145 >> v2:172.21.15.155:3300/0 conn(0x7fdb380c92c0 0x7fdb380b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/2880813145 >> 172.21.15.70:0/2880813145 conn(0x7fdb380affd0 msgr2=0x7fdb380b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:18.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/2880813145 shutdown_connections 2023-08-25T09:46:18.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.582+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/2880813145 wait complete. 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.584+0000 7fdb3e2c2640 1 Processor -- start 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.584+0000 7fdb3e2c2640 1 -- start start 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.585+0000 7fdb3e2c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb380b4150 0x7fdb38160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.585+0000 7fdb3e2c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 0x7fdb380c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.585+0000 7fdb3e2c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb380c96a0 0x7fdb381ea350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.585+0000 7fdb3e2c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c1920 con 0x7fdb380c92c0 2023-08-25T09:46:18.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.585+0000 7fdb3e2c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c8530 con 0x7fdb380c96a0 2023-08-25T09:46:18.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.585+0000 7fdb3e2c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c4c40 con 0x7fdb380b4150 2023-08-25T09:46:18.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 0x7fdb380c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 0x7fdb380c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51398/0 (socket says 172.21.15.70:51398) 2023-08-25T09:46:18.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 -- 172.21.15.70:0/118882017 learned_addr learned my addr 172.21.15.70:0/118882017 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:18.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3d2c0640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb380b4150 0x7fdb38160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3dac1640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb380c96a0 0x7fdb381ea350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 -- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb380b4150 msgr2=0x7fdb38160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb380b4150 0x7fdb38160130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 -- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb380c96a0 msgr2=0x7fdb381ea350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb380c96a0 0x7fdb381ea350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 -- 172.21.15.70:0/118882017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28009400 con 0x7fdb380c92c0 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.586+0000 7fdb3cabf640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 0x7fdb380c7e50 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdb28000c00 tx=0x7fdb280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.587+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803aa60 con 0x7fdb380c92c0 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.587+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2803abe0 con 0x7fdb380c92c0 2023-08-25T09:46:18.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.587+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb280443f0 con 0x7fdb380c92c0 2023-08-25T09:46:18.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.587+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb380c4f20 con 0x7fdb380c92c0 2023-08-25T09:46:18.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.587+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3815fe80 con 0x7fdb380c92c0 2023-08-25T09:46:18.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.588+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38066fb0 con 0x7fdb380c92c0 2023-08-25T09:46:18.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.589+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdb28044550 con 0x7fdb380c92c0 2023-08-25T09:46:18.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.590+0000 7fdb2dffb640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb1c06a050 0x7fdb1c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:18.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.590+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdb280aa8e0 con 0x7fdb380c92c0 2023-08-25T09:46:18.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.590+0000 7fdb3d2c0640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb1c06a050 0x7fdb1c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:18.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.591+0000 7fdb3d2c0640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb1c06a050 0x7fdb1c06c510 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdb380b2340 tx=0x7fdb34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:18.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.594+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdb280760e0 con 0x7fdb380c92c0 2023-08-25T09:46:18.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.778+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb380c4db0 con 0x7fdb1c06a050 2023-08-25T09:46:18.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.787+0000 7fdb2dffb640 1 -- 172.21.15.70:0/118882017 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303433 (secure 0 0 0) 0x7fdb380c4db0 con 0x7fdb1c06a050 2023-08-25T09:46:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:18.795 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:46:18.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb1c06a050 msgr2=0x7fdb1c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb1c06a050 0x7fdb1c06c510 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdb380b2340 tx=0x7fdb34008040 comp rx=0 tx=0).stop 2023-08-25T09:46:18.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 msgr2=0x7fdb380c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:18.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 0x7fdb380c7e50 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fdb28000c00 tx=0x7fdb280046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 shutdown_connections 2023-08-25T09:46:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdb1c06a050 0x7fdb1c06c510 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdb380c96a0 0x7fdb381ea350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.794+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdb380c92c0 0x7fdb380c7e50 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.795+0000 7fdb3e2c2640 1 --2- 172.21.15.70:0/118882017 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdb380b4150 0x7fdb38160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.795+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 >> 172.21.15.70:0/118882017 conn(0x7fdb380affd0 msgr2=0x7fdb380cad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.795+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 shutdown_connections 2023-08-25T09:46:18.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:18.795+0000 7fdb3e2c2640 1 -- 172.21.15.70:0/118882017 wait complete. 2023-08-25T09:46:18.817 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-08-25T09:46:17.468304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7214,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555285,"statfs":{"total":1600000000,"available":1592612402,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.662923"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852515+0000","last_change":"2023-08-25T09:46:13.851390+0000","last_active":"2023-08-25T09:46:13.852515+0000","last_peered":"2023-08-25T09:46:13.852515+0000","last_clean":"2023-08-25T09:46:13.852515+0000","last_became_active":"2023-08-25T09:46:13.849187+0000","last_became_peered":"2023-08-25T09:46:13.849187+0000","last_unstale":"2023-08-25T09:46:13.852515+0000","last_undegraded":"2023-08-25T09:46:13.852515+0000","last_fullsized":"2023-08-25T09:46:13.852515+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:26.295460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854795+0000","last_change":"2023-08-25T09:46:13.854719+0000","last_active":"2023-08-25T09:46:13.854795+0000","last_peered":"2023-08-25T09:46:13.854795+0000","last_clean":"2023-08-25T09:46:13.854795+0000","last_became_active":"2023-08-25T09:46:13.854454+0000","last_became_peered":"2023-08-25T09:46:13.854454+0000","last_unstale":"2023-08-25T09:46:13.854795+0000","last_undegraded":"2023-08-25T09:46:13.854795+0000","last_fullsized":"2023-08-25T09:46:13.854795+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:49.662888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852531+0000","last_change":"2023-08-25T09:46:13.852310+0000","last_active":"2023-08-25T09:46:13.852531+0000","last_peered":"2023-08-25T09:46:13.852531+0000","last_clean":"2023-08-25T09:46:13.852531+0000","last_became_active":"2023-08-25T09:46:13.851416+0000","last_became_peered":"2023-08-25T09:46:13.851416+0000","last_unstale":"2023-08-25T09:46:13.852531+0000","last_undegraded":"2023-08-25T09:46:13.852531+0000","last_fullsized":"2023-08-25T09:46:13.852531+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:21.909471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837138+0000","last_change":"2023-08-25T09:46:13.836516+0000","last_active":"2023-08-25T09:46:13.837138+0000","last_peered":"2023-08-25T09:46:13.837138+0000","last_clean":"2023-08-25T09:46:13.837138+0000","last_became_active":"2023-08-25T09:46:13.835894+0000","last_became_peered":"2023-08-25T09:46:13.835894+0000","last_unstale":"2023-08-25T09:46:13.837138+0000","last_undegraded":"2023-08-25T09:46:13.837138+0000","last_fullsized":"2023-08-25T09:46:13.837138+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:35.225952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837083+0000","last_change":"2023-08-25T09:46:13.835218+0000","last_active":"2023-08-25T09:46:13.837083+0000","last_peered":"2023-08-25T09:46:13.837083+0000","last_clean":"2023-08-25T09:46:13.837083+0000","last_became_active":"2023-08-25T09:46:13.834503+0000","last_became_peered":"2023-08-25T09:46:13.834503+0000","last_unstale":"2023-08-25T09:46:13.837083+0000","last_undegraded":"2023-08-25T09:46:13.837083+0000","last_fullsized":"2023-08-25T09:46:13.837083+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:02.087906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858257+0000","last_change":"2023-08-25T09:46:13.858094+0000","last_active":"2023-08-25T09:46:13.858257+0000","last_peered":"2023-08-25T09:46:13.858257+0000","last_clean":"2023-08-25T09:46:13.858257+0000","last_became_active":"2023-08-25T09:46:13.857822+0000","last_became_peered":"2023-08-25T09:46:13.857822+0000","last_unstale":"2023-08-25T09:46:13.858257+0000","last_undegraded":"2023-08-25T09:46:13.858257+0000","last_fullsized":"2023-08-25T09:46:13.858257+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:14.209373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861577+0000","last_change":"2023-08-25T09:46:13.861529+0000","last_active":"2023-08-25T09:46:13.861577+0000","last_peered":"2023-08-25T09:46:13.861577+0000","last_clean":"2023-08-25T09:46:13.861577+0000","last_became_active":"2023-08-25T09:46:13.861015+0000","last_became_peered":"2023-08-25T09:46:13.861015+0000","last_unstale":"2023-08-25T09:46:13.861577+0000","last_undegraded":"2023-08-25T09:46:13.861577+0000","last_fullsized":"2023-08-25T09:46:13.861577+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:45:39.512662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860687+0000","last_change":"2023-08-25T09:46:13.860520+0000","last_active":"2023-08-25T09:46:13.860687+0000","last_peered":"2023-08-25T09:46:13.860687+0000","last_clean":"2023-08-25T09:46:13.860687+0000","last_became_active":"2023-08-25T09:46:13.859561+0000","last_became_peered":"2023-08-25T09:46:13.859561+0000","last_unstale":"2023-08-25T09:46:13.860687+0000","last_undegraded":"2023-08-25T09:46:13.860687+0000","last_fullsized":"2023-08-25T09:46:13.860687+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:11.935170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836135+0000","last_change":"2023-08-25T09:46:13.836031+0000","last_active":"2023-08-25T09:46:13.836135+0000","last_peered":"2023-08-25T09:46:13.836135+0000","last_clean":"2023-08-25T09:46:13.836135+0000","last_became_active":"2023-08-25T09:46:13.835741+0000","last_became_peered":"2023-08-25T09:46:13.835741+0000","last_unstale":"2023-08-25T09:46:13.836135+0000","last_undegraded":"2023-08-25T09:46:13.836135+0000","last_fullsized":"2023-08-25T09:46:13.836135+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:31:37.550284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.841682+0000","last_change":"2023-08-25T09:46:13.841414+0000","last_active":"2023-08-25T09:46:13.841682+0000","last_peered":"2023-08-25T09:46:13.841682+0000","last_clean":"2023-08-25T09:46:13.841682+0000","last_became_active":"2023-08-25T09:46:13.840612+0000","last_became_peered":"2023-08-25T09:46:13.840612+0000","last_unstale":"2023-08-25T09:46:13.841682+0000","last_undegraded":"2023-08-25T09:46:13.841682+0000","last_fullsized":"2023-08-25T09:46:13.841682+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:47.969760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837422+0000","last_change":"2023-08-25T09:46:13.837329+0000","last_active":"2023-08-25T09:46:13.837422+0000","last_peered":"2023-08-25T09:46:13.837422+0000","last_clean":"2023-08-25T09:46:13.837422+0000","last_became_active":"2023-08-25T09:46:13.837038+0000","last_became_peered":"2023-08-25T09:46:13.837038+0000","last_unstale":"2023-08-25T09:46:13.837422+0000","last_undegraded":"2023-08-25T09:46:13.837422+0000","last_fullsized":"2023-08-25T09:46:13.837422+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:47:16.394476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.855844+0000","last_change":"2023-08-25T09:46:13.855298+0000","last_active":"2023-08-25T09:46:13.855844+0000","last_peered":"2023-08-25T09:46:13.855844+0000","last_clean":"2023-08-25T09:46:13.855844+0000","last_became_active":"2023-08-25T09:46:13.855025+0000","last_became_peered":"2023-08-25T09:46:13.855025+0000","last_unstale":"2023-08-25T09:46:13.855844+0000","last_undegraded":"2023-08-25T09:46:13.855844+0000","last_fullsized":"2023-08-25T09:46:13.855844+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:13.250629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853311+0000","last_change":"2023-08-25T09:46:13.852919+0000","last_active":"2023-08-25T09:46:13.853311+0000","last_peered":"2023-08-25T09:46:13.853311+0000","last_clean":"2023-08-25T09:46:13.853311+0000","last_became_active":"2023-08-25T09:46:13.852191+0000","last_became_peered":"2023-08-25T09:46:13.852191+0000","last_unstale":"2023-08-25T09:46:13.853311+0000","last_undegraded":"2023-08-25T09:46:13.853311+0000","last_fullsized":"2023-08-25T09:46:13.853311+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:46:33.084456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861004+0000","last_change":"2023-08-25T09:46:13.860886+0000","last_active":"2023-08-25T09:46:13.861004+0000","last_peered":"2023-08-25T09:46:13.861004+0000","last_clean":"2023-08-25T09:46:13.861004+0000","last_became_active":"2023-08-25T09:46:13.856276+0000","last_became_peered":"2023-08-25T09:46:13.856276+0000","last_unstale":"2023-08-25T09:46:13.861004+0000","last_undegraded":"2023-08-25T09:46:13.861004+0000","last_fullsized":"2023-08-25T09:46:13.861004+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:35:05.042117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842504+0000","last_change":"2023-08-25T09:46:13.841857+0000","last_active":"2023-08-25T09:46:13.842504+0000","last_peered":"2023-08-25T09:46:13.842504+0000","last_clean":"2023-08-25T09:46:13.842504+0000","last_became_active":"2023-08-25T09:46:13.841316+0000","last_became_peered":"2023-08-25T09:46:13.841316+0000","last_unstale":"2023-08-25T09:46:13.842504+0000","last_undegraded":"2023-08-25T09:46:13.842504+0000","last_fullsized":"2023-08-25T09:46:13.842504+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:46:12.295419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.856792+0000","last_change":"2023-08-25T09:46:13.856705+0000","last_active":"2023-08-25T09:46:13.856792+0000","last_peered":"2023-08-25T09:46:13.856792+0000","last_clean":"2023-08-25T09:46:13.856792+0000","last_became_active":"2023-08-25T09:46:13.856195+0000","last_became_peered":"2023-08-25T09:46:13.856195+0000","last_unstale":"2023-08-25T09:46:13.856792+0000","last_undegraded":"2023-08-25T09:46:13.856792+0000","last_fullsized":"2023-08-25T09:46:13.856792+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:10.404156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854392+0000","last_change":"2023-08-25T09:46:13.853493+0000","last_active":"2023-08-25T09:46:13.854392+0000","last_peered":"2023-08-25T09:46:13.854392+0000","last_clean":"2023-08-25T09:46:13.854392+0000","last_became_active":"2023-08-25T09:46:13.853149+0000","last_became_peered":"2023-08-25T09:46:13.853149+0000","last_unstale":"2023-08-25T09:46:13.854392+0000","last_undegraded":"2023-08-25T09:46:13.854392+0000","last_fullsized":"2023-08-25T09:46:13.854392+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:43.082659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836016+0000","last_change":"2023-08-25T09:46:13.835272+0000","last_active":"2023-08-25T09:46:13.836016+0000","last_peered":"2023-08-25T09:46:13.836016+0000","last_clean":"2023-08-25T09:46:13.836016+0000","last_became_active":"2023-08-25T09:46:13.834977+0000","last_became_peered":"2023-08-25T09:46:13.834977+0000","last_unstale":"2023-08-25T09:46:13.836016+0000","last_undegraded":"2023-08-25T09:46:13.836016+0000","last_fullsized":"2023-08-25T09:46:13.836016+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:59:53.950833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862636+0000","last_change":"2023-08-25T09:46:13.862068+0000","last_active":"2023-08-25T09:46:13.862636+0000","last_peered":"2023-08-25T09:46:13.862636+0000","last_clean":"2023-08-25T09:46:13.862636+0000","last_became_active":"2023-08-25T09:46:13.859588+0000","last_became_peered":"2023-08-25T09:46:13.859588+0000","last_unstale":"2023-08-25T09:46:13.862636+0000","last_undegraded":"2023-08-25T09:46:13.862636+0000","last_fullsized":"2023-08-25T09:46:13.862636+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:06:02.889505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845157+0000","last_change":"2023-08-25T09:46:13.844903+0000","last_active":"2023-08-25T09:46:13.845157+0000","last_peered":"2023-08-25T09:46:13.845157+0000","last_clean":"2023-08-25T09:46:13.845157+0000","last_became_active":"2023-08-25T09:46:13.844627+0000","last_became_peered":"2023-08-25T09:46:13.844627+0000","last_unstale":"2023-08-25T09:46:13.845157+0000","last_undegraded":"2023-08-25T09:46:13.845157+0000","last_fullsized":"2023-08-25T09:46:13.845157+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:17:15.261803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845669+0000","last_change":"2023-08-25T09:46:13.845521+0000","last_active":"2023-08-25T09:46:13.845669+0000","last_peered":"2023-08-25T09:46:13.845669+0000","last_clean":"2023-08-25T09:46:13.845669+0000","last_became_active":"2023-08-25T09:46:13.845045+0000","last_became_peered":"2023-08-25T09:46:13.845045+0000","last_unstale":"2023-08-25T09:46:13.845669+0000","last_undegraded":"2023-08-25T09:46:13.845669+0000","last_fullsized":"2023-08-25T09:46:13.845669+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:18.897684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860826+0000","last_change":"2023-08-25T09:46:13.860719+0000","last_active":"2023-08-25T09:46:13.860826+0000","last_peered":"2023-08-25T09:46:13.860826+0000","last_clean":"2023-08-25T09:46:13.860826+0000","last_became_active":"2023-08-25T09:46:13.853489+0000","last_became_peered":"2023-08-25T09:46:13.853489+0000","last_unstale":"2023-08-25T09:46:13.860826+0000","last_undegraded":"2023-08-25T09:46:13.860826+0000","last_fullsized":"2023-08-25T09:46:13.860826+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:15:54.817525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845590+0000","last_change":"2023-08-25T09:46:13.845316+0000","last_active":"2023-08-25T09:46:13.845590+0000","last_peered":"2023-08-25T09:46:13.845590+0000","last_clean":"2023-08-25T09:46:13.845590+0000","last_became_active":"2023-08-25T09:46:13.843482+0000","last_became_peered":"2023-08-25T09:46:13.843482+0000","last_unstale":"2023-08-25T09:46:13.845590+0000","last_undegraded":"2023-08-25T09:46:13.845590+0000","last_fullsized":"2023-08-25T09:46:13.845590+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:26.953645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842416+0000","last_change":"2023-08-25T09:46:13.842254+0000","last_active":"2023-08-25T09:46:13.842416+0000","last_peered":"2023-08-25T09:46:13.842416+0000","last_clean":"2023-08-25T09:46:13.842416+0000","last_became_active":"2023-08-25T09:46:13.841795+0000","last_became_peered":"2023-08-25T09:46:13.841795+0000","last_unstale":"2023-08-25T09:46:13.842416+0000","last_undegraded":"2023-08-25T09:46:13.842416+0000","last_fullsized":"2023-08-25T09:46:13.842416+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:33.117508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.841784+0000","last_change":"2023-08-25T09:46:13.841555+0000","last_active":"2023-08-25T09:46:13.841784+0000","last_peered":"2023-08-25T09:46:13.841784+0000","last_clean":"2023-08-25T09:46:13.841784+0000","last_became_active":"2023-08-25T09:46:13.841262+0000","last_became_peered":"2023-08-25T09:46:13.841262+0000","last_unstale":"2023-08-25T09:46:13.841784+0000","last_undegraded":"2023-08-25T09:46:13.841784+0000","last_fullsized":"2023-08-25T09:46:13.841784+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:54:34.882076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.844071+0000","last_change":"2023-08-25T09:46:13.843898+0000","last_active":"2023-08-25T09:46:13.844071+0000","last_peered":"2023-08-25T09:46:13.844071+0000","last_clean":"2023-08-25T09:46:13.844071+0000","last_became_active":"2023-08-25T09:46:13.843646+0000","last_became_peered":"2023-08-25T09:46:13.843646+0000","last_unstale":"2023-08-25T09:46:13.844071+0000","last_undegraded":"2023-08-25T09:46:13.844071+0000","last_fullsized":"2023-08-25T09:46:13.844071+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:24:31.987367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.841967+0000","last_change":"2023-08-25T09:46:13.841873+0000","last_active":"2023-08-25T09:46:13.841967+0000","last_peered":"2023-08-25T09:46:13.841967+0000","last_clean":"2023-08-25T09:46:13.841967+0000","last_became_active":"2023-08-25T09:46:13.841298+0000","last_became_peered":"2023-08-25T09:46:13.841298+0000","last_unstale":"2023-08-25T09:46:13.841967+0000","last_undegraded":"2023-08-25T09:46:13.841967+0000","last_fullsized":"2023-08-25T09:46:13.841967+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:56.784316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853597+0000","last_change":"2023-08-25T09:46:13.853496+0000","last_active":"2023-08-25T09:46:13.853597+0000","last_peered":"2023-08-25T09:46:13.853597+0000","last_clean":"2023-08-25T09:46:13.853597+0000","last_became_active":"2023-08-25T09:46:13.853245+0000","last_became_peered":"2023-08-25T09:46:13.853245+0000","last_unstale":"2023-08-25T09:46:13.853597+0000","last_undegraded":"2023-08-25T09:46:13.853597+0000","last_fullsized":"2023-08-25T09:46:13.853597+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:33.068910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854702+0000","last_change":"2023-08-25T09:46:13.854531+0000","last_active":"2023-08-25T09:46:13.854702+0000","last_peered":"2023-08-25T09:46:13.854702+0000","last_clean":"2023-08-25T09:46:13.854702+0000","last_became_active":"2023-08-25T09:46:13.853351+0000","last_became_peered":"2023-08-25T09:46:13.853351+0000","last_unstale":"2023-08-25T09:46:13.854702+0000","last_undegraded":"2023-08-25T09:46:13.854702+0000","last_fullsized":"2023-08-25T09:46:13.854702+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:10.427818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.840288+0000","last_change":"2023-08-25T09:46:13.840029+0000","last_active":"2023-08-25T09:46:13.840288+0000","last_peered":"2023-08-25T09:46:13.840288+0000","last_clean":"2023-08-25T09:46:13.840288+0000","last_became_active":"2023-08-25T09:46:13.839139+0000","last_became_peered":"2023-08-25T09:46:13.839139+0000","last_unstale":"2023-08-25T09:46:13.840288+0000","last_undegraded":"2023-08-25T09:46:13.840288+0000","last_fullsized":"2023-08-25T09:46:13.840288+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:14.126703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854468+0000","last_change":"2023-08-25T09:46:13.854407+0000","last_active":"2023-08-25T09:46:13.854468+0000","last_peered":"2023-08-25T09:46:13.854468+0000","last_clean":"2023-08-25T09:46:13.854468+0000","last_became_active":"2023-08-25T09:46:13.854123+0000","last_became_peered":"2023-08-25T09:46:13.854123+0000","last_unstale":"2023-08-25T09:46:13.854468+0000","last_undegraded":"2023-08-25T09:46:13.854468+0000","last_fullsized":"2023-08-25T09:46:13.854468+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:14.437683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.848739+0000","last_change":"2023-08-25T09:46:13.848242+0000","last_active":"2023-08-25T09:46:13.848739+0000","last_peered":"2023-08-25T09:46:13.848739+0000","last_clean":"2023-08-25T09:46:13.848739+0000","last_became_active":"2023-08-25T09:46:13.847826+0000","last_became_peered":"2023-08-25T09:46:13.847826+0000","last_unstale":"2023-08-25T09:46:13.848739+0000","last_undegraded":"2023-08-25T09:46:13.848739+0000","last_fullsized":"2023-08-25T09:46:13.848739+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:37:08.773278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854415+0000","last_change":"2023-08-25T09:46:13.854048+0000","last_active":"2023-08-25T09:46:13.854415+0000","last_peered":"2023-08-25T09:46:13.854415+0000","last_clean":"2023-08-25T09:46:13.854415+0000","last_became_active":"2023-08-25T09:46:13.853308+0000","last_became_peered":"2023-08-25T09:46:13.853308+0000","last_unstale":"2023-08-25T09:46:13.854415+0000","last_undegraded":"2023-08-25T09:46:13.854415+0000","last_fullsized":"2023-08-25T09:46:13.854415+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:06.285281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"132'4","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858589+0000","last_change":"2023-08-25T09:46:12.837594+0000","last_active":"2023-08-25T09:46:13.858589+0000","last_peered":"2023-08-25T09:46:13.858589+0000","last_clean":"2023-08-25T09:46:13.858589+0000","last_became_active":"2023-08-25T09:46:12.837159+0000","last_became_peered":"2023-08-25T09:46:12.837159+0000","last_unstale":"2023-08-25T09:46:13.858589+0000","last_undegraded":"2023-08-25T09:46:13.858589+0000","last_fullsized":"2023-08-25T09:46:13.858589+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:35.672374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.846063+0000","last_change":"2023-08-25T09:46:13.845952+0000","last_active":"2023-08-25T09:46:13.846063+0000","last_peered":"2023-08-25T09:46:13.846063+0000","last_clean":"2023-08-25T09:46:13.846063+0000","last_became_active":"2023-08-25T09:46:13.844786+0000","last_became_peered":"2023-08-25T09:46:13.844786+0000","last_unstale":"2023-08-25T09:46:13.846063+0000","last_undegraded":"2023-08-25T09:46:13.846063+0000","last_fullsized":"2023-08-25T09:46:13.846063+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:00:59.929787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862339+0000","last_change":"2023-08-25T09:46:12.844400+0000","last_active":"2023-08-25T09:46:13.862339+0000","last_peered":"2023-08-25T09:46:13.862339+0000","last_clean":"2023-08-25T09:46:13.862339+0000","last_became_active":"2023-08-25T09:46:12.842665+0000","last_became_peered":"2023-08-25T09:46:12.842665+0000","last_unstale":"2023-08-25T09:46:13.862339+0000","last_undegraded":"2023-08-25T09:46:13.862339+0000","last_fullsized":"2023-08-25T09:46:13.862339+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:25.432915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.839926+0000","last_change":"2023-08-25T09:46:13.839824+0000","last_active":"2023-08-25T09:46:13.839926+0000","last_peered":"2023-08-25T09:46:13.839926+0000","last_clean":"2023-08-25T09:46:13.839926+0000","last_became_active":"2023-08-25T09:46:13.839537+0000","last_became_peered":"2023-08-25T09:46:13.839537+0000","last_unstale":"2023-08-25T09:46:13.839926+0000","last_undegraded":"2023-08-25T09:46:13.839926+0000","last_fullsized":"2023-08-25T09:46:13.839926+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:07:21.793789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860568+0000","last_change":"2023-08-25T09:46:12.842539+0000","last_active":"2023-08-25T09:46:13.860568+0000","last_peered":"2023-08-25T09:46:13.860568+0000","last_clean":"2023-08-25T09:46:13.860568+0000","last_became_active":"2023-08-25T09:46:12.842300+0000","last_became_peered":"2023-08-25T09:46:12.842300+0000","last_unstale":"2023-08-25T09:46:13.860568+0000","last_undegraded":"2023-08-25T09:46:13.860568+0000","last_fullsized":"2023-08-25T09:46:13.860568+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:12:44.732933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853424+0000","last_change":"2023-08-25T09:46:13.853308+0000","last_active":"2023-08-25T09:46:13.853424+0000","last_peered":"2023-08-25T09:46:13.853424+0000","last_clean":"2023-08-25T09:46:13.853424+0000","last_became_active":"2023-08-25T09:46:13.853058+0000","last_became_peered":"2023-08-25T09:46:13.853058+0000","last_unstale":"2023-08-25T09:46:13.853424+0000","last_undegraded":"2023-08-25T09:46:13.853424+0000","last_fullsized":"2023-08-25T09:46:13.853424+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:28:52.961461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861676+0000","last_change":"2023-08-25T09:46:12.845899+0000","last_active":"2023-08-25T09:46:13.861676+0000","last_peered":"2023-08-25T09:46:13.861676+0000","last_clean":"2023-08-25T09:46:13.861676+0000","last_became_active":"2023-08-25T09:46:12.844324+0000","last_became_peered":"2023-08-25T09:46:12.844324+0000","last_unstale":"2023-08-25T09:46:13.861676+0000","last_undegraded":"2023-08-25T09:46:13.861676+0000","last_fullsized":"2023-08-25T09:46:13.861676+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:51.985346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861509+0000","last_change":"2023-08-25T09:46:13.860977+0000","last_active":"2023-08-25T09:46:13.861509+0000","last_peered":"2023-08-25T09:46:13.861509+0000","last_clean":"2023-08-25T09:46:13.861509+0000","last_became_active":"2023-08-25T09:46:13.859736+0000","last_became_peered":"2023-08-25T09:46:13.859736+0000","last_unstale":"2023-08-25T09:46:13.861509+0000","last_undegraded":"2023-08-25T09:46:13.861509+0000","last_fullsized":"2023-08-25T09:46:13.861509+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:27:09.256131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.857628+0000","last_change":"2023-08-25T09:46:12.840203+0000","last_active":"2023-08-25T09:46:13.857628+0000","last_peered":"2023-08-25T09:46:13.857628+0000","last_clean":"2023-08-25T09:46:13.857628+0000","last_became_active":"2023-08-25T09:46:12.839865+0000","last_became_peered":"2023-08-25T09:46:12.839865+0000","last_unstale":"2023-08-25T09:46:13.857628+0000","last_undegraded":"2023-08-25T09:46:13.857628+0000","last_fullsized":"2023-08-25T09:46:13.857628+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:49:30.054349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860451+0000","last_change":"2023-08-25T09:46:13.860372+0000","last_active":"2023-08-25T09:46:13.860451+0000","last_peered":"2023-08-25T09:46:13.860451+0000","last_clean":"2023-08-25T09:46:13.860451+0000","last_became_active":"2023-08-25T09:46:13.856515+0000","last_became_peered":"2023-08-25T09:46:13.856515+0000","last_unstale":"2023-08-25T09:46:13.860451+0000","last_undegraded":"2023-08-25T09:46:13.860451+0000","last_fullsized":"2023-08-25T09:46:13.860451+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:58.562158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858853+0000","last_change":"2023-08-25T09:46:12.842406+0000","last_active":"2023-08-25T09:46:13.858853+0000","last_peered":"2023-08-25T09:46:13.858853+0000","last_clean":"2023-08-25T09:46:13.858853+0000","last_became_active":"2023-08-25T09:46:12.842102+0000","last_became_peered":"2023-08-25T09:46:12.842102+0000","last_unstale":"2023-08-25T09:46:13.858853+0000","last_undegraded":"2023-08-25T09:46:13.858853+0000","last_fullsized":"2023-08-25T09:46:13.858853+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:27:26.027427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842391+0000","last_change":"2023-08-25T09:46:13.841967+0000","last_active":"2023-08-25T09:46:13.842391+0000","last_peered":"2023-08-25T09:46:13.842391+0000","last_clean":"2023-08-25T09:46:13.842391+0000","last_became_active":"2023-08-25T09:46:13.841735+0000","last_became_peered":"2023-08-25T09:46:13.841735+0000","last_unstale":"2023-08-25T09:46:13.842391+0000","last_undegraded":"2023-08-25T09:46:13.842391+0000","last_fullsized":"2023-08-25T09:46:13.842391+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:28.200630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.838120+0000","last_change":"2023-08-25T09:46:12.844774+0000","last_active":"2023-08-25T09:46:13.838120+0000","last_peered":"2023-08-25T09:46:13.838120+0000","last_clean":"2023-08-25T09:46:13.838120+0000","last_became_active":"2023-08-25T09:46:12.844487+0000","last_became_peered":"2023-08-25T09:46:12.844487+0000","last_unstale":"2023-08-25T09:46:13.838120+0000","last_undegraded":"2023-08-25T09:46:13.838120+0000","last_fullsized":"2023-08-25T09:46:13.838120+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:54:35.314750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852488+0000","last_change":"2023-08-25T09:46:13.850782+0000","last_active":"2023-08-25T09:46:13.852488+0000","last_peered":"2023-08-25T09:46:13.852488+0000","last_clean":"2023-08-25T09:46:13.852488+0000","last_became_active":"2023-08-25T09:46:13.850179+0000","last_became_peered":"2023-08-25T09:46:13.850179+0000","last_unstale":"2023-08-25T09:46:13.852488+0000","last_undegraded":"2023-08-25T09:46:13.852488+0000","last_fullsized":"2023-08-25T09:46:13.852488+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:00:32.388159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.846809+0000","last_change":"2023-08-25T09:46:12.848337+0000","last_active":"2023-08-25T09:46:13.846809+0000","last_peered":"2023-08-25T09:46:13.846809+0000","last_clean":"2023-08-25T09:46:13.846809+0000","last_became_active":"2023-08-25T09:46:12.847749+0000","last_became_peered":"2023-08-25T09:46:12.847749+0000","last_unstale":"2023-08-25T09:46:13.846809+0000","last_undegraded":"2023-08-25T09:46:13.846809+0000","last_fullsized":"2023-08-25T09:46:13.846809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:56.362629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852539+0000","last_change":"2023-08-25T09:46:13.851570+0000","last_active":"2023-08-25T09:46:13.852539+0000","last_peered":"2023-08-25T09:46:13.852539+0000","last_clean":"2023-08-25T09:46:13.852539+0000","last_became_active":"2023-08-25T09:46:13.850670+0000","last_became_peered":"2023-08-25T09:46:13.850670+0000","last_unstale":"2023-08-25T09:46:13.852539+0000","last_undegraded":"2023-08-25T09:46:13.852539+0000","last_fullsized":"2023-08-25T09:46:13.852539+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:01:50.987872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.847838+0000","last_change":"2023-08-25T09:46:12.837637+0000","last_active":"2023-08-25T09:46:13.847838+0000","last_peered":"2023-08-25T09:46:13.847838+0000","last_clean":"2023-08-25T09:46:13.847838+0000","last_became_active":"2023-08-25T09:46:12.837142+0000","last_became_peered":"2023-08-25T09:46:12.837142+0000","last_unstale":"2023-08-25T09:46:13.847838+0000","last_undegraded":"2023-08-25T09:46:13.847838+0000","last_fullsized":"2023-08-25T09:46:13.847838+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:48.099737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860332+0000","last_change":"2023-08-25T09:46:13.860023+0000","last_active":"2023-08-25T09:46:13.860332+0000","last_peered":"2023-08-25T09:46:13.860332+0000","last_clean":"2023-08-25T09:46:13.860332+0000","last_became_active":"2023-08-25T09:46:13.849300+0000","last_became_peered":"2023-08-25T09:46:13.849300+0000","last_unstale":"2023-08-25T09:46:13.860332+0000","last_undegraded":"2023-08-25T09:46:13.860332+0000","last_fullsized":"2023-08-25T09:46:13.860332+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:04.821820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.835958+0000","last_change":"2023-08-25T09:46:12.847429+0000","last_active":"2023-08-25T09:46:13.835958+0000","last_peered":"2023-08-25T09:46:13.835958+0000","last_clean":"2023-08-25T09:46:13.835958+0000","last_became_active":"2023-08-25T09:46:12.846940+0000","last_became_peered":"2023-08-25T09:46:12.846940+0000","last_unstale":"2023-08-25T09:46:13.835958+0000","last_undegraded":"2023-08-25T09:46:13.835958+0000","last_fullsized":"2023-08-25T09:46:13.835958+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:24:26.779363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860666+0000","last_change":"2023-08-25T09:46:13.860500+0000","last_active":"2023-08-25T09:46:13.860666+0000","last_peered":"2023-08-25T09:46:13.860666+0000","last_clean":"2023-08-25T09:46:13.860666+0000","last_became_active":"2023-08-25T09:46:13.860258+0000","last_became_peered":"2023-08-25T09:46:13.860258+0000","last_unstale":"2023-08-25T09:46:13.860666+0000","last_undegraded":"2023-08-25T09:46:13.860666+0000","last_fullsized":"2023-08-25T09:46:13.860666+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:52:34.877190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"132'3","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858266+0000","last_change":"2023-08-25T09:46:12.828008+0000","last_active":"2023-08-25T09:46:13.858266+0000","last_peered":"2023-08-25T09:46:13.858266+0000","last_clean":"2023-08-25T09:46:13.858266+0000","last_became_active":"2023-08-25T09:46:12.827587+0000","last_became_peered":"2023-08-25T09:46:12.827587+0000","last_unstale":"2023-08-25T09:46:13.858266+0000","last_undegraded":"2023-08-25T09:46:13.858266+0000","last_fullsized":"2023-08-25T09:46:13.858266+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:37:00.505165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854933+0000","last_change":"2023-08-25T09:46:13.854489+0000","last_active":"2023-08-25T09:46:13.854933+0000","last_peered":"2023-08-25T09:46:13.854933+0000","last_clean":"2023-08-25T09:46:13.854933+0000","last_became_active":"2023-08-25T09:46:13.854221+0000","last_became_peered":"2023-08-25T09:46:13.854221+0000","last_unstale":"2023-08-25T09:46:13.854933+0000","last_undegraded":"2023-08-25T09:46:13.854933+0000","last_fullsized":"2023-08-25T09:46:13.854933+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:31:53.955946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"132'6","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862688+0000","last_change":"2023-08-25T09:46:12.845025+0000","last_active":"2023-08-25T09:46:13.862688+0000","last_peered":"2023-08-25T09:46:13.862688+0000","last_clean":"2023-08-25T09:46:13.862688+0000","last_became_active":"2023-08-25T09:46:12.844573+0000","last_became_peered":"2023-08-25T09:46:12.844573+0000","last_unstale":"2023-08-25T09:46:13.862688+0000","last_undegraded":"2023-08-25T09:46:13.862688+0000","last_fullsized":"2023-08-25T09:46:13.862688+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:29:34.685465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836091+0000","last_change":"2023-08-25T09:46:13.835776+0000","last_active":"2023-08-25T09:46:13.836091+0000","last_peered":"2023-08-25T09:46:13.836091+0000","last_clean":"2023-08-25T09:46:13.836091+0000","last_became_active":"2023-08-25T09:46:13.835514+0000","last_became_peered":"2023-08-25T09:46:13.835514+0000","last_unstale":"2023-08-25T09:46:13.836091+0000","last_undegraded":"2023-08-25T09:46:13.836091+0000","last_fullsized":"2023-08-25T09:46:13.836091+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:12:05.190445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.865675+0000","last_change":"2023-08-25T09:46:12.850998+0000","last_active":"2023-08-25T09:46:13.865675+0000","last_peered":"2023-08-25T09:46:13.865675+0000","last_clean":"2023-08-25T09:46:13.865675+0000","last_became_active":"2023-08-25T09:46:12.848344+0000","last_became_peered":"2023-08-25T09:46:12.848344+0000","last_unstale":"2023-08-25T09:46:13.865675+0000","last_undegraded":"2023-08-25T09:46:13.865675+0000","last_fullsized":"2023-08-25T09:46:13.865675+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:13:33.371729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.843751+0000","last_change":"2023-08-25T09:46:13.843518+0000","last_active":"2023-08-25T09:46:13.843751+0000","last_peered":"2023-08-25T09:46:13.843751+0000","last_clean":"2023-08-25T09:46:13.843751+0000","last_became_active":"2023-08-25T09:46:13.843246+0000","last_became_peered":"2023-08-25T09:46:13.843246+0000","last_unstale":"2023-08-25T09:46:13.843751+0000","last_undegraded":"2023-08-25T09:46:13.843751+0000","last_fullsized":"2023-08-25T09:46:13.843751+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:06:39.875204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836274+0000","last_change":"2023-08-25T09:46:12.846154+0000","last_active":"2023-08-25T09:46:13.836274+0000","last_peered":"2023-08-25T09:46:13.836274+0000","last_clean":"2023-08-25T09:46:13.836274+0000","last_became_active":"2023-08-25T09:46:12.844908+0000","last_became_peered":"2023-08-25T09:46:12.844908+0000","last_unstale":"2023-08-25T09:46:13.836274+0000","last_undegraded":"2023-08-25T09:46:13.836274+0000","last_fullsized":"2023-08-25T09:46:13.836274+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:21:31.715559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860203+0000","last_change":"2023-08-25T09:46:13.859633+0000","last_active":"2023-08-25T09:46:13.860203+0000","last_peered":"2023-08-25T09:46:13.860203+0000","last_clean":"2023-08-25T09:46:13.860203+0000","last_became_active":"2023-08-25T09:46:13.859231+0000","last_became_peered":"2023-08-25T09:46:13.859231+0000","last_unstale":"2023-08-25T09:46:13.860203+0000","last_undegraded":"2023-08-25T09:46:13.860203+0000","last_fullsized":"2023-08-25T09:46:13.860203+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:43.082659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.827940+0000","last_change":"2023-08-25T09:46:12.846602+0000","last_active":"2023-08-25T09:46:13.827940+0000","last_peered":"2023-08-25T09:46:13.827940+0000","last_clean":"2023-08-25T09:46:13.827940+0000","last_became_active":"2023-08-25T09:46:12.846000+0000","last_became_peered":"2023-08-25T09:46:12.846000+0000","last_unstale":"2023-08-25T09:46:13.827940+0000","last_undegraded":"2023-08-25T09:46:13.827940+0000","last_fullsized":"2023-08-25T09:46:13.827940+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:50:05.747460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854435+0000","last_change":"2023-08-25T09:46:13.854222+0000","last_active":"2023-08-25T09:46:13.854435+0000","last_peered":"2023-08-25T09:46:13.854435+0000","last_clean":"2023-08-25T09:46:13.854435+0000","last_became_active":"2023-08-25T09:46:13.853725+0000","last_became_peered":"2023-08-25T09:46:13.853725+0000","last_unstale":"2023-08-25T09:46:13.854435+0000","last_undegraded":"2023-08-25T09:46:13.854435+0000","last_fullsized":"2023-08-25T09:46:13.854435+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:45:06.643619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"132'8","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.855899+0000","last_change":"2023-08-25T09:46:12.841852+0000","last_active":"2023-08-25T09:46:13.855899+0000","last_peered":"2023-08-25T09:46:13.855899+0000","last_clean":"2023-08-25T09:46:13.855899+0000","last_became_active":"2023-08-25T09:46:12.841528+0000","last_became_peered":"2023-08-25T09:46:12.841528+0000","last_unstale":"2023-08-25T09:46:13.855899+0000","last_undegraded":"2023-08-25T09:46:13.855899+0000","last_fullsized":"2023-08-25T09:46:13.855899+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:10:00.933841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.851100+0000","last_change":"2023-08-25T09:46:13.850722+0000","last_active":"2023-08-25T09:46:13.851100+0000","last_peered":"2023-08-25T09:46:13.851100+0000","last_clean":"2023-08-25T09:46:13.851100+0000","last_became_active":"2023-08-25T09:46:13.850323+0000","last_became_peered":"2023-08-25T09:46:13.850323+0000","last_unstale":"2023-08-25T09:46:13.851100+0000","last_undegraded":"2023-08-25T09:46:13.851100+0000","last_fullsized":"2023-08-25T09:46:13.851100+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:21:42.100149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861243+0000","last_change":"2023-08-25T09:46:12.845881+0000","last_active":"2023-08-25T09:46:13.861243+0000","last_peered":"2023-08-25T09:46:13.861243+0000","last_clean":"2023-08-25T09:46:13.861243+0000","last_became_active":"2023-08-25T09:46:12.845604+0000","last_became_peered":"2023-08-25T09:46:12.845604+0000","last_unstale":"2023-08-25T09:46:13.861243+0000","last_undegraded":"2023-08-25T09:46:13.861243+0000","last_fullsized":"2023-08-25T09:46:13.861243+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:56.362629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845957+0000","last_change":"2023-08-25T09:46:13.845840+0000","last_active":"2023-08-25T09:46:13.845957+0000","last_peered":"2023-08-25T09:46:13.845957+0000","last_clean":"2023-08-25T09:46:13.845957+0000","last_became_active":"2023-08-25T09:46:13.845608+0000","last_became_peered":"2023-08-25T09:46:13.845608+0000","last_unstale":"2023-08-25T09:46:13.845957+0000","last_undegraded":"2023-08-25T09:46:13.845957+0000","last_fullsized":"2023-08-25T09:46:13.845957+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:29:09.902573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.834911+0000","last_change":"2023-08-25T09:46:12.841006+0000","last_active":"2023-08-25T09:46:13.834911+0000","last_peered":"2023-08-25T09:46:13.834911+0000","last_clean":"2023-08-25T09:46:13.834911+0000","last_became_active":"2023-08-25T09:46:12.833094+0000","last_became_peered":"2023-08-25T09:46:12.833094+0000","last_unstale":"2023-08-25T09:46:13.834911+0000","last_undegraded":"2023-08-25T09:46:13.834911+0000","last_fullsized":"2023-08-25T09:46:13.834911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:35.672374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860862+0000","last_change":"2023-08-25T09:46:13.860715+0000","last_active":"2023-08-25T09:46:13.860862+0000","last_peered":"2023-08-25T09:46:13.860862+0000","last_clean":"2023-08-25T09:46:13.860862+0000","last_became_active":"2023-08-25T09:46:13.860124+0000","last_became_peered":"2023-08-25T09:46:13.860124+0000","last_unstale":"2023-08-25T09:46:13.860862+0000","last_undegraded":"2023-08-25T09:46:13.860862+0000","last_fullsized":"2023-08-25T09:46:13.860862+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:48.476261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833317+0000","last_change":"2023-08-25T09:46:12.841401+0000","last_active":"2023-08-25T09:46:13.833317+0000","last_peered":"2023-08-25T09:46:13.833317+0000","last_clean":"2023-08-25T09:46:13.833317+0000","last_became_active":"2023-08-25T09:46:12.840778+0000","last_became_peered":"2023-08-25T09:46:12.840778+0000","last_unstale":"2023-08-25T09:46:13.833317+0000","last_undegraded":"2023-08-25T09:46:13.833317+0000","last_fullsized":"2023-08-25T09:46:13.833317+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:48.141548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.842250+0000","last_change":"2023-08-25T09:46:13.842044+0000","last_active":"2023-08-25T09:46:13.842250+0000","last_peered":"2023-08-25T09:46:13.842250+0000","last_clean":"2023-08-25T09:46:13.842250+0000","last_became_active":"2023-08-25T09:46:13.841629+0000","last_became_peered":"2023-08-25T09:46:13.841629+0000","last_unstale":"2023-08-25T09:46:13.842250+0000","last_undegraded":"2023-08-25T09:46:13.842250+0000","last_fullsized":"2023-08-25T09:46:13.842250+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:40.787728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833774+0000","last_change":"2023-08-25T09:46:12.851464+0000","last_active":"2023-08-25T09:46:13.833774+0000","last_peered":"2023-08-25T09:46:13.833774+0000","last_clean":"2023-08-25T09:46:13.833774+0000","last_became_active":"2023-08-25T09:46:12.850409+0000","last_became_peered":"2023-08-25T09:46:12.850409+0000","last_unstale":"2023-08-25T09:46:13.833774+0000","last_undegraded":"2023-08-25T09:46:13.833774+0000","last_fullsized":"2023-08-25T09:46:13.833774+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:59:08.643583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.855752+0000","last_change":"2023-08-25T09:46:13.855661+0000","last_active":"2023-08-25T09:46:13.855752+0000","last_peered":"2023-08-25T09:46:13.855752+0000","last_clean":"2023-08-25T09:46:13.855752+0000","last_became_active":"2023-08-25T09:46:13.855350+0000","last_became_peered":"2023-08-25T09:46:13.855350+0000","last_unstale":"2023-08-25T09:46:13.855752+0000","last_undegraded":"2023-08-25T09:46:13.855752+0000","last_fullsized":"2023-08-25T09:46:13.855752+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:47.738941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.834084+0000","last_change":"2023-08-25T09:46:12.849108+0000","last_active":"2023-08-25T09:46:13.834084+0000","last_peered":"2023-08-25T09:46:13.834084+0000","last_clean":"2023-08-25T09:46:13.834084+0000","last_became_active":"2023-08-25T09:46:12.848826+0000","last_became_peered":"2023-08-25T09:46:12.848826+0000","last_unstale":"2023-08-25T09:46:13.834084+0000","last_undegraded":"2023-08-25T09:46:13.834084+0000","last_fullsized":"2023-08-25T09:46:13.834084+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:53:28.856588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.862527+0000","last_change":"2023-08-25T09:46:13.862062+0000","last_active":"2023-08-25T09:46:13.862527+0000","last_peered":"2023-08-25T09:46:13.862527+0000","last_clean":"2023-08-25T09:46:13.862527+0000","last_became_active":"2023-08-25T09:46:13.860672+0000","last_became_peered":"2023-08-25T09:46:13.860672+0000","last_unstale":"2023-08-25T09:46:13.862527+0000","last_undegraded":"2023-08-25T09:46:13.862527+0000","last_fullsized":"2023-08-25T09:46:13.862527+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:51:35.329934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837155+0000","last_change":"2023-08-25T09:46:12.839898+0000","last_active":"2023-08-25T09:46:13.837155+0000","last_peered":"2023-08-25T09:46:13.837155+0000","last_clean":"2023-08-25T09:46:13.837155+0000","last_became_active":"2023-08-25T09:46:12.834307+0000","last_became_peered":"2023-08-25T09:46:12.834307+0000","last_unstale":"2023-08-25T09:46:13.837155+0000","last_undegraded":"2023-08-25T09:46:13.837155+0000","last_fullsized":"2023-08-25T09:46:13.837155+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:39:02.892983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852078+0000","last_change":"2023-08-25T09:46:13.851428+0000","last_active":"2023-08-25T09:46:13.852078+0000","last_peered":"2023-08-25T09:46:13.852078+0000","last_clean":"2023-08-25T09:46:13.852078+0000","last_became_active":"2023-08-25T09:46:13.850982+0000","last_became_peered":"2023-08-25T09:46:13.850982+0000","last_unstale":"2023-08-25T09:46:13.852078+0000","last_undegraded":"2023-08-25T09:46:13.852078+0000","last_fullsized":"2023-08-25T09:46:13.852078+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:13.085449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.839768+0000","last_change":"2023-08-25T09:46:12.848152+0000","last_active":"2023-08-25T09:46:13.839768+0000","last_peered":"2023-08-25T09:46:13.839768+0000","last_clean":"2023-08-25T09:46:13.839768+0000","last_became_active":"2023-08-25T09:46:12.847883+0000","last_became_peered":"2023-08-25T09:46:12.847883+0000","last_unstale":"2023-08-25T09:46:13.839768+0000","last_undegraded":"2023-08-25T09:46:13.839768+0000","last_fullsized":"2023-08-25T09:46:13.839768+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:14:39.383791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.853263+0000","last_change":"2023-08-25T09:46:13.853177+0000","last_active":"2023-08-25T09:46:13.853263+0000","last_peered":"2023-08-25T09:46:13.853263+0000","last_clean":"2023-08-25T09:46:13.853263+0000","last_became_active":"2023-08-25T09:46:13.852940+0000","last_became_peered":"2023-08-25T09:46:13.852940+0000","last_unstale":"2023-08-25T09:46:13.853263+0000","last_undegraded":"2023-08-25T09:46:13.853263+0000","last_fullsized":"2023-08-25T09:46:13.853263+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:41:13.253957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.838303+0000","last_change":"2023-08-25T09:46:12.840878+0000","last_active":"2023-08-25T09:46:13.838303+0000","last_peered":"2023-08-25T09:46:13.838303+0000","last_clean":"2023-08-25T09:46:13.838303+0000","last_became_active":"2023-08-25T09:46:12.833247+0000","last_became_peered":"2023-08-25T09:46:12.833247+0000","last_unstale":"2023-08-25T09:46:13.838303+0000","last_undegraded":"2023-08-25T09:46:13.838303+0000","last_fullsized":"2023-08-25T09:46:13.838303+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:00:40.793263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861425+0000","last_change":"2023-08-25T09:46:13.861304+0000","last_active":"2023-08-25T09:46:13.861425+0000","last_peered":"2023-08-25T09:46:13.861425+0000","last_clean":"2023-08-25T09:46:13.861425+0000","last_became_active":"2023-08-25T09:46:13.856296+0000","last_became_peered":"2023-08-25T09:46:13.856296+0000","last_unstale":"2023-08-25T09:46:13.861425+0000","last_undegraded":"2023-08-25T09:46:13.861425+0000","last_fullsized":"2023-08-25T09:46:13.861425+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:40.300356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.864417+0000","last_change":"2023-08-25T09:46:12.851284+0000","last_active":"2023-08-25T09:46:13.864417+0000","last_peered":"2023-08-25T09:46:13.864417+0000","last_clean":"2023-08-25T09:46:13.864417+0000","last_became_active":"2023-08-25T09:46:12.850203+0000","last_became_peered":"2023-08-25T09:46:12.850203+0000","last_unstale":"2023-08-25T09:46:13.864417+0000","last_undegraded":"2023-08-25T09:46:13.864417+0000","last_fullsized":"2023-08-25T09:46:13.864417+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:43.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":362,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.838223+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:46:13.838223+0000","last_peered":"2023-08-25T09:46:13.838223+0000","last_clean":"2023-08-25T09:46:13.838223+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:46:13.838223+0000","last_undegraded":"2023-08-25T09:46:13.838223+0000","last_fullsized":"2023-08-25T09:46:13.838223+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:39:02.299585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.834187+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:46:13.834187+0000","last_peered":"2023-08-25T09:46:13.834187+0000","last_clean":"2023-08-25T09:46:13.834187+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:46:13.834187+0000","last_undegraded":"2023-08-25T09:46:13.834187+0000","last_fullsized":"2023-08-25T09:46:13.834187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:39:48.229465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.856775+0000","last_change":"2023-08-25T09:46:13.856507+0000","last_active":"2023-08-25T09:46:13.856775+0000","last_peered":"2023-08-25T09:46:13.856775+0000","last_clean":"2023-08-25T09:46:13.856775+0000","last_became_active":"2023-08-25T09:46:13.855894+0000","last_became_peered":"2023-08-25T09:46:13.855894+0000","last_unstale":"2023-08-25T09:46:13.856775+0000","last_undegraded":"2023-08-25T09:46:13.856775+0000","last_fullsized":"2023-08-25T09:46:13.856775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:38:40.979862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.860616+0000","last_change":"2023-08-25T09:46:12.842641+0000","last_active":"2023-08-25T09:46:13.860616+0000","last_peered":"2023-08-25T09:46:13.860616+0000","last_clean":"2023-08-25T09:46:13.860616+0000","last_became_active":"2023-08-25T09:46:12.842361+0000","last_became_peered":"2023-08-25T09:46:12.842361+0000","last_unstale":"2023-08-25T09:46:13.860616+0000","last_undegraded":"2023-08-25T09:46:13.860616+0000","last_fullsized":"2023-08-25T09:46:13.860616+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:42:28.503309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833594+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:46:13.833594+0000","last_peered":"2023-08-25T09:46:13.833594+0000","last_clean":"2023-08-25T09:46:13.833594+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:46:13.833594+0000","last_undegraded":"2023-08-25T09:46:13.833594+0000","last_fullsized":"2023-08-25T09:46:13.833594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:36:23.556533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.859548+0000","last_change":"2023-08-25T09:46:13.859466+0000","last_active":"2023-08-25T09:46:13.859548+0000","last_peered":"2023-08-25T09:46:13.859548+0000","last_clean":"2023-08-25T09:46:13.859548+0000","last_became_active":"2023-08-25T09:46:13.858905+0000","last_became_peered":"2023-08-25T09:46:13.858905+0000","last_unstale":"2023-08-25T09:46:13.859548+0000","last_undegraded":"2023-08-25T09:46:13.859548+0000","last_fullsized":"2023-08-25T09:46:13.859548+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:25.787736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.851088+0000","last_change":"2023-08-25T09:46:12.841819+0000","last_active":"2023-08-25T09:46:13.851088+0000","last_peered":"2023-08-25T09:46:13.851088+0000","last_clean":"2023-08-25T09:46:13.851088+0000","last_became_active":"2023-08-25T09:46:12.840186+0000","last_became_peered":"2023-08-25T09:46:12.840186+0000","last_unstale":"2023-08-25T09:46:13.851088+0000","last_undegraded":"2023-08-25T09:46:13.851088+0000","last_fullsized":"2023-08-25T09:46:13.851088+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:05.625951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":234,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.848275+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:46:13.848275+0000","last_peered":"2023-08-25T09:46:13.848275+0000","last_clean":"2023-08-25T09:46:13.848275+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:46:13.848275+0000","last_undegraded":"2023-08-25T09:46:13.848275+0000","last_fullsized":"2023-08-25T09:46:13.848275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:01:39.470904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.845726+0000","last_change":"2023-08-25T09:46:13.845634+0000","last_active":"2023-08-25T09:46:13.845726+0000","last_peered":"2023-08-25T09:46:13.845726+0000","last_clean":"2023-08-25T09:46:13.845726+0000","last_became_active":"2023-08-25T09:46:13.843965+0000","last_became_peered":"2023-08-25T09:46:13.843965+0000","last_unstale":"2023-08-25T09:46:13.845726+0000","last_undegraded":"2023-08-25T09:46:13.845726+0000","last_fullsized":"2023-08-25T09:46:13.845726+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:17:08.706354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"132'1","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.854548+0000","last_change":"2023-08-25T09:46:12.841273+0000","last_active":"2023-08-25T09:46:13.854548+0000","last_peered":"2023-08-25T09:46:13.854548+0000","last_clean":"2023-08-25T09:46:13.854548+0000","last_became_active":"2023-08-25T09:46:12.841026+0000","last_became_peered":"2023-08-25T09:46:12.841026+0000","last_unstale":"2023-08-25T09:46:13.854548+0000","last_undegraded":"2023-08-25T09:46:13.854548+0000","last_fullsized":"2023-08-25T09:46:13.854548+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:19.493227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833072+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:46:13.833072+0000","last_peered":"2023-08-25T09:46:13.833072+0000","last_clean":"2023-08-25T09:46:13.833072+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:46:13.833072+0000","last_undegraded":"2023-08-25T09:46:13.833072+0000","last_fullsized":"2023-08-25T09:46:13.833072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:33.664260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.857145+0000","last_change":"2023-08-25T09:46:13.857037+0000","last_active":"2023-08-25T09:46:13.857145+0000","last_peered":"2023-08-25T09:46:13.857145+0000","last_clean":"2023-08-25T09:46:13.857145+0000","last_became_active":"2023-08-25T09:46:13.853078+0000","last_became_peered":"2023-08-25T09:46:13.853078+0000","last_unstale":"2023-08-25T09:46:13.857145+0000","last_undegraded":"2023-08-25T09:46:13.857145+0000","last_fullsized":"2023-08-25T09:46:13.857145+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:01:15.965690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.846440+0000","last_change":"2023-08-25T09:46:12.850794+0000","last_active":"2023-08-25T09:46:13.846440+0000","last_peered":"2023-08-25T09:46:13.846440+0000","last_clean":"2023-08-25T09:46:13.846440+0000","last_became_active":"2023-08-25T09:46:12.850570+0000","last_became_peered":"2023-08-25T09:46:12.850570+0000","last_unstale":"2023-08-25T09:46:13.846440+0000","last_undegraded":"2023-08-25T09:46:13.846440+0000","last_fullsized":"2023-08-25T09:46:13.846440+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:18:20.187390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":208,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836951+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:46:13.836951+0000","last_peered":"2023-08-25T09:46:13.836951+0000","last_clean":"2023-08-25T09:46:13.836951+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:46:13.836951+0000","last_undegraded":"2023-08-25T09:46:13.836951+0000","last_fullsized":"2023-08-25T09:46:13.836951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:42:30.185692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.852561+0000","last_change":"2023-08-25T09:46:13.851749+0000","last_active":"2023-08-25T09:46:13.852561+0000","last_peered":"2023-08-25T09:46:13.852561+0000","last_clean":"2023-08-25T09:46:13.852561+0000","last_became_active":"2023-08-25T09:46:13.851078+0000","last_became_peered":"2023-08-25T09:46:13.851078+0000","last_unstale":"2023-08-25T09:46:13.852561+0000","last_undegraded":"2023-08-25T09:46:13.852561+0000","last_fullsized":"2023-08-25T09:46:13.852561+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:45:48.631058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.836698+0000","last_change":"2023-08-25T09:46:12.846366+0000","last_active":"2023-08-25T09:46:13.836698+0000","last_peered":"2023-08-25T09:46:13.836698+0000","last_clean":"2023-08-25T09:46:13.836698+0000","last_became_active":"2023-08-25T09:46:12.845484+0000","last_became_peered":"2023-08-25T09:46:12.845484+0000","last_unstale":"2023-08-25T09:46:13.836698+0000","last_undegraded":"2023-08-25T09:46:13.836698+0000","last_fullsized":"2023-08-25T09:46:13.836698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:46:32.336383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.833406+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:46:13.833406+0000","last_peered":"2023-08-25T09:46:13.833406+0000","last_clean":"2023-08-25T09:46:13.833406+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:46:13.833406+0000","last_undegraded":"2023-08-25T09:46:13.833406+0000","last_fullsized":"2023-08-25T09:46:13.833406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:10:54.659715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.861620+0000","last_change":"2023-08-25T09:46:13.861119+0000","last_active":"2023-08-25T09:46:13.861620+0000","last_peered":"2023-08-25T09:46:13.861620+0000","last_clean":"2023-08-25T09:46:13.861620+0000","last_became_active":"2023-08-25T09:46:13.858979+0000","last_became_peered":"2023-08-25T09:46:13.858979+0000","last_unstale":"2023-08-25T09:46:13.861620+0000","last_undegraded":"2023-08-25T09:46:13.861620+0000","last_fullsized":"2023-08-25T09:46:13.861620+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:31:59.027427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"132'2","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.858819+0000","last_change":"2023-08-25T09:46:12.848200+0000","last_active":"2023-08-25T09:46:13.858819+0000","last_peered":"2023-08-25T09:46:13.858819+0000","last_clean":"2023-08-25T09:46:13.858819+0000","last_became_active":"2023-08-25T09:46:12.847931+0000","last_became_peered":"2023-08-25T09:46:12.847931+0000","last_unstale":"2023-08-25T09:46:13.858819+0000","last_undegraded":"2023-08-25T09:46:13.858819+0000","last_fullsized":"2023-08-25T09:46:13.858819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:31:52.940795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":204,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.827729+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:46:13.827729+0000","last_peered":"2023-08-25T09:46:13.827729+0000","last_clean":"2023-08-25T09:46:13.827729+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:46:13.827729+0000","last_undegraded":"2023-08-25T09:46:13.827729+0000","last_fullsized":"2023-08-25T09:46:13.827729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:54.344637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.856822+0000","last_change":"2023-08-25T09:46:13.856436+0000","last_active":"2023-08-25T09:46:13.856822+0000","last_peered":"2023-08-25T09:46:13.856822+0000","last_clean":"2023-08-25T09:46:13.856822+0000","last_became_active":"2023-08-25T09:46:13.856203+0000","last_became_peered":"2023-08-25T09:46:13.856203+0000","last_unstale":"2023-08-25T09:46:13.856822+0000","last_undegraded":"2023-08-25T09:46:13.856822+0000","last_fullsized":"2023-08-25T09:46:13.856822+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":131,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:12.804965+0000","last_clean_scrub_stamp":"2023-08-25T09:46:12.804965+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:21.909471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":11,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.837957+0000","last_change":"2023-08-25T09:46:12.840595+0000","last_active":"2023-08-25T09:46:13.837957+0000","last_peered":"2023-08-25T09:46:13.837957+0000","last_clean":"2023-08-25T09:46:13.837957+0000","last_became_active":"2023-08-25T09:46:12.832232+0000","last_became_peered":"2023-08-25T09:46:12.832232+0000","last_unstale":"2023-08-25T09:46:13.837957+0000","last_undegraded":"2023-08-25T09:46:13.837957+0000","last_fullsized":"2023-08-25T09:46:13.837957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:11.789814+0000","last_clean_scrub_stamp":"2023-08-25T09:46:11.789814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:18:10.133619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":232,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-08-25T09:46:13.832931+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:46:13.832931+0000","last_peered":"2023-08-25T09:46:13.832931+0000","last_clean":"2023-08-25T09:46:13.832931+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:46:13.832931+0000","last_undegraded":"2023-08-25T09:46:13.832931+0000","last_fullsized":"2023-08-25T09:46:13.832931+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:22:30.690053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220366,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":1.1599999999999999,"15min":1.0669999999999999},"min":{"1min":0.67500000000000004,"5min":0.59699999999999998,"15min":0.58899999999999997},"max":{"1min":1.3049999999999999,"5min":10.747,"15min":10.747},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.125,"15min":1.046},"min":{"1min":0.83999999999999997,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3819999999999999,"5min":1.855,"15min":1.855},"last":1.0620000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.7649999999999999,"5min":1.323,"15min":1.177},"min":{"1min":0.66900000000000004,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":11.010999999999999,"5min":11.010999999999999,"15min":11.010999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.127,"5min":1.173,"15min":1.103},"min":{"1min":0.93100000000000005,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4019999999999999,"5min":1.6679999999999999,"15min":1.7769999999999999},"last":0.93999999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1599999999999999,"15min":1.0900000000000001},"min":{"1min":0.748,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.403,"5min":1.619,"15min":1.619},"last":1.111},{"interface":"front","average":{"1min":1.224,"5min":1.304,"15min":1.232},"min":{"1min":0.75,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6859999999999999,"5min":1.98,"15min":1.98},"last":1.335}]},{"osd":3,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2410000000000001,"15min":1.177},"min":{"1min":0.93300000000000005,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5349999999999999,"5min":1.6699999999999999,"15min":1.7110000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.371,"15min":1.335},"min":{"1min":0.89200000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.6719999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.0149999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.3320000000000001,"15min":1.2809999999999999},"min":{"1min":0.99299999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.7110000000000001,"5min":7.1639999999999997,"15min":7.1639999999999997},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.1830000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.292}]},{"osd":5,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4339999999999999,"15min":1.3540000000000001},"min":{"1min":0.94299999999999995,"5min":0.67700000000000005,"15min":0.52700000000000002},"max":{"1min":1.7130000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.448,"15min":1.3879999999999999},"min":{"1min":1.1000000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.7330000000000001,"5min":1.9830000000000001,"15min":2.0739999999999998},"last":1.228}]},{"osd":6,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.538,"5min":1.5569999999999999,"15min":1.456},"min":{"1min":1.274,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.819,"5min":2.032,"15min":2.1720000000000002},"last":1.381},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4890000000000001,"15min":1.4139999999999999},"min":{"1min":1.1859999999999999,"5min":0.71199999999999997,"15min":0.61399999999999999},"max":{"1min":1.7789999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.2}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220346,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":1.1499999999999999,"15min":1.1459999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.615,"5min":1.804,"15min":1.804},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.1990000000000001,"15min":1.1479999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.734,"5min":5.5410000000000004,"15min":5.5410000000000004},"last":1.3620000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.2330000000000001,"15min":1.196},"min":{"1min":0.72699999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.726,"5min":5.6799999999999997,"15min":5.6799999999999997},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.202,"5min":1.236,"15min":1.1379999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7669999999999999,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":1.492}]},{"osd":2,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1819999999999999,"15min":1.1140000000000001},"min":{"1min":0.63400000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5269999999999999,"5min":1.708,"15min":1.708},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.329,"5min":1.425,"15min":1.2330000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7070000000000001,"5min":13.193,"15min":13.193},"last":1.3100000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.2410000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6479999999999999,"5min":5.6239999999999997,"15min":5.6239999999999997},"last":1.21},{"interface":"front","average":{"1min":1.278,"5min":1.27,"15min":1.2589999999999999},"min":{"1min":0.70699999999999996,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.76,"5min":1.76,"15min":1.9239999999999999},"last":1.7430000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3400000000000001,"15min":1.266},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8089999999999999,"5min":1.881,"15min":2.484},"last":1.702},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.405,"15min":1.3300000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.7789999999999999,"5min":5.4699999999999998,"15min":5.4699999999999998},"last":1.4359999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.3140000000000001,"15min":1.2809999999999999},"min":{"1min":0.67700000000000005,"5min":0.624,"15min":0.59599999999999997},"max":{"1min":1.663,"5min":1.837,"15min":2.0049999999999999},"last":1.859},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.502,"15min":1.3919999999999999},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.8129999999999999,"5min":1.978,"15min":2.3450000000000002},"last":1.792}]},{"osd":7,"last update":"Fri Aug 25 09:45:52 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5620000000000001,"15min":1.5009999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.622},"max":{"1min":1.9490000000000001,"5min":6.7709999999999999,"15min":6.7709999999999999},"last":1.669},{"interface":"front","average":{"1min":1.446,"5min":1.47,"15min":1.474},"min":{"1min":0.74299999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.97,"5min":1.97,"15min":2.1890000000000001},"last":1.946}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193982,"statfs":{"total":200000000,"available":198637778,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2070000000000001,"15min":1.153},"min":{"1min":0.63,"5min":0.497,"15min":0.497},"max":{"1min":1.6060000000000001,"5min":2.081,"15min":2.081},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.2050000000000001,"15min":1.2030000000000001},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.4410000000000001,"5min":1.8839999999999999,"15min":1.974},"last":1.4039999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1539999999999999,"15min":1.1870000000000001},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6830000000000001,"5min":1.851,"15min":2.1779999999999999},"last":1.208},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.222,"15min":1.1299999999999999},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.583,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.3660000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2629999999999999,"15min":1.1919999999999999},"min":{"1min":0.73899999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.5669999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.278,"15min":1.224},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.6890000000000001,"5min":1.8049999999999999,"15min":1.962},"last":1.0209999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.3759999999999999,"15min":1.2969999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.536,"5min":17.356999999999999,"15min":17.356999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.282,"5min":1.3400000000000001,"15min":1.29},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.806,"5min":1.806,"15min":1.9650000000000001},"last":1.5880000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.3340000000000001,"15min":1.3},"min":{"1min":0.77600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.7410000000000001,"5min":2.625,"15min":2.625},"last":1.129},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.343,"15min":1.3620000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.58799999999999997},"max":{"1min":1.845,"5min":1.8620000000000001,"15min":2.2789999999999999},"last":1.0760000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.448,"15min":1.4299999999999999},"min":{"1min":0.85499999999999998,"5min":0.752,"15min":0.58899999999999997},"max":{"1min":1.9219999999999999,"5min":2.4820000000000002,"15min":2.4820000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.522,"15min":1.4259999999999999},"min":{"1min":0.88700000000000001,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.028,"5min":2.5539999999999998,"15min":2.5539999999999998},"last":1.244}]},{"osd":7,"last update":"Fri Aug 25 09:45:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.472,"15min":1.4399999999999999},"min":{"1min":0.80800000000000005,"5min":0.63,"15min":0.63},"max":{"1min":1.9630000000000001,"5min":2.2149999999999999,"15min":2.2149999999999999},"last":1.272},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.591,"15min":1.575},"min":{"1min":0.93100000000000005,"5min":0.876,"15min":0.63900000000000001},"max":{"1min":1.994,"5min":2.0790000000000002,"15min":2.3700000000000001},"last":1.3140000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":753,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199228146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.234,"15min":1.171},"min":{"1min":0.80100000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6519999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.3340000000000001,"15min":1.179},"min":{"1min":0.95899999999999996,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":1.635,"5min":7.367,"15min":7.367},"last":1.347}]},{"osd":1,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.1799999999999999,"15min":1.133},"min":{"1min":0.89000000000000001,"5min":0.72199999999999998,"15min":0.71299999999999997},"max":{"1min":2.032,"5min":2.032,"15min":2.032},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.228,"15min":1.373},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.954,"5min":1.954,"15min":8.5850000000000009},"last":1.115}]},{"osd":2,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.206,"15min":1.284},"min":{"1min":0.85299999999999998,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.6240000000000001,"5min":1.6899999999999999,"15min":4.1669999999999998},"last":1.232},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2889999999999999,"15min":1.1899999999999999},"min":{"1min":0.85299999999999998,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.6739999999999999,"5min":1.6799999999999999,"15min":1.8520000000000001},"last":1.5129999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.274,"15min":1.2310000000000001},"min":{"1min":0.77100000000000002,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.764,"5min":7.2560000000000002,"15min":7.2560000000000002},"last":1.46},{"interface":"front","average":{"1min":1.4119999999999999,"5min":1.3799999999999999,"15min":1.3740000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.887,"5min":1.887,"15min":1.911},"last":1.5820000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.286,"15min":1.2909999999999999},"min":{"1min":0.77600000000000002,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.1850000000000001,"5min":2.1850000000000001,"15min":2.4420000000000002},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3660000000000001,"15min":1.264},"min":{"1min":0.84199999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.9299999999999999,"5min":8.2409999999999997,"15min":8.2409999999999997},"last":1.7569999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.524,"5min":1.456,"15min":1.4410000000000001},"min":{"1min":0.68400000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":3.5880000000000001,"5min":3.5880000000000001,"15min":3.5880000000000001},"last":1.7030000000000001},{"interface":"front","average":{"1min":1.746,"5min":1.5800000000000001,"15min":1.569},"min":{"1min":1.244,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":3.5270000000000001,"5min":3.5270000000000001,"15min":4.0229999999999997},"last":1.629}]},{"osd":7,"last update":"Fri Aug 25 09:45:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5840000000000001,"15min":1.4370000000000001},"min":{"1min":0.93999999999999995,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.976,"5min":8.1760000000000002,"15min":8.1760000000000002},"last":1.651},{"interface":"front","average":{"1min":1.653,"5min":1.621,"15min":1.6000000000000001},"min":{"1min":0.80900000000000005,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.1819999999999999,"5min":2.1819999999999999,"15min":2.7930000000000001},"last":1.806}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":762,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194550,"statfs":{"total":200000000,"available":199219406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.94099999999999995,"15min":0.86399999999999999},"min":{"1min":0.58199999999999996,"5min":0.432,"15min":0.29299999999999998},"max":{"1min":1.6399999999999999,"5min":1.6399999999999999,"15min":1.6830000000000001},"last":1.1200000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":1.0129999999999999,"15min":0.91600000000000004},"min":{"1min":0.54200000000000004,"5min":0.503,"15min":0.41699999999999998},"max":{"1min":1.292,"5min":1.631,"15min":1.631},"last":1.0089999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.90800000000000003,"15min":0.81399999999999995},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.35699999999999998},"max":{"1min":1.3580000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":1.044,"15min":0.98099999999999998},"min":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":0.94799999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.018,"15min":0.89200000000000002},"min":{"1min":0.56200000000000006,"5min":0.45000000000000001,"15min":0.378},"max":{"1min":1.3939999999999999,"5min":1.919,"15min":1.919},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.167,"15min":1.119},"min":{"1min":0.56999999999999995,"5min":0.47699999999999998,"15min":0.44900000000000001},"max":{"1min":2.0870000000000002,"5min":2.0870000000000002,"15min":3.399},"last":1.2869999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2529999999999999,"15min":1.198},"min":{"1min":0.82199999999999995,"5min":0.76100000000000001,"15min":0.66800000000000004},"max":{"1min":1.6519999999999999,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":1.194},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2569999999999999,"15min":1.204},"min":{"1min":0.73299999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.984,"5min":2.1840000000000002,"15min":2.1840000000000002},"last":1.23}]},{"osd":5,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.3640000000000001,"15min":1.319},"min":{"1min":0.86899999999999999,"5min":0.73599999999999999,"15min":0.71899999999999997},"max":{"1min":2.407,"5min":2.407,"15min":2.407},"last":1.47},{"interface":"front","average":{"1min":1.264,"5min":1.3220000000000001,"15min":1.276},"min":{"1min":0.83599999999999997,"5min":0.81499999999999995,"15min":0.64500000000000002},"max":{"1min":1.7949999999999999,"5min":1.8100000000000001,"15min":1.8100000000000001},"last":1.335}]},{"osd":6,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.3100000000000001,"15min":1.252},"min":{"1min":0.85199999999999998,"5min":0.78200000000000003,"15min":0.73799999999999999},"max":{"1min":2.3450000000000002,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.4370000000000001,"15min":1.409},"min":{"1min":0.95399999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.298,"5min":2.6699999999999999,"15min":3.496},"last":1.405}]},{"osd":7,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.4279999999999999,"15min":1.3680000000000001},"min":{"1min":0.995,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.659,"5min":2.044,"15min":2.2989999999999999},"last":1.5},{"interface":"front","average":{"1min":1.385,"5min":1.4410000000000001,"15min":1.3149999999999999},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":2.1509999999999998,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.3560000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":753,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194558,"statfs":{"total":200000000,"available":199228146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.85899999999999999,"15min":0.83499999999999996},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":2.383},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.95099999999999996,"15min":1.0449999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.546,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.026,"15min":0.997},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.45900000000000002},"max":{"1min":1.698,"5min":1.9350000000000001,"15min":2.3180000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":1.026,"5min":1.0389999999999999,"15min":1.056},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.7470000000000001,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.891,"5min":1.1479999999999999,"15min":1.133},"min":{"1min":0.47799999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":18.821000000000002,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.502},{"interface":"front","average":{"1min":1.089,"5min":1.1240000000000001,"15min":1.109},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.738,"5min":2.145,"15min":2.2040000000000002},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3240000000000001,"15min":1.387},"min":{"1min":0.751,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.833,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.284,"15min":1.331},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":1.036}]},{"osd":5,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.246,"15min":1.353},"min":{"1min":0.78300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.798,"5min":2.798,"15min":5.1269999999999998},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.381,"15min":1.395},"min":{"1min":0.85199999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1179999999999999,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.121}]},{"osd":6,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3859999999999999,"15min":1.417},"min":{"1min":0.77700000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.6709999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.395,"5min":1.4039999999999999,"15min":1.47},"min":{"1min":0.874,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":2.903,"5min":2.903,"15min":5.0599999999999996},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Aug 25 09:45:51 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.377,"15min":1.4379999999999999},"min":{"1min":0.77000000000000002,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.331,"5min":2.331,"15min":2.968},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4299999999999999,"15min":1.492},"min":{"1min":0.81599999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.0169999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1330,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193982,"statfs":{"total":200000000,"available":198637778,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.86399999999999999,"15min":0.78700000000000003},"min":{"1min":0.38700000000000001,"5min":0.36299999999999999,"15min":0.29699999999999999},"max":{"1min":1.5049999999999999,"5min":1.5700000000000001,"15min":1.627},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.94399999999999995,"15min":0.81100000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.39300000000000002},"max":{"1min":1.776,"5min":1.895,"15min":1.895},"last":0.69399999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.93700000000000006,"15min":0.82399999999999995},"min":{"1min":0.46200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":1.056,"5min":1.038,"15min":0.92400000000000004},"min":{"1min":0.53300000000000003,"5min":0.41899999999999998,"15min":0.33600000000000002},"max":{"1min":1.506,"5min":1.746,"15min":1.746},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1919999999999999,"15min":1.0600000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.372},"max":{"1min":1.8680000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.077,"15min":0.96499999999999997},"min":{"1min":0.56999999999999995,"5min":0.48799999999999999,"15min":0.45000000000000001},"max":{"1min":1.6240000000000001,"5min":1.6439999999999999,"15min":1.764},"last":0.629}]},{"osd":4,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1919999999999999,"15min":1.1020000000000001},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.5029999999999999,"5min":1.694,"15min":1.694},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3480000000000001,"15min":1.236},"min":{"1min":0.998,"5min":0.629,"15min":0.629},"max":{"1min":1.722,"5min":2.008,"15min":2.008},"last":0.95199999999999996}]},{"osd":5,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.2809999999999999,"15min":1.204},"min":{"1min":0.82199999999999995,"5min":0.751,"15min":0.68600000000000005},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.323,"5min":1.2829999999999999,"15min":1.1890000000000001},"min":{"1min":0.83199999999999996,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.4510000000000001,"15min":1.3220000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1339999999999999,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.002},{"interface":"front","average":{"1min":1.452,"5min":1.425,"15min":1.3140000000000001},"min":{"1min":0.92400000000000004,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9890000000000001,"5min":2.004,"15min":2.004},"last":1.0369999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:45:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4670000000000001,"5min":1.472,"15min":1.369},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.9630000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.98199999999999998},{"interface":"front","average":{"1min":1.53,"5min":1.4610000000000001,"15min":1.3380000000000001},"min":{"1min":0.91600000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.218,"5min":2.218,"15min":2.218},"last":1.0600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":761,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194551,"statfs":{"total":200000000,"available":199220436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":0.86399999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.313},"max":{"1min":1.3939999999999999,"5min":11.257,"15min":11.257},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0149999999999999,"15min":0.96199999999999997},"min":{"1min":0.58099999999999996,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":1.474,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":0.85499999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.96899999999999997,"15min":0.94199999999999995},"min":{"1min":0.59999999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.724,"15min":1.724},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.016,"15min":1.0800000000000001},"min":{"1min":0.498,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4259999999999999,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.94699999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92100000000000004,"15min":0.90500000000000003},"min":{"1min":0.437,"5min":0.42699999999999999,"15min":0.40699999999999997},"max":{"1min":1.625,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.121},"min":{"1min":0.55500000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6339999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.367}]},{"osd":4,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.272,"15min":1.24},"min":{"1min":0.82799999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.78,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.379,"5min":1.3759999999999999,"15min":1.357},"min":{"1min":0.95299999999999996,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.482}]},{"osd":5,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.2909999999999999,"15min":1.347},"min":{"1min":0.82499999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.9530000000000001,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.294},"min":{"1min":0.89900000000000002,"5min":0.73699999999999999,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.0469999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.347,"15min":1.3440000000000001},"min":{"1min":0.86399999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"max":{"1min":1.821,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.343,"15min":1.3080000000000001},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.72699999999999998},"max":{"1min":2.2090000000000001,"5min":2.331,"15min":2.331},"last":1.0700000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4139999999999999,"15min":1.4019999999999999},"min":{"1min":0.94999999999999996,"5min":0.79000000000000004,"15min":0.71099999999999997},"max":{"1min":2.085,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.337,"15min":1.323},"min":{"1min":0.86899999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.1469999999999998,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":1.2410000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:46:18.824 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:46:18.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:46:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.078+0000 7f86eeb22640 1 Processor -- start 2023-08-25T09:46:19.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.078+0000 7f86eeb22640 1 -- start start 2023-08-25T09:46:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.079+0000 7f86eeb22640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f86e80cd850 0x7f86e80cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.079+0000 7f86eeb22640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80ce170 con 0x7f86e815cfe0 2023-08-25T09:46:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.079+0000 7f86eeb22640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e80d1500 con 0x7f86e80b8510 2023-08-25T09:46:19.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.079+0000 7f86eeb22640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e80d1310 con 0x7f86e80cd850 2023-08-25T09:46:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.080+0000 7f86e7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f86e815cfe0 0x7f86e80d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52158/0 (socket says 172.21.15.70:52158) 2023-08-25T09:46:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.080+0000 7f86e7fff640 1 -- 172.21.15.70:0/2918401728 learned_addr learned my addr 172.21.15.70:0/2918401728 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.080+0000 7f86ec8be640 1 --2- 172.21.15.70:0/2918401728 >> v2:172.21.15.155:3300/0 conn(0x7f86e80cd850 0x7f86e80cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.080+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216217131 0 0) 0x7f86e80ce170 con 0x7f86e815cfe0 2023-08-25T09:46:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.080+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d0003520 con 0x7f86e815cfe0 2023-08-25T09:46:19.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2238301026 0 0) 0x7f86d0003520 con 0x7f86e815cfe0 2023-08-25T09:46:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 >> v2:172.21.15.155:3300/0 conn(0x7f86e80cd850 msgr2=0x7f86e80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 --2- 172.21.15.70:0/2918401728 >> v2:172.21.15.155:3300/0 conn(0x7f86e80cd850 0x7f86e80cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 >> v1:172.21.15.155:6789/0 conn(0x7f86e80b8510 legacy=0x7f86e80b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80d17e0 con 0x7f86e815cfe0 2023-08-25T09:46:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f86dc001670 con 0x7f86e815cfe0 2023-08-25T09:46:19.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 >> v1:172.21.15.70:6789/0 conn(0x7f86e815cfe0 legacy=0x7f86e80d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86d0003970 0x7f86d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d00047c0 0x7f86d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d0005180 0x7f86d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d000bff0 con 0x7f86d00047c0 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d000c160 con 0x7f86d0005180 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d000c320 con 0x7f86d0003970 2023-08-25T09:46:19.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86e7fff640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d00047c0 0x7f86d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.081+0000 7f86ec8be640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86d0003970 0x7f86d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86e7fff640 1 -- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86d0003970 msgr2=0x7f86d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86e7fff640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86d0003970 0x7f86d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86e7fff640 1 -- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d0005180 msgr2=0x7f86d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86e7fff640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d0005180 0x7f86d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86e7fff640 1 -- 172.21.15.70:0/2918401728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0003520 con 0x7f86d00047c0 2023-08-25T09:46:19.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86ed0bf640 1 -- 172.21.15.70:0/2918401728 reap_dead start 2023-08-25T09:46:19.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.082+0000 7f86ec8be640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86d0003970 0x7f86d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:19.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86e7fff640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d00047c0 0x7f86d0004c40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c210 tx=0x7f86d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfedc058243cf0d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc005ad0 con 0x7f86d00047c0 2023-08-25T09:46:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc005c50 con 0x7f86d00047c0 2023-08-25T09:46:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86e77fe640 1 -- 172.21.15.70:0/2918401728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc02c890 con 0x7f86d00047c0 2023-08-25T09:46:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86eeb22640 1 -- 172.21.15.70:0/2918401728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d00047c0 msgr2=0x7f86d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86eeb22640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d00047c0 0x7f86d0004c40 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00c210 tx=0x7f86d00057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86eeb22640 1 -- 172.21.15.70:0/2918401728 shutdown_connections 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86eeb22640 1 --2- 172.21.15.70:0/2918401728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86d00047c0 0x7f86d0004c40 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.083+0000 7f86eeb22640 1 -- 172.21.15.70:0/2918401728 >> 172.21.15.70:0/2918401728 conn(0x7f86e8069850 msgr2=0x7f86e8069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.084+0000 7f86eeb22640 1 -- 172.21.15.70:0/2918401728 shutdown_connections 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.084+0000 7f86eeb22640 1 -- 172.21.15.70:0/2918401728 wait complete. 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.085+0000 7f86eeb22640 1 Processor -- start 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.085+0000 7f86eeb22640 1 -- start start 2023-08-25T09:46:19.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.086+0000 7f86eeb22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d00047c0 0x7f86e81f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.086+0000 7f86eeb22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 0x7f86e80cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.086+0000 7f86eeb22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86e80b2570 0x7f86e80b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.086+0000 7f86eeb22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81f66b0 con 0x7f86e80cb0d0 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.086+0000 7f86eeb22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e81515f0 con 0x7f86d00047c0 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.086+0000 7f86eeb22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e80ca030 con 0x7f86e80b2570 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 0x7f86e80cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 0x7f86e80cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37046/0 (socket says 172.21.15.70:37046) 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 -- 172.21.15.70:0/4155679673 learned_addr learned my addr 172.21.15.70:0/4155679673 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86ec8be640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d00047c0 0x7f86e81f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 -- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86e80b2570 msgr2=0x7f86e80b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86e80b2570 0x7f86e80b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 -- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d00047c0 msgr2=0x7f86e81f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d00047c0 0x7f86e81f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 -- 172.21.15.70:0/4155679673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e80ca310 con 0x7f86e80cb0d0 2023-08-25T09:46:19.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.087+0000 7f86e7fff640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 0x7f86e80cbf30 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f86dc001930 tx=0x7f86dc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.088+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc03c040 con 0x7f86e80cb0d0 2023-08-25T09:46:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.088+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc02cca0 con 0x7f86e80cb0d0 2023-08-25T09:46:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.088+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc007270 con 0x7f86e80cb0d0 2023-08-25T09:46:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.088+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e80ca5f0 con 0x7f86e80cb0d0 2023-08-25T09:46:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.088+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e80cabb0 con 0x7f86e80cb0d0 2023-08-25T09:46:19.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.089+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e80cad40 con 0x7f86e80cb0d0 2023-08-25T09:46:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.090+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f86dc032070 con 0x7f86e80cb0d0 2023-08-25T09:46:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.091+0000 7f86e57fa640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f86d406a080 0x7f86d406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.091+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86dc0a9c10 con 0x7f86e80cb0d0 2023-08-25T09:46:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.091+0000 7f86ec8be640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f86d406a080 0x7f86d406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.092+0000 7f86ec8be640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f86d406a080 0x7f86d406c540 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f86e804ee40 tx=0x7f86d800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:19.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.095+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f86dc075260 con 0x7f86e80cb0d0 2023-08-25T09:46:19.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.351+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f86e8066fb0 con 0x7f86e80cb0d0 2023-08-25T09:46:19.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.357+0000 7f86e57fa640 1 -- 172.21.15.70:0/4155679673 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f86dc078880 con 0x7f86e80cb0d0 2023-08-25T09:46:19.360 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:46:19.360 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:46:19.360 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:46:19.360 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:46:19.360 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:46:19.361 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:46:19.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f86d406a080 msgr2=0x7f86d406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f86d406a080 0x7f86d406c540 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f86e804ee40 tx=0x7f86d800b040 comp rx=0 tx=0).stop 2023-08-25T09:46:19.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 msgr2=0x7f86e80cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 0x7f86e80cbf30 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f86dc001930 tx=0x7f86dc02e070 comp rx=0 tx=0).stop 2023-08-25T09:46:19.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 shutdown_connections 2023-08-25T09:46:19.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f86d406a080 0x7f86d406c540 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f86e80b2570 0x7f86e80b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f86e80cb0d0 0x7f86e80cbf30 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 --2- 172.21.15.70:0/4155679673 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f86d00047c0 0x7f86e81f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 >> 172.21.15.70:0/4155679673 conn(0x7f86e8069850 msgr2=0x7f86e815e280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:19.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 shutdown_connections 2023-08-25T09:46:19.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.360+0000 7f86eeb22640 1 -- 172.21.15.70:0/4155679673 wait complete. 2023-08-25T09:46:19.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:19.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.586+0000 7fde387cf640 1 Processor -- start 2023-08-25T09:46:19.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.586+0000 7fde387cf640 1 -- start start 2023-08-25T09:46:19.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde387cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde3015e830 0x7fde3006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde387cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015ec10 0x7fde300b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde387cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde300b1b60 con 0x7fde3006b2b0 2023-08-25T09:46:19.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde387cf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3006bd80 con 0x7fde3015e830 2023-08-25T09:46:19.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde387cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006b7c0 con 0x7fde3015ec10 2023-08-25T09:46:19.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde35d6a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fde3006b2b0 0x7fde3015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52164/0 (socket says 172.21.15.70:52164) 2023-08-25T09:46:19.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde35d6a640 1 -- 172.21.15.70:0/2536612311 learned_addr learned my addr 172.21.15.70:0/2536612311 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:19.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.588+0000 7fde3656b640 1 --2- 172.21.15.70:0/2536612311 >> v2:172.21.15.70:3300/0 conn(0x7fde3015e830 0x7fde3006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde3656b640 1 -- 172.21.15.70:0/2536612311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015ec10 msgr2=0x7fde300b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:19.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde36d6c640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015ec10 0x7fde300b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde35569640 1 -- 172.21.15.70:0/2536612311 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461694998 0 0) 0x7fde300b1b60 con 0x7fde3006b2b0 2023-08-25T09:46:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde3656b640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015ec10 0x7fde300b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde3656b640 1 -- 172.21.15.70:0/2536612311 >> v1:172.21.15.70:6789/0 conn(0x7fde3006b2b0 legacy=0x7fde3015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde3656b640 1 -- 172.21.15.70:0/2536612311 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3006baa0 con 0x7fde3015e830 2023-08-25T09:46:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.589+0000 7fde36d6c640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015ec10 0x7fde300b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:19.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.590+0000 7fde3656b640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015e830 0x7fde3006ad70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0097a0 tx=0x7fde2c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f059feff880b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.590+0000 7fde35569640 1 -- 172.21.15.70:0/2536612311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c02d040 con 0x7fde3015e830 2023-08-25T09:46:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.590+0000 7fde35569640 1 -- 172.21.15.70:0/2536612311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c031810 con 0x7fde3015e830 2023-08-25T09:46:19.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.590+0000 7fde35569640 1 -- 172.21.15.70:0/2536612311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c031ba0 con 0x7fde3015e830 2023-08-25T09:46:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 -- 172.21.15.70:0/2536612311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015e830 msgr2=0x7fde3006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015e830 0x7fde3006ad70 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0097a0 tx=0x7fde2c02b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 -- 172.21.15.70:0/2536612311 shutdown_connections 2023-08-25T09:46:19.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015ec10 0x7fde300b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 --2- 172.21.15.70:0/2536612311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015e830 0x7fde3006ad70 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 -- 172.21.15.70:0/2536612311 >> 172.21.15.70:0/2536612311 conn(0x7fde30069850 msgr2=0x7fde30069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:19.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 -- 172.21.15.70:0/2536612311 shutdown_connections 2023-08-25T09:46:19.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.591+0000 7fde387cf640 1 -- 172.21.15.70:0/2536612311 wait complete. 2023-08-25T09:46:19.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.592+0000 7fde387cf640 1 Processor -- start 2023-08-25T09:46:19.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.593+0000 7fde387cf640 1 -- start start 2023-08-25T09:46:19.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.593+0000 7fde387cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3006b2b0 0x7fde300c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.593+0000 7fde387cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015e830 0x7fde30155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.593+0000 7fde387cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 0x7fde300c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.593+0000 7fde387cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde300c4b40 con 0x7fde3015ec10 2023-08-25T09:46:19.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde387cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006ed80 con 0x7fde3006b2b0 2023-08-25T09:46:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde387cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3006f060 con 0x7fde3015e830 2023-08-25T09:46:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 0x7fde300c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 0x7fde300c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37058/0 (socket says 172.21.15.70:37058) 2023-08-25T09:46:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 -- 172.21.15.70:0/532158426 learned_addr learned my addr 172.21.15.70:0/532158426 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde35d6a640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015e830 0x7fde30155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 -- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015e830 msgr2=0x7fde30155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015e830 0x7fde30155630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 -- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3006b2b0 msgr2=0x7fde300c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3006b2b0 0x7fde300c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.594+0000 7fde36d6c640 1 -- 172.21.15.70:0/532158426 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009400 con 0x7fde3015ec10 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.595+0000 7fde36d6c640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 0x7fde300c40f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fde2800cd50 tx=0x7fde2800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.595+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800f680 con 0x7fde3015ec10 2023-08-25T09:46:19.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.595+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2800f800 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.595+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800fbb0 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.595+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3015ca20 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.595+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde300c4e20 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.596+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3004e420 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.597+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fde2800b040 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.598+0000 7fde26ffd640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1406a050 0x7fde1406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.598+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fde28089070 con 0x7fde3015ec10 2023-08-25T09:46:19.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.598+0000 7fde3656b640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1406a050 0x7fde1406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:19.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.599+0000 7fde3656b640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1406a050 0x7fde1406c510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fde2c036000 tx=0x7fde2c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:19.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.602+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fde280bb990 con 0x7fde3015ec10 2023-08-25T09:46:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.826+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde30156190 con 0x7fde3015ec10 2023-08-25T09:46:19.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.827+0000 7fde26ffd640 1 -- 172.21.15.70:0/532158426 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7fde2801d020 con 0x7fde3015ec10 2023-08-25T09:46:19.830 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:19.830 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:19.830 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:19.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.829+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1406a050 msgr2=0x7fde1406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.829+0000 7fde387cf640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1406a050 0x7fde1406c510 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fde2c036000 tx=0x7fde2c030000 comp rx=0 tx=0).stop 2023-08-25T09:46:19.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.829+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 msgr2=0x7fde300c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:19.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.829+0000 7fde387cf640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 0x7fde300c40f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fde2800cd50 tx=0x7fde2800a650 comp rx=0 tx=0).stop 2023-08-25T09:46:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 shutdown_connections 2023-08-25T09:46:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde1406a050 0x7fde1406c510 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde3015ec10 0x7fde300c40f0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde3015e830 0x7fde30155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 --2- 172.21.15.70:0/532158426 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde3006b2b0 0x7fde300c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 >> 172.21.15.70:0/532158426 conn(0x7fde30069850 msgr2=0x7fde3015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:19.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 shutdown_connections 2023-08-25T09:46:19.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:19.830+0000 7fde387cf640 1 -- 172.21.15.70:0/532158426 wait complete. 2023-08-25T09:46:20.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:46:13.310400+0000', 'modified': '2023-08-25T09:46:16.886817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7746}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7746': {'gid': 7746, 'name': 'a', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6836/59115020', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 59115020}, {'type': 'v1', 'addr': '172.21.15.70:6836', 'nonce': 59115020}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:46:20.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:46:20.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T09:46:20.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.478+0000 7effaebb2640 1 Processor -- start 2023-08-25T09:46:20.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.478+0000 7effaebb2640 1 -- start start 2023-08-25T09:46:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.479+0000 7effaebb2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7effa815abb0 0x7effa80b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.479+0000 7effaebb2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7effa8071850 0x7effa806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.479+0000 7effaebb2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa815e4a0 con 0x7effa815a7d0 2023-08-25T09:46:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.479+0000 7effaebb2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa80c53b0 con 0x7effa8071850 2023-08-25T09:46:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.479+0000 7effaebb2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa815f020 con 0x7effa815abb0 2023-08-25T09:46:20.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7effa8071850 0x7effa806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7effa8071850 0x7effa806ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37072/0 (socket says 172.21.15.70:37072) 2023-08-25T09:46:20.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 -- 172.21.15.70:0/940680769 learned_addr learned my addr 172.21.15.70:0/940680769 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effad3af640 1 --2- 172.21.15.70:0/940680769 >> v2:172.21.15.155:3300/0 conn(0x7effa815abb0 0x7effa80b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 -- 172.21.15.70:0/940680769 >> v2:172.21.15.155:3300/0 conn(0x7effa815abb0 msgr2=0x7effa80b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 --2- 172.21.15.70:0/940680769 >> v2:172.21.15.155:3300/0 conn(0x7effa815abb0 0x7effa80b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 -- 172.21.15.70:0/940680769 >> v1:172.21.15.155:6789/0 conn(0x7effa815a7d0 legacy=0x7effa80b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.480+0000 7effae3b1640 1 -- 172.21.15.70:0/940680769 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa815e6f0 con 0x7effa8071850 2023-08-25T09:46:20.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.481+0000 7effae3b1640 1 --2- 172.21.15.70:0/940680769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa8071850 0x7effa806ea70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7effa80c1f50 tx=0x7effa002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ed0c9b6bfbf2f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effacbae640 1 -- 172.21.15.70:0/940680769 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa002c960 con 0x7effa8071850 2023-08-25T09:46:20.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effacbae640 1 -- 172.21.15.70:0/940680769 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa002cac0 con 0x7effa8071850 2023-08-25T09:46:20.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effacbae640 1 -- 172.21.15.70:0/940680769 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa002ce50 con 0x7effa8071850 2023-08-25T09:46:20.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 -- 172.21.15.70:0/940680769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa8071850 msgr2=0x7effa806ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 --2- 172.21.15.70:0/940680769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa8071850 0x7effa806ea70 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7effa80c1f50 tx=0x7effa002bdc0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 -- 172.21.15.70:0/940680769 shutdown_connections 2023-08-25T09:46:20.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 --2- 172.21.15.70:0/940680769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa8071850 0x7effa806ea70 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 --2- 172.21.15.70:0/940680769 >> v2:172.21.15.155:3300/0 conn(0x7effa815abb0 0x7effa80b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 -- 172.21.15.70:0/940680769 >> 172.21.15.70:0/940680769 conn(0x7effa80aff60 msgr2=0x7effa80b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 -- 172.21.15.70:0/940680769 shutdown_connections 2023-08-25T09:46:20.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.482+0000 7effaebb2640 1 -- 172.21.15.70:0/940680769 wait complete. 2023-08-25T09:46:20.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.484+0000 7effaebb2640 1 Processor -- start 2023-08-25T09:46:20.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.484+0000 7effaebb2640 1 -- start start 2023-08-25T09:46:20.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effaebb2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8071850 0x7effa80c2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effaebb2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 0x7effa80c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effaebb2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effa81f43e0 0x7effa80c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effaebb2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8141320 con 0x7effa815a7d0 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effaebb2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa8141040 con 0x7effa8071850 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effaebb2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa81f3900 con 0x7effa81f43e0 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effad3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 0x7effa80c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effad3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 0x7effa80c7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37074/0 (socket says 172.21.15.70:37074) 2023-08-25T09:46:20.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.485+0000 7effad3af640 1 -- 172.21.15.70:0/116216831 learned_addr learned my addr 172.21.15.70:0/116216831 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effad3af640 1 -- 172.21.15.70:0/116216831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effa81f43e0 msgr2=0x7effa80c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effad3af640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effa81f43e0 0x7effa80c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effad3af640 1 -- 172.21.15.70:0/116216831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8071850 msgr2=0x7effa80c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effad3af640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8071850 0x7effa80c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effad3af640 1 -- 172.21.15.70:0/116216831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa0009b30 con 0x7effa815a7d0 2023-08-25T09:46:20.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effad3af640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 0x7effa80c7a50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7eff9800d5f0 tx=0x7eff9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7eff9e7fc640 1 -- 172.21.15.70:0/116216831 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980125f0 con 0x7effa815a7d0 2023-08-25T09:46:20.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7eff9e7fc640 1 -- 172.21.15.70:0/116216831 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98012750 con 0x7effa815a7d0 2023-08-25T09:46:20.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7eff9e7fc640 1 -- 172.21.15.70:0/116216831 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff98012b00 con 0x7effa815a7d0 2023-08-25T09:46:20.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.486+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa81f3be0 con 0x7effa815a7d0 2023-08-25T09:46:20.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.487+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa80c2650 con 0x7effa815a7d0 2023-08-25T09:46:20.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.489+0000 7eff9e7fc640 1 -- 172.21.15.70:0/116216831 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7eff98012c60 con 0x7effa815a7d0 2023-08-25T09:46:20.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.489+0000 7eff9e7fc640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff8c06a050 0x7eff8c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.490+0000 7eff9e7fc640 1 -- 172.21.15.70:0/116216831 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff98088ab0 con 0x7effa815a7d0 2023-08-25T09:46:20.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.490+0000 7effadbb0640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff8c06a050 0x7eff8c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.490+0000 7effadbb0640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff8c06a050 0x7eff8c06c510 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7effa4001560 tx=0x7effa4007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.491+0000 7effb298a740 1 Processor -- start 2023-08-25T09:46:20.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.491+0000 7effb298a740 1 -- start start 2023-08-25T09:46:20.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015867070 0x560015849150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effb298a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015846780 0x56001581cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effb298a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56001578c190 con 0x560015864b90 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015789880 con 0x560015867070 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560015785170 con 0x560015846780 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37082/0 (socket says 172.21.15.70:37082) 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 -- 172.21.15.70:0/2610919867 learned_addr learned my addr 172.21.15.70:0/2610919867 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effadbb0640 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015867070 0x560015849150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 -- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015846780 msgr2=0x56001581cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015846780 0x56001581cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 -- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015867070 msgr2=0x560015849150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015867070 0x560015849150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.492+0000 7effad3af640 1 -- 172.21.15.70:0/2610919867 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600157837f0 con 0x560015864b90 2023-08-25T09:46:20.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7effad3af640 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849690 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7eff980542c0 tx=0x7eff98058510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460eebd3e33c816 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7eff8b7fe640 1 -- 172.21.15.70:0/2610919867 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9805fad0 con 0x560015864b90 2023-08-25T09:46:20.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7eff8b7fe640 1 -- 172.21.15.70:0/2610919867 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98056160 con 0x560015864b90 2023-08-25T09:46:20.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7eff8b7fe640 1 -- 172.21.15.70:0/2610919867 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9805f640 con 0x560015864b90 2023-08-25T09:46:20.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7effb298a740 1 -- 172.21.15.70:0/2610919867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 msgr2=0x560015849690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7effb298a740 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849690 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7eff980542c0 tx=0x7eff98058510 comp rx=0 tx=0).stop 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7effb298a740 1 -- 172.21.15.70:0/2610919867 shutdown_connections 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.493+0000 7effb298a740 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015846780 0x56001581cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.494+0000 7effb298a740 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849690 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.494+0000 7effb298a740 1 --2- 172.21.15.70:0/2610919867 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015867070 0x560015849150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.494+0000 7effb298a740 1 -- 172.21.15.70:0/2610919867 >> 172.21.15.70:0/2610919867 conn(0x56001581b5d0 msgr2=0x560015864150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.494+0000 7effb298a740 1 -- 172.21.15.70:0/2610919867 shutdown_connections 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.494+0000 7effb298a740 1 -- 172.21.15.70:0/2610919867 wait complete. 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.494+0000 7effb298a740 1 -- start start 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.495+0000 7effb298a740 1 Processor -- start 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.495+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015849040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015849580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effb298a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x56001583f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effb298a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015724d20 con 0x56001584a190 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600157249c0 con 0x560015864b90 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600154eaf20 con 0x560015867070 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effb298a740 10 client.0 _ll_register_callbacks cb 0x56001573c630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x56001583f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x56001583f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37092/0 (socket says 172.21.15.70:37092) 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 -- 172.21.15.70:0/1758225744 learned_addr learned my addr 172.21.15.70:0/1758225744 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effadbb0640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015849040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effad3af640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015849580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 -- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 msgr2=0x560015849580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015849580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 -- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 msgr2=0x560015849040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015849040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effae3b1640 1 -- 172.21.15.70:0/1758225744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa80c2650 con 0x56001584a190 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.496+0000 7effadbb0640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015849040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7effae3b1640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x56001583f720 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7effa0009f40 tx=0x7effa0034620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e3a3e8beafaa75 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7eff88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:46:20.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7eff88ff9640 1 -- 172.21.15.70:0/1758225744 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa00349a0 con 0x56001584a190 2023-08-25T09:46:20.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560015759540 con 0x56001584a190 2023-08-25T09:46:20.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7eff88ff9640 1 -- 172.21.15.70:0/1758225744 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effa0034b00 con 0x56001584a190 2023-08-25T09:46:20.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7eff88ff9640 1 -- 172.21.15.70:0/1758225744 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa0034cc0 con 0x56001584a190 2023-08-25T09:46:20.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7effb298a740 10 client.7908 fetch_fsmap learned FSMap version 57 2023-08-25T09:46:20.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56001577cf30 con 0x56001584a190 2023-08-25T09:46:20.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.497+0000 7eff88ff9640 1 -- 172.21.15.70:0/1758225744 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560015759540 con 0x56001584a190 2023-08-25T09:46:20.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7eff88ff9640 1 -- 172.21.15.70:0/1758225744 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effa003c070 con 0x56001584a190 2023-08-25T09:46:20.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effb298a740 10 client.7908 fetch_fsmap finished waiting for FSMap version 57 2023-08-25T09:46:20.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effb298a740 10 client.7908 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T09:46:20.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effb298a740 20 client.7908 populate_metadata read hostname 'smithi070' 2023-08-25T09:46:20.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effb298a740 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x56001587bb60 0x56001587bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effb298a740 4 client.7908 mds_command: new command op to 7746 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:46:20.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 --> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56001577e150 con 0x56001587bb60 2023-08-25T09:46:20.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effadbb0640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x56001587bb60 0x56001587bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7effadbb0640 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x56001587bb60 0x56001587bf40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74931d0ffa675672 server_cookie=cd0af1a25ed36349 in_seq=0 out_seq=0 2023-08-25T09:46:20.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.498+0000 7eff88ff9640 10 client.7908 ms_handle_connect on v2:172.21.15.70:6834/59115020 2023-08-25T09:46:20.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.499+0000 7eff88ff9640 1 -- 172.21.15.70:0/1758225744 <== mds.0 v2:172.21.15.70:6834/59115020 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56001577e150 con 0x56001587bb60 2023-08-25T09:46:20.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.499+0000 7eff88ff9640 10 client.7908 handle_command_reply: tid=0 2023-08-25T09:46:20.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.499+0000 7effb298a740 1 client.7908 shutdown 2023-08-25T09:46:20.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.499+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 msgr2=0x56001583f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.499+0000 7effb298a740 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x56001583f720 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7effa0009f40 tx=0x7effa0034620 comp rx=0 tx=0).stop 2023-08-25T09:46:20.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 shutdown_connections 2023-08-25T09:46:20.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x56001583f720 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015849580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x56001587bb60 0x56001587bf40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 --2- 172.21.15.70:0/1758225744 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015849040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 >> 172.21.15.70:0/1758225744 conn(0x560015846780 msgr2=0x560015818aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7eff88ff9640 0 client.7908 ms_handle_reset on v2:172.21.15.70:6834/59115020 2023-08-25T09:46:20.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 shutdown_connections 2023-08-25T09:46:20.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 1 -- 172.21.15.70:0/1758225744 wait complete. 2023-08-25T09:46:20.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.500+0000 7effb298a740 20 client.7908 trim_cache size 0 max 16384 2023-08-25T09:46:20.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.516+0000 7effb298a740 1 Processor -- start 2023-08-25T09:46:20.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.516+0000 7effb298a740 1 -- start start 2023-08-25T09:46:20.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.517+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56001584a190 0x560015849040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.517+0000 7effb298a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.517+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015845320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.517+0000 7effb298a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015814e40 con 0x560015864b90 2023-08-25T09:46:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.517+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600158152e0 con 0x56001584a190 2023-08-25T09:46:20.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.517+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560015849ea0 con 0x560015867070 2023-08-25T09:46:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37104/0 (socket says 172.21.15.70:37104) 2023-08-25T09:46:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 -- 172.21.15.70:0/2594691754 learned_addr learned my addr 172.21.15.70:0/2594691754 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 -- 172.21.15.70:0/2594691754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 msgr2=0x560015845320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015845320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 -- 172.21.15.70:0/2594691754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56001584a190 msgr2=0x560015849040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56001584a190 0x560015849040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 -- 172.21.15.70:0/2594691754 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560015723a10 con 0x560015864b90 2023-08-25T09:46:20.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7effad3af640 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849580 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7eff98013040 tx=0x7eff98020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9920e0cc968c093a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7eff8bfff640 1 -- 172.21.15.70:0/2594691754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9805fad0 con 0x560015864b90 2023-08-25T09:46:20.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7eff8bfff640 1 -- 172.21.15.70:0/2594691754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff980544d0 con 0x560015864b90 2023-08-25T09:46:20.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.518+0000 7eff8bfff640 1 -- 172.21.15.70:0/2594691754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9805f880 con 0x560015864b90 2023-08-25T09:46:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 -- 172.21.15.70:0/2594691754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 msgr2=0x560015849580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849580 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7eff98013040 tx=0x7eff98020000 comp rx=0 tx=0).stop 2023-08-25T09:46:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 -- 172.21.15.70:0/2594691754 shutdown_connections 2023-08-25T09:46:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015845320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x560015864b90 0x560015849580 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 --2- 172.21.15.70:0/2594691754 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56001584a190 0x560015849040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 -- 172.21.15.70:0/2594691754 >> 172.21.15.70:0/2594691754 conn(0x560015846780 msgr2=0x56001588aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 -- 172.21.15.70:0/2594691754 shutdown_connections 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.519+0000 7effb298a740 1 -- 172.21.15.70:0/2594691754 wait complete. 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.520+0000 7effb298a740 1 -- start start 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.520+0000 7effb298a740 1 Processor -- start 2023-08-25T09:46:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x560015844120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015844660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015844ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015882630 con 0x56001584a190 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x560015842ca0 con 0x560015864b90 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x560015842a50 con 0x560015867070 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effb298a740 10 client.0 _ll_register_callbacks cb 0x7eff98012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:46:20.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effadbb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x560015844120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effadbb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x560015844120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37114/0 (socket says 172.21.15.70:37114) 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.521+0000 7effadbb0640 1 -- 172.21.15.70:0/2624037486 learned_addr learned my addr 172.21.15.70:0/2624037486 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effad3af640 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015844660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effadbb0640 1 -- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 msgr2=0x560015844ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effadbb0640 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015844ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effadbb0640 1 -- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 msgr2=0x560015844660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effadbb0640 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015844660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effadbb0640 1 -- 172.21.15.70:0/2624037486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600157837f0 con 0x56001584a190 2023-08-25T09:46:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effadbb0640 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x560015844120 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7effa4005430 tx=0x7effa4001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a5e3f3599aefcd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7eff88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:46:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7eff88ff9640 1 -- 172.21.15.70:0/2624037486 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4026030 con 0x56001584a190 2023-08-25T09:46:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7eff88ff9640 1 -- 172.21.15.70:0/2624037486 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7effa4027070 con 0x56001584a190 2023-08-25T09:46:20.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.522+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff98056160 con 0x56001584a190 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7eff88ff9640 1 -- 172.21.15.70:0/2624037486 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa40215b0 con 0x56001584a190 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 10 client.7914 fetch_fsmap learned FSMap version 57 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x560015748de0 con 0x56001584a190 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7eff88ff9640 1 -- 172.21.15.70:0/2624037486 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff98056160 con 0x56001584a190 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7eff88ff9640 1 -- 172.21.15.70:0/2624037486 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7effa401f040 con 0x56001584a190 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 10 client.7914 fetch_fsmap finished waiting for FSMap version 57 2023-08-25T09:46:20.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 10 client.7914 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T09:46:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 20 client.7914 populate_metadata read hostname 'smithi070' 2023-08-25T09:46:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x5600158451d0 0x5600158455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 4 client.7914 mds_command: new command op to 7746 tid=0[{"prefix": "status"}] 2023-08-25T09:46:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 --> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5600154345e0 con 0x5600158451d0 2023-08-25T09:46:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.523+0000 7effad3af640 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x5600158451d0 0x5600158455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effad3af640 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x5600158451d0 0x5600158455b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4eb4b155396b46 server_cookie=524e50bf5df6b0e1 in_seq=0 out_seq=0 2023-08-25T09:46:20.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7eff88ff9640 10 client.7914 ms_handle_connect on v2:172.21.15.70:6834/59115020 2023-08-25T09:46:20.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7eff88ff9640 1 -- 172.21.15.70:0/2624037486 <== mds.0 v2:172.21.15.70:6834/59115020 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5600154345e0 con 0x5600158451d0 2023-08-25T09:46:20.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7eff88ff9640 10 client.7914 handle_command_reply: tid=0 2023-08-25T09:46:20.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 client.7914 shutdown 2023-08-25T09:46:20.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 msgr2=0x560015844120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x560015844120 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7effa4005430 tx=0x7effa4001020 comp rx=0 tx=0).stop 2023-08-25T09:46:20.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 shutdown_connections 2023-08-25T09:46:20.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x560015867070 0x560015844ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:6834/59115020,v1:172.21.15.70:6836/59115020] conn(0x5600158451d0 0x5600158455b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x560015864b90 0x560015844660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 --2- 172.21.15.70:0/2624037486 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56001584a190 0x560015844120 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7eff88ff9640 0 client.7914 ms_handle_reset on v2:172.21.15.70:6834/59115020 2023-08-25T09:46:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 >> 172.21.15.70:0/2624037486 conn(0x560015846780 msgr2=0x560015849040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 shutdown_connections 2023-08-25T09:46:20.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.524+0000 7effb298a740 1 -- 172.21.15.70:0/2624037486 wait complete. 2023-08-25T09:46:20.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effb298a740 20 client.7914 trim_cache size 0 max 16384 2023-08-25T09:46:20.539 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:20.539 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7746, 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.7006829750000003, 2023-08-25T09:46:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 57, 2023-08-25T09:46:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 133, 2023-08-25T09:46:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 133, 2023-08-25T09:46:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.112237035 2023-08-25T09:46:20.541 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:20.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff8c06a050 msgr2=0x7eff8c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff8c06a050 0x7eff8c06c510 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7effa4001560 tx=0x7effa4007990 comp rx=0 tx=0).stop 2023-08-25T09:46:20.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 msgr2=0x7effa80c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 0x7effa80c7a50 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7eff9800d5f0 tx=0x7eff9800dab0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 shutdown_connections 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff8c06a050 0x7eff8c06c510 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7effa81f43e0 0x7effa80c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7effa815a7d0 0x7effa80c7a50 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 --2- 172.21.15.70:0/116216831 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7effa8071850 0x7effa80c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 >> 172.21.15.70:0/116216831 conn(0x7effa80aff60 msgr2=0x7effa8159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 shutdown_connections 2023-08-25T09:46:20.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.525+0000 7effaebb2640 1 -- 172.21.15.70:0/116216831 wait complete. 2023-08-25T09:46:20.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:46:20.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.743+0000 7f6987c90640 1 Processor -- start 2023-08-25T09:46:20.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.743+0000 7f6987c90640 1 -- start start 2023-08-25T09:46:20.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6987c90640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69800ce5b0 0x7f69800ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6987c90640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f69800b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6987c90640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f69800b2680 0x7f698006be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6987c90640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69800b2b10 con 0x7f69800ce5b0 2023-08-25T09:46:20.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6987c90640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6980159d10 con 0x7f69800b2680 2023-08-25T09:46:20.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6987c90640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69800ceed0 con 0x7f698015aa10 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6985a2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69800ce5b0 0x7f69800ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6985a2c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f69800ce5b0 0x7f69800ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37124/0 (socket says 172.21.15.70:37124) 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.745+0000 7f6985a2c640 1 -- 172.21.15.70:0/3463235682 learned_addr learned my addr 172.21.15.70:0/3463235682 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f698522b640 1 --2- 172.21.15.70:0/3463235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f69800b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f698622d640 1 --2- 172.21.15.70:0/3463235682 >> v2:172.21.15.155:3300/0 conn(0x7f69800b2680 0x7f698006be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f6985a2c640 1 -- 172.21.15.70:0/3463235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 msgr2=0x7f69800b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f6985a2c640 1 --2- 172.21.15.70:0/3463235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f69800b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f6985a2c640 1 -- 172.21.15.70:0/3463235682 >> v2:172.21.15.155:3300/0 conn(0x7f69800b2680 msgr2=0x7f698006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:20.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f6985a2c640 1 --2- 172.21.15.70:0/3463235682 >> v2:172.21.15.155:3300/0 conn(0x7f69800b2680 0x7f698006be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.746+0000 7f6985a2c640 1 -- 172.21.15.70:0/3463235682 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698015d3f0 con 0x7f69800ce5b0 2023-08-25T09:46:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.747+0000 7f6985a2c640 1 --2- 172.21.15.70:0/3463235682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800ce5b0 0x7f69800ce990 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f698006a760 tx=0x7f697c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3481a683886e89 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.747+0000 7f6984a2a640 1 -- 172.21.15.70:0/3463235682 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c031870 con 0x7f69800ce5b0 2023-08-25T09:46:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.747+0000 7f6984a2a640 1 -- 172.21.15.70:0/3463235682 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c0319d0 con 0x7f69800ce5b0 2023-08-25T09:46:20.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.747+0000 7f6984a2a640 1 -- 172.21.15.70:0/3463235682 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c031d60 con 0x7f69800ce5b0 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 -- 172.21.15.70:0/3463235682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800ce5b0 msgr2=0x7f69800ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 --2- 172.21.15.70:0/3463235682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800ce5b0 0x7f69800ce990 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f698006a760 tx=0x7f697c02b910 comp rx=0 tx=0).stop 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 -- 172.21.15.70:0/3463235682 shutdown_connections 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 --2- 172.21.15.70:0/3463235682 >> v2:172.21.15.155:3300/0 conn(0x7f69800b2680 0x7f698006be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 --2- 172.21.15.70:0/3463235682 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f69800b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 --2- 172.21.15.70:0/3463235682 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800ce5b0 0x7f69800ce990 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 -- 172.21.15.70:0/3463235682 >> 172.21.15.70:0/3463235682 conn(0x7f6980069850 msgr2=0x7f6980069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 -- 172.21.15.70:0/3463235682 shutdown_connections 2023-08-25T09:46:20.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.748+0000 7f6987c90640 1 -- 172.21.15.70:0/3463235682 wait complete. 2023-08-25T09:46:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.750+0000 7f6987c90640 1 Processor -- start 2023-08-25T09:46:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.750+0000 7f6987c90640 1 -- start start 2023-08-25T09:46:20.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6987c90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 0x7f6980151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6987c90640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69800ce5b0 0x7f698015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6987c90640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f698015c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6987c90640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980151e50 con 0x7f69800b2680 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6987c90640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980155eb0 con 0x7f69800ce5b0 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6987c90640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69801520a0 con 0x7f698015aa10 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6985a2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 0x7f6980151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6985a2c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 0x7f6980151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37130/0 (socket says 172.21.15.70:37130) 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.751+0000 7f6985a2c640 1 -- 172.21.15.70:0/671140938 learned_addr learned my addr 172.21.15.70:0/671140938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6985a2c640 1 -- 172.21.15.70:0/671140938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 msgr2=0x7f698015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6985a2c640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f698015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6985a2c640 1 -- 172.21.15.70:0/671140938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69800ce5b0 msgr2=0x7f698015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6985a2c640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69800ce5b0 0x7f698015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6985a2c640 1 -- 172.21.15.70:0/671140938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0094d0 con 0x7f69800b2680 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6985a2c640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 0x7f6980151800 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f697c02b950 tx=0x7f697c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c03ad30 con 0x7f69800b2680 2023-08-25T09:46:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c0443f0 con 0x7f69800b2680 2023-08-25T09:46:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69800ae2c0 con 0x7f69800b2680 2023-08-25T09:46:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.752+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c0447a0 con 0x7f69800b2680 2023-08-25T09:46:20.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.753+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69800ae880 con 0x7f69800b2680 2023-08-25T09:46:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.754+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698004e420 con 0x7f69800b2680 2023-08-25T09:46:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.755+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f697c04d450 con 0x7f69800b2680 2023-08-25T09:46:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.755+0000 7f69767fc640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696406a050 0x7f696406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.756+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f697c02d040 con 0x7f69800b2680 2023-08-25T09:46:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.756+0000 7f698522b640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696406a050 0x7f696406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:20.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.756+0000 7f698522b640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696406a050 0x7f696406c510 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6970006fd0 tx=0x7f6970008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:20.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.759+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f697c076300 con 0x7f69800b2680 2023-08-25T09:46:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.987+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69800cae50 con 0x7f69800b2680 2023-08-25T09:46:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.987+0000 7f69767fc640 1 -- 172.21.15.70:0/671140938 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f697c0aae60 con 0x7f69800b2680 2023-08-25T09:46:20.990 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:46:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696406a050 msgr2=0x7f696406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696406a050 0x7f696406c510 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6970006fd0 tx=0x7f6970008040 comp rx=0 tx=0).stop 2023-08-25T09:46:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 msgr2=0x7f6980151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 0x7f6980151800 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f697c02b950 tx=0x7f697c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 shutdown_connections 2023-08-25T09:46:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f696406a050 0x7f696406c510 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f698015aa10 0x7f698015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69800ce5b0 0x7f698015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 --2- 172.21.15.70:0/671140938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69800b2680 0x7f6980151800 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:20.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 >> 172.21.15.70:0/671140938 conn(0x7f6980069850 msgr2=0x7f6980069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:20.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 shutdown_connections 2023-08-25T09:46:20.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:20.990+0000 7f6987c90640 1 -- 172.21.15.70:0/671140938 wait complete. 2023-08-25T09:46:21.011 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:46:21.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:46:21.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:46:21.012 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:46:21.012 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:21.012 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:46:21.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:46:21.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:46:21.013 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:46:21.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.014 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:46:21.064 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:46:21.064 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:46:21.065 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:46:21.065 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.067 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:46:21.067 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:46:21.067 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:21.067 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:46:21.067 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:21.068 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:46:21.069 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41895sec preferred_lft 41895sec 2023-08-25T09:46:21.069 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:46:21.069 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.069 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:46:21.070 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:21.070 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:46:21.070 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.070 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:46:21.070 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.070 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:46:21.071 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:21.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.073 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:46:21.212 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:46:21.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.217 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:46:21.275 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:46:21.275 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:46:21.276 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:46:21.276 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:46:21.276 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:46:21.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.277 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:46:21.277 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:46:21.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:46:21.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:46:21.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:46:21.278 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:46:21.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.464 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:46:21.519 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:46:21.519 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:46:21.521 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:46:21.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:46:21.627 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:46:21.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:46:21.628 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:21.684 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:46:21.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:46:21.686 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:21.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:46:21.836 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:46:21.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:46:21.929 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:46:21.933 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:46:21.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:21.935 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:46:21.999 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:46:21.999 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:46:22.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:46:22.098 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:46:22.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:46:22.182 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:46:22.185 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:46:22.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:22.187 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:46:22.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:46:22.207+0000 7f59993f8480 -1 init, newargv = 0x55689f84d230 newargc=15 2023-08-25T09:46:22.210 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[113769]: starting ceph client 2023-08-25T09:46:22.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[113769]: starting fuse 2023-08-25T09:46:22.233 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:46:22.233 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:46:22.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:46:22.234 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:46:22.234 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:46:22.235 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:22.235 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:46:22.235 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:22.236 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:46:22.236 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:46:22.236 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:46:22.237 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:46:22.237 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:22.237 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:46:22.238 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:46:22.238 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:46:22.238 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:46:22.238 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:22.239 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:46:22.239 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:46:22.239 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:46:22.240 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:46:22.241 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:46:22.241 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:46:22.241 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:46:22.241 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:46:22.243 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:22.243 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:46:22.244 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:22.445 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:22.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:22.462 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:22.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:22.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113769.asok status 2023-08-25T09:46:22.701 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:22.701 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "113769", 2023-08-25T09:46:22.702 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7926, 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:46:22.703 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:46:22.704 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 7926 2023-08-25T09:46:22.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:22.704 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:22.704 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:22.704 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:22.704 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:22.705 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.7926 192.168.0.1:0/3843564457", 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3843564457", 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 57, 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 133, 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:46:22.706 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:22.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:22.716 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:46:22.716 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:22.774 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:46:22.774 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:22.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:22.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:22.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:46:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.140+0000 7fb065e3f640 1 Processor -- start 2023-08-25T09:46:23.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.141+0000 7fb065e3f640 1 -- start start 2023-08-25T09:46:23.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.142+0000 7fb065e3f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0600ce480 0x7fb06015da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.142+0000 7fb065e3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0600b0ec0 0x7fb0600b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.142+0000 7fb065e3f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0600cec90 con 0x7fb06015d6b0 2023-08-25T09:46:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.142+0000 7fb065e3f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0600cdf90 con 0x7fb0600ce480 2023-08-25T09:46:23.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.142+0000 7fb065e3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b1890 con 0x7fb0600b0ec0 2023-08-25T09:46:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.143+0000 7fb05effd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0600ce480 0x7fb06015da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.143+0000 7fb05ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0600b0ec0 0x7fb0600b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.143+0000 7fb05effd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0600ce480 0x7fb06015da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:59308/0 (socket says 172.21.15.70:59308) 2023-08-25T09:46:23.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.143+0000 7fb05effd640 1 -- 172.21.15.70:0/1569430609 learned_addr learned my addr 172.21.15.70:0/1569430609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.144+0000 7fb05ffff640 1 -- 172.21.15.70:0/1569430609 >> v2:172.21.15.155:3300/0 conn(0x7fb0600ce480 msgr2=0x7fb06015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.144+0000 7fb05ffff640 1 --2- 172.21.15.70:0/1569430609 >> v2:172.21.15.155:3300/0 conn(0x7fb0600ce480 0x7fb06015da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.144+0000 7fb05ffff640 1 -- 172.21.15.70:0/1569430609 >> v1:172.21.15.155:6789/0 conn(0x7fb06015d6b0 legacy=0x7fb0600cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:23.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.144+0000 7fb05ffff640 1 -- 172.21.15.70:0/1569430609 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0600cb490 con 0x7fb0600b0ec0 2023-08-25T09:46:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.145+0000 7fb05ffff640 1 --2- 172.21.15.70:0/1569430609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0600b0ec0 0x7fb0600b1350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb0580097a0 tx=0x7fb05802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1afd19855166309 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb05e7fc640 1 -- 172.21.15.70:0/1569430609 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05802d040 con 0x7fb0600b0ec0 2023-08-25T09:46:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb05e7fc640 1 -- 172.21.15.70:0/1569430609 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058031810 con 0x7fb0600b0ec0 2023-08-25T09:46:23.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb05e7fc640 1 -- 172.21.15.70:0/1569430609 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058031ba0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 -- 172.21.15.70:0/1569430609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0600b0ec0 msgr2=0x7fb0600b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:23.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 --2- 172.21.15.70:0/1569430609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0600b0ec0 0x7fb0600b1350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb0580097a0 tx=0x7fb05802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 -- 172.21.15.70:0/1569430609 shutdown_connections 2023-08-25T09:46:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 --2- 172.21.15.70:0/1569430609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0600b0ec0 0x7fb0600b1350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 --2- 172.21.15.70:0/1569430609 >> v2:172.21.15.155:3300/0 conn(0x7fb0600ce480 0x7fb06015da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 -- 172.21.15.70:0/1569430609 >> 172.21.15.70:0/1569430609 conn(0x7fb060069c70 msgr2=0x7fb06006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.146+0000 7fb065e3f640 1 -- 172.21.15.70:0/1569430609 shutdown_connections 2023-08-25T09:46:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.147+0000 7fb065e3f640 1 -- 172.21.15.70:0/1569430609 wait complete. 2023-08-25T09:46:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.149+0000 7fb065e3f640 1 Processor -- start 2023-08-25T09:46:23.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.149+0000 7fb065e3f640 1 -- start start 2023-08-25T09:46:23.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb065e3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 0x7fb0600c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb065e3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0600ce480 0x7fb0600b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb065e3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb06015d6b0 0x7fb06015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb065e3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b2ac0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb065e3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b2030 con 0x7fb0600ce480 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb065e3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0600b2220 con 0x7fb06015d6b0 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 0x7fb0600c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:23.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb05f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 0x7fb0600c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37136/0 (socket says 172.21.15.70:37136) 2023-08-25T09:46:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.150+0000 7fb05f7fe640 1 -- 172.21.15.70:0/854092267 learned_addr learned my addr 172.21.15.70:0/854092267 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:23.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05effd640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0600ce480 0x7fb0600b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05ffff640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb06015d6b0 0x7fb06015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05f7fe640 1 -- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb06015d6b0 msgr2=0x7fb06015a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05f7fe640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb06015d6b0 0x7fb06015a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05f7fe640 1 -- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0600ce480 msgr2=0x7fb0600b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05f7fe640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0600ce480 0x7fb0600b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05f7fe640 1 -- 172.21.15.70:0/854092267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009400 con 0x7fb0600b0ec0 2023-08-25T09:46:23.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb05f7fe640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 0x7fb0600c8b00 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c870 tx=0x7fb05000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:23.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.151+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050003b70 con 0x7fb0600b0ec0 2023-08-25T09:46:23.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.152+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb050003cf0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.152+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0600b2500 con 0x7fb0600b0ec0 2023-08-25T09:46:23.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.152+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05000aa90 con 0x7fb0600b0ec0 2023-08-25T09:46:23.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.152+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0600b2e90 con 0x7fb0600b0ec0 2023-08-25T09:46:23.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.153+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0600b2cf0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.154+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb05000abf0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.155+0000 7fb057fff640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb04006a050 0x7fb04006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:23.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.156+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb050088510 con 0x7fb0600b0ec0 2023-08-25T09:46:23.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.156+0000 7fb05effd640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb04006a050 0x7fb04006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:23.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.156+0000 7fb05effd640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb04006a050 0x7fb04006c510 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb0600ad840 tx=0x7fb048008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:23.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.160+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0600b2cf0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.388+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0600b2cf0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.388+0000 7fb057fff640 1 -- 172.21.15.70:0/854092267 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0600b2cf0 con 0x7fb0600b0ec0 2023-08-25T09:46:23.391 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:46:23.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.391+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb04006a050 msgr2=0x7fb04006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:23.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb04006a050 0x7fb04006c510 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb0600ad840 tx=0x7fb048008040 comp rx=0 tx=0).stop 2023-08-25T09:46:23.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 msgr2=0x7fb0600c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:23.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 0x7fb0600c8b00 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb05000c870 tx=0x7fb05000cd30 comp rx=0 tx=0).stop 2023-08-25T09:46:23.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 shutdown_connections 2023-08-25T09:46:23.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb04006a050 0x7fb04006c510 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb06015d6b0 0x7fb06015a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0600ce480 0x7fb0600b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 --2- 172.21.15.70:0/854092267 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0600b0ec0 0x7fb0600c8b00 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:23.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 >> 172.21.15.70:0/854092267 conn(0x7fb060069c70 msgr2=0x7fb0600c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:23.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 shutdown_connections 2023-08-25T09:46:23.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:23.392+0000 7fb065e3f640 1 -- 172.21.15.70:0/854092267 wait complete. 2023-08-25T09:46:23.414 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:46:23.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:46:23.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:46:23.416 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:46:23.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:23.416 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:46:23.417 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:46:23.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:46:23.417 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:46:23.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:23.419 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:46:23.480 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:46:23.480 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:46:23.480 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:46:23.480 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:46:23.481 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:23.482 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:46:23.482 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:46:23.482 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:23.482 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:46:23.482 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:23.483 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:46:23.483 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41906sec preferred_lft 41906sec 2023-08-25T09:46:23.483 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:46:23.483 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.483 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:46:23.483 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:46:23.484 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.485 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:46:23.485 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:46:23.485 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:46:23.485 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:46:23.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:23.487 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:46:23.582 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:46:23.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:23.587 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:46:23.643 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:46:23.644 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:46:23.644 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:46:23.644 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:46:23.645 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:46:23.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:23.646 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:46:23.646 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:46:23.647 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:46:23.648 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:46:23.648 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:46:23.649 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:46:23.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:23.838 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:46:23.894 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:46:23.895 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:46:23.895 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:46:23.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:23.898 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:46:23.991 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:46:23.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:46:23.992 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:24.043 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:46:24.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:46:24.044 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:24.093 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:46:24.182 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:46:24.183 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:46:24.277 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:46:24.280 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:46:24.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:24.282 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:46:24.337 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:46:24.338 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:46:24.343 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:46:24.422 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:46:24.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:46:24.497 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:46:24.500 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:46:24.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:24.501 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:46:24.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:46:24.530+0000 7fecb016e480 -1 init, newargv = 0x55bb0e9baf50 newargc=15 2023-08-25T09:46:24.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[91335]: starting ceph client 2023-08-25T09:46:24.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[91335]: starting fuse 2023-08-25T09:46:24.548 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:46:24.549 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:46:24.549 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:46:24.550 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:46:24.551 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:46:24.551 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:46:24.552 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:46:24.553 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:46:24.553 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:46:24.554 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:46:24.555 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:46:24.555 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:46:24.556 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:46:24.557 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:46:24.558 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:46:24.558 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:46:24.559 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:46:24.560 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:46:24.560 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:46:24.561 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:46:24.562 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:46:24.562 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:46:24.563 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:46:24.564 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:46:24.564 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:46:24.565 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:46:24.566 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:46:24.567 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:46:24.567 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:46:24.751 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.91335.asok 2023-08-25T09:46:24.756 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:24.768 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.91335.asok 2023-08-25T09:46:24.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:24.769 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.91335.asok status 2023-08-25T09:46:25.007 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:46:25.007 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:46:25.007 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:46:25.007 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:46:25.007 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "91335", 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:46:25.008 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5797, 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5797 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:46:25.009 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3409367006 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:46:25.010 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 3409367006 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5797 192.168.0.1:0/3409367006", 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/3409367006", 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 57, 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 133, 2023-08-25T09:46:25.011 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:46:25.012 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:46:25.012 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:46:25.012 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:46:25.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:25.022 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:46:25.022 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:25.072 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:46:25.072 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:25.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:25.073 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:25.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:25.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.376+0000 7f15d49c8640 1 Processor -- start 2023-08-25T09:46:25.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.376+0000 7f15d49c8640 1 -- start start 2023-08-25T09:46:25.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15d49c8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f15d00ce5b0 0x7f15d00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15d49c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15d015aa10 0x7f15d00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15d49c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00b2680 0x7f15d006be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15d49c8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d0159d10 con 0x7f15d015aa10 2023-08-25T09:46:25.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15d49c8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d00b2af0 con 0x7f15d00ce5b0 2023-08-25T09:46:25.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15d49c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00ceed0 con 0x7f15d00b2680 2023-08-25T09:46:25.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15cdd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15d015aa10 0x7f15d00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15cdd9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f15d015aa10 0x7f15d00b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37140/0 (socket says 172.21.15.70:37140) 2023-08-25T09:46:25.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.378+0000 7f15cdd9b640 1 -- 172.21.15.70:0/296021739 learned_addr learned my addr 172.21.15.70:0/296021739 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15ced9d640 1 --2- 172.21.15.70:0/296021739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00b2680 0x7f15d006be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15ce59c640 1 --2- 172.21.15.70:0/296021739 >> v2:172.21.15.155:3300/0 conn(0x7f15d00ce5b0 0x7f15d00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15cdd9b640 1 -- 172.21.15.70:0/296021739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00b2680 msgr2=0x7f15d006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:25.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15cdd9b640 1 --2- 172.21.15.70:0/296021739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00b2680 0x7f15d006be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15cdd9b640 1 -- 172.21.15.70:0/296021739 >> v2:172.21.15.155:3300/0 conn(0x7f15d00ce5b0 msgr2=0x7f15d00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15cdd9b640 1 --2- 172.21.15.70:0/296021739 >> v2:172.21.15.155:3300/0 conn(0x7f15d00ce5b0 0x7f15d00ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.379+0000 7f15cdd9b640 1 -- 172.21.15.70:0/296021739 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d015d3f0 con 0x7f15d015aa10 2023-08-25T09:46:25.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.380+0000 7f15cdd9b640 1 --2- 172.21.15.70:0/296021739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d00b2140 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f15d015d5e0 tx=0x7f15b802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e38b8fb030eacad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.380+0000 7f15cd59a640 1 -- 172.21.15.70:0/296021739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b802c9d0 con 0x7f15d015aa10 2023-08-25T09:46:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.380+0000 7f15cd59a640 1 -- 172.21.15.70:0/296021739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b802cb30 con 0x7f15d015aa10 2023-08-25T09:46:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.380+0000 7f15cd59a640 1 -- 172.21.15.70:0/296021739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b80343f0 con 0x7f15d015aa10 2023-08-25T09:46:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 -- 172.21.15.70:0/296021739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 msgr2=0x7f15d00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:25.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 --2- 172.21.15.70:0/296021739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d00b2140 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f15d015d5e0 tx=0x7f15b802be30 comp rx=0 tx=0).stop 2023-08-25T09:46:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 -- 172.21.15.70:0/296021739 shutdown_connections 2023-08-25T09:46:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 --2- 172.21.15.70:0/296021739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00b2680 0x7f15d006be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 --2- 172.21.15.70:0/296021739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d00b2140 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 --2- 172.21.15.70:0/296021739 >> v2:172.21.15.155:3300/0 conn(0x7f15d00ce5b0 0x7f15d00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 -- 172.21.15.70:0/296021739 >> 172.21.15.70:0/296021739 conn(0x7f15d0069850 msgr2=0x7f15d0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:25.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 -- 172.21.15.70:0/296021739 shutdown_connections 2023-08-25T09:46:25.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.381+0000 7f15d49c8640 1 -- 172.21.15.70:0/296021739 wait complete. 2023-08-25T09:46:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.383+0000 7f15d49c8640 1 Processor -- start 2023-08-25T09:46:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.383+0000 7f15d49c8640 1 -- start start 2023-08-25T09:46:25.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15d49c8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d00b2680 0x7f15d0154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15d49c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00ce5b0 0x7f15d01f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15d49c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d01f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15d49c8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01f8230 con 0x7f15d015aa10 2023-08-25T09:46:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15d49c8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01f8510 con 0x7f15d00b2680 2023-08-25T09:46:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15d49c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01f87f0 con 0x7f15d00ce5b0 2023-08-25T09:46:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.384+0000 7f15ced9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d01f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d01f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37152/0 (socket says 172.21.15.70:37152) 2023-08-25T09:46:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 -- 172.21.15.70:0/4118392200 learned_addr learned my addr 172.21.15.70:0/4118392200 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ce59c640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d00b2680 0x7f15d0154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15cdd9b640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00ce5b0 0x7f15d01f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 -- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00ce5b0 msgr2=0x7f15d01f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00ce5b0 0x7f15d01f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 -- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d00b2680 msgr2=0x7f15d0154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d00b2680 0x7f15d0154160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 -- 172.21.15.70:0/4118392200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8009b30 con 0x7f15d015aa10 2023-08-25T09:46:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15ced9d640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d01f92d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c550 tx=0x7f15c800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:25.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8004220 con 0x7f15d015aa10 2023-08-25T09:46:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8004380 con 0x7f15d015aa10 2023-08-25T09:46:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.385+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8003830 con 0x7f15d015aa10 2023-08-25T09:46:25.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.386+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d01f8ad0 con 0x7f15d015aa10 2023-08-25T09:46:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.386+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d01546a0 con 0x7f15d015aa10 2023-08-25T09:46:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.387+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d01f8ff0 con 0x7f15d015aa10 2023-08-25T09:46:25.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.388+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f15c8003990 con 0x7f15d015aa10 2023-08-25T09:46:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.389+0000 7f15c6ffd640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15b406a050 0x7f15b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.389+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15c8088270 con 0x7f15d015aa10 2023-08-25T09:46:25.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.389+0000 7f15ce59c640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15b406a050 0x7f15b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.390+0000 7f15ce59c640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15b406a050 0x7f15b406c510 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f15d006cc90 tx=0x7f15c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.393+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f15c8053960 con 0x7f15d015aa10 2023-08-25T09:46:25.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.622+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d004e420 con 0x7f15d015aa10 2023-08-25T09:46:25.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.623+0000 7f15c6ffd640 1 -- 172.21.15.70:0/4118392200 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7f15c8056f80 con 0x7f15d015aa10 2023-08-25T09:46:25.626 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:25.626 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:25.626 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:25.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15b406a050 msgr2=0x7f15b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:25.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15b406a050 0x7f15b406c510 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f15d006cc90 tx=0x7f15c0008040 comp rx=0 tx=0).stop 2023-08-25T09:46:25.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 msgr2=0x7f15d01f92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:25.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d01f92d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f15c800c550 tx=0x7f15c800ca10 comp rx=0 tx=0).stop 2023-08-25T09:46:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 shutdown_connections 2023-08-25T09:46:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f15b406a050 0x7f15b406c510 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15d015aa10 0x7f15d01f92d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15d00ce5b0 0x7f15d01f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 --2- 172.21.15.70:0/4118392200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15d00b2680 0x7f15d0154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:25.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.626+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 >> 172.21.15.70:0/4118392200 conn(0x7f15d0069850 msgr2=0x7f15d0158c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:25.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.627+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 shutdown_connections 2023-08-25T09:46:25.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:25.627+0000 7f15d49c8640 1 -- 172.21.15.70:0/4118392200 wait complete. 2023-08-25T09:46:25.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:46:25.880 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":7746,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.038717093000001,"mdsmap_epoch":57,"osdmap_epoch":133,"osdmap_epoch_barrier":133,"uptime":18.45028211} 2023-08-25T09:46:25.881 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 7746, "mdsmap_epoch": 57, "osdmap_epoch": 133, "osdmap_epoch_barrier": 133, "rank_uptime": 12.038717093, "state": "up:active", "uptime": 18.45028211, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:46:25.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-08-25T09:46:26.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.093+0000 7f06983c8640 1 Processor -- start 2023-08-25T09:46:26.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.093+0000 7f06983c8640 1 -- start start 2023-08-25T09:46:26.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f06983c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f06900ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:26.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f06983c8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06900ca820 0x7f069015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:26.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f06983c8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06900cacb0 con 0x7f0690155120 2023-08-25T09:46:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f06983c8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06900caf30 con 0x7f06900ca820 2023-08-25T09:46:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f06983c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069015e6f0 con 0x7f06900cda60 2023-08-25T09:46:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f0696965640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06900ca820 0x7f069015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.095+0000 7f0696965640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f06900ca820 0x7f069015af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37156/0 (socket says 172.21.15.70:37156) 2023-08-25T09:46:26.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.096+0000 7f0696965640 1 -- 172.21.15.70:0/2827175599 learned_addr learned my addr 172.21.15.70:0/2827175599 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.096+0000 7f0696965640 1 -- 172.21.15.70:0/2827175599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 msgr2=0x7f06900ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.096+0000 7f0695963640 1 --2- 172.21.15.70:0/2827175599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f06900ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.096+0000 7f0696965640 1 --2- 172.21.15.70:0/2827175599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f06900ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.096+0000 7f0696965640 1 -- 172.21.15.70:0/2827175599 >> v1:172.21.15.155:6789/0 conn(0x7f0690155120 legacy=0x7f06900cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:26.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.096+0000 7f0696965640 1 -- 172.21.15.70:0/2827175599 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069015e9d0 con 0x7f06900ca820 2023-08-25T09:46:26.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.097+0000 7f0696965640 1 --2- 172.21.15.70:0/2827175599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06900ca820 0x7f069015af60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f068802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eeb0bb5b6f0ec92 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:26.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.097+0000 7f0695162640 1 -- 172.21.15.70:0/2827175599 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688030840 con 0x7f06900ca820 2023-08-25T09:46:26.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f0695162640 1 -- 172.21.15.70:0/2827175599 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06880309a0 con 0x7f06900ca820 2023-08-25T09:46:26.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f0695162640 1 -- 172.21.15.70:0/2827175599 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688030d30 con 0x7f06900ca820 2023-08-25T09:46:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 -- 172.21.15.70:0/2827175599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06900ca820 msgr2=0x7f069015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 --2- 172.21.15.70:0/2827175599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06900ca820 0x7f069015af60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f068802b980 comp rx=0 tx=0).stop 2023-08-25T09:46:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 -- 172.21.15.70:0/2827175599 shutdown_connections 2023-08-25T09:46:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 --2- 172.21.15.70:0/2827175599 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06900ca820 0x7f069015af60 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 --2- 172.21.15.70:0/2827175599 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f06900ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 -- 172.21.15.70:0/2827175599 >> 172.21.15.70:0/2827175599 conn(0x7f0690069c70 msgr2=0x7f069006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 -- 172.21.15.70:0/2827175599 shutdown_connections 2023-08-25T09:46:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.098+0000 7f06983c8640 1 -- 172.21.15.70:0/2827175599 wait complete. 2023-08-25T09:46:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.100+0000 7f06983c8640 1 Processor -- start 2023-08-25T09:46:26.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.100+0000 7f06983c8640 1 -- start start 2023-08-25T09:46:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.101+0000 7f06983c8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06900ca820 0x7f06901f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.101+0000 7f06983c8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f069014d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.101+0000 7f06983c8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 0x7f06900c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:26.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.101+0000 7f06983c8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06901f2190 con 0x7f0690155120 2023-08-25T09:46:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.101+0000 7f06983c8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06901f1580 con 0x7f06900ca820 2023-08-25T09:46:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.101+0000 7f06983c8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06901f1860 con 0x7f06900cda60 2023-08-25T09:46:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 0x7f06900c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:26.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 0x7f06900c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37172/0 (socket says 172.21.15.70:37172) 2023-08-25T09:46:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 -- 172.21.15.70:0/4209117385 learned_addr learned my addr 172.21.15.70:0/4209117385 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696164640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06900ca820 0x7f06901f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0695963640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f069014d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:26.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 -- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 msgr2=0x7f069014d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f069014d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 -- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06900ca820 msgr2=0x7f06901f0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06900ca820 0x7f06901f0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 -- 172.21.15.70:0/4209117385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880094d0 con 0x7f0690155120 2023-08-25T09:46:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0695963640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f069014d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.102+0000 7f0696965640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 0x7f06900c7140 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f0688004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.103+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880307d0 con 0x7f0690155120 2023-08-25T09:46:26.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.103+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688039d90 con 0x7f0690155120 2023-08-25T09:46:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.103+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688043650 con 0x7f0690155120 2023-08-25T09:46:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.103+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06901f1b40 con 0x7f0690155120 2023-08-25T09:46:26.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.103+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069014d970 con 0x7f0690155120 2023-08-25T09:46:26.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.104+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06901f2300 con 0x7f0690155120 2023-08-25T09:46:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.105+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f068802f390 con 0x7f0690155120 2023-08-25T09:46:26.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.106+0000 7f0686ffd640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f067406a050 0x7f067406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.106+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06880aa900 con 0x7f0690155120 2023-08-25T09:46:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.106+0000 7f0696164640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f067406a050 0x7f067406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.107+0000 7f0696164640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f067406a050 0x7f067406c510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f06900b0b90 tx=0x7f068c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:26.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.110+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0688076100 con 0x7f0690155120 2023-08-25T09:46:26.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.300+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f069004e420 con 0x7f0690155120 2023-08-25T09:46:26.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.533+0000 7f0686ffd640 1 -- 172.21.15.70:0/4209117385 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v134) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0688079720 con 0x7f0690155120 2023-08-25T09:46:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-08-25T09:46:26.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f067406a050 msgr2=0x7f067406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:26.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f067406a050 0x7f067406c510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f06900b0b90 tx=0x7f068c009040 comp rx=0 tx=0).stop 2023-08-25T09:46:26.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 msgr2=0x7f06900c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:26.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 0x7f06900c7140 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f0688009870 tx=0x7f0688004730 comp rx=0 tx=0).stop 2023-08-25T09:46:26.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 shutdown_connections 2023-08-25T09:46:26.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f067406a050 0x7f067406c510 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0690155120 0x7f06900c7140 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06900cda60 0x7f069014d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 --2- 172.21.15.70:0/4209117385 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06900ca820 0x7f06901f0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:26.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.537+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 >> 172.21.15.70:0/4209117385 conn(0x7f0690069c70 msgr2=0x7f06900cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:26.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.538+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 shutdown_connections 2023-08-25T09:46:26.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26.538+0000 7f06983c8640 1 -- 172.21.15.70:0/4209117385 wait complete. 2023-08-25T09:46:26.562 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:46:26.563 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:26.564 DEBUG:teuthology.orchestra.run.smithi070:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-08-25T09:46:26.564 DEBUG:teuthology.orchestra.run.smithi070:> os.fchmod(fd, 0O666) 2023-08-25T09:46:26.564 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(fd) 2023-08-25T09:46:26.565 DEBUG:teuthology.orchestra.run.smithi070:> os.close(fd) 2023-08-25T09:46:26.565 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:26.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:26.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:46:26.737 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:46:26.737 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:46:26.737 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:26.738 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:46:26.738 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:26.738 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:46:26.739 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:46:26.739 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:46:26.739 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:46:26.740 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:26.740 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:46:26.740 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:46:26.740 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:46:26.741 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:46:26.741 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:26.741 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:46:26.742 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:46:26.742 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:46:26.742 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:46:26.743 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:46:26.743 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:46:26.744 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:46:26.744 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:46:26.744 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:26.745 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:46:26.745 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:26.962 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:26.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:26.979 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:26.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:26.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113769.asok status 2023-08-25T09:46:27.221 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:27.221 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:46:27.221 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "113769", 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:46:27.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:27.223 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 1, 2023-08-25T09:46:27.223 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:46:27.223 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7926, 2023-08-25T09:46:27.223 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:46:27.224 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:46:27.224 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:46:27.224 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 7926 2023-08-25T09:46:27.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:27.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:27.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:27.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:27.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:27.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:27.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.7926 192.168.0.1:0/3843564457", 2023-08-25T09:46:27.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3843564457", 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 2, 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 57, 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 133, 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 133, 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:46:27.227 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:27.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-08-25T09:46:27.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.453+0000 7f947b59e640 1 Processor -- start 2023-08-25T09:46:27.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.453+0000 7f947b59e640 1 -- start start 2023-08-25T09:46:27.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.454+0000 7f947b59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f947c159930 0x7f947c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f947b59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c159d10 con 0x7f947c0b3c80 2023-08-25T09:46:27.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f947b59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c0b2d10 con 0x7f947c0cb200 2023-08-25T09:46:27.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f947b59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c0cb690 con 0x7f947c159930 2023-08-25T09:46:27.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f9479d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f947c0b3c80 0x7f947c0caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:52166/0 (socket says 172.21.15.70:52166) 2023-08-25T09:46:27.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f9479d9b640 1 -- 172.21.15.70:0/1404742311 learned_addr learned my addr 172.21.15.70:0/1404742311 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:27.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088606910 0 0) 0x7f947c159d10 con 0x7f947c0b3c80 2023-08-25T09:46:27.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9468003520 con 0x7f947c0b3c80 2023-08-25T09:46:27.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.455+0000 7f947a59c640 1 --2- 172.21.15.70:0/1404742311 >> v2:172.21.15.155:3300/0 conn(0x7f947c159930 0x7f947c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:27.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1581956423 0 0) 0x7f9468003520 con 0x7f947c0b3c80 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 >> v2:172.21.15.155:3300/0 conn(0x7f947c159930 msgr2=0x7f947c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 --2- 172.21.15.70:0/1404742311 >> v2:172.21.15.155:3300/0 conn(0x7f947c159930 0x7f947c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 >> v1:172.21.15.155:6789/0 conn(0x7f947c0cb200 legacy=0x7f947c0cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0b2ff0 con 0x7f947c0b3c80 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9464001670 con 0x7f947c0b3c80 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 >> v1:172.21.15.70:6789/0 conn(0x7f947c0b3c80 legacy=0x7f947c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94680039e0 0x7f9468003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94680047c0 0x7f9468004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f946800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0cb690 con 0x7f94680052a0 2023-08-25T09:46:27.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946800c1f0 con 0x7f94680039e0 2023-08-25T09:46:27.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946800c3b0 con 0x7f94680047c0 2023-08-25T09:46:27.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.456+0000 7f947ad9d640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f946800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f947a59c640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94680039e0 0x7f9468003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f9479d9b640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94680047c0 0x7f9468004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f947ad9d640 1 -- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94680047c0 msgr2=0x7f9468004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f947ad9d640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94680047c0 0x7f9468004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f947ad9d640 1 -- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94680039e0 msgr2=0x7f9468003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f947ad9d640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94680039e0 0x7f9468003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.457+0000 7f947ad9d640 1 -- 172.21.15.70:0/1404742311 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9468003520 con 0x7f94680052a0 2023-08-25T09:46:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.458+0000 7f947ad9d640 1 -- 172.21.15.70:0/1404742311 reap_dead start 2023-08-25T09:46:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.458+0000 7f947ad9d640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f946800b980 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9474001720 tx=0x7f94680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed55c5c97f050e54 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.458+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740052d0 con 0x7f94680052a0 2023-08-25T09:46:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.458+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947402e050 con 0x7f94680052a0 2023-08-25T09:46:27.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.458+0000 7f947959a640 1 -- 172.21.15.70:0/1404742311 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740035f0 con 0x7f94680052a0 2023-08-25T09:46:27.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 -- 172.21.15.70:0/1404742311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 msgr2=0x7f946800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:27.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f946800b980 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f9474001720 tx=0x7f94680057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 -- 172.21.15.70:0/1404742311 shutdown_connections 2023-08-25T09:46:27.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 --2- 172.21.15.70:0/1404742311 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f946800b980 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 -- 172.21.15.70:0/1404742311 >> 172.21.15.70:0/1404742311 conn(0x7f947c0aff60 msgr2=0x7f947c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:27.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 -- 172.21.15.70:0/1404742311 shutdown_connections 2023-08-25T09:46:27.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.459+0000 7f947b59e640 1 -- 172.21.15.70:0/1404742311 wait complete. 2023-08-25T09:46:27.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.461+0000 7f947b59e640 1 Processor -- start 2023-08-25T09:46:27.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.461+0000 7f947b59e640 1 -- start start 2023-08-25T09:46:27.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f947c1f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f947c0c3f70 0x7f947c0acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947b59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f947c15b7d0 0x7f947c0c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c1f53f0 con 0x7f94680052a0 2023-08-25T09:46:27.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947b59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0c4450 con 0x7f947c15b7d0 2023-08-25T09:46:27.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0c4730 con 0x7f947c0c3f70 2023-08-25T09:46:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f947c1f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f947c1f1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37192/0 (socket says 172.21.15.70:37192) 2023-08-25T09:46:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 -- 172.21.15.70:0/3107444572 learned_addr learned my addr 172.21.15.70:0/3107444572 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 -- 172.21.15.70:0/3107444572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f947c0c3f70 msgr2=0x7f947c0acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f947c0c3f70 0x7f947c0acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 -- 172.21.15.70:0/3107444572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f947c15b7d0 msgr2=0x7f947c0c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:27.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f947c15b7d0 0x7f947c0c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:27.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.462+0000 7f947a59c640 1 -- 172.21.15.70:0/3107444572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0c4a10 con 0x7f94680052a0 2023-08-25T09:46:27.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.463+0000 7f947a59c640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f947c1f1e70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f946c007e60 tx=0x7f946c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:27.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.463+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c0145f0 con 0x7f94680052a0 2023-08-25T09:46:27.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.463+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c014750 con 0x7f94680052a0 2023-08-25T09:46:27.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.463+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c0c4cf0 con 0x7f94680052a0 2023-08-25T09:46:27.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.463+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c014ae0 con 0x7f94680052a0 2023-08-25T09:46:27.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.463+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c0ad030 con 0x7f94680052a0 2023-08-25T09:46:27.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.464+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c0b32b0 con 0x7f94680052a0 2023-08-25T09:46:27.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.465+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f946c014c40 con 0x7f94680052a0 2023-08-25T09:46:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.466+0000 7f9472ffd640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f945c06a050 0x7f945c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.466+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f946c089320 con 0x7f94680052a0 2023-08-25T09:46:27.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.467+0000 7f9479d9b640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f945c06a050 0x7f945c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.468+0000 7f9479d9b640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f945c06a050 0x7f945c06c510 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9464000ff0 tx=0x7f9464009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:27.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.471+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f947c0b32b0 con 0x7f94680052a0 2023-08-25T09:46:27.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:27.670+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f947c0b32b0 con 0x7f94680052a0 2023-08-25T09:46:28.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.549+0000 7f9472ffd640 1 -- 172.21.15.70:0/3107444572 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v135) v1 ==== 93+0+0 (secure 0 0 0) 0x7f947c0b32b0 con 0x7f94680052a0 2023-08-25T09:46:28.552 INFO:teuthology.orchestra.run.smithi070.stderr:pauserd,pausewr is set 2023-08-25T09:46:28.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.552+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f945c06a050 msgr2=0x7f945c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:28.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f945c06a050 0x7f945c06c510 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9464000ff0 tx=0x7f9464009040 comp rx=0 tx=0).stop 2023-08-25T09:46:28.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 msgr2=0x7f947c1f1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:28.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f947c1f1e70 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f946c007e60 tx=0x7f946c00f520 comp rx=0 tx=0).stop 2023-08-25T09:46:28.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 shutdown_connections 2023-08-25T09:46:28.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f945c06a050 0x7f945c06c510 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f947c15b7d0 0x7f947c0c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f947c0c3f70 0x7f947c0acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 --2- 172.21.15.70:0/3107444572 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94680052a0 0x7f947c1f1e70 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 >> 172.21.15.70:0/3107444572 conn(0x7f947c0aff60 msgr2=0x7f947c15c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:28.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 shutdown_connections 2023-08-25T09:46:28.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.553+0000 7f947b59e640 1 -- 172.21.15.70:0/3107444572 wait complete. 2023-08-25T09:46:28.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-08-25T09:46:28.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.798+0000 7f117db39640 1 Processor -- start 2023-08-25T09:46:28.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.798+0000 7f117db39640 1 -- start start 2023-08-25T09:46:28.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.799+0000 7f117db39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f117815ccf0 0x7f11780b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:28.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.799+0000 7f117db39640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11780cab30 0x7f11780b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:28.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.799+0000 7f117db39640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1178161600 con 0x7f11780c7a70 2023-08-25T09:46:28.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.799+0000 7f117db39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1178071000 con 0x7f117815ccf0 2023-08-25T09:46:28.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.799+0000 7f117db39640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117815e0e0 con 0x7f11780cab30 2023-08-25T09:46:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f117815ccf0 0x7f11780b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f117815ccf0 0x7f11780b0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37206/0 (socket says 172.21.15.70:37206) 2023-08-25T09:46:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 -- 172.21.15.70:0/2290253239 learned_addr learned my addr 172.21.15.70:0/2290253239 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:28.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 -- 172.21.15.70:0/2290253239 >> v2:172.21.15.155:3300/0 conn(0x7f11780cab30 msgr2=0x7f11780b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 --2- 172.21.15.70:0/2290253239 >> v2:172.21.15.155:3300/0 conn(0x7f11780cab30 0x7f11780b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 -- 172.21.15.70:0/2290253239 >> v1:172.21.15.155:6789/0 conn(0x7f11780c7a70 legacy=0x7f11780ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.800+0000 7f11777fe640 1 -- 172.21.15.70:0/2290253239 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11781609f0 con 0x7f117815ccf0 2023-08-25T09:46:28.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.801+0000 7f11777fe640 1 --2- 172.21.15.70:0/2290253239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11780b0a10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f11780cf530 tx=0x7f116802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d4d3a577d398db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:28.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.802+0000 7f1176ffd640 1 -- 172.21.15.70:0/2290253239 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168004230 con 0x7f117815ccf0 2023-08-25T09:46:28.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.802+0000 7f1176ffd640 1 -- 172.21.15.70:0/2290253239 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168004390 con 0x7f117815ccf0 2023-08-25T09:46:28.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.802+0000 7f1176ffd640 1 -- 172.21.15.70:0/2290253239 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168003810 con 0x7f117815ccf0 2023-08-25T09:46:28.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.802+0000 7f117db39640 1 -- 172.21.15.70:0/2290253239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 msgr2=0x7f11780b0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:28.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.802+0000 7f117db39640 1 --2- 172.21.15.70:0/2290253239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11780b0a10 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f11780cf530 tx=0x7f116802b880 comp rx=0 tx=0).stop 2023-08-25T09:46:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.803+0000 7f117db39640 1 -- 172.21.15.70:0/2290253239 shutdown_connections 2023-08-25T09:46:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.803+0000 7f117db39640 1 --2- 172.21.15.70:0/2290253239 >> v2:172.21.15.155:3300/0 conn(0x7f11780cab30 0x7f11780b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.803+0000 7f117db39640 1 --2- 172.21.15.70:0/2290253239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11780b0a10 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.803+0000 7f117db39640 1 -- 172.21.15.70:0/2290253239 >> 172.21.15.70:0/2290253239 conn(0x7f11780ce760 msgr2=0x7f11780ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.803+0000 7f117db39640 1 -- 172.21.15.70:0/2290253239 shutdown_connections 2023-08-25T09:46:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.803+0000 7f117db39640 1 -- 172.21.15.70:0/2290253239 wait complete. 2023-08-25T09:46:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.805+0000 7f117db39640 1 Processor -- start 2023-08-25T09:46:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.805+0000 7f117db39640 1 -- start start 2023-08-25T09:46:28.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.806+0000 7f117db39640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11780c7a70 0x7f1178151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:28.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.806+0000 7f117db39640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11780cab30 0x7f1178155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:28.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.806+0000 7f117db39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11781562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:28.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.806+0000 7f117db39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11781f6990 con 0x7f117815ccf0 2023-08-25T09:46:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.806+0000 7f117db39640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11781f5d80 con 0x7f11780cab30 2023-08-25T09:46:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.806+0000 7f117db39640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11781f6060 con 0x7f11780c7a70 2023-08-25T09:46:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11781562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:28.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11781562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:37210/0 (socket says 172.21.15.70:37210) 2023-08-25T09:46:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 -- 172.21.15.70:0/109023581 learned_addr learned my addr 172.21.15.70:0/109023581 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f11777fe640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11780c7a70 0x7f1178151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:28.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f116ffff640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11780cab30 0x7f1178155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 -- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11780c7a70 msgr2=0x7f1178151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11780c7a70 0x7f1178151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 -- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11780cab30 msgr2=0x7f1178155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11780cab30 0x7f1178155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168009460 con 0x7f117815ccf0 2023-08-25T09:46:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.807+0000 7f1177fff640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11781562a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f117000b9e0 tx=0x7f117000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.808+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117000cb20 con 0x7f117815ccf0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.808+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117000cc80 con 0x7f117815ccf0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.808+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11781f6340 con 0x7f117815ccf0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.808+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170014680 con 0x7f117815ccf0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.808+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11781533d0 con 0x7f117815ccf0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.809+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11781f6b40 con 0x7f117815ccf0 2023-08-25T09:46:28.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.810+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f117001d3b0 con 0x7f117815ccf0 2023-08-25T09:46:28.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.811+0000 7f1174ff9640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f115806a080 0x7f115806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:28.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.811+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11700883c0 con 0x7f117815ccf0 2023-08-25T09:46:28.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.811+0000 7f11777fe640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f115806a080 0x7f115806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:28.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.811+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=136+}) v3 -- 0x7f11580721d0 con 0x7f117815ccf0 2023-08-25T09:46:28.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.812+0000 7f11777fe640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f115806a080 0x7f115806c540 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1168034000 tx=0x7f1168008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:28.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:28.815+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f11781f6b40 con 0x7f117815ccf0 2023-08-25T09:46:29.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.015+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f117804e420 con 0x7f117815ccf0 2023-08-25T09:46:29.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.570+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(136..136 src has 1..136) v4 ==== 288+0+0 (secure 0 0 0) 0x7f117000ce30 con 0x7f117815ccf0 2023-08-25T09:46:29.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.570+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=137}) v3 -- 0x7f1158072a80 con 0x7f117815ccf0 2023-08-25T09:46:29.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.575+0000 7f1174ff9640 1 -- 172.21.15.70:0/109023581 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1170019070 con 0x7f117815ccf0 2023-08-25T09:46:29.578 INFO:teuthology.orchestra.run.smithi070.stderr:pauserd,pausewr is unset 2023-08-25T09:46:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f115806a080 msgr2=0x7f115806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:29.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f115806a080 0x7f115806c540 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1168034000 tx=0x7f1168008030 comp rx=0 tx=0).stop 2023-08-25T09:46:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 msgr2=0x7f11781562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11781562a0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f117000b9e0 tx=0x7f117000bea0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 shutdown_connections 2023-08-25T09:46:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f115806a080 0x7f115806c540 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f117815ccf0 0x7f11781562a0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11780cab30 0x7f1178155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 --2- 172.21.15.70:0/109023581 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11780c7a70 0x7f1178151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.580+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 >> 172.21.15.70:0/109023581 conn(0x7f11780ce760 msgr2=0x7f117815d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.581+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 shutdown_connections 2023-08-25T09:46:29.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.581+0000 7f117db39640 1 -- 172.21.15.70:0/109023581 wait complete. 2023-08-25T09:46:29.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:29.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.818+0000 7f53409c4640 1 Processor -- start 2023-08-25T09:46:29.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.818+0000 7f53409c4640 1 -- start start 2023-08-25T09:46:29.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.819+0000 7f53409c4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f533c15ec10 0x7f533c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.819+0000 7f53409c4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f533c15c6d0 0x7f533c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.819+0000 7f53409c4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0b1b60 con 0x7f533c15e830 2023-08-25T09:46:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.819+0000 7f53409c4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c06bd80 con 0x7f533c15ec10 2023-08-25T09:46:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.819+0000 7f53409c4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c06b7c0 con 0x7f533c15c6d0 2023-08-25T09:46:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f533a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f533c15e830 0x7f533c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45424/0 (socket says 172.21.15.70:45424) 2023-08-25T09:46:29.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f533a59c640 1 -- 172.21.15.70:0/3342523430 learned_addr learned my addr 172.21.15.70:0/3342523430 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:29.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f5339d9b640 1 --2- 172.21.15.70:0/3342523430 >> v2:172.21.15.70:3300/0 conn(0x7f533c15ec10 0x7f533c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:29.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f533ad9d640 1 --2- 172.21.15.70:0/3342523430 >> v2:172.21.15.155:3300/0 conn(0x7f533c15c6d0 0x7f533c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:29.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f533959a640 1 -- 172.21.15.70:0/3342523430 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049062200 0 0) 0x7f533c0b1b60 con 0x7f533c15e830 2023-08-25T09:46:29.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f5339d9b640 1 -- 172.21.15.70:0/3342523430 >> v2:172.21.15.155:3300/0 conn(0x7f533c15c6d0 msgr2=0x7f533c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:29.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f5339d9b640 1 --2- 172.21.15.70:0/3342523430 >> v2:172.21.15.155:3300/0 conn(0x7f533c15c6d0 0x7f533c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f5339d9b640 1 -- 172.21.15.70:0/3342523430 >> v1:172.21.15.70:6789/0 conn(0x7f533c15e830 legacy=0x7f533c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:29.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f5339d9b640 1 -- 172.21.15.70:0/3342523430 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c06baa0 con 0x7f533c15ec10 2023-08-25T09:46:29.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.820+0000 7f533ad9d640 1 --2- 172.21.15.70:0/3342523430 >> v2:172.21.15.155:3300/0 conn(0x7f533c15c6d0 0x7f533c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.821+0000 7f5339d9b640 1 --2- 172.21.15.70:0/3342523430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15ec10 0x7f533c15c190 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f533c0cf460 tx=0x7f532402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4145e917247d6f33 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.821+0000 7f533959a640 1 -- 172.21.15.70:0/3342523430 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532402e070 con 0x7f533c15ec10 2023-08-25T09:46:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.821+0000 7f533959a640 1 -- 172.21.15.70:0/3342523430 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53240315a0 con 0x7f533c15ec10 2023-08-25T09:46:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.821+0000 7f533959a640 1 -- 172.21.15.70:0/3342523430 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5324031930 con 0x7f533c15ec10 2023-08-25T09:46:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 -- 172.21.15.70:0/3342523430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15ec10 msgr2=0x7f533c15c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:29.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 --2- 172.21.15.70:0/3342523430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15ec10 0x7f533c15c190 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f533c0cf460 tx=0x7f532402b840 comp rx=0 tx=0).stop 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 -- 172.21.15.70:0/3342523430 shutdown_connections 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 --2- 172.21.15.70:0/3342523430 >> v2:172.21.15.155:3300/0 conn(0x7f533c15c6d0 0x7f533c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 --2- 172.21.15.70:0/3342523430 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15ec10 0x7f533c15c190 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 -- 172.21.15.70:0/3342523430 >> 172.21.15.70:0/3342523430 conn(0x7f533c069850 msgr2=0x7f533c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 -- 172.21.15.70:0/3342523430 shutdown_connections 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.822+0000 7f53409c4640 1 -- 172.21.15.70:0/3342523430 wait complete. 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.824+0000 7f53409c4640 1 Processor -- start 2023-08-25T09:46:29.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.824+0000 7f53409c4640 1 -- start start 2023-08-25T09:46:29.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f53409c4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 0x7f533c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f53409c4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f533c15e830 0x7f533c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f53409c4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f533c15ec10 0x7f533c145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f53409c4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c143980 con 0x7f533c15c6d0 2023-08-25T09:46:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f53409c4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c154ca0 con 0x7f533c15ec10 2023-08-25T09:46:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f53409c4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c154f80 con 0x7f533c15e830 2023-08-25T09:46:29.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f533a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 0x7f533c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f533a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 0x7f533c152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47516/0 (socket says 172.21.15.70:47516) 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.825+0000 7f533a59c640 1 -- 172.21.15.70:0/122787801 learned_addr learned my addr 172.21.15.70:0/122787801 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533ad9d640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f533c15ec10 0x7f533c145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533a59c640 1 -- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f533c15e830 msgr2=0x7f533c152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533a59c640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f533c15e830 0x7f533c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533a59c640 1 -- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f533c15ec10 msgr2=0x7f533c145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:29.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533a59c640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f533c15ec10 0x7f533c145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533a59c640 1 -- 172.21.15.70:0/122787801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5324009400 con 0x7f533c15c6d0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f533a59c640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 0x7f533c152a00 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c4d0 tx=0x7f532c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0074e0 con 0x7f533c15c6d0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c007640 con 0x7f533c15c6d0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.826+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c155260 con 0x7f533c15c6d0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.827+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c004a20 con 0x7f533c15c6d0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.827+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1549b0 con 0x7f533c15c6d0 2023-08-25T09:46:29.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.828+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c155780 con 0x7f533c15c6d0 2023-08-25T09:46:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.829+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f532c004b80 con 0x7f533c15c6d0 2023-08-25T09:46:29.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.831+0000 7f5332ffd640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f531c06a050 0x7f531c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:29.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.831+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f532c087af0 con 0x7f533c15c6d0 2023-08-25T09:46:29.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.831+0000 7f5339d9b640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f531c06a050 0x7f531c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:29.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.831+0000 7f5339d9b640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f531c06a050 0x7f531c06c510 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5324036000 tx=0x7f5324030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:29.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:29.834+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f533c155780 con 0x7f533c15c6d0 2023-08-25T09:46:30.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.027+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f533c155780 con 0x7f533c15c6d0 2023-08-25T09:46:30.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.030+0000 7f5332ffd640 1 -- 172.21.15.70:0/122787801 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f533c155780 con 0x7f533c15c6d0 2023-08-25T09:46:30.033 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:30.033 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":136,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:29.557323+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:30.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.032+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f531c06a050 msgr2=0x7f531c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:30.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.032+0000 7f53409c4640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f531c06a050 0x7f531c06c510 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5324036000 tx=0x7f5324030000 comp rx=0 tx=0).stop 2023-08-25T09:46:30.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.032+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 msgr2=0x7f533c152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:30.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.032+0000 7f53409c4640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 0x7f533c152a00 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f532c00c4d0 tx=0x7f532c00c990 comp rx=0 tx=0).stop 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 shutdown_connections 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f531c06a050 0x7f531c06c510 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f533c15ec10 0x7f533c145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f533c15e830 0x7f533c152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 --2- 172.21.15.70:0/122787801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f533c15c6d0 0x7f533c152a00 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 >> 172.21.15.70:0/122787801 conn(0x7f533c069850 msgr2=0x7f533c0ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:30.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 shutdown_connections 2023-08-25T09:46:30.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.033+0000 7f53409c4640 1 -- 172.21.15.70:0/122787801 wait complete. 2023-08-25T09:46:30.055 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:46:30.056 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:30.056 DEBUG:teuthology.orchestra.run.smithi070:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-08-25T09:46:30.056 DEBUG:teuthology.orchestra.run.smithi070:> os.fchmod(fd, 0O666) 2023-08-25T09:46:30.057 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(fd) 2023-08-25T09:46:30.057 DEBUG:teuthology.orchestra.run.smithi070:> os.close(fd) 2023-08-25T09:46:30.057 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:30.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:30.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:46:30.220 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:46:30.221 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:46:30.221 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:30.222 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:46:30.222 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:30.222 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:46:30.223 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:46:30.223 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:46:30.224 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:46:30.224 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:30.225 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:46:30.225 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:46:30.225 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:46:30.226 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:46:30.226 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:30.227 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:46:30.227 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:46:30.227 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:46:30.228 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:46:30.228 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:46:30.229 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:46:30.229 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:46:30.229 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:46:30.230 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:30.230 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:46:30.230 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:30.445 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:30.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:30.461 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:30.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:30.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113769.asok status 2023-08-25T09:46:30.703 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:46:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:46:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:46:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:46:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:46:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "113769", 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 1, 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7926, 2023-08-25T09:46:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:46:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:46:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:46:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 7926 2023-08-25T09:46:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.7926 192.168.0.1:0/3843564457", 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3843564457", 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 2, 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 57, 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 133, 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 133, 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:46:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:46:30.710 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:30.718 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.924+0000 7fc5fd724640 1 Processor -- start 2023-08-25T09:46:30.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.925+0000 7fc5fd724640 1 -- start start 2023-08-25T09:46:30.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.926+0000 7fc5fd724640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5f80b1ff0 0x7fc5f80b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:30.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.926+0000 7fc5fd724640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f80b1b60 con 0x7fc5f815ec10 2023-08-25T09:46:30.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.926+0000 7fc5fd724640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f806b7c0 con 0x7fc5f815e830 2023-08-25T09:46:30.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.926+0000 7fc5fd724640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f806bd80 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5f77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc5f80b1ff0 0x7fc5f80b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:30.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5effff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc5f815ec10 0x7fc5f815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45428/0 (socket says 172.21.15.70:45428) 2023-08-25T09:46:30.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5effff640 1 -- 172.21.15.70:0/1143858614 learned_addr learned my addr 172.21.15.70:0/1143858614 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:30.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5f77fe640 1 -- 172.21.15.70:0/1143858614 >> v1:172.21.15.155:6789/0 conn(0x7fc5f815e830 legacy=0x7fc5f806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:30.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5f77fe640 1 -- 172.21.15.70:0/1143858614 >> v1:172.21.15.70:6789/0 conn(0x7fc5f815ec10 legacy=0x7fc5f815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:30.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5f77fe640 1 -- 172.21.15.70:0/1143858614 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f806baa0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.927+0000 7fc5f67fc640 1 -- 172.21.15.70:0/1143858614 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775728658 0 0) 0x7fc5f80b1b60 con 0x7fc5f815ec10 2023-08-25T09:46:30.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.928+0000 7fc5f77fe640 1 --2- 172.21.15.70:0/1143858614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80b2440 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009d90 tx=0x7fc5f002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d03bf832e17c10e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:30.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.928+0000 7fc5f67fc640 1 -- 172.21.15.70:0/1143858614 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f002cc60 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5f67fc640 1 -- 172.21.15.70:0/1143858614 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f002cdc0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5f67fc640 1 -- 172.21.15.70:0/1143858614 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0034660 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 -- 172.21.15.70:0/1143858614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 msgr2=0x7fc5f80b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:30.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 --2- 172.21.15.70:0/1143858614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80b2440 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009d90 tx=0x7fc5f002beb0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 -- 172.21.15.70:0/1143858614 shutdown_connections 2023-08-25T09:46:30.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 --2- 172.21.15.70:0/1143858614 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80b2440 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 -- 172.21.15.70:0/1143858614 >> 172.21.15.70:0/1143858614 conn(0x7fc5f8069850 msgr2=0x7fc5f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:30.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 -- 172.21.15.70:0/1143858614 shutdown_connections 2023-08-25T09:46:30.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.929+0000 7fc5fd724640 1 -- 172.21.15.70:0/1143858614 wait complete. 2023-08-25T09:46:30.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.931+0000 7fc5fd724640 1 Processor -- start 2023-08-25T09:46:30.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.931+0000 7fc5fd724640 1 -- start start 2023-08-25T09:46:30.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5fd724640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5fd724640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f815e830 0x7fc5f81f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5fd724640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f815ec10 0x7fc5f814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5fd724640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c79d0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5fd724640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c7c20 con 0x7fc5f815ec10 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5fd724640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c7f00 con 0x7fc5f815e830 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5f6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:30.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5f6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47534/0 (socket says 172.21.15.70:47534) 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.932+0000 7fc5f6ffd640 1 -- 172.21.15.70:0/347422188 learned_addr learned my addr 172.21.15.70:0/347422188 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f6ffd640 1 -- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f815e830 msgr2=0x7fc5f81f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f6ffd640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f815e830 0x7fc5f81f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f6ffd640 1 -- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f815ec10 msgr2=0x7fc5f814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f77fe640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f815ec10 0x7fc5f814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f6ffd640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f815ec10 0x7fc5f814cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:30.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f6ffd640 1 -- 172.21.15.70:0/347422188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0009a70 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f77fe640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f815ec10 0x7fc5f814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:30.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5f6ffd640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80c9c40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c430 tx=0x7fc5e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:30.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8011790 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e80118f0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.933+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f80c81e0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.934+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8011ce0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.934+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80ca180 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.935+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f80c89e0 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.936+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc5e801c460 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.937+0000 7fc5ef7fe640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc5d806a050 0x7fc5d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:30.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.937+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5e8088520 con 0x7fc5f80b1ff0 2023-08-25T09:46:30.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.937+0000 7fc5effff640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc5d806a050 0x7fc5d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:30.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.938+0000 7fc5effff640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc5d806a050 0x7fc5d806c510 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0007fd0 tx=0x7fc5e0007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:30.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:30.941+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc5f80c89e0 con 0x7fc5f80b1ff0 2023-08-25T09:46:31.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.172+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5f80c89e0 con 0x7fc5f80b1ff0 2023-08-25T09:46:31.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.173+0000 7fc5ef7fe640 1 -- 172.21.15.70:0/347422188 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7fc5f80c89e0 con 0x7fc5f80b1ff0 2023-08-25T09:46:31.176 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:31.177 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:31.177 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc5d806a050 msgr2=0x7fc5d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc5d806a050 0x7fc5d806c510 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5e0007fd0 tx=0x7fc5e0007960 comp rx=0 tx=0).stop 2023-08-25T09:46:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 msgr2=0x7fc5f80c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80c9c40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fc5e800c430 tx=0x7fc5e800c8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:31.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 shutdown_connections 2023-08-25T09:46:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc5d806a050 0x7fc5d806c510 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc5f815ec10 0x7fc5f814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:31.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc5f815e830 0x7fc5f81f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 --2- 172.21.15.70:0/347422188 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc5f80b1ff0 0x7fc5f80c9c40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.176+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 >> 172.21.15.70:0/347422188 conn(0x7fc5f8069850 msgr2=0x7fc5f81e61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.177+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 shutdown_connections 2023-08-25T09:46:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31.177+0000 7fc5fd724640 1 -- 172.21.15.70:0/347422188 wait complete. 2023-08-25T09:46:31.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 136 2023-08-25T09:46:31.428 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-08-25T09:46:31.428 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:46:31.429 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:31.429 DEBUG:teuthology.orchestra.run.smithi070:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-08-25T09:46:31.429 DEBUG:teuthology.orchestra.run.smithi070:> os.fchmod(fd, 0O666) 2023-08-25T09:46:31.429 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(fd) 2023-08-25T09:46:31.430 DEBUG:teuthology.orchestra.run.smithi070:> os.close(fd) 2023-08-25T09:46:31.430 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:31.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:31.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:46:31.594 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:46:31.594 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:46:31.594 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:31.595 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:46:31.595 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:31.595 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:46:31.595 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:46:31.596 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:46:31.596 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:46:31.596 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:31.597 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:46:31.597 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:46:31.597 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:46:31.598 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:46:31.598 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:31.598 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:46:31.598 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:46:31.599 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:46:31.599 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:46:31.599 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:46:31.600 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:46:31.600 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:46:31.600 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:46:31.601 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:31.601 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:46:31.601 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:31.818 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:31.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:31.836 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:31.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:31.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113769.asok status 2023-08-25T09:46:32.076 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:46:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:46:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:46:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:46:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:46:32.077 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "113769", 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 2, 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7926, 2023-08-25T09:46:32.078 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 7926 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:32.079 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:32.080 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.7926 192.168.0.1:0/3843564457", 2023-08-25T09:46:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3843564457", 2023-08-25T09:46:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 3, 2023-08-25T09:46:32.081 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 57, 2023-08-25T09:46:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 136, 2023-08-25T09:46:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 136, 2023-08-25T09:46:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:46:32.082 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:46:32.082 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:32.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:46:32.092 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:46:32.092 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:46:32.092 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:46:32.093 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:46:32.093 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:32.093 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:46:32.094 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:46:32.094 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:46:32.094 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:46:32.095 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:32.095 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:46:32.095 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:46:32.096 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:46:32.097 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:46:32.097 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:32.098 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:46:32.098 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:46:32.098 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:46:32.099 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:46:32.099 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:46:32.099 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:46:32.100 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:46:32.100 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:46:32.100 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:46:32.100 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:46:32.101 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:46:32.280 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:32.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:46:32.298 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113769.asok 2023-08-25T09:46:32.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:32.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113769.asok status 2023-08-25T09:46:32.540 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:32.540 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:46:32.540 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:46:32.540 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:46:32.541 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "113769", 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 2, 2023-08-25T09:46:32.542 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 7926, 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 7926 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.543 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:32.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:32.544 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:32.544 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:32.544 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:32.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.545 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:46:32.545 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:46:32.545 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:46:32.545 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3843564457 2023-08-25T09:46:32.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:32.545 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.7926 192.168.0.1:0/3843564457", 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3843564457", 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 3, 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 57, 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 136, 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 136, 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:46:32.546 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:46:32.547 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:32.554 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:46:32.555 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:46:32.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:46:32.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:46:32.781 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:46:32.781 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:46:32.782 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:32.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.063+0000 7f2e6afe1640 1 Processor -- start 2023-08-25T09:46:33.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.063+0000 7f2e6afe1640 1 -- start start 2023-08-25T09:46:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e6afe1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e6415cb40 0x7f2e640ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e6afe1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e640ac870 0x7f2e640ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e6afe1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e6406f050 con 0x7f2e640b3880 2023-08-25T09:46:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e6afe1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e6406f330 con 0x7f2e6415cb40 2023-08-25T09:46:33.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e6afe1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6406f610 con 0x7f2e640ac870 2023-08-25T09:46:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e68d7d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e6415cb40 0x7f2e640ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e68d7d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2e6415cb40 0x7f2e640ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47538/0 (socket says 172.21.15.70:47538) 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e68d7d640 1 -- 172.21.15.70:0/4119032420 learned_addr learned my addr 172.21.15.70:0/4119032420 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e6957e640 1 --1- 172.21.15.70:0/4119032420 >> v1:172.21.15.70:6789/0 conn(0x7f2e640b3880 0x7f2e64160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45444/0 (socket says 172.21.15.70:45444) 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.065+0000 7f2e63fff640 1 --2- 172.21.15.70:0/4119032420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e640ac870 0x7f2e640ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.066+0000 7f2e68d7d640 1 -- 172.21.15.70:0/4119032420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e640ac870 msgr2=0x7f2e640ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.066+0000 7f2e68d7d640 1 --2- 172.21.15.70:0/4119032420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e640ac870 0x7f2e640ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.066+0000 7f2e68d7d640 1 -- 172.21.15.70:0/4119032420 >> v1:172.21.15.70:6789/0 conn(0x7f2e640b3880 legacy=0x7f2e64160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.066+0000 7f2e637fe640 1 -- 172.21.15.70:0/4119032420 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984814914 0 0) 0x7f2e6406f050 con 0x7f2e640b3880 2023-08-25T09:46:33.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.066+0000 7f2e68d7d640 1 -- 172.21.15.70:0/4119032420 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6406f8f0 con 0x7f2e6415cb40 2023-08-25T09:46:33.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.067+0000 7f2e68d7d640 1 --2- 172.21.15.70:0/4119032420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6415cb40 0x7f2e640ce320 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d07713ed31aaab3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.067+0000 7f2e637fe640 1 -- 172.21.15.70:0/4119032420 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58030840 con 0x7f2e6415cb40 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.067+0000 7f2e637fe640 1 -- 172.21.15.70:0/4119032420 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e580309a0 con 0x7f2e6415cb40 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.067+0000 7f2e637fe640 1 -- 172.21.15.70:0/4119032420 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e58030d30 con 0x7f2e6415cb40 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 -- 172.21.15.70:0/4119032420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6415cb40 msgr2=0x7f2e640ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/4119032420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6415cb40 0x7f2e640ce320 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009870 tx=0x7f2e5802b980 comp rx=0 tx=0).stop 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 -- 172.21.15.70:0/4119032420 shutdown_connections 2023-08-25T09:46:33.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/4119032420 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e640ac870 0x7f2e640ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/4119032420 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e6415cb40 0x7f2e640ce320 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 -- 172.21.15.70:0/4119032420 >> 172.21.15.70:0/4119032420 conn(0x7f2e64069850 msgr2=0x7f2e64069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:33.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 -- 172.21.15.70:0/4119032420 shutdown_connections 2023-08-25T09:46:33.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.068+0000 7f2e6afe1640 1 -- 172.21.15.70:0/4119032420 wait complete. 2023-08-25T09:46:33.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.070+0000 7f2e6afe1640 1 Processor -- start 2023-08-25T09:46:33.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.070+0000 7f2e6afe1640 1 -- start start 2023-08-25T09:46:33.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6afe1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e640ac870 0x7f2e64158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6afe1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 0x7f2e640cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6afe1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e6415cb40 0x7f2e6415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6afe1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64158c60 con 0x7f2e640b3880 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6afe1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64158f40 con 0x7f2e640ac870 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6afe1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64159220 con 0x7f2e6415cb40 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 0x7f2e640cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 0x7f2e640cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47548/0 (socket says 172.21.15.70:47548) 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e63fff640 1 -- 172.21.15.70:0/1672081363 learned_addr learned my addr 172.21.15.70:0/1672081363 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e68d7d640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e640ac870 0x7f2e64158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e6957e640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e6415cb40 0x7f2e6415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e63fff640 1 -- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e6415cb40 msgr2=0x7f2e6415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.071+0000 7f2e63fff640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e6415cb40 0x7f2e6415d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e63fff640 1 -- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e640ac870 msgr2=0x7f2e64158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e63fff640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e640ac870 0x7f2e64158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e63fff640 1 -- 172.21.15.70:0/1672081363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580094d0 con 0x7f2e640b3880 2023-08-25T09:46:33.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e6957e640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e6415cb40 0x7f2e6415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:33.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e63fff640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 0x7f2e640cd260 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000d5f0 tx=0x7f2e5000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500125f0 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50012750 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e64159500 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50012b00 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.072+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6415fda0 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.073+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6404e420 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.074+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2e50012c60 con 0x7f2e640b3880 2023-08-25T09:46:33.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.075+0000 7f2e617fa640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e4806a050 0x7f2e4806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.075+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e50088ad0 con 0x7f2e640b3880 2023-08-25T09:46:33.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.076+0000 7f2e68d7d640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e4806a050 0x7f2e4806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.076+0000 7f2e68d7d640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e4806a050 0x7f2e4806c510 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2e58036000 tx=0x7f2e58008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.080+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2e50054120 con 0x7f2e640b3880 2023-08-25T09:46:33.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.270+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e640aed70 con 0x7f2e640b3880 2023-08-25T09:46:33.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.273+0000 7f2e617fa640 1 -- 172.21.15.70:0/1672081363 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f2e50057740 con 0x7f2e640b3880 2023-08-25T09:46:33.275 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:33.275 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":136,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:29.557323+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.275+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e4806a050 msgr2=0x7f2e4806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.275+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e4806a050 0x7f2e4806c510 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2e58036000 tx=0x7f2e58008030 comp rx=0 tx=0).stop 2023-08-25T09:46:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.275+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 msgr2=0x7f2e640cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 0x7f2e640cd260 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000d5f0 tx=0x7f2e5000dab0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 shutdown_connections 2023-08-25T09:46:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2e4806a050 0x7f2e4806c510 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2e6415cb40 0x7f2e6415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2e640b3880 0x7f2e640cd260 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 --2- 172.21.15.70:0/1672081363 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2e640ac870 0x7f2e64158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 >> 172.21.15.70:0/1672081363 conn(0x7f2e64069850 msgr2=0x7f2e64069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 shutdown_connections 2023-08-25T09:46:33.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.276+0000 7f2e6afe1640 1 -- 172.21.15.70:0/1672081363 wait complete. 2023-08-25T09:46:33.296 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:46:11.556010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:46:12.318797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:33.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:33.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.512+0000 7f1cbb59e640 1 Processor -- start 2023-08-25T09:46:33.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.512+0000 7f1cbb59e640 1 -- start start 2023-08-25T09:46:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbb59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1cbc0cb880 0x7f1cbc15da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbb59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc160660 0x7f1cbc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbb59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1cbc160ae0 0x7f1cbc06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbb59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc161600 con 0x7f1cbc160ae0 2023-08-25T09:46:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbb59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc15d070 con 0x7f1cbc0cb880 2023-08-25T09:46:33.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbb59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15dfc0 con 0x7f1cbc160660 2023-08-25T09:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1cbc160ae0 0x7f1cbc06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1cbc160ae0 0x7f1cbc06ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47554/0 (socket says 172.21.15.70:47554) 2023-08-25T09:46:33.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cbad9d640 1 -- 172.21.15.70:0/2587577754 learned_addr learned my addr 172.21.15.70:0/2587577754 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cb9d9b640 1 --2- 172.21.15.70:0/2587577754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc160660 0x7f1cbc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.514+0000 7f1cba59c640 1 --2- 172.21.15.70:0/2587577754 >> v2:172.21.15.155:3300/0 conn(0x7f1cbc0cb880 0x7f1cbc15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.515+0000 7f1cbad9d640 1 -- 172.21.15.70:0/2587577754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc160660 msgr2=0x7f1cbc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.515+0000 7f1cbad9d640 1 --2- 172.21.15.70:0/2587577754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc160660 0x7f1cbc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.515+0000 7f1cbad9d640 1 -- 172.21.15.70:0/2587577754 >> v2:172.21.15.155:3300/0 conn(0x7f1cbc0cb880 msgr2=0x7f1cbc15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.515+0000 7f1cbad9d640 1 --2- 172.21.15.70:0/2587577754 >> v2:172.21.15.155:3300/0 conn(0x7f1cbc0cb880 0x7f1cbc15da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.515+0000 7f1cbad9d640 1 -- 172.21.15.70:0/2587577754 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc161040 con 0x7f1cbc160ae0 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.516+0000 7f1cbad9d640 1 --2- 172.21.15.70:0/2587577754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc06ec60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc070950 tx=0x7f1cb402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c715c684b0e48c80 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.516+0000 7f1cb959a640 1 -- 172.21.15.70:0/2587577754 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb402ca70 con 0x7f1cbc160ae0 2023-08-25T09:46:33.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.516+0000 7f1cb959a640 1 -- 172.21.15.70:0/2587577754 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb402cbd0 con 0x7f1cbc160ae0 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.516+0000 7f1cb959a640 1 -- 172.21.15.70:0/2587577754 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40343f0 con 0x7f1cbc160ae0 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 -- 172.21.15.70:0/2587577754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 msgr2=0x7f1cbc06ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/2587577754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc06ec60 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc070950 tx=0x7f1cb402bed0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 -- 172.21.15.70:0/2587577754 shutdown_connections 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/2587577754 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc06ec60 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/2587577754 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc160660 0x7f1cbc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/2587577754 >> v2:172.21.15.155:3300/0 conn(0x7f1cbc0cb880 0x7f1cbc15da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 -- 172.21.15.70:0/2587577754 >> 172.21.15.70:0/2587577754 conn(0x7f1cbc0d06b0 msgr2=0x7f1cbc0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:33.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 -- 172.21.15.70:0/2587577754 shutdown_connections 2023-08-25T09:46:33.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.517+0000 7f1cbb59e640 1 -- 172.21.15.70:0/2587577754 wait complete. 2023-08-25T09:46:33.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.519+0000 7f1cbb59e640 1 Processor -- start 2023-08-25T09:46:33.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.519+0000 7f1cbb59e640 1 -- start start 2023-08-25T09:46:33.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbb59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc0cb880 0x7f1cbc1f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbb59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cbc160660 0x7f1cbc1556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbb59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbb59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06a7d0 con 0x7f1cbc160ae0 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbb59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06a4f0 con 0x7f1cbc160660 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbb59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06ab20 con 0x7f1cbc0cb880 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47562/0 (socket says 172.21.15.70:47562) 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 -- 172.21.15.70:0/390392340 learned_addr learned my addr 172.21.15.70:0/390392340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cb9d9b640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cbc160660 0x7f1cbc1556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cba59c640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc0cb880 0x7f1cbc1f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 -- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc0cb880 msgr2=0x7f1cbc1f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc0cb880 0x7f1cbc1f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 -- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cbc160660 msgr2=0x7f1cbc1556b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cbc160660 0x7f1cbc1556b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cbad9d640 1 -- 172.21.15.70:0/390392340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009b30 con 0x7f1cbc160ae0 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.520+0000 7f1cba59c640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc0cb880 0x7f1cbc1f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:33.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.521+0000 7f1cbad9d640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc155bf0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4036040 tx=0x7f1cb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.521+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4040670 con 0x7f1cbc160ae0 2023-08-25T09:46:33.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.521+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb40407d0 con 0x7f1cbc160ae0 2023-08-25T09:46:33.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.521+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4040b80 con 0x7f1cbc160ae0 2023-08-25T09:46:33.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.521+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc06ae00 con 0x7f1cbc160ae0 2023-08-25T09:46:33.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.521+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc1f2210 con 0x7f1cbc160ae0 2023-08-25T09:46:33.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.522+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc06a980 con 0x7f1cbc160ae0 2023-08-25T09:46:33.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.523+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1cb4040ce0 con 0x7f1cbc160ae0 2023-08-25T09:46:33.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.524+0000 7f1caaffd640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c9c06a0a0 0x7f1c9c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.524+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1cb40ab620 con 0x7f1cbc160ae0 2023-08-25T09:46:33.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.525+0000 7f1cba59c640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c9c06a0a0 0x7f1c9c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.525+0000 7f1cba59c640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c9c06a0a0 0x7f1c9c06c560 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006fd0 tx=0x7f1cac008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.529+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1cbc06a980 con 0x7f1cbc160ae0 2023-08-25T09:46:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.719+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cbc06a980 con 0x7f1cbc160ae0 2023-08-25T09:46:33.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.720+0000 7f1caaffd640 1 -- 172.21.15.70:0/390392340 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cbc06a980 con 0x7f1cbc160ae0 2023-08-25T09:46:33.722 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:33.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c9c06a0a0 msgr2=0x7f1c9c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c9c06a0a0 0x7f1c9c06c560 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006fd0 tx=0x7f1cac008040 comp rx=0 tx=0).stop 2023-08-25T09:46:33.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 msgr2=0x7f1cbc155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc155bf0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4036040 tx=0x7f1cb4004a40 comp rx=0 tx=0).stop 2023-08-25T09:46:33.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 shutdown_connections 2023-08-25T09:46:33.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1c9c06a0a0 0x7f1c9c06c560 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1cbc160ae0 0x7f1cbc155bf0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1cbc160660 0x7f1cbc1556b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 --2- 172.21.15.70:0/390392340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1cbc0cb880 0x7f1cbc1f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 >> 172.21.15.70:0/390392340 conn(0x7f1cbc0d06b0 msgr2=0x7f1cbc15b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:33.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 shutdown_connections 2023-08-25T09:46:33.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.723+0000 7f1cbb59e640 1 -- 172.21.15.70:0/390392340 wait complete. 2023-08-25T09:46:33.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:33.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.961+0000 7f6384a03640 1 Processor -- start 2023-08-25T09:46:33.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.961+0000 7f6384a03640 1 -- start start 2023-08-25T09:46:33.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.962+0000 7f6384a03640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0320 0x7f63800c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.962+0000 7f6384a03640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f638015a920 0x7f63800b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.962+0000 7f6384a03640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63800adbb0 con 0x7f63800b0700 2023-08-25T09:46:33.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f6384a03640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63800b2220 con 0x7f638015a920 2023-08-25T09:46:33.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f6384a03640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63800c90a0 con 0x7f63800b0320 2023-08-25T09:46:33.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f637f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63800b0700 0x7f63800ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45458/0 (socket says 172.21.15.70:45458) 2023-08-25T09:46:33.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f637f59e640 1 -- 172.21.15.70:0/2715103505 learned_addr learned my addr 172.21.15.70:0/2715103505 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:33.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f637e59c640 1 --2- 172.21.15.70:0/2715103505 >> v2:172.21.15.70:3300/0 conn(0x7f638015a920 0x7f63800b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f637ed9d640 1 --2- 172.21.15.70:0/2715103505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0320 0x7f63800c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f637dd9b640 1 -- 172.21.15.70:0/2715103505 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469742170 0 0) 0x7f63800adbb0 con 0x7f63800b0700 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.963+0000 7f637dd9b640 1 -- 172.21.15.70:0/2715103505 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6368003520 con 0x7f63800b0700 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.964+0000 7f637e59c640 1 -- 172.21.15.70:0/2715103505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0320 msgr2=0x7f63800c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.964+0000 7f637e59c640 1 --2- 172.21.15.70:0/2715103505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0320 0x7f63800c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.964+0000 7f637e59c640 1 -- 172.21.15.70:0/2715103505 >> v1:172.21.15.70:6789/0 conn(0x7f63800b0700 legacy=0x7f63800ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.964+0000 7f637e59c640 1 -- 172.21.15.70:0/2715103505 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800c9380 con 0x7f638015a920 2023-08-25T09:46:33.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.965+0000 7f637e59c640 1 --2- 172.21.15.70:0/2715103505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f638015a920 0x7f63800b1ce0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f63740097a0 tx=0x7f637402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474293feaafb52c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.965+0000 7f637dd9b640 1 -- 172.21.15.70:0/2715103505 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637402d040 con 0x7f638015a920 2023-08-25T09:46:33.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.965+0000 7f637dd9b640 1 -- 172.21.15.70:0/2715103505 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6374031810 con 0x7f638015a920 2023-08-25T09:46:33.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.965+0000 7f637dd9b640 1 -- 172.21.15.70:0/2715103505 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374031ba0 con 0x7f638015a920 2023-08-25T09:46:33.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.965+0000 7f6384a03640 1 -- 172.21.15.70:0/2715103505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f638015a920 msgr2=0x7f63800b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.965+0000 7f6384a03640 1 --2- 172.21.15.70:0/2715103505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f638015a920 0x7f63800b1ce0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f63740097a0 tx=0x7f637402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.966+0000 7f6384a03640 1 -- 172.21.15.70:0/2715103505 shutdown_connections 2023-08-25T09:46:33.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.966+0000 7f6384a03640 1 --2- 172.21.15.70:0/2715103505 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f638015a920 0x7f63800b1ce0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.966+0000 7f6384a03640 1 --2- 172.21.15.70:0/2715103505 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0320 0x7f63800c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.966+0000 7f6384a03640 1 -- 172.21.15.70:0/2715103505 >> 172.21.15.70:0/2715103505 conn(0x7f63800ce830 msgr2=0x7f63800cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:33.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.966+0000 7f6384a03640 1 -- 172.21.15.70:0/2715103505 shutdown_connections 2023-08-25T09:46:33.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.966+0000 7f6384a03640 1 -- 172.21.15.70:0/2715103505 wait complete. 2023-08-25T09:46:33.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.967+0000 7f6384a03640 1 Processor -- start 2023-08-25T09:46:33.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.968+0000 7f6384a03640 1 -- start start 2023-08-25T09:46:33.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f6384a03640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 0x7f63801f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f6384a03640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0700 0x7f63800c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f6384a03640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638015a920 0x7f6380157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f6384a03640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800c78e0 con 0x7f63800b0320 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f6384a03640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800c7450 con 0x7f638015a920 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f6384a03640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63801f30d0 con 0x7f63800b0700 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 0x7f63801f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 0x7f63801f2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47572/0 (socket says 172.21.15.70:47572) 2023-08-25T09:46:33.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 -- 172.21.15.70:0/857171878 learned_addr learned my addr 172.21.15.70:0/857171878 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637e59c640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0700 0x7f63800c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 -- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0700 msgr2=0x7f63800c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0700 0x7f63800c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 -- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638015a920 msgr2=0x7f6380157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637f59e640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638015a920 0x7f6380157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638015a920 0x7f6380157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:33.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.969+0000 7f637ed9d640 1 -- 172.21.15.70:0/857171878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374009400 con 0x7f63800b0320 2023-08-25T09:46:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.970+0000 7f637ed9d640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 0x7f63801f2b90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f637000d800 tx=0x7f637000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.970+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000b6c0 con 0x7f63800b0320 2023-08-25T09:46:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.970+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637000b820 con 0x7f63800b0320 2023-08-25T09:46:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.970+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000bbd0 con 0x7f63800b0320 2023-08-25T09:46:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.970+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63800c3f80 con 0x7f63800b0320 2023-08-25T09:46:33.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.970+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63800c6b20 con 0x7f63800b0320 2023-08-25T09:46:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.971+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63800b2040 con 0x7f63800b0320 2023-08-25T09:46:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.972+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f637000bd30 con 0x7f63800b0320 2023-08-25T09:46:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.973+0000 7f63677fe640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f636c06a050 0x7f636c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:33.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.973+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6370088b00 con 0x7f63800b0320 2023-08-25T09:46:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.974+0000 7f637e59c640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f636c06a050 0x7f636c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:33.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.974+0000 7f637e59c640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f636c06a050 0x7f636c06c510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6374036000 tx=0x7f637403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:33.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:33.977+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f637008ae60 con 0x7f63800b0320 2023-08-25T09:46:34.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.167+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6380061460 con 0x7f63800b0320 2023-08-25T09:46:34.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.168+0000 7f63677fe640 1 -- 172.21.15.70:0/857171878 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6370054150 con 0x7f63800b0320 2023-08-25T09:46:34.169 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f636c06a050 msgr2=0x7f636c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f636c06a050 0x7f636c06c510 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6374036000 tx=0x7f637403c040 comp rx=0 tx=0).stop 2023-08-25T09:46:34.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 msgr2=0x7f63801f2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 0x7f63801f2b90 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f637000d800 tx=0x7f637000dcc0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 shutdown_connections 2023-08-25T09:46:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f636c06a050 0x7f636c06c510 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f638015a920 0x7f6380157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63800b0700 0x7f63800c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 --2- 172.21.15.70:0/857171878 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63800b0320 0x7f63801f2b90 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 >> 172.21.15.70:0/857171878 conn(0x7f63800ce830 msgr2=0x7f63800aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 shutdown_connections 2023-08-25T09:46:34.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.170+0000 7f6384a03640 1 -- 172.21.15.70:0/857171878 wait complete. 2023-08-25T09:46:34.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:46:34.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.401+0000 7fbfc8712640 1 Processor -- start 2023-08-25T09:46:34.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.402+0000 7fbfc8712640 1 -- start start 2023-08-25T09:46:34.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.403+0000 7fbfc8712640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfc00ce8e0 0x7fbfc00cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.403+0000 7fbfc8712640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbfc00cdc80 0x7fbfc00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.403+0000 7fbfc8712640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00cf200 0x7fbfc006d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.403+0000 7fbfc8712640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00cba70 con 0x7fbfc00ce8e0 2023-08-25T09:46:34.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.403+0000 7fbfc8712640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc00cbd50 con 0x7fbfc00cdc80 2023-08-25T09:46:34.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.403+0000 7fbfc8712640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00cc030 con 0x7fbfc00cf200 2023-08-25T09:46:34.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfc00ce8e0 0x7fbfc00cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbfc00ce8e0 0x7fbfc00cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47584/0 (socket says 172.21.15.70:47584) 2023-08-25T09:46:34.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 -- 172.21.15.70:0/4147100336 learned_addr learned my addr 172.21.15.70:0/4147100336 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc6caf640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00cf200 0x7fbfc006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc5cad640 1 --2- 172.21.15.70:0/4147100336 >> v2:172.21.15.155:3300/0 conn(0x7fbfc00cdc80 0x7fbfc00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 -- 172.21.15.70:0/4147100336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00cf200 msgr2=0x7fbfc006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00cf200 0x7fbfc006d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 -- 172.21.15.70:0/4147100336 >> v2:172.21.15.155:3300/0 conn(0x7fbfc00cdc80 msgr2=0x7fbfc00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 --2- 172.21.15.70:0/4147100336 >> v2:172.21.15.155:3300/0 conn(0x7fbfc00cdc80 0x7fbfc00cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc64ae640 1 -- 172.21.15.70:0/4147100336 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00cc310 con 0x7fbfc00ce8e0 2023-08-25T09:46:34.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc6caf640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00cf200 0x7fbfc006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:34.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.404+0000 7fbfc5cad640 1 --2- 172.21.15.70:0/4147100336 >> v2:172.21.15.155:3300/0 conn(0x7fbfc00cdc80 0x7fbfc00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:34.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.405+0000 7fbfc64ae640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00ce8e0 0x7fbfc00cb470 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00c40a0 tx=0x7fbfbc02b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abd419d53494f94 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:34.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.406+0000 7fbfc54ac640 1 -- 172.21.15.70:0/4147100336 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004030 con 0x7fbfc00ce8e0 2023-08-25T09:46:34.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.406+0000 7fbfc54ac640 1 -- 172.21.15.70:0/4147100336 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc004190 con 0x7fbfc00ce8e0 2023-08-25T09:46:34.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.406+0000 7fbfc54ac640 1 -- 172.21.15.70:0/4147100336 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc031640 con 0x7fbfc00ce8e0 2023-08-25T09:46:34.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.406+0000 7fbfc8712640 1 -- 172.21.15.70:0/4147100336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00ce8e0 msgr2=0x7fbfc00cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.406+0000 7fbfc8712640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00ce8e0 0x7fbfc00cb470 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00c40a0 tx=0x7fbfbc02b8e0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 -- 172.21.15.70:0/4147100336 shutdown_connections 2023-08-25T09:46:34.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00cf200 0x7fbfc006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 --2- 172.21.15.70:0/4147100336 >> v2:172.21.15.155:3300/0 conn(0x7fbfc00cdc80 0x7fbfc00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 --2- 172.21.15.70:0/4147100336 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00ce8e0 0x7fbfc00cb470 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 -- 172.21.15.70:0/4147100336 >> 172.21.15.70:0/4147100336 conn(0x7fbfc00d0860 msgr2=0x7fbfc00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:34.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 -- 172.21.15.70:0/4147100336 shutdown_connections 2023-08-25T09:46:34.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.407+0000 7fbfc8712640 1 -- 172.21.15.70:0/4147100336 wait complete. 2023-08-25T09:46:34.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.409+0000 7fbfc8712640 1 Processor -- start 2023-08-25T09:46:34.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.409+0000 7fbfc8712640 1 -- start start 2023-08-25T09:46:34.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc8712640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc00cdc80 0x7fbfc01d65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc8712640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00ce8e0 0x7fbfc00ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc8712640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 0x7fbfc00c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc8712640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00ae410 con 0x7fbfc00cf200 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc8712640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00aede0 con 0x7fbfc00cdc80 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc8712640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00ad800 con 0x7fbfc00ce8e0 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc6caf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 0x7fbfc00c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc6caf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 0x7fbfc00c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47598/0 (socket says 172.21.15.70:47598) 2023-08-25T09:46:34.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc6caf640 1 -- 172.21.15.70:0/481746570 learned_addr learned my addr 172.21.15.70:0/481746570 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:34.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.410+0000 7fbfc5cad640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00ce8e0 0x7fbfc00ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc6caf640 1 -- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00ce8e0 msgr2=0x7fbfc00ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc6caf640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00ce8e0 0x7fbfc00ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc6caf640 1 -- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc00cdc80 msgr2=0x7fbfc01d65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc64ae640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc00cdc80 0x7fbfc01d65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc6caf640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc00cdc80 0x7fbfc01d65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc6caf640 1 -- 172.21.15.70:0/481746570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc009460 con 0x7fbfc00cf200 2023-08-25T09:46:34.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc6caf640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 0x7fbfc00c5320 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800cef0 tx=0x7fbfb800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:34.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800d840 con 0x7fbfc00cf200 2023-08-25T09:46:34.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb800d9c0 con 0x7fbfc00cf200 2023-08-25T09:46:34.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800dd70 con 0x7fbfc00cf200 2023-08-25T09:46:34.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc00adae0 con 0x7fbfc00cf200 2023-08-25T09:46:34.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.411+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc00ae0a0 con 0x7fbfc00cf200 2023-08-25T09:46:34.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.413+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc00ae5c0 con 0x7fbfc00cf200 2023-08-25T09:46:34.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.413+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbfb801e3b0 con 0x7fbfc00cf200 2023-08-25T09:46:34.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.414+0000 7fbfb6ffd640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbfa406a080 0x7fbfa406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.415+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbfb8088ff0 con 0x7fbfc00cf200 2023-08-25T09:46:34.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.415+0000 7fbfc64ae640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbfa406a080 0x7fbfa406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.415+0000 7fbfc64ae640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbfa406a080 0x7fbfa406c540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc035000 tx=0x7fbfbc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:34.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.419+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbfc00ae5c0 con 0x7fbfc00cf200 2023-08-25T09:46:34.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.609+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfc0066fb0 con 0x7fbfc00cf200 2023-08-25T09:46:34.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.610+0000 7fbfb6ffd640 1 -- 172.21.15.70:0/481746570 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfb801a0a0 con 0x7fbfc00cf200 2023-08-25T09:46:34.685 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:46:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbfa406a080 msgr2=0x7fbfa406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbfa406a080 0x7fbfa406c540 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc035000 tx=0x7fbfbc008030 comp rx=0 tx=0).stop 2023-08-25T09:46:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 msgr2=0x7fbfc00c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 0x7fbfc00c5320 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800cef0 tx=0x7fbfb800a710 comp rx=0 tx=0).stop 2023-08-25T09:46:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 shutdown_connections 2023-08-25T09:46:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbfa406a080 0x7fbfa406c540 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbfc00cf200 0x7fbfc00c5320 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbfc00ce8e0 0x7fbfc00ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 --2- 172.21.15.70:0/481746570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbfc00cdc80 0x7fbfc01d65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 >> 172.21.15.70:0/481746570 conn(0x7fbfc00d0860 msgr2=0x7fbfc00b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 shutdown_connections 2023-08-25T09:46:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.613+0000 7fbfc8712640 1 -- 172.21.15.70:0/481746570 wait complete. 2023-08-25T09:46:34.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:46:34.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.914+0000 7ff2a3fff640 1 Processor -- start 2023-08-25T09:46:34.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.914+0000 7ff2a3fff640 1 -- start start 2023-08-25T09:46:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.915+0000 7ff2a3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff2a40b2850 0x7ff2a40cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.915+0000 7ff2a3fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40cd800 con 0x7ff2a40b2470 2023-08-25T09:46:34.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.915+0000 7ff2a3fff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a415a110 con 0x7ff2a40c8d20 2023-08-25T09:46:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.915+0000 7ff2a3fff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a415ad20 con 0x7ff2a40b2850 2023-08-25T09:46:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a2ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff2a40b2470 0x7ff2a4159930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45472/0 (socket says 172.21.15.70:45472) 2023-08-25T09:46:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a2ffd640 1 -- 172.21.15.70:0/3229121063 learned_addr learned my addr 172.21.15.70:0/3229121063 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:34.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a27fc640 1 --2- 172.21.15.70:0/3229121063 >> v2:172.21.15.70:3300/0 conn(0x7ff2a40b2850 0x7ff2a40cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a1ffb640 1 -- 172.21.15.70:0/3229121063 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076252223 0 0) 0x7ff2a40cd800 con 0x7ff2a40b2470 2023-08-25T09:46:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a1ffb640 1 -- 172.21.15.70:0/3229121063 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff28c003520 con 0x7ff2a40b2470 2023-08-25T09:46:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a27fc640 1 -- 172.21.15.70:0/3229121063 >> v1:172.21.15.155:6789/0 conn(0x7ff2a40c8d20 legacy=0x7ff2a40c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:34.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a27fc640 1 -- 172.21.15.70:0/3229121063 >> v1:172.21.15.70:6789/0 conn(0x7ff2a40b2470 legacy=0x7ff2a4159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.916+0000 7ff2a27fc640 1 -- 172.21.15.70:0/3229121063 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a415a3f0 con 0x7ff2a40b2850 2023-08-25T09:46:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.917+0000 7ff2a27fc640 1 --2- 172.21.15.70:0/3229121063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a40cdb30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097a0 tx=0x7ff29802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e74c3318de33dd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a1ffb640 1 -- 172.21.15.70:0/3229121063 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298031830 con 0x7ff2a40b2850 2023-08-25T09:46:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a1ffb640 1 -- 172.21.15.70:0/3229121063 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298031990 con 0x7ff2a40b2850 2023-08-25T09:46:34.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a1ffb640 1 -- 172.21.15.70:0/3229121063 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298031d00 con 0x7ff2a40b2850 2023-08-25T09:46:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 -- 172.21.15.70:0/3229121063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 msgr2=0x7ff2a40cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/3229121063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a40cdb30 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097a0 tx=0x7ff29802b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 -- 172.21.15.70:0/3229121063 shutdown_connections 2023-08-25T09:46:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/3229121063 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a40cdb30 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 -- 172.21.15.70:0/3229121063 >> 172.21.15.70:0/3229121063 conn(0x7ff2a40ce830 msgr2=0x7ff2a40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 -- 172.21.15.70:0/3229121063 shutdown_connections 2023-08-25T09:46:34.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.918+0000 7ff2a3fff640 1 -- 172.21.15.70:0/3229121063 wait complete. 2023-08-25T09:46:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.920+0000 7ff2a3fff640 1 Processor -- start 2023-08-25T09:46:34.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.920+0000 7ff2a3fff640 1 -- start start 2023-08-25T09:46:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.921+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a40b2470 0x7ff2a4154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.921+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a415d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.921+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2a40c8d20 0x7ff2a415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.921+0000 7ff2a3fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a41547d0 con 0x7ff2a40b2850 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.921+0000 7ff2a3fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415c070 con 0x7ff2a40b2470 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.921+0000 7ff2a3fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415c350 con 0x7ff2a40c8d20 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a415d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47608/0 (socket says 172.21.15.70:47608) 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 -- 172.21.15.70:0/4038994456 learned_addr learned my addr 172.21.15.70:0/4038994456 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a2ffd640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a40b2470 0x7ff2a4154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 -- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2a40c8d20 msgr2=0x7ff2a415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2a40c8d20 0x7ff2a415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 -- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a40b2470 msgr2=0x7ff2a4154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a40b2470 0x7ff2a4154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 -- 172.21.15.70:0/4038994456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298009400 con 0x7ff2a40b2850 2023-08-25T09:46:34.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a2ffd640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a40b2470 0x7ff2a4154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.922+0000 7ff2a27fc640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a415d280 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff29802b910 tx=0x7ff2980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.923+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29803a630 con 0x7ff2a40b2850 2023-08-25T09:46:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.923+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29803a790 con 0x7ff2a40b2850 2023-08-25T09:46:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.923+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29803ab40 con 0x7ff2a40b2850 2023-08-25T09:46:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.923+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a415c630 con 0x7ff2a40b2850 2023-08-25T09:46:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.923+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a415cbf0 con 0x7ff2a40b2850 2023-08-25T09:46:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.924+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4061460 con 0x7ff2a40b2850 2023-08-25T09:46:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.925+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff29803aca0 con 0x7ff2a40b2850 2023-08-25T09:46:34.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.926+0000 7ff28b7fe640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff29006a050 0x7ff29006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.926+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2980a9ff0 con 0x7ff2a40b2850 2023-08-25T09:46:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.926+0000 7ff2a2ffd640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff29006a050 0x7ff29006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:34.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.927+0000 7ff2a2ffd640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff29006a050 0x7ff29006c510 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff294007fd0 tx=0x7ff294009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:34.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:34.931+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff2980dc990 con 0x7ff2a40b2850 2023-08-25T09:46:35.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.119+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2a4071d30 con 0x7ff2a40b2850 2023-08-25T09:46:35.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.120+0000 7ff28b7fe640 1 -- 172.21.15.70:0/4038994456 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2980350a0 con 0x7ff2a40b2850 2023-08-25T09:46:35.122 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:46:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff29006a050 msgr2=0x7ff29006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff29006a050 0x7ff29006c510 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff294007fd0 tx=0x7ff294009040 comp rx=0 tx=0).stop 2023-08-25T09:46:35.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 msgr2=0x7ff2a415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a415d280 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7ff29802b910 tx=0x7ff2980046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 shutdown_connections 2023-08-25T09:46:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff29006a050 0x7ff29006c510 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff2a40c8d20 0x7ff2a415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff2a40b2850 0x7ff2a415d280 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 --2- 172.21.15.70:0/4038994456 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff2a40b2470 0x7ff2a4154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.122+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 >> 172.21.15.70:0/4038994456 conn(0x7ff2a40ce830 msgr2=0x7ff2a40cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.123+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 shutdown_connections 2023-08-25T09:46:35.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.123+0000 7ff2a3fff640 1 -- 172.21.15.70:0/4038994456 wait complete. 2023-08-25T09:46:35.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:46:35.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.344+0000 7f83e98f3640 1 Processor -- start 2023-08-25T09:46:35.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.344+0000 7f83e98f3640 1 -- start start 2023-08-25T09:46:35.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83e98f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:35.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83e98f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f83e40ca780 0x7f83e40cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:35.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83e98f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e40b12b0 con 0x7f83e40b3cf0 2023-08-25T09:46:35.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83e98f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e406c0e0 con 0x7f83e40ca780 2023-08-25T09:46:35.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83e98f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e406ae20 con 0x7f83e415acd0 2023-08-25T09:46:35.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83dbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f83e40ca780 0x7f83e40cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:35.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83dbfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f83e40ca780 0x7f83e40cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47622/0 (socket says 172.21.15.70:47622) 2023-08-25T09:46:35.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.346+0000 7f83dbfff640 1 -- 172.21.15.70:0/1995173376 learned_addr learned my addr 172.21.15.70:0/1995173376 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:35.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.347+0000 7f83e2ffd640 1 --2- 172.21.15.70:0/1995173376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.347+0000 7f83dbfff640 1 -- 172.21.15.70:0/1995173376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 msgr2=0x7f83e415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.347+0000 7f83dbfff640 1 --2- 172.21.15.70:0/1995173376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.347+0000 7f83dbfff640 1 -- 172.21.15.70:0/1995173376 >> v1:172.21.15.155:6789/0 conn(0x7f83e40b3cf0 legacy=0x7f83e40b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.347+0000 7f83dbfff640 1 -- 172.21.15.70:0/1995173376 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e406b910 con 0x7f83e40ca780 2023-08-25T09:46:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.348+0000 7f83dbfff640 1 --2- 172.21.15.70:0/1995173376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e40cac00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009870 tx=0x7f83cc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c554fe038ae342 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:35.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.348+0000 7f83e27fc640 1 -- 172.21.15.70:0/1995173376 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0317c0 con 0x7f83e40ca780 2023-08-25T09:46:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.348+0000 7f83e27fc640 1 -- 172.21.15.70:0/1995173376 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc031920 con 0x7f83e40ca780 2023-08-25T09:46:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.348+0000 7f83e27fc640 1 -- 172.21.15.70:0/1995173376 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc031cb0 con 0x7f83e40ca780 2023-08-25T09:46:35.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 -- 172.21.15.70:0/1995173376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 msgr2=0x7f83e40cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 --2- 172.21.15.70:0/1995173376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e40cac00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009870 tx=0x7f83cc02b860 comp rx=0 tx=0).stop 2023-08-25T09:46:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 -- 172.21.15.70:0/1995173376 shutdown_connections 2023-08-25T09:46:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 --2- 172.21.15.70:0/1995173376 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e40cac00 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 --2- 172.21.15.70:0/1995173376 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e415b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 -- 172.21.15.70:0/1995173376 >> 172.21.15.70:0/1995173376 conn(0x7f83e40ce760 msgr2=0x7f83e40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:35.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 -- 172.21.15.70:0/1995173376 shutdown_connections 2023-08-25T09:46:35.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.349+0000 7f83e98f3640 1 -- 172.21.15.70:0/1995173376 wait complete. 2023-08-25T09:46:35.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.351+0000 7f83e98f3640 1 Processor -- start 2023-08-25T09:46:35.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.351+0000 7f83e98f3640 1 -- start start 2023-08-25T09:46:35.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83e98f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e40b3cf0 0x7f83e4152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:35.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83e98f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e41585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:35.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83e98f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e4158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:35.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83e98f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40ae5d0 con 0x7f83e40ca780 2023-08-25T09:46:35.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83e98f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40ad530 con 0x7f83e40b3cf0 2023-08-25T09:46:35.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83e98f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40ad810 con 0x7f83e415acd0 2023-08-25T09:46:35.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e41585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:35.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83dbfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e41585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47626/0 (socket says 172.21.15.70:47626) 2023-08-25T09:46:35.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.352+0000 7f83dbfff640 1 -- 172.21.15.70:0/2354117824 learned_addr learned my addr 172.21.15.70:0/2354117824 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:35.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83e2ffd640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e40b3cf0 0x7f83e4152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:35.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83e37fe640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e4158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83dbfff640 1 -- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 msgr2=0x7f83e4158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83dbfff640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e4158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83dbfff640 1 -- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e40b3cf0 msgr2=0x7f83e4152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83dbfff640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e40b3cf0 0x7f83e4152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83dbfff640 1 -- 172.21.15.70:0/2354117824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc0094d0 con 0x7f83e40ca780 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83e37fe640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83dbfff640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e41585f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009870 tx=0x7f83cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc032960 con 0x7f83e40ca780 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc032ac0 con 0x7f83e40ca780 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.353+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e40adaf0 con 0x7f83e40ca780 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.354+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc032e70 con 0x7f83e40ca780 2023-08-25T09:46:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.354+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e40ae0b0 con 0x7f83e40ca780 2023-08-25T09:46:35.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.355+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e40ae2a0 con 0x7f83e40ca780 2023-08-25T09:46:35.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.356+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f83cc04e450 con 0x7f83e40ca780 2023-08-25T09:46:35.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.356+0000 7f83e0ff9640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83c406a050 0x7f83c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:35.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.357+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83cc0aab20 con 0x7f83e40ca780 2023-08-25T09:46:35.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.357+0000 7f83e2ffd640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83c406a050 0x7f83c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:35.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.357+0000 7f83e2ffd640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83c406a050 0x7f83c406c510 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83e41598e0 tx=0x7f83d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:35.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.361+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f83cc0760e0 con 0x7f83e40ca780 2023-08-25T09:46:35.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.575+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83e404e420 con 0x7f83e40ca780 2023-08-25T09:46:35.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.576+0000 7f83e0ff9640 1 -- 172.21.15.70:0/2354117824 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v136) v1 ==== 80+0+0 (secure 0 0 0) 0x7f83cc079700 con 0x7f83e40ca780 2023-08-25T09:46:35.577 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:46:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83c406a050 msgr2=0x7f83c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83c406a050 0x7f83c406c510 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f83e41598e0 tx=0x7f83d4008040 comp rx=0 tx=0).stop 2023-08-25T09:46:35.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 msgr2=0x7f83e41585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:35.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e41585f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f83cc009870 tx=0x7f83cc0046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 shutdown_connections 2023-08-25T09:46:35.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f83c406a050 0x7f83c406c510 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f83e415acd0 0x7f83e4158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f83e40ca780 0x7f83e41585f0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 --2- 172.21.15.70:0/2354117824 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f83e40b3cf0 0x7f83e4152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 >> 172.21.15.70:0/2354117824 conn(0x7f83e40ce760 msgr2=0x7f83e40cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:35.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 shutdown_connections 2023-08-25T09:46:35.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:35.579+0000 7f83e98f3640 1 -- 172.21.15.70:0/2354117824 wait complete. 2023-08-25T09:46:35.600 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:46:35.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:35.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:46:35.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[113769]: fuse finished with error 0 and tester_r 0 2023-08-25T09:46:35.683 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:46:35.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:35.684 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:46:35.810 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:46:35.811 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:35.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:36.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.069+0000 7ff8bf0ec640 1 Processor -- start 2023-08-25T09:46:36.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.069+0000 7ff8bf0ec640 1 -- start start 2023-08-25T09:46:36.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.070+0000 7ff8bf0ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8b815e830 0x7ff8b806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.070+0000 7ff8bf0ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b806b2b0 0x7ff8b815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bf0ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b80b1b60 con 0x7ff8b815c6d0 2023-08-25T09:46:36.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bf0ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b806bd80 con 0x7ff8b815e830 2023-08-25T09:46:36.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bf0ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b806b7c0 con 0x7ff8b806b2b0 2023-08-25T09:46:36.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bd689640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff8b815c6d0 0x7ff8b80b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45474/0 (socket says 172.21.15.70:45474) 2023-08-25T09:46:36.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bce88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff8b815e830 0x7ff8b806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bd689640 1 -- 172.21.15.70:0/2083246427 learned_addr learned my addr 172.21.15.70:0/2083246427 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:36.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8b7fff640 1 --2- 172.21.15.70:0/2083246427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b806b2b0 0x7ff8b815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bce88640 1 -- 172.21.15.70:0/2083246427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b806b2b0 msgr2=0x7ff8b815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:36.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bce88640 1 --2- 172.21.15.70:0/2083246427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b806b2b0 0x7ff8b815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bce88640 1 -- 172.21.15.70:0/2083246427 >> v1:172.21.15.70:6789/0 conn(0x7ff8b815c6d0 legacy=0x7ff8b80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.071+0000 7ff8bce88640 1 -- 172.21.15.70:0/2083246427 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b806baa0 con 0x7ff8b815e830 2023-08-25T09:46:36.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.072+0000 7ff8bce88640 1 --2- 172.21.15.70:0/2083246427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b815e830 0x7ff8b806ad70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80097a0 tx=0x7ff8a802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa97ccd6fdeacaf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8b77fe640 1 -- 172.21.15.70:0/2083246427 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a802d040 con 0x7ff8b815e830 2023-08-25T09:46:36.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8b77fe640 1 -- 172.21.15.70:0/2083246427 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a8031810 con 0x7ff8b815e830 2023-08-25T09:46:36.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8b77fe640 1 -- 172.21.15.70:0/2083246427 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a8031ba0 con 0x7ff8b815e830 2023-08-25T09:46:36.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/2083246427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b815e830 msgr2=0x7ff8b806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/2083246427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b815e830 0x7ff8b806ad70 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7ff8a80097a0 tx=0x7ff8a802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/2083246427 shutdown_connections 2023-08-25T09:46:36.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/2083246427 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b806b2b0 0x7ff8b815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/2083246427 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b815e830 0x7ff8b806ad70 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/2083246427 >> 172.21.15.70:0/2083246427 conn(0x7ff8b8069850 msgr2=0x7ff8b8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:36.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/2083246427 shutdown_connections 2023-08-25T09:46:36.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.073+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/2083246427 wait complete. 2023-08-25T09:46:36.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.075+0000 7ff8bf0ec640 1 Processor -- start 2023-08-25T09:46:36.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.075+0000 7ff8bf0ec640 1 -- start start 2023-08-25T09:46:36.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bf0ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 0x7ff8b813f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bf0ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b815c6d0 0x7ff8b813c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bf0ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b815e830 0x7ff8b815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bf0ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b81507e0 con 0x7ff8b806b2b0 2023-08-25T09:46:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bf0ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b813fe00 con 0x7ff8b815c6d0 2023-08-25T09:46:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bf0ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b8152cc0 con 0x7ff8b815e830 2023-08-25T09:46:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bce88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 0x7ff8b813f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bce88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 0x7ff8b813f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47634/0 (socket says 172.21.15.70:47634) 2023-08-25T09:46:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.076+0000 7ff8bce88640 1 -- 172.21.15.70:0/753566911 learned_addr learned my addr 172.21.15.70:0/753566911 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bce88640 1 -- 172.21.15.70:0/753566911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b815e830 msgr2=0x7ff8b815cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bce88640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b815e830 0x7ff8b815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bce88640 1 -- 172.21.15.70:0/753566911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b815c6d0 msgr2=0x7ff8b813c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bce88640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b815c6d0 0x7ff8b813c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bce88640 1 -- 172.21.15.70:0/753566911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a8009400 con 0x7ff8b806b2b0 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bce88640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 0x7ff8b813f8c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8000c00 tx=0x7ff8a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a803ad60 con 0x7ff8b806b2b0 2023-08-25T09:46:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a80443f0 con 0x7ff8b806b2b0 2023-08-25T09:46:36.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a80447a0 con 0x7ff8b806b2b0 2023-08-25T09:46:36.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b8152fa0 con 0x7ff8b806b2b0 2023-08-25T09:46:36.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.077+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b8153560 con 0x7ff8b806b2b0 2023-08-25T09:46:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.078+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b804e420 con 0x7ff8b806b2b0 2023-08-25T09:46:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.079+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff8a804d450 con 0x7ff8b806b2b0 2023-08-25T09:46:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.080+0000 7ff8b57fa640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff89806a050 0x7ff89806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.080+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8a80aa940 con 0x7ff8b806b2b0 2023-08-25T09:46:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.080+0000 7ff8b7fff640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff89806a050 0x7ff89806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.081+0000 7ff8b7fff640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff89806a050 0x7ff89806c510 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff8b815ecc0 tx=0x7ff8a0007390 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.084+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff8a80760b0 con 0x7ff8b806b2b0 2023-08-25T09:46:36.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.275+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8b806c620 con 0x7ff8b806b2b0 2023-08-25T09:46:36.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.277+0000 7ff8b57fa640 1 -- 172.21.15.70:0/753566911 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18332 (secure 0 0 0) 0x7ff8a8035090 con 0x7ff8b806b2b0 2023-08-25T09:46:36.278 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:36.279 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":136,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:29.557323+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:36.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff89806a050 msgr2=0x7ff89806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff89806a050 0x7ff89806c510 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff8b815ecc0 tx=0x7ff8a0007390 comp rx=0 tx=0).stop 2023-08-25T09:46:36.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 msgr2=0x7ff8b813f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 0x7ff8b813f8c0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7ff8a8000c00 tx=0x7ff8a80046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 shutdown_connections 2023-08-25T09:46:36.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff89806a050 0x7ff89806c510 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff8b815e830 0x7ff8b815cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff8b815c6d0 0x7ff8b813c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.279+0000 7ff8bf0ec640 1 --2- 172.21.15.70:0/753566911 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff8b806b2b0 0x7ff8b813f8c0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.280+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 >> 172.21.15.70:0/753566911 conn(0x7ff8b8069850 msgr2=0x7ff8b8159020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:36.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.280+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 shutdown_connections 2023-08-25T09:46:36.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.280+0000 7ff8bf0ec640 1 -- 172.21.15.70:0/753566911 wait complete. 2023-08-25T09:46:36.299 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:46:11.556010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:46:12.318797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:36.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:36.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.515+0000 7fdcd8e63640 1 Processor -- start 2023-08-25T09:46:36.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.515+0000 7fdcd8e63640 1 -- start start 2023-08-25T09:46:36.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.516+0000 7fdcd8e63640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 0x7fdcd40cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.516+0000 7fdcd8e63640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd40ce170 con 0x7fdcd415cfe0 2023-08-25T09:46:36.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.516+0000 7fdcd8e63640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd40d1500 con 0x7fdcd40b8510 2023-08-25T09:46:36.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.516+0000 7fdcd8e63640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40d1310 con 0x7fdcd40cd850 2023-08-25T09:46:36.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.516+0000 7fdcd1d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdcd415cfe0 0x7fdcd40d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45476/0 (socket says 172.21.15.70:45476) 2023-08-25T09:46:36.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.516+0000 7fdcd1d9b640 1 -- 172.21.15.70:0/2334104935 learned_addr learned my addr 172.21.15.70:0/2334104935 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:36.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426742220 0 0) 0x7fdcd40ce170 con 0x7fdcd415cfe0 2023-08-25T09:46:36.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcc0003520 con 0x7fdcd415cfe0 2023-08-25T09:46:36.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd259c640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 0x7fdcd40cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2012948681 0 0) 0x7fdcc0003520 con 0x7fdcd415cfe0 2023-08-25T09:46:36.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 msgr2=0x7fdcd40cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:36.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 0x7fdcd40cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 >> v1:172.21.15.155:6789/0 conn(0x7fdcd40b8510 legacy=0x7fdcd40b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40d17e0 con 0x7fdcd415cfe0 2023-08-25T09:46:36.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fdcbc001670 con 0x7fdcd415cfe0 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 >> v1:172.21.15.70:6789/0 conn(0x7fdcd415cfe0 legacy=0x7fdcd40d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc00039a0 0x7fdcc0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 0x7fdcc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc000ba80 con 0x7fdcc00047c0 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc000bbf0 con 0x7fdcc00039a0 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.517+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc000bdb0 con 0x7fdcd40cd850 2023-08-25T09:46:36.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd1d9b640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd259c640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc00039a0 0x7fdcc0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd1d9b640 1 -- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 msgr2=0x7fdcc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd1d9b640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd40cd850 0x7fdcc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd1d9b640 1 -- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc00039a0 msgr2=0x7fdcc0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd1d9b640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcc00039a0 0x7fdcc0003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.518+0000 7fdcd1d9b640 1 -- 172.21.15.70:0/2334104935 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0003520 con 0x7fdcc00047c0 2023-08-25T09:46:36.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.519+0000 7fdcd1d9b640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcc0004c40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c210 tx=0x7fdcc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5426b8b2a3b61803 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.519+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc005ad0 con 0x7fdcc00047c0 2023-08-25T09:46:36.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.519+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc005c30 con 0x7fdcc00047c0 2023-08-25T09:46:36.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.519+0000 7fdcd159a640 1 -- 172.21.15.70:0/2334104935 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02c890 con 0x7fdcc00047c0 2023-08-25T09:46:36.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd8e63640 1 -- 172.21.15.70:0/2334104935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 msgr2=0x7fdcc0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/2334104935 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcc0004c40 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00c210 tx=0x7fdcc00057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd2d9d640 1 -- 172.21.15.70:0/2334104935 reap_dead start 2023-08-25T09:46:36.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd8e63640 1 -- 172.21.15.70:0/2334104935 shutdown_connections 2023-08-25T09:46:36.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd8e63640 1 -- 172.21.15.70:0/2334104935 >> 172.21.15.70:0/2334104935 conn(0x7fdcd4069850 msgr2=0x7fdcd4069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd8e63640 1 -- 172.21.15.70:0/2334104935 shutdown_connections 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.520+0000 7fdcd8e63640 1 -- 172.21.15.70:0/2334104935 wait complete. 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.522+0000 7fdcd8e63640 1 Processor -- start 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.522+0000 7fdcd8e63640 1 -- start start 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd8e63640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcd41f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd8e63640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd414daa0 0x7fdcd414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd8e63640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd414ba80 0x7fdcd414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd8e63640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd414e440 con 0x7fdcc00047c0 2023-08-25T09:46:36.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd8e63640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd414d870 con 0x7fdcd414ba80 2023-08-25T09:46:36.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd8e63640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd414c7d0 con 0x7fdcd414daa0 2023-08-25T09:46:36.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcd41f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcd41f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47648/0 (socket says 172.21.15.70:47648) 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.523+0000 7fdcd259c640 1 -- 172.21.15.70:0/4062263539 learned_addr learned my addr 172.21.15.70:0/4062263539 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd259c640 1 -- 172.21.15.70:0/4062263539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd414daa0 msgr2=0x7fdcd414df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd259c640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd414daa0 0x7fdcd414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd259c640 1 -- 172.21.15.70:0/4062263539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd414ba80 msgr2=0x7fdcd414bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd259c640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd414ba80 0x7fdcd414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd259c640 1 -- 172.21.15.70:0/4062263539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd414cab0 con 0x7fdcc00047c0 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd259c640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcd41f1df0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007ee0 tx=0x7fdcc400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40145f0 con 0x7fdcc00047c0 2023-08-25T09:46:36.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4014750 con 0x7fdcc00047c0 2023-08-25T09:46:36.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4014ae0 con 0x7fdcc00047c0 2023-08-25T09:46:36.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd414cd90 con 0x7fdcc00047c0 2023-08-25T09:46:36.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.524+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd414d350 con 0x7fdcc00047c0 2023-08-25T09:46:36.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.526+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd4066fb0 con 0x7fdcc00047c0 2023-08-25T09:46:36.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.526+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdcc4014c40 con 0x7fdcc00047c0 2023-08-25T09:46:36.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.527+0000 7fdccaffd640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdcb406a050 0x7fdcb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.528+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdcc40892b0 con 0x7fdcc00047c0 2023-08-25T09:46:36.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.528+0000 7fdcd1d9b640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdcb406a050 0x7fdcb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.529+0000 7fdcd1d9b640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdcb406a050 0x7fdcb406c510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc034000 tx=0x7fdcbc007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.532+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdcc4054a30 con 0x7fdcc00047c0 2023-08-25T09:46:36.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.719+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcd4069f10 con 0x7fdcc00047c0 2023-08-25T09:46:36.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.720+0000 7fdccaffd640 1 -- 172.21.15.70:0/4062263539 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcc4058050 con 0x7fdcc00047c0 2023-08-25T09:46:36.722 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.722+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdcb406a050 msgr2=0x7fdcb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.722+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdcb406a050 0x7fdcb406c510 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc034000 tx=0x7fdcbc007000 comp rx=0 tx=0).stop 2023-08-25T09:46:36.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.722+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 msgr2=0x7fdcd41f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.722+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcd41f1df0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007ee0 tx=0x7fdcc400f520 comp rx=0 tx=0).stop 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 shutdown_connections 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdcb406a050 0x7fdcb406c510 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdcd414ba80 0x7fdcd414bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdcd414daa0 0x7fdcd414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 --2- 172.21.15.70:0/4062263539 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdcc00047c0 0x7fdcd41f1df0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 >> 172.21.15.70:0/4062263539 conn(0x7fdcd4069850 msgr2=0x7fdcd40d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 shutdown_connections 2023-08-25T09:46:36.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.723+0000 7fdcd8e63640 1 -- 172.21.15.70:0/4062263539 wait complete. 2023-08-25T09:46:36.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:36.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.965+0000 7fea428a7640 1 Processor -- start 2023-08-25T09:46:36.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.965+0000 7fea428a7640 1 -- start start 2023-08-25T09:46:36.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea428a7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea3c15cd70 0x7fea3c0aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea428a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fea3c072820 0x7fea3c072c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea428a7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c0b31a0 con 0x7fea3c0b2c30 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea428a7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c15d150 con 0x7fea3c15cd70 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea428a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c0731b0 con 0x7fea3c072820 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea3bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea3c15cd70 0x7fea3c0aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea3bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fea3c15cd70 0x7fea3c0aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47662/0 (socket says 172.21.15.70:47662) 2023-08-25T09:46:36.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.967+0000 7fea3bfff640 1 -- 172.21.15.70:0/454739403 learned_addr learned my addr 172.21.15.70:0/454739403 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:36.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.968+0000 7fea3bfff640 1 -- 172.21.15.70:0/454739403 >> v2:172.21.15.155:3300/0 conn(0x7fea3c072820 msgr2=0x7fea3c072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:36.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.968+0000 7fea3bfff640 1 --2- 172.21.15.70:0/454739403 >> v2:172.21.15.155:3300/0 conn(0x7fea3c072820 0x7fea3c072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.968+0000 7fea3bfff640 1 -- 172.21.15.70:0/454739403 >> v1:172.21.15.155:6789/0 conn(0x7fea3c0b2c30 legacy=0x7fea3c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:36.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.968+0000 7fea3bfff640 1 -- 172.21.15.70:0/454739403 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c06c320 con 0x7fea3c15cd70 2023-08-25T09:46:36.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.969+0000 7fea3bfff640 1 --2- 172.21.15.70:0/454739403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c15cd70 0x7fea3c0aca80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fea3c06b2e0 tx=0x7fea2c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfcf1f046ecc1da server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.969+0000 7fea3affd640 1 -- 172.21.15.70:0/454739403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0317e0 con 0x7fea3c15cd70 2023-08-25T09:46:36.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.969+0000 7fea3affd640 1 -- 172.21.15.70:0/454739403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c031940 con 0x7fea3c15cd70 2023-08-25T09:46:36.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.969+0000 7fea3affd640 1 -- 172.21.15.70:0/454739403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c031cd0 con 0x7fea3c15cd70 2023-08-25T09:46:36.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 -- 172.21.15.70:0/454739403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c15cd70 msgr2=0x7fea3c0aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 --2- 172.21.15.70:0/454739403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c15cd70 0x7fea3c0aca80 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fea3c06b2e0 tx=0x7fea2c02b880 comp rx=0 tx=0).stop 2023-08-25T09:46:36.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 -- 172.21.15.70:0/454739403 shutdown_connections 2023-08-25T09:46:36.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 --2- 172.21.15.70:0/454739403 >> v2:172.21.15.155:3300/0 conn(0x7fea3c072820 0x7fea3c072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 --2- 172.21.15.70:0/454739403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c15cd70 0x7fea3c0aca80 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 -- 172.21.15.70:0/454739403 >> 172.21.15.70:0/454739403 conn(0x7fea3c0cfc70 msgr2=0x7fea3c0d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:36.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 -- 172.21.15.70:0/454739403 shutdown_connections 2023-08-25T09:46:36.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.970+0000 7fea428a7640 1 -- 172.21.15.70:0/454739403 wait complete. 2023-08-25T09:46:36.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.972+0000 7fea428a7640 1 Processor -- start 2023-08-25T09:46:36.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.972+0000 7fea428a7640 1 -- start start 2023-08-25T09:46:36.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea428a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c072820 0x7fea3c15e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea428a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 0x7fea3c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea428a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c15cd70 0x7fea3c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea428a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c160120 con 0x7fea3c0b2c30 2023-08-25T09:46:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea428a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c157310 con 0x7fea3c072820 2023-08-25T09:46:36.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea428a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c1575f0 con 0x7fea3c15cd70 2023-08-25T09:46:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.973+0000 7fea3b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 0x7fea3c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 0x7fea3c150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47666/0 (socket says 172.21.15.70:47666) 2023-08-25T09:46:36.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 -- 172.21.15.70:0/1256192319 learned_addr learned my addr 172.21.15.70:0/1256192319 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 -- 172.21.15.70:0/1256192319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c15cd70 msgr2=0x7fea3c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c15cd70 0x7fea3c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 -- 172.21.15.70:0/1256192319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c072820 msgr2=0x7fea3c15e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c072820 0x7fea3c15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 -- 172.21.15.70:0/1256192319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0094d0 con 0x7fea3c0b2c30 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea3b7fe640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 0x7fea3c150180 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fea3000c410 tx=0x7fea3000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300035f0 con 0x7fea3c0b2c30 2023-08-25T09:46:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30003750 con 0x7fea3c0b2c30 2023-08-25T09:46:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30003b40 con 0x7fea3c0b2c30 2023-08-25T09:46:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.974+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c15dd40 con 0x7fea3c0b2c30 2023-08-25T09:46:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.975+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c15e300 con 0x7fea3c0b2c30 2023-08-25T09:46:36.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.976+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c066fb0 con 0x7fea3c0b2c30 2023-08-25T09:46:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.977+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fea30003ca0 con 0x7fea3c0b2c30 2023-08-25T09:46:36.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.978+0000 7fea38ff9640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea2806a080 0x7fea2806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.978+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea300877e0 con 0x7fea3c0b2c30 2023-08-25T09:46:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.978+0000 7fea3bfff640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea2806a080 0x7fea2806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.979+0000 7fea3bfff640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea2806a080 0x7fea2806c540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fea2c036000 tx=0x7fea2c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:36.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:36.982+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fea30052e30 con 0x7fea3c0b2c30 2023-08-25T09:46:37.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.170+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea3c159460 con 0x7fea3c0b2c30 2023-08-25T09:46:37.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.171+0000 7fea38ff9640 1 -- 172.21.15.70:0/1256192319 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea30056450 con 0x7fea3c0b2c30 2023-08-25T09:46:37.173 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:37.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea2806a080 msgr2=0x7fea2806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea2806a080 0x7fea2806c540 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fea2c036000 tx=0x7fea2c030000 comp rx=0 tx=0).stop 2023-08-25T09:46:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 msgr2=0x7fea3c150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 0x7fea3c150180 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fea3000c410 tx=0x7fea3000c8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 shutdown_connections 2023-08-25T09:46:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fea2806a080 0x7fea2806c540 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fea3c15cd70 0x7fea3c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fea3c0b2c30 0x7fea3c150180 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 --2- 172.21.15.70:0/1256192319 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fea3c072820 0x7fea3c15e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 >> 172.21.15.70:0/1256192319 conn(0x7fea3c0cfc70 msgr2=0x7fea3c0b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 shutdown_connections 2023-08-25T09:46:37.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.174+0000 7fea428a7640 1 -- 172.21.15.70:0/1256192319 wait complete. 2023-08-25T09:46:37.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:46:37.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.406+0000 7fae15749640 1 Processor -- start 2023-08-25T09:46:37.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.406+0000 7fae15749640 1 -- start start 2023-08-25T09:46:37.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.408+0000 7fae15749640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fae10072c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.408+0000 7fae15749640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fae10143860 0x7fae10143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.408+0000 7fae15749640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1006ce20 con 0x7fae1006aa20 2023-08-25T09:46:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.408+0000 7fae15749640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae10073140 con 0x7fae10143860 2023-08-25T09:46:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.408+0000 7fae15749640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae10069980 con 0x7fae10072820 2023-08-25T09:46:37.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.409+0000 7fae0f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fae1006aa20 0x7fae1006c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45484/0 (socket says 172.21.15.70:45484) 2023-08-25T09:46:37.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.409+0000 7fae0f7fe640 1 -- 172.21.15.70:0/1330377381 learned_addr learned my addr 172.21.15.70:0/1330377381 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.409+0000 7fae0ffff640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fae10072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.409+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948847517 0 0) 0x7fae04001110 con 0x7fae1006aa20 2023-08-25T09:46:37.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.409+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf8003520 con 0x7fae1006aa20 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.409+0000 7fae14f48640 1 --2- 172.21.15.70:0/1330377381 >> v2:172.21.15.155:3300/0 conn(0x7fae10143860 0x7fae10143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862775765 0 0) 0x7fadf8003520 con 0x7fae1006aa20 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 msgr2=0x7fae10072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fae10072c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 >> v2:172.21.15.155:3300/0 conn(0x7fae10143860 msgr2=0x7fae10143cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 --2- 172.21.15.70:0/1330377381 >> v2:172.21.15.155:3300/0 conn(0x7fae10143860 0x7fae10143cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae10069c60 con 0x7fae1006aa20 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fae04001800 con 0x7fae1006aa20 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 >> v1:172.21.15.70:6789/0 conn(0x7fae1006aa20 legacy=0x7fae1006c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf8003970 0x7fadf8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf80047c0 0x7fadf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fadf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae10073140 con 0x7fadf8003970 2023-08-25T09:46:37.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae10069980 con 0x7fadf80047c0 2023-08-25T09:46:37.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf800ba30 con 0x7fae10072820 2023-08-25T09:46:37.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0ffff640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf8003970 0x7fadf8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae14f48640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fadf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0ffff640 1 -- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 msgr2=0x7fadf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0ffff640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fadf8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.410+0000 7fae0ffff640 1 -- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf80047c0 msgr2=0x7fadf8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.411+0000 7fae0ffff640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf80047c0 0x7fadf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.411+0000 7fae0ffff640 1 -- 172.21.15.70:0/1330377381 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf8003520 con 0x7fadf8003970 2023-08-25T09:46:37.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.411+0000 7fae14f48640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae10072820 0x7fadf8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae0ffff640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf8003970 0x7fadf8003d90 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fae00009a90 tx=0x7fadf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21c06f533ad1be3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0002c5a0 con 0x7fadf8003970 2023-08-25T09:46:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0002c700 con 0x7fadf8003970 2023-08-25T09:46:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae0effd640 1 -- 172.21.15.70:0/1330377381 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0002ca90 con 0x7fadf8003970 2023-08-25T09:46:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae15749640 1 -- 172.21.15.70:0/1330377381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf8003970 msgr2=0x7fadf8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae15749640 1 --2- 172.21.15.70:0/1330377381 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fadf8003970 0x7fadf8003d90 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fae00009a90 tx=0x7fadf80057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.412+0000 7fae14f48640 1 -- 172.21.15.70:0/1330377381 reap_dead start 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.413+0000 7fae15749640 1 -- 172.21.15.70:0/1330377381 shutdown_connections 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.413+0000 7fae15749640 1 -- 172.21.15.70:0/1330377381 >> 172.21.15.70:0/1330377381 conn(0x7fae100cc060 msgr2=0x7fae100cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.413+0000 7fae15749640 1 -- 172.21.15.70:0/1330377381 shutdown_connections 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.413+0000 7fae15749640 1 -- 172.21.15.70:0/1330377381 wait complete. 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.415+0000 7fae15749640 1 Processor -- start 2023-08-25T09:46:37.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.415+0000 7fae15749640 1 -- start start 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae15749640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf8003970 0x7fae101f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae15749640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 0x7fae1015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae15749640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100af320 0x7fae10070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae15749640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1013ac20 con 0x7fae100ae9d0 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae15749640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1015c9f0 con 0x7fadf8003970 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae15749640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae100b1a10 con 0x7fae100af320 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 0x7fae1015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 0x7fae1015c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47678/0 (socket says 172.21.15.70:47678) 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae0f7fe640 1 -- 172.21.15.70:0/843327983 learned_addr learned my addr 172.21.15.70:0/843327983 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae0ffff640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf8003970 0x7fae101f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.416+0000 7fae14f48640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100af320 0x7fae10070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0f7fe640 1 -- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100af320 msgr2=0x7fae10070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0f7fe640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100af320 0x7fae10070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0f7fe640 1 -- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf8003970 msgr2=0x7fae101f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0f7fe640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf8003970 0x7fae101f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0f7fe640 1 -- 172.21.15.70:0/843327983 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae100b1450 con 0x7fae100ae9d0 2023-08-25T09:46:37.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0ffff640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf8003970 0x7fae101f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:37.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae14f48640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100af320 0x7fae10070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:37.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0f7fe640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 0x7fae1015c4b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fae0400c7a0 tx=0x7fae0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:37.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040074e0 con 0x7fae100ae9d0 2023-08-25T09:46:37.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04007660 con 0x7fae100ae9d0 2023-08-25T09:46:37.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04004a20 con 0x7fae100ae9d0 2023-08-25T09:46:37.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae100b1730 con 0x7fae100ae9d0 2023-08-25T09:46:37.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.417+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae101391d0 con 0x7fae100ae9d0 2023-08-25T09:46:37.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.419+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1013ad90 con 0x7fae100ae9d0 2023-08-25T09:46:37.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.420+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae04004b80 con 0x7fae100ae9d0 2023-08-25T09:46:37.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.421+0000 7fae0cff9640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadfc06a050 0x7fadfc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.421+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fae04087890 con 0x7fae100ae9d0 2023-08-25T09:46:37.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.421+0000 7fae0ffff640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadfc06a050 0x7fadfc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.422+0000 7fae0ffff640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadfc06a050 0x7fadfc06c510 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fae00036000 tx=0x7fae00030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:37.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.426+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae04052f80 con 0x7fae100ae9d0 2023-08-25T09:46:37.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.616+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae10061460 con 0x7fae100ae9d0 2023-08-25T09:46:37.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.617+0000 7fae0cff9640 1 -- 172.21.15.70:0/843327983 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v136) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae04018020 con 0x7fae100ae9d0 2023-08-25T09:46:37.618 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:46:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadfc06a050 msgr2=0x7fadfc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadfc06a050 0x7fadfc06c510 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fae00036000 tx=0x7fae00030000 comp rx=0 tx=0).stop 2023-08-25T09:46:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 msgr2=0x7fae1015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 0x7fae1015c4b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fae0400c7a0 tx=0x7fae0400cc60 comp rx=0 tx=0).stop 2023-08-25T09:46:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 shutdown_connections 2023-08-25T09:46:37.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fadfc06a050 0x7fadfc06c510 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae100af320 0x7fae10070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae100ae9d0 0x7fae1015c4b0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 --2- 172.21.15.70:0/843327983 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fadf8003970 0x7fae101f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.619+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 >> 172.21.15.70:0/843327983 conn(0x7fae100cc060 msgr2=0x7fae10141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:37.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.620+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 shutdown_connections 2023-08-25T09:46:37.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.620+0000 7fae15749640 1 -- 172.21.15.70:0/843327983 wait complete. 2023-08-25T09:46:37.639 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:46:37.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.860+0000 7f286f46a640 1 Processor -- start 2023-08-25T09:46:37.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.860+0000 7f286f46a640 1 -- start start 2023-08-25T09:46:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.861+0000 7f286f46a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f286815a920 0x7f28680c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.861+0000 7f286f46a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28680adbb0 con 0x7f28680b1ce0 2023-08-25T09:46:37.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.861+0000 7f286f46a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28680c90a0 con 0x7f28680b0320 2023-08-25T09:46:37.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.861+0000 7f286f46a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28680c9380 con 0x7f286815a920 2023-08-25T09:46:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.861+0000 7f286ec69640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f28680b1ce0 0x7f28680ad3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45486/0 (socket says 172.21.15.70:45486) 2023-08-25T09:46:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.861+0000 7f286ec69640 1 -- 172.21.15.70:0/459224583 learned_addr learned my addr 172.21.15.70:0/459224583 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173623785 0 0) 0x7f28680adbb0 con 0x7f28680b1ce0 2023-08-25T09:46:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2854003520 con 0x7f28680b1ce0 2023-08-25T09:46:37.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286dc67640 1 --2- 172.21.15.70:0/459224583 >> v2:172.21.15.155:3300/0 conn(0x7f286815a920 0x7f28680c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2000364149 0 0) 0x7f2854003520 con 0x7f28680b1ce0 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 >> v2:172.21.15.155:3300/0 conn(0x7f286815a920 msgr2=0x7f28680c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 --2- 172.21.15.70:0/459224583 >> v2:172.21.15.155:3300/0 conn(0x7f286815a920 0x7f28680c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 >> v1:172.21.15.155:6789/0 conn(0x7f28680b0320 legacy=0x7f28680b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680c9660 con 0x7f28680b1ce0 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f28600019f0 con 0x7f28680b1ce0 2023-08-25T09:46:37.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286dc67640 1 --2- 172.21.15.70:0/459224583 >> v2:172.21.15.155:3300/0 conn(0x7f286815a920 0x7f28680c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 >> v1:172.21.15.70:6789/0 conn(0x7f28680b1ce0 legacy=0x7f28680ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2854003970 0x7f2854003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2854004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2854005180 0x7f285400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c9380 con 0x7f28540047c0 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285400c0d0 con 0x7f2854003970 2023-08-25T09:46:37.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.862+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285400c290 con 0x7f2854005180 2023-08-25T09:46:37.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286dc67640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2854004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286e468640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2854003970 0x7f2854003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286dc67640 1 -- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2854005180 msgr2=0x7f285400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286dc67640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2854005180 0x7f285400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286dc67640 1 -- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2854003970 msgr2=0x7f2854003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286dc67640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2854003970 0x7f2854003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286dc67640 1 -- 172.21.15.70:0/459224583 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854003520 con 0x7f28540047c0 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.863+0000 7f286ec69640 1 -- 172.21.15.70:0/459224583 reap_dead start 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.864+0000 7f286dc67640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2854004c40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f286815ad30 tx=0x7f28540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9457c17fcc331c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.864+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858004230 con 0x7f28540047c0 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.864+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858004390 con 0x7f28540047c0 2023-08-25T09:46:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.864+0000 7f286d466640 1 -- 172.21.15.70:0/459224583 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858003830 con 0x7f28540047c0 2023-08-25T09:46:37.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 -- 172.21.15.70:0/459224583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 msgr2=0x7f2854004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2854004c40 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f286815ad30 tx=0x7f28540057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 -- 172.21.15.70:0/459224583 shutdown_connections 2023-08-25T09:46:37.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 --2- 172.21.15.70:0/459224583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2854004c40 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 -- 172.21.15.70:0/459224583 >> 172.21.15.70:0/459224583 conn(0x7f28680ce830 msgr2=0x7f28680cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:37.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 -- 172.21.15.70:0/459224583 shutdown_connections 2023-08-25T09:46:37.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.865+0000 7f286f46a640 1 -- 172.21.15.70:0/459224583 wait complete. 2023-08-25T09:46:37.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.867+0000 7f286f46a640 1 Processor -- start 2023-08-25T09:46:37.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.867+0000 7f286f46a640 1 -- start start 2023-08-25T09:46:37.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286f46a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2868154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286f46a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28680c3f10 0x7f28680c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286f46a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28681596d0 0x7f28681ed2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286f46a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2868154780 con 0x7f28540047c0 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286f46a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c1eb0 con 0x7f28681596d0 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286f46a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c2750 con 0x7f28680c3f10 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286e468640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2868154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286e468640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2868154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47682/0 (socket says 172.21.15.70:47682) 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.868+0000 7f286e468640 1 -- 172.21.15.70:0/948203285 learned_addr learned my addr 172.21.15.70:0/948203285 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286e468640 1 -- 172.21.15.70:0/948203285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28680c3f10 msgr2=0x7f28680c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286e468640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28680c3f10 0x7f28680c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286e468640 1 -- 172.21.15.70:0/948203285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28681596d0 msgr2=0x7f28681ed2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286e468640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28681596d0 0x7f28681ed2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286e468640 1 -- 172.21.15.70:0/948203285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680c4390 con 0x7f28540047c0 2023-08-25T09:46:37.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286e468640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2868154240 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f286400e410 tx=0x7f286400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:37.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28640052e0 con 0x7f28540047c0 2023-08-25T09:46:37.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2864005440 con 0x7f28540047c0 2023-08-25T09:46:37.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2864003860 con 0x7f28540047c0 2023-08-25T09:46:37.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.869+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2868154970 con 0x7f28540047c0 2023-08-25T09:46:37.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.870+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28680c23e0 con 0x7f28540047c0 2023-08-25T09:46:37.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.871+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2868061460 con 0x7f28540047c0 2023-08-25T09:46:37.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.872+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f28640039c0 con 0x7f28540047c0 2023-08-25T09:46:37.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.872+0000 7f285effd640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f284c06a050 0x7f284c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:37.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.873+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2864087d20 con 0x7f28540047c0 2023-08-25T09:46:37.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.873+0000 7f286dc67640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f284c06a050 0x7f284c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:37.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.873+0000 7f286dc67640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f284c06a050 0x7f284c06c510 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2858035000 tx=0x7f2858030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:37.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:37.877+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f28640534b0 con 0x7f28540047c0 2023-08-25T09:46:38.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.064+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28680cc3f0 con 0x7f28540047c0 2023-08-25T09:46:38.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.065+0000 7f285effd640 1 -- 172.21.15.70:0/948203285 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v136) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2864019020 con 0x7f28540047c0 2023-08-25T09:46:38.067 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:46:38.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f284c06a050 msgr2=0x7f284c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f284c06a050 0x7f284c06c510 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2858035000 tx=0x7f2858030000 comp rx=0 tx=0).stop 2023-08-25T09:46:38.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 msgr2=0x7f2868154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2868154240 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f286400e410 tx=0x7f286400e8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 shutdown_connections 2023-08-25T09:46:38.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f284c06a050 0x7f284c06c510 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28681596d0 0x7f28681ed2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28680c3f10 0x7f28680c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 --2- 172.21.15.70:0/948203285 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28540047c0 0x7f2868154240 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.067+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 >> 172.21.15.70:0/948203285 conn(0x7f28680ce830 msgr2=0x7f2868157870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:38.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.068+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 shutdown_connections 2023-08-25T09:46:38.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.068+0000 7f286f46a640 1 -- 172.21.15.70:0/948203285 wait complete. 2023-08-25T09:46:38.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:46:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.293+0000 7fe5682ec640 1 Processor -- start 2023-08-25T09:46:38.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.293+0000 7fe5682ec640 1 -- start start 2023-08-25T09:46:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe5682ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5601593a0 0x7fe5600cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:38.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe5682ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56006ed50 0x7fe56006f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe5682ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe560159780 con 0x7fe5600c8db0 2023-08-25T09:46:38.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe5682ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56006f720 con 0x7fe5601593a0 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe5682ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600b21d0 con 0x7fe56006ed50 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe566088640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5601593a0 0x7fe5600cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe566088640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe5601593a0 0x7fe5600cabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47698/0 (socket says 172.21.15.70:47698) 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.295+0000 7fe566088640 1 -- 172.21.15.70:0/1835725985 learned_addr learned my addr 172.21.15.70:0/1835725985 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.296+0000 7fe566088640 1 -- 172.21.15.70:0/1835725985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56006ed50 msgr2=0x7fe56006f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.296+0000 7fe566088640 1 --2- 172.21.15.70:0/1835725985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56006ed50 0x7fe56006f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.296+0000 7fe566088640 1 -- 172.21.15.70:0/1835725985 >> v1:172.21.15.70:6789/0 conn(0x7fe5600c8db0 legacy=0x7fe5600c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.296+0000 7fe566088640 1 -- 172.21.15.70:0/1835725985 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560070740 con 0x7fe5601593a0 2023-08-25T09:46:38.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.297+0000 7fe566088640 1 --2- 172.21.15.70:0/1835725985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600cabc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009850 tx=0x7fe55c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12c88f440f81b3a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.297+0000 7fe565086640 1 -- 172.21.15.70:0/1835725985 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c004230 con 0x7fe5601593a0 2023-08-25T09:46:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.297+0000 7fe565086640 1 -- 172.21.15.70:0/1835725985 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55c004390 con 0x7fe5601593a0 2023-08-25T09:46:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.297+0000 7fe565086640 1 -- 172.21.15.70:0/1835725985 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55c031830 con 0x7fe5601593a0 2023-08-25T09:46:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 -- 172.21.15.70:0/1835725985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 msgr2=0x7fe5600cabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 --2- 172.21.15.70:0/1835725985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600cabc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009850 tx=0x7fe55c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 -- 172.21.15.70:0/1835725985 shutdown_connections 2023-08-25T09:46:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 --2- 172.21.15.70:0/1835725985 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe56006ed50 0x7fe56006f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 --2- 172.21.15.70:0/1835725985 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600cabc0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 -- 172.21.15.70:0/1835725985 >> 172.21.15.70:0/1835725985 conn(0x7fe5600ce760 msgr2=0x7fe5600ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 -- 172.21.15.70:0/1835725985 shutdown_connections 2023-08-25T09:46:38.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.298+0000 7fe5682ec640 1 -- 172.21.15.70:0/1835725985 wait complete. 2023-08-25T09:46:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.300+0000 7fe5682ec640 1 Processor -- start 2023-08-25T09:46:38.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.300+0000 7fe5682ec640 1 -- start start 2023-08-25T09:46:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.301+0000 7fe5682ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56006ed50 0x7fe5601550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.301+0000 7fe5682ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5600c8db0 0x7fe5601e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.301+0000 7fe5682ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.301+0000 7fe5682ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601e1760 con 0x7fe5601593a0 2023-08-25T09:46:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.301+0000 7fe5682ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601e25c0 con 0x7fe56006ed50 2023-08-25T09:46:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.301+0000 7fe5682ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5601e19b0 con 0x7fe5600c8db0 2023-08-25T09:46:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47702/0 (socket says 172.21.15.70:47702) 2023-08-25T09:46:38.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 -- 172.21.15.70:0/3965348079 learned_addr learned my addr 172.21.15.70:0/3965348079 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 -- 172.21.15.70:0/3965348079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5600c8db0 msgr2=0x7fe5601e2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5600c8db0 0x7fe5601e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 -- 172.21.15.70:0/3965348079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56006ed50 msgr2=0x7fe5601550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56006ed50 0x7fe5601550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 -- 172.21.15.70:0/3965348079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009460 con 0x7fe5601593a0 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.302+0000 7fe566889640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600ccc90 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c7d0 tx=0x7fe55800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.303+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558003b70 con 0x7fe5601593a0 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.303+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe558003cf0 con 0x7fe5601593a0 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.303+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55800aa90 con 0x7fe5601593a0 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.303+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5601e1c90 con 0x7fe5601593a0 2023-08-25T09:46:38.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.303+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5601e2250 con 0x7fe5601593a0 2023-08-25T09:46:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.304+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56004e420 con 0x7fe5601593a0 2023-08-25T09:46:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.305+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe55800abf0 con 0x7fe5601593a0 2023-08-25T09:46:38.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.306+0000 7fe556ffd640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54406a050 0x7fe54406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.306+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe5580882f0 con 0x7fe5601593a0 2023-08-25T09:46:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.306+0000 7fe566088640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54406a050 0x7fe54406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.307+0000 7fe566088640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54406a050 0x7fe54406c510 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe55c035000 tx=0x7fe55c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:38.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.310+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe558053940 con 0x7fe5601593a0 2023-08-25T09:46:38.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.511+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe560159780 con 0x7fe5601593a0 2023-08-25T09:46:38.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.511+0000 7fe556ffd640 1 -- 172.21.15.70:0/3965348079 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v136) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe560159780 con 0x7fe5601593a0 2023-08-25T09:46:38.513 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:46:38.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54406a050 msgr2=0x7fe54406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54406a050 0x7fe54406c510 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe55c035000 tx=0x7fe55c030000 comp rx=0 tx=0).stop 2023-08-25T09:46:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 msgr2=0x7fe5600ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:38.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600ccc90 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe55800c7d0 tx=0x7fe55800cc90 comp rx=0 tx=0).stop 2023-08-25T09:46:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 shutdown_connections 2023-08-25T09:46:38.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe54406a050 0x7fe54406c510 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe5601593a0 0x7fe5600ccc90 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe5600c8db0 0x7fe5601e2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 --2- 172.21.15.70:0/3965348079 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe56006ed50 0x7fe5601550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.514+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 >> 172.21.15.70:0/3965348079 conn(0x7fe5600ce760 msgr2=0x7fe5600c9690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:38.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.515+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 shutdown_connections 2023-08-25T09:46:38.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:38.515+0000 7fe5682ec640 1 -- 172.21.15.70:0/3965348079 wait complete. 2023-08-25T09:46:38.534 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:46:38.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:38.535 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:46:38.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[91335]: fuse finished with error 0 and tester_r 0 2023-08-25T09:46:38.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:46:38.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:38.642 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:46:38.738 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:46:38.738 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:38.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:46:38.799 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:46:38.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:38.799 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:46:38.849 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:46:38.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:46:38.851 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:46:38.851 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:38.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:46:38.911 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:46:38.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:46:38.912 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:46:38.965 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:46:38.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:46:38.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:39.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.189+0000 7fe64f67c640 1 Processor -- start 2023-08-25T09:46:39.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.189+0000 7fe64f67c640 1 -- start start 2023-08-25T09:46:39.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64f67c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe64815e830 0x7fe64806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64f67c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe64815ec10 0x7fe6480b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64f67c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6480b1b60 con 0x7fe64806b2b0 2023-08-25T09:46:39.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64f67c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64806bd80 con 0x7fe64815e830 2023-08-25T09:46:39.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64f67c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64806b7c0 con 0x7fe64815ec10 2023-08-25T09:46:39.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64d418640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe64815e830 0x7fe64806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64d418640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe64815e830 0x7fe64806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43466/0 (socket says 172.21.15.70:43466) 2023-08-25T09:46:39.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.191+0000 7fe64d418640 1 -- 172.21.15.70:0/3078097616 learned_addr learned my addr 172.21.15.70:0/3078097616 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:39.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.192+0000 7fe64d418640 1 -- 172.21.15.70:0/3078097616 >> v2:172.21.15.155:3300/0 conn(0x7fe64815ec10 msgr2=0x7fe6480b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:39.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.192+0000 7fe64d418640 1 --2- 172.21.15.70:0/3078097616 >> v2:172.21.15.155:3300/0 conn(0x7fe64815ec10 0x7fe6480b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.192+0000 7fe64d418640 1 -- 172.21.15.70:0/3078097616 >> v1:172.21.15.155:6789/0 conn(0x7fe64806b2b0 legacy=0x7fe64815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:39.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.192+0000 7fe64d418640 1 -- 172.21.15.70:0/3078097616 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64806baa0 con 0x7fe64815e830 2023-08-25T09:46:39.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.193+0000 7fe64d418640 1 --2- 172.21.15.70:0/3078097616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe64806ad70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe64806a710 tx=0x7fe64402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d7791a09451ccc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:39.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.193+0000 7fe63ffff640 1 -- 172.21.15.70:0/3078097616 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64402ca80 con 0x7fe64815e830 2023-08-25T09:46:39.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.193+0000 7fe63ffff640 1 -- 172.21.15.70:0/3078097616 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64402cbe0 con 0x7fe64815e830 2023-08-25T09:46:39.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.193+0000 7fe63ffff640 1 -- 172.21.15.70:0/3078097616 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6440343f0 con 0x7fe64815e830 2023-08-25T09:46:39.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 -- 172.21.15.70:0/3078097616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 msgr2=0x7fe64806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 --2- 172.21.15.70:0/3078097616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe64806ad70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe64806a710 tx=0x7fe64402bee0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 -- 172.21.15.70:0/3078097616 shutdown_connections 2023-08-25T09:46:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 --2- 172.21.15.70:0/3078097616 >> v2:172.21.15.155:3300/0 conn(0x7fe64815ec10 0x7fe6480b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 --2- 172.21.15.70:0/3078097616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe64806ad70 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 -- 172.21.15.70:0/3078097616 >> 172.21.15.70:0/3078097616 conn(0x7fe648069850 msgr2=0x7fe648069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 -- 172.21.15.70:0/3078097616 shutdown_connections 2023-08-25T09:46:39.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.194+0000 7fe64f67c640 1 -- 172.21.15.70:0/3078097616 wait complete. 2023-08-25T09:46:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.197+0000 7fe64f67c640 1 Processor -- start 2023-08-25T09:46:39.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.197+0000 7fe64f67c640 1 -- start start 2023-08-25T09:46:39.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64f67c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe64806b2b0 0x7fe6481f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64f67c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe648155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64f67c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe64815ec10 0x7fe6481f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64f67c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481f2440 con 0x7fe64815e830 2023-08-25T09:46:39.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64f67c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481f5ee0 con 0x7fe64806b2b0 2023-08-25T09:46:39.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64f67c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6481f61c0 con 0x7fe64815ec10 2023-08-25T09:46:39.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64cc17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe648155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64cc17640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe648155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43472/0 (socket says 172.21.15.70:43472) 2023-08-25T09:46:39.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.198+0000 7fe64cc17640 1 -- 172.21.15.70:0/803659295 learned_addr learned my addr 172.21.15.70:0/803659295 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:39.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64cc17640 1 -- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe64815ec10 msgr2=0x7fe6481f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64cc17640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe64815ec10 0x7fe6481f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64cc17640 1 -- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe64806b2b0 msgr2=0x7fe6481f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64d418640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe64806b2b0 0x7fe6481f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64cc17640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe64806b2b0 0x7fe6481f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64cc17640 1 -- 172.21.15.70:0/803659295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe644009b30 con 0x7fe64815e830 2023-08-25T09:46:39.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64d418640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe64806b2b0 0x7fe6481f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:39.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64cc17640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe648155050 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c410 tx=0x7fe63800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:39.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6380035f0 con 0x7fe64815e830 2023-08-25T09:46:39.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe638003750 con 0x7fe64815e830 2023-08-25T09:46:39.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.199+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64815ca20 con 0x7fe64815e830 2023-08-25T09:46:39.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.200+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638003b40 con 0x7fe64815e830 2023-08-25T09:46:39.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.200+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6481f20a0 con 0x7fe64815e830 2023-08-25T09:46:39.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.201+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64815e3c0 con 0x7fe64815e830 2023-08-25T09:46:39.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.202+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe638003ca0 con 0x7fe64815e830 2023-08-25T09:46:39.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.202+0000 7fe63dffb640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe62c06a050 0x7fe62c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.203+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe638087d10 con 0x7fe64815e830 2023-08-25T09:46:39.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.203+0000 7fe64d418640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe62c06a050 0x7fe62c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.204+0000 7fe64d418640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe62c06a050 0x7fe62c06c510 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe644035000 tx=0x7fe644030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:39.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.207+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe63808d020 con 0x7fe64815e830 2023-08-25T09:46:39.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.453+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe64804e420 con 0x7fe64815e830 2023-08-25T09:46:39.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.454+0000 7fe63dffb640 1 -- 172.21.15.70:0/803659295 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7fe638017020 con 0x7fe64815e830 2023-08-25T09:46:39.456 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:39.457 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:39.457 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:39.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe62c06a050 msgr2=0x7fe62c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe62c06a050 0x7fe62c06c510 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe644035000 tx=0x7fe644030000 comp rx=0 tx=0).stop 2023-08-25T09:46:39.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 msgr2=0x7fe648155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe648155050 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c410 tx=0x7fe63800c8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 shutdown_connections 2023-08-25T09:46:39.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe62c06a050 0x7fe62c06c510 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe64815ec10 0x7fe6481f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe64815e830 0x7fe648155050 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 --2- 172.21.15.70:0/803659295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe64806b2b0 0x7fe6481f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 >> 172.21.15.70:0/803659295 conn(0x7fe648069850 msgr2=0x7fe6480ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:39.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 shutdown_connections 2023-08-25T09:46:39.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.457+0000 7fe64f67c640 1 -- 172.21.15.70:0/803659295 wait complete. 2023-08-25T09:46:39.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:39.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.689+0000 7f5bed63f640 1 Processor -- start 2023-08-25T09:46:39.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.689+0000 7f5bed63f640 1 -- start start 2023-08-25T09:46:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5bed63f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 0x7f5be806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5bed63f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be80b1b60 con 0x7f5be806b2b0 2023-08-25T09:46:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5bed63f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5be806b7c0 con 0x7f5be815ec10 2023-08-25T09:46:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5bed63f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be806bd80 con 0x7f5be815e830 2023-08-25T09:46:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5be67fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5be806b2b0 0x7f5be815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42540/0 (socket says 172.21.15.70:42540) 2023-08-25T09:46:39.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5be67fc640 1 -- 172.21.15.70:0/3099634991 learned_addr learned my addr 172.21.15.70:0/3099634991 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:39.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 0x7f5be806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.691+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531532911 0 0) 0x7f5be80b1b60 con 0x7f5be806b2b0 2023-08-25T09:46:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bd0003520 con 0x7f5be806b2b0 2023-08-25T09:46:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1538126515 0 0) 0x7f5bd0003520 con 0x7f5be806b2b0 2023-08-25T09:46:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 msgr2=0x7f5be806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 0x7f5be806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 >> v1:172.21.15.155:6789/0 conn(0x7f5be815ec10 legacy=0x7f5be80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be806baa0 con 0x7f5be806b2b0 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5bdc001670 con 0x7f5be806b2b0 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 >> v1:172.21.15.70:6789/0 conn(0x7f5be806b2b0 legacy=0x7f5be815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd00039a0 0x7f5bd0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bd00047c0 0x7f5bd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 0x7f5bd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be806bd80 con 0x7f5bd00039a0 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd000ba30 con 0x7f5bd00047c0 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd000bbf0 con 0x7f5be815e830 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.692+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd00039a0 0x7f5bd0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.693+0000 7f5be6ffd640 1 -- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 msgr2=0x7f5bd0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.693+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5be815e830 0x7f5bd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.693+0000 7f5be67fc640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bd00047c0 0x7f5bd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.693+0000 7f5be6ffd640 1 -- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bd00047c0 msgr2=0x7f5bd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:39.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.693+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bd00047c0 0x7f5bd0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.693+0000 7f5be6ffd640 1 -- 172.21.15.70:0/3099634991 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd0003520 con 0x7f5bd00039a0 2023-08-25T09:46:39.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.694+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd00039a0 0x7f5bd0003dc0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009a90 tx=0x7f5bd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18525e3d2e31121 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:39.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.694+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd802c6b0 con 0x7f5bd00039a0 2023-08-25T09:46:39.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.694+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd802c810 con 0x7f5bd00039a0 2023-08-25T09:46:39.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.694+0000 7f5be5ffb640 1 -- 172.21.15.70:0/3099634991 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd802cba0 con 0x7f5bd00039a0 2023-08-25T09:46:39.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5bed63f640 1 -- 172.21.15.70:0/3099634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd00039a0 msgr2=0x7f5bd0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5bed63f640 1 --2- 172.21.15.70:0/3099634991 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bd00039a0 0x7f5bd0003dc0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009a90 tx=0x7f5bd00057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5be77fe640 1 -- 172.21.15.70:0/3099634991 reap_dead start 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5bed63f640 1 -- 172.21.15.70:0/3099634991 shutdown_connections 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5bed63f640 1 -- 172.21.15.70:0/3099634991 >> 172.21.15.70:0/3099634991 conn(0x7f5be8069850 msgr2=0x7f5be8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5bed63f640 1 -- 172.21.15.70:0/3099634991 shutdown_connections 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.695+0000 7f5bed63f640 1 -- 172.21.15.70:0/3099634991 wait complete. 2023-08-25T09:46:39.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.697+0000 7f5bed63f640 1 Processor -- start 2023-08-25T09:46:39.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.697+0000 7f5bed63f640 1 -- start start 2023-08-25T09:46:39.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5bed63f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd00039a0 0x7f5be80a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5bed63f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 0x7f5be814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5bed63f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80a8460 0x7f5be80cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5bed63f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be814e750 con 0x7f5be8157a10 2023-08-25T09:46:39.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5bed63f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be8157e90 con 0x7f5be80a8460 2023-08-25T09:46:39.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5bed63f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5be80a8bd0 con 0x7f5bd00039a0 2023-08-25T09:46:39.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5be67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 0x7f5be814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5be67fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 0x7f5be814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43502/0 (socket says 172.21.15.70:43502) 2023-08-25T09:46:39.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.698+0000 7f5be67fc640 1 -- 172.21.15.70:0/2183370987 learned_addr learned my addr 172.21.15.70:0/2183370987 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:39.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd00039a0 0x7f5be80a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be77fe640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80a8460 0x7f5be80cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be67fc640 1 -- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd00039a0 msgr2=0x7f5be80a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be67fc640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd00039a0 0x7f5be80a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be67fc640 1 -- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80a8460 msgr2=0x7f5be80cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be67fc640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80a8460 0x7f5be80cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be67fc640 1 -- 172.21.15.70:0/2183370987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be815ca20 con 0x7f5be8157a10 2023-08-25T09:46:39.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5be67fc640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 0x7f5be814e210 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00d810 tx=0x7f5bdc00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:39.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0074e0 con 0x7f5be8157a10 2023-08-25T09:46:39.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc007640 con 0x7f5be8157a10 2023-08-25T09:46:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0049e0 con 0x7f5be8157a10 2023-08-25T09:46:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.699+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be80a8eb0 con 0x7f5be8157a10 2023-08-25T09:46:39.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.700+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be80a9470 con 0x7f5be8157a10 2023-08-25T09:46:39.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.701+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be804e420 con 0x7f5be8157a10 2023-08-25T09:46:39.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.702+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5bdc004b40 con 0x7f5be8157a10 2023-08-25T09:46:39.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.702+0000 7f5bcf7fe640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bd406a080 0x7f5bd406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:39.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.703+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5bdc087f10 con 0x7f5be8157a10 2023-08-25T09:46:39.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.703+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bd406a080 0x7f5bd406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:39.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.703+0000 7f5be6ffd640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bd406a080 0x7f5bd406c540 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8036000 tx=0x7f5bd8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:39.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.707+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5bdc0535f0 con 0x7f5be8157a10 2023-08-25T09:46:39.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.899+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5be815c4a0 con 0x7f5be8157a10 2023-08-25T09:46:39.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.901+0000 7f5bcf7fe640 1 -- 172.21.15.70:0/2183370987 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+18332 (secure 0 0 0) 0x7f5bdc019020 con 0x7f5be8157a10 2023-08-25T09:46:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:39.903 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":136,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:29.557323+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bd406a080 msgr2=0x7f5bd406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bd406a080 0x7f5bd406c540 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8036000 tx=0x7f5bd8030000 comp rx=0 tx=0).stop 2023-08-25T09:46:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 msgr2=0x7f5be814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:39.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 0x7f5be814e210 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00d810 tx=0x7f5bdc00dcd0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 shutdown_connections 2023-08-25T09:46:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5bd406a080 0x7f5bd406c540 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5be80a8460 0x7f5be80cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5be8157a10 0x7f5be814e210 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 --2- 172.21.15.70:0/2183370987 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bd00039a0 0x7f5be80a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:39.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 >> 172.21.15.70:0/2183370987 conn(0x7f5be8069850 msgr2=0x7f5be815d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:39.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 shutdown_connections 2023-08-25T09:46:39.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:39.904+0000 7f5bed63f640 1 -- 172.21.15.70:0/2183370987 wait complete. 2023-08-25T09:46:39.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:46:11.556010+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:46:12.318797+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:39.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:40.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.127+0000 7f8280dfc640 1 Processor -- start 2023-08-25T09:46:40.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.128+0000 7f8280dfc640 1 -- start start 2023-08-25T09:46:40.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8280dfc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f827c15a920 0x7f827c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8280dfc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f827c0b1ce0 0x7f827c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8280dfc640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827c0ad8f0 con 0x7f827c0b0320 2023-08-25T09:46:40.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8280dfc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0adbb0 con 0x7f827c15a920 2023-08-25T09:46:40.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8280dfc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827c0c90a0 con 0x7f827c0b1ce0 2023-08-25T09:46:40.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8273fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f827c15a920 0x7f827c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8273fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f827c15a920 0x7f827c0c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43514/0 (socket says 172.21.15.70:43514) 2023-08-25T09:46:40.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.129+0000 7f8273fff640 1 -- 172.21.15.70:0/3640448135 learned_addr learned my addr 172.21.15.70:0/3640448135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:40.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.130+0000 7f8273fff640 1 -- 172.21.15.70:0/3640448135 >> v2:172.21.15.155:3300/0 conn(0x7f827c0b1ce0 msgr2=0x7f827c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:40.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.130+0000 7f827bfff640 1 --2- 172.21.15.70:0/3640448135 >> v2:172.21.15.155:3300/0 conn(0x7f827c0b1ce0 0x7f827c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.130+0000 7f8273fff640 1 --2- 172.21.15.70:0/3640448135 >> v2:172.21.15.155:3300/0 conn(0x7f827c0b1ce0 0x7f827c0ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.130+0000 7f8273fff640 1 -- 172.21.15.70:0/3640448135 >> v1:172.21.15.155:6789/0 conn(0x7f827c0b0320 legacy=0x7f827c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:40.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.130+0000 7f8273fff640 1 -- 172.21.15.70:0/3640448135 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827c0c9380 con 0x7f827c15a920 2023-08-25T09:46:40.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.131+0000 7f8273fff640 1 --2- 172.21.15.70:0/3640448135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c0c9e60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f827c0c8f30 tx=0x7f826402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a0f498bfb25a95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:40.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.131+0000 7f827affd640 1 -- 172.21.15.70:0/3640448135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264031870 con 0x7f827c15a920 2023-08-25T09:46:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.131+0000 7f827affd640 1 -- 172.21.15.70:0/3640448135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82640319d0 con 0x7f827c15a920 2023-08-25T09:46:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.131+0000 7f827affd640 1 -- 172.21.15.70:0/3640448135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264031d60 con 0x7f827c15a920 2023-08-25T09:46:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 -- 172.21.15.70:0/3640448135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 msgr2=0x7f827c0c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 --2- 172.21.15.70:0/3640448135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c0c9e60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f827c0c8f30 tx=0x7f826402b910 comp rx=0 tx=0).stop 2023-08-25T09:46:40.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 -- 172.21.15.70:0/3640448135 shutdown_connections 2023-08-25T09:46:40.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 --2- 172.21.15.70:0/3640448135 >> v2:172.21.15.155:3300/0 conn(0x7f827c0b1ce0 0x7f827c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 --2- 172.21.15.70:0/3640448135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c0c9e60 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 -- 172.21.15.70:0/3640448135 >> 172.21.15.70:0/3640448135 conn(0x7f827c0ce830 msgr2=0x7f827c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:40.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 -- 172.21.15.70:0/3640448135 shutdown_connections 2023-08-25T09:46:40.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.132+0000 7f8280dfc640 1 -- 172.21.15.70:0/3640448135 wait complete. 2023-08-25T09:46:40.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.133+0000 7f8280dfc640 1 Processor -- start 2023-08-25T09:46:40.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.134+0000 7f8280dfc640 1 -- start start 2023-08-25T09:46:40.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.134+0000 7f8280dfc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f827c0b0320 0x7f827c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.134+0000 7f8280dfc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f827c0b1ce0 0x7f827c15c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.134+0000 7f8280dfc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.134+0000 7f8280dfc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0c1eb0 con 0x7f827c15a920 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.134+0000 7f8280dfc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0c20c0 con 0x7f827c0b1ce0 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f8280dfc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827c0c23a0 con 0x7f827c0b0320 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43530/0 (socket says 172.21.15.70:43530) 2023-08-25T09:46:40.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 -- 172.21.15.70:0/4157829508 learned_addr learned my addr 172.21.15.70:0/4157829508 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827b7fe640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f827c0b0320 0x7f827c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 -- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f827c0b0320 msgr2=0x7f827c0ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f827c0b0320 0x7f827c0ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 -- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f827c0b1ce0 msgr2=0x7f827c15c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f827c0b1ce0 0x7f827c15c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 -- 172.21.15.70:0/4157829508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640094d0 con 0x7f827c15a920 2023-08-25T09:46:40.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.135+0000 7f827bfff640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c15c550 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f827400cd70 tx=0x7f827400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.136+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274003b90 con 0x7f827c15a920 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.136+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8274003cf0 con 0x7f827c15a920 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.136+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8274004240 con 0x7f827c15a920 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.136+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f827c0c2680 con 0x7f827c15a920 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.136+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f827c0c2e80 con 0x7f827c15a920 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.137+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827c0c2ba0 con 0x7f827c15a920 2023-08-25T09:46:40.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.138+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f82740043a0 con 0x7f827c15a920 2023-08-25T09:46:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.139+0000 7f8278ff9640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f825c06a050 0x7f825c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.139+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8274012070 con 0x7f827c15a920 2023-08-25T09:46:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.139+0000 7f827b7fe640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f825c06a050 0x7f825c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.139+0000 7f827b7fe640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f825c06a050 0x7f825c06c510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f826c007fd0 tx=0x7f826c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:40.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.142+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8274054700 con 0x7f827c15a920 2023-08-25T09:46:40.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.330+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f827c0ad3b0 con 0x7f827c15a920 2023-08-25T09:46:40.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.331+0000 7f8278ff9640 1 -- 172.21.15.70:0/4157829508 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8274057d20 con 0x7f827c15a920 2023-08-25T09:46:40.333 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f825c06a050 msgr2=0x7f825c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f825c06a050 0x7f825c06c510 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f826c007fd0 tx=0x7f826c009040 comp rx=0 tx=0).stop 2023-08-25T09:46:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 msgr2=0x7f827c15c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c15c550 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f827400cd70 tx=0x7f827400a660 comp rx=0 tx=0).stop 2023-08-25T09:46:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 shutdown_connections 2023-08-25T09:46:40.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f825c06a050 0x7f825c06c510 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f827c15a920 0x7f827c15c550 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f827c0b1ce0 0x7f827c15c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 --2- 172.21.15.70:0/4157829508 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f827c0b0320 0x7f827c0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 >> 172.21.15.70:0/4157829508 conn(0x7f827c0ce830 msgr2=0x7f827c070aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 shutdown_connections 2023-08-25T09:46:40.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.334+0000 7f8280dfc640 1 -- 172.21.15.70:0/4157829508 wait complete. 2023-08-25T09:46:40.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:40.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.567+0000 7fd572df3640 1 Processor -- start 2023-08-25T09:46:40.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.568+0000 7fd572df3640 1 -- start start 2023-08-25T09:46:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.569+0000 7fd572df3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd56c15ae50 0x7fd56c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.569+0000 7fd572df3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c069f60 0x7fd56c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.569+0000 7fd572df3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56c0cc5f0 con 0x7fd56c0cdc20 2023-08-25T09:46:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.569+0000 7fd572df3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56c070650 con 0x7fd56c15ae50 2023-08-25T09:46:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.569+0000 7fd572df3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c070930 con 0x7fd56c069f60 2023-08-25T09:46:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.570+0000 7fd570b8f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd56c15ae50 0x7fd56c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.570+0000 7fd570b8f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd56c15ae50 0x7fd56c0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:41492/0 (socket says 172.21.15.70:41492) 2023-08-25T09:46:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.570+0000 7fd570b8f640 1 -- 172.21.15.70:0/1260321741 learned_addr learned my addr 172.21.15.70:0/1260321741 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:40.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.570+0000 7fd571390640 1 --2- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c069f60 0x7fd56c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.571+0000 7fd570b8f640 1 -- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c069f60 msgr2=0x7fd56c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.571+0000 7fd570b8f640 1 --2- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c069f60 0x7fd56c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.571+0000 7fd570b8f640 1 -- 172.21.15.70:0/1260321741 >> v1:172.21.15.155:6789/0 conn(0x7fd56c0cdc20 legacy=0x7fd56c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.571+0000 7fd570b8f640 1 -- 172.21.15.70:0/1260321741 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56c070c10 con 0x7fd56c15ae50 2023-08-25T09:46:40.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.573+0000 7fd570b8f640 1 --2- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c15ae50 0x7fd56c0c2110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56002ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383436f9773a4d92 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.573+0000 7fd56b7fe640 1 -- 172.21.15.70:0/1260321741 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56002f840 con 0x7fd56c15ae50 2023-08-25T09:46:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.574+0000 7fd56b7fe640 1 -- 172.21.15.70:0/1260321741 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56002f9a0 con 0x7fd56c15ae50 2023-08-25T09:46:40.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.574+0000 7fd56b7fe640 1 -- 172.21.15.70:0/1260321741 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56002fd30 con 0x7fd56c15ae50 2023-08-25T09:46:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.574+0000 7fd572df3640 1 -- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c15ae50 msgr2=0x7fd56c0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.574+0000 7fd572df3640 1 --2- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c15ae50 0x7fd56c0c2110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd560009870 tx=0x7fd56002ba20 comp rx=0 tx=0).stop 2023-08-25T09:46:40.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.575+0000 7fd572df3640 1 -- 172.21.15.70:0/1260321741 shutdown_connections 2023-08-25T09:46:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.575+0000 7fd572df3640 1 --2- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c069f60 0x7fd56c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.575+0000 7fd572df3640 1 --2- 172.21.15.70:0/1260321741 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c15ae50 0x7fd56c0c2110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.575+0000 7fd572df3640 1 -- 172.21.15.70:0/1260321741 >> 172.21.15.70:0/1260321741 conn(0x7fd56c0cb0d0 msgr2=0x7fd56c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.575+0000 7fd572df3640 1 -- 172.21.15.70:0/1260321741 shutdown_connections 2023-08-25T09:46:40.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.575+0000 7fd572df3640 1 -- 172.21.15.70:0/1260321741 wait complete. 2023-08-25T09:46:40.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.577+0000 7fd572df3640 1 Processor -- start 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.577+0000 7fd572df3640 1 -- start start 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd572df3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c069f60 0x7fd56c14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd572df3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 0x7fd56c1f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd572df3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c15ae50 0x7fd56c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd572df3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c15b3b0 con 0x7fd56c0cdc20 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd572df3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c0c6bd0 con 0x7fd56c069f60 2023-08-25T09:46:40.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd572df3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56c14e1a0 con 0x7fd56c15ae50 2023-08-25T09:46:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd56bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 0x7fd56c1f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd56bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 0x7fd56c1f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43536/0 (socket says 172.21.15.70:43536) 2023-08-25T09:46:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.578+0000 7fd56bfff640 1 -- 172.21.15.70:0/2442829162 learned_addr learned my addr 172.21.15.70:0/2442829162 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd56bfff640 1 -- 172.21.15.70:0/2442829162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c15ae50 msgr2=0x7fd56c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd56bfff640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c15ae50 0x7fd56c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd56bfff640 1 -- 172.21.15.70:0/2442829162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c069f60 msgr2=0x7fd56c14dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd56bfff640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c069f60 0x7fd56c14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd56bfff640 1 -- 172.21.15.70:0/2442829162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600094d0 con 0x7fd56c0cdc20 2023-08-25T09:46:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd56bfff640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 0x7fd56c1f5820 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd55800c450 tx=0x7fd55800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:40.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5580052d0 con 0x7fd56c0cdc20 2023-08-25T09:46:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd558005430 con 0x7fd56c0cdc20 2023-08-25T09:46:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd558003830 con 0x7fd56c0cdc20 2023-08-25T09:46:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.579+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c0c7170 con 0x7fd56c0cdc20 2023-08-25T09:46:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.580+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56c1f6190 con 0x7fd56c0cdc20 2023-08-25T09:46:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.581+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56c066fb0 con 0x7fd56c0cdc20 2023-08-25T09:46:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.582+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd558003990 con 0x7fd56c0cdc20 2023-08-25T09:46:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.583+0000 7fd5697fa640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd55006a050 0x7fd55006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.583+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd558087b60 con 0x7fd56c0cdc20 2023-08-25T09:46:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.583+0000 7fd570b8f640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd55006a050 0x7fd55006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:40.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.583+0000 7fd570b8f640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd55006a050 0x7fd55006c510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd560036000 tx=0x7fd560039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:40.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.587+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd558053250 con 0x7fd56c0cdc20 2023-08-25T09:46:40.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.780+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd56c0c7300 con 0x7fd56c0cdc20 2023-08-25T09:46:40.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.781+0000 7fd5697fa640 1 -- 172.21.15.70:0/2442829162 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v137) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd558017020 con 0x7fd56c0cdc20 2023-08-25T09:46:40.782 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd55006a050 msgr2=0x7fd55006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd55006a050 0x7fd55006c510 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd560036000 tx=0x7fd560039040 comp rx=0 tx=0).stop 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 msgr2=0x7fd56c1f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 0x7fd56c1f5820 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fd55800c450 tx=0x7fd55800c910 comp rx=0 tx=0).stop 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 shutdown_connections 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd55006a050 0x7fd55006c510 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd56c15ae50 0x7fd56c0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd56c0cdc20 0x7fd56c1f5820 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 --2- 172.21.15.70:0/2442829162 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd56c069f60 0x7fd56c14dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 >> 172.21.15.70:0/2442829162 conn(0x7fd56c0cb0d0 msgr2=0x7fd56c0c4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 shutdown_connections 2023-08-25T09:46:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:40.783+0000 7fd572df3640 1 -- 172.21.15.70:0/2442829162 wait complete. 2023-08-25T09:46:40.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:46:41.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.021+0000 7eff6bb0f640 1 Processor -- start 2023-08-25T09:46:41.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.022+0000 7eff6bb0f640 1 -- start start 2023-08-25T09:46:41.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.023+0000 7eff6bb0f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7eff64160290 0x7eff640cd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.023+0000 7eff6bb0f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff64071c10 con 0x7eff64160670 2023-08-25T09:46:41.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.023+0000 7eff6bb0f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff64071000 con 0x7eff64069850 2023-08-25T09:46:41.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.023+0000 7eff6bb0f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640712e0 con 0x7eff64160290 2023-08-25T09:46:41.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff690aa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7eff64160670 0x7eff640cfc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42552/0 (socket says 172.21.15.70:42552) 2023-08-25T09:46:41.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff690aa640 1 -- 172.21.15.70:0/3994658990 learned_addr learned my addr 172.21.15.70:0/3994658990 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:41.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff698ab640 1 --2- 172.21.15.70:0/3994658990 >> v2:172.21.15.155:3300/0 conn(0x7eff64160290 0x7eff640cd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467098549 0 0) 0x7eff64071c10 con 0x7eff64160670 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff4c003520 con 0x7eff64160670 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 788742061 0 0) 0x7eff4c003520 con 0x7eff64160670 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 >> v2:172.21.15.155:3300/0 conn(0x7eff64160290 msgr2=0x7eff640cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 --2- 172.21.15.70:0/3994658990 >> v2:172.21.15.155:3300/0 conn(0x7eff64160290 0x7eff640cd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 >> v1:172.21.15.155:6789/0 conn(0x7eff64069850 legacy=0x7eff64160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff640715c0 con 0x7eff64160670 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7eff58001670 con 0x7eff64160670 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.024+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 >> v1:172.21.15.70:6789/0 conn(0x7eff64160670 legacy=0x7eff640cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff688a9640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff4c003970 0x7eff4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff688a9640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff4c0047c0 0x7eff4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff688a9640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640712e0 con 0x7eff4c005180 2023-08-25T09:46:41.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c00c020 con 0x7eff4c003970 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c00c1e0 con 0x7eff4c0047c0 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff6a0ac640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff698ab640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff4c003970 0x7eff4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff6a0ac640 1 -- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff4c0047c0 msgr2=0x7eff4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff690aa640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff4c0047c0 0x7eff4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff6a0ac640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff4c0047c0 0x7eff4c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff6a0ac640 1 -- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff4c003970 msgr2=0x7eff4c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff6a0ac640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff4c003970 0x7eff4c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff6a0ac640 1 -- 172.21.15.70:0/3994658990 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c003520 con 0x7eff4c005180 2023-08-25T09:46:41.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.025+0000 7eff698ab640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff4c003970 0x7eff4c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:41.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.026+0000 7eff6a0ac640 1 -- 172.21.15.70:0/3994658990 reap_dead start 2023-08-25T09:46:41.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.026+0000 7eff6a0ac640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff4c00b980 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7eff5c001720 tx=0x7eff4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a5c431b551bb51 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0052d0 con 0x7eff4c005180 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c02e050 con 0x7eff4c005180 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff688a9640 1 -- 172.21.15.70:0/3994658990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0035f0 con 0x7eff4c005180 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 -- 172.21.15.70:0/3994658990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 msgr2=0x7eff4c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff4c00b980 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7eff5c001720 tx=0x7eff4c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 -- 172.21.15.70:0/3994658990 shutdown_connections 2023-08-25T09:46:41.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/3994658990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff4c00b980 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 -- 172.21.15.70:0/3994658990 >> 172.21.15.70:0/3994658990 conn(0x7eff6406ba60 msgr2=0x7eff6406be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 -- 172.21.15.70:0/3994658990 shutdown_connections 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.027+0000 7eff6bb0f640 1 -- 172.21.15.70:0/3994658990 wait complete. 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.029+0000 7eff6bb0f640 1 Processor -- start 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.029+0000 7eff6bb0f640 1 -- start start 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff6bb0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff640b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff6bb0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff640b61d0 0x7eff64153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff6bb0f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff640cfae0 0x7eff640cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff6bb0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640d04d0 con 0x7eff4c005180 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff6bb0f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640b4f60 con 0x7eff640cfae0 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff6bb0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff640b3ec0 con 0x7eff640b61d0 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff698ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff640b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff698ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff640b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43550/0 (socket says 172.21.15.70:43550) 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.030+0000 7eff698ab640 1 -- 172.21.15.70:0/2525042898 learned_addr learned my addr 172.21.15.70:0/2525042898 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:41.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff698ab640 1 -- 172.21.15.70:0/2525042898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff640b61d0 msgr2=0x7eff64153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff698ab640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff640b61d0 0x7eff64153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff698ab640 1 -- 172.21.15.70:0/2525042898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff640cfae0 msgr2=0x7eff640cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff698ab640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff640cfae0 0x7eff640cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff698ab640 1 -- 172.21.15.70:0/2525042898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff640b41a0 con 0x7eff4c005180 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff698ab640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff640b37c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7eff5400f380 tx=0x7eff5400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54003b90 con 0x7eff4c005180 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54011040 con 0x7eff4c005180 2023-08-25T09:46:41.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54004020 con 0x7eff4c005180 2023-08-25T09:46:41.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff640b4480 con 0x7eff4c005180 2023-08-25T09:46:41.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.031+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff640b4a40 con 0x7eff4c005180 2023-08-25T09:46:41.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.032+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff640b4c00 con 0x7eff4c005180 2023-08-25T09:46:41.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.033+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7eff54004180 con 0x7eff4c005180 2023-08-25T09:46:41.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.034+0000 7eff627fc640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5006a080 0x7eff5006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.034+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6178+0+0 (secure 0 0 0) 0x7eff54088eb0 con 0x7eff4c005180 2023-08-25T09:46:41.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.034+0000 7eff690aa640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5006a080 0x7eff5006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.035+0000 7eff690aa640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5006a080 0x7eff5006c540 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7eff580017a0 tx=0x7eff58009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.039+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7eff54054480 con 0x7eff4c005180 2023-08-25T09:46:41.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.227+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff64066fb0 con 0x7eff4c005180 2023-08-25T09:46:41.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.228+0000 7eff627fc640 1 -- 172.21.15.70:0/2525042898 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff54057aa0 con 0x7eff4c005180 2023-08-25T09:46:41.229 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:46:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5006a080 msgr2=0x7eff5006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5006a080 0x7eff5006c540 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7eff580017a0 tx=0x7eff58009040 comp rx=0 tx=0).stop 2023-08-25T09:46:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 msgr2=0x7eff640b37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff640b37c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7eff5400f380 tx=0x7eff5400f840 comp rx=0 tx=0).stop 2023-08-25T09:46:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 shutdown_connections 2023-08-25T09:46:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5006a080 0x7eff5006c540 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff640cfae0 0x7eff640cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff640b61d0 0x7eff64153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 --2- 172.21.15.70:0/2525042898 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff4c005180 0x7eff640b37c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 >> 172.21.15.70:0/2525042898 conn(0x7eff6406ba60 msgr2=0x7eff640ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 shutdown_connections 2023-08-25T09:46:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.230+0000 7eff6bb0f640 1 -- 172.21.15.70:0/2525042898 wait complete. 2023-08-25T09:46:41.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:46:41.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.471+0000 7f6315f42640 1 Processor -- start 2023-08-25T09:46:41.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.472+0000 7f6315f42640 1 -- start start 2023-08-25T09:46:41.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.473+0000 7f6315f42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 0x7f63100c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.473+0000 7f6315f42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6310069f60 0x7f63100ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.473+0000 7f6315f42640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63100cc5f0 con 0x7f63100cdc20 2023-08-25T09:46:41.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.473+0000 7f6315f42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6310070650 con 0x7f6310069f60 2023-08-25T09:46:41.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.473+0000 7f6315f42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6310070930 con 0x7f631015ae50 2023-08-25T09:46:41.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f63100cdc20 0x7f6310069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42568/0 (socket says 172.21.15.70:42568) 2023-08-25T09:46:41.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630effd640 1 -- 172.21.15.70:0/1985165391 learned_addr learned my addr 172.21.15.70:0/1985165391 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:41.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 0x7f63100c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630ffff640 1 --2- 172.21.15.70:0/1985165391 >> v2:172.21.15.155:3300/0 conn(0x7f6310069f60 0x7f63100ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959128454 0 0) 0x7f63100cc5f0 con 0x7f63100cdc20 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62fc003520 con 0x7f63100cdc20 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3544543918 0 0) 0x7f62fc003520 con 0x7f63100cdc20 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 msgr2=0x7f63100c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 0x7f63100c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 >> v2:172.21.15.155:3300/0 conn(0x7f6310069f60 msgr2=0x7f63100ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 --2- 172.21.15.70:0/1985165391 >> v2:172.21.15.155:3300/0 conn(0x7f6310069f60 0x7f63100ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310070c10 con 0x7f63100cdc20 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f62f8001670 con 0x7f63100cdc20 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 >> v1:172.21.15.70:6789/0 conn(0x7f63100cdc20 legacy=0x7f6310069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc003970 0x7f62fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.474+0000 7f630e7fc640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0047c0 0x7f62fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630e7fc640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 0x7f62fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6310070650 con 0x7f62fc0047c0 2023-08-25T09:46:41.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6310070930 con 0x7f62fc003970 2023-08-25T09:46:41.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc00fd90 con 0x7f631015ae50 2023-08-25T09:46:41.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630effd640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0047c0 0x7f62fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630ffff640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 0x7f62fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc003970 0x7f62fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630effd640 1 -- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 msgr2=0x7f62fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630effd640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f631015ae50 0x7f62fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630effd640 1 -- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc003970 msgr2=0x7f62fc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630effd640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc003970 0x7f62fc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630effd640 1 -- 172.21.15.70:0/1985165391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc003520 con 0x7f62fc0047c0 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.475+0000 7f630f7fe640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc003970 0x7f62fc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.476+0000 7f630effd640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0047c0 0x7f62fc004c40 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c240 tx=0x7f62fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cd89753ac196f4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f802cb00 con 0x7f62fc0047c0 2023-08-25T09:46:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f802cc60 con 0x7f62fc0047c0 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f630e7fc640 1 -- 172.21.15.70:0/1985165391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f8005af0 con 0x7f62fc0047c0 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f6315f42640 1 -- 172.21.15.70:0/1985165391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0047c0 msgr2=0x7f62fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f6315f42640 1 --2- 172.21.15.70:0/1985165391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62fc0047c0 0x7f62fc004c40 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c240 tx=0x7f62fc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f630ffff640 1 -- 172.21.15.70:0/1985165391 reap_dead start 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f6315f42640 1 -- 172.21.15.70:0/1985165391 shutdown_connections 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f6315f42640 1 -- 172.21.15.70:0/1985165391 >> 172.21.15.70:0/1985165391 conn(0x7f63100cb0d0 msgr2=0x7f63100cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:41.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.477+0000 7f6315f42640 1 -- 172.21.15.70:0/1985165391 shutdown_connections 2023-08-25T09:46:41.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.478+0000 7f6315f42640 1 -- 172.21.15.70:0/1985165391 wait complete. 2023-08-25T09:46:41.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.479+0000 7f6315f42640 1 Processor -- start 2023-08-25T09:46:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.479+0000 7f6315f42640 1 -- start start 2023-08-25T09:46:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f6315f42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc0047c0 0x7f631015dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f6315f42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63100adf10 0x7f63101569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f6315f42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 0x7f6310156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f6315f42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100ae8b0 con 0x7f63100ae390 2023-08-25T09:46:41.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f6315f42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100aeb90 con 0x7f62fc0047c0 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f6315f42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63100aee70 con 0x7f63100adf10 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.480+0000 7f630ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 0x7f6310156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 0x7f6310156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43562/0 (socket says 172.21.15.70:43562) 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 -- 172.21.15.70:0/2788528146 learned_addr learned my addr 172.21.15.70:0/2788528146 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 -- 172.21.15.70:0/2788528146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63100adf10 msgr2=0x7f63101569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63100adf10 0x7f63101569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 -- 172.21.15.70:0/2788528146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc0047c0 msgr2=0x7f631015dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc0047c0 0x7f631015dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 -- 172.21.15.70:0/2788528146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100af150 con 0x7f63100ae390 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f630ffff640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 0x7f6310156f00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f63100c51a0 tx=0x7f6308011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.481+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308004500 con 0x7f63100ae390 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.482+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6308003610 con 0x7f63100ae390 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.482+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63080039a0 con 0x7f63100ae390 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.482+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63100af430 con 0x7f63100ae390 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.482+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631015e2d0 con 0x7f63100ae390 2023-08-25T09:46:41.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.483+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100af620 con 0x7f63100ae390 2023-08-25T09:46:41.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.484+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6308003b00 con 0x7f63100ae390 2023-08-25T09:46:41.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.484+0000 7f6307fff640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f006a050 0x7f62f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.484+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f6308088280 con 0x7f63100ae390 2023-08-25T09:46:41.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.485+0000 7f630f7fe640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f006a050 0x7f62f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.485+0000 7f630f7fe640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f006a050 0x7f62f006c510 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f63100c60d0 tx=0x7f6300009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.489+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f63080539a0 con 0x7f63100ae390 2023-08-25T09:46:41.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.685+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f631015e2d0 con 0x7f63100ae390 2023-08-25T09:46:41.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.686+0000 7f6307fff640 1 -- 172.21.15.70:0/2788528146 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v138) v1 ==== 100+0+11 (secure 0 0 0) 0x7f631015e2d0 con 0x7f63100ae390 2023-08-25T09:46:41.687 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:46:41.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f006a050 msgr2=0x7f62f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f006a050 0x7f62f006c510 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f63100c60d0 tx=0x7f6300009750 comp rx=0 tx=0).stop 2023-08-25T09:46:41.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 msgr2=0x7f6310156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 0x7f6310156f00 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f63100c51a0 tx=0x7f6308011920 comp rx=0 tx=0).stop 2023-08-25T09:46:41.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 shutdown_connections 2023-08-25T09:46:41.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62f006a050 0x7f62f006c510 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f63100ae390 0x7f6310156f00 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f63100adf10 0x7f63101569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 --2- 172.21.15.70:0/2788528146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62fc0047c0 0x7f631015dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.688+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 >> 172.21.15.70:0/2788528146 conn(0x7f63100cb0d0 msgr2=0x7f63100cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:41.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.689+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 shutdown_connections 2023-08-25T09:46:41.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.689+0000 7f6315f42640 1 -- 172.21.15.70:0/2788528146 wait complete. 2023-08-25T09:46:41.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:41.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.918+0000 7f0901234640 1 Processor -- start 2023-08-25T09:46:41.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.918+0000 7f0901234640 1 -- start start 2023-08-25T09:46:41.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.920+0000 7f0901234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 0x7f08fc0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.920+0000 7f0901234640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0cba70 con 0x7f08fc0cecc0 2023-08-25T09:46:41.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.920+0000 7f0901234640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc0cbd50 con 0x7f08fc0ce8e0 2023-08-25T09:46:41.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.920+0000 7f0901234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0cc030 con 0x7f08fc06e090 2023-08-25T09:46:41.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.920+0000 7f08fb59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f08fc0cecc0 0x7f08fc0cf140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42572/0 (socket says 172.21.15.70:42572) 2023-08-25T09:46:41.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.920+0000 7f08fb59e640 1 -- 172.21.15.70:0/687484329 learned_addr learned my addr 172.21.15.70:0/687484329 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:41.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08f3fff640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 0x7f08fc0cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614666458 0 0) 0x7f08fc0cba70 con 0x7f08fc0cecc0 2023-08-25T09:46:41.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08e8003520 con 0x7f08fc0cecc0 2023-08-25T09:46:41.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277317365 0 0) 0x7f08e8003520 con 0x7f08fc0cecc0 2023-08-25T09:46:41.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 msgr2=0x7f08fc0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 0x7f08fc0cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 >> v1:172.21.15.155:6789/0 conn(0x7f08fc0ce8e0 legacy=0x7f08fc0cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0cc310 con 0x7f08fc0cecc0 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f08f40019f0 con 0x7f08fc0cecc0 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 >> v1:172.21.15.70:6789/0 conn(0x7f08fc0cecc0 legacy=0x7f08fc0cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08e80039a0 0x7f08e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80047c0 0x7f08e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 0x7f08e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0cc030 con 0x7f08e80039a0 2023-08-25T09:46:41.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e800ba60 con 0x7f08e80047c0 2023-08-25T09:46:41.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.921+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e800bc20 con 0x7f08fc06e090 2023-08-25T09:46:41.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fad9d640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08e80039a0 0x7f08e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08f3fff640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80047c0 0x7f08e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fb59e640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 0x7f08e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fad9d640 1 -- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 msgr2=0x7f08e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fad9d640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc06e090 0x7f08e8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fad9d640 1 -- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80047c0 msgr2=0x7f08e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:41.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fad9d640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80047c0 0x7f08e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08fad9d640 1 -- 172.21.15.70:0/687484329 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e8003520 con 0x7f08e80039a0 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.922+0000 7f08f3fff640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80047c0 0x7f08e8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.923+0000 7f08fad9d640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08e80039a0 0x7f08e8003dc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f08ec007fd0 tx=0x7f08e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b52725f565cae41 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0052d0 con 0x7f08e80039a0 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08ec02e050 con 0x7f08e80039a0 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f08fa59c640 1 -- 172.21.15.70:0/687484329 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08ec003640 con 0x7f08e80039a0 2023-08-25T09:46:41.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f0901234640 1 -- 172.21.15.70:0/687484329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08e80039a0 msgr2=0x7f08e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f0901234640 1 --2- 172.21.15.70:0/687484329 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08e80039a0 0x7f08e8003dc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f08ec007fd0 tx=0x7f08e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f08fb59e640 1 -- 172.21.15.70:0/687484329 reap_dead start 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f0901234640 1 -- 172.21.15.70:0/687484329 shutdown_connections 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f0901234640 1 -- 172.21.15.70:0/687484329 >> 172.21.15.70:0/687484329 conn(0x7f08fc0d0860 msgr2=0x7f08fc0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f0901234640 1 -- 172.21.15.70:0/687484329 shutdown_connections 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.924+0000 7f0901234640 1 -- 172.21.15.70:0/687484329 wait complete. 2023-08-25T09:46:41.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.926+0000 7f0901234640 1 Processor -- start 2023-08-25T09:46:41.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.926+0000 7f0901234640 1 -- start start 2023-08-25T09:46:41.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.927+0000 7f0901234640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80039a0 0x7f08fc1dd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.927+0000 7f0901234640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 0x7f08fc1e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.927+0000 7f0901234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc1e1140 0x7f08fc0b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.927+0000 7f0901234640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc1e15f0 con 0x7f08fc1e03b0 2023-08-25T09:46:41.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.927+0000 7f0901234640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc1ddb60 con 0x7f08e80039a0 2023-08-25T09:46:41.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.927+0000 7f0901234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0b6170 con 0x7f08fc1e1140 2023-08-25T09:46:41.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 0x7f08fc1e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 0x7f08fc1e0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43588/0 (socket says 172.21.15.70:43588) 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 -- 172.21.15.70:0/252615648 learned_addr learned my addr 172.21.15.70:0/252615648 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 -- 172.21.15.70:0/252615648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc1e1140 msgr2=0x7f08fc0b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc1e1140 0x7f08fc0b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 -- 172.21.15.70:0/252615648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80039a0 msgr2=0x7f08fc1dd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80039a0 0x7f08fc1dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 -- 172.21.15.70:0/252615648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc1ddfb0 con 0x7f08fc1e03b0 2023-08-25T09:46:41.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.928+0000 7f08f3fff640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 0x7f08fc1e0c00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f08e4010890 tx=0x7f08e4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.929+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400eb00 con 0x7f08fc1e03b0 2023-08-25T09:46:41.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.929+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400ec60 con 0x7f08fc1e03b0 2023-08-25T09:46:41.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.929+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e401c400 con 0x7f08fc1e03b0 2023-08-25T09:46:41.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.929+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc1e3520 con 0x7f08fc1e03b0 2023-08-25T09:46:41.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.929+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc0b5690 con 0x7f08fc1e03b0 2023-08-25T09:46:41.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.930+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc04e420 con 0x7f08fc1e03b0 2023-08-25T09:46:41.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.931+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08e401c560 con 0x7f08fc1e03b0 2023-08-25T09:46:41.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.932+0000 7f08f37fe640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08dc06a0a0 0x7f08dc06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:41.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.932+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f08e4088ac0 con 0x7f08fc1e03b0 2023-08-25T09:46:41.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.932+0000 7f08fad9d640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08dc06a0a0 0x7f08dc06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:41.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.932+0000 7f08fad9d640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08dc06a0a0 0x7f08dc06c560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f08ec033000 tx=0x7f08ec02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:41.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:41.936+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08e4054110 con 0x7f08fc1e03b0 2023-08-25T09:46:42.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.165+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08fc0add20 con 0x7f08fc1e03b0 2023-08-25T09:46:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.166+0000 7f08f37fe640 1 -- 172.21.15.70:0/252615648 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7f08e4006c30 con 0x7f08fc1e03b0 2023-08-25T09:46:42.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:42.168 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:42.169 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:42.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08dc06a0a0 msgr2=0x7f08dc06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08dc06a0a0 0x7f08dc06c560 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f08ec033000 tx=0x7f08ec02f000 comp rx=0 tx=0).stop 2023-08-25T09:46:42.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 msgr2=0x7f08fc1e0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 0x7f08fc1e0c00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f08e4010890 tx=0x7f08e4010d50 comp rx=0 tx=0).stop 2023-08-25T09:46:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 shutdown_connections 2023-08-25T09:46:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08dc06a0a0 0x7f08dc06c560 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08fc1e1140 0x7f08fc0b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08fc1e03b0 0x7f08fc1e0c00 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 --2- 172.21.15.70:0/252615648 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08e80039a0 0x7f08fc1dd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.168+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 >> 172.21.15.70:0/252615648 conn(0x7f08fc0d0860 msgr2=0x7f08fc0ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.169+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 shutdown_connections 2023-08-25T09:46:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.169+0000 7f0901234640 1 -- 172.21.15.70:0/252615648 wait complete. 2023-08-25T09:46:42.192 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:42.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.404+0000 7f6c3f542640 1 Processor -- start 2023-08-25T09:46:42.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.404+0000 7f6c3f542640 1 -- start start 2023-08-25T09:46:42.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.405+0000 7f6c3f542640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 0x7f6c380b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.406+0000 7f6c3f542640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3815d3f0 con 0x7f6c380b2680 2023-08-25T09:46:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.406+0000 7f6c3f542640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c380b2d20 con 0x7f6c380ce5b0 2023-08-25T09:46:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.406+0000 7f6c3f542640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380b2b10 con 0x7f6c3815aa10 2023-08-25T09:46:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.406+0000 7f6c3dadf640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6c380b2680 0x7f6c3806f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42582/0 (socket says 172.21.15.70:42582) 2023-08-25T09:46:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.406+0000 7f6c3dadf640 1 -- 172.21.15.70:0/1605389857 learned_addr learned my addr 172.21.15.70:0/1605389857 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.406+0000 7f6c3cadd640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 0x7f6c380b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976558429 0 0) 0x7f6c3815d3f0 con 0x7f6c380b2680 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c24005730 con 0x7f6c380b2680 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964403852 0 0) 0x7f6c24005730 con 0x7f6c380b2680 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 msgr2=0x7f6c380b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 0x7f6c380b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 >> v1:172.21.15.155:6789/0 conn(0x7f6c380ce5b0 legacy=0x7f6c380ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3815d6b0 con 0x7f6c380b2680 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6c300019f0 con 0x7f6c380b2680 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 >> v1:172.21.15.70:6789/0 conn(0x7f6c380b2680 legacy=0x7f6c3806f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c24005bb0 0x7f6c24005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240069d0 0x7f6c24006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 0x7f6c24007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380b2b10 con 0x7f6c240069d0 2023-08-25T09:46:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400dc70 con 0x7f6c24005bb0 2023-08-25T09:46:42.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2400de30 con 0x7f6c3815aa10 2023-08-25T09:46:42.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.407+0000 7f6c3cadd640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240069d0 0x7f6c24006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.408+0000 7f6c3cadd640 1 -- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 msgr2=0x7f6c24007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:42.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.408+0000 7f6c3cadd640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c3815aa10 0x7f6c24007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.408+0000 7f6c3cadd640 1 -- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c24005bb0 msgr2=0x7f6c24005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:42.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.408+0000 7f6c3cadd640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c24005bb0 0x7f6c24005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.408+0000 7f6c3cadd640 1 -- 172.21.15.70:0/1605389857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24005730 con 0x7f6c240069d0 2023-08-25T09:46:42.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.409+0000 7f6c3cadd640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240069d0 0x7f6c24006e50 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009a90 tx=0x7f6c24007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110598ba48c5e886 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:42.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.409+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2802c690 con 0x7f6c240069d0 2023-08-25T09:46:42.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.409+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2802c7f0 con 0x7f6c240069d0 2023-08-25T09:46:42.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.409+0000 7f6c2ffff640 1 -- 172.21.15.70:0/1605389857 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2802cb80 con 0x7f6c240069d0 2023-08-25T09:46:42.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.409+0000 7f6c3f542640 1 -- 172.21.15.70:0/1605389857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240069d0 msgr2=0x7f6c24006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.409+0000 7f6c3f542640 1 --2- 172.21.15.70:0/1605389857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c240069d0 0x7f6c24006e50 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009a90 tx=0x7f6c24007a00 comp rx=0 tx=0).stop 2023-08-25T09:46:42.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.410+0000 7f6c3dadf640 1 -- 172.21.15.70:0/1605389857 reap_dead start 2023-08-25T09:46:42.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.410+0000 7f6c3f542640 1 -- 172.21.15.70:0/1605389857 shutdown_connections 2023-08-25T09:46:42.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.410+0000 7f6c3f542640 1 -- 172.21.15.70:0/1605389857 >> 172.21.15.70:0/1605389857 conn(0x7f6c38069850 msgr2=0x7f6c38069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:42.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.410+0000 7f6c3f542640 1 -- 172.21.15.70:0/1605389857 shutdown_connections 2023-08-25T09:46:42.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.410+0000 7f6c3f542640 1 -- 172.21.15.70:0/1605389857 wait complete. 2023-08-25T09:46:42.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.411+0000 7f6c3f542640 1 Processor -- start 2023-08-25T09:46:42.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.412+0000 7f6c3f542640 1 -- start start 2023-08-25T09:46:42.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.412+0000 7f6c3f542640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c240069d0 0x7f6c380cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.412+0000 7f6c3f542640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c380b0e90 0x7f6c380ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.412+0000 7f6c3f542640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 0x7f6c380cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3f542640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3806ce40 con 0x7f6c3814bb00 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3f542640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380cc4c0 con 0x7f6c240069d0 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3f542640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3806c910 con 0x7f6c380b0e90 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 0x7f6c380cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 0x7f6c380cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43602/0 (socket says 172.21.15.70:43602) 2023-08-25T09:46:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 -- 172.21.15.70:0/3048505971 learned_addr learned my addr 172.21.15.70:0/3048505971 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3d2de640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c240069d0 0x7f6c380cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3cadd640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c380b0e90 0x7f6c380ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 -- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c380b0e90 msgr2=0x7f6c380ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c380b0e90 0x7f6c380ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 -- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c240069d0 msgr2=0x7f6c380cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c240069d0 0x7f6c380cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3dadf640 1 -- 172.21.15.70:0/3048505971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3806cb60 con 0x7f6c3814bb00 2023-08-25T09:46:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.413+0000 7f6c3d2de640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c240069d0 0x7f6c380cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.414+0000 7f6c3dadf640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 0x7f6c380cd070 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f6c380cae20 tx=0x7f6c3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.414+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000a610 con 0x7f6c3814bb00 2023-08-25T09:46:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.414+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3000a790 con 0x7f6c3814bb00 2023-08-25T09:46:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.414+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c380b03b0 con 0x7f6c3814bb00 2023-08-25T09:46:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.414+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000ab40 con 0x7f6c3814bb00 2023-08-25T09:46:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.414+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380b0970 con 0x7f6c3814bb00 2023-08-25T09:46:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.415+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c380b0210 con 0x7f6c3814bb00 2023-08-25T09:46:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.416+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6c3000aca0 con 0x7f6c3814bb00 2023-08-25T09:46:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.417+0000 7f6c2dffb640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c1c06a080 0x7f6c1c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.417+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f6c30087ac0 con 0x7f6c3814bb00 2023-08-25T09:46:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.417+0000 7f6c3d2de640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c1c06a080 0x7f6c1c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.418+0000 7f6c3d2de640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c1c06a080 0x7f6c1c06c540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6c34001710 tx=0x7f6c34009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:42.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.422+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6c380b0210 con 0x7f6c3814bb00 2023-08-25T09:46:42.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.622+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c380b0210 con 0x7f6c3814bb00 2023-08-25T09:46:42.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.625+0000 7f6c2dffb640 1 -- 172.21.15.70:0/3048505971 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18581 (secure 0 0 0) 0x7f6c380b0210 con 0x7f6c3814bb00 2023-08-25T09:46:42.628 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:42.628 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":138,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:41.631656+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.11","mappings":[{"from":4,"to":6}]},{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"16.1e","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:42.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c1c06a080 msgr2=0x7f6c1c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c1c06a080 0x7f6c1c06c540 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6c34001710 tx=0x7f6c34009040 comp rx=0 tx=0).stop 2023-08-25T09:46:42.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 msgr2=0x7f6c380cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 0x7f6c380cd070 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f6c380cae20 tx=0x7f6c3000ccb0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 shutdown_connections 2023-08-25T09:46:42.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c1c06a080 0x7f6c1c06c540 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c3814bb00 0x7f6c380cd070 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6c380b0e90 0x7f6c380ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 --2- 172.21.15.70:0/3048505971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c240069d0 0x7f6c380cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 >> 172.21.15.70:0/3048505971 conn(0x7f6c38069850 msgr2=0x7f6c3806b720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:42.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 shutdown_connections 2023-08-25T09:46:42.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.628+0000 7f6c3f542640 1 -- 172.21.15.70:0/3048505971 wait complete. 2023-08-25T09:46:42.649 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:46:42.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.859+0000 7f472b339640 1 Processor -- start 2023-08-25T09:46:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.860+0000 7f472b339640 1 -- start start 2023-08-25T09:46:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.861+0000 7f472b339640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4724160ff0 0x7f472406b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.861+0000 7f472b339640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4724161ae0 0x7f47240ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.861+0000 7f472b339640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47241620b0 con 0x7f47240ce5f0 2023-08-25T09:46:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.861+0000 7f472b339640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47240cefe0 con 0x7f4724160ff0 2023-08-25T09:46:42.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.861+0000 7f472b339640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472406ad70 con 0x7f4724161ae0 2023-08-25T09:46:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4724160ff0 0x7f472406b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4724160ff0 0x7f472406b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43618/0 (socket says 172.21.15.70:43618) 2023-08-25T09:46:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 -- 172.21.15.70:0/1749129592 learned_addr learned my addr 172.21.15.70:0/1749129592 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:42.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 -- 172.21.15.70:0/1749129592 >> v2:172.21.15.155:3300/0 conn(0x7f4724161ae0 msgr2=0x7f47240ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 --2- 172.21.15.70:0/1749129592 >> v2:172.21.15.155:3300/0 conn(0x7f4724161ae0 0x7f47240ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 -- 172.21.15.70:0/1749129592 >> v1:172.21.15.155:6789/0 conn(0x7f47240ce5f0 legacy=0x7f47241613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.862+0000 7f47290d5640 1 -- 172.21.15.70:0/1749129592 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4724162390 con 0x7f4724160ff0 2023-08-25T09:46:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.863+0000 7f47290d5640 1 --2- 172.21.15.70:0/1749129592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4724160ff0 0x7f472406b780 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f47240b2a30 tx=0x7f472002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30dccb0d1be8305 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.863+0000 7f471bfff640 1 -- 172.21.15.70:0/1749129592 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720031870 con 0x7f4724160ff0 2023-08-25T09:46:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.863+0000 7f471bfff640 1 -- 172.21.15.70:0/1749129592 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47200319d0 con 0x7f4724160ff0 2023-08-25T09:46:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.863+0000 7f471bfff640 1 -- 172.21.15.70:0/1749129592 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720031d60 con 0x7f4724160ff0 2023-08-25T09:46:42.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 -- 172.21.15.70:0/1749129592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4724160ff0 msgr2=0x7f472406b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 --2- 172.21.15.70:0/1749129592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4724160ff0 0x7f472406b780 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f47240b2a30 tx=0x7f472002b910 comp rx=0 tx=0).stop 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 -- 172.21.15.70:0/1749129592 shutdown_connections 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 --2- 172.21.15.70:0/1749129592 >> v2:172.21.15.155:3300/0 conn(0x7f4724161ae0 0x7f47240ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 --2- 172.21.15.70:0/1749129592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4724160ff0 0x7f472406b780 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 -- 172.21.15.70:0/1749129592 >> 172.21.15.70:0/1749129592 conn(0x7f4724069850 msgr2=0x7f4724069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 -- 172.21.15.70:0/1749129592 shutdown_connections 2023-08-25T09:46:42.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.864+0000 7f472b339640 1 -- 172.21.15.70:0/1749129592 wait complete. 2023-08-25T09:46:42.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.866+0000 7f472b339640 1 Processor -- start 2023-08-25T09:46:42.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.866+0000 7f472b339640 1 -- start start 2023-08-25T09:46:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f472b339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 0x7f47240cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f472b339640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724160ff0 0x7f472414f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f472b339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724161ae0 0x7f47240af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f472b339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240cbf90 con 0x7f47240ce5f0 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f472b339640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240cd2e0 con 0x7f4724160ff0 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f472b339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240cc240 con 0x7f4724161ae0 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f47290d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 0x7f47240cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f47290d5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 0x7f47240cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43630/0 (socket says 172.21.15.70:43630) 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.867+0000 7f47290d5640 1 -- 172.21.15.70:0/742068351 learned_addr learned my addr 172.21.15.70:0/742068351 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f47290d5640 1 -- 172.21.15.70:0/742068351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724161ae0 msgr2=0x7f47240af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f47290d5640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724161ae0 0x7f47240af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f47290d5640 1 -- 172.21.15.70:0/742068351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724160ff0 msgr2=0x7f472414f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f47290d5640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724160ff0 0x7f472414f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:42.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f47290d5640 1 -- 172.21.15.70:0/742068351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47200094d0 con 0x7f47240ce5f0 2023-08-25T09:46:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f47290d5640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 0x7f47240cba50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4720000c00 tx=0x7f47200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472003ad10 con 0x7f47240ce5f0 2023-08-25T09:46:42.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472003ae70 con 0x7f47240ce5f0 2023-08-25T09:46:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4720044610 con 0x7f47240ce5f0 2023-08-25T09:46:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.868+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47240cc520 con 0x7f47240ce5f0 2023-08-25T09:46:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.869+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47240ccae0 con 0x7f47240ce5f0 2023-08-25T09:46:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.870+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472404e420 con 0x7f47240ce5f0 2023-08-25T09:46:42.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.870+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f472004d450 con 0x7f47240ce5f0 2023-08-25T09:46:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.871+0000 7f4719ffb640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f470806a050 0x7f470806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.871+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f472002d040 con 0x7f47240ce5f0 2023-08-25T09:46:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.872+0000 7f47288d4640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f470806a050 0x7f470806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:42.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.873+0000 7f47288d4640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f470806a050 0x7f470806c510 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f472406f250 tx=0x7f4714007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:42.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:42.876+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f47200763f0 con 0x7f47240ce5f0 2023-08-25T09:46:43.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.103+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f472406c8b0 con 0x7f47240ce5f0 2023-08-25T09:46:43.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.104+0000 7f4719ffb640 1 -- 172.21.15.70:0/742068351 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7f4720079a10 con 0x7f47240ce5f0 2023-08-25T09:46:43.105 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:43.106 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:43.106 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:43.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f470806a050 msgr2=0x7f470806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f470806a050 0x7f470806c510 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f472406f250 tx=0x7f4714007990 comp rx=0 tx=0).stop 2023-08-25T09:46:43.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 msgr2=0x7f47240cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 0x7f47240cba50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f4720000c00 tx=0x7f47200046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 shutdown_connections 2023-08-25T09:46:43.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f470806a050 0x7f470806c510 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4724161ae0 0x7f47240af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4724160ff0 0x7f472414f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 --2- 172.21.15.70:0/742068351 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f47240ce5f0 0x7f47240cba50 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.106+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 >> 172.21.15.70:0/742068351 conn(0x7f4724069850 msgr2=0x7f4724070330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:43.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.107+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 shutdown_connections 2023-08-25T09:46:43.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.107+0000 7f472b339640 1 -- 172.21.15.70:0/742068351 wait complete. 2023-08-25T09:46:43.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:43.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.327+0000 7f048cbb5640 1 Processor -- start 2023-08-25T09:46:43.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.327+0000 7f048cbb5640 1 -- start start 2023-08-25T09:46:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f048cbb5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f048815aae0 0x7f04880cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f048cbb5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0488071500 0x7f04880b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f048cbb5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04880b1860 con 0x7f04880b1440 2023-08-25T09:46:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f048cbb5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880ca5b0 con 0x7f0488071500 2023-08-25T09:46:43.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f048cbb5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04880cb5b0 con 0x7f048815aae0 2023-08-25T09:46:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f0485d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0488071500 0x7f04880b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f0485d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0488071500 0x7f04880b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43642/0 (socket says 172.21.15.70:43642) 2023-08-25T09:46:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f0485d9b640 1 -- 172.21.15.70:0/2427596260 learned_addr learned my addr 172.21.15.70:0/2427596260 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.329+0000 7f048659c640 1 --2- 172.21.15.70:0/2427596260 >> v2:172.21.15.155:3300/0 conn(0x7f048815aae0 0x7f04880cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.330+0000 7f0485d9b640 1 -- 172.21.15.70:0/2427596260 >> v2:172.21.15.155:3300/0 conn(0x7f048815aae0 msgr2=0x7f04880cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.330+0000 7f0485d9b640 1 --2- 172.21.15.70:0/2427596260 >> v2:172.21.15.155:3300/0 conn(0x7f048815aae0 0x7f04880cacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.330+0000 7f0485d9b640 1 -- 172.21.15.70:0/2427596260 >> v1:172.21.15.155:6789/0 conn(0x7f04880b1440 legacy=0x7f04880cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:43.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.330+0000 7f0485d9b640 1 -- 172.21.15.70:0/2427596260 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048815e4f0 con 0x7f0488071500 2023-08-25T09:46:43.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.331+0000 7f0485d9b640 1 --2- 172.21.15.70:0/2427596260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0488071500 0x7f04880b0f00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0488066510 tx=0x7f047002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a16fe9c8f28136 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:43.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.331+0000 7f048559a640 1 -- 172.21.15.70:0/2427596260 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470031870 con 0x7f0488071500 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.331+0000 7f048559a640 1 -- 172.21.15.70:0/2427596260 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04700319d0 con 0x7f0488071500 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.331+0000 7f048559a640 1 -- 172.21.15.70:0/2427596260 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470031d60 con 0x7f0488071500 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 -- 172.21.15.70:0/2427596260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0488071500 msgr2=0x7f04880b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 --2- 172.21.15.70:0/2427596260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0488071500 0x7f04880b0f00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f0488066510 tx=0x7f047002b910 comp rx=0 tx=0).stop 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 -- 172.21.15.70:0/2427596260 shutdown_connections 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 --2- 172.21.15.70:0/2427596260 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0488071500 0x7f04880b0f00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 --2- 172.21.15.70:0/2427596260 >> v2:172.21.15.155:3300/0 conn(0x7f048815aae0 0x7f04880cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 -- 172.21.15.70:0/2427596260 >> 172.21.15.70:0/2427596260 conn(0x7f0488069c70 msgr2=0x7f048806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:43.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 -- 172.21.15.70:0/2427596260 shutdown_connections 2023-08-25T09:46:43.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.332+0000 7f048cbb5640 1 -- 172.21.15.70:0/2427596260 wait complete. 2023-08-25T09:46:43.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.334+0000 7f048cbb5640 1 Processor -- start 2023-08-25T09:46:43.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.334+0000 7f048cbb5640 1 -- start start 2023-08-25T09:46:43.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048cbb5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0488071500 0x7f04881f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048cbb5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 0x7f04880aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048cbb5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048815aae0 0x7f04880acf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048cbb5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b2e70 con 0x7f04880b1440 2023-08-25T09:46:43.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048cbb5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b36d0 con 0x7f048815aae0 2023-08-25T09:46:43.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048cbb5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04880b2aa0 con 0x7f0488071500 2023-08-25T09:46:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 0x7f04880aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f0485d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 0x7f04880aca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43654/0 (socket says 172.21.15.70:43654) 2023-08-25T09:46:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f0485d9b640 1 -- 172.21.15.70:0/846135123 learned_addr learned my addr 172.21.15.70:0/846135123 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f048659c640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0488071500 0x7f04881f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.335+0000 7f0485d9b640 1 -- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0488071500 msgr2=0x7f04881f1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0485d9b640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0488071500 0x7f04881f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0485d9b640 1 -- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048815aae0 msgr2=0x7f04880acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0486d9d640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048815aae0 0x7f04880acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0485d9b640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048815aae0 0x7f04880acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0485d9b640 1 -- 172.21.15.70:0/846135123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04700094d0 con 0x7f04880b1440 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0486d9d640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048815aae0 0x7f04880acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f0485d9b640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 0x7f04880aca10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f047002b950 tx=0x7f04700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047003abe0 con 0x7f04880b1440 2023-08-25T09:46:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047003ad40 con 0x7f04880b1440 2023-08-25T09:46:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0470044610 con 0x7f04880b1440 2023-08-25T09:46:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04880b3080 con 0x7f04880b1440 2023-08-25T09:46:43.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.336+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04881f2110 con 0x7f04880b1440 2023-08-25T09:46:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.337+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04880b3860 con 0x7f04880b1440 2023-08-25T09:46:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.338+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0470044770 con 0x7f04880b1440 2023-08-25T09:46:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.339+0000 7f047effd640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f046806a050 0x7f046806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.339+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f047002d040 con 0x7f04880b1440 2023-08-25T09:46:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.340+0000 7f048659c640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f046806a050 0x7f046806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.340+0000 7f048659c640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f046806a050 0x7f046806c510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0488070ee0 tx=0x7f047800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.343+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0470076330 con 0x7f04880b1440 2023-08-25T09:46:43.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.571+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f048804e420 con 0x7f04880b1440 2023-08-25T09:46:43.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.572+0000 7f047effd640 1 -- 172.21.15.70:0/846135123 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4871 (secure 0 0 0) 0x7f0470035020 con 0x7f04880b1440 2023-08-25T09:46:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:43.574 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":57,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:13.310400+0000","modified":"2023-08-25T09:46:16.886817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7746},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/59115020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":59115020},{"type":"v1","addr":"172.21.15.70:6836","nonce":59115020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-08-25T09:46:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 57 2023-08-25T09:46:43.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f046806a050 msgr2=0x7f046806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f046806a050 0x7f046806c510 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0488070ee0 tx=0x7f047800b040 comp rx=0 tx=0).stop 2023-08-25T09:46:43.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 msgr2=0x7f04880aca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 0x7f04880aca10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f047002b950 tx=0x7f04700046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 shutdown_connections 2023-08-25T09:46:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f046806a050 0x7f046806c510 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f048815aae0 0x7f04880acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04880b1440 0x7f04880aca10 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 --2- 172.21.15.70:0/846135123 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0488071500 0x7f04881f1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 >> 172.21.15.70:0/846135123 conn(0x7f0488069c70 msgr2=0x7f04880cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 shutdown_connections 2023-08-25T09:46:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.575+0000 7f048cbb5640 1 -- 172.21.15.70:0/846135123 wait complete. 2023-08-25T09:46:43.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:43.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.809+0000 7f080e5ab640 1 Processor -- start 2023-08-25T09:46:43.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.809+0000 7f080e5ab640 1 -- start start 2023-08-25T09:46:43.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f080e5ab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f08080ce5b0 0x7f08080ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f080e5ab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08080b2850 0x7f080806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f080e5ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08080ceed0 con 0x7f080815aa10 2023-08-25T09:46:43.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f080e5ab640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0808159d10 con 0x7f08080b2850 2023-08-25T09:46:43.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f080e5ab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080815d3f0 con 0x7f08080ce5b0 2023-08-25T09:46:43.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f080cb48640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08080b2850 0x7f080806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f08077fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f080815aa10 0x7f08080b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42586/0 (socket says 172.21.15.70:42586) 2023-08-25T09:46:43.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.811+0000 7f08077fe640 1 -- 172.21.15.70:0/871589091 learned_addr learned my addr 172.21.15.70:0/871589091 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:43.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f0807fff640 1 --2- 172.21.15.70:0/871589091 >> v2:172.21.15.155:3300/0 conn(0x7f08080ce5b0 0x7f08080ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f080cb48640 1 -- 172.21.15.70:0/871589091 >> v2:172.21.15.155:3300/0 conn(0x7f08080ce5b0 msgr2=0x7f08080ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:43.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f080cb48640 1 --2- 172.21.15.70:0/871589091 >> v2:172.21.15.155:3300/0 conn(0x7f08080ce5b0 0x7f08080ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f080cb48640 1 -- 172.21.15.70:0/871589091 >> v1:172.21.15.70:6789/0 conn(0x7f080815aa10 legacy=0x7f08080b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f080cb48640 1 -- 172.21.15.70:0/871589091 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08080b2d20 con 0x7f08080b2850 2023-08-25T09:46:43.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f0806ffd640 1 -- 172.21.15.70:0/871589091 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222544022 0 0) 0x7f08080ceed0 con 0x7f080815aa10 2023-08-25T09:46:43.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.812+0000 7f0807fff640 1 --2- 172.21.15.70:0/871589091 >> v2:172.21.15.155:3300/0 conn(0x7f08080ce5b0 0x7f08080ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:43.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.813+0000 7f080cb48640 1 --2- 172.21.15.70:0/871589091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08080b2850 0x7f080806f9f0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f080806f470 tx=0x7f080002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9090970f471af8d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:43.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.813+0000 7f0806ffd640 1 -- 172.21.15.70:0/871589091 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080002cbf0 con 0x7f08080b2850 2023-08-25T09:46:43.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.813+0000 7f0806ffd640 1 -- 172.21.15.70:0/871589091 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080002cd50 con 0x7f08080b2850 2023-08-25T09:46:43.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.813+0000 7f0806ffd640 1 -- 172.21.15.70:0/871589091 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800034660 con 0x7f08080b2850 2023-08-25T09:46:43.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 -- 172.21.15.70:0/871589091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08080b2850 msgr2=0x7f080806f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 --2- 172.21.15.70:0/871589091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08080b2850 0x7f080806f9f0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f080806f470 tx=0x7f080002be40 comp rx=0 tx=0).stop 2023-08-25T09:46:43.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 -- 172.21.15.70:0/871589091 shutdown_connections 2023-08-25T09:46:43.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 --2- 172.21.15.70:0/871589091 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08080b2850 0x7f080806f9f0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 --2- 172.21.15.70:0/871589091 >> v2:172.21.15.155:3300/0 conn(0x7f08080ce5b0 0x7f08080ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 -- 172.21.15.70:0/871589091 >> 172.21.15.70:0/871589091 conn(0x7f0808069850 msgr2=0x7f0808069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:43.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 -- 172.21.15.70:0/871589091 shutdown_connections 2023-08-25T09:46:43.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.814+0000 7f080e5ab640 1 -- 172.21.15.70:0/871589091 wait complete. 2023-08-25T09:46:43.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.816+0000 7f080e5ab640 1 Processor -- start 2023-08-25T09:46:43.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.816+0000 7f080e5ab640 1 -- start start 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080e5ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08080b2850 0x7f08081557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080e5ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08080ce5b0 0x7f08081572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080e5ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 0x7f08081f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080e5ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0808155310 con 0x7f080815aa10 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080e5ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08081f2c00 con 0x7f08080b2850 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080e5ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08081f2ee0 con 0x7f08080ce5b0 2023-08-25T09:46:43.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080cb48640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 0x7f08081f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080cb48640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 0x7f08081f2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43662/0 (socket says 172.21.15.70:43662) 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.817+0000 7f080cb48640 1 -- 172.21.15.70:0/367489886 learned_addr learned my addr 172.21.15.70:0/367489886 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f08077fe640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08080ce5b0 0x7f08081572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080cb48640 1 -- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08080ce5b0 msgr2=0x7f08081572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080cb48640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08080ce5b0 0x7f08081572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080cb48640 1 -- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08080b2850 msgr2=0x7f08081557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:43.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f0807fff640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08080b2850 0x7f08081557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080cb48640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08080b2850 0x7f08081557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080cb48640 1 -- 172.21.15.70:0/367489886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009a70 con 0x7f080815aa10 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f0807fff640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08080b2850 0x7f08081557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080cb48640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 0x7f08081f2130 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0800009e80 tx=0x7f0800004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800040650 con 0x7f080815aa10 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000407d0 con 0x7f080815aa10 2023-08-25T09:46:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0800040b80 con 0x7f080815aa10 2023-08-25T09:46:43.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.818+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0808154fa0 con 0x7f080815aa10 2023-08-25T09:46:43.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.819+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0808155ce0 con 0x7f080815aa10 2023-08-25T09:46:43.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.820+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08081f3100 con 0x7f080815aa10 2023-08-25T09:46:43.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.821+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0800040ce0 con 0x7f080815aa10 2023-08-25T09:46:43.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.822+0000 7f0804ff9640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f07f406a050 0x7f07f406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:43.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.822+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f08000ab5a0 con 0x7f080815aa10 2023-08-25T09:46:43.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.822+0000 7f0807fff640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f07f406a050 0x7f07f406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:43.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.823+0000 7f0807fff640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f07f406a050 0x7f07f406c510 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f08081efcc0 tx=0x7f07f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:43.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:43.826+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0800076c10 con 0x7f080815aa10 2023-08-25T09:46:44.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.014+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08081554c0 con 0x7f080815aa10 2023-08-25T09:46:44.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.018+0000 7f0804ff9640 1 -- 172.21.15.70:0/367489886 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+18581 (secure 0 0 0) 0x7f08081554c0 con 0x7f080815aa10 2023-08-25T09:46:44.020 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:44.020 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":138,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:41.631656+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:11.556010+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:12.318797+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.a","mappings":[{"from":4,"to":3}]},{"pgid":"16.11","mappings":[{"from":4,"to":6}]},{"pgid":"16.1d","mappings":[{"from":4,"to":3}]},{"pgid":"16.1e","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:44.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f07f406a050 msgr2=0x7f07f406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f07f406a050 0x7f07f406c510 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f08081efcc0 tx=0x7f07f8008040 comp rx=0 tx=0).stop 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 msgr2=0x7f08081f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 0x7f08081f2130 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f0800009e80 tx=0x7f0800004a40 comp rx=0 tx=0).stop 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 shutdown_connections 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f07f406a050 0x7f07f406c510 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f080815aa10 0x7f08081f2130 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08080ce5b0 0x7f08081572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.020+0000 7f080e5ab640 1 --2- 172.21.15.70:0/367489886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08080b2850 0x7f08081557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.021+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 >> 172.21.15.70:0/367489886 conn(0x7f0808069850 msgr2=0x7f08080b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:44.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.021+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 shutdown_connections 2023-08-25T09:46:44.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.021+0000 7f080e5ab640 1 -- 172.21.15.70:0/367489886 wait complete. 2023-08-25T09:46:44.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:46:44.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.255+0000 7f17e9c83640 1 Processor -- start 2023-08-25T09:46:44.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.256+0000 7f17e9c83640 1 -- start start 2023-08-25T09:46:44.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e9c83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f17e4072820 0x7f17e4072c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e9c83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17e415ce60 0x7f17e4069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e9c83640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e406b960 con 0x7f17e4069d90 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e9c83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e406bfe0 con 0x7f17e4072820 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e9c83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e40b2170 con 0x7f17e415ce60 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f17e4072820 0x7f17e4072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f17e4072820 0x7f17e4072c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43670/0 (socket says 172.21.15.70:43670) 2023-08-25T09:46:44.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.257+0000 7f17e37fe640 1 -- 172.21.15.70:0/3607620010 learned_addr learned my addr 172.21.15.70:0/3607620010 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:44.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.258+0000 7f17e2ffd640 1 --2- 172.21.15.70:0/3607620010 >> v2:172.21.15.155:3300/0 conn(0x7f17e415ce60 0x7f17e4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.258+0000 7f17e37fe640 1 -- 172.21.15.70:0/3607620010 >> v2:172.21.15.155:3300/0 conn(0x7f17e415ce60 msgr2=0x7f17e4069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.258+0000 7f17e37fe640 1 --2- 172.21.15.70:0/3607620010 >> v2:172.21.15.155:3300/0 conn(0x7f17e415ce60 0x7f17e4069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.258+0000 7f17e37fe640 1 -- 172.21.15.70:0/3607620010 >> v1:172.21.15.155:6789/0 conn(0x7f17e4069d90 legacy=0x7f17e406c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.258+0000 7f17e37fe640 1 -- 172.21.15.70:0/3607620010 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e4073140 con 0x7f17e4072820 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.258+0000 7f17e2ffd640 1 --2- 172.21.15.70:0/3607620010 >> v2:172.21.15.155:3300/0 conn(0x7f17e415ce60 0x7f17e4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.259+0000 7f17e37fe640 1 --2- 172.21.15.70:0/3607620010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4072820 0x7f17e4072c00 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b23c0 tx=0x7f17d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5994bec1029fd9d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:44.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.259+0000 7f17e27fc640 1 -- 172.21.15.70:0/3607620010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d402ca80 con 0x7f17e4072820 2023-08-25T09:46:44.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.259+0000 7f17e27fc640 1 -- 172.21.15.70:0/3607620010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d402cbe0 con 0x7f17e4072820 2023-08-25T09:46:44.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.259+0000 7f17e27fc640 1 -- 172.21.15.70:0/3607620010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d40343f0 con 0x7f17e4072820 2023-08-25T09:46:44.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 -- 172.21.15.70:0/3607620010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4072820 msgr2=0x7f17e4072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3607620010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4072820 0x7f17e4072c00 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b23c0 tx=0x7f17d402bee0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 -- 172.21.15.70:0/3607620010 shutdown_connections 2023-08-25T09:46:44.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3607620010 >> v2:172.21.15.155:3300/0 conn(0x7f17e415ce60 0x7f17e4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3607620010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4072820 0x7f17e4072c00 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 -- 172.21.15.70:0/3607620010 >> 172.21.15.70:0/3607620010 conn(0x7f17e40b12c0 msgr2=0x7f17e40b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 -- 172.21.15.70:0/3607620010 shutdown_connections 2023-08-25T09:46:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.260+0000 7f17e9c83640 1 -- 172.21.15.70:0/3607620010 wait complete. 2023-08-25T09:46:44.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.262+0000 7f17e9c83640 1 Processor -- start 2023-08-25T09:46:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.262+0000 7f17e9c83640 1 -- start start 2023-08-25T09:46:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e9c83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 0x7f17e40c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e9c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17e4072820 0x7f17e40ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e9c83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e40c8d50 0x7f17e40ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e9c83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c9870 con 0x7f17e4069d90 2023-08-25T09:46:44.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e9c83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c92b0 con 0x7f17e40c8d50 2023-08-25T09:46:44.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e9c83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e40c9590 con 0x7f17e4072820 2023-08-25T09:46:44.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 0x7f17e40c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 0x7f17e40c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43676/0 (socket says 172.21.15.70:43676) 2023-08-25T09:46:44.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.263+0000 7f17e37fe640 1 -- 172.21.15.70:0/3750543742 learned_addr learned my addr 172.21.15.70:0/3750543742 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:44.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e37fe640 1 -- 172.21.15.70:0/3750543742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17e4072820 msgr2=0x7f17e40ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e37fe640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17e4072820 0x7f17e40ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e37fe640 1 -- 172.21.15.70:0/3750543742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e40c8d50 msgr2=0x7f17e40ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e37fe640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e40c8d50 0x7f17e40ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e37fe640 1 -- 172.21.15.70:0/3750543742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4009b30 con 0x7f17e4069d90 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e37fe640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 0x7f17e40c8810 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f17d4000f80 tx=0x7f17d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d403e450 con 0x7f17e4069d90 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d403e5f0 con 0x7f17e4069d90 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d403e9a0 con 0x7f17e4069d90 2023-08-25T09:46:44.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.264+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e40b5af0 con 0x7f17e4069d90 2023-08-25T09:46:44.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.265+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e4154f60 con 0x7f17e4069d90 2023-08-25T09:46:44.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.266+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e40b5950 con 0x7f17e4069d90 2023-08-25T09:46:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.266+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17d402cd50 con 0x7f17e4069d90 2023-08-25T09:46:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.267+0000 7f17dbfff640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17c406a080 0x7f17c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.268+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6178+0+0 (secure 0 0 0) 0x7f17d40aaf90 con 0x7f17e4069d90 2023-08-25T09:46:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.268+0000 7f17e2ffd640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17c406a080 0x7f17c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.269+0000 7f17e2ffd640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17c406a080 0x7f17c406c540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f17cc006fd0 tx=0x7f17cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:44.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.272+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17e40b5950 con 0x7f17e4069d90 2023-08-25T09:46:44.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.491+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f17e40b5950 con 0x7f17e4069d90 2023-08-25T09:46:44.494 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:46:44.493+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:46:44.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.501+0000 7f17dbfff640 1 -- 172.21.15.70:0/3750543742 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f17e40b5950 con 0x7f17e4069d90 2023-08-25T09:46:44.504 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:46:44.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.504+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17c406a080 msgr2=0x7f17c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.504+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17c406a080 0x7f17c406c540 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f17cc006fd0 tx=0x7f17cc008040 comp rx=0 tx=0).stop 2023-08-25T09:46:44.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.504+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 msgr2=0x7f17e40c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.504+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 0x7f17e40c8810 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f17d4000f80 tx=0x7f17d4004a40 comp rx=0 tx=0).stop 2023-08-25T09:46:44.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.504+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 shutdown_connections 2023-08-25T09:46:44.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.504+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17c406a080 0x7f17c406c540 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.505+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17e40c8d50 0x7f17e40ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.505+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17e4072820 0x7f17e40ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.505+0000 7f17e9c83640 1 --2- 172.21.15.70:0/3750543742 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17e4069d90 0x7f17e40c8810 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.505+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 >> 172.21.15.70:0/3750543742 conn(0x7f17e40b12c0 msgr2=0x7f17e40b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:44.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.505+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 shutdown_connections 2023-08-25T09:46:44.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.505+0000 7f17e9c83640 1 -- 172.21.15.70:0/3750543742 wait complete. 2023-08-25T09:46:44.509 INFO:tasks.ceph.mds.a.smithi070.stderr: -3556> 2023-08-25T09:46:18.368+0000 7f1880fa5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:46:44.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:46:44.568 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:46:44.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.721+0000 7f08c7f4e640 1 Processor -- start 2023-08-25T09:46:44.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.721+0000 7f08c7f4e640 1 -- start start 2023-08-25T09:46:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.722+0000 7f08c7f4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c006ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.722+0000 7f08c7f4e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08c015b030 0x7f08c015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.722+0000 7f08c7f4e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c006bfb0 con 0x7f08c015cdc0 2023-08-25T09:46:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.722+0000 7f08c7f4e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c0159d80 con 0x7f08c015b030 2023-08-25T09:46:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.722+0000 7f08c7f4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c015d1a0 con 0x7f08c006b610 2023-08-25T09:46:44.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08c015b030 0x7f08c015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f08c015b030 0x7f08c015b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43704/0 (socket says 172.21.15.70:43704) 2023-08-25T09:46:44.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 -- 172.21.15.70:0/844081719 learned_addr learned my addr 172.21.15.70:0/844081719 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:44.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c54e9640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c006ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 -- 172.21.15.70:0/844081719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 msgr2=0x7f08c006ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:44.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c006ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 -- 172.21.15.70:0/844081719 >> v1:172.21.15.155:6789/0 conn(0x7f08c015cdc0 legacy=0x7f08c00b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:44.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c64eb640 1 -- 172.21.15.70:0/844081719 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00b3710 con 0x7f08c015b030 2023-08-25T09:46:44.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.723+0000 7f08c54e9640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c006ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:44.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.724+0000 7f08c64eb640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c015b4a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009e50 tx=0x7f08b802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a09fae3a25e6e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:44.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.724+0000 7f08c4ce8640 1 -- 172.21.15.70:0/844081719 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b802ca80 con 0x7f08c015b030 2023-08-25T09:46:44.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.724+0000 7f08c4ce8640 1 -- 172.21.15.70:0/844081719 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b802cbe0 con 0x7f08c015b030 2023-08-25T09:46:44.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.724+0000 7f08c4ce8640 1 -- 172.21.15.70:0/844081719 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80343f0 con 0x7f08c015b030 2023-08-25T09:46:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 -- 172.21.15.70:0/844081719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 msgr2=0x7f08c015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c015b4a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009e50 tx=0x7f08b802bee0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 -- 172.21.15.70:0/844081719 shutdown_connections 2023-08-25T09:46:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c015b4a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/844081719 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c006ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 -- 172.21.15.70:0/844081719 >> 172.21.15.70:0/844081719 conn(0x7f08c0069850 msgr2=0x7f08c0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 -- 172.21.15.70:0/844081719 shutdown_connections 2023-08-25T09:46:44.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.725+0000 7f08c7f4e640 1 -- 172.21.15.70:0/844081719 wait complete. 2023-08-25T09:46:44.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.727+0000 7f08c7f4e640 1 Processor -- start 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.727+0000 7f08c7f4e640 1 -- start start 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c7f4e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c0156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c7f4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c00acb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c7f4e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08c015cdc0 0x7f08c0157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c7f4e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01f3200 con 0x7f08c015b030 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c7f4e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01f2680 con 0x7f08c015cdc0 2023-08-25T09:46:44.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c7f4e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c01f2960 con 0x7f08c006b610 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c54e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c00acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c54e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c00acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43710/0 (socket says 172.21.15.70:43710) 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.728+0000 7f08c54e9640 1 -- 172.21.15.70:0/3051710229 learned_addr learned my addr 172.21.15.70:0/3051710229 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c5cea640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c0156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c54e9640 1 -- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 msgr2=0x7f08c0156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c54e9640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c0156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c54e9640 1 -- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08c015cdc0 msgr2=0x7f08c0157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c64eb640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08c015cdc0 0x7f08c0157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c54e9640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08c015cdc0 0x7f08c0157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c54e9640 1 -- 172.21.15.70:0/3051710229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009b30 con 0x7f08c015b030 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c54e9640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c00acb10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c7f0 tx=0x7f08b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4007820 con 0x7f08c015b030 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b4007980 con 0x7f08c015b030 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c01f2c40 con 0x7f08c015b030 2023-08-25T09:46:44.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b4007cf0 con 0x7f08c015b030 2023-08-25T09:46:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.729+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c0157480 con 0x7f08c015b030 2023-08-25T09:46:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.730+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c004e420 con 0x7f08c015b030 2023-08-25T09:46:44.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.731+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08b40113f0 con 0x7f08c015b030 2023-08-25T09:46:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.733+0000 7f08b27fc640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08a406a050 0x7f08a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.733+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f08b4088a00 con 0x7f08c015b030 2023-08-25T09:46:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.733+0000 7f08c5cea640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08a406a050 0x7f08a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:44.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.734+0000 7f08c5cea640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08a406a050 0x7f08a406c510 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08bc003d90 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:44.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.737+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08b4053f80 con 0x7f08c015b030 2023-08-25T09:46:44.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:44.953+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f08c0151d20 con 0x7f08c015b030 2023-08-25T09:46:45.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.505+0000 7f08b27fc640 1 -- 172.21.15.70:0/3051710229 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7f08b40575a0 con 0x7f08c015b030 2023-08-25T09:46:45.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08a406a050 msgr2=0x7f08a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:45.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08a406a050 0x7f08a406c510 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08bc003d90 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2023-08-25T09:46:45.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 msgr2=0x7f08c00acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:45.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c00acb10 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c7f0 tx=0x7f08b400ccb0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 shutdown_connections 2023-08-25T09:46:45.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08a406a050 0x7f08a406c510 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f08c015cdc0 0x7f08c0157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f08c015b030 0x7f08c00acb10 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 --2- 172.21.15.70:0/3051710229 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f08c006b610 0x7f08c0156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.509+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 >> 172.21.15.70:0/3051710229 conn(0x7f08c0069850 msgr2=0x7f08c00ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:45.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.510+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 shutdown_connections 2023-08-25T09:46:45.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.510+0000 7f08c7f4e640 1 -- 172.21.15.70:0/3051710229 wait complete. 2023-08-25T09:46:45.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:46:45.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.796+0000 7f80c9a26640 1 Processor -- start 2023-08-25T09:46:45.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.797+0000 7f80c9a26640 1 -- start start 2023-08-25T09:46:45.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c9a26640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f80c415c950 0x7f80c40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:45.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c9a26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c406b180 0x7f80c41606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:45.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c9a26640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c40b2300 con 0x7f80c415cd30 2023-08-25T09:46:45.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c9a26640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c40b16f0 con 0x7f80c415c950 2023-08-25T09:46:45.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c9a26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c40b19d0 con 0x7f80c406b180 2023-08-25T09:46:45.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f80c415c950 0x7f80c40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:45.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80bbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f80c415cd30 0x7f80c406aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42604/0 (socket says 172.21.15.70:42604) 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80bbfff640 1 -- 172.21.15.70:0/2213640209 learned_addr learned my addr 172.21.15.70:0/2213640209 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c37fe640 1 --2- 172.21.15.70:0/2213640209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c406b180 0x7f80c41606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c2ffd640 1 -- 172.21.15.70:0/2213640209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c406b180 msgr2=0x7f80c41606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c2ffd640 1 --2- 172.21.15.70:0/2213640209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c406b180 0x7f80c41606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.799+0000 7f80c2ffd640 1 -- 172.21.15.70:0/2213640209 >> v1:172.21.15.70:6789/0 conn(0x7f80c415cd30 legacy=0x7f80c406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.800+0000 7f80c2ffd640 1 -- 172.21.15.70:0/2213640209 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40b1cb0 con 0x7f80c415c950 2023-08-25T09:46:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.800+0000 7f80c27fc640 1 -- 172.21.15.70:0/2213640209 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824079555 0 0) 0x7f80c40b2300 con 0x7f80c415cd30 2023-08-25T09:46:45.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.801+0000 7f80c2ffd640 1 --2- 172.21.15.70:0/2213640209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c415c950 0x7f80c40b4190 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0af18794c6d1c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:45.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.801+0000 7f80c27fc640 1 -- 172.21.15.70:0/2213640209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b402d040 con 0x7f80c415c950 2023-08-25T09:46:45.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.801+0000 7f80c27fc640 1 -- 172.21.15.70:0/2213640209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4031810 con 0x7f80c415c950 2023-08-25T09:46:45.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.801+0000 7f80c27fc640 1 -- 172.21.15.70:0/2213640209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4031ba0 con 0x7f80c415c950 2023-08-25T09:46:45.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.801+0000 7f80c9a26640 1 -- 172.21.15.70:0/2213640209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c415c950 msgr2=0x7f80c40b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:45.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.801+0000 7f80c9a26640 1 --2- 172.21.15.70:0/2213640209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c415c950 0x7f80c40b4190 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f80b40097a0 tx=0x7f80b402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.802+0000 7f80c9a26640 1 -- 172.21.15.70:0/2213640209 shutdown_connections 2023-08-25T09:46:45.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.802+0000 7f80c9a26640 1 --2- 172.21.15.70:0/2213640209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c406b180 0x7f80c41606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.802+0000 7f80c9a26640 1 --2- 172.21.15.70:0/2213640209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c415c950 0x7f80c40b4190 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.802+0000 7f80c9a26640 1 -- 172.21.15.70:0/2213640209 >> 172.21.15.70:0/2213640209 conn(0x7f80c4069c70 msgr2=0x7f80c406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:45.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.802+0000 7f80c9a26640 1 -- 172.21.15.70:0/2213640209 shutdown_connections 2023-08-25T09:46:45.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.802+0000 7f80c9a26640 1 -- 172.21.15.70:0/2213640209 wait complete. 2023-08-25T09:46:45.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.803+0000 7f80c9a26640 1 Processor -- start 2023-08-25T09:46:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.804+0000 7f80c9a26640 1 -- start start 2023-08-25T09:46:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c9a26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 0x7f80c415baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c9a26640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c415c950 0x7f80c40ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c9a26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c415cd30 0x7f80c4140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c9a26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4159480 con 0x7f80c406b180 2023-08-25T09:46:45.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c9a26640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c4154600 con 0x7f80c415c950 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c9a26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c41591a0 con 0x7f80c415cd30 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 0x7f80c415baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 0x7f80c415baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43734/0 (socket says 172.21.15.70:43734) 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 -- 172.21.15.70:0/168598356 learned_addr learned my addr 172.21.15.70:0/168598356 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 -- 172.21.15.70:0/168598356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c415cd30 msgr2=0x7f80c4140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c415cd30 0x7f80c4140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 -- 172.21.15.70:0/168598356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c415c950 msgr2=0x7f80c40ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c415c950 0x7f80c40ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.805+0000 7f80c2ffd640 1 -- 172.21.15.70:0/168598356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4009400 con 0x7f80c406b180 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.806+0000 7f80c2ffd640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 0x7f80c415baf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f80b40098c0 tx=0x7f80b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.806+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b403ad60 con 0x7f80c406b180 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.806+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4044400 con 0x7f80c406b180 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.806+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40447b0 con 0x7f80c406b180 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.806+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c4154810 con 0x7f80c406b180 2023-08-25T09:46:45.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.806+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c4154dd0 con 0x7f80c406b180 2023-08-25T09:46:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.807+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c404e420 con 0x7f80c406b180 2023-08-25T09:46:45.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.808+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f80b404d440 con 0x7f80c406b180 2023-08-25T09:46:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.809+0000 7f80c0ff9640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f80a406a050 0x7f80a406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.809+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6264+0+0 (secure 0 0 0) 0x7f80b40aab80 con 0x7f80c406b180 2023-08-25T09:46:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.809+0000 7f80bbfff640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f80a406a050 0x7f80a406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:45.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.810+0000 7f80bbfff640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f80a406a050 0x7f80a406c510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f80ac007fd0 tx=0x7f80ac007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:45.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:45.813+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f80b40761a0 con 0x7f80c406b180 2023-08-25T09:46:46.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.002+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80c40b2ce0 con 0x7f80c406b180 2023-08-25T09:46:46.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:46:46.152 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:46:46.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.694+0000 7f80c0ff9640 1 -- 172.21.15.70:0/168598356 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v140) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80b40797c0 con 0x7f80c406b180 2023-08-25T09:46:46.697 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:46:46.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.698+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f80a406a050 msgr2=0x7f80a406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:46.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.698+0000 7f80c9a26640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f80a406a050 0x7f80a406c510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f80ac007fd0 tx=0x7f80ac007960 comp rx=0 tx=0).stop 2023-08-25T09:46:46.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.698+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 msgr2=0x7f80c415baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:46.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.698+0000 7f80c9a26640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 0x7f80c415baf0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f80b40098c0 tx=0x7f80b40046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 shutdown_connections 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f80a406a050 0x7f80a406c510 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80c415cd30 0x7f80c4140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80c415c950 0x7f80c40ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 --2- 172.21.15.70:0/168598356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80c406b180 0x7f80c415baf0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 >> 172.21.15.70:0/168598356 conn(0x7f80c4069c70 msgr2=0x7f80c4070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.699+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 shutdown_connections 2023-08-25T09:46:46.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.700+0000 7f80c9a26640 1 -- 172.21.15.70:0/168598356 wait complete. 2023-08-25T09:46:46.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:46:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.922+0000 7fdf96dab640 1 Processor -- start 2023-08-25T09:46:46.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.922+0000 7fdf96dab640 1 -- start start 2023-08-25T09:46:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf96dab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 0x7fdf900cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf96dab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdf900ca9f0 0x7fdf9015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf96dab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9015e6f0 con 0x7fdf900cd890 2023-08-25T09:46:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf96dab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900caf30 con 0x7fdf900ca9f0 2023-08-25T09:46:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf96dab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900cdd10 con 0x7fdf90155120 2023-08-25T09:46:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf8ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdf900cd890 0x7fdf900ca2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42610/0 (socket says 172.21.15.70:42610) 2023-08-25T09:46:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.924+0000 7fdf8ffff640 1 -- 172.21.15.70:0/1410806265 learned_addr learned my addr 172.21.15.70:0/1410806265 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:46.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf94b47640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 0x7fdf900cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf95348640 1 --2- 172.21.15.70:0/1410806265 >> v2:172.21.15.155:3300/0 conn(0x7fdf900ca9f0 0x7fdf9015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838980368 0 0) 0x7fdf9015e6f0 con 0x7fdf900cd890 2023-08-25T09:46:46.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf80003520 con 0x7fdf900cd890 2023-08-25T09:46:46.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4199932706 0 0) 0x7fdf80003520 con 0x7fdf900cd890 2023-08-25T09:46:46.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 msgr2=0x7fdf900cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 0x7fdf900cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 >> v2:172.21.15.155:3300/0 conn(0x7fdf900ca9f0 msgr2=0x7fdf9015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 --2- 172.21.15.70:0/1410806265 >> v2:172.21.15.155:3300/0 conn(0x7fdf900ca9f0 0x7fdf9015af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9015e9d0 con 0x7fdf900cd890 2023-08-25T09:46:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fdf7c001670 con 0x7fdf900cd890 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 >> v1:172.21.15.70:6789/0 conn(0x7fdf900cd890 legacy=0x7fdf900ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 0x7fdf80003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80003fa0 0x7fdf800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf80004d00 0x7fdf80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900cdd10 con 0x7fdf80003fa0 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8000bb10 con 0x7fdf80004d00 2023-08-25T09:46:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8000bcd0 con 0x7fdf90155120 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf95348640 1 --2- 172.21.15.70:0/1410806265 >> v2:172.21.15.155:3300/0 conn(0x7fdf900ca9f0 0x7fdf9015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.925+0000 7fdf8ffff640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80003fa0 0x7fdf800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf94b47640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 0x7fdf80003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf8ffff640 1 -- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 msgr2=0x7fdf80003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf8ffff640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf90155120 0x7fdf80003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf8ffff640 1 -- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf80004d00 msgr2=0x7fdf80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:46.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf95348640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf80004d00 0x7fdf80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf8ffff640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf80004d00 0x7fdf80005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.926+0000 7fdf8ffff640 1 -- 172.21.15.70:0/1410806265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf80003520 con 0x7fdf80003fa0 2023-08-25T09:46:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.927+0000 7fdf8ffff640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80003fa0 0x7fdf800047c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c240 tx=0x7fdf800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77731f2670397766 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:46.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.927+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c02cb00 con 0x7fdf80003fa0 2023-08-25T09:46:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.927+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c02cc60 con 0x7fdf80003fa0 2023-08-25T09:46:46.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.927+0000 7fdf8f7fe640 1 -- 172.21.15.70:0/1410806265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c005af0 con 0x7fdf80003fa0 2023-08-25T09:46:46.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf96dab640 1 -- 172.21.15.70:0/1410806265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80003fa0 msgr2=0x7fdf800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:46.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1410806265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf80003fa0 0x7fdf800047c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c240 tx=0x7fdf800057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf95348640 1 -- 172.21.15.70:0/1410806265 reap_dead start 2023-08-25T09:46:46.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf96dab640 1 -- 172.21.15.70:0/1410806265 shutdown_connections 2023-08-25T09:46:46.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf96dab640 1 -- 172.21.15.70:0/1410806265 >> 172.21.15.70:0/1410806265 conn(0x7fdf90069c70 msgr2=0x7fdf9006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:46.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf96dab640 1 -- 172.21.15.70:0/1410806265 shutdown_connections 2023-08-25T09:46:46.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.928+0000 7fdf96dab640 1 -- 172.21.15.70:0/1410806265 wait complete. 2023-08-25T09:46:46.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.930+0000 7fdf96dab640 1 Processor -- start 2023-08-25T09:46:46.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.930+0000 7fdf96dab640 1 -- start start 2023-08-25T09:46:46.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf96dab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf80003fa0 0x7fdf901400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf96dab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf900cc290 0x7fdf900cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf96dab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 0x7fdf90159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf96dab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901405f0 con 0x7fdf90159910 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf96dab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9014d110 con 0x7fdf900cc290 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf96dab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9015a2d0 con 0x7fdf80003fa0 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 0x7fdf90159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 0x7fdf90159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43762/0 (socket says 172.21.15.70:43762) 2023-08-25T09:46:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 -- 172.21.15.70:0/1903668010 learned_addr learned my addr 172.21.15.70:0/1903668010 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf94b47640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf80003fa0 0x7fdf901400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf8ffff640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf900cc290 0x7fdf900cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 -- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf80003fa0 msgr2=0x7fdf901400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf80003fa0 0x7fdf901400b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 -- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf900cc290 msgr2=0x7fdf900cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf900cc290 0x7fdf900cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf95348640 1 -- 172.21.15.70:0/1903668010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9015d410 con 0x7fdf90159910 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf8ffff640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf900cc290 0x7fdf900cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.931+0000 7fdf94b47640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf80003fa0 0x7fdf901400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.932+0000 7fdf95348640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 0x7fdf90159d90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fdf9006ebb0 tx=0x7fdf8800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.932+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88008e80 con 0x7fdf90159910 2023-08-25T09:46:46.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.932+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8800be80 con 0x7fdf90159910 2023-08-25T09:46:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.932+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9015d6f0 con 0x7fdf90159910 2023-08-25T09:46:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.932+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88009800 con 0x7fdf90159910 2023-08-25T09:46:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.932+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9015dcb0 con 0x7fdf90159910 2023-08-25T09:46:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.933+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9004e420 con 0x7fdf90159910 2023-08-25T09:46:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.934+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdf88009960 con 0x7fdf90159910 2023-08-25T09:46:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.935+0000 7fdf8d7fa640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf7406a050 0x7fdf7406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.935+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5826+0+0 (secure 0 0 0) 0x7fdf88088940 con 0x7fdf90159910 2023-08-25T09:46:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.935+0000 7fdf94b47640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf7406a050 0x7fdf7406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.936+0000 7fdf94b47640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf7406a050 0x7fdf7406c510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006fd0 tx=0x7fdf84008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:46.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:46.940+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdf88054070 con 0x7fdf90159910 2023-08-25T09:46:47.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.129+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf900715c0 con 0x7fdf90159910 2023-08-25T09:46:47.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.695+0000 7fdf8d7fa640 1 -- 172.21.15.70:0/1903668010 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v141) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf88088070 con 0x7fdf90159910 2023-08-25T09:46:47.698 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:46:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf7406a050 msgr2=0x7fdf7406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf7406a050 0x7fdf7406c510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006fd0 tx=0x7fdf84008040 comp rx=0 tx=0).stop 2023-08-25T09:46:47.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 msgr2=0x7fdf90159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 0x7fdf90159d90 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fdf9006ebb0 tx=0x7fdf8800b7a0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 shutdown_connections 2023-08-25T09:46:47.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf7406a050 0x7fdf7406c510 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdf90159910 0x7fdf90159d90 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdf900cc290 0x7fdf900cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 --2- 172.21.15.70:0/1903668010 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdf80003fa0 0x7fdf901400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 >> 172.21.15.70:0/1903668010 conn(0x7fdf90069c70 msgr2=0x7fdf900b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.700+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 shutdown_connections 2023-08-25T09:46:47.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.701+0000 7fdf96dab640 1 -- 172.21.15.70:0/1903668010 wait complete. 2023-08-25T09:46:47.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:46:47.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.941+0000 7fa872aec640 1 Processor -- start 2023-08-25T09:46:47.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.941+0000 7fa872aec640 1 -- start start 2023-08-25T09:46:47.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.942+0000 7fa872aec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c0cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:47.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.942+0000 7fa872aec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa86c0cdf30 0x7fa86c0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:47.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.942+0000 7fa872aec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0b4e20 con 0x7fa86c0b49e0 2023-08-25T09:46:47.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.942+0000 7fa872aec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0cd4d0 con 0x7fa86c0cdf30 2023-08-25T09:46:47.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.942+0000 7fa872aec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0d1820 con 0x7fa86c15cd10 2023-08-25T09:46:47.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.943+0000 7fa870888640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c0cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:47.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.943+0000 7fa86bfff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa86c0cdf30 0x7fa86c0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:47.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.943+0000 7fa870888640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c0cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:51498/0 (socket says 172.21.15.70:51498) 2023-08-25T09:46:47.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.943+0000 7fa870888640 1 -- 172.21.15.70:0/164804276 learned_addr learned my addr 172.21.15.70:0/164804276 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:47.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.944+0000 7fa870888640 1 -- 172.21.15.70:0/164804276 >> v2:172.21.15.155:3300/0 conn(0x7fa86c0cdf30 msgr2=0x7fa86c0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:47.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.944+0000 7fa870888640 1 --2- 172.21.15.70:0/164804276 >> v2:172.21.15.155:3300/0 conn(0x7fa86c0cdf30 0x7fa86c0b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.944+0000 7fa870888640 1 -- 172.21.15.70:0/164804276 >> v1:172.21.15.155:6789/0 conn(0x7fa86c0b49e0 legacy=0x7fa86c06fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:47.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.944+0000 7fa870888640 1 -- 172.21.15.70:0/164804276 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0d1b00 con 0x7fa86c15cd10 2023-08-25T09:46:47.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.944+0000 7fa86bfff640 1 --2- 172.21.15.70:0/164804276 >> v2:172.21.15.155:3300/0 conn(0x7fa86c0cdf30 0x7fa86c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:46:47.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.945+0000 7fa870888640 1 --2- 172.21.15.70:0/164804276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c0cd9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009e50 tx=0x7fa85c02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28bf5f24b501db25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:47.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.946+0000 7fa86b7fe640 1 -- 172.21.15.70:0/164804276 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c02cb90 con 0x7fa86c15cd10 2023-08-25T09:46:47.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.946+0000 7fa86b7fe640 1 -- 172.21.15.70:0/164804276 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c02ccf0 con 0x7fa86c15cd10 2023-08-25T09:46:47.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.946+0000 7fa86b7fe640 1 -- 172.21.15.70:0/164804276 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c034610 con 0x7fa86c15cd10 2023-08-25T09:46:47.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 -- 172.21.15.70:0/164804276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 msgr2=0x7fa86c0cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 --2- 172.21.15.70:0/164804276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c0cd9f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009e50 tx=0x7fa85c02bff0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 -- 172.21.15.70:0/164804276 shutdown_connections 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 --2- 172.21.15.70:0/164804276 >> v2:172.21.15.155:3300/0 conn(0x7fa86c0cdf30 0x7fa86c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 --2- 172.21.15.70:0/164804276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c0cd9f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 -- 172.21.15.70:0/164804276 >> 172.21.15.70:0/164804276 conn(0x7fa86c0d2a20 msgr2=0x7fa86c0d2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 -- 172.21.15.70:0/164804276 shutdown_connections 2023-08-25T09:46:47.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.947+0000 7fa872aec640 1 -- 172.21.15.70:0/164804276 wait complete. 2023-08-25T09:46:47.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.949+0000 7fa872aec640 1 Processor -- start 2023-08-25T09:46:47.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.949+0000 7fa872aec640 1 -- start start 2023-08-25T09:46:47.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa872aec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86c0b49e0 0x7fa86c0c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:47.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa872aec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 0x7fa86c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:47.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa872aec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:47.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa872aec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c1538b0 con 0x7fa86c0cdf30 2023-08-25T09:46:47.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa872aec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0c9380 con 0x7fa86c0b49e0 2023-08-25T09:46:47.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa872aec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c153b50 con 0x7fa86c15cd10 2023-08-25T09:46:47.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.950+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 0x7fa86c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:47.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 0x7fa86c0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43778/0 (socket says 172.21.15.70:43778) 2023-08-25T09:46:47.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 -- 172.21.15.70:0/2814264144 learned_addr learned my addr 172.21.15.70:0/2814264144 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:47.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa870888640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86c0b49e0 0x7fa86c0c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:47.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 -- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 msgr2=0x7fa86c144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:47.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 -- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86c0b49e0 msgr2=0x7fa86c0c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:47.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86c0b49e0 0x7fa86c0c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 -- 172.21.15.70:0/2814264144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa854009400 con 0x7fa86c0cdf30 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa870888640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86c0b49e0 0x7fa86c0c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa86bfff640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 0x7fa86c0cb840 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa85400e9b0 tx=0x7fa85400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.951+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400ca00 con 0x7fa86c0cdf30 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.952+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85400cb60 con 0x7fa86c0cdf30 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.952+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c009b30 con 0x7fa86c0cdf30 2023-08-25T09:46:47.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.952+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854013470 con 0x7fa86c0cdf30 2023-08-25T09:46:47.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.952+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0cc5e0 con 0x7fa86c0cdf30 2023-08-25T09:46:47.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.953+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c04e420 con 0x7fa86c0cdf30 2023-08-25T09:46:47.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.954+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8540135d0 con 0x7fa86c0cdf30 2023-08-25T09:46:47.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.954+0000 7fa8697fa640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:47.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.955+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa854088290 con 0x7fa86c0cdf30 2023-08-25T09:46:47.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.955+0000 7fa870888640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:47.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.956+0000 7fa870888640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa85c035000 tx=0x7fa85c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:47.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:47.959+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa854053c00 con 0x7fa86c0cdf30 2023-08-25T09:46:48.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.148+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa86c0b5d20 con 0x7fa86c0cdf30 2023-08-25T09:46:48.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.157+0000 7fa8697fa640 1 -- 172.21.15.70:0/2814264144 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa854057220 con 0x7fa86c0cdf30 2023-08-25T09:46:48.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 msgr2=0x7fa84c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa85c035000 tx=0x7fa85c030000 comp rx=0 tx=0).stop 2023-08-25T09:46:48.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 msgr2=0x7fa86c0cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 0x7fa86c0cb840 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fa85400e9b0 tx=0x7fa85400ee70 comp rx=0 tx=0).stop 2023-08-25T09:46:48.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 shutdown_connections 2023-08-25T09:46:48.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86c15cd10 0x7fa86c144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.159+0000 7fa872aec640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86c0cdf30 0x7fa86c0cb840 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.160+0000 7fa872aec640 1 --2- 172.21.15.70:0/2814264144 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86c0b49e0 0x7fa86c0c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.160+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 >> 172.21.15.70:0/2814264144 conn(0x7fa86c0d2a20 msgr2=0x7fa86c0b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.160+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 shutdown_connections 2023-08-25T09:46:48.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.160+0000 7fa872aec640 1 -- 172.21.15.70:0/2814264144 wait complete. 2023-08-25T09:46:48.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-08-25T09:46:48.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.380+0000 7f2fffee1640 1 Processor -- start 2023-08-25T09:46:48.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.380+0000 7f2fffee1640 1 -- start start 2023-08-25T09:46:48.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.382+0000 7f2fffee1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ff815ec10 0x7f2ff815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.382+0000 7f2fffee1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815c6d0 0x7f2ff80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.382+0000 7f2fffee1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b1b60 con 0x7f2ff815e830 2023-08-25T09:46:48.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.382+0000 7f2fffee1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff806bd80 con 0x7f2ff815ec10 2023-08-25T09:46:48.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.382+0000 7f2fffee1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff806b7c0 con 0x7f2ff815c6d0 2023-08-25T09:46:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.382+0000 7f2ffd47c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ff815ec10 0x7f2ff815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffd47c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ff815ec10 0x7f2ff815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43780/0 (socket says 172.21.15.70:43780) 2023-08-25T09:46:48.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffd47c640 1 -- 172.21.15.70:0/3466139863 learned_addr learned my addr 172.21.15.70:0/3466139863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:48.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffd47c640 1 -- 172.21.15.70:0/3466139863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815c6d0 msgr2=0x7f2ff80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:48.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffe47e640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815c6d0 0x7f2ff80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffd47c640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815c6d0 0x7f2ff80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffd47c640 1 -- 172.21.15.70:0/3466139863 >> v1:172.21.15.155:6789/0 conn(0x7f2ff815e830 legacy=0x7f2ff806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:48.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffd47c640 1 -- 172.21.15.70:0/3466139863 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff806baa0 con 0x7f2ff815ec10 2023-08-25T09:46:48.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.383+0000 7f2ffe47e640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815c6d0 0x7f2ff80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:48.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.384+0000 7f2ffd47c640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815ec10 0x7f2ff815c190 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009870 tx=0x7f2fe802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424252d0978dec55 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:48.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2ffcc7b640 1 -- 172.21.15.70:0/3466139863 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8031640 con 0x7f2ff815ec10 2023-08-25T09:46:48.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2ffcc7b640 1 -- 172.21.15.70:0/3466139863 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80317a0 con 0x7f2ff815ec10 2023-08-25T09:46:48.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2ffcc7b640 1 -- 172.21.15.70:0/3466139863 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8031b30 con 0x7f2ff815ec10 2023-08-25T09:46:48.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2fffee1640 1 -- 172.21.15.70:0/3466139863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815ec10 msgr2=0x7f2ff815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2fffee1640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815ec10 0x7f2ff815c190 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009870 tx=0x7f2fe802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2fffee1640 1 -- 172.21.15.70:0/3466139863 shutdown_connections 2023-08-25T09:46:48.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2fffee1640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815c6d0 0x7f2ff80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2fffee1640 1 --2- 172.21.15.70:0/3466139863 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815ec10 0x7f2ff815c190 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.385+0000 7f2fffee1640 1 -- 172.21.15.70:0/3466139863 >> 172.21.15.70:0/3466139863 conn(0x7f2ff8069850 msgr2=0x7f2ff8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:48.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.386+0000 7f2fffee1640 1 -- 172.21.15.70:0/3466139863 shutdown_connections 2023-08-25T09:46:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.386+0000 7f2fffee1640 1 -- 172.21.15.70:0/3466139863 wait complete. 2023-08-25T09:46:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.387+0000 7f2fffee1640 1 Processor -- start 2023-08-25T09:46:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 -- start start 2023-08-25T09:46:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 0x7f2ff806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815e830 0x7f2ff8149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff815ec10 0x7f2ff80a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff81568e0 con 0x7f2ff815c6d0 2023-08-25T09:46:48.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff81481b0 con 0x7f2ff815ec10 2023-08-25T09:46:48.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.388+0000 7f2fffee1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff81475a0 con 0x7f2ff815e830 2023-08-25T09:46:48.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 0x7f2ff806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 0x7f2ff806ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43796/0 (socket says 172.21.15.70:43796) 2023-08-25T09:46:48.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 -- 172.21.15.70:0/1333766666 learned_addr learned my addr 172.21.15.70:0/1333766666 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:48.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffd47c640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815e830 0x7f2ff8149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 -- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815e830 msgr2=0x7f2ff8149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815e830 0x7f2ff8149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffe47e640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff815ec10 0x7f2ff80a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 -- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff815ec10 msgr2=0x7f2ff80a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff815ec10 0x7f2ff80a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 -- 172.21.15.70:0/1333766666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe80094d0 con 0x7f2ff815c6d0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.389+0000 7f2ffdc7d640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 0x7f2ff806ece0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400c430 tx=0x7f2ff400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.390+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4011790 con 0x7f2ff815c6d0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.390+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff40118f0 con 0x7f2ff815c6d0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.390+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff8147880 con 0x7f2ff815c6d0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.390+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4011ce0 con 0x7f2ff815c6d0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.390+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff8147e40 con 0x7f2ff815c6d0 2023-08-25T09:46:48.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.391+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff804e420 con 0x7f2ff815c6d0 2023-08-25T09:46:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.392+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2ff401c460 con 0x7f2ff815c6d0 2023-08-25T09:46:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.393+0000 7f2fee7fc640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fdc06a080 0x7f2fdc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.393+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2ff4088220 con 0x7f2ff815c6d0 2023-08-25T09:46:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.393+0000 7f2ffd47c640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fdc06a080 0x7f2fdc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.394+0000 7f2ffd47c640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fdc06a080 0x7f2fdc06c540 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8036000 tx=0x7f2fe803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:48.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.397+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ff4053c20 con 0x7f2ff815c6d0 2023-08-25T09:46:48.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.637+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f2ff8147fd0 con 0x7f2ff815c6d0 2023-08-25T09:46:48.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.711+0000 7f2fee7fc640 1 -- 172.21.15.70:0/1333766666 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v427) v1 ==== 128+0+0 (secure 0 0 0) 0x7f2ff4057240 con 0x7f2ff815c6d0 2023-08-25T09:46:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fdc06a080 msgr2=0x7f2fdc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fdc06a080 0x7f2fdc06c540 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8036000 tx=0x7f2fe803c040 comp rx=0 tx=0).stop 2023-08-25T09:46:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 msgr2=0x7f2ff806ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 0x7f2ff806ece0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400c430 tx=0x7f2ff400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 shutdown_connections 2023-08-25T09:46:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2fdc06a080 0x7f2fdc06c540 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ff815ec10 0x7f2ff80a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ff815e830 0x7f2ff8149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 --2- 172.21.15.70:0/1333766666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ff815c6d0 0x7f2ff806ece0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 >> 172.21.15.70:0/1333766666 conn(0x7f2ff8069850 msgr2=0x7f2ff8159230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 shutdown_connections 2023-08-25T09:46:48.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.714+0000 7f2fffee1640 1 -- 172.21.15.70:0/1333766666 wait complete. 2023-08-25T09:46:48.734 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-08-25T09:46:48.734 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-08-25T09:46:48.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.957+0000 7f48c7814640 1 Processor -- start 2023-08-25T09:46:48.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.958+0000 7f48c7814640 1 -- start start 2023-08-25T09:46:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c7814640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48c00ce5b0 0x7f48c00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c7814640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f48c015aa10 0x7f48c00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c7814640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00b2aa0 con 0x7f48c00b2680 2023-08-25T09:46:48.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c7814640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c0159d10 con 0x7f48c00ce5b0 2023-08-25T09:46:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c7814640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c00ceed0 con 0x7f48c015aa10 2023-08-25T09:46:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c55b0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f48c00ce5b0 0x7f48c00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c5db1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f48c00b2680 0x7f48c006be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42620/0 (socket says 172.21.15.70:42620) 2023-08-25T09:46:48.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.959+0000 7f48c5db1640 1 -- 172.21.15.70:0/1732238500 learned_addr learned my addr 172.21.15.70:0/1732238500 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.960+0000 7f48c55b0640 1 -- 172.21.15.70:0/1732238500 >> v2:172.21.15.155:3300/0 conn(0x7f48c015aa10 msgr2=0x7f48c00b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.960+0000 7f48c55b0640 1 --2- 172.21.15.70:0/1732238500 >> v2:172.21.15.155:3300/0 conn(0x7f48c015aa10 0x7f48c00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.960+0000 7f48c55b0640 1 -- 172.21.15.70:0/1732238500 >> v1:172.21.15.70:6789/0 conn(0x7f48c00b2680 legacy=0x7f48c006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.960+0000 7f48c55b0640 1 -- 172.21.15.70:0/1732238500 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c015d3f0 con 0x7f48c00ce5b0 2023-08-25T09:46:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.961+0000 7f48c55b0640 1 --2- 172.21.15.70:0/1732238500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00ce5b0 0x7f48c00ce990 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f48c00b16e0 tx=0x7f48bc02d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f94d8276d44e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.961+0000 7f48b7fff640 1 -- 172.21.15.70:0/1732238500 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc033770 con 0x7f48c00ce5b0 2023-08-25T09:46:48.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.961+0000 7f48b7fff640 1 -- 172.21.15.70:0/1732238500 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc0338d0 con 0x7f48c00ce5b0 2023-08-25T09:46:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.961+0000 7f48b7fff640 1 -- 172.21.15.70:0/1732238500 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc033c60 con 0x7f48c00ce5b0 2023-08-25T09:46:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 -- 172.21.15.70:0/1732238500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00ce5b0 msgr2=0x7f48c00ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 --2- 172.21.15.70:0/1732238500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00ce5b0 0x7f48c00ce990 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f48c00b16e0 tx=0x7f48bc02d890 comp rx=0 tx=0).stop 2023-08-25T09:46:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 -- 172.21.15.70:0/1732238500 shutdown_connections 2023-08-25T09:46:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 --2- 172.21.15.70:0/1732238500 >> v2:172.21.15.155:3300/0 conn(0x7f48c015aa10 0x7f48c00b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 --2- 172.21.15.70:0/1732238500 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00ce5b0 0x7f48c00ce990 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 -- 172.21.15.70:0/1732238500 >> 172.21.15.70:0/1732238500 conn(0x7f48c0069850 msgr2=0x7f48c0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:48.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 -- 172.21.15.70:0/1732238500 shutdown_connections 2023-08-25T09:46:48.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.962+0000 7f48c7814640 1 -- 172.21.15.70:0/1732238500 wait complete. 2023-08-25T09:46:48.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.964+0000 7f48c7814640 1 Processor -- start 2023-08-25T09:46:48.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.964+0000 7f48c7814640 1 -- start start 2023-08-25T09:46:48.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c7814640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 0x7f48c00aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c7814640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c00ce5b0 0x7f48c0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c7814640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c015aa10 0x7f48c014a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c7814640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00ae070 con 0x7f48c00b2680 2023-08-25T09:46:48.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c7814640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0157450 con 0x7f48c00ce5b0 2023-08-25T09:46:48.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c7814640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0157730 con 0x7f48c015aa10 2023-08-25T09:46:48.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c55b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 0x7f48c00aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c55b0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 0x7f48c00aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43814/0 (socket says 172.21.15.70:43814) 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.965+0000 7f48c55b0640 1 -- 172.21.15.70:0/2958178349 learned_addr learned my addr 172.21.15.70:0/2958178349 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c55b0640 1 -- 172.21.15.70:0/2958178349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c015aa10 msgr2=0x7f48c014a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c55b0640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c015aa10 0x7f48c014a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c55b0640 1 -- 172.21.15.70:0/2958178349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c00ce5b0 msgr2=0x7f48c0156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c55b0640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c00ce5b0 0x7f48c0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c55b0640 1 -- 172.21.15.70:0/2958178349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc009400 con 0x7f48c00b2680 2023-08-25T09:46:48.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c55b0640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 0x7f48c00aec10 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f48bc036040 tx=0x7f48bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:48.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc034bc0 con 0x7f48c00b2680 2023-08-25T09:46:48.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc034d40 con 0x7f48c00b2680 2023-08-25T09:46:48.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc047630 con 0x7f48c00b2680 2023-08-25T09:46:48.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.966+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c0157a10 con 0x7f48c00b2680 2023-08-25T09:46:48.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.967+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c0157fd0 con 0x7f48c00b2680 2023-08-25T09:46:48.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.968+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c004e420 con 0x7f48c00b2680 2023-08-25T09:46:48.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.969+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f48bc050440 con 0x7f48c00b2680 2023-08-25T09:46:48.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.970+0000 7f48b5ffb640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f48a406a080 0x7f48a406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:48.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.970+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f48bc0aa9a0 con 0x7f48c00b2680 2023-08-25T09:46:48.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.970+0000 7f48c4daf640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f48a406a080 0x7f48a406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:48.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.971+0000 7f48c4daf640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f48a406a080 0x7f48a406c540 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f48c0158840 tx=0x7f48b0006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:48.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:48.974+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48bc076280 con 0x7f48c00b2680 2023-08-25T09:46:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.162+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48c01581c0 con 0x7f48c00b2680 2023-08-25T09:46:49.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.163+0000 7f48b5ffb640 1 -- 172.21.15.70:0/2958178349 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) v1 ==== 74+0+14790 (secure 0 0 0) 0x7f48bc0798a0 con 0x7f48c00b2680 2023-08-25T09:46:49.165 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:49.166 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":141,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:47.686723+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6836/59115020":"2023-08-26T09:46:44.493115+0000","172.21.15.70:6834/59115020":"2023-08-26T09:46:44.493115+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:49.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.166+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f48a406a080 msgr2=0x7f48a406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f48a406a080 0x7f48a406c540 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f48c0158840 tx=0x7f48b0006c90 comp rx=0 tx=0).stop 2023-08-25T09:46:49.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 msgr2=0x7f48c00aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 0x7f48c00aec10 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f48bc036040 tx=0x7f48bc0046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 shutdown_connections 2023-08-25T09:46:49.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f48a406a080 0x7f48a406c540 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f48c015aa10 0x7f48c014a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f48c00ce5b0 0x7f48c0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 --2- 172.21.15.70:0/2958178349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f48c00b2680 0x7f48c00aec10 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 >> 172.21.15.70:0/2958178349 conn(0x7f48c0069850 msgr2=0x7f48c015af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:49.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 shutdown_connections 2023-08-25T09:46:49.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.167+0000 7f48c7814640 1 -- 172.21.15.70:0/2958178349 wait complete. 2023-08-25T09:46:49.186 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:49.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:49.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.396+0000 7f146bfff640 1 Processor -- start 2023-08-25T09:46:49.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.396+0000 7f146bfff640 1 -- start start 2023-08-25T09:46:49.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.397+0000 7f146bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 0x7f146c164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.397+0000 7f146bfff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0cd800 con 0x7f146c0b2850 2023-08-25T09:46:49.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.397+0000 7f146bfff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c15a110 con 0x7f146c0b2470 2023-08-25T09:46:49.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.397+0000 7f146bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15ad20 con 0x7f146c162630 2023-08-25T09:46:49.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f146a7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f146c0b2850 0x7f146c0cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57572/0 (socket says 172.21.15.70:57572) 2023-08-25T09:46:49.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f146a7fc640 1 -- 172.21.15.70:0/3461582143 learned_addr learned my addr 172.21.15.70:0/3461582143 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:49.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f146b7fe640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 0x7f146c164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457464030 0 0) 0x7f146c0cd800 con 0x7f146c0b2850 2023-08-25T09:46:49.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1458005730 con 0x7f146c0b2850 2023-08-25T09:46:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017119075 0 0) 0x7f1458005730 con 0x7f146c0b2850 2023-08-25T09:46:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 msgr2=0x7f146c164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.398+0000 7f1469ffb640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 0x7f146c164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 >> v1:172.21.15.155:6789/0 conn(0x7f146c0b2470 legacy=0x7f146c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c15a3f0 con 0x7f146c0b2850 2023-08-25T09:46:49.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f145c0019f0 con 0x7f146c0b2850 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 >> v1:172.21.15.70:6789/0 conn(0x7f146c0b2850 legacy=0x7f146c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1458005bb0 0x7f1458005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 0x7f14580069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458006f10 0x7f1458007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c15ad20 con 0x7f1458006f10 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145800dc70 con 0x7f1458005bb0 2023-08-25T09:46:49.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145800de30 con 0x7f146c162630 2023-08-25T09:46:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f146b7fe640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458006f10 0x7f1458007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.399+0000 7f146a7fc640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 0x7f14580069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.400+0000 7f146affd640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1458005bb0 0x7f1458005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.400+0000 7f146b7fe640 1 -- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 msgr2=0x7f14580069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.400+0000 7f146b7fe640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f146c162630 0x7f14580069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.400+0000 7f146b7fe640 1 -- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1458005bb0 msgr2=0x7f1458005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.400+0000 7f146b7fe640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1458005bb0 0x7f1458005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.400+0000 7f146b7fe640 1 -- 172.21.15.70:0/3461582143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1458005730 con 0x7f1458006f10 2023-08-25T09:46:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.401+0000 7f146b7fe640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458006f10 0x7f1458007390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f146c0b0ee0 tx=0x7f1458007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db367a5fbe61f119 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.401+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146402c660 con 0x7f1458006f10 2023-08-25T09:46:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.401+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146402c7c0 con 0x7f1458006f10 2023-08-25T09:46:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.401+0000 7f1469ffb640 1 -- 172.21.15.70:0/3461582143 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146402cb70 con 0x7f1458006f10 2023-08-25T09:46:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146bfff640 1 -- 172.21.15.70:0/3461582143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458006f10 msgr2=0x7f1458007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146bfff640 1 --2- 172.21.15.70:0/3461582143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1458006f10 0x7f1458007390 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f146c0b0ee0 tx=0x7f1458007a00 comp rx=0 tx=0).stop 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146b7fe640 1 -- 172.21.15.70:0/3461582143 reap_dead start 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146bfff640 1 -- 172.21.15.70:0/3461582143 shutdown_connections 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146bfff640 1 -- 172.21.15.70:0/3461582143 >> 172.21.15.70:0/3461582143 conn(0x7f146c0ce830 msgr2=0x7f146c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146bfff640 1 -- 172.21.15.70:0/3461582143 shutdown_connections 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.402+0000 7f146bfff640 1 -- 172.21.15.70:0/3461582143 wait complete. 2023-08-25T09:46:49.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.404+0000 7f146bfff640 1 Processor -- start 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.404+0000 7f146bfff640 1 -- start start 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1458006f10 0x7f146c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 0x7f146c06b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f146c08ca40 0x7f146c099bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c158990 con 0x7f146c1578f0 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146bfff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c157d80 con 0x7f146c08ca40 2023-08-25T09:46:49.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c158060 con 0x7f1458006f10 2023-08-25T09:46:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 0x7f146c06b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 0x7f146c06b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43678/0 (socket says 172.21.15.70:43678) 2023-08-25T09:46:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 -- 172.21.15.70:0/379785346 learned_addr learned my addr 172.21.15.70:0/379785346 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146affd640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1458006f10 0x7f146c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 -- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1458006f10 msgr2=0x7f146c158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1458006f10 0x7f146c158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 -- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f146c08ca40 msgr2=0x7f146c099bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f146c08ca40 0x7f146c099bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.405+0000 7f146a7fc640 1 -- 172.21.15.70:0/379785346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c158340 con 0x7f146c1578f0 2023-08-25T09:46:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.406+0000 7f146a7fc640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 0x7f146c06b1c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f145c007ae0 tx=0x7f145c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.406+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c00ee90 con 0x7f146c1578f0 2023-08-25T09:46:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.406+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c013420 con 0x7f146c1578f0 2023-08-25T09:46:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.406+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0137d0 con 0x7f146c1578f0 2023-08-25T09:46:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.406+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c158620 con 0x7f146c1578f0 2023-08-25T09:46:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.406+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c09ba50 con 0x7f146c1578f0 2023-08-25T09:46:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.407+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c158b40 con 0x7f146c1578f0 2023-08-25T09:46:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.408+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f145c013930 con 0x7f146c1578f0 2023-08-25T09:46:49.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.409+0000 7f1463fff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f144c06a080 0x7f144c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.409+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f145c088b50 con 0x7f146c1578f0 2023-08-25T09:46:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.410+0000 7f146affd640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f144c06a080 0x7f144c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.410+0000 7f146affd640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f144c06a080 0x7f144c06c540 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1454001640 tx=0x7f1454009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:49.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.414+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f146c158b40 con 0x7f146c1578f0 2023-08-25T09:46:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.601+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f146c061460 con 0x7f146c1578f0 2023-08-25T09:46:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.602+0000 7f1463fff640 1 -- 172.21.15.70:0/379785346 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) v1 ==== 93+0+10 (secure 0 0 0) 0x7f145c057f50 con 0x7f146c1578f0 2023-08-25T09:46:49.604 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f144c06a080 msgr2=0x7f144c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f144c06a080 0x7f144c06c540 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1454001640 tx=0x7f1454009040 comp rx=0 tx=0).stop 2023-08-25T09:46:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 msgr2=0x7f146c06b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 0x7f146c06b1c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f145c007ae0 tx=0x7f145c00e530 comp rx=0 tx=0).stop 2023-08-25T09:46:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 shutdown_connections 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f144c06a080 0x7f144c06c540 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f146c08ca40 0x7f146c099bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f146c1578f0 0x7f146c06b1c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 --2- 172.21.15.70:0/379785346 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1458006f10 0x7f146c158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.604+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 >> 172.21.15.70:0/379785346 conn(0x7f146c0ce830 msgr2=0x7f146c069c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.605+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 shutdown_connections 2023-08-25T09:46:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.605+0000 7f146bfff640 1 -- 172.21.15.70:0/379785346 wait complete. 2023-08-25T09:46:49.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.843+0000 7ff5311a4640 1 Processor -- start 2023-08-25T09:46:49.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.844+0000 7ff5311a4640 1 -- start start 2023-08-25T09:46:49.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.845+0000 7ff5311a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff52c069850 0x7ff52c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.845+0000 7ff5311a4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff52c0c9620 0x7ff52c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.845+0000 7ff5311a4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c0ac880 con 0x7ff52c160be0 2023-08-25T09:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.845+0000 7ff5311a4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0ad400 con 0x7ff52c0c9620 2023-08-25T09:46:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.845+0000 7ff5311a4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0ad890 con 0x7ff52c069850 2023-08-25T09:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.845+0000 7ff523fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff52c0c9620 0x7ff52c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff523fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff52c0c9620 0x7ff52c069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43692/0 (socket says 172.21.15.70:43692) 2023-08-25T09:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff523fff640 1 -- 172.21.15.70:0/3090940269 learned_addr learned my addr 172.21.15.70:0/3090940269 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff52b59e640 1 --1- 172.21.15.70:0/3090940269 >> v1:172.21.15.70:6789/0 conn(0x7ff52c160be0 0x7ff52c161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57578/0 (socket says 172.21.15.70:57578) 2023-08-25T09:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff52ad9d640 1 --2- 172.21.15.70:0/3090940269 >> v2:172.21.15.155:3300/0 conn(0x7ff52c069850 0x7ff52c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff523fff640 1 -- 172.21.15.70:0/3090940269 >> v2:172.21.15.155:3300/0 conn(0x7ff52c069850 msgr2=0x7ff52c15e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff523fff640 1 --2- 172.21.15.70:0/3090940269 >> v2:172.21.15.155:3300/0 conn(0x7ff52c069850 0x7ff52c15e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff523fff640 1 -- 172.21.15.70:0/3090940269 >> v1:172.21.15.70:6789/0 conn(0x7ff52c160be0 legacy=0x7ff52c161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.846+0000 7ff523fff640 1 -- 172.21.15.70:0/3090940269 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0acad0 con 0x7ff52c0c9620 2023-08-25T09:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.847+0000 7ff523fff640 1 --2- 172.21.15.70:0/3090940269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c0c9620 0x7ff52c069c30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0cc0d0 tx=0x7ff51402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d220df61904580 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:49.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff52a59c640 1 -- 172.21.15.70:0/3090940269 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514031750 con 0x7ff52c0c9620 2023-08-25T09:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff52a59c640 1 -- 172.21.15.70:0/3090940269 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5140318b0 con 0x7ff52c0c9620 2023-08-25T09:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff52a59c640 1 -- 172.21.15.70:0/3090940269 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514031c40 con 0x7ff52c0c9620 2023-08-25T09:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 -- 172.21.15.70:0/3090940269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c0c9620 msgr2=0x7ff52c069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 --2- 172.21.15.70:0/3090940269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c0c9620 0x7ff52c069c30 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0cc0d0 tx=0x7ff51402b870 comp rx=0 tx=0).stop 2023-08-25T09:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 -- 172.21.15.70:0/3090940269 shutdown_connections 2023-08-25T09:46:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 --2- 172.21.15.70:0/3090940269 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c0c9620 0x7ff52c069c30 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 --2- 172.21.15.70:0/3090940269 >> v2:172.21.15.155:3300/0 conn(0x7ff52c069850 0x7ff52c15e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 -- 172.21.15.70:0/3090940269 >> 172.21.15.70:0/3090940269 conn(0x7ff52c0cb0d0 msgr2=0x7ff52c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.848+0000 7ff5311a4640 1 -- 172.21.15.70:0/3090940269 shutdown_connections 2023-08-25T09:46:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.849+0000 7ff5311a4640 1 -- 172.21.15.70:0/3090940269 wait complete. 2023-08-25T09:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.850+0000 7ff5311a4640 1 Processor -- start 2023-08-25T09:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.850+0000 7ff5311a4640 1 -- start start 2023-08-25T09:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.851+0000 7ff5311a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 0x7ff52c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.851+0000 7ff5311a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff52c0c9620 0x7ff52c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.851+0000 7ff5311a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff52c160be0 0x7ff52c0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.851+0000 7ff5311a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0afeb0 con 0x7ff52c069850 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.851+0000 7ff5311a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0b3c90 con 0x7ff52c160be0 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.851+0000 7ff5311a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0b1270 con 0x7ff52c0c9620 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 0x7ff52c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 0x7ff52c0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43696/0 (socket says 172.21.15.70:43696) 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 -- 172.21.15.70:0/713258853 learned_addr learned my addr 172.21.15.70:0/713258853 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:49.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 -- 172.21.15.70:0/713258853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff52c0c9620 msgr2=0x7ff52c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff52c0c9620 0x7ff52c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 -- 172.21.15.70:0/713258853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff52c160be0 msgr2=0x7ff52c0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff52c160be0 0x7ff52c0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 -- 172.21.15.70:0/713258853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5140094d0 con 0x7ff52c069850 2023-08-25T09:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.852+0000 7ff52ad9d640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 0x7ff52c0c7320 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00d800 tx=0x7ff51c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:49.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.853+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00b6c0 con 0x7ff52c069850 2023-08-25T09:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.853+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c00b820 con 0x7ff52c069850 2023-08-25T09:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.853+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c00bbb0 con 0x7ff52c069850 2023-08-25T09:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.853+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c0b00a0 con 0x7ff52c069850 2023-08-25T09:46:49.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.853+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c0b1760 con 0x7ff52c069850 2023-08-25T09:46:49.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.854+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c066fb0 con 0x7ff52c069850 2023-08-25T09:46:49.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.855+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff51c00bd10 con 0x7ff52c069850 2023-08-25T09:46:49.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.856+0000 7ff5237fe640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff50c06a050 0x7ff50c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.856+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff51c088890 con 0x7ff52c069850 2023-08-25T09:46:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.856+0000 7ff523fff640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff50c06a050 0x7ff50c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:49.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.857+0000 7ff523fff640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff50c06a050 0x7ff50c06c510 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff514035000 tx=0x7ff514030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:49.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:49.860+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff51c054200 con 0x7ff52c069850 2023-08-25T09:46:50.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.045+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff52c06a810 con 0x7ff52c069850 2023-08-25T09:46:50.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.046+0000 7ff5237fe640 1 -- 172.21.15.70:0/713258853 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v141) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff51c057820 con 0x7ff52c069850 2023-08-25T09:46:50.048 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:50.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff50c06a050 msgr2=0x7ff50c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff50c06a050 0x7ff50c06c510 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff514035000 tx=0x7ff514030000 comp rx=0 tx=0).stop 2023-08-25T09:46:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 msgr2=0x7ff52c0c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 0x7ff52c0c7320 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00d800 tx=0x7ff51c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 shutdown_connections 2023-08-25T09:46:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff50c06a050 0x7ff50c06c510 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff52c160be0 0x7ff52c0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff52c0c9620 0x7ff52c0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 --2- 172.21.15.70:0/713258853 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff52c069850 0x7ff52c0c7320 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 >> 172.21.15.70:0/713258853 conn(0x7ff52c0cb0d0 msgr2=0x7ff52c06ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:50.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 shutdown_connections 2023-08-25T09:46:50.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.049+0000 7ff5311a4640 1 -- 172.21.15.70:0/713258853 wait complete. 2023-08-25T09:46:50.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-08-25T09:46:50.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.263+0000 7fe0c2167640 1 Processor -- start 2023-08-25T09:46:50.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.263+0000 7fe0c2167640 1 -- start start 2023-08-25T09:46:50.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.264+0000 7fe0c2167640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 0x7fe0bc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.264+0000 7fe0c2167640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0bc0cad20 0x7fe0bc0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.264+0000 7fe0c2167640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0cb6c0 con 0x7fe0bc0b12f0 2023-08-25T09:46:50.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.264+0000 7fe0c2167640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc15e7c0 con 0x7fe0bc0cad20 2023-08-25T09:46:50.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.264+0000 7fe0c2167640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc071b50 con 0x7fe0bc15a7c0 2023-08-25T09:46:50.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.265+0000 7fe0c0964640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe0bc0b12f0 0x7fe0bc15ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57590/0 (socket says 172.21.15.70:57590) 2023-08-25T09:46:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.265+0000 7fe0c0964640 1 -- 172.21.15.70:0/1996258142 learned_addr learned my addr 172.21.15.70:0/1996258142 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.265+0000 7fe0c1165640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 0x7fe0bc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.265+0000 7fe0c1966640 1 --2- 172.21.15.70:0/1996258142 >> v2:172.21.15.155:3300/0 conn(0x7fe0bc0cad20 0x7fe0bc0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.265+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124399614 0 0) 0x7fe0bc0cb6c0 con 0x7fe0bc0b12f0 2023-08-25T09:46:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.265+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a4003520 con 0x7fe0bc0b12f0 2023-08-25T09:46:50.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1887023338 0 0) 0x7fe0a4003520 con 0x7fe0bc0b12f0 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 msgr2=0x7fe0bc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 0x7fe0bc0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 >> v2:172.21.15.155:3300/0 conn(0x7fe0bc0cad20 msgr2=0x7fe0bc0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 --2- 172.21.15.70:0/1996258142 >> v2:172.21.15.155:3300/0 conn(0x7fe0bc0cad20 0x7fe0bc0cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc15e490 con 0x7fe0bc0b12f0 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe0b0001670 con 0x7fe0bc0b12f0 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 >> v1:172.21.15.70:6789/0 conn(0x7fe0bc0b12f0 legacy=0x7fe0bc15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0a4003970 0x7fe0a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 0x7fe0a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc15e7c0 con 0x7fe0a40047c0 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc071b50 con 0x7fe0a4003970 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a400b980 con 0x7fe0bc15a7c0 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c0964640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c0964640 1 -- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 msgr2=0x7fe0a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c0964640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc15a7c0 0x7fe0a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c1165640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0a4003970 0x7fe0a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c0964640 1 -- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0a4003970 msgr2=0x7fe0a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c0964640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0a4003970 0x7fe0a4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.266+0000 7fe0c0964640 1 -- 172.21.15.70:0/1996258142 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a4003520 con 0x7fe0a40047c0 2023-08-25T09:46:50.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.267+0000 7fe0c0964640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0a4004c40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c210 tx=0x7fe0a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d597477a27ac4b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b002cae0 con 0x7fe0a40047c0 2023-08-25T09:46:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b002cc40 con 0x7fe0a40047c0 2023-08-25T09:46:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0bbfff640 1 -- 172.21.15.70:0/1996258142 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b00058d0 con 0x7fe0a40047c0 2023-08-25T09:46:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c2167640 1 -- 172.21.15.70:0/1996258142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 msgr2=0x7fe0a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c2167640 1 --2- 172.21.15.70:0/1996258142 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0a4004c40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe0b000c210 tx=0x7fe0a40057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c1966640 1 -- 172.21.15.70:0/1996258142 reap_dead start 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c2167640 1 -- 172.21.15.70:0/1996258142 shutdown_connections 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c2167640 1 -- 172.21.15.70:0/1996258142 >> 172.21.15.70:0/1996258142 conn(0x7fe0bc0affd0 msgr2=0x7fe0bc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c2167640 1 -- 172.21.15.70:0/1996258142 shutdown_connections 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.268+0000 7fe0c2167640 1 -- 172.21.15.70:0/1996258142 wait complete. 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.270+0000 7fe0c2167640 1 Processor -- start 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.270+0000 7fe0c2167640 1 -- start start 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c2167640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0bc1523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c2167640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc0cc070 0x7fe0bc0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c2167640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc0cc4b0 0x7fe0bc0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c2167640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0c2510 con 0x7fe0a40047c0 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c2167640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1f2080 con 0x7fe0bc0cc070 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c2167640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1f2360 con 0x7fe0bc0cc4b0 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c1165640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0bc1523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c1165640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0bc1523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43712/0 (socket says 172.21.15.70:43712) 2023-08-25T09:46:50.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.271+0000 7fe0c1165640 1 -- 172.21.15.70:0/555575180 learned_addr learned my addr 172.21.15.70:0/555575180 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:50.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c1165640 1 -- 172.21.15.70:0/555575180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc0cc4b0 msgr2=0x7fe0bc0acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c1165640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc0cc4b0 0x7fe0bc0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c1165640 1 -- 172.21.15.70:0/555575180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc0cc070 msgr2=0x7fe0bc0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c1165640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc0cc070 0x7fe0bc0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c1165640 1 -- 172.21.15.70:0/555575180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc1f2640 con 0x7fe0a40047c0 2023-08-25T09:46:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c1165640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0bc1523f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00f350 tx=0x7fe0ac00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:50.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00fee0 con 0x7fe0a40047c0 2023-08-25T09:46:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac011040 con 0x7fe0a40047c0 2023-08-25T09:46:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0bc1f2920 con 0x7fe0a40047c0 2023-08-25T09:46:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac017860 con 0x7fe0a40047c0 2023-08-25T09:46:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.272+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc1f2c00 con 0x7fe0a40047c0 2023-08-25T09:46:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.274+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc0b2be0 con 0x7fe0a40047c0 2023-08-25T09:46:50.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.275+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe0ac0179c0 con 0x7fe0a40047c0 2023-08-25T09:46:50.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.276+0000 7fe0b9ffb640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0a806a080 0x7fe0a806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.276+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe0ac088810 con 0x7fe0a40047c0 2023-08-25T09:46:50.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.276+0000 7fe0c0964640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0a806a080 0x7fe0a806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.277+0000 7fe0c0964640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0a806a080 0x7fe0a806c540 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0034000 tx=0x7fe0b002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:50.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.280+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0ac0542c0 con 0x7fe0a40047c0 2023-08-25T09:46:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.530+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7fe0bc066fb0 con 0x7fe0a40047c0 2023-08-25T09:46:50.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.724+0000 7fe0b9ffb640 1 -- 172.21.15.70:0/555575180 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v429) v1 ==== 143+0+0 (secure 0 0 0) 0x7fe0ac01c0d0 con 0x7fe0a40047c0 2023-08-25T09:46:50.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0a806a080 msgr2=0x7fe0a806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0a806a080 0x7fe0a806c540 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0034000 tx=0x7fe0b002e000 comp rx=0 tx=0).stop 2023-08-25T09:46:50.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 msgr2=0x7fe0bc1523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0bc1523f0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00f350 tx=0x7fe0ac00f810 comp rx=0 tx=0).stop 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 shutdown_connections 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0a806a080 0x7fe0a806c540 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0bc0cc4b0 0x7fe0bc0acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0bc0cc070 0x7fe0bc0ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 --2- 172.21.15.70:0/555575180 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0a40047c0 0x7fe0bc1523f0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 >> 172.21.15.70:0/555575180 conn(0x7fe0bc0affd0 msgr2=0x7fe0bc152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 shutdown_connections 2023-08-25T09:46:50.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.727+0000 7fe0c2167640 1 -- 172.21.15.70:0/555575180 wait complete. 2023-08-25T09:46:50.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:46:50.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.966+0000 7f3ad7d5d640 1 Processor -- start 2023-08-25T09:46:50.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.966+0000 7f3ad7d5d640 1 -- start start 2023-08-25T09:46:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad7d5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 0x7f3ad00c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad7d5d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ad00cdc20 0x7f3ad0069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad7d5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad00cc5f0 con 0x7f3ad00ce060 2023-08-25T09:46:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad7d5d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad0070650 con 0x7f3ad00cdc20 2023-08-25T09:46:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad7d5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0070930 con 0x7f3ad015ae50 2023-08-25T09:46:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad62fa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ad00ce060 0x7f3ad00ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57600/0 (socket says 172.21.15.70:57600) 2023-08-25T09:46:50.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad62fa640 1 -- 172.21.15.70:0/825703815 learned_addr learned my addr 172.21.15.70:0/825703815 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:50.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.968+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539513755 0 0) 0x7f3ad00cc5f0 con 0x7f3ad00ce060 2023-08-25T09:46:50.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3abc003520 con 0x7f3ad00ce060 2023-08-25T09:46:50.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/825703815 >> v2:172.21.15.155:3300/0 conn(0x7f3ad00cdc20 0x7f3ad0069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad5af9640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 0x7f3ad00c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3048812507 0 0) 0x7f3abc003520 con 0x7f3ad00ce060 2023-08-25T09:46:50.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 msgr2=0x7f3ad00c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 0x7f3ad00c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 >> v2:172.21.15.155:3300/0 conn(0x7f3ad00cdc20 msgr2=0x7f3ad0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 --2- 172.21.15.70:0/825703815 >> v2:172.21.15.155:3300/0 conn(0x7f3ad00cdc20 0x7f3ad0069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0070c10 con 0x7f3ad00ce060 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3ac80019f0 con 0x7f3ad00ce060 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 >> v1:172.21.15.70:6789/0 conn(0x7f3ad00ce060 legacy=0x7f3ad00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 0x7f3abc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc003fa0 0x7f3abc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3abc004d00 0x7f3abc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0070650 con 0x7f3abc003fa0 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad0070930 con 0x7f3abc004d00 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc00bc20 con 0x7f3ad015ae50 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/825703815 >> v2:172.21.15.155:3300/0 conn(0x7f3ad00cdc20 0x7f3ad0069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.969+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc003fa0 0x7f3abc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad5af9640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 0x7f3abc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad62fa640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3abc004d00 0x7f3abc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad52f8640 1 -- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 msgr2=0x7f3abc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ad015ae50 0x7f3abc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad52f8640 1 -- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3abc004d00 msgr2=0x7f3abc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3abc004d00 0x7f3abc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad52f8640 1 -- 172.21.15.70:0/825703815 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc003520 con 0x7f3abc003fa0 2023-08-25T09:46:50.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.970+0000 7f3ad62fa640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3abc004d00 0x7f3abc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:50.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.971+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc003fa0 0x7f3abc0047c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000aa70 tx=0x7f3abc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0d5528ba6fa9c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.971+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0004220 con 0x7f3abc003fa0 2023-08-25T09:46:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.971+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac0004380 con 0x7f3abc003fa0 2023-08-25T09:46:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.971+0000 7f3ad4af7640 1 -- 172.21.15.70:0/825703815 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0003830 con 0x7f3abc003fa0 2023-08-25T09:46:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/825703815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc003fa0 msgr2=0x7f3abc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/825703815 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3abc003fa0 0x7f3abc0047c0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000aa70 tx=0x7f3abc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad62fa640 1 -- 172.21.15.70:0/825703815 reap_dead start 2023-08-25T09:46:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/825703815 shutdown_connections 2023-08-25T09:46:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/825703815 >> 172.21.15.70:0/825703815 conn(0x7f3ad00cb0d0 msgr2=0x7f3ad00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/825703815 shutdown_connections 2023-08-25T09:46:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.972+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/825703815 wait complete. 2023-08-25T09:46:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.974+0000 7f3ad7d5d640 1 Processor -- start 2023-08-25T09:46:50.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.974+0000 7f3ad7d5d640 1 -- start start 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad7d5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3abc003fa0 0x7f3ad0149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad7d5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 0x7f3ad015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad7d5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad0146ae0 0x7f3ad015d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad7d5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00c8510 con 0x7f3ad00c5790 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad7d5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad014a050 con 0x7f3ad0146ae0 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad7d5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad00c89a0 con 0x7f3abc003fa0 2023-08-25T09:46:50.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad52f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 0x7f3ad015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad52f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 0x7f3ad015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43732/0 (socket says 172.21.15.70:43732) 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad52f8640 1 -- 172.21.15.70:0/1416335186 learned_addr learned my addr 172.21.15.70:0/1416335186 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad52f8640 1 -- 172.21.15.70:0/1416335186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3abc003fa0 msgr2=0x7f3ad0149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.975+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3abc003fa0 0x7f3ad0149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ad52f8640 1 -- 172.21.15.70:0/1416335186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad0146ae0 msgr2=0x7f3ad015d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad0146ae0 0x7f3ad015d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ad52f8640 1 -- 172.21.15.70:0/1416335186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad00c7900 con 0x7f3ad00c5790 2023-08-25T09:46:50.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ad52f8640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 0x7f3ad015d3c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ab90 tx=0x7f3ac0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:50.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0037810 con 0x7f3ad00c5790 2023-08-25T09:46:50.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac003d040 con 0x7f3ad00c5790 2023-08-25T09:46:50.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad00c7be0 con 0x7f3ad00c5790 2023-08-25T09:46:50.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0037bd0 con 0x7f3ad00c5790 2023-08-25T09:46:50.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.976+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad00c81a0 con 0x7f3ad00c5790 2023-08-25T09:46:50.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.977+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad0066fb0 con 0x7f3ad00c5790 2023-08-25T09:46:50.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.978+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3ac0037d30 con 0x7f3ad00c5790 2023-08-25T09:46:50.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.979+0000 7f3ac67fc640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ab406a080 0x7f3ab406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:50.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.979+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3ac00a9ff0 con 0x7f3ad00c5790 2023-08-25T09:46:50.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.979+0000 7f3ad5af9640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ab406a080 0x7f3ab406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:50.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.980+0000 7f3ad5af9640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ab406a080 0x7f3ab406c540 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3acc006a90 tx=0x7f3acc006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:50.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:50.983+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ac0075aa0 con 0x7f3ad00c5790 2023-08-25T09:46:51.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.170+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3ad01e7940 con 0x7f3ad00c5790 2023-08-25T09:46:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.177+0000 7f3ac67fc640 1 -- 172.21.15.70:0/1416335186 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3ac00790c0 con 0x7f3ad00c5790 2023-08-25T09:46:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ab406a080 msgr2=0x7f3ab406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:51.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ab406a080 0x7f3ab406c540 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3acc006a90 tx=0x7f3acc006a20 comp rx=0 tx=0).stop 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 msgr2=0x7f3ad015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 0x7f3ad015d3c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000ab90 tx=0x7f3ac0004730 comp rx=0 tx=0).stop 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 shutdown_connections 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3ab406a080 0x7f3ab406c540 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ad0146ae0 0x7f3ad015d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ad00c5790 0x7f3ad015d3c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 --2- 172.21.15.70:0/1416335186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3abc003fa0 0x7f3ad0149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 >> 172.21.15.70:0/1416335186 conn(0x7f3ad00cb0d0 msgr2=0x7f3ad006a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 shutdown_connections 2023-08-25T09:46:51.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.180+0000 7f3ad7d5d640 1 -- 172.21.15.70:0/1416335186 wait complete. 2023-08-25T09:46:51.199 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:46:51.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:46:51.304 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:46:51.431 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:46:51.432 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:46:51.432 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:51.492 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:46:51.493 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:46:51.493 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:51.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:46:51.554 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:46:51.554 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:51.614 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:46:51.614 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:46:51.615 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:46:51.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:46:51.676 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:46:51.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:46:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.901+0000 7fb60dea6640 1 Processor -- start 2023-08-25T09:46:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.902+0000 7fb60dea6640 1 -- start start 2023-08-25T09:46:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.903+0000 7fb60dea6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb60815e830 0x7fb60806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.903+0000 7fb60dea6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60806b2b0 0x7fb60815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.903+0000 7fb60dea6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6080b1b60 con 0x7fb60815c6d0 2023-08-25T09:46:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.903+0000 7fb60dea6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60806bd80 con 0x7fb60815e830 2023-08-25T09:46:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.903+0000 7fb60dea6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60806b7c0 con 0x7fb60806b2b0 2023-08-25T09:46:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.903+0000 7fb6077fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb60815e830 0x7fb60806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.904+0000 7fb6077fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb60815e830 0x7fb60806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43740/0 (socket says 172.21.15.70:43740) 2023-08-25T09:46:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.904+0000 7fb6077fe640 1 -- 172.21.15.70:0/3991844674 learned_addr learned my addr 172.21.15.70:0/3991844674 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.904+0000 7fb6077fe640 1 -- 172.21.15.70:0/3991844674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60806b2b0 msgr2=0x7fb60815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.904+0000 7fb6077fe640 1 --2- 172.21.15.70:0/3991844674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60806b2b0 0x7fb60815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.904+0000 7fb6077fe640 1 -- 172.21.15.70:0/3991844674 >> v1:172.21.15.155:6789/0 conn(0x7fb60815c6d0 legacy=0x7fb6080b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.904+0000 7fb6077fe640 1 -- 172.21.15.70:0/3991844674 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60806baa0 con 0x7fb60815e830 2023-08-25T09:46:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.905+0000 7fb6077fe640 1 --2- 172.21.15.70:0/3991844674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e830 0x7fb60806ad70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ef1438589a7b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.905+0000 7fb606ffd640 1 -- 172.21.15.70:0/3991844674 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8031640 con 0x7fb60815e830 2023-08-25T09:46:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.905+0000 7fb606ffd640 1 -- 172.21.15.70:0/3991844674 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f80317a0 con 0x7fb60815e830 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.905+0000 7fb606ffd640 1 -- 172.21.15.70:0/3991844674 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8031b30 con 0x7fb60815e830 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 -- 172.21.15.70:0/3991844674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e830 msgr2=0x7fb60806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 --2- 172.21.15.70:0/3991844674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e830 0x7fb60806ad70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 -- 172.21.15.70:0/3991844674 shutdown_connections 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 --2- 172.21.15.70:0/3991844674 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60806b2b0 0x7fb60815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 --2- 172.21.15.70:0/3991844674 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60815e830 0x7fb60806ad70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 -- 172.21.15.70:0/3991844674 >> 172.21.15.70:0/3991844674 conn(0x7fb608069850 msgr2=0x7fb608069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 -- 172.21.15.70:0/3991844674 shutdown_connections 2023-08-25T09:46:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.906+0000 7fb60dea6640 1 -- 172.21.15.70:0/3991844674 wait complete. 2023-08-25T09:46:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.908+0000 7fb60dea6640 1 Processor -- start 2023-08-25T09:46:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.908+0000 7fb60dea6640 1 -- start start 2023-08-25T09:46:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.909+0000 7fb60dea6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 0x7fb608148120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.909+0000 7fb60dea6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60815c6d0 0x7fb6080c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.909+0000 7fb60dea6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb60815e830 0x7fb60806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.909+0000 7fb60dea6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6081570a0 con 0x7fb60806b2b0 2023-08-25T09:46:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.909+0000 7fb60dea6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb608148660 con 0x7fb60815e830 2023-08-25T09:46:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.909+0000 7fb60dea6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6080ca580 con 0x7fb60815c6d0 2023-08-25T09:46:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 0x7fb608148120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 0x7fb608148120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43752/0 (socket says 172.21.15.70:43752) 2023-08-25T09:46:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 -- 172.21.15.70:0/59863450 learned_addr learned my addr 172.21.15.70:0/59863450 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 -- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60815c6d0 msgr2=0x7fb6080c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb5fffff640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60815c6d0 0x7fb6080c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60815c6d0 0x7fb6080c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb607fff640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb60815e830 0x7fb60806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 -- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb60815e830 msgr2=0x7fb60806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb60815e830 0x7fb60806ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 -- 172.21.15.70:0/59863450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80094d0 con 0x7fb60806b2b0 2023-08-25T09:46:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb5fffff640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60815c6d0 0x7fb6080c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.910+0000 7fb6077fe640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 0x7fb608148120 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.911+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f803ac60 con 0x7fb60806b2b0 2023-08-25T09:46:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.911+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f803adc0 con 0x7fb60806b2b0 2023-08-25T09:46:51.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.911+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8044630 con 0x7fb60806b2b0 2023-08-25T09:46:51.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.911+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6080ca860 con 0x7fb60806b2b0 2023-08-25T09:46:51.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.911+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb608156d30 con 0x7fb60806b2b0 2023-08-25T09:46:51.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.912+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6080cef50 con 0x7fb60806b2b0 2023-08-25T09:46:51.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.913+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb5f802f050 con 0x7fb60806b2b0 2023-08-25T09:46:51.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.914+0000 7fb604ff9640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb5e806a050 0x7fb5e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:51.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.914+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb5f80aa300 con 0x7fb60806b2b0 2023-08-25T09:46:51.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.914+0000 7fb5fffff640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb5e806a050 0x7fb5e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:51.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.915+0000 7fb5fffff640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb5e806a050 0x7fb5e806c510 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0006fd0 tx=0x7fb5f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:51.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:51.919+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb5f8075c70 con 0x7fb60806b2b0 2023-08-25T09:46:52.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.143+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb60804e420 con 0x7fb60806b2b0 2023-08-25T09:46:52.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.144+0000 7fb604ff9640 1 -- 172.21.15.70:0/59863450 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fb5f8031e60 con 0x7fb60806b2b0 2023-08-25T09:46:52.146 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:52.146 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/4064441994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":4064441994},{"type":"v1","addr":"172.21.15.70:6837","nonce":4064441994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5222,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/881198064","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":881198064},{"type":"v1","addr":"172.21.15.155:6833","nonce":881198064}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":5719,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/390590528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":390590528},{"type":"v1","addr":"172.21.15.155:6835","nonce":390590528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":8139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/4089556563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":4089556563},{"type":"v1","addr":"172.21.15.70:6836","nonce":4089556563}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[]} 2023-08-25T09:46:52.146 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 59 2023-08-25T09:46:52.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.146+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb5e806a050 msgr2=0x7fb5e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.146+0000 7fb60dea6640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb5e806a050 0x7fb5e806c510 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0006fd0 tx=0x7fb5f0008040 comp rx=0 tx=0).stop 2023-08-25T09:46:52.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 msgr2=0x7fb608148120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 0x7fb608148120 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009870 tx=0x7fb5f80046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 shutdown_connections 2023-08-25T09:46:52.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb5e806a050 0x7fb5e806c510 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb60815e830 0x7fb60806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb60815c6d0 0x7fb6080c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 --2- 172.21.15.70:0/59863450 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb60806b2b0 0x7fb608148120 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 >> 172.21.15.70:0/59863450 conn(0x7fb608069850 msgr2=0x7fb6080ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 shutdown_connections 2023-08-25T09:46:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.147+0000 7fb60dea6640 1 -- 172.21.15.70:0/59863450 wait complete. 2023-08-25T09:46:52.167 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:46:52.168 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:46:52.168 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:46:52.168 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:52.169 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:46:52.169 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:46:52.169 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:46:52.169 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:52.170 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:46:52.170 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:46:52.170 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:46:52.170 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:52.171 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:46:52.171 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:46:52.171 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:46:52.171 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:46:52.187 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:46:52.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:46:52.189 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:46:52.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:46:52.200 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:46:52.201 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:46:52.206 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:46:52.206 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:46:52.237 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:46:52.237 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:52.244 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:46:52.244 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:52.254 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:46:52.255 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:52.262 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:46:52.262 DEBUG:teuthology.parallel:result is None 2023-08-25T09:46:52.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:46:52.394 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:46:52.402 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:46:52.420 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:46:52.420 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:46:52.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.458+0000 7f7856271640 1 Processor -- start 2023-08-25T09:46:52.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.458+0000 7f7856271640 1 -- start start 2023-08-25T09:46:52.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.459+0000 7f7856271640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f785015acd0 0x7f785015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:52.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.459+0000 7f7856271640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78500b12b0 con 0x7f78500b4400 2023-08-25T09:46:52.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f7856271640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785006ae20 con 0x7f78500ca780 2023-08-25T09:46:52.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f7856271640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78500b47e0 con 0x7f785015acd0 2023-08-25T09:46:52.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f784ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f785015acd0 0x7f785015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f785480e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f78500b4400 0x7f78500c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57628/0 (socket says 172.21.15.70:57628) 2023-08-25T09:46:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f785480e640 1 -- 172.21.15.70:0/719541683 learned_addr learned my addr 172.21.15.70:0/719541683 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f784ffff640 1 -- 172.21.15.70:0/719541683 >> v1:172.21.15.155:6789/0 conn(0x7f78500ca780 legacy=0x7f78500b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f784ffff640 1 -- 172.21.15.70:0/719541683 >> v1:172.21.15.70:6789/0 conn(0x7f78500b4400 legacy=0x7f78500c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.460+0000 7f784ffff640 1 -- 172.21.15.70:0/719541683 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785006b910 con 0x7f785015acd0 2023-08-25T09:46:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.461+0000 7f784ffff640 1 --2- 172.21.15.70:0/719541683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f785015acd0 0x7f785015b0b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7840009790 tx=0x7f784002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e741a1b36ee8fe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f784effd640 1 -- 172.21.15.70:0/719541683 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784002d040 con 0x7f785015acd0 2023-08-25T09:46:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f784effd640 1 -- 172.21.15.70:0/719541683 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7840031810 con 0x7f785015acd0 2023-08-25T09:46:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f784effd640 1 -- 172.21.15.70:0/719541683 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7840031ba0 con 0x7f785015acd0 2023-08-25T09:46:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 -- 172.21.15.70:0/719541683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f785015acd0 msgr2=0x7f785015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 --2- 172.21.15.70:0/719541683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f785015acd0 0x7f785015b0b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f7840009790 tx=0x7f784002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 -- 172.21.15.70:0/719541683 shutdown_connections 2023-08-25T09:46:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 --2- 172.21.15.70:0/719541683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f785015acd0 0x7f785015b0b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 -- 172.21.15.70:0/719541683 >> 172.21.15.70:0/719541683 conn(0x7f78500ce760 msgr2=0x7f78500ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 -- 172.21.15.70:0/719541683 shutdown_connections 2023-08-25T09:46:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.462+0000 7f7856271640 1 -- 172.21.15.70:0/719541683 wait complete. 2023-08-25T09:46:52.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.464+0000 7f7856271640 1 Processor -- start 2023-08-25T09:46:52.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.464+0000 7f7856271640 1 -- start start 2023-08-25T09:46:52.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f7856271640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 0x7f7850149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:52.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f7856271640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78500ca780 0x7f78500b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:52.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f7856271640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f785015acd0 0x7f78500ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:52.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f7856271640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7850149770 con 0x7f78500b4400 2023-08-25T09:46:52.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f7856271640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78500ad640 con 0x7f785015acd0 2023-08-25T09:46:52.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f7856271640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78500ad920 con 0x7f78500ca780 2023-08-25T09:46:52.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f784ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 0x7f7850149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:52.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f784ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 0x7f7850149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43808/0 (socket says 172.21.15.70:43808) 2023-08-25T09:46:52.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.465+0000 7f784ffff640 1 -- 172.21.15.70:0/3136112408 learned_addr learned my addr 172.21.15.70:0/3136112408 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:52.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784ffff640 1 -- 172.21.15.70:0/3136112408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78500ca780 msgr2=0x7f78500b01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784ffff640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78500ca780 0x7f78500b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784ffff640 1 -- 172.21.15.70:0/3136112408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f785015acd0 msgr2=0x7f78500ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:52.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784ffff640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f785015acd0 0x7f78500ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:52.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784ffff640 1 -- 172.21.15.70:0/3136112408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400093a0 con 0x7f78500b4400 2023-08-25T09:46:52.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784ffff640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 0x7f7850149230 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7840000c00 tx=0x7f78400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:52.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784003ad60 con 0x7f78500b4400 2023-08-25T09:46:52.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78400443f0 con 0x7f78500b4400 2023-08-25T09:46:52.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78400447a0 con 0x7f78500b4400 2023-08-25T09:46:52.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.466+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78500c3940 con 0x7f78500b4400 2023-08-25T09:46:52.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.467+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78500c3f00 con 0x7f78500b4400 2023-08-25T09:46:52.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.468+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78500c40f0 con 0x7f78500b4400 2023-08-25T09:46:52.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.468+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f784004d450 con 0x7f78500b4400 2023-08-25T09:46:52.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.469+0000 7f784cff9640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f783c06a020 0x7f783c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:52.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.469+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f78400aa6b0 con 0x7f78500b4400 2023-08-25T09:46:52.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.470+0000 7f784f7fe640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f783c06a020 0x7f783c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:52.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.470+0000 7f784f7fe640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f783c06a020 0x7f783c06c4e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7844000f60 tx=0x7f78440079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:52.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.474+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7840076020 con 0x7f78500b4400 2023-08-25T09:46:52.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:52.681+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f785004e420 con 0x7f78500b4400 2023-08-25T09:46:53.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.758+0000 7f784cff9640 1 -- 172.21.15.70:0/3136112408 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v146) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7840079640 con 0x7f78500b4400 2023-08-25T09:46:53.763 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:46:53.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f783c06a020 msgr2=0x7f783c06c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:53.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f783c06a020 0x7f783c06c4e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7844000f60 tx=0x7f78440079b0 comp rx=0 tx=0).stop 2023-08-25T09:46:53.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 msgr2=0x7f7850149230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:53.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 0x7f7850149230 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7840000c00 tx=0x7f78400046c0 comp rx=0 tx=0).stop 2023-08-25T09:46:53.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 shutdown_connections 2023-08-25T09:46:53.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f783c06a020 0x7f783c06c4e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:53.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f785015acd0 0x7f78500ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:53.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78500ca780 0x7f78500b01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 --2- 172.21.15.70:0/3136112408 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78500b4400 0x7f7850149230 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 >> 172.21.15.70:0/3136112408 conn(0x7f78500ce760 msgr2=0x7f785015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 shutdown_connections 2023-08-25T09:46:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.763+0000 7f7856271640 1 -- 172.21.15.70:0/3136112408 wait complete. 2023-08-25T09:46:53.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:46:53.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.995+0000 7f0f125e1640 1 Processor -- start 2023-08-25T09:46:53.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.995+0000 7f0f125e1640 1 -- start start 2023-08-25T09:46:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f125e1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0f0c15cb40 0x7f0f0c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f125e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f0c0b2b40 0x7f0f0c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f125e1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f0c06f050 con 0x7f0f0c0ac870 2023-08-25T09:46:53.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f125e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c06f330 con 0x7f0f0c0b2b40 2023-08-25T09:46:54.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f125e1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f0c06f610 con 0x7f0f0c15cb40 2023-08-25T09:46:54.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f10b7e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f0c0b2b40 0x7f0f0c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f10b7e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0f0c0b2b40 0x7f0f0c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43820/0 (socket says 172.21.15.70:43820) 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f10b7e640 1 -- 172.21.15.70:0/78018158 learned_addr learned my addr 172.21.15.70:0/78018158 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.997+0000 7f0f0bfff640 1 --2- 172.21.15.70:0/78018158 >> v2:172.21.15.155:3300/0 conn(0x7f0f0c15cb40 0x7f0f0c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.998+0000 7f0f10b7e640 1 -- 172.21.15.70:0/78018158 >> v2:172.21.15.155:3300/0 conn(0x7f0f0c15cb40 msgr2=0x7f0f0c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.998+0000 7f0f10b7e640 1 --2- 172.21.15.70:0/78018158 >> v2:172.21.15.155:3300/0 conn(0x7f0f0c15cb40 0x7f0f0c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.998+0000 7f0f10b7e640 1 -- 172.21.15.70:0/78018158 >> v1:172.21.15.155:6789/0 conn(0x7f0f0c0ac870 legacy=0x7f0f0c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:54.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.998+0000 7f0f10b7e640 1 -- 172.21.15.70:0/78018158 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c06f8f0 con 0x7f0f0c0b2b40 2023-08-25T09:46:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.999+0000 7f0f10b7e640 1 --2- 172.21.15.70:0/78018158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c0ce860 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c06a7b0 tx=0x7f0f0402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd6fccf0f835f0b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.999+0000 7f0f0b7fe640 1 -- 172.21.15.70:0/78018158 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0402ca80 con 0x7f0f0c0b2b40 2023-08-25T09:46:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.999+0000 7f0f0b7fe640 1 -- 172.21.15.70:0/78018158 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0402cbe0 con 0x7f0f0c0b2b40 2023-08-25T09:46:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:53.999+0000 7f0f0b7fe640 1 -- 172.21.15.70:0/78018158 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f040343f0 con 0x7f0f0c0b2b40 2023-08-25T09:46:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 -- 172.21.15.70:0/78018158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c0b2b40 msgr2=0x7f0f0c0ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 --2- 172.21.15.70:0/78018158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c0ce860 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c06a7b0 tx=0x7f0f0402bee0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 -- 172.21.15.70:0/78018158 shutdown_connections 2023-08-25T09:46:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 --2- 172.21.15.70:0/78018158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c0ce860 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 --2- 172.21.15.70:0/78018158 >> v2:172.21.15.155:3300/0 conn(0x7f0f0c15cb40 0x7f0f0c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 -- 172.21.15.70:0/78018158 >> 172.21.15.70:0/78018158 conn(0x7f0f0c069850 msgr2=0x7f0f0c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 -- 172.21.15.70:0/78018158 shutdown_connections 2023-08-25T09:46:54.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.000+0000 7f0f125e1640 1 -- 172.21.15.70:0/78018158 wait complete. 2023-08-25T09:46:54.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.002+0000 7f0f125e1640 1 Processor -- start 2023-08-25T09:46:54.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.002+0000 7f0f125e1640 1 -- start start 2023-08-25T09:46:54.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.003+0000 7f0f125e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f0c0ac870 0x7f0f0c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.003+0000 7f0f125e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.003+0000 7f0f125e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 0x7f0f0c0af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.003+0000 7f0f125e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c14ac40 con 0x7f0f0c15cb40 2023-08-25T09:46:54.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.003+0000 7f0f125e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c157980 con 0x7f0f0c0b2b40 2023-08-25T09:46:54.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.003+0000 7f0f125e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c158500 con 0x7f0f0c0ac870 2023-08-25T09:46:54.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 0x7f0f0c0af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 0x7f0f0c0af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43828/0 (socket says 172.21.15.70:43828) 2023-08-25T09:46:54.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 -- 172.21.15.70:0/1362849814 learned_addr learned my addr 172.21.15.70:0/1362849814 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:54.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f03fff640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f0bfff640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f0c0ac870 0x7f0f0c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 -- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f0c0ac870 msgr2=0x7f0f0c14b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f0c0ac870 0x7f0f0c14b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 -- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f0c0b2b40 msgr2=0x7f0f0c153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c153240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 -- 172.21.15.70:0/1362849814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f04009b30 con 0x7f0f0c15cb40 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f03fff640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.004+0000 7f0f10b7e640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 0x7f0f0c0af540 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009f40 tx=0x7f0f04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.005+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f04009510 con 0x7f0f0c15cb40 2023-08-25T09:46:54.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.005+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0403e450 con 0x7f0f0c15cb40 2023-08-25T09:46:54.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.005+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0403e7e0 con 0x7f0f0c15cb40 2023-08-25T09:46:54.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.005+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0c153780 con 0x7f0f0c15cb40 2023-08-25T09:46:54.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.005+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0c157eb0 con 0x7f0f0c15cb40 2023-08-25T09:46:54.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.006+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f0c158690 con 0x7f0f0c15cb40 2023-08-25T09:46:54.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.007+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0f0402cd50 con 0x7f0f0c15cb40 2023-08-25T09:46:54.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.008+0000 7f0f097fa640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0eec06a050 0x7f0eec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.008+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0f040ab0f0 con 0x7f0f0c15cb40 2023-08-25T09:46:54.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.008+0000 7f0f0bfff640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0eec06a050 0x7f0eec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.009+0000 7f0f0bfff640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0eec06a050 0x7f0eec06c510 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0efc006fd0 tx=0x7f0efc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.012+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0f04076a90 con 0x7f0f0c15cb40 2023-08-25T09:46:54.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.271+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0f0c04e420 con 0x7f0f0c15cb40 2023-08-25T09:46:54.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.272+0000 7f0f097fa640 1 -- 172.21.15.70:0/1362849814 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0f0407a0b0 con 0x7f0f0c15cb40 2023-08-25T09:46:54.275 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:54.275 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:46:54.275 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:46:54.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.275 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.277 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:54.278 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:54.278 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.278 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.279 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.280 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:46:54.281 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:46:54.282 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.283 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.286 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:46:54.288 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:46:54.289 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:46:54.292 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.293 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.294 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:46:54.294 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:46:54.294 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:46:54.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:46:54.295 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.296 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:46:54.296 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:46:54.296 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:46:54.297 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:54.298 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:54.300 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:46:54.300 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:46:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0eec06a050 msgr2=0x7f0eec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0eec06a050 0x7f0eec06c510 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0efc006fd0 tx=0x7f0efc008040 comp rx=0 tx=0).stop 2023-08-25T09:46:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 msgr2=0x7f0f0c0af540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 0x7f0f0c0af540 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009f40 tx=0x7f0f04004a40 comp rx=0 tx=0).stop 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 shutdown_connections 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0eec06a050 0x7f0eec06c510 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0f0c15cb40 0x7f0f0c0af540 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0f0c0b2b40 0x7f0f0c153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 --2- 172.21.15.70:0/1362849814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0f0c0ac870 0x7f0f0c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 >> 172.21.15.70:0/1362849814 conn(0x7f0f0c069850 msgr2=0x7f0f0c160c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 shutdown_connections 2023-08-25T09:46:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.275+0000 7f0f125e1640 1 -- 172.21.15.70:0/1362849814 wait complete. 2023-08-25T09:46:54.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:54.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.515+0000 7fa06b396640 1 Processor -- start 2023-08-25T09:46:54.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.515+0000 7fa06b396640 1 -- start start 2023-08-25T09:46:54.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.517+0000 7fa06b396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa06406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.517+0000 7fa06b396640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0640b1b60 con 0x7fa06415ec10 2023-08-25T09:46:54.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.517+0000 7fa06b396640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa06406b7c0 con 0x7fa06406b2b0 2023-08-25T09:46:54.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.517+0000 7fa06b396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06406bd80 con 0x7fa06415e830 2023-08-25T09:46:54.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa069933640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa06415ec10 0x7fa0640b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57638/0 (socket says 172.21.15.70:57638) 2023-08-25T09:46:54.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa069933640 1 -- 172.21.15.70:0/273221848 learned_addr learned my addr 172.21.15.70:0/273221848 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:54.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa069132640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa06406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908025449 0 0) 0x7fa0640b1b60 con 0x7fa06415ec10 2023-08-25T09:46:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa050003520 con 0x7fa06415ec10 2023-08-25T09:46:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3719654468 0 0) 0x7fa050003520 con 0x7fa06415ec10 2023-08-25T09:46:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 msgr2=0x7fa06406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa06406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 >> v1:172.21.15.155:6789/0 conn(0x7fa06406b2b0 legacy=0x7fa06415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.518+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06406baa0 con 0x7fa06415ec10 2023-08-25T09:46:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa05c0019f0 con 0x7fa06415ec10 2023-08-25T09:46:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 >> v1:172.21.15.70:6789/0 conn(0x7fa06415ec10 legacy=0x7fa0640b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0500039a0 0x7fa050003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa0500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa050005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06406bd80 con 0x7fa050004d00 2023-08-25T09:46:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05000ba30 con 0x7fa0500039a0 2023-08-25T09:46:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05000bbf0 con 0x7fa06415e830 2023-08-25T09:46:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069933640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa050005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069132640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0500039a0 0x7fa050003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa068931640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa0500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069933640 1 -- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 msgr2=0x7fa0500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069933640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa0500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069933640 1 -- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0500039a0 msgr2=0x7fa050003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069933640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0500039a0 0x7fa050003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa069933640 1 -- 172.21.15.70:0/273221848 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa050003520 con 0x7fa050004d00 2023-08-25T09:46:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.519+0000 7fa068931640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa06415e830 0x7fa0500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:54.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.520+0000 7fa069933640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa050005180 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00a7f0 tx=0x7fa0500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e966c2f91036cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0072d0 con 0x7fa050004d00 2023-08-25T09:46:54.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c02f070 con 0x7fa050004d00 2023-08-25T09:46:54.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa05bfff640 1 -- 172.21.15.70:0/273221848 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0076b0 con 0x7fa050004d00 2023-08-25T09:46:54.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa06b396640 1 -- 172.21.15.70:0/273221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 msgr2=0x7fa050005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa06b396640 1 --2- 172.21.15.70:0/273221848 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa050005180 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa05c00a7f0 tx=0x7fa0500057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa069933640 1 -- 172.21.15.70:0/273221848 reap_dead start 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa06b396640 1 -- 172.21.15.70:0/273221848 shutdown_connections 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.521+0000 7fa06b396640 1 -- 172.21.15.70:0/273221848 >> 172.21.15.70:0/273221848 conn(0x7fa064069850 msgr2=0x7fa064069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.522+0000 7fa06b396640 1 -- 172.21.15.70:0/273221848 shutdown_connections 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.522+0000 7fa06b396640 1 -- 172.21.15.70:0/273221848 wait complete. 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.523+0000 7fa06b396640 1 Processor -- start 2023-08-25T09:46:54.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.524+0000 7fa06b396640 1 -- start start 2023-08-25T09:46:54.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa06b396640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa064154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa06b396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa064150cd0 0x7fa0641f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa06b396640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0641f6720 0x7fa06415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa06b396640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa064154670 con 0x7fa050004d00 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa06b396640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa064154fd0 con 0x7fa0641f6720 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa06b396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0641f6ca0 con 0x7fa064150cd0 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa064154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa064154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43860/0 (socket says 172.21.15.70:43860) 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 -- 172.21.15.70:0/3652582555 learned_addr learned my addr 172.21.15.70:0/3652582555 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa068931640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa064150cd0 0x7fa0641f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069933640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0641f6720 0x7fa06415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 -- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa064150cd0 msgr2=0x7fa0641f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa064150cd0 0x7fa0641f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 -- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0641f6720 msgr2=0x7fa06415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0641f6720 0x7fa06415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.525+0000 7fa069132640 1 -- 172.21.15.70:0/3652582555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06415ca20 con 0x7fa050004d00 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.526+0000 7fa069132640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa064154a90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa060011600 tx=0x7fa060011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.526+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000f920 con 0x7fa050004d00 2023-08-25T09:46:54.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.526+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06000fa80 con 0x7fa050004d00 2023-08-25T09:46:54.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.526+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000fdf0 con 0x7fa050004d00 2023-08-25T09:46:54.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.526+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0641f6f80 con 0x7fa050004d00 2023-08-25T09:46:54.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.526+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa064151740 con 0x7fa050004d00 2023-08-25T09:46:54.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.527+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0641548e0 con 0x7fa050004d00 2023-08-25T09:46:54.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.528+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa060015460 con 0x7fa050004d00 2023-08-25T09:46:54.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.529+0000 7fa059ffb640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa04806a050 0x7fa04806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.529+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa060088810 con 0x7fa050004d00 2023-08-25T09:46:54.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.530+0000 7fa068931640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa04806a050 0x7fa04806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.530+0000 7fa068931640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa04806a050 0x7fa04806c510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa054007fd0 tx=0x7fa054009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.533+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa0600541b0 con 0x7fa050004d00 2023-08-25T09:46:54.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.723+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa06404e420 con 0x7fa050004d00 2023-08-25T09:46:54.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.725+0000 7fa059ffb640 1 -- 172.21.15.70:0/3652582555 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+14665 (secure 0 0 0) 0x7fa0600577d0 con 0x7fa050004d00 2023-08-25T09:46:54.727 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:54.728 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":146,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:53.752585+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:54.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa04806a050 msgr2=0x7fa04806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa04806a050 0x7fa04806c510 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa054007fd0 tx=0x7fa054009040 comp rx=0 tx=0).stop 2023-08-25T09:46:54.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 msgr2=0x7fa064154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa064154a90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa060011600 tx=0x7fa060011ac0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 shutdown_connections 2023-08-25T09:46:54.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa04806a050 0x7fa04806c510 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa0641f6720 0x7fa06415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa064150cd0 0x7fa0641f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 --2- 172.21.15.70:0/3652582555 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa050004d00 0x7fa064154a90 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 >> 172.21.15.70:0/3652582555 conn(0x7fa064069850 msgr2=0x7fa06406ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:54.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 shutdown_connections 2023-08-25T09:46:54.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.728+0000 7fa06b396640 1 -- 172.21.15.70:0/3652582555 wait complete. 2023-08-25T09:46:54.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:46:54.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:46:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.965+0000 7f2188c6a640 1 Processor -- start 2023-08-25T09:46:54.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.965+0000 7f2188c6a640 1 -- start start 2023-08-25T09:46:54.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2188c6a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f21840c2110 0x7f2184069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2188c6a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2184069d90 0x7f21840ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2188c6a640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21840c2550 con 0x7f218415ae50 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2188c6a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218406a200 con 0x7f2184069d90 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2188c6a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21840ce330 con 0x7f21840c2110 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2182d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2184069d90 0x7f21840ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2182d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2184069d90 0x7f21840ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43864/0 (socket says 172.21.15.70:43864) 2023-08-25T09:46:54.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.967+0000 7f2182d9d640 1 -- 172.21.15.70:0/3230248283 learned_addr learned my addr 172.21.15.70:0/3230248283 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:54.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.968+0000 7f217bfff640 1 --2- 172.21.15.70:0/3230248283 >> v2:172.21.15.155:3300/0 conn(0x7f21840c2110 0x7f2184069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.968+0000 7f2182d9d640 1 -- 172.21.15.70:0/3230248283 >> v2:172.21.15.155:3300/0 conn(0x7f21840c2110 msgr2=0x7f2184069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.968+0000 7f2182d9d640 1 --2- 172.21.15.70:0/3230248283 >> v2:172.21.15.155:3300/0 conn(0x7f21840c2110 0x7f2184069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.968+0000 7f2182d9d640 1 -- 172.21.15.70:0/3230248283 >> v1:172.21.15.70:6789/0 conn(0x7f218415ae50 legacy=0x7f21840cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.968+0000 7f2182d9d640 1 -- 172.21.15.70:0/3230248283 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840cc5f0 con 0x7f2184069d90 2023-08-25T09:46:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.968+0000 7f217bfff640 1 --2- 172.21.15.70:0/3230248283 >> v2:172.21.15.155:3300/0 conn(0x7f21840c2110 0x7f2184069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.969+0000 7f2182d9d640 1 --2- 172.21.15.70:0/3230248283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2184069d90 0x7f21840ccad0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f21840cb030 tx=0x7f217c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d33ffac0cf4a16 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.969+0000 7f2181d9b640 1 -- 172.21.15.70:0/3230248283 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c02cbb0 con 0x7f2184069d90 2023-08-25T09:46:54.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.969+0000 7f2181d9b640 1 -- 172.21.15.70:0/3230248283 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c02cd10 con 0x7f2184069d90 2023-08-25T09:46:54.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.969+0000 7f2181d9b640 1 -- 172.21.15.70:0/3230248283 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c034610 con 0x7f2184069d90 2023-08-25T09:46:54.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 -- 172.21.15.70:0/3230248283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2184069d90 msgr2=0x7f21840ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 --2- 172.21.15.70:0/3230248283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2184069d90 0x7f21840ccad0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f21840cb030 tx=0x7f217c02c010 comp rx=0 tx=0).stop 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 -- 172.21.15.70:0/3230248283 shutdown_connections 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 --2- 172.21.15.70:0/3230248283 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2184069d90 0x7f21840ccad0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 --2- 172.21.15.70:0/3230248283 >> v2:172.21.15.155:3300/0 conn(0x7f21840c2110 0x7f2184069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 -- 172.21.15.70:0/3230248283 >> 172.21.15.70:0/3230248283 conn(0x7f21840cb0d0 msgr2=0x7f21840cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 -- 172.21.15.70:0/3230248283 shutdown_connections 2023-08-25T09:46:54.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.970+0000 7f2188c6a640 1 -- 172.21.15.70:0/3230248283 wait complete. 2023-08-25T09:46:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.973+0000 7f2188c6a640 1 Processor -- start 2023-08-25T09:46:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.973+0000 7f2188c6a640 1 -- start start 2023-08-25T09:46:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2188c6a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2184069d90 0x7f21840c3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2188c6a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840c2110 0x7f21840acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2188c6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 0x7f218414d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2188c6a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21841f5bb0 con 0x7f218415ae50 2023-08-25T09:46:54.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2188c6a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21841f5e90 con 0x7f2184069d90 2023-08-25T09:46:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2188c6a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21841f6170 con 0x7f21840c2110 2023-08-25T09:46:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.974+0000 7f2182d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 0x7f218414d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 0x7f218414d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43878/0 (socket says 172.21.15.70:43878) 2023-08-25T09:46:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 -- 172.21.15.70:0/145370403 learned_addr learned my addr 172.21.15.70:0/145370403 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:54.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f218259c640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2184069d90 0x7f21840c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 -- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840c2110 msgr2=0x7f21840acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840c2110 0x7f21840acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 -- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2184069d90 msgr2=0x7f21840c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2184069d90 0x7f21840c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 -- 172.21.15.70:0/145370403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c009b30 con 0x7f218415ae50 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f2182d9d640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 0x7f218414d950 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f217c036040 tx=0x7f217c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.975+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c009510 con 0x7f218415ae50 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.976+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c03e450 con 0x7f218415ae50 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.976+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c03e800 con 0x7f218415ae50 2023-08-25T09:46:54.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.976+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21841f6450 con 0x7f218415ae50 2023-08-25T09:46:54.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.976+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21840c3a50 con 0x7f218415ae50 2023-08-25T09:46:54.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.977+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21840c3c10 con 0x7f218415ae50 2023-08-25T09:46:54.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.978+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f217c03e960 con 0x7f218415ae50 2023-08-25T09:46:54.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.979+0000 7f217affd640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f216406a050 0x7f216406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:54.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.979+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f217c0ab0e0 con 0x7f218415ae50 2023-08-25T09:46:54.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.979+0000 7f218259c640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f216406a050 0x7f216406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:54.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.980+0000 7f218259c640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f216406a050 0x7f216406c510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2174000f60 tx=0x7f2174009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:54.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:54.983+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f217c076ce0 con 0x7f218415ae50 2023-08-25T09:46:55.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.178+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2184066fb0 con 0x7f218415ae50 2023-08-25T09:46:55.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.179+0000 7f217affd640 1 -- 172.21.15.70:0/145370403 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v146) v1 ==== 93+0+10 (secure 0 0 0) 0x7f217c07a300 con 0x7f218415ae50 2023-08-25T09:46:55.182 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:46:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.181+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f216406a050 msgr2=0x7f216406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.181+0000 7f2188c6a640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f216406a050 0x7f216406c510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2174000f60 tx=0x7f2174009040 comp rx=0 tx=0).stop 2023-08-25T09:46:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.181+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 msgr2=0x7f218414d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.181+0000 7f2188c6a640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 0x7f218414d950 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f217c036040 tx=0x7f217c004a40 comp rx=0 tx=0).stop 2023-08-25T09:46:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 shutdown_connections 2023-08-25T09:46:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f216406a050 0x7f216406c510 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f218415ae50 0x7f218414d950 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f21840c2110 0x7f21840acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 --2- 172.21.15.70:0/145370403 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2184069d90 0x7f21840c3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 >> 172.21.15.70:0/145370403 conn(0x7f21840cb0d0 msgr2=0x7f2184159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 shutdown_connections 2023-08-25T09:46:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.182+0000 7f2188c6a640 1 -- 172.21.15.70:0/145370403 wait complete. 2023-08-25T09:46:55.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:46:55.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.417+0000 7f4f6a9c4640 1 Processor -- start 2023-08-25T09:46:55.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.417+0000 7f4f6a9c4640 1 -- start start 2023-08-25T09:46:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.418+0000 7f4f6a9c4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f64159bf0 0x7f4f6415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.418+0000 7f4f6a9c4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4f6406f0e0 0x7f4f64070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.418+0000 7f4f6a9c4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f640cd660 0x7f4f640cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.418+0000 7f4f6a9c4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415e780 con 0x7f4f640cd660 2023-08-25T09:46:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f6a9c4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f640ce000 con 0x7f4f6406f0e0 2023-08-25T09:46:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f6a9c4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6406f8e0 con 0x7f4f64159bf0 2023-08-25T09:46:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f640cd660 0x7f4f640cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4f640cd660 0x7f4f640cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43886/0 (socket says 172.21.15.70:43886) 2023-08-25T09:46:55.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 -- 172.21.15.70:0/1482277842 learned_addr learned my addr 172.21.15.70:0/1482277842 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f5bfff640 1 --2- 172.21.15.70:0/1482277842 >> v2:172.21.15.155:3300/0 conn(0x7f4f6406f0e0 0x7f4f64070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f63fff640 1 --2- 172.21.15.70:0/1482277842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f64159bf0 0x7f4f6415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 -- 172.21.15.70:0/1482277842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f64159bf0 msgr2=0x7f4f6415e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 --2- 172.21.15.70:0/1482277842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f64159bf0 0x7f4f6415e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 -- 172.21.15.70:0/1482277842 >> v2:172.21.15.155:3300/0 conn(0x7f4f6406f0e0 msgr2=0x7f4f64070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 --2- 172.21.15.70:0/1482277842 >> v2:172.21.15.155:3300/0 conn(0x7f4f6406f0e0 0x7f4f64070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.419+0000 7f4f68f61640 1 -- 172.21.15.70:0/1482277842 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6406fbc0 con 0x7f4f640cd660 2023-08-25T09:46:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.421+0000 7f4f68f61640 1 --2- 172.21.15.70:0/1482277842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f640cdac0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4f64071d50 tx=0x7f4f5c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18e88fba35004b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:55.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.421+0000 7f4f637fe640 1 -- 172.21.15.70:0/1482277842 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c02ca70 con 0x7f4f640cd660 2023-08-25T09:46:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.421+0000 7f4f637fe640 1 -- 172.21.15.70:0/1482277842 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c02cbd0 con 0x7f4f640cd660 2023-08-25T09:46:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.421+0000 7f4f637fe640 1 -- 172.21.15.70:0/1482277842 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0343f0 con 0x7f4f640cd660 2023-08-25T09:46:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1482277842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 msgr2=0x7f4f640cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1482277842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f640cdac0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f4f64071d50 tx=0x7f4f5c02bed0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1482277842 shutdown_connections 2023-08-25T09:46:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1482277842 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f640cdac0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1482277842 >> v2:172.21.15.155:3300/0 conn(0x7f4f6406f0e0 0x7f4f64070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1482277842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f64159bf0 0x7f4f6415e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1482277842 >> 172.21.15.70:0/1482277842 conn(0x7f4f640cb0d0 msgr2=0x7f4f640cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:55.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1482277842 shutdown_connections 2023-08-25T09:46:55.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.422+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1482277842 wait complete. 2023-08-25T09:46:55.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.424+0000 7f4f6a9c4640 1 Processor -- start 2023-08-25T09:46:55.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.424+0000 7f4f6a9c4640 1 -- start start 2023-08-25T09:46:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f6a9c4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f6406f0e0 0x7f4f6415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f6a9c4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f64154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f6a9c4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f64159bf0 0x7f4f640c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f6a9c4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c5690 con 0x7f4f640cd660 2023-08-25T09:46:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f6a9c4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415b910 con 0x7f4f64159bf0 2023-08-25T09:46:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f6a9c4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c33c0 con 0x7f4f6406f0e0 2023-08-25T09:46:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f64154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f64154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43898/0 (socket says 172.21.15.70:43898) 2023-08-25T09:46:55.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 -- 172.21.15.70:0/1355842224 learned_addr learned my addr 172.21.15.70:0/1355842224 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:55.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f68f61640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f64159bf0 0x7f4f640c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f63fff640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f6406f0e0 0x7f4f6415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 -- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f6406f0e0 msgr2=0x7f4f6415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f6406f0e0 0x7f4f6415b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 -- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f64159bf0 msgr2=0x7f4f640c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f64159bf0 0x7f4f640c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.425+0000 7f4f5bfff640 1 -- 172.21.15.70:0/1355842224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c009b30 con 0x7f4f640cd660 2023-08-25T09:46:55.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.426+0000 7f4f5bfff640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f64154f40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00d620 tx=0x7f4f4c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:55.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.426+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00b620 con 0x7f4f640cd660 2023-08-25T09:46:55.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.426+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4c00b780 con 0x7f4f640cd660 2023-08-25T09:46:55.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.426+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c00bb30 con 0x7f4f640cd660 2023-08-25T09:46:55.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.426+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640c81b0 con 0x7f4f640cd660 2023-08-25T09:46:55.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.426+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640c58e0 con 0x7f4f640cd660 2023-08-25T09:46:55.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.427+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f64066fb0 con 0x7f4f640cd660 2023-08-25T09:46:55.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.428+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4f4c00bc90 con 0x7f4f640cd660 2023-08-25T09:46:55.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.429+0000 7f4f617fa640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4f4406a080 0x7f4f4406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.429+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4f4c0886b0 con 0x7f4f640cd660 2023-08-25T09:46:55.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.429+0000 7f4f63fff640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4f4406a080 0x7f4f4406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.430+0000 7f4f63fff640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4f4406a080 0x7f4f4406c540 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4f54006fd0 tx=0x7f4f54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:55.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.433+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4f4c0540e0 con 0x7f4f640cd660 2023-08-25T09:46:55.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.614+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4f640ca290 con 0x7f4f640cd660 2023-08-25T09:46:55.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.615+0000 7f4f617fa640 1 -- 172.21.15.70:0/1355842224 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v146) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4f4c057700 con 0x7f4f640cd660 2023-08-25T09:46:55.618 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:46:55.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4f4406a080 msgr2=0x7f4f4406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4f4406a080 0x7f4f4406c540 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4f54006fd0 tx=0x7f4f54008040 comp rx=0 tx=0).stop 2023-08-25T09:46:55.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 msgr2=0x7f4f64154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f64154f40 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c00d620 tx=0x7f4f4c00dae0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 shutdown_connections 2023-08-25T09:46:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4f4406a080 0x7f4f4406c540 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4f64159bf0 0x7f4f640c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4f640cd660 0x7f4f64154f40 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 --2- 172.21.15.70:0/1355842224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4f6406f0e0 0x7f4f6415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 >> 172.21.15.70:0/1355842224 conn(0x7f4f640cb0d0 msgr2=0x7f4f64155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:55.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.618+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 shutdown_connections 2023-08-25T09:46:55.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.619+0000 7f4f6a9c4640 1 -- 172.21.15.70:0/1355842224 wait complete. 2023-08-25T09:46:55.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:46:55.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.845+0000 7f9a6f9b5640 1 Processor -- start 2023-08-25T09:46:55.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.845+0000 7f9a6f9b5640 1 -- start start 2023-08-25T09:46:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6f9b5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9a6815e830 0x7f9a6806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6f9b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 0x7f9a6815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6f9b5640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a680b1b60 con 0x7f9a6815c6d0 2023-08-25T09:46:55.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6f9b5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6806bd80 con 0x7f9a6815e830 2023-08-25T09:46:55.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6f9b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6806b7c0 con 0x7f9a6806b2b0 2023-08-25T09:46:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6df52640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9a6815c6d0 0x7f9a680b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57644/0 (socket says 172.21.15.70:57644) 2023-08-25T09:46:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.847+0000 7f9a6df52640 1 -- 172.21.15.70:0/4285167606 learned_addr learned my addr 172.21.15.70:0/4285167606 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 0x7f9a6815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a6d751640 1 --2- 172.21.15.70:0/4285167606 >> v2:172.21.15.155:3300/0 conn(0x7f9a6815e830 0x7f9a6806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704003005 0 0) 0x7f9a680b1b60 con 0x7f9a6815c6d0 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a54003520 con 0x7f9a6815c6d0 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421597025 0 0) 0x7f9a54003520 con 0x7f9a6815c6d0 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 msgr2=0x7f9a6815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 0x7f9a6815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 >> v2:172.21.15.155:3300/0 conn(0x7f9a6815e830 msgr2=0x7f9a6806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 --2- 172.21.15.70:0/4285167606 >> v2:172.21.15.155:3300/0 conn(0x7f9a6815e830 0x7f9a6806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6806baa0 con 0x7f9a6815c6d0 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9a600019f0 con 0x7f9a6815c6d0 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 >> v1:172.21.15.70:6789/0 conn(0x7f9a6815c6d0 legacy=0x7f9a680b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 0x7f9a54003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004d00 0x7f9a54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6806bd80 con 0x7f9a54003ef0 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6806b7c0 con 0x7f9a54004d00 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.848+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5400b980 con 0x7f9a6806b2b0 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6cf50640 1 -- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 msgr2=0x7f9a54003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6d751640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 0x7f9a54003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a6806b2b0 0x7f9a54003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6df52640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004d00 0x7f9a54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6cf50640 1 -- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004d00 msgr2=0x7f9a54005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:55.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a54004d00 0x7f9a54005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.849+0000 7f9a6cf50640 1 -- 172.21.15.70:0/4285167606 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54003520 con 0x7f9a54003ef0 2023-08-25T09:46:55.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.850+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a540047c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009a90 tx=0x7f9a540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616d9e5041b45f4e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:55.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.850+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5802c6b0 con 0x7f9a54003ef0 2023-08-25T09:46:55.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.850+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5802c810 con 0x7f9a54003ef0 2023-08-25T09:46:55.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.850+0000 7f9a5ffff640 1 -- 172.21.15.70:0/4285167606 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5802cba0 con 0x7f9a54003ef0 2023-08-25T09:46:55.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/4285167606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 msgr2=0x7f9a540047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/4285167606 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a540047c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009a90 tx=0x7f9a540057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6df52640 1 -- 172.21.15.70:0/4285167606 reap_dead start 2023-08-25T09:46:55.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/4285167606 shutdown_connections 2023-08-25T09:46:55.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/4285167606 >> 172.21.15.70:0/4285167606 conn(0x7f9a68069850 msgr2=0x7f9a68069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:55.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/4285167606 shutdown_connections 2023-08-25T09:46:55.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.851+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/4285167606 wait complete. 2023-08-25T09:46:55.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.853+0000 7f9a6f9b5640 1 Processor -- start 2023-08-25T09:46:55.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.853+0000 7f9a6f9b5640 1 -- start start 2023-08-25T09:46:55.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6f9b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a68147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6f9b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a680cd390 0x7f9a6806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6f9b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a680a8460 0x7f9a680a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6f9b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68147d70 con 0x7f9a54003ef0 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6f9b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a680a8e20 con 0x7f9a680a8460 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6f9b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68147fd0 con 0x7f9a680cd390 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6d751640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a68147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6d751640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a68147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43914/0 (socket says 172.21.15.70:43914) 2023-08-25T09:46:55.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.854+0000 7f9a6d751640 1 -- 172.21.15.70:0/3224798992 learned_addr learned my addr 172.21.15.70:0/3224798992 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:55.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6d751640 1 -- 172.21.15.70:0/3224798992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a680cd390 msgr2=0x7f9a6806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6d751640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a680cd390 0x7f9a6806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6d751640 1 -- 172.21.15.70:0/3224798992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a680a8460 msgr2=0x7f9a680a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6d751640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a680a8460 0x7f9a680a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6d751640 1 -- 172.21.15.70:0/3224798992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6806ff00 con 0x7f9a54003ef0 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6d751640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a68147830 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9a640108e0 tx=0x7f9a64010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400eb60 con 0x7f9a54003ef0 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400ecc0 con 0x7f9a54003ef0 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6401c400 con 0x7f9a54003ef0 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.855+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a680701e0 con 0x7f9a54003ef0 2023-08-25T09:46:55.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.856+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a68148220 con 0x7f9a54003ef0 2023-08-25T09:46:55.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.857+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6804e420 con 0x7f9a54003ef0 2023-08-25T09:46:55.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.857+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a6401c560 con 0x7f9a54003ef0 2023-08-25T09:46:55.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.858+0000 7f9a5dffb640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9a4c06a080 0x7f9a4c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:55.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.858+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9a640888f0 con 0x7f9a54003ef0 2023-08-25T09:46:55.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.859+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9a4c06a080 0x7f9a4c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:55.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.859+0000 7f9a6cf50640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9a4c06a080 0x7f9a4c06c540 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a58036000 tx=0x7f9a5802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:55.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:55.863+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a640543d0 con 0x7f9a54003ef0 2023-08-25T09:46:56.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.045+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9a6806c620 con 0x7f9a54003ef0 2023-08-25T09:46:56.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.046+0000 7f9a5dffb640 1 -- 172.21.15.70:0/3224798992 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v146) v1 ==== 84+0+126 (secure 0 0 0) 0x7f9a640579f0 con 0x7f9a54003ef0 2023-08-25T09:46:56.049 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:56.049 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:46:56.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:46:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:46:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:46:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:46:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:46:56.050 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:46:56.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:46:56.051 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:46:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9a4c06a080 msgr2=0x7f9a4c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9a4c06a080 0x7f9a4c06c540 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a58036000 tx=0x7f9a5802e000 comp rx=0 tx=0).stop 2023-08-25T09:46:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 msgr2=0x7f9a68147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a68147830 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9a640108e0 tx=0x7f9a64010da0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 shutdown_connections 2023-08-25T09:46:56.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9a4c06a080 0x7f9a4c06c540 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a680a8460 0x7f9a680a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a680cd390 0x7f9a6806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 --2- 172.21.15.70:0/3224798992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9a54003ef0 0x7f9a68147830 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 >> 172.21.15.70:0/3224798992 conn(0x7f9a68069850 msgr2=0x7f9a6815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 shutdown_connections 2023-08-25T09:46:56.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.049+0000 7f9a6f9b5640 1 -- 172.21.15.70:0/3224798992 wait complete. 2023-08-25T09:46:56.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:46:56.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:46:56.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.285+0000 7f1e3d0a6640 1 Processor -- start 2023-08-25T09:46:56.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.285+0000 7f1e3d0a6640 1 -- start start 2023-08-25T09:46:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.286+0000 7f1e3d0a6640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1e38155120 0x7f1e380cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3d0a6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380cd890 0x7f1e380ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3d0a6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380cacb0 con 0x7f1e380ca820 2023-08-25T09:46:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3d0a6640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380caf30 con 0x7f1e38155120 2023-08-25T09:46:56.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3d0a6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380cdd10 con 0x7f1e380cd890 2023-08-25T09:46:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3759e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1e380ca820 0x7f1e3806aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57654/0 (socket says 172.21.15.70:57654) 2023-08-25T09:46:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3759e640 1 -- 172.21.15.70:0/215634024 learned_addr learned my addr 172.21.15.70:0/215634024 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e36d9d640 1 --2- 172.21.15.70:0/215634024 >> v2:172.21.15.155:3300/0 conn(0x7f1e38155120 0x7f1e380cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e3659c640 1 --2- 172.21.15.70:0/215634024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380cd890 0x7f1e380ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.287+0000 7f1e35d9b640 1 -- 172.21.15.70:0/215634024 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101544695 0 0) 0x7f1e380cacb0 con 0x7f1e380ca820 2023-08-25T09:46:56.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.288+0000 7f1e35d9b640 1 -- 172.21.15.70:0/215634024 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e24003520 con 0x7f1e380ca820 2023-08-25T09:46:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.288+0000 7f1e3659c640 1 -- 172.21.15.70:0/215634024 >> v2:172.21.15.155:3300/0 conn(0x7f1e38155120 msgr2=0x7f1e380cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.288+0000 7f1e3659c640 1 --2- 172.21.15.70:0/215634024 >> v2:172.21.15.155:3300/0 conn(0x7f1e38155120 0x7f1e380cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.288+0000 7f1e3659c640 1 -- 172.21.15.70:0/215634024 >> v1:172.21.15.70:6789/0 conn(0x7f1e380ca820 legacy=0x7f1e3806aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.288+0000 7f1e3659c640 1 -- 172.21.15.70:0/215634024 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3815e6f0 con 0x7f1e380cd890 2023-08-25T09:46:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.289+0000 7f1e3659c640 1 --2- 172.21.15.70:0/215634024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380cd890 0x7f1e380ca2e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e200097a0 tx=0x7f1e2002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909635b5a8e9c6d1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.290+0000 7f1e35d9b640 1 -- 172.21.15.70:0/215634024 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2002d040 con 0x7f1e380cd890 2023-08-25T09:46:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.290+0000 7f1e35d9b640 1 -- 172.21.15.70:0/215634024 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20031810 con 0x7f1e380cd890 2023-08-25T09:46:56.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.290+0000 7f1e35d9b640 1 -- 172.21.15.70:0/215634024 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20031ba0 con 0x7f1e380cd890 2023-08-25T09:46:56.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/215634024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380cd890 msgr2=0x7f1e380ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/215634024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380cd890 0x7f1e380ca2e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1e200097a0 tx=0x7f1e2002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/215634024 shutdown_connections 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/215634024 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380cd890 0x7f1e380ca2e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/215634024 >> v2:172.21.15.155:3300/0 conn(0x7f1e38155120 0x7f1e380cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/215634024 >> 172.21.15.70:0/215634024 conn(0x7f1e38069c70 msgr2=0x7f1e3806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/215634024 shutdown_connections 2023-08-25T09:46:56.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.291+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/215634024 wait complete. 2023-08-25T09:46:56.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.293+0000 7f1e3d0a6640 1 Processor -- start 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.293+0000 7f1e3d0a6640 1 -- start start 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.294+0000 7f1e3d0a6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380ca820 0x7f1e38150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.294+0000 7f1e3d0a6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e380cd890 0x7f1e380c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.294+0000 7f1e3d0a6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 0x7f1e380c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.294+0000 7f1e3d0a6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380c9ab0 con 0x7f1e38155120 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.294+0000 7f1e3d0a6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381513f0 con 0x7f1e380cd890 2023-08-25T09:46:56.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.294+0000 7f1e3d0a6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380c9620 con 0x7f1e380ca820 2023-08-25T09:46:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 0x7f1e380c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 0x7f1e380c5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43916/0 (socket says 172.21.15.70:43916) 2023-08-25T09:46:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 -- 172.21.15.70:0/926465525 learned_addr learned my addr 172.21.15.70:0/926465525 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3659c640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e380cd890 0x7f1e380c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e36d9d640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380ca820 0x7f1e38150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:56.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 -- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380ca820 msgr2=0x7f1e38150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380ca820 0x7f1e38150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 -- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e380cd890 msgr2=0x7f1e380c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e380cd890 0x7f1e380c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 -- 172.21.15.70:0/926465525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e20009400 con 0x7f1e38155120 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.295+0000 7f1e3759e640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 0x7f1e380c5ee0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c390 tx=0x7f1e3000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.296+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30005ce0 con 0x7f1e38155120 2023-08-25T09:46:56.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.296+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e30005e40 con 0x7f1e38155120 2023-08-25T09:46:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.296+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30007310 con 0x7f1e38155120 2023-08-25T09:46:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.296+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e380c8a10 con 0x7f1e38155120 2023-08-25T09:46:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.296+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e380c8fd0 con 0x7f1e38155120 2023-08-25T09:46:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.297+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3804e420 con 0x7f1e38155120 2023-08-25T09:46:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.298+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e30007470 con 0x7f1e38155120 2023-08-25T09:46:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.299+0000 7f1e2f7fe640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e1806a050 0x7f1e1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:56.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.300+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f1e30087820 con 0x7f1e38155120 2023-08-25T09:46:56.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.300+0000 7f1e36d9d640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e1806a050 0x7f1e1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:56.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.300+0000 7f1e36d9d640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e1806a050 0x7f1e1806c510 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e280074a0 tx=0x7f1e2800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:56.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.304+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1e30053260 con 0x7f1e38155120 2023-08-25T09:46:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.481+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1e380c9760 con 0x7f1e38155120 2023-08-25T09:46:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.783+0000 7f1e2f7fe640 1 -- 172.21.15.70:0/926465525 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v147) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1e30056880 con 0x7f1e38155120 2023-08-25T09:46:56.786 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:46:56.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.788+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e1806a050 msgr2=0x7f1e1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.788+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e1806a050 0x7f1e1806c510 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e280074a0 tx=0x7f1e2800b040 comp rx=0 tx=0).stop 2023-08-25T09:46:56.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.788+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 msgr2=0x7f1e380c5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:56.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.788+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 0x7f1e380c5ee0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000c390 tx=0x7f1e3000c850 comp rx=0 tx=0).stop 2023-08-25T09:46:56.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.789+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 shutdown_connections 2023-08-25T09:46:56.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.789+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e1806a050 0x7f1e1806c510 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.789+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e38155120 0x7f1e380c5ee0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.789+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e380cd890 0x7f1e380c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.789+0000 7f1e3d0a6640 1 --2- 172.21.15.70:0/926465525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e380ca820 0x7f1e38150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:56.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.789+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 >> 172.21.15.70:0/926465525 conn(0x7f1e38069c70 msgr2=0x7f1e380cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:56.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.790+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 shutdown_connections 2023-08-25T09:46:56.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:56.790+0000 7f1e3d0a6640 1 -- 172.21.15.70:0/926465525 wait complete. 2023-08-25T09:46:56.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:46:57.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.033+0000 7f008ea14640 1 Processor -- start 2023-08-25T09:46:57.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.033+0000 7f008ea14640 1 -- start start 2023-08-25T09:46:57.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.035+0000 7f008ea14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 0x7f00880cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.035+0000 7f008ea14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0088071c10 con 0x7f0088160290 2023-08-25T09:46:57.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.035+0000 7f008ea14640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0088071000 con 0x7f00880cda70 2023-08-25T09:46:57.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.035+0000 7f008ea14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00880712e0 con 0x7f0088160670 2023-08-25T09:46:57.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.035+0000 7f0087fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0088160290 0x7f00880cd360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57660/0 (socket says 172.21.15.70:57660) 2023-08-25T09:46:57.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.035+0000 7f0087fff640 1 -- 172.21.15.70:0/3035650372 learned_addr learned my addr 172.21.15.70:0/3035650372 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:46:57.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663194958 0 0) 0x7f0088071c10 con 0x7f0088160290 2023-08-25T09:46:57.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f007ffff640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 0x7f00880cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0074003520 con 0x7f0088160290 2023-08-25T09:46:57.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3067705169 0 0) 0x7f0074003520 con 0x7f0088160290 2023-08-25T09:46:57.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 msgr2=0x7f00880cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:57.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 0x7f00880cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 >> v1:172.21.15.155:6789/0 conn(0x7f00880cda70 legacy=0x7f00880d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880715c0 con 0x7f0088160290 2023-08-25T09:46:57.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0078001670 con 0x7f0088160290 2023-08-25T09:46:57.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 >> v1:172.21.15.70:6789/0 conn(0x7f0088160290 legacy=0x7f00880cd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 0x7f00740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.036+0000 7f00877fe640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0074003fd0 0x7f00740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f00877fe640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0074004d00 0x7f0074005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00880712e0 con 0x7f0074004d00 2023-08-25T09:46:57.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007400bb40 con 0x7f0074003fd0 2023-08-25T09:46:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007400bd00 con 0x7f0088160670 2023-08-25T09:46:57.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f008cfb1640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0074004d00 0x7f0074005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f0087fff640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 0x7f00740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f007ffff640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0074003fd0 0x7f00740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f008cfb1640 1 -- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 msgr2=0x7f00740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:57.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f008cfb1640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0088160670 0x7f00740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f008cfb1640 1 -- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0074003fd0 msgr2=0x7f00740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:57.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f008cfb1640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0074003fd0 0x7f00740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.037+0000 7f008cfb1640 1 -- 172.21.15.70:0/3035650372 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0074003520 con 0x7f0074004d00 2023-08-25T09:46:57.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.038+0000 7f008cfb1640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0074004d00 0x7f0074005180 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0080001e70 tx=0x7f00740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c26282b89d5011 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:57.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080003b70 con 0x7f0074004d00 2023-08-25T09:46:57.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080003cd0 con 0x7f0074004d00 2023-08-25T09:46:57.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f00877fe640 1 -- 172.21.15.70:0/3035650372 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0080005310 con 0x7f0074004d00 2023-08-25T09:46:57.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008ea14640 1 -- 172.21.15.70:0/3035650372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0074004d00 msgr2=0x7f0074005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008ea14640 1 --2- 172.21.15.70:0/3035650372 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0074004d00 0x7f0074005180 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0080001e70 tx=0x7f00740057f0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008cfb1640 1 -- 172.21.15.70:0/3035650372 reap_dead start 2023-08-25T09:46:57.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008ea14640 1 -- 172.21.15.70:0/3035650372 shutdown_connections 2023-08-25T09:46:57.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008ea14640 1 -- 172.21.15.70:0/3035650372 >> 172.21.15.70:0/3035650372 conn(0x7f008806ba60 msgr2=0x7f008806be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:57.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008ea14640 1 -- 172.21.15.70:0/3035650372 shutdown_connections 2023-08-25T09:46:57.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.039+0000 7f008ea14640 1 -- 172.21.15.70:0/3035650372 wait complete. 2023-08-25T09:46:57.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.041+0000 7f008ea14640 1 Processor -- start 2023-08-25T09:46:57.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.041+0000 7f008ea14640 1 -- start start 2023-08-25T09:46:57.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f008ea14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0074004d00 0x7f00880b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f008ea14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 0x7f00880ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f008ea14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f008815cd30 0x7f00880ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f008ea14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00881f2360 con 0x7f008815d660 2023-08-25T09:46:57.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f008ea14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00881f2620 con 0x7f008815cd30 2023-08-25T09:46:57.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f008ea14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00881f2900 con 0x7f0074004d00 2023-08-25T09:46:57.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f007ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 0x7f00880ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f007ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 0x7f00880ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43932/0 (socket says 172.21.15.70:43932) 2023-08-25T09:46:57.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.042+0000 7f007ffff640 1 -- 172.21.15.70:0/728573797 learned_addr learned my addr 172.21.15.70:0/728573797 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:57.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f008cfb1640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f008815cd30 0x7f00880ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f0087fff640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0074004d00 0x7f00880b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f007ffff640 1 -- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0074004d00 msgr2=0x7f00880b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f007ffff640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0074004d00 0x7f00880b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f007ffff640 1 -- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f008815cd30 msgr2=0x7f00880ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f007ffff640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f008815cd30 0x7f00880ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f007ffff640 1 -- 172.21.15.70:0/728573797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00881f2be0 con 0x7f008815d660 2023-08-25T09:46:57.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f007ffff640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 0x7f00880ca2b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f007000f580 tx=0x7f007000fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:57.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.043+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070014870 con 0x7f008815d660 2023-08-25T09:46:57.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.044+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00700149d0 con 0x7f008815d660 2023-08-25T09:46:57.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.044+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070014d80 con 0x7f008815d660 2023-08-25T09:46:57.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.044+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00881f2ec0 con 0x7f008815d660 2023-08-25T09:46:57.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.044+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008815d1b0 con 0x7f008815d660 2023-08-25T09:46:57.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.045+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0088066fb0 con 0x7f008815d660 2023-08-25T09:46:57.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.046+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0070015410 con 0x7f008815d660 2023-08-25T09:46:57.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.047+0000 7f00857fa640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f006806a050 0x7f006806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:57.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.047+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f0070088e50 con 0x7f008815d660 2023-08-25T09:46:57.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.047+0000 7f0087fff640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f006806a050 0x7f006806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:57.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.048+0000 7f0087fff640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f006806a050 0x7f006806c510 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0078001eb0 tx=0x7f0078001de0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:57.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.052+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0070054690 con 0x7f008815d660 2023-08-25T09:46:57.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.245+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f00880b68f0 con 0x7f008815d660 2023-08-25T09:46:57.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.789+0000 7f00857fa640 1 -- 172.21.15.70:0/728573797 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v148) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0070057cb0 con 0x7f008815d660 2023-08-25T09:46:57.800 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:46:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f006806a050 msgr2=0x7f006806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f006806a050 0x7f006806c510 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0078001eb0 tx=0x7f0078001de0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 msgr2=0x7f00880ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 0x7f00880ca2b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f007000f580 tx=0x7f007000fa40 comp rx=0 tx=0).stop 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 shutdown_connections 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f006806a050 0x7f006806c510 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f008815cd30 0x7f00880ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f008815d660 0x7f00880ca2b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 --2- 172.21.15.70:0/728573797 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0074004d00 0x7f00880b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 >> 172.21.15.70:0/728573797 conn(0x7f008806ba60 msgr2=0x7f00880aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 shutdown_connections 2023-08-25T09:46:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:57.806+0000 7f008ea14640 1 -- 172.21.15.70:0/728573797 wait complete. 2023-08-25T09:46:57.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:46:58.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.023+0000 7fe1eecb0640 1 Processor -- start 2023-08-25T09:46:58.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.023+0000 7fe1eecb0640 1 -- start start 2023-08-25T09:46:58.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.024+0000 7fe1eecb0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1e8159650 0x7fe1e815dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:58.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.024+0000 7fe1eecb0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80ccee0 0x7fe1e80ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:58.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.024+0000 7fe1eecb0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e80acda0 con 0x7fe1e80b0300 2023-08-25T09:46:58.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.024+0000 7fe1eecb0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e80cd650 con 0x7fe1e8159650 2023-08-25T09:46:58.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.024+0000 7fe1eecb0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80ac1d0 con 0x7fe1e80ccee0 2023-08-25T09:46:58.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.024+0000 7fe1edcae640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1e8159650 0x7fe1e815dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:58.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1edcae640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1e8159650 0x7fe1e815dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43934/0 (socket says 172.21.15.70:43934) 2023-08-25T09:46:58.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1edcae640 1 -- 172.21.15.70:0/2894114712 learned_addr learned my addr 172.21.15.70:0/2894114712 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:58.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1ed4ad640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80ccee0 0x7fe1e80ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:58.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1edcae640 1 -- 172.21.15.70:0/2894114712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80ccee0 msgr2=0x7fe1e80ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:58.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1edcae640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80ccee0 0x7fe1e80ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1edcae640 1 -- 172.21.15.70:0/2894114712 >> v1:172.21.15.155:6789/0 conn(0x7fe1e80b0300 legacy=0x7fe1e80c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:58.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1edcae640 1 -- 172.21.15.70:0/2894114712 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e80ac470 con 0x7fe1e8159650 2023-08-25T09:46:58.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.025+0000 7fe1ed4ad640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80ccee0 0x7fe1e80ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:58.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.026+0000 7fe1edcae640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e815dd90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009850 tx=0x7fe1e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccb2c0caa477d13 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:58.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.026+0000 7fe1eccac640 1 -- 172.21.15.70:0/2894114712 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4004230 con 0x7fe1e8159650 2023-08-25T09:46:58.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.026+0000 7fe1eccac640 1 -- 172.21.15.70:0/2894114712 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4004390 con 0x7fe1e8159650 2023-08-25T09:46:58.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.026+0000 7fe1eccac640 1 -- 172.21.15.70:0/2894114712 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4031830 con 0x7fe1e8159650 2023-08-25T09:46:58.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 -- 172.21.15.70:0/2894114712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 msgr2=0x7fe1e815dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e815dd90 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009850 tx=0x7fe1e402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 -- 172.21.15.70:0/2894114712 shutdown_connections 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80ccee0 0x7fe1e80ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/2894114712 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e815dd90 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 -- 172.21.15.70:0/2894114712 >> 172.21.15.70:0/2894114712 conn(0x7fe1e80ce810 msgr2=0x7fe1e80cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 -- 172.21.15.70:0/2894114712 shutdown_connections 2023-08-25T09:46:58.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.027+0000 7fe1eecb0640 1 -- 172.21.15.70:0/2894114712 wait complete. 2023-08-25T09:46:58.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.029+0000 7fe1eecb0640 1 Processor -- start 2023-08-25T09:46:58.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.029+0000 7fe1eecb0640 1 -- start start 2023-08-25T09:46:58.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.030+0000 7fe1eecb0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80b0300 0x7fe1e806c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.030+0000 7fe1eecb0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e80ccee0 0x7fe1e81f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.030+0000 7fe1eecb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e80c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.030+0000 7fe1eecb0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e806caf0 con 0x7fe1e8159650 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.030+0000 7fe1eecb0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e81f36f0 con 0x7fe1e80ccee0 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.030+0000 7fe1eecb0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e8159a30 con 0x7fe1e80b0300 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e80c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e80c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43950/0 (socket says 172.21.15.70:43950) 2023-08-25T09:46:58.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 -- 172.21.15.70:0/1828950160 learned_addr learned my addr 172.21.15.70:0/1828950160 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:58.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1edcae640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80b0300 0x7fe1e806c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:58.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ed4ad640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e80ccee0 0x7fe1e81f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:58.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 -- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80b0300 msgr2=0x7fe1e806c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:58.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80b0300 0x7fe1e806c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 -- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e80ccee0 msgr2=0x7fe1e81f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:58.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e80ccee0 0x7fe1e81f31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 -- 172.21.15.70:0/1828950160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009460 con 0x7fe1e8159650 2023-08-25T09:46:58.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1ee4af640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e80c4060 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c780 tx=0x7fe1e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:58.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.031+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0004b60 con 0x7fe1e8159650 2023-08-25T09:46:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.032+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e000ce90 con 0x7fe1e8159650 2023-08-25T09:46:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.032+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000a850 con 0x7fe1e8159650 2023-08-25T09:46:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.032+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e80b2720 con 0x7fe1e8159650 2023-08-25T09:46:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.032+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e81f64c0 con 0x7fe1e8159650 2023-08-25T09:46:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.033+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e8061460 con 0x7fe1e8159650 2023-08-25T09:46:58.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.034+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe1e000a9b0 con 0x7fe1e8159650 2023-08-25T09:46:58.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.034+0000 7fe1de7fc640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1cc06a050 0x7fe1cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:58.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.035+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe1e00887a0 con 0x7fe1e8159650 2023-08-25T09:46:58.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.035+0000 7fe1edcae640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1cc06a050 0x7fe1cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:58.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.035+0000 7fe1edcae640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1cc06a050 0x7fe1cc06c510 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4035000 tx=0x7fe1e4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:58.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.039+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe1e0053e50 con 0x7fe1e8159650 2023-08-25T09:46:58.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.264+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe1e8159b70 con 0x7fe1e8159650 2023-08-25T09:46:58.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.821+0000 7fe1de7fc640 1 -- 172.21.15.70:0/1828950160 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe1e0057470 con 0x7fe1e8159650 2023-08-25T09:46:58.824 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 17 and data pool 18 2023-08-25T09:46:58.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1cc06a050 msgr2=0x7fe1cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:58.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1cc06a050 0x7fe1cc06c510 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4035000 tx=0x7fe1e4008030 comp rx=0 tx=0).stop 2023-08-25T09:46:58.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 msgr2=0x7fe1e80c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:58.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e80c4060 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c780 tx=0x7fe1e000cc40 comp rx=0 tx=0).stop 2023-08-25T09:46:58.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 shutdown_connections 2023-08-25T09:46:58.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe1cc06a050 0x7fe1cc06c510 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1e8159650 0x7fe1e80c4060 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1e80ccee0 0x7fe1e81f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 --2- 172.21.15.70:0/1828950160 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1e80b0300 0x7fe1e806c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:58.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.826+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 >> 172.21.15.70:0/1828950160 conn(0x7fe1e80ce810 msgr2=0x7fe1e80ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:58.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.827+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 shutdown_connections 2023-08-25T09:46:58.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:58.827+0000 7fe1eecb0640 1 -- 172.21.15.70:0/1828950160 wait complete. 2023-08-25T09:46:58.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.073+0000 7f29cf8bf640 1 Processor -- start 2023-08-25T09:46:59.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.073+0000 7f29cf8bf640 1 -- start start 2023-08-25T09:46:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cf8bf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29c815cc00 0x7f29c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cf8bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c80cf580 0x7f29c806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cf8bf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c806e340 con 0x7f29c806def0 2023-08-25T09:46:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cf8bf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c815cfe0 con 0x7f29c815cc00 2023-08-25T09:46:59.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cf8bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80cfdb0 con 0x7f29c80cf580 2023-08-25T09:46:59.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cd65b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29c815cc00 0x7f29c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cd65b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f29c815cc00 0x7f29c806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60244/0 (socket says 172.21.15.70:60244) 2023-08-25T09:46:59.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cd65b640 1 -- 172.21.15.70:0/648131436 learned_addr learned my addr 172.21.15.70:0/648131436 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cce5a640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c80cf580 0x7f29c806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cd65b640 1 -- 172.21.15.70:0/648131436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c80cf580 msgr2=0x7f29c806d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:59.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29cd65b640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c80cf580 0x7f29c806d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.076+0000 7f29cd65b640 1 -- 172.21.15.70:0/648131436 >> v1:172.21.15.70:6789/0 conn(0x7f29c806def0 legacy=0x7f29c806a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.076+0000 7f29cd65b640 1 -- 172.21.15.70:0/648131436 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c8069850 con 0x7f29c815cc00 2023-08-25T09:46:59.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.075+0000 7f29bffff640 1 -- 172.21.15.70:0/648131436 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734067709 0 0) 0x7f29c0001490 con 0x7f29c806def0 2023-08-25T09:46:59.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.076+0000 7f29cce5a640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c80cf580 0x7f29c806d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.077+0000 7f29cd65b640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c815cc00 0x7f29c806d3d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009850 tx=0x7f29c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ebcfc80180d3bb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.077+0000 7f29bffff640 1 -- 172.21.15.70:0/648131436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4004230 con 0x7f29c815cc00 2023-08-25T09:46:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.077+0000 7f29bffff640 1 -- 172.21.15.70:0/648131436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4004390 con 0x7f29c815cc00 2023-08-25T09:46:59.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.077+0000 7f29bffff640 1 -- 172.21.15.70:0/648131436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4003810 con 0x7f29c815cc00 2023-08-25T09:46:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.077+0000 7f29cf8bf640 1 -- 172.21.15.70:0/648131436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c815cc00 msgr2=0x7f29c806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.077+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c815cc00 0x7f29c806d3d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f29c4009850 tx=0x7f29c402b960 comp rx=0 tx=0).stop 2023-08-25T09:46:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.078+0000 7f29cf8bf640 1 -- 172.21.15.70:0/648131436 shutdown_connections 2023-08-25T09:46:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.078+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c80cf580 0x7f29c806d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.078+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/648131436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c815cc00 0x7f29c806d3d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.078+0000 7f29cf8bf640 1 -- 172.21.15.70:0/648131436 >> 172.21.15.70:0/648131436 conn(0x7f29c80d0860 msgr2=0x7f29c80d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.078+0000 7f29cf8bf640 1 -- 172.21.15.70:0/648131436 shutdown_connections 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.078+0000 7f29cf8bf640 1 -- 172.21.15.70:0/648131436 wait complete. 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.079+0000 7f29cf8bf640 1 Processor -- start 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.080+0000 7f29cf8bf640 1 -- start start 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cf8bf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c806def0 0x7f29c8155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cf8bf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 0x7f29c80ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cf8bf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c815cc00 0x7f29c80b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cf8bf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80c7c60 con 0x7f29c80cf580 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cf8bf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80c5770 con 0x7f29c815cc00 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cf8bf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80c59c0 con 0x7f29c806def0 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 0x7f29c80ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 0x7f29c80ca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60254/0 (socket says 172.21.15.70:60254) 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 -- 172.21.15.70:0/1600912443 learned_addr learned my addr 172.21.15.70:0/1600912443 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 -- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c806def0 msgr2=0x7f29c8155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cd65b640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c806def0 0x7f29c8155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c806def0 0x7f29c8155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 -- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c815cc00 msgr2=0x7f29c80b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c815cc00 0x7f29c80b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.081+0000 7f29cce5a640 1 -- 172.21.15.70:0/1600912443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c4009460 con 0x7f29c80cf580 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.082+0000 7f29cce5a640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 0x7f29c80ca560 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c7f0 tx=0x7f29b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.082+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8007a30 con 0x7f29c80cf580 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.082+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b8007b90 con 0x7f29c80cf580 2023-08-25T09:46:59.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.082+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800ce40 con 0x7f29c80cf580 2023-08-25T09:46:59.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.082+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c80c5ca0 con 0x7f29c80cf580 2023-08-25T09:46:59.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.082+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c80c6260 con 0x7f29c80cf580 2023-08-25T09:46:59.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.083+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c804e420 con 0x7f29c80cf580 2023-08-25T09:46:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.084+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29b80113f0 con 0x7f29c80cf580 2023-08-25T09:46:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.085+0000 7f29bdffb640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29ac06a050 0x7f29ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.085+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29b8088710 con 0x7f29c80cf580 2023-08-25T09:46:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.086+0000 7f29cd65b640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29ac06a050 0x7f29ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.086+0000 7f29cd65b640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29ac06a050 0x7f29ac06c510 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29c4035000 tx=0x7f29c4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.090+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f29b8053df0 con 0x7f29c80cf580 2023-08-25T09:46:59.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.279+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29c8065c70 con 0x7f29c80cf580 2023-08-25T09:46:59.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.282+0000 7f29bdffb640 1 -- 172.21.15.70:0/1600912443 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f29b8086e60 con 0x7f29c80cf580 2023-08-25T09:46:59.285 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:59.285 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":149,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:58.266720+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:59.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.284+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29ac06a050 msgr2=0x7f29ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.284+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29ac06a050 0x7f29ac06c510 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f29c4035000 tx=0x7f29c4008030 comp rx=0 tx=0).stop 2023-08-25T09:46:59.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.284+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 msgr2=0x7f29c80ca560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.284+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 0x7f29c80ca560 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c7f0 tx=0x7f29b800cbf0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 shutdown_connections 2023-08-25T09:46:59.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f29ac06a050 0x7f29ac06c510 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29c815cc00 0x7f29c80b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f29c80cf580 0x7f29c80ca560 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 --2- 172.21.15.70:0/1600912443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29c806def0 0x7f29c8155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 >> 172.21.15.70:0/1600912443 conn(0x7f29c80d0860 msgr2=0x7f29c80b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:59.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 shutdown_connections 2023-08-25T09:46:59.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.285+0000 7f29cf8bf640 1 -- 172.21.15.70:0/1600912443 wait complete. 2023-08-25T09:46:59.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:46:59.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.519+0000 7f23e595d640 1 Processor -- start 2023-08-25T09:46:59.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.519+0000 7f23e595d640 1 -- start start 2023-08-25T09:46:59.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.520+0000 7f23e595d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00ce5b0 0x7f23e00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.520+0000 7f23e595d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23e015aa10 0x7f23e00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.520+0000 7f23e595d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00ceed0 con 0x7f23e00b2680 2023-08-25T09:46:59.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.520+0000 7f23e595d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00b2b10 con 0x7f23e015aa10 2023-08-25T09:46:59.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.520+0000 7f23e595d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e0159d10 con 0x7f23e00ce5b0 2023-08-25T09:46:59.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23e015aa10 0x7f23e00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23e015aa10 0x7f23e00b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60268/0 (socket says 172.21.15.70:60268) 2023-08-25T09:46:59.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 -- 172.21.15.70:0/3255445404 learned_addr learned my addr 172.21.15.70:0/3255445404 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:59.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23deffd640 1 --2- 172.21.15.70:0/3255445404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00ce5b0 0x7f23e00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 -- 172.21.15.70:0/3255445404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00ce5b0 msgr2=0x7f23e00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:59.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 --2- 172.21.15.70:0/3255445404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00ce5b0 0x7f23e00ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 -- 172.21.15.70:0/3255445404 >> v1:172.21.15.155:6789/0 conn(0x7f23e00b2680 legacy=0x7f23e006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:59.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.521+0000 7f23d7fff640 1 -- 172.21.15.70:0/3255445404 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e015d3f0 con 0x7f23e015aa10 2023-08-25T09:46:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.523+0000 7f23d7fff640 1 --2- 172.21.15.70:0/3255445404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00b2140 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f23c800bb10 tx=0x7f23c802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f05c1fb6a21b21 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.523+0000 7f23de7fc640 1 -- 172.21.15.70:0/3255445404 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c802ee70 con 0x7f23e015aa10 2023-08-25T09:46:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.523+0000 7f23de7fc640 1 -- 172.21.15.70:0/3255445404 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c8036440 con 0x7f23e015aa10 2023-08-25T09:46:59.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.523+0000 7f23de7fc640 1 -- 172.21.15.70:0/3255445404 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c80367d0 con 0x7f23e015aa10 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 -- 172.21.15.70:0/3255445404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 msgr2=0x7f23e00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 --2- 172.21.15.70:0/3255445404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00b2140 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f23c800bb10 tx=0x7f23c802e0c0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 -- 172.21.15.70:0/3255445404 shutdown_connections 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 --2- 172.21.15.70:0/3255445404 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00b2140 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 --2- 172.21.15.70:0/3255445404 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00ce5b0 0x7f23e00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 -- 172.21.15.70:0/3255445404 >> 172.21.15.70:0/3255445404 conn(0x7f23e0069850 msgr2=0x7f23e0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 -- 172.21.15.70:0/3255445404 shutdown_connections 2023-08-25T09:46:59.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.524+0000 7f23e595d640 1 -- 172.21.15.70:0/3255445404 wait complete. 2023-08-25T09:46:59.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.526+0000 7f23e595d640 1 Processor -- start 2023-08-25T09:46:59.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.526+0000 7f23e595d640 1 -- start start 2023-08-25T09:46:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23e595d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00b2680 0x7f23e00b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23e595d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23e00ce5b0 0x7f23e014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23e595d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00af260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23e595d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00afd20 con 0x7f23e015aa10 2023-08-25T09:46:59.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23e595d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b0000 con 0x7f23e00ce5b0 2023-08-25T09:46:59.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23e595d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b02e0 con 0x7f23e00b2680 2023-08-25T09:46:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60272/0 (socket says 172.21.15.70:60272) 2023-08-25T09:46:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 -- 172.21.15.70:0/141952048 learned_addr learned my addr 172.21.15.70:0/141952048 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23deffd640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00b2680 0x7f23e00b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23d7fff640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23e00ce5b0 0x7f23e014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 -- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00b2680 msgr2=0x7f23e00b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00b2680 0x7f23e00b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 -- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23e00ce5b0 msgr2=0x7f23e014b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23e00ce5b0 0x7f23e014b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23df7fe640 1 -- 172.21.15.70:0/141952048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c800b770 con 0x7f23e015aa10 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23deffd640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00b2680 0x7f23e00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.527+0000 7f23d7fff640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23e00ce5b0 0x7f23e014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:59.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.528+0000 7f23df7fe640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00af260 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f23d800b970 tx=0x7f23d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.528+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d800cb40 con 0x7f23e015aa10 2023-08-25T09:46:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.528+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d800cca0 con 0x7f23e015aa10 2023-08-25T09:46:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.528+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e00b05c0 con 0x7f23e015aa10 2023-08-25T09:46:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.528+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d8014680 con 0x7f23e015aa10 2023-08-25T09:46:59.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.528+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e014b690 con 0x7f23e015aa10 2023-08-25T09:46:59.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.529+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e00b0ae0 con 0x7f23e015aa10 2023-08-25T09:46:59.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.530+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f23d801d3b0 con 0x7f23e015aa10 2023-08-25T09:46:59.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.531+0000 7f23dcff9640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f23c006a050 0x7f23c006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.531+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23d8088390 con 0x7f23e015aa10 2023-08-25T09:46:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.532+0000 7f23deffd640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f23c006a050 0x7f23c006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.532+0000 7f23deffd640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f23c006a050 0x7f23c006c510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006fd0 tx=0x7f23d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.536+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23d8053a80 con 0x7f23e015aa10 2023-08-25T09:46:59.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.727+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23e004e420 con 0x7f23e015aa10 2023-08-25T09:46:59.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.730+0000 7f23dcff9640 1 -- 172.21.15.70:0/141952048 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+18381 (secure 0 0 0) 0x7f23d80570a0 con 0x7f23e015aa10 2023-08-25T09:46:59.733 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:46:59.734 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":150,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:59.376525+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:46:59.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.733+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f23c006a050 msgr2=0x7f23c006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.733+0000 7f23e595d640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f23c006a050 0x7f23c006c510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f23d0006fd0 tx=0x7f23d0008040 comp rx=0 tx=0).stop 2023-08-25T09:46:59.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.733+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 msgr2=0x7f23e00af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.733+0000 7f23e595d640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00af260 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f23d800b970 tx=0x7f23d800be30 comp rx=0 tx=0).stop 2023-08-25T09:46:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.733+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 shutdown_connections 2023-08-25T09:46:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.733+0000 7f23e595d640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f23c006a050 0x7f23c006c510 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.734+0000 7f23e595d640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23e015aa10 0x7f23e00af260 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.734+0000 7f23e595d640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f23e00ce5b0 0x7f23e014b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.734+0000 7f23e595d640 1 --2- 172.21.15.70:0/141952048 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23e00b2680 0x7f23e00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.734+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 >> 172.21.15.70:0/141952048 conn(0x7f23e0069850 msgr2=0x7f23e006ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:59.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.734+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 shutdown_connections 2023-08-25T09:46:59.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.734+0000 7f23e595d640 1 -- 172.21.15.70:0/141952048 wait complete. 2023-08-25T09:46:59.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:46:59.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.974+0000 7f1884199640 1 Processor -- start 2023-08-25T09:46:59.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.974+0000 7f1884199640 1 -- start start 2023-08-25T09:46:59.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1884199640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f187c069850 0x7f187c15e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1884199640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f187c0cca20 0x7f187c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1884199640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f187c0ac880 con 0x7f187c0c9620 2023-08-25T09:46:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1884199640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0ad400 con 0x7f187c069850 2023-08-25T09:46:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1884199640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f187c0ad890 con 0x7f187c0cca20 2023-08-25T09:46:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1881f35640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f187c069850 0x7f187c15e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1881f35640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f187c069850 0x7f187c15e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60274/0 (socket says 172.21.15.70:60274) 2023-08-25T09:46:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1881f35640 1 -- 172.21.15.70:0/1677351045 learned_addr learned my addr 172.21.15.70:0/1677351045 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.976+0000 7f1882736640 1 --2- 172.21.15.70:0/1677351045 >> v2:172.21.15.155:3300/0 conn(0x7f187c0cca20 0x7f187c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.977+0000 7f1881f35640 1 -- 172.21.15.70:0/1677351045 >> v2:172.21.15.155:3300/0 conn(0x7f187c0cca20 msgr2=0x7f187c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.977+0000 7f1881f35640 1 --2- 172.21.15.70:0/1677351045 >> v2:172.21.15.155:3300/0 conn(0x7f187c0cca20 0x7f187c0cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.977+0000 7f1881f35640 1 -- 172.21.15.70:0/1677351045 >> v1:172.21.15.155:6789/0 conn(0x7f187c0c9620 legacy=0x7f187c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:46:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.977+0000 7f1881f35640 1 -- 172.21.15.70:0/1677351045 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0acad0 con 0x7f187c069850 2023-08-25T09:46:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.977+0000 7f1882736640 1 --2- 172.21.15.70:0/1677351045 >> v2:172.21.15.155:3300/0 conn(0x7f187c0cca20 0x7f187c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:46:59.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.978+0000 7f1881f35640 1 --2- 172.21.15.70:0/1677351045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c069850 0x7f187c15e210 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f187c0c5610 tx=0x7f187802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1ef166ea7ad381 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.978+0000 7f1880f33640 1 -- 172.21.15.70:0/1677351045 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878030860 con 0x7f187c069850 2023-08-25T09:46:59.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.978+0000 7f1880f33640 1 -- 172.21.15.70:0/1677351045 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18780309c0 con 0x7f187c069850 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.978+0000 7f1880f33640 1 -- 172.21.15.70:0/1677351045 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1878030d50 con 0x7f187c069850 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 -- 172.21.15.70:0/1677351045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c069850 msgr2=0x7f187c15e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 --2- 172.21.15.70:0/1677351045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c069850 0x7f187c15e210 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f187c0c5610 tx=0x7f187802b9a0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 -- 172.21.15.70:0/1677351045 shutdown_connections 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 --2- 172.21.15.70:0/1677351045 >> v2:172.21.15.155:3300/0 conn(0x7f187c0cca20 0x7f187c0cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 --2- 172.21.15.70:0/1677351045 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c069850 0x7f187c15e210 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 -- 172.21.15.70:0/1677351045 >> 172.21.15.70:0/1677351045 conn(0x7f187c0cb0d0 msgr2=0x7f187c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:46:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 -- 172.21.15.70:0/1677351045 shutdown_connections 2023-08-25T09:46:59.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.979+0000 7f1884199640 1 -- 172.21.15.70:0/1677351045 wait complete. 2023-08-25T09:46:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.981+0000 7f1884199640 1 Processor -- start 2023-08-25T09:46:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.981+0000 7f1884199640 1 -- start start 2023-08-25T09:46:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1884199640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c069850 0x7f187c0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1884199640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 0x7f187c145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1884199640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c0cca20 0x7f187c0c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1884199640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c0b1960 con 0x7f187c0c9620 2023-08-25T09:46:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1884199640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f187c1455c0 con 0x7f187c069850 2023-08-25T09:46:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1884199640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f187c145ac0 con 0x7f187c0cca20 2023-08-25T09:46:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1881734640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 0x7f187c145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1881734640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 0x7f187c145080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60282/0 (socket says 172.21.15.70:60282) 2023-08-25T09:46:59.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.982+0000 7f1881734640 1 -- 172.21.15.70:0/3950682578 learned_addr learned my addr 172.21.15.70:0/3950682578 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:46:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881734640 1 -- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c0cca20 msgr2=0x7f187c0c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881f35640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c069850 0x7f187c0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881734640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c0cca20 0x7f187c0c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881734640 1 -- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c069850 msgr2=0x7f187c0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:46:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881734640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c069850 0x7f187c0c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:46:59.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881734640 1 -- 172.21.15.70:0/3950682578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18780094d0 con 0x7f187c0c9620 2023-08-25T09:46:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1881734640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 0x7f187c145080 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c430 tx=0x7f186c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c011790 con 0x7f187c0c9620 2023-08-25T09:46:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186c0118f0 con 0x7f187c0c9620 2023-08-25T09:46:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186c011ce0 con 0x7f187c0c9620 2023-08-25T09:46:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.983+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f187c1f6a00 con 0x7f187c0c9620 2023-08-25T09:46:59.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.984+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f187c0b1310 con 0x7f187c0c9620 2023-08-25T09:46:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.985+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f187c0b1b10 con 0x7f187c0c9620 2023-08-25T09:46:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.986+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f186c01c460 con 0x7f187c0c9620 2023-08-25T09:46:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.986+0000 7f18727fc640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f186006a050 0x7f186006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:46:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.987+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f186c0885b0 con 0x7f187c0c9620 2023-08-25T09:46:59.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.987+0000 7f1881f35640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f186006a050 0x7f186006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:46:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.988+0000 7f1881f35640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f186006a050 0x7f186006c510 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1878036000 tx=0x7f187803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:46:59.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:46:59.991+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f187c0b1b10 con 0x7f187c0c9620 2023-08-25T09:47:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.216+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f187c066fb0 con 0x7f187c0c9620 2023-08-25T09:47:00.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.835+0000 7f18727fc640 1 -- 172.21.15.70:0/3950682578 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f186c053c00 con 0x7f187c0c9620 2023-08-25T09:47:00.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.839+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f186006a050 msgr2=0x7f186006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:00.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.839+0000 7f1884199640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f186006a050 0x7f186006c510 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1878036000 tx=0x7f187803b040 comp rx=0 tx=0).stop 2023-08-25T09:47:00.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.839+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 msgr2=0x7f187c145080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:00.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.839+0000 7f1884199640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 0x7f187c145080 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f186c00c430 tx=0x7f186c00c8f0 comp rx=0 tx=0).stop 2023-08-25T09:47:00.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 shutdown_connections 2023-08-25T09:47:00.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f186006a050 0x7f186006c510 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:00.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f187c0cca20 0x7f187c0c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:00.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f187c0c9620 0x7f187c145080 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:00.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 --2- 172.21.15.70:0/3950682578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f187c069850 0x7f187c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:00.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 >> 172.21.15.70:0/3950682578 conn(0x7f187c0cb0d0 msgr2=0x7f187c0cce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:00.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 shutdown_connections 2023-08-25T09:47:00.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:00.840+0000 7f1884199640 1 -- 172.21.15.70:0/3950682578 wait complete. 2023-08-25T09:47:00.864 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:47:00.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:47:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.174+0000 7f6ba85dc640 1 Processor -- start 2023-08-25T09:47:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.175+0000 7f6ba85dc640 1 -- start start 2023-08-25T09:47:01.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.176+0000 7f6ba85dc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 0x7f6ba015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.176+0000 7f6ba85dc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6ba006f0e0 0x7f6ba0070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.176+0000 7f6ba85dc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba015e780 con 0x7f6ba00cd660 2023-08-25T09:47:01.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.176+0000 7f6ba85dc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba006f8e0 con 0x7f6ba006f0e0 2023-08-25T09:47:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.176+0000 7f6ba85dc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba006fbc0 con 0x7f6ba0159bf0 2023-08-25T09:47:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.177+0000 7f6ba6b79640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6ba00cd660 0x7f6ba00c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56076/0 (socket says 172.21.15.70:56076) 2023-08-25T09:47:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.177+0000 7f6ba6b79640 1 -- 172.21.15.70:0/1528890410 learned_addr learned my addr 172.21.15.70:0/1528890410 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.177+0000 7f6ba5b77640 1 --2- 172.21.15.70:0/1528890410 >> v2:172.21.15.155:3300/0 conn(0x7f6ba006f0e0 0x7f6ba0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.177+0000 7f6ba6378640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 0x7f6ba015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.177+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319382302 0 0) 0x7f6ba015e780 con 0x7f6ba00cd660 2023-08-25T09:47:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.177+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b88005730 con 0x7f6ba00cd660 2023-08-25T09:47:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1544906441 0 0) 0x7f6b88005730 con 0x7f6ba00cd660 2023-08-25T09:47:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 msgr2=0x7f6ba015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 0x7f6ba015e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 >> v2:172.21.15.155:3300/0 conn(0x7f6ba006f0e0 msgr2=0x7f6ba0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 --2- 172.21.15.70:0/1528890410 >> v2:172.21.15.155:3300/0 conn(0x7f6ba006f0e0 0x7f6ba0070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba006fea0 con 0x7f6ba00cd660 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6b98001670 con 0x7f6ba00cd660 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 >> v1:172.21.15.70:6789/0 conn(0x7f6ba00cd660 legacy=0x7f6ba00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b88005b80 0x7f6b88005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b880069d0 0x7f6b88006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 0x7f6b88007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba006f8e0 con 0x7f6b88005b80 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba006fbc0 con 0x7f6b880069d0 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8800db90 con 0x7f6ba0159bf0 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba6378640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b88005b80 0x7f6b88005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5b77640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b880069d0 0x7f6b88006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba6378640 1 -- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 msgr2=0x7f6b88007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba6378640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6ba0159bf0 0x7f6b88007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba6378640 1 -- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b880069d0 msgr2=0x7f6b88006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba6378640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b880069d0 0x7f6b88006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba6378640 1 -- 172.21.15.70:0/1528890410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b88005730 con 0x7f6b88005b80 2023-08-25T09:47:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.178+0000 7f6ba5b77640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b880069d0 0x7f6b88006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:01.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba6378640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b88005b80 0x7f6b88005fa0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009fc0 tx=0x7f6b88007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1227c0e997359445 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:01.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9002cc60 con 0x7f6b88005b80 2023-08-25T09:47:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9000f040 con 0x7f6b88005b80 2023-08-25T09:47:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba5376640 1 -- 172.21.15.70:0/1528890410 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b90034440 con 0x7f6b88005b80 2023-08-25T09:47:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba85dc640 1 -- 172.21.15.70:0/1528890410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b88005b80 msgr2=0x7f6b88005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/1528890410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b88005b80 0x7f6b88005fa0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009fc0 tx=0x7f6b88007a00 comp rx=0 tx=0).stop 2023-08-25T09:47:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba6b79640 1 -- 172.21.15.70:0/1528890410 reap_dead start 2023-08-25T09:47:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba85dc640 1 -- 172.21.15.70:0/1528890410 shutdown_connections 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.180+0000 7f6ba85dc640 1 -- 172.21.15.70:0/1528890410 >> 172.21.15.70:0/1528890410 conn(0x7f6ba00cb0d0 msgr2=0x7f6ba00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.181+0000 7f6ba85dc640 1 -- 172.21.15.70:0/1528890410 shutdown_connections 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.181+0000 7f6ba85dc640 1 -- 172.21.15.70:0/1528890410 wait complete. 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.182+0000 7f6ba85dc640 1 Processor -- start 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.182+0000 7f6ba85dc640 1 -- start start 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.183+0000 7f6ba85dc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b88005b80 0x7f6ba01f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.183+0000 7f6ba85dc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba0154000 0x7f6ba0156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.183+0000 7f6ba85dc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 0x7f6ba0159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.183+0000 7f6ba85dc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba01f2410 con 0x7f6ba0156880 2023-08-25T09:47:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.183+0000 7f6ba85dc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba0155eb0 con 0x7f6ba0154000 2023-08-25T09:47:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.183+0000 7f6ba85dc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba0158060 con 0x7f6b88005b80 2023-08-25T09:47:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 0x7f6ba0159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 0x7f6ba0159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60304/0 (socket says 172.21.15.70:60304) 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 -- 172.21.15.70:0/2728317722 learned_addr learned my addr 172.21.15.70:0/2728317722 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba5b77640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba0154000 0x7f6ba0156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6378640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b88005b80 0x7f6ba01f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 -- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b88005b80 msgr2=0x7f6ba01f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b88005b80 0x7f6ba01f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 -- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba0154000 msgr2=0x7f6ba0156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba0154000 0x7f6ba0156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 -- 172.21.15.70:0/2728317722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba0158340 con 0x7f6ba0156880 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba5b77640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba0154000 0x7f6ba0156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.184+0000 7f6ba6b79640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 0x7f6ba0159fd0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00c8d90 tx=0x7f6b9800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.185+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98007500 con 0x7f6ba0156880 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.185+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b98007660 con 0x7f6ba0156880 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.185+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98004a20 con 0x7f6ba0156880 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.185+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba0155b40 con 0x7f6ba0156880 2023-08-25T09:47:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.185+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba0157ce0 con 0x7f6ba0156880 2023-08-25T09:47:01.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.186+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba0158580 con 0x7f6ba0156880 2023-08-25T09:47:01.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.187+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6b98004b80 con 0x7f6ba0156880 2023-08-25T09:47:01.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.188+0000 7f6b9effd640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b8c06a050 0x7f6b8c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:01.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.188+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6b98087b20 con 0x7f6ba0156880 2023-08-25T09:47:01.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.188+0000 7f6ba6378640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b8c06a050 0x7f6b8c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:01.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.189+0000 7f6ba6378640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b8c06a050 0x7f6b8c06c510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b90036000 tx=0x7f6b9002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:01.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.192+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6ba0158580 con 0x7f6ba0156880 2023-08-25T09:47:01.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.420+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6ba01559a0 con 0x7f6ba0156880 2023-08-25T09:47:01.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.846+0000 7f6b9effd640 1 -- 172.21.15.70:0/2728317722 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ba01559a0 con 0x7f6ba0156880 2023-08-25T09:47:01.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.849+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b8c06a050 msgr2=0x7f6b8c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:01.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.849+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b8c06a050 0x7f6b8c06c510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6b90036000 tx=0x7f6b9002e000 comp rx=0 tx=0).stop 2023-08-25T09:47:01.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.849+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 msgr2=0x7f6ba0159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:01.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.849+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 0x7f6ba0159fd0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6ba00c8d90 tx=0x7f6b9800c8b0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 shutdown_connections 2023-08-25T09:47:01.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6b8c06a050 0x7f6b8c06c510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6ba0156880 0x7f6ba0159fd0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6ba0154000 0x7f6ba0156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 --2- 172.21.15.70:0/2728317722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b88005b80 0x7f6ba01f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:01.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 >> 172.21.15.70:0/2728317722 conn(0x7f6ba00cb0d0 msgr2=0x7f6ba00ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:01.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 shutdown_connections 2023-08-25T09:47:01.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:01.850+0000 7f6ba85dc640 1 -- 172.21.15.70:0/2728317722 wait complete. 2023-08-25T09:47:01.874 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:47:01.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:02.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.073+0000 7ffb55f22640 1 Processor -- start 2023-08-25T09:47:02.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.073+0000 7ffb55f22640 1 -- start start 2023-08-25T09:47:02.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb55f22640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb5015aa10 0x7ffb500b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb55f22640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ffb500b2680 0x7ffb5006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb55f22640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb500b2ad0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb55f22640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb50159d10 con 0x7ffb5015aa10 2023-08-25T09:47:02.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb55f22640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5015d3f0 con 0x7ffb500b2680 2023-08-25T09:47:02.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb5015aa10 0x7ffb500b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffb5015aa10 0x7ffb500b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60306/0 (socket says 172.21.15.70:60306) 2023-08-25T09:47:02.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 -- 172.21.15.70:0/3304970603 learned_addr learned my addr 172.21.15.70:0/3304970603 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:02.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 -- 172.21.15.70:0/3304970603 >> v2:172.21.15.155:3300/0 conn(0x7ffb500b2680 msgr2=0x7ffb5006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 --2- 172.21.15.70:0/3304970603 >> v2:172.21.15.155:3300/0 conn(0x7ffb500b2680 0x7ffb5006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 -- 172.21.15.70:0/3304970603 >> v1:172.21.15.155:6789/0 conn(0x7ffb500ce5b0 legacy=0x7ffb500ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.075+0000 7ffb47fff640 1 -- 172.21.15.70:0/3304970603 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500b2d20 con 0x7ffb5015aa10 2023-08-25T09:47:02.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.076+0000 7ffb47fff640 1 --2- 172.21.15.70:0/3304970603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5015aa10 0x7ffb500b2140 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009870 tx=0x7ffb3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db91b3ab0711e224 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb4effd640 1 -- 172.21.15.70:0/3304970603 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380317e0 con 0x7ffb5015aa10 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb4effd640 1 -- 172.21.15.70:0/3304970603 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb38031940 con 0x7ffb5015aa10 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb4effd640 1 -- 172.21.15.70:0/3304970603 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38031cd0 con 0x7ffb5015aa10 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb55f22640 1 -- 172.21.15.70:0/3304970603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5015aa10 msgr2=0x7ffb500b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb55f22640 1 --2- 172.21.15.70:0/3304970603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5015aa10 0x7ffb500b2140 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009870 tx=0x7ffb3802b880 comp rx=0 tx=0).stop 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb55f22640 1 -- 172.21.15.70:0/3304970603 shutdown_connections 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb55f22640 1 --2- 172.21.15.70:0/3304970603 >> v2:172.21.15.155:3300/0 conn(0x7ffb500b2680 0x7ffb5006f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb55f22640 1 --2- 172.21.15.70:0/3304970603 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb5015aa10 0x7ffb500b2140 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.077+0000 7ffb55f22640 1 -- 172.21.15.70:0/3304970603 >> 172.21.15.70:0/3304970603 conn(0x7ffb50069850 msgr2=0x7ffb50069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:02.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.078+0000 7ffb55f22640 1 -- 172.21.15.70:0/3304970603 shutdown_connections 2023-08-25T09:47:02.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.078+0000 7ffb55f22640 1 -- 172.21.15.70:0/3304970603 wait complete. 2023-08-25T09:47:02.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.079+0000 7ffb55f22640 1 Processor -- start 2023-08-25T09:47:02.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.079+0000 7ffb55f22640 1 -- start start 2023-08-25T09:47:02.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb55f22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb500b2680 0x7ffb50156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb55f22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 0x7ffb500b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb55f22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5015aa10 0x7ffb50151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb55f22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501542e0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb55f22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb500b17a0 con 0x7ffb500b2680 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb55f22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb501536d0 con 0x7ffb5015aa10 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 0x7ffb500b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.080+0000 7ffb47fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 0x7ffb500b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60316/0 (socket says 172.21.15.70:60316) 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 -- 172.21.15.70:0/2748963059 learned_addr learned my addr 172.21.15.70:0/2748963059 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 -- 172.21.15.70:0/2748963059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5015aa10 msgr2=0x7ffb50151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5015aa10 0x7ffb50151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 -- 172.21.15.70:0/2748963059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb500b2680 msgr2=0x7ffb50156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb500b2680 0x7ffb50156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 -- 172.21.15.70:0/2748963059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380094d0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb47fff640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 0x7ffb500b1260 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7ffb3802b8c0 tx=0x7ffb380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38032880 con 0x7ffb500ce5b0 2023-08-25T09:47:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb380329e0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38032d90 con 0x7ffb500ce5b0 2023-08-25T09:47:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.081+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb501539b0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.082+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb50153f70 con 0x7ffb500ce5b0 2023-08-25T09:47:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.083+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5004e420 con 0x7ffb500ce5b0 2023-08-25T09:47:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.083+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffb3804d3f0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.084+0000 7ffb4cff9640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffb3006a080 0x7ffb3006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.084+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffb380aa4c0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.085+0000 7ffb4f7fe640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffb3006a080 0x7ffb3006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.085+0000 7ffb4f7fe640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffb3006a080 0x7ffb3006c540 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffb40007fd0 tx=0x7ffb40007980 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.088+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffb38075bc0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.312+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb500cae50 con 0x7ffb500ce5b0 2023-08-25T09:47:02.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.313+0000 7ffb4cff9640 1 -- 172.21.15.70:0/2748963059 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7ffb380791e0 con 0x7ffb500ce5b0 2023-08-25T09:47:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:02.316 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:02.316 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:02.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffb3006a080 msgr2=0x7ffb3006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffb3006a080 0x7ffb3006c540 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ffb40007fd0 tx=0x7ffb40007980 comp rx=0 tx=0).stop 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 msgr2=0x7ffb500b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 0x7ffb500b1260 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7ffb3802b8c0 tx=0x7ffb380046c0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 shutdown_connections 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffb3006a080 0x7ffb3006c540 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffb5015aa10 0x7ffb50151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffb500ce5b0 0x7ffb500b1260 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 --2- 172.21.15.70:0/2748963059 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffb500b2680 0x7ffb50156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 >> 172.21.15.70:0/2748963059 conn(0x7ffb50069850 msgr2=0x7ffb5015af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:02.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 shutdown_connections 2023-08-25T09:47:02.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.316+0000 7ffb55f22640 1 -- 172.21.15.70:0/2748963059 wait complete. 2023-08-25T09:47:02.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.549+0000 7f57e00ec640 1 Processor -- start 2023-08-25T09:47:02.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.549+0000 7f57e00ec640 1 -- start start 2023-08-25T09:47:02.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57e00ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f57d815ec10 0x7f57d815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57e00ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815c6d0 0x7f57d80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57e00ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d80b1b60 con 0x7f57d815e830 2023-08-25T09:47:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57e00ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57d806bd80 con 0x7f57d815ec10 2023-08-25T09:47:02.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57e00ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d806b7c0 con 0x7f57d815c6d0 2023-08-25T09:47:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57dd687640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f57d815ec10 0x7f57d815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57dd687640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f57d815ec10 0x7f57d815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60324/0 (socket says 172.21.15.70:60324) 2023-08-25T09:47:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.551+0000 7f57dd687640 1 -- 172.21.15.70:0/3903661575 learned_addr learned my addr 172.21.15.70:0/3903661575 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:02.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.552+0000 7f57de689640 1 --2- 172.21.15.70:0/3903661575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815c6d0 0x7f57d80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.552+0000 7f57dd687640 1 -- 172.21.15.70:0/3903661575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815c6d0 msgr2=0x7f57d80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.552+0000 7f57dd687640 1 --2- 172.21.15.70:0/3903661575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815c6d0 0x7f57d80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.552+0000 7f57dd687640 1 -- 172.21.15.70:0/3903661575 >> v1:172.21.15.155:6789/0 conn(0x7f57d815e830 legacy=0x7f57d806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.552+0000 7f57dd687640 1 -- 172.21.15.70:0/3903661575 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d806baa0 con 0x7f57d815ec10 2023-08-25T09:47:02.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.553+0000 7f57dd687640 1 --2- 172.21.15.70:0/3903661575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815ec10 0x7f57d815c190 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90e8130ea513cc5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.553+0000 7f57dce86640 1 -- 172.21.15.70:0/3903661575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8031640 con 0x7f57d815ec10 2023-08-25T09:47:02.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.553+0000 7f57dce86640 1 -- 172.21.15.70:0/3903661575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c80317a0 con 0x7f57d815ec10 2023-08-25T09:47:02.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.553+0000 7f57dce86640 1 -- 172.21.15.70:0/3903661575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8031b30 con 0x7f57d815ec10 2023-08-25T09:47:02.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 -- 172.21.15.70:0/3903661575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815ec10 msgr2=0x7f57d815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 --2- 172.21.15.70:0/3903661575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815ec10 0x7f57d815c190 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009870 tx=0x7f57c802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 -- 172.21.15.70:0/3903661575 shutdown_connections 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 --2- 172.21.15.70:0/3903661575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815c6d0 0x7f57d80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 --2- 172.21.15.70:0/3903661575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815ec10 0x7f57d815c190 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 -- 172.21.15.70:0/3903661575 >> 172.21.15.70:0/3903661575 conn(0x7f57d8069850 msgr2=0x7f57d8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 -- 172.21.15.70:0/3903661575 shutdown_connections 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.554+0000 7f57e00ec640 1 -- 172.21.15.70:0/3903661575 wait complete. 2023-08-25T09:47:02.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.555+0000 7f57e00ec640 1 Processor -- start 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.556+0000 7f57e00ec640 1 -- start start 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57e00ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 0x7f57d814d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57e00ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815e830 0x7f57d80a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57e00ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d815ec10 0x7f57d80a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57e00ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d806fcd0 con 0x7f57d815c6d0 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57e00ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d806ffb0 con 0x7f57d815ec10 2023-08-25T09:47:02.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57e00ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d8070290 con 0x7f57d815e830 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 0x7f57d814d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 0x7f57d814d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60336/0 (socket says 172.21.15.70:60336) 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 -- 172.21.15.70:0/2305116020 learned_addr learned my addr 172.21.15.70:0/2305116020 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dd687640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815e830 0x7f57d80a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 -- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815e830 msgr2=0x7f57d80a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815e830 0x7f57d80a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 -- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d815ec10 msgr2=0x7f57d80a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d815ec10 0x7f57d80a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.557+0000 7f57dde88640 1 -- 172.21.15.70:0/2305116020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c80094d0 con 0x7f57d815c6d0 2023-08-25T09:47:02.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.558+0000 7f57dde88640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 0x7f57d814d590 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c430 tx=0x7f57d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.558+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4011790 con 0x7f57d815c6d0 2023-08-25T09:47:02.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.558+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d40118f0 con 0x7f57d815c6d0 2023-08-25T09:47:02.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.558+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d8070570 con 0x7f57d815c6d0 2023-08-25T09:47:02.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.558+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d4011ce0 con 0x7f57d815c6d0 2023-08-25T09:47:02.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.558+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d806ef80 con 0x7f57d815c6d0 2023-08-25T09:47:02.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.559+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d804e420 con 0x7f57d815c6d0 2023-08-25T09:47:02.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.560+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f57d401c460 con 0x7f57d815c6d0 2023-08-25T09:47:02.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.561+0000 7f57ce7fc640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57bc06a050 0x7f57bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.561+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57d4088570 con 0x7f57d815c6d0 2023-08-25T09:47:02.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.561+0000 7f57dd687640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57bc06a050 0x7f57bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.562+0000 7f57dd687640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57bc06a050 0x7f57bc06c510 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f57c8036000 tx=0x7f57c803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.566+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f57d4053c50 con 0x7f57d815c6d0 2023-08-25T09:47:02.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.742+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d806b450 con 0x7f57d815c6d0 2023-08-25T09:47:02.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.744+0000 7f57ce7fc640 1 -- 172.21.15.70:0/2305116020 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+18381 (secure 0 0 0) 0x7f57d4057270 con 0x7f57d815c6d0 2023-08-25T09:47:02.747 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:02.747 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":150,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:46:59.376525+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:47:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.746+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57bc06a050 msgr2=0x7f57bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.746+0000 7f57e00ec640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57bc06a050 0x7f57bc06c510 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f57c8036000 tx=0x7f57c803c040 comp rx=0 tx=0).stop 2023-08-25T09:47:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.746+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 msgr2=0x7f57d814d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.746+0000 7f57e00ec640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 0x7f57d814d590 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f57d400c430 tx=0x7f57d400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 shutdown_connections 2023-08-25T09:47:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f57bc06a050 0x7f57bc06c510 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f57d815ec10 0x7f57d80a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f57d815e830 0x7f57d80a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 --2- 172.21.15.70:0/2305116020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f57d815c6d0 0x7f57d814d590 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 >> 172.21.15.70:0/2305116020 conn(0x7f57d8069850 msgr2=0x7f57d80b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:02.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 shutdown_connections 2023-08-25T09:47:02.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.747+0000 7f57e00ec640 1 -- 172.21.15.70:0/2305116020 wait complete. 2023-08-25T09:47:02.768 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:47:02.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:47:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.964+0000 7fd981845640 1 Processor -- start 2023-08-25T09:47:02.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.964+0000 7fd981845640 1 -- start start 2023-08-25T09:47:02.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.966+0000 7fd981845640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd97c0cad20 0x7fd97c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.966+0000 7fd981845640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c15e7c0 con 0x7fd97c15a7c0 2023-08-25T09:47:02.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.966+0000 7fd981845640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97c071b50 con 0x7fd97c0b12f0 2023-08-25T09:47:02.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.966+0000 7fd981845640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c0cb6f0 con 0x7fd97c0cad20 2023-08-25T09:47:02.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.966+0000 7fd980843640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd97c15a7c0 0x7fd97c0b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56082/0 (socket says 172.21.15.70:56082) 2023-08-25T09:47:02.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.966+0000 7fd980843640 1 -- 172.21.15.70:0/3664244250 learned_addr learned my addr 172.21.15.70:0/3664244250 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:02.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd981044640 1 --2- 172.21.15.70:0/3664244250 >> v2:172.21.15.155:3300/0 conn(0x7fd97c0cad20 0x7fd97c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213565051 0 0) 0x7fd97c15e7c0 con 0x7fd97c15a7c0 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd96c003520 con 0x7fd97c15a7c0 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818317188 0 0) 0x7fd96c003520 con 0x7fd97c15a7c0 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 >> v2:172.21.15.155:3300/0 conn(0x7fd97c0cad20 msgr2=0x7fd97c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 --2- 172.21.15.70:0/3664244250 >> v2:172.21.15.155:3300/0 conn(0x7fd97c0cad20 0x7fd97c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 >> v1:172.21.15.155:6789/0 conn(0x7fd97c0b12f0 legacy=0x7fd97c15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c15e490 con 0x7fd97c15a7c0 2023-08-25T09:47:02.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fd970001670 con 0x7fd97c15a7c0 2023-08-25T09:47:02.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 >> v1:172.21.15.70:6789/0 conn(0x7fd97c15a7c0 legacy=0x7fd97c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd96c003970 0x7fd96c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd96c0047c0 0x7fd96c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c005180 0x7fd96c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0cb6f0 con 0x7fd96c003970 2023-08-25T09:47:02.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c00c000 con 0x7fd96c005180 2023-08-25T09:47:02.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.967+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c00c1c0 con 0x7fd96c0047c0 2023-08-25T09:47:02.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd980843640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd96c003970 0x7fd96c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd980843640 1 -- 172.21.15.70:0/3664244250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd96c0047c0 msgr2=0x7fd96c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd980843640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd96c0047c0 0x7fd96c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd980843640 1 -- 172.21.15.70:0/3664244250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c005180 msgr2=0x7fd96c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:02.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd980843640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c005180 0x7fd96c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd980843640 1 -- 172.21.15.70:0/3664244250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c003520 con 0x7fd96c003970 2023-08-25T09:47:02.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.968+0000 7fd981044640 1 -- 172.21.15.70:0/3664244250 reap_dead start 2023-08-25T09:47:02.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.969+0000 7fd980843640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd96c003970 0x7fd96c003d70 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd97000c190 tx=0x7fd96c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d6152410b74859 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.969+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd970005ad0 con 0x7fd96c003970 2023-08-25T09:47:02.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.969+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd970005c50 con 0x7fd96c003970 2023-08-25T09:47:02.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.969+0000 7fd97b7fe640 1 -- 172.21.15.70:0/3664244250 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97002c8b0 con 0x7fd96c003970 2023-08-25T09:47:02.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 -- 172.21.15.70:0/3664244250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd96c003970 msgr2=0x7fd96c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd96c003970 0x7fd96c003d70 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd97000c190 tx=0x7fd96c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 -- 172.21.15.70:0/3664244250 shutdown_connections 2023-08-25T09:47:02.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 --2- 172.21.15.70:0/3664244250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd96c003970 0x7fd96c003d70 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 -- 172.21.15.70:0/3664244250 >> 172.21.15.70:0/3664244250 conn(0x7fd97c0affd0 msgr2=0x7fd97c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:02.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 -- 172.21.15.70:0/3664244250 shutdown_connections 2023-08-25T09:47:02.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.970+0000 7fd981845640 1 -- 172.21.15.70:0/3664244250 wait complete. 2023-08-25T09:47:02.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.972+0000 7fd981845640 1 Processor -- start 2023-08-25T09:47:02.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.972+0000 7fd981845640 1 -- start start 2023-08-25T09:47:02.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd981845640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c003970 0x7fd97c0c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd981845640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 0x7fd97c0c2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd981845640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd97c150c40 0x7fd97c0c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd981845640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c3960 con 0x7fd97c150720 2023-08-25T09:47:02.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd981845640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c5570 con 0x7fd96c003970 2023-08-25T09:47:02.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd981845640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0c5850 con 0x7fd97c150c40 2023-08-25T09:47:02.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd97bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 0x7fd97c0c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd97bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 0x7fd97c0c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60352/0 (socket says 172.21.15.70:60352) 2023-08-25T09:47:02.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.973+0000 7fd97bfff640 1 -- 172.21.15.70:0/2362503877 learned_addr learned my addr 172.21.15.70:0/2362503877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:02.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd981044640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd97c150c40 0x7fd97c0c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd980843640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c003970 0x7fd97c0c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd97bfff640 1 -- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd97c150c40 msgr2=0x7fd97c0c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd97bfff640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd97c150c40 0x7fd97c0c2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd97bfff640 1 -- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c003970 msgr2=0x7fd97c0c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:02.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd97bfff640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c003970 0x7fd97c0c1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:02.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd97bfff640 1 -- 172.21.15.70:0/2362503877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c0c5b30 con 0x7fd97c150720 2023-08-25T09:47:02.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd97bfff640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 0x7fd97c0c2510 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c3c0 tx=0x7fd96800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968011790 con 0x7fd97c150720 2023-08-25T09:47:02.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9680118f0 con 0x7fd97c150720 2023-08-25T09:47:02.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c0c5e10 con 0x7fd97c150720 2023-08-25T09:47:02.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968011ce0 con 0x7fd97c150720 2023-08-25T09:47:02.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.974+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c0c6610 con 0x7fd97c150720 2023-08-25T09:47:02.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.975+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c066fb0 con 0x7fd97c150720 2023-08-25T09:47:02.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.976+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd96801c460 con 0x7fd97c150720 2023-08-25T09:47:02.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.977+0000 7fd9797fa640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd96006a050 0x7fd96006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:02.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.977+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd968088580 con 0x7fd97c150720 2023-08-25T09:47:02.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.978+0000 7fd980843640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd96006a050 0x7fd96006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:02.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.978+0000 7fd980843640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd96006a050 0x7fd96006c510 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd970034000 tx=0x7fd970007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:02.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:02.982+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd968053bd0 con 0x7fd97c150720 2023-08-25T09:47:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.164+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd97c0c59c0 con 0x7fd96006a050 2023-08-25T09:47:03.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.178+0000 7fd9797fa640 1 -- 172.21.15.70:0/2362503877 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303420 (secure 0 0 0) 0x7fd97c0c59c0 con 0x7fd96006a050 2023-08-25T09:47:03.184 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:03.186 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:47:03.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.184+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd96006a050 msgr2=0x7fd96006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.184+0000 7fd981845640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd96006a050 0x7fd96006c510 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd970034000 tx=0x7fd970007000 comp rx=0 tx=0).stop 2023-08-25T09:47:03.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.184+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 msgr2=0x7fd97c0c2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.184+0000 7fd981845640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 0x7fd97c0c2510 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c3c0 tx=0x7fd96800c7c0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 shutdown_connections 2023-08-25T09:47:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd96006a050 0x7fd96006c510 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd97c150c40 0x7fd97c0c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd97c150720 0x7fd97c0c2510 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 --2- 172.21.15.70:0/2362503877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd96c003970 0x7fd97c0c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 >> 172.21.15.70:0/2362503877 conn(0x7fd97c0affd0 msgr2=0x7fd97c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:03.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 shutdown_connections 2023-08-25T09:47:03.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.185+0000 7fd981845640 1 -- 172.21.15.70:0/2362503877 wait complete. 2023-08-25T09:47:03.207 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2023-08-25T09:47:01.481970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8063,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554436,"statfs":{"total":1600000000,"available":1591742778,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.706446"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.410114+0000","last_change":"2023-08-25T09:46:58.410027+0000","last_active":"2023-08-25T09:46:58.410114+0000","last_peered":"2023-08-25T09:46:58.410114+0000","last_clean":"2023-08-25T09:46:58.410114+0000","last_became_active":"2023-08-25T09:46:58.408039+0000","last_became_peered":"2023-08-25T09:46:58.408039+0000","last_unstale":"2023-08-25T09:46:58.410114+0000","last_undegraded":"2023-08-25T09:46:58.410114+0000","last_fullsized":"2023-08-25T09:46:58.410114+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:43.122225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382132+0000","last_change":"2023-08-25T09:46:58.408833+0000","last_active":"2023-08-25T09:46:59.382132+0000","last_peered":"2023-08-25T09:46:59.382132+0000","last_clean":"2023-08-25T09:46:59.382132+0000","last_became_active":"2023-08-25T09:46:58.408584+0000","last_became_peered":"2023-08-25T09:46:58.408584+0000","last_unstale":"2023-08-25T09:46:59.382132+0000","last_undegraded":"2023-08-25T09:46:59.382132+0000","last_fullsized":"2023-08-25T09:46:59.382132+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:37:41.988324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386709+0000","last_change":"2023-08-25T09:46:58.398252+0000","last_active":"2023-08-25T09:46:59.386709+0000","last_peered":"2023-08-25T09:46:59.386709+0000","last_clean":"2023-08-25T09:46:59.386709+0000","last_became_active":"2023-08-25T09:46:58.397685+0000","last_became_peered":"2023-08-25T09:46:58.397685+0000","last_unstale":"2023-08-25T09:46:59.386709+0000","last_undegraded":"2023-08-25T09:46:59.386709+0000","last_fullsized":"2023-08-25T09:46:59.386709+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:14.268254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391977+0000","last_change":"2023-08-25T09:46:58.391642+0000","last_active":"2023-08-25T09:46:58.391977+0000","last_peered":"2023-08-25T09:46:58.391977+0000","last_clean":"2023-08-25T09:46:58.391977+0000","last_became_active":"2023-08-25T09:46:58.389974+0000","last_became_peered":"2023-08-25T09:46:58.389974+0000","last_unstale":"2023-08-25T09:46:58.391977+0000","last_undegraded":"2023-08-25T09:46:58.391977+0000","last_fullsized":"2023-08-25T09:46:58.391977+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:49.387469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.404040+0000","last_change":"2023-08-25T09:46:58.403928+0000","last_active":"2023-08-25T09:46:58.404040+0000","last_peered":"2023-08-25T09:46:58.404040+0000","last_clean":"2023-08-25T09:46:58.404040+0000","last_became_active":"2023-08-25T09:46:58.403595+0000","last_became_peered":"2023-08-25T09:46:58.403595+0000","last_unstale":"2023-08-25T09:46:58.404040+0000","last_undegraded":"2023-08-25T09:46:58.404040+0000","last_fullsized":"2023-08-25T09:46:58.404040+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:39.656363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382915+0000","last_change":"2023-08-25T09:46:58.408452+0000","last_active":"2023-08-25T09:46:59.382915+0000","last_peered":"2023-08-25T09:46:59.382915+0000","last_clean":"2023-08-25T09:46:59.382915+0000","last_became_active":"2023-08-25T09:46:58.406452+0000","last_became_peered":"2023-08-25T09:46:58.406452+0000","last_unstale":"2023-08-25T09:46:59.382915+0000","last_undegraded":"2023-08-25T09:46:59.382915+0000","last_fullsized":"2023-08-25T09:46:59.382915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:37:00.602117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382463+0000","last_change":"2023-08-25T09:46:58.400258+0000","last_active":"2023-08-25T09:46:59.382463+0000","last_peered":"2023-08-25T09:46:59.382463+0000","last_clean":"2023-08-25T09:46:59.382463+0000","last_became_active":"2023-08-25T09:46:58.399966+0000","last_became_peered":"2023-08-25T09:46:58.399966+0000","last_unstale":"2023-08-25T09:46:59.382463+0000","last_undegraded":"2023-08-25T09:46:59.382463+0000","last_fullsized":"2023-08-25T09:46:59.382463+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:56:20.426610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.396513+0000","last_change":"2023-08-25T09:46:58.395499+0000","last_active":"2023-08-25T09:46:58.396513+0000","last_peered":"2023-08-25T09:46:58.396513+0000","last_clean":"2023-08-25T09:46:58.396513+0000","last_became_active":"2023-08-25T09:46:58.394182+0000","last_became_peered":"2023-08-25T09:46:58.394182+0000","last_unstale":"2023-08-25T09:46:58.396513+0000","last_undegraded":"2023-08-25T09:46:58.396513+0000","last_fullsized":"2023-08-25T09:46:58.396513+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:56.684770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.404849+0000","last_change":"2023-08-25T09:46:58.404708+0000","last_active":"2023-08-25T09:46:58.404849+0000","last_peered":"2023-08-25T09:46:58.404849+0000","last_clean":"2023-08-25T09:46:58.404849+0000","last_became_active":"2023-08-25T09:46:58.404278+0000","last_became_peered":"2023-08-25T09:46:58.404278+0000","last_unstale":"2023-08-25T09:46:58.404849+0000","last_undegraded":"2023-08-25T09:46:58.404849+0000","last_fullsized":"2023-08-25T09:46:58.404849+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:06:03.539567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382578+0000","last_change":"2023-08-25T09:46:58.403349+0000","last_active":"2023-08-25T09:46:59.382578+0000","last_peered":"2023-08-25T09:46:59.382578+0000","last_clean":"2023-08-25T09:46:59.382578+0000","last_became_active":"2023-08-25T09:46:58.402710+0000","last_became_peered":"2023-08-25T09:46:58.402710+0000","last_unstale":"2023-08-25T09:46:59.382578+0000","last_undegraded":"2023-08-25T09:46:59.382578+0000","last_fullsized":"2023-08-25T09:46:59.382578+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:15:30.773638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409894+0000","last_change":"2023-08-25T09:46:58.408735+0000","last_active":"2023-08-25T09:46:58.409894+0000","last_peered":"2023-08-25T09:46:58.409894+0000","last_clean":"2023-08-25T09:46:58.409894+0000","last_became_active":"2023-08-25T09:46:58.407589+0000","last_became_peered":"2023-08-25T09:46:58.407589+0000","last_unstale":"2023-08-25T09:46:58.409894+0000","last_undegraded":"2023-08-25T09:46:58.409894+0000","last_fullsized":"2023-08-25T09:46:58.409894+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:40:22.145671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385653+0000","last_change":"2023-08-25T09:46:58.404188+0000","last_active":"2023-08-25T09:46:59.385653+0000","last_peered":"2023-08-25T09:46:59.385653+0000","last_clean":"2023-08-25T09:46:59.385653+0000","last_became_active":"2023-08-25T09:46:58.403816+0000","last_became_peered":"2023-08-25T09:46:58.403816+0000","last_unstale":"2023-08-25T09:46:59.385653+0000","last_undegraded":"2023-08-25T09:46:59.385653+0000","last_fullsized":"2023-08-25T09:46:59.385653+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:42:14.053437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382608+0000","last_change":"2023-08-25T09:46:58.400494+0000","last_active":"2023-08-25T09:46:59.382608+0000","last_peered":"2023-08-25T09:46:59.382608+0000","last_clean":"2023-08-25T09:46:59.382608+0000","last_became_active":"2023-08-25T09:46:58.400143+0000","last_became_peered":"2023-08-25T09:46:58.400143+0000","last_unstale":"2023-08-25T09:46:59.382608+0000","last_undegraded":"2023-08-25T09:46:59.382608+0000","last_fullsized":"2023-08-25T09:46:59.382608+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:29.729115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385687+0000","last_change":"2023-08-25T09:46:58.408730+0000","last_active":"2023-08-25T09:46:59.385687+0000","last_peered":"2023-08-25T09:46:59.385687+0000","last_clean":"2023-08-25T09:46:59.385687+0000","last_became_active":"2023-08-25T09:46:58.408363+0000","last_became_peered":"2023-08-25T09:46:58.408363+0000","last_unstale":"2023-08-25T09:46:59.385687+0000","last_undegraded":"2023-08-25T09:46:59.385687+0000","last_fullsized":"2023-08-25T09:46:59.385687+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:10:08.157400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409398+0000","last_change":"2023-08-25T09:46:58.409240+0000","last_active":"2023-08-25T09:46:58.409398+0000","last_peered":"2023-08-25T09:46:58.409398+0000","last_clean":"2023-08-25T09:46:58.409398+0000","last_became_active":"2023-08-25T09:46:58.405588+0000","last_became_peered":"2023-08-25T09:46:58.405588+0000","last_unstale":"2023-08-25T09:46:58.409398+0000","last_undegraded":"2023-08-25T09:46:58.409398+0000","last_fullsized":"2023-08-25T09:46:58.409398+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:36:07.055865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387177+0000","last_change":"2023-08-25T09:46:58.400189+0000","last_active":"2023-08-25T09:46:59.387177+0000","last_peered":"2023-08-25T09:46:59.387177+0000","last_clean":"2023-08-25T09:46:59.387177+0000","last_became_active":"2023-08-25T09:46:58.399709+0000","last_became_peered":"2023-08-25T09:46:58.399709+0000","last_unstale":"2023-08-25T09:46:59.387177+0000","last_undegraded":"2023-08-25T09:46:59.387177+0000","last_fullsized":"2023-08-25T09:46:59.387177+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:19:35.074854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385983+0000","last_change":"2023-08-25T09:46:58.407130+0000","last_active":"2023-08-25T09:46:59.385983+0000","last_peered":"2023-08-25T09:46:59.385983+0000","last_clean":"2023-08-25T09:46:59.385983+0000","last_became_active":"2023-08-25T09:46:58.406902+0000","last_became_peered":"2023-08-25T09:46:58.406902+0000","last_unstale":"2023-08-25T09:46:59.385983+0000","last_undegraded":"2023-08-25T09:46:59.385983+0000","last_fullsized":"2023-08-25T09:46:59.385983+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:58:37.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385848+0000","last_change":"2023-08-25T09:46:58.403983+0000","last_active":"2023-08-25T09:46:59.385848+0000","last_peered":"2023-08-25T09:46:59.385848+0000","last_clean":"2023-08-25T09:46:59.385848+0000","last_became_active":"2023-08-25T09:46:58.402479+0000","last_became_peered":"2023-08-25T09:46:58.402479+0000","last_unstale":"2023-08-25T09:46:59.385848+0000","last_undegraded":"2023-08-25T09:46:59.385848+0000","last_fullsized":"2023-08-25T09:46:59.385848+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:58.773715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.411031+0000","last_change":"2023-08-25T09:46:58.410811+0000","last_active":"2023-08-25T09:46:58.411031+0000","last_peered":"2023-08-25T09:46:58.411031+0000","last_clean":"2023-08-25T09:46:58.411031+0000","last_became_active":"2023-08-25T09:46:58.407902+0000","last_became_peered":"2023-08-25T09:46:58.407902+0000","last_unstale":"2023-08-25T09:46:58.411031+0000","last_undegraded":"2023-08-25T09:46:58.411031+0000","last_fullsized":"2023-08-25T09:46:58.411031+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:57:25.269137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382963+0000","last_change":"2023-08-25T09:46:58.403176+0000","last_active":"2023-08-25T09:46:59.382963+0000","last_peered":"2023-08-25T09:46:59.382963+0000","last_clean":"2023-08-25T09:46:59.382963+0000","last_became_active":"2023-08-25T09:46:58.402356+0000","last_became_peered":"2023-08-25T09:46:58.402356+0000","last_unstale":"2023-08-25T09:46:59.382963+0000","last_undegraded":"2023-08-25T09:46:59.382963+0000","last_fullsized":"2023-08-25T09:46:59.382963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:40.112749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.410381+0000","last_change":"2023-08-25T09:46:58.410312+0000","last_active":"2023-08-25T09:46:58.410381+0000","last_peered":"2023-08-25T09:46:58.410381+0000","last_clean":"2023-08-25T09:46:58.410381+0000","last_became_active":"2023-08-25T09:46:58.410049+0000","last_became_peered":"2023-08-25T09:46:58.410049+0000","last_unstale":"2023-08-25T09:46:58.410381+0000","last_undegraded":"2023-08-25T09:46:58.410381+0000","last_fullsized":"2023-08-25T09:46:58.410381+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:58.219410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408629+0000","last_change":"2023-08-25T09:46:58.408468+0000","last_active":"2023-08-25T09:46:58.408629+0000","last_peered":"2023-08-25T09:46:58.408629+0000","last_clean":"2023-08-25T09:46:58.408629+0000","last_became_active":"2023-08-25T09:46:58.408171+0000","last_became_peered":"2023-08-25T09:46:58.408171+0000","last_unstale":"2023-08-25T09:46:58.408629+0000","last_undegraded":"2023-08-25T09:46:58.408629+0000","last_fullsized":"2023-08-25T09:46:58.408629+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:52.783619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.407122+0000","last_change":"2023-08-25T09:46:58.407047+0000","last_active":"2023-08-25T09:46:58.407122+0000","last_peered":"2023-08-25T09:46:58.407122+0000","last_clean":"2023-08-25T09:46:58.407122+0000","last_became_active":"2023-08-25T09:46:58.406822+0000","last_became_peered":"2023-08-25T09:46:58.406822+0000","last_unstale":"2023-08-25T09:46:58.407122+0000","last_undegraded":"2023-08-25T09:46:58.407122+0000","last_fullsized":"2023-08-25T09:46:58.407122+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:13:33.254186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409605+0000","last_change":"2023-08-25T09:46:58.409474+0000","last_active":"2023-08-25T09:46:58.409605+0000","last_peered":"2023-08-25T09:46:58.409605+0000","last_clean":"2023-08-25T09:46:58.409605+0000","last_became_active":"2023-08-25T09:46:58.409020+0000","last_became_peered":"2023-08-25T09:46:58.409020+0000","last_unstale":"2023-08-25T09:46:58.409605+0000","last_undegraded":"2023-08-25T09:46:58.409605+0000","last_fullsized":"2023-08-25T09:46:58.409605+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:37:41.601980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408670+0000","last_change":"2023-08-25T09:46:58.408433+0000","last_active":"2023-08-25T09:46:58.408670+0000","last_peered":"2023-08-25T09:46:58.408670+0000","last_clean":"2023-08-25T09:46:58.408670+0000","last_became_active":"2023-08-25T09:46:58.408198+0000","last_became_peered":"2023-08-25T09:46:58.408198+0000","last_unstale":"2023-08-25T09:46:58.408670+0000","last_undegraded":"2023-08-25T09:46:58.408670+0000","last_fullsized":"2023-08-25T09:46:58.408670+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:49.170623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409567+0000","last_change":"2023-08-25T09:46:58.409478+0000","last_active":"2023-08-25T09:46:58.409567+0000","last_peered":"2023-08-25T09:46:58.409567+0000","last_clean":"2023-08-25T09:46:58.409567+0000","last_became_active":"2023-08-25T09:46:58.409208+0000","last_became_peered":"2023-08-25T09:46:58.409208+0000","last_unstale":"2023-08-25T09:46:58.409567+0000","last_undegraded":"2023-08-25T09:46:58.409567+0000","last_fullsized":"2023-08-25T09:46:58.409567+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:24:05.357659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408711+0000","last_change":"2023-08-25T09:46:58.408261+0000","last_active":"2023-08-25T09:46:58.408711+0000","last_peered":"2023-08-25T09:46:58.408711+0000","last_clean":"2023-08-25T09:46:58.408711+0000","last_became_active":"2023-08-25T09:46:58.407962+0000","last_became_peered":"2023-08-25T09:46:58.407962+0000","last_unstale":"2023-08-25T09:46:58.408711+0000","last_undegraded":"2023-08-25T09:46:58.408711+0000","last_fullsized":"2023-08-25T09:46:58.408711+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:58.054230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386260+0000","last_change":"2023-08-25T09:46:58.405379+0000","last_active":"2023-08-25T09:46:59.386260+0000","last_peered":"2023-08-25T09:46:59.386260+0000","last_clean":"2023-08-25T09:46:59.386260+0000","last_became_active":"2023-08-25T09:46:58.405025+0000","last_became_peered":"2023-08-25T09:46:58.405025+0000","last_unstale":"2023-08-25T09:46:59.386260+0000","last_undegraded":"2023-08-25T09:46:59.386260+0000","last_fullsized":"2023-08-25T09:46:59.386260+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:20.680767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385743+0000","last_change":"2023-08-25T09:46:58.401820+0000","last_active":"2023-08-25T09:46:59.385743+0000","last_peered":"2023-08-25T09:46:59.385743+0000","last_clean":"2023-08-25T09:46:59.385743+0000","last_became_active":"2023-08-25T09:46:58.399296+0000","last_became_peered":"2023-08-25T09:46:58.399296+0000","last_unstale":"2023-08-25T09:46:59.385743+0000","last_undegraded":"2023-08-25T09:46:59.385743+0000","last_fullsized":"2023-08-25T09:46:59.385743+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:42:18.572044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387900+0000","last_change":"2023-08-25T09:46:58.407120+0000","last_active":"2023-08-25T09:46:59.387900+0000","last_peered":"2023-08-25T09:46:59.387900+0000","last_clean":"2023-08-25T09:46:59.387900+0000","last_became_active":"2023-08-25T09:46:58.406817+0000","last_became_peered":"2023-08-25T09:46:58.406817+0000","last_unstale":"2023-08-25T09:46:59.387900+0000","last_undegraded":"2023-08-25T09:46:59.387900+0000","last_fullsized":"2023-08-25T09:46:59.387900+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:28:06.714953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402873+0000","last_change":"2023-08-25T09:46:58.401705+0000","last_active":"2023-08-25T09:46:58.402873+0000","last_peered":"2023-08-25T09:46:58.402873+0000","last_clean":"2023-08-25T09:46:58.402873+0000","last_became_active":"2023-08-25T09:46:58.401131+0000","last_became_peered":"2023-08-25T09:46:58.401131+0000","last_unstale":"2023-08-25T09:46:58.402873+0000","last_undegraded":"2023-08-25T09:46:58.402873+0000","last_fullsized":"2023-08-25T09:46:58.402873+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:13:22.777432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402828+0000","last_change":"2023-08-25T09:46:58.400062+0000","last_active":"2023-08-25T09:46:58.402828+0000","last_peered":"2023-08-25T09:46:58.402828+0000","last_clean":"2023-08-25T09:46:58.402828+0000","last_became_active":"2023-08-25T09:46:58.399454+0000","last_became_peered":"2023-08-25T09:46:58.399454+0000","last_unstale":"2023-08-25T09:46:58.402828+0000","last_undegraded":"2023-08-25T09:46:58.402828+0000","last_fullsized":"2023-08-25T09:46:58.402828+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:46:43.565623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.400325+0000","last_change":"2023-08-25T09:46:58.400230+0000","last_active":"2023-08-25T09:46:58.400325+0000","last_peered":"2023-08-25T09:46:58.400325+0000","last_clean":"2023-08-25T09:46:58.400325+0000","last_became_active":"2023-08-25T09:46:58.399880+0000","last_became_peered":"2023-08-25T09:46:58.399880+0000","last_unstale":"2023-08-25T09:46:58.400325+0000","last_undegraded":"2023-08-25T09:46:58.400325+0000","last_fullsized":"2023-08-25T09:46:58.400325+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:02.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"149'4","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.840035+0000","last_change":"2023-08-25T09:46:57.796710+0000","last_active":"2023-08-25T09:46:58.840035+0000","last_peered":"2023-08-25T09:46:58.840035+0000","last_clean":"2023-08-25T09:46:58.840035+0000","last_became_active":"2023-08-25T09:46:57.796414+0000","last_became_peered":"2023-08-25T09:46:57.796414+0000","last_unstale":"2023-08-25T09:46:58.840035+0000","last_undegraded":"2023-08-25T09:46:58.840035+0000","last_fullsized":"2023-08-25T09:46:58.840035+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:35:49.005173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383314+0000","last_change":"2023-08-25T09:46:58.402995+0000","last_active":"2023-08-25T09:46:59.383314+0000","last_peered":"2023-08-25T09:46:59.383314+0000","last_clean":"2023-08-25T09:46:59.383314+0000","last_became_active":"2023-08-25T09:46:58.401926+0000","last_became_peered":"2023-08-25T09:46:58.401926+0000","last_unstale":"2023-08-25T09:46:59.383314+0000","last_undegraded":"2023-08-25T09:46:59.383314+0000","last_fullsized":"2023-08-25T09:46:59.383314+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:33:04.852695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"149'1","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386459+0000","last_change":"2023-08-25T09:46:57.807213+0000","last_active":"2023-08-25T09:46:59.386459+0000","last_peered":"2023-08-25T09:46:59.386459+0000","last_clean":"2023-08-25T09:46:59.386459+0000","last_became_active":"2023-08-25T09:46:57.806785+0000","last_became_peered":"2023-08-25T09:46:57.806785+0000","last_unstale":"2023-08-25T09:46:59.386459+0000","last_undegraded":"2023-08-25T09:46:59.386459+0000","last_fullsized":"2023-08-25T09:46:59.386459+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:23.429545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.406688+0000","last_change":"2023-08-25T09:46:58.405960+0000","last_active":"2023-08-25T09:46:58.406688+0000","last_peered":"2023-08-25T09:46:58.406688+0000","last_clean":"2023-08-25T09:46:58.406688+0000","last_became_active":"2023-08-25T09:46:58.405638+0000","last_became_peered":"2023-08-25T09:46:58.405638+0000","last_unstale":"2023-08-25T09:46:58.406688+0000","last_undegraded":"2023-08-25T09:46:58.406688+0000","last_fullsized":"2023-08-25T09:46:58.406688+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:21.391199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"149'2","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.839410+0000","last_change":"2023-08-25T09:46:57.804438+0000","last_active":"2023-08-25T09:46:58.839410+0000","last_peered":"2023-08-25T09:46:58.839410+0000","last_clean":"2023-08-25T09:46:58.839410+0000","last_became_active":"2023-08-25T09:46:57.804124+0000","last_became_peered":"2023-08-25T09:46:57.804124+0000","last_unstale":"2023-08-25T09:46:58.839410+0000","last_undegraded":"2023-08-25T09:46:58.839410+0000","last_fullsized":"2023-08-25T09:46:58.839410+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:05.689473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391002+0000","last_change":"2023-08-25T09:46:58.390625+0000","last_active":"2023-08-25T09:46:58.391002+0000","last_peered":"2023-08-25T09:46:58.391002+0000","last_clean":"2023-08-25T09:46:58.391002+0000","last_became_active":"2023-08-25T09:46:58.389486+0000","last_became_peered":"2023-08-25T09:46:58.389486+0000","last_unstale":"2023-08-25T09:46:58.391002+0000","last_undegraded":"2023-08-25T09:46:58.391002+0000","last_fullsized":"2023-08-25T09:46:58.391002+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:27:39.787823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383836+0000","last_change":"2023-08-25T09:46:57.806745+0000","last_active":"2023-08-25T09:46:59.383836+0000","last_peered":"2023-08-25T09:46:59.383836+0000","last_clean":"2023-08-25T09:46:59.383836+0000","last_became_active":"2023-08-25T09:46:57.806189+0000","last_became_peered":"2023-08-25T09:46:57.806189+0000","last_unstale":"2023-08-25T09:46:59.383836+0000","last_undegraded":"2023-08-25T09:46:59.383836+0000","last_fullsized":"2023-08-25T09:46:59.383836+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:10:47.537865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402713+0000","last_change":"2023-08-25T09:46:58.401213+0000","last_active":"2023-08-25T09:46:58.402713+0000","last_peered":"2023-08-25T09:46:58.402713+0000","last_clean":"2023-08-25T09:46:58.402713+0000","last_became_active":"2023-08-25T09:46:58.400822+0000","last_became_peered":"2023-08-25T09:46:58.400822+0000","last_unstale":"2023-08-25T09:46:58.402713+0000","last_undegraded":"2023-08-25T09:46:58.402713+0000","last_fullsized":"2023-08-25T09:46:58.402713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:11.015122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"149'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.836433+0000","last_change":"2023-08-25T09:46:57.802141+0000","last_active":"2023-08-25T09:46:58.836433+0000","last_peered":"2023-08-25T09:46:58.836433+0000","last_clean":"2023-08-25T09:46:58.836433+0000","last_became_active":"2023-08-25T09:46:57.801615+0000","last_became_peered":"2023-08-25T09:46:57.801615+0000","last_unstale":"2023-08-25T09:46:58.836433+0000","last_undegraded":"2023-08-25T09:46:58.836433+0000","last_fullsized":"2023-08-25T09:46:58.836433+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:17:39.769119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402777+0000","last_change":"2023-08-25T09:46:58.394876+0000","last_active":"2023-08-25T09:46:58.402777+0000","last_peered":"2023-08-25T09:46:58.402777+0000","last_clean":"2023-08-25T09:46:58.402777+0000","last_became_active":"2023-08-25T09:46:58.394202+0000","last_became_peered":"2023-08-25T09:46:58.394202+0000","last_unstale":"2023-08-25T09:46:58.402777+0000","last_undegraded":"2023-08-25T09:46:58.402777+0000","last_fullsized":"2023-08-25T09:46:58.402777+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:34:24.862537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"149'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.839134+0000","last_change":"2023-08-25T09:46:57.808292+0000","last_active":"2023-08-25T09:46:58.839134+0000","last_peered":"2023-08-25T09:46:58.839134+0000","last_clean":"2023-08-25T09:46:58.839134+0000","last_became_active":"2023-08-25T09:46:57.808056+0000","last_became_peered":"2023-08-25T09:46:57.808056+0000","last_unstale":"2023-08-25T09:46:58.839134+0000","last_undegraded":"2023-08-25T09:46:58.839134+0000","last_fullsized":"2023-08-25T09:46:58.839134+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:40.774858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387067+0000","last_change":"2023-08-25T09:46:58.393063+0000","last_active":"2023-08-25T09:46:59.387067+0000","last_peered":"2023-08-25T09:46:59.387067+0000","last_clean":"2023-08-25T09:46:59.387067+0000","last_became_active":"2023-08-25T09:46:58.392680+0000","last_became_peered":"2023-08-25T09:46:58.392680+0000","last_unstale":"2023-08-25T09:46:59.387067+0000","last_undegraded":"2023-08-25T09:46:59.387067+0000","last_fullsized":"2023-08-25T09:46:59.387067+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:33:43.049388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.387985+0000","last_change":"2023-08-25T09:46:57.804467+0000","last_active":"2023-08-25T09:46:58.387985+0000","last_peered":"2023-08-25T09:46:58.387985+0000","last_clean":"2023-08-25T09:46:58.387985+0000","last_became_active":"2023-08-25T09:46:57.804133+0000","last_became_peered":"2023-08-25T09:46:57.804133+0000","last_unstale":"2023-08-25T09:46:58.387985+0000","last_undegraded":"2023-08-25T09:46:58.387985+0000","last_fullsized":"2023-08-25T09:46:58.387985+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:10.928801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408802+0000","last_change":"2023-08-25T09:46:58.408726+0000","last_active":"2023-08-25T09:46:58.408802+0000","last_peered":"2023-08-25T09:46:58.408802+0000","last_clean":"2023-08-25T09:46:58.408802+0000","last_became_active":"2023-08-25T09:46:58.408481+0000","last_became_peered":"2023-08-25T09:46:58.408481+0000","last_unstale":"2023-08-25T09:46:58.408802+0000","last_undegraded":"2023-08-25T09:46:58.408802+0000","last_fullsized":"2023-08-25T09:46:58.408802+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:42.466186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388194+0000","last_change":"2023-08-25T09:46:57.795116+0000","last_active":"2023-08-25T09:46:59.388194+0000","last_peered":"2023-08-25T09:46:59.388194+0000","last_clean":"2023-08-25T09:46:59.388194+0000","last_became_active":"2023-08-25T09:46:57.794578+0000","last_became_peered":"2023-08-25T09:46:57.794578+0000","last_unstale":"2023-08-25T09:46:59.388194+0000","last_undegraded":"2023-08-25T09:46:59.388194+0000","last_fullsized":"2023-08-25T09:46:59.388194+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:33:50.170382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.400327+0000","last_change":"2023-08-25T09:46:58.397926+0000","last_active":"2023-08-25T09:46:58.400327+0000","last_peered":"2023-08-25T09:46:58.400327+0000","last_clean":"2023-08-25T09:46:58.400327+0000","last_became_active":"2023-08-25T09:46:58.397314+0000","last_became_peered":"2023-08-25T09:46:58.397314+0000","last_unstale":"2023-08-25T09:46:58.400327+0000","last_undegraded":"2023-08-25T09:46:58.400327+0000","last_fullsized":"2023-08-25T09:46:58.400327+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:23.468922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"149'1","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386801+0000","last_change":"2023-08-25T09:46:57.807033+0000","last_active":"2023-08-25T09:46:59.386801+0000","last_peered":"2023-08-25T09:46:59.386801+0000","last_clean":"2023-08-25T09:46:59.386801+0000","last_became_active":"2023-08-25T09:46:57.806435+0000","last_became_peered":"2023-08-25T09:46:57.806435+0000","last_unstale":"2023-08-25T09:46:59.386801+0000","last_undegraded":"2023-08-25T09:46:59.386801+0000","last_fullsized":"2023-08-25T09:46:59.386801+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:20.802523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.410556+0000","last_change":"2023-08-25T09:46:58.410469+0000","last_active":"2023-08-25T09:46:58.410556+0000","last_peered":"2023-08-25T09:46:58.410556+0000","last_clean":"2023-08-25T09:46:58.410556+0000","last_became_active":"2023-08-25T09:46:58.408436+0000","last_became_peered":"2023-08-25T09:46:58.408436+0000","last_unstale":"2023-08-25T09:46:58.410556+0000","last_undegraded":"2023-08-25T09:46:58.410556+0000","last_fullsized":"2023-08-25T09:46:58.410556+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:54.212755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.389029+0000","last_change":"2023-08-25T09:46:57.804036+0000","last_active":"2023-08-25T09:46:59.389029+0000","last_peered":"2023-08-25T09:46:59.389029+0000","last_clean":"2023-08-25T09:46:59.389029+0000","last_became_active":"2023-08-25T09:46:57.803048+0000","last_became_peered":"2023-08-25T09:46:57.803048+0000","last_unstale":"2023-08-25T09:46:59.389029+0000","last_undegraded":"2023-08-25T09:46:59.389029+0000","last_fullsized":"2023-08-25T09:46:59.389029+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:48:36.771533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383675+0000","last_change":"2023-08-25T09:46:58.410561+0000","last_active":"2023-08-25T09:46:59.383675+0000","last_peered":"2023-08-25T09:46:59.383675+0000","last_clean":"2023-08-25T09:46:59.383675+0000","last_became_active":"2023-08-25T09:46:58.410158+0000","last_became_peered":"2023-08-25T09:46:58.410158+0000","last_unstale":"2023-08-25T09:46:59.383675+0000","last_undegraded":"2023-08-25T09:46:59.383675+0000","last_fullsized":"2023-08-25T09:46:59.383675+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:28:25.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"149'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.839479+0000","last_change":"2023-08-25T09:46:57.802901+0000","last_active":"2023-08-25T09:46:58.839479+0000","last_peered":"2023-08-25T09:46:58.839479+0000","last_clean":"2023-08-25T09:46:58.839479+0000","last_became_active":"2023-08-25T09:46:57.802360+0000","last_became_peered":"2023-08-25T09:46:57.802360+0000","last_unstale":"2023-08-25T09:46:58.839479+0000","last_undegraded":"2023-08-25T09:46:58.839479+0000","last_fullsized":"2023-08-25T09:46:58.839479+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:31:04.471433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.411710+0000","last_change":"2023-08-25T09:46:58.411092+0000","last_active":"2023-08-25T09:46:58.411710+0000","last_peered":"2023-08-25T09:46:58.411710+0000","last_clean":"2023-08-25T09:46:58.411710+0000","last_became_active":"2023-08-25T09:46:58.409848+0000","last_became_peered":"2023-08-25T09:46:58.409848+0000","last_unstale":"2023-08-25T09:46:58.411710+0000","last_undegraded":"2023-08-25T09:46:58.411710+0000","last_fullsized":"2023-08-25T09:46:58.411710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:30:18.037690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.387888+0000","last_change":"2023-08-25T09:46:57.807397+0000","last_active":"2023-08-25T09:46:58.387888+0000","last_peered":"2023-08-25T09:46:58.387888+0000","last_clean":"2023-08-25T09:46:58.387888+0000","last_became_active":"2023-08-25T09:46:57.806671+0000","last_became_peered":"2023-08-25T09:46:57.806671+0000","last_unstale":"2023-08-25T09:46:58.387888+0000","last_undegraded":"2023-08-25T09:46:58.387888+0000","last_fullsized":"2023-08-25T09:46:58.387888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:00:37.854739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383469+0000","last_change":"2023-08-25T09:46:58.407697+0000","last_active":"2023-08-25T09:46:59.383469+0000","last_peered":"2023-08-25T09:46:59.383469+0000","last_clean":"2023-08-25T09:46:59.383469+0000","last_became_active":"2023-08-25T09:46:58.402997+0000","last_became_peered":"2023-08-25T09:46:58.402997+0000","last_unstale":"2023-08-25T09:46:59.383469+0000","last_undegraded":"2023-08-25T09:46:59.383469+0000","last_fullsized":"2023-08-25T09:46:59.383469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:53.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386798+0000","last_change":"2023-08-25T09:46:57.809731+0000","last_active":"2023-08-25T09:46:59.386798+0000","last_peered":"2023-08-25T09:46:59.386798+0000","last_clean":"2023-08-25T09:46:59.386798+0000","last_became_active":"2023-08-25T09:46:57.808952+0000","last_became_peered":"2023-08-25T09:46:57.808952+0000","last_unstale":"2023-08-25T09:46:59.386798+0000","last_undegraded":"2023-08-25T09:46:59.386798+0000","last_fullsized":"2023-08-25T09:46:59.386798+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:42:17.566319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388367+0000","last_change":"2023-08-25T09:46:58.409531+0000","last_active":"2023-08-25T09:46:59.388367+0000","last_peered":"2023-08-25T09:46:59.388367+0000","last_clean":"2023-08-25T09:46:59.388367+0000","last_became_active":"2023-08-25T09:46:58.409239+0000","last_became_peered":"2023-08-25T09:46:58.409239+0000","last_unstale":"2023-08-25T09:46:59.388367+0000","last_undegraded":"2023-08-25T09:46:59.388367+0000","last_fullsized":"2023-08-25T09:46:59.388367+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:33:43.049388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.396704+0000","last_change":"2023-08-25T09:46:57.802821+0000","last_active":"2023-08-25T09:46:58.396704+0000","last_peered":"2023-08-25T09:46:58.396704+0000","last_clean":"2023-08-25T09:46:58.396704+0000","last_became_active":"2023-08-25T09:46:57.802505+0000","last_became_peered":"2023-08-25T09:46:57.802505+0000","last_unstale":"2023-08-25T09:46:58.396704+0000","last_undegraded":"2023-08-25T09:46:58.396704+0000","last_fullsized":"2023-08-25T09:46:58.396704+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:17.002089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385389+0000","last_change":"2023-08-25T09:46:58.401628+0000","last_active":"2023-08-25T09:46:59.385389+0000","last_peered":"2023-08-25T09:46:59.385389+0000","last_clean":"2023-08-25T09:46:59.385389+0000","last_became_active":"2023-08-25T09:46:58.398832+0000","last_became_peered":"2023-08-25T09:46:58.398832+0000","last_unstale":"2023-08-25T09:46:59.385389+0000","last_undegraded":"2023-08-25T09:46:59.385389+0000","last_fullsized":"2023-08-25T09:46:59.385389+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:24.435271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388487+0000","last_change":"2023-08-25T09:46:57.808957+0000","last_active":"2023-08-25T09:46:59.388487+0000","last_peered":"2023-08-25T09:46:59.388487+0000","last_clean":"2023-08-25T09:46:59.388487+0000","last_became_active":"2023-08-25T09:46:57.806512+0000","last_became_peered":"2023-08-25T09:46:57.806512+0000","last_unstale":"2023-08-25T09:46:59.388487+0000","last_undegraded":"2023-08-25T09:46:59.388487+0000","last_fullsized":"2023-08-25T09:46:59.388487+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:42:32.794430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402850+0000","last_change":"2023-08-25T09:46:58.400694+0000","last_active":"2023-08-25T09:46:58.402850+0000","last_peered":"2023-08-25T09:46:58.402850+0000","last_clean":"2023-08-25T09:46:58.402850+0000","last_became_active":"2023-08-25T09:46:58.400039+0000","last_became_peered":"2023-08-25T09:46:58.400039+0000","last_unstale":"2023-08-25T09:46:58.402850+0000","last_undegraded":"2023-08-25T09:46:58.402850+0000","last_fullsized":"2023-08-25T09:46:58.402850+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:39.077929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.399851+0000","last_change":"2023-08-25T09:46:57.806778+0000","last_active":"2023-08-25T09:46:58.399851+0000","last_peered":"2023-08-25T09:46:58.399851+0000","last_clean":"2023-08-25T09:46:58.399851+0000","last_became_active":"2023-08-25T09:46:57.806536+0000","last_became_peered":"2023-08-25T09:46:57.806536+0000","last_unstale":"2023-08-25T09:46:58.399851+0000","last_undegraded":"2023-08-25T09:46:58.399851+0000","last_fullsized":"2023-08-25T09:46:58.399851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:50.604157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388682+0000","last_change":"2023-08-25T09:46:58.398350+0000","last_active":"2023-08-25T09:46:59.388682+0000","last_peered":"2023-08-25T09:46:59.388682+0000","last_clean":"2023-08-25T09:46:59.388682+0000","last_became_active":"2023-08-25T09:46:58.397574+0000","last_became_peered":"2023-08-25T09:46:58.397574+0000","last_unstale":"2023-08-25T09:46:59.388682+0000","last_undegraded":"2023-08-25T09:46:59.388682+0000","last_fullsized":"2023-08-25T09:46:59.388682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:02.773948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383262+0000","last_change":"2023-08-25T09:46:57.805782+0000","last_active":"2023-08-25T09:46:59.383262+0000","last_peered":"2023-08-25T09:46:59.383262+0000","last_clean":"2023-08-25T09:46:59.383262+0000","last_became_active":"2023-08-25T09:46:57.805464+0000","last_became_peered":"2023-08-25T09:46:57.805464+0000","last_unstale":"2023-08-25T09:46:59.383262+0000","last_undegraded":"2023-08-25T09:46:59.383262+0000","last_fullsized":"2023-08-25T09:46:59.383262+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:25.565647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386122+0000","last_change":"2023-08-25T09:46:58.403952+0000","last_active":"2023-08-25T09:46:59.386122+0000","last_peered":"2023-08-25T09:46:59.386122+0000","last_clean":"2023-08-25T09:46:59.386122+0000","last_became_active":"2023-08-25T09:46:58.403694+0000","last_became_peered":"2023-08-25T09:46:58.403694+0000","last_unstale":"2023-08-25T09:46:59.386122+0000","last_undegraded":"2023-08-25T09:46:59.386122+0000","last_fullsized":"2023-08-25T09:46:59.386122+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:21.808249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.395991+0000","last_change":"2023-08-25T09:46:57.807436+0000","last_active":"2023-08-25T09:46:58.395991+0000","last_peered":"2023-08-25T09:46:58.395991+0000","last_clean":"2023-08-25T09:46:58.395991+0000","last_became_active":"2023-08-25T09:46:57.807148+0000","last_became_peered":"2023-08-25T09:46:57.807148+0000","last_unstale":"2023-08-25T09:46:58.395991+0000","last_undegraded":"2023-08-25T09:46:58.395991+0000","last_fullsized":"2023-08-25T09:46:58.395991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:33.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.405360+0000","last_change":"2023-08-25T09:46:58.405270+0000","last_active":"2023-08-25T09:46:58.405360+0000","last_peered":"2023-08-25T09:46:58.405360+0000","last_clean":"2023-08-25T09:46:58.405360+0000","last_became_active":"2023-08-25T09:46:58.404402+0000","last_became_peered":"2023-08-25T09:46:58.404402+0000","last_unstale":"2023-08-25T09:46:58.405360+0000","last_undegraded":"2023-08-25T09:46:58.405360+0000","last_fullsized":"2023-08-25T09:46:58.405360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:08:53.762969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.398849+0000","last_change":"2023-08-25T09:46:57.807245+0000","last_active":"2023-08-25T09:46:58.398849+0000","last_peered":"2023-08-25T09:46:58.398849+0000","last_clean":"2023-08-25T09:46:58.398849+0000","last_became_active":"2023-08-25T09:46:57.805588+0000","last_became_peered":"2023-08-25T09:46:57.805588+0000","last_unstale":"2023-08-25T09:46:58.398849+0000","last_undegraded":"2023-08-25T09:46:58.398849+0000","last_fullsized":"2023-08-25T09:46:58.398849+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:46:23.475717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387822+0000","last_change":"2023-08-25T09:46:58.395055+0000","last_active":"2023-08-25T09:46:59.387822+0000","last_peered":"2023-08-25T09:46:59.387822+0000","last_clean":"2023-08-25T09:46:59.387822+0000","last_became_active":"2023-08-25T09:46:58.394608+0000","last_became_peered":"2023-08-25T09:46:58.394608+0000","last_unstale":"2023-08-25T09:46:59.387822+0000","last_undegraded":"2023-08-25T09:46:59.387822+0000","last_fullsized":"2023-08-25T09:46:59.387822+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:52.386906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.389319+0000","last_change":"2023-08-25T09:46:57.804522+0000","last_active":"2023-08-25T09:46:59.389319+0000","last_peered":"2023-08-25T09:46:59.389319+0000","last_clean":"2023-08-25T09:46:59.389319+0000","last_became_active":"2023-08-25T09:46:57.803573+0000","last_became_peered":"2023-08-25T09:46:57.803573+0000","last_unstale":"2023-08-25T09:46:59.389319+0000","last_undegraded":"2023-08-25T09:46:59.389319+0000","last_fullsized":"2023-08-25T09:46:59.389319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:51.381181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386099+0000","last_change":"2023-08-25T09:46:58.407934+0000","last_active":"2023-08-25T09:46:59.386099+0000","last_peered":"2023-08-25T09:46:59.386099+0000","last_clean":"2023-08-25T09:46:59.386099+0000","last_became_active":"2023-08-25T09:46:58.406811+0000","last_became_peered":"2023-08-25T09:46:58.406811+0000","last_unstale":"2023-08-25T09:46:59.386099+0000","last_undegraded":"2023-08-25T09:46:59.386099+0000","last_fullsized":"2023-08-25T09:46:59.386099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:22.513670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.400162+0000","last_change":"2023-08-25T09:46:57.802100+0000","last_active":"2023-08-25T09:46:58.400162+0000","last_peered":"2023-08-25T09:46:58.400162+0000","last_clean":"2023-08-25T09:46:58.400162+0000","last_became_active":"2023-08-25T09:46:57.801521+0000","last_became_peered":"2023-08-25T09:46:57.801521+0000","last_unstale":"2023-08-25T09:46:58.400162+0000","last_undegraded":"2023-08-25T09:46:58.400162+0000","last_fullsized":"2023-08-25T09:46:58.400162+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:55:14.134737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383008+0000","last_change":"2023-08-25T09:46:58.402356+0000","last_active":"2023-08-25T09:46:59.383008+0000","last_peered":"2023-08-25T09:46:59.383008+0000","last_clean":"2023-08-25T09:46:59.383008+0000","last_became_active":"2023-08-25T09:46:58.401869+0000","last_became_peered":"2023-08-25T09:46:58.401869+0000","last_unstale":"2023-08-25T09:46:59.383008+0000","last_undegraded":"2023-08-25T09:46:59.383008+0000","last_fullsized":"2023-08-25T09:46:59.383008+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:01.602066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.403647+0000","last_change":"2023-08-25T09:46:57.806218+0000","last_active":"2023-08-25T09:46:58.403647+0000","last_peered":"2023-08-25T09:46:58.403647+0000","last_clean":"2023-08-25T09:46:58.403647+0000","last_became_active":"2023-08-25T09:46:57.805985+0000","last_became_peered":"2023-08-25T09:46:57.805985+0000","last_unstale":"2023-08-25T09:46:58.403647+0000","last_undegraded":"2023-08-25T09:46:58.403647+0000","last_fullsized":"2023-08-25T09:46:58.403647+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:45.868036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.405439+0000","last_change":"2023-08-25T09:46:58.405343+0000","last_active":"2023-08-25T09:46:58.405439+0000","last_peered":"2023-08-25T09:46:58.405439+0000","last_clean":"2023-08-25T09:46:58.405439+0000","last_became_active":"2023-08-25T09:46:58.404510+0000","last_became_peered":"2023-08-25T09:46:58.404510+0000","last_unstale":"2023-08-25T09:46:58.405439+0000","last_undegraded":"2023-08-25T09:46:58.405439+0000","last_fullsized":"2023-08-25T09:46:58.405439+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:34.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383346+0000","last_change":"2023-08-25T09:46:57.807376+0000","last_active":"2023-08-25T09:46:59.383346+0000","last_peered":"2023-08-25T09:46:59.383346+0000","last_clean":"2023-08-25T09:46:59.383346+0000","last_became_active":"2023-08-25T09:46:57.806906+0000","last_became_peered":"2023-08-25T09:46:57.806906+0000","last_unstale":"2023-08-25T09:46:59.383346+0000","last_undegraded":"2023-08-25T09:46:59.383346+0000","last_fullsized":"2023-08-25T09:46:59.383346+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:51:23.503362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.398466+0000","last_change":"2023-08-25T09:46:58.397974+0000","last_active":"2023-08-25T09:46:58.398466+0000","last_peered":"2023-08-25T09:46:58.398466+0000","last_clean":"2023-08-25T09:46:58.398466+0000","last_became_active":"2023-08-25T09:46:58.396730+0000","last_became_peered":"2023-08-25T09:46:58.396730+0000","last_unstale":"2023-08-25T09:46:58.398466+0000","last_undegraded":"2023-08-25T09:46:58.398466+0000","last_fullsized":"2023-08-25T09:46:58.398466+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:40:49.259136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.399236+0000","last_change":"2023-08-25T09:46:57.805862+0000","last_active":"2023-08-25T09:46:58.399236+0000","last_peered":"2023-08-25T09:46:58.399236+0000","last_clean":"2023-08-25T09:46:58.399236+0000","last_became_active":"2023-08-25T09:46:57.805106+0000","last_became_peered":"2023-08-25T09:46:57.805106+0000","last_unstale":"2023-08-25T09:46:58.399236+0000","last_undegraded":"2023-08-25T09:46:58.399236+0000","last_fullsized":"2023-08-25T09:46:58.399236+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:37:45.483372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402031+0000","last_change":"2023-08-25T09:46:58.401894+0000","last_active":"2023-08-25T09:46:58.402031+0000","last_peered":"2023-08-25T09:46:58.402031+0000","last_clean":"2023-08-25T09:46:58.402031+0000","last_became_active":"2023-08-25T09:46:58.401306+0000","last_became_peered":"2023-08-25T09:46:58.401306+0000","last_unstale":"2023-08-25T09:46:58.402031+0000","last_undegraded":"2023-08-25T09:46:58.402031+0000","last_fullsized":"2023-08-25T09:46:58.402031+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:59.406464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"149'8","reported_seq":19,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.840256+0000","last_change":"2023-08-25T09:46:57.795642+0000","last_active":"2023-08-25T09:46:58.840256+0000","last_peered":"2023-08-25T09:46:58.840256+0000","last_clean":"2023-08-25T09:46:58.840256+0000","last_became_active":"2023-08-25T09:46:57.794931+0000","last_became_peered":"2023-08-25T09:46:57.794931+0000","last_unstale":"2023-08-25T09:46:58.840256+0000","last_undegraded":"2023-08-25T09:46:58.840256+0000","last_fullsized":"2023-08-25T09:46:58.840256+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:23.955445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":393,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.396950+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:46:58.396950+0000","last_peered":"2023-08-25T09:46:58.396950+0000","last_clean":"2023-08-25T09:46:58.396950+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:46:58.396950+0000","last_undegraded":"2023-08-25T09:46:58.396950+0000","last_fullsized":"2023-08-25T09:46:58.396950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:12:22.955616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391476+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:46:58.391476+0000","last_peered":"2023-08-25T09:46:58.391476+0000","last_clean":"2023-08-25T09:46:58.391476+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:46:58.391476+0000","last_undegraded":"2023-08-25T09:46:58.391476+0000","last_fullsized":"2023-08-25T09:46:58.391476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:21.353922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.401170+0000","last_change":"2023-08-25T09:46:58.401037+0000","last_active":"2023-08-25T09:46:58.401170+0000","last_peered":"2023-08-25T09:46:58.401170+0000","last_clean":"2023-08-25T09:46:58.401170+0000","last_became_active":"2023-08-25T09:46:58.400553+0000","last_became_peered":"2023-08-25T09:46:58.400553+0000","last_unstale":"2023-08-25T09:46:58.401170+0000","last_undegraded":"2023-08-25T09:46:58.401170+0000","last_fullsized":"2023-08-25T09:46:58.401170+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:59.178154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"149'2","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.838119+0000","last_change":"2023-08-25T09:46:57.805528+0000","last_active":"2023-08-25T09:46:58.838119+0000","last_peered":"2023-08-25T09:46:58.838119+0000","last_clean":"2023-08-25T09:46:58.838119+0000","last_became_active":"2023-08-25T09:46:57.805190+0000","last_became_peered":"2023-08-25T09:46:57.805190+0000","last_unstale":"2023-08-25T09:46:58.838119+0000","last_undegraded":"2023-08-25T09:46:58.838119+0000","last_fullsized":"2023-08-25T09:46:58.838119+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:34.056911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":263,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.389600+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:46:58.389600+0000","last_peered":"2023-08-25T09:46:58.389600+0000","last_clean":"2023-08-25T09:46:58.389600+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:46:58.389600+0000","last_undegraded":"2023-08-25T09:46:58.389600+0000","last_fullsized":"2023-08-25T09:46:58.389600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:17.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382915+0000","last_change":"2023-08-25T09:46:58.407371+0000","last_active":"2023-08-25T09:46:59.382915+0000","last_peered":"2023-08-25T09:46:59.382915+0000","last_clean":"2023-08-25T09:46:59.382915+0000","last_became_active":"2023-08-25T09:46:58.406298+0000","last_became_peered":"2023-08-25T09:46:58.406298+0000","last_unstale":"2023-08-25T09:46:59.382915+0000","last_undegraded":"2023-08-25T09:46:59.382915+0000","last_fullsized":"2023-08-25T09:46:59.382915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:47.053310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386634+0000","last_change":"2023-08-25T09:46:57.808138+0000","last_active":"2023-08-25T09:46:59.386634+0000","last_peered":"2023-08-25T09:46:59.386634+0000","last_clean":"2023-08-25T09:46:59.386634+0000","last_became_active":"2023-08-25T09:46:57.807894+0000","last_became_peered":"2023-08-25T09:46:57.807894+0000","last_unstale":"2023-08-25T09:46:59.386634+0000","last_undegraded":"2023-08-25T09:46:59.386634+0000","last_fullsized":"2023-08-25T09:46:59.386634+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:19:34.069129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":266,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387293+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:46:59.387293+0000","last_peered":"2023-08-25T09:46:59.387293+0000","last_clean":"2023-08-25T09:46:59.387293+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:46:59.387293+0000","last_undegraded":"2023-08-25T09:46:59.387293+0000","last_fullsized":"2023-08-25T09:46:59.387293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:04:50.990790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387806+0000","last_change":"2023-08-25T09:46:58.405793+0000","last_active":"2023-08-25T09:46:59.387806+0000","last_peered":"2023-08-25T09:46:59.387806+0000","last_clean":"2023-08-25T09:46:59.387806+0000","last_became_active":"2023-08-25T09:46:58.405501+0000","last_became_peered":"2023-08-25T09:46:58.405501+0000","last_unstale":"2023-08-25T09:46:59.387806+0000","last_undegraded":"2023-08-25T09:46:59.387806+0000","last_fullsized":"2023-08-25T09:46:59.387806+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:14.268254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.390922+0000","last_change":"2023-08-25T09:46:57.790379+0000","last_active":"2023-08-25T09:46:58.390922+0000","last_peered":"2023-08-25T09:46:58.390922+0000","last_clean":"2023-08-25T09:46:58.390922+0000","last_became_active":"2023-08-25T09:46:57.789516+0000","last_became_peered":"2023-08-25T09:46:57.789516+0000","last_unstale":"2023-08-25T09:46:58.390922+0000","last_undegraded":"2023-08-25T09:46:58.390922+0000","last_fullsized":"2023-08-25T09:46:58.390922+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:00.448961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.388529+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:46:58.388529+0000","last_peered":"2023-08-25T09:46:58.388529+0000","last_clean":"2023-08-25T09:46:58.388529+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:46:58.388529+0000","last_undegraded":"2023-08-25T09:46:58.388529+0000","last_fullsized":"2023-08-25T09:46:58.388529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:55.270769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.406599+0000","last_change":"2023-08-25T09:46:58.406492+0000","last_active":"2023-08-25T09:46:58.406599+0000","last_peered":"2023-08-25T09:46:58.406599+0000","last_clean":"2023-08-25T09:46:58.406599+0000","last_became_active":"2023-08-25T09:46:58.405642+0000","last_became_peered":"2023-08-25T09:46:58.405642+0000","last_unstale":"2023-08-25T09:46:58.406599+0000","last_undegraded":"2023-08-25T09:46:58.406599+0000","last_fullsized":"2023-08-25T09:46:58.406599+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:10.349935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.381275+0000","last_change":"2023-08-25T09:46:57.794364+0000","last_active":"2023-08-25T09:46:58.381275+0000","last_peered":"2023-08-25T09:46:58.381275+0000","last_clean":"2023-08-25T09:46:58.381275+0000","last_became_active":"2023-08-25T09:46:57.793965+0000","last_became_peered":"2023-08-25T09:46:57.793965+0000","last_unstale":"2023-08-25T09:46:58.381275+0000","last_undegraded":"2023-08-25T09:46:58.381275+0000","last_fullsized":"2023-08-25T09:46:58.381275+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:50:56.754615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":240,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383920+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:46:59.383920+0000","last_peered":"2023-08-25T09:46:59.383920+0000","last_clean":"2023-08-25T09:46:59.383920+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:46:59.383920+0000","last_undegraded":"2023-08-25T09:46:59.383920+0000","last_fullsized":"2023-08-25T09:46:59.383920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:09:18.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391050+0000","last_change":"2023-08-25T09:46:58.390802+0000","last_active":"2023-08-25T09:46:58.391050+0000","last_peered":"2023-08-25T09:46:58.391050+0000","last_clean":"2023-08-25T09:46:58.391050+0000","last_became_active":"2023-08-25T09:46:58.389873+0000","last_became_peered":"2023-08-25T09:46:58.389873+0000","last_unstale":"2023-08-25T09:46:58.391050+0000","last_undegraded":"2023-08-25T09:46:58.391050+0000","last_fullsized":"2023-08-25T09:46:58.391050+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:48.165528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.382098+0000","last_change":"2023-08-25T09:46:57.805400+0000","last_active":"2023-08-25T09:46:58.382098+0000","last_peered":"2023-08-25T09:46:58.382098+0000","last_clean":"2023-08-25T09:46:58.382098+0000","last_became_active":"2023-08-25T09:46:57.805171+0000","last_became_peered":"2023-08-25T09:46:57.805171+0000","last_unstale":"2023-08-25T09:46:58.382098+0000","last_undegraded":"2023-08-25T09:46:58.382098+0000","last_fullsized":"2023-08-25T09:46:58.382098+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:04:20.143242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.389302+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:46:58.389302+0000","last_peered":"2023-08-25T09:46:58.389302+0000","last_clean":"2023-08-25T09:46:58.389302+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:46:58.389302+0000","last_undegraded":"2023-08-25T09:46:58.389302+0000","last_fullsized":"2023-08-25T09:46:58.389302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:03.918320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387511+0000","last_change":"2023-08-25T09:46:58.410284+0000","last_active":"2023-08-25T09:46:59.387511+0000","last_peered":"2023-08-25T09:46:59.387511+0000","last_clean":"2023-08-25T09:46:59.387511+0000","last_became_active":"2023-08-25T09:46:58.410046+0000","last_became_peered":"2023-08-25T09:46:58.410046+0000","last_unstale":"2023-08-25T09:46:59.387511+0000","last_undegraded":"2023-08-25T09:46:59.387511+0000","last_fullsized":"2023-08-25T09:46:59.387511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:48:37.777258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"149'3","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.838595+0000","last_change":"2023-08-25T09:46:57.802862+0000","last_active":"2023-08-25T09:46:58.838595+0000","last_peered":"2023-08-25T09:46:58.838595+0000","last_clean":"2023-08-25T09:46:58.838595+0000","last_became_active":"2023-08-25T09:46:57.802548+0000","last_became_peered":"2023-08-25T09:46:57.802548+0000","last_unstale":"2023-08-25T09:46:58.838595+0000","last_undegraded":"2023-08-25T09:46:58.838595+0000","last_fullsized":"2023-08-25T09:46:58.838595+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:16:32.564166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":234,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.389315+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:46:58.389315+0000","last_peered":"2023-08-25T09:46:58.389315+0000","last_clean":"2023-08-25T09:46:58.389315+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:46:58.389315+0000","last_undegraded":"2023-08-25T09:46:58.389315+0000","last_fullsized":"2023-08-25T09:46:58.389315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:20:53.122003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.407039+0000","last_change":"2023-08-25T09:46:58.406965+0000","last_active":"2023-08-25T09:46:58.407039+0000","last_peered":"2023-08-25T09:46:58.407039+0000","last_clean":"2023-08-25T09:46:58.407039+0000","last_became_active":"2023-08-25T09:46:58.406620+0000","last_became_peered":"2023-08-25T09:46:58.406620+0000","last_unstale":"2023-08-25T09:46:58.407039+0000","last_undegraded":"2023-08-25T09:46:58.407039+0000","last_fullsized":"2023-08-25T09:46:58.407039+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:19:06.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"149'6","reported_seq":19,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382308+0000","last_change":"2023-08-25T09:46:57.803942+0000","last_active":"2023-08-25T09:46:59.382308+0000","last_peered":"2023-08-25T09:46:59.382308+0000","last_clean":"2023-08-25T09:46:59.382308+0000","last_became_active":"2023-08-25T09:46:57.803577+0000","last_became_peered":"2023-08-25T09:46:57.803577+0000","last_unstale":"2023-08-25T09:46:59.382308+0000","last_undegraded":"2023-08-25T09:46:59.382308+0000","last_fullsized":"2023-08-25T09:46:59.382308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:12.075905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.388228+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:46:58.388228+0000","last_peered":"2023-08-25T09:46:58.388228+0000","last_clean":"2023-08-25T09:46:58.388228+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:46:58.388228+0000","last_undegraded":"2023-08-25T09:46:58.388228+0000","last_fullsized":"2023-08-25T09:46:58.388228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:06.274850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115546,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.738,"5min":1.7909999999999999,"15min":10.747},"last":0.872},{"interface":"front","average":{"1min":1.077,"5min":1.129,"15min":1.0509999999999999},"min":{"1min":0.61699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6499999999999999,"5min":1.855,"15min":1.855},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.298,"15min":1.177},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.665,"5min":11.010999999999999,"15min":11.010999999999999},"last":1.278},{"interface":"front","average":{"1min":1.179,"5min":1.1619999999999999,"15min":1.1120000000000001},"min":{"1min":0.70899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6559999999999999,"5min":1.667,"15min":1.7769999999999999},"last":1.347}]},{"osd":2,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1459999999999999,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4550000000000001,"5min":1.619,"15min":1.619},"last":1.077},{"interface":"front","average":{"1min":1.22,"5min":1.294,"15min":1.234},"min":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.718,"5min":1.98,"15min":1.98},"last":1.419}]},{"osd":3,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.232,"15min":1.177},"min":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5580000000000001,"5min":1.6699999999999999,"15min":1.7110000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3660000000000001,"15min":1.3360000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.651,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.141}]},{"osd":4,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2889999999999999,"15min":1.2829999999999999},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.462},{"interface":"front","average":{"1min":1.335,"5min":1.3620000000000001,"15min":1.3080000000000001},"min":{"1min":0.53800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.651,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.181}]},{"osd":5,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4350000000000001,"15min":1.363},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":1.8280000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.544},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.452,"15min":1.397},"min":{"1min":0.67700000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.806}]},{"osd":6,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5529999999999999,"15min":1.464},"min":{"1min":0.72999999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0259999999999998,"5min":2.032,"15min":2.1720000000000002},"last":1.597},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4930000000000001,"15min":1.4199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":1.8029999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.7470000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115524,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1519999999999999,"15min":1.1439999999999999},"min":{"1min":0.69799999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.796,"5min":1.804,"15min":1.804},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1539999999999999,"15min":1.1499999999999999},"min":{"1min":0.753,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.992,"5min":1.992,"15min":5.5410000000000004},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77000000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.198,"5min":2.198,"15min":5.6799999999999997},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.22,"15min":1.143},"min":{"1min":0.69199999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.1240000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.157,"15min":1.117},"min":{"1min":0.78400000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.708},"last":1.052},{"interface":"front","average":{"1min":1.147,"5min":1.2829999999999999,"15min":1.2350000000000001},"min":{"1min":0.79900000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5760000000000001,"5min":1.768,"15min":13.193},"last":1.3240000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3,"15min":1.2509999999999999},"min":{"1min":0.94499999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.2609999999999999,"15min":1.2569999999999999},"min":{"1min":0.85299999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7430000000000001,"5min":1.76,"15min":1.9239999999999999},"last":0.93899999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3460000000000001,"15min":1.278},"min":{"1min":0.68400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.484},"last":1.464},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.369,"15min":1.341},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.671}]},{"osd":5,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3300000000000001,"15min":1.2909999999999999},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":1.859,"15min":2.0049999999999999},"last":1.425},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4970000000000001,"15min":1.4039999999999999},"min":{"1min":0.73099999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.4420000000000002,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.5509999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5249999999999999,"15min":1.51},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.49,"15min":1.4830000000000001},"min":{"1min":0.81699999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.1890000000000001},"last":1.278}]}]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193872,"statfs":{"total":200000000,"available":198525178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.196,"15min":1.157},"min":{"1min":0.64000000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.7809999999999999,"5min":2.081,"15min":2.081},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.2210000000000001,"15min":1.204},"min":{"1min":0.77600000000000002,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.7629999999999999,"5min":1.8839999999999999,"15min":1.974},"last":1.1479999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.145,"15min":1.1830000000000001},"min":{"1min":0.73499999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6539999999999999,"5min":1.851,"15min":2.1779999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.1870000000000001,"15min":1.133},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.696,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.204}]},{"osd":2,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2450000000000001,"15min":1.1990000000000001},"min":{"1min":0.80400000000000005,"5min":0.73899999999999999,"15min":0.73599999999999999},"max":{"1min":1.6459999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.27,"15min":1.228},"min":{"1min":0.82899999999999996,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.724,"5min":1.778,"15min":1.962},"last":1.2929999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2030000000000001,"15min":1.292},"min":{"1min":0.65900000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.704,"5min":2.3439999999999999,"15min":17.356999999999999},"last":1.323},{"interface":"front","average":{"1min":1.369,"5min":1.343,"15min":1.2969999999999999},"min":{"1min":0.91600000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9650000000000001},"last":1.5629999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3220000000000001,"15min":1.3049999999999999},"min":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.921,"5min":2.2799999999999998,"15min":2.625},"last":1.385},{"interface":"front","average":{"1min":1.337,"5min":1.351,"15min":1.359},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.2290000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":2.048,"5min":2.048,"15min":2.4820000000000002},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.492,"15min":1.4350000000000001},"min":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0379999999999998,"5min":2.4700000000000002,"15min":2.5539999999999998},"last":1.2729999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.474,"15min":1.4419999999999999},"min":{"1min":0.85399999999999998,"5min":0.63,"15min":0.63},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.5860000000000001,"15min":1.5760000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115490,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.24,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.657,"5min":1.657,"15min":1.8160000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2709999999999999,"15min":1.1879999999999999},"min":{"1min":0.878,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":1.5629999999999999,"5min":1.976,"15min":7.367},"last":1.325}]},{"osd":1,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.173,"15min":1.1299999999999999},"min":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"max":{"1min":1.6240000000000001,"5min":2.032,"15min":2.032},"last":1.133},{"interface":"front","average":{"1min":1.22,"5min":1.2350000000000001,"15min":1.3560000000000001},"min":{"1min":0.77100000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.5940000000000001,"5min":1.954,"15min":8.5850000000000009},"last":1.0980000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.2010000000000001,"15min":1.268},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.76800000000000002},"max":{"1min":1.708,"5min":1.708,"15min":4.1669999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3100000000000001,"15min":1.2},"min":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5589999999999999,"5min":1.6799999999999999,"15min":1.8520000000000001},"last":1.2}]},{"osd":3,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2150000000000001,"15min":1.2350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.673,"5min":1.764,"15min":7.2560000000000002},"last":1.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.377},"min":{"1min":1.0329999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.911},"last":1.2290000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.2969999999999999,"15min":1.2949999999999999},"min":{"1min":0.64800000000000002,"5min":0.56000000000000005,"15min":0.55200000000000005},"max":{"1min":1.7230000000000001,"5min":2.1850000000000001,"15min":2.4420000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.302,"15min":1.2709999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.802,"5min":1.9299999999999999,"15min":8.2409999999999997},"last":1.516}]},{"osd":6,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.4730000000000001,"15min":1.4399999999999999},"min":{"1min":0.84999999999999998,"5min":0.68400000000000005,"15min":0.52000000000000002},"max":{"1min":2.02,"5min":3.5880000000000001,"15min":3.5880000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5980000000000001,"15min":1.569},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.9299999999999999,"5min":3.5270000000000001,"15min":4.0229999999999997},"last":1.444}]},{"osd":7,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.508,"15min":1.4419999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.63800000000000001},"max":{"1min":1.792,"5min":2.3570000000000002,"15min":8.1760000000000002},"last":1.409},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6419999999999999,"15min":1.601},"min":{"1min":0.96799999999999997,"5min":0.80900000000000005,"15min":0.77500000000000002},"max":{"1min":2.1150000000000002,"5min":2.1819999999999999,"15min":2.7930000000000001},"last":1.375}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115456,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.91600000000000004,"15min":0.85799999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.29299999999999998},"max":{"1min":1.579,"5min":1.6399999999999999,"15min":1.6830000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0209999999999999,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.503,"15min":0.41699999999999998},"max":{"1min":1.486,"5min":1.631,"15min":1.631},"last":1.3300000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.88300000000000001,"15min":0.81699999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.35699999999999998},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.004,"15min":0.97599999999999998},"min":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.45700000000000002},"max":{"1min":1.5649999999999999,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.0029999999999999,"15min":0.89500000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.378},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.919},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.145,"15min":1.1120000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.44900000000000001},"max":{"1min":1.8120000000000001,"5min":2.0870000000000002,"15min":3.399},"last":1.264}]},{"osd":4,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2370000000000001,"15min":1.1950000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.66800000000000004},"max":{"1min":1.538,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":1.431},{"interface":"front","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2010000000000001},"min":{"1min":0.84599999999999997,"5min":0.73299999999999998,"15min":0.65700000000000003},"max":{"1min":1.8600000000000001,"5min":1.984,"15min":2.1840000000000002},"last":1.385}]},{"osd":5,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.343,"15min":1.3149999999999999},"min":{"1min":0.91700000000000004,"5min":0.73599999999999999,"15min":0.71899999999999997},"max":{"1min":1.5429999999999999,"5min":2.407,"15min":2.407},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.3049999999999999,"15min":1.274},"min":{"1min":0.84899999999999998,"5min":0.83599999999999997,"15min":0.64500000000000002},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.556}]},{"osd":6,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.3080000000000001,"15min":1.2549999999999999},"min":{"1min":0.871,"5min":0.85199999999999998,"15min":0.73799999999999999},"max":{"1min":1.9279999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.488},{"interface":"front","average":{"1min":1.26,"5min":1.409,"15min":1.399},"min":{"1min":0.83199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7170000000000001,"5min":2.6699999999999999,"15min":3.496},"last":1.413}]},{"osd":7,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.403,"15min":1.3660000000000001},"min":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":2.2989999999999999},"last":1.881},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.413,"15min":1.319},"min":{"1min":0.84199999999999997,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":1.9159999999999999,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6850000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115546,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.83399999999999996},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.706,"5min":1.9279999999999999,"15min":2.383},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.96099999999999997,"15min":1.028},"min":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.645,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":1.1240000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.016,"15min":0.99099999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.978,"5min":1.978,"15min":2.3180000000000001},"last":1.048},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0389999999999999,"15min":1.048},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.6830000000000001,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.157,"15min":1.117},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6839999999999999,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.1040000000000001,"15min":1.0980000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.8999999999999999,"5min":2.145,"15min":2.2040000000000002},"last":1.2829999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.302,"15min":1.365},"min":{"1min":0.74199999999999999,"5min":0.73899999999999999,"15min":0.72099999999999997},"max":{"1min":2.0059999999999998,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2929999999999999,"15min":1.319},"min":{"1min":0.75700000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5609999999999999,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.26,"15min":1.3420000000000001},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.7559999999999998,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":1.101},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.3480000000000001,"15min":1.381},"min":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0920000000000001,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4159999999999999,"15min":1.417},"min":{"1min":0.88300000000000001,"5min":0.755,"15min":0.73199999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.389,"15min":1.431},"min":{"1min":0.85499999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.3149999999999999,"5min":2.331,"15min":2.968},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.76500000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.96,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.4930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193872,"statfs":{"total":200000000,"available":198525178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.86299999999999999,"15min":0.79000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.29699999999999999},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.627},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.81699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2490000000000001,"5min":1.895,"15min":1.895},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.90800000000000003,"15min":0.82299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.397,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98599999999999999,"15min":0.92300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.33600000000000002},"max":{"1min":1.2669999999999999,"5min":1.6220000000000001,"15min":1.746},"last":1.508}]},{"osd":3,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.1339999999999999,"15min":1.056},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.372},"max":{"1min":1.4950000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.103},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.042,"15min":0.96599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5389999999999999,"5min":1.6439999999999999,"15min":1.764},"last":1.0680000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1040000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.238},{"interface":"front","average":{"1min":1.169,"5min":1.2989999999999999,"15min":1.2350000000000001},"min":{"1min":0.81299999999999994,"5min":0.77700000000000002,"15min":0.629},"max":{"1min":1.5700000000000001,"5min":1.8160000000000001,"15min":2.008},"last":1.379}]},{"osd":5,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.238,"15min":1.2030000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.68600000000000005},"max":{"1min":1.7330000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.161},{"interface":"front","average":{"1min":1.177,"5min":1.2609999999999999,"15min":1.1919999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.61199999999999999},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.014}]},{"osd":6,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.415,"15min":1.3240000000000001},"min":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.462},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.403,"15min":1.3160000000000001},"min":{"1min":0.88,"5min":0.82199999999999995,"15min":0.72099999999999997},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.409}]},{"osd":7,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.417,"15min":1.3700000000000001},"min":{"1min":0.96699999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8100000000000001,"5min":1.968,"15min":2.9529999999999998},"last":1.214},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4410000000000001,"15min":1.345},"min":{"1min":0.92200000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.0659999999999998,"5min":2.218,"15min":2.218},"last":1.331}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":864,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199114860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":0.86399999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.313},"max":{"1min":1.3939999999999999,"5min":11.257,"15min":11.257},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0149999999999999,"15min":0.96199999999999997},"min":{"1min":0.58099999999999996,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":1.474,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.246}]},{"osd":2,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.96899999999999997,"15min":0.94199999999999995},"min":{"1min":0.59999999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.724,"15min":1.724},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.016,"15min":1.0800000000000001},"min":{"1min":0.498,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4259999999999999,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.86499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92100000000000004,"15min":0.90500000000000003},"min":{"1min":0.437,"5min":0.42699999999999999,"15min":0.40699999999999997},"max":{"1min":1.625,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.121},"min":{"1min":0.55500000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6339999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.458}]},{"osd":4,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.272,"15min":1.24},"min":{"1min":0.82799999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.78,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.3759999999999999,"15min":1.357},"min":{"1min":0.95299999999999996,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.589}]},{"osd":5,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.2909999999999999,"15min":1.347},"min":{"1min":0.82499999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.9530000000000001,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.335},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.294},"min":{"1min":0.89900000000000002,"5min":0.73699999999999999,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.1950000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.347,"15min":1.3440000000000001},"min":{"1min":0.86399999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"max":{"1min":1.821,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.343,"15min":1.3080000000000001},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.72699999999999998},"max":{"1min":2.2090000000000001,"5min":2.331,"15min":2.331},"last":1.429}]},{"osd":7,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4139999999999999,"15min":1.4019999999999999},"min":{"1min":0.94999999999999996,"5min":0.79000000000000004,"15min":0.71099999999999997},"max":{"1min":2.085,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.482},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.337,"15min":1.323},"min":{"1min":0.86899999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.1469999999999998,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":1.2969999999999999}]}]}],"pool_statfs":[]}} 2023-08-25T09:47:03.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:47:03.388 INFO:tasks.ceph.mds.d.smithi155.stderr:2023-08-25T09:47:03.386+0000 7f0c92f26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:47:03.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.465+0000 7fcef6396640 1 Processor -- start 2023-08-25T09:47:03.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.465+0000 7fcef6396640 1 -- start start 2023-08-25T09:47:03.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.467+0000 7fcef6396640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcef00ac870 0x7fcef00b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.467+0000 7fcef6396640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef006f050 con 0x7fcef00cea30 2023-08-25T09:47:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.467+0000 7fcef6396640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef006f330 con 0x7fcef015cb40 2023-08-25T09:47:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.467+0000 7fcef6396640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef006f610 con 0x7fcef00ac870 2023-08-25T09:47:03.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fcee7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcef00cea30 0x7fcef00b1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56096/0 (socket says 172.21.15.70:56096) 2023-08-25T09:47:03.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fcee7fff640 1 -- 172.21.15.70:0/2141823305 learned_addr learned my addr 172.21.15.70:0/2141823305 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:03.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fcef4933640 1 --2- 172.21.15.70:0/2141823305 >> v2:172.21.15.155:3300/0 conn(0x7fcef00ac870 0x7fcef00b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452920992 0 0) 0x7fcef006f050 con 0x7fcef00cea30 2023-08-25T09:47:03.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcedc003520 con 0x7fcef00cea30 2023-08-25T09:47:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3077388241 0 0) 0x7fcedc003520 con 0x7fcef00cea30 2023-08-25T09:47:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 >> v2:172.21.15.155:3300/0 conn(0x7fcef00ac870 msgr2=0x7fcef00b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 --2- 172.21.15.70:0/2141823305 >> v2:172.21.15.155:3300/0 conn(0x7fcef00ac870 0x7fcef00b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 >> v1:172.21.15.155:6789/0 conn(0x7fcef015cb40 legacy=0x7fcef00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef006f8f0 con 0x7fcef00cea30 2023-08-25T09:47:03.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fced8001670 con 0x7fcef00cea30 2023-08-25T09:47:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 >> v1:172.21.15.70:6789/0 conn(0x7fcef00cea30 legacy=0x7fcef00b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc003970 0x7fcedc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0047c0 0x7fcedc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.468+0000 7fceef7fe640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc005180 0x7fcedc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef006f610 con 0x7fcedc005180 2023-08-25T09:47:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc00c020 con 0x7fcedc003970 2023-08-25T09:47:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc00c1e0 con 0x7fcedc0047c0 2023-08-25T09:47:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fcef4933640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc005180 0x7fcedc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fcef4933640 1 -- 172.21.15.70:0/2141823305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0047c0 msgr2=0x7fcedc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fcef4933640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc0047c0 0x7fcedc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fcef4933640 1 -- 172.21.15.70:0/2141823305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc003970 msgr2=0x7fcedc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fcef4933640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcedc003970 0x7fcedc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.469+0000 7fcef4933640 1 -- 172.21.15.70:0/2141823305 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc003520 con 0x7fcedc005180 2023-08-25T09:47:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.470+0000 7fcef4933640 1 -- 172.21.15.70:0/2141823305 reap_dead start 2023-08-25T09:47:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.470+0000 7fcef4933640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc005180 0x7fcedc00b980 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fc0 tx=0x7fcedc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cba098b20a9945b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.470+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee802cc60 con 0x7fcedc005180 2023-08-25T09:47:03.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee800f040 con 0x7fcedc005180 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fceef7fe640 1 -- 172.21.15.70:0/2141823305 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8003990 con 0x7fcedc005180 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 -- 172.21.15.70:0/2141823305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc005180 msgr2=0x7fcedc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc005180 0x7fcedc00b980 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009fc0 tx=0x7fcedc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 -- 172.21.15.70:0/2141823305 shutdown_connections 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 --2- 172.21.15.70:0/2141823305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcedc005180 0x7fcedc00b980 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 -- 172.21.15.70:0/2141823305 >> 172.21.15.70:0/2141823305 conn(0x7fcef0069850 msgr2=0x7fcef0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:03.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 -- 172.21.15.70:0/2141823305 shutdown_connections 2023-08-25T09:47:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.471+0000 7fcef6396640 1 -- 172.21.15.70:0/2141823305 wait complete. 2023-08-25T09:47:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.473+0000 7fcef6396640 1 Processor -- start 2023-08-25T09:47:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.473+0000 7fcef6396640 1 -- start start 2023-08-25T09:47:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.474+0000 7fcef6396640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc005180 0x7fcef013f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.474+0000 7fcef6396640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef0153bc0 0x7fcef0154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.474+0000 7fcef6396640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 0x7fcef0141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.474+0000 7fcef6396640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef013fc00 con 0x7fcef0140ef0 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.474+0000 7fcef6396640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0140a60 con 0x7fcef0153bc0 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.474+0000 7fcef6396640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef0154560 con 0x7fcedc005180 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 0x7fcef0141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 0x7fcef0141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60370/0 (socket says 172.21.15.70:60370) 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 -- 172.21.15.70:0/365327067 learned_addr learned my addr 172.21.15.70:0/365327067 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fceeffff640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc005180 0x7fcef013f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 -- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc005180 msgr2=0x7fcef013f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcee7fff640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef0153bc0 0x7fcef0154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc005180 0x7fcef013f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 -- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef0153bc0 msgr2=0x7fcef0154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef0153bc0 0x7fcef0154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 -- 172.21.15.70:0/365327067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef013f6d0 con 0x7fcef0140ef0 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fceeffff640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc005180 0x7fcef013f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcee7fff640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef0153bc0 0x7fcef0154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.475+0000 7fcef4933640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 0x7fcef0141370 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcee802c6d0 tx=0x7fcee8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.476+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8035e40 con 0x7fcef0140ef0 2023-08-25T09:47:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.476+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee8003990 con 0x7fcef0140ef0 2023-08-25T09:47:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.476+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef013fe50 con 0x7fcef0140ef0 2023-08-25T09:47:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.476+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8003d80 con 0x7fcef0140ef0 2023-08-25T09:47:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.476+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef0140410 con 0x7fcef0140ef0 2023-08-25T09:47:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.477+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef006fbb0 con 0x7fcef0140ef0 2023-08-25T09:47:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.478+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcee8004030 con 0x7fcef0140ef0 2023-08-25T09:47:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.479+0000 7fceed7fa640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fced006a050 0x7fced006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.479+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcee80ab380 con 0x7fcef0140ef0 2023-08-25T09:47:03.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.479+0000 7fceeffff640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fced006a050 0x7fced006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.480+0000 7fceeffff640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fced006a050 0x7fced006c510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcee0001660 tx=0x7fcee0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:03.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.484+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcef006fbb0 con 0x7fcef0140ef0 2023-08-25T09:47:03.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.686+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcef006ece0 con 0x7fced006a050 2023-08-25T09:47:03.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.698+0000 7fceed7fa640 1 -- 172.21.15.70:0/365327067 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303420 (secure 0 0 0) 0x7fcee0001c40 con 0x7fced006a050 2023-08-25T09:47:03.702 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:03.707 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:47:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fced006a050 msgr2=0x7fced006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fced006a050 0x7fced006c510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcee0001660 tx=0x7fcee0009040 comp rx=0 tx=0).stop 2023-08-25T09:47:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 msgr2=0x7fcef0141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 0x7fcef0141370 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fcee802c6d0 tx=0x7fcee8009750 comp rx=0 tx=0).stop 2023-08-25T09:47:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 shutdown_connections 2023-08-25T09:47:03.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fced006a050 0x7fced006c510 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcef0140ef0 0x7fcef0141370 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcef0153bc0 0x7fcef0154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 --2- 172.21.15.70:0/365327067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcedc005180 0x7fcef013f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 >> 172.21.15.70:0/365327067 conn(0x7fcef0069850 msgr2=0x7fcef00adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 shutdown_connections 2023-08-25T09:47:03.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.706+0000 7fcef6396640 1 -- 172.21.15.70:0/365327067 wait complete. 2023-08-25T09:47:03.727 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2023-08-25T09:47:03.482439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8063,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554436,"statfs":{"total":1600000000,"available":1591742778,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001914"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.410114+0000","last_change":"2023-08-25T09:46:58.410027+0000","last_active":"2023-08-25T09:46:58.410114+0000","last_peered":"2023-08-25T09:46:58.410114+0000","last_clean":"2023-08-25T09:46:58.410114+0000","last_became_active":"2023-08-25T09:46:58.408039+0000","last_became_peered":"2023-08-25T09:46:58.408039+0000","last_unstale":"2023-08-25T09:46:58.410114+0000","last_undegraded":"2023-08-25T09:46:58.410114+0000","last_fullsized":"2023-08-25T09:46:58.410114+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:43.122225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382132+0000","last_change":"2023-08-25T09:46:58.408833+0000","last_active":"2023-08-25T09:46:59.382132+0000","last_peered":"2023-08-25T09:46:59.382132+0000","last_clean":"2023-08-25T09:46:59.382132+0000","last_became_active":"2023-08-25T09:46:58.408584+0000","last_became_peered":"2023-08-25T09:46:58.408584+0000","last_unstale":"2023-08-25T09:46:59.382132+0000","last_undegraded":"2023-08-25T09:46:59.382132+0000","last_fullsized":"2023-08-25T09:46:59.382132+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:37:41.988324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386709+0000","last_change":"2023-08-25T09:46:58.398252+0000","last_active":"2023-08-25T09:46:59.386709+0000","last_peered":"2023-08-25T09:46:59.386709+0000","last_clean":"2023-08-25T09:46:59.386709+0000","last_became_active":"2023-08-25T09:46:58.397685+0000","last_became_peered":"2023-08-25T09:46:58.397685+0000","last_unstale":"2023-08-25T09:46:59.386709+0000","last_undegraded":"2023-08-25T09:46:59.386709+0000","last_fullsized":"2023-08-25T09:46:59.386709+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:14.268254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391977+0000","last_change":"2023-08-25T09:46:58.391642+0000","last_active":"2023-08-25T09:46:58.391977+0000","last_peered":"2023-08-25T09:46:58.391977+0000","last_clean":"2023-08-25T09:46:58.391977+0000","last_became_active":"2023-08-25T09:46:58.389974+0000","last_became_peered":"2023-08-25T09:46:58.389974+0000","last_unstale":"2023-08-25T09:46:58.391977+0000","last_undegraded":"2023-08-25T09:46:58.391977+0000","last_fullsized":"2023-08-25T09:46:58.391977+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:25:49.387469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.404040+0000","last_change":"2023-08-25T09:46:58.403928+0000","last_active":"2023-08-25T09:46:58.404040+0000","last_peered":"2023-08-25T09:46:58.404040+0000","last_clean":"2023-08-25T09:46:58.404040+0000","last_became_active":"2023-08-25T09:46:58.403595+0000","last_became_peered":"2023-08-25T09:46:58.403595+0000","last_unstale":"2023-08-25T09:46:58.404040+0000","last_undegraded":"2023-08-25T09:46:58.404040+0000","last_fullsized":"2023-08-25T09:46:58.404040+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:16:39.656363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382915+0000","last_change":"2023-08-25T09:46:58.408452+0000","last_active":"2023-08-25T09:46:59.382915+0000","last_peered":"2023-08-25T09:46:59.382915+0000","last_clean":"2023-08-25T09:46:59.382915+0000","last_became_active":"2023-08-25T09:46:58.406452+0000","last_became_peered":"2023-08-25T09:46:58.406452+0000","last_unstale":"2023-08-25T09:46:59.382915+0000","last_undegraded":"2023-08-25T09:46:59.382915+0000","last_fullsized":"2023-08-25T09:46:59.382915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:37:00.602117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382463+0000","last_change":"2023-08-25T09:46:58.400258+0000","last_active":"2023-08-25T09:46:59.382463+0000","last_peered":"2023-08-25T09:46:59.382463+0000","last_clean":"2023-08-25T09:46:59.382463+0000","last_became_active":"2023-08-25T09:46:58.399966+0000","last_became_peered":"2023-08-25T09:46:58.399966+0000","last_unstale":"2023-08-25T09:46:59.382463+0000","last_undegraded":"2023-08-25T09:46:59.382463+0000","last_fullsized":"2023-08-25T09:46:59.382463+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:56:20.426610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.396513+0000","last_change":"2023-08-25T09:46:58.395499+0000","last_active":"2023-08-25T09:46:58.396513+0000","last_peered":"2023-08-25T09:46:58.396513+0000","last_clean":"2023-08-25T09:46:58.396513+0000","last_became_active":"2023-08-25T09:46:58.394182+0000","last_became_peered":"2023-08-25T09:46:58.394182+0000","last_unstale":"2023-08-25T09:46:58.396513+0000","last_undegraded":"2023-08-25T09:46:58.396513+0000","last_fullsized":"2023-08-25T09:46:58.396513+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:56.684770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.404849+0000","last_change":"2023-08-25T09:46:58.404708+0000","last_active":"2023-08-25T09:46:58.404849+0000","last_peered":"2023-08-25T09:46:58.404849+0000","last_clean":"2023-08-25T09:46:58.404849+0000","last_became_active":"2023-08-25T09:46:58.404278+0000","last_became_peered":"2023-08-25T09:46:58.404278+0000","last_unstale":"2023-08-25T09:46:58.404849+0000","last_undegraded":"2023-08-25T09:46:58.404849+0000","last_fullsized":"2023-08-25T09:46:58.404849+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:06:03.539567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382578+0000","last_change":"2023-08-25T09:46:58.403349+0000","last_active":"2023-08-25T09:46:59.382578+0000","last_peered":"2023-08-25T09:46:59.382578+0000","last_clean":"2023-08-25T09:46:59.382578+0000","last_became_active":"2023-08-25T09:46:58.402710+0000","last_became_peered":"2023-08-25T09:46:58.402710+0000","last_unstale":"2023-08-25T09:46:59.382578+0000","last_undegraded":"2023-08-25T09:46:59.382578+0000","last_fullsized":"2023-08-25T09:46:59.382578+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:15:30.773638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409894+0000","last_change":"2023-08-25T09:46:58.408735+0000","last_active":"2023-08-25T09:46:58.409894+0000","last_peered":"2023-08-25T09:46:58.409894+0000","last_clean":"2023-08-25T09:46:58.409894+0000","last_became_active":"2023-08-25T09:46:58.407589+0000","last_became_peered":"2023-08-25T09:46:58.407589+0000","last_unstale":"2023-08-25T09:46:58.409894+0000","last_undegraded":"2023-08-25T09:46:58.409894+0000","last_fullsized":"2023-08-25T09:46:58.409894+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:40:22.145671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385653+0000","last_change":"2023-08-25T09:46:58.404188+0000","last_active":"2023-08-25T09:46:59.385653+0000","last_peered":"2023-08-25T09:46:59.385653+0000","last_clean":"2023-08-25T09:46:59.385653+0000","last_became_active":"2023-08-25T09:46:58.403816+0000","last_became_peered":"2023-08-25T09:46:58.403816+0000","last_unstale":"2023-08-25T09:46:59.385653+0000","last_undegraded":"2023-08-25T09:46:59.385653+0000","last_fullsized":"2023-08-25T09:46:59.385653+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:42:14.053437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382608+0000","last_change":"2023-08-25T09:46:58.400494+0000","last_active":"2023-08-25T09:46:59.382608+0000","last_peered":"2023-08-25T09:46:59.382608+0000","last_clean":"2023-08-25T09:46:59.382608+0000","last_became_active":"2023-08-25T09:46:58.400143+0000","last_became_peered":"2023-08-25T09:46:58.400143+0000","last_unstale":"2023-08-25T09:46:59.382608+0000","last_undegraded":"2023-08-25T09:46:59.382608+0000","last_fullsized":"2023-08-25T09:46:59.382608+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:29.729115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385687+0000","last_change":"2023-08-25T09:46:58.408730+0000","last_active":"2023-08-25T09:46:59.385687+0000","last_peered":"2023-08-25T09:46:59.385687+0000","last_clean":"2023-08-25T09:46:59.385687+0000","last_became_active":"2023-08-25T09:46:58.408363+0000","last_became_peered":"2023-08-25T09:46:58.408363+0000","last_unstale":"2023-08-25T09:46:59.385687+0000","last_undegraded":"2023-08-25T09:46:59.385687+0000","last_fullsized":"2023-08-25T09:46:59.385687+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:10:08.157400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409398+0000","last_change":"2023-08-25T09:46:58.409240+0000","last_active":"2023-08-25T09:46:58.409398+0000","last_peered":"2023-08-25T09:46:58.409398+0000","last_clean":"2023-08-25T09:46:58.409398+0000","last_became_active":"2023-08-25T09:46:58.405588+0000","last_became_peered":"2023-08-25T09:46:58.405588+0000","last_unstale":"2023-08-25T09:46:58.409398+0000","last_undegraded":"2023-08-25T09:46:58.409398+0000","last_fullsized":"2023-08-25T09:46:58.409398+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:36:07.055865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387177+0000","last_change":"2023-08-25T09:46:58.400189+0000","last_active":"2023-08-25T09:46:59.387177+0000","last_peered":"2023-08-25T09:46:59.387177+0000","last_clean":"2023-08-25T09:46:59.387177+0000","last_became_active":"2023-08-25T09:46:58.399709+0000","last_became_peered":"2023-08-25T09:46:58.399709+0000","last_unstale":"2023-08-25T09:46:59.387177+0000","last_undegraded":"2023-08-25T09:46:59.387177+0000","last_fullsized":"2023-08-25T09:46:59.387177+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:19:35.074854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385983+0000","last_change":"2023-08-25T09:46:58.407130+0000","last_active":"2023-08-25T09:46:59.385983+0000","last_peered":"2023-08-25T09:46:59.385983+0000","last_clean":"2023-08-25T09:46:59.385983+0000","last_became_active":"2023-08-25T09:46:58.406902+0000","last_became_peered":"2023-08-25T09:46:58.406902+0000","last_unstale":"2023-08-25T09:46:59.385983+0000","last_undegraded":"2023-08-25T09:46:59.385983+0000","last_fullsized":"2023-08-25T09:46:59.385983+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:58:37.291808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385848+0000","last_change":"2023-08-25T09:46:58.403983+0000","last_active":"2023-08-25T09:46:59.385848+0000","last_peered":"2023-08-25T09:46:59.385848+0000","last_clean":"2023-08-25T09:46:59.385848+0000","last_became_active":"2023-08-25T09:46:58.402479+0000","last_became_peered":"2023-08-25T09:46:58.402479+0000","last_unstale":"2023-08-25T09:46:59.385848+0000","last_undegraded":"2023-08-25T09:46:59.385848+0000","last_fullsized":"2023-08-25T09:46:59.385848+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:20:58.773715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.411031+0000","last_change":"2023-08-25T09:46:58.410811+0000","last_active":"2023-08-25T09:46:58.411031+0000","last_peered":"2023-08-25T09:46:58.411031+0000","last_clean":"2023-08-25T09:46:58.411031+0000","last_became_active":"2023-08-25T09:46:58.407902+0000","last_became_peered":"2023-08-25T09:46:58.407902+0000","last_unstale":"2023-08-25T09:46:58.411031+0000","last_undegraded":"2023-08-25T09:46:58.411031+0000","last_fullsized":"2023-08-25T09:46:58.411031+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:57:25.269137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382963+0000","last_change":"2023-08-25T09:46:58.403176+0000","last_active":"2023-08-25T09:46:59.382963+0000","last_peered":"2023-08-25T09:46:59.382963+0000","last_clean":"2023-08-25T09:46:59.382963+0000","last_became_active":"2023-08-25T09:46:58.402356+0000","last_became_peered":"2023-08-25T09:46:58.402356+0000","last_unstale":"2023-08-25T09:46:59.382963+0000","last_undegraded":"2023-08-25T09:46:59.382963+0000","last_fullsized":"2023-08-25T09:46:59.382963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:40.112749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.410381+0000","last_change":"2023-08-25T09:46:58.410312+0000","last_active":"2023-08-25T09:46:58.410381+0000","last_peered":"2023-08-25T09:46:58.410381+0000","last_clean":"2023-08-25T09:46:58.410381+0000","last_became_active":"2023-08-25T09:46:58.410049+0000","last_became_peered":"2023-08-25T09:46:58.410049+0000","last_unstale":"2023-08-25T09:46:58.410381+0000","last_undegraded":"2023-08-25T09:46:58.410381+0000","last_fullsized":"2023-08-25T09:46:58.410381+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:01:58.219410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408629+0000","last_change":"2023-08-25T09:46:58.408468+0000","last_active":"2023-08-25T09:46:58.408629+0000","last_peered":"2023-08-25T09:46:58.408629+0000","last_clean":"2023-08-25T09:46:58.408629+0000","last_became_active":"2023-08-25T09:46:58.408171+0000","last_became_peered":"2023-08-25T09:46:58.408171+0000","last_unstale":"2023-08-25T09:46:58.408629+0000","last_undegraded":"2023-08-25T09:46:58.408629+0000","last_fullsized":"2023-08-25T09:46:58.408629+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:52.783619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.407122+0000","last_change":"2023-08-25T09:46:58.407047+0000","last_active":"2023-08-25T09:46:58.407122+0000","last_peered":"2023-08-25T09:46:58.407122+0000","last_clean":"2023-08-25T09:46:58.407122+0000","last_became_active":"2023-08-25T09:46:58.406822+0000","last_became_peered":"2023-08-25T09:46:58.406822+0000","last_unstale":"2023-08-25T09:46:58.407122+0000","last_undegraded":"2023-08-25T09:46:58.407122+0000","last_fullsized":"2023-08-25T09:46:58.407122+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:13:33.254186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409605+0000","last_change":"2023-08-25T09:46:58.409474+0000","last_active":"2023-08-25T09:46:58.409605+0000","last_peered":"2023-08-25T09:46:58.409605+0000","last_clean":"2023-08-25T09:46:58.409605+0000","last_became_active":"2023-08-25T09:46:58.409020+0000","last_became_peered":"2023-08-25T09:46:58.409020+0000","last_unstale":"2023-08-25T09:46:58.409605+0000","last_undegraded":"2023-08-25T09:46:58.409605+0000","last_fullsized":"2023-08-25T09:46:58.409605+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:37:41.601980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408670+0000","last_change":"2023-08-25T09:46:58.408433+0000","last_active":"2023-08-25T09:46:58.408670+0000","last_peered":"2023-08-25T09:46:58.408670+0000","last_clean":"2023-08-25T09:46:58.408670+0000","last_became_active":"2023-08-25T09:46:58.408198+0000","last_became_peered":"2023-08-25T09:46:58.408198+0000","last_unstale":"2023-08-25T09:46:58.408670+0000","last_undegraded":"2023-08-25T09:46:58.408670+0000","last_fullsized":"2023-08-25T09:46:58.408670+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:49.170623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.409567+0000","last_change":"2023-08-25T09:46:58.409478+0000","last_active":"2023-08-25T09:46:58.409567+0000","last_peered":"2023-08-25T09:46:58.409567+0000","last_clean":"2023-08-25T09:46:58.409567+0000","last_became_active":"2023-08-25T09:46:58.409208+0000","last_became_peered":"2023-08-25T09:46:58.409208+0000","last_unstale":"2023-08-25T09:46:58.409567+0000","last_undegraded":"2023-08-25T09:46:58.409567+0000","last_fullsized":"2023-08-25T09:46:58.409567+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:24:05.357659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408711+0000","last_change":"2023-08-25T09:46:58.408261+0000","last_active":"2023-08-25T09:46:58.408711+0000","last_peered":"2023-08-25T09:46:58.408711+0000","last_clean":"2023-08-25T09:46:58.408711+0000","last_became_active":"2023-08-25T09:46:58.407962+0000","last_became_peered":"2023-08-25T09:46:58.407962+0000","last_unstale":"2023-08-25T09:46:58.408711+0000","last_undegraded":"2023-08-25T09:46:58.408711+0000","last_fullsized":"2023-08-25T09:46:58.408711+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:59:58.054230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386260+0000","last_change":"2023-08-25T09:46:58.405379+0000","last_active":"2023-08-25T09:46:59.386260+0000","last_peered":"2023-08-25T09:46:59.386260+0000","last_clean":"2023-08-25T09:46:59.386260+0000","last_became_active":"2023-08-25T09:46:58.405025+0000","last_became_peered":"2023-08-25T09:46:58.405025+0000","last_unstale":"2023-08-25T09:46:59.386260+0000","last_undegraded":"2023-08-25T09:46:59.386260+0000","last_fullsized":"2023-08-25T09:46:59.386260+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:27:20.680767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385743+0000","last_change":"2023-08-25T09:46:58.401820+0000","last_active":"2023-08-25T09:46:59.385743+0000","last_peered":"2023-08-25T09:46:59.385743+0000","last_clean":"2023-08-25T09:46:59.385743+0000","last_became_active":"2023-08-25T09:46:58.399296+0000","last_became_peered":"2023-08-25T09:46:58.399296+0000","last_unstale":"2023-08-25T09:46:59.385743+0000","last_undegraded":"2023-08-25T09:46:59.385743+0000","last_fullsized":"2023-08-25T09:46:59.385743+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:42:18.572044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387900+0000","last_change":"2023-08-25T09:46:58.407120+0000","last_active":"2023-08-25T09:46:59.387900+0000","last_peered":"2023-08-25T09:46:59.387900+0000","last_clean":"2023-08-25T09:46:59.387900+0000","last_became_active":"2023-08-25T09:46:58.406817+0000","last_became_peered":"2023-08-25T09:46:58.406817+0000","last_unstale":"2023-08-25T09:46:59.387900+0000","last_undegraded":"2023-08-25T09:46:59.387900+0000","last_fullsized":"2023-08-25T09:46:59.387900+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:28:06.714953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402873+0000","last_change":"2023-08-25T09:46:58.401705+0000","last_active":"2023-08-25T09:46:58.402873+0000","last_peered":"2023-08-25T09:46:58.402873+0000","last_clean":"2023-08-25T09:46:58.402873+0000","last_became_active":"2023-08-25T09:46:58.401131+0000","last_became_peered":"2023-08-25T09:46:58.401131+0000","last_unstale":"2023-08-25T09:46:58.402873+0000","last_undegraded":"2023-08-25T09:46:58.402873+0000","last_fullsized":"2023-08-25T09:46:58.402873+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:13:22.777432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402828+0000","last_change":"2023-08-25T09:46:58.400062+0000","last_active":"2023-08-25T09:46:58.402828+0000","last_peered":"2023-08-25T09:46:58.402828+0000","last_clean":"2023-08-25T09:46:58.402828+0000","last_became_active":"2023-08-25T09:46:58.399454+0000","last_became_peered":"2023-08-25T09:46:58.399454+0000","last_unstale":"2023-08-25T09:46:58.402828+0000","last_undegraded":"2023-08-25T09:46:58.402828+0000","last_fullsized":"2023-08-25T09:46:58.402828+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:46:43.565623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.400325+0000","last_change":"2023-08-25T09:46:58.400230+0000","last_active":"2023-08-25T09:46:58.400325+0000","last_peered":"2023-08-25T09:46:58.400325+0000","last_clean":"2023-08-25T09:46:58.400325+0000","last_became_active":"2023-08-25T09:46:58.399880+0000","last_became_peered":"2023-08-25T09:46:58.399880+0000","last_unstale":"2023-08-25T09:46:58.400325+0000","last_undegraded":"2023-08-25T09:46:58.400325+0000","last_fullsized":"2023-08-25T09:46:58.400325+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:30:02.737125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"149'4","reported_seq":15,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.840035+0000","last_change":"2023-08-25T09:46:57.796710+0000","last_active":"2023-08-25T09:46:58.840035+0000","last_peered":"2023-08-25T09:46:58.840035+0000","last_clean":"2023-08-25T09:46:58.840035+0000","last_became_active":"2023-08-25T09:46:57.796414+0000","last_became_peered":"2023-08-25T09:46:57.796414+0000","last_unstale":"2023-08-25T09:46:58.840035+0000","last_undegraded":"2023-08-25T09:46:58.840035+0000","last_fullsized":"2023-08-25T09:46:58.840035+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:35:49.005173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383314+0000","last_change":"2023-08-25T09:46:58.402995+0000","last_active":"2023-08-25T09:46:59.383314+0000","last_peered":"2023-08-25T09:46:59.383314+0000","last_clean":"2023-08-25T09:46:59.383314+0000","last_became_active":"2023-08-25T09:46:58.401926+0000","last_became_peered":"2023-08-25T09:46:58.401926+0000","last_unstale":"2023-08-25T09:46:59.383314+0000","last_undegraded":"2023-08-25T09:46:59.383314+0000","last_fullsized":"2023-08-25T09:46:59.383314+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:33:04.852695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"149'1","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386459+0000","last_change":"2023-08-25T09:46:57.807213+0000","last_active":"2023-08-25T09:46:59.386459+0000","last_peered":"2023-08-25T09:46:59.386459+0000","last_clean":"2023-08-25T09:46:59.386459+0000","last_became_active":"2023-08-25T09:46:57.806785+0000","last_became_peered":"2023-08-25T09:46:57.806785+0000","last_unstale":"2023-08-25T09:46:59.386459+0000","last_undegraded":"2023-08-25T09:46:59.386459+0000","last_fullsized":"2023-08-25T09:46:59.386459+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:23.429545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.406688+0000","last_change":"2023-08-25T09:46:58.405960+0000","last_active":"2023-08-25T09:46:58.406688+0000","last_peered":"2023-08-25T09:46:58.406688+0000","last_clean":"2023-08-25T09:46:58.406688+0000","last_became_active":"2023-08-25T09:46:58.405638+0000","last_became_peered":"2023-08-25T09:46:58.405638+0000","last_unstale":"2023-08-25T09:46:58.406688+0000","last_undegraded":"2023-08-25T09:46:58.406688+0000","last_fullsized":"2023-08-25T09:46:58.406688+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:21.391199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"149'2","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.839410+0000","last_change":"2023-08-25T09:46:57.804438+0000","last_active":"2023-08-25T09:46:58.839410+0000","last_peered":"2023-08-25T09:46:58.839410+0000","last_clean":"2023-08-25T09:46:58.839410+0000","last_became_active":"2023-08-25T09:46:57.804124+0000","last_became_peered":"2023-08-25T09:46:57.804124+0000","last_unstale":"2023-08-25T09:46:58.839410+0000","last_undegraded":"2023-08-25T09:46:58.839410+0000","last_fullsized":"2023-08-25T09:46:58.839410+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:05.689473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391002+0000","last_change":"2023-08-25T09:46:58.390625+0000","last_active":"2023-08-25T09:46:58.391002+0000","last_peered":"2023-08-25T09:46:58.391002+0000","last_clean":"2023-08-25T09:46:58.391002+0000","last_became_active":"2023-08-25T09:46:58.389486+0000","last_became_peered":"2023-08-25T09:46:58.389486+0000","last_unstale":"2023-08-25T09:46:58.391002+0000","last_undegraded":"2023-08-25T09:46:58.391002+0000","last_fullsized":"2023-08-25T09:46:58.391002+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:27:39.787823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383836+0000","last_change":"2023-08-25T09:46:57.806745+0000","last_active":"2023-08-25T09:46:59.383836+0000","last_peered":"2023-08-25T09:46:59.383836+0000","last_clean":"2023-08-25T09:46:59.383836+0000","last_became_active":"2023-08-25T09:46:57.806189+0000","last_became_peered":"2023-08-25T09:46:57.806189+0000","last_unstale":"2023-08-25T09:46:59.383836+0000","last_undegraded":"2023-08-25T09:46:59.383836+0000","last_fullsized":"2023-08-25T09:46:59.383836+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:10:47.537865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402713+0000","last_change":"2023-08-25T09:46:58.401213+0000","last_active":"2023-08-25T09:46:58.402713+0000","last_peered":"2023-08-25T09:46:58.402713+0000","last_clean":"2023-08-25T09:46:58.402713+0000","last_became_active":"2023-08-25T09:46:58.400822+0000","last_became_peered":"2023-08-25T09:46:58.400822+0000","last_unstale":"2023-08-25T09:46:58.402713+0000","last_undegraded":"2023-08-25T09:46:58.402713+0000","last_fullsized":"2023-08-25T09:46:58.402713+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:07:11.015122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"149'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.836433+0000","last_change":"2023-08-25T09:46:57.802141+0000","last_active":"2023-08-25T09:46:58.836433+0000","last_peered":"2023-08-25T09:46:58.836433+0000","last_clean":"2023-08-25T09:46:58.836433+0000","last_became_active":"2023-08-25T09:46:57.801615+0000","last_became_peered":"2023-08-25T09:46:57.801615+0000","last_unstale":"2023-08-25T09:46:58.836433+0000","last_undegraded":"2023-08-25T09:46:58.836433+0000","last_fullsized":"2023-08-25T09:46:58.836433+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:17:39.769119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402777+0000","last_change":"2023-08-25T09:46:58.394876+0000","last_active":"2023-08-25T09:46:58.402777+0000","last_peered":"2023-08-25T09:46:58.402777+0000","last_clean":"2023-08-25T09:46:58.402777+0000","last_became_active":"2023-08-25T09:46:58.394202+0000","last_became_peered":"2023-08-25T09:46:58.394202+0000","last_unstale":"2023-08-25T09:46:58.402777+0000","last_undegraded":"2023-08-25T09:46:58.402777+0000","last_fullsized":"2023-08-25T09:46:58.402777+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:34:24.862537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"149'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.839134+0000","last_change":"2023-08-25T09:46:57.808292+0000","last_active":"2023-08-25T09:46:58.839134+0000","last_peered":"2023-08-25T09:46:58.839134+0000","last_clean":"2023-08-25T09:46:58.839134+0000","last_became_active":"2023-08-25T09:46:57.808056+0000","last_became_peered":"2023-08-25T09:46:57.808056+0000","last_unstale":"2023-08-25T09:46:58.839134+0000","last_undegraded":"2023-08-25T09:46:58.839134+0000","last_fullsized":"2023-08-25T09:46:58.839134+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:07:40.774858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387067+0000","last_change":"2023-08-25T09:46:58.393063+0000","last_active":"2023-08-25T09:46:59.387067+0000","last_peered":"2023-08-25T09:46:59.387067+0000","last_clean":"2023-08-25T09:46:59.387067+0000","last_became_active":"2023-08-25T09:46:58.392680+0000","last_became_peered":"2023-08-25T09:46:58.392680+0000","last_unstale":"2023-08-25T09:46:59.387067+0000","last_undegraded":"2023-08-25T09:46:59.387067+0000","last_fullsized":"2023-08-25T09:46:59.387067+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:33:43.049388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.387985+0000","last_change":"2023-08-25T09:46:57.804467+0000","last_active":"2023-08-25T09:46:58.387985+0000","last_peered":"2023-08-25T09:46:58.387985+0000","last_clean":"2023-08-25T09:46:58.387985+0000","last_became_active":"2023-08-25T09:46:57.804133+0000","last_became_peered":"2023-08-25T09:46:57.804133+0000","last_unstale":"2023-08-25T09:46:58.387985+0000","last_undegraded":"2023-08-25T09:46:58.387985+0000","last_fullsized":"2023-08-25T09:46:58.387985+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:10.928801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.408802+0000","last_change":"2023-08-25T09:46:58.408726+0000","last_active":"2023-08-25T09:46:58.408802+0000","last_peered":"2023-08-25T09:46:58.408802+0000","last_clean":"2023-08-25T09:46:58.408802+0000","last_became_active":"2023-08-25T09:46:58.408481+0000","last_became_peered":"2023-08-25T09:46:58.408481+0000","last_unstale":"2023-08-25T09:46:58.408802+0000","last_undegraded":"2023-08-25T09:46:58.408802+0000","last_fullsized":"2023-08-25T09:46:58.408802+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:42.466186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388194+0000","last_change":"2023-08-25T09:46:57.795116+0000","last_active":"2023-08-25T09:46:59.388194+0000","last_peered":"2023-08-25T09:46:59.388194+0000","last_clean":"2023-08-25T09:46:59.388194+0000","last_became_active":"2023-08-25T09:46:57.794578+0000","last_became_peered":"2023-08-25T09:46:57.794578+0000","last_unstale":"2023-08-25T09:46:59.388194+0000","last_undegraded":"2023-08-25T09:46:59.388194+0000","last_fullsized":"2023-08-25T09:46:59.388194+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:33:50.170382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.400327+0000","last_change":"2023-08-25T09:46:58.397926+0000","last_active":"2023-08-25T09:46:58.400327+0000","last_peered":"2023-08-25T09:46:58.400327+0000","last_clean":"2023-08-25T09:46:58.400327+0000","last_became_active":"2023-08-25T09:46:58.397314+0000","last_became_peered":"2023-08-25T09:46:58.397314+0000","last_unstale":"2023-08-25T09:46:58.400327+0000","last_undegraded":"2023-08-25T09:46:58.400327+0000","last_fullsized":"2023-08-25T09:46:58.400327+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:23.468922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"149'1","reported_seq":14,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386801+0000","last_change":"2023-08-25T09:46:57.807033+0000","last_active":"2023-08-25T09:46:59.386801+0000","last_peered":"2023-08-25T09:46:59.386801+0000","last_clean":"2023-08-25T09:46:59.386801+0000","last_became_active":"2023-08-25T09:46:57.806435+0000","last_became_peered":"2023-08-25T09:46:57.806435+0000","last_unstale":"2023-08-25T09:46:59.386801+0000","last_undegraded":"2023-08-25T09:46:59.386801+0000","last_fullsized":"2023-08-25T09:46:59.386801+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:20.802523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.410556+0000","last_change":"2023-08-25T09:46:58.410469+0000","last_active":"2023-08-25T09:46:58.410556+0000","last_peered":"2023-08-25T09:46:58.410556+0000","last_clean":"2023-08-25T09:46:58.410556+0000","last_became_active":"2023-08-25T09:46:58.408436+0000","last_became_peered":"2023-08-25T09:46:58.408436+0000","last_unstale":"2023-08-25T09:46:58.410556+0000","last_undegraded":"2023-08-25T09:46:58.410556+0000","last_fullsized":"2023-08-25T09:46:58.410556+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:54.212755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.389029+0000","last_change":"2023-08-25T09:46:57.804036+0000","last_active":"2023-08-25T09:46:59.389029+0000","last_peered":"2023-08-25T09:46:59.389029+0000","last_clean":"2023-08-25T09:46:59.389029+0000","last_became_active":"2023-08-25T09:46:57.803048+0000","last_became_peered":"2023-08-25T09:46:57.803048+0000","last_unstale":"2023-08-25T09:46:59.389029+0000","last_undegraded":"2023-08-25T09:46:59.389029+0000","last_fullsized":"2023-08-25T09:46:59.389029+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:48:36.771533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383675+0000","last_change":"2023-08-25T09:46:58.410561+0000","last_active":"2023-08-25T09:46:59.383675+0000","last_peered":"2023-08-25T09:46:59.383675+0000","last_clean":"2023-08-25T09:46:59.383675+0000","last_became_active":"2023-08-25T09:46:58.410158+0000","last_became_peered":"2023-08-25T09:46:58.410158+0000","last_unstale":"2023-08-25T09:46:59.383675+0000","last_undegraded":"2023-08-25T09:46:59.383675+0000","last_fullsized":"2023-08-25T09:46:59.383675+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:28:25.650973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"149'1","reported_seq":12,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.839479+0000","last_change":"2023-08-25T09:46:57.802901+0000","last_active":"2023-08-25T09:46:58.839479+0000","last_peered":"2023-08-25T09:46:58.839479+0000","last_clean":"2023-08-25T09:46:58.839479+0000","last_became_active":"2023-08-25T09:46:57.802360+0000","last_became_peered":"2023-08-25T09:46:57.802360+0000","last_unstale":"2023-08-25T09:46:58.839479+0000","last_undegraded":"2023-08-25T09:46:58.839479+0000","last_fullsized":"2023-08-25T09:46:58.839479+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:31:04.471433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.411710+0000","last_change":"2023-08-25T09:46:58.411092+0000","last_active":"2023-08-25T09:46:58.411710+0000","last_peered":"2023-08-25T09:46:58.411710+0000","last_clean":"2023-08-25T09:46:58.411710+0000","last_became_active":"2023-08-25T09:46:58.409848+0000","last_became_peered":"2023-08-25T09:46:58.409848+0000","last_unstale":"2023-08-25T09:46:58.411710+0000","last_undegraded":"2023-08-25T09:46:58.411710+0000","last_fullsized":"2023-08-25T09:46:58.411710+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:30:18.037690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.387888+0000","last_change":"2023-08-25T09:46:57.807397+0000","last_active":"2023-08-25T09:46:58.387888+0000","last_peered":"2023-08-25T09:46:58.387888+0000","last_clean":"2023-08-25T09:46:58.387888+0000","last_became_active":"2023-08-25T09:46:57.806671+0000","last_became_peered":"2023-08-25T09:46:57.806671+0000","last_unstale":"2023-08-25T09:46:58.387888+0000","last_undegraded":"2023-08-25T09:46:58.387888+0000","last_fullsized":"2023-08-25T09:46:58.387888+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:00:37.854739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383469+0000","last_change":"2023-08-25T09:46:58.407697+0000","last_active":"2023-08-25T09:46:59.383469+0000","last_peered":"2023-08-25T09:46:59.383469+0000","last_clean":"2023-08-25T09:46:59.383469+0000","last_became_active":"2023-08-25T09:46:58.402997+0000","last_became_peered":"2023-08-25T09:46:58.402997+0000","last_unstale":"2023-08-25T09:46:59.383469+0000","last_undegraded":"2023-08-25T09:46:59.383469+0000","last_fullsized":"2023-08-25T09:46:59.383469+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:53.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386798+0000","last_change":"2023-08-25T09:46:57.809731+0000","last_active":"2023-08-25T09:46:59.386798+0000","last_peered":"2023-08-25T09:46:59.386798+0000","last_clean":"2023-08-25T09:46:59.386798+0000","last_became_active":"2023-08-25T09:46:57.808952+0000","last_became_peered":"2023-08-25T09:46:57.808952+0000","last_unstale":"2023-08-25T09:46:59.386798+0000","last_undegraded":"2023-08-25T09:46:59.386798+0000","last_fullsized":"2023-08-25T09:46:59.386798+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:42:17.566319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388367+0000","last_change":"2023-08-25T09:46:58.409531+0000","last_active":"2023-08-25T09:46:59.388367+0000","last_peered":"2023-08-25T09:46:59.388367+0000","last_clean":"2023-08-25T09:46:59.388367+0000","last_became_active":"2023-08-25T09:46:58.409239+0000","last_became_peered":"2023-08-25T09:46:58.409239+0000","last_unstale":"2023-08-25T09:46:59.388367+0000","last_undegraded":"2023-08-25T09:46:59.388367+0000","last_fullsized":"2023-08-25T09:46:59.388367+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:33:43.049388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.396704+0000","last_change":"2023-08-25T09:46:57.802821+0000","last_active":"2023-08-25T09:46:58.396704+0000","last_peered":"2023-08-25T09:46:58.396704+0000","last_clean":"2023-08-25T09:46:58.396704+0000","last_became_active":"2023-08-25T09:46:57.802505+0000","last_became_peered":"2023-08-25T09:46:57.802505+0000","last_unstale":"2023-08-25T09:46:58.396704+0000","last_undegraded":"2023-08-25T09:46:58.396704+0000","last_fullsized":"2023-08-25T09:46:58.396704+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:17.002089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.385389+0000","last_change":"2023-08-25T09:46:58.401628+0000","last_active":"2023-08-25T09:46:59.385389+0000","last_peered":"2023-08-25T09:46:59.385389+0000","last_clean":"2023-08-25T09:46:59.385389+0000","last_became_active":"2023-08-25T09:46:58.398832+0000","last_became_peered":"2023-08-25T09:46:58.398832+0000","last_unstale":"2023-08-25T09:46:59.385389+0000","last_undegraded":"2023-08-25T09:46:59.385389+0000","last_fullsized":"2023-08-25T09:46:59.385389+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:09:24.435271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388487+0000","last_change":"2023-08-25T09:46:57.808957+0000","last_active":"2023-08-25T09:46:59.388487+0000","last_peered":"2023-08-25T09:46:59.388487+0000","last_clean":"2023-08-25T09:46:59.388487+0000","last_became_active":"2023-08-25T09:46:57.806512+0000","last_became_peered":"2023-08-25T09:46:57.806512+0000","last_unstale":"2023-08-25T09:46:59.388487+0000","last_undegraded":"2023-08-25T09:46:59.388487+0000","last_fullsized":"2023-08-25T09:46:59.388487+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:42:32.794430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402850+0000","last_change":"2023-08-25T09:46:58.400694+0000","last_active":"2023-08-25T09:46:58.402850+0000","last_peered":"2023-08-25T09:46:58.402850+0000","last_clean":"2023-08-25T09:46:58.402850+0000","last_became_active":"2023-08-25T09:46:58.400039+0000","last_became_peered":"2023-08-25T09:46:58.400039+0000","last_unstale":"2023-08-25T09:46:58.402850+0000","last_undegraded":"2023-08-25T09:46:58.402850+0000","last_fullsized":"2023-08-25T09:46:58.402850+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:39.077929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.399851+0000","last_change":"2023-08-25T09:46:57.806778+0000","last_active":"2023-08-25T09:46:58.399851+0000","last_peered":"2023-08-25T09:46:58.399851+0000","last_clean":"2023-08-25T09:46:58.399851+0000","last_became_active":"2023-08-25T09:46:57.806536+0000","last_became_peered":"2023-08-25T09:46:57.806536+0000","last_unstale":"2023-08-25T09:46:58.399851+0000","last_undegraded":"2023-08-25T09:46:58.399851+0000","last_fullsized":"2023-08-25T09:46:58.399851+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:33:50.604157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.388682+0000","last_change":"2023-08-25T09:46:58.398350+0000","last_active":"2023-08-25T09:46:59.388682+0000","last_peered":"2023-08-25T09:46:59.388682+0000","last_clean":"2023-08-25T09:46:59.388682+0000","last_became_active":"2023-08-25T09:46:58.397574+0000","last_became_peered":"2023-08-25T09:46:58.397574+0000","last_unstale":"2023-08-25T09:46:59.388682+0000","last_undegraded":"2023-08-25T09:46:59.388682+0000","last_fullsized":"2023-08-25T09:46:59.388682+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:13:02.773948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"149'2","reported_seq":15,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383262+0000","last_change":"2023-08-25T09:46:57.805782+0000","last_active":"2023-08-25T09:46:59.383262+0000","last_peered":"2023-08-25T09:46:59.383262+0000","last_clean":"2023-08-25T09:46:59.383262+0000","last_became_active":"2023-08-25T09:46:57.805464+0000","last_became_peered":"2023-08-25T09:46:57.805464+0000","last_unstale":"2023-08-25T09:46:59.383262+0000","last_undegraded":"2023-08-25T09:46:59.383262+0000","last_fullsized":"2023-08-25T09:46:59.383262+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:25.565647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386122+0000","last_change":"2023-08-25T09:46:58.403952+0000","last_active":"2023-08-25T09:46:59.386122+0000","last_peered":"2023-08-25T09:46:59.386122+0000","last_clean":"2023-08-25T09:46:59.386122+0000","last_became_active":"2023-08-25T09:46:58.403694+0000","last_became_peered":"2023-08-25T09:46:58.403694+0000","last_unstale":"2023-08-25T09:46:59.386122+0000","last_undegraded":"2023-08-25T09:46:59.386122+0000","last_fullsized":"2023-08-25T09:46:59.386122+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:21.808249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.395991+0000","last_change":"2023-08-25T09:46:57.807436+0000","last_active":"2023-08-25T09:46:58.395991+0000","last_peered":"2023-08-25T09:46:58.395991+0000","last_clean":"2023-08-25T09:46:58.395991+0000","last_became_active":"2023-08-25T09:46:57.807148+0000","last_became_peered":"2023-08-25T09:46:57.807148+0000","last_unstale":"2023-08-25T09:46:58.395991+0000","last_undegraded":"2023-08-25T09:46:58.395991+0000","last_fullsized":"2023-08-25T09:46:58.395991+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:33.077943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.405360+0000","last_change":"2023-08-25T09:46:58.405270+0000","last_active":"2023-08-25T09:46:58.405360+0000","last_peered":"2023-08-25T09:46:58.405360+0000","last_clean":"2023-08-25T09:46:58.405360+0000","last_became_active":"2023-08-25T09:46:58.404402+0000","last_became_peered":"2023-08-25T09:46:58.404402+0000","last_unstale":"2023-08-25T09:46:58.405360+0000","last_undegraded":"2023-08-25T09:46:58.405360+0000","last_fullsized":"2023-08-25T09:46:58.405360+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:08:53.762969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.398849+0000","last_change":"2023-08-25T09:46:57.807245+0000","last_active":"2023-08-25T09:46:58.398849+0000","last_peered":"2023-08-25T09:46:58.398849+0000","last_clean":"2023-08-25T09:46:58.398849+0000","last_became_active":"2023-08-25T09:46:57.805588+0000","last_became_peered":"2023-08-25T09:46:57.805588+0000","last_unstale":"2023-08-25T09:46:58.398849+0000","last_undegraded":"2023-08-25T09:46:58.398849+0000","last_fullsized":"2023-08-25T09:46:58.398849+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:46:23.475717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387822+0000","last_change":"2023-08-25T09:46:58.395055+0000","last_active":"2023-08-25T09:46:59.387822+0000","last_peered":"2023-08-25T09:46:59.387822+0000","last_clean":"2023-08-25T09:46:59.387822+0000","last_became_active":"2023-08-25T09:46:58.394608+0000","last_became_peered":"2023-08-25T09:46:58.394608+0000","last_unstale":"2023-08-25T09:46:59.387822+0000","last_undegraded":"2023-08-25T09:46:59.387822+0000","last_fullsized":"2023-08-25T09:46:59.387822+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:52.386906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.389319+0000","last_change":"2023-08-25T09:46:57.804522+0000","last_active":"2023-08-25T09:46:59.389319+0000","last_peered":"2023-08-25T09:46:59.389319+0000","last_clean":"2023-08-25T09:46:59.389319+0000","last_became_active":"2023-08-25T09:46:57.803573+0000","last_became_peered":"2023-08-25T09:46:57.803573+0000","last_unstale":"2023-08-25T09:46:59.389319+0000","last_undegraded":"2023-08-25T09:46:59.389319+0000","last_fullsized":"2023-08-25T09:46:59.389319+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:51.381181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386099+0000","last_change":"2023-08-25T09:46:58.407934+0000","last_active":"2023-08-25T09:46:59.386099+0000","last_peered":"2023-08-25T09:46:59.386099+0000","last_clean":"2023-08-25T09:46:59.386099+0000","last_became_active":"2023-08-25T09:46:58.406811+0000","last_became_peered":"2023-08-25T09:46:58.406811+0000","last_unstale":"2023-08-25T09:46:59.386099+0000","last_undegraded":"2023-08-25T09:46:59.386099+0000","last_fullsized":"2023-08-25T09:46:59.386099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:22.513670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.400162+0000","last_change":"2023-08-25T09:46:57.802100+0000","last_active":"2023-08-25T09:46:58.400162+0000","last_peered":"2023-08-25T09:46:58.400162+0000","last_clean":"2023-08-25T09:46:58.400162+0000","last_became_active":"2023-08-25T09:46:57.801521+0000","last_became_peered":"2023-08-25T09:46:57.801521+0000","last_unstale":"2023-08-25T09:46:58.400162+0000","last_undegraded":"2023-08-25T09:46:58.400162+0000","last_fullsized":"2023-08-25T09:46:58.400162+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:55:14.134737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383008+0000","last_change":"2023-08-25T09:46:58.402356+0000","last_active":"2023-08-25T09:46:59.383008+0000","last_peered":"2023-08-25T09:46:59.383008+0000","last_clean":"2023-08-25T09:46:59.383008+0000","last_became_active":"2023-08-25T09:46:58.401869+0000","last_became_peered":"2023-08-25T09:46:58.401869+0000","last_unstale":"2023-08-25T09:46:59.383008+0000","last_undegraded":"2023-08-25T09:46:59.383008+0000","last_fullsized":"2023-08-25T09:46:59.383008+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:01.602066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.403647+0000","last_change":"2023-08-25T09:46:57.806218+0000","last_active":"2023-08-25T09:46:58.403647+0000","last_peered":"2023-08-25T09:46:58.403647+0000","last_clean":"2023-08-25T09:46:58.403647+0000","last_became_active":"2023-08-25T09:46:57.805985+0000","last_became_peered":"2023-08-25T09:46:57.805985+0000","last_unstale":"2023-08-25T09:46:58.403647+0000","last_undegraded":"2023-08-25T09:46:58.403647+0000","last_fullsized":"2023-08-25T09:46:58.403647+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:45.868036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.405439+0000","last_change":"2023-08-25T09:46:58.405343+0000","last_active":"2023-08-25T09:46:58.405439+0000","last_peered":"2023-08-25T09:46:58.405439+0000","last_clean":"2023-08-25T09:46:58.405439+0000","last_became_active":"2023-08-25T09:46:58.404510+0000","last_became_peered":"2023-08-25T09:46:58.404510+0000","last_unstale":"2023-08-25T09:46:58.405439+0000","last_undegraded":"2023-08-25T09:46:58.405439+0000","last_fullsized":"2023-08-25T09:46:58.405439+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:34.344098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383346+0000","last_change":"2023-08-25T09:46:57.807376+0000","last_active":"2023-08-25T09:46:59.383346+0000","last_peered":"2023-08-25T09:46:59.383346+0000","last_clean":"2023-08-25T09:46:59.383346+0000","last_became_active":"2023-08-25T09:46:57.806906+0000","last_became_peered":"2023-08-25T09:46:57.806906+0000","last_unstale":"2023-08-25T09:46:59.383346+0000","last_undegraded":"2023-08-25T09:46:59.383346+0000","last_fullsized":"2023-08-25T09:46:59.383346+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:51:23.503362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.398466+0000","last_change":"2023-08-25T09:46:58.397974+0000","last_active":"2023-08-25T09:46:58.398466+0000","last_peered":"2023-08-25T09:46:58.398466+0000","last_clean":"2023-08-25T09:46:58.398466+0000","last_became_active":"2023-08-25T09:46:58.396730+0000","last_became_peered":"2023-08-25T09:46:58.396730+0000","last_unstale":"2023-08-25T09:46:58.398466+0000","last_undegraded":"2023-08-25T09:46:58.398466+0000","last_fullsized":"2023-08-25T09:46:58.398466+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:40:49.259136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.399236+0000","last_change":"2023-08-25T09:46:57.805862+0000","last_active":"2023-08-25T09:46:58.399236+0000","last_peered":"2023-08-25T09:46:58.399236+0000","last_clean":"2023-08-25T09:46:58.399236+0000","last_became_active":"2023-08-25T09:46:57.805106+0000","last_became_peered":"2023-08-25T09:46:57.805106+0000","last_unstale":"2023-08-25T09:46:58.399236+0000","last_undegraded":"2023-08-25T09:46:58.399236+0000","last_fullsized":"2023-08-25T09:46:58.399236+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:37:45.483372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.402031+0000","last_change":"2023-08-25T09:46:58.401894+0000","last_active":"2023-08-25T09:46:58.402031+0000","last_peered":"2023-08-25T09:46:58.402031+0000","last_clean":"2023-08-25T09:46:58.402031+0000","last_became_active":"2023-08-25T09:46:58.401306+0000","last_became_peered":"2023-08-25T09:46:58.401306+0000","last_unstale":"2023-08-25T09:46:58.402031+0000","last_undegraded":"2023-08-25T09:46:58.402031+0000","last_fullsized":"2023-08-25T09:46:58.402031+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:40:59.406464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"149'8","reported_seq":19,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.840256+0000","last_change":"2023-08-25T09:46:57.795642+0000","last_active":"2023-08-25T09:46:58.840256+0000","last_peered":"2023-08-25T09:46:58.840256+0000","last_clean":"2023-08-25T09:46:58.840256+0000","last_became_active":"2023-08-25T09:46:57.794931+0000","last_became_peered":"2023-08-25T09:46:57.794931+0000","last_unstale":"2023-08-25T09:46:58.840256+0000","last_undegraded":"2023-08-25T09:46:58.840256+0000","last_fullsized":"2023-08-25T09:46:58.840256+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:23.955445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":393,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.396950+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:46:58.396950+0000","last_peered":"2023-08-25T09:46:58.396950+0000","last_clean":"2023-08-25T09:46:58.396950+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:46:58.396950+0000","last_undegraded":"2023-08-25T09:46:58.396950+0000","last_fullsized":"2023-08-25T09:46:58.396950+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:12:22.955616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391476+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:46:58.391476+0000","last_peered":"2023-08-25T09:46:58.391476+0000","last_clean":"2023-08-25T09:46:58.391476+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:46:58.391476+0000","last_undegraded":"2023-08-25T09:46:58.391476+0000","last_fullsized":"2023-08-25T09:46:58.391476+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:21.353922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.401170+0000","last_change":"2023-08-25T09:46:58.401037+0000","last_active":"2023-08-25T09:46:58.401170+0000","last_peered":"2023-08-25T09:46:58.401170+0000","last_clean":"2023-08-25T09:46:58.401170+0000","last_became_active":"2023-08-25T09:46:58.400553+0000","last_became_peered":"2023-08-25T09:46:58.400553+0000","last_unstale":"2023-08-25T09:46:58.401170+0000","last_undegraded":"2023-08-25T09:46:58.401170+0000","last_fullsized":"2023-08-25T09:46:58.401170+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:59.178154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"149'2","reported_seq":13,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.838119+0000","last_change":"2023-08-25T09:46:57.805528+0000","last_active":"2023-08-25T09:46:58.838119+0000","last_peered":"2023-08-25T09:46:58.838119+0000","last_clean":"2023-08-25T09:46:58.838119+0000","last_became_active":"2023-08-25T09:46:57.805190+0000","last_became_peered":"2023-08-25T09:46:57.805190+0000","last_unstale":"2023-08-25T09:46:58.838119+0000","last_undegraded":"2023-08-25T09:46:58.838119+0000","last_fullsized":"2023-08-25T09:46:58.838119+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:34.056911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":263,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.389600+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:46:58.389600+0000","last_peered":"2023-08-25T09:46:58.389600+0000","last_clean":"2023-08-25T09:46:58.389600+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:46:58.389600+0000","last_undegraded":"2023-08-25T09:46:58.389600+0000","last_fullsized":"2023-08-25T09:46:58.389600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:06:17.741556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382915+0000","last_change":"2023-08-25T09:46:58.407371+0000","last_active":"2023-08-25T09:46:59.382915+0000","last_peered":"2023-08-25T09:46:59.382915+0000","last_clean":"2023-08-25T09:46:59.382915+0000","last_became_active":"2023-08-25T09:46:58.406298+0000","last_became_peered":"2023-08-25T09:46:58.406298+0000","last_unstale":"2023-08-25T09:46:59.382915+0000","last_undegraded":"2023-08-25T09:46:59.382915+0000","last_fullsized":"2023-08-25T09:46:59.382915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:47.053310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.386634+0000","last_change":"2023-08-25T09:46:57.808138+0000","last_active":"2023-08-25T09:46:59.386634+0000","last_peered":"2023-08-25T09:46:59.386634+0000","last_clean":"2023-08-25T09:46:59.386634+0000","last_became_active":"2023-08-25T09:46:57.807894+0000","last_became_peered":"2023-08-25T09:46:57.807894+0000","last_unstale":"2023-08-25T09:46:59.386634+0000","last_undegraded":"2023-08-25T09:46:59.386634+0000","last_fullsized":"2023-08-25T09:46:59.386634+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:19:34.069129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":266,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387293+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:46:59.387293+0000","last_peered":"2023-08-25T09:46:59.387293+0000","last_clean":"2023-08-25T09:46:59.387293+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:46:59.387293+0000","last_undegraded":"2023-08-25T09:46:59.387293+0000","last_fullsized":"2023-08-25T09:46:59.387293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:04:50.990790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387806+0000","last_change":"2023-08-25T09:46:58.405793+0000","last_active":"2023-08-25T09:46:59.387806+0000","last_peered":"2023-08-25T09:46:59.387806+0000","last_clean":"2023-08-25T09:46:59.387806+0000","last_became_active":"2023-08-25T09:46:58.405501+0000","last_became_peered":"2023-08-25T09:46:58.405501+0000","last_unstale":"2023-08-25T09:46:59.387806+0000","last_undegraded":"2023-08-25T09:46:59.387806+0000","last_fullsized":"2023-08-25T09:46:59.387806+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:14.268254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.390922+0000","last_change":"2023-08-25T09:46:57.790379+0000","last_active":"2023-08-25T09:46:58.390922+0000","last_peered":"2023-08-25T09:46:58.390922+0000","last_clean":"2023-08-25T09:46:58.390922+0000","last_became_active":"2023-08-25T09:46:57.789516+0000","last_became_peered":"2023-08-25T09:46:57.789516+0000","last_unstale":"2023-08-25T09:46:58.390922+0000","last_undegraded":"2023-08-25T09:46:58.390922+0000","last_fullsized":"2023-08-25T09:46:58.390922+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:20:00.448961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.388529+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:46:58.388529+0000","last_peered":"2023-08-25T09:46:58.388529+0000","last_clean":"2023-08-25T09:46:58.388529+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:46:58.388529+0000","last_undegraded":"2023-08-25T09:46:58.388529+0000","last_fullsized":"2023-08-25T09:46:58.388529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:03:55.270769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.406599+0000","last_change":"2023-08-25T09:46:58.406492+0000","last_active":"2023-08-25T09:46:58.406599+0000","last_peered":"2023-08-25T09:46:58.406599+0000","last_clean":"2023-08-25T09:46:58.406599+0000","last_became_active":"2023-08-25T09:46:58.405642+0000","last_became_peered":"2023-08-25T09:46:58.405642+0000","last_unstale":"2023-08-25T09:46:58.406599+0000","last_undegraded":"2023-08-25T09:46:58.406599+0000","last_fullsized":"2023-08-25T09:46:58.406599+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:10.349935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.381275+0000","last_change":"2023-08-25T09:46:57.794364+0000","last_active":"2023-08-25T09:46:58.381275+0000","last_peered":"2023-08-25T09:46:58.381275+0000","last_clean":"2023-08-25T09:46:58.381275+0000","last_became_active":"2023-08-25T09:46:57.793965+0000","last_became_peered":"2023-08-25T09:46:57.793965+0000","last_unstale":"2023-08-25T09:46:58.381275+0000","last_undegraded":"2023-08-25T09:46:58.381275+0000","last_fullsized":"2023-08-25T09:46:58.381275+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:50:56.754615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":240,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.383920+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:46:59.383920+0000","last_peered":"2023-08-25T09:46:59.383920+0000","last_clean":"2023-08-25T09:46:59.383920+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:46:59.383920+0000","last_undegraded":"2023-08-25T09:46:59.383920+0000","last_fullsized":"2023-08-25T09:46:59.383920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:09:18.750849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.391050+0000","last_change":"2023-08-25T09:46:58.390802+0000","last_active":"2023-08-25T09:46:58.391050+0000","last_peered":"2023-08-25T09:46:58.391050+0000","last_clean":"2023-08-25T09:46:58.391050+0000","last_became_active":"2023-08-25T09:46:58.389873+0000","last_became_peered":"2023-08-25T09:46:58.389873+0000","last_unstale":"2023-08-25T09:46:58.391050+0000","last_undegraded":"2023-08-25T09:46:58.391050+0000","last_fullsized":"2023-08-25T09:46:58.391050+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:48.165528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.382098+0000","last_change":"2023-08-25T09:46:57.805400+0000","last_active":"2023-08-25T09:46:58.382098+0000","last_peered":"2023-08-25T09:46:58.382098+0000","last_clean":"2023-08-25T09:46:58.382098+0000","last_became_active":"2023-08-25T09:46:57.805171+0000","last_became_peered":"2023-08-25T09:46:57.805171+0000","last_unstale":"2023-08-25T09:46:58.382098+0000","last_undegraded":"2023-08-25T09:46:58.382098+0000","last_fullsized":"2023-08-25T09:46:58.382098+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:04:20.143242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.389302+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:46:58.389302+0000","last_peered":"2023-08-25T09:46:58.389302+0000","last_clean":"2023-08-25T09:46:58.389302+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:46:58.389302+0000","last_undegraded":"2023-08-25T09:46:58.389302+0000","last_fullsized":"2023-08-25T09:46:58.389302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:02:03.918320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":11,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.387511+0000","last_change":"2023-08-25T09:46:58.410284+0000","last_active":"2023-08-25T09:46:59.387511+0000","last_peered":"2023-08-25T09:46:59.387511+0000","last_clean":"2023-08-25T09:46:59.387511+0000","last_became_active":"2023-08-25T09:46:58.410046+0000","last_became_peered":"2023-08-25T09:46:58.410046+0000","last_unstale":"2023-08-25T09:46:59.387511+0000","last_undegraded":"2023-08-25T09:46:59.387511+0000","last_fullsized":"2023-08-25T09:46:59.387511+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:48:37.777258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"149'3","reported_seq":14,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.838595+0000","last_change":"2023-08-25T09:46:57.802862+0000","last_active":"2023-08-25T09:46:58.838595+0000","last_peered":"2023-08-25T09:46:58.838595+0000","last_clean":"2023-08-25T09:46:58.838595+0000","last_became_active":"2023-08-25T09:46:57.802548+0000","last_became_peered":"2023-08-25T09:46:57.802548+0000","last_unstale":"2023-08-25T09:46:58.838595+0000","last_undegraded":"2023-08-25T09:46:58.838595+0000","last_fullsized":"2023-08-25T09:46:58.838595+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:16:32.564166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":234,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.389315+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:46:58.389315+0000","last_peered":"2023-08-25T09:46:58.389315+0000","last_clean":"2023-08-25T09:46:58.389315+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:46:58.389315+0000","last_undegraded":"2023-08-25T09:46:58.389315+0000","last_fullsized":"2023-08-25T09:46:58.389315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:20:53.122003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.407039+0000","last_change":"2023-08-25T09:46:58.406965+0000","last_active":"2023-08-25T09:46:58.407039+0000","last_peered":"2023-08-25T09:46:58.407039+0000","last_clean":"2023-08-25T09:46:58.407039+0000","last_became_active":"2023-08-25T09:46:58.406620+0000","last_became_peered":"2023-08-25T09:46:58.406620+0000","last_unstale":"2023-08-25T09:46:58.407039+0000","last_undegraded":"2023-08-25T09:46:58.407039+0000","last_fullsized":"2023-08-25T09:46:58.407039+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:57.773745+0000","last_clean_scrub_stamp":"2023-08-25T09:46:57.773745+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:19:06.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"149'6","reported_seq":19,"reported_epoch":150,"state":"active+clean","last_fresh":"2023-08-25T09:46:59.382308+0000","last_change":"2023-08-25T09:46:57.803942+0000","last_active":"2023-08-25T09:46:59.382308+0000","last_peered":"2023-08-25T09:46:59.382308+0000","last_clean":"2023-08-25T09:46:59.382308+0000","last_became_active":"2023-08-25T09:46:57.803577+0000","last_became_peered":"2023-08-25T09:46:57.803577+0000","last_unstale":"2023-08-25T09:46:59.382308+0000","last_undegraded":"2023-08-25T09:46:59.382308+0000","last_fullsized":"2023-08-25T09:46:59.382308+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":147,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:46:56.768020+0000","last_clean_scrub_stamp":"2023-08-25T09:46:56.768020+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:57:12.075905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":262,"reported_epoch":149,"state":"active+clean","last_fresh":"2023-08-25T09:46:58.388228+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:46:58.388228+0000","last_peered":"2023-08-25T09:46:58.388228+0000","last_clean":"2023-08-25T09:46:58.388228+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:46:58.388228+0000","last_undegraded":"2023-08-25T09:46:58.388228+0000","last_fullsized":"2023-08-25T09:46:58.388228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:06.274850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574950,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115546,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0660000000000001,"15min":1.073},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.738,"5min":1.7909999999999999,"15min":10.747},"last":0.872},{"interface":"front","average":{"1min":1.077,"5min":1.129,"15min":1.0509999999999999},"min":{"1min":0.61699999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.6499999999999999,"5min":1.855,"15min":1.855},"last":0.92700000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.298,"15min":1.177},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.665,"5min":11.010999999999999,"15min":11.010999999999999},"last":1.278},{"interface":"front","average":{"1min":1.179,"5min":1.1619999999999999,"15min":1.1120000000000001},"min":{"1min":0.70899999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6559999999999999,"5min":1.667,"15min":1.7769999999999999},"last":1.347}]},{"osd":2,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1459999999999999,"15min":1.0920000000000001},"min":{"1min":0.66800000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.4550000000000001,"5min":1.619,"15min":1.619},"last":1.077},{"interface":"front","average":{"1min":1.22,"5min":1.294,"15min":1.234},"min":{"1min":0.67700000000000005,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.718,"5min":1.98,"15min":1.98},"last":1.419}]},{"osd":3,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.232,"15min":1.177},"min":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5580000000000001,"5min":1.6699999999999999,"15min":1.7110000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3660000000000001,"15min":1.3360000000000001},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.651,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.141}]},{"osd":4,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2889999999999999,"15min":1.2829999999999999},"min":{"1min":0.51300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.462},{"interface":"front","average":{"1min":1.335,"5min":1.3620000000000001,"15min":1.3080000000000001},"min":{"1min":0.53800000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.651,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.181}]},{"osd":5,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4350000000000001,"15min":1.363},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":1.8280000000000001,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.544},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.452,"15min":1.397},"min":{"1min":0.67700000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.806}]},{"osd":6,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.5529999999999999,"15min":1.464},"min":{"1min":0.72999999999999998,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.0259999999999998,"5min":2.032,"15min":2.1720000000000002},"last":1.597},{"interface":"front","average":{"1min":1.4550000000000001,"5min":1.4930000000000001,"15min":1.4199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":1.8029999999999999,"5min":2.0710000000000002,"15min":2.0710000000000002},"last":1.7470000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574950,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115524,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1519999999999999,"15min":1.1439999999999999},"min":{"1min":0.69799999999999995,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.796,"5min":1.804,"15min":1.804},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1539999999999999,"15min":1.1499999999999999},"min":{"1min":0.753,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.992,"5min":1.992,"15min":5.5410000000000004},"last":1.3759999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.179,"15min":1.1950000000000001},"min":{"1min":0.77000000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":2.198,"5min":2.198,"15min":5.6799999999999997},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.22,"15min":1.143},"min":{"1min":0.69199999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.3079999999999998,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.1240000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.157,"15min":1.117},"min":{"1min":0.78400000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.708},"last":1.052},{"interface":"front","average":{"1min":1.147,"5min":1.2829999999999999,"15min":1.2350000000000001},"min":{"1min":0.79900000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.5760000000000001,"5min":1.768,"15min":13.193},"last":1.3240000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3,"15min":1.2509999999999999},"min":{"1min":0.94499999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.2609999999999999,"15min":1.2569999999999999},"min":{"1min":0.85299999999999998,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7430000000000001,"5min":1.76,"15min":1.9239999999999999},"last":0.93899999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.3460000000000001,"15min":1.278},"min":{"1min":0.68400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.484},"last":1.464},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.369,"15min":1.341},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.671}]},{"osd":5,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.3300000000000001,"15min":1.2909999999999999},"min":{"1min":0.65000000000000002,"5min":0.624,"15min":0.59599999999999997},"max":{"1min":1.859,"5min":1.859,"15min":2.0049999999999999},"last":1.425},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4970000000000001,"15min":1.4039999999999999},"min":{"1min":0.73099999999999998,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.4420000000000002,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.5509999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:46:54 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.5249999999999999,"15min":1.51},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":2.6579999999999999,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.605,"5min":1.49,"15min":1.4830000000000001},"min":{"1min":0.81699999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.9710000000000001,"5min":1.9710000000000001,"15min":2.1890000000000001},"last":1.278}]}]},{"osd":5,"up_from":13,"seq":55834574950,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193872,"statfs":{"total":200000000,"available":198525178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.196,"15min":1.157},"min":{"1min":0.64000000000000001,"5min":0.497,"15min":0.497},"max":{"1min":1.7809999999999999,"5min":2.081,"15min":2.081},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.2210000000000001,"15min":1.204},"min":{"1min":0.77600000000000002,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.7629999999999999,"5min":1.8839999999999999,"15min":1.974},"last":1.1479999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.145,"15min":1.1830000000000001},"min":{"1min":0.73499999999999999,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6539999999999999,"5min":1.851,"15min":2.1779999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.1870000000000001,"15min":1.133},"min":{"1min":0.81399999999999995,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.696,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.204}]},{"osd":2,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2450000000000001,"15min":1.1990000000000001},"min":{"1min":0.80400000000000005,"5min":0.73899999999999999,"15min":0.73599999999999999},"max":{"1min":1.6459999999999999,"5min":2.3690000000000002,"15min":2.3690000000000002},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.27,"15min":1.228},"min":{"1min":0.82899999999999996,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.724,"5min":1.778,"15min":1.962},"last":1.2929999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.2030000000000001,"15min":1.292},"min":{"1min":0.65900000000000003,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.704,"5min":2.3439999999999999,"15min":17.356999999999999},"last":1.323},{"interface":"front","average":{"1min":1.369,"5min":1.343,"15min":1.2969999999999999},"min":{"1min":0.91600000000000004,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9650000000000001},"last":1.5629999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.3220000000000001,"15min":1.3049999999999999},"min":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.921,"5min":2.2799999999999998,"15min":2.625},"last":1.385},{"interface":"front","average":{"1min":1.337,"5min":1.351,"15min":1.359},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":2.1230000000000002,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.2290000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4359999999999999,"15min":1.4319999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":2.048,"5min":2.048,"15min":2.4820000000000002},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.492,"15min":1.4350000000000001},"min":{"1min":0.65500000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.0379999999999998,"5min":2.4700000000000002,"15min":2.5539999999999998},"last":1.2729999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:46:53 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.474,"15min":1.4419999999999999},"min":{"1min":0.85399999999999998,"5min":0.63,"15min":0.63},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.5860000000000001,"15min":1.5760000000000001},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":2.3580000000000001,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574950,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115490,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.24,"15min":1.177},"min":{"1min":0.80300000000000005,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.657,"5min":1.657,"15min":1.8160000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2709999999999999,"15min":1.1879999999999999},"min":{"1min":0.878,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":1.5629999999999999,"5min":1.976,"15min":7.367},"last":1.325}]},{"osd":1,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.173,"15min":1.1299999999999999},"min":{"1min":0.73899999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"max":{"1min":1.6240000000000001,"5min":2.032,"15min":2.032},"last":1.133},{"interface":"front","average":{"1min":1.22,"5min":1.2350000000000001,"15min":1.3560000000000001},"min":{"1min":0.77100000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.5940000000000001,"5min":1.954,"15min":8.5850000000000009},"last":1.0980000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.2010000000000001,"15min":1.268},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.76800000000000002},"max":{"1min":1.708,"5min":1.708,"15min":4.1669999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3100000000000001,"15min":1.2},"min":{"1min":0.83199999999999996,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5589999999999999,"5min":1.6799999999999999,"15min":1.8520000000000001},"last":1.2}]},{"osd":3,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2150000000000001,"15min":1.2350000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.673,"5min":1.764,"15min":7.2560000000000002},"last":1.468},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.377},"min":{"1min":1.0329999999999999,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.899,"5min":1.899,"15min":1.911},"last":1.2290000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.2969999999999999,"15min":1.2949999999999999},"min":{"1min":0.64800000000000002,"5min":0.56000000000000005,"15min":0.55200000000000005},"max":{"1min":1.7230000000000001,"5min":2.1850000000000001,"15min":2.4420000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.302,"15min":1.2709999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.802,"5min":1.9299999999999999,"15min":8.2409999999999997},"last":1.516}]},{"osd":6,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.4730000000000001,"15min":1.4399999999999999},"min":{"1min":0.84999999999999998,"5min":0.68400000000000005,"15min":0.52000000000000002},"max":{"1min":2.02,"5min":3.5880000000000001,"15min":3.5880000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5980000000000001,"15min":1.569},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.9299999999999999,"5min":3.5270000000000001,"15min":4.0229999999999997},"last":1.444}]},{"osd":7,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.508,"15min":1.4419999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.63800000000000001},"max":{"1min":1.792,"5min":2.3570000000000002,"15min":8.1760000000000002},"last":1.409},{"interface":"front","average":{"1min":1.6120000000000001,"5min":1.6419999999999999,"15min":1.601},"min":{"1min":0.96799999999999997,"5min":0.80900000000000005,"15min":0.77500000000000002},"max":{"1min":2.1150000000000002,"5min":2.1819999999999999,"15min":2.7930000000000001},"last":1.375}]}]},{"osd":3,"up_from":13,"seq":55834574950,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115456,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.91600000000000004,"15min":0.85799999999999998},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.29299999999999998},"max":{"1min":1.579,"5min":1.6399999999999999,"15min":1.6830000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.0209999999999999,"15min":0.91700000000000004},"min":{"1min":0.51300000000000001,"5min":0.503,"15min":0.41699999999999998},"max":{"1min":1.486,"5min":1.631,"15min":1.631},"last":1.3300000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.88300000000000001,"15min":0.81699999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.35699999999999998},"max":{"1min":1.702,"5min":1.702,"15min":1.702},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.88400000000000001,"5min":1.004,"15min":0.97599999999999998},"min":{"1min":0.58799999999999997,"5min":0.52800000000000002,"15min":0.45700000000000002},"max":{"1min":1.5649999999999999,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":1.0029999999999999,"15min":0.89500000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.378},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.919},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.008,"5min":1.145,"15min":1.1120000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.44900000000000001},"max":{"1min":1.8120000000000001,"5min":2.0870000000000002,"15min":3.399},"last":1.264}]},{"osd":4,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.2370000000000001,"15min":1.1950000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.66800000000000004},"max":{"1min":1.538,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":1.431},{"interface":"front","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2010000000000001},"min":{"1min":0.84599999999999997,"5min":0.73299999999999998,"15min":0.65700000000000003},"max":{"1min":1.8600000000000001,"5min":1.984,"15min":2.1840000000000002},"last":1.385}]},{"osd":5,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.343,"15min":1.3149999999999999},"min":{"1min":0.91700000000000004,"5min":0.73599999999999999,"15min":0.71899999999999997},"max":{"1min":1.5429999999999999,"5min":2.407,"15min":2.407},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.3049999999999999,"15min":1.274},"min":{"1min":0.84899999999999998,"5min":0.83599999999999997,"15min":0.64500000000000002},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.556}]},{"osd":6,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.3080000000000001,"15min":1.2549999999999999},"min":{"1min":0.871,"5min":0.85199999999999998,"15min":0.73799999999999999},"max":{"1min":1.9279999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.488},{"interface":"front","average":{"1min":1.26,"5min":1.409,"15min":1.399},"min":{"1min":0.83199999999999996,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.7170000000000001,"5min":2.6699999999999999,"15min":3.496},"last":1.413}]},{"osd":7,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.403,"15min":1.3660000000000001},"min":{"1min":0.89400000000000002,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":2.2989999999999999},"last":1.881},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.413,"15min":1.319},"min":{"1min":0.84199999999999997,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":1.9159999999999999,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.6850000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574950,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":863,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199115546,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.83399999999999996},"min":{"1min":0.38400000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.706,"5min":1.9279999999999999,"15min":2.383},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.96099999999999997,"15min":1.028},"min":{"1min":0.41099999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.645,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":1.1240000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.016,"15min":0.99099999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.978,"5min":1.978,"15min":2.3180000000000001},"last":1.048},{"interface":"front","average":{"1min":0.94599999999999995,"5min":1.0389999999999999,"15min":1.048},"min":{"1min":0.47999999999999998,"5min":0.439,"15min":0.439},"max":{"1min":1.6830000000000001,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":1.0089999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":1.157,"15min":1.117},"min":{"1min":0.501,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6839999999999999,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.1040000000000001,"15min":1.0980000000000001},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.8999999999999999,"5min":2.145,"15min":2.2040000000000002},"last":1.2829999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.302,"15min":1.365},"min":{"1min":0.74199999999999999,"5min":0.73899999999999999,"15min":0.72099999999999997},"max":{"1min":2.0059999999999998,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2929999999999999,"15min":1.319},"min":{"1min":0.75700000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5609999999999999,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":1.3220000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.26,"15min":1.3420000000000001},"min":{"1min":0.73199999999999998,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":3.7559999999999998,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":1.101},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.3480000000000001,"15min":1.381},"min":{"1min":0.84899999999999998,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0920000000000001,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.4630000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4159999999999999,"15min":1.417},"min":{"1min":0.88300000000000001,"5min":0.755,"15min":0.73199999999999998},"max":{"1min":4.0170000000000003,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.4239999999999999,"15min":1.462},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":3.4140000000000001,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":1.1739999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:46:55 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.389,"15min":1.431},"min":{"1min":0.85499999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.3149999999999999,"5min":2.331,"15min":2.968},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.423,"15min":1.4810000000000001},"min":{"1min":0.76500000000000001,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.96,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.4930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1440,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193872,"statfs":{"total":200000000,"available":198525178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.86299999999999999,"15min":0.79000000000000004},"min":{"1min":0.36499999999999999,"5min":0.36399999999999999,"15min":0.29699999999999999},"max":{"1min":1.5600000000000001,"5min":1.5700000000000001,"15min":1.627},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.91700000000000004,"15min":0.81699999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2490000000000001,"5min":1.895,"15min":1.895},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.90800000000000003,"15min":0.82299999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.397,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.98599999999999999,"15min":0.92300000000000004},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.33600000000000002},"max":{"1min":1.2669999999999999,"5min":1.6220000000000001,"15min":1.746},"last":1.508}]},{"osd":3,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93999999999999995,"5min":1.1339999999999999,"15min":1.056},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.372},"max":{"1min":1.4950000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.103},{"interface":"front","average":{"1min":0.92500000000000004,"5min":1.042,"15min":0.96599999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.5389999999999999,"5min":1.6439999999999999,"15min":1.764},"last":1.0680000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.1739999999999999,"15min":1.1040000000000001},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.238},{"interface":"front","average":{"1min":1.169,"5min":1.2989999999999999,"15min":1.2350000000000001},"min":{"1min":0.81299999999999994,"5min":0.77700000000000002,"15min":0.629},"max":{"1min":1.5700000000000001,"5min":1.8160000000000001,"15min":2.008},"last":1.379}]},{"osd":5,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.238,"15min":1.2030000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.68600000000000005},"max":{"1min":1.7330000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.161},{"interface":"front","average":{"1min":1.177,"5min":1.2609999999999999,"15min":1.1919999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.61199999999999999},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.014}]},{"osd":6,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.415,"15min":1.3240000000000001},"min":{"1min":0.78900000000000003,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.7929999999999999,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.462},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.403,"15min":1.3160000000000001},"min":{"1min":0.88,"5min":0.82199999999999995,"15min":0.72099999999999997},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.409}]},{"osd":7,"last update":"Fri Aug 25 09:46:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.417,"15min":1.3700000000000001},"min":{"1min":0.96699999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.8100000000000001,"5min":1.968,"15min":2.9529999999999998},"last":1.214},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.4410000000000001,"15min":1.345},"min":{"1min":0.92200000000000004,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":2.0659999999999998,"5min":2.218,"15min":2.218},"last":1.331}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":864,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194448,"statfs":{"total":200000000,"available":199114860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":0.86399999999999999},"min":{"1min":0.36799999999999999,"5min":0.33900000000000002,"15min":0.313},"max":{"1min":1.3939999999999999,"5min":11.257,"15min":11.257},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.97699999999999998,"5min":1.0149999999999999,"15min":0.96199999999999997},"min":{"1min":0.58099999999999996,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":1.474,"5min":6.4260000000000002,"15min":6.4260000000000002},"last":1.246}]},{"osd":2,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":0.96899999999999997,"15min":0.94199999999999995},"min":{"1min":0.59999999999999998,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":1.4019999999999999,"5min":1.724,"15min":1.724},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.038,"5min":1.016,"15min":1.0800000000000001},"min":{"1min":0.498,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4259999999999999,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.86499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.92100000000000004,"15min":0.90500000000000003},"min":{"1min":0.437,"5min":0.42699999999999999,"15min":0.40699999999999997},"max":{"1min":1.625,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.121},"min":{"1min":0.55500000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6339999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.458}]},{"osd":4,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.272,"15min":1.24},"min":{"1min":0.82799999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.78,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.3759999999999999,"15min":1.357},"min":{"1min":0.95299999999999996,"5min":0.83299999999999996,"15min":0.66300000000000003},"max":{"1min":1.754,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.589}]},{"osd":5,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.2909999999999999,"15min":1.347},"min":{"1min":0.82499999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.9530000000000001,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.335},{"interface":"front","average":{"1min":1.25,"5min":1.276,"15min":1.294},"min":{"1min":0.89900000000000002,"5min":0.73699999999999999,"15min":0.66200000000000003},"max":{"1min":1.6040000000000001,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.1950000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.347,"15min":1.3440000000000001},"min":{"1min":0.86399999999999999,"5min":0.73699999999999999,"15min":0.73299999999999998},"max":{"1min":1.821,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.343,"15min":1.3080000000000001},"min":{"1min":0.76300000000000001,"5min":0.752,"15min":0.72699999999999998},"max":{"1min":2.2090000000000001,"5min":2.331,"15min":2.331},"last":1.429}]},{"osd":7,"last update":"Fri Aug 25 09:45:59 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4139999999999999,"15min":1.4019999999999999},"min":{"1min":0.94999999999999996,"5min":0.79000000000000004,"15min":0.71099999999999997},"max":{"1min":2.085,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.482},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.337,"15min":1.323},"min":{"1min":0.86899999999999999,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.1469999999999998,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":1.2969999999999999}]}]}],"pool_statfs":[]}} 2023-08-25T09:47:03.734 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:47:03.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:47:03.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.992+0000 7f2827838640 1 Processor -- start 2023-08-25T09:47:03.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.992+0000 7f2827838640 1 -- start start 2023-08-25T09:47:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f2827838640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 0x7f2820072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f2827838640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2820072d60 0x7f28200ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f2827838640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282015d340 con 0x7f28200b2870 2023-08-25T09:47:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f2827838640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28200731f0 con 0x7f2820072d60 2023-08-25T09:47:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f2827838640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2820154150 con 0x7f2820153d70 2023-08-25T09:47:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f28255d4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 0x7f2820072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.994+0000 7f2825dd5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2820072d60 0x7f28200ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f28255d4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 0x7f2820072820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46638/0 (socket says 172.21.15.70:46638) 2023-08-25T09:47:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f28255d4640 1 -- 172.21.15.70:0/1332737419 learned_addr learned my addr 172.21.15.70:0/1332737419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f2825dd5640 1 -- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 msgr2=0x7f2820072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f2825dd5640 1 --2- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 0x7f2820072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f2825dd5640 1 -- 172.21.15.70:0/1332737419 >> v1:172.21.15.155:6789/0 conn(0x7f28200b2870 legacy=0x7f28200acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f2825dd5640 1 -- 172.21.15.70:0/1332737419 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282015d620 con 0x7f2820072d60 2023-08-25T09:47:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.995+0000 7f28255d4640 1 --2- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 0x7f2820072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2825dd5640 1 --2- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f28200ad380 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2818009d90 tx=0x7f281802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8c13f2a4b481bf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2817fff640 1 -- 172.21.15.70:0/1332737419 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281802cb40 con 0x7f2820072d60 2023-08-25T09:47:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2817fff640 1 -- 172.21.15.70:0/1332737419 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281802cca0 con 0x7f2820072d60 2023-08-25T09:47:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2817fff640 1 -- 172.21.15.70:0/1332737419 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818034660 con 0x7f2820072d60 2023-08-25T09:47:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 -- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 msgr2=0x7f28200ad380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 --2- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f28200ad380 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2818009d90 tx=0x7f281802bd90 comp rx=0 tx=0).stop 2023-08-25T09:47:04.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 -- 172.21.15.70:0/1332737419 shutdown_connections 2023-08-25T09:47:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 --2- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f28200ad380 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 --2- 172.21.15.70:0/1332737419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2820153d70 0x7f2820072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 -- 172.21.15.70:0/1332737419 >> 172.21.15.70:0/1332737419 conn(0x7f28200cfc70 msgr2=0x7f28200d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 -- 172.21.15.70:0/1332737419 shutdown_connections 2023-08-25T09:47:04.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:03.998+0000 7f2827838640 1 -- 172.21.15.70:0/1332737419 wait complete. 2023-08-25T09:47:04.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.001+0000 7f2827838640 1 Processor -- start 2023-08-25T09:47:04.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.001+0000 7f2827838640 1 -- start start 2023-08-25T09:47:04.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2827838640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f282006bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2827838640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28200b2870 0x7f2820148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2827838640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 0x7f2820156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2827838640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201f5410 con 0x7f2820153d70 2023-08-25T09:47:04.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2827838640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28201f56f0 con 0x7f2820072d60 2023-08-25T09:47:04.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2827838640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28201f59d0 con 0x7f28200b2870 2023-08-25T09:47:04.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 0x7f2820156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 0x7f2820156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60374/0 (socket says 172.21.15.70:60374) 2023-08-25T09:47:04.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 -- 172.21.15.70:0/3693352382 learned_addr learned my addr 172.21.15.70:0/3693352382 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:04.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 -- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28200b2870 msgr2=0x7f2820148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2824dd3640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28200b2870 0x7f2820148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f28255d4640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f282006bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28200b2870 0x7f2820148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 -- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 msgr2=0x7f282006bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f282006bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 -- 172.21.15.70:0/3693352382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0094d0 con 0x7f2820153d70 2023-08-25T09:47:04.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2825dd5640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 0x7f2820156ad0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f2818009d90 tx=0x7f2818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:04.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818040650 con 0x7f2820153d70 2023-08-25T09:47:04.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28180407b0 con 0x7f2820153d70 2023-08-25T09:47:04.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.002+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2818040b60 con 0x7f2820153d70 2023-08-25T09:47:04.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.003+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2818009a70 con 0x7f2820153d70 2023-08-25T09:47:04.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.003+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28201f5f90 con 0x7f2820153d70 2023-08-25T09:47:04.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.004+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28200738b0 con 0x7f2820153d70 2023-08-25T09:47:04.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.005+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2818040cc0 con 0x7f2820153d70 2023-08-25T09:47:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.005+0000 7f2815ffb640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f280406a050 0x7f280406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.006+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28180ab4c0 con 0x7f2820153d70 2023-08-25T09:47:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.006+0000 7f28255d4640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f280406a050 0x7f280406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.006+0000 7f28255d4640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f280406a050 0x7f280406c510 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f281c009b70 tx=0x7f281c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:04.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.010+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2818076b10 con 0x7f2820153d70 2023-08-25T09:47:04.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.244+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2820066fb0 con 0x7f2820153d70 2023-08-25T09:47:04.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.252+0000 7f2815ffb640 1 -- 172.21.15.70:0/3693352382 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f281807a130 con 0x7f2820153d70 2023-08-25T09:47:04.255 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:47:04.255 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:47:04.255 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:47:04.255 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:47:04.256 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:47:04.257 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:47:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f280406a050 msgr2=0x7f280406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f280406a050 0x7f280406c510 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f281c009b70 tx=0x7f281c008040 comp rx=0 tx=0).stop 2023-08-25T09:47:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 msgr2=0x7f2820156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 0x7f2820156ad0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f2818009d90 tx=0x7f2818004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 shutdown_connections 2023-08-25T09:47:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f280406a050 0x7f280406c510 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2820153d70 0x7f2820156ad0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28200b2870 0x7f2820148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 --2- 172.21.15.70:0/3693352382 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2820072d60 0x7f282006bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 >> 172.21.15.70:0/3693352382 conn(0x7f28200cfc70 msgr2=0x7f28200d0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 shutdown_connections 2023-08-25T09:47:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.254+0000 7f2827838640 1 -- 172.21.15.70:0/3693352382 wait complete. 2023-08-25T09:47:04.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:04.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.481+0000 7fb8d5cee640 1 Processor -- start 2023-08-25T09:47:04.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.481+0000 7fb8d5cee640 1 -- start start 2023-08-25T09:47:04.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.482+0000 7fb8d5cee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015e830 0x7fb8d006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.482+0000 7fb8d5cee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8d015ec10 0x7fb8d00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.482+0000 7fb8d5cee640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d00b1b60 con 0x7fb8d006b2b0 2023-08-25T09:47:04.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8d5cee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d006bd80 con 0x7fb8d015ec10 2023-08-25T09:47:04.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8d5cee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d006b7c0 con 0x7fb8d015e830 2023-08-25T09:47:04.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8d015ec10 0x7fb8d00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8c7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb8d006b2b0 0x7fb8d015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56102/0 (socket says 172.21.15.70:56102) 2023-08-25T09:47:04.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8c7fff640 1 -- 172.21.15.70:0/3744741424 learned_addr learned my addr 172.21.15.70:0/3744741424 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:04.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cf7fe640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015e830 0x7fb8d006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cffff640 1 -- 172.21.15.70:0/3744741424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015e830 msgr2=0x7fb8d006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:04.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cffff640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015e830 0x7fb8d006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cffff640 1 -- 172.21.15.70:0/3744741424 >> v1:172.21.15.70:6789/0 conn(0x7fb8d006b2b0 legacy=0x7fb8d015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8ceffd640 1 -- 172.21.15.70:0/3744741424 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278649107 0 0) 0x7fb8d00b1b60 con 0x7fb8d006b2b0 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cffff640 1 -- 172.21.15.70:0/3744741424 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d006baa0 con 0x7fb8d015ec10 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.483+0000 7fb8cf7fe640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015e830 0x7fb8d006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.484+0000 7fb8cffff640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015ec10 0x7fb8d00b1ff0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009d90 tx=0x7fb8c802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3654be5691e60388 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8ceffd640 1 -- 172.21.15.70:0/3744741424 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c802cc50 con 0x7fb8d015ec10 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8ceffd640 1 -- 172.21.15.70:0/3744741424 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c802cdb0 con 0x7fb8d015ec10 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8ceffd640 1 -- 172.21.15.70:0/3744741424 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8034660 con 0x7fb8d015ec10 2023-08-25T09:47:04.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3744741424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015ec10 msgr2=0x7fb8d00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015ec10 0x7fb8d00b1ff0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009d90 tx=0x7fb8c802bea0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3744741424 shutdown_connections 2023-08-25T09:47:04.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015ec10 0x7fb8d00b1ff0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3744741424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015e830 0x7fb8d006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3744741424 >> 172.21.15.70:0/3744741424 conn(0x7fb8d0069850 msgr2=0x7fb8d0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:04.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3744741424 shutdown_connections 2023-08-25T09:47:04.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.485+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3744741424 wait complete. 2023-08-25T09:47:04.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.487+0000 7fb8d5cee640 1 Processor -- start 2023-08-25T09:47:04.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.487+0000 7fb8d5cee640 1 -- start start 2023-08-25T09:47:04.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8d5cee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8d006b2b0 0x7fb8d00a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8d5cee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 0x7fb8d00c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8d5cee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015ec10 0x7fb8d014fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8d5cee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00a99e0 con 0x7fb8d015e830 2023-08-25T09:47:04.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8d5cee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00a8d60 con 0x7fb8d006b2b0 2023-08-25T09:47:04.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8d5cee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00a9040 con 0x7fb8d015ec10 2023-08-25T09:47:04.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.488+0000 7fb8c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 0x7fb8d00c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 0x7fb8d00c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60396/0 (socket says 172.21.15.70:60396) 2023-08-25T09:47:04.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 -- 172.21.15.70:0/3570308209 learned_addr learned my addr 172.21.15.70:0/3570308209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:04.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 -- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015ec10 msgr2=0x7fb8d014fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015ec10 0x7fb8d014fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 -- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8d006b2b0 msgr2=0x7fb8d00a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8cf7fe640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8d006b2b0 0x7fb8d00a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8d006b2b0 0x7fb8d00a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 -- 172.21.15.70:0/3570308209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8009a70 con 0x7fb8d015e830 2023-08-25T09:47:04.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8c7fff640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 0x7fb8d00c9e10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c4b0 tx=0x7fb8b800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:04.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80074e0 con 0x7fb8d015e830 2023-08-25T09:47:04.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b8007660 con 0x7fb8d015e830 2023-08-25T09:47:04.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8004a20 con 0x7fb8d015e830 2023-08-25T09:47:04.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.489+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d015ca20 con 0x7fb8d015e830 2023-08-25T09:47:04.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.490+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d00a9600 con 0x7fb8d015e830 2023-08-25T09:47:04.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.491+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d00a9b20 con 0x7fb8d015e830 2023-08-25T09:47:04.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.492+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb8b8004b80 con 0x7fb8d015e830 2023-08-25T09:47:04.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.493+0000 7fb8ccff9640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb8b006a050 0x7fb8b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:04.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.493+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8b8087790 con 0x7fb8d015e830 2023-08-25T09:47:04.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.493+0000 7fb8cf7fe640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb8b006a050 0x7fb8b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:04.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.494+0000 7fb8cf7fe640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb8b006a050 0x7fb8b006c510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b1c30 tx=0x7fb8c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:04.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.497+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb8d00a9b20 con 0x7fb8d015e830 2023-08-25T09:47:04.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.709+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8d00a9b20 con 0x7fb8d015e830 2023-08-25T09:47:04.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.710+0000 7fb8ccff9640 1 -- 172.21.15.70:0/3570308209 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fb8d00a9b20 con 0x7fb8d015e830 2023-08-25T09:47:04.712 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:04.712 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:04.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb8b006a050 msgr2=0x7fb8b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb8b006a050 0x7fb8b006c510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00b1c30 tx=0x7fb8c0008040 comp rx=0 tx=0).stop 2023-08-25T09:47:04.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 msgr2=0x7fb8d00c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:04.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 0x7fb8d00c9e10 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c4b0 tx=0x7fb8b800c970 comp rx=0 tx=0).stop 2023-08-25T09:47:04.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 shutdown_connections 2023-08-25T09:47:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb8b006a050 0x7fb8b006c510 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb8d015ec10 0x7fb8d014fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8d015e830 0x7fb8d00c9e10 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 --2- 172.21.15.70:0/3570308209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8d006b2b0 0x7fb8d00a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:04.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.712+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 >> 172.21.15.70:0/3570308209 conn(0x7fb8d0069850 msgr2=0x7fb8d00b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:04.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.713+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 shutdown_connections 2023-08-25T09:47:04.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:04.713+0000 7fb8d5cee640 1 -- 172.21.15.70:0/3570308209 wait complete. 2023-08-25T09:47:05.135 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:46:58.267207+0000', 'modified': '2023-08-25T09:47:01.840742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8232': {'gid': 8232, 'name': 'd', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6834/1074652443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 1074652443}, {'type': 'v1', 'addr': '172.21.15.155:6834', 'nonce': 1074652443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:47:05.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:47:05.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-25T09:47:05.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.362+0000 7f2632796640 1 Processor -- start 2023-08-25T09:47:05.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.362+0000 7f2632796640 1 -- start start 2023-08-25T09:47:05.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.363+0000 7f2632796640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f262c159aa0 0x7f262c070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f2632796640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c0cace0 con 0x7f262c15df10 2023-08-25T09:47:05.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f2632796640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c06aec0 con 0x7f262c06bc80 2023-08-25T09:47:05.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f2632796640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c159e80 con 0x7f262c159aa0 2023-08-25T09:47:05.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f262c159aa0 0x7f262c070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f262c159aa0 0x7f262c070410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60400/0 (socket says 172.21.15.70:60400) 2023-08-25T09:47:05.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262bfff640 1 -- 172.21.15.70:0/3646525816 learned_addr learned my addr 172.21.15.70:0/3646525816 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262bfff640 1 -- 172.21.15.70:0/3646525816 >> v1:172.21.15.155:6789/0 conn(0x7f262c06bc80 legacy=0x7f262c0c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262bfff640 1 -- 172.21.15.70:0/3646525816 >> v1:172.21.15.70:6789/0 conn(0x7f262c15df10 legacy=0x7f262c0ca5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262bfff640 1 -- 172.21.15.70:0/3646525816 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c06b1a0 con 0x7f262c159aa0 2023-08-25T09:47:05.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.364+0000 7f262affd640 1 -- 172.21.15.70:0/3646525816 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494510643 0 0) 0x7f262c0cace0 con 0x7f262c15df10 2023-08-25T09:47:05.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.365+0000 7f262bfff640 1 --2- 172.21.15.70:0/3646525816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c070410 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f261c009850 tx=0x7f261c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8082d8c39ac10260 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f262affd640 1 -- 172.21.15.70:0/3646525816 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c004230 con 0x7f262c159aa0 2023-08-25T09:47:05.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f262affd640 1 -- 172.21.15.70:0/3646525816 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c004390 con 0x7f262c159aa0 2023-08-25T09:47:05.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f262affd640 1 -- 172.21.15.70:0/3646525816 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c030840 con 0x7f262c159aa0 2023-08-25T09:47:05.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 -- 172.21.15.70:0/3646525816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 msgr2=0x7f262c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 --2- 172.21.15.70:0/3646525816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c070410 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f261c009850 tx=0x7f261c02b980 comp rx=0 tx=0).stop 2023-08-25T09:47:05.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 -- 172.21.15.70:0/3646525816 shutdown_connections 2023-08-25T09:47:05.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 --2- 172.21.15.70:0/3646525816 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c070410 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 -- 172.21.15.70:0/3646525816 >> 172.21.15.70:0/3646525816 conn(0x7f262c0ce760 msgr2=0x7f262c0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 -- 172.21.15.70:0/3646525816 shutdown_connections 2023-08-25T09:47:05.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.366+0000 7f2632796640 1 -- 172.21.15.70:0/3646525816 wait complete. 2023-08-25T09:47:05.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.368+0000 7f2632796640 1 Processor -- start 2023-08-25T09:47:05.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.368+0000 7f2632796640 1 -- start start 2023-08-25T09:47:05.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f2632796640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f2632796640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f262c151ae0 0x7f262c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f2632796640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f262c0c8260 0x7f262c157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f2632796640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c152c70 con 0x7f262c159aa0 2023-08-25T09:47:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f2632796640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c155c20 con 0x7f262c0c8260 2023-08-25T09:47:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f2632796640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c155f00 con 0x7f262c151ae0 2023-08-25T09:47:05.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f262bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f262bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c14f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60402/0 (socket says 172.21.15.70:60402) 2023-08-25T09:47:05.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.369+0000 7f262bfff640 1 -- 172.21.15.70:0/3344822857 learned_addr learned my addr 172.21.15.70:0/3344822857 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f262bfff640 1 -- 172.21.15.70:0/3344822857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f262c151ae0 msgr2=0x7f262c156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f262bfff640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f262c151ae0 0x7f262c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f262bfff640 1 -- 172.21.15.70:0/3344822857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f262c0c8260 msgr2=0x7f262c157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f262bfff640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f262c0c8260 0x7f262c157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f262bfff640 1 -- 172.21.15.70:0/3344822857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c009460 con 0x7f262c159aa0 2023-08-25T09:47:05.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f262bfff640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c14f290 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f261c0090b0 tx=0x7f261c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f2628ff9640 1 -- 172.21.15.70:0/3344822857 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c03e070 con 0x7f262c159aa0 2023-08-25T09:47:05.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f2628ff9640 1 -- 172.21.15.70:0/3344822857 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c004500 con 0x7f262c159aa0 2023-08-25T09:47:05.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f2628ff9640 1 -- 172.21.15.70:0/3344822857 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c030e00 con 0x7f262c159aa0 2023-08-25T09:47:05.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.370+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c1561e0 con 0x7f262c159aa0 2023-08-25T09:47:05.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.371+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c1567a0 con 0x7f262c159aa0 2023-08-25T09:47:05.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.374+0000 7f2628ff9640 1 -- 172.21.15.70:0/3344822857 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f261c003610 con 0x7f262c159aa0 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.374+0000 7f2628ff9640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f261806a050 0x7f261806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.375+0000 7f2628ff9640 1 -- 172.21.15.70:0/3344822857 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f261c0aa3a0 con 0x7f262c159aa0 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.375+0000 7f262b7fe640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f261806a050 0x7f261806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.375+0000 7f26353e7740 1 Processor -- start 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.375+0000 7f26353e7740 1 -- start start 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.375+0000 7f262b7fe640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f261806a050 0x7f261806c510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f262c15ced0 tx=0x7f26200079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a8e80 0x56430d18af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f26353e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 0x56430d18b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d188590 0x56430d15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f26353e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d0ce190 con 0x56430d1a69a0 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d0cb880 con 0x56430d188590 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x56430d0c7170 con 0x56430d1a8e80 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 0x56430d18b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.376+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 0x56430d18b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60418/0 (socket says 172.21.15.70:60418) 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 -- 172.21.15.70:0/64456565 learned_addr learned my addr 172.21.15.70:0/64456565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 -- 172.21.15.70:0/64456565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a8e80 msgr2=0x56430d18af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a8e80 0x56430d18af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 -- 172.21.15.70:0/64456565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d188590 msgr2=0x56430d15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d188590 0x56430d15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 -- 172.21.15.70:0/64456565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56430d0c57f0 con 0x56430d1a69a0 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f262b7fe640 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 0x56430d18b4a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f26200054e0 tx=0x7f262000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78edda7c2b90a1c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f2611ffb640 1 -- 172.21.15.70:0/64456565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620005060 con 0x56430d1a69a0 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f2611ffb640 1 -- 172.21.15.70:0/64456565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620017850 con 0x56430d1a69a0 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.377+0000 7f2611ffb640 1 -- 172.21.15.70:0/64456565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620017c00 con 0x56430d1a69a0 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 -- 172.21.15.70:0/64456565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 msgr2=0x56430d18b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 0x56430d18b4a0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f26200054e0 tx=0x7f262000f000 comp rx=0 tx=0).stop 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 -- 172.21.15.70:0/64456565 shutdown_connections 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d188590 0x56430d15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a69a0 0x56430d18b4a0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 --2- 172.21.15.70:0/64456565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a8e80 0x56430d18af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 -- 172.21.15.70:0/64456565 >> 172.21.15.70:0/64456565 conn(0x56430d15d3e0 msgr2=0x56430d1a5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 -- 172.21.15.70:0/64456565 shutdown_connections 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.378+0000 7f26353e7740 1 -- 172.21.15.70:0/64456565 wait complete. 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.379+0000 7f26353e7740 1 -- start start 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.379+0000 7f26353e7740 1 Processor -- start 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d1837c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d189e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d183ec0 0x56430d18a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d066d20 con 0x56430d1a8e80 2023-08-25T09:47:05.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d0669c0 con 0x56430d183ec0 2023-08-25T09:47:05.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x56430ce2cf20 con 0x56430d1a69a0 2023-08-25T09:47:05.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f26353e7740 10 client.0 _ll_register_callbacks cb 0x56430d07e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:47:05.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d189e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f262b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d189e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60420/0 (socket says 172.21.15.70:60420) 2023-08-25T09:47:05.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.380+0000 7f262b7fe640 1 -- 172.21.15.70:0/1640596337 learned_addr learned my addr 172.21.15.70:0/1640596337 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262b7fe640 1 -- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 msgr2=0x56430d1837c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f2630d33640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d183ec0 0x56430d18a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262bfff640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d1837c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262b7fe640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d1837c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262b7fe640 1 -- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d183ec0 msgr2=0x56430d18a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262b7fe640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d183ec0 0x56430d18a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262b7fe640 1 -- 172.21.15.70:0/1640596337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26200051c0 con 0x56430d1a8e80 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f262b7fe640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d189e90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2620016000 tx=0x7f26200037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3883543f0a13547a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f260affd640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:47:05.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f260affd640 1 -- 172.21.15.70:0/1640596337 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2620027070 con 0x56430d1a8e80 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f260affd640 1 -- 172.21.15.70:0/1640596337 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f262002dc70 con 0x56430d1a8e80 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.381+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56430d09b540 con 0x56430d1a8e80 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f260affd640 1 -- 172.21.15.70:0/1640596337 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f262002de30 con 0x56430d1a8e80 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 10 client.8373 fetch_fsmap learned FSMap version 65 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x56430d0bef30 con 0x56430d1a8e80 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f260affd640 1 -- 172.21.15.70:0/1640596337 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56430d09b540 con 0x56430d1a8e80 2023-08-25T09:47:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f260affd640 1 -- 172.21.15.70:0/1640596337 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2620007030 con 0x56430d1a8e80 2023-08-25T09:47:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 10 client.8373 fetch_fsmap finished waiting for FSMap version 65 2023-08-25T09:47:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 10 client.8373 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-25T09:47:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 20 client.8373 populate_metadata read hostname 'smithi070' 2023-08-25T09:47:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d18c470 0x56430d18c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 4 client.8373 mds_command: new command op to 8232 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:47:05.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.382+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 --> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56430d0c0150 con 0x56430d18c470 2023-08-25T09:47:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.383+0000 7f262bfff640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d18c470 0x56430d18c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.384+0000 7f262bfff640 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d18c470 0x56430d18c850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a052a73ea6b687 server_cookie=e76231de21eac3a6 in_seq=0 out_seq=0 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.384+0000 7f260affd640 10 client.8373 ms_handle_connect on v2:172.21.15.155:6832/1074652443 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.385+0000 7f260affd640 1 -- 172.21.15.70:0/1640596337 <== mds.0 v2:172.21.15.155:6832/1074652443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56430d0c0150 con 0x56430d18c470 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.385+0000 7f260affd640 10 client.8373 handle_command_reply: tid=0 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.385+0000 7f26353e7740 1 client.8373 shutdown 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 msgr2=0x56430d189e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d189e90 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2620016000 tx=0x7f26200037d0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 shutdown_connections 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d18c470 0x56430d18c850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d183ec0 0x56430d18a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d189e90 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f260affd640 0 client.8373 ms_handle_reset on v2:172.21.15.155:6832/1074652443 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 --2- 172.21.15.70:0/1640596337 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d1837c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 >> 172.21.15.70:0/1640596337 conn(0x56430d188590 msgr2=0x56430d15a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 shutdown_connections 2023-08-25T09:47:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 1 -- 172.21.15.70:0/1640596337 wait complete. 2023-08-25T09:47:05.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.386+0000 7f26353e7740 20 client.8373 trim_cache size 0 max 16384 2023-08-25T09:47:05.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.402+0000 7f26353e7740 1 Processor -- start 2023-08-25T09:47:05.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.402+0000 7f26353e7740 1 -- start start 2023-08-25T09:47:05.405 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f26353e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 0x56430d18a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d18aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a8e80 0x56430d18d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f26353e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d156a00 con 0x56430d183ec0 2023-08-25T09:47:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d18a7c0 con 0x56430d1a8e80 2023-08-25T09:47:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x56430d1a5bc0 con 0x56430d1a69a0 2023-08-25T09:47:05.406 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f262bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 0x56430d18a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f262bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 0x56430d18a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60428/0 (socket says 172.21.15.70:60428) 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.403+0000 7f262bfff640 1 -- 172.21.15.70:0/118693561 learned_addr learned my addr 172.21.15.70:0/118693561 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f262bfff640 1 -- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 msgr2=0x56430d18aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f262bfff640 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d18aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f2630d33640 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a8e80 0x56430d18d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f262bfff640 1 -- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a8e80 msgr2=0x56430d18d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f262bfff640 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a8e80 0x56430d18d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f262bfff640 1 -- 172.21.15.70:0/118693561 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56430d065a10 con 0x56430d183ec0 2023-08-25T09:47:05.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f262bfff640 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 0x56430d18a280 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f261c0796b0 tx=0x7f261c04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375afe9c90986ad1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f26127fc640 1 -- 172.21.15.70:0/118693561 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c077bb0 con 0x56430d183ec0 2023-08-25T09:47:05.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f26127fc640 1 -- 172.21.15.70:0/118693561 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261c0aa0b0 con 0x56430d183ec0 2023-08-25T09:47:05.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.404+0000 7f26127fc640 1 -- 172.21.15.70:0/118693561 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261c0aa220 con 0x56430d183ec0 2023-08-25T09:47:05.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 -- 172.21.15.70:0/118693561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 msgr2=0x56430d18a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 0x56430d18a280 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f261c0796b0 tx=0x7f261c04a000 comp rx=0 tx=0).stop 2023-08-25T09:47:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 -- 172.21.15.70:0/118693561 shutdown_connections 2023-08-25T09:47:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a8e80 0x56430d18d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d1a69a0 0x56430d18aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 --2- 172.21.15.70:0/118693561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d183ec0 0x56430d18a280 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 -- 172.21.15.70:0/118693561 >> 172.21.15.70:0/118693561 conn(0x56430d188590 msgr2=0x56430d1835d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 -- 172.21.15.70:0/118693561 shutdown_connections 2023-08-25T09:47:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 -- 172.21.15.70:0/118693561 wait complete. 2023-08-25T09:47:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.405+0000 7f26353e7740 1 -- start start 2023-08-25T09:47:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.406+0000 7f26353e7740 1 Processor -- start 2023-08-25T09:47:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d183ec0 0x56430d215f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a69a0 0x56430d1a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d1a8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d1891c0 con 0x56430d1a8e80 2023-08-25T09:47:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x56430d1add60 con 0x56430d1a69a0 2023-08-25T09:47:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x56430d15c840 con 0x56430d183ec0 2023-08-25T09:47:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f26353e7740 10 client.0 _ll_register_callbacks cb 0x56430d160eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:47:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d1a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d1a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60432/0 (socket says 172.21.15.70:60432) 2023-08-25T09:47:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 -- 172.21.15.70:0/1768251609 learned_addr learned my addr 172.21.15.70:0/1768251609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f262b7fe640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a69a0 0x56430d1a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f262bfff640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d183ec0 0x56430d215f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 -- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d183ec0 msgr2=0x56430d215f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d183ec0 0x56430d215f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 -- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a69a0 msgr2=0x56430d1a7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a69a0 0x56430d1a7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.407+0000 7f2630d33640 1 -- 172.21.15.70:0/1768251609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c1567a0 con 0x56430d1a8e80 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f2630d33640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d1a8480 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2624002160 tx=0x7f262400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2808842fee1f096 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f260affd640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:47:05.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f260affd640 1 -- 172.21.15.70:0/1768251609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624003b70 con 0x56430d1a8e80 2023-08-25T09:47:05.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f260affd640 1 -- 172.21.15.70:0/1768251609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2624003cd0 con 0x56430d1a8e80 2023-08-25T09:47:05.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2620017850 con 0x56430d1a8e80 2023-08-25T09:47:05.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f260affd640 1 -- 172.21.15.70:0/1768251609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2624003e90 con 0x56430d1a8e80 2023-08-25T09:47:05.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f26353e7740 10 client.8379 fetch_fsmap learned FSMap version 65 2023-08-25T09:47:05.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x56430d08ade0 con 0x56430d1a8e80 2023-08-25T09:47:05.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f260affd640 1 -- 172.21.15.70:0/1768251609 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2620017850 con 0x56430d1a8e80 2023-08-25T09:47:05.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.408+0000 7f260affd640 1 -- 172.21.15.70:0/1768251609 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f26240050d0 con 0x56430d1a8e80 2023-08-25T09:47:05.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f26353e7740 10 client.8379 fetch_fsmap finished waiting for FSMap version 65 2023-08-25T09:47:05.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f26353e7740 10 client.8379 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-25T09:47:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f26353e7740 20 client.8379 populate_metadata read hostname 'smithi070' 2023-08-25T09:47:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f26353e7740 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d1a75a0 0x56430d1aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f26353e7740 4 client.8379 mds_command: new command op to 8232 tid=0[{"prefix": "status"}] 2023-08-25T09:47:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 --> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56430cd765e0 con 0x56430d1a75a0 2023-08-25T09:47:05.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.409+0000 7f262bfff640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d1a75a0 0x56430d1aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.410+0000 7f262bfff640 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d1a75a0 0x56430d1aca20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9dd3499ec11118 server_cookie=8437501685fafd2b in_seq=0 out_seq=0 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.410+0000 7f260affd640 10 client.8379 ms_handle_connect on v2:172.21.15.155:6832/1074652443 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f260affd640 1 -- 172.21.15.70:0/1768251609 <== mds.0 v2:172.21.15.155:6832/1074652443 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56430cd765e0 con 0x56430d1a75a0 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f260affd640 10 client.8379 handle_command_reply: tid=0 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 client.8379 shutdown 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 msgr2=0x56430d1a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d1a8480 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2624002160 tx=0x7f262400e590 comp rx=0 tx=0).stop 2023-08-25T09:47:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 shutdown_connections 2023-08-25T09:47:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:6832/1074652443,v1:172.21.15.155:6834/1074652443] conn(0x56430d1a75a0 0x56430d1aca20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x56430d1a8e80 0x56430d1a8480 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x56430d1a69a0 0x56430d1a7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 --2- 172.21.15.70:0/1768251609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x56430d183ec0 0x56430d215f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f260affd640 0 client.8379 ms_handle_reset on v2:172.21.15.155:6832/1074652443 2023-08-25T09:47:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 >> 172.21.15.70:0/1768251609 conn(0x56430d188590 msgr2=0x56430d15be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 shutdown_connections 2023-08-25T09:47:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 1 -- 172.21.15.70:0/1768251609 wait complete. 2023-08-25T09:47:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.411+0000 7f26353e7740 20 client.8379 trim_cache size 0 max 16384 2023-08-25T09:47:05.429 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:05.430 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:05.430 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:47:05.430 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8232, 2023-08-25T09:47:05.430 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:47:05.430 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:47:05.431 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:47:05.431 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.5896603000000002, 2023-08-25T09:47:05.431 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 65, 2023-08-25T09:47:05.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 150, 2023-08-25T09:47:05.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 150, 2023-08-25T09:47:05.431 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.988076591 2023-08-25T09:47:05.432 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f261806a050 msgr2=0x7f261806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f261806a050 0x7f261806c510 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f262c15ced0 tx=0x7f26200079b0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 msgr2=0x7f262c14f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c14f290 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f261c0090b0 tx=0x7f261c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 shutdown_connections 2023-08-25T09:47:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f261806a050 0x7f261806c510 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f262c0c8260 0x7f262c157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f262c151ae0 0x7f262c156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 --2- 172.21.15.70:0/3344822857 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f262c159aa0 0x7f262c14f290 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 >> 172.21.15.70:0/3344822857 conn(0x7f262c0ce760 msgr2=0x7f262c15c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 shutdown_connections 2023-08-25T09:47:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.412+0000 7f2632796640 1 -- 172.21.15.70:0/3344822857 wait complete. 2023-08-25T09:47:05.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:47:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.698+0000 7fc9acae0640 1 Processor -- start 2023-08-25T09:47:05.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.698+0000 7fc9acae0640 1 -- start start 2023-08-25T09:47:05.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.699+0000 7fc9acae0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 0x7fc9a8072ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.699+0000 7fc9acae0640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc9a815af50 0x7fc9a815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.699+0000 7fc9acae0640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a806bd60 con 0x7fc9a8158480 2023-08-25T09:47:05.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9acae0640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a80731e0 con 0x7fc9a815af50 2023-08-25T09:47:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9acae0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8158860 con 0x7fc9a8072820 2023-08-25T09:47:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9a77fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc9a8158480 0x7fc9a806cab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:56120/0 (socket says 172.21.15.70:56120) 2023-08-25T09:47:05.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9a77fe640 1 -- 172.21.15.70:0/3749414828 learned_addr learned my addr 172.21.15.70:0/3749414828 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:05.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9a7fff640 1 --2- 172.21.15.70:0/3749414828 >> v2:172.21.15.155:3300/0 conn(0x7fc9a815af50 0x7fc9a815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 0x7fc9a8072ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283528897 0 0) 0x7fc9a806bd60 con 0x7fc9a8158480 2023-08-25T09:47:05.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.700+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc990003520 con 0x7fc9a8158480 2023-08-25T09:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 699438318 0 0) 0x7fc990003520 con 0x7fc9a8158480 2023-08-25T09:47:05.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 msgr2=0x7fc9a8072ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 0x7fc9a8072ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 >> v2:172.21.15.155:3300/0 conn(0x7fc9a815af50 msgr2=0x7fc9a815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 --2- 172.21.15.70:0/3749414828 >> v2:172.21.15.155:3300/0 conn(0x7fc9a815af50 0x7fc9a815b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a806bfd0 con 0x7fc9a8158480 2023-08-25T09:47:05.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc998001670 con 0x7fc9a8158480 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 >> v1:172.21.15.70:6789/0 conn(0x7fc9a8158480 legacy=0x7fc9a806cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc990003970 0x7fc990003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9900047c0 0x7fc990004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 0x7fc990005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80731e0 con 0x7fc9900047c0 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8158860 con 0x7fc990003970 2023-08-25T09:47:05.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99000ba30 con 0x7fc9a8072820 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.701+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9900047c0 0x7fc990004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a77fe640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc990003970 0x7fc990003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a7fff640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 0x7fc990005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 msgr2=0x7fc990005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a8072820 0x7fc990005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc990003970 msgr2=0x7fc990003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:05.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc990003970 0x7fc990003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.702+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/3749414828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990003520 con 0x7fc9900047c0 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9900047c0 0x7fc990004c40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009fc0 tx=0x7fc9900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70da676de210ebcd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c02cc20 con 0x7fc9900047c0 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c00f040 con 0x7fc9900047c0 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9a67fc640 1 -- 172.21.15.70:0/3749414828 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c034440 con 0x7fc9900047c0 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9acae0640 1 -- 172.21.15.70:0/3749414828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9900047c0 msgr2=0x7fc990004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9acae0640 1 --2- 172.21.15.70:0/3749414828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9900047c0 0x7fc990004c40 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009fc0 tx=0x7fc9900057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9a7fff640 1 -- 172.21.15.70:0/3749414828 reap_dead start 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9acae0640 1 -- 172.21.15.70:0/3749414828 shutdown_connections 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.703+0000 7fc9acae0640 1 -- 172.21.15.70:0/3749414828 >> 172.21.15.70:0/3749414828 conn(0x7fc9a80cc0a0 msgr2=0x7fc9a80cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.704+0000 7fc9acae0640 1 -- 172.21.15.70:0/3749414828 shutdown_connections 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.704+0000 7fc9acae0640 1 -- 172.21.15.70:0/3749414828 wait complete. 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.705+0000 7fc9acae0640 1 Processor -- start 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.705+0000 7fc9acae0640 1 -- start start 2023-08-25T09:47:05.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.706+0000 7fc9acae0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9900047c0 0x7fc9a80b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.706+0000 7fc9acae0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 0x7fc9a80b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.706+0000 7fc9acae0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a80b1730 0x7fc9a81f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.706+0000 7fc9acae0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80b1bb0 con 0x7fc9a81545b0 2023-08-25T09:47:05.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.706+0000 7fc9acae0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80b2670 con 0x7fc9a80b1730 2023-08-25T09:47:05.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.706+0000 7fc9acae0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81f2c30 con 0x7fc9900047c0 2023-08-25T09:47:05.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 0x7fc9a80b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 0x7fc9a80b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60448/0 (socket says 172.21.15.70:60448) 2023-08-25T09:47:05.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/4052213872 learned_addr learned my addr 172.21.15.70:0/4052213872 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:05.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9900047c0 msgr2=0x7fc9a80b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a77fe640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9900047c0 0x7fc9a80b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9900047c0 0x7fc9a80b5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a80b1730 msgr2=0x7fc9a81f1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a80b1730 0x7fc9a81f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 -- 172.21.15.70:0/4052213872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a81f2f10 con 0x7fc9a81545b0 2023-08-25T09:47:05.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.707+0000 7fc9a6ffd640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 0x7fc9a80b11f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00e230 tx=0x7fc99c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.708+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c034b50 con 0x7fc9a81545b0 2023-08-25T09:47:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.708+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c034cb0 con 0x7fc9a81545b0 2023-08-25T09:47:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.708+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c003bb0 con 0x7fc9a81545b0 2023-08-25T09:47:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.708+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a80b4850 con 0x7fc9a81545b0 2023-08-25T09:47:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.708+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a80b4e10 con 0x7fc9a81545b0 2023-08-25T09:47:05.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.709+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a80b46b0 con 0x7fc9a81545b0 2023-08-25T09:47:05.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.710+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc99c040450 con 0x7fc9a81545b0 2023-08-25T09:47:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.711+0000 7fc98ffff640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99406a050 0x7fc99406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.711+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc99c0aacd0 con 0x7fc9a81545b0 2023-08-25T09:47:05.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.711+0000 7fc9a77fe640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99406a050 0x7fc99406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:05.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.712+0000 7fc9a77fe640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99406a050 0x7fc99406c510 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc9980058b0 tx=0x7fc998000fe0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:05.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.715+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc9a80b46b0 con 0x7fc9a81545b0 2023-08-25T09:47:05.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.950+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9a80b46b0 con 0x7fc9a81545b0 2023-08-25T09:47:05.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.951+0000 7fc98ffff640 1 -- 172.21.15.70:0/4052213872 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc9a80b46b0 con 0x7fc9a81545b0 2023-08-25T09:47:05.953 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:47:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99406a050 msgr2=0x7fc99406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99406a050 0x7fc99406c510 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc9980058b0 tx=0x7fc998000fe0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 msgr2=0x7fc9a80b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:05.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 0x7fc9a80b11f0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00e230 tx=0x7fc99c009750 comp rx=0 tx=0).stop 2023-08-25T09:47:05.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 shutdown_connections 2023-08-25T09:47:05.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc99406a050 0x7fc99406c510 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a80b1730 0x7fc9a81f1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a81545b0 0x7fc9a80b11f0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 --2- 172.21.15.70:0/4052213872 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9900047c0 0x7fc9a80b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:05.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.953+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 >> 172.21.15.70:0/4052213872 conn(0x7fc9a80cc0a0 msgr2=0x7fc9a80c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:05.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.954+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 shutdown_connections 2023-08-25T09:47:05.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:05.954+0000 7fc9acae0640 1 -- 172.21.15.70:0/4052213872 wait complete. 2023-08-25T09:47:05.975 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:47:05.976 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:47:05.976 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:47:05.977 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:47:05.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:05.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:47:05.978 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:47:05.978 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:47:05.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:47:05.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:05.980 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:47:06.032 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:47:06.032 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:47:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:47:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:47:06.033 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.034 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:47:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:47:06.034 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:47:06.034 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:47:06.035 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:47:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:06.035 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:47:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:06.035 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:47:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41850sec preferred_lft 41850sec 2023-08-25T09:47:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:47:06.038 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.039 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:47:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:47:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:47:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.040 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:47:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:47:06.040 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:47:06.041 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:06.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:06.043 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:47:06.137 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:47:06.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:06.142 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:47:06.200 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:47:06.200 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:47:06.200 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:47:06.201 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:47:06.201 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:47:06.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:06.202 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:47:06.202 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:47:06.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:47:06.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:47:06.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:47:06.204 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:47:06.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:06.392 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:47:06.448 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:47:06.448 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:47:06.449 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:47:06.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:06.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:47:06.543 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:47:06.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:47:06.545 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:06.602 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:47:06.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:47:06.603 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:06.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:47:06.757 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:47:06.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:47:06.858 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:47:06.861 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:47:06.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:06.863 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:47:06.917 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:47:06.918 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:47:06.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:47:07.008 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:47:07.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:47:07.084 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:47:07.086 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:47:07.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:07.088 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:47:07.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:47:07.122+0000 7f212cc1c480 -1 init, newargv = 0x556ffb1ba040 newargc=15 2023-08-25T09:47:07.124 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[117333]: starting ceph client 2023-08-25T09:47:07.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[117333]: starting fuse 2023-08-25T09:47:08.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:47:08.342 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:47:08.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:47:08.445 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:47:08.449 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:47:08.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:08.450 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:47:08.507 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:47:08.508 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:47:08.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:47:08.510 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:47:08.510 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:47:08.511 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:47:08.511 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:47:08.511 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:47:08.512 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:47:08.512 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:47:08.512 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:47:08.513 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:47:08.513 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:47:08.513 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:47:08.513 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:47:08.514 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:47:08.514 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:47:08.514 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:47:08.515 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:47:08.515 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:47:08.516 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:47:08.516 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:47:08.516 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:47:08.517 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:47:08.517 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:47:08.517 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:47:08.518 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:47:08.518 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:47:08.518 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:47:08.702 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.117333.asok 2023-08-25T09:47:08.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:47:08.719 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.117333.asok 2023-08-25T09:47:08.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:08.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.117333.asok status 2023-08-25T09:47:08.961 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:47:08.962 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "117333", 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8391, 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:47:08.963 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 8391 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 778141390 2023-08-25T09:47:08.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 778141390 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:08.965 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.8391 192.168.0.1:0/778141390", 2023-08-25T09:47:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/778141390", 2023-08-25T09:47:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:47:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 65, 2023-08-25T09:47:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 150, 2023-08-25T09:47:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:47:08.966 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:47:08.967 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:47:08.967 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:08.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:08.976 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:47:08.977 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:09.033 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:47:09.033 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:09.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:09.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:09.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:47:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.347+0000 7f31d5006640 1 Processor -- start 2023-08-25T09:47:09.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.347+0000 7f31d5006640 1 -- start start 2023-08-25T09:47:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.348+0000 7f31d5006640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f31d006b180 0x7f31d0070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.348+0000 7f31d5006640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00ce4d0 0x7f31d0071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31d5006640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d0160c60 con 0x7f31d00b2ea0 2023-08-25T09:47:09.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31d5006640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d0161850 con 0x7f31d006b180 2023-08-25T09:47:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31d5006640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0160eb0 con 0x7f31d00ce4d0 2023-08-25T09:47:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31ced9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f31d00b2ea0 0x7f31d006aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42680/0 (socket says 172.21.15.70:42680) 2023-08-25T09:47:09.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31ced9d640 1 -- 172.21.15.70:0/3950440050 learned_addr learned my addr 172.21.15.70:0/3950440050 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3950440050 >> v2:172.21.15.70:3300/0 conn(0x7f31d006b180 0x7f31d0070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31cf59e640 1 --2- 172.21.15.70:0/3950440050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00ce4d0 0x7f31d0071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31cdd9b640 1 -- 172.21.15.70:0/3950440050 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805619353 0 0) 0x7f31d0160c60 con 0x7f31d00b2ea0 2023-08-25T09:47:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.349+0000 7f31cdd9b640 1 -- 172.21.15.70:0/3950440050 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b8003520 con 0x7f31d00b2ea0 2023-08-25T09:47:09.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.350+0000 7f31ce59c640 1 -- 172.21.15.70:0/3950440050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00ce4d0 msgr2=0x7f31d0071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.350+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3950440050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00ce4d0 0x7f31d0071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.350+0000 7f31ce59c640 1 -- 172.21.15.70:0/3950440050 >> v1:172.21.15.70:6789/0 conn(0x7f31d00b2ea0 legacy=0x7f31d006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.350+0000 7f31ce59c640 1 -- 172.21.15.70:0/3950440050 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0161190 con 0x7f31d006b180 2023-08-25T09:47:09.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.351+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3950440050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0070ef0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f31c40097a0 tx=0x7f31c402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832951540e91d05d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.351+0000 7f31cdd9b640 1 -- 172.21.15.70:0/3950440050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4031780 con 0x7f31d006b180 2023-08-25T09:47:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.351+0000 7f31cdd9b640 1 -- 172.21.15.70:0/3950440050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c40318e0 con 0x7f31d006b180 2023-08-25T09:47:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.351+0000 7f31cdd9b640 1 -- 172.21.15.70:0/3950440050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c4031c70 con 0x7f31d006b180 2023-08-25T09:47:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 -- 172.21.15.70:0/3950440050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 msgr2=0x7f31d0070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:09.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 --2- 172.21.15.70:0/3950440050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0070ef0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f31c40097a0 tx=0x7f31c402b820 comp rx=0 tx=0).stop 2023-08-25T09:47:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 -- 172.21.15.70:0/3950440050 shutdown_connections 2023-08-25T09:47:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 --2- 172.21.15.70:0/3950440050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0070ef0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 --2- 172.21.15.70:0/3950440050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00ce4d0 0x7f31d0071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 -- 172.21.15.70:0/3950440050 >> 172.21.15.70:0/3950440050 conn(0x7f31d00cd380 msgr2=0x7f31d00cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 -- 172.21.15.70:0/3950440050 shutdown_connections 2023-08-25T09:47:09.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.352+0000 7f31d5006640 1 -- 172.21.15.70:0/3950440050 wait complete. 2023-08-25T09:47:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.353+0000 7f31d5006640 1 Processor -- start 2023-08-25T09:47:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.354+0000 7f31d5006640 1 -- start start 2023-08-25T09:47:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.354+0000 7f31d5006640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.354+0000 7f31d5006640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00b2ea0 0x7f31d015e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31d5006640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31d00ce4d0 0x7f31d00b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31d5006640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d0156c60 con 0x7f31d006b180 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31d5006640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d015e680 con 0x7f31d00ce4d0 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31d5006640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d01f6d60 con 0x7f31d00b2ea0 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38728/0 (socket says 172.21.15.70:38728) 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 -- 172.21.15.70:0/3704729929 learned_addr learned my addr 172.21.15.70:0/3704729929 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00b2ea0 0x7f31d015e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 -- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00b2ea0 msgr2=0x7f31d015e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31cf59e640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31d00ce4d0 0x7f31d00b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00b2ea0 0x7f31d015e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 -- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31d00ce4d0 msgr2=0x7f31d00b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31d00ce4d0 0x7f31d00b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ced9d640 1 -- 172.21.15.70:0/3704729929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4009400 con 0x7f31d006b180 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00b2ea0 0x7f31d015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.355+0000 7f31cf59e640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31d00ce4d0 0x7f31d00b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.356+0000 7f31ced9d640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0156720 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f31c000cbc0 tx=0x7f31c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.356+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c000ad40 con 0x7f31d006b180 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.356+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0012400 con 0x7f31d006b180 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.356+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c000aea0 con 0x7f31d006b180 2023-08-25T09:47:09.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.356+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d01f61e0 con 0x7f31d006b180 2023-08-25T09:47:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.356+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d01f67a0 con 0x7f31d006b180 2023-08-25T09:47:09.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.357+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d015efc0 con 0x7f31d006b180 2023-08-25T09:47:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.358+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f31c0007070 con 0x7f31d006b180 2023-08-25T09:47:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.360+0000 7f31b77fe640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31bc06a050 0x7f31bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.360+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31c0088600 con 0x7f31d006b180 2023-08-25T09:47:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.360+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31bc06a050 0x7f31bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:09.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.361+0000 7f31ce59c640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31bc06a050 0x7f31bc06c510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f31c4036000 tx=0x7f31c403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:09.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.365+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f31d015efc0 con 0x7f31d006b180 2023-08-25T09:47:09.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.593+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31d015efc0 con 0x7f31d006b180 2023-08-25T09:47:09.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.594+0000 7f31b77fe640 1 -- 172.21.15.70:0/3704729929 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31d015efc0 con 0x7f31d006b180 2023-08-25T09:47:09.595 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:47:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31bc06a050 msgr2=0x7f31bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31bc06a050 0x7f31bc06c510 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f31c4036000 tx=0x7f31c403d040 comp rx=0 tx=0).stop 2023-08-25T09:47:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 msgr2=0x7f31d0156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0156720 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f31c000cbc0 tx=0x7f31c000a430 comp rx=0 tx=0).stop 2023-08-25T09:47:09.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 shutdown_connections 2023-08-25T09:47:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31bc06a050 0x7f31bc06c510 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31d00ce4d0 0x7f31d00b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31d00b2ea0 0x7f31d015e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 --2- 172.21.15.70:0/3704729929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31d006b180 0x7f31d0156720 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 >> 172.21.15.70:0/3704729929 conn(0x7f31d00cd380 msgr2=0x7f31d00b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 shutdown_connections 2023-08-25T09:47:09.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:09.596+0000 7f31d5006640 1 -- 172.21.15.70:0/3704729929 wait complete. 2023-08-25T09:47:09.615 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:47:09.616 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:47:09.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:47:09.617 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:47:09.617 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:09.618 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:47:09.618 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:47:09.618 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:47:09.619 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:47:09.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:09.619 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:47:09.680 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:47:09.680 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:47:09.680 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:47:09.681 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:09.682 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41859sec preferred_lft 41859sec 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:47:09.683 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.684 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:47:09.684 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.684 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:47:09.684 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:47:09.684 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:47:09.684 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.685 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:47:09.685 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:47:09.685 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:47:09.685 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:47:09.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:09.687 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:47:09.782 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:47:09.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:09.786 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:47:09.841 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:47:09.842 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:47:09.843 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:47:09.843 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:47:09.843 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:47:09.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:09.844 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:47:09.845 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:47:09.845 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:47:09.845 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:47:09.845 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:47:09.846 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:47:10.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:10.035 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:47:10.088 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:47:10.088 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:47:10.089 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:47:10.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:10.091 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:47:10.170 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:47:10.171 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:47:10.172 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:10.222 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:47:10.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:47:10.223 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:10.277 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:47:10.373 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:47:10.373 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:47:10.463 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:47:10.466 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:47:10.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:10.468 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:47:10.521 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:47:10.522 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:47:10.528 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:47:10.605 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:47:10.606 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:47:10.680 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:47:10.684 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:47:10.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:10.685 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:47:10.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:47:10.718+0000 7f35e1837480 -1 init, newargv = 0x55f2d4443e10 newargc=15 2023-08-25T09:47:10.721 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[91974]: starting ceph client 2023-08-25T09:47:10.728 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[91974]: starting fuse 2023-08-25T09:47:10.732 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:47:10.733 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:47:10.733 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:47:10.734 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:47:10.734 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:47:10.734 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:47:10.735 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:47:10.735 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:47:10.735 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:47:10.736 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:47:10.736 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:47:10.736 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:47:10.736 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:47:10.737 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:47:10.737 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:47:10.737 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:47:10.738 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:47:10.738 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:47:10.738 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:47:10.739 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:47:10.739 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:47:10.739 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:47:10.739 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:47:10.741 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:47:10.741 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:47:10.741 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:47:10.742 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:47:10.742 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:47:10.742 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:47:10.942 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.91974.asok 2023-08-25T09:47:10.946 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:47:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:47:10.960 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.91974.asok 2023-08-25T09:47:10.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:10.962 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.91974.asok status 2023-08-25T09:47:11.204 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:47:11.205 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "91974", 2023-08-25T09:47:11.206 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:47:11.206 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:47:11.206 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:47:11.207 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:47:11.208 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5941, 2023-08-25T09:47:11.208 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:47:11.208 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:47:11.209 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:47:11.209 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5941 2023-08-25T09:47:11.209 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:47:11.209 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:47:11.210 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:47:11.210 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:47:11.211 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 876087750 2023-08-25T09:47:11.211 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:47:11.212 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:47:11.212 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:47:11.212 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:47:11.213 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:47:11.214 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 876087750 2023-08-25T09:47:11.214 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:47:11.214 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5941 192.168.0.1:0/876087750", 2023-08-25T09:47:11.215 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/876087750", 2023-08-25T09:47:11.215 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:47:11.216 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 65, 2023-08-25T09:47:11.216 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 150, 2023-08-25T09:47:11.217 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:47:11.217 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:47:11.218 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:47:11.218 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:47:11.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:11.222 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:47:11.223 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:11.277 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:47:11.278 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:11.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:11.280 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:11.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:11.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.604+0000 7f62d1be8640 1 Processor -- start 2023-08-25T09:47:11.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.604+0000 7f62d1be8640 1 -- start start 2023-08-25T09:47:11.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.606+0000 7f62d1be8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f62cc0b1ff0 0x7f62cc0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:11.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.606+0000 7f62d1be8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc0b1b60 con 0x7f62cc15e830 2023-08-25T09:47:11.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.606+0000 7f62d1be8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc06b7c0 con 0x7f62cc15ec10 2023-08-25T09:47:11.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.606+0000 7f62d1be8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc06bd80 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.606+0000 7f62c3fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f62cc15ec10 0x7f62cc15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:55406/0 (socket says 172.21.15.70:55406) 2023-08-25T09:47:11.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.606+0000 7f62c3fff640 1 -- 172.21.15.70:0/866442629 learned_addr learned my addr 172.21.15.70:0/866442629 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:11.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.607+0000 7f62cbfff640 1 --2- 172.21.15.70:0/866442629 >> v2:172.21.15.155:3300/0 conn(0x7f62cc0b1ff0 0x7f62cc0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:11.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.607+0000 7f62cbfff640 1 -- 172.21.15.70:0/866442629 >> v1:172.21.15.155:6789/0 conn(0x7f62cc15ec10 legacy=0x7f62cc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.607+0000 7f62cbfff640 1 -- 172.21.15.70:0/866442629 >> v1:172.21.15.70:6789/0 conn(0x7f62cc15e830 legacy=0x7f62cc06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.607+0000 7f62cbfff640 1 -- 172.21.15.70:0/866442629 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc06baa0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.607+0000 7f62caffd640 1 -- 172.21.15.70:0/866442629 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432510723 0 0) 0x7f62cc0b1b60 con 0x7f62cc15e830 2023-08-25T09:47:11.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.609+0000 7f62cbfff640 1 --2- 172.21.15.70:0/866442629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0b2440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009d90 tx=0x7f62c402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6703df42c4f877 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:11.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.609+0000 7f62caffd640 1 -- 172.21.15.70:0/866442629 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c402ccf0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.609+0000 7f62caffd640 1 -- 172.21.15.70:0/866442629 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c402ce50 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.609+0000 7f62caffd640 1 -- 172.21.15.70:0/866442629 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4034660 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 -- 172.21.15.70:0/866442629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0b1ff0 msgr2=0x7f62cc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 --2- 172.21.15.70:0/866442629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0b2440 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f62c4009d90 tx=0x7f62c402bf40 comp rx=0 tx=0).stop 2023-08-25T09:47:11.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 -- 172.21.15.70:0/866442629 shutdown_connections 2023-08-25T09:47:11.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 --2- 172.21.15.70:0/866442629 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0b2440 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 -- 172.21.15.70:0/866442629 >> 172.21.15.70:0/866442629 conn(0x7f62cc069850 msgr2=0x7f62cc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:11.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 -- 172.21.15.70:0/866442629 shutdown_connections 2023-08-25T09:47:11.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.610+0000 7f62d1be8640 1 -- 172.21.15.70:0/866442629 wait complete. 2023-08-25T09:47:11.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.612+0000 7f62d1be8640 1 Processor -- start 2023-08-25T09:47:11.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.612+0000 7f62d1be8640 1 -- start start 2023-08-25T09:47:11.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.613+0000 7f62d1be8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:11.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.613+0000 7f62d1be8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc15e830 0x7f62cc15cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:11.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.613+0000 7f62d1be8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc15ec10 0x7f62cc15d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:11.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.613+0000 7f62d1be8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0aeeb0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.613+0000 7f62d1be8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0ae2a0 con 0x7f62cc15ec10 2023-08-25T09:47:11.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.613+0000 7f62d1be8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0ae580 con 0x7f62cc15e830 2023-08-25T09:47:11.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.614+0000 7f62cb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:11.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.614+0000 7f62cb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38730/0 (socket says 172.21.15.70:38730) 2023-08-25T09:47:11.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.614+0000 7f62c3fff640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc15e830 0x7f62cc15cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:11.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.614+0000 7f62cb7fe640 1 -- 172.21.15.70:0/1339735758 learned_addr learned my addr 172.21.15.70:0/1339735758 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:11.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.614+0000 7f62cbfff640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc15ec10 0x7f62cc15d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:11.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cb7fe640 1 -- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc15e830 msgr2=0x7f62cc15cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cb7fe640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc15e830 0x7f62cc15cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cb7fe640 1 -- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc15ec10 msgr2=0x7f62cc15d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cb7fe640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc15ec10 0x7f62cc15d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cb7fe640 1 -- 172.21.15.70:0/1339735758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c4009a70 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cbfff640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc15ec10 0x7f62cc15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:11.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.615+0000 7f62cb7fe640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0ad970 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c4b0 tx=0x7f62bc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:11.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.616+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc005480 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.616+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc0035f0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.616+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0039e0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.616+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc0ae860 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.616+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc0adeb0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.617+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc04e420 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.622+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f62bc003b40 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.623+0000 7f62c8ff9640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62b006a050 0x7f62b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:11.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.623+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62bc087b40 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.623+0000 7f62c3fff640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62b006a050 0x7f62b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:11.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.624+0000 7f62c3fff640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62b006a050 0x7f62b006c510 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0ac750 tx=0x7f62b4007e80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:11.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.628+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f62bc053250 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.853+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62cc0caca0 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.854+0000 7f62c8ff9640 1 -- 172.21.15.70:0/1339735758 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f62bc056870 con 0x7f62cc0b1ff0 2023-08-25T09:47:11.856 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:11.856 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62b006a050 msgr2=0x7f62b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62b006a050 0x7f62b006c510 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0ac750 tx=0x7f62b4007e80 comp rx=0 tx=0).stop 2023-08-25T09:47:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 msgr2=0x7f62cc0ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0ad970 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00c4b0 tx=0x7f62bc00c970 comp rx=0 tx=0).stop 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 shutdown_connections 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f62b006a050 0x7f62b006c510 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f62cc15ec10 0x7f62cc15d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f62cc15e830 0x7f62cc15cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 --2- 172.21.15.70:0/1339735758 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f62cc0b1ff0 0x7f62cc0ad970 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 >> 172.21.15.70:0/1339735758 conn(0x7f62cc069850 msgr2=0x7f62cc15a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 shutdown_connections 2023-08-25T09:47:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:11.857+0000 7f62d1be8640 1 -- 172.21.15.70:0/1339735758 wait complete. 2023-08-25T09:47:11.877 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-08-25T09:47:12.121 INFO:teuthology.orchestra.run.smithi155.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":8232,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.27917274,"mdsmap_epoch":65,"osdmap_epoch":150,"osdmap_epoch_barrier":150,"uptime":19.677592178000001} 2023-08-25T09:47:12.121 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 8232, "mdsmap_epoch": 65, "osdmap_epoch": 150, "osdmap_epoch_barrier": 150, "rank_uptime": 13.27917274, "state": "up:active", "uptime": 19.677592178, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:47:12.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-08-25T09:47:12.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.352+0000 7f5858c83640 1 Processor -- start 2023-08-25T09:47:12.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.352+0000 7f5858c83640 1 -- start start 2023-08-25T09:47:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5858c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f585406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5858c83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5854072820 0x7f5854072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5858c83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585406b1d0 con 0x7f585406a0b0 2023-08-25T09:47:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5858c83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58540731f0 con 0x7f5854072820 2023-08-25T09:47:12.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5858c83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585415d3d0 con 0x7f585415cff0 2023-08-25T09:47:12.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5852d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5854072820 0x7f5854072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:12.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5851d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f585406a0b0 0x7f585406bd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42706/0 (socket says 172.21.15.70:42706) 2023-08-25T09:47:12.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f5851d9b640 1 -- 172.21.15.70:0/2196947534 learned_addr learned my addr 172.21.15.70:0/2196947534 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:12.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.354+0000 7f585259c640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f585406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:12.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.355+0000 7f5852d9d640 1 -- 172.21.15.70:0/2196947534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 msgr2=0x7f585406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:12.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.355+0000 7f5852d9d640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f585406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:12.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.355+0000 7f5852d9d640 1 -- 172.21.15.70:0/2196947534 >> v1:172.21.15.70:6789/0 conn(0x7f585406a0b0 legacy=0x7f585406bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:12.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.355+0000 7f5852d9d640 1 -- 172.21.15.70:0/2196947534 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585406c4a0 con 0x7f5854072820 2023-08-25T09:47:12.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.355+0000 7f585159a640 1 -- 172.21.15.70:0/2196947534 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157430264 0 0) 0x7f585406b1d0 con 0x7f585406a0b0 2023-08-25T09:47:12.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.355+0000 7f585259c640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f585406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:12.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.356+0000 7f5852d9d640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5854072820 0x7f5854072cb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f584c009d90 tx=0x7f584c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536745c5748f97bd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:12.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.356+0000 7f585159a640 1 -- 172.21.15.70:0/2196947534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c02cbc0 con 0x7f5854072820 2023-08-25T09:47:12.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.356+0000 7f585159a640 1 -- 172.21.15.70:0/2196947534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c02cd20 con 0x7f5854072820 2023-08-25T09:47:12.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.356+0000 7f585159a640 1 -- 172.21.15.70:0/2196947534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c034660 con 0x7f5854072820 2023-08-25T09:47:12.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 -- 172.21.15.70:0/2196947534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5854072820 msgr2=0x7f5854072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:12.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5854072820 0x7f5854072cb0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f584c009d90 tx=0x7f584c02be10 comp rx=0 tx=0).stop 2023-08-25T09:47:12.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 -- 172.21.15.70:0/2196947534 shutdown_connections 2023-08-25T09:47:12.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5854072820 0x7f5854072cb0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 --2- 172.21.15.70:0/2196947534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f585406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 -- 172.21.15.70:0/2196947534 >> 172.21.15.70:0/2196947534 conn(0x7f58540cfb80 msgr2=0x7f58540cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 -- 172.21.15.70:0/2196947534 shutdown_connections 2023-08-25T09:47:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.357+0000 7f5858c83640 1 -- 172.21.15.70:0/2196947534 wait complete. 2023-08-25T09:47:12.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.359+0000 7f5858c83640 1 Processor -- start 2023-08-25T09:47:12.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.359+0000 7f5858c83640 1 -- start start 2023-08-25T09:47:12.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5858c83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 0x7f585415f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5858c83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5854072820 0x7f58540c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5858c83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f5854158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5858c83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b2500 con 0x7f585406a0b0 2023-08-25T09:47:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5858c83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b27c0 con 0x7f5854072820 2023-08-25T09:47:12.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5858c83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58540b2aa0 con 0x7f585415cff0 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 0x7f585415f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 0x7f585415f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38746/0 (socket says 172.21.15.70:38746) 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 -- 172.21.15.70:0/3736668327 learned_addr learned my addr 172.21.15.70:0/3736668327 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f5852d9d640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f5854158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 -- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 msgr2=0x7f5854158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f5854158860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 -- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5854072820 msgr2=0x7f58540c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:12.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5854072820 0x7f58540c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.360+0000 7f585259c640 1 -- 172.21.15.70:0/3736668327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c009a70 con 0x7f585406a0b0 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.361+0000 7f585259c640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 0x7f585415f880 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f584400c7f0 tx=0x7f584400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.361+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5844007a30 con 0x7f585406a0b0 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.361+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844007b90 con 0x7f585406a0b0 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.361+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584400ce40 con 0x7f585406a0b0 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.361+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58540b2d80 con 0x7f585406a0b0 2023-08-25T09:47:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.361+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585415fdc0 con 0x7f585406a0b0 2023-08-25T09:47:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.362+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58540b2fc0 con 0x7f585406a0b0 2023-08-25T09:47:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.363+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f58440113f0 con 0x7f585406a0b0 2023-08-25T09:47:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.364+0000 7f584affd640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f583406a050 0x7f583406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.364+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5844088810 con 0x7f585406a0b0 2023-08-25T09:47:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.364+0000 7f5851d9b640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f583406a050 0x7f583406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:12.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.365+0000 7f5851d9b640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f583406a050 0x7f583406c510 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f583c007fd0 tx=0x7f583c007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:12.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.368+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58540b2fc0 con 0x7f585406a0b0 2023-08-25T09:47:12.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:12.561+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f58540b2fc0 con 0x7f585406a0b0 2023-08-25T09:47:13.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.573+0000 7f584affd640 1 -- 172.21.15.70:0/3736668327 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v151) v1 ==== 181+0+0 (secure 0 0 0) 0x7f58440578f0 con 0x7f585406a0b0 2023-08-25T09:47:13.577 INFO:teuthology.orchestra.run.smithi070.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-08-25T09:47:13.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f583406a050 msgr2=0x7f583406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:13.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f583406a050 0x7f583406c510 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f583c007fd0 tx=0x7f583c007960 comp rx=0 tx=0).stop 2023-08-25T09:47:13.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 msgr2=0x7f585415f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:13.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 0x7f585415f880 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f584400c7f0 tx=0x7f584400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:47:13.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 shutdown_connections 2023-08-25T09:47:13.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f583406a050 0x7f583406c510 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:13.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f585415cff0 0x7f5854158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:13.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5854072820 0x7f58540c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:13.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 --2- 172.21.15.70:0/3736668327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f585406a0b0 0x7f585415f880 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:13.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.577+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 >> 172.21.15.70:0/3736668327 conn(0x7f58540cfb80 msgr2=0x7f58540c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:13.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.578+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 shutdown_connections 2023-08-25T09:47:13.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:13.578+0000 7f5858c83640 1 -- 172.21.15.70:0/3736668327 wait complete. 2023-08-25T09:47:13.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:47:13.835 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:47:13.836 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:47:13.836 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-08-25T09:47:13.836 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:47:13.837 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:13.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:13.896 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-08-25T09:47:14.377 INFO:teuthology.orchestra.run.smithi070.stderr:16+0 records in 2023-08-25T09:47:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:16+0 records out 2023-08-25T09:47:14.378 INFO:teuthology.orchestra.run.smithi070.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.371537 s, 45.2 MB/s 2023-08-25T09:47:14.379 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:47:14.381 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:14.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:14.440 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-08-25T09:47:14.782 INFO:teuthology.orchestra.run.smithi070.stderr:17+0 records in 2023-08-25T09:47:14.783 INFO:teuthology.orchestra.run.smithi070.stderr:17+0 records out 2023-08-25T09:47:14.783 INFO:teuthology.orchestra.run.smithi070.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.285553 s, 62.4 MB/s 2023-08-25T09:47:14.784 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-08-25T09:47:14.785 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:15.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.007+0000 7f51a759e640 1 Processor -- start 2023-08-25T09:47:15.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.008+0000 7f51a759e640 1 -- start start 2023-08-25T09:47:15.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.009+0000 7f51a759e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51a806b330 0x7f51a80cc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:15.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.009+0000 7f51a759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a80ca1a0 con 0x7f51a80c9d30 2023-08-25T09:47:15.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.009+0000 7f51a759e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a80c98a0 con 0x7f51a8156980 2023-08-25T09:47:15.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.009+0000 7f51a759e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a80c8c90 con 0x7f51a806b330 2023-08-25T09:47:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f51a6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f51a806b330 0x7f51a80cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f519dd9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f51a80c9d30 0x7f51a806ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42720/0 (socket says 172.21.15.70:42720) 2023-08-25T09:47:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f519dd9b640 1 -- 172.21.15.70:0/3010961835 learned_addr learned my addr 172.21.15.70:0/3010961835 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f51a6d9d640 1 -- 172.21.15.70:0/3010961835 >> v1:172.21.15.155:6789/0 conn(0x7f51a8156980 legacy=0x7f51a80ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f51a6d9d640 1 -- 172.21.15.70:0/3010961835 >> v1:172.21.15.70:6789/0 conn(0x7f51a80c9d30 legacy=0x7f51a806ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f51a6d9d640 1 -- 172.21.15.70:0/3010961835 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80c8f70 con 0x7f51a806b330 2023-08-25T09:47:15.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.010+0000 7f51a5d9b640 1 -- 172.21.15.70:0/3010961835 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2393665828 0 0) 0x7f51a80ca1a0 con 0x7f51a80c9d30 2023-08-25T09:47:15.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.011+0000 7f51a6d9d640 1 --2- 172.21.15.70:0/3010961835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a806b330 0x7f51a80cc070 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f51a000aea0 tx=0x7f51a002d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941df6bf25ef73af server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:15.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.011+0000 7f51a5d9b640 1 -- 172.21.15.70:0/3010961835 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a002dbe0 con 0x7f51a806b330 2023-08-25T09:47:15.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.011+0000 7f51a5d9b640 1 -- 172.21.15.70:0/3010961835 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a002dd40 con 0x7f51a806b330 2023-08-25T09:47:15.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.011+0000 7f51a5d9b640 1 -- 172.21.15.70:0/3010961835 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0008660 con 0x7f51a806b330 2023-08-25T09:47:15.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 -- 172.21.15.70:0/3010961835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a806b330 msgr2=0x7f51a80cc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 --2- 172.21.15.70:0/3010961835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a806b330 0x7f51a80cc070 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f51a000aea0 tx=0x7f51a002d040 comp rx=0 tx=0).stop 2023-08-25T09:47:15.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 -- 172.21.15.70:0/3010961835 shutdown_connections 2023-08-25T09:47:15.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 --2- 172.21.15.70:0/3010961835 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a806b330 0x7f51a80cc070 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 -- 172.21.15.70:0/3010961835 >> 172.21.15.70:0/3010961835 conn(0x7f51a80affd0 msgr2=0x7f51a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:15.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 -- 172.21.15.70:0/3010961835 shutdown_connections 2023-08-25T09:47:15.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.012+0000 7f51a759e640 1 -- 172.21.15.70:0/3010961835 wait complete. 2023-08-25T09:47:15.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.014+0000 7f51a759e640 1 Processor -- start 2023-08-25T09:47:15.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.014+0000 7f51a759e640 1 -- start start 2023-08-25T09:47:15.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f51a759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a806b330 0x7f51a80c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:15.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f51a759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 0x7f51a80c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:15.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f51a759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51a8156980 0x7f51a8151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:15.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f51a759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c5ab0 con 0x7f51a80c9d30 2023-08-25T09:47:15.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f51a759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c5ca0 con 0x7f51a806b330 2023-08-25T09:47:15.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f51a759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80c5f80 con 0x7f51a8156980 2023-08-25T09:47:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f519dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 0x7f51a80c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:15.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f519dd9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 0x7f51a80c5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38762/0 (socket says 172.21.15.70:38762) 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.015+0000 7f519dd9b640 1 -- 172.21.15.70:0/4168078565 learned_addr learned my addr 172.21.15.70:0/4168078565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f51a659c640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a806b330 0x7f51a80c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519dd9b640 1 -- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51a8156980 msgr2=0x7f51a8151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f51a6d9d640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51a8156980 0x7f51a8151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519dd9b640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51a8156980 0x7f51a8151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519dd9b640 1 -- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a806b330 msgr2=0x7f51a80c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519dd9b640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a806b330 0x7f51a80c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519dd9b640 1 -- 172.21.15.70:0/4168078565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a000ab80 con 0x7f51a80c9d30 2023-08-25T09:47:15.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f51a659c640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a806b330 0x7f51a80c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:15.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519dd9b640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 0x7f51a80c5570 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f519000c4a0 tx=0x7f519000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:15.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900074e0 con 0x7f51a80c9d30 2023-08-25T09:47:15.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190007640 con 0x7f51a80c9d30 2023-08-25T09:47:15.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.016+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190004a00 con 0x7f51a80c9d30 2023-08-25T09:47:15.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.017+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80c6260 con 0x7f51a80c9d30 2023-08-25T09:47:15.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.017+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80c6820 con 0x7f51a80c9d30 2023-08-25T09:47:15.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.018+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8066fb0 con 0x7f51a80c9d30 2023-08-25T09:47:15.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.019+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5190004b60 con 0x7f51a80c9d30 2023-08-25T09:47:15.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.020+0000 7f519f7fe640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f518806a050 0x7f518806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:15.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.020+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5190087910 con 0x7f51a80c9d30 2023-08-25T09:47:15.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.020+0000 7f51a659c640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f518806a050 0x7f518806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:15.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.021+0000 7f51a659c640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f518806a050 0x7f518806c510 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5198000f60 tx=0x7f5198009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:15.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.026+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5190052ed0 con 0x7f51a80c9d30 2023-08-25T09:47:15.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.219+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f51a80cbd70 con 0x7f51a80c9d30 2023-08-25T09:47:15.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.221+0000 7f519f7fe640 1 -- 172.21.15.70:0/4168078565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+37035 (secure 0 0 0) 0x7f5190018030 con 0x7f51a80c9d30 2023-08-25T09:47:15.223 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:15.223 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 151, 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:13.557064+0000", 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:15.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:15.225 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:15.226 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:15.228 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:15.229 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:15.229 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:15.229 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:15.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:15.231 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:15.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:15.233 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:15.234 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:15.235 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:15.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:15.237 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:15.238 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:15.238 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:15.238 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:15.239 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:15.239 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.240 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:15.241 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:15.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:15.243 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:15.244 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:15.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:15.246 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:15.247 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:15.248 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:15.249 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:15.250 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:15.251 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:15.252 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:15.254 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:15.255 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:15.256 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:15.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:15.258 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:15.259 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:15.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:15.261 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:15.262 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:15.263 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:15.264 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.265 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:15.266 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:15.267 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:15.268 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "151", 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:15.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:15.270 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:15.271 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:15.271 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:15.272 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.273 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:15.274 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:15.275 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:15.276 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.277 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.278 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.279 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.281 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:15.281 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.282 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.284 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.285 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.288 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.289 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.290 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.291 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.293 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.294 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:15.295 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.297 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.298 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.299 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.300 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:15.301 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.304 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:15.307 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:15.308 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:15.309 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.310 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:15.311 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.312 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.314 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.316 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:15.318 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.319 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.320 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.321 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.322 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.326 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:15.328 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.329 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.330 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.331 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.334 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.336 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.337 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:15.338 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:15.339 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.340 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.341 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.342 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.344 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.346 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:15.348 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.349 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:15.350 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:15.352 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.354 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:15.358 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.359 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.360 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.361 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.362 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.364 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.365 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:15.366 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.368 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:15.369 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:15.370 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:15.371 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:15.372 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:15.373 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:15.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:15.374 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:15.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.224+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f518806a050 msgr2=0x7f518806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.224+0000 7f51a759e640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f518806a050 0x7f518806c510 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5198000f60 tx=0x7f5198009040 comp rx=0 tx=0).stop 2023-08-25T09:47:15.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.224+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 msgr2=0x7f51a80c5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.224+0000 7f51a759e640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 0x7f51a80c5570 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f519000c4a0 tx=0x7f519000c960 comp rx=0 tx=0).stop 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 shutdown_connections 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f518806a050 0x7f518806c510 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f51a8156980 0x7f51a8151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f51a80c9d30 0x7f51a80c5570 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 --2- 172.21.15.70:0/4168078565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f51a806b330 0x7f51a80c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:15.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 >> 172.21.15.70:0/4168078565 conn(0x7f51a80affd0 msgr2=0x7f51a81588b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 shutdown_connections 2023-08-25T09:47:15.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:15.225+0000 7f51a759e640 1 -- 172.21.15.70:0/4168078565 wait complete. 2023-08-25T09:47:15.377 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:47:16.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:47:16.306 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:47:20.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:20.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.601+0000 7fcf1e226640 1 Processor -- start 2023-08-25T09:47:20.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.601+0000 7fcf1e226640 1 -- start start 2023-08-25T09:47:20.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf1e226640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf1815cb40 0x7fcf180ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf1e226640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf180ac870 0x7fcf180ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf1e226640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcf180b3880 0x7fcf18160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf1e226640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1806f050 con 0x7fcf1815cb40 2023-08-25T09:47:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf1e226640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1806f330 con 0x7fcf180b3880 2023-08-25T09:47:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf1e226640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1806f610 con 0x7fcf180ac870 2023-08-25T09:47:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf1815cb40 0x7fcf180ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcf1815cb40 0x7fcf180ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54886/0 (socket says 172.21.15.70:54886) 2023-08-25T09:47:20.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 -- 172.21.15.70:0/3219885938 learned_addr learned my addr 172.21.15.70:0/3219885938 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:20.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 -- 172.21.15.70:0/3219885938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf180ac870 msgr2=0x7fcf180ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:20.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 --2- 172.21.15.70:0/3219885938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf180ac870 0x7fcf180ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 -- 172.21.15.70:0/3219885938 >> v2:172.21.15.155:3300/0 conn(0x7fcf180b3880 msgr2=0x7fcf18160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:20.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 --2- 172.21.15.70:0/3219885938 >> v2:172.21.15.155:3300/0 conn(0x7fcf180b3880 0x7fcf18160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.603+0000 7fcf177fe640 1 -- 172.21.15.70:0/3219885938 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1806f8f0 con 0x7fcf1815cb40 2023-08-25T09:47:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.604+0000 7fcf177fe640 1 --2- 172.21.15.70:0/3219885938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf1815cb40 0x7fcf180ce320 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fcf180adbd0 tx=0x7fcf0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f477e52a051ada server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf167fc640 1 -- 172.21.15.70:0/3219885938 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08031870 con 0x7fcf1815cb40 2023-08-25T09:47:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf167fc640 1 -- 172.21.15.70:0/3219885938 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf080319d0 con 0x7fcf1815cb40 2023-08-25T09:47:20.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf167fc640 1 -- 172.21.15.70:0/3219885938 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf08031d60 con 0x7fcf1815cb40 2023-08-25T09:47:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 -- 172.21.15.70:0/3219885938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf1815cb40 msgr2=0x7fcf180ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 --2- 172.21.15.70:0/3219885938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf1815cb40 0x7fcf180ce320 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fcf180adbd0 tx=0x7fcf0802b910 comp rx=0 tx=0).stop 2023-08-25T09:47:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 -- 172.21.15.70:0/3219885938 shutdown_connections 2023-08-25T09:47:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 --2- 172.21.15.70:0/3219885938 >> v2:172.21.15.155:3300/0 conn(0x7fcf180b3880 0x7fcf18160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 --2- 172.21.15.70:0/3219885938 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf180ac870 0x7fcf180ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 --2- 172.21.15.70:0/3219885938 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf1815cb40 0x7fcf180ce320 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 -- 172.21.15.70:0/3219885938 >> 172.21.15.70:0/3219885938 conn(0x7fcf18069850 msgr2=0x7fcf18069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 -- 172.21.15.70:0/3219885938 shutdown_connections 2023-08-25T09:47:20.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.605+0000 7fcf1e226640 1 -- 172.21.15.70:0/3219885938 wait complete. 2023-08-25T09:47:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.607+0000 7fcf1e226640 1 Processor -- start 2023-08-25T09:47:20.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.607+0000 7fcf1e226640 1 -- start start 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf1e226640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf180ac870 0x7fcf180ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf1e226640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 0x7fcf1814e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf1e226640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf1815cb40 0x7fcf1814e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf1e226640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180cc6d0 con 0x7fcf180b3880 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf1e226640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180ccfe0 con 0x7fcf180ac870 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf1e226640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf180cb630 con 0x7fcf1815cb40 2023-08-25T09:47:20.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.608+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 0x7fcf1814e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 0x7fcf1814e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54902/0 (socket says 172.21.15.70:54902) 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 -- 172.21.15.70:0/648729556 learned_addr learned my addr 172.21.15.70:0/648729556 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf177fe640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf180ac870 0x7fcf180ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf17fff640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf1815cb40 0x7fcf1814e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 -- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf1815cb40 msgr2=0x7fcf1814e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf1815cb40 0x7fcf1814e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 -- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf180ac870 msgr2=0x7fcf180ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:20.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf180ac870 0x7fcf180ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 -- 172.21.15.70:0/648729556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf080094d0 con 0x7fcf180b3880 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf17fff640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf1815cb40 0x7fcf1814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.609+0000 7fcf16ffd640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 0x7fcf1814e100 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b410 tx=0x7fcf0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.610+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c011830 con 0x7fcf180b3880 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.610+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c0119b0 con 0x7fcf180b3880 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.610+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c011d60 con 0x7fcf180b3880 2023-08-25T09:47:20.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.610+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf180cb910 con 0x7fcf180b3880 2023-08-25T09:47:20.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.610+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf180cbed0 con 0x7fcf180b3880 2023-08-25T09:47:20.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.611+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1804e420 con 0x7fcf180b3880 2023-08-25T09:47:20.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.612+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf0c01a400 con 0x7fcf180b3880 2023-08-25T09:47:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.614+0000 7fcefffff640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf0406a050 0x7fcf0406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.614+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf0c088770 con 0x7fcf180b3880 2023-08-25T09:47:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.614+0000 7fcf177fe640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf0406a050 0x7fcf0406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.615+0000 7fcf177fe640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf0406a050 0x7fcf0406c510 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcf08036000 tx=0x7fcf0803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.618+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcf0c053e50 con 0x7fcf180b3880 2023-08-25T09:47:20.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.809+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fcf18157de0 con 0x7fcf180b3880 2023-08-25T09:47:20.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.813+0000 7fcefffff640 1 -- 172.21.15.70:0/648729556 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v151) v1 ==== 81+0+37035 (secure 0 0 0) 0x7fcf0c057470 con 0x7fcf180b3880 2023-08-25T09:47:20.815 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:20.815 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:20.815 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 151, 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:13.557064+0000", 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:20.816 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:20.817 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:20.818 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:20.818 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:20.819 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:20.820 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:20.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:20.821 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:20.821 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:20.821 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:20.821 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:20.821 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:20.822 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:20.822 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:20.822 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:20.823 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:20.823 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:20.823 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:20.823 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:20.823 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:20.824 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:20.824 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:20.824 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:20.824 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:20.824 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:20.824 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:20.825 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:20.825 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:20.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.825 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:20.826 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:20.827 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:20.827 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:20.827 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:20.827 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:20.828 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:20.828 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:20.829 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:20.829 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:20.830 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:20.830 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:20.830 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:20.830 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:20.830 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:20.830 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:20.831 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:20.831 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:20.831 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:20.831 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:20.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:20.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.832 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:20.832 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:20.832 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:20.832 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:20.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.834 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:20.835 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:20.836 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:20.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.837 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:20.837 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:20.837 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:20.837 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:20.837 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:20.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:20.838 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:20.838 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:20.838 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:20.838 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:20.838 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:20.839 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:20.839 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:20.839 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:20.839 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:20.839 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:20.840 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:20.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:20.842 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:20.843 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:20.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:20.845 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:20.846 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:20.846 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:20.846 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:20.846 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:20.846 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:20.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:20.848 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:20.849 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:20.850 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:20.851 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:20.852 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:20.853 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:20.854 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:20.855 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:20.855 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:20.855 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:20.855 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:20.855 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:20.856 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:20.857 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:20.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:20.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.857 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:20.857 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:20.857 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:20.858 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:20.859 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:20.859 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:20.859 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:20.859 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:20.859 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:20.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:20.860 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:20.861 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:20.861 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:20.861 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:20.861 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:20.861 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:20.861 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:20.862 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:20.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.862 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:20.863 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:20.863 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:20.863 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:20.863 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:20.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:20.863 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:20.864 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:20.864 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:20.864 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:20.864 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:20.864 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:20.865 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:20.865 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:20.865 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:20.865 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:20.865 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:20.865 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:20.866 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "151", 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:20.867 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:20.868 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:20.869 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:20.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:20.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.871 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:20.871 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:20.871 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:20.871 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:20.871 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:20.872 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:20.874 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:20.875 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.876 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.877 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.877 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.877 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.877 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.877 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.878 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:20.878 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.878 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.879 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:20.880 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.881 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.881 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.882 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.882 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:20.882 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.883 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.883 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.885 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:20.885 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.886 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:20.886 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:20.886 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.886 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.886 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:20.887 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:20.887 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:20.887 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:20.887 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.887 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.887 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.888 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.889 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:20.889 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.890 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.893 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:20.893 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.893 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.893 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.893 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.894 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.894 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:20.895 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.895 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:20.895 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.897 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.898 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.899 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.900 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.902 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:20.903 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:20.905 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.910 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:20.911 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:20.911 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:20.911 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:20.911 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.911 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.911 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.912 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.913 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:20.914 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.915 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.915 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:20.915 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.915 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.916 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.916 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.916 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.916 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:20.917 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:20.918 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.918 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.918 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.919 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.919 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.919 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:20.919 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.920 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.920 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:20.920 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.920 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.921 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.921 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.921 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.921 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:20.922 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.922 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.922 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:20.923 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.925 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.926 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.926 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.926 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.926 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.926 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:20.927 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.928 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.928 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.928 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.928 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:20.929 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.930 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:20.932 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.934 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.935 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.936 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.937 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.937 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.937 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:20.937 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.939 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.939 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.939 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.939 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:20.939 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.940 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.941 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.941 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.941 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.941 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:20.942 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.943 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.943 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.943 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.943 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.944 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.944 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:20.944 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.945 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.945 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:20.945 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:20.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.945 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.946 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:20.946 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:20.946 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:20.946 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:20.946 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.947 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.947 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.947 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.947 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.948 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.949 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.950 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:20.950 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.950 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.951 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:20.951 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.951 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.951 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.953 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.954 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.955 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.956 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:20.957 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:20.958 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:20.959 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.960 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.961 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:20.962 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.964 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:20.964 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.964 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.964 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:20.964 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.967 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:20.967 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:20.967 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:20.967 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.967 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:20.968 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.969 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.970 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:20.971 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.972 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.974 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.976 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.978 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.979 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.980 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:20.982 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:20.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:20.984 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:20.985 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:20.986 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:20.987 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:20.987 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:20.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf0406a050 msgr2=0x7fcf0406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:20.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf0406a050 0x7fcf0406c510 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcf08036000 tx=0x7fcf0803c040 comp rx=0 tx=0).stop 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 msgr2=0x7fcf1814e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 0x7fcf1814e100 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b410 tx=0x7fcf0c00b8d0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 shutdown_connections 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcf0406a050 0x7fcf0406c510 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcf1815cb40 0x7fcf1814e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcf180b3880 0x7fcf1814e100 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 --2- 172.21.15.70:0/648729556 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcf180ac870 0x7fcf180ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 >> 172.21.15.70:0/648729556 conn(0x7fcf18069850 msgr2=0x7fcf18157050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 shutdown_connections 2023-08-25T09:47:20.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:20.816+0000 7fcf1e226640 1 -- 172.21.15.70:0/648729556 wait complete. 2023-08-25T09:47:20.991 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:47:25.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:26.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.214+0000 7f423556a640 1 Processor -- start 2023-08-25T09:47:26.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.214+0000 7f423556a640 1 -- start start 2023-08-25T09:47:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f423556a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015ec10 0x7f423015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:26.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f423556a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f423015c6d0 0x7f42300b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f423556a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42300b1b60 con 0x7f423015e830 2023-08-25T09:47:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f423556a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f423006bd80 con 0x7f423015c6d0 2023-08-25T09:47:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f423556a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423006b7c0 con 0x7f423015ec10 2023-08-25T09:47:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f422f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f423015c6d0 0x7f42300b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f422f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f423015c6d0 0x7f42300b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54908/0 (socket says 172.21.15.70:54908) 2023-08-25T09:47:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f422f7fe640 1 -- 172.21.15.70:0/3911989429 learned_addr learned my addr 172.21.15.70:0/3911989429 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f4227fff640 1 --2- 172.21.15.70:0/3911989429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015ec10 0x7f423015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f422f7fe640 1 -- 172.21.15.70:0/3911989429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015ec10 msgr2=0x7f423015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.216+0000 7f422f7fe640 1 --2- 172.21.15.70:0/3911989429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015ec10 0x7f423015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.217+0000 7f422f7fe640 1 -- 172.21.15.70:0/3911989429 >> v1:172.21.15.155:6789/0 conn(0x7f423015e830 legacy=0x7f423006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.217+0000 7f422f7fe640 1 -- 172.21.15.70:0/3911989429 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423006baa0 con 0x7f423015c6d0 2023-08-25T09:47:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.218+0000 7f422f7fe640 1 --2- 172.21.15.70:0/3911989429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015c6d0 0x7f42300b1ff0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4228009e50 tx=0x7f422802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f45f3c7d5f8318b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.218+0000 7f422e7fc640 1 -- 172.21.15.70:0/3911989429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422802c9e0 con 0x7f423015c6d0 2023-08-25T09:47:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.218+0000 7f422e7fc640 1 -- 172.21.15.70:0/3911989429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422802cb40 con 0x7f423015c6d0 2023-08-25T09:47:26.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.218+0000 7f422e7fc640 1 -- 172.21.15.70:0/3911989429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42280343f0 con 0x7f423015c6d0 2023-08-25T09:47:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 -- 172.21.15.70:0/3911989429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015c6d0 msgr2=0x7f42300b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 --2- 172.21.15.70:0/3911989429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015c6d0 0x7f42300b1ff0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4228009e50 tx=0x7f422802be40 comp rx=0 tx=0).stop 2023-08-25T09:47:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 -- 172.21.15.70:0/3911989429 shutdown_connections 2023-08-25T09:47:26.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 --2- 172.21.15.70:0/3911989429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015c6d0 0x7f42300b1ff0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 --2- 172.21.15.70:0/3911989429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015ec10 0x7f423015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 -- 172.21.15.70:0/3911989429 >> 172.21.15.70:0/3911989429 conn(0x7f4230069850 msgr2=0x7f4230069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:26.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 -- 172.21.15.70:0/3911989429 shutdown_connections 2023-08-25T09:47:26.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.219+0000 7f423556a640 1 -- 172.21.15.70:0/3911989429 wait complete. 2023-08-25T09:47:26.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.221+0000 7f423556a640 1 Processor -- start 2023-08-25T09:47:26.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.221+0000 7f423556a640 1 -- start start 2023-08-25T09:47:26.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f423556a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423015c6d0 0x7f42300ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:26.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f423556a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015e830 0x7f42301580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:26.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f423556a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 0x7f4230070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:26.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f423556a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423015bf20 con 0x7f423015ec10 2023-08-25T09:47:26.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f423556a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42301f2230 con 0x7f423015c6d0 2023-08-25T09:47:26.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f423556a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423015bc40 con 0x7f423015e830 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 0x7f4230070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 0x7f4230070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54920/0 (socket says 172.21.15.70:54920) 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 -- 172.21.15.70:0/3461317167 learned_addr learned my addr 172.21.15.70:0/3461317167 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422effd640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423015c6d0 0x7f42300ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f4227fff640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015e830 0x7f42301580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 -- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015e830 msgr2=0x7f42301580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:26.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015e830 0x7f42301580f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 -- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423015c6d0 msgr2=0x7f42300ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:26.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423015c6d0 0x7f42300ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.222+0000 7f422f7fe640 1 -- 172.21.15.70:0/3461317167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228009b30 con 0x7f423015ec10 2023-08-25T09:47:26.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.223+0000 7f422f7fe640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 0x7f4230070140 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f4228036040 tx=0x7f4228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:26.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.223+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228009510 con 0x7f423015ec10 2023-08-25T09:47:26.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.223+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422803e450 con 0x7f423015ec10 2023-08-25T09:47:26.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.223+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42301f2440 con 0x7f423015ec10 2023-08-25T09:47:26.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.223+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422803e7e0 con 0x7f423015ec10 2023-08-25T09:47:26.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.223+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42301f2a00 con 0x7f423015ec10 2023-08-25T09:47:26.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.224+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42301f2c20 con 0x7f423015ec10 2023-08-25T09:47:26.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.225+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f422803e940 con 0x7f423015ec10 2023-08-25T09:47:26.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.226+0000 7f42277fe640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f421006a050 0x7f421006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:26.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.226+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f42280ab320 con 0x7f423015ec10 2023-08-25T09:47:26.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.227+0000 7f422effd640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f421006a050 0x7f421006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:26.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.227+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7f42100721a0 con 0x7f423015ec10 2023-08-25T09:47:26.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.227+0000 7f422effd640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f421006a050 0x7f421006c510 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4220004ea0 tx=0x7f4220009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:26.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.231+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4228043060 con 0x7f423015ec10 2023-08-25T09:47:26.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.422+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f423004e420 con 0x7f423015ec10 2023-08-25T09:47:26.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.426+0000 7f42277fe640 1 -- 172.21.15.70:0/3461317167 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v152) v1 ==== 81+0+37054 (secure 0 0 0) 0x7f4228043030 con 0x7f423015ec10 2023-08-25T09:47:26.429 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:26.430 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:26.430 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 152, 2023-08-25T09:47:26.430 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:26.430 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:26.430 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:22.383126+0000", 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:26.431 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:26.432 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:26.432 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:26.432 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:26.432 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:26.433 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:26.433 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:26.433 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:26.433 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:26.433 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:26.433 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:26.434 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:26.434 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:26.434 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:26.434 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:26.435 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:26.436 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:26.437 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:26.437 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:26.437 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:26.437 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:26.438 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:26.438 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:26.438 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:26.438 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:26.439 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:26.440 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:26.440 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:26.441 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:26.441 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:26.441 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:26.441 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:26.441 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:26.442 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:26.443 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:26.444 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:26.444 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:26.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.447 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:26.447 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:26.447 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:26.447 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:26.448 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.449 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:26.450 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:26.451 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:26.452 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:26.454 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:26.455 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:26.456 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:26.457 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:26.458 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:26.459 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:26.460 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:26.461 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:26.462 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:26.463 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:26.464 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:26.465 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:26.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:26.467 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:26.468 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:26.469 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:26.470 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:26.471 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:26.472 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:26.473 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.474 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:26.475 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:26.476 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:26.477 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "152", 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:26.478 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:26.479 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:26.480 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:26.481 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:26.482 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:26.483 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:26.485 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.486 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.487 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:26.490 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.493 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:26.495 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.497 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.498 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.498 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.498 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.498 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.498 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.499 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.499 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.500 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:26.502 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.505 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:26.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:26.507 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.508 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.508 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.510 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:26.510 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.510 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.511 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.511 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.511 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.513 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:26.513 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.514 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:26.516 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.516 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.521 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.522 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.523 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.525 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.526 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.528 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.530 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:26.530 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.530 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.530 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:26.531 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:26.531 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.531 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.531 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:26.531 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.533 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.535 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:26.538 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.539 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.541 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.542 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.543 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:26.543 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.543 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.544 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.545 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.545 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.545 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.545 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.545 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:26.546 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.547 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.548 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:26.548 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.549 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.549 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.551 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.553 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:26.554 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.556 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.557 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.557 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.557 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.557 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.557 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.557 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.559 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.561 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.563 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.564 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:26.565 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:26.567 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.568 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.569 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:26.570 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.571 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.571 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:26.571 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:26.572 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.573 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.574 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:26.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:26.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.574 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:26.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:26.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:26.576 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:26.577 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.579 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:26.579 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.579 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.579 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.579 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:26.580 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:26.582 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.583 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.584 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.586 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:26.587 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:26.588 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:26.589 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:26.590 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:26.590 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:26.590 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:26.590 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:26.590 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:26.590 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:26.591 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:26.592 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:26.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f421006a050 msgr2=0x7f421006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:26.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f421006a050 0x7f421006c510 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4220004ea0 tx=0x7f4220009040 comp rx=0 tx=0).stop 2023-08-25T09:47:26.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 msgr2=0x7f4230070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:26.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 0x7f4230070140 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f4228036040 tx=0x7f4228004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:26.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 shutdown_connections 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f421006a050 0x7f421006c510 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f423015ec10 0x7f4230070140 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f423015e830 0x7f42301580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 --2- 172.21.15.70:0/3461317167 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f423015c6d0 0x7f42300ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.429+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 >> 172.21.15.70:0/3461317167 conn(0x7f4230069850 msgr2=0x7f423015ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.430+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 shutdown_connections 2023-08-25T09:47:26.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:26.430+0000 7f423556a640 1 -- 172.21.15.70:0/3461317167 wait complete. 2023-08-25T09:47:26.596 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-08-25T09:47:26.597 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:47:26.597 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:26.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:26.659 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-08-25T09:47:26.883 INFO:teuthology.orchestra.run.smithi070.stderr:dd: error writing 'large_file_b': No space left on device 2023-08-25T09:47:26.883 INFO:teuthology.orchestra.run.smithi070.stderr:dd: closing output file 'large_file_b': No space left on device 2023-08-25T09:47:26.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:47:26.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.105+0000 7f898e5fd640 1 Processor -- start 2023-08-25T09:47:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.105+0000 7f898e5fd640 1 -- start start 2023-08-25T09:47:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.107+0000 7f898e5fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 0x7f89881594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.107+0000 7f898e5fd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898806ae10 con 0x7f8988159100 2023-08-25T09:47:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.107+0000 7f898e5fd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f898806b0b0 con 0x7f89880709a0 2023-08-25T09:47:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.107+0000 7f898e5fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898806ba50 con 0x7f89880b2070 2023-08-25T09:47:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.107+0000 7f8987fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8988159100 0x7f898806be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41094/0 (socket says 172.21.15.70:41094) 2023-08-25T09:47:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.107+0000 7f8987fff640 1 -- 172.21.15.70:0/2600022995 learned_addr learned my addr 172.21.15.70:0/2600022995 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289531067 0 0) 0x7f898806ae10 con 0x7f8988159100 2023-08-25T09:47:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8974003520 con 0x7f8988159100 2023-08-25T09:47:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f898cb9a640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 0x7f89881594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327565626 0 0) 0x7f8974003520 con 0x7f8988159100 2023-08-25T09:47:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 msgr2=0x7f89881594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:27.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 0x7f89881594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 >> v1:172.21.15.155:6789/0 conn(0x7f89880709a0 legacy=0x7f89880b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898806b390 con 0x7f8988159100 2023-08-25T09:47:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f89780019f0 con 0x7f8988159100 2023-08-25T09:47:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 >> v1:172.21.15.70:6789/0 conn(0x7f8988159100 legacy=0x7f898806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 0x7f89740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8974004d00 0x7f8974005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898806ba50 con 0x7f8974003fd0 2023-08-25T09:47:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897400bbc0 con 0x7f8974004d00 2023-08-25T09:47:27.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897400bd80 con 0x7f89880b2070 2023-08-25T09:47:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.108+0000 7f89877fe640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f8987fff640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 0x7f89740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f89877fe640 1 -- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 msgr2=0x7f89740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f89877fe640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89880b2070 0x7f89740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f89877fe640 1 -- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8974004d00 msgr2=0x7f8974005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:27.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f898cb9a640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8974004d00 0x7f8974005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f89877fe640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8974004d00 0x7f8974005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f89877fe640 1 -- 172.21.15.70:0/2600022995 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8974003520 con 0x7f8974003fd0 2023-08-25T09:47:27.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.109+0000 7f898cb9a640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8974004d00 0x7f8974005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:27.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.110+0000 7f89877fe640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89740047c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8970001720 tx=0x7f89740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705993dc50bced9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:27.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.110+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89700052d0 con 0x7f8974003fd0 2023-08-25T09:47:27.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.110+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897002e050 con 0x7f8974003fd0 2023-08-25T09:47:27.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.110+0000 7f8986ffd640 1 -- 172.21.15.70:0/2600022995 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8970003830 con 0x7f8974003fd0 2023-08-25T09:47:27.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898e5fd640 1 -- 172.21.15.70:0/2600022995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 msgr2=0x7f89740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898e5fd640 1 --2- 172.21.15.70:0/2600022995 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89740047c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f8970001720 tx=0x7f89740057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898cb9a640 1 -- 172.21.15.70:0/2600022995 reap_dead start 2023-08-25T09:47:27.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898e5fd640 1 -- 172.21.15.70:0/2600022995 shutdown_connections 2023-08-25T09:47:27.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898e5fd640 1 -- 172.21.15.70:0/2600022995 >> 172.21.15.70:0/2600022995 conn(0x7f8988069850 msgr2=0x7f8988069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:27.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898e5fd640 1 -- 172.21.15.70:0/2600022995 shutdown_connections 2023-08-25T09:47:27.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.111+0000 7f898e5fd640 1 -- 172.21.15.70:0/2600022995 wait complete. 2023-08-25T09:47:27.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.113+0000 7f898e5fd640 1 Processor -- start 2023-08-25T09:47:27.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.113+0000 7f898e5fd640 1 -- start start 2023-08-25T09:47:27.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f898e5fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89880cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f898e5fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89881524c0 0x7f89881594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f898e5fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89881e3ad0 0x7f89881e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f898e5fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898815b410 con 0x7f8974003fd0 2023-08-25T09:47:27.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f898e5fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881e4490 con 0x7f89881e3ad0 2023-08-25T09:47:27.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f898e5fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8988152940 con 0x7f89881524c0 2023-08-25T09:47:27.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f8987fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89880cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f8987fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89880cc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54938/0 (socket says 172.21.15.70:54938) 2023-08-25T09:47:27.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.114+0000 7f8987fff640 1 -- 172.21.15.70:0/1111087502 learned_addr learned my addr 172.21.15.70:0/1111087502 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8987fff640 1 -- 172.21.15.70:0/1111087502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89881524c0 msgr2=0x7f89881594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8987fff640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89881524c0 0x7f89881594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8987fff640 1 -- 172.21.15.70:0/1111087502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89881e3ad0 msgr2=0x7f89881e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8987fff640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89881e3ad0 0x7f89881e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8987fff640 1 -- 172.21.15.70:0/1111087502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988159a20 con 0x7f8974003fd0 2023-08-25T09:47:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8987fff640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89880cc050 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f8978007a10 tx=0x7f897800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800edf0 con 0x7f8974003fd0 2023-08-25T09:47:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978013420 con 0x7f8974003fd0 2023-08-25T09:47:27.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89780137d0 con 0x7f8974003fd0 2023-08-25T09:47:27.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.115+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8988153830 con 0x7f8974003fd0 2023-08-25T09:47:27.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.116+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8988153550 con 0x7f8974003fd0 2023-08-25T09:47:27.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.117+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898804e420 con 0x7f8974003fd0 2023-08-25T09:47:27.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.118+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8978013930 con 0x7f8974003fd0 2023-08-25T09:47:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.119+0000 7f8984ff9640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896806a080 0x7f896806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.119+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8978088cd0 con 0x7f8974003fd0 2023-08-25T09:47:27.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.119+0000 7f89877fe640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896806a080 0x7f896806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.119+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7f89680721d0 con 0x7f8974003fd0 2023-08-25T09:47:27.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.120+0000 7f89877fe640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896806a080 0x7f896806c540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8970033000 tx=0x7f897002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:27.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.123+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f89780543b0 con 0x7f8974003fd0 2023-08-25T09:47:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.313+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f89880cb300 con 0x7f8974003fd0 2023-08-25T09:47:27.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.315+0000 7f8984ff9640 1 -- 172.21.15.70:0/1111087502 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v152) v1 ==== 81+0+37054 (secure 0 0 0) 0x7f89780579d0 con 0x7f8974003fd0 2023-08-25T09:47:27.317 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 152, 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:22.383126+0000", 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:27.318 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:27.319 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:27.320 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.321 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:27.322 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:27.323 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:27.324 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:27.324 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:27.324 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:27.324 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:27.324 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:27.324 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:27.325 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:27.326 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:27.327 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:27.328 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:27.330 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:27.331 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:27.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:27.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:27.334 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:27.335 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:27.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:27.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:27.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:27.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:27.338 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:27.338 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:27.338 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:27.338 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:27.339 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:27.339 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:27.339 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:27.340 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:27.341 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:27.342 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:27.343 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:27.344 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.345 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:27.346 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:27.347 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:27.348 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:27.349 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:27.350 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:27.351 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:27.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:27.354 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:27.355 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:27.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:47:27.357 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:27.358 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:27.359 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "152", 2023-08-25T09:47:27.360 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:27.361 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:27.362 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:27.363 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:27.364 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:27.365 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.366 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:27.367 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:27.368 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.369 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:27.370 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:27.376 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:27.377 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.378 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.379 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:27.380 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:27.386 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:27.387 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.389 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.391 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.394 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.396 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.396 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:27.396 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.399 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.400 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.401 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:27.403 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.404 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.405 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.407 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.408 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.409 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.410 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:27.413 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.414 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:27.416 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.418 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.419 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.420 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.421 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:27.423 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.424 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.425 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.427 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.429 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.430 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.431 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.432 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.433 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:27.434 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.437 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:27.439 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:27.441 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.442 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:27.444 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:27.447 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:27.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.449 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.450 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.451 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:27.452 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.453 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.454 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:27.455 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:27.457 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:27.458 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:27.459 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:27.460 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:27.461 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:27.462 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:27.463 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:27.463 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896806a080 msgr2=0x7f896806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896806a080 0x7f896806c540 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8970033000 tx=0x7f897002f000 comp rx=0 tx=0).stop 2023-08-25T09:47:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 msgr2=0x7f89880cc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89880cc050 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f8978007a10 tx=0x7f897800e530 comp rx=0 tx=0).stop 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 shutdown_connections 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f896806a080 0x7f896806c540 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89881e3ad0 0x7f89881e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89881524c0 0x7f89881594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 --2- 172.21.15.70:0/1111087502 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8974003fd0 0x7f89880cc050 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 >> 172.21.15.70:0/1111087502 conn(0x7f8988069850 msgr2=0x7f89880ae9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.318+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 shutdown_connections 2023-08-25T09:47:27.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.319+0000 7f898e5fd640 1 -- 172.21.15.70:0/1111087502 wait complete. 2023-08-25T09:47:27.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:27.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.686+0000 7fd131784640 1 Processor -- start 2023-08-25T09:47:27.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.686+0000 7fd131784640 1 -- start start 2023-08-25T09:47:27.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd131784640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd12c0ce5b0 0x7fd12c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd131784640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd12c15aa10 0x7fd12c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd131784640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c0b2aa0 con 0x7fd12c0b2680 2023-08-25T09:47:27.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd131784640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c159d10 con 0x7fd12c15aa10 2023-08-25T09:47:27.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd131784640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0ceed0 con 0x7fd12c0ce5b0 2023-08-25T09:47:27.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd1227fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd12c15aa10 0x7fd12c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd1227fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd12c15aa10 0x7fd12c0b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54954/0 (socket says 172.21.15.70:54954) 2023-08-25T09:47:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.688+0000 7fd1227fc640 1 -- 172.21.15.70:0/875898872 learned_addr learned my addr 172.21.15.70:0/875898872 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.689+0000 7fd1227fc640 1 -- 172.21.15.70:0/875898872 >> v2:172.21.15.155:3300/0 conn(0x7fd12c0ce5b0 msgr2=0x7fd12c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.689+0000 7fd1227fc640 1 --2- 172.21.15.70:0/875898872 >> v2:172.21.15.155:3300/0 conn(0x7fd12c0ce5b0 0x7fd12c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.689+0000 7fd1227fc640 1 -- 172.21.15.70:0/875898872 >> v1:172.21.15.155:6789/0 conn(0x7fd12c0b2680 legacy=0x7fd12c06be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:27.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.689+0000 7fd1227fc640 1 -- 172.21.15.70:0/875898872 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c15d3f0 con 0x7fd12c15aa10 2023-08-25T09:47:27.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.690+0000 7fd1227fc640 1 --2- 172.21.15.70:0/875898872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c15aa10 0x7fd12c0b2140 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06f470 tx=0x7fd11402da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79aae3051d933523 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:27.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.690+0000 7fd12a7fc640 1 -- 172.21.15.70:0/875898872 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11402f850 con 0x7fd12c15aa10 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.690+0000 7fd12a7fc640 1 -- 172.21.15.70:0/875898872 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11402f9b0 con 0x7fd12c15aa10 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.690+0000 7fd12a7fc640 1 -- 172.21.15.70:0/875898872 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11402fd40 con 0x7fd12c15aa10 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 -- 172.21.15.70:0/875898872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c15aa10 msgr2=0x7fd12c0b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 --2- 172.21.15.70:0/875898872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c15aa10 0x7fd12c0b2140 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06f470 tx=0x7fd11402da90 comp rx=0 tx=0).stop 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 -- 172.21.15.70:0/875898872 shutdown_connections 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 --2- 172.21.15.70:0/875898872 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c15aa10 0x7fd12c0b2140 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 --2- 172.21.15.70:0/875898872 >> v2:172.21.15.155:3300/0 conn(0x7fd12c0ce5b0 0x7fd12c0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 -- 172.21.15.70:0/875898872 >> 172.21.15.70:0/875898872 conn(0x7fd12c069850 msgr2=0x7fd12c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:27.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 -- 172.21.15.70:0/875898872 shutdown_connections 2023-08-25T09:47:27.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.691+0000 7fd131784640 1 -- 172.21.15.70:0/875898872 wait complete. 2023-08-25T09:47:27.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.693+0000 7fd131784640 1 Processor -- start 2023-08-25T09:47:27.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.693+0000 7fd131784640 1 -- start start 2023-08-25T09:47:27.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd131784640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 0x7fd12c14b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd131784640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd12c0ce5b0 0x7fd12c0cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd131784640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd12c15aa10 0x7fd12c1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd131784640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0ccd60 con 0x7fd12c0b2680 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd131784640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c14b720 con 0x7fd12c15aa10 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd131784640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0cbe40 con 0x7fd12c0ce5b0 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 0x7fd12c14b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 0x7fd12c14b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54956/0 (socket says 172.21.15.70:54956) 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 -- 172.21.15.70:0/211718082 learned_addr learned my addr 172.21.15.70:0/211718082 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12b7fe640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd12c15aa10 0x7fd12c1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd1227fc640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd12c0ce5b0 0x7fd12c0cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 -- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd12c0ce5b0 msgr2=0x7fd12c0cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd12c0ce5b0 0x7fd12c0cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 -- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd12c15aa10 msgr2=0x7fd12c1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd12c15aa10 0x7fd12c1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12affd640 1 -- 172.21.15.70:0/211718082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140094d0 con 0x7fd12c0b2680 2023-08-25T09:47:27.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.694+0000 7fd12b7fe640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd12c15aa10 0x7fd12c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.695+0000 7fd12affd640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 0x7fd12c14b1e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00ba80 tx=0x7fd11c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.695+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00cc00 con 0x7fd12c0b2680 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.695+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c00cd80 con 0x7fd12c0b2680 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.695+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c0cc0e0 con 0x7fd12c0b2680 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.695+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c014680 con 0x7fd12c0b2680 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.695+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c0cc6a0 con 0x7fd12c0b2680 2023-08-25T09:47:27.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.696+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c0b1550 con 0x7fd12c0b2680 2023-08-25T09:47:27.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.697+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd11c01d3b0 con 0x7fd12c0b2680 2023-08-25T09:47:27.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.698+0000 7fd123fff640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd10c06a050 0x7fd10c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:27.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.698+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd11c0885a0 con 0x7fd12c0b2680 2023-08-25T09:47:27.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.698+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7fd10c0721a0 con 0x7fd12c0b2680 2023-08-25T09:47:27.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.699+0000 7fd1227fc640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd10c06a050 0x7fd10c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:27.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.699+0000 7fd1227fc640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd10c06a050 0x7fd10c06c510 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd114038000 tx=0x7fd114008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:27.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.703+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd11c053c80 con 0x7fd12c0b2680 2023-08-25T09:47:27.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.928+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd12c0ccea0 con 0x7fd12c0b2680 2023-08-25T09:47:27.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.929+0000 7fd123fff640 1 -- 172.21.15.70:0/211718082 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fd11c019030 con 0x7fd12c0b2680 2023-08-25T09:47:27.932 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:27.933 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:27.933 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:27.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd10c06a050 msgr2=0x7fd10c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd10c06a050 0x7fd10c06c510 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd114038000 tx=0x7fd114008030 comp rx=0 tx=0).stop 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 msgr2=0x7fd12c14b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 0x7fd12c14b1e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00ba80 tx=0x7fd11c00bf40 comp rx=0 tx=0).stop 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 shutdown_connections 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd10c06a050 0x7fd10c06c510 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd12c15aa10 0x7fd12c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd12c0ce5b0 0x7fd12c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 --2- 172.21.15.70:0/211718082 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd12c0b2680 0x7fd12c14b1e0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:27.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 >> 172.21.15.70:0/211718082 conn(0x7fd12c069850 msgr2=0x7fd12c156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:27.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 shutdown_connections 2023-08-25T09:47:27.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:27.932+0000 7fd131784640 1 -- 172.21.15.70:0/211718082 wait complete. 2023-08-25T09:47:27.952 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-08-25T09:47:28.195 INFO:teuthology.orchestra.run.smithi155.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":8232,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":29.358236077000001,"mdsmap_epoch":65,"osdmap_epoch":152,"osdmap_epoch_barrier":150,"uptime":35.756645763999998} 2023-08-25T09:47:28.196 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 8232, "mdsmap_epoch": 65, "osdmap_epoch": 152, "osdmap_epoch_barrier": 150, "rank_uptime": 29.358236077, "state": "up:active", "uptime": 35.756645764, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:47:28.196 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:47:28.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:28.197 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-08-25T09:47:28.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:28.270 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-08-25T09:47:28.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:28.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.557+0000 7f3630eec640 1 Processor -- start 2023-08-25T09:47:28.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.557+0000 7f3630eec640 1 -- start start 2023-08-25T09:47:28.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3630eec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f362c15cb40 0x7f362c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3630eec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 0x7f362c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3630eec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c06f050 con 0x7f362c0ac870 2023-08-25T09:47:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3630eec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c06f330 con 0x7f362c15cb40 2023-08-25T09:47:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3630eec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c06f610 con 0x7f362c0b2b40 2023-08-25T09:47:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3629d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f362c0ac870 0x7f362c160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41110/0 (socket says 172.21.15.70:41110) 2023-08-25T09:47:28.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.559+0000 7f3629d9b640 1 -- 172.21.15.70:0/2295477668 learned_addr learned my addr 172.21.15.70:0/2295477668 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362a59c640 1 --2- 172.21.15.70:0/2295477668 >> v2:172.21.15.155:3300/0 conn(0x7f362c15cb40 0x7f362c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911755691 0 0) 0x7f362c06f050 con 0x7f362c0ac870 2023-08-25T09:47:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3614003520 con 0x7f362c0ac870 2023-08-25T09:47:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362ad9d640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 0x7f362c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 351979619 0 0) 0x7f3614003520 con 0x7f362c0ac870 2023-08-25T09:47:28.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 msgr2=0x7f362c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 0x7f362c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 >> v2:172.21.15.155:3300/0 conn(0x7f362c15cb40 msgr2=0x7f362c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 --2- 172.21.15.70:0/2295477668 >> v2:172.21.15.155:3300/0 conn(0x7f362c15cb40 0x7f362c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c06f8f0 con 0x7f362c0ac870 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3620001670 con 0x7f362c0ac870 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 >> v1:172.21.15.70:6789/0 conn(0x7f362c0ac870 legacy=0x7f362c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3614003970 0x7f3614003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 0x7f36140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3614004d00 0x7f3614005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c06f330 con 0x7f3614004d00 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c06f610 con 0x7f3614003970 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.560+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361400b980 con 0x7f362c0b2b40 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362ad9d640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3614004d00 0x7f3614005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f3629d9b640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 0x7f36140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362a59c640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3614003970 0x7f3614003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362ad9d640 1 -- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 msgr2=0x7f36140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362ad9d640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f362c0b2b40 0x7f36140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362ad9d640 1 -- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3614003970 msgr2=0x7f3614003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362ad9d640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3614003970 0x7f3614003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362ad9d640 1 -- 172.21.15.70:0/2295477668 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614003520 con 0x7f3614004d00 2023-08-25T09:47:28.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.561+0000 7f362a59c640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3614003970 0x7f3614003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.562+0000 7f362ad9d640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3614004d00 0x7f3614005180 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3624007360 tx=0x7f36140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512c9a2149d7c47 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.562+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362402cc80 con 0x7f3614004d00 2023-08-25T09:47:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.562+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362402cde0 con 0x7f3614004d00 2023-08-25T09:47:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.562+0000 7f362959a640 1 -- 172.21.15.70:0/2295477668 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3624034660 con 0x7f3614004d00 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f3630eec640 1 -- 172.21.15.70:0/2295477668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3614004d00 msgr2=0x7f3614005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f3630eec640 1 --2- 172.21.15.70:0/2295477668 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3614004d00 0x7f3614005180 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f3624007360 tx=0x7f36140057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f362ad9d640 1 -- 172.21.15.70:0/2295477668 reap_dead start 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f3630eec640 1 -- 172.21.15.70:0/2295477668 shutdown_connections 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f3630eec640 1 -- 172.21.15.70:0/2295477668 >> 172.21.15.70:0/2295477668 conn(0x7f362c069850 msgr2=0x7f362c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f3630eec640 1 -- 172.21.15.70:0/2295477668 shutdown_connections 2023-08-25T09:47:28.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.563+0000 7f3630eec640 1 -- 172.21.15.70:0/2295477668 wait complete. 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.565+0000 7f3630eec640 1 Processor -- start 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.565+0000 7f3630eec640 1 -- start start 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3630eec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3614004d00 0x7f362c15ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3630eec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 0x7f362c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3630eec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c0b0690 0x7f362c06c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3630eec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c15b5c0 con 0x7f362c1557a0 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3630eec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c15a9b0 con 0x7f362c0b0690 2023-08-25T09:47:28.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3630eec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c15ac90 con 0x7f3614004d00 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 0x7f362c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3629d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 0x7f362c147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54988/0 (socket says 172.21.15.70:54988) 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.566+0000 7f3629d9b640 1 -- 172.21.15.70:0/1849518029 learned_addr learned my addr 172.21.15.70:0/1849518029 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3629d9b640 1 -- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3614004d00 msgr2=0x7f362c15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f362a59c640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3614004d00 0x7f362c15ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3629d9b640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3614004d00 0x7f362c15ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3629d9b640 1 -- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c0b0690 msgr2=0x7f362c06c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:28.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3629d9b640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c0b0690 0x7f362c06c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3629d9b640 1 -- 172.21.15.70:0/1849518029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c15af70 con 0x7f362c1557a0 2023-08-25T09:47:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3629d9b640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 0x7f362c147210 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f362000e840 tx=0x7f362000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362000c850 con 0x7f362c1557a0 2023-08-25T09:47:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362000c9b0 con 0x7f362c1557a0 2023-08-25T09:47:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362000cd60 con 0x7f362c1557a0 2023-08-25T09:47:28.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.567+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c15b250 con 0x7f362c1557a0 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.568+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c0b0d20 con 0x7f362c1557a0 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.569+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c04e420 con 0x7f362c1557a0 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.570+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f362001d460 con 0x7f362c1557a0 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.570+0000 7f361affd640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f360c06a050 0x7f360c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.571+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3620088600 con 0x7f362c1557a0 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.571+0000 7f362a59c640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f360c06a050 0x7f360c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:28.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.571+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7f360c0721a0 con 0x7f362c1557a0 2023-08-25T09:47:28.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.572+0000 7f362a59c640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f360c06a050 0x7f360c06c510 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f361c009a40 tx=0x7f361c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:28.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.575+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3620053ce0 con 0x7f362c1557a0 2023-08-25T09:47:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.766+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f362c15b700 con 0x7f362c1557a0 2023-08-25T09:47:28.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.768+0000 7f361affd640 1 -- 172.21.15.70:0/1849518029 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v152) v1 ==== 81+0+37054 (secure 0 0 0) 0x7f3620057300 con 0x7f362c1557a0 2023-08-25T09:47:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:28.772 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 152, 2023-08-25T09:47:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:22.383126+0000", 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "152", 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.870 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:28.870 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.870 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.872 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:28.874 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.876 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.878 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.879 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:28.880 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.882 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:28.883 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.885 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.886 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.887 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.889 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:28.894 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:28.895 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:28.897 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:28.898 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.900 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:28.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:28.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:28.905 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:28.906 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:28.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:28.908 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.909 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.910 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.913 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.914 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:28.915 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:28.917 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:28.918 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:28.919 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:28.920 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:28.921 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:28.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f360c06a050 msgr2=0x7f360c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:28.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f360c06a050 0x7f360c06c510 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f361c009a40 tx=0x7f361c008040 comp rx=0 tx=0).stop 2023-08-25T09:47:28.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 msgr2=0x7f362c147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:28.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 0x7f362c147210 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f362000e840 tx=0x7f362000ed00 comp rx=0 tx=0).stop 2023-08-25T09:47:28.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 shutdown_connections 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f360c06a050 0x7f360c06c510 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f362c0b0690 0x7f362c06c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f362c1557a0 0x7f362c147210 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 --2- 172.21.15.70:0/1849518029 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3614004d00 0x7f362c15ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.771+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 >> 172.21.15.70:0/1849518029 conn(0x7f362c069850 msgr2=0x7f362c161130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.772+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 shutdown_connections 2023-08-25T09:47:28.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:28.772+0000 7f3630eec640 1 -- 172.21.15.70:0/1849518029 wait complete. 2023-08-25T09:47:28.925 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:47:33.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.148+0000 7fef2e747640 1 Processor -- start 2023-08-25T09:47:34.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.148+0000 7fef2e747640 1 -- start start 2023-08-25T09:47:34.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2e747640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef2806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:34.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2e747640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef2806b2b0 0x7fef2815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:34.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2e747640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef280b1b60 con 0x7fef2815c6d0 2023-08-25T09:47:34.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2e747640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef2806bd80 con 0x7fef2806b2b0 2023-08-25T09:47:34.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2e747640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2806b7c0 con 0x7fef2815e830 2023-08-25T09:47:34.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef1ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fef2806b2b0 0x7fef2815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2cce4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fef2815c6d0 0x7fef280b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:36326/0 (socket says 172.21.15.70:36326) 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef2cce4640 1 -- 172.21.15.70:0/3495451133 learned_addr learned my addr 172.21.15.70:0/3495451133 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.150+0000 7fef27fff640 1 --2- 172.21.15.70:0/3495451133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef2806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.151+0000 7fef1ffff640 1 -- 172.21.15.70:0/3495451133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 msgr2=0x7fef2806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.151+0000 7fef1ffff640 1 --2- 172.21.15.70:0/3495451133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef2806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.151+0000 7fef1ffff640 1 -- 172.21.15.70:0/3495451133 >> v1:172.21.15.70:6789/0 conn(0x7fef2815c6d0 legacy=0x7fef280b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:34.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.151+0000 7fef1ffff640 1 -- 172.21.15.70:0/3495451133 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2806baa0 con 0x7fef2806b2b0 2023-08-25T09:47:34.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.151+0000 7fef277fe640 1 -- 172.21.15.70:0/3495451133 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876414218 0 0) 0x7fef280b1b60 con 0x7fef2815c6d0 2023-08-25T09:47:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.152+0000 7fef1ffff640 1 --2- 172.21.15.70:0/3495451133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef2815c190 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fef100097a0 tx=0x7fef1002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff4dea82add733 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.152+0000 7fef277fe640 1 -- 172.21.15.70:0/3495451133 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1002d040 con 0x7fef2806b2b0 2023-08-25T09:47:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.152+0000 7fef277fe640 1 -- 172.21.15.70:0/3495451133 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef10031810 con 0x7fef2806b2b0 2023-08-25T09:47:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.152+0000 7fef277fe640 1 -- 172.21.15.70:0/3495451133 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10031ba0 con 0x7fef2806b2b0 2023-08-25T09:47:34.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 -- 172.21.15.70:0/3495451133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 msgr2=0x7fef2815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 --2- 172.21.15.70:0/3495451133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef2815c190 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fef100097a0 tx=0x7fef1002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 -- 172.21.15.70:0/3495451133 shutdown_connections 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 --2- 172.21.15.70:0/3495451133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef2815c190 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 --2- 172.21.15.70:0/3495451133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef2806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 -- 172.21.15.70:0/3495451133 >> 172.21.15.70:0/3495451133 conn(0x7fef28069850 msgr2=0x7fef28069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 -- 172.21.15.70:0/3495451133 shutdown_connections 2023-08-25T09:47:34.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.153+0000 7fef2e747640 1 -- 172.21.15.70:0/3495451133 wait complete. 2023-08-25T09:47:34.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.155+0000 7fef2e747640 1 Processor -- start 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.155+0000 7fef2e747640 1 -- start start 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef2e747640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef281f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef2e747640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef2815c6d0 0x7fef281f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef2e747640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef28149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef2e747640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2814a2c0 con 0x7fef2806b2b0 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef2e747640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2814a5a0 con 0x7fef2815c6d0 2023-08-25T09:47:34.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef2e747640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2814a880 con 0x7fef2815e830 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef281f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef281f19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36680/0 (socket says 172.21.15.70:36680) 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 -- 172.21.15.70:0/2744960170 learned_addr learned my addr 172.21.15.70:0/2744960170 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 -- 172.21.15.70:0/2744960170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 msgr2=0x7fef28149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef28149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 -- 172.21.15.70:0/2744960170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef2815c6d0 msgr2=0x7fef281f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:34.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef2815c6d0 0x7fef281f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.156+0000 7fef27fff640 1 -- 172.21.15.70:0/2744960170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef10009400 con 0x7fef2806b2b0 2023-08-25T09:47:34.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.157+0000 7fef27fff640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef281f19c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d5f0 tx=0x7fef1800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.157+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180125f0 con 0x7fef2806b2b0 2023-08-25T09:47:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.157+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef18012750 con 0x7fef2806b2b0 2023-08-25T09:47:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.157+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18012b00 con 0x7fef2806b2b0 2023-08-25T09:47:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.157+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef281f5770 con 0x7fef2806b2b0 2023-08-25T09:47:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.157+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef281f1f00 con 0x7fef2806b2b0 2023-08-25T09:47:34.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.158+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2804e420 con 0x7fef2806b2b0 2023-08-25T09:47:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.159+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fef18012c60 con 0x7fef2806b2b0 2023-08-25T09:47:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.160+0000 7fef257fa640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef0806a050 0x7fef0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.161+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef18088a90 con 0x7fef2806b2b0 2023-08-25T09:47:34.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.161+0000 7fef1ffff640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef0806a050 0x7fef0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.161+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=153}) v3 -- 0x7fef080721a0 con 0x7fef2806b2b0 2023-08-25T09:47:34.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.161+0000 7fef1ffff640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef0806a050 0x7fef0806c510 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef10036000 tx=0x7fef10030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:34.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.165+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fef18054170 con 0x7fef2806b2b0 2023-08-25T09:47:34.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.357+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fef2806c620 con 0x7fef2806b2b0 2023-08-25T09:47:34.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.361+0000 7fef257fa640 1 -- 172.21.15.70:0/2744960170 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v152) v1 ==== 81+0+37054 (secure 0 0 0) 0x7fef18057790 con 0x7fef2806b2b0 2023-08-25T09:47:34.365 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:34.365 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 152, 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:22.383126+0000", 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:34.366 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:34.367 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:34.368 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:34.369 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:34.370 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:34.370 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:34.370 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:34.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:34.370 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:34.370 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:34.371 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:34.372 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:34.372 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:34.372 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:34.372 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:34.373 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:34.373 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:34.373 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:34.374 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:34.375 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:34.376 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:34.377 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:34.378 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:34.379 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:34.380 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.382 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:34.382 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:34.382 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:34.382 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:34.382 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:34.383 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:34.384 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:34.386 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:34.387 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:34.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:34.389 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:34.390 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:34.391 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:34.393 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:34.394 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:34.395 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:34.396 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:34.397 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:34.398 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:34.399 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:34.400 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:34.401 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:34.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:34.403 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:34.405 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:34.406 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:34.407 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "152", 2023-08-25T09:47:34.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:34.409 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:34.410 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:34.411 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:34.412 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:34.413 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.414 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:34.415 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:34.416 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.417 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:34.418 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.419 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:34.420 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.423 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.424 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.425 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.426 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.427 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.428 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.429 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.430 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.432 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:34.433 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.434 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:34.438 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.439 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:34.440 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.441 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.442 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.443 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.445 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:34.447 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.448 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:34.449 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.450 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.451 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.452 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.454 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.455 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:34.457 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.458 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.459 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.460 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.461 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.462 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.464 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.465 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:34.467 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.468 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.469 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.470 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.471 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.472 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.474 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.475 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:34.477 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.478 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.479 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.480 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.481 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.482 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.484 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.485 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:34.495 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:34.516 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:34.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef0806a050 msgr2=0x7fef0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef0806a050 0x7fef0806c510 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef10036000 tx=0x7fef10030000 comp rx=0 tx=0).stop 2023-08-25T09:47:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 msgr2=0x7fef281f19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef281f19c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fef1800d5f0 tx=0x7fef1800dab0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 shutdown_connections 2023-08-25T09:47:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fef0806a050 0x7fef0806c510 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fef2815e830 0x7fef28149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fef2815c6d0 0x7fef281f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 --2- 172.21.15.70:0/2744960170 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fef2806b2b0 0x7fef281f19c0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 >> 172.21.15.70:0/2744960170 conn(0x7fef28069850 msgr2=0x7fef280ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 shutdown_connections 2023-08-25T09:47:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:34.364+0000 7fef2e747640 1 -- 172.21.15.70:0/2744960170 wait complete. 2023-08-25T09:47:34.520 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:47:39.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:39.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.746+0000 7f25c6234640 1 Processor -- start 2023-08-25T09:47:39.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.746+0000 7f25c6234640 1 -- start start 2023-08-25T09:47:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.748+0000 7f25c6234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.748+0000 7f25c6234640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f25c015aa10 0x7f25c00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.748+0000 7f25c6234640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c00ceed0 con 0x7f25c00b2680 2023-08-25T09:47:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.748+0000 7f25c6234640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c00b2b10 con 0x7f25c015aa10 2023-08-25T09:47:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.748+0000 7f25c6234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c0159d10 con 0x7f25c00ce5b0 2023-08-25T09:47:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.749+0000 7f25bf7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.749+0000 7f25bf7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c00ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:56484/0 (socket says 172.21.15.70:56484) 2023-08-25T09:47:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.749+0000 7f25bf7fe640 1 -- 172.21.15.70:0/3855990171 learned_addr learned my addr 172.21.15.70:0/3855990171 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.749+0000 7f25bffff640 1 --1- 172.21.15.70:0/3855990171 >> v1:172.21.15.155:6789/0 conn(0x7f25c00b2680 0x7f25c006be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:43406/0 (socket says 172.21.15.70:43406) 2023-08-25T09:47:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.749+0000 7f25beffd640 1 --2- 172.21.15.70:0/3855990171 >> v2:172.21.15.155:3300/0 conn(0x7f25c015aa10 0x7f25c00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.750+0000 7f25beffd640 1 -- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 msgr2=0x7f25c00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.750+0000 7f25beffd640 1 --2- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c00ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.750+0000 7f25beffd640 1 -- 172.21.15.70:0/3855990171 >> v1:172.21.15.155:6789/0 conn(0x7f25c00b2680 legacy=0x7f25c006be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.750+0000 7f25beffd640 1 -- 172.21.15.70:0/3855990171 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c015d3f0 con 0x7f25c015aa10 2023-08-25T09:47:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.750+0000 7f25bf7fe640 1 --2- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.751+0000 7f25beffd640 1 --2- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c015aa10 0x7f25c00b2140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25b40097a0 tx=0x7f25b402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ecdaa32458cc400 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.752+0000 7f25be7fc640 1 -- 172.21.15.70:0/3855990171 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b402d040 con 0x7f25c015aa10 2023-08-25T09:47:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.752+0000 7f25be7fc640 1 -- 172.21.15.70:0/3855990171 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4031810 con 0x7f25c015aa10 2023-08-25T09:47:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.752+0000 7f25be7fc640 1 -- 172.21.15.70:0/3855990171 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4031ba0 con 0x7f25c015aa10 2023-08-25T09:47:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.752+0000 7f25c6234640 1 -- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c015aa10 msgr2=0x7f25c00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.752+0000 7f25c6234640 1 --2- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c015aa10 0x7f25c00b2140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f25b40097a0 tx=0x7f25b402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:47:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.753+0000 7f25c6234640 1 -- 172.21.15.70:0/3855990171 shutdown_connections 2023-08-25T09:47:39.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.753+0000 7f25c6234640 1 --2- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c015aa10 0x7f25c00b2140 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:39.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.753+0000 7f25c6234640 1 --2- 172.21.15.70:0/3855990171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:39.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.753+0000 7f25c6234640 1 -- 172.21.15.70:0/3855990171 >> 172.21.15.70:0/3855990171 conn(0x7f25c0069850 msgr2=0x7f25c0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.753+0000 7f25c6234640 1 -- 172.21.15.70:0/3855990171 shutdown_connections 2023-08-25T09:47:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.753+0000 7f25c6234640 1 -- 172.21.15.70:0/3855990171 wait complete. 2023-08-25T09:47:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.754+0000 7f25c6234640 1 Processor -- start 2023-08-25T09:47:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.755+0000 7f25c6234640 1 -- start start 2023-08-25T09:47:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.755+0000 7f25c6234640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c00b2680 0x7f25c00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25c6234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c0152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25c6234640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 0x7f25c014c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25c6234640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00b0e00 con 0x7f25c015aa10 2023-08-25T09:47:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25c6234640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00af820 con 0x7f25c00b2680 2023-08-25T09:47:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25c6234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c00afb00 con 0x7f25c00ce5b0 2023-08-25T09:47:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 0x7f25c014c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 0x7f25c014c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47516/0 (socket says 172.21.15.70:47516) 2023-08-25T09:47:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 -- 172.21.15.70:0/4259067919 learned_addr learned my addr 172.21.15.70:0/4259067919 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:39.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bf7fe640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c00b2680 0x7f25c00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:39.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25beffd640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c0152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:39.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 -- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 msgr2=0x7f25c0152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:39.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c0152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:39.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 -- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c00b2680 msgr2=0x7f25c00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:39.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c00b2680 0x7f25c00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:39.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.756+0000 7f25bffff640 1 -- 172.21.15.70:0/4259067919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00094d0 con 0x7f25c015aa10 2023-08-25T09:47:39.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.757+0000 7f25bffff640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 0x7f25c014c2f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f25b800c730 tx=0x7f25b800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:39.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.757+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004d60 con 0x7f25c015aa10 2023-08-25T09:47:39.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.757+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b800a640 con 0x7f25c015aa10 2023-08-25T09:47:39.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.757+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800a9f0 con 0x7f25c015aa10 2023-08-25T09:47:39.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.757+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b4009400 con 0x7f25c015aa10 2023-08-25T09:47:39.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.757+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c00b00c0 con 0x7f25c015aa10 2023-08-25T09:47:39.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.759+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c004e420 con 0x7f25c015aa10 2023-08-25T09:47:39.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.759+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f25b800ab50 con 0x7f25c015aa10 2023-08-25T09:47:39.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.760+0000 7f25a7fff640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f25ac06a050 0x7f25ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:39.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.760+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f25b8087890 con 0x7f25c015aa10 2023-08-25T09:47:39.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.761+0000 7f25bf7fe640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f25ac06a050 0x7f25ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:39.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.761+0000 7f25bf7fe640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f25ac06a050 0x7f25ac06c510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009cf0 tx=0x7f25b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:39.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.765+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f25b8056a10 con 0x7f25c015aa10 2023-08-25T09:47:39.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.959+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f25c00cae50 con 0x7f25c015aa10 2023-08-25T09:47:39.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.961+0000 7f25a7fff640 1 -- 172.21.15.70:0/4259067919 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v153) v1 ==== 81+0+37035 (secure 0 0 0) 0x7f25b80565a0 con 0x7f25c015aa10 2023-08-25T09:47:39.965 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:39.965 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:39.966 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 153, 2023-08-25T09:47:39.966 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:39.966 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:39.967 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:37.389506+0000", 2023-08-25T09:47:39.967 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:39.967 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:39.967 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:39.968 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:39.968 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:39.968 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:39.969 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:39.969 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:39.969 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:39.969 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:39.970 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:39.970 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:39.970 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:39.970 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:39.971 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:39.971 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:39.971 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:39.972 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:39.972 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:39.972 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:39.972 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:39.974 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:39.975 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:39.975 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:39.975 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:39.975 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:39.975 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:39.975 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:39.976 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:39.977 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:39.977 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:39.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.977 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:39.977 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:39.977 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:39.978 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:39.979 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:39.980 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:39.980 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:39.980 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:39.980 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:39.980 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:39.980 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:39.982 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:39.983 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:39.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.984 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:39.984 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:39.984 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:39.984 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:39.984 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.985 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:39.986 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:39.987 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:39.988 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:39.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:39.990 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:39.991 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:39.992 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:39.993 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:39.994 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:39.995 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:39.996 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:39.997 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:39.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:39.999 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:40.000 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:40.001 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:40.002 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:40.003 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:40.004 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:40.005 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:40.007 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:40.008 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:40.009 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:40.010 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:40.011 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:40.012 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:40.013 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:40.014 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:40.016 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:40.017 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "153", 2023-08-25T09:47:40.018 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:40.019 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:40.020 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:40.021 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:40.022 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:40.023 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:40.023 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:40.023 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:40.024 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:40.025 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:40.026 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:40.027 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:40.028 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:40.030 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:40.032 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:40.033 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.034 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.035 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.036 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.038 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.041 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.042 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.043 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.044 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:40.045 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.046 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.047 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.048 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.049 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.050 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.051 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.053 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:40.054 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.055 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.056 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.057 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.058 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:40.060 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.062 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.063 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.064 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:40.065 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.066 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:40.067 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.068 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.069 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.070 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:40.071 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.072 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.073 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:40.073 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.073 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.074 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.077 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.078 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.079 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.080 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.082 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.083 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.084 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.085 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.086 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.087 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.092 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.093 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.094 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:40.095 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.096 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:40.097 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.098 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:40.099 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.100 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:40.101 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.102 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.103 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.104 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.105 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.106 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.107 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.108 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.109 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:40.110 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.111 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:40.112 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.113 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.114 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.115 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.116 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.117 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.118 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.119 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:40.120 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.122 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.124 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:40.125 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.126 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.129 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.135 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:40.136 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.137 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.138 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.139 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.140 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.141 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.142 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:40.143 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.144 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.145 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:40.146 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.147 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.148 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:40.149 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:40.150 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:40.151 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:40.152 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:40.153 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:40.154 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:40.154 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:40.154 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.154 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f25ac06a050 msgr2=0x7f25ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f25ac06a050 0x7f25ac06c510 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009cf0 tx=0x7f25b0008040 comp rx=0 tx=0).stop 2023-08-25T09:47:40.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 msgr2=0x7f25c014c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 0x7f25c014c2f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f25b800c730 tx=0x7f25b800cb30 comp rx=0 tx=0).stop 2023-08-25T09:47:40.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 shutdown_connections 2023-08-25T09:47:40.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f25ac06a050 0x7f25ac06c510 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f25c015aa10 0x7f25c014c2f0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f25c00ce5b0 0x7f25c0152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 --2- 172.21.15.70:0/4259067919 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f25c00b2680 0x7f25c00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.964+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 >> 172.21.15.70:0/4259067919 conn(0x7f25c0069850 msgr2=0x7f25c006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:40.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.965+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 shutdown_connections 2023-08-25T09:47:40.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:39.965+0000 7f25c6234640 1 -- 172.21.15.70:0/4259067919 wait complete. 2023-08-25T09:47:40.158 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-08-25T09:47:40.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:47:40.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.376+0000 7f982b7b5640 1 Processor -- start 2023-08-25T09:47:40.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.376+0000 7f982b7b5640 1 -- start start 2023-08-25T09:47:40.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f982b7b5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f982415cb40 0x7f98240ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f982b7b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 0x7f98240ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f982b7b5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98240b3880 0x7f9824160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f982b7b5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982406f050 con 0x7f98240b3880 2023-08-25T09:47:40.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f982b7b5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f982406f330 con 0x7f982415cb40 2023-08-25T09:47:40.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f982b7b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982406f610 con 0x7f98240ac870 2023-08-25T09:47:40.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f9829d52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98240b3880 0x7f9824160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:40.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f9829d52640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f98240b3880 0x7f9824160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47518/0 (socket says 172.21.15.70:47518) 2023-08-25T09:47:40.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.378+0000 7f9829d52640 1 -- 172.21.15.70:0/3601501076 learned_addr learned my addr 172.21.15.70:0/3601501076 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:40.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.379+0000 7f9829551640 1 --2- 172.21.15.70:0/3601501076 >> v2:172.21.15.155:3300/0 conn(0x7f982415cb40 0x7f98240ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:40.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.379+0000 7f9829d52640 1 -- 172.21.15.70:0/3601501076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 msgr2=0x7f98240ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:40.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.379+0000 7f9829d52640 1 --2- 172.21.15.70:0/3601501076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 0x7f98240ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.379+0000 7f9829d52640 1 -- 172.21.15.70:0/3601501076 >> v2:172.21.15.155:3300/0 conn(0x7f982415cb40 msgr2=0x7f98240ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:40.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.379+0000 7f9829d52640 1 --2- 172.21.15.70:0/3601501076 >> v2:172.21.15.155:3300/0 conn(0x7f982415cb40 0x7f98240ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.379+0000 7f9829d52640 1 -- 172.21.15.70:0/3601501076 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982406f8f0 con 0x7f98240b3880 2023-08-25T09:47:40.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.380+0000 7f9829d52640 1 --2- 172.21.15.70:0/3601501076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98240b3880 0x7f9824160790 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f982406a7b0 tx=0x7f981c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad5d5836e12c229 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:40.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.380+0000 7f981bfff640 1 -- 172.21.15.70:0/3601501076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c02ca70 con 0x7f98240b3880 2023-08-25T09:47:40.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.380+0000 7f981bfff640 1 -- 172.21.15.70:0/3601501076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c02cbd0 con 0x7f98240b3880 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.380+0000 7f981bfff640 1 -- 172.21.15.70:0/3601501076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c0343f0 con 0x7f98240b3880 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 -- 172.21.15.70:0/3601501076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98240b3880 msgr2=0x7f9824160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 --2- 172.21.15.70:0/3601501076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98240b3880 0x7f9824160790 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f982406a7b0 tx=0x7f981c02bed0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 -- 172.21.15.70:0/3601501076 shutdown_connections 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 --2- 172.21.15.70:0/3601501076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f98240b3880 0x7f9824160790 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 --2- 172.21.15.70:0/3601501076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 0x7f98240ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 --2- 172.21.15.70:0/3601501076 >> v2:172.21.15.155:3300/0 conn(0x7f982415cb40 0x7f98240ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 -- 172.21.15.70:0/3601501076 >> 172.21.15.70:0/3601501076 conn(0x7f9824069850 msgr2=0x7f9824069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:40.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 -- 172.21.15.70:0/3601501076 shutdown_connections 2023-08-25T09:47:40.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.381+0000 7f982b7b5640 1 -- 172.21.15.70:0/3601501076 wait complete. 2023-08-25T09:47:40.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.383+0000 7f982b7b5640 1 Processor -- start 2023-08-25T09:47:40.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.383+0000 7f982b7b5640 1 -- start start 2023-08-25T09:47:40.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f982b7b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 0x7f982414b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f982b7b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98240b3880 0x7f982406b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f982b7b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 0x7f982406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f982b7b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982414b720 con 0x7f982415cb40 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f982b7b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98241518e0 con 0x7f98240b3880 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f982b7b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9824151bc0 con 0x7f98240ac870 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f9829d52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 0x7f982406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f9829d52640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 0x7f982406ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47532/0 (socket says 172.21.15.70:47532) 2023-08-25T09:47:40.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.384+0000 7f9829d52640 1 -- 172.21.15.70:0/4019866640 learned_addr learned my addr 172.21.15.70:0/4019866640 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9829d52640 1 -- 172.21.15.70:0/4019866640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 msgr2=0x7f982414b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9829d52640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 0x7f982414b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9829d52640 1 -- 172.21.15.70:0/4019866640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98240b3880 msgr2=0x7f982406b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9829d52640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98240b3880 0x7f982406b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9829d52640 1 -- 172.21.15.70:0/4019866640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c009b30 con 0x7f982415cb40 2023-08-25T09:47:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9829d52640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 0x7f982406ba00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f981c036040 tx=0x7f981c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:40.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c009510 con 0x7f982415cb40 2023-08-25T09:47:40.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981c03e450 con 0x7f982415cb40 2023-08-25T09:47:40.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c03e800 con 0x7f982415cb40 2023-08-25T09:47:40.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.385+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9824151ea0 con 0x7f982415cb40 2023-08-25T09:47:40.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.386+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f982414f3d0 con 0x7f982415cb40 2023-08-25T09:47:40.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.387+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9824069c30 con 0x7f982415cb40 2023-08-25T09:47:40.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.388+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f981c03e960 con 0x7f982415cb40 2023-08-25T09:47:40.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.389+0000 7f9819ffb640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f980806a050 0x7f980806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.389+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f981c0ab4d0 con 0x7f982415cb40 2023-08-25T09:47:40.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.389+0000 7f9829551640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f980806a050 0x7f980806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:40.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.390+0000 7f9829551640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f980806a050 0x7f980806c510 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f982000a810 tx=0x7f9820008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:40.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.393+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f981c076d00 con 0x7f982415cb40 2023-08-25T09:47:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.584+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9824152360 con 0x7f982415cb40 2023-08-25T09:47:40.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.588+0000 7f9819ffb640 1 -- 172.21.15.70:0/4019866640 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v153) v1 ==== 81+0+37035 (secure 0 0 0) 0x7f981c07a320 con 0x7f982415cb40 2023-08-25T09:47:40.590 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:40.590 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:47:40.590 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 153, 2023-08-25T09:47:40.590 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:47:40.591 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:47:40.591 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:47:37.389506+0000", 2023-08-25T09:47:40.591 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:47:40.591 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:47:40.591 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:47:40.591 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.592 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:47:40.593 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:47:40.593 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:47:40.593 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:47:40.593 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:47:40.593 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 18, 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:47:40.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.595 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:47:40.595 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:47:40.595 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:40.596 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:40.597 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:40.597 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:40.597 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:40.597 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:40.597 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:40.598 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:47:40.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:47:40.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:47:40.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:47:40.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:47:40.600 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:40.601 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:40.601 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:40.601 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:40.601 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:40.602 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:40.602 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:40.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.602 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:40.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:40.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:40.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:40.603 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:40.604 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:40.604 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:40.605 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:40.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:40.605 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:40.606 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:40.606 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:40.606 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:40.606 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:40.607 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:40.607 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:40.607 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:40.608 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:40.608 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:40.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:40.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:40.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:40.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:40.610 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:40.610 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:40.610 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:40.610 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:40.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.611 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:40.611 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:40.612 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:40.612 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:40.612 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:40.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:40.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:47:40.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:47:40.614 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:40.616 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:40.617 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:40.618 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:40.619 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:40.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:40.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:40.619 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:40.620 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:40.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:47:40.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:40.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:40.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:40.621 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:40.622 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:40.623 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:40.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:40.625 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:47:40.626 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 17, 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:56.533109+0000", 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:40.628 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:40.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:40.630 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:47:40.630 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:47:40.630 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:47:40.630 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:47:40.631 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:47:40.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:40.631 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:40.631 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:40.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:40.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:40.634 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:40.634 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:40.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "149", 2023-08-25T09:47:40.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:40.635 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:40.636 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:40.636 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:47:40.636 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:40.636 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:40.636 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:40.637 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:40.638 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:40.638 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:40.638 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:40.638 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:40.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:40.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.639 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:40.639 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:40.639 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:40.640 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:40.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:40.642 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:47:40.642 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.642 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:40.642 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:47:40.642 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:40.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.644 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 18, 2023-08-25T09:47:40.644 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:47:40.644 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:46:57.303907+0000", 2023-08-25T09:47:40.644 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:47:40.645 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:47:40.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:47:40.647 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:47:40.648 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:47:40.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "153", 2023-08-25T09:47:40.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:47:40.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:47:40.649 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:47:40.650 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:47:40.650 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:47:40.650 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:47:40.650 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:47:40.651 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:47:40.652 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:47:40.652 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:47:40.652 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:47:40.652 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:47:40.652 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:47:40.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:47:40.653 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:47:40.654 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:47:40.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.655 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:47:40.655 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:47:40.655 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:47:40.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.3799999952316284, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.3799999952316284, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.3799999952316284, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.3799999952316284, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:47:40.656 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:40.657 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.658 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.660 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.661 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:47:40.662 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:47:40.662 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.663 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.663 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.663 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:47:40.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.665 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.667 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:47:40.668 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:47:40.668 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:47:40.668 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:47:40.668 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.668 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.668 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.669 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.670 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.672 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:47:40.673 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.674 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.674 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:47:40.674 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.674 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:47:40.675 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.676 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:47:40.677 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.678 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:47:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:47:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:47:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:47:40.679 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:40.680 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.681 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.681 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.681 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.681 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.682 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.684 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:47:40.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.687 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.687 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:47:40.687 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.688 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:47:40.690 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:47:40.690 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.690 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.691 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.692 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.693 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:47:40.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.695 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.697 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.697 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:47:40.697 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.697 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.697 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.699 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.699 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:47:40.699 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.699 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.699 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.700 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.700 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.700 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:47:40.701 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:47:40.702 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.703 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.703 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.703 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.703 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:40.704 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.704 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.704 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.704 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.704 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:47:40.706 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.707 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.708 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.708 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:47:40.708 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.709 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.709 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:47:40.709 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.709 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.709 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:47:40.710 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.711 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.711 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:47:40.711 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.711 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.712 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.712 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.712 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.712 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.712 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:47:40.713 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:47:40.714 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.715 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.716 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:47:40.717 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.718 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.718 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:47:40.718 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.718 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.719 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.719 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.719 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.719 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:47:40.720 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.721 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.722 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:47:40.723 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.724 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.725 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.726 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.727 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.729 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:47:40.730 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.733 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:47:40.734 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.735 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.736 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.737 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:40.737 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:47:40.737 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:47:40.737 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 148, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:47:40.738 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:47:40.742 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:47:40.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:47:40.745 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:47:40.746 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:47:40.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.748 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.750 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.751 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:47:40.752 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.753 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.754 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:47:40.755 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:47:40.757 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:47:40.758 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:47:40.759 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:47:40.760 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:47:40.760 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:47:40.760 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:47:40.760 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:47:40.761 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:47:40.762 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:47:40.762 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:47:40.762 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:47:40.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.590+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f980806a050 msgr2=0x7f980806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.590+0000 7f982b7b5640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f980806a050 0x7f980806c510 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f982000a810 tx=0x7f9820008040 comp rx=0 tx=0).stop 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.590+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 msgr2=0x7f982406ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.590+0000 7f982b7b5640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 0x7f982406ba00 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f981c036040 tx=0x7f981c004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 shutdown_connections 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f980806a050 0x7f980806c510 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f982415cb40 0x7f982406ba00 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f98240b3880 0x7f982406b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 --2- 172.21.15.70:0/4019866640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f98240ac870 0x7f982414b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 >> 172.21.15.70:0/4019866640 conn(0x7f9824069850 msgr2=0x7f98240accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:40.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 shutdown_connections 2023-08-25T09:47:40.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.591+0000 7f982b7b5640 1 -- 172.21.15.70:0/4019866640 wait complete. 2023-08-25T09:47:40.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:40.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.986+0000 7f14ff5e3640 1 Processor -- start 2023-08-25T09:47:40.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.986+0000 7f14ff5e3640 1 -- start start 2023-08-25T09:47:40.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14ff5e3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14f815e830 0x7f14f806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14ff5e3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f80b1b60 con 0x7f14f806b2b0 2023-08-25T09:47:40.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14ff5e3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f806b7c0 con 0x7f14f815ec10 2023-08-25T09:47:40.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14ff5e3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f806bd80 con 0x7f14f815e830 2023-08-25T09:47:40.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14fd37f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14f815e830 0x7f14f806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:40.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14fd37f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f14f815e830 0x7f14f806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47548/0 (socket says 172.21.15.70:47548) 2023-08-25T09:47:40.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14fd37f640 1 -- 172.21.15.70:0/628936942 learned_addr learned my addr 172.21.15.70:0/628936942 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:40.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.988+0000 7f14fcb7e640 1 --1- 172.21.15.70:0/628936942 >> v1:172.21.15.70:6789/0 conn(0x7f14f806b2b0 0x7f14f815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41380/0 (socket says 172.21.15.70:41380) 2023-08-25T09:47:40.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.989+0000 7f14fd37f640 1 -- 172.21.15.70:0/628936942 >> v1:172.21.15.155:6789/0 conn(0x7f14f815ec10 legacy=0x7f14f80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:40.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.989+0000 7f14fd37f640 1 -- 172.21.15.70:0/628936942 >> v1:172.21.15.70:6789/0 conn(0x7f14f806b2b0 legacy=0x7f14f815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.989+0000 7f14fd37f640 1 -- 172.21.15.70:0/628936942 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f806baa0 con 0x7f14f815e830 2023-08-25T09:47:40.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.989+0000 7f14effff640 1 -- 172.21.15.70:0/628936942 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2191004450 0 0) 0x7f14f80b1b60 con 0x7f14f806b2b0 2023-08-25T09:47:40.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.990+0000 7f14fd37f640 1 --2- 172.21.15.70:0/628936942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e830 0x7f14f806ad70 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e50 tx=0x7f14f402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0052a2a061ad92 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.990+0000 7f14effff640 1 -- 172.21.15.70:0/628936942 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f402cb20 con 0x7f14f815e830 2023-08-25T09:47:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.990+0000 7f14effff640 1 -- 172.21.15.70:0/628936942 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f402cc80 con 0x7f14f815e830 2023-08-25T09:47:40.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.990+0000 7f14effff640 1 -- 172.21.15.70:0/628936942 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f40343f0 con 0x7f14f815e830 2023-08-25T09:47:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 -- 172.21.15.70:0/628936942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e830 msgr2=0x7f14f806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/628936942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e830 0x7f14f806ad70 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009e50 tx=0x7f14f402bf80 comp rx=0 tx=0).stop 2023-08-25T09:47:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 -- 172.21.15.70:0/628936942 shutdown_connections 2023-08-25T09:47:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/628936942 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f815e830 0x7f14f806ad70 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 -- 172.21.15.70:0/628936942 >> 172.21.15.70:0/628936942 conn(0x7f14f8069850 msgr2=0x7f14f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:40.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 -- 172.21.15.70:0/628936942 shutdown_connections 2023-08-25T09:47:40.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.991+0000 7f14ff5e3640 1 -- 172.21.15.70:0/628936942 wait complete. 2023-08-25T09:47:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.993+0000 7f14ff5e3640 1 Processor -- start 2023-08-25T09:47:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.993+0000 7f14ff5e3640 1 -- start start 2023-08-25T09:47:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14ff5e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 0x7f14f80b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14ff5e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f815e830 0x7f14f80b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:40.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14ff5e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f815ec10 0x7f14f80c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14ff5e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f81e8810 con 0x7f14f806b2b0 2023-08-25T09:47:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14ff5e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f8146a30 con 0x7f14f815ec10 2023-08-25T09:47:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14ff5e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f81e8db0 con 0x7f14f815e830 2023-08-25T09:47:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14fd37f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 0x7f14f80b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14fd37f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 0x7f14f80b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47558/0 (socket says 172.21.15.70:47558) 2023-08-25T09:47:41.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.994+0000 7f14fd37f640 1 -- 172.21.15.70:0/2457184519 learned_addr learned my addr 172.21.15.70:0/2457184519 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14fd37f640 1 -- 172.21.15.70:0/2457184519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f815e830 msgr2=0x7f14f80b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14fd37f640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f815e830 0x7f14f80b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14fd37f640 1 -- 172.21.15.70:0/2457184519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f815ec10 msgr2=0x7f14f80c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14fd37f640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f815ec10 0x7f14f80c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14fd37f640 1 -- 172.21.15.70:0/2457184519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f4009b30 con 0x7f14f806b2b0 2023-08-25T09:47:41.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14fd37f640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 0x7f14f80b0330 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f14f4036040 tx=0x7f14f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4009510 con 0x7f14f806b2b0 2023-08-25T09:47:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f403e450 con 0x7f14f806b2b0 2023-08-25T09:47:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f403e800 con 0x7f14f806b2b0 2023-08-25T09:47:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.995+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f815ca20 con 0x7f14f806b2b0 2023-08-25T09:47:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.996+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f8146750 con 0x7f14f806b2b0 2023-08-25T09:47:41.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.997+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f804e420 con 0x7f14f806b2b0 2023-08-25T09:47:41.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.998+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14f403e960 con 0x7f14f806b2b0 2023-08-25T09:47:41.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.999+0000 7f14edffb640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14dc06a080 0x7f14dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.999+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14f40aafe0 con 0x7f14f806b2b0 2023-08-25T09:47:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:40.999+0000 7f14fcb7e640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14dc06a080 0x7f14dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:41.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.000+0000 7f14fcb7e640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14dc06a080 0x7f14dc06c540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007fd0 tx=0x7f14e8007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:41.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.003+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f14f4076730 con 0x7f14f806b2b0 2023-08-25T09:47:41.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.226+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14f80cef10 con 0x7f14f806b2b0 2023-08-25T09:47:41.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.228+0000 7f14edffb640 1 -- 172.21.15.70:0/2457184519 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f14f4079d50 con 0x7f14f806b2b0 2023-08-25T09:47:41.230 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:41.230 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:41.230 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:41.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14dc06a080 msgr2=0x7f14dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14dc06a080 0x7f14dc06c540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f14e8007fd0 tx=0x7f14e8007920 comp rx=0 tx=0).stop 2023-08-25T09:47:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 msgr2=0x7f14f80b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 0x7f14f80b0330 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f14f4036040 tx=0x7f14f4004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 shutdown_connections 2023-08-25T09:47:41.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f14dc06a080 0x7f14dc06c540 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f14f815ec10 0x7f14f80c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f14f815e830 0x7f14f80b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 --2- 172.21.15.70:0/2457184519 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f14f806b2b0 0x7f14f80b0330 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 >> 172.21.15.70:0/2457184519 conn(0x7f14f8069850 msgr2=0x7f14f815b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 shutdown_connections 2023-08-25T09:47:41.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:41.230+0000 7f14ff5e3640 1 -- 172.21.15.70:0/2457184519 wait complete. 2023-08-25T09:47:41.250 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json status 2023-08-25T09:47:41.493 INFO:teuthology.orchestra.run.smithi155.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":8232,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":42.656025147000001,"mdsmap_epoch":65,"osdmap_epoch":153,"osdmap_epoch_barrier":152,"uptime":49.054442504999997} 2023-08-25T09:47:41.494 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 8232, "mdsmap_epoch": 65, "osdmap_epoch": 153, "osdmap_epoch_barrier": 152, "rank_uptime": 42.656025147, "state": "up:active", "uptime": 49.054442505, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:47:41.494 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:47:41.495 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:47:41.495 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:41.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:41.557 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-08-25T09:47:42.568 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records in 2023-08-25T09:47:42.569 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records out 2023-08-25T09:47:42.569 INFO:teuthology.orchestra.run.smithi070.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.947391 s, 55.3 MB/s 2023-08-25T09:47:42.570 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:47:42.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:47:42.695 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:47:42.830 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:47:42.831 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:47:42.832 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:42.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:43.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.111+0000 7f123b35d640 1 Processor -- start 2023-08-25T09:47:43.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.112+0000 7f123b35d640 1 -- start start 2023-08-25T09:47:43.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.113+0000 7f123b35d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123415cb40 0x7f12340ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.113+0000 7f123b35d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f12340ac870 0x7f12340ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.113+0000 7f123b35d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f123406f050 con 0x7f12340b3880 2023-08-25T09:47:43.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.113+0000 7f123b35d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123406f330 con 0x7f12340ac870 2023-08-25T09:47:43.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.113+0000 7f123b35d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f123406f610 con 0x7f123415cb40 2023-08-25T09:47:43.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f12398fa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f12340b3880 0x7f1234160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41382/0 (socket says 172.21.15.70:41382) 2023-08-25T09:47:43.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f12398fa640 1 -- 172.21.15.70:0/3326239465 learned_addr learned my addr 172.21.15.70:0/3326239465 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:43.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f12388f8640 1 --2- 172.21.15.70:0/3326239465 >> v2:172.21.15.155:3300/0 conn(0x7f12340ac870 0x7f12340ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190844860 0 0) 0x7f123406f050 con 0x7f12340b3880 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1220003520 con 0x7f12340b3880 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f12390f9640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123415cb40 0x7f12340ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3933658309 0 0) 0x7f1220003520 con 0x7f12340b3880 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123415cb40 msgr2=0x7f12340ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123415cb40 0x7f12340ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 >> v2:172.21.15.155:3300/0 conn(0x7f12340ac870 msgr2=0x7f12340ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 --2- 172.21.15.70:0/3326239465 >> v2:172.21.15.155:3300/0 conn(0x7f12340ac870 0x7f12340ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f123406f8f0 con 0x7f12340b3880 2023-08-25T09:47:43.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f122c0019f0 con 0x7f12340b3880 2023-08-25T09:47:43.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.114+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 >> v1:172.21.15.70:6789/0 conn(0x7f12340b3880 legacy=0x7f1234160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f122bfff640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220003970 0x7f1220003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f122bfff640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12200047c0 0x7f1220004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f122bfff640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220005180 0x7f122000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123406f330 con 0x7f12200047c0 2023-08-25T09:47:43.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f123406f610 con 0x7f1220003970 2023-08-25T09:47:43.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122000bff0 con 0x7f1220005180 2023-08-25T09:47:43.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12390f9640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f123415cb40 0x7f12340ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:43.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12388f8640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12200047c0 0x7f1220004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12388f8640 1 -- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220005180 msgr2=0x7f122000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12388f8640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1220005180 0x7f122000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12388f8640 1 -- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220003970 msgr2=0x7f1220003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12388f8640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1220003970 0x7f1220003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.115+0000 7f12388f8640 1 -- 172.21.15.70:0/3326239465 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220003520 con 0x7f12200047c0 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.116+0000 7f12388f8640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12200047c0 0x7f1220004c40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f1224009ac0 tx=0x7f12200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d7cac1fd25b07 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.116+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122402c710 con 0x7f12200047c0 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122402c870 con 0x7f12200047c0 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f122bfff640 1 -- 172.21.15.70:0/3326239465 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122402cc00 con 0x7f12200047c0 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f123b35d640 1 -- 172.21.15.70:0/3326239465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12200047c0 msgr2=0x7f1220004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f123b35d640 1 --2- 172.21.15.70:0/3326239465 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12200047c0 0x7f1220004c40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f1224009ac0 tx=0x7f12200057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f12398fa640 1 -- 172.21.15.70:0/3326239465 reap_dead start 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f123b35d640 1 -- 172.21.15.70:0/3326239465 shutdown_connections 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f123b35d640 1 -- 172.21.15.70:0/3326239465 >> 172.21.15.70:0/3326239465 conn(0x7f1234069850 msgr2=0x7f1234069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f123b35d640 1 -- 172.21.15.70:0/3326239465 shutdown_connections 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.117+0000 7f123b35d640 1 -- 172.21.15.70:0/3326239465 wait complete. 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.119+0000 7f123b35d640 1 Processor -- start 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.119+0000 7f123b35d640 1 -- start start 2023-08-25T09:47:43.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f123b35d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12200047c0 0x7f1234156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f123b35d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 0x7f1234154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f123b35d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1234155460 0x7f1234152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f123b35d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12341558f0 con 0x7f12341591b0 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f123b35d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1234159ee0 con 0x7f12200047c0 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f123b35d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12340cabc0 con 0x7f1234155460 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f12388f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 0x7f1234154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f12388f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 0x7f1234154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47588/0 (socket says 172.21.15.70:47588) 2023-08-25T09:47:43.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.120+0000 7f12388f8640 1 -- 172.21.15.70:0/296243873 learned_addr learned my addr 172.21.15.70:0/296243873 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12388f8640 1 -- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1234155460 msgr2=0x7f1234152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12398fa640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1234155460 0x7f1234152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12388f8640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1234155460 0x7f1234152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12388f8640 1 -- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12200047c0 msgr2=0x7f1234156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12388f8640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12200047c0 0x7f1234156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12388f8640 1 -- 172.21.15.70:0/296243873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12340cadf0 con 0x7f12341591b0 2023-08-25T09:47:43.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f12388f8640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 0x7f1234154f20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1224037040 tx=0x7f1224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:43.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.121+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224003810 con 0x7f12341591b0 2023-08-25T09:47:43.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.122+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122402de90 con 0x7f12341591b0 2023-08-25T09:47:43.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.122+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123415d210 con 0x7f12341591b0 2023-08-25T09:47:43.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.122+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1224003b70 con 0x7f12341591b0 2023-08-25T09:47:43.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.122+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1234159c00 con 0x7f12341591b0 2023-08-25T09:47:43.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.123+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1234155870 con 0x7f12341591b0 2023-08-25T09:47:43.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.124+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1224003cd0 con 0x7f12341591b0 2023-08-25T09:47:43.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.125+0000 7f1229ffb640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f121806a050 0x7f121806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.125+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12240aa590 con 0x7f12341591b0 2023-08-25T09:47:43.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.125+0000 7f12390f9640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f121806a050 0x7f121806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.125+0000 7f12390f9640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f121806a050 0x7f121806c510 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f123406a7b0 tx=0x7f1230008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:43.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.129+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1224075d20 con 0x7f12341591b0 2023-08-25T09:47:43.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.318+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f123404e420 con 0x7f12341591b0 2023-08-25T09:47:43.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.321+0000 7f1229ffb640 1 -- 172.21.15.70:0/296243873 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+18388 (secure 0 0 0) 0x7f1224079340 con 0x7f12341591b0 2023-08-25T09:47:43.323 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:43.324 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":153,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:47:37.389506+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:47:43.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f121806a050 msgr2=0x7f121806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f121806a050 0x7f121806c510 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f123406a7b0 tx=0x7f1230008040 comp rx=0 tx=0).stop 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 msgr2=0x7f1234154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 0x7f1234154f20 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f1224037040 tx=0x7f1224004730 comp rx=0 tx=0).stop 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 shutdown_connections 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f121806a050 0x7f121806c510 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1234155460 0x7f1234152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f12341591b0 0x7f1234154f20 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 --2- 172.21.15.70:0/296243873 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f12200047c0 0x7f1234156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.323+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 >> 172.21.15.70:0/296243873 conn(0x7f1234069850 msgr2=0x7f12340b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:43.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.324+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 shutdown_connections 2023-08-25T09:47:43.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.324+0000 7f123b35d640 1 -- 172.21.15.70:0/296243873 wait complete. 2023-08-25T09:47:43.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:46:56.533109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:46:57.303907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:47:43.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:47:43.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.563+0000 7f7167283640 1 Processor -- start 2023-08-25T09:47:43.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.563+0000 7f7167283640 1 -- start start 2023-08-25T09:47:43.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.564+0000 7f7167283640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f716015e240 0x7f7160070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.564+0000 7f7167283640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 0x7f71600cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.564+0000 7f7167283640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716006f8e0 con 0x7f7160159bf0 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.564+0000 7f7167283640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71600ce030 con 0x7f716015e240 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.564+0000 7f7167283640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716006fbc0 con 0x7f71600cd660 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.565+0000 7f716501f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7160159bf0 0x7f716006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41396/0 (socket says 172.21.15.70:41396) 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.565+0000 7f716501f640 1 -- 172.21.15.70:0/3948739036 learned_addr learned my addr 172.21.15.70:0/3948739036 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:43.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.565+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298253202 0 0) 0x7f716006f8e0 con 0x7f7160159bf0 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.565+0000 7f716481e640 1 --2- 172.21.15.70:0/3948739036 >> v2:172.21.15.155:3300/0 conn(0x7f716015e240 0x7f7160070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.565+0000 7f7165820640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 0x7f71600cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.565+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f714c005730 con 0x7f7160159bf0 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608449967 0 0) 0x7f714c005730 con 0x7f7160159bf0 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 msgr2=0x7f71600cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 0x7f71600cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 >> v2:172.21.15.155:3300/0 conn(0x7f716015e240 msgr2=0x7f7160070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 --2- 172.21.15.70:0/3948739036 >> v2:172.21.15.155:3300/0 conn(0x7f716015e240 0x7f7160070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716006fea0 con 0x7f7160159bf0 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f715c001670 con 0x7f7160159bf0 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7165820640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 0x7f71600cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 >> v1:172.21.15.70:6789/0 conn(0x7f7160159bf0 legacy=0x7f716006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f714c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f714c0069d0 0x7f714c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 0x7f714c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71600ce030 con 0x7f714c005b80 2023-08-25T09:47:43.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716006fbc0 con 0x7f714c0069d0 2023-08-25T09:47:43.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c00db90 con 0x7f71600cd660 2023-08-25T09:47:43.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f716501f640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f714c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.566+0000 7f716481e640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f714c0069d0 0x7f714c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.567+0000 7f716501f640 1 -- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 msgr2=0x7f714c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.567+0000 7f716501f640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71600cd660 0x7f714c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.567+0000 7f716501f640 1 -- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f714c0069d0 msgr2=0x7f714c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.567+0000 7f716501f640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f714c0069d0 0x7f714c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.567+0000 7f716501f640 1 -- 172.21.15.70:0/3948739036 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c005730 con 0x7f714c005b80 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.567+0000 7f716481e640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f714c0069d0 0x7f714c006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f716501f640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f714c005fa0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c270 tx=0x7f714c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d58218394b8754 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:43.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c02cae0 con 0x7f714c005b80 2023-08-25T09:47:43.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c02cc40 con 0x7f714c005b80 2023-08-25T09:47:43.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f7157fff640 1 -- 172.21.15.70:0/3948739036 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c0058d0 con 0x7f714c005b80 2023-08-25T09:47:43.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f7167283640 1 -- 172.21.15.70:0/3948739036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 msgr2=0x7f714c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f7167283640 1 --2- 172.21.15.70:0/3948739036 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f714c005fa0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c270 tx=0x7f714c007a00 comp rx=0 tx=0).stop 2023-08-25T09:47:43.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.568+0000 7f7165820640 1 -- 172.21.15.70:0/3948739036 reap_dead start 2023-08-25T09:47:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.569+0000 7f7167283640 1 -- 172.21.15.70:0/3948739036 shutdown_connections 2023-08-25T09:47:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.569+0000 7f7167283640 1 -- 172.21.15.70:0/3948739036 >> 172.21.15.70:0/3948739036 conn(0x7f71600cb0d0 msgr2=0x7f71600cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.569+0000 7f7167283640 1 -- 172.21.15.70:0/3948739036 shutdown_connections 2023-08-25T09:47:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.569+0000 7f7167283640 1 -- 172.21.15.70:0/3948739036 wait complete. 2023-08-25T09:47:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.570+0000 7f7167283640 1 Processor -- start 2023-08-25T09:47:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.571+0000 7f7167283640 1 -- start start 2023-08-25T09:47:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7167283640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f7160149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7167283640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f716015d1f0 0x7f7160155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7167283640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f716015bc00 0x7f71600b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7167283640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716015a270 con 0x7f714c005b80 2023-08-25T09:47:43.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7167283640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716015c150 con 0x7f716015d1f0 2023-08-25T09:47:43.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7167283640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716015c430 con 0x7f716015bc00 2023-08-25T09:47:43.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f7160149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f7160149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47612/0 (socket says 172.21.15.70:47612) 2023-08-25T09:47:43.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 -- 172.21.15.70:0/3177854171 learned_addr learned my addr 172.21.15.70:0/3177854171 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:43.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716481e640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f716015d1f0 0x7f7160155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f7165820640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f716015bc00 0x7f71600b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 -- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f716015bc00 msgr2=0x7f71600b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f716015bc00 0x7f71600b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 -- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f716015d1f0 msgr2=0x7f7160155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f716015d1f0 0x7f7160155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716501f640 1 -- 172.21.15.70:0/3177854171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716015c710 con 0x7f714c005b80 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.572+0000 7f716481e640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f716015d1f0 0x7f7160155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.573+0000 7f716501f640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f7160149700 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c210 tx=0x7f715c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.573+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c03c040 con 0x7f714c005b80 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.573+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c001630 con 0x7f714c005b80 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.573+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c007290 con 0x7f714c005b80 2023-08-25T09:47:43.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.573+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716015c9f0 con 0x7f714c005b80 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.573+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7160149c40 con 0x7f714c005b80 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.574+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7160066fb0 con 0x7f714c005b80 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.575+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f715c032070 con 0x7f714c005b80 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.576+0000 7f7155ffb640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f714406a050 0x7f714406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.576+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f715c0a9930 con 0x7f714c005b80 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.576+0000 7f716481e640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f714406a050 0x7f714406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:43.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.577+0000 7f716481e640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f714406a050 0x7f714406c510 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f716015d5f0 tx=0x7f7150008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:43.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.580+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f715c074f80 con 0x7f714c005b80 2023-08-25T09:47:43.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.774+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f71600671e0 con 0x7f714c005b80 2023-08-25T09:47:43.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.775+0000 7f7155ffb640 1 -- 172.21.15.70:0/3177854171 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7f715c046020 con 0x7f714c005b80 2023-08-25T09:47:43.778 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:47:43.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f714406a050 msgr2=0x7f714406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f714406a050 0x7f714406c510 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f716015d5f0 tx=0x7f7150008040 comp rx=0 tx=0).stop 2023-08-25T09:47:43.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 msgr2=0x7f7160149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:43.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f7160149700 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c210 tx=0x7f715c001a00 comp rx=0 tx=0).stop 2023-08-25T09:47:43.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 shutdown_connections 2023-08-25T09:47:43.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f714406a050 0x7f714406c510 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f716015bc00 0x7f71600b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f716015d1f0 0x7f7160155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 --2- 172.21.15.70:0/3177854171 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f714c005b80 0x7f7160149700 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 >> 172.21.15.70:0/3177854171 conn(0x7f71600cb0d0 msgr2=0x7f716015d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 shutdown_connections 2023-08-25T09:47:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:43.778+0000 7f7167283640 1 -- 172.21.15.70:0/3177854171 wait complete. 2023-08-25T09:47:43.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:47:44.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.013+0000 7fb0a67c2640 1 Processor -- start 2023-08-25T09:47:44.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.013+0000 7fb0a67c2640 1 -- start start 2023-08-25T09:47:44.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb0a67c2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb0a015e240 0x7fb0a0070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb0a67c2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a006f8e0 con 0x7fb0a0159bf0 2023-08-25T09:47:44.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb0a67c2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a006fbc0 con 0x7fb0a00cd660 2023-08-25T09:47:44.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb0a67c2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a006fea0 con 0x7fb0a015e240 2023-08-25T09:47:44.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb09ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb0a0159bf0 0x7fb0a006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41402/0 (socket says 172.21.15.70:41402) 2023-08-25T09:47:44.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb09ffff640 1 -- 172.21.15.70:0/4096861977 learned_addr learned my addr 172.21.15.70:0/4096861977 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:44.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.015+0000 7fb09f7fe640 1 --2- 172.21.15.70:0/4096861977 >> v2:172.21.15.70:3300/0 conn(0x7fb0a015e240 0x7fb0a0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.016+0000 7fb09effd640 1 -- 172.21.15.70:0/4096861977 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377692988 0 0) 0x7fb0a006f8e0 con 0x7fb0a0159bf0 2023-08-25T09:47:44.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.016+0000 7fb09effd640 1 -- 172.21.15.70:0/4096861977 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb088003520 con 0x7fb0a0159bf0 2023-08-25T09:47:44.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.016+0000 7fb09f7fe640 1 -- 172.21.15.70:0/4096861977 >> v1:172.21.15.155:6789/0 conn(0x7fb0a00cd660 legacy=0x7fb0a00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.016+0000 7fb09f7fe640 1 -- 172.21.15.70:0/4096861977 >> v1:172.21.15.70:6789/0 conn(0x7fb0a0159bf0 legacy=0x7fb0a006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.016+0000 7fb09f7fe640 1 -- 172.21.15.70:0/4096861977 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0070180 con 0x7fb0a015e240 2023-08-25T09:47:44.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.017+0000 7fb09f7fe640 1 --2- 172.21.15.70:0/4096861977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a0070980 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb0940097a0 tx=0x7fb09402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07c5c28071dcd5f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb09effd640 1 -- 172.21.15.70:0/4096861977 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094030610 con 0x7fb0a015e240 2023-08-25T09:47:44.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb09effd640 1 -- 172.21.15.70:0/4096861977 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb094030770 con 0x7fb0a015e240 2023-08-25T09:47:44.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb09effd640 1 -- 172.21.15.70:0/4096861977 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb094030b00 con 0x7fb0a015e240 2023-08-25T09:47:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 -- 172.21.15.70:0/4096861977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 msgr2=0x7fb0a0070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/4096861977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a0070980 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7fb0940097a0 tx=0x7fb09402b960 comp rx=0 tx=0).stop 2023-08-25T09:47:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 -- 172.21.15.70:0/4096861977 shutdown_connections 2023-08-25T09:47:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/4096861977 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a0070980 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 -- 172.21.15.70:0/4096861977 >> 172.21.15.70:0/4096861977 conn(0x7fb0a00cb0d0 msgr2=0x7fb0a00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:44.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 -- 172.21.15.70:0/4096861977 shutdown_connections 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.018+0000 7fb0a67c2640 1 -- 172.21.15.70:0/4096861977 wait complete. 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.020+0000 7fb0a67c2640 1 Processor -- start 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.020+0000 7fb0a67c2640 1 -- start start 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a67c2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cd660 0x7fb0a00acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a67c2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0159bf0 0x7fb0a014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a67c2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a00c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a67c2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00c70f0 con 0x7fb0a015e240 2023-08-25T09:47:44.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a67c2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00c78c0 con 0x7fb0a0159bf0 2023-08-25T09:47:44.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a67c2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a00c7300 con 0x7fb0a00cd660 2023-08-25T09:47:44.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.021+0000 7fb0a4d5f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a00c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a00c3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47628/0 (socket says 172.21.15.70:47628) 2023-08-25T09:47:44.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 -- 172.21.15.70:0/3978964739 learned_addr learned my addr 172.21.15.70:0/3978964739 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:44.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb09ffff640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cd660 0x7fb0a00acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb09f7fe640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0159bf0 0x7fb0a014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 -- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cd660 msgr2=0x7fb0a00acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cd660 0x7fb0a00acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 -- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0159bf0 msgr2=0x7fb0a014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0159bf0 0x7fb0a014f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 -- 172.21.15.70:0/3978964739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094009400 con 0x7fb0a015e240 2023-08-25T09:47:44.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.022+0000 7fb0a4d5f640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a00c3930 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c7a0 tx=0x7fb09800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.023+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098003b70 con 0x7fb0a015e240 2023-08-25T09:47:44.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.023+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb098003cf0 con 0x7fb0a015e240 2023-08-25T09:47:44.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.023+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098003ea0 con 0x7fb0a015e240 2023-08-25T09:47:44.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.023+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a00c75e0 con 0x7fb0a015e240 2023-08-25T09:47:44.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.023+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a00c24e0 con 0x7fb0a015e240 2023-08-25T09:47:44.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.024+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a00c26d0 con 0x7fb0a015e240 2023-08-25T09:47:44.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.025+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb09800aa90 con 0x7fb0a015e240 2023-08-25T09:47:44.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.026+0000 7fb09cff9640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb08c06a050 0x7fb08c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.026+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb098088400 con 0x7fb0a015e240 2023-08-25T09:47:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.026+0000 7fb09ffff640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb08c06a050 0x7fb08c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.027+0000 7fb09ffff640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb08c06a050 0x7fb08c06c510 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb0900011e0 tx=0x7fb090009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.030+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb098053ae0 con 0x7fb0a015e240 2023-08-25T09:47:44.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.218+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb0a0066fb0 con 0x7fb0a015e240 2023-08-25T09:47:44.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.219+0000 7fb09cff9640 1 -- 172.21.15.70:0/3978964739 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v153) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb098057100 con 0x7fb0a015e240 2023-08-25T09:47:44.220 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:47:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb08c06a050 msgr2=0x7fb08c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb08c06a050 0x7fb08c06c510 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb0900011e0 tx=0x7fb090009040 comp rx=0 tx=0).stop 2023-08-25T09:47:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 msgr2=0x7fb0a00c3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a00c3930 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb09800c7a0 tx=0x7fb09800cc60 comp rx=0 tx=0).stop 2023-08-25T09:47:44.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 shutdown_connections 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb08c06a050 0x7fb08c06c510 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a015e240 0x7fb0a00c3930 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a0159bf0 0x7fb0a014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 --2- 172.21.15.70:0/3978964739 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0a00cd660 0x7fb0a00acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.221+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 >> 172.21.15.70:0/3978964739 conn(0x7fb0a00cb0d0 msgr2=0x7fb0a006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.222+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 shutdown_connections 2023-08-25T09:47:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.222+0000 7fb0a67c2640 1 -- 172.21.15.70:0/3978964739 wait complete. 2023-08-25T09:47:44.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:47:44.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.458+0000 7f59d0cfa640 1 Processor -- start 2023-08-25T09:47:44.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.458+0000 7f59d0cfa640 1 -- start start 2023-08-25T09:47:44.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59d0cfa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59cc15ae50 0x7f59cc0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59d0cfa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0cdc20 0x7f59cc069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59d0cfa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc0cc5f0 con 0x7f59cc0ce060 2023-08-25T09:47:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59d0cfa640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc070650 con 0x7f59cc15ae50 2023-08-25T09:47:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59d0cfa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc070930 con 0x7f59cc0cdc20 2023-08-25T09:47:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59ca59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59cc15ae50 0x7f59cc0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59ca59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f59cc15ae50 0x7f59cc0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47640/0 (socket says 172.21.15.70:47640) 2023-08-25T09:47:44.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59ca59c640 1 -- 172.21.15.70:0/2365805265 learned_addr learned my addr 172.21.15.70:0/2365805265 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:44.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.460+0000 7f59c9d9b640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0cdc20 0x7f59cc069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.461+0000 7f59ca59c640 1 -- 172.21.15.70:0/2365805265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0cdc20 msgr2=0x7f59cc069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:44.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.461+0000 7f59ca59c640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0cdc20 0x7f59cc069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.461+0000 7f59ca59c640 1 -- 172.21.15.70:0/2365805265 >> v1:172.21.15.70:6789/0 conn(0x7f59cc0ce060 legacy=0x7f59cc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.461+0000 7f59ca59c640 1 -- 172.21.15.70:0/2365805265 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc070c10 con 0x7f59cc15ae50 2023-08-25T09:47:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.461+0000 7f59c9d9b640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0cdc20 0x7f59cc069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:44.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.462+0000 7f59ca59c640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc0c2110 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009870 tx=0x7f59bc02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a348ca17727a7b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.462+0000 7f59c959a640 1 -- 172.21.15.70:0/2365805265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc02f830 con 0x7f59cc15ae50 2023-08-25T09:47:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.462+0000 7f59c959a640 1 -- 172.21.15.70:0/2365805265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc02f990 con 0x7f59cc15ae50 2023-08-25T09:47:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.462+0000 7f59c959a640 1 -- 172.21.15.70:0/2365805265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc02fd20 con 0x7f59cc15ae50 2023-08-25T09:47:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 -- 172.21.15.70:0/2365805265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 msgr2=0x7f59cc0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc0c2110 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009870 tx=0x7f59bc02ba10 comp rx=0 tx=0).stop 2023-08-25T09:47:44.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 -- 172.21.15.70:0/2365805265 shutdown_connections 2023-08-25T09:47:44.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0cdc20 0x7f59cc069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/2365805265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc0c2110 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 -- 172.21.15.70:0/2365805265 >> 172.21.15.70:0/2365805265 conn(0x7f59cc0cb0d0 msgr2=0x7f59cc0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:44.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 -- 172.21.15.70:0/2365805265 shutdown_connections 2023-08-25T09:47:44.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.463+0000 7f59d0cfa640 1 -- 172.21.15.70:0/2365805265 wait complete. 2023-08-25T09:47:44.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.465+0000 7f59d0cfa640 1 Processor -- start 2023-08-25T09:47:44.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.465+0000 7f59d0cfa640 1 -- start start 2023-08-25T09:47:44.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59d0cfa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59cc0cdc20 0x7f59cc14d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59d0cfa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0ce060 0x7f59cc14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59d0cfa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59d0cfa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc14d5c0 con 0x7f59cc15ae50 2023-08-25T09:47:44.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59d0cfa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc14c520 con 0x7f59cc0cdc20 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59d0cfa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc14c800 con 0x7f59cc0ce060 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59cad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc14b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47652/0 (socket says 172.21.15.70:47652) 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.466+0000 7f59cad9d640 1 -- 172.21.15.70:0/1783428466 learned_addr learned my addr 172.21.15.70:0/1783428466 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59c9d9b640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0ce060 0x7f59cc14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59ca59c640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59cc0cdc20 0x7f59cc14d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59cad9d640 1 -- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0ce060 msgr2=0x7f59cc14ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59cad9d640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0ce060 0x7f59cc14ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59cad9d640 1 -- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59cc0cdc20 msgr2=0x7f59cc14d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59cad9d640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59cc0cdc20 0x7f59cc14d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59cad9d640 1 -- 172.21.15.70:0/1783428466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc0094d0 con 0x7f59cc15ae50 2023-08-25T09:47:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59cad9d640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc14b830 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c730 tx=0x7f59c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4003b70 con 0x7f59cc15ae50 2023-08-25T09:47:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c4003cd0 con 0x7f59cc15ae50 2023-08-25T09:47:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400aa90 con 0x7f59cc15ae50 2023-08-25T09:47:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.467+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc14cae0 con 0x7f59cc15ae50 2023-08-25T09:47:44.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.468+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc14d0a0 con 0x7f59cc15ae50 2023-08-25T09:47:44.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.469+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc066fb0 con 0x7f59cc15ae50 2023-08-25T09:47:44.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.470+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f59c400abf0 con 0x7f59cc15ae50 2023-08-25T09:47:44.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.470+0000 7f59baffd640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f59ac06a050 0x7f59ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.471+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f59c4088390 con 0x7f59cc15ae50 2023-08-25T09:47:44.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.471+0000 7f59ca59c640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f59ac06a050 0x7f59ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.472+0000 7f59ca59c640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f59ac06a050 0x7f59ac06c510 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f59bc036000 tx=0x7f59bc039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.475+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f59c4053a70 con 0x7f59cc15ae50 2023-08-25T09:47:44.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.662+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f59cc0ca510 con 0x7f59cc15ae50 2023-08-25T09:47:44.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.663+0000 7f59baffd640 1 -- 172.21.15.70:0/1783428466 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v153) v1 ==== 104+0+11 (secure 0 0 0) 0x7f59c4057090 con 0x7f59cc15ae50 2023-08-25T09:47:44.665 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:47:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f59ac06a050 msgr2=0x7f59ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f59ac06a050 0x7f59ac06c510 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f59bc036000 tx=0x7f59bc039040 comp rx=0 tx=0).stop 2023-08-25T09:47:44.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 msgr2=0x7f59cc14b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc14b830 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c730 tx=0x7f59c400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 shutdown_connections 2023-08-25T09:47:44.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f59ac06a050 0x7f59ac06c510 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f59cc15ae50 0x7f59cc14b830 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f59cc0ce060 0x7f59cc14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 --2- 172.21.15.70:0/1783428466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f59cc0cdc20 0x7f59cc14d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 >> 172.21.15.70:0/1783428466 conn(0x7f59cc0cb0d0 msgr2=0x7f59cc0c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 shutdown_connections 2023-08-25T09:47:44.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.666+0000 7f59d0cfa640 1 -- 172.21.15.70:0/1783428466 wait complete. 2023-08-25T09:47:44.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:47:44.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.904+0000 7f0fe515d640 1 Processor -- start 2023-08-25T09:47:44.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.904+0000 7f0fe515d640 1 -- start start 2023-08-25T09:47:44.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fe515d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fe515d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0fe0069f60 0x7f0fe00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fe515d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe00cc5f0 con 0x7f0fe00cdc20 2023-08-25T09:47:44.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fe515d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe0070650 con 0x7f0fe0069f60 2023-08-25T09:47:44.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fe515d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0070930 con 0x7f0fe015ae50 2023-08-25T09:47:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fd7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0fe00cdc20 0x7f0fe0069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41420/0 (socket says 172.21.15.70:41420) 2023-08-25T09:47:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fd7fff640 1 -- 172.21.15.70:0/2846745483 learned_addr learned my addr 172.21.15.70:0/2846745483 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fdf59e640 1 --2- 172.21.15.70:0/2846745483 >> v2:172.21.15.70:3300/0 conn(0x7f0fe0069f60 0x7f0fe00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.906+0000 7f0fded9d640 1 --2- 172.21.15.70:0/2846745483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.907+0000 7f0fdf59e640 1 -- 172.21.15.70:0/2846745483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 msgr2=0x7f0fe00c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.907+0000 7f0fdf59e640 1 --2- 172.21.15.70:0/2846745483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.907+0000 7f0fdf59e640 1 -- 172.21.15.70:0/2846745483 >> v1:172.21.15.70:6789/0 conn(0x7f0fe00cdc20 legacy=0x7f0fe0069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.907+0000 7f0fdf59e640 1 -- 172.21.15.70:0/2846745483 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0070c10 con 0x7f0fe0069f60 2023-08-25T09:47:44.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.907+0000 7f0fde59c640 1 -- 172.21.15.70:0/2846745483 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2332965460 0 0) 0x7f0fe00cc5f0 con 0x7f0fe00cdc20 2023-08-25T09:47:44.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.908+0000 7f0fdf59e640 1 --2- 172.21.15.70:0/2846745483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe00ccad0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009d90 tx=0x7f0fd802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2a57b28d200f7e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.908+0000 7f0fde59c640 1 -- 172.21.15.70:0/2846745483 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd802cce0 con 0x7f0fe0069f60 2023-08-25T09:47:44.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.908+0000 7f0fde59c640 1 -- 172.21.15.70:0/2846745483 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd802ce40 con 0x7f0fe0069f60 2023-08-25T09:47:44.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.908+0000 7f0fde59c640 1 -- 172.21.15.70:0/2846745483 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8034660 con 0x7f0fe0069f60 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 -- 172.21.15.70:0/2846745483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 msgr2=0x7f0fe00ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 --2- 172.21.15.70:0/2846745483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe00ccad0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8009d90 tx=0x7f0fd802bf30 comp rx=0 tx=0).stop 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 -- 172.21.15.70:0/2846745483 shutdown_connections 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 --2- 172.21.15.70:0/2846745483 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe00ccad0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 --2- 172.21.15.70:0/2846745483 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 -- 172.21.15.70:0/2846745483 >> 172.21.15.70:0/2846745483 conn(0x7f0fe00cb0d0 msgr2=0x7f0fe00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:44.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 -- 172.21.15.70:0/2846745483 shutdown_connections 2023-08-25T09:47:44.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.909+0000 7f0fe515d640 1 -- 172.21.15.70:0/2846745483 wait complete. 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.911+0000 7f0fe515d640 1 Processor -- start 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.911+0000 7f0fe515d640 1 -- start start 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fe515d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe014a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fe515d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00cdc20 0x7f0fe014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fe515d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fe515d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe015b930 con 0x7f0fe0069f60 2023-08-25T09:47:44.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fe515d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe014e320 con 0x7f0fe00cdc20 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fe515d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe014dd60 con 0x7f0fe015ae50 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fded9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe014a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fded9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe014a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47678/0 (socket says 172.21.15.70:47678) 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fded9d640 1 -- 172.21.15.70:0/1240841708 learned_addr learned my addr 172.21.15.70:0/1240841708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fded9d640 1 -- 172.21.15.70:0/1240841708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 msgr2=0x7f0fe00adf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.912+0000 7f0fded9d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fded9d640 1 -- 172.21.15.70:0/1240841708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00cdc20 msgr2=0x7f0fe014d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:44.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fded9d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00cdc20 0x7f0fe014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:44.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fded9d640 1 -- 172.21.15.70:0/1240841708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8009a70 con 0x7f0fe0069f60 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fded9d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe014a970 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d5f0 tx=0x7f0fd000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00125f0 con 0x7f0fe0069f60 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0012750 con 0x7f0fe0069f60 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0012ae0 con 0x7f0fe0069f60 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe014e040 con 0x7f0fe0069f60 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.913+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe014aeb0 con 0x7f0fe0069f60 2023-08-25T09:47:44.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.914+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe0066fb0 con 0x7f0fe0069f60 2023-08-25T09:47:44.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.915+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0fd0012c40 con 0x7f0fe0069f60 2023-08-25T09:47:44.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.916+0000 7f0fd77fe640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fc006a050 0x7f0fc006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:44.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.916+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0fd0088a80 con 0x7f0fe0069f60 2023-08-25T09:47:44.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.917+0000 7f0fd7fff640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fc006a050 0x7f0fc006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:44.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.917+0000 7f0fd7fff640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fc006a050 0x7f0fc006c510 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8007fd0 tx=0x7f0fc8007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:44.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:44.921+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0fd0054160 con 0x7f0fe0069f60 2023-08-25T09:47:45.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.104+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fe006bec0 con 0x7f0fe0069f60 2023-08-25T09:47:45.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.105+0000 7f0fd77fe640 1 -- 172.21.15.70:0/1240841708 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v153) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fd0057780 con 0x7f0fe0069f60 2023-08-25T09:47:45.107 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:47:45.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.107+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fc006a050 msgr2=0x7f0fc006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fc006a050 0x7f0fc006c510 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8007fd0 tx=0x7f0fc8007920 comp rx=0 tx=0).stop 2023-08-25T09:47:45.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 msgr2=0x7f0fe014a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe014a970 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000d5f0 tx=0x7f0fd000dab0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 shutdown_connections 2023-08-25T09:47:45.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fc006a050 0x7f0fc006c510 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0fe015ae50 0x7f0fe00adf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fe00cdc20 0x7f0fe014d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 --2- 172.21.15.70:0/1240841708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fe0069f60 0x7f0fe014a970 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 >> 172.21.15.70:0/1240841708 conn(0x7f0fe00cb0d0 msgr2=0x7f0fe00c46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:45.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 shutdown_connections 2023-08-25T09:47:45.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.108+0000 7f0fe515d640 1 -- 172.21.15.70:0/1240841708 wait complete. 2023-08-25T09:47:45.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:47:45.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.330+0000 7f31fac45640 1 Processor -- start 2023-08-25T09:47:45.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.331+0000 7f31fac45640 1 -- start start 2023-08-25T09:47:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.332+0000 7f31fac45640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 0x7f31f40b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.332+0000 7f31fac45640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31f4071850 0x7f31f406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.332+0000 7f31fac45640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f415e4a0 con 0x7f31f415a7d0 2023-08-25T09:47:45.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.332+0000 7f31fac45640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f40c53b0 con 0x7f31f4071850 2023-08-25T09:47:45.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.332+0000 7f31fac45640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f415f020 con 0x7f31f415abb0 2023-08-25T09:47:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.333+0000 7f31f9442640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 0x7f31f40b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:45.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.333+0000 7f31f9c43640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f31f415a7d0 0x7f31f40b3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:43450/0 (socket says 172.21.15.70:43450) 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.333+0000 7f31f9c43640 1 -- 172.21.15.70:0/1570110749 learned_addr learned my addr 172.21.15.70:0/1570110749 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.333+0000 7f31fa444640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31f4071850 0x7f31f406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.333+0000 7f31f9442640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 0x7f31f40b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:56630/0 (socket says 172.21.15.70:56630) 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.334+0000 7f31f9442640 1 -- 172.21.15.70:0/1570110749 >> v2:172.21.15.155:3300/0 conn(0x7f31f4071850 msgr2=0x7f31f406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.334+0000 7f31f9442640 1 --2- 172.21.15.70:0/1570110749 >> v2:172.21.15.155:3300/0 conn(0x7f31f4071850 0x7f31f406ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.334+0000 7f31f9442640 1 -- 172.21.15.70:0/1570110749 >> v1:172.21.15.155:6789/0 conn(0x7f31f415a7d0 legacy=0x7f31f40b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.334+0000 7f31f9442640 1 -- 172.21.15.70:0/1570110749 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f415e6f0 con 0x7f31f415abb0 2023-08-25T09:47:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.336+0000 7f31f9442640 1 --2- 172.21.15.70:0/1570110749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 0x7f31f40b3d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f31e40097a0 tx=0x7f31e402b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd24da67486d8bbf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.336+0000 7f31f8c41640 1 -- 172.21.15.70:0/1570110749 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e40316f0 con 0x7f31f415abb0 2023-08-25T09:47:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.336+0000 7f31f8c41640 1 -- 172.21.15.70:0/1570110749 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e4031850 con 0x7f31f415abb0 2023-08-25T09:47:45.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.336+0000 7f31f8c41640 1 -- 172.21.15.70:0/1570110749 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4031be0 con 0x7f31f415abb0 2023-08-25T09:47:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 -- 172.21.15.70:0/1570110749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 msgr2=0x7f31f40b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 --2- 172.21.15.70:0/1570110749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 0x7f31f40b3d30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f31e40097a0 tx=0x7f31e402b790 comp rx=0 tx=0).stop 2023-08-25T09:47:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 -- 172.21.15.70:0/1570110749 shutdown_connections 2023-08-25T09:47:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 --2- 172.21.15.70:0/1570110749 >> v2:172.21.15.155:3300/0 conn(0x7f31f4071850 0x7f31f406ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 --2- 172.21.15.70:0/1570110749 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f415abb0 0x7f31f40b3d30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 -- 172.21.15.70:0/1570110749 >> 172.21.15.70:0/1570110749 conn(0x7f31f40aff60 msgr2=0x7f31f40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:45.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 -- 172.21.15.70:0/1570110749 shutdown_connections 2023-08-25T09:47:45.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.337+0000 7f31fac45640 1 -- 172.21.15.70:0/1570110749 wait complete. 2023-08-25T09:47:45.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.339+0000 7f31fac45640 1 Processor -- start 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.339+0000 7f31fac45640 1 -- start start 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.340+0000 7f31fac45640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f4071850 0x7f31f40c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.340+0000 7f31fac45640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 0x7f31f4140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.340+0000 7f31fac45640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f40c34a0 0x7f31f40c1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.340+0000 7f31fac45640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40c4550 con 0x7f31f415a7d0 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.340+0000 7f31fac45640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f41412a0 con 0x7f31f40c34a0 2023-08-25T09:47:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.340+0000 7f31fac45640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f40c3940 con 0x7f31f4071850 2023-08-25T09:47:45.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 0x7f31f4140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:45.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 0x7f31f4140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47684/0 (socket says 172.21.15.70:47684) 2023-08-25T09:47:45.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 -- 172.21.15.70:0/3715743370 learned_addr learned my addr 172.21.15.70:0/3715743370 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:45.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9c43640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f4071850 0x7f31f40c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:45.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31fa444640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f40c34a0 0x7f31f40c1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:45.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 -- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f4071850 msgr2=0x7f31f40c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f4071850 0x7f31f40c49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 -- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f40c34a0 msgr2=0x7f31f40c1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f40c34a0 0x7f31f40c1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31f9442640 1 -- 172.21.15.70:0/3715743370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e4009400 con 0x7f31f415a7d0 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.341+0000 7f31fa444640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f40c34a0 0x7f31f40c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.342+0000 7f31f9442640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 0x7f31f4140d60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f31e40098c0 tx=0x7f31e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.342+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4034680 con 0x7f31f415a7d0 2023-08-25T09:47:45.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.342+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e40347e0 con 0x7f31f415a7d0 2023-08-25T09:47:45.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.342+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e4034b90 con 0x7f31f415a7d0 2023-08-25T09:47:45.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.342+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f40c3c20 con 0x7f31f415a7d0 2023-08-25T09:47:45.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.342+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f40c41e0 con 0x7f31f415a7d0 2023-08-25T09:47:45.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.343+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f40c46c0 con 0x7f31f415a7d0 2023-08-25T09:47:45.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.344+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f31e403b040 con 0x7f31f415a7d0 2023-08-25T09:47:45.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.345+0000 7f31ea7fc640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31d806a050 0x7f31d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:45.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.345+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31e40aa240 con 0x7f31f415a7d0 2023-08-25T09:47:45.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.345+0000 7f31f9c43640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31d806a050 0x7f31d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:45.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.346+0000 7f31f9c43640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31d806a050 0x7f31d806c510 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f31f0005880 tx=0x7f31f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:45.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.350+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f31e4075890 con 0x7f31f415a7d0 2023-08-25T09:47:45.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.552+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f31f4066fb0 con 0x7f31f415a7d0 2023-08-25T09:47:45.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.553+0000 7f31ea7fc640 1 -- 172.21.15.70:0/3715743370 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v153) v1 ==== 80+0+0 (secure 0 0 0) 0x7f31e4078eb0 con 0x7f31f415a7d0 2023-08-25T09:47:45.555 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:47:45.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.555+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31d806a050 msgr2=0x7f31d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.555+0000 7f31fac45640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31d806a050 0x7f31d806c510 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f31f0005880 tx=0x7f31f0009040 comp rx=0 tx=0).stop 2023-08-25T09:47:45.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.555+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 msgr2=0x7f31f4140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:45.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.555+0000 7f31fac45640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 0x7f31f4140d60 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f31e40098c0 tx=0x7f31e40046c0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 shutdown_connections 2023-08-25T09:47:45.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f31d806a050 0x7f31d806c510 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31f40c34a0 0x7f31f40c1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31f415a7d0 0x7f31f4140d60 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 --2- 172.21.15.70:0/3715743370 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31f4071850 0x7f31f40c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 >> 172.21.15.70:0/3715743370 conn(0x7f31f40aff60 msgr2=0x7f31f4155780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 shutdown_connections 2023-08-25T09:47:45.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:45.556+0000 7f31fac45640 1 -- 172.21.15.70:0/3715743370 wait complete. 2023-08-25T09:47:45.577 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:47:45.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:45.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:47:45.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[117333]: fuse finished with error 0 and tester_r 0 2023-08-25T09:47:45.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:47:45.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:45.691 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:47:45.759 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:47:45.760 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:45.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.008+0000 7f3a02bb2640 1 Processor -- start 2023-08-25T09:47:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.009+0000 7f3a02bb2640 1 -- start start 2023-08-25T09:47:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.010+0000 7f3a02bb2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f39fc0b12f0 0x7f39fc15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.010+0000 7f3a02bb2640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc15e7c0 con 0x7f39fc15a7c0 2023-08-25T09:47:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.010+0000 7f3a02bb2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39fc071b50 con 0x7f39fc15f190 2023-08-25T09:47:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.010+0000 7f3a02bb2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39fc15e490 con 0x7f39fc0b12f0 2023-08-25T09:47:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.010+0000 7f3a01bb0640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f39fc15a7c0 0x7f39fc0b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41428/0 (socket says 172.21.15.70:41428) 2023-08-25T09:47:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.010+0000 7f3a01bb0640 1 -- 172.21.15.70:0/800582545 learned_addr learned my addr 172.21.15.70:0/800582545 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964233080 0 0) 0x7f39fc15e7c0 con 0x7f39fc15a7c0 2023-08-25T09:47:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e8003520 con 0x7f39fc15a7c0 2023-08-25T09:47:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a013af640 1 --2- 172.21.15.70:0/800582545 >> v2:172.21.15.155:3300/0 conn(0x7f39fc0b12f0 0x7f39fc15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1173197404 0 0) 0x7f39e8003520 con 0x7f39fc15a7c0 2023-08-25T09:47:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 >> v2:172.21.15.155:3300/0 conn(0x7f39fc0b12f0 msgr2=0x7f39fc15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 --2- 172.21.15.70:0/800582545 >> v2:172.21.15.155:3300/0 conn(0x7f39fc0b12f0 0x7f39fc15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 >> v1:172.21.15.155:6789/0 conn(0x7f39fc15f190 legacy=0x7f39fc0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0b3d60 con 0x7f39fc15a7c0 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f39f8001670 con 0x7f39fc15a7c0 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 >> v1:172.21.15.70:6789/0 conn(0x7f39fc15a7c0 legacy=0x7f39fc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39e8003970 0x7f39e8003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39e80052a0 0x7f39e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc15e490 con 0x7f39e80047c0 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e800c1a0 con 0x7f39e80052a0 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e800c360 con 0x7f39e8003970 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.011+0000 7f3a013af640 1 --2- 172.21.15.70:0/800582545 >> v2:172.21.15.155:3300/0 conn(0x7f39fc0b12f0 0x7f39fc15ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a013af640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a01bb0640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39e8003970 0x7f39e8003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a013af640 1 -- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39e8003970 msgr2=0x7f39e8003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a013af640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39e8003970 0x7f39e8003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a013af640 1 -- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39e80052a0 msgr2=0x7f39e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a013af640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39e80052a0 0x7f39e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a013af640 1 -- 172.21.15.70:0/800582545 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8003520 con 0x7f39e80047c0 2023-08-25T09:47:46.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.012+0000 7f3a023b1640 1 -- 172.21.15.70:0/800582545 reap_dead start 2023-08-25T09:47:46.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.013+0000 7f3a013af640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39e8004c40 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00aa40 tx=0x7f39e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded8e9f3ea6ef19b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.013+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec004220 con 0x7f39e80047c0 2023-08-25T09:47:46.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.013+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec004380 con 0x7f39e80047c0 2023-08-25T09:47:46.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.013+0000 7f3a00bae640 1 -- 172.21.15.70:0/800582545 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec003830 con 0x7f39e80047c0 2023-08-25T09:47:46.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 -- 172.21.15.70:0/800582545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 msgr2=0x7f39e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39e8004c40 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f39ec00aa40 tx=0x7f39e80057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 -- 172.21.15.70:0/800582545 shutdown_connections 2023-08-25T09:47:46.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/800582545 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39e8004c40 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 -- 172.21.15.70:0/800582545 >> 172.21.15.70:0/800582545 conn(0x7f39fc0affd0 msgr2=0x7f39fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:46.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 -- 172.21.15.70:0/800582545 shutdown_connections 2023-08-25T09:47:46.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.014+0000 7f3a02bb2640 1 -- 172.21.15.70:0/800582545 wait complete. 2023-08-25T09:47:46.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.016+0000 7f3a02bb2640 1 Processor -- start 2023-08-25T09:47:46.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.017+0000 7f3a02bb2640 1 -- start start 2023-08-25T09:47:46.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a02bb2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39fc070890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a02bb2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39fc143630 0x7f39fc0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a02bb2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39fc0c8f10 0x7f39fc0c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a02bb2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0c93c0 con 0x7f39e80047c0 2023-08-25T09:47:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a02bb2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc0c1940 con 0x7f39fc143630 2023-08-25T09:47:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a02bb2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39fc06faf0 con 0x7f39fc0c8f10 2023-08-25T09:47:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39fc070890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39fc070890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47694/0 (socket says 172.21.15.70:47694) 2023-08-25T09:47:46.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 -- 172.21.15.70:0/194740233 learned_addr learned my addr 172.21.15.70:0/194740233 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 -- 172.21.15.70:0/194740233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39fc0c8f10 msgr2=0x7f39fc0c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39fc0c8f10 0x7f39fc0c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 -- 172.21.15.70:0/194740233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39fc143630 msgr2=0x7f39fc0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39fc143630 0x7f39fc0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.018+0000 7f3a01bb0640 1 -- 172.21.15.70:0/194740233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0c8360 con 0x7f39e80047c0 2023-08-25T09:47:46.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.019+0000 7f3a01bb0640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39fc070890 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f39f8001200 tx=0x7f39f800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.019+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800c6c0 con 0x7f39e80047c0 2023-08-25T09:47:46.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.019+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f800c840 con 0x7f39e80047c0 2023-08-25T09:47:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.019+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f800cbf0 con 0x7f39e80047c0 2023-08-25T09:47:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.019+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39fc06fdb0 con 0x7f39e80047c0 2023-08-25T09:47:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.019+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39fc070370 con 0x7f39e80047c0 2023-08-25T09:47:46.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.020+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39fc066fb0 con 0x7f39e80047c0 2023-08-25T09:47:46.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.021+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f39f801d460 con 0x7f39e80047c0 2023-08-25T09:47:46.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.022+0000 7f39f27fc640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f39e006a080 0x7f39e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.022+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f39f8088760 con 0x7f39e80047c0 2023-08-25T09:47:46.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.022+0000 7f3a013af640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f39e006a080 0x7f39e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.023+0000 7f3a013af640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f39e006a080 0x7f39e006c540 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f39ec035000 tx=0x7f39ec030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.026+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f39f8053db0 con 0x7f39e80047c0 2023-08-25T09:47:46.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.215+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39fc0b0d00 con 0x7f39e80047c0 2023-08-25T09:47:46.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.219+0000 7f39f27fc640 1 -- 172.21.15.70:0/194740233 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+18388 (secure 0 0 0) 0x7f39f80573d0 con 0x7f39e80047c0 2023-08-25T09:47:46.221 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:46.221 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":153,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:47:37.389506+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:47:46.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.221+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f39e006a080 msgr2=0x7f39e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.221+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f39e006a080 0x7f39e006c540 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f39ec035000 tx=0x7f39ec030000 comp rx=0 tx=0).stop 2023-08-25T09:47:46.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.221+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 msgr2=0x7f39fc070890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.221+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39fc070890 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f39f8001200 tx=0x7f39f800eb90 comp rx=0 tx=0).stop 2023-08-25T09:47:46.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 shutdown_connections 2023-08-25T09:47:46.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f39e006a080 0x7f39e006c540 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f39fc0c8f10 0x7f39fc0c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f39fc143630 0x7f39fc0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 --2- 172.21.15.70:0/194740233 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f39e80047c0 0x7f39fc070890 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 >> 172.21.15.70:0/194740233 conn(0x7f39fc0affd0 msgr2=0x7f39fc0ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:46.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 shutdown_connections 2023-08-25T09:47:46.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.222+0000 7f3a02bb2640 1 -- 172.21.15.70:0/194740233 wait complete. 2023-08-25T09:47:46.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:46:56.533109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:46:57.303907+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:47:46.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:47:46.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:47:46.454 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:47:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.461+0000 7fa87ffa4640 1 Processor -- start 2023-08-25T09:47:46.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.461+0000 7fa87ffa4640 1 -- start start 2023-08-25T09:47:46.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.462+0000 7fa87ffa4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa878159bf0 0x7fa87815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.462+0000 7fa87ffa4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8780cd660 0x7fa8780cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.462+0000 7fa87ffa4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87815e780 con 0x7fa87806f0e0 2023-08-25T09:47:46.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.462+0000 7fa87ffa4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780ce030 con 0x7fa8780cd660 2023-08-25T09:47:46.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.462+0000 7fa87ffa4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa87806f8e0 con 0x7fa878159bf0 2023-08-25T09:47:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8780cd660 0x7fa8780cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8780cd660 0x7fa8780cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47710/0 (socket says 172.21.15.70:47710) 2023-08-25T09:47:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 -- 172.21.15.70:0/3046983474 learned_addr learned my addr 172.21.15.70:0/3046983474 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87dd40640 1 --2- 172.21.15.70:0/3046983474 >> v2:172.21.15.155:3300/0 conn(0x7fa878159bf0 0x7fa87815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 -- 172.21.15.70:0/3046983474 >> v2:172.21.15.155:3300/0 conn(0x7fa878159bf0 msgr2=0x7fa87815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 --2- 172.21.15.70:0/3046983474 >> v2:172.21.15.155:3300/0 conn(0x7fa878159bf0 0x7fa87815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 -- 172.21.15.70:0/3046983474 >> v1:172.21.15.155:6789/0 conn(0x7fa87806f0e0 legacy=0x7fa878070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87e541640 1 -- 172.21.15.70:0/3046983474 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87806fbc0 con 0x7fa8780cd660 2023-08-25T09:47:46.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.463+0000 7fa87dd40640 1 --2- 172.21.15.70:0/3046983474 >> v2:172.21.15.155:3300/0 conn(0x7fa878159bf0 0x7fa87815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.465+0000 7fa87e541640 1 --2- 172.21.15.70:0/3046983474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8780cd660 0x7fa8780cdaf0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa87806d1f0 tx=0x7fa87002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb436868e01a7ce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.465+0000 7fa87cd3e640 1 -- 172.21.15.70:0/3046983474 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87002ca80 con 0x7fa8780cd660 2023-08-25T09:47:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.465+0000 7fa87cd3e640 1 -- 172.21.15.70:0/3046983474 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87002cbe0 con 0x7fa8780cd660 2023-08-25T09:47:46.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.465+0000 7fa87cd3e640 1 -- 172.21.15.70:0/3046983474 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8700343f0 con 0x7fa8780cd660 2023-08-25T09:47:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 -- 172.21.15.70:0/3046983474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8780cd660 msgr2=0x7fa8780cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/3046983474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8780cd660 0x7fa8780cdaf0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa87806d1f0 tx=0x7fa87002bee0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 -- 172.21.15.70:0/3046983474 shutdown_connections 2023-08-25T09:47:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/3046983474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8780cd660 0x7fa8780cdaf0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/3046983474 >> v2:172.21.15.155:3300/0 conn(0x7fa878159bf0 0x7fa87815e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 -- 172.21.15.70:0/3046983474 >> 172.21.15.70:0/3046983474 conn(0x7fa8780cb0d0 msgr2=0x7fa8780cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 -- 172.21.15.70:0/3046983474 shutdown_connections 2023-08-25T09:47:46.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.466+0000 7fa87ffa4640 1 -- 172.21.15.70:0/3046983474 wait complete. 2023-08-25T09:47:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.468+0000 7fa87ffa4640 1 Processor -- start 2023-08-25T09:47:46.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.468+0000 7fa87ffa4640 1 -- start start 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87ffa4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 0x7fa87814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87ffa4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8780cd660 0x7fa878146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87ffa4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa878159bf0 0x7fa878148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87ffa4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878149fe0 con 0x7fa87806f0e0 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87ffa4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa878147450 con 0x7fa878159bf0 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87ffa4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa878149360 con 0x7fa8780cd660 2023-08-25T09:47:46.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87dd40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 0x7fa87814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87dd40640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 0x7fa87814a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47716/0 (socket says 172.21.15.70:47716) 2023-08-25T09:47:46.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87dd40640 1 -- 172.21.15.70:0/607851921 learned_addr learned my addr 172.21.15.70:0/607851921 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:46.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87d53f640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8780cd660 0x7fa878146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.469+0000 7fa87e541640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa878159bf0 0x7fa878148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87dd40640 1 -- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8780cd660 msgr2=0x7fa878146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87dd40640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8780cd660 0x7fa878146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87dd40640 1 -- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa878159bf0 msgr2=0x7fa878148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87dd40640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa878159bf0 0x7fa878148ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87dd40640 1 -- 172.21.15.70:0/607851921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870009b30 con 0x7fa87806f0e0 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87dd40640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 0x7fa87814a400 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c7f0 tx=0x7fa87400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8740079d0 con 0x7fa87806f0e0 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa874007b30 con 0x7fa87806f0e0 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87400ce40 con 0x7fa87806f0e0 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa878149640 con 0x7fa87806f0e0 2023-08-25T09:47:46.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.470+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa878149c00 con 0x7fa87806f0e0 2023-08-25T09:47:46.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.472+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87814a250 con 0x7fa87806f0e0 2023-08-25T09:47:46.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.473+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8740113f0 con 0x7fa87806f0e0 2023-08-25T09:47:46.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.474+0000 7fa86e7fc640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85c06a050 0x7fa85c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.474+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa874088790 con 0x7fa87806f0e0 2023-08-25T09:47:46.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.474+0000 7fa87d53f640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85c06a050 0x7fa85c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.475+0000 7fa87d53f640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85c06a050 0x7fa85c06c510 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fd0 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.478+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa874053de0 con 0x7fa87806f0e0 2023-08-25T09:47:46.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.672+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa878066fb0 con 0x7fa87806f0e0 2023-08-25T09:47:46.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.673+0000 7fa86e7fc640 1 -- 172.21.15.70:0/607851921 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa874057400 con 0x7fa87806f0e0 2023-08-25T09:47:46.675 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:47:46.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85c06a050 msgr2=0x7fa85c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85c06a050 0x7fa85c06c510 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fd0 tx=0x7fa868008040 comp rx=0 tx=0).stop 2023-08-25T09:47:46.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 msgr2=0x7fa87814a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 0x7fa87814a400 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fa87400c7f0 tx=0x7fa87400cbf0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 shutdown_connections 2023-08-25T09:47:46.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa85c06a050 0x7fa85c06c510 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa878159bf0 0x7fa878148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8780cd660 0x7fa878146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 --2- 172.21.15.70:0/607851921 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa87806f0e0 0x7fa87814a400 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.676+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 >> 172.21.15.70:0/607851921 conn(0x7fa8780cb0d0 msgr2=0x7fa878069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:46.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.677+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 shutdown_connections 2023-08-25T09:47:46.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.677+0000 7fa87ffa4640 1 -- 172.21.15.70:0/607851921 wait complete. 2023-08-25T09:47:46.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:47:46.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.914+0000 7fdfb9eb8640 1 Processor -- start 2023-08-25T09:47:46.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.914+0000 7fdfb9eb8640 1 -- start start 2023-08-25T09:47:46.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfb9eb8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdfb4159bf0 0x7fdfb415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfb9eb8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb40cd660 0x7fdfb40cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfb9eb8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb415e780 con 0x7fdfb406f0e0 2023-08-25T09:47:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfb9eb8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb40ce030 con 0x7fdfb4159bf0 2023-08-25T09:47:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfb9eb8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb406f8e0 con 0x7fdfb40cd660 2023-08-25T09:47:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfb37fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdfb4159bf0 0x7fdfb415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfabfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdfb406f0e0 0x7fdfb4070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41442/0 (socket says 172.21.15.70:41442) 2023-08-25T09:47:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.916+0000 7fdfabfff640 1 -- 172.21.15.70:0/857178965 learned_addr learned my addr 172.21.15.70:0/857178965 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.917+0000 7fdfb37fe640 1 -- 172.21.15.70:0/857178965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb40cd660 msgr2=0x7fdfb40cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:46.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.917+0000 7fdfb3fff640 1 --2- 172.21.15.70:0/857178965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb40cd660 0x7fdfb40cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.917+0000 7fdfb37fe640 1 --2- 172.21.15.70:0/857178965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb40cd660 0x7fdfb40cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.917+0000 7fdfb2ffd640 1 -- 172.21.15.70:0/857178965 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216587860 0 0) 0x7fdfb415e780 con 0x7fdfb406f0e0 2023-08-25T09:47:46.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.917+0000 7fdfb37fe640 1 -- 172.21.15.70:0/857178965 >> v1:172.21.15.70:6789/0 conn(0x7fdfb406f0e0 legacy=0x7fdfb4070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.917+0000 7fdfb37fe640 1 -- 172.21.15.70:0/857178965 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb406fbc0 con 0x7fdfb4159bf0 2023-08-25T09:47:46.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.918+0000 7fdfb37fe640 1 --2- 172.21.15.70:0/857178965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb415e240 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40097a0 tx=0x7fdfa402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4a71187b66a56a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.918+0000 7fdfb2ffd640 1 -- 172.21.15.70:0/857178965 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa402d040 con 0x7fdfb4159bf0 2023-08-25T09:47:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.918+0000 7fdfb2ffd640 1 -- 172.21.15.70:0/857178965 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa4031810 con 0x7fdfb4159bf0 2023-08-25T09:47:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.918+0000 7fdfb2ffd640 1 -- 172.21.15.70:0/857178965 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4031ba0 con 0x7fdfb4159bf0 2023-08-25T09:47:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/857178965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 msgr2=0x7fdfb415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/857178965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb415e240 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40097a0 tx=0x7fdfa402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/857178965 shutdown_connections 2023-08-25T09:47:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/857178965 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb40cd660 0x7fdfb40cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/857178965 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb415e240 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/857178965 >> 172.21.15.70:0/857178965 conn(0x7fdfb40cb0d0 msgr2=0x7fdfb40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/857178965 shutdown_connections 2023-08-25T09:47:46.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.919+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/857178965 wait complete. 2023-08-25T09:47:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.921+0000 7fdfb9eb8640 1 Processor -- start 2023-08-25T09:47:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.921+0000 7fdfb9eb8640 1 -- start start 2023-08-25T09:47:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb9eb8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb406f0e0 0x7fdfb41f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb9eb8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfb40cd660 0x7fdfb4156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb9eb8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb4153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb9eb8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb40c8660 con 0x7fdfb4159bf0 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb9eb8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb41f6130 con 0x7fdfb40cd660 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb9eb8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb4156580 con 0x7fdfb406f0e0 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb4153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb4153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47720/0 (socket says 172.21.15.70:47720) 2023-08-25T09:47:46.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb3fff640 1 -- 172.21.15.70:0/3733170235 learned_addr learned my addr 172.21.15.70:0/3733170235 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb37fe640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb406f0e0 0x7fdfb41f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfabfff640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfb40cd660 0x7fdfb4156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb3fff640 1 -- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb406f0e0 msgr2=0x7fdfb41f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.922+0000 7fdfb3fff640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb406f0e0 0x7fdfb41f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb3fff640 1 -- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfb40cd660 msgr2=0x7fdfb4156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb3fff640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfb40cd660 0x7fdfb4156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb3fff640 1 -- 172.21.15.70:0/3733170235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4009400 con 0x7fdfb4159bf0 2023-08-25T09:47:46.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfabfff640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfb40cd660 0x7fdfb4156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb3fff640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb4153d50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00cd70 tx=0x7fdfac00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003b90 con 0x7fdfb4159bf0 2023-08-25T09:47:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac003cf0 con 0x7fdfb4159bf0 2023-08-25T09:47:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac004240 con 0x7fdfb4159bf0 2023-08-25T09:47:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb4156820 con 0x7fdfb4159bf0 2023-08-25T09:47:46.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.923+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb41f66b0 con 0x7fdfb4159bf0 2023-08-25T09:47:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.924+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb41f68d0 con 0x7fdfb4159bf0 2023-08-25T09:47:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.925+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdfac0043a0 con 0x7fdfb4159bf0 2023-08-25T09:47:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.926+0000 7fdfb0ff9640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf9406a050 0x7fdf9406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.926+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdfac012070 con 0x7fdfb4159bf0 2023-08-25T09:47:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.926+0000 7fdfb37fe640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf9406a050 0x7fdf9406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:46.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.927+0000 7fdfb37fe640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf9406a050 0x7fdf9406c510 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4036000 tx=0x7fdfa403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:46.930+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdfac054660 con 0x7fdfb4159bf0 2023-08-25T09:47:47.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.120+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdfb4066fb0 con 0x7fdfb4159bf0 2023-08-25T09:47:47.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.121+0000 7fdfb0ff9640 1 -- 172.21.15.70:0/3733170235 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v153) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdfac057c80 con 0x7fdfb4159bf0 2023-08-25T09:47:47.123 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:47:47.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf9406a050 msgr2=0x7fdf9406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf9406a050 0x7fdf9406c510 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4036000 tx=0x7fdfa403c040 comp rx=0 tx=0).stop 2023-08-25T09:47:47.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 msgr2=0x7fdfb4153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb4153d50 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00cd70 tx=0x7fdfac00a660 comp rx=0 tx=0).stop 2023-08-25T09:47:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 shutdown_connections 2023-08-25T09:47:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdf9406a050 0x7fdf9406c510 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfb4159bf0 0x7fdfb4153d50 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfb40cd660 0x7fdfb4156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 --2- 172.21.15.70:0/3733170235 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfb406f0e0 0x7fdfb41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.123+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 >> 172.21.15.70:0/3733170235 conn(0x7fdfb40cb0d0 msgr2=0x7fdfb40ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:47.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.124+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 shutdown_connections 2023-08-25T09:47:47.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.124+0000 7fdfb9eb8640 1 -- 172.21.15.70:0/3733170235 wait complete. 2023-08-25T09:47:47.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:47:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.345+0000 7f3811257640 1 Processor -- start 2023-08-25T09:47:47.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.345+0000 7f3811257640 1 -- start start 2023-08-25T09:47:47.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.346+0000 7f3811257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 0x7f380c069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.346+0000 7f3811257640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c0ac880 con 0x7f380c069850 2023-08-25T09:47:47.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.346+0000 7f3811257640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c0ad400 con 0x7f380c0cca20 2023-08-25T09:47:47.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.346+0000 7f3811257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c15e650 con 0x7f380c15e210 2023-08-25T09:47:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.347+0000 7f380ad9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f380c069850 0x7f380c0c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41448/0 (socket says 172.21.15.70:41448) 2023-08-25T09:47:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.347+0000 7f380ad9d640 1 -- 172.21.15.70:0/3421469609 learned_addr learned my addr 172.21.15.70:0/3421469609 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.347+0000 7f3803fff640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 0x7f380c069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.347+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258523552 0 0) 0x7f380c0ac880 con 0x7f380c069850 2023-08-25T09:47:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.347+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f8003520 con 0x7f380c069850 2023-08-25T09:47:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4201372349 0 0) 0x7f37f8003520 con 0x7f380c069850 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 msgr2=0x7f380c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 0x7f380c069c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 >> v1:172.21.15.155:6789/0 conn(0x7f380c0cca20 legacy=0x7f380c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0ad890 con 0x7f380c069850 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f37fc001670 con 0x7f380c069850 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 >> v1:172.21.15.70:6789/0 conn(0x7f380c069850 legacy=0x7f380c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 0x7f37f80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f8003f20 0x7f37f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8004d00 0x7f37f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c15e650 con 0x7f37f8003f20 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f800baf0 con 0x7f37f8004d00 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f800bcb0 con 0x7f380c15e210 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f3803fff640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f8003f20 0x7f37f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f380b59e640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8004d00 0x7f37f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f3803fff640 1 -- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 msgr2=0x7f37f80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f3803fff640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c15e210 0x7f37f80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f3803fff640 1 -- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8004d00 msgr2=0x7f37f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f3803fff640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8004d00 0x7f37f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.348+0000 7f3803fff640 1 -- 172.21.15.70:0/3421469609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8003520 con 0x7f37f8003f20 2023-08-25T09:47:47.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.349+0000 7f380b59e640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8004d00 0x7f37f8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3803fff640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f8003f20 0x7f37f80047c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009ac0 tx=0x7f37f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb366c76f76897b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f402c710 con 0x7f37f8003f20 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f402c870 con 0x7f37f8003f20 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f380a59c640 1 -- 172.21.15.70:0/3421469609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f402cc00 con 0x7f37f8003f20 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3811257640 1 -- 172.21.15.70:0/3421469609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f8003f20 msgr2=0x7f37f80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3811257640 1 --2- 172.21.15.70:0/3421469609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f37f8003f20 0x7f37f80047c0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009ac0 tx=0x7f37f80057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f380b59e640 1 -- 172.21.15.70:0/3421469609 reap_dead start 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3811257640 1 -- 172.21.15.70:0/3421469609 shutdown_connections 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3811257640 1 -- 172.21.15.70:0/3421469609 >> 172.21.15.70:0/3421469609 conn(0x7f380c0cb0d0 msgr2=0x7f380c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3811257640 1 -- 172.21.15.70:0/3421469609 shutdown_connections 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.350+0000 7f3811257640 1 -- 172.21.15.70:0/3421469609 wait complete. 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.352+0000 7f3811257640 1 Processor -- start 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.352+0000 7f3811257640 1 -- start start 2023-08-25T09:47:47.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.353+0000 7f3811257640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8003f20 0x7f380c0c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.353+0000 7f3811257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 0x7f380c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.353+0000 7f3811257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c146390 0x7f380c0ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.353+0000 7f3811257640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0c70c0 con 0x7f380c0c74e0 2023-08-25T09:47:47.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.353+0000 7f3811257640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0b1090 con 0x7f37f8003f20 2023-08-25T09:47:47.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.353+0000 7f3811257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0c6440 con 0x7f380c146390 2023-08-25T09:47:47.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 0x7f380c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 0x7f380c0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47736/0 (socket says 172.21.15.70:47736) 2023-08-25T09:47:47.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 -- 172.21.15.70:0/1573903943 learned_addr learned my addr 172.21.15.70:0/1573903943 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:47.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f380ad9d640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8003f20 0x7f380c0c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 -- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c146390 msgr2=0x7f380c0ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f380b59e640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c146390 0x7f380c0ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c146390 0x7f380c0ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 -- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8003f20 msgr2=0x7f380c0c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8003f20 0x7f380c0c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f3803fff640 1 -- 172.21.15.70:0/1573903943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0c6720 con 0x7f380c0c74e0 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.354+0000 7f380ad9d640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8003f20 0x7f380c0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.355+0000 7f3803fff640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 0x7f380c0b0b50 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f37f402c2b0 tx=0x7f37f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.355+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4003810 con 0x7f380c0c74e0 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.355+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f402de90 con 0x7f380c0c74e0 2023-08-25T09:47:47.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.355+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f4003b90 con 0x7f380c0c74e0 2023-08-25T09:47:47.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.355+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c0c6a00 con 0x7f380c0c74e0 2023-08-25T09:47:47.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.355+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c146810 con 0x7f380c0c74e0 2023-08-25T09:47:47.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.356+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380c066fb0 con 0x7f380c0c74e0 2023-08-25T09:47:47.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.357+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f37f4003cf0 con 0x7f380c0c74e0 2023-08-25T09:47:47.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.358+0000 7f38037fe640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37ec06a050 0x7f37ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.358+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f37f40aa5c0 con 0x7f380c0c74e0 2023-08-25T09:47:47.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.358+0000 7f380ad9d640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37ec06a050 0x7f37ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.359+0000 7f380ad9d640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37ec06a050 0x7f37ec06c510 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f37fc001eb0 tx=0x7f37fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:47.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.362+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f37f4075de0 con 0x7f380c0c74e0 2023-08-25T09:47:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.556+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f380c06a810 con 0x7f380c0c74e0 2023-08-25T09:47:47.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.557+0000 7f38037fe640 1 -- 172.21.15.70:0/1573903943 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v154) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37f4079400 con 0x7f380c0c74e0 2023-08-25T09:47:47.559 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:47:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37ec06a050 msgr2=0x7f37ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37ec06a050 0x7f37ec06c510 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f37fc001eb0 tx=0x7f37fc009040 comp rx=0 tx=0).stop 2023-08-25T09:47:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 msgr2=0x7f380c0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 0x7f380c0b0b50 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f37f402c2b0 tx=0x7f37f4004730 comp rx=0 tx=0).stop 2023-08-25T09:47:47.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 shutdown_connections 2023-08-25T09:47:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f37ec06a050 0x7f37ec06c510 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f380c146390 0x7f380c0ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f380c0c74e0 0x7f380c0b0b50 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 --2- 172.21.15.70:0/1573903943 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f37f8003f20 0x7f380c0c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 >> 172.21.15.70:0/1573903943 conn(0x7f380c0cb0d0 msgr2=0x7f380c06c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:47.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.560+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 shutdown_connections 2023-08-25T09:47:47.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.561+0000 7f3811257640 1 -- 172.21.15.70:0/1573903943 wait complete. 2023-08-25T09:47:47.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:47:47.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.794+0000 7f209aead640 1 Processor -- start 2023-08-25T09:47:47.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.795+0000 7f209aead640 1 -- start start 2023-08-25T09:47:47.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.796+0000 7f209aead640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f209415cb60 0x7f20940cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.796+0000 7f209aead640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 0x7f20940b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.796+0000 7f209aead640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20940d0fe0 con 0x7f20940d0b00 2023-08-25T09:47:47.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.796+0000 7f209aead640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20940d12c0 con 0x7f209415cb60 2023-08-25T09:47:47.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.796+0000 7f209aead640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20940d15a0 con 0x7f20940b5060 2023-08-25T09:47:47.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f2093fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f20940d0b00 0x7f20940b4950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41464/0 (socket says 172.21.15.70:41464) 2023-08-25T09:47:47.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f2093fff640 1 -- 172.21.15.70:0/1352460575 learned_addr learned my addr 172.21.15.70:0/1352460575 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:47.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965012859 0 0) 0x7f20940d0fe0 con 0x7f20940d0b00 2023-08-25T09:47:47.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2084003520 con 0x7f20940d0b00 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f2098c49640 1 --2- 172.21.15.70:0/1352460575 >> v2:172.21.15.155:3300/0 conn(0x7f209415cb60 0x7f20940cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f209944a640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 0x7f20940b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2683879315 0 0) 0x7f2084003520 con 0x7f20940d0b00 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 msgr2=0x7f20940b5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 0x7f20940b5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 >> v2:172.21.15.155:3300/0 conn(0x7f209415cb60 msgr2=0x7f20940cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 --2- 172.21.15.70:0/1352460575 >> v2:172.21.15.155:3300/0 conn(0x7f209415cb60 0x7f20940cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940d1880 con 0x7f20940d0b00 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f2080001670 con 0x7f20940d0b00 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 >> v1:172.21.15.70:6789/0 conn(0x7f20940d0b00 legacy=0x7f20940b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 0x7f2084003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.797+0000 7f20937fe640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f20937fe640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2084004d00 0x7f2084005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20940d12c0 con 0x7f2084003fa0 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20940d15a0 con 0x7f2084004d00 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208400bb10 con 0x7f20940b5060 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2093fff640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2098c49640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 0x7f2084003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f209944a640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2084004d00 0x7f2084005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2093fff640 1 -- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 msgr2=0x7f2084003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2093fff640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940b5060 0x7f2084003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2093fff640 1 -- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2084004d00 msgr2=0x7f2084005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:47.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2093fff640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2084004d00 0x7f2084005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.798+0000 7f2093fff640 1 -- 172.21.15.70:0/1352460575 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084003520 con 0x7f2084003fa0 2023-08-25T09:47:47.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.799+0000 7f2093fff640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20840047c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2080007fd0 tx=0x7f20840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b568f6b9269ac3bf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:47.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080005ad0 con 0x7f2084003fa0 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080005c30 con 0x7f2084003fa0 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f20937fe640 1 -- 172.21.15.70:0/1352460575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208002cab0 con 0x7f2084003fa0 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209aead640 1 -- 172.21.15.70:0/1352460575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 msgr2=0x7f20840047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209aead640 1 --2- 172.21.15.70:0/1352460575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20840047c0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2080007fd0 tx=0x7f20840057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209944a640 1 -- 172.21.15.70:0/1352460575 reap_dead start 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209aead640 1 -- 172.21.15.70:0/1352460575 shutdown_connections 2023-08-25T09:47:47.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209aead640 1 -- 172.21.15.70:0/1352460575 >> 172.21.15.70:0/1352460575 conn(0x7f2094070080 msgr2=0x7f2094070490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209aead640 1 -- 172.21.15.70:0/1352460575 shutdown_connections 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.800+0000 7f209aead640 1 -- 172.21.15.70:0/1352460575 wait complete. 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.802+0000 7f209aead640 1 Processor -- start 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.802+0000 7f209aead640 1 -- start start 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.803+0000 7f209aead640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20940c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.803+0000 7f209aead640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940cb9f0 0x7f20940cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.803+0000 7f209aead640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2094149e10 0x7f209414a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.803+0000 7f209aead640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20940cc390 con 0x7f2084003fa0 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.803+0000 7f209aead640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209414b770 con 0x7f2094149e10 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.803+0000 7f209aead640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20940c94c0 con 0x7f20940cb9f0 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20940c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20940c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47756/0 (socket says 172.21.15.70:47756) 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 -- 172.21.15.70:0/558593437 learned_addr learned my addr 172.21.15.70:0/558593437 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:47.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 -- 172.21.15.70:0/558593437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940cb9f0 msgr2=0x7f20940cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940cb9f0 0x7f20940cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 -- 172.21.15.70:0/558593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2094149e10 msgr2=0x7f209414a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2094149e10 0x7f209414a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 -- 172.21.15.70:0/558593437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209414ab60 con 0x7f2084003fa0 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.804+0000 7f2098c49640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20940c8f80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f208800f600 tx=0x7f208800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.805+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088014870 con 0x7f2084003fa0 2023-08-25T09:47:47.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.805+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20880149d0 con 0x7f2084003fa0 2023-08-25T09:47:47.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.805+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088014d40 con 0x7f2084003fa0 2023-08-25T09:47:47.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.805+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209414ae40 con 0x7f2084003fa0 2023-08-25T09:47:47.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.805+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209414b400 con 0x7f2084003fa0 2023-08-25T09:47:47.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.806+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2094066fb0 con 0x7f2084003fa0 2023-08-25T09:47:47.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.807+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f208800d490 con 0x7f2084003fa0 2023-08-25T09:47:47.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.808+0000 7f20917fa640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f207806a050 0x7f207806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:47.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.808+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2088089110 con 0x7f2084003fa0 2023-08-25T09:47:47.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.808+0000 7f2093fff640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f207806a050 0x7f207806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:47.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.808+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f20780721a0 con 0x7f2084003fa0 2023-08-25T09:47:47.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.809+0000 7f2093fff640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f207806a050 0x7f207806c510 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2080034000 tx=0x7f2080039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:47.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.812+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2088054760 con 0x7f2084003fa0 2023-08-25T09:47:48.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:47.999+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20940b5e80 con 0x7f2084003fa0 2023-08-25T09:47:48.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.000+0000 7f20917fa640 1 -- 172.21.15.70:0/558593437 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v154) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2088057d80 con 0x7f2084003fa0 2023-08-25T09:47:48.002 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:47:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f207806a050 msgr2=0x7f207806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f207806a050 0x7f207806c510 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2080034000 tx=0x7f2080039040 comp rx=0 tx=0).stop 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 msgr2=0x7f20940c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20940c8f80 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f208800f600 tx=0x7f208800fac0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 shutdown_connections 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f207806a050 0x7f207806c510 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2094149e10 0x7f209414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f20940cb9f0 0x7f20940cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 --2- 172.21.15.70:0/558593437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2084003fa0 0x7f20940c8f80 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 >> 172.21.15.70:0/558593437 conn(0x7f2094070080 msgr2=0x7f20940b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 shutdown_connections 2023-08-25T09:47:48.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.003+0000 7f209aead640 1 -- 172.21.15.70:0/558593437 wait complete. 2023-08-25T09:47:48.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:47:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.228+0000 7fd651621640 1 Processor -- start 2023-08-25T09:47:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.228+0000 7fd651621640 1 -- start start 2023-08-25T09:47:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd651621640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 0x7fd64c0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd651621640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c15e4a0 con 0x7fd64c15abb0 2023-08-25T09:47:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd651621640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c15f020 con 0x7fd64c071850 2023-08-25T09:47:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd651621640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0c53b0 con 0x7fd64c15a7d0 2023-08-25T09:47:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd643fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd64c15abb0 0x7fd64c0b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41466/0 (socket says 172.21.15.70:41466) 2023-08-25T09:47:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd643fff640 1 -- 172.21.15.70:0/776605182 learned_addr learned my addr 172.21.15.70:0/776605182 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227274473 0 0) 0x7fd64c15e4a0 con 0x7fd64c15abb0 2023-08-25T09:47:48.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd638003520 con 0x7fd64c15abb0 2023-08-25T09:47:48.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.230+0000 7fd64bfff640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 0x7fd64c0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:48.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2372546371 0 0) 0x7fd638003520 con 0x7fd64c15abb0 2023-08-25T09:47:48.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 msgr2=0x7fd64c0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:48.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 0x7fd64c0b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 >> v1:172.21.15.155:6789/0 conn(0x7fd64c071850 legacy=0x7fd64c06ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c15e6f0 con 0x7fd64c15abb0 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fd634001670 con 0x7fd64c15abb0 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 >> v1:172.21.15.70:6789/0 conn(0x7fd64c15abb0 legacy=0x7fd64c0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd638003a10 0x7fd638003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd638004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 0x7fd63800fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638005590 con 0x7fd6380047c0 2023-08-25T09:47:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6380102d0 con 0x7fd638003a10 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd638010410 con 0x7fd64c15a7d0 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.231+0000 7fd643fff640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd638004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.232+0000 7fd650e20640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 0x7fd63800fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.232+0000 7fd643fff640 1 -- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 msgr2=0x7fd63800fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.232+0000 7fd643fff640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c15a7d0 0x7fd63800fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.232+0000 7fd643fff640 1 -- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd638003a10 msgr2=0x7fd638003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.232+0000 7fd643fff640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd638003a10 0x7fd638003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.232+0000 7fd643fff640 1 -- 172.21.15.70:0/776605182 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd638003520 con 0x7fd6380047c0 2023-08-25T09:47:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.233+0000 7fd643fff640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd638004c40 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c210 tx=0x7fd6380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd1b2a7cd0af2fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.233+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63402cae0 con 0x7fd6380047c0 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.233+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63402cc60 con 0x7fd6380047c0 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.233+0000 7fd64b7fe640 1 -- 172.21.15.70:0/776605182 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340058d0 con 0x7fd6380047c0 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd651621640 1 -- 172.21.15.70:0/776605182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 msgr2=0x7fd638004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd651621640 1 --2- 172.21.15.70:0/776605182 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd638004c40 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fd63400c210 tx=0x7fd6380057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd650e20640 1 -- 172.21.15.70:0/776605182 reap_dead start 2023-08-25T09:47:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd651621640 1 -- 172.21.15.70:0/776605182 shutdown_connections 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd651621640 1 -- 172.21.15.70:0/776605182 >> 172.21.15.70:0/776605182 conn(0x7fd64c0aff60 msgr2=0x7fd64c0b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd651621640 1 -- 172.21.15.70:0/776605182 shutdown_connections 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.234+0000 7fd651621640 1 -- 172.21.15.70:0/776605182 wait complete. 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.235+0000 7fd651621640 1 Processor -- start 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 -- start start 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd64c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64c0cc000 0x7fd64c150290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c154740 0x7fd64c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c155100 con 0x7fd6380047c0 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c153050 con 0x7fd64c0cc000 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.236+0000 7fd651621640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1511c0 con 0x7fd64c154740 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd64c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd64c155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:47772/0 (socket says 172.21.15.70:47772) 2023-08-25T09:47:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 -- 172.21.15.70:0/2247218197 learned_addr learned my addr 172.21.15.70:0/2247218197 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 -- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c154740 msgr2=0x7fd64c154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd650e20640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c154740 0x7fd64c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c154740 0x7fd64c154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 -- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64c0cc000 msgr2=0x7fd64c150290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64c0cc000 0x7fd64c150290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd64bfff640 1 -- 172.21.15.70:0/2247218197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0c2e90 con 0x7fd6380047c0 2023-08-25T09:47:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.237+0000 7fd650e20640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c154740 0x7fd64c154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:48.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.238+0000 7fd64bfff640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd64c155620 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00d800 tx=0x7fd63c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:48.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.238+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00b6c0 con 0x7fd6380047c0 2023-08-25T09:47:48.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.238+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63c00b840 con 0x7fd6380047c0 2023-08-25T09:47:48.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.238+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c0cc7e0 con 0x7fd6380047c0 2023-08-25T09:47:48.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.238+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c00bbf0 con 0x7fd6380047c0 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.238+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c0ccda0 con 0x7fd6380047c0 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.239+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c066fb0 con 0x7fd6380047c0 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.240+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd63c00bd50 con 0x7fd6380047c0 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.241+0000 7fd6497fa640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd62c06a050 0x7fd62c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.241+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd63c088b30 con 0x7fd6380047c0 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.242+0000 7fd643fff640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd62c06a050 0x7fd62c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:48.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.242+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fd62c0721a0 con 0x7fd6380047c0 2023-08-25T09:47:48.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.242+0000 7fd643fff640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd62c06a050 0x7fd62c06c510 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd634034000 tx=0x7fd634007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:48.247 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.246+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd63c05be60 con 0x7fd6380047c0 2023-08-25T09:47:48.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.449+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd64c0b0d60 con 0x7fd6380047c0 2023-08-25T09:47:48.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.450+0000 7fd6497fa640 1 -- 172.21.15.70:0/2247218197 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v154) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd63c018030 con 0x7fd6380047c0 2023-08-25T09:47:48.451 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:47:48.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd62c06a050 msgr2=0x7fd62c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd62c06a050 0x7fd62c06c510 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fd634034000 tx=0x7fd634007000 comp rx=0 tx=0).stop 2023-08-25T09:47:48.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 msgr2=0x7fd64c155620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:48.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd64c155620 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00d800 tx=0x7fd63c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 shutdown_connections 2023-08-25T09:47:48.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd62c06a050 0x7fd62c06c510 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd64c154740 0x7fd64c154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd64c0cc000 0x7fd64c150290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 --2- 172.21.15.70:0/2247218197 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6380047c0 0x7fd64c155620 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:48.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 >> 172.21.15.70:0/2247218197 conn(0x7fd64c0aff60 msgr2=0x7fd64c1561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:48.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 shutdown_connections 2023-08-25T09:47:48.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:48.452+0000 7fd651621640 1 -- 172.21.15.70:0/2247218197 wait complete. 2023-08-25T09:47:48.471 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:47:48.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:48.472 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:47:48.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[91974]: fuse finished with error 0 and tester_r 0 2023-08-25T09:47:48.582 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:47:48.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:48.583 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:47:48.762 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:47:48.763 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:48.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:47:48.824 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:47:48.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:48.825 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:47:48.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:47:48.884 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:47:48.884 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:47:48.885 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:47:48.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:47:48.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:47:48.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:47:48.952 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:47:49.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:47:49.009 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:47:49.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:49.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.235+0000 7fe75e234640 1 Processor -- start 2023-08-25T09:47:49.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.235+0000 7fe75e234640 1 -- start start 2023-08-25T09:47:49.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe75e234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75815cb40 0x7fe7580ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe75e234640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7580ac870 0x7fe7580ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe75e234640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7580b3880 0x7fe758160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe75e234640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75806f050 con 0x7fe7580b3880 2023-08-25T09:47:49.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe75e234640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75806f330 con 0x7fe7580ac870 2023-08-25T09:47:49.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe75e234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75806f610 con 0x7fe75815cb40 2023-08-25T09:47:49.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe757fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7580b3880 0x7fe758160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe757fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe7580b3880 0x7fe758160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58836/0 (socket says 172.21.15.70:58836) 2023-08-25T09:47:49.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe757fff640 1 -- 172.21.15.70:0/919866752 learned_addr learned my addr 172.21.15.70:0/919866752 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:49.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe7577fe640 1 --2- 172.21.15.70:0/919866752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75815cb40 0x7fe7580ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe756ffd640 1 --2- 172.21.15.70:0/919866752 >> v2:172.21.15.155:3300/0 conn(0x7fe7580ac870 0x7fe7580ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.237+0000 7fe757fff640 1 -- 172.21.15.70:0/919866752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75815cb40 msgr2=0x7fe7580ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:49.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.238+0000 7fe757fff640 1 --2- 172.21.15.70:0/919866752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75815cb40 0x7fe7580ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.238+0000 7fe757fff640 1 -- 172.21.15.70:0/919866752 >> v2:172.21.15.155:3300/0 conn(0x7fe7580ac870 msgr2=0x7fe7580ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:49.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.238+0000 7fe757fff640 1 --2- 172.21.15.70:0/919866752 >> v2:172.21.15.155:3300/0 conn(0x7fe7580ac870 0x7fe7580ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.238+0000 7fe757fff640 1 -- 172.21.15.70:0/919866752 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75806f8f0 con 0x7fe7580b3880 2023-08-25T09:47:49.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.238+0000 7fe756ffd640 1 --2- 172.21.15.70:0/919866752 >> v2:172.21.15.155:3300/0 conn(0x7fe7580ac870 0x7fe7580ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:49.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.239+0000 7fe757fff640 1 --2- 172.21.15.70:0/919866752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580b3880 0x7fe758160790 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fe75806a7e0 tx=0x7fe75002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296cc6ec233cf202 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:49.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.239+0000 7fe7567fc640 1 -- 172.21.15.70:0/919866752 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75002ca70 con 0x7fe7580b3880 2023-08-25T09:47:49.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.239+0000 7fe7567fc640 1 -- 172.21.15.70:0/919866752 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75002cbd0 con 0x7fe7580b3880 2023-08-25T09:47:49.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.239+0000 7fe7567fc640 1 -- 172.21.15.70:0/919866752 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7500343f0 con 0x7fe7580b3880 2023-08-25T09:47:49.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 -- 172.21.15.70:0/919866752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580b3880 msgr2=0x7fe758160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 --2- 172.21.15.70:0/919866752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580b3880 0x7fe758160790 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fe75806a7e0 tx=0x7fe75002bed0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 -- 172.21.15.70:0/919866752 shutdown_connections 2023-08-25T09:47:49.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 --2- 172.21.15.70:0/919866752 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7580b3880 0x7fe758160790 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 --2- 172.21.15.70:0/919866752 >> v2:172.21.15.155:3300/0 conn(0x7fe7580ac870 0x7fe7580ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 --2- 172.21.15.70:0/919866752 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe75815cb40 0x7fe7580ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 -- 172.21.15.70:0/919866752 >> 172.21.15.70:0/919866752 conn(0x7fe758069850 msgr2=0x7fe758069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:49.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 -- 172.21.15.70:0/919866752 shutdown_connections 2023-08-25T09:47:49.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.240+0000 7fe75e234640 1 -- 172.21.15.70:0/919866752 wait complete. 2023-08-25T09:47:49.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.242+0000 7fe75e234640 1 Processor -- start 2023-08-25T09:47:49.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.242+0000 7fe75e234640 1 -- start start 2023-08-25T09:47:49.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe75e234640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7580ac870 0x7fe7581f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe75e234640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7580b3880 0x7fe75814b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe75e234640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 0x7fe75814dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe75e234640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7581f6c90 con 0x7fe75815cb40 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe75e234640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75814bcc0 con 0x7fe7580b3880 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe75e234640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75814b720 con 0x7fe7580ac870 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 0x7fe75814dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.249 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 0x7fe75814dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58848/0 (socket says 172.21.15.70:58848) 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 -- 172.21.15.70:0/1644657660 learned_addr learned my addr 172.21.15.70:0/1644657660 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 -- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7580ac870 msgr2=0x7fe7581f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe756ffd640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7580b3880 0x7fe75814b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe7577fe640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7580ac870 0x7fe7581f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7580ac870 0x7fe7581f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 -- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7580b3880 msgr2=0x7fe75814b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.243+0000 7fe757fff640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7580b3880 0x7fe75814b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.250 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe757fff640 1 -- 172.21.15.70:0/1644657660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750009b30 con 0x7fe75815cb40 2023-08-25T09:47:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe757fff640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 0x7fe75814dbd0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe750036040 tx=0x7fe750004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750009510 con 0x7fe75815cb40 2023-08-25T09:47:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75003e450 con 0x7fe75815cb40 2023-08-25T09:47:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75003e800 con 0x7fe75815cb40 2023-08-25T09:47:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe75814cfb0 con 0x7fe75815cb40 2023-08-25T09:47:49.251 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.244+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75814c1f0 con 0x7fe75815cb40 2023-08-25T09:47:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.245+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75804e420 con 0x7fe75815cb40 2023-08-25T09:47:49.252 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.246+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe75003e960 con 0x7fe75815cb40 2023-08-25T09:47:49.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.247+0000 7fe73ffff640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe74406a050 0x7fe74406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.247+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe7500ab410 con 0x7fe75815cb40 2023-08-25T09:47:49.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.248+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fe7440721a0 con 0x7fe75815cb40 2023-08-25T09:47:49.253 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.248+0000 7fe7577fe640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe74406a050 0x7fe74406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.248+0000 7fe7577fe640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe74406a050 0x7fe74406c510 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b7c0 tx=0x7fe748008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:49.254 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.252+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe750076ba0 con 0x7fe75815cb40 2023-08-25T09:47:49.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.477+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe75815c0c0 con 0x7fe75815cb40 2023-08-25T09:47:49.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.478+0000 7fe73ffff640 1 -- 172.21.15.70:0/1644657660 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7fe750043070 con 0x7fe75815cb40 2023-08-25T09:47:49.483 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:49.483 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:49.484 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:49.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe74406a050 msgr2=0x7fe74406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe74406a050 0x7fe74406c510 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe74800b7c0 tx=0x7fe748008040 comp rx=0 tx=0).stop 2023-08-25T09:47:49.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 msgr2=0x7fe75814dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 0x7fe75814dbd0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fe750036040 tx=0x7fe750004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:49.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 shutdown_connections 2023-08-25T09:47:49.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe74406a050 0x7fe74406c510 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe75815cb40 0x7fe75814dbd0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7580b3880 0x7fe75814b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 --2- 172.21.15.70:0/1644657660 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7580ac870 0x7fe7581f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 >> 172.21.15.70:0/1644657660 conn(0x7fe758069850 msgr2=0x7fe7580654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 shutdown_connections 2023-08-25T09:47:49.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.481+0000 7fe75e234640 1 -- 172.21.15.70:0/1644657660 wait complete. 2023-08-25T09:47:49.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:49.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.720+0000 7fc1aaeec640 1 Processor -- start 2023-08-25T09:47:49.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.721+0000 7fc1aaeec640 1 -- start start 2023-08-25T09:47:49.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.722+0000 7fc1aaeec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a415cc90 0x7fc1a40d0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.722+0000 7fc1aaeec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.722+0000 7fc1aaeec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40d1fd0 con 0x7fc1a415d070 2023-08-25T09:47:49.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.722+0000 7fc1aaeec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a40d28a0 con 0x7fc1a415cc90 2023-08-25T09:47:49.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.722+0000 7fc1aaeec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40d13c0 con 0x7fc1a40cfa10 2023-08-25T09:47:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.722+0000 7fc1a8c88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a415cc90 0x7fc1a40d0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a8c88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc1a415cc90 0x7fc1a40d0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58852/0 (socket says 172.21.15.70:58852) 2023-08-25T09:47:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a8c88640 1 -- 172.21.15.70:0/761561032 learned_addr learned my addr 172.21.15.70:0/761561032 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:49.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a9489640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a8c88640 1 -- 172.21.15.70:0/761561032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 msgr2=0x7fc1a40b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a8c88640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a8c88640 1 -- 172.21.15.70:0/761561032 >> v1:172.21.15.155:6789/0 conn(0x7fc1a415d070 legacy=0x7fc1a40cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a8c88640 1 -- 172.21.15.70:0/761561032 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a40d16a0 con 0x7fc1a415cc90 2023-08-25T09:47:49.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.723+0000 7fc1a9489640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.725+0000 7fc1a8c88640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a40d0cc0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009870 tx=0x7fc18c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af805e70eb71e0e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.725+0000 7fc1a37fe640 1 -- 172.21.15.70:0/761561032 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c031640 con 0x7fc1a415cc90 2023-08-25T09:47:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.725+0000 7fc1a37fe640 1 -- 172.21.15.70:0/761561032 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c0317a0 con 0x7fc1a415cc90 2023-08-25T09:47:49.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.725+0000 7fc1a37fe640 1 -- 172.21.15.70:0/761561032 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c031b30 con 0x7fc1a415cc90 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 -- 172.21.15.70:0/761561032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 msgr2=0x7fc1a40d0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a40d0cc0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009870 tx=0x7fc18c02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 -- 172.21.15.70:0/761561032 shutdown_connections 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/761561032 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a40d0cc0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 -- 172.21.15.70:0/761561032 >> 172.21.15.70:0/761561032 conn(0x7fc1a4070260 msgr2=0x7fc1a4070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:49.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 -- 172.21.15.70:0/761561032 shutdown_connections 2023-08-25T09:47:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.726+0000 7fc1aaeec640 1 -- 172.21.15.70:0/761561032 wait complete. 2023-08-25T09:47:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.727+0000 7fc1aaeec640 1 Processor -- start 2023-08-25T09:47:49.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.728+0000 7fc1aaeec640 1 -- start start 2023-08-25T09:47:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1aaeec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1aaeec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1aaeec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a415d070 0x7fc1a415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1aaeec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40b3920 con 0x7fc1a415cc90 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1aaeec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40b3640 con 0x7fc1a415d070 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1aaeec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a40b3e60 con 0x7fc1a40cfa10 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a4155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58856/0 (socket says 172.21.15.70:58856) 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 -- 172.21.15.70:0/429360769 learned_addr learned my addr 172.21.15.70:0/429360769 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 -- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 msgr2=0x7fc1a40b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 -- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a415d070 msgr2=0x7fc1a415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a9489640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a415d070 0x7fc1a415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a415d070 0x7fc1a415f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.729+0000 7fc1a3fff640 1 -- 172.21.15.70:0/429360769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0094d0 con 0x7fc1a415cc90 2023-08-25T09:47:49.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.730+0000 7fc1a3fff640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a4155340 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc19400ba50 tx=0x7fc19400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:49.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.730+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400cbe0 con 0x7fc1a415cc90 2023-08-25T09:47:49.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.730+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19400cd80 con 0x7fc1a415cc90 2023-08-25T09:47:49.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.730+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194014680 con 0x7fc1a415cc90 2023-08-25T09:47:49.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.730+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a40b4140 con 0x7fc1a415cc90 2023-08-25T09:47:49.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.730+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a415fc90 con 0x7fc1a415cc90 2023-08-25T09:47:49.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.731+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a40b4660 con 0x7fc1a415cc90 2023-08-25T09:47:49.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.732+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc1940147e0 con 0x7fc1a415cc90 2023-08-25T09:47:49.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.733+0000 7fc1a17fa640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:49.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.733+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc194088410 con 0x7fc1a415cc90 2023-08-25T09:47:49.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.733+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fc1840721d0 con 0x7fc1a415cc90 2023-08-25T09:47:49.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.733+0000 7fc1a8c88640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:49.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.734+0000 7fc1a8c88640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc18c036000 tx=0x7fc18c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.737+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc1a40b4660 con 0x7fc1a415cc90 2023-08-25T09:47:49.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.925+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1a40b4660 con 0x7fc1a415cc90 2023-08-25T09:47:49.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.929+0000 7fc1a17fa640 1 -- 172.21.15.70:0/429360769 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18407 (secure 0 0 0) 0x7fc1a40b4660 con 0x7fc1a415cc90 2023-08-25T09:47:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:49.931 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":154,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:47:47.394216+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:47:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 msgr2=0x7fc18406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc18c036000 tx=0x7fc18c030000 comp rx=0 tx=0).stop 2023-08-25T09:47:49.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 msgr2=0x7fc1a4155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a4155340 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fc19400ba50 tx=0x7fc19400bf10 comp rx=0 tx=0).stop 2023-08-25T09:47:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 shutdown_connections 2023-08-25T09:47:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc18406a080 0x7fc18406c540 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1a415d070 0x7fc1a415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1a415cc90 0x7fc1a4155340 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 --2- 172.21.15.70:0/429360769 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1a40cfa10 0x7fc1a40b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.931+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 >> 172.21.15.70:0/429360769 conn(0x7fc1a4070260 msgr2=0x7fc1a40c7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.932+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 shutdown_connections 2023-08-25T09:47:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:49.932+0000 7fc1aaeec640 1 -- 172.21.15.70:0/429360769 wait complete. 2023-08-25T09:47:49.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:46:56.533109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:46:57.303907+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:47:49.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:47:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.177+0000 7f1811214640 1 Processor -- start 2023-08-25T09:47:50.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.178+0000 7f1811214640 1 -- start start 2023-08-25T09:47:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.179+0000 7f1811214640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c15ae20 0x7f180c071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.179+0000 7f1811214640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f180c0b37c0 0x7f180c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.179+0000 7f1811214640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c0cf9a0 con 0x7f180c0cef50 2023-08-25T09:47:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.179+0000 7f1811214640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c15b200 con 0x7f180c0b37c0 2023-08-25T09:47:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.179+0000 7f1811214640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0cc9f0 con 0x7f180c15ae20 2023-08-25T09:47:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f180c0cef50 0x7f180c0cfbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34142/0 (socket says 172.21.15.70:34142) 2023-08-25T09:47:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180a59c640 1 -- 172.21.15.70:0/143666133 learned_addr learned my addr 172.21.15.70:0/143666133 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180b59e640 1 --2- 172.21.15.70:0/143666133 >> v2:172.21.15.70:3300/0 conn(0x7f180c0b37c0 0x7f180c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180ad9d640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c15ae20 0x7f180c071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f1809d9b640 1 -- 172.21.15.70:0/143666133 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303226899 0 0) 0x7f180c0cf9a0 con 0x7f180c0cef50 2023-08-25T09:47:50.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f1809d9b640 1 -- 172.21.15.70:0/143666133 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f8003520 con 0x7f180c0cef50 2023-08-25T09:47:50.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180b59e640 1 -- 172.21.15.70:0/143666133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c15ae20 msgr2=0x7f180c071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:50.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180b59e640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c15ae20 0x7f180c071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180b59e640 1 -- 172.21.15.70:0/143666133 >> v1:172.21.15.70:6789/0 conn(0x7f180c0cef50 legacy=0x7f180c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180b59e640 1 -- 172.21.15.70:0/143666133 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0cc3e0 con 0x7f180c0b37c0 2023-08-25T09:47:50.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.180+0000 7f180ad9d640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c15ae20 0x7f180c071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:47:50.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.181+0000 7f180b59e640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b37c0 0x7f180c0cce80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1804009d90 tx=0x7f180402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487a506e8a851c62 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:50.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1809d9b640 1 -- 172.21.15.70:0/143666133 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180402cce0 con 0x7f180c0b37c0 2023-08-25T09:47:50.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1809d9b640 1 -- 172.21.15.70:0/143666133 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180402ce40 con 0x7f180c0b37c0 2023-08-25T09:47:50.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1809d9b640 1 -- 172.21.15.70:0/143666133 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1804034640 con 0x7f180c0b37c0 2023-08-25T09:47:50.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 -- 172.21.15.70:0/143666133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b37c0 msgr2=0x7f180c0cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b37c0 0x7f180c0cce80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f1804009d90 tx=0x7f180402bf30 comp rx=0 tx=0).stop 2023-08-25T09:47:50.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 -- 172.21.15.70:0/143666133 shutdown_connections 2023-08-25T09:47:50.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b37c0 0x7f180c0cce80 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 --2- 172.21.15.70:0/143666133 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c15ae20 0x7f180c071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 -- 172.21.15.70:0/143666133 >> 172.21.15.70:0/143666133 conn(0x7f180c06ba60 msgr2=0x7f180c06be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:50.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.182+0000 7f1811214640 1 -- 172.21.15.70:0/143666133 shutdown_connections 2023-08-25T09:47:50.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.183+0000 7f1811214640 1 -- 172.21.15.70:0/143666133 wait complete. 2023-08-25T09:47:50.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.184+0000 7f1811214640 1 Processor -- start 2023-08-25T09:47:50.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.184+0000 7f1811214640 1 -- start start 2023-08-25T09:47:50.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.185+0000 7f1811214640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c0b37c0 0x7f180c0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.185+0000 7f1811214640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f180c15ae20 0x7f180c0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.185+0000 7f1811214640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 0x7f180c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.185+0000 7f1811214640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0c3a80 con 0x7f180c0b0280 2023-08-25T09:47:50.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.185+0000 7f1811214640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c1503e0 con 0x7f180c15ae20 2023-08-25T09:47:50.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.185+0000 7f1811214640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c15dba0 con 0x7f180c0b37c0 2023-08-25T09:47:50.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 0x7f180c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 0x7f180c0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58868/0 (socket says 172.21.15.70:58868) 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 -- 172.21.15.70:0/1961794165 learned_addr learned my addr 172.21.15.70:0/1961794165 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180ad9d640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c0b37c0 0x7f180c0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180a59c640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f180c15ae20 0x7f180c0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 -- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c0b37c0 msgr2=0x7f180c0c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c0b37c0 0x7f180c0c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 -- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f180c15ae20 msgr2=0x7f180c0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f180c15ae20 0x7f180c0c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 -- 172.21.15.70:0/1961794165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804009a70 con 0x7f180c0b0280 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180ad9d640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c0b37c0 0x7f180c0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180a59c640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f180c15ae20 0x7f180c0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.186+0000 7f180b59e640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 0x7f180c0c9970 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1804009d90 tx=0x7f1804004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.187+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180403d600 con 0x7f180c0b0280 2023-08-25T09:47:50.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.187+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180403d780 con 0x7f180c0b0280 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.187+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c15de80 con 0x7f180c0b0280 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.187+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180403db30 con 0x7f180c0b0280 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.187+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c0c86f0 con 0x7f180c0b0280 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.188+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c066fb0 con 0x7f180c0b0280 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.189+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f180403b040 con 0x7f180c0b0280 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.190+0000 7f18037fe640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17ec06a050 0x7f17ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.190+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f18040aaa70 con 0x7f180c0b0280 2023-08-25T09:47:50.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.190+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f17ec0721a0 con 0x7f180c0b0280 2023-08-25T09:47:50.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.190+0000 7f180ad9d640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17ec06a050 0x7f17ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.191+0000 7f180ad9d640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17ec06a050 0x7f17ec06c510 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006fd0 tx=0x7f17fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:50.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.194+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1804039060 con 0x7f180c0b0280 2023-08-25T09:47:50.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.383+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f180c0b3df0 con 0x7f180c0b0280 2023-08-25T09:47:50.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.384+0000 7f18037fe640 1 -- 172.21.15.70:0/1961794165 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) v1 ==== 93+0+10 (secure 0 0 0) 0x7f180404e020 con 0x7f180c0b0280 2023-08-25T09:47:50.389 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:47:50.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17ec06a050 msgr2=0x7f17ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17ec06a050 0x7f17ec06c510 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f17fc006fd0 tx=0x7f17fc008040 comp rx=0 tx=0).stop 2023-08-25T09:47:50.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 msgr2=0x7f180c0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 0x7f180c0c9970 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1804009d90 tx=0x7f1804004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:50.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 shutdown_connections 2023-08-25T09:47:50.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17ec06a050 0x7f17ec06c510 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f180c0b0280 0x7f180c0c9970 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f180c15ae20 0x7f180c0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 --2- 172.21.15.70:0/1961794165 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f180c0b37c0 0x7f180c0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 >> 172.21.15.70:0/1961794165 conn(0x7f180c06ba60 msgr2=0x7f180c159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 shutdown_connections 2023-08-25T09:47:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.386+0000 7f1811214640 1 -- 172.21.15.70:0/1961794165 wait complete. 2023-08-25T09:47:50.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:47:50.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.621+0000 7f5ddf28a640 1 Processor -- start 2023-08-25T09:47:50.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.621+0000 7f5ddf28a640 1 -- start start 2023-08-25T09:47:50.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.622+0000 7f5ddf28a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5dd80b0320 0x7f5dd80c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.622+0000 7f5ddf28a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5dd80b1ce0 0x7f5dd80ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.622+0000 7f5ddf28a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd80ad8f0 con 0x7f5dd815a920 2023-08-25T09:47:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.622+0000 7f5ddf28a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd80adbb0 con 0x7f5dd80b1ce0 2023-08-25T09:47:50.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.622+0000 7f5ddf28a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd80c90a0 con 0x7f5dd80b0320 2023-08-25T09:47:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5dd80b1ce0 0x7f5dd80ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5dd80b1ce0 0x7f5dd80ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58870/0 (socket says 172.21.15.70:58870) 2023-08-25T09:47:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 -- 172.21.15.70:0/64389035 learned_addr learned my addr 172.21.15.70:0/64389035 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:50.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5dde288640 1 --2- 172.21.15.70:0/64389035 >> v2:172.21.15.155:3300/0 conn(0x7f5dd80b0320 0x7f5dd80c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 -- 172.21.15.70:0/64389035 >> v2:172.21.15.155:3300/0 conn(0x7f5dd80b0320 msgr2=0x7f5dd80c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 --2- 172.21.15.70:0/64389035 >> v2:172.21.15.155:3300/0 conn(0x7f5dd80b0320 0x7f5dd80c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 -- 172.21.15.70:0/64389035 >> v1:172.21.15.155:6789/0 conn(0x7f5dd815a920 legacy=0x7f5dd80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:50.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.623+0000 7f5ddea89640 1 -- 172.21.15.70:0/64389035 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd80c9380 con 0x7f5dd80b1ce0 2023-08-25T09:47:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.624+0000 7f5ddea89640 1 --2- 172.21.15.70:0/64389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b1ce0 0x7f5dd80ad3b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80b29f0 tx=0x7f5dd002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b346c79b0ab2d6b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddd286640 1 -- 172.21.15.70:0/64389035 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd002ca80 con 0x7f5dd80b1ce0 2023-08-25T09:47:50.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddd286640 1 -- 172.21.15.70:0/64389035 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd002cbe0 con 0x7f5dd80b1ce0 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddd286640 1 -- 172.21.15.70:0/64389035 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00343f0 con 0x7f5dd80b1ce0 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 -- 172.21.15.70:0/64389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b1ce0 msgr2=0x7f5dd80ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/64389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b1ce0 0x7f5dd80ad3b0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80b29f0 tx=0x7f5dd002bee0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 -- 172.21.15.70:0/64389035 shutdown_connections 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/64389035 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b1ce0 0x7f5dd80ad3b0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/64389035 >> v2:172.21.15.155:3300/0 conn(0x7f5dd80b0320 0x7f5dd80c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 -- 172.21.15.70:0/64389035 >> 172.21.15.70:0/64389035 conn(0x7f5dd80ce830 msgr2=0x7f5dd80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:50.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 -- 172.21.15.70:0/64389035 shutdown_connections 2023-08-25T09:47:50.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.625+0000 7f5ddf28a640 1 -- 172.21.15.70:0/64389035 wait complete. 2023-08-25T09:47:50.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.627+0000 7f5ddf28a640 1 Processor -- start 2023-08-25T09:47:50.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.627+0000 7f5ddf28a640 1 -- start start 2023-08-25T09:47:50.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5ddf28a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 0x7f5dd806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5ddf28a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dd80b1ce0 0x7f5dd80c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5ddf28a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd815a920 0x7f5dd81f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5ddf28a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80c1eb0 con 0x7f5dd80b0320 2023-08-25T09:47:50.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5ddf28a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80c5f10 con 0x7f5dd815a920 2023-08-25T09:47:50.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5ddf28a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80c61f0 con 0x7f5dd80b1ce0 2023-08-25T09:47:50.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5dde288640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 0x7f5dd806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5dde288640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 0x7f5dd806fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58882/0 (socket says 172.21.15.70:58882) 2023-08-25T09:47:50.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.629+0000 7f5dde288640 1 -- 172.21.15.70:0/2106572056 learned_addr learned my addr 172.21.15.70:0/2106572056 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:50.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5ddea89640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd815a920 0x7f5dd81f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dde288640 1 -- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dd80b1ce0 msgr2=0x7f5dd80c5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5ddda87640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dd80b1ce0 0x7f5dd80c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dde288640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dd80b1ce0 0x7f5dd80c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dde288640 1 -- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd815a920 msgr2=0x7f5dd81f2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dde288640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd815a920 0x7f5dd81f2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dde288640 1 -- 172.21.15.70:0/2106572056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0009b30 con 0x7f5dd80b0320 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dde288640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 0x7f5dd806fc20 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d7d0 tx=0x7f5dd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.630+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40096c0 con 0x7f5dd80b0320 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.631+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd4009820 con 0x7f5dd80b0320 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.631+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd80c2100 con 0x7f5dd80b0320 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.631+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4009bd0 con 0x7f5dd80b0320 2023-08-25T09:47:50.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.631+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd80c6f30 con 0x7f5dd80b0320 2023-08-25T09:47:50.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.632+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd8061460 con 0x7f5dd80b0320 2023-08-25T09:47:50.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.633+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5dd4009d30 con 0x7f5dd80b0320 2023-08-25T09:47:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.633+0000 7f5dceffd640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5dbc06a080 0x7f5dbc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.634+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5dd4088d00 con 0x7f5dd80b0320 2023-08-25T09:47:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.634+0000 7f5ddda87640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5dbc06a080 0x7f5dbc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.634+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f5dbc0721d0 con 0x7f5dd80b0320 2023-08-25T09:47:50.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.634+0000 7f5ddda87640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5dbc06a080 0x7f5dbc06c540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80c7660 tx=0x7f5dc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.638+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5dd4057e70 con 0x7f5dd80b0320 2023-08-25T09:47:50.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.821+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5dd80c1c60 con 0x7f5dd80b0320 2023-08-25T09:47:50.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.822+0000 7f5dceffd640 1 -- 172.21.15.70:0/2106572056 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v154) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5dd4057a00 con 0x7f5dd80b0320 2023-08-25T09:47:50.823 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:47:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5dbc06a080 msgr2=0x7f5dbc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5dbc06a080 0x7f5dbc06c540 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80c7660 tx=0x7f5dc8009040 comp rx=0 tx=0).stop 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 msgr2=0x7f5dd806fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 0x7f5dd806fc20 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f5dd400d7d0 tx=0x7f5dd400dc90 comp rx=0 tx=0).stop 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 shutdown_connections 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5dbc06a080 0x7f5dbc06c540 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5dd815a920 0x7f5dd81f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5dd80b1ce0 0x7f5dd80c5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 --2- 172.21.15.70:0/2106572056 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5dd80b0320 0x7f5dd806fc20 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 >> 172.21.15.70:0/2106572056 conn(0x7f5dd80ce830 msgr2=0x7f5dd80adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 shutdown_connections 2023-08-25T09:47:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:50.825+0000 7f5ddf28a640 1 -- 172.21.15.70:0/2106572056 wait complete. 2023-08-25T09:47:50.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:47:51.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.047+0000 7f5897294640 1 Processor -- start 2023-08-25T09:47:51.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.047+0000 7f5897294640 1 -- start start 2023-08-25T09:47:51.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5897294640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f589015ae50 0x7f58900c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5897294640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58900cdc20 0x7f5890069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5897294640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58900cc5f0 con 0x7f58900ce060 2023-08-25T09:47:51.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5897294640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5890070650 con 0x7f589015ae50 2023-08-25T09:47:51.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5897294640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5890070930 con 0x7f58900cdc20 2023-08-25T09:47:51.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5895030640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f589015ae50 0x7f58900c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5895030640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f589015ae50 0x7f58900c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58890/0 (socket says 172.21.15.70:58890) 2023-08-25T09:47:51.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f5895030640 1 -- 172.21.15.70:0/909940892 learned_addr learned my addr 172.21.15.70:0/909940892 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:51.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.049+0000 7f589482f640 1 --2- 172.21.15.70:0/909940892 >> v2:172.21.15.155:3300/0 conn(0x7f58900cdc20 0x7f5890069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.050+0000 7f5895030640 1 -- 172.21.15.70:0/909940892 >> v2:172.21.15.155:3300/0 conn(0x7f58900cdc20 msgr2=0x7f5890069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:51.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.050+0000 7f5895030640 1 --2- 172.21.15.70:0/909940892 >> v2:172.21.15.155:3300/0 conn(0x7f58900cdc20 0x7f5890069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.050+0000 7f5895030640 1 -- 172.21.15.70:0/909940892 >> v1:172.21.15.70:6789/0 conn(0x7f58900ce060 legacy=0x7f58900ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.050+0000 7f5895030640 1 -- 172.21.15.70:0/909940892 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5890070c10 con 0x7f589015ae50 2023-08-25T09:47:51.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.050+0000 7f589482f640 1 --2- 172.21.15.70:0/909940892 >> v2:172.21.15.155:3300/0 conn(0x7f58900cdc20 0x7f5890069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:51.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.051+0000 7f5895030640 1 --2- 172.21.15.70:0/909940892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f589015ae50 0x7f58900c2110 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f58900cb030 tx=0x7f588c02ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3036aa9f691da55c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.051+0000 7f5887fff640 1 -- 172.21.15.70:0/909940892 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c02f860 con 0x7f589015ae50 2023-08-25T09:47:51.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.051+0000 7f5887fff640 1 -- 172.21.15.70:0/909940892 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588c02f9c0 con 0x7f589015ae50 2023-08-25T09:47:51.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.051+0000 7f5887fff640 1 -- 172.21.15.70:0/909940892 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588c02fd50 con 0x7f589015ae50 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.051+0000 7f5897294640 1 -- 172.21.15.70:0/909940892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f589015ae50 msgr2=0x7f58900c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.051+0000 7f5897294640 1 --2- 172.21.15.70:0/909940892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f589015ae50 0x7f58900c2110 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f58900cb030 tx=0x7f588c02ba40 comp rx=0 tx=0).stop 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.052+0000 7f5897294640 1 -- 172.21.15.70:0/909940892 shutdown_connections 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.052+0000 7f5897294640 1 --2- 172.21.15.70:0/909940892 >> v2:172.21.15.155:3300/0 conn(0x7f58900cdc20 0x7f5890069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.052+0000 7f5897294640 1 --2- 172.21.15.70:0/909940892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f589015ae50 0x7f58900c2110 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.052+0000 7f5897294640 1 -- 172.21.15.70:0/909940892 >> 172.21.15.70:0/909940892 conn(0x7f58900cb0d0 msgr2=0x7f58900cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.052+0000 7f5897294640 1 -- 172.21.15.70:0/909940892 shutdown_connections 2023-08-25T09:47:51.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.052+0000 7f5897294640 1 -- 172.21.15.70:0/909940892 wait complete. 2023-08-25T09:47:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.053+0000 7f5897294640 1 Processor -- start 2023-08-25T09:47:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 -- start start 2023-08-25T09:47:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58900cdc20 0x7f589015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 0x7f58900c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589015ae50 0x7f58900c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58901586d0 con 0x7f58900ce060 2023-08-25T09:47:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58900c5a50 con 0x7f58900cdc20 2023-08-25T09:47:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.054+0000 7f5897294640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58900c5d30 con 0x7f589015ae50 2023-08-25T09:47:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 0x7f58900c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 0x7f58900c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58892/0 (socket says 172.21.15.70:58892) 2023-08-25T09:47:51.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 -- 172.21.15.70:0/2935363819 learned_addr learned my addr 172.21.15.70:0/2935363819 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:51.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 -- 172.21.15.70:0/2935363819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589015ae50 msgr2=0x7f58900c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589015ae50 0x7f58900c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 -- 172.21.15.70:0/2935363819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58900cdc20 msgr2=0x7f589015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58900cdc20 0x7f589015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 -- 172.21.15.70:0/2935363819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c0094d0 con 0x7f58900ce060 2023-08-25T09:47:51.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f589482f640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 0x7f58900c2910 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f588000c7f0 tx=0x7f588000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.055+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880007a30 con 0x7f58900ce060 2023-08-25T09:47:51.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.056+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5880007b90 con 0x7f58900ce060 2023-08-25T09:47:51.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.056+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588000ce40 con 0x7f58900ce060 2023-08-25T09:47:51.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.056+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58900c6010 con 0x7f58900ce060 2023-08-25T09:47:51.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.056+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58900c65d0 con 0x7f58900ce060 2023-08-25T09:47:51.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.057+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58901f24a0 con 0x7f58900ce060 2023-08-25T09:47:51.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.057+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f58800113f0 con 0x7f58900ce060 2023-08-25T09:47:51.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.058+0000 7f5885ffb640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f587406a050 0x7f587406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.058+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f58800887a0 con 0x7f58900ce060 2023-08-25T09:47:51.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.059+0000 7f5895030640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f587406a050 0x7f587406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.059+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f58740721a0 con 0x7f58900ce060 2023-08-25T09:47:51.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.059+0000 7f5895030640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f587406a050 0x7f587406c510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f588c036000 tx=0x7f588c039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.062+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5880053df0 con 0x7f58900ce060 2023-08-25T09:47:51.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.235+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5890066fb0 con 0x7f58900ce060 2023-08-25T09:47:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.235+0000 7f5885ffb640 1 -- 172.21.15.70:0/2935363819 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v154) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5880057410 con 0x7f58900ce060 2023-08-25T09:47:51.237 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:47:51.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f587406a050 msgr2=0x7f587406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f587406a050 0x7f587406c510 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f588c036000 tx=0x7f588c039040 comp rx=0 tx=0).stop 2023-08-25T09:47:51.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 msgr2=0x7f58900c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 0x7f58900c2910 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f588000c7f0 tx=0x7f588000cbf0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 shutdown_connections 2023-08-25T09:47:51.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f587406a050 0x7f587406c510 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f589015ae50 0x7f58900c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58900ce060 0x7f58900c2910 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 --2- 172.21.15.70:0/2935363819 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58900cdc20 0x7f589015cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 >> 172.21.15.70:0/2935363819 conn(0x7f58900cb0d0 msgr2=0x7f589006c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:51.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 shutdown_connections 2023-08-25T09:47:51.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.238+0000 7f5897294640 1 -- 172.21.15.70:0/2935363819 wait complete. 2023-08-25T09:47:51.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:47:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.453+0000 7f09a4d53640 1 Processor -- start 2023-08-25T09:47:51.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.453+0000 7f09a4d53640 1 -- start start 2023-08-25T09:47:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f09a4d53640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f09a00cf9f0 0x7f09a00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f09a4d53640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a015d0b0 con 0x7f09a00d09f0 2023-08-25T09:47:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f09a4d53640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a00b58d0 con 0x7f09a015ccd0 2023-08-25T09:47:51.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f09a4d53640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a00d25d0 con 0x7f09a00cf9f0 2023-08-25T09:47:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f099effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f09a00d09f0 0x7f09a00cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34156/0 (socket says 172.21.15.70:34156) 2023-08-25T09:47:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f099effd640 1 -- 172.21.15.70:0/625713645 learned_addr learned my addr 172.21.15.70:0/625713645 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f099ffff640 1 --2- 172.21.15.70:0/625713645 >> v2:172.21.15.155:3300/0 conn(0x7f09a00cf9f0 0x7f09a00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276631704 0 0) 0x7f09a015d0b0 con 0x7f09a00d09f0 2023-08-25T09:47:51.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.455+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0988003520 con 0x7f09a00d09f0 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836913296 0 0) 0x7f0988003520 con 0x7f09a00d09f0 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 >> v2:172.21.15.155:3300/0 conn(0x7f09a00cf9f0 msgr2=0x7f09a00b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 --2- 172.21.15.70:0/625713645 >> v2:172.21.15.155:3300/0 conn(0x7f09a00cf9f0 0x7f09a00b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 >> v1:172.21.15.155:6789/0 conn(0x7f09a015ccd0 legacy=0x7f09a00d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a00d0e80 con 0x7f09a00d09f0 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0994001690 con 0x7f09a00d09f0 2023-08-25T09:47:51.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 >> v1:172.21.15.70:6789/0 conn(0x7f09a00d09f0 legacy=0x7f09a00cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0988003970 0x7f0988003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09880047c0 0x7f0988004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0988005180 0x7f098800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a00d25d0 con 0x7f09880047c0 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098800c020 con 0x7f0988005180 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098800c1e0 con 0x7f0988003970 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.456+0000 7f099effd640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09880047c0 0x7f0988004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099f7fe640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0988003970 0x7f0988003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099effd640 1 -- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0988003970 msgr2=0x7f0988003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099effd640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0988003970 0x7f0988003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099effd640 1 -- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0988005180 msgr2=0x7f098800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099effd640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0988005180 0x7f098800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099effd640 1 -- 172.21.15.70:0/625713645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0988003520 con 0x7f09880047c0 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099ffff640 1 -- 172.21.15.70:0/625713645 reap_dead start 2023-08-25T09:47:51.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.457+0000 7f099f7fe640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0988003970 0x7f0988003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.458+0000 7f099effd640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09880047c0 0x7f0988004c40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f09a00cced0 tx=0x7f09880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0049ad13c917bbf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.458+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994005ad0 con 0x7f09880047c0 2023-08-25T09:47:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.458+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994005c30 con 0x7f09880047c0 2023-08-25T09:47:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.458+0000 7f099e7fc640 1 -- 172.21.15.70:0/625713645 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099402c890 con 0x7f09880047c0 2023-08-25T09:47:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.458+0000 7f09a4d53640 1 -- 172.21.15.70:0/625713645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09880047c0 msgr2=0x7f0988004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.458+0000 7f09a4d53640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09880047c0 0x7f0988004c40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f09a00cced0 tx=0x7f09880057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.459+0000 7f09a4d53640 1 -- 172.21.15.70:0/625713645 shutdown_connections 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.459+0000 7f09a4d53640 1 --2- 172.21.15.70:0/625713645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09880047c0 0x7f0988004c40 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.459+0000 7f09a4d53640 1 -- 172.21.15.70:0/625713645 >> 172.21.15.70:0/625713645 conn(0x7f09a0069850 msgr2=0x7f09a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.459+0000 7f09a4d53640 1 -- 172.21.15.70:0/625713645 shutdown_connections 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.459+0000 7f09a4d53640 1 -- 172.21.15.70:0/625713645 wait complete. 2023-08-25T09:47:51.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.460+0000 7f09a4d53640 1 Processor -- start 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.460+0000 7f09a4d53640 1 -- start start 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.461+0000 7f09a4d53640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09880047c0 0x7f09a00c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.461+0000 7f09a4d53640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09a01e6410 0x7f09a01e6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.461+0000 7f09a4d53640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 0x7f09a015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.461+0000 7f09a4d53640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a01e6dd0 con 0x7f09a015e7f0 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.461+0000 7f09a4d53640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a015fb80 con 0x7f09a01e6410 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.461+0000 7f09a4d53640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a0152c30 con 0x7f09880047c0 2023-08-25T09:47:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 0x7f09a015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 0x7f09a015ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58906/0 (socket says 172.21.15.70:58906) 2023-08-25T09:47:51.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 -- 172.21.15.70:0/3433842349 learned_addr learned my addr 172.21.15.70:0/3433842349 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:51.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 -- 172.21.15.70:0/3433842349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09880047c0 msgr2=0x7f09a00c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09880047c0 0x7f09a00c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 -- 172.21.15.70:0/3433842349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09a01e6410 msgr2=0x7f09a01e6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09a01e6410 0x7f09a01e6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 -- 172.21.15.70:0/3433842349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a015f5c0 con 0x7f09a015e7f0 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f099ffff640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 0x7f09a015ec70 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f09a008bd60 tx=0x7f099800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.462+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998010040 con 0x7f09a015e7f0 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.463+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099800ba50 con 0x7f09a015e7f0 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.463+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800be00 con 0x7f09a015e7f0 2023-08-25T09:47:51.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.463+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a015f8a0 con 0x7f09a015e7f0 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.463+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a015f1b0 con 0x7f09a015e7f0 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.464+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a0152720 con 0x7f09a015e7f0 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.465+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f099801d400 con 0x7f09a015e7f0 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.465+0000 7f0987fff640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f098c06a080 0x7f098c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.466+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0998088d80 con 0x7f09a015e7f0 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.466+0000 7f099f7fe640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f098c06a080 0x7f098c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.466+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f098c0721d0 con 0x7f09a015e7f0 2023-08-25T09:47:51.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.466+0000 7f099f7fe640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f098c06a080 0x7f098c06c540 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0990007fd0 tx=0x7f0990009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.470+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09a0152720 con 0x7f09a015e7f0 2023-08-25T09:47:51.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.661+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09a0152720 con 0x7f09a015e7f0 2023-08-25T09:47:51.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.662+0000 7f0987fff640 1 -- 172.21.15.70:0/3433842349 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v154) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09a0152720 con 0x7f09a015e7f0 2023-08-25T09:47:51.663 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:47:51.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f098c06a080 msgr2=0x7f098c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f098c06a080 0x7f098c06c540 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0990007fd0 tx=0x7f0990009040 comp rx=0 tx=0).stop 2023-08-25T09:47:51.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 msgr2=0x7f09a015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 0x7f09a015ec70 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f09a008bd60 tx=0x7f099800eea0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 shutdown_connections 2023-08-25T09:47:51.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f098c06a080 0x7f098c06c540 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09a015e7f0 0x7f09a015ec70 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09a01e6410 0x7f09a01e6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 --2- 172.21.15.70:0/3433842349 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f09880047c0 0x7f09a00c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.664+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 >> 172.21.15.70:0/3433842349 conn(0x7f09a0069850 msgr2=0x7f09a00cbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:51.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.665+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 shutdown_connections 2023-08-25T09:47:51.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.665+0000 7f09a4d53640 1 -- 172.21.15.70:0/3433842349 wait complete. 2023-08-25T09:47:51.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:51.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.899+0000 7faf7c3fd640 1 Processor -- start 2023-08-25T09:47:51.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.899+0000 7faf7c3fd640 1 -- start start 2023-08-25T09:47:51.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf7c3fd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf7415ec10 0x7faf7415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf7c3fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415c6d0 0x7faf740b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf7c3fd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf740b1b60 con 0x7faf7415e830 2023-08-25T09:47:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf7c3fd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7406bd80 con 0x7faf7415ec10 2023-08-25T09:47:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf7c3fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7406b7c0 con 0x7faf7415c6d0 2023-08-25T09:47:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf79998640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf7415ec10 0x7faf7415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf79998640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7faf7415ec10 0x7faf7415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58922/0 (socket says 172.21.15.70:58922) 2023-08-25T09:47:51.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf79998640 1 -- 172.21.15.70:0/1045308852 learned_addr learned my addr 172.21.15.70:0/1045308852 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:51.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf7a99a640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415c6d0 0x7faf740b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf79998640 1 -- 172.21.15.70:0/1045308852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415c6d0 msgr2=0x7faf740b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.901+0000 7faf79998640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415c6d0 0x7faf740b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.902+0000 7faf79998640 1 -- 172.21.15.70:0/1045308852 >> v1:172.21.15.70:6789/0 conn(0x7faf7415e830 legacy=0x7faf7406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.902+0000 7faf79998640 1 -- 172.21.15.70:0/1045308852 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7406baa0 con 0x7faf7415ec10 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.902+0000 7faf79197640 1 -- 172.21.15.70:0/1045308852 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528512293 0 0) 0x7faf740b1b60 con 0x7faf7415e830 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.902+0000 7faf7a99a640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415c6d0 0x7faf740b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:51.905 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.903+0000 7faf79998640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415ec10 0x7faf7415c190 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7faf64009870 tx=0x7faf6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a9bc8f8691c235 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.903+0000 7faf79197640 1 -- 172.21.15.70:0/1045308852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64030840 con 0x7faf7415ec10 2023-08-25T09:47:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.903+0000 7faf79197640 1 -- 172.21.15.70:0/1045308852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf640309a0 con 0x7faf7415ec10 2023-08-25T09:47:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.903+0000 7faf79197640 1 -- 172.21.15.70:0/1045308852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64030d30 con 0x7faf7415ec10 2023-08-25T09:47:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.903+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1045308852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415ec10 msgr2=0x7faf7415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.906 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.903+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415ec10 0x7faf7415c190 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7faf64009870 tx=0x7faf6402b980 comp rx=0 tx=0).stop 2023-08-25T09:47:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.904+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1045308852 shutdown_connections 2023-08-25T09:47:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.904+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415ec10 0x7faf7415c190 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.904+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1045308852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415c6d0 0x7faf740b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.904+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1045308852 >> 172.21.15.70:0/1045308852 conn(0x7faf74069850 msgr2=0x7faf74069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.904+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1045308852 shutdown_connections 2023-08-25T09:47:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.904+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1045308852 wait complete. 2023-08-25T09:47:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.905+0000 7faf7c3fd640 1 Processor -- start 2023-08-25T09:47:51.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.906+0000 7faf7c3fd640 1 -- start start 2023-08-25T09:47:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.906+0000 7faf7c3fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 0x7faf74154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7c3fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415e830 0x7faf74156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7c3fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf7415ec10 0x7faf740ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7c3fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf74156dd0 con 0x7faf7415c6d0 2023-08-25T09:47:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7c3fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf740c7a70 con 0x7faf7415ec10 2023-08-25T09:47:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7c3fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf74155230 con 0x7faf7415e830 2023-08-25T09:47:51.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 0x7faf74154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 0x7faf74154b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58936/0 (socket says 172.21.15.70:58936) 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 -- 172.21.15.70:0/1378607928 learned_addr learned my addr 172.21.15.70:0/1378607928 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 -- 172.21.15.70:0/1378607928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415e830 msgr2=0x7faf74156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415e830 0x7faf74156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 -- 172.21.15.70:0/1378607928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf7415ec10 msgr2=0x7faf740ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf7415ec10 0x7faf740ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:51.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.907+0000 7faf7a199640 1 -- 172.21.15.70:0/1378607928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640094d0 con 0x7faf7415c6d0 2023-08-25T09:47:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.908+0000 7faf7a199640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 0x7faf74154b20 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c4b0 tx=0x7faf7000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.908+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70005480 con 0x7faf7415c6d0 2023-08-25T09:47:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.908+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf700035f0 con 0x7faf7415c6d0 2023-08-25T09:47:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.908+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700039e0 con 0x7faf7415c6d0 2023-08-25T09:47:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.908+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf74155510 con 0x7faf7415c6d0 2023-08-25T09:47:51.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.908+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf74155ad0 con 0x7faf7415c6d0 2023-08-25T09:47:51.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.909+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf740c7c30 con 0x7faf7415c6d0 2023-08-25T09:47:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.910+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7faf70003b40 con 0x7faf7415c6d0 2023-08-25T09:47:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.911+0000 7faf6affd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf5806a050 0x7faf5806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.912+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf70087bd0 con 0x7faf7415c6d0 2023-08-25T09:47:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.912+0000 7faf79998640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf5806a050 0x7faf5806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:51.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.912+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7faf580721a0 con 0x7faf7415c6d0 2023-08-25T09:47:51.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.912+0000 7faf79998640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf5806a050 0x7faf5806c510 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faf64036000 tx=0x7faf64030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:51.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:51.916+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7faf70017020 con 0x7faf7415c6d0 2023-08-25T09:47:52.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.144+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf7415f280 con 0x7faf7415c6d0 2023-08-25T09:47:52.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.145+0000 7faf6affd640 1 -- 172.21.15.70:0/1378607928 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7faf700532b0 con 0x7faf7415c6d0 2023-08-25T09:47:52.147 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:52.148 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:52.148 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf5806a050 msgr2=0x7faf5806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf5806a050 0x7faf5806c510 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faf64036000 tx=0x7faf64030000 comp rx=0 tx=0).stop 2023-08-25T09:47:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 msgr2=0x7faf74154b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 0x7faf74154b20 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c4b0 tx=0x7faf7000c970 comp rx=0 tx=0).stop 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 shutdown_connections 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faf5806a050 0x7faf5806c510 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7faf7415ec10 0x7faf740ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7faf7415e830 0x7faf74156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 --2- 172.21.15.70:0/1378607928 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7faf7415c6d0 0x7faf74154b20 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.148+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 >> 172.21.15.70:0/1378607928 conn(0x7faf74069850 msgr2=0x7faf740ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:52.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.149+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 shutdown_connections 2023-08-25T09:47:52.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.149+0000 7faf7c3fd640 1 -- 172.21.15.70:0/1378607928 wait complete. 2023-08-25T09:47:52.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.372+0000 7fc85ca3f640 1 Processor -- start 2023-08-25T09:47:52.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.372+0000 7fc85ca3f640 1 -- start start 2023-08-25T09:47:52.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc85ca3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc85806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.377 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc85ca3f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc85806b2b0 0x7fc85815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc85ca3f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580b1b60 con 0x7fc85815c6d0 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc85ca3f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85806bd80 con 0x7fc85806b2b0 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc85ca3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85806b7c0 con 0x7fc85815e830 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc85659c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc85806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.374+0000 7fc855d9b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc85806b2b0 0x7fc85815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc85659c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc85806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:57710/0 (socket says 172.21.15.70:57710) 2023-08-25T09:47:52.378 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc85659c640 1 -- 172.21.15.70:0/2076454882 learned_addr learned my addr 172.21.15.70:0/2076454882 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc85659c640 1 -- 172.21.15.70:0/2076454882 >> v2:172.21.15.155:3300/0 conn(0x7fc85806b2b0 msgr2=0x7fc85815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc85659c640 1 --2- 172.21.15.70:0/2076454882 >> v2:172.21.15.155:3300/0 conn(0x7fc85806b2b0 0x7fc85815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc85659c640 1 -- 172.21.15.70:0/2076454882 >> v1:172.21.15.155:6789/0 conn(0x7fc85815c6d0 legacy=0x7fc8580b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc85659c640 1 -- 172.21.15.70:0/2076454882 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85806baa0 con 0x7fc85815e830 2023-08-25T09:47:52.379 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.375+0000 7fc855d9b640 1 --2- 172.21.15.70:0/2076454882 >> v2:172.21.15.155:3300/0 conn(0x7fc85806b2b0 0x7fc85815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.377+0000 7fc85659c640 1 --2- 172.21.15.70:0/2076454882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc85806ad70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc848009870 tx=0x7fc84802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b3f411d3eb2e05 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:52.380 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.377+0000 7fc85559a640 1 -- 172.21.15.70:0/2076454882 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848030840 con 0x7fc85815e830 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.377+0000 7fc85559a640 1 -- 172.21.15.70:0/2076454882 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8480309a0 con 0x7fc85815e830 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.377+0000 7fc85559a640 1 -- 172.21.15.70:0/2076454882 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848030d30 con 0x7fc85815e830 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 -- 172.21.15.70:0/2076454882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 msgr2=0x7fc85806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/2076454882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc85806ad70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc848009870 tx=0x7fc84802b980 comp rx=0 tx=0).stop 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 -- 172.21.15.70:0/2076454882 shutdown_connections 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/2076454882 >> v2:172.21.15.155:3300/0 conn(0x7fc85806b2b0 0x7fc85815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/2076454882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc85806ad70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 -- 172.21.15.70:0/2076454882 >> 172.21.15.70:0/2076454882 conn(0x7fc858069850 msgr2=0x7fc858069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 -- 172.21.15.70:0/2076454882 shutdown_connections 2023-08-25T09:47:52.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.378+0000 7fc85ca3f640 1 -- 172.21.15.70:0/2076454882 wait complete. 2023-08-25T09:47:52.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.380+0000 7fc85ca3f640 1 Processor -- start 2023-08-25T09:47:52.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.380+0000 7fc85ca3f640 1 -- start start 2023-08-25T09:47:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85ca3f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 0x7fc858147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85ca3f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc85815c6d0 0x7fc858144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85ca3f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc8581488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85ca3f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85814ad20 con 0x7fc85806b2b0 2023-08-25T09:47:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85ca3f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581483a0 con 0x7fc85815c6d0 2023-08-25T09:47:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85ca3f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc858145740 con 0x7fc85815e830 2023-08-25T09:47:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.381+0000 7fc85659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 0x7fc858147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 0x7fc858147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58938/0 (socket says 172.21.15.70:58938) 2023-08-25T09:47:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 -- 172.21.15.70:0/1772535488 learned_addr learned my addr 172.21.15.70:0/1772535488 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 -- 172.21.15.70:0/1772535488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 msgr2=0x7fc8581488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc8581488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 -- 172.21.15.70:0/1772535488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc85815c6d0 msgr2=0x7fc858144a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc85815c6d0 0x7fc858144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 -- 172.21.15.70:0/1772535488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c009400 con 0x7fc85806b2b0 2023-08-25T09:47:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.382+0000 7fc85659c640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 0x7fc858147e60 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc84802b9c0 tx=0x7fc8480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.383+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84803e070 con 0x7fc85806b2b0 2023-08-25T09:47:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.383+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc848039a10 con 0x7fc85806b2b0 2023-08-25T09:47:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.383+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848039de0 con 0x7fc85806b2b0 2023-08-25T09:47:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.383+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8480094d0 con 0x7fc85806b2b0 2023-08-25T09:47:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.383+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858145d00 con 0x7fc85806b2b0 2023-08-25T09:47:52.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.384+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85804e420 con 0x7fc85806b2b0 2023-08-25T09:47:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.385+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc84804b400 con 0x7fc85806b2b0 2023-08-25T09:47:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.386+0000 7fc846ffd640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc83806a050 0x7fc83806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.386+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc8480aac70 con 0x7fc85806b2b0 2023-08-25T09:47:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.386+0000 7fc855d9b640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc83806a050 0x7fc83806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.386+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fc8380721a0 con 0x7fc85806b2b0 2023-08-25T09:47:52.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.387+0000 7fc855d9b640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc83806a050 0x7fc83806c510 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc84c006e80 tx=0x7fc84c006dd0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:52.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.390+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc848076510 con 0x7fc85806b2b0 2023-08-25T09:47:52.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.600+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc85815a2d0 con 0x7fc85806b2b0 2023-08-25T09:47:52.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.603+0000 7fc846ffd640 1 -- 172.21.15.70:0/1772535488 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18407 (secure 0 0 0) 0x7fc84807c090 con 0x7fc85806b2b0 2023-08-25T09:47:52.605 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:52.605 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":154,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:47:47.394216+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:47:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.605+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc83806a050 msgr2=0x7fc83806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.605+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc83806a050 0x7fc83806c510 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc84c006e80 tx=0x7fc84c006dd0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.605+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 msgr2=0x7fc858147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.605+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 0x7fc858147e60 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fc84802b9c0 tx=0x7fc8480046c0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 shutdown_connections 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc83806a050 0x7fc83806c510 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc85815e830 0x7fc8581488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc85815c6d0 0x7fc858144a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 --2- 172.21.15.70:0/1772535488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc85806b2b0 0x7fc858147e60 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 >> 172.21.15.70:0/1772535488 conn(0x7fc858069850 msgr2=0x7fc8580b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:52.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 shutdown_connections 2023-08-25T09:47:52.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.606+0000 7fc85ca3f640 1 -- 172.21.15.70:0/1772535488 wait complete. 2023-08-25T09:47:52.625 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:47:52.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.818+0000 7f38b4ffd640 1 Processor -- start 2023-08-25T09:47:52.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.818+0000 7f38b4ffd640 1 -- start start 2023-08-25T09:47:52.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38b4ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38b00ce8e0 0x7f38b00cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38b4ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f38b00cdc80 0x7f38b00cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38b4ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 0x7f38b006d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38b4ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b00cba70 con 0x7f38b00ce8e0 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38b4ffd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b00cbd50 con 0x7f38b00cdc80 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38b4ffd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b00cc030 con 0x7f38b00cf200 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38ae59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38b00ce8e0 0x7f38b00cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38ae59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38b00ce8e0 0x7f38b00cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58940/0 (socket says 172.21.15.70:58940) 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.820+0000 7f38ae59c640 1 -- 172.21.15.70:0/197165406 learned_addr learned my addr 172.21.15.70:0/197165406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.821+0000 7f38ae59c640 1 -- 172.21.15.70:0/197165406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 msgr2=0x7f38b006d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:52.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.821+0000 7f38ae59c640 1 --2- 172.21.15.70:0/197165406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 0x7f38b006d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.821+0000 7f38ae59c640 1 -- 172.21.15.70:0/197165406 >> v2:172.21.15.155:3300/0 conn(0x7f38b00cdc80 msgr2=0x7f38b00cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:52.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.821+0000 7f38ae59c640 1 --2- 172.21.15.70:0/197165406 >> v2:172.21.15.155:3300/0 conn(0x7f38b00cdc80 0x7f38b00cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.821+0000 7f38ae59c640 1 -- 172.21.15.70:0/197165406 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00cc310 con 0x7f38b00ce8e0 2023-08-25T09:47:52.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.822+0000 7f38ae59c640 1 --2- 172.21.15.70:0/197165406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b00cb470 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f38b00c40a0 tx=0x7f38a002b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37b21a323d18b88 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:52.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.822+0000 7f38ad59a640 1 -- 172.21.15.70:0/197165406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0004030 con 0x7f38b00ce8e0 2023-08-25T09:47:52.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.822+0000 7f38ad59a640 1 -- 172.21.15.70:0/197165406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0004190 con 0x7f38b00ce8e0 2023-08-25T09:47:52.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.822+0000 7f38ad59a640 1 -- 172.21.15.70:0/197165406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0031640 con 0x7f38b00ce8e0 2023-08-25T09:47:52.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 -- 172.21.15.70:0/197165406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 msgr2=0x7f38b00cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/197165406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b00cb470 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f38b00c40a0 tx=0x7f38a002b8e0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 -- 172.21.15.70:0/197165406 shutdown_connections 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/197165406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 0x7f38b006d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/197165406 >> v2:172.21.15.155:3300/0 conn(0x7f38b00cdc80 0x7f38b00cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/197165406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b00cb470 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 -- 172.21.15.70:0/197165406 >> 172.21.15.70:0/197165406 conn(0x7f38b00d0860 msgr2=0x7f38b00d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 -- 172.21.15.70:0/197165406 shutdown_connections 2023-08-25T09:47:52.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.823+0000 7f38b4ffd640 1 -- 172.21.15.70:0/197165406 wait complete. 2023-08-25T09:47:52.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.825+0000 7f38b4ffd640 1 Processor -- start 2023-08-25T09:47:52.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.825+0000 7f38b4ffd640 1 -- start start 2023-08-25T09:47:52.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38b4ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38b00cdc80 0x7f38b00ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38b4ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b01dd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38b4ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 0x7f38b00c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38b4ffd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01de530 con 0x7f38b00ce8e0 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38b4ffd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01dd650 con 0x7f38b00cdc80 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38b4ffd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01dd8b0 con 0x7f38b00cf200 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b01dd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b01dd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58946/0 (socket says 172.21.15.70:58946) 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 -- 172.21.15.70:0/1176541926 learned_addr learned my addr 172.21.15.70:0/1176541926 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:52.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38ae59c640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38b00cdc80 0x7f38b00ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 -- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 msgr2=0x7f38b00c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 0x7f38b00c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 -- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38b00cdc80 msgr2=0x7f38b00ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38b00cdc80 0x7f38b00ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38add9b640 1 -- 172.21.15.70:0/1176541926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009460 con 0x7f38b00ce8e0 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.826+0000 7f38ae59c640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38b00cdc80 0x7f38b00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:52.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.827+0000 7f38add9b640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b01dd110 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b410 tx=0x7f38a400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:52.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.827+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4011830 con 0x7f38b00ce8e0 2023-08-25T09:47:52.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.827+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a40119b0 con 0x7f38b00ce8e0 2023-08-25T09:47:52.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.827+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4011d60 con 0x7f38b00ce8e0 2023-08-25T09:47:52.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.827+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b01ddb90 con 0x7f38b00ce8e0 2023-08-25T09:47:52.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.827+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b01de150 con 0x7f38b00ce8e0 2023-08-25T09:47:52.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.828+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b01de670 con 0x7f38b00ce8e0 2023-08-25T09:47:52.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.829+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f38a401a400 con 0x7f38b00ce8e0 2023-08-25T09:47:52.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.830+0000 7f389effd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f389006a050 0x7f389006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:52.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.830+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38a40886f0 con 0x7f38b00ce8e0 2023-08-25T09:47:52.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.830+0000 7f38ae59c640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f389006a050 0x7f389006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:52.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.831+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f38900721a0 con 0x7f38b00ce8e0 2023-08-25T09:47:52.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.831+0000 7f38ae59c640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f389006a050 0x7f389006c510 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f38a0035000 tx=0x7f38a0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:52.836 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:52.835+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f38b01de670 con 0x7f38b00ce8e0 2023-08-25T09:47:53.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.058+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38b01de670 con 0x7f38b00ce8e0 2023-08-25T09:47:53.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.059+0000 7f389effd640 1 -- 172.21.15.70:0/1176541926 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f38b01de670 con 0x7f38b00ce8e0 2023-08-25T09:47:53.060 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:53.061 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f389006a050 msgr2=0x7f389006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f389006a050 0x7f389006c510 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f38a0035000 tx=0x7f38a0008030 comp rx=0 tx=0).stop 2023-08-25T09:47:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 msgr2=0x7f38b01dd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b01dd110 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b410 tx=0x7f38a400b8d0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 shutdown_connections 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f389006a050 0x7f389006c510 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38b00cf200 0x7f38b00c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38b00ce8e0 0x7f38b01dd110 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 --2- 172.21.15.70:0/1176541926 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38b00cdc80 0x7f38b00ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 >> 172.21.15.70:0/1176541926 conn(0x7f38b00d0860 msgr2=0x7f38b01e1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:53.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 shutdown_connections 2023-08-25T09:47:53.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.062+0000 7f38b4ffd640 1 -- 172.21.15.70:0/1176541926 wait complete. 2023-08-25T09:47:53.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:47:53.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.293+0000 7f06c48a7640 1 Processor -- start 2023-08-25T09:47:53.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.293+0000 7f06c48a7640 1 -- start start 2023-08-25T09:47:53.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06c48a7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 0x7f06c00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06c48a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06c48a7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c015d3f0 con 0x7f06c00ce5b0 2023-08-25T09:47:53.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06c48a7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00b2b10 con 0x7f06c015aa10 2023-08-25T09:47:53.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06c48a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0159d10 con 0x7f06c00b2680 2023-08-25T09:47:53.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06b7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 0x7f06c00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.295+0000 7f06bed9d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06b7fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 0x7f06c00b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:56754/0 (socket says 172.21.15.70:56754) 2023-08-25T09:47:53.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06b7fff640 1 -- 172.21.15.70:0/556913704 learned_addr learned my addr 172.21.15.70:0/556913704 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:53.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06bed9d640 1 -- 172.21.15.70:0/556913704 >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 msgr2=0x7f06c00b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:53.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06bed9d640 1 --2- 172.21.15.70:0/556913704 >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 0x7f06c00b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06bed9d640 1 -- 172.21.15.70:0/556913704 >> v1:172.21.15.155:6789/0 conn(0x7f06c00ce5b0 legacy=0x7f06c00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06bed9d640 1 -- 172.21.15.70:0/556913704 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00b2d20 con 0x7f06c00b2680 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.296+0000 7f06b7fff640 1 --2- 172.21.15.70:0/556913704 >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 0x7f06c00b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.298+0000 7f06bed9d640 1 --2- 172.21.15.70:0/556913704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c006f9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009dc0 tx=0x7f06b802bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54734b776927e64 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.298+0000 7f06bdd9b640 1 -- 172.21.15.70:0/556913704 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b802cb20 con 0x7f06c00b2680 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.298+0000 7f06bdd9b640 1 -- 172.21.15.70:0/556913704 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b802cc80 con 0x7f06c00b2680 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.298+0000 7f06bdd9b640 1 -- 172.21.15.70:0/556913704 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8034440 con 0x7f06c00b2680 2023-08-25T09:47:53.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 -- 172.21.15.70:0/556913704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 msgr2=0x7f06c006f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 --2- 172.21.15.70:0/556913704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c006f9f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009dc0 tx=0x7f06b802bd70 comp rx=0 tx=0).stop 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 -- 172.21.15.70:0/556913704 shutdown_connections 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 --2- 172.21.15.70:0/556913704 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c006f9f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 --2- 172.21.15.70:0/556913704 >> v2:172.21.15.155:3300/0 conn(0x7f06c015aa10 0x7f06c00b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 -- 172.21.15.70:0/556913704 >> 172.21.15.70:0/556913704 conn(0x7f06c0069850 msgr2=0x7f06c0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 -- 172.21.15.70:0/556913704 shutdown_connections 2023-08-25T09:47:53.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.299+0000 7f06c48a7640 1 -- 172.21.15.70:0/556913704 wait complete. 2023-08-25T09:47:53.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.301+0000 7f06c48a7640 1 Processor -- start 2023-08-25T09:47:53.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.302+0000 7f06c48a7640 1 -- start start 2023-08-25T09:47:53.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.302+0000 7f06c48a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c01451f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06c48a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 0x7f06c00ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06c48a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06c015aa10 0x7f06c0153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06c48a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00af3d0 con 0x7f06c00ce5b0 2023-08-25T09:47:53.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06c48a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c01e30b0 con 0x7f06c015aa10 2023-08-25T09:47:53.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06c48a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00cd630 con 0x7f06c00b2680 2023-08-25T09:47:53.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 0x7f06c00ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 0x7f06c00ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58960/0 (socket says 172.21.15.70:58960) 2023-08-25T09:47:53.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 -- 172.21.15.70:0/4129353745 learned_addr learned my addr 172.21.15.70:0/4129353745 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:53.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06be59c640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c01451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06bed9d640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06c015aa10 0x7f06c0153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 -- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 msgr2=0x7f06c01451f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c01451f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 -- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06c015aa10 msgr2=0x7f06c0153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06c015aa10 0x7f06c0153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06b7fff640 1 -- 172.21.15.70:0/4129353745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80094d0 con 0x7f06c00ce5b0 2023-08-25T09:47:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.303+0000 7f06bed9d640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06c015aa10 0x7f06c0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:53.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.304+0000 7f06b7fff640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 0x7f06c00ccc50 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e9d0 tx=0x7f06a800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:53.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.304+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800ca60 con 0x7f06c00ce5b0 2023-08-25T09:47:53.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.304+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800cbc0 con 0x7f06c00ce5b0 2023-08-25T09:47:53.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.304+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8013470 con 0x7f06c00ce5b0 2023-08-25T09:47:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.304+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b8009a70 con 0x7f06c00ce5b0 2023-08-25T09:47:53.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.304+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c00af900 con 0x7f06c00ce5b0 2023-08-25T09:47:53.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.305+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c004e420 con 0x7f06c00ce5b0 2023-08-25T09:47:53.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.306+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f06a80135d0 con 0x7f06c00ce5b0 2023-08-25T09:47:53.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.308+0000 7f06b6ffd640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f06a406a080 0x7f06a406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.308+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f06a8088970 con 0x7f06c00ce5b0 2023-08-25T09:47:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.308+0000 7f06be59c640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f06a406a080 0x7f06a406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.308+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f06a4072250 con 0x7f06c00ce5b0 2023-08-25T09:47:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.309+0000 7f06be59c640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f06a406a080 0x7f06a406c540 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f06b0007fd0 tx=0x7f06b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:53.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.312+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f06a8053fc0 con 0x7f06c00ce5b0 2023-08-25T09:47:53.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.543+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06c00cae50 con 0x7f06c00ce5b0 2023-08-25T09:47:53.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.545+0000 7f06b6ffd640 1 -- 172.21.15.70:0/4129353745 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4883 (secure 0 0 0) 0x7f06a8021020 con 0x7f06c00ce5b0 2023-08-25T09:47:53.546 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:53.547 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":65,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:46:58.267207+0000","modified":"2023-08-25T09:47:01.840742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8232":{"gid":8232,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/1074652443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1074652443},{"type":"v1","addr":"172.21.15.155:6834","nonce":1074652443}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-08-25T09:47:53.547 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 65 2023-08-25T09:47:53.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f06a406a080 msgr2=0x7f06a406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f06a406a080 0x7f06a406c540 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f06b0007fd0 tx=0x7f06b0009040 comp rx=0 tx=0).stop 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 msgr2=0x7f06c00ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 0x7f06c00ccc50 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e9d0 tx=0x7f06a800ee90 comp rx=0 tx=0).stop 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 shutdown_connections 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f06a406a080 0x7f06a406c540 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f06c015aa10 0x7f06c0153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f06c00ce5b0 0x7f06c00ccc50 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 --2- 172.21.15.70:0/4129353745 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f06c00b2680 0x7f06c01451f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.547+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 >> 172.21.15.70:0/4129353745 conn(0x7f06c0069850 msgr2=0x7f06c0069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:53.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.548+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 shutdown_connections 2023-08-25T09:47:53.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.548+0000 7f06c48a7640 1 -- 172.21.15.70:0/4129353745 wait complete. 2023-08-25T09:47:53.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:53.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.779+0000 7f6526afa640 1 Processor -- start 2023-08-25T09:47:53.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.779+0000 7f6526afa640 1 -- start start 2023-08-25T09:47:53.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f6526afa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65200cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f6526afa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652015cfb0 con 0x7f65201606c0 2023-08-25T09:47:53.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f6526afa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200cd4e0 con 0x7f65200d2a00 2023-08-25T09:47:53.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f6526afa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6520160f10 con 0x7f652015cbd0 2023-08-25T09:47:53.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f651ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f65201606c0 0x7f65200cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34170/0 (socket says 172.21.15.70:34170) 2023-08-25T09:47:53.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f651ffff640 1 -- 172.21.15.70:0/2004355125 learned_addr learned my addr 172.21.15.70:0/2004355125 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:53.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f6524896640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65200cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744361463 0 0) 0x7f652015cfb0 con 0x7f65201606c0 2023-08-25T09:47:53.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.781+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6510003520 con 0x7f65201606c0 2023-08-25T09:47:53.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172585198 0 0) 0x7f6510003520 con 0x7f65201606c0 2023-08-25T09:47:53.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 msgr2=0x7f65200cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:53.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65200cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 >> v1:172.21.15.155:6789/0 conn(0x7f65200d2a00 legacy=0x7f65200b5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65201611f0 con 0x7f65201606c0 2023-08-25T09:47:53.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f650c001670 con 0x7f65201606c0 2023-08-25T09:47:53.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 >> v1:172.21.15.70:6789/0 conn(0x7f65201606c0 legacy=0x7f65200cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65100039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6510004d00 0x7f6510005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6520160f10 con 0x7f6510003fd0 2023-08-25T09:47:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651000bb40 con 0x7f6510004d00 2023-08-25T09:47:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651000bd00 con 0x7f652015cbd0 2023-08-25T09:47:53.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.782+0000 7f651ffff640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f651ffff640 1 -- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 msgr2=0x7f65100039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:53.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f6524896640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65100039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f6525097640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6510004d00 0x7f6510005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f651ffff640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65100039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f651ffff640 1 -- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6510004d00 msgr2=0x7f6510005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:53.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f651ffff640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6510004d00 0x7f6510005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f651ffff640 1 -- 172.21.15.70:0/2004355125 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510003520 con 0x7f6510003fd0 2023-08-25T09:47:53.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.783+0000 7f6524896640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015cbd0 0x7f65100039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:53.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f651ffff640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65100047c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c240 tx=0x7f65100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41a554d89471d6d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:53.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c005ad0 con 0x7f6510003fd0 2023-08-25T09:47:53.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c005c30 con 0x7f6510003fd0 2023-08-25T09:47:53.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f651f7fe640 1 -- 172.21.15.70:0/2004355125 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c02caf0 con 0x7f6510003fd0 2023-08-25T09:47:53.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f6526afa640 1 -- 172.21.15.70:0/2004355125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 msgr2=0x7f65100047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f6526afa640 1 --2- 172.21.15.70:0/2004355125 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65100047c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c240 tx=0x7f65100057f0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f6525097640 1 -- 172.21.15.70:0/2004355125 reap_dead start 2023-08-25T09:47:53.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f6526afa640 1 -- 172.21.15.70:0/2004355125 shutdown_connections 2023-08-25T09:47:53.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.784+0000 7f6526afa640 1 -- 172.21.15.70:0/2004355125 >> 172.21.15.70:0/2004355125 conn(0x7f6520070260 msgr2=0x7f6520070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:53.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.785+0000 7f6526afa640 1 -- 172.21.15.70:0/2004355125 shutdown_connections 2023-08-25T09:47:53.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.785+0000 7f6526afa640 1 -- 172.21.15.70:0/2004355125 wait complete. 2023-08-25T09:47:53.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.786+0000 7f6526afa640 1 Processor -- start 2023-08-25T09:47:53.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.786+0000 7f6526afa640 1 -- start start 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6526afa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65200b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6526afa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65200cbeb0 0x7f65200cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6526afa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015f2f0 0x7f652015f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6526afa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f5a90 con 0x7f6510003fd0 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6526afa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f5ce0 con 0x7f65200cbeb0 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6526afa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65201f5fc0 con 0x7f652015f2f0 2023-08-25T09:47:53.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6524896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65200b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6524896640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65200b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58984/0 (socket says 172.21.15.70:58984) 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.787+0000 7f6524896640 1 -- 172.21.15.70:0/945787421 learned_addr learned my addr 172.21.15.70:0/945787421 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6524896640 1 -- 172.21.15.70:0/945787421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015f2f0 msgr2=0x7f652015f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6524896640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015f2f0 0x7f652015f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6524896640 1 -- 172.21.15.70:0/945787421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65200cbeb0 msgr2=0x7f65200cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6524896640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65200cbeb0 0x7f65200cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6524896640 1 -- 172.21.15.70:0/945787421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65201f62a0 con 0x7f6510003fd0 2023-08-25T09:47:53.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6524896640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65200b75d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f651400e7e0 tx=0x7f651400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400c6d0 con 0x7f6510003fd0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400c830 con 0x7f6510003fd0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400cba0 con 0x7f6510003fd0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.788+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65201f6580 con 0x7f6510003fd0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.789+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652015fcb0 con 0x7f6510003fd0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.790+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652004e420 con 0x7f6510003fd0 2023-08-25T09:47:53.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.791+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f651400cd00 con 0x7f6510003fd0 2023-08-25T09:47:53.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.791+0000 7f651d7fa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f650406a050 0x7f650406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:53.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.792+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6514088380 con 0x7f6510003fd0 2023-08-25T09:47:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.792+0000 7f651ffff640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f650406a050 0x7f650406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.792+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f65040721a0 con 0x7f6510003fd0 2023-08-25T09:47:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.792+0000 7f651ffff640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f650406a050 0x7f650406c510 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f650c034000 tx=0x7f650c007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:53.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.795+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6514053a60 con 0x7f6510003fd0 2023-08-25T09:47:53.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.971+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65201614a0 con 0x7f6510003fd0 2023-08-25T09:47:53.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.973+0000 7f651d7fa640 1 -- 172.21.15.70:0/945787421 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) v1 ==== 74+0+18407 (secure 0 0 0) 0x7f6514057080 con 0x7f6510003fd0 2023-08-25T09:47:53.976 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:47:53.976 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":154,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:47:47.394216+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:46:56.533109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-08-25T09:46:57.303907+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:47:53.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f650406a050 msgr2=0x7f650406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f650406a050 0x7f650406c510 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f650c034000 tx=0x7f650c007000 comp rx=0 tx=0).stop 2023-08-25T09:47:53.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 msgr2=0x7f65200b75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:53.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65200b75d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f651400e7e0 tx=0x7f651400eca0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 shutdown_connections 2023-08-25T09:47:53.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f650406a050 0x7f650406c510 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f652015f2f0 0x7f652015f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f65200cbeb0 0x7f65200cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 --2- 172.21.15.70:0/945787421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6510003fd0 0x7f65200b75d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:53.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 >> 172.21.15.70:0/945787421 conn(0x7f6520070260 msgr2=0x7f65200b68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:53.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 shutdown_connections 2023-08-25T09:47:53.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:53.976+0000 7f6526afa640 1 -- 172.21.15.70:0/945787421 wait complete. 2023-08-25T09:47:53.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:47:54.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.197+0000 7f78e5657640 1 Processor -- start 2023-08-25T09:47:54.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.197+0000 7f78e5657640 1 -- start start 2023-08-25T09:47:54.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78e5657640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78e5657640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 0x7f78e00cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78e5657640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e00b12b0 con 0x7f78e00b3cf0 2023-08-25T09:47:54.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78e5657640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e006c0e0 con 0x7f78e00ca780 2023-08-25T09:47:54.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78e5657640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006ae20 con 0x7f78e015acd0 2023-08-25T09:47:54.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78deffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78de7fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 0x7f78e00cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78de7fc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 0x7f78e00cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:56788/0 (socket says 172.21.15.70:56788) 2023-08-25T09:47:54.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.199+0000 7f78de7fc640 1 -- 172.21.15.70:0/221412672 learned_addr learned my addr 172.21.15.70:0/221412672 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:54.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.200+0000 7f78deffd640 1 -- 172.21.15.70:0/221412672 >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 msgr2=0x7f78e00cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:54.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.200+0000 7f78deffd640 1 --2- 172.21.15.70:0/221412672 >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 0x7f78e00cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.200+0000 7f78deffd640 1 -- 172.21.15.70:0/221412672 >> v1:172.21.15.155:6789/0 conn(0x7f78e00b3cf0 legacy=0x7f78e00b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.200+0000 7f78deffd640 1 -- 172.21.15.70:0/221412672 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e006b910 con 0x7f78e015acd0 2023-08-25T09:47:54.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.200+0000 7f78de7fc640 1 --2- 172.21.15.70:0/221412672 >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 0x7f78e00cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:54.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.202+0000 7f78deffd640 1 --2- 172.21.15.70:0/221412672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e015b0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009790 tx=0x7f78d002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd1664f1be5c4e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:54.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.202+0000 7f78ddffb640 1 -- 172.21.15.70:0/221412672 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0031780 con 0x7f78e015acd0 2023-08-25T09:47:54.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78ddffb640 1 -- 172.21.15.70:0/221412672 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d00318e0 con 0x7f78e015acd0 2023-08-25T09:47:54.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78ddffb640 1 -- 172.21.15.70:0/221412672 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d0031c70 con 0x7f78e015acd0 2023-08-25T09:47:54.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 -- 172.21.15.70:0/221412672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 msgr2=0x7f78e015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 --2- 172.21.15.70:0/221412672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e015b0b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009790 tx=0x7f78d002b820 comp rx=0 tx=0).stop 2023-08-25T09:47:54.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 -- 172.21.15.70:0/221412672 shutdown_connections 2023-08-25T09:47:54.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 --2- 172.21.15.70:0/221412672 >> v2:172.21.15.155:3300/0 conn(0x7f78e00ca780 0x7f78e00cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 --2- 172.21.15.70:0/221412672 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e015b0b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 -- 172.21.15.70:0/221412672 >> 172.21.15.70:0/221412672 conn(0x7f78e00ce760 msgr2=0x7f78e00ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:54.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 -- 172.21.15.70:0/221412672 shutdown_connections 2023-08-25T09:47:54.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.203+0000 7f78e5657640 1 -- 172.21.15.70:0/221412672 wait complete. 2023-08-25T09:47:54.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.205+0000 7f78e5657640 1 Processor -- start 2023-08-25T09:47:54.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.205+0000 7f78e5657640 1 -- start start 2023-08-25T09:47:54.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78e5657640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78e00b3cf0 0x7f78e015d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78e5657640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 0x7f78e006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78e5657640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e01536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78e5657640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006fcd0 con 0x7f78e00ca780 2023-08-25T09:47:54.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78e5657640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006f710 con 0x7f78e00b3cf0 2023-08-25T09:47:54.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78e5657640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e006f9f0 con 0x7f78e015acd0 2023-08-25T09:47:54.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78de7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 0x7f78e006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78de7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 0x7f78e006ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:58990/0 (socket says 172.21.15.70:58990) 2023-08-25T09:47:54.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.206+0000 7f78de7fc640 1 -- 172.21.15.70:0/1832689534 learned_addr learned my addr 172.21.15.70:0/1832689534 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78de7fc640 1 -- 172.21.15.70:0/1832689534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 msgr2=0x7f78e01536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78de7fc640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e01536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78de7fc640 1 -- 172.21.15.70:0/1832689534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78e00b3cf0 msgr2=0x7f78e015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78de7fc640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78e00b3cf0 0x7f78e015d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78de7fc640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d40094d0 con 0x7f78e00ca780 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78de7fc640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 0x7f78e006ed50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e9b0 tx=0x7f78d400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:54.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d400ca00 con 0x7f78e00ca780 2023-08-25T09:47:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d400cb60 con 0x7f78e00ca780 2023-08-25T09:47:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d4013470 con 0x7f78e00ca780 2023-08-25T09:47:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.207+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d00093a0 con 0x7f78e00ca780 2023-08-25T09:47:54.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.208+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e0156490 con 0x7f78e00ca780 2023-08-25T09:47:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.209+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e004e420 con 0x7f78e00ca780 2023-08-25T09:47:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.209+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f78d40135d0 con 0x7f78e00ca780 2023-08-25T09:47:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.210+0000 7f78c77fe640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78cc06a050 0x7f78cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.210+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f78d4088940 con 0x7f78e00ca780 2023-08-25T09:47:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.211+0000 7f78deffd640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78cc06a050 0x7f78cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.211+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f78cc0721a0 con 0x7f78e00ca780 2023-08-25T09:47:54.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.211+0000 7f78deffd640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78cc06a050 0x7f78cc06c510 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f78d0036000 tx=0x7f78d0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:54.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.215+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f78d4053fa0 con 0x7f78e00ca780 2023-08-25T09:47:54.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.449+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f78e006bec0 con 0x7f78e00ca780 2023-08-25T09:47:54.452 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:47:54.451+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:47:54.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.454+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(155..155 src has 1..155) v4 ==== 374+0+0 (secure 0 0 0) 0x7f78d40760d0 con 0x7f78e00ca780 2023-08-25T09:47:54.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.454+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f78cc072a50 con 0x7f78e00ca780 2023-08-25T09:47:54.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.457+0000 7f78c77fe640 1 -- 172.21.15.70:0/1832689534 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f78d4021020 con 0x7f78e00ca780 2023-08-25T09:47:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:47:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.459+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78cc06a050 msgr2=0x7f78cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.459+0000 7f78e5657640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78cc06a050 0x7f78cc06c510 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f78d0036000 tx=0x7f78d0030000 comp rx=0 tx=0).stop 2023-08-25T09:47:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.459+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 msgr2=0x7f78e006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.459+0000 7f78e5657640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 0x7f78e006ed50 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f78d400e9b0 tx=0x7f78d400ee70 comp rx=0 tx=0).stop 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.460+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 shutdown_connections 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.460+0000 7f78e5657640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78cc06a050 0x7f78cc06c510 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.460+0000 7f78e5657640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78e015acd0 0x7f78e01536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.460+0000 7f78e5657640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78e00ca780 0x7f78e006ed50 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.460+0000 7f78e5657640 1 --2- 172.21.15.70:0/1832689534 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78e00b3cf0 0x7f78e015d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.460+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 >> 172.21.15.70:0/1832689534 conn(0x7f78e00ce760 msgr2=0x7f78e00b2210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:54.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.461+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 shutdown_connections 2023-08-25T09:47:54.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.461+0000 7f78e5657640 1 -- 172.21.15.70:0/1832689534 wait complete. 2023-08-25T09:47:54.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:47:54.495 INFO:tasks.ceph.mds.d.smithi155.stderr: -8433> 2023-08-25T09:47:03.386+0000 7f0c92f26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:47:54.573 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:47:54.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.686+0000 7f0e48ffd640 1 Processor -- start 2023-08-25T09:47:54.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.686+0000 7f0e48ffd640 1 -- start start 2023-08-25T09:47:54.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e48ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e48ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e4406f300 0x7f0e4406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e48ffd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4415be60 con 0x7f0e44070770 2023-08-25T09:47:54.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e48ffd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4415b3d0 con 0x7f0e4406f300 2023-08-25T09:47:54.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e48ffd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4406ee70 con 0x7f0e440ce410 2023-08-25T09:47:54.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e41d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0e44070770 0x7f0e440ce7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34184/0 (socket says 172.21.15.70:34184) 2023-08-25T09:47:54.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e42d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0e4406f300 0x7f0e4406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e41d9b640 1 -- 172.21.15.70:0/3813066115 learned_addr learned my addr 172.21.15.70:0/3813066115 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e4259c640 1 --2- 172.21.15.70:0/3813066115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e4159a640 1 -- 172.21.15.70:0/3813066115 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899318719 0 0) 0x7f0e4415be60 con 0x7f0e44070770 2023-08-25T09:47:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.688+0000 7f0e42d9d640 1 -- 172.21.15.70:0/3813066115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 msgr2=0x7f0e4415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.689+0000 7f0e42d9d640 1 --2- 172.21.15.70:0/3813066115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.689+0000 7f0e42d9d640 1 -- 172.21.15.70:0/3813066115 >> v1:172.21.15.70:6789/0 conn(0x7f0e44070770 legacy=0x7f0e440ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.689+0000 7f0e42d9d640 1 -- 172.21.15.70:0/3813066115 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4415b5c0 con 0x7f0e4406f300 2023-08-25T09:47:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.690+0000 7f0e42d9d640 1 --2- 172.21.15.70:0/3813066115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e4406f300 0x7f0e4406f790 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009d90 tx=0x7f0e3c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d22a323a5916ab server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:54.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.690+0000 7f0e4159a640 1 -- 172.21.15.70:0/3813066115 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c02cc60 con 0x7f0e4406f300 2023-08-25T09:47:54.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.690+0000 7f0e4159a640 1 -- 172.21.15.70:0/3813066115 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3c02cdc0 con 0x7f0e4406f300 2023-08-25T09:47:54.694 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.690+0000 7f0e4159a640 1 -- 172.21.15.70:0/3813066115 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3c034660 con 0x7f0e4406f300 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.690+0000 7f0e48ffd640 1 -- 172.21.15.70:0/3813066115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e4406f300 msgr2=0x7f0e4406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.690+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/3813066115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e4406f300 0x7f0e4406f790 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c009d90 tx=0x7f0e3c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.691+0000 7f0e48ffd640 1 -- 172.21.15.70:0/3813066115 shutdown_connections 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.691+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/3813066115 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e4406f300 0x7f0e4406f790 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.691+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/3813066115 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.691+0000 7f0e48ffd640 1 -- 172.21.15.70:0/3813066115 >> 172.21.15.70:0/3813066115 conn(0x7f0e44069850 msgr2=0x7f0e44069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:54.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.691+0000 7f0e48ffd640 1 -- 172.21.15.70:0/3813066115 shutdown_connections 2023-08-25T09:47:54.696 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.691+0000 7f0e48ffd640 1 -- 172.21.15.70:0/3813066115 wait complete. 2023-08-25T09:47:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.692+0000 7f0e48ffd640 1 Processor -- start 2023-08-25T09:47:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.693+0000 7f0e48ffd640 1 -- start start 2023-08-25T09:47:54.697 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.693+0000 7f0e48ffd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4406f300 0x7f0e4414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.693+0000 7f0e48ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 0x7f0e4414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.693+0000 7f0e48ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e48ffd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4415db30 con 0x7f0e44070770 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e48ffd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e441503e0 con 0x7f0e4406f300 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e48ffd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b1ef0 con 0x7f0e440ce410 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 0x7f0e4414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.698 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 0x7f0e4414fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59010/0 (socket says 172.21.15.70:59010) 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 -- 172.21.15.70:0/2236515728 learned_addr learned my addr 172.21.15.70:0/2236515728 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 -- 172.21.15.70:0/2236515728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 msgr2=0x7f0e4415d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 -- 172.21.15.70:0/2236515728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4406f300 msgr2=0x7f0e4414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4406f300 0x7f0e4414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 -- 172.21.15.70:0/2236515728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c009a70 con 0x7f0e44070770 2023-08-25T09:47:54.699 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.694+0000 7f0e41d9b640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 0x7f0e4414fd70 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c4d0 tx=0x7f0e3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.695+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e380074e0 con 0x7f0e44070770 2023-08-25T09:47:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.695+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38007660 con 0x7f0e44070770 2023-08-25T09:47:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.695+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38004a20 con 0x7f0e44070770 2023-08-25T09:47:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.695+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e440b21d0 con 0x7f0e44070770 2023-08-25T09:47:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.695+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e44151af0 con 0x7f0e44070770 2023-08-25T09:47:54.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.696+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44151d00 con 0x7f0e44070770 2023-08-25T09:47:54.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.697+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0e38004b80 con 0x7f0e44070770 2023-08-25T09:47:54.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.698+0000 7f0e32ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e2406a080 0x7f0e2406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:54.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.698+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0e380877c0 con 0x7f0e44070770 2023-08-25T09:47:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.698+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f0e240722e0 con 0x7f0e44070770 2023-08-25T09:47:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.698+0000 7f0e4259c640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e2406a080 0x7f0e2406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:54.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.699+0000 7f0e4259c640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e2406a080 0x7f0e2406c540 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:54.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.702+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0e38052e40 con 0x7f0e44070770 2023-08-25T09:47:54.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:54.926+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0e4404e420 con 0x7f0e44070770 2023-08-25T09:47:55.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.462+0000 7f0e32ffd640 1 -- 172.21.15.70:0/2236515728 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e38056460 con 0x7f0e44070770 2023-08-25T09:47:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e2406a080 msgr2=0x7f0e2406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e2406a080 0x7f0e2406c540 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f0e34006fd0 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2023-08-25T09:47:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 msgr2=0x7f0e4414fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 0x7f0e4414fd70 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c4d0 tx=0x7f0e3800c990 comp rx=0 tx=0).stop 2023-08-25T09:47:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 shutdown_connections 2023-08-25T09:47:55.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0e2406a080 0x7f0e2406c540 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0e440ce410 0x7f0e4415d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0e44070770 0x7f0e4414fd70 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 --2- 172.21.15.70:0/2236515728 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0e4406f300 0x7f0e4414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 >> 172.21.15.70:0/2236515728 conn(0x7f0e44069850 msgr2=0x7f0e441573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 shutdown_connections 2023-08-25T09:47:55.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.465+0000 7f0e48ffd640 1 -- 172.21.15.70:0/2236515728 wait complete. 2023-08-25T09:47:55.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:47:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.704+0000 7f30a966a640 1 Processor -- start 2023-08-25T09:47:55.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.705+0000 7f30a966a640 1 -- start start 2023-08-25T09:47:55.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.706+0000 7f30a966a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f30a415c950 0x7f30a40b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:55.708 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.706+0000 7f30a966a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f30a406b180 0x7f30a41606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.706+0000 7f30a966a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a40b2300 con 0x7f30a415cd30 2023-08-25T09:47:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.706+0000 7f30a966a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a40b16f0 con 0x7f30a415c950 2023-08-25T09:47:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a966a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a40b19d0 con 0x7f30a406b180 2023-08-25T09:47:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f30a415c950 0x7f30a40b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f30a415c950 0x7f30a40b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59016/0 (socket says 172.21.15.70:59016) 2023-08-25T09:47:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 -- 172.21.15.70:0/4118768862 learned_addr learned my addr 172.21.15.70:0/4118768862 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:55.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 -- 172.21.15.70:0/4118768862 >> v2:172.21.15.155:3300/0 conn(0x7f30a406b180 msgr2=0x7f30a41606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:55.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 --2- 172.21.15.70:0/4118768862 >> v2:172.21.15.155:3300/0 conn(0x7f30a406b180 0x7f30a41606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 -- 172.21.15.70:0/4118768862 >> v1:172.21.15.155:6789/0 conn(0x7f30a415cd30 legacy=0x7f30a406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:55.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.707+0000 7f30a2ffd640 1 -- 172.21.15.70:0/4118768862 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40b1cb0 con 0x7f30a415c950 2023-08-25T09:47:55.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.708+0000 7f30a2ffd640 1 --2- 172.21.15.70:0/4118768862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415c950 0x7f30a40b4190 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f30a40b2270 tx=0x7f309402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef75fa8c978f985 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:55.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a27fc640 1 -- 172.21.15.70:0/4118768862 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094031870 con 0x7f30a415c950 2023-08-25T09:47:55.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a27fc640 1 -- 172.21.15.70:0/4118768862 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30940319d0 con 0x7f30a415c950 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a27fc640 1 -- 172.21.15.70:0/4118768862 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094031d60 con 0x7f30a415c950 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a966a640 1 -- 172.21.15.70:0/4118768862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415c950 msgr2=0x7f30a40b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a966a640 1 --2- 172.21.15.70:0/4118768862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415c950 0x7f30a40b4190 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f30a40b2270 tx=0x7f309402b910 comp rx=0 tx=0).stop 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a966a640 1 -- 172.21.15.70:0/4118768862 shutdown_connections 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a966a640 1 --2- 172.21.15.70:0/4118768862 >> v2:172.21.15.155:3300/0 conn(0x7f30a406b180 0x7f30a41606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a966a640 1 --2- 172.21.15.70:0/4118768862 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415c950 0x7f30a40b4190 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.709+0000 7f30a966a640 1 -- 172.21.15.70:0/4118768862 >> 172.21.15.70:0/4118768862 conn(0x7f30a4069c70 msgr2=0x7f30a406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:55.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.710+0000 7f30a966a640 1 -- 172.21.15.70:0/4118768862 shutdown_connections 2023-08-25T09:47:55.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.710+0000 7f30a966a640 1 -- 172.21.15.70:0/4118768862 wait complete. 2023-08-25T09:47:55.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.711+0000 7f30a966a640 1 Processor -- start 2023-08-25T09:47:55.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.711+0000 7f30a966a640 1 -- start start 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.712+0000 7f30a966a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30a406b180 0x7f30a40cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.712+0000 7f30a966a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30a415c950 0x7f30a40c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.712+0000 7f30a966a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 0x7f30a40c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.712+0000 7f30a966a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40cab30 con 0x7f30a415cd30 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.712+0000 7f30a966a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40c9f20 con 0x7f30a415c950 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.712+0000 7f30a966a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40ca200 con 0x7f30a406b180 2023-08-25T09:47:55.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 0x7f30a40c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 0x7f30a40c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59026/0 (socket says 172.21.15.70:59026) 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 -- 172.21.15.70:0/1565822868 learned_addr learned my addr 172.21.15.70:0/1565822868 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a2ffd640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30a406b180 0x7f30a40cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f309a7fc640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30a415c950 0x7f30a40c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 -- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30a406b180 msgr2=0x7f30a40cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30a406b180 0x7f30a40cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.717 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 -- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30a415c950 msgr2=0x7f30a40c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30a415c950 0x7f30a40c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.713+0000 7f30a37fe640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30940094d0 con 0x7f30a415cd30 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.714+0000 7f30a37fe640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 0x7f30a40c98a0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b970 tx=0x7f309c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.714+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c00cb40 con 0x7f30a415cd30 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.714+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c00ccc0 con 0x7f30a415cd30 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.714+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c014680 con 0x7f30a415cd30 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.714+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a40ca4e0 con 0x7f30a415cd30 2023-08-25T09:47:55.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.714+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a4152490 con 0x7f30a415cd30 2023-08-25T09:47:55.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.715+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a4071240 con 0x7f30a415cd30 2023-08-25T09:47:55.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.716+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f309c01d3b0 con 0x7f30a415cd30 2023-08-25T09:47:55.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.717+0000 7f309bfff640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f308806a050 0x7f308806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:55.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.717+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f309c088600 con 0x7f30a415cd30 2023-08-25T09:47:55.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.717+0000 7f30a2ffd640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f308806a050 0x7f308806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:55.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.717+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=156}) v3 -- 0x7f30880722b0 con 0x7f30a415cd30 2023-08-25T09:47:55.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.718+0000 7f30a2ffd640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f308806a050 0x7f308806c510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3094036000 tx=0x7f309403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:55.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.722+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f309c053bf0 con 0x7f30a415cd30 2023-08-25T09:47:55.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:55.914+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30a40b4650 con 0x7f30a415cd30 2023-08-25T09:47:56.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.671+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(156..156 src has 1..156) v4 ==== 296+0+0 (secure 0 0 0) 0x7f309c00ce30 con 0x7f30a415cd30 2023-08-25T09:47:56.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.672+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f30880706e0 con 0x7f30a415cd30 2023-08-25T09:47:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.676+0000 7f309bfff640 1 -- 172.21.15.70:0/1565822868 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v156) v1 ==== 181+0+0 (secure 0 0 0) 0x7f309c019030 con 0x7f30a415cd30 2023-08-25T09:47:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:47:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.686+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f308806a050 msgr2=0x7f308806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:56.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.686+0000 7f30a966a640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f308806a050 0x7f308806c510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3094036000 tx=0x7f309403c040 comp rx=0 tx=0).stop 2023-08-25T09:47:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.687+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 msgr2=0x7f30a40c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.687+0000 7f30a966a640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 0x7f30a40c98a0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f309c00b970 tx=0x7f309c00be30 comp rx=0 tx=0).stop 2023-08-25T09:47:56.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 shutdown_connections 2023-08-25T09:47:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f308806a050 0x7f308806c510 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f30a415cd30 0x7f30a40c98a0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f30a415c950 0x7f30a40c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 --2- 172.21.15.70:0/1565822868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f30a406b180 0x7f30a40cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 >> 172.21.15.70:0/1565822868 conn(0x7f30a4069c70 msgr2=0x7f30a40b0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:56.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 shutdown_connections 2023-08-25T09:47:56.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.688+0000 7f30a966a640 1 -- 172.21.15.70:0/1565822868 wait complete. 2023-08-25T09:47:56.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:47:56.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.922+0000 7f2be7814640 1 Processor -- start 2023-08-25T09:47:56.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.923+0000 7f2be7814640 1 -- start start 2023-08-25T09:47:56.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.924+0000 7f2be7814640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be00cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:56.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.924+0000 7f2be7814640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2be0143ef0 0x7f2be0146740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.924+0000 7f2be7814640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be0147470 con 0x7f2be0143b10 2023-08-25T09:47:56.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.924+0000 7f2be7814640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be00ca9f0 con 0x7f2be0143ef0 2023-08-25T09:47:56.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.924+0000 7f2be7814640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00cacd0 con 0x7f2be006b180 2023-08-25T09:47:56.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.925+0000 7f2be4daf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be00cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:56.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.925+0000 7f2be55b0640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2be0143b10 0x7f2be006aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:35728/0 (socket says 172.21.15.70:35728) 2023-08-25T09:47:56.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.925+0000 7f2be55b0640 1 -- 172.21.15.70:0/651408834 learned_addr learned my addr 172.21.15.70:0/651408834 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.925+0000 7f2be5db1640 1 --2- 172.21.15.70:0/651408834 >> v2:172.21.15.155:3300/0 conn(0x7f2be0143ef0 0x7f2be0146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.926+0000 7f2be4daf640 1 -- 172.21.15.70:0/651408834 >> v2:172.21.15.155:3300/0 conn(0x7f2be0143ef0 msgr2=0x7f2be0146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.926+0000 7f2be4daf640 1 --2- 172.21.15.70:0/651408834 >> v2:172.21.15.155:3300/0 conn(0x7f2be0143ef0 0x7f2be0146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.926+0000 7f2be4daf640 1 -- 172.21.15.70:0/651408834 >> v1:172.21.15.155:6789/0 conn(0x7f2be0143b10 legacy=0x7f2be006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.926+0000 7f2be4daf640 1 -- 172.21.15.70:0/651408834 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00cafb0 con 0x7f2be006b180 2023-08-25T09:47:56.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.926+0000 7f2be5db1640 1 --2- 172.21.15.70:0/651408834 >> v2:172.21.15.155:3300/0 conn(0x7f2be0143ef0 0x7f2be0146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:47:56.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.928+0000 7f2be4daf640 1 --2- 172.21.15.70:0/651408834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be00cb7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00097a0 tx=0x7f2bd002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3dd7b78f690 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:56.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.928+0000 7f2bd7fff640 1 -- 172.21.15.70:0/651408834 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd002d040 con 0x7f2be006b180 2023-08-25T09:47:56.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.928+0000 7f2bd7fff640 1 -- 172.21.15.70:0/651408834 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd0031810 con 0x7f2be006b180 2023-08-25T09:47:56.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.928+0000 7f2bd7fff640 1 -- 172.21.15.70:0/651408834 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0031ba0 con 0x7f2be006b180 2023-08-25T09:47:56.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 -- 172.21.15.70:0/651408834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 msgr2=0x7f2be00cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:56.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 --2- 172.21.15.70:0/651408834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be00cb7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00097a0 tx=0x7f2bd002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 -- 172.21.15.70:0/651408834 shutdown_connections 2023-08-25T09:47:56.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 --2- 172.21.15.70:0/651408834 >> v2:172.21.15.155:3300/0 conn(0x7f2be0143ef0 0x7f2be0146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 --2- 172.21.15.70:0/651408834 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be00cb7b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 -- 172.21.15.70:0/651408834 >> 172.21.15.70:0/651408834 conn(0x7f2be0069c70 msgr2=0x7f2be006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:56.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 -- 172.21.15.70:0/651408834 shutdown_connections 2023-08-25T09:47:56.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.929+0000 7f2be7814640 1 -- 172.21.15.70:0/651408834 wait complete. 2023-08-25T09:47:56.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.931+0000 7f2be7814640 1 Processor -- start 2023-08-25T09:47:56.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.932+0000 7f2be7814640 1 -- start start 2023-08-25T09:47:56.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be7814640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be013ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:56.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be7814640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2be0143b10 0x7f2be015a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:56.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be7814640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 0x7f2be01570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:56.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be7814640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be015aaa0 con 0x7f2be0143ef0 2023-08-25T09:47:56.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be7814640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be0140660 con 0x7f2be0143b10 2023-08-25T09:47:56.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be7814640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be0140940 con 0x7f2be006b180 2023-08-25T09:47:56.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be5db1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 0x7f2be01570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:56.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be5db1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 0x7f2be01570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59038/0 (socket says 172.21.15.70:59038) 2023-08-25T09:47:56.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.933+0000 7f2be5db1640 1 -- 172.21.15.70:0/982089391 learned_addr learned my addr 172.21.15.70:0/982089391 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:56.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be55b0640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be013ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:56.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be4daf640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2be0143b10 0x7f2be015a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be5db1640 1 -- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 msgr2=0x7f2be013ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be5db1640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be013ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be5db1640 1 -- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2be0143b10 msgr2=0x7f2be015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be5db1640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2be0143b10 0x7f2be015a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be5db1640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8009a70 con 0x7f2be0143ef0 2023-08-25T09:47:56.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.934+0000 7f2be5db1640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 0x7f2be01570e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800efc0 tx=0x7f2bd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.935+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8009cd0 con 0x7f2be0143ef0 2023-08-25T09:47:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.935+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd800f040 con 0x7f2be0143ef0 2023-08-25T09:47:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.935+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8003990 con 0x7f2be0143ef0 2023-08-25T09:47:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.935+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd0009400 con 0x7f2be0143ef0 2023-08-25T09:47:56.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.935+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be0140f00 con 0x7f2be0143ef0 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.936+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be0070a80 con 0x7f2be0143ef0 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.937+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2bd8014480 con 0x7f2be0143ef0 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.938+0000 7f2bd5ffb640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2bc406a050 0x7f2bc406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.938+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f2bd80890c0 con 0x7f2be0143ef0 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.938+0000 7f2be55b0640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2bc406a050 0x7f2bc406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.938+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=157}) v3 -- 0x7f2bc4071440 con 0x7f2be0143ef0 2023-08-25T09:47:56.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.939+0000 7f2be55b0640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2bc406a050 0x7f2bc406c510 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005880 tx=0x7f2bdc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:56.942+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2bd8058300 con 0x7f2be0143ef0 2023-08-25T09:47:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.130+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2be004e420 con 0x7f2be0143ef0 2023-08-25T09:47:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.676+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(157..157 src has 1..157) v4 ==== 296+0+0 (secure 0 0 0) 0x7f2bd8058bc0 con 0x7f2be0143ef0 2023-08-25T09:47:57.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.676+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=158}) v3 -- 0x7f2bc40701d0 con 0x7f2be0143ef0 2023-08-25T09:47:57.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.685+0000 7f2bd5ffb640 1 -- 172.21.15.70:0/982089391 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v157) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2bd80180d0 con 0x7f2be0143ef0 2023-08-25T09:47:57.687 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:47:57.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.690+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2bc406a050 msgr2=0x7f2bc406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:57.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.690+0000 7f2be7814640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2bc406a050 0x7f2bc406c510 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc005880 tx=0x7f2bdc009040 comp rx=0 tx=0).stop 2023-08-25T09:47:57.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.690+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 msgr2=0x7f2be01570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:57.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.690+0000 7f2be7814640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 0x7f2be01570e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800efc0 tx=0x7f2bd800c490 comp rx=0 tx=0).stop 2023-08-25T09:47:57.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 shutdown_connections 2023-08-25T09:47:57.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2bc406a050 0x7f2bc406c510 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:57.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2be0143ef0 0x7f2be01570e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:57.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2be0143b10 0x7f2be015a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:57.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 --2- 172.21.15.70:0/982089391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2be006b180 0x7f2be013ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:57.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 >> 172.21.15.70:0/982089391 conn(0x7f2be0069c70 msgr2=0x7f2be006fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:57.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 shutdown_connections 2023-08-25T09:47:57.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:57.691+0000 7f2be7814640 1 -- 172.21.15.70:0/982089391 wait complete. 2023-08-25T09:47:57.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:47:58.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.151+0000 7f1794891640 1 Processor -- start 2023-08-25T09:47:58.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.152+0000 7f1794891640 1 -- start start 2023-08-25T09:47:58.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.153+0000 7f1794891640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f179015cce0 0x7f17900cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.153+0000 7f1794891640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17900cd130 0x7f17900cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.153+0000 7f1794891640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17900d0010 con 0x7f17900cfc00 2023-08-25T09:47:58.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.153+0000 7f1794891640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17900b8160 con 0x7f179015cce0 2023-08-25T09:47:58.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.153+0000 7f1794891640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f179015d0c0 con 0x7f17900cd130 2023-08-25T09:47:58.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.153+0000 7f178f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f179015cce0 0x7f17900cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.154+0000 7f178f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f179015cce0 0x7f17900cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59044/0 (socket says 172.21.15.70:59044) 2023-08-25T09:47:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.154+0000 7f178f7fe640 1 -- 172.21.15.70:0/3916532109 learned_addr learned my addr 172.21.15.70:0/3916532109 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.154+0000 7f178f7fe640 1 -- 172.21.15.70:0/3916532109 >> v2:172.21.15.155:3300/0 conn(0x7f17900cd130 msgr2=0x7f17900cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.154+0000 7f178f7fe640 1 --2- 172.21.15.70:0/3916532109 >> v2:172.21.15.155:3300/0 conn(0x7f17900cd130 0x7f17900cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.154+0000 7f178f7fe640 1 -- 172.21.15.70:0/3916532109 >> v1:172.21.15.155:6789/0 conn(0x7f17900cfc00 legacy=0x7f17900b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.154+0000 7f178f7fe640 1 -- 172.21.15.70:0/3916532109 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17900cde70 con 0x7f179015cce0 2023-08-25T09:47:58.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.155+0000 7f178f7fe640 1 --2- 172.21.15.70:0/3916532109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f179015cce0 0x7f17900cd930 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f1790070d50 tx=0x7f178002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5aa2f87198a725 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.155+0000 7f178e7fc640 1 -- 172.21.15.70:0/3916532109 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178002c960 con 0x7f179015cce0 2023-08-25T09:47:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f178e7fc640 1 -- 172.21.15.70:0/3916532109 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178002cac0 con 0x7f179015cce0 2023-08-25T09:47:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f178e7fc640 1 -- 172.21.15.70:0/3916532109 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178002ce50 con 0x7f179015cce0 2023-08-25T09:47:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 -- 172.21.15.70:0/3916532109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f179015cce0 msgr2=0x7f17900cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 --2- 172.21.15.70:0/3916532109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f179015cce0 0x7f17900cd930 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f1790070d50 tx=0x7f178002bdc0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 -- 172.21.15.70:0/3916532109 shutdown_connections 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 --2- 172.21.15.70:0/3916532109 >> v2:172.21.15.155:3300/0 conn(0x7f17900cd130 0x7f17900cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 --2- 172.21.15.70:0/3916532109 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f179015cce0 0x7f17900cd930 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 -- 172.21.15.70:0/3916532109 >> 172.21.15.70:0/3916532109 conn(0x7f1790070050 msgr2=0x7f1790070460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 -- 172.21.15.70:0/3916532109 shutdown_connections 2023-08-25T09:47:58.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.156+0000 7f1794891640 1 -- 172.21.15.70:0/3916532109 wait complete. 2023-08-25T09:47:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.158+0000 7f1794891640 1 Processor -- start 2023-08-25T09:47:58.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.158+0000 7f1794891640 1 -- start start 2023-08-25T09:47:58.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.159+0000 7f1794891640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 0x7f17900c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.159+0000 7f1794891640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17900cfc00 0x7f1790156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.159+0000 7f1794891640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f179015cce0 0x7f1790156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.159+0000 7f1794891640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900d29d0 con 0x7f17900cd130 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.159+0000 7f1794891640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17900d2cb0 con 0x7f179015cce0 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.159+0000 7f1794891640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17900d2f90 con 0x7f17900cfc00 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 0x7f17900c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 0x7f17900c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59050/0 (socket says 172.21.15.70:59050) 2023-08-25T09:47:58.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 -- 172.21.15.70:0/3906059605 learned_addr learned my addr 172.21.15.70:0/3906059605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178ffff640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f179015cce0 0x7f1790156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 -- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17900cfc00 msgr2=0x7f1790156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17900cfc00 0x7f1790156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 -- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f179015cce0 msgr2=0x7f1790156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f179015cce0 0x7f1790156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 -- 172.21.15.70:0/3906059605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1780009b30 con 0x7f17900cd130 2023-08-25T09:47:58.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178ffff640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f179015cce0 0x7f1790156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.160+0000 7f178f7fe640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 0x7f17900c83c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f1780009e50 tx=0x7f1780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.161+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178003e450 con 0x7f17900cd130 2023-08-25T09:47:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.161+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178003e5b0 con 0x7f17900cd130 2023-08-25T09:47:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.161+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17900d3270 con 0x7f17900cd130 2023-08-25T09:47:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.161+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178003e960 con 0x7f17900cd130 2023-08-25T09:47:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.161+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17900c8900 con 0x7f17900cd130 2023-08-25T09:47:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.162+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17901f3810 con 0x7f17900cd130 2023-08-25T09:47:58.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.163+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f178003eac0 con 0x7f17900cd130 2023-08-25T09:47:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.164+0000 7f1787fff640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f177006a080 0x7f177006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.164+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f17800aafe0 con 0x7f17900cd130 2023-08-25T09:47:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.164+0000 7f178effd640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f177006a080 0x7f177006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.165+0000 7f178effd640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f177006a080 0x7f177006c540 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17900b72d0 tx=0x7f1778006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:58.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.168+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17901f3810 con 0x7f17900cd130 2023-08-25T09:47:58.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.362+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f17901f3810 con 0x7f17900cd130 2023-08-25T09:47:58.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.368+0000 7f1787fff640 1 -- 172.21.15.70:0/3906059605 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17901f3810 con 0x7f17900cd130 2023-08-25T09:47:58.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.370+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f177006a080 msgr2=0x7f177006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.370+0000 7f1794891640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f177006a080 0x7f177006c540 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17900b72d0 tx=0x7f1778006bf0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.370+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 msgr2=0x7f17900c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.370+0000 7f1794891640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 0x7f17900c83c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f1780009e50 tx=0x7f1780004a40 comp rx=0 tx=0).stop 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 shutdown_connections 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f177006a080 0x7f177006c540 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f179015cce0 0x7f1790156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17900cfc00 0x7f1790156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 --2- 172.21.15.70:0/3906059605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17900cd130 0x7f17900c83c0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 >> 172.21.15.70:0/3906059605 conn(0x7f1790070050 msgr2=0x7f17900d0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:58.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 shutdown_connections 2023-08-25T09:47:58.375 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.371+0000 7f1794891640 1 -- 172.21.15.70:0/3906059605 wait complete. 2023-08-25T09:47:58.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-08-25T09:47:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.605+0000 7f4caf153640 1 Processor -- start 2023-08-25T09:47:58.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.605+0000 7f4caf153640 1 -- start start 2023-08-25T09:47:58.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4caf153640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ca80b12f0 0x7f4ca815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4caf153640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4caf153640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ca815e7c0 con 0x7f4ca815a7c0 2023-08-25T09:47:58.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4caf153640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ca8071b50 con 0x7f4ca80b12f0 2023-08-25T09:47:58.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4caf153640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca815e490 con 0x7f4ca815f190 2023-08-25T09:47:58.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4cae151640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4ca815a7c0 0x7f4ca80b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34188/0 (socket says 172.21.15.70:34188) 2023-08-25T09:47:58.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4cae151640 1 -- 172.21.15.70:0/3901283341 learned_addr learned my addr 172.21.15.70:0/3901283341 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:47:58.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.607+0000 7f4cad950640 1 --2- 172.21.15.70:0/3901283341 >> v2:172.21.15.70:3300/0 conn(0x7f4ca80b12f0 0x7f4ca815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cae952640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cad950640 1 -- 172.21.15.70:0/3901283341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 msgr2=0x7f4ca80cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cad14f640 1 -- 172.21.15.70:0/3901283341 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1150832899 0 0) 0x7f4ca815e7c0 con 0x7f4ca815a7c0 2023-08-25T09:47:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cad950640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cad950640 1 -- 172.21.15.70:0/3901283341 >> v1:172.21.15.70:6789/0 conn(0x7f4ca815a7c0 legacy=0x7f4ca80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cad950640 1 -- 172.21.15.70:0/3901283341 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca80b3d60 con 0x7f4ca80b12f0 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.608+0000 7f4cae952640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.609+0000 7f4cad950640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca815ec50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f4c980097a0 tx=0x7f4c9802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775a3d5ead7f1d22 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.609+0000 7f4cad14f640 1 -- 172.21.15.70:0/3901283341 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9802d040 con 0x7f4ca80b12f0 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.609+0000 7f4cad14f640 1 -- 172.21.15.70:0/3901283341 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98031810 con 0x7f4ca80b12f0 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.609+0000 7f4cad14f640 1 -- 172.21.15.70:0/3901283341 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c98031ba0 con 0x7f4ca80b12f0 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 -- 172.21.15.70:0/3901283341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 msgr2=0x7f4ca815ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca815ec50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f4c980097a0 tx=0x7f4c9802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 -- 172.21.15.70:0/3901283341 shutdown_connections 2023-08-25T09:47:58.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca815ec50 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 --2- 172.21.15.70:0/3901283341 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 -- 172.21.15.70:0/3901283341 >> 172.21.15.70:0/3901283341 conn(0x7f4ca80affd0 msgr2=0x7f4ca80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:58.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 -- 172.21.15.70:0/3901283341 shutdown_connections 2023-08-25T09:47:58.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.610+0000 7f4caf153640 1 -- 172.21.15.70:0/3901283341 wait complete. 2023-08-25T09:47:58.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.612+0000 7f4caf153640 1 Processor -- start 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.612+0000 7f4caf153640 1 -- start start 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4caf153640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca814f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4caf153640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca815a7c0 0x7f4ca80acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4caf153640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4caf153640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca806f9c0 con 0x7f4ca80b12f0 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4caf153640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81f2a40 con 0x7f4ca815a7c0 2023-08-25T09:47:58.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4caf153640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca81f2d20 con 0x7f4ca815f190 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4cae151640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca814f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4cae151640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca814f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:59074/0 (socket says 172.21.15.70:59074) 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.613+0000 7f4cae151640 1 -- 172.21.15.70:0/1729965812 learned_addr learned my addr 172.21.15.70:0/1729965812 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4cae151640 1 -- 172.21.15.70:0/1729965812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 msgr2=0x7f4ca80cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4cae151640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4cae151640 1 -- 172.21.15.70:0/1729965812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca815a7c0 msgr2=0x7f4ca80acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4cae151640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca815a7c0 0x7f4ca80acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:58.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4cae151640 1 -- 172.21.15.70:0/1729965812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98009400 con 0x7f4ca80b12f0 2023-08-25T09:47:58.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4cae151640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca814f730 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c430 tx=0x7f4ca400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:58.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca40074e0 con 0x7f4ca80b12f0 2023-08-25T09:47:58.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4007660 con 0x7f4ca80b12f0 2023-08-25T09:47:58.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4004a20 con 0x7f4ca80b12f0 2023-08-25T09:47:58.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.614+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca81f3000 con 0x7f4ca80b12f0 2023-08-25T09:47:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.615+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca814fc70 con 0x7f4ca80b12f0 2023-08-25T09:47:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.616+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca8066fb0 con 0x7f4ca80b12f0 2023-08-25T09:47:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.617+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4ca4004b80 con 0x7f4ca80b12f0 2023-08-25T09:47:58.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.618+0000 7f4c9effd640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8c06a050 0x7f4c8c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:58.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.618+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4ca40881a0 con 0x7f4ca80b12f0 2023-08-25T09:47:58.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.618+0000 7f4cad950640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8c06a050 0x7f4c8c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:58.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.619+0000 7f4cad950640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8c06a050 0x7f4c8c06c510 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4c98036000 tx=0x7f4c98030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:58.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.622+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ca4053b10 con 0x7f4ca80b12f0 2023-08-25T09:47:58.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:58.877+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f4ca80cf1c0 con 0x7f4ca80b12f0 2023-08-25T09:47:59.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.713+0000 7f4c9effd640 1 -- 172.21.15.70:0/1729965812 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v486) v1 ==== 140+0+0 (secure 0 0 0) 0x7f4ca4057130 con 0x7f4ca80b12f0 2023-08-25T09:47:59.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8c06a050 msgr2=0x7f4c8c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:59.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8c06a050 0x7f4c8c06c510 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4c98036000 tx=0x7f4c98030000 comp rx=0 tx=0).stop 2023-08-25T09:47:59.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 msgr2=0x7f4ca814f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:59.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca814f730 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c430 tx=0x7f4ca400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 shutdown_connections 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8c06a050 0x7f4c8c06c510 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ca815f190 0x7f4ca80cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ca815a7c0 0x7f4ca80acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 --2- 172.21.15.70:0/1729965812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ca80b12f0 0x7f4ca814f730 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 >> 172.21.15.70:0/1729965812 conn(0x7f4ca80affd0 msgr2=0x7f4ca80af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:59.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 shutdown_connections 2023-08-25T09:47:59.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.717+0000 7f4caf153640 1 -- 172.21.15.70:0/1729965812 wait complete. 2023-08-25T09:47:59.737 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-08-25T09:47:59.738 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-08-25T09:47:59.739 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:47:59.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.932+0000 7fddb97a4640 1 Processor -- start 2023-08-25T09:47:59.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.933+0000 7fddb97a4640 1 -- start start 2023-08-25T09:47:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.934+0000 7fddb97a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb40cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.934+0000 7fddb97a4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fddb40b0ec0 0x7fddb40b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.934+0000 7fddb97a4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb40cc790 con 0x7fddb40ce480 2023-08-25T09:47:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.934+0000 7fddb97a4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb40cdf90 con 0x7fddb40b0ec0 2023-08-25T09:47:59.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.934+0000 7fddb97a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40b1890 con 0x7fddb415d6b0 2023-08-25T09:47:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.934+0000 7fddb2ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb40cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb2ffd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb40cc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:47246/0 (socket says 172.21.15.70:47246) 2023-08-25T09:47:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb2ffd640 1 -- 172.21.15.70:0/3321209250 learned_addr learned my addr 172.21.15.70:0/3321209250 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb37fe640 1 --2- 172.21.15.70:0/3321209250 >> v2:172.21.15.155:3300/0 conn(0x7fddb40b0ec0 0x7fddb40b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:59.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb1ffb640 1 -- 172.21.15.70:0/3321209250 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246833591 0 0) 0x7fddb40cc790 con 0x7fddb40ce480 2023-08-25T09:47:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb1ffb640 1 -- 172.21.15.70:0/3321209250 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd94003520 con 0x7fddb40ce480 2023-08-25T09:47:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb2ffd640 1 -- 172.21.15.70:0/3321209250 >> v2:172.21.15.155:3300/0 conn(0x7fddb40b0ec0 msgr2=0x7fddb40b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:47:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb2ffd640 1 --2- 172.21.15.70:0/3321209250 >> v2:172.21.15.155:3300/0 conn(0x7fddb40b0ec0 0x7fddb40b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb2ffd640 1 -- 172.21.15.70:0/3321209250 >> v1:172.21.15.155:6789/0 conn(0x7fddb40ce480 legacy=0x7fddb415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb2ffd640 1 -- 172.21.15.70:0/3321209250 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40cec90 con 0x7fddb415d6b0 2023-08-25T09:47:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.935+0000 7fddb37fe640 1 --2- 172.21.15.70:0/3321209250 >> v2:172.21.15.155:3300/0 conn(0x7fddb40b0ec0 0x7fddb40b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:47:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.937+0000 7fddb2ffd640 1 --2- 172.21.15.70:0/3321209250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb40cc250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009870 tx=0x7fdda402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f17c535c6c9c3814 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:59.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.937+0000 7fddb1ffb640 1 -- 172.21.15.70:0/3321209250 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4030840 con 0x7fddb415d6b0 2023-08-25T09:47:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.937+0000 7fddb1ffb640 1 -- 172.21.15.70:0/3321209250 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda40309a0 con 0x7fddb415d6b0 2023-08-25T09:47:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.937+0000 7fddb1ffb640 1 -- 172.21.15.70:0/3321209250 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4030d30 con 0x7fddb415d6b0 2023-08-25T09:47:59.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 -- 172.21.15.70:0/3321209250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 msgr2=0x7fddb40cc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 --2- 172.21.15.70:0/3321209250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb40cc250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdda4009870 tx=0x7fdda402b980 comp rx=0 tx=0).stop 2023-08-25T09:47:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 -- 172.21.15.70:0/3321209250 shutdown_connections 2023-08-25T09:47:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 --2- 172.21.15.70:0/3321209250 >> v2:172.21.15.155:3300/0 conn(0x7fddb40b0ec0 0x7fddb40b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 --2- 172.21.15.70:0/3321209250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb40cc250 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 -- 172.21.15.70:0/3321209250 >> 172.21.15.70:0/3321209250 conn(0x7fddb4069c70 msgr2=0x7fddb406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:47:59.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 -- 172.21.15.70:0/3321209250 shutdown_connections 2023-08-25T09:47:59.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.938+0000 7fddb97a4640 1 -- 172.21.15.70:0/3321209250 wait complete. 2023-08-25T09:47:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.939+0000 7fddb97a4640 1 Processor -- start 2023-08-25T09:47:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.940+0000 7fddb97a4640 1 -- start start 2023-08-25T09:47:59.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.940+0000 7fddb97a4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddb40b0ec0 0x7fddb40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:59.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.940+0000 7fddb97a4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 0x7fddb414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.940+0000 7fddb97a4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb41f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.940+0000 7fddb97a4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40b4f40 con 0x7fddb40ce480 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb97a4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb414d0f0 con 0x7fddb40b0ec0 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb97a4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb414c600 con 0x7fddb415d6b0 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 0x7fddb414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 0x7fddb414c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39224/0 (socket says 172.21.15.70:39224) 2023-08-25T09:47:59.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 -- 172.21.15.70:0/4010042264 learned_addr learned my addr 172.21.15.70:0/4010042264 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 -- 172.21.15.70:0/4010042264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 msgr2=0x7fddb41f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb41f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 -- 172.21.15.70:0/4010042264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddb40b0ec0 msgr2=0x7fddb40b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddb40b0ec0 0x7fddb40b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 -- 172.21.15.70:0/4010042264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda40094d0 con 0x7fddb40ce480 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.941+0000 7fddb27fc640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 0x7fddb414c0c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00c800 tx=0x7fdd9c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:59.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.942+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c003b70 con 0x7fddb40ce480 2023-08-25T09:47:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.942+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c003cd0 con 0x7fddb40ce480 2023-08-25T09:47:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.942+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00aa90 con 0x7fddb40ce480 2023-08-25T09:47:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.942+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb41f2120 con 0x7fddb40ce480 2023-08-25T09:47:59.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.942+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb414cb30 con 0x7fddb40ce480 2023-08-25T09:47:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.943+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb404e420 con 0x7fddb40ce480 2023-08-25T09:47:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.944+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd9c00abf0 con 0x7fddb40ce480 2023-08-25T09:47:59.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.945+0000 7fddab7fe640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd9806a050 0x7fdd9806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:47:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.945+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdd9c0881f0 con 0x7fddb40ce480 2023-08-25T09:47:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.945+0000 7fddb2ffd640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd9806a050 0x7fdd9806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:47:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.946+0000 7fddb2ffd640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd9806a050 0x7fdd9806c510 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdda4036000 tx=0x7fdda403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:47:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:47:59.949+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdd9c053bf0 con 0x7fddb40ce480 2023-08-25T09:48:00.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.121+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fddb40cb460 con 0x7fddb40ce480 2023-08-25T09:48:00.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.122+0000 7fddab7fe640 1 -- 172.21.15.70:0/4010042264 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+14796 (secure 0 0 0) 0x7fdd9c057210 con 0x7fddb40ce480 2023-08-25T09:48:00.124 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:00.125 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":157,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:47:57.671567+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6834/1074652443":"2023-08-26T09:47:54.451254+0000","172.21.15.155:6832/1074652443":"2023-08-26T09:47:54.451254+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:48:00.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd9806a050 msgr2=0x7fdd9806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd9806a050 0x7fdd9806c510 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdda4036000 tx=0x7fdda403b040 comp rx=0 tx=0).stop 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 msgr2=0x7fddb414c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 0x7fddb414c0c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00c800 tx=0x7fdd9c00ccc0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 shutdown_connections 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd9806a050 0x7fdd9806c510 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fddb415d6b0 0x7fddb41f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fddb40ce480 0x7fddb414c0c0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 --2- 172.21.15.70:0/4010042264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fddb40b0ec0 0x7fddb40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 >> 172.21.15.70:0/4010042264 conn(0x7fddb4069c70 msgr2=0x7fddb406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:00.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 shutdown_connections 2023-08-25T09:48:00.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.125+0000 7fddb97a4640 1 -- 172.21.15.70:0/4010042264 wait complete. 2023-08-25T09:48:00.143 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:48:00.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:48:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.347+0000 7f2f4fba4640 1 Processor -- start 2023-08-25T09:48:00.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.347+0000 7f2f4fba4640 1 -- start start 2023-08-25T09:48:00.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4fba4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f48069850 0x7f2f4815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4fba4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2f480c9620 0x7f2f48069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4fba4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2f48160be0 0x7f2f48161040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4fba4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480ac880 con 0x7f2f48160be0 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4fba4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480ad400 con 0x7f2f480c9620 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4fba4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480ad890 con 0x7f2f48069850 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4e141640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2f48160be0 0x7f2f48161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4e141640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2f48160be0 0x7f2f48161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39238/0 (socket says 172.21.15.70:39238) 2023-08-25T09:48:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.349+0000 7f2f4e141640 1 -- 172.21.15.70:0/3445454224 learned_addr learned my addr 172.21.15.70:0/3445454224 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4d940640 1 --2- 172.21.15.70:0/3445454224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f48069850 0x7f2f4815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4d13f640 1 --2- 172.21.15.70:0/3445454224 >> v2:172.21.15.155:3300/0 conn(0x7f2f480c9620 0x7f2f48069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4e141640 1 -- 172.21.15.70:0/3445454224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f48069850 msgr2=0x7f2f4815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4e141640 1 --2- 172.21.15.70:0/3445454224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f48069850 0x7f2f4815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4e141640 1 -- 172.21.15.70:0/3445454224 >> v2:172.21.15.155:3300/0 conn(0x7f2f480c9620 msgr2=0x7f2f48069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4e141640 1 --2- 172.21.15.70:0/3445454224 >> v2:172.21.15.155:3300/0 conn(0x7f2f480c9620 0x7f2f48069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.350+0000 7f2f4e141640 1 -- 172.21.15.70:0/3445454224 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480acad0 con 0x7f2f48160be0 2023-08-25T09:48:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.351+0000 7f2f4e141640 1 --2- 172.21.15.70:0/3445454224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48160be0 0x7f2f48161040 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2f48071d50 tx=0x7f2f4002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c47daf551d346e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.351+0000 7f2f4c93e640 1 -- 172.21.15.70:0/3445454224 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4002cb00 con 0x7f2f48160be0 2023-08-25T09:48:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.351+0000 7f2f4c93e640 1 -- 172.21.15.70:0/3445454224 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4002cc60 con 0x7f2f48160be0 2023-08-25T09:48:00.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.351+0000 7f2f4c93e640 1 -- 172.21.15.70:0/3445454224 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400343f0 con 0x7f2f48160be0 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 -- 172.21.15.70:0/3445454224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48160be0 msgr2=0x7f2f48161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/3445454224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48160be0 0x7f2f48161040 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f2f48071d50 tx=0x7f2f4002bf60 comp rx=0 tx=0).stop 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 -- 172.21.15.70:0/3445454224 shutdown_connections 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/3445454224 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48160be0 0x7f2f48161040 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/3445454224 >> v2:172.21.15.155:3300/0 conn(0x7f2f480c9620 0x7f2f48069c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/3445454224 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f48069850 0x7f2f4815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 -- 172.21.15.70:0/3445454224 >> 172.21.15.70:0/3445454224 conn(0x7f2f480cb0d0 msgr2=0x7f2f480cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 -- 172.21.15.70:0/3445454224 shutdown_connections 2023-08-25T09:48:00.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.352+0000 7f2f4fba4640 1 -- 172.21.15.70:0/3445454224 wait complete. 2023-08-25T09:48:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.354+0000 7f2f4fba4640 1 Processor -- start 2023-08-25T09:48:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.354+0000 7f2f4fba4640 1 -- start start 2023-08-25T09:48:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4fba4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 0x7f2f48149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4fba4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f480c9620 0x7f2f481f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4fba4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f48160be0 0x7f2f481f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4fba4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b0d90 con 0x7f2f48069850 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4fba4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f48149aa0 con 0x7f2f48160be0 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4fba4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b0180 con 0x7f2f480c9620 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4d940640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 0x7f2f48149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4d940640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 0x7f2f48149560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39254/0 (socket says 172.21.15.70:39254) 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.355+0000 7f2f4d940640 1 -- 172.21.15.70:0/1646995886 learned_addr learned my addr 172.21.15.70:0/1646995886 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d940640 1 -- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f480c9620 msgr2=0x7f2f481f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d13f640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f480c9620 0x7f2f481f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d940640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f480c9620 0x7f2f481f6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d940640 1 -- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f48160be0 msgr2=0x7f2f481f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d940640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f48160be0 0x7f2f481f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d940640 1 -- 172.21.15.70:0/1646995886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009b30 con 0x7f2f48069850 2023-08-25T09:48:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4d940640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 0x7f2f48149560 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400d830 tx=0x7f2f4400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400be10 con 0x7f2f48069850 2023-08-25T09:48:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440098b0 con 0x7f2f48069850 2023-08-25T09:48:00.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44009a40 con 0x7f2f48069850 2023-08-25T09:48:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.356+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f480b0460 con 0x7f2f48069850 2023-08-25T09:48:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.357+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f480b0a20 con 0x7f2f48069850 2023-08-25T09:48:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.358+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48070fe0 con 0x7f2f48069850 2023-08-25T09:48:00.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.360+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f44009ba0 con 0x7f2f48069850 2023-08-25T09:48:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.361+0000 7f2f3e7fc640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f2c06a050 0x7f2f2c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.361+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2f44088a70 con 0x7f2f48069850 2023-08-25T09:48:00.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.361+0000 7f2f4d13f640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f2c06a050 0x7f2f2c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.362+0000 7f2f4d13f640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f2c06a050 0x7f2f2c06c510 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.364+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f44054460 con 0x7f2f48069850 2023-08-25T09:48:00.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.553+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f48066fb0 con 0x7f2f48069850 2023-08-25T09:48:00.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.554+0000 7f2f3e7fc640 1 -- 172.21.15.70:0/1646995886 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f44018020 con 0x7f2f48069850 2023-08-25T09:48:00.555 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:48:00.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.556+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f2c06a050 msgr2=0x7f2f2c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.556+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f2c06a050 0x7f2f2c06c510 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38008040 comp rx=0 tx=0).stop 2023-08-25T09:48:00.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.556+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 msgr2=0x7f2f48149560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.556+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 0x7f2f48149560 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400d830 tx=0x7f2f4400dcf0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 shutdown_connections 2023-08-25T09:48:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2f2c06a050 0x7f2f2c06c510 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2f48160be0 0x7f2f481f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2f480c9620 0x7f2f481f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 --2- 172.21.15.70:0/1646995886 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2f48069850 0x7f2f48149560 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 >> 172.21.15.70:0/1646995886 conn(0x7f2f480cb0d0 msgr2=0x7f2f480cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:00.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 shutdown_connections 2023-08-25T09:48:00.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.557+0000 7f2f4fba4640 1 -- 172.21.15.70:0/1646995886 wait complete. 2023-08-25T09:48:00.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:48:00.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.784+0000 7f5b2e606640 1 Processor -- start 2023-08-25T09:48:00.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.784+0000 7f5b2e606640 1 -- start start 2023-08-25T09:48:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.785+0000 7f5b2e606640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b2815ae50 0x7f5b280c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.785+0000 7f5b2e606640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5b28069f60 0x7f5b280ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.785+0000 7f5b2e606640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b280cc5f0 con 0x7f5b280cdc20 2023-08-25T09:48:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b2e606640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b28070650 con 0x7f5b2815ae50 2023-08-25T09:48:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b2e606640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b28070930 con 0x7f5b28069f60 2023-08-25T09:48:00.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b2815ae50 0x7f5b280c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5b2815ae50 0x7f5b280c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39258/0 (socket says 172.21.15.70:39258) 2023-08-25T09:48:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 -- 172.21.15.70:0/1567893305 learned_addr learned my addr 172.21.15.70:0/1567893305 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 -- 172.21.15.70:0/1567893305 >> v2:172.21.15.155:3300/0 conn(0x7f5b28069f60 msgr2=0x7f5b280ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 --2- 172.21.15.70:0/1567893305 >> v2:172.21.15.155:3300/0 conn(0x7f5b28069f60 0x7f5b280ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 -- 172.21.15.70:0/1567893305 >> v1:172.21.15.70:6789/0 conn(0x7f5b280cdc20 legacy=0x7f5b28069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.786+0000 7f5b27fff640 1 -- 172.21.15.70:0/1567893305 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28070c10 con 0x7f5b2815ae50 2023-08-25T09:48:00.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.787+0000 7f5b27fff640 1 --2- 172.21.15.70:0/1567893305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b2815ae50 0x7f5b280c2110 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f5b280b00d0 tx=0x7f5b1802ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5fe5b4d045ce2b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.787+0000 7f5b277fe640 1 -- 172.21.15.70:0/1567893305 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1802f870 con 0x7f5b2815ae50 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b277fe640 1 -- 172.21.15.70:0/1567893305 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1802f9d0 con 0x7f5b2815ae50 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b277fe640 1 -- 172.21.15.70:0/1567893305 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1802fd60 con 0x7f5b2815ae50 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 -- 172.21.15.70:0/1567893305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b2815ae50 msgr2=0x7f5b280c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 --2- 172.21.15.70:0/1567893305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b2815ae50 0x7f5b280c2110 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f5b280b00d0 tx=0x7f5b1802ba50 comp rx=0 tx=0).stop 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 -- 172.21.15.70:0/1567893305 shutdown_connections 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 --2- 172.21.15.70:0/1567893305 >> v2:172.21.15.155:3300/0 conn(0x7f5b28069f60 0x7f5b280ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 --2- 172.21.15.70:0/1567893305 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b2815ae50 0x7f5b280c2110 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 -- 172.21.15.70:0/1567893305 >> 172.21.15.70:0/1567893305 conn(0x7f5b280cb0d0 msgr2=0x7f5b280cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 -- 172.21.15.70:0/1567893305 shutdown_connections 2023-08-25T09:48:00.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.788+0000 7f5b2e606640 1 -- 172.21.15.70:0/1567893305 wait complete. 2023-08-25T09:48:00.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.790+0000 7f5b2e606640 1 Processor -- start 2023-08-25T09:48:00.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.790+0000 7f5b2e606640 1 -- start start 2023-08-25T09:48:00.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b2e606640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 0x7f5b281f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b2e606640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b280cdc20 0x7f5b280c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b2e606640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b2815ae50 0x7f5b280acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b2e606640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2814b810 con 0x7f5b28069f60 2023-08-25T09:48:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b2e606640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2814a600 con 0x7f5b2815ae50 2023-08-25T09:48:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b2e606640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b281f6560 con 0x7f5b280cdc20 2023-08-25T09:48:00.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 0x7f5b281f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b27fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 0x7f5b281f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39260/0 (socket says 172.21.15.70:39260) 2023-08-25T09:48:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.791+0000 7f5b27fff640 1 -- 172.21.15.70:0/3345523262 learned_addr learned my addr 172.21.15.70:0/3345523262 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b27fff640 1 -- 172.21.15.70:0/3345523262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b280cdc20 msgr2=0x7f5b280c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b27fff640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b280cdc20 0x7f5b280c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b27fff640 1 -- 172.21.15.70:0/3345523262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b2815ae50 msgr2=0x7f5b280acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b27fff640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b2815ae50 0x7f5b280acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b27fff640 1 -- 172.21.15.70:0/3345523262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b180094d0 con 0x7f5b28069f60 2023-08-25T09:48:00.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b27fff640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 0x7f5b281f6020 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.792+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1802f800 con 0x7f5b28069f60 2023-08-25T09:48:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.793+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18041400 con 0x7f5b28069f60 2023-08-25T09:48:00.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.793+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2814ac00 con 0x7f5b28069f60 2023-08-25T09:48:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.793+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18041790 con 0x7f5b28069f60 2023-08-25T09:48:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.793+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2814b1c0 con 0x7f5b28069f60 2023-08-25T09:48:00.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.794+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b28066fb0 con 0x7f5b28069f60 2023-08-25T09:48:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.795+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5b180418f0 con 0x7f5b28069f60 2023-08-25T09:48:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.796+0000 7f5b257fa640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b0806a050 0x7f5b0806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:00.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.797+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5b1803c070 con 0x7f5b28069f60 2023-08-25T09:48:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.797+0000 7f5b1f7fe640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b0806a050 0x7f5b0806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.797+0000 7f5b1f7fe640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b0806a050 0x7f5b0806c510 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b100012e0 tx=0x7f5b10007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:00.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.801+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5b18076c20 con 0x7f5b28069f60 2023-08-25T09:48:00.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.988+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5b280008d0 con 0x7f5b28069f60 2023-08-25T09:48:00.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.989+0000 7f5b257fa640 1 -- 172.21.15.70:0/3345523262 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v157) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5b1807a240 con 0x7f5b28069f60 2023-08-25T09:48:00.991 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:48:00.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.991+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b0806a050 msgr2=0x7f5b0806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.991+0000 7f5b2e606640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b0806a050 0x7f5b0806c510 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b100012e0 tx=0x7f5b10007d30 comp rx=0 tx=0).stop 2023-08-25T09:48:00.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.991+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 msgr2=0x7f5b281f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:00.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.991+0000 7f5b2e606640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 0x7f5b281f6020 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5b18000c00 tx=0x7f5b18004730 comp rx=0 tx=0).stop 2023-08-25T09:48:00.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 shutdown_connections 2023-08-25T09:48:00.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b0806a050 0x7f5b0806c510 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5b2815ae50 0x7f5b280acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5b280cdc20 0x7f5b280c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 --2- 172.21.15.70:0/3345523262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5b28069f60 0x7f5b281f6020 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:00.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 >> 172.21.15.70:0/3345523262 conn(0x7f5b280cb0d0 msgr2=0x7f5b280c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:00.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 shutdown_connections 2023-08-25T09:48:00.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:00.992+0000 7f5b2e606640 1 -- 172.21.15.70:0/3345523262 wait complete. 2023-08-25T09:48:01.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-08-25T09:48:01.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.215+0000 7f875eaa7640 1 Processor -- start 2023-08-25T09:48:01.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.215+0000 7f875eaa7640 1 -- start start 2023-08-25T09:48:01.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.217+0000 7f875eaa7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f875806bc80 0x7f875806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.217+0000 7f875eaa7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875806aec0 con 0x7f8758159aa0 2023-08-25T09:48:01.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.217+0000 7f875eaa7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875806b1a0 con 0x7f875815df10 2023-08-25T09:48:01.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.217+0000 7f875eaa7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8758159e80 con 0x7f875806bc80 2023-08-25T09:48:01.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.217+0000 7f875c843640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8758159aa0 0x7f8758070410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42892/0 (socket says 172.21.15.70:42892) 2023-08-25T09:48:01.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.217+0000 7f875c843640 1 -- 172.21.15.70:0/3354689968 learned_addr learned my addr 172.21.15.70:0/3354689968 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:01.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753209281 0 0) 0x7f875806aec0 con 0x7f8758159aa0 2023-08-25T09:48:01.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8740005730 con 0x7f8758159aa0 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f8757fff640 1 --2- 172.21.15.70:0/3354689968 >> v2:172.21.15.155:3300/0 conn(0x7f875806bc80 0x7f875806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1431803145 0 0) 0x7f8740005730 con 0x7f8758159aa0 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 >> v2:172.21.15.155:3300/0 conn(0x7f875806bc80 msgr2=0x7f875806ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 --2- 172.21.15.70:0/3354689968 >> v2:172.21.15.155:3300/0 conn(0x7f875806bc80 0x7f875806ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 >> v1:172.21.15.155:6789/0 conn(0x7f875815df10 legacy=0x7f875806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875806b480 con 0x7f8758159aa0 2023-08-25T09:48:01.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8748001810 con 0x7f8758159aa0 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 >> v1:172.21.15.70:6789/0 conn(0x7f8758159aa0 legacy=0x7f8758070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8740005b80 0x7f8740005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87400069d0 0x7f8740006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740007390 0x7f874000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8758159e80 con 0x7f8740007390 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874000e2e0 con 0x7f8740005b80 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.218+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874000e4a0 con 0x7f87400069d0 2023-08-25T09:48:01.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875d044640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740007390 0x7f874000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875c843640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8740005b80 0x7f8740005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875d044640 1 -- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87400069d0 msgr2=0x7f8740006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:01.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875d044640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f87400069d0 0x7f8740006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875d044640 1 -- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8740005b80 msgr2=0x7f8740005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:01.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875d044640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8740005b80 0x7f8740005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875d044640 1 -- 172.21.15.70:0/3354689968 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740005730 con 0x7f8740007390 2023-08-25T09:48:01.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.219+0000 7f875c843640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8740005b80 0x7f8740005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:48:01.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.220+0000 7f875d044640 1 -- 172.21.15.70:0/3354689968 reap_dead start 2023-08-25T09:48:01.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.220+0000 7f875d044640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740007390 0x7f874000db90 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f8750001e50 tx=0x7f8740007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b4b213255d3939 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:01.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750003b70 con 0x7f8740007390 2023-08-25T09:48:01.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750003cf0 con 0x7f8740007390 2023-08-25T09:48:01.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f87577fe640 1 -- 172.21.15.70:0/3354689968 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750005310 con 0x7f8740007390 2023-08-25T09:48:01.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 -- 172.21.15.70:0/3354689968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740007390 msgr2=0x7f874000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740007390 0x7f874000db90 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f8750001e50 tx=0x7f8740007a00 comp rx=0 tx=0).stop 2023-08-25T09:48:01.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 -- 172.21.15.70:0/3354689968 shutdown_connections 2023-08-25T09:48:01.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 --2- 172.21.15.70:0/3354689968 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8740007390 0x7f874000db90 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 -- 172.21.15.70:0/3354689968 >> 172.21.15.70:0/3354689968 conn(0x7f87580ce760 msgr2=0x7f87580ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:01.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 -- 172.21.15.70:0/3354689968 shutdown_connections 2023-08-25T09:48:01.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.221+0000 7f875eaa7640 1 -- 172.21.15.70:0/3354689968 wait complete. 2023-08-25T09:48:01.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.223+0000 7f875eaa7640 1 Processor -- start 2023-08-25T09:48:01.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.223+0000 7f875eaa7640 1 -- start start 2023-08-25T09:48:01.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875eaa7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8740007390 0x7f87581f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875eaa7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87580a8d50 0x7f87580ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875eaa7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 0x7f875814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875eaa7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875814d8c0 con 0x7f875814c980 2023-08-25T09:48:01.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875eaa7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581f3060 con 0x7f87580a8d50 2023-08-25T09:48:01.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875eaa7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87581f5670 con 0x7f8740007390 2023-08-25T09:48:01.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.224+0000 7f875d044640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 0x7f875814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 0x7f875814ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39270/0 (socket says 172.21.15.70:39270) 2023-08-25T09:48:01.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 -- 172.21.15.70:0/4012078722 learned_addr learned my addr 172.21.15.70:0/4012078722 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875c843640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8740007390 0x7f87581f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f8757fff640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87580a8d50 0x7f87580ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 -- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8740007390 msgr2=0x7f87581f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8740007390 0x7f87581f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 -- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87580a8d50 msgr2=0x7f87580ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87580a8d50 0x7f87580ca180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 -- 172.21.15.70:0/4012078722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875814d340 con 0x7f875814c980 2023-08-25T09:48:01.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.225+0000 7f875d044640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 0x7f875814ce00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8750001f70 tx=0x7f87500014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:01.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.226+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750006d50 con 0x7f875814c980 2023-08-25T09:48:01.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.226+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875003d070 con 0x7f875814c980 2023-08-25T09:48:01.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.226+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8750007230 con 0x7f875814c980 2023-08-25T09:48:01.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.226+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87580a92c0 con 0x7f875814c980 2023-08-25T09:48:01.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.226+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87581f5ba0 con 0x7f875814c980 2023-08-25T09:48:01.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.227+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f875804e420 con 0x7f875814c980 2023-08-25T09:48:01.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.228+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8750007390 con 0x7f875814c980 2023-08-25T09:48:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.230+0000 7f87557fa640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f874406a080 0x7f874406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.230+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f87500a9f90 con 0x7f875814c980 2023-08-25T09:48:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.230+0000 7f875c843640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f874406a080 0x7f874406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.231+0000 7f875c843640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f874406a080 0x7f874406c540 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f87480010e0 tx=0x7f8748000f70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:01.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.235+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8750075900 con 0x7f875814c980 2023-08-25T09:48:01.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.484+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f87580cf670 con 0x7f875814c980 2023-08-25T09:48:01.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.726+0000 7f87557fa640 1 -- 172.21.15.70:0/4012078722 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v488) v1 ==== 135+0+0 (secure 0 0 0) 0x7f8750003e60 con 0x7f875814c980 2023-08-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f874406a080 msgr2=0x7f874406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f874406a080 0x7f874406c540 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f87480010e0 tx=0x7f8748000f70 comp rx=0 tx=0).stop 2023-08-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 msgr2=0x7f875814ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 0x7f875814ce00 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f8750001f70 tx=0x7f87500014a0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 shutdown_connections 2023-08-25T09:48:01.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f874406a080 0x7f874406c540 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f875814c980 0x7f875814ce00 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f87580a8d50 0x7f87580ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 --2- 172.21.15.70:0/4012078722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8740007390 0x7f87581f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 >> 172.21.15.70:0/4012078722 conn(0x7f87580ce760 msgr2=0x7f87580cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:01.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 shutdown_connections 2023-08-25T09:48:01.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.729+0000 7f875eaa7640 1 -- 172.21.15.70:0/4012078722 wait complete. 2023-08-25T09:48:01.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:48:01.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.970+0000 7f7160731640 1 Processor -- start 2023-08-25T09:48:01.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.970+0000 7f7160731640 1 -- start start 2023-08-25T09:48:01.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f7160731640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f7158069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f7160731640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7158069d90 0x7f71580ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f7160731640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71580c2550 con 0x7f715815ae50 2023-08-25T09:48:01.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f7160731640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f715806a220 con 0x7f7158069d90 2023-08-25T09:48:01.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f7160731640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71580ce330 con 0x7f71580c2110 2023-08-25T09:48:01.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f715ecce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7158069d90 0x7f71580ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f715ecce640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7158069d90 0x7f71580ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39282/0 (socket says 172.21.15.70:39282) 2023-08-25T09:48:01.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f715ecce640 1 -- 172.21.15.70:0/624017582 learned_addr learned my addr 172.21.15.70:0/624017582 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:01.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.972+0000 7f715dccc640 1 --2- 172.21.15.70:0/624017582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f7158069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.973+0000 7f715ecce640 1 -- 172.21.15.70:0/624017582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 msgr2=0x7f7158069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:01.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.973+0000 7f715ecce640 1 --2- 172.21.15.70:0/624017582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f7158069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.973+0000 7f715ecce640 1 -- 172.21.15.70:0/624017582 >> v1:172.21.15.155:6789/0 conn(0x7f715815ae50 legacy=0x7f71580cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:01.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.973+0000 7f715ecce640 1 -- 172.21.15.70:0/624017582 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71580cc5f0 con 0x7f7158069d90 2023-08-25T09:48:01.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.974+0000 7f715ecce640 1 --2- 172.21.15.70:0/624017582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580ccad0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7150009e50 tx=0x7f715002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b379d7cebaaa1420 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:01.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.974+0000 7f715d4cb640 1 -- 172.21.15.70:0/624017582 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715002cb00 con 0x7f7158069d90 2023-08-25T09:48:01.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.974+0000 7f715d4cb640 1 -- 172.21.15.70:0/624017582 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715002cc60 con 0x7f7158069d90 2023-08-25T09:48:01.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.974+0000 7f715d4cb640 1 -- 172.21.15.70:0/624017582 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71500343f0 con 0x7f7158069d90 2023-08-25T09:48:01.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 -- 172.21.15.70:0/624017582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 msgr2=0x7f71580ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 --2- 172.21.15.70:0/624017582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580ccad0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7150009e50 tx=0x7f715002bf60 comp rx=0 tx=0).stop 2023-08-25T09:48:01.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 -- 172.21.15.70:0/624017582 shutdown_connections 2023-08-25T09:48:01.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 --2- 172.21.15.70:0/624017582 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580ccad0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 --2- 172.21.15.70:0/624017582 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f7158069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 -- 172.21.15.70:0/624017582 >> 172.21.15.70:0/624017582 conn(0x7f71580cb0d0 msgr2=0x7f71580cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:01.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 -- 172.21.15.70:0/624017582 shutdown_connections 2023-08-25T09:48:01.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.975+0000 7f7160731640 1 -- 172.21.15.70:0/624017582 wait complete. 2023-08-25T09:48:01.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.977+0000 7f7160731640 1 Processor -- start 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.977+0000 7f7160731640 1 -- start start 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f7160731640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f7160731640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f71580c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f7160731640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f715815ae50 0x7f71580c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f7160731640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71581f0fd0 con 0x7f7158069d90 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f7160731640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71580b4040 con 0x7f715815ae50 2023-08-25T09:48:01.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f7160731640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71581f0450 con 0x7f71580c2110 2023-08-25T09:48:01.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f715e4cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f715e4cd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39288/0 (socket says 172.21.15.70:39288) 2023-08-25T09:48:01.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.978+0000 7f715e4cd640 1 -- 172.21.15.70:0/2796083138 learned_addr learned my addr 172.21.15.70:0/2796083138 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:01.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715ecce640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f715815ae50 0x7f71580c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715dccc640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f71580c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715e4cd640 1 -- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 msgr2=0x7f71580c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715e4cd640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f71580c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715e4cd640 1 -- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f715815ae50 msgr2=0x7f71580c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715e4cd640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f715815ae50 0x7f71580c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715e4cd640 1 -- 172.21.15.70:0/2796083138 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150009b30 con 0x7f7158069d90 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715ecce640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f715815ae50 0x7f71580c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f715e4cd640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580c67e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f715400c430 tx=0x7f715400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:01.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011790 con 0x7f7158069d90 2023-08-25T09:48:01.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71540118f0 con 0x7f7158069d90 2023-08-25T09:48:01.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011ce0 con 0x7f7158069d90 2023-08-25T09:48:01.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71581f0730 con 0x7f7158069d90 2023-08-25T09:48:01.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.979+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71581f0cf0 con 0x7f7158069d90 2023-08-25T09:48:01.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.980+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7158066fb0 con 0x7f7158069d90 2023-08-25T09:48:01.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.981+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f715401c460 con 0x7f7158069d90 2023-08-25T09:48:01.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.983+0000 7f714effd640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713c06a050 0x7f713c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:01.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.983+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f71540882e0 con 0x7f7158069d90 2023-08-25T09:48:01.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.983+0000 7f715dccc640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713c06a050 0x7f713c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:01.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.984+0000 7f715dccc640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713c06a050 0x7f713c06c510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f71580b13f0 tx=0x7f7148008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:01.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:01.987+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7154053c50 con 0x7f7158069d90 2023-08-25T09:48:02.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.176+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f71580cb9e0 con 0x7f7158069d90 2023-08-25T09:48:02.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.185+0000 7f714effd640 1 -- 172.21.15.70:0/2796083138 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7154057270 con 0x7f7158069d90 2023-08-25T09:48:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.187+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713c06a050 msgr2=0x7f713c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.187+0000 7f7160731640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713c06a050 0x7f713c06c510 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f71580b13f0 tx=0x7f7148008040 comp rx=0 tx=0).stop 2023-08-25T09:48:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.187+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 msgr2=0x7f71580c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.187+0000 7f7160731640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580c67e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f715400c430 tx=0x7f715400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 shutdown_connections 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f713c06a050 0x7f713c06c510 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f715815ae50 0x7f71580c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71580c2110 0x7f71580c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 --2- 172.21.15.70:0/2796083138 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7158069d90 0x7f71580c67e0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 >> 172.21.15.70:0/2796083138 conn(0x7f71580cb0d0 msgr2=0x7f71580c2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 shutdown_connections 2023-08-25T09:48:02.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.188+0000 7f7160731640 1 -- 172.21.15.70:0/2796083138 wait complete. 2023-08-25T09:48:02.209 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:48:02.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:48:02.323 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:48:02.448 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:48:02.449 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:48:02.449 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:48:02.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:48:02.510 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:48:02.511 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:48:02.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:48:02.571 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:48:02.572 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:48:02.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:48:02.631 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:48:02.632 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:48:02.694 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:48:02.694 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:48:02.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:48:02.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.919+0000 7f603ac21640 1 Processor -- start 2023-08-25T09:48:02.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.919+0000 7f603ac21640 1 -- start start 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.920+0000 7f603ac21640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60340c92c0 0x7f60340b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.920+0000 7f603ac21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.920+0000 7f603ac21640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f60340b4150 0x7f60340cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.920+0000 7f603ac21640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60341615b0 con 0x7f60340c92c0 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.920+0000 7f603ac21640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340cd330 con 0x7f60340b4150 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.920+0000 7f603ac21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340cb7c0 con 0x7f60340c96a0 2023-08-25T09:48:02.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60340c92c0 0x7f60340b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f60340c92c0 0x7f60340b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39296/0 (socket says 172.21.15.70:39296) 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 -- 172.21.15.70:0/3669979464 learned_addr learned my addr 172.21.15.70:0/3669979464 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 -- 172.21.15.70:0/3669979464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 msgr2=0x7f60340cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 --2- 172.21.15.70:0/3669979464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 -- 172.21.15.70:0/3669979464 >> v2:172.21.15.155:3300/0 conn(0x7f60340b4150 msgr2=0x7f60340cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 --2- 172.21.15.70:0/3669979464 >> v2:172.21.15.155:3300/0 conn(0x7f60340b4150 0x7f60340cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.921+0000 7f6039c1f640 1 -- 172.21.15.70:0/3669979464 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340b0e80 con 0x7f60340c92c0 2023-08-25T09:48:02.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.922+0000 7f6039c1f640 1 --2- 172.21.15.70:0/3669979464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b12f0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f60340b0b50 tx=0x7f603002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4e044238bf1abd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:02.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.922+0000 7f6038c1d640 1 -- 172.21.15.70:0/3669979464 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030031880 con 0x7f60340c92c0 2023-08-25T09:48:02.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.922+0000 7f6038c1d640 1 -- 172.21.15.70:0/3669979464 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60300319e0 con 0x7f60340c92c0 2023-08-25T09:48:02.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.922+0000 7f6038c1d640 1 -- 172.21.15.70:0/3669979464 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6030031d70 con 0x7f60340c92c0 2023-08-25T09:48:02.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 -- 172.21.15.70:0/3669979464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 msgr2=0x7f60340b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:02.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 --2- 172.21.15.70:0/3669979464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b12f0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f60340b0b50 tx=0x7f603002b920 comp rx=0 tx=0).stop 2023-08-25T09:48:02.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 -- 172.21.15.70:0/3669979464 shutdown_connections 2023-08-25T09:48:02.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 --2- 172.21.15.70:0/3669979464 >> v2:172.21.15.155:3300/0 conn(0x7f60340b4150 0x7f60340cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 --2- 172.21.15.70:0/3669979464 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 --2- 172.21.15.70:0/3669979464 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b12f0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 -- 172.21.15.70:0/3669979464 >> 172.21.15.70:0/3669979464 conn(0x7f60340affd0 msgr2=0x7f60340b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:02.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 -- 172.21.15.70:0/3669979464 shutdown_connections 2023-08-25T09:48:02.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.923+0000 7f603ac21640 1 -- 172.21.15.70:0/3669979464 wait complete. 2023-08-25T09:48:02.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.925+0000 7f603ac21640 1 Processor -- start 2023-08-25T09:48:02.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.926+0000 7f603ac21640 1 -- start start 2023-08-25T09:48:02.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603ac21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60340b4150 0x7f60341f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603ac21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603ac21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603ac21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6034150fa0 con 0x7f60340c92c0 2023-08-25T09:48:02.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603ac21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60341f6680 con 0x7f60340b4150 2023-08-25T09:48:02.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603ac21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60341fabf0 con 0x7f60340c96a0 2023-08-25T09:48:02.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603941e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:02.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603941e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39304/0 (socket says 172.21.15.70:39304) 2023-08-25T09:48:02.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603941e640 1 -- 172.21.15.70:0/510713609 learned_addr learned my addr 172.21.15.70:0/510713609 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:02.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f6039c1f640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60340b4150 0x7f60341f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:02.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.927+0000 7f603a420640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:02.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603941e640 1 -- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 msgr2=0x7f60340b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:02.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603941e640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603941e640 1 -- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60340b4150 msgr2=0x7f60341f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:02.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603941e640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60340b4150 0x7f60341f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:02.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603941e640 1 -- 172.21.15.70:0/510713609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300094d0 con 0x7f60340c92c0 2023-08-25T09:48:02.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603941e640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b10a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f602400b410 tx=0x7f602400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:02.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024011830 con 0x7f60340c92c0 2023-08-25T09:48:02.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60240119b0 con 0x7f60340c92c0 2023-08-25T09:48:02.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6024011d60 con 0x7f60340c92c0 2023-08-25T09:48:02.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60341508e0 con 0x7f60340c92c0 2023-08-25T09:48:02.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.928+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60341505a0 con 0x7f60340c92c0 2023-08-25T09:48:02.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.929+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6034066fb0 con 0x7f60340c92c0 2023-08-25T09:48:02.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.931+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f602401a400 con 0x7f60340c92c0 2023-08-25T09:48:02.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.932+0000 7f602a7fc640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601806a050 0x7f601806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:02.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.932+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f60240883e0 con 0x7f60340c92c0 2023-08-25T09:48:02.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.932+0000 7f6039c1f640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601806a050 0x7f601806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:02.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.933+0000 7f6039c1f640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601806a050 0x7f601806c510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6030036000 tx=0x7f603003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:02.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:02.938+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6024053de0 con 0x7f60340c92c0 2023-08-25T09:48:03.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.164+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60340c8aa0 con 0x7f60340c92c0 2023-08-25T09:48:03.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.165+0000 7f602a7fc640 1 -- 172.21.15.70:0/510713609 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6024015e90 con 0x7f60340c92c0 2023-08-25T09:48:03.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:03.168 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5893,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1983452128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1983452128},{"type":"v1","addr":"172.21.15.155:6835","nonce":1983452128}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":6025,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/1363491956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":1363491956},{"type":"v1","addr":"172.21.15.155:6834","nonce":1363491956}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/3922833473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":3922833473},{"type":"v1","addr":"172.21.15.70:6835","nonce":3922833473}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/2037179249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":2037179249},{"type":"v1","addr":"172.21.15.70:6837","nonce":2037179249}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2023-08-25T09:48:03.169 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 67 2023-08-25T09:48:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601806a050 msgr2=0x7f601806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601806a050 0x7f601806c510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6030036000 tx=0x7f603003c040 comp rx=0 tx=0).stop 2023-08-25T09:48:03.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 msgr2=0x7f60340b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b10a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f602400b410 tx=0x7f602400b8d0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 shutdown_connections 2023-08-25T09:48:03.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f601806a050 0x7f601806c510 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f60340c96a0 0x7f60340b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f60340c92c0 0x7f60340b10a0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 --2- 172.21.15.70:0/510713609 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f60340b4150 0x7f60341f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 >> 172.21.15.70:0/510713609 conn(0x7f60340affd0 msgr2=0x7f60340cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 shutdown_connections 2023-08-25T09:48:03.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.168+0000 7f603ac21640 1 -- 172.21.15.70:0/510713609 wait complete. 2023-08-25T09:48:03.191 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:48:03.192 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:48:03.192 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:48:03.192 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:48:03.193 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:48:03.193 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:48:03.194 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:48:03.194 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:48:03.195 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:48:03.195 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:48:03.198 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:48:03.199 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:48:03.199 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:48:03.200 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:48:03.200 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:48:03.201 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:48:03.210 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:48:03.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:48:03.218 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:48:03.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:48:03.222 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:48:03.223 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:48:03.231 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:48:03.232 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:48:03.265 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:48:03.266 DEBUG:teuthology.parallel:result is None 2023-08-25T09:48:03.273 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:48:03.274 DEBUG:teuthology.parallel:result is None 2023-08-25T09:48:03.279 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:48:03.280 DEBUG:teuthology.parallel:result is None 2023-08-25T09:48:03.288 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:48:03.288 DEBUG:teuthology.parallel:result is None 2023-08-25T09:48:03.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:48:03.427 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:48:03.429 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:48:03.457 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:48:03.467 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:48:03.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.479+0000 7fe6f18ad640 1 Processor -- start 2023-08-25T09:48:03.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.479+0000 7fe6f18ad640 1 -- start start 2023-08-25T09:48:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.480+0000 7fe6f18ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.480+0000 7fe6f18ad640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe6ec0b13f0 0x7fe6ec0ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.480+0000 7fe6f18ad640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6ec0cd0a0 con 0x7fe6ec0cabd0 2023-08-25T09:48:03.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.480+0000 7fe6f18ad640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6ec0ae6d0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.480+0000 7fe6f18ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec0ce4c0 con 0x7fe6ec06c020 2023-08-25T09:48:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eb7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe6ec0b13f0 0x7fe6ec0ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eaffd640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe6ec0cabd0 0x7fe6ec0b0ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:48026/0 (socket says 172.21.15.70:48026) 2023-08-25T09:48:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eb7fe640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe6ec0b13f0 0x7fe6ec0ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:55570/0 (socket says 172.21.15.70:55570) 2023-08-25T09:48:03.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eaffd640 1 -- 172.21.15.70:0/1445418817 learned_addr learned my addr 172.21.15.70:0/1445418817 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:03.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6ea7fc640 1 --2- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eb7fe640 1 -- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 msgr2=0x7fe6ec0ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:03.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eb7fe640 1 --2- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec0ce9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eb7fe640 1 -- 172.21.15.70:0/1445418817 >> v1:172.21.15.155:6789/0 conn(0x7fe6ec0cabd0 legacy=0x7fe6ec0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6eb7fe640 1 -- 172.21.15.70:0/1445418817 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec161520 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.481+0000 7fe6e9ffb640 1 -- 172.21.15.70:0/1445418817 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769347680 0 0) 0x7fe6dc001110 con 0x7fe6ec0cabd0 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.482+0000 7fe6eb7fe640 1 --2- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0ccb60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009d90 tx=0x7fe6e402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7326b79019217dfc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6e9ffb640 1 -- 172.21.15.70:0/1445418817 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e402cc50 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6e9ffb640 1 -- 172.21.15.70:0/1445418817 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e402cdb0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6e9ffb640 1 -- 172.21.15.70:0/1445418817 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4034660 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 -- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0b13f0 msgr2=0x7fe6ec0ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0ccb60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009d90 tx=0x7fe6e402bea0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 -- 172.21.15.70:0/1445418817 shutdown_connections 2023-08-25T09:48:03.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0ccb60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/1445418817 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec0ce9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 -- 172.21.15.70:0/1445418817 >> 172.21.15.70:0/1445418817 conn(0x7fe6ec069a60 msgr2=0x7fe6ec069e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:03.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 -- 172.21.15.70:0/1445418817 shutdown_connections 2023-08-25T09:48:03.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.483+0000 7fe6f18ad640 1 -- 172.21.15.70:0/1445418817 wait complete. 2023-08-25T09:48:03.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.485+0000 7fe6f18ad640 1 Processor -- start 2023-08-25T09:48:03.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.485+0000 7fe6f18ad640 1 -- start start 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6f18ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec15a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6f18ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6f18ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0cabd0 0x7fe6ec15f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6f18ad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec15ab90 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6f18ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec15e890 con 0x7fe6ec0cabd0 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6f18ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec15eb70 con 0x7fe6ec06c020 2023-08-25T09:48:03.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6ea7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6ea7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39340/0 (socket says 172.21.15.70:39340) 2023-08-25T09:48:03.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.486+0000 7fe6ea7fc640 1 -- 172.21.15.70:0/3038341746 learned_addr learned my addr 172.21.15.70:0/3038341746 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:03.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6eaffd640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec15a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6eb7fe640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0cabd0 0x7fe6ec15f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6ea7fc640 1 -- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 msgr2=0x7fe6ec15a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6ea7fc640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec15a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6ea7fc640 1 -- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0cabd0 msgr2=0x7fe6ec15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6ea7fc640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0cabd0 0x7fe6ec15f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6ea7fc640 1 -- 172.21.15.70:0/3038341746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e4009a70 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.487+0000 7fe6ea7fc640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0b1b90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000e870 tx=0x7fe6e000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:03.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.488+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e00094f0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.488+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e000c940 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.488+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000ccf0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.488+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ec15ee50 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.488+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ec15f410 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.489+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ec04e420 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.490+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe6e0013470 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.491+0000 7fe6d3fff640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6d806a050 0x7fe6d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.491+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe6e00886a0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.491+0000 7fe6eaffd640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6d806a050 0x7fe6d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.492+0000 7fe6eaffd640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6d806a050 0x7fe6d806c510 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc000f60 tx=0x7fe6dc007b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:03.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.495+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6e00540c0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.698+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe6ec06bcb0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.751+0000 7fe6d3fff640 1 -- 172.21.15.70:0/3038341746 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v159) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe6e00576e0 con 0x7fe6ec0b13f0 2023-08-25T09:48:03.754 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:48:03.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6d806a050 msgr2=0x7fe6d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6d806a050 0x7fe6d806c510 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc000f60 tx=0x7fe6dc007b90 comp rx=0 tx=0).stop 2023-08-25T09:48:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 msgr2=0x7fe6ec0b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0b1b90 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000e870 tx=0x7fe6e000ed30 comp rx=0 tx=0).stop 2023-08-25T09:48:03.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 shutdown_connections 2023-08-25T09:48:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6d806a050 0x7fe6d806c510 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec0cabd0 0x7fe6ec15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec0b13f0 0x7fe6ec0b1b90 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 --2- 172.21.15.70:0/3038341746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec06c020 0x7fe6ec15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.754+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 >> 172.21.15.70:0/3038341746 conn(0x7fe6ec069a60 msgr2=0x7fe6ec0b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:03.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.755+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 shutdown_connections 2023-08-25T09:48:03.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.755+0000 7fe6f18ad640 1 -- 172.21.15.70:0/3038341746 wait complete. 2023-08-25T09:48:03.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:48:03.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.983+0000 7f2c8c4b4640 1 Processor -- start 2023-08-25T09:48:03.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.983+0000 7f2c8c4b4640 1 -- start start 2023-08-25T09:48:03.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.984+0000 7f2c8c4b4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c8415e830 0x7f2c8406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.984+0000 7f2c8c4b4640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2c8415ec10 0x7f2c840b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.984+0000 7f2c8c4b4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c840b1b60 con 0x7f2c8406b2b0 2023-08-25T09:48:03.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.984+0000 7f2c8c4b4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8406bd80 con 0x7f2c8415e830 2023-08-25T09:48:03.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.984+0000 7f2c8c4b4640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8406b7c0 con 0x7f2c8415ec10 2023-08-25T09:48:03.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c8415e830 0x7f2c8406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c8415e830 0x7f2c8406ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39342/0 (socket says 172.21.15.70:39342) 2023-08-25T09:48:03.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 -- 172.21.15.70:0/2887953781 learned_addr learned my addr 172.21.15.70:0/2887953781 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:03.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8aa51640 1 --2- 172.21.15.70:0/2887953781 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415ec10 0x7f2c840b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 -- 172.21.15.70:0/2887953781 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415ec10 msgr2=0x7f2c840b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:03.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 --2- 172.21.15.70:0/2887953781 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415ec10 0x7f2c840b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 -- 172.21.15.70:0/2887953781 >> v1:172.21.15.155:6789/0 conn(0x7f2c8406b2b0 legacy=0x7f2c8415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:03.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8a250640 1 -- 172.21.15.70:0/2887953781 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8406baa0 con 0x7f2c8415e830 2023-08-25T09:48:03.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.985+0000 7f2c8aa51640 1 --2- 172.21.15.70:0/2887953781 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415ec10 0x7f2c840b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:03.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.986+0000 7f2c8a250640 1 --2- 172.21.15.70:0/2887953781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c8406ad70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2c8406a710 tx=0x7f2c8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee4c6181f803e56 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:03.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.987+0000 7f2c8924e640 1 -- 172.21.15.70:0/2887953781 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80031870 con 0x7f2c8415e830 2023-08-25T09:48:03.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.987+0000 7f2c8924e640 1 -- 172.21.15.70:0/2887953781 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c800319d0 con 0x7f2c8415e830 2023-08-25T09:48:03.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.987+0000 7f2c8924e640 1 -- 172.21.15.70:0/2887953781 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c80031d60 con 0x7f2c8415e830 2023-08-25T09:48:03.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.987+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/2887953781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 msgr2=0x7f2c8406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.987+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/2887953781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c8406ad70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2c8406a710 tx=0x7f2c8002b910 comp rx=0 tx=0).stop 2023-08-25T09:48:03.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.988+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/2887953781 shutdown_connections 2023-08-25T09:48:03.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.988+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/2887953781 >> v2:172.21.15.155:3300/0 conn(0x7f2c8415ec10 0x7f2c840b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.988+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/2887953781 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c8406ad70 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.988+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/2887953781 >> 172.21.15.70:0/2887953781 conn(0x7f2c84069850 msgr2=0x7f2c84069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:03.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.988+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/2887953781 shutdown_connections 2023-08-25T09:48:03.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.988+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/2887953781 wait complete. 2023-08-25T09:48:03.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.989+0000 7f2c8c4b4640 1 Processor -- start 2023-08-25T09:48:03.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.990+0000 7f2c8c4b4640 1 -- start start 2023-08-25T09:48:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.990+0000 7f2c8c4b4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8406b2b0 0x7f2c840ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.990+0000 7f2c8c4b4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c840a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c8c4b4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415ec10 0x7f2c840ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c8c4b4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840acba0 con 0x7f2c8415e830 2023-08-25T09:48:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c8c4b4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840a9f80 con 0x7f2c8406b2b0 2023-08-25T09:48:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c8c4b4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840a8ee0 con 0x7f2c8415ec10 2023-08-25T09:48:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c840a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:03.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c840a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39348/0 (socket says 172.21.15.70:39348) 2023-08-25T09:48:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 -- 172.21.15.70:0/4101808588 learned_addr learned my addr 172.21.15.70:0/4101808588 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 -- 172.21.15.70:0/4101808588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415ec10 msgr2=0x7f2c840ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415ec10 0x7f2c840ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 -- 172.21.15.70:0/4101808588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8406b2b0 msgr2=0x7f2c840ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8406b2b0 0x7f2c840ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.991+0000 7f2c89a4f640 1 -- 172.21.15.70:0/4101808588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c800094d0 con 0x7f2c8415e830 2023-08-25T09:48:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.992+0000 7f2c89a4f640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c840a8460 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c430 tx=0x7f2c7400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.992+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74011790 con 0x7f2c8415e830 2023-08-25T09:48:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.992+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c740118f0 con 0x7f2c8415e830 2023-08-25T09:48:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.992+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c74011ce0 con 0x7f2c8415e830 2023-08-25T09:48:03.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.992+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8415ca20 con 0x7f2c8415e830 2023-08-25T09:48:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.992+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c840a94a0 con 0x7f2c8415e830 2023-08-25T09:48:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.993+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8404e420 con 0x7f2c8415e830 2023-08-25T09:48:04.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.994+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2c7401c460 con 0x7f2c8415e830 2023-08-25T09:48:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.995+0000 7f2c7affd640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a050 0x7f2c6806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.995+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2c740882b0 con 0x7f2c8415e830 2023-08-25T09:48:04.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.995+0000 7f2c8a250640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a050 0x7f2c6806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.996+0000 7f2c8a250640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a050 0x7f2c6806c510 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2c80036000 tx=0x7f2c80030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:04.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:03.999+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c74053c50 con 0x7f2c8415e830 2023-08-25T09:48:04.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.254+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2c8406c620 con 0x7f2c8415e830 2023-08-25T09:48:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.256+0000 7f2c7affd640 1 -- 172.21.15.70:0/4101808588 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2c74057270 con 0x7f2c8415e830 2023-08-25T09:48:04.260 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:04.261 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:48:04.261 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:48:04.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.262 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:48:04.262 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:48:04.262 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.262 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.263 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:48:04.263 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:48:04.263 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.263 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.264 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:48:04.264 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:48:04.265 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.265 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.265 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:48:04.265 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:48:04.265 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.267 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:48:04.267 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:48:04.267 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.267 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.268 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:48:04.268 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:48:04.268 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.269 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:48:04.269 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:48:04.269 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.270 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.270 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:48:04.270 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:48:04.271 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.271 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.272 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:48:04.272 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:48:04.273 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.273 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.273 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.274 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.274 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.275 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:48:04.275 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:48:04.276 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.276 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.276 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.276 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.277 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:48:04.278 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:48:04.278 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.278 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.278 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.279 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.279 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.280 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:48:04.280 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:48:04.280 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.280 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.281 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.281 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.282 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:48:04.283 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:48:04.283 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.283 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.284 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.284 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.284 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:48:04.285 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:48:04.285 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.285 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.285 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.285 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.286 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:48:04.286 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:48:04.287 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.287 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.287 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.288 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:48:04.288 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:48:04.288 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.288 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:48:04.288 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.289 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:48:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:48:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.290 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:48:04.291 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:48:04.291 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.291 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:48:04.292 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:48:04.292 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:48:04.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.293 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:48:04.294 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:48:04.296 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.296 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:48:04.297 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:48:04.297 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:48:04.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.298 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:48:04.298 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:48:04.298 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.299 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:48:04.299 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:48:04.299 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:48:04.299 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:48:04.300 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.300 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:48:04.301 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:48:04.301 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.301 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:48:04.302 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:48:04.302 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:48:04.302 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.303 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:48:04.303 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:48:04.304 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.304 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:48:04.304 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:48:04.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:04.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:04.306 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:48:04.306 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:48:04.306 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:48:04.306 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:48:04.306 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:48:04.307 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:48:04.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:04.307 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:48:04.308 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:48:04.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a050 msgr2=0x7f2c6806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a050 0x7f2c6806c510 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f2c80036000 tx=0x7f2c80030000 comp rx=0 tx=0).stop 2023-08-25T09:48:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 msgr2=0x7f2c840a8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c840a8460 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2c7400c430 tx=0x7f2c7400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 shutdown_connections 2023-08-25T09:48:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6806a050 0x7f2c6806c510 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8415ec10 0x7f2c840ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8415e830 0x7f2c840a8460 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 --2- 172.21.15.70:0/4101808588 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8406b2b0 0x7f2c840ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 >> 172.21.15.70:0/4101808588 conn(0x7f2c84069850 msgr2=0x7f2c840cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:04.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.258+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 shutdown_connections 2023-08-25T09:48:04.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.259+0000 7f2c8c4b4640 1 -- 172.21.15.70:0/4101808588 wait complete. 2023-08-25T09:48:04.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:48:04.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.526+0000 7f9169d2f640 1 Processor -- start 2023-08-25T09:48:04.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.526+0000 7f9169d2f640 1 -- start start 2023-08-25T09:48:04.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.527+0000 7f9169d2f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91640ce5b0 0x7f91640ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9169d2f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f916415aa10 0x7f91640b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9169d2f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91640b2aa0 con 0x7f91640b2680 2023-08-25T09:48:04.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9169d2f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9164159d10 con 0x7f91640ce5b0 2023-08-25T09:48:04.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9169d2f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91640ceed0 con 0x7f916415aa10 2023-08-25T09:48:04.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f91637fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91640ce5b0 0x7f91640ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9163fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f91640b2680 0x7f916406be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42908/0 (socket says 172.21.15.70:42908) 2023-08-25T09:48:04.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9163fff640 1 -- 172.21.15.70:0/3874544684 learned_addr learned my addr 172.21.15.70:0/3874544684 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f915bfff640 1 --2- 172.21.15.70:0/3874544684 >> v2:172.21.15.155:3300/0 conn(0x7f916415aa10 0x7f91640b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f91637fe640 1 -- 172.21.15.70:0/3874544684 >> v2:172.21.15.155:3300/0 conn(0x7f916415aa10 msgr2=0x7f91640b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f9162ffd640 1 -- 172.21.15.70:0/3874544684 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524030817 0 0) 0x7f91640b2aa0 con 0x7f91640b2680 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f91637fe640 1 --2- 172.21.15.70:0/3874544684 >> v2:172.21.15.155:3300/0 conn(0x7f916415aa10 0x7f91640b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f91637fe640 1 -- 172.21.15.70:0/3874544684 >> v1:172.21.15.70:6789/0 conn(0x7f91640b2680 legacy=0x7f916406be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f91637fe640 1 -- 172.21.15.70:0/3874544684 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916415d3f0 con 0x7f91640ce5b0 2023-08-25T09:48:04.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.528+0000 7f915bfff640 1 --2- 172.21.15.70:0/3874544684 >> v2:172.21.15.155:3300/0 conn(0x7f916415aa10 0x7f91640b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:04.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.530+0000 7f91637fe640 1 --2- 172.21.15.70:0/3874544684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91640ce5b0 0x7f91640ce990 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f91640b16e0 tx=0x7f915402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d57bd38e1313d3a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.530+0000 7f9162ffd640 1 -- 172.21.15.70:0/3874544684 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915402e070 con 0x7f91640ce5b0 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.530+0000 7f9162ffd640 1 -- 172.21.15.70:0/3874544684 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91540315a0 con 0x7f91640ce5b0 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.530+0000 7f9162ffd640 1 -- 172.21.15.70:0/3874544684 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154031930 con 0x7f91640ce5b0 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.530+0000 7f9169d2f640 1 -- 172.21.15.70:0/3874544684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91640ce5b0 msgr2=0x7f91640ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.530+0000 7f9169d2f640 1 --2- 172.21.15.70:0/3874544684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91640ce5b0 0x7f91640ce990 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f91640b16e0 tx=0x7f915402b840 comp rx=0 tx=0).stop 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.531+0000 7f9169d2f640 1 -- 172.21.15.70:0/3874544684 shutdown_connections 2023-08-25T09:48:04.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.531+0000 7f9169d2f640 1 --2- 172.21.15.70:0/3874544684 >> v2:172.21.15.155:3300/0 conn(0x7f916415aa10 0x7f91640b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.531+0000 7f9169d2f640 1 --2- 172.21.15.70:0/3874544684 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91640ce5b0 0x7f91640ce990 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.531+0000 7f9169d2f640 1 -- 172.21.15.70:0/3874544684 >> 172.21.15.70:0/3874544684 conn(0x7f9164069850 msgr2=0x7f9164069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:04.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.531+0000 7f9169d2f640 1 -- 172.21.15.70:0/3874544684 shutdown_connections 2023-08-25T09:48:04.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.531+0000 7f9169d2f640 1 -- 172.21.15.70:0/3874544684 wait complete. 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.532+0000 7f9169d2f640 1 Processor -- start 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.532+0000 7f9169d2f640 1 -- start start 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.533+0000 7f9169d2f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91640b2680 0x7f91640acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.533+0000 7f9169d2f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91640ce5b0 0x7f91640b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.533+0000 7f9169d2f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 0x7f9164147c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.533+0000 7f9169d2f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91640cbcc0 con 0x7f916415aa10 2023-08-25T09:48:04.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.533+0000 7f9169d2f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9164156570 con 0x7f91640ce5b0 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.533+0000 7f9169d2f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91640cbf60 con 0x7f91640b2680 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 0x7f9164147c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 0x7f9164147c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39372/0 (socket says 172.21.15.70:39372) 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 -- 172.21.15.70:0/2150092114 learned_addr learned my addr 172.21.15.70:0/2150092114 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f91637fe640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91640b2680 0x7f91640acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f915bfff640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91640ce5b0 0x7f91640b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 -- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91640b2680 msgr2=0x7f91640acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91640b2680 0x7f91640acae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 -- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91640ce5b0 msgr2=0x7f91640b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91640ce5b0 0x7f91640b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 -- 172.21.15.70:0/2150092114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9154009400 con 0x7f916415aa10 2023-08-25T09:48:04.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f91637fe640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91640b2680 0x7f91640acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:04.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f915bfff640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91640ce5b0 0x7f91640b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:04.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.534+0000 7f9163fff640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 0x7f9164147c10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c780 tx=0x7f915c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:04.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.535+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00ce90 con 0x7f916415aa10 2023-08-25T09:48:04.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.535+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c00a640 con 0x7f916415aa10 2023-08-25T09:48:04.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.535+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00a9d0 con 0x7f916415aa10 2023-08-25T09:48:04.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.535+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91640cc240 con 0x7f916415aa10 2023-08-25T09:48:04.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.535+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91640ad020 con 0x7f916415aa10 2023-08-25T09:48:04.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.536+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91640cc480 con 0x7f916415aa10 2023-08-25T09:48:04.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.537+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f915c00ab30 con 0x7f916415aa10 2023-08-25T09:48:04.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.538+0000 7f9160ff9640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f914406a080 0x7f914406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.539+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f915c0875c0 con 0x7f916415aa10 2023-08-25T09:48:04.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.539+0000 7f91637fe640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f914406a080 0x7f914406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.539+0000 7f91637fe640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f914406a080 0x7f914406c540 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9154036000 tx=0x7f915403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:04.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.543+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91640cc480 con 0x7f916415aa10 2023-08-25T09:48:04.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.732+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91640cc430 con 0x7f916415aa10 2023-08-25T09:48:04.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.734+0000 7f9160ff9640 1 -- 172.21.15.70:0/2150092114 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+14665 (secure 0 0 0) 0x7f915c053090 con 0x7f916415aa10 2023-08-25T09:48:04.737 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:04.737 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":161,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:48:03.759653+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:48:04.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f914406a080 msgr2=0x7f914406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f914406a080 0x7f914406c540 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9154036000 tx=0x7f915403d040 comp rx=0 tx=0).stop 2023-08-25T09:48:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 msgr2=0x7f9164147c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 0x7f9164147c10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c780 tx=0x7f915c00cc40 comp rx=0 tx=0).stop 2023-08-25T09:48:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 shutdown_connections 2023-08-25T09:48:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f914406a080 0x7f914406c540 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f916415aa10 0x7f9164147c10 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91640ce5b0 0x7f91640b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 --2- 172.21.15.70:0/2150092114 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91640b2680 0x7f91640acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.737+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 >> 172.21.15.70:0/2150092114 conn(0x7f9164069850 msgr2=0x7f916415b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.738+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 shutdown_connections 2023-08-25T09:48:04.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.738+0000 7f9169d2f640 1 -- 172.21.15.70:0/2150092114 wait complete. 2023-08-25T09:48:04.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:48:04.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:48:04.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.977+0000 7fe170183640 1 Processor -- start 2023-08-25T09:48:04.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.977+0000 7fe170183640 1 -- start start 2023-08-25T09:48:04.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe170183640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe168069850 0x7fe16815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe170183640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1680cca20 0x7fe1680cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe170183640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1680ac880 con 0x7fe1680c9620 2023-08-25T09:48:04.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe170183640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1680ad400 con 0x7fe1680cca20 2023-08-25T09:48:04.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe170183640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1680ad890 con 0x7fe168069850 2023-08-25T09:48:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe16e720640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1680cca20 0x7fe1680cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe16e720640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1680cca20 0x7fe1680cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39382/0 (socket says 172.21.15.70:39382) 2023-08-25T09:48:04.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe16e720640 1 -- 172.21.15.70:0/2870221616 learned_addr learned my addr 172.21.15.70:0/2870221616 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.979+0000 7fe16df1f640 1 --2- 172.21.15.70:0/2870221616 >> v2:172.21.15.155:3300/0 conn(0x7fe168069850 0x7fe16815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.980+0000 7fe16e720640 1 -- 172.21.15.70:0/2870221616 >> v2:172.21.15.155:3300/0 conn(0x7fe168069850 msgr2=0x7fe16815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.980+0000 7fe16e720640 1 --2- 172.21.15.70:0/2870221616 >> v2:172.21.15.155:3300/0 conn(0x7fe168069850 0x7fe16815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.980+0000 7fe16e720640 1 -- 172.21.15.70:0/2870221616 >> v1:172.21.15.155:6789/0 conn(0x7fe1680c9620 legacy=0x7fe168069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.980+0000 7fe16e720640 1 -- 172.21.15.70:0/2870221616 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680acad0 con 0x7fe1680cca20 2023-08-25T09:48:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.981+0000 7fe16e720640 1 --2- 172.21.15.70:0/2870221616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680cceb0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe1680cc0d0 tx=0x7fe16002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25477fdedd1a7b5a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:04.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.981+0000 7fe16cf1d640 1 -- 172.21.15.70:0/2870221616 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16002cb20 con 0x7fe1680cca20 2023-08-25T09:48:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.981+0000 7fe16cf1d640 1 -- 172.21.15.70:0/2870221616 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16002cc80 con 0x7fe1680cca20 2023-08-25T09:48:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.981+0000 7fe16cf1d640 1 -- 172.21.15.70:0/2870221616 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1600343f0 con 0x7fe1680cca20 2023-08-25T09:48:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 -- 172.21.15.70:0/2870221616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 msgr2=0x7fe1680cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:04.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 --2- 172.21.15.70:0/2870221616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680cceb0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe1680cc0d0 tx=0x7fe16002bf80 comp rx=0 tx=0).stop 2023-08-25T09:48:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 -- 172.21.15.70:0/2870221616 shutdown_connections 2023-08-25T09:48:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 --2- 172.21.15.70:0/2870221616 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680cceb0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 --2- 172.21.15.70:0/2870221616 >> v2:172.21.15.155:3300/0 conn(0x7fe168069850 0x7fe16815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:04.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 -- 172.21.15.70:0/2870221616 >> 172.21.15.70:0/2870221616 conn(0x7fe1680cb0d0 msgr2=0x7fe1680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 -- 172.21.15.70:0/2870221616 shutdown_connections 2023-08-25T09:48:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.982+0000 7fe170183640 1 -- 172.21.15.70:0/2870221616 wait complete. 2023-08-25T09:48:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.984+0000 7fe170183640 1 Processor -- start 2023-08-25T09:48:04.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.984+0000 7fe170183640 1 -- start start 2023-08-25T09:48:04.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.985+0000 7fe170183640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe168069850 0x7fe1680b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.985+0000 7fe170183640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1680c9620 0x7fe1680c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.985+0000 7fe170183640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:04.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.985+0000 7fe170183640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680c6860 con 0x7fe1680cca20 2023-08-25T09:48:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.985+0000 7fe170183640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680c1680 con 0x7fe168069850 2023-08-25T09:48:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.985+0000 7fe170183640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680c49c0 con 0x7fe1680c9620 2023-08-25T09:48:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680c22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39388/0 (socket says 172.21.15.70:39388) 2023-08-25T09:48:04.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 -- 172.21.15.70:0/401716265 learned_addr learned my addr 172.21.15.70:0/401716265 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:04.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16df1f640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe168069850 0x7fe1680b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16d71e640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1680c9620 0x7fe1680c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 -- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1680c9620 msgr2=0x7fe1680c4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1680c9620 0x7fe1680c4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 -- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe168069850 msgr2=0x7fe1680b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe168069850 0x7fe1680b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.986+0000 7fe16e720640 1 -- 172.21.15.70:0/401716265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160009b30 con 0x7fe1680cca20 2023-08-25T09:48:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.987+0000 7fe16e720640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680c22b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fe160036040 tx=0x7fe160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.987+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe160009510 con 0x7fe1680cca20 2023-08-25T09:48:05.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.987+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe160007e50 con 0x7fe1680cca20 2023-08-25T09:48:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.987+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1680c5c50 con 0x7fe1680cca20 2023-08-25T09:48:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.987+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16003e690 con 0x7fe1680cca20 2023-08-25T09:48:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.987+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1680c6210 con 0x7fe1680cca20 2023-08-25T09:48:05.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.988+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1680c9a40 con 0x7fe1680cca20 2023-08-25T09:48:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.989+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe16003e7f0 con 0x7fe1680cca20 2023-08-25T09:48:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.990+0000 7fe15e7fc640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe14c06a050 0x7fe14c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.990+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe1600aae90 con 0x7fe1680cca20 2023-08-25T09:48:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.990+0000 7fe16df1f640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe14c06a050 0x7fe14c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.991+0000 7fe16df1f640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe14c06a050 0x7fe14c06c510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe164006fd0 tx=0x7fe164008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:04.994+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe1680c9a40 con 0x7fe1680cca20 2023-08-25T09:48:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.182+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1680c9a40 con 0x7fe1680cca20 2023-08-25T09:48:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.182+0000 7fe15e7fc640 1 -- 172.21.15.70:0/401716265 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v161) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1680c9a40 con 0x7fe1680cca20 2023-08-25T09:48:05.186 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:48:05.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe14c06a050 msgr2=0x7fe14c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe14c06a050 0x7fe14c06c510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe164006fd0 tx=0x7fe164008040 comp rx=0 tx=0).stop 2023-08-25T09:48:05.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 msgr2=0x7fe1680c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680c22b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7fe160036040 tx=0x7fe160004a40 comp rx=0 tx=0).stop 2023-08-25T09:48:05.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 shutdown_connections 2023-08-25T09:48:05.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe14c06a050 0x7fe14c06c510 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1680cca20 0x7fe1680c22b0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1680c9620 0x7fe1680c4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 --2- 172.21.15.70:0/401716265 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe168069850 0x7fe1680b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 >> 172.21.15.70:0/401716265 conn(0x7fe1680cb0d0 msgr2=0x7fe1681415b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:05.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 shutdown_connections 2023-08-25T09:48:05.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.185+0000 7fe170183640 1 -- 172.21.15.70:0/401716265 wait complete. 2023-08-25T09:48:05.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:48:05.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.412+0000 7fbf28ca7640 1 Processor -- start 2023-08-25T09:48:05.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.412+0000 7fbf28ca7640 1 -- start start 2023-08-25T09:48:05.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf28ca7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf2415e210 0x7fbf24069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf28ca7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf28ca7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240ac880 con 0x7fbf24069850 2023-08-25T09:48:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf28ca7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2415e650 con 0x7fbf2415e210 2023-08-25T09:48:05.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf28ca7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ad400 con 0x7fbf240cca20 2023-08-25T09:48:05.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf21d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf2415e210 0x7fbf24069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf21d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbf2415e210 0x7fbf24069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39402/0 (socket says 172.21.15.70:39402) 2023-08-25T09:48:05.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf21d9b640 1 -- 172.21.15.70:0/108321632 learned_addr learned my addr 172.21.15.70:0/108321632 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.414+0000 7fbf22d9d640 1 --2- 172.21.15.70:0/108321632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.415+0000 7fbf21d9b640 1 -- 172.21.15.70:0/108321632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 msgr2=0x7fbf240cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.415+0000 7fbf21d9b640 1 --2- 172.21.15.70:0/108321632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.415+0000 7fbf21d9b640 1 -- 172.21.15.70:0/108321632 >> v1:172.21.15.155:6789/0 conn(0x7fbf24069850 legacy=0x7fbf240c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.415+0000 7fbf21d9b640 1 -- 172.21.15.70:0/108321632 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf240ad890 con 0x7fbf2415e210 2023-08-25T09:48:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.416+0000 7fbf21d9b640 1 --2- 172.21.15.70:0/108321632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf2415e210 0x7fbf24069c30 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009870 tx=0x7fbf0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeb2457c394fe96 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.416+0000 7fbf2159a640 1 -- 172.21.15.70:0/108321632 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c030840 con 0x7fbf2415e210 2023-08-25T09:48:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.416+0000 7fbf2159a640 1 -- 172.21.15.70:0/108321632 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0309a0 con 0x7fbf2415e210 2023-08-25T09:48:05.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.416+0000 7fbf2159a640 1 -- 172.21.15.70:0/108321632 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c030d30 con 0x7fbf2415e210 2023-08-25T09:48:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 -- 172.21.15.70:0/108321632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf2415e210 msgr2=0x7fbf24069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/108321632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf2415e210 0x7fbf24069c30 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009870 tx=0x7fbf0c02b980 comp rx=0 tx=0).stop 2023-08-25T09:48:05.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 -- 172.21.15.70:0/108321632 shutdown_connections 2023-08-25T09:48:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/108321632 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/108321632 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf2415e210 0x7fbf24069c30 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 -- 172.21.15.70:0/108321632 >> 172.21.15.70:0/108321632 conn(0x7fbf240cb0d0 msgr2=0x7fbf240cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:05.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 -- 172.21.15.70:0/108321632 shutdown_connections 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.417+0000 7fbf28ca7640 1 -- 172.21.15.70:0/108321632 wait complete. 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.418+0000 7fbf28ca7640 1 Processor -- start 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.419+0000 7fbf28ca7640 1 -- start start 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.419+0000 7fbf28ca7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 0x7fbf2414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.419+0000 7fbf28ca7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf28ca7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf2415e210 0x7fbf2415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf28ca7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240c4c80 con 0x7fbf24069850 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf28ca7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240aeb40 con 0x7fbf2415e210 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf28ca7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2414f220 con 0x7fbf240cca20 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 0x7fbf2414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 0x7fbf2414ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39406/0 (socket says 172.21.15.70:39406) 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 -- 172.21.15.70:0/293864203 learned_addr learned my addr 172.21.15.70:0/293864203 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf21d9b640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf22d9d640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf2415e210 0x7fbf2415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 -- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 msgr2=0x7fbf240c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 -- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf2415e210 msgr2=0x7fbf2415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf2415e210 0x7fbf2415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.420+0000 7fbf2259c640 1 -- 172.21.15.70:0/293864203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0094d0 con 0x7fbf24069850 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.421+0000 7fbf2259c640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 0x7fbf2414ece0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c430 tx=0x7fbf1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.421+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011790 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.421+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf140118f0 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.421+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011ce0 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.421+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240ae210 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.421+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf240ae7d0 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.422+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24066fb0 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.423+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbf1401c460 con 0x7fbf24069850 2023-08-25T09:48:05.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.424+0000 7fbf1affd640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbf0406a080 0x7fbf0406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.424+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbf14088280 con 0x7fbf24069850 2023-08-25T09:48:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.424+0000 7fbf21d9b640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbf0406a080 0x7fbf0406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.425+0000 7fbf21d9b640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbf0406a080 0x7fbf0406c540 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c036000 tx=0x7fbf0c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.428+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbf14053c20 con 0x7fbf24069850 2023-08-25T09:48:05.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.615+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbf2406a810 con 0x7fbf24069850 2023-08-25T09:48:05.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.616+0000 7fbf1affd640 1 -- 172.21.15.70:0/293864203 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v161) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbf14057240 con 0x7fbf24069850 2023-08-25T09:48:05.619 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:48:05.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.618+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbf0406a080 msgr2=0x7fbf0406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.618+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbf0406a080 0x7fbf0406c540 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c036000 tx=0x7fbf0c03b040 comp rx=0 tx=0).stop 2023-08-25T09:48:05.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.618+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 msgr2=0x7fbf2414ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.618+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 0x7fbf2414ece0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c430 tx=0x7fbf1400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 shutdown_connections 2023-08-25T09:48:05.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbf0406a080 0x7fbf0406c540 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbf2415e210 0x7fbf2415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbf240cca20 0x7fbf240c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 --2- 172.21.15.70:0/293864203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbf24069850 0x7fbf2414ece0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 >> 172.21.15.70:0/293864203 conn(0x7fbf240cb0d0 msgr2=0x7fbf2406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:05.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 shutdown_connections 2023-08-25T09:48:05.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.619+0000 7fbf28ca7640 1 -- 172.21.15.70:0/293864203 wait complete. 2023-08-25T09:48:05.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:48:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.859+0000 7fe88ccd3640 1 Processor -- start 2023-08-25T09:48:05.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.860+0000 7fe88ccd3640 1 -- start start 2023-08-25T09:48:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.861+0000 7fe88ccd3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815e830 0x7fe88806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.861+0000 7fe88ccd3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe88806b2b0 0x7fe88815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.861+0000 7fe88ccd3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe88815c6d0 0x7fe8880b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.861+0000 7fe88ccd3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88806bd80 con 0x7fe88815c6d0 2023-08-25T09:48:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.861+0000 7fe88ccd3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8880b1b60 con 0x7fe88806b2b0 2023-08-25T09:48:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.861+0000 7fe88ccd3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88806b7c0 con 0x7fe88815e830 2023-08-25T09:48:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe88815c6d0 0x7fe8880b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe88815c6d0 0x7fe8880b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39420/0 (socket says 172.21.15.70:39420) 2023-08-25T09:48:05.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 -- 172.21.15.70:0/549904320 learned_addr learned my addr 172.21.15.70:0/549904320 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe87ffff640 1 --2- 172.21.15.70:0/549904320 >> v2:172.21.15.155:3300/0 conn(0x7fe88806b2b0 0x7fe88815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe88659c640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815e830 0x7fe88806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 -- 172.21.15.70:0/549904320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815e830 msgr2=0x7fe88806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815e830 0x7fe88806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 -- 172.21.15.70:0/549904320 >> v2:172.21.15.155:3300/0 conn(0x7fe88806b2b0 msgr2=0x7fe88815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 --2- 172.21.15.70:0/549904320 >> v2:172.21.15.155:3300/0 conn(0x7fe88806b2b0 0x7fe88815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe886d9d640 1 -- 172.21.15.70:0/549904320 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88806baa0 con 0x7fe88815c6d0 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.862+0000 7fe88659c640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815e830 0x7fe88806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.863+0000 7fe886d9d640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815c6d0 0x7fe8880b1ff0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe888071120 tx=0x7fe88002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70691e9cc90eebef server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe885d9b640 1 -- 172.21.15.70:0/549904320 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88002ca80 con 0x7fe88815c6d0 2023-08-25T09:48:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe885d9b640 1 -- 172.21.15.70:0/549904320 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88002cbe0 con 0x7fe88815c6d0 2023-08-25T09:48:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe885d9b640 1 -- 172.21.15.70:0/549904320 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8800343f0 con 0x7fe88815c6d0 2023-08-25T09:48:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 -- 172.21.15.70:0/549904320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815c6d0 msgr2=0x7fe8880b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815c6d0 0x7fe8880b1ff0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fe888071120 tx=0x7fe88002bee0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 -- 172.21.15.70:0/549904320 shutdown_connections 2023-08-25T09:48:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815c6d0 0x7fe8880b1ff0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/549904320 >> v2:172.21.15.155:3300/0 conn(0x7fe88806b2b0 0x7fe88815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/549904320 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815e830 0x7fe88806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 -- 172.21.15.70:0/549904320 >> 172.21.15.70:0/549904320 conn(0x7fe888069850 msgr2=0x7fe888069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 -- 172.21.15.70:0/549904320 shutdown_connections 2023-08-25T09:48:05.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.864+0000 7fe88ccd3640 1 -- 172.21.15.70:0/549904320 wait complete. 2023-08-25T09:48:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.866+0000 7fe88ccd3640 1 Processor -- start 2023-08-25T09:48:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.866+0000 7fe88ccd3640 1 -- start start 2023-08-25T09:48:05.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe88ccd3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88806b2b0 0x7fe8880aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe88ccd3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815c6d0 0x7fe8880a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe88ccd3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 0x7fe8880cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe88ccd3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880aab80 con 0x7fe88815e830 2023-08-25T09:48:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe88ccd3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880aae60 con 0x7fe88806b2b0 2023-08-25T09:48:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe88ccd3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880ab140 con 0x7fe88815c6d0 2023-08-25T09:48:05.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe886d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 0x7fe8880cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe886d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 0x7fe8880cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39432/0 (socket says 172.21.15.70:39432) 2023-08-25T09:48:05.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.867+0000 7fe886d9d640 1 -- 172.21.15.70:0/2291733278 learned_addr learned my addr 172.21.15.70:0/2291733278 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:05.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe886d9d640 1 -- 172.21.15.70:0/2291733278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815c6d0 msgr2=0x7fe8880a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe886d9d640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815c6d0 0x7fe8880a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe886d9d640 1 -- 172.21.15.70:0/2291733278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88806b2b0 msgr2=0x7fe8880aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe886d9d640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88806b2b0 0x7fe8880aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe886d9d640 1 -- 172.21.15.70:0/2291733278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe880009b30 con 0x7fe88815e830 2023-08-25T09:48:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe886d9d640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 0x7fe8880cca00 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe880036040 tx=0x7fe880004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880009510 con 0x7fe88815e830 2023-08-25T09:48:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88003e450 con 0x7fe88815e830 2023-08-25T09:48:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88003e800 con 0x7fe88815e830 2023-08-25T09:48:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.868+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8880acb00 con 0x7fe88815e830 2023-08-25T09:48:05.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.869+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8880af1f0 con 0x7fe88815e830 2023-08-25T09:48:05.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.870+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8880ac960 con 0x7fe88815e830 2023-08-25T09:48:05.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.871+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe88002cd50 con 0x7fe88815e830 2023-08-25T09:48:05.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.871+0000 7fe87effd640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe86806a050 0x7fe86806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:05.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.872+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe8800aae10 con 0x7fe88815e830 2023-08-25T09:48:05.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.872+0000 7fe88659c640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe86806a050 0x7fe86806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:05.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.872+0000 7fe88659c640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe86806a050 0x7fe86806c510 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe878006fd0 tx=0x7fe878008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:05.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:05.876+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe8880ac960 con 0x7fe88815e830 2023-08-25T09:48:06.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.068+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe8880ac960 con 0x7fe88815e830 2023-08-25T09:48:06.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.069+0000 7fe87effd640 1 -- 172.21.15.70:0/2291733278 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v161) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe8880ac960 con 0x7fe88815e830 2023-08-25T09:48:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:06.072 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:48:06.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:48:06.073 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:06.074 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:48:06.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.073+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe86806a050 msgr2=0x7fe86806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.073+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe86806a050 0x7fe86806c510 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fe878006fd0 tx=0x7fe878008040 comp rx=0 tx=0).stop 2023-08-25T09:48:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.073+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 msgr2=0x7fe8880cca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.073+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 0x7fe8880cca00 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fe880036040 tx=0x7fe880004a40 comp rx=0 tx=0).stop 2023-08-25T09:48:06.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 shutdown_connections 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe86806a050 0x7fe86806c510 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe88815e830 0x7fe8880cca00 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe88815c6d0 0x7fe8880a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 --2- 172.21.15.70:0/2291733278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe88806b2b0 0x7fe8880aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 >> 172.21.15.70:0/2291733278 conn(0x7fe888069850 msgr2=0x7fe88815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 shutdown_connections 2023-08-25T09:48:06.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.074+0000 7fe88ccd3640 1 -- 172.21.15.70:0/2291733278 wait complete. 2023-08-25T09:48:06.095 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:48:06.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:48:06.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.305+0000 7f359d0ad640 1 Processor -- start 2023-08-25T09:48:06.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.306+0000 7f359d0ad640 1 -- start start 2023-08-25T09:48:06.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.307+0000 7f359d0ad640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3598153cd0 0x7f359815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:06.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.307+0000 7f359d0ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980b8470 0x7f35980d0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:06.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.307+0000 7f359d0ad640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35980cd2f0 con 0x7f35980d1460 2023-08-25T09:48:06.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.307+0000 7f359d0ad640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359815d6f0 con 0x7f3598153cd0 2023-08-25T09:48:06.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.307+0000 7f359d0ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359815e1a0 con 0x7f35980b8470 2023-08-25T09:48:06.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3598153cd0 0x7f359815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3598153cd0 0x7f359815d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39448/0 (socket says 172.21.15.70:39448) 2023-08-25T09:48:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 -- 172.21.15.70:0/102656402 learned_addr learned my addr 172.21.15.70:0/102656402 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 -- 172.21.15.70:0/102656402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980b8470 msgr2=0x7f35980d0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f359659c640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980b8470 0x7f35980d0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:06.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980b8470 0x7f35980d0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 -- 172.21.15.70:0/102656402 >> v1:172.21.15.70:6789/0 conn(0x7f35980d1460 legacy=0x7f3598160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f3596d9d640 1 -- 172.21.15.70:0/102656402 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35981540b0 con 0x7f3598153cd0 2023-08-25T09:48:06.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.308+0000 7f359659c640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980b8470 0x7f35980d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:06.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.310+0000 7f3596d9d640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598153cd0 0x7f359815d1b0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f359815d830 tx=0x7f358802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea52ec0d2ac6a540 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:06.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.310+0000 7f3595d9b640 1 -- 172.21.15.70:0/102656402 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358802c9c0 con 0x7f3598153cd0 2023-08-25T09:48:06.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.310+0000 7f3595d9b640 1 -- 172.21.15.70:0/102656402 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358802cb20 con 0x7f3598153cd0 2023-08-25T09:48:06.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.310+0000 7f3595d9b640 1 -- 172.21.15.70:0/102656402 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35880343f0 con 0x7f3598153cd0 2023-08-25T09:48:06.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 -- 172.21.15.70:0/102656402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598153cd0 msgr2=0x7f359815d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598153cd0 0x7f359815d1b0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f359815d830 tx=0x7f358802be20 comp rx=0 tx=0).stop 2023-08-25T09:48:06.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 -- 172.21.15.70:0/102656402 shutdown_connections 2023-08-25T09:48:06.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35980b8470 0x7f35980d0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 --2- 172.21.15.70:0/102656402 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3598153cd0 0x7f359815d1b0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 -- 172.21.15.70:0/102656402 >> 172.21.15.70:0/102656402 conn(0x7f35980704a0 msgr2=0x7f35980cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:06.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 -- 172.21.15.70:0/102656402 shutdown_connections 2023-08-25T09:48:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.311+0000 7f359d0ad640 1 -- 172.21.15.70:0/102656402 wait complete. 2023-08-25T09:48:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.313+0000 7f359d0ad640 1 Processor -- start 2023-08-25T09:48:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.313+0000 7f359d0ad640 1 -- start start 2023-08-25T09:48:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359d0ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35980b8470 0x7f35980cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:06.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359d0ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 0x7f35980cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359d0ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3598153cd0 0x7f35980c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359d0ad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35980ccb80 con 0x7f35980d1460 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359d0ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35980c6820 con 0x7f35980b8470 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359d0ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35980cc2e0 con 0x7f3598153cd0 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 0x7f35980cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 0x7f35980cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39458/0 (socket says 172.21.15.70:39458) 2023-08-25T09:48:06.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f359659c640 1 -- 172.21.15.70:0/2571358715 learned_addr learned my addr 172.21.15.70:0/2571358715 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.314+0000 7f3596d9d640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35980b8470 0x7f35980cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359759e640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3598153cd0 0x7f35980c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359659c640 1 -- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3598153cd0 msgr2=0x7f35980c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359659c640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3598153cd0 0x7f35980c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359659c640 1 -- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35980b8470 msgr2=0x7f35980cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359659c640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35980b8470 0x7f35980cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359659c640 1 -- 172.21.15.70:0/2571358715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588009b30 con 0x7f35980d1460 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359759e640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3598153cd0 0x7f35980c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f359659c640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 0x7f35980cdba0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f358000c7f0 tx=0x7f358000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580007820 con 0x7f35980d1460 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3580007980 con 0x7f35980d1460 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.315+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580007cf0 con 0x7f35980d1460 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.316+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35980cc5c0 con 0x7f35980d1460 2023-08-25T09:48:06.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.316+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35980cbfb0 con 0x7f35980d1460 2023-08-25T09:48:06.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.317+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35980c69d0 con 0x7f35980d1460 2023-08-25T09:48:06.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.318+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f35800113f0 con 0x7f35980d1460 2023-08-25T09:48:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.319+0000 7f358f7fe640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a080 0x7f357806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.319+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f35800885a0 con 0x7f35980d1460 2023-08-25T09:48:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.319+0000 7f3596d9d640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a080 0x7f357806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.319+0000 7f3596d9d640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a080 0x7f357806c540 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3588035000 tx=0x7f3588030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.324+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35980c69d0 con 0x7f35980d1460 2023-08-25T09:48:06.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.515+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f35980c6990 con 0x7f35980d1460 2023-08-25T09:48:06.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.780+0000 7f358f7fe640 1 -- 172.21.15.70:0/2571358715 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v162) v1 ==== 138+0+0 (secure 0 0 0) 0x7f35800579d0 con 0x7f35980d1460 2023-08-25T09:48:06.783 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:48:06.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.788+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a080 msgr2=0x7f357806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.788+0000 7f359d0ad640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a080 0x7f357806c540 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3588035000 tx=0x7f3588030000 comp rx=0 tx=0).stop 2023-08-25T09:48:06.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.788+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 msgr2=0x7f35980cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:06.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.788+0000 7f359d0ad640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 0x7f35980cdba0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f358000c7f0 tx=0x7f358000ccb0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 shutdown_connections 2023-08-25T09:48:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f357806a080 0x7f357806c540 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3598153cd0 0x7f35980c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35980d1460 0x7f35980cdba0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 --2- 172.21.15.70:0/2571358715 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35980b8470 0x7f35980cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 >> 172.21.15.70:0/2571358715 conn(0x7f35980704a0 msgr2=0x7f35980b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:06.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.789+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 shutdown_connections 2023-08-25T09:48:06.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:06.790+0000 7f359d0ad640 1 -- 172.21.15.70:0/2571358715 wait complete. 2023-08-25T09:48:06.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:48:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.025+0000 7fa40c0d3640 1 Processor -- start 2023-08-25T09:48:07.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.026+0000 7fa40c0d3640 1 -- start start 2023-08-25T09:48:07.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40c0d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa4040cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40c0d3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa40415cfe0 0x7fa4040d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40c0d3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4040b8510 0x7fa4040b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40c0d3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040ce170 con 0x7fa4040b8510 2023-08-25T09:48:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40c0d3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4040d1500 con 0x7fa40415cfe0 2023-08-25T09:48:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40c0d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040d17e0 con 0x7fa4040cd850 2023-08-25T09:48:07.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40a670640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4040b8510 0x7fa4040b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.027+0000 7fa40a670640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa4040b8510 0x7fa4040b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39468/0 (socket says 172.21.15.70:39468) 2023-08-25T09:48:07.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40a670640 1 -- 172.21.15.70:0/1753447262 learned_addr learned my addr 172.21.15.70:0/1753447262 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40966e640 1 --2- 172.21.15.70:0/1753447262 >> v2:172.21.15.155:3300/0 conn(0x7fa40415cfe0 0x7fa4040d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa409e6f640 1 --2- 172.21.15.70:0/1753447262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa4040cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40a670640 1 -- 172.21.15.70:0/1753447262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 msgr2=0x7fa4040cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:07.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40a670640 1 --2- 172.21.15.70:0/1753447262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa4040cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40a670640 1 -- 172.21.15.70:0/1753447262 >> v2:172.21.15.155:3300/0 conn(0x7fa40415cfe0 msgr2=0x7fa4040d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:07.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40a670640 1 --2- 172.21.15.70:0/1753447262 >> v2:172.21.15.155:3300/0 conn(0x7fa40415cfe0 0x7fa4040d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40a670640 1 -- 172.21.15.70:0/1753447262 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040d1ac0 con 0x7fa4040b8510 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.028+0000 7fa40966e640 1 --2- 172.21.15.70:0/1753447262 >> v2:172.21.15.155:3300/0 conn(0x7fa40415cfe0 0x7fa4040d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.029+0000 7fa40a670640 1 --2- 172.21.15.70:0/1753447262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa4040b4590 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fa4040b56c0 tx=0x7fa3fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d646633df524d634 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.029+0000 7fa408e6d640 1 -- 172.21.15.70:0/1753447262 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc031640 con 0x7fa4040b8510 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa408e6d640 1 -- 172.21.15.70:0/1753447262 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc0317a0 con 0x7fa4040b8510 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa408e6d640 1 -- 172.21.15.70:0/1753447262 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc031b30 con 0x7fa4040b8510 2023-08-25T09:48:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 -- 172.21.15.70:0/1753447262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 msgr2=0x7fa4040b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/1753447262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa4040b4590 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fa4040b56c0 tx=0x7fa3fc02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 -- 172.21.15.70:0/1753447262 shutdown_connections 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/1753447262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa4040b4590 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/1753447262 >> v2:172.21.15.155:3300/0 conn(0x7fa40415cfe0 0x7fa4040d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/1753447262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa4040cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 -- 172.21.15.70:0/1753447262 >> 172.21.15.70:0/1753447262 conn(0x7fa404069850 msgr2=0x7fa404069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 -- 172.21.15.70:0/1753447262 shutdown_connections 2023-08-25T09:48:07.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.030+0000 7fa40c0d3640 1 -- 172.21.15.70:0/1753447262 wait complete. 2023-08-25T09:48:07.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.032+0000 7fa40c0d3640 1 Processor -- start 2023-08-25T09:48:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.032+0000 7fa40c0d3640 1 -- start start 2023-08-25T09:48:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.033+0000 7fa40c0d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa404154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.033+0000 7fa40c0d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa40415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.033+0000 7fa40c0d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa40415cfe0 0x7fa40415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.033+0000 7fa40c0d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040c94f0 con 0x7fa4040b8510 2023-08-25T09:48:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.033+0000 7fa40c0d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040b4c90 con 0x7fa40415cfe0 2023-08-25T09:48:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.033+0000 7fa40c0d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4040b3ed0 con 0x7fa4040cd850 2023-08-25T09:48:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa404154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa404154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39476/0 (socket says 172.21.15.70:39476) 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 -- 172.21.15.70:0/142317154 learned_addr learned my addr 172.21.15.70:0/142317154 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa40a670640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa40415cfe0 0x7fa40415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa40966e640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa40415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 -- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 msgr2=0x7fa40415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa40415d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 -- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa40415cfe0 msgr2=0x7fa40415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa40415cfe0 0x7fa40415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 -- 172.21.15.70:0/142317154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc0094b0 con 0x7fa4040b8510 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa40a670640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa40415cfe0 0x7fa40415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.034+0000 7fa409e6f640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa404154950 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400ce10 tx=0x7fa3f400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.035+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4003b90 con 0x7fa4040b8510 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.035+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f4003cf0 con 0x7fa4040b8510 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.035+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4040b41b0 con 0x7fa4040b8510 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.035+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f4004260 con 0x7fa4040b8510 2023-08-25T09:48:07.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.035+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4040b4770 con 0x7fa4040b8510 2023-08-25T09:48:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.036+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40406a700 con 0x7fa4040b8510 2023-08-25T09:48:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.037+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3f40043c0 con 0x7fa4040b8510 2023-08-25T09:48:07.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.038+0000 7fa4027fc640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3f006a050 0x7fa3f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.038+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fa3f4012070 con 0x7fa4040b8510 2023-08-25T09:48:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.038+0000 7fa40966e640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3f006a050 0x7fa3f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:07.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.039+0000 7fa40966e640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3f006a050 0x7fa3f006c510 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa4040b1b20 tx=0x7fa3f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:07.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.045+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3f4054690 con 0x7fa4040b8510 2023-08-25T09:48:07.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.236+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa4040b4990 con 0x7fa4040b8510 2023-08-25T09:48:07.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.792+0000 7fa4027fc640 1 -- 172.21.15.70:0/142317154 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v163) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa3f4057cb0 con 0x7fa4040b8510 2023-08-25T09:48:07.802 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:48:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3f006a050 msgr2=0x7fa3f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3f006a050 0x7fa3f006c510 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa4040b1b20 tx=0x7fa3f8008040 comp rx=0 tx=0).stop 2023-08-25T09:48:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 msgr2=0x7fa404154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa404154950 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fa3f400ce10 tx=0x7fa3f400a660 comp rx=0 tx=0).stop 2023-08-25T09:48:07.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 shutdown_connections 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3f006a050 0x7fa3f006c510 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa40415cfe0 0x7fa40415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4040cd850 0x7fa40415d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 --2- 172.21.15.70:0/142317154 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4040b8510 0x7fa404154950 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 >> 172.21.15.70:0/142317154 conn(0x7fa404069850 msgr2=0x7fa4040b72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.806+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 shutdown_connections 2023-08-25T09:48:07.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:07.807+0000 7fa40c0d3640 1 -- 172.21.15.70:0/142317154 wait complete. 2023-08-25T09:48:07.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:48:08.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.028+0000 7f94b041f640 1 Processor -- start 2023-08-25T09:48:08.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.028+0000 7f94b041f640 1 -- start start 2023-08-25T09:48:08.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.029+0000 7f94b041f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 0x7f94a815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.029+0000 7f94b041f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a815e780 con 0x7f94a806f0e0 2023-08-25T09:48:08.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.029+0000 7f94b041f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a806f8e0 con 0x7f94a80cd660 2023-08-25T09:48:08.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.029+0000 7f94b041f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a806fbc0 con 0x7f94a8159bf0 2023-08-25T09:48:08.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad9ba640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f94a806f0e0 0x7f94a8070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42914/0 (socket says 172.21.15.70:42914) 2023-08-25T09:48:08.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad9ba640 1 -- 172.21.15.70:0/2668913394 learned_addr learned my addr 172.21.15.70:0/2668913394 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:08.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 0x7f94a815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117815413 0 0) 0x7f94a815e780 con 0x7f94a806f0e0 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9494003520 con 0x7f94a806f0e0 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3186188429 0 0) 0x7f9494003520 con 0x7f94a806f0e0 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 msgr2=0x7f94a815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 0x7f94a815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 >> v1:172.21.15.155:6789/0 conn(0x7f94a80cd660 legacy=0x7f94a80c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a806fea0 con 0x7f94a806f0e0 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9498001670 con 0x7f94a806f0e0 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.030+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 >> v1:172.21.15.70:6789/0 conn(0x7f94a806f0e0 legacy=0x7f94a8070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad1b9640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f9494003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad1b9640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 0x7f94940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad1b9640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494004d00 0x7f9494005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a806fbc0 con 0x7f94940039a0 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949400ba30 con 0x7f9494004d00 2023-08-25T09:48:08.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949400bbf0 con 0x7f94a8159bf0 2023-08-25T09:48:08.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f9494003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae1bb640 1 -- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 msgr2=0x7f94940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ad9ba640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 0x7f94940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae9bc640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494004d00 0x7f9494005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a8159bf0 0x7f94940047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae1bb640 1 -- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494004d00 msgr2=0x7f9494005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494004d00 0x7f9494005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae1bb640 1 -- 172.21.15.70:0/2668913394 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494003520 con 0x7f94940039a0 2023-08-25T09:48:08.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.031+0000 7f94ae9bc640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9494004d00 0x7f9494005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.032+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f9494003dc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009a90 tx=0x7f94940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea64cfbcd7758633 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a402c680 con 0x7f94940039a0 2023-08-25T09:48:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a402c7e0 con 0x7f94940039a0 2023-08-25T09:48:08.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94ad1b9640 1 -- 172.21.15.70:0/2668913394 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a402cb70 con 0x7f94940039a0 2023-08-25T09:48:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94b041f640 1 -- 172.21.15.70:0/2668913394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 msgr2=0x7f9494003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94b041f640 1 --2- 172.21.15.70:0/2668913394 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f9494003dc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f94a4009a90 tx=0x7f94940057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94ae9bc640 1 -- 172.21.15.70:0/2668913394 reap_dead start 2023-08-25T09:48:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94b041f640 1 -- 172.21.15.70:0/2668913394 shutdown_connections 2023-08-25T09:48:08.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94b041f640 1 -- 172.21.15.70:0/2668913394 >> 172.21.15.70:0/2668913394 conn(0x7f94a80cb0d0 msgr2=0x7f94a80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.033+0000 7f94b041f640 1 -- 172.21.15.70:0/2668913394 shutdown_connections 2023-08-25T09:48:08.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.034+0000 7f94b041f640 1 -- 172.21.15.70:0/2668913394 wait complete. 2023-08-25T09:48:08.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.035+0000 7f94b041f640 1 Processor -- start 2023-08-25T09:48:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.035+0000 7f94b041f640 1 -- start start 2023-08-25T09:48:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.036+0000 7f94b041f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f94a80a93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.036+0000 7f94b041f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a80c7000 0x7f94a815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.036+0000 7f94b041f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a814f8e0 0x7f94a814fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.036+0000 7f94b041f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80c75a0 con 0x7f94940039a0 2023-08-25T09:48:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.036+0000 7f94b041f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80c7880 con 0x7f94a80c7000 2023-08-25T09:48:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.036+0000 7f94b041f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80c7b60 con 0x7f94a814f8e0 2023-08-25T09:48:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f94a80a93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f94a80a93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39498/0 (socket says 172.21.15.70:39498) 2023-08-25T09:48:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 -- 172.21.15.70:0/3591346689 learned_addr learned my addr 172.21.15.70:0/3591346689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:08.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ad9ba640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a80c7000 0x7f94a815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae9bc640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a814f8e0 0x7f94a814fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 -- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a814f8e0 msgr2=0x7f94a814fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a814f8e0 0x7f94a814fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 -- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a80c7000 msgr2=0x7f94a815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a80c7000 0x7f94a815c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 -- 172.21.15.70:0/3591346689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80c7e40 con 0x7f94940039a0 2023-08-25T09:48:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.037+0000 7f94ae1bb640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f94a80a93b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f94a402c220 tx=0x7f94a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.038+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4003810 con 0x7f94940039a0 2023-08-25T09:48:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.038+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a4003970 con 0x7f94940039a0 2023-08-25T09:48:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.038+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a80c8120 con 0x7f94940039a0 2023-08-25T09:48:08.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.038+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a4003ce0 con 0x7f94940039a0 2023-08-25T09:48:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.038+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a80a98f0 con 0x7f94940039a0 2023-08-25T09:48:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.039+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a8066fb0 con 0x7f94940039a0 2023-08-25T09:48:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.040+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f94a4034440 con 0x7f94940039a0 2023-08-25T09:48:08.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.041+0000 7f949effd640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f948c06a080 0x7f948c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.041+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f94a40aa4c0 con 0x7f94940039a0 2023-08-25T09:48:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.041+0000 7f94ad9ba640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f948c06a080 0x7f948c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:08.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.042+0000 7f94ad9ba640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f948c06a080 0x7f948c06c540 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b00a0 tx=0x7f9498009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:08.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.046+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94a4075d40 con 0x7f94940039a0 2023-08-25T09:48:08.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.272+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f94a80aae00 con 0x7f94940039a0 2023-08-25T09:48:08.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.831+0000 7f949effd640 1 -- 172.21.15.70:0/3591346689 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7f94a40044e0 con 0x7f94940039a0 2023-08-25T09:48:08.834 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 19 and data pool 20 2023-08-25T09:48:08.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f948c06a080 msgr2=0x7f948c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.837 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f948c06a080 0x7f948c06c540 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f94a80b00a0 tx=0x7f9498009040 comp rx=0 tx=0).stop 2023-08-25T09:48:08.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 msgr2=0x7f94a80a93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:08.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f94a80a93b0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f94a402c220 tx=0x7f94a4004730 comp rx=0 tx=0).stop 2023-08-25T09:48:08.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 shutdown_connections 2023-08-25T09:48:08.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f948c06a080 0x7f948c06c540 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94a814f8e0 0x7f94a814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94a80c7000 0x7f94a815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 --2- 172.21.15.70:0/3591346689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94940039a0 0x7f94a80a93b0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.834+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 >> 172.21.15.70:0/3591346689 conn(0x7f94a80cb0d0 msgr2=0x7f94a80c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.835+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 shutdown_connections 2023-08-25T09:48:08.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:08.835+0000 7f94b041f640 1 -- 172.21.15.70:0/3591346689 wait complete. 2023-08-25T09:48:08.863 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:48:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.053+0000 7f7683fff640 1 Processor -- start 2023-08-25T09:48:09.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.053+0000 7f7683fff640 1 -- start start 2023-08-25T09:48:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f7683fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f76840c9d30 0x7f7684156980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f7683fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 0x7f76840caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f7683fff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7684156ec0 con 0x7f76840adb10 2023-08-25T09:48:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f7683fff640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76840c98a0 con 0x7f76840c9d30 2023-08-25T09:48:09.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f7683fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76840ca110 con 0x7f76840ad6d0 2023-08-25T09:48:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f76837fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f76840adb10 0x7f768406ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57288/0 (socket says 172.21.15.70:57288) 2023-08-25T09:48:09.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.055+0000 7f76837fe640 1 -- 172.21.15.70:0/2731930008 learned_addr learned my addr 172.21.15.70:0/2731930008 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2731930008 >> v2:172.21.15.155:3300/0 conn(0x7f76840c9d30 0x7f7684156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f76827fc640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 0x7f76840caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235222651 0 0) 0x7f7684156ec0 con 0x7f76840adb10 2023-08-25T09:48:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f766c003520 con 0x7f76840adb10 2023-08-25T09:48:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4195684474 0 0) 0x7f766c003520 con 0x7f76840adb10 2023-08-25T09:48:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 msgr2=0x7f76840caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 0x7f76840caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 >> v2:172.21.15.155:3300/0 conn(0x7f76840c9d30 msgr2=0x7f7684156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 --2- 172.21.15.70:0/2731930008 >> v2:172.21.15.155:3300/0 conn(0x7f76840c9d30 0x7f7684156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840c8c90 con 0x7f76840adb10 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f767c001670 con 0x7f76840adb10 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 >> v1:172.21.15.70:6789/0 conn(0x7f76840adb10 legacy=0x7f768406ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f766c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f766c0047c0 0x7f766c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.056+0000 7f7681ffb640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 0x7f766c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76840c98a0 con 0x7f766c003970 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76840ca110 con 0x7f766c0047c0 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f766c00db90 con 0x7f76840ad6d0 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f766c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7682ffd640 1 -- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 msgr2=0x7f766c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840ad6d0 0x7f766c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7682ffd640 1 -- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f766c0047c0 msgr2=0x7f766c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f766c0047c0 0x7f766c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.057+0000 7f7682ffd640 1 -- 172.21.15.70:0/2731930008 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c003520 con 0x7f766c003970 2023-08-25T09:48:09.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.058+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f766c003d90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7674009ac0 tx=0x7f766c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a74b6b2df3cd2b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.058+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767402c710 con 0x7f766c003970 2023-08-25T09:48:09.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.058+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f767402c870 con 0x7f766c003970 2023-08-25T09:48:09.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.058+0000 7f7681ffb640 1 -- 172.21.15.70:0/2731930008 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767402cc00 con 0x7f766c003970 2023-08-25T09:48:09.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f7683fff640 1 -- 172.21.15.70:0/2731930008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 msgr2=0x7f766c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f7683fff640 1 --2- 172.21.15.70:0/2731930008 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f766c003d90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7674009ac0 tx=0x7f766c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f76837fe640 1 -- 172.21.15.70:0/2731930008 reap_dead start 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f7683fff640 1 -- 172.21.15.70:0/2731930008 shutdown_connections 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f7683fff640 1 -- 172.21.15.70:0/2731930008 >> 172.21.15.70:0/2731930008 conn(0x7f76840affd0 msgr2=0x7f76840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f7683fff640 1 -- 172.21.15.70:0/2731930008 shutdown_connections 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.059+0000 7f7683fff640 1 -- 172.21.15.70:0/2731930008 wait complete. 2023-08-25T09:48:09.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.061+0000 7f7683fff640 1 Processor -- start 2023-08-25T09:48:09.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.061+0000 7f7683fff640 1 -- start start 2023-08-25T09:48:09.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7683fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f768414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7683fff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768414e6d0 0x7f76841f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7683fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840707a0 0x7f76841f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7683fff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768414d4f0 con 0x7f766c003970 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7683fff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768414c970 con 0x7f768414e6d0 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7683fff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f768414cc50 con 0x7f76840707a0 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7682ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f768414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7682ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f768414e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44508/0 (socket says 172.21.15.70:44508) 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.062+0000 7f7682ffd640 1 -- 172.21.15.70:0/2905688146 learned_addr learned my addr 172.21.15.70:0/2905688146 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7682ffd640 1 -- 172.21.15.70:0/2905688146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840707a0 msgr2=0x7f76841f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840707a0 0x7f76841f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7682ffd640 1 -- 172.21.15.70:0/2905688146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768414e6d0 msgr2=0x7f76841f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768414e6d0 0x7f76841f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7682ffd640 1 -- 172.21.15.70:0/2905688146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768414cf30 con 0x7f766c003970 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7682ffd640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f768414e190 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f767402c2b0 tx=0x7f7674004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674003810 con 0x7f766c003970 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674003970 con 0x7f766c003970 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674003d00 con 0x7f766c003970 2023-08-25T09:48:09.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.063+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768414d210 con 0x7f766c003970 2023-08-25T09:48:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.064+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768414c700 con 0x7f766c003970 2023-08-25T09:48:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.065+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768406e970 con 0x7f766c003970 2023-08-25T09:48:09.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.066+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7674040450 con 0x7f766c003970 2023-08-25T09:48:09.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.067+0000 7f766b7fe640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f767006a080 0x7f767006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.067+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76740aa440 con 0x7f766c003970 2023-08-25T09:48:09.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.067+0000 7f76827fc640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f767006a080 0x7f767006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.068+0000 7f76827fc640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f767006a080 0x7f767006c540 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f76780070b0 tx=0x7f767800d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.071+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7674075b20 con 0x7f766c003970 2023-08-25T09:48:09.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.260+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7684066fb0 con 0x7f766c003970 2023-08-25T09:48:09.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.263+0000 7f766b7fe640 1 -- 172.21.15.70:0/2905688146 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+18322 (secure 0 0 0) 0x7f7674003e60 con 0x7f766c003970 2023-08-25T09:48:09.266 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:09.266 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":164,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:48:08.274400+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f767006a080 msgr2=0x7f767006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f767006a080 0x7f767006c540 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f76780070b0 tx=0x7f767800d040 comp rx=0 tx=0).stop 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 msgr2=0x7f768414e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f768414e190 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f767402c2b0 tx=0x7f7674004730 comp rx=0 tx=0).stop 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 shutdown_connections 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f767006a080 0x7f767006c540 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f76840707a0 0x7f76841f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f768414e6d0 0x7f76841f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 --2- 172.21.15.70:0/2905688146 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f766c003970 0x7f768414e190 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 >> 172.21.15.70:0/2905688146 conn(0x7f76840affd0 msgr2=0x7f76840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:09.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 shutdown_connections 2023-08-25T09:48:09.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.266+0000 7f7683fff640 1 -- 172.21.15.70:0/2905688146 wait complete. 2023-08-25T09:48:09.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:48:09.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.488+0000 7f11df786640 1 Processor -- start 2023-08-25T09:48:09.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.488+0000 7f11df786640 1 -- start start 2023-08-25T09:48:09.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.489+0000 7f11df786640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11d815e830 0x7f11d806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.489+0000 7f11df786640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d80b1b60 con 0x7f11d806b2b0 2023-08-25T09:48:09.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.489+0000 7f11df786640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d806b7c0 con 0x7f11d815ec10 2023-08-25T09:48:09.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.489+0000 7f11df786640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d806bd80 con 0x7f11d815e830 2023-08-25T09:48:09.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.490+0000 7f11dcd21640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11d806b2b0 0x7f11d815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57294/0 (socket says 172.21.15.70:57294) 2023-08-25T09:48:09.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.490+0000 7f11dcd21640 1 -- 172.21.15.70:0/213024074 learned_addr learned my addr 172.21.15.70:0/213024074 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:09.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.490+0000 7f11dd522640 1 --2- 172.21.15.70:0/213024074 >> v2:172.21.15.155:3300/0 conn(0x7f11d815e830 0x7f11d806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.490+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677943488 0 0) 0x7f11d80b1b60 con 0x7f11d806b2b0 2023-08-25T09:48:09.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.490+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11c4003520 con 0x7f11d806b2b0 2023-08-25T09:48:09.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 790331393 0 0) 0x7f11c4003520 con 0x7f11d806b2b0 2023-08-25T09:48:09.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 >> v2:172.21.15.155:3300/0 conn(0x7f11d815e830 msgr2=0x7f11d806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 --2- 172.21.15.70:0/213024074 >> v2:172.21.15.155:3300/0 conn(0x7f11d815e830 0x7f11d806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 >> v1:172.21.15.155:6789/0 conn(0x7f11d815ec10 legacy=0x7f11d80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d806baa0 con 0x7f11d806b2b0 2023-08-25T09:48:09.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f11c8001670 con 0x7f11d806b2b0 2023-08-25T09:48:09.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 >> v1:172.21.15.70:6789/0 conn(0x7f11d806b2b0 legacy=0x7f11d815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4003970 0x7f11c4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11c40047c0 0x7f11c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4005180 0x7f11c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d806bd80 con 0x7f11c4003970 2023-08-25T09:48:09.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c400c000 con 0x7f11c4005180 2023-08-25T09:48:09.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c400c1c0 con 0x7f11c40047c0 2023-08-25T09:48:09.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11dd522640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4003970 0x7f11c4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11dd522640 1 -- 172.21.15.70:0/213024074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11c40047c0 msgr2=0x7f11c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11dd522640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11c40047c0 0x7f11c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11dd522640 1 -- 172.21.15.70:0/213024074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4005180 msgr2=0x7f11c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11dd522640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4005180 0x7f11c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11dd522640 1 -- 172.21.15.70:0/213024074 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c4003520 con 0x7f11c4003970 2023-08-25T09:48:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.491+0000 7f11ddd23640 1 -- 172.21.15.70:0/213024074 reap_dead start 2023-08-25T09:48:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.492+0000 7f11dd522640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4003970 0x7f11c4003d70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009ac0 tx=0x7f11c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be2e8b0ccaa44d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d402c710 con 0x7f11c4003970 2023-08-25T09:48:09.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d402c870 con 0x7f11c4003970 2023-08-25T09:48:09.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11cffff640 1 -- 172.21.15.70:0/213024074 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d402cc00 con 0x7f11c4003970 2023-08-25T09:48:09.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 -- 172.21.15.70:0/213024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4003970 msgr2=0x7f11c4003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4003970 0x7f11c4003d70 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009ac0 tx=0x7f11c40057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 -- 172.21.15.70:0/213024074 shutdown_connections 2023-08-25T09:48:09.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 --2- 172.21.15.70:0/213024074 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11c4003970 0x7f11c4003d70 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 -- 172.21.15.70:0/213024074 >> 172.21.15.70:0/213024074 conn(0x7f11d8069850 msgr2=0x7f11d8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:09.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 -- 172.21.15.70:0/213024074 shutdown_connections 2023-08-25T09:48:09.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.493+0000 7f11df786640 1 -- 172.21.15.70:0/213024074 wait complete. 2023-08-25T09:48:09.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.495+0000 7f11df786640 1 Processor -- start 2023-08-25T09:48:09.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.495+0000 7f11df786640 1 -- start start 2023-08-25T09:48:09.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.496+0000 7f11df786640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4003970 0x7f11d806f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.496+0000 7f11df786640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 0x7f11d814bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.496+0000 7f11df786640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11d80c9d90 0x7f11d814c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.496+0000 7f11df786640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80a8d90 con 0x7f11d80c9910 2023-08-25T09:48:09.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.496+0000 7f11df786640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80ca320 con 0x7f11c4003970 2023-08-25T09:48:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.496+0000 7f11df786640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d80a8460 con 0x7f11d80c9d90 2023-08-25T09:48:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 0x7f11d814bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 0x7f11d814bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44520/0 (socket says 172.21.15.70:44520) 2023-08-25T09:48:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 -- 172.21.15.70:0/3728007497 learned_addr learned my addr 172.21.15.70:0/3728007497 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:09.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11ddd23640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11d80c9d90 0x7f11d814c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 -- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11d80c9d90 msgr2=0x7f11d814c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11d80c9d90 0x7f11d814c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dd522640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4003970 0x7f11d806f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 -- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4003970 msgr2=0x7f11d806f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4003970 0x7f11d806f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 -- 172.21.15.70:0/3728007497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d80a8740 con 0x7f11d80c9910 2023-08-25T09:48:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.497+0000 7f11dcd21640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 0x7f11d814bd50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f11c800c770 tx=0x7f11c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.511 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.498+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c80074e0 con 0x7f11d80c9910 2023-08-25T09:48:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.498+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c8007640 con 0x7f11d80c9910 2023-08-25T09:48:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.498+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d80a8a20 con 0x7f11d80c9910 2023-08-25T09:48:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.498+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c8004a00 con 0x7f11d80c9910 2023-08-25T09:48:09.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.498+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d806fce0 con 0x7f11d80c9910 2023-08-25T09:48:09.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.499+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d80a8f40 con 0x7f11d80c9910 2023-08-25T09:48:09.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.500+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f11c8004b60 con 0x7f11d80c9910 2023-08-25T09:48:09.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.501+0000 7f11cdffb640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11bc06a050 0x7f11bc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.501+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11c8087c90 con 0x7f11d80c9910 2023-08-25T09:48:09.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.501+0000 7f11dd522640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11bc06a050 0x7f11bc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.502+0000 7f11dd522640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11bc06a050 0x7f11bc06c510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f11d4036000 tx=0x7f11d4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.505+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f11d80a8f40 con 0x7f11d80c9910 2023-08-25T09:48:09.700 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.698+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11d806fce0 con 0x7f11d80c9910 2023-08-25T09:48:09.702 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.700+0000 7f11cdffb640 1 -- 172.21.15.70:0/3728007497 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+18309 (secure 0 0 0) 0x7f11d806fce0 con 0x7f11d80c9910 2023-08-25T09:48:09.703 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:09.703 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":165,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:48:09.412707+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:48:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11bc06a050 msgr2=0x7f11bc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11bc06a050 0x7f11bc06c510 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f11d4036000 tx=0x7f11d4030000 comp rx=0 tx=0).stop 2023-08-25T09:48:09.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 msgr2=0x7f11d814bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 0x7f11d814bd50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f11c800c770 tx=0x7f11c800cc30 comp rx=0 tx=0).stop 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 shutdown_connections 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11bc06a050 0x7f11bc06c510 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11d80c9d90 0x7f11d814c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11d80c9910 0x7f11d814bd50 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 --2- 172.21.15.70:0/3728007497 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11c4003970 0x7f11d806f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 >> 172.21.15.70:0/3728007497 conn(0x7f11d8069850 msgr2=0x7f11d80b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:09.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 shutdown_connections 2023-08-25T09:48:09.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.703+0000 7f11df786640 1 -- 172.21.15.70:0/3728007497 wait complete. 2023-08-25T09:48:09.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:48:09.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.943+0000 7f9c68376640 1 Processor -- start 2023-08-25T09:48:09.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.944+0000 7f9c68376640 1 -- start start 2023-08-25T09:48:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.945+0000 7f9c68376640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6015ae50 0x7f9c600c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.945+0000 7f9c68376640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c60069f60 0x7f9c600ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.945+0000 7f9c68376640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c600cc5f0 con 0x7f9c600cdc20 2023-08-25T09:48:09.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.945+0000 7f9c68376640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c60070650 con 0x7f9c6015ae50 2023-08-25T09:48:09.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.945+0000 7f9c68376640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c60070930 con 0x7f9c60069f60 2023-08-25T09:48:09.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.945+0000 7f9c66112640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9c6015ae50 0x7f9c600c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c65911640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9c600cdc20 0x7f9c60069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57302/0 (socket says 172.21.15.70:57302) 2023-08-25T09:48:09.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c65911640 1 -- 172.21.15.70:0/2128580722 learned_addr learned my addr 172.21.15.70:0/2128580722 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:09.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c66913640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c60069f60 0x7f9c600ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c66112640 1 -- 172.21.15.70:0/2128580722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c60069f60 msgr2=0x7f9c600ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c66112640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c60069f60 0x7f9c600ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c66112640 1 -- 172.21.15.70:0/2128580722 >> v1:172.21.15.70:6789/0 conn(0x7f9c600cdc20 legacy=0x7f9c60069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c65110640 1 -- 172.21.15.70:0/2128580722 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3673068883 0 0) 0x7f9c600cc5f0 con 0x7f9c600cdc20 2023-08-25T09:48:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c66112640 1 -- 172.21.15.70:0/2128580722 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c60070c10 con 0x7f9c6015ae50 2023-08-25T09:48:09.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.946+0000 7f9c66913640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c60069f60 0x7f9c600ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.947+0000 7f9c66112640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6015ae50 0x7f9c600c2110 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0097a0 tx=0x7f9c5c02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607f72fe1cb0e2a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c65110640 1 -- 172.21.15.70:0/2128580722 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c030800 con 0x7f9c6015ae50 2023-08-25T09:48:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c65110640 1 -- 172.21.15.70:0/2128580722 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c030960 con 0x7f9c6015ae50 2023-08-25T09:48:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c65110640 1 -- 172.21.15.70:0/2128580722 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c030cf0 con 0x7f9c6015ae50 2023-08-25T09:48:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 -- 172.21.15.70:0/2128580722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6015ae50 msgr2=0x7f9c600c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6015ae50 0x7f9c600c2110 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0097a0 tx=0x7f9c5c02b940 comp rx=0 tx=0).stop 2023-08-25T09:48:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 -- 172.21.15.70:0/2128580722 shutdown_connections 2023-08-25T09:48:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c60069f60 0x7f9c600ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 --2- 172.21.15.70:0/2128580722 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c6015ae50 0x7f9c600c2110 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 -- 172.21.15.70:0/2128580722 >> 172.21.15.70:0/2128580722 conn(0x7f9c600cb0d0 msgr2=0x7f9c600cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:09.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 -- 172.21.15.70:0/2128580722 shutdown_connections 2023-08-25T09:48:09.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.948+0000 7f9c68376640 1 -- 172.21.15.70:0/2128580722 wait complete. 2023-08-25T09:48:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.950+0000 7f9c68376640 1 Processor -- start 2023-08-25T09:48:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.950+0000 7f9c68376640 1 -- start start 2023-08-25T09:48:09.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.951+0000 7f9c68376640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 0x7f9c601f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.951+0000 7f9c68376640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c600cdc20 0x7f9c6014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.951+0000 7f9c68376640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6015ae50 0x7f9c6014e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.951+0000 7f9c68376640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6014c060 con 0x7f9c60069f60 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.951+0000 7f9c68376640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6014ef10 con 0x7f9c6015ae50 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.951+0000 7f9c68376640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c601f61f0 con 0x7f9c600cdc20 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 0x7f9c601f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 0x7f9c601f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44542/0 (socket says 172.21.15.70:44542) 2023-08-25T09:48:09.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 -- 172.21.15.70:0/1443403537 learned_addr learned my addr 172.21.15.70:0/1443403537 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 -- 172.21.15.70:0/1443403537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c600cdc20 msgr2=0x7f9c6014b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c600cdc20 0x7f9c6014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 -- 172.21.15.70:0/1443403537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6015ae50 msgr2=0x7f9c6014e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6015ae50 0x7f9c6014e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 -- 172.21.15.70:0/1443403537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c009400 con 0x7f9c60069f60 2023-08-25T09:48:09.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.952+0000 7f9c66112640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 0x7f9c601f5c10 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0098c0 tx=0x7f9c5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.953+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c03e070 con 0x7f9c60069f60 2023-08-25T09:48:09.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.953+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c039ba0 con 0x7f9c60069f60 2023-08-25T09:48:09.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.953+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c0423f0 con 0x7f9c60069f60 2023-08-25T09:48:09.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.953+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c601f64d0 con 0x7f9c60069f60 2023-08-25T09:48:09.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.953+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c601f6a90 con 0x7f9c60069f60 2023-08-25T09:48:09.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.954+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c60066fb0 con 0x7f9c60069f60 2023-08-25T09:48:09.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.955+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9c5c042550 con 0x7f9c60069f60 2023-08-25T09:48:09.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.956+0000 7f9c56ffd640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:09.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.956+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c5c0aac20 con 0x7f9c60069f60 2023-08-25T09:48:09.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.956+0000 7f9c65911640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:09.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.957+0000 7f9c65911640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007fd0 tx=0x7f9c50007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:09.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:09.960+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9c5c076300 con 0x7f9c60069f60 2023-08-25T09:48:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.184+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9c60159bb0 con 0x7f9c60069f60 2023-08-25T09:48:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.855+0000 7f9c56ffd640 1 -- 172.21.15.70:0/1443403537 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9c5c0340b0 con 0x7f9c60069f60 2023-08-25T09:48:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 msgr2=0x7f9c4406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9c50007fd0 tx=0x7f9c50007920 comp rx=0 tx=0).stop 2023-08-25T09:48:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 msgr2=0x7f9c601f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 0x7f9c601f5c10 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c0098c0 tx=0x7f9c5c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:48:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 shutdown_connections 2023-08-25T09:48:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9c4406a050 0x7f9c4406c510 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9c6015ae50 0x7f9c6014e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9c600cdc20 0x7f9c6014b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.858+0000 7f9c68376640 1 --2- 172.21.15.70:0/1443403537 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9c60069f60 0x7f9c601f5c10 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.859+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 >> 172.21.15.70:0/1443403537 conn(0x7f9c600cb0d0 msgr2=0x7f9c600ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.859+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 shutdown_connections 2023-08-25T09:48:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:10.859+0000 7f9c68376640 1 -- 172.21.15.70:0/1443403537 wait complete. 2023-08-25T09:48:10.882 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:48:10.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:48:11.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.103+0000 7f93cb59e640 1 Processor -- start 2023-08-25T09:48:11.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.103+0000 7f93cb59e640 1 -- start start 2023-08-25T09:48:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.104+0000 7f93cb59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93cc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.104+0000 7f93cb59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f93cc0b0300 0x7f93cc0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.104+0000 7f93cb59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93cc0acda0 con 0x7f93cc0ccee0 2023-08-25T09:48:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.104+0000 7f93cb59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93cc0cd650 con 0x7f93cc0b0300 2023-08-25T09:48:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.104+0000 7f93cb59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0b0cd0 con 0x7f93cc159650 2023-08-25T09:48:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.104+0000 7f93c3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f93cc0ccee0 0x7f93cc0ad230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57304/0 (socket says 172.21.15.70:57304) 2023-08-25T09:48:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c3fff640 1 -- 172.21.15.70:0/2184355703 learned_addr learned my addr 172.21.15.70:0/2184355703 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355756130 0 0) 0x7f93cc0acda0 con 0x7f93cc0ccee0 2023-08-25T09:48:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93b8003520 con 0x7f93cc0ccee0 2023-08-25T09:48:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93cc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2184355703 >> v2:172.21.15.155:3300/0 conn(0x7f93cc0b0300 0x7f93cc0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1162760792 0 0) 0x7f93b8003520 con 0x7f93cc0ccee0 2023-08-25T09:48:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 msgr2=0x7f93cc15dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93cc15dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 >> v2:172.21.15.155:3300/0 conn(0x7f93cc0b0300 msgr2=0x7f93cc0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 --2- 172.21.15.70:0/2184355703 >> v2:172.21.15.155:3300/0 conn(0x7f93cc0b0300 0x7f93cc0b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc0ac1d0 con 0x7f93cc0ccee0 2023-08-25T09:48:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f93b4001670 con 0x7f93cc0ccee0 2023-08-25T09:48:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2184355703 >> v2:172.21.15.155:3300/0 conn(0x7f93cc0b0300 0x7f93cc0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:48:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.105+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 >> v1:172.21.15.70:6789/0 conn(0x7f93cc0ccee0 legacy=0x7f93cc0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c9d9b640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c9d9b640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8003fa0 0x7f93b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c9d9b640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0cd650 con 0x7f93b8003fa0 2023-08-25T09:48:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0b0cd0 con 0x7f93b8004d00 2023-08-25T09:48:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b800dd50 con 0x7f93cc159650 2023-08-25T09:48:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c3fff640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8003fa0 0x7f93b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c3fff640 1 -- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 msgr2=0x7f93b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c3fff640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93b8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c3fff640 1 -- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8004d00 msgr2=0x7f93b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:11.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c3fff640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93c3fff640 1 -- 172.21.15.70:0/2184355703 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8003520 con 0x7f93b8003fa0 2023-08-25T09:48:11.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc159650 0x7f93b8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:11.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.106+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8004d00 0x7f93b8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.107+0000 7f93c3fff640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8003fa0 0x7f93b80047c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c240 tx=0x7f93b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0268525a14875cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4005ad0 con 0x7f93b8003fa0 2023-08-25T09:48:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4005c50 con 0x7f93b8003fa0 2023-08-25T09:48:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93c9d9b640 1 -- 172.21.15.70:0/2184355703 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b402c8d0 con 0x7f93b8003fa0 2023-08-25T09:48:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cb59e640 1 -- 172.21.15.70:0/2184355703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8003fa0 msgr2=0x7f93b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2184355703 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93b8003fa0 0x7f93b80047c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c240 tx=0x7f93b80057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cad9d640 1 -- 172.21.15.70:0/2184355703 reap_dead start 2023-08-25T09:48:11.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cb59e640 1 -- 172.21.15.70:0/2184355703 shutdown_connections 2023-08-25T09:48:11.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cb59e640 1 -- 172.21.15.70:0/2184355703 >> 172.21.15.70:0/2184355703 conn(0x7f93cc0ce810 msgr2=0x7f93cc0cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:11.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cb59e640 1 -- 172.21.15.70:0/2184355703 shutdown_connections 2023-08-25T09:48:11.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.108+0000 7f93cb59e640 1 -- 172.21.15.70:0/2184355703 wait complete. 2023-08-25T09:48:11.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.110+0000 7f93cb59e640 1 Processor -- start 2023-08-25T09:48:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.110+0000 7f93cb59e640 1 -- start start 2023-08-25T09:48:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.111+0000 7f93cb59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8003fa0 0x7f93cc0c1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.111+0000 7f93cb59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc0710a0 0x7f93cc06bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.111+0000 7f93cb59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 0x7f93cc1f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.111+0000 7f93cb59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0718d0 con 0x7f93cc15b750 2023-08-25T09:48:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.111+0000 7f93cb59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0715f0 con 0x7f93b8003fa0 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.111+0000 7f93cb59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc06b450 con 0x7f93cc0710a0 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 0x7f93cc1f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 0x7f93cc1f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44568/0 (socket says 172.21.15.70:44568) 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 -- 172.21.15.70:0/2572299050 learned_addr learned my addr 172.21.15.70:0/2572299050 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93c3fff640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc0710a0 0x7f93cc06bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8003fa0 0x7f93cc0c1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 -- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc0710a0 msgr2=0x7f93cc06bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:11.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc0710a0 0x7f93cc06bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 -- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8003fa0 msgr2=0x7f93cc0c1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:11.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8003fa0 0x7f93cc0c1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93cad9d640 1 -- 172.21.15.70:0/2572299050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc06b730 con 0x7f93cc15b750 2023-08-25T09:48:11.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8003fa0 0x7f93cc0c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.112+0000 7f93c3fff640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc0710a0 0x7f93cc06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.113+0000 7f93cad9d640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 0x7f93cc1f15c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0c5360 tx=0x7f93c4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.113+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4007c70 con 0x7f93cc15b750 2023-08-25T09:48:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.113+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c4007dd0 con 0x7f93cc15b750 2023-08-25T09:48:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.113+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93cc06ba10 con 0x7f93cc15b750 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.113+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4014640 con 0x7f93cc15b750 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.113+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc06c470 con 0x7f93cc15b750 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.114+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93cc06b2b0 con 0x7f93cc15b750 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.115+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f93c401d3b0 con 0x7f93cc15b750 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.116+0000 7f93c2ffd640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f93ac06a050 0x7f93ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.116+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93c4088330 con 0x7f93cc15b750 2023-08-25T09:48:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.116+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f93ac06a050 0x7f93ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:11.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.117+0000 7f93ca59c640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f93ac06a050 0x7f93ac06c510 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f93bc006a30 tx=0x7f93bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:11.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.120+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f93cc06b2b0 con 0x7f93cc15b750 2023-08-25T09:48:11.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.349+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f93cc06b2b0 con 0x7f93cc15b750 2023-08-25T09:48:11.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.858+0000 7f93c2ffd640 1 -- 172.21.15.70:0/2572299050 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7f93cc06b2b0 con 0x7f93cc15b750 2023-08-25T09:48:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f93ac06a050 msgr2=0x7f93ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:11.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f93ac06a050 0x7f93ac06c510 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f93bc006a30 tx=0x7f93bc008040 comp rx=0 tx=0).stop 2023-08-25T09:48:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 msgr2=0x7f93cc1f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 0x7f93cc1f15c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0c5360 tx=0x7f93c4004770 comp rx=0 tx=0).stop 2023-08-25T09:48:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 shutdown_connections 2023-08-25T09:48:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f93ac06a050 0x7f93ac06c510 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f93cc15b750 0x7f93cc1f15c0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f93cc0710a0 0x7f93cc06bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 --2- 172.21.15.70:0/2572299050 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f93b8003fa0 0x7f93cc0c1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 >> 172.21.15.70:0/2572299050 conn(0x7f93cc0ce810 msgr2=0x7f93cc0b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 shutdown_connections 2023-08-25T09:48:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:11.862+0000 7f93cb59e640 1 -- 172.21.15.70:0/2572299050 wait complete. 2023-08-25T09:48:11.887 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:48:11.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:48:12.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.097+0000 7f2edce56640 1 Processor -- start 2023-08-25T09:48:12.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.097+0000 7f2edce56640 1 -- start start 2023-08-25T09:48:12.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.098+0000 7f2edce56640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ed8158fe0 0x7f2ed80cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.098+0000 7f2edce56640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2ed80cf9f0 0x7f2ed80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.098+0000 7f2edce56640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ed80cebe0 con 0x7f2ed80ce790 2023-08-25T09:48:12.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.098+0000 7f2edce56640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed80cd8f0 con 0x7f2ed8158fe0 2023-08-25T09:48:12.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.098+0000 7f2edce56640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ed806f380 con 0x7f2ed80cf9f0 2023-08-25T09:48:12.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.098+0000 7f2ed77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ed8158fe0 0x7f2ed80cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.099+0000 7f2ed77fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2ed8158fe0 0x7f2ed80cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44574/0 (socket says 172.21.15.70:44574) 2023-08-25T09:48:12.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.099+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2895626018 learned_addr learned my addr 172.21.15.70:0/2895626018 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:12.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.099+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2895626018 >> v2:172.21.15.155:3300/0 conn(0x7f2ed80cf9f0 msgr2=0x7f2ed80ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:12.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.099+0000 7f2ed77fe640 1 --2- 172.21.15.70:0/2895626018 >> v2:172.21.15.155:3300/0 conn(0x7f2ed80cf9f0 0x7f2ed80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.099+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2895626018 >> v1:172.21.15.155:6789/0 conn(0x7f2ed80ce790 legacy=0x7f2ed815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:12.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.099+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2895626018 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed81593c0 con 0x7f2ed8158fe0 2023-08-25T09:48:12.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.100+0000 7f2ed77fe640 1 --2- 172.21.15.70:0/2895626018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8158fe0 0x7f2ed80cc9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cb790 tx=0x7f2ec802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b99c3a3698ff832 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:12.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.100+0000 7f2ed6ffd640 1 -- 172.21.15.70:0/2895626018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec802ee00 con 0x7f2ed8158fe0 2023-08-25T09:48:12.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.100+0000 7f2ed6ffd640 1 -- 172.21.15.70:0/2895626018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8036440 con 0x7f2ed8158fe0 2023-08-25T09:48:12.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.100+0000 7f2ed6ffd640 1 -- 172.21.15.70:0/2895626018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80367d0 con 0x7f2ed8158fe0 2023-08-25T09:48:12.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 -- 172.21.15.70:0/2895626018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8158fe0 msgr2=0x7f2ed80cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 --2- 172.21.15.70:0/2895626018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8158fe0 0x7f2ed80cc9c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80cb790 tx=0x7f2ec802e050 comp rx=0 tx=0).stop 2023-08-25T09:48:12.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 -- 172.21.15.70:0/2895626018 shutdown_connections 2023-08-25T09:48:12.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 --2- 172.21.15.70:0/2895626018 >> v2:172.21.15.155:3300/0 conn(0x7f2ed80cf9f0 0x7f2ed80ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 --2- 172.21.15.70:0/2895626018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed8158fe0 0x7f2ed80cc9c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 -- 172.21.15.70:0/2895626018 >> 172.21.15.70:0/2895626018 conn(0x7f2ed80cedf0 msgr2=0x7f2ed80cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:12.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 -- 172.21.15.70:0/2895626018 shutdown_connections 2023-08-25T09:48:12.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.101+0000 7f2edce56640 1 -- 172.21.15.70:0/2895626018 wait complete. 2023-08-25T09:48:12.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.103+0000 7f2edce56640 1 Processor -- start 2023-08-25T09:48:12.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.103+0000 7f2edce56640 1 -- start start 2023-08-25T09:48:12.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2edce56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 0x7f2ed814a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2edce56640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed80cf9f0 0x7f2ed8147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2edce56640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed8158fe0 0x7f2ed814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2edce56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8149c50 con 0x7f2ed80ce790 2023-08-25T09:48:12.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2edce56640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed814b910 con 0x7f2ed8158fe0 2023-08-25T09:48:12.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2edce56640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed8148fd0 con 0x7f2ed80cf9f0 2023-08-25T09:48:12.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2ed77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 0x7f2ed814a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2ed77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 0x7f2ed814a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44586/0 (socket says 172.21.15.70:44586) 2023-08-25T09:48:12.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.104+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2046833787 learned_addr learned my addr 172.21.15.70:0/2046833787 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:12.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed80cf9f0 msgr2=0x7f2ed8147ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed77fe640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed80cf9f0 0x7f2ed8147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed8158fe0 msgr2=0x7f2ed814d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed7fff640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed8158fe0 0x7f2ed814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed77fe640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed8158fe0 0x7f2ed814d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed77fe640 1 -- 172.21.15.70:0/2046833787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec800b770 con 0x7f2ed80ce790 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed7fff640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed8158fe0 0x7f2ed814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed77fe640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 0x7f2ed814a070 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2ec802e090 tx=0x7f2ec8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034950 con 0x7f2ed80ce790 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.105+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec8034ab0 con 0x7f2ed80ce790 2023-08-25T09:48:12.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.106+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8034e60 con 0x7f2ed80ce790 2023-08-25T09:48:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.106+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed81492b0 con 0x7f2ed80ce790 2023-08-25T09:48:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.106+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed8149870 con 0x7f2ed80ce790 2023-08-25T09:48:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.107+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed8149ec0 con 0x7f2ed80ce790 2023-08-25T09:48:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.108+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2ec8041480 con 0x7f2ed80ce790 2023-08-25T09:48:12.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.108+0000 7f2ed4ff9640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2eb806a050 0x7f2eb806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.109+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ec80ab740 con 0x7f2ed80ce790 2023-08-25T09:48:12.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.109+0000 7f2ecffff640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2eb806a050 0x7f2eb806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.109+0000 7f2ecffff640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2eb806a050 0x7f2eb806c510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80c6e40 tx=0x7f2ec0006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:12.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.113+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ec8076e20 con 0x7f2ed80ce790 2023-08-25T09:48:12.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.348+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ed80cebe0 con 0x7f2ed80ce790 2023-08-25T09:48:12.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.349+0000 7f2ed4ff9640 1 -- 172.21.15.70:0/2046833787 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f2ed80cebe0 con 0x7f2ed80ce790 2023-08-25T09:48:12.352 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:12.353 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:48:12.353 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:48:12.354 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2eb806a050 msgr2=0x7f2eb806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2eb806a050 0x7f2eb806c510 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2ed80c6e40 tx=0x7f2ec0006c90 comp rx=0 tx=0).stop 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 msgr2=0x7f2ed814a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 0x7f2ed814a070 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f2ec802e090 tx=0x7f2ec8004730 comp rx=0 tx=0).stop 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 shutdown_connections 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2eb806a050 0x7f2eb806c510 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2ed8158fe0 0x7f2ed814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.355 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2ed80cf9f0 0x7f2ed8147ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 --2- 172.21.15.70:0/2046833787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2ed80ce790 0x7f2ed814a070 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 >> 172.21.15.70:0/2046833787 conn(0x7f2ed80cedf0 msgr2=0x7f2ed80cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:12.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 shutdown_connections 2023-08-25T09:48:12.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.352+0000 7f2edce56640 1 -- 172.21.15.70:0/2046833787 wait complete. 2023-08-25T09:48:12.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:48:12.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.578+0000 7fba9855b640 1 Processor -- start 2023-08-25T09:48:12.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.578+0000 7fba9855b640 1 -- start start 2023-08-25T09:48:12.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.578+0000 7fba9855b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 0x7fba9006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.578+0000 7fba9855b640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba901620b0 con 0x7fba900ce5f0 2023-08-25T09:48:12.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.578+0000 7fba9855b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba90162390 con 0x7fba90161ae0 2023-08-25T09:48:12.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.578+0000 7fba9855b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9006ad70 con 0x7fba90160ff0 2023-08-25T09:48:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.579+0000 7fba95af6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fba900ce5f0 0x7fba901613d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57308/0 (socket says 172.21.15.70:57308) 2023-08-25T09:48:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.579+0000 7fba95af6640 1 -- 172.21.15.70:0/414277166 learned_addr learned my addr 172.21.15.70:0/414277166 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.579+0000 7fba962f7640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 0x7fba9006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.579+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843940326 0 0) 0x7fba901620b0 con 0x7fba900ce5f0 2023-08-25T09:48:12.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.579+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba7c003520 con 0x7fba900ce5f0 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683284559 0 0) 0x7fba7c003520 con 0x7fba900ce5f0 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 msgr2=0x7fba9006b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 0x7fba9006b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 >> v1:172.21.15.155:6789/0 conn(0x7fba90161ae0 legacy=0x7fba90162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9006af80 con 0x7fba900ce5f0 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fba80001670 con 0x7fba900ce5f0 2023-08-25T09:48:12.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 >> v1:172.21.15.70:6789/0 conn(0x7fba900ce5f0 legacy=0x7fba901613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba7c0039a0 0x7fba7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba7c0047c0 0x7fba7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 0x7fba7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9006ad70 con 0x7fba7c0039a0 2023-08-25T09:48:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c00ba30 con 0x7fba7c0047c0 2023-08-25T09:48:12.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c00bbf0 con 0x7fba90160ff0 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.580+0000 7fba962f7640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba7c0039a0 0x7fba7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.581+0000 7fba95af6640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba7c0047c0 0x7fba7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.581+0000 7fba962f7640 1 -- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 msgr2=0x7fba7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.581+0000 7fba962f7640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba90160ff0 0x7fba7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.581+0000 7fba962f7640 1 -- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba7c0047c0 msgr2=0x7fba7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.581+0000 7fba962f7640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba7c0047c0 0x7fba7c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.581+0000 7fba962f7640 1 -- 172.21.15.70:0/414277166 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c003520 con 0x7fba7c0039a0 2023-08-25T09:48:12.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.582+0000 7fba962f7640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba7c0039a0 0x7fba7c003dc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009a90 tx=0x7fba7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9987c802dcc8a36 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:12.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.582+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c02c6b0 con 0x7fba7c0039a0 2023-08-25T09:48:12.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.582+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c02c810 con 0x7fba7c0039a0 2023-08-25T09:48:12.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.582+0000 7fba952f5640 1 -- 172.21.15.70:0/414277166 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c02cba0 con 0x7fba7c0039a0 2023-08-25T09:48:12.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba9855b640 1 -- 172.21.15.70:0/414277166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba7c0039a0 msgr2=0x7fba7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba9855b640 1 --2- 172.21.15.70:0/414277166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba7c0039a0 0x7fba7c003dc0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fba8c009a90 tx=0x7fba7c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba96af8640 1 -- 172.21.15.70:0/414277166 reap_dead start 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba9855b640 1 -- 172.21.15.70:0/414277166 shutdown_connections 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba9855b640 1 -- 172.21.15.70:0/414277166 >> 172.21.15.70:0/414277166 conn(0x7fba90069850 msgr2=0x7fba90069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba9855b640 1 -- 172.21.15.70:0/414277166 shutdown_connections 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.583+0000 7fba9855b640 1 -- 172.21.15.70:0/414277166 wait complete. 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.585+0000 7fba9855b640 1 Processor -- start 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.585+0000 7fba9855b640 1 -- start start 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba9855b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba7c0039a0 0x7fba901434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba9855b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 0x7fba900ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba9855b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba901457e0 0x7fba90159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba9855b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba900af5e0 con 0x7fba90146180 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba9855b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba90143a20 con 0x7fba901457e0 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba9855b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba900ade40 con 0x7fba7c0039a0 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba95af6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 0x7fba900ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba95af6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 0x7fba900ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44616/0 (socket says 172.21.15.70:44616) 2023-08-25T09:48:12.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba95af6640 1 -- 172.21.15.70:0/2634230794 learned_addr learned my addr 172.21.15.70:0/2634230794 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba962f7640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba7c0039a0 0x7fba901434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.586+0000 7fba95af6640 1 -- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba7c0039a0 msgr2=0x7fba901434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba95af6640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba7c0039a0 0x7fba901434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba95af6640 1 -- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba901457e0 msgr2=0x7fba90159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba95af6640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba901457e0 0x7fba90159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba95af6640 1 -- 172.21.15.70:0/2634230794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba900af810 con 0x7fba90146180 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba95af6640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 0x7fba900ad900 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d750 tx=0x7fba8000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:12.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba800074e0 con 0x7fba90146180 2023-08-25T09:48:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba80007640 con 0x7fba90146180 2023-08-25T09:48:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba80004a20 con 0x7fba90146180 2023-08-25T09:48:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba900afaf0 con 0x7fba90146180 2023-08-25T09:48:12.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.587+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba900b00b0 con 0x7fba90146180 2023-08-25T09:48:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.588+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9004e420 con 0x7fba90146180 2023-08-25T09:48:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.589+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fba80004b80 con 0x7fba90146180 2023-08-25T09:48:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.590+0000 7fba86ffd640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba7406a050 0x7fba7406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.590+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba80087fe0 con 0x7fba90146180 2023-08-25T09:48:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.590+0000 7fba962f7640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba7406a050 0x7fba7406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:12.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.591+0000 7fba962f7640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba7406a050 0x7fba7406c510 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fba8c036000 tx=0x7fba8c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:12.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.595+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fba800535a0 con 0x7fba90146180 2023-08-25T09:48:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.786+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba9006c8b0 con 0x7fba90146180 2023-08-25T09:48:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.788+0000 7fba86ffd640 1 -- 172.21.15.70:0/2634230794 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+18309 (secure 0 0 0) 0x7fba80019070 con 0x7fba90146180 2023-08-25T09:48:12.793 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:12.793 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":165,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:48:09.412707+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:48:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba7406a050 msgr2=0x7fba7406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba7406a050 0x7fba7406c510 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fba8c036000 tx=0x7fba8c030000 comp rx=0 tx=0).stop 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 msgr2=0x7fba900ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 0x7fba900ad900 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d750 tx=0x7fba8000dc10 comp rx=0 tx=0).stop 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 shutdown_connections 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba7406a050 0x7fba7406c510 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fba901457e0 0x7fba90159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba90146180 0x7fba900ad900 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 --2- 172.21.15.70:0/2634230794 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba7c0039a0 0x7fba901434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 >> 172.21.15.70:0/2634230794 conn(0x7fba90069850 msgr2=0x7fba9015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 shutdown_connections 2023-08-25T09:48:12.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:12.791+0000 7fba9855b640 1 -- 172.21.15.70:0/2634230794 wait complete. 2023-08-25T09:48:12.811 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:48:12.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:48:13.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.013+0000 7f5e4f82d640 1 Processor -- start 2023-08-25T09:48:13.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.013+0000 7f5e4f82d640 1 -- start start 2023-08-25T09:48:13.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4f82d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e4815e830 0x7f5e4806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4f82d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e4815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4f82d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480b1b60 con 0x7f5e4815c6d0 2023-08-25T09:48:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4f82d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4806bd80 con 0x7f5e4815e830 2023-08-25T09:48:13.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4f82d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4806b7c0 con 0x7f5e4806b2b0 2023-08-25T09:48:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e4815e830 0x7f5e4806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5e4815e830 0x7f5e4806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44624/0 (socket says 172.21.15.70:44624) 2023-08-25T09:48:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 -- 172.21.15.70:0/2723305590 learned_addr learned my addr 172.21.15.70:0/2723305590 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4cdc8640 1 --2- 172.21.15.70:0/2723305590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e4815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 -- 172.21.15.70:0/2723305590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 msgr2=0x7f5e4815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:13.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 --2- 172.21.15.70:0/2723305590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e4815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 -- 172.21.15.70:0/2723305590 >> v1:172.21.15.155:6789/0 conn(0x7f5e4815c6d0 legacy=0x7f5e480b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:13.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.015+0000 7f5e4d5c9640 1 -- 172.21.15.70:0/2723305590 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4806baa0 con 0x7f5e4815e830 2023-08-25T09:48:13.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.016+0000 7f5e4d5c9640 1 --2- 172.21.15.70:0/2723305590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815e830 0x7f5e4806ad70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e3802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619e8a195880629a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:13.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.017+0000 7f5e47fff640 1 -- 172.21.15.70:0/2723305590 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38031640 con 0x7f5e4815e830 2023-08-25T09:48:13.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.017+0000 7f5e47fff640 1 -- 172.21.15.70:0/2723305590 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e380317a0 con 0x7f5e4815e830 2023-08-25T09:48:13.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.017+0000 7f5e47fff640 1 -- 172.21.15.70:0/2723305590 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38031b30 con 0x7f5e4815e830 2023-08-25T09:48:13.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.017+0000 7f5e4f82d640 1 -- 172.21.15.70:0/2723305590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815e830 msgr2=0x7f5e4806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.017+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/2723305590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815e830 0x7f5e4806ad70 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e3802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.018+0000 7f5e4f82d640 1 -- 172.21.15.70:0/2723305590 shutdown_connections 2023-08-25T09:48:13.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.018+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/2723305590 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e4815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.018+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/2723305590 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815e830 0x7f5e4806ad70 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.018+0000 7f5e4f82d640 1 -- 172.21.15.70:0/2723305590 >> 172.21.15.70:0/2723305590 conn(0x7f5e48069850 msgr2=0x7f5e48069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:13.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.018+0000 7f5e4f82d640 1 -- 172.21.15.70:0/2723305590 shutdown_connections 2023-08-25T09:48:13.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.018+0000 7f5e4f82d640 1 -- 172.21.15.70:0/2723305590 wait complete. 2023-08-25T09:48:13.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.019+0000 7f5e4f82d640 1 Processor -- start 2023-08-25T09:48:13.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.019+0000 7f5e4f82d640 1 -- start start 2023-08-25T09:48:13.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.020+0000 7f5e4f82d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e48150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.020+0000 7f5e4f82d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 0x7f5e480c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.020+0000 7f5e4f82d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4815e830 0x7f5e4815cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.020+0000 7f5e4f82d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48070660 con 0x7f5e4815c6d0 2023-08-25T09:48:13.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.020+0000 7f5e4f82d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48150850 con 0x7f5e4815e830 2023-08-25T09:48:13.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.020+0000 7f5e4f82d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4806fa50 con 0x7f5e4806b2b0 2023-08-25T09:48:13.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 0x7f5e480c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 0x7f5e480c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44638/0 (socket says 172.21.15.70:44638) 2023-08-25T09:48:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 -- 172.21.15.70:0/3126363641 learned_addr learned my addr 172.21.15.70:0/3126363641 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 -- 172.21.15.70:0/3126363641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 msgr2=0x7f5e48150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e48150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 -- 172.21.15.70:0/3126363641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4815e830 msgr2=0x7f5e4815cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4815e830 0x7f5e4815cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 -- 172.21.15.70:0/3126363641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380094d0 con 0x7f5e4815c6d0 2023-08-25T09:48:13.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4cdc8640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 0x7f5e480c8250 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c7f0 tx=0x7f5e3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:13.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c004220 con 0x7f5e4815c6d0 2023-08-25T09:48:13.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c004380 con 0x7f5e4815c6d0 2023-08-25T09:48:13.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c007840 con 0x7f5e4815c6d0 2023-08-25T09:48:13.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.021+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4806fd30 con 0x7f5e4815c6d0 2023-08-25T09:48:13.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.022+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480702f0 con 0x7f5e4815c6d0 2023-08-25T09:48:13.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.023+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4804e420 con 0x7f5e4815c6d0 2023-08-25T09:48:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.023+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e3c007bb0 con 0x7f5e4815c6d0 2023-08-25T09:48:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.024+0000 7f5e45ffb640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e3406a050 0x7f5e3406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.024+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e3c088560 con 0x7f5e4815c6d0 2023-08-25T09:48:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.025+0000 7f5e4d5c9640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e3406a050 0x7f5e3406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.025+0000 7f5e4d5c9640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e3406a050 0x7f5e3406c510 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5e38036000 tx=0x7f5e38030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.029+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e3c053c50 con 0x7f5e4815c6d0 2023-08-25T09:48:13.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.212+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e480b17f0 con 0x7f5e3406a050 2023-08-25T09:48:13.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.227+0000 7f5e45ffb640 1 -- 172.21.15.70:0/3126363641 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303412 (secure 0 0 0) 0x7f5e38031a10 con 0x7f5e3406a050 2023-08-25T09:48:13.234 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:13.235 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:48:13.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e3406a050 msgr2=0x7f5e3406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e3406a050 0x7f5e3406c510 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5e38036000 tx=0x7f5e38030000 comp rx=0 tx=0).stop 2023-08-25T09:48:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 msgr2=0x7f5e480c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 0x7f5e480c8250 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c7f0 tx=0x7f5e3c00ccb0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 shutdown_connections 2023-08-25T09:48:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5e3406a050 0x7f5e3406c510 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5e4815e830 0x7f5e4815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5e4815c6d0 0x7f5e480c8250 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 --2- 172.21.15.70:0/3126363641 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5e4806b2b0 0x7f5e48150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 >> 172.21.15.70:0/3126363641 conn(0x7f5e48069850 msgr2=0x7f5e480b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 shutdown_connections 2023-08-25T09:48:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.233+0000 7f5e4f82d640 1 -- 172.21.15.70:0/3126363641 wait complete. 2023-08-25T09:48:13.254 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2023-08-25T09:48:11.506759+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553709,"statfs":{"total":1600000000,"available":1590998876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.727911"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446207+0000","last_change":"2023-08-25T09:48:08.445972+0000","last_active":"2023-08-25T09:48:08.446207+0000","last_peered":"2023-08-25T09:48:08.446207+0000","last_clean":"2023-08-25T09:48:08.446207+0000","last_became_active":"2023-08-25T09:48:08.444843+0000","last_became_peered":"2023-08-25T09:48:08.444843+0000","last_unstale":"2023-08-25T09:48:08.446207+0000","last_undegraded":"2023-08-25T09:48:08.446207+0000","last_fullsized":"2023-08-25T09:48:08.446207+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:22.285471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442914+0000","last_change":"2023-08-25T09:48:08.441613+0000","last_active":"2023-08-25T09:48:08.442914+0000","last_peered":"2023-08-25T09:48:08.442914+0000","last_clean":"2023-08-25T09:48:08.442914+0000","last_became_active":"2023-08-25T09:48:08.440821+0000","last_became_peered":"2023-08-25T09:48:08.440821+0000","last_unstale":"2023-08-25T09:48:08.442914+0000","last_undegraded":"2023-08-25T09:48:08.442914+0000","last_fullsized":"2023-08-25T09:48:08.442914+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:08:04.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447045+0000","last_change":"2023-08-25T09:48:08.446273+0000","last_active":"2023-08-25T09:48:08.447045+0000","last_peered":"2023-08-25T09:48:08.447045+0000","last_clean":"2023-08-25T09:48:08.447045+0000","last_became_active":"2023-08-25T09:48:08.444885+0000","last_became_peered":"2023-08-25T09:48:08.444885+0000","last_unstale":"2023-08-25T09:48:08.447045+0000","last_undegraded":"2023-08-25T09:48:08.447045+0000","last_fullsized":"2023-08-25T09:48:08.447045+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:15:12.785972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.449216+0000","last_change":"2023-08-25T09:48:08.448477+0000","last_active":"2023-08-25T09:48:08.449216+0000","last_peered":"2023-08-25T09:48:08.449216+0000","last_clean":"2023-08-25T09:48:08.449216+0000","last_became_active":"2023-08-25T09:48:08.443560+0000","last_became_peered":"2023-08-25T09:48:08.443560+0000","last_unstale":"2023-08-25T09:48:08.449216+0000","last_undegraded":"2023-08-25T09:48:08.449216+0000","last_fullsized":"2023-08-25T09:48:08.449216+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:00.991827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442914+0000","last_change":"2023-08-25T09:48:08.442812+0000","last_active":"2023-08-25T09:48:08.442914+0000","last_peered":"2023-08-25T09:48:08.442914+0000","last_clean":"2023-08-25T09:48:08.442914+0000","last_became_active":"2023-08-25T09:48:08.441305+0000","last_became_peered":"2023-08-25T09:48:08.441305+0000","last_unstale":"2023-08-25T09:48:08.442914+0000","last_undegraded":"2023-08-25T09:48:08.442914+0000","last_fullsized":"2023-08-25T09:48:08.442914+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:29.438853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.431332+0000","last_change":"2023-08-25T09:48:08.425993+0000","last_active":"2023-08-25T09:48:08.431332+0000","last_peered":"2023-08-25T09:48:08.431332+0000","last_clean":"2023-08-25T09:48:08.431332+0000","last_became_active":"2023-08-25T09:48:08.425208+0000","last_became_peered":"2023-08-25T09:48:08.425208+0000","last_unstale":"2023-08-25T09:48:08.431332+0000","last_undegraded":"2023-08-25T09:48:08.431332+0000","last_fullsized":"2023-08-25T09:48:08.431332+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:17:45.563574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441460+0000","last_change":"2023-08-25T09:48:08.441337+0000","last_active":"2023-08-25T09:48:08.441460+0000","last_peered":"2023-08-25T09:48:08.441460+0000","last_clean":"2023-08-25T09:48:08.441460+0000","last_became_active":"2023-08-25T09:48:08.440296+0000","last_became_peered":"2023-08-25T09:48:08.440296+0000","last_unstale":"2023-08-25T09:48:08.441460+0000","last_undegraded":"2023-08-25T09:48:08.441460+0000","last_fullsized":"2023-08-25T09:48:08.441460+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:52:34.591238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447029+0000","last_change":"2023-08-25T09:48:08.446084+0000","last_active":"2023-08-25T09:48:08.447029+0000","last_peered":"2023-08-25T09:48:08.447029+0000","last_clean":"2023-08-25T09:48:08.447029+0000","last_became_active":"2023-08-25T09:48:08.444444+0000","last_became_peered":"2023-08-25T09:48:08.444444+0000","last_unstale":"2023-08-25T09:48:08.447029+0000","last_undegraded":"2023-08-25T09:48:08.447029+0000","last_fullsized":"2023-08-25T09:48:08.447029+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:54:50.721713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446073+0000","last_change":"2023-08-25T09:48:08.445893+0000","last_active":"2023-08-25T09:48:08.446073+0000","last_peered":"2023-08-25T09:48:08.446073+0000","last_clean":"2023-08-25T09:48:08.446073+0000","last_became_active":"2023-08-25T09:48:08.442540+0000","last_became_peered":"2023-08-25T09:48:08.442540+0000","last_unstale":"2023-08-25T09:48:08.446073+0000","last_undegraded":"2023-08-25T09:48:08.446073+0000","last_fullsized":"2023-08-25T09:48:08.446073+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:43:52.258967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443302+0000","last_change":"2023-08-25T09:48:08.443111+0000","last_active":"2023-08-25T09:48:08.443302+0000","last_peered":"2023-08-25T09:48:08.443302+0000","last_clean":"2023-08-25T09:48:08.443302+0000","last_became_active":"2023-08-25T09:48:08.441724+0000","last_became_peered":"2023-08-25T09:48:08.441724+0000","last_unstale":"2023-08-25T09:48:08.443302+0000","last_undegraded":"2023-08-25T09:48:08.443302+0000","last_fullsized":"2023-08-25T09:48:08.443302+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:17.179847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450353+0000","last_change":"2023-08-25T09:48:08.450278+0000","last_active":"2023-08-25T09:48:08.450353+0000","last_peered":"2023-08-25T09:48:08.450353+0000","last_clean":"2023-08-25T09:48:08.450353+0000","last_became_active":"2023-08-25T09:48:08.450053+0000","last_became_peered":"2023-08-25T09:48:08.450053+0000","last_unstale":"2023-08-25T09:48:08.450353+0000","last_undegraded":"2023-08-25T09:48:08.450353+0000","last_fullsized":"2023-08-25T09:48:08.450353+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:49:07.032545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442387+0000","last_change":"2023-08-25T09:48:08.442270+0000","last_active":"2023-08-25T09:48:08.442387+0000","last_peered":"2023-08-25T09:48:08.442387+0000","last_clean":"2023-08-25T09:48:08.442387+0000","last_became_active":"2023-08-25T09:48:08.440508+0000","last_became_peered":"2023-08-25T09:48:08.440508+0000","last_unstale":"2023-08-25T09:48:08.442387+0000","last_undegraded":"2023-08-25T09:48:08.442387+0000","last_fullsized":"2023-08-25T09:48:08.442387+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:51:18.775838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436209+0000","last_change":"2023-08-25T09:48:08.436087+0000","last_active":"2023-08-25T09:48:08.436209+0000","last_peered":"2023-08-25T09:48:08.436209+0000","last_clean":"2023-08-25T09:48:08.436209+0000","last_became_active":"2023-08-25T09:48:08.435677+0000","last_became_peered":"2023-08-25T09:48:08.435677+0000","last_unstale":"2023-08-25T09:48:08.436209+0000","last_undegraded":"2023-08-25T09:48:08.436209+0000","last_fullsized":"2023-08-25T09:48:08.436209+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:35:51.412561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450285+0000","last_change":"2023-08-25T09:48:08.450196+0000","last_active":"2023-08-25T09:48:08.450285+0000","last_peered":"2023-08-25T09:48:08.450285+0000","last_clean":"2023-08-25T09:48:08.450285+0000","last_became_active":"2023-08-25T09:48:08.450053+0000","last_became_peered":"2023-08-25T09:48:08.450053+0000","last_unstale":"2023-08-25T09:48:08.450285+0000","last_undegraded":"2023-08-25T09:48:08.450285+0000","last_fullsized":"2023-08-25T09:48:08.450285+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:33.877843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439971+0000","last_change":"2023-08-25T09:48:08.439832+0000","last_active":"2023-08-25T09:48:08.439971+0000","last_peered":"2023-08-25T09:48:08.439971+0000","last_clean":"2023-08-25T09:48:08.439971+0000","last_became_active":"2023-08-25T09:48:08.439445+0000","last_became_peered":"2023-08-25T09:48:08.439445+0000","last_unstale":"2023-08-25T09:48:08.439971+0000","last_undegraded":"2023-08-25T09:48:08.439971+0000","last_fullsized":"2023-08-25T09:48:08.439971+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:58.494293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.445961+0000","last_change":"2023-08-25T09:48:08.445458+0000","last_active":"2023-08-25T09:48:08.445961+0000","last_peered":"2023-08-25T09:48:08.445961+0000","last_clean":"2023-08-25T09:48:08.445961+0000","last_became_active":"2023-08-25T09:48:08.444539+0000","last_became_peered":"2023-08-25T09:48:08.444539+0000","last_unstale":"2023-08-25T09:48:08.445961+0000","last_undegraded":"2023-08-25T09:48:08.445961+0000","last_fullsized":"2023-08-25T09:48:08.445961+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:36:47.168629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.437939+0000","last_change":"2023-08-25T09:48:08.436562+0000","last_active":"2023-08-25T09:48:08.437939+0000","last_peered":"2023-08-25T09:48:08.437939+0000","last_clean":"2023-08-25T09:48:08.437939+0000","last_became_active":"2023-08-25T09:48:08.432548+0000","last_became_peered":"2023-08-25T09:48:08.432548+0000","last_unstale":"2023-08-25T09:48:08.437939+0000","last_undegraded":"2023-08-25T09:48:08.437939+0000","last_fullsized":"2023-08-25T09:48:08.437939+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:09.453496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444248+0000","last_change":"2023-08-25T09:48:08.443617+0000","last_active":"2023-08-25T09:48:08.444248+0000","last_peered":"2023-08-25T09:48:08.444248+0000","last_clean":"2023-08-25T09:48:08.444248+0000","last_became_active":"2023-08-25T09:48:08.443058+0000","last_became_peered":"2023-08-25T09:48:08.443058+0000","last_unstale":"2023-08-25T09:48:08.444248+0000","last_undegraded":"2023-08-25T09:48:08.444248+0000","last_fullsized":"2023-08-25T09:48:08.444248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:32:46.687002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450224+0000","last_change":"2023-08-25T09:48:08.450148+0000","last_active":"2023-08-25T09:48:08.450224+0000","last_peered":"2023-08-25T09:48:08.450224+0000","last_clean":"2023-08-25T09:48:08.450224+0000","last_became_active":"2023-08-25T09:48:08.449893+0000","last_became_peered":"2023-08-25T09:48:08.449893+0000","last_unstale":"2023-08-25T09:48:08.450224+0000","last_undegraded":"2023-08-25T09:48:08.450224+0000","last_fullsized":"2023-08-25T09:48:08.450224+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:07.891935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442801+0000","last_change":"2023-08-25T09:48:08.442695+0000","last_active":"2023-08-25T09:48:08.442801+0000","last_peered":"2023-08-25T09:48:08.442801+0000","last_clean":"2023-08-25T09:48:08.442801+0000","last_became_active":"2023-08-25T09:48:08.442347+0000","last_became_peered":"2023-08-25T09:48:08.442347+0000","last_unstale":"2023-08-25T09:48:08.442801+0000","last_undegraded":"2023-08-25T09:48:08.442801+0000","last_fullsized":"2023-08-25T09:48:08.442801+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:13:26.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447602+0000","last_change":"2023-08-25T09:48:08.442351+0000","last_active":"2023-08-25T09:48:08.447602+0000","last_peered":"2023-08-25T09:48:08.447602+0000","last_clean":"2023-08-25T09:48:08.447602+0000","last_became_active":"2023-08-25T09:48:08.442041+0000","last_became_peered":"2023-08-25T09:48:08.442041+0000","last_unstale":"2023-08-25T09:48:08.447602+0000","last_undegraded":"2023-08-25T09:48:08.447602+0000","last_fullsized":"2023-08-25T09:48:08.447602+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:56.775903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443826+0000","last_change":"2023-08-25T09:48:08.443646+0000","last_active":"2023-08-25T09:48:08.443826+0000","last_peered":"2023-08-25T09:48:08.443826+0000","last_clean":"2023-08-25T09:48:08.443826+0000","last_became_active":"2023-08-25T09:48:08.443378+0000","last_became_peered":"2023-08-25T09:48:08.443378+0000","last_unstale":"2023-08-25T09:48:08.443826+0000","last_undegraded":"2023-08-25T09:48:08.443826+0000","last_fullsized":"2023-08-25T09:48:08.443826+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:27:21.865053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448157+0000","last_change":"2023-08-25T09:48:08.448088+0000","last_active":"2023-08-25T09:48:08.448157+0000","last_peered":"2023-08-25T09:48:08.448157+0000","last_clean":"2023-08-25T09:48:08.448157+0000","last_became_active":"2023-08-25T09:48:08.447687+0000","last_became_peered":"2023-08-25T09:48:08.447687+0000","last_unstale":"2023-08-25T09:48:08.448157+0000","last_undegraded":"2023-08-25T09:48:08.448157+0000","last_fullsized":"2023-08-25T09:48:08.448157+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:10:58.564019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.445422+0000","last_change":"2023-08-25T09:48:08.445300+0000","last_active":"2023-08-25T09:48:08.445422+0000","last_peered":"2023-08-25T09:48:08.445422+0000","last_clean":"2023-08-25T09:48:08.445422+0000","last_became_active":"2023-08-25T09:48:08.441646+0000","last_became_peered":"2023-08-25T09:48:08.441646+0000","last_unstale":"2023-08-25T09:48:08.445422+0000","last_undegraded":"2023-08-25T09:48:08.445422+0000","last_fullsized":"2023-08-25T09:48:08.445422+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:59:28.925816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446998+0000","last_change":"2023-08-25T09:48:08.445279+0000","last_active":"2023-08-25T09:48:08.446998+0000","last_peered":"2023-08-25T09:48:08.446998+0000","last_clean":"2023-08-25T09:48:08.446998+0000","last_became_active":"2023-08-25T09:48:08.438573+0000","last_became_peered":"2023-08-25T09:48:08.438573+0000","last_unstale":"2023-08-25T09:48:08.446998+0000","last_undegraded":"2023-08-25T09:48:08.446998+0000","last_fullsized":"2023-08-25T09:48:08.446998+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:53.098145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450154+0000","last_change":"2023-08-25T09:48:08.450042+0000","last_active":"2023-08-25T09:48:08.450154+0000","last_peered":"2023-08-25T09:48:08.450154+0000","last_clean":"2023-08-25T09:48:08.450154+0000","last_became_active":"2023-08-25T09:48:08.449229+0000","last_became_peered":"2023-08-25T09:48:08.449229+0000","last_unstale":"2023-08-25T09:48:08.450154+0000","last_undegraded":"2023-08-25T09:48:08.450154+0000","last_fullsized":"2023-08-25T09:48:08.450154+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:50.738200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444328+0000","last_change":"2023-08-25T09:48:08.444233+0000","last_active":"2023-08-25T09:48:08.444328+0000","last_peered":"2023-08-25T09:48:08.444328+0000","last_clean":"2023-08-25T09:48:08.444328+0000","last_became_active":"2023-08-25T09:48:08.434945+0000","last_became_peered":"2023-08-25T09:48:08.434945+0000","last_unstale":"2023-08-25T09:48:08.444328+0000","last_undegraded":"2023-08-25T09:48:08.444328+0000","last_fullsized":"2023-08-25T09:48:08.444328+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:58:01.789008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.425544+0000","last_change":"2023-08-25T09:48:08.425364+0000","last_active":"2023-08-25T09:48:08.425544+0000","last_peered":"2023-08-25T09:48:08.425544+0000","last_clean":"2023-08-25T09:48:08.425544+0000","last_became_active":"2023-08-25T09:48:08.424671+0000","last_became_peered":"2023-08-25T09:48:08.424671+0000","last_unstale":"2023-08-25T09:48:08.425544+0000","last_undegraded":"2023-08-25T09:48:08.425544+0000","last_fullsized":"2023-08-25T09:48:08.425544+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:11:06.014011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446058+0000","last_change":"2023-08-25T09:48:08.445865+0000","last_active":"2023-08-25T09:48:08.446058+0000","last_peered":"2023-08-25T09:48:08.446058+0000","last_clean":"2023-08-25T09:48:08.446058+0000","last_became_active":"2023-08-25T09:48:08.445501+0000","last_became_peered":"2023-08-25T09:48:08.445501+0000","last_unstale":"2023-08-25T09:48:08.446058+0000","last_undegraded":"2023-08-25T09:48:08.446058+0000","last_fullsized":"2023-08-25T09:48:08.446058+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:22:31.286612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450678+0000","last_change":"2023-08-25T09:48:08.450569+0000","last_active":"2023-08-25T09:48:08.450678+0000","last_peered":"2023-08-25T09:48:08.450678+0000","last_clean":"2023-08-25T09:48:08.450678+0000","last_became_active":"2023-08-25T09:48:08.450310+0000","last_became_peered":"2023-08-25T09:48:08.450310+0000","last_unstale":"2023-08-25T09:48:08.450678+0000","last_undegraded":"2023-08-25T09:48:08.450678+0000","last_fullsized":"2023-08-25T09:48:08.450678+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:18:18.990796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441703+0000","last_change":"2023-08-25T09:48:08.441589+0000","last_active":"2023-08-25T09:48:08.441703+0000","last_peered":"2023-08-25T09:48:08.441703+0000","last_clean":"2023-08-25T09:48:08.441703+0000","last_became_active":"2023-08-25T09:48:08.441284+0000","last_became_peered":"2023-08-25T09:48:08.441284+0000","last_unstale":"2023-08-25T09:48:08.441703+0000","last_undegraded":"2023-08-25T09:48:08.441703+0000","last_fullsized":"2023-08-25T09:48:08.441703+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:45:02.619368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441011+0000","last_change":"2023-08-25T09:48:08.440904+0000","last_active":"2023-08-25T09:48:08.441011+0000","last_peered":"2023-08-25T09:48:08.441011+0000","last_clean":"2023-08-25T09:48:08.441011+0000","last_became_active":"2023-08-25T09:48:08.440508+0000","last_became_peered":"2023-08-25T09:48:08.440508+0000","last_unstale":"2023-08-25T09:48:08.441011+0000","last_undegraded":"2023-08-25T09:48:08.441011+0000","last_fullsized":"2023-08-25T09:48:08.441011+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:03.327396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446978+0000","last_change":"2023-08-25T09:48:08.445129+0000","last_active":"2023-08-25T09:48:08.446978+0000","last_peered":"2023-08-25T09:48:08.446978+0000","last_clean":"2023-08-25T09:48:08.446978+0000","last_became_active":"2023-08-25T09:48:08.438083+0000","last_became_peered":"2023-08-25T09:48:08.438083+0000","last_unstale":"2023-08-25T09:48:08.446978+0000","last_undegraded":"2023-08-25T09:48:08.446978+0000","last_fullsized":"2023-08-25T09:48:08.446978+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:58:27.581262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"164'4","reported_seq":15,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846593+0000","last_change":"2023-08-25T09:48:07.808083+0000","last_active":"2023-08-25T09:48:08.846593+0000","last_peered":"2023-08-25T09:48:08.846593+0000","last_clean":"2023-08-25T09:48:08.846593+0000","last_became_active":"2023-08-25T09:48:07.807728+0000","last_became_peered":"2023-08-25T09:48:07.807728+0000","last_unstale":"2023-08-25T09:48:08.846593+0000","last_undegraded":"2023-08-25T09:48:08.846593+0000","last_fullsized":"2023-08-25T09:48:08.846593+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:50.560573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448296+0000","last_change":"2023-08-25T09:48:08.448179+0000","last_active":"2023-08-25T09:48:08.448296+0000","last_peered":"2023-08-25T09:48:08.448296+0000","last_clean":"2023-08-25T09:48:08.448296+0000","last_became_active":"2023-08-25T09:48:08.446644+0000","last_became_peered":"2023-08-25T09:48:08.446644+0000","last_unstale":"2023-08-25T09:48:08.448296+0000","last_undegraded":"2023-08-25T09:48:08.448296+0000","last_fullsized":"2023-08-25T09:48:08.448296+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:27:51.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.847133+0000","last_change":"2023-08-25T09:48:07.814371+0000","last_active":"2023-08-25T09:48:08.847133+0000","last_peered":"2023-08-25T09:48:08.847133+0000","last_clean":"2023-08-25T09:48:08.847133+0000","last_became_active":"2023-08-25T09:48:07.813571+0000","last_became_peered":"2023-08-25T09:48:07.813571+0000","last_unstale":"2023-08-25T09:48:08.847133+0000","last_undegraded":"2023-08-25T09:48:08.847133+0000","last_fullsized":"2023-08-25T09:48:08.847133+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:26.094932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447990+0000","last_change":"2023-08-25T09:48:08.447912+0000","last_active":"2023-08-25T09:48:08.447990+0000","last_peered":"2023-08-25T09:48:08.447990+0000","last_clean":"2023-08-25T09:48:08.447990+0000","last_became_active":"2023-08-25T09:48:08.447183+0000","last_became_peered":"2023-08-25T09:48:08.447183+0000","last_unstale":"2023-08-25T09:48:08.447990+0000","last_undegraded":"2023-08-25T09:48:08.447990+0000","last_fullsized":"2023-08-25T09:48:08.447990+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:35.688792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.844309+0000","last_change":"2023-08-25T09:48:07.816558+0000","last_active":"2023-08-25T09:48:08.844309+0000","last_peered":"2023-08-25T09:48:08.844309+0000","last_clean":"2023-08-25T09:48:08.844309+0000","last_became_active":"2023-08-25T09:48:07.816259+0000","last_became_peered":"2023-08-25T09:48:07.816259+0000","last_unstale":"2023-08-25T09:48:08.844309+0000","last_undegraded":"2023-08-25T09:48:08.844309+0000","last_fullsized":"2023-08-25T09:48:08.844309+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:59:16.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448219+0000","last_change":"2023-08-25T09:48:08.448142+0000","last_active":"2023-08-25T09:48:08.448219+0000","last_peered":"2023-08-25T09:48:08.448219+0000","last_clean":"2023-08-25T09:48:08.448219+0000","last_became_active":"2023-08-25T09:48:08.447799+0000","last_became_peered":"2023-08-25T09:48:08.447799+0000","last_unstale":"2023-08-25T09:48:08.448219+0000","last_undegraded":"2023-08-25T09:48:08.448219+0000","last_fullsized":"2023-08-25T09:48:08.448219+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:05:01.547555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846095+0000","last_change":"2023-08-25T09:48:07.802767+0000","last_active":"2023-08-25T09:48:08.846095+0000","last_peered":"2023-08-25T09:48:08.846095+0000","last_clean":"2023-08-25T09:48:08.846095+0000","last_became_active":"2023-08-25T09:48:07.802312+0000","last_became_peered":"2023-08-25T09:48:07.802312+0000","last_unstale":"2023-08-25T09:48:08.846095+0000","last_undegraded":"2023-08-25T09:48:08.846095+0000","last_fullsized":"2023-08-25T09:48:08.846095+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:02:53.485651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436951+0000","last_change":"2023-08-25T09:48:08.431617+0000","last_active":"2023-08-25T09:48:08.436951+0000","last_peered":"2023-08-25T09:48:08.436951+0000","last_clean":"2023-08-25T09:48:08.436951+0000","last_became_active":"2023-08-25T09:48:08.430838+0000","last_became_peered":"2023-08-25T09:48:08.430838+0000","last_unstale":"2023-08-25T09:48:08.436951+0000","last_undegraded":"2023-08-25T09:48:08.436951+0000","last_fullsized":"2023-08-25T09:48:08.436951+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:39.601468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.845740+0000","last_change":"2023-08-25T09:48:07.812478+0000","last_active":"2023-08-25T09:48:08.845740+0000","last_peered":"2023-08-25T09:48:08.845740+0000","last_clean":"2023-08-25T09:48:08.845740+0000","last_became_active":"2023-08-25T09:48:07.812336+0000","last_became_peered":"2023-08-25T09:48:07.812336+0000","last_unstale":"2023-08-25T09:48:08.845740+0000","last_undegraded":"2023-08-25T09:48:08.845740+0000","last_fullsized":"2023-08-25T09:48:08.845740+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:24.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450368+0000","last_change":"2023-08-25T09:48:08.450275+0000","last_active":"2023-08-25T09:48:08.450368+0000","last_peered":"2023-08-25T09:48:08.450368+0000","last_clean":"2023-08-25T09:48:08.450368+0000","last_became_active":"2023-08-25T09:48:08.448989+0000","last_became_peered":"2023-08-25T09:48:08.448989+0000","last_unstale":"2023-08-25T09:48:08.450368+0000","last_undegraded":"2023-08-25T09:48:08.450368+0000","last_fullsized":"2023-08-25T09:48:08.450368+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:28.295414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.849291+0000","last_change":"2023-08-25T09:48:07.798393+0000","last_active":"2023-08-25T09:48:08.849291+0000","last_peered":"2023-08-25T09:48:08.849291+0000","last_clean":"2023-08-25T09:48:08.849291+0000","last_became_active":"2023-08-25T09:48:07.796692+0000","last_became_peered":"2023-08-25T09:48:07.796692+0000","last_unstale":"2023-08-25T09:48:08.849291+0000","last_undegraded":"2023-08-25T09:48:08.849291+0000","last_fullsized":"2023-08-25T09:48:08.849291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:49:37.676551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447602+0000","last_change":"2023-08-25T09:48:08.446984+0000","last_active":"2023-08-25T09:48:08.447602+0000","last_peered":"2023-08-25T09:48:08.447602+0000","last_clean":"2023-08-25T09:48:08.447602+0000","last_became_active":"2023-08-25T09:48:08.444577+0000","last_became_peered":"2023-08-25T09:48:08.444577+0000","last_unstale":"2023-08-25T09:48:08.447602+0000","last_undegraded":"2023-08-25T09:48:08.447602+0000","last_fullsized":"2023-08-25T09:48:08.447602+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:39.717951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.425127+0000","last_change":"2023-08-25T09:48:07.811447+0000","last_active":"2023-08-25T09:48:08.425127+0000","last_peered":"2023-08-25T09:48:08.425127+0000","last_clean":"2023-08-25T09:48:08.425127+0000","last_became_active":"2023-08-25T09:48:07.811003+0000","last_became_peered":"2023-08-25T09:48:07.811003+0000","last_unstale":"2023-08-25T09:48:08.425127+0000","last_undegraded":"2023-08-25T09:48:08.425127+0000","last_fullsized":"2023-08-25T09:48:08.425127+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:09.631434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447014+0000","last_change":"2023-08-25T09:48:08.445321+0000","last_active":"2023-08-25T09:48:08.447014+0000","last_peered":"2023-08-25T09:48:08.447014+0000","last_clean":"2023-08-25T09:48:08.447014+0000","last_became_active":"2023-08-25T09:48:08.441335+0000","last_became_peered":"2023-08-25T09:48:08.441335+0000","last_unstale":"2023-08-25T09:48:08.447014+0000","last_undegraded":"2023-08-25T09:48:08.447014+0000","last_fullsized":"2023-08-25T09:48:08.447014+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:50.342277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.429588+0000","last_change":"2023-08-25T09:48:07.804809+0000","last_active":"2023-08-25T09:48:08.429588+0000","last_peered":"2023-08-25T09:48:08.429588+0000","last_clean":"2023-08-25T09:48:08.429588+0000","last_became_active":"2023-08-25T09:48:07.804396+0000","last_became_peered":"2023-08-25T09:48:07.804396+0000","last_unstale":"2023-08-25T09:48:08.429588+0000","last_undegraded":"2023-08-25T09:48:08.429588+0000","last_fullsized":"2023-08-25T09:48:08.429588+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:10.703210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444207+0000","last_change":"2023-08-25T09:48:08.444108+0000","last_active":"2023-08-25T09:48:08.444207+0000","last_peered":"2023-08-25T09:48:08.444207+0000","last_clean":"2023-08-25T09:48:08.444207+0000","last_became_active":"2023-08-25T09:48:08.443760+0000","last_became_peered":"2023-08-25T09:48:08.443760+0000","last_unstale":"2023-08-25T09:48:08.444207+0000","last_undegraded":"2023-08-25T09:48:08.444207+0000","last_fullsized":"2023-08-25T09:48:08.444207+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:18.708379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.843504+0000","last_change":"2023-08-25T09:48:07.807308+0000","last_active":"2023-08-25T09:48:08.843504+0000","last_peered":"2023-08-25T09:48:08.843504+0000","last_clean":"2023-08-25T09:48:08.843504+0000","last_became_active":"2023-08-25T09:48:07.807077+0000","last_became_peered":"2023-08-25T09:48:07.807077+0000","last_unstale":"2023-08-25T09:48:08.843504+0000","last_undegraded":"2023-08-25T09:48:08.843504+0000","last_fullsized":"2023-08-25T09:48:08.843504+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:50.590459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.438464+0000","last_change":"2023-08-25T09:48:08.438301+0000","last_active":"2023-08-25T09:48:08.438464+0000","last_peered":"2023-08-25T09:48:08.438464+0000","last_clean":"2023-08-25T09:48:08.438464+0000","last_became_active":"2023-08-25T09:48:08.437811+0000","last_became_peered":"2023-08-25T09:48:08.437811+0000","last_unstale":"2023-08-25T09:48:08.438464+0000","last_undegraded":"2023-08-25T09:48:08.438464+0000","last_fullsized":"2023-08-25T09:48:08.438464+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:09:28.011677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.844149+0000","last_change":"2023-08-25T09:48:07.815830+0000","last_active":"2023-08-25T09:48:08.844149+0000","last_peered":"2023-08-25T09:48:08.844149+0000","last_clean":"2023-08-25T09:48:08.844149+0000","last_became_active":"2023-08-25T09:48:07.815571+0000","last_became_peered":"2023-08-25T09:48:07.815571+0000","last_unstale":"2023-08-25T09:48:08.844149+0000","last_undegraded":"2023-08-25T09:48:08.844149+0000","last_fullsized":"2023-08-25T09:48:08.844149+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:49.781500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441711+0000","last_change":"2023-08-25T09:48:08.441613+0000","last_active":"2023-08-25T09:48:08.441711+0000","last_peered":"2023-08-25T09:48:08.441711+0000","last_clean":"2023-08-25T09:48:08.441711+0000","last_became_active":"2023-08-25T09:48:08.441365+0000","last_became_peered":"2023-08-25T09:48:08.441365+0000","last_unstale":"2023-08-25T09:48:08.441711+0000","last_undegraded":"2023-08-25T09:48:08.441711+0000","last_fullsized":"2023-08-25T09:48:08.441711+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:12:04.798252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.848697+0000","last_change":"2023-08-25T09:48:07.810838+0000","last_active":"2023-08-25T09:48:08.848697+0000","last_peered":"2023-08-25T09:48:08.848697+0000","last_clean":"2023-08-25T09:48:08.848697+0000","last_became_active":"2023-08-25T09:48:07.810479+0000","last_became_peered":"2023-08-25T09:48:07.810479+0000","last_unstale":"2023-08-25T09:48:08.848697+0000","last_undegraded":"2023-08-25T09:48:08.848697+0000","last_fullsized":"2023-08-25T09:48:08.848697+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:09.745781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439133+0000","last_change":"2023-08-25T09:48:08.438968+0000","last_active":"2023-08-25T09:48:08.439133+0000","last_peered":"2023-08-25T09:48:08.439133+0000","last_clean":"2023-08-25T09:48:08.439133+0000","last_became_active":"2023-08-25T09:48:08.438236+0000","last_became_peered":"2023-08-25T09:48:08.438236+0000","last_unstale":"2023-08-25T09:48:08.439133+0000","last_undegraded":"2023-08-25T09:48:08.439133+0000","last_fullsized":"2023-08-25T09:48:08.439133+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:48.669279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.433126+0000","last_change":"2023-08-25T09:48:07.808622+0000","last_active":"2023-08-25T09:48:08.433126+0000","last_peered":"2023-08-25T09:48:08.433126+0000","last_clean":"2023-08-25T09:48:08.433126+0000","last_became_active":"2023-08-25T09:48:07.807217+0000","last_became_peered":"2023-08-25T09:48:07.807217+0000","last_unstale":"2023-08-25T09:48:08.433126+0000","last_undegraded":"2023-08-25T09:48:08.433126+0000","last_fullsized":"2023-08-25T09:48:08.433126+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:42:39.683019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.430693+0000","last_change":"2023-08-25T09:48:08.430490+0000","last_active":"2023-08-25T09:48:08.430693+0000","last_peered":"2023-08-25T09:48:08.430693+0000","last_clean":"2023-08-25T09:48:08.430693+0000","last_became_active":"2023-08-25T09:48:08.429665+0000","last_became_peered":"2023-08-25T09:48:08.429665+0000","last_unstale":"2023-08-25T09:48:08.430693+0000","last_undegraded":"2023-08-25T09:48:08.430693+0000","last_fullsized":"2023-08-25T09:48:08.430693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:22:29.317808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846900+0000","last_change":"2023-08-25T09:48:07.806915+0000","last_active":"2023-08-25T09:48:08.846900+0000","last_peered":"2023-08-25T09:48:08.846900+0000","last_clean":"2023-08-25T09:48:08.846900+0000","last_became_active":"2023-08-25T09:48:07.806658+0000","last_became_peered":"2023-08-25T09:48:07.806658+0000","last_unstale":"2023-08-25T09:48:08.846900+0000","last_undegraded":"2023-08-25T09:48:08.846900+0000","last_fullsized":"2023-08-25T09:48:08.846900+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:31.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444338+0000","last_change":"2023-08-25T09:48:08.444130+0000","last_active":"2023-08-25T09:48:08.444338+0000","last_peered":"2023-08-25T09:48:08.444338+0000","last_clean":"2023-08-25T09:48:08.444338+0000","last_became_active":"2023-08-25T09:48:08.443178+0000","last_became_peered":"2023-08-25T09:48:08.443178+0000","last_unstale":"2023-08-25T09:48:08.444338+0000","last_undegraded":"2023-08-25T09:48:08.444338+0000","last_fullsized":"2023-08-25T09:48:08.444338+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:02:50.345147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.425462+0000","last_change":"2023-08-25T09:48:07.807095+0000","last_active":"2023-08-25T09:48:08.425462+0000","last_peered":"2023-08-25T09:48:08.425462+0000","last_clean":"2023-08-25T09:48:08.425462+0000","last_became_active":"2023-08-25T09:48:07.806424+0000","last_became_peered":"2023-08-25T09:48:07.806424+0000","last_unstale":"2023-08-25T09:48:08.425462+0000","last_undegraded":"2023-08-25T09:48:08.425462+0000","last_fullsized":"2023-08-25T09:48:08.425462+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:18:31.379808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.438573+0000","last_change":"2023-08-25T09:48:08.438238+0000","last_active":"2023-08-25T09:48:08.438573+0000","last_peered":"2023-08-25T09:48:08.438573+0000","last_clean":"2023-08-25T09:48:08.438573+0000","last_became_active":"2023-08-25T09:48:08.437998+0000","last_became_peered":"2023-08-25T09:48:08.437998+0000","last_unstale":"2023-08-25T09:48:08.438573+0000","last_undegraded":"2023-08-25T09:48:08.438573+0000","last_fullsized":"2023-08-25T09:48:08.438573+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:30:09.804471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.429952+0000","last_change":"2023-08-25T09:48:07.804632+0000","last_active":"2023-08-25T09:48:08.429952+0000","last_peered":"2023-08-25T09:48:08.429952+0000","last_clean":"2023-08-25T09:48:08.429952+0000","last_became_active":"2023-08-25T09:48:07.804074+0000","last_became_peered":"2023-08-25T09:48:07.804074+0000","last_unstale":"2023-08-25T09:48:08.429952+0000","last_undegraded":"2023-08-25T09:48:08.429952+0000","last_fullsized":"2023-08-25T09:48:08.429952+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:20.161074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439748+0000","last_change":"2023-08-25T09:48:08.438574+0000","last_active":"2023-08-25T09:48:08.439748+0000","last_peered":"2023-08-25T09:48:08.439748+0000","last_clean":"2023-08-25T09:48:08.439748+0000","last_became_active":"2023-08-25T09:48:08.438317+0000","last_became_peered":"2023-08-25T09:48:08.438317+0000","last_unstale":"2023-08-25T09:48:08.439748+0000","last_undegraded":"2023-08-25T09:48:08.439748+0000","last_fullsized":"2023-08-25T09:48:08.439748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:46:47.214540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.420872+0000","last_change":"2023-08-25T09:48:07.812944+0000","last_active":"2023-08-25T09:48:08.420872+0000","last_peered":"2023-08-25T09:48:08.420872+0000","last_clean":"2023-08-25T09:48:08.420872+0000","last_became_active":"2023-08-25T09:48:07.812556+0000","last_became_peered":"2023-08-25T09:48:07.812556+0000","last_unstale":"2023-08-25T09:48:08.420872+0000","last_undegraded":"2023-08-25T09:48:08.420872+0000","last_fullsized":"2023-08-25T09:48:08.420872+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:16:14.871551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448076+0000","last_change":"2023-08-25T09:48:08.447996+0000","last_active":"2023-08-25T09:48:08.448076+0000","last_peered":"2023-08-25T09:48:08.448076+0000","last_clean":"2023-08-25T09:48:08.448076+0000","last_became_active":"2023-08-25T09:48:08.446309+0000","last_became_peered":"2023-08-25T09:48:08.446309+0000","last_unstale":"2023-08-25T09:48:08.448076+0000","last_undegraded":"2023-08-25T09:48:08.448076+0000","last_fullsized":"2023-08-25T09:48:08.448076+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:13:32.773833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.848707+0000","last_change":"2023-08-25T09:48:07.809138+0000","last_active":"2023-08-25T09:48:08.848707+0000","last_peered":"2023-08-25T09:48:08.848707+0000","last_clean":"2023-08-25T09:48:08.848707+0000","last_became_active":"2023-08-25T09:48:07.808036+0000","last_became_peered":"2023-08-25T09:48:07.808036+0000","last_unstale":"2023-08-25T09:48:08.848707+0000","last_undegraded":"2023-08-25T09:48:08.848707+0000","last_fullsized":"2023-08-25T09:48:08.848707+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:02:49.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443187+0000","last_change":"2023-08-25T09:48:08.443026+0000","last_active":"2023-08-25T09:48:08.443187+0000","last_peered":"2023-08-25T09:48:08.443187+0000","last_clean":"2023-08-25T09:48:08.443187+0000","last_became_active":"2023-08-25T09:48:08.442362+0000","last_became_peered":"2023-08-25T09:48:08.442362+0000","last_unstale":"2023-08-25T09:48:08.443187+0000","last_undegraded":"2023-08-25T09:48:08.443187+0000","last_fullsized":"2023-08-25T09:48:08.443187+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:58:45.559676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.432173+0000","last_change":"2023-08-25T09:48:07.805014+0000","last_active":"2023-08-25T09:48:08.432173+0000","last_peered":"2023-08-25T09:48:08.432173+0000","last_clean":"2023-08-25T09:48:08.432173+0000","last_became_active":"2023-08-25T09:48:07.804053+0000","last_became_peered":"2023-08-25T09:48:07.804053+0000","last_unstale":"2023-08-25T09:48:08.432173+0000","last_undegraded":"2023-08-25T09:48:08.432173+0000","last_fullsized":"2023-08-25T09:48:08.432173+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:06:10.131696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439813+0000","last_change":"2023-08-25T09:48:08.439577+0000","last_active":"2023-08-25T09:48:08.439813+0000","last_peered":"2023-08-25T09:48:08.439813+0000","last_clean":"2023-08-25T09:48:08.439813+0000","last_became_active":"2023-08-25T09:48:08.439064+0000","last_became_peered":"2023-08-25T09:48:08.439064+0000","last_unstale":"2023-08-25T09:48:08.439813+0000","last_undegraded":"2023-08-25T09:48:08.439813+0000","last_fullsized":"2023-08-25T09:48:08.439813+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:26.980981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416447+0000","last_change":"2023-08-25T09:48:07.793569+0000","last_active":"2023-08-25T09:48:08.416447+0000","last_peered":"2023-08-25T09:48:08.416447+0000","last_clean":"2023-08-25T09:48:08.416447+0000","last_became_active":"2023-08-25T09:48:07.793062+0000","last_became_peered":"2023-08-25T09:48:07.793062+0000","last_unstale":"2023-08-25T09:48:08.416447+0000","last_undegraded":"2023-08-25T09:48:08.416447+0000","last_fullsized":"2023-08-25T09:48:08.416447+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:27.562821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442396+0000","last_change":"2023-08-25T09:48:08.442303+0000","last_active":"2023-08-25T09:48:08.442396+0000","last_peered":"2023-08-25T09:48:08.442396+0000","last_clean":"2023-08-25T09:48:08.442396+0000","last_became_active":"2023-08-25T09:48:08.442007+0000","last_became_peered":"2023-08-25T09:48:08.442007+0000","last_unstale":"2023-08-25T09:48:08.442396+0000","last_undegraded":"2023-08-25T09:48:08.442396+0000","last_fullsized":"2023-08-25T09:48:08.442396+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:14:03.786808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.431062+0000","last_change":"2023-08-25T09:48:07.816291+0000","last_active":"2023-08-25T09:48:08.431062+0000","last_peered":"2023-08-25T09:48:08.431062+0000","last_clean":"2023-08-25T09:48:08.431062+0000","last_became_active":"2023-08-25T09:48:07.815563+0000","last_became_peered":"2023-08-25T09:48:07.815563+0000","last_unstale":"2023-08-25T09:48:08.431062+0000","last_undegraded":"2023-08-25T09:48:08.431062+0000","last_fullsized":"2023-08-25T09:48:08.431062+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:27.441758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.437986+0000","last_change":"2023-08-25T09:48:08.437231+0000","last_active":"2023-08-25T09:48:08.437986+0000","last_peered":"2023-08-25T09:48:08.437986+0000","last_clean":"2023-08-25T09:48:08.437986+0000","last_became_active":"2023-08-25T09:48:08.436924+0000","last_became_peered":"2023-08-25T09:48:08.436924+0000","last_unstale":"2023-08-25T09:48:08.437986+0000","last_undegraded":"2023-08-25T09:48:08.437986+0000","last_fullsized":"2023-08-25T09:48:08.437986+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:24.475584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.435451+0000","last_change":"2023-08-25T09:48:07.810821+0000","last_active":"2023-08-25T09:48:08.435451+0000","last_peered":"2023-08-25T09:48:08.435451+0000","last_clean":"2023-08-25T09:48:08.435451+0000","last_became_active":"2023-08-25T09:48:07.810473+0000","last_became_peered":"2023-08-25T09:48:07.810473+0000","last_unstale":"2023-08-25T09:48:08.435451+0000","last_undegraded":"2023-08-25T09:48:08.435451+0000","last_fullsized":"2023-08-25T09:48:08.435451+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:51.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.445153+0000","last_change":"2023-08-25T09:48:08.445040+0000","last_active":"2023-08-25T09:48:08.445153+0000","last_peered":"2023-08-25T09:48:08.445153+0000","last_clean":"2023-08-25T09:48:08.445153+0000","last_became_active":"2023-08-25T09:48:08.444543+0000","last_became_peered":"2023-08-25T09:48:08.444543+0000","last_unstale":"2023-08-25T09:48:08.445153+0000","last_undegraded":"2023-08-25T09:48:08.445153+0000","last_fullsized":"2023-08-25T09:48:08.445153+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:40.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436248+0000","last_change":"2023-08-25T09:48:07.804988+0000","last_active":"2023-08-25T09:48:08.436248+0000","last_peered":"2023-08-25T09:48:08.436248+0000","last_clean":"2023-08-25T09:48:08.436248+0000","last_became_active":"2023-08-25T09:48:07.804645+0000","last_became_peered":"2023-08-25T09:48:07.804645+0000","last_unstale":"2023-08-25T09:48:08.436248+0000","last_undegraded":"2023-08-25T09:48:08.436248+0000","last_fullsized":"2023-08-25T09:48:08.436248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:25.662743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.434083+0000","last_change":"2023-08-25T09:48:08.433943+0000","last_active":"2023-08-25T09:48:08.434083+0000","last_peered":"2023-08-25T09:48:08.434083+0000","last_clean":"2023-08-25T09:48:08.434083+0000","last_became_active":"2023-08-25T09:48:08.433506+0000","last_became_peered":"2023-08-25T09:48:08.433506+0000","last_unstale":"2023-08-25T09:48:08.434083+0000","last_undegraded":"2023-08-25T09:48:08.434083+0000","last_fullsized":"2023-08-25T09:48:08.434083+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:57.923814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.424095+0000","last_change":"2023-08-25T09:48:07.814082+0000","last_active":"2023-08-25T09:48:08.424095+0000","last_peered":"2023-08-25T09:48:08.424095+0000","last_clean":"2023-08-25T09:48:08.424095+0000","last_became_active":"2023-08-25T09:48:07.813447+0000","last_became_peered":"2023-08-25T09:48:07.813447+0000","last_unstale":"2023-08-25T09:48:08.424095+0000","last_undegraded":"2023-08-25T09:48:08.424095+0000","last_fullsized":"2023-08-25T09:48:08.424095+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:21:18.043288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446130+0000","last_change":"2023-08-25T09:48:08.446057+0000","last_active":"2023-08-25T09:48:08.446130+0000","last_peered":"2023-08-25T09:48:08.446130+0000","last_clean":"2023-08-25T09:48:08.446130+0000","last_became_active":"2023-08-25T09:48:08.445799+0000","last_became_peered":"2023-08-25T09:48:08.445799+0000","last_unstale":"2023-08-25T09:48:08.446130+0000","last_undegraded":"2023-08-25T09:48:08.446130+0000","last_fullsized":"2023-08-25T09:48:08.446130+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:03:20.353797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.417678+0000","last_change":"2023-08-25T09:48:07.802544+0000","last_active":"2023-08-25T09:48:08.417678+0000","last_peered":"2023-08-25T09:48:08.417678+0000","last_clean":"2023-08-25T09:48:08.417678+0000","last_became_active":"2023-08-25T09:48:07.801902+0000","last_became_peered":"2023-08-25T09:48:07.801902+0000","last_unstale":"2023-08-25T09:48:08.417678+0000","last_undegraded":"2023-08-25T09:48:08.417678+0000","last_fullsized":"2023-08-25T09:48:08.417678+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:42.811868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443342+0000","last_change":"2023-08-25T09:48:08.443256+0000","last_active":"2023-08-25T09:48:08.443342+0000","last_peered":"2023-08-25T09:48:08.443342+0000","last_clean":"2023-08-25T09:48:08.443342+0000","last_became_active":"2023-08-25T09:48:08.442684+0000","last_became_peered":"2023-08-25T09:48:08.442684+0000","last_unstale":"2023-08-25T09:48:08.443342+0000","last_undegraded":"2023-08-25T09:48:08.443342+0000","last_fullsized":"2023-08-25T09:48:08.443342+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:32:43.629812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.418804+0000","last_change":"2023-08-25T09:48:07.811130+0000","last_active":"2023-08-25T09:48:08.418804+0000","last_peered":"2023-08-25T09:48:08.418804+0000","last_clean":"2023-08-25T09:48:08.418804+0000","last_became_active":"2023-08-25T09:48:07.810908+0000","last_became_peered":"2023-08-25T09:48:07.810908+0000","last_unstale":"2023-08-25T09:48:08.418804+0000","last_undegraded":"2023-08-25T09:48:08.418804+0000","last_fullsized":"2023-08-25T09:48:08.418804+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:53:58.196255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":419,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.427619+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:48:08.427619+0000","last_peered":"2023-08-25T09:48:08.427619+0000","last_clean":"2023-08-25T09:48:08.427619+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:48:08.427619+0000","last_undegraded":"2023-08-25T09:48:08.427619+0000","last_fullsized":"2023-08-25T09:48:08.427619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:50:17.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.419019+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:48:08.419019+0000","last_peered":"2023-08-25T09:48:08.419019+0000","last_clean":"2023-08-25T09:48:08.419019+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:48:08.419019+0000","last_undegraded":"2023-08-25T09:48:08.419019+0000","last_fullsized":"2023-08-25T09:48:08.419019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:01.515335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.449210+0000","last_change":"2023-08-25T09:48:08.449113+0000","last_active":"2023-08-25T09:48:08.449210+0000","last_peered":"2023-08-25T09:48:08.449210+0000","last_clean":"2023-08-25T09:48:08.449210+0000","last_became_active":"2023-08-25T09:48:08.445991+0000","last_became_peered":"2023-08-25T09:48:08.445991+0000","last_unstale":"2023-08-25T09:48:08.449210+0000","last_undegraded":"2023-08-25T09:48:08.449210+0000","last_fullsized":"2023-08-25T09:48:08.449210+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:22:59.694751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436120+0000","last_change":"2023-08-25T09:48:07.806255+0000","last_active":"2023-08-25T09:48:08.436120+0000","last_peered":"2023-08-25T09:48:08.436120+0000","last_clean":"2023-08-25T09:48:08.436120+0000","last_became_active":"2023-08-25T09:48:07.805900+0000","last_became_peered":"2023-08-25T09:48:07.805900+0000","last_unstale":"2023-08-25T09:48:08.436120+0000","last_undegraded":"2023-08-25T09:48:08.436120+0000","last_fullsized":"2023-08-25T09:48:08.436120+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:27:33.818964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.417147+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:48:08.417147+0000","last_peered":"2023-08-25T09:48:08.417147+0000","last_clean":"2023-08-25T09:48:08.417147+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:48:08.417147+0000","last_undegraded":"2023-08-25T09:48:08.417147+0000","last_fullsized":"2023-08-25T09:48:08.417147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:28.380493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.431969+0000","last_change":"2023-08-25T09:48:08.431044+0000","last_active":"2023-08-25T09:48:08.431969+0000","last_peered":"2023-08-25T09:48:08.431969+0000","last_clean":"2023-08-25T09:48:08.431969+0000","last_became_active":"2023-08-25T09:48:08.430122+0000","last_became_peered":"2023-08-25T09:48:08.430122+0000","last_unstale":"2023-08-25T09:48:08.431969+0000","last_undegraded":"2023-08-25T09:48:08.431969+0000","last_fullsized":"2023-08-25T09:48:08.431969+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:39:40.045358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"164'8","reported_seq":19,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.854021+0000","last_change":"2023-08-25T09:48:07.813874+0000","last_active":"2023-08-25T09:48:08.854021+0000","last_peered":"2023-08-25T09:48:08.854021+0000","last_clean":"2023-08-25T09:48:08.854021+0000","last_became_active":"2023-08-25T09:48:07.813007+0000","last_became_peered":"2023-08-25T09:48:07.813007+0000","last_unstale":"2023-08-25T09:48:08.854021+0000","last_undegraded":"2023-08-25T09:48:08.854021+0000","last_fullsized":"2023-08-25T09:48:08.854021+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:58:27.018451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":293,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.422221+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:48:08.422221+0000","last_peered":"2023-08-25T09:48:08.422221+0000","last_clean":"2023-08-25T09:48:08.422221+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:48:08.422221+0000","last_undegraded":"2023-08-25T09:48:08.422221+0000","last_fullsized":"2023-08-25T09:48:08.422221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:38:02.817543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444918+0000","last_change":"2023-08-25T09:48:08.444817+0000","last_active":"2023-08-25T09:48:08.444918+0000","last_peered":"2023-08-25T09:48:08.444918+0000","last_clean":"2023-08-25T09:48:08.444918+0000","last_became_active":"2023-08-25T09:48:08.444177+0000","last_became_peered":"2023-08-25T09:48:08.444177+0000","last_unstale":"2023-08-25T09:48:08.444918+0000","last_undegraded":"2023-08-25T09:48:08.444918+0000","last_fullsized":"2023-08-25T09:48:08.444918+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:23.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.850397+0000","last_change":"2023-08-25T09:48:07.816102+0000","last_active":"2023-08-25T09:48:08.850397+0000","last_peered":"2023-08-25T09:48:08.850397+0000","last_clean":"2023-08-25T09:48:08.850397+0000","last_became_active":"2023-08-25T09:48:07.814766+0000","last_became_peered":"2023-08-25T09:48:07.814766+0000","last_unstale":"2023-08-25T09:48:08.850397+0000","last_undegraded":"2023-08-25T09:48:08.850397+0000","last_fullsized":"2023-08-25T09:48:08.850397+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:24.774411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416645+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:48:08.416645+0000","last_peered":"2023-08-25T09:48:08.416645+0000","last_clean":"2023-08-25T09:48:08.416645+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:48:08.416645+0000","last_undegraded":"2023-08-25T09:48:08.416645+0000","last_fullsized":"2023-08-25T09:48:08.416645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:44:49.052732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.451178+0000","last_change":"2023-08-25T09:48:08.451100+0000","last_active":"2023-08-25T09:48:08.451178+0000","last_peered":"2023-08-25T09:48:08.451178+0000","last_clean":"2023-08-25T09:48:08.451178+0000","last_became_active":"2023-08-25T09:48:08.450868+0000","last_became_peered":"2023-08-25T09:48:08.450868+0000","last_unstale":"2023-08-25T09:48:08.451178+0000","last_undegraded":"2023-08-25T09:48:08.451178+0000","last_fullsized":"2023-08-25T09:48:08.451178+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:21.857578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"164'3","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846648+0000","last_change":"2023-08-25T09:48:07.805184+0000","last_active":"2023-08-25T09:48:08.846648+0000","last_peered":"2023-08-25T09:48:08.846648+0000","last_clean":"2023-08-25T09:48:08.846648+0000","last_became_active":"2023-08-25T09:48:07.804397+0000","last_became_peered":"2023-08-25T09:48:07.804397+0000","last_unstale":"2023-08-25T09:48:08.846648+0000","last_undegraded":"2023-08-25T09:48:08.846648+0000","last_fullsized":"2023-08-25T09:48:08.846648+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:13.216905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":265,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.429811+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:48:08.429811+0000","last_peered":"2023-08-25T09:48:08.429811+0000","last_clean":"2023-08-25T09:48:08.429811+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:48:08.429811+0000","last_undegraded":"2023-08-25T09:48:08.429811+0000","last_fullsized":"2023-08-25T09:48:08.429811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:00:36.662651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450132+0000","last_change":"2023-08-25T09:48:08.449960+0000","last_active":"2023-08-25T09:48:08.450132+0000","last_peered":"2023-08-25T09:48:08.450132+0000","last_clean":"2023-08-25T09:48:08.450132+0000","last_became_active":"2023-08-25T09:48:08.447890+0000","last_became_peered":"2023-08-25T09:48:08.447890+0000","last_unstale":"2023-08-25T09:48:08.450132+0000","last_undegraded":"2023-08-25T09:48:08.450132+0000","last_fullsized":"2023-08-25T09:48:08.450132+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:46.866121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"164'6","reported_seq":17,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.848005+0000","last_change":"2023-08-25T09:48:07.802045+0000","last_active":"2023-08-25T09:48:08.848005+0000","last_peered":"2023-08-25T09:48:08.848005+0000","last_clean":"2023-08-25T09:48:08.848005+0000","last_became_active":"2023-08-25T09:48:07.801824+0000","last_became_peered":"2023-08-25T09:48:07.801824+0000","last_unstale":"2023-08-25T09:48:08.848005+0000","last_undegraded":"2023-08-25T09:48:08.848005+0000","last_fullsized":"2023-08-25T09:48:08.848005+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:48.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416844+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:48:08.416844+0000","last_peered":"2023-08-25T09:48:08.416844+0000","last_clean":"2023-08-25T09:48:08.416844+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:48:08.416844+0000","last_undegraded":"2023-08-25T09:48:08.416844+0000","last_fullsized":"2023-08-25T09:48:08.416844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:33:51.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443470+0000","last_change":"2023-08-25T09:48:08.443374+0000","last_active":"2023-08-25T09:48:08.443470+0000","last_peered":"2023-08-25T09:48:08.443470+0000","last_clean":"2023-08-25T09:48:08.443470+0000","last_became_active":"2023-08-25T09:48:08.442594+0000","last_became_peered":"2023-08-25T09:48:08.442594+0000","last_unstale":"2023-08-25T09:48:08.443470+0000","last_undegraded":"2023-08-25T09:48:08.443470+0000","last_fullsized":"2023-08-25T09:48:08.443470+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:15:08.002236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.428959+0000","last_change":"2023-08-25T09:48:07.799252+0000","last_active":"2023-08-25T09:48:08.428959+0000","last_peered":"2023-08-25T09:48:08.428959+0000","last_clean":"2023-08-25T09:48:08.428959+0000","last_became_active":"2023-08-25T09:48:07.798809+0000","last_became_peered":"2023-08-25T09:48:07.798809+0000","last_unstale":"2023-08-25T09:48:08.428959+0000","last_undegraded":"2023-08-25T09:48:08.428959+0000","last_fullsized":"2023-08-25T09:48:08.428959+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:46:04.683565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":263,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.433435+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:48:08.433435+0000","last_peered":"2023-08-25T09:48:08.433435+0000","last_clean":"2023-08-25T09:48:08.433435+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:48:08.433435+0000","last_undegraded":"2023-08-25T09:48:08.433435+0000","last_fullsized":"2023-08-25T09:48:08.433435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:22.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.435985+0000","last_change":"2023-08-25T09:48:08.435834+0000","last_active":"2023-08-25T09:48:08.435985+0000","last_peered":"2023-08-25T09:48:08.435985+0000","last_clean":"2023-08-25T09:48:08.435985+0000","last_became_active":"2023-08-25T09:48:08.435058+0000","last_became_peered":"2023-08-25T09:48:08.435058+0000","last_unstale":"2023-08-25T09:48:08.435985+0000","last_undegraded":"2023-08-25T09:48:08.435985+0000","last_fullsized":"2023-08-25T09:48:08.435985+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:58.315658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.432686+0000","last_change":"2023-08-25T09:48:07.814733+0000","last_active":"2023-08-25T09:48:08.432686+0000","last_peered":"2023-08-25T09:48:08.432686+0000","last_clean":"2023-08-25T09:48:08.432686+0000","last_became_active":"2023-08-25T09:48:07.814463+0000","last_became_peered":"2023-08-25T09:48:07.814463+0000","last_unstale":"2023-08-25T09:48:08.432686+0000","last_undegraded":"2023-08-25T09:48:08.432686+0000","last_fullsized":"2023-08-25T09:48:08.432686+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:05:00.706211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416330+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:48:08.416330+0000","last_peered":"2023-08-25T09:48:08.416330+0000","last_clean":"2023-08-25T09:48:08.416330+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:48:08.416330+0000","last_undegraded":"2023-08-25T09:48:08.416330+0000","last_fullsized":"2023-08-25T09:48:08.416330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:33.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":948,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194364,"statfs":{"total":200000000,"available":199028987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.095,"15min":1.0900000000000001},"min":{"1min":0.79500000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.7210000000000001,"5min":1.7909999999999999,"15min":10.747},"last":1.137},{"interface":"front","average":{"1min":1.216,"5min":1.159,"15min":1.0660000000000001},"min":{"1min":0.91300000000000003,"5min":0.61699999999999999,"15min":0.43099999999999999},"max":{"1min":1.651,"5min":1.855,"15min":1.855},"last":1.218}]},{"osd":1,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3240000000000001,"15min":1.1919999999999999},"min":{"1min":0.85999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.72,"5min":11.010999999999999,"15min":11.010999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.2110000000000001,"15min":1.1339999999999999},"min":{"1min":1.095,"5min":0.66500000000000004,"15min":0.57999999999999996},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7769999999999999},"last":1.024}]},{"osd":2,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1479999999999999,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.66800000000000004,"15min":0.50600000000000001},"max":{"1min":1.4590000000000001,"5min":1.619,"15min":1.619},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.327,"5min":1.302,"15min":1.2430000000000001},"min":{"1min":0.79500000000000004,"5min":0.67700000000000005,"15min":0.66800000000000004},"max":{"1min":1.704,"5min":1.98,"15min":1.98},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.236,"15min":1.181},"min":{"1min":0.90500000000000003,"5min":0.59799999999999998,"15min":0.56799999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.3700000000000001,"15min":1.3440000000000001},"min":{"1min":1.077,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6679999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2849999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.288,"15min":1.2829999999999999},"min":{"1min":0.80700000000000005,"5min":0.51300000000000001,"15min":0.48599999999999999},"max":{"1min":1.7649999999999999,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3740000000000001,"15min":1.3120000000000001},"min":{"1min":0.93700000000000006,"5min":0.53800000000000003,"15min":0.46300000000000002},"max":{"1min":1.798,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.7}]},{"osd":5,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.456,"15min":1.381},"min":{"1min":1.3160000000000001,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":1.849,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.532},{"interface":"front","average":{"1min":1.488,"5min":1.462,"15min":1.4059999999999999},"min":{"1min":1.113,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.895,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.357}]},{"osd":6,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5489999999999999,"15min":1.474},"min":{"1min":0.995,"5min":0.72999999999999998,"15min":0.68799999999999994},"max":{"1min":1.98,"5min":2.0259999999999998,"15min":2.1720000000000002},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.5,"15min":1.4319999999999999},"min":{"1min":1.292,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.595}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194357,"statfs":{"total":200000000,"available":199021825,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1539999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5569999999999999,"5min":1.804,"15min":1.804},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.169,"15min":1.1519999999999999},"min":{"1min":0.79100000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.488,"5min":1.992,"15min":5.5410000000000004},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2,"15min":1.2},"min":{"1min":0.83299999999999996,"5min":0.72699999999999998,"15min":0.54700000000000004},"max":{"1min":1.6259999999999999,"5min":2.198,"15min":5.6799999999999997},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.214,"15min":1.1459999999999999},"min":{"1min":0.66800000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.909,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.091}]},{"osd":2,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.165,"15min":1.123},"min":{"1min":0.82199999999999995,"5min":0.63400000000000001,"15min":0.60399999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.708},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3029999999999999,"15min":1.2450000000000001},"min":{"1min":0.67000000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":13.193},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3080000000000001,"15min":1.2569999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.661,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2749999999999999,"15min":1.26},"min":{"1min":0.88300000000000001,"5min":0.70699999999999996,"15min":0.69199999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.9239999999999999},"last":1.5309999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.363,"15min":1.2869999999999999},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8009999999999999,"5min":2.1110000000000002,"15min":2.484},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.385,"15min":1.349},"min":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7509999999999999,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.3480000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.343,"15min":1.296},"min":{"1min":0.74199999999999999,"5min":0.65000000000000002,"15min":0.59599999999999997},"max":{"1min":1.74,"5min":1.859,"15min":2.0049999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5089999999999999,"15min":1.413},"min":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.994,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.421}]},{"osd":7,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.54,"15min":1.512},"min":{"1min":0.80000000000000004,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":1.944,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.476},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5069999999999999,"15min":1.484},"min":{"1min":0.85899999999999999,"5min":0.74299999999999999,"15min":0.57399999999999995},"max":{"1min":1.7949999999999999,"5min":1.9710000000000001,"15min":2.1890000000000001},"last":1.655}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1531,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193780,"statfs":{"total":200000000,"available":198431491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.179,"15min":1.161},"min":{"1min":0.71999999999999997,"5min":0.624,"15min":0.497},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.081},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.218,"15min":1.2090000000000001},"min":{"1min":0.82199999999999995,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.974},"last":1.139}]},{"osd":1,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.67800000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.177,"15min":1.1399999999999999},"min":{"1min":0.76600000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.704,"5min":1.8640000000000001,"15min":2.1869999999999998},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.2609999999999999,"15min":1.208},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.528},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.286,"15min":1.234},"min":{"1min":0.86399999999999999,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.623,"5min":1.778,"15min":1.962},"last":1.268}]},{"osd":3,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.22,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.8109999999999999,"5min":2.3439999999999999,"15min":17.356999999999999},"last":1.296},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3580000000000001,"15min":1.304},"min":{"1min":0.78700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.964,"5min":1.964,"15min":1.9650000000000001},"last":0.94499999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2949999999999999,"15min":1.3049999999999999},"min":{"1min":0.80800000000000005,"5min":0.58299999999999996,"15min":0.55500000000000005},"max":{"1min":1.9059999999999999,"5min":1.9530000000000001,"15min":2.625},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.3460000000000001,"15min":1.3560000000000001},"min":{"1min":0.83699999999999997,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":1.913,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.476}]},{"osd":6,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.4670000000000001,"15min":1.4379999999999999},"min":{"1min":0.95099999999999996,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":2.028,"5min":2.048,"15min":2.4820000000000002},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.492,"15min":1.446},"min":{"1min":0.96899999999999997,"5min":0.65500000000000003,"15min":0.60399999999999998},"max":{"1min":2.044,"5min":2.1059999999999999,"15min":2.5539999999999998},"last":1.605}]},{"osd":7,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.4830000000000001,"15min":1.452},"min":{"1min":1.0700000000000001,"5min":0.80800000000000005,"15min":0.63},"max":{"1min":2.048,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.605,"15min":1.5800000000000001},"min":{"1min":1.1639999999999999,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":2.1040000000000001,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.4239999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194357,"statfs":{"total":200000000,"available":199021789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.246,"15min":1.1819999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.8160000000000001},"last":4.4260000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.3029999999999999,"15min":1.204},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.9750000000000001,"5min":1.976,"15min":7.367},"last":2.5150000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.149,"15min":1.1299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7330000000000001,"5min":2.032,"15min":2.032},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2150000000000001,"15min":1.333},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7110000000000001,"5min":1.954,"15min":8.5850000000000009},"last":4.351}]},{"osd":2,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2030000000000001,"15min":1.2569999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.613,"5min":1.708,"15min":4.1669999999999998},"last":1.224},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3089999999999999,"15min":1.2110000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.8520000000000001},"last":4.7320000000000002}]},{"osd":3,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.248,"15min":1.244},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":7.2560000000000002},"last":2.5680000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.4059999999999999,"15min":1.3740000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8100000000000001,"5min":1.899,"15min":1.911},"last":1.151}]},{"osd":5,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3009999999999999,"15min":1.2949999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.6879999999999999,"5min":2.1850000000000001,"15min":2.4420000000000002},"last":5.048},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.282},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.931,"5min":1.931,"15min":8.2409999999999997},"last":2.6469999999999998}]},{"osd":6,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.4470000000000001,"15min":1.4359999999999999},"min":{"1min":0.72899999999999998,"5min":0.68400000000000005,"15min":0.52000000000000002},"max":{"1min":1.722,"5min":3.5880000000000001,"15min":3.5880000000000001},"last":6.2220000000000004},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.585,"15min":1.5680000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.1720000000000002,"5min":3.5270000000000001,"15min":4.0229999999999997},"last":7.0990000000000002}]},{"osd":7,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5049999999999999,"15min":1.454},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.8859999999999999,"5min":2.0649999999999999,"15min":8.1760000000000002},"last":3.5409999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.621,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.77500000000000002},"max":{"1min":1.909,"5min":2.1819999999999999,"15min":2.7930000000000001},"last":3.831}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194356,"statfs":{"total":200000000,"available":199021277,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.93799999999999994,"15min":0.871},"min":{"1min":0.46600000000000003,"5min":0.41099999999999998,"15min":0.29299999999999998},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0029999999999999,"15min":0.92700000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.595,"5min":1.631,"15min":1.631},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85599999999999998,"15min":0.81999999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.35699999999999998},"max":{"1min":1.163,"5min":1.702,"15min":1.702},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1,"15min":0.98199999999999998},"min":{"1min":0.58499999999999996,"5min":0.52800000000000002,"15min":0.45700000000000002},"max":{"1min":1.6339999999999999,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.99099999999999999,"15min":0.90800000000000003},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.378},"max":{"1min":1.5469999999999999,"5min":1.7450000000000001,"15min":1.919},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.147,"15min":1.1160000000000001},"min":{"1min":0.63500000000000001,"5min":0.47399999999999998,"15min":0.44900000000000001},"max":{"1min":1.8080000000000001,"5min":2.0870000000000002,"15min":3.399},"last":0.876}]},{"osd":4,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2150000000000001,"15min":1.1970000000000001},"min":{"1min":0.93300000000000005,"5min":0.73299999999999998,"15min":0.66800000000000004},"max":{"1min":1.655,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.65700000000000003},"max":{"1min":1.5529999999999999,"5min":1.984,"15min":2.1840000000000002},"last":1.056}]},{"osd":5,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.345,"15min":1.3220000000000001},"min":{"1min":0.98899999999999999,"5min":0.73599999999999999,"15min":0.71899999999999997},"max":{"1min":2.008,"5min":2.407,"15min":2.407},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.64500000000000002},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.2250000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2889999999999999,"15min":1.256},"min":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.73799999999999999},"max":{"1min":1.5549999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3999999999999999},"min":{"1min":0.84899999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1120000000000001,"5min":2.6699999999999999,"15min":3.496},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.369},"min":{"1min":0.871,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.9910000000000001,"15min":2.2989999999999999},"last":1.347},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.389,"15min":1.329},"min":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":1.843,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.367}]}]},{"osd":2,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194357,"statfs":{"total":200000000,"available":199021859,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.41099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2050000000000001,"5min":1.9279999999999999,"15min":2.383},"last":0.627},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96099999999999997,"15min":1.0169999999999999},"min":{"1min":0.59699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.625,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.625,"5min":0.442,"15min":0.442},"max":{"1min":1.5549999999999999,"5min":1.978,"15min":2.3180000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.0229999999999999,"15min":1.044},"min":{"1min":0.59099999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.5369999999999999,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.131,"15min":1.101},"min":{"1min":0.46800000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3460000000000001,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.091,"15min":1.0940000000000001},"min":{"1min":0.69299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4850000000000001,"5min":2.145,"15min":2.2040000000000002},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.29,"15min":1.351},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.72099999999999997},"max":{"1min":1.5409999999999999,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.274,"15min":1.3120000000000001},"min":{"1min":0.92100000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.631,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2450000000000001,"15min":1.3260000000000001},"min":{"1min":0.83599999999999997,"5min":0.73199999999999998,"15min":0.70799999999999996},"max":{"1min":1.5149999999999999,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.375},"min":{"1min":0.99399999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.631,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3999999999999999,"15min":1.4099999999999999},"min":{"1min":1.02,"5min":0.755,"15min":0.73199999999999998},"max":{"1min":1.714,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.409,"15min":1.4510000000000001},"min":{"1min":1.024,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.855,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":1.109}]},{"osd":7,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.369,"15min":1.4199999999999999},"min":{"1min":0.87,"5min":0.74199999999999999,"15min":0.65700000000000003},"max":{"1min":1.7110000000000001,"5min":2.331,"15min":2.968},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.371,"5min":1.413,"15min":1.4690000000000001},"min":{"1min":1.0409999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7470000000000001,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.077}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1531,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193780,"statfs":{"total":200000000,"available":198431491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84499999999999997,"15min":0.79600000000000004},"min":{"1min":0.46000000000000002,"5min":0.36399999999999999,"15min":0.29699999999999999},"max":{"1min":1.26,"5min":1.5600000000000001,"15min":1.627},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.94099999999999995,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.5860000000000001,"5min":1.776,"15min":1.895},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.83199999999999996},"min":{"1min":0.57299999999999995,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3320000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.98199999999999998,"15min":0.92900000000000005},"min":{"1min":0.53500000000000003,"5min":0.46000000000000002,"15min":0.33600000000000002},"max":{"1min":1.508,"5min":1.6220000000000001,"15min":1.746},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1200000000000001,"15min":1.0649999999999999},"min":{"1min":0.55000000000000004,"5min":0.50800000000000001,"15min":0.372},"max":{"1min":1.518,"5min":1.8680000000000001,"15min":1.9450000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0489999999999999,"15min":0.97599999999999998},"min":{"1min":0.62,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.655,"5min":1.655,"15min":1.764},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.167,"15min":1.113},"min":{"1min":0.82999999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.476,"5min":1.768,"15min":1.768},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.2450000000000001},"min":{"1min":0.85499999999999998,"5min":0.77700000000000002,"15min":0.629},"max":{"1min":1.5229999999999999,"5min":1.722,"15min":2.008},"last":1.0880000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2529999999999999,"15min":1.212},"min":{"1min":0.80400000000000005,"5min":0.70499999999999996,"15min":0.68600000000000005},"max":{"1min":1.6479999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.248,"5min":1.264,"15min":1.198},"min":{"1min":0.79200000000000004,"5min":0.78300000000000003,"15min":0.61199999999999999},"max":{"1min":1.6180000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.02}]},{"osd":6,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4019999999999999,"15min":1.3340000000000001},"min":{"1min":0.83299999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8879999999999999,"5min":2.1339999999999999,"15min":2.9940000000000002},"last":1.28},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3979999999999999,"15min":1.325},"min":{"1min":0.876,"5min":0.82199999999999995,"15min":0.72099999999999997},"max":{"1min":1.8129999999999999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.22}]},{"osd":7,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4219999999999999,"15min":1.377},"min":{"1min":0.93300000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6930000000000001,"5min":1.968,"15min":2.9529999999999998},"last":1.381},{"interface":"front","average":{"1min":1.484,"5min":1.4510000000000001,"15min":1.359},"min":{"1min":0.86099999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.847,"5min":2.218,"15min":2.218},"last":1.2430000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":956,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194355,"statfs":{"total":200000000,"available":199020157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.875,"15min":0.87},"min":{"1min":0.433,"5min":0.33900000000000002,"15min":0.313},"max":{"1min":1.2989999999999999,"5min":1.9350000000000001,"15min":11.257},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":0.95399999999999996},"min":{"1min":0.52700000000000002,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":1.5309999999999999,"5min":1.829,"15min":6.4260000000000002},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.96699999999999997,"15min":0.94599999999999995},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":1.3899999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.02},{"interface":"front","average":{"1min":1.01,"5min":1.02,"15min":1.0600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.421,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.97599999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.2270000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.629,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1459999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2729999999999999,"15min":1.2390000000000001},"min":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6930000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3720000000000001,"15min":1.357},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7549999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.4179999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2829999999999999,"15min":1.3260000000000001},"min":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.68000000000000005},"max":{"1min":1.488,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2669999999999999,"15min":1.284},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.8500000000000001,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3600000000000001,"15min":1.343},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7030000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.244},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3480000000000001,"15min":1.3089999999999999},"min":{"1min":0.80500000000000005,"5min":0.752,"15min":0.72699999999999998},"max":{"1min":1.7889999999999999,"5min":2.331,"15min":2.331},"last":1.268}]},{"osd":7,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4139999999999999,"15min":1.3959999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.6830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.349,"15min":1.3240000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7949999999999999,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":1.3260000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:48:13.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:48:13.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.514+0000 7f0ff8322640 1 Processor -- start 2023-08-25T09:48:13.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.515+0000 7f0ff8322640 1 -- start start 2023-08-25T09:48:13.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.516+0000 7f0ff8322640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 0x7f0ff015e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.516+0000 7f0ff8322640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0161640 con 0x7f0ff00cce70 2023-08-25T09:48:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.516+0000 7f0ff8322640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff0161920 con 0x7f0ff006ca40 2023-08-25T09:48:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.516+0000 7f0ff8322640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff015d880 con 0x7f0ff00adf80 2023-08-25T09:48:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.516+0000 7f0ff60be640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0ff00cce70 0x7f0ff0160d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57316/0 (socket says 172.21.15.70:57316) 2023-08-25T09:48:13.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.516+0000 7f0ff60be640 1 -- 172.21.15.70:0/485118148 learned_addr learned my addr 172.21.15.70:0/485118148 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:13.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231418480 0 0) 0x7f0ff0161640 con 0x7f0ff00cce70 2023-08-25T09:48:13.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff58bd640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 0x7f0ff015e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fdc003520 con 0x7f0ff00cce70 2023-08-25T09:48:13.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 669358553 0 0) 0x7f0fdc003520 con 0x7f0ff00cce70 2023-08-25T09:48:13.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 msgr2=0x7f0ff015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:13.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 0x7f0ff015e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 >> v1:172.21.15.155:6789/0 conn(0x7f0ff006ca40 legacy=0x7f0ff006b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff015db60 con 0x7f0ff00cce70 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0fec001670 con 0x7f0ff00cce70 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 >> v1:172.21.15.70:6789/0 conn(0x7f0ff00cce70 legacy=0x7f0ff0160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.517+0000 7f0ff50bc640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 0x7f0fdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff50bc640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc004d00 0x7f0fdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff015d880 con 0x7f0fdc0039a0 2023-08-25T09:48:13.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc00ba60 con 0x7f0fdc004d00 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc00bc20 con 0x7f0ff00adf80 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff60be640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff60be640 1 -- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 msgr2=0x7f0fdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff60be640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00adf80 0x7f0fdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff60be640 1 -- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc004d00 msgr2=0x7f0fdc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff60be640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0fdc004d00 0x7f0fdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.518+0000 7f0ff60be640 1 -- 172.21.15.70:0/485118148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc003520 con 0x7f0fdc0039a0 2023-08-25T09:48:13.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.519+0000 7f0ff60be640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c210 tx=0x7f0fdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3c763acdfc1760 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.519+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec005ad0 con 0x7f0fdc0039a0 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.519+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec005c30 con 0x7f0fdc0039a0 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.519+0000 7f0ff50bc640 1 -- 172.21.15.70:0/485118148 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec02c890 con 0x7f0fdc0039a0 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff8322640 1 -- 172.21.15.70:0/485118148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 msgr2=0x7f0fdc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff8322640 1 --2- 172.21.15.70:0/485118148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0fdc003dc0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c210 tx=0x7f0fdc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff68bf640 1 -- 172.21.15.70:0/485118148 reap_dead start 2023-08-25T09:48:13.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff8322640 1 -- 172.21.15.70:0/485118148 shutdown_connections 2023-08-25T09:48:13.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff8322640 1 -- 172.21.15.70:0/485118148 >> 172.21.15.70:0/485118148 conn(0x7f0ff00ceae0 msgr2=0x7f0ff00ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:13.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff8322640 1 -- 172.21.15.70:0/485118148 shutdown_connections 2023-08-25T09:48:13.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.520+0000 7f0ff8322640 1 -- 172.21.15.70:0/485118148 wait complete. 2023-08-25T09:48:13.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.522+0000 7f0ff8322640 1 Processor -- start 2023-08-25T09:48:13.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.522+0000 7f0ff8322640 1 -- start start 2023-08-25T09:48:13.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff8322640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0ff00c3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff8322640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ff00c29b0 0x7f0ff014af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff8322640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00c4190 0x7f0ff00c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff8322640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00c5240 con 0x7f0fdc0039a0 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff8322640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00c4630 con 0x7f0ff00c29b0 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff8322640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00c4910 con 0x7f0ff00c4190 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff60be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0ff00c3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff60be640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0ff00c3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44652/0 (socket says 172.21.15.70:44652) 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.523+0000 7f0ff60be640 1 -- 172.21.15.70:0/3697429800 learned_addr learned my addr 172.21.15.70:0/3697429800 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff60be640 1 -- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00c4190 msgr2=0x7f0ff00c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff68bf640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00c4190 0x7f0ff00c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff58bd640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ff00c29b0 0x7f0ff014af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff60be640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00c4190 0x7f0ff00c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff60be640 1 -- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ff00c29b0 msgr2=0x7f0ff014af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff60be640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ff00c29b0 0x7f0ff014af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff60be640 1 -- 172.21.15.70:0/3697429800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00c4bf0 con 0x7f0fdc0039a0 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff60be640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0ff00c3c50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0fec02c430 tx=0x7f0fec001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec039070 con 0x7f0fdc0039a0 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec02cca0 con 0x7f0fdc0039a0 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec03c040 con 0x7f0fdc0039a0 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff00c4ed0 con 0x7f0fdc0039a0 2023-08-25T09:48:13.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.524+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff00c58b0 con 0x7f0fdc0039a0 2023-08-25T09:48:13.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.526+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff004e420 con 0x7f0fdc0039a0 2023-08-25T09:48:13.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.527+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0fec032070 con 0x7f0fdc0039a0 2023-08-25T09:48:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.527+0000 7f0fe6ffd640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fd406a050 0x7f0fd406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.528+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0fec0a9a00 con 0x7f0fdc0039a0 2023-08-25T09:48:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.528+0000 7f0ff58bd640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fd406a050 0x7f0fd406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:13.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.529+0000 7f0ff58bd640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fd406a050 0x7f0fd406c510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00c2f40 tx=0x7f0fe0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:13.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.532+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0fec075050 con 0x7f0fdc0039a0 2023-08-25T09:48:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.717+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ff0161c00 con 0x7f0fd406a050 2023-08-25T09:48:13.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.741+0000 7f0fe6ffd640 1 -- 172.21.15.70:0/3697429800 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+303412 (secure 0 0 0) 0x7f0fe0009e40 con 0x7f0fd406a050 2023-08-25T09:48:13.746 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:48:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.746+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fd406a050 msgr2=0x7f0fd406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.746+0000 7f0ff8322640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fd406a050 0x7f0fd406c510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00c2f40 tx=0x7f0fe0008040 comp rx=0 tx=0).stop 2023-08-25T09:48:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.746+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 msgr2=0x7f0ff00c3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.746+0000 7f0ff8322640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0ff00c3c50 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f0fec02c430 tx=0x7f0fec001a00 comp rx=0 tx=0).stop 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 shutdown_connections 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0fd406a050 0x7f0fd406c510 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0ff00c4190 0x7f0ff00c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0ff00c29b0 0x7f0ff014af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 --2- 172.21.15.70:0/3697429800 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0fdc0039a0 0x7f0ff00c3c50 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 >> 172.21.15.70:0/3697429800 conn(0x7f0ff00ceae0 msgr2=0x7f0ff00af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 shutdown_connections 2023-08-25T09:48:13.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:13.747+0000 7f0ff8322640 1 -- 172.21.15.70:0/3697429800 wait complete. 2023-08-25T09:48:13.770 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2023-08-25T09:48:13.507300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8790,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553709,"statfs":{"total":1600000000,"available":1590998876,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001537"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446207+0000","last_change":"2023-08-25T09:48:08.445972+0000","last_active":"2023-08-25T09:48:08.446207+0000","last_peered":"2023-08-25T09:48:08.446207+0000","last_clean":"2023-08-25T09:48:08.446207+0000","last_became_active":"2023-08-25T09:48:08.444843+0000","last_became_peered":"2023-08-25T09:48:08.444843+0000","last_unstale":"2023-08-25T09:48:08.446207+0000","last_undegraded":"2023-08-25T09:48:08.446207+0000","last_fullsized":"2023-08-25T09:48:08.446207+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:11:22.285471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442914+0000","last_change":"2023-08-25T09:48:08.441613+0000","last_active":"2023-08-25T09:48:08.442914+0000","last_peered":"2023-08-25T09:48:08.442914+0000","last_clean":"2023-08-25T09:48:08.442914+0000","last_became_active":"2023-08-25T09:48:08.440821+0000","last_became_peered":"2023-08-25T09:48:08.440821+0000","last_unstale":"2023-08-25T09:48:08.442914+0000","last_undegraded":"2023-08-25T09:48:08.442914+0000","last_fullsized":"2023-08-25T09:48:08.442914+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:08:04.719495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447045+0000","last_change":"2023-08-25T09:48:08.446273+0000","last_active":"2023-08-25T09:48:08.447045+0000","last_peered":"2023-08-25T09:48:08.447045+0000","last_clean":"2023-08-25T09:48:08.447045+0000","last_became_active":"2023-08-25T09:48:08.444885+0000","last_became_peered":"2023-08-25T09:48:08.444885+0000","last_unstale":"2023-08-25T09:48:08.447045+0000","last_undegraded":"2023-08-25T09:48:08.447045+0000","last_fullsized":"2023-08-25T09:48:08.447045+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:15:12.785972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.449216+0000","last_change":"2023-08-25T09:48:08.448477+0000","last_active":"2023-08-25T09:48:08.449216+0000","last_peered":"2023-08-25T09:48:08.449216+0000","last_clean":"2023-08-25T09:48:08.449216+0000","last_became_active":"2023-08-25T09:48:08.443560+0000","last_became_peered":"2023-08-25T09:48:08.443560+0000","last_unstale":"2023-08-25T09:48:08.449216+0000","last_undegraded":"2023-08-25T09:48:08.449216+0000","last_fullsized":"2023-08-25T09:48:08.449216+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:00.991827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442914+0000","last_change":"2023-08-25T09:48:08.442812+0000","last_active":"2023-08-25T09:48:08.442914+0000","last_peered":"2023-08-25T09:48:08.442914+0000","last_clean":"2023-08-25T09:48:08.442914+0000","last_became_active":"2023-08-25T09:48:08.441305+0000","last_became_peered":"2023-08-25T09:48:08.441305+0000","last_unstale":"2023-08-25T09:48:08.442914+0000","last_undegraded":"2023-08-25T09:48:08.442914+0000","last_fullsized":"2023-08-25T09:48:08.442914+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:29.438853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.431332+0000","last_change":"2023-08-25T09:48:08.425993+0000","last_active":"2023-08-25T09:48:08.431332+0000","last_peered":"2023-08-25T09:48:08.431332+0000","last_clean":"2023-08-25T09:48:08.431332+0000","last_became_active":"2023-08-25T09:48:08.425208+0000","last_became_peered":"2023-08-25T09:48:08.425208+0000","last_unstale":"2023-08-25T09:48:08.431332+0000","last_undegraded":"2023-08-25T09:48:08.431332+0000","last_fullsized":"2023-08-25T09:48:08.431332+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:17:45.563574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441460+0000","last_change":"2023-08-25T09:48:08.441337+0000","last_active":"2023-08-25T09:48:08.441460+0000","last_peered":"2023-08-25T09:48:08.441460+0000","last_clean":"2023-08-25T09:48:08.441460+0000","last_became_active":"2023-08-25T09:48:08.440296+0000","last_became_peered":"2023-08-25T09:48:08.440296+0000","last_unstale":"2023-08-25T09:48:08.441460+0000","last_undegraded":"2023-08-25T09:48:08.441460+0000","last_fullsized":"2023-08-25T09:48:08.441460+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:52:34.591238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447029+0000","last_change":"2023-08-25T09:48:08.446084+0000","last_active":"2023-08-25T09:48:08.447029+0000","last_peered":"2023-08-25T09:48:08.447029+0000","last_clean":"2023-08-25T09:48:08.447029+0000","last_became_active":"2023-08-25T09:48:08.444444+0000","last_became_peered":"2023-08-25T09:48:08.444444+0000","last_unstale":"2023-08-25T09:48:08.447029+0000","last_undegraded":"2023-08-25T09:48:08.447029+0000","last_fullsized":"2023-08-25T09:48:08.447029+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:54:50.721713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446073+0000","last_change":"2023-08-25T09:48:08.445893+0000","last_active":"2023-08-25T09:48:08.446073+0000","last_peered":"2023-08-25T09:48:08.446073+0000","last_clean":"2023-08-25T09:48:08.446073+0000","last_became_active":"2023-08-25T09:48:08.442540+0000","last_became_peered":"2023-08-25T09:48:08.442540+0000","last_unstale":"2023-08-25T09:48:08.446073+0000","last_undegraded":"2023-08-25T09:48:08.446073+0000","last_fullsized":"2023-08-25T09:48:08.446073+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:43:52.258967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443302+0000","last_change":"2023-08-25T09:48:08.443111+0000","last_active":"2023-08-25T09:48:08.443302+0000","last_peered":"2023-08-25T09:48:08.443302+0000","last_clean":"2023-08-25T09:48:08.443302+0000","last_became_active":"2023-08-25T09:48:08.441724+0000","last_became_peered":"2023-08-25T09:48:08.441724+0000","last_unstale":"2023-08-25T09:48:08.443302+0000","last_undegraded":"2023-08-25T09:48:08.443302+0000","last_fullsized":"2023-08-25T09:48:08.443302+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:17.179847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450353+0000","last_change":"2023-08-25T09:48:08.450278+0000","last_active":"2023-08-25T09:48:08.450353+0000","last_peered":"2023-08-25T09:48:08.450353+0000","last_clean":"2023-08-25T09:48:08.450353+0000","last_became_active":"2023-08-25T09:48:08.450053+0000","last_became_peered":"2023-08-25T09:48:08.450053+0000","last_unstale":"2023-08-25T09:48:08.450353+0000","last_undegraded":"2023-08-25T09:48:08.450353+0000","last_fullsized":"2023-08-25T09:48:08.450353+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:49:07.032545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442387+0000","last_change":"2023-08-25T09:48:08.442270+0000","last_active":"2023-08-25T09:48:08.442387+0000","last_peered":"2023-08-25T09:48:08.442387+0000","last_clean":"2023-08-25T09:48:08.442387+0000","last_became_active":"2023-08-25T09:48:08.440508+0000","last_became_peered":"2023-08-25T09:48:08.440508+0000","last_unstale":"2023-08-25T09:48:08.442387+0000","last_undegraded":"2023-08-25T09:48:08.442387+0000","last_fullsized":"2023-08-25T09:48:08.442387+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:51:18.775838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436209+0000","last_change":"2023-08-25T09:48:08.436087+0000","last_active":"2023-08-25T09:48:08.436209+0000","last_peered":"2023-08-25T09:48:08.436209+0000","last_clean":"2023-08-25T09:48:08.436209+0000","last_became_active":"2023-08-25T09:48:08.435677+0000","last_became_peered":"2023-08-25T09:48:08.435677+0000","last_unstale":"2023-08-25T09:48:08.436209+0000","last_undegraded":"2023-08-25T09:48:08.436209+0000","last_fullsized":"2023-08-25T09:48:08.436209+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:35:51.412561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450285+0000","last_change":"2023-08-25T09:48:08.450196+0000","last_active":"2023-08-25T09:48:08.450285+0000","last_peered":"2023-08-25T09:48:08.450285+0000","last_clean":"2023-08-25T09:48:08.450285+0000","last_became_active":"2023-08-25T09:48:08.450053+0000","last_became_peered":"2023-08-25T09:48:08.450053+0000","last_unstale":"2023-08-25T09:48:08.450285+0000","last_undegraded":"2023-08-25T09:48:08.450285+0000","last_fullsized":"2023-08-25T09:48:08.450285+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:16:33.877843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439971+0000","last_change":"2023-08-25T09:48:08.439832+0000","last_active":"2023-08-25T09:48:08.439971+0000","last_peered":"2023-08-25T09:48:08.439971+0000","last_clean":"2023-08-25T09:48:08.439971+0000","last_became_active":"2023-08-25T09:48:08.439445+0000","last_became_peered":"2023-08-25T09:48:08.439445+0000","last_unstale":"2023-08-25T09:48:08.439971+0000","last_undegraded":"2023-08-25T09:48:08.439971+0000","last_fullsized":"2023-08-25T09:48:08.439971+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:19:58.494293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.445961+0000","last_change":"2023-08-25T09:48:08.445458+0000","last_active":"2023-08-25T09:48:08.445961+0000","last_peered":"2023-08-25T09:48:08.445961+0000","last_clean":"2023-08-25T09:48:08.445961+0000","last_became_active":"2023-08-25T09:48:08.444539+0000","last_became_peered":"2023-08-25T09:48:08.444539+0000","last_unstale":"2023-08-25T09:48:08.445961+0000","last_undegraded":"2023-08-25T09:48:08.445961+0000","last_fullsized":"2023-08-25T09:48:08.445961+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:36:47.168629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.437939+0000","last_change":"2023-08-25T09:48:08.436562+0000","last_active":"2023-08-25T09:48:08.437939+0000","last_peered":"2023-08-25T09:48:08.437939+0000","last_clean":"2023-08-25T09:48:08.437939+0000","last_became_active":"2023-08-25T09:48:08.432548+0000","last_became_peered":"2023-08-25T09:48:08.432548+0000","last_unstale":"2023-08-25T09:48:08.437939+0000","last_undegraded":"2023-08-25T09:48:08.437939+0000","last_fullsized":"2023-08-25T09:48:08.437939+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:18:09.453496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444248+0000","last_change":"2023-08-25T09:48:08.443617+0000","last_active":"2023-08-25T09:48:08.444248+0000","last_peered":"2023-08-25T09:48:08.444248+0000","last_clean":"2023-08-25T09:48:08.444248+0000","last_became_active":"2023-08-25T09:48:08.443058+0000","last_became_peered":"2023-08-25T09:48:08.443058+0000","last_unstale":"2023-08-25T09:48:08.444248+0000","last_undegraded":"2023-08-25T09:48:08.444248+0000","last_fullsized":"2023-08-25T09:48:08.444248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:32:46.687002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450224+0000","last_change":"2023-08-25T09:48:08.450148+0000","last_active":"2023-08-25T09:48:08.450224+0000","last_peered":"2023-08-25T09:48:08.450224+0000","last_clean":"2023-08-25T09:48:08.450224+0000","last_became_active":"2023-08-25T09:48:08.449893+0000","last_became_peered":"2023-08-25T09:48:08.449893+0000","last_unstale":"2023-08-25T09:48:08.450224+0000","last_undegraded":"2023-08-25T09:48:08.450224+0000","last_fullsized":"2023-08-25T09:48:08.450224+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:07.891935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442801+0000","last_change":"2023-08-25T09:48:08.442695+0000","last_active":"2023-08-25T09:48:08.442801+0000","last_peered":"2023-08-25T09:48:08.442801+0000","last_clean":"2023-08-25T09:48:08.442801+0000","last_became_active":"2023-08-25T09:48:08.442347+0000","last_became_peered":"2023-08-25T09:48:08.442347+0000","last_unstale":"2023-08-25T09:48:08.442801+0000","last_undegraded":"2023-08-25T09:48:08.442801+0000","last_fullsized":"2023-08-25T09:48:08.442801+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:13:26.158786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447602+0000","last_change":"2023-08-25T09:48:08.442351+0000","last_active":"2023-08-25T09:48:08.447602+0000","last_peered":"2023-08-25T09:48:08.447602+0000","last_clean":"2023-08-25T09:48:08.447602+0000","last_became_active":"2023-08-25T09:48:08.442041+0000","last_became_peered":"2023-08-25T09:48:08.442041+0000","last_unstale":"2023-08-25T09:48:08.447602+0000","last_undegraded":"2023-08-25T09:48:08.447602+0000","last_fullsized":"2023-08-25T09:48:08.447602+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:56.775903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443826+0000","last_change":"2023-08-25T09:48:08.443646+0000","last_active":"2023-08-25T09:48:08.443826+0000","last_peered":"2023-08-25T09:48:08.443826+0000","last_clean":"2023-08-25T09:48:08.443826+0000","last_became_active":"2023-08-25T09:48:08.443378+0000","last_became_peered":"2023-08-25T09:48:08.443378+0000","last_unstale":"2023-08-25T09:48:08.443826+0000","last_undegraded":"2023-08-25T09:48:08.443826+0000","last_fullsized":"2023-08-25T09:48:08.443826+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:27:21.865053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448157+0000","last_change":"2023-08-25T09:48:08.448088+0000","last_active":"2023-08-25T09:48:08.448157+0000","last_peered":"2023-08-25T09:48:08.448157+0000","last_clean":"2023-08-25T09:48:08.448157+0000","last_became_active":"2023-08-25T09:48:08.447687+0000","last_became_peered":"2023-08-25T09:48:08.447687+0000","last_unstale":"2023-08-25T09:48:08.448157+0000","last_undegraded":"2023-08-25T09:48:08.448157+0000","last_fullsized":"2023-08-25T09:48:08.448157+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:10:58.564019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.445422+0000","last_change":"2023-08-25T09:48:08.445300+0000","last_active":"2023-08-25T09:48:08.445422+0000","last_peered":"2023-08-25T09:48:08.445422+0000","last_clean":"2023-08-25T09:48:08.445422+0000","last_became_active":"2023-08-25T09:48:08.441646+0000","last_became_peered":"2023-08-25T09:48:08.441646+0000","last_unstale":"2023-08-25T09:48:08.445422+0000","last_undegraded":"2023-08-25T09:48:08.445422+0000","last_fullsized":"2023-08-25T09:48:08.445422+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:59:28.925816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446998+0000","last_change":"2023-08-25T09:48:08.445279+0000","last_active":"2023-08-25T09:48:08.446998+0000","last_peered":"2023-08-25T09:48:08.446998+0000","last_clean":"2023-08-25T09:48:08.446998+0000","last_became_active":"2023-08-25T09:48:08.438573+0000","last_became_peered":"2023-08-25T09:48:08.438573+0000","last_unstale":"2023-08-25T09:48:08.446998+0000","last_undegraded":"2023-08-25T09:48:08.446998+0000","last_fullsized":"2023-08-25T09:48:08.446998+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:34:53.098145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450154+0000","last_change":"2023-08-25T09:48:08.450042+0000","last_active":"2023-08-25T09:48:08.450154+0000","last_peered":"2023-08-25T09:48:08.450154+0000","last_clean":"2023-08-25T09:48:08.450154+0000","last_became_active":"2023-08-25T09:48:08.449229+0000","last_became_peered":"2023-08-25T09:48:08.449229+0000","last_unstale":"2023-08-25T09:48:08.450154+0000","last_undegraded":"2023-08-25T09:48:08.450154+0000","last_fullsized":"2023-08-25T09:48:08.450154+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:50.738200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444328+0000","last_change":"2023-08-25T09:48:08.444233+0000","last_active":"2023-08-25T09:48:08.444328+0000","last_peered":"2023-08-25T09:48:08.444328+0000","last_clean":"2023-08-25T09:48:08.444328+0000","last_became_active":"2023-08-25T09:48:08.434945+0000","last_became_peered":"2023-08-25T09:48:08.434945+0000","last_unstale":"2023-08-25T09:48:08.444328+0000","last_undegraded":"2023-08-25T09:48:08.444328+0000","last_fullsized":"2023-08-25T09:48:08.444328+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:58:01.789008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.425544+0000","last_change":"2023-08-25T09:48:08.425364+0000","last_active":"2023-08-25T09:48:08.425544+0000","last_peered":"2023-08-25T09:48:08.425544+0000","last_clean":"2023-08-25T09:48:08.425544+0000","last_became_active":"2023-08-25T09:48:08.424671+0000","last_became_peered":"2023-08-25T09:48:08.424671+0000","last_unstale":"2023-08-25T09:48:08.425544+0000","last_undegraded":"2023-08-25T09:48:08.425544+0000","last_fullsized":"2023-08-25T09:48:08.425544+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:11:06.014011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446058+0000","last_change":"2023-08-25T09:48:08.445865+0000","last_active":"2023-08-25T09:48:08.446058+0000","last_peered":"2023-08-25T09:48:08.446058+0000","last_clean":"2023-08-25T09:48:08.446058+0000","last_became_active":"2023-08-25T09:48:08.445501+0000","last_became_peered":"2023-08-25T09:48:08.445501+0000","last_unstale":"2023-08-25T09:48:08.446058+0000","last_undegraded":"2023-08-25T09:48:08.446058+0000","last_fullsized":"2023-08-25T09:48:08.446058+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:22:31.286612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450678+0000","last_change":"2023-08-25T09:48:08.450569+0000","last_active":"2023-08-25T09:48:08.450678+0000","last_peered":"2023-08-25T09:48:08.450678+0000","last_clean":"2023-08-25T09:48:08.450678+0000","last_became_active":"2023-08-25T09:48:08.450310+0000","last_became_peered":"2023-08-25T09:48:08.450310+0000","last_unstale":"2023-08-25T09:48:08.450678+0000","last_undegraded":"2023-08-25T09:48:08.450678+0000","last_fullsized":"2023-08-25T09:48:08.450678+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:18:18.990796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441703+0000","last_change":"2023-08-25T09:48:08.441589+0000","last_active":"2023-08-25T09:48:08.441703+0000","last_peered":"2023-08-25T09:48:08.441703+0000","last_clean":"2023-08-25T09:48:08.441703+0000","last_became_active":"2023-08-25T09:48:08.441284+0000","last_became_peered":"2023-08-25T09:48:08.441284+0000","last_unstale":"2023-08-25T09:48:08.441703+0000","last_undegraded":"2023-08-25T09:48:08.441703+0000","last_fullsized":"2023-08-25T09:48:08.441703+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:45:02.619368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441011+0000","last_change":"2023-08-25T09:48:08.440904+0000","last_active":"2023-08-25T09:48:08.441011+0000","last_peered":"2023-08-25T09:48:08.441011+0000","last_clean":"2023-08-25T09:48:08.441011+0000","last_became_active":"2023-08-25T09:48:08.440508+0000","last_became_peered":"2023-08-25T09:48:08.440508+0000","last_unstale":"2023-08-25T09:48:08.441011+0000","last_undegraded":"2023-08-25T09:48:08.441011+0000","last_fullsized":"2023-08-25T09:48:08.441011+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:48:03.327396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446978+0000","last_change":"2023-08-25T09:48:08.445129+0000","last_active":"2023-08-25T09:48:08.446978+0000","last_peered":"2023-08-25T09:48:08.446978+0000","last_clean":"2023-08-25T09:48:08.446978+0000","last_became_active":"2023-08-25T09:48:08.438083+0000","last_became_peered":"2023-08-25T09:48:08.438083+0000","last_unstale":"2023-08-25T09:48:08.446978+0000","last_undegraded":"2023-08-25T09:48:08.446978+0000","last_fullsized":"2023-08-25T09:48:08.446978+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:58:27.581262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1f","version":"164'4","reported_seq":15,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846593+0000","last_change":"2023-08-25T09:48:07.808083+0000","last_active":"2023-08-25T09:48:08.846593+0000","last_peered":"2023-08-25T09:48:08.846593+0000","last_clean":"2023-08-25T09:48:08.846593+0000","last_became_active":"2023-08-25T09:48:07.807728+0000","last_became_peered":"2023-08-25T09:48:07.807728+0000","last_unstale":"2023-08-25T09:48:08.846593+0000","last_undegraded":"2023-08-25T09:48:08.846593+0000","last_fullsized":"2023-08-25T09:48:08.846593+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:55:50.560573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448296+0000","last_change":"2023-08-25T09:48:08.448179+0000","last_active":"2023-08-25T09:48:08.448296+0000","last_peered":"2023-08-25T09:48:08.448296+0000","last_clean":"2023-08-25T09:48:08.448296+0000","last_became_active":"2023-08-25T09:48:08.446644+0000","last_became_peered":"2023-08-25T09:48:08.446644+0000","last_unstale":"2023-08-25T09:48:08.448296+0000","last_undegraded":"2023-08-25T09:48:08.448296+0000","last_fullsized":"2023-08-25T09:48:08.448296+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:27:51.035464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.1e","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.847133+0000","last_change":"2023-08-25T09:48:07.814371+0000","last_active":"2023-08-25T09:48:08.847133+0000","last_peered":"2023-08-25T09:48:08.847133+0000","last_clean":"2023-08-25T09:48:08.847133+0000","last_became_active":"2023-08-25T09:48:07.813571+0000","last_became_peered":"2023-08-25T09:48:07.813571+0000","last_unstale":"2023-08-25T09:48:08.847133+0000","last_undegraded":"2023-08-25T09:48:08.847133+0000","last_fullsized":"2023-08-25T09:48:08.847133+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:46:26.094932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447990+0000","last_change":"2023-08-25T09:48:08.447912+0000","last_active":"2023-08-25T09:48:08.447990+0000","last_peered":"2023-08-25T09:48:08.447990+0000","last_clean":"2023-08-25T09:48:08.447990+0000","last_became_active":"2023-08-25T09:48:08.447183+0000","last_became_peered":"2023-08-25T09:48:08.447183+0000","last_unstale":"2023-08-25T09:48:08.447990+0000","last_undegraded":"2023-08-25T09:48:08.447990+0000","last_fullsized":"2023-08-25T09:48:08.447990+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:35.688792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1d","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.844309+0000","last_change":"2023-08-25T09:48:07.816558+0000","last_active":"2023-08-25T09:48:08.844309+0000","last_peered":"2023-08-25T09:48:08.844309+0000","last_clean":"2023-08-25T09:48:08.844309+0000","last_became_active":"2023-08-25T09:48:07.816259+0000","last_became_peered":"2023-08-25T09:48:07.816259+0000","last_unstale":"2023-08-25T09:48:08.844309+0000","last_undegraded":"2023-08-25T09:48:08.844309+0000","last_fullsized":"2023-08-25T09:48:08.844309+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:59:16.099743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448219+0000","last_change":"2023-08-25T09:48:08.448142+0000","last_active":"2023-08-25T09:48:08.448219+0000","last_peered":"2023-08-25T09:48:08.448219+0000","last_clean":"2023-08-25T09:48:08.448219+0000","last_became_active":"2023-08-25T09:48:08.447799+0000","last_became_peered":"2023-08-25T09:48:08.447799+0000","last_unstale":"2023-08-25T09:48:08.448219+0000","last_undegraded":"2023-08-25T09:48:08.448219+0000","last_fullsized":"2023-08-25T09:48:08.448219+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:05:01.547555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1c","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846095+0000","last_change":"2023-08-25T09:48:07.802767+0000","last_active":"2023-08-25T09:48:08.846095+0000","last_peered":"2023-08-25T09:48:08.846095+0000","last_clean":"2023-08-25T09:48:08.846095+0000","last_became_active":"2023-08-25T09:48:07.802312+0000","last_became_peered":"2023-08-25T09:48:07.802312+0000","last_unstale":"2023-08-25T09:48:08.846095+0000","last_undegraded":"2023-08-25T09:48:08.846095+0000","last_fullsized":"2023-08-25T09:48:08.846095+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:02:53.485651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436951+0000","last_change":"2023-08-25T09:48:08.431617+0000","last_active":"2023-08-25T09:48:08.436951+0000","last_peered":"2023-08-25T09:48:08.436951+0000","last_clean":"2023-08-25T09:48:08.436951+0000","last_became_active":"2023-08-25T09:48:08.430838+0000","last_became_peered":"2023-08-25T09:48:08.430838+0000","last_unstale":"2023-08-25T09:48:08.436951+0000","last_undegraded":"2023-08-25T09:48:08.436951+0000","last_fullsized":"2023-08-25T09:48:08.436951+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:56:39.601468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1b","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.845740+0000","last_change":"2023-08-25T09:48:07.812478+0000","last_active":"2023-08-25T09:48:08.845740+0000","last_peered":"2023-08-25T09:48:08.845740+0000","last_clean":"2023-08-25T09:48:08.845740+0000","last_became_active":"2023-08-25T09:48:07.812336+0000","last_became_peered":"2023-08-25T09:48:07.812336+0000","last_unstale":"2023-08-25T09:48:08.845740+0000","last_undegraded":"2023-08-25T09:48:08.845740+0000","last_fullsized":"2023-08-25T09:48:08.845740+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:20:24.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450368+0000","last_change":"2023-08-25T09:48:08.450275+0000","last_active":"2023-08-25T09:48:08.450368+0000","last_peered":"2023-08-25T09:48:08.450368+0000","last_clean":"2023-08-25T09:48:08.450368+0000","last_became_active":"2023-08-25T09:48:08.448989+0000","last_became_peered":"2023-08-25T09:48:08.448989+0000","last_unstale":"2023-08-25T09:48:08.450368+0000","last_undegraded":"2023-08-25T09:48:08.450368+0000","last_fullsized":"2023-08-25T09:48:08.450368+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:28.295414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1a","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.849291+0000","last_change":"2023-08-25T09:48:07.798393+0000","last_active":"2023-08-25T09:48:08.849291+0000","last_peered":"2023-08-25T09:48:08.849291+0000","last_clean":"2023-08-25T09:48:08.849291+0000","last_became_active":"2023-08-25T09:48:07.796692+0000","last_became_peered":"2023-08-25T09:48:07.796692+0000","last_unstale":"2023-08-25T09:48:08.849291+0000","last_undegraded":"2023-08-25T09:48:08.849291+0000","last_fullsized":"2023-08-25T09:48:08.849291+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:49:37.676551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447602+0000","last_change":"2023-08-25T09:48:08.446984+0000","last_active":"2023-08-25T09:48:08.447602+0000","last_peered":"2023-08-25T09:48:08.447602+0000","last_clean":"2023-08-25T09:48:08.447602+0000","last_became_active":"2023-08-25T09:48:08.444577+0000","last_became_peered":"2023-08-25T09:48:08.444577+0000","last_unstale":"2023-08-25T09:48:08.447602+0000","last_undegraded":"2023-08-25T09:48:08.447602+0000","last_fullsized":"2023-08-25T09:48:08.447602+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:39.717951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.425127+0000","last_change":"2023-08-25T09:48:07.811447+0000","last_active":"2023-08-25T09:48:08.425127+0000","last_peered":"2023-08-25T09:48:08.425127+0000","last_clean":"2023-08-25T09:48:08.425127+0000","last_became_active":"2023-08-25T09:48:07.811003+0000","last_became_peered":"2023-08-25T09:48:07.811003+0000","last_unstale":"2023-08-25T09:48:08.425127+0000","last_undegraded":"2023-08-25T09:48:08.425127+0000","last_fullsized":"2023-08-25T09:48:08.425127+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:09.631434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.447014+0000","last_change":"2023-08-25T09:48:08.445321+0000","last_active":"2023-08-25T09:48:08.447014+0000","last_peered":"2023-08-25T09:48:08.447014+0000","last_clean":"2023-08-25T09:48:08.447014+0000","last_became_active":"2023-08-25T09:48:08.441335+0000","last_became_peered":"2023-08-25T09:48:08.441335+0000","last_unstale":"2023-08-25T09:48:08.447014+0000","last_undegraded":"2023-08-25T09:48:08.447014+0000","last_fullsized":"2023-08-25T09:48:08.447014+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:50.342277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.429588+0000","last_change":"2023-08-25T09:48:07.804809+0000","last_active":"2023-08-25T09:48:08.429588+0000","last_peered":"2023-08-25T09:48:08.429588+0000","last_clean":"2023-08-25T09:48:08.429588+0000","last_became_active":"2023-08-25T09:48:07.804396+0000","last_became_peered":"2023-08-25T09:48:07.804396+0000","last_unstale":"2023-08-25T09:48:08.429588+0000","last_undegraded":"2023-08-25T09:48:08.429588+0000","last_fullsized":"2023-08-25T09:48:08.429588+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:10.703210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444207+0000","last_change":"2023-08-25T09:48:08.444108+0000","last_active":"2023-08-25T09:48:08.444207+0000","last_peered":"2023-08-25T09:48:08.444207+0000","last_clean":"2023-08-25T09:48:08.444207+0000","last_became_active":"2023-08-25T09:48:08.443760+0000","last_became_peered":"2023-08-25T09:48:08.443760+0000","last_unstale":"2023-08-25T09:48:08.444207+0000","last_undegraded":"2023-08-25T09:48:08.444207+0000","last_fullsized":"2023-08-25T09:48:08.444207+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:18.708379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.f","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.843504+0000","last_change":"2023-08-25T09:48:07.807308+0000","last_active":"2023-08-25T09:48:08.843504+0000","last_peered":"2023-08-25T09:48:08.843504+0000","last_clean":"2023-08-25T09:48:08.843504+0000","last_became_active":"2023-08-25T09:48:07.807077+0000","last_became_peered":"2023-08-25T09:48:07.807077+0000","last_unstale":"2023-08-25T09:48:08.843504+0000","last_undegraded":"2023-08-25T09:48:08.843504+0000","last_fullsized":"2023-08-25T09:48:08.843504+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:38:50.590459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.438464+0000","last_change":"2023-08-25T09:48:08.438301+0000","last_active":"2023-08-25T09:48:08.438464+0000","last_peered":"2023-08-25T09:48:08.438464+0000","last_clean":"2023-08-25T09:48:08.438464+0000","last_became_active":"2023-08-25T09:48:08.437811+0000","last_became_peered":"2023-08-25T09:48:08.437811+0000","last_unstale":"2023-08-25T09:48:08.438464+0000","last_undegraded":"2023-08-25T09:48:08.438464+0000","last_fullsized":"2023-08-25T09:48:08.438464+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:09:28.011677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.e","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.844149+0000","last_change":"2023-08-25T09:48:07.815830+0000","last_active":"2023-08-25T09:48:08.844149+0000","last_peered":"2023-08-25T09:48:08.844149+0000","last_clean":"2023-08-25T09:48:08.844149+0000","last_became_active":"2023-08-25T09:48:07.815571+0000","last_became_peered":"2023-08-25T09:48:07.815571+0000","last_unstale":"2023-08-25T09:48:08.844149+0000","last_undegraded":"2023-08-25T09:48:08.844149+0000","last_fullsized":"2023-08-25T09:48:08.844149+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:49.781500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.441711+0000","last_change":"2023-08-25T09:48:08.441613+0000","last_active":"2023-08-25T09:48:08.441711+0000","last_peered":"2023-08-25T09:48:08.441711+0000","last_clean":"2023-08-25T09:48:08.441711+0000","last_became_active":"2023-08-25T09:48:08.441365+0000","last_became_peered":"2023-08-25T09:48:08.441365+0000","last_unstale":"2023-08-25T09:48:08.441711+0000","last_undegraded":"2023-08-25T09:48:08.441711+0000","last_fullsized":"2023-08-25T09:48:08.441711+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:12:04.798252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.d","version":"164'1","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.848697+0000","last_change":"2023-08-25T09:48:07.810838+0000","last_active":"2023-08-25T09:48:08.848697+0000","last_peered":"2023-08-25T09:48:08.848697+0000","last_clean":"2023-08-25T09:48:08.848697+0000","last_became_active":"2023-08-25T09:48:07.810479+0000","last_became_peered":"2023-08-25T09:48:07.810479+0000","last_unstale":"2023-08-25T09:48:08.848697+0000","last_undegraded":"2023-08-25T09:48:08.848697+0000","last_fullsized":"2023-08-25T09:48:08.848697+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:09.745781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439133+0000","last_change":"2023-08-25T09:48:08.438968+0000","last_active":"2023-08-25T09:48:08.439133+0000","last_peered":"2023-08-25T09:48:08.439133+0000","last_clean":"2023-08-25T09:48:08.439133+0000","last_became_active":"2023-08-25T09:48:08.438236+0000","last_became_peered":"2023-08-25T09:48:08.438236+0000","last_unstale":"2023-08-25T09:48:08.439133+0000","last_undegraded":"2023-08-25T09:48:08.439133+0000","last_fullsized":"2023-08-25T09:48:08.439133+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:32:48.669279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.433126+0000","last_change":"2023-08-25T09:48:07.808622+0000","last_active":"2023-08-25T09:48:08.433126+0000","last_peered":"2023-08-25T09:48:08.433126+0000","last_clean":"2023-08-25T09:48:08.433126+0000","last_became_active":"2023-08-25T09:48:07.807217+0000","last_became_peered":"2023-08-25T09:48:07.807217+0000","last_unstale":"2023-08-25T09:48:08.433126+0000","last_undegraded":"2023-08-25T09:48:08.433126+0000","last_fullsized":"2023-08-25T09:48:08.433126+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:42:39.683019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.430693+0000","last_change":"2023-08-25T09:48:08.430490+0000","last_active":"2023-08-25T09:48:08.430693+0000","last_peered":"2023-08-25T09:48:08.430693+0000","last_clean":"2023-08-25T09:48:08.430693+0000","last_became_active":"2023-08-25T09:48:08.429665+0000","last_became_peered":"2023-08-25T09:48:08.429665+0000","last_unstale":"2023-08-25T09:48:08.430693+0000","last_undegraded":"2023-08-25T09:48:08.430693+0000","last_fullsized":"2023-08-25T09:48:08.430693+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:22:29.317808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.b","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846900+0000","last_change":"2023-08-25T09:48:07.806915+0000","last_active":"2023-08-25T09:48:08.846900+0000","last_peered":"2023-08-25T09:48:08.846900+0000","last_clean":"2023-08-25T09:48:08.846900+0000","last_became_active":"2023-08-25T09:48:07.806658+0000","last_became_peered":"2023-08-25T09:48:07.806658+0000","last_unstale":"2023-08-25T09:48:08.846900+0000","last_undegraded":"2023-08-25T09:48:08.846900+0000","last_fullsized":"2023-08-25T09:48:08.846900+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:56:31.199417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444338+0000","last_change":"2023-08-25T09:48:08.444130+0000","last_active":"2023-08-25T09:48:08.444338+0000","last_peered":"2023-08-25T09:48:08.444338+0000","last_clean":"2023-08-25T09:48:08.444338+0000","last_became_active":"2023-08-25T09:48:08.443178+0000","last_became_peered":"2023-08-25T09:48:08.443178+0000","last_unstale":"2023-08-25T09:48:08.444338+0000","last_undegraded":"2023-08-25T09:48:08.444338+0000","last_fullsized":"2023-08-25T09:48:08.444338+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:02:50.345147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.425462+0000","last_change":"2023-08-25T09:48:07.807095+0000","last_active":"2023-08-25T09:48:08.425462+0000","last_peered":"2023-08-25T09:48:08.425462+0000","last_clean":"2023-08-25T09:48:08.425462+0000","last_became_active":"2023-08-25T09:48:07.806424+0000","last_became_peered":"2023-08-25T09:48:07.806424+0000","last_unstale":"2023-08-25T09:48:08.425462+0000","last_undegraded":"2023-08-25T09:48:08.425462+0000","last_fullsized":"2023-08-25T09:48:08.425462+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:18:31.379808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.438573+0000","last_change":"2023-08-25T09:48:08.438238+0000","last_active":"2023-08-25T09:48:08.438573+0000","last_peered":"2023-08-25T09:48:08.438573+0000","last_clean":"2023-08-25T09:48:08.438573+0000","last_became_active":"2023-08-25T09:48:08.437998+0000","last_became_peered":"2023-08-25T09:48:08.437998+0000","last_unstale":"2023-08-25T09:48:08.438573+0000","last_undegraded":"2023-08-25T09:48:08.438573+0000","last_fullsized":"2023-08-25T09:48:08.438573+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:30:09.804471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.429952+0000","last_change":"2023-08-25T09:48:07.804632+0000","last_active":"2023-08-25T09:48:08.429952+0000","last_peered":"2023-08-25T09:48:08.429952+0000","last_clean":"2023-08-25T09:48:08.429952+0000","last_became_active":"2023-08-25T09:48:07.804074+0000","last_became_peered":"2023-08-25T09:48:07.804074+0000","last_unstale":"2023-08-25T09:48:08.429952+0000","last_undegraded":"2023-08-25T09:48:08.429952+0000","last_fullsized":"2023-08-25T09:48:08.429952+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:20.161074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439748+0000","last_change":"2023-08-25T09:48:08.438574+0000","last_active":"2023-08-25T09:48:08.439748+0000","last_peered":"2023-08-25T09:48:08.439748+0000","last_clean":"2023-08-25T09:48:08.439748+0000","last_became_active":"2023-08-25T09:48:08.438317+0000","last_became_peered":"2023-08-25T09:48:08.438317+0000","last_unstale":"2023-08-25T09:48:08.439748+0000","last_undegraded":"2023-08-25T09:48:08.439748+0000","last_fullsized":"2023-08-25T09:48:08.439748+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:46:47.214540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.420872+0000","last_change":"2023-08-25T09:48:07.812944+0000","last_active":"2023-08-25T09:48:08.420872+0000","last_peered":"2023-08-25T09:48:08.420872+0000","last_clean":"2023-08-25T09:48:08.420872+0000","last_became_active":"2023-08-25T09:48:07.812556+0000","last_became_peered":"2023-08-25T09:48:07.812556+0000","last_unstale":"2023-08-25T09:48:08.420872+0000","last_undegraded":"2023-08-25T09:48:08.420872+0000","last_fullsized":"2023-08-25T09:48:08.420872+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:16:14.871551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.448076+0000","last_change":"2023-08-25T09:48:08.447996+0000","last_active":"2023-08-25T09:48:08.448076+0000","last_peered":"2023-08-25T09:48:08.448076+0000","last_clean":"2023-08-25T09:48:08.448076+0000","last_became_active":"2023-08-25T09:48:08.446309+0000","last_became_peered":"2023-08-25T09:48:08.446309+0000","last_unstale":"2023-08-25T09:48:08.448076+0000","last_undegraded":"2023-08-25T09:48:08.448076+0000","last_fullsized":"2023-08-25T09:48:08.448076+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:13:32.773833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.7","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.848707+0000","last_change":"2023-08-25T09:48:07.809138+0000","last_active":"2023-08-25T09:48:08.848707+0000","last_peered":"2023-08-25T09:48:08.848707+0000","last_clean":"2023-08-25T09:48:08.848707+0000","last_became_active":"2023-08-25T09:48:07.808036+0000","last_became_peered":"2023-08-25T09:48:07.808036+0000","last_unstale":"2023-08-25T09:48:08.848707+0000","last_undegraded":"2023-08-25T09:48:08.848707+0000","last_fullsized":"2023-08-25T09:48:08.848707+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:02:49.789356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443187+0000","last_change":"2023-08-25T09:48:08.443026+0000","last_active":"2023-08-25T09:48:08.443187+0000","last_peered":"2023-08-25T09:48:08.443187+0000","last_clean":"2023-08-25T09:48:08.443187+0000","last_became_active":"2023-08-25T09:48:08.442362+0000","last_became_peered":"2023-08-25T09:48:08.442362+0000","last_unstale":"2023-08-25T09:48:08.443187+0000","last_undegraded":"2023-08-25T09:48:08.443187+0000","last_fullsized":"2023-08-25T09:48:08.443187+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:58:45.559676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.432173+0000","last_change":"2023-08-25T09:48:07.805014+0000","last_active":"2023-08-25T09:48:08.432173+0000","last_peered":"2023-08-25T09:48:08.432173+0000","last_clean":"2023-08-25T09:48:08.432173+0000","last_became_active":"2023-08-25T09:48:07.804053+0000","last_became_peered":"2023-08-25T09:48:07.804053+0000","last_unstale":"2023-08-25T09:48:08.432173+0000","last_undegraded":"2023-08-25T09:48:08.432173+0000","last_fullsized":"2023-08-25T09:48:08.432173+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:06:10.131696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.439813+0000","last_change":"2023-08-25T09:48:08.439577+0000","last_active":"2023-08-25T09:48:08.439813+0000","last_peered":"2023-08-25T09:48:08.439813+0000","last_clean":"2023-08-25T09:48:08.439813+0000","last_became_active":"2023-08-25T09:48:08.439064+0000","last_became_peered":"2023-08-25T09:48:08.439064+0000","last_unstale":"2023-08-25T09:48:08.439813+0000","last_undegraded":"2023-08-25T09:48:08.439813+0000","last_fullsized":"2023-08-25T09:48:08.439813+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:26.980981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416447+0000","last_change":"2023-08-25T09:48:07.793569+0000","last_active":"2023-08-25T09:48:08.416447+0000","last_peered":"2023-08-25T09:48:08.416447+0000","last_clean":"2023-08-25T09:48:08.416447+0000","last_became_active":"2023-08-25T09:48:07.793062+0000","last_became_peered":"2023-08-25T09:48:07.793062+0000","last_unstale":"2023-08-25T09:48:08.416447+0000","last_undegraded":"2023-08-25T09:48:08.416447+0000","last_fullsized":"2023-08-25T09:48:08.416447+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:27.562821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.442396+0000","last_change":"2023-08-25T09:48:08.442303+0000","last_active":"2023-08-25T09:48:08.442396+0000","last_peered":"2023-08-25T09:48:08.442396+0000","last_clean":"2023-08-25T09:48:08.442396+0000","last_became_active":"2023-08-25T09:48:08.442007+0000","last_became_peered":"2023-08-25T09:48:08.442007+0000","last_unstale":"2023-08-25T09:48:08.442396+0000","last_undegraded":"2023-08-25T09:48:08.442396+0000","last_fullsized":"2023-08-25T09:48:08.442396+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:14:03.786808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.431062+0000","last_change":"2023-08-25T09:48:07.816291+0000","last_active":"2023-08-25T09:48:08.431062+0000","last_peered":"2023-08-25T09:48:08.431062+0000","last_clean":"2023-08-25T09:48:08.431062+0000","last_became_active":"2023-08-25T09:48:07.815563+0000","last_became_peered":"2023-08-25T09:48:07.815563+0000","last_unstale":"2023-08-25T09:48:08.431062+0000","last_undegraded":"2023-08-25T09:48:08.431062+0000","last_fullsized":"2023-08-25T09:48:08.431062+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:27.441758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.437986+0000","last_change":"2023-08-25T09:48:08.437231+0000","last_active":"2023-08-25T09:48:08.437986+0000","last_peered":"2023-08-25T09:48:08.437986+0000","last_clean":"2023-08-25T09:48:08.437986+0000","last_became_active":"2023-08-25T09:48:08.436924+0000","last_became_peered":"2023-08-25T09:48:08.436924+0000","last_unstale":"2023-08-25T09:48:08.437986+0000","last_undegraded":"2023-08-25T09:48:08.437986+0000","last_fullsized":"2023-08-25T09:48:08.437986+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:59:24.475584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.435451+0000","last_change":"2023-08-25T09:48:07.810821+0000","last_active":"2023-08-25T09:48:08.435451+0000","last_peered":"2023-08-25T09:48:08.435451+0000","last_clean":"2023-08-25T09:48:08.435451+0000","last_became_active":"2023-08-25T09:48:07.810473+0000","last_became_peered":"2023-08-25T09:48:07.810473+0000","last_unstale":"2023-08-25T09:48:08.435451+0000","last_undegraded":"2023-08-25T09:48:08.435451+0000","last_fullsized":"2023-08-25T09:48:08.435451+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:00:51.290998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.445153+0000","last_change":"2023-08-25T09:48:08.445040+0000","last_active":"2023-08-25T09:48:08.445153+0000","last_peered":"2023-08-25T09:48:08.445153+0000","last_clean":"2023-08-25T09:48:08.445153+0000","last_became_active":"2023-08-25T09:48:08.444543+0000","last_became_peered":"2023-08-25T09:48:08.444543+0000","last_unstale":"2023-08-25T09:48:08.445153+0000","last_undegraded":"2023-08-25T09:48:08.445153+0000","last_fullsized":"2023-08-25T09:48:08.445153+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:40.297657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436248+0000","last_change":"2023-08-25T09:48:07.804988+0000","last_active":"2023-08-25T09:48:08.436248+0000","last_peered":"2023-08-25T09:48:08.436248+0000","last_clean":"2023-08-25T09:48:08.436248+0000","last_became_active":"2023-08-25T09:48:07.804645+0000","last_became_peered":"2023-08-25T09:48:07.804645+0000","last_unstale":"2023-08-25T09:48:08.436248+0000","last_undegraded":"2023-08-25T09:48:08.436248+0000","last_fullsized":"2023-08-25T09:48:08.436248+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:05:25.662743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.434083+0000","last_change":"2023-08-25T09:48:08.433943+0000","last_active":"2023-08-25T09:48:08.434083+0000","last_peered":"2023-08-25T09:48:08.434083+0000","last_clean":"2023-08-25T09:48:08.434083+0000","last_became_active":"2023-08-25T09:48:08.433506+0000","last_became_peered":"2023-08-25T09:48:08.433506+0000","last_unstale":"2023-08-25T09:48:08.434083+0000","last_undegraded":"2023-08-25T09:48:08.434083+0000","last_fullsized":"2023-08-25T09:48:08.434083+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:57.923814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.424095+0000","last_change":"2023-08-25T09:48:07.814082+0000","last_active":"2023-08-25T09:48:08.424095+0000","last_peered":"2023-08-25T09:48:08.424095+0000","last_clean":"2023-08-25T09:48:08.424095+0000","last_became_active":"2023-08-25T09:48:07.813447+0000","last_became_peered":"2023-08-25T09:48:07.813447+0000","last_unstale":"2023-08-25T09:48:08.424095+0000","last_undegraded":"2023-08-25T09:48:08.424095+0000","last_fullsized":"2023-08-25T09:48:08.424095+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:21:18.043288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.446130+0000","last_change":"2023-08-25T09:48:08.446057+0000","last_active":"2023-08-25T09:48:08.446130+0000","last_peered":"2023-08-25T09:48:08.446130+0000","last_clean":"2023-08-25T09:48:08.446130+0000","last_became_active":"2023-08-25T09:48:08.445799+0000","last_became_peered":"2023-08-25T09:48:08.445799+0000","last_unstale":"2023-08-25T09:48:08.446130+0000","last_undegraded":"2023-08-25T09:48:08.446130+0000","last_fullsized":"2023-08-25T09:48:08.446130+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:03:20.353797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.417678+0000","last_change":"2023-08-25T09:48:07.802544+0000","last_active":"2023-08-25T09:48:08.417678+0000","last_peered":"2023-08-25T09:48:08.417678+0000","last_clean":"2023-08-25T09:48:08.417678+0000","last_became_active":"2023-08-25T09:48:07.801902+0000","last_became_peered":"2023-08-25T09:48:07.801902+0000","last_unstale":"2023-08-25T09:48:08.417678+0000","last_undegraded":"2023-08-25T09:48:08.417678+0000","last_fullsized":"2023-08-25T09:48:08.417678+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:31:42.811868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443342+0000","last_change":"2023-08-25T09:48:08.443256+0000","last_active":"2023-08-25T09:48:08.443342+0000","last_peered":"2023-08-25T09:48:08.443342+0000","last_clean":"2023-08-25T09:48:08.443342+0000","last_became_active":"2023-08-25T09:48:08.442684+0000","last_became_peered":"2023-08-25T09:48:08.442684+0000","last_unstale":"2023-08-25T09:48:08.443342+0000","last_undegraded":"2023-08-25T09:48:08.443342+0000","last_fullsized":"2023-08-25T09:48:08.443342+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:32:43.629812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.418804+0000","last_change":"2023-08-25T09:48:07.811130+0000","last_active":"2023-08-25T09:48:08.418804+0000","last_peered":"2023-08-25T09:48:08.418804+0000","last_clean":"2023-08-25T09:48:08.418804+0000","last_became_active":"2023-08-25T09:48:07.810908+0000","last_became_peered":"2023-08-25T09:48:07.810908+0000","last_unstale":"2023-08-25T09:48:08.418804+0000","last_undegraded":"2023-08-25T09:48:08.418804+0000","last_fullsized":"2023-08-25T09:48:08.418804+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:53:58.196255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":419,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.427619+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:48:08.427619+0000","last_peered":"2023-08-25T09:48:08.427619+0000","last_clean":"2023-08-25T09:48:08.427619+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:48:08.427619+0000","last_undegraded":"2023-08-25T09:48:08.427619+0000","last_fullsized":"2023-08-25T09:48:08.427619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:50:17.345621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.419019+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:48:08.419019+0000","last_peered":"2023-08-25T09:48:08.419019+0000","last_clean":"2023-08-25T09:48:08.419019+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:48:08.419019+0000","last_undegraded":"2023-08-25T09:48:08.419019+0000","last_fullsized":"2023-08-25T09:48:08.419019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:48:01.515335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.449210+0000","last_change":"2023-08-25T09:48:08.449113+0000","last_active":"2023-08-25T09:48:08.449210+0000","last_peered":"2023-08-25T09:48:08.449210+0000","last_clean":"2023-08-25T09:48:08.449210+0000","last_became_active":"2023-08-25T09:48:08.445991+0000","last_became_peered":"2023-08-25T09:48:08.445991+0000","last_unstale":"2023-08-25T09:48:08.449210+0000","last_undegraded":"2023-08-25T09:48:08.449210+0000","last_fullsized":"2023-08-25T09:48:08.449210+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:22:59.694751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.436120+0000","last_change":"2023-08-25T09:48:07.806255+0000","last_active":"2023-08-25T09:48:08.436120+0000","last_peered":"2023-08-25T09:48:08.436120+0000","last_clean":"2023-08-25T09:48:08.436120+0000","last_became_active":"2023-08-25T09:48:07.805900+0000","last_became_peered":"2023-08-25T09:48:07.805900+0000","last_unstale":"2023-08-25T09:48:08.436120+0000","last_undegraded":"2023-08-25T09:48:08.436120+0000","last_fullsized":"2023-08-25T09:48:08.436120+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:27:33.818964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.417147+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:48:08.417147+0000","last_peered":"2023-08-25T09:48:08.417147+0000","last_clean":"2023-08-25T09:48:08.417147+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:48:08.417147+0000","last_undegraded":"2023-08-25T09:48:08.417147+0000","last_fullsized":"2023-08-25T09:48:08.417147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:34:28.380493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.431969+0000","last_change":"2023-08-25T09:48:08.431044+0000","last_active":"2023-08-25T09:48:08.431969+0000","last_peered":"2023-08-25T09:48:08.431969+0000","last_clean":"2023-08-25T09:48:08.431969+0000","last_became_active":"2023-08-25T09:48:08.430122+0000","last_became_peered":"2023-08-25T09:48:08.430122+0000","last_unstale":"2023-08-25T09:48:08.431969+0000","last_undegraded":"2023-08-25T09:48:08.431969+0000","last_fullsized":"2023-08-25T09:48:08.431969+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:39:40.045358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"164'8","reported_seq":19,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.854021+0000","last_change":"2023-08-25T09:48:07.813874+0000","last_active":"2023-08-25T09:48:08.854021+0000","last_peered":"2023-08-25T09:48:08.854021+0000","last_clean":"2023-08-25T09:48:08.854021+0000","last_became_active":"2023-08-25T09:48:07.813007+0000","last_became_peered":"2023-08-25T09:48:07.813007+0000","last_unstale":"2023-08-25T09:48:08.854021+0000","last_undegraded":"2023-08-25T09:48:08.854021+0000","last_fullsized":"2023-08-25T09:48:08.854021+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:58:27.018451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":293,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.422221+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:48:08.422221+0000","last_peered":"2023-08-25T09:48:08.422221+0000","last_clean":"2023-08-25T09:48:08.422221+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:48:08.422221+0000","last_undegraded":"2023-08-25T09:48:08.422221+0000","last_fullsized":"2023-08-25T09:48:08.422221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:38:02.817543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.444918+0000","last_change":"2023-08-25T09:48:08.444817+0000","last_active":"2023-08-25T09:48:08.444918+0000","last_peered":"2023-08-25T09:48:08.444918+0000","last_clean":"2023-08-25T09:48:08.444918+0000","last_became_active":"2023-08-25T09:48:08.444177+0000","last_became_peered":"2023-08-25T09:48:08.444177+0000","last_unstale":"2023-08-25T09:48:08.444918+0000","last_undegraded":"2023-08-25T09:48:08.444918+0000","last_fullsized":"2023-08-25T09:48:08.444918+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:23.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.13","version":"164'2","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.850397+0000","last_change":"2023-08-25T09:48:07.816102+0000","last_active":"2023-08-25T09:48:08.850397+0000","last_peered":"2023-08-25T09:48:08.850397+0000","last_clean":"2023-08-25T09:48:08.850397+0000","last_became_active":"2023-08-25T09:48:07.814766+0000","last_became_peered":"2023-08-25T09:48:07.814766+0000","last_unstale":"2023-08-25T09:48:08.850397+0000","last_undegraded":"2023-08-25T09:48:08.850397+0000","last_fullsized":"2023-08-25T09:48:08.850397+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:58:24.774411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416645+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:48:08.416645+0000","last_peered":"2023-08-25T09:48:08.416645+0000","last_clean":"2023-08-25T09:48:08.416645+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:48:08.416645+0000","last_undegraded":"2023-08-25T09:48:08.416645+0000","last_fullsized":"2023-08-25T09:48:08.416645+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:44:49.052732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.451178+0000","last_change":"2023-08-25T09:48:08.451100+0000","last_active":"2023-08-25T09:48:08.451178+0000","last_peered":"2023-08-25T09:48:08.451178+0000","last_clean":"2023-08-25T09:48:08.451178+0000","last_became_active":"2023-08-25T09:48:08.450868+0000","last_became_peered":"2023-08-25T09:48:08.450868+0000","last_unstale":"2023-08-25T09:48:08.451178+0000","last_undegraded":"2023-08-25T09:48:08.451178+0000","last_fullsized":"2023-08-25T09:48:08.451178+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:21.857578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.15","version":"164'3","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.846648+0000","last_change":"2023-08-25T09:48:07.805184+0000","last_active":"2023-08-25T09:48:08.846648+0000","last_peered":"2023-08-25T09:48:08.846648+0000","last_clean":"2023-08-25T09:48:08.846648+0000","last_became_active":"2023-08-25T09:48:07.804397+0000","last_became_peered":"2023-08-25T09:48:07.804397+0000","last_unstale":"2023-08-25T09:48:08.846648+0000","last_undegraded":"2023-08-25T09:48:08.846648+0000","last_fullsized":"2023-08-25T09:48:08.846648+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:13.216905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":265,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.429811+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:48:08.429811+0000","last_peered":"2023-08-25T09:48:08.429811+0000","last_clean":"2023-08-25T09:48:08.429811+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:48:08.429811+0000","last_undegraded":"2023-08-25T09:48:08.429811+0000","last_fullsized":"2023-08-25T09:48:08.429811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:00:36.662651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.450132+0000","last_change":"2023-08-25T09:48:08.449960+0000","last_active":"2023-08-25T09:48:08.450132+0000","last_peered":"2023-08-25T09:48:08.450132+0000","last_clean":"2023-08-25T09:48:08.450132+0000","last_became_active":"2023-08-25T09:48:08.447890+0000","last_became_peered":"2023-08-25T09:48:08.447890+0000","last_unstale":"2023-08-25T09:48:08.450132+0000","last_undegraded":"2023-08-25T09:48:08.450132+0000","last_fullsized":"2023-08-25T09:48:08.450132+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:29:46.866121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"164'6","reported_seq":17,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.848005+0000","last_change":"2023-08-25T09:48:07.802045+0000","last_active":"2023-08-25T09:48:08.848005+0000","last_peered":"2023-08-25T09:48:08.848005+0000","last_clean":"2023-08-25T09:48:08.848005+0000","last_became_active":"2023-08-25T09:48:07.801824+0000","last_became_peered":"2023-08-25T09:48:07.801824+0000","last_unstale":"2023-08-25T09:48:08.848005+0000","last_undegraded":"2023-08-25T09:48:08.848005+0000","last_fullsized":"2023-08-25T09:48:08.848005+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:48.175474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416844+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:48:08.416844+0000","last_peered":"2023-08-25T09:48:08.416844+0000","last_clean":"2023-08-25T09:48:08.416844+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:48:08.416844+0000","last_undegraded":"2023-08-25T09:48:08.416844+0000","last_fullsized":"2023-08-25T09:48:08.416844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:33:51.834741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.443470+0000","last_change":"2023-08-25T09:48:08.443374+0000","last_active":"2023-08-25T09:48:08.443470+0000","last_peered":"2023-08-25T09:48:08.443470+0000","last_clean":"2023-08-25T09:48:08.443470+0000","last_became_active":"2023-08-25T09:48:08.442594+0000","last_became_peered":"2023-08-25T09:48:08.442594+0000","last_unstale":"2023-08-25T09:48:08.443470+0000","last_undegraded":"2023-08-25T09:48:08.443470+0000","last_fullsized":"2023-08-25T09:48:08.443470+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:15:08.002236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.428959+0000","last_change":"2023-08-25T09:48:07.799252+0000","last_active":"2023-08-25T09:48:08.428959+0000","last_peered":"2023-08-25T09:48:08.428959+0000","last_clean":"2023-08-25T09:48:08.428959+0000","last_became_active":"2023-08-25T09:48:07.798809+0000","last_became_peered":"2023-08-25T09:48:07.798809+0000","last_unstale":"2023-08-25T09:48:08.428959+0000","last_undegraded":"2023-08-25T09:48:08.428959+0000","last_fullsized":"2023-08-25T09:48:08.428959+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:46:04.683565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":263,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.433435+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:48:08.433435+0000","last_peered":"2023-08-25T09:48:08.433435+0000","last_clean":"2023-08-25T09:48:08.433435+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:48:08.433435+0000","last_undegraded":"2023-08-25T09:48:08.433435+0000","last_fullsized":"2023-08-25T09:48:08.433435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:51:22.280833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.435985+0000","last_change":"2023-08-25T09:48:08.435834+0000","last_active":"2023-08-25T09:48:08.435985+0000","last_peered":"2023-08-25T09:48:08.435985+0000","last_clean":"2023-08-25T09:48:08.435985+0000","last_became_active":"2023-08-25T09:48:08.435058+0000","last_became_peered":"2023-08-25T09:48:08.435058+0000","last_unstale":"2023-08-25T09:48:08.435985+0000","last_undegraded":"2023-08-25T09:48:08.435985+0000","last_fullsized":"2023-08-25T09:48:08.435985+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:07.777608+0000","last_clean_scrub_stamp":"2023-08-25T09:48:07.777608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:44:58.315658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.432686+0000","last_change":"2023-08-25T09:48:07.814733+0000","last_active":"2023-08-25T09:48:08.432686+0000","last_peered":"2023-08-25T09:48:08.432686+0000","last_clean":"2023-08-25T09:48:08.432686+0000","last_became_active":"2023-08-25T09:48:07.814463+0000","last_became_peered":"2023-08-25T09:48:07.814463+0000","last_unstale":"2023-08-25T09:48:08.432686+0000","last_undegraded":"2023-08-25T09:48:08.432686+0000","last_fullsized":"2023-08-25T09:48:08.432686+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:48:06.762225+0000","last_clean_scrub_stamp":"2023-08-25T09:48:06.762225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:05:00.706211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":291,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-08-25T09:48:08.416330+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:48:08.416330+0000","last_peered":"2023-08-25T09:48:08.416330+0000","last_clean":"2023-08-25T09:48:08.416330+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:48:08.416330+0000","last_undegraded":"2023-08-25T09:48:08.416330+0000","last_fullsized":"2023-08-25T09:48:08.416330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:47:33.677357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":948,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194364,"statfs":{"total":200000000,"available":199028987,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2569999999999999,"5min":1.095,"15min":1.0900000000000001},"min":{"1min":0.79500000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.7210000000000001,"5min":1.7909999999999999,"15min":10.747},"last":1.137},{"interface":"front","average":{"1min":1.216,"5min":1.159,"15min":1.0660000000000001},"min":{"1min":0.91300000000000003,"5min":0.61699999999999999,"15min":0.43099999999999999},"max":{"1min":1.651,"5min":1.855,"15min":1.855},"last":1.218}]},{"osd":1,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3240000000000001,"15min":1.1919999999999999},"min":{"1min":0.85999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.72,"5min":11.010999999999999,"15min":11.010999999999999},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.2110000000000001,"15min":1.1339999999999999},"min":{"1min":1.095,"5min":0.66500000000000004,"15min":0.57999999999999996},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7769999999999999},"last":1.024}]},{"osd":2,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1479999999999999,"15min":1.1020000000000001},"min":{"1min":0.79400000000000004,"5min":0.66800000000000004,"15min":0.50600000000000001},"max":{"1min":1.4590000000000001,"5min":1.619,"15min":1.619},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.327,"5min":1.302,"15min":1.2430000000000001},"min":{"1min":0.79500000000000004,"5min":0.67700000000000005,"15min":0.66800000000000004},"max":{"1min":1.704,"5min":1.98,"15min":1.98},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.236,"15min":1.181},"min":{"1min":0.90500000000000003,"5min":0.59799999999999998,"15min":0.56799999999999995},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.421,"5min":1.3700000000000001,"15min":1.3440000000000001},"min":{"1min":1.077,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.6679999999999999,"5min":2.1030000000000002,"15min":2.1030000000000002},"last":1.2849999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.288,"15min":1.2829999999999999},"min":{"1min":0.80700000000000005,"5min":0.51300000000000001,"15min":0.48599999999999999},"max":{"1min":1.7649999999999999,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3740000000000001,"15min":1.3120000000000001},"min":{"1min":0.93700000000000006,"5min":0.53800000000000003,"15min":0.46300000000000002},"max":{"1min":1.798,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.7}]},{"osd":5,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.456,"15min":1.381},"min":{"1min":1.3160000000000001,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":1.849,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.532},{"interface":"front","average":{"1min":1.488,"5min":1.462,"15min":1.4059999999999999},"min":{"1min":1.113,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.895,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.357}]},{"osd":6,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5489999999999999,"15min":1.474},"min":{"1min":0.995,"5min":0.72999999999999998,"15min":0.68799999999999994},"max":{"1min":1.98,"5min":2.0259999999999998,"15min":2.1720000000000002},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.5,"15min":1.4319999999999999},"min":{"1min":1.292,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":2.3220000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.595}]}]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194357,"statfs":{"total":200000000,"available":199021825,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.1539999999999999,"15min":1.1419999999999999},"min":{"1min":0.68700000000000006,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5569999999999999,"5min":1.804,"15min":1.804},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.155,"5min":1.169,"15min":1.1519999999999999},"min":{"1min":0.79100000000000004,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.488,"5min":1.992,"15min":5.5410000000000004},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2,"15min":1.2},"min":{"1min":0.83299999999999996,"5min":0.72699999999999998,"15min":0.54700000000000004},"max":{"1min":1.6259999999999999,"5min":2.198,"15min":5.6799999999999997},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.214,"15min":1.1459999999999999},"min":{"1min":0.66800000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.909,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":1.091}]},{"osd":2,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.165,"15min":1.123},"min":{"1min":0.82199999999999995,"5min":0.63400000000000001,"15min":0.60399999999999998},"max":{"1min":1.631,"5min":1.631,"15min":1.708},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3029999999999999,"15min":1.2450000000000001},"min":{"1min":0.67000000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":13.193},"last":1.1859999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3080000000000001,"15min":1.2569999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.661,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.2749999999999999,"15min":1.26},"min":{"1min":0.88300000000000001,"5min":0.70699999999999996,"15min":0.69199999999999995},"max":{"1min":1.8660000000000001,"5min":1.8660000000000001,"15min":1.9239999999999999},"last":1.5309999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.363,"15min":1.2869999999999999},"min":{"1min":0.55300000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.8009999999999999,"5min":2.1110000000000002,"15min":2.484},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.385,"15min":1.349},"min":{"1min":0.70099999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7509999999999999,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.3480000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.343,"15min":1.296},"min":{"1min":0.74199999999999999,"5min":0.65000000000000002,"15min":0.59599999999999997},"max":{"1min":1.74,"5min":1.859,"15min":2.0049999999999999},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5089999999999999,"15min":1.413},"min":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.994,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.421}]},{"osd":7,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.54,"15min":1.512},"min":{"1min":0.80000000000000004,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":1.944,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.476},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.5069999999999999,"15min":1.484},"min":{"1min":0.85899999999999999,"5min":0.74299999999999999,"15min":0.57399999999999995},"max":{"1min":1.7949999999999999,"5min":1.9710000000000001,"15min":2.1890000000000001},"last":1.655}]}]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1531,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193780,"statfs":{"total":200000000,"available":198431491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.179,"15min":1.161},"min":{"1min":0.71999999999999997,"5min":0.624,"15min":0.497},"max":{"1min":1.8109999999999999,"5min":1.8109999999999999,"15min":2.081},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.218,"15min":1.2090000000000001},"min":{"1min":0.82199999999999995,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.974},"last":1.139}]},{"osd":1,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1659999999999999,"15min":1.1819999999999999},"min":{"1min":0.67800000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.177,"15min":1.1399999999999999},"min":{"1min":0.76600000000000001,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.704,"5min":1.8640000000000001,"15min":2.1869999999999998},"last":0.97499999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.2609999999999999,"15min":1.208},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.528},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.286,"15min":1.234},"min":{"1min":0.86399999999999999,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.623,"5min":1.778,"15min":1.962},"last":1.268}]},{"osd":3,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.22,"15min":1.292},"min":{"1min":0.72199999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.8109999999999999,"5min":2.3439999999999999,"15min":17.356999999999999},"last":1.296},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3580000000000001,"15min":1.304},"min":{"1min":0.78700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.964,"5min":1.964,"15min":1.9650000000000001},"last":0.94499999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.2949999999999999,"15min":1.3049999999999999},"min":{"1min":0.80800000000000005,"5min":0.58299999999999996,"15min":0.55500000000000005},"max":{"1min":1.9059999999999999,"5min":1.9530000000000001,"15min":2.625},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.3460000000000001,"15min":1.3560000000000001},"min":{"1min":0.83699999999999997,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":1.913,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.476}]},{"osd":6,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":1.4670000000000001,"15min":1.4379999999999999},"min":{"1min":0.95099999999999996,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":2.028,"5min":2.048,"15min":2.4820000000000002},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.492,"15min":1.446},"min":{"1min":0.96899999999999997,"5min":0.65500000000000003,"15min":0.60399999999999998},"max":{"1min":2.044,"5min":2.1059999999999999,"15min":2.5539999999999998},"last":1.605}]},{"osd":7,"last update":"Fri Aug 25 09:47:55 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.4830000000000001,"15min":1.452},"min":{"1min":1.0700000000000001,"5min":0.80800000000000005,"15min":0.63},"max":{"1min":2.048,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.6339999999999999,"5min":1.605,"15min":1.5800000000000001},"min":{"1min":1.1639999999999999,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":2.1040000000000001,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.4239999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574964,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194357,"statfs":{"total":200000000,"available":199021789,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.246,"15min":1.1819999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":1.8160000000000001},"last":4.4260000000000002},{"interface":"front","average":{"1min":1.321,"5min":1.3029999999999999,"15min":1.204},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.9750000000000001,"5min":1.976,"15min":7.367},"last":2.5150000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.149,"15min":1.1299999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.7330000000000001,"5min":2.032,"15min":2.032},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.2150000000000001,"15min":1.333},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7110000000000001,"5min":1.954,"15min":8.5850000000000009},"last":4.351}]},{"osd":2,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2030000000000001,"15min":1.2569999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.613,"5min":1.708,"15min":4.1669999999999998},"last":1.224},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.3089999999999999,"15min":1.2110000000000001},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.835,"5min":1.835,"15min":1.8520000000000001},"last":4.7320000000000002}]},{"osd":3,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.248,"15min":1.244},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":7.2560000000000002},"last":2.5680000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.4059999999999999,"15min":1.3740000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8100000000000001,"5min":1.899,"15min":1.911},"last":1.151}]},{"osd":5,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3009999999999999,"15min":1.2949999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.6879999999999999,"5min":2.1850000000000001,"15min":2.4420000000000002},"last":5.048},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.325,"15min":1.282},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.931,"5min":1.931,"15min":8.2409999999999997},"last":2.6469999999999998}]},{"osd":6,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.4470000000000001,"15min":1.4359999999999999},"min":{"1min":0.72899999999999998,"5min":0.68400000000000005,"15min":0.52000000000000002},"max":{"1min":1.722,"5min":3.5880000000000001,"15min":3.5880000000000001},"last":6.2220000000000004},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.585,"15min":1.5680000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":2.1720000000000002,"5min":3.5270000000000001,"15min":4.0229999999999997},"last":7.0990000000000002}]},{"osd":7,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5049999999999999,"15min":1.454},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.8859999999999999,"5min":2.0649999999999999,"15min":8.1760000000000002},"last":3.5409999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.621,"15min":1.597},"min":{"1min":0.85699999999999998,"5min":0.80900000000000005,"15min":0.77500000000000002},"max":{"1min":1.909,"5min":2.1819999999999999,"15min":2.7930000000000001},"last":3.831}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194356,"statfs":{"total":200000000,"available":199021277,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":0.93799999999999994,"15min":0.871},"min":{"1min":0.46600000000000003,"5min":0.41099999999999998,"15min":0.29299999999999998},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.0029999999999999,"15min":0.92700000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.595,"5min":1.631,"15min":1.631},"last":0.94199999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.85599999999999998,"15min":0.81999999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.35699999999999998},"max":{"1min":1.163,"5min":1.702,"15min":1.702},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1,"15min":0.98199999999999998},"min":{"1min":0.58499999999999996,"5min":0.52800000000000002,"15min":0.45700000000000002},"max":{"1min":1.6339999999999999,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":0.99099999999999999,"15min":0.90800000000000003},"min":{"1min":0.51500000000000001,"5min":0.44600000000000001,"15min":0.378},"max":{"1min":1.5469999999999999,"5min":1.7450000000000001,"15min":1.919},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.165,"5min":1.147,"15min":1.1160000000000001},"min":{"1min":0.63500000000000001,"5min":0.47399999999999998,"15min":0.44900000000000001},"max":{"1min":1.8080000000000001,"5min":2.0870000000000002,"15min":3.399},"last":0.876}]},{"osd":4,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.2150000000000001,"15min":1.1970000000000001},"min":{"1min":0.93300000000000005,"5min":0.73299999999999998,"15min":0.66800000000000004},"max":{"1min":1.655,"5min":2.0289999999999999,"15min":2.1949999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.65700000000000003},"max":{"1min":1.5529999999999999,"5min":1.984,"15min":2.1840000000000002},"last":1.056}]},{"osd":5,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.345,"15min":1.3220000000000001},"min":{"1min":0.98899999999999999,"5min":0.73599999999999999,"15min":0.71899999999999997},"max":{"1min":2.008,"5min":2.407,"15min":2.407},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.2769999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.64500000000000002},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.2250000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.2889999999999999,"15min":1.256},"min":{"1min":0.86699999999999999,"5min":0.85199999999999998,"15min":0.73799999999999999},"max":{"1min":1.5549999999999999,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.401,"15min":1.3999999999999999},"min":{"1min":0.84899999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":2.1120000000000001,"5min":2.6699999999999999,"15min":3.496},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.3879999999999999,"15min":1.369},"min":{"1min":0.871,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":1.7330000000000001,"5min":1.9910000000000001,"15min":2.2989999999999999},"last":1.347},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.389,"15min":1.329},"min":{"1min":0.84699999999999998,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":1.843,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.367}]}]},{"osd":2,"up_from":13,"seq":55834574964,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":955,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194357,"statfs":{"total":200000000,"available":199021859,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.41099999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.2050000000000001,"5min":1.9279999999999999,"15min":2.383},"last":0.627},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.96099999999999997,"15min":1.0169999999999999},"min":{"1min":0.59699999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.625,"5min":1.6839999999999999,"15min":2.7309999999999999},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.996,"15min":0.98699999999999999},"min":{"1min":0.625,"5min":0.442,"15min":0.442},"max":{"1min":1.5549999999999999,"5min":1.978,"15min":2.3180000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.012,"5min":1.0229999999999999,"15min":1.044},"min":{"1min":0.59099999999999997,"5min":0.439,"15min":0.439},"max":{"1min":1.5369999999999999,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":1.131,"15min":1.101},"min":{"1min":0.46800000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.3460000000000001,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.091,"15min":1.0940000000000001},"min":{"1min":0.69299999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.4850000000000001,"5min":2.145,"15min":2.2040000000000002},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.29,"15min":1.351},"min":{"1min":0.85199999999999998,"5min":0.74199999999999999,"15min":0.72099999999999997},"max":{"1min":1.5409999999999999,"5min":2.2919999999999998,"15min":3.0150000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.274,"15min":1.3120000000000001},"min":{"1min":0.92100000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.631,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.2450000000000001,"15min":1.3260000000000001},"min":{"1min":0.83599999999999997,"5min":0.73199999999999998,"15min":0.70799999999999996},"max":{"1min":1.5149999999999999,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.3400000000000001,"15min":1.375},"min":{"1min":0.99399999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.631,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3999999999999999,"15min":1.4099999999999999},"min":{"1min":1.02,"5min":0.755,"15min":0.73199999999999998},"max":{"1min":1.714,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.409,"15min":1.4510000000000001},"min":{"1min":1.024,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.855,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":1.109}]},{"osd":7,"last update":"Fri Aug 25 09:47:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.369,"15min":1.4199999999999999},"min":{"1min":0.87,"5min":0.74199999999999999,"15min":0.65700000000000003},"max":{"1min":1.7110000000000001,"5min":2.331,"15min":2.968},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.371,"5min":1.413,"15min":1.4690000000000001},"min":{"1min":1.0409999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7470000000000001,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.077}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1531,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193780,"statfs":{"total":200000000,"available":198431491,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84499999999999997,"15min":0.79600000000000004},"min":{"1min":0.46000000000000002,"5min":0.36399999999999999,"15min":0.29699999999999999},"max":{"1min":1.26,"5min":1.5600000000000001,"15min":1.627},"last":0.54000000000000004},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.94099999999999995,"15min":0.83499999999999996},"min":{"1min":0.47499999999999998,"5min":0.317,"15min":0.317},"max":{"1min":1.5860000000000001,"5min":1.776,"15min":1.895},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.90900000000000003,"15min":0.83199999999999996},"min":{"1min":0.57299999999999995,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3320000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.98199999999999998,"15min":0.92900000000000005},"min":{"1min":0.53500000000000003,"5min":0.46000000000000002,"15min":0.33600000000000002},"max":{"1min":1.508,"5min":1.6220000000000001,"15min":1.746},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.1200000000000001,"15min":1.0649999999999999},"min":{"1min":0.55000000000000004,"5min":0.50800000000000001,"15min":0.372},"max":{"1min":1.518,"5min":1.8680000000000001,"15min":1.9450000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.0549999999999999,"5min":1.0489999999999999,"15min":0.97599999999999998},"min":{"1min":0.62,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.655,"5min":1.655,"15min":1.764},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.167,"15min":1.113},"min":{"1min":0.82999999999999996,"5min":0.627,"15min":0.627},"max":{"1min":1.476,"5min":1.768,"15min":1.768},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.292,"15min":1.2450000000000001},"min":{"1min":0.85499999999999998,"5min":0.77700000000000002,"15min":0.629},"max":{"1min":1.5229999999999999,"5min":1.722,"15min":2.008},"last":1.0880000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.2529999999999999,"15min":1.212},"min":{"1min":0.80400000000000005,"5min":0.70499999999999996,"15min":0.68600000000000005},"max":{"1min":1.6479999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":1.248,"5min":1.264,"15min":1.198},"min":{"1min":0.79200000000000004,"5min":0.78300000000000003,"15min":0.61199999999999999},"max":{"1min":1.6180000000000001,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.02}]},{"osd":6,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4019999999999999,"15min":1.3340000000000001},"min":{"1min":0.83299999999999996,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8879999999999999,"5min":2.1339999999999999,"15min":2.9940000000000002},"last":1.28},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3979999999999999,"15min":1.325},"min":{"1min":0.876,"5min":0.82199999999999995,"15min":0.72099999999999997},"max":{"1min":1.8129999999999999,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.22}]},{"osd":7,"last update":"Fri Aug 25 09:47:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4219999999999999,"15min":1.377},"min":{"1min":0.93300000000000005,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.6930000000000001,"5min":1.968,"15min":2.9529999999999998},"last":1.381},{"interface":"front","average":{"1min":1.484,"5min":1.4510000000000001,"15min":1.359},"min":{"1min":0.86099999999999999,"5min":0.81899999999999995,"15min":0.81899999999999995},"max":{"1min":1.847,"5min":2.218,"15min":2.218},"last":1.2430000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":956,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194355,"statfs":{"total":200000000,"available":199020157,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.875,"15min":0.87},"min":{"1min":0.433,"5min":0.33900000000000002,"15min":0.313},"max":{"1min":1.2989999999999999,"5min":1.9350000000000001,"15min":11.257},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":0.95399999999999996},"min":{"1min":0.52700000000000002,"5min":0.34799999999999998,"15min":0.32800000000000001},"max":{"1min":1.5309999999999999,"5min":1.829,"15min":6.4260000000000002},"last":1.0920000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.96699999999999997,"15min":0.94599999999999995},"min":{"1min":0.44,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":1.3899999999999999,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.02},{"interface":"front","average":{"1min":1.01,"5min":1.02,"15min":1.0600000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.421,"5min":1.9950000000000001,"15min":3.4889999999999999},"last":0.97599999999999998}]},{"osd":3,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.91800000000000004,"15min":0.90100000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.2270000000000001,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.629,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":1.1459999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2729999999999999,"15min":1.2390000000000001},"min":{"1min":0.62,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.6930000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3720000000000001,"15min":1.357},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7549999999999999,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":1.4179999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2829999999999999,"15min":1.3260000000000001},"min":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.68000000000000005},"max":{"1min":1.488,"5min":2.4359999999999999,"15min":3.3769999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.2669999999999999,"15min":1.284},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.8500000000000001,"5min":2.2999999999999998,"15min":2.3220000000000001},"last":1.2050000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3600000000000001,"15min":1.343},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7030000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.244},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.3480000000000001,"15min":1.3089999999999999},"min":{"1min":0.80500000000000005,"5min":0.752,"15min":0.72699999999999998},"max":{"1min":1.7889999999999999,"5min":2.331,"15min":2.331},"last":1.268}]},{"osd":7,"last update":"Fri Aug 25 09:48:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4139999999999999,"15min":1.3959999999999999},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.6830000000000001,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.349,"15min":1.3240000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.7949999999999999,"5min":2.7410000000000001,"15min":2.7410000000000001},"last":1.3260000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:48:13.778 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:48:13.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:48:14.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.037+0000 7fa3cf59e640 1 Processor -- start 2023-08-25T09:48:14.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.038+0000 7fa3cf59e640 1 -- start start 2023-08-25T09:48:14.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3cf59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa3d015cc00 0x7fa3d00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3cf59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d00cfe20 0x7fa3d00b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3cf59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d00d09d0 con 0x7fa3d00cd6a0 2023-08-25T09:48:14.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3cf59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d00d1270 con 0x7fa3d015cc00 2023-08-25T09:48:14.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3cf59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d015cfe0 con 0x7fa3d00cfe20 2023-08-25T09:48:14.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3ce59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa3d015cc00 0x7fa3d00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3ce59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa3d015cc00 0x7fa3d00cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44656/0 (socket says 172.21.15.70:44656) 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3ce59c640 1 -- 172.21.15.70:0/1808776406 learned_addr learned my addr 172.21.15.70:0/1808776406 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.039+0000 7fa3c7fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa3d00cd6a0 0x7fa3d00cf710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57318/0 (socket says 172.21.15.70:57318) 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3ced9d640 1 --2- 172.21.15.70:0/1808776406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d00cfe20 0x7fa3d00b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3ce59c640 1 -- 172.21.15.70:0/1808776406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d00cfe20 msgr2=0x7fa3d00b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3ce59c640 1 --2- 172.21.15.70:0/1808776406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d00cfe20 0x7fa3d00b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3ce59c640 1 -- 172.21.15.70:0/1808776406 >> v1:172.21.15.70:6789/0 conn(0x7fa3d00cd6a0 legacy=0x7fa3d00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3cdd9b640 1 -- 172.21.15.70:0/1808776406 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70278391 0 0) 0x7fa3d00d09d0 con 0x7fa3d00cd6a0 2023-08-25T09:48:14.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.040+0000 7fa3ce59c640 1 -- 172.21.15.70:0/1808776406 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00cdc20 con 0x7fa3d015cc00 2023-08-25T09:48:14.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.041+0000 7fa3ce59c640 1 --2- 172.21.15.70:0/1808776406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d015cc00 0x7fa3d00cd160 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009870 tx=0x7fa3c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3827b61c8dabf6a3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:14.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.041+0000 7fa3cdd9b640 1 -- 172.21.15.70:0/1808776406 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0030840 con 0x7fa3d015cc00 2023-08-25T09:48:14.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.041+0000 7fa3cdd9b640 1 -- 172.21.15.70:0/1808776406 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c00309a0 con 0x7fa3d015cc00 2023-08-25T09:48:14.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.041+0000 7fa3cdd9b640 1 -- 172.21.15.70:0/1808776406 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0030d30 con 0x7fa3d015cc00 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1808776406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d015cc00 msgr2=0x7fa3d00cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1808776406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d015cc00 0x7fa3d00cd160 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009870 tx=0x7fa3c002b980 comp rx=0 tx=0).stop 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1808776406 shutdown_connections 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1808776406 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d00cfe20 0x7fa3d00b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1808776406 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d015cc00 0x7fa3d00cd160 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1808776406 >> 172.21.15.70:0/1808776406 conn(0x7fa3d0070050 msgr2=0x7fa3d0070430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1808776406 shutdown_connections 2023-08-25T09:48:14.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.042+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1808776406 wait complete. 2023-08-25T09:48:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.044+0000 7fa3cf59e640 1 Processor -- start 2023-08-25T09:48:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.044+0000 7fa3cf59e640 1 -- start start 2023-08-25T09:48:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3cf59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d00cd6a0 0x7fa3d01f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3cf59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 0x7fa3d0153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3cf59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d015cc00 0x7fa3d0155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3cf59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d01f38e0 con 0x7fa3d00cfe20 2023-08-25T09:48:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3cf59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d01f30d0 con 0x7fa3d00cd6a0 2023-08-25T09:48:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3cf59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d00c64d0 con 0x7fa3d015cc00 2023-08-25T09:48:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 0x7fa3d0153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3c7fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 0x7fa3d0153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44668/0 (socket says 172.21.15.70:44668) 2023-08-25T09:48:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.045+0000 7fa3c7fff640 1 -- 172.21.15.70:0/1836952596 learned_addr learned my addr 172.21.15.70:0/1836952596 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:14.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c7fff640 1 -- 172.21.15.70:0/1836952596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d015cc00 msgr2=0x7fa3d0155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c7fff640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d015cc00 0x7fa3d0155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c7fff640 1 -- 172.21.15.70:0/1836952596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d00cd6a0 msgr2=0x7fa3d01f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c7fff640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d00cd6a0 0x7fa3d01f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c7fff640 1 -- 172.21.15.70:0/1836952596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c00094d0 con 0x7fa3d00cfe20 2023-08-25T09:48:14.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c7fff640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 0x7fa3d0153240 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c4d0 tx=0x7fa3b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:14.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80074e0 con 0x7fa3d00cfe20 2023-08-25T09:48:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b8007660 con 0x7fa3d00cfe20 2023-08-25T09:48:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8004a20 con 0x7fa3d00cfe20 2023-08-25T09:48:14.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.046+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d00c67b0 con 0x7fa3d00cfe20 2023-08-25T09:48:14.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.047+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d00c6d70 con 0x7fa3d00cfe20 2023-08-25T09:48:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.048+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d00d3a50 con 0x7fa3d00cfe20 2023-08-25T09:48:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.049+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3b8004b80 con 0x7fa3d00cfe20 2023-08-25T09:48:14.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.050+0000 7fa3c6ffd640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b006a050 0x7fa3b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.050+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3b8087690 con 0x7fa3d00cfe20 2023-08-25T09:48:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.050+0000 7fa3ce59c640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b006a050 0x7fa3b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.051+0000 7fa3ce59c640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b006a050 0x7fa3b006c510 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0036000 tx=0x7fa3c003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:14.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.054+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3b8052e10 con 0x7fa3d00cfe20 2023-08-25T09:48:14.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.310+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fa3d0061460 con 0x7fa3d00cfe20 2023-08-25T09:48:14.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.318+0000 7fa3c6ffd640 1 -- 172.21.15.70:0/1836952596 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7fa3b8056430 con 0x7fa3d00cfe20 2023-08-25T09:48:14.320 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:48:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:48:14.320 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:48:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:48:14.321 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:48:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:48:14.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b006a050 msgr2=0x7fa3b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b006a050 0x7fa3b006c510 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0036000 tx=0x7fa3c003b040 comp rx=0 tx=0).stop 2023-08-25T09:48:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 msgr2=0x7fa3d0153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 0x7fa3d0153240 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800c4d0 tx=0x7fa3b800c990 comp rx=0 tx=0).stop 2023-08-25T09:48:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 shutdown_connections 2023-08-25T09:48:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3b006a050 0x7fa3b006c510 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3d015cc00 0x7fa3d0155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3d00cfe20 0x7fa3d0153240 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 --2- 172.21.15.70:0/1836952596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3d00cd6a0 0x7fa3d01f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 >> 172.21.15.70:0/1836952596 conn(0x7fa3d0070050 msgr2=0x7fa3d00b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 shutdown_connections 2023-08-25T09:48:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.320+0000 7fa3cf59e640 1 -- 172.21.15.70:0/1836952596 wait complete. 2023-08-25T09:48:14.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:48:14.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.551+0000 7fab9bc34640 1 Processor -- start 2023-08-25T09:48:14.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.551+0000 7fab9bc34640 1 -- start start 2023-08-25T09:48:14.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab9bc34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab9415ec10 0x7fab9415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab9bc34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415c6d0 0x7fab940b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab9bc34640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab940b1b60 con 0x7fab9415e830 2023-08-25T09:48:14.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab9bc34640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9406bd80 con 0x7fab9415ec10 2023-08-25T09:48:14.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab9bc34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9406b7c0 con 0x7fab9415c6d0 2023-08-25T09:48:14.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab991cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab9415ec10 0x7fab9415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab991cf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fab9415ec10 0x7fab9415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44684/0 (socket says 172.21.15.70:44684) 2023-08-25T09:48:14.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.553+0000 7fab991cf640 1 -- 172.21.15.70:0/3671762156 learned_addr learned my addr 172.21.15.70:0/3671762156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.554+0000 7fab991cf640 1 -- 172.21.15.70:0/3671762156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415c6d0 msgr2=0x7fab940b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.554+0000 7fab991cf640 1 --2- 172.21.15.70:0/3671762156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415c6d0 0x7fab940b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.554+0000 7fab991cf640 1 -- 172.21.15.70:0/3671762156 >> v1:172.21.15.155:6789/0 conn(0x7fab9415e830 legacy=0x7fab9406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.554+0000 7fab991cf640 1 -- 172.21.15.70:0/3671762156 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9406baa0 con 0x7fab9415ec10 2023-08-25T09:48:14.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.555+0000 7fab991cf640 1 --2- 172.21.15.70:0/3671762156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab9415c190 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fab84009870 tx=0x7fab8402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31395aff9425c225 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:14.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.555+0000 7fab989ce640 1 -- 172.21.15.70:0/3671762156 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84031640 con 0x7fab9415ec10 2023-08-25T09:48:14.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.555+0000 7fab989ce640 1 -- 172.21.15.70:0/3671762156 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab840317a0 con 0x7fab9415ec10 2023-08-25T09:48:14.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.555+0000 7fab989ce640 1 -- 172.21.15.70:0/3671762156 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84031b30 con 0x7fab9415ec10 2023-08-25T09:48:14.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 -- 172.21.15.70:0/3671762156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 msgr2=0x7fab9415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 --2- 172.21.15.70:0/3671762156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab9415c190 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fab84009870 tx=0x7fab8402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 -- 172.21.15.70:0/3671762156 shutdown_connections 2023-08-25T09:48:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 --2- 172.21.15.70:0/3671762156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415c6d0 0x7fab940b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 --2- 172.21.15.70:0/3671762156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab9415c190 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 -- 172.21.15.70:0/3671762156 >> 172.21.15.70:0/3671762156 conn(0x7fab94069850 msgr2=0x7fab94069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 -- 172.21.15.70:0/3671762156 shutdown_connections 2023-08-25T09:48:14.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.556+0000 7fab9bc34640 1 -- 172.21.15.70:0/3671762156 wait complete. 2023-08-25T09:48:14.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.558+0000 7fab9bc34640 1 Processor -- start 2023-08-25T09:48:14.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.558+0000 7fab9bc34640 1 -- start start 2023-08-25T09:48:14.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9bc34640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab9415c6d0 0x7fab94148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9bc34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415e830 0x7fab940c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9bc34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab941f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9bc34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab941492b0 con 0x7fab9415ec10 2023-08-25T09:48:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9bc34640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab941e4760 con 0x7fab9415c6d0 2023-08-25T09:48:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9bc34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab941e4a40 con 0x7fab9415e830 2023-08-25T09:48:14.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab941f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab941f21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44688/0 (socket says 172.21.15.70:44688) 2023-08-25T09:48:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 -- 172.21.15.70:0/1232290591 learned_addr learned my addr 172.21.15.70:0/1232290591 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab991cf640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415e830 0x7fab940c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab999d0640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab9415c6d0 0x7fab94148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 -- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415e830 msgr2=0x7fab940c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415e830 0x7fab940c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 -- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab9415c6d0 msgr2=0x7fab94148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab9415c6d0 0x7fab94148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab9a1d1640 1 -- 172.21.15.70:0/1232290591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab840094d0 con 0x7fab9415ec10 2023-08-25T09:48:14.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab991cf640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415e830 0x7fab940c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:14.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.559+0000 7fab999d0640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab9415c6d0 0x7fab94148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:14.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.560+0000 7fab9a1d1640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab941f21a0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00b970 tx=0x7fab8c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:14.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.560+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00cb40 con 0x7fab9415ec10 2023-08-25T09:48:14.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.560+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00ccc0 con 0x7fab9415ec10 2023-08-25T09:48:14.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.560+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab941e4d20 con 0x7fab9415ec10 2023-08-25T09:48:14.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.560+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c014680 con 0x7fab9415ec10 2023-08-25T09:48:14.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.560+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab940ca350 con 0x7fab9415ec10 2023-08-25T09:48:14.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.561+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9404e420 con 0x7fab9415ec10 2023-08-25T09:48:14.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.562+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fab8c01d3b0 con 0x7fab9415ec10 2023-08-25T09:48:14.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.563+0000 7fab8a7fc640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab7806a050 0x7fab7806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.563+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab8c088520 con 0x7fab9415ec10 2023-08-25T09:48:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.563+0000 7fab999d0640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab7806a050 0x7fab7806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.564+0000 7fab999d0640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab7806a050 0x7fab7806c510 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fab90007fd0 tx=0x7fab90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.567+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fab8c053c10 con 0x7fab9415ec10 2023-08-25T09:48:14.761 INFO:tasks.ceph.mds.c.smithi070.stderr:2023-08-25T09:48:14.759+0000 7fecc4bee640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:48:14.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.796+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab9406c620 con 0x7fab9415ec10 2023-08-25T09:48:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.797+0000 7fab8a7fc640 1 -- 172.21.15.70:0/1232290591 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fab8c057230 con 0x7fab9415ec10 2023-08-25T09:48:14.799 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:14.799 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:48:14.800 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:48:14.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab7806a050 msgr2=0x7fab7806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab7806a050 0x7fab7806c510 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fab90007fd0 tx=0x7fab90009040 comp rx=0 tx=0).stop 2023-08-25T09:48:14.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 msgr2=0x7fab941f21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:14.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab941f21a0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00b970 tx=0x7fab8c00be30 comp rx=0 tx=0).stop 2023-08-25T09:48:14.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 shutdown_connections 2023-08-25T09:48:14.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fab7806a050 0x7fab7806c510 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fab9415ec10 0x7fab941f21a0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fab9415e830 0x7fab940c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 --2- 172.21.15.70:0/1232290591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fab9415c6d0 0x7fab94148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:14.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 >> 172.21.15.70:0/1232290591 conn(0x7fab94069850 msgr2=0x7fab9406f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:14.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.800+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 shutdown_connections 2023-08-25T09:48:14.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:14.801+0000 7fab9bc34640 1 -- 172.21.15.70:0/1232290591 wait complete. 2023-08-25T09:48:15.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:48:08.274966+0000', 'modified': '2023-08-25T09:48:11.852564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8718}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8718': {'gid': 8718, 'name': 'c', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6837/1719004988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6835', 'nonce': 1719004988}, {'type': 'v1', 'addr': '172.21.15.70:6837', 'nonce': 1719004988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:48:15.224 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:48:15.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-08-25T09:48:15.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.446+0000 7fc3eb742640 1 Processor -- start 2023-08-25T09:48:15.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.446+0000 7fc3eb742640 1 -- start start 2023-08-25T09:48:15.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3eb742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40ca780 0x7fc3e40cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3eb742640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3e40b3cf0 0x7fc3e40b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3eb742640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e40b12b0 con 0x7fc3e415acd0 2023-08-25T09:48:15.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3eb742640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e406c0e0 con 0x7fc3e40b3cf0 2023-08-25T09:48:15.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3eb742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e406ae20 con 0x7fc3e40ca780 2023-08-25T09:48:15.450 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3e9cdf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3e40b3cf0 0x7fc3e40b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3e9cdf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc3e40b3cf0 0x7fc3e40b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44702/0 (socket says 172.21.15.70:44702) 2023-08-25T09:48:15.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.448+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/499840836 learned_addr learned my addr 172.21.15.70:0/499840836 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.449+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/499840836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40ca780 msgr2=0x7fc3e40cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.451 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.449+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/499840836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40ca780 0x7fc3e40cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.449+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/499840836 >> v1:172.21.15.155:6789/0 conn(0x7fc3e415acd0 legacy=0x7fc3e415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.449+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/499840836 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e415b7c0 con 0x7fc3e40b3cf0 2023-08-25T09:48:15.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.450+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/499840836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e40b3cf0 0x7fc3e40b4140 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009870 tx=0x7fc3dc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e5128454b4a835 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.450+0000 7fc3e3fff640 1 -- 172.21.15.70:0/499840836 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0317c0 con 0x7fc3e40b3cf0 2023-08-25T09:48:15.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.450+0000 7fc3e3fff640 1 -- 172.21.15.70:0/499840836 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc031920 con 0x7fc3e40b3cf0 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.450+0000 7fc3e3fff640 1 -- 172.21.15.70:0/499840836 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc031cb0 con 0x7fc3e40b3cf0 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 -- 172.21.15.70:0/499840836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e40b3cf0 msgr2=0x7fc3e40b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 --2- 172.21.15.70:0/499840836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e40b3cf0 0x7fc3e40b4140 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009870 tx=0x7fc3dc02b860 comp rx=0 tx=0).stop 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 -- 172.21.15.70:0/499840836 shutdown_connections 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 --2- 172.21.15.70:0/499840836 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e40b3cf0 0x7fc3e40b4140 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 --2- 172.21.15.70:0/499840836 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40ca780 0x7fc3e40cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 -- 172.21.15.70:0/499840836 >> 172.21.15.70:0/499840836 conn(0x7fc3e40ce760 msgr2=0x7fc3e40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 -- 172.21.15.70:0/499840836 shutdown_connections 2023-08-25T09:48:15.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.451+0000 7fc3eb742640 1 -- 172.21.15.70:0/499840836 wait complete. 2023-08-25T09:48:15.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.453+0000 7fc3eb742640 1 Processor -- start 2023-08-25T09:48:15.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.454+0000 7fc3eb742640 1 -- start start 2023-08-25T09:48:15.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3eb742640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40b3cf0 0x7fc3e4145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3eb742640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e40ca780 0x7fc3e4145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3eb742640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 0x7fc3e4147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3eb742640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4146700 con 0x7fc3e415acd0 2023-08-25T09:48:15.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3eb742640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41469e0 con 0x7fc3e40ca780 2023-08-25T09:48:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3eb742640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4146cc0 con 0x7fc3e40b3cf0 2023-08-25T09:48:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 0x7fc3e4147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 0x7fc3e4147d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44712/0 (socket says 172.21.15.70:44712) 2023-08-25T09:48:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/3003004270 learned_addr learned my addr 172.21.15.70:0/3003004270 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e40ca780 0x7fc3e4145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.458 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40b3cf0 msgr2=0x7fc3e4145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40b3cf0 0x7fc3e4145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40b3cf0 0x7fc3e4145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e40ca780 msgr2=0x7fc3e4145cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e40ca780 0x7fc3e4145cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/3003004270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0094d0 con 0x7fc3e415acd0 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.455+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e40ca780 0x7fc3e4145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:15.459 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40b3cf0 0x7fc3e4145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:15.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 0x7fc3e4147d40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc02b8a0 tx=0x7fc3dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3e1ffb640 1 -- 172.21.15.70:0/3003004270 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc032920 con 0x7fc3e415acd0 2023-08-25T09:48:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3e1ffb640 1 -- 172.21.15.70:0/3003004270 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc032aa0 con 0x7fc3e415acd0 2023-08-25T09:48:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3e1ffb640 1 -- 172.21.15.70:0/3003004270 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc032e50 con 0x7fc3e415acd0 2023-08-25T09:48:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e4146fa0 con 0x7fc3e415acd0 2023-08-25T09:48:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.456+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e41458c0 con 0x7fc3e415acd0 2023-08-25T09:48:15.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.459+0000 7fc3e1ffb640 1 -- 172.21.15.70:0/3003004270 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc3dc04d3f0 con 0x7fc3e415acd0 2023-08-25T09:48:15.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.460+0000 7fc3e1ffb640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3d006a080 0x7fc3d006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.461+0000 7fc3e1ffb640 1 -- 172.21.15.70:0/3003004270 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3dc0aa5f0 con 0x7fc3e415acd0 2023-08-25T09:48:15.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.461+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3d006a080 0x7fc3d006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.461+0000 7fc3ee38e740 1 Processor -- start 2023-08-25T09:48:15.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.461+0000 7fc3ee38e740 1 -- start start 2023-08-25T09:48:15.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.461+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3d006a080 0x7fc3d006c540 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4007fd0 tx=0x7fc3d4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 0x55c5bdf79f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 0x55c5bdf4dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdebd190 con 0x55c5bdf77590 2023-08-25T09:48:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdeba880 con 0x55c5bdf959a0 2023-08-25T09:48:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdeb6170 con 0x55c5bdf97e80 2023-08-25T09:48:15.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3e9cdf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 0x55c5bdf4dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3e9cdf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 0x55c5bdf4dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44716/0 (socket says 172.21.15.70:44716) 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.462+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/778498882 learned_addr learned my addr 172.21.15.70:0/778498882 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/778498882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 msgr2=0x55c5bdf79f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 0x55c5bdf79f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/778498882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 msgr2=0x55c5bdf7a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/778498882 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5bdeb47f0 con 0x55c5bdf77590 2023-08-25T09:48:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 0x55c5bdf4dcd0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc079800 tx=0x7fc3dc075ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc1e02e6fc34b2d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3caffd640 1 -- 172.21.15.70:0/778498882 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc077c10 con 0x55c5bdf77590 2023-08-25T09:48:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3caffd640 1 -- 172.21.15.70:0/778498882 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc075f80 con 0x55c5bdf77590 2023-08-25T09:48:15.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.463+0000 7fc3caffd640 1 -- 172.21.15.70:0/778498882 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc0813a0 con 0x55c5bdf77590 2023-08-25T09:48:15.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 -- 172.21.15.70:0/778498882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 msgr2=0x55c5bdf4dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 0x55c5bdf4dcd0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc079800 tx=0x7fc3dc075ce0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 -- 172.21.15.70:0/778498882 shutdown_connections 2023-08-25T09:48:15.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf77590 0x55c5bdf4dcd0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf7a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/778498882 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 0x55c5bdf79f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 -- 172.21.15.70:0/778498882 >> 172.21.15.70:0/778498882 conn(0x55c5bdf4c3e0 msgr2=0x55c5bdf94f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 -- 172.21.15.70:0/778498882 shutdown_connections 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.464+0000 7fc3ee38e740 1 -- 172.21.15.70:0/778498882 wait complete. 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.465+0000 7fc3ee38e740 1 -- start start 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.465+0000 7fc3ee38e740 1 Processor -- start 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 0x55c5bdf701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 0x55c5bdf78850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf79990 0x55c5bdf79e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bde55d20 con 0x55c5bdf959a0 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bde559c0 con 0x55c5bdf79990 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdc1bf20 con 0x55c5bdf97e80 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3ee38e740 10 client.0 _ll_register_callbacks cb 0x55c5bde6d630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3e94de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 0x55c5bdf701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3e94de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 0x55c5bdf701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44730/0 (socket says 172.21.15.70:44730) 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.466+0000 7fc3e94de640 1 -- 172.21.15.70:0/2733415761 learned_addr learned my addr 172.21.15.70:0/2733415761 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3e94de640 1 -- 172.21.15.70:0/2733415761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 msgr2=0x55c5bdf78850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3e94de640 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 0x55c5bdf78850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3e94de640 1 -- 172.21.15.70:0/2733415761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf79990 msgr2=0x55c5bdf79e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3e94de640 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf79990 0x55c5bdf79e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3e94de640 1 -- 172.21.15.70:0/2733415761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e41458c0 con 0x55c5bdf959a0 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3e94de640 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 0x55c5bdf701c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400df00 tx=0x7fc3d40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3c22b29b26f05 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3bffff640 1 -- 172.21.15.70:0/2733415761 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4012820 con 0x55c5bdf959a0 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3bffff640 1 -- 172.21.15.70:0/2733415761 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3d401eb10 con 0x55c5bdf959a0 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5bde8a540 con 0x55c5bdf959a0 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3bffff640 1 -- 172.21.15.70:0/2733415761 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3d401ecd0 con 0x55c5bdf959a0 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3ee38e740 10 client.8871 fetch_fsmap learned FSMap version 74 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.467+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55c5bdeadf30 con 0x55c5bdf959a0 2023-08-25T09:48:15.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3bffff640 1 -- 172.21.15.70:0/2733415761 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5bde8a540 con 0x55c5bdf959a0 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3bffff640 1 -- 172.21.15.70:0/2733415761 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3d4012980 con 0x55c5bdf959a0 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3ee38e740 10 client.8871 fetch_fsmap finished waiting for FSMap version 74 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3ee38e740 10 client.8871 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3ee38e740 20 client.8871 populate_metadata read hostname 'smithi070' 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdfacf10 0x55c5bdfad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3ee38e740 4 client.8871 mds_command: new command op to 8718 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 --> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5bdeaf150 con 0x55c5bdfacf10 2023-08-25T09:48:15.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.468+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdfacf10 0x55c5bdfad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.469+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdfacf10 0x55c5bdfad2f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16c92c4207ccea1a server_cookie=e180e6bd25592314 in_seq=0 out_seq=0 2023-08-25T09:48:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.469+0000 7fc3bffff640 10 client.8871 ms_handle_connect on v2:172.21.15.70:6835/1719004988 2023-08-25T09:48:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.469+0000 7fc3bffff640 1 -- 172.21.15.70:0/2733415761 <== mds.0 v2:172.21.15.70:6835/1719004988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c5bdeaf150 con 0x55c5bdfacf10 2023-08-25T09:48:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.469+0000 7fc3bffff640 10 client.8871 handle_command_reply: tid=0 2023-08-25T09:48:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.469+0000 7fc3ee38e740 1 client.8871 shutdown 2023-08-25T09:48:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 msgr2=0x55c5bdf701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 0x55c5bdf701c0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400df00 tx=0x7fc3d40054a0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 shutdown_connections 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdfacf10 0x55c5bdfad2f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf79990 0x55c5bdf79e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf97e80 0x55c5bdf78850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2733415761 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf959a0 0x55c5bdf701c0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3bffff640 0 client.8871 ms_handle_reset on v2:172.21.15.70:6835/1719004988 2023-08-25T09:48:15.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 >> 172.21.15.70:0/2733415761 conn(0x55c5bdf77590 msgr2=0x55c5bdf498a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 shutdown_connections 2023-08-25T09:48:15.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2733415761 wait complete. 2023-08-25T09:48:15.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.470+0000 7fc3ee38e740 20 client.8871 trim_cache size 0 max 16384 2023-08-25T09:48:15.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.486+0000 7fc3ee38e740 1 Processor -- start 2023-08-25T09:48:15.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 -- start start 2023-08-25T09:48:15.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf79990 0x55c5bdfb3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf76a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 0x55c5bdf79dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdf77e40 con 0x55c5bdf97e80 2023-08-25T09:48:15.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdf50930 con 0x55c5bdf959a0 2023-08-25T09:48:15.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.487+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdf4f030 con 0x55c5bdf79990 2023-08-25T09:48:15.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 0x55c5bdf79dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 0x55c5bdf79dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44742/0 (socket says 172.21.15.70:44742) 2023-08-25T09:48:15.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/2635679101 learned_addr learned my addr 172.21.15.70:0/2635679101 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e94de640 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf79990 0x55c5bdfb3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf79990 msgr2=0x55c5bdfb3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf76a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf79990 0x55c5bdfb3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 msgr2=0x55c5bdf76a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf76a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.488+0000 7fc3e9cdf640 1 -- 172.21.15.70:0/2635679101 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5bde54a10 con 0x55c5bdf97e80 2023-08-25T09:48:15.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3e9cdf640 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 0x55c5bdf79dd0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc079800 tx=0x7fc3dc07d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d581746bf60256b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3cb7fe640 1 -- 172.21.15.70:0/2635679101 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc079640 con 0x55c5bdf97e80 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3cb7fe640 1 -- 172.21.15.70:0/2635679101 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc0760f0 con 0x55c5bdf97e80 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3cb7fe640 1 -- 172.21.15.70:0/2635679101 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc081180 con 0x55c5bdf97e80 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2635679101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 msgr2=0x55c5bdf79dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 0x55c5bdf79dd0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc079800 tx=0x7fc3dc07d000 comp rx=0 tx=0).stop 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2635679101 shutdown_connections 2023-08-25T09:48:15.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf97e80 0x55c5bdf79dd0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf959a0 0x55c5bdf76a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/2635679101 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf79990 0x55c5bdfb3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2635679101 >> 172.21.15.70:0/2635679101 conn(0x55c5bdf77590 msgr2=0x55c5bdf4a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2635679101 shutdown_connections 2023-08-25T09:48:15.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.489+0000 7fc3ee38e740 1 -- 172.21.15.70:0/2635679101 wait complete. 2023-08-25T09:48:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.490+0000 7fc3ee38e740 1 -- start start 2023-08-25T09:48:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.490+0000 7fc3ee38e740 1 Processor -- start 2023-08-25T09:48:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 0x55c5bdf730a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf959a0 0x55c5bdf4e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf97e80 0x55c5be006d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5be007260 con 0x55c5bdf79990 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdf91970 con 0x55c5bdf97e80 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5bdf73610 con 0x55c5bdf959a0 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3ee38e740 10 client.0 _ll_register_callbacks cb 0x7fc3dc075c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3e94de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 0x55c5bdf730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3e94de640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 0x55c5bdf730a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44754/0 (socket says 172.21.15.70:44754) 2023-08-25T09:48:15.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.491+0000 7fc3e94de640 1 -- 172.21.15.70:0/3726627187 learned_addr learned my addr 172.21.15.70:0/3726627187 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3e94de640 1 -- 172.21.15.70:0/3726627187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf959a0 msgr2=0x55c5bdf4e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf959a0 0x55c5bdf4e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3e94de640 1 -- 172.21.15.70:0/3726627187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf97e80 msgr2=0x55c5be006d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf97e80 0x55c5be006d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3e94de640 1 -- 172.21.15.70:0/3726627187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5bdeb47f0 con 0x55c5bdf79990 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3e94de640 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 0x55c5bdf730a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40054e0 tx=0x7fc3d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76893dcb7e4cc128 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:48:15.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3bffff640 1 -- 172.21.15.70:0/3726627187 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4012a30 con 0x55c5bdf79990 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3bffff640 1 -- 172.21.15.70:0/3726627187 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3d4033040 con 0x55c5bdf79990 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.492+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3dc075f80 con 0x55c5bdf79990 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3bffff640 1 -- 172.21.15.70:0/3726627187 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3d4037440 con 0x55c5bdf79990 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 10 client.8877 fetch_fsmap learned FSMap version 74 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55c5bde79de0 con 0x55c5bdf79990 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3bffff640 1 -- 172.21.15.70:0/3726627187 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc075f80 con 0x55c5bdf79990 2023-08-25T09:48:15.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3bffff640 1 -- 172.21.15.70:0/3726627187 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3d403f070 con 0x55c5bdf79990 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 10 client.8877 fetch_fsmap finished waiting for FSMap version 74 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 10 client.8877 resolve_mds: resolved name 'c' to daemon mds.c 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 20 client.8877 populate_metadata read hostname 'smithi070' 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdf90d10 0x55c5bdf910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 4 client.8877 mds_command: new command op to 8718 tid=0[{"prefix": "status"}] 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 --> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c5bdb655e0 con 0x55c5bdf90d10 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.493+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdf90d10 0x55c5bdf910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3e8cdd640 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdf90d10 0x55c5bdf910f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0445eba7076f78 server_cookie=d0bc0cb890e17d6e in_seq=0 out_seq=0 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3bffff640 10 client.8877 ms_handle_connect on v2:172.21.15.70:6835/1719004988 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3bffff640 1 -- 172.21.15.70:0/3726627187 <== mds.0 v2:172.21.15.70:6835/1719004988 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c5bdb655e0 con 0x55c5bdf90d10 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3bffff640 10 client.8877 handle_command_reply: tid=0 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 client.8877 shutdown 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 msgr2=0x55c5bdf730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 0x55c5bdf730a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40054e0 tx=0x7fc3d4011000 comp rx=0 tx=0).stop 2023-08-25T09:48:15.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 shutdown_connections 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:6835/1719004988,v1:172.21.15.70:6837/1719004988] conn(0x55c5bdf90d10 0x55c5bdf910f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x55c5bdf97e80 0x55c5be006d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x55c5bdf959a0 0x55c5bdf4e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 --2- 172.21.15.70:0/3726627187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x55c5bdf79990 0x55c5bdf730a0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3bffff640 0 client.8877 ms_handle_reset on v2:172.21.15.70:6835/1719004988 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 >> 172.21.15.70:0/3726627187 conn(0x55c5bdf77590 msgr2=0x55c5bdfb3320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 shutdown_connections 2023-08-25T09:48:15.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.494+0000 7fc3ee38e740 1 -- 172.21.15.70:0/3726627187 wait complete. 2023-08-25T09:48:15.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3ee38e740 20 client.8877 trim_cache size 0 max 16384 2023-08-25T09:48:15.504 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:48:15.504 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:48:15.504 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8718, 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.6648797000000002, 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 74, 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 165, 2023-08-25T09:48:15.505 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 165, 2023-08-25T09:48:15.506 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 12.061723646000001 2023-08-25T09:48:15.506 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:48:15.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3d006a080 msgr2=0x7fc3d006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3d006a080 0x7fc3d006c540 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4007fd0 tx=0x7fc3d4009040 comp rx=0 tx=0).stop 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 msgr2=0x7fc3e4147d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 0x7fc3e4147d40 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc02b8a0 tx=0x7fc3dc0046c0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 shutdown_connections 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3d006a080 0x7fc3d006c540 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3e415acd0 0x7fc3e4147d40 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3e40ca780 0x7fc3e4145cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 --2- 172.21.15.70:0/3003004270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3e40b3cf0 0x7fc3e4145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 >> 172.21.15.70:0/3003004270 conn(0x7fc3e40ce760 msgr2=0x7fc3e415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 shutdown_connections 2023-08-25T09:48:15.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.495+0000 7fc3eb742640 1 -- 172.21.15.70:0/3003004270 wait complete. 2023-08-25T09:48:15.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:48:15.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.723+0000 7fd50720d640 1 Processor -- start 2023-08-25T09:48:15.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.723+0000 7fd50720d640 1 -- start start 2023-08-25T09:48:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.724+0000 7fd50720d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50015aa10 0x7fd5000b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.724+0000 7fd50720d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5000b2680 0x7fd50006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.724+0000 7fd50720d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50015d3f0 con 0x7fd5000ce5b0 2023-08-25T09:48:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.724+0000 7fd50720d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5000b2b10 con 0x7fd5000b2680 2023-08-25T09:48:15.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.724+0000 7fd50720d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd500159d10 con 0x7fd50015aa10 2023-08-25T09:48:15.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.725+0000 7fd5057aa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5000b2680 0x7fd50006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.725+0000 7fd504fa9640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fd5000ce5b0 0x7fd5000ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:43740/0 (socket says 172.21.15.70:43740) 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.725+0000 7fd5057aa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd5000b2680 0x7fd50006f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44764/0 (socket says 172.21.15.70:44764) 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.725+0000 7fd504fa9640 1 -- 172.21.15.70:0/2601761708 learned_addr learned my addr 172.21.15.70:0/2601761708 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.725+0000 7fd4fffff640 1 --2- 172.21.15.70:0/2601761708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50015aa10 0x7fd5000b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.726+0000 7fd5057aa640 1 -- 172.21.15.70:0/2601761708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50015aa10 msgr2=0x7fd5000b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.726+0000 7fd5057aa640 1 --2- 172.21.15.70:0/2601761708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50015aa10 0x7fd5000b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.726+0000 7fd5057aa640 1 -- 172.21.15.70:0/2601761708 >> v1:172.21.15.155:6789/0 conn(0x7fd5000ce5b0 legacy=0x7fd5000ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.726+0000 7fd5057aa640 1 -- 172.21.15.70:0/2601761708 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000b2d20 con 0x7fd5000b2680 2023-08-25T09:48:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.727+0000 7fd5057aa640 1 --2- 172.21.15.70:0/2601761708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50006f9f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80097a0 tx=0x7fd4f802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec52b2e6a1900cf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.727+0000 7fd4ff7fe640 1 -- 172.21.15.70:0/2601761708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f802d040 con 0x7fd5000b2680 2023-08-25T09:48:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.727+0000 7fd4ff7fe640 1 -- 172.21.15.70:0/2601761708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f8031810 con 0x7fd5000b2680 2023-08-25T09:48:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.727+0000 7fd4ff7fe640 1 -- 172.21.15.70:0/2601761708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f8031ba0 con 0x7fd5000b2680 2023-08-25T09:48:15.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 -- 172.21.15.70:0/2601761708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 msgr2=0x7fd50006f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 --2- 172.21.15.70:0/2601761708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50006f9f0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fd4f80097a0 tx=0x7fd4f802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 -- 172.21.15.70:0/2601761708 shutdown_connections 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 --2- 172.21.15.70:0/2601761708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50006f9f0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 --2- 172.21.15.70:0/2601761708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd50015aa10 0x7fd5000b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 -- 172.21.15.70:0/2601761708 >> 172.21.15.70:0/2601761708 conn(0x7fd500069850 msgr2=0x7fd500069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 -- 172.21.15.70:0/2601761708 shutdown_connections 2023-08-25T09:48:15.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.728+0000 7fd50720d640 1 -- 172.21.15.70:0/2601761708 wait complete. 2023-08-25T09:48:15.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.729+0000 7fd50720d640 1 Processor -- start 2023-08-25T09:48:15.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.730+0000 7fd50720d640 1 -- start start 2023-08-25T09:48:15.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.730+0000 7fd50720d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50014f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd50720d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5000ce5b0 0x7fd5000ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd50720d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd50015aa10 0x7fd5000acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd50720d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5001f2ba0 con 0x7fd5000b2680 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd50720d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b1790 con 0x7fd50015aa10 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd50720d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50014f7f0 con 0x7fd5000ce5b0 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50014f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50014f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44774/0 (socket says 172.21.15.70:44774) 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 -- 172.21.15.70:0/2962074591 learned_addr learned my addr 172.21.15.70:0/2962074591 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:15.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 -- 172.21.15.70:0/2962074591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5000ce5b0 msgr2=0x7fd5000ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5000ce5b0 0x7fd5000ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 -- 172.21.15.70:0/2962074591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd50015aa10 msgr2=0x7fd5000acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd50015aa10 0x7fd5000acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 -- 172.21.15.70:0/2962074591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009400 con 0x7fd5000b2680 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.731+0000 7fd504fa9640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50014f2b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000cba0 tx=0x7fd4f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.732+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000ad40 con 0x7fd5000b2680 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.732+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f0012400 con 0x7fd5000b2680 2023-08-25T09:48:15.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.732+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000aea0 con 0x7fd5000b2680 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.732+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5001f2270 con 0x7fd5000b2680 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.732+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5001f2830 con 0x7fd5000b2680 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.733+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50004e420 con 0x7fd5000b2680 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.734+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd4f0007070 con 0x7fd5000b2680 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.735+0000 7fd4fd7fa640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4ec06a050 0x7fd4ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.735+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4f0088470 con 0x7fd5000b2680 2023-08-25T09:48:15.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.735+0000 7fd4fffff640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4ec06a050 0x7fd4ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:15.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.736+0000 7fd4fffff640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4ec06a050 0x7fd4ec06c510 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd500152c70 tx=0x7fd4f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:15.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.740+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd4f0053c10 con 0x7fd5000b2680 2023-08-25T09:48:15.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.955+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd5000cae50 con 0x7fd5000b2680 2023-08-25T09:48:15.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.955+0000 7fd4fd7fa640 1 -- 172.21.15.70:0/2962074591 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd4f0057230 con 0x7fd5000b2680 2023-08-25T09:48:15.958 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:48:15.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4ec06a050 msgr2=0x7fd4ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4ec06a050 0x7fd4ec06c510 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd500152c70 tx=0x7fd4f4008040 comp rx=0 tx=0).stop 2023-08-25T09:48:15.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 msgr2=0x7fd50014f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:15.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50014f2b0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000cba0 tx=0x7fd4f000a430 comp rx=0 tx=0).stop 2023-08-25T09:48:15.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 shutdown_connections 2023-08-25T09:48:15.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd4ec06a050 0x7fd4ec06c510 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd50015aa10 0x7fd5000acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd5000ce5b0 0x7fd5000ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 --2- 172.21.15.70:0/2962074591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd5000b2680 0x7fd50014f2b0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:15.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 >> 172.21.15.70:0/2962074591 conn(0x7fd500069850 msgr2=0x7fd50015b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:15.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 shutdown_connections 2023-08-25T09:48:15.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:15.958+0000 7fd50720d640 1 -- 172.21.15.70:0/2962074591 wait complete. 2023-08-25T09:48:15.978 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:48:15.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:48:15.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:48:15.979 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:48:15.979 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:48:15.979 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:48:15.979 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:48:15.980 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:48:15.980 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:48:15.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:15.981 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:48:16.032 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:48:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:48:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:48:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:48:16.033 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.033 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:16.034 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41780sec preferred_lft 41780sec 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:16.035 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:48:16.036 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.037 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:48:16.037 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:48:16.037 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:48:16.037 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:16.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:16.038 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:48:16.126 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:48:16.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:16.130 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:48:16.183 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:48:16.184 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:48:16.184 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:48:16.184 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:48:16.185 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:48:16.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:16.186 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:48:16.186 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:48:16.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:48:16.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:48:16.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:48:16.188 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:48:16.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:16.373 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:48:16.424 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:48:16.425 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:48:16.425 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:48:16.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:16.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:48:16.530 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:48:16.530 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:48:16.531 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:48:16.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:48:16.586 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:48:16.588 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:48:16.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:48:16.589 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:48:16.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:48:16.757 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:48:16.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:48:16.851 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:48:16.855 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:48:16.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:16.856 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:48:16.907 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:48:16.908 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:48:16.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:48:16.996 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:48:16.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:48:17.072 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:48:17.074 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:48:17.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:17.076 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:48:17.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:48:17.098+0000 7f10b3c64480 -1 init, newargv = 0x561ea60a7040 newargc=15 2023-08-25T09:48:17.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[121171]: starting ceph client 2023-08-25T09:48:17.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[121171]: starting fuse 2023-08-25T09:48:17.120 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:48:17.120 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:48:17.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:48:17.121 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:48:17.121 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:48:17.122 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:48:17.122 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:48:17.122 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:17.123 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:48:17.123 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:48:17.123 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:48:17.124 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:48:17.124 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:17.124 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:48:17.125 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:48:17.125 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:48:17.125 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:48:17.125 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:17.126 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:48:17.126 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:48:17.126 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:48:17.127 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:48:17.127 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:48:17.127 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:48:17.128 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:48:17.128 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:48:17.128 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:17.129 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:48:17.129 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:48:17.328 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.121171.asok 2023-08-25T09:48:17.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:48:17.346 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.121171.asok 2023-08-25T09:48:17.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:17.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.121171.asok status 2023-08-25T09:48:17.582 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:48:17.583 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:48:17.583 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:48:17.583 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:48:17.583 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:48:17.583 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:48:17.583 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:48:17.584 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "121171", 2023-08-25T09:48:17.584 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:48:17.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:17.584 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:48:17.584 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:48:17.584 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 8892, 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 8892 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:48:17.585 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3814328231 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:48:17.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3814328231 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.8892 192.168.0.1:0/3814328231", 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/3814328231", 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 74, 2023-08-25T09:48:17.587 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 165, 2023-08-25T09:48:17.588 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:48:17.588 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:48:17.588 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:48:17.588 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:48:17.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:17.599 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:48:17.600 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:48:17.654 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:48:17.654 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:48:17.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:17.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:48:17.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:48:17.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.967+0000 7f3ac05fd640 1 Processor -- start 2023-08-25T09:48:17.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.967+0000 7f3ac05fd640 1 -- start start 2023-08-25T09:48:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3ac05fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 0x7f3ab806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3ac05fd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ab815ec10 0x7f3ab80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3ac05fd640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80b1b60 con 0x7f3ab806b2b0 2023-08-25T09:48:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3ac05fd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab806bd80 con 0x7f3ab815ec10 2023-08-25T09:48:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3ac05fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab806b7c0 con 0x7f3ab815e830 2023-08-25T09:48:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3abdb98640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ab806b2b0 0x7f3ab815c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57332/0 (socket says 172.21.15.70:57332) 2023-08-25T09:48:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.969+0000 7f3abdb98640 1 -- 172.21.15.70:0/4094325208 learned_addr learned my addr 172.21.15.70:0/4094325208 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abeb9a640 1 --2- 172.21.15.70:0/4094325208 >> v2:172.21.15.155:3300/0 conn(0x7f3ab815ec10 0x7f3ab80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abe399640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 0x7f3ab806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455162115 0 0) 0x7f3ab80b1b60 con 0x7f3ab806b2b0 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa4003520 con 0x7f3ab806b2b0 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362658473 0 0) 0x7f3aa4003520 con 0x7f3ab806b2b0 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 msgr2=0x7f3ab806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 0x7f3ab806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 >> v2:172.21.15.155:3300/0 conn(0x7f3ab815ec10 msgr2=0x7f3ab80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 --2- 172.21.15.70:0/4094325208 >> v2:172.21.15.155:3300/0 conn(0x7f3ab815ec10 0x7f3ab80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab806baa0 con 0x7f3ab806b2b0 2023-08-25T09:48:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3aa8001670 con 0x7f3ab806b2b0 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 >> v1:172.21.15.70:6789/0 conn(0x7f3ab806b2b0 legacy=0x7f3ab815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa4003970 0x7f3aa4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aa40047c0 0x7f3aa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 0x7f3aa4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab806bd80 con 0x7f3aa40047c0 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab806b7c0 con 0x7f3aa4003970 2023-08-25T09:48:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.970+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa400b980 con 0x7f3ab815e830 2023-08-25T09:48:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.971+0000 7f3abdb98640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aa40047c0 0x7f3aa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.971+0000 7f3abdb98640 1 -- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 msgr2=0x7f3aa4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.971+0000 7f3abdb98640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab815e830 0x7f3aa4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.971+0000 7f3abdb98640 1 -- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa4003970 msgr2=0x7f3aa4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.971+0000 7f3abdb98640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa4003970 0x7f3aa4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.971+0000 7f3abdb98640 1 -- 172.21.15.70:0/4094325208 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4003520 con 0x7f3aa40047c0 2023-08-25T09:48:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.972+0000 7f3abdb98640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aa40047c0 0x7f3aa4004c40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806b6f0 tx=0x7f3aa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ced155e2eb5322 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.972+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8005ad0 con 0x7f3aa40047c0 2023-08-25T09:48:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.972+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa8005c30 con 0x7f3aa40047c0 2023-08-25T09:48:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.972+0000 7f3abd397640 1 -- 172.21.15.70:0/4094325208 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa802c890 con 0x7f3aa40047c0 2023-08-25T09:48:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3ac05fd640 1 -- 172.21.15.70:0/4094325208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aa40047c0 msgr2=0x7f3aa4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/4094325208 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aa40047c0 0x7f3aa4004c40 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f3ab806b6f0 tx=0x7f3aa40057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3abeb9a640 1 -- 172.21.15.70:0/4094325208 reap_dead start 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3ac05fd640 1 -- 172.21.15.70:0/4094325208 shutdown_connections 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3ac05fd640 1 -- 172.21.15.70:0/4094325208 >> 172.21.15.70:0/4094325208 conn(0x7f3ab8069850 msgr2=0x7f3ab8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3ac05fd640 1 -- 172.21.15.70:0/4094325208 shutdown_connections 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.973+0000 7f3ac05fd640 1 -- 172.21.15.70:0/4094325208 wait complete. 2023-08-25T09:48:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.975+0000 7f3ac05fd640 1 Processor -- start 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.975+0000 7f3ac05fd640 1 -- start start 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3ac05fd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa40047c0 0x7f3ab80aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3ac05fd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 0x7f3ab814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3ac05fd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab80ac9e0 0x7f3ab80ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3ac05fd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80ad3a0 con 0x7f3ab80a8e70 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3ac05fd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80a9ed0 con 0x7f3aa40047c0 2023-08-25T09:48:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3ac05fd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80a9630 con 0x7f3ab80ac9e0 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3abdb98640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 0x7f3ab814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3abdb98640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 0x7f3ab814b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:44788/0 (socket says 172.21.15.70:44788) 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.976+0000 7f3abdb98640 1 -- 172.21.15.70:0/1870516732 learned_addr learned my addr 172.21.15.70:0/1870516732 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abeb9a640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab80ac9e0 0x7f3ab80ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abe399640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa40047c0 0x7f3ab80aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abdb98640 1 -- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab80ac9e0 msgr2=0x7f3ab80ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abdb98640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab80ac9e0 0x7f3ab80ace60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abdb98640 1 -- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa40047c0 msgr2=0x7f3ab80aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abdb98640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa40047c0 0x7f3ab80aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abdb98640 1 -- 172.21.15.70:0/1870516732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab815ca20 con 0x7f3ab80a8e70 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abe399640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa40047c0 0x7f3ab80aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abeb9a640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab80ac9e0 0x7f3ab80ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.977+0000 7f3abdb98640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 0x7f3ab814b020 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c300 tx=0x7f3ab80b1ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.978+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa802cca0 con 0x7f3ab80a8e70 2023-08-25T09:48:17.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.978+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa802ce00 con 0x7f3ab80a8e70 2023-08-25T09:48:17.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.978+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8007290 con 0x7f3ab80a8e70 2023-08-25T09:48:17.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.978+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80a9910 con 0x7f3ab80a8e70 2023-08-25T09:48:17.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.978+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80aa1d0 con 0x7f3ab80a8e70 2023-08-25T09:48:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.979+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab804e420 con 0x7f3ab80a8e70 2023-08-25T09:48:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.980+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3aa8032070 con 0x7f3ab80a8e70 2023-08-25T09:48:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.981+0000 7f3aaeffd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3a9c06a050 0x7f3a9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.981+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3aa803c040 con 0x7f3ab80a8e70 2023-08-25T09:48:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.981+0000 7f3abe399640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3a9c06a050 0x7f3a9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.982+0000 7f3abe399640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3a9c06a050 0x7f3a9c06c510 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009950 tx=0x7f3ab4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:17.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:17.986+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3aa80751e0 con 0x7f3ab80a8e70 2023-08-25T09:48:18.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.213+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ab806c620 con 0x7f3ab80a8e70 2023-08-25T09:48:18.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.214+0000 7f3aaeffd640 1 -- 172.21.15.70:0/1870516732 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3aa8045060 con 0x7f3ab80a8e70 2023-08-25T09:48:18.216 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:48:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3a9c06a050 msgr2=0x7f3a9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3a9c06a050 0x7f3a9c06c510 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009950 tx=0x7f3ab4008040 comp rx=0 tx=0).stop 2023-08-25T09:48:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 msgr2=0x7f3ab814b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:18.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 0x7f3ab814b020 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c300 tx=0x7f3ab80b1ad0 comp rx=0 tx=0).stop 2023-08-25T09:48:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 shutdown_connections 2023-08-25T09:48:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3a9c06a050 0x7f3a9c06c510 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ab80ac9e0 0x7f3ab80ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:18.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3ab80a8e70 0x7f3ab814b020 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:18.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 --2- 172.21.15.70:0/1870516732 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aa40047c0 0x7f3ab80aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:18.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 >> 172.21.15.70:0/1870516732 conn(0x7f3ab8069850 msgr2=0x7f3ab80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:18.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 shutdown_connections 2023-08-25T09:48:18.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:18.217+0000 7f3ac05fd640 1 -- 172.21.15.70:0/1870516732 wait complete. 2023-08-25T09:48:18.235 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:48:18.236 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:48:18.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:48:18.237 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:48:18.237 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:48:18.237 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:48:18.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:48:18.239 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:48:18.239 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:48:18.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:18.240 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:48:18.302 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:48:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:48:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:48:18.302 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:48:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.303 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:48:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:18.303 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:48:18.304 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:48:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:18.304 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:48:18.305 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:48:18.305 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:18.305 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:48:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:48:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41791sec preferred_lft 41791sec 2023-08-25T09:48:18.306 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:48:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.307 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:48:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:48:18.307 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:48:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:48:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.308 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:48:18.308 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:48:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:48:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.309 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:48:18.309 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:48:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:48:18.310 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:48:18.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:18.311 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:48:18.404 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:48:18.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:18.409 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:48:18.458 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:48:18.459 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:48:18.459 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:48:18.459 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:48:18.459 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:48:18.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:18.460 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:48:18.460 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:48:18.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:48:18.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:48:18.461 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:48:18.462 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:48:18.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:18.646 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:48:18.695 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:48:18.696 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:48:18.696 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:48:18.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:18.697 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:48:18.788 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:48:18.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:48:18.789 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:48:18.834 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:48:18.835 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:48:18.835 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:48:18.934 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:48:19.033 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:48:19.034 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:48:19.120 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:48:19.123 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:48:19.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:19.125 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:48:19.175 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:48:19.176 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:48:19.224 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:48:19.310 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:48:19.310 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:48:19.382 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:48:19.385 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:48:19.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:19.389 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:48:19.422 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:48:19.420+0000 7f8b7563a480 -1 init, newargv = 0x562e42836d60 newargc=15 2023-08-25T09:48:19.422 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[92718]: starting ceph client 2023-08-25T09:48:19.431 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[92718]: starting fuse 2023-08-25T09:48:19.434 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:48:19.434 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:48:19.435 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:48:19.436 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:48:19.436 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:48:19.437 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:48:19.438 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:48:19.438 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:48:19.439 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:48:19.439 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:48:19.441 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:48:19.442 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:48:19.442 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:48:19.443 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:48:19.444 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:48:19.444 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:48:19.445 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:48:19.445 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:48:19.446 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:48:19.447 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:48:19.447 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:48:19.448 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:48:19.449 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:48:19.449 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:48:19.450 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:48:19.451 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:48:19.452 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:48:19.453 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:48:19.454 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:48:19.642 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.92718.asok 2023-08-25T09:48:19.646 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:48:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:48:19.659 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.92718.asok 2023-08-25T09:48:19.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:19.660 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.92718.asok status 2023-08-25T09:48:19.904 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:48:19.904 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:48:19.904 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "92718", 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:48:19.905 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 6115, 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 6115 2023-08-25T09:48:19.906 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 99006261 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:48:19.907 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 99006261 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.6115 192.168.0.1:0/99006261", 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/99006261", 2023-08-25T09:48:19.908 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:48:19.909 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 74, 2023-08-25T09:48:19.909 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 165, 2023-08-25T09:48:19.909 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:48:19.909 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:48:19.909 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:48:19.909 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:48:19.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:19.920 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:48:19.920 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:48:19.975 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:48:19.976 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:48:19.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:48:19.977 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:48:20.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:48:20.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.293+0000 7f12013eb640 1 Processor -- start 2023-08-25T09:48:20.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.293+0000 7f12013eb640 1 -- start start 2023-08-25T09:48:20.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f12013eb640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f11fc156980 0x7f11fc0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f12013eb640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0c98a0 con 0x7f11fc0ad6d0 2023-08-25T09:48:20.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f12013eb640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0c8c90 con 0x7f11fc06b330 2023-08-25T09:48:20.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f12013eb640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc0c8f70 con 0x7f11fc156980 2023-08-25T09:48:20.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f11f3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11fc0ad6d0 0x7f11fc06ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57200/0 (socket says 172.21.15.70:57200) 2023-08-25T09:48:20.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f11f3fff640 1 -- 172.21.15.70:0/3170196574 learned_addr learned my addr 172.21.15.70:0/3170196574 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:20.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.295+0000 7f11fbfff640 1 --2- 172.21.15.70:0/3170196574 >> v2:172.21.15.155:3300/0 conn(0x7f11fc156980 0x7f11fc0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168771813 0 0) 0x7f11fc0c98a0 con 0x7f11fc0ad6d0 2023-08-25T09:48:20.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11dc003440 con 0x7f11fc0ad6d0 2023-08-25T09:48:20.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707597229 0 0) 0x7f11dc003440 con 0x7f11fc0ad6d0 2023-08-25T09:48:20.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 >> v2:172.21.15.155:3300/0 conn(0x7f11fc156980 msgr2=0x7f11fc0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:20.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 --2- 172.21.15.70:0/3170196574 >> v2:172.21.15.155:3300/0 conn(0x7f11fc156980 0x7f11fc0c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 >> v1:172.21.15.155:6789/0 conn(0x7f11fc06b330 legacy=0x7f11fc0cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0c9250 con 0x7f11fc0ad6d0 2023-08-25T09:48:20.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f11e4001670 con 0x7f11fc0ad6d0 2023-08-25T09:48:20.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 >> v1:172.21.15.70:6789/0 conn(0x7f11fc0ad6d0 legacy=0x7f11fc06ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc003a60 0x7f11dc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11dc008850 0x7f11dc00ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11dc00b480 0x7f11dc00d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0c8f70 con 0x7f11dc008850 2023-08-25T09:48:20.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc00e220 con 0x7f11dc003a60 2023-08-25T09:48:20.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.296+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11dc00e3e0 con 0x7f11dc00b480 2023-08-25T09:48:20.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11f3fff640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11dc008850 0x7f11dc00ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11fbfff640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc003a60 0x7f11dc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11f3fff640 1 -- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11dc00b480 msgr2=0x7f11dc00d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:20.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11f3fff640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11dc00b480 0x7f11dc00d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11f3fff640 1 -- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc003a60 msgr2=0x7f11dc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:20.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11f3fff640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc003a60 0x7f11dc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11f3fff640 1 -- 172.21.15.70:0/3170196574 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc003440 con 0x7f11dc008850 2023-08-25T09:48:20.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f11fbfff640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc003a60 0x7f11dc003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:48:20.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.297+0000 7f1200bea640 1 -- 172.21.15.70:0/3170196574 reap_dead start 2023-08-25T09:48:20.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.298+0000 7f11f3fff640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11dc008850 0x7f11dc00ac80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c240 tx=0x7f11dc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495805423e239ce8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:20.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.298+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4005ad0 con 0x7f11dc008850 2023-08-25T09:48:20.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.298+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e4005c50 con 0x7f11dc008850 2023-08-25T09:48:20.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.298+0000 7f11fb7fe640 1 -- 172.21.15.70:0/3170196574 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e402c8b0 con 0x7f11dc008850 2023-08-25T09:48:20.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 -- 172.21.15.70:0/3170196574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11dc008850 msgr2=0x7f11dc00ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11dc008850 0x7f11dc00ac80 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c240 tx=0x7f11dc0036f0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 -- 172.21.15.70:0/3170196574 shutdown_connections 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 --2- 172.21.15.70:0/3170196574 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11dc008850 0x7f11dc00ac80 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 -- 172.21.15.70:0/3170196574 >> 172.21.15.70:0/3170196574 conn(0x7f11fc0affd0 msgr2=0x7f11fc0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 -- 172.21.15.70:0/3170196574 shutdown_connections 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.299+0000 7f12013eb640 1 -- 172.21.15.70:0/3170196574 wait complete. 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.301+0000 7f12013eb640 1 Processor -- start 2023-08-25T09:48:20.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.301+0000 7f12013eb640 1 -- start start 2023-08-25T09:48:20.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f12013eb640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc008850 0x7f11fc1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f12013eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 0x7f11fc0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f12013eb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc0acdc0 0x7f11fc06eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f12013eb640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0ad270 con 0x7f11fc14f370 2023-08-25T09:48:20.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f12013eb640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0718a0 con 0x7f11dc008850 2023-08-25T09:48:20.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f12013eb640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc1f2330 con 0x7f11fc0acdc0 2023-08-25T09:48:20.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f11f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 0x7f11fc0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f11f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 0x7f11fc0ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33874/0 (socket says 172.21.15.70:33874) 2023-08-25T09:48:20.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.302+0000 7f11f3fff640 1 -- 172.21.15.70:0/970758239 learned_addr learned my addr 172.21.15.70:0/970758239 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:20.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11fbfff640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc008850 0x7f11fc1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f1200bea640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc0acdc0 0x7f11fc06eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f3fff640 1 -- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc0acdc0 msgr2=0x7f11fc06eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f3fff640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc0acdc0 0x7f11fc06eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f3fff640 1 -- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc008850 msgr2=0x7f11fc1f1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f3fff640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc008850 0x7f11fc1f1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f3fff640 1 -- 172.21.15.70:0/970758239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc070d20 con 0x7f11fc14f370 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f3fff640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 0x7f11fc0ac880 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c240 tx=0x7f11e402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e403c040 con 0x7f11fc14f370 2023-08-25T09:48:20.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e402cd80 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc071000 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4007270 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.303+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0715c0 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.304+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc066fb0 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.305+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f11e4032070 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.306+0000 7f11f97fa640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11e006a080 0x7f11e006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.306+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11e40a9b50 con 0x7f11fc14f370 2023-08-25T09:48:20.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.307+0000 7f11fbfff640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11e006a080 0x7f11e006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:20.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.307+0000 7f11fbfff640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11e006a080 0x7f11e006c540 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009b20 tx=0x7f11ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:20.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.311+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f11e40752c0 con 0x7f11fc14f370 2023-08-25T09:48:20.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.531+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11fc1f2470 con 0x7f11fc14f370 2023-08-25T09:48:20.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.532+0000 7f11f97fa640 1 -- 172.21.15.70:0/970758239 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f11e40788e0 con 0x7f11fc14f370 2023-08-25T09:48:20.534 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:48:20.534 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:48:20.535 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:48:20.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11e006a080 msgr2=0x7f11e006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11e006a080 0x7f11e006c540 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009b20 tx=0x7f11ec008040 comp rx=0 tx=0).stop 2023-08-25T09:48:20.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 msgr2=0x7f11fc0ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:20.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 0x7f11fc0ac880 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c240 tx=0x7f11e402e070 comp rx=0 tx=0).stop 2023-08-25T09:48:20.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 shutdown_connections 2023-08-25T09:48:20.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f11e006a080 0x7f11e006c540 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11fc0acdc0 0x7f11fc06eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f11fc14f370 0x7f11fc0ac880 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 --2- 172.21.15.70:0/970758239 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f11dc008850 0x7f11fc1f1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 >> 172.21.15.70:0/970758239 conn(0x7f11fc0affd0 msgr2=0x7f11fc0aded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 shutdown_connections 2023-08-25T09:48:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.535+0000 7f12013eb640 1 -- 172.21.15.70:0/970758239 wait complete. 2023-08-25T09:48:20.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-08-25T09:48:20.783 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":8718,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.938021644999999,"mdsmap_epoch":74,"osdmap_epoch":165,"osdmap_epoch_barrier":165,"uptime":17.334901170999998} 2023-08-25T09:48:20.784 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 8718, "mdsmap_epoch": 74, "osdmap_epoch": 165, "osdmap_epoch_barrier": 165, "rank_uptime": 11.938021645, "state": "up:active", "uptime": 17.334901171, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:48:20.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-08-25T09:48:21.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.998+0000 7f494bbcf640 1 Processor -- start 2023-08-25T09:48:21.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:20.999+0000 7f494bbcf640 1 -- start start 2023-08-25T09:48:21.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.000+0000 7f494bbcf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f49440cda60 0x7f49440ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.000+0000 7f494bbcf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494415e6f0 con 0x7f4944155120 2023-08-25T09:48:21.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.000+0000 7f494bbcf640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f494415e9d0 con 0x7f49440ca820 2023-08-25T09:48:21.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.000+0000 7f494bbcf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49440caf30 con 0x7f49440cda60 2023-08-25T09:48:21.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.000+0000 7f494996b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4944155120 0x7f49440cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:57210/0 (socket says 172.21.15.70:57210) 2023-08-25T09:48:21.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.000+0000 7f494996b640 1 -- 172.21.15.70:0/1663558092 learned_addr learned my addr 172.21.15.70:0/1663558092 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:48:21.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439758197 0 0) 0x7f494415e6f0 con 0x7f4944155120 2023-08-25T09:48:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f492c003520 con 0x7f4944155120 2023-08-25T09:48:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f494916a640 1 --2- 172.21.15.70:0/1663558092 >> v2:172.21.15.155:3300/0 conn(0x7f49440cda60 0x7f49440ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2047284769 0 0) 0x7f492c003520 con 0x7f4944155120 2023-08-25T09:48:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 >> v2:172.21.15.155:3300/0 conn(0x7f49440cda60 msgr2=0x7f49440ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 --2- 172.21.15.70:0/1663558092 >> v2:172.21.15.155:3300/0 conn(0x7f49440cda60 0x7f49440ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 >> v1:172.21.15.155:6789/0 conn(0x7f49440ca820 legacy=0x7f494415af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494415ecb0 con 0x7f4944155120 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4934001670 con 0x7f4944155120 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 >> v1:172.21.15.70:6789/0 conn(0x7f4944155120 legacy=0x7f49440cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c003970 0x7f492c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c0047c0 0x7f492c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f492c005180 0x7f492c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00c080 con 0x7f492c003970 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00c1f0 con 0x7f492c005180 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.001+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c00c3b0 con 0x7f492c0047c0 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494996b640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c003970 0x7f492c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494996b640 1 -- 172.21.15.70:0/1663558092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c0047c0 msgr2=0x7f492c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494996b640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c0047c0 0x7f492c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494996b640 1 -- 172.21.15.70:0/1663558092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f492c005180 msgr2=0x7f492c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494996b640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f492c005180 0x7f492c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494996b640 1 -- 172.21.15.70:0/1663558092 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c003520 con 0x7f492c003970 2023-08-25T09:48:21.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.002+0000 7f494a16c640 1 -- 172.21.15.70:0/1663558092 reap_dead start 2023-08-25T09:48:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.003+0000 7f494996b640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c003970 0x7f492c003d70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4934001eb0 tx=0x7f492c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6479398713b92d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.003+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934005ad0 con 0x7f492c003970 2023-08-25T09:48:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.003+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934005c50 con 0x7f492c003970 2023-08-25T09:48:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.003+0000 7f4948969640 1 -- 172.21.15.70:0/1663558092 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493402c8b0 con 0x7f492c003970 2023-08-25T09:48:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 -- 172.21.15.70:0/1663558092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c003970 msgr2=0x7f492c003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c003970 0x7f492c003d70 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4934001eb0 tx=0x7f492c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 -- 172.21.15.70:0/1663558092 shutdown_connections 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 --2- 172.21.15.70:0/1663558092 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f492c003970 0x7f492c003d70 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 -- 172.21.15.70:0/1663558092 >> 172.21.15.70:0/1663558092 conn(0x7f4944069c70 msgr2=0x7f494406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 -- 172.21.15.70:0/1663558092 shutdown_connections 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.004+0000 7f494bbcf640 1 -- 172.21.15.70:0/1663558092 wait complete. 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.006+0000 7f494bbcf640 1 Processor -- start 2023-08-25T09:48:21.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.006+0000 7f494bbcf640 1 -- start start 2023-08-25T09:48:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494bbcf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c003970 0x7f49441f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494bbcf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 0x7f49440b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494bbcf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49441431e0 0x7f4944143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494bbcf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4944140e10 con 0x7f49440b1fe0 2023-08-25T09:48:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494bbcf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49441f5ff0 con 0x7f49441431e0 2023-08-25T09:48:21.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494bbcf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49440b2980 con 0x7f492c003970 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494916a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 0x7f49440b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494916a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 0x7f49440b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33894/0 (socket says 172.21.15.70:33894) 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.007+0000 7f494916a640 1 -- 172.21.15.70:0/3764315118 learned_addr learned my addr 172.21.15.70:0/3764315118 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494996b640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c003970 0x7f49441f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494916a640 1 -- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c003970 msgr2=0x7f49441f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494916a640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c003970 0x7f49441f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494916a640 1 -- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49441431e0 msgr2=0x7f4944143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494916a640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49441431e0 0x7f4944143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494916a640 1 -- 172.21.15.70:0/3764315118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49441408f0 con 0x7f49440b1fe0 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494916a640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 0x7f49440b2440 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f493800d590 tx=0x7f493800da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380125f0 con 0x7f49440b1fe0 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4938012750 con 0x7f49440b1fe0 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938012b00 con 0x7f49440b1fe0 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.008+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4944143ba0 con 0x7f49440b1fe0 2023-08-25T09:48:21.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.009+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49441f6500 con 0x7f49440b1fe0 2023-08-25T09:48:21.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.010+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4944140f50 con 0x7f49440b1fe0 2023-08-25T09:48:21.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.011+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4938012c60 con 0x7f49440b1fe0 2023-08-25T09:48:21.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.011+0000 7f49427fc640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:48:21.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.012+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f49380888e0 con 0x7f49440b1fe0 2023-08-25T09:48:21.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.012+0000 7f494996b640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:48:21.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.012+0000 7f494996b640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4934034000 tx=0x7f4934009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:48:21.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.015+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4944140f50 con 0x7f49440b1fe0 2023-08-25T09:48:21.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.209+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f4944140f50 con 0x7f49440b1fe0 2023-08-25T09:48:21.584 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.583+0000 7f49427fc640 1 -- 172.21.15.70:0/3764315118 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v166) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4944140f50 con 0x7f49440b1fe0 2023-08-25T09:48:21.585 INFO:teuthology.orchestra.run.smithi070.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-08-25T09:48:21.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.586+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 msgr2=0x7f493006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:21.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.586+0000 7f494bbcf640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4934034000 tx=0x7f4934009030 comp rx=0 tx=0).stop 2023-08-25T09:48:21.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.586+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 msgr2=0x7f49440b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:48:21.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.586+0000 7f494bbcf640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 0x7f49440b2440 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f493800d590 tx=0x7f493800da50 comp rx=0 tx=0).stop 2023-08-25T09:48:21.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 shutdown_connections 2023-08-25T09:48:21.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f493006a050 0x7f493006c510 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f49441431e0 0x7f4944143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f49440b1fe0 0x7f49440b2440 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 --2- 172.21.15.70:0/3764315118 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f492c003970 0x7f49441f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:48:21.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 >> 172.21.15.70:0/3764315118 conn(0x7f4944069c70 msgr2=0x7f494406ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:48:21.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 shutdown_connections 2023-08-25T09:48:21.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:48:21.587+0000 7f494bbcf640 1 -- 172.21.15.70:0/3764315118 wait complete. 2023-08-25T09:48:21.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:48:21.843 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:48:21.843 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:48:21.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-08-25T09:48:22.065 INFO:teuthology.orchestra.run.smithi070.stdout:{"mon_tick_interval":"5"} 2023-08-25T09:48:22.066 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-08-25T09:48:22.066 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-08-25T09:48:22.067 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:48:22.067 DEBUG:teuthology.orchestra.run.smithi070:> import time 2023-08-25T09:48:22.067 DEBUG:teuthology.orchestra.run.smithi070:> import datetime 2023-08-25T09:48:22.068 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:48:22.068 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:48:22.069 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.069 DEBUG:teuthology.orchestra.run.smithi070:> # Write some buffered data through before going full, all should be well 2023-08-25T09:48:22.069 DEBUG:teuthology.orchestra.run.smithi070:> print("writing some data through which we expect to succeed") 2023-08-25T09:48:22.070 DEBUG:teuthology.orchestra.run.smithi070:> bytes = 0 2023-08-25T09:48:22.070 DEBUG:teuthology.orchestra.run.smithi070:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-08-25T09:48:22.070 DEBUG:teuthology.orchestra.run.smithi070:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-08-25T09:48:22.071 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-08-25T09:48:22.071 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:48:22.071 DEBUG:teuthology.orchestra.run.smithi070:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-08-25T09:48:22.072 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.072 DEBUG:teuthology.orchestra.run.smithi070:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-08-25T09:48:22.073 DEBUG:teuthology.orchestra.run.smithi070:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-08-25T09:48:22.073 DEBUG:teuthology.orchestra.run.smithi070:> # buffered IO, the error should always only appear from fsync and not 2023-08-25T09:48:22.073 DEBUG:teuthology.orchestra.run.smithi070:> # from write 2023-08-25T09:48:22.073 DEBUG:teuthology.orchestra.run.smithi070:> full = False 2023-08-25T09:48:22.074 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.074 DEBUG:teuthology.orchestra.run.smithi070:> for n in range(0, int(32 * 0.9)): 2023-08-25T09:48:22.074 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-08-25T09:48:22.074 DEBUG:teuthology.orchestra.run.smithi070:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-08-25T09:48:22.075 DEBUG:teuthology.orchestra.run.smithi070:> print("done writing {0} bytes".format(bytes)) 2023-08-25T09:48:22.075 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.075 DEBUG:teuthology.orchestra.run.smithi070:> # OK, now we should sneak in under the full condition 2023-08-25T09:48:22.076 DEBUG:teuthology.orchestra.run.smithi070:> # due to the time it takes the OSDs to report to the 2023-08-25T09:48:22.076 DEBUG:teuthology.orchestra.run.smithi070:> # mons, and get a successful fsync on our full-making data 2023-08-25T09:48:22.077 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:48:22.077 DEBUG:teuthology.orchestra.run.smithi070:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-08-25T09:48:22.078 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.078 DEBUG:teuthology.orchestra.run.smithi070:> # buffered write, add more dirty data to the buffer 2023-08-25T09:48:22.079 DEBUG:teuthology.orchestra.run.smithi070:> print("starting buffered write") 2023-08-25T09:48:22.079 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:48:22.079 DEBUG:teuthology.orchestra.run.smithi070:> for n in range(0, int(32 * 0.2)): 2023-08-25T09:48:22.080 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-08-25T09:48:22.080 DEBUG:teuthology.orchestra.run.smithi070:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-08-25T09:48:22.080 DEBUG:teuthology.orchestra.run.smithi070:> time.sleep(15.0) 2023-08-25T09:48:22.081 DEBUG:teuthology.orchestra.run.smithi070:> except OSError: 2023-08-25T09:48:22.081 DEBUG:teuthology.orchestra.run.smithi070:> pass; 2023-08-25T09:48:22.081 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.082 DEBUG:teuthology.orchestra.run.smithi070:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-08-25T09:48:22.083 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.083 DEBUG:teuthology.orchestra.run.smithi070:> # Wait long enough for a background flush that should fail 2023-08-25T09:48:22.084 DEBUG:teuthology.orchestra.run.smithi070:> time.sleep(30) 2023-08-25T09:48:22.084 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.084 DEBUG:teuthology.orchestra.run.smithi070:> if True: 2023-08-25T09:48:22.085 DEBUG:teuthology.orchestra.run.smithi070:> # ...and check that the failed background flush is reflected in fclose 2023-08-25T09:48:22.085 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:48:22.086 DEBUG:teuthology.orchestra.run.smithi070:> os.close(f) 2023-08-25T09:48:22.086 DEBUG:teuthology.orchestra.run.smithi070:> except OSError: 2023-08-25T09:48:22.086 DEBUG:teuthology.orchestra.run.smithi070:> print("close() returned an error as expected") 2023-08-25T09:48:22.087 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:48:22.087 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("close() failed to raise error") 2023-08-25T09:48:22.087 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:48:22.088 DEBUG:teuthology.orchestra.run.smithi070:> # The kernel cephfs client does not raise errors on fclose 2023-08-25T09:48:22.088 DEBUG:teuthology.orchestra.run.smithi070:> os.close(f) 2023-08-25T09:48:22.089 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:48:22.089 DEBUG:teuthology.orchestra.run.smithi070:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-08-25T09:48:22.089 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:48:46.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:48:46.818 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:49:16.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:49:16.975 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:49:47.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:49:47.119 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:50:08.949 INFO:teuthology.orchestra.run.smithi070.stdout:writing some data through which we expect to succeed 2023-08-25T09:50:08.949 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-08-25T09:50:08.949 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-08-25T09:50:08.949 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-08-25T09:50:08.950 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-08-25T09:50:08.951 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-08-25T09:50:08.952 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:done writing 29884416 bytes 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:successfully fsync'ed prior to getting full state reported 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:starting buffered write 2023-08-25T09:50:08.953 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:50:08.954 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:50:08.954 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:50:08.954 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:50:08.954 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:50:08.954 INFO:teuthology.orchestra.run.smithi070.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-08-25T09:50:08.954 INFO:teuthology.orchestra.run.smithi070.stdout:close() returned an error as expected 2023-08-25T09:50:08.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:50:08.982 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:50:08.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:50:09.107 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:50:09.235 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:50:09.236 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:50:09.236 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:09.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:09.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.520+0000 7f4b210d7640 1 Processor -- start 2023-08-25T09:50:09.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.521+0000 7f4b210d7640 1 -- start start 2023-08-25T09:50:09.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.522+0000 7f4b210d7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c15e830 0x7f4b1c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.522+0000 7f4b210d7640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4b1c06b2b0 0x7f4b1c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.522+0000 7f4b210d7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c0b1b60 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.522+0000 7f4b210d7640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b1c06bd80 con 0x7f4b1c06b2b0 2023-08-25T09:50:09.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.522+0000 7f4b210d7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c06b7c0 con 0x7f4b1c15e830 2023-08-25T09:50:09.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1b59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4b1c15c6d0 0x7f4b1c0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59808/0 (socket says 172.21.15.70:59808) 2023-08-25T09:50:09.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1b59e640 1 -- 172.21.15.70:0/4229970558 learned_addr learned my addr 172.21.15.70:0/4229970558 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1ad9d640 1 --2- 172.21.15.70:0/4229970558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c15e830 0x7f4b1c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1a59c640 1 --2- 172.21.15.70:0/4229970558 >> v2:172.21.15.155:3300/0 conn(0x7f4b1c06b2b0 0x7f4b1c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b19d9b640 1 -- 172.21.15.70:0/4229970558 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704192871 0 0) 0x7f4b1c0b1b60 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b19d9b640 1 -- 172.21.15.70:0/4229970558 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b08003520 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1ad9d640 1 -- 172.21.15.70:0/4229970558 >> v2:172.21.15.155:3300/0 conn(0x7f4b1c06b2b0 msgr2=0x7f4b1c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1ad9d640 1 --2- 172.21.15.70:0/4229970558 >> v2:172.21.15.155:3300/0 conn(0x7f4b1c06b2b0 0x7f4b1c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1ad9d640 1 -- 172.21.15.70:0/4229970558 >> v1:172.21.15.70:6789/0 conn(0x7f4b1c15c6d0 legacy=0x7f4b1c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.523+0000 7f4b1ad9d640 1 -- 172.21.15.70:0/4229970558 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c06baa0 con 0x7f4b1c15e830 2023-08-25T09:50:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.525+0000 7f4b1ad9d640 1 --2- 172.21.15.70:0/4229970558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c15e830 0x7f4b1c06ad70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097a0 tx=0x7f4b0c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9cfa0b1757b019 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.525+0000 7f4b19d9b640 1 -- 172.21.15.70:0/4229970558 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c02d040 con 0x7f4b1c15e830 2023-08-25T09:50:09.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.525+0000 7f4b19d9b640 1 -- 172.21.15.70:0/4229970558 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0c031810 con 0x7f4b1c15e830 2023-08-25T09:50:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.525+0000 7f4b19d9b640 1 -- 172.21.15.70:0/4229970558 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0c031ba0 con 0x7f4b1c15e830 2023-08-25T09:50:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 -- 172.21.15.70:0/4229970558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c15e830 msgr2=0x7f4b1c06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 --2- 172.21.15.70:0/4229970558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c15e830 0x7f4b1c06ad70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c0097a0 tx=0x7f4b0c02b8b0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 -- 172.21.15.70:0/4229970558 shutdown_connections 2023-08-25T09:50:09.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 --2- 172.21.15.70:0/4229970558 >> v2:172.21.15.155:3300/0 conn(0x7f4b1c06b2b0 0x7f4b1c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 --2- 172.21.15.70:0/4229970558 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c15e830 0x7f4b1c06ad70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 -- 172.21.15.70:0/4229970558 >> 172.21.15.70:0/4229970558 conn(0x7f4b1c069850 msgr2=0x7f4b1c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 -- 172.21.15.70:0/4229970558 shutdown_connections 2023-08-25T09:50:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.526+0000 7f4b210d7640 1 -- 172.21.15.70:0/4229970558 wait complete. 2023-08-25T09:50:09.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.528+0000 7f4b210d7640 1 Processor -- start 2023-08-25T09:50:09.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.528+0000 7f4b210d7640 1 -- start start 2023-08-25T09:50:09.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b210d7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c06b2b0 0x7f4b1c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b210d7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 0x7f4b1c1f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b210d7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c15e830 0x7f4b1c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b210d7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c1f1fb0 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b210d7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0c9d20 con 0x7f4b1c15e830 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b210d7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0a8820 con 0x7f4b1c06b2b0 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 0x7f4b1c1f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 0x7f4b1c1f1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39872/0 (socket says 172.21.15.70:39872) 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1a59c640 1 -- 172.21.15.70:0/2857598939 learned_addr learned my addr 172.21.15.70:0/2857598939 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1ad9d640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c06b2b0 0x7f4b1c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1a59c640 1 -- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c06b2b0 msgr2=0x7f4b1c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1a59c640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c06b2b0 0x7f4b1c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1b59e640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c15e830 0x7f4b1c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.529+0000 7f4b1a59c640 1 -- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c15e830 msgr2=0x7f4b1c0b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b1a59c640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c15e830 0x7f4b1c0b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b1a59c640 1 -- 172.21.15.70:0/2857598939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c009400 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b1b59e640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c15e830 0x7f4b1c0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b1a59c640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 0x7f4b1c1f1a70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400efc0 tx=0x7f4b0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04009be0 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0400f040 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04014680 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c0a8b00 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.530+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c0a90c0 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.531+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c06f420 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.532+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4b040147e0 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.533+0000 7f4b137fe640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4afc06a050 0x7f4afc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.533+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f4b040894d0 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.534+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f4afc072380 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.534+0000 7f4b1ad9d640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4afc06a050 0x7f4afc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.534+0000 7f4b1ad9d640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4afc06a050 0x7f4afc06c510 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c036000 tx=0x7f4b0c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:09.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.538+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b04058520 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.728+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b1c04e420 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.731+0000 7f4b137fe640 1 -- 172.21.15.70:0/2857598939 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+18637 (secure 0 0 0) 0x7f4b040580b0 con 0x7f4b1c15c6d0 2023-08-25T09:50:09.734 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:09.734 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":169,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:49:22.433958+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"20.13","mappings":[{"from":3,"to":7}]},{"pgid":"20.1d","mappings":[{"from":3,"to":1}]},{"pgid":"20.1e","mappings":[{"from":3,"to":7}]},{"pgid":"20.38","mappings":[{"from":3,"to":7}]},{"pgid":"20.3c","mappings":[{"from":3,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:09.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4afc06a050 msgr2=0x7f4afc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4afc06a050 0x7f4afc06c510 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f4b0c036000 tx=0x7f4b0c03c040 comp rx=0 tx=0).stop 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 msgr2=0x7f4b1c1f1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 0x7f4b1c1f1a70 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400efc0 tx=0x7f4b0400c490 comp rx=0 tx=0).stop 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 shutdown_connections 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4afc06a050 0x7f4afc06c510 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b1c15e830 0x7f4b1c0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b1c15c6d0 0x7f4b1c1f1a70 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 --2- 172.21.15.70:0/2857598939 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b1c06b2b0 0x7f4b1c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 >> 172.21.15.70:0/2857598939 conn(0x7f4b1c069850 msgr2=0x7f4b1c06ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 shutdown_connections 2023-08-25T09:50:09.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.734+0000 7f4b210d7640 1 -- 172.21.15.70:0/2857598939 wait complete. 2023-08-25T09:50:09.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:48:06.570383+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:48:07.289165+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:50:09.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:50:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.974+0000 7f22cecd3640 1 Processor -- start 2023-08-25T09:50:09.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.974+0000 7f22cecd3640 1 -- start start 2023-08-25T09:50:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22cecd3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f22c806cdd0 0x7f22c80ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22cecd3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f22c80cd730 0x7f22c80cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22cecd3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c80ade30 con 0x7f22c8159fb0 2023-08-25T09:50:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22cecd3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80c94b0 con 0x7f22c806cdd0 2023-08-25T09:50:09.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22cecd3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80cc550 con 0x7f22c80cd730 2023-08-25T09:50:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f22c806cdd0 0x7f22c80ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22c7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f22c806cdd0 0x7f22c80ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39884/0 (socket says 172.21.15.70:39884) 2023-08-25T09:50:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.976+0000 7f22c7fff640 1 -- 172.21.15.70:0/2242560557 learned_addr learned my addr 172.21.15.70:0/2242560557 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.977+0000 7f22cd270640 1 --2- 172.21.15.70:0/2242560557 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cd730 0x7f22c80cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.977+0000 7f22c7fff640 1 -- 172.21.15.70:0/2242560557 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cd730 msgr2=0x7f22c80cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.977+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2242560557 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cd730 0x7f22c80cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.977+0000 7f22c7fff640 1 -- 172.21.15.70:0/2242560557 >> v1:172.21.15.155:6789/0 conn(0x7f22c8159fb0 legacy=0x7f22c80ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.977+0000 7f22c7fff640 1 -- 172.21.15.70:0/2242560557 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80c8c10 con 0x7f22c806cdd0 2023-08-25T09:50:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.978+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2242560557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c806cdd0 0x7f22c80ad8f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cc140 tx=0x7f22b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5850801b38d47ba3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.978+0000 7f22c77fe640 1 -- 172.21.15.70:0/2242560557 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40317e0 con 0x7f22c806cdd0 2023-08-25T09:50:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.978+0000 7f22c77fe640 1 -- 172.21.15.70:0/2242560557 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b4031940 con 0x7f22c806cdd0 2023-08-25T09:50:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.978+0000 7f22c77fe640 1 -- 172.21.15.70:0/2242560557 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4031cd0 con 0x7f22c806cdd0 2023-08-25T09:50:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 -- 172.21.15.70:0/2242560557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c806cdd0 msgr2=0x7f22c80ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 --2- 172.21.15.70:0/2242560557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c806cdd0 0x7f22c80ad8f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cc140 tx=0x7f22b402b880 comp rx=0 tx=0).stop 2023-08-25T09:50:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 -- 172.21.15.70:0/2242560557 shutdown_connections 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 --2- 172.21.15.70:0/2242560557 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cd730 0x7f22c80cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 --2- 172.21.15.70:0/2242560557 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c806cdd0 0x7f22c80ad8f0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 -- 172.21.15.70:0/2242560557 >> 172.21.15.70:0/2242560557 conn(0x7f22c80cb0d0 msgr2=0x7f22c80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 -- 172.21.15.70:0/2242560557 shutdown_connections 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.979+0000 7f22cecd3640 1 -- 172.21.15.70:0/2242560557 wait complete. 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.981+0000 7f22cecd3640 1 Processor -- start 2023-08-25T09:50:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.981+0000 7f22cecd3640 1 -- start start 2023-08-25T09:50:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cecd3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c806cdd0 0x7f22c815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cecd3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80cd730 0x7f22c815aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cecd3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 0x7f22c81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cecd3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c815da30 con 0x7f22c8159fb0 2023-08-25T09:50:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cecd3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c815af70 con 0x7f22c80cd730 2023-08-25T09:50:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cecd3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c815ba20 con 0x7f22c806cdd0 2023-08-25T09:50:09.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cd270640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 0x7f22c81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cd270640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 0x7f22c81555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39892/0 (socket says 172.21.15.70:39892) 2023-08-25T09:50:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.982+0000 7f22cd270640 1 -- 172.21.15.70:0/1985608310 learned_addr learned my addr 172.21.15.70:0/1985608310 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cd270640 1 -- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c806cdd0 msgr2=0x7f22c815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22c7fff640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80cd730 0x7f22c815aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cca6f640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c806cdd0 0x7f22c815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cd270640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c806cdd0 0x7f22c815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cd270640 1 -- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80cd730 msgr2=0x7f22c815aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cd270640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80cd730 0x7f22c815aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cd270640 1 -- 172.21.15.70:0/1985608310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40094d0 con 0x7f22c8159fb0 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cca6f640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c806cdd0 0x7f22c815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22cd270640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 0x7f22c81555d0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f22c0007e40 tx=0x7f22c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.983+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0014620 con 0x7f22c8159fb0 2023-08-25T09:50:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.984+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c000ee90 con 0x7f22c8159fb0 2023-08-25T09:50:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.984+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00149e0 con 0x7f22c8159fb0 2023-08-25T09:50:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.984+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c815b180 con 0x7f22c8159fb0 2023-08-25T09:50:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.984+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c815b740 con 0x7f22c8159fb0 2023-08-25T09:50:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.985+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c80c8a60 con 0x7f22c8159fb0 2023-08-25T09:50:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.986+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f22c0014b40 con 0x7f22c8159fb0 2023-08-25T09:50:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.987+0000 7f22c57fa640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22ac06a050 0x7f22ac06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.987+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f22c0088ff0 con 0x7f22c8159fb0 2023-08-25T09:50:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.988+0000 7f22cca6f640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22ac06a050 0x7f22ac06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.988+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f22ac072380 con 0x7f22c8159fb0 2023-08-25T09:50:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.988+0000 7f22cca6f640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22ac06a050 0x7f22ac06c510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22bc003eb0 tx=0x7f22bc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:09.992+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22c00bb990 con 0x7f22c8159fb0 2023-08-25T09:50:10.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.181+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22c8066fb0 con 0x7f22c8159fb0 2023-08-25T09:50:10.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.181+0000 7f22c57fa640 1 -- 172.21.15.70:0/1985608310 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22c0058110 con 0x7f22c8159fb0 2023-08-25T09:50:10.183 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:50:10.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.183+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22ac06a050 msgr2=0x7f22ac06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.183+0000 7f22cecd3640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22ac06a050 0x7f22ac06c510 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f22bc003eb0 tx=0x7f22bc009040 comp rx=0 tx=0).stop 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.183+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 msgr2=0x7f22c81555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.183+0000 7f22cecd3640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 0x7f22c81555d0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f22c0007e40 tx=0x7f22c000e4b0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 shutdown_connections 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22ac06a050 0x7f22ac06c510 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22c8159fb0 0x7f22c81555d0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80cd730 0x7f22c815aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 --2- 172.21.15.70:0/1985608310 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c806cdd0 0x7f22c815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 >> 172.21.15.70:0/1985608310 conn(0x7f22c80cb0d0 msgr2=0x7f22c81580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 shutdown_connections 2023-08-25T09:50:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.184+0000 7f22cecd3640 1 -- 172.21.15.70:0/1985608310 wait complete. 2023-08-25T09:50:10.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:50:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.424+0000 7f17fdeb8640 1 Processor -- start 2023-08-25T09:50:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.424+0000 7f17fdeb8640 1 -- start start 2023-08-25T09:50:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17fdeb8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f80c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17fdeb8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17f80cdc20 0x7f17f8069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17fdeb8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f80cc5f0 con 0x7f17f80ce060 2023-08-25T09:50:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17fdeb8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f8070650 con 0x7f17f80cdc20 2023-08-25T09:50:10.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17fdeb8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8070930 con 0x7f17f815ae50 2023-08-25T09:50:10.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17f6ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17f80cdc20 0x7f17f8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.426+0000 7f17f77fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f80c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f6ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f17f80cdc20 0x7f17f8069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:33122/0 (socket says 172.21.15.70:33122) 2023-08-25T09:50:10.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f6ffd640 1 -- 172.21.15.70:0/2672305790 learned_addr learned my addr 172.21.15.70:0/2672305790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:10.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f77fe640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f80c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:36418/0 (socket says 172.21.15.70:36418) 2023-08-25T09:50:10.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f6ffd640 1 -- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 msgr2=0x7f17f80c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:10.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f6ffd640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f80c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f6ffd640 1 -- 172.21.15.70:0/2672305790 >> v1:172.21.15.155:6789/0 conn(0x7f17f80ce060 legacy=0x7f17f80ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f6ffd640 1 -- 172.21.15.70:0/2672305790 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f8070c10 con 0x7f17f80cdc20 2023-08-25T09:50:10.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.427+0000 7f17f77fe640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f80c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:50:10.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.429+0000 7f17f6ffd640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f8069850 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009870 tx=0x7f17ec02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2e0dc3405b1abd4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.429+0000 7f17f67fc640 1 -- 172.21.15.70:0/2672305790 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec02f830 con 0x7f17f80cdc20 2023-08-25T09:50:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.429+0000 7f17f67fc640 1 -- 172.21.15.70:0/2672305790 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec02f990 con 0x7f17f80cdc20 2023-08-25T09:50:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.429+0000 7f17f67fc640 1 -- 172.21.15.70:0/2672305790 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec02fd20 con 0x7f17f80cdc20 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 -- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 msgr2=0x7f17f8069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f8069850 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009870 tx=0x7f17ec02ba10 comp rx=0 tx=0).stop 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 -- 172.21.15.70:0/2672305790 shutdown_connections 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f8069850 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/2672305790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f80c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 -- 172.21.15.70:0/2672305790 >> 172.21.15.70:0/2672305790 conn(0x7f17f80cb0d0 msgr2=0x7f17f80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 -- 172.21.15.70:0/2672305790 shutdown_connections 2023-08-25T09:50:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.430+0000 7f17fdeb8640 1 -- 172.21.15.70:0/2672305790 wait complete. 2023-08-25T09:50:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.432+0000 7f17fdeb8640 1 Processor -- start 2023-08-25T09:50:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.432+0000 7f17fdeb8640 1 -- start start 2023-08-25T09:50:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17fdeb8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f81f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17fdeb8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 0x7f17f80c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17fdeb8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f8158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17fdeb8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f81f6700 con 0x7f17f80ce060 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17fdeb8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f815b3b0 con 0x7f17f80cdc20 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17fdeb8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f815b600 con 0x7f17f815ae50 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17f6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 0x7f17f80c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17f6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 0x7f17f80c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39906/0 (socket says 172.21.15.70:39906) 2023-08-25T09:50:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.433+0000 7f17f6ffd640 1 -- 172.21.15.70:0/1885766156 learned_addr learned my addr 172.21.15.70:0/1885766156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f7fff640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f8158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f77fe640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f81f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f6ffd640 1 -- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 msgr2=0x7f17f8158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f6ffd640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f8158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f6ffd640 1 -- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 msgr2=0x7f17f81f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f6ffd640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f81f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f6ffd640 1 -- 172.21.15.70:0/1885766156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8009400 con 0x7f17f80ce060 2023-08-25T09:50:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f77fe640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f81f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17f6ffd640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 0x7f17f80c5690 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f17ec000c00 tx=0x7f17ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec03b040 con 0x7f17f80ce060 2023-08-25T09:50:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec037be0 con 0x7f17f80ce060 2023-08-25T09:50:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17ec0094d0 con 0x7f17f80ce060 2023-08-25T09:50:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.434+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec042480 con 0x7f17f80ce060 2023-08-25T09:50:10.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.435+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f815bbc0 con 0x7f17f80ce060 2023-08-25T09:50:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.436+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f8066fb0 con 0x7f17f80ce060 2023-08-25T09:50:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.437+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17ec0425e0 con 0x7f17f80ce060 2023-08-25T09:50:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.438+0000 7f17dffff640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17e406a050 0x7f17e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.438+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f17ec0ab7a0 con 0x7f17f80ce060 2023-08-25T09:50:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.438+0000 7f17f77fe640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17e406a050 0x7f17e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.438+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f17e4072380 con 0x7f17f80ce060 2023-08-25T09:50:10.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.439+0000 7f17f77fe640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17e406a050 0x7f17e406c510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009bb0 tx=0x7f17e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:10.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.442+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17ec0a9e60 con 0x7f17f80ce060 2023-08-25T09:50:10.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.629+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f17f815a320 con 0x7f17f80ce060 2023-08-25T09:50:10.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.630+0000 7f17dffff640 1 -- 172.21.15.70:0/1885766156 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v169) v1 ==== 92+0+10 (secure 0 0 0) 0x7f17ec07a770 con 0x7f17f80ce060 2023-08-25T09:50:10.631 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:50:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.632+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17e406a050 msgr2=0x7f17e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.632+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17e406a050 0x7f17e406c510 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f17e8009bb0 tx=0x7f17e8008040 comp rx=0 tx=0).stop 2023-08-25T09:50:10.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.632+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 msgr2=0x7f17f80c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.632+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 0x7f17f80c5690 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f17ec000c00 tx=0x7f17ec0046c0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 shutdown_connections 2023-08-25T09:50:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f17e406a050 0x7f17e406c510 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f17f815ae50 0x7f17f8158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f17f80ce060 0x7f17f80c5690 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 --2- 172.21.15.70:0/1885766156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f17f80cdc20 0x7f17f81f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 >> 172.21.15.70:0/1885766156 conn(0x7f17f80cb0d0 msgr2=0x7f17f80cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:10.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 shutdown_connections 2023-08-25T09:50:10.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.633+0000 7f17fdeb8640 1 -- 172.21.15.70:0/1885766156 wait complete. 2023-08-25T09:50:10.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:50:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.869+0000 7eff6e155640 1 Processor -- start 2023-08-25T09:50:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.869+0000 7eff6e155640 1 -- start start 2023-08-25T09:50:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.870+0000 7eff6e155640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff6815e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff6e155640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7eff680cd660 0x7eff680cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff6e155640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6815e780 con 0x7eff6806f0e0 2023-08-25T09:50:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff6e155640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6806f8e0 con 0x7eff680cd660 2023-08-25T09:50:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff6e155640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6806fbc0 con 0x7eff68159bf0 2023-08-25T09:50:10.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff66ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7eff6806f0e0 0x7eff68070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59818/0 (socket says 172.21.15.70:59818) 2023-08-25T09:50:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff66ffd640 1 -- 172.21.15.70:0/2943095377 learned_addr learned my addr 172.21.15.70:0/2943095377 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff67fff640 1 --2- 172.21.15.70:0/2943095377 >> v2:172.21.15.155:3300/0 conn(0x7eff680cd660 0x7eff680cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.871+0000 7eff677fe640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff6815e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155654653 0 0) 0x7eff6815e780 con 0x7eff6806f0e0 2023-08-25T09:50:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff50003520 con 0x7eff6806f0e0 2023-08-25T09:50:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1109533382 0 0) 0x7eff50003520 con 0x7eff6806f0e0 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 msgr2=0x7eff6815e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff6815e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 >> v2:172.21.15.155:3300/0 conn(0x7eff680cd660 msgr2=0x7eff680cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 --2- 172.21.15.70:0/2943095377 >> v2:172.21.15.155:3300/0 conn(0x7eff680cd660 0x7eff680cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680ce000 con 0x7eff6806f0e0 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7eff5c001670 con 0x7eff6806f0e0 2023-08-25T09:50:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 >> v1:172.21.15.70:6789/0 conn(0x7eff6806f0e0 legacy=0x7eff68070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff50003970 0x7eff50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50004d00 0x7eff50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6806f8e0 con 0x7eff50003970 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6806fbc0 con 0x7eff50004d00 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5000b980 con 0x7eff68159bf0 2023-08-25T09:50:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.872+0000 7eff677fe640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff50003970 0x7eff50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff677fe640 1 -- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 msgr2=0x7eff500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff66ffd640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff67fff640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50004d00 0x7eff50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff677fe640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff677fe640 1 -- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50004d00 msgr2=0x7eff50005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff677fe640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50004d00 0x7eff50005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff677fe640 1 -- 172.21.15.70:0/2943095377 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff50003520 con 0x7eff50003970 2023-08-25T09:50:10.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff66ffd640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff68159bf0 0x7eff500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.873+0000 7eff67fff640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50004d00 0x7eff50005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.874+0000 7eff677fe640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff50003970 0x7eff50003d90 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7eff58009ac0 tx=0x7eff500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9660c1f07b3886d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.874+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5802c6b0 con 0x7eff50003970 2023-08-25T09:50:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.874+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5802c810 con 0x7eff50003970 2023-08-25T09:50:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.874+0000 7eff667fc640 1 -- 172.21.15.70:0/2943095377 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5802cba0 con 0x7eff50003970 2023-08-25T09:50:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff6e155640 1 -- 172.21.15.70:0/2943095377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff50003970 msgr2=0x7eff50003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff6e155640 1 --2- 172.21.15.70:0/2943095377 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff50003970 0x7eff50003d90 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7eff58009ac0 tx=0x7eff500057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff67fff640 1 -- 172.21.15.70:0/2943095377 reap_dead start 2023-08-25T09:50:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff6e155640 1 -- 172.21.15.70:0/2943095377 shutdown_connections 2023-08-25T09:50:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff6e155640 1 -- 172.21.15.70:0/2943095377 >> 172.21.15.70:0/2943095377 conn(0x7eff680cb0d0 msgr2=0x7eff680cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff6e155640 1 -- 172.21.15.70:0/2943095377 shutdown_connections 2023-08-25T09:50:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.875+0000 7eff6e155640 1 -- 172.21.15.70:0/2943095377 wait complete. 2023-08-25T09:50:10.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.877+0000 7eff6e155640 1 Processor -- start 2023-08-25T09:50:10.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.877+0000 7eff6e155640 1 -- start start 2023-08-25T09:50:10.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff6e155640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50003970 0x7eff68153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff6e155640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 0x7eff680ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff6e155640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff680acdc0 0x7eff680aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff6e155640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff680ad240 con 0x7eff680ae5a0 2023-08-25T09:50:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff6e155640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff681f2170 con 0x7eff50003970 2023-08-25T09:50:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff6e155640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff680af970 con 0x7eff680acdc0 2023-08-25T09:50:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 0x7eff680ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 0x7eff680ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39930/0 (socket says 172.21.15.70:39930) 2023-08-25T09:50:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.878+0000 7eff66ffd640 1 -- 172.21.15.70:0/595151986 learned_addr learned my addr 172.21.15.70:0/595151986 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff677fe640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50003970 0x7eff68153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff66ffd640 1 -- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff680acdc0 msgr2=0x7eff680aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff66ffd640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff680acdc0 0x7eff680aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff66ffd640 1 -- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50003970 msgr2=0x7eff68153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:10.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff66ffd640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50003970 0x7eff68153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:10.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff66ffd640 1 -- 172.21.15.70:0/595151986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680adac0 con 0x7eff680ae5a0 2023-08-25T09:50:10.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff66ffd640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 0x7eff680ac880 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00d810 tx=0x7eff5c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:10.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0074e0 con 0x7eff680ae5a0 2023-08-25T09:50:10.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c007640 con 0x7eff680ae5a0 2023-08-25T09:50:10.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c0049e0 con 0x7eff680ae5a0 2023-08-25T09:50:10.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff680adda0 con 0x7eff680ae5a0 2023-08-25T09:50:10.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.879+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff680afc50 con 0x7eff680ae5a0 2023-08-25T09:50:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.880+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff68066fb0 con 0x7eff680ae5a0 2023-08-25T09:50:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.881+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7eff5c004b40 con 0x7eff680ae5a0 2023-08-25T09:50:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.882+0000 7eff4ffff640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5406a080 0x7eff5406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.882+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7eff5c087f20 con 0x7eff680ae5a0 2023-08-25T09:50:10.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.882+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7eff540723b0 con 0x7eff680ae5a0 2023-08-25T09:50:10.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.882+0000 7eff677fe640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5406a080 0x7eff5406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:10.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.883+0000 7eff677fe640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5406a080 0x7eff5406c540 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7eff58036000 tx=0x7eff58030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:10.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:10.886+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7eff5c019020 con 0x7eff680ae5a0 2023-08-25T09:50:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.074+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff680c4040 con 0x7eff680ae5a0 2023-08-25T09:50:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.075+0000 7eff4ffff640 1 -- 172.21.15.70:0/595151986 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff5c053570 con 0x7eff680ae5a0 2023-08-25T09:50:11.077 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:50:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.077+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5406a080 msgr2=0x7eff5406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.077+0000 7eff6e155640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5406a080 0x7eff5406c540 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7eff58036000 tx=0x7eff58030000 comp rx=0 tx=0).stop 2023-08-25T09:50:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.077+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 msgr2=0x7eff680ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.077+0000 7eff6e155640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 0x7eff680ac880 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00d810 tx=0x7eff5c00dcd0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 shutdown_connections 2023-08-25T09:50:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7eff5406a080 0x7eff5406c540 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7eff680acdc0 0x7eff680aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7eff680ae5a0 0x7eff680ac880 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 --2- 172.21.15.70:0/595151986 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7eff50003970 0x7eff68153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 >> 172.21.15.70:0/595151986 conn(0x7eff680cb0d0 msgr2=0x7eff680c20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 shutdown_connections 2023-08-25T09:50:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.078+0000 7eff6e155640 1 -- 172.21.15.70:0/595151986 wait complete. 2023-08-25T09:50:11.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:50:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.303+0000 7fba1a0f3640 1 Processor -- start 2023-08-25T09:50:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.303+0000 7fba1a0f3640 1 -- start start 2023-08-25T09:50:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba1a0f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fba14165f00 0x7fba141682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba1a0f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba140ac880 con 0x7fba14069850 2023-08-25T09:50:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba1a0f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba140ad400 con 0x7fba1415e210 2023-08-25T09:50:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba1a0f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1415e650 con 0x7fba14165f00 2023-08-25T09:50:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba137fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fba14069850 0x7fba140c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59830/0 (socket says 172.21.15.70:59830) 2023-08-25T09:50:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba137fe640 1 -- 172.21.15.70:0/1503533292 learned_addr learned my addr 172.21.15.70:0/1503533292 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba13fff640 1 --2- 172.21.15.70:0/1503533292 >> v2:172.21.15.155:3300/0 conn(0x7fba14165f00 0x7fba141682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860404659 0 0) 0x7fba140ac880 con 0x7fba14069850 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.305+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9fc003520 con 0x7fba14069850 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044142288 0 0) 0x7fb9fc003520 con 0x7fba14069850 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 >> v2:172.21.15.155:3300/0 conn(0x7fba14165f00 msgr2=0x7fba141682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 --2- 172.21.15.70:0/1503533292 >> v2:172.21.15.155:3300/0 conn(0x7fba14165f00 0x7fba141682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 >> v1:172.21.15.155:6789/0 conn(0x7fba1415e210 legacy=0x7fba14069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba140ad890 con 0x7fba14069850 2023-08-25T09:50:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fba04001670 con 0x7fba14069850 2023-08-25T09:50:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 >> v1:172.21.15.70:6789/0 conn(0x7fba14069850 legacy=0x7fba140c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9fc003970 0x7fb9fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9fc0047c0 0x7fb9fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc005180 0x7fb9fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1415e650 con 0x7fb9fc003970 2023-08-25T09:50:11.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc00c030 con 0x7fb9fc005180 2023-08-25T09:50:11.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9fc00c1f0 con 0x7fb9fc0047c0 2023-08-25T09:50:11.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.306+0000 7fba137fe640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9fc003970 0x7fb9fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba13fff640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc005180 0x7fb9fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba137fe640 1 -- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9fc0047c0 msgr2=0x7fb9fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba12ffd640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9fc0047c0 0x7fb9fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba137fe640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb9fc0047c0 0x7fb9fc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba137fe640 1 -- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc005180 msgr2=0x7fb9fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba137fe640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc005180 0x7fb9fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba137fe640 1 -- 172.21.15.70:0/1503533292 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9fc003520 con 0x7fb9fc003970 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba13fff640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc005180 0x7fb9fc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.307+0000 7fba13fff640 1 -- 172.21.15.70:0/1503533292 reap_dead start 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.308+0000 7fba137fe640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9fc003970 0x7fb9fc003d90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c190 tx=0x7fb9fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b91bb7acb11e98d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.308+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04005ad0 con 0x7fb9fc003970 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.308+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04005c30 con 0x7fb9fc003970 2023-08-25T09:50:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.308+0000 7fba127fc640 1 -- 172.21.15.70:0/1503533292 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0402ca70 con 0x7fb9fc003970 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 -- 172.21.15.70:0/1503533292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9fc003970 msgr2=0x7fb9fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9fc003970 0x7fb9fc003d90 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c190 tx=0x7fb9fc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 -- 172.21.15.70:0/1503533292 shutdown_connections 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/1503533292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb9fc003970 0x7fb9fc003d90 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 -- 172.21.15.70:0/1503533292 >> 172.21.15.70:0/1503533292 conn(0x7fba140cb0d0 msgr2=0x7fba140cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 -- 172.21.15.70:0/1503533292 shutdown_connections 2023-08-25T09:50:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.309+0000 7fba1a0f3640 1 -- 172.21.15.70:0/1503533292 wait complete. 2023-08-25T09:50:11.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.311+0000 7fba1a0f3640 1 Processor -- start 2023-08-25T09:50:11.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.311+0000 7fba1a0f3640 1 -- start start 2023-08-25T09:50:11.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba1a0f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc003970 0x7fba14155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba1a0f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 0x7fba14149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba1a0f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba140c7db0 0x7fba140c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba1a0f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba14155710 con 0x7fba14154670 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba1a0f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba141560e0 con 0x7fb9fc003970 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba1a0f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba14154b00 con 0x7fba140c7db0 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 0x7fba14149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 0x7fba14149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39944/0 (socket says 172.21.15.70:39944) 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 -- 172.21.15.70:0/2621992564 learned_addr learned my addr 172.21.15.70:0/2621992564 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba137fe640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc003970 0x7fba14155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba13fff640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba140c7db0 0x7fba140c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 -- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba140c7db0 msgr2=0x7fba140c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba140c7db0 0x7fba140c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 -- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc003970 msgr2=0x7fba14155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc003970 0x7fba14155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.312+0000 7fba12ffd640 1 -- 172.21.15.70:0/2621992564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba14154de0 con 0x7fba14154670 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.313+0000 7fba12ffd640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 0x7fba14149c40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fba0800d470 tx=0x7fba0800d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.313+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08005480 con 0x7fba14154670 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.313+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba08003610 con 0x7fba14154670 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.313+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08003980 con 0x7fba14154670 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.313+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba141550c0 con 0x7fba14154670 2023-08-25T09:50:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.313+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1414a180 con 0x7fba14154670 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.314+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba14066fb0 con 0x7fba14154670 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.315+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fba08003ae0 con 0x7fba14154670 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.316+0000 7fb9fbfff640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba0006a050 0x7fba0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.316+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fba08088490 con 0x7fba14154670 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.316+0000 7fba137fe640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba0006a050 0x7fba0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.316+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7fba00072400 con 0x7fba14154670 2023-08-25T09:50:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.317+0000 7fba137fe640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba0006a050 0x7fba0006c510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fba04034000 tx=0x7fba04009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:11.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.321+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fba08053ae0 con 0x7fba14154670 2023-08-25T09:50:11.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.494+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba14155850 con 0x7fba14154670 2023-08-25T09:50:11.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.495+0000 7fb9fbfff640 1 -- 172.21.15.70:0/2621992564 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba08057100 con 0x7fba14154670 2023-08-25T09:50:11.497 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:50:11.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba0006a050 msgr2=0x7fba0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba0006a050 0x7fba0006c510 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fba04034000 tx=0x7fba04009030 comp rx=0 tx=0).stop 2023-08-25T09:50:11.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 msgr2=0x7fba14149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 0x7fba14149c40 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fba0800d470 tx=0x7fba0800d930 comp rx=0 tx=0).stop 2023-08-25T09:50:11.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 shutdown_connections 2023-08-25T09:50:11.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fba0006a050 0x7fba0006c510 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fba140c7db0 0x7fba140c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fba14154670 0x7fba14149c40 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 --2- 172.21.15.70:0/2621992564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb9fc003970 0x7fba14155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 >> 172.21.15.70:0/2621992564 conn(0x7fba140cb0d0 msgr2=0x7fba1406c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:11.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 shutdown_connections 2023-08-25T09:50:11.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.498+0000 7fba1a0f3640 1 -- 172.21.15.70:0/2621992564 wait complete. 2023-08-25T09:50:11.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:50:11.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.730+0000 7fda96314640 1 Processor -- start 2023-08-25T09:50:11.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.730+0000 7fda96314640 1 -- start start 2023-08-25T09:50:11.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda96314640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fda9015a7d0 0x7fda900b3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda96314640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda9015abb0 0x7fda900b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda96314640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda90071850 0x7fda9006ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda96314640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda900c53b0 con 0x7fda9015abb0 2023-08-25T09:50:11.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda96314640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9015e4a0 con 0x7fda9015a7d0 2023-08-25T09:50:11.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda96314640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9015f020 con 0x7fda90071850 2023-08-25T09:50:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda9015abb0 0x7fda900b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fda9015abb0 0x7fda900b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39954/0 (socket says 172.21.15.70:39954) 2023-08-25T09:50:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 -- 172.21.15.70:0/2347123988 learned_addr learned my addr 172.21.15.70:0/2347123988 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:11.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 -- 172.21.15.70:0/2347123988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda90071850 msgr2=0x7fda9006ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 --2- 172.21.15.70:0/2347123988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda90071850 0x7fda9006ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 -- 172.21.15.70:0/2347123988 >> v2:172.21.15.155:3300/0 conn(0x7fda9015a7d0 msgr2=0x7fda900b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 --2- 172.21.15.70:0/2347123988 >> v2:172.21.15.155:3300/0 conn(0x7fda9015a7d0 0x7fda900b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.732+0000 7fda94b11640 1 -- 172.21.15.70:0/2347123988 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9015e6f0 con 0x7fda9015abb0 2023-08-25T09:50:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.733+0000 7fda94b11640 1 --2- 172.21.15.70:0/2347123988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9015abb0 0x7fda900b3b60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fda9006cef0 tx=0x7fda8002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd68d84fe7266f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.734+0000 7fda87fff640 1 -- 172.21.15.70:0/2347123988 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80031750 con 0x7fda9015abb0 2023-08-25T09:50:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.734+0000 7fda87fff640 1 -- 172.21.15.70:0/2347123988 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda800318b0 con 0x7fda9015abb0 2023-08-25T09:50:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.734+0000 7fda87fff640 1 -- 172.21.15.70:0/2347123988 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda80031c40 con 0x7fda9015abb0 2023-08-25T09:50:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.734+0000 7fda96314640 1 -- 172.21.15.70:0/2347123988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9015abb0 msgr2=0x7fda900b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.734+0000 7fda96314640 1 --2- 172.21.15.70:0/2347123988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9015abb0 0x7fda900b3b60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fda9006cef0 tx=0x7fda8002b7f0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 -- 172.21.15.70:0/2347123988 shutdown_connections 2023-08-25T09:50:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 --2- 172.21.15.70:0/2347123988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda90071850 0x7fda9006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 --2- 172.21.15.70:0/2347123988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda9015abb0 0x7fda900b3b60 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 --2- 172.21.15.70:0/2347123988 >> v2:172.21.15.155:3300/0 conn(0x7fda9015a7d0 0x7fda900b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 -- 172.21.15.70:0/2347123988 >> 172.21.15.70:0/2347123988 conn(0x7fda900aff60 msgr2=0x7fda900b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 -- 172.21.15.70:0/2347123988 shutdown_connections 2023-08-25T09:50:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.735+0000 7fda96314640 1 -- 172.21.15.70:0/2347123988 wait complete. 2023-08-25T09:50:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.737+0000 7fda96314640 1 Processor -- start 2023-08-25T09:50:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.737+0000 7fda96314640 1 -- start start 2023-08-25T09:50:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda96314640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 0x7fda901595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda96314640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda9015a7d0 0x7fda900c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda96314640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90159af0 0x7fda900c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda96314640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda901f2f20 con 0x7fda90071850 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda96314640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900c8450 con 0x7fda90159af0 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda96314640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda901f22a0 con 0x7fda9015a7d0 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda95312640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 0x7fda901595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda95312640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 0x7fda901595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39962/0 (socket says 172.21.15.70:39962) 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.738+0000 7fda95312640 1 -- 172.21.15.70:0/3973599525 learned_addr learned my addr 172.21.15.70:0/3973599525 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:11.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda94b11640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda9015a7d0 0x7fda900c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda95312640 1 -- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda9015a7d0 msgr2=0x7fda900c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda95312640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda9015a7d0 0x7fda900c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda95312640 1 -- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90159af0 msgr2=0x7fda900c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda95312640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90159af0 0x7fda900c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda95312640 1 -- 172.21.15.70:0/3973599525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda800094d0 con 0x7fda90071850 2023-08-25T09:50:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda94b11640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda9015a7d0 0x7fda900c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda95312640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 0x7fda901595b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00b4a0 tx=0x7fda8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c010610 con 0x7fda90071850 2023-08-25T09:50:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.739+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c010770 con 0x7fda90071850 2023-08-25T09:50:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.740+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda901f2580 con 0x7fda90071850 2023-08-25T09:50:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.740+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c010b20 con 0x7fda90071850 2023-08-25T09:50:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.740+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda901f2b40 con 0x7fda90071850 2023-08-25T09:50:11.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.741+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda90066fb0 con 0x7fda90071850 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.742+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fda8c010c80 con 0x7fda90071850 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.742+0000 7fda85ffb640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fda7406a050 0x7fda7406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.743+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fda8c088250 con 0x7fda90071850 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.743+0000 7fda94b11640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fda7406a050 0x7fda7406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.743+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7fda74072380 con 0x7fda90071850 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.744+0000 7fda94b11640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fda7406a050 0x7fda7406c510 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fda80035000 tx=0x7fda80008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.747+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fda8c0538a0 con 0x7fda90071850 2023-08-25T09:50:11.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.947+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda900b1270 con 0x7fda90071850 2023-08-25T09:50:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.947+0000 7fda85ffb640 1 -- 172.21.15.70:0/3973599525 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v169) v1 ==== 80+0+0 (secure 0 0 0) 0x7fda8c01e080 con 0x7fda90071850 2023-08-25T09:50:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:50:11.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fda7406a050 msgr2=0x7fda7406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fda7406a050 0x7fda7406c510 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fda80035000 tx=0x7fda80008030 comp rx=0 tx=0).stop 2023-08-25T09:50:11.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 msgr2=0x7fda901595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:11.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 0x7fda901595b0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00b4a0 tx=0x7fda8c00b960 comp rx=0 tx=0).stop 2023-08-25T09:50:11.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 shutdown_connections 2023-08-25T09:50:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fda7406a050 0x7fda7406c510 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fda90159af0 0x7fda900c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fda9015a7d0 0x7fda900c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 --2- 172.21.15.70:0/3973599525 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fda90071850 0x7fda901595b0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:11.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 >> 172.21.15.70:0/3973599525 conn(0x7fda900aff60 msgr2=0x7fda90070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 shutdown_connections 2023-08-25T09:50:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:11.950+0000 7fda96314640 1 -- 172.21.15.70:0/3973599525 wait complete. 2023-08-25T09:50:11.971 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:50:11.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:11.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:12.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[121171]: fuse finished with error 0 and tester_r 0 2023-08-25T09:50:12.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:50:12.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:12.061 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:50:12.236 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:50:12.237 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:12.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.524+0000 7f9f79bc2640 1 Processor -- start 2023-08-25T09:50:12.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.524+0000 7f9f79bc2640 1 -- start start 2023-08-25T09:50:12.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f79bc2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7415aa10 0x7f9f740b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f79bc2640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9f740b2680 0x7f9f7406f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f79bc2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7415d3f0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f79bc2640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f740b2b10 con 0x7f9f740b2680 2023-08-25T09:50:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f79bc2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74159d10 con 0x7f9f7415aa10 2023-08-25T09:50:12.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f73fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9f740b2680 0x7f9f7406f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f73fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9f740b2680 0x7f9f7406f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39972/0 (socket says 172.21.15.70:39972) 2023-08-25T09:50:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f73fff640 1 -- 172.21.15.70:0/24040128 learned_addr learned my addr 172.21.15.70:0/24040128 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:12.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.526+0000 7f9f6bfff640 1 --2- 172.21.15.70:0/24040128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7415aa10 0x7f9f740b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.527+0000 7f9f73fff640 1 -- 172.21.15.70:0/24040128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7415aa10 msgr2=0x7f9f740b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.527+0000 7f9f73fff640 1 --2- 172.21.15.70:0/24040128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7415aa10 0x7f9f740b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.527+0000 7f9f73fff640 1 -- 172.21.15.70:0/24040128 >> v1:172.21.15.155:6789/0 conn(0x7f9f740ce5b0 legacy=0x7f9f740ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.527+0000 7f9f73fff640 1 -- 172.21.15.70:0/24040128 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740b2d20 con 0x7f9f740b2680 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.528+0000 7f9f73fff640 1 --2- 172.21.15.70:0/24040128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740b2680 0x7f9f7406f9f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00d150 tx=0x7f9f6c02e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3310656a26d819f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.528+0000 7f9f72ffd640 1 -- 172.21.15.70:0/24040128 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00b0d0 con 0x7f9f740b2680 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.528+0000 7f9f72ffd640 1 -- 172.21.15.70:0/24040128 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c00b230 con 0x7f9f740b2680 2023-08-25T09:50:12.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.528+0000 7f9f72ffd640 1 -- 172.21.15.70:0/24040128 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c009600 con 0x7f9f740b2680 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 -- 172.21.15.70:0/24040128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740b2680 msgr2=0x7f9f7406f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/24040128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740b2680 0x7f9f7406f9f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00d150 tx=0x7f9f6c02e160 comp rx=0 tx=0).stop 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 -- 172.21.15.70:0/24040128 shutdown_connections 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/24040128 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740b2680 0x7f9f7406f9f0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/24040128 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f7415aa10 0x7f9f740b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 -- 172.21.15.70:0/24040128 >> 172.21.15.70:0/24040128 conn(0x7f9f74069850 msgr2=0x7f9f74069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:12.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 -- 172.21.15.70:0/24040128 shutdown_connections 2023-08-25T09:50:12.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.529+0000 7f9f79bc2640 1 -- 172.21.15.70:0/24040128 wait complete. 2023-08-25T09:50:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.531+0000 7f9f79bc2640 1 Processor -- start 2023-08-25T09:50:12.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.531+0000 7f9f79bc2640 1 -- start start 2023-08-25T09:50:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f79bc2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f740b2680 0x7f9f740aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f79bc2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 0x7f9f740cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f79bc2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f7415aa10 0x7f9f74146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f79bc2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740cd3f0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f79bc2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740af360 con 0x7f9f7415aa10 2023-08-25T09:50:12.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f79bc2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740af640 con 0x7f9f740b2680 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 0x7f9f740cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 0x7f9f740cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39984/0 (socket says 172.21.15.70:39984) 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 -- 172.21.15.70:0/2377257136 learned_addr learned my addr 172.21.15.70:0/2377257136 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 -- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f740b2680 msgr2=0x7f9f740aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f740b2680 0x7f9f740aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f73fff640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f7415aa10 0x7f9f74146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 -- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f7415aa10 msgr2=0x7f9f74146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f7415aa10 0x7f9f74146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.532+0000 7f9f6bfff640 1 -- 172.21.15.70:0/2377257136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f6c00bd40 con 0x7f9f740ce5b0 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.533+0000 7f9f6bfff640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 0x7f9f740cceb0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00d800 tx=0x7f9f5c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.533+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00b6c0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.533+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c00b820 con 0x7f9f740ce5b0 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.533+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c00bbb0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.533+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f740af920 con 0x7f9f740ce5b0 2023-08-25T09:50:12.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.533+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f741465b0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.534+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7404e420 con 0x7f9f740ce5b0 2023-08-25T09:50:12.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.535+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f5c00bd10 con 0x7f9f740ce5b0 2023-08-25T09:50:12.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.537+0000 7f9f70ff9640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f5406a080 0x7f9f5406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.537+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9f5c088b60 con 0x7f9f740ce5b0 2023-08-25T09:50:12.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.537+0000 7f9f737fe640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f5406a080 0x7f9f5406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.537+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f9f540723b0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.538+0000 7f9f737fe640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f5406a080 0x7f9f5406c540 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9f64000f60 tx=0x7f9f640079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:12.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.541+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9f5c0541b0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.731+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f740cae50 con 0x7f9f740ce5b0 2023-08-25T09:50:12.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.734+0000 7f9f70ff9640 1 -- 172.21.15.70:0/2377257136 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+18637 (secure 0 0 0) 0x7f9f5c0577d0 con 0x7f9f740ce5b0 2023-08-25T09:50:12.736 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:12.737 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":169,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:49:22.433958+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"20.13","mappings":[{"from":3,"to":7}]},{"pgid":"20.1d","mappings":[{"from":3,"to":1}]},{"pgid":"20.1e","mappings":[{"from":3,"to":7}]},{"pgid":"20.38","mappings":[{"from":3,"to":7}]},{"pgid":"20.3c","mappings":[{"from":3,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:12.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f5406a080 msgr2=0x7f9f5406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f5406a080 0x7f9f5406c540 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9f64000f60 tx=0x7f9f640079b0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 msgr2=0x7f9f740cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 0x7f9f740cceb0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00d800 tx=0x7f9f5c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 shutdown_connections 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9f5406a080 0x7f9f5406c540 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9f7415aa10 0x7f9f74146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9f740ce5b0 0x7f9f740cceb0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 --2- 172.21.15.70:0/2377257136 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9f740b2680 0x7f9f740aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.737+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 >> 172.21.15.70:0/2377257136 conn(0x7f9f74069850 msgr2=0x7f9f74159270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.738+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 shutdown_connections 2023-08-25T09:50:12.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.738+0000 7f9f79bc2640 1 -- 172.21.15.70:0/2377257136 wait complete. 2023-08-25T09:50:12.757 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:48:06.570383+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:48:07.289165+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:50:12.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:50:12.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.971+0000 7f15405e8640 1 Processor -- start 2023-08-25T09:50:12.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.971+0000 7f15405e8640 1 -- start start 2023-08-25T09:50:12.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f15405e8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f153815cfe0 0x7f15380d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f15405e8640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15380d1500 con 0x7f15380cd850 2023-08-25T09:50:12.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f15405e8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15380d17e0 con 0x7f15380b8510 2023-08-25T09:50:12.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f15405e8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15380d1ac0 con 0x7f153815cfe0 2023-08-25T09:50:12.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f153e384640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f15380cd850 0x7f15380cdc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59844/0 (socket says 172.21.15.70:59844) 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f153e384640 1 -- 172.21.15.70:0/384756877 learned_addr learned my addr 172.21.15.70:0/384756877 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f153db83640 1 --2- 172.21.15.70:0/384756877 >> v2:172.21.15.155:3300/0 conn(0x7f153815cfe0 0x7f15380d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.973+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586623488 0 0) 0x7f15380d1500 con 0x7f15380cd850 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1524003520 con 0x7f15380cd850 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3251744494 0 0) 0x7f1524003520 con 0x7f15380cd850 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 >> v2:172.21.15.155:3300/0 conn(0x7f153815cfe0 msgr2=0x7f15380d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:12.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 --2- 172.21.15.70:0/384756877 >> v2:172.21.15.155:3300/0 conn(0x7f153815cfe0 0x7f15380d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 >> v1:172.21.15.155:6789/0 conn(0x7f15380b8510 legacy=0x7f15380b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15380d1da0 con 0x7f15380cd850 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1534001670 con 0x7f15380cd850 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 >> v1:172.21.15.70:6789/0 conn(0x7f15380cd850 legacy=0x7f15380cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1524003970 0x7f1524003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15240047c0 0x7f1524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1524005180 0x7f152400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15380d1ac0 con 0x7f1524005180 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152400c0d0 con 0x7f15240047c0 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152400c290 con 0x7f1524003970 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153db83640 1 --2- 172.21.15.70:0/384756877 >> v2:172.21.15.155:3300/0 conn(0x7f153815cfe0 0x7f15380d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.974+0000 7f153eb85640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1524005180 0x7f152400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.975+0000 7f153eb85640 1 -- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1524003970 msgr2=0x7f1524003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:12.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.975+0000 7f153eb85640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1524003970 0x7f1524003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.975+0000 7f153eb85640 1 -- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15240047c0 msgr2=0x7f1524004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.975+0000 7f153db83640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15240047c0 0x7f1524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.975+0000 7f153eb85640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f15240047c0 0x7f1524004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.975+0000 7f153eb85640 1 -- 172.21.15.70:0/384756877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524003520 con 0x7f1524005180 2023-08-25T09:50:12.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.976+0000 7f153eb85640 1 -- 172.21.15.70:0/384756877 reap_dead start 2023-08-25T09:50:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.976+0000 7f153eb85640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1524005180 0x7f152400b980 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f1530001e70 tx=0x7f15240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8967d1b658551918 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.976+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530003bc0 con 0x7f1524005180 2023-08-25T09:50:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.976+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530003d60 con 0x7f1524005180 2023-08-25T09:50:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.976+0000 7f153d382640 1 -- 172.21.15.70:0/384756877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530005310 con 0x7f1524005180 2023-08-25T09:50:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 -- 172.21.15.70:0/384756877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1524005180 msgr2=0x7f152400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1524005180 0x7f152400b980 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f1530001e70 tx=0x7f15240057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 -- 172.21.15.70:0/384756877 shutdown_connections 2023-08-25T09:50:12.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 --2- 172.21.15.70:0/384756877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1524005180 0x7f152400b980 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 -- 172.21.15.70:0/384756877 >> 172.21.15.70:0/384756877 conn(0x7f1538069850 msgr2=0x7f1538069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:12.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 -- 172.21.15.70:0/384756877 shutdown_connections 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.977+0000 7f15405e8640 1 -- 172.21.15.70:0/384756877 wait complete. 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.979+0000 7f15405e8640 1 Processor -- start 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.979+0000 7f15405e8640 1 -- start start 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f15405e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1524005180 0x7f15381513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f15405e8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15380a9f00 0x7f15380cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f15405e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 0x7f15380cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f15405e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15380ab560 con 0x7f15380aa380 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f15405e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15380aa950 con 0x7f1524005180 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f15405e8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15380aac30 con 0x7f15380a9f00 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f153eb85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 0x7f15380cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f153eb85640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 0x7f15380cc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40006/0 (socket says 172.21.15.70:40006) 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.980+0000 7f153eb85640 1 -- 172.21.15.70:0/3127351554 learned_addr learned my addr 172.21.15.70:0/3127351554 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153e384640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1524005180 0x7f15381513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153eb85640 1 -- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15380a9f00 msgr2=0x7f15380cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153eb85640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15380a9f00 0x7f15380cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153eb85640 1 -- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1524005180 msgr2=0x7f15381513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153eb85640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1524005180 0x7f15381513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153eb85640 1 -- 172.21.15.70:0/3127351554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15380aaf10 con 0x7f15380aa380 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153e384640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1524005180 0x7f15381513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f153eb85640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 0x7f15380cc520 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1530005040 tx=0x7f15300014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:12.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530006da0 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530039460 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15300397f0 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15380ab1f0 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.981+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15380c9be0 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.982+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1538066fb0 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.984+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1530039950 con 0x7f15380aa380 2023-08-25T09:50:12.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.984+0000 7f152effd640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f151c06a080 0x7f151c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:12.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.985+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f15300aa350 con 0x7f15380aa380 2023-08-25T09:50:12.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.985+0000 7f153e384640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f151c06a080 0x7f151c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:12.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.985+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f151c0723b0 con 0x7f15380aa380 2023-08-25T09:50:12.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.985+0000 7f153e384640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f151c06a080 0x7f151c06c540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1534007a70 tx=0x7f1534009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:12.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:12.989+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1530075910 con 0x7f15380aa380 2023-08-25T09:50:13.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.184+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15380c93c0 con 0x7f15380aa380 2023-08-25T09:50:13.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.185+0000 7f152effd640 1 -- 172.21.15.70:0/3127351554 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1530078f30 con 0x7f15380aa380 2023-08-25T09:50:13.187 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:50:13.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f151c06a080 msgr2=0x7f151c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f151c06a080 0x7f151c06c540 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1534007a70 tx=0x7f1534009040 comp rx=0 tx=0).stop 2023-08-25T09:50:13.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 msgr2=0x7f15380cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 0x7f15380cc520 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1530005040 tx=0x7f15300014f0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 shutdown_connections 2023-08-25T09:50:13.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f151c06a080 0x7f151c06c540 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f15380aa380 0x7f15380cc520 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f15380a9f00 0x7f15380cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 --2- 172.21.15.70:0/3127351554 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1524005180 0x7f15381513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 >> 172.21.15.70:0/3127351554 conn(0x7f1538069850 msgr2=0x7f153815c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:13.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.188+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 shutdown_connections 2023-08-25T09:50:13.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.189+0000 7f15405e8640 1 -- 172.21.15.70:0/3127351554 wait complete. 2023-08-25T09:50:13.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:50:13.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.424+0000 7f8a1a238640 1 Processor -- start 2023-08-25T09:50:13.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.424+0000 7f8a1a238640 1 -- start start 2023-08-25T09:50:13.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a1a238640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 0x7f8a14069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a1a238640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8a14069d90 0x7f8a140ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a1a238640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a140c2550 con 0x7f8a1415ae50 2023-08-25T09:50:13.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a1a238640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1406a220 con 0x7f8a14069d90 2023-08-25T09:50:13.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a1a238640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140ce330 con 0x7f8a140c2110 2023-08-25T09:50:13.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a137fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8a1415ae50 0x7f8a140cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59850/0 (socket says 172.21.15.70:59850) 2023-08-25T09:50:13.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a137fe640 1 -- 172.21.15.70:0/2129584584 learned_addr learned my addr 172.21.15.70:0/2129584584 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:13.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a12ffd640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 0x7f8a14069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991786124 0 0) 0x7f8a140c2550 con 0x7f8a1415ae50 2023-08-25T09:50:13.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89fc003520 con 0x7f8a1415ae50 2023-08-25T09:50:13.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.426+0000 7f8a13fff640 1 --2- 172.21.15.70:0/2129584584 >> v2:172.21.15.155:3300/0 conn(0x7f8a14069d90 0x7f8a140ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1495291842 0 0) 0x7f89fc003520 con 0x7f8a1415ae50 2023-08-25T09:50:13.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 msgr2=0x7f8a14069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:13.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 0x7f8a14069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 >> v2:172.21.15.155:3300/0 conn(0x7f8a14069d90 msgr2=0x7f8a140ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:13.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 --2- 172.21.15.70:0/2129584584 >> v2:172.21.15.155:3300/0 conn(0x7f8a14069d90 0x7f8a140ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140cc5f0 con 0x7f8a1415ae50 2023-08-25T09:50:13.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8a04001670 con 0x7f8a1415ae50 2023-08-25T09:50:13.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 >> v1:172.21.15.70:6789/0 conn(0x7f8a1415ae50 legacy=0x7f8a140cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89fc003970 0x7f89fc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 0x7f89fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc004d00 0x7f89fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1406a220 con 0x7f89fc003970 2023-08-25T09:50:13.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140ce330 con 0x7f89fc004d00 2023-08-25T09:50:13.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc00ba40 con 0x7f8a140c2110 2023-08-25T09:50:13.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.427+0000 7f8a137fe640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89fc003970 0x7f89fc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a12ffd640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 0x7f89fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a13fff640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc004d00 0x7f89fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a137fe640 1 -- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 msgr2=0x7f89fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:13.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a137fe640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a140c2110 0x7f89fc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a137fe640 1 -- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc004d00 msgr2=0x7f89fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:13.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a137fe640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc004d00 0x7f89fc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.428+0000 7f8a137fe640 1 -- 172.21.15.70:0/2129584584 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc003520 con 0x7f89fc003970 2023-08-25T09:50:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a137fe640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89fc003970 0x7f89fc003d90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c240 tx=0x7f89fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89e22d8498ead3d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0402cb00 con 0x7f89fc003970 2023-08-25T09:50:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0402cc60 con 0x7f89fc003970 2023-08-25T09:50:13.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a127fc640 1 -- 172.21.15.70:0/2129584584 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04005af0 con 0x7f89fc003970 2023-08-25T09:50:13.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a1a238640 1 -- 172.21.15.70:0/2129584584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89fc003970 msgr2=0x7f89fc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a1a238640 1 --2- 172.21.15.70:0/2129584584 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89fc003970 0x7f89fc003d90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c240 tx=0x7f89fc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a13fff640 1 -- 172.21.15.70:0/2129584584 reap_dead start 2023-08-25T09:50:13.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.429+0000 7f8a1a238640 1 -- 172.21.15.70:0/2129584584 shutdown_connections 2023-08-25T09:50:13.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.430+0000 7f8a1a238640 1 -- 172.21.15.70:0/2129584584 >> 172.21.15.70:0/2129584584 conn(0x7f8a140cb0d0 msgr2=0x7f8a140cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:13.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.430+0000 7f8a1a238640 1 -- 172.21.15.70:0/2129584584 shutdown_connections 2023-08-25T09:50:13.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.430+0000 7f8a1a238640 1 -- 172.21.15.70:0/2129584584 wait complete. 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.431+0000 7f8a1a238640 1 Processor -- start 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.431+0000 7f8a1a238640 1 -- start start 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.432+0000 7f8a1a238640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc003970 0x7f8a140b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.432+0000 7f8a1a238640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a141571a0 0x7f8a140c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.432+0000 7f8a1a238640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 0x7f8a14158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.432+0000 7f8a1a238640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14159400 con 0x7f8a14158a40 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.432+0000 7f8a1a238640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140b23b0 con 0x7f89fc003970 2023-08-25T09:50:13.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.432+0000 7f8a1a238640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140c5bd0 con 0x7f8a141571a0 2023-08-25T09:50:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 0x7f8a14158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 0x7f8a14158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40022/0 (socket says 172.21.15.70:40022) 2023-08-25T09:50:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 -- 172.21.15.70:0/335912178 learned_addr learned my addr 172.21.15.70:0/335912178 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a137fe640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc003970 0x7f8a140b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 -- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a141571a0 msgr2=0x7f8a140c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a141571a0 0x7f8a140c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 -- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc003970 msgr2=0x7f8a140b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc003970 0x7f8a140b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 -- 172.21.15.70:0/335912178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140c5e70 con 0x7f8a14158a40 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f8a13fff640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 0x7f8a14158ec0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8a1406d1f0 tx=0x7f8a0c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.433+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c007cf0 con 0x7f8a14158a40 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.434+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0c007e70 con 0x7f8a14158a40 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.434+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c014620 con 0x7f8a14158a40 2023-08-25T09:50:13.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.434+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a140c6150 con 0x7f8a14158a40 2023-08-25T09:50:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.434+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a140c6710 con 0x7f8a14158a40 2023-08-25T09:50:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.435+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140c68d0 con 0x7f8a14158a40 2023-08-25T09:50:13.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.436+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8a0c01d3b0 con 0x7f8a14158a40 2023-08-25T09:50:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.437+0000 7f89fbfff640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a0006a080 0x7f8a0006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.437+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f8a0c088350 con 0x7f8a14158a40 2023-08-25T09:50:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.437+0000 7f8a137fe640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a0006a080 0x7f8a0006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.438+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f8a000723b0 con 0x7f8a14158a40 2023-08-25T09:50:13.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.438+0000 7f8a137fe640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a0006a080 0x7f8a0006c540 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8a04034000 tx=0x7f8a04007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:13.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.442+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8a0c053910 con 0x7f8a14158a40 2023-08-25T09:50:13.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.628+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8a14070bf0 con 0x7f8a14158a40 2023-08-25T09:50:13.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.632+0000 7f89fbfff640 1 -- 172.21.15.70:0/335912178 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v169) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8a14070bf0 con 0x7f8a14158a40 2023-08-25T09:50:13.634 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a0006a080 msgr2=0x7f8a0006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a0006a080 0x7f8a0006c540 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8a04034000 tx=0x7f8a04007000 comp rx=0 tx=0).stop 2023-08-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 msgr2=0x7f8a14158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 0x7f8a14158ec0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f8a1406d1f0 tx=0x7f8a0c009d40 comp rx=0 tx=0).stop 2023-08-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 shutdown_connections 2023-08-25T09:50:13.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8a0006a080 0x7f8a0006c540 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8a14158a40 0x7f8a14158ec0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8a141571a0 0x7f8a140c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 --2- 172.21.15.70:0/335912178 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89fc003970 0x7f8a140b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 >> 172.21.15.70:0/335912178 conn(0x7f8a140cb0d0 msgr2=0x7f8a140cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 shutdown_connections 2023-08-25T09:50:13.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.635+0000 7f8a1a238640 1 -- 172.21.15.70:0/335912178 wait complete. 2023-08-25T09:50:13.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:50:13.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.865+0000 7fc8254d3640 1 Processor -- start 2023-08-25T09:50:13.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.865+0000 7fc8254d3640 1 -- start start 2023-08-25T09:50:13.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc8254d3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc82015e240 0x7fc820070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc8254d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc8254d3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82006f8e0 con 0x7fc820159bf0 2023-08-25T09:50:13.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc8254d3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8200ce030 con 0x7fc82015e240 2023-08-25T09:50:13.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc8254d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82006fbc0 con 0x7fc8200cd660 2023-08-25T09:50:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc817fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc82015e240 0x7fc820070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc817fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc82015e240 0x7fc820070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40024/0 (socket says 172.21.15.70:40024) 2023-08-25T09:50:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.867+0000 7fc817fff640 1 -- 172.21.15.70:0/1268925891 learned_addr learned my addr 172.21.15.70:0/1268925891 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.868+0000 7fc817fff640 1 -- 172.21.15.70:0/1268925891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 msgr2=0x7fc8200cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.868+0000 7fc81f7fe640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.868+0000 7fc817fff640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.868+0000 7fc817fff640 1 -- 172.21.15.70:0/1268925891 >> v1:172.21.15.155:6789/0 conn(0x7fc820159bf0 legacy=0x7fc82006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:13.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.868+0000 7fc817fff640 1 -- 172.21.15.70:0/1268925891 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82006fea0 con 0x7fc82015e240 2023-08-25T09:50:13.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.868+0000 7fc81f7fe640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:13.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.869+0000 7fc817fff640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc820070980 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc808009870 tx=0x7fc80802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f225b3f788a9c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:13.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.869+0000 7fc81e7fc640 1 -- 172.21.15.70:0/1268925891 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808031640 con 0x7fc82015e240 2023-08-25T09:50:13.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc81e7fc640 1 -- 172.21.15.70:0/1268925891 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8080317a0 con 0x7fc82015e240 2023-08-25T09:50:13.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc81e7fc640 1 -- 172.21.15.70:0/1268925891 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808031b30 con 0x7fc82015e240 2023-08-25T09:50:13.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc8254d3640 1 -- 172.21.15.70:0/1268925891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 msgr2=0x7fc820070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc8254d3640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc820070980 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fc808009870 tx=0x7fc80802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc8254d3640 1 -- 172.21.15.70:0/1268925891 shutdown_connections 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc8254d3640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc8254d3640 1 --2- 172.21.15.70:0/1268925891 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc820070980 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.870+0000 7fc8254d3640 1 -- 172.21.15.70:0/1268925891 >> 172.21.15.70:0/1268925891 conn(0x7fc8200cb0d0 msgr2=0x7fc8200cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.871+0000 7fc8254d3640 1 -- 172.21.15.70:0/1268925891 shutdown_connections 2023-08-25T09:50:13.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.871+0000 7fc8254d3640 1 -- 172.21.15.70:0/1268925891 wait complete. 2023-08-25T09:50:13.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.872+0000 7fc8254d3640 1 Processor -- start 2023-08-25T09:50:13.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.872+0000 7fc8254d3640 1 -- start start 2023-08-25T09:50:13.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.873+0000 7fc8254d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200c68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.873+0000 7fc8254d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc820159bf0 0x7fc8200c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.873+0000 7fc8254d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc8200cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.873+0000 7fc8254d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200c6140 con 0x7fc82015e240 2023-08-25T09:50:13.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.873+0000 7fc8254d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8200c6e10 con 0x7fc820159bf0 2023-08-25T09:50:13.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.873+0000 7fc8254d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8201507d0 con 0x7fc8200cd660 2023-08-25T09:50:13.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc8200cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc8200cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40038/0 (socket says 172.21.15.70:40038) 2023-08-25T09:50:13.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 -- 172.21.15.70:0/217267340 learned_addr learned my addr 172.21.15.70:0/217267340 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:13.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc817fff640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc820159bf0 0x7fc8200c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81effd640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200c68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 -- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 msgr2=0x7fc8200c68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200c68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 -- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc820159bf0 msgr2=0x7fc8200c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc820159bf0 0x7fc8200c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 -- 172.21.15.70:0/217267340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080094d0 con 0x7fc82015e240 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc817fff640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc820159bf0 0x7fc8200c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:13.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.874+0000 7fc81f7fe640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc8200cc550 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fc81800cef0 tx=0x7fc81800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.875+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800d840 con 0x7fc82015e240 2023-08-25T09:50:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.875+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81800d9c0 con 0x7fc82015e240 2023-08-25T09:50:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.875+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81800dd70 con 0x7fc82015e240 2023-08-25T09:50:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.875+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc820150ab0 con 0x7fc82015e240 2023-08-25T09:50:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.875+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8200c8540 con 0x7fc82015e240 2023-08-25T09:50:13.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.876+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8200c65f0 con 0x7fc82015e240 2023-08-25T09:50:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.877+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc81801e3b0 con 0x7fc82015e240 2023-08-25T09:50:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.878+0000 7fc8177fe640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80006a050 0x7fc80006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.878+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fc818089130 con 0x7fc82015e240 2023-08-25T09:50:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.878+0000 7fc81effd640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80006a050 0x7fc80006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.878+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7fc800072380 con 0x7fc82015e240 2023-08-25T09:50:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.879+0000 7fc81effd640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80006a050 0x7fc80006c510 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc810003d90 tx=0x7fc810009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:13.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:13.883+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc8200c65f0 con 0x7fc82015e240 2023-08-25T09:50:14.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.071+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8200c65f0 con 0x7fc82015e240 2023-08-25T09:50:14.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.072+0000 7fc8177fe640 1 -- 172.21.15.70:0/217267340 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v169) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8200c65f0 con 0x7fc82015e240 2023-08-25T09:50:14.073 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:50:14.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.074+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80006a050 msgr2=0x7fc80006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.074+0000 7fc8254d3640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80006a050 0x7fc80006c510 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc810003d90 tx=0x7fc810009040 comp rx=0 tx=0).stop 2023-08-25T09:50:14.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.074+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 msgr2=0x7fc8200cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.074+0000 7fc8254d3640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc8200cc550 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fc81800cef0 tx=0x7fc81800a710 comp rx=0 tx=0).stop 2023-08-25T09:50:14.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 shutdown_connections 2023-08-25T09:50:14.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc80006a050 0x7fc80006c510 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc82015e240 0x7fc8200cc550 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc820159bf0 0x7fc8200c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 --2- 172.21.15.70:0/217267340 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc8200cd660 0x7fc8200c68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 >> 172.21.15.70:0/217267340 conn(0x7fc8200cb0d0 msgr2=0x7fc8201542e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:14.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 shutdown_connections 2023-08-25T09:50:14.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.075+0000 7fc8254d3640 1 -- 172.21.15.70:0/217267340 wait complete. 2023-08-25T09:50:14.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:50:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.317+0000 7efe0c68e640 1 Processor -- start 2023-08-25T09:50:14.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.317+0000 7efe0c68e640 1 -- start start 2023-08-25T09:50:14.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.318+0000 7efe0c68e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe04069f60 0x7efe040ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.318+0000 7efe0c68e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040c2550 con 0x7efe040c2110 2023-08-25T09:50:14.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.318+0000 7efe0c68e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe040ce330 con 0x7efe0415ae50 2023-08-25T09:50:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.318+0000 7efe0c68e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040cc5f0 con 0x7efe04069f60 2023-08-25T09:50:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.319+0000 7efe09c29640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efe040c2110 0x7efe04069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59858/0 (socket says 172.21.15.70:59858) 2023-08-25T09:50:14.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.319+0000 7efe09c29640 1 -- 172.21.15.70:0/1674619659 learned_addr learned my addr 172.21.15.70:0/1674619659 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.319+0000 7efe0ac2b640 1 --2- 172.21.15.70:0/1674619659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe04069f60 0x7efe040ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.319+0000 7efe09428640 1 -- 172.21.15.70:0/1674619659 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399854854 0 0) 0x7efe040c2550 con 0x7efe040c2110 2023-08-25T09:50:14.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.319+0000 7efe09428640 1 -- 172.21.15.70:0/1674619659 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf0003520 con 0x7efe040c2110 2023-08-25T09:50:14.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.320+0000 7efe0ac2b640 1 -- 172.21.15.70:0/1674619659 >> v1:172.21.15.155:6789/0 conn(0x7efe0415ae50 legacy=0x7efe040cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.320+0000 7efe09428640 1 -- 172.21.15.70:0/1674619659 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089109806 0 0) 0x7efdf0003520 con 0x7efe040c2110 2023-08-25T09:50:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.320+0000 7efe0ac2b640 1 -- 172.21.15.70:0/1674619659 >> v1:172.21.15.70:6789/0 conn(0x7efe040c2110 legacy=0x7efe04069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.320+0000 7efe0ac2b640 1 -- 172.21.15.70:0/1674619659 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04070650 con 0x7efe04069f60 2023-08-25T09:50:14.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.321+0000 7efe0ac2b640 1 --2- 172.21.15.70:0/1674619659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe04069f60 0x7efe040ccad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009dc0 tx=0x7efdfc02be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aaba658ebbffa83 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe09428640 1 -- 172.21.15.70:0/1674619659 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4001670 con 0x7efe04069f60 2023-08-25T09:50:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe09428640 1 -- 172.21.15.70:0/1674619659 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc02c9a0 con 0x7efe04069f60 2023-08-25T09:50:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe09428640 1 -- 172.21.15.70:0/1674619659 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc02cd10 con 0x7efe04069f60 2023-08-25T09:50:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe0c68e640 1 -- 172.21.15.70:0/1674619659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe04069f60 msgr2=0x7efe040ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe0c68e640 1 --2- 172.21.15.70:0/1674619659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe04069f60 0x7efe040ccad0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009dc0 tx=0x7efdfc02be70 comp rx=0 tx=0).stop 2023-08-25T09:50:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe0c68e640 1 -- 172.21.15.70:0/1674619659 shutdown_connections 2023-08-25T09:50:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe0c68e640 1 --2- 172.21.15.70:0/1674619659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe04069f60 0x7efe040ccad0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe0c68e640 1 -- 172.21.15.70:0/1674619659 >> 172.21.15.70:0/1674619659 conn(0x7efe040cb0d0 msgr2=0x7efe040cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.322+0000 7efe0c68e640 1 -- 172.21.15.70:0/1674619659 shutdown_connections 2023-08-25T09:50:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.323+0000 7efe0c68e640 1 -- 172.21.15.70:0/1674619659 wait complete. 2023-08-25T09:50:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.324+0000 7efe0c68e640 1 Processor -- start 2023-08-25T09:50:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.324+0000 7efe0c68e640 1 -- start start 2023-08-25T09:50:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.325+0000 7efe0c68e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04069f60 0x7efe04157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.325+0000 7efe0c68e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 0x7efe040c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.325+0000 7efe0c68e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe0415ae50 0x7efe040c2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.325+0000 7efe0c68e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c5690 con 0x7efe040c2110 2023-08-25T09:50:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.325+0000 7efe0c68e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041f35e0 con 0x7efe04069f60 2023-08-25T09:50:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.325+0000 7efe0c68e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe040c58a0 con 0x7efe0415ae50 2023-08-25T09:50:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 0x7efe040c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 0x7efe040c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40054/0 (socket says 172.21.15.70:40054) 2023-08-25T09:50:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 -- 172.21.15.70:0/364991661 learned_addr learned my addr 172.21.15.70:0/364991661 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe0a42a640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04069f60 0x7efe04157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 -- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe0415ae50 msgr2=0x7efe040c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe0415ae50 0x7efe040c2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 -- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04069f60 msgr2=0x7efe04157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04069f60 0x7efe04157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 -- 172.21.15.70:0/364991661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc009a70 con 0x7efe040c2110 2023-08-25T09:50:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe0a42a640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04069f60 0x7efe04157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.326+0000 7efe09c29640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 0x7efe040c28b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c760 tx=0x7efdf400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.327+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4005ad0 con 0x7efe040c2110 2023-08-25T09:50:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.327+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf4005c30 con 0x7efe040c2110 2023-08-25T09:50:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.327+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf4007070 con 0x7efe040c2110 2023-08-25T09:50:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.327+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe040c5b80 con 0x7efe040c2110 2023-08-25T09:50:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.327+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041581a0 con 0x7efe040c2110 2023-08-25T09:50:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.328+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe040c60a0 con 0x7efe040c2110 2023-08-25T09:50:14.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.329+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efdf40071d0 con 0x7efe040c2110 2023-08-25T09:50:14.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.330+0000 7efdfaffd640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efde806a050 0x7efde806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.330+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7efdf4088450 con 0x7efe040c2110 2023-08-25T09:50:14.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.330+0000 7efe0a42a640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efde806a050 0x7efde806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.330+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7efde8072380 con 0x7efe040c2110 2023-08-25T09:50:14.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.331+0000 7efe0a42a640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efde806a050 0x7efde806c510 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efe000015e0 tx=0x7efe00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:14.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.334+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efe040c60a0 con 0x7efe040c2110 2023-08-25T09:50:14.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.523+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe040c60a0 con 0x7efe040c2110 2023-08-25T09:50:14.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.524+0000 7efdfaffd640 1 -- 172.21.15.70:0/364991661 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v169) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe040c60a0 con 0x7efe040c2110 2023-08-25T09:50:14.525 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:50:14.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efde806a050 msgr2=0x7efde806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efde806a050 0x7efde806c510 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efe000015e0 tx=0x7efe00009040 comp rx=0 tx=0).stop 2023-08-25T09:50:14.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 msgr2=0x7efe040c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 0x7efe040c28b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7efdf400c760 tx=0x7efdf400cc20 comp rx=0 tx=0).stop 2023-08-25T09:50:14.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 shutdown_connections 2023-08-25T09:50:14.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efde806a050 0x7efde806c510 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efe0415ae50 0x7efe040c2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efe040c2110 0x7efe040c28b0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 --2- 172.21.15.70:0/364991661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efe04069f60 0x7efe04157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.526+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 >> 172.21.15.70:0/364991661 conn(0x7efe040cb0d0 msgr2=0x7efe0415a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:14.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.527+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 shutdown_connections 2023-08-25T09:50:14.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.527+0000 7efe0c68e640 1 -- 172.21.15.70:0/364991661 wait complete. 2023-08-25T09:50:14.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:50:14.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.759+0000 7f892df22640 1 Processor -- start 2023-08-25T09:50:14.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.759+0000 7f892df22640 1 -- start start 2023-08-25T09:50:14.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f892df22640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8928159aa0 0x7f8928070410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f892df22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892806bc80 0x7f8928070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f892df22640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892806aec0 con 0x7f892815df10 2023-08-25T09:50:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f892df22640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8928159e80 con 0x7f8928159aa0 2023-08-25T09:50:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f892df22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892806b1a0 con 0x7f892806bc80 2023-08-25T09:50:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f8927fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f892815df10 0x7f89280c9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59864/0 (socket says 172.21.15.70:59864) 2023-08-25T09:50:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.761+0000 7f8927fff640 1 -- 172.21.15.70:0/1315127640 learned_addr learned my addr 172.21.15.70:0/1315127640 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:14.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f8926ffd640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892806bc80 0x7f8928070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89277fe640 1 --2- 172.21.15.70:0/1315127640 >> v2:172.21.15.155:3300/0 conn(0x7f8928159aa0 0x7f8928070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728343316 0 0) 0x7f892806aec0 con 0x7f892815df10 2023-08-25T09:50:14.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8910003520 con 0x7f892815df10 2023-08-25T09:50:14.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1064027912 0 0) 0x7f8910003520 con 0x7f892815df10 2023-08-25T09:50:14.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892806bc80 msgr2=0x7f8928070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f892806bc80 0x7f8928070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 >> v2:172.21.15.155:3300/0 conn(0x7f8928159aa0 msgr2=0x7f8928070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 --2- 172.21.15.70:0/1315127640 >> v2:172.21.15.155:3300/0 conn(0x7f8928159aa0 0x7f8928070410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892806b480 con 0x7f892815df10 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f89200019f0 con 0x7f892815df10 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 >> v1:172.21.15.70:6789/0 conn(0x7f892815df10 legacy=0x7f89280c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8910003970 0x7f8910003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89100042d0 0x7f89100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8910004ec0 0x7f891000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8928159e80 con 0x7f89100042d0 2023-08-25T09:50:14.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892806b1a0 con 0x7f8910004ec0 2023-08-25T09:50:14.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.762+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891000bec0 con 0x7f8910003970 2023-08-25T09:50:14.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8926ffd640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89100042d0 0x7f89100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f89277fe640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8910003970 0x7f8910003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8927fff640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8910004ec0 0x7f891000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8926ffd640 1 -- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8910003970 msgr2=0x7f8910003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8926ffd640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8910003970 0x7f8910003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8926ffd640 1 -- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8910004ec0 msgr2=0x7f891000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8926ffd640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8910004ec0 0x7f891000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f8926ffd640 1 -- 172.21.15.70:0/1315127640 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8910003520 con 0x7f89100042d0 2023-08-25T09:50:14.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.763+0000 7f89277fe640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8910003970 0x7f8910003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:14.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.764+0000 7f8926ffd640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89100042d0 0x7f89100047c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ac0 tx=0x7f89100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feebfa0af12dbfcd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:14.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c02c6b0 con 0x7f89100042d0 2023-08-25T09:50:14.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c02c810 con 0x7f89100042d0 2023-08-25T09:50:14.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f89267fc640 1 -- 172.21.15.70:0/1315127640 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c02cba0 con 0x7f89100042d0 2023-08-25T09:50:14.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f892df22640 1 -- 172.21.15.70:0/1315127640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89100042d0 msgr2=0x7f89100047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f892df22640 1 --2- 172.21.15.70:0/1315127640 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89100042d0 0x7f89100047c0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ac0 tx=0x7f89100057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f8927fff640 1 -- 172.21.15.70:0/1315127640 reap_dead start 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f892df22640 1 -- 172.21.15.70:0/1315127640 shutdown_connections 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f892df22640 1 -- 172.21.15.70:0/1315127640 >> 172.21.15.70:0/1315127640 conn(0x7f89280ce760 msgr2=0x7f89280ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f892df22640 1 -- 172.21.15.70:0/1315127640 shutdown_connections 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.765+0000 7f892df22640 1 -- 172.21.15.70:0/1315127640 wait complete. 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.767+0000 7f892df22640 1 Processor -- start 2023-08-25T09:50:14.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.767+0000 7f892df22640 1 -- start start 2023-08-25T09:50:14.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f892df22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89100042d0 0x7f89281466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f892df22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 0x7f89281f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f892df22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89280a8460 0x7f89280a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f892df22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89280a8e30 con 0x7f89281f6650 2023-08-25T09:50:14.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f892df22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892815b550 con 0x7f89100042d0 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f892df22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89281f2d20 con 0x7f89280a8460 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f8926ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 0x7f89281f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.768+0000 7f8926ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 0x7f89281f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40072/0 (socket says 172.21.15.70:40072) 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 -- 172.21.15.70:0/2747734203 learned_addr learned my addr 172.21.15.70:0/2747734203 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8927fff640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89280a8460 0x7f89280a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 -- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89280a8460 msgr2=0x7f89280a88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89280a8460 0x7f89280a88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f89277fe640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89100042d0 0x7f89281466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 -- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89100042d0 msgr2=0x7f89281466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89100042d0 0x7f89281466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 -- 172.21.15.70:0/2747734203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89281f3000 con 0x7f89281f6650 2023-08-25T09:50:14.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.769+0000 7f8926ffd640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 0x7f89281f23b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ac0 tx=0x7f891c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:14.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.770+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c003810 con 0x7f89281f6650 2023-08-25T09:50:14.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.770+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c003970 con 0x7f89281f6650 2023-08-25T09:50:14.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.770+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892815b1e0 con 0x7f89281f6650 2023-08-25T09:50:14.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.770+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c003d00 con 0x7f89281f6650 2023-08-25T09:50:14.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.770+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8928146c20 con 0x7f89281f6650 2023-08-25T09:50:14.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.771+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892815b6e0 con 0x7f89281f6650 2023-08-25T09:50:14.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.772+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f891c040450 con 0x7f89281f6650 2023-08-25T09:50:14.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.773+0000 7f890ffff640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f891406a050 0x7f891406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:14.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.773+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f891c0aa500 con 0x7f89281f6650 2023-08-25T09:50:14.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.773+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f8914072380 con 0x7f89281f6650 2023-08-25T09:50:14.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.773+0000 7f89277fe640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f891406a050 0x7f891406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:14.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.774+0000 7f89277fe640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f891406a050 0x7f891406c510 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f89280cb300 tx=0x7f8918008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:14.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.777+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f891c075b50 con 0x7f89281f6650 2023-08-25T09:50:14.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.982+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f892804e420 con 0x7f89281f6650 2023-08-25T09:50:14.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.982+0000 7f890ffff640 1 -- 172.21.15.70:0/2747734203 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v169) v1 ==== 80+0+0 (secure 0 0 0) 0x7f891c003e60 con 0x7f89281f6650 2023-08-25T09:50:14.985 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:50:14.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f891406a050 msgr2=0x7f891406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f891406a050 0x7f891406c510 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f89280cb300 tx=0x7f8918008040 comp rx=0 tx=0).stop 2023-08-25T09:50:14.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 msgr2=0x7f89281f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:14.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 0x7f89281f23b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ac0 tx=0x7f891c004730 comp rx=0 tx=0).stop 2023-08-25T09:50:14.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 shutdown_connections 2023-08-25T09:50:14.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f891406a050 0x7f891406c510 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f89280a8460 0x7f89280a88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f89281f6650 0x7f89281f23b0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 --2- 172.21.15.70:0/2747734203 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f89100042d0 0x7f89281466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:14.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 >> 172.21.15.70:0/2747734203 conn(0x7f89280ce760 msgr2=0x7f892806ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:14.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 shutdown_connections 2023-08-25T09:50:14.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:14.985+0000 7f892df22640 1 -- 172.21.15.70:0/2747734203 wait complete. 2023-08-25T09:50:15.006 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:50:15.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:15.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:15.108 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[92718]: fuse finished with error 0 and tester_r 0 2023-08-25T09:50:15.111 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:50:15.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:15.112 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:50:15.185 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:50:15.185 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:15.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:50:15.249 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:50:15.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:15.250 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:50:15.304 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:50:15.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:50:15.307 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:50:15.307 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:15.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:50:15.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:50:15.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:15.370 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:50:15.423 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:50:15.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:50:15.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:15.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.648+0000 7f409dc21640 1 Processor -- start 2023-08-25T09:50:15.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.648+0000 7f409dc21640 1 -- start start 2023-08-25T09:50:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.649+0000 7f409dc21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 0x7f40980b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.649+0000 7f409dc21640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f40980b12f0 0x7f409815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:15.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f409dc21640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409815e7c0 con 0x7f409815f190 2023-08-25T09:50:15.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f409dc21640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4098071b50 con 0x7f40980b12f0 2023-08-25T09:50:15.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f409dc21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409815e490 con 0x7f409815a7c0 2023-08-25T09:50:15.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f4097fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f40980b12f0 0x7f409815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:15.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f409cc1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 0x7f40980b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:15.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f409cc1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 0x7f40980b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:36602/0 (socket says 172.21.15.70:36602) 2023-08-25T09:50:15.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.650+0000 7f409cc1f640 1 -- 172.21.15.70:0/284630501 learned_addr learned my addr 172.21.15.70:0/284630501 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:15.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.651+0000 7f409cc1f640 1 -- 172.21.15.70:0/284630501 >> v2:172.21.15.155:3300/0 conn(0x7f40980b12f0 msgr2=0x7f409815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:15.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.651+0000 7f409cc1f640 1 --2- 172.21.15.70:0/284630501 >> v2:172.21.15.155:3300/0 conn(0x7f40980b12f0 0x7f409815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.651+0000 7f409cc1f640 1 -- 172.21.15.70:0/284630501 >> v1:172.21.15.155:6789/0 conn(0x7f409815f190 legacy=0x7f40980cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:15.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.651+0000 7f409cc1f640 1 -- 172.21.15.70:0/284630501 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40980b3aa0 con 0x7f409815a7c0 2023-08-25T09:50:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.652+0000 7f409cc1f640 1 --2- 172.21.15.70:0/284630501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 0x7f40980b3560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f408c009870 tx=0x7f408c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4258f905b52716be server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.653+0000 7f40977fe640 1 -- 172.21.15.70:0/284630501 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c030840 con 0x7f409815a7c0 2023-08-25T09:50:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.653+0000 7f40977fe640 1 -- 172.21.15.70:0/284630501 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c0309a0 con 0x7f409815a7c0 2023-08-25T09:50:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.653+0000 7f40977fe640 1 -- 172.21.15.70:0/284630501 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408c030d30 con 0x7f409815a7c0 2023-08-25T09:50:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 -- 172.21.15.70:0/284630501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 msgr2=0x7f40980b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:15.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 --2- 172.21.15.70:0/284630501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 0x7f40980b3560 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f408c009870 tx=0x7f408c02b980 comp rx=0 tx=0).stop 2023-08-25T09:50:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 -- 172.21.15.70:0/284630501 shutdown_connections 2023-08-25T09:50:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 --2- 172.21.15.70:0/284630501 >> v2:172.21.15.155:3300/0 conn(0x7f40980b12f0 0x7f409815ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 --2- 172.21.15.70:0/284630501 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f409815a7c0 0x7f40980b3560 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 -- 172.21.15.70:0/284630501 >> 172.21.15.70:0/284630501 conn(0x7f40980affd0 msgr2=0x7f40980b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 -- 172.21.15.70:0/284630501 shutdown_connections 2023-08-25T09:50:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.654+0000 7f409dc21640 1 -- 172.21.15.70:0/284630501 wait complete. 2023-08-25T09:50:15.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.656+0000 7f409dc21640 1 Processor -- start 2023-08-25T09:50:15.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.656+0000 7f409dc21640 1 -- start start 2023-08-25T09:50:15.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409dc21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40980b12f0 0x7f40980c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:15.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409dc21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 0x7f40980c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:15.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409dc21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409815f190 0x7f409814e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:15.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409dc21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409806fd30 con 0x7f409815a7c0 2023-08-25T09:50:15.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409dc21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4098070010 con 0x7f409815f190 2023-08-25T09:50:15.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409dc21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40980702f0 con 0x7f40980b12f0 2023-08-25T09:50:15.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f4097fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 0x7f40980c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f4097fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 0x7f40980c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40076/0 (socket says 172.21.15.70:40076) 2023-08-25T09:50:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f4097fff640 1 -- 172.21.15.70:0/1891947694 learned_addr learned my addr 172.21.15.70:0/1891947694 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409cc1f640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40980b12f0 0x7f40980c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:15.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f409d420640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409815f190 0x7f409814e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f4097fff640 1 -- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40980b12f0 msgr2=0x7f40980c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.658+0000 7f4097fff640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40980b12f0 0x7f40980c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f4097fff640 1 -- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409815f190 msgr2=0x7f409814e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f4097fff640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409815f190 0x7f409814e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f4097fff640 1 -- 172.21.15.70:0/1891947694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408c0094d0 con 0x7f409815a7c0 2023-08-25T09:50:15.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f409d420640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409815f190 0x7f409814e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:15.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f4097fff640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 0x7f40980c57e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f408800e870 tx=0x7f408800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:15.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880094f0 con 0x7f409815a7c0 2023-08-25T09:50:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408800c940 con 0x7f409815a7c0 2023-08-25T09:50:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408800ccf0 con 0x7f409815a7c0 2023-08-25T09:50:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40980705d0 con 0x7f409815a7c0 2023-08-25T09:50:15.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.659+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40980c5d20 con 0x7f409815a7c0 2023-08-25T09:50:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.661+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4098066fb0 con 0x7f409815a7c0 2023-08-25T09:50:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.661+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4088013470 con 0x7f409815a7c0 2023-08-25T09:50:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.662+0000 7f40957fa640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f408406a050 0x7f408406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:15.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.662+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f40880886d0 con 0x7f409815a7c0 2023-08-25T09:50:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.663+0000 7f409cc1f640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f408406a050 0x7f408406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.663+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f4084072380 con 0x7f409815a7c0 2023-08-25T09:50:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.663+0000 7f409cc1f640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f408406a050 0x7f408406c510 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f408c036000 tx=0x7f408c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:15.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.667+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4088053d30 con 0x7f409815a7c0 2023-08-25T09:50:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.894+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40980c2ef0 con 0x7f409815a7c0 2023-08-25T09:50:15.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.895+0000 7f40957fa640 1 -- 172.21.15.70:0/1891947694 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f4088021020 con 0x7f409815a7c0 2023-08-25T09:50:15.897 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:15.897 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:50:15.897 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:50:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f408406a050 msgr2=0x7f408406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f408406a050 0x7f408406c510 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f408c036000 tx=0x7f408c008030 comp rx=0 tx=0).stop 2023-08-25T09:50:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 msgr2=0x7f40980c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:15.900 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 0x7f40980c57e0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f408800e870 tx=0x7f408800ed30 comp rx=0 tx=0).stop 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 shutdown_connections 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f408406a050 0x7f408406c510 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f409815f190 0x7f409814e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f409815a7c0 0x7f40980c57e0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 --2- 172.21.15.70:0/1891947694 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f40980b12f0 0x7f40980c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 >> 172.21.15.70:0/1891947694 conn(0x7f40980affd0 msgr2=0x7f40980c9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.898+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 shutdown_connections 2023-08-25T09:50:15.902 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:15.899+0000 7f409dc21640 1 -- 172.21.15.70:0/1891947694 wait complete. 2023-08-25T09:50:15.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.136+0000 7fe34e96b640 1 Processor -- start 2023-08-25T09:50:16.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.136+0000 7fe34e96b640 1 -- start start 2023-08-25T09:50:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34e96b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 0x7fe3480b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34e96b640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe34806f0a0 0x7fe3480b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34e96b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34806ae10 con 0x7fe348159100 2023-08-25T09:50:16.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34e96b640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34806ba50 con 0x7fe34806f0a0 2023-08-25T09:50:16.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34e96b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34806b0b0 con 0x7fe3480709a0 2023-08-25T09:50:16.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34cf08640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe34806f0a0 0x7fe3480b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe33f7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 0x7fe3480b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe347fff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fe348159100 0x7fe34806be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:59664/0 (socket says 172.21.15.70:59664) 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe347fff640 1 -- 172.21.15.70:0/1112307665 learned_addr learned my addr 172.21.15.70:0/1112307665 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.138+0000 7fe34cf08640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe34806f0a0 0x7fe3480b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:33336/0 (socket says 172.21.15.70:33336) 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe33f7fe640 1 --2- 172.21.15.70:0/1112307665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 0x7fe3480b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:36622/0 (socket says 172.21.15.70:36622) 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe33f7fe640 1 -- 172.21.15.70:0/1112307665 >> v2:172.21.15.155:3300/0 conn(0x7fe34806f0a0 msgr2=0x7fe3480b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:16.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe33f7fe640 1 --2- 172.21.15.70:0/1112307665 >> v2:172.21.15.155:3300/0 conn(0x7fe34806f0a0 0x7fe3480b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe33f7fe640 1 -- 172.21.15.70:0/1112307665 >> v1:172.21.15.155:6789/0 conn(0x7fe348159100 legacy=0x7fe34806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.139+0000 7fe33f7fe640 1 -- 172.21.15.70:0/1112307665 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34806b390 con 0x7fe3480709a0 2023-08-25T09:50:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.141+0000 7fe33f7fe640 1 --2- 172.21.15.70:0/1112307665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 0x7fe3480b1960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb7c8ead4f0788b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:16.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.141+0000 7fe3477fe640 1 -- 172.21.15.70:0/1112307665 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33002e070 con 0x7fe3480709a0 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.141+0000 7fe3477fe640 1 -- 172.21.15.70:0/1112307665 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3300315a0 con 0x7fe3480709a0 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.141+0000 7fe3477fe640 1 -- 172.21.15.70:0/1112307665 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330031930 con 0x7fe3480709a0 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 -- 172.21.15.70:0/1112307665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 msgr2=0x7fe3480b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 --2- 172.21.15.70:0/1112307665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 0x7fe3480b1960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097a0 tx=0x7fe33002b840 comp rx=0 tx=0).stop 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 -- 172.21.15.70:0/1112307665 shutdown_connections 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 --2- 172.21.15.70:0/1112307665 >> v2:172.21.15.155:3300/0 conn(0x7fe34806f0a0 0x7fe3480b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 --2- 172.21.15.70:0/1112307665 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3480709a0 0x7fe3480b1960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 -- 172.21.15.70:0/1112307665 >> 172.21.15.70:0/1112307665 conn(0x7fe348069850 msgr2=0x7fe348069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 -- 172.21.15.70:0/1112307665 shutdown_connections 2023-08-25T09:50:16.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.142+0000 7fe34e96b640 1 -- 172.21.15.70:0/1112307665 wait complete. 2023-08-25T09:50:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.144+0000 7fe34e96b640 1 Processor -- start 2023-08-25T09:50:16.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.144+0000 7fe34e96b640 1 -- start start 2023-08-25T09:50:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34e96b640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe34806f0a0 0x7fe34813d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34e96b640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3480709a0 0x7fe3480ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34e96b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 0x7fe3480acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34e96b640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34813d640 con 0x7fe348159100 2023-08-25T09:50:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34e96b640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34814e920 con 0x7fe3480709a0 2023-08-25T09:50:16.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34e96b640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34814ec00 con 0x7fe34806f0a0 2023-08-25T09:50:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 0x7fe3480acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 0x7fe3480acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40078/0 (socket says 172.21.15.70:40078) 2023-08-25T09:50:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 -- 172.21.15.70:0/126053327 learned_addr learned my addr 172.21.15.70:0/126053327 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe33f7fe640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3480709a0 0x7fe3480ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe347fff640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe34806f0a0 0x7fe34813d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 -- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe34806f0a0 msgr2=0x7fe34813d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe34806f0a0 0x7fe34813d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 -- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3480709a0 msgr2=0x7fe3480ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3480709a0 0x7fe3480ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe34cf08640 1 -- 172.21.15.70:0/126053327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe330009400 con 0x7fe348159100 2023-08-25T09:50:16.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.145+0000 7fe347fff640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe34806f0a0 0x7fe34813d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.146+0000 7fe34cf08640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 0x7fe3480acdb0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe34000e930 tx=0x7fe34000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.146+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340009690 con 0x7fe348159100 2023-08-25T09:50:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.146+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34000c980 con 0x7fe348159100 2023-08-25T09:50:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.146+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34000cd30 con 0x7fe348159100 2023-08-25T09:50:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.146+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34814eee0 con 0x7fe348159100 2023-08-25T09:50:16.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.146+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34814f4a0 con 0x7fe348159100 2023-08-25T09:50:16.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.147+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34814f6c0 con 0x7fe348159100 2023-08-25T09:50:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.148+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe340013470 con 0x7fe348159100 2023-08-25T09:50:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.149+0000 7fe3457fa640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe32806a050 0x7fe32806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.149+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe3400888d0 con 0x7fe348159100 2023-08-25T09:50:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.149+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7fe328072380 con 0x7fe348159100 2023-08-25T09:50:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.150+0000 7fe347fff640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe32806a050 0x7fe32806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.150+0000 7fe347fff640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe32806a050 0x7fe32806c510 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe338005c00 tx=0x7fe338009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:16.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.154+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3400579e0 con 0x7fe348159100 2023-08-25T09:50:16.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.343+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe34804e420 con 0x7fe348159100 2023-08-25T09:50:16.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.346+0000 7fe3457fa640 1 -- 172.21.15.70:0/126053327 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+18637 (secure 0 0 0) 0x7fe340021070 con 0x7fe348159100 2023-08-25T09:50:16.348 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:16.348 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":169,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:49:22.433958+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"20.13","mappings":[{"from":3,"to":7}]},{"pgid":"20.1d","mappings":[{"from":3,"to":1}]},{"pgid":"20.1e","mappings":[{"from":3,"to":7}]},{"pgid":"20.38","mappings":[{"from":3,"to":7}]},{"pgid":"20.3c","mappings":[{"from":3,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:16.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe32806a050 msgr2=0x7fe32806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe32806a050 0x7fe32806c510 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe338005c00 tx=0x7fe338009040 comp rx=0 tx=0).stop 2023-08-25T09:50:16.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 msgr2=0x7fe3480acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 0x7fe3480acdb0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe34000e930 tx=0x7fe34000edf0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 shutdown_connections 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe32806a050 0x7fe32806c510 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe348159100 0x7fe3480acdb0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3480709a0 0x7fe3480ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 --2- 172.21.15.70:0/126053327 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe34806f0a0 0x7fe34813d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 >> 172.21.15.70:0/126053327 conn(0x7fe348069850 msgr2=0x7fe34806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:16.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 shutdown_connections 2023-08-25T09:50:16.352 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.349+0000 7fe34e96b640 1 -- 172.21.15.70:0/126053327 wait complete. 2023-08-25T09:50:16.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:48:06.570383+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:48:07.289165+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:50:16.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:50:16.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.588+0000 7f09088b7640 1 Processor -- start 2023-08-25T09:50:16.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.588+0000 7f09088b7640 1 -- start start 2023-08-25T09:50:16.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f09088b7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0904162630 0x7f0904164a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f09088b7640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09040cd800 con 0x7f09040b2470 2023-08-25T09:50:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f09088b7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090415a110 con 0x7f09040b2850 2023-08-25T09:50:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f09088b7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090415ad20 con 0x7f0904162630 2023-08-25T09:50:16.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f0902d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f09040b2470 0x7f0904159930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59872/0 (socket says 172.21.15.70:59872) 2023-08-25T09:50:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f0902d9d640 1 -- 172.21.15.70:0/3628494298 learned_addr learned my addr 172.21.15.70:0/3628494298 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f090359e640 1 --2- 172.21.15.70:0/3628494298 >> v2:172.21.15.70:3300/0 conn(0x7f0904162630 0x7f0904164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f090259c640 1 -- 172.21.15.70:0/3628494298 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694240403 0 0) 0x7f09040cd800 con 0x7f09040b2470 2023-08-25T09:50:16.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f090359e640 1 -- 172.21.15.70:0/3628494298 >> v1:172.21.15.155:6789/0 conn(0x7f09040b2850 legacy=0x7f09040cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f090359e640 1 -- 172.21.15.70:0/3628494298 >> v1:172.21.15.70:6789/0 conn(0x7f09040b2470 legacy=0x7f0904159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.590+0000 7f090359e640 1 -- 172.21.15.70:0/3628494298 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090415a3f0 con 0x7f0904162630 2023-08-25T09:50:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.592+0000 7f090359e640 1 --2- 172.21.15.70:0/3628494298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0904162630 0x7f0904164a20 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d90 tx=0x7f08fc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e4caccf5ebe66d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.592+0000 7f090259c640 1 -- 172.21.15.70:0/3628494298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc02cc60 con 0x7f0904162630 2023-08-25T09:50:16.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.592+0000 7f090259c640 1 -- 172.21.15.70:0/3628494298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc02cdc0 con 0x7f0904162630 2023-08-25T09:50:16.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.592+0000 7f090259c640 1 -- 172.21.15.70:0/3628494298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc034660 con 0x7f0904162630 2023-08-25T09:50:16.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 -- 172.21.15.70:0/3628494298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0904162630 msgr2=0x7f0904164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 --2- 172.21.15.70:0/3628494298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0904162630 0x7f0904164a20 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f08fc009d90 tx=0x7f08fc02beb0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 -- 172.21.15.70:0/3628494298 shutdown_connections 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 --2- 172.21.15.70:0/3628494298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0904162630 0x7f0904164a20 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 -- 172.21.15.70:0/3628494298 >> 172.21.15.70:0/3628494298 conn(0x7f09040ce830 msgr2=0x7f09040cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 -- 172.21.15.70:0/3628494298 shutdown_connections 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.593+0000 7f09088b7640 1 -- 172.21.15.70:0/3628494298 wait complete. 2023-08-25T09:50:16.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.595+0000 7f09088b7640 1 Processor -- start 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.595+0000 7f09088b7640 1 -- start start 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f09088b7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 0x7f09041f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f09088b7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09040b2850 0x7f09040c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f09088b7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0904162630 0x7f09041f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f09088b7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090406ea00 con 0x7f09040b2470 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f09088b7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090406ece0 con 0x7f09040b2850 2023-08-25T09:50:16.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f09088b7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090406efc0 con 0x7f0904162630 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f0902d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 0x7f09041f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f0902d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 0x7f09041f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40096/0 (socket says 172.21.15.70:40096) 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.596+0000 7f0902d9d640 1 -- 172.21.15.70:0/216272276 learned_addr learned my addr 172.21.15.70:0/216272276 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f0902d9d640 1 -- 172.21.15.70:0/216272276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0904162630 msgr2=0x7f09041f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f0902d9d640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0904162630 0x7f09041f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f0902d9d640 1 -- 172.21.15.70:0/216272276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09040b2850 msgr2=0x7f09040c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f0902d9d640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09040b2850 0x7f09040c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f0902d9d640 1 -- 172.21.15.70:0/216272276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc009a70 con 0x7f09040b2470 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f0902d9d640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 0x7f09041f3320 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c4d0 tx=0x7f08f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4007480 con 0x7f09040b2470 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f40075e0 con 0x7f09040b2470 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f40059a0 con 0x7f09040b2470 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.597+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090406f2a0 con 0x7f09040b2470 2023-08-25T09:50:16.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.598+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09040c4890 con 0x7f09040b2470 2023-08-25T09:50:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.599+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09040b2cb0 con 0x7f09040b2470 2023-08-25T09:50:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.600+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08f4005b00 con 0x7f09040b2470 2023-08-25T09:50:16.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.601+0000 7f08fb7fe640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a050 0x7f08e406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.601+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f08f40881b0 con 0x7f09040b2470 2023-08-25T09:50:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.601+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f08e4072380 con 0x7f09040b2470 2023-08-25T09:50:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.601+0000 7f08fbfff640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a050 0x7f08e406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.602+0000 7f08fbfff640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a050 0x7f08e406c510 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f09040abc40 tx=0x7f08ec0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:16.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.605+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09040b2cb0 con 0x7f09040b2470 2023-08-25T09:50:16.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.795+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09040b2cb0 con 0x7f09040b2470 2023-08-25T09:50:16.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.796+0000 7f08fb7fe640 1 -- 172.21.15.70:0/216272276 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09040b2cb0 con 0x7f09040b2470 2023-08-25T09:50:16.798 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:50:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a050 msgr2=0x7f08e406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a050 0x7f08e406c510 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f09040abc40 tx=0x7f08ec0079b0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 msgr2=0x7f09041f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 0x7f09041f3320 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c4d0 tx=0x7f08f400c990 comp rx=0 tx=0).stop 2023-08-25T09:50:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 shutdown_connections 2023-08-25T09:50:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08e406a050 0x7f08e406c510 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0904162630 0x7f09041f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f09040b2850 0x7f09040c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 --2- 172.21.15.70:0/216272276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f09040b2470 0x7f09041f3320 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.798+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 >> 172.21.15.70:0/216272276 conn(0x7f09040ce830 msgr2=0x7f090406a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.799+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 shutdown_connections 2023-08-25T09:50:16.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:16.799+0000 7f09088b7640 1 -- 172.21.15.70:0/216272276 wait complete. 2023-08-25T09:50:16.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:50:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.034+0000 7f9d95a57640 1 Processor -- start 2023-08-25T09:50:17.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.034+0000 7f9d95a57640 1 -- start start 2023-08-25T09:50:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.035+0000 7f9d95a57640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f9d90069f60 0x7f9d900ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.035+0000 7f9d95a57640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900c2550 con 0x7f9d9015ae50 2023-08-25T09:50:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.035+0000 7f9d95a57640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900ce330 con 0x7f9d900c2110 2023-08-25T09:50:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.035+0000 7f9d95a57640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900cc5f0 con 0x7f9d90069f60 2023-08-25T09:50:17.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9d9015ae50 0x7f9d900cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59888/0 (socket says 172.21.15.70:59888) 2023-08-25T09:50:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8effd640 1 -- 172.21.15.70:0/59308658 learned_addr learned my addr 172.21.15.70:0/59308658 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:17.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8f7fe640 1 --2- 172.21.15.70:0/59308658 >> v2:172.21.15.70:3300/0 conn(0x7f9d90069f60 0x7f9d900ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8e7fc640 1 -- 172.21.15.70:0/59308658 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061414985 0 0) 0x7f9d900c2550 con 0x7f9d9015ae50 2023-08-25T09:50:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8e7fc640 1 -- 172.21.15.70:0/59308658 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7c003520 con 0x7f9d9015ae50 2023-08-25T09:50:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8f7fe640 1 -- 172.21.15.70:0/59308658 >> v1:172.21.15.155:6789/0 conn(0x7f9d900c2110 legacy=0x7f9d90069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8f7fe640 1 -- 172.21.15.70:0/59308658 >> v1:172.21.15.70:6789/0 conn(0x7f9d9015ae50 legacy=0x7f9d900cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.036+0000 7f9d8f7fe640 1 -- 172.21.15.70:0/59308658 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90070650 con 0x7f9d90069f60 2023-08-25T09:50:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.037+0000 7f9d8f7fe640 1 --2- 172.21.15.70:0/59308658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d90069f60 0x7f9d900ccad0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009d90 tx=0x7f9d8802bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bc8d753ce56f51 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d8e7fc640 1 -- 172.21.15.70:0/59308658 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8802cd80 con 0x7f9d90069f60 2023-08-25T09:50:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d8e7fc640 1 -- 172.21.15.70:0/59308658 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d88034440 con 0x7f9d90069f60 2023-08-25T09:50:17.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d8e7fc640 1 -- 172.21.15.70:0/59308658 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d880347b0 con 0x7f9d90069f60 2023-08-25T09:50:17.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d95a57640 1 -- 172.21.15.70:0/59308658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d90069f60 msgr2=0x7f9d900ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d95a57640 1 --2- 172.21.15.70:0/59308658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d90069f60 0x7f9d900ccad0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f9d88009d90 tx=0x7f9d8802bfd0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d95a57640 1 -- 172.21.15.70:0/59308658 shutdown_connections 2023-08-25T09:50:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d95a57640 1 --2- 172.21.15.70:0/59308658 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d90069f60 0x7f9d900ccad0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.038+0000 7f9d95a57640 1 -- 172.21.15.70:0/59308658 >> 172.21.15.70:0/59308658 conn(0x7f9d900cb0d0 msgr2=0x7f9d900cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.039+0000 7f9d95a57640 1 -- 172.21.15.70:0/59308658 shutdown_connections 2023-08-25T09:50:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.039+0000 7f9d95a57640 1 -- 172.21.15.70:0/59308658 wait complete. 2023-08-25T09:50:17.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.040+0000 7f9d95a57640 1 Processor -- start 2023-08-25T09:50:17.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.040+0000 7f9d95a57640 1 -- start start 2023-08-25T09:50:17.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.041+0000 7f9d95a57640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d90069f60 0x7f9d900c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.041+0000 7f9d95a57640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 0x7f9d900afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.041+0000 7f9d95a57640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9015ae50 0x7f9d900b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.041+0000 7f9d95a57640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900aecc0 con 0x7f9d900c2110 2023-08-25T09:50:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.041+0000 7f9d95a57640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900aefa0 con 0x7f9d90069f60 2023-08-25T09:50:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.041+0000 7f9d95a57640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900af280 con 0x7f9d9015ae50 2023-08-25T09:50:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 0x7f9d900afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 0x7f9d900afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40110/0 (socket says 172.21.15.70:40110) 2023-08-25T09:50:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 -- 172.21.15.70:0/2605857514 learned_addr learned my addr 172.21.15.70:0/2605857514 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d8effd640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d90069f60 0x7f9d900c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d8f7fe640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9015ae50 0x7f9d900b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 -- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9015ae50 msgr2=0x7f9d900b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9015ae50 0x7f9d900b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 -- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d90069f60 msgr2=0x7f9d900c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d90069f60 0x7f9d900c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.042+0000 7f9d87fff640 1 -- 172.21.15.70:0/2605857514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d88009a70 con 0x7f9d900c2110 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.043+0000 7f9d87fff640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 0x7f9d900afd60 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c400 tx=0x7f9d7800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.043+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d780052d0 con 0x7f9d900c2110 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.043+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78005450 con 0x7f9d900c2110 2023-08-25T09:50:17.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.043+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78007260 con 0x7f9d900c2110 2023-08-25T09:50:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.043+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900af560 con 0x7f9d900c2110 2023-08-25T09:50:17.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.043+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d900c61a0 con 0x7f9d900c2110 2023-08-25T09:50:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.044+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d900afa80 con 0x7f9d900c2110 2023-08-25T09:50:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.045+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d780073c0 con 0x7f9d900c2110 2023-08-25T09:50:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.046+0000 7f9d8cff9640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d7006a050 0x7f9d7006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.046+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f9d78087a60 con 0x7f9d900c2110 2023-08-25T09:50:17.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.046+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=170}) v3 -- 0x7f9d70072380 con 0x7f9d900c2110 2023-08-25T09:50:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.047+0000 7f9d8effd640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d7006a050 0x7f9d7006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.047+0000 7f9d8effd640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d7006a050 0x7f9d7006c510 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9d800011e0 tx=0x7f9d80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.051+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d900afa80 con 0x7f9d900c2110 2023-08-25T09:50:17.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.237+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9d900afa80 con 0x7f9d900c2110 2023-08-25T09:50:17.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.238+0000 7f9d8cff9640 1 -- 172.21.15.70:0/2605857514 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v169) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d900afa80 con 0x7f9d900c2110 2023-08-25T09:50:17.240 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:50:17.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d7006a050 msgr2=0x7f9d7006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d7006a050 0x7f9d7006c510 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9d800011e0 tx=0x7f9d80009040 comp rx=0 tx=0).stop 2023-08-25T09:50:17.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 msgr2=0x7f9d900afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 0x7f9d900afd60 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800c400 tx=0x7f9d7800c8c0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 shutdown_connections 2023-08-25T09:50:17.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f9d7006a050 0x7f9d7006c510 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9d9015ae50 0x7f9d900b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9d900c2110 0x7f9d900afd60 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 --2- 172.21.15.70:0/2605857514 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9d90069f60 0x7f9d900c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 >> 172.21.15.70:0/2605857514 conn(0x7f9d900cb0d0 msgr2=0x7f9d90159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:17.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 shutdown_connections 2023-08-25T09:50:17.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.241+0000 7f9d95a57640 1 -- 172.21.15.70:0/2605857514 wait complete. 2023-08-25T09:50:17.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:50:17.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:50:17.271 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:50:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.452+0000 7f73c2f0c640 1 Processor -- start 2023-08-25T09:50:17.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.452+0000 7f73c2f0c640 1 -- start start 2023-08-25T09:50:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.453+0000 7f73c2f0c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f73bc069f60 0x7f73bc0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.453+0000 7f73c2f0c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc0c2550 con 0x7f73bc15ae50 2023-08-25T09:50:17.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73c2f0c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc0ce330 con 0x7f73bc0c2110 2023-08-25T09:50:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73c2f0c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc0cc5f0 con 0x7f73bc069f60 2023-08-25T09:50:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73c0ca8640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f73bc15ae50 0x7f73bc0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59894/0 (socket says 172.21.15.70:59894) 2023-08-25T09:50:17.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73c0ca8640 1 -- 172.21.15.70:0/1331222934 learned_addr learned my addr 172.21.15.70:0/1331222934 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73c14a9640 1 --2- 172.21.15.70:0/1331222934 >> v2:172.21.15.155:3300/0 conn(0x7f73bc069f60 0x7f73bc0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199282619 0 0) 0x7f73bc0c2550 con 0x7f73bc15ae50 2023-08-25T09:50:17.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.454+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a4003520 con 0x7f73bc15ae50 2023-08-25T09:50:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3371369890 0 0) 0x7f73a4003520 con 0x7f73bc15ae50 2023-08-25T09:50:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 >> v2:172.21.15.155:3300/0 conn(0x7f73bc069f60 msgr2=0x7f73bc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 --2- 172.21.15.70:0/1331222934 >> v2:172.21.15.155:3300/0 conn(0x7f73bc069f60 0x7f73bc0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 >> v1:172.21.15.155:6789/0 conn(0x7f73bc0c2110 legacy=0x7f73bc069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc070650 con 0x7f73bc15ae50 2023-08-25T09:50:17.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f73b80019f0 con 0x7f73bc15ae50 2023-08-25T09:50:17.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 >> v1:172.21.15.70:6789/0 conn(0x7f73bc15ae50 legacy=0x7f73bc0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a4003970 0x7f73a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a40047c0 0x7f73a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73a4005180 0x7f73a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0cc5f0 con 0x7f73a4005180 2023-08-25T09:50:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a400c030 con 0x7f73a40047c0 2023-08-25T09:50:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a400c1f0 con 0x7f73a4003970 2023-08-25T09:50:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73c14a9640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73a4005180 0x7f73a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.455+0000 7f73b3fff640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a40047c0 0x7f73a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.456+0000 7f73c0ca8640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a4003970 0x7f73a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.456+0000 7f73c14a9640 1 -- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a4003970 msgr2=0x7f73a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.456+0000 7f73c14a9640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73a4003970 0x7f73a4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.456+0000 7f73c14a9640 1 -- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a40047c0 msgr2=0x7f73a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.456+0000 7f73c14a9640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a40047c0 0x7f73a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.456+0000 7f73c14a9640 1 -- 172.21.15.70:0/1331222934 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a4003520 con 0x7f73a4005180 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.457+0000 7f73c14a9640 1 -- 172.21.15.70:0/1331222934 reap_dead start 2023-08-25T09:50:17.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.457+0000 7f73c14a9640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73a4005180 0x7f73a400b980 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f73bc15e980 tx=0x7f73a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b6926a5fb69e69 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.457+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b402c680 con 0x7f73a4005180 2023-08-25T09:50:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.457+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b402c800 con 0x7f73a4005180 2023-08-25T09:50:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.457+0000 7f73b37fe640 1 -- 172.21.15.70:0/1331222934 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b402cbb0 con 0x7f73a4005180 2023-08-25T09:50:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 -- 172.21.15.70:0/1331222934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73a4005180 msgr2=0x7f73a400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73a4005180 0x7f73a400b980 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f73bc15e980 tx=0x7f73a40057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 -- 172.21.15.70:0/1331222934 shutdown_connections 2023-08-25T09:50:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/1331222934 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73a4005180 0x7f73a400b980 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 -- 172.21.15.70:0/1331222934 >> 172.21.15.70:0/1331222934 conn(0x7f73bc0cb0d0 msgr2=0x7f73bc0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 -- 172.21.15.70:0/1331222934 shutdown_connections 2023-08-25T09:50:17.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.458+0000 7f73c2f0c640 1 -- 172.21.15.70:0/1331222934 wait complete. 2023-08-25T09:50:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.461+0000 7f73c2f0c640 1 Processor -- start 2023-08-25T09:50:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.462+0000 7f73c2f0c640 1 -- start start 2023-08-25T09:50:17.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c2f0c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4005180 0x7f73bc154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c2f0c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc157f50 0x7f73bc0c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c2f0c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 0x7f73bc156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c2f0c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc1571c0 con 0x7f73bc1583d0 2023-08-25T09:50:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c2f0c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0c5830 con 0x7f73a4005180 2023-08-25T09:50:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c2f0c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0c5b10 con 0x7f73bc157f50 2023-08-25T09:50:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c14a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 0x7f73bc156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c14a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 0x7f73bc156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40118/0 (socket says 172.21.15.70:40118) 2023-08-25T09:50:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.463+0000 7f73c14a9640 1 -- 172.21.15.70:0/2438721655 learned_addr learned my addr 172.21.15.70:0/2438721655 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.464+0000 7f73c14a9640 1 -- 172.21.15.70:0/2438721655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc157f50 msgr2=0x7f73bc0c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.464+0000 7f73c14a9640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc157f50 0x7f73bc0c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.464+0000 7f73c14a9640 1 -- 172.21.15.70:0/2438721655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4005180 msgr2=0x7f73bc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.464+0000 7f73c14a9640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4005180 0x7f73bc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.464+0000 7f73c14a9640 1 -- 172.21.15.70:0/2438721655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0c5df0 con 0x7f73bc1583d0 2023-08-25T09:50:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.464+0000 7f73c14a9640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 0x7f73bc156250 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f73b4037040 tx=0x7f73b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.465+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4034990 con 0x7f73bc1583d0 2023-08-25T09:50:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.465+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b4034af0 con 0x7f73bc1583d0 2023-08-25T09:50:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.465+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4034e60 con 0x7f73bc1583d0 2023-08-25T09:50:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.465+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc0c60d0 con 0x7f73bc1583d0 2023-08-25T09:50:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.465+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc154e90 con 0x7f73bc1583d0 2023-08-25T09:50:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.466+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc0c65f0 con 0x7f73bc1583d0 2023-08-25T09:50:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.467+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73b403b040 con 0x7f73bc1583d0 2023-08-25T09:50:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.468+0000 7f73b17fa640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.468+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f73b40a93e0 con 0x7f73bc1583d0 2023-08-25T09:50:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.469+0000 7f73c0ca8640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.469+0000 7f73c0ca8640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f73b80012e0 tx=0x7f73b80014d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.473+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73bc0c65f0 con 0x7f73bc1583d0 2023-08-25T09:50:17.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.656+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73bc0c65f0 con 0x7f73bc1583d0 2023-08-25T09:50:17.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.656+0000 7f73b17fa640 1 -- 172.21.15.70:0/2438721655 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v170) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73bc0c65f0 con 0x7f73bc1583d0 2023-08-25T09:50:17.658 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:50:17.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.658+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 msgr2=0x7f73a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.658+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f73b80012e0 tx=0x7f73b80014d0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.658+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 msgr2=0x7f73bc156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.658+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 0x7f73bc156250 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f73b4037040 tx=0x7f73b4004730 comp rx=0 tx=0).stop 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 shutdown_connections 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc1583d0 0x7f73bc156250 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc157f50 0x7f73bc0c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 --2- 172.21.15.70:0/2438721655 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73a4005180 0x7f73bc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 >> 172.21.15.70:0/2438721655 conn(0x7f73bc0cb0d0 msgr2=0x7f73bc159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 shutdown_connections 2023-08-25T09:50:17.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.659+0000 7f73c2f0c640 1 -- 172.21.15.70:0/2438721655 wait complete. 2023-08-25T09:50:17.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:50:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.882+0000 7fd08b470640 1 Processor -- start 2023-08-25T09:50:17.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.883+0000 7fd08b470640 1 -- start start 2023-08-25T09:50:17.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.884+0000 7fd08b470640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd084159bf0 0x7fd08415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.884+0000 7fd08b470640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd08406f0e0 0x7fd084070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.884+0000 7fd08b470640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08415e780 con 0x7fd0840cd660 2023-08-25T09:50:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.884+0000 7fd08b470640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08406f8e0 con 0x7fd084159bf0 2023-08-25T09:50:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.884+0000 7fd08b470640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08406fbc0 con 0x7fd08406f0e0 2023-08-25T09:50:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd084159bf0 0x7fd08415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd084159bf0 0x7fd08415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40128/0 (socket says 172.21.15.70:40128) 2023-08-25T09:50:17.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 -- 172.21.15.70:0/2878680937 learned_addr learned my addr 172.21.15.70:0/2878680937 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd089a0d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd0840cd660 0x7fd0840c8eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59896/0 (socket says 172.21.15.70:59896) 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd088a0b640 1 --2- 172.21.15.70:0/2878680937 >> v2:172.21.15.155:3300/0 conn(0x7fd08406f0e0 0x7fd084070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 -- 172.21.15.70:0/2878680937 >> v2:172.21.15.155:3300/0 conn(0x7fd08406f0e0 msgr2=0x7fd084070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 --2- 172.21.15.70:0/2878680937 >> v2:172.21.15.155:3300/0 conn(0x7fd08406f0e0 0x7fd084070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 -- 172.21.15.70:0/2878680937 >> v1:172.21.15.70:6789/0 conn(0x7fd0840cd660 legacy=0x7fd0840c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd08920c640 1 -- 172.21.15.70:0/2878680937 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08406fea0 con 0x7fd084159bf0 2023-08-25T09:50:17.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.885+0000 7fd07bfff640 1 -- 172.21.15.70:0/2878680937 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064287936 0 0) 0x7fd08415e780 con 0x7fd0840cd660 2023-08-25T09:50:17.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.886+0000 7fd08920c640 1 --2- 172.21.15.70:0/2878680937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd084159bf0 0x7fd08415e240 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd08406af90 tx=0x7fd08002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fe65a8dd8f9a96 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.887+0000 7fd07bfff640 1 -- 172.21.15.70:0/2878680937 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080036440 con 0x7fd084159bf0 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.887+0000 7fd07bfff640 1 -- 172.21.15.70:0/2878680937 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0800365a0 con 0x7fd084159bf0 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.887+0000 7fd07bfff640 1 -- 172.21.15.70:0/2878680937 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080036930 con 0x7fd084159bf0 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.887+0000 7fd08b470640 1 -- 172.21.15.70:0/2878680937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd084159bf0 msgr2=0x7fd08415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.887+0000 7fd08b470640 1 --2- 172.21.15.70:0/2878680937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd084159bf0 0x7fd08415e240 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fd08406af90 tx=0x7fd08002e170 comp rx=0 tx=0).stop 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.888+0000 7fd08b470640 1 -- 172.21.15.70:0/2878680937 shutdown_connections 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.888+0000 7fd08b470640 1 --2- 172.21.15.70:0/2878680937 >> v2:172.21.15.155:3300/0 conn(0x7fd08406f0e0 0x7fd084070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.888+0000 7fd08b470640 1 --2- 172.21.15.70:0/2878680937 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd084159bf0 0x7fd08415e240 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.888+0000 7fd08b470640 1 -- 172.21.15.70:0/2878680937 >> 172.21.15.70:0/2878680937 conn(0x7fd0840cb0d0 msgr2=0x7fd0840cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.888+0000 7fd08b470640 1 -- 172.21.15.70:0/2878680937 shutdown_connections 2023-08-25T09:50:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.888+0000 7fd08b470640 1 -- 172.21.15.70:0/2878680937 wait complete. 2023-08-25T09:50:17.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.890+0000 7fd08b470640 1 Processor -- start 2023-08-25T09:50:17.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.890+0000 7fd08b470640 1 -- start start 2023-08-25T09:50:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.891+0000 7fd08b470640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd08406f0e0 0x7fd08414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.891+0000 7fd08b470640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 0x7fd08415b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.891+0000 7fd08b470640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd084159bf0 0x7fd0840c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.891+0000 7fd08b470640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0841ef640 con 0x7fd0840cd660 2023-08-25T09:50:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.891+0000 7fd08b470640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0841ee300 con 0x7fd08406f0e0 2023-08-25T09:50:17.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.891+0000 7fd08b470640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0841eee70 con 0x7fd084159bf0 2023-08-25T09:50:17.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 0x7fd08415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 0x7fd08415b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40144/0 (socket says 172.21.15.70:40144) 2023-08-25T09:50:17.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 -- 172.21.15.70:0/3932732510 learned_addr learned my addr 172.21.15.70:0/3932732510 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd08920c640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd08406f0e0 0x7fd08414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 -- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd084159bf0 msgr2=0x7fd0840c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd089a0d640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd084159bf0 0x7fd0840c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd084159bf0 0x7fd0840c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 -- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd08406f0e0 msgr2=0x7fd08414e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd08406f0e0 0x7fd08414e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd088a0b640 1 -- 172.21.15.70:0/3932732510 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08000b770 con 0x7fd0840cd660 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.892+0000 7fd08920c640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd08406f0e0 0x7fd08414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.893+0000 7fd088a0b640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 0x7fd08415b080 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fd07400d800 tx=0x7fd07400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.893+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07400b6c0 con 0x7fd0840cd660 2023-08-25T09:50:17.896 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.893+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07400b820 con 0x7fd0840cd660 2023-08-25T09:50:17.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.893+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08415b5c0 con 0x7fd0840cd660 2023-08-25T09:50:17.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.893+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07400bbb0 con 0x7fd0840cd660 2023-08-25T09:50:17.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.893+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0841ef130 con 0x7fd0840cd660 2023-08-25T09:50:17.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.894+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd084066fb0 con 0x7fd0840cd660 2023-08-25T09:50:17.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.895+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd07400bd10 con 0x7fd0840cd660 2023-08-25T09:50:17.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.897+0000 7fd079ffb640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd06806a050 0x7fd06806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:17.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.897+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fd074088bc0 con 0x7fd0840cd660 2023-08-25T09:50:17.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.897+0000 7fd08920c640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd06806a050 0x7fd06806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:17.899 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.898+0000 7fd08920c640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd06806a050 0x7fd06806c510 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd080038000 tx=0x7fd080032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:17.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:17.900+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd074054210 con 0x7fd0840cd660 2023-08-25T09:50:18.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.088+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0840cd320 con 0x7fd0840cd660 2023-08-25T09:50:18.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.089+0000 7fd079ffb640 1 -- 172.21.15.70:0/3932732510 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v170) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd074057830 con 0x7fd0840cd660 2023-08-25T09:50:18.090 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:50:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd06806a050 msgr2=0x7fd06806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd06806a050 0x7fd06806c510 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd080038000 tx=0x7fd080032000 comp rx=0 tx=0).stop 2023-08-25T09:50:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 msgr2=0x7fd08415b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 0x7fd08415b080 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fd07400d800 tx=0x7fd07400dcc0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 shutdown_connections 2023-08-25T09:50:18.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd06806a050 0x7fd06806c510 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd084159bf0 0x7fd0840c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0840cd660 0x7fd08415b080 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 --2- 172.21.15.70:0/3932732510 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd08406f0e0 0x7fd08414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.091+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 >> 172.21.15.70:0/3932732510 conn(0x7fd0840cb0d0 msgr2=0x7fd0840ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.092+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 shutdown_connections 2023-08-25T09:50:18.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.092+0000 7fd08b470640 1 -- 172.21.15.70:0/3932732510 wait complete. 2023-08-25T09:50:18.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:18.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.307+0000 7f58a7e83640 1 Processor -- start 2023-08-25T09:50:18.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.307+0000 7f58a7e83640 1 -- start start 2023-08-25T09:50:18.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a7e83640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f58a00709a0 0x7f58a00b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a7e83640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f58a006f0a0 0x7f58a00b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a7e83640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a006ae10 con 0x7f58a0159100 2023-08-25T09:50:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a7e83640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a006ba50 con 0x7f58a00709a0 2023-08-25T09:50:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a7e83640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a006b0b0 con 0x7f58a006f0a0 2023-08-25T09:50:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a5c1f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f58a0159100 0x7f58a006be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:59898/0 (socket says 172.21.15.70:59898) 2023-08-25T09:50:18.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a5c1f640 1 -- 172.21.15.70:0/2907242081 learned_addr learned my addr 172.21.15.70:0/2907242081 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a541e640 1 --2- 172.21.15.70:0/2907242081 >> v2:172.21.15.70:3300/0 conn(0x7f58a00709a0 0x7f58a00b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.309+0000 7f58a6420640 1 --2- 172.21.15.70:0/2907242081 >> v2:172.21.15.155:3300/0 conn(0x7f58a006f0a0 0x7f58a00b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.310+0000 7f58a541e640 1 -- 172.21.15.70:0/2907242081 >> v2:172.21.15.155:3300/0 conn(0x7f58a006f0a0 msgr2=0x7f58a00b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.310+0000 7f58a541e640 1 --2- 172.21.15.70:0/2907242081 >> v2:172.21.15.155:3300/0 conn(0x7f58a006f0a0 0x7f58a00b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.310+0000 7f58a541e640 1 -- 172.21.15.70:0/2907242081 >> v1:172.21.15.70:6789/0 conn(0x7f58a0159100 legacy=0x7f58a006be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.310+0000 7f58a541e640 1 -- 172.21.15.70:0/2907242081 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a006b390 con 0x7f58a00709a0 2023-08-25T09:50:18.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.310+0000 7f58a4c1d640 1 -- 172.21.15.70:0/2907242081 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914282758 0 0) 0x7f58a006ae10 con 0x7f58a0159100 2023-08-25T09:50:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.311+0000 7f58a541e640 1 --2- 172.21.15.70:0/2907242081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00b1960 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f58a0071010 tx=0x7f589002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3e6f1c3aea8519 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.311+0000 7f58a4c1d640 1 -- 172.21.15.70:0/2907242081 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589002e070 con 0x7f58a00709a0 2023-08-25T09:50:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.311+0000 7f58a4c1d640 1 -- 172.21.15.70:0/2907242081 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58900315a0 con 0x7f58a00709a0 2023-08-25T09:50:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.311+0000 7f58a4c1d640 1 -- 172.21.15.70:0/2907242081 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890031930 con 0x7f58a00709a0 2023-08-25T09:50:18.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.311+0000 7f58a7e83640 1 -- 172.21.15.70:0/2907242081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 msgr2=0x7f58a00b1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.311+0000 7f58a7e83640 1 --2- 172.21.15.70:0/2907242081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00b1960 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f58a0071010 tx=0x7f589002b840 comp rx=0 tx=0).stop 2023-08-25T09:50:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.312+0000 7f58a7e83640 1 -- 172.21.15.70:0/2907242081 shutdown_connections 2023-08-25T09:50:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.312+0000 7f58a7e83640 1 --2- 172.21.15.70:0/2907242081 >> v2:172.21.15.155:3300/0 conn(0x7f58a006f0a0 0x7f58a00b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.312+0000 7f58a7e83640 1 --2- 172.21.15.70:0/2907242081 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00b1960 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.312+0000 7f58a7e83640 1 -- 172.21.15.70:0/2907242081 >> 172.21.15.70:0/2907242081 conn(0x7f58a0069850 msgr2=0x7f58a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:18.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.312+0000 7f58a7e83640 1 -- 172.21.15.70:0/2907242081 shutdown_connections 2023-08-25T09:50:18.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.312+0000 7f58a7e83640 1 -- 172.21.15.70:0/2907242081 wait complete. 2023-08-25T09:50:18.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.313+0000 7f58a7e83640 1 Processor -- start 2023-08-25T09:50:18.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.314+0000 7f58a7e83640 1 -- start start 2023-08-25T09:50:18.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.314+0000 7f58a7e83640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a006f0a0 0x7f58a0149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.314+0000 7f58a7e83640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a7e83640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58a0159100 0x7f58a00acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a7e83640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00c35c0 con 0x7f58a00709a0 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a7e83640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a014a350 con 0x7f58a006f0a0 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a7e83640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0159ef0 con 0x7f58a0159100 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40162/0 (socket says 172.21.15.70:40162) 2023-08-25T09:50:18.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 -- 172.21.15.70:0/1721751988 learned_addr learned my addr 172.21.15.70:0/1721751988 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a6420640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58a0159100 0x7f58a00acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a5c1f640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a006f0a0 0x7f58a0149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 -- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58a0159100 msgr2=0x7f58a00acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58a0159100 0x7f58a00acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 -- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a006f0a0 msgr2=0x7f58a0149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a006f0a0 0x7f58a0149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.315+0000 7f58a541e640 1 -- 172.21.15.70:0/1721751988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5890009400 con 0x7f58a00709a0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.316+0000 7f58a541e640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00ac870 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f58900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.316+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890032900 con 0x7f58a00709a0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.316+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890032a80 con 0x7f58a00709a0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.316+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890032e30 con 0x7f58a00709a0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.316+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00c3810 con 0x7f58a00709a0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.316+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a0154680 con 0x7f58a00709a0 2023-08-25T09:50:18.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.317+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a0147650 con 0x7f58a00709a0 2023-08-25T09:50:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.318+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f589004d3f0 con 0x7f58a00709a0 2023-08-25T09:50:18.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.319+0000 7f58967fc640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f588406a050 0x7f588406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.319+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f58900aa1a0 con 0x7f58a00709a0 2023-08-25T09:50:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.319+0000 7f58a5c1f640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f588406a050 0x7f588406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.320+0000 7f58a5c1f640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f588406a050 0x7f588406c510 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f589c000f60 tx=0x7f589c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:18.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.323+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f58a0147650 con 0x7f58a00709a0 2023-08-25T09:50:18.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.533+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a0147650 con 0x7f58a00709a0 2023-08-25T09:50:18.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.534+0000 7f58967fc640 1 -- 172.21.15.70:0/1721751988 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7f58a0147650 con 0x7f58a00709a0 2023-08-25T09:50:18.536 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:18.537 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:50:18.537 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:50:18.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f588406a050 msgr2=0x7f588406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f588406a050 0x7f588406c510 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f589c000f60 tx=0x7f589c009040 comp rx=0 tx=0).stop 2023-08-25T09:50:18.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 msgr2=0x7f58a00ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00ac870 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f58900046c0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 shutdown_connections 2023-08-25T09:50:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f588406a050 0x7f588406c510 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f58a0159100 0x7f58a00acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f58a00709a0 0x7f58a00ac870 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 --2- 172.21.15.70:0/1721751988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f58a006f0a0 0x7f58a0149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 >> 172.21.15.70:0/1721751988 conn(0x7f58a0069850 msgr2=0x7f58a006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 shutdown_connections 2023-08-25T09:50:18.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.537+0000 7f58a7e83640 1 -- 172.21.15.70:0/1721751988 wait complete. 2023-08-25T09:50:18.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:18.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.748+0000 7f7af01e8640 1 Processor -- start 2023-08-25T09:50:18.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.749+0000 7f7af01e8640 1 -- start start 2023-08-25T09:50:18.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.750+0000 7f7af01e8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ae815e830 0x7f7ae806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.750+0000 7f7af01e8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7ae806b2b0 0x7f7ae815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.750+0000 7f7af01e8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae80b1b60 con 0x7f7ae815c6d0 2023-08-25T09:50:18.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.750+0000 7f7af01e8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae806bd80 con 0x7f7ae815e830 2023-08-25T09:50:18.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.750+0000 7f7af01e8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae806b7c0 con 0x7f7ae806b2b0 2023-08-25T09:50:18.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ae815e830 0x7f7ae806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7ae815e830 0x7f7ae806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40164/0 (socket says 172.21.15.70:40164) 2023-08-25T09:50:18.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 -- 172.21.15.70:0/3736119672 learned_addr learned my addr 172.21.15.70:0/3736119672 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:18.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 -- 172.21.15.70:0/3736119672 >> v2:172.21.15.155:3300/0 conn(0x7f7ae806b2b0 msgr2=0x7f7ae815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:18.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 --2- 172.21.15.70:0/3736119672 >> v2:172.21.15.155:3300/0 conn(0x7f7ae806b2b0 0x7f7ae815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 -- 172.21.15.70:0/3736119672 >> v1:172.21.15.155:6789/0 conn(0x7f7ae815c6d0 legacy=0x7f7ae80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:18.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.751+0000 7f7aedf84640 1 -- 172.21.15.70:0/3736119672 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae806baa0 con 0x7f7ae815e830 2023-08-25T09:50:18.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.752+0000 7f7aedf84640 1 --2- 172.21.15.70:0/3736119672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae806ad70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80cf6f0 tx=0x7f7ae402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b2f3b740742537 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.752+0000 7f7aecf82640 1 -- 172.21.15.70:0/3736119672 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4031870 con 0x7f7ae815e830 2023-08-25T09:50:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.752+0000 7f7aecf82640 1 -- 172.21.15.70:0/3736119672 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae40319d0 con 0x7f7ae815e830 2023-08-25T09:50:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.752+0000 7f7aecf82640 1 -- 172.21.15.70:0/3736119672 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4031d60 con 0x7f7ae815e830 2023-08-25T09:50:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 -- 172.21.15.70:0/3736119672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 msgr2=0x7f7ae806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3736119672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae806ad70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80cf6f0 tx=0x7f7ae402b910 comp rx=0 tx=0).stop 2023-08-25T09:50:18.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 -- 172.21.15.70:0/3736119672 shutdown_connections 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3736119672 >> v2:172.21.15.155:3300/0 conn(0x7f7ae806b2b0 0x7f7ae815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3736119672 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae806ad70 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 -- 172.21.15.70:0/3736119672 >> 172.21.15.70:0/3736119672 conn(0x7f7ae8069850 msgr2=0x7f7ae8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 -- 172.21.15.70:0/3736119672 shutdown_connections 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.753+0000 7f7af01e8640 1 -- 172.21.15.70:0/3736119672 wait complete. 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.755+0000 7f7af01e8640 1 Processor -- start 2023-08-25T09:50:18.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.755+0000 7f7af01e8640 1 -- start start 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7af01e8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae806b2b0 0x7f7ae81ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7af01e8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ae815c6d0 0x7f7ae80c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7af01e8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae81f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7af01e8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8158ad0 con 0x7f7ae815e830 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7af01e8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80ca270 con 0x7f7ae806b2b0 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7af01e8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae81f31a0 con 0x7f7ae815c6d0 2023-08-25T09:50:18.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae81f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae81f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:40174/0 (socket says 172.21.15.70:40174) 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 -- 172.21.15.70:0/3217661892 learned_addr learned my addr 172.21.15.70:0/3217661892 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aedf84640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae806b2b0 0x7f7ae81ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 -- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ae815c6d0 msgr2=0x7f7ae80c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ae815c6d0 0x7f7ae80c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 -- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae806b2b0 msgr2=0x7f7ae81ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae806b2b0 0x7f7ae81ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.756+0000 7f7aee785640 1 -- 172.21.15.70:0/3217661892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae40094d0 con 0x7f7ae815e830 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.757+0000 7f7aee785640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae81f2c60 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c780 tx=0x7f7ae000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.757+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0003d80 con 0x7f7ae815e830 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.757+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae000ce90 con 0x7f7ae815e830 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.757+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae00052b0 con 0x7f7ae815e830 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.757+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae80c9660 con 0x7f7ae815e830 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.757+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae80c9c20 con 0x7f7ae815e830 2023-08-25T09:50:18.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.758+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae804e420 con 0x7f7ae815e830 2023-08-25T09:50:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.759+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ae0005410 con 0x7f7ae815e830 2023-08-25T09:50:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.760+0000 7f7ade7fc640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7acc06a050 0x7f7acc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.760+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f7ae0087bd0 con 0x7f7ae815e830 2023-08-25T09:50:18.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.760+0000 7f7aedf84640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7acc06a050 0x7f7acc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:18.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.761+0000 7f7aedf84640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7acc06a050 0x7f7acc06c510 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4036000 tx=0x7f7ae403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:18.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.764+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ae0053190 con 0x7f7ae815e830 2023-08-25T09:50:18.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.951+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ae80ca3b0 con 0x7f7ae815e830 2023-08-25T09:50:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.954+0000 7f7ade7fc640 1 -- 172.21.15.70:0/3217661892 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18618 (secure 0 0 0) 0x7f7ae0017090 con 0x7f7ae815e830 2023-08-25T09:50:18.955 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:18.956 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":170,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:17.456255+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"20.13","mappings":[{"from":3,"to":7}]},{"pgid":"20.1d","mappings":[{"from":3,"to":1}]},{"pgid":"20.1e","mappings":[{"from":3,"to":7}]},{"pgid":"20.38","mappings":[{"from":3,"to":7}]},{"pgid":"20.3c","mappings":[{"from":3,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:18.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7acc06a050 msgr2=0x7f7acc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7acc06a050 0x7f7acc06c510 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4036000 tx=0x7f7ae403c040 comp rx=0 tx=0).stop 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 msgr2=0x7f7ae81f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae81f2c60 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000c780 tx=0x7f7ae000cc40 comp rx=0 tx=0).stop 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 shutdown_connections 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7acc06a050 0x7f7acc06c510 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7ae815e830 0x7f7ae81f2c60 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7ae815c6d0 0x7f7ae80c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 --2- 172.21.15.70:0/3217661892 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7ae806b2b0 0x7f7ae81ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 >> 172.21.15.70:0/3217661892 conn(0x7f7ae8069850 msgr2=0x7f7ae815b700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 shutdown_connections 2023-08-25T09:50:18.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:18.956+0000 7f7af01e8640 1 -- 172.21.15.70:0/3217661892 wait complete. 2023-08-25T09:50:18.975 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:50:18.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:19.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.183+0000 7fb415463640 1 Processor -- start 2023-08-25T09:50:19.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.183+0000 7fb415463640 1 -- start start 2023-08-25T09:50:19.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb415463640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4100b12f0 0x7fb41015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb415463640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41015e7c0 con 0x7fb41015f190 2023-08-25T09:50:19.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb415463640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb410071b50 con 0x7fb41015a7c0 2023-08-25T09:50:19.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb415463640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41015e490 con 0x7fb4100b12f0 2023-08-25T09:50:19.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb40f7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb4100b12f0 0x7fb41015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb414c62640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb41015f190 0x7fb4100cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34448/0 (socket says 172.21.15.70:34448) 2023-08-25T09:50:19.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb414c62640 1 -- 172.21.15.70:0/1906037824 learned_addr learned my addr 172.21.15.70:0/1906037824 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:19.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb40f7fe640 1 -- 172.21.15.70:0/1906037824 >> v1:172.21.15.155:6789/0 conn(0x7fb41015a7c0 legacy=0x7fb4100b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:19.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb40f7fe640 1 -- 172.21.15.70:0/1906037824 >> v1:172.21.15.70:6789/0 conn(0x7fb41015f190 legacy=0x7fb4100cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.185+0000 7fb40f7fe640 1 -- 172.21.15.70:0/1906037824 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100b3d60 con 0x7fb4100b12f0 2023-08-25T09:50:19.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.186+0000 7fb40f7fe640 1 --2- 172.21.15.70:0/1906037824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4100b12f0 0x7fb41015ec50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fb4040097a0 tx=0x7fb40402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9bae35a5265b95 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:19.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb40effd640 1 -- 172.21.15.70:0/1906037824 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404031830 con 0x7fb4100b12f0 2023-08-25T09:50:19.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb40effd640 1 -- 172.21.15.70:0/1906037824 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404031990 con 0x7fb4100b12f0 2023-08-25T09:50:19.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb40effd640 1 -- 172.21.15.70:0/1906037824 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404031d20 con 0x7fb4100b12f0 2023-08-25T09:50:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb415463640 1 -- 172.21.15.70:0/1906037824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4100b12f0 msgr2=0x7fb41015ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb415463640 1 --2- 172.21.15.70:0/1906037824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4100b12f0 0x7fb41015ec50 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fb4040097a0 tx=0x7fb40402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb415463640 1 -- 172.21.15.70:0/1906037824 shutdown_connections 2023-08-25T09:50:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb415463640 1 --2- 172.21.15.70:0/1906037824 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb4100b12f0 0x7fb41015ec50 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb415463640 1 -- 172.21.15.70:0/1906037824 >> 172.21.15.70:0/1906037824 conn(0x7fb4100affd0 msgr2=0x7fb4100b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:19.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.187+0000 7fb415463640 1 -- 172.21.15.70:0/1906037824 shutdown_connections 2023-08-25T09:50:19.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.188+0000 7fb415463640 1 -- 172.21.15.70:0/1906037824 wait complete. 2023-08-25T09:50:19.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.189+0000 7fb415463640 1 Processor -- start 2023-08-25T09:50:19.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.189+0000 7fb415463640 1 -- start start 2023-08-25T09:50:19.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb415463640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4100b12f0 0x7fb4101f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb415463640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 0x7fb4100cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb415463640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb41015f190 0x7fb41015ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb415463640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410070a80 con 0x7fb41015a7c0 2023-08-25T09:50:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb415463640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101f3620 con 0x7fb41015f190 2023-08-25T09:50:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb415463640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41006eb60 con 0x7fb4100b12f0 2023-08-25T09:50:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.190+0000 7fb40f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 0x7fb4100cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 0x7fb4100cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39654/0 (socket says 172.21.15.70:39654) 2023-08-25T09:50:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 -- 172.21.15.70:0/3453646322 learned_addr learned my addr 172.21.15.70:0/3453646322 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:19.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 -- 172.21.15.70:0/3453646322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4100b12f0 msgr2=0x7fb4101f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4100b12f0 0x7fb4101f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 -- 172.21.15.70:0/3453646322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb41015f190 msgr2=0x7fb41015ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb41015f190 0x7fb41015ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 -- 172.21.15.70:0/3453646322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009400 con 0x7fb41015a7c0 2023-08-25T09:50:19.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.191+0000 7fb40f7fe640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 0x7fb4100cd480 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb4040098c0 tx=0x7fb4040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.192+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40403aa60 con 0x7fb41015a7c0 2023-08-25T09:50:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.192+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40403abc0 con 0x7fb41015a7c0 2023-08-25T09:50:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.192+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040423f0 con 0x7fb41015a7c0 2023-08-25T09:50:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.192+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4101f2c50 con 0x7fb41015a7c0 2023-08-25T09:50:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.192+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41006edb0 con 0x7fb41015a7c0 2023-08-25T09:50:19.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.193+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410066fb0 con 0x7fb41015a7c0 2023-08-25T09:50:19.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.194+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb404030380 con 0x7fb41015a7c0 2023-08-25T09:50:19.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.195+0000 7fb40cff9640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb3fc06a050 0x7fb3fc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.195+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fb4040aa270 con 0x7fb41015a7c0 2023-08-25T09:50:19.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.195+0000 7fb40ffff640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb3fc06a050 0x7fb3fc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.196+0000 7fb40ffff640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb3fc06a050 0x7fb3fc06c510 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c5060 tx=0x7fb4000079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:19.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.199+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb404075830 con 0x7fb41015a7c0 2023-08-25T09:50:19.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.409+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4101f2de0 con 0x7fb41015a7c0 2023-08-25T09:50:19.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.410+0000 7fb40cff9640 1 -- 172.21.15.70:0/3453646322 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fb404078e50 con 0x7fb41015a7c0 2023-08-25T09:50:19.412 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:19.413 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:50:19.413 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:50:19.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb3fc06a050 msgr2=0x7fb3fc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb3fc06a050 0x7fb3fc06c510 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb4100c5060 tx=0x7fb4000079b0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 msgr2=0x7fb4100cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 0x7fb4100cd480 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb4040098c0 tx=0x7fb4040046c0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 shutdown_connections 2023-08-25T09:50:19.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb3fc06a050 0x7fb3fc06c510 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb41015f190 0x7fb41015ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb41015a7c0 0x7fb4100cd480 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 --2- 172.21.15.70:0/3453646322 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb4100b12f0 0x7fb4101f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 >> 172.21.15.70:0/3453646322 conn(0x7fb4100affd0 msgr2=0x7fb4100c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:19.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 shutdown_connections 2023-08-25T09:50:19.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.413+0000 7fb415463640 1 -- 172.21.15.70:0/3453646322 wait complete. 2023-08-25T09:50:19.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:19.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.640+0000 7fe5a3463640 1 Processor -- start 2023-08-25T09:50:19.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.640+0000 7fe5a3463640 1 -- start start 2023-08-25T09:50:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a3463640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a3463640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe59c06e0c0 0x7fe59c166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a3463640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c15cfe0 con 0x7fe59c0cf580 2023-08-25T09:50:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a3463640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c0cfdb0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a3463640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c069850 con 0x7fe59c15cc00 2023-08-25T09:50:19.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a1a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe59c06e0c0 0x7fe59c166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a1a00640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe59c06e0c0 0x7fe59c166760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39660/0 (socket says 172.21.15.70:39660) 2023-08-25T09:50:19.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a1a00640 1 -- 172.21.15.70:0/3671178575 learned_addr learned my addr 172.21.15.70:0/3671178575 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:19.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.642+0000 7fe5a11ff640 1 --2- 172.21.15.70:0/3671178575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.643+0000 7fe5a1a00640 1 -- 172.21.15.70:0/3671178575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 msgr2=0x7fe59c06d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:19.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.643+0000 7fe5a1a00640 1 --2- 172.21.15.70:0/3671178575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c06d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.643+0000 7fe5a1a00640 1 -- 172.21.15.70:0/3671178575 >> v1:172.21.15.155:6789/0 conn(0x7fe59c0cf580 legacy=0x7fe59c06d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.643+0000 7fe5a1a00640 1 -- 172.21.15.70:0/3671178575 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c069b30 con 0x7fe59c06e0c0 2023-08-25T09:50:19.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.644+0000 7fe5a1a00640 1 --2- 172.21.15.70:0/3671178575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c166760 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe594009870 tx=0x7fe59402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefb09068a32b6a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:19.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.644+0000 7fe58ffff640 1 -- 172.21.15.70:0/3671178575 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594031640 con 0x7fe59c06e0c0 2023-08-25T09:50:19.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.644+0000 7fe58ffff640 1 -- 172.21.15.70:0/3671178575 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5940317a0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.644+0000 7fe58ffff640 1 -- 172.21.15.70:0/3671178575 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe594031b30 con 0x7fe59c06e0c0 2023-08-25T09:50:19.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 -- 172.21.15.70:0/3671178575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 msgr2=0x7fe59c166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 --2- 172.21.15.70:0/3671178575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c166760 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fe594009870 tx=0x7fe59402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 -- 172.21.15.70:0/3671178575 shutdown_connections 2023-08-25T09:50:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 --2- 172.21.15.70:0/3671178575 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c166760 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 --2- 172.21.15.70:0/3671178575 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c06d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 -- 172.21.15.70:0/3671178575 >> 172.21.15.70:0/3671178575 conn(0x7fe59c0d0860 msgr2=0x7fe59c0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:19.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 -- 172.21.15.70:0/3671178575 shutdown_connections 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.645+0000 7fe5a3463640 1 -- 172.21.15.70:0/3671178575 wait complete. 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.647+0000 7fe5a3463640 1 Processor -- start 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.647+0000 7fe5a3463640 1 -- start start 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a3463640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a3463640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe59c0cf580 0x7fe59c0b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a3463640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c09db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a3463640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c09cdd0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a3463640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c09d0b0 con 0x7fe59c0cf580 2023-08-25T09:50:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a3463640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c09d390 con 0x7fe59c15cc00 2023-08-25T09:50:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a11ff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a11ff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c1593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39672/0 (socket says 172.21.15.70:39672) 2023-08-25T09:50:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a11ff640 1 -- 172.21.15.70:0/983516438 learned_addr learned my addr 172.21.15.70:0/983516438 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:19.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a09fe640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe59c0cf580 0x7fe59c0b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.648+0000 7fe5a1a00640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c09db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a11ff640 1 -- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 msgr2=0x7fe59c09db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a11ff640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c09db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a11ff640 1 -- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe59c0cf580 msgr2=0x7fe59c0b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a11ff640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe59c0cf580 0x7fe59c0b3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a11ff640 1 -- 172.21.15.70:0/983516438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940094d0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a11ff640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c1593d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fe59000d7d0 tx=0x7fe59000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:19.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900096c0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590009820 con 0x7fe59c06e0c0 2023-08-25T09:50:19.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590009bd0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.649+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c09d670 con 0x7fe59c06e0c0 2023-08-25T09:50:19.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.650+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c09c9f0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.651+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c0cf9c0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.652+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe590009d30 con 0x7fe59c06e0c0 2023-08-25T09:50:19.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.652+0000 7fe58dffb640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe58006a050 0x7fe58006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:19.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.653+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7fe590088e90 con 0x7fe59c06e0c0 2023-08-25T09:50:19.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.653+0000 7fe5a09fe640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe58006a050 0x7fe58006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:19.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.653+0000 7fe5a09fe640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe58006a050 0x7fe58006c510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe59c09aa80 tx=0x7fe598009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:19.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.657+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe59c0cf9c0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.884+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe59c0cf9c0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.884+0000 7fe58dffb640 1 -- 172.21.15.70:0/983516438 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4880 (secure 0 0 0) 0x7fe59c0cf9c0 con 0x7fe59c06e0c0 2023-08-25T09:50:19.888 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:19.888 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:48:08.274966+0000","modified":"2023-08-25T09:48:11.852564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8718},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8718":{"gid":8718,"name":"c","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6837/1719004988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1719004988},{"type":"v1","addr":"172.21.15.70:6837","nonce":1719004988}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-08-25T09:50:19.889 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 74 2023-08-25T09:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe58006a050 msgr2=0x7fe58006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe58006a050 0x7fe58006c510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe59c09aa80 tx=0x7fe598009040 comp rx=0 tx=0).stop 2023-08-25T09:50:19.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 msgr2=0x7fe59c1593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c1593d0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fe59000d7d0 tx=0x7fe59000dc90 comp rx=0 tx=0).stop 2023-08-25T09:50:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 shutdown_connections 2023-08-25T09:50:19.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe58006a050 0x7fe58006c510 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe59c15cc00 0x7fe59c09db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe59c0cf580 0x7fe59c0b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 --2- 172.21.15.70:0/983516438 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe59c06e0c0 0x7fe59c1593d0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:19.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 >> 172.21.15.70:0/983516438 conn(0x7fe59c0d0860 msgr2=0x7fe59c0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:19.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 shutdown_connections 2023-08-25T09:50:19.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:19.887+0000 7fe5a3463640 1 -- 172.21.15.70:0/983516438 wait complete. 2023-08-25T09:50:19.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:20.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.118+0000 7ff3f282d640 1 Processor -- start 2023-08-25T09:50:20.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.118+0000 7ff3f282d640 1 -- start start 2023-08-25T09:50:20.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3f282d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3ec15aa10 0x7ff3ec0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3f282d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0b2ad0 con 0x7ff3ec0b2680 2023-08-25T09:50:20.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3f282d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec15d3f0 con 0x7ff3ec0ce5b0 2023-08-25T09:50:20.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3f282d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec159d10 con 0x7ff3ec15aa10 2023-08-25T09:50:20.124 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3eb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3ec15aa10 0x7ff3ec0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3eb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff3ec15aa10 0x7ff3ec0b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39682/0 (socket says 172.21.15.70:39682) 2023-08-25T09:50:20.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3eb7fe640 1 -- 172.21.15.70:0/1814362787 learned_addr learned my addr 172.21.15.70:0/1814362787 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:20.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.120+0000 7ff3f0dca640 1 --1- 172.21.15.70:0/1814362787 >> v1:172.21.15.70:6789/0 conn(0x7ff3ec0b2680 0x7ff3ec06f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34450/0 (socket says 172.21.15.70:34450) 2023-08-25T09:50:20.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.121+0000 7ff3eb7fe640 1 -- 172.21.15.70:0/1814362787 >> v1:172.21.15.155:6789/0 conn(0x7ff3ec0ce5b0 legacy=0x7ff3ec0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:20.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.121+0000 7ff3eb7fe640 1 -- 172.21.15.70:0/1814362787 >> v1:172.21.15.70:6789/0 conn(0x7ff3ec0b2680 legacy=0x7ff3ec06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.121+0000 7ff3eb7fe640 1 -- 172.21.15.70:0/1814362787 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0b2d20 con 0x7ff3ec15aa10 2023-08-25T09:50:20.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.122+0000 7ff3eb7fe640 1 --2- 172.21.15.70:0/1814362787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec0b2140 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40098a0 tx=0x7ff3d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1466ab7a70d312 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:20.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.122+0000 7ff3eaffd640 1 -- 172.21.15.70:0/1814362787 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4031640 con 0x7ff3ec15aa10 2023-08-25T09:50:20.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.122+0000 7ff3eaffd640 1 -- 172.21.15.70:0/1814362787 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d40317a0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.122+0000 7ff3eaffd640 1 -- 172.21.15.70:0/1814362787 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4031b30 con 0x7ff3ec15aa10 2023-08-25T09:50:20.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 -- 172.21.15.70:0/1814362787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 msgr2=0x7ff3ec0b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 --2- 172.21.15.70:0/1814362787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec0b2140 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40098a0 tx=0x7ff3d402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 -- 172.21.15.70:0/1814362787 shutdown_connections 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 --2- 172.21.15.70:0/1814362787 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec0b2140 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 -- 172.21.15.70:0/1814362787 >> 172.21.15.70:0/1814362787 conn(0x7ff3ec069850 msgr2=0x7ff3ec069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 -- 172.21.15.70:0/1814362787 shutdown_connections 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.123+0000 7ff3f282d640 1 -- 172.21.15.70:0/1814362787 wait complete. 2023-08-25T09:50:20.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.124+0000 7ff3f282d640 1 Processor -- start 2023-08-25T09:50:20.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.125+0000 7ff3f282d640 1 -- start start 2023-08-25T09:50:20.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f282d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0b2680 0x7ff3ec14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f282d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec0ce5b0 0x7ff3ec154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f282d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f282d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec14ec80 con 0x7ff3ec15aa10 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f282d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec155080 con 0x7ff3ec0ce5b0 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f282d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0af4e0 con 0x7ff3ec0b2680 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec1f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39684/0 (socket says 172.21.15.70:39684) 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 -- 172.21.15.70:0/3589173034 learned_addr learned my addr 172.21.15.70:0/3589173034 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3ebfff640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0b2680 0x7ff3ec14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3eb7fe640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec0ce5b0 0x7ff3ec154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 -- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0b2680 msgr2=0x7ff3ec14e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0b2680 0x7ff3ec14e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 -- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec0ce5b0 msgr2=0x7ff3ec154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec0ce5b0 0x7ff3ec154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.126+0000 7ff3f0dca640 1 -- 172.21.15.70:0/3589173034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d40094d0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.127+0000 7ff3f0dca640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec1f2a00 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c400 tx=0x7ff3e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.127+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40074e0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.127+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4007640 con 0x7ff3ec15aa10 2023-08-25T09:50:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.127+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40049e0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.127+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec0af7c0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.127+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec0afd80 con 0x7ff3ec15aa10 2023-08-25T09:50:20.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.128+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec04e420 con 0x7ff3ec15aa10 2023-08-25T09:50:20.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.129+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3e4004b40 con 0x7ff3ec15aa10 2023-08-25T09:50:20.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.130+0000 7ff3e8ff9640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3cc06a080 0x7ff3cc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.130+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7ff3e4088040 con 0x7ff3ec15aa10 2023-08-25T09:50:20.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.130+0000 7ff3ebfff640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3cc06a080 0x7ff3cc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.131+0000 7ff3ebfff640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3cc06a080 0x7ff3cc06c540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc005520 tx=0x7ff3dc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:20.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.134+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3e40536a0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.322+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3ec0cae50 con 0x7ff3ec15aa10 2023-08-25T09:50:20.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.325+0000 7ff3e8ff9640 1 -- 172.21.15.70:0/3589173034 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+18618 (secure 0 0 0) 0x7ff3e4056cc0 con 0x7ff3ec15aa10 2023-08-25T09:50:20.328 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:20.329 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":170,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:17.456255+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:48:06.570383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-08-25T09:48:07.289165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"20.13","mappings":[{"from":3,"to":7}]},{"pgid":"20.1d","mappings":[{"from":3,"to":1}]},{"pgid":"20.1e","mappings":[{"from":3,"to":7}]},{"pgid":"20.38","mappings":[{"from":3,"to":7}]},{"pgid":"20.3c","mappings":[{"from":3,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:20.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.327+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3cc06a080 msgr2=0x7ff3cc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.327+0000 7ff3f282d640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3cc06a080 0x7ff3cc06c540 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc005520 tx=0x7ff3dc009040 comp rx=0 tx=0).stop 2023-08-25T09:50:20.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.327+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 msgr2=0x7ff3ec1f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.327+0000 7ff3f282d640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec1f2a00 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400c400 tx=0x7ff3e400c8c0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 shutdown_connections 2023-08-25T09:50:20.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff3cc06a080 0x7ff3cc06c540 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff3ec15aa10 0x7ff3ec1f2a00 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff3ec0ce5b0 0x7ff3ec154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 --2- 172.21.15.70:0/3589173034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff3ec0b2680 0x7ff3ec14e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 >> 172.21.15.70:0/3589173034 conn(0x7ff3ec069850 msgr2=0x7ff3ec070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:20.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 shutdown_connections 2023-08-25T09:50:20.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.328+0000 7ff3f282d640 1 -- 172.21.15.70:0/3589173034 wait complete. 2023-08-25T09:50:20.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:50:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.557+0000 7f2265996640 1 Processor -- start 2023-08-25T09:50:20.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.558+0000 7f2265996640 1 -- start start 2023-08-25T09:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.559+0000 7f2265996640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f226015ac30 0x7f22600cb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.559+0000 7f2265996640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f22600cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.559+0000 7f2265996640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600cef00 con 0x7f22600cd1d0 2023-08-25T09:50:20.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.559+0000 7f2265996640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226006d1b0 con 0x7f226015ac30 2023-08-25T09:50:20.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.559+0000 7f2265996640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22600cc410 con 0x7f22600cba70 2023-08-25T09:50:20.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.560+0000 7f225e7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f22600cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.560+0000 7f225effd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f226015ac30 0x7f22600cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.560+0000 7f225e7fc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f22600cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:50970/0 (socket says 172.21.15.70:50970) 2023-08-25T09:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.560+0000 7f225e7fc640 1 -- 172.21.15.70:0/257295417 learned_addr learned my addr 172.21.15.70:0/257295417 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:20.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.561+0000 7f225e7fc640 1 -- 172.21.15.70:0/257295417 >> v2:172.21.15.155:3300/0 conn(0x7f226015ac30 msgr2=0x7f22600cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.561+0000 7f225e7fc640 1 --2- 172.21.15.70:0/257295417 >> v2:172.21.15.155:3300/0 conn(0x7f226015ac30 0x7f22600cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.561+0000 7f225e7fc640 1 -- 172.21.15.70:0/257295417 >> v1:172.21.15.155:6789/0 conn(0x7f22600cd1d0 legacy=0x7f22600cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.561+0000 7f225e7fc640 1 -- 172.21.15.70:0/257295417 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600cf150 con 0x7f22600cba70 2023-08-25T09:50:20.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.562+0000 7f225e7fc640 1 --2- 172.21.15.70:0/257295417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f22600cbed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5185f6c5208d5d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:20.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.563+0000 7f225dffb640 1 -- 172.21.15.70:0/257295417 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248031640 con 0x7f22600cba70 2023-08-25T09:50:20.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.563+0000 7f225dffb640 1 -- 172.21.15.70:0/257295417 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22480317a0 con 0x7f22600cba70 2023-08-25T09:50:20.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.563+0000 7f225dffb640 1 -- 172.21.15.70:0/257295417 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248031b30 con 0x7f22600cba70 2023-08-25T09:50:20.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 -- 172.21.15.70:0/257295417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 msgr2=0x7f22600cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 --2- 172.21.15.70:0/257295417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f22600cbed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 -- 172.21.15.70:0/257295417 shutdown_connections 2023-08-25T09:50:20.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 --2- 172.21.15.70:0/257295417 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f22600cbed0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 --2- 172.21.15.70:0/257295417 >> v2:172.21.15.155:3300/0 conn(0x7f226015ac30 0x7f22600cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 -- 172.21.15.70:0/257295417 >> 172.21.15.70:0/257295417 conn(0x7f22600d0920 msgr2=0x7f22600d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:20.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 -- 172.21.15.70:0/257295417 shutdown_connections 2023-08-25T09:50:20.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.564+0000 7f2265996640 1 -- 172.21.15.70:0/257295417 wait complete. 2023-08-25T09:50:20.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.566+0000 7f2265996640 1 Processor -- start 2023-08-25T09:50:20.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.567+0000 7f2265996640 1 -- start start 2023-08-25T09:50:20.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f2265996640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f226006da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f2265996640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22600cd1d0 0x7f22600b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f2265996640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 0x7f22600b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f2265996640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2260144540 con 0x7f226015ac30 2023-08-25T09:50:20.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f2265996640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2260144260 con 0x7f22600cd1d0 2023-08-25T09:50:20.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f2265996640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22601f18a0 con 0x7f22600cba70 2023-08-25T09:50:20.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f225f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 0x7f22600b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f225f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 0x7f22600b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39700/0 (socket says 172.21.15.70:39700) 2023-08-25T09:50:20.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f225f7fe640 1 -- 172.21.15.70:0/3663575135 learned_addr learned my addr 172.21.15.70:0/3663575135 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.568+0000 7f225effd640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f226006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225e7fc640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22600cd1d0 0x7f22600b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225f7fe640 1 -- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 msgr2=0x7f226006da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225f7fe640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f226006da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225f7fe640 1 -- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22600cd1d0 msgr2=0x7f22600b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225f7fe640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22600cd1d0 0x7f22600b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225f7fe640 1 -- 172.21.15.70:0/3663575135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22480094d0 con 0x7f226015ac30 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225e7fc640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22600cd1d0 0x7f22600b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f225f7fe640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 0x7f22600b07c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2258001eb0 tx=0x7f225800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.569+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580052d0 con 0x7f226015ac30 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.570+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2258005430 con 0x7f226015ac30 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.570+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22601f1b80 con 0x7f226015ac30 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.570+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258003810 con 0x7f226015ac30 2023-08-25T09:50:20.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.570+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226006df90 con 0x7f226015ac30 2023-08-25T09:50:20.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.571+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22601f1700 con 0x7f226015ac30 2023-08-25T09:50:20.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.572+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2258003970 con 0x7f226015ac30 2023-08-25T09:50:20.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.573+0000 7f2257fff640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f224006a050 0x7f224006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:20.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.573+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f2258087bd0 con 0x7f226015ac30 2023-08-25T09:50:20.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.573+0000 7f225effd640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f224006a050 0x7f224006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:20.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.574+0000 7f225effd640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f224006a050 0x7f224006c510 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2250011440 tx=0x7f225000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:20.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.578+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22601f1700 con 0x7f226015ac30 2023-08-25T09:50:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.804+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f22601f20a0 con 0x7f226015ac30 2023-08-25T09:50:20.808 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:50:20.806+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:50:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.817+0000 7f2257fff640 1 -- 172.21.15.70:0/3663575135 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f22601f20a0 con 0x7f226015ac30 2023-08-25T09:50:20.820 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:50:20.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f224006a050 msgr2=0x7f224006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f224006a050 0x7f224006c510 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2250011440 tx=0x7f225000b040 comp rx=0 tx=0).stop 2023-08-25T09:50:20.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 msgr2=0x7f22600b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 0x7f22600b07c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f2258001eb0 tx=0x7f225800c890 comp rx=0 tx=0).stop 2023-08-25T09:50:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 shutdown_connections 2023-08-25T09:50:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f224006a050 0x7f224006c510 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f226015ac30 0x7f22600b07c0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22600cd1d0 0x7f22600b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 --2- 172.21.15.70:0/3663575135 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22600cba70 0x7f226006da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 >> 172.21.15.70:0/3663575135 conn(0x7f22600d0920 msgr2=0x7f22600ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 shutdown_connections 2023-08-25T09:50:20.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:20.821+0000 7f2265996640 1 -- 172.21.15.70:0/3663575135 wait complete. 2023-08-25T09:50:20.829 INFO:tasks.ceph.mds.c.smithi070.stderr: -9304> 2023-08-25T09:48:14.759+0000 7fecc4bee640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:50:20.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:50:20.886 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:50:21.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.032+0000 7fe10aab4640 1 Processor -- start 2023-08-25T09:50:21.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.032+0000 7fe10aab4640 1 -- start start 2023-08-25T09:50:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.034+0000 7fe10aab4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1040cec00 0x7fe1040b2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:21.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.034+0000 7fe10aab4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10406c8b0 con 0x7fe1041607e0 2023-08-25T09:50:21.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.034+0000 7fe10aab4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10406b510 con 0x7fe10415cbb0 2023-08-25T09:50:21.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.034+0000 7fe10aab4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe10406b7f0 con 0x7fe1040cec00 2023-08-25T09:50:21.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.034+0000 7fe109051640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1040cec00 0x7fe1040b2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.035+0000 7fe109051640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe1040cec00 0x7fe1040b2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39716/0 (socket says 172.21.15.70:39716) 2023-08-25T09:50:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.035+0000 7fe109051640 1 -- 172.21.15.70:0/593840219 learned_addr learned my addr 172.21.15.70:0/593840219 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.035+0000 7fe103fff640 1 --1- 172.21.15.70:0/593840219 >> v1:172.21.15.70:6789/0 conn(0x7fe1041607e0 0x7fe1040ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34464/0 (socket says 172.21.15.70:34464) 2023-08-25T09:50:21.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.035+0000 7fe109051640 1 -- 172.21.15.70:0/593840219 >> v1:172.21.15.155:6789/0 conn(0x7fe10415cbb0 legacy=0x7fe10406ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:21.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.035+0000 7fe109051640 1 -- 172.21.15.70:0/593840219 >> v1:172.21.15.70:6789/0 conn(0x7fe1041607e0 legacy=0x7fe1040ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:21.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.035+0000 7fe109051640 1 -- 172.21.15.70:0/593840219 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10406bad0 con 0x7fe1040cec00 2023-08-25T09:50:21.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.036+0000 7fe109051640 1 --2- 172.21.15.70:0/593840219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1040cec00 0x7fe1040b2810 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009e50 tx=0x7fe0fc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f94884422abd9f1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:21.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.036+0000 7fe1037fe640 1 -- 172.21.15.70:0/593840219 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc02cbb0 con 0x7fe1040cec00 2023-08-25T09:50:21.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.036+0000 7fe1037fe640 1 -- 172.21.15.70:0/593840219 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc02cd10 con 0x7fe1040cec00 2023-08-25T09:50:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.036+0000 7fe1037fe640 1 -- 172.21.15.70:0/593840219 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc034610 con 0x7fe1040cec00 2023-08-25T09:50:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 -- 172.21.15.70:0/593840219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1040cec00 msgr2=0x7fe1040b2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 --2- 172.21.15.70:0/593840219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1040cec00 0x7fe1040b2810 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009e50 tx=0x7fe0fc02c010 comp rx=0 tx=0).stop 2023-08-25T09:50:21.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 -- 172.21.15.70:0/593840219 shutdown_connections 2023-08-25T09:50:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 --2- 172.21.15.70:0/593840219 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe1040cec00 0x7fe1040b2810 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 -- 172.21.15.70:0/593840219 >> 172.21.15.70:0/593840219 conn(0x7fe104069850 msgr2=0x7fe104069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 -- 172.21.15.70:0/593840219 shutdown_connections 2023-08-25T09:50:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.037+0000 7fe10aab4640 1 -- 172.21.15.70:0/593840219 wait complete. 2023-08-25T09:50:21.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.040+0000 7fe10aab4640 1 Processor -- start 2023-08-25T09:50:21.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.040+0000 7fe10aab4640 1 -- start start 2023-08-25T09:50:21.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe10aab4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1040cec00 0x7fe1040b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:21.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe10aab4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 0x7fe104144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:21.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe10aab4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1041607e0 0x7fe1041f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:21.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe10aab4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1041f62c0 con 0x7fe10415cbb0 2023-08-25T09:50:21.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe10aab4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1041450a0 con 0x7fe1040cec00 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe10aab4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe104070ea0 con 0x7fe1041607e0 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe103fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 0x7fe104144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe103fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 0x7fe104144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39718/0 (socket says 172.21.15.70:39718) 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.041+0000 7fe103fff640 1 -- 172.21.15.70:0/2645350858 learned_addr learned my addr 172.21.15.70:0/2645350858 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe108850640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1040cec00 0x7fe1040b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe103fff640 1 -- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1041607e0 msgr2=0x7fe1041f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:21.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe103fff640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1041607e0 0x7fe1041f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe103fff640 1 -- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1040cec00 msgr2=0x7fe1040b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe103fff640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1040cec00 0x7fe1040b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe103fff640 1 -- 172.21.15.70:0/2645350858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0fc009b30 con 0x7fe10415cbb0 2023-08-25T09:50:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe103fff640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 0x7fe104144b60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000c4d0 tx=0x7fe0f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0007480 con 0x7fe10415cbb0 2023-08-25T09:50:21.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f00075e0 con 0x7fe10415cbb0 2023-08-25T09:50:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.042+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1040708e0 con 0x7fe10415cbb0 2023-08-25T09:50:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.043+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00049e0 con 0x7fe10415cbb0 2023-08-25T09:50:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.043+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1041f5ee0 con 0x7fe10415cbb0 2023-08-25T09:50:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.044+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe10404e420 con 0x7fe10415cbb0 2023-08-25T09:50:21.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.045+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe0f0004b40 con 0x7fe10415cbb0 2023-08-25T09:50:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.046+0000 7fe1017fa640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0e806a050 0x7fe0e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.046+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe0f0087b10 con 0x7fe10415cbb0 2023-08-25T09:50:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.046+0000 7fe108850640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0e806a050 0x7fe0e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:21.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.047+0000 7fe108850640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0e806a050 0x7fe0e806c510 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe1041f5d10 tx=0x7fe0f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:21.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.050+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0f00531a0 con 0x7fe10415cbb0 2023-08-25T09:50:21.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.258+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe10406f140 con 0x7fe10415cbb0 2023-08-25T09:50:21.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.816+0000 7fe1017fa640 1 -- 172.21.15.70:0/2645350858 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0f00567c0 con 0x7fe10415cbb0 2023-08-25T09:50:21.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0e806a050 msgr2=0x7fe0e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:21.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0e806a050 0x7fe0e806c510 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe1041f5d10 tx=0x7fe0f8009040 comp rx=0 tx=0).stop 2023-08-25T09:50:21.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 msgr2=0x7fe104144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:21.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 0x7fe104144b60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000c4d0 tx=0x7fe0f000c990 comp rx=0 tx=0).stop 2023-08-25T09:50:21.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 shutdown_connections 2023-08-25T09:50:21.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0e806a050 0x7fe0e806c510 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe1041607e0 0x7fe1041f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe10415cbb0 0x7fe104144b60 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 --2- 172.21.15.70:0/2645350858 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe1040cec00 0x7fe1040b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:21.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.820+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 >> 172.21.15.70:0/2645350858 conn(0x7fe104069850 msgr2=0x7fe104069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:21.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.821+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 shutdown_connections 2023-08-25T09:50:21.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:21.821+0000 7fe10aab4640 1 -- 172.21.15.70:0/2645350858 wait complete. 2023-08-25T09:50:21.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:50:22.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.097+0000 7fd67917f640 1 Processor -- start 2023-08-25T09:50:22.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.097+0000 7fd67917f640 1 -- start start 2023-08-25T09:50:22.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd67917f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd6740cb520 0x7fd6740b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:22.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd67917f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6740cc1e0 con 0x7fd674161060 2023-08-25T09:50:22.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd67917f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6740cc4c0 con 0x7fd6740c18e0 2023-08-25T09:50:22.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd67917f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6740cc7a0 con 0x7fd6740cb520 2023-08-25T09:50:22.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd6737fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd6740cb520 0x7fd6740b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:22.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd6737fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd6740cb520 0x7fd6740b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39724/0 (socket says 172.21.15.70:39724) 2023-08-25T09:50:22.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd673fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd674161060 0x7fd6740cae10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34472/0 (socket says 172.21.15.70:34472) 2023-08-25T09:50:22.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.099+0000 7fd6737fe640 1 -- 172.21.15.70:0/4056866287 learned_addr learned my addr 172.21.15.70:0/4056866287 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:22.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.100+0000 7fd6737fe640 1 -- 172.21.15.70:0/4056866287 >> v1:172.21.15.155:6789/0 conn(0x7fd6740c18e0 legacy=0x7fd6740cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:22.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.100+0000 7fd6737fe640 1 -- 172.21.15.70:0/4056866287 >> v1:172.21.15.70:6789/0 conn(0x7fd674161060 legacy=0x7fd6740cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:22.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.100+0000 7fd6737fe640 1 -- 172.21.15.70:0/4056866287 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740cca80 con 0x7fd6740cb520 2023-08-25T09:50:22.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.100+0000 7fd672ffd640 1 -- 172.21.15.70:0/4056866287 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2963141725 0 0) 0x7fd6740cc1e0 con 0x7fd674161060 2023-08-25T09:50:22.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.101+0000 7fd6737fe640 1 --2- 172.21.15.70:0/4056866287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740cb520 0x7fd6740b1450 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd668009870 tx=0x7fd66802ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f89d37b8511b2fa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:22.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.101+0000 7fd672ffd640 1 -- 172.21.15.70:0/4056866287 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66802f840 con 0x7fd6740cb520 2023-08-25T09:50:22.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.101+0000 7fd672ffd640 1 -- 172.21.15.70:0/4056866287 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66802f9a0 con 0x7fd6740cb520 2023-08-25T09:50:22.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.101+0000 7fd672ffd640 1 -- 172.21.15.70:0/4056866287 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66802fd30 con 0x7fd6740cb520 2023-08-25T09:50:22.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.101+0000 7fd67917f640 1 -- 172.21.15.70:0/4056866287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740cb520 msgr2=0x7fd6740b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:22.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.101+0000 7fd67917f640 1 --2- 172.21.15.70:0/4056866287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740cb520 0x7fd6740b1450 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd668009870 tx=0x7fd66802ba20 comp rx=0 tx=0).stop 2023-08-25T09:50:22.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.102+0000 7fd67917f640 1 -- 172.21.15.70:0/4056866287 shutdown_connections 2023-08-25T09:50:22.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.102+0000 7fd67917f640 1 --2- 172.21.15.70:0/4056866287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740cb520 0x7fd6740b1450 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.102+0000 7fd67917f640 1 -- 172.21.15.70:0/4056866287 >> 172.21.15.70:0/4056866287 conn(0x7fd6740aff60 msgr2=0x7fd6740b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:22.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.102+0000 7fd67917f640 1 -- 172.21.15.70:0/4056866287 shutdown_connections 2023-08-25T09:50:22.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.102+0000 7fd67917f640 1 -- 172.21.15.70:0/4056866287 wait complete. 2023-08-25T09:50:22.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.103+0000 7fd67917f640 1 Processor -- start 2023-08-25T09:50:22.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.103+0000 7fd67917f640 1 -- start start 2023-08-25T09:50:22.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd67917f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 0x7fd6740c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:22.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd67917f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6740cb520 0x7fd67414d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:22.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd67917f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd674161060 0x7fd674150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:22.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd67917f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67414d7a0 con 0x7fd6740c18e0 2023-08-25T09:50:22.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd67917f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6741509c0 con 0x7fd674161060 2023-08-25T09:50:22.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd67917f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740c36c0 con 0x7fd6740cb520 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.104+0000 7fd673fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 0x7fd6740c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 0x7fd6740c30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39736/0 (socket says 172.21.15.70:39736) 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 -- 172.21.15.70:0/1291852906 learned_addr learned my addr 172.21.15.70:0/1291852906 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 -- 172.21.15.70:0/1291852906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6740cb520 msgr2=0x7fd67414d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6740cb520 0x7fd67414d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 -- 172.21.15.70:0/1291852906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd674161060 msgr2=0x7fd674150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd674161060 0x7fd674150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 -- 172.21.15.70:0/1291852906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6680094d0 con 0x7fd6740c18e0 2023-08-25T09:50:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd673fff640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 0x7fd6740c30e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c460 tx=0x7fd66400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6640074e0 con 0x7fd6740c18e0 2023-08-25T09:50:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664007660 con 0x7fd6740c18e0 2023-08-25T09:50:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664004a20 con 0x7fd6740c18e0 2023-08-25T09:50:22.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.105+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6740c39a0 con 0x7fd6740c18e0 2023-08-25T09:50:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.106+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6740c3f60 con 0x7fd6740c18e0 2023-08-25T09:50:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.107+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd674066fb0 con 0x7fd6740c18e0 2023-08-25T09:50:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.107+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd664004b80 con 0x7fd6740c18e0 2023-08-25T09:50:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.108+0000 7fd670ff9640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66006a050 0x7fd66006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.108+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd664087940 con 0x7fd6740c18e0 2023-08-25T09:50:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.108+0000 7fd6737fe640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66006a050 0x7fd66006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:22.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.109+0000 7fd6737fe640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66006a050 0x7fd66006c510 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd668036000 tx=0x7fd668039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:22.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.112+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd664052ea0 con 0x7fd6740c18e0 2023-08-25T09:50:22.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.302+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd674160810 con 0x7fd6740c18e0 2023-08-25T09:50:22.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.828+0000 7fd670ff9640 1 -- 172.21.15.70:0/1291852906 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd6640180d0 con 0x7fd6740c18e0 2023-08-25T09:50:22.830 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:50:22.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66006a050 msgr2=0x7fd66006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:22.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66006a050 0x7fd66006c510 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd668036000 tx=0x7fd668039040 comp rx=0 tx=0).stop 2023-08-25T09:50:22.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 msgr2=0x7fd6740c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:22.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 0x7fd6740c30e0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c460 tx=0x7fd66400c920 comp rx=0 tx=0).stop 2023-08-25T09:50:22.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 shutdown_connections 2023-08-25T09:50:22.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd66006a050 0x7fd66006c510 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd674161060 0x7fd674150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd6740cb520 0x7fd67414d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 --2- 172.21.15.70:0/1291852906 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd6740c18e0 0x7fd6740c30e0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:22.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 >> 172.21.15.70:0/1291852906 conn(0x7fd6740aff60 msgr2=0x7fd6740aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:22.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 shutdown_connections 2023-08-25T09:50:22.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:22.831+0000 7fd67917f640 1 -- 172.21.15.70:0/1291852906 wait complete. 2023-08-25T09:50:22.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:50:23.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.073+0000 7f9a0f74f640 1 Processor -- start 2023-08-25T09:50:23.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.073+0000 7f9a0f74f640 1 -- start start 2023-08-25T09:50:23.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0f74f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f9a080ca9f0 0x7f9a081658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0f74f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0815e6f0 con 0x7f9a08155120 2023-08-25T09:50:23.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0f74f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0815e9d0 con 0x7f9a080cda60 2023-08-25T09:50:23.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0f74f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a080caf30 con 0x7f9a080ca9f0 2023-08-25T09:50:23.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0d4eb640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f9a08155120 0x7f9a080cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34482/0 (socket says 172.21.15.70:34482) 2023-08-25T09:50:23.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/192724653 learned_addr learned my addr 172.21.15.70:0/192724653 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:23.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.075+0000 7f9a0dcec640 1 --2- 172.21.15.70:0/192724653 >> v2:172.21.15.155:3300/0 conn(0x7f9a080ca9f0 0x7f9a081658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221090037 0 0) 0x7f9a0815e6f0 con 0x7f9a08155120 2023-08-25T09:50:23.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f4003520 con 0x7f9a08155120 2023-08-25T09:50:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909220696 0 0) 0x7f99f4003520 con 0x7f9a08155120 2023-08-25T09:50:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 >> v2:172.21.15.155:3300/0 conn(0x7f9a080ca9f0 msgr2=0x7f9a081658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 --2- 172.21.15.70:0/192724653 >> v2:172.21.15.155:3300/0 conn(0x7f9a080ca9f0 0x7f9a081658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 >> v1:172.21.15.155:6789/0 conn(0x7f9a080cda60 legacy=0x7f9a080ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0815ecb0 con 0x7f9a08155120 2023-08-25T09:50:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f9a04001670 con 0x7f9a08155120 2023-08-25T09:50:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 >> v1:172.21.15.70:6789/0 conn(0x7f9a08155120 legacy=0x7f9a080cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f99f4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99f40047c0 0x7f99f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99f4005180 0x7f99f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f400c0f0 con 0x7f99f4003970 2023-08-25T09:50:23.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f400c260 con 0x7f99f40047c0 2023-08-25T09:50:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f400c420 con 0x7f99f4005180 2023-08-25T09:50:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.076+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f99f4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0ccea640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99f40047c0 0x7f99f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99f4005180 msgr2=0x7f99f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f99f4005180 0x7f99f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99f40047c0 msgr2=0x7f99f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f99f40047c0 0x7f99f4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/192724653 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4003520 con 0x7f99f4003970 2023-08-25T09:50:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.077+0000 7f9a0dcec640 1 -- 172.21.15.70:0/192724653 reap_dead start 2023-08-25T09:50:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.078+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f99f4003d90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c240 tx=0x7f99f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909684bad1706ee9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.078+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04005ad0 con 0x7f99f4003970 2023-08-25T09:50:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.078+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a04005c30 con 0x7f99f4003970 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.078+0000 7f99fffff640 1 -- 172.21.15.70:0/192724653 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0402cab0 con 0x7f99f4003970 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 -- 172.21.15.70:0/192724653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 msgr2=0x7f99f4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f99f4003d90 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400c240 tx=0x7f99f40057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 -- 172.21.15.70:0/192724653 shutdown_connections 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/192724653 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f99f4003d90 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 -- 172.21.15.70:0/192724653 >> 172.21.15.70:0/192724653 conn(0x7f9a08069c70 msgr2=0x7f9a0806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 -- 172.21.15.70:0/192724653 shutdown_connections 2023-08-25T09:50:23.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.079+0000 7f9a0f74f640 1 -- 172.21.15.70:0/192724653 wait complete. 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.081+0000 7f9a0f74f640 1 Processor -- start 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.081+0000 7f9a0f74f640 1 -- start start 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0f74f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f9a08150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0f74f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08151810 0x7f9a0814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0f74f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a08152210 0x7f9a08152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0f74f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a081505c0 con 0x7f99f4003970 2023-08-25T09:50:23.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0f74f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08152bb0 con 0x7f9a08151810 2023-08-25T09:50:23.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0f74f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08151450 con 0x7f9a08152210 2023-08-25T09:50:23.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f9a08150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f9a08150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39744/0 (socket says 172.21.15.70:39744) 2023-08-25T09:50:23.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/2177076689 learned_addr learned my addr 172.21.15.70:0/2177076689 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:23.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0ccea640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08151810 0x7f9a0814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0dcec640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a08152210 0x7f9a08152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a08152210 msgr2=0x7f9a08152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a08152210 0x7f9a08152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08151810 msgr2=0x7f9a0814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08151810 0x7f9a0814efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.082+0000 7f9a0d4eb640 1 -- 172.21.15.70:0/2177076689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a08151c90 con 0x7f99f4003970 2023-08-25T09:50:23.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.083+0000 7f9a0d4eb640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f9a08150d90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9a0402c410 tx=0x7f9a0402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:23.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.083+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0403c040 con 0x7f99f4003970 2023-08-25T09:50:23.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.083+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0402cc80 con 0x7f99f4003970 2023-08-25T09:50:23.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.083+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04007070 con 0x7f99f4003970 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.083+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0814f800 con 0x7f99f4003970 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.083+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0814fdc0 con 0x7f99f4003970 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.084+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a080b2870 con 0x7f99f4003970 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.085+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a04032070 con 0x7f99f4003970 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.086+0000 7f99fdffb640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f99ec06a050 0x7f99ec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.086+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5855+0+0 (secure 0 0 0) 0x7f9a040a9790 con 0x7f99f4003970 2023-08-25T09:50:23.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.087+0000 7f9a0ccea640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f99ec06a050 0x7f99ec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.087+0000 7f9a0ccea640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f99ec06a050 0x7f99ec06c510 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f99f8000c00 tx=0x7f99f8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:23.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.091+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a040751b0 con 0x7f99f4003970 2023-08-25T09:50:23.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.282+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a0804e420 con 0x7f99f4003970 2023-08-25T09:50:23.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.843+0000 7f99fdffb640 1 -- 172.21.15.70:0/2177076689 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a040460d0 con 0x7f99f4003970 2023-08-25T09:50:23.847 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:50:23.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f99ec06a050 msgr2=0x7f99ec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f99ec06a050 0x7f99ec06c510 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f99f8000c00 tx=0x7f99f8009040 comp rx=0 tx=0).stop 2023-08-25T09:50:23.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 msgr2=0x7f9a08150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:23.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f9a08150d90 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f9a0402c410 tx=0x7f9a0402e070 comp rx=0 tx=0).stop 2023-08-25T09:50:23.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 shutdown_connections 2023-08-25T09:50:23.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f99ec06a050 0x7f99ec06c510 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9a08152210 0x7f9a08152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9a08151810 0x7f9a0814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 --2- 172.21.15.70:0/2177076689 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f99f4003970 0x7f9a08150d90 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:23.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.850+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 >> 172.21.15.70:0/2177076689 conn(0x7f9a08069c70 msgr2=0x7f9a080cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:23.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.851+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 shutdown_connections 2023-08-25T09:50:23.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:23.851+0000 7f9a0f74f640 1 -- 172.21.15.70:0/2177076689 wait complete. 2023-08-25T09:50:23.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:50:24.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.075+0000 7f7223d0f640 1 Processor -- start 2023-08-25T09:50:24.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.075+0000 7f7223d0f640 1 -- start start 2023-08-25T09:50:24.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.077+0000 7f7223d0f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f721c15cdc0 0x7f721c0b3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.077+0000 7f7223d0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c06bd20 0x7f721c15b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.077+0000 7f7223d0f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c15d1a0 con 0x7f721c0b3540 2023-08-25T09:50:24.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.077+0000 7f7223d0f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c159d80 con 0x7f721c15cdc0 2023-08-25T09:50:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.077+0000 7f7223d0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c159f90 con 0x7f721c06bd20 2023-08-25T09:50:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.077+0000 7f7221aab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f721c15cdc0 0x7f721c0b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.078+0000 7f7221aab640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f721c15cdc0 0x7f721c0b3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39746/0 (socket says 172.21.15.70:39746) 2023-08-25T09:50:24.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.078+0000 7f7221aab640 1 -- 172.21.15.70:0/1368064613 learned_addr learned my addr 172.21.15.70:0/1368064613 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.078+0000 7f7221aab640 1 -- 172.21.15.70:0/1368064613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c06bd20 msgr2=0x7f721c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.078+0000 7f7221aab640 1 --2- 172.21.15.70:0/1368064613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c06bd20 0x7f721c15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.078+0000 7f7221aab640 1 -- 172.21.15.70:0/1368064613 >> v1:172.21.15.155:6789/0 conn(0x7f721c0b3540 legacy=0x7f721c06b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:24.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.078+0000 7f7221aab640 1 -- 172.21.15.70:0/1368064613 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c15a270 con 0x7f721c15cdc0 2023-08-25T09:50:24.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.079+0000 7f7221aab640 1 --2- 172.21.15.70:0/1368064613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c15cdc0 0x7f721c0b3000 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f7218009870 tx=0x7f721802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f272c436d02e29 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.079+0000 7f7220aa9640 1 -- 172.21.15.70:0/1368064613 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72180317c0 con 0x7f721c15cdc0 2023-08-25T09:50:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.079+0000 7f7220aa9640 1 -- 172.21.15.70:0/1368064613 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218031920 con 0x7f721c15cdc0 2023-08-25T09:50:24.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.079+0000 7f7220aa9640 1 -- 172.21.15.70:0/1368064613 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218031cb0 con 0x7f721c15cdc0 2023-08-25T09:50:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 -- 172.21.15.70:0/1368064613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c15cdc0 msgr2=0x7f721c0b3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 --2- 172.21.15.70:0/1368064613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c15cdc0 0x7f721c0b3000 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f7218009870 tx=0x7f721802b860 comp rx=0 tx=0).stop 2023-08-25T09:50:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 -- 172.21.15.70:0/1368064613 shutdown_connections 2023-08-25T09:50:24.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 --2- 172.21.15.70:0/1368064613 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c06bd20 0x7f721c15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 --2- 172.21.15.70:0/1368064613 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c15cdc0 0x7f721c0b3000 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 -- 172.21.15.70:0/1368064613 >> 172.21.15.70:0/1368064613 conn(0x7f721c069850 msgr2=0x7f721c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:24.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 -- 172.21.15.70:0/1368064613 shutdown_connections 2023-08-25T09:50:24.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.080+0000 7f7223d0f640 1 -- 172.21.15.70:0/1368064613 wait complete. 2023-08-25T09:50:24.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.082+0000 7f7223d0f640 1 Processor -- start 2023-08-25T09:50:24.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.083+0000 7f7223d0f640 1 -- start start 2023-08-25T09:50:24.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7223d0f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f721c06bd20 0x7f721c14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7223d0f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 0x7f721c14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7223d0f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c15cdc0 0x7f721c06fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7223d0f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cc460 con 0x7f721c0b3540 2023-08-25T09:50:24.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7223d0f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cb850 con 0x7f721c06bd20 2023-08-25T09:50:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7223d0f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cbb30 con 0x7f721c15cdc0 2023-08-25T09:50:24.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f72212aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 0x7f721c14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f72212aa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 0x7f721c14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39752/0 (socket says 172.21.15.70:39752) 2023-08-25T09:50:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f72212aa640 1 -- 172.21.15.70:0/334699315 learned_addr learned my addr 172.21.15.70:0/334699315 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:24.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.084+0000 7f7221aab640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f721c06bd20 0x7f721c14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72212aa640 1 -- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c15cdc0 msgr2=0x7f721c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72222ac640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c15cdc0 0x7f721c06fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72212aa640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c15cdc0 0x7f721c06fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72212aa640 1 -- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f721c06bd20 msgr2=0x7f721c14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72212aa640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f721c06bd20 0x7f721c14feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72212aa640 1 -- 172.21.15.70:0/334699315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180094d0 con 0x7f721c0b3540 2023-08-25T09:50:24.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72212aa640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 0x7f721c14dfc0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c410 tx=0x7f720c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:24.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0035f0 con 0x7f721c0b3540 2023-08-25T09:50:24.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c003750 con 0x7f721c0b3540 2023-08-25T09:50:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c003b40 con 0x7f721c0b3540 2023-08-25T09:50:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.085+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c0cbe10 con 0x7f721c0b3540 2023-08-25T09:50:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.086+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c1503f0 con 0x7f721c0b3540 2023-08-25T09:50:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.087+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c0cc610 con 0x7f721c0b3540 2023-08-25T09:50:24.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.088+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f720c003ca0 con 0x7f721c0b3540 2023-08-25T09:50:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.088+0000 7f72127fc640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f720006a050 0x7f720006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.089+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f720c087490 con 0x7f721c0b3540 2023-08-25T09:50:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.089+0000 7f7221aab640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f720006a050 0x7f720006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.089+0000 7f7221aab640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f720006a050 0x7f720006c510 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7218036000 tx=0x7f721803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:24.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.093+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f721c0cc610 con 0x7f721c0b3540 2023-08-25T09:50:24.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.281+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f721c04e420 con 0x7f721c0b3540 2023-08-25T09:50:24.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.287+0000 7f72127fc640 1 -- 172.21.15.70:0/334699315 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f720c052e90 con 0x7f721c0b3540 2023-08-25T09:50:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f720006a050 msgr2=0x7f720006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f720006a050 0x7f720006c510 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f7218036000 tx=0x7f721803d040 comp rx=0 tx=0).stop 2023-08-25T09:50:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 msgr2=0x7f721c14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 0x7f721c14dfc0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f720c00c410 tx=0x7f720c00c8d0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 shutdown_connections 2023-08-25T09:50:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f720006a050 0x7f720006c510 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721c15cdc0 0x7f721c06fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721c0b3540 0x7f721c14dfc0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 --2- 172.21.15.70:0/334699315 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f721c06bd20 0x7f721c14feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 >> 172.21.15.70:0/334699315 conn(0x7f721c069850 msgr2=0x7f721c06b480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:24.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 shutdown_connections 2023-08-25T09:50:24.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.290+0000 7f7223d0f640 1 -- 172.21.15.70:0/334699315 wait complete. 2023-08-25T09:50:24.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-08-25T09:50:24.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.522+0000 7f3ac7391640 1 Processor -- start 2023-08-25T09:50:24.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.522+0000 7f3ac7391640 1 -- start start 2023-08-25T09:50:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.523+0000 7f3ac7391640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3ac015a7c0 0x7f3ac00b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac7391640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 0x7f3ac00cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac7391640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac015e7c0 con 0x7f3ac00b12f0 2023-08-25T09:50:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac7391640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ac00cb6f0 con 0x7f3ac015a7c0 2023-08-25T09:50:24.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac7391640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0071b50 con 0x7f3ac00cad20 2023-08-25T09:50:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac5b8e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3ac00b12f0 0x7f3ac015ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34486/0 (socket says 172.21.15.70:34486) 2023-08-25T09:50:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac5b8e640 1 -- 172.21.15.70:0/1892097982 learned_addr learned my addr 172.21.15.70:0/1892097982 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac6b90640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 0x7f3ac00cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1892097982 >> v2:172.21.15.155:3300/0 conn(0x7f3ac015a7c0 0x7f3ac00b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.524+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259374856 0 0) 0x7f3ac015e7c0 con 0x7f3ac00b12f0 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aac003520 con 0x7f3ac00b12f0 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2064145344 0 0) 0x7f3aac003520 con 0x7f3ac00b12f0 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 msgr2=0x7f3ac00cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 0x7f3ac00cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 >> v2:172.21.15.155:3300/0 conn(0x7f3ac015a7c0 msgr2=0x7f3ac00b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:24.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 --2- 172.21.15.70:0/1892097982 >> v2:172.21.15.155:3300/0 conn(0x7f3ac015a7c0 0x7f3ac00b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac015e490 con 0x7f3ac00b12f0 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3ab0001670 con 0x7f3ac00b12f0 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 >> v1:172.21.15.70:6789/0 conn(0x7f3ac00b12f0 legacy=0x7f3ac015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3aac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac0047c0 0x7f3aac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 0x7f3aac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00cb6f0 con 0x7f3aac003970 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac0071b50 con 0x7f3aac0047c0 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac00b9b0 con 0x7f3ac00cad20 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.525+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3aac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.526+0000 7f3ac5b8e640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac0047c0 0x7f3aac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.526+0000 7f3ac638f640 1 -- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 msgr2=0x7f3aac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.526+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00cad20 0x7f3aac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.526+0000 7f3ac638f640 1 -- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac0047c0 msgr2=0x7f3aac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.526+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3aac0047c0 0x7f3aac004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.526+0000 7f3ac638f640 1 -- 172.21.15.70:0/1892097982 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac003520 con 0x7f3aac003970 2023-08-25T09:50:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.527+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3aac003d90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00ba30 tx=0x7f3aac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd6ff19d8b0bfe3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:24.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.527+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc032680 con 0x7f3aac003970 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.527+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc0327e0 con 0x7f3aac003970 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.527+0000 7f3ac538d640 1 -- 172.21.15.70:0/1892097982 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc032b70 con 0x7f3aac003970 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac7391640 1 -- 172.21.15.70:0/1892097982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 msgr2=0x7f3aac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1892097982 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3aac003d90 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00ba30 tx=0x7f3aac0057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac6b90640 1 -- 172.21.15.70:0/1892097982 reap_dead start 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac7391640 1 -- 172.21.15.70:0/1892097982 shutdown_connections 2023-08-25T09:50:24.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac7391640 1 -- 172.21.15.70:0/1892097982 >> 172.21.15.70:0/1892097982 conn(0x7f3ac00affd0 msgr2=0x7f3ac00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac7391640 1 -- 172.21.15.70:0/1892097982 shutdown_connections 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.528+0000 7f3ac7391640 1 -- 172.21.15.70:0/1892097982 wait complete. 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.530+0000 7f3ac7391640 1 Processor -- start 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.530+0000 7f3ac7391640 1 -- start start 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac7391640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3ac00c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac7391640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac01f2af0 0x7f3ac00ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac7391640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00acdc0 0x7f3ac01f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac7391640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01f1e20 con 0x7f3aac003970 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac7391640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac01f63e0 con 0x7f3ac01f2af0 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac7391640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac00ad240 con 0x7f3ac00acdc0 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac638f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3ac00c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac638f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3ac00c3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39772/0 (socket says 172.21.15.70:39772) 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.531+0000 7f3ac638f640 1 -- 172.21.15.70:0/1578846449 learned_addr learned my addr 172.21.15.70:0/1578846449 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:24.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac6b90640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00acdc0 0x7f3ac01f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac638f640 1 -- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00acdc0 msgr2=0x7f3ac01f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac5b8e640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac01f2af0 0x7f3ac00ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00acdc0 0x7f3ac01f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac638f640 1 -- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac01f2af0 msgr2=0x7f3ac00ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac01f2af0 0x7f3ac00ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac638f640 1 -- 172.21.15.70:0/1578846449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac01f2660 con 0x7f3aac003970 2023-08-25T09:50:24.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac6b90640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00acdc0 0x7f3ac01f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.532+0000 7f3ac638f640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3ac00c3e80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00bb50 tx=0x7f3abc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.533+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0308a0 con 0x7f3aac003970 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.533+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc030a20 con 0x7f3aac003970 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.533+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc030dd0 con 0x7f3aac003970 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.533+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac01f6100 con 0x7f3aac003970 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.533+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac01f22f0 con 0x7f3aac003970 2023-08-25T09:50:24.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.534+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac00b3d40 con 0x7f3aac003970 2023-08-25T09:50:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.535+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3abc03b420 con 0x7f3aac003970 2023-08-25T09:50:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.536+0000 7f3ab6ffd640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a0d0 0x7f3aa406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.536+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3abc0aa7f0 con 0x7f3aac003970 2023-08-25T09:50:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.536+0000 7f3ac5b8e640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a0d0 0x7f3aa406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.537+0000 7f3ac5b8e640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a0d0 0x7f3aa406c590 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00018e0 tx=0x7f3ab0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.540+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ac00b3d40 con 0x7f3aac003970 2023-08-25T09:50:24.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.798+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7f3ac00b3d40 con 0x7f3aac003970 2023-08-25T09:50:24.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.842+0000 7f3ab6ffd640 1 -- 172.21.15.70:0/1578846449 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v576) v1 ==== 132+0+0 (secure 0 0 0) 0x7f3ac00b3d40 con 0x7f3aac003970 2023-08-25T09:50:24.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a0d0 msgr2=0x7f3aa406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a0d0 0x7f3aa406c590 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00018e0 tx=0x7f3ab0009040 comp rx=0 tx=0).stop 2023-08-25T09:50:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 msgr2=0x7f3ac00c3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3ac00c3e80 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3abc00bb50 tx=0x7f3abc004730 comp rx=0 tx=0).stop 2023-08-25T09:50:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 shutdown_connections 2023-08-25T09:50:24.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3aa406a0d0 0x7f3aa406c590 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3ac00acdc0 0x7f3ac01f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3ac01f2af0 0x7f3ac00ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 --2- 172.21.15.70:0/1578846449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3aac003970 0x7f3ac00c3e80 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 >> 172.21.15.70:0/1578846449 conn(0x7f3ac00affd0 msgr2=0x7f3ac006ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.845+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 shutdown_connections 2023-08-25T09:50:24.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:24.846+0000 7f3ac7391640 1 -- 172.21.15.70:0/1578846449 wait complete. 2023-08-25T09:50:24.868 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-08-25T09:50:24.868 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-08-25T09:50:24.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:25.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.082+0000 7f1a26979640 1 Processor -- start 2023-08-25T09:50:25.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.082+0000 7f1a26979640 1 -- start start 2023-08-25T09:50:25.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.083+0000 7f1a26979640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a200b1ff0 0x7f1a200b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.083+0000 7f1a26979640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a200b1b60 con 0x7f1a2015ec10 2023-08-25T09:50:25.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.083+0000 7f1a26979640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a2006b7c0 con 0x7f1a2015e830 2023-08-25T09:50:25.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.083+0000 7f1a26979640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a2006bd80 con 0x7f1a200b1ff0 2023-08-25T09:50:25.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a24f16640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a200b1ff0 0x7f1a200b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a24f16640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f1a200b1ff0 0x7f1a200b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39780/0 (socket says 172.21.15.70:39780) 2023-08-25T09:50:25.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a24f16640 1 -- 172.21.15.70:0/1551426597 learned_addr learned my addr 172.21.15.70:0/1551426597 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:25.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a1f7fe640 1 --1- 172.21.15.70:0/1551426597 >> v1:172.21.15.70:6789/0 conn(0x7f1a2015ec10 0x7f1a2015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34494/0 (socket says 172.21.15.70:34494) 2023-08-25T09:50:25.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a24f16640 1 -- 172.21.15.70:0/1551426597 >> v1:172.21.15.155:6789/0 conn(0x7f1a2015e830 legacy=0x7f1a2006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:25.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a24f16640 1 -- 172.21.15.70:0/1551426597 >> v1:172.21.15.70:6789/0 conn(0x7f1a2015ec10 legacy=0x7f1a2015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a24f16640 1 -- 172.21.15.70:0/1551426597 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2006baa0 con 0x7f1a200b1ff0 2023-08-25T09:50:25.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.084+0000 7f1a1effd640 1 -- 172.21.15.70:0/1551426597 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786643238 0 0) 0x7f1a200b1b60 con 0x7f1a2015ec10 2023-08-25T09:50:25.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.085+0000 7f1a24f16640 1 --2- 172.21.15.70:0/1551426597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a200b1ff0 0x7f1a200b2440 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a1802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9ed98c8d3f82da server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a1effd640 1 -- 172.21.15.70:0/1551426597 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18030850 con 0x7f1a200b1ff0 2023-08-25T09:50:25.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a1effd640 1 -- 172.21.15.70:0/1551426597 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a180309b0 con 0x7f1a200b1ff0 2023-08-25T09:50:25.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a1effd640 1 -- 172.21.15.70:0/1551426597 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18030d40 con 0x7f1a200b1ff0 2023-08-25T09:50:25.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a26979640 1 -- 172.21.15.70:0/1551426597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a200b1ff0 msgr2=0x7f1a200b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a26979640 1 --2- 172.21.15.70:0/1551426597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a200b1ff0 0x7f1a200b2440 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a1802b990 comp rx=0 tx=0).stop 2023-08-25T09:50:25.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a26979640 1 -- 172.21.15.70:0/1551426597 shutdown_connections 2023-08-25T09:50:25.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a26979640 1 --2- 172.21.15.70:0/1551426597 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a200b1ff0 0x7f1a200b2440 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.086+0000 7f1a26979640 1 -- 172.21.15.70:0/1551426597 >> 172.21.15.70:0/1551426597 conn(0x7f1a20069850 msgr2=0x7f1a20069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:25.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.087+0000 7f1a26979640 1 -- 172.21.15.70:0/1551426597 shutdown_connections 2023-08-25T09:50:25.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.087+0000 7f1a26979640 1 -- 172.21.15.70:0/1551426597 wait complete. 2023-08-25T09:50:25.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.088+0000 7f1a26979640 1 Processor -- start 2023-08-25T09:50:25.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.088+0000 7f1a26979640 1 -- start start 2023-08-25T09:50:25.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.089+0000 7f1a26979640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b1ff0 0x7f1a200b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.089+0000 7f1a26979640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 0x7f1a20155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.089+0000 7f1a26979640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2015ec10 0x7f1a20156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.089+0000 7f1a26979640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20070ab0 con 0x7f1a2015e830 2023-08-25T09:50:25.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.089+0000 7f1a26979640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20156940 con 0x7f1a200b1ff0 2023-08-25T09:50:25.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.089+0000 7f1a26979640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20156c20 con 0x7f1a2015ec10 2023-08-25T09:50:25.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 0x7f1a20155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 0x7f1a20155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39782/0 (socket says 172.21.15.70:39782) 2023-08-25T09:50:25.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 -- 172.21.15.70:0/141558485 learned_addr learned my addr 172.21.15.70:0/141558485 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:25.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 -- 172.21.15.70:0/141558485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2015ec10 msgr2=0x7f1a20156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2015ec10 0x7f1a20156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 -- 172.21.15.70:0/141558485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b1ff0 msgr2=0x7f1a200b0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b1ff0 0x7f1a200b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 -- 172.21.15.70:0/141558485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180094d0 con 0x7f1a2015e830 2023-08-25T09:50:25.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1f7fe640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 0x7f1a20155dd0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c780 tx=0x7f1a0800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800ce90 con 0x7f1a2015e830 2023-08-25T09:50:25.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0800a640 con 0x7f1a2015e830 2023-08-25T09:50:25.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.090+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800a9d0 con 0x7f1a2015e830 2023-08-25T09:50:25.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.091+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a20156f00 con 0x7f1a2015e830 2023-08-25T09:50:25.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.091+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a201574c0 con 0x7f1a2015e830 2023-08-25T09:50:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.092+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a201576b0 con 0x7f1a2015e830 2023-08-25T09:50:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.093+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1a0800ab30 con 0x7f1a2015e830 2023-08-25T09:50:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.094+0000 7f1a1cff9640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a0006a050 0x7f1a0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.094+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1a08087890 con 0x7f1a2015e830 2023-08-25T09:50:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.094+0000 7f1a1ffff640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a0006a050 0x7f1a0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.094+0000 7f1a1ffff640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a0006a050 0x7f1a0006c510 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1a10003cc0 tx=0x7f1a100079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.098+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1a08053200 con 0x7f1a2015e830 2023-08-25T09:50:25.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.275+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a2004e420 con 0x7f1a2015e830 2023-08-25T09:50:25.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.277+0000 7f1a1cff9640 1 -- 172.21.15.70:0/141558485 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+14794 (secure 0 0 0) 0x7f1a08018020 con 0x7f1a2015e830 2023-08-25T09:50:25.280 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:25.280 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":173,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:23.825607+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6837/1719004988":"2023-08-26T09:50:20.807034+0000","172.21.15.70:6835/1719004988":"2023-08-26T09:50:20.807034+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:25.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a0006a050 msgr2=0x7f1a0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a0006a050 0x7f1a0006c510 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1a10003cc0 tx=0x7f1a100079b0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 msgr2=0x7f1a20155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 0x7f1a20155dd0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800c780 tx=0x7f1a0800cc40 comp rx=0 tx=0).stop 2023-08-25T09:50:25.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 shutdown_connections 2023-08-25T09:50:25.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1a0006a050 0x7f1a0006c510 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1a2015ec10 0x7f1a20156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1a2015e830 0x7f1a20155dd0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 --2- 172.21.15.70:0/141558485 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1a200b1ff0 0x7f1a200b0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 >> 172.21.15.70:0/141558485 conn(0x7f1a20069850 msgr2=0x7f1a2015c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:25.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 shutdown_connections 2023-08-25T09:50:25.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.280+0000 7f1a26979640 1 -- 172.21.15.70:0/141558485 wait complete. 2023-08-25T09:50:25.299 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:50:25.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:50:25.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.503+0000 7f0b1a309640 1 Processor -- start 2023-08-25T09:50:25.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.503+0000 7f0b1a309640 1 -- start start 2023-08-25T09:50:25.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.505+0000 7f0b1a309640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 0x7f0b140b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.505+0000 7f0b1a309640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1415d0b0 con 0x7f0b140d09f0 2023-08-25T09:50:25.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.505+0000 7f0b1a309640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b140b58d0 con 0x7f0b1415ccd0 2023-08-25T09:50:25.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.505+0000 7f0b1a309640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140d25d0 con 0x7f0b140cf9f0 2023-08-25T09:50:25.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.505+0000 7f0b18b06640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0b140d09f0 0x7f0b140cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34496/0 (socket says 172.21.15.70:34496) 2023-08-25T09:50:25.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.505+0000 7f0b18b06640 1 -- 172.21.15.70:0/2768003710 learned_addr learned my addr 172.21.15.70:0/2768003710 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:25.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b19b08640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 0x7f0b140b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832423863 0 0) 0x7f0b1415d0b0 con 0x7f0b140d09f0 2023-08-25T09:50:25.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00003520 con 0x7f0b140d09f0 2023-08-25T09:50:25.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1127640149 0 0) 0x7f0b00003520 con 0x7f0b140d09f0 2023-08-25T09:50:25.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 msgr2=0x7f0b140b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:25.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 0x7f0b140b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 >> v1:172.21.15.155:6789/0 conn(0x7f0b1415ccd0 legacy=0x7f0b140d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b140d0e80 con 0x7f0b140d09f0 2023-08-25T09:50:25.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0b04001690 con 0x7f0b140d09f0 2023-08-25T09:50:25.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 >> v1:172.21.15.70:6789/0 conn(0x7f0b140d09f0 legacy=0x7f0b140cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b000039a0 0x7f0b00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 0x7f0b000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b140d25d0 con 0x7f0b00004d00 2023-08-25T09:50:25.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0000ba60 con 0x7f0b000039a0 2023-08-25T09:50:25.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.506+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0000bc20 con 0x7f0b140cf9f0 2023-08-25T09:50:25.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.507+0000 7f0b19b08640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.507+0000 7f0b19b08640 1 -- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 msgr2=0x7f0b000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:25.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.507+0000 7f0b19b08640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b140cf9f0 0x7f0b000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.507+0000 7f0b19b08640 1 -- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b000039a0 msgr2=0x7f0b00003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:25.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.507+0000 7f0b19b08640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b000039a0 0x7f0b00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.507+0000 7f0b19b08640 1 -- 172.21.15.70:0/2768003710 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00003520 con 0x7f0b00004d00 2023-08-25T09:50:25.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.508+0000 7f0b19b08640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b00005180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009f90 tx=0x7f0b000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7699670f48167ae1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.508+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c02cc00 con 0x7f0b00004d00 2023-08-25T09:50:25.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.508+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00f040 con 0x7f0b00004d00 2023-08-25T09:50:25.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.508+0000 7f0b0bfff640 1 -- 172.21.15.70:0/2768003710 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c034440 con 0x7f0b00004d00 2023-08-25T09:50:25.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b1a309640 1 -- 172.21.15.70:0/2768003710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 msgr2=0x7f0b00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b1a309640 1 --2- 172.21.15.70:0/2768003710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b00005180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009f90 tx=0x7f0b000057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b19b08640 1 -- 172.21.15.70:0/2768003710 reap_dead start 2023-08-25T09:50:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b1a309640 1 -- 172.21.15.70:0/2768003710 shutdown_connections 2023-08-25T09:50:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b1a309640 1 -- 172.21.15.70:0/2768003710 >> 172.21.15.70:0/2768003710 conn(0x7f0b14069850 msgr2=0x7f0b14069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:25.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b1a309640 1 -- 172.21.15.70:0/2768003710 shutdown_connections 2023-08-25T09:50:25.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.509+0000 7f0b1a309640 1 -- 172.21.15.70:0/2768003710 wait complete. 2023-08-25T09:50:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.511+0000 7f0b1a309640 1 Processor -- start 2023-08-25T09:50:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.511+0000 7f0b1a309640 1 -- start start 2023-08-25T09:50:25.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b1a309640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b1415fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b1a309640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b14154080 0x7f0b141f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b1a309640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b14154f10 0x7f0b14155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b1a309640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b141558d0 con 0x7f0b00004d00 2023-08-25T09:50:25.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b1a309640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14153bf0 con 0x7f0b14154f10 2023-08-25T09:50:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b1a309640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b14152fe0 con 0x7f0b14154080 2023-08-25T09:50:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b19307640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b1415fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b19307640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b1415fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39800/0 (socket says 172.21.15.70:39800) 2023-08-25T09:50:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.512+0000 7f0b19307640 1 -- 172.21.15.70:0/90005695 learned_addr learned my addr 172.21.15.70:0/90005695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b19307640 1 -- 172.21.15.70:0/90005695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b14154080 msgr2=0x7f0b141f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b19307640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b14154080 0x7f0b141f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b19307640 1 -- 172.21.15.70:0/90005695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b14154f10 msgr2=0x7f0b14155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b19307640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b14154f10 0x7f0b14155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b19307640 1 -- 172.21.15.70:0/90005695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b141532c0 con 0x7f0b00004d00 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b19307640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b1415fba0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c480 tx=0x7f0b1000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b100052d0 con 0x7f0b00004d00 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b10005430 con 0x7f0b00004d00 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b141535a0 con 0x7f0b00004d00 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b10012870 con 0x7f0b00004d00 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.513+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b14154500 con 0x7f0b00004d00 2023-08-25T09:50:25.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.514+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b14061460 con 0x7f0b00004d00 2023-08-25T09:50:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.515+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0b100129d0 con 0x7f0b00004d00 2023-08-25T09:50:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.516+0000 7f0b09ffb640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0af806a050 0x7f0af806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.516+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0b10087e00 con 0x7f0b00004d00 2023-08-25T09:50:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.517+0000 7f0b18b06640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0af806a050 0x7f0af806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.517+0000 7f0b18b06640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0af806a050 0x7f0af806c510 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0b040017c0 tx=0x7f0b04001490 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.520+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0b10053880 con 0x7f0b00004d00 2023-08-25T09:50:25.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.712+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b140cb490 con 0x7f0b00004d00 2023-08-25T09:50:25.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.713+0000 7f0b09ffb640 1 -- 172.21.15.70:0/90005695 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v173) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b10056ea0 con 0x7f0b00004d00 2023-08-25T09:50:25.715 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:50:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0af806a050 msgr2=0x7f0af806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0af806a050 0x7f0af806c510 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0b040017c0 tx=0x7f0b04001490 comp rx=0 tx=0).stop 2023-08-25T09:50:25.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 msgr2=0x7f0b1415fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b1415fba0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c480 tx=0x7f0b1000c940 comp rx=0 tx=0).stop 2023-08-25T09:50:25.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 shutdown_connections 2023-08-25T09:50:25.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0af806a050 0x7f0af806c510 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0b14154f10 0x7f0b14155390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0b14154080 0x7f0b141f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 --2- 172.21.15.70:0/90005695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0b00004d00 0x7f0b1415fba0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 >> 172.21.15.70:0/90005695 conn(0x7f0b14069850 msgr2=0x7f0b140cad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 shutdown_connections 2023-08-25T09:50:25.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.716+0000 7f0b1a309640 1 -- 172.21.15.70:0/90005695 wait complete. 2023-08-25T09:50:25.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:50:25.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.949+0000 7fcda3057640 1 Processor -- start 2023-08-25T09:50:25.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.949+0000 7fcda3057640 1 -- start start 2023-08-25T09:50:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda3057640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcd9c15e240 0x7fcd9c070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda3057640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd9c0cd660 0x7fcd9c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda3057640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c06f8e0 con 0x7fcd9c159bf0 2023-08-25T09:50:25.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda3057640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c0ce030 con 0x7fcd9c0cd660 2023-08-25T09:50:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda3057640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c06fbc0 con 0x7fcd9c15e240 2023-08-25T09:50:25.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda15f4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd9c0cd660 0x7fcd9c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda15f4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcd9c0cd660 0x7fcd9c0cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39816/0 (socket says 172.21.15.70:39816) 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcda15f4640 1 -- 172.21.15.70:0/2727701918 learned_addr learned my addr 172.21.15.70:0/2727701918 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.951+0000 7fcd9bfff640 1 --2- 172.21.15.70:0/2727701918 >> v2:172.21.15.155:3300/0 conn(0x7fcd9c15e240 0x7fcd9c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.952+0000 7fcda15f4640 1 -- 172.21.15.70:0/2727701918 >> v2:172.21.15.155:3300/0 conn(0x7fcd9c15e240 msgr2=0x7fcd9c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.952+0000 7fcda15f4640 1 --2- 172.21.15.70:0/2727701918 >> v2:172.21.15.155:3300/0 conn(0x7fcd9c15e240 0x7fcd9c070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.952+0000 7fcda15f4640 1 -- 172.21.15.70:0/2727701918 >> v1:172.21.15.155:6789/0 conn(0x7fcd9c159bf0 legacy=0x7fcd9c06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:25.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.952+0000 7fcda15f4640 1 -- 172.21.15.70:0/2727701918 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c06fea0 con 0x7fcd9c0cd660 2023-08-25T09:50:25.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.953+0000 7fcda15f4640 1 --2- 172.21.15.70:0/2727701918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0cdaf0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06af90 tx=0x7fcd9402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d22fd1853b69ca server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.953+0000 7fcd9b7fe640 1 -- 172.21.15.70:0/2727701918 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9402ca80 con 0x7fcd9c0cd660 2023-08-25T09:50:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.953+0000 7fcd9b7fe640 1 -- 172.21.15.70:0/2727701918 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9402cbe0 con 0x7fcd9c0cd660 2023-08-25T09:50:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.953+0000 7fcd9b7fe640 1 -- 172.21.15.70:0/2727701918 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940343f0 con 0x7fcd9c0cd660 2023-08-25T09:50:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 -- 172.21.15.70:0/2727701918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 msgr2=0x7fcd9c0cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 --2- 172.21.15.70:0/2727701918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0cdaf0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06af90 tx=0x7fcd9402bee0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 -- 172.21.15.70:0/2727701918 shutdown_connections 2023-08-25T09:50:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 --2- 172.21.15.70:0/2727701918 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0cdaf0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 --2- 172.21.15.70:0/2727701918 >> v2:172.21.15.155:3300/0 conn(0x7fcd9c15e240 0x7fcd9c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 -- 172.21.15.70:0/2727701918 >> 172.21.15.70:0/2727701918 conn(0x7fcd9c0cb0d0 msgr2=0x7fcd9c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 -- 172.21.15.70:0/2727701918 shutdown_connections 2023-08-25T09:50:25.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.954+0000 7fcda3057640 1 -- 172.21.15.70:0/2727701918 wait complete. 2023-08-25T09:50:25.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.956+0000 7fcda3057640 1 Processor -- start 2023-08-25T09:50:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.956+0000 7fcda3057640 1 -- start start 2023-08-25T09:50:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda3057640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda3057640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9c159bf0 0x7fcd9c14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda3057640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9c15e240 0x7fcd9c15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda3057640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c14b270 con 0x7fcd9c0cd660 2023-08-25T09:50:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda3057640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15c7a0 con 0x7fcd9c15e240 2023-08-25T09:50:25.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda3057640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15ca80 con 0x7fcd9c159bf0 2023-08-25T09:50:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39824/0 (socket says 172.21.15.70:39824) 2023-08-25T09:50:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 -- 172.21.15.70:0/720661460 learned_addr learned my addr 172.21.15.70:0/720661460 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 -- 172.21.15.70:0/720661460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9c159bf0 msgr2=0x7fcd9c14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9c159bf0 0x7fcd9c14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 -- 172.21.15.70:0/720661460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9c15e240 msgr2=0x7fcd9c15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9c15e240 0x7fcd9c15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.957+0000 7fcda0df3640 1 -- 172.21.15.70:0/720661460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94009b30 con 0x7fcd9c0cd660 2023-08-25T09:50:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.958+0000 7fcda0df3640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0a94f0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c430 tx=0x7fcd9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.958+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90011790 con 0x7fcd9c0cd660 2023-08-25T09:50:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.958+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd900118f0 con 0x7fcd9c0cd660 2023-08-25T09:50:25.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.958+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90011ce0 con 0x7fcd9c0cd660 2023-08-25T09:50:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.958+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c15cd60 con 0x7fcd9c0cd660 2023-08-25T09:50:25.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.958+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c15d320 con 0x7fcd9c0cd660 2023-08-25T09:50:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.960+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c0aad90 con 0x7fcd9c0cd660 2023-08-25T09:50:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.960+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcd9001c460 con 0x7fcd9c0cd660 2023-08-25T09:50:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.961+0000 7fcd997fa640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd8006a050 0x7fcd8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.961+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcd90088290 con 0x7fcd9c0cd660 2023-08-25T09:50:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.962+0000 7fcd9bfff640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd8006a050 0x7fcd8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:25.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.962+0000 7fcd9bfff640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd8006a050 0x7fcd8006c510 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcd88006fd0 tx=0x7fcd88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:25.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:25.966+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd90053bf0 con 0x7fcd9c0cd660 2023-08-25T09:50:26.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.153+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd9c066fb0 con 0x7fcd9c0cd660 2023-08-25T09:50:26.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.154+0000 7fcd997fa640 1 -- 172.21.15.70:0/720661460 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v173) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd90057210 con 0x7fcd9c0cd660 2023-08-25T09:50:26.157 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:50:26.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.156+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd8006a050 msgr2=0x7fcd8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.156+0000 7fcda3057640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd8006a050 0x7fcd8006c510 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcd88006fd0 tx=0x7fcd88008040 comp rx=0 tx=0).stop 2023-08-25T09:50:26.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.156+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 msgr2=0x7fcd9c0a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.156+0000 7fcda3057640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0a94f0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fcd9000c430 tx=0x7fcd9000c8f0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 shutdown_connections 2023-08-25T09:50:26.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcd8006a050 0x7fcd8006c510 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcd9c15e240 0x7fcd9c15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcd9c159bf0 0x7fcd9c14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 --2- 172.21.15.70:0/720661460 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcd9c0cd660 0x7fcd9c0a94f0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 >> 172.21.15.70:0/720661460 conn(0x7fcd9c0cb0d0 msgr2=0x7fcd9c0c4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:26.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 shutdown_connections 2023-08-25T09:50:26.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.157+0000 7fcda3057640 1 -- 172.21.15.70:0/720661460 wait complete. 2023-08-25T09:50:26.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-08-25T09:50:26.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.384+0000 7fa77475d640 1 Processor -- start 2023-08-25T09:50:26.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.384+0000 7fa77475d640 1 -- start start 2023-08-25T09:50:26.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa77475d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa76c0ca780 0x7fa76c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:26.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa77475d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa76c0b3cf0 0x7fa76c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:26.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa77475d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0b12b0 con 0x7fa76c15acd0 2023-08-25T09:50:26.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa77475d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c06c0e0 con 0x7fa76c0ca780 2023-08-25T09:50:26.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa77475d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c06ae20 con 0x7fa76c0b3cf0 2023-08-25T09:50:26.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa7724f9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa76c15acd0 0x7fa76c15b0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34504/0 (socket says 172.21.15.70:34504) 2023-08-25T09:50:26.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa7724f9640 1 -- 172.21.15.70:0/2463773495 learned_addr learned my addr 172.21.15.70:0/2463773495 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:26.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.386+0000 7fa771cf8640 1 --2- 172.21.15.70:0/2463773495 >> v2:172.21.15.70:3300/0 conn(0x7fa76c0ca780 0x7fa76c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:26.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa771cf8640 1 -- 172.21.15.70:0/2463773495 >> v2:172.21.15.155:3300/0 conn(0x7fa76c0b3cf0 msgr2=0x7fa76c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:26.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa772cfa640 1 --2- 172.21.15.70:0/2463773495 >> v2:172.21.15.155:3300/0 conn(0x7fa76c0b3cf0 0x7fa76c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:26.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa771cf8640 1 --2- 172.21.15.70:0/2463773495 >> v2:172.21.15.155:3300/0 conn(0x7fa76c0b3cf0 0x7fa76c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa771cf8640 1 -- 172.21.15.70:0/2463773495 >> v1:172.21.15.70:6789/0 conn(0x7fa76c15acd0 legacy=0x7fa76c15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa7714f7640 1 -- 172.21.15.70:0/2463773495 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2440504294 0 0) 0x7fa76c0b12b0 con 0x7fa76c15acd0 2023-08-25T09:50:26.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa771cf8640 1 -- 172.21.15.70:0/2463773495 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c15b7c0 con 0x7fa76c0ca780 2023-08-25T09:50:26.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.387+0000 7fa772cfa640 1 --2- 172.21.15.70:0/2463773495 >> v2:172.21.15.155:3300/0 conn(0x7fa76c0b3cf0 0x7fa76c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:26.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.388+0000 7fa771cf8640 1 --2- 172.21.15.70:0/2463773495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c0cac30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0c8d30 tx=0x7fa75c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf116fe4274f16bb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:26.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.388+0000 7fa7714f7640 1 -- 172.21.15.70:0/2463773495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c02e070 con 0x7fa76c0ca780 2023-08-25T09:50:26.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.388+0000 7fa7714f7640 1 -- 172.21.15.70:0/2463773495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c0315a0 con 0x7fa76c0ca780 2023-08-25T09:50:26.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.388+0000 7fa7714f7640 1 -- 172.21.15.70:0/2463773495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c031930 con 0x7fa76c0ca780 2023-08-25T09:50:26.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 -- 172.21.15.70:0/2463773495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 msgr2=0x7fa76c0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 --2- 172.21.15.70:0/2463773495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c0cac30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0c8d30 tx=0x7fa75c02b840 comp rx=0 tx=0).stop 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 -- 172.21.15.70:0/2463773495 shutdown_connections 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 --2- 172.21.15.70:0/2463773495 >> v2:172.21.15.155:3300/0 conn(0x7fa76c0b3cf0 0x7fa76c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 --2- 172.21.15.70:0/2463773495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c0cac30 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 -- 172.21.15.70:0/2463773495 >> 172.21.15.70:0/2463773495 conn(0x7fa76c0ce760 msgr2=0x7fa76c0ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 -- 172.21.15.70:0/2463773495 shutdown_connections 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.389+0000 7fa77475d640 1 -- 172.21.15.70:0/2463773495 wait complete. 2023-08-25T09:50:26.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.391+0000 7fa77475d640 1 Processor -- start 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.391+0000 7fa77475d640 1 -- start start 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa77475d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa76c0b3cf0 0x7fa76c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa77475d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa77475d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa76c15acd0 0x7fa76c155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa77475d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0cda70 con 0x7fa76c0ca780 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa77475d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c154470 con 0x7fa76c0b3cf0 2023-08-25T09:50:26.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa77475d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c154750 con 0x7fa76c15acd0 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa771cf8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa771cf8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39838/0 (socket says 172.21.15.70:39838) 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.392+0000 7fa771cf8640 1 -- 172.21.15.70:0/415322368 learned_addr learned my addr 172.21.15.70:0/415322368 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa7724f9640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa76c0b3cf0 0x7fa76c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa771cf8640 1 -- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa76c15acd0 msgr2=0x7fa76c155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa771cf8640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa76c15acd0 0x7fa76c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa771cf8640 1 -- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa76c0b3cf0 msgr2=0x7fa76c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa771cf8640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa76c0b3cf0 0x7fa76c0cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa771cf8640 1 -- 172.21.15.70:0/415322368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c009400 con 0x7fa76c0ca780 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa7724f9640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa76c0b3cf0 0x7fa76c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa771cf8640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c153a60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa75c000c00 tx=0x7fa75c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c032900 con 0x7fa76c0ca780 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c032a60 con 0x7fa76c0ca780 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c032dd0 con 0x7fa76c0ca780 2023-08-25T09:50:26.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.393+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c154a30 con 0x7fa76c0ca780 2023-08-25T09:50:26.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.394+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c154ff0 con 0x7fa76c0ca780 2023-08-25T09:50:26.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.395+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c06b330 con 0x7fa76c0ca780 2023-08-25T09:50:26.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.395+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa75c04d3f0 con 0x7fa76c0ca780 2023-08-25T09:50:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.396+0000 7fa762ffd640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.396+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa75c0a9f60 con 0x7fa76c0ca780 2023-08-25T09:50:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.397+0000 7fa7724f9640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.397+0000 7fa7724f9640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa7680016b0 tx=0x7fa768009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.401+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa76c06b330 con 0x7fa76c0ca780 2023-08-25T09:50:26.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.654+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fa76c06b330 con 0x7fa76c0ca780 2023-08-25T09:50:26.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.855+0000 7fa762ffd640 1 -- 172.21.15.70:0/415322368 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v578) v1 ==== 134+0+0 (secure 0 0 0) 0x7fa76c06b330 con 0x7fa76c0ca780 2023-08-25T09:50:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 msgr2=0x7fa75006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa7680016b0 tx=0x7fa768009040 comp rx=0 tx=0).stop 2023-08-25T09:50:26.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 msgr2=0x7fa76c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c153a60 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fa75c000c00 tx=0x7fa75c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 shutdown_connections 2023-08-25T09:50:26.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa75006a050 0x7fa75006c510 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa76c15acd0 0x7fa76c155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa76c0ca780 0x7fa76c153a60 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 --2- 172.21.15.70:0/415322368 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa76c0b3cf0 0x7fa76c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:26.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 >> 172.21.15.70:0/415322368 conn(0x7fa76c0ce760 msgr2=0x7fa76c0c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:26.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 shutdown_connections 2023-08-25T09:50:26.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:26.858+0000 7fa77475d640 1 -- 172.21.15.70:0/415322368 wait complete. 2023-08-25T09:50:26.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:50:27.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.090+0000 7f5575973640 1 Processor -- start 2023-08-25T09:50:27.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.090+0000 7f5575973640 1 -- start start 2023-08-25T09:50:27.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.092+0000 7f5575973640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f557015a920 0x7f55700c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:27.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.092+0000 7f5575973640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b1ce0 0x7f55700ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:27.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.092+0000 7f5575973640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55700ad8f0 con 0x7f55700b0320 2023-08-25T09:50:27.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.092+0000 7f5575973640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55700adbb0 con 0x7f557015a920 2023-08-25T09:50:27.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.092+0000 7f5575973640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55700c90a0 con 0x7f55700b1ce0 2023-08-25T09:50:27.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f556ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f557015a920 0x7f55700c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:27.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f5575172640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b1ce0 0x7f55700ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:27.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f556ffff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f557015a920 0x7f55700c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:36324/0 (socket says 172.21.15.70:36324) 2023-08-25T09:50:27.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f556ffff640 1 -- 172.21.15.70:0/4279816563 learned_addr learned my addr 172.21.15.70:0/4279816563 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:27.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f556ffff640 1 -- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b1ce0 msgr2=0x7f55700ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:27.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f556ffff640 1 --2- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b1ce0 0x7f55700ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.093+0000 7f556ffff640 1 -- 172.21.15.70:0/4279816563 >> v1:172.21.15.155:6789/0 conn(0x7f55700b0320 legacy=0x7f55700b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:27.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.094+0000 7f556ffff640 1 -- 172.21.15.70:0/4279816563 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55700c9380 con 0x7f557015a920 2023-08-25T09:50:27.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.095+0000 7f556ffff640 1 --2- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557015a920 0x7f55700c9e60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f555c009870 tx=0x7f555c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d562627b06defb8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:27.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.095+0000 7f556f7fe640 1 -- 172.21.15.70:0/4279816563 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c030840 con 0x7f557015a920 2023-08-25T09:50:27.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.095+0000 7f556f7fe640 1 -- 172.21.15.70:0/4279816563 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c0309a0 con 0x7f557015a920 2023-08-25T09:50:27.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.095+0000 7f556f7fe640 1 -- 172.21.15.70:0/4279816563 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c030d30 con 0x7f557015a920 2023-08-25T09:50:27.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 -- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557015a920 msgr2=0x7f55700c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:27.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 --2- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557015a920 0x7f55700c9e60 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f555c009870 tx=0x7f555c02b980 comp rx=0 tx=0).stop 2023-08-25T09:50:27.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 -- 172.21.15.70:0/4279816563 shutdown_connections 2023-08-25T09:50:27.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 --2- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b1ce0 0x7f55700ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 --2- 172.21.15.70:0/4279816563 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f557015a920 0x7f55700c9e60 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 -- 172.21.15.70:0/4279816563 >> 172.21.15.70:0/4279816563 conn(0x7f55700ce830 msgr2=0x7f55700cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 -- 172.21.15.70:0/4279816563 shutdown_connections 2023-08-25T09:50:27.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.096+0000 7f5575973640 1 -- 172.21.15.70:0/4279816563 wait complete. 2023-08-25T09:50:27.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.099+0000 7f5575973640 1 Processor -- start 2023-08-25T09:50:27.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.099+0000 7f5575973640 1 -- start start 2023-08-25T09:50:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575973640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b0320 0x7f55701f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:27.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575973640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55700b1ce0 0x7f55700c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575973640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 0x7f55701f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575973640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701edb70 con 0x7f557015a920 2023-08-25T09:50:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575973640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55701ede50 con 0x7f55700b1ce0 2023-08-25T09:50:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575973640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55701ee130 con 0x7f55700b0320 2023-08-25T09:50:27.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575172640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 0x7f55701f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575172640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 0x7f55701f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39848/0 (socket says 172.21.15.70:39848) 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.100+0000 7f5575172640 1 -- 172.21.15.70:0/1370560979 learned_addr learned my addr 172.21.15.70:0/1370560979 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f556ffff640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55700b1ce0 0x7f55700c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5574971640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b0320 0x7f55701f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5575172640 1 -- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b0320 msgr2=0x7f55701f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5575172640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b0320 0x7f55701f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5575172640 1 -- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55700b1ce0 msgr2=0x7f55700c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5575172640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55700b1ce0 0x7f55700c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5575172640 1 -- 172.21.15.70:0/1370560979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0094d0 con 0x7f557015a920 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f5575172640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 0x7f55701f3520 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f556800efc0 tx=0x7f556800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.101+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568014480 con 0x7f557015a920 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.102+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556800f040 con 0x7f557015a920 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.102+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568014820 con 0x7f557015a920 2023-08-25T09:50:27.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.102+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55701ee410 con 0x7f557015a920 2023-08-25T09:50:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.102+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55701ee9d0 con 0x7f557015a920 2023-08-25T09:50:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.103+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55701eebc0 con 0x7f557015a920 2023-08-25T09:50:27.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.104+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5568014980 con 0x7f557015a920 2023-08-25T09:50:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.105+0000 7f556d7fa640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f555406a050 0x7f555406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.105+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5568088e90 con 0x7f557015a920 2023-08-25T09:50:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.105+0000 7f5574971640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f555406a050 0x7f555406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:27.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.106+0000 7f5574971640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f555406a050 0x7f555406c510 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5564007fd0 tx=0x7f5564009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.109+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5568054880 con 0x7f557015a920 2023-08-25T09:50:27.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.297+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5570061460 con 0x7f557015a920 2023-08-25T09:50:27.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.306+0000 7f556d7fa640 1 -- 172.21.15.70:0/1370560979 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5568057ea0 con 0x7f557015a920 2023-08-25T09:50:27.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f555406a050 msgr2=0x7f555406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:27.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f555406a050 0x7f555406c510 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5564007fd0 tx=0x7f5564009040 comp rx=0 tx=0).stop 2023-08-25T09:50:27.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 msgr2=0x7f55701f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:27.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 0x7f55701f3520 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f556800efc0 tx=0x7f556800c490 comp rx=0 tx=0).stop 2023-08-25T09:50:27.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 shutdown_connections 2023-08-25T09:50:27.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f555406a050 0x7f555406c510 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f557015a920 0x7f55701f3520 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f55700b1ce0 0x7f55700c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 --2- 172.21.15.70:0/1370560979 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f55700b0320 0x7f55701f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:27.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 >> 172.21.15.70:0/1370560979 conn(0x7f55700ce830 msgr2=0x7f5570070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:27.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 shutdown_connections 2023-08-25T09:50:27.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:27.308+0000 7f5575973640 1 -- 172.21.15.70:0/1370560979 wait complete. 2023-08-25T09:50:27.329 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:50:27.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:50:27.445 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:50:27.568 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:50:27.569 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:50:27.569 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:27.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:50:27.631 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:50:27.632 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:27.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:50:27.698 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:50:27.698 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:27.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:50:27.762 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:50:27.763 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:50:27.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:50:27.830 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:50:27.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:28.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.053+0000 7fa375347640 1 Processor -- start 2023-08-25T09:50:28.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.053+0000 7fa375347640 1 -- start start 2023-08-25T09:50:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa375347640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa37015e830 0x7fa37006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa375347640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 0x7fa3700b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa375347640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3700b1b60 con 0x7fa37006b2b0 2023-08-25T09:50:28.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa375347640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa37006bd80 con 0x7fa37015e830 2023-08-25T09:50:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa375347640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37006b7c0 con 0x7fa37015ec10 2023-08-25T09:50:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa36e7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa37006b2b0 0x7fa37015c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34512/0 (socket says 172.21.15.70:34512) 2023-08-25T09:50:28.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.055+0000 7fa36e7fc640 1 -- 172.21.15.70:0/1127821199 learned_addr learned my addr 172.21.15.70:0/1127821199 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36effd640 1 --2- 172.21.15.70:0/1127821199 >> v2:172.21.15.155:3300/0 conn(0x7fa37015e830 0x7fa37006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36f7fe640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 0x7fa3700b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409476937 0 0) 0x7fa3700b1b60 con 0x7fa37006b2b0 2023-08-25T09:50:28.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa358003520 con 0x7fa37006b2b0 2023-08-25T09:50:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069321688 0 0) 0x7fa358003520 con 0x7fa37006b2b0 2023-08-25T09:50:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 msgr2=0x7fa3700b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:28.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 0x7fa3700b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 >> v2:172.21.15.155:3300/0 conn(0x7fa37015e830 msgr2=0x7fa37006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 --2- 172.21.15.70:0/1127821199 >> v2:172.21.15.155:3300/0 conn(0x7fa37015e830 0x7fa37006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37006baa0 con 0x7fa37006b2b0 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa364001670 con 0x7fa37006b2b0 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 >> v1:172.21.15.70:6789/0 conn(0x7fa37006b2b0 legacy=0x7fa37015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa358003970 0x7fa358003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3580047c0 0x7fa358004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 0x7fa358005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37006bd80 con 0x7fa358003970 2023-08-25T09:50:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37006b7c0 con 0x7fa3580047c0 2023-08-25T09:50:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.056+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35800b980 con 0x7fa37015ec10 2023-08-25T09:50:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.057+0000 7fa36effd640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa358003970 0x7fa358003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.057+0000 7fa36effd640 1 -- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 msgr2=0x7fa358005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.057+0000 7fa36effd640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa37015ec10 0x7fa358005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.057+0000 7fa36effd640 1 -- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3580047c0 msgr2=0x7fa358004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.057+0000 7fa36effd640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3580047c0 0x7fa358004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.057+0000 7fa36effd640 1 -- 172.21.15.70:0/1127821199 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358003520 con 0x7fa358003970 2023-08-25T09:50:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.058+0000 7fa36effd640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa358003970 0x7fa358003d90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa360009ac0 tx=0x7fa3580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4237e339be5152 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.058+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36002c710 con 0x7fa358003970 2023-08-25T09:50:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.058+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36002c870 con 0x7fa358003970 2023-08-25T09:50:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.058+0000 7fa36dffb640 1 -- 172.21.15.70:0/1127821199 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36002cc00 con 0x7fa358003970 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa375347640 1 -- 172.21.15.70:0/1127821199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa358003970 msgr2=0x7fa358003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa375347640 1 --2- 172.21.15.70:0/1127821199 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa358003970 0x7fa358003d90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fa360009ac0 tx=0x7fa3580057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa36f7fe640 1 -- 172.21.15.70:0/1127821199 reap_dead start 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa375347640 1 -- 172.21.15.70:0/1127821199 shutdown_connections 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa375347640 1 -- 172.21.15.70:0/1127821199 >> 172.21.15.70:0/1127821199 conn(0x7fa370069850 msgr2=0x7fa370069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa375347640 1 -- 172.21.15.70:0/1127821199 shutdown_connections 2023-08-25T09:50:28.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.059+0000 7fa375347640 1 -- 172.21.15.70:0/1127821199 wait complete. 2023-08-25T09:50:28.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.061+0000 7fa375347640 1 Processor -- start 2023-08-25T09:50:28.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.061+0000 7fa375347640 1 -- start start 2023-08-25T09:50:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa375347640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358003970 0x7fa37014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa375347640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3700c3c60 0x7fa3700c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa375347640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 0x7fa3700c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa375347640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c7240 con 0x7fa3700c5ae0 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa375347640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c64a0 con 0x7fa358003970 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa375347640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700c52c0 con 0x7fa3700c3c60 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa36f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 0x7fa3700c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa36f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 0x7fa3700c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39860/0 (socket says 172.21.15.70:39860) 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.062+0000 7fa36f7fe640 1 -- 172.21.15.70:0/3636072860 learned_addr learned my addr 172.21.15.70:0/3636072860 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36effd640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358003970 0x7fa37014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36e7fc640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3700c3c60 0x7fa3700c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36f7fe640 1 -- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3700c3c60 msgr2=0x7fa3700c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36f7fe640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3700c3c60 0x7fa3700c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36f7fe640 1 -- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358003970 msgr2=0x7fa37014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36f7fe640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358003970 0x7fa37014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36f7fe640 1 -- 172.21.15.70:0/3636072860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37015ca20 con 0x7fa3700c5ae0 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36effd640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358003970 0x7fa37014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa36f7fe640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 0x7fa3700c5f60 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa37006a620 tx=0x7fa36800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368010040 con 0x7fa3700c5ae0 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36800bae0 con 0x7fa3700c5ae0 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36800be70 con 0x7fa3700c5ae0 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37014ff60 con 0x7fa3700c5ae0 2023-08-25T09:50:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.063+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3700c4990 con 0x7fa3700c5ae0 2023-08-25T09:50:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.064+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3700c5430 con 0x7fa3700c5ae0 2023-08-25T09:50:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.065+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa36801e410 con 0x7fa3700c5ae0 2023-08-25T09:50:28.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.066+0000 7fa3577fe640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa35c06a050 0x7fa35c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.066+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa3680886e0 con 0x7fa3700c5ae0 2023-08-25T09:50:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.067+0000 7fa36effd640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa35c06a050 0x7fa35c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.067+0000 7fa36effd640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa35c06a050 0x7fa35c06c510 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa360036000 tx=0x7fa360030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.071+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa368057a50 con 0x7fa3700c5ae0 2023-08-25T09:50:28.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.297+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa37004e420 con 0x7fa3700c5ae0 2023-08-25T09:50:28.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.298+0000 7fa3577fe640 1 -- 172.21.15.70:0/3636072860 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3699 (secure 0 0 0) 0x7fa3680575e0 con 0x7fa3700c5ae0 2023-08-25T09:50:28.302 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:28.302 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5498,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6833/4249276757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4249276757},{"type":"v1","addr":"172.21.15.155:6833","nonce":4249276757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":5501,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1974322213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6834","nonce":1974322213},{"type":"v1","addr":"172.21.15.155:6835","nonce":1974322213}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/250669163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":250669163},{"type":"v1","addr":"172.21.15.70:6836","nonce":250669163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":9069,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/901965747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":901965747},{"type":"v1","addr":"172.21.15.70:6837","nonce":901965747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76}],"filesystems":[]} 2023-08-25T09:50:28.302 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 76 2023-08-25T09:50:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa35c06a050 msgr2=0x7fa35c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa35c06a050 0x7fa35c06c510 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa360036000 tx=0x7fa360030000 comp rx=0 tx=0).stop 2023-08-25T09:50:28.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 msgr2=0x7fa3700c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 0x7fa3700c5f60 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fa37006a620 tx=0x7fa36800b480 comp rx=0 tx=0).stop 2023-08-25T09:50:28.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 shutdown_connections 2023-08-25T09:50:28.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa35c06a050 0x7fa35c06c510 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3700c5ae0 0x7fa3700c5f60 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3700c3c60 0x7fa3700c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 --2- 172.21.15.70:0/3636072860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa358003970 0x7fa37014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 >> 172.21.15.70:0/3636072860 conn(0x7fa370069850 msgr2=0x7fa3700b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:28.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 shutdown_connections 2023-08-25T09:50:28.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.301+0000 7fa375347640 1 -- 172.21.15.70:0/3636072860 wait complete. 2023-08-25T09:50:28.323 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:50:28.324 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:50:28.324 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:50:28.324 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:50:28.324 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:50:28.325 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:50:28.325 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:50:28.325 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:50:28.325 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:50:28.325 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:50:28.326 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:50:28.326 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:50:28.326 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:50:28.326 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:50:28.327 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:50:28.327 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:50:28.340 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:50:28.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:50:28.341 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:50:28.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:50:28.355 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:50:28.355 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:50:28.385 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:50:28.386 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:50:28.394 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:50:28.395 DEBUG:teuthology.parallel:result is None 2023-08-25T09:50:28.404 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:50:28.405 DEBUG:teuthology.parallel:result is None 2023-08-25T09:50:28.435 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:50:28.436 DEBUG:teuthology.parallel:result is None 2023-08-25T09:50:28.442 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:50:28.442 DEBUG:teuthology.parallel:result is None 2023-08-25T09:50:28.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:50:28.558 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:50:28.560 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:50:28.605 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:50:28.606 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:50:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.633+0000 7f92bde7c640 1 Processor -- start 2023-08-25T09:50:28.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.633+0000 7f92bde7c640 1 -- start start 2023-08-25T09:50:28.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.634+0000 7f92bde7c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f92b815acd0 0x7f92b815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.634+0000 7f92bde7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b80cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.634+0000 7f92bde7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92b80b3cf0 0x7f92b80b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.634+0000 7f92bde7c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b806c0e0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.634+0000 7f92bde7c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b80b12b0 con 0x7f92b815acd0 2023-08-25T09:50:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.634+0000 7f92bde7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b806ae20 con 0x7f92b80ca780 2023-08-25T09:50:28.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92b80b3cf0 0x7f92b80b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f92b80b3cf0 0x7f92b80b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39882/0 (socket says 172.21.15.70:39882) 2023-08-25T09:50:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 -- 172.21.15.70:0/1950018992 learned_addr learned my addr 172.21.15.70:0/1950018992 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b77fe640 1 --2- 172.21.15.70:0/1950018992 >> v2:172.21.15.155:3300/0 conn(0x7f92b815acd0 0x7f92b815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92affff640 1 --2- 172.21.15.70:0/1950018992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b80cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 -- 172.21.15.70:0/1950018992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 msgr2=0x7f92b80cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 --2- 172.21.15.70:0/1950018992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b80cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 -- 172.21.15.70:0/1950018992 >> v2:172.21.15.155:3300/0 conn(0x7f92b815acd0 msgr2=0x7f92b815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 --2- 172.21.15.70:0/1950018992 >> v2:172.21.15.155:3300/0 conn(0x7f92b815acd0 0x7f92b815b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.635+0000 7f92b7fff640 1 -- 172.21.15.70:0/1950018992 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b806b910 con 0x7f92b80b3cf0 2023-08-25T09:50:28.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.636+0000 7f92b7fff640 1 --2- 172.21.15.70:0/1950018992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b80b4140 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f92b80714b0 tx=0x7f92b002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a2d34a3331a1e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92b6ffd640 1 -- 172.21.15.70:0/1950018992 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b002c9f0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92b6ffd640 1 -- 172.21.15.70:0/1950018992 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b002cb50 con 0x7f92b80b3cf0 2023-08-25T09:50:28.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92b6ffd640 1 -- 172.21.15.70:0/1950018992 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b00343f0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 -- 172.21.15.70:0/1950018992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 msgr2=0x7f92b80b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 --2- 172.21.15.70:0/1950018992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b80b4140 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f92b80714b0 tx=0x7f92b002be50 comp rx=0 tx=0).stop 2023-08-25T09:50:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 -- 172.21.15.70:0/1950018992 shutdown_connections 2023-08-25T09:50:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 --2- 172.21.15.70:0/1950018992 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b80b4140 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 --2- 172.21.15.70:0/1950018992 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b80cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 --2- 172.21.15.70:0/1950018992 >> v2:172.21.15.155:3300/0 conn(0x7f92b815acd0 0x7f92b815b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 -- 172.21.15.70:0/1950018992 >> 172.21.15.70:0/1950018992 conn(0x7f92b80ce760 msgr2=0x7f92b80ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 -- 172.21.15.70:0/1950018992 shutdown_connections 2023-08-25T09:50:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.637+0000 7f92bde7c640 1 -- 172.21.15.70:0/1950018992 wait complete. 2023-08-25T09:50:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.639+0000 7f92bde7c640 1 Processor -- start 2023-08-25T09:50:28.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.639+0000 7f92bde7c640 1 -- start start 2023-08-25T09:50:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.640+0000 7f92bde7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.640+0000 7f92bde7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b8145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.640+0000 7f92bde7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92b815acd0 0x7f92b80cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.640+0000 7f92bde7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b814efe0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.640+0000 7f92bde7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8145650 con 0x7f92b815acd0 2023-08-25T09:50:28.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.640+0000 7f92bde7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b814f1d0 con 0x7f92b80ca780 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b814eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39886/0 (socket says 172.21.15.70:39886) 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 -- 172.21.15.70:0/2303175121 learned_addr learned my addr 172.21.15.70:0/2303175121 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b7fff640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92b815acd0 0x7f92b80cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92affff640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b8145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 -- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 msgr2=0x7f92b8145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b8145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.646 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 -- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92b815acd0 msgr2=0x7f92b80cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92b815acd0 0x7f92b80cbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 -- 172.21.15.70:0/2303175121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b0009b30 con 0x7f92b80b3cf0 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92affff640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b8145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.641+0000 7f92b77fe640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b814eaa0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e900 tx=0x7f92a800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.642+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a8009660 con 0x7f92b80b3cf0 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.642+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a800c9a0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.642+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800cd10 con 0x7f92b80b3cf0 2023-08-25T09:50:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.642+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b814f510 con 0x7f92b80b3cf0 2023-08-25T09:50:28.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.642+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b814fd50 con 0x7f92b80b3cf0 2023-08-25T09:50:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.643+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b814ff10 con 0x7f92b80b3cf0 2023-08-25T09:50:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.644+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f92a8013470 con 0x7f92b80b3cf0 2023-08-25T09:50:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.645+0000 7f92b4ff9640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f929806a080 0x7f929806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.645+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f92a80887d0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.645+0000 7f92affff640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f929806a080 0x7f929806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:28.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.646+0000 7f92affff640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f929806a080 0x7f929806c540 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f92b80cdc30 tx=0x7f92a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:28.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.649+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f92a80541f0 con 0x7f92b80b3cf0 2023-08-25T09:50:28.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:28.851+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f92b804e420 con 0x7f92b80b3cf0 2023-08-25T09:50:29.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.887+0000 7f92b4ff9640 1 -- 172.21.15.70:0/2303175121 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) v1 ==== 97+0+0 (secure 0 0 0) 0x7f92a8057810 con 0x7f92b80b3cf0 2023-08-25T09:50:29.891 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:50:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f929806a080 msgr2=0x7f929806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f929806a080 0x7f929806c540 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f92b80cdc30 tx=0x7f92a0008040 comp rx=0 tx=0).stop 2023-08-25T09:50:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 msgr2=0x7f92b814eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b814eaa0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e900 tx=0x7f92a800edc0 comp rx=0 tx=0).stop 2023-08-25T09:50:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 shutdown_connections 2023-08-25T09:50:29.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f929806a080 0x7f929806c540 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:29.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f92b815acd0 0x7f92b80cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:29.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f92b80ca780 0x7f92b8145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:29.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 --2- 172.21.15.70:0/2303175121 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f92b80b3cf0 0x7f92b814eaa0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:29.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 >> 172.21.15.70:0/2303175121 conn(0x7f92b80ce760 msgr2=0x7f92b815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:29.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 shutdown_connections 2023-08-25T09:50:29.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:29.891+0000 7f92bde7c640 1 -- 172.21.15.70:0/2303175121 wait complete. 2023-08-25T09:50:29.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:50:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.120+0000 7f70c98ad640 1 Processor -- start 2023-08-25T09:50:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.121+0000 7f70c98ad640 1 -- start start 2023-08-25T09:50:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c98ad640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70c40b12f0 0x7f70c415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.125 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c98ad640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c415e7c0 con 0x7f70c415f190 2023-08-25T09:50:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c98ad640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c4071b50 con 0x7f70c415a7c0 2023-08-25T09:50:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c98ad640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c415e490 con 0x7f70c40b12f0 2023-08-25T09:50:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70c40b12f0 0x7f70c415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f70c40b12f0 0x7f70c415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38542/0 (socket says 172.21.15.70:38542) 2023-08-25T09:50:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c3fff640 1 -- 172.21.15.70:0/1197041631 learned_addr learned my addr 172.21.15.70:0/1197041631 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:30.126 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.122+0000 7f70c90ac640 1 --1- 172.21.15.70:0/1197041631 >> v1:172.21.15.70:6789/0 conn(0x7f70c415f190 0x7f70c40cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42126/0 (socket says 172.21.15.70:42126) 2023-08-25T09:50:30.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.123+0000 7f70c3fff640 1 -- 172.21.15.70:0/1197041631 >> v1:172.21.15.155:6789/0 conn(0x7f70c415a7c0 legacy=0x7f70c40b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:30.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.123+0000 7f70c3fff640 1 -- 172.21.15.70:0/1197041631 >> v1:172.21.15.70:6789/0 conn(0x7f70c415f190 legacy=0x7f70c40cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.123+0000 7f70c3fff640 1 -- 172.21.15.70:0/1197041631 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40b3d60 con 0x7f70c40b12f0 2023-08-25T09:50:30.127 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.123+0000 7f70c37fe640 1 -- 172.21.15.70:0/1197041631 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588173908 0 0) 0x7f70c415e7c0 con 0x7f70c415f190 2023-08-25T09:50:30.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.124+0000 7f70c3fff640 1 --2- 172.21.15.70:0/1197041631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c415ec50 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009870 tx=0x7f70b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11014f6096033da server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:30.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c37fe640 1 -- 172.21.15.70:0/1197041631 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4030860 con 0x7f70c40b12f0 2023-08-25T09:50:30.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c37fe640 1 -- 172.21.15.70:0/1197041631 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b40309c0 con 0x7f70c40b12f0 2023-08-25T09:50:30.128 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c37fe640 1 -- 172.21.15.70:0/1197041631 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4030d50 con 0x7f70c40b12f0 2023-08-25T09:50:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 -- 172.21.15.70:0/1197041631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 msgr2=0x7f70c415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 --2- 172.21.15.70:0/1197041631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c415ec50 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009870 tx=0x7f70b402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 -- 172.21.15.70:0/1197041631 shutdown_connections 2023-08-25T09:50:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 --2- 172.21.15.70:0/1197041631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c415ec50 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 -- 172.21.15.70:0/1197041631 >> 172.21.15.70:0/1197041631 conn(0x7f70c40affd0 msgr2=0x7f70c40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:30.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 -- 172.21.15.70:0/1197041631 shutdown_connections 2023-08-25T09:50:30.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.125+0000 7f70c98ad640 1 -- 172.21.15.70:0/1197041631 wait complete. 2023-08-25T09:50:30.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.127+0000 7f70c98ad640 1 Processor -- start 2023-08-25T09:50:30.130 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.127+0000 7f70c98ad640 1 -- start start 2023-08-25T09:50:30.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.128+0000 7f70c98ad640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.128+0000 7f70c98ad640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70c415a7c0 0x7f70c4070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.131 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.128+0000 7f70c98ad640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c415f190 0x7f70c41507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.128+0000 7f70c98ad640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40c8520 con 0x7f70c40b12f0 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.128+0000 7f70c98ad640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40c4450 con 0x7f70c415f190 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.128+0000 7f70c98ad640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c40c3840 con 0x7f70c415a7c0 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c40ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38558/0 (socket says 172.21.15.70:38558) 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 -- 172.21.15.70:0/4211139264 learned_addr learned my addr 172.21.15.70:0/4211139264 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:30.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 -- 172.21.15.70:0/4211139264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70c415a7c0 msgr2=0x7f70c4070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70c415a7c0 0x7f70c4070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 -- 172.21.15.70:0/4211139264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c415f190 msgr2=0x7f70c41507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c415f190 0x7f70c41507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 -- 172.21.15.70:0/4211139264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40094d0 con 0x7f70c40b12f0 2023-08-25T09:50:30.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.129+0000 7f70c88ab640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c40ce3e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c460 tx=0x7f70b800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:30.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.130+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8012830 con 0x7f70c40b12f0 2023-08-25T09:50:30.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.130+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b80129d0 con 0x7f70c40b12f0 2023-08-25T09:50:30.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.130+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8012d80 con 0x7f70c40b12f0 2023-08-25T09:50:30.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.130+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c40c3b20 con 0x7f70c40b12f0 2023-08-25T09:50:30.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.130+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c40c40e0 con 0x7f70c40b12f0 2023-08-25T09:50:30.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.131+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c4066fb0 con 0x7f70c40b12f0 2023-08-25T09:50:30.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.132+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70b801a3b0 con 0x7f70c40b12f0 2023-08-25T09:50:30.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.133+0000 7f70c17fa640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70b006a050 0x7f70b006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.133+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f70b8087eb0 con 0x7f70c40b12f0 2023-08-25T09:50:30.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.133+0000 7f70c3fff640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70b006a050 0x7f70b006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.134+0000 7f70c3fff640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70b006a050 0x7f70b006c510 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f70b4036000 tx=0x7f70b403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:30.139 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.137+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70b80538e0 con 0x7f70c40b12f0 2023-08-25T09:50:30.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.393+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f70c40b0d00 con 0x7f70c40b12f0 2023-08-25T09:50:30.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.394+0000 7f70c17fa640 1 -- 172.21.15.70:0/4211139264 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f70b8056f00 con 0x7f70c40b12f0 2023-08-25T09:50:30.397 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:30.397 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:50:30.397 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:50:30.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.397 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:50:30.397 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.399 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:50:30.399 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:50:30.401 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.401 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.401 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:50:30.401 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:50:30.401 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.402 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:50:30.403 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.405 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:50:30.405 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:50:30.405 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.405 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.405 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:50:30.406 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.407 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.409 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:50:30.409 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:50:30.409 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.409 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.409 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.410 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.410 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.411 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:50:30.411 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:50:30.411 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.412 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.412 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.412 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.413 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:50:30.413 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:50:30.413 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.414 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.414 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.414 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.415 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.416 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:50:30.417 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:50:30.417 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:50:30.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.419 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:50:30.420 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:50:30.420 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:50:30.420 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.420 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:50:30.421 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:50:30.422 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:50:30.423 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.424 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:50:30.424 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:50:30.424 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:50:30.424 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.425 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:50:30.425 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:50:30.425 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.425 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:50:30.425 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:50:30.426 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:50:30.427 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:50:30.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70b006a050 msgr2=0x7f70b006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70b006a050 0x7f70b006c510 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f70b4036000 tx=0x7f70b403b040 comp rx=0 tx=0).stop 2023-08-25T09:50:30.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 msgr2=0x7f70c40ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c40ce3e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f70b800c460 tx=0x7f70b800c920 comp rx=0 tx=0).stop 2023-08-25T09:50:30.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 shutdown_connections 2023-08-25T09:50:30.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f70b006a050 0x7f70b006c510 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f70c415f190 0x7f70c41507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f70c415a7c0 0x7f70c4070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 --2- 172.21.15.70:0/4211139264 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f70c40b12f0 0x7f70c40ce3e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 >> 172.21.15.70:0/4211139264 conn(0x7f70c40affd0 msgr2=0x7f70c40b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:30.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 shutdown_connections 2023-08-25T09:50:30.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.397+0000 7f70c98ad640 1 -- 172.21.15.70:0/4211139264 wait complete. 2023-08-25T09:50:30.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:30.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.646+0000 7fd00a309640 1 Processor -- start 2023-08-25T09:50:30.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.646+0000 7fd00a309640 1 -- start start 2023-08-25T09:50:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.647+0000 7fd00a309640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 0x7fd0040d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.647+0000 7fd00a309640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd00415ce80 0x7fd0040cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.647+0000 7fd00a309640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0040d11d0 con 0x7fd0040b8100 2023-08-25T09:50:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.647+0000 7fd00a309640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0040d1470 con 0x7fd00415ce80 2023-08-25T09:50:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.647+0000 7fd00a309640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0040d1750 con 0x7fd00415caa0 2023-08-25T09:50:30.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.648+0000 7fd009307640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 0x7fd0040d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.648+0000 7fd008b06640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd00415ce80 0x7fd0040cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.648+0000 7fd009307640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 0x7fd0040d0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:59774/0 (socket says 172.21.15.70:59774) 2023-08-25T09:50:30.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.648+0000 7fd009307640 1 -- 172.21.15.70:0/2734661712 learned_addr learned my addr 172.21.15.70:0/2734661712 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:30.651 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.649+0000 7fd008b06640 1 -- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 msgr2=0x7fd0040d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.649+0000 7fd008b06640 1 --2- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 0x7fd0040d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.649+0000 7fd008b06640 1 -- 172.21.15.70:0/2734661712 >> v1:172.21.15.155:6789/0 conn(0x7fd0040b8100 legacy=0x7fd004069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.649+0000 7fd008b06640 1 -- 172.21.15.70:0/2734661712 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0040d1a30 con 0x7fd00415ce80 2023-08-25T09:50:30.652 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.649+0000 7fd009307640 1 --2- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 0x7fd0040d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:50:30.653 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.651+0000 7fd008b06640 1 --2- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415ce80 0x7fd0040cf290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcff40097a0 tx=0x7fcff402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d169661777bbc2c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:30.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.651+0000 7fcffbfff640 1 -- 172.21.15.70:0/2734661712 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff402d040 con 0x7fd00415ce80 2023-08-25T09:50:30.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.651+0000 7fcffbfff640 1 -- 172.21.15.70:0/2734661712 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff4031810 con 0x7fd00415ce80 2023-08-25T09:50:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.651+0000 7fcffbfff640 1 -- 172.21.15.70:0/2734661712 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff4031ba0 con 0x7fd00415ce80 2023-08-25T09:50:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 -- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415ce80 msgr2=0x7fd0040cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 --2- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415ce80 0x7fd0040cf290 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcff40097a0 tx=0x7fcff402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 -- 172.21.15.70:0/2734661712 shutdown_connections 2023-08-25T09:50:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 --2- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415ce80 0x7fd0040cf290 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 --2- 172.21.15.70:0/2734661712 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415caa0 0x7fd0040d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 -- 172.21.15.70:0/2734661712 >> 172.21.15.70:0/2734661712 conn(0x7fd00406fe70 msgr2=0x7fd004070250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:30.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 -- 172.21.15.70:0/2734661712 shutdown_connections 2023-08-25T09:50:30.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.652+0000 7fd00a309640 1 -- 172.21.15.70:0/2734661712 wait complete. 2023-08-25T09:50:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.655+0000 7fd00a309640 1 Processor -- start 2023-08-25T09:50:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.655+0000 7fd00a309640 1 -- start start 2023-08-25T09:50:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 0x7fd00414d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415caa0 0x7fd0040c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415ce80 0x7fd0040d2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004158ee0 con 0x7fd0040b8100 2023-08-25T09:50:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041591c0 con 0x7fd00415caa0 2023-08-25T09:50:30.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0041594a0 con 0x7fd00415ce80 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 0x7fd00414d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 0x7fd00414d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38560/0 (socket says 172.21.15.70:38560) 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 -- 172.21.15.70:0/4177582812 learned_addr learned my addr 172.21.15.70:0/4177582812 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 -- 172.21.15.70:0/4177582812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415ce80 msgr2=0x7fd0040d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415ce80 0x7fd0040d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 -- 172.21.15.70:0/4177582812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415caa0 msgr2=0x7fd0040c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415caa0 0x7fd0040c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 -- 172.21.15.70:0/4177582812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000094d0 con 0x7fd0040b8100 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd009307640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 0x7fd00414d010 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e9d0 tx=0x7fd00000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00000ca60 con 0x7fd0040b8100 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00000cbc0 con 0x7fd0040b8100 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000013470 con 0x7fd0040b8100 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.657+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff4009400 con 0x7fd0040b8100 2023-08-25T09:50:30.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.658+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd004159a60 con 0x7fd0040b8100 2023-08-25T09:50:30.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.659+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd004066fb0 con 0x7fd0040b8100 2023-08-25T09:50:30.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.660+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd0000135d0 con 0x7fd0040b8100 2023-08-25T09:50:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.661+0000 7fcff9ffb640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcfe806a080 0x7fcfe806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.661+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd000088590 con 0x7fd0040b8100 2023-08-25T09:50:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.661+0000 7fd008b06640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcfe806a080 0x7fcfe806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:30.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.662+0000 7fd008b06640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcfe806a080 0x7fcfe806c540 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcff4036000 tx=0x7fcff4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:30.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.665+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd000053fc0 con 0x7fd0040b8100 2023-08-25T09:50:30.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.854+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0040c5d80 con 0x7fd0040b8100 2023-08-25T09:50:30.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.856+0000 7fcff9ffb640 1 -- 172.21.15.70:0/4177582812 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+14665 (secure 0 0 0) 0x7fd0000575e0 con 0x7fd0040b8100 2023-08-25T09:50:30.858 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:30.859 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":178,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:29.880976+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":167,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.858+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcfe806a080 msgr2=0x7fcfe806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.858+0000 7fd00a309640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcfe806a080 0x7fcfe806c540 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcff4036000 tx=0x7fcff4030000 comp rx=0 tx=0).stop 2023-08-25T09:50:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.858+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 msgr2=0x7fd00414d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.858+0000 7fd00a309640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 0x7fd00414d010 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fd00000e9d0 tx=0x7fd00000ee90 comp rx=0 tx=0).stop 2023-08-25T09:50:30.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.858+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 shutdown_connections 2023-08-25T09:50:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.858+0000 7fd00a309640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcfe806a080 0x7fcfe806c540 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.859+0000 7fd00a309640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd00415ce80 0x7fd0040d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.859+0000 7fd00a309640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd00415caa0 0x7fd0040c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.859+0000 7fd00a309640 1 --2- 172.21.15.70:0/4177582812 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd0040b8100 0x7fd00414d010 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:30.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.859+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 >> 172.21.15.70:0/4177582812 conn(0x7fd00406fe70 msgr2=0x7fd0040cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.859+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 shutdown_connections 2023-08-25T09:50:30.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:30.859+0000 7fd00a309640 1 -- 172.21.15.70:0/4177582812 wait complete. 2023-08-25T09:50:30.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:50:30.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:50:31.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.096+0000 7f6cb87cf640 1 Processor -- start 2023-08-25T09:50:31.098 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.096+0000 7f6cb87cf640 1 -- start start 2023-08-25T09:50:31.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.098+0000 7f6cb87cf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6cb015cd70 0x7f6cb00aca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.099 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.098+0000 7f6cb87cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 0x7f6cb006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.098+0000 7f6cb87cf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cb00b31a0 con 0x7f6cb0072820 2023-08-25T09:50:31.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.098+0000 7f6cb87cf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb015d150 con 0x7f6cb015cd70 2023-08-25T09:50:31.100 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.098+0000 7f6cb87cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00ad360 con 0x7f6cb00b2c30 2023-08-25T09:50:31.101 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb6d6c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f6cb0072820 0x7f6cb00b3bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42142/0 (socket says 172.21.15.70:42142) 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb6d6c640 1 -- 172.21.15.70:0/1328787140 learned_addr learned my addr 172.21.15.70:0/1328787140 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb656b640 1 --2- 172.21.15.70:0/1328787140 >> v2:172.21.15.155:3300/0 conn(0x7f6cb015cd70 0x7f6cb00aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5d6a640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 0x7f6cb006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701526752 0 0) 0x7f6cb00b31a0 con 0x7f6cb0072820 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c9c003520 con 0x7f6cb0072820 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248897544 0 0) 0x7f6c9c003520 con 0x7f6cb0072820 2023-08-25T09:50:31.102 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 msgr2=0x7f6cb006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 0x7f6cb006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 >> v2:172.21.15.155:3300/0 conn(0x7f6cb015cd70 msgr2=0x7f6cb00aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 --2- 172.21.15.70:0/1328787140 >> v2:172.21.15.155:3300/0 conn(0x7f6cb015cd70 0x7f6cb00aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb00ad640 con 0x7f6cb0072820 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6ca80019f0 con 0x7f6cb0072820 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 >> v1:172.21.15.70:6789/0 conn(0x7f6cb0072820 legacy=0x7f6cb00b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.099+0000 7f6cb5569640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c9c003970 0x7f6c9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb5569640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 0x7f6c9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb5569640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6c9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb015d150 con 0x7f6c9c004d00 2023-08-25T09:50:31.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00ad360 con 0x7f6c9c003970 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c00b980 con 0x7f6cb00b2c30 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb6d6c640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6c9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb5d6a640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 0x7f6c9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb6d6c640 1 -- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 msgr2=0x7f6c9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb6d6c640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb00b2c30 0x7f6c9c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb6d6c640 1 -- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c9c003970 msgr2=0x7f6c9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb656b640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c9c003970 0x7f6c9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb6d6c640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6c9c003970 0x7f6c9c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.106 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.100+0000 7f6cb6d6c640 1 -- 172.21.15.70:0/1328787140 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c003520 con 0x7f6c9c004d00 2023-08-25T09:50:31.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.101+0000 7f6cb6d6c640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6c9c005180 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800a8d0 tx=0x7f6c9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e162e874c3d657 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8034630 con 0x7f6c9c004d00 2023-08-25T09:50:31.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca802f070 con 0x7f6c9c004d00 2023-08-25T09:50:31.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb5569640 1 -- 172.21.15.70:0/1328787140 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80349d0 con 0x7f6c9c004d00 2023-08-25T09:50:31.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb87cf640 1 -- 172.21.15.70:0/1328787140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 msgr2=0x7f6c9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/1328787140 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6c9c005180 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800a8d0 tx=0x7f6c9c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb6d6c640 1 -- 172.21.15.70:0/1328787140 reap_dead start 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb87cf640 1 -- 172.21.15.70:0/1328787140 shutdown_connections 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb87cf640 1 -- 172.21.15.70:0/1328787140 >> 172.21.15.70:0/1328787140 conn(0x7f6cb00cfc70 msgr2=0x7f6cb00d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb87cf640 1 -- 172.21.15.70:0/1328787140 shutdown_connections 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.102+0000 7f6cb87cf640 1 -- 172.21.15.70:0/1328787140 wait complete. 2023-08-25T09:50:31.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.104+0000 7f6cb87cf640 1 Processor -- start 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.104+0000 7f6cb87cf640 1 -- start start 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb87cf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6cb015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb87cf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb014f630 0x7f6cb015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb87cf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb00b08b0 0x7f6cb00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb87cf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb00b1270 con 0x7f6c9c004d00 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb87cf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb015f750 con 0x7f6cb00b08b0 2023-08-25T09:50:31.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb87cf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb015b160 con 0x7f6cb014f630 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb656b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6cb015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb656b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6cb015fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38582/0 (socket says 172.21.15.70:38582) 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.105+0000 7f6cb656b640 1 -- 172.21.15.70:0/3757617398 learned_addr learned my addr 172.21.15.70:0/3757617398 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb656b640 1 -- 172.21.15.70:0/3757617398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb014f630 msgr2=0x7f6cb015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb656b640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb014f630 0x7f6cb015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb656b640 1 -- 172.21.15.70:0/3757617398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb00b08b0 msgr2=0x7f6cb00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb656b640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb00b08b0 0x7f6cb00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb656b640 1 -- 172.21.15.70:0/3757617398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb015ac80 con 0x7f6c9c004d00 2023-08-25T09:50:31.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb656b640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6cb015fa40 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f6cac010910 tx=0x7f6cac010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac009690 con 0x7f6c9c004d00 2023-08-25T09:50:31.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00eba0 con 0x7f6c9c004d00 2023-08-25T09:50:31.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac01c400 con 0x7f6c9c004d00 2023-08-25T09:50:31.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.106+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb015ee20 con 0x7f6c9c004d00 2023-08-25T09:50:31.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.107+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb015f3e0 con 0x7f6c9c004d00 2023-08-25T09:50:31.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.108+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb015ae10 con 0x7f6c9c004d00 2023-08-25T09:50:31.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.109+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6cac01c560 con 0x7f6c9c004d00 2023-08-25T09:50:31.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.110+0000 7f6ca6ffd640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c9406a080 0x7f6c9406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.110+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6cac088810 con 0x7f6c9c004d00 2023-08-25T09:50:31.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.110+0000 7f6cb5d6a640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c9406a080 0x7f6c9406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.111+0000 7f6cb5d6a640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c9406a080 0x7f6c9406c540 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00b4200 tx=0x7f6ca0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.114+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6cac0542f0 con 0x7f6c9c004d00 2023-08-25T09:50:31.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.303+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6cb0066fb0 con 0x7f6c9c004d00 2023-08-25T09:50:31.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.304+0000 7f6ca6ffd640 1 -- 172.21.15.70:0/3757617398 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v178) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6cac057910 con 0x7f6c9c004d00 2023-08-25T09:50:31.305 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:50:31.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c9406a080 msgr2=0x7f6c9406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c9406a080 0x7f6c9406c540 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00b4200 tx=0x7f6ca0008040 comp rx=0 tx=0).stop 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 msgr2=0x7f6cb015fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6cb015fa40 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f6cac010910 tx=0x7f6cac010dd0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 shutdown_connections 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6c9406a080 0x7f6c9406c540 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6cb00b08b0 0x7f6cb00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6cb014f630 0x7f6cb015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 --2- 172.21.15.70:0/3757617398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6c9c004d00 0x7f6cb015fa40 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 >> 172.21.15.70:0/3757617398 conn(0x7f6cb00cfc70 msgr2=0x7f6cb0069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 shutdown_connections 2023-08-25T09:50:31.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.307+0000 7f6cb87cf640 1 -- 172.21.15.70:0/3757617398 wait complete. 2023-08-25T09:50:31.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:50:31.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.542+0000 7f4ef963f640 1 Processor -- start 2023-08-25T09:50:31.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.542+0000 7f4ef963f640 1 -- start start 2023-08-25T09:50:31.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.543+0000 7f4ef963f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ef415ae50 0x7f4ef40c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.543+0000 7f4ef963f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4ef40cdc20 0x7f4ef4069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.543+0000 7f4ef963f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef40ce060 0x7f4ef4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.543+0000 7f4ef963f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef40cc5f0 con 0x7f4ef415ae50 2023-08-25T09:50:31.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.543+0000 7f4ef963f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef4070650 con 0x7f4ef40cdc20 2023-08-25T09:50:31.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.543+0000 7f4ef963f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef4070930 con 0x7f4ef40ce060 2023-08-25T09:50:31.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ef415ae50 0x7f4ef40c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4ef415ae50 0x7f4ef40c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38590/0 (socket says 172.21.15.70:38590) 2023-08-25T09:50:31.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/407017315 learned_addr learned my addr 172.21.15.70:0/407017315 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef27fc640 1 --2- 172.21.15.70:0/407017315 >> v2:172.21.15.155:3300/0 conn(0x7f4ef40cdc20 0x7f4ef4069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef37fe640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef40ce060 0x7f4ef4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/407017315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef40ce060 msgr2=0x7f4ef4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef40ce060 0x7f4ef4069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/407017315 >> v2:172.21.15.155:3300/0 conn(0x7f4ef40cdc20 msgr2=0x7f4ef4069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 --2- 172.21.15.70:0/407017315 >> v2:172.21.15.155:3300/0 conn(0x7f4ef40cdc20 0x7f4ef4069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/407017315 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef4070c10 con 0x7f4ef415ae50 2023-08-25T09:50:31.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef37fe640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef40ce060 0x7f4ef4069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:31.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.544+0000 7f4ef27fc640 1 --2- 172.21.15.70:0/407017315 >> v2:172.21.15.155:3300/0 conn(0x7f4ef40cdc20 0x7f4ef4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:31.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.545+0000 7f4ef2ffd640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef415ae50 0x7f4ef40c2110 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40cb030 tx=0x7f4ee402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf04f3425fecbd90 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef1ffb640 1 -- 172.21.15.70:0/407017315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee402cb00 con 0x7f4ef415ae50 2023-08-25T09:50:31.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef1ffb640 1 -- 172.21.15.70:0/407017315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee402cc60 con 0x7f4ef415ae50 2023-08-25T09:50:31.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef1ffb640 1 -- 172.21.15.70:0/407017315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40343f0 con 0x7f4ef415ae50 2023-08-25T09:50:31.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 -- 172.21.15.70:0/407017315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef415ae50 msgr2=0x7f4ef40c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef415ae50 0x7f4ef40c2110 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40cb030 tx=0x7f4ee402bf60 comp rx=0 tx=0).stop 2023-08-25T09:50:31.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 -- 172.21.15.70:0/407017315 shutdown_connections 2023-08-25T09:50:31.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef40ce060 0x7f4ef4069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 --2- 172.21.15.70:0/407017315 >> v2:172.21.15.155:3300/0 conn(0x7f4ef40cdc20 0x7f4ef4069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 --2- 172.21.15.70:0/407017315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef415ae50 0x7f4ef40c2110 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 -- 172.21.15.70:0/407017315 >> 172.21.15.70:0/407017315 conn(0x7f4ef40cb0d0 msgr2=0x7f4ef40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:31.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 -- 172.21.15.70:0/407017315 shutdown_connections 2023-08-25T09:50:31.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.546+0000 7f4ef963f640 1 -- 172.21.15.70:0/407017315 wait complete. 2023-08-25T09:50:31.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.548+0000 7f4ef963f640 1 Processor -- start 2023-08-25T09:50:31.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.548+0000 7f4ef963f640 1 -- start start 2023-08-25T09:50:31.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.549+0000 7f4ef963f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 0x7f4ef4158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.549+0000 7f4ef963f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef40ce060 0x7f4ef41e4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef963f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef415ae50 0x7f4ef41e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef963f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40c4040 con 0x7f4ef40cdc20 2023-08-25T09:50:31.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef963f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40c57a0 con 0x7f4ef40ce060 2023-08-25T09:50:31.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef963f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40c5a80 con 0x7f4ef415ae50 2023-08-25T09:50:31.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 0x7f4ef4158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 0x7f4ef4158370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38604/0 (socket says 172.21.15.70:38604) 2023-08-25T09:50:31.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/70281434 learned_addr learned my addr 172.21.15.70:0/70281434 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:31.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/70281434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef415ae50 msgr2=0x7f4ef41e4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef415ae50 0x7f4ef41e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/70281434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef40ce060 msgr2=0x7f4ef41e4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef40ce060 0x7f4ef41e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.550+0000 7f4ef2ffd640 1 -- 172.21.15.70:0/70281434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee4009b30 con 0x7f4ef40cdc20 2023-08-25T09:50:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.551+0000 7f4ef2ffd640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 0x7f4ef4158370 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4000f80 tx=0x7f4ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.551+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee403e450 con 0x7f4ef40cdc20 2023-08-25T09:50:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.551+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee403e5f0 con 0x7f4ef40cdc20 2023-08-25T09:50:31.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.551+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee403e9a0 con 0x7f4ef40cdc20 2023-08-25T09:50:31.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.551+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef40c5d60 con 0x7f4ef40cdc20 2023-08-25T09:50:31.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.551+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef41588b0 con 0x7f4ef40cdc20 2023-08-25T09:50:31.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.552+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef40ad210 con 0x7f4ef40cdc20 2023-08-25T09:50:31.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.553+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4ee403eb00 con 0x7f4ef40cdc20 2023-08-25T09:50:31.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.554+0000 7f4eeb7fe640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4ed406a050 0x7f4ed406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.554+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4ee40ab250 con 0x7f4ef40cdc20 2023-08-25T09:50:31.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.554+0000 7f4ef27fc640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4ed406a050 0x7f4ed406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.555+0000 7f4ef27fc640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4ed406a050 0x7f4ed406c510 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4edc006fd0 tx=0x7f4edc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.558+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ee4076da0 con 0x7f4ef40cdc20 2023-08-25T09:50:31.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.747+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4ef4066fb0 con 0x7f4ef40cdc20 2023-08-25T09:50:31.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.748+0000 7f4eeb7fe640 1 -- 172.21.15.70:0/70281434 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v178) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4ee403edb0 con 0x7f4ef40cdc20 2023-08-25T09:50:31.749 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4ed406a050 msgr2=0x7f4ed406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4ed406a050 0x7f4ed406c510 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4edc006fd0 tx=0x7f4edc008040 comp rx=0 tx=0).stop 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 msgr2=0x7f4ef4158370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 0x7f4ef4158370 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4000f80 tx=0x7f4ee4004a40 comp rx=0 tx=0).stop 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 shutdown_connections 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4ed406a050 0x7f4ed406c510 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4ef415ae50 0x7f4ef41e4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4ef40ce060 0x7f4ef41e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 --2- 172.21.15.70:0/70281434 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4ef40cdc20 0x7f4ef4158370 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.750+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 >> 172.21.15.70:0/70281434 conn(0x7f4ef40cb0d0 msgr2=0x7f4ef40adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:31.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.751+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 shutdown_connections 2023-08-25T09:50:31.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.751+0000 7f4ef963f640 1 -- 172.21.15.70:0/70281434 wait complete. 2023-08-25T09:50:31.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:50:31.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.966+0000 7fa867984640 1 Processor -- start 2023-08-25T09:50:31.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.966+0000 7fa867984640 1 -- start start 2023-08-25T09:50:31.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa867984640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa86015a8e0 0x7fa8600cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa867984640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86015acc0 0x7fa86006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa867984640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa860070500 0x7fa860072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa867984640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86006fdd0 con 0x7fa86015a8e0 2023-08-25T09:50:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa867984640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8600cb850 con 0x7fa860070500 2023-08-25T09:50:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa867984640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600cbb30 con 0x7fa86015acc0 2023-08-25T09:50:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa865720640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa86015a8e0 0x7fa8600cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa865720640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa86015a8e0 0x7fa8600cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38618/0 (socket says 172.21.15.70:38618) 2023-08-25T09:50:31.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.968+0000 7fa865720640 1 -- 172.21.15.70:0/2740926646 learned_addr learned my addr 172.21.15.70:0/2740926646 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:31.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa864f1f640 1 --2- 172.21.15.70:0/2740926646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86015acc0 0x7fa86006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa865f21640 1 --2- 172.21.15.70:0/2740926646 >> v2:172.21.15.155:3300/0 conn(0x7fa860070500 0x7fa860072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa865720640 1 -- 172.21.15.70:0/2740926646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86015acc0 msgr2=0x7fa86006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa865720640 1 --2- 172.21.15.70:0/2740926646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86015acc0 0x7fa86006aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa865720640 1 -- 172.21.15.70:0/2740926646 >> v2:172.21.15.155:3300/0 conn(0x7fa860070500 msgr2=0x7fa860072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:31.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa865720640 1 --2- 172.21.15.70:0/2740926646 >> v2:172.21.15.155:3300/0 conn(0x7fa860070500 0x7fa860072820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.969+0000 7fa865720640 1 -- 172.21.15.70:0/2740926646 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600cbe10 con 0x7fa86015a8e0 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.970+0000 7fa865720640 1 --2- 172.21.15.70:0/2740926646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa8600cb250 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa8600cf770 tx=0x7fa85002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193913efffb3eee6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.970+0000 7fa85ffff640 1 -- 172.21.15.70:0/2740926646 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85002ca80 con 0x7fa86015a8e0 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.970+0000 7fa85ffff640 1 -- 172.21.15.70:0/2740926646 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85002cbe0 con 0x7fa86015a8e0 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.970+0000 7fa85ffff640 1 -- 172.21.15.70:0/2740926646 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8500343f0 con 0x7fa86015a8e0 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 -- 172.21.15.70:0/2740926646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 msgr2=0x7fa8600cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 --2- 172.21.15.70:0/2740926646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa8600cb250 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fa8600cf770 tx=0x7fa85002bee0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 -- 172.21.15.70:0/2740926646 shutdown_connections 2023-08-25T09:50:31.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 --2- 172.21.15.70:0/2740926646 >> v2:172.21.15.155:3300/0 conn(0x7fa860070500 0x7fa860072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 --2- 172.21.15.70:0/2740926646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa86015acc0 0x7fa86006aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 --2- 172.21.15.70:0/2740926646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa8600cb250 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 -- 172.21.15.70:0/2740926646 >> 172.21.15.70:0/2740926646 conn(0x7fa8600cf9c0 msgr2=0x7fa8600cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:31.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 -- 172.21.15.70:0/2740926646 shutdown_connections 2023-08-25T09:50:31.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.971+0000 7fa867984640 1 -- 172.21.15.70:0/2740926646 wait complete. 2023-08-25T09:50:31.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.973+0000 7fa867984640 1 Processor -- start 2023-08-25T09:50:31.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.973+0000 7fa867984640 1 -- start start 2023-08-25T09:50:31.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa867984640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa860070500 0x7fa8601f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa867984640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa860158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa867984640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86015acc0 0x7fa8600b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa867984640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601f6660 con 0x7fa86015a8e0 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa867984640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860159500 con 0x7fa86015acc0 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa867984640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600b44d0 con 0x7fa860070500 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa864f1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa860158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa864f1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa860158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38628/0 (socket says 172.21.15.70:38628) 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.974+0000 7fa864f1f640 1 -- 172.21.15.70:0/2522315173 learned_addr learned my addr 172.21.15.70:0/2522315173 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa864f1f640 1 -- 172.21.15.70:0/2522315173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa860070500 msgr2=0x7fa8601f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa864f1f640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa860070500 0x7fa8601f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa864f1f640 1 -- 172.21.15.70:0/2522315173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86015acc0 msgr2=0x7fa8600b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:31.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa864f1f640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86015acc0 0x7fa8600b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:31.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa864f1f640 1 -- 172.21.15.70:0/2522315173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850009b30 con 0x7fa86015a8e0 2023-08-25T09:50:31.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa864f1f640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa860158fc0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c7f0 tx=0x7fa85400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540077c0 con 0x7fa86015a8e0 2023-08-25T09:50:31.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854007920 con 0x7fa86015a8e0 2023-08-25T09:50:31.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854007c90 con 0x7fa86015a8e0 2023-08-25T09:50:31.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8600b1bd0 con 0x7fa86015a8e0 2023-08-25T09:50:31.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.975+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8600b2190 con 0x7fa86015a8e0 2023-08-25T09:50:31.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.976+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86004e420 con 0x7fa86015a8e0 2023-08-25T09:50:31.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.977+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8540113f0 con 0x7fa86015a8e0 2023-08-25T09:50:31.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.978+0000 7fa85dffb640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:31.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.978+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa8540881a0 con 0x7fa86015a8e0 2023-08-25T09:50:31.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.978+0000 7fa865720640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:31.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.979+0000 7fa865720640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa850035000 tx=0x7fa85002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:31.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:31.982+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa854053b40 con 0x7fa86015a8e0 2023-08-25T09:50:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.159+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa86006a010 con 0x7fa86015a8e0 2023-08-25T09:50:32.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.160+0000 7fa85dffb640 1 -- 172.21.15.70:0/2522315173 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v178) v1 ==== 84+0+126 (secure 0 0 0) 0x7fa854057160 con 0x7fa86015a8e0 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:32.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:50:32.163 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:50:32.163 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:50:32.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:32.163 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:50:32.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 msgr2=0x7fa84c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:32.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa850035000 tx=0x7fa85002e000 comp rx=0 tx=0).stop 2023-08-25T09:50:32.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 msgr2=0x7fa860158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:32.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa860158fc0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa85400c7f0 tx=0x7fa85400ccb0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 shutdown_connections 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa84c06a050 0x7fa84c06c510 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa86015acc0 0x7fa8600b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa86015a8e0 0x7fa860158fc0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 --2- 172.21.15.70:0/2522315173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa860070500 0x7fa8601f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 >> 172.21.15.70:0/2522315173 conn(0x7fa8600cf9c0 msgr2=0x7fa86006b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:32.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 shutdown_connections 2023-08-25T09:50:32.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.162+0000 7fa867984640 1 -- 172.21.15.70:0/2522315173 wait complete. 2023-08-25T09:50:32.181 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:50:32.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:50:32.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.385+0000 7facdffff640 1 Processor -- start 2023-08-25T09:50:32.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.385+0000 7facdffff640 1 -- start start 2023-08-25T09:50:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.386+0000 7facdffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face01597d0 0x7face00ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.386+0000 7facdffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7face00b0740 0x7face00cc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdffff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face00b2d40 con 0x7face0159bb0 2023-08-25T09:50:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdffff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face00b3020 con 0x7face00b0740 2023-08-25T09:50:32.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00b3300 con 0x7face01597d0 2023-08-25T09:50:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7face00b0740 0x7face00cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7face00b0740 0x7face00cc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38638/0 (socket says 172.21.15.70:38638) 2023-08-25T09:50:32.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 -- 172.21.15.70:0/1613928933 learned_addr learned my addr 172.21.15.70:0/1613928933 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdeffd640 1 --2- 172.21.15.70:0/1613928933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face01597d0 0x7face00ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 -- 172.21.15.70:0/1613928933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face01597d0 msgr2=0x7face00ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 --2- 172.21.15.70:0/1613928933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face01597d0 0x7face00ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 -- 172.21.15.70:0/1613928933 >> v1:172.21.15.155:6789/0 conn(0x7face0159bb0 legacy=0x7face00b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:32.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.387+0000 7facdf7fe640 1 -- 172.21.15.70:0/1613928933 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face00b35e0 con 0x7face00b0740 2023-08-25T09:50:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facdf7fe640 1 --2- 172.21.15.70:0/1613928933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face00b0740 0x7face00cc180 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7facd800d150 tx=0x7facd802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7176e2c6a64e5d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facddffb640 1 -- 172.21.15.70:0/1613928933 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd802e9e0 con 0x7face00b0740 2023-08-25T09:50:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facddffb640 1 -- 172.21.15.70:0/1613928933 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd802eb40 con 0x7face00b0740 2023-08-25T09:50:32.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facddffb640 1 -- 172.21.15.70:0/1613928933 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8009600 con 0x7face00b0740 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facdffff640 1 -- 172.21.15.70:0/1613928933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face00b0740 msgr2=0x7face00cc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facdffff640 1 --2- 172.21.15.70:0/1613928933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face00b0740 0x7face00cc180 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7facd800d150 tx=0x7facd802e040 comp rx=0 tx=0).stop 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facdffff640 1 -- 172.21.15.70:0/1613928933 shutdown_connections 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.389+0000 7facdffff640 1 --2- 172.21.15.70:0/1613928933 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face00b0740 0x7face00cc180 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.390+0000 7facdffff640 1 --2- 172.21.15.70:0/1613928933 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face01597d0 0x7face00ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.390+0000 7facdffff640 1 -- 172.21.15.70:0/1613928933 >> 172.21.15.70:0/1613928933 conn(0x7face006b560 msgr2=0x7face00cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:32.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.390+0000 7facdffff640 1 -- 172.21.15.70:0/1613928933 shutdown_connections 2023-08-25T09:50:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.390+0000 7facdffff640 1 -- 172.21.15.70:0/1613928933 wait complete. 2023-08-25T09:50:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.391+0000 7facdffff640 1 Processor -- start 2023-08-25T09:50:32.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.391+0000 7facdffff640 1 -- start start 2023-08-25T09:50:32.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.392+0000 7facdffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7face00b0740 0x7face014e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.392+0000 7facdffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 0x7face00c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.392+0000 7facdffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face0159bb0 0x7face015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.392+0000 7facdffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face00c3670 con 0x7face01597d0 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.392+0000 7facdffff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face00c2a60 con 0x7face00b0740 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.392+0000 7facdffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face00c2d40 con 0x7face0159bb0 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 0x7face00c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:32.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 0x7face00c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38644/0 (socket says 172.21.15.70:38644) 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 -- 172.21.15.70:0/231268210 learned_addr learned my addr 172.21.15.70:0/231268210 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facdeffd640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7face00b0740 0x7face014e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 -- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face0159bb0 msgr2=0x7face015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face0159bb0 0x7face015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 -- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7face00b0740 msgr2=0x7face014e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7face00b0740 0x7face014e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 -- 172.21.15.70:0/231268210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd800bd40 con 0x7face01597d0 2023-08-25T09:50:32.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.393+0000 7facde7fc640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 0x7face00c20c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7facd400b3c0 tx=0x7facd400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:32.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.394+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4011830 con 0x7face01597d0 2023-08-25T09:50:32.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.394+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4011990 con 0x7face01597d0 2023-08-25T09:50:32.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.394+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4011d80 con 0x7face01597d0 2023-08-25T09:50:32.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.394+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face00c3020 con 0x7face01597d0 2023-08-25T09:50:32.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.394+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face0071570 con 0x7face01597d0 2023-08-25T09:50:32.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.395+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face0066fb0 con 0x7face01597d0 2023-08-25T09:50:32.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.396+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7facd401b460 con 0x7face01597d0 2023-08-25T09:50:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.397+0000 7facc77fe640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faccc06a080 0x7faccc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.397+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 5210+0+0 (secure 0 0 0) 0x7facd4087e50 con 0x7face01597d0 2023-08-25T09:50:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.397+0000 7facdeffd640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faccc06a080 0x7faccc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:32.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.398+0000 7facdeffd640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faccc06a080 0x7faccc06c540 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7facd0006fd0 tx=0x7facd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:32.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.401+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7facd40537f0 con 0x7face01597d0 2023-08-25T09:50:32.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:32.591+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7face00adee0 con 0x7face01597d0 2023-08-25T09:50:33.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.482+0000 7facc77fe640 1 -- 172.21.15.70:0/231268210 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v179) v1 ==== 138+0+0 (secure 0 0 0) 0x7facd4056e10 con 0x7face01597d0 2023-08-25T09:50:33.484 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:50:33.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faccc06a080 msgr2=0x7faccc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faccc06a080 0x7faccc06c540 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7facd0006fd0 tx=0x7facd0008040 comp rx=0 tx=0).stop 2023-08-25T09:50:33.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 msgr2=0x7face00c20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 0x7face00c20c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7facd400b3c0 tx=0x7facd400b880 comp rx=0 tx=0).stop 2023-08-25T09:50:33.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 shutdown_connections 2023-08-25T09:50:33.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7faccc06a080 0x7faccc06c540 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7face0159bb0 0x7face015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7face01597d0 0x7face00c20c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 --2- 172.21.15.70:0/231268210 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7face00b0740 0x7face014e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.487+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 >> 172.21.15.70:0/231268210 conn(0x7face006b560 msgr2=0x7face0158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:33.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.488+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 shutdown_connections 2023-08-25T09:50:33.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.488+0000 7facdffff640 1 -- 172.21.15.70:0/231268210 wait complete. 2023-08-25T09:50:33.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:50:33.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.731+0000 7f16a574f640 1 Processor -- start 2023-08-25T09:50:33.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.731+0000 7f16a574f640 1 -- start start 2023-08-25T09:50:33.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.733+0000 7f16a574f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f16a00b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.733+0000 7f16a574f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a006bfc0 con 0x7f16a0159870 2023-08-25T09:50:33.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.733+0000 7f16a574f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a006c2a0 con 0x7f16a00700f0 2023-08-25T09:50:33.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.733+0000 7f16a574f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0159c50 con 0x7f16a0070550 2023-08-25T09:50:33.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.733+0000 7f169ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f16a0159870 0x7f16a015a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42156/0 (socket says 172.21.15.70:42156) 2023-08-25T09:50:33.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.733+0000 7f169ffff640 1 -- 172.21.15.70:0/117931679 learned_addr learned my addr 172.21.15.70:0/117931679 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:33.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644898991 0 0) 0x7f16a006bfc0 con 0x7f16a0159870 2023-08-25T09:50:33.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1688005730 con 0x7f16a0159870 2023-08-25T09:50:33.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f16a4f4e640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f16a00b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:33.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 51989120 0 0) 0x7f1688005730 con 0x7f16a0159870 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 msgr2=0x7f16a00b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f16a00b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 >> v1:172.21.15.155:6789/0 conn(0x7f16a00700f0 legacy=0x7f16a00b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a006c580 con 0x7f16a0159870 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1690001670 con 0x7f16a0159870 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 >> v1:172.21.15.70:6789/0 conn(0x7f16a0159870 legacy=0x7f16a015a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f1688005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16880069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1688006f10 0x7f1688007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0159c50 con 0x7f16880061e0 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168800dd50 con 0x7f1688006f10 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.734+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168800df10 con 0x7f16a0070550 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169f7fe640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16880069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169ffff640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f1688005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:33.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169f7fe640 1 -- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 msgr2=0x7f1688005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:33.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169f7fe640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f1688005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169f7fe640 1 -- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1688006f10 msgr2=0x7f1688007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:33.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169f7fe640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1688006f10 0x7f1688007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169f7fe640 1 -- 172.21.15.70:0/117931679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688005730 con 0x7f16880061e0 2023-08-25T09:50:33.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.735+0000 7f169ffff640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0070550 0x7f1688005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:33.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.736+0000 7f169f7fe640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16880069d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f1694001750 tx=0x7f1688007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360bcacfa835620 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:33.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.736+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16940052d0 con 0x7f16880061e0 2023-08-25T09:50:33.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.736+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169402e050 con 0x7f16880061e0 2023-08-25T09:50:33.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.736+0000 7f169effd640 1 -- 172.21.15.70:0/117931679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1694003830 con 0x7f16880061e0 2023-08-25T09:50:33.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a574f640 1 -- 172.21.15.70:0/117931679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 msgr2=0x7f16880069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a574f640 1 --2- 172.21.15.70:0/117931679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16880069d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f1694001750 tx=0x7f1688007a00 comp rx=0 tx=0).stop 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a4f4e640 1 -- 172.21.15.70:0/117931679 reap_dead start 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a574f640 1 -- 172.21.15.70:0/117931679 shutdown_connections 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a574f640 1 -- 172.21.15.70:0/117931679 >> 172.21.15.70:0/117931679 conn(0x7f16a0069f60 msgr2=0x7f16a00b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a574f640 1 -- 172.21.15.70:0/117931679 shutdown_connections 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.737+0000 7f16a574f640 1 -- 172.21.15.70:0/117931679 wait complete. 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.739+0000 7f16a574f640 1 Processor -- start 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.739+0000 7f16a574f640 1 -- start start 2023-08-25T09:50:33.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f16a574f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16a01474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f16a574f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a0155500 0x7f16a00cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f16a574f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0155980 0x7f16a00cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f16a574f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01479e0 con 0x7f16880061e0 2023-08-25T09:50:33.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f16a574f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a015b260 con 0x7f16a0155500 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f16a574f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a015b540 con 0x7f16a0155980 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16a01474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16a01474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38662/0 (socket says 172.21.15.70:38662) 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 -- 172.21.15.70:0/736290316 learned_addr learned my addr 172.21.15.70:0/736290316 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 -- 172.21.15.70:0/736290316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0155980 msgr2=0x7f16a00cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0155980 0x7f16a00cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 -- 172.21.15.70:0/736290316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a0155500 msgr2=0x7f16a00cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a0155500 0x7f16a00cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.740+0000 7f169ffff640 1 -- 172.21.15.70:0/736290316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a015b820 con 0x7f16880061e0 2023-08-25T09:50:33.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.741+0000 7f169ffff640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16a01474a0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f169000d8e0 tx=0x7f169000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.741+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900074e0 con 0x7f16880061e0 2023-08-25T09:50:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.741+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690007640 con 0x7f16880061e0 2023-08-25T09:50:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.741+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a015bb00 con 0x7f16880061e0 2023-08-25T09:50:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.741+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900049e0 con 0x7f16880061e0 2023-08-25T09:50:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.741+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a015c300 con 0x7f16880061e0 2023-08-25T09:50:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.742+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0066fb0 con 0x7f16880061e0 2023-08-25T09:50:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.744+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1690004b40 con 0x7f16880061e0 2023-08-25T09:50:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.745+0000 7f169cff9640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f168c06a080 0x7f168c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.745+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f1690088700 con 0x7f16880061e0 2023-08-25T09:50:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.745+0000 7f169f7fe640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f168c06a080 0x7f168c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:33.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.746+0000 7f169f7fe640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f168c06a080 0x7f168c06c540 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1694033000 tx=0x7f169402f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:33.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.750+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1690057a40 con 0x7f16880061e0 2023-08-25T09:50:33.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:33.940+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f16a006b5b0 con 0x7f16880061e0 2023-08-25T09:50:34.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.510+0000 7f169cff9640 1 -- 172.21.15.70:0/736290316 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v180) v1 ==== 170+0+0 (secure 0 0 0) 0x7f16900575d0 con 0x7f16880061e0 2023-08-25T09:50:34.518 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:50:34.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.527+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f168c06a080 msgr2=0x7f168c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:34.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.527+0000 7f16a574f640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f168c06a080 0x7f168c06c540 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f1694033000 tx=0x7f169402f000 comp rx=0 tx=0).stop 2023-08-25T09:50:34.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.527+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 msgr2=0x7f16a01474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:34.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.527+0000 7f16a574f640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16a01474a0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f169000d8e0 tx=0x7f169000dda0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.528+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 shutdown_connections 2023-08-25T09:50:34.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.528+0000 7f16a574f640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f168c06a080 0x7f168c06c540 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.528+0000 7f16a574f640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f16a0155980 0x7f16a00cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.528+0000 7f16a574f640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f16a0155500 0x7f16a00cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.528+0000 7f16a574f640 1 --2- 172.21.15.70:0/736290316 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f16880061e0 0x7f16a01474a0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.528+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 >> 172.21.15.70:0/736290316 conn(0x7f16a0069f60 msgr2=0x7f16a00cc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:34.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.529+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 shutdown_connections 2023-08-25T09:50:34.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.529+0000 7f16a574f640 1 -- 172.21.15.70:0/736290316 wait complete. 2023-08-25T09:50:34.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:50:34.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.745+0000 7f66e72cc640 1 Processor -- start 2023-08-25T09:50:34.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.745+0000 7f66e72cc640 1 -- start start 2023-08-25T09:50:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.746+0000 7f66e72cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e015e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.746+0000 7f66e72cc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f66e006f0e0 0x7f66e0070980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.746+0000 7f66e72cc640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e015e780 con 0x7f66e00cd660 2023-08-25T09:50:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.746+0000 7f66e72cc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e006f8e0 con 0x7f66e006f0e0 2023-08-25T09:50:34.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.746+0000 7f66e72cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e006fbc0 con 0x7f66e0159bf0 2023-08-25T09:50:34.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.747+0000 7f66e5068640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:34.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.747+0000 7f66e5068640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e015e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:59910/0 (socket says 172.21.15.70:59910) 2023-08-25T09:50:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.747+0000 7f66e5068640 1 -- 172.21.15.70:0/2782443863 learned_addr learned my addr 172.21.15.70:0/2782443863 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.747+0000 7f66e4867640 1 --2- 172.21.15.70:0/2782443863 >> v2:172.21.15.155:3300/0 conn(0x7f66e006f0e0 0x7f66e0070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.748+0000 7f66e5068640 1 -- 172.21.15.70:0/2782443863 >> v2:172.21.15.155:3300/0 conn(0x7f66e006f0e0 msgr2=0x7f66e0070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:34.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.748+0000 7f66e5068640 1 --2- 172.21.15.70:0/2782443863 >> v2:172.21.15.155:3300/0 conn(0x7f66e006f0e0 0x7f66e0070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.748+0000 7f66e5068640 1 -- 172.21.15.70:0/2782443863 >> v1:172.21.15.155:6789/0 conn(0x7f66e00cd660 legacy=0x7f66e00c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:34.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.748+0000 7f66e5068640 1 -- 172.21.15.70:0/2782443863 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e006fea0 con 0x7f66e0159bf0 2023-08-25T09:50:34.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.749+0000 7f66e5068640 1 --2- 172.21.15.70:0/2782443863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e015e240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009870 tx=0x7f66dc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=307080d79278129 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:34.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66d7fff640 1 -- 172.21.15.70:0/2782443863 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc030840 con 0x7f66e0159bf0 2023-08-25T09:50:34.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66d7fff640 1 -- 172.21.15.70:0/2782443863 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc0309a0 con 0x7f66e0159bf0 2023-08-25T09:50:34.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66d7fff640 1 -- 172.21.15.70:0/2782443863 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc030d30 con 0x7f66e0159bf0 2023-08-25T09:50:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 -- 172.21.15.70:0/2782443863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 msgr2=0x7f66e015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 --2- 172.21.15.70:0/2782443863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e015e240 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66dc009870 tx=0x7f66dc02b980 comp rx=0 tx=0).stop 2023-08-25T09:50:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 -- 172.21.15.70:0/2782443863 shutdown_connections 2023-08-25T09:50:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 --2- 172.21.15.70:0/2782443863 >> v2:172.21.15.155:3300/0 conn(0x7f66e006f0e0 0x7f66e0070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 --2- 172.21.15.70:0/2782443863 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e015e240 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 -- 172.21.15.70:0/2782443863 >> 172.21.15.70:0/2782443863 conn(0x7f66e00cb0d0 msgr2=0x7f66e00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:34.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 -- 172.21.15.70:0/2782443863 shutdown_connections 2023-08-25T09:50:34.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.750+0000 7f66e72cc640 1 -- 172.21.15.70:0/2782443863 wait complete. 2023-08-25T09:50:34.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.752+0000 7f66e72cc640 1 Processor -- start 2023-08-25T09:50:34.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.752+0000 7f66e72cc640 1 -- start start 2023-08-25T09:50:34.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.753+0000 7f66e72cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 0x7f66e00cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:34.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.753+0000 7f66e72cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66e00cd660 0x7f66e00cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:34.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.753+0000 7f66e72cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e0152dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:34.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.753+0000 7f66e72cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e01522b0 con 0x7f66e006f0e0 2023-08-25T09:50:34.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.753+0000 7f66e72cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0151980 con 0x7f66e00cd660 2023-08-25T09:50:34.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.753+0000 7f66e72cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0151c60 con 0x7f66e0159bf0 2023-08-25T09:50:34.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 0x7f66e00cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:34.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 0x7f66e00cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38664/0 (socket says 172.21.15.70:38664) 2023-08-25T09:50:34.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 -- 172.21.15.70:0/3225569276 learned_addr learned my addr 172.21.15.70:0/3225569276 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:34.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e4867640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66e00cd660 0x7f66e00cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:34.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5869640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e0152dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:34.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 -- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 msgr2=0x7f66e0152dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:34.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e0152dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 -- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66e00cd660 msgr2=0x7f66e00cca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:34.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66e00cd660 0x7f66e00cca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:34.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.754+0000 7f66e5068640 1 -- 172.21.15.70:0/3225569276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0094d0 con 0x7f66e006f0e0 2023-08-25T09:50:34.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.755+0000 7f66e5068640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 0x7f66e00cc550 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000c00 tx=0x7f66dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:34.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.755+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc039a90 con 0x7f66e006f0e0 2023-08-25T09:50:34.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.755+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc039bf0 con 0x7f66e006f0e0 2023-08-25T09:50:34.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.755+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc043410 con 0x7f66e006f0e0 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.755+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e0151f40 con 0x7f66e006f0e0 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.755+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e015ccc0 con 0x7f66e006f0e0 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.756+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e0066fb0 con 0x7f66e006f0e0 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.758+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f66dc02f390 con 0x7f66e006f0e0 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.759+0000 7f66d5ffb640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66c406a050 0x7f66c406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.759+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f66dc0aa7a0 con 0x7f66e006f0e0 2023-08-25T09:50:34.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.759+0000 7f66e4867640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66c406a050 0x7f66c406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:34.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.760+0000 7f66e4867640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66c406a050 0x7f66c406c510 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66e0158350 tx=0x7f66d0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:34.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.764+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f66dc076000 con 0x7f66e006f0e0 2023-08-25T09:50:34.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:34.992+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f66e00cdc40 con 0x7f66e006f0e0 2023-08-25T09:50:35.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.508+0000 7f66d5ffb640 1 -- 172.21.15.70:0/3225569276 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7f66dc079620 con 0x7f66e006f0e0 2023-08-25T09:50:35.516 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 21 and data pool 22 2023-08-25T09:50:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.523+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66c406a050 msgr2=0x7f66c406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.523+0000 7f66e72cc640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66c406a050 0x7f66c406c510 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f66e0158350 tx=0x7f66d0008040 comp rx=0 tx=0).stop 2023-08-25T09:50:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 msgr2=0x7f66e00cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 0x7f66e00cc550 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000c00 tx=0x7f66dc004730 comp rx=0 tx=0).stop 2023-08-25T09:50:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 shutdown_connections 2023-08-25T09:50:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66c406a050 0x7f66c406c510 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f66e0159bf0 0x7f66e0152dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f66e00cd660 0x7f66e00cca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 --2- 172.21.15.70:0/3225569276 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f66e006f0e0 0x7f66e00cc550 secure :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000c00 tx=0x7f66dc004730 comp rx=0 tx=0).stop 2023-08-25T09:50:35.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.524+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 >> 172.21.15.70:0/3225569276 conn(0x7f66e00cb0d0 msgr2=0x7f66e00c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:35.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.529+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 shutdown_connections 2023-08-25T09:50:35.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.529+0000 7f66e72cc640 1 -- 172.21.15.70:0/3225569276 wait complete. 2023-08-25T09:50:35.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:35.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.741+0000 7f5f12755640 1 Processor -- start 2023-08-25T09:50:35.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.741+0000 7f5f12755640 1 -- start start 2023-08-25T09:50:35.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.743+0000 7f5f12755640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5f0c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.743+0000 7f5f12755640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c15d3f0 con 0x7f5f0c0ce5b0 2023-08-25T09:50:35.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.743+0000 7f5f12755640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c0b2d20 con 0x7f5f0c0b2680 2023-08-25T09:50:35.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.743+0000 7f5f12755640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0b2b10 con 0x7f5f0c15aa10 2023-08-25T09:50:35.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.743+0000 7f5f0bfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5f0c0ce5b0 0x7f5f0c0ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42168/0 (socket says 172.21.15.70:42168) 2023-08-25T09:50:35.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.743+0000 7f5f0bfff640 1 -- 172.21.15.70:0/1558667020 learned_addr learned my addr 172.21.15.70:0/1558667020 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:35.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f037fe640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5f0c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:35.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453995775 0 0) 0x7f5f0c15d3f0 con 0x7f5f0c0ce5b0 2023-08-25T09:50:35.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef8003520 con 0x7f5f0c0ce5b0 2023-08-25T09:50:35.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2430711166 0 0) 0x7f5ef8003520 con 0x7f5f0c0ce5b0 2023-08-25T09:50:35.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 msgr2=0x7f5f0c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5f0c0b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 >> v1:172.21.15.155:6789/0 conn(0x7f5f0c0b2680 legacy=0x7f5f0c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c15d6b0 con 0x7f5f0c0ce5b0 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5efc0019f0 con 0x7f5f0c0ce5b0 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 >> v1:172.21.15.70:6789/0 conn(0x7f5f0c0ce5b0 legacy=0x7f5f0c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5ef8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5ef8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0b2b10 con 0x7f5ef80039a0 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef800dc70 con 0x7f5ef80047c0 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.744+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef800de30 con 0x7f5f0c15aa10 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5ef8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f037fe640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:35.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f0bfff640 1 -- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 msgr2=0x7f5ef8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:35.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f10cf2640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5ef8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:35.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5ef8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f0bfff640 1 -- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef80047c0 msgr2=0x7f5ef8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:35.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f0bfff640 1 -- 172.21.15.70:0/1558667020 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef8003520 con 0x7f5ef80039a0 2023-08-25T09:50:35.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.745+0000 7f5f10cf2640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c15aa10 0x7f5ef8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.746+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5ef8003dc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00a800 tx=0x7f5ef80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470c3291c7686208 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0072d0 con 0x7f5ef80039a0 2023-08-25T09:50:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc02f070 con 0x7f5ef80039a0 2023-08-25T09:50:35.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f0b7fe640 1 -- 172.21.15.70:0/1558667020 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc007670 con 0x7f5ef80039a0 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f12755640 1 -- 172.21.15.70:0/1558667020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 msgr2=0x7f5ef8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f12755640 1 --2- 172.21.15.70:0/1558667020 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5ef8003dc0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00a800 tx=0x7f5ef80057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f10cf2640 1 -- 172.21.15.70:0/1558667020 reap_dead start 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f12755640 1 -- 172.21.15.70:0/1558667020 shutdown_connections 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f12755640 1 -- 172.21.15.70:0/1558667020 >> 172.21.15.70:0/1558667020 conn(0x7f5f0c069850 msgr2=0x7f5f0c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f12755640 1 -- 172.21.15.70:0/1558667020 shutdown_connections 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.747+0000 7f5f12755640 1 -- 172.21.15.70:0/1558667020 wait complete. 2023-08-25T09:50:35.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.749+0000 7f5f12755640 1 Processor -- start 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.749+0000 7f5f12755640 1 -- start start 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.750+0000 7f5f12755640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5f0c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.750+0000 7f5f12755640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c146100 0x7f5f0c0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.750+0000 7f5f12755640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0c0acdb0 0x7f5f0c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.750+0000 7f5f12755640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0aeab0 con 0x7f5ef80039a0 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.750+0000 7f5f12755640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0ad230 con 0x7f5f0c0acdb0 2023-08-25T09:50:35.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.750+0000 7f5f12755640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0ade30 con 0x7f5f0c146100 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5f0c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5f0c0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38680/0 (socket says 172.21.15.70:38680) 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 -- 172.21.15.70:0/565642445 learned_addr learned my addr 172.21.15.70:0/565642445 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 -- 172.21.15.70:0/565642445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c146100 msgr2=0x7f5f0c0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c146100 0x7f5f0c0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 -- 172.21.15.70:0/565642445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0c0acdb0 msgr2=0x7f5f0c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0c0acdb0 0x7f5f0c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 -- 172.21.15.70:0/565642445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0ae110 con 0x7f5ef80039a0 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.751+0000 7f5f0bfff640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5f0c0aeed0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00a800 tx=0x7f5efc001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.752+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0074e0 con 0x7f5ef80039a0 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.752+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc007640 con 0x7f5ef80039a0 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.752+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0079d0 con 0x7f5ef80039a0 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.752+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c0ae3f0 con 0x7f5ef80039a0 2023-08-25T09:50:35.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.752+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c0af410 con 0x7f5ef80039a0 2023-08-25T09:50:35.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.753+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c0aed20 con 0x7f5ef80039a0 2023-08-25T09:50:35.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.754+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5efc031030 con 0x7f5ef80039a0 2023-08-25T09:50:35.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.755+0000 7f5f097fa640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5eec06a050 0x7f5eec06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:35.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.755+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5efc0a9af0 con 0x7f5ef80039a0 2023-08-25T09:50:35.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.755+0000 7f5f037fe640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5eec06a050 0x7f5eec06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:35.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.756+0000 7f5f037fe640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5eec06a050 0x7f5eec06c510 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c154890 tx=0x7f5ef4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:35.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.759+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5efc075140 con 0x7f5ef80039a0 2023-08-25T09:50:35.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.950+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f0c04e420 con 0x7f5ef80039a0 2023-08-25T09:50:35.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.954+0000 7f5f097fa640 1 -- 172.21.15.70:0/565642445 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f5efc078760 con 0x7f5ef80039a0 2023-08-25T09:50:35.956 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:35.957 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":181,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:34.993617+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:35.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5eec06a050 msgr2=0x7f5eec06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5eec06a050 0x7f5eec06c510 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c154890 tx=0x7f5ef4008040 comp rx=0 tx=0).stop 2023-08-25T09:50:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 msgr2=0x7f5f0c0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5f0c0aeed0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00a800 tx=0x7f5efc001d80 comp rx=0 tx=0).stop 2023-08-25T09:50:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 shutdown_connections 2023-08-25T09:50:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5eec06a050 0x7f5eec06c510 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f0c0acdb0 0x7f5f0c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f0c146100 0x7f5f0c0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 --2- 172.21.15.70:0/565642445 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5ef80039a0 0x7f5f0c0aeed0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:35.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.956+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 >> 172.21.15.70:0/565642445 conn(0x7f5f0c069850 msgr2=0x7f5f0c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:35.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.957+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 shutdown_connections 2023-08-25T09:50:35.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:35.957+0000 7f5f12755640 1 -- 172.21.15.70:0/565642445 wait complete. 2023-08-25T09:50:35.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:36.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.186+0000 7f1ebf283640 1 Processor -- start 2023-08-25T09:50:36.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.186+0000 7f1ebf283640 1 -- start start 2023-08-25T09:50:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.188+0000 7f1ebf283640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1eb815e830 0x7f1eb806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.188+0000 7f1ebf283640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.188+0000 7f1ebf283640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80b1b60 con 0x7f1eb815c6d0 2023-08-25T09:50:36.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.188+0000 7f1ebf283640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb806bd80 con 0x7f1eb815e830 2023-08-25T09:50:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.188+0000 7f1ebf283640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb806b7c0 con 0x7f1eb806b2b0 2023-08-25T09:50:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.188+0000 7f1ebd01f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1eb815e830 0x7f1eb806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1ebd01f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1eb815e830 0x7f1eb806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:54998/0 (socket says 172.21.15.70:54998) 2023-08-25T09:50:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1ebd01f640 1 -- 172.21.15.70:0/2687111157 learned_addr learned my addr 172.21.15.70:0/2687111157 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1ebc81e640 1 --2- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1ebd01f640 1 -- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 msgr2=0x7f1eb815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:36.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1ebd01f640 1 --2- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1ebd01f640 1 -- 172.21.15.70:0/2687111157 >> v1:172.21.15.155:6789/0 conn(0x7f1eb815c6d0 legacy=0x7f1eb80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.190+0000 7f1ebd01f640 1 -- 172.21.15.70:0/2687111157 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb806baa0 con 0x7f1eb815e830 2023-08-25T09:50:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.189+0000 7f1eaffff640 1 -- 172.21.15.70:0/2687111157 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018927176 0 0) 0x7f1eb80b1b60 con 0x7f1eb815c6d0 2023-08-25T09:50:36.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.191+0000 7f1ebd01f640 1 --2- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 0x7f1eb806ad70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c7ce93a2776c679 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1eaffff640 1 -- 172.21.15.70:0/2687111157 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4030840 con 0x7f1eb815e830 2023-08-25T09:50:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1eaffff640 1 -- 172.21.15.70:0/2687111157 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40309a0 con 0x7f1eb815e830 2023-08-25T09:50:36.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1eaffff640 1 -- 172.21.15.70:0/2687111157 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4030d30 con 0x7f1eb815e830 2023-08-25T09:50:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1ebf283640 1 -- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 msgr2=0x7f1eb806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 0x7f1eb806ad70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402b980 comp rx=0 tx=0).stop 2023-08-25T09:50:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1ebf283640 1 -- 172.21.15.70:0/2687111157 shutdown_connections 2023-08-25T09:50:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb815c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2687111157 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 0x7f1eb806ad70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.192+0000 7f1ebf283640 1 -- 172.21.15.70:0/2687111157 >> 172.21.15.70:0/2687111157 conn(0x7f1eb8069850 msgr2=0x7f1eb8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.193+0000 7f1ebf283640 1 -- 172.21.15.70:0/2687111157 shutdown_connections 2023-08-25T09:50:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.193+0000 7f1ebf283640 1 -- 172.21.15.70:0/2687111157 wait complete. 2023-08-25T09:50:36.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.194+0000 7f1ebf283640 1 Processor -- start 2023-08-25T09:50:36.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.194+0000 7f1ebf283640 1 -- start start 2023-08-25T09:50:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.195+0000 7f1ebf283640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb8153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.195+0000 7f1ebf283640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 0x7f1eb81f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.195+0000 7f1ebf283640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 0x7f1eb81f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.195+0000 7f1ebf283640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c8a20 con 0x7f1eb815c6d0 2023-08-25T09:50:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.195+0000 7f1ebf283640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c8d00 con 0x7f1eb815e830 2023-08-25T09:50:36.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.195+0000 7f1ebf283640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb80c8fe0 con 0x7f1eb806b2b0 2023-08-25T09:50:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 0x7f1eb81f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 0x7f1eb81f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38694/0 (socket says 172.21.15.70:38694) 2023-08-25T09:50:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 -- 172.21.15.70:0/2630181468 learned_addr learned my addr 172.21.15.70:0/2630181468 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 -- 172.21.15.70:0/2630181468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 msgr2=0x7f1eb8153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb8153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 -- 172.21.15.70:0/2630181468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 msgr2=0x7f1eb81f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 0x7f1eb81f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 -- 172.21.15.70:0/2630181468 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40094d0 con 0x7f1eb815c6d0 2023-08-25T09:50:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.196+0000 7f1ebc81e640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 0x7f1eb81f2160 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800e980 tx=0x7f1ea800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.197+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8009670 con 0x7f1eb815c6d0 2023-08-25T09:50:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.197+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea800ca00 con 0x7f1eb815c6d0 2023-08-25T09:50:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.197+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800cd90 con 0x7f1eb815c6d0 2023-08-25T09:50:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.197+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb80c92c0 con 0x7f1eb815c6d0 2023-08-25T09:50:36.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.197+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb80c9ac0 con 0x7f1eb815c6d0 2023-08-25T09:50:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.198+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb804e420 con 0x7f1eb815c6d0 2023-08-25T09:50:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.199+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1ea8013470 con 0x7f1eb815c6d0 2023-08-25T09:50:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.200+0000 7f1eadffb640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.200+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ea8088840 con 0x7f1eb815c6d0 2023-08-25T09:50:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.200+0000 7f1ebd01f640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.201+0000 7f1ebd01f640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4036000 tx=0x7f1eb403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:36.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.204+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1ea8053f20 con 0x7f1eb815c6d0 2023-08-25T09:50:36.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.389+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eb806c620 con 0x7f1eb815c6d0 2023-08-25T09:50:36.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.392+0000 7f1eadffb640 1 -- 172.21.15.70:0/2630181468 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+18394 (secure 0 0 0) 0x7f1ea8057540 con 0x7f1eb815c6d0 2023-08-25T09:50:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:36.394 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":181,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:34.993617+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 msgr2=0x7f1e9c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4036000 tx=0x7f1eb403b040 comp rx=0 tx=0).stop 2023-08-25T09:50:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 msgr2=0x7f1eb81f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 0x7f1eb81f2160 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800e980 tx=0x7f1ea800ee40 comp rx=0 tx=0).stop 2023-08-25T09:50:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 shutdown_connections 2023-08-25T09:50:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e9c06a050 0x7f1e9c06c510 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1eb815e830 0x7f1eb81f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1eb815c6d0 0x7f1eb81f2160 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 --2- 172.21.15.70:0/2630181468 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1eb806b2b0 0x7f1eb8153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 >> 172.21.15.70:0/2630181468 conn(0x7f1eb8069850 msgr2=0x7f1eb80b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 shutdown_connections 2023-08-25T09:50:36.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.395+0000 7f1ebf283640 1 -- 172.21.15.70:0/2630181468 wait complete. 2023-08-25T09:50:36.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:50:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.620+0000 7fc379438640 1 Processor -- start 2023-08-25T09:50:36.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.621+0000 7fc379438640 1 -- start start 2023-08-25T09:50:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc379438640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 0x7fc3740c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc379438640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc3740cdc20 0x7fc374069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc379438640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3740cc5f0 con 0x7fc3740ce060 2023-08-25T09:50:36.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc379438640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc374070650 con 0x7fc3740cdc20 2023-08-25T09:50:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc379438640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc374070930 con 0x7fc37415ae50 2023-08-25T09:50:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc3737fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc3740ce060 0x7fc3740ccad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42184/0 (socket says 172.21.15.70:42184) 2023-08-25T09:50:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc3737fe640 1 -- 172.21.15.70:0/2089794147 learned_addr learned my addr 172.21.15.70:0/2089794147 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3229744195 0 0) 0x7fc3740cc5f0 con 0x7fc3740ce060 2023-08-25T09:50:36.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc360003520 con 0x7fc3740ce060 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc3727fc640 1 --2- 172.21.15.70:0/2089794147 >> v2:172.21.15.155:3300/0 conn(0x7fc3740cdc20 0x7fc374069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.622+0000 7fc372ffd640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 0x7fc3740c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3348642546 0 0) 0x7fc360003520 con 0x7fc3740ce060 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 msgr2=0x7fc3740c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 0x7fc3740c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 >> v2:172.21.15.155:3300/0 conn(0x7fc3740cdc20 msgr2=0x7fc374069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:36.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 --2- 172.21.15.70:0/2089794147 >> v2:172.21.15.155:3300/0 conn(0x7fc3740cdc20 0x7fc374069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc374070c10 con 0x7fc3740ce060 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc36c001670 con 0x7fc3740ce060 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 >> v1:172.21.15.70:6789/0 conn(0x7fc3740ce060 legacy=0x7fc3740ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc360003970 0x7fc360003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 0x7fc3600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360004d00 0x7fc360005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374070650 con 0x7fc360003970 2023-08-25T09:50:36.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc374070930 con 0x7fc360004d00 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36000bab0 con 0x7fc37415ae50 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc372ffd640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc360003970 0x7fc360003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc372ffd640 1 -- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 msgr2=0x7fc3600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc372ffd640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc37415ae50 0x7fc3600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc372ffd640 1 -- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360004d00 msgr2=0x7fc360005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc372ffd640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360004d00 0x7fc360005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.623+0000 7fc372ffd640 1 -- 172.21.15.70:0/2089794147 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360003520 con 0x7fc360003970 2023-08-25T09:50:36.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.624+0000 7fc372ffd640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc360003970 0x7fc360003d90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fc364007360 tx=0x7fc3600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea781f0f3ee0c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36402ccf0 con 0x7fc360003970 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36402ce50 con 0x7fc360003970 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc371ffb640 1 -- 172.21.15.70:0/2089794147 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364034660 con 0x7fc360003970 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc379438640 1 -- 172.21.15.70:0/2089794147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc360003970 msgr2=0x7fc360003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc379438640 1 --2- 172.21.15.70:0/2089794147 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc360003970 0x7fc360003d90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fc364007360 tx=0x7fc3600057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc3737fe640 1 -- 172.21.15.70:0/2089794147 reap_dead start 2023-08-25T09:50:36.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc379438640 1 -- 172.21.15.70:0/2089794147 shutdown_connections 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc379438640 1 -- 172.21.15.70:0/2089794147 >> 172.21.15.70:0/2089794147 conn(0x7fc3740cb0d0 msgr2=0x7fc3740cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc379438640 1 -- 172.21.15.70:0/2089794147 shutdown_connections 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.625+0000 7fc379438640 1 -- 172.21.15.70:0/2089794147 wait complete. 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.627+0000 7fc379438640 1 Processor -- start 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.627+0000 7fc379438640 1 -- start start 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc379438640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360003970 0x7fc3741f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc379438640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 0x7fc3741f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc379438640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc374152b70 0x7fc374153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc379438640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37415e320 con 0x7fc3740c4090 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc379438640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c32f0 con 0x7fc360003970 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc379438640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740c35d0 con 0x7fc374152b70 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.628+0000 7fc3727fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 0x7fc3741f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 0x7fc3741f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38720/0 (socket says 172.21.15.70:38720) 2023-08-25T09:50:36.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 -- 172.21.15.70:0/1070335200 learned_addr learned my addr 172.21.15.70:0/1070335200 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 -- 172.21.15.70:0/1070335200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc374152b70 msgr2=0x7fc374153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc374152b70 0x7fc374153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 -- 172.21.15.70:0/1070335200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360003970 msgr2=0x7fc3741f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360003970 0x7fc3741f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 -- 172.21.15.70:0/1070335200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3740c38b0 con 0x7fc3740c4090 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc3727fc640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 0x7fc3741f3990 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00d800 tx=0x7fc35c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00b6c0 con 0x7fc3740c4090 2023-08-25T09:50:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc35c00b820 con 0x7fc3740c4090 2023-08-25T09:50:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc35c00bbd0 con 0x7fc3740c4090 2023-08-25T09:50:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.629+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3740c3b90 con 0x7fc3740c4090 2023-08-25T09:50:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.630+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3741f3ed0 con 0x7fc3740c4090 2023-08-25T09:50:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.631+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc374066fb0 con 0x7fc3740c4090 2023-08-25T09:50:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.632+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc35c00bd30 con 0x7fc3740c4090 2023-08-25T09:50:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.632+0000 7fc36b7fe640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc35406a050 0x7fc35406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.632+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc35c088a10 con 0x7fc3740c4090 2023-08-25T09:50:36.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.633+0000 7fc372ffd640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc35406a050 0x7fc35406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:36.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.633+0000 7fc372ffd640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc35406a050 0x7fc35406c510 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc364036000 tx=0x7fc36402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:36.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.636+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc35c054100 con 0x7fc3740c4090 2023-08-25T09:50:36.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:36.848+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc3740cd030 con 0x7fc3740c4090 2023-08-25T09:50:37.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.551+0000 7fc36b7fe640 1 -- 172.21.15.70:0/1070335200 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc35c057720 con 0x7fc3740c4090 2023-08-25T09:50:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.554+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc35406a050 msgr2=0x7fc35406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.554+0000 7fc379438640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc35406a050 0x7fc35406c510 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc364036000 tx=0x7fc36402e000 comp rx=0 tx=0).stop 2023-08-25T09:50:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.554+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 msgr2=0x7fc3741f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.554+0000 7fc379438640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 0x7fc3741f3990 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00d800 tx=0x7fc35c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 shutdown_connections 2023-08-25T09:50:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc35406a050 0x7fc35406c510 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc374152b70 0x7fc374153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3740c4090 0x7fc3741f3990 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 --2- 172.21.15.70:0/1070335200 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc360003970 0x7fc3741f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 >> 172.21.15.70:0/1070335200 conn(0x7fc3740cb0d0 msgr2=0x7fc3740ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 shutdown_connections 2023-08-25T09:50:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.555+0000 7fc379438640 1 -- 172.21.15.70:0/1070335200 wait complete. 2023-08-25T09:50:37.577 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:50:37.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:50:37.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.794+0000 7ffa23eec640 1 Processor -- start 2023-08-25T09:50:37.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.794+0000 7ffa23eec640 1 -- start start 2023-08-25T09:50:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa23eec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa1c159bf0 0x7ffa1c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa23eec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c15e780 con 0x7ffa1c0cd660 2023-08-25T09:50:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa23eec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c06f8e0 con 0x7ffa1c06f0e0 2023-08-25T09:50:37.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa23eec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1c06fbc0 con 0x7ffa1c159bf0 2023-08-25T09:50:37.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa21c88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa1c159bf0 0x7ffa1c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:37.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa21c88640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ffa1c159bf0 0x7ffa1c15e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38724/0 (socket says 172.21.15.70:38724) 2023-08-25T09:50:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.796+0000 7ffa21c88640 1 -- 172.21.15.70:0/695918877 learned_addr learned my addr 172.21.15.70:0/695918877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.797+0000 7ffa21c88640 1 -- 172.21.15.70:0/695918877 >> v1:172.21.15.155:6789/0 conn(0x7ffa1c06f0e0 legacy=0x7ffa1c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.797+0000 7ffa21c88640 1 -- 172.21.15.70:0/695918877 >> v1:172.21.15.70:6789/0 conn(0x7ffa1c0cd660 legacy=0x7ffa1c0c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.797+0000 7ffa21c88640 1 -- 172.21.15.70:0/695918877 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c06fea0 con 0x7ffa1c159bf0 2023-08-25T09:50:37.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.797+0000 7ffa20c86640 1 -- 172.21.15.70:0/695918877 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003679095 0 0) 0x7ffa1c15e780 con 0x7ffa1c0cd660 2023-08-25T09:50:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.798+0000 7ffa21c88640 1 --2- 172.21.15.70:0/695918877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c15e240 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800bb10 tx=0x7ffa1802e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7c26b76db804c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.798+0000 7ffa20c86640 1 -- 172.21.15.70:0/695918877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18036440 con 0x7ffa1c159bf0 2023-08-25T09:50:37.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.798+0000 7ffa20c86640 1 -- 172.21.15.70:0/695918877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa180365a0 con 0x7ffa1c159bf0 2023-08-25T09:50:37.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.798+0000 7ffa20c86640 1 -- 172.21.15.70:0/695918877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18036930 con 0x7ffa1c159bf0 2023-08-25T09:50:37.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 -- 172.21.15.70:0/695918877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 msgr2=0x7ffa1c15e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:37.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 --2- 172.21.15.70:0/695918877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c15e240 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800bb10 tx=0x7ffa1802e170 comp rx=0 tx=0).stop 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 -- 172.21.15.70:0/695918877 shutdown_connections 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 --2- 172.21.15.70:0/695918877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c15e240 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 -- 172.21.15.70:0/695918877 >> 172.21.15.70:0/695918877 conn(0x7ffa1c0cb0d0 msgr2=0x7ffa1c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 -- 172.21.15.70:0/695918877 shutdown_connections 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.799+0000 7ffa23eec640 1 -- 172.21.15.70:0/695918877 wait complete. 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.801+0000 7ffa23eec640 1 Processor -- start 2023-08-25T09:50:37.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.801+0000 7ffa23eec640 1 -- start start 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa23eec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa1c06f0e0 0x7ffa1c0acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa23eec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa1c0cd660 0x7ffa1c14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa23eec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c14de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa23eec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c1f3680 con 0x7ffa1c159bf0 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa23eec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c14e340 con 0x7ffa1c06f0e0 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa23eec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c14e860 con 0x7ffa1c0cd660 2023-08-25T09:50:37.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa22489640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c14de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa22489640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c14de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38734/0 (socket says 172.21.15.70:38734) 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa22489640 1 -- 172.21.15.70:0/1507167546 learned_addr learned my addr 172.21.15.70:0/1507167546 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.802+0000 7ffa21c88640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa1c06f0e0 0x7ffa1c0acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa21487640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa1c0cd660 0x7ffa1c14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa22489640 1 -- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa1c0cd660 msgr2=0x7ffa1c14d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa22489640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa1c0cd660 0x7ffa1c14d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa22489640 1 -- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa1c06f0e0 msgr2=0x7ffa1c0acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa22489640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa1c06f0e0 0x7ffa1c0acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa22489640 1 -- 172.21.15.70:0/1507167546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1800b770 con 0x7ffa1c159bf0 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa22489640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c14de00 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400c760 tx=0x7ffa1400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14003da0 con 0x7ffa1c159bf0 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1400ce70 con 0x7ffa1c159bf0 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa14005290 con 0x7ffa1c159bf0 2023-08-25T09:50:37.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c1f2d40 con 0x7ffa1c159bf0 2023-08-25T09:50:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.803+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c1f2990 con 0x7ffa1c159bf0 2023-08-25T09:50:37.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.805+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c1f37c0 con 0x7ffa1c159bf0 2023-08-25T09:50:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.805+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffa140053f0 con 0x7ffa1c159bf0 2023-08-25T09:50:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.806+0000 7ffa127fc640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffa0006a050 0x7ffa0006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.806+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffa14088400 con 0x7ffa1c159bf0 2023-08-25T09:50:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.806+0000 7ffa21c88640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffa0006a050 0x7ffa0006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:37.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.807+0000 7ffa21c88640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffa0006a050 0x7ffa0006c510 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffa18038000 tx=0x7ffa18032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:37.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:37.811+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffa1c1f37c0 con 0x7ffa1c159bf0 2023-08-25T09:50:38.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.035+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffa1c1f37c0 con 0x7ffa1c159bf0 2023-08-25T09:50:38.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.565+0000 7ffa127fc640 1 -- 172.21.15.70:0/1507167546 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffa1c1f37c0 con 0x7ffa1c159bf0 2023-08-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffa0006a050 msgr2=0x7ffa0006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffa0006a050 0x7ffa0006c510 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffa18038000 tx=0x7ffa18032000 comp rx=0 tx=0).stop 2023-08-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 msgr2=0x7ffa1c14de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c14de00 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400c760 tx=0x7ffa1400cc20 comp rx=0 tx=0).stop 2023-08-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 shutdown_connections 2023-08-25T09:50:38.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ffa0006a050 0x7ffa0006c510 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ffa1c159bf0 0x7ffa1c14de00 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ffa1c0cd660 0x7ffa1c14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 --2- 172.21.15.70:0/1507167546 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ffa1c06f0e0 0x7ffa1c0acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.569+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 >> 172.21.15.70:0/1507167546 conn(0x7ffa1c0cb0d0 msgr2=0x7ffa1c0cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:38.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.570+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 shutdown_connections 2023-08-25T09:50:38.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.570+0000 7ffa23eec640 1 -- 172.21.15.70:0/1507167546 wait complete. 2023-08-25T09:50:38.593 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:50:38.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:38.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.810+0000 7f3da19ab640 1 Processor -- start 2023-08-25T09:50:38.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.810+0000 7f3da19ab640 1 -- start start 2023-08-25T09:50:38.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.812+0000 7f3da19ab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3d9c0b1ff0 0x7f3d9c0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.812+0000 7f3da19ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9c0b1b60 con 0x7f3d9c15e830 2023-08-25T09:50:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.812+0000 7f3da19ab640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9c06b7c0 con 0x7f3d9c15ec10 2023-08-25T09:50:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.812+0000 7f3da19ab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c06bd80 con 0x7f3d9c0b1ff0 2023-08-25T09:50:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.812+0000 7f3d9affd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3d9c15e830 0x7f3d9c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42190/0 (socket says 172.21.15.70:42190) 2023-08-25T09:50:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.812+0000 7f3d9affd640 1 -- 172.21.15.70:0/2550246148 learned_addr learned my addr 172.21.15.70:0/2550246148 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d9b7fe640 1 --2- 172.21.15.70:0/2550246148 >> v2:172.21.15.155:3300/0 conn(0x7f3d9c0b1ff0 0x7f3d9c0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421030916 0 0) 0x7f3d9c0b1b60 con 0x7f3d9c15e830 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d84003520 con 0x7f3d9c15e830 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980559466 0 0) 0x7f3d84003520 con 0x7f3d9c15e830 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 >> v2:172.21.15.155:3300/0 conn(0x7f3d9c0b1ff0 msgr2=0x7f3d9c0b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 --2- 172.21.15.70:0/2550246148 >> v2:172.21.15.155:3300/0 conn(0x7f3d9c0b1ff0 0x7f3d9c0b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 >> v1:172.21.15.155:6789/0 conn(0x7f3d9c15ec10 legacy=0x7f3d9c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c06baa0 con 0x7f3d9c15e830 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3d8c001670 con 0x7f3d9c15e830 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 >> v1:172.21.15.70:6789/0 conn(0x7f3d9c15e830 legacy=0x7f3d9c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d84003970 0x7f3d84003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d840047c0 0x7f3d84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d84005180 0x7f3d8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c06bd80 con 0x7f3d840047c0 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8400c0d0 con 0x7f3d84005180 2023-08-25T09:50:38.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.813+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8400c290 con 0x7f3d84003970 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9a7fc640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d840047c0 0x7f3d84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9a7fc640 1 -- 172.21.15.70:0/2550246148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d84003970 msgr2=0x7f3d84003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9a7fc640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d84003970 0x7f3d84003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9a7fc640 1 -- 172.21.15.70:0/2550246148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d84005180 msgr2=0x7f3d8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9a7fc640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d84005180 0x7f3d8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9b7fe640 1 -- 172.21.15.70:0/2550246148 reap_dead start 2023-08-25T09:50:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.814+0000 7f3d9a7fc640 1 -- 172.21.15.70:0/2550246148 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84003520 con 0x7f3d840047c0 2023-08-25T09:50:38.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.815+0000 7f3d9a7fc640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d840047c0 0x7f3d84004c40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3d90001720 tx=0x7f3d840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6486ae89fa40c140 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:38.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.815+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d900052d0 con 0x7f3d840047c0 2023-08-25T09:50:38.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.815+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9002e050 con 0x7f3d840047c0 2023-08-25T09:50:38.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.815+0000 7f3d99ffb640 1 -- 172.21.15.70:0/2550246148 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90003830 con 0x7f3d840047c0 2023-08-25T09:50:38.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 -- 172.21.15.70:0/2550246148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d840047c0 msgr2=0x7f3d84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d840047c0 0x7f3d84004c40 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f3d90001720 tx=0x7f3d840057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 -- 172.21.15.70:0/2550246148 shutdown_connections 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 --2- 172.21.15.70:0/2550246148 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d840047c0 0x7f3d84004c40 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 -- 172.21.15.70:0/2550246148 >> 172.21.15.70:0/2550246148 conn(0x7f3d9c069850 msgr2=0x7f3d9c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 -- 172.21.15.70:0/2550246148 shutdown_connections 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.816+0000 7f3da19ab640 1 -- 172.21.15.70:0/2550246148 wait complete. 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.818+0000 7f3da19ab640 1 Processor -- start 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.818+0000 7f3da19ab640 1 -- start start 2023-08-25T09:50:38.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3da19ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d840047c0 0x7f3d9c1433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3da19ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0c8b50 0x7f3d9c0c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3da19ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 0x7f3d9c06f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3da19ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c06f6a0 con 0x7f3d9c06ece0 2023-08-25T09:50:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3da19ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0c9b90 con 0x7f3d9c0c8b50 2023-08-25T09:50:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3da19ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c1440e0 con 0x7f3d840047c0 2023-08-25T09:50:38.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3d9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 0x7f3d9c06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3d9b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 0x7f3d9c06f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:38746/0 (socket says 172.21.15.70:38746) 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.819+0000 7f3d9b7fe640 1 -- 172.21.15.70:0/1771613348 learned_addr learned my addr 172.21.15.70:0/1771613348 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9b7fe640 1 -- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d840047c0 msgr2=0x7f3d9c1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9b7fe640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d840047c0 0x7f3d9c1433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9a7fc640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0c8b50 0x7f3d9c0c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9b7fe640 1 -- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0c8b50 msgr2=0x7f3d9c0c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:38.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9b7fe640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0c8b50 0x7f3d9c0c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9b7fe640 1 -- 172.21.15.70:0/1771613348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c143930 con 0x7f3d9c06ece0 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d9b7fe640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 0x7f3d9c06f160 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0cf160 tx=0x7f3d9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400edc0 con 0x7f3d9c06ece0 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d94014420 con 0x7f3d9c06ece0 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940147d0 con 0x7f3d9c06ece0 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c0c95d0 con 0x7f3d9c06ece0 2023-08-25T09:50:38.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.820+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c143d70 con 0x7f3d9c06ece0 2023-08-25T09:50:38.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.822+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c04e420 con 0x7f3d9c06ece0 2023-08-25T09:50:38.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.822+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3d94014930 con 0x7f3d9c06ece0 2023-08-25T09:50:38.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.823+0000 7f3d83fff640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3d8806a080 0x7f3d8806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:38.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.824+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d94088c90 con 0x7f3d9c06ece0 2023-08-25T09:50:38.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.824+0000 7f3d9affd640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3d8806a080 0x7f3d8806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:38.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.824+0000 7f3d9affd640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3d8806a080 0x7f3d8806c540 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c000ff0 tx=0x7f3d8c0014b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:38.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:38.827+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3d940543b0 con 0x7f3d9c06ece0 2023-08-25T09:50:39.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.054+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d9c144220 con 0x7f3d9c06ece0 2023-08-25T09:50:39.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.055+0000 7f3d83fff640 1 -- 172.21.15.70:0/1771613348 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f3d940579d0 con 0x7f3d9c06ece0 2023-08-25T09:50:39.057 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:39.057 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:50:39.058 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:50:39.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3d8806a080 msgr2=0x7f3d8806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3d8806a080 0x7f3d8806c540 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c000ff0 tx=0x7f3d8c0014b0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 msgr2=0x7f3d9c06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 0x7f3d9c06f160 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0cf160 tx=0x7f3d9400e4b0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 shutdown_connections 2023-08-25T09:50:39.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3d8806a080 0x7f3d8806c540 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3d9c06ece0 0x7f3d9c06f160 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3d9c0c8b50 0x7f3d9c0c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 --2- 172.21.15.70:0/1771613348 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3d840047c0 0x7f3d9c1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 >> 172.21.15.70:0/1771613348 conn(0x7f3d9c069850 msgr2=0x7f3d9c0ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:39.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 shutdown_connections 2023-08-25T09:50:39.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.058+0000 7f3da19ab640 1 -- 172.21.15.70:0/1771613348 wait complete. 2023-08-25T09:50:39.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:50:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.291+0000 7f174a4d3640 1 Processor -- start 2023-08-25T09:50:39.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.291+0000 7f174a4d3640 1 -- start start 2023-08-25T09:50:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.293+0000 7f174a4d3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f174415e830 0x7f174406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.293+0000 7f174a4d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f17440b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.293+0000 7f174a4d3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17440b1b60 con 0x7f174406b2b0 2023-08-25T09:50:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.293+0000 7f174a4d3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f174406bd80 con 0x7f174415e830 2023-08-25T09:50:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.293+0000 7f174a4d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174406b7c0 con 0x7f174415ec10 2023-08-25T09:50:39.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1743fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f174415e830 0x7f174406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1748a70640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f17440b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1743fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f174415e830 0x7f174406ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:55608/0 (socket says 172.21.15.70:55608) 2023-08-25T09:50:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1743fff640 1 -- 172.21.15.70:0/2727123982 learned_addr learned my addr 172.21.15.70:0/2727123982 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:39.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1748a70640 1 -- 172.21.15.70:0/2727123982 >> v2:172.21.15.155:3300/0 conn(0x7f174415e830 msgr2=0x7f174406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1748a70640 1 --2- 172.21.15.70:0/2727123982 >> v2:172.21.15.155:3300/0 conn(0x7f174415e830 0x7f174406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1748a70640 1 -- 172.21.15.70:0/2727123982 >> v1:172.21.15.155:6789/0 conn(0x7f174406b2b0 legacy=0x7f174415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.294+0000 7f1748a70640 1 -- 172.21.15.70:0/2727123982 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174406baa0 con 0x7f174415ec10 2023-08-25T09:50:39.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.296+0000 7f1748a70640 1 --2- 172.21.15.70:0/2727123982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f17440b1ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f173c009d90 tx=0x7f173c02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59dfb4abf41fa081 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.297+0000 7f17437fe640 1 -- 172.21.15.70:0/2727123982 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c02cc60 con 0x7f174415ec10 2023-08-25T09:50:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.297+0000 7f17437fe640 1 -- 172.21.15.70:0/2727123982 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c02cdc0 con 0x7f174415ec10 2023-08-25T09:50:39.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.297+0000 7f17437fe640 1 -- 172.21.15.70:0/2727123982 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c034660 con 0x7f174415ec10 2023-08-25T09:50:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.297+0000 7f174a4d3640 1 -- 172.21.15.70:0/2727123982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 msgr2=0x7f17440b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.297+0000 7f174a4d3640 1 --2- 172.21.15.70:0/2727123982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f17440b1ff0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f173c009d90 tx=0x7f173c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.298+0000 7f174a4d3640 1 -- 172.21.15.70:0/2727123982 shutdown_connections 2023-08-25T09:50:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.298+0000 7f174a4d3640 1 --2- 172.21.15.70:0/2727123982 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f17440b1ff0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.298+0000 7f174a4d3640 1 --2- 172.21.15.70:0/2727123982 >> v2:172.21.15.155:3300/0 conn(0x7f174415e830 0x7f174406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.298+0000 7f174a4d3640 1 -- 172.21.15.70:0/2727123982 >> 172.21.15.70:0/2727123982 conn(0x7f1744069850 msgr2=0x7f1744069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.298+0000 7f174a4d3640 1 -- 172.21.15.70:0/2727123982 shutdown_connections 2023-08-25T09:50:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.298+0000 7f174a4d3640 1 -- 172.21.15.70:0/2727123982 wait complete. 2023-08-25T09:50:39.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.299+0000 7f174a4d3640 1 Processor -- start 2023-08-25T09:50:39.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.299+0000 7f174a4d3640 1 -- start start 2023-08-25T09:50:39.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.300+0000 7f174a4d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174406b2b0 0x7f1744158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.300+0000 7f174a4d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 0x7f174415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.300+0000 7f174a4d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f174415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.300+0000 7f174a4d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441f32e0 con 0x7f174415e830 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.300+0000 7f174a4d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744070ca0 con 0x7f174406b2b0 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.300+0000 7f174a4d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17440ca780 con 0x7f174415ec10 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 0x7f174415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 0x7f174415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51126/0 (socket says 172.21.15.70:51126) 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 -- 172.21.15.70:0/1612399793 learned_addr learned my addr 172.21.15.70:0/1612399793 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 -- 172.21.15.70:0/1612399793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 msgr2=0x7f174415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f174415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 -- 172.21.15.70:0/1612399793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174406b2b0 msgr2=0x7f1744158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174406b2b0 0x7f1744158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.301+0000 7f173b7fe640 1 -- 172.21.15.70:0/1612399793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173c009a70 con 0x7f174415e830 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.302+0000 7f173b7fe640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 0x7f174415b4d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c450 tx=0x7f172c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.302+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0052d0 con 0x7f174415e830 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.302+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c005430 con 0x7f174415e830 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.302+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c012870 con 0x7f174415e830 2023-08-25T09:50:39.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.302+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174415ca20 con 0x7f174415e830 2023-08-25T09:50:39.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.302+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17441f2800 con 0x7f174415e830 2023-08-25T09:50:39.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.303+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174404e420 con 0x7f174415e830 2023-08-25T09:50:39.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.304+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f172c0129d0 con 0x7f174415e830 2023-08-25T09:50:39.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.305+0000 7f17417fa640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f172406a050 0x7f172406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.306+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f172c088010 con 0x7f174415e830 2023-08-25T09:50:39.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.306+0000 7f1743fff640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f172406a050 0x7f172406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.306+0000 7f1743fff640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f172406a050 0x7f172406c510 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f17340094a0 tx=0x7f17340093f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:39.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.310+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f172c053790 con 0x7f174415e830 2023-08-25T09:50:39.486 INFO:tasks.ceph.mds.b.smithi155.stderr:2023-08-25T09:50:39.484+0000 7f474e99e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:50:39.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.500+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f174406c620 con 0x7f174415e830 2023-08-25T09:50:39.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.503+0000 7f17417fa640 1 -- 172.21.15.70:0/1612399793 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v182) v1 ==== 74+0+18381 (secure 0 0 0) 0x7f172c056db0 con 0x7f174415e830 2023-08-25T09:50:39.505 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:39.505 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":182,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:50:36.501261+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:50:39.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.506+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f172406a050 msgr2=0x7f172406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.506+0000 7f174a4d3640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f172406a050 0x7f172406c510 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f17340094a0 tx=0x7f17340093f0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.506+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 msgr2=0x7f174415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.506+0000 7f174a4d3640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 0x7f174415b4d0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c450 tx=0x7f172c00c910 comp rx=0 tx=0).stop 2023-08-25T09:50:39.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 shutdown_connections 2023-08-25T09:50:39.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f172406a050 0x7f172406c510 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f174415ec10 0x7f174415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f174415e830 0x7f174415b4d0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.509 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 --2- 172.21.15.70:0/1612399793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f174406b2b0 0x7f1744158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 >> 172.21.15.70:0/1612399793 conn(0x7f1744069850 msgr2=0x7f174406f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:39.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 shutdown_connections 2023-08-25T09:50:39.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.507+0000 7f174a4d3640 1 -- 172.21.15.70:0/1612399793 wait complete. 2023-08-25T09:50:39.526 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:50:39.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:50:39.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.743+0000 7f1b23112640 1 Processor -- start 2023-08-25T09:50:39.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.743+0000 7f1b23112640 1 -- start start 2023-08-25T09:50:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b23112640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1b1c15aa10 0x7f1b1c0b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b23112640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c0b2ad0 con 0x7f1b1c0b2680 2023-08-25T09:50:39.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b23112640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1c15d3f0 con 0x7f1b1c0ce5b0 2023-08-25T09:50:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b23112640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1c159d10 con 0x7f1b1c15aa10 2023-08-25T09:50:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b216af640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1b1c0b2680 0x7f1b1c06f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41772/0 (socket says 172.21.15.70:41772) 2023-08-25T09:50:39.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b216af640 1 -- 172.21.15.70:0/4180415278 learned_addr learned my addr 172.21.15.70:0/4180415278 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:39.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.745+0000 7f1b1bfff640 1 --2- 172.21.15.70:0/4180415278 >> v2:172.21.15.155:3300/0 conn(0x7f1b1c15aa10 0x7f1b1c0b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094861214 0 0) 0x7f1b1c0b2ad0 con 0x7f1b1c0b2680 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b08003520 con 0x7f1b1c0b2680 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177695463 0 0) 0x7f1b08003520 con 0x7f1b1c0b2680 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 >> v2:172.21.15.155:3300/0 conn(0x7f1b1c15aa10 msgr2=0x7f1b1c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 --2- 172.21.15.70:0/4180415278 >> v2:172.21.15.155:3300/0 conn(0x7f1b1c15aa10 0x7f1b1c0b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 >> v1:172.21.15.155:6789/0 conn(0x7f1b1c0ce5b0 legacy=0x7f1b1c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0b2d20 con 0x7f1b1c0b2680 2023-08-25T09:50:39.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1b14001670 con 0x7f1b1c0b2680 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 >> v1:172.21.15.70:6789/0 conn(0x7f1b1c0b2680 legacy=0x7f1b1c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b080039a0 0x7f1b08003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080047c0 0x7f1b08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b080051d0 0x7f1b0800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c159d10 con 0x7f1b080051d0 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0800e310 con 0x7f1b080047c0 2023-08-25T09:50:39.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0800e4d0 con 0x7f1b080039a0 2023-08-25T09:50:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.746+0000 7f1b216af640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b080051d0 0x7f1b0800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.747+0000 7f1b216af640 1 -- 172.21.15.70:0/4180415278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b080039a0 msgr2=0x7f1b08003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.747+0000 7f1b216af640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b080039a0 0x7f1b08003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.747+0000 7f1b216af640 1 -- 172.21.15.70:0/4180415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080047c0 msgr2=0x7f1b08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.747+0000 7f1b216af640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080047c0 0x7f1b08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.747+0000 7f1b216af640 1 -- 172.21.15.70:0/4180415278 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b08003520 con 0x7f1b080051d0 2023-08-25T09:50:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.748+0000 7f1b216af640 1 -- 172.21.15.70:0/4180415278 reap_dead start 2023-08-25T09:50:39.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.748+0000 7f1b216af640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b080051d0 0x7f1b0800db90 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c160 tx=0x7f1b080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c9fd3846faed2a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.748+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b14005ad0 con 0x7f1b080051d0 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.748+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b14005c70 con 0x7f1b080051d0 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.748+0000 7f1b1b7fe640 1 -- 172.21.15.70:0/4180415278 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1402c870 con 0x7f1b080051d0 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 -- 172.21.15.70:0/4180415278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b080051d0 msgr2=0x7f1b0800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b080051d0 0x7f1b0800db90 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c160 tx=0x7f1b080057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 -- 172.21.15.70:0/4180415278 shutdown_connections 2023-08-25T09:50:39.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 --2- 172.21.15.70:0/4180415278 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b080051d0 0x7f1b0800db90 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 -- 172.21.15.70:0/4180415278 >> 172.21.15.70:0/4180415278 conn(0x7f1b1c069850 msgr2=0x7f1b1c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 -- 172.21.15.70:0/4180415278 shutdown_connections 2023-08-25T09:50:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.749+0000 7f1b23112640 1 -- 172.21.15.70:0/4180415278 wait complete. 2023-08-25T09:50:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.751+0000 7f1b23112640 1 Processor -- start 2023-08-25T09:50:39.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.751+0000 7f1b23112640 1 -- start start 2023-08-25T09:50:39.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b23112640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080051d0 0x7f1b1c0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b23112640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 0x7f1b1c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b23112640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b1c1ee260 0x7f1b1c1ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b23112640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0af060 con 0x7f1b1c0add20 2023-08-25T09:50:39.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b23112640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0ae1a0 con 0x7f1b080051d0 2023-08-25T09:50:39.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b23112640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1c0ae450 con 0x7f1b1c1ee260 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 0x7f1b1c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 0x7f1b1c151b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51156/0 (socket says 172.21.15.70:51156) 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 -- 172.21.15.70:0/2190021250 learned_addr learned my addr 172.21.15.70:0/2190021250 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b20eae640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080051d0 0x7f1b1c0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b216af640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b1c1ee260 0x7f1b1c1ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 -- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b1c1ee260 msgr2=0x7f1b1c1ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b1c1ee260 0x7f1b1c1ee6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 -- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080051d0 msgr2=0x7f1b1c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.752+0000 7f1b1bfff640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080051d0 0x7f1b1c0cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b1bfff640 1 -- 172.21.15.70:0/2190021250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1c0ae730 con 0x7f1b1c0add20 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b20eae640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080051d0 0x7f1b1c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b1bfff640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 0x7f1b1c151b20 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400e7a0 tx=0x7f1b0400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400eeb0 con 0x7f1b1c0add20 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0400c940 con 0x7f1b1c0add20 2023-08-25T09:50:39.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400cd10 con 0x7f1b1c0add20 2023-08-25T09:50:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b1c0aea10 con 0x7f1b1c0add20 2023-08-25T09:50:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.753+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b1c152060 con 0x7f1b1c0add20 2023-08-25T09:50:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.755+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1c15d690 con 0x7f1b1c0add20 2023-08-25T09:50:39.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.755+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1b04013470 con 0x7f1b1c0add20 2023-08-25T09:50:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.756+0000 7f1b197fa640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1afc06a080 0x7f1afc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.756+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b04088700 con 0x7f1b1c0add20 2023-08-25T09:50:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.757+0000 7f1b20eae640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1afc06a080 0x7f1afc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:39.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.757+0000 7f1b20eae640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1afc06a080 0x7f1afc06c540 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0ad220 tx=0x7f1b0c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:39.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.761+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1b1c15d690 con 0x7f1b1c0add20 2023-08-25T09:50:39.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.946+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b1c15d990 con 0x7f1afc06a080 2023-08-25T09:50:39.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.970+0000 7f1b197fa640 1 -- 172.21.15.70:0/2190021250 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302890 (secure 0 0 0) 0x7f1b0c001a70 con 0x7f1afc06a080 2023-08-25T09:50:39.973 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:39.977 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:50:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1afc06a080 msgr2=0x7f1afc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1afc06a080 0x7f1afc06c540 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0ad220 tx=0x7f1b0c009040 comp rx=0 tx=0).stop 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 msgr2=0x7f1b1c151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 0x7f1b1c151b20 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1b0400e7a0 tx=0x7f1b0400ec60 comp rx=0 tx=0).stop 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 shutdown_connections 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1afc06a080 0x7f1afc06c540 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1b1c1ee260 0x7f1b1c1ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1b1c0add20 0x7f1b1c151b20 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 --2- 172.21.15.70:0/2190021250 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1b080051d0 0x7f1b1c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.976+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 >> 172.21.15.70:0/2190021250 conn(0x7f1b1c069850 msgr2=0x7f1b1c070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.977+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 shutdown_connections 2023-08-25T09:50:39.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:39.977+0000 7f1b23112640 1 -- 172.21.15.70:0/2190021250 wait complete. 2023-08-25T09:50:39.998 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2023-08-25T09:50:39.562367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552811,"statfs":{"total":1600000000,"available":1590079428,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001262"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505591+0000","last_change":"2023-08-25T09:50:35.537407+0000","last_active":"2023-08-25T09:50:36.505591+0000","last_peered":"2023-08-25T09:50:36.505591+0000","last_clean":"2023-08-25T09:50:36.505591+0000","last_became_active":"2023-08-25T09:50:35.537174+0000","last_became_peered":"2023-08-25T09:50:35.537174+0000","last_unstale":"2023-08-25T09:50:36.505591+0000","last_undegraded":"2023-08-25T09:50:36.505591+0000","last_fullsized":"2023-08-25T09:50:36.505591+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:49.485164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510204+0000","last_change":"2023-08-25T09:50:35.531429+0000","last_active":"2023-08-25T09:50:36.510204+0000","last_peered":"2023-08-25T09:50:36.510204+0000","last_clean":"2023-08-25T09:50:36.510204+0000","last_became_active":"2023-08-25T09:50:35.531175+0000","last_became_peered":"2023-08-25T09:50:35.531175+0000","last_unstale":"2023-08-25T09:50:36.510204+0000","last_undegraded":"2023-08-25T09:50:36.510204+0000","last_fullsized":"2023-08-25T09:50:36.510204+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:51:42.860626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510814+0000","last_change":"2023-08-25T09:50:35.524262+0000","last_active":"2023-08-25T09:50:36.510814+0000","last_peered":"2023-08-25T09:50:36.510814+0000","last_clean":"2023-08-25T09:50:36.510814+0000","last_became_active":"2023-08-25T09:50:35.524041+0000","last_became_peered":"2023-08-25T09:50:35.524041+0000","last_unstale":"2023-08-25T09:50:36.510814+0000","last_undegraded":"2023-08-25T09:50:36.510814+0000","last_fullsized":"2023-08-25T09:50:36.510814+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:12.844946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656059+0000","last_change":"2023-08-25T09:50:35.534809+0000","last_active":"2023-08-25T09:50:36.656059+0000","last_peered":"2023-08-25T09:50:36.656059+0000","last_clean":"2023-08-25T09:50:36.656059+0000","last_became_active":"2023-08-25T09:50:35.534101+0000","last_became_peered":"2023-08-25T09:50:35.534101+0000","last_unstale":"2023-08-25T09:50:36.656059+0000","last_undegraded":"2023-08-25T09:50:36.656059+0000","last_fullsized":"2023-08-25T09:50:36.656059+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:19.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511154+0000","last_change":"2023-08-25T09:50:35.523079+0000","last_active":"2023-08-25T09:50:36.511154+0000","last_peered":"2023-08-25T09:50:36.511154+0000","last_clean":"2023-08-25T09:50:36.511154+0000","last_became_active":"2023-08-25T09:50:35.522792+0000","last_became_peered":"2023-08-25T09:50:35.522792+0000","last_unstale":"2023-08-25T09:50:36.511154+0000","last_undegraded":"2023-08-25T09:50:36.511154+0000","last_fullsized":"2023-08-25T09:50:36.511154+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:59:11.178849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658462+0000","last_change":"2023-08-25T09:50:35.519479+0000","last_active":"2023-08-25T09:50:36.658462+0000","last_peered":"2023-08-25T09:50:36.658462+0000","last_clean":"2023-08-25T09:50:36.658462+0000","last_became_active":"2023-08-25T09:50:35.519256+0000","last_became_peered":"2023-08-25T09:50:35.519256+0000","last_unstale":"2023-08-25T09:50:36.658462+0000","last_undegraded":"2023-08-25T09:50:36.658462+0000","last_fullsized":"2023-08-25T09:50:36.658462+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:51.419901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506124+0000","last_change":"2023-08-25T09:50:35.521872+0000","last_active":"2023-08-25T09:50:36.506124+0000","last_peered":"2023-08-25T09:50:36.506124+0000","last_clean":"2023-08-25T09:50:36.506124+0000","last_became_active":"2023-08-25T09:50:35.520647+0000","last_became_peered":"2023-08-25T09:50:35.520647+0000","last_unstale":"2023-08-25T09:50:36.506124+0000","last_undegraded":"2023-08-25T09:50:36.506124+0000","last_fullsized":"2023-08-25T09:50:36.506124+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:33.209821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656199+0000","last_change":"2023-08-25T09:50:35.533938+0000","last_active":"2023-08-25T09:50:36.656199+0000","last_peered":"2023-08-25T09:50:36.656199+0000","last_clean":"2023-08-25T09:50:36.656199+0000","last_became_active":"2023-08-25T09:50:35.533577+0000","last_became_peered":"2023-08-25T09:50:35.533577+0000","last_unstale":"2023-08-25T09:50:36.656199+0000","last_undegraded":"2023-08-25T09:50:36.656199+0000","last_fullsized":"2023-08-25T09:50:36.656199+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:44.154342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510352+0000","last_change":"2023-08-25T09:50:35.536587+0000","last_active":"2023-08-25T09:50:36.510352+0000","last_peered":"2023-08-25T09:50:36.510352+0000","last_clean":"2023-08-25T09:50:36.510352+0000","last_became_active":"2023-08-25T09:50:35.536283+0000","last_became_peered":"2023-08-25T09:50:35.536283+0000","last_unstale":"2023-08-25T09:50:36.510352+0000","last_undegraded":"2023-08-25T09:50:36.510352+0000","last_fullsized":"2023-08-25T09:50:36.510352+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:26:24.330633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512402+0000","last_change":"2023-08-25T09:50:35.516669+0000","last_active":"2023-08-25T09:50:36.512402+0000","last_peered":"2023-08-25T09:50:36.512402+0000","last_clean":"2023-08-25T09:50:36.512402+0000","last_became_active":"2023-08-25T09:50:35.516432+0000","last_became_peered":"2023-08-25T09:50:35.516432+0000","last_unstale":"2023-08-25T09:50:36.512402+0000","last_undegraded":"2023-08-25T09:50:36.512402+0000","last_fullsized":"2023-08-25T09:50:36.512402+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:38.092636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659026+0000","last_change":"2023-08-25T09:50:35.519309+0000","last_active":"2023-08-25T09:50:36.659026+0000","last_peered":"2023-08-25T09:50:36.659026+0000","last_clean":"2023-08-25T09:50:36.659026+0000","last_became_active":"2023-08-25T09:50:35.519052+0000","last_became_peered":"2023-08-25T09:50:35.519052+0000","last_unstale":"2023-08-25T09:50:36.659026+0000","last_undegraded":"2023-08-25T09:50:36.659026+0000","last_fullsized":"2023-08-25T09:50:36.659026+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:24.221872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657112+0000","last_change":"2023-08-25T09:50:35.531731+0000","last_active":"2023-08-25T09:50:36.657112+0000","last_peered":"2023-08-25T09:50:36.657112+0000","last_clean":"2023-08-25T09:50:36.657112+0000","last_became_active":"2023-08-25T09:50:35.531401+0000","last_became_peered":"2023-08-25T09:50:35.531401+0000","last_unstale":"2023-08-25T09:50:36.657112+0000","last_undegraded":"2023-08-25T09:50:36.657112+0000","last_fullsized":"2023-08-25T09:50:36.657112+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:45:50.225326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.509910+0000","last_change":"2023-08-25T09:50:35.515361+0000","last_active":"2023-08-25T09:50:36.509910+0000","last_peered":"2023-08-25T09:50:36.509910+0000","last_clean":"2023-08-25T09:50:36.509910+0000","last_became_active":"2023-08-25T09:50:35.515100+0000","last_became_peered":"2023-08-25T09:50:35.515100+0000","last_unstale":"2023-08-25T09:50:36.509910+0000","last_undegraded":"2023-08-25T09:50:36.509910+0000","last_fullsized":"2023-08-25T09:50:36.509910+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:34.123853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658448+0000","last_change":"2023-08-25T09:50:35.530046+0000","last_active":"2023-08-25T09:50:36.658448+0000","last_peered":"2023-08-25T09:50:36.658448+0000","last_clean":"2023-08-25T09:50:36.658448+0000","last_became_active":"2023-08-25T09:50:35.527466+0000","last_became_peered":"2023-08-25T09:50:35.527466+0000","last_unstale":"2023-08-25T09:50:36.658448+0000","last_undegraded":"2023-08-25T09:50:36.658448+0000","last_fullsized":"2023-08-25T09:50:36.658448+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:33.557742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658844+0000","last_change":"2023-08-25T09:50:35.530507+0000","last_active":"2023-08-25T09:50:36.658844+0000","last_peered":"2023-08-25T09:50:36.658844+0000","last_clean":"2023-08-25T09:50:36.658844+0000","last_became_active":"2023-08-25T09:50:35.530267+0000","last_became_peered":"2023-08-25T09:50:35.530267+0000","last_unstale":"2023-08-25T09:50:36.658844+0000","last_undegraded":"2023-08-25T09:50:36.658844+0000","last_fullsized":"2023-08-25T09:50:36.658844+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:01:03.485056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658562+0000","last_change":"2023-08-25T09:50:35.523707+0000","last_active":"2023-08-25T09:50:36.658562+0000","last_peered":"2023-08-25T09:50:36.658562+0000","last_clean":"2023-08-25T09:50:36.658562+0000","last_became_active":"2023-08-25T09:50:35.523483+0000","last_became_peered":"2023-08-25T09:50:35.523483+0000","last_unstale":"2023-08-25T09:50:36.658562+0000","last_undegraded":"2023-08-25T09:50:36.658562+0000","last_fullsized":"2023-08-25T09:50:36.658562+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:39.457561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657605+0000","last_change":"2023-08-25T09:50:35.528094+0000","last_active":"2023-08-25T09:50:36.657605+0000","last_peered":"2023-08-25T09:50:36.657605+0000","last_clean":"2023-08-25T09:50:36.657605+0000","last_became_active":"2023-08-25T09:50:35.527574+0000","last_became_peered":"2023-08-25T09:50:35.527574+0000","last_unstale":"2023-08-25T09:50:36.657605+0000","last_undegraded":"2023-08-25T09:50:36.657605+0000","last_fullsized":"2023-08-25T09:50:36.657605+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:07:06.557564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510886+0000","last_change":"2023-08-25T09:50:35.515892+0000","last_active":"2023-08-25T09:50:36.510886+0000","last_peered":"2023-08-25T09:50:36.510886+0000","last_clean":"2023-08-25T09:50:36.510886+0000","last_became_active":"2023-08-25T09:50:35.515626+0000","last_became_peered":"2023-08-25T09:50:35.515626+0000","last_unstale":"2023-08-25T09:50:36.510886+0000","last_undegraded":"2023-08-25T09:50:36.510886+0000","last_fullsized":"2023-08-25T09:50:36.510886+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:38.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505609+0000","last_change":"2023-08-25T09:50:35.525792+0000","last_active":"2023-08-25T09:50:36.505609+0000","last_peered":"2023-08-25T09:50:36.505609+0000","last_clean":"2023-08-25T09:50:36.505609+0000","last_became_active":"2023-08-25T09:50:35.522122+0000","last_became_peered":"2023-08-25T09:50:35.522122+0000","last_unstale":"2023-08-25T09:50:36.505609+0000","last_undegraded":"2023-08-25T09:50:36.505609+0000","last_fullsized":"2023-08-25T09:50:36.505609+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:59:39.522298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659490+0000","last_change":"2023-08-25T09:50:35.519299+0000","last_active":"2023-08-25T09:50:36.659490+0000","last_peered":"2023-08-25T09:50:36.659490+0000","last_clean":"2023-08-25T09:50:36.659490+0000","last_became_active":"2023-08-25T09:50:35.519010+0000","last_became_peered":"2023-08-25T09:50:35.519010+0000","last_unstale":"2023-08-25T09:50:36.659490+0000","last_undegraded":"2023-08-25T09:50:36.659490+0000","last_fullsized":"2023-08-25T09:50:36.659490+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:53.862704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659218+0000","last_change":"2023-08-25T09:50:35.535260+0000","last_active":"2023-08-25T09:50:36.659218+0000","last_peered":"2023-08-25T09:50:36.659218+0000","last_clean":"2023-08-25T09:50:36.659218+0000","last_became_active":"2023-08-25T09:50:35.534954+0000","last_became_peered":"2023-08-25T09:50:35.534954+0000","last_unstale":"2023-08-25T09:50:36.659218+0000","last_undegraded":"2023-08-25T09:50:36.659218+0000","last_fullsized":"2023-08-25T09:50:36.659218+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:44.831905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506440+0000","last_change":"2023-08-25T09:50:35.531609+0000","last_active":"2023-08-25T09:50:36.506440+0000","last_peered":"2023-08-25T09:50:36.506440+0000","last_clean":"2023-08-25T09:50:36.506440+0000","last_became_active":"2023-08-25T09:50:35.531231+0000","last_became_peered":"2023-08-25T09:50:35.531231+0000","last_unstale":"2023-08-25T09:50:36.506440+0000","last_undegraded":"2023-08-25T09:50:36.506440+0000","last_fullsized":"2023-08-25T09:50:36.506440+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:18.997018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656321+0000","last_change":"2023-08-25T09:50:35.535757+0000","last_active":"2023-08-25T09:50:36.656321+0000","last_peered":"2023-08-25T09:50:36.656321+0000","last_clean":"2023-08-25T09:50:36.656321+0000","last_became_active":"2023-08-25T09:50:35.535496+0000","last_became_peered":"2023-08-25T09:50:35.535496+0000","last_unstale":"2023-08-25T09:50:36.656321+0000","last_undegraded":"2023-08-25T09:50:36.656321+0000","last_fullsized":"2023-08-25T09:50:36.656321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:01:59.019297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510510+0000","last_change":"2023-08-25T09:50:35.517766+0000","last_active":"2023-08-25T09:50:36.510510+0000","last_peered":"2023-08-25T09:50:36.510510+0000","last_clean":"2023-08-25T09:50:36.510510+0000","last_became_active":"2023-08-25T09:50:35.517540+0000","last_became_peered":"2023-08-25T09:50:35.517540+0000","last_unstale":"2023-08-25T09:50:36.510510+0000","last_undegraded":"2023-08-25T09:50:36.510510+0000","last_fullsized":"2023-08-25T09:50:36.510510+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:32.565910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505651+0000","last_change":"2023-08-25T09:50:35.517583+0000","last_active":"2023-08-25T09:50:36.505651+0000","last_peered":"2023-08-25T09:50:36.505651+0000","last_clean":"2023-08-25T09:50:36.505651+0000","last_became_active":"2023-08-25T09:50:35.517346+0000","last_became_peered":"2023-08-25T09:50:35.517346+0000","last_unstale":"2023-08-25T09:50:36.505651+0000","last_undegraded":"2023-08-25T09:50:36.505651+0000","last_fullsized":"2023-08-25T09:50:36.505651+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:24:41.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658174+0000","last_change":"2023-08-25T09:50:35.532787+0000","last_active":"2023-08-25T09:50:36.658174+0000","last_peered":"2023-08-25T09:50:36.658174+0000","last_clean":"2023-08-25T09:50:36.658174+0000","last_became_active":"2023-08-25T09:50:35.532531+0000","last_became_peered":"2023-08-25T09:50:35.532531+0000","last_unstale":"2023-08-25T09:50:36.658174+0000","last_undegraded":"2023-08-25T09:50:36.658174+0000","last_fullsized":"2023-08-25T09:50:36.658174+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:21:14.023646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511379+0000","last_change":"2023-08-25T09:50:35.531823+0000","last_active":"2023-08-25T09:50:36.511379+0000","last_peered":"2023-08-25T09:50:36.511379+0000","last_clean":"2023-08-25T09:50:36.511379+0000","last_became_active":"2023-08-25T09:50:35.531592+0000","last_became_peered":"2023-08-25T09:50:35.531592+0000","last_unstale":"2023-08-25T09:50:36.511379+0000","last_undegraded":"2023-08-25T09:50:36.511379+0000","last_fullsized":"2023-08-25T09:50:36.511379+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:54:30.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510230+0000","last_change":"2023-08-25T09:50:35.520768+0000","last_active":"2023-08-25T09:50:36.510230+0000","last_peered":"2023-08-25T09:50:36.510230+0000","last_clean":"2023-08-25T09:50:36.510230+0000","last_became_active":"2023-08-25T09:50:35.520534+0000","last_became_peered":"2023-08-25T09:50:35.520534+0000","last_unstale":"2023-08-25T09:50:36.510230+0000","last_undegraded":"2023-08-25T09:50:36.510230+0000","last_fullsized":"2023-08-25T09:50:36.510230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:09:15.827159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656651+0000","last_change":"2023-08-25T09:50:35.534327+0000","last_active":"2023-08-25T09:50:36.656651+0000","last_peered":"2023-08-25T09:50:36.656651+0000","last_clean":"2023-08-25T09:50:36.656651+0000","last_became_active":"2023-08-25T09:50:35.533158+0000","last_became_peered":"2023-08-25T09:50:35.533158+0000","last_unstale":"2023-08-25T09:50:36.656651+0000","last_undegraded":"2023-08-25T09:50:36.656651+0000","last_fullsized":"2023-08-25T09:50:36.656651+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:40:22.636730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512229+0000","last_change":"2023-08-25T09:50:35.531744+0000","last_active":"2023-08-25T09:50:36.512229+0000","last_peered":"2023-08-25T09:50:36.512229+0000","last_clean":"2023-08-25T09:50:36.512229+0000","last_became_active":"2023-08-25T09:50:35.531508+0000","last_became_peered":"2023-08-25T09:50:35.531508+0000","last_unstale":"2023-08-25T09:50:36.512229+0000","last_undegraded":"2023-08-25T09:50:36.512229+0000","last_fullsized":"2023-08-25T09:50:36.512229+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:51:50.203775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659387+0000","last_change":"2023-08-25T09:50:35.521820+0000","last_active":"2023-08-25T09:50:36.659387+0000","last_peered":"2023-08-25T09:50:36.659387+0000","last_clean":"2023-08-25T09:50:36.659387+0000","last_became_active":"2023-08-25T09:50:35.521582+0000","last_became_peered":"2023-08-25T09:50:35.521582+0000","last_unstale":"2023-08-25T09:50:36.659387+0000","last_undegraded":"2023-08-25T09:50:36.659387+0000","last_fullsized":"2023-08-25T09:50:36.659387+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:14:18.772984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657176+0000","last_change":"2023-08-25T09:50:35.525792+0000","last_active":"2023-08-25T09:50:36.657176+0000","last_peered":"2023-08-25T09:50:36.657176+0000","last_clean":"2023-08-25T09:50:36.657176+0000","last_became_active":"2023-08-25T09:50:35.525255+0000","last_became_peered":"2023-08-25T09:50:35.525255+0000","last_unstale":"2023-08-25T09:50:36.657176+0000","last_undegraded":"2023-08-25T09:50:36.657176+0000","last_fullsized":"2023-08-25T09:50:36.657176+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:48:10.259157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659577+0000","last_change":"2023-08-25T09:50:35.527708+0000","last_active":"2023-08-25T09:50:36.659577+0000","last_peered":"2023-08-25T09:50:36.659577+0000","last_clean":"2023-08-25T09:50:36.659577+0000","last_became_active":"2023-08-25T09:50:35.527243+0000","last_became_peered":"2023-08-25T09:50:35.527243+0000","last_unstale":"2023-08-25T09:50:36.659577+0000","last_undegraded":"2023-08-25T09:50:36.659577+0000","last_fullsized":"2023-08-25T09:50:36.659577+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:39.085915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"181'4","reported_seq":17,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657558+0000","last_change":"2023-08-25T09:50:34.520850+0000","last_active":"2023-08-25T09:50:36.657558+0000","last_peered":"2023-08-25T09:50:36.657558+0000","last_clean":"2023-08-25T09:50:36.657558+0000","last_became_active":"2023-08-25T09:50:34.520601+0000","last_became_peered":"2023-08-25T09:50:34.520601+0000","last_unstale":"2023-08-25T09:50:36.657558+0000","last_undegraded":"2023-08-25T09:50:36.657558+0000","last_fullsized":"2023-08-25T09:50:36.657558+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:51:49.197623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511432+0000","last_change":"2023-08-25T09:50:35.531143+0000","last_active":"2023-08-25T09:50:36.511432+0000","last_peered":"2023-08-25T09:50:36.511432+0000","last_clean":"2023-08-25T09:50:36.511432+0000","last_became_active":"2023-08-25T09:50:35.530915+0000","last_became_peered":"2023-08-25T09:50:35.530915+0000","last_unstale":"2023-08-25T09:50:36.511432+0000","last_undegraded":"2023-08-25T09:50:36.511432+0000","last_fullsized":"2023-08-25T09:50:36.511432+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:50:56.211336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656483+0000","last_change":"2023-08-25T09:50:34.525079+0000","last_active":"2023-08-25T09:50:36.656483+0000","last_peered":"2023-08-25T09:50:36.656483+0000","last_clean":"2023-08-25T09:50:36.656483+0000","last_became_active":"2023-08-25T09:50:34.524831+0000","last_became_peered":"2023-08-25T09:50:34.524831+0000","last_unstale":"2023-08-25T09:50:36.656483+0000","last_undegraded":"2023-08-25T09:50:36.656483+0000","last_fullsized":"2023-08-25T09:50:36.656483+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:41:23.608995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656296+0000","last_change":"2023-08-25T09:50:35.529049+0000","last_active":"2023-08-25T09:50:36.656296+0000","last_peered":"2023-08-25T09:50:36.656296+0000","last_clean":"2023-08-25T09:50:36.656296+0000","last_became_active":"2023-08-25T09:50:35.528787+0000","last_became_peered":"2023-08-25T09:50:35.528787+0000","last_unstale":"2023-08-25T09:50:36.656296+0000","last_undegraded":"2023-08-25T09:50:36.656296+0000","last_fullsized":"2023-08-25T09:50:36.656296+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:35.694521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658026+0000","last_change":"2023-08-25T09:50:34.516199+0000","last_active":"2023-08-25T09:50:36.658026+0000","last_peered":"2023-08-25T09:50:36.658026+0000","last_clean":"2023-08-25T09:50:36.658026+0000","last_became_active":"2023-08-25T09:50:34.515459+0000","last_became_peered":"2023-08-25T09:50:34.515459+0000","last_unstale":"2023-08-25T09:50:36.658026+0000","last_undegraded":"2023-08-25T09:50:36.658026+0000","last_fullsized":"2023-08-25T09:50:36.658026+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:51:43.426280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656581+0000","last_change":"2023-08-25T09:50:35.519927+0000","last_active":"2023-08-25T09:50:36.656581+0000","last_peered":"2023-08-25T09:50:36.656581+0000","last_clean":"2023-08-25T09:50:36.656581+0000","last_became_active":"2023-08-25T09:50:35.519528+0000","last_became_peered":"2023-08-25T09:50:35.519528+0000","last_unstale":"2023-08-25T09:50:36.656581+0000","last_undegraded":"2023-08-25T09:50:36.656581+0000","last_fullsized":"2023-08-25T09:50:36.656581+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:55.364627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511886+0000","last_change":"2023-08-25T09:50:34.526826+0000","last_active":"2023-08-25T09:50:36.511886+0000","last_peered":"2023-08-25T09:50:36.511886+0000","last_clean":"2023-08-25T09:50:36.511886+0000","last_became_active":"2023-08-25T09:50:34.526200+0000","last_became_peered":"2023-08-25T09:50:34.526200+0000","last_unstale":"2023-08-25T09:50:36.511886+0000","last_undegraded":"2023-08-25T09:50:36.511886+0000","last_fullsized":"2023-08-25T09:50:36.511886+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:22:37.241070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659089+0000","last_change":"2023-08-25T09:50:35.520980+0000","last_active":"2023-08-25T09:50:36.659089+0000","last_peered":"2023-08-25T09:50:36.659089+0000","last_clean":"2023-08-25T09:50:36.659089+0000","last_became_active":"2023-08-25T09:50:35.520355+0000","last_became_peered":"2023-08-25T09:50:35.520355+0000","last_unstale":"2023-08-25T09:50:36.659089+0000","last_undegraded":"2023-08-25T09:50:36.659089+0000","last_fullsized":"2023-08-25T09:50:36.659089+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:27.119175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657459+0000","last_change":"2023-08-25T09:50:34.529984+0000","last_active":"2023-08-25T09:50:36.657459+0000","last_peered":"2023-08-25T09:50:36.657459+0000","last_clean":"2023-08-25T09:50:36.657459+0000","last_became_active":"2023-08-25T09:50:34.529286+0000","last_became_peered":"2023-08-25T09:50:34.529286+0000","last_unstale":"2023-08-25T09:50:36.657459+0000","last_undegraded":"2023-08-25T09:50:36.657459+0000","last_fullsized":"2023-08-25T09:50:36.657459+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:00:12.218056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658686+0000","last_change":"2023-08-25T09:50:35.525317+0000","last_active":"2023-08-25T09:50:36.658686+0000","last_peered":"2023-08-25T09:50:36.658686+0000","last_clean":"2023-08-25T09:50:36.658686+0000","last_became_active":"2023-08-25T09:50:35.524382+0000","last_became_peered":"2023-08-25T09:50:35.524382+0000","last_unstale":"2023-08-25T09:50:36.658686+0000","last_undegraded":"2023-08-25T09:50:36.658686+0000","last_fullsized":"2023-08-25T09:50:36.658686+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:40:27.511906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510448+0000","last_change":"2023-08-25T09:50:34.522225+0000","last_active":"2023-08-25T09:50:36.510448+0000","last_peered":"2023-08-25T09:50:36.510448+0000","last_clean":"2023-08-25T09:50:36.510448+0000","last_became_active":"2023-08-25T09:50:34.521947+0000","last_became_peered":"2023-08-25T09:50:34.521947+0000","last_unstale":"2023-08-25T09:50:36.510448+0000","last_undegraded":"2023-08-25T09:50:36.510448+0000","last_fullsized":"2023-08-25T09:50:36.510448+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:42.504958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657564+0000","last_change":"2023-08-25T09:50:35.524325+0000","last_active":"2023-08-25T09:50:36.657564+0000","last_peered":"2023-08-25T09:50:36.657564+0000","last_clean":"2023-08-25T09:50:36.657564+0000","last_became_active":"2023-08-25T09:50:35.523972+0000","last_became_peered":"2023-08-25T09:50:35.523972+0000","last_unstale":"2023-08-25T09:50:36.657564+0000","last_undegraded":"2023-08-25T09:50:36.657564+0000","last_fullsized":"2023-08-25T09:50:36.657564+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:52:51.584851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659265+0000","last_change":"2023-08-25T09:50:34.523651+0000","last_active":"2023-08-25T09:50:36.659265+0000","last_peered":"2023-08-25T09:50:36.659265+0000","last_clean":"2023-08-25T09:50:36.659265+0000","last_became_active":"2023-08-25T09:50:34.523010+0000","last_became_peered":"2023-08-25T09:50:34.523010+0000","last_unstale":"2023-08-25T09:50:36.659265+0000","last_undegraded":"2023-08-25T09:50:36.659265+0000","last_fullsized":"2023-08-25T09:50:36.659265+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:16:59.601307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657372+0000","last_change":"2023-08-25T09:50:35.525925+0000","last_active":"2023-08-25T09:50:36.657372+0000","last_peered":"2023-08-25T09:50:36.657372+0000","last_clean":"2023-08-25T09:50:36.657372+0000","last_became_active":"2023-08-25T09:50:35.525559+0000","last_became_peered":"2023-08-25T09:50:35.525559+0000","last_unstale":"2023-08-25T09:50:36.657372+0000","last_undegraded":"2023-08-25T09:50:36.657372+0000","last_fullsized":"2023-08-25T09:50:36.657372+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:47.712163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511549+0000","last_change":"2023-08-25T09:50:34.524845+0000","last_active":"2023-08-25T09:50:36.511549+0000","last_peered":"2023-08-25T09:50:36.511549+0000","last_clean":"2023-08-25T09:50:36.511549+0000","last_became_active":"2023-08-25T09:50:34.524122+0000","last_became_peered":"2023-08-25T09:50:34.524122+0000","last_unstale":"2023-08-25T09:50:36.511549+0000","last_undegraded":"2023-08-25T09:50:36.511549+0000","last_fullsized":"2023-08-25T09:50:36.511549+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:14.771032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.507248+0000","last_change":"2023-08-25T09:50:35.528714+0000","last_active":"2023-08-25T09:50:36.507248+0000","last_peered":"2023-08-25T09:50:36.507248+0000","last_clean":"2023-08-25T09:50:36.507248+0000","last_became_active":"2023-08-25T09:50:35.521351+0000","last_became_peered":"2023-08-25T09:50:35.521351+0000","last_unstale":"2023-08-25T09:50:36.507248+0000","last_undegraded":"2023-08-25T09:50:36.507248+0000","last_fullsized":"2023-08-25T09:50:36.507248+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:01.386869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510622+0000","last_change":"2023-08-25T09:50:34.516216+0000","last_active":"2023-08-25T09:50:36.510622+0000","last_peered":"2023-08-25T09:50:36.510622+0000","last_clean":"2023-08-25T09:50:36.510622+0000","last_became_active":"2023-08-25T09:50:34.515369+0000","last_became_peered":"2023-08-25T09:50:34.515369+0000","last_unstale":"2023-08-25T09:50:36.510622+0000","last_undegraded":"2023-08-25T09:50:36.510622+0000","last_fullsized":"2023-08-25T09:50:36.510622+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:58.487878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657301+0000","last_change":"2023-08-25T09:50:35.532194+0000","last_active":"2023-08-25T09:50:36.657301+0000","last_peered":"2023-08-25T09:50:36.657301+0000","last_clean":"2023-08-25T09:50:36.657301+0000","last_became_active":"2023-08-25T09:50:35.530499+0000","last_became_peered":"2023-08-25T09:50:35.530499+0000","last_unstale":"2023-08-25T09:50:36.657301+0000","last_undegraded":"2023-08-25T09:50:36.657301+0000","last_fullsized":"2023-08-25T09:50:36.657301+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:16:21.461028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657719+0000","last_change":"2023-08-25T09:50:34.525090+0000","last_active":"2023-08-25T09:50:36.657719+0000","last_peered":"2023-08-25T09:50:36.657719+0000","last_clean":"2023-08-25T09:50:36.657719+0000","last_became_active":"2023-08-25T09:50:34.524648+0000","last_became_peered":"2023-08-25T09:50:34.524648+0000","last_unstale":"2023-08-25T09:50:36.657719+0000","last_undegraded":"2023-08-25T09:50:36.657719+0000","last_fullsized":"2023-08-25T09:50:36.657719+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:49.980136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657850+0000","last_change":"2023-08-25T09:50:35.526155+0000","last_active":"2023-08-25T09:50:36.657850+0000","last_peered":"2023-08-25T09:50:36.657850+0000","last_clean":"2023-08-25T09:50:36.657850+0000","last_became_active":"2023-08-25T09:50:35.525788+0000","last_became_peered":"2023-08-25T09:50:35.525788+0000","last_unstale":"2023-08-25T09:50:36.657850+0000","last_undegraded":"2023-08-25T09:50:36.657850+0000","last_fullsized":"2023-08-25T09:50:36.657850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:35.763214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512236+0000","last_change":"2023-08-25T09:50:34.526641+0000","last_active":"2023-08-25T09:50:36.512236+0000","last_peered":"2023-08-25T09:50:36.512236+0000","last_clean":"2023-08-25T09:50:36.512236+0000","last_became_active":"2023-08-25T09:50:34.525590+0000","last_became_peered":"2023-08-25T09:50:34.525590+0000","last_unstale":"2023-08-25T09:50:36.512236+0000","last_undegraded":"2023-08-25T09:50:36.512236+0000","last_fullsized":"2023-08-25T09:50:36.512236+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:11.985415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656318+0000","last_change":"2023-08-25T09:50:35.534488+0000","last_active":"2023-08-25T09:50:36.656318+0000","last_peered":"2023-08-25T09:50:36.656318+0000","last_clean":"2023-08-25T09:50:36.656318+0000","last_became_active":"2023-08-25T09:50:35.533454+0000","last_became_peered":"2023-08-25T09:50:35.533454+0000","last_unstale":"2023-08-25T09:50:36.656318+0000","last_undegraded":"2023-08-25T09:50:36.656318+0000","last_fullsized":"2023-08-25T09:50:36.656318+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:03.639440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656824+0000","last_change":"2023-08-25T09:50:34.525109+0000","last_active":"2023-08-25T09:50:36.656824+0000","last_peered":"2023-08-25T09:50:36.656824+0000","last_clean":"2023-08-25T09:50:36.656824+0000","last_became_active":"2023-08-25T09:50:34.524875+0000","last_became_peered":"2023-08-25T09:50:34.524875+0000","last_unstale":"2023-08-25T09:50:36.656824+0000","last_undegraded":"2023-08-25T09:50:36.656824+0000","last_fullsized":"2023-08-25T09:50:36.656824+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:09.400893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505755+0000","last_change":"2023-08-25T09:50:35.525415+0000","last_active":"2023-08-25T09:50:36.505755+0000","last_peered":"2023-08-25T09:50:36.505755+0000","last_clean":"2023-08-25T09:50:36.505755+0000","last_became_active":"2023-08-25T09:50:35.525185+0000","last_became_peered":"2023-08-25T09:50:35.525185+0000","last_unstale":"2023-08-25T09:50:36.505755+0000","last_undegraded":"2023-08-25T09:50:36.505755+0000","last_fullsized":"2023-08-25T09:50:36.505755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:36.753754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511116+0000","last_change":"2023-08-25T09:50:34.516618+0000","last_active":"2023-08-25T09:50:36.511116+0000","last_peered":"2023-08-25T09:50:36.511116+0000","last_clean":"2023-08-25T09:50:36.511116+0000","last_became_active":"2023-08-25T09:50:34.516139+0000","last_became_peered":"2023-08-25T09:50:34.516139+0000","last_unstale":"2023-08-25T09:50:36.511116+0000","last_undegraded":"2023-08-25T09:50:36.511116+0000","last_fullsized":"2023-08-25T09:50:36.511116+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:31:58.290329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658038+0000","last_change":"2023-08-25T09:50:35.520559+0000","last_active":"2023-08-25T09:50:36.658038+0000","last_peered":"2023-08-25T09:50:36.658038+0000","last_clean":"2023-08-25T09:50:36.658038+0000","last_became_active":"2023-08-25T09:50:35.520300+0000","last_became_peered":"2023-08-25T09:50:35.520300+0000","last_unstale":"2023-08-25T09:50:36.658038+0000","last_undegraded":"2023-08-25T09:50:36.658038+0000","last_fullsized":"2023-08-25T09:50:36.658038+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:46.389809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656392+0000","last_change":"2023-08-25T09:50:34.522545+0000","last_active":"2023-08-25T09:50:36.656392+0000","last_peered":"2023-08-25T09:50:36.656392+0000","last_clean":"2023-08-25T09:50:36.656392+0000","last_became_active":"2023-08-25T09:50:34.522307+0000","last_became_peered":"2023-08-25T09:50:34.522307+0000","last_unstale":"2023-08-25T09:50:36.656392+0000","last_undegraded":"2023-08-25T09:50:36.656392+0000","last_fullsized":"2023-08-25T09:50:36.656392+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:04.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506026+0000","last_change":"2023-08-25T09:50:35.525621+0000","last_active":"2023-08-25T09:50:36.506026+0000","last_peered":"2023-08-25T09:50:36.506026+0000","last_clean":"2023-08-25T09:50:36.506026+0000","last_became_active":"2023-08-25T09:50:35.521785+0000","last_became_peered":"2023-08-25T09:50:35.521785+0000","last_unstale":"2023-08-25T09:50:36.506026+0000","last_undegraded":"2023-08-25T09:50:36.506026+0000","last_fullsized":"2023-08-25T09:50:36.506026+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:37:24.308622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511703+0000","last_change":"2023-08-25T09:50:34.524688+0000","last_active":"2023-08-25T09:50:36.511703+0000","last_peered":"2023-08-25T09:50:36.511703+0000","last_clean":"2023-08-25T09:50:36.511703+0000","last_became_active":"2023-08-25T09:50:34.523666+0000","last_became_peered":"2023-08-25T09:50:34.523666+0000","last_unstale":"2023-08-25T09:50:36.511703+0000","last_undegraded":"2023-08-25T09:50:36.511703+0000","last_fullsized":"2023-08-25T09:50:36.511703+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:42:31.126738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658703+0000","last_change":"2023-08-25T09:50:35.523437+0000","last_active":"2023-08-25T09:50:36.658703+0000","last_peered":"2023-08-25T09:50:36.658703+0000","last_clean":"2023-08-25T09:50:36.658703+0000","last_became_active":"2023-08-25T09:50:35.523194+0000","last_became_peered":"2023-08-25T09:50:35.523194+0000","last_unstale":"2023-08-25T09:50:36.658703+0000","last_undegraded":"2023-08-25T09:50:36.658703+0000","last_fullsized":"2023-08-25T09:50:36.658703+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:18.738839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657502+0000","last_change":"2023-08-25T09:50:34.524307+0000","last_active":"2023-08-25T09:50:36.657502+0000","last_peered":"2023-08-25T09:50:36.657502+0000","last_clean":"2023-08-25T09:50:36.657502+0000","last_became_active":"2023-08-25T09:50:34.523860+0000","last_became_peered":"2023-08-25T09:50:34.523860+0000","last_unstale":"2023-08-25T09:50:36.657502+0000","last_undegraded":"2023-08-25T09:50:36.657502+0000","last_fullsized":"2023-08-25T09:50:36.657502+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:23:08.713308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659276+0000","last_change":"2023-08-25T09:50:35.521160+0000","last_active":"2023-08-25T09:50:36.659276+0000","last_peered":"2023-08-25T09:50:36.659276+0000","last_clean":"2023-08-25T09:50:36.659276+0000","last_became_active":"2023-08-25T09:50:35.520738+0000","last_became_peered":"2023-08-25T09:50:35.520738+0000","last_unstale":"2023-08-25T09:50:36.659276+0000","last_undegraded":"2023-08-25T09:50:36.659276+0000","last_fullsized":"2023-08-25T09:50:36.659276+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:14.072195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657828+0000","last_change":"2023-08-25T09:50:34.529605+0000","last_active":"2023-08-25T09:50:36.657828+0000","last_peered":"2023-08-25T09:50:36.657828+0000","last_clean":"2023-08-25T09:50:36.657828+0000","last_became_active":"2023-08-25T09:50:34.528881+0000","last_became_peered":"2023-08-25T09:50:34.528881+0000","last_unstale":"2023-08-25T09:50:36.657828+0000","last_undegraded":"2023-08-25T09:50:36.657828+0000","last_fullsized":"2023-08-25T09:50:36.657828+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:06:27.489325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658911+0000","last_change":"2023-08-25T09:50:35.527860+0000","last_active":"2023-08-25T09:50:36.658911+0000","last_peered":"2023-08-25T09:50:36.658911+0000","last_clean":"2023-08-25T09:50:36.658911+0000","last_became_active":"2023-08-25T09:50:35.525088+0000","last_became_peered":"2023-08-25T09:50:35.525088+0000","last_unstale":"2023-08-25T09:50:36.658911+0000","last_undegraded":"2023-08-25T09:50:36.658911+0000","last_fullsized":"2023-08-25T09:50:36.658911+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:25.624985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"182'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656963+0000","last_change":"2023-08-25T09:50:34.529128+0000","last_active":"2023-08-25T09:50:36.656963+0000","last_peered":"2023-08-25T09:50:36.656963+0000","last_clean":"2023-08-25T09:50:36.656963+0000","last_became_active":"2023-08-25T09:50:34.528569+0000","last_became_peered":"2023-08-25T09:50:34.528569+0000","last_unstale":"2023-08-25T09:50:36.656963+0000","last_undegraded":"2023-08-25T09:50:36.656963+0000","last_fullsized":"2023-08-25T09:50:36.656963+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:31.527354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506037+0000","last_change":"2023-08-25T09:50:35.524624+0000","last_active":"2023-08-25T09:50:36.506037+0000","last_peered":"2023-08-25T09:50:36.506037+0000","last_clean":"2023-08-25T09:50:36.506037+0000","last_became_active":"2023-08-25T09:50:35.524356+0000","last_became_peered":"2023-08-25T09:50:35.524356+0000","last_unstale":"2023-08-25T09:50:36.506037+0000","last_undegraded":"2023-08-25T09:50:36.506037+0000","last_fullsized":"2023-08-25T09:50:36.506037+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:15:42.562887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657600+0000","last_change":"2023-08-25T09:50:34.526702+0000","last_active":"2023-08-25T09:50:36.657600+0000","last_peered":"2023-08-25T09:50:36.657600+0000","last_clean":"2023-08-25T09:50:36.657600+0000","last_became_active":"2023-08-25T09:50:34.526229+0000","last_became_peered":"2023-08-25T09:50:34.526229+0000","last_unstale":"2023-08-25T09:50:36.657600+0000","last_undegraded":"2023-08-25T09:50:36.657600+0000","last_fullsized":"2023-08-25T09:50:36.657600+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:35:05.547294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510880+0000","last_change":"2023-08-25T09:50:35.512965+0000","last_active":"2023-08-25T09:50:36.510880+0000","last_peered":"2023-08-25T09:50:36.510880+0000","last_clean":"2023-08-25T09:50:36.510880+0000","last_became_active":"2023-08-25T09:50:35.512704+0000","last_became_peered":"2023-08-25T09:50:35.512704+0000","last_unstale":"2023-08-25T09:50:36.510880+0000","last_undegraded":"2023-08-25T09:50:36.510880+0000","last_fullsized":"2023-08-25T09:50:36.510880+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:11.163303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658794+0000","last_change":"2023-08-25T09:50:34.519204+0000","last_active":"2023-08-25T09:50:36.658794+0000","last_peered":"2023-08-25T09:50:36.658794+0000","last_clean":"2023-08-25T09:50:36.658794+0000","last_became_active":"2023-08-25T09:50:34.518943+0000","last_became_peered":"2023-08-25T09:50:34.518943+0000","last_unstale":"2023-08-25T09:50:36.658794+0000","last_undegraded":"2023-08-25T09:50:36.658794+0000","last_fullsized":"2023-08-25T09:50:36.658794+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:14:48.356922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659124+0000","last_change":"2023-08-25T09:50:35.529740+0000","last_active":"2023-08-25T09:50:36.659124+0000","last_peered":"2023-08-25T09:50:36.659124+0000","last_clean":"2023-08-25T09:50:36.659124+0000","last_became_active":"2023-08-25T09:50:35.527026+0000","last_became_peered":"2023-08-25T09:50:35.527026+0000","last_unstale":"2023-08-25T09:50:36.659124+0000","last_undegraded":"2023-08-25T09:50:36.659124+0000","last_fullsized":"2023-08-25T09:50:36.659124+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:56:17.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658221+0000","last_change":"2023-08-25T09:50:34.522459+0000","last_active":"2023-08-25T09:50:36.658221+0000","last_peered":"2023-08-25T09:50:36.658221+0000","last_clean":"2023-08-25T09:50:36.658221+0000","last_became_active":"2023-08-25T09:50:34.522116+0000","last_became_peered":"2023-08-25T09:50:34.522116+0000","last_unstale":"2023-08-25T09:50:36.658221+0000","last_undegraded":"2023-08-25T09:50:36.658221+0000","last_fullsized":"2023-08-25T09:50:36.658221+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:45.453693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656715+0000","last_change":"2023-08-25T09:50:35.519780+0000","last_active":"2023-08-25T09:50:36.656715+0000","last_peered":"2023-08-25T09:50:36.656715+0000","last_clean":"2023-08-25T09:50:36.656715+0000","last_became_active":"2023-08-25T09:50:35.519210+0000","last_became_peered":"2023-08-25T09:50:35.519210+0000","last_unstale":"2023-08-25T09:50:36.656715+0000","last_undegraded":"2023-08-25T09:50:36.656715+0000","last_fullsized":"2023-08-25T09:50:36.656715+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:56.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658199+0000","last_change":"2023-08-25T09:50:34.522546+0000","last_active":"2023-08-25T09:50:36.658199+0000","last_peered":"2023-08-25T09:50:36.658199+0000","last_clean":"2023-08-25T09:50:36.658199+0000","last_became_active":"2023-08-25T09:50:34.522293+0000","last_became_peered":"2023-08-25T09:50:34.522293+0000","last_unstale":"2023-08-25T09:50:36.658199+0000","last_undegraded":"2023-08-25T09:50:36.658199+0000","last_fullsized":"2023-08-25T09:50:36.658199+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:48:16.439493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512065+0000","last_change":"2023-08-25T09:50:35.522642+0000","last_active":"2023-08-25T09:50:36.512065+0000","last_peered":"2023-08-25T09:50:36.512065+0000","last_clean":"2023-08-25T09:50:36.512065+0000","last_became_active":"2023-08-25T09:50:35.522367+0000","last_became_peered":"2023-08-25T09:50:35.522367+0000","last_unstale":"2023-08-25T09:50:36.512065+0000","last_undegraded":"2023-08-25T09:50:36.512065+0000","last_fullsized":"2023-08-25T09:50:36.512065+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:20:45.947016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506111+0000","last_change":"2023-08-25T09:50:34.520876+0000","last_active":"2023-08-25T09:50:36.506111+0000","last_peered":"2023-08-25T09:50:36.506111+0000","last_clean":"2023-08-25T09:50:36.506111+0000","last_became_active":"2023-08-25T09:50:34.520535+0000","last_became_peered":"2023-08-25T09:50:34.520535+0000","last_unstale":"2023-08-25T09:50:36.506111+0000","last_undegraded":"2023-08-25T09:50:36.506111+0000","last_fullsized":"2023-08-25T09:50:36.506111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:01:22.062472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657414+0000","last_change":"2023-08-25T09:50:35.531887+0000","last_active":"2023-08-25T09:50:36.657414+0000","last_peered":"2023-08-25T09:50:36.657414+0000","last_clean":"2023-08-25T09:50:36.657414+0000","last_became_active":"2023-08-25T09:50:35.530226+0000","last_became_peered":"2023-08-25T09:50:35.530226+0000","last_unstale":"2023-08-25T09:50:36.657414+0000","last_undegraded":"2023-08-25T09:50:36.657414+0000","last_fullsized":"2023-08-25T09:50:36.657414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:37.495837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659457+0000","last_change":"2023-08-25T09:50:34.523504+0000","last_active":"2023-08-25T09:50:36.659457+0000","last_peered":"2023-08-25T09:50:36.659457+0000","last_clean":"2023-08-25T09:50:36.659457+0000","last_became_active":"2023-08-25T09:50:34.522671+0000","last_became_peered":"2023-08-25T09:50:34.522671+0000","last_unstale":"2023-08-25T09:50:36.659457+0000","last_undegraded":"2023-08-25T09:50:36.659457+0000","last_fullsized":"2023-08-25T09:50:36.659457+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:54:18.858105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657183+0000","last_change":"2023-08-25T09:50:35.538181+0000","last_active":"2023-08-25T09:50:36.657183+0000","last_peered":"2023-08-25T09:50:36.657183+0000","last_clean":"2023-08-25T09:50:36.657183+0000","last_became_active":"2023-08-25T09:50:35.537924+0000","last_became_peered":"2023-08-25T09:50:35.537924+0000","last_unstale":"2023-08-25T09:50:36.657183+0000","last_undegraded":"2023-08-25T09:50:36.657183+0000","last_fullsized":"2023-08-25T09:50:36.657183+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:19:07.985209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"181'6","reported_seq":19,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657603+0000","last_change":"2023-08-25T09:50:34.527589+0000","last_active":"2023-08-25T09:50:36.657603+0000","last_peered":"2023-08-25T09:50:36.657603+0000","last_clean":"2023-08-25T09:50:36.657603+0000","last_became_active":"2023-08-25T09:50:34.527167+0000","last_became_peered":"2023-08-25T09:50:34.527167+0000","last_unstale":"2023-08-25T09:50:36.657603+0000","last_undegraded":"2023-08-25T09:50:36.657603+0000","last_fullsized":"2023-08-25T09:50:36.657603+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:11.985415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":452,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656756+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:50:36.656756+0000","last_peered":"2023-08-25T09:50:36.656756+0000","last_clean":"2023-08-25T09:50:36.656756+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:50:36.656756+0000","last_undegraded":"2023-08-25T09:50:36.656756+0000","last_fullsized":"2023-08-25T09:50:36.656756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:35.520201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505827+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:50:36.505827+0000","last_peered":"2023-08-25T09:50:36.505827+0000","last_clean":"2023-08-25T09:50:36.505827+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:50:36.505827+0000","last_undegraded":"2023-08-25T09:50:36.505827+0000","last_fullsized":"2023-08-25T09:50:36.505827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:23.838150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656189+0000","last_change":"2023-08-25T09:50:35.534639+0000","last_active":"2023-08-25T09:50:36.656189+0000","last_peered":"2023-08-25T09:50:36.656189+0000","last_clean":"2023-08-25T09:50:36.656189+0000","last_became_active":"2023-08-25T09:50:35.533785+0000","last_became_peered":"2023-08-25T09:50:35.533785+0000","last_unstale":"2023-08-25T09:50:36.656189+0000","last_undegraded":"2023-08-25T09:50:36.656189+0000","last_fullsized":"2023-08-25T09:50:36.656189+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:39.636863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657388+0000","last_change":"2023-08-25T09:50:34.527689+0000","last_active":"2023-08-25T09:50:36.657388+0000","last_peered":"2023-08-25T09:50:36.657388+0000","last_clean":"2023-08-25T09:50:36.657388+0000","last_became_active":"2023-08-25T09:50:34.527128+0000","last_became_peered":"2023-08-25T09:50:34.527128+0000","last_unstale":"2023-08-25T09:50:36.657388+0000","last_undegraded":"2023-08-25T09:50:36.657388+0000","last_fullsized":"2023-08-25T09:50:36.657388+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:58.147091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506130+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:50:36.506130+0000","last_peered":"2023-08-25T09:50:36.506130+0000","last_clean":"2023-08-25T09:50:36.506130+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:50:36.506130+0000","last_undegraded":"2023-08-25T09:50:36.506130+0000","last_fullsized":"2023-08-25T09:50:36.506130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:15.595005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658391+0000","last_change":"2023-08-25T09:50:35.535211+0000","last_active":"2023-08-25T09:50:36.658391+0000","last_peered":"2023-08-25T09:50:36.658391+0000","last_clean":"2023-08-25T09:50:36.658391+0000","last_became_active":"2023-08-25T09:50:35.535008+0000","last_became_peered":"2023-08-25T09:50:35.535008+0000","last_unstale":"2023-08-25T09:50:36.658391+0000","last_undegraded":"2023-08-25T09:50:36.658391+0000","last_fullsized":"2023-08-25T09:50:36.658391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:55:03.841472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657579+0000","last_change":"2023-08-25T09:50:34.523756+0000","last_active":"2023-08-25T09:50:36.657579+0000","last_peered":"2023-08-25T09:50:36.657579+0000","last_clean":"2023-08-25T09:50:36.657579+0000","last_became_active":"2023-08-25T09:50:34.522806+0000","last_became_peered":"2023-08-25T09:50:34.522806+0000","last_unstale":"2023-08-25T09:50:36.657579+0000","last_undegraded":"2023-08-25T09:50:36.657579+0000","last_fullsized":"2023-08-25T09:50:36.657579+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:17:57.369548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511427+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:50:36.511427+0000","last_peered":"2023-08-25T09:50:36.511427+0000","last_clean":"2023-08-25T09:50:36.511427+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:50:36.511427+0000","last_undegraded":"2023-08-25T09:50:36.511427+0000","last_fullsized":"2023-08-25T09:50:36.511427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:15:56.083282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510525+0000","last_change":"2023-08-25T09:50:35.525628+0000","last_active":"2023-08-25T09:50:36.510525+0000","last_peered":"2023-08-25T09:50:36.510525+0000","last_clean":"2023-08-25T09:50:36.510525+0000","last_became_active":"2023-08-25T09:50:35.525339+0000","last_became_peered":"2023-08-25T09:50:35.525339+0000","last_unstale":"2023-08-25T09:50:36.510525+0000","last_undegraded":"2023-08-25T09:50:36.510525+0000","last_fullsized":"2023-08-25T09:50:36.510525+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:26.134099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"181'3","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657962+0000","last_change":"2023-08-25T09:50:34.519932+0000","last_active":"2023-08-25T09:50:36.657962+0000","last_peered":"2023-08-25T09:50:36.657962+0000","last_clean":"2023-08-25T09:50:36.657962+0000","last_became_active":"2023-08-25T09:50:34.519519+0000","last_became_peered":"2023-08-25T09:50:34.519519+0000","last_unstale":"2023-08-25T09:50:36.657962+0000","last_undegraded":"2023-08-25T09:50:36.657962+0000","last_fullsized":"2023-08-25T09:50:36.657962+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:37.086484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.507432+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:50:36.507432+0000","last_peered":"2023-08-25T09:50:36.507432+0000","last_clean":"2023-08-25T09:50:36.507432+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:50:36.507432+0000","last_undegraded":"2023-08-25T09:50:36.507432+0000","last_fullsized":"2023-08-25T09:50:36.507432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:53:07.982145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656448+0000","last_change":"2023-08-25T09:50:35.533797+0000","last_active":"2023-08-25T09:50:36.656448+0000","last_peered":"2023-08-25T09:50:36.656448+0000","last_clean":"2023-08-25T09:50:36.656448+0000","last_became_active":"2023-08-25T09:50:35.533297+0000","last_became_peered":"2023-08-25T09:50:35.533297+0000","last_unstale":"2023-08-25T09:50:36.656448+0000","last_undegraded":"2023-08-25T09:50:36.656448+0000","last_fullsized":"2023-08-25T09:50:36.656448+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:48.759947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658483+0000","last_change":"2023-08-25T09:50:34.527144+0000","last_active":"2023-08-25T09:50:36.658483+0000","last_peered":"2023-08-25T09:50:36.658483+0000","last_clean":"2023-08-25T09:50:36.658483+0000","last_became_active":"2023-08-25T09:50:34.526894+0000","last_became_peered":"2023-08-25T09:50:34.526894+0000","last_unstale":"2023-08-25T09:50:36.658483+0000","last_undegraded":"2023-08-25T09:50:36.658483+0000","last_fullsized":"2023-08-25T09:50:36.658483+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:50.540210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":296,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659615+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:50:36.659615+0000","last_peered":"2023-08-25T09:50:36.659615+0000","last_clean":"2023-08-25T09:50:36.659615+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:50:36.659615+0000","last_undegraded":"2023-08-25T09:50:36.659615+0000","last_fullsized":"2023-08-25T09:50:36.659615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:06.966295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511705+0000","last_change":"2023-08-25T09:50:35.524658+0000","last_active":"2023-08-25T09:50:36.511705+0000","last_peered":"2023-08-25T09:50:36.511705+0000","last_clean":"2023-08-25T09:50:36.511705+0000","last_became_active":"2023-08-25T09:50:35.524426+0000","last_became_peered":"2023-08-25T09:50:35.524426+0000","last_unstale":"2023-08-25T09:50:36.511705+0000","last_undegraded":"2023-08-25T09:50:36.511705+0000","last_fullsized":"2023-08-25T09:50:36.511705+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:02:02.673773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658000+0000","last_change":"2023-08-25T09:50:34.528642+0000","last_active":"2023-08-25T09:50:36.658000+0000","last_peered":"2023-08-25T09:50:36.658000+0000","last_clean":"2023-08-25T09:50:36.658000+0000","last_became_active":"2023-08-25T09:50:34.528407+0000","last_became_peered":"2023-08-25T09:50:34.528407+0000","last_unstale":"2023-08-25T09:50:36.658000+0000","last_undegraded":"2023-08-25T09:50:36.658000+0000","last_fullsized":"2023-08-25T09:50:36.658000+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:46:39.388437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505723+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:50:36.505723+0000","last_peered":"2023-08-25T09:50:36.505723+0000","last_clean":"2023-08-25T09:50:36.505723+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:50:36.505723+0000","last_undegraded":"2023-08-25T09:50:36.505723+0000","last_fullsized":"2023-08-25T09:50:36.505723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:45.377747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.509947+0000","last_change":"2023-08-25T09:50:35.519573+0000","last_active":"2023-08-25T09:50:36.509947+0000","last_peered":"2023-08-25T09:50:36.509947+0000","last_clean":"2023-08-25T09:50:36.509947+0000","last_became_active":"2023-08-25T09:50:35.519267+0000","last_became_peered":"2023-08-25T09:50:35.519267+0000","last_unstale":"2023-08-25T09:50:36.509947+0000","last_undegraded":"2023-08-25T09:50:36.509947+0000","last_fullsized":"2023-08-25T09:50:36.509947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:00:19.244622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657709+0000","last_change":"2023-08-25T09:50:34.523381+0000","last_active":"2023-08-25T09:50:36.657709+0000","last_peered":"2023-08-25T09:50:36.657709+0000","last_clean":"2023-08-25T09:50:36.657709+0000","last_became_active":"2023-08-25T09:50:34.522247+0000","last_became_peered":"2023-08-25T09:50:34.522247+0000","last_unstale":"2023-08-25T09:50:36.657709+0000","last_undegraded":"2023-08-25T09:50:36.657709+0000","last_fullsized":"2023-08-25T09:50:36.657709+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:13:34.836401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":294,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657992+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:50:36.657992+0000","last_peered":"2023-08-25T09:50:36.657992+0000","last_clean":"2023-08-25T09:50:36.657992+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:50:36.657992+0000","last_undegraded":"2023-08-25T09:50:36.657992+0000","last_fullsized":"2023-08-25T09:50:36.657992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:10:51.199339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657659+0000","last_change":"2023-08-25T09:50:35.518017+0000","last_active":"2023-08-25T09:50:36.657659+0000","last_peered":"2023-08-25T09:50:36.657659+0000","last_clean":"2023-08-25T09:50:36.657659+0000","last_became_active":"2023-08-25T09:50:35.517632+0000","last_became_peered":"2023-08-25T09:50:35.517632+0000","last_unstale":"2023-08-25T09:50:36.657659+0000","last_undegraded":"2023-08-25T09:50:36.657659+0000","last_fullsized":"2023-08-25T09:50:36.657659+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:20:09.606894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"181'8","reported_seq":21,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656809+0000","last_change":"2023-08-25T09:50:34.529284+0000","last_active":"2023-08-25T09:50:36.656809+0000","last_peered":"2023-08-25T09:50:36.656809+0000","last_clean":"2023-08-25T09:50:36.656809+0000","last_became_active":"2023-08-25T09:50:34.528893+0000","last_became_peered":"2023-08-25T09:50:34.528893+0000","last_unstale":"2023-08-25T09:50:36.656809+0000","last_undegraded":"2023-08-25T09:50:36.656809+0000","last_fullsized":"2023-08-25T09:50:36.656809+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:01.942785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506839+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:50:36.506839+0000","last_peered":"2023-08-25T09:50:36.506839+0000","last_clean":"2023-08-25T09:50:36.506839+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:50:36.506839+0000","last_undegraded":"2023-08-25T09:50:36.506839+0000","last_fullsized":"2023-08-25T09:50:36.506839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:04:35.651763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908011,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.131,"15min":1.113},"min":{"1min":0.76300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6279999999999999,"5min":1.738,"15min":10.747},"last":1.829},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.137,"15min":1.089},"min":{"1min":0.85699999999999998,"5min":0.61699999999999999,"15min":0.43099999999999999},"max":{"1min":1.639,"5min":1.651,"15min":1.855},"last":0.69699999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.383,"15min":1.2270000000000001},"min":{"1min":0.94699999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.823,"5min":11.010999999999999,"15min":11.010999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.25,"15min":1.165},"min":{"1min":0.97699999999999998,"5min":0.70899999999999996,"15min":0.57999999999999996},"max":{"1min":1.6279999999999999,"5min":1.7370000000000001,"15min":1.7769999999999999},"last":1.363}]},{"osd":2,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.1719999999999999,"15min":1.129},"min":{"1min":0.90800000000000003,"5min":0.66800000000000004,"15min":0.50600000000000001},"max":{"1min":1.552,"5min":1.5529999999999999,"15min":1.619},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.2849999999999999,"15min":1.262},"min":{"1min":0.91200000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.98},"last":1.6020000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.236,"15min":1.2010000000000001},"min":{"1min":0.79800000000000004,"5min":0.59799999999999998,"15min":0.56799999999999995},"max":{"1min":1.665,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.373,"15min":1.359},"min":{"1min":1.0229999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.833,"5min":1.833,"15min":2.1030000000000002},"last":1.7230000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.286,"15min":1.2869999999999999},"min":{"1min":0.91400000000000003,"5min":0.51300000000000001,"15min":0.48599999999999999},"max":{"1min":1.845,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.5},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.90600000000000003,"5min":0.53800000000000003,"15min":0.46300000000000002},"max":{"1min":1.6060000000000001,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4370000000000001,"15min":1.399},"min":{"1min":0.91100000000000003,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":1.919,"5min":1.919,"15min":2.2389999999999999},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.472,"15min":1.4219999999999999},"min":{"1min":1.0760000000000001,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.73,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.907}]},{"osd":6,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.5589999999999999,"15min":1.5009999999999999},"min":{"1min":1.2589999999999999,"5min":0.72999999999999998,"15min":0.68799999999999994},"max":{"1min":1.8700000000000001,"5min":2.0259999999999998,"15min":2.1720000000000002},"last":1.788},{"interface":"front","average":{"1min":1.419,"5min":1.4850000000000001,"15min":1.448},"min":{"1min":1.163,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":1.7370000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.4139999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908053,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1399999999999999,"15min":1.1479999999999999},"min":{"1min":0.79900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8029999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1759999999999999,"15min":1.167},"min":{"1min":0.86399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.645,"5min":1.992,"15min":5.5410000000000004},"last":1.042}]},{"osd":1,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2190000000000001,"15min":1.21},"min":{"1min":0.91200000000000003,"5min":0.72699999999999998,"15min":0.54700000000000004},"max":{"1min":1.6359999999999999,"5min":2.198,"15min":5.6799999999999997},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.196,"15min":1.167},"min":{"1min":0.80400000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7509999999999999,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.169,"15min":1.1419999999999999},"min":{"1min":0.92800000000000005,"5min":0.63400000000000001,"15min":0.60399999999999998},"max":{"1min":1.581,"5min":1.631,"15min":1.708},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.2989999999999999,"15min":1.278},"min":{"1min":0.95199999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7889999999999999,"5min":1.7969999999999999,"15min":13.193},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.337,"15min":1.286},"min":{"1min":0.97499999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.718,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2729999999999999,"15min":1.266},"min":{"1min":0.96399999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5509999999999999,"5min":1.8660000000000001,"15min":1.9239999999999999},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.357,"15min":1.304},"min":{"1min":0.90500000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7250000000000001,"5min":2.1110000000000002,"15min":2.484},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.393,"15min":1.3620000000000001},"min":{"1min":0.95099999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.764,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3540000000000001,"15min":1.3149999999999999},"min":{"1min":0.95299999999999996,"5min":0.65000000000000002,"15min":0.59599999999999997},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.0049999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4890000000000001,"15min":1.4379999999999999},"min":{"1min":1.153,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.1890000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5429999999999999,"15min":1.5229999999999999},"min":{"1min":1.145,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":1.893,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.399},{"interface":"front","average":{"1min":1.512,"5min":1.5229999999999999,"15min":1.4930000000000001},"min":{"1min":1.071,"5min":0.74299999999999999,"15min":0.57399999999999995},"max":{"1min":1.996,"5min":1.996,"15min":2.1890000000000001},"last":0.94599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193667,"statfs":{"total":200000000,"available":198315471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.252,"15min":1.1910000000000001},"min":{"1min":0.874,"5min":0.63,"15min":0.497},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.081},"last":1.329},{"interface":"front","average":{"1min":1.278,"5min":1.244,"15min":1.2210000000000001},"min":{"1min":1.026,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.6950000000000001,"5min":1.8080000000000001,"15min":1.974},"last":1.3939999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.92800000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6799999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1759999999999999,"15min":1.153},"min":{"1min":0.86199999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6499999999999999,"5min":1.704,"15min":2.1869999999999998},"last":1.4390000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2829999999999999,"15min":1.2310000000000001},"min":{"1min":1.0509999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.756,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":0.875},{"interface":"front","average":{"1min":1.341,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.92300000000000004,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.847,"5min":1.847,"15min":1.962},"last":0.96299999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.204,"15min":1.29},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7649999999999999,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.222},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.3180000000000001},"min":{"1min":1.145,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7490000000000001,"5min":1.964,"15min":1.9650000000000001},"last":1.288}]},{"osd":4,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2869999999999999,"15min":1.3100000000000001},"min":{"1min":0.96199999999999997,"5min":0.58299999999999996,"15min":0.55500000000000005},"max":{"1min":1.6319999999999999,"5min":1.921,"15min":2.625},"last":1.157},{"interface":"front","average":{"1min":1.266,"5min":1.292,"15min":1.3440000000000001},"min":{"1min":0.89200000000000002,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":1.6739999999999999,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.05}]},{"osd":6,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.45,"15min":1.444},"min":{"1min":1.0349999999999999,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":1.9339999999999999,"5min":2.048,"15min":2.4820000000000002},"last":1.484},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4850000000000001,"15min":1.466},"min":{"1min":1.075,"5min":0.65500000000000003,"15min":0.60399999999999998},"max":{"1min":1.891,"5min":2.044,"15min":2.5539999999999998},"last":1.593}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.466,"15min":1.46},"min":{"1min":1.157,"5min":0.80800000000000005,"15min":0.63},"max":{"1min":2.0529999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.516},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5680000000000001,"15min":1.581},"min":{"1min":1.0289999999999999,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":1.9970000000000001,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.6419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194245,"statfs":{"total":200000000,"available":198907507,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.2130000000000001},"min":{"1min":0.88900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.54,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.177},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2869999999999999,"15min":1.232},"min":{"1min":1.0669999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.605,"5min":2.5150000000000001,"15min":7.367},"last":1.446}]},{"osd":1,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.84399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5760000000000001,"5min":2.032,"15min":2.032},"last":1.254},{"interface":"front","average":{"1min":1.226,"5min":1.2789999999999999,"15min":1.3160000000000001},"min":{"1min":0.82399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.732,"5min":4.351,"15min":8.5850000000000009},"last":0.997}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.198,"15min":1.242},"min":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5669999999999999,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.079},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.248},"min":{"1min":0.92700000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.831,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.93000000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5429999999999999,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.401,"15min":1.381},"min":{"1min":1.218,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6830000000000001,"5min":1.899,"15min":1.911},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.347,"15min":1.3049999999999999},"min":{"1min":0.82499999999999996,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.8819999999999999,"5min":5.048,"15min":5.048},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3220000000000001,"15min":1.296},"min":{"1min":0.86499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.659,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.645}]},{"osd":6,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4550000000000001,"15min":1.4399999999999999},"min":{"1min":0.86399999999999999,"5min":0.68400000000000005,"15min":0.52000000000000002},"max":{"1min":1.7410000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.653,"15min":1.5860000000000001},"min":{"1min":0.89600000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9410000000000001,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.53}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5,"15min":1.476},"min":{"1min":1.133,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.7230000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.611,"15min":1.601},"min":{"1min":1.2789999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8,"5min":3.831,"15min":3.831},"last":1.6839999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1067,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194244,"statfs":{"total":200000000,"available":198906511,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96999999999999997,"15min":0.89900000000000002},"min":{"1min":0.72799999999999998,"5min":0.41099999999999998,"15min":0.29299999999999998},"max":{"1min":1.569,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.151,"5min":1.022,"15min":0.96199999999999997},"min":{"1min":0.90500000000000003,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.4610000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.87,"15min":0.84499999999999997},"min":{"1min":0.76800000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.286,"5min":1.702,"15min":1.702},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.0289999999999999,"15min":1.0049999999999999},"min":{"1min":0.86099999999999999,"5min":0.52800000000000002,"15min":0.45700000000000002},"max":{"1min":1.5820000000000001,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":1.4450000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":0.99199999999999999,"15min":0.93899999999999995},"min":{"1min":0.79800000000000004,"5min":0.44600000000000001,"15min":0.378},"max":{"1min":1.623,"5min":1.7450000000000001,"15min":1.919},"last":1.226},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.1930000000000001,"15min":1.1419999999999999},"min":{"1min":0.90100000000000002,"5min":0.47399999999999998,"15min":0.44900000000000001},"max":{"1min":1.792,"5min":2.0870000000000002,"15min":3.399},"last":1.173}]},{"osd":4,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.252,"15min":1.222},"min":{"1min":1.1870000000000001,"5min":0.73299999999999998,"15min":0.66800000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2130000000000001,"15min":1.2110000000000001},"min":{"1min":1.069,"5min":0.73299999999999998,"15min":0.65700000000000003},"max":{"1min":1.79,"5min":1.984,"15min":2.1840000000000002},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.397,"15min":1.3440000000000001},"min":{"1min":1.196,"5min":0.86899999999999999,"15min":0.71899999999999997},"max":{"1min":1.6890000000000001,"5min":2.407,"15min":2.407},"last":1.331},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.319,"15min":1.2949999999999999},"min":{"1min":1.1319999999999999,"5min":0.81599999999999995,"15min":0.64500000000000002},"max":{"1min":1.7509999999999999,"5min":1.895,"15min":1.895},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.294,"15min":1.272},"min":{"1min":0.96899999999999997,"5min":0.85199999999999998,"15min":0.73799999999999999},"max":{"1min":1.875,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.282},{"interface":"front","average":{"1min":1.587,"5min":1.4339999999999999,"15min":1.417},"min":{"1min":1.2490000000000001,"5min":0.83199999999999996,"15min":0.82199999999999995},"max":{"1min":2.0960000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.3919999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.411,"15min":1.391},"min":{"1min":1.3160000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.905,"5min":1.9910000000000001,"15min":2.2989999999999999},"last":1.367},{"interface":"front","average":{"1min":1.546,"5min":1.4139999999999999,"15min":1.363},"min":{"1min":1.28,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":1.9350000000000001,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.476}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908123,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.70799999999999996,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.202,"5min":1.9279999999999999,"15min":2.383},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.92800000000000005,"15min":0.99399999999999999},"min":{"1min":0.70099999999999996,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.466,"5min":1.645,"15min":2.7309999999999999},"last":1.05}]},{"osd":1,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.78600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.335,"5min":1.978,"15min":2.3180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.0369999999999999,"15min":1.048},"min":{"1min":0.91600000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.704,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.131,"15min":1.077},"min":{"1min":0.79900000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.3779999999999999,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.204,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.96799999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7390000000000001,"5min":1.8999999999999999,"15min":2.2040000000000002},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.2829999999999999,"15min":1.341},"min":{"1min":1.1759999999999999,"5min":0.74199999999999999,"15min":0.72099999999999997},"max":{"1min":1.8440000000000001,"5min":2.0059999999999998,"15min":3.0150000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":1.1240000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5589999999999999,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.2769999999999999,"15min":1.3140000000000001},"min":{"1min":1.0629999999999999,"5min":0.73199999999999998,"15min":0.70799999999999996},"max":{"1min":1.7290000000000001,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":1.21},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.341,"15min":1.375},"min":{"1min":1.113,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.647,"5min":2.1179999999999999,"15min":2.4220000000000002},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.4139999999999999,"15min":1.411},"min":{"1min":1.2,"5min":0.77700000000000002,"15min":0.73199999999999998},"max":{"1min":1.744,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.4279999999999999,"15min":1.4450000000000001},"min":{"1min":1.3149999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.718,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3819999999999999,"15min":1.4119999999999999},"min":{"1min":1.2529999999999999,"5min":0.77000000000000002,"15min":0.65700000000000003},"max":{"1min":1.7589999999999999,"5min":2.331,"15min":2.968},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.4159999999999999,"15min":1.458},"min":{"1min":1.034,"5min":0.76500000000000001,"15min":0.70499999999999996},"max":{"1min":1.7789999999999999,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193669,"statfs":{"total":200000000,"available":198317733,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.81299999999999994},"min":{"1min":0.63500000000000001,"5min":0.36499999999999999,"15min":0.29699999999999999},"max":{"1min":1.401,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.97599999999999998,"15min":0.874},"min":{"1min":0.72099999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.4019999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.21}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96299999999999997,"15min":0.874},"min":{"1min":0.72899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4490000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.872},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.996,"15min":0.95399999999999996},"min":{"1min":0.85899999999999999,"5min":0.46000000000000002,"15min":0.33600000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.746},"last":0.98899999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.135,"15min":1.095},"min":{"1min":0.94599999999999995,"5min":0.50800000000000001,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.8680000000000001,"15min":1.9450000000000001},"last":1.254},{"interface":"front","average":{"1min":1.163,"5min":1.071,"15min":1.004},"min":{"1min":0.89100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.542,"5min":1.655,"15min":1.764},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.1359999999999999},"min":{"1min":0.97499999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6519999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.2749999999999999,"15min":1.26},"min":{"1min":1.0409999999999999,"5min":0.72499999999999998,"15min":0.629},"max":{"1min":1.651,"5min":1.722,"15min":2.008},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.274,"15min":1.2310000000000001},"min":{"1min":1.0669999999999999,"5min":0.70499999999999996,"15min":0.68600000000000005},"max":{"1min":1.5860000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.341},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.298,"15min":1.2290000000000001},"min":{"1min":1.0960000000000001,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.6970000000000001,"5min":2.867,"15min":2.867},"last":1.24}]},{"osd":6,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3720000000000001,"15min":1.349},"min":{"1min":1.0640000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8,"5min":2.1339999999999999,"15min":2.9940000000000002},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.411,"15min":1.351},"min":{"1min":1.214,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.7789999999999999,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4350000000000001,"15min":1.3979999999999999},"min":{"1min":1.1559999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.788,"5min":1.9630000000000001,"15min":2.9529999999999998},"last":1.486},{"interface":"front","average":{"1min":1.48,"5min":1.456,"15min":1.383},"min":{"1min":1.069,"5min":0.86099999999999999,"15min":0.81899999999999995},"max":{"1min":1.6990000000000001,"5min":2.218,"15min":2.218},"last":1.29}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908019,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.91100000000000003,"15min":0.88800000000000001},"min":{"1min":0.68600000000000005,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":1.52,"5min":2.2759999999999998,"15min":11.257},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.109,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.72899999999999998,"5min":0.41099999999999998,"15min":0.32800000000000001},"max":{"1min":1.621,"5min":1.829,"15min":6.4260000000000002},"last":0.63900000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.008,"15min":0.95799999999999996},"min":{"1min":0.83799999999999997,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.482,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.081,"15min":1.0669999999999999},"min":{"1min":0.90700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.714,"5min":1.714,"15min":3.4889999999999999},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.96599999999999997,"15min":0.92300000000000004},"min":{"1min":0.77300000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.4159999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.145,"15min":1.119},"min":{"1min":0.96599999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5469999999999999,"5min":1.825,"15min":2.3809999999999998},"last":1.4330000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.28,"15min":1.252},"min":{"1min":0.95399999999999996,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.6930000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.411,"15min":1.3720000000000001},"min":{"1min":1.3240000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":2.0339999999999998,"15min":2.5499999999999998},"last":1.599}]},{"osd":5,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.325,"15min":1.323},"min":{"1min":1.151,"5min":0.73599999999999999,"15min":0.68000000000000005},"max":{"1min":1.7390000000000001,"5min":2.0939999999999999,"15min":3.3769999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.282,"15min":1.2849999999999999},"min":{"1min":1.1879999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5640000000000001,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.222}]},{"osd":6,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.403,"15min":1.357},"min":{"1min":1.083,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8100000000000001,"5min":2.1480000000000001,"15min":2.6869999999999998},"last":1.518},{"interface":"front","average":{"1min":1.502,"5min":1.3899999999999999,"15min":1.329},"min":{"1min":1.2490000000000001,"5min":0.76300000000000001,"15min":0.72699999999999998},"max":{"1min":1.829,"5min":2.2090000000000001,"15min":2.331},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4339999999999999,"15min":1.409},"min":{"1min":1.2090000000000001,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.8859999999999999,"5min":2.4590000000000001,"15min":2.6429999999999998},"last":1.401},{"interface":"front","average":{"1min":1.429,"5min":1.363,"15min":1.3340000000000001},"min":{"1min":1.1359999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.776,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.2769999999999999}]}]}],"pool_statfs":[]}} 2023-08-25T09:50:40.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:50:40.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.263+0000 7f35d9be4640 1 Processor -- start 2023-08-25T09:50:40.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.263+0000 7f35d9be4640 1 -- start start 2023-08-25T09:50:40.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d9be4640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35d40cadb0 0x7f35d40cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d9be4640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40b2aa0 con 0x7f35d415d3d0 2023-08-25T09:50:40.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d9be4640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40b4450 con 0x7f35d4159840 2023-08-25T09:50:40.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d9be4640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40c8b60 con 0x7f35d40cadb0 2023-08-25T09:50:40.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f35d415d3d0 0x7f35d40b3d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41774/0 (socket says 172.21.15.70:41774) 2023-08-25T09:50:40.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d3fff640 1 -- 172.21.15.70:0/3999026962 learned_addr learned my addr 172.21.15.70:0/3999026962 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:40.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.265+0000 7f35d93e3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f35d40cadb0 0x7f35d40cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.266+0000 7f35d93e3640 1 -- 172.21.15.70:0/3999026962 >> v1:172.21.15.155:6789/0 conn(0x7f35d4159840 legacy=0x7f35d40b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.266+0000 7f35d93e3640 1 -- 172.21.15.70:0/3999026962 >> v1:172.21.15.70:6789/0 conn(0x7f35d415d3d0 legacy=0x7f35d40b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.266+0000 7f35d93e3640 1 -- 172.21.15.70:0/3999026962 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d415d890 con 0x7f35d40cadb0 2023-08-25T09:50:40.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.266+0000 7f35d37fe640 1 -- 172.21.15.70:0/3999026962 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569338237 0 0) 0x7f35d40b2aa0 con 0x7f35d415d3d0 2023-08-25T09:50:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.267+0000 7f35d93e3640 1 --2- 172.21.15.70:0/3999026962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d40cadb0 0x7f35d40cb1d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009d90 tx=0x7f35cc02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46eabab31d64e015 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.267+0000 7f35d37fe640 1 -- 172.21.15.70:0/3999026962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc02ccf0 con 0x7f35d40cadb0 2023-08-25T09:50:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.267+0000 7f35d37fe640 1 -- 172.21.15.70:0/3999026962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc02ce50 con 0x7f35d40cadb0 2023-08-25T09:50:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.267+0000 7f35d37fe640 1 -- 172.21.15.70:0/3999026962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc034660 con 0x7f35d40cadb0 2023-08-25T09:50:40.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 -- 172.21.15.70:0/3999026962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d40cadb0 msgr2=0x7f35d40cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 --2- 172.21.15.70:0/3999026962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d40cadb0 0x7f35d40cb1d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009d90 tx=0x7f35cc02bf40 comp rx=0 tx=0).stop 2023-08-25T09:50:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 -- 172.21.15.70:0/3999026962 shutdown_connections 2023-08-25T09:50:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 --2- 172.21.15.70:0/3999026962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d40cadb0 0x7f35d40cb1d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 -- 172.21.15.70:0/3999026962 >> 172.21.15.70:0/3999026962 conn(0x7f35d40affd0 msgr2=0x7f35d40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 -- 172.21.15.70:0/3999026962 shutdown_connections 2023-08-25T09:50:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.268+0000 7f35d9be4640 1 -- 172.21.15.70:0/3999026962 wait complete. 2023-08-25T09:50:40.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.270+0000 7f35d9be4640 1 Processor -- start 2023-08-25T09:50:40.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.270+0000 7f35d9be4640 1 -- start start 2023-08-25T09:50:40.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d9be4640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d40cadb0 0x7f35d40c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d9be4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 0x7f35d40cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d9be4640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d415d3d0 0x7f35d415c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d9be4640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d41566a0 con 0x7f35d4159840 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d9be4640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40cc360 con 0x7f35d415d3d0 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d9be4640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d41f0eb0 con 0x7f35d40cadb0 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 0x7f35d40cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 0x7f35d40cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51172/0 (socket says 172.21.15.70:51172) 2023-08-25T09:50:40.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.271+0000 7f35d3fff640 1 -- 172.21.15.70:0/2145228247 learned_addr learned my addr 172.21.15.70:0/2145228247 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d8be2640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d40cadb0 0x7f35d40c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d3fff640 1 -- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d40cadb0 msgr2=0x7f35d40c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d3fff640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d40cadb0 0x7f35d40c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d3fff640 1 -- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d415d3d0 msgr2=0x7f35d415c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d3fff640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d415d3d0 0x7f35d415c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d3fff640 1 -- 172.21.15.70:0/2145228247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009a70 con 0x7f35d4159840 2023-08-25T09:50:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d8be2640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d40cadb0 0x7f35d40c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d3fff640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 0x7f35d40cce20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c3f0 tx=0x7f35c000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c00074e0 con 0x7f35d4159840 2023-08-25T09:50:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0007640 con 0x7f35d4159840 2023-08-25T09:50:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d40cc620 con 0x7f35d4159840 2023-08-25T09:50:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0004a00 con 0x7f35d4159840 2023-08-25T09:50:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.272+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40cd360 con 0x7f35d4159840 2023-08-25T09:50:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.274+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d40ccb40 con 0x7f35d4159840 2023-08-25T09:50:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.274+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f35c0004b60 con 0x7f35d4159840 2023-08-25T09:50:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.275+0000 7f35d17fa640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.276+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35c00883e0 con 0x7f35d4159840 2023-08-25T09:50:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.276+0000 7f35d8be2640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.276+0000 7f35d8be2640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f35d40c88c0 tx=0x7f35c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:40.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.280+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f35d40ccb40 con 0x7f35d4159840 2023-08-25T09:50:40.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.464+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35d40cc1e0 con 0x7f35b806a050 2023-08-25T09:50:40.488 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.486+0000 7f35d17fa640 1 -- 172.21.15.70:0/2145228247 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302890 (secure 0 0 0) 0x7f35c8001cd0 con 0x7f35b806a050 2023-08-25T09:50:40.489 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:40.492 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:50:40.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 msgr2=0x7f35b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f35d40c88c0 tx=0x7f35c8009040 comp rx=0 tx=0).stop 2023-08-25T09:50:40.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 msgr2=0x7f35d40cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 0x7f35d40cce20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f35c000c3f0 tx=0x7f35c000c8b0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 shutdown_connections 2023-08-25T09:50:40.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f35b806a050 0x7f35b806c510 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f35d415d3d0 0x7f35d415c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f35d4159840 0x7f35d40cce20 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 --2- 172.21.15.70:0/2145228247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35d40cadb0 0x7f35d40c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 >> 172.21.15.70:0/2145228247 conn(0x7f35d40affd0 msgr2=0x7f35d40713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 shutdown_connections 2023-08-25T09:50:40.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.493+0000 7f35d9be4640 1 -- 172.21.15.70:0/2145228247 wait complete. 2023-08-25T09:50:40.512 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2023-08-25T09:50:39.562367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9688,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552811,"statfs":{"total":1600000000,"available":1590079428,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001262"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505591+0000","last_change":"2023-08-25T09:50:35.537407+0000","last_active":"2023-08-25T09:50:36.505591+0000","last_peered":"2023-08-25T09:50:36.505591+0000","last_clean":"2023-08-25T09:50:36.505591+0000","last_became_active":"2023-08-25T09:50:35.537174+0000","last_became_peered":"2023-08-25T09:50:35.537174+0000","last_unstale":"2023-08-25T09:50:36.505591+0000","last_undegraded":"2023-08-25T09:50:36.505591+0000","last_fullsized":"2023-08-25T09:50:36.505591+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:49.485164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510204+0000","last_change":"2023-08-25T09:50:35.531429+0000","last_active":"2023-08-25T09:50:36.510204+0000","last_peered":"2023-08-25T09:50:36.510204+0000","last_clean":"2023-08-25T09:50:36.510204+0000","last_became_active":"2023-08-25T09:50:35.531175+0000","last_became_peered":"2023-08-25T09:50:35.531175+0000","last_unstale":"2023-08-25T09:50:36.510204+0000","last_undegraded":"2023-08-25T09:50:36.510204+0000","last_fullsized":"2023-08-25T09:50:36.510204+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:51:42.860626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510814+0000","last_change":"2023-08-25T09:50:35.524262+0000","last_active":"2023-08-25T09:50:36.510814+0000","last_peered":"2023-08-25T09:50:36.510814+0000","last_clean":"2023-08-25T09:50:36.510814+0000","last_became_active":"2023-08-25T09:50:35.524041+0000","last_became_peered":"2023-08-25T09:50:35.524041+0000","last_unstale":"2023-08-25T09:50:36.510814+0000","last_undegraded":"2023-08-25T09:50:36.510814+0000","last_fullsized":"2023-08-25T09:50:36.510814+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:12.844946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656059+0000","last_change":"2023-08-25T09:50:35.534809+0000","last_active":"2023-08-25T09:50:36.656059+0000","last_peered":"2023-08-25T09:50:36.656059+0000","last_clean":"2023-08-25T09:50:36.656059+0000","last_became_active":"2023-08-25T09:50:35.534101+0000","last_became_peered":"2023-08-25T09:50:35.534101+0000","last_unstale":"2023-08-25T09:50:36.656059+0000","last_undegraded":"2023-08-25T09:50:36.656059+0000","last_fullsized":"2023-08-25T09:50:36.656059+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:19.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511154+0000","last_change":"2023-08-25T09:50:35.523079+0000","last_active":"2023-08-25T09:50:36.511154+0000","last_peered":"2023-08-25T09:50:36.511154+0000","last_clean":"2023-08-25T09:50:36.511154+0000","last_became_active":"2023-08-25T09:50:35.522792+0000","last_became_peered":"2023-08-25T09:50:35.522792+0000","last_unstale":"2023-08-25T09:50:36.511154+0000","last_undegraded":"2023-08-25T09:50:36.511154+0000","last_fullsized":"2023-08-25T09:50:36.511154+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:59:11.178849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658462+0000","last_change":"2023-08-25T09:50:35.519479+0000","last_active":"2023-08-25T09:50:36.658462+0000","last_peered":"2023-08-25T09:50:36.658462+0000","last_clean":"2023-08-25T09:50:36.658462+0000","last_became_active":"2023-08-25T09:50:35.519256+0000","last_became_peered":"2023-08-25T09:50:35.519256+0000","last_unstale":"2023-08-25T09:50:36.658462+0000","last_undegraded":"2023-08-25T09:50:36.658462+0000","last_fullsized":"2023-08-25T09:50:36.658462+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:51.419901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506124+0000","last_change":"2023-08-25T09:50:35.521872+0000","last_active":"2023-08-25T09:50:36.506124+0000","last_peered":"2023-08-25T09:50:36.506124+0000","last_clean":"2023-08-25T09:50:36.506124+0000","last_became_active":"2023-08-25T09:50:35.520647+0000","last_became_peered":"2023-08-25T09:50:35.520647+0000","last_unstale":"2023-08-25T09:50:36.506124+0000","last_undegraded":"2023-08-25T09:50:36.506124+0000","last_fullsized":"2023-08-25T09:50:36.506124+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:33.209821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656199+0000","last_change":"2023-08-25T09:50:35.533938+0000","last_active":"2023-08-25T09:50:36.656199+0000","last_peered":"2023-08-25T09:50:36.656199+0000","last_clean":"2023-08-25T09:50:36.656199+0000","last_became_active":"2023-08-25T09:50:35.533577+0000","last_became_peered":"2023-08-25T09:50:35.533577+0000","last_unstale":"2023-08-25T09:50:36.656199+0000","last_undegraded":"2023-08-25T09:50:36.656199+0000","last_fullsized":"2023-08-25T09:50:36.656199+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:44.154342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510352+0000","last_change":"2023-08-25T09:50:35.536587+0000","last_active":"2023-08-25T09:50:36.510352+0000","last_peered":"2023-08-25T09:50:36.510352+0000","last_clean":"2023-08-25T09:50:36.510352+0000","last_became_active":"2023-08-25T09:50:35.536283+0000","last_became_peered":"2023-08-25T09:50:35.536283+0000","last_unstale":"2023-08-25T09:50:36.510352+0000","last_undegraded":"2023-08-25T09:50:36.510352+0000","last_fullsized":"2023-08-25T09:50:36.510352+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:26:24.330633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512402+0000","last_change":"2023-08-25T09:50:35.516669+0000","last_active":"2023-08-25T09:50:36.512402+0000","last_peered":"2023-08-25T09:50:36.512402+0000","last_clean":"2023-08-25T09:50:36.512402+0000","last_became_active":"2023-08-25T09:50:35.516432+0000","last_became_peered":"2023-08-25T09:50:35.516432+0000","last_unstale":"2023-08-25T09:50:36.512402+0000","last_undegraded":"2023-08-25T09:50:36.512402+0000","last_fullsized":"2023-08-25T09:50:36.512402+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:38.092636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659026+0000","last_change":"2023-08-25T09:50:35.519309+0000","last_active":"2023-08-25T09:50:36.659026+0000","last_peered":"2023-08-25T09:50:36.659026+0000","last_clean":"2023-08-25T09:50:36.659026+0000","last_became_active":"2023-08-25T09:50:35.519052+0000","last_became_peered":"2023-08-25T09:50:35.519052+0000","last_unstale":"2023-08-25T09:50:36.659026+0000","last_undegraded":"2023-08-25T09:50:36.659026+0000","last_fullsized":"2023-08-25T09:50:36.659026+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:24.221872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657112+0000","last_change":"2023-08-25T09:50:35.531731+0000","last_active":"2023-08-25T09:50:36.657112+0000","last_peered":"2023-08-25T09:50:36.657112+0000","last_clean":"2023-08-25T09:50:36.657112+0000","last_became_active":"2023-08-25T09:50:35.531401+0000","last_became_peered":"2023-08-25T09:50:35.531401+0000","last_unstale":"2023-08-25T09:50:36.657112+0000","last_undegraded":"2023-08-25T09:50:36.657112+0000","last_fullsized":"2023-08-25T09:50:36.657112+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:45:50.225326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.509910+0000","last_change":"2023-08-25T09:50:35.515361+0000","last_active":"2023-08-25T09:50:36.509910+0000","last_peered":"2023-08-25T09:50:36.509910+0000","last_clean":"2023-08-25T09:50:36.509910+0000","last_became_active":"2023-08-25T09:50:35.515100+0000","last_became_peered":"2023-08-25T09:50:35.515100+0000","last_unstale":"2023-08-25T09:50:36.509910+0000","last_undegraded":"2023-08-25T09:50:36.509910+0000","last_fullsized":"2023-08-25T09:50:36.509910+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:56:34.123853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658448+0000","last_change":"2023-08-25T09:50:35.530046+0000","last_active":"2023-08-25T09:50:36.658448+0000","last_peered":"2023-08-25T09:50:36.658448+0000","last_clean":"2023-08-25T09:50:36.658448+0000","last_became_active":"2023-08-25T09:50:35.527466+0000","last_became_peered":"2023-08-25T09:50:35.527466+0000","last_unstale":"2023-08-25T09:50:36.658448+0000","last_undegraded":"2023-08-25T09:50:36.658448+0000","last_fullsized":"2023-08-25T09:50:36.658448+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:33.557742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658844+0000","last_change":"2023-08-25T09:50:35.530507+0000","last_active":"2023-08-25T09:50:36.658844+0000","last_peered":"2023-08-25T09:50:36.658844+0000","last_clean":"2023-08-25T09:50:36.658844+0000","last_became_active":"2023-08-25T09:50:35.530267+0000","last_became_peered":"2023-08-25T09:50:35.530267+0000","last_unstale":"2023-08-25T09:50:36.658844+0000","last_undegraded":"2023-08-25T09:50:36.658844+0000","last_fullsized":"2023-08-25T09:50:36.658844+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:01:03.485056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658562+0000","last_change":"2023-08-25T09:50:35.523707+0000","last_active":"2023-08-25T09:50:36.658562+0000","last_peered":"2023-08-25T09:50:36.658562+0000","last_clean":"2023-08-25T09:50:36.658562+0000","last_became_active":"2023-08-25T09:50:35.523483+0000","last_became_peered":"2023-08-25T09:50:35.523483+0000","last_unstale":"2023-08-25T09:50:36.658562+0000","last_undegraded":"2023-08-25T09:50:36.658562+0000","last_fullsized":"2023-08-25T09:50:36.658562+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:39.457561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657605+0000","last_change":"2023-08-25T09:50:35.528094+0000","last_active":"2023-08-25T09:50:36.657605+0000","last_peered":"2023-08-25T09:50:36.657605+0000","last_clean":"2023-08-25T09:50:36.657605+0000","last_became_active":"2023-08-25T09:50:35.527574+0000","last_became_peered":"2023-08-25T09:50:35.527574+0000","last_unstale":"2023-08-25T09:50:36.657605+0000","last_undegraded":"2023-08-25T09:50:36.657605+0000","last_fullsized":"2023-08-25T09:50:36.657605+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:07:06.557564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510886+0000","last_change":"2023-08-25T09:50:35.515892+0000","last_active":"2023-08-25T09:50:36.510886+0000","last_peered":"2023-08-25T09:50:36.510886+0000","last_clean":"2023-08-25T09:50:36.510886+0000","last_became_active":"2023-08-25T09:50:35.515626+0000","last_became_peered":"2023-08-25T09:50:35.515626+0000","last_unstale":"2023-08-25T09:50:36.510886+0000","last_undegraded":"2023-08-25T09:50:36.510886+0000","last_fullsized":"2023-08-25T09:50:36.510886+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:38.715309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505609+0000","last_change":"2023-08-25T09:50:35.525792+0000","last_active":"2023-08-25T09:50:36.505609+0000","last_peered":"2023-08-25T09:50:36.505609+0000","last_clean":"2023-08-25T09:50:36.505609+0000","last_became_active":"2023-08-25T09:50:35.522122+0000","last_became_peered":"2023-08-25T09:50:35.522122+0000","last_unstale":"2023-08-25T09:50:36.505609+0000","last_undegraded":"2023-08-25T09:50:36.505609+0000","last_fullsized":"2023-08-25T09:50:36.505609+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:59:39.522298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659490+0000","last_change":"2023-08-25T09:50:35.519299+0000","last_active":"2023-08-25T09:50:36.659490+0000","last_peered":"2023-08-25T09:50:36.659490+0000","last_clean":"2023-08-25T09:50:36.659490+0000","last_became_active":"2023-08-25T09:50:35.519010+0000","last_became_peered":"2023-08-25T09:50:35.519010+0000","last_unstale":"2023-08-25T09:50:36.659490+0000","last_undegraded":"2023-08-25T09:50:36.659490+0000","last_fullsized":"2023-08-25T09:50:36.659490+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:53.862704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659218+0000","last_change":"2023-08-25T09:50:35.535260+0000","last_active":"2023-08-25T09:50:36.659218+0000","last_peered":"2023-08-25T09:50:36.659218+0000","last_clean":"2023-08-25T09:50:36.659218+0000","last_became_active":"2023-08-25T09:50:35.534954+0000","last_became_peered":"2023-08-25T09:50:35.534954+0000","last_unstale":"2023-08-25T09:50:36.659218+0000","last_undegraded":"2023-08-25T09:50:36.659218+0000","last_fullsized":"2023-08-25T09:50:36.659218+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:19:44.831905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506440+0000","last_change":"2023-08-25T09:50:35.531609+0000","last_active":"2023-08-25T09:50:36.506440+0000","last_peered":"2023-08-25T09:50:36.506440+0000","last_clean":"2023-08-25T09:50:36.506440+0000","last_became_active":"2023-08-25T09:50:35.531231+0000","last_became_peered":"2023-08-25T09:50:35.531231+0000","last_unstale":"2023-08-25T09:50:36.506440+0000","last_undegraded":"2023-08-25T09:50:36.506440+0000","last_fullsized":"2023-08-25T09:50:36.506440+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:45:18.997018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656321+0000","last_change":"2023-08-25T09:50:35.535757+0000","last_active":"2023-08-25T09:50:36.656321+0000","last_peered":"2023-08-25T09:50:36.656321+0000","last_clean":"2023-08-25T09:50:36.656321+0000","last_became_active":"2023-08-25T09:50:35.535496+0000","last_became_peered":"2023-08-25T09:50:35.535496+0000","last_unstale":"2023-08-25T09:50:36.656321+0000","last_undegraded":"2023-08-25T09:50:36.656321+0000","last_fullsized":"2023-08-25T09:50:36.656321+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:01:59.019297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510510+0000","last_change":"2023-08-25T09:50:35.517766+0000","last_active":"2023-08-25T09:50:36.510510+0000","last_peered":"2023-08-25T09:50:36.510510+0000","last_clean":"2023-08-25T09:50:36.510510+0000","last_became_active":"2023-08-25T09:50:35.517540+0000","last_became_peered":"2023-08-25T09:50:35.517540+0000","last_unstale":"2023-08-25T09:50:36.510510+0000","last_undegraded":"2023-08-25T09:50:36.510510+0000","last_fullsized":"2023-08-25T09:50:36.510510+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:33:32.565910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505651+0000","last_change":"2023-08-25T09:50:35.517583+0000","last_active":"2023-08-25T09:50:36.505651+0000","last_peered":"2023-08-25T09:50:36.505651+0000","last_clean":"2023-08-25T09:50:36.505651+0000","last_became_active":"2023-08-25T09:50:35.517346+0000","last_became_peered":"2023-08-25T09:50:35.517346+0000","last_unstale":"2023-08-25T09:50:36.505651+0000","last_undegraded":"2023-08-25T09:50:36.505651+0000","last_fullsized":"2023-08-25T09:50:36.505651+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:24:41.394209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658174+0000","last_change":"2023-08-25T09:50:35.532787+0000","last_active":"2023-08-25T09:50:36.658174+0000","last_peered":"2023-08-25T09:50:36.658174+0000","last_clean":"2023-08-25T09:50:36.658174+0000","last_became_active":"2023-08-25T09:50:35.532531+0000","last_became_peered":"2023-08-25T09:50:35.532531+0000","last_unstale":"2023-08-25T09:50:36.658174+0000","last_undegraded":"2023-08-25T09:50:36.658174+0000","last_fullsized":"2023-08-25T09:50:36.658174+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:21:14.023646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511379+0000","last_change":"2023-08-25T09:50:35.531823+0000","last_active":"2023-08-25T09:50:36.511379+0000","last_peered":"2023-08-25T09:50:36.511379+0000","last_clean":"2023-08-25T09:50:36.511379+0000","last_became_active":"2023-08-25T09:50:35.531592+0000","last_became_peered":"2023-08-25T09:50:35.531592+0000","last_unstale":"2023-08-25T09:50:36.511379+0000","last_undegraded":"2023-08-25T09:50:36.511379+0000","last_fullsized":"2023-08-25T09:50:36.511379+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:54:30.392648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510230+0000","last_change":"2023-08-25T09:50:35.520768+0000","last_active":"2023-08-25T09:50:36.510230+0000","last_peered":"2023-08-25T09:50:36.510230+0000","last_clean":"2023-08-25T09:50:36.510230+0000","last_became_active":"2023-08-25T09:50:35.520534+0000","last_became_peered":"2023-08-25T09:50:35.520534+0000","last_unstale":"2023-08-25T09:50:36.510230+0000","last_undegraded":"2023-08-25T09:50:36.510230+0000","last_fullsized":"2023-08-25T09:50:36.510230+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:09:15.827159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656651+0000","last_change":"2023-08-25T09:50:35.534327+0000","last_active":"2023-08-25T09:50:36.656651+0000","last_peered":"2023-08-25T09:50:36.656651+0000","last_clean":"2023-08-25T09:50:36.656651+0000","last_became_active":"2023-08-25T09:50:35.533158+0000","last_became_peered":"2023-08-25T09:50:35.533158+0000","last_unstale":"2023-08-25T09:50:36.656651+0000","last_undegraded":"2023-08-25T09:50:36.656651+0000","last_fullsized":"2023-08-25T09:50:36.656651+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:40:22.636730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512229+0000","last_change":"2023-08-25T09:50:35.531744+0000","last_active":"2023-08-25T09:50:36.512229+0000","last_peered":"2023-08-25T09:50:36.512229+0000","last_clean":"2023-08-25T09:50:36.512229+0000","last_became_active":"2023-08-25T09:50:35.531508+0000","last_became_peered":"2023-08-25T09:50:35.531508+0000","last_unstale":"2023-08-25T09:50:36.512229+0000","last_undegraded":"2023-08-25T09:50:36.512229+0000","last_fullsized":"2023-08-25T09:50:36.512229+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:51:50.203775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659387+0000","last_change":"2023-08-25T09:50:35.521820+0000","last_active":"2023-08-25T09:50:36.659387+0000","last_peered":"2023-08-25T09:50:36.659387+0000","last_clean":"2023-08-25T09:50:36.659387+0000","last_became_active":"2023-08-25T09:50:35.521582+0000","last_became_peered":"2023-08-25T09:50:35.521582+0000","last_unstale":"2023-08-25T09:50:36.659387+0000","last_undegraded":"2023-08-25T09:50:36.659387+0000","last_fullsized":"2023-08-25T09:50:36.659387+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:14:18.772984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657176+0000","last_change":"2023-08-25T09:50:35.525792+0000","last_active":"2023-08-25T09:50:36.657176+0000","last_peered":"2023-08-25T09:50:36.657176+0000","last_clean":"2023-08-25T09:50:36.657176+0000","last_became_active":"2023-08-25T09:50:35.525255+0000","last_became_peered":"2023-08-25T09:50:35.525255+0000","last_unstale":"2023-08-25T09:50:36.657176+0000","last_undegraded":"2023-08-25T09:50:36.657176+0000","last_fullsized":"2023-08-25T09:50:36.657176+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:48:10.259157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659577+0000","last_change":"2023-08-25T09:50:35.527708+0000","last_active":"2023-08-25T09:50:36.659577+0000","last_peered":"2023-08-25T09:50:36.659577+0000","last_clean":"2023-08-25T09:50:36.659577+0000","last_became_active":"2023-08-25T09:50:35.527243+0000","last_became_peered":"2023-08-25T09:50:35.527243+0000","last_unstale":"2023-08-25T09:50:36.659577+0000","last_undegraded":"2023-08-25T09:50:36.659577+0000","last_fullsized":"2023-08-25T09:50:36.659577+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:39.085915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1f","version":"181'4","reported_seq":17,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657558+0000","last_change":"2023-08-25T09:50:34.520850+0000","last_active":"2023-08-25T09:50:36.657558+0000","last_peered":"2023-08-25T09:50:36.657558+0000","last_clean":"2023-08-25T09:50:36.657558+0000","last_became_active":"2023-08-25T09:50:34.520601+0000","last_became_peered":"2023-08-25T09:50:34.520601+0000","last_unstale":"2023-08-25T09:50:36.657558+0000","last_undegraded":"2023-08-25T09:50:36.657558+0000","last_fullsized":"2023-08-25T09:50:36.657558+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:51:49.197623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511432+0000","last_change":"2023-08-25T09:50:35.531143+0000","last_active":"2023-08-25T09:50:36.511432+0000","last_peered":"2023-08-25T09:50:36.511432+0000","last_clean":"2023-08-25T09:50:36.511432+0000","last_became_active":"2023-08-25T09:50:35.530915+0000","last_became_peered":"2023-08-25T09:50:35.530915+0000","last_unstale":"2023-08-25T09:50:36.511432+0000","last_undegraded":"2023-08-25T09:50:36.511432+0000","last_fullsized":"2023-08-25T09:50:36.511432+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:50:56.211336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656483+0000","last_change":"2023-08-25T09:50:34.525079+0000","last_active":"2023-08-25T09:50:36.656483+0000","last_peered":"2023-08-25T09:50:36.656483+0000","last_clean":"2023-08-25T09:50:36.656483+0000","last_became_active":"2023-08-25T09:50:34.524831+0000","last_became_peered":"2023-08-25T09:50:34.524831+0000","last_unstale":"2023-08-25T09:50:36.656483+0000","last_undegraded":"2023-08-25T09:50:36.656483+0000","last_fullsized":"2023-08-25T09:50:36.656483+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:41:23.608995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656296+0000","last_change":"2023-08-25T09:50:35.529049+0000","last_active":"2023-08-25T09:50:36.656296+0000","last_peered":"2023-08-25T09:50:36.656296+0000","last_clean":"2023-08-25T09:50:36.656296+0000","last_became_active":"2023-08-25T09:50:35.528787+0000","last_became_peered":"2023-08-25T09:50:35.528787+0000","last_unstale":"2023-08-25T09:50:36.656296+0000","last_undegraded":"2023-08-25T09:50:36.656296+0000","last_fullsized":"2023-08-25T09:50:36.656296+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:35.694521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1d","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658026+0000","last_change":"2023-08-25T09:50:34.516199+0000","last_active":"2023-08-25T09:50:36.658026+0000","last_peered":"2023-08-25T09:50:36.658026+0000","last_clean":"2023-08-25T09:50:36.658026+0000","last_became_active":"2023-08-25T09:50:34.515459+0000","last_became_peered":"2023-08-25T09:50:34.515459+0000","last_unstale":"2023-08-25T09:50:36.658026+0000","last_undegraded":"2023-08-25T09:50:36.658026+0000","last_fullsized":"2023-08-25T09:50:36.658026+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:51:43.426280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656581+0000","last_change":"2023-08-25T09:50:35.519927+0000","last_active":"2023-08-25T09:50:36.656581+0000","last_peered":"2023-08-25T09:50:36.656581+0000","last_clean":"2023-08-25T09:50:36.656581+0000","last_became_active":"2023-08-25T09:50:35.519528+0000","last_became_peered":"2023-08-25T09:50:35.519528+0000","last_unstale":"2023-08-25T09:50:36.656581+0000","last_undegraded":"2023-08-25T09:50:36.656581+0000","last_fullsized":"2023-08-25T09:50:36.656581+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:47:55.364627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1c","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511886+0000","last_change":"2023-08-25T09:50:34.526826+0000","last_active":"2023-08-25T09:50:36.511886+0000","last_peered":"2023-08-25T09:50:36.511886+0000","last_clean":"2023-08-25T09:50:36.511886+0000","last_became_active":"2023-08-25T09:50:34.526200+0000","last_became_peered":"2023-08-25T09:50:34.526200+0000","last_unstale":"2023-08-25T09:50:36.511886+0000","last_undegraded":"2023-08-25T09:50:36.511886+0000","last_fullsized":"2023-08-25T09:50:36.511886+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:22:37.241070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659089+0000","last_change":"2023-08-25T09:50:35.520980+0000","last_active":"2023-08-25T09:50:36.659089+0000","last_peered":"2023-08-25T09:50:36.659089+0000","last_clean":"2023-08-25T09:50:36.659089+0000","last_became_active":"2023-08-25T09:50:35.520355+0000","last_became_peered":"2023-08-25T09:50:35.520355+0000","last_unstale":"2023-08-25T09:50:36.659089+0000","last_undegraded":"2023-08-25T09:50:36.659089+0000","last_fullsized":"2023-08-25T09:50:36.659089+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:43:27.119175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.1b","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657459+0000","last_change":"2023-08-25T09:50:34.529984+0000","last_active":"2023-08-25T09:50:36.657459+0000","last_peered":"2023-08-25T09:50:36.657459+0000","last_clean":"2023-08-25T09:50:36.657459+0000","last_became_active":"2023-08-25T09:50:34.529286+0000","last_became_peered":"2023-08-25T09:50:34.529286+0000","last_unstale":"2023-08-25T09:50:36.657459+0000","last_undegraded":"2023-08-25T09:50:36.657459+0000","last_fullsized":"2023-08-25T09:50:36.657459+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:00:12.218056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658686+0000","last_change":"2023-08-25T09:50:35.525317+0000","last_active":"2023-08-25T09:50:36.658686+0000","last_peered":"2023-08-25T09:50:36.658686+0000","last_clean":"2023-08-25T09:50:36.658686+0000","last_became_active":"2023-08-25T09:50:35.524382+0000","last_became_peered":"2023-08-25T09:50:35.524382+0000","last_unstale":"2023-08-25T09:50:36.658686+0000","last_undegraded":"2023-08-25T09:50:36.658686+0000","last_fullsized":"2023-08-25T09:50:36.658686+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:40:27.511906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1a","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510448+0000","last_change":"2023-08-25T09:50:34.522225+0000","last_active":"2023-08-25T09:50:36.510448+0000","last_peered":"2023-08-25T09:50:36.510448+0000","last_clean":"2023-08-25T09:50:36.510448+0000","last_became_active":"2023-08-25T09:50:34.521947+0000","last_became_peered":"2023-08-25T09:50:34.521947+0000","last_unstale":"2023-08-25T09:50:36.510448+0000","last_undegraded":"2023-08-25T09:50:36.510448+0000","last_fullsized":"2023-08-25T09:50:36.510448+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:54:42.504958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657564+0000","last_change":"2023-08-25T09:50:35.524325+0000","last_active":"2023-08-25T09:50:36.657564+0000","last_peered":"2023-08-25T09:50:36.657564+0000","last_clean":"2023-08-25T09:50:36.657564+0000","last_became_active":"2023-08-25T09:50:35.523972+0000","last_became_peered":"2023-08-25T09:50:35.523972+0000","last_unstale":"2023-08-25T09:50:36.657564+0000","last_undegraded":"2023-08-25T09:50:36.657564+0000","last_fullsized":"2023-08-25T09:50:36.657564+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:52:51.584851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659265+0000","last_change":"2023-08-25T09:50:34.523651+0000","last_active":"2023-08-25T09:50:36.659265+0000","last_peered":"2023-08-25T09:50:36.659265+0000","last_clean":"2023-08-25T09:50:36.659265+0000","last_became_active":"2023-08-25T09:50:34.523010+0000","last_became_peered":"2023-08-25T09:50:34.523010+0000","last_unstale":"2023-08-25T09:50:36.659265+0000","last_undegraded":"2023-08-25T09:50:36.659265+0000","last_fullsized":"2023-08-25T09:50:36.659265+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:16:59.601307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657372+0000","last_change":"2023-08-25T09:50:35.525925+0000","last_active":"2023-08-25T09:50:36.657372+0000","last_peered":"2023-08-25T09:50:36.657372+0000","last_clean":"2023-08-25T09:50:36.657372+0000","last_became_active":"2023-08-25T09:50:35.525559+0000","last_became_peered":"2023-08-25T09:50:35.525559+0000","last_unstale":"2023-08-25T09:50:36.657372+0000","last_undegraded":"2023-08-25T09:50:36.657372+0000","last_fullsized":"2023-08-25T09:50:36.657372+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:47.712163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511549+0000","last_change":"2023-08-25T09:50:34.524845+0000","last_active":"2023-08-25T09:50:36.511549+0000","last_peered":"2023-08-25T09:50:36.511549+0000","last_clean":"2023-08-25T09:50:36.511549+0000","last_became_active":"2023-08-25T09:50:34.524122+0000","last_became_peered":"2023-08-25T09:50:34.524122+0000","last_unstale":"2023-08-25T09:50:36.511549+0000","last_undegraded":"2023-08-25T09:50:36.511549+0000","last_fullsized":"2023-08-25T09:50:36.511549+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:14.771032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.507248+0000","last_change":"2023-08-25T09:50:35.528714+0000","last_active":"2023-08-25T09:50:36.507248+0000","last_peered":"2023-08-25T09:50:36.507248+0000","last_clean":"2023-08-25T09:50:36.507248+0000","last_became_active":"2023-08-25T09:50:35.521351+0000","last_became_peered":"2023-08-25T09:50:35.521351+0000","last_unstale":"2023-08-25T09:50:36.507248+0000","last_undegraded":"2023-08-25T09:50:36.507248+0000","last_fullsized":"2023-08-25T09:50:36.507248+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:36:01.386869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.f","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510622+0000","last_change":"2023-08-25T09:50:34.516216+0000","last_active":"2023-08-25T09:50:36.510622+0000","last_peered":"2023-08-25T09:50:36.510622+0000","last_clean":"2023-08-25T09:50:36.510622+0000","last_became_active":"2023-08-25T09:50:34.515369+0000","last_became_peered":"2023-08-25T09:50:34.515369+0000","last_unstale":"2023-08-25T09:50:36.510622+0000","last_undegraded":"2023-08-25T09:50:36.510622+0000","last_fullsized":"2023-08-25T09:50:36.510622+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:33:58.487878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657301+0000","last_change":"2023-08-25T09:50:35.532194+0000","last_active":"2023-08-25T09:50:36.657301+0000","last_peered":"2023-08-25T09:50:36.657301+0000","last_clean":"2023-08-25T09:50:36.657301+0000","last_became_active":"2023-08-25T09:50:35.530499+0000","last_became_peered":"2023-08-25T09:50:35.530499+0000","last_unstale":"2023-08-25T09:50:36.657301+0000","last_undegraded":"2023-08-25T09:50:36.657301+0000","last_fullsized":"2023-08-25T09:50:36.657301+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:16:21.461028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.e","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657719+0000","last_change":"2023-08-25T09:50:34.525090+0000","last_active":"2023-08-25T09:50:36.657719+0000","last_peered":"2023-08-25T09:50:36.657719+0000","last_clean":"2023-08-25T09:50:36.657719+0000","last_became_active":"2023-08-25T09:50:34.524648+0000","last_became_peered":"2023-08-25T09:50:34.524648+0000","last_unstale":"2023-08-25T09:50:36.657719+0000","last_undegraded":"2023-08-25T09:50:36.657719+0000","last_fullsized":"2023-08-25T09:50:36.657719+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:49:49.980136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657850+0000","last_change":"2023-08-25T09:50:35.526155+0000","last_active":"2023-08-25T09:50:36.657850+0000","last_peered":"2023-08-25T09:50:36.657850+0000","last_clean":"2023-08-25T09:50:36.657850+0000","last_became_active":"2023-08-25T09:50:35.525788+0000","last_became_peered":"2023-08-25T09:50:35.525788+0000","last_unstale":"2023-08-25T09:50:36.657850+0000","last_undegraded":"2023-08-25T09:50:36.657850+0000","last_fullsized":"2023-08-25T09:50:36.657850+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:00:35.763214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.d","version":"181'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512236+0000","last_change":"2023-08-25T09:50:34.526641+0000","last_active":"2023-08-25T09:50:36.512236+0000","last_peered":"2023-08-25T09:50:36.512236+0000","last_clean":"2023-08-25T09:50:36.512236+0000","last_became_active":"2023-08-25T09:50:34.525590+0000","last_became_peered":"2023-08-25T09:50:34.525590+0000","last_unstale":"2023-08-25T09:50:36.512236+0000","last_undegraded":"2023-08-25T09:50:36.512236+0000","last_fullsized":"2023-08-25T09:50:36.512236+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:11.985415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656318+0000","last_change":"2023-08-25T09:50:35.534488+0000","last_active":"2023-08-25T09:50:36.656318+0000","last_peered":"2023-08-25T09:50:36.656318+0000","last_clean":"2023-08-25T09:50:36.656318+0000","last_became_active":"2023-08-25T09:50:35.533454+0000","last_became_peered":"2023-08-25T09:50:35.533454+0000","last_unstale":"2023-08-25T09:50:36.656318+0000","last_undegraded":"2023-08-25T09:50:36.656318+0000","last_fullsized":"2023-08-25T09:50:36.656318+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:03.639440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656824+0000","last_change":"2023-08-25T09:50:34.525109+0000","last_active":"2023-08-25T09:50:36.656824+0000","last_peered":"2023-08-25T09:50:36.656824+0000","last_clean":"2023-08-25T09:50:36.656824+0000","last_became_active":"2023-08-25T09:50:34.524875+0000","last_became_peered":"2023-08-25T09:50:34.524875+0000","last_unstale":"2023-08-25T09:50:36.656824+0000","last_undegraded":"2023-08-25T09:50:36.656824+0000","last_fullsized":"2023-08-25T09:50:36.656824+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:09.400893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505755+0000","last_change":"2023-08-25T09:50:35.525415+0000","last_active":"2023-08-25T09:50:36.505755+0000","last_peered":"2023-08-25T09:50:36.505755+0000","last_clean":"2023-08-25T09:50:36.505755+0000","last_became_active":"2023-08-25T09:50:35.525185+0000","last_became_peered":"2023-08-25T09:50:35.525185+0000","last_unstale":"2023-08-25T09:50:36.505755+0000","last_undegraded":"2023-08-25T09:50:36.505755+0000","last_fullsized":"2023-08-25T09:50:36.505755+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:36.753754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.b","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511116+0000","last_change":"2023-08-25T09:50:34.516618+0000","last_active":"2023-08-25T09:50:36.511116+0000","last_peered":"2023-08-25T09:50:36.511116+0000","last_clean":"2023-08-25T09:50:36.511116+0000","last_became_active":"2023-08-25T09:50:34.516139+0000","last_became_peered":"2023-08-25T09:50:34.516139+0000","last_unstale":"2023-08-25T09:50:36.511116+0000","last_undegraded":"2023-08-25T09:50:36.511116+0000","last_fullsized":"2023-08-25T09:50:36.511116+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:31:58.290329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658038+0000","last_change":"2023-08-25T09:50:35.520559+0000","last_active":"2023-08-25T09:50:36.658038+0000","last_peered":"2023-08-25T09:50:36.658038+0000","last_clean":"2023-08-25T09:50:36.658038+0000","last_became_active":"2023-08-25T09:50:35.520300+0000","last_became_peered":"2023-08-25T09:50:35.520300+0000","last_unstale":"2023-08-25T09:50:36.658038+0000","last_undegraded":"2023-08-25T09:50:36.658038+0000","last_fullsized":"2023-08-25T09:50:36.658038+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:46.389809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656392+0000","last_change":"2023-08-25T09:50:34.522545+0000","last_active":"2023-08-25T09:50:36.656392+0000","last_peered":"2023-08-25T09:50:36.656392+0000","last_clean":"2023-08-25T09:50:36.656392+0000","last_became_active":"2023-08-25T09:50:34.522307+0000","last_became_peered":"2023-08-25T09:50:34.522307+0000","last_unstale":"2023-08-25T09:50:36.656392+0000","last_undegraded":"2023-08-25T09:50:36.656392+0000","last_fullsized":"2023-08-25T09:50:36.656392+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:01:04.961527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506026+0000","last_change":"2023-08-25T09:50:35.525621+0000","last_active":"2023-08-25T09:50:36.506026+0000","last_peered":"2023-08-25T09:50:36.506026+0000","last_clean":"2023-08-25T09:50:36.506026+0000","last_became_active":"2023-08-25T09:50:35.521785+0000","last_became_peered":"2023-08-25T09:50:35.521785+0000","last_unstale":"2023-08-25T09:50:36.506026+0000","last_undegraded":"2023-08-25T09:50:36.506026+0000","last_fullsized":"2023-08-25T09:50:36.506026+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:37:24.308622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511703+0000","last_change":"2023-08-25T09:50:34.524688+0000","last_active":"2023-08-25T09:50:36.511703+0000","last_peered":"2023-08-25T09:50:36.511703+0000","last_clean":"2023-08-25T09:50:36.511703+0000","last_became_active":"2023-08-25T09:50:34.523666+0000","last_became_peered":"2023-08-25T09:50:34.523666+0000","last_unstale":"2023-08-25T09:50:36.511703+0000","last_undegraded":"2023-08-25T09:50:36.511703+0000","last_fullsized":"2023-08-25T09:50:36.511703+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:42:31.126738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658703+0000","last_change":"2023-08-25T09:50:35.523437+0000","last_active":"2023-08-25T09:50:36.658703+0000","last_peered":"2023-08-25T09:50:36.658703+0000","last_clean":"2023-08-25T09:50:36.658703+0000","last_became_active":"2023-08-25T09:50:35.523194+0000","last_became_peered":"2023-08-25T09:50:35.523194+0000","last_unstale":"2023-08-25T09:50:36.658703+0000","last_undegraded":"2023-08-25T09:50:36.658703+0000","last_fullsized":"2023-08-25T09:50:36.658703+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:18.738839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657502+0000","last_change":"2023-08-25T09:50:34.524307+0000","last_active":"2023-08-25T09:50:36.657502+0000","last_peered":"2023-08-25T09:50:36.657502+0000","last_clean":"2023-08-25T09:50:36.657502+0000","last_became_active":"2023-08-25T09:50:34.523860+0000","last_became_peered":"2023-08-25T09:50:34.523860+0000","last_unstale":"2023-08-25T09:50:36.657502+0000","last_undegraded":"2023-08-25T09:50:36.657502+0000","last_fullsized":"2023-08-25T09:50:36.657502+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:23:08.713308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659276+0000","last_change":"2023-08-25T09:50:35.521160+0000","last_active":"2023-08-25T09:50:36.659276+0000","last_peered":"2023-08-25T09:50:36.659276+0000","last_clean":"2023-08-25T09:50:36.659276+0000","last_became_active":"2023-08-25T09:50:35.520738+0000","last_became_peered":"2023-08-25T09:50:35.520738+0000","last_unstale":"2023-08-25T09:50:36.659276+0000","last_undegraded":"2023-08-25T09:50:36.659276+0000","last_fullsized":"2023-08-25T09:50:36.659276+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:40:14.072195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.7","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657828+0000","last_change":"2023-08-25T09:50:34.529605+0000","last_active":"2023-08-25T09:50:36.657828+0000","last_peered":"2023-08-25T09:50:36.657828+0000","last_clean":"2023-08-25T09:50:36.657828+0000","last_became_active":"2023-08-25T09:50:34.528881+0000","last_became_peered":"2023-08-25T09:50:34.528881+0000","last_unstale":"2023-08-25T09:50:36.657828+0000","last_undegraded":"2023-08-25T09:50:36.657828+0000","last_fullsized":"2023-08-25T09:50:36.657828+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:06:27.489325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658911+0000","last_change":"2023-08-25T09:50:35.527860+0000","last_active":"2023-08-25T09:50:36.658911+0000","last_peered":"2023-08-25T09:50:36.658911+0000","last_clean":"2023-08-25T09:50:36.658911+0000","last_became_active":"2023-08-25T09:50:35.525088+0000","last_became_peered":"2023-08-25T09:50:35.525088+0000","last_unstale":"2023-08-25T09:50:36.658911+0000","last_undegraded":"2023-08-25T09:50:36.658911+0000","last_fullsized":"2023-08-25T09:50:36.658911+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:11:25.624985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"182'1","reported_seq":14,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656963+0000","last_change":"2023-08-25T09:50:34.529128+0000","last_active":"2023-08-25T09:50:36.656963+0000","last_peered":"2023-08-25T09:50:36.656963+0000","last_clean":"2023-08-25T09:50:36.656963+0000","last_became_active":"2023-08-25T09:50:34.528569+0000","last_became_peered":"2023-08-25T09:50:34.528569+0000","last_unstale":"2023-08-25T09:50:36.656963+0000","last_undegraded":"2023-08-25T09:50:36.656963+0000","last_fullsized":"2023-08-25T09:50:36.656963+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:31.527354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506037+0000","last_change":"2023-08-25T09:50:35.524624+0000","last_active":"2023-08-25T09:50:36.506037+0000","last_peered":"2023-08-25T09:50:36.506037+0000","last_clean":"2023-08-25T09:50:36.506037+0000","last_became_active":"2023-08-25T09:50:35.524356+0000","last_became_peered":"2023-08-25T09:50:35.524356+0000","last_unstale":"2023-08-25T09:50:36.506037+0000","last_undegraded":"2023-08-25T09:50:36.506037+0000","last_fullsized":"2023-08-25T09:50:36.506037+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:15:42.562887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657600+0000","last_change":"2023-08-25T09:50:34.526702+0000","last_active":"2023-08-25T09:50:36.657600+0000","last_peered":"2023-08-25T09:50:36.657600+0000","last_clean":"2023-08-25T09:50:36.657600+0000","last_became_active":"2023-08-25T09:50:34.526229+0000","last_became_peered":"2023-08-25T09:50:34.526229+0000","last_unstale":"2023-08-25T09:50:36.657600+0000","last_undegraded":"2023-08-25T09:50:36.657600+0000","last_fullsized":"2023-08-25T09:50:36.657600+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:35:05.547294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510880+0000","last_change":"2023-08-25T09:50:35.512965+0000","last_active":"2023-08-25T09:50:36.510880+0000","last_peered":"2023-08-25T09:50:36.510880+0000","last_clean":"2023-08-25T09:50:36.510880+0000","last_became_active":"2023-08-25T09:50:35.512704+0000","last_became_peered":"2023-08-25T09:50:35.512704+0000","last_unstale":"2023-08-25T09:50:36.510880+0000","last_undegraded":"2023-08-25T09:50:36.510880+0000","last_fullsized":"2023-08-25T09:50:36.510880+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:11.163303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658794+0000","last_change":"2023-08-25T09:50:34.519204+0000","last_active":"2023-08-25T09:50:36.658794+0000","last_peered":"2023-08-25T09:50:36.658794+0000","last_clean":"2023-08-25T09:50:36.658794+0000","last_became_active":"2023-08-25T09:50:34.518943+0000","last_became_peered":"2023-08-25T09:50:34.518943+0000","last_unstale":"2023-08-25T09:50:36.658794+0000","last_undegraded":"2023-08-25T09:50:36.658794+0000","last_fullsized":"2023-08-25T09:50:36.658794+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:14:48.356922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659124+0000","last_change":"2023-08-25T09:50:35.529740+0000","last_active":"2023-08-25T09:50:36.659124+0000","last_peered":"2023-08-25T09:50:36.659124+0000","last_clean":"2023-08-25T09:50:36.659124+0000","last_became_active":"2023-08-25T09:50:35.527026+0000","last_became_peered":"2023-08-25T09:50:35.527026+0000","last_unstale":"2023-08-25T09:50:36.659124+0000","last_undegraded":"2023-08-25T09:50:36.659124+0000","last_fullsized":"2023-08-25T09:50:36.659124+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:56:17.064313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658221+0000","last_change":"2023-08-25T09:50:34.522459+0000","last_active":"2023-08-25T09:50:36.658221+0000","last_peered":"2023-08-25T09:50:36.658221+0000","last_clean":"2023-08-25T09:50:36.658221+0000","last_became_active":"2023-08-25T09:50:34.522116+0000","last_became_peered":"2023-08-25T09:50:34.522116+0000","last_unstale":"2023-08-25T09:50:36.658221+0000","last_undegraded":"2023-08-25T09:50:36.658221+0000","last_fullsized":"2023-08-25T09:50:36.658221+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:45.453693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656715+0000","last_change":"2023-08-25T09:50:35.519780+0000","last_active":"2023-08-25T09:50:36.656715+0000","last_peered":"2023-08-25T09:50:36.656715+0000","last_clean":"2023-08-25T09:50:36.656715+0000","last_became_active":"2023-08-25T09:50:35.519210+0000","last_became_peered":"2023-08-25T09:50:35.519210+0000","last_unstale":"2023-08-25T09:50:36.656715+0000","last_undegraded":"2023-08-25T09:50:36.656715+0000","last_fullsized":"2023-08-25T09:50:36.656715+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:08:56.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658199+0000","last_change":"2023-08-25T09:50:34.522546+0000","last_active":"2023-08-25T09:50:36.658199+0000","last_peered":"2023-08-25T09:50:36.658199+0000","last_clean":"2023-08-25T09:50:36.658199+0000","last_became_active":"2023-08-25T09:50:34.522293+0000","last_became_peered":"2023-08-25T09:50:34.522293+0000","last_unstale":"2023-08-25T09:50:36.658199+0000","last_undegraded":"2023-08-25T09:50:36.658199+0000","last_fullsized":"2023-08-25T09:50:36.658199+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:48:16.439493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.512065+0000","last_change":"2023-08-25T09:50:35.522642+0000","last_active":"2023-08-25T09:50:36.512065+0000","last_peered":"2023-08-25T09:50:36.512065+0000","last_clean":"2023-08-25T09:50:36.512065+0000","last_became_active":"2023-08-25T09:50:35.522367+0000","last_became_peered":"2023-08-25T09:50:35.522367+0000","last_unstale":"2023-08-25T09:50:36.512065+0000","last_undegraded":"2023-08-25T09:50:36.512065+0000","last_fullsized":"2023-08-25T09:50:36.512065+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:20:45.947016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506111+0000","last_change":"2023-08-25T09:50:34.520876+0000","last_active":"2023-08-25T09:50:36.506111+0000","last_peered":"2023-08-25T09:50:36.506111+0000","last_clean":"2023-08-25T09:50:36.506111+0000","last_became_active":"2023-08-25T09:50:34.520535+0000","last_became_peered":"2023-08-25T09:50:34.520535+0000","last_unstale":"2023-08-25T09:50:36.506111+0000","last_undegraded":"2023-08-25T09:50:36.506111+0000","last_fullsized":"2023-08-25T09:50:36.506111+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:01:22.062472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657414+0000","last_change":"2023-08-25T09:50:35.531887+0000","last_active":"2023-08-25T09:50:36.657414+0000","last_peered":"2023-08-25T09:50:36.657414+0000","last_clean":"2023-08-25T09:50:36.657414+0000","last_became_active":"2023-08-25T09:50:35.530226+0000","last_became_peered":"2023-08-25T09:50:35.530226+0000","last_unstale":"2023-08-25T09:50:36.657414+0000","last_undegraded":"2023-08-25T09:50:36.657414+0000","last_fullsized":"2023-08-25T09:50:36.657414+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:37.495837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659457+0000","last_change":"2023-08-25T09:50:34.523504+0000","last_active":"2023-08-25T09:50:36.659457+0000","last_peered":"2023-08-25T09:50:36.659457+0000","last_clean":"2023-08-25T09:50:36.659457+0000","last_became_active":"2023-08-25T09:50:34.522671+0000","last_became_peered":"2023-08-25T09:50:34.522671+0000","last_unstale":"2023-08-25T09:50:36.659457+0000","last_undegraded":"2023-08-25T09:50:36.659457+0000","last_fullsized":"2023-08-25T09:50:36.659457+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:54:18.858105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657183+0000","last_change":"2023-08-25T09:50:35.538181+0000","last_active":"2023-08-25T09:50:36.657183+0000","last_peered":"2023-08-25T09:50:36.657183+0000","last_clean":"2023-08-25T09:50:36.657183+0000","last_became_active":"2023-08-25T09:50:35.537924+0000","last_became_peered":"2023-08-25T09:50:35.537924+0000","last_unstale":"2023-08-25T09:50:36.657183+0000","last_undegraded":"2023-08-25T09:50:36.657183+0000","last_fullsized":"2023-08-25T09:50:36.657183+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:19:07.985209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.14","version":"181'6","reported_seq":19,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657603+0000","last_change":"2023-08-25T09:50:34.527589+0000","last_active":"2023-08-25T09:50:36.657603+0000","last_peered":"2023-08-25T09:50:36.657603+0000","last_clean":"2023-08-25T09:50:36.657603+0000","last_became_active":"2023-08-25T09:50:34.527167+0000","last_became_peered":"2023-08-25T09:50:34.527167+0000","last_unstale":"2023-08-25T09:50:36.657603+0000","last_undegraded":"2023-08-25T09:50:36.657603+0000","last_fullsized":"2023-08-25T09:50:36.657603+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:21:11.985415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":452,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656756+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:50:36.656756+0000","last_peered":"2023-08-25T09:50:36.656756+0000","last_clean":"2023-08-25T09:50:36.656756+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:50:36.656756+0000","last_undegraded":"2023-08-25T09:50:36.656756+0000","last_fullsized":"2023-08-25T09:50:36.656756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:45:35.520201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505827+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:50:36.505827+0000","last_peered":"2023-08-25T09:50:36.505827+0000","last_clean":"2023-08-25T09:50:36.505827+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:50:36.505827+0000","last_undegraded":"2023-08-25T09:50:36.505827+0000","last_fullsized":"2023-08-25T09:50:36.505827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:41:23.838150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656189+0000","last_change":"2023-08-25T09:50:35.534639+0000","last_active":"2023-08-25T09:50:36.656189+0000","last_peered":"2023-08-25T09:50:36.656189+0000","last_clean":"2023-08-25T09:50:36.656189+0000","last_became_active":"2023-08-25T09:50:35.533785+0000","last_became_peered":"2023-08-25T09:50:35.533785+0000","last_unstale":"2023-08-25T09:50:36.656189+0000","last_undegraded":"2023-08-25T09:50:36.656189+0000","last_fullsized":"2023-08-25T09:50:36.656189+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:28:39.636863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657388+0000","last_change":"2023-08-25T09:50:34.527689+0000","last_active":"2023-08-25T09:50:36.657388+0000","last_peered":"2023-08-25T09:50:36.657388+0000","last_clean":"2023-08-25T09:50:36.657388+0000","last_became_active":"2023-08-25T09:50:34.527128+0000","last_became_peered":"2023-08-25T09:50:34.527128+0000","last_unstale":"2023-08-25T09:50:36.657388+0000","last_undegraded":"2023-08-25T09:50:36.657388+0000","last_fullsized":"2023-08-25T09:50:36.657388+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:53:58.147091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506130+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:50:36.506130+0000","last_peered":"2023-08-25T09:50:36.506130+0000","last_clean":"2023-08-25T09:50:36.506130+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:50:36.506130+0000","last_undegraded":"2023-08-25T09:50:36.506130+0000","last_fullsized":"2023-08-25T09:50:36.506130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:15.595005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658391+0000","last_change":"2023-08-25T09:50:35.535211+0000","last_active":"2023-08-25T09:50:36.658391+0000","last_peered":"2023-08-25T09:50:36.658391+0000","last_clean":"2023-08-25T09:50:36.658391+0000","last_became_active":"2023-08-25T09:50:35.535008+0000","last_became_peered":"2023-08-25T09:50:35.535008+0000","last_unstale":"2023-08-25T09:50:36.658391+0000","last_undegraded":"2023-08-25T09:50:36.658391+0000","last_fullsized":"2023-08-25T09:50:36.658391+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:55:03.841472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657579+0000","last_change":"2023-08-25T09:50:34.523756+0000","last_active":"2023-08-25T09:50:36.657579+0000","last_peered":"2023-08-25T09:50:36.657579+0000","last_clean":"2023-08-25T09:50:36.657579+0000","last_became_active":"2023-08-25T09:50:34.522806+0000","last_became_peered":"2023-08-25T09:50:34.522806+0000","last_unstale":"2023-08-25T09:50:36.657579+0000","last_undegraded":"2023-08-25T09:50:36.657579+0000","last_fullsized":"2023-08-25T09:50:36.657579+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:17:57.369548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511427+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:50:36.511427+0000","last_peered":"2023-08-25T09:50:36.511427+0000","last_clean":"2023-08-25T09:50:36.511427+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:50:36.511427+0000","last_undegraded":"2023-08-25T09:50:36.511427+0000","last_fullsized":"2023-08-25T09:50:36.511427+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:15:56.083282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.510525+0000","last_change":"2023-08-25T09:50:35.525628+0000","last_active":"2023-08-25T09:50:36.510525+0000","last_peered":"2023-08-25T09:50:36.510525+0000","last_clean":"2023-08-25T09:50:36.510525+0000","last_became_active":"2023-08-25T09:50:35.525339+0000","last_became_peered":"2023-08-25T09:50:35.525339+0000","last_unstale":"2023-08-25T09:50:36.510525+0000","last_undegraded":"2023-08-25T09:50:36.510525+0000","last_fullsized":"2023-08-25T09:50:36.510525+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:26.134099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"181'3","reported_seq":16,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657962+0000","last_change":"2023-08-25T09:50:34.519932+0000","last_active":"2023-08-25T09:50:36.657962+0000","last_peered":"2023-08-25T09:50:36.657962+0000","last_clean":"2023-08-25T09:50:36.657962+0000","last_became_active":"2023-08-25T09:50:34.519519+0000","last_became_peered":"2023-08-25T09:50:34.519519+0000","last_unstale":"2023-08-25T09:50:36.657962+0000","last_undegraded":"2023-08-25T09:50:36.657962+0000","last_fullsized":"2023-08-25T09:50:36.657962+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:37.086484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.507432+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:50:36.507432+0000","last_peered":"2023-08-25T09:50:36.507432+0000","last_clean":"2023-08-25T09:50:36.507432+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:50:36.507432+0000","last_undegraded":"2023-08-25T09:50:36.507432+0000","last_fullsized":"2023-08-25T09:50:36.507432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:53:07.982145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656448+0000","last_change":"2023-08-25T09:50:35.533797+0000","last_active":"2023-08-25T09:50:36.656448+0000","last_peered":"2023-08-25T09:50:36.656448+0000","last_clean":"2023-08-25T09:50:36.656448+0000","last_became_active":"2023-08-25T09:50:35.533297+0000","last_became_peered":"2023-08-25T09:50:35.533297+0000","last_unstale":"2023-08-25T09:50:36.656448+0000","last_undegraded":"2023-08-25T09:50:36.656448+0000","last_fullsized":"2023-08-25T09:50:36.656448+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:48.759947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.13","version":"181'2","reported_seq":15,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658483+0000","last_change":"2023-08-25T09:50:34.527144+0000","last_active":"2023-08-25T09:50:36.658483+0000","last_peered":"2023-08-25T09:50:36.658483+0000","last_clean":"2023-08-25T09:50:36.658483+0000","last_became_active":"2023-08-25T09:50:34.526894+0000","last_became_peered":"2023-08-25T09:50:34.526894+0000","last_unstale":"2023-08-25T09:50:36.658483+0000","last_undegraded":"2023-08-25T09:50:36.658483+0000","last_fullsized":"2023-08-25T09:50:36.658483+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:50.540210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":296,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.659615+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:50:36.659615+0000","last_peered":"2023-08-25T09:50:36.659615+0000","last_clean":"2023-08-25T09:50:36.659615+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:50:36.659615+0000","last_undegraded":"2023-08-25T09:50:36.659615+0000","last_fullsized":"2023-08-25T09:50:36.659615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:57:06.966295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.511705+0000","last_change":"2023-08-25T09:50:35.524658+0000","last_active":"2023-08-25T09:50:36.511705+0000","last_peered":"2023-08-25T09:50:36.511705+0000","last_clean":"2023-08-25T09:50:36.511705+0000","last_became_active":"2023-08-25T09:50:35.524426+0000","last_became_peered":"2023-08-25T09:50:35.524426+0000","last_unstale":"2023-08-25T09:50:36.511705+0000","last_undegraded":"2023-08-25T09:50:36.511705+0000","last_fullsized":"2023-08-25T09:50:36.511705+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:02:02.673773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.658000+0000","last_change":"2023-08-25T09:50:34.528642+0000","last_active":"2023-08-25T09:50:36.658000+0000","last_peered":"2023-08-25T09:50:36.658000+0000","last_clean":"2023-08-25T09:50:36.658000+0000","last_became_active":"2023-08-25T09:50:34.528407+0000","last_became_peered":"2023-08-25T09:50:34.528407+0000","last_unstale":"2023-08-25T09:50:36.658000+0000","last_undegraded":"2023-08-25T09:50:36.658000+0000","last_fullsized":"2023-08-25T09:50:36.658000+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:46:39.388437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.505723+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:50:36.505723+0000","last_peered":"2023-08-25T09:50:36.505723+0000","last_clean":"2023-08-25T09:50:36.505723+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:50:36.505723+0000","last_undegraded":"2023-08-25T09:50:36.505723+0000","last_fullsized":"2023-08-25T09:50:36.505723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:23:45.377747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.509947+0000","last_change":"2023-08-25T09:50:35.519573+0000","last_active":"2023-08-25T09:50:36.509947+0000","last_peered":"2023-08-25T09:50:36.509947+0000","last_clean":"2023-08-25T09:50:36.509947+0000","last_became_active":"2023-08-25T09:50:35.519267+0000","last_became_peered":"2023-08-25T09:50:35.519267+0000","last_unstale":"2023-08-25T09:50:36.509947+0000","last_undegraded":"2023-08-25T09:50:36.509947+0000","last_fullsized":"2023-08-25T09:50:36.509947+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:00:19.244622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657709+0000","last_change":"2023-08-25T09:50:34.523381+0000","last_active":"2023-08-25T09:50:36.657709+0000","last_peered":"2023-08-25T09:50:36.657709+0000","last_clean":"2023-08-25T09:50:36.657709+0000","last_became_active":"2023-08-25T09:50:34.522247+0000","last_became_peered":"2023-08-25T09:50:34.522247+0000","last_unstale":"2023-08-25T09:50:36.657709+0000","last_undegraded":"2023-08-25T09:50:36.657709+0000","last_fullsized":"2023-08-25T09:50:36.657709+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:13:34.836401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":294,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657992+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:50:36.657992+0000","last_peered":"2023-08-25T09:50:36.657992+0000","last_clean":"2023-08-25T09:50:36.657992+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:50:36.657992+0000","last_undegraded":"2023-08-25T09:50:36.657992+0000","last_fullsized":"2023-08-25T09:50:36.657992+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:10:51.199339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":11,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.657659+0000","last_change":"2023-08-25T09:50:35.518017+0000","last_active":"2023-08-25T09:50:36.657659+0000","last_peered":"2023-08-25T09:50:36.657659+0000","last_clean":"2023-08-25T09:50:36.657659+0000","last_became_active":"2023-08-25T09:50:35.517632+0000","last_became_peered":"2023-08-25T09:50:35.517632+0000","last_unstale":"2023-08-25T09:50:36.657659+0000","last_undegraded":"2023-08-25T09:50:36.657659+0000","last_fullsized":"2023-08-25T09:50:36.657659+0000","mapping_epoch":180,"log_start":"0'0","ondisk_log_start":"0'0","created":180,"last_epoch_clean":181,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:34.480290+0000","last_clean_scrub_stamp":"2023-08-25T09:50:34.480290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:20:09.606894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.10","version":"181'8","reported_seq":21,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.656809+0000","last_change":"2023-08-25T09:50:34.529284+0000","last_active":"2023-08-25T09:50:36.656809+0000","last_peered":"2023-08-25T09:50:36.656809+0000","last_clean":"2023-08-25T09:50:36.656809+0000","last_became_active":"2023-08-25T09:50:34.528893+0000","last_became_peered":"2023-08-25T09:50:34.528893+0000","last_unstale":"2023-08-25T09:50:36.656809+0000","last_undegraded":"2023-08-25T09:50:36.656809+0000","last_fullsized":"2023-08-25T09:50:36.656809+0000","mapping_epoch":179,"log_start":"0'0","ondisk_log_start":"0'0","created":179,"last_epoch_clean":180,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:50:33.474138+0000","last_clean_scrub_stamp":"2023-08-25T09:50:33.474138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:41:01.942785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":328,"reported_epoch":182,"state":"active+clean","last_fresh":"2023-08-25T09:50:36.506839+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:50:36.506839+0000","last_peered":"2023-08-25T09:50:36.506839+0000","last_clean":"2023-08-25T09:50:36.506839+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:50:36.506839+0000","last_undegraded":"2023-08-25T09:50:36.506839+0000","last_fullsized":"2023-08-25T09:50:36.506839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:04:35.651763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908011,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.131,"15min":1.113},"min":{"1min":0.76300000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.6279999999999999,"5min":1.738,"15min":10.747},"last":1.829},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.137,"15min":1.089},"min":{"1min":0.85699999999999998,"5min":0.61699999999999999,"15min":0.43099999999999999},"max":{"1min":1.639,"5min":1.651,"15min":1.855},"last":0.69699999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.383,"15min":1.2270000000000001},"min":{"1min":0.94699999999999995,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.823,"5min":11.010999999999999,"15min":11.010999999999999},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.25,"15min":1.165},"min":{"1min":0.97699999999999998,"5min":0.70899999999999996,"15min":0.57999999999999996},"max":{"1min":1.6279999999999999,"5min":1.7370000000000001,"15min":1.7769999999999999},"last":1.363}]},{"osd":2,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.1719999999999999,"15min":1.129},"min":{"1min":0.90800000000000003,"5min":0.66800000000000004,"15min":0.50600000000000001},"max":{"1min":1.552,"5min":1.5529999999999999,"15min":1.619},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.2849999999999999,"15min":1.262},"min":{"1min":0.91200000000000003,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.873,"5min":1.873,"15min":1.98},"last":1.6020000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.236,"15min":1.2010000000000001},"min":{"1min":0.79800000000000004,"5min":0.59799999999999998,"15min":0.56799999999999995},"max":{"1min":1.665,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.373,"15min":1.359},"min":{"1min":1.0229999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.833,"5min":1.833,"15min":2.1030000000000002},"last":1.7230000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.286,"15min":1.2869999999999999},"min":{"1min":0.91400000000000003,"5min":0.51300000000000001,"15min":0.48599999999999999},"max":{"1min":1.845,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.5},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.3400000000000001,"15min":1.3140000000000001},"min":{"1min":0.90600000000000003,"5min":0.53800000000000003,"15min":0.46300000000000002},"max":{"1min":1.6060000000000001,"5min":1.8540000000000001,"15min":1.8779999999999999},"last":1.2569999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.4370000000000001,"15min":1.399},"min":{"1min":0.91100000000000003,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":1.919,"5min":1.919,"15min":2.2389999999999999},"last":2.0059999999999998},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.472,"15min":1.4219999999999999},"min":{"1min":1.0760000000000001,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.73,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.907}]},{"osd":6,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.548,"5min":1.5589999999999999,"15min":1.5009999999999999},"min":{"1min":1.2589999999999999,"5min":0.72999999999999998,"15min":0.68799999999999994},"max":{"1min":1.8700000000000001,"5min":2.0259999999999998,"15min":2.1720000000000002},"last":1.788},{"interface":"front","average":{"1min":1.419,"5min":1.4850000000000001,"15min":1.448},"min":{"1min":1.163,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":1.7370000000000001,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.4139999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908053,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1399999999999999,"15min":1.1479999999999999},"min":{"1min":0.79900000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.8029999999999999,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1759999999999999,"15min":1.167},"min":{"1min":0.86399999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.645,"5min":1.992,"15min":5.5410000000000004},"last":1.042}]},{"osd":1,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2190000000000001,"15min":1.21},"min":{"1min":0.91200000000000003,"5min":0.72699999999999998,"15min":0.54700000000000004},"max":{"1min":1.6359999999999999,"5min":2.198,"15min":5.6799999999999997},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.196,"15min":1.167},"min":{"1min":0.80400000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.7509999999999999,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.88200000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.169,"15min":1.1419999999999999},"min":{"1min":0.92800000000000005,"5min":0.63400000000000001,"15min":0.60399999999999998},"max":{"1min":1.581,"5min":1.631,"15min":1.708},"last":0.72299999999999998},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.2989999999999999,"15min":1.278},"min":{"1min":0.95199999999999996,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.7889999999999999,"5min":1.7969999999999999,"15min":13.193},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.337,"15min":1.286},"min":{"1min":0.97499999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.718,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2729999999999999,"15min":1.266},"min":{"1min":0.96399999999999997,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.5509999999999999,"5min":1.8660000000000001,"15min":1.9239999999999999},"last":0.85099999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.357,"15min":1.304},"min":{"1min":0.90500000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7250000000000001,"5min":2.1110000000000002,"15min":2.484},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.393,"15min":1.3620000000000001},"min":{"1min":0.95099999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.764,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.3460000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3540000000000001,"15min":1.3149999999999999},"min":{"1min":0.95299999999999996,"5min":0.65000000000000002,"15min":0.59599999999999997},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":2.0049999999999999},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.4890000000000001,"15min":1.4379999999999999},"min":{"1min":1.153,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.851,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.1890000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5429999999999999,"15min":1.5229999999999999},"min":{"1min":1.145,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":1.893,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.399},{"interface":"front","average":{"1min":1.512,"5min":1.5229999999999999,"15min":1.4930000000000001},"min":{"1min":1.071,"5min":0.74299999999999999,"15min":0.57399999999999995},"max":{"1min":1.996,"5min":1.996,"15min":2.1890000000000001},"last":0.94599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193667,"statfs":{"total":200000000,"available":198315471,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.252,"15min":1.1910000000000001},"min":{"1min":0.874,"5min":0.63,"15min":0.497},"max":{"1min":1.9770000000000001,"5min":1.9770000000000001,"15min":2.081},"last":1.329},{"interface":"front","average":{"1min":1.278,"5min":1.244,"15min":1.2210000000000001},"min":{"1min":1.026,"5min":0.67500000000000004,"15min":0.65400000000000003},"max":{"1min":1.6950000000000001,"5min":1.8080000000000001,"15min":1.974},"last":1.3939999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.1899999999999999,"15min":1.1799999999999999},"min":{"1min":0.92800000000000005,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.6799999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.1759999999999999,"15min":1.153},"min":{"1min":0.86199999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.6499999999999999,"5min":1.704,"15min":2.1869999999999998},"last":1.4390000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2829999999999999,"15min":1.2310000000000001},"min":{"1min":1.0509999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.756,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":0.875},{"interface":"front","average":{"1min":1.341,"5min":1.294,"15min":1.2509999999999999},"min":{"1min":0.92300000000000004,"5min":0.76400000000000001,"15min":0.72599999999999998},"max":{"1min":1.847,"5min":1.847,"15min":1.962},"last":0.96299999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.204,"15min":1.29},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7649999999999999,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.222},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3520000000000001,"15min":1.3180000000000001},"min":{"1min":1.145,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7490000000000001,"5min":1.964,"15min":1.9650000000000001},"last":1.288}]},{"osd":4,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2869999999999999,"15min":1.3100000000000001},"min":{"1min":0.96199999999999997,"5min":0.58299999999999996,"15min":0.55500000000000005},"max":{"1min":1.6319999999999999,"5min":1.921,"15min":2.625},"last":1.157},{"interface":"front","average":{"1min":1.266,"5min":1.292,"15min":1.3440000000000001},"min":{"1min":0.89200000000000002,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":1.6739999999999999,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.05}]},{"osd":6,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.45,"15min":1.444},"min":{"1min":1.0349999999999999,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":1.9339999999999999,"5min":2.048,"15min":2.4820000000000002},"last":1.484},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4850000000000001,"15min":1.466},"min":{"1min":1.075,"5min":0.65500000000000003,"15min":0.60399999999999998},"max":{"1min":1.891,"5min":2.044,"15min":2.5539999999999998},"last":1.593}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.466,"15min":1.46},"min":{"1min":1.157,"5min":0.80800000000000005,"15min":0.63},"max":{"1min":2.0529999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.516},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.5680000000000001,"15min":1.581},"min":{"1min":1.0289999999999999,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":1.9970000000000001,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.6419999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194245,"statfs":{"total":200000000,"available":198907507,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.29,"15min":1.2130000000000001},"min":{"1min":0.88900000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.54,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.177},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.2869999999999999,"15min":1.232},"min":{"1min":1.0669999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.605,"5min":2.5150000000000001,"15min":7.367},"last":1.446}]},{"osd":1,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1479999999999999,"15min":1.137},"min":{"1min":0.84399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5760000000000001,"5min":2.032,"15min":2.032},"last":1.254},{"interface":"front","average":{"1min":1.226,"5min":1.2789999999999999,"15min":1.3160000000000001},"min":{"1min":0.82399999999999995,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.732,"5min":4.351,"15min":8.5850000000000009},"last":0.997}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.198,"15min":1.242},"min":{"1min":0.71999999999999997,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.5669999999999999,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.079},{"interface":"front","average":{"1min":1.341,"5min":1.341,"15min":1.248},"min":{"1min":0.92700000000000005,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.831,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.93000000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.5429999999999999,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.401,"15min":1.381},"min":{"1min":1.218,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.6830000000000001,"5min":1.899,"15min":1.911},"last":1.4830000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.347,"15min":1.3049999999999999},"min":{"1min":0.82499999999999996,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.8819999999999999,"5min":5.048,"15min":5.048},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3220000000000001,"15min":1.296},"min":{"1min":0.86499999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.659,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.645}]},{"osd":6,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.4550000000000001,"15min":1.4399999999999999},"min":{"1min":0.86399999999999999,"5min":0.68400000000000005,"15min":0.52000000000000002},"max":{"1min":1.7410000000000001,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.653,"15min":1.5860000000000001},"min":{"1min":0.89600000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.9410000000000001,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.53}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.5,"15min":1.476},"min":{"1min":1.133,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.7230000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.7470000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.611,"15min":1.601},"min":{"1min":1.2789999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8,"5min":3.831,"15min":3.831},"last":1.6839999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1067,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194244,"statfs":{"total":200000000,"available":198906511,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.96999999999999997,"15min":0.89900000000000002},"min":{"1min":0.72799999999999998,"5min":0.41099999999999998,"15min":0.29299999999999998},"max":{"1min":1.569,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":1.151,"5min":1.022,"15min":0.96199999999999997},"min":{"1min":0.90500000000000003,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.4610000000000001,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97399999999999998,"5min":0.87,"15min":0.84499999999999997},"min":{"1min":0.76800000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.286,"5min":1.702,"15min":1.702},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.0289999999999999,"15min":1.0049999999999999},"min":{"1min":0.86099999999999999,"5min":0.52800000000000002,"15min":0.45700000000000002},"max":{"1min":1.5820000000000001,"5min":1.9119999999999999,"15min":2.3399999999999999},"last":1.4450000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":0.99199999999999999,"15min":0.93899999999999995},"min":{"1min":0.79800000000000004,"5min":0.44600000000000001,"15min":0.378},"max":{"1min":1.623,"5min":1.7450000000000001,"15min":1.919},"last":1.226},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.1930000000000001,"15min":1.1419999999999999},"min":{"1min":0.90100000000000002,"5min":0.47399999999999998,"15min":0.44900000000000001},"max":{"1min":1.792,"5min":2.0870000000000002,"15min":3.399},"last":1.173}]},{"osd":4,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4350000000000001,"5min":1.252,"15min":1.222},"min":{"1min":1.1870000000000001,"5min":0.73299999999999998,"15min":0.66800000000000004},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2130000000000001,"15min":1.2110000000000001},"min":{"1min":1.069,"5min":0.73299999999999998,"15min":0.65700000000000003},"max":{"1min":1.79,"5min":1.984,"15min":2.1840000000000002},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.397,"15min":1.3440000000000001},"min":{"1min":1.196,"5min":0.86899999999999999,"15min":0.71899999999999997},"max":{"1min":1.6890000000000001,"5min":2.407,"15min":2.407},"last":1.331},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.319,"15min":1.2949999999999999},"min":{"1min":1.1319999999999999,"5min":0.81599999999999995,"15min":0.64500000000000002},"max":{"1min":1.7509999999999999,"5min":1.895,"15min":1.895},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.294,"15min":1.272},"min":{"1min":0.96899999999999997,"5min":0.85199999999999998,"15min":0.73799999999999999},"max":{"1min":1.875,"5min":2.3450000000000002,"15min":2.3450000000000002},"last":1.282},{"interface":"front","average":{"1min":1.587,"5min":1.4339999999999999,"15min":1.417},"min":{"1min":1.2490000000000001,"5min":0.83199999999999996,"15min":0.82199999999999995},"max":{"1min":2.0960000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.3919999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:50:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.411,"15min":1.391},"min":{"1min":1.3160000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.905,"5min":1.9910000000000001,"15min":2.2989999999999999},"last":1.367},{"interface":"front","average":{"1min":1.546,"5min":1.4139999999999999,"15min":1.363},"min":{"1min":1.28,"5min":0.83599999999999997,"15min":0.68400000000000005},"max":{"1min":1.9350000000000001,"5min":2.1509999999999998,"15min":2.1509999999999998},"last":1.476}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908123,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.70799999999999996,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.202,"5min":1.9279999999999999,"15min":2.383},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0409999999999999,"5min":0.92800000000000005,"15min":0.99399999999999999},"min":{"1min":0.70099999999999996,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.466,"5min":1.645,"15min":2.7309999999999999},"last":1.05}]},{"osd":1,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.78600000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.335,"5min":1.978,"15min":2.3180000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.0369999999999999,"15min":1.048},"min":{"1min":0.91600000000000004,"5min":0.439,"15min":0.439},"max":{"1min":1.704,"5min":1.7470000000000001,"15min":2.8599999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.131,"15min":1.077},"min":{"1min":0.79900000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.3779999999999999,"5min":18.821000000000002,"15min":18.821000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.204,"5min":1.0900000000000001,"15min":1.103},"min":{"1min":0.96799999999999997,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.7390000000000001,"5min":1.8999999999999999,"15min":2.2040000000000002},"last":1.0660000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.2829999999999999,"15min":1.341},"min":{"1min":1.1759999999999999,"5min":0.74199999999999999,"15min":0.72099999999999997},"max":{"1min":1.8440000000000001,"5min":2.0059999999999998,"15min":3.0150000000000001},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.2589999999999999,"15min":1.3029999999999999},"min":{"1min":1.1240000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5589999999999999,"5min":2.4359999999999999,"15min":2.7639999999999998},"last":1.1910000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.2769999999999999,"15min":1.3140000000000001},"min":{"1min":1.0629999999999999,"5min":0.73199999999999998,"15min":0.70799999999999996},"max":{"1min":1.7290000000000001,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":1.21},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.341,"15min":1.375},"min":{"1min":1.113,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.647,"5min":2.1179999999999999,"15min":2.4220000000000002},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.4139999999999999,"15min":1.411},"min":{"1min":1.2,"5min":0.77700000000000002,"15min":0.73199999999999998},"max":{"1min":1.744,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.4279999999999999,"15min":1.4450000000000001},"min":{"1min":1.3149999999999999,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.718,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Aug 25 09:50:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3819999999999999,"15min":1.4119999999999999},"min":{"1min":1.2529999999999999,"5min":0.77000000000000002,"15min":0.65700000000000003},"max":{"1min":1.7589999999999999,"5min":2.331,"15min":2.968},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.51,"5min":1.4159999999999999,"15min":1.458},"min":{"1min":1.034,"5min":0.76500000000000001,"15min":0.70499999999999996},"max":{"1min":1.7789999999999999,"5min":2.9980000000000002,"15min":5.2990000000000004},"last":1.109}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1642,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193669,"statfs":{"total":200000000,"available":198317733,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85399999999999998,"15min":0.81299999999999994},"min":{"1min":0.63500000000000001,"5min":0.36499999999999999,"15min":0.29699999999999999},"max":{"1min":1.401,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":0.97599999999999998,"15min":0.874},"min":{"1min":0.72099999999999997,"5min":0.317,"15min":0.317},"max":{"1min":1.4019999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.21}]},{"osd":2,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.96299999999999997,"15min":0.874},"min":{"1min":0.72899999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.4490000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.872},{"interface":"front","average":{"1min":1.0780000000000001,"5min":0.996,"15min":0.95399999999999996},"min":{"1min":0.85899999999999999,"5min":0.46000000000000002,"15min":0.33600000000000002},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.746},"last":0.98899999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.135,"15min":1.095},"min":{"1min":0.94599999999999995,"5min":0.50800000000000001,"15min":0.372},"max":{"1min":1.5069999999999999,"5min":1.8680000000000001,"15min":1.9450000000000001},"last":1.254},{"interface":"front","average":{"1min":1.163,"5min":1.071,"15min":1.004},"min":{"1min":0.89100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.542,"5min":1.655,"15min":1.764},"last":1.0489999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.179,"15min":1.1359999999999999},"min":{"1min":0.97499999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6519999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.2749999999999999,"15min":1.26},"min":{"1min":1.0409999999999999,"5min":0.72499999999999998,"15min":0.629},"max":{"1min":1.651,"5min":1.722,"15min":2.008},"last":1.3700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.274,"15min":1.2310000000000001},"min":{"1min":1.0669999999999999,"5min":0.70499999999999996,"15min":0.68600000000000005},"max":{"1min":1.5860000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.341},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.298,"15min":1.2290000000000001},"min":{"1min":1.0960000000000001,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.6970000000000001,"5min":2.867,"15min":2.867},"last":1.24}]},{"osd":6,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3720000000000001,"15min":1.349},"min":{"1min":1.0640000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8,"5min":2.1339999999999999,"15min":2.9940000000000002},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.411,"15min":1.351},"min":{"1min":1.214,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.7789999999999999,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:49:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.4350000000000001,"15min":1.3979999999999999},"min":{"1min":1.1559999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"max":{"1min":1.788,"5min":1.9630000000000001,"15min":2.9529999999999998},"last":1.486},{"interface":"front","average":{"1min":1.48,"5min":1.456,"15min":1.383},"min":{"1min":1.069,"5min":0.86099999999999999,"15min":0.81899999999999995},"max":{"1min":1.6990000000000001,"5min":2.218,"15min":2.218},"last":1.29}]}]},{"osd":0,"up_from":13,"seq":55834574995,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194246,"statfs":{"total":200000000,"available":198908019,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.91100000000000003,"15min":0.88800000000000001},"min":{"1min":0.68600000000000005,"5min":0.36799999999999999,"15min":0.313},"max":{"1min":1.52,"5min":2.2759999999999998,"15min":11.257},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.109,"5min":0.96799999999999997,"15min":0.96599999999999997},"min":{"1min":0.72899999999999998,"5min":0.41099999999999998,"15min":0.32800000000000001},"max":{"1min":1.621,"5min":1.829,"15min":6.4260000000000002},"last":0.63900000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.022,"5min":1.008,"15min":0.95799999999999996},"min":{"1min":0.83799999999999997,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.482,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":1.3169999999999999},{"interface":"front","average":{"1min":1.238,"5min":1.081,"15min":1.0669999999999999},"min":{"1min":0.90700000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.714,"5min":1.714,"15min":3.4889999999999999},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.96599999999999997,"15min":0.92300000000000004},"min":{"1min":0.77300000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.4159999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.145,"15min":1.119},"min":{"1min":0.96599999999999997,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5469999999999999,"5min":1.825,"15min":2.3809999999999998},"last":1.4330000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.28,"15min":1.252},"min":{"1min":0.95399999999999996,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.6930000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.411,"15min":1.3720000000000001},"min":{"1min":1.3240000000000001,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.667,"5min":2.0339999999999998,"15min":2.5499999999999998},"last":1.599}]},{"osd":5,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.325,"15min":1.323},"min":{"1min":1.151,"5min":0.73599999999999999,"15min":0.68000000000000005},"max":{"1min":1.7390000000000001,"5min":2.0939999999999999,"15min":3.3769999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.282,"15min":1.2849999999999999},"min":{"1min":1.1879999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.5640000000000001,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.222}]},{"osd":6,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.403,"15min":1.357},"min":{"1min":1.083,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.8100000000000001,"5min":2.1480000000000001,"15min":2.6869999999999998},"last":1.518},{"interface":"front","average":{"1min":1.502,"5min":1.3899999999999999,"15min":1.329},"min":{"1min":1.2490000000000001,"5min":0.76300000000000001,"15min":0.72699999999999998},"max":{"1min":1.829,"5min":2.2090000000000001,"15min":2.331},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:50:05 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.4339999999999999,"15min":1.409},"min":{"1min":1.2090000000000001,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.8859999999999999,"5min":2.4590000000000001,"15min":2.6429999999999998},"last":1.401},{"interface":"front","average":{"1min":1.429,"5min":1.363,"15min":1.3340000000000001},"min":{"1min":1.1359999999999999,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.776,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.2769999999999999}]}]}],"pool_statfs":[]}} 2023-08-25T09:50:40.519 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:50:40.520 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:50:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.783+0000 7f7e81083640 1 Processor -- start 2023-08-25T09:50:40.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.784+0000 7f7e81083640 1 -- start start 2023-08-25T09:50:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e81083640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e7c0cd850 0x7f7e7c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e81083640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c0ce170 con 0x7f7e7c0b8510 2023-08-25T09:50:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e81083640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c0d1500 con 0x7f7e7c15cfe0 2023-08-25T09:50:40.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e81083640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c0d1310 con 0x7f7e7c0cd850 2023-08-25T09:50:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e7ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e7c0cd850 0x7f7e7c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e7ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7e7c0cd850 0x7f7e7c0cdc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51176/0 (socket says 172.21.15.70:51176) 2023-08-25T09:50:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.785+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/1523425274 learned_addr learned my addr 172.21.15.70:0/1523425274 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:40.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.786+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/1523425274 >> v1:172.21.15.155:6789/0 conn(0x7f7e7c15cfe0 legacy=0x7f7e7c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:40.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.786+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/1523425274 >> v1:172.21.15.70:6789/0 conn(0x7f7e7c0b8510 legacy=0x7f7e7c0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.786+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/1523425274 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c0d17e0 con 0x7f7e7c0cd850 2023-08-25T09:50:40.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.787+0000 7f7e7ad9d640 1 --2- 172.21.15.70:0/1523425274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0cd850 0x7f7e7c0cdc30 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51edbdadd6ca38ef server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:40.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.787+0000 7f7e79d9b640 1 -- 172.21.15.70:0/1523425274 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c02cb10 con 0x7f7e7c0cd850 2023-08-25T09:50:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.787+0000 7f7e79d9b640 1 -- 172.21.15.70:0/1523425274 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c02cc70 con 0x7f7e7c0cd850 2023-08-25T09:50:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.787+0000 7f7e79d9b640 1 -- 172.21.15.70:0/1523425274 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0343f0 con 0x7f7e7c0cd850 2023-08-25T09:50:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 -- 172.21.15.70:0/1523425274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0cd850 msgr2=0x7f7e7c0cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 --2- 172.21.15.70:0/1523425274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0cd850 0x7f7e7c0cdc30 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c02bf70 comp rx=0 tx=0).stop 2023-08-25T09:50:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 -- 172.21.15.70:0/1523425274 shutdown_connections 2023-08-25T09:50:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 --2- 172.21.15.70:0/1523425274 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0cd850 0x7f7e7c0cdc30 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 -- 172.21.15.70:0/1523425274 >> 172.21.15.70:0/1523425274 conn(0x7f7e7c069850 msgr2=0x7f7e7c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 -- 172.21.15.70:0/1523425274 shutdown_connections 2023-08-25T09:50:40.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.788+0000 7f7e81083640 1 -- 172.21.15.70:0/1523425274 wait complete. 2023-08-25T09:50:40.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.790+0000 7f7e81083640 1 Processor -- start 2023-08-25T09:50:40.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.790+0000 7f7e81083640 1 -- start start 2023-08-25T09:50:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e81083640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 0x7f7e7c1532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e81083640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0cd850 0x7f7e7c1f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e81083640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15cfe0 0x7f7e7c1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e81083640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1f4a50 con 0x7f7e7c0b8510 2023-08-25T09:50:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e81083640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1f3dd0 con 0x7f7e7c0cd850 2023-08-25T09:50:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e81083640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1f40b0 con 0x7f7e7c15cfe0 2023-08-25T09:50:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 0x7f7e7c1532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 0x7f7e7c1532b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51184/0 (socket says 172.21.15.70:51184) 2023-08-25T09:50:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/4087705683 learned_addr learned my addr 172.21.15.70:0/4087705683 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/4087705683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15cfe0 msgr2=0x7f7e7c1f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15cfe0 0x7f7e7c1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/4087705683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0cd850 msgr2=0x7f7e7c1f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:40.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0cd850 0x7f7e7c1f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.791+0000 7f7e7ad9d640 1 -- 172.21.15.70:0/4087705683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c009b30 con 0x7f7e7c0b8510 2023-08-25T09:50:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.792+0000 7f7e7ad9d640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 0x7f7e7c1532b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.792+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c03e450 con 0x7f7e7c0b8510 2023-08-25T09:50:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.792+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c03e5f0 con 0x7f7e7c0b8510 2023-08-25T09:50:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.792+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c03e9a0 con 0x7f7e7c0b8510 2023-08-25T09:50:40.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.792+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c1f4390 con 0x7f7e7c0b8510 2023-08-25T09:50:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.792+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c1f39a0 con 0x7f7e7c0b8510 2023-08-25T09:50:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.793+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c1f3b30 con 0x7f7e7c0b8510 2023-08-25T09:50:40.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.794+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7e6c03eb00 con 0x7f7e7c0b8510 2023-08-25T09:50:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.795+0000 7f7e737fe640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5c06a050 0x7f7e5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.795+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e6c0ab500 con 0x7f7e7c0b8510 2023-08-25T09:50:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.795+0000 7f7e7a59c640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5c06a050 0x7f7e5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:40.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.796+0000 7f7e7a59c640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5c06a050 0x7f7e5c06c510 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e64006fd0 tx=0x7f7e64006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:40.800+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7e6c076d00 con 0x7f7e7c0b8510 2023-08-25T09:50:41.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.055+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7e7c066fb0 con 0x7f7e7c0b8510 2023-08-25T09:50:41.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.062+0000 7f7e737fe640 1 -- 172.21.15.70:0/4087705683 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7e6c03edb0 con 0x7f7e7c0b8510 2023-08-25T09:50:41.064 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:50:41.064 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:50:41.064 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:50:41.064 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:50:41.064 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:50:41.065 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:50:41.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5c06a050 msgr2=0x7f7e5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5c06a050 0x7f7e5c06c510 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7e64006fd0 tx=0x7f7e64006bc0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 msgr2=0x7f7e7c1532b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 0x7f7e7c1532b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c009e50 tx=0x7f7e6c004a40 comp rx=0 tx=0).stop 2023-08-25T09:50:41.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 shutdown_connections 2023-08-25T09:50:41.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7e5c06a050 0x7f7e5c06c510 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7e7c15cfe0 0x7f7e7c1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7e7c0cd850 0x7f7e7c1f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 --2- 172.21.15.70:0/4087705683 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7e7c0b8510 0x7f7e7c1532b0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 >> 172.21.15.70:0/4087705683 conn(0x7f7e7c069850 msgr2=0x7f7e7c0cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:41.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 shutdown_connections 2023-08-25T09:50:41.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.065+0000 7f7e81083640 1 -- 172.21.15.70:0/4087705683 wait complete. 2023-08-25T09:50:41.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:41.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.288+0000 7fe0f135d640 1 Processor -- start 2023-08-25T09:50:41.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.288+0000 7fe0f135d640 1 -- start start 2023-08-25T09:50:41.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.290+0000 7fe0f135d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0ec0ce5b0 0x7fe0ec0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:41.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.290+0000 7fe0f135d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec0b2850 0x7fe0ec06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.290+0000 7fe0f135d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ec0ceed0 con 0x7fe0ec15aa10 2023-08-25T09:50:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.290+0000 7fe0f135d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec159d10 con 0x7fe0ec0ce5b0 2023-08-25T09:50:41.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.290+0000 7fe0f135d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15d3f0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.290+0000 7fe0eaffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0ec0ce5b0 0x7fe0ec0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec0b2850 0x7fe0ec06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:41.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eaffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe0ec0ce5b0 0x7fe0ec0ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:55696/0 (socket says 172.21.15.70:55696) 2023-08-25T09:50:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2443114657 learned_addr learned my addr 172.21.15.70:0/2443114657 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec0b2850 msgr2=0x7fe0ec06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eaffd640 1 --2- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec0b2850 0x7fe0ec06f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2443114657 >> v1:172.21.15.155:6789/0 conn(0x7fe0ec15aa10 legacy=0x7fe0ec0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.291+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2443114657 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0b2d20 con 0x7fe0ec0ce5b0 2023-08-25T09:50:41.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.293+0000 7fe0eaffd640 1 --2- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 0x7fe0ec0ce990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009870 tx=0x7fe0dc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7300d1b1a98c3004 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.293+0000 7fe0ea7fc640 1 -- 172.21.15.70:0/2443114657 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc031640 con 0x7fe0ec0ce5b0 2023-08-25T09:50:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.293+0000 7fe0ea7fc640 1 -- 172.21.15.70:0/2443114657 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc0317a0 con 0x7fe0ec0ce5b0 2023-08-25T09:50:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.293+0000 7fe0ea7fc640 1 -- 172.21.15.70:0/2443114657 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc031b30 con 0x7fe0ec0ce5b0 2023-08-25T09:50:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 -- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 msgr2=0x7fe0ec0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 0x7fe0ec0ce990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc009870 tx=0x7fe0dc02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 -- 172.21.15.70:0/2443114657 shutdown_connections 2023-08-25T09:50:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec0b2850 0x7fe0ec06f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2443114657 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 0x7fe0ec0ce990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 -- 172.21.15.70:0/2443114657 >> 172.21.15.70:0/2443114657 conn(0x7fe0ec069850 msgr2=0x7fe0ec069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 -- 172.21.15.70:0/2443114657 shutdown_connections 2023-08-25T09:50:41.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.294+0000 7fe0f135d640 1 -- 172.21.15.70:0/2443114657 wait complete. 2023-08-25T09:50:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.296+0000 7fe0f135d640 1 Processor -- start 2023-08-25T09:50:41.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.297+0000 7fe0f135d640 1 -- start start 2023-08-25T09:50:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0f135d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 0x7fe0ec0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0f135d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 0x7fe0ec0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:41.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0f135d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec15aa10 0x7fe0ec0acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0f135d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec0b1500 con 0x7fe0ec0b2850 2023-08-25T09:50:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0f135d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec14f9f0 con 0x7fe0ec0ce5b0 2023-08-25T09:50:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0f135d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec0cbec0 con 0x7fe0ec15aa10 2023-08-25T09:50:41.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 0x7fe0ec0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 0x7fe0ec0b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51196/0 (socket says 172.21.15.70:51196) 2023-08-25T09:50:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2256323384 learned_addr learned my addr 172.21.15.70:0/2256323384 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:41.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2256323384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec15aa10 msgr2=0x7fe0ec0acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec15aa10 0x7fe0ec0acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2256323384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 msgr2=0x7fe0ec0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.298+0000 7fe0eaffd640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 0x7fe0ec0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0eaffd640 1 -- 172.21.15.70:0/2256323384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc0094d0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0eaffd640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 0x7fe0ec0b0fc0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc000c00 tx=0x7fe0dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc03a8b0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc03aa10 con 0x7fe0ec0b2850 2023-08-25T09:50:41.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc03adc0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec0cc1a0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.299+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec0cc760 con 0x7fe0ec0b2850 2023-08-25T09:50:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.300+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec04e420 con 0x7fe0ec0b2850 2023-08-25T09:50:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.301+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe0dc02f050 con 0x7fe0ec0b2850 2023-08-25T09:50:41.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.302+0000 7fe0e3fff640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0cc06a050 0x7fe0cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:41.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.302+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe0dc0aa450 con 0x7fe0ec0b2850 2023-08-25T09:50:41.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.303+0000 7fe0e27fc640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0cc06a050 0x7fe0cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:41.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.303+0000 7fe0e27fc640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0cc06a050 0x7fe0cc06c510 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec158dd0 tx=0x7fe0d4007950 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:41.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.306+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe0dc075aa0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.514+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0ec0cae50 con 0x7fe0ec0b2850 2023-08-25T09:50:41.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.515+0000 7fe0e3fff640 1 -- 172.21.15.70:0/2256323384 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fe0dc0790c0 con 0x7fe0ec0b2850 2023-08-25T09:50:41.516 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:41.516 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:50:41.517 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:50:41.518 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0cc06a050 msgr2=0x7fe0cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0cc06a050 0x7fe0cc06c510 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec158dd0 tx=0x7fe0d4007950 comp rx=0 tx=0).stop 2023-08-25T09:50:41.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 msgr2=0x7fe0ec0b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:41.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 0x7fe0ec0b0fc0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc000c00 tx=0x7fe0dc0046c0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 shutdown_connections 2023-08-25T09:50:41.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe0cc06a050 0x7fe0cc06c510 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe0ec15aa10 0x7fe0ec0acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe0ec0ce5b0 0x7fe0ec0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 --2- 172.21.15.70:0/2256323384 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe0ec0b2850 0x7fe0ec0b0fc0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:41.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.517+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 >> 172.21.15.70:0/2256323384 conn(0x7fe0ec069850 msgr2=0x7fe0ec15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:41.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.518+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 shutdown_connections 2023-08-25T09:50:41.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:41.518+0000 7fe0f135d640 1 -- 172.21.15.70:0/2256323384 wait complete. 2023-08-25T09:50:41.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:50:34.994169+0000', 'modified': '2023-08-25T09:50:38.548823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9162}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9162': {'gid': 9162, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.155:6834/3137406973', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.155:6832', 'nonce': 3137406973}, {'type': 'v1', 'addr': '172.21.15.155:6834', 'nonce': 3137406973}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:50:41.943 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:50:41.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-08-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.164+0000 7f5593ccc640 1 Processor -- start 2023-08-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.164+0000 7f5593ccc640 1 -- start start 2023-08-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5593ccc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f558c15cd90 0x7f558c0718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5593ccc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c06ad20 0x7f558c06b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5593ccc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f558c0b2360 0x7f558c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5593ccc640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0cdd60 con 0x7f558c15cd90 2023-08-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5593ccc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0cc9e0 con 0x7f558c0b2360 2023-08-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5593ccc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0cccc0 con 0x7f558c06ad20 2023-08-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f558c15cd90 0x7f558c0718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f558c15cd90 0x7f558c0718e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51210/0 (socket says 172.21.15.70:51210) 2023-08-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 -- 172.21.15.70:0/1698577449 learned_addr learned my addr 172.21.15.70:0/1698577449 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 -- 172.21.15.70:0/1698577449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c06ad20 msgr2=0x7f558c06b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 --2- 172.21.15.70:0/1698577449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c06ad20 0x7f558c06b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 -- 172.21.15.70:0/1698577449 >> v2:172.21.15.155:3300/0 conn(0x7f558c0b2360 msgr2=0x7f558c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 --2- 172.21.15.70:0/1698577449 >> v2:172.21.15.155:3300/0 conn(0x7f558c0b2360 0x7f558c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.166+0000 7f5591a68640 1 -- 172.21.15.70:0/1698577449 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c0ccfa0 con 0x7f558c15cd90 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.167+0000 7f5591a68640 1 --2- 172.21.15.70:0/1698577449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0718e0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f558c06df50 tx=0x7f558802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654348dc33bd53b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.168+0000 7f5590a66640 1 -- 172.21.15.70:0/1698577449 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558802ca80 con 0x7f558c15cd90 2023-08-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.168+0000 7f5590a66640 1 -- 172.21.15.70:0/1698577449 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558802cbe0 con 0x7f558c15cd90 2023-08-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.168+0000 7f5590a66640 1 -- 172.21.15.70:0/1698577449 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55880343f0 con 0x7f558c15cd90 2023-08-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.168+0000 7f5593ccc640 1 -- 172.21.15.70:0/1698577449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 msgr2=0x7f558c0718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.168+0000 7f5593ccc640 1 --2- 172.21.15.70:0/1698577449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0718e0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f558c06df50 tx=0x7f558802bee0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 -- 172.21.15.70:0/1698577449 shutdown_connections 2023-08-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 --2- 172.21.15.70:0/1698577449 >> v2:172.21.15.155:3300/0 conn(0x7f558c0b2360 0x7f558c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 --2- 172.21.15.70:0/1698577449 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c06ad20 0x7f558c06b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 --2- 172.21.15.70:0/1698577449 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0718e0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 -- 172.21.15.70:0/1698577449 >> 172.21.15.70:0/1698577449 conn(0x7f558c06cf60 msgr2=0x7f558c06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 -- 172.21.15.70:0/1698577449 shutdown_connections 2023-08-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.169+0000 7f5593ccc640 1 -- 172.21.15.70:0/1698577449 wait complete. 2023-08-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.170+0000 7f5593ccc640 1 Processor -- start 2023-08-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 -- start start 2023-08-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c06ad20 0x7f558c0c5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0b2360 0x7f558c0c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0b5b80 con 0x7f558c15cd90 2023-08-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0b51e0 con 0x7f558c06ad20 2023-08-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.171+0000 7f5593ccc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0b54c0 con 0x7f558c0b2360 2023-08-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51220/0 (socket says 172.21.15.70:51220) 2023-08-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 -- 172.21.15.70:0/147767058 learned_addr learned my addr 172.21.15.70:0/147767058 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5591267640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0b2360 0x7f558c0c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5591a68640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c06ad20 0x7f558c0c5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 -- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0b2360 msgr2=0x7f558c0c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0b2360 0x7f558c0c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 -- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c06ad20 msgr2=0x7f558c0c5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c06ad20 0x7f558c0c5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.172+0000 7f5592269640 1 -- 172.21.15.70:0/147767058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588009b30 con 0x7f558c15cd90 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.173+0000 7f5592269640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0c62f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5584009fd0 tx=0x7f558400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.173+0000 7f55827fc640 1 -- 172.21.15.70:0/147767058 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584011830 con 0x7f558c15cd90 2023-08-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.173+0000 7f55827fc640 1 -- 172.21.15.70:0/147767058 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584011990 con 0x7f558c15cd90 2023-08-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.173+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c0b57a0 con 0x7f558c15cd90 2023-08-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.173+0000 7f55827fc640 1 -- 172.21.15.70:0/147767058 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584011d20 con 0x7f558c15cd90 2023-08-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.173+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c152950 con 0x7f558c15cd90 2023-08-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.175+0000 7f55827fc640 1 -- 172.21.15.70:0/147767058 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f558401b460 con 0x7f558c15cd90 2023-08-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.176+0000 7f55827fc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.176+0000 7f55827fc640 1 -- 172.21.15.70:0/147767058 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f55840883f0 con 0x7f558c15cd90 2023-08-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.176+0000 7f5591a68640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.177+0000 7f5591a68640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5588035000 tx=0x7f5588030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.177+0000 7f5596847740 1 Processor -- start 2023-08-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 -- start start 2023-08-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 0x564dc80d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a2130 0x564dc80cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc8015190 con 0x564dc80a5cd0 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc8012880 con 0x564dc80d2150 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.178+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dc800e170 con 0x564dc80a2130 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51226/0 (socket says 172.21.15.70:51226) 2023-08-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 -- 172.21.15.70:0/2168775697 learned_addr learned my addr 172.21.15.70:0/2168775697 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 -- 172.21.15.70:0/2168775697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a2130 msgr2=0x564dc80cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a2130 0x564dc80cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 -- 172.21.15.70:0/2168775697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 msgr2=0x564dc80d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 0x564dc80d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 -- 172.21.15.70:0/2168775697 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc800c7f0 con 0x564dc80a5cd0 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.179+0000 7f5592269640 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80cf900 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5584057400 tx=0x7f558405f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290cbf386389dd3d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f556f7fe640 1 -- 172.21.15.70:0/2168775697 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584055900 con 0x564dc80a5cd0 2023-08-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f556f7fe640 1 -- 172.21.15.70:0/2168775697 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584053e10 con 0x564dc80a5cd0 2023-08-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f556f7fe640 1 -- 172.21.15.70:0/2168775697 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584053f80 con 0x564dc80a5cd0 2023-08-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 -- 172.21.15.70:0/2168775697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 msgr2=0x564dc80cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80cf900 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f5584057400 tx=0x7f558405f410 comp rx=0 tx=0).stop 2023-08-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 -- 172.21.15.70:0/2168775697 shutdown_connections 2023-08-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80cf900 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a2130 0x564dc80cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 --2- 172.21.15.70:0/2168775697 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 0x564dc80d2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 -- 172.21.15.70:0/2168775697 >> 172.21.15.70:0/2168775697 conn(0x564dc80a43e0 msgr2=0x564dc80ecf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.180+0000 7f5596847740 1 -- 172.21.15.70:0/2168775697 shutdown_connections 2023-08-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.181+0000 7f5596847740 1 -- 172.21.15.70:0/2168775697 wait complete. 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.181+0000 7f5596847740 1 -- start start 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.181+0000 7f5596847740 1 Processor -- start 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80d2150 0x564dc80cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc7fadd20 con 0x564dc80a5cd0 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc7fad9c0 con 0x564dc80bce10 2023-08-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dc7d73f20 con 0x564dc80d2150 2023-08-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.182+0000 7f5596847740 10 client.0 _ll_register_callbacks cb 0x564dc7fc5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51230/0 (socket says 172.21.15.70:51230) 2023-08-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 -- 172.21.15.70:0/3794197360 learned_addr learned my addr 172.21.15.70:0/3794197360 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591267640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80d2150 0x564dc80cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5592269640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 -- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80d2150 msgr2=0x564dc80cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80d2150 0x564dc80cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 -- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 msgr2=0x564dc80bd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80bd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 -- 172.21.15.70:0/3794197360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c152950 con 0x564dc80a5cd0 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f5591a68640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5900 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f558802c4f0 tx=0x7f5588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a018b00ae93bb8f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.183+0000 7f556cff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f556cff9640 1 -- 172.21.15.70:0/3794197360 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588046070 con 0x564dc80a5cd0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f556cff9640 1 -- 172.21.15.70:0/3794197360 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5588041c40 con 0x564dc80a5cd0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dc7fe2540 con 0x564dc80a5cd0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f556cff9640 1 -- 172.21.15.70:0/3794197360 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5588041e00 con 0x564dc80a5cd0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 10 client.9288 fetch_fsmap learned FSMap version 82 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x564dc8005f30 con 0x564dc80a5cd0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f556cff9640 1 -- 172.21.15.70:0/3794197360 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dc7fe2540 con 0x564dc80a5cd0 2023-08-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f556cff9640 1 -- 172.21.15.70:0/3794197360 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f558803c040 con 0x564dc80a5cd0 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 10 client.9288 fetch_fsmap finished waiting for FSMap version 82 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 10 client.9288 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 20 client.9288 populate_metadata read hostname 'smithi070' 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80fd7e0 0x564dc80fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 4 client.9288 mds_command: new command op to 9162 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.184+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 --> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dc8007150 con 0x564dc80fd7e0 2023-08-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.185+0000 7f5591267640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80fd7e0 0x564dc80fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.186+0000 7f5591267640 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80fd7e0 0x564dc80fdbc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72b22e02550fdc1 server_cookie=ead9c9ad552bd7b0 in_seq=0 out_seq=0 2023-08-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.186+0000 7f556cff9640 10 client.9288 ms_handle_connect on v2:172.21.15.155:6832/3137406973 2023-08-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.188+0000 7f556cff9640 1 -- 172.21.15.70:0/3794197360 <== mds.0 v2:172.21.15.155:6832/3137406973 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x564dc8007150 con 0x564dc80fd7e0 2023-08-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.188+0000 7f556cff9640 10 client.9288 handle_command_reply: tid=0 2023-08-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.188+0000 7f5596847740 1 client.9288 shutdown 2023-08-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.188+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 msgr2=0x564dc80d5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.188+0000 7f5596847740 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5900 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f558802c4f0 tx=0x7f5588004a40 comp rx=0 tx=0).stop 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 shutdown_connections 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80fd7e0 0x564dc80fdbc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80bd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80d2150 0x564dc80cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 --2- 172.21.15.70:0/3794197360 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5900 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f556cff9640 0 client.9288 ms_handle_reset on v2:172.21.15.155:6832/3137406973 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 >> 172.21.15.70:0/3794197360 conn(0x564dc80a2130 msgr2=0x564dc80a6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 shutdown_connections 2023-08-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 1 -- 172.21.15.70:0/3794197360 wait complete. 2023-08-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.189+0000 7f5596847740 20 client.9288 trim_cache size 0 max 16384 2023-08-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.205+0000 7f5596847740 1 Processor -- start 2023-08-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.205+0000 7f5596847740 1 -- start start 2023-08-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.206+0000 7f5596847740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.206+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80bce10 0x564dc809d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.206+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 0x564dc80cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.206+0000 7f5596847740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc80f0f30 con 0x564dc80a5cd0 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.206+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc80c7da0 con 0x564dc80d2150 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.206+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dc80efe70 con 0x564dc80bce10 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51240/0 (socket says 172.21.15.70:51240) 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 -- 172.21.15.70:0/3499328695 learned_addr learned my addr 172.21.15.70:0/3499328695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 -- 172.21.15.70:0/3499328695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80bce10 msgr2=0x564dc809d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80bce10 0x564dc809d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 -- 172.21.15.70:0/3499328695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 msgr2=0x564dc80cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 0x564dc80cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 -- 172.21.15.70:0/3499328695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dc7faca10 con 0x564dc80a5cd0 2023-08-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f5591a68640 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5960 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f558803e8c0 tx=0x7f5588046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ca525a8ca167be server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f556ffff640 1 -- 172.21.15.70:0/3499328695 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588041c40 con 0x564dc80a5cd0 2023-08-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f556ffff640 1 -- 172.21.15.70:0/3499328695 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5588041da0 con 0x564dc80a5cd0 2023-08-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.207+0000 7f556ffff640 1 -- 172.21.15.70:0/3499328695 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558803ee90 con 0x564dc80a5cd0 2023-08-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 -- 172.21.15.70:0/3499328695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 msgr2=0x564dc80d5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5960 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f558803e8c0 tx=0x7f5588046000 comp rx=0 tx=0).stop 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 -- 172.21.15.70:0/3499328695 shutdown_connections 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80d2150 0x564dc80cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80bce10 0x564dc809d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 --2- 172.21.15.70:0/3499328695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80a5cd0 0x564dc80d5960 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 -- 172.21.15.70:0/3499328695 >> 172.21.15.70:0/3499328695 conn(0x564dc80a2130 msgr2=0x564dc80c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 -- 172.21.15.70:0/3499328695 shutdown_connections 2023-08-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.208+0000 7f5596847740 1 -- 172.21.15.70:0/3499328695 wait complete. 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.209+0000 7f5596847740 1 -- start start 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.209+0000 7f5596847740 1 Processor -- start 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a5cd0 0x564dc80d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 0x564dc8156f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc8107da0 con 0x564dc80d2150 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dc80cb250 con 0x564dc80bce10 2023-08-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dc80a7990 con 0x564dc80a5cd0 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5596847740 10 client.0 _ll_register_callbacks cb 0x7f5584057550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5592269640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 0x564dc8156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5592269640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 0x564dc8156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51244/0 (socket says 172.21.15.70:51244) 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.210+0000 7f5592269640 1 -- 172.21.15.70:0/2379423701 learned_addr learned my addr 172.21.15.70:0/2379423701 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5591a68640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a5cd0 0x564dc80d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5591267640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5592269640 1 -- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a5cd0 msgr2=0x564dc80d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5592269640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a5cd0 0x564dc80d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5592269640 1 -- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 msgr2=0x564dc80a6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5592269640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80a6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5592269640 1 -- 172.21.15.70:0/2379423701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564dc8005f30 con 0x564dc80d2150 2023-08-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5591a68640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a5cd0 0x564dc80d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5592269640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 0x564dc8156f20 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f5584057640 tx=0x7f55840865f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ac4508e9203c9f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f556cff9640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f556cff9640 1 -- 172.21.15.70:0/2379423701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558405ef80 con 0x564dc80d2150 2023-08-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f556cff9640 1 -- 172.21.15.70:0/2379423701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f558405f0e0 con 0x564dc80d2150 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5584053e10 con 0x564dc80d2150 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f556cff9640 1 -- 172.21.15.70:0/2379423701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f558405d0b0 con 0x564dc80d2150 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5596847740 10 client.9294 fetch_fsmap learned FSMap version 82 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.211+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x564dc7fd1de0 con 0x564dc80d2150 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f556cff9640 1 -- 172.21.15.70:0/2379423701 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584053e10 con 0x564dc80d2150 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f556cff9640 1 -- 172.21.15.70:0/2379423701 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f558405d210 con 0x564dc80d2150 2023-08-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5596847740 10 client.9294 fetch_fsmap finished waiting for FSMap version 82 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5596847740 10 client.9294 resolve_mds: resolved name 'b' to daemon mds.b 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5596847740 20 client.9294 populate_metadata read hostname 'smithi070' 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5596847740 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80cef70 0x564dc80cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5596847740 4 client.9294 mds_command: new command op to 9162 tid=0[{"prefix": "status"}] 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 --> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564dc7cbd5e0 con 0x564dc80cef70 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.212+0000 7f5591a68640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80cef70 0x564dc80cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.213+0000 7f5591a68640 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80cef70 0x564dc80cf350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ab9c3fd493b8a2 server_cookie=668994808ac46011 in_seq=0 out_seq=0 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.213+0000 7f556cff9640 10 client.9294 ms_handle_connect on v2:172.21.15.155:6832/3137406973 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f556cff9640 1 -- 172.21.15.70:0/2379423701 <== mds.0 v2:172.21.15.155:6832/3137406973 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x564dc7cbd5e0 con 0x564dc80cef70 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f556cff9640 10 client.9294 handle_command_reply: tid=0 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 client.9294 shutdown 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 msgr2=0x564dc8156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 0x564dc8156f20 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f5584057640 tx=0x7f55840865f0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 shutdown_connections 2023-08-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:6832/3137406973,v1:172.21.15.155:6834/3137406973] conn(0x564dc80cef70 0x564dc80cf350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x564dc80d2150 0x564dc8156f20 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x564dc80bce10 0x564dc80a6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 --2- 172.21.15.70:0/2379423701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x564dc80a5cd0 0x564dc80d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f556cff9640 0 client.9294 ms_handle_reset on v2:172.21.15.155:6832/3137406973 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 >> 172.21.15.70:0/2379423701 conn(0x564dc80a2130 msgr2=0x564dc80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 shutdown_connections 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.214+0000 7f5596847740 1 -- 172.21.15.70:0/2379423701 wait complete. 2023-08-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5596847740 20 client.9294 trim_cache size 0 max 16384 2023-08-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 9162, 2023-08-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 6.7112730139999996, 2023-08-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 82, 2023-08-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 182, 2023-08-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 182, 2023-08-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 13.605031904000001 2023-08-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 msgr2=0x7f557006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5588035000 tx=0x7f5588030000 comp rx=0 tx=0).stop 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 msgr2=0x7f558c0c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0c62f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f5584009fd0 tx=0x7f558400b8e0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 shutdown_connections 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f557006a050 0x7f557006c510 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f558c15cd90 0x7f558c0c62f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f558c0b2360 0x7f558c0c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 --2- 172.21.15.70:0/147767058 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f558c06ad20 0x7f558c0c5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 >> 172.21.15.70:0/147767058 conn(0x7f558c06cf60 msgr2=0x7f558c15f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 shutdown_connections 2023-08-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.215+0000 7f5593ccc640 1 -- 172.21.15.70:0/147767058 wait complete. 2023-08-25T09:50:42.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:50:42.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.417+0000 7fe8f8a14640 1 Processor -- start 2023-08-25T09:50:42.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.417+0000 7fe8f8a14640 1 -- start start 2023-08-25T09:50:42.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f8a14640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe8f40cea30 0x7fe8f40b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f8a14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f406f050 con 0x7fe8f415cb40 2023-08-25T09:50:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f8a14640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f406f330 con 0x7fe8f40ac870 2023-08-25T09:50:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f8a14640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f406f610 con 0x7fe8f40cea30 2023-08-25T09:50:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe8f415cb40 0x7fe8f40ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:41790/0 (socket says 172.21.15.70:41790) 2023-08-25T09:50:42.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f259c640 1 -- 172.21.15.70:0/3866118686 learned_addr learned my addr 172.21.15.70:0/3866118686 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:42.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/3866118686 >> v2:172.21.15.155:3300/0 conn(0x7fe8f40cea30 0x7fe8f40b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549227344 0 0) 0x7fe8f406f050 con 0x7fe8f415cb40 2023-08-25T09:50:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.419+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e0003520 con 0x7fe8f415cb40 2023-08-25T09:50:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3663290706 0 0) 0x7fe8e0003520 con 0x7fe8f415cb40 2023-08-25T09:50:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 >> v2:172.21.15.155:3300/0 conn(0x7fe8f40cea30 msgr2=0x7fe8f40b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 --2- 172.21.15.70:0/3866118686 >> v2:172.21.15.155:3300/0 conn(0x7fe8f40cea30 0x7fe8f40b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 >> v1:172.21.15.155:6789/0 conn(0x7fe8f40ac870 legacy=0x7fe8f40b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f406f8f0 con 0x7fe8f415cb40 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe8e4001670 con 0x7fe8f415cb40 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 >> v1:172.21.15.70:6789/0 conn(0x7fe8f415cb40 legacy=0x7fe8f40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e0003970 0x7fe8e0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e00047c0 0x7fe8e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e0005180 0x7fe8e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f406f610 con 0x7fe8e00047c0 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e000c000 con 0x7fe8e0003970 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e000c1c0 con 0x7fe8e0005180 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.420+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e00047c0 0x7fe8e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8f259c640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e0003970 0x7fe8e0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8ebfff640 1 -- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e0005180 msgr2=0x7fe8e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8e0005180 0x7fe8e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8ebfff640 1 -- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e0003970 msgr2=0x7fe8e0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:42.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e0003970 0x7fe8e0003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8ebfff640 1 -- 172.21.15.70:0/3866118686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0003520 con 0x7fe8e00047c0 2023-08-25T09:50:42.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.421+0000 7fe8f2d9d640 1 -- 172.21.15.70:0/3866118686 reap_dead start 2023-08-25T09:50:42.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.422+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e00047c0 0x7fe8e0004c40 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009ac0 tx=0x7fe8e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a650edd37de81a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.422+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc02c6d0 con 0x7fe8e00047c0 2023-08-25T09:50:42.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.422+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc02c830 con 0x7fe8e00047c0 2023-08-25T09:50:42.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.422+0000 7fe8f1d9b640 1 -- 172.21.15.70:0/3866118686 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc02cbc0 con 0x7fe8e00047c0 2023-08-25T09:50:42.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 -- 172.21.15.70:0/3866118686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e00047c0 msgr2=0x7fe8e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e00047c0 0x7fe8e0004c40 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc009ac0 tx=0x7fe8e00057f0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 -- 172.21.15.70:0/3866118686 shutdown_connections 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/3866118686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8e00047c0 0x7fe8e0004c40 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 -- 172.21.15.70:0/3866118686 >> 172.21.15.70:0/3866118686 conn(0x7fe8f4069850 msgr2=0x7fe8f4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 -- 172.21.15.70:0/3866118686 shutdown_connections 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.423+0000 7fe8f8a14640 1 -- 172.21.15.70:0/3866118686 wait complete. 2023-08-25T09:50:42.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.424+0000 7fe8f8a14640 1 Processor -- start 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.425+0000 7fe8f8a14640 1 -- start start 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.425+0000 7fe8f8a14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e00047c0 0x7fe8f40cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.425+0000 7fe8f8a14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 0x7fe8f4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.425+0000 7fe8f8a14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8f4154d90 0x7fe8f4155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8f8a14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41f6590 con 0x7fe8f40ccf70 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8f8a14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f41416a0 con 0x7fe8e00047c0 2023-08-25T09:50:42.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8f8a14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f40cd760 con 0x7fe8f4154d90 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 0x7fe8f4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 0x7fe8f4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51260/0 (socket says 172.21.15.70:51260) 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 -- 172.21.15.70:0/97041925 learned_addr learned my addr 172.21.15.70:0/97041925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8f259c640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e00047c0 0x7fe8f40cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 -- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8f4154d90 msgr2=0x7fe8f4155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8f4154d90 0x7fe8f4155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 -- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e00047c0 msgr2=0x7fe8f40cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e00047c0 0x7fe8f40cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8ebfff640 1 -- 172.21.15.70:0/97041925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f40cda40 con 0x7fe8f40ccf70 2023-08-25T09:50:42.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.426+0000 7fe8f259c640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e00047c0 0x7fe8f40cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:42.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.427+0000 7fe8ebfff640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 0x7fe8f4154850 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc037040 tx=0x7fe8dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.427+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003810 con 0x7fe8f40ccf70 2023-08-25T09:50:42.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.427+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8dc003970 con 0x7fe8f40ccf70 2023-08-25T09:50:42.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.427+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003d00 con 0x7fe8f40ccf70 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.427+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f4141050 con 0x7fe8f40ccf70 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.427+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f40cd3f0 con 0x7fe8f40ccf70 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.428+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f404e420 con 0x7fe8f40ccf70 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.429+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe8dc040450 con 0x7fe8f40ccf70 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.430+0000 7fe8eaffd640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe8d406a050 0x7fe8d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.430+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8dc0aa960 con 0x7fe8f40ccf70 2023-08-25T09:50:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.430+0000 7fe8f259c640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe8d406a050 0x7fe8d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.431+0000 7fe8f259c640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe8d406a050 0x7fe8d406c510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40079d0 tx=0x7fe8e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:42.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.434+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe8dc075fb0 con 0x7fe8f40ccf70 2023-08-25T09:50:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.666+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8f41417e0 con 0x7fe8f40ccf70 2023-08-25T09:50:42.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.667+0000 7fe8eaffd640 1 -- 172.21.15.70:0/97041925 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8dc003e60 con 0x7fe8f40ccf70 2023-08-25T09:50:42.669 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:50:42.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe8d406a050 msgr2=0x7fe8d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe8d406a050 0x7fe8d406c510 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe8e40079d0 tx=0x7fe8e4009040 comp rx=0 tx=0).stop 2023-08-25T09:50:42.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 msgr2=0x7fe8f4154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:42.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 0x7fe8f4154850 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc037040 tx=0x7fe8dc004730 comp rx=0 tx=0).stop 2023-08-25T09:50:42.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 shutdown_connections 2023-08-25T09:50:42.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe8d406a050 0x7fe8d406c510 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe8f4154d90 0x7fe8f4155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe8f40ccf70 0x7fe8f4154850 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 --2- 172.21.15.70:0/97041925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe8e00047c0 0x7fe8f40cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.669+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 >> 172.21.15.70:0/97041925 conn(0x7fe8f4069850 msgr2=0x7fe8f4069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.670+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 shutdown_connections 2023-08-25T09:50:42.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:42.670+0000 7fe8f8a14640 1 -- 172.21.15.70:0/97041925 wait complete. 2023-08-25T09:50:42.688 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:50:42.689 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:50:42.689 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:50:42.689 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:50:42.690 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:42.690 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:50:42.690 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:50:42.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:50:42.691 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:50:42.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:42.692 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:50:42.740 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:50:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:50:42.740 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:50:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:50:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.741 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:50:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:42.741 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:50:42.742 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:50:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:50:42.742 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:50:42.742 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41633sec preferred_lft 41633sec 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.743 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:50:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:50:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:50:42.744 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:50:42.745 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:42.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:42.746 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:50:42.831 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:50:42.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:42.836 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:50:42.888 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:50:42.889 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:50:42.889 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:50:42.889 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:50:42.889 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:50:42.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:42.890 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:50:42.890 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:50:42.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:50:42.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:50:42.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:50:42.892 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:50:43.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:43.081 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:50:43.130 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:50:43.131 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:50:43.131 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:50:43.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:43.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:50:43.225 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:50:43.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:50:43.226 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:43.278 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:50:43.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:50:43.279 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:43.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:50:43.429 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:50:43.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:50:43.525 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:50:43.530 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:50:43.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:43.531 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:50:43.583 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:50:43.584 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:50:43.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:50:43.673 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:50:43.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:50:43.746 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:50:43.748 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:50:43.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:43.750 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:50:43.784 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:50:43.783+0000 7f11eae93480 -1 init, newargv = 0x5648821b5e20 newargc=15 2023-08-25T09:50:43.785 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[124460]: starting ceph client 2023-08-25T09:50:43.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[124460]: starting fuse 2023-08-25T09:50:44.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:50:44.993 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:50:44.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:50:45.086 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:50:45.090 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:50:45.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:45.091 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:50:45.146 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:50:45.146 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:50:45.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:50:45.147 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:50:45.147 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:50:45.148 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:50:45.148 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:50:45.148 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:45.149 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:50:45.149 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:50:45.149 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:50:45.150 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:50:45.150 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:45.150 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:50:45.151 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:50:45.151 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:50:45.151 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:50:45.152 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:45.154 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:50:45.155 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:50:45.155 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:50:45.156 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:50:45.157 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:50:45.158 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:50:45.158 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:50:45.158 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:50:45.159 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:45.159 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:50:45.160 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:50:45.338 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.124460.asok 2023-08-25T09:50:45.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:50:45.356 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.124460.asok 2023-08-25T09:50:45.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:45.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.124460.asok status 2023-08-25T09:50:45.597 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:50:45.597 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:50:45.598 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:50:45.598 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:50:45.598 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:50:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:50:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:50:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "124460", 2023-08-25T09:50:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:50:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:50:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:50:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 9303, 2023-08-25T09:50:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:50:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:50:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:50:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 9303 2023-08-25T09:50:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:50:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:50:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:50:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 436442486 2023-08-25T09:50:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:50:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:50:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:50:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:50:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 436442486 2023-08-25T09:50:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:50:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.9303 192.168.0.1:0/436442486", 2023-08-25T09:50:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/436442486", 2023-08-25T09:50:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:50:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 82, 2023-08-25T09:50:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 184, 2023-08-25T09:50:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:50:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:50:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:50:45.609 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:50:45.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:45.614 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:50:45.614 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:45.673 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:50:45.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:45.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:45.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:50:45.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:50:45.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.992+0000 7f6b0a67c640 1 Processor -- start 2023-08-25T09:50:45.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.992+0000 7f6b0a67c640 1 -- start start 2023-08-25T09:50:45.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.993+0000 7f6b0a67c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f6b040ce5b0 0x7f6b040ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:45.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.993+0000 7f6b0a67c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b0415aa10 0x7f6b040b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:45.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.993+0000 7f6b0a67c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b040b2680 0x7f6b0406be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.993+0000 7f6b0a67c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b040b2b10 con 0x7f6b040b2680 2023-08-25T09:50:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.993+0000 7f6b0a67c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b04159d10 con 0x7f6b040ce5b0 2023-08-25T09:50:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.993+0000 7f6b0a67c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040ceed0 con 0x7f6b0415aa10 2023-08-25T09:50:45.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b040b2680 0x7f6b0406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f6b040b2680 0x7f6b0406be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51272/0 (socket says 172.21.15.70:51272) 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 -- 172.21.15.70:0/3807434969 learned_addr learned my addr 172.21.15.70:0/3807434969 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b03fff640 1 --2- 172.21.15.70:0/3807434969 >> v2:172.21.15.155:3300/0 conn(0x7f6b040ce5b0 0x7f6b040ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b037fe640 1 --2- 172.21.15.70:0/3807434969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b0415aa10 0x7f6b040b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 -- 172.21.15.70:0/3807434969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b0415aa10 msgr2=0x7f6b040b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 --2- 172.21.15.70:0/3807434969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b0415aa10 0x7f6b040b2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:45.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 -- 172.21.15.70:0/3807434969 >> v2:172.21.15.155:3300/0 conn(0x7f6b040ce5b0 msgr2=0x7f6b040ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:46.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 --2- 172.21.15.70:0/3807434969 >> v2:172.21.15.155:3300/0 conn(0x7f6b040ce5b0 0x7f6b040ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.994+0000 7f6b08c19640 1 -- 172.21.15.70:0/3807434969 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0415d3f0 con 0x7f6b040b2680 2023-08-25T09:50:46.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.995+0000 7f6b08c19640 1 --2- 172.21.15.70:0/3807434969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040b2680 0x7f6b0406be80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f6b0406f470 tx=0x7f6afc02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c717d0d8c0a19b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:46.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b02ffd640 1 -- 172.21.15.70:0/3807434969 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc02ee80 con 0x7f6b040b2680 2023-08-25T09:50:46.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b02ffd640 1 -- 172.21.15.70:0/3807434969 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6afc036440 con 0x7f6b040b2680 2023-08-25T09:50:46.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b02ffd640 1 -- 172.21.15.70:0/3807434969 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc0367b0 con 0x7f6b040b2680 2023-08-25T09:50:46.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 -- 172.21.15.70:0/3807434969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040b2680 msgr2=0x7f6b0406be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:46.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/3807434969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040b2680 0x7f6b0406be80 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f6b0406f470 tx=0x7f6afc02e0d0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 -- 172.21.15.70:0/3807434969 shutdown_connections 2023-08-25T09:50:46.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/3807434969 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040b2680 0x7f6b0406be80 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/3807434969 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b0415aa10 0x7f6b040b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/3807434969 >> v2:172.21.15.155:3300/0 conn(0x7f6b040ce5b0 0x7f6b040ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.996+0000 7f6b0a67c640 1 -- 172.21.15.70:0/3807434969 >> 172.21.15.70:0/3807434969 conn(0x7f6b04069850 msgr2=0x7f6b04069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:46.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.997+0000 7f6b0a67c640 1 -- 172.21.15.70:0/3807434969 shutdown_connections 2023-08-25T09:50:46.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.997+0000 7f6b0a67c640 1 -- 172.21.15.70:0/3807434969 wait complete. 2023-08-25T09:50:46.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.998+0000 7f6b0a67c640 1 Processor -- start 2023-08-25T09:50:46.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.998+0000 7f6b0a67c640 1 -- start start 2023-08-25T09:50:46.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.999+0000 7f6b0a67c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b040b2680 0x7f6b0414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:46.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.999+0000 7f6b0a67c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 0x7f6b040cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:46.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.999+0000 7f6b0a67c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b0415aa10 0x7f6b040b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:46.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.999+0000 7f6b0a67c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0413bbc0 con 0x7f6b040ce5b0 2023-08-25T09:50:46.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.999+0000 7f6b0a67c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040cb660 con 0x7f6b0415aa10 2023-08-25T09:50:46.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:45.999+0000 7f6b0a67c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b040cc4c0 con 0x7f6b040b2680 2023-08-25T09:50:46.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 0x7f6b040cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:46.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 0x7f6b040cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51276/0 (socket says 172.21.15.70:51276) 2023-08-25T09:50:46.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 -- 172.21.15.70:0/903269175 learned_addr learned my addr 172.21.15.70:0/903269175 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:46.008 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 -- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b040b2680 msgr2=0x7f6b0414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:46.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b040b2680 0x7f6b0414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b08c19640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b0415aa10 0x7f6b040b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:46.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 -- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b0415aa10 msgr2=0x7f6b040b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:46.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b0415aa10 0x7f6b040b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 -- 172.21.15.70:0/903269175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc00b770 con 0x7f6b040ce5b0 2023-08-25T09:50:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.000+0000 7f6b037fe640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 0x7f6b040cc950 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00cd00 tx=0x7f6aec00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.001+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec003990 con 0x7f6b040ce5b0 2023-08-25T09:50:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.001+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aec003af0 con 0x7f6b040ce5b0 2023-08-25T09:50:46.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.001+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aec003e80 con 0x7f6b040ce5b0 2023-08-25T09:50:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.001+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0414e500 con 0x7f6b040ce5b0 2023-08-25T09:50:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.001+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b040cbe70 con 0x7f6b040ce5b0 2023-08-25T09:50:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.002+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b040cb910 con 0x7f6b040ce5b0 2023-08-25T09:50:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.003+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6aec004020 con 0x7f6b040ce5b0 2023-08-25T09:50:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.004+0000 7f6b00ff9640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6ae406a050 0x7f6ae406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.004+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f6aec088980 con 0x7f6b040ce5b0 2023-08-25T09:50:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.004+0000 7f6b03fff640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6ae406a050 0x7f6ae406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.005+0000 7f6b03fff640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6ae406a050 0x7f6ae406c510 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6af4006fd0 tx=0x7f6af4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.008+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6aec053fb0 con 0x7f6b040ce5b0 2023-08-25T09:50:46.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.237+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6b0404e420 con 0x7f6b040ce5b0 2023-08-25T09:50:46.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.238+0000 7f6b00ff9640 1 -- 172.21.15.70:0/903269175 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6aec01d020 con 0x7f6b040ce5b0 2023-08-25T09:50:46.239 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:50:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.240+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6ae406a050 msgr2=0x7f6ae406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.240+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6ae406a050 0x7f6ae406c510 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6af4006fd0 tx=0x7f6af4008040 comp rx=0 tx=0).stop 2023-08-25T09:50:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 msgr2=0x7f6b040cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 0x7f6b040cc950 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f6aec00cd00 tx=0x7f6aec00a5c0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 shutdown_connections 2023-08-25T09:50:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f6ae406a050 0x7f6ae406c510 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6b0415aa10 0x7f6b040b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6b040ce5b0 0x7f6b040cc950 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 --2- 172.21.15.70:0/903269175 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6b040b2680 0x7f6b0414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:46.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 >> 172.21.15.70:0/903269175 conn(0x7f6b04069850 msgr2=0x7f6b0406be20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:46.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 shutdown_connections 2023-08-25T09:50:46.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:46.241+0000 7f6b0a67c640 1 -- 172.21.15.70:0/903269175 wait complete. 2023-08-25T09:50:46.261 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:50:46.262 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:50:46.262 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:50:46.263 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:50:46.263 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:46.263 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:50:46.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:50:46.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:50:46.265 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:50:46.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:46.266 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:50:46.327 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:50:46.327 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:50:46.327 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:50:46.327 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.328 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:50:46.328 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.328 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:50:46.328 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:46.328 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:50:46.329 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:50:46.329 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:46.329 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:50:46.329 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:50:46.329 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:46.330 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:50:46.330 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:46.330 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:50:46.330 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41643sec preferred_lft 41643sec 2023-08-25T09:50:46.330 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:50:46.330 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.331 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:50:46.331 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:50:46.331 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:50:46.331 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.331 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:50:46.331 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.332 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:50:46.332 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:50:46.332 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:50:46.332 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.332 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:50:46.332 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:50:46.333 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:50:46.333 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:50:46.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:46.334 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:50:46.427 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:50:46.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:46.432 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:50:46.489 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:50:46.489 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:50:46.489 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:50:46.489 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:50:46.490 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:50:46.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:46.490 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:50:46.491 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:50:46.491 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:50:46.491 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:50:46.492 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:50:46.493 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:50:46.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:46.679 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:50:46.731 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:50:46.731 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:50:46.732 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:50:46.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:46.733 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:50:46.831 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:50:46.832 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:50:46.832 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:46.885 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:50:46.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:50:46.886 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:46.939 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:50:47.039 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:50:47.039 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:50:47.130 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:50:47.134 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:50:47.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:47.135 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:50:47.191 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:50:47.191 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:50:47.197 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:50:47.273 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:50:47.273 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:50:47.354 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:50:47.357 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:50:47.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:47.358 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:50:47.387 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:50:47.386+0000 7fdbd743a480 -1 init, newargv = 0x557b85ce2eb0 newargc=15 2023-08-25T09:50:47.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[93415]: starting ceph client 2023-08-25T09:50:47.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[93415]: starting fuse 2023-08-25T09:50:47.402 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:50:47.402 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:50:47.403 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:50:47.403 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:50:47.404 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:50:47.404 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:50:47.404 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:50:47.405 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:50:47.405 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:50:47.406 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:50:47.406 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:50:47.406 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:50:47.407 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:50:47.407 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:50:47.408 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:50:47.408 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:50:47.408 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:50:47.409 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:50:47.409 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:50:47.409 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:50:47.409 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:50:47.410 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:50:47.410 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:50:47.411 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:50:47.411 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:50:47.411 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:50:47.412 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:50:47.412 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:50:47.412 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:50:47.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:50:47.422 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:50:47.597 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.93415.asok 2023-08-25T09:50:47.601 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:50:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:50:47.613 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93415.asok 2023-08-25T09:50:47.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:47.614 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.93415.asok status 2023-08-25T09:50:47.794 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:50:47.795 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "93415", 2023-08-25T09:50:47.796 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:50:47.796 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:50:47.796 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:50:47.796 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:50:47.802 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5675, 2023-08-25T09:50:47.802 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:50:47.802 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:50:47.802 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5675 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2210988941 2023-08-25T09:50:47.803 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 2210988941 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5675 192.168.0.1:0/2210988941", 2023-08-25T09:50:47.804 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/2210988941", 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 82, 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 184, 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:50:47.805 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:50:47.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:47.809 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:50:47.809 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:47.862 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:50:47.862 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:47.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:50:47.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:50:47.952 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:50:48.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.177+0000 7fd8bf339640 1 Processor -- start 2023-08-25T09:50:48.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.177+0000 7fd8bf339640 1 -- start start 2023-08-25T09:50:48.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bf339640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8b80ce8e0 0x7fd8b80cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:48.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bf339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cdc80 0x7fd8b80cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:48.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bf339640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80cba70 con 0x7fd8b80cf200 2023-08-25T09:50:48.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bf339640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b80cbd50 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bf339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80cc030 con 0x7fd8b80cdc80 2023-08-25T09:50:48.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bd0d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8b80ce8e0 0x7fd8b80cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:48.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bd0d5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd8b80ce8e0 0x7fd8b80cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51284/0 (socket says 172.21.15.70:51284) 2023-08-25T09:50:48.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.179+0000 7fd8bd0d5640 1 -- 172.21.15.70:0/3081781158 learned_addr learned my addr 172.21.15.70:0/3081781158 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:48.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.180+0000 7fd8bd0d5640 1 -- 172.21.15.70:0/3081781158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cdc80 msgr2=0x7fd8b80cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:48.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.180+0000 7fd8bc8d4640 1 --2- 172.21.15.70:0/3081781158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cdc80 0x7fd8b80cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:48.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.180+0000 7fd8bd0d5640 1 --2- 172.21.15.70:0/3081781158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cdc80 0x7fd8b80cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.180+0000 7fd8bd0d5640 1 -- 172.21.15.70:0/3081781158 >> v1:172.21.15.155:6789/0 conn(0x7fd8b80cf200 legacy=0x7fd8b806d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:48.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.180+0000 7fd8bd0d5640 1 -- 172.21.15.70:0/3081781158 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b80cc310 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.181+0000 7fd8bd0d5640 1 --2- 172.21.15.70:0/3081781158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b80cb470 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009850 tx=0x7fd8b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547b77d51a03ffd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:48.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8affff640 1 -- 172.21.15.70:0/3081781158 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4004230 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8affff640 1 -- 172.21.15.70:0/3081781158 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4004390 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8affff640 1 -- 172.21.15.70:0/3081781158 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4031830 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8bf339640 1 -- 172.21.15.70:0/3081781158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 msgr2=0x7fd8b80cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:48.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3081781158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b80cb470 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009850 tx=0x7fd8b402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8bf339640 1 -- 172.21.15.70:0/3081781158 shutdown_connections 2023-08-25T09:50:48.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3081781158 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cdc80 0x7fd8b80cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3081781158 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b80cb470 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.182+0000 7fd8bf339640 1 -- 172.21.15.70:0/3081781158 >> 172.21.15.70:0/3081781158 conn(0x7fd8b80d0860 msgr2=0x7fd8b80d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:48.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.183+0000 7fd8bf339640 1 -- 172.21.15.70:0/3081781158 shutdown_connections 2023-08-25T09:50:48.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.183+0000 7fd8bf339640 1 -- 172.21.15.70:0/3081781158 wait complete. 2023-08-25T09:50:48.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.184+0000 7fd8bf339640 1 Processor -- start 2023-08-25T09:50:48.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.185+0000 7fd8bf339640 1 -- start start 2023-08-25T09:50:48.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bf339640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b80cdc80 0x7fd8b81e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:48.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bf339640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b81ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:48.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bf339640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cf200 0x7fd8b80c3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:48.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bf339640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81e17c0 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bf339640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81e1aa0 con 0x7fd8b80cdc80 2023-08-25T09:50:48.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bf339640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81e1d80 con 0x7fd8b80cf200 2023-08-25T09:50:48.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bc8d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b81ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:48.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bc8d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b81ddf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51296/0 (socket says 172.21.15.70:51296) 2023-08-25T09:50:48.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bc8d4640 1 -- 172.21.15.70:0/3283209186 learned_addr learned my addr 172.21.15.70:0/3283209186 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:48.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bd0d5640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b80cdc80 0x7fd8b81e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:48.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.186+0000 7fd8bd8d6640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cf200 0x7fd8b80c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:48.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bc8d4640 1 -- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cf200 msgr2=0x7fd8b80c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:48.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bc8d4640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cf200 0x7fd8b80c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bc8d4640 1 -- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b80cdc80 msgr2=0x7fd8b81e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:48.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bc8d4640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b80cdc80 0x7fd8b81e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bc8d4640 1 -- 172.21.15.70:0/3283209186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4009460 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bd0d5640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b80cdc80 0x7fd8b81e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:50:48.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bd8d6640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cf200 0x7fd8b80c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:50:48.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bc8d4640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b81ddf40 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7f0 tx=0x7fd8a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:48.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004220 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8004380 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b81e2060 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.187+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8007840 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.188+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b80b3660 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.189+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b80b38e0 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.190+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd8a8007bb0 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.191+0000 7fd8adffb640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd89c06a050 0x7fd89c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:48.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.191+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd8a80884f0 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.191+0000 7fd8bd0d5640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd89c06a050 0x7fd89c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:48.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.192+0000 7fd8bd0d5640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd89c06a050 0x7fd89c06c510 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4035000 tx=0x7fd8b4008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:48.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.196+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd8a8053bb0 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.421+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8b804e420 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.422+0000 7fd8adffb640 1 -- 172.21.15.70:0/3283209186 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fd8a8007e60 con 0x7fd8b80ce8e0 2023-08-25T09:50:48.721 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:50:48.722 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:50:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:50:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd89c06a050 msgr2=0x7fd89c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd89c06a050 0x7fd89c06c510 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4035000 tx=0x7fd8b4008030 comp rx=0 tx=0).stop 2023-08-25T09:50:48.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 msgr2=0x7fd8b81ddf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b81ddf40 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7f0 tx=0x7fd8a800ccb0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 shutdown_connections 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd89c06a050 0x7fd89c06c510 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8b80cf200 0x7fd8b80c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd8b80ce8e0 0x7fd8b81ddf40 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 --2- 172.21.15.70:0/3283209186 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd8b80cdc80 0x7fd8b81e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:48.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 >> 172.21.15.70:0/3283209186 conn(0x7fd8b80d0860 msgr2=0x7fd8b80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:48.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 shutdown_connections 2023-08-25T09:50:48.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:48.424+0000 7fd8bf339640 1 -- 172.21.15.70:0/3283209186 wait complete. 2023-08-25T09:50:48.727 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2023-08-25T09:50:48.972 INFO:teuthology.orchestra.run.smithi155.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":9162,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.45352959,"mdsmap_epoch":82,"osdmap_epoch":184,"osdmap_epoch_barrier":182,"uptime":20.347292997} 2023-08-25T09:50:48.973 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 9162, "mdsmap_epoch": 82, "osdmap_epoch": 184, "osdmap_epoch_barrier": 182, "rank_uptime": 13.45352959, "state": "up:active", "uptime": 20.347292997, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:50:48.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-08-25T09:50:49.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.196+0000 7f3070f96640 1 Processor -- start 2023-08-25T09:50:49.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.196+0000 7f3070f96640 1 -- start start 2023-08-25T09:50:49.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.197+0000 7f3070f96640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f306c155120 0x7f306c0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:49.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.197+0000 7f3070f96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c15af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:49.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.197+0000 7f3070f96640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c15e6f0 con 0x7f306c0cd890 2023-08-25T09:50:49.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.197+0000 7f3070f96640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0cdd40 con 0x7f306c155120 2023-08-25T09:50:49.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.197+0000 7f3070f96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0caf30 con 0x7f306c0ca9f0 2023-08-25T09:50:49.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.198+0000 7f306a59c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f306c155120 0x7f306c0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:49.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.198+0000 7f3069d9b640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f306c0cd890 0x7f306c0ca2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:47280/0 (socket says 172.21.15.70:47280) 2023-08-25T09:50:49.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.198+0000 7f3069d9b640 1 -- 172.21.15.70:0/1004357148 learned_addr learned my addr 172.21.15.70:0/1004357148 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:50:49.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306ad9d640 1 --2- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c15af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:49.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306959a640 1 -- 172.21.15.70:0/1004357148 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822588034 0 0) 0x7f306c15e6f0 con 0x7f306c0cd890 2023-08-25T09:50:49.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306959a640 1 -- 172.21.15.70:0/1004357148 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3058003520 con 0x7f306c0cd890 2023-08-25T09:50:49.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306a59c640 1 -- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 msgr2=0x7f306c15af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:50:49.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306a59c640 1 --2- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c15af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306a59c640 1 -- 172.21.15.70:0/1004357148 >> v1:172.21.15.155:6789/0 conn(0x7f306c0cd890 legacy=0x7f306c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:49.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.199+0000 7f306a59c640 1 -- 172.21.15.70:0/1004357148 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c15e9d0 con 0x7f306c155120 2023-08-25T09:50:49.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.201+0000 7f306a59c640 1 --2- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c155120 0x7f306c0cd350 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f305c0097a0 tx=0x7f305c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb242d201b9080a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.201+0000 7f306959a640 1 -- 172.21.15.70:0/1004357148 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c031830 con 0x7f306c155120 2023-08-25T09:50:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.201+0000 7f306959a640 1 -- 172.21.15.70:0/1004357148 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c031990 con 0x7f306c155120 2023-08-25T09:50:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.201+0000 7f306959a640 1 -- 172.21.15.70:0/1004357148 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c031d20 con 0x7f306c155120 2023-08-25T09:50:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 -- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c155120 msgr2=0x7f306c0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 --2- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c155120 0x7f306c0cd350 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f305c0097a0 tx=0x7f305c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 -- 172.21.15.70:0/1004357148 shutdown_connections 2023-08-25T09:50:49.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 --2- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c15af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 --2- 172.21.15.70:0/1004357148 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c155120 0x7f306c0cd350 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 -- 172.21.15.70:0/1004357148 >> 172.21.15.70:0/1004357148 conn(0x7f306c069c70 msgr2=0x7f306c06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 -- 172.21.15.70:0/1004357148 shutdown_connections 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.202+0000 7f3070f96640 1 -- 172.21.15.70:0/1004357148 wait complete. 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.204+0000 7f3070f96640 1 Processor -- start 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.204+0000 7f3070f96640 1 -- start start 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f3070f96640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:49.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f3070f96640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0cd890 0x7f306c159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f3070f96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 0x7f306c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f3070f96640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c14fa70 con 0x7f306c155120 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f3070f96640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c15c3b0 con 0x7f306c0cd890 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f3070f96640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c15c690 con 0x7f306c0ca9f0 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 0x7f306c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f306ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 0x7f306c159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:56592/0 (socket says 172.21.15.70:56592) 2023-08-25T09:50:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.205+0000 7f306ad9d640 1 -- 172.21.15.70:0/1691042027 learned_addr learned my addr 172.21.15.70:0/1691042027 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:50:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306a59c640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f3069d9b640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0cd890 0x7f306c159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306ad9d640 1 -- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 msgr2=0x7f306c0ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306ad9d640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c0ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306ad9d640 1 -- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0cd890 msgr2=0x7f306c159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:49.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306ad9d640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0cd890 0x7f306c159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306ad9d640 1 -- 172.21.15.70:0/1691042027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009400 con 0x7f306c155120 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f3069d9b640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0cd890 0x7f306c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.206+0000 7f306ad9d640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 0x7f306c159e50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f306400e7f0 tx=0x7f306400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.207+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30640094b0 con 0x7f306c155120 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.207+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306400c730 con 0x7f306c155120 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.207+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306400cae0 con 0x7f306c155120 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.207+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c15c970 con 0x7f306c155120 2023-08-25T09:50:49.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.207+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1f2000 con 0x7f306c155120 2023-08-25T09:50:49.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.208+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c1f2280 con 0x7f306c155120 2023-08-25T09:50:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.209+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f306400cc40 con 0x7f306c155120 2023-08-25T09:50:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.210+0000 7f3062ffd640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f304c06a050 0x7f304c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:50:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.210+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3064088750 con 0x7f306c155120 2023-08-25T09:50:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.210+0000 7f306a59c640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f304c06a050 0x7f304c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:50:49.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.211+0000 7f306a59c640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f304c06a050 0x7f304c06c510 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f305c036000 tx=0x7f305c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:50:49.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.215+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f306c1f2280 con 0x7f306c155120 2023-08-25T09:50:49.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.406+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f306c1f2280 con 0x7f306c155120 2023-08-25T09:50:49.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.499+0000 7f3062ffd640 1 -- 172.21.15.70:0/1691042027 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v185) v1 ==== 181+0+0 (secure 0 0 0) 0x7f306c1f2280 con 0x7f306c155120 2023-08-25T09:50:49.501 INFO:teuthology.orchestra.run.smithi070.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-08-25T09:50:49.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f304c06a050 msgr2=0x7f304c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:49.504 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f304c06a050 0x7f304c06c510 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f305c036000 tx=0x7f305c03c040 comp rx=0 tx=0).stop 2023-08-25T09:50:49.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 msgr2=0x7f306c159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:50:49.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 0x7f306c159e50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f306400e7f0 tx=0x7f306400ecb0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 shutdown_connections 2023-08-25T09:50:49.505 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f304c06a050 0x7f304c06c510 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f306c155120 0x7f306c159e50 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f306c0cd890 0x7f306c159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 --2- 172.21.15.70:0/1691042027 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f306c0ca9f0 0x7f306c0ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:50:49.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 >> 172.21.15.70:0/1691042027 conn(0x7f306c069c70 msgr2=0x7f306c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:50:49.506 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.503+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 shutdown_connections 2023-08-25T09:50:49.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:50:49.504+0000 7f3070f96640 1 -- 172.21.15.70:0/1691042027 wait complete. 2023-08-25T09:50:49.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:50:49.752 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:50:49.753 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:50:49.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-08-25T09:50:49.977 INFO:teuthology.orchestra.run.smithi070.stdout:{"mon_tick_interval":"5"} 2023-08-25T09:50:49.977 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-08-25T09:50:49.978 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-08-25T09:50:49.978 DEBUG:teuthology.orchestra.run.smithi070:> stdin-killer -- python3 -c ' 2023-08-25T09:50:49.979 DEBUG:teuthology.orchestra.run.smithi070:> import time 2023-08-25T09:50:49.980 DEBUG:teuthology.orchestra.run.smithi070:> import datetime 2023-08-25T09:50:49.980 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:50:49.980 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:50:49.980 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.981 DEBUG:teuthology.orchestra.run.smithi070:> # Write some buffered data through before going full, all should be well 2023-08-25T09:50:49.981 DEBUG:teuthology.orchestra.run.smithi070:> print("writing some data through which we expect to succeed") 2023-08-25T09:50:49.982 DEBUG:teuthology.orchestra.run.smithi070:> bytes = 0 2023-08-25T09:50:49.982 DEBUG:teuthology.orchestra.run.smithi070:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-08-25T09:50:49.982 DEBUG:teuthology.orchestra.run.smithi070:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-08-25T09:50:49.982 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-08-25T09:50:49.983 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:50:49.983 DEBUG:teuthology.orchestra.run.smithi070:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-08-25T09:50:49.983 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.984 DEBUG:teuthology.orchestra.run.smithi070:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-08-25T09:50:49.984 DEBUG:teuthology.orchestra.run.smithi070:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-08-25T09:50:49.984 DEBUG:teuthology.orchestra.run.smithi070:> # buffered IO, the error should always only appear from fsync and not 2023-08-25T09:50:49.985 DEBUG:teuthology.orchestra.run.smithi070:> # from write 2023-08-25T09:50:49.985 DEBUG:teuthology.orchestra.run.smithi070:> full = False 2023-08-25T09:50:49.985 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.986 DEBUG:teuthology.orchestra.run.smithi070:> for n in range(0, int(32 * 1.1)): 2023-08-25T09:50:49.986 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:50:49.986 DEBUG:teuthology.orchestra.run.smithi070:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-08-25T09:50:49.987 DEBUG:teuthology.orchestra.run.smithi070:> print("wrote bytes via buffered write, moving on to fsync") 2023-08-25T09:50:49.987 DEBUG:teuthology.orchestra.run.smithi070:> except OSError as e: 2023-08-25T09:50:49.987 DEBUG:teuthology.orchestra.run.smithi070:> if True: 2023-08-25T09:50:49.987 DEBUG:teuthology.orchestra.run.smithi070:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-08-25T09:50:49.988 DEBUG:teuthology.orchestra.run.smithi070:> raise 2023-08-25T09:50:49.988 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:50:49.988 DEBUG:teuthology.orchestra.run.smithi070:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-08-25T09:50:49.989 DEBUG:teuthology.orchestra.run.smithi070:> full = True 2023-08-25T09:50:49.989 DEBUG:teuthology.orchestra.run.smithi070:> break 2023-08-25T09:50:49.989 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.990 DEBUG:teuthology.orchestra.run.smithi070:> try: 2023-08-25T09:50:49.990 DEBUG:teuthology.orchestra.run.smithi070:> os.fsync(f) 2023-08-25T09:50:49.990 DEBUG:teuthology.orchestra.run.smithi070:> print("fsync'"'"'ed successfully") 2023-08-25T09:50:49.990 DEBUG:teuthology.orchestra.run.smithi070:> except OSError as e: 2023-08-25T09:50:49.991 DEBUG:teuthology.orchestra.run.smithi070:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-08-25T09:50:49.991 DEBUG:teuthology.orchestra.run.smithi070:> full = True 2023-08-25T09:50:49.991 DEBUG:teuthology.orchestra.run.smithi070:> break 2023-08-25T09:50:49.992 DEBUG:teuthology.orchestra.run.smithi070:> else: 2023-08-25T09:50:49.992 DEBUG:teuthology.orchestra.run.smithi070:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-08-25T09:50:49.992 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.992 DEBUG:teuthology.orchestra.run.smithi070:> if n > 32 * 0.9: 2023-08-25T09:50:49.993 DEBUG:teuthology.orchestra.run.smithi070:> # Be cautious in the last region where we expect to hit 2023-08-25T09:50:49.993 DEBUG:teuthology.orchestra.run.smithi070:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-08-25T09:50:49.993 DEBUG:teuthology.orchestra.run.smithi070:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-08-25T09:50:49.994 DEBUG:teuthology.orchestra.run.smithi070:> time.sleep(15.0) 2023-08-25T09:50:49.994 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.994 DEBUG:teuthology.orchestra.run.smithi070:> if not full: 2023-08-25T09:50:49.995 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-08-25T09:50:49.995 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.995 DEBUG:teuthology.orchestra.run.smithi070:> # close() should not raise an error because we already caught it in 2023-08-25T09:50:49.996 DEBUG:teuthology.orchestra.run.smithi070:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-08-25T09:50:49.996 DEBUG:teuthology.orchestra.run.smithi070:> # since then because all IOs got cancelled on the full flag. 2023-08-25T09:50:49.996 DEBUG:teuthology.orchestra.run.smithi070:> print("calling close") 2023-08-25T09:50:49.997 DEBUG:teuthology.orchestra.run.smithi070:> os.close(f) 2023-08-25T09:50:49.997 DEBUG:teuthology.orchestra.run.smithi070:> print("close() did not raise error") 2023-08-25T09:50:49.997 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:50:49.997 DEBUG:teuthology.orchestra.run.smithi070:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-08-25T09:50:49.998 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:51:17.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:51:17.572 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:51:39.270 INFO:teuthology.orchestra.run.smithi070.stdout:writing some data through which we expect to succeed 2023-08-25T09:51:39.270 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-08-25T09:51:39.270 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.270 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.270 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 1.00 MB 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 2.00 MB 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 3.00 MB 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.271 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 4.00 MB 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 5.00 MB 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.272 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 6.00 MB 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 7.00 MB 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 8.00 MB 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.273 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 9.00 MB 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 10.00 MB 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 11.00 MB 2023-08-25T09:51:39.274 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 12.00 MB 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 13.00 MB 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.275 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 14.00 MB 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 15.00 MB 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 16.00 MB 2023-08-25T09:51:39.276 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 17.00 MB 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 18.00 MB 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.277 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 19.00 MB 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 20.00 MB 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 21.00 MB 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.278 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 22.00 MB 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 23.00 MB 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 24.00 MB 2023-08-25T09:51:39.279 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 25.00 MB 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 26.00 MB 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.280 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 27.00 MB 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 28.00 MB 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 29.00 MB 2023-08-25T09:51:39.281 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 30.00 MB 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 31.00 MB 2023-08-25T09:51:39.282 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:fsync'ed successfully 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:Not full yet after 32.00 MB 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:wrote bytes via buffered write, moving on to fsync 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:Reached fullness after 33.00 MB 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:calling close 2023-08-25T09:51:39.283 INFO:teuthology.orchestra.run.smithi070.stdout:close() did not raise error 2023-08-25T09:51:39.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:51:39.302 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:51:39.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:51:39.423 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:51:39.554 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:51:39.555 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:51:39.555 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:39.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:51:39.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.820+0000 7f12a2784640 1 Processor -- start 2023-08-25T09:51:39.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.820+0000 7f12a2784640 1 -- start start 2023-08-25T09:51:39.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.821+0000 7f12a2784640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:39.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.821+0000 7f12a2784640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f129c0b2850 0x7f129c06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:39.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.821+0000 7f12a2784640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129c0ceed0 con 0x7f129c15aa10 2023-08-25T09:51:39.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.821+0000 7f12a2784640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129c159d10 con 0x7f129c0b2850 2023-08-25T09:51:39.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.821+0000 7f12a2784640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c15d3f0 con 0x7f129c0ce5b0 2023-08-25T09:51:39.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f12a0d21640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f129c0b2850 0x7f129c06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:39.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f1293fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f129c15aa10 0x7f129c0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42124/0 (socket says 172.21.15.70:42124) 2023-08-25T09:51:39.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f1293fff640 1 -- 172.21.15.70:0/2210209751 learned_addr learned my addr 172.21.15.70:0/2210209751 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:39.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f129bfff640 1 --2- 172.21.15.70:0/2210209751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:39.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f12a0d21640 1 -- 172.21.15.70:0/2210209751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 msgr2=0x7f129c0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:39.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f12a0d21640 1 --2- 172.21.15.70:0/2210209751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:39.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f12a0d21640 1 -- 172.21.15.70:0/2210209751 >> v1:172.21.15.70:6789/0 conn(0x7f129c15aa10 legacy=0x7f129c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:39.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f12a0d21640 1 -- 172.21.15.70:0/2210209751 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c0b2d20 con 0x7f129c0b2850 2023-08-25T09:51:39.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.822+0000 7f129b7fe640 1 -- 172.21.15.70:0/2210209751 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554273369 0 0) 0x7f129c0ceed0 con 0x7f129c15aa10 2023-08-25T09:51:39.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.823+0000 7f12a0d21640 1 --2- 172.21.15.70:0/2210209751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c06f9f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1294009d90 tx=0x7f129402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f470d85069263e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:39.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.824+0000 7f129b7fe640 1 -- 172.21.15.70:0/2210209751 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129402cc50 con 0x7f129c0b2850 2023-08-25T09:51:39.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.824+0000 7f129b7fe640 1 -- 172.21.15.70:0/2210209751 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129402cdb0 con 0x7f129c0b2850 2023-08-25T09:51:39.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.824+0000 7f129b7fe640 1 -- 172.21.15.70:0/2210209751 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294034660 con 0x7f129c0b2850 2023-08-25T09:51:39.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.824+0000 7f12a2784640 1 -- 172.21.15.70:0/2210209751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 msgr2=0x7f129c06f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:39.826 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.824+0000 7f12a2784640 1 --2- 172.21.15.70:0/2210209751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c06f9f0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1294009d90 tx=0x7f129402bea0 comp rx=0 tx=0).stop 2023-08-25T09:51:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.825+0000 7f12a2784640 1 -- 172.21.15.70:0/2210209751 shutdown_connections 2023-08-25T09:51:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.825+0000 7f12a2784640 1 --2- 172.21.15.70:0/2210209751 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c06f9f0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.825+0000 7f12a2784640 1 --2- 172.21.15.70:0/2210209751 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.825+0000 7f12a2784640 1 -- 172.21.15.70:0/2210209751 >> 172.21.15.70:0/2210209751 conn(0x7f129c069850 msgr2=0x7f129c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.825+0000 7f12a2784640 1 -- 172.21.15.70:0/2210209751 shutdown_connections 2023-08-25T09:51:39.827 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.825+0000 7f12a2784640 1 -- 172.21.15.70:0/2210209751 wait complete. 2023-08-25T09:51:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.826+0000 7f12a2784640 1 Processor -- start 2023-08-25T09:51:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 -- start start 2023-08-25T09:51:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c14e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f129c15aa10 0x7f129c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c152ee0 con 0x7f129c0b2850 2023-08-25T09:51:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129c152610 con 0x7f129c15aa10 2023-08-25T09:51:39.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.827+0000 7f12a2784640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129c1503c0 con 0x7f129c0ce5b0 2023-08-25T09:51:39.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c14e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:39.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c14e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50912/0 (socket says 172.21.15.70:50912) 2023-08-25T09:51:39.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 -- 172.21.15.70:0/322551736 learned_addr learned my addr 172.21.15.70:0/322551736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:39.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f1293fff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:39.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 -- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 msgr2=0x7f129c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:39.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:39.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 -- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f129c15aa10 msgr2=0x7f129c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f129c15aa10 0x7f129c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f129bfff640 1 -- 172.21.15.70:0/322551736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1294009a70 con 0x7f129c0b2850 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.828+0000 7f1293fff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.829+0000 7f129bfff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c14e360 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f128c00d800 tx=0x7f128c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.829+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00b6c0 con 0x7f129c0b2850 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.829+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c00b840 con 0x7f129c0b2850 2023-08-25T09:51:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.829+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c00bbf0 con 0x7f129c0b2850 2023-08-25T09:51:39.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.829+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129c1500e0 con 0x7f129c0b2850 2023-08-25T09:51:39.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.829+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129c152b00 con 0x7f129c0b2850 2023-08-25T09:51:39.833 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.830+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129c153020 con 0x7f129c0b2850 2023-08-25T09:51:39.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.831+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f128c00bd50 con 0x7f129c0b2850 2023-08-25T09:51:39.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.832+0000 7f12997fa640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f127c06a050 0x7f127c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:39.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.832+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f128c088b60 con 0x7f129c0b2850 2023-08-25T09:51:39.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.832+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f127c072200 con 0x7f129c0b2850 2023-08-25T09:51:39.834 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.832+0000 7f1293fff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f127c06a050 0x7f127c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:39.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.833+0000 7f1293fff640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f127c06a050 0x7f127c06c510 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f129c14f560 tx=0x7f1284009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:39.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:39.837+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f129c153020 con 0x7f129c0b2850 2023-08-25T09:51:40.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.026+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f129c152cc0 con 0x7f129c0b2850 2023-08-25T09:51:40.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.029+0000 7f12997fa640 1 -- 172.21.15.70:0/322551736 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+18457 (secure 0 0 0) 0x7f128c057cc0 con 0x7f129c0b2850 2023-08-25T09:51:40.031 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:40.031 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":186,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:27.484002+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:51:40.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f127c06a050 msgr2=0x7f127c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f127c06a050 0x7f127c06c510 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f129c14f560 tx=0x7f1284009040 comp rx=0 tx=0).stop 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 msgr2=0x7f129c14e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c14e360 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f128c00d800 tx=0x7f128c00dcc0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 shutdown_connections 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f127c06a050 0x7f127c06c510 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f129c15aa10 0x7f129c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f129c0ce5b0 0x7f129c0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 --2- 172.21.15.70:0/322551736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f129c0b2850 0x7f129c14e360 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 >> 172.21.15.70:0/322551736 conn(0x7f129c069850 msgr2=0x7f129c069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 shutdown_connections 2023-08-25T09:51:40.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.032+0000 7f12a2784640 1 -- 172.21.15.70:0/322551736 wait complete. 2023-08-25T09:51:40.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:50:32.642193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.75, 'score_stable': 2.75, 'optimal_score': 1, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:50:33.991027+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:51:40.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:51:40.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.270+0000 7fd85ffff640 1 Processor -- start 2023-08-25T09:51:40.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.270+0000 7fd85ffff640 1 -- start start 2023-08-25T09:51:40.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 0x7fd8600c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85ffff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600b2170 con 0x7fd86015a920 2023-08-25T09:51:40.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85ffff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600adbb0 con 0x7fd8600b1ce0 2023-08-25T09:51:40.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600c90a0 con 0x7fd8600b0320 2023-08-25T09:51:40.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85e7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd86015a920 0x7fd8600b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42136/0 (socket says 172.21.15.70:42136) 2023-08-25T09:51:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85e7fc640 1 -- 172.21.15.70:0/2731511421 learned_addr learned my addr 172.21.15.70:0/2731511421 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.272+0000 7fd85effd640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 0x7fd8600c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6082495 0 0) 0x7fd8600b2170 con 0x7fd86015a920 2023-08-25T09:51:40.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd848003520 con 0x7fd86015a920 2023-08-25T09:51:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276648998 0 0) 0x7fd848003520 con 0x7fd86015a920 2023-08-25T09:51:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 msgr2=0x7fd8600c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 0x7fd8600c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 >> v1:172.21.15.155:6789/0 conn(0x7fd8600b1ce0 legacy=0x7fd8600ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600c9380 con 0x7fd86015a920 2023-08-25T09:51:40.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fd854001670 con 0x7fd86015a920 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 >> v1:172.21.15.70:6789/0 conn(0x7fd86015a920 legacy=0x7fd8600b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 0x7fd8480039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848003f20 0x7fd8480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd848004d00 0x7fd848005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600c90a0 con 0x7fd848003f20 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84800ba60 con 0x7fd848004d00 2023-08-25T09:51:40.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84800bc20 con 0x7fd8600b0320 2023-08-25T09:51:40.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.273+0000 7fd85e7fc640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848003f20 0x7fd8480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85effd640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 0x7fd8480039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85f7fe640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd848004d00 0x7fd848005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85e7fc640 1 -- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 msgr2=0x7fd8480039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:40.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85e7fc640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd8600b0320 0x7fd8480039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85e7fc640 1 -- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd848004d00 msgr2=0x7fd848005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:40.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85e7fc640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd848004d00 0x7fd848005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.274+0000 7fd85e7fc640 1 -- 172.21.15.70:0/2731511421 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848003520 con 0x7fd848003f20 2023-08-25T09:51:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.275+0000 7fd85e7fc640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848003f20 0x7fd8480047c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd85400c240 tx=0x7fd8480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581a47d2ea5b1cf6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.275+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd854005ad0 con 0x7fd848003f20 2023-08-25T09:51:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.275+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd854005c30 con 0x7fd848003f20 2023-08-25T09:51:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.275+0000 7fd85dffb640 1 -- 172.21.15.70:0/2731511421 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85402caf0 con 0x7fd848003f20 2023-08-25T09:51:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85ffff640 1 -- 172.21.15.70:0/2731511421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848003f20 msgr2=0x7fd8480047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85ffff640 1 --2- 172.21.15.70:0/2731511421 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd848003f20 0x7fd8480047c0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fd85400c240 tx=0x7fd8480057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85f7fe640 1 -- 172.21.15.70:0/2731511421 reap_dead start 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85ffff640 1 -- 172.21.15.70:0/2731511421 shutdown_connections 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85ffff640 1 -- 172.21.15.70:0/2731511421 >> 172.21.15.70:0/2731511421 conn(0x7fd8600ce830 msgr2=0x7fd8600cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85ffff640 1 -- 172.21.15.70:0/2731511421 shutdown_connections 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.276+0000 7fd85ffff640 1 -- 172.21.15.70:0/2731511421 wait complete. 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.278+0000 7fd85ffff640 1 Processor -- start 2023-08-25T09:51:40.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.278+0000 7fd85ffff640 1 -- start start 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd848003f20 0x7fd86015d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd860154390 0x7fd8600c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 0x7fd860070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85ffff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600c2600 con 0x7fd860070b30 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85ffff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600714f0 con 0x7fd860154390 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85ffff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600c3400 con 0x7fd848003f20 2023-08-25T09:51:40.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 0x7fd860070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 0x7fd860070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50940/0 (socket says 172.21.15.70:50940) 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.279+0000 7fd85f7fe640 1 -- 172.21.15.70:0/3284265474 learned_addr learned my addr 172.21.15.70:0/3284265474 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85effd640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd848003f20 0x7fd86015d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85e7fc640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd860154390 0x7fd8600c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85f7fe640 1 -- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd848003f20 msgr2=0x7fd86015d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85f7fe640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd848003f20 0x7fd86015d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85f7fe640 1 -- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd860154390 msgr2=0x7fd8600c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85f7fe640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd860154390 0x7fd8600c20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85f7fe640 1 -- 172.21.15.70:0/3284265474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600c27f0 con 0x7fd860070b30 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85f7fe640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 0x7fd860070fb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd86009c330 tx=0x7fd85800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858003b70 con 0x7fd860070b30 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858003cd0 con 0x7fd860070b30 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8580052b0 con 0x7fd860070b30 2023-08-25T09:51:40.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8600c2ad0 con 0x7fd860070b30 2023-08-25T09:51:40.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.280+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8600c3090 con 0x7fd860070b30 2023-08-25T09:51:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.281+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8600c4dc0 con 0x7fd860070b30 2023-08-25T09:51:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.283+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd858005410 con 0x7fd860070b30 2023-08-25T09:51:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.283+0000 7fd8477fe640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd84c06a050 0x7fd84c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.284+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd8580880b0 con 0x7fd860070b30 2023-08-25T09:51:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.284+0000 7fd85effd640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd84c06a050 0x7fd84c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.284+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fd84c072200 con 0x7fd860070b30 2023-08-25T09:51:40.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.285+0000 7fd85effd640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd84c06a050 0x7fd84c06c510 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd850009940 tx=0x7fd850008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:40.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.288+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd858018070 con 0x7fd860070b30 2023-08-25T09:51:40.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.473+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8600ac130 con 0x7fd860070b30 2023-08-25T09:51:40.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.474+0000 7fd8477fe640 1 -- 172.21.15.70:0/3284265474 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd858057170 con 0x7fd860070b30 2023-08-25T09:51:40.475 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:51:40.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd84c06a050 msgr2=0x7fd84c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd84c06a050 0x7fd84c06c510 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd850009940 tx=0x7fd850008040 comp rx=0 tx=0).stop 2023-08-25T09:51:40.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 msgr2=0x7fd860070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 0x7fd860070fb0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fd86009c330 tx=0x7fd85800dc00 comp rx=0 tx=0).stop 2023-08-25T09:51:40.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 shutdown_connections 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd84c06a050 0x7fd84c06c510 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd860070b30 0x7fd860070fb0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd860154390 0x7fd8600c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 --2- 172.21.15.70:0/3284265474 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd848003f20 0x7fd86015d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 >> 172.21.15.70:0/3284265474 conn(0x7fd8600ce830 msgr2=0x7fd8600ade70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 shutdown_connections 2023-08-25T09:51:40.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.476+0000 7fd85ffff640 1 -- 172.21.15.70:0/3284265474 wait complete. 2023-08-25T09:51:40.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:51:40.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.722+0000 7fd798a71640 1 Processor -- start 2023-08-25T09:51:40.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.722+0000 7fd798a71640 1 -- start start 2023-08-25T09:51:40.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd798a71640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd79415e210 0x7fd794069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd798a71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd798a71640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7940ac880 con 0x7fd794069850 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd798a71640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79415e650 con 0x7fd79415e210 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd798a71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940ad400 con 0x7fd7940cca20 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd78bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd79415e210 0x7fd794069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd78bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd79415e210 0x7fd794069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50942/0 (socket says 172.21.15.70:50942) 2023-08-25T09:51:40.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.724+0000 7fd78bfff640 1 -- 172.21.15.70:0/3829742650 learned_addr learned my addr 172.21.15.70:0/3829742650 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:40.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.725+0000 7fd792d9d640 1 --2- 172.21.15.70:0/3829742650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.725+0000 7fd78bfff640 1 -- 172.21.15.70:0/3829742650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 msgr2=0x7fd7940cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:40.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.725+0000 7fd78bfff640 1 --2- 172.21.15.70:0/3829742650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.725+0000 7fd78bfff640 1 -- 172.21.15.70:0/3829742650 >> v1:172.21.15.155:6789/0 conn(0x7fd794069850 legacy=0x7fd7940c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:40.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.725+0000 7fd78bfff640 1 -- 172.21.15.70:0/3829742650 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940ad890 con 0x7fd79415e210 2023-08-25T09:51:40.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.726+0000 7fd78bfff640 1 --2- 172.21.15.70:0/3829742650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd79415e210 0x7fd794069c30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009870 tx=0x7fd77c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac365ca39926924 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd791d9b640 1 -- 172.21.15.70:0/3829742650 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c030840 con 0x7fd79415e210 2023-08-25T09:51:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd791d9b640 1 -- 172.21.15.70:0/3829742650 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c0309a0 con 0x7fd79415e210 2023-08-25T09:51:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd791d9b640 1 -- 172.21.15.70:0/3829742650 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c030d30 con 0x7fd79415e210 2023-08-25T09:51:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd798a71640 1 -- 172.21.15.70:0/3829742650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd79415e210 msgr2=0x7fd794069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd798a71640 1 --2- 172.21.15.70:0/3829742650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd79415e210 0x7fd794069c30 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009870 tx=0x7fd77c02b980 comp rx=0 tx=0).stop 2023-08-25T09:51:40.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd798a71640 1 -- 172.21.15.70:0/3829742650 shutdown_connections 2023-08-25T09:51:40.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd798a71640 1 --2- 172.21.15.70:0/3829742650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd798a71640 1 --2- 172.21.15.70:0/3829742650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd79415e210 0x7fd794069c30 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.727+0000 7fd798a71640 1 -- 172.21.15.70:0/3829742650 >> 172.21.15.70:0/3829742650 conn(0x7fd7940cb0d0 msgr2=0x7fd7940cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:40.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.728+0000 7fd798a71640 1 -- 172.21.15.70:0/3829742650 shutdown_connections 2023-08-25T09:51:40.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.728+0000 7fd798a71640 1 -- 172.21.15.70:0/3829742650 wait complete. 2023-08-25T09:51:40.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.729+0000 7fd798a71640 1 Processor -- start 2023-08-25T09:51:40.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.729+0000 7fd798a71640 1 -- start start 2023-08-25T09:51:40.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.730+0000 7fd798a71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 0x7fd794146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.730+0000 7fd798a71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.730+0000 7fd798a71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79415e210 0x7fd7940c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.730+0000 7fd798a71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940c1d00 con 0x7fd794069850 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.730+0000 7fd798a71640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940af2f0 con 0x7fd79415e210 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.730+0000 7fd798a71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7940af5d0 con 0x7fd7940cca20 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 0x7fd794146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 0x7fd794146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50946/0 (socket says 172.21.15.70:50946) 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 -- 172.21.15.70:0/1954309984 learned_addr learned my addr 172.21.15.70:0/1954309984 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd792d9d640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79415e210 0x7fd7940c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 -- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 msgr2=0x7fd7940aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd78bfff640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 -- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79415e210 msgr2=0x7fd7940c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79415e210 0x7fd7940c3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 -- 172.21.15.70:0/1954309984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0094d0 con 0x7fd794069850 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.731+0000 7fd79259c640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 0x7fd794146c10 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd78400c430 tx=0x7fd78400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.732+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011790 con 0x7fd794069850 2023-08-25T09:51:40.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.732+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7840118f0 con 0x7fd794069850 2023-08-25T09:51:40.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.732+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011ce0 con 0x7fd794069850 2023-08-25T09:51:40.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.732+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7940af8b0 con 0x7fd794069850 2023-08-25T09:51:40.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.732+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd794147150 con 0x7fd794069850 2023-08-25T09:51:40.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.733+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7940c21e0 con 0x7fd794069850 2023-08-25T09:51:40.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.734+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd78401c460 con 0x7fd794069850 2023-08-25T09:51:40.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.735+0000 7fd78affd640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd77406a050 0x7fd77406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:40.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.735+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd7840884c0 con 0x7fd794069850 2023-08-25T09:51:40.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.735+0000 7fd78bfff640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd77406a050 0x7fd77406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:40.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.735+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fd7740721f0 con 0x7fd794069850 2023-08-25T09:51:40.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.736+0000 7fd78bfff640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd77406a050 0x7fd77406c510 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd77c036000 tx=0x7fd77c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:40.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.739+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd7940c21e0 con 0x7fd794069850 2023-08-25T09:51:40.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.922+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd7940c21e0 con 0x7fd794069850 2023-08-25T09:51:40.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.923+0000 7fd78affd640 1 -- 172.21.15.70:0/1954309984 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v186) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd7940c21e0 con 0x7fd794069850 2023-08-25T09:51:40.924 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:51:40.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd77406a050 msgr2=0x7fd77406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd77406a050 0x7fd77406c510 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd77c036000 tx=0x7fd77c03b040 comp rx=0 tx=0).stop 2023-08-25T09:51:40.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 msgr2=0x7fd794146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:40.927 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 0x7fd794146c10 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd78400c430 tx=0x7fd78400c8f0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 shutdown_connections 2023-08-25T09:51:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd77406a050 0x7fd77406c510 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd79415e210 0x7fd7940c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7940cca20 0x7fd7940aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 --2- 172.21.15.70:0/1954309984 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd794069850 0x7fd794146c10 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:40.928 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 >> 172.21.15.70:0/1954309984 conn(0x7fd7940cb0d0 msgr2=0x7fd79406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.925+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 shutdown_connections 2023-08-25T09:51:40.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:40.926+0000 7fd798a71640 1 -- 172.21.15.70:0/1954309984 wait complete. 2023-08-25T09:51:40.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:51:41.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.153+0000 7f2672924640 1 Processor -- start 2023-08-25T09:51:41.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.154+0000 7f2672924640 1 -- start start 2023-08-25T09:51:41.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.155+0000 7f2672924640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c0c2110 0x7f266c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.155+0000 7f2672924640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 0x7f266c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.155+0000 7f2672924640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c0c2550 con 0x7f266c15ae50 2023-08-25T09:51:41.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.155+0000 7f2672924640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c06a220 con 0x7f266c069d90 2023-08-25T09:51:41.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.155+0000 7f2672924640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0ce330 con 0x7f266c0c2110 2023-08-25T09:51:41.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.156+0000 7f2670ec1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 0x7f266c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.156+0000 7f2670ec1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 0x7f266c0ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:47924/0 (socket says 172.21.15.70:47924) 2023-08-25T09:51:41.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.156+0000 7f2670ec1640 1 -- 172.21.15.70:0/4189502354 learned_addr learned my addr 172.21.15.70:0/4189502354 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.156+0000 7f266b7fe640 1 --2- 172.21.15.70:0/4189502354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c0c2110 0x7f266c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.157+0000 7f266b7fe640 1 -- 172.21.15.70:0/4189502354 >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 msgr2=0x7f266c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.157+0000 7f266b7fe640 1 --2- 172.21.15.70:0/4189502354 >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 0x7f266c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.157+0000 7f266b7fe640 1 -- 172.21.15.70:0/4189502354 >> v1:172.21.15.155:6789/0 conn(0x7f266c15ae50 legacy=0x7f266c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.157+0000 7f266b7fe640 1 -- 172.21.15.70:0/4189502354 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0cc5f0 con 0x7f266c0c2110 2023-08-25T09:51:41.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.157+0000 7f2670ec1640 1 --2- 172.21.15.70:0/4189502354 >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 0x7f266c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:51:41.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.158+0000 7f266b7fe640 1 --2- 172.21.15.70:0/4189502354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c0c2110 0x7f266c069850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26540097a0 tx=0x7f265402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d445e00d1ddfe8d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:41.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f266affd640 1 -- 172.21.15.70:0/4189502354 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265402d040 con 0x7f266c0c2110 2023-08-25T09:51:41.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f266affd640 1 -- 172.21.15.70:0/4189502354 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654031810 con 0x7f266c0c2110 2023-08-25T09:51:41.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f266affd640 1 -- 172.21.15.70:0/4189502354 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654031ba0 con 0x7f266c0c2110 2023-08-25T09:51:41.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f2672924640 1 -- 172.21.15.70:0/4189502354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c0c2110 msgr2=0x7f266c069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f2672924640 1 --2- 172.21.15.70:0/4189502354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c0c2110 0x7f266c069850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f26540097a0 tx=0x7f265402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f2672924640 1 -- 172.21.15.70:0/4189502354 shutdown_connections 2023-08-25T09:51:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f2672924640 1 --2- 172.21.15.70:0/4189502354 >> v2:172.21.15.155:3300/0 conn(0x7f266c069d90 0x7f266c0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f2672924640 1 --2- 172.21.15.70:0/4189502354 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c0c2110 0x7f266c069850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.159+0000 7f2672924640 1 -- 172.21.15.70:0/4189502354 >> 172.21.15.70:0/4189502354 conn(0x7f266c0cb0d0 msgr2=0x7f266c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.160+0000 7f2672924640 1 -- 172.21.15.70:0/4189502354 shutdown_connections 2023-08-25T09:51:41.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.160+0000 7f2672924640 1 -- 172.21.15.70:0/4189502354 wait complete. 2023-08-25T09:51:41.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.162+0000 7f2672924640 1 Processor -- start 2023-08-25T09:51:41.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.162+0000 7f2672924640 1 -- start start 2023-08-25T09:51:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.163+0000 7f2672924640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c069d90 0x7f266c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.163+0000 7f2672924640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 0x7f266c0c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.163+0000 7f2672924640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c15ae50 0x7f266c148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.163+0000 7f2672924640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0c7160 con 0x7f266c0c2110 2023-08-25T09:51:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.163+0000 7f2672924640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0c7360 con 0x7f266c069d90 2023-08-25T09:51:41.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.163+0000 7f2672924640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0c7640 con 0x7f266c15ae50 2023-08-25T09:51:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 0x7f266c0c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 0x7f266c0c6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50948/0 (socket says 172.21.15.70:50948) 2023-08-25T09:51:41.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 -- 172.21.15.70:0/2750607143 learned_addr learned my addr 172.21.15.70:0/2750607143 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266bfff640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c069d90 0x7f266c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f2670ec1640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c15ae50 0x7f266c148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 -- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c15ae50 msgr2=0x7f266c148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c15ae50 0x7f266c148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 -- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c069d90 msgr2=0x7f266c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c069d90 0x7f266c0c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f266b7fe640 1 -- 172.21.15.70:0/2750607143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640094d0 con 0x7f266c0c2110 2023-08-25T09:51:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.164+0000 7f2670ec1640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c15ae50 0x7f266c148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.165+0000 7f266b7fe640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 0x7f266c0c6c20 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f26540098c0 tx=0x7f26540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.165+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654006e10 con 0x7f266c0c2110 2023-08-25T09:51:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.165+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265403ad60 con 0x7f266c0c2110 2023-08-25T09:51:41.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.165+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654044630 con 0x7f266c0c2110 2023-08-25T09:51:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.165+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2654009400 con 0x7f266c0c2110 2023-08-25T09:51:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.165+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c0c7c00 con 0x7f266c0c2110 2023-08-25T09:51:41.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.166+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c066fb0 con 0x7f266c0c2110 2023-08-25T09:51:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.167+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f265404d450 con 0x7f266c0c2110 2023-08-25T09:51:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.168+0000 7f2668ff9640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f264c06a050 0x7f264c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.168+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f26540aa8d0 con 0x7f266c0c2110 2023-08-25T09:51:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.168+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f264c072200 con 0x7f266c0c2110 2023-08-25T09:51:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.168+0000 7f266bfff640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f264c06a050 0x7f264c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.169+0000 7f266bfff640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f264c06a050 0x7f264c06c510 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f266c06cbb0 tx=0x7f265c0094d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:41.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.174+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2654075f90 con 0x7f266c0c2110 2023-08-25T09:51:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.362+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f266c0ccc70 con 0x7f266c0c2110 2023-08-25T09:51:41.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.363+0000 7f2668ff9640 1 -- 172.21.15.70:0/2750607143 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26540795b0 con 0x7f266c0c2110 2023-08-25T09:51:41.365 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:51:41.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.365+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f264c06a050 msgr2=0x7f264c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.365+0000 7f2672924640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f264c06a050 0x7f264c06c510 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f266c06cbb0 tx=0x7f265c0094d0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.365+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 msgr2=0x7f266c0c6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.365+0000 7f2672924640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 0x7f266c0c6c20 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f26540098c0 tx=0x7f26540046c0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.365+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 shutdown_connections 2023-08-25T09:51:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.365+0000 7f2672924640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f264c06a050 0x7f264c06c510 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.366+0000 7f2672924640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f266c15ae50 0x7f266c148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.366+0000 7f2672924640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f266c0c2110 0x7f266c0c6c20 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.366+0000 7f2672924640 1 --2- 172.21.15.70:0/2750607143 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f266c069d90 0x7f266c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.366+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 >> 172.21.15.70:0/2750607143 conn(0x7f266c0cb0d0 msgr2=0x7f266c0ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:41.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.366+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 shutdown_connections 2023-08-25T09:51:41.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.366+0000 7f2672924640 1 -- 172.21.15.70:0/2750607143 wait complete. 2023-08-25T09:51:41.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:51:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.598+0000 7fb2a2f14640 1 Processor -- start 2023-08-25T09:51:41.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.599+0000 7fb2a2f14640 1 -- start start 2023-08-25T09:51:41.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.600+0000 7fb2a2f14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.600+0000 7fb2a2f14640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb29c0b1ce0 0x7fb29c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.600+0000 7fb2a2f14640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29c0b2170 con 0x7fb29c0b0320 2023-08-25T09:51:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.600+0000 7fb2a2f14640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29c0ad8f0 con 0x7fb29c0b1ce0 2023-08-25T09:51:41.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.600+0000 7fb2a2f14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0adbb0 con 0x7fb29c15a920 2023-08-25T09:51:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.600+0000 7fb2a1f12640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb29c0b0320 0x7fb29c0b0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42148/0 (socket says 172.21.15.70:42148) 2023-08-25T09:51:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a1f12640 1 -- 172.21.15.70:0/3713324203 learned_addr learned my addr 172.21.15.70:0/3713324203 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a2713640 1 --2- 172.21.15.70:0/3713324203 >> v2:172.21.15.70:3300/0 conn(0x7fb29c0b1ce0 0x7fb29c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a1711640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a0f10640 1 -- 172.21.15.70:0/3713324203 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047619073 0 0) 0x7fb29c0b2170 con 0x7fb29c0b0320 2023-08-25T09:51:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a0f10640 1 -- 172.21.15.70:0/3713324203 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb288003520 con 0x7fb29c0b0320 2023-08-25T09:51:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a2713640 1 -- 172.21.15.70:0/3713324203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 msgr2=0x7fb29c0c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a2713640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a2713640 1 -- 172.21.15.70:0/3713324203 >> v1:172.21.15.70:6789/0 conn(0x7fb29c0b0320 legacy=0x7fb29c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a2713640 1 -- 172.21.15.70:0/3713324203 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0c90a0 con 0x7fb29c0b1ce0 2023-08-25T09:51:41.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.601+0000 7fb2a1711640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:51:41.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.603+0000 7fb2a2713640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b1ce0 0x7fb29c0ad3b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d90 tx=0x7fb29402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3286e945a45eb5b7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:41.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.603+0000 7fb2a0f10640 1 -- 172.21.15.70:0/3713324203 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29402cc50 con 0x7fb29c0b1ce0 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.603+0000 7fb2a0f10640 1 -- 172.21.15.70:0/3713324203 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29402cdb0 con 0x7fb29c0b1ce0 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.603+0000 7fb2a0f10640 1 -- 172.21.15.70:0/3713324203 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294034660 con 0x7fb29c0b1ce0 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 -- 172.21.15.70:0/3713324203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b1ce0 msgr2=0x7fb29c0ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b1ce0 0x7fb29c0ad3b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d90 tx=0x7fb29402bea0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 -- 172.21.15.70:0/3713324203 shutdown_connections 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b1ce0 0x7fb29c0ad3b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/3713324203 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 -- 172.21.15.70:0/3713324203 >> 172.21.15.70:0/3713324203 conn(0x7fb29c0ce830 msgr2=0x7fb29c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 -- 172.21.15.70:0/3713324203 shutdown_connections 2023-08-25T09:51:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.604+0000 7fb2a2f14640 1 -- 172.21.15.70:0/3713324203 wait complete. 2023-08-25T09:51:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.606+0000 7fb2a2f14640 1 Processor -- start 2023-08-25T09:51:41.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.606+0000 7fb2a2f14640 1 -- start start 2023-08-25T09:51:41.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a2f14640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 0x7fb29c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a2f14640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c0b1ce0 0x7fb29c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a2f14640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a2f14640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0c7e10 con 0x7fb29c0b0320 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a2f14640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0c30a0 con 0x7fb29c0b1ce0 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a2f14640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0c3380 con 0x7fb29c15a920 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 0x7fb29c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 0x7fb29c0ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50958/0 (socket says 172.21.15.70:50958) 2023-08-25T09:51:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 -- 172.21.15.70:0/1573557515 learned_addr learned my addr 172.21.15.70:0/1573557515 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 -- 172.21.15.70:0/1573557515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 msgr2=0x7fb29c0c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 -- 172.21.15.70:0/1573557515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c0b1ce0 msgr2=0x7fb29c0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c0b1ce0 0x7fb29c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.607+0000 7fb2a1f12640 1 -- 172.21.15.70:0/1573557515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009a70 con 0x7fb29c0b0320 2023-08-25T09:51:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.608+0000 7fb2a1f12640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 0x7fb29c0ae0e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fb298004b00 tx=0x7fb29800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.608+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298005ce0 con 0x7fb29c0b0320 2023-08-25T09:51:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.608+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb298005e40 con 0x7fb29c0b0320 2023-08-25T09:51:41.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.608+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298007270 con 0x7fb29c0b0320 2023-08-25T09:51:41.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.608+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c0c3660 con 0x7fb29c0b0320 2023-08-25T09:51:41.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.608+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0c3e40 con 0x7fb29c0b0320 2023-08-25T09:51:41.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.609+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c0c3fd0 con 0x7fb29c0b0320 2023-08-25T09:51:41.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.610+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb298007490 con 0x7fb29c0b0320 2023-08-25T09:51:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.611+0000 7fb28e7fc640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb28006a050 0x7fb28006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.611+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fb298087bd0 con 0x7fb29c0b0320 2023-08-25T09:51:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.611+0000 7fb2a1711640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb28006a050 0x7fb28006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.611+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fb280072200 con 0x7fb29c0b0320 2023-08-25T09:51:41.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.612+0000 7fb2a1711640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb28006a050 0x7fb28006c510 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb290006e80 tx=0x7fb290008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:41.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.615+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb298017020 con 0x7fb29c0b0320 2023-08-25T09:51:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.803+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb29c061460 con 0x7fb29c0b0320 2023-08-25T09:51:41.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.804+0000 7fb28e7fc640 1 -- 172.21.15.70:0/1573557515 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb298053290 con 0x7fb29c0b0320 2023-08-25T09:51:41.806 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:51:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb28006a050 msgr2=0x7fb28006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb28006a050 0x7fb28006c510 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fb290006e80 tx=0x7fb290008040 comp rx=0 tx=0).stop 2023-08-25T09:51:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 msgr2=0x7fb29c0ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 0x7fb29c0ae0e0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fb298004b00 tx=0x7fb29800c960 comp rx=0 tx=0).stop 2023-08-25T09:51:41.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 shutdown_connections 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb28006a050 0x7fb28006c510 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb29c15a920 0x7fb29c0c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb29c0b1ce0 0x7fb29c0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.806+0000 7fb2a2f14640 1 --2- 172.21.15.70:0/1573557515 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb29c0b0320 0x7fb29c0ae0e0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.807+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 >> 172.21.15.70:0/1573557515 conn(0x7fb29c0ce830 msgr2=0x7fb29c0af160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.807+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 shutdown_connections 2023-08-25T09:51:41.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:41.807+0000 7fb2a2f14640 1 -- 172.21.15.70:0/1573557515 wait complete. 2023-08-25T09:51:41.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:51:42.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.033+0000 7f19fb0ee640 1 Processor -- start 2023-08-25T09:51:42.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.033+0000 7f19fb0ee640 1 -- start start 2023-08-25T09:51:42.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.034+0000 7f19fb0ee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19f40c7a70 0x7f19f40ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.034+0000 7f19fb0ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40ca960 0x7f19f415d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19fb0ee640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f4161600 con 0x7f19f415ccf0 2023-08-25T09:51:42.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19fb0ee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40cae10 con 0x7f19f40c7a70 2023-08-25T09:51:42.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19fb0ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4071000 con 0x7f19f40ca960 2023-08-25T09:51:42.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f8e8a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f19f415ccf0 0x7f19f40b0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42158/0 (socket says 172.21.15.70:42158) 2023-08-25T09:51:42.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f3fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f19f40c7a70 0x7f19f40ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f8e8a640 1 -- 172.21.15.70:0/946288922 learned_addr learned my addr 172.21.15.70:0/946288922 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:42.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f3fff640 1 -- 172.21.15.70:0/946288922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40ca960 msgr2=0x7f19f415d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:42.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f968b640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40ca960 0x7f19f415d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f3fff640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40ca960 0x7f19f415d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f3fff640 1 -- 172.21.15.70:0/946288922 >> v1:172.21.15.70:6789/0 conn(0x7f19f415ccf0 legacy=0x7f19f40b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f3fff640 1 -- 172.21.15.70:0/946288922 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f41609f0 con 0x7f19f40c7a70 2023-08-25T09:51:42.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f968b640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40ca960 0x7f19f415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:42.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.035+0000 7f19f37fe640 1 -- 172.21.15.70:0/946288922 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486568438 0 0) 0x7f19f4161600 con 0x7f19f415ccf0 2023-08-25T09:51:42.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19f3fff640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40c7a70 0x7f19f40ca420 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e29410574334ebd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:42.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19f37fe640 1 -- 172.21.15.70:0/946288922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0031780 con 0x7f19f40c7a70 2023-08-25T09:51:42.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19f37fe640 1 -- 172.21.15.70:0/946288922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e00318e0 con 0x7f19f40c7a70 2023-08-25T09:51:42.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19f37fe640 1 -- 172.21.15.70:0/946288922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0031c70 con 0x7f19f40c7a70 2023-08-25T09:51:42.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19fb0ee640 1 -- 172.21.15.70:0/946288922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40c7a70 msgr2=0x7f19f40ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40c7a70 0x7f19f40ca420 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e002b820 comp rx=0 tx=0).stop 2023-08-25T09:51:42.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19fb0ee640 1 -- 172.21.15.70:0/946288922 shutdown_connections 2023-08-25T09:51:42.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40c7a70 0x7f19f40ca420 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.037+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/946288922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40ca960 0x7f19f415d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.038+0000 7f19fb0ee640 1 -- 172.21.15.70:0/946288922 >> 172.21.15.70:0/946288922 conn(0x7f19f40ce760 msgr2=0x7f19f40ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.038+0000 7f19fb0ee640 1 -- 172.21.15.70:0/946288922 shutdown_connections 2023-08-25T09:51:42.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.038+0000 7f19fb0ee640 1 -- 172.21.15.70:0/946288922 wait complete. 2023-08-25T09:51:42.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.039+0000 7f19fb0ee640 1 Processor -- start 2023-08-25T09:51:42.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.039+0000 7f19fb0ee640 1 -- start start 2023-08-25T09:51:42.042 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.040+0000 7f19fb0ee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40c7a70 0x7f19f4153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.040+0000 7f19fb0ee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 0x7f19f4150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.040+0000 7f19fb0ee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19f415ccf0 0x7f19f40b43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.040+0000 7f19fb0ee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4154050 con 0x7f19f40ca960 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.040+0000 7f19fb0ee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4155310 con 0x7f19f415ccf0 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.040+0000 7f19fb0ee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41555f0 con 0x7f19f40c7a70 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 0x7f19f4150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.043 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 0x7f19f4150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50974/0 (socket says 172.21.15.70:50974) 2023-08-25T09:51:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 -- 172.21.15.70:0/2713922695 learned_addr learned my addr 172.21.15.70:0/2713922695 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 -- 172.21.15.70:0/2713922695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40c7a70 msgr2=0x7f19f4153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40c7a70 0x7f19f4153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 -- 172.21.15.70:0/2713922695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19f415ccf0 msgr2=0x7f19f40b43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19f415ccf0 0x7f19f40b43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 -- 172.21.15.70:0/2713922695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0009400 con 0x7f19f40ca960 2023-08-25T09:51:42.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.041+0000 7f19f3fff640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 0x7f19f4150f80 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:42.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.042+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004230 con 0x7f19f40ca960 2023-08-25T09:51:42.045 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.042+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e00043b0 con 0x7f19f40ca960 2023-08-25T09:51:42.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.042+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004520 con 0x7f19f40ca960 2023-08-25T09:51:42.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.042+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f41558d0 con 0x7f19f40ca960 2023-08-25T09:51:42.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.042+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40b4910 con 0x7f19f40ca960 2023-08-25T09:51:42.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.043+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f4155df0 con 0x7f19f40ca960 2023-08-25T09:51:42.046 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.044+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f19e0032860 con 0x7f19f40ca960 2023-08-25T09:51:42.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.045+0000 7f19f17fa640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f19d806a050 0x7f19d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.045+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f19e00aa890 con 0x7f19f40ca960 2023-08-25T09:51:42.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.045+0000 7f19f8e8a640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f19d806a050 0x7f19d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.045+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f19d8072200 con 0x7f19f40ca960 2023-08-25T09:51:42.047 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.046+0000 7f19f8e8a640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f19d806a050 0x7f19d806c510 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f19e80010e0 tx=0x7f19e80077c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:42.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.049+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f19e0046e60 con 0x7f19f40ca960 2023-08-25T09:51:42.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.236+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f19f4161260 con 0x7f19f40ca960 2023-08-25T09:51:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.236+0000 7f19f17fa640 1 -- 172.21.15.70:0/2713922695 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v186) v1 ==== 80+0+0 (secure 0 0 0) 0x7f19e0046e60 con 0x7f19f40ca960 2023-08-25T09:51:42.238 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:51:42.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f19d806a050 msgr2=0x7f19d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f19d806a050 0x7f19d806c510 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f19e80010e0 tx=0x7f19e80077c0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 msgr2=0x7f19f4150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 0x7f19f4150f80 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f19e00097a0 tx=0x7f19e00046c0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 shutdown_connections 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f19d806a050 0x7f19d806c510 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f19f415ccf0 0x7f19f40b43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f19f40ca960 0x7f19f4150f80 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 --2- 172.21.15.70:0/2713922695 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f19f40c7a70 0x7f19f4153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 >> 172.21.15.70:0/2713922695 conn(0x7f19f40ce760 msgr2=0x7f19f40caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.239+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 shutdown_connections 2023-08-25T09:51:42.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.240+0000 7f19fb0ee640 1 -- 172.21.15.70:0/2713922695 wait complete. 2023-08-25T09:51:42.258 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:51:42.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:51:42.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:51:42.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[124460]: fuse finished with error 0 and tester_r 0 2023-08-25T09:51:42.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:51:42.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:51:42.347 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:51:42.466 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:51:42.467 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:42.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:51:42.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.752+0000 7f091b796640 1 Processor -- start 2023-08-25T09:51:42.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.753+0000 7f091b796640 1 -- start start 2023-08-25T09:51:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.754+0000 7f091b796640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091415e830 0x7f091406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.754+0000 7f091b796640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f091415ec10 0x7f09140b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f091b796640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09140b1b60 con 0x7f091406b2b0 2023-08-25T09:51:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f091b796640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f091406bd80 con 0x7f091415ec10 2023-08-25T09:51:42.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f091b796640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091406b7c0 con 0x7f091415e830 2023-08-25T09:51:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0919d33640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f091415ec10 0x7f09140b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0918d31640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f091406b2b0 0x7f091415c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42162/0 (socket says 172.21.15.70:42162) 2023-08-25T09:51:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0918d31640 1 -- 172.21.15.70:0/2006062018 learned_addr learned my addr 172.21.15.70:0/2006062018 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0919d33640 1 -- 172.21.15.70:0/2006062018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091415e830 msgr2=0x7f091406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0919d33640 1 --2- 172.21.15.70:0/2006062018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091415e830 0x7f091406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0919d33640 1 -- 172.21.15.70:0/2006062018 >> v1:172.21.15.70:6789/0 conn(0x7f091406b2b0 legacy=0x7f091415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f090bfff640 1 -- 172.21.15.70:0/2006062018 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3545916523 0 0) 0x7f09140b1b60 con 0x7f091406b2b0 2023-08-25T09:51:42.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.755+0000 7f0919d33640 1 -- 172.21.15.70:0/2006062018 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091406baa0 con 0x7f091415ec10 2023-08-25T09:51:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.756+0000 7f0919d33640 1 --2- 172.21.15.70:0/2006062018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09140b1ff0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d90 tx=0x7f090c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fae7727cf7d00e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f090bfff640 1 -- 172.21.15.70:0/2006062018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c02cc50 con 0x7f091415ec10 2023-08-25T09:51:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f090bfff640 1 -- 172.21.15.70:0/2006062018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c02cdb0 con 0x7f091415ec10 2023-08-25T09:51:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f090bfff640 1 -- 172.21.15.70:0/2006062018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c034660 con 0x7f091415ec10 2023-08-25T09:51:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 -- 172.21.15.70:0/2006062018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 msgr2=0x7f09140b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 --2- 172.21.15.70:0/2006062018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09140b1ff0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d90 tx=0x7f090c02bea0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 -- 172.21.15.70:0/2006062018 shutdown_connections 2023-08-25T09:51:42.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 --2- 172.21.15.70:0/2006062018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09140b1ff0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 --2- 172.21.15.70:0/2006062018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091415e830 0x7f091406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 -- 172.21.15.70:0/2006062018 >> 172.21.15.70:0/2006062018 conn(0x7f0914069850 msgr2=0x7f0914069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 -- 172.21.15.70:0/2006062018 shutdown_connections 2023-08-25T09:51:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.757+0000 7f091b796640 1 -- 172.21.15.70:0/2006062018 wait complete. 2023-08-25T09:51:42.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.759+0000 7f091b796640 1 Processor -- start 2023-08-25T09:51:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.759+0000 7f091b796640 1 -- start start 2023-08-25T09:51:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f091b796640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091406b2b0 0x7f09140c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f091b796640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091415e830 0x7f09141ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f091b796640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09141ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f091b796640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09140ca5e0 con 0x7f091415ec10 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f091b796640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0914145330 con 0x7f091415e830 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f091b796640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09140c5560 con 0x7f091406b2b0 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f0919d33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09141ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f0919d33640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09141ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:50996/0 (socket says 172.21.15.70:50996) 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.760+0000 7f0919d33640 1 -- 172.21.15.70:0/4172341481 learned_addr learned my addr 172.21.15.70:0/4172341481 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919532640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091406b2b0 0x7f09140c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919d33640 1 -- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091406b2b0 msgr2=0x7f09140c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919d33640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091406b2b0 0x7f09140c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0918d31640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091415e830 0x7f09141ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919d33640 1 -- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091415e830 msgr2=0x7f09141ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919d33640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091415e830 0x7f09141ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919d33640 1 -- 172.21.15.70:0/4172341481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c009a70 con 0x7f091415ec10 2023-08-25T09:51:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919532640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091406b2b0 0x7f09140c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0919d33640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09141ef8c0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d90 tx=0x7f090c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.761+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c03e610 con 0x7f091415ec10 2023-08-25T09:51:42.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.762+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c03e790 con 0x7f091415ec10 2023-08-25T09:51:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.762+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c03eb40 con 0x7f091415ec10 2023-08-25T09:51:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.762+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091415ca20 con 0x7f091415ec10 2023-08-25T09:51:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.762+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09141f3c00 con 0x7f091415ec10 2023-08-25T09:51:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.763+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09141454f0 con 0x7f091415ec10 2023-08-25T09:51:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.764+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f090c03b050 con 0x7f091415ec10 2023-08-25T09:51:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.764+0000 7f0909ffb640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08f806a050 0x7f08f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.765+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f090c0aafe0 con 0x7f091415ec10 2023-08-25T09:51:42.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.765+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f08f8072200 con 0x7f091415ec10 2023-08-25T09:51:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.765+0000 7f0919532640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08f806a050 0x7f08f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:42.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.765+0000 7f0919532640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08f806a050 0x7f08f806c510 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0910006fd0 tx=0x7f0910008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:42.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.769+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f090c039070 con 0x7f091415ec10 2023-08-25T09:51:42.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.958+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f091404e420 con 0x7f091415ec10 2023-08-25T09:51:42.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.962+0000 7f0909ffb640 1 -- 172.21.15.70:0/4172341481 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+18457 (secure 0 0 0) 0x7f090c04e030 con 0x7f091415ec10 2023-08-25T09:51:42.963 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:42.964 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":186,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:27.484002+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:51:42.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08f806a050 msgr2=0x7f08f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08f806a050 0x7f08f806c510 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0910006fd0 tx=0x7f0910008040 comp rx=0 tx=0).stop 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 msgr2=0x7f09141ef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09141ef8c0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f090c009d90 tx=0x7f090c004a40 comp rx=0 tx=0).stop 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 shutdown_connections 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f08f806a050 0x7f08f806c510 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f091415ec10 0x7f09141ef8c0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f091415e830 0x7f09141ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 --2- 172.21.15.70:0/4172341481 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f091406b2b0 0x7f09140c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:42.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 >> 172.21.15.70:0/4172341481 conn(0x7f0914069850 msgr2=0x7f091415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:42.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 shutdown_connections 2023-08-25T09:51:42.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:42.964+0000 7f091b796640 1 -- 172.21.15.70:0/4172341481 wait complete. 2023-08-25T09:51:42.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:50:32.642193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.75, 'score_stable': 2.75, 'optimal_score': 1, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:50:33.991027+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:51:42.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:51:43.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.202+0000 7fd7f600d640 1 Processor -- start 2023-08-25T09:51:43.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.202+0000 7fd7f600d640 1 -- start start 2023-08-25T09:51:43.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.204+0000 7fd7f600d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7f00c2110 0x7fd7f0069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.204+0000 7fd7f600d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd7f0069d90 0x7fd7f00ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.204+0000 7fd7f600d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f00c2550 con 0x7fd7f015ae50 2023-08-25T09:51:43.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.204+0000 7fd7f600d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f006a200 con 0x7fd7f00c2110 2023-08-25T09:51:43.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.204+0000 7fd7f600d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00ce330 con 0x7fd7f0069d90 2023-08-25T09:51:43.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.204+0000 7fd7eeffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7f00c2110 0x7fd7f0069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7eeffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd7f00c2110 0x7fd7f0069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51000/0 (socket says 172.21.15.70:51000) 2023-08-25T09:51:43.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7eeffd640 1 -- 172.21.15.70:0/3362392585 learned_addr learned my addr 172.21.15.70:0/3362392585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:43.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7effff640 1 --2- 172.21.15.70:0/3362392585 >> v2:172.21.15.155:3300/0 conn(0x7fd7f0069d90 0x7fd7f00ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7eeffd640 1 -- 172.21.15.70:0/3362392585 >> v2:172.21.15.155:3300/0 conn(0x7fd7f0069d90 msgr2=0x7fd7f00ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:43.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7eeffd640 1 --2- 172.21.15.70:0/3362392585 >> v2:172.21.15.155:3300/0 conn(0x7fd7f0069d90 0x7fd7f00ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7eeffd640 1 -- 172.21.15.70:0/3362392585 >> v1:172.21.15.155:6789/0 conn(0x7fd7f015ae50 legacy=0x7fd7f00cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:43.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.205+0000 7fd7eeffd640 1 -- 172.21.15.70:0/3362392585 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00cc5f0 con 0x7fd7f00c2110 2023-08-25T09:51:43.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.206+0000 7fd7eeffd640 1 --2- 172.21.15.70:0/3362392585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f0069850 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b00d0 tx=0x7fd7e402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e75aacba46a3447 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:43.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.207+0000 7fd7ee7fc640 1 -- 172.21.15.70:0/3362392585 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e402f850 con 0x7fd7f00c2110 2023-08-25T09:51:43.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.207+0000 7fd7ee7fc640 1 -- 172.21.15.70:0/3362392585 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e402f9b0 con 0x7fd7f00c2110 2023-08-25T09:51:43.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.207+0000 7fd7ee7fc640 1 -- 172.21.15.70:0/3362392585 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e402fd40 con 0x7fd7f00c2110 2023-08-25T09:51:43.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.207+0000 7fd7f600d640 1 -- 172.21.15.70:0/3362392585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 msgr2=0x7fd7f0069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.207+0000 7fd7f600d640 1 --2- 172.21.15.70:0/3362392585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f0069850 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b00d0 tx=0x7fd7e402ba30 comp rx=0 tx=0).stop 2023-08-25T09:51:43.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.208+0000 7fd7f600d640 1 -- 172.21.15.70:0/3362392585 shutdown_connections 2023-08-25T09:51:43.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.208+0000 7fd7f600d640 1 --2- 172.21.15.70:0/3362392585 >> v2:172.21.15.155:3300/0 conn(0x7fd7f0069d90 0x7fd7f00ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.208+0000 7fd7f600d640 1 --2- 172.21.15.70:0/3362392585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f0069850 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.208+0000 7fd7f600d640 1 -- 172.21.15.70:0/3362392585 >> 172.21.15.70:0/3362392585 conn(0x7fd7f00cb0d0 msgr2=0x7fd7f00cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:43.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.208+0000 7fd7f600d640 1 -- 172.21.15.70:0/3362392585 shutdown_connections 2023-08-25T09:51:43.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.208+0000 7fd7f600d640 1 -- 172.21.15.70:0/3362392585 wait complete. 2023-08-25T09:51:43.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.209+0000 7fd7f600d640 1 Processor -- start 2023-08-25T09:51:43.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.209+0000 7fd7f600d640 1 -- start start 2023-08-25T09:51:43.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.210+0000 7fd7f600d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7f0069d90 0x7fd7f00ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.210+0000 7fd7f600d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f00acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.210+0000 7fd7f600d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7f015ae50 0x7fd7f00c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.210+0000 7fd7f600d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c2830 con 0x7fd7f00c2110 2023-08-25T09:51:43.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.210+0000 7fd7f600d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c2b10 con 0x7fd7f015ae50 2023-08-25T09:51:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.210+0000 7fd7f600d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c2df0 con 0x7fd7f0069d90 2023-08-25T09:51:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f00acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f00acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51008/0 (socket says 172.21.15.70:51008) 2023-08-25T09:51:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 -- 172.21.15.70:0/2579204773 learned_addr learned my addr 172.21.15.70:0/2579204773 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7ef7fe640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7f0069d90 0x7fd7f00ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 -- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7f0069d90 msgr2=0x7fd7f00ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7f0069d90 0x7fd7f00ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 -- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7f015ae50 msgr2=0x7fd7f00c5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7f015ae50 0x7fd7f00c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 -- 172.21.15.70:0/2579204773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40094d0 con 0x7fd7f00c2110 2023-08-25T09:51:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.211+0000 7fd7eeffd640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f00acdc0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd7e402ba70 tx=0x7fd7e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:43.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.212+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4037d10 con 0x7fd7f00c2110 2023-08-25T09:51:43.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.212+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e4037e90 con 0x7fd7f00c2110 2023-08-25T09:51:43.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.212+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e4041640 con 0x7fd7f00c2110 2023-08-25T09:51:43.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.212+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f00c30d0 con 0x7fd7f00c2110 2023-08-25T09:51:43.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.212+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f0150960 con 0x7fd7f00c2110 2023-08-25T09:51:43.219 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.213+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f0150af0 con 0x7fd7f00c2110 2023-08-25T09:51:43.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.214+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd7e40417a0 con 0x7fd7f00c2110 2023-08-25T09:51:43.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.215+0000 7fd7d7fff640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7dc06a080 0x7fd7dc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.215+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fd7e403c070 con 0x7fd7f00c2110 2023-08-25T09:51:43.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.215+0000 7fd7ef7fe640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7dc06a080 0x7fd7dc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.215+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fd7dc072230 con 0x7fd7f00c2110 2023-08-25T09:51:43.221 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.216+0000 7fd7ef7fe640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7dc06a080 0x7fd7dc06c540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000f60 tx=0x7fd7e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:43.222 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.219+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd7e4076a90 con 0x7fd7f00c2110 2023-08-25T09:51:43.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.412+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7f0066fb0 con 0x7fd7f00c2110 2023-08-25T09:51:43.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.413+0000 7fd7d7fff640 1 -- 172.21.15.70:0/2579204773 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7e407a0b0 con 0x7fd7f00c2110 2023-08-25T09:51:43.414 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:51:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7dc06a080 msgr2=0x7fd7dc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7dc06a080 0x7fd7dc06c540 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0000f60 tx=0x7fd7e0009040 comp rx=0 tx=0).stop 2023-08-25T09:51:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 msgr2=0x7fd7f00acdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f00acdc0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd7e402ba70 tx=0x7fd7e4004730 comp rx=0 tx=0).stop 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 shutdown_connections 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd7dc06a080 0x7fd7dc06c540 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd7f015ae50 0x7fd7f00c5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd7f00c2110 0x7fd7f00acdc0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 --2- 172.21.15.70:0/2579204773 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd7f0069d90 0x7fd7f00ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 >> 172.21.15.70:0/2579204773 conn(0x7fd7f00cb0d0 msgr2=0x7fd7f00ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:43.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 shutdown_connections 2023-08-25T09:51:43.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.416+0000 7fd7f600d640 1 -- 172.21.15.70:0/2579204773 wait complete. 2023-08-25T09:51:43.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:51:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.655+0000 7f3bd6a5d640 1 Processor -- start 2023-08-25T09:51:43.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.655+0000 7f3bd6a5d640 1 -- start start 2023-08-25T09:51:43.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.656+0000 7f3bd6a5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bd00b2470 0x7f3bd0159930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.656+0000 7f3bd6a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00c8d20 0x7f3bd00c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.656+0000 7f3bd6a5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd0159e70 con 0x7f3bd00b2850 2023-08-25T09:51:43.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.656+0000 7f3bd6a5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd015ad20 con 0x7f3bd00b2470 2023-08-25T09:51:43.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.656+0000 7f3bd6a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00cd800 con 0x7f3bd00c8d20 2023-08-25T09:51:43.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd5a5b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bd00b2470 0x7f3bd0159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd525a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3bd00b2850 0x7f3bd00cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42170/0 (socket says 172.21.15.70:42170) 2023-08-25T09:51:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd5a5b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3bd00b2470 0x7f3bd0159930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51014/0 (socket says 172.21.15.70:51014) 2023-08-25T09:51:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd525a640 1 -- 172.21.15.70:0/1140895111 learned_addr learned my addr 172.21.15.70:0/1140895111 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd625c640 1 --2- 172.21.15.70:0/1140895111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00c8d20 0x7f3bd00c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd5a5b640 1 -- 172.21.15.70:0/1140895111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00c8d20 msgr2=0x7f3bd00c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd5a5b640 1 --2- 172.21.15.70:0/1140895111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00c8d20 0x7f3bd00c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd5a5b640 1 -- 172.21.15.70:0/1140895111 >> v1:172.21.15.70:6789/0 conn(0x7f3bd00b2850 legacy=0x7f3bd00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd5a5b640 1 -- 172.21.15.70:0/1140895111 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd015a110 con 0x7f3bd00b2470 2023-08-25T09:51:43.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.657+0000 7f3bd4a59640 1 -- 172.21.15.70:0/1140895111 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530984718 0 0) 0x7f3bd0159e70 con 0x7f3bd00b2850 2023-08-25T09:51:43.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.658+0000 7f3bd5a5b640 1 --2- 172.21.15.70:0/1140895111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2470 0x7f3bd0159930 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009790 tx=0x7f3bcc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0372ea9ace8eee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:43.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd4a59640 1 -- 172.21.15.70:0/1140895111 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc031800 con 0x7f3bd00b2470 2023-08-25T09:51:43.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd4a59640 1 -- 172.21.15.70:0/1140895111 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc031960 con 0x7f3bd00b2470 2023-08-25T09:51:43.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd4a59640 1 -- 172.21.15.70:0/1140895111 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc031cf0 con 0x7f3bd00b2470 2023-08-25T09:51:43.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/1140895111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2470 msgr2=0x7f3bd0159930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/1140895111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2470 0x7f3bd0159930 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009790 tx=0x7f3bcc02b8a0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/1140895111 shutdown_connections 2023-08-25T09:51:43.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/1140895111 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00c8d20 0x7f3bd00c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/1140895111 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2470 0x7f3bd0159930 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.659+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/1140895111 >> 172.21.15.70:0/1140895111 conn(0x7f3bd00ce830 msgr2=0x7f3bd00cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.660+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/1140895111 shutdown_connections 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.660+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/1140895111 wait complete. 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.661+0000 7f3bd6a5d640 1 Processor -- start 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.661+0000 7f3bd6a5d640 1 -- start start 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.662+0000 7f3bd6a5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00b2470 0x7f3bd0071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.662+0000 7f3bd6a5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 0x7f3bd015b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.662+0000 7f3bd6a5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bd00c8d20 0x7f3bd01551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.662+0000 7f3bd6a5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd015b6f0 con 0x7f3bd00b2850 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.662+0000 7f3bd6a5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd015b920 con 0x7f3bd00c8d20 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.662+0000 7f3bd6a5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd015bc00 con 0x7f3bd00b2470 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 0x7f3bd015b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 0x7f3bd015b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51024/0 (socket says 172.21.15.70:51024) 2023-08-25T09:51:43.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 -- 172.21.15.70:0/3540461679 learned_addr learned my addr 172.21.15.70:0/3540461679 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 -- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00b2470 msgr2=0x7f3bd0071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00b2470 0x7f3bd0071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 -- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bd00c8d20 msgr2=0x7f3bd01551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd625c640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bd00c8d20 0x7f3bd01551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bd00c8d20 0x7f3bd01551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 -- 172.21.15.70:0/3540461679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0093a0 con 0x7f3bd00b2850 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd625c640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bd00c8d20 0x7f3bd01551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.663+0000 7f3bd525a640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 0x7f3bd015b1b0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c4d0 tx=0x7f3bc000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.664+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00074e0 con 0x7f3bd00b2850 2023-08-25T09:51:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.664+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0007660 con 0x7f3bd00b2850 2023-08-25T09:51:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.664+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0004a20 con 0x7f3bd00b2850 2023-08-25T09:51:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.664+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd015bee0 con 0x7f3bd00b2850 2023-08-25T09:51:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.664+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd015c4a0 con 0x7f3bd00b2850 2023-08-25T09:51:43.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.665+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd00c82a0 con 0x7f3bd00b2850 2023-08-25T09:51:43.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.666+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3bc0004b80 con 0x7f3bd00b2850 2023-08-25T09:51:43.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.667+0000 7f3bc67fc640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3bb406a050 0x7f3bb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:43.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.667+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f3bc0087780 con 0x7f3bd00b2850 2023-08-25T09:51:43.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.668+0000 7f3bd5a5b640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3bb406a050 0x7f3bb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:43.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.668+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f3bb4072200 con 0x7f3bd00b2850 2023-08-25T09:51:43.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.668+0000 7f3bd5a5b640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3bb406a050 0x7f3bb406c510 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc036000 tx=0x7f3bcc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:43.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.672+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3bc0052e30 con 0x7f3bd00b2850 2023-08-25T09:51:43.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.864+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3bd0061460 con 0x7f3bd00b2850 2023-08-25T09:51:43.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.865+0000 7f3bc67fc640 1 -- 172.21.15.70:0/3540461679 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v186) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3bc0018030 con 0x7f3bd00b2850 2023-08-25T09:51:43.866 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:51:43.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3bb406a050 msgr2=0x7f3bb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3bb406a050 0x7f3bb406c510 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc036000 tx=0x7f3bcc030000 comp rx=0 tx=0).stop 2023-08-25T09:51:43.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 msgr2=0x7f3bd015b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:43.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 0x7f3bd015b1b0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000c4d0 tx=0x7f3bc000c990 comp rx=0 tx=0).stop 2023-08-25T09:51:43.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 shutdown_connections 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3bb406a050 0x7f3bb406c510 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3bd00c8d20 0x7f3bd01551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3bd00b2850 0x7f3bd015b1b0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 --2- 172.21.15.70:0/3540461679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3bd00b2470 0x7f3bd0071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.867+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 >> 172.21.15.70:0/3540461679 conn(0x7f3bd00ce830 msgr2=0x7f3bd00c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.868+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 shutdown_connections 2023-08-25T09:51:43.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:43.868+0000 7f3bd6a5d640 1 -- 172.21.15.70:0/3540461679 wait complete. 2023-08-25T09:51:43.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:51:44.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.101+0000 7f1ea5604640 1 Processor -- start 2023-08-25T09:51:44.103 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.102+0000 7f1ea5604640 1 -- start start 2023-08-25T09:51:44.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.103+0000 7f1ea5604640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f1ea015ae20 0x7f1ea0071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.104 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.103+0000 7f1ea5604640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea00cf9a0 con 0x7f1ea00b37c0 2023-08-25T09:51:44.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.103+0000 7f1ea5604640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea00cc9f0 con 0x7f1ea00cef50 2023-08-25T09:51:44.105 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.103+0000 7f1ea5604640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea015b200 con 0x7f1ea015ae20 2023-08-25T09:51:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.104+0000 7f1e9f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f1ea00b37c0 0x7f1ea00cce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42172/0 (socket says 172.21.15.70:42172) 2023-08-25T09:51:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.104+0000 7f1e9f7fe640 1 -- 172.21.15.70:0/633107132 learned_addr learned my addr 172.21.15.70:0/633107132 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.104+0000 7f1e9effd640 1 --2- 172.21.15.70:0/633107132 >> v2:172.21.15.155:3300/0 conn(0x7f1ea015ae20 0x7f1ea0071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.104+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303831026 0 0) 0x7f1ea00cf9a0 con 0x7f1ea00b37c0 2023-08-25T09:51:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.104+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e8c003520 con 0x7f1ea00b37c0 2023-08-25T09:51:44.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597836972 0 0) 0x7f1e8c003520 con 0x7f1ea00b37c0 2023-08-25T09:51:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 >> v2:172.21.15.155:3300/0 conn(0x7f1ea015ae20 msgr2=0x7f1ea0071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 --2- 172.21.15.70:0/633107132 >> v2:172.21.15.155:3300/0 conn(0x7f1ea015ae20 0x7f1ea0071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 >> v1:172.21.15.155:6789/0 conn(0x7f1ea00cef50 legacy=0x7f1ea00cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.108 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00cc3e0 con 0x7f1ea00b37c0 2023-08-25T09:51:44.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f1e98001670 con 0x7f1ea00b37c0 2023-08-25T09:51:44.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 >> v1:172.21.15.70:6789/0 conn(0x7f1ea00b37c0 legacy=0x7f1ea00cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c003970 0x7f1e8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e8c0047c0 0x7f1e8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c005180 0x7f1e8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c00c000 con 0x7f1e8c003970 2023-08-25T09:51:44.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c00c170 con 0x7f1e8c005180 2023-08-25T09:51:44.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c00c330 con 0x7f1e8c0047c0 2023-08-25T09:51:44.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9effd640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c003970 0x7f1e8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9f7fe640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c005180 0x7f1e8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9effd640 1 -- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e8c0047c0 msgr2=0x7f1e8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.110 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9effd640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1e8c0047c0 0x7f1e8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.105+0000 7f1e9effd640 1 -- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c005180 msgr2=0x7f1e8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.106+0000 7f1e9effd640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c005180 0x7f1e8c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.106+0000 7f1e9effd640 1 -- 172.21.15.70:0/633107132 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c003520 con 0x7f1e8c003970 2023-08-25T09:51:44.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.106+0000 7f1e9f7fe640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c005180 0x7f1e8c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:44.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.106+0000 7f1e9f7fe640 1 -- 172.21.15.70:0/633107132 reap_dead start 2023-08-25T09:51:44.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.107+0000 7f1e9effd640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c003970 0x7f1e8c003d90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009fc0 tx=0x7f1e8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acb5d99efb0924e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.107+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9002cc20 con 0x7f1e8c003970 2023-08-25T09:51:44.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.107+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9000f040 con 0x7f1e8c003970 2023-08-25T09:51:44.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.107+0000 7f1e9dffb640 1 -- 172.21.15.70:0/633107132 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90034440 con 0x7f1e8c003970 2023-08-25T09:51:44.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.107+0000 7f1ea5604640 1 -- 172.21.15.70:0/633107132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c003970 msgr2=0x7f1e8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.107+0000 7f1ea5604640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c003970 0x7f1e8c003d90 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009fc0 tx=0x7f1e8c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.108+0000 7f1ea5604640 1 -- 172.21.15.70:0/633107132 shutdown_connections 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.108+0000 7f1ea5604640 1 --2- 172.21.15.70:0/633107132 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1e8c003970 0x7f1e8c003d90 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.108+0000 7f1ea5604640 1 -- 172.21.15.70:0/633107132 >> 172.21.15.70:0/633107132 conn(0x7f1ea006ba60 msgr2=0x7f1ea006be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.108+0000 7f1ea5604640 1 -- 172.21.15.70:0/633107132 shutdown_connections 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.108+0000 7f1ea5604640 1 -- 172.21.15.70:0/633107132 wait complete. 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.109+0000 7f1ea5604640 1 Processor -- start 2023-08-25T09:51:44.114 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.110+0000 7f1ea5604640 1 -- start start 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1ea5604640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c003970 0x7f1ea00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1ea5604640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 0x7f1ea01f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1ea5604640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea015da60 0x7f1ea00b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1ea5604640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea01f2840 con 0x7f1ea00c6330 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1ea5604640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c68a0 con 0x7f1e8c003970 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1ea5604640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c6b80 con 0x7f1ea015da60 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 0x7f1ea01f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.115 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 0x7f1ea01f2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51042/0 (socket says 172.21.15.70:51042) 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 -- 172.21.15.70:0/2668979084 learned_addr learned my addr 172.21.15.70:0/2668979084 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9effd640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c003970 0x7f1ea00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9f7fe640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea015da60 0x7f1ea00b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 -- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea015da60 msgr2=0x7f1ea00b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea015da60 0x7f1ea00b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 -- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c003970 msgr2=0x7f1ea00b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c003970 0x7f1ea00b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.111+0000 7f1e9e7fc640 1 -- 172.21.15.70:0/2668979084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea01f24d0 con 0x7f1ea00c6330 2023-08-25T09:51:44.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.112+0000 7f1e9e7fc640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 0x7f1ea01f2cd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c500 tx=0x7f1e8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.112+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e880052d0 con 0x7f1ea00c6330 2023-08-25T09:51:44.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.112+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e88005430 con 0x7f1ea00c6330 2023-08-25T09:51:44.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.112+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88012870 con 0x7f1ea00c6330 2023-08-25T09:51:44.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.112+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea00b3230 con 0x7f1ea00c6330 2023-08-25T09:51:44.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.112+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea01f3210 con 0x7f1ea00c6330 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.113+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea01f2330 con 0x7f1ea00c6330 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.114+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1e880129d0 con 0x7f1ea00c6330 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.115+0000 7f1e977fe640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e8006a050 0x7f1e8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.115+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f1e88088210 con 0x7f1ea00c6330 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.115+0000 7f1e9effd640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e8006a050 0x7f1e8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.115+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f1e80072200 con 0x7f1ea00c6330 2023-08-25T09:51:44.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.116+0000 7f1e9effd640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e8006a050 0x7f1e8006c510 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1e90036000 tx=0x7f1e9002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.120+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1ea01f2330 con 0x7f1ea00c6330 2023-08-25T09:51:44.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.306+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1ea01f29f0 con 0x7f1ea00c6330 2023-08-25T09:51:44.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.307+0000 7f1e977fe640 1 -- 172.21.15.70:0/2668979084 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v186) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ea01f29f0 con 0x7f1ea00c6330 2023-08-25T09:51:44.308 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:51:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.309+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e8006a050 msgr2=0x7f1e8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.309+0000 7f1ea5604640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e8006a050 0x7f1e8006c510 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f1e90036000 tx=0x7f1e9002e000 comp rx=0 tx=0).stop 2023-08-25T09:51:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.309+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 msgr2=0x7f1ea01f2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.309+0000 7f1ea5604640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 0x7f1ea01f2cd0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800c500 tx=0x7f1e8800c9c0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 shutdown_connections 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f1e8006a050 0x7f1e8006c510 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f1ea015da60 0x7f1ea00b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1ea00c6330 0x7f1ea01f2cd0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 --2- 172.21.15.70:0/2668979084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1e8c003970 0x7f1ea00b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 >> 172.21.15.70:0/2668979084 conn(0x7f1ea006ba60 msgr2=0x7f1ea0071780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:44.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 shutdown_connections 2023-08-25T09:51:44.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.310+0000 7f1ea5604640 1 -- 172.21.15.70:0/2668979084 wait complete. 2023-08-25T09:51:44.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:51:44.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.549+0000 7f4413ea6640 1 Processor -- start 2023-08-25T09:51:44.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.549+0000 7f4413ea6640 1 -- start start 2023-08-25T09:51:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4413ea6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 0x7f440c1682f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4413ea6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0ac880 con 0x7f440c069850 2023-08-25T09:51:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4413ea6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0ad400 con 0x7f440c15e210 2023-08-25T09:51:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4413ea6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15e650 con 0x7f440c165f00 2023-08-25T09:51:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4411c42640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f440c069850 0x7f440c0c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42178/0 (socket says 172.21.15.70:42178) 2023-08-25T09:51:44.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4411c42640 1 -- 172.21.15.70:0/332215034 learned_addr learned my addr 172.21.15.70:0/332215034 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:44.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.551+0000 7f4412443640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 0x7f440c1682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93334741 0 0) 0x7f440c0ac880 con 0x7f440c069850 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f8003520 con 0x7f440c069850 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 641238848 0 0) 0x7f43f8003520 con 0x7f440c069850 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 msgr2=0x7f440c1682f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 0x7f440c1682f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 >> v1:172.21.15.155:6789/0 conn(0x7f440c15e210 legacy=0x7f440c069c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0ad890 con 0x7f440c069850 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4408001670 con 0x7f440c069850 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 >> v1:172.21.15.70:6789/0 conn(0x7f440c069850 legacy=0x7f440c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 0x7f43f80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43f8003f20 0x7f43f80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8004d00 0x7f43f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c15e650 con 0x7f43f8003f20 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800baf0 con 0x7f43f8004d00 2023-08-25T09:51:44.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.552+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800bcb0 con 0x7f440c165f00 2023-08-25T09:51:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411441640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43f8003f20 0x7f43f80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4412443640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8004d00 0x7f43f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411c42640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 0x7f43f80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411441640 1 -- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 msgr2=0x7f43f80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411441640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c165f00 0x7f43f80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411441640 1 -- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8004d00 msgr2=0x7f43f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411441640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8004d00 0x7f43f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.553+0000 7f4411441640 1 -- 172.21.15.70:0/332215034 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8003520 con 0x7f43f8003f20 2023-08-25T09:51:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.554+0000 7f4411441640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43f8003f20 0x7f43f80047c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f43fc001750 tx=0x7f43f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df38db599eb1eb7a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.554+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0052d0 con 0x7f43f8003f20 2023-08-25T09:51:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.554+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc02e050 con 0x7f43f8003f20 2023-08-25T09:51:44.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.554+0000 7f4410c40640 1 -- 172.21.15.70:0/332215034 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc003830 con 0x7f43f8003f20 2023-08-25T09:51:44.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4413ea6640 1 -- 172.21.15.70:0/332215034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43f8003f20 msgr2=0x7f43f80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4413ea6640 1 --2- 172.21.15.70:0/332215034 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f43f8003f20 0x7f43f80047c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f43fc001750 tx=0x7f43f80057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4412443640 1 -- 172.21.15.70:0/332215034 reap_dead start 2023-08-25T09:51:44.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4413ea6640 1 -- 172.21.15.70:0/332215034 shutdown_connections 2023-08-25T09:51:44.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4413ea6640 1 -- 172.21.15.70:0/332215034 >> 172.21.15.70:0/332215034 conn(0x7f440c0cb0d0 msgr2=0x7f440c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:44.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4413ea6640 1 -- 172.21.15.70:0/332215034 shutdown_connections 2023-08-25T09:51:44.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.555+0000 7f4413ea6640 1 -- 172.21.15.70:0/332215034 wait complete. 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.557+0000 7f4413ea6640 1 Processor -- start 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.557+0000 7f4413ea6640 1 -- start start 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4413ea6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8003f20 0x7f440c146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4413ea6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 0x7f440c150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4413ea6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c0c4420 0x7f440c151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4413ea6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c149470 con 0x7f440c06ebe0 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4413ea6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c149c30 con 0x7f43f8003f20 2023-08-25T09:51:44.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4413ea6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c152890 con 0x7f440c0c4420 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4411441640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 0x7f440c150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4411441640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 0x7f440c150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51062/0 (socket says 172.21.15.70:51062) 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.558+0000 7f4411441640 1 -- 172.21.15.70:0/2775393436 learned_addr learned my addr 172.21.15.70:0/2775393436 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4412443640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c0c4420 0x7f440c151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411c42640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8003f20 0x7f440c146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411441640 1 -- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c0c4420 msgr2=0x7f440c151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411441640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c0c4420 0x7f440c151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411441640 1 -- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8003f20 msgr2=0x7f440c146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411441640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8003f20 0x7f440c146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411441640 1 -- 172.21.15.70:0/2775393436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c152ae0 con 0x7f440c06ebe0 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411c42640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8003f20 0x7f440c146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.559+0000 7f4411441640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 0x7f440c150ed0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f43fc005040 tx=0x7f43fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.560+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc0076c0 con 0x7f440c06ebe0 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.560+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43fc007a30 con 0x7f440c06ebe0 2023-08-25T09:51:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.560+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43fc006c00 con 0x7f440c06ebe0 2023-08-25T09:51:44.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.560+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c14b920 con 0x7f440c06ebe0 2023-08-25T09:51:44.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.560+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c146e60 con 0x7f440c06ebe0 2023-08-25T09:51:44.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.561+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c14b780 con 0x7f440c06ebe0 2023-08-25T09:51:44.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.562+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f43fc0035f0 con 0x7f440c06ebe0 2023-08-25T09:51:44.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.563+0000 7f44027fc640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f43f006a080 0x7f43f006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.563+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f43fc0aa410 con 0x7f440c06ebe0 2023-08-25T09:51:44.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.563+0000 7f4411c42640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f43f006a080 0x7f43f006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.563+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f43f0072230 con 0x7f440c06ebe0 2023-08-25T09:51:44.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.564+0000 7f4411c42640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f43f006a080 0x7f43f006c540 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4408001050 tx=0x7f4408001d40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.568 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.567+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f440c14b780 con 0x7f440c06ebe0 2023-08-25T09:51:44.755 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.754+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f440c149de0 con 0x7f440c06ebe0 2023-08-25T09:51:44.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.755+0000 7f44027fc640 1 -- 172.21.15.70:0/2775393436 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v186) v1 ==== 100+0+11 (secure 0 0 0) 0x7f440c149de0 con 0x7f440c06ebe0 2023-08-25T09:51:44.758 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:51:44.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f43f006a080 msgr2=0x7f43f006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f43f006a080 0x7f43f006c540 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4408001050 tx=0x7f4408001d40 comp rx=0 tx=0).stop 2023-08-25T09:51:44.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 msgr2=0x7f440c150ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 0x7f440c150ed0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f43fc005040 tx=0x7f43fc001120 comp rx=0 tx=0).stop 2023-08-25T09:51:44.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 shutdown_connections 2023-08-25T09:51:44.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f43f006a080 0x7f43f006c540 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f440c0c4420 0x7f440c151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f440c06ebe0 0x7f440c150ed0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 --2- 172.21.15.70:0/2775393436 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f43f8003f20 0x7f440c146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 >> 172.21.15.70:0/2775393436 conn(0x7f440c0cb0d0 msgr2=0x7f440c14d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:44.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 shutdown_connections 2023-08-25T09:51:44.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.758+0000 7f4413ea6640 1 -- 172.21.15.70:0/2775393436 wait complete. 2023-08-25T09:51:44.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:51:44.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.988+0000 7f387359e640 1 Processor -- start 2023-08-25T09:51:44.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.989+0000 7f387359e640 1 -- start start 2023-08-25T09:51:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.990+0000 7f387359e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3874159930 0x7f38740b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.990+0000 7f387359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38740b3c80 0x7f38740caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.990+0000 7f387359e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3874159d10 con 0x7f38740cb030 2023-08-25T09:51:44.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.990+0000 7f387359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38740b2d10 con 0x7f38740b3c80 2023-08-25T09:51:44.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.990+0000 7f387359e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38740b2ff0 con 0x7f3874159930 2023-08-25T09:51:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f386bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38740b3c80 0x7f38740caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f3872d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f38740cb030 0x7f38740c8aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42192/0 (socket says 172.21.15.70:42192) 2023-08-25T09:51:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f3872d9d640 1 -- 172.21.15.70:0/2211816131 learned_addr learned my addr 172.21.15.70:0/2211816131 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f387259c640 1 --2- 172.21.15.70:0/2211816131 >> v2:172.21.15.155:3300/0 conn(0x7f3874159930 0x7f38740b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f3871d9b640 1 -- 172.21.15.70:0/2211816131 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3797578539 0 0) 0x7f3874159d10 con 0x7f38740cb030 2023-08-25T09:51:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f3871d9b640 1 -- 172.21.15.70:0/2211816131 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3860005730 con 0x7f38740cb030 2023-08-25T09:51:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f386bfff640 1 -- 172.21.15.70:0/2211816131 >> v2:172.21.15.155:3300/0 conn(0x7f3874159930 msgr2=0x7f38740b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f386bfff640 1 --2- 172.21.15.70:0/2211816131 >> v2:172.21.15.155:3300/0 conn(0x7f3874159930 0x7f38740b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f386bfff640 1 -- 172.21.15.70:0/2211816131 >> v1:172.21.15.70:6789/0 conn(0x7f38740cb030 legacy=0x7f38740c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.991+0000 7f386bfff640 1 -- 172.21.15.70:0/2211816131 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38740b32d0 con 0x7f38740b3c80 2023-08-25T09:51:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.992+0000 7f386bfff640 1 --2- 172.21.15.70:0/2211816131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740b3c80 0x7f38740caaf0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f38740b0ce0 tx=0x7f385c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ff10966d3b495c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.993+0000 7f3871d9b640 1 -- 172.21.15.70:0/2211816131 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c02e070 con 0x7f38740b3c80 2023-08-25T09:51:44.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.993+0000 7f3871d9b640 1 -- 172.21.15.70:0/2211816131 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c0315a0 con 0x7f38740b3c80 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.993+0000 7f3871d9b640 1 -- 172.21.15.70:0/2211816131 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c031910 con 0x7f38740b3c80 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.993+0000 7f387359e640 1 -- 172.21.15.70:0/2211816131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740b3c80 msgr2=0x7f38740caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.993+0000 7f387359e640 1 --2- 172.21.15.70:0/2211816131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740b3c80 0x7f38740caaf0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f38740b0ce0 tx=0x7f385c02b840 comp rx=0 tx=0).stop 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.994+0000 7f387359e640 1 -- 172.21.15.70:0/2211816131 shutdown_connections 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.994+0000 7f387359e640 1 --2- 172.21.15.70:0/2211816131 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740b3c80 0x7f38740caaf0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.994+0000 7f387359e640 1 --2- 172.21.15.70:0/2211816131 >> v2:172.21.15.155:3300/0 conn(0x7f3874159930 0x7f38740b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:44.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.994+0000 7f387359e640 1 -- 172.21.15.70:0/2211816131 >> 172.21.15.70:0/2211816131 conn(0x7f38740aff60 msgr2=0x7f38740b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.994+0000 7f387359e640 1 -- 172.21.15.70:0/2211816131 shutdown_connections 2023-08-25T09:51:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.994+0000 7f387359e640 1 -- 172.21.15.70:0/2211816131 wait complete. 2023-08-25T09:51:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.996+0000 7f387359e640 1 Processor -- start 2023-08-25T09:51:44.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.996+0000 7f387359e640 1 -- start start 2023-08-25T09:51:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387359e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38740b3c80 0x7f3874156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 0x7f38741e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387359e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3874159930 0x7f38741e6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38741572d0 con 0x7f38740cb030 2023-08-25T09:51:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387359e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c3830 con 0x7f38740b3c80 2023-08-25T09:51:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387359e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38740c3b10 con 0x7f3874159930 2023-08-25T09:51:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 0x7f38741e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 0x7f38741e65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51084/0 (socket says 172.21.15.70:51084) 2023-08-25T09:51:44.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 -- 172.21.15.70:0/1526878284 learned_addr learned my addr 172.21.15.70:0/1526878284 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:45.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f387259c640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38740b3c80 0x7f3874156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 -- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3874159930 msgr2=0x7f38741e6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3874159930 0x7f38741e6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 -- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38740b3c80 msgr2=0x7f3874156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38740b3c80 0x7f3874156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.997+0000 7f386bfff640 1 -- 172.21.15.70:0/1526878284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c009400 con 0x7f38740cb030 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.998+0000 7f386bfff640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 0x7f38741e65a0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f385c034040 tx=0x7f385c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.998+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c003810 con 0x7f38740cb030 2023-08-25T09:51:45.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.998+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c040070 con 0x7f38740cb030 2023-08-25T09:51:45.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.998+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38740c3df0 con 0x7f38740cb030 2023-08-25T09:51:45.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.998+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c003b70 con 0x7f38740cb030 2023-08-25T09:51:45.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.998+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38740c2ae0 con 0x7f38740cb030 2023-08-25T09:51:45.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:44.999+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387406f410 con 0x7f38740cb030 2023-08-25T09:51:45.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.000+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f385c003cd0 con 0x7f38740cb030 2023-08-25T09:51:45.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.002+0000 7f386affd640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f385406a050 0x7f385406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.002+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f385c042070 con 0x7f38740cb030 2023-08-25T09:51:45.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.002+0000 7f387259c640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f385406a050 0x7f385406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.002+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f3854072200 con 0x7f38740cb030 2023-08-25T09:51:45.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.003+0000 7f387259c640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f385406a050 0x7f385406c510 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3864006fd0 tx=0x7f3864008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:45.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.006+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f387406f410 con 0x7f38740cb030 2023-08-25T09:51:45.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.213+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f387406f410 con 0x7f38740cb030 2023-08-25T09:51:45.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.214+0000 7f386affd640 1 -- 172.21.15.70:0/1526878284 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v186) v1 ==== 80+0+0 (secure 0 0 0) 0x7f387406f410 con 0x7f38740cb030 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f385406a050 msgr2=0x7f385406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f385406a050 0x7f385406c510 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3864006fd0 tx=0x7f3864008040 comp rx=0 tx=0).stop 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 msgr2=0x7f38741e65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 0x7f38741e65a0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f385c034040 tx=0x7f385c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 shutdown_connections 2023-08-25T09:51:45.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f385406a050 0x7f385406c510 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3874159930 0x7f38741e6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38740cb030 0x7f38741e65a0 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 --2- 172.21.15.70:0/1526878284 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38740b3c80 0x7f3874156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 >> 172.21.15.70:0/1526878284 conn(0x7f38740aff60 msgr2=0x7f3874158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:45.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 shutdown_connections 2023-08-25T09:51:45.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.217+0000 7f387359e640 1 -- 172.21.15.70:0/1526878284 wait complete. 2023-08-25T09:51:45.285 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:51:45.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:51:45.286 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:51:45.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[93415]: fuse finished with error 0 and tester_r 0 2023-08-25T09:51:45.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:51:45.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:51:45.394 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:51:45.461 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:51:45.462 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:45.528 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:51:45.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:51:45.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:51:45.529 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:51:45.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:51:45.594 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:51:45.595 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:51:45.595 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:45.658 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:51:45.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:51:45.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:51:45.659 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:51:45.717 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:51:45.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:51:45.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:51:45.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.938+0000 7fc108da6640 1 Processor -- start 2023-08-25T09:51:45.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.938+0000 7fc108da6640 1 -- start start 2023-08-25T09:51:45.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.940+0000 7fc108da6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 0x7fc1040ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.940+0000 7fc108da6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1041615b0 con 0x7fc1040b2b40 2023-08-25T09:51:45.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.940+0000 7fc108da6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10406f050 con 0x7fc1040ac870 2023-08-25T09:51:45.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.940+0000 7fc108da6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10406f330 con 0x7fc10415cb40 2023-08-25T09:51:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.940+0000 7fc102d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc1040b2b40 0x7fc104160ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42208/0 (socket says 172.21.15.70:42208) 2023-08-25T09:51:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.940+0000 7fc102d9d640 1 -- 172.21.15.70:0/80542513 learned_addr learned my addr 172.21.15.70:0/80542513 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:45.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10259c640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 0x7fc1040ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761194324 0 0) 0x7fc1041615b0 con 0x7fc1040b2b40 2023-08-25T09:51:45.944 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0ec003520 con 0x7fc1040b2b40 2023-08-25T09:51:45.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640298457 0 0) 0x7fc0ec003520 con 0x7fc1040b2b40 2023-08-25T09:51:45.945 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 msgr2=0x7fc1040ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:45.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 0x7fc1040ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 >> v1:172.21.15.155:6789/0 conn(0x7fc1040ac870 legacy=0x7fc104160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10406f610 con 0x7fc1040b2b40 2023-08-25T09:51:45.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc0fc0019f0 con 0x7fc1040b2b40 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 >> v1:172.21.15.70:6789/0 conn(0x7fc1040b2b40 legacy=0x7fc104160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec0039a0 0x7fc0ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 0x7fc0ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0ec004d00 0x7fc0ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10406f330 con 0x7fc0ec004d00 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec00ba30 con 0x7fc0ec0039a0 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.941+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec00bbf0 con 0x7fc10415cb40 2023-08-25T09:51:45.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc102d9d640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0ec004d00 0x7fc0ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc10259c640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec0039a0 0x7fc0ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc102d9d640 1 -- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 msgr2=0x7fc0ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:45.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc102d9d640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc10415cb40 0x7fc0ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc102d9d640 1 -- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec0039a0 msgr2=0x7fc0ec003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:45.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc102d9d640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec0039a0 0x7fc0ec003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.942+0000 7fc102d9d640 1 -- 172.21.15.70:0/80542513 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec003520 con 0x7fc0ec004d00 2023-08-25T09:51:45.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.943+0000 7fc102d9d640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0ec004d00 0x7fc0ec005180 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc1040b3830 tx=0x7fc0ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9ef2ef0845f6d3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:45.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0072d0 con 0x7fc0ec004d00 2023-08-25T09:51:45.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc02f070 con 0x7fc0ec004d00 2023-08-25T09:51:45.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc10159a640 1 -- 172.21.15.70:0/80542513 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0076b0 con 0x7fc0ec004d00 2023-08-25T09:51:45.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc108da6640 1 -- 172.21.15.70:0/80542513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0ec004d00 msgr2=0x7fc0ec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc108da6640 1 --2- 172.21.15.70:0/80542513 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc0ec004d00 0x7fc0ec005180 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc1040b3830 tx=0x7fc0ec0057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc102d9d640 1 -- 172.21.15.70:0/80542513 reap_dead start 2023-08-25T09:51:45.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc108da6640 1 -- 172.21.15.70:0/80542513 shutdown_connections 2023-08-25T09:51:45.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.944+0000 7fc108da6640 1 -- 172.21.15.70:0/80542513 >> 172.21.15.70:0/80542513 conn(0x7fc104069850 msgr2=0x7fc104069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:45.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.945+0000 7fc108da6640 1 -- 172.21.15.70:0/80542513 shutdown_connections 2023-08-25T09:51:45.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.945+0000 7fc108da6640 1 -- 172.21.15.70:0/80542513 wait complete. 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.946+0000 7fc108da6640 1 Processor -- start 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.946+0000 7fc108da6640 1 -- start start 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.947+0000 7fc108da6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec004d00 0x7fc104150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.947+0000 7fc108da6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 0x7fc10414c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.947+0000 7fc108da6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc104152cf0 0x7fc1041531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.947+0000 7fc108da6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041536e0 con 0x7fc10414e880 2023-08-25T09:51:45.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.947+0000 7fc108da6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10414ed00 con 0x7fc0ec004d00 2023-08-25T09:51:45.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.947+0000 7fc108da6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10415d520 con 0x7fc104152cf0 2023-08-25T09:51:45.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 0x7fc10414c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 0x7fc10414c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51098/0 (socket says 172.21.15.70:51098) 2023-08-25T09:51:45.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 -- 172.21.15.70:0/373885734 learned_addr learned my addr 172.21.15.70:0/373885734 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:45.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 -- 172.21.15.70:0/373885734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc104152cf0 msgr2=0x7fc1041531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc104152cf0 0x7fc1041531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 -- 172.21.15.70:0/373885734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec004d00 msgr2=0x7fc104150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:45.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec004d00 0x7fc104150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:45.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 -- 172.21.15.70:0/373885734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10414ef80 con 0x7fc10414e880 2023-08-25T09:51:45.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.948+0000 7fc101d9b640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 0x7fc10414c9d0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c480 tx=0x7fc0f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:45.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.949+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80052d0 con 0x7fc10414e880 2023-08-25T09:51:45.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.949+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f8005470 con 0x7fc10414e880 2023-08-25T09:51:45.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.949+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8003830 con 0x7fc10414e880 2023-08-25T09:51:45.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.949+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc10414f260 con 0x7fc10414e880 2023-08-25T09:51:45.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.949+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc10414f820 con 0x7fc10414e880 2023-08-25T09:51:45.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.950+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10415d850 con 0x7fc10414e880 2023-08-25T09:51:45.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.951+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc0f8003990 con 0x7fc10414e880 2023-08-25T09:51:45.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.952+0000 7fc0f2ffd640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc0e406a080 0x7fc0e406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:45.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.952+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fc0f80879e0 con 0x7fc10414e880 2023-08-25T09:51:45.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.952+0000 7fc10259c640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc0e406a080 0x7fc0e406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:45.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.952+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fc0e4072230 con 0x7fc10414e880 2023-08-25T09:51:45.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.952+0000 7fc10259c640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc0e406a080 0x7fc0e406c540 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc104154840 tx=0x7fc0f4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:45.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:45.956+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc0f8017060 con 0x7fc10414e880 2023-08-25T09:51:46.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.185+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc10414f9b0 con 0x7fc10414e880 2023-08-25T09:51:46.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.186+0000 7fc0f2ffd640 1 -- 172.21.15.70:0/373885734 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fc0f8053150 con 0x7fc10414e880 2023-08-25T09:51:46.188 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:46.188 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:51:46.188 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:51:46.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.188+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc0e406a080 msgr2=0x7fc0e406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.188+0000 7fc108da6640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc0e406a080 0x7fc0e406c540 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fc104154840 tx=0x7fc0f4008040 comp rx=0 tx=0).stop 2023-08-25T09:51:46.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.188+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 msgr2=0x7fc10414c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.188+0000 7fc108da6640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 0x7fc10414c9d0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800c480 tx=0x7fc0f800c940 comp rx=0 tx=0).stop 2023-08-25T09:51:46.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.188+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 shutdown_connections 2023-08-25T09:51:46.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc0e406a080 0x7fc0e406c540 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc104152cf0 0x7fc1041531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc10414e880 0x7fc10414c9d0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 --2- 172.21.15.70:0/373885734 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc0ec004d00 0x7fc104150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 >> 172.21.15.70:0/373885734 conn(0x7fc104069850 msgr2=0x7fc10406c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:46.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 shutdown_connections 2023-08-25T09:51:46.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.189+0000 7fc108da6640 1 -- 172.21.15.70:0/373885734 wait complete. 2023-08-25T09:51:46.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:51:46.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.426+0000 7f00396ec640 1 Processor -- start 2023-08-25T09:51:46.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.426+0000 7f00396ec640 1 -- start start 2023-08-25T09:51:46.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.428+0000 7f00396ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f003415e830 0x7f003406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.428+0000 7f00396ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.428+0000 7f00396ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00340b1b60 con 0x7f003406b2b0 2023-08-25T09:51:46.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.428+0000 7f00396ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003406bd80 con 0x7f003415e830 2023-08-25T09:51:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.428+0000 7f00396ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003406b7c0 con 0x7f003415ec10 2023-08-25T09:51:46.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f0032ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f003415e830 0x7f003406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:42796/0 (socket says 172.21.15.70:42796) 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 -- 172.21.15.70:0/1274057679 learned_addr learned my addr 172.21.15.70:0/1274057679 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f002bfff640 1 --1- 172.21.15.70:0/1274057679 >> v1:172.21.15.155:6789/0 conn(0x7f003406b2b0 0x7f003415c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:47628/0 (socket says 172.21.15.70:47628) 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 -- 172.21.15.70:0/1274057679 >> v2:172.21.15.155:3300/0 conn(0x7f003415e830 msgr2=0x7f003406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 --2- 172.21.15.70:0/1274057679 >> v2:172.21.15.155:3300/0 conn(0x7f003415e830 0x7f003406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 -- 172.21.15.70:0/1274057679 >> v1:172.21.15.155:6789/0 conn(0x7f003406b2b0 legacy=0x7f003415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.429+0000 7f00337fe640 1 -- 172.21.15.70:0/1274057679 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003406baa0 con 0x7f003415ec10 2023-08-25T09:51:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.431+0000 7f00337fe640 1 --2- 172.21.15.70:0/1274057679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340b1ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f002c009e50 tx=0x7f002c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=113f7e0cf9db6a1a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:46.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.431+0000 7f00327fc640 1 -- 172.21.15.70:0/1274057679 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c02cb00 con 0x7f003415ec10 2023-08-25T09:51:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00327fc640 1 -- 172.21.15.70:0/1274057679 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c02cc60 con 0x7f003415ec10 2023-08-25T09:51:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00327fc640 1 -- 172.21.15.70:0/1274057679 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c0343f0 con 0x7f003415ec10 2023-08-25T09:51:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 -- 172.21.15.70:0/1274057679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 msgr2=0x7f00340b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 --2- 172.21.15.70:0/1274057679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340b1ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f002c009e50 tx=0x7f002c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:51:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 -- 172.21.15.70:0/1274057679 shutdown_connections 2023-08-25T09:51:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 --2- 172.21.15.70:0/1274057679 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340b1ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 --2- 172.21.15.70:0/1274057679 >> v2:172.21.15.155:3300/0 conn(0x7f003415e830 0x7f003406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 -- 172.21.15.70:0/1274057679 >> 172.21.15.70:0/1274057679 conn(0x7f0034069850 msgr2=0x7f0034069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:46.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 -- 172.21.15.70:0/1274057679 shutdown_connections 2023-08-25T09:51:46.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.432+0000 7f00396ec640 1 -- 172.21.15.70:0/1274057679 wait complete. 2023-08-25T09:51:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.435+0000 7f00396ec640 1 Processor -- start 2023-08-25T09:51:46.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.435+0000 7f00396ec640 1 -- start start 2023-08-25T09:51:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f00396ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003406b2b0 0x7f00341553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f00396ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 0x7f0034152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f00396ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f00396ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0034154f30 con 0x7f003415e830 2023-08-25T09:51:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f00396ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0034154040 con 0x7f003406b2b0 2023-08-25T09:51:46.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f00396ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0034154320 con 0x7f003415ec10 2023-08-25T09:51:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f002bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 0x7f0034152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f002bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 0x7f0034152d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51114/0 (socket says 172.21.15.70:51114) 2023-08-25T09:51:46.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.436+0000 7f002bfff640 1 -- 172.21.15.70:0/3129692656 learned_addr learned my addr 172.21.15.70:0/3129692656 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:46.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f0032ffd640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003406b2b0 0x7f00341553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f00337fe640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002bfff640 1 -- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 msgr2=0x7f00340aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002bfff640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002bfff640 1 -- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003406b2b0 msgr2=0x7f00341553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002bfff640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003406b2b0 0x7f00341553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002bfff640 1 -- 172.21.15.70:0/3129692656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009b30 con 0x7f003415e830 2023-08-25T09:51:46.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002bfff640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 0x7f0034152d60 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f001c00c4d0 tx=0x7f001c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.437+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c007480 con 0x7f003415e830 2023-08-25T09:51:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.438+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c0075e0 con 0x7f003415e830 2023-08-25T09:51:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.438+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c0049e0 con 0x7f003415e830 2023-08-25T09:51:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.438+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003415ca20 con 0x7f003415e830 2023-08-25T09:51:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.438+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00341548e0 con 0x7f003415e830 2023-08-25T09:51:46.442 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.439+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00341550a0 con 0x7f003415e830 2023-08-25T09:51:46.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.440+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f001c004b40 con 0x7f003415e830 2023-08-25T09:51:46.443 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.440+0000 7f002b7fe640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f001406a050 0x7f001406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.441+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f001c0876b0 con 0x7f003415e830 2023-08-25T09:51:46.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.441+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f0014072200 con 0x7f003415e830 2023-08-25T09:51:46.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.441+0000 7f0032ffd640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f001406a050 0x7f001406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.444 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.441+0000 7f0032ffd640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f001406a050 0x7f001406c510 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0024009ad0 tx=0x7f0024008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:46.446 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.445+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f001c052da0 con 0x7f003415e830 2023-08-25T09:51:46.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.637+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f003404e420 con 0x7f003415e830 2023-08-25T09:51:46.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.638+0000 7f002b7fe640 1 -- 172.21.15.70:0/3129692656 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) v1 ==== 74+0+18457 (secure 0 0 0) 0x7f001c018020 con 0x7f003415e830 2023-08-25T09:51:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:46.640 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":186,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:27.484002+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:51:46.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f001406a050 msgr2=0x7f001406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f001406a050 0x7f001406c510 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0024009ad0 tx=0x7f0024008040 comp rx=0 tx=0).stop 2023-08-25T09:51:46.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 msgr2=0x7f0034152d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 0x7f0034152d60 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f001c00c4d0 tx=0x7f001c00c990 comp rx=0 tx=0).stop 2023-08-25T09:51:46.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 shutdown_connections 2023-08-25T09:51:46.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f001406a050 0x7f001406c510 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f003415ec10 0x7f00340aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f003415e830 0x7f0034152d60 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 --2- 172.21.15.70:0/3129692656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f003406b2b0 0x7f00341553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.641+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 >> 172.21.15.70:0/3129692656 conn(0x7f0034069850 msgr2=0x7f00340b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:46.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.642+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 shutdown_connections 2023-08-25T09:51:46.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.642+0000 7f00396ec640 1 -- 172.21.15.70:0/3129692656 wait complete. 2023-08-25T09:51:46.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:50:32.642193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '181', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.75, 'score_stable': 2.75, 'optimal_score': 1, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:50:33.991027+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:51:46.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:51:46.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.875+0000 7f2cc0257640 1 Processor -- start 2023-08-25T09:51:46.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.876+0000 7f2cc0257640 1 -- start start 2023-08-25T09:51:46.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.877+0000 7f2cc0257640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2cb80d1ba0 0x7f2cb80cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.877+0000 7f2cc0257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.877+0000 7f2cc0257640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80b7140 con 0x7f2cb815ccd0 2023-08-25T09:51:46.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.877+0000 7f2cc0257640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb80cfe10 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.877+0000 7f2cc0257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80b7420 con 0x7f2cb80cf980 2023-08-25T09:51:46.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.878+0000 7f2cbdff3640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2cb815ccd0 0x7f2cb80b8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:47636/0 (socket says 172.21.15.70:47636) 2023-08-25T09:51:46.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.878+0000 7f2cbdff3640 1 -- 172.21.15.70:0/911454231 learned_addr learned my addr 172.21.15.70:0/911454231 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:46.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.878+0000 7f2cbe7f4640 1 --2- 172.21.15.70:0/911454231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.878+0000 7f2cbd7f2640 1 --2- 172.21.15.70:0/911454231 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80d1ba0 0x7f2cb80cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.879+0000 7f2cbcff1640 1 -- 172.21.15.70:0/911454231 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496223218 0 0) 0x7f2cb80b7140 con 0x7f2cb815ccd0 2023-08-25T09:51:46.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.879+0000 7f2cbcff1640 1 -- 172.21.15.70:0/911454231 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca0003520 con 0x7f2cb815ccd0 2023-08-25T09:51:46.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.879+0000 7f2cbe7f4640 1 -- 172.21.15.70:0/911454231 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80d1ba0 msgr2=0x7f2cb80cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:46.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.879+0000 7f2cbe7f4640 1 --2- 172.21.15.70:0/911454231 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80d1ba0 0x7f2cb80cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.879+0000 7f2cbe7f4640 1 -- 172.21.15.70:0/911454231 >> v1:172.21.15.155:6789/0 conn(0x7f2cb815ccd0 legacy=0x7f2cb80b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.879+0000 7f2cbe7f4640 1 -- 172.21.15.70:0/911454231 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80b7700 con 0x7f2cb80cf980 2023-08-25T09:51:46.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.880+0000 7f2cbe7f4640 1 --2- 172.21.15.70:0/911454231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80b5a50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097a0 tx=0x7f2cb002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deeba09d037526f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:46.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.881+0000 7f2cbcff1640 1 -- 172.21.15.70:0/911454231 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0030610 con 0x7f2cb80cf980 2023-08-25T09:51:46.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.881+0000 7f2cbcff1640 1 -- 172.21.15.70:0/911454231 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0030770 con 0x7f2cb80cf980 2023-08-25T09:51:46.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.881+0000 7f2cbcff1640 1 -- 172.21.15.70:0/911454231 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0030ae0 con 0x7f2cb80cf980 2023-08-25T09:51:46.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.881+0000 7f2cc0257640 1 -- 172.21.15.70:0/911454231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 msgr2=0x7f2cb80b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.881+0000 7f2cc0257640 1 --2- 172.21.15.70:0/911454231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80b5a50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2cb00097a0 tx=0x7f2cb002b960 comp rx=0 tx=0).stop 2023-08-25T09:51:46.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.882+0000 7f2cc0257640 1 -- 172.21.15.70:0/911454231 shutdown_connections 2023-08-25T09:51:46.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.882+0000 7f2cc0257640 1 --2- 172.21.15.70:0/911454231 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80b5a50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.882+0000 7f2cc0257640 1 --2- 172.21.15.70:0/911454231 >> v2:172.21.15.155:3300/0 conn(0x7f2cb80d1ba0 0x7f2cb80cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.882+0000 7f2cc0257640 1 -- 172.21.15.70:0/911454231 >> 172.21.15.70:0/911454231 conn(0x7f2cb8070260 msgr2=0x7f2cb8070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:46.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.882+0000 7f2cc0257640 1 -- 172.21.15.70:0/911454231 shutdown_connections 2023-08-25T09:51:46.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.882+0000 7f2cc0257640 1 -- 172.21.15.70:0/911454231 wait complete. 2023-08-25T09:51:46.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.883+0000 7f2cc0257640 1 Processor -- start 2023-08-25T09:51:46.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.884+0000 7f2cc0257640 1 -- start start 2023-08-25T09:51:46.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.884+0000 7f2cc0257640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cc0257640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 0x7f2cb8150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cc0257640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ccd0 0x7f2cb80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cc0257640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80c8e10 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cc0257640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80ccf10 con 0x7f2cb815ccd0 2023-08-25T09:51:46.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cc0257640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb815d350 con 0x7f2cb80cf980 2023-08-25T09:51:46.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 0x7f2cb8150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 0x7f2cb8150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51116/0 (socket says 172.21.15.70:51116) 2023-08-25T09:51:46.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 -- 172.21.15.70:0/2319998201 learned_addr learned my addr 172.21.15.70:0/2319998201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:46.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbe7f4640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ccd0 0x7f2cb80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbdff3640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 -- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 msgr2=0x7f2cb80c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 -- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ccd0 msgr2=0x7f2cb80cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:46.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ccd0 0x7f2cb80cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:46.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.885+0000 7f2cbd7f2640 1 -- 172.21.15.70:0/2319998201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb0009400 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.886+0000 7f2cbd7f2640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 0x7f2cb8150aa0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e9b0 tx=0x7f2cac00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:46.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.886+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac009670 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.886+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac00c9a0 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.886+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb8151470 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.886+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac00cd50 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.886+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb80c8850 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.887+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb815d490 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.888+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2cac013470 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.897 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.889+0000 7f2cb67fc640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ca406a050 0x7f2ca406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:46.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.889+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f2cac0888b0 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.889+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7f2ca4072200 con 0x7f2cb80d1ba0 2023-08-25T09:51:46.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.889+0000 7f2cbdff3640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ca406a050 0x7f2ca406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:46.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.890+0000 7f2cbdff3640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ca406a050 0x7f2ca406c510 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80012e0 tx=0x7f2ca8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:46.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:46.893+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2cb815d490 con 0x7f2cb80d1ba0 2023-08-25T09:51:47.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.080+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cb815d490 con 0x7f2cb80d1ba0 2023-08-25T09:51:47.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.082+0000 7f2cb67fc640 1 -- 172.21.15.70:0/2319998201 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v186) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cb815d490 con 0x7f2cb80d1ba0 2023-08-25T09:51:47.084 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:51:47.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ca406a050 msgr2=0x7f2ca406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ca406a050 0x7f2ca406c510 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2ca80012e0 tx=0x7f2ca8009040 comp rx=0 tx=0).stop 2023-08-25T09:51:47.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 msgr2=0x7f2cb8150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 0x7f2cb8150aa0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00e9b0 tx=0x7f2cac00ee70 comp rx=0 tx=0).stop 2023-08-25T09:51:47.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 shutdown_connections 2023-08-25T09:51:47.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2ca406a050 0x7f2ca406c510 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2cb815ccd0 0x7f2cb80cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2cb80d1ba0 0x7f2cb8150aa0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 --2- 172.21.15.70:0/2319998201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2cb80cf980 0x7f2cb80c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.084+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 >> 172.21.15.70:0/2319998201 conn(0x7f2cb8070260 msgr2=0x7f2cb8157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:47.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.085+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 shutdown_connections 2023-08-25T09:51:47.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.085+0000 7f2cc0257640 1 -- 172.21.15.70:0/2319998201 wait complete. 2023-08-25T09:51:47.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:51:47.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.323+0000 7fe7064f3640 1 Processor -- start 2023-08-25T09:51:47.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.324+0000 7fe7064f3640 1 -- start start 2023-08-25T09:51:47.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.325+0000 7fe7064f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe700162910 0x7fe700164d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.325+0000 7fe7064f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000abd30 con 0x7fe7000ce940 2023-08-25T09:51:47.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.325+0000 7fe7064f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000cdb90 con 0x7fe7000ad820 2023-08-25T09:51:47.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.325+0000 7fe7064f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7000adc00 con 0x7fe700162910 2023-08-25T09:51:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.325+0000 7fe6ff7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe7000ce940 0x7fe70015dd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42224/0 (socket says 172.21.15.70:42224) 2023-08-25T09:51:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.325+0000 7fe6ff7fe640 1 -- 172.21.15.70:0/3255983713 learned_addr learned my addr 172.21.15.70:0/3255983713 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:47.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe704a90640 1 --2- 172.21.15.70:0/3255983713 >> v2:172.21.15.155:3300/0 conn(0x7fe700162910 0x7fe700164d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230059469 0 0) 0x7fe7000abd30 con 0x7fe7000ce940 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ec003520 con 0x7fe7000ce940 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1691296747 0 0) 0x7fe6ec003520 con 0x7fe7000ce940 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 >> v2:172.21.15.155:3300/0 conn(0x7fe700162910 msgr2=0x7fe700164d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 --2- 172.21.15.70:0/3255983713 >> v2:172.21.15.155:3300/0 conn(0x7fe700162910 0x7fe700164d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 >> v1:172.21.15.155:6789/0 conn(0x7fe7000ad820 legacy=0x7fe70015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe700159f00 con 0x7fe7000ce940 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe6e8001670 con 0x7fe7000ce940 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 >> v1:172.21.15.70:6789/0 conn(0x7fe7000ce940 legacy=0x7fe70015dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.326+0000 7fe6feffd640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec003970 0x7fe6ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6feffd640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0047c0 0x7fe6ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6feffd640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec005180 0x7fe6ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00c0f0 con 0x7fe6ec005180 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00c260 con 0x7fe6ec003970 2023-08-25T09:51:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00c420 con 0x7fe6ec0047c0 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe704a90640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec005180 0x7fe6ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6ff7fe640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0047c0 0x7fe6ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6fffff640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec003970 0x7fe6ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe704a90640 1 -- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0047c0 msgr2=0x7fe6ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe704a90640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0047c0 0x7fe6ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe704a90640 1 -- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec003970 msgr2=0x7fe6ec003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:47.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe704a90640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec003970 0x7fe6ec003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe704a90640 1 -- 172.21.15.70:0/3255983713 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec003520 con 0x7fe6ec005180 2023-08-25T09:51:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6ff7fe640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6ec0047c0 0x7fe6ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:47.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.327+0000 7fe6fffff640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec003970 0x7fe6ec003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:47.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.328+0000 7fe704a90640 1 -- 172.21.15.70:0/3255983713 reap_dead start 2023-08-25T09:51:47.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.328+0000 7fe704a90640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec005180 0x7fe6ec00b980 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009a90 tx=0x7fe6ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afb1bc0abcbe13f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.329+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f802c6b0 con 0x7fe6ec005180 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.329+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f802c810 con 0x7fe6ec005180 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.329+0000 7fe6feffd640 1 -- 172.21.15.70:0/3255983713 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f802cbc0 con 0x7fe6ec005180 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.329+0000 7fe7064f3640 1 -- 172.21.15.70:0/3255983713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec005180 msgr2=0x7fe6ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.329+0000 7fe7064f3640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec005180 0x7fe6ec00b980 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009a90 tx=0x7fe6ec0057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.330+0000 7fe7064f3640 1 -- 172.21.15.70:0/3255983713 shutdown_connections 2023-08-25T09:51:47.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.330+0000 7fe7064f3640 1 --2- 172.21.15.70:0/3255983713 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6ec005180 0x7fe6ec00b980 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.330+0000 7fe7064f3640 1 -- 172.21.15.70:0/3255983713 >> 172.21.15.70:0/3255983713 conn(0x7fe7000cb960 msgr2=0x7fe7000cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.330+0000 7fe7064f3640 1 -- 172.21.15.70:0/3255983713 shutdown_connections 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.330+0000 7fe7064f3640 1 -- 172.21.15.70:0/3255983713 wait complete. 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.331+0000 7fe7064f3640 1 Processor -- start 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.332+0000 7fe7064f3640 1 -- start start 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.332+0000 7fe7064f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec005180 0x7fe7000c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe7064f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe70009b8a0 0x7fe70009c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe7064f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 0x7fe70015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe7064f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000c68e0 con 0x7fe7001513a0 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe7064f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe700151dc0 con 0x7fe6ec005180 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe7064f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70009c890 con 0x7fe70009b8a0 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 0x7fe70015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 0x7fe70015c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51134/0 (socket says 172.21.15.70:51134) 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 -- 172.21.15.70:0/406015650 learned_addr learned my addr 172.21.15.70:0/406015650 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:47.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 -- 172.21.15.70:0/406015650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe70009b8a0 msgr2=0x7fe70009c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe70009b8a0 0x7fe70009c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 -- 172.21.15.70:0/406015650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec005180 msgr2=0x7fe7000c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec005180 0x7fe7000c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.333+0000 7fe704a90640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000c5fb0 con 0x7fe7001513a0 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.334+0000 7fe704a90640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 0x7fe70015c360 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009bb0 tx=0x7fe6f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.334+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80359a0 con 0x7fe7001513a0 2023-08-25T09:51:47.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.334+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8035b00 con 0x7fe7001513a0 2023-08-25T09:51:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.334+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8035e90 con 0x7fe7001513a0 2023-08-25T09:51:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.334+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7000c6290 con 0x7fe7001513a0 2023-08-25T09:51:47.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.334+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe700151820 con 0x7fe7001513a0 2023-08-25T09:51:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.335+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7001519e0 con 0x7fe7001513a0 2023-08-25T09:51:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.336+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe6f803b040 con 0x7fe7001513a0 2023-08-25T09:51:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.337+0000 7fe6fcff9640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6e006a070 0x7fe6e006c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.337+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fe6f80a9440 con 0x7fe7001513a0 2023-08-25T09:51:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.337+0000 7fe6fffff640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6e006a070 0x7fe6e006c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.337+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=187}) v3 -- 0x7fe6e0072220 con 0x7fe7001513a0 2023-08-25T09:51:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.338+0000 7fe6fffff640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6e006a070 0x7fe6e006c530 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe7001628e0 tx=0x7fe6f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:47.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.341+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6f8074b00 con 0x7fe7001513a0 2023-08-25T09:51:47.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.495+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(187..187 src has 1..187) v4 ==== 708+0+0 (secure 0 0 0) 0x7fe6f8078e50 con 0x7fe7001513a0 2023-08-25T09:51:47.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.495+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=188}) v3 -- 0x7fe6e0072db0 con 0x7fe7001513a0 2023-08-25T09:51:47.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.541+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe700066fb0 con 0x7fe7001513a0 2023-08-25T09:51:47.543 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.542+0000 7fe6fcff9640 1 -- 172.21.15.70:0/406015650 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v187) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6f8039070 con 0x7fe7001513a0 2023-08-25T09:51:47.543 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:51:47.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.544+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6e006a070 msgr2=0x7fe6e006c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.544+0000 7fe7064f3640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6e006a070 0x7fe6e006c530 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe7001628e0 tx=0x7fe6f0009040 comp rx=0 tx=0).stop 2023-08-25T09:51:47.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.544+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 msgr2=0x7fe70015c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.544+0000 7fe7064f3640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 0x7fe70015c360 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009bb0 tx=0x7fe6f8004730 comp rx=0 tx=0).stop 2023-08-25T09:51:47.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.544+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 shutdown_connections 2023-08-25T09:51:47.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe6e006a070 0x7fe6e006c530 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7001513a0 0x7fe70015c360 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe70009b8a0 0x7fe70009c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 --2- 172.21.15.70:0/406015650 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6ec005180 0x7fe7000c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 >> 172.21.15.70:0/406015650 conn(0x7fe7000cb960 msgr2=0x7fe70006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:47.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 shutdown_connections 2023-08-25T09:51:47.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.545+0000 7fe7064f3640 1 -- 172.21.15.70:0/406015650 wait complete. 2023-08-25T09:51:47.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:51:47.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:51:47.724 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:51:47.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.777+0000 7f312aefa640 1 Processor -- start 2023-08-25T09:51:47.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.778+0000 7f312aefa640 1 -- start start 2023-08-25T09:51:47.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f312aefa640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f312415e210 0x7f3124069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f312aefa640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31240ac880 con 0x7f31240cca20 2023-08-25T09:51:47.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f312aefa640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31240ad400 con 0x7f3124069850 2023-08-25T09:51:47.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f312aefa640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312415e650 con 0x7f312415e210 2023-08-25T09:51:47.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f3128c96640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3124069850 0x7f31240c9620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:47642/0 (socket says 172.21.15.70:47642) 2023-08-25T09:51:47.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f3128c96640 1 -- 172.21.15.70:0/3364785668 learned_addr learned my addr 172.21.15.70:0/3364785668 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.779+0000 7f3123fff640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f312415e210 0x7f3124069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.780+0000 7f3123fff640 1 -- 172.21.15.70:0/3364785668 >> v1:172.21.15.155:6789/0 conn(0x7f3124069850 legacy=0x7f31240c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.780+0000 7f3123fff640 1 -- 172.21.15.70:0/3364785668 >> v1:172.21.15.70:6789/0 conn(0x7f31240cca20 legacy=0x7f31240cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.780+0000 7f3123fff640 1 -- 172.21.15.70:0/3364785668 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31240ad890 con 0x7f312415e210 2023-08-25T09:51:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.781+0000 7f3123fff640 1 --2- 172.21.15.70:0/3364785668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312415e210 0x7f3124069c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31100097a0 tx=0x7f311002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecfeb93eda78819 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.781+0000 7f31237fe640 1 -- 172.21.15.70:0/3364785668 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311002f810 con 0x7f312415e210 2023-08-25T09:51:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f31237fe640 1 -- 172.21.15.70:0/3364785668 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311002f970 con 0x7f312415e210 2023-08-25T09:51:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f31237fe640 1 -- 172.21.15.70:0/3364785668 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311002fd00 con 0x7f312415e210 2023-08-25T09:51:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 -- 172.21.15.70:0/3364785668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312415e210 msgr2=0x7f3124069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 --2- 172.21.15.70:0/3364785668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312415e210 0x7f3124069c30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f31100097a0 tx=0x7f311002b9f0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 -- 172.21.15.70:0/3364785668 shutdown_connections 2023-08-25T09:51:47.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 --2- 172.21.15.70:0/3364785668 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f312415e210 0x7f3124069c30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 -- 172.21.15.70:0/3364785668 >> 172.21.15.70:0/3364785668 conn(0x7f31240cb0d0 msgr2=0x7f31240cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 -- 172.21.15.70:0/3364785668 shutdown_connections 2023-08-25T09:51:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.782+0000 7f312aefa640 1 -- 172.21.15.70:0/3364785668 wait complete. 2023-08-25T09:51:47.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.784+0000 7f312aefa640 1 Processor -- start 2023-08-25T09:51:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.784+0000 7f312aefa640 1 -- start start 2023-08-25T09:51:47.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f312aefa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 0x7f31241f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f312aefa640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31240cca20 0x7f31240c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f312aefa640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f312415e210 0x7f31240b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f312aefa640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31241f02c0 con 0x7f3124069850 2023-08-25T09:51:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f312aefa640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31240b3e60 con 0x7f31240cca20 2023-08-25T09:51:47.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f312aefa640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31240b4280 con 0x7f312415e210 2023-08-25T09:51:47.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.785+0000 7f3128c96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 0x7f31241f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 0x7f31241f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51138/0 (socket says 172.21.15.70:51138) 2023-08-25T09:51:47.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 -- 172.21.15.70:0/1067584667 learned_addr learned my addr 172.21.15.70:0/1067584667 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:47.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3123fff640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31240cca20 0x7f31240c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3129497640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f312415e210 0x7f31240b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 -- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f312415e210 msgr2=0x7f31240b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f312415e210 0x7f31240b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 -- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31240cca20 msgr2=0x7f31240c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31240cca20 0x7f31240c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 -- 172.21.15.70:0/1067584667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3110009400 con 0x7f3124069850 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3123fff640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31240cca20 0x7f31240c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f3128c96640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 0x7f31241f3260 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f311800c4d0 tx=0x7f311800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.786+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118005ce0 con 0x7f3124069850 2023-08-25T09:51:47.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.787+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31240b1560 con 0x7f3124069850 2023-08-25T09:51:47.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.787+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31240b34c0 con 0x7f3124069850 2023-08-25T09:51:47.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.787+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118005e40 con 0x7f3124069850 2023-08-25T09:51:47.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.787+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118007270 con 0x7f3124069850 2023-08-25T09:51:47.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.788+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3104005250 con 0x7f3124069850 2023-08-25T09:51:47.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.790+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f31180073d0 con 0x7f3124069850 2023-08-25T09:51:47.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.791+0000 7f31217fa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f310806a050 0x7f310806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:47.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.791+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f31180879e0 con 0x7f3124069850 2023-08-25T09:51:47.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.791+0000 7f3123fff640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f310806a050 0x7f310806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:47.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.792+0000 7f3123fff640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f310806a050 0x7f310806c510 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3110036000 tx=0x7f3110008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:47.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.796+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3118056c00 con 0x7f3124069850 2023-08-25T09:51:47.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.986+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3104005a80 con 0x7f3124069850 2023-08-25T09:51:47.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.987+0000 7f31217fa640 1 -- 172.21.15.70:0/1067584667 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v187) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31180011e0 con 0x7f3124069850 2023-08-25T09:51:47.989 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:51:47.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f310806a050 msgr2=0x7f310806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f310806a050 0x7f310806c510 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f3110036000 tx=0x7f3110008030 comp rx=0 tx=0).stop 2023-08-25T09:51:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 msgr2=0x7f31241f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 0x7f31241f3260 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f311800c4d0 tx=0x7f311800c990 comp rx=0 tx=0).stop 2023-08-25T09:51:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 shutdown_connections 2023-08-25T09:51:47.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f310806a050 0x7f310806c510 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f312415e210 0x7f31240b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31240cca20 0x7f31240c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 --2- 172.21.15.70:0/1067584667 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3124069850 0x7f31241f3260 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 >> 172.21.15.70:0/1067584667 conn(0x7f31240cb0d0 msgr2=0x7f312406ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 shutdown_connections 2023-08-25T09:51:47.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:47.990+0000 7f312aefa640 1 -- 172.21.15.70:0/1067584667 wait complete. 2023-08-25T09:51:48.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:51:48.224 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.223+0000 7fa4248ec640 1 Processor -- start 2023-08-25T09:51:48.225 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.223+0000 7fa4248ec640 1 -- start start 2023-08-25T09:51:48.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa4248ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 0x7fa4200cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa4248ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42006f8e0 con 0x7fa42015e240 2023-08-25T09:51:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa4248ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42006fbc0 con 0x7fa420159bf0 2023-08-25T09:51:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa4248ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200ce030 con 0x7fa4200cd660 2023-08-25T09:51:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa41dd9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa42015e240 0x7fa420070980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42240/0 (socket says 172.21.15.70:42240) 2023-08-25T09:51:48.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa41dd9b640 1 -- 172.21.15.70:0/280772843 learned_addr learned my addr 172.21.15.70:0/280772843 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.225+0000 7fa41ed9d640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 0x7fa4200cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.229 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253743233 0 0) 0x7fa42006f8e0 con 0x7fa42015e240 2023-08-25T09:51:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa408003520 con 0x7fa42015e240 2023-08-25T09:51:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3494590777 0 0) 0x7fa408003520 con 0x7fa42015e240 2023-08-25T09:51:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 msgr2=0x7fa4200cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 0x7fa4200cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 >> v1:172.21.15.155:6789/0 conn(0x7fa420159bf0 legacy=0x7fa42006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42006fea0 con 0x7fa42015e240 2023-08-25T09:51:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa414001670 con 0x7fa42015e240 2023-08-25T09:51:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 >> v1:172.21.15.70:6789/0 conn(0x7fa42015e240 legacy=0x7fa420070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4080039a0 0x7fa408003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 0x7fa4080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa408004d00 0x7fa408005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200ce030 con 0x7fa4080039a0 2023-08-25T09:51:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40800ba60 con 0x7fa408004d00 2023-08-25T09:51:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40800bc20 con 0x7fa4200cd660 2023-08-25T09:51:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.226+0000 7fa41e59c640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4080039a0 0x7fa408003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.227+0000 7fa41e59c640 1 -- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 msgr2=0x7fa4080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:48.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.227+0000 7fa41dd9b640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 0x7fa4080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.227+0000 7fa41e59c640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4200cd660 0x7fa4080047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.227+0000 7fa41e59c640 1 -- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa408004d00 msgr2=0x7fa408005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.227+0000 7fa41e59c640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa408004d00 0x7fa408005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.227+0000 7fa41e59c640 1 -- 172.21.15.70:0/280772843 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa408003520 con 0x7fa4080039a0 2023-08-25T09:51:48.233 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.228+0000 7fa41e59c640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4080039a0 0x7fa408003dc0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa410001720 tx=0x7fa4080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a0df4f6ac89c67 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:48.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.228+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100052d0 con 0x7fa4080039a0 2023-08-25T09:51:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.228+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41002e050 con 0x7fa4080039a0 2023-08-25T09:51:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.228+0000 7fa41d59a640 1 -- 172.21.15.70:0/280772843 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4100035f0 con 0x7fa4080039a0 2023-08-25T09:51:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa4248ec640 1 -- 172.21.15.70:0/280772843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4080039a0 msgr2=0x7fa408003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa4248ec640 1 --2- 172.21.15.70:0/280772843 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4080039a0 0x7fa408003dc0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa410001720 tx=0x7fa4080057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa41ed9d640 1 -- 172.21.15.70:0/280772843 reap_dead start 2023-08-25T09:51:48.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa4248ec640 1 -- 172.21.15.70:0/280772843 shutdown_connections 2023-08-25T09:51:48.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa4248ec640 1 -- 172.21.15.70:0/280772843 >> 172.21.15.70:0/280772843 conn(0x7fa4200cb0d0 msgr2=0x7fa4200cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:48.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa4248ec640 1 -- 172.21.15.70:0/280772843 shutdown_connections 2023-08-25T09:51:48.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.229+0000 7fa4248ec640 1 -- 172.21.15.70:0/280772843 wait complete. 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.231+0000 7fa4248ec640 1 Processor -- start 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.231+0000 7fa4248ec640 1 -- start start 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa4248ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4080039a0 0x7fa4200b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa4248ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4200b0910 0x7fa4201f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa4248ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 0x7fa4200acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa4248ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200ad240 con 0x7fa4200ac880 2023-08-25T09:51:48.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa4248ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200c6160 con 0x7fa4200b0910 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa4248ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42015c480 con 0x7fa4080039a0 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa41ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 0x7fa4200acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa41ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 0x7fa4200acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51158/0 (socket says 172.21.15.70:51158) 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.232+0000 7fa41ed9d640 1 -- 172.21.15.70:0/4123503551 learned_addr learned my addr 172.21.15.70:0/4123503551 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41dd9b640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4200b0910 0x7fa4201f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41e59c640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4080039a0 0x7fa4200b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41ed9d640 1 -- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4080039a0 msgr2=0x7fa4200b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41ed9d640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4080039a0 0x7fa4200b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41ed9d640 1 -- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4200b0910 msgr2=0x7fa4201f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41ed9d640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4200b0910 0x7fa4201f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41ed9d640 1 -- 172.21.15.70:0/4123503551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42015c760 con 0x7fa4200ac880 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41dd9b640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4200b0910 0x7fa4201f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa41ed9d640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 0x7fa4200acd00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa42015d720 tx=0x7fa41800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418010040 con 0x7fa4200ac880 2023-08-25T09:51:48.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41800bd10 con 0x7fa4200ac880 2023-08-25T09:51:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa418014640 con 0x7fa4200ac880 2023-08-25T09:51:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42015ca40 con 0x7fa4200ac880 2023-08-25T09:51:48.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.233+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4201f24a0 con 0x7fa4200ac880 2023-08-25T09:51:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.235+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4200c6340 con 0x7fa4200ac880 2023-08-25T09:51:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.236+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa4180147a0 con 0x7fa4200ac880 2023-08-25T09:51:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.236+0000 7fa40effd640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa40006a050 0x7fa40006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.237+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fa418088d10 con 0x7fa4200ac880 2023-08-25T09:51:48.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.237+0000 7fa41e59c640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa40006a050 0x7fa40006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.237+0000 7fa41e59c640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa40006a050 0x7fa40006c510 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa410033000 tx=0x7fa410009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:48.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.241+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa4200c6340 con 0x7fa4200ac880 2023-08-25T09:51:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.427+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4200c6340 con 0x7fa4200ac880 2023-08-25T09:51:48.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.428+0000 7fa40effd640 1 -- 172.21.15.70:0/4123503551 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v187) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa418018020 con 0x7fa4200ac880 2023-08-25T09:51:48.430 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:51:48.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa40006a050 msgr2=0x7fa40006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa40006a050 0x7fa40006c510 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa410033000 tx=0x7fa410009030 comp rx=0 tx=0).stop 2023-08-25T09:51:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 msgr2=0x7fa4200acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 0x7fa4200acd00 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa42015d720 tx=0x7fa41800b480 comp rx=0 tx=0).stop 2023-08-25T09:51:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 shutdown_connections 2023-08-25T09:51:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa40006a050 0x7fa40006c510 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa4200ac880 0x7fa4200acd00 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa4200b0910 0x7fa4201f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 --2- 172.21.15.70:0/4123503551 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa4080039a0 0x7fa4200b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.430+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 >> 172.21.15.70:0/4123503551 conn(0x7fa4200cb0d0 msgr2=0x7fa42015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:48.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.431+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 shutdown_connections 2023-08-25T09:51:48.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.431+0000 7fa4248ec640 1 -- 172.21.15.70:0/4123503551 wait complete. 2023-08-25T09:51:48.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:51:48.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.664+0000 7fdd46574640 1 Processor -- start 2023-08-25T09:51:48.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.664+0000 7fdd46574640 1 -- start start 2023-08-25T09:51:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd46574640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdd400cadb0 0x7fdd400cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd46574640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd400b2aa0 con 0x7fdd40159840 2023-08-25T09:51:48.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd46574640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd400b4450 con 0x7fdd4015d3d0 2023-08-25T09:51:48.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd46574640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd400c8b60 con 0x7fdd400cadb0 2023-08-25T09:51:48.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd45572640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdd40159840 0x7fdd400b2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:42254/0 (socket says 172.21.15.70:42254) 2023-08-25T09:51:48.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd45572640 1 -- 172.21.15.70:0/4213489166 learned_addr learned my addr 172.21.15.70:0/4213489166 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:48.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd45d73640 1 --2- 172.21.15.70:0/4213489166 >> v2:172.21.15.70:3300/0 conn(0x7fdd400cadb0 0x7fdd400cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd3ffff640 1 -- 172.21.15.70:0/4213489166 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267985487 0 0) 0x7fdd400b2aa0 con 0x7fdd40159840 2023-08-25T09:51:48.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd3ffff640 1 -- 172.21.15.70:0/4213489166 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd28003520 con 0x7fdd40159840 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd45d73640 1 -- 172.21.15.70:0/4213489166 >> v1:172.21.15.155:6789/0 conn(0x7fdd4015d3d0 legacy=0x7fdd400b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.666+0000 7fdd45d73640 1 -- 172.21.15.70:0/4213489166 >> v1:172.21.15.70:6789/0 conn(0x7fdd40159840 legacy=0x7fdd400b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.667+0000 7fdd45d73640 1 -- 172.21.15.70:0/4213489166 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4015d890 con 0x7fdd400cadb0 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.668+0000 7fdd45d73640 1 --2- 172.21.15.70:0/4213489166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd400cadb0 0x7fdd400cb1d0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fdd380097a0 tx=0x7fdd3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16bc150a5122cd82 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.668+0000 7fdd3ffff640 1 -- 172.21.15.70:0/4213489166 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38031830 con 0x7fdd400cadb0 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.668+0000 7fdd3ffff640 1 -- 172.21.15.70:0/4213489166 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd38031990 con 0x7fdd400cadb0 2023-08-25T09:51:48.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.668+0000 7fdd3ffff640 1 -- 172.21.15.70:0/4213489166 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38031d20 con 0x7fdd400cadb0 2023-08-25T09:51:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.668+0000 7fdd46574640 1 -- 172.21.15.70:0/4213489166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd400cadb0 msgr2=0x7fdd400cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.668+0000 7fdd46574640 1 --2- 172.21.15.70:0/4213489166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd400cadb0 0x7fdd400cb1d0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fdd380097a0 tx=0x7fdd3802b8d0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.669+0000 7fdd46574640 1 -- 172.21.15.70:0/4213489166 shutdown_connections 2023-08-25T09:51:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.669+0000 7fdd46574640 1 --2- 172.21.15.70:0/4213489166 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd400cadb0 0x7fdd400cb1d0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.669+0000 7fdd46574640 1 -- 172.21.15.70:0/4213489166 >> 172.21.15.70:0/4213489166 conn(0x7fdd400affd0 msgr2=0x7fdd400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:48.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.669+0000 7fdd46574640 1 -- 172.21.15.70:0/4213489166 shutdown_connections 2023-08-25T09:51:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.669+0000 7fdd46574640 1 -- 172.21.15.70:0/4213489166 wait complete. 2023-08-25T09:51:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.670+0000 7fdd46574640 1 Processor -- start 2023-08-25T09:51:48.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 -- start start 2023-08-25T09:51:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400cadb0 0x7fdd40148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd40159840 0x7fdd4014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 0x7fdd400ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4014cbb0 con 0x7fdd4015d3d0 2023-08-25T09:51:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd40168a50 con 0x7fdd40159840 2023-08-25T09:51:48.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.671+0000 7fdd46574640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4014ce30 con 0x7fdd400cadb0 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 0x7fdd400ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 0x7fdd400ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:51176/0 (socket says 172.21.15.70:51176) 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 -- 172.21.15.70:0/3235467612 learned_addr learned my addr 172.21.15.70:0/3235467612 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45572640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400cadb0 0x7fdd40148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 -- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400cadb0 msgr2=0x7fdd40148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400cadb0 0x7fdd40148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 -- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd40159840 msgr2=0x7fdd4014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd44d71640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd40159840 0x7fdd4014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd40159840 0x7fdd4014c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45d73640 1 -- 172.21.15.70:0/3235467612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38009400 con 0x7fdd4015d3d0 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.672+0000 7fdd45572640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400cadb0 0x7fdd40148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.673+0000 7fdd45d73640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 0x7fdd400ac880 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098c0 tx=0x7fdd380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.673+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3803a630 con 0x7fdd4015d3d0 2023-08-25T09:51:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.673+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3803a790 con 0x7fdd4015d3d0 2023-08-25T09:51:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.673+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3803ab20 con 0x7fdd4015d3d0 2023-08-25T09:51:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.673+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4014d110 con 0x7fdd4015d3d0 2023-08-25T09:51:48.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.673+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4014d6d0 con 0x7fdd4015d3d0 2023-08-25T09:51:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.674+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd400ae180 con 0x7fdd4015d3d0 2023-08-25T09:51:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.675+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdd3803ac80 con 0x7fdd4015d3d0 2023-08-25T09:51:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.677+0000 7fdd3dffb640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd2c06a050 0x7fdd2c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.677+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fdd380aa400 con 0x7fdd4015d3d0 2023-08-25T09:51:48.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.677+0000 7fdd45572640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd2c06a050 0x7fdd2c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:48.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.677+0000 7fdd45572640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd2c06a050 0x7fdd2c06c510 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007fd0 tx=0x7fdd30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:48.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.681+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdd400ae180 con 0x7fdd4015d3d0 2023-08-25T09:51:48.907 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.906+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd400ae180 con 0x7fdd4015d3d0 2023-08-25T09:51:48.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.907+0000 7fdd3dffb640 1 -- 172.21.15.70:0/3235467612 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fdd400ae180 con 0x7fdd4015d3d0 2023-08-25T09:51:48.909 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:48.909 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:51:48.910 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:51:48.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd2c06a050 msgr2=0x7fdd2c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd2c06a050 0x7fdd2c06c510 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdd30007fd0 tx=0x7fdd30009040 comp rx=0 tx=0).stop 2023-08-25T09:51:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 msgr2=0x7fdd400ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:48.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 0x7fdd400ac880 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fdd380098c0 tx=0x7fdd380046c0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 shutdown_connections 2023-08-25T09:51:48.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdd2c06a050 0x7fdd2c06c510 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdd4015d3d0 0x7fdd400ac880 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdd40159840 0x7fdd4014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 --2- 172.21.15.70:0/3235467612 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdd400cadb0 0x7fdd40148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 >> 172.21.15.70:0/3235467612 conn(0x7fdd400affd0 msgr2=0x7fdd40151250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:48.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 shutdown_connections 2023-08-25T09:51:48.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:48.910+0000 7fdd46574640 1 -- 172.21.15.70:0/3235467612 wait complete. 2023-08-25T09:51:48.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:51:49.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.149+0000 7fcc13683640 1 Processor -- start 2023-08-25T09:51:49.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.149+0000 7fcc13683640 1 -- start start 2023-08-25T09:51:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc13683640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fcc0c072e70 0x7fcc0c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc13683640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcc0c06aea0 0x7fcc0c06c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc13683640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c0ceb30 con 0x7fcc0c072a90 2023-08-25T09:51:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc13683640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cee10 con 0x7fcc0c072e70 2023-08-25T09:51:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc13683640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cf0f0 con 0x7fcc0c06aea0 2023-08-25T09:51:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc1141f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcc0c072a90 0x7fcc0c06b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44402/0 (socket says 172.21.15.70:44402) 2023-08-25T09:51:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc1141f640 1 -- 172.21.15.70:0/992703473 learned_addr learned my addr 172.21.15.70:0/992703473 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:49.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.151+0000 7fcc10c1e640 1 --2- 172.21.15.70:0/992703473 >> v2:172.21.15.70:3300/0 conn(0x7fcc0c072e70 0x7fcc0c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.152+0000 7fcc10c1e640 1 -- 172.21.15.70:0/992703473 >> v2:172.21.15.155:3300/0 conn(0x7fcc0c06aea0 msgr2=0x7fcc0c06c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.152+0000 7fcc10c1e640 1 --2- 172.21.15.70:0/992703473 >> v2:172.21.15.155:3300/0 conn(0x7fcc0c06aea0 0x7fcc0c06c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.152+0000 7fcc10c1e640 1 -- 172.21.15.70:0/992703473 >> v1:172.21.15.70:6789/0 conn(0x7fcc0c072a90 legacy=0x7fcc0c06b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.152+0000 7fcc03fff640 1 -- 172.21.15.70:0/992703473 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379573783 0 0) 0x7fcc0c0ceb30 con 0x7fcc0c072a90 2023-08-25T09:51:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.152+0000 7fcc10c1e640 1 -- 172.21.15.70:0/992703473 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0cf3d0 con 0x7fcc0c072e70 2023-08-25T09:51:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.153+0000 7fcc10c1e640 1 --2- 172.21.15.70:0/992703473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072e70 0x7fcc0c06a960 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c073300 tx=0x7fcbfc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa37368574e32af server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:49.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.153+0000 7fcc03fff640 1 -- 172.21.15.70:0/992703473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc02e070 con 0x7fcc0c072e70 2023-08-25T09:51:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.153+0000 7fcc03fff640 1 -- 172.21.15.70:0/992703473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc0315a0 con 0x7fcc0c072e70 2023-08-25T09:51:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.153+0000 7fcc03fff640 1 -- 172.21.15.70:0/992703473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc031930 con 0x7fcc0c072e70 2023-08-25T09:51:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 -- 172.21.15.70:0/992703473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072e70 msgr2=0x7fcc0c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 --2- 172.21.15.70:0/992703473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072e70 0x7fcc0c06a960 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c073300 tx=0x7fcbfc02b840 comp rx=0 tx=0).stop 2023-08-25T09:51:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 -- 172.21.15.70:0/992703473 shutdown_connections 2023-08-25T09:51:49.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 --2- 172.21.15.70:0/992703473 >> v2:172.21.15.155:3300/0 conn(0x7fcc0c06aea0 0x7fcc0c06c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 --2- 172.21.15.70:0/992703473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072e70 0x7fcc0c06a960 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 -- 172.21.15.70:0/992703473 >> 172.21.15.70:0/992703473 conn(0x7fcc0c0cf9c0 msgr2=0x7fcc0c0cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 -- 172.21.15.70:0/992703473 shutdown_connections 2023-08-25T09:51:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.154+0000 7fcc13683640 1 -- 172.21.15.70:0/992703473 wait complete. 2023-08-25T09:51:49.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.156+0000 7fcc13683640 1 Processor -- start 2023-08-25T09:51:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.156+0000 7fcc13683640 1 -- start start 2023-08-25T09:51:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc13683640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 0x7fcc0c15b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc13683640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc0c072e70 0x7fcc0c146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc13683640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0c0c9040 0x7fcc0c159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc13683640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c15aee0 con 0x7fcc0c072a90 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc13683640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c147130 con 0x7fcc0c0c9040 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc13683640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c159fc0 con 0x7fcc0c072e70 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc1141f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 0x7fcc0c15b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc1141f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 0x7fcc0c15b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39002/0 (socket says 172.21.15.70:39002) 2023-08-25T09:51:49.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.157+0000 7fcc1141f640 1 -- 172.21.15.70:0/2470261729 learned_addr learned my addr 172.21.15.70:0/2470261729 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:49.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc11c20640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0c0c9040 0x7fcc0c159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc1141f640 1 -- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc0c072e70 msgr2=0x7fcc0c146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc1141f640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc0c072e70 0x7fcc0c146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc1141f640 1 -- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0c0c9040 msgr2=0x7fcc0c159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc1141f640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0c0c9040 0x7fcc0c159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc1141f640 1 -- 172.21.15.70:0/2470261729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc009400 con 0x7fcc0c072a90 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc1141f640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 0x7fcc0c15b300 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800c4d0 tx=0x7fcc0800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:49.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc080074e0 con 0x7fcc0c072a90 2023-08-25T09:51:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc08007660 con 0x7fcc0c072a90 2023-08-25T09:51:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08004a20 con 0x7fcc0c072a90 2023-08-25T09:51:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.158+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c15a260 con 0x7fcc0c072a90 2023-08-25T09:51:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.159+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c15a820 con 0x7fcc0c072a90 2023-08-25T09:51:49.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.160+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0671e0 con 0x7fcc0c072a90 2023-08-25T09:51:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.161+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcc08004b80 con 0x7fcc0c072a90 2023-08-25T09:51:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.161+0000 7fcc01ffb640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbf006a050 0x7fcbf006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.162+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7fcc08087860 con 0x7fcc0c072a90 2023-08-25T09:51:49.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.162+0000 7fcc10c1e640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbf006a050 0x7fcbf006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.163+0000 7fcc10c1e640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbf006a050 0x7fcbf006c510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc036000 tx=0x7fcbfc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:49.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.166+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcc08052e90 con 0x7fcc0c072a90 2023-08-25T09:51:49.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.360+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc0c06cd60 con 0x7fcc0c072a90 2023-08-25T09:51:49.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.363+0000 7fcc01ffb640 1 -- 172.21.15.70:0/2470261729 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+18438 (secure 0 0 0) 0x7fcc080564b0 con 0x7fcc0c072a90 2023-08-25T09:51:49.366 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:49.366 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":187,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:47.491625+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:51:49.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbf006a050 msgr2=0x7fcbf006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbf006a050 0x7fcbf006c510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc036000 tx=0x7fcbfc030000 comp rx=0 tx=0).stop 2023-08-25T09:51:49.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 msgr2=0x7fcc0c15b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 0x7fcc0c15b300 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800c4d0 tx=0x7fcc0800c990 comp rx=0 tx=0).stop 2023-08-25T09:51:49.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 shutdown_connections 2023-08-25T09:51:49.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcbf006a050 0x7fcbf006c510 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcc0c0c9040 0x7fcc0c159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcc0c072e70 0x7fcc0c146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 --2- 172.21.15.70:0/2470261729 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcc0c072a90 0x7fcc0c15b300 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 >> 172.21.15.70:0/2470261729 conn(0x7fcc0c0cf9c0 msgr2=0x7fcc0c0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:49.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 shutdown_connections 2023-08-25T09:51:49.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.366+0000 7fcc13683640 1 -- 172.21.15.70:0/2470261729 wait complete. 2023-08-25T09:51:49.386 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:51:49.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:51:49.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.598+0000 7f7dee139640 1 Processor -- start 2023-08-25T09:51:49.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.598+0000 7f7dee139640 1 -- start start 2023-08-25T09:51:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.599+0000 7f7dee139640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7de815ec10 0x7f7de815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.599+0000 7f7dee139640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.599+0000 7f7dee139640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de80b1b60 con 0x7f7de815e830 2023-08-25T09:51:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.599+0000 7f7dee139640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de806bd80 con 0x7f7de815ec10 2023-08-25T09:51:49.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.599+0000 7f7dee139640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de806b7c0 con 0x7f7de815c6d0 2023-08-25T09:51:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.600+0000 7f7de6ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7de815ec10 0x7f7de815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.600+0000 7f7de7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.603 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.600+0000 7f7de7fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de80b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:48960/0 (socket says 172.21.15.70:48960) 2023-08-25T09:51:49.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.600+0000 7f7de6ffd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f7de815ec10 0x7f7de815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:43184/0 (socket says 172.21.15.70:43184) 2023-08-25T09:51:49.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.600+0000 7f7de7fff640 1 -- 172.21.15.70:0/1506905509 learned_addr learned my addr 172.21.15.70:0/1506905509 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:49.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.601+0000 7f7de6ffd640 1 -- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 msgr2=0x7f7de80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:49.604 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.601+0000 7f7de6ffd640 1 --2- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.601+0000 7f7de6ffd640 1 -- 172.21.15.70:0/1506905509 >> v1:172.21.15.155:6789/0 conn(0x7f7de815e830 legacy=0x7f7de806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.601+0000 7f7de6ffd640 1 -- 172.21.15.70:0/1506905509 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de806baa0 con 0x7f7de815ec10 2023-08-25T09:51:49.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.602+0000 7f7de6ffd640 1 --2- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de815c190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009d90 tx=0x7f7dd002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5b21cbb2a38ac9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.603+0000 7f7de67fc640 1 -- 172.21.15.70:0/1506905509 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002cc50 con 0x7f7de815ec10 2023-08-25T09:51:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.603+0000 7f7de67fc640 1 -- 172.21.15.70:0/1506905509 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd002cdb0 con 0x7f7de815ec10 2023-08-25T09:51:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.603+0000 7f7de67fc640 1 -- 172.21.15.70:0/1506905509 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0034660 con 0x7f7de815ec10 2023-08-25T09:51:49.607 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 -- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 msgr2=0x7f7de815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 --2- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de815c190 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009d90 tx=0x7f7dd002bea0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 -- 172.21.15.70:0/1506905509 shutdown_connections 2023-08-25T09:51:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 --2- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 --2- 172.21.15.70:0/1506905509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de815c190 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 -- 172.21.15.70:0/1506905509 >> 172.21.15.70:0/1506905509 conn(0x7f7de8069850 msgr2=0x7f7de8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 -- 172.21.15.70:0/1506905509 shutdown_connections 2023-08-25T09:51:49.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.604+0000 7f7dee139640 1 -- 172.21.15.70:0/1506905509 wait complete. 2023-08-25T09:51:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.606+0000 7f7dee139640 1 Processor -- start 2023-08-25T09:51:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.606+0000 7f7dee139640 1 -- start start 2023-08-25T09:51:49.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7dee139640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de8158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7dee139640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 0x7f7de80c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7dee139640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de80b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7dee139640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de80c8d20 con 0x7f7de815e830 2023-08-25T09:51:49.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7dee139640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de80c80a0 con 0x7f7de815ec10 2023-08-25T09:51:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7dee139640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de80c8380 con 0x7f7de815c6d0 2023-08-25T09:51:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.607+0000 7f7de6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 0x7f7de80c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 0x7f7de80c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39004/0 (socket says 172.21.15.70:39004) 2023-08-25T09:51:49.614 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 -- 172.21.15.70:0/2599892084 learned_addr learned my addr 172.21.15.70:0/2599892084 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de7fff640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de80b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de77fe640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de8158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 -- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 msgr2=0x7f7de8158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de8158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 -- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 msgr2=0x7f7de80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de80b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 -- 172.21.15.70:0/2599892084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd0009a70 con 0x7f7de815e830 2023-08-25T09:51:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de77fe640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de8158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de7fff640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.608+0000 7f7de6ffd640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 0x7f7de80c77a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009d90 tx=0x7f7dd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:49.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.609+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0040650 con 0x7f7de815e830 2023-08-25T09:51:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.609+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd00407b0 con 0x7f7de815e830 2023-08-25T09:51:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.609+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de80c8660 con 0x7f7de815e830 2023-08-25T09:51:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.609+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0040b60 con 0x7f7de815e830 2023-08-25T09:51:49.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.609+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de81588b0 con 0x7f7de815e830 2023-08-25T09:51:49.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.610+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de80c8e60 con 0x7f7de815e830 2023-08-25T09:51:49.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.611+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7dd0040cc0 con 0x7f7de815e830 2023-08-25T09:51:49.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.612+0000 7f7ddffff640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7dc806a050 0x7f7dc806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:49.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.612+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f7dd00ab450 con 0x7f7de815e830 2023-08-25T09:51:49.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.612+0000 7f7de77fe640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7dc806a050 0x7f7dc806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:49.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.613+0000 7f7de77fe640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7dc806a050 0x7f7dc806c510 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000f60 tx=0x7f7dd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:49.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.616+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7de80c8e60 con 0x7f7de815e830 2023-08-25T09:51:49.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.845+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7de80c8e60 con 0x7f7de815e830 2023-08-25T09:51:49.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.847+0000 7f7ddffff640 1 -- 172.21.15.70:0/2599892084 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f7de80c8e60 con 0x7f7de815e830 2023-08-25T09:51:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:49.849 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:51:49.849 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:51:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.849+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7dc806a050 msgr2=0x7f7dc806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.849+0000 7f7dee139640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7dc806a050 0x7f7dc806c510 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8000f60 tx=0x7f7dd8009040 comp rx=0 tx=0).stop 2023-08-25T09:51:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.849+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 msgr2=0x7f7de80c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.849+0000 7f7dee139640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 0x7f7de80c77a0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009d90 tx=0x7f7dd0004a40 comp rx=0 tx=0).stop 2023-08-25T09:51:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 shutdown_connections 2023-08-25T09:51:49.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7dc806a050 0x7f7dc806c510 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7de815ec10 0x7f7de80b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7de815e830 0x7f7de80c77a0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 --2- 172.21.15.70:0/2599892084 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7de815c6d0 0x7f7de8158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 >> 172.21.15.70:0/2599892084 conn(0x7f7de8069850 msgr2=0x7f7de80ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 shutdown_connections 2023-08-25T09:51:49.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:49.850+0000 7f7dee139640 1 -- 172.21.15.70:0/2599892084 wait complete. 2023-08-25T09:51:49.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:51:50.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.064+0000 7f27a759e640 1 Processor -- start 2023-08-25T09:51:50.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.064+0000 7f27a759e640 1 -- start start 2023-08-25T09:51:50.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.065+0000 7f27a759e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f27a80cad20 0x7f27a80cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f27a759e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a815e7c0 con 0x7f27a80b12f0 2023-08-25T09:51:50.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f27a759e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a8071b50 con 0x7f27a815a7c0 2023-08-25T09:51:50.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f27a759e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a80cb6f0 con 0x7f27a80cad20 2023-08-25T09:51:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f279ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f27a80b12f0 0x7f27a815ec50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44414/0 (socket says 172.21.15.70:44414) 2023-08-25T09:51:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f279ffff640 1 -- 172.21.15.70:0/1907365664 learned_addr learned my addr 172.21.15.70:0/1907365664 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f27a6d9d640 1 --2- 172.21.15.70:0/1907365664 >> v2:172.21.15.155:3300/0 conn(0x7f27a80cad20 0x7f27a80cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381245551 0 0) 0x7f27a815e7c0 con 0x7f27a80b12f0 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.066+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2794003520 con 0x7f27a80b12f0 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033873862 0 0) 0x7f2794003520 con 0x7f27a80b12f0 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 >> v2:172.21.15.155:3300/0 conn(0x7f27a80cad20 msgr2=0x7f27a80cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 --2- 172.21.15.70:0/1907365664 >> v2:172.21.15.155:3300/0 conn(0x7f27a80cad20 0x7f27a80cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 >> v1:172.21.15.155:6789/0 conn(0x7f27a815a7c0 legacy=0x7f27a80b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a815e490 con 0x7f27a80b12f0 2023-08-25T09:51:50.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f2790001670 con 0x7f27a80b12f0 2023-08-25T09:51:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 >> v1:172.21.15.70:6789/0 conn(0x7f27a80b12f0 legacy=0x7f27a815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2794003970 0x7f2794003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27940047c0 0x7f2794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f279400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80cb6f0 con 0x7f2794005180 2023-08-25T09:51:50.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279400c0d0 con 0x7f27940047c0 2023-08-25T09:51:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279400c290 con 0x7f2794003970 2023-08-25T09:51:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a6d9d640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f279400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a659c640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2794003970 0x7f2794003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f279ffff640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27940047c0 0x7f2794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a6d9d640 1 -- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2794003970 msgr2=0x7f2794003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a6d9d640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2794003970 0x7f2794003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.067+0000 7f27a6d9d640 1 -- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27940047c0 msgr2=0x7f2794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.068+0000 7f27a6d9d640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27940047c0 0x7f2794004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.068+0000 7f27a6d9d640 1 -- 172.21.15.70:0/1907365664 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2794003520 con 0x7f2794005180 2023-08-25T09:51:50.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.068+0000 7f279ffff640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27940047c0 0x7f2794004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.068+0000 7f27a6d9d640 1 -- 172.21.15.70:0/1907365664 reap_dead start 2023-08-25T09:51:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a6d9d640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f279400b980 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f27a0007360 tx=0x7f27940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d8923d0d7027c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a002ccf0 con 0x7f2794005180 2023-08-25T09:51:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a002ce90 con 0x7f2794005180 2023-08-25T09:51:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a5d9b640 1 -- 172.21.15.70:0/1907365664 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0003bd0 con 0x7f2794005180 2023-08-25T09:51:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a759e640 1 -- 172.21.15.70:0/1907365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 msgr2=0x7f279400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a759e640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f279400b980 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f27a0007360 tx=0x7f27940057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a759e640 1 -- 172.21.15.70:0/1907365664 shutdown_connections 2023-08-25T09:51:50.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a759e640 1 --2- 172.21.15.70:0/1907365664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f279400b980 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.069+0000 7f27a759e640 1 -- 172.21.15.70:0/1907365664 >> 172.21.15.70:0/1907365664 conn(0x7f27a80affd0 msgr2=0x7f27a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:50.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.070+0000 7f27a759e640 1 -- 172.21.15.70:0/1907365664 shutdown_connections 2023-08-25T09:51:50.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.070+0000 7f27a759e640 1 -- 172.21.15.70:0/1907365664 wait complete. 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.071+0000 7f27a759e640 1 Processor -- start 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.071+0000 7f27a759e640 1 -- start start 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a759e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f27a8154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a759e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27a8159d30 0x7f27a8070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a759e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a80708a0 0x7f27a80c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a759e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8070d70 con 0x7f2794005180 2023-08-25T09:51:50.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a759e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80c1fd0 con 0x7f27a80708a0 2023-08-25T09:51:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a759e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a80c22b0 con 0x7f27a8159d30 2023-08-25T09:51:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f27a8154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.072+0000 7f27a659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f27a8154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39026/0 (socket says 172.21.15.70:39026) 2023-08-25T09:51:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 -- 172.21.15.70:0/3295120962 learned_addr learned my addr 172.21.15.70:0/3295120962 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 -- 172.21.15.70:0/3295120962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27a8159d30 msgr2=0x7f27a8070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27a8159d30 0x7f27a8070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 -- 172.21.15.70:0/3295120962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a80708a0 msgr2=0x7f27a80c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a80708a0 0x7f27a80c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 -- 172.21.15.70:0/3295120962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a80c2590 con 0x7f2794005180 2023-08-25T09:51:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a659c640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f27a8154080 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f279800d570 tx=0x7f279800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798003610 con 0x7f2794005180 2023-08-25T09:51:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798003770 con 0x7f2794005180 2023-08-25T09:51:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798003ae0 con 0x7f2794005180 2023-08-25T09:51:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.073+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a80c2870 con 0x7f2794005180 2023-08-25T09:51:50.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.074+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a80c7150 con 0x7f2794005180 2023-08-25T09:51:50.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.075+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a80c73d0 con 0x7f2794005180 2023-08-25T09:51:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.076+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2798003c40 con 0x7f2794005180 2023-08-25T09:51:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.076+0000 7f279effd640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f278806a050 0x7f278806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.077+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f2798088560 con 0x7f2794005180 2023-08-25T09:51:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.077+0000 7f279ffff640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f278806a050 0x7f278806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.077+0000 7f279ffff640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f278806a050 0x7f278806c510 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2790000f60 tx=0x7f2790009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.081+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2798053c50 con 0x7f2794005180 2023-08-25T09:51:50.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.286+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27a8066fb0 con 0x7f2794005180 2023-08-25T09:51:50.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.287+0000 7f279effd640 1 -- 172.21.15.70:0/3295120962 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f2798057270 con 0x7f2794005180 2023-08-25T09:51:50.288 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:50.289 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":82,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:50:34.994169+0000","modified":"2023-08-25T09:50:38.548823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9162},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9162":{"gid":9162,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.155:6834/3137406973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":3137406973},{"type":"v1","addr":"172.21.15.155:6834","nonce":3137406973}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-08-25T09:51:50.289 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 82 2023-08-25T09:51:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f278806a050 msgr2=0x7f278806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f278806a050 0x7f278806c510 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f2790000f60 tx=0x7f2790009040 comp rx=0 tx=0).stop 2023-08-25T09:51:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 msgr2=0x7f27a8154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f27a8154080 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f279800d570 tx=0x7f279800da30 comp rx=0 tx=0).stop 2023-08-25T09:51:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 shutdown_connections 2023-08-25T09:51:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f278806a050 0x7f278806c510 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f27a80708a0 0x7f27a80c6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f27a8159d30 0x7f27a8070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 --2- 172.21.15.70:0/3295120962 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2794005180 0x7f27a8154080 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 >> 172.21.15.70:0/3295120962 conn(0x7f27a80affd0 msgr2=0x7f27a80c5850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:50.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 shutdown_connections 2023-08-25T09:51:50.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.289+0000 7f27a759e640 1 -- 172.21.15.70:0/3295120962 wait complete. 2023-08-25T09:51:50.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:51:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.512+0000 7f91bcc26640 1 Processor -- start 2023-08-25T09:51:50.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.513+0000 7f91bcc26640 1 -- start start 2023-08-25T09:51:50.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.514+0000 7f91bcc26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.514+0000 7f91bcc26640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91b806b2b0 0x7f91b815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.514+0000 7f91bcc26640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80b1b60 con 0x7f91b815c6d0 2023-08-25T09:51:50.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.514+0000 7f91bcc26640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b806bd80 con 0x7f91b806b2b0 2023-08-25T09:51:50.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.514+0000 7f91bcc26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b806b7c0 con 0x7f91b815e830 2023-08-25T09:51:50.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.514+0000 7f91affff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91b806b2b0 0x7f91b815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91affff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f91b806b2b0 0x7f91b815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39038/0 (socket says 172.21.15.70:39038) 2023-08-25T09:51:50.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91affff640 1 -- 172.21.15.70:0/3369457205 learned_addr learned my addr 172.21.15.70:0/3369457205 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:50.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91b659c640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91affff640 1 -- 172.21.15.70:0/3369457205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 msgr2=0x7f91b806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91affff640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b806ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91affff640 1 -- 172.21.15.70:0/3369457205 >> v1:172.21.15.70:6789/0 conn(0x7f91b815c6d0 legacy=0x7f91b80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.520 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91affff640 1 -- 172.21.15.70:0/3369457205 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b806baa0 con 0x7f91b806b2b0 2023-08-25T09:51:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.515+0000 7f91b659c640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.516+0000 7f91affff640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b806b2b0 0x7f91b815c190 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009870 tx=0x7f91a002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c41f11ce9893e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.521 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.516+0000 7f91b5d9b640 1 -- 172.21.15.70:0/3369457205 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0030840 con 0x7f91b806b2b0 2023-08-25T09:51:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.516+0000 7f91b5d9b640 1 -- 172.21.15.70:0/3369457205 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a00309a0 con 0x7f91b806b2b0 2023-08-25T09:51:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.516+0000 7f91b5d9b640 1 -- 172.21.15.70:0/3369457205 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0030d30 con 0x7f91b806b2b0 2023-08-25T09:51:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 -- 172.21.15.70:0/3369457205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b806b2b0 msgr2=0x7f91b815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.522 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b806b2b0 0x7f91b815c190 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009870 tx=0x7f91a002b980 comp rx=0 tx=0).stop 2023-08-25T09:51:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 -- 172.21.15.70:0/3369457205 shutdown_connections 2023-08-25T09:51:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b806b2b0 0x7f91b815c190 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3369457205 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b806ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 -- 172.21.15.70:0/3369457205 >> 172.21.15.70:0/3369457205 conn(0x7f91b8069850 msgr2=0x7f91b8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 -- 172.21.15.70:0/3369457205 shutdown_connections 2023-08-25T09:51:50.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.517+0000 7f91bcc26640 1 -- 172.21.15.70:0/3369457205 wait complete. 2023-08-25T09:51:50.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.519+0000 7f91bcc26640 1 Processor -- start 2023-08-25T09:51:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.519+0000 7f91bcc26640 1 -- start start 2023-08-25T09:51:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.520+0000 7f91bcc26640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b806b2b0 0x7f91b81538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.520+0000 7f91bcc26640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 0x7f91b81f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.520+0000 7f91bcc26640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b80c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.520+0000 7f91bcc26640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b814cc40 con 0x7f91b815c6d0 2023-08-25T09:51:50.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.520+0000 7f91bcc26640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80c66d0 con 0x7f91b806b2b0 2023-08-25T09:51:50.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.520+0000 7f91bcc26640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b81f4550 con 0x7f91b815e830 2023-08-25T09:51:50.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 0x7f91b81f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 0x7f91b81f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39054/0 (socket says 172.21.15.70:39054) 2023-08-25T09:51:50.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 -- 172.21.15.70:0/3639431981 learned_addr learned my addr 172.21.15.70:0/3639431981 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:50.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91b659c640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b806b2b0 0x7f91b81538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91b6d9d640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b80c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 -- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 msgr2=0x7f91b80c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b80c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 -- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b806b2b0 msgr2=0x7f91b81538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b806b2b0 0x7f91b81538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 -- 172.21.15.70:0/3639431981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00094d0 con 0x7f91b815c6d0 2023-08-25T09:51:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91b659c640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b806b2b0 0x7f91b81538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.521+0000 7f91affff640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 0x7f91b81f3640 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f91a002b9c0 tx=0x7f91a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.522+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0030810 con 0x7f91b815c6d0 2023-08-25T09:51:50.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.522+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a0044480 con 0x7f91b815c6d0 2023-08-25T09:51:50.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.522+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0044850 con 0x7f91b815c6d0 2023-08-25T09:51:50.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.522+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b815d350 con 0x7f91b815c6d0 2023-08-25T09:51:50.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.522+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81f3f90 con 0x7f91b815c6d0 2023-08-25T09:51:50.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.523+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b804e420 con 0x7f91b815c6d0 2023-08-25T09:51:50.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.524+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f91a002f390 con 0x7f91b815c6d0 2023-08-25T09:51:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.525+0000 7f91aeffd640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919806a050 0x7f919806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.525+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f91a003e070 con 0x7f91b815c6d0 2023-08-25T09:51:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.525+0000 7f91b659c640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919806a050 0x7f919806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.526+0000 7f91b659c640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919806a050 0x7f919806c510 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.529+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91a00af070 con 0x7f91b815c6d0 2023-08-25T09:51:50.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.718+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91b806c620 con 0x7f91b815c6d0 2023-08-25T09:51:50.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.721+0000 7f91aeffd640 1 -- 172.21.15.70:0/3639431981 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+18438 (secure 0 0 0) 0x7f91a0049090 con 0x7f91b815c6d0 2023-08-25T09:51:50.722 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:50.723 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":187,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:47.491625+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:50:32.642193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"181","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.75,"score_stable":2.75,"optimal_score":1,"raw_score_acting":2.75,"raw_score_stable":2.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-08-25T09:50:33.991027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:51:50.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.723+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919806a050 msgr2=0x7f919806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.723+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919806a050 0x7f919806c510 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f91a8006fd0 tx=0x7f91a8008040 comp rx=0 tx=0).stop 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.723+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 msgr2=0x7f91b81f3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.723+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 0x7f91b81f3640 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f91a002b9c0 tx=0x7f91a0004730 comp rx=0 tx=0).stop 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 shutdown_connections 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919806a050 0x7f919806c510 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b815e830 0x7f91b80c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b815c6d0 0x7f91b81f3640 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 --2- 172.21.15.70:0/3639431981 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b806b2b0 0x7f91b81538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 >> 172.21.15.70:0/3639431981 conn(0x7f91b8069850 msgr2=0x7f91b80b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:50.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 shutdown_connections 2023-08-25T09:51:50.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.724+0000 7f91bcc26640 1 -- 172.21.15.70:0/3639431981 wait complete. 2023-08-25T09:51:50.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:51:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.949+0000 7f8e0b8e0640 1 Processor -- start 2023-08-25T09:51:50.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.949+0000 7f8e0b8e0640 1 -- start start 2023-08-25T09:51:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.951+0000 7f8e0b8e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c9130 0x7f8e0406caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.951+0000 7f8e0b8e0640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e0406cfe0 0x7f8e0406f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.951+0000 7f8e0b8e0640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e040cd1f0 con 0x7f8e040c21f0 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.951+0000 7f8e0b8e0640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e040cb330 con 0x7f8e0406cfe0 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.951+0000 7f8e0b8e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0406d470 con 0x7f8e040c9130 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e0406cfe0 0x7f8e0406f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f8e0406cfe0 0x7f8e0406f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39056/0 (socket says 172.21.15.70:39056) 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 -- 172.21.15.70:0/1392741680 learned_addr learned my addr 172.21.15.70:0/1392741680 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:50.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e08e7b640 1 --2- 172.21.15.70:0/1392741680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c9130 0x7f8e0406caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 -- 172.21.15.70:0/1392741680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c9130 msgr2=0x7f8e0406caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 --2- 172.21.15.70:0/1392741680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c9130 0x7f8e0406caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 -- 172.21.15.70:0/1392741680 >> v1:172.21.15.155:6789/0 conn(0x7f8e040c21f0 legacy=0x7f8e040cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:50.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.952+0000 7f8e09e7d640 1 -- 172.21.15.70:0/1392741680 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0406f2a0 con 0x7f8e0406cfe0 2023-08-25T09:51:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.953+0000 7f8e09e7d640 1 --2- 172.21.15.70:0/1392741680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0406cfe0 0x7f8e0406f780 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009e50 tx=0x7f8dfc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb45efa8cc001d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.953+0000 7f8dfbfff640 1 -- 172.21.15.70:0/1392741680 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc02c9f0 con 0x7f8e0406cfe0 2023-08-25T09:51:50.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.953+0000 7f8dfbfff640 1 -- 172.21.15.70:0/1392741680 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc02cb50 con 0x7f8e0406cfe0 2023-08-25T09:51:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.953+0000 7f8dfbfff640 1 -- 172.21.15.70:0/1392741680 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0343f0 con 0x7f8e0406cfe0 2023-08-25T09:51:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/1392741680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0406cfe0 msgr2=0x7f8e0406f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/1392741680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0406cfe0 0x7f8e0406f780 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009e50 tx=0x7f8dfc02be50 comp rx=0 tx=0).stop 2023-08-25T09:51:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/1392741680 shutdown_connections 2023-08-25T09:51:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/1392741680 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e0406cfe0 0x7f8e0406f780 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.957 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/1392741680 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c9130 0x7f8e0406caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/1392741680 >> 172.21.15.70:0/1392741680 conn(0x7f8e040ceae0 msgr2=0x7f8e040ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/1392741680 shutdown_connections 2023-08-25T09:51:50.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.954+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/1392741680 wait complete. 2023-08-25T09:51:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.956+0000 7f8e0b8e0640 1 Processor -- start 2023-08-25T09:51:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.956+0000 7f8e0b8e0640 1 -- start start 2023-08-25T09:51:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e0b8e0640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0406cfe0 0x7f8e0415e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e0b8e0640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c21f0 0x7f8e040c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e0b8e0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 0x7f8e040c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e0b8e0640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040c37c0 con 0x7f8e040c9130 2023-08-25T09:51:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e0b8e0640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040c2bb0 con 0x7f8e0406cfe0 2023-08-25T09:51:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e0b8e0640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040c2e90 con 0x7f8e040c21f0 2023-08-25T09:51:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e09e7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 0x7f8e040c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e09e7d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 0x7f8e040c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39060/0 (socket says 172.21.15.70:39060) 2023-08-25T09:51:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.957+0000 7f8e09e7d640 1 -- 172.21.15.70:0/584925922 learned_addr learned my addr 172.21.15.70:0/584925922 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e09e7d640 1 -- 172.21.15.70:0/584925922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c21f0 msgr2=0x7f8e040c3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e09e7d640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c21f0 0x7f8e040c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e09e7d640 1 -- 172.21.15.70:0/584925922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0406cfe0 msgr2=0x7f8e0415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e09e7d640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0406cfe0 0x7f8e0415e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:50.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e09e7d640 1 -- 172.21.15.70:0/584925922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc009b30 con 0x7f8e040c9130 2023-08-25T09:51:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e09e7d640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 0x7f8e040c3fa0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009e50 tx=0x7f8dfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc009510 con 0x7f8e040c9130 2023-08-25T09:51:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc03e450 con 0x7f8e040c9130 2023-08-25T09:51:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e040c3170 con 0x7f8e040c9130 2023-08-25T09:51:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.958+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc03e800 con 0x7f8e040c9130 2023-08-25T09:51:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.959+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e04154ae0 con 0x7f8e040c9130 2023-08-25T09:51:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.960+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0404e420 con 0x7f8e040c9130 2023-08-25T09:51:50.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.960+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8dfc03e960 con 0x7f8e040c9130 2023-08-25T09:51:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.961+0000 7f8df9ffb640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8de806a020 0x7f8de806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.962+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6091+0+0 (secure 0 0 0) 0x7f8dfc0ab470 con 0x7f8e040c9130 2023-08-25T09:51:50.964 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.962+0000 7f8e0967c640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8de806a020 0x7f8de806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.962+0000 7f8e0967c640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8de806a020 0x7f8de806c4e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8e00003cc0 tx=0x7f8e000079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:50.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:50.965+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8dfc076bc0 con 0x7f8e040c9130 2023-08-25T09:51:51.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.176+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8e0406fa40 con 0x7f8e040c9130 2023-08-25T09:51:51.179 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:51:51.178+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:51:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.185+0000 7f8df9ffb640 1 -- 172.21.15.70:0/584925922 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8dfc07a1e0 con 0x7f8e040c9130 2023-08-25T09:51:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:51:51.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.187+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8de806a020 msgr2=0x7f8de806c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:51.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.187+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8de806a020 0x7f8de806c4e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8e00003cc0 tx=0x7f8e000079b0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 msgr2=0x7f8e040c3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:51.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 0x7f8e040c3fa0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009e50 tx=0x7f8dfc004a40 comp rx=0 tx=0).stop 2023-08-25T09:51:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 shutdown_connections 2023-08-25T09:51:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8de806a020 0x7f8de806c4e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8e040c9130 0x7f8e040c3fa0 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8e040c21f0 0x7f8e040c3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 --2- 172.21.15.70:0/584925922 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8e0406cfe0 0x7f8e0415e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 >> 172.21.15.70:0/584925922 conn(0x7f8e040ceae0 msgr2=0x7f8e040cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 shutdown_connections 2023-08-25T09:51:51.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.188+0000 7f8e0b8e0640 1 -- 172.21.15.70:0/584925922 wait complete. 2023-08-25T09:51:51.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:51:51.223 INFO:tasks.ceph.mds.b.smithi155.stderr: -8609> 2023-08-25T09:50:39.484+0000 7f474e99e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:51:51.310 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:51:51.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.413+0000 7fbee1404640 1 Processor -- start 2023-08-25T09:51:51.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.414+0000 7fbee1404640 1 -- start start 2023-08-25T09:51:51.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.415+0000 7fbee1404640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbedc15c910 0x7fbedc069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:51.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.415+0000 7fbee1404640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fbedc15ccf0 0x7fbedc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:51.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.415+0000 7fbee1404640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc1606d0 con 0x7fbedc06f3b0 2023-08-25T09:51:51.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.415+0000 7fbee1404640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc06f7e0 con 0x7fbedc15c910 2023-08-25T09:51:51.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.415+0000 7fbee1404640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc0cfc40 con 0x7fbedc15ccf0 2023-08-25T09:51:51.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbedc15c910 0x7fbedc069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:51.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbedc15c910 0x7fbedc069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39072/0 (socket says 172.21.15.70:39072) 2023-08-25T09:51:51.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedb7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fbedc06f3b0 0x7fbedc0b2280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44438/0 (socket says 172.21.15.70:44438) 2023-08-25T09:51:51.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 -- 172.21.15.70:0/504177292 learned_addr learned my addr 172.21.15.70:0/504177292 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:51.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 -- 172.21.15.70:0/504177292 >> v2:172.21.15.155:3300/0 conn(0x7fbedc15ccf0 msgr2=0x7fbedc06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:51.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 --2- 172.21.15.70:0/504177292 >> v2:172.21.15.155:3300/0 conn(0x7fbedc15ccf0 0x7fbedc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 -- 172.21.15.70:0/504177292 >> v1:172.21.15.70:6789/0 conn(0x7fbedc06f3b0 legacy=0x7fbedc0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbedaffd640 1 -- 172.21.15.70:0/504177292 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc0cff20 con 0x7fbedc15c910 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.416+0000 7fbed9ffb640 1 -- 172.21.15.70:0/504177292 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194093697 0 0) 0x7fbedc1606d0 con 0x7fbedc06f3b0 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.417+0000 7fbedaffd640 1 --2- 172.21.15.70:0/504177292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc15c910 0x7fbedc069850 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0cde00 tx=0x7fbecc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78bd1ff9ae44269 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.417+0000 7fbed9ffb640 1 -- 172.21.15.70:0/504177292 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc030860 con 0x7fbedc15c910 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbed9ffb640 1 -- 172.21.15.70:0/504177292 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc0309c0 con 0x7fbedc15c910 2023-08-25T09:51:51.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbed9ffb640 1 -- 172.21.15.70:0/504177292 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc030d50 con 0x7fbedc15c910 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 -- 172.21.15.70:0/504177292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc15c910 msgr2=0x7fbedc069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 --2- 172.21.15.70:0/504177292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc15c910 0x7fbedc069850 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0cde00 tx=0x7fbecc02b9a0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 -- 172.21.15.70:0/504177292 shutdown_connections 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 --2- 172.21.15.70:0/504177292 >> v2:172.21.15.155:3300/0 conn(0x7fbedc15ccf0 0x7fbedc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 --2- 172.21.15.70:0/504177292 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc15c910 0x7fbedc069850 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 -- 172.21.15.70:0/504177292 >> 172.21.15.70:0/504177292 conn(0x7fbedc0cd280 msgr2=0x7fbedc0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:51.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 -- 172.21.15.70:0/504177292 shutdown_connections 2023-08-25T09:51:51.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.418+0000 7fbee1404640 1 -- 172.21.15.70:0/504177292 wait complete. 2023-08-25T09:51:51.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.420+0000 7fbee1404640 1 Processor -- start 2023-08-25T09:51:51.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.420+0000 7fbee1404640 1 -- start start 2023-08-25T09:51:51.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbee1404640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 0x7fbedc0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:51.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbee1404640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc15c910 0x7fbedc0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:51.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbee1404640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc15ccf0 0x7fbedc0c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:51.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbee1404640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c5ae0 con 0x7fbedc06f3b0 2023-08-25T09:51:51.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbee1404640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c4d20 con 0x7fbedc15ccf0 2023-08-25T09:51:51.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbee1404640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0c5000 con 0x7fbedc15c910 2023-08-25T09:51:51.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbedaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 0x7fbedc0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:51.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbedaffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 0x7fbedc0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39074/0 (socket says 172.21.15.70:39074) 2023-08-25T09:51:51.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.421+0000 7fbedaffd640 1 -- 172.21.15.70:0/3026822433 learned_addr learned my addr 172.21.15.70:0/3026822433 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:51.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbedaffd640 1 -- 172.21.15.70:0/3026822433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc15c910 msgr2=0x7fbedc0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:51.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbedaffd640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc15c910 0x7fbedc0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbedaffd640 1 -- 172.21.15.70:0/3026822433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc15ccf0 msgr2=0x7fbedc0c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbedaffd640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc15ccf0 0x7fbedc0c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbedaffd640 1 -- 172.21.15.70:0/3026822433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0094d0 con 0x7fbedc06f3b0 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbedaffd640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 0x7fbedc0af8e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fbecc000c00 tx=0x7fbecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc030830 con 0x7fbedc06f3b0 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbecc039d90 con 0x7fbedc06f3b0 2023-08-25T09:51:51.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbecc043650 con 0x7fbedc06f3b0 2023-08-25T09:51:51.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.422+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc0c52e0 con 0x7fbedc06f3b0 2023-08-25T09:51:51.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.423+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc0c4a30 con 0x7fbedc06f3b0 2023-08-25T09:51:51.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.424+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc04e420 con 0x7fbedc06f3b0 2023-08-25T09:51:51.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.425+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbecc02f3b0 con 0x7fbedc06f3b0 2023-08-25T09:51:51.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.426+0000 7fbed37fe640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbebc06a050 0x7fbebc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:51.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.426+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fbecc0aab10 con 0x7fbedc06f3b0 2023-08-25T09:51:51.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.426+0000 7fbeda7fc640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbebc06a050 0x7fbebc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:51.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.427+0000 7fbeda7fc640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbebc06a050 0x7fbebc06c510 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0cc8f0 tx=0x7fbec4006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:51.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.430+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbecc076290 con 0x7fbedc06f3b0 2023-08-25T09:51:51.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:51.654+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbedc0c5e40 con 0x7fbedc06f3b0 2023-08-25T09:51:52.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.195+0000 7fbed37fe640 1 -- 172.21.15.70:0/3026822433 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbecc0798b0 con 0x7fbedc06f3b0 2023-08-25T09:51:52.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.198+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbebc06a050 msgr2=0x7fbebc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:52.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.198+0000 7fbee1404640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbebc06a050 0x7fbebc06c510 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0cc8f0 tx=0x7fbec4006bf0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.198+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 msgr2=0x7fbedc0af8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:52.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.198+0000 7fbee1404640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 0x7fbedc0af8e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fbecc000c00 tx=0x7fbecc004730 comp rx=0 tx=0).stop 2023-08-25T09:51:52.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 shutdown_connections 2023-08-25T09:51:52.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbebc06a050 0x7fbebc06c510 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbedc15ccf0 0x7fbedc0c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbedc15c910 0x7fbedc0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 --2- 172.21.15.70:0/3026822433 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbedc06f3b0 0x7fbedc0af8e0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 >> 172.21.15.70:0/3026822433 conn(0x7fbedc0cd280 msgr2=0x7fbedc15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:52.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 shutdown_connections 2023-08-25T09:51:52.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.199+0000 7fbee1404640 1 -- 172.21.15.70:0/3026822433 wait complete. 2023-08-25T09:51:52.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:51:52.447 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.446+0000 7fa8cee56640 1 Processor -- start 2023-08-25T09:51:52.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.446+0000 7fa8cee56640 1 -- start start 2023-08-25T09:51:52.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.447+0000 7fa8cee56640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa8c815b380 0x7fa8c815b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:52.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.447+0000 7fa8cee56640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8c806ee30 0x7fa8c80cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:52.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.447+0000 7fa8cee56640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c80cbb90 con 0x7fa8c80ce200 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.447+0000 7fa8cee56640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c815f2d0 con 0x7fa8c806ee30 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8cee56640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c815bca0 con 0x7fa8c815b380 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8c806ee30 0x7fa8c80cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa8c806ee30 0x7fa8c80cc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39076/0 (socket says 172.21.15.70:39076) 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 -- 172.21.15.70:0/2416442664 learned_addr learned my addr 172.21.15.70:0/2416442664 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8cd653640 1 --1- 172.21.15.70:0/2416442664 >> v1:172.21.15.70:6789/0 conn(0x7fa8c80ce200 0x7fa8c806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44454/0 (socket says 172.21.15.70:44454) 2023-08-25T09:51:52.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8cde54640 1 --2- 172.21.15.70:0/2416442664 >> v2:172.21.15.155:3300/0 conn(0x7fa8c815b380 0x7fa8c815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:52.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 -- 172.21.15.70:0/2416442664 >> v2:172.21.15.155:3300/0 conn(0x7fa8c815b380 msgr2=0x7fa8c815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:52.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 --2- 172.21.15.70:0/2416442664 >> v2:172.21.15.155:3300/0 conn(0x7fa8c815b380 0x7fa8c815b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 -- 172.21.15.70:0/2416442664 >> v1:172.21.15.70:6789/0 conn(0x7fa8c80ce200 legacy=0x7fa8c806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:52.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8ce655640 1 -- 172.21.15.70:0/2416442664 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c815e750 con 0x7fa8c806ee30 2023-08-25T09:51:52.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.448+0000 7fa8cce52640 1 -- 172.21.15.70:0/2416442664 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1851509081 0 0) 0x7fa8c80cbb90 con 0x7fa8c80ce200 2023-08-25T09:51:52.455 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.449+0000 7fa8ce655640 1 --2- 172.21.15.70:0/2416442664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c806ee30 0x7fa8c80cc030 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80d1470 tx=0x7fa8c002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ada6a58e7c3278 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:52.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cce52640 1 -- 172.21.15.70:0/2416442664 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c002cb20 con 0x7fa8c806ee30 2023-08-25T09:51:52.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cce52640 1 -- 172.21.15.70:0/2416442664 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c002cc80 con 0x7fa8c806ee30 2023-08-25T09:51:52.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cce52640 1 -- 172.21.15.70:0/2416442664 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00343f0 con 0x7fa8c806ee30 2023-08-25T09:51:52.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 -- 172.21.15.70:0/2416442664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c806ee30 msgr2=0x7fa8c80cc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:52.456 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 --2- 172.21.15.70:0/2416442664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c806ee30 0x7fa8c80cc030 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80d1470 tx=0x7fa8c002bf80 comp rx=0 tx=0).stop 2023-08-25T09:51:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 -- 172.21.15.70:0/2416442664 shutdown_connections 2023-08-25T09:51:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 --2- 172.21.15.70:0/2416442664 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c806ee30 0x7fa8c80cc030 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 --2- 172.21.15.70:0/2416442664 >> v2:172.21.15.155:3300/0 conn(0x7fa8c815b380 0x7fa8c815b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 -- 172.21.15.70:0/2416442664 >> 172.21.15.70:0/2416442664 conn(0x7fa8c80d0630 msgr2=0x7fa8c80d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 -- 172.21.15.70:0/2416442664 shutdown_connections 2023-08-25T09:51:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.450+0000 7fa8cee56640 1 -- 172.21.15.70:0/2416442664 wait complete. 2023-08-25T09:51:52.460 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.452+0000 7fa8cee56640 1 Processor -- start 2023-08-25T09:51:52.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.452+0000 7fa8cee56640 1 -- start start 2023-08-25T09:51:52.461 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cee56640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c806ee30 0x7fa8c8157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:52.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cee56640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 0x7fa8c80c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:52.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cee56640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c815b380 0x7fa8c80ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:52.462 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cee56640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8158300 con 0x7fa8c80ce200 2023-08-25T09:51:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cee56640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80c8050 con 0x7fa8c815b380 2023-08-25T09:51:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cee56640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c80c8330 con 0x7fa8c806ee30 2023-08-25T09:51:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.453+0000 7fa8cd653640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 0x7fa8c80c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 0x7fa8c80c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39092/0 (socket says 172.21.15.70:39092) 2023-08-25T09:51:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 -- 172.21.15.70:0/876832964 learned_addr learned my addr 172.21.15.70:0/876832964 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:52.463 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 -- 172.21.15.70:0/876832964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c806ee30 msgr2=0x7fa8c8157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:52.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c806ee30 0x7fa8c8157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 -- 172.21.15.70:0/876832964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c815b380 msgr2=0x7fa8c80ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:52.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c815b380 0x7fa8c80ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:52.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 -- 172.21.15.70:0/876832964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0009b30 con 0x7fa8c80ce200 2023-08-25T09:51:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8cd653640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 0x7fa8c80c7700 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c4b0 tx=0x7fa8bc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0074e0 con 0x7fa8c80ce200 2023-08-25T09:51:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc007660 con 0x7fa8c80ce200 2023-08-25T09:51:52.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.454+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc004a20 con 0x7fa8c80ce200 2023-08-25T09:51:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.455+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c81f2d00 con 0x7fa8c80ce200 2023-08-25T09:51:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.455+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c81f32c0 con 0x7fa8c80ce200 2023-08-25T09:51:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.456+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c81f3500 con 0x7fa8c80ce200 2023-08-25T09:51:52.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.457+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa8bc004b80 con 0x7fa8c80ce200 2023-08-25T09:51:52.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.458+0000 7fa8c67fc640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8b406a080 0x7fa8b406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:52.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.458+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 6177+0+0 (secure 0 0 0) 0x7fa8bc087800 con 0x7fa8c80ce200 2023-08-25T09:51:52.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.458+0000 7fa8cde54640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8b406a080 0x7fa8b406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:52.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.459+0000 7fa8cde54640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8b406a080 0x7fa8b406c540 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006fd0 tx=0x7fa8b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:52.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.462+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa8c81f3500 con 0x7fa8c80ce200 2023-08-25T09:51:52.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:52.655+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8c81f3500 con 0x7fa8c80ce200 2023-08-25T09:51:53.508 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.507+0000 7fa8c67fc640 1 -- 172.21.15.70:0/876832964 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v189) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa8c81f3500 con 0x7fa8c80ce200 2023-08-25T09:51:53.511 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:51:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.513+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8b406a080 msgr2=0x7fa8b406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.513+0000 7fa8cee56640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8b406a080 0x7fa8b406c540 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006fd0 tx=0x7fa8b8008040 comp rx=0 tx=0).stop 2023-08-25T09:51:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.513+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 msgr2=0x7fa8c80c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:53.515 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.513+0000 7fa8cee56640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 0x7fa8c80c7700 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c4b0 tx=0x7fa8bc00c970 comp rx=0 tx=0).stop 2023-08-25T09:51:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 shutdown_connections 2023-08-25T09:51:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa8b406a080 0x7fa8b406c540 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa8c815b380 0x7fa8c80ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa8c80ce200 0x7fa8c80c7700 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.516 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 --2- 172.21.15.70:0/876832964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa8c806ee30 0x7fa8c8157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 >> 172.21.15.70:0/876832964 conn(0x7fa8c80d0630 msgr2=0x7fa8c80ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:53.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 shutdown_connections 2023-08-25T09:51:53.517 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.514+0000 7fa8cee56640 1 -- 172.21.15.70:0/876832964 wait complete. 2023-08-25T09:51:53.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:51:53.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.756+0000 7f39011b5640 1 Processor -- start 2023-08-25T09:51:53.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.756+0000 7f39011b5640 1 -- start start 2023-08-25T09:51:53.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.757+0000 7f39011b5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38fc155120 0x7f38fc0cd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:53.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.757+0000 7f39011b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:53.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.757+0000 7f39011b5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38fc0cacb0 con 0x7f38fc0ca820 2023-08-25T09:51:53.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.757+0000 7f39011b5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38fc0caf30 con 0x7f38fc155120 2023-08-25T09:51:53.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.757+0000 7f39011b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0cdd10 con 0x7f38fc0cd890 2023-08-25T09:51:53.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38fc155120 0x7f38fc0cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:53.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f38fc155120 0x7f38fc0cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39094/0 (socket says 172.21.15.70:39094) 2023-08-25T09:51:53.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 -- 172.21.15.70:0/1215259912 learned_addr learned my addr 172.21.15.70:0/1215259912 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:53.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 -- 172.21.15.70:0/1215259912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 msgr2=0x7f38fc0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:53.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fa59c640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:53.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 -- 172.21.15.70:0/1215259912 >> v1:172.21.15.155:6789/0 conn(0x7f38fc0ca820 legacy=0x7f38fc06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:53.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fad9d640 1 -- 172.21.15.70:0/1215259912 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc15e6f0 con 0x7f38fc155120 2023-08-25T09:51:53.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.758+0000 7f38fa59c640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:53.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.759+0000 7f38fad9d640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc155120 0x7f38fc0cd350 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009870 tx=0x7f38e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b60fb674aaf62a0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:53.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.760+0000 7f38f9d9b640 1 -- 172.21.15.70:0/1215259912 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4031640 con 0x7f38fc155120 2023-08-25T09:51:53.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.760+0000 7f38f9d9b640 1 -- 172.21.15.70:0/1215259912 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e40317a0 con 0x7f38fc155120 2023-08-25T09:51:53.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.760+0000 7f38f9d9b640 1 -- 172.21.15.70:0/1215259912 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4031b30 con 0x7f38fc155120 2023-08-25T09:51:53.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.760+0000 7f39011b5640 1 -- 172.21.15.70:0/1215259912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc155120 msgr2=0x7f38fc0cd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:53.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc155120 0x7f38fc0cd350 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009870 tx=0x7f38e402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 -- 172.21.15.70:0/1215259912 shutdown_connections 2023-08-25T09:51:53.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 --2- 172.21.15.70:0/1215259912 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc155120 0x7f38fc0cd350 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 -- 172.21.15.70:0/1215259912 >> 172.21.15.70:0/1215259912 conn(0x7f38fc069c70 msgr2=0x7f38fc06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:53.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 -- 172.21.15.70:0/1215259912 shutdown_connections 2023-08-25T09:51:53.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.761+0000 7f39011b5640 1 -- 172.21.15.70:0/1215259912 wait complete. 2023-08-25T09:51:53.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.763+0000 7f39011b5640 1 Processor -- start 2023-08-25T09:51:53.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.763+0000 7f39011b5640 1 -- start start 2023-08-25T09:51:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.764+0000 7f39011b5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 0x7f38fc0c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.764+0000 7f39011b5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.764+0000 7f39011b5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38fc155120 0x7f38fc0cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.764+0000 7f39011b5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc1518b0 con 0x7f38fc0ca820 2023-08-25T09:51:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.764+0000 7f39011b5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0c77d0 con 0x7f38fc155120 2023-08-25T09:51:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.764+0000 7f39011b5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38fc0c6350 con 0x7f38fc0cd890 2023-08-25T09:51:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 0x7f38fc0c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 0x7f38fc0c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39110/0 (socket says 172.21.15.70:39110) 2023-08-25T09:51:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 -- 172.21.15.70:0/3964806896 learned_addr learned my addr 172.21.15.70:0/3964806896 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 -- 172.21.15.70:0/3964806896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 msgr2=0x7f38fc151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 -- 172.21.15.70:0/3964806896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38fc155120 msgr2=0x7f38fc0cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38fc155120 0x7f38fc0cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 -- 172.21.15.70:0/3964806896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40094d0 con 0x7f38fc0ca820 2023-08-25T09:51:53.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.765+0000 7f38fad9d640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 0x7f38fc0c5720 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f38e4000c00 tx=0x7f38e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.766+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e403ac60 con 0x7f38fc0ca820 2023-08-25T09:51:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.766+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e403ade0 con 0x7f38fc0ca820 2023-08-25T09:51:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.766+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38fc0c9a30 con 0x7f38fc0ca820 2023-08-25T09:51:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.766+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4044630 con 0x7f38fc0ca820 2023-08-25T09:51:53.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.766+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38fc0c6070 con 0x7f38fc0ca820 2023-08-25T09:51:53.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.767+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38fc04e420 con 0x7f38fc0ca820 2023-08-25T09:51:53.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.768+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f38e402f050 con 0x7f38fc0ca820 2023-08-25T09:51:53.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.769+0000 7f38f37fe640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f38dc06a050 0x7f38dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:53.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.769+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f38e40aa4d0 con 0x7f38fc0ca820 2023-08-25T09:51:53.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.769+0000 7f38fa59c640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f38dc06a050 0x7f38dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:53.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.770+0000 7f38fa59c640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f38dc06a050 0x7f38dc06c510 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0ceee0 tx=0x7f38ec008330 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:53.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.773+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f38e4075db0 con 0x7f38fc0ca820 2023-08-25T09:51:53.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:53.965+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38fc0715c0 con 0x7f38fc0ca820 2023-08-25T09:51:54.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.517+0000 7f38f37fe640 1 -- 172.21.15.70:0/3964806896 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v190) v1 ==== 169+0+0 (secure 0 0 0) 0x7f38e40793d0 con 0x7f38fc0ca820 2023-08-25T09:51:54.519 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:51:54.523 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.521+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f38dc06a050 msgr2=0x7f38dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.521+0000 7f39011b5640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f38dc06a050 0x7f38dc06c510 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0ceee0 tx=0x7f38ec008330 comp rx=0 tx=0).stop 2023-08-25T09:51:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.521+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 msgr2=0x7f38fc0c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.521+0000 7f39011b5640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 0x7f38fc0c5720 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f38e4000c00 tx=0x7f38e40046c0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.524 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 shutdown_connections 2023-08-25T09:51:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f38dc06a050 0x7f38dc06c510 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f38fc155120 0x7f38fc0cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f38fc0cd890 0x7f38fc151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 --2- 172.21.15.70:0/3964806896 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f38fc0ca820 0x7f38fc0c5720 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 >> 172.21.15.70:0/3964806896 conn(0x7f38fc069c70 msgr2=0x7f38fc070150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 shutdown_connections 2023-08-25T09:51:54.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.522+0000 7f39011b5640 1 -- 172.21.15.70:0/3964806896 wait complete. 2023-08-25T09:51:54.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:51:54.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.765+0000 7fcdcaa21640 1 Processor -- start 2023-08-25T09:51:54.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.765+0000 7fcdcaa21640 1 -- start start 2023-08-25T09:51:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdcaa21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdc415abb0 0x7fcdc40b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdcaa21640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fcdc4071850 0x7fcdc406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdcaa21640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc415e4a0 con 0x7fcdc415a7d0 2023-08-25T09:51:54.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdcaa21640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40c53b0 con 0x7fcdc4071850 2023-08-25T09:51:54.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdcaa21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415f020 con 0x7fcdc415abb0 2023-08-25T09:51:54.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdc9a1f640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fcdc415a7d0 0x7fcdc40b3620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44470/0 (socket says 172.21.15.70:44470) 2023-08-25T09:51:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdc9a1f640 1 -- 172.21.15.70:0/894654596 learned_addr learned my addr 172.21.15.70:0/894654596 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdc921e640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdc415abb0 0x7fcdc40b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.767+0000 7fcdca220640 1 --2- 172.21.15.70:0/894654596 >> v2:172.21.15.155:3300/0 conn(0x7fcdc4071850 0x7fcdc406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904838172 0 0) 0x7fcdc415e4a0 con 0x7fcdc415a7d0 2023-08-25T09:51:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb0003520 con 0x7fcdc415a7d0 2023-08-25T09:51:54.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2282583200 0 0) 0x7fcdb0003520 con 0x7fcdc415a7d0 2023-08-25T09:51:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdc415abb0 msgr2=0x7fcdc40b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdc415abb0 0x7fcdc40b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 >> v2:172.21.15.155:3300/0 conn(0x7fcdc4071850 msgr2=0x7fcdc406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 --2- 172.21.15.70:0/894654596 >> v2:172.21.15.155:3300/0 conn(0x7fcdc4071850 0x7fcdc406ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc415e6f0 con 0x7fcdc415a7d0 2023-08-25T09:51:54.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fcdc0001670 con 0x7fcdc415a7d0 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 >> v1:172.21.15.70:6789/0 conn(0x7fcdc415a7d0 legacy=0x7fcdc40b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdb0003970 0x7fcdb0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb00047c0 0x7fcdb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb0005180 0x7fcdb000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc415f020 con 0x7fcdb00047c0 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb000bec0 con 0x7fcdb0003970 2023-08-25T09:51:54.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.768+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb000c000 con 0x7fcdb0005180 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc921e640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb00047c0 0x7fcdb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc921e640 1 -- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb0005180 msgr2=0x7fcdb000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdca220640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb0005180 0x7fcdb000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc921e640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb0005180 0x7fcdb000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc9a1f640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdb0003970 0x7fcdb0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc921e640 1 -- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdb0003970 msgr2=0x7fcdb0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:54.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc921e640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdb0003970 0x7fcdb0003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdc921e640 1 -- 172.21.15.70:0/894654596 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0003520 con 0x7fcdb00047c0 2023-08-25T09:51:54.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.769+0000 7fcdca220640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb0005180 0x7fcdb000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:54.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.770+0000 7fcdc921e640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb00047c0 0x7fcdb0004c40 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009ac0 tx=0x7fcdb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b371a1b5849b6438 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:54.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.770+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb402c6b0 con 0x7fcdb00047c0 2023-08-25T09:51:54.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.770+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb402c810 con 0x7fcdb00047c0 2023-08-25T09:51:54.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdc8a1d640 1 -- 172.21.15.70:0/894654596 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb402cba0 con 0x7fcdb00047c0 2023-08-25T09:51:54.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdcaa21640 1 -- 172.21.15.70:0/894654596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb00047c0 msgr2=0x7fcdb0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/894654596 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdb00047c0 0x7fcdb0004c40 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009ac0 tx=0x7fcdb00057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdca220640 1 -- 172.21.15.70:0/894654596 reap_dead start 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdcaa21640 1 -- 172.21.15.70:0/894654596 shutdown_connections 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdcaa21640 1 -- 172.21.15.70:0/894654596 >> 172.21.15.70:0/894654596 conn(0x7fcdc40aff60 msgr2=0x7fcdc40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdcaa21640 1 -- 172.21.15.70:0/894654596 shutdown_connections 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.771+0000 7fcdcaa21640 1 -- 172.21.15.70:0/894654596 wait complete. 2023-08-25T09:51:54.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.773+0000 7fcdcaa21640 1 Processor -- start 2023-08-25T09:51:54.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.773+0000 7fcdcaa21640 1 -- start start 2023-08-25T09:51:54.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdcaa21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb00047c0 0x7fcdc40cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdcaa21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 0x7fcdc4150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdcaa21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdc4151100 0x7fcdc40cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdcaa21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4151580 con 0x7fcdc414a1a0 2023-08-25T09:51:54.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdcaa21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40ce020 con 0x7fcdc4151100 2023-08-25T09:51:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdcaa21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4158e70 con 0x7fcdb00047c0 2023-08-25T09:51:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.774+0000 7fcdc921e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 0x7fcdc4150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 0x7fcdc4150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39130/0 (socket says 172.21.15.70:39130) 2023-08-25T09:51:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 -- 172.21.15.70:0/3763758529 learned_addr learned my addr 172.21.15.70:0/3763758529 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 -- 172.21.15.70:0/3763758529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb00047c0 msgr2=0x7fcdc40cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb00047c0 0x7fcdc40cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 -- 172.21.15.70:0/3763758529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdc4151100 msgr2=0x7fcdc40cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdc4151100 0x7fcdc40cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 -- 172.21.15.70:0/3763758529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4159150 con 0x7fcdc414a1a0 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.775+0000 7fcdc921e640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 0x7fcdc4150bc0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4037040 tx=0x7fcdb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.776+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4003810 con 0x7fcdc414a1a0 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.776+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb4003970 con 0x7fcdc414a1a0 2023-08-25T09:51:54.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.776+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4003ce0 con 0x7fcdc414a1a0 2023-08-25T09:51:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.776+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc4159430 con 0x7fcdc414a1a0 2023-08-25T09:51:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.776+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc41599f0 con 0x7fcdc414a1a0 2023-08-25T09:51:54.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.777+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc4151520 con 0x7fcdc414a1a0 2023-08-25T09:51:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.778+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcdb4047450 con 0x7fcdc414a1a0 2023-08-25T09:51:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.779+0000 7fcdba7fc640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcda806a080 0x7fcda806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.779+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcdb40aa4d0 con 0x7fcdc414a1a0 2023-08-25T09:51:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.780+0000 7fcdc9a1f640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcda806a080 0x7fcda806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:54.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.780+0000 7fcdc9a1f640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcda806a080 0x7fcda806c540 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001eb0 tx=0x7fcdc0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:54.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.783+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcdc4151520 con 0x7fcdc414a1a0 2023-08-25T09:51:54.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.970+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcdc4151520 con 0x7fcdc414a1a0 2023-08-25T09:51:54.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.977+0000 7fcdba7fc640 1 -- 172.21.15.70:0/3763758529 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcdb4079a10 con 0x7fcdc414a1a0 2023-08-25T09:51:54.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcda806a080 msgr2=0x7fcda806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcda806a080 0x7fcda806c540 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0001eb0 tx=0x7fcdc0009040 comp rx=0 tx=0).stop 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 msgr2=0x7fcdc4150bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 0x7fcdc4150bc0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4037040 tx=0x7fcdb4004730 comp rx=0 tx=0).stop 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 shutdown_connections 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fcda806a080 0x7fcda806c540 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fcdc4151100 0x7fcdc40cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fcdc414a1a0 0x7fcdc4150bc0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 --2- 172.21.15.70:0/3763758529 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fcdb00047c0 0x7fcdc40cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:54.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 >> 172.21.15.70:0/3763758529 conn(0x7fcdc40aff60 msgr2=0x7fcdc414f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:54.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 shutdown_connections 2023-08-25T09:51:54.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:54.980+0000 7fcdcaa21640 1 -- 172.21.15.70:0/3763758529 wait complete. 2023-08-25T09:51:55.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-08-25T09:51:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.200+0000 7fdfd7ade640 1 Processor -- start 2023-08-25T09:51:55.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.200+0000 7fdfd7ade640 1 -- start start 2023-08-25T09:51:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd7ade640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fdfd015ec10 0x7fdfd015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd7ade640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fdfd015c6d0 0x7fdfd00b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd7ade640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd00b1b60 con 0x7fdfd015e830 2023-08-25T09:51:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd7ade640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd006bd80 con 0x7fdfd015c6d0 2023-08-25T09:51:55.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd7ade640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd006b7c0 con 0x7fdfd015ec10 2023-08-25T09:51:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd587a640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fdfd015e830 0x7fdfd006ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44482/0 (socket says 172.21.15.70:44482) 2023-08-25T09:51:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd587a640 1 -- 172.21.15.70:0/1762758205 learned_addr learned my addr 172.21.15.70:0/1762758205 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:55.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd607b640 1 --2- 172.21.15.70:0/1762758205 >> v2:172.21.15.70:3300/0 conn(0x7fdfd015c6d0 0x7fdfd00b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd607b640 1 -- 172.21.15.70:0/1762758205 >> v2:172.21.15.155:3300/0 conn(0x7fdfd015ec10 msgr2=0x7fdfd015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd5079640 1 --2- 172.21.15.70:0/1762758205 >> v2:172.21.15.155:3300/0 conn(0x7fdfd015ec10 0x7fdfd015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.203+0000 7fdfd607b640 1 --2- 172.21.15.70:0/1762758205 >> v2:172.21.15.155:3300/0 conn(0x7fdfd015ec10 0x7fdfd015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.203+0000 7fdfd607b640 1 -- 172.21.15.70:0/1762758205 >> v1:172.21.15.70:6789/0 conn(0x7fdfd015e830 legacy=0x7fdfd006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.202+0000 7fdfd4878640 1 -- 172.21.15.70:0/1762758205 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371967073 0 0) 0x7fdfd00b1b60 con 0x7fdfd015e830 2023-08-25T09:51:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.203+0000 7fdfd607b640 1 -- 172.21.15.70:0/1762758205 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd006baa0 con 0x7fdfd015c6d0 2023-08-25T09:51:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.203+0000 7fdfd5079640 1 --2- 172.21.15.70:0/1762758205 >> v2:172.21.15.155:3300/0 conn(0x7fdfd015ec10 0x7fdfd015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:55.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.204+0000 7fdfd607b640 1 --2- 172.21.15.70:0/1762758205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015c6d0 0x7fdfd00b1ff0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00cf6f0 tx=0x7fdfc802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8df19e8faebfdd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:55.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.204+0000 7fdfd4878640 1 -- 172.21.15.70:0/1762758205 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc802cbf0 con 0x7fdfd015c6d0 2023-08-25T09:51:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.204+0000 7fdfd4878640 1 -- 172.21.15.70:0/1762758205 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc802cd50 con 0x7fdfd015c6d0 2023-08-25T09:51:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.204+0000 7fdfd4878640 1 -- 172.21.15.70:0/1762758205 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8034660 con 0x7fdfd015c6d0 2023-08-25T09:51:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 -- 172.21.15.70:0/1762758205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015c6d0 msgr2=0x7fdfd00b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/1762758205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015c6d0 0x7fdfd00b1ff0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00cf6f0 tx=0x7fdfc802be40 comp rx=0 tx=0).stop 2023-08-25T09:51:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 -- 172.21.15.70:0/1762758205 shutdown_connections 2023-08-25T09:51:55.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/1762758205 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015c6d0 0x7fdfd00b1ff0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/1762758205 >> v2:172.21.15.155:3300/0 conn(0x7fdfd015ec10 0x7fdfd015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 -- 172.21.15.70:0/1762758205 >> 172.21.15.70:0/1762758205 conn(0x7fdfd0069850 msgr2=0x7fdfd0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 -- 172.21.15.70:0/1762758205 shutdown_connections 2023-08-25T09:51:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.205+0000 7fdfd7ade640 1 -- 172.21.15.70:0/1762758205 wait complete. 2023-08-25T09:51:55.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.207+0000 7fdfd7ade640 1 Processor -- start 2023-08-25T09:51:55.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.207+0000 7fdfd7ade640 1 -- start start 2023-08-25T09:51:55.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd7ade640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd015c6d0 0x7fdfd01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd7ade640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfd015e830 0x7fdfd00cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd7ade640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 0x7fdfd00ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd7ade640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01f6470 con 0x7fdfd015ec10 2023-08-25T09:51:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd7ade640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd00aca40 con 0x7fdfd015c6d0 2023-08-25T09:51:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd7ade640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd01f2230 con 0x7fdfd015e830 2023-08-25T09:51:55.213 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd607b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 0x7fdfd00ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd607b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 0x7fdfd00ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39144/0 (socket says 172.21.15.70:39144) 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.208+0000 7fdfd607b640 1 -- 172.21.15.70:0/2521462487 learned_addr learned my addr 172.21.15.70:0/2521462487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd5079640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfd015e830 0x7fdfd00cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd587a640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd015c6d0 0x7fdfd01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd607b640 1 -- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfd015e830 msgr2=0x7fdfd00cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd607b640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfd015e830 0x7fdfd00cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd607b640 1 -- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd015c6d0 msgr2=0x7fdfd01595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd607b640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd015c6d0 0x7fdfd01595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd607b640 1 -- 172.21.15.70:0/2521462487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc8009a70 con 0x7fdfd015ec10 2023-08-25T09:51:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd607b640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 0x7fdfd00ccca0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e80 tx=0x7fdfc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8040650 con 0x7fdfd015ec10 2023-08-25T09:51:55.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc80407d0 con 0x7fdfd015ec10 2023-08-25T09:51:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8040b80 con 0x7fdfd015ec10 2023-08-25T09:51:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.209+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd01f2510 con 0x7fdfd015ec10 2023-08-25T09:51:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.210+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd01f2ad0 con 0x7fdfd015ec10 2023-08-25T09:51:55.216 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.211+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd004e420 con 0x7fdfd015ec10 2023-08-25T09:51:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.212+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdfc8040ce0 con 0x7fdfd015ec10 2023-08-25T09:51:55.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.213+0000 7fdfc67fc640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdfb406a050 0x7fdfb406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.213+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdfc80ab250 con 0x7fdfd015ec10 2023-08-25T09:51:55.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.213+0000 7fdfd587a640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdfb406a050 0x7fdfb406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.214+0000 7fdfd587a640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdfb406a050 0x7fdfb406c510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000f60 tx=0x7fdfcc007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:55.218 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.217+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdfc8076bc0 con 0x7fdfd015ec10 2023-08-25T09:51:55.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.474+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7fdfd006c620 con 0x7fdfd015ec10 2023-08-25T09:51:55.519 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.518+0000 7fdfc67fc640 1 -- 172.21.15.70:0/2521462487 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v640) v1 ==== 131+0+0 (secure 0 0 0) 0x7fdfc807a1e0 con 0x7fdfd015ec10 2023-08-25T09:51:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdfb406a050 msgr2=0x7fdfb406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.525 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdfb406a050 0x7fdfb406c510 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc000f60 tx=0x7fdfcc007b50 comp rx=0 tx=0).stop 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 msgr2=0x7fdfd00ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 0x7fdfd00ccca0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8009e80 tx=0x7fdfc8004a40 comp rx=0 tx=0).stop 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 shutdown_connections 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fdfb406a050 0x7fdfb406c510 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fdfd015ec10 0x7fdfd00ccca0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fdfd015e830 0x7fdfd00cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 --2- 172.21.15.70:0/2521462487 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fdfd015c6d0 0x7fdfd01595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 >> 172.21.15.70:0/2521462487 conn(0x7fdfd0069850 msgr2=0x7fdfd0070020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:55.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 shutdown_connections 2023-08-25T09:51:55.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.521+0000 7fdfd7ade640 1 -- 172.21.15.70:0/2521462487 wait complete. 2023-08-25T09:51:55.542 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-08-25T09:51:55.543 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-08-25T09:51:55.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:51:55.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.755+0000 7fc1d3294640 1 Processor -- start 2023-08-25T09:51:55.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.755+0000 7fc1d3294640 1 -- start start 2023-08-25T09:51:55.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.757+0000 7fc1d3294640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 0x7fc1cc0b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.757+0000 7fc1d3294640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc1cc0ac870 0x7fc1cc0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.757+0000 7fc1d3294640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1cc06f050 con 0x7fc1cc15cb40 2023-08-25T09:51:55.760 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.757+0000 7fc1d3294640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1cc06f330 con 0x7fc1cc0ac870 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.757+0000 7fc1d3294640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc06f610 con 0x7fc1cc0cea30 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1d1030640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc1cc15cb40 0x7fc1cc0ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44494/0 (socket says 172.21.15.70:44494) 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1d1030640 1 -- 172.21.15.70:0/54255828 learned_addr learned my addr 172.21.15.70:0/54255828 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1d082f640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 0x7fc1cc0b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1d1831640 1 --2- 172.21.15.70:0/54255828 >> v2:172.21.15.155:3300/0 conn(0x7fc1cc0ac870 0x7fc1cc0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016657538 0 0) 0x7fc1cc06f050 con 0x7fc1cc15cb40 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b8003520 con 0x7fc1cc15cb40 2023-08-25T09:51:55.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 421607356 0 0) 0x7fc1b8003520 con 0x7fc1cc15cb40 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 msgr2=0x7fc1cc0b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1c3fff640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 0x7fc1cc0b1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.758+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 >> v2:172.21.15.155:3300/0 conn(0x7fc1cc0ac870 msgr2=0x7fc1cc0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 --2- 172.21.15.70:0/54255828 >> v2:172.21.15.155:3300/0 conn(0x7fc1cc0ac870 0x7fc1cc0b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc06f8f0 con 0x7fc1cc15cb40 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc1c8001670 con 0x7fc1cc15cb40 2023-08-25T09:51:55.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 >> v1:172.21.15.70:6789/0 conn(0x7fc1cc15cb40 legacy=0x7fc1cc0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 0x7fc1b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8003ef0 0x7fc1b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc06f330 con 0x7fc1b8004d00 2023-08-25T09:51:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc06f610 con 0x7fc1b8003ef0 2023-08-25T09:51:55.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b800b980 con 0x7fc1cc0cea30 2023-08-25T09:51:55.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1831640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d082f640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8003ef0 0x7fc1b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1030640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 0x7fc1b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1831640 1 -- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 msgr2=0x7fc1b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1831640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc0cea30 0x7fc1b8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1831640 1 -- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8003ef0 msgr2=0x7fc1b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1831640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8003ef0 0x7fc1b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d1831640 1 -- 172.21.15.70:0/54255828 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b8003520 con 0x7fc1b8004d00 2023-08-25T09:51:55.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.759+0000 7fc1d082f640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1b8003ef0 0x7fc1b80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.760+0000 7fc1d1831640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1b8005180 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fc0 tx=0x7fc1b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512d82c4c113f5d0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c402cc40 con 0x7fc1b8004d00 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c400f040 con 0x7fc1b8004d00 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1c3fff640 1 -- 172.21.15.70:0/54255828 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4034440 con 0x7fc1b8004d00 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d3294640 1 -- 172.21.15.70:0/54255828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 msgr2=0x7fc1b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d3294640 1 --2- 172.21.15.70:0/54255828 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1b8005180 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4009fc0 tx=0x7fc1b80057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d1831640 1 -- 172.21.15.70:0/54255828 reap_dead start 2023-08-25T09:51:55.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d3294640 1 -- 172.21.15.70:0/54255828 shutdown_connections 2023-08-25T09:51:55.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d3294640 1 -- 172.21.15.70:0/54255828 >> 172.21.15.70:0/54255828 conn(0x7fc1cc069850 msgr2=0x7fc1cc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:55.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d3294640 1 -- 172.21.15.70:0/54255828 shutdown_connections 2023-08-25T09:51:55.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.761+0000 7fc1d3294640 1 -- 172.21.15.70:0/54255828 wait complete. 2023-08-25T09:51:55.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.763+0000 7fc1d3294640 1 Processor -- start 2023-08-25T09:51:55.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.763+0000 7fc1d3294640 1 -- start start 2023-08-25T09:51:55.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.764+0000 7fc1d3294640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1cc1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.764+0000 7fc1d3294640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc15d1c0 0x7fc1cc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.764+0000 7fc1d3294640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0b1a30 0x7fc1cc0b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.764+0000 7fc1d3294640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0b2420 con 0x7fc1b8004d00 2023-08-25T09:51:55.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.764+0000 7fc1d3294640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc15d6c0 con 0x7fc1cc0b1a30 2023-08-25T09:51:55.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.764+0000 7fc1d3294640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc15d9a0 con 0x7fc1cc15d1c0 2023-08-25T09:51:55.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1cc1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1cc1546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39170/0 (socket says 172.21.15.70:39170) 2023-08-25T09:51:55.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 -- 172.21.15.70:0/4001334067 learned_addr learned my addr 172.21.15.70:0/4001334067 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:55.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 -- 172.21.15.70:0/4001334067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc15d1c0 msgr2=0x7fc1cc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc15d1c0 0x7fc1cc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 -- 172.21.15.70:0/4001334067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0b1a30 msgr2=0x7fc1cc0b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0b1a30 0x7fc1cc0b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 -- 172.21.15.70:0/4001334067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc15dc80 con 0x7fc1b8004d00 2023-08-25T09:51:55.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1d1030640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1cc1546a0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800d8e0 tx=0x7fc1c800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:55.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.765+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c80074e0 con 0x7fc1b8004d00 2023-08-25T09:51:55.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.766+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c8007640 con 0x7fc1b8004d00 2023-08-25T09:51:55.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.766+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc15df60 con 0x7fc1b8004d00 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.766+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8004a20 con 0x7fc1b8004d00 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.766+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc15c1e0 con 0x7fc1b8004d00 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.767+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc04e420 con 0x7fc1b8004d00 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.768+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc1c8004b80 con 0x7fc1b8004d00 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.769+0000 7fc1c1ffb640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1b006a080 0x7fc1b006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.769+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc1c8087ff0 con 0x7fc1b8004d00 2023-08-25T09:51:55.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.769+0000 7fc1d082f640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1b006a080 0x7fc1b006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:55.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.770+0000 7fc1d082f640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1b006a080 0x7fc1b006c540 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0b2610 tx=0x7fc1bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:55.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.773+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc1c80574b0 con 0x7fc1b8004d00 2023-08-25T09:51:55.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.962+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1cc155ab0 con 0x7fc1b8004d00 2023-08-25T09:51:55.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.964+0000 7fc1c1ffb640 1 -- 172.21.15.70:0/4001334067 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+14796 (secure 0 0 0) 0x7fc1c8057040 con 0x7fc1b8004d00 2023-08-25T09:51:55.966 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:55.966 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":190,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:54.500951+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.155:6834/3137406973":"2023-08-26T09:51:51.178223+0000","172.21.15.155:6832/3137406973":"2023-08-26T09:51:51.178223+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:51:55.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.966+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1b006a080 msgr2=0x7fc1b006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.966+0000 7fc1d3294640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1b006a080 0x7fc1b006c540 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0b2610 tx=0x7fc1bc008040 comp rx=0 tx=0).stop 2023-08-25T09:51:55.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.966+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 msgr2=0x7fc1cc1546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:55.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.966+0000 7fc1d3294640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1cc1546a0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800d8e0 tx=0x7fc1c800dda0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 shutdown_connections 2023-08-25T09:51:55.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc1b006a080 0x7fc1b006c540 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc1cc0b1a30 0x7fc1cc0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc1cc15d1c0 0x7fc1cc15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 --2- 172.21.15.70:0/4001334067 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc1b8004d00 0x7fc1cc1546a0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 >> 172.21.15.70:0/4001334067 conn(0x7fc1cc069850 msgr2=0x7fc1cc1553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 shutdown_connections 2023-08-25T09:51:55.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:55.967+0000 7fc1d3294640 1 -- 172.21.15.70:0/4001334067 wait complete. 2023-08-25T09:51:55.988 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:51:55.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:51:56.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.199+0000 7fd052a06640 1 Processor -- start 2023-08-25T09:51:56.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.200+0000 7fd052a06640 1 -- start start 2023-08-25T09:51:56.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.201+0000 7fd052a06640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd04c15ae50 0x7fd04c0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.201+0000 7fd052a06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd04c0cdc20 0x7fd04c069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.201+0000 7fd052a06640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c0cc5f0 con 0x7fd04c0ce060 2023-08-25T09:51:56.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.201+0000 7fd052a06640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c070650 con 0x7fd04c0cdc20 2023-08-25T09:51:56.203 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.201+0000 7fd052a06640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c070930 con 0x7fd04c15ae50 2023-08-25T09:51:56.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.201+0000 7fd043fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd04c0cdc20 0x7fd04c069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd043fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd04c0cdc20 0x7fd04c069850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39184/0 (socket says 172.21.15.70:39184) 2023-08-25T09:51:56.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd043fff640 1 -- 172.21.15.70:0/1011310364 learned_addr learned my addr 172.21.15.70:0/1011310364 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:56.204 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd04bfff640 1 --2- 172.21.15.70:0/1011310364 >> v2:172.21.15.155:3300/0 conn(0x7fd04c15ae50 0x7fd04c0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd043fff640 1 -- 172.21.15.70:0/1011310364 >> v2:172.21.15.155:3300/0 conn(0x7fd04c15ae50 msgr2=0x7fd04c0c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:56.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd043fff640 1 --2- 172.21.15.70:0/1011310364 >> v2:172.21.15.155:3300/0 conn(0x7fd04c15ae50 0x7fd04c0c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd043fff640 1 -- 172.21.15.70:0/1011310364 >> v1:172.21.15.155:6789/0 conn(0x7fd04c0ce060 legacy=0x7fd04c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:56.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd043fff640 1 -- 172.21.15.70:0/1011310364 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c070c10 con 0x7fd04c0cdc20 2023-08-25T09:51:56.205 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.202+0000 7fd04bfff640 1 --2- 172.21.15.70:0/1011310364 >> v2:172.21.15.155:3300/0 conn(0x7fd04c15ae50 0x7fd04c0c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:56.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.203+0000 7fd043fff640 1 --2- 172.21.15.70:0/1011310364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c069850 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0b1210 tx=0x7fd03402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f508315a087db8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:56.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd04b7fe640 1 -- 172.21.15.70:0/1011310364 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034030860 con 0x7fd04c0cdc20 2023-08-25T09:51:56.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd04b7fe640 1 -- 172.21.15.70:0/1011310364 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0340309c0 con 0x7fd04c0cdc20 2023-08-25T09:51:56.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd04b7fe640 1 -- 172.21.15.70:0/1011310364 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034030d50 con 0x7fd04c0cdc20 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 -- 172.21.15.70:0/1011310364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 msgr2=0x7fd04c069850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 --2- 172.21.15.70:0/1011310364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c069850 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0b1210 tx=0x7fd03402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 -- 172.21.15.70:0/1011310364 shutdown_connections 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 --2- 172.21.15.70:0/1011310364 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c069850 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 --2- 172.21.15.70:0/1011310364 >> v2:172.21.15.155:3300/0 conn(0x7fd04c15ae50 0x7fd04c0c2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 -- 172.21.15.70:0/1011310364 >> 172.21.15.70:0/1011310364 conn(0x7fd04c0cb0d0 msgr2=0x7fd04c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:56.207 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 -- 172.21.15.70:0/1011310364 shutdown_connections 2023-08-25T09:51:56.208 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.204+0000 7fd052a06640 1 -- 172.21.15.70:0/1011310364 wait complete. 2023-08-25T09:51:56.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.206+0000 7fd052a06640 1 Processor -- start 2023-08-25T09:51:56.209 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.206+0000 7fd052a06640 1 -- start start 2023-08-25T09:51:56.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd052a06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd052a06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd04c0ce060 0x7fd04c148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd052a06640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd04c15ae50 0x7fd04c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd052a06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c149850 con 0x7fd04c0cdc20 2023-08-25T09:51:56.210 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd052a06640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0c72c0 con 0x7fd04c15ae50 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd052a06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0c75a0 con 0x7fd04c0ce060 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39194/0 (socket says 172.21.15.70:39194) 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.207+0000 7fd04bfff640 1 -- 172.21.15.70:0/1530917877 learned_addr learned my addr 172.21.15.70:0/1530917877 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd04bfff640 1 -- 172.21.15.70:0/1530917877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd04c0ce060 msgr2=0x7fd04c148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd04bfff640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd04c0ce060 0x7fd04c148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.211 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd04bfff640 1 -- 172.21.15.70:0/1530917877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd04c15ae50 msgr2=0x7fd04c0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd04bfff640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd04c15ae50 0x7fd04c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd04bfff640 1 -- 172.21.15.70:0/1530917877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340094d0 con 0x7fd04c0cdc20 2023-08-25T09:51:56.212 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd04bfff640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c148a80 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c7f0 tx=0x7fd03c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:56.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c007820 con 0x7fd04c0cdc20 2023-08-25T09:51:56.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c007980 con 0x7fd04c0cdc20 2023-08-25T09:51:56.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c007cf0 con 0x7fd04c0cdc20 2023-08-25T09:51:56.214 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.208+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c0c7880 con 0x7fd04c0cdc20 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.209+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c0c7e40 con 0x7fd04c0cdc20 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.210+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c066fb0 con 0x7fd04c0cdc20 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.211+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd03c0113f0 con 0x7fd04c0cdc20 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.212+0000 7fd0497fa640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd02c06a050 0x7fd02c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.212+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd03c088670 con 0x7fd04c0cdc20 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.212+0000 7fd043fff640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd02c06a050 0x7fd02c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.215 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.213+0000 7fd043fff640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd02c06a050 0x7fd02c06c510 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd034036000 tx=0x7fd03403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:56.217 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.216+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd03c053fe0 con 0x7fd04c0cdc20 2023-08-25T09:51:56.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.401+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd04c0ccf70 con 0x7fd04c0cdc20 2023-08-25T09:51:56.403 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.402+0000 7fd0497fa640 1 -- 172.21.15.70:0/1530917877 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd03c057600 con 0x7fd04c0cdc20 2023-08-25T09:51:56.404 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:51:56.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd02c06a050 msgr2=0x7fd02c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd02c06a050 0x7fd02c06c510 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd034036000 tx=0x7fd03403b040 comp rx=0 tx=0).stop 2023-08-25T09:51:56.407 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 msgr2=0x7fd04c148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c148a80 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fd03c00c7f0 tx=0x7fd03c00ccb0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.408 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 shutdown_connections 2023-08-25T09:51:56.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd02c06a050 0x7fd02c06c510 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd04c15ae50 0x7fd04c0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd04c0ce060 0x7fd04c148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.409 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 --2- 172.21.15.70:0/1530917877 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd04c0cdc20 0x7fd04c148a80 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 >> 172.21.15.70:0/1530917877 conn(0x7fd04c0cb0d0 msgr2=0x7fd04c0c2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:56.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 shutdown_connections 2023-08-25T09:51:56.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.405+0000 7fd052a06640 1 -- 172.21.15.70:0/1530917877 wait complete. 2023-08-25T09:51:56.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:51:56.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.642+0000 7fad7664d640 1 Processor -- start 2023-08-25T09:51:56.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.642+0000 7fad7664d640 1 -- start start 2023-08-25T09:51:56.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.644+0000 7fad7664d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fad7015ae50 0x7fad700c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.644+0000 7fad7664d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700cc5f0 con 0x7fad700cdc20 2023-08-25T09:51:56.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.644+0000 7fad7664d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad70070650 con 0x7fad70069f60 2023-08-25T09:51:56.645 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.644+0000 7fad7664d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad70070930 con 0x7fad7015ae50 2023-08-25T09:51:56.647 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.644+0000 7fad6f7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fad700cdc20 0x7fad70069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44508/0 (socket says 172.21.15.70:44508) 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.644+0000 7fad6f7fe640 1 -- 172.21.15.70:0/3938991626 learned_addr learned my addr 172.21.15.70:0/3938991626 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256513879 0 0) 0x7fad700cc5f0 con 0x7fad700cdc20 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad58003520 con 0x7fad700cdc20 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2488192183 0 0) 0x7fad58003520 con 0x7fad700cdc20 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 >> v2:172.21.15.155:3300/0 conn(0x7fad7015ae50 msgr2=0x7fad700c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 --2- 172.21.15.70:0/3938991626 >> v2:172.21.15.155:3300/0 conn(0x7fad7015ae50 0x7fad700c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 >> v1:172.21.15.155:6789/0 conn(0x7fad70069f60 legacy=0x7fad700ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70070c10 con 0x7fad700cdc20 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fad64001670 con 0x7fad700cdc20 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 >> v1:172.21.15.70:6789/0 conn(0x7fad700cdc20 legacy=0x7fad70069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad58003970 0x7fad58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad580047c0 0x7fad58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580052a0 0x7fad5800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad70070930 con 0x7fad580047c0 2023-08-25T09:51:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5800c190 con 0x7fad58003970 2023-08-25T09:51:56.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.645+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5800c350 con 0x7fad580052a0 2023-08-25T09:51:56.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad580047c0 0x7fad58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.650 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6ffff640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad58003970 0x7fad58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6f7fe640 1 -- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580052a0 msgr2=0x7fad5800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:56.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580052a0 0x7fad5800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6f7fe640 1 -- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad58003970 msgr2=0x7fad58003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:56.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad58003970 0x7fad58003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad74bea640 1 -- 172.21.15.70:0/3938991626 reap_dead start 2023-08-25T09:51:56.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.646+0000 7fad6f7fe640 1 -- 172.21.15.70:0/3938991626 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58003520 con 0x7fad580047c0 2023-08-25T09:51:56.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.647+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad580047c0 0x7fad58004c40 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fad6400c240 tx=0x7fad580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f04977630d681c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:56.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.647+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64005ad0 con 0x7fad580047c0 2023-08-25T09:51:56.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.647+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad64005c30 con 0x7fad580047c0 2023-08-25T09:51:56.658 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.647+0000 7fad6effd640 1 -- 172.21.15.70:0/3938991626 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6402caf0 con 0x7fad580047c0 2023-08-25T09:51:56.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 -- 172.21.15.70:0/3938991626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad580047c0 msgr2=0x7fad58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad580047c0 0x7fad58004c40 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fad6400c240 tx=0x7fad580057f0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 -- 172.21.15.70:0/3938991626 shutdown_connections 2023-08-25T09:51:56.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 --2- 172.21.15.70:0/3938991626 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad580047c0 0x7fad58004c40 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.659 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 -- 172.21.15.70:0/3938991626 >> 172.21.15.70:0/3938991626 conn(0x7fad700cb0d0 msgr2=0x7fad700cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 -- 172.21.15.70:0/3938991626 shutdown_connections 2023-08-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.648+0000 7fad7664d640 1 -- 172.21.15.70:0/3938991626 wait complete. 2023-08-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.650+0000 7fad7664d640 1 Processor -- start 2023-08-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.650+0000 7fad7664d640 1 -- start start 2023-08-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad7664d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580047c0 0x7fad700ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad7664d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 0x7fad7015c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad7664d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad700c61b0 0x7fad700c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad7664d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c6b80 con 0x7fad7015d290 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad7664d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015c9a0 con 0x7fad700c61b0 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad7664d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7015d850 con 0x7fad580047c0 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 0x7fad7015c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad6f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 0x7fad7015c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39218/0 (socket says 172.21.15.70:39218) 2023-08-25T09:51:56.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.651+0000 7fad6f7fe640 1 -- 172.21.15.70:0/2683093287 learned_addr learned my addr 172.21.15.70:0/2683093287 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:56.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6f7fe640 1 -- 172.21.15.70:0/2683093287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580047c0 msgr2=0x7fad700ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580047c0 0x7fad700ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.662 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6f7fe640 1 -- 172.21.15.70:0/2683093287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad700c61b0 msgr2=0x7fad700c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad700c61b0 0x7fad700c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6f7fe640 1 -- 172.21.15.70:0/2683093287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7015db30 con 0x7fad7015d290 2023-08-25T09:51:56.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6f7fe640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 0x7fad7015c460 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fad64001930 tx=0x7fad6402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:56.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6403c040 con 0x7fad7015d290 2023-08-25T09:51:56.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6402cd80 con 0x7fad7015d290 2023-08-25T09:51:56.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64007290 con 0x7fad7015d290 2023-08-25T09:51:56.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.652+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7015de10 con 0x7fad7015d290 2023-08-25T09:51:56.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.653+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad701ee270 con 0x7fad7015d290 2023-08-25T09:51:56.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.654+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad701ee400 con 0x7fad7015d290 2023-08-25T09:51:56.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.655+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fad64032070 con 0x7fad7015d290 2023-08-25T09:51:56.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.656+0000 7fad6cff9640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad5c069fe0 0x7fad5c06c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:56.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.656+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fad640a98d0 con 0x7fad7015d290 2023-08-25T09:51:56.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.656+0000 7fad6ffff640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad5c069fe0 0x7fad5c06c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:56.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.656+0000 7fad6ffff640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad5c069fe0 0x7fad5c06c4a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fad60006dd0 tx=0x7fad60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:56.666 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.660+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fad64075360 con 0x7fad7015d290 2023-08-25T09:51:56.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.847+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad70066fb0 con 0x7fad7015d290 2023-08-25T09:51:56.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.848+0000 7fad6cff9640 1 -- 172.21.15.70:0/2683093287 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v190) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad64078980 con 0x7fad7015d290 2023-08-25T09:51:56.850 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:51:56.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad5c069fe0 msgr2=0x7fad5c06c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad5c069fe0 0x7fad5c06c4a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fad60006dd0 tx=0x7fad60008040 comp rx=0 tx=0).stop 2023-08-25T09:51:56.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 msgr2=0x7fad7015c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:56.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 0x7fad7015c460 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fad64001930 tx=0x7fad6402e070 comp rx=0 tx=0).stop 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 shutdown_connections 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fad5c069fe0 0x7fad5c06c4a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fad700c61b0 0x7fad700c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fad7015d290 0x7fad7015c460 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 --2- 172.21.15.70:0/2683093287 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fad580047c0 0x7fad700ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.850+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 >> 172.21.15.70:0/2683093287 conn(0x7fad700cb0d0 msgr2=0x7fad70069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:56.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.851+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 shutdown_connections 2023-08-25T09:51:56.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:56.851+0000 7fad7664d640 1 -- 172.21.15.70:0/2683093287 wait complete. 2023-08-25T09:51:56.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-08-25T09:51:57.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.075+0000 7f05987e3640 1 Processor -- start 2023-08-25T09:51:57.077 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.076+0000 7f05987e3640 1 -- start start 2023-08-25T09:51:57.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.077+0000 7f05987e3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f059006d3d0 0x7f059006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.077+0000 7f05987e3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f059006def0 0x7f0590166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.077+0000 7f05987e3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059006e320 con 0x7f059015cc00 2023-08-25T09:51:57.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.077+0000 7f05987e3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059015cfe0 con 0x7f059006def0 2023-08-25T09:51:57.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.077+0000 7f05987e3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05900cfdb0 con 0x7f059006d3d0 2023-08-25T09:51:57.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f059006def0 0x7f0590166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f059006def0 0x7f0590166760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39232/0 (socket says 172.21.15.70:39232) 2023-08-25T09:51:57.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 -- 172.21.15.70:0/3260745591 learned_addr learned my addr 172.21.15.70:0/3260745591 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:57.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0595d7e640 1 --2- 172.21.15.70:0/3260745591 >> v2:172.21.15.155:3300/0 conn(0x7f059006d3d0 0x7f059006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 -- 172.21.15.70:0/3260745591 >> v2:172.21.15.155:3300/0 conn(0x7f059006d3d0 msgr2=0x7f059006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:57.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 --2- 172.21.15.70:0/3260745591 >> v2:172.21.15.155:3300/0 conn(0x7f059006d3d0 0x7f059006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 -- 172.21.15.70:0/3260745591 >> v1:172.21.15.155:6789/0 conn(0x7f059015cc00 legacy=0x7f05900cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:57.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.078+0000 7f0596d80640 1 -- 172.21.15.70:0/3260745591 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590069850 con 0x7f059006def0 2023-08-25T09:51:57.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.079+0000 7f0596d80640 1 --2- 172.21.15.70:0/3260745591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059006def0 0x7f0590166760 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f05900ce3c0 tx=0x7f058802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0ea6546f5d28a9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:57.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.079+0000 7f059557d640 1 -- 172.21.15.70:0/3260745591 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058802ca80 con 0x7f059006def0 2023-08-25T09:51:57.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.079+0000 7f059557d640 1 -- 172.21.15.70:0/3260745591 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058802cbe0 con 0x7f059006def0 2023-08-25T09:51:57.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.079+0000 7f059557d640 1 -- 172.21.15.70:0/3260745591 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05880343f0 con 0x7f059006def0 2023-08-25T09:51:57.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 -- 172.21.15.70:0/3260745591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059006def0 msgr2=0x7f0590166760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 --2- 172.21.15.70:0/3260745591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059006def0 0x7f0590166760 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f05900ce3c0 tx=0x7f058802bee0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 -- 172.21.15.70:0/3260745591 shutdown_connections 2023-08-25T09:51:57.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 --2- 172.21.15.70:0/3260745591 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059006def0 0x7f0590166760 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 --2- 172.21.15.70:0/3260745591 >> v2:172.21.15.155:3300/0 conn(0x7f059006d3d0 0x7f059006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 -- 172.21.15.70:0/3260745591 >> 172.21.15.70:0/3260745591 conn(0x7f05900d0860 msgr2=0x7f05900d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 -- 172.21.15.70:0/3260745591 shutdown_connections 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.080+0000 7f05987e3640 1 -- 172.21.15.70:0/3260745591 wait complete. 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.082+0000 7f05987e3640 1 Processor -- start 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.082+0000 7f05987e3640 1 -- start start 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.083+0000 7f05987e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059006d3d0 0x7f05900aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.083+0000 7f05987e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f059006def0 0x7f0590155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.083+0000 7f05987e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 0x7f05900cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.083+0000 7f05987e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05900c9980 con 0x7f059015cc00 2023-08-25T09:51:57.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.083+0000 7f05987e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05900c9c60 con 0x7f059006d3d0 2023-08-25T09:51:57.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.083+0000 7f05987e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05900ae460 con 0x7f059006def0 2023-08-25T09:51:57.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 0x7f05900cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 0x7f05900cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39234/0 (socket says 172.21.15.70:39234) 2023-08-25T09:51:57.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 -- 172.21.15.70:0/1704895611 learned_addr learned my addr 172.21.15.70:0/1704895611 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:57.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f059657f640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059006d3d0 0x7f05900aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0595d7e640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f059006def0 0x7f0590155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 -- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f059006def0 msgr2=0x7f0590155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f059006def0 0x7f0590155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 -- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059006d3d0 msgr2=0x7f05900aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059006d3d0 0x7f05900aec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.084+0000 7f0596d80640 1 -- 172.21.15.70:0/1704895611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0588009b30 con 0x7f059015cc00 2023-08-25T09:51:57.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.085+0000 7f0596d80640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 0x7f05900cae70 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f0588036040 tx=0x7f0588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:57.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.085+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588009510 con 0x7f059015cc00 2023-08-25T09:51:57.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.085+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058803e450 con 0x7f059015cc00 2023-08-25T09:51:57.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.085+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05900ae740 con 0x7f059015cc00 2023-08-25T09:51:57.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.085+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058803e7e0 con 0x7f059015cc00 2023-08-25T09:51:57.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.085+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0590155d90 con 0x7f059015cc00 2023-08-25T09:51:57.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.086+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05900ae930 con 0x7f059015cc00 2023-08-25T09:51:57.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.087+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f058802cd50 con 0x7f059015cc00 2023-08-25T09:51:57.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.089+0000 7f058effd640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f057c06a050 0x7f057c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.089+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f05880aaeb0 con 0x7f059015cc00 2023-08-25T09:51:57.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.089+0000 7f059657f640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f057c06a050 0x7f057c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.090+0000 7f059657f640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f057c06a050 0x7f057c06c510 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f058000b7f0 tx=0x7f0580008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:57.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.094+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f05880768b0 con 0x7f059015cc00 2023-08-25T09:51:57.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.347+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f059004e420 con 0x7f059015cc00 2023-08-25T09:51:57.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.532+0000 7f058effd640 1 -- 172.21.15.70:0/1704895611 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v642) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0588043020 con 0x7f059015cc00 2023-08-25T09:51:57.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f057c06a050 msgr2=0x7f057c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f057c06a050 0x7f057c06c510 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f058000b7f0 tx=0x7f0580008040 comp rx=0 tx=0).stop 2023-08-25T09:51:57.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 msgr2=0x7f05900cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 0x7f05900cae70 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f0588036040 tx=0x7f0588004a40 comp rx=0 tx=0).stop 2023-08-25T09:51:57.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 shutdown_connections 2023-08-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f057c06a050 0x7f057c06c510 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f059015cc00 0x7f05900cae70 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f059006def0 0x7f0590155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 --2- 172.21.15.70:0/1704895611 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f059006d3d0 0x7f05900aec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.535+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 >> 172.21.15.70:0/1704895611 conn(0x7f05900d0860 msgr2=0x7f05900d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.536+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 shutdown_connections 2023-08-25T09:51:57.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.536+0000 7f05987e3640 1 -- 172.21.15.70:0/1704895611 wait complete. 2023-08-25T09:51:57.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T09:51:57.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.766+0000 7f2ca3af3640 1 Processor -- start 2023-08-25T09:51:57.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.767+0000 7f2ca3af3640 1 -- start start 2023-08-25T09:51:57.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.768+0000 7f2ca3af3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c9c159bf0 0x7f2c9c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.768+0000 7f2ca3af3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c0cd660 0x7f2c9c0cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.768+0000 7f2ca3af3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c9c15e780 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.768+0000 7f2ca3af3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c9c0ce030 con 0x7f2c9c159bf0 2023-08-25T09:51:57.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.768+0000 7f2ca3af3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c06f8e0 con 0x7f2c9c0cd660 2023-08-25T09:51:57.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c9c159bf0 0x7f2c9c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2c9c159bf0 0x7f2c9c15e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39238/0 (socket says 172.21.15.70:39238) 2023-08-25T09:51:57.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 -- 172.21.15.70:0/3952691006 learned_addr learned my addr 172.21.15.70:0/3952691006 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:57.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 -- 172.21.15.70:0/3952691006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c0cd660 msgr2=0x7f2c9c0cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:57.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca2090640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c0cd660 0x7f2c9c0cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c0cd660 0x7f2c9c0cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 -- 172.21.15.70:0/3952691006 >> v1:172.21.15.155:6789/0 conn(0x7f2c9c06f0e0 legacy=0x7f2c9c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:57.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca188f640 1 -- 172.21.15.70:0/3952691006 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c06fbc0 con 0x7f2c9c159bf0 2023-08-25T09:51:57.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.769+0000 7f2ca2090640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c0cd660 0x7f2c9c0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:57.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.770+0000 7f2ca188f640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c159bf0 0x7f2c9c15e240 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009870 tx=0x7f2c8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a0b6f9401902fb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:57.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca088d640 1 -- 172.21.15.70:0/3952691006 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c031640 con 0x7f2c9c159bf0 2023-08-25T09:51:57.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca088d640 1 -- 172.21.15.70:0/3952691006 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c0317a0 con 0x7f2c9c159bf0 2023-08-25T09:51:57.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca088d640 1 -- 172.21.15.70:0/3952691006 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c031b30 con 0x7f2c9c159bf0 2023-08-25T09:51:57.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3952691006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c159bf0 msgr2=0x7f2c9c15e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c159bf0 0x7f2c9c15e240 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009870 tx=0x7f2c8c02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3952691006 shutdown_connections 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c0cd660 0x7f2c9c0cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3952691006 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c159bf0 0x7f2c9c15e240 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3952691006 >> 172.21.15.70:0/3952691006 conn(0x7f2c9c0cb0d0 msgr2=0x7f2c9c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3952691006 shutdown_connections 2023-08-25T09:51:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.771+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3952691006 wait complete. 2023-08-25T09:51:57.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.773+0000 7f2ca3af3640 1 Processor -- start 2023-08-25T09:51:57.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.773+0000 7f2ca3af3640 1 -- start start 2023-08-25T09:51:57.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca3af3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 0x7f2c9c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca3af3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9c0cd660 0x7f2c9c0c3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca3af3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c159bf0 0x7f2c9c14f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca3af3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c1f9fb0 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca3af3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c1fae80 con 0x7f2c9c0cd660 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca3af3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9c1fa200 con 0x7f2c9c159bf0 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.774+0000 7f2ca188f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 0x7f2c9c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 0x7f2c9c0c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39240/0 (socket says 172.21.15.70:39240) 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 -- 172.21.15.70:0/3992653247 learned_addr learned my addr 172.21.15.70:0/3992653247 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:57.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 -- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c159bf0 msgr2=0x7f2c9c14f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c159bf0 0x7f2c9c14f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 -- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9c0cd660 msgr2=0x7f2c9c0c3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca108e640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9c0cd660 0x7f2c9c0c3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9c0cd660 0x7f2c9c0c3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 -- 172.21.15.70:0/3992653247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0094d0 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.775+0000 7f2ca188f640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 0x7f2c9c0c9c30 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c000c00 tx=0x7f2c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.776+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c03ac60 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.776+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c03ade0 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.776+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c044630 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.776+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9c1fa4e0 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.776+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9c1faaa0 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.777+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c9c0c5690 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.778+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2c8c02f050 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.779+0000 7f2c9a7fc640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c8806a050 0x7f2c8806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:57.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.779+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2c8c0aa3c0 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.779+0000 7f2ca108e640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c8806a050 0x7f2c8806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:57.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.780+0000 7f2ca108e640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c8806a050 0x7f2c8806c510 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0ae180 tx=0x7f2c90009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:57.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.783+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c9c0c5690 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.976+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2c9c0c5690 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.984+0000 7f2c9a7fc640 1 -- 172.21.15.70:0/3992653247 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c9c0c5690 con 0x7f2c9c06f0e0 2023-08-25T09:51:57.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c8806a050 msgr2=0x7f2c8806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c8806a050 0x7f2c8806c510 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0ae180 tx=0x7f2c90009040 comp rx=0 tx=0).stop 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 msgr2=0x7f2c9c0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 0x7f2c9c0c9c30 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c000c00 tx=0x7f2c8c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 shutdown_connections 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c8806a050 0x7f2c8806c510 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c9c159bf0 0x7f2c9c14f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c9c0cd660 0x7f2c9c0c3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 --2- 172.21.15.70:0/3992653247 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c9c06f0e0 0x7f2c9c0c9c30 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:57.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 >> 172.21.15.70:0/3992653247 conn(0x7f2c9c0cb0d0 msgr2=0x7f2c9c1589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:57.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 shutdown_connections 2023-08-25T09:51:57.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:57.987+0000 7f2ca3af3640 1 -- 172.21.15.70:0/3992653247 wait complete. 2023-08-25T09:51:58.007 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:51:58.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:51:58.130 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:51:58.250 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:51:58.250 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:51:58.251 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:58.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:51:58.310 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:51:58.311 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:58.372 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:51:58.373 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:51:58.374 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:58.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:51:58.435 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:51:58.435 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:51:58.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:51:58.495 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T09:51:58.495 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:51:58.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.718+0000 7f76757e3640 1 Processor -- start 2023-08-25T09:51:58.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.718+0000 7f76757e3640 1 -- start start 2023-08-25T09:51:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f76757e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015e830 0x7f767006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f76757e3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f767006b2b0 0x7f767015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:58.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f76757e3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76700b1b60 con 0x7f767015c6d0 2023-08-25T09:51:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f76757e3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006bd80 con 0x7f767006b2b0 2023-08-25T09:51:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f76757e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767006b7c0 con 0x7f767015e830 2023-08-25T09:51:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f766e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f767006b2b0 0x7f767015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f766e7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f767006b2b0 0x7f767015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39248/0 (socket says 172.21.15.70:39248) 2023-08-25T09:51:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f766e7fc640 1 -- 172.21.15.70:0/4151035952 learned_addr learned my addr 172.21.15.70:0/4151035952 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:58.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f766f7fe640 1 --1- 172.21.15.70:0/4151035952 >> v1:172.21.15.70:6789/0 conn(0x7f767015c6d0 0x7f76700b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:44522/0 (socket says 172.21.15.70:44522) 2023-08-25T09:51:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.720+0000 7f766effd640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015e830 0x7f767006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.721+0000 7f766e7fc640 1 -- 172.21.15.70:0/4151035952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015e830 msgr2=0x7f767006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.721+0000 7f766e7fc640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015e830 0x7f767006ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.721+0000 7f766e7fc640 1 -- 172.21.15.70:0/4151035952 >> v1:172.21.15.70:6789/0 conn(0x7f767015c6d0 legacy=0x7f76700b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.721+0000 7f766e7fc640 1 -- 172.21.15.70:0/4151035952 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767006baa0 con 0x7f767006b2b0 2023-08-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.721+0000 7f766dffb640 1 -- 172.21.15.70:0/4151035952 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1788955932 0 0) 0x7f76700b1b60 con 0x7f767015c6d0 2023-08-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.721+0000 7f766effd640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015e830 0x7f767006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.722+0000 7f766e7fc640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767006b2b0 0x7f767015c190 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7658009870 tx=0x7f765802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e5f0ca1cb204e3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:58.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.722+0000 7f766dffb640 1 -- 172.21.15.70:0/4151035952 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658030840 con 0x7f767006b2b0 2023-08-25T09:51:58.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.722+0000 7f766dffb640 1 -- 172.21.15.70:0/4151035952 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76580309a0 con 0x7f767006b2b0 2023-08-25T09:51:58.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.722+0000 7f766dffb640 1 -- 172.21.15.70:0/4151035952 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658030d30 con 0x7f767006b2b0 2023-08-25T09:51:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 -- 172.21.15.70:0/4151035952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767006b2b0 msgr2=0x7f767015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767006b2b0 0x7f767015c190 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7658009870 tx=0x7f765802b980 comp rx=0 tx=0).stop 2023-08-25T09:51:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 -- 172.21.15.70:0/4151035952 shutdown_connections 2023-08-25T09:51:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767006b2b0 0x7f767015c190 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 --2- 172.21.15.70:0/4151035952 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015e830 0x7f767006ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 -- 172.21.15.70:0/4151035952 >> 172.21.15.70:0/4151035952 conn(0x7f7670069850 msgr2=0x7f7670069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 -- 172.21.15.70:0/4151035952 shutdown_connections 2023-08-25T09:51:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.723+0000 7f76757e3640 1 -- 172.21.15.70:0/4151035952 wait complete. 2023-08-25T09:51:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.725+0000 7f76757e3640 1 Processor -- start 2023-08-25T09:51:58.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.725+0000 7f76757e3640 1 -- start start 2023-08-25T09:51:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f76757e3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f767006b2b0 0x7f76701450f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f76757e3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015c6d0 0x7f7670158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f76757e3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 0x7f76701592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f76757e3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701f09c0 con 0x7f767015e830 2023-08-25T09:51:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f76757e3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7670145630 con 0x7f767006b2b0 2023-08-25T09:51:58.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f76757e3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7670153b40 con 0x7f767015c6d0 2023-08-25T09:51:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 0x7f76701592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 0x7f76701592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39264/0 (socket says 172.21.15.70:39264) 2023-08-25T09:51:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 -- 172.21.15.70:0/1475988094 learned_addr learned my addr 172.21.15.70:0/1475988094 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766effd640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f767006b2b0 0x7f76701450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:58.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 -- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015c6d0 msgr2=0x7f7670158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:58.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015c6d0 0x7f7670158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 -- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f767006b2b0 msgr2=0x7f76701450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:58.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f767006b2b0 0x7f76701450f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.726+0000 7f766f7fe640 1 -- 172.21.15.70:0/1475988094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76580094d0 con 0x7f767015e830 2023-08-25T09:51:58.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.727+0000 7f766f7fe640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 0x7f76701592b0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f766800c450 tx=0x7f766800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:58.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.727+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680074e0 con 0x7f767015e830 2023-08-25T09:51:58.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.727+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7668007660 con 0x7f767015e830 2023-08-25T09:51:58.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.727+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7668004a20 con 0x7f767015e830 2023-08-25T09:51:58.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.727+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7670153e20 con 0x7f767015e830 2023-08-25T09:51:58.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.727+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76701f05b0 con 0x7f767015e830 2023-08-25T09:51:58.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.728+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76701f0b00 con 0x7f767015e830 2023-08-25T09:51:58.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.729+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7668004b80 con 0x7f767015e830 2023-08-25T09:51:58.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.730+0000 7f76677fe640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f765006a080 0x7f765006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:58.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.730+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7668087450 con 0x7f767015e830 2023-08-25T09:51:58.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.731+0000 7f766effd640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f765006a080 0x7f765006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:58.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.731+0000 7f766effd640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f765006a080 0x7f765006c540 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ed0 tx=0x7f766000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:58.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.735+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f76701f0b00 con 0x7f767015e830 2023-08-25T09:51:58.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.965+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76701f0b00 con 0x7f767015e830 2023-08-25T09:51:58.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.965+0000 7f76677fe640 1 -- 172.21.15.70:0/1475988094 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f7668018e90 con 0x7f767015e830 2023-08-25T09:51:58.967 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:51:58.967 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5726,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/4254012465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":4254012465},{"type":"v1","addr":"172.21.15.155:6834","nonce":4254012465}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6265,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1353493507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1353493507},{"type":"v1","addr":"172.21.15.155:6835","nonce":1353493507}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9150,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6835/1886984477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1886984477},{"type":"v1","addr":"172.21.15.70:6835","nonce":1886984477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9153,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/330061328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6836","nonce":330061328},{"type":"v1","addr":"172.21.15.70:6837","nonce":330061328}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[]} 2023-08-25T09:51:58.968 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 84 2023-08-25T09:51:58.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.968+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f765006a080 msgr2=0x7f765006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:58.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.968+0000 7f76757e3640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f765006a080 0x7f765006c540 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f7660009ed0 tx=0x7f766000b040 comp rx=0 tx=0).stop 2023-08-25T09:51:58.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.968+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 msgr2=0x7f76701592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:58.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.968+0000 7f76757e3640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 0x7f76701592b0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f766800c450 tx=0x7f766800c910 comp rx=0 tx=0).stop 2023-08-25T09:51:58.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 shutdown_connections 2023-08-25T09:51:58.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f765006a080 0x7f765006c540 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f767015e830 0x7f76701592b0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f767015c6d0 0x7f7670158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 --2- 172.21.15.70:0/1475988094 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f767006b2b0 0x7f76701450f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:58.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 >> 172.21.15.70:0/1475988094 conn(0x7f7670069850 msgr2=0x7f76700b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:58.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 shutdown_connections 2023-08-25T09:51:58.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:58.969+0000 7f76757e3640 1 -- 172.21.15.70:0/1475988094 wait complete. 2023-08-25T09:51:58.989 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T09:51:58.989 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T09:51:58.989 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:51:58.989 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:51:58.990 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T09:51:58.990 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T09:51:58.990 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:51:58.990 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:51:58.991 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T09:51:58.991 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T09:51:58.991 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:51:58.991 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:51:58.991 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T09:51:58.992 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T09:51:58.992 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:51:58.992 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:51:59.006 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:51:59.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T09:51:59.014 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:51:59.015 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T09:51:59.017 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:51:59.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T09:51:59.020 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:51:59.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T09:51:59.065 INFO:tasks.ceph.mds.b:Started 2023-08-25T09:51:59.066 DEBUG:teuthology.parallel:result is None 2023-08-25T09:51:59.068 INFO:tasks.ceph.mds.a:Started 2023-08-25T09:51:59.068 DEBUG:teuthology.parallel:result is None 2023-08-25T09:51:59.073 INFO:tasks.ceph.mds.d:Started 2023-08-25T09:51:59.074 DEBUG:teuthology.parallel:result is None 2023-08-25T09:51:59.076 INFO:tasks.ceph.mds.c:Started 2023-08-25T09:51:59.076 DEBUG:teuthology.parallel:result is None 2023-08-25T09:51:59.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T09:51:59.230 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:51:59.231 INFO:tasks.ceph.mds.c.smithi070.stdout:starting mds.c at 2023-08-25T09:51:59.241 INFO:tasks.ceph.mds.d.smithi155.stdout:starting mds.d at 2023-08-25T09:51:59.241 INFO:tasks.ceph.mds.b.smithi155.stdout:starting mds.b at 2023-08-25T09:51:59.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.265+0000 7fe7a5214640 1 Processor -- start 2023-08-25T09:51:59.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.265+0000 7fe7a5214640 1 -- start start 2023-08-25T09:51:59.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.266+0000 7fe7a5214640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.266+0000 7fe7a5214640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe7a0160ec0 0x7fe7a00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.266+0000 7fe7a5214640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00d0c00 con 0x7fe7a015cfc0 2023-08-25T09:51:59.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.266+0000 7fe7a5214640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00b5cb0 con 0x7fe7a0160ec0 2023-08-25T09:51:59.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.266+0000 7fe7a5214640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01613d0 con 0x7fe7a015cbe0 2023-08-25T09:51:59.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe7a015cfc0 0x7fe7a00b4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35192/0 (socket says 172.21.15.70:35192) 2023-08-25T09:51:59.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79f59e640 1 -- 172.21.15.70:0/3874264221 learned_addr learned my addr 172.21.15.70:0/3874264221 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:59.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79e59c640 1 --2- 172.21.15.70:0/3874264221 >> v2:172.21.15.155:3300/0 conn(0x7fe7a0160ec0 0x7fe7a00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79ed9d640 1 --2- 172.21.15.70:0/3874264221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79dd9b640 1 -- 172.21.15.70:0/3874264221 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270849355 0 0) 0x7fe7a00d0c00 con 0x7fe7a015cfc0 2023-08-25T09:51:59.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79dd9b640 1 -- 172.21.15.70:0/3874264221 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe788003520 con 0x7fe7a015cfc0 2023-08-25T09:51:59.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79ed9d640 1 -- 172.21.15.70:0/3874264221 >> v2:172.21.15.155:3300/0 conn(0x7fe7a0160ec0 msgr2=0x7fe7a00cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:59.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79ed9d640 1 --2- 172.21.15.70:0/3874264221 >> v2:172.21.15.155:3300/0 conn(0x7fe7a0160ec0 0x7fe7a00cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79ed9d640 1 -- 172.21.15.70:0/3874264221 >> v1:172.21.15.70:6789/0 conn(0x7fe7a015cfc0 legacy=0x7fe7a00b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.267+0000 7fe79ed9d640 1 -- 172.21.15.70:0/3874264221 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a01616b0 con 0x7fe7a015cbe0 2023-08-25T09:51:59.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.268+0000 7fe79ed9d640 1 --2- 172.21.15.70:0/3874264221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a0160980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7900097d0 tx=0x7fe79002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0506ed1801f2af server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:59.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe79dd9b640 1 -- 172.21.15.70:0/3874264221 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790031800 con 0x7fe7a015cbe0 2023-08-25T09:51:59.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe79dd9b640 1 -- 172.21.15.70:0/3874264221 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe790031960 con 0x7fe7a015cbe0 2023-08-25T09:51:59.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe79dd9b640 1 -- 172.21.15.70:0/3874264221 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790031cf0 con 0x7fe7a015cbe0 2023-08-25T09:51:59.272 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 -- 172.21.15.70:0/3874264221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 msgr2=0x7fe7a0160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 --2- 172.21.15.70:0/3874264221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a0160980 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe7900097d0 tx=0x7fe79002b8a0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 -- 172.21.15.70:0/3874264221 shutdown_connections 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 --2- 172.21.15.70:0/3874264221 >> v2:172.21.15.155:3300/0 conn(0x7fe7a0160ec0 0x7fe7a00cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 --2- 172.21.15.70:0/3874264221 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a0160980 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 -- 172.21.15.70:0/3874264221 >> 172.21.15.70:0/3874264221 conn(0x7fe7a0069850 msgr2=0x7fe7a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 -- 172.21.15.70:0/3874264221 shutdown_connections 2023-08-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.269+0000 7fe7a5214640 1 -- 172.21.15.70:0/3874264221 wait complete. 2023-08-25T09:51:59.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.271+0000 7fe7a5214640 1 Processor -- start 2023-08-25T09:51:59.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.271+0000 7fe7a5214640 1 -- start start 2023-08-25T09:51:59.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe7a5214640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a00cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe7a5214640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a015cfc0 0x7fe7a015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe7a5214640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 0x7fe7a015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe7a5214640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01f37b0 con 0x7fe7a0160ec0 2023-08-25T09:51:59.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe7a5214640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01f2ba0 con 0x7fe7a015cfc0 2023-08-25T09:51:59.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe7a5214640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01f2e80 con 0x7fe7a015cbe0 2023-08-25T09:51:59.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.272+0000 7fe79f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 0x7fe7a015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 0x7fe7a015fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35804/0 (socket says 172.21.15.70:35804) 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 -- 172.21.15.70:0/2223657651 learned_addr learned my addr 172.21.15.70:0/2223657651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79ed9d640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a00cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 -- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 msgr2=0x7fe7a00cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a00cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 -- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a015cfc0 msgr2=0x7fe7a015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a015cfc0 0x7fe7a015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 -- 172.21.15.70:0/2223657651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790009400 con 0x7fe7a0160ec0 2023-08-25T09:51:59.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe79f59e640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 0x7fe7a015fa70 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe79800ca70 tx=0x7fe79800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:59.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79800ab70 con 0x7fe7a0160ec0 2023-08-25T09:51:59.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.273+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79800acd0 con 0x7fe7a0160ec0 2023-08-25T09:51:59.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.274+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a01f3160 con 0x7fe7a0160ec0 2023-08-25T09:51:59.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.274+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798012620 con 0x7fe7a0160ec0 2023-08-25T09:51:59.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.274+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a01f2800 con 0x7fe7a0160ec0 2023-08-25T09:51:59.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.275+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a004e420 con 0x7fe7a0160ec0 2023-08-25T09:51:59.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.276+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe79801b3b0 con 0x7fe7a0160ec0 2023-08-25T09:51:59.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.277+0000 7fe7877fe640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe78c06a050 0x7fe78c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.277+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe798088150 con 0x7fe7a0160ec0 2023-08-25T09:51:59.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.277+0000 7fe79ed9d640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe78c06a050 0x7fe78c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.278+0000 7fe79ed9d640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe78c06a050 0x7fe78c06c510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe790036000 tx=0x7fe79003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:59.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.281+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe798053b70 con 0x7fe7a0160ec0 2023-08-25T09:51:59.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.482+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe7a00b5880 con 0x7fe7a0160ec0 2023-08-25T09:51:59.567 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.565+0000 7fe7877fe640 1 -- 172.21.15.70:0/2223657651 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v192) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe798057190 con 0x7fe7a0160ec0 2023-08-25T09:51:59.567 INFO:teuthology.orchestra.run.smithi070.stderr: removed all blocklist entries 2023-08-25T09:51:59.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.569+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe78c06a050 msgr2=0x7fe78c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.569+0000 7fe7a5214640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe78c06a050 0x7fe78c06c510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe790036000 tx=0x7fe79003c040 comp rx=0 tx=0).stop 2023-08-25T09:51:59.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.569+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 msgr2=0x7fe7a015fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.572 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.569+0000 7fe7a5214640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 0x7fe7a015fa70 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe79800ca70 tx=0x7fe79800cf30 comp rx=0 tx=0).stop 2023-08-25T09:51:59.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 shutdown_connections 2023-08-25T09:51:59.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe78c06a050 0x7fe78c06c510 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.573 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe7a0160ec0 0x7fe7a015fa70 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe7a015cfc0 0x7fe7a015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 --2- 172.21.15.70:0/2223657651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe7a015cbe0 0x7fe7a00cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 >> 172.21.15.70:0/2223657651 conn(0x7fe7a0069850 msgr2=0x7fe7a00cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 shutdown_connections 2023-08-25T09:51:59.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.570+0000 7fe7a5214640 1 -- 172.21.15.70:0/2223657651 wait complete. 2023-08-25T09:51:59.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T09:51:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.788+0000 7f6729583640 1 Processor -- start 2023-08-25T09:51:59.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.788+0000 7f6729583640 1 -- start start 2023-08-25T09:51:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6729583640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f67240cea30 0x7f67240b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6729583640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f67240ac870 0x7f67240b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6729583640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672406f050 con 0x7f672415cb40 2023-08-25T09:51:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6729583640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672406f330 con 0x7f67240cea30 2023-08-25T09:51:59.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6729583640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f672406f610 con 0x7f67240ac870 2023-08-25T09:51:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6722ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f672415cb40 0x7f67240ce320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35198/0 (socket says 172.21.15.70:35198) 2023-08-25T09:51:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f6722ffd640 1 -- 172.21.15.70:0/1240656019 learned_addr learned my addr 172.21.15.70:0/1240656019 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:51:59.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f671bfff640 1 --2- 172.21.15.70:0/1240656019 >> v2:172.21.15.70:3300/0 conn(0x7f67240cea30 0x7f67240b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f67227fc640 1 -- 172.21.15.70:0/1240656019 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004989334 0 0) 0x7f672406f050 con 0x7f672415cb40 2023-08-25T09:51:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f67227fc640 1 -- 172.21.15.70:0/1240656019 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6710003520 con 0x7f672415cb40 2023-08-25T09:51:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f67237fe640 1 --2- 172.21.15.70:0/1240656019 >> v2:172.21.15.155:3300/0 conn(0x7f67240ac870 0x7f67240b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f671bfff640 1 -- 172.21.15.70:0/1240656019 >> v2:172.21.15.155:3300/0 conn(0x7f67240ac870 msgr2=0x7f67240b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:51:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f671bfff640 1 --2- 172.21.15.70:0/1240656019 >> v2:172.21.15.155:3300/0 conn(0x7f67240ac870 0x7f67240b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f671bfff640 1 -- 172.21.15.70:0/1240656019 >> v1:172.21.15.70:6789/0 conn(0x7f672415cb40 legacy=0x7f67240ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f671bfff640 1 -- 172.21.15.70:0/1240656019 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672406f8f0 con 0x7f67240cea30 2023-08-25T09:51:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.790+0000 7f67237fe640 1 --2- 172.21.15.70:0/1240656019 >> v2:172.21.15.155:3300/0 conn(0x7f67240ac870 0x7f67240b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:51:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.792+0000 7f671bfff640 1 --2- 172.21.15.70:0/1240656019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67240cea30 0x7f67240b1b30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f67240adbd0 tx=0x7f670c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd25b070f631a2df server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:59.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.792+0000 7f67227fc640 1 -- 172.21.15.70:0/1240656019 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c031830 con 0x7f67240cea30 2023-08-25T09:51:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.792+0000 7f67227fc640 1 -- 172.21.15.70:0/1240656019 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670c031990 con 0x7f67240cea30 2023-08-25T09:51:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.792+0000 7f67227fc640 1 -- 172.21.15.70:0/1240656019 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c031d20 con 0x7f67240cea30 2023-08-25T09:51:59.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.792+0000 7f6729583640 1 -- 172.21.15.70:0/1240656019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67240cea30 msgr2=0x7f67240b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.792+0000 7f6729583640 1 --2- 172.21.15.70:0/1240656019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67240cea30 0x7f67240b1b30 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f67240adbd0 tx=0x7f670c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.793+0000 7f6729583640 1 -- 172.21.15.70:0/1240656019 shutdown_connections 2023-08-25T09:51:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.793+0000 7f6729583640 1 --2- 172.21.15.70:0/1240656019 >> v2:172.21.15.155:3300/0 conn(0x7f67240ac870 0x7f67240b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.793+0000 7f6729583640 1 --2- 172.21.15.70:0/1240656019 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f67240cea30 0x7f67240b1b30 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.793+0000 7f6729583640 1 -- 172.21.15.70:0/1240656019 >> 172.21.15.70:0/1240656019 conn(0x7f6724069850 msgr2=0x7f6724069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:51:59.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.793+0000 7f6729583640 1 -- 172.21.15.70:0/1240656019 shutdown_connections 2023-08-25T09:51:59.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.793+0000 7f6729583640 1 -- 172.21.15.70:0/1240656019 wait complete. 2023-08-25T09:51:59.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.794+0000 7f6729583640 1 Processor -- start 2023-08-25T09:51:59.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.795+0000 7f6729583640 1 -- start start 2023-08-25T09:51:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.795+0000 7f6729583640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67240ac870 0x7f67241f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f6729583640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67240cea30 0x7f6724150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f6729583640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 0x7f672414ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f6729583640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240aebb0 con 0x7f672415cb40 2023-08-25T09:51:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f6729583640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6724150f10 con 0x7f67240ac870 2023-08-25T09:51:59.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f6729583640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672414b360 con 0x7f67240cea30 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 0x7f672414ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 0x7f672414ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35820/0 (socket says 172.21.15.70:35820) 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 -- 172.21.15.70:0/459769686 learned_addr learned my addr 172.21.15.70:0/459769686 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 -- 172.21.15.70:0/459769686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67240cea30 msgr2=0x7f6724150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67240cea30 0x7f6724150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 -- 172.21.15.70:0/459769686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67240ac870 msgr2=0x7f67241f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:51:59.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67240ac870 0x7f67241f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 -- 172.21.15.70:0/459769686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c009400 con 0x7f672415cb40 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.796+0000 7f67237fe640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 0x7f672414ae20 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f671c00cef0 tx=0x7f671c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.797+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00d840 con 0x7f672415cb40 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.797+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c00d9c0 con 0x7f672415cb40 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.797+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c00dd70 con 0x7f672415cb40 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.797+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67241f5c90 con 0x7f672415cb40 2023-08-25T09:51:59.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.797+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67241f6250 con 0x7f672415cb40 2023-08-25T09:51:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.798+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672404e420 con 0x7f672415cb40 2023-08-25T09:51:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.799+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f671c01e3b0 con 0x7f672415cb40 2023-08-25T09:51:59.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.800+0000 7f671b7fe640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f670406a050 0x7f670406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:51:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.800+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f671c088de0 con 0x7f672415cb40 2023-08-25T09:51:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.800+0000 7f6722ffd640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f670406a050 0x7f670406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:51:59.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.800+0000 7f6722ffd640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f670406a050 0x7f670406c510 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f672406c090 tx=0x7f6714007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:51:59.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:51:59.804+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f671c054840 con 0x7f672415cb40 2023-08-25T09:52:00.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.054+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6724151050 con 0x7f672415cb40 2023-08-25T09:52:00.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.055+0000 7f671b7fe640 1 -- 172.21.15.70:0/459769686 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f671c01a090 con 0x7f672415cb40 2023-08-25T09:52:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:00.057 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_dump": [ 2023-08-25T09:52:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.a", 2023-08-25T09:52:00.057 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhk+HFCNhAAV8tcHlcp2HpAVO3Xlt8dpQ==", 2023-08-25T09:52:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:52:00.058 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.b", 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhk719cHhAA2rd1vsrVUGTrXRvh0CpbMA==", 2023-08-25T09:52:00.059 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.c", 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkfEJdDBAAS6bbyBOvPzy/+WQk3sRYGQ==", 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:52:00.061 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mds.d", 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCJduhkqY8XLxAAm0h95KeZZOQAxdlMowg9sw==", 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.062 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.0", 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkpeOyIBAAbJccyAjjBv0O/C7BAUeFgA==", 2023-08-25T09:52:00.063 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.064 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.1", 2023-08-25T09:52:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCMduhkYiesLhAAUZb6xKl+HK2P4y5O7BvyhA==", 2023-08-25T09:52:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.065 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.066 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.2", 2023-08-25T09:52:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkvmviARAASgwPDaoCX3oxh9knInf/vA==", 2023-08-25T09:52:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.067 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.3", 2023-08-25T09:52:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCNduhkapfMEBAAMcJ8kL36U3k4PQVSHhcaqw==", 2023-08-25T09:52:00.069 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.070 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.4", 2023-08-25T09:52:00.071 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhk4KXyGBAABvhYUFZKgFugwDoDy+pg9w==", 2023-08-25T09:52:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.072 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.5", 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkSR9FKhAAfQPKXOpaPBGM48P4vCkaUQ==", 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.073 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.074 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.6", 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCRduhkwHjXOhAA8VWDta8EAITkhNTUiKpRsw==", 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "osd.7", 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCSduhkpb+7DhAAvdMBA0Vcflsg82Gb8FKGUg==", 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow profile osd", 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile osd", 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.076 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.0", 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg==", 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.077 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow r", 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.1", 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhk88OYGhAAIIol2sIvblntJ+Lsx8Tq1g==", 2023-08-25T09:52:00.078 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:52:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:52:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:52:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.2", 2023-08-25T09:52:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkhIt6DhAARQvSFE81b0g7lzLu5B4l7g==", 2023-08-25T09:52:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.080 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.3", 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCKduhkalZSJhAAo0QKxoTobLmpEOlUNUTsNA==", 2023-08-25T09:52:00.081 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow", 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow r", 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow rw", 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow rwx" 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.082 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "client.admin", 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCHduhk+hVgKhAAeYt+vH1BXfIdynPctPYQpg==", 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": "allow *", 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow *", 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.083 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.x", 2023-08-25T09:52:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkX4veKRAAdMDsLlrikYkiR7k1PAtqeQ==", 2023-08-25T09:52:00.084 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:52:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:52:00.085 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:00.086 INFO:teuthology.orchestra.run.smithi070.stdout: "entity": "mgr.y", 2023-08-25T09:52:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "AQCIduhkrZF3HRAATOEVVwWRhKU5PXJOBXxXJg==", 2023-08-25T09:52:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "caps": { 2023-08-25T09:52:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "mds": "allow *", 2023-08-25T09:52:00.087 INFO:teuthology.orchestra.run.smithi070.stdout: "mon": "allow profile mgr", 2023-08-25T09:52:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": "allow *" 2023-08-25T09:52:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:00.088 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:00.089 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.057+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f670406a050 msgr2=0x7f670406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.057+0000 7f6729583640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f670406a050 0x7f670406c510 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f672406c090 tx=0x7f6714007930 comp rx=0 tx=0).stop 2023-08-25T09:52:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.057+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 msgr2=0x7f672414ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.057+0000 7f6729583640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 0x7f672414ae20 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f671c00cef0 tx=0x7f671c00a710 comp rx=0 tx=0).stop 2023-08-25T09:52:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 shutdown_connections 2023-08-25T09:52:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f670406a050 0x7f670406c510 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f672415cb40 0x7f672414ae20 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f67240cea30 0x7f6724150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 --2- 172.21.15.70:0/459769686 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f67240ac870 0x7f67241f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 >> 172.21.15.70:0/459769686 conn(0x7f6724069850 msgr2=0x7f672406c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 shutdown_connections 2023-08-25T09:52:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.058+0000 7f6729583640 1 -- 172.21.15.70:0/459769686 wait complete. 2023-08-25T09:52:00.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:00.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.303+0000 7ff4be75d640 1 Processor -- start 2023-08-25T09:52:00.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.303+0000 7ff4be75d640 1 -- start start 2023-08-25T09:52:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4be75d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 0x7ff4b80ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4be75d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff4b815aa10 0x7ff4b80b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4be75d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80ceed0 con 0x7ff4b80b2680 2023-08-25T09:52:00.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4be75d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80b2b10 con 0x7ff4b815aa10 2023-08-25T09:52:00.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4be75d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8159d10 con 0x7ff4b80ce5b0 2023-08-25T09:52:00.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4bccfa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7ff4b80b2680 0x7ff4b806be80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35206/0 (socket says 172.21.15.70:35206) 2023-08-25T09:52:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.304+0000 7ff4bccfa640 1 -- 172.21.15.70:0/1939784383 learned_addr learned my addr 172.21.15.70:0/1939784383 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616982485 0 0) 0x7ff4b80ceed0 con 0x7ff4b80b2680 2023-08-25T09:52:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a4003520 con 0x7ff4b80b2680 2023-08-25T09:52:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b7fff640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 0x7ff4b80ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4af7fe640 1 --2- 172.21.15.70:0/1939784383 >> v2:172.21.15.155:3300/0 conn(0x7ff4b815aa10 0x7ff4b80b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373716596 0 0) 0x7ff4a4003520 con 0x7ff4b80b2680 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 msgr2=0x7ff4b80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 0x7ff4b80ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 >> v2:172.21.15.155:3300/0 conn(0x7ff4b815aa10 msgr2=0x7ff4b80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 --2- 172.21.15.70:0/1939784383 >> v2:172.21.15.155:3300/0 conn(0x7ff4b815aa10 0x7ff4b80b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b815d3f0 con 0x7ff4b80b2680 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7ff4b00019f0 con 0x7ff4b80b2680 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 >> v1:172.21.15.70:6789/0 conn(0x7ff4b80b2680 legacy=0x7ff4b806be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 0x7ff4a4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.305+0000 7ff4b77fe640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a4003fa0 0x7ff4a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4b77fe640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4004d00 0x7ff4a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80b2b10 con 0x7ff4a4004d00 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8159d10 con 0x7ff4a4003fa0 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400ba80 con 0x7ff4b80ce5b0 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4004d00 0x7ff4a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4b7fff640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 0x7ff4a4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4bccfa640 1 -- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 msgr2=0x7ff4a4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80ce5b0 0x7ff4a4003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4bccfa640 1 -- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a4003fa0 msgr2=0x7ff4a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4a4003fa0 0x7ff4a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.306+0000 7ff4bccfa640 1 -- 172.21.15.70:0/1939784383 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4003520 con 0x7ff4a4004d00 2023-08-25T09:52:00.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.307+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4004d00 0x7ff4a4005180 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000a440 tx=0x7ff4a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83fa55bdc03f1432 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:00.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.307+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0035640 con 0x7ff4a4004d00 2023-08-25T09:52:00.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b00357a0 con 0x7ff4a4004d00 2023-08-25T09:52:00.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4b77fe640 1 -- 172.21.15.70:0/1939784383 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0035bb0 con 0x7ff4a4004d00 2023-08-25T09:52:00.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4be75d640 1 -- 172.21.15.70:0/1939784383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4004d00 msgr2=0x7ff4a4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4be75d640 1 --2- 172.21.15.70:0/1939784383 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4a4004d00 0x7ff4a4005180 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff4b000a440 tx=0x7ff4a40057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4bccfa640 1 -- 172.21.15.70:0/1939784383 reap_dead start 2023-08-25T09:52:00.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4be75d640 1 -- 172.21.15.70:0/1939784383 shutdown_connections 2023-08-25T09:52:00.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4be75d640 1 -- 172.21.15.70:0/1939784383 >> 172.21.15.70:0/1939784383 conn(0x7ff4b8069850 msgr2=0x7ff4b8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:00.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4be75d640 1 -- 172.21.15.70:0/1939784383 shutdown_connections 2023-08-25T09:52:00.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.308+0000 7ff4be75d640 1 -- 172.21.15.70:0/1939784383 wait complete. 2023-08-25T09:52:00.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.310+0000 7ff4be75d640 1 Processor -- start 2023-08-25T09:52:00.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.310+0000 7ff4be75d640 1 -- start start 2023-08-25T09:52:00.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4be75d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b09d0 0x7ff4b80b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4be75d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80ae2c0 0x7ff4b80ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4be75d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 0x7ff4b815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4be75d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b815c950 con 0x7ff4b815bf60 2023-08-25T09:52:00.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4be75d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80aee10 con 0x7ff4b80ae2c0 2023-08-25T09:52:00.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4be75d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80af0f0 con 0x7ff4b80b09d0 2023-08-25T09:52:00.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4bccfa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 0x7ff4b815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4bccfa640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 0x7ff4b815c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35850/0 (socket says 172.21.15.70:35850) 2023-08-25T09:52:00.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.311+0000 7ff4bccfa640 1 -- 172.21.15.70:0/4111723868 learned_addr learned my addr 172.21.15.70:0/4111723868 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:00.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4bccfa640 1 -- 172.21.15.70:0/4111723868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b09d0 msgr2=0x7ff4b80b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b09d0 0x7ff4b80b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4bccfa640 1 -- 172.21.15.70:0/4111723868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80ae2c0 msgr2=0x7ff4b80ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80ae2c0 0x7ff4b80ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4bccfa640 1 -- 172.21.15.70:0/4111723868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80af3d0 con 0x7ff4b815bf60 2023-08-25T09:52:00.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4bccfa640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 0x7ff4b815c410 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff4b002edb0 tx=0x7ff4b0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:00.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00019e0 con 0x7ff4b815bf60 2023-08-25T09:52:00.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0035910 con 0x7ff4b815bf60 2023-08-25T09:52:00.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0035ca0 con 0x7ff4b815bf60 2023-08-25T09:52:00.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.312+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b80af6b0 con 0x7ff4b815bf60 2023-08-25T09:52:00.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.313+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b8155470 con 0x7ff4b815bf60 2023-08-25T09:52:00.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.314+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8158160 con 0x7ff4b815bf60 2023-08-25T09:52:00.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.314+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff4b0041430 con 0x7ff4b815bf60 2023-08-25T09:52:00.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.315+0000 7ff4b57fa640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff49806a080 0x7ff49806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.315+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff4b00abfe0 con 0x7ff4b815bf60 2023-08-25T09:52:00.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.316+0000 7ff4b7fff640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff49806a080 0x7ff49806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.316+0000 7ff4b7fff640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff49806a080 0x7ff49806c540 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006a90 tx=0x7ff4a8006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:00.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.320+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff4b0077a90 con 0x7ff4b815bf60 2023-08-25T09:52:00.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.493+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4b804e420 con 0x7ff4b815bf60 2023-08-25T09:52:00.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.495+0000 7ff4b57fa640 1 -- 172.21.15.70:0/4111723868 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+14665 (secure 0 0 0) 0x7ff4b007b0b0 con 0x7ff4b815bf60 2023-08-25T09:52:00.497 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:00.497 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":194,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:51:59.578115+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":180,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:00.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff49806a080 msgr2=0x7ff49806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff49806a080 0x7ff49806c540 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8006a90 tx=0x7ff4a8006a20 comp rx=0 tx=0).stop 2023-08-25T09:52:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 msgr2=0x7ff4b815c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 0x7ff4b815c410 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff4b002edb0 tx=0x7ff4b0001d80 comp rx=0 tx=0).stop 2023-08-25T09:52:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 shutdown_connections 2023-08-25T09:52:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff49806a080 0x7ff49806c540 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff4b815bf60 0x7ff4b815c410 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff4b80ae2c0 0x7ff4b80ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 --2- 172.21.15.70:0/4111723868 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff4b80b09d0 0x7ff4b80b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 >> 172.21.15.70:0/4111723868 conn(0x7ff4b8069850 msgr2=0x7ff4b8158440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:00.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 shutdown_connections 2023-08-25T09:52:00.503 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.498+0000 7ff4be75d640 1 -- 172.21.15.70:0/4111723868 wait complete. 2023-08-25T09:52:00.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:52:00.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:52:00.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.725+0000 7fde5622d640 1 Processor -- start 2023-08-25T09:52:00.727 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.725+0000 7fde5622d640 1 -- start start 2023-08-25T09:52:00.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde5622d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde5015e210 0x7fde50069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde5622d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500ac880 con 0x7fde500cca20 2023-08-25T09:52:00.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde5622d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde500ad400 con 0x7fde50069850 2023-08-25T09:52:00.728 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde5622d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5015e650 con 0x7fde5015e210 2023-08-25T09:52:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde4effd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fde5015e210 0x7fde50069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde4ffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fde500cca20 0x7fde500cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35222/0 (socket says 172.21.15.70:35222) 2023-08-25T09:52:00.729 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.727+0000 7fde4ffff640 1 -- 172.21.15.70:0/3855302990 learned_addr learned my addr 172.21.15.70:0/3855302990 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.728+0000 7fde4e7fc640 1 -- 172.21.15.70:0/3855302990 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405309925 0 0) 0x7fde500ac880 con 0x7fde500cca20 2023-08-25T09:52:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.728+0000 7fde4e7fc640 1 -- 172.21.15.70:0/3855302990 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde38003520 con 0x7fde500cca20 2023-08-25T09:52:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.728+0000 7fde4effd640 1 -- 172.21.15.70:0/3855302990 >> v1:172.21.15.155:6789/0 conn(0x7fde50069850 legacy=0x7fde500c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.728+0000 7fde4effd640 1 -- 172.21.15.70:0/3855302990 >> v1:172.21.15.70:6789/0 conn(0x7fde500cca20 legacy=0x7fde500cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.728+0000 7fde4effd640 1 -- 172.21.15.70:0/3855302990 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde500ad890 con 0x7fde5015e210 2023-08-25T09:52:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.729+0000 7fde4effd640 1 --2- 172.21.15.70:0/3855302990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5015e210 0x7fde50069c30 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fde440097a0 tx=0x7fde4402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61806385620b7fb3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.729+0000 7fde4e7fc640 1 -- 172.21.15.70:0/3855302990 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4402f810 con 0x7fde5015e210 2023-08-25T09:52:00.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.729+0000 7fde4e7fc640 1 -- 172.21.15.70:0/3855302990 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4402f970 con 0x7fde5015e210 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.729+0000 7fde4e7fc640 1 -- 172.21.15.70:0/3855302990 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4402fd00 con 0x7fde5015e210 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.729+0000 7fde5622d640 1 -- 172.21.15.70:0/3855302990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5015e210 msgr2=0x7fde50069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.729+0000 7fde5622d640 1 --2- 172.21.15.70:0/3855302990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5015e210 0x7fde50069c30 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fde440097a0 tx=0x7fde4402b9f0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.730+0000 7fde5622d640 1 -- 172.21.15.70:0/3855302990 shutdown_connections 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.730+0000 7fde5622d640 1 --2- 172.21.15.70:0/3855302990 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde5015e210 0x7fde50069c30 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.730+0000 7fde5622d640 1 -- 172.21.15.70:0/3855302990 >> 172.21.15.70:0/3855302990 conn(0x7fde500cb0d0 msgr2=0x7fde500cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:00.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.730+0000 7fde5622d640 1 -- 172.21.15.70:0/3855302990 shutdown_connections 2023-08-25T09:52:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.730+0000 7fde5622d640 1 -- 172.21.15.70:0/3855302990 wait complete. 2023-08-25T09:52:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.731+0000 7fde5622d640 1 Processor -- start 2023-08-25T09:52:00.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.731+0000 7fde5622d640 1 -- start start 2023-08-25T09:52:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.732+0000 7fde5622d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 0x7fde500c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.732+0000 7fde5622d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde500cca20 0x7fde500c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.732+0000 7fde5622d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde5015e210 0x7fde501f1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.732+0000 7fde5622d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c3140 con 0x7fde50069850 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.732+0000 7fde5622d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c2530 con 0x7fde5015e210 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.732+0000 7fde5622d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde500c2810 con 0x7fde500cca20 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 0x7fde500c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 0x7fde500c1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35860/0 (socket says 172.21.15.70:35860) 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 -- 172.21.15.70:0/2877521701 learned_addr learned my addr 172.21.15.70:0/2877521701 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 -- 172.21.15.70:0/2877521701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde500cca20 msgr2=0x7fde500c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde500cca20 0x7fde500c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 -- 172.21.15.70:0/2877521701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde5015e210 msgr2=0x7fde501f1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde5015e210 0x7fde501f1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 -- 172.21.15.70:0/2877521701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44009400 con 0x7fde50069850 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.733+0000 7fde4f7fe640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 0x7fde500c1d00 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fde4000c450 tx=0x7fde4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.734+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde400052d0 con 0x7fde50069850 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.734+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde40005430 con 0x7fde50069850 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.734+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde40012870 con 0x7fde50069850 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.734+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde500c2af0 con 0x7fde50069850 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.734+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde500c2240 con 0x7fde50069850 2023-08-25T09:52:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.735+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde50066fb0 con 0x7fde50069850 2023-08-25T09:52:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.736+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fde400129d0 con 0x7fde50069850 2023-08-25T09:52:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.737+0000 7fde37fff640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde3c06a050 0x7fde3c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:00.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.737+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fde40087e30 con 0x7fde50069850 2023-08-25T09:52:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.738+0000 7fde4effd640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde3c06a050 0x7fde3c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:00.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.738+0000 7fde4effd640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde3c06a050 0x7fde3c06c510 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fde44036000 tx=0x7fde44039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:00.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.742+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fde400537d0 con 0x7fde50069850 2023-08-25T09:52:00.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.930+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde500ad060 con 0x7fde50069850 2023-08-25T09:52:00.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.931+0000 7fde37fff640 1 -- 172.21.15.70:0/2877521701 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v194) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde40056df0 con 0x7fde50069850 2023-08-25T09:52:00.932 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:52:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde3c06a050 msgr2=0x7fde3c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde3c06a050 0x7fde3c06c510 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fde44036000 tx=0x7fde44039040 comp rx=0 tx=0).stop 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 msgr2=0x7fde500c1d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 0x7fde500c1d00 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fde4000c450 tx=0x7fde4000c910 comp rx=0 tx=0).stop 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 shutdown_connections 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fde3c06a050 0x7fde3c06c510 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fde5015e210 0x7fde501f1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fde500cca20 0x7fde500c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 --2- 172.21.15.70:0/2877521701 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fde50069850 0x7fde500c1d00 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 >> 172.21.15.70:0/2877521701 conn(0x7fde500cb0d0 msgr2=0x7fde5006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 shutdown_connections 2023-08-25T09:52:00.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:00.933+0000 7fde5622d640 1 -- 172.21.15.70:0/2877521701 wait complete. 2023-08-25T09:52:00.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:52:01.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.175+0000 7f05f24d3640 1 Processor -- start 2023-08-25T09:52:01.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.175+0000 7f05f24d3640 1 -- start start 2023-08-25T09:52:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.176+0000 7f05f24d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 0x7f05ec0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05f24d3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f05ec069f60 0x7f05ec0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05f24d3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ec0cc5f0 con 0x7f05ec0cdc20 2023-08-25T09:52:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05f24d3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ec070650 con 0x7f05ec069f60 2023-08-25T09:52:01.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05f24d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec070930 con 0x7f05ec15ae50 2023-08-25T09:52:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05e37fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f05ec0cdc20 0x7f05ec069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35236/0 (socket says 172.21.15.70:35236) 2023-08-25T09:52:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05e37fe640 1 -- 172.21.15.70:0/2244400988 learned_addr learned my addr 172.21.15.70:0/2244400988 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05ebfff640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 0x7f05ec0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.177+0000 7f05f0a70640 1 --2- 172.21.15.70:0/2244400988 >> v2:172.21.15.155:3300/0 conn(0x7f05ec069f60 0x7f05ec0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584368518 0 0) 0x7f05ec0cc5f0 con 0x7f05ec0cdc20 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05d8003520 con 0x7f05ec0cdc20 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1174623724 0 0) 0x7f05d8003520 con 0x7f05ec0cdc20 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 msgr2=0x7f05ec0c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 0x7f05ec0c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 >> v2:172.21.15.155:3300/0 conn(0x7f05ec069f60 msgr2=0x7f05ec0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 --2- 172.21.15.70:0/2244400988 >> v2:172.21.15.155:3300/0 conn(0x7f05ec069f60 0x7f05ec0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec070c10 con 0x7f05ec0cdc20 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f05d40019f0 con 0x7f05ec0cdc20 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 >> v1:172.21.15.70:6789/0 conn(0x7f05ec0cdc20 legacy=0x7f05ec069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05d8003970 0x7f05d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05d80047c0 0x7f05d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 0x7f05d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec070650 con 0x7f05d80047c0 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec070930 con 0x7f05d8003970 2023-08-25T09:52:01.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05d800b980 con 0x7f05ec15ae50 2023-08-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.178+0000 7f05e37fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05d80047c0 0x7f05d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.179+0000 7f05e37fe640 1 -- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 msgr2=0x7f05d8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.179+0000 7f05e37fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05ec15ae50 0x7f05d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.179+0000 7f05e37fe640 1 -- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05d8003970 msgr2=0x7f05d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.179+0000 7f05e37fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05d8003970 0x7f05d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.179+0000 7f05e37fe640 1 -- 172.21.15.70:0/2244400988 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d8003520 con 0x7f05d80047c0 2023-08-25T09:52:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.180+0000 7f05e37fe640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05d80047c0 0x7f05d8004c40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f05d400a860 tx=0x7f05d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f800774d6d32f56d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.180+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4034630 con 0x7f05d80047c0 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.180+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d402f070 con 0x7f05d80047c0 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.180+0000 7f05eb7fe640 1 -- 172.21.15.70:0/2244400988 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40349f0 con 0x7f05d80047c0 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f24d3640 1 -- 172.21.15.70:0/2244400988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05d80047c0 msgr2=0x7f05d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f24d3640 1 --2- 172.21.15.70:0/2244400988 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05d80047c0 0x7f05d8004c40 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f05d400a860 tx=0x7f05d80057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f0a70640 1 -- 172.21.15.70:0/2244400988 reap_dead start 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f24d3640 1 -- 172.21.15.70:0/2244400988 shutdown_connections 2023-08-25T09:52:01.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f24d3640 1 -- 172.21.15.70:0/2244400988 >> 172.21.15.70:0/2244400988 conn(0x7f05ec0cb0d0 msgr2=0x7f05ec0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:01.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f24d3640 1 -- 172.21.15.70:0/2244400988 shutdown_connections 2023-08-25T09:52:01.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.181+0000 7f05f24d3640 1 -- 172.21.15.70:0/2244400988 wait complete. 2023-08-25T09:52:01.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.183+0000 7f05f24d3640 1 Processor -- start 2023-08-25T09:52:01.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.183+0000 7f05f24d3640 1 -- start start 2023-08-25T09:52:01.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05f24d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05d80047c0 0x7f05ec15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05f24d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 0x7f05ec0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05f24d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05ec148e30 0x7f05ec1f0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05f24d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0b3140 con 0x7f05ec154f50 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05f24d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0b2b50 con 0x7f05ec148e30 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05f24d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ec0b21f0 con 0x7f05d80047c0 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 0x7f05ec0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05e37fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 0x7f05ec0b35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35880/0 (socket says 172.21.15.70:35880) 2023-08-25T09:52:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.184+0000 7f05e37fe640 1 -- 172.21.15.70:0/3018878700 learned_addr learned my addr 172.21.15.70:0/3018878700 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05ebfff640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05d80047c0 0x7f05ec15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e37fe640 1 -- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05d80047c0 msgr2=0x7f05ec15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05f0a70640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05ec148e30 0x7f05ec1f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e37fe640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05d80047c0 0x7f05ec15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e37fe640 1 -- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05ec148e30 msgr2=0x7f05ec1f0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e37fe640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05ec148e30 0x7f05ec1f0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e37fe640 1 -- 172.21.15.70:0/3018878700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0b24d0 con 0x7f05ec154f50 2023-08-25T09:52:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e37fe640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 0x7f05ec0b35d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f05d40012e0 tx=0x7f05d4001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4034840 con 0x7f05ec154f50 2023-08-25T09:52:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40349a0 con 0x7f05ec154f50 2023-08-25T09:52:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d4034d50 con 0x7f05ec154f50 2023-08-25T09:52:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ec0b27b0 con 0x7f05ec154f50 2023-08-25T09:52:01.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.185+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ec15e4b0 con 0x7f05ec154f50 2023-08-25T09:52:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.187+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ec0b32f0 con 0x7f05ec154f50 2023-08-25T09:52:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.187+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f05d4031030 con 0x7f05ec154f50 2023-08-25T09:52:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.188+0000 7f05e97fa640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05cc06a050 0x7f05cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.188+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f05d40a9a40 con 0x7f05ec154f50 2023-08-25T09:52:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.188+0000 7f05ebfff640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05cc06a050 0x7f05cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.189+0000 7f05ebfff640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05cc06a050 0x7f05cc06c510 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009890 tx=0x7f05dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:01.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.193+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f05ec0b32f0 con 0x7f05ec154f50 2023-08-25T09:52:01.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.379+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05ec066fb0 con 0x7f05ec154f50 2023-08-25T09:52:01.381 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.380+0000 7f05e97fa640 1 -- 172.21.15.70:0/3018878700 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v194) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05d4046020 con 0x7f05ec154f50 2023-08-25T09:52:01.382 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:52:01.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05cc06a050 msgr2=0x7f05cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05cc06a050 0x7f05cc06c510 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f05dc009890 tx=0x7f05dc008040 comp rx=0 tx=0).stop 2023-08-25T09:52:01.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 msgr2=0x7f05ec0b35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 0x7f05ec0b35d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f05d40012e0 tx=0x7f05d4001d80 comp rx=0 tx=0).stop 2023-08-25T09:52:01.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 shutdown_connections 2023-08-25T09:52:01.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f05cc06a050 0x7f05cc06c510 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f05ec148e30 0x7f05ec1f0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f05ec154f50 0x7f05ec0b35d0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 --2- 172.21.15.70:0/3018878700 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f05d80047c0 0x7f05ec15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 >> 172.21.15.70:0/3018878700 conn(0x7f05ec0cb0d0 msgr2=0x7f05ec0aeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 shutdown_connections 2023-08-25T09:52:01.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.383+0000 7f05f24d3640 1 -- 172.21.15.70:0/3018878700 wait complete. 2023-08-25T09:52:01.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T09:52:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.608+0000 7fae80f39640 1 Processor -- start 2023-08-25T09:52:01.610 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.609+0000 7fae80f39640 1 -- start start 2023-08-25T09:52:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae80f39640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae7c161ae0 0x7fae7c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae80f39640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c1620b0 con 0x7fae7c0ce5f0 2023-08-25T09:52:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae80f39640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c162390 con 0x7fae7c160ff0 2023-08-25T09:52:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae80f39640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae7c0cefe0 con 0x7fae7c161ae0 2023-08-25T09:52:01.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae7ad9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fae7c161ae0 0x7fae7c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae79d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fae7c0ce5f0 0x7fae7c1613d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35250/0 (socket says 172.21.15.70:35250) 2023-08-25T09:52:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.610+0000 7fae79d9b640 1 -- 172.21.15.70:0/952242798 learned_addr learned my addr 172.21.15.70:0/952242798 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.611+0000 7fae7ad9d640 1 -- 172.21.15.70:0/952242798 >> v1:172.21.15.155:6789/0 conn(0x7fae7c160ff0 legacy=0x7fae7c06b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.611+0000 7fae7ad9d640 1 -- 172.21.15.70:0/952242798 >> v1:172.21.15.70:6789/0 conn(0x7fae7c0ce5f0 legacy=0x7fae7c1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.611+0000 7fae7ad9d640 1 -- 172.21.15.70:0/952242798 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c06af80 con 0x7fae7c161ae0 2023-08-25T09:52:01.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.611+0000 7fae7959a640 1 -- 172.21.15.70:0/952242798 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449637573 0 0) 0x7fae7c1620b0 con 0x7fae7c0ce5f0 2023-08-25T09:52:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.612+0000 7fae7ad9d640 1 --2- 172.21.15.70:0/952242798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c0ceaa0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fae740097a0 tx=0x7fae7402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572ba37c074715db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.612+0000 7fae7959a640 1 -- 172.21.15.70:0/952242798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74031830 con 0x7fae7c161ae0 2023-08-25T09:52:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.612+0000 7fae7959a640 1 -- 172.21.15.70:0/952242798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae74031990 con 0x7fae7c161ae0 2023-08-25T09:52:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.612+0000 7fae7959a640 1 -- 172.21.15.70:0/952242798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74031d20 con 0x7fae7c161ae0 2023-08-25T09:52:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 -- 172.21.15.70:0/952242798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 msgr2=0x7fae7c0ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 --2- 172.21.15.70:0/952242798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c0ceaa0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fae740097a0 tx=0x7fae7402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 -- 172.21.15.70:0/952242798 shutdown_connections 2023-08-25T09:52:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 --2- 172.21.15.70:0/952242798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c0ceaa0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 -- 172.21.15.70:0/952242798 >> 172.21.15.70:0/952242798 conn(0x7fae7c069850 msgr2=0x7fae7c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 -- 172.21.15.70:0/952242798 shutdown_connections 2023-08-25T09:52:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.613+0000 7fae80f39640 1 -- 172.21.15.70:0/952242798 wait complete. 2023-08-25T09:52:01.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.615+0000 7fae80f39640 1 Processor -- start 2023-08-25T09:52:01.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.615+0000 7fae80f39640 1 -- start start 2023-08-25T09:52:01.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae80f39640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae7c0ce5f0 0x7fae7c0ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae80f39640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae7c160ff0 0x7fae7c1ea320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae80f39640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c1ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae80f39640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1eae90 con 0x7fae7c161ae0 2023-08-25T09:52:01.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae80f39640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1eb170 con 0x7fae7c0ce5f0 2023-08-25T09:52:01.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae80f39640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c1eb450 con 0x7fae7c160ff0 2023-08-25T09:52:01.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae7ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c1ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae7ad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c1ea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35894/0 (socket says 172.21.15.70:35894) 2023-08-25T09:52:01.621 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.616+0000 7fae7ad9d640 1 -- 172.21.15.70:0/1411021334 learned_addr learned my addr 172.21.15.70:0/1411021334 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae79d9b640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae7c160ff0 0x7fae7c1ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae7ad9d640 1 -- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae7c160ff0 msgr2=0x7fae7c1ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae7ad9d640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae7c160ff0 0x7fae7c1ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae7ad9d640 1 -- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae7c0ce5f0 msgr2=0x7fae7c0ac810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae7ad9d640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae7c0ce5f0 0x7fae7c0ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae7ad9d640 1 -- 172.21.15.70:0/1411021334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae74009400 con 0x7fae7c161ae0 2023-08-25T09:52:01.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae79d9b640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae7c160ff0 0x7fae7c1ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:01.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae7ad9d640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c1ea860 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fae740097a0 tx=0x7fae740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:01.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7403a9e0 con 0x7fae7c161ae0 2023-08-25T09:52:01.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7403ab40 con 0x7fae7c161ae0 2023-08-25T09:52:01.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae740443f0 con 0x7fae7c161ae0 2023-08-25T09:52:01.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.617+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7c1eb730 con 0x7fae7c161ae0 2023-08-25T09:52:01.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.618+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c0cd370 con 0x7fae7c161ae0 2023-08-25T09:52:01.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.619+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c1ebc50 con 0x7fae7c161ae0 2023-08-25T09:52:01.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.620+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae7404d450 con 0x7fae7c161ae0 2023-08-25T09:52:01.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.620+0000 7fae72ffd640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae5c06a050 0x7fae5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:01.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.621+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fae740aa510 con 0x7fae7c161ae0 2023-08-25T09:52:01.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.621+0000 7fae7a59c640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae5c06a050 0x7fae5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:01.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.621+0000 7fae7a59c640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae5c06a050 0x7fae5c06c510 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fae7c1e9670 tx=0x7fae6c007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:01.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.625+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae7c1ebc50 con 0x7fae7c161ae0 2023-08-25T09:52:01.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.816+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fae7c1ebc50 con 0x7fae7c161ae0 2023-08-25T09:52:01.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.816+0000 7fae72ffd640 1 -- 172.21.15.70:0/1411021334 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v194) v1 ==== 84+0+126 (secure 0 0 0) 0x7fae7c1ebc50 con 0x7fae7c161ae0 2023-08-25T09:52:01.818 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:01.818 INFO:teuthology.orchestra.run.smithi070.stdout:[ 2023-08-25T09:52:01.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:01.819 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 1, 2023-08-25T09:52:01.819 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": ".mgr" 2023-08-25T09:52:01.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:01.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:01.819 INFO:teuthology.orchestra.run.smithi070.stdout: "poolnum": 2, 2023-08-25T09:52:01.820 INFO:teuthology.orchestra.run.smithi070.stdout: "poolname": "rbd" 2023-08-25T09:52:01.821 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:01.821 INFO:teuthology.orchestra.run.smithi070.stdout:] 2023-08-25T09:52:01.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae5c06a050 msgr2=0x7fae5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae5c06a050 0x7fae5c06c510 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fae7c1e9670 tx=0x7fae6c007d30 comp rx=0 tx=0).stop 2023-08-25T09:52:01.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 msgr2=0x7fae7c1ea860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:01.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c1ea860 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fae740097a0 tx=0x7fae740046c0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 shutdown_connections 2023-08-25T09:52:01.823 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fae5c06a050 0x7fae5c06c510 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fae7c161ae0 0x7fae7c1ea860 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fae7c160ff0 0x7fae7c1ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.824 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 --2- 172.21.15.70:0/1411021334 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fae7c0ce5f0 0x7fae7c0ac810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:01.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 >> 172.21.15.70:0/1411021334 conn(0x7fae7c069850 msgr2=0x7fae7c070560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:01.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 shutdown_connections 2023-08-25T09:52:01.825 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:01.819+0000 7fae80f39640 1 -- 172.21.15.70:0/1411021334 wait complete. 2023-08-25T09:52:01.840 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T09:52:01.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T09:52:02.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.057+0000 7f530c95d640 1 Processor -- start 2023-08-25T09:52:02.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.057+0000 7f530c95d640 1 -- start start 2023-08-25T09:52:02.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530c95d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f53080cda60 0x7f53080ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530c95d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530815e6f0 con 0x7f5308155120 2023-08-25T09:52:02.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530c95d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530815e9d0 con 0x7f53080ca820 2023-08-25T09:52:02.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530c95d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53080caf30 con 0x7f53080cda60 2023-08-25T09:52:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530659c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5308155120 0x7f53080cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35266/0 (socket says 172.21.15.70:35266) 2023-08-25T09:52:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530659c640 1 -- 172.21.15.70:0/3300939870 learned_addr learned my addr 172.21.15.70:0/3300939870 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.059+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070527041 0 0) 0x7f530815e6f0 con 0x7f5308155120 2023-08-25T09:52:02.062 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f0003520 con 0x7f5308155120 2023-08-25T09:52:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f5305d9b640 1 --2- 172.21.15.70:0/3300939870 >> v2:172.21.15.155:3300/0 conn(0x7f53080cda60 0x7f53080ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2168087713 0 0) 0x7f52f0003520 con 0x7f5308155120 2023-08-25T09:52:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 >> v2:172.21.15.155:3300/0 conn(0x7f53080cda60 msgr2=0x7f53080ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 --2- 172.21.15.70:0/3300939870 >> v2:172.21.15.155:3300/0 conn(0x7f53080cda60 0x7f53080ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 >> v1:172.21.15.155:6789/0 conn(0x7f53080ca820 legacy=0x7f530815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.063 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530815ecb0 con 0x7f5308155120 2023-08-25T09:52:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f52f8001670 con 0x7f5308155120 2023-08-25T09:52:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 >> v1:172.21.15.70:6789/0 conn(0x7f5308155120 legacy=0x7f53080cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f52f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52f00047c0 0x7f52f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0005180 0x7f52f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f000c0a0 con 0x7f52f0003970 2023-08-25T09:52:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f000c210 con 0x7f52f0005180 2023-08-25T09:52:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f000c3d0 con 0x7f52f00047c0 2023-08-25T09:52:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.060+0000 7f530659c640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f52f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.065 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f5306d9d640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0005180 0x7f52f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f5305d9b640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52f00047c0 0x7f52f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f530659c640 1 -- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52f00047c0 msgr2=0x7f52f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f530659c640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f52f00047c0 0x7f52f0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f530659c640 1 -- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0005180 msgr2=0x7f52f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f530659c640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f52f0005180 0x7f52f000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f530659c640 1 -- 172.21.15.70:0/3300939870 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0003520 con 0x7f52f0003970 2023-08-25T09:52:02.066 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.061+0000 7f5306d9d640 1 -- 172.21.15.70:0/3300939870 reap_dead start 2023-08-25T09:52:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.062+0000 7f530659c640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f52f0003d90 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f52f800c240 tx=0x7f52f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c53f54786cd693 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.062+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8005ad0 con 0x7f52f0003970 2023-08-25T09:52:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.062+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f8005c30 con 0x7f52f0003970 2023-08-25T09:52:02.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.062+0000 7f530559a640 1 -- 172.21.15.70:0/3300939870 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f802caf0 con 0x7f52f0003970 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 -- 172.21.15.70:0/3300939870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 msgr2=0x7f52f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f52f0003d90 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f52f800c240 tx=0x7f52f00057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 -- 172.21.15.70:0/3300939870 shutdown_connections 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 --2- 172.21.15.70:0/3300939870 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f52f0003d90 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 -- 172.21.15.70:0/3300939870 >> 172.21.15.70:0/3300939870 conn(0x7f5308069c70 msgr2=0x7f530806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 -- 172.21.15.70:0/3300939870 shutdown_connections 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.063+0000 7f530c95d640 1 -- 172.21.15.70:0/3300939870 wait complete. 2023-08-25T09:52:02.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.065+0000 7f530c95d640 1 Processor -- start 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.065+0000 7f530c95d640 1 -- start start 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530c95d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f5308152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530c95d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53080b1fe0 0x7f53080b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530c95d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308158410 0x7f5308158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530c95d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5308156320 con 0x7f52f0003970 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530c95d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080b29a0 con 0x7f5308158410 2023-08-25T09:52:02.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530c95d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53081556a0 con 0x7f53080b1fe0 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f5308152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530659c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f5308152c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35912/0 (socket says 172.21.15.70:35912) 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.066+0000 7f530659c640 1 -- 172.21.15.70:0/3235335410 learned_addr learned my addr 172.21.15.70:0/3235335410 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530659c640 1 -- 172.21.15.70:0/3235335410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53080b1fe0 msgr2=0x7f53080b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530659c640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53080b1fe0 0x7f53080b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530659c640 1 -- 172.21.15.70:0/3235335410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308158410 msgr2=0x7f5308158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530659c640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308158410 0x7f5308158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.071 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530659c640 1 -- 172.21.15.70:0/3235335410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5308155980 con 0x7f52f0003970 2023-08-25T09:52:02.072 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530659c640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f5308152c90 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f52f8001930 tx=0x7f52f802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f803c040 con 0x7f52f0003970 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f802cd80 con 0x7f52f0003970 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f80072b0 con 0x7f52f0003970 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5308155c60 con 0x7f52f0003970 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.067+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53081f1ee0 con 0x7f52f0003970 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.069+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5308156590 con 0x7f52f0003970 2023-08-25T09:52:02.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.070+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52f8032070 con 0x7f52f0003970 2023-08-25T09:52:02.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.071+0000 7f52f6ffd640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52e806a050 0x7f52e806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.071+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f52f80a98a0 con 0x7f52f0003970 2023-08-25T09:52:02.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.071+0000 7f5305d9b640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52e806a050 0x7f52e806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.071+0000 7f5305d9b640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52e806a050 0x7f52e806c510 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f52fc006a90 tx=0x7f52fc006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:02.075 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.074+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52f8075240 con 0x7f52f0003970 2023-08-25T09:52:02.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.246+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f530804e420 con 0x7f52f0003970 2023-08-25T09:52:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.583+0000 7f52f6ffd640 1 -- 172.21.15.70:0/3235335410 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v195) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52f8078860 con 0x7f52f0003970 2023-08-25T09:52:02.586 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' created 2023-08-25T09:52:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.587+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52e806a050 msgr2=0x7f52e806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.587+0000 7f530c95d640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52e806a050 0x7f52e806c510 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f52fc006a90 tx=0x7f52fc006a20 comp rx=0 tx=0).stop 2023-08-25T09:52:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.587+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 msgr2=0x7f5308152c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.587+0000 7f530c95d640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f5308152c90 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f52f8001930 tx=0x7f52f802e070 comp rx=0 tx=0).stop 2023-08-25T09:52:02.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 shutdown_connections 2023-08-25T09:52:02.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f52e806a050 0x7f52e806c510 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5308158410 0x7f5308158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f53080b1fe0 0x7f53080b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 --2- 172.21.15.70:0/3235335410 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f52f0003970 0x7f5308152c90 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 >> 172.21.15.70:0/3235335410 conn(0x7f5308069c70 msgr2=0x7f530806eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 shutdown_connections 2023-08-25T09:52:02.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.588+0000 7f530c95d640 1 -- 172.21.15.70:0/3235335410 wait complete. 2023-08-25T09:52:02.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T09:52:02.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.835+0000 7f01eab2f640 1 Processor -- start 2023-08-25T09:52:02.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.835+0000 7f01eab2f640 1 -- start start 2023-08-25T09:52:02.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.837+0000 7f01eab2f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 0x7f01e415e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.837+0000 7f01eab2f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f01e40cd660 0x7f01e40cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.837+0000 7f01eab2f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e415e780 con 0x7f01e406f0e0 2023-08-25T09:52:02.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.837+0000 7f01eab2f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e406f8e0 con 0x7f01e40cd660 2023-08-25T09:52:02.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.837+0000 7f01eab2f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e406fbc0 con 0x7f01e4159bf0 2023-08-25T09:52:02.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.838+0000 7f01e88cb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 0x7f01e415e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.838+0000 7f01e88cb640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 0x7f01e415e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:48640/0 (socket says 172.21.15.70:48640) 2023-08-25T09:52:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.838+0000 7f01e88cb640 1 -- 172.21.15.70:0/811847418 learned_addr learned my addr 172.21.15.70:0/811847418 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.838+0000 7f01e90cc640 1 --2- 172.21.15.70:0/811847418 >> v2:172.21.15.155:3300/0 conn(0x7f01e40cd660 0x7f01e40cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.839+0000 7f01e88cb640 1 -- 172.21.15.70:0/811847418 >> v2:172.21.15.155:3300/0 conn(0x7f01e40cd660 msgr2=0x7f01e40cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.839+0000 7f01e88cb640 1 --2- 172.21.15.70:0/811847418 >> v2:172.21.15.155:3300/0 conn(0x7f01e40cd660 0x7f01e40cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.839+0000 7f01e88cb640 1 -- 172.21.15.70:0/811847418 >> v1:172.21.15.155:6789/0 conn(0x7f01e406f0e0 legacy=0x7f01e4070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.839+0000 7f01e88cb640 1 -- 172.21.15.70:0/811847418 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40ce000 con 0x7f01e4159bf0 2023-08-25T09:52:02.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.840+0000 7f01e88cb640 1 --2- 172.21.15.70:0/811847418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 0x7f01e415e240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009870 tx=0x7f01d802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc989d509ca1c1a7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:02.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.841+0000 7f01e37fe640 1 -- 172.21.15.70:0/811847418 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8030840 con 0x7f01e4159bf0 2023-08-25T09:52:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.841+0000 7f01e37fe640 1 -- 172.21.15.70:0/811847418 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d80309a0 con 0x7f01e4159bf0 2023-08-25T09:52:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.841+0000 7f01e37fe640 1 -- 172.21.15.70:0/811847418 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8030d30 con 0x7f01e4159bf0 2023-08-25T09:52:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 -- 172.21.15.70:0/811847418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 msgr2=0x7f01e415e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 --2- 172.21.15.70:0/811847418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 0x7f01e415e240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f01d8009870 tx=0x7f01d802b980 comp rx=0 tx=0).stop 2023-08-25T09:52:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 -- 172.21.15.70:0/811847418 shutdown_connections 2023-08-25T09:52:02.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 --2- 172.21.15.70:0/811847418 >> v2:172.21.15.155:3300/0 conn(0x7f01e40cd660 0x7f01e40cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 --2- 172.21.15.70:0/811847418 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e4159bf0 0x7f01e415e240 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 -- 172.21.15.70:0/811847418 >> 172.21.15.70:0/811847418 conn(0x7f01e40cb0d0 msgr2=0x7f01e40cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 -- 172.21.15.70:0/811847418 shutdown_connections 2023-08-25T09:52:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.842+0000 7f01eab2f640 1 -- 172.21.15.70:0/811847418 wait complete. 2023-08-25T09:52:02.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.844+0000 7f01eab2f640 1 Processor -- start 2023-08-25T09:52:02.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.845+0000 7f01eab2f640 1 -- start start 2023-08-25T09:52:02.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01eab2f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e406f0e0 0x7f01e40c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01eab2f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 0x7f01e40c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01eab2f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e4159bf0 0x7f01e4158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01eab2f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e414a110 con 0x7f01e40cd660 2023-08-25T09:52:02.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01eab2f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c4d70 con 0x7f01e4159bf0 2023-08-25T09:52:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01eab2f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c3790 con 0x7f01e406f0e0 2023-08-25T09:52:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 0x7f01e40c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01e3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 0x7f01e40c2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35914/0 (socket says 172.21.15.70:35914) 2023-08-25T09:52:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.846+0000 7f01e3fff640 1 -- 172.21.15.70:0/633599156 learned_addr learned my addr 172.21.15.70:0/633599156 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:02.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e88cb640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e406f0e0 0x7f01e40c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e90cc640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e4159bf0 0x7f01e4158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e3fff640 1 -- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e406f0e0 msgr2=0x7f01e40c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e3fff640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e406f0e0 0x7f01e40c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e3fff640 1 -- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e4159bf0 msgr2=0x7f01e4158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e3fff640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e4159bf0 0x7f01e4158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e3fff640 1 -- 172.21.15.70:0/633599156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80094d0 con 0x7f01e40cd660 2023-08-25T09:52:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e88cb640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e406f0e0 0x7f01e40c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:52:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.847+0000 7f01e3fff640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 0x7f01e40c2ae0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c450 tx=0x7f01d000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.848+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00052d0 con 0x7f01e40cd660 2023-08-25T09:52:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.848+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0005430 con 0x7f01e40cd660 2023-08-25T09:52:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.848+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0012870 con 0x7f01e40cd660 2023-08-25T09:52:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.848+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e40c3a70 con 0x7f01e40cd660 2023-08-25T09:52:02.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.848+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e40c4030 con 0x7f01e40cd660 2023-08-25T09:52:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.849+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e4066fb0 con 0x7f01e40cd660 2023-08-25T09:52:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.850+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f01d00129d0 con 0x7f01e40cd660 2023-08-25T09:52:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.851+0000 7f01e17fa640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01c806a050 0x7f01c806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.851+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f01d0087fa0 con 0x7f01e40cd660 2023-08-25T09:52:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.851+0000 7f01e88cb640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01c806a050 0x7f01c806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.852+0000 7f01e88cb640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01c806a050 0x7f01c806c510 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f01d8036000 tx=0x7f01d803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:02.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:02.856+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f01d00537d0 con 0x7f01e40cd660 2023-08-25T09:52:03.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.046+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f01e4150d50 con 0x7f01e40cd660 2023-08-25T09:52:03.606 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.601+0000 7f01e17fa640 1 -- 172.21.15.70:0/633599156 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v196) v1 ==== 170+0+0 (secure 0 0 0) 0x7f01d0056df0 con 0x7f01e40cd660 2023-08-25T09:52:03.612 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' created 2023-08-25T09:52:03.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01c806a050 msgr2=0x7f01c806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:03.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01c806a050 0x7f01c806c510 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f01d8036000 tx=0x7f01d803b040 comp rx=0 tx=0).stop 2023-08-25T09:52:03.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 msgr2=0x7f01e40c2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 0x7f01e40c2ae0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c450 tx=0x7f01d000c910 comp rx=0 tx=0).stop 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 shutdown_connections 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f01c806a050 0x7f01c806c510 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f01e4159bf0 0x7f01e4158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f01e40cd660 0x7f01e40c2ae0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 --2- 172.21.15.70:0/633599156 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f01e406f0e0 0x7f01e40c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 >> 172.21.15.70:0/633599156 conn(0x7f01e40cb0d0 msgr2=0x7f01e40c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:03.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 shutdown_connections 2023-08-25T09:52:03.620 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.616+0000 7f01eab2f640 1 -- 172.21.15.70:0/633599156 wait complete. 2023-08-25T09:52:03.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T09:52:03.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.842+0000 7efd787b1640 1 Processor -- start 2023-08-25T09:52:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.842+0000 7efd787b1640 1 -- start start 2023-08-25T09:52:03.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd787b1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd700c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd787b1640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efd70069f60 0x7efd700ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd787b1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd700cc5f0 con 0x7efd700cdc20 2023-08-25T09:52:03.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd787b1640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd70070650 con 0x7efd70069f60 2023-08-25T09:52:03.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd787b1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd70070930 con 0x7efd7015ae50 2023-08-25T09:52:03.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd75d4c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efd700cdc20 0x7efd70069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35282/0 (socket says 172.21.15.70:35282) 2023-08-25T09:52:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd75d4c640 1 -- 172.21.15.70:0/1937294917 learned_addr learned my addr 172.21.15.70:0/1937294917 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178289054 0 0) 0x7efd700cc5f0 con 0x7efd700cdc20 2023-08-25T09:52:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd7654d640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd700c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1937294917 >> v2:172.21.15.155:3300/0 conn(0x7efd70069f60 0x7efd700ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.844+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd58003520 con 0x7efd700cdc20 2023-08-25T09:52:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372035025 0 0) 0x7efd58003520 con 0x7efd700cdc20 2023-08-25T09:52:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 msgr2=0x7efd700c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd700c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 >> v2:172.21.15.155:3300/0 conn(0x7efd70069f60 msgr2=0x7efd700ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:03.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 --2- 172.21.15.70:0/1937294917 >> v2:172.21.15.155:3300/0 conn(0x7efd70069f60 0x7efd700ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70070c10 con 0x7efd700cdc20 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7efd6c0019f0 con 0x7efd700cdc20 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 >> v1:172.21.15.70:6789/0 conn(0x7efd700cdc20 legacy=0x7efd70069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58003970 0x7efd58003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd58004d00 0x7efd58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70070650 con 0x7efd58004d00 2023-08-25T09:52:03.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd70070930 con 0x7efd58003970 2023-08-25T09:52:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd5800bab0 con 0x7efd7015ae50 2023-08-25T09:52:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.845+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd58004d00 0x7efd58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd75d4c640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd76d4e640 1 -- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 msgr2=0x7efd580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd7654d640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58003970 0x7efd58003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd76d4e640 1 -- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58003970 msgr2=0x7efd58003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58003970 0x7efd58003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd76d4e640 1 -- 172.21.15.70:0/1937294917 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd58003520 con 0x7efd58004d00 2023-08-25T09:52:03.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.846+0000 7efd75d4c640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd7015ae50 0x7efd580047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.847+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd58004d00 0x7efd58005180 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7efd6800ba30 tx=0x7efd580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05f8ae8ffb6b82b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.847+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68032680 con 0x7efd58004d00 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.847+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd680327e0 con 0x7efd58004d00 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.847+0000 7efd7554b640 1 -- 172.21.15.70:0/1937294917 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68032b70 con 0x7efd58004d00 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd787b1640 1 -- 172.21.15.70:0/1937294917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd58004d00 msgr2=0x7efd58005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd787b1640 1 --2- 172.21.15.70:0/1937294917 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd58004d00 0x7efd58005180 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7efd6800ba30 tx=0x7efd580057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd76d4e640 1 -- 172.21.15.70:0/1937294917 reap_dead start 2023-08-25T09:52:03.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd787b1640 1 -- 172.21.15.70:0/1937294917 shutdown_connections 2023-08-25T09:52:03.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd787b1640 1 -- 172.21.15.70:0/1937294917 >> 172.21.15.70:0/1937294917 conn(0x7efd700cb0d0 msgr2=0x7efd700cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:03.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd787b1640 1 -- 172.21.15.70:0/1937294917 shutdown_connections 2023-08-25T09:52:03.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.848+0000 7efd787b1640 1 -- 172.21.15.70:0/1937294917 wait complete. 2023-08-25T09:52:03.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.850+0000 7efd787b1640 1 Processor -- start 2023-08-25T09:52:03.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.850+0000 7efd787b1640 1 -- start start 2023-08-25T09:52:03.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd787b1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58004d00 0x7efd7014af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd787b1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd701f3ab0 0x7efd700a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd787b1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 0x7efd701f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd787b1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd700a8e30 con 0x7efd700a89b0 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd787b1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd701e09f0 con 0x7efd58004d00 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd787b1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd701f3680 con 0x7efd701f3ab0 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd76d4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 0x7efd701f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd76d4e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 0x7efd701f2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35942/0 (socket says 172.21.15.70:35942) 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd76d4e640 1 -- 172.21.15.70:0/1459793215 learned_addr learned my addr 172.21.15.70:0/1459793215 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd7654d640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58004d00 0x7efd7014af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.851+0000 7efd75d4c640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd701f3ab0 0x7efd700a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd76d4e640 1 -- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd701f3ab0 msgr2=0x7efd700a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd701f3ab0 0x7efd700a8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd76d4e640 1 -- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58004d00 msgr2=0x7efd7014af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:03.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58004d00 0x7efd7014af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd76d4e640 1 -- 172.21.15.70:0/1459793215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd701e00c0 con 0x7efd700a89b0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd76d4e640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 0x7efd701f2950 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7efd68035040 tx=0x7efd68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd68004220 con 0x7efd700a89b0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd68004380 con 0x7efd700a89b0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6803b830 con 0x7efd700a89b0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.852+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd701e03a0 con 0x7efd700a89b0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.853+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd701f3f30 con 0x7efd700a89b0 2023-08-25T09:52:03.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.854+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd70066fb0 con 0x7efd700a89b0 2023-08-25T09:52:03.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.855+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efd6803b990 con 0x7efd700a89b0 2023-08-25T09:52:03.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.856+0000 7efd66ffd640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd5c06a050 0x7efd5c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:03.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.856+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 5962+0+0 (secure 0 0 0) 0x7efd680aaca0 con 0x7efd700a89b0 2023-08-25T09:52:03.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.856+0000 7efd7654d640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd5c06a050 0x7efd5c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:03.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.857+0000 7efd7654d640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd5c06a050 0x7efd5c06c510 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efd701f5b10 tx=0x7efd60008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:03.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:03.860+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efd68076350 con 0x7efd700a89b0 2023-08-25T09:52:04.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.086+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efd701e0b30 con 0x7efd700a89b0 2023-08-25T09:52:04.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.630+0000 7efd66ffd640 1 -- 172.21.15.70:0/1459793215 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v87) v1 ==== 174+0+0 (secure 0 0 0) 0x7efd680360c0 con 0x7efd700a89b0 2023-08-25T09:52:04.633 INFO:teuthology.orchestra.run.smithi070.stderr:new fs with metadata pool 23 and data pool 24 2023-08-25T09:52:04.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd5c06a050 msgr2=0x7efd5c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:04.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd5c06a050 0x7efd5c06c510 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7efd701f5b10 tx=0x7efd60008040 comp rx=0 tx=0).stop 2023-08-25T09:52:04.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 msgr2=0x7efd701f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:04.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 0x7efd701f2950 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7efd68035040 tx=0x7efd68004730 comp rx=0 tx=0).stop 2023-08-25T09:52:04.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 shutdown_connections 2023-08-25T09:52:04.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd5c06a050 0x7efd5c06c510 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd700a89b0 0x7efd701f2950 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd701f3ab0 0x7efd700a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 --2- 172.21.15.70:0/1459793215 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd58004d00 0x7efd7014af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 >> 172.21.15.70:0/1459793215 conn(0x7efd700cb0d0 msgr2=0x7efd7015a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:04.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.634+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 shutdown_connections 2023-08-25T09:52:04.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.635+0000 7efd787b1640 1 -- 172.21.15.70:0/1459793215 wait complete. 2023-08-25T09:52:04.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:04.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.842+0000 7f613c3ab640 1 Processor -- start 2023-08-25T09:52:04.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.842+0000 7f613c3ab640 1 -- start start 2023-08-25T09:52:04.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613c3ab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f613415cb40 0x7f61340ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613c3ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 0x7f61340ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613c3ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613406f050 con 0x7f61340ac870 2023-08-25T09:52:04.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613c3ab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613406f330 con 0x7f613415cb40 2023-08-25T09:52:04.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613c3ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613406f610 con 0x7f61340b2b40 2023-08-25T09:52:04.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f6139946640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f61340ac870 0x7f6134160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35290/0 (socket says 172.21.15.70:35290) 2023-08-25T09:52:04.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f6139946640 1 -- 172.21.15.70:0/591822018 learned_addr learned my addr 172.21.15.70:0/591822018 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613a948640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 0x7f61340ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.844+0000 7f613a147640 1 --2- 172.21.15.70:0/591822018 >> v2:172.21.15.155:3300/0 conn(0x7f613415cb40 0x7f61340ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579622963 0 0) 0x7f613406f050 con 0x7f61340ac870 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6120003520 con 0x7f61340ac870 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1670264790 0 0) 0x7f6120003520 con 0x7f61340ac870 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 msgr2=0x7f61340ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 0x7f61340ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 >> v2:172.21.15.155:3300/0 conn(0x7f613415cb40 msgr2=0x7f61340ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:04.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 --2- 172.21.15.70:0/591822018 >> v2:172.21.15.155:3300/0 conn(0x7f613415cb40 0x7f61340ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613406f8f0 con 0x7f61340ac870 2023-08-25T09:52:04.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f6124001670 con 0x7f61340ac870 2023-08-25T09:52:04.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 >> v1:172.21.15.70:6789/0 conn(0x7f61340ac870 legacy=0x7f6134160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 0x7f6120003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6120003ef0 0x7f61200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6120004d00 0x7f6120005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613406f330 con 0x7f6120004d00 2023-08-25T09:52:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613406f610 con 0x7f6120003ef0 2023-08-25T09:52:04.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.845+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612000b980 con 0x7f61340b2b40 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a948640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6120004d00 0x7f6120005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a147640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 0x7f6120003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a948640 1 -- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 msgr2=0x7f6120003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a948640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f61340b2b40 0x7f6120003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a948640 1 -- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6120003ef0 msgr2=0x7f61200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a948640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6120003ef0 0x7f61200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.846+0000 7f613a948640 1 -- 172.21.15.70:0/591822018 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120003520 con 0x7f6120004d00 2023-08-25T09:52:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.847+0000 7f613a948640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6120004d00 0x7f6120005180 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f612c009f90 tx=0x7f61200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c59f89f7c56a7a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.847+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c02cbd0 con 0x7f6120004d00 2023-08-25T09:52:04.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.847+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612c00f040 con 0x7f6120004d00 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.847+0000 7f6139145640 1 -- 172.21.15.70:0/591822018 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612c034440 con 0x7f6120004d00 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613c3ab640 1 -- 172.21.15.70:0/591822018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6120004d00 msgr2=0x7f6120005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613c3ab640 1 --2- 172.21.15.70:0/591822018 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f6120004d00 0x7f6120005180 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f612c009f90 tx=0x7f61200057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613a948640 1 -- 172.21.15.70:0/591822018 reap_dead start 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613c3ab640 1 -- 172.21.15.70:0/591822018 shutdown_connections 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613c3ab640 1 -- 172.21.15.70:0/591822018 >> 172.21.15.70:0/591822018 conn(0x7f6134069850 msgr2=0x7f6134069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:04.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613c3ab640 1 -- 172.21.15.70:0/591822018 shutdown_connections 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.848+0000 7f613c3ab640 1 -- 172.21.15.70:0/591822018 wait complete. 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.850+0000 7f613c3ab640 1 Processor -- start 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.850+0000 7f613c3ab640 1 -- start start 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613c3ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6120004d00 0x7f613414c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613c3ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 0x7f613414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613c3ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f613414f630 0x7f61340b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613c3ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61341504d0 con 0x7f613415f6b0 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613c3ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613414ecd0 con 0x7f613414f630 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613c3ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613414dfb0 con 0x7f6120004d00 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f6139946640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 0x7f613414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f6139946640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 0x7f613414e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35950/0 (socket says 172.21.15.70:35950) 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f6139946640 1 -- 172.21.15.70:0/3945785241 learned_addr learned my addr 172.21.15.70:0/3945785241 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613a948640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f613414f630 0x7f61340b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.851+0000 7f613a147640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6120004d00 0x7f613414c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f6139946640 1 -- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6120004d00 msgr2=0x7f613414c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f6139946640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6120004d00 0x7f613414c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f6139946640 1 -- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f613414f630 msgr2=0x7f61340b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f6139946640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f613414f630 0x7f61340b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f6139946640 1 -- 172.21.15.70:0/3945785241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613414ff10 con 0x7f613415f6b0 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f6139946640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 0x7f613414e790 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f612400c7a0 tx=0x7f612400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240074e0 con 0x7f613415f6b0 2023-08-25T09:52:04.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6124007640 con 0x7f613415f6b0 2023-08-25T09:52:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61240049e0 con 0x7f613415f6b0 2023-08-25T09:52:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61341501f0 con 0x7f613415f6b0 2023-08-25T09:52:04.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.852+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613414fab0 con 0x7f613415f6b0 2023-08-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.854+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613414fca0 con 0x7f613415f6b0 2023-08-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.855+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6124004b40 con 0x7f613415f6b0 2023-08-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.856+0000 7f612affd640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f611806a080 0x7f611806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.856+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6124087ac0 con 0x7f613415f6b0 2023-08-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.856+0000 7f613a147640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f611806a080 0x7f611806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:04.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.857+0000 7f613a147640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f611806a080 0x7f611806c540 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6130009a90 tx=0x7f6130008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:04.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:04.861+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6124053110 con 0x7f613415f6b0 2023-08-25T09:52:05.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.053+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f613404e420 con 0x7f613415f6b0 2023-08-25T09:52:05.057 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.055+0000 7f612affd640 1 -- 172.21.15.70:0/3945785241 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+18326 (secure 0 0 0) 0x7f6124018090 con 0x7f613415f6b0 2023-08-25T09:52:05.057 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:05.058 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":197,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:04.088977+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f611806a080 msgr2=0x7f611806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f611806a080 0x7f611806c540 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f6130009a90 tx=0x7f6130008040 comp rx=0 tx=0).stop 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 msgr2=0x7f613414e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 0x7f613414e790 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f612400c7a0 tx=0x7f612400cc60 comp rx=0 tx=0).stop 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 shutdown_connections 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f611806a080 0x7f611806c540 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f613414f630 0x7f61340b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f613415f6b0 0x7f613414e790 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 --2- 172.21.15.70:0/3945785241 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f6120004d00 0x7f613414c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.057+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 >> 172.21.15.70:0/3945785241 conn(0x7f6134069850 msgr2=0x7f613406bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.058+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 shutdown_connections 2023-08-25T09:52:05.061 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.058+0000 7f613c3ab640 1 -- 172.21.15.70:0/3945785241 wait complete. 2023-08-25T09:52:05.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:05.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.277+0000 7fb0c094e640 1 Processor -- start 2023-08-25T09:52:05.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.277+0000 7fb0c094e640 1 -- start start 2023-08-25T09:52:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0c094e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb0bc0ce5b0 0x7fb0bc0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0c094e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 0x7fb0bc06f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0c094e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0bc0ceed0 con 0x7fb0bc15aa10 2023-08-25T09:52:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0c094e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc159d10 con 0x7fb0bc0ce5b0 2023-08-25T09:52:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0c094e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc15d3f0 con 0x7fb0bc0b2850 2023-08-25T09:52:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0b9d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb0bc15aa10 0x7fb0bc0b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35292/0 (socket says 172.21.15.70:35292) 2023-08-25T09:52:05.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0b9d9b640 1 -- 172.21.15.70:0/1148763841 learned_addr learned my addr 172.21.15.70:0/1148763841 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679842208 0 0) 0x7fb0bc0ceed0 con 0x7fb0bc15aa10 2023-08-25T09:52:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0ba59c640 1 --2- 172.21.15.70:0/1148763841 >> v2:172.21.15.155:3300/0 conn(0x7fb0bc0ce5b0 0x7fb0bc0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a4005730 con 0x7fb0bc15aa10 2023-08-25T09:52:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.279+0000 7fb0bad9d640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 0x7fb0bc06f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963347628 0 0) 0x7fb0a4005730 con 0x7fb0bc15aa10 2023-08-25T09:52:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 msgr2=0x7fb0bc06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 0x7fb0bc06f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 >> v2:172.21.15.155:3300/0 conn(0x7fb0bc0ce5b0 msgr2=0x7fb0bc0ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 --2- 172.21.15.70:0/1148763841 >> v2:172.21.15.155:3300/0 conn(0x7fb0bc0ce5b0 0x7fb0bc0ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0b2d20 con 0x7fb0bc15aa10 2023-08-25T09:52:05.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb0b0001690 con 0x7fb0bc15aa10 2023-08-25T09:52:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 >> v1:172.21.15.70:6789/0 conn(0x7fb0bc15aa10 legacy=0x7fb0bc0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 0x7fb0a4005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a4006100 0x7fb0a40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0a4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc159d10 con 0x7fb0a4006f10 2023-08-25T09:52:05.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc15d3f0 con 0x7fb0a4006100 2023-08-25T09:52:05.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a400dc90 con 0x7fb0bc0b2850 2023-08-25T09:52:05.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0bad9d640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0a4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0ba59c640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 0x7fb0a4005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.280+0000 7fb0b9d9b640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a4006100 0x7fb0a40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.281+0000 7fb0bad9d640 1 -- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 msgr2=0x7fb0a4005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:05.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.281+0000 7fb0bad9d640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0b2850 0x7fb0a4005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.281+0000 7fb0bad9d640 1 -- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a4006100 msgr2=0x7fb0a40069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:05.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.281+0000 7fb0bad9d640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0a4006100 0x7fb0a40069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.281+0000 7fb0bad9d640 1 -- 172.21.15.70:0/1148763841 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a4005730 con 0x7fb0a4006f10 2023-08-25T09:52:05.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.282+0000 7fb0bad9d640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0a4007390 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4007360 tx=0x7fb0a4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4723b98bd8489cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:05.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b402cc80 con 0x7fb0a4006f10 2023-08-25T09:52:05.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b402cde0 con 0x7fb0a4006f10 2023-08-25T09:52:05.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0b959a640 1 -- 172.21.15.70:0/1148763841 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4034660 con 0x7fb0a4006f10 2023-08-25T09:52:05.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0c094e640 1 -- 172.21.15.70:0/1148763841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 msgr2=0x7fb0a4007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0c094e640 1 --2- 172.21.15.70:0/1148763841 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0a4007390 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4007360 tx=0x7fb0a4007a00 comp rx=0 tx=0).stop 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0bad9d640 1 -- 172.21.15.70:0/1148763841 reap_dead start 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0c094e640 1 -- 172.21.15.70:0/1148763841 shutdown_connections 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0c094e640 1 -- 172.21.15.70:0/1148763841 >> 172.21.15.70:0/1148763841 conn(0x7fb0bc069850 msgr2=0x7fb0bc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0c094e640 1 -- 172.21.15.70:0/1148763841 shutdown_connections 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.283+0000 7fb0c094e640 1 -- 172.21.15.70:0/1148763841 wait complete. 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.285+0000 7fb0c094e640 1 Processor -- start 2023-08-25T09:52:05.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.285+0000 7fb0c094e640 1 -- start start 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0c094e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0bc1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0c094e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0bc0aaa10 0x7fb0bc0aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0c094e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0acc50 0x7fb0bc0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0c094e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0ab3d0 con 0x7fb0a4006f10 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0c094e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0af700 con 0x7fb0bc0aaa10 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0c094e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc147730 con 0x7fb0bc0acc50 2023-08-25T09:52:05.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0ba59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0bc1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0ba59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0bc1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35980/0 (socket says 172.21.15.70:35980) 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.286+0000 7fb0ba59c640 1 -- 172.21.15.70:0/4065509492 learned_addr learned my addr 172.21.15.70:0/4065509492 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0ba59c640 1 -- 172.21.15.70:0/4065509492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0acc50 msgr2=0x7fb0bc0ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0ba59c640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0acc50 0x7fb0bc0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0ba59c640 1 -- 172.21.15.70:0/4065509492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0bc0aaa10 msgr2=0x7fb0bc0aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0ba59c640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0bc0aaa10 0x7fb0bc0aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0ba59c640 1 -- 172.21.15.70:0/4065509492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0ad9f0 con 0x7fb0a4006f10 2023-08-25T09:52:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0ba59c640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0bc1471f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac011700 tx=0x7fb0ac011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00f940 con 0x7fb0a4006f10 2023-08-25T09:52:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac00faa0 con 0x7fb0a4006f10 2023-08-25T09:52:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00fe10 con 0x7fb0a4006f10 2023-08-25T09:52:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.287+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc0adcd0 con 0x7fb0a4006f10 2023-08-25T09:52:05.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.288+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc0adfb0 con 0x7fb0a4006f10 2023-08-25T09:52:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.289+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc04e420 con 0x7fb0a4006f10 2023-08-25T09:52:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.290+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb0ac015460 con 0x7fb0a4006f10 2023-08-25T09:52:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.290+0000 7fb0aaffd640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09c06a080 0x7fb09c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.291+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb0ac088b10 con 0x7fb0a4006f10 2023-08-25T09:52:05.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.291+0000 7fb0b9d9b640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09c06a080 0x7fb09c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.291+0000 7fb0b9d9b640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09c06a080 0x7fb09c06c540 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0ac820 tx=0x7fb0b0001d90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:05.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.295+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb0ac054160 con 0x7fb0a4006f10 2023-08-25T09:52:05.489 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.487+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0bc0cae50 con 0x7fb0a4006f10 2023-08-25T09:52:05.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.490+0000 7fb0aaffd640 1 -- 172.21.15.70:0/4065509492 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+18326 (secure 0 0 0) 0x7fb0ac057780 con 0x7fb0a4006f10 2023-08-25T09:52:05.493 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:05.493 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":197,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:04.088977+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:05.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09c06a080 msgr2=0x7fb09c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09c06a080 0x7fb09c06c540 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0ac820 tx=0x7fb0b0001d90 comp rx=0 tx=0).stop 2023-08-25T09:52:05.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 msgr2=0x7fb0bc1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0bc1471f0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac011700 tx=0x7fb0ac011bc0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 shutdown_connections 2023-08-25T09:52:05.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb09c06a080 0x7fb09c06c540 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb0bc0acc50 0x7fb0bc0ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb0bc0aaa10 0x7fb0bc0aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 --2- 172.21.15.70:0/4065509492 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb0a4006f10 0x7fb0bc1471f0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 >> 172.21.15.70:0/4065509492 conn(0x7fb0bc069850 msgr2=0x7fb0bc15af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:05.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 shutdown_connections 2023-08-25T09:52:05.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.493+0000 7fb0c094e640 1 -- 172.21.15.70:0/4065509492 wait complete. 2023-08-25T09:52:05.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T09:52:05.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.732+0000 7f47008d3640 1 Processor -- start 2023-08-25T09:52:05.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.732+0000 7f47008d3640 1 -- start start 2023-08-25T09:52:05.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f47008d3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46fc15ae50 0x7f46fc0c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f47008d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f47008d3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc0cc5f0 con 0x7f46fc0cdc20 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f47008d3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc070650 con 0x7f46fc15ae50 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f47008d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc070930 con 0x7f46fc069f60 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f46fa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46fc15ae50 0x7f46fc0c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f46fa59c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46fc15ae50 0x7f46fc0c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:35996/0 (socket says 172.21.15.70:35996) 2023-08-25T09:52:05.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.734+0000 7f46fa59c640 1 -- 172.21.15.70:0/2183724561 learned_addr learned my addr 172.21.15.70:0/2183724561 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:05.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.735+0000 7f46fad9d640 1 --2- 172.21.15.70:0/2183724561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.735+0000 7f46fa59c640 1 -- 172.21.15.70:0/2183724561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 msgr2=0x7f46fc0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.735+0000 7f46fa59c640 1 --2- 172.21.15.70:0/2183724561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.735+0000 7f46fa59c640 1 -- 172.21.15.70:0/2183724561 >> v1:172.21.15.155:6789/0 conn(0x7f46fc0cdc20 legacy=0x7f46fc069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.735+0000 7f46fa59c640 1 -- 172.21.15.70:0/2183724561 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc070c10 con 0x7f46fc15ae50 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.736+0000 7f46fa59c640 1 --2- 172.21.15.70:0/2183724561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0c2110 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009e50 tx=0x7f46ec02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6105b62dfadaba server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.736+0000 7f46f959a640 1 -- 172.21.15.70:0/2183724561 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec02cb00 con 0x7f46fc15ae50 2023-08-25T09:52:05.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.736+0000 7f46f959a640 1 -- 172.21.15.70:0/2183724561 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ec02cc60 con 0x7f46fc15ae50 2023-08-25T09:52:05.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.736+0000 7f46f959a640 1 -- 172.21.15.70:0/2183724561 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0343f0 con 0x7f46fc15ae50 2023-08-25T09:52:05.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 -- 172.21.15.70:0/2183724561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 msgr2=0x7f46fc0c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 --2- 172.21.15.70:0/2183724561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0c2110 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009e50 tx=0x7f46ec02bf60 comp rx=0 tx=0).stop 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 -- 172.21.15.70:0/2183724561 shutdown_connections 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 --2- 172.21.15.70:0/2183724561 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 --2- 172.21.15.70:0/2183724561 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0c2110 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 -- 172.21.15.70:0/2183724561 >> 172.21.15.70:0/2183724561 conn(0x7f46fc0cb0d0 msgr2=0x7f46fc0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 -- 172.21.15.70:0/2183724561 shutdown_connections 2023-08-25T09:52:05.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.737+0000 7f47008d3640 1 -- 172.21.15.70:0/2183724561 wait complete. 2023-08-25T09:52:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.739+0000 7f47008d3640 1 Processor -- start 2023-08-25T09:52:05.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.739+0000 7f47008d3640 1 -- start start 2023-08-25T09:52:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f47008d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f47008d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46fc0cdc20 0x7f46fc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f47008d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f47008d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15e260 con 0x7f46fc15ae50 2023-08-25T09:52:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f47008d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc15c530 con 0x7f46fc0cdc20 2023-08-25T09:52:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f47008d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0ad590 con 0x7f46fc069f60 2023-08-25T09:52:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f46fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f46fad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36012/0 (socket says 172.21.15.70:36012) 2023-08-25T09:52:05.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.740+0000 7f46fad9d640 1 -- 172.21.15.70:0/4173761753 learned_addr learned my addr 172.21.15.70:0/4173761753 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:05.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fa59c640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46f9d9b640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46fc0cdc20 0x7f46fc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fad9d640 1 -- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 msgr2=0x7f46fc0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fad9d640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fad9d640 1 -- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46fc0cdc20 msgr2=0x7f46fc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fad9d640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46fc0cdc20 0x7f46fc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fad9d640 1 -- 172.21.15.70:0/4173761753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec009b30 con 0x7f46fc15ae50 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46f9d9b640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46fc0cdc20 0x7f46fc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46fad9d640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0ca1f0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f46f400d5a0 tx=0x7f46f400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:05.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f40125f0 con 0x7f46fc15ae50 2023-08-25T09:52:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f4012750 con 0x7f46fc15ae50 2023-08-25T09:52:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f4012ae0 con 0x7f46fc15ae50 2023-08-25T09:52:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.741+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc15df80 con 0x7f46fc15ae50 2023-08-25T09:52:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.742+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc15cab0 con 0x7f46fc15ae50 2023-08-25T09:52:05.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.743+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc066fb0 con 0x7f46fc15ae50 2023-08-25T09:52:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.743+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f46f4012c40 con 0x7f46fc15ae50 2023-08-25T09:52:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.744+0000 7f46f2ffd640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f46dc06a050 0x7f46dc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.744+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46f4088a50 con 0x7f46fc15ae50 2023-08-25T09:52:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.745+0000 7f46fa59c640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f46dc06a050 0x7f46dc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:05.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.745+0000 7f46fa59c640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f46dc06a050 0x7f46dc06c510 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f46ec035000 tx=0x7f46ec030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:05.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.749+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f46f40540b0 con 0x7f46fc15ae50 2023-08-25T09:52:05.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:05.970+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f46fc04dc90 con 0x7f46fc15ae50 2023-08-25T09:52:06.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.667+0000 7f46f2ffd640 1 -- 172.21.15.70:0/4173761753 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7f46f40576d0 con 0x7f46fc15ae50 2023-08-25T09:52:06.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f46dc06a050 msgr2=0x7f46dc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:06.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f46dc06a050 0x7f46dc06c510 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f46ec035000 tx=0x7f46ec030000 comp rx=0 tx=0).stop 2023-08-25T09:52:06.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 msgr2=0x7f46fc0ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:06.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0ca1f0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f46f400d5a0 tx=0x7f46f400da60 comp rx=0 tx=0).stop 2023-08-25T09:52:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 shutdown_connections 2023-08-25T09:52:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f46dc06a050 0x7f46dc06c510 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46fc15ae50 0x7f46fc0ca1f0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46fc0cdc20 0x7f46fc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 --2- 172.21.15.70:0/4173761753 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46fc069f60 0x7f46fc0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 >> 172.21.15.70:0/4173761753 conn(0x7f46fc0cb0d0 msgr2=0x7f46fc0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 shutdown_connections 2023-08-25T09:52:06.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.671+0000 7f47008d3640 1 -- 172.21.15.70:0/4173761753 wait complete. 2023-08-25T09:52:06.696 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T09:52:06.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T09:52:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.917+0000 7f94ee4b8640 1 Processor -- start 2023-08-25T09:52:06.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.917+0000 7f94ee4b8640 1 -- start start 2023-08-25T09:52:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.918+0000 7f94ee4b8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f94e80cd850 0x7f94e80cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.918+0000 7f94ee4b8640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94e80b8510 0x7f94e80b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:06.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.918+0000 7f94ee4b8640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e80ce170 con 0x7f94e815cfe0 2023-08-25T09:52:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.918+0000 7f94ee4b8640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e80d1310 con 0x7f94e80b8510 2023-08-25T09:52:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.918+0000 7f94ee4b8640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e80d1500 con 0x7f94e80cd850 2023-08-25T09:52:06.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94e80b8510 0x7f94e80b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94e80b8510 0x7f94e80b5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36020/0 (socket says 172.21.15.70:36020) 2023-08-25T09:52:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 -- 172.21.15.70:0/2350290801 learned_addr learned my addr 172.21.15.70:0/2350290801 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94e7fff640 1 --2- 172.21.15.70:0/2350290801 >> v2:172.21.15.155:3300/0 conn(0x7f94e80cd850 0x7f94e80cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:06.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 -- 172.21.15.70:0/2350290801 >> v2:172.21.15.155:3300/0 conn(0x7f94e80cd850 msgr2=0x7f94e80cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 --2- 172.21.15.70:0/2350290801 >> v2:172.21.15.155:3300/0 conn(0x7f94e80cd850 0x7f94e80cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.925 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 -- 172.21.15.70:0/2350290801 >> v1:172.21.15.155:6789/0 conn(0x7f94e815cfe0 legacy=0x7f94e80d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:06.926 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.919+0000 7f94eca55640 1 -- 172.21.15.70:0/2350290801 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80d17e0 con 0x7f94e80b8510 2023-08-25T09:52:06.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.920+0000 7f94eca55640 1 --2- 172.21.15.70:0/2350290801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80b8510 0x7f94e80b5cb0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f94e80b5690 tx=0x7f94e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebb0fe05b35493a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:06.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94e6ffd640 1 -- 172.21.15.70:0/2350290801 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0031640 con 0x7f94e80b8510 2023-08-25T09:52:06.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94e6ffd640 1 -- 172.21.15.70:0/2350290801 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e00317a0 con 0x7f94e80b8510 2023-08-25T09:52:06.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94e6ffd640 1 -- 172.21.15.70:0/2350290801 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0031b30 con 0x7f94e80b8510 2023-08-25T09:52:06.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 -- 172.21.15.70:0/2350290801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80b8510 msgr2=0x7f94e80b5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:06.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/2350290801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80b8510 0x7f94e80b5cb0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f94e80b5690 tx=0x7f94e002b8f0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 -- 172.21.15.70:0/2350290801 shutdown_connections 2023-08-25T09:52:06.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/2350290801 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e80b8510 0x7f94e80b5cb0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/2350290801 >> v2:172.21.15.155:3300/0 conn(0x7f94e80cd850 0x7f94e80cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 -- 172.21.15.70:0/2350290801 >> 172.21.15.70:0/2350290801 conn(0x7f94e8069850 msgr2=0x7f94e8069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:06.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.921+0000 7f94ee4b8640 1 -- 172.21.15.70:0/2350290801 shutdown_connections 2023-08-25T09:52:06.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.922+0000 7f94ee4b8640 1 -- 172.21.15.70:0/2350290801 wait complete. 2023-08-25T09:52:06.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.923+0000 7f94ee4b8640 1 Processor -- start 2023-08-25T09:52:06.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.923+0000 7f94ee4b8640 1 -- start start 2023-08-25T09:52:06.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.924+0000 7f94ee4b8640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80b8510 0x7f94e80c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:06.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.924+0000 7f94ee4b8640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94e80cd850 0x7f94e80cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:06.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.924+0000 7f94ee4b8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 0x7f94e80b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:06.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.924+0000 7f94ee4b8640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e8154890 con 0x7f94e815cfe0 2023-08-25T09:52:06.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.924+0000 7f94ee4b8640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80ccef0 con 0x7f94e80b8510 2023-08-25T09:52:06.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.924+0000 7f94ee4b8640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e80b33c0 con 0x7f94e80cd850 2023-08-25T09:52:06.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 0x7f94e80b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:06.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 0x7f94e80b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36034/0 (socket says 172.21.15.70:36034) 2023-08-25T09:52:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 -- 172.21.15.70:0/1352848509 learned_addr learned my addr 172.21.15.70:0/1352848509 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94e7fff640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80b8510 0x7f94e80c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94e77fe640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94e80cd850 0x7f94e80cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:06.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 -- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94e80cd850 msgr2=0x7f94e80cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94e80cd850 0x7f94e80cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 -- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80b8510 msgr2=0x7f94e80c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80b8510 0x7f94e80c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:06.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 -- 172.21.15.70:0/1352848509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00094b0 con 0x7f94e815cfe0 2023-08-25T09:52:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94e7fff640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80b8510 0x7f94e80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.925+0000 7f94eca55640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 0x7f94e80b2570 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f94e0034040 tx=0x7f94e0004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:06.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.926+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0039b70 con 0x7f94e815cfe0 2023-08-25T09:52:06.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.926+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e003e070 con 0x7f94e815cfe0 2023-08-25T09:52:06.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.926+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0045480 con 0x7f94e815cfe0 2023-08-25T09:52:06.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.926+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e80c7750 con 0x7f94e815cfe0 2023-08-25T09:52:06.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.926+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e80c7d10 con 0x7f94e815cfe0 2023-08-25T09:52:06.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.927+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e806a700 con 0x7f94e815cfe0 2023-08-25T09:52:06.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.928+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f94e002f050 con 0x7f94e815cfe0 2023-08-25T09:52:06.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.929+0000 7f94e4ff9640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94c806a080 0x7f94c806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.929+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94e0040070 con 0x7f94e815cfe0 2023-08-25T09:52:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.929+0000 7f94e7fff640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94c806a080 0x7f94c806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.930+0000 7f94e7fff640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94c806a080 0x7f94c806c540 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f94d8007330 tx=0x7f94d800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:06.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:06.933+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94e0075cf0 con 0x7f94e815cfe0 2023-08-25T09:52:07.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.162+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94e8066fb0 con 0x7f94e815cfe0 2023-08-25T09:52:07.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.673+0000 7f94e4ff9640 1 -- 172.21.15.70:0/1352848509 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7f94e0079310 con 0x7f94e815cfe0 2023-08-25T09:52:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.676+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94c806a080 msgr2=0x7f94c806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.676+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94c806a080 0x7f94c806c540 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f94d8007330 tx=0x7f94d800b040 comp rx=0 tx=0).stop 2023-08-25T09:52:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.676+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 msgr2=0x7f94e80b2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:07.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.676+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 0x7f94e80b2570 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f94e0034040 tx=0x7f94e0004710 comp rx=0 tx=0).stop 2023-08-25T09:52:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 shutdown_connections 2023-08-25T09:52:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f94c806a080 0x7f94c806c540 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94e815cfe0 0x7f94e80b2570 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94e80cd850 0x7f94e80cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 --2- 172.21.15.70:0/1352848509 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94e80b8510 0x7f94e80c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 >> 172.21.15.70:0/1352848509 conn(0x7f94e8069850 msgr2=0x7f94e8069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 shutdown_connections 2023-08-25T09:52:07.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.677+0000 7f94ee4b8640 1 -- 172.21.15.70:0/1352848509 wait complete. 2023-08-25T09:52:07.701 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T09:52:07.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:52:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.906+0000 7f671c755640 1 Processor -- start 2023-08-25T09:52:07.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.906+0000 7f671c755640 1 -- start start 2023-08-25T09:52:07.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671c755640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f671415e830 0x7f671406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:07.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671c755640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f671406b2b0 0x7f671415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:07.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671c755640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67140b1b60 con 0x7f671415c6d0 2023-08-25T09:52:07.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671c755640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671406bd80 con 0x7f671406b2b0 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671c755640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f671406b7c0 con 0x7f671415e830 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671acf2640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f671415c6d0 0x7f67140b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35302/0 (socket says 172.21.15.70:35302) 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671acf2640 1 -- 172.21.15.70:0/1357867580 learned_addr learned my addr 172.21.15.70:0/1357867580 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f6719cf0640 1 --2- 172.21.15.70:0/1357867580 >> v2:172.21.15.70:3300/0 conn(0x7f671406b2b0 0x7f671415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.908+0000 7f671a4f1640 1 --2- 172.21.15.70:0/1357867580 >> v2:172.21.15.155:3300/0 conn(0x7f671415e830 0x7f671406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.909+0000 7f67194ef640 1 -- 172.21.15.70:0/1357867580 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233646414 0 0) 0x7f67140b1b60 con 0x7f671415c6d0 2023-08-25T09:52:07.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.909+0000 7f6719cf0640 1 -- 172.21.15.70:0/1357867580 >> v2:172.21.15.155:3300/0 conn(0x7f671415e830 msgr2=0x7f671406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:07.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.909+0000 7f6719cf0640 1 --2- 172.21.15.70:0/1357867580 >> v2:172.21.15.155:3300/0 conn(0x7f671415e830 0x7f671406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.909+0000 7f6719cf0640 1 -- 172.21.15.70:0/1357867580 >> v1:172.21.15.70:6789/0 conn(0x7f671415c6d0 legacy=0x7f67140b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:07.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.909+0000 7f6719cf0640 1 -- 172.21.15.70:0/1357867580 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671406baa0 con 0x7f671406b2b0 2023-08-25T09:52:07.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.910+0000 7f6719cf0640 1 --2- 172.21.15.70:0/1357867580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671415c190 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f671406bf10 tx=0x7f670402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ff5b8c91ab602 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:07.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.910+0000 7f67194ef640 1 -- 172.21.15.70:0/1357867580 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031830 con 0x7f671406b2b0 2023-08-25T09:52:07.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.910+0000 7f67194ef640 1 -- 172.21.15.70:0/1357867580 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6704031990 con 0x7f671406b2b0 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.910+0000 7f67194ef640 1 -- 172.21.15.70:0/1357867580 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6704031d20 con 0x7f671406b2b0 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 -- 172.21.15.70:0/1357867580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 msgr2=0x7f671415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 --2- 172.21.15.70:0/1357867580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671415c190 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f671406bf10 tx=0x7f670402b8d0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 -- 172.21.15.70:0/1357867580 shutdown_connections 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 --2- 172.21.15.70:0/1357867580 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671415c190 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 --2- 172.21.15.70:0/1357867580 >> v2:172.21.15.155:3300/0 conn(0x7f671415e830 0x7f671406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 -- 172.21.15.70:0/1357867580 >> 172.21.15.70:0/1357867580 conn(0x7f6714069850 msgr2=0x7f6714069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:07.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 -- 172.21.15.70:0/1357867580 shutdown_connections 2023-08-25T09:52:07.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.911+0000 7f671c755640 1 -- 172.21.15.70:0/1357867580 wait complete. 2023-08-25T09:52:07.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.913+0000 7f671c755640 1 Processor -- start 2023-08-25T09:52:07.917 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.913+0000 7f671c755640 1 -- start start 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671c755640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671414d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671c755640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671415c6d0 0x7f67140c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671c755640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671415e830 0x7f671414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671c755640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671414d940 con 0x7f671406b2b0 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671c755640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671414af00 con 0x7f671415c6d0 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671c755640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f671414b1e0 con 0x7f671415e830 2023-08-25T09:52:07.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671414d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671414d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36062/0 (socket says 172.21.15.70:36062) 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 -- 172.21.15.70:0/3800715294 learned_addr learned my addr 172.21.15.70:0/3800715294 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f6719cf0640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671415c6d0 0x7f67140c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 -- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671415e830 msgr2=0x7f671414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671415e830 0x7f671414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 -- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671415c6d0 msgr2=0x7f67140c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:07.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671415c6d0 0x7f67140c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.914+0000 7f671a4f1640 1 -- 172.21.15.70:0/3800715294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009400 con 0x7f671406b2b0 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.915+0000 7f671a4f1640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671414d400 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f671000d5f0 tx=0x7f671000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.915+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67100125f0 con 0x7f671406b2b0 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.915+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710012750 con 0x7f671406b2b0 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.915+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710012b00 con 0x7f671406b2b0 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.915+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671414b4c0 con 0x7f671406b2b0 2023-08-25T09:52:07.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.915+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671414ba80 con 0x7f671406b2b0 2023-08-25T09:52:07.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.916+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671404e420 con 0x7f671406b2b0 2023-08-25T09:52:07.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.917+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6710012c60 con 0x7f671406b2b0 2023-08-25T09:52:07.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.918+0000 7f670affd640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66f806a050 0x7f66f806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:07.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.918+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6710088af0 con 0x7f671406b2b0 2023-08-25T09:52:07.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.918+0000 7f6719cf0640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66f806a050 0x7f66f806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:07.922 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.919+0000 7f6719cf0640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66f806a050 0x7f66f806c510 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6704036000 tx=0x7f6704008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:07.924 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:07.922+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6710054140 con 0x7f671406b2b0 2023-08-25T09:52:08.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.137+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671406c620 con 0x7f671406b2b0 2023-08-25T09:52:08.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.139+0000 7f670affd640 1 -- 172.21.15.70:0/3800715294 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f6710057760 con 0x7f671406b2b0 2023-08-25T09:52:08.142 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:08.143 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:52:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:52:08.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66f806a050 msgr2=0x7f66f806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66f806a050 0x7f66f806c510 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6704036000 tx=0x7f6704008030 comp rx=0 tx=0).stop 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 msgr2=0x7f671414d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671414d400 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f671000d5f0 tx=0x7f671000dab0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 shutdown_connections 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f66f806a050 0x7f66f806c510 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f671415e830 0x7f671414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.145 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f671415c6d0 0x7f67140c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 --2- 172.21.15.70:0/3800715294 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f671406b2b0 0x7f671414d400 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 >> 172.21.15.70:0/3800715294 conn(0x7f6714069850 msgr2=0x7f671415dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:08.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 shutdown_connections 2023-08-25T09:52:08.146 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.141+0000 7f671c755640 1 -- 172.21.15.70:0/3800715294 wait complete. 2023-08-25T09:52:08.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.353+0000 7f78cfc1f640 1 Processor -- start 2023-08-25T09:52:08.356 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.353+0000 7f78cfc1f640 1 -- start start 2023-08-25T09:52:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78cfc1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78c815ab70 0x7f78c806d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78cfc1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c806ae60 0x7f78c8072820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78cfc1f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c806cf90 con 0x7f78c806a910 2023-08-25T09:52:08.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78cfc1f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c80ae780 con 0x7f78c815ab70 2023-08-25T09:52:08.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78cfc1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80aea60 con 0x7f78c806ae60 2023-08-25T09:52:08.358 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78cd9bb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f78c815ab70 0x7f78c806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78ce1bc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f78c806a910 0x7f78c806fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35314/0 (socket says 172.21.15.70:35314) 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.355+0000 7f78ce1bc640 1 -- 172.21.15.70:0/1650655629 learned_addr learned my addr 172.21.15.70:0/1650655629 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.356+0000 7f78cd1ba640 1 --2- 172.21.15.70:0/1650655629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c806ae60 0x7f78c8072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.356+0000 7f78cd9bb640 1 -- 172.21.15.70:0/1650655629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c806ae60 msgr2=0x7f78c8072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.356+0000 7f78cd9bb640 1 --2- 172.21.15.70:0/1650655629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c806ae60 0x7f78c8072820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.356+0000 7f78cd9bb640 1 -- 172.21.15.70:0/1650655629 >> v1:172.21.15.70:6789/0 conn(0x7f78c806a910 legacy=0x7f78c806fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.356+0000 7f78cd9bb640 1 -- 172.21.15.70:0/1650655629 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c806cc20 con 0x7f78c815ab70 2023-08-25T09:52:08.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.356+0000 7f78cc9b9640 1 -- 172.21.15.70:0/1650655629 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715595348 0 0) 0x7f78c806cf90 con 0x7f78c806a910 2023-08-25T09:52:08.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.357+0000 7f78cd9bb640 1 --2- 172.21.15.70:0/1650655629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c815ab70 0x7f78c806d420 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f78c40097a0 tx=0x7f78c402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0be8328b432f967 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.357+0000 7f78cc9b9640 1 -- 172.21.15.70:0/1650655629 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c402d040 con 0x7f78c815ab70 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.357+0000 7f78cc9b9640 1 -- 172.21.15.70:0/1650655629 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4031810 con 0x7f78c815ab70 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.357+0000 7f78cc9b9640 1 -- 172.21.15.70:0/1650655629 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4031ba0 con 0x7f78c815ab70 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 -- 172.21.15.70:0/1650655629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c815ab70 msgr2=0x7f78c806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/1650655629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c815ab70 0x7f78c806d420 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f78c40097a0 tx=0x7f78c402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 -- 172.21.15.70:0/1650655629 shutdown_connections 2023-08-25T09:52:08.361 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/1650655629 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c806ae60 0x7f78c8072820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/1650655629 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c815ab70 0x7f78c806d420 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 -- 172.21.15.70:0/1650655629 >> 172.21.15.70:0/1650655629 conn(0x7f78c80cf7e0 msgr2=0x7f78c80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 -- 172.21.15.70:0/1650655629 shutdown_connections 2023-08-25T09:52:08.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.358+0000 7f78cfc1f640 1 -- 172.21.15.70:0/1650655629 wait complete. 2023-08-25T09:52:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.360+0000 7f78cfc1f640 1 Processor -- start 2023-08-25T09:52:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.360+0000 7f78cfc1f640 1 -- start start 2023-08-25T09:52:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cfc1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 0x7f78c8155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cfc1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c806ae60 0x7f78c80b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cfc1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c815ab70 0x7f78c80b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cfc1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80b0890 con 0x7f78c806a910 2023-08-25T09:52:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cfc1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81f5c90 con 0x7f78c806ae60 2023-08-25T09:52:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cfc1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c81f5ee0 con 0x7f78c815ab70 2023-08-25T09:52:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 0x7f78c8155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 0x7f78c8155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36082/0 (socket says 172.21.15.70:36082) 2023-08-25T09:52:08.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 -- 172.21.15.70:0/2253806435 learned_addr learned my addr 172.21.15.70:0/2253806435 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:08.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 -- 172.21.15.70:0/2253806435 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c815ab70 msgr2=0x7f78c80b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c815ab70 0x7f78c80b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 -- 172.21.15.70:0/2253806435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c806ae60 msgr2=0x7f78c80b0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c806ae60 0x7f78c80b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.361+0000 7f78cd9bb640 1 -- 172.21.15.70:0/2253806435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c4009400 con 0x7f78c806a910 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.362+0000 7f78cd9bb640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 0x7f78c8155440 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f78c4000c00 tx=0x7f78c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.362+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c403ad60 con 0x7f78c806a910 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.362+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c40443f0 con 0x7f78c806a910 2023-08-25T09:52:08.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.362+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c40447a0 con 0x7f78c806a910 2023-08-25T09:52:08.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.362+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c81f61c0 con 0x7f78c806a910 2023-08-25T09:52:08.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.362+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c81f6780 con 0x7f78c806a910 2023-08-25T09:52:08.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.363+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c81f6970 con 0x7f78c806a910 2023-08-25T09:52:08.367 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.364+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f78c404d450 con 0x7f78c806a910 2023-08-25T09:52:08.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.365+0000 7f78be7fc640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78ac06a080 0x7f78ac06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.365+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f78c40aa730 con 0x7f78c806a910 2023-08-25T09:52:08.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.365+0000 7f78cd1ba640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78ac06a080 0x7f78ac06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.366+0000 7f78cd1ba640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78ac06a080 0x7f78ac06c540 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f78b8006fd0 tx=0x7f78b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:08.371 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.369+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f78c4075eb0 con 0x7f78c806a910 2023-08-25T09:52:08.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.562+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78c804e420 con 0x7f78c806a910 2023-08-25T09:52:08.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.564+0000 7f78be7fc640 1 -- 172.21.15.70:0/2253806435 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+18313 (secure 0 0 0) 0x7f78c40794d0 con 0x7f78c806a910 2023-08-25T09:52:08.567 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:08.567 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":198,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:05.588988+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"198","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:08.569 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78ac06a080 msgr2=0x7f78ac06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78ac06a080 0x7f78ac06c540 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f78b8006fd0 tx=0x7f78b8008040 comp rx=0 tx=0).stop 2023-08-25T09:52:08.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 msgr2=0x7f78c8155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 0x7f78c8155440 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f78c4000c00 tx=0x7f78c40046c0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 shutdown_connections 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f78ac06a080 0x7f78ac06c540 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f78c815ab70 0x7f78c80b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f78c806ae60 0x7f78c80b0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 --2- 172.21.15.70:0/2253806435 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f78c806a910 0x7f78c8155440 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 >> 172.21.15.70:0/2253806435 conn(0x7f78c80cf7e0 msgr2=0x7f78c806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 shutdown_connections 2023-08-25T09:52:08.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.567+0000 7f78cfc1f640 1 -- 172.21.15.70:0/2253806435 wait complete. 2023-08-25T09:52:08.588 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T09:52:08.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:52:08.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.803+0000 7f22cf434640 1 Processor -- start 2023-08-25T09:52:08.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.803+0000 7f22cf434640 1 -- start start 2023-08-25T09:52:08.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.804+0000 7f22cf434640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 0x7f22c80b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.804+0000 7f22cf434640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f22c80cced0 0x7f22c80b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.804+0000 7f22cf434640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22c80d03c0 con 0x7f22c81608d0 2023-08-25T09:52:08.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.804+0000 7f22cf434640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22c80b3240 con 0x7f22c80cced0 2023-08-25T09:52:08.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.804+0000 7f22cf434640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80cde20 con 0x7f22c81614c0 2023-08-25T09:52:08.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.805+0000 7f22cd9d1640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f22c81608d0 0x7f22c815c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:35328/0 (socket says 172.21.15.70:35328) 2023-08-25T09:52:08.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.805+0000 7f22cd9d1640 1 -- 172.21.15.70:0/2919486004 learned_addr learned my addr 172.21.15.70:0/2919486004 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:08.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.805+0000 7f22cc9cf640 1 --2- 172.21.15.70:0/2919486004 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cced0 0x7f22c80b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.805+0000 7f22cd1d0640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 0x7f22c80b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.805+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836481014 0 0) 0x7f22c80d03c0 con 0x7f22c81608d0 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.805+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b0003520 con 0x7f22c81608d0 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436099040 0 0) 0x7f22b0003520 con 0x7f22c81608d0 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 msgr2=0x7f22c80b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 0x7f22c80b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cced0 msgr2=0x7f22c80b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2919486004 >> v2:172.21.15.155:3300/0 conn(0x7f22c80cced0 0x7f22c80b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80ce100 con 0x7f22c81608d0 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f22c0001670 con 0x7f22c81608d0 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 >> v1:172.21.15.70:6789/0 conn(0x7f22c81608d0 legacy=0x7f22c815c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 0x7f22b0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22b0003ef0 0x7f22b00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80b3240 con 0x7f22b0004d00 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80cde20 con 0x7f22b0003ef0 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b000b9b0 con 0x7f22c81614c0 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.806+0000 7f22cd9d1640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cd1d0640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 0x7f22b0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cc9cf640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22b0003ef0 0x7f22b00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cd9d1640 1 -- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 msgr2=0x7f22b0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:08.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cd9d1640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c81614c0 0x7f22b0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cd9d1640 1 -- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22b0003ef0 msgr2=0x7f22b00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:08.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cd9d1640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22b0003ef0 0x7f22b00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cd9d1640 1 -- 172.21.15.70:0/2919486004 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0003520 con 0x7f22b0004d00 2023-08-25T09:52:08.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.807+0000 7f22cc9cf640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22b0003ef0 0x7f22b00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:08.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.808+0000 7f22cd9d1640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22b0005180 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f22c8160d60 tx=0x7f22b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738cd65ef86a533b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:08.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.808+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0005ad0 con 0x7f22b0004d00 2023-08-25T09:52:08.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.808+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c0005c30 con 0x7f22b0004d00 2023-08-25T09:52:08.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.808+0000 7f22c7fff640 1 -- 172.21.15.70:0/2919486004 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c002c890 con 0x7f22b0004d00 2023-08-25T09:52:08.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cf434640 1 -- 172.21.15.70:0/2919486004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 msgr2=0x7f22b0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cf434640 1 --2- 172.21.15.70:0/2919486004 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22b0005180 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f22c8160d60 tx=0x7f22b00057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cd9d1640 1 -- 172.21.15.70:0/2919486004 reap_dead start 2023-08-25T09:52:08.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cf434640 1 -- 172.21.15.70:0/2919486004 shutdown_connections 2023-08-25T09:52:08.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cf434640 1 -- 172.21.15.70:0/2919486004 >> 172.21.15.70:0/2919486004 conn(0x7f22c80cd300 msgr2=0x7f22c80cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cf434640 1 -- 172.21.15.70:0/2919486004 shutdown_connections 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.809+0000 7f22cf434640 1 -- 172.21.15.70:0/2919486004 wait complete. 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.811+0000 7f22cf434640 1 Processor -- start 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.811+0000 7f22cf434640 1 -- start start 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cf434640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22c81f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cf434640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80b50c0 0x7f22c80ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cf434640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c80b59c0 0x7f22c80ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cf434640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c81f3430 con 0x7f22b0004d00 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cf434640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80ac880 con 0x7f22c80b50c0 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cf434640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c80acad0 con 0x7f22c80b59c0 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cd1d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22c81f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cd1d0640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22c81f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:36098/0 (socket says 172.21.15.70:36098) 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cd1d0640 1 -- 172.21.15.70:0/3021665852 learned_addr learned my addr 172.21.15.70:0/3021665852 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:08.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.812+0000 7f22cc9cf640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80b50c0 0x7f22c80ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd9d1640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c80b59c0 0x7f22c80ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd1d0640 1 -- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c80b59c0 msgr2=0x7f22c80ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd1d0640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c80b59c0 0x7f22c80ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd1d0640 1 -- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80b50c0 msgr2=0x7f22c80ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:08.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd1d0640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80b50c0 0x7f22c80ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:08.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd1d0640 1 -- 172.21.15.70:0/3021665852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c80acdb0 con 0x7f22b0004d00 2023-08-25T09:52:08.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cd1d0640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22c81f2ef0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f22b800f350 tx=0x7f22b800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b8017600 con 0x7f22b0004d00 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b8011040 con 0x7f22b0004d00 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b80179c0 con 0x7f22b0004d00 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c80ad090 con 0x7f22b0004d00 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.813+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c80ad650 con 0x7f22b0004d00 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.814+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c804e420 con 0x7f22b0004d00 2023-08-25T09:52:08.821 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.815+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f22b8017b20 con 0x7f22b0004d00 2023-08-25T09:52:08.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.816+0000 7f22c5ffb640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22b406a050 0x7f22b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:08.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.816+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22b8088cc0 con 0x7f22b0004d00 2023-08-25T09:52:08.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.816+0000 7f22cc9cf640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22b406a050 0x7f22b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:08.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.817+0000 7f22cc9cf640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22b406a050 0x7f22b406c510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cae30 tx=0x7f22bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:08.822 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:08.820+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22b808f080 con 0x7f22b0004d00 2023-08-25T09:52:09.009 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.006+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22c80d0050 con 0x7f22b406a050 2023-08-25T09:52:09.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.030+0000 7f22c5ffb640 1 -- 172.21.15.70:0/3021665852 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297781 (secure 0 0 0) 0x7f22bc0129b0 con 0x7f22b406a050 2023-08-25T09:52:09.033 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:09.047 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:52:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.034+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22b406a050 msgr2=0x7f22b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.034+0000 7f22cf434640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22b406a050 0x7f22b406c510 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f22c80cae30 tx=0x7f22bc008040 comp rx=0 tx=0).stop 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 msgr2=0x7f22c81f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22c81f2ef0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f22b800f350 tx=0x7f22b800f810 comp rx=0 tx=0).stop 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 shutdown_connections 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f22b406a050 0x7f22b406c510 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f22c80b59c0 0x7f22c80ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f22c80b50c0 0x7f22c80ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.049 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 --2- 172.21.15.70:0/3021665852 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f22b0004d00 0x7f22c81f2ef0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 >> 172.21.15.70:0/3021665852 conn(0x7f22c80cd300 msgr2=0x7f22c80b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 shutdown_connections 2023-08-25T09:52:09.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.035+0000 7f22cf434640 1 -- 172.21.15.70:0/3021665852 wait complete. 2023-08-25T09:52:09.057 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":594,"stamp":"2023-08-25T09:52:07.592093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552163,"statfs":{"total":1600000000,"available":1589414948,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013135"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.598037+0000","last_change":"2023-08-25T09:52:03.598037+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.598037+0000","last_undegraded":"2023-08-25T09:52:03.598037+0000","last_fullsized":"2023-08-25T09:52:03.598037+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.599450+0000","last_change":"2023-08-25T09:52:03.599450+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.599450+0000","last_undegraded":"2023-08-25T09:52:03.599450+0000","last_fullsized":"2023-08-25T09:52:03.599450+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.600397+0000","last_change":"2023-08-25T09:52:03.600397+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.600397+0000","last_undegraded":"2023-08-25T09:52:03.600397+0000","last_fullsized":"2023-08-25T09:52:03.600397+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.603937+0000","last_change":"2023-08-25T09:52:03.603937+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.603937+0000","last_undegraded":"2023-08-25T09:52:03.603937+0000","last_fullsized":"2023-08-25T09:52:03.603937+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609415+0000","last_change":"2023-08-25T09:52:03.609415+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609415+0000","last_undegraded":"2023-08-25T09:52:03.609415+0000","last_fullsized":"2023-08-25T09:52:03.609415+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597169+0000","last_change":"2023-08-25T09:52:03.597169+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597169+0000","last_undegraded":"2023-08-25T09:52:03.597169+0000","last_fullsized":"2023-08-25T09:52:03.597169+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.601229+0000","last_change":"2023-08-25T09:52:03.601229+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.601229+0000","last_undegraded":"2023-08-25T09:52:03.601229+0000","last_fullsized":"2023-08-25T09:52:03.601229+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605774+0000","last_change":"2023-08-25T09:52:03.605774+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605774+0000","last_undegraded":"2023-08-25T09:52:03.605774+0000","last_fullsized":"2023-08-25T09:52:03.605774+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605247+0000","last_change":"2023-08-25T09:52:03.605247+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605247+0000","last_undegraded":"2023-08-25T09:52:03.605247+0000","last_fullsized":"2023-08-25T09:52:03.605247+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609019+0000","last_change":"2023-08-25T09:52:03.609019+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609019+0000","last_undegraded":"2023-08-25T09:52:03.609019+0000","last_fullsized":"2023-08-25T09:52:03.609019+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596405+0000","last_change":"2023-08-25T09:52:03.596405+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596405+0000","last_undegraded":"2023-08-25T09:52:03.596405+0000","last_fullsized":"2023-08-25T09:52:03.596405+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.603780+0000","last_change":"2023-08-25T09:52:03.603780+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.603780+0000","last_undegraded":"2023-08-25T09:52:03.603780+0000","last_fullsized":"2023-08-25T09:52:03.603780+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.598799+0000","last_change":"2023-08-25T09:52:03.598799+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.598799+0000","last_undegraded":"2023-08-25T09:52:03.598799+0000","last_fullsized":"2023-08-25T09:52:03.598799+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.611880+0000","last_change":"2023-08-25T09:52:03.611880+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.611880+0000","last_undegraded":"2023-08-25T09:52:03.611880+0000","last_fullsized":"2023-08-25T09:52:03.611880+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596692+0000","last_change":"2023-08-25T09:52:03.596692+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596692+0000","last_undegraded":"2023-08-25T09:52:03.596692+0000","last_fullsized":"2023-08-25T09:52:03.596692+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595634+0000","last_change":"2023-08-25T09:52:03.595634+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595634+0000","last_undegraded":"2023-08-25T09:52:03.595634+0000","last_fullsized":"2023-08-25T09:52:03.595634+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604783+0000","last_change":"2023-08-25T09:52:03.604783+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604783+0000","last_undegraded":"2023-08-25T09:52:03.604783+0000","last_fullsized":"2023-08-25T09:52:03.604783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604441+0000","last_change":"2023-08-25T09:52:03.604441+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604441+0000","last_undegraded":"2023-08-25T09:52:03.604441+0000","last_fullsized":"2023-08-25T09:52:03.604441+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597476+0000","last_change":"2023-08-25T09:52:03.597476+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597476+0000","last_undegraded":"2023-08-25T09:52:03.597476+0000","last_fullsized":"2023-08-25T09:52:03.597476+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607624+0000","last_change":"2023-08-25T09:52:03.607624+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607624+0000","last_undegraded":"2023-08-25T09:52:03.607624+0000","last_fullsized":"2023-08-25T09:52:03.607624+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597284+0000","last_change":"2023-08-25T09:52:03.597284+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597284+0000","last_undegraded":"2023-08-25T09:52:03.597284+0000","last_fullsized":"2023-08-25T09:52:03.597284+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595744+0000","last_change":"2023-08-25T09:52:03.595744+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595744+0000","last_undegraded":"2023-08-25T09:52:03.595744+0000","last_fullsized":"2023-08-25T09:52:03.595744+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607681+0000","last_change":"2023-08-25T09:52:03.607681+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607681+0000","last_undegraded":"2023-08-25T09:52:03.607681+0000","last_fullsized":"2023-08-25T09:52:03.607681+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596686+0000","last_change":"2023-08-25T09:52:03.596686+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596686+0000","last_undegraded":"2023-08-25T09:52:03.596686+0000","last_fullsized":"2023-08-25T09:52:03.596686+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596928+0000","last_change":"2023-08-25T09:52:03.596928+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596928+0000","last_undegraded":"2023-08-25T09:52:03.596928+0000","last_fullsized":"2023-08-25T09:52:03.596928+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.610788+0000","last_change":"2023-08-25T09:52:03.610788+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.610788+0000","last_undegraded":"2023-08-25T09:52:03.610788+0000","last_fullsized":"2023-08-25T09:52:03.610788+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.598589+0000","last_change":"2023-08-25T09:52:03.598589+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.598589+0000","last_undegraded":"2023-08-25T09:52:03.598589+0000","last_fullsized":"2023-08-25T09:52:03.598589+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597994+0000","last_change":"2023-08-25T09:52:03.597994+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597994+0000","last_undegraded":"2023-08-25T09:52:03.597994+0000","last_fullsized":"2023-08-25T09:52:03.597994+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607464+0000","last_change":"2023-08-25T09:52:03.607464+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607464+0000","last_undegraded":"2023-08-25T09:52:03.607464+0000","last_fullsized":"2023-08-25T09:52:03.607464+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594960+0000","last_change":"2023-08-25T09:52:03.594960+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594960+0000","last_undegraded":"2023-08-25T09:52:03.594960+0000","last_fullsized":"2023-08-25T09:52:03.594960+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.602630+0000","last_change":"2023-08-25T09:52:03.602630+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.602630+0000","last_undegraded":"2023-08-25T09:52:03.602630+0000","last_fullsized":"2023-08-25T09:52:03.602630+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597732+0000","last_change":"2023-08-25T09:52:03.597732+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597732+0000","last_undegraded":"2023-08-25T09:52:03.597732+0000","last_fullsized":"2023-08-25T09:52:03.597732+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597830+0000","last_change":"2023-08-25T09:52:03.597830+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597830+0000","last_undegraded":"2023-08-25T09:52:03.597830+0000","last_fullsized":"2023-08-25T09:52:03.597830+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.606094+0000","last_change":"2023-08-25T09:52:03.605509+0000","last_active":"2023-08-25T09:52:03.606094+0000","last_peered":"2023-08-25T09:52:03.606094+0000","last_clean":"2023-08-25T09:52:03.606094+0000","last_became_active":"2023-08-25T09:52:03.596779+0000","last_became_peered":"2023-08-25T09:52:03.596779+0000","last_unstale":"2023-08-25T09:52:03.606094+0000","last_undegraded":"2023-08-25T09:52:03.606094+0000","last_fullsized":"2023-08-25T09:52:03.606094+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:10:33.811352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595211+0000","last_change":"2023-08-25T09:52:03.595211+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595211+0000","last_undegraded":"2023-08-25T09:52:03.595211+0000","last_fullsized":"2023-08-25T09:52:03.595211+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.618472+0000","last_change":"2023-08-25T09:52:03.618376+0000","last_active":"2023-08-25T09:52:03.618472+0000","last_peered":"2023-08-25T09:52:03.618472+0000","last_clean":"2023-08-25T09:52:03.618472+0000","last_became_active":"2023-08-25T09:52:03.617343+0000","last_became_peered":"2023-08-25T09:52:03.617343+0000","last_unstale":"2023-08-25T09:52:03.618472+0000","last_undegraded":"2023-08-25T09:52:03.618472+0000","last_fullsized":"2023-08-25T09:52:03.618472+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:36.308598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607124+0000","last_change":"2023-08-25T09:52:03.607124+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607124+0000","last_undegraded":"2023-08-25T09:52:03.607124+0000","last_fullsized":"2023-08-25T09:52:03.607124+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615672+0000","last_change":"2023-08-25T09:52:03.615544+0000","last_active":"2023-08-25T09:52:03.615672+0000","last_peered":"2023-08-25T09:52:03.615672+0000","last_clean":"2023-08-25T09:52:03.615672+0000","last_became_active":"2023-08-25T09:52:03.615200+0000","last_became_peered":"2023-08-25T09:52:03.615200+0000","last_unstale":"2023-08-25T09:52:03.615672+0000","last_undegraded":"2023-08-25T09:52:03.615672+0000","last_fullsized":"2023-08-25T09:52:03.615672+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:55:24.277587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604724+0000","last_change":"2023-08-25T09:52:03.604724+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604724+0000","last_undegraded":"2023-08-25T09:52:03.604724+0000","last_fullsized":"2023-08-25T09:52:03.604724+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615770+0000","last_change":"2023-08-25T09:52:03.615665+0000","last_active":"2023-08-25T09:52:03.615770+0000","last_peered":"2023-08-25T09:52:03.615770+0000","last_clean":"2023-08-25T09:52:03.615770+0000","last_became_active":"2023-08-25T09:52:03.615398+0000","last_became_peered":"2023-08-25T09:52:03.615398+0000","last_unstale":"2023-08-25T09:52:03.615770+0000","last_undegraded":"2023-08-25T09:52:03.615770+0000","last_fullsized":"2023-08-25T09:52:03.615770+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:14.615337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595394+0000","last_change":"2023-08-25T09:52:03.595394+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595394+0000","last_undegraded":"2023-08-25T09:52:03.595394+0000","last_fullsized":"2023-08-25T09:52:03.595394+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.612841+0000","last_change":"2023-08-25T09:52:03.612691+0000","last_active":"2023-08-25T09:52:03.612841+0000","last_peered":"2023-08-25T09:52:03.612841+0000","last_clean":"2023-08-25T09:52:03.612841+0000","last_became_active":"2023-08-25T09:52:03.609712+0000","last_became_peered":"2023-08-25T09:52:03.609712+0000","last_unstale":"2023-08-25T09:52:03.612841+0000","last_undegraded":"2023-08-25T09:52:03.612841+0000","last_fullsized":"2023-08-25T09:52:03.612841+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:48:57.407979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594550+0000","last_change":"2023-08-25T09:52:03.594550+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594550+0000","last_undegraded":"2023-08-25T09:52:03.594550+0000","last_fullsized":"2023-08-25T09:52:03.594550+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.613629+0000","last_change":"2023-08-25T09:52:03.612785+0000","last_active":"2023-08-25T09:52:03.613629+0000","last_peered":"2023-08-25T09:52:03.613629+0000","last_clean":"2023-08-25T09:52:03.613629+0000","last_became_active":"2023-08-25T09:52:03.612129+0000","last_became_peered":"2023-08-25T09:52:03.612129+0000","last_unstale":"2023-08-25T09:52:03.613629+0000","last_undegraded":"2023-08-25T09:52:03.613629+0000","last_fullsized":"2023-08-25T09:52:03.613629+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:31.665821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604133+0000","last_change":"2023-08-25T09:52:03.604133+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604133+0000","last_undegraded":"2023-08-25T09:52:03.604133+0000","last_fullsized":"2023-08-25T09:52:03.604133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.618800+0000","last_change":"2023-08-25T09:52:03.618689+0000","last_active":"2023-08-25T09:52:03.618800+0000","last_peered":"2023-08-25T09:52:03.618800+0000","last_clean":"2023-08-25T09:52:03.618800+0000","last_became_active":"2023-08-25T09:52:03.617671+0000","last_became_peered":"2023-08-25T09:52:03.617671+0000","last_unstale":"2023-08-25T09:52:03.618800+0000","last_undegraded":"2023-08-25T09:52:03.618800+0000","last_fullsized":"2023-08-25T09:52:03.618800+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:52.014238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.606075+0000","last_change":"2023-08-25T09:52:03.606075+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.606075+0000","last_undegraded":"2023-08-25T09:52:03.606075+0000","last_fullsized":"2023-08-25T09:52:03.606075+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.604103+0000","last_change":"2023-08-25T09:52:03.603589+0000","last_active":"2023-08-25T09:52:03.604103+0000","last_peered":"2023-08-25T09:52:03.604103+0000","last_clean":"2023-08-25T09:52:03.604103+0000","last_became_active":"2023-08-25T09:52:03.602798+0000","last_became_peered":"2023-08-25T09:52:03.602798+0000","last_unstale":"2023-08-25T09:52:03.604103+0000","last_undegraded":"2023-08-25T09:52:03.604103+0000","last_fullsized":"2023-08-25T09:52:03.604103+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:06.723741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.612908+0000","last_change":"2023-08-25T09:52:03.612908+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.612908+0000","last_undegraded":"2023-08-25T09:52:03.612908+0000","last_fullsized":"2023-08-25T09:52:03.612908+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.619037+0000","last_change":"2023-08-25T09:52:03.618945+0000","last_active":"2023-08-25T09:52:03.619037+0000","last_peered":"2023-08-25T09:52:03.619037+0000","last_clean":"2023-08-25T09:52:03.619037+0000","last_became_active":"2023-08-25T09:52:03.618057+0000","last_became_peered":"2023-08-25T09:52:03.618057+0000","last_unstale":"2023-08-25T09:52:03.619037+0000","last_undegraded":"2023-08-25T09:52:03.619037+0000","last_fullsized":"2023-08-25T09:52:03.619037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:29.871817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594548+0000","last_change":"2023-08-25T09:52:03.594548+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594548+0000","last_undegraded":"2023-08-25T09:52:03.594548+0000","last_fullsized":"2023-08-25T09:52:03.594548+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.617181+0000","last_change":"2023-08-25T09:52:03.617112+0000","last_active":"2023-08-25T09:52:03.617181+0000","last_peered":"2023-08-25T09:52:03.617181+0000","last_clean":"2023-08-25T09:52:03.617181+0000","last_became_active":"2023-08-25T09:52:03.616870+0000","last_became_peered":"2023-08-25T09:52:03.616870+0000","last_unstale":"2023-08-25T09:52:03.617181+0000","last_undegraded":"2023-08-25T09:52:03.617181+0000","last_fullsized":"2023-08-25T09:52:03.617181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:17:30.422505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594730+0000","last_change":"2023-08-25T09:52:03.594730+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594730+0000","last_undegraded":"2023-08-25T09:52:03.594730+0000","last_fullsized":"2023-08-25T09:52:03.594730+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.597181+0000","last_change":"2023-08-25T09:52:03.596983+0000","last_active":"2023-08-25T09:52:03.597181+0000","last_peered":"2023-08-25T09:52:03.597181+0000","last_clean":"2023-08-25T09:52:03.597181+0000","last_became_active":"2023-08-25T09:52:03.596298+0000","last_became_peered":"2023-08-25T09:52:03.596298+0000","last_unstale":"2023-08-25T09:52:03.597181+0000","last_undegraded":"2023-08-25T09:52:03.597181+0000","last_fullsized":"2023-08-25T09:52:03.597181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:22.805922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594769+0000","last_change":"2023-08-25T09:52:03.594769+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594769+0000","last_undegraded":"2023-08-25T09:52:03.594769+0000","last_fullsized":"2023-08-25T09:52:03.594769+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615972+0000","last_change":"2023-08-25T09:52:03.614128+0000","last_active":"2023-08-25T09:52:03.615972+0000","last_peered":"2023-08-25T09:52:03.615972+0000","last_clean":"2023-08-25T09:52:03.615972+0000","last_became_active":"2023-08-25T09:52:03.610416+0000","last_became_peered":"2023-08-25T09:52:03.610416+0000","last_unstale":"2023-08-25T09:52:03.615972+0000","last_undegraded":"2023-08-25T09:52:03.615972+0000","last_fullsized":"2023-08-25T09:52:03.615972+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:31.051627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.591632+0000","last_change":"2023-08-25T09:52:03.591632+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.591632+0000","last_undegraded":"2023-08-25T09:52:03.591632+0000","last_fullsized":"2023-08-25T09:52:03.591632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.617647+0000","last_change":"2023-08-25T09:52:03.617559+0000","last_active":"2023-08-25T09:52:03.617647+0000","last_peered":"2023-08-25T09:52:03.617647+0000","last_clean":"2023-08-25T09:52:03.617647+0000","last_became_active":"2023-08-25T09:52:03.616663+0000","last_became_peered":"2023-08-25T09:52:03.616663+0000","last_unstale":"2023-08-25T09:52:03.617647+0000","last_undegraded":"2023-08-25T09:52:03.617647+0000","last_fullsized":"2023-08-25T09:52:03.617647+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:11.295939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.610240+0000","last_change":"2023-08-25T09:52:03.610240+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.610240+0000","last_undegraded":"2023-08-25T09:52:03.610240+0000","last_fullsized":"2023-08-25T09:52:03.610240+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.610090+0000","last_change":"2023-08-25T09:52:03.609849+0000","last_active":"2023-08-25T09:52:03.610090+0000","last_peered":"2023-08-25T09:52:03.610090+0000","last_clean":"2023-08-25T09:52:03.610090+0000","last_became_active":"2023-08-25T09:52:03.609198+0000","last_became_peered":"2023-08-25T09:52:03.609198+0000","last_unstale":"2023-08-25T09:52:03.610090+0000","last_undegraded":"2023-08-25T09:52:03.610090+0000","last_fullsized":"2023-08-25T09:52:03.610090+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:30.008208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.613991+0000","last_change":"2023-08-25T09:52:03.613991+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.613991+0000","last_undegraded":"2023-08-25T09:52:03.613991+0000","last_fullsized":"2023-08-25T09:52:03.613991+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.613308+0000","last_change":"2023-08-25T09:52:03.613206+0000","last_active":"2023-08-25T09:52:03.613308+0000","last_peered":"2023-08-25T09:52:03.613308+0000","last_clean":"2023-08-25T09:52:03.613308+0000","last_became_active":"2023-08-25T09:52:03.612871+0000","last_became_peered":"2023-08-25T09:52:03.612871+0000","last_unstale":"2023-08-25T09:52:03.613308+0000","last_undegraded":"2023-08-25T09:52:03.613308+0000","last_fullsized":"2023-08-25T09:52:03.613308+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:12.582747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609476+0000","last_change":"2023-08-25T09:52:03.609476+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609476+0000","last_undegraded":"2023-08-25T09:52:03.609476+0000","last_fullsized":"2023-08-25T09:52:03.609476+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.614627+0000","last_change":"2023-08-25T09:52:03.614440+0000","last_active":"2023-08-25T09:52:03.614627+0000","last_peered":"2023-08-25T09:52:03.614627+0000","last_clean":"2023-08-25T09:52:03.614627+0000","last_became_active":"2023-08-25T09:52:03.613170+0000","last_became_peered":"2023-08-25T09:52:03.613170+0000","last_unstale":"2023-08-25T09:52:03.614627+0000","last_undegraded":"2023-08-25T09:52:03.614627+0000","last_fullsized":"2023-08-25T09:52:03.614627+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:12:38.030116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604775+0000","last_change":"2023-08-25T09:52:03.604775+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604775+0000","last_undegraded":"2023-08-25T09:52:03.604775+0000","last_fullsized":"2023-08-25T09:52:03.604775+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.606882+0000","last_change":"2023-08-25T09:52:03.606748+0000","last_active":"2023-08-25T09:52:03.606882+0000","last_peered":"2023-08-25T09:52:03.606882+0000","last_clean":"2023-08-25T09:52:03.606882+0000","last_became_active":"2023-08-25T09:52:03.606384+0000","last_became_peered":"2023-08-25T09:52:03.606384+0000","last_unstale":"2023-08-25T09:52:03.606882+0000","last_undegraded":"2023-08-25T09:52:03.606882+0000","last_fullsized":"2023-08-25T09:52:03.606882+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:18.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605505+0000","last_change":"2023-08-25T09:52:03.605505+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605505+0000","last_undegraded":"2023-08-25T09:52:03.605505+0000","last_fullsized":"2023-08-25T09:52:03.605505+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.598266+0000","last_change":"2023-08-25T09:52:03.598008+0000","last_active":"2023-08-25T09:52:03.598266+0000","last_peered":"2023-08-25T09:52:03.598266+0000","last_clean":"2023-08-25T09:52:03.598266+0000","last_became_active":"2023-08-25T09:52:03.597171+0000","last_became_peered":"2023-08-25T09:52:03.597171+0000","last_unstale":"2023-08-25T09:52:03.598266+0000","last_undegraded":"2023-08-25T09:52:03.598266+0000","last_fullsized":"2023-08-25T09:52:03.598266+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:58:35.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.613359+0000","last_change":"2023-08-25T09:52:03.613359+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.613359+0000","last_undegraded":"2023-08-25T09:52:03.613359+0000","last_fullsized":"2023-08-25T09:52:03.613359+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.620006+0000","last_change":"2023-08-25T09:52:03.619922+0000","last_active":"2023-08-25T09:52:03.620006+0000","last_peered":"2023-08-25T09:52:03.620006+0000","last_clean":"2023-08-25T09:52:03.620006+0000","last_became_active":"2023-08-25T09:52:03.619288+0000","last_became_peered":"2023-08-25T09:52:03.619288+0000","last_unstale":"2023-08-25T09:52:03.620006+0000","last_undegraded":"2023-08-25T09:52:03.620006+0000","last_fullsized":"2023-08-25T09:52:03.620006+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:18.360502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.593490+0000","last_change":"2023-08-25T09:52:03.593490+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.593490+0000","last_undegraded":"2023-08-25T09:52:03.593490+0000","last_fullsized":"2023-08-25T09:52:03.593490+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615995+0000","last_change":"2023-08-25T09:52:03.614922+0000","last_active":"2023-08-25T09:52:03.615995+0000","last_peered":"2023-08-25T09:52:03.615995+0000","last_clean":"2023-08-25T09:52:03.615995+0000","last_became_active":"2023-08-25T09:52:03.614577+0000","last_became_peered":"2023-08-25T09:52:03.614577+0000","last_unstale":"2023-08-25T09:52:03.615995+0000","last_undegraded":"2023-08-25T09:52:03.615995+0000","last_fullsized":"2023-08-25T09:52:03.615995+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:24.261532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609814+0000","last_change":"2023-08-25T09:52:03.609814+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609814+0000","last_undegraded":"2023-08-25T09:52:03.609814+0000","last_fullsized":"2023-08-25T09:52:03.609814+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.614928+0000","last_change":"2023-08-25T09:52:03.614822+0000","last_active":"2023-08-25T09:52:03.614928+0000","last_peered":"2023-08-25T09:52:03.614928+0000","last_clean":"2023-08-25T09:52:03.614928+0000","last_became_active":"2023-08-25T09:52:03.613922+0000","last_became_peered":"2023-08-25T09:52:03.613922+0000","last_unstale":"2023-08-25T09:52:03.614928+0000","last_undegraded":"2023-08-25T09:52:03.614928+0000","last_fullsized":"2023-08-25T09:52:03.614928+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:02.268146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605404+0000","last_change":"2023-08-25T09:52:03.605404+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605404+0000","last_undegraded":"2023-08-25T09:52:03.605404+0000","last_fullsized":"2023-08-25T09:52:03.605404+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.612471+0000","last_change":"2023-08-25T09:52:03.610194+0000","last_active":"2023-08-25T09:52:03.612471+0000","last_peered":"2023-08-25T09:52:03.612471+0000","last_clean":"2023-08-25T09:52:03.612471+0000","last_became_active":"2023-08-25T09:52:03.609009+0000","last_became_peered":"2023-08-25T09:52:03.609009+0000","last_unstale":"2023-08-25T09:52:03.612471+0000","last_undegraded":"2023-08-25T09:52:03.612471+0000","last_fullsized":"2023-08-25T09:52:03.612471+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:46:32.529133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.606546+0000","last_change":"2023-08-25T09:52:03.606546+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.606546+0000","last_undegraded":"2023-08-25T09:52:03.606546+0000","last_fullsized":"2023-08-25T09:52:03.606546+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.608748+0000","last_change":"2023-08-25T09:52:03.608521+0000","last_active":"2023-08-25T09:52:03.608748+0000","last_peered":"2023-08-25T09:52:03.608748+0000","last_clean":"2023-08-25T09:52:03.608748+0000","last_became_active":"2023-08-25T09:52:03.605881+0000","last_became_peered":"2023-08-25T09:52:03.605881+0000","last_unstale":"2023-08-25T09:52:03.608748+0000","last_undegraded":"2023-08-25T09:52:03.608748+0000","last_fullsized":"2023-08-25T09:52:03.608748+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:31:19.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.593285+0000","last_change":"2023-08-25T09:52:03.593285+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.593285+0000","last_undegraded":"2023-08-25T09:52:03.593285+0000","last_fullsized":"2023-08-25T09:52:03.593285+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.614837+0000","last_change":"2023-08-25T09:52:03.614418+0000","last_active":"2023-08-25T09:52:03.614837+0000","last_peered":"2023-08-25T09:52:03.614837+0000","last_clean":"2023-08-25T09:52:03.614837+0000","last_became_active":"2023-08-25T09:52:03.614127+0000","last_became_peered":"2023-08-25T09:52:03.614127+0000","last_unstale":"2023-08-25T09:52:03.614837+0000","last_undegraded":"2023-08-25T09:52:03.614837+0000","last_fullsized":"2023-08-25T09:52:03.614837+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:20.246360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604894+0000","last_change":"2023-08-25T09:52:03.604894+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604894+0000","last_undegraded":"2023-08-25T09:52:03.604894+0000","last_fullsized":"2023-08-25T09:52:03.604894+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.619131+0000","last_change":"2023-08-25T09:52:03.619038+0000","last_active":"2023-08-25T09:52:03.619131+0000","last_peered":"2023-08-25T09:52:03.619131+0000","last_clean":"2023-08-25T09:52:03.619131+0000","last_became_active":"2023-08-25T09:52:03.618767+0000","last_became_peered":"2023-08-25T09:52:03.618767+0000","last_unstale":"2023-08-25T09:52:03.619131+0000","last_undegraded":"2023-08-25T09:52:03.619131+0000","last_fullsized":"2023-08-25T09:52:03.619131+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:05:24.291732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":475,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.598927+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:52:03.598927+0000","last_peered":"2023-08-25T09:52:03.598927+0000","last_clean":"2023-08-25T09:52:03.598927+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:52:03.598927+0000","last_undegraded":"2023-08-25T09:52:03.598927+0000","last_fullsized":"2023-08-25T09:52:03.598927+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:07.239907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.602122+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:52:03.602122+0000","last_peered":"2023-08-25T09:52:03.602122+0000","last_clean":"2023-08-25T09:52:03.602122+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:52:03.602122+0000","last_undegraded":"2023-08-25T09:52:03.602122+0000","last_fullsized":"2023-08-25T09:52:03.602122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:33:51.823664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595499+0000","last_change":"2023-08-25T09:52:03.595499+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595499+0000","last_undegraded":"2023-08-25T09:52:03.595499+0000","last_fullsized":"2023-08-25T09:52:03.595499+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.609570+0000","last_change":"2023-08-25T09:52:03.608811+0000","last_active":"2023-08-25T09:52:03.609570+0000","last_peered":"2023-08-25T09:52:03.609570+0000","last_clean":"2023-08-25T09:52:03.609570+0000","last_became_active":"2023-08-25T09:52:03.608140+0000","last_became_peered":"2023-08-25T09:52:03.608140+0000","last_unstale":"2023-08-25T09:52:03.609570+0000","last_undegraded":"2023-08-25T09:52:03.609570+0000","last_fullsized":"2023-08-25T09:52:03.609570+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:24:40.118198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.599796+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:52:03.599796+0000","last_peered":"2023-08-25T09:52:03.599796+0000","last_clean":"2023-08-25T09:52:03.599796+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:52:03.599796+0000","last_undegraded":"2023-08-25T09:52:03.599796+0000","last_fullsized":"2023-08-25T09:52:03.599796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:15.025918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596135+0000","last_change":"2023-08-25T09:52:03.596135+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596135+0000","last_undegraded":"2023-08-25T09:52:03.596135+0000","last_fullsized":"2023-08-25T09:52:03.596135+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.616009+0000","last_change":"2023-08-25T09:52:03.615496+0000","last_active":"2023-08-25T09:52:03.616009+0000","last_peered":"2023-08-25T09:52:03.616009+0000","last_clean":"2023-08-25T09:52:03.616009+0000","last_became_active":"2023-08-25T09:52:03.615224+0000","last_became_peered":"2023-08-25T09:52:03.615224+0000","last_unstale":"2023-08-25T09:52:03.616009+0000","last_undegraded":"2023-08-25T09:52:03.616009+0000","last_fullsized":"2023-08-25T09:52:03.616009+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:00.452072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":355,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.590094+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:52:03.590094+0000","last_peered":"2023-08-25T09:52:03.590094+0000","last_clean":"2023-08-25T09:52:03.590094+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:52:03.590094+0000","last_undegraded":"2023-08-25T09:52:03.590094+0000","last_fullsized":"2023-08-25T09:52:03.590094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:51.224218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595733+0000","last_change":"2023-08-25T09:52:03.595733+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595733+0000","last_undegraded":"2023-08-25T09:52:03.595733+0000","last_fullsized":"2023-08-25T09:52:03.595733+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.616028+0000","last_change":"2023-08-25T09:52:03.615906+0000","last_active":"2023-08-25T09:52:03.616028+0000","last_peered":"2023-08-25T09:52:03.616028+0000","last_clean":"2023-08-25T09:52:03.616028+0000","last_became_active":"2023-08-25T09:52:03.615475+0000","last_became_peered":"2023-08-25T09:52:03.615475+0000","last_unstale":"2023-08-25T09:52:03.616028+0000","last_undegraded":"2023-08-25T09:52:03.616028+0000","last_fullsized":"2023-08-25T09:52:03.616028+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:00:02.156359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.600084+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:52:03.600084+0000","last_peered":"2023-08-25T09:52:03.600084+0000","last_clean":"2023-08-25T09:52:03.600084+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:52:03.600084+0000","last_undegraded":"2023-08-25T09:52:03.600084+0000","last_fullsized":"2023-08-25T09:52:03.600084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:56:48.511610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605516+0000","last_change":"2023-08-25T09:52:03.605516+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605516+0000","last_undegraded":"2023-08-25T09:52:03.605516+0000","last_fullsized":"2023-08-25T09:52:03.605516+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.613201+0000","last_change":"2023-08-25T09:52:03.613108+0000","last_active":"2023-08-25T09:52:03.613201+0000","last_peered":"2023-08-25T09:52:03.613201+0000","last_clean":"2023-08-25T09:52:03.613201+0000","last_became_active":"2023-08-25T09:52:03.612787+0000","last_became_peered":"2023-08-25T09:52:03.612787+0000","last_unstale":"2023-08-25T09:52:03.613201+0000","last_undegraded":"2023-08-25T09:52:03.613201+0000","last_fullsized":"2023-08-25T09:52:03.613201+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:30.915836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.590217+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:52:03.590217+0000","last_peered":"2023-08-25T09:52:03.590217+0000","last_clean":"2023-08-25T09:52:03.590217+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:52:03.590217+0000","last_undegraded":"2023-08-25T09:52:03.590217+0000","last_fullsized":"2023-08-25T09:52:03.590217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:17:35.995249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.616369+0000","last_change":"2023-08-25T09:52:03.616369+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.616369+0000","last_undegraded":"2023-08-25T09:52:03.616369+0000","last_fullsized":"2023-08-25T09:52:03.616369+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615749+0000","last_change":"2023-08-25T09:52:03.615255+0000","last_active":"2023-08-25T09:52:03.615749+0000","last_peered":"2023-08-25T09:52:03.615749+0000","last_clean":"2023-08-25T09:52:03.615749+0000","last_became_active":"2023-08-25T09:52:03.613680+0000","last_became_peered":"2023-08-25T09:52:03.613680+0000","last_unstale":"2023-08-25T09:52:03.615749+0000","last_undegraded":"2023-08-25T09:52:03.615749+0000","last_fullsized":"2023-08-25T09:52:03.615749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:11.723113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.599484+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:52:03.599484+0000","last_peered":"2023-08-25T09:52:03.599484+0000","last_clean":"2023-08-25T09:52:03.599484+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:52:03.599484+0000","last_undegraded":"2023-08-25T09:52:03.599484+0000","last_fullsized":"2023-08-25T09:52:03.599484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:37.067424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.601630+0000","last_change":"2023-08-25T09:52:03.601630+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.601630+0000","last_undegraded":"2023-08-25T09:52:03.601630+0000","last_fullsized":"2023-08-25T09:52:03.601630+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.608543+0000","last_change":"2023-08-25T09:52:03.608367+0000","last_active":"2023-08-25T09:52:03.608543+0000","last_peered":"2023-08-25T09:52:03.608543+0000","last_clean":"2023-08-25T09:52:03.608543+0000","last_became_active":"2023-08-25T09:52:03.605716+0000","last_became_peered":"2023-08-25T09:52:03.605716+0000","last_unstale":"2023-08-25T09:52:03.608543+0000","last_undegraded":"2023-08-25T09:52:03.608543+0000","last_fullsized":"2023-08-25T09:52:03.608543+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:18:13.745741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":321,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.599505+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:52:03.599505+0000","last_peered":"2023-08-25T09:52:03.599505+0000","last_clean":"2023-08-25T09:52:03.599505+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:52:03.599505+0000","last_undegraded":"2023-08-25T09:52:03.599505+0000","last_fullsized":"2023-08-25T09:52:03.599505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:08:03.681656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594627+0000","last_change":"2023-08-25T09:52:03.594627+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594627+0000","last_undegraded":"2023-08-25T09:52:03.594627+0000","last_fullsized":"2023-08-25T09:52:03.594627+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.612060+0000","last_change":"2023-08-25T09:52:03.611964+0000","last_active":"2023-08-25T09:52:03.612060+0000","last_peered":"2023-08-25T09:52:03.612060+0000","last_clean":"2023-08-25T09:52:03.612060+0000","last_became_active":"2023-08-25T09:52:03.611724+0000","last_became_peered":"2023-08-25T09:52:03.611724+0000","last_unstale":"2023-08-25T09:52:03.612060+0000","last_undegraded":"2023-08-25T09:52:03.612060+0000","last_fullsized":"2023-08-25T09:52:03.612060+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:08.582145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.598776+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:52:03.598776+0000","last_peered":"2023-08-25T09:52:03.598776+0000","last_clean":"2023-08-25T09:52:03.598776+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:52:03.598776+0000","last_undegraded":"2023-08-25T09:52:03.598776+0000","last_fullsized":"2023-08-25T09:52:03.598776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:43.545960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194168,"statfs":{"total":200000000,"available":198828593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.179,"15min":1.1259999999999999},"min":{"1min":0.754,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.829,"5min":1.829,"15min":10.747},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.1539999999999999,"15min":1.0980000000000001},"min":{"1min":0.69699999999999995,"5min":0.61699999999999999,"15min":0.43099999999999999},"max":{"1min":1.637,"5min":1.651,"15min":1.855},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.28,"15min":1.2370000000000001},"min":{"1min":0.66000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":11.010999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.2689999999999999,"15min":1.177},"min":{"1min":0.83699999999999997,"5min":0.70899999999999996,"15min":0.57999999999999996},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1830000000000001,"15min":1.137},"min":{"1min":0.85799999999999998,"5min":0.66800000000000004,"15min":0.50600000000000001},"max":{"1min":1.46,"5min":1.5529999999999999,"15min":1.619},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.304,"15min":1.27},"min":{"1min":0.99199999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8149999999999999,"5min":1.873,"15min":1.98},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.23,"15min":1.21},"min":{"1min":0.81399999999999995,"5min":0.59799999999999998,"15min":0.56799999999999995},"max":{"1min":1.5680000000000001,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.456},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.385,"15min":1.363},"min":{"1min":1.018,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7270000000000001,"5min":1.833,"15min":2.1030000000000002},"last":1.135}]},{"osd":4,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.268,"15min":1.2869999999999999},"min":{"1min":0.82999999999999996,"5min":0.51300000000000001,"15min":0.48599999999999999},"max":{"1min":1.7010000000000001,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.94599999999999995,"5min":0.53800000000000003,"15min":0.46300000000000002},"max":{"1min":1.74,"5min":1.798,"15min":1.8779999999999999},"last":1.5700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.46,"15min":1.411},"min":{"1min":0.90700000000000003,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.522},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.468,"15min":1.4279999999999999},"min":{"1min":1.1060000000000001,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.907,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.6890000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5589999999999999,"15min":1.51},"min":{"1min":1.3400000000000001,"5min":0.72999999999999998,"15min":0.68799999999999994},"max":{"1min":1.984,"5min":2.0259999999999998,"15min":2.1720000000000002},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.488,"15min":1.456},"min":{"1min":0.90600000000000003,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":1.877,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6339999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1539999999999999,"15min":1.1459999999999999},"min":{"1min":0.78800000000000003,"5min":0.68700000000000006,"15min":0.54400000000000004},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.1779999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5980000000000001,"5min":1.992,"15min":5.5410000000000004},"last":0.97299999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.222,"15min":1.2110000000000001},"min":{"1min":0.78800000000000003,"5min":0.77000000000000002,"15min":0.54700000000000004},"max":{"1min":1.8380000000000001,"5min":2.198,"15min":5.6799999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1919999999999999,"15min":1.1739999999999999},"min":{"1min":0.79000000000000004,"5min":0.66800000000000004,"15min":0.50800000000000001},"max":{"1min":1.6220000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1739999999999999,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.526,"5min":1.631,"15min":1.708},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.288,"15min":1.288},"min":{"1min":0.88700000000000001,"5min":0.67000000000000004,"15min":0.47299999999999998},"max":{"1min":1.6140000000000001,"5min":1.7969999999999999,"15min":13.193},"last":1.093}]},{"osd":3,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.345,"15min":1.2949999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7010000000000001,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.85099999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3400000000000001,"15min":1.3100000000000001},"min":{"1min":0.93999999999999995,"5min":0.55300000000000005,"15min":0.42599999999999999},"max":{"1min":1.7609999999999999,"5min":2.1110000000000002,"15min":2.484},"last":1.165},{"interface":"front","average":{"1min":1.454,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.78900000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.101,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.2969999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3640000000000001,"15min":1.3169999999999999},"min":{"1min":0.75,"5min":0.65000000000000002,"15min":0.59599999999999997},"max":{"1min":1.831,"5min":1.8640000000000001,"15min":2.0049999999999999},"last":1.9670000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4770000000000001,"15min":1.444},"min":{"1min":1.0980000000000001,"5min":0.61099999999999999,"15min":0.59099999999999997},"max":{"1min":1.716,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.534}]},{"osd":7,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.518,"15min":1.518},"min":{"1min":0.89400000000000002,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":1.917,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.532,"15min":1.494},"min":{"1min":0.872,"5min":0.81699999999999995,"15min":0.57399999999999995},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":2.0590000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575011,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1720,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193592,"statfs":{"total":200000000,"available":198238225,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2529999999999999,"15min":1.196},"min":{"1min":0.753,"5min":0.64000000000000001,"15min":0.497},"max":{"1min":1.845,"5min":1.9770000000000001,"15min":2.081},"last":1.024},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2529999999999999,"15min":1.22},"min":{"1min":0.747,"5min":0.747,"15min":0.65400000000000003},"max":{"1min":1.621,"5min":1.8080000000000001,"15min":1.974},"last":1.1479999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.177,"15min":1.1719999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.6499999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1559999999999999,"15min":1.153},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.492,"5min":1.704,"15min":2.1869999999999998},"last":1.111}]},{"osd":2,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.296,"15min":1.2370000000000001},"min":{"1min":0.874,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.772,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.204},{"interface":"front","average":{"1min":1.22,"5min":1.2869999999999999,"15min":1.252},"min":{"1min":0.75,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2290000000000001,"15min":1.2829999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.57999999999999996},"max":{"1min":1.597,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3540000000000001,"15min":1.3200000000000001},"min":{"1min":0.87,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.7410000000000001,"5min":1.964,"15min":1.9650000000000001},"last":0.95299999999999996}]},{"osd":4,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3169999999999999,"15min":1.3129999999999999},"min":{"1min":0.91400000000000003,"5min":0.58299999999999996,"15min":0.55500000000000005},"max":{"1min":1.671,"5min":1.921,"15min":2.625},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3220000000000001,"15min":1.3420000000000001},"min":{"1min":0.90500000000000003,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":1.7290000000000001,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4790000000000001,"15min":1.446},"min":{"1min":1.131,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":1.794,"5min":2.048,"15min":2.4820000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5169999999999999,"15min":1.4750000000000001},"min":{"1min":1.268,"5min":0.65500000000000003,"15min":0.60399999999999998},"max":{"1min":1.901,"5min":2.044,"15min":2.5539999999999998},"last":1.4279999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4910000000000001,"15min":1.46},"min":{"1min":1.099,"5min":0.85399999999999998,"15min":0.63},"max":{"1min":1.8069999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.585,"5min":1.5920000000000001,"15min":1.581},"min":{"1min":1.22,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":1.8899999999999999,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.2869999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.28,"15min":1.224},"min":{"1min":0.624,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.589,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.405},{"interface":"front","average":{"1min":1.264,"5min":1.2709999999999999,"15min":1.2490000000000001},"min":{"1min":0.91600000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.615,"5min":2.5150000000000001,"15min":7.367},"last":1.135}]},{"osd":1,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.466,"5min":1.7330000000000001,"15min":2.032},"last":0.88},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.234,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.571,"5min":4.351,"15min":8.5850000000000009},"last":1.3200000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.214},"min":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.556,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.294,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5940000000000001,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.5209999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.284,"15min":1.2629999999999999},"min":{"1min":0.79400000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.665,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.351},{"interface":"front","average":{"1min":1.359,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.911},"last":1.548}]},{"osd":5,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3140000000000001,"15min":1.3009999999999999},"min":{"1min":0.67100000000000004,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.7370000000000001,"5min":5.048,"15min":5.048},"last":1.214},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.2729999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4179999999999999,"15min":1.4319999999999999},"min":{"1min":0.88100000000000001,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":1.927,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.597,"15min":1.579},"min":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.968,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.496,"15min":1.49},"min":{"1min":0.75700000000000001,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.865,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.494},{"interface":"front","average":{"1min":1.504,"5min":1.569,"15min":1.5920000000000001},"min":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8340000000000001,"5min":3.831,"15min":3.831},"last":1.4450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98599999999999999,"15min":0.91000000000000003},"min":{"1min":0.628,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.8180000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0649999999999999,"15min":0.98199999999999998},"min":{"1min":0.69999999999999996,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.579,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.089}]},{"osd":1,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.64600000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.258,"5min":1.413,"15min":1.702},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.087,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.623,"5min":1.6339999999999999,"15min":2.3399999999999999},"last":0.94399999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.034,"15min":0.96499999999999997},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.5980000000000001,"5min":1.623,"15min":1.919},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2050000000000001,"15min":1.1439999999999999},"min":{"1min":0.745,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.901,"5min":1.901,"15min":3.399},"last":1.137}]},{"osd":4,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3009999999999999,"15min":1.236},"min":{"1min":1.1399999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.204,"15min":1.2090000000000001},"min":{"1min":1.0109999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.405,"5min":1.79,"15min":2.1840000000000002},"last":1.341}]},{"osd":5,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3480000000000001},"min":{"1min":1.137,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":2.1509999999999998,"5min":2.1930000000000001,"15min":2.407},"last":1.284},{"interface":"front","average":{"1min":1.357,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.0349999999999999,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.804,"5min":1.895,"15min":1.895},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.302,"15min":1.282},"min":{"1min":1.054,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5980000000000001,"5min":1.875,"15min":2.3450000000000002},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.47,"15min":1.4199999999999999},"min":{"1min":1.28,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":2.2330000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.3700000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.464,"15min":1.405},"min":{"1min":1.0840000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.976,"5min":1.976,"15min":2.2989999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.3839999999999999},"min":{"1min":1.1259999999999999,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":1.7589999999999999,"5min":1.9350000000000001,"15min":2.1509999999999998},"last":1.444}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194168,"statfs":{"total":200000000,"available":198828593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.6930000000000001,"5min":1.706,"15min":2.383},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.92100000000000004,"15min":0.97699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":1.367,"5min":1.645,"15min":2.7309999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.95399999999999996,"15min":0.98299999999999998},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.446,"5min":1.978,"15min":2.3180000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.02,"15min":1.04},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.528,"5min":1.704,"15min":2.8599999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.92600000000000005,"15min":1.0580000000000001},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.341,"5min":1.6839999999999999,"15min":18.821000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.54600000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.486,"5min":1.8999999999999999,"15min":2.2040000000000002},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2689999999999999,"15min":1.329},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9139999999999999,"5min":2.0059999999999998,"15min":3.0150000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.2549999999999999,"15min":1.294},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8,"5min":1.8,"15min":2.7639999999999998},"last":1.6459999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2569999999999999,"15min":1.296},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4990000000000001,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3400000000000001,"15min":1.371},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0920000000000001,"15min":2.4220000000000002},"last":1.3240000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3979999999999999,"15min":1.403},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8109999999999999,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.409,"15min":1.4299999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":1.23}]},{"osd":7,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.367,"15min":1.3999999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.706,"5min":2.3149999999999999,"15min":2.968},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4379999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7330000000000001,"5min":2.96,"15min":5.2990000000000004},"last":1.0740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1726,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193585,"statfs":{"total":200000000,"available":198231613,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.82699999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.29699999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97899999999999998,"15min":0.89800000000000002},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.7869999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.88100000000000001},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.35899999999999999},"max":{"1min":1.2809999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.022,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.389,"5min":1.6619999999999999,"15min":1.746},"last":1.153}]},{"osd":3,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.109},"min":{"1min":0.496,"5min":0.496,"15min":0.372},"max":{"1min":1.647,"5min":1.647,"15min":1.9450000000000001},"last":1.419},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0109999999999999},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.333,"5min":1.655,"15min":1.764},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.254,"5min":1.2929999999999999,"15min":1.276},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":1.6830000000000001,"5min":1.9419999999999999,"15min":2.008},"last":1.5529999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.2430000000000001},"min":{"1min":0.94699999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.665,"15min":1.9650000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3049999999999999,"15min":1.2450000000000001},"min":{"1min":0.82699999999999996,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.677,"5min":2.867,"15min":2.867},"last":0.97399999999999998}]},{"osd":6,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.363},"min":{"1min":1.016,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":1.633,"5min":2.0779999999999998,"15min":2.9940000000000002},"last":1.605},{"interface":"front","average":{"1min":1.331,"5min":1.411,"15min":1.3660000000000001},"min":{"1min":0.95099999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.603,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4490000000000001,"15min":1.4139999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.79200000000000004},"max":{"1min":1.772,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.431,"15min":1.399},"min":{"1min":0.88200000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.738,"5min":1.913,"15min":2.218},"last":1.1870000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.41499999999999998,"5min":0.40699999999999997,"15min":0.313},"max":{"1min":1.2929999999999999,"5min":2.2759999999999998,"15min":11.257},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.96099999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.405,"5min":1.829,"15min":6.4260000000000002},"last":1.107}]},{"osd":2,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.95899999999999996},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.522,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.5860000000000001,"5min":1.714,"15min":3.4889999999999999},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.92400000000000004},"min":{"1min":0.46500000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.532,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1240000000000001,"15min":1.1160000000000001},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5860000000000001,"5min":1.825,"15min":2.3809999999999998},"last":0.745}]},{"osd":4,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.278,"15min":1.2529999999999999},"min":{"1min":0.83599999999999997,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.782,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7589999999999999,"5min":2.0339999999999998,"15min":2.5499999999999998},"last":1.282}]},{"osd":5,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.599,"5min":2.0939999999999999,"15min":3.3769999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.27,"15min":1.276},"min":{"1min":0.747,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6299999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.079}]},{"osd":6,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4119999999999999,"15min":1.3600000000000001},"min":{"1min":0.88900000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9910000000000001,"5min":2.1480000000000001,"15min":2.6869999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.863,"5min":1.9710000000000001,"15min":2.331},"last":1.1839999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.423,"15min":1.405},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.669,"5min":2.4590000000000001,"15min":2.6429999999999998},"last":1.143},{"interface":"front","average":{"1min":1.385,"5min":1.3740000000000001,"15min":1.3380000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.847,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.157}]}]}],"pool_statfs":[]}} 2023-08-25T09:52:09.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:52:09.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.321+0000 7f3c48887640 1 Processor -- start 2023-08-25T09:52:09.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.321+0000 7f3c48887640 1 -- start start 2023-08-25T09:52:09.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.322+0000 7f3c48887640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c440b1ff0 0x7f3c440b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.322+0000 7f3c48887640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c440b1b60 con 0x7f3c4415e830 2023-08-25T09:52:09.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.322+0000 7f3c48887640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4406b7c0 con 0x7f3c4415ec10 2023-08-25T09:52:09.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.322+0000 7f3c48887640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4406bd80 con 0x7f3c440b1ff0 2023-08-25T09:52:09.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4259c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3c4415e830 0x7f3c4406ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45768/0 (socket says 172.21.15.70:45768) 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4259c640 1 -- 172.21.15.70:0/1386021116 learned_addr learned my addr 172.21.15.70:0/1386021116 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254667677 0 0) 0x7f3c440b1b60 con 0x7f3c4415e830 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c42d9d640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c440b1ff0 0x7f3c440b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c2c003520 con 0x7f3c4415e830 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4150926693 0 0) 0x7f3c2c003520 con 0x7f3c4415e830 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c440b1ff0 msgr2=0x7f3c440b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:09.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.323+0000 7f3c4159a640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c440b1ff0 0x7f3c440b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 >> v1:172.21.15.155:6789/0 conn(0x7f3c4415ec10 legacy=0x7f3c4415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4406baa0 con 0x7f3c4415e830 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f3c34001670 con 0x7f3c4415e830 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 >> v1:172.21.15.70:6789/0 conn(0x7f3c4415e830 legacy=0x7f3c4406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c2c0039a0 0x7f3c2c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c2c005180 0x7f3c2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4406bd80 con 0x7f3c2c0047c0 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c00c020 con 0x7f3c2c0039a0 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c00c1e0 con 0x7f3c2c005180 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c42d9d640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c440b1ff0 0x7f3c440b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c41d9b640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4259c640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c2c0039a0 0x7f3c2c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c41d9b640 1 -- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c2c005180 msgr2=0x7f3c2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:09.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c41d9b640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c2c005180 0x7f3c2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c41d9b640 1 -- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c2c0039a0 msgr2=0x7f3c2c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c41d9b640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c2c0039a0 0x7f3c2c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c41d9b640 1 -- 172.21.15.70:0/1386021116 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c003520 con 0x7f3c2c0047c0 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.324+0000 7f3c4259c640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c2c0039a0 0x7f3c2c003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.325+0000 7f3c41d9b640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c2c004c40 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f3c38001720 tx=0x7f3c2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a0ccb8613ff7c1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380052d0 con 0x7f3c2c0047c0 2023-08-25T09:52:09.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3802e050 con 0x7f3c2c0047c0 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c4159a640 1 -- 172.21.15.70:0/1386021116 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c380035f0 con 0x7f3c2c0047c0 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c48887640 1 -- 172.21.15.70:0/1386021116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 msgr2=0x7f3c2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c48887640 1 --2- 172.21.15.70:0/1386021116 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c2c004c40 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f3c38001720 tx=0x7f3c2c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c42d9d640 1 -- 172.21.15.70:0/1386021116 reap_dead start 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c48887640 1 -- 172.21.15.70:0/1386021116 shutdown_connections 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c48887640 1 -- 172.21.15.70:0/1386021116 >> 172.21.15.70:0/1386021116 conn(0x7f3c44069850 msgr2=0x7f3c44069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c48887640 1 -- 172.21.15.70:0/1386021116 shutdown_connections 2023-08-25T09:52:09.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.326+0000 7f3c48887640 1 -- 172.21.15.70:0/1386021116 wait complete. 2023-08-25T09:52:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.328+0000 7f3c48887640 1 Processor -- start 2023-08-25T09:52:09.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.328+0000 7f3c48887640 1 -- start start 2023-08-25T09:52:09.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.329+0000 7f3c48887640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c440705c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.329+0000 7f3c48887640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c44150790 0x7f3c440c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.329+0000 7f3c48887640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c441554c0 0x7f3c44147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.329+0000 7f3c48887640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440c85c0 con 0x7f3c2c0047c0 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.329+0000 7f3c48887640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440c79d0 con 0x7f3c441554c0 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.329+0000 7f3c48887640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c440c7c90 con 0x7f3c44150790 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c440705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c440705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43064/0 (socket says 172.21.15.70:43064) 2023-08-25T09:52:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 -- 172.21.15.70:0/1411935201 learned_addr learned my addr 172.21.15.70:0/1411935201 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 -- 172.21.15.70:0/1411935201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c44150790 msgr2=0x7f3c440c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c44150790 0x7f3c440c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 -- 172.21.15.70:0/1411935201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c441554c0 msgr2=0x7f3c44147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c441554c0 0x7f3c44147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 -- 172.21.15.70:0/1411935201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c440c7f70 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.330+0000 7f3c4259c640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c440705c0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d8e0 tx=0x7f3c3400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.331+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c340074e0 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.331+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c34007640 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.331+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c340049e0 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.331+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c440c8250 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.331+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c440c8f90 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.332+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c440cbb00 con 0x7f3c2c0047c0 2023-08-25T09:52:09.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.333+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3c34004b40 con 0x7f3c2c0047c0 2023-08-25T09:52:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.334+0000 7f3c32ffd640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c2406a050 0x7f3c2406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.334+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3c34087fa0 con 0x7f3c2c0047c0 2023-08-25T09:52:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.334+0000 7f3c41d9b640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c2406a050 0x7f3c2406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.335+0000 7f3c41d9b640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c2406a050 0x7f3c2406c510 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3c38033000 tx=0x7f3c3802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.338+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3c34057130 con 0x7f3c2c0047c0 2023-08-25T09:52:09.527 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.525+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c440b17f0 con 0x7f3c2406a050 2023-08-25T09:52:09.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.536+0000 7f3c32ffd640 1 -- 172.21.15.70:0/1411935201 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+297781 (secure 0 0 0) 0x7f3c38005340 con 0x7f3c2406a050 2023-08-25T09:52:09.540 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:09.543 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:52:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c2406a050 msgr2=0x7f3c2406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c2406a050 0x7f3c2406c510 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3c38033000 tx=0x7f3c3802f000 comp rx=0 tx=0).stop 2023-08-25T09:52:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 msgr2=0x7f3c440705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c440705c0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400d8e0 tx=0x7f3c3400dda0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 shutdown_connections 2023-08-25T09:52:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3c2406a050 0x7f3c2406c510 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3c441554c0 0x7f3c44147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3c44150790 0x7f3c440c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 --2- 172.21.15.70:0/1411935201 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3c2c0047c0 0x7f3c440705c0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.546 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 >> 172.21.15.70:0/1411935201 conn(0x7f3c44069850 msgr2=0x7f3c4415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:09.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 shutdown_connections 2023-08-25T09:52:09.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.543+0000 7f3c48887640 1 -- 172.21.15.70:0/1411935201 wait complete. 2023-08-25T09:52:09.554 INFO:tasks.ceph.mds.a.smithi070.stderr:2023-08-25T09:52:09.552+0000 7f9914bc5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:52:09.565 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":594,"stamp":"2023-08-25T09:52:07.592093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10336,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552163,"statfs":{"total":1600000000,"available":1589414948,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013135"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.598037+0000","last_change":"2023-08-25T09:52:03.598037+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.598037+0000","last_undegraded":"2023-08-25T09:52:03.598037+0000","last_fullsized":"2023-08-25T09:52:03.598037+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.599450+0000","last_change":"2023-08-25T09:52:03.599450+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.599450+0000","last_undegraded":"2023-08-25T09:52:03.599450+0000","last_fullsized":"2023-08-25T09:52:03.599450+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.600397+0000","last_change":"2023-08-25T09:52:03.600397+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.600397+0000","last_undegraded":"2023-08-25T09:52:03.600397+0000","last_fullsized":"2023-08-25T09:52:03.600397+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.603937+0000","last_change":"2023-08-25T09:52:03.603937+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.603937+0000","last_undegraded":"2023-08-25T09:52:03.603937+0000","last_fullsized":"2023-08-25T09:52:03.603937+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609415+0000","last_change":"2023-08-25T09:52:03.609415+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609415+0000","last_undegraded":"2023-08-25T09:52:03.609415+0000","last_fullsized":"2023-08-25T09:52:03.609415+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597169+0000","last_change":"2023-08-25T09:52:03.597169+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597169+0000","last_undegraded":"2023-08-25T09:52:03.597169+0000","last_fullsized":"2023-08-25T09:52:03.597169+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.601229+0000","last_change":"2023-08-25T09:52:03.601229+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.601229+0000","last_undegraded":"2023-08-25T09:52:03.601229+0000","last_fullsized":"2023-08-25T09:52:03.601229+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605774+0000","last_change":"2023-08-25T09:52:03.605774+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605774+0000","last_undegraded":"2023-08-25T09:52:03.605774+0000","last_fullsized":"2023-08-25T09:52:03.605774+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605247+0000","last_change":"2023-08-25T09:52:03.605247+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605247+0000","last_undegraded":"2023-08-25T09:52:03.605247+0000","last_fullsized":"2023-08-25T09:52:03.605247+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609019+0000","last_change":"2023-08-25T09:52:03.609019+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609019+0000","last_undegraded":"2023-08-25T09:52:03.609019+0000","last_fullsized":"2023-08-25T09:52:03.609019+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596405+0000","last_change":"2023-08-25T09:52:03.596405+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596405+0000","last_undegraded":"2023-08-25T09:52:03.596405+0000","last_fullsized":"2023-08-25T09:52:03.596405+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.603780+0000","last_change":"2023-08-25T09:52:03.603780+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.603780+0000","last_undegraded":"2023-08-25T09:52:03.603780+0000","last_fullsized":"2023-08-25T09:52:03.603780+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.598799+0000","last_change":"2023-08-25T09:52:03.598799+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.598799+0000","last_undegraded":"2023-08-25T09:52:03.598799+0000","last_fullsized":"2023-08-25T09:52:03.598799+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.611880+0000","last_change":"2023-08-25T09:52:03.611880+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.611880+0000","last_undegraded":"2023-08-25T09:52:03.611880+0000","last_fullsized":"2023-08-25T09:52:03.611880+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596692+0000","last_change":"2023-08-25T09:52:03.596692+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596692+0000","last_undegraded":"2023-08-25T09:52:03.596692+0000","last_fullsized":"2023-08-25T09:52:03.596692+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595634+0000","last_change":"2023-08-25T09:52:03.595634+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595634+0000","last_undegraded":"2023-08-25T09:52:03.595634+0000","last_fullsized":"2023-08-25T09:52:03.595634+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604783+0000","last_change":"2023-08-25T09:52:03.604783+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604783+0000","last_undegraded":"2023-08-25T09:52:03.604783+0000","last_fullsized":"2023-08-25T09:52:03.604783+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604441+0000","last_change":"2023-08-25T09:52:03.604441+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604441+0000","last_undegraded":"2023-08-25T09:52:03.604441+0000","last_fullsized":"2023-08-25T09:52:03.604441+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597476+0000","last_change":"2023-08-25T09:52:03.597476+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597476+0000","last_undegraded":"2023-08-25T09:52:03.597476+0000","last_fullsized":"2023-08-25T09:52:03.597476+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607624+0000","last_change":"2023-08-25T09:52:03.607624+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607624+0000","last_undegraded":"2023-08-25T09:52:03.607624+0000","last_fullsized":"2023-08-25T09:52:03.607624+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597284+0000","last_change":"2023-08-25T09:52:03.597284+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597284+0000","last_undegraded":"2023-08-25T09:52:03.597284+0000","last_fullsized":"2023-08-25T09:52:03.597284+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595744+0000","last_change":"2023-08-25T09:52:03.595744+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595744+0000","last_undegraded":"2023-08-25T09:52:03.595744+0000","last_fullsized":"2023-08-25T09:52:03.595744+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607681+0000","last_change":"2023-08-25T09:52:03.607681+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607681+0000","last_undegraded":"2023-08-25T09:52:03.607681+0000","last_fullsized":"2023-08-25T09:52:03.607681+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596686+0000","last_change":"2023-08-25T09:52:03.596686+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596686+0000","last_undegraded":"2023-08-25T09:52:03.596686+0000","last_fullsized":"2023-08-25T09:52:03.596686+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596928+0000","last_change":"2023-08-25T09:52:03.596928+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596928+0000","last_undegraded":"2023-08-25T09:52:03.596928+0000","last_fullsized":"2023-08-25T09:52:03.596928+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.610788+0000","last_change":"2023-08-25T09:52:03.610788+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.610788+0000","last_undegraded":"2023-08-25T09:52:03.610788+0000","last_fullsized":"2023-08-25T09:52:03.610788+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.598589+0000","last_change":"2023-08-25T09:52:03.598589+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.598589+0000","last_undegraded":"2023-08-25T09:52:03.598589+0000","last_fullsized":"2023-08-25T09:52:03.598589+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597994+0000","last_change":"2023-08-25T09:52:03.597994+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597994+0000","last_undegraded":"2023-08-25T09:52:03.597994+0000","last_fullsized":"2023-08-25T09:52:03.597994+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607464+0000","last_change":"2023-08-25T09:52:03.607464+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607464+0000","last_undegraded":"2023-08-25T09:52:03.607464+0000","last_fullsized":"2023-08-25T09:52:03.607464+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594960+0000","last_change":"2023-08-25T09:52:03.594960+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594960+0000","last_undegraded":"2023-08-25T09:52:03.594960+0000","last_fullsized":"2023-08-25T09:52:03.594960+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.602630+0000","last_change":"2023-08-25T09:52:03.602630+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.602630+0000","last_undegraded":"2023-08-25T09:52:03.602630+0000","last_fullsized":"2023-08-25T09:52:03.602630+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597732+0000","last_change":"2023-08-25T09:52:03.597732+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597732+0000","last_undegraded":"2023-08-25T09:52:03.597732+0000","last_fullsized":"2023-08-25T09:52:03.597732+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.597830+0000","last_change":"2023-08-25T09:52:03.597830+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.597830+0000","last_undegraded":"2023-08-25T09:52:03.597830+0000","last_fullsized":"2023-08-25T09:52:03.597830+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.606094+0000","last_change":"2023-08-25T09:52:03.605509+0000","last_active":"2023-08-25T09:52:03.606094+0000","last_peered":"2023-08-25T09:52:03.606094+0000","last_clean":"2023-08-25T09:52:03.606094+0000","last_became_active":"2023-08-25T09:52:03.596779+0000","last_became_peered":"2023-08-25T09:52:03.596779+0000","last_unstale":"2023-08-25T09:52:03.606094+0000","last_undegraded":"2023-08-25T09:52:03.606094+0000","last_fullsized":"2023-08-25T09:52:03.606094+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:10:33.811352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595211+0000","last_change":"2023-08-25T09:52:03.595211+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595211+0000","last_undegraded":"2023-08-25T09:52:03.595211+0000","last_fullsized":"2023-08-25T09:52:03.595211+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.618472+0000","last_change":"2023-08-25T09:52:03.618376+0000","last_active":"2023-08-25T09:52:03.618472+0000","last_peered":"2023-08-25T09:52:03.618472+0000","last_clean":"2023-08-25T09:52:03.618472+0000","last_became_active":"2023-08-25T09:52:03.617343+0000","last_became_peered":"2023-08-25T09:52:03.617343+0000","last_unstale":"2023-08-25T09:52:03.618472+0000","last_undegraded":"2023-08-25T09:52:03.618472+0000","last_fullsized":"2023-08-25T09:52:03.618472+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:36.308598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.607124+0000","last_change":"2023-08-25T09:52:03.607124+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.607124+0000","last_undegraded":"2023-08-25T09:52:03.607124+0000","last_fullsized":"2023-08-25T09:52:03.607124+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615672+0000","last_change":"2023-08-25T09:52:03.615544+0000","last_active":"2023-08-25T09:52:03.615672+0000","last_peered":"2023-08-25T09:52:03.615672+0000","last_clean":"2023-08-25T09:52:03.615672+0000","last_became_active":"2023-08-25T09:52:03.615200+0000","last_became_peered":"2023-08-25T09:52:03.615200+0000","last_unstale":"2023-08-25T09:52:03.615672+0000","last_undegraded":"2023-08-25T09:52:03.615672+0000","last_fullsized":"2023-08-25T09:52:03.615672+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:55:24.277587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604724+0000","last_change":"2023-08-25T09:52:03.604724+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604724+0000","last_undegraded":"2023-08-25T09:52:03.604724+0000","last_fullsized":"2023-08-25T09:52:03.604724+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615770+0000","last_change":"2023-08-25T09:52:03.615665+0000","last_active":"2023-08-25T09:52:03.615770+0000","last_peered":"2023-08-25T09:52:03.615770+0000","last_clean":"2023-08-25T09:52:03.615770+0000","last_became_active":"2023-08-25T09:52:03.615398+0000","last_became_peered":"2023-08-25T09:52:03.615398+0000","last_unstale":"2023-08-25T09:52:03.615770+0000","last_undegraded":"2023-08-25T09:52:03.615770+0000","last_fullsized":"2023-08-25T09:52:03.615770+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:14.615337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595394+0000","last_change":"2023-08-25T09:52:03.595394+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595394+0000","last_undegraded":"2023-08-25T09:52:03.595394+0000","last_fullsized":"2023-08-25T09:52:03.595394+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.612841+0000","last_change":"2023-08-25T09:52:03.612691+0000","last_active":"2023-08-25T09:52:03.612841+0000","last_peered":"2023-08-25T09:52:03.612841+0000","last_clean":"2023-08-25T09:52:03.612841+0000","last_became_active":"2023-08-25T09:52:03.609712+0000","last_became_peered":"2023-08-25T09:52:03.609712+0000","last_unstale":"2023-08-25T09:52:03.612841+0000","last_undegraded":"2023-08-25T09:52:03.612841+0000","last_fullsized":"2023-08-25T09:52:03.612841+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:48:57.407979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594550+0000","last_change":"2023-08-25T09:52:03.594550+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594550+0000","last_undegraded":"2023-08-25T09:52:03.594550+0000","last_fullsized":"2023-08-25T09:52:03.594550+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.613629+0000","last_change":"2023-08-25T09:52:03.612785+0000","last_active":"2023-08-25T09:52:03.613629+0000","last_peered":"2023-08-25T09:52:03.613629+0000","last_clean":"2023-08-25T09:52:03.613629+0000","last_became_active":"2023-08-25T09:52:03.612129+0000","last_became_peered":"2023-08-25T09:52:03.612129+0000","last_unstale":"2023-08-25T09:52:03.613629+0000","last_undegraded":"2023-08-25T09:52:03.613629+0000","last_fullsized":"2023-08-25T09:52:03.613629+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:31.665821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604133+0000","last_change":"2023-08-25T09:52:03.604133+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604133+0000","last_undegraded":"2023-08-25T09:52:03.604133+0000","last_fullsized":"2023-08-25T09:52:03.604133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.618800+0000","last_change":"2023-08-25T09:52:03.618689+0000","last_active":"2023-08-25T09:52:03.618800+0000","last_peered":"2023-08-25T09:52:03.618800+0000","last_clean":"2023-08-25T09:52:03.618800+0000","last_became_active":"2023-08-25T09:52:03.617671+0000","last_became_peered":"2023-08-25T09:52:03.617671+0000","last_unstale":"2023-08-25T09:52:03.618800+0000","last_undegraded":"2023-08-25T09:52:03.618800+0000","last_fullsized":"2023-08-25T09:52:03.618800+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:52.014238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.606075+0000","last_change":"2023-08-25T09:52:03.606075+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.606075+0000","last_undegraded":"2023-08-25T09:52:03.606075+0000","last_fullsized":"2023-08-25T09:52:03.606075+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.604103+0000","last_change":"2023-08-25T09:52:03.603589+0000","last_active":"2023-08-25T09:52:03.604103+0000","last_peered":"2023-08-25T09:52:03.604103+0000","last_clean":"2023-08-25T09:52:03.604103+0000","last_became_active":"2023-08-25T09:52:03.602798+0000","last_became_peered":"2023-08-25T09:52:03.602798+0000","last_unstale":"2023-08-25T09:52:03.604103+0000","last_undegraded":"2023-08-25T09:52:03.604103+0000","last_fullsized":"2023-08-25T09:52:03.604103+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:13:06.723741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.612908+0000","last_change":"2023-08-25T09:52:03.612908+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.612908+0000","last_undegraded":"2023-08-25T09:52:03.612908+0000","last_fullsized":"2023-08-25T09:52:03.612908+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.619037+0000","last_change":"2023-08-25T09:52:03.618945+0000","last_active":"2023-08-25T09:52:03.619037+0000","last_peered":"2023-08-25T09:52:03.619037+0000","last_clean":"2023-08-25T09:52:03.619037+0000","last_became_active":"2023-08-25T09:52:03.618057+0000","last_became_peered":"2023-08-25T09:52:03.618057+0000","last_unstale":"2023-08-25T09:52:03.619037+0000","last_undegraded":"2023-08-25T09:52:03.619037+0000","last_fullsized":"2023-08-25T09:52:03.619037+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:29.871817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594548+0000","last_change":"2023-08-25T09:52:03.594548+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594548+0000","last_undegraded":"2023-08-25T09:52:03.594548+0000","last_fullsized":"2023-08-25T09:52:03.594548+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.617181+0000","last_change":"2023-08-25T09:52:03.617112+0000","last_active":"2023-08-25T09:52:03.617181+0000","last_peered":"2023-08-25T09:52:03.617181+0000","last_clean":"2023-08-25T09:52:03.617181+0000","last_became_active":"2023-08-25T09:52:03.616870+0000","last_became_peered":"2023-08-25T09:52:03.616870+0000","last_unstale":"2023-08-25T09:52:03.617181+0000","last_undegraded":"2023-08-25T09:52:03.617181+0000","last_fullsized":"2023-08-25T09:52:03.617181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:17:30.422505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594730+0000","last_change":"2023-08-25T09:52:03.594730+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594730+0000","last_undegraded":"2023-08-25T09:52:03.594730+0000","last_fullsized":"2023-08-25T09:52:03.594730+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.597181+0000","last_change":"2023-08-25T09:52:03.596983+0000","last_active":"2023-08-25T09:52:03.597181+0000","last_peered":"2023-08-25T09:52:03.597181+0000","last_clean":"2023-08-25T09:52:03.597181+0000","last_became_active":"2023-08-25T09:52:03.596298+0000","last_became_peered":"2023-08-25T09:52:03.596298+0000","last_unstale":"2023-08-25T09:52:03.597181+0000","last_undegraded":"2023-08-25T09:52:03.597181+0000","last_fullsized":"2023-08-25T09:52:03.597181+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:22.805922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594769+0000","last_change":"2023-08-25T09:52:03.594769+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594769+0000","last_undegraded":"2023-08-25T09:52:03.594769+0000","last_fullsized":"2023-08-25T09:52:03.594769+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615972+0000","last_change":"2023-08-25T09:52:03.614128+0000","last_active":"2023-08-25T09:52:03.615972+0000","last_peered":"2023-08-25T09:52:03.615972+0000","last_clean":"2023-08-25T09:52:03.615972+0000","last_became_active":"2023-08-25T09:52:03.610416+0000","last_became_peered":"2023-08-25T09:52:03.610416+0000","last_unstale":"2023-08-25T09:52:03.615972+0000","last_undegraded":"2023-08-25T09:52:03.615972+0000","last_fullsized":"2023-08-25T09:52:03.615972+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:31.051627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.591632+0000","last_change":"2023-08-25T09:52:03.591632+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.591632+0000","last_undegraded":"2023-08-25T09:52:03.591632+0000","last_fullsized":"2023-08-25T09:52:03.591632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.617647+0000","last_change":"2023-08-25T09:52:03.617559+0000","last_active":"2023-08-25T09:52:03.617647+0000","last_peered":"2023-08-25T09:52:03.617647+0000","last_clean":"2023-08-25T09:52:03.617647+0000","last_became_active":"2023-08-25T09:52:03.616663+0000","last_became_peered":"2023-08-25T09:52:03.616663+0000","last_unstale":"2023-08-25T09:52:03.617647+0000","last_undegraded":"2023-08-25T09:52:03.617647+0000","last_fullsized":"2023-08-25T09:52:03.617647+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:59:11.295939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.610240+0000","last_change":"2023-08-25T09:52:03.610240+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.610240+0000","last_undegraded":"2023-08-25T09:52:03.610240+0000","last_fullsized":"2023-08-25T09:52:03.610240+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.610090+0000","last_change":"2023-08-25T09:52:03.609849+0000","last_active":"2023-08-25T09:52:03.610090+0000","last_peered":"2023-08-25T09:52:03.610090+0000","last_clean":"2023-08-25T09:52:03.610090+0000","last_became_active":"2023-08-25T09:52:03.609198+0000","last_became_peered":"2023-08-25T09:52:03.609198+0000","last_unstale":"2023-08-25T09:52:03.610090+0000","last_undegraded":"2023-08-25T09:52:03.610090+0000","last_fullsized":"2023-08-25T09:52:03.610090+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:30.008208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.613991+0000","last_change":"2023-08-25T09:52:03.613991+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.613991+0000","last_undegraded":"2023-08-25T09:52:03.613991+0000","last_fullsized":"2023-08-25T09:52:03.613991+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.613308+0000","last_change":"2023-08-25T09:52:03.613206+0000","last_active":"2023-08-25T09:52:03.613308+0000","last_peered":"2023-08-25T09:52:03.613308+0000","last_clean":"2023-08-25T09:52:03.613308+0000","last_became_active":"2023-08-25T09:52:03.612871+0000","last_became_peered":"2023-08-25T09:52:03.612871+0000","last_unstale":"2023-08-25T09:52:03.613308+0000","last_undegraded":"2023-08-25T09:52:03.613308+0000","last_fullsized":"2023-08-25T09:52:03.613308+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:10:12.582747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609476+0000","last_change":"2023-08-25T09:52:03.609476+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609476+0000","last_undegraded":"2023-08-25T09:52:03.609476+0000","last_fullsized":"2023-08-25T09:52:03.609476+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.614627+0000","last_change":"2023-08-25T09:52:03.614440+0000","last_active":"2023-08-25T09:52:03.614627+0000","last_peered":"2023-08-25T09:52:03.614627+0000","last_clean":"2023-08-25T09:52:03.614627+0000","last_became_active":"2023-08-25T09:52:03.613170+0000","last_became_peered":"2023-08-25T09:52:03.613170+0000","last_unstale":"2023-08-25T09:52:03.614627+0000","last_undegraded":"2023-08-25T09:52:03.614627+0000","last_fullsized":"2023-08-25T09:52:03.614627+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:12:38.030116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604775+0000","last_change":"2023-08-25T09:52:03.604775+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604775+0000","last_undegraded":"2023-08-25T09:52:03.604775+0000","last_fullsized":"2023-08-25T09:52:03.604775+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.606882+0000","last_change":"2023-08-25T09:52:03.606748+0000","last_active":"2023-08-25T09:52:03.606882+0000","last_peered":"2023-08-25T09:52:03.606882+0000","last_clean":"2023-08-25T09:52:03.606882+0000","last_became_active":"2023-08-25T09:52:03.606384+0000","last_became_peered":"2023-08-25T09:52:03.606384+0000","last_unstale":"2023-08-25T09:52:03.606882+0000","last_undegraded":"2023-08-25T09:52:03.606882+0000","last_fullsized":"2023-08-25T09:52:03.606882+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:18.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605505+0000","last_change":"2023-08-25T09:52:03.605505+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605505+0000","last_undegraded":"2023-08-25T09:52:03.605505+0000","last_fullsized":"2023-08-25T09:52:03.605505+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.598266+0000","last_change":"2023-08-25T09:52:03.598008+0000","last_active":"2023-08-25T09:52:03.598266+0000","last_peered":"2023-08-25T09:52:03.598266+0000","last_clean":"2023-08-25T09:52:03.598266+0000","last_became_active":"2023-08-25T09:52:03.597171+0000","last_became_peered":"2023-08-25T09:52:03.597171+0000","last_unstale":"2023-08-25T09:52:03.598266+0000","last_undegraded":"2023-08-25T09:52:03.598266+0000","last_fullsized":"2023-08-25T09:52:03.598266+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:58:35.671897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.613359+0000","last_change":"2023-08-25T09:52:03.613359+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.613359+0000","last_undegraded":"2023-08-25T09:52:03.613359+0000","last_fullsized":"2023-08-25T09:52:03.613359+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.620006+0000","last_change":"2023-08-25T09:52:03.619922+0000","last_active":"2023-08-25T09:52:03.620006+0000","last_peered":"2023-08-25T09:52:03.620006+0000","last_clean":"2023-08-25T09:52:03.620006+0000","last_became_active":"2023-08-25T09:52:03.619288+0000","last_became_peered":"2023-08-25T09:52:03.619288+0000","last_unstale":"2023-08-25T09:52:03.620006+0000","last_undegraded":"2023-08-25T09:52:03.620006+0000","last_fullsized":"2023-08-25T09:52:03.620006+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:18.360502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.593490+0000","last_change":"2023-08-25T09:52:03.593490+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.593490+0000","last_undegraded":"2023-08-25T09:52:03.593490+0000","last_fullsized":"2023-08-25T09:52:03.593490+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615995+0000","last_change":"2023-08-25T09:52:03.614922+0000","last_active":"2023-08-25T09:52:03.615995+0000","last_peered":"2023-08-25T09:52:03.615995+0000","last_clean":"2023-08-25T09:52:03.615995+0000","last_became_active":"2023-08-25T09:52:03.614577+0000","last_became_peered":"2023-08-25T09:52:03.614577+0000","last_unstale":"2023-08-25T09:52:03.615995+0000","last_undegraded":"2023-08-25T09:52:03.615995+0000","last_fullsized":"2023-08-25T09:52:03.615995+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:24.261532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.609814+0000","last_change":"2023-08-25T09:52:03.609814+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.609814+0000","last_undegraded":"2023-08-25T09:52:03.609814+0000","last_fullsized":"2023-08-25T09:52:03.609814+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.614928+0000","last_change":"2023-08-25T09:52:03.614822+0000","last_active":"2023-08-25T09:52:03.614928+0000","last_peered":"2023-08-25T09:52:03.614928+0000","last_clean":"2023-08-25T09:52:03.614928+0000","last_became_active":"2023-08-25T09:52:03.613922+0000","last_became_peered":"2023-08-25T09:52:03.613922+0000","last_unstale":"2023-08-25T09:52:03.614928+0000","last_undegraded":"2023-08-25T09:52:03.614928+0000","last_fullsized":"2023-08-25T09:52:03.614928+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:22:02.268146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605404+0000","last_change":"2023-08-25T09:52:03.605404+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605404+0000","last_undegraded":"2023-08-25T09:52:03.605404+0000","last_fullsized":"2023-08-25T09:52:03.605404+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.612471+0000","last_change":"2023-08-25T09:52:03.610194+0000","last_active":"2023-08-25T09:52:03.612471+0000","last_peered":"2023-08-25T09:52:03.612471+0000","last_clean":"2023-08-25T09:52:03.612471+0000","last_became_active":"2023-08-25T09:52:03.609009+0000","last_became_peered":"2023-08-25T09:52:03.609009+0000","last_unstale":"2023-08-25T09:52:03.612471+0000","last_undegraded":"2023-08-25T09:52:03.612471+0000","last_fullsized":"2023-08-25T09:52:03.612471+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:46:32.529133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.606546+0000","last_change":"2023-08-25T09:52:03.606546+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.606546+0000","last_undegraded":"2023-08-25T09:52:03.606546+0000","last_fullsized":"2023-08-25T09:52:03.606546+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.608748+0000","last_change":"2023-08-25T09:52:03.608521+0000","last_active":"2023-08-25T09:52:03.608748+0000","last_peered":"2023-08-25T09:52:03.608748+0000","last_clean":"2023-08-25T09:52:03.608748+0000","last_became_active":"2023-08-25T09:52:03.605881+0000","last_became_peered":"2023-08-25T09:52:03.605881+0000","last_unstale":"2023-08-25T09:52:03.608748+0000","last_undegraded":"2023-08-25T09:52:03.608748+0000","last_fullsized":"2023-08-25T09:52:03.608748+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:31:19.665979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.593285+0000","last_change":"2023-08-25T09:52:03.593285+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.593285+0000","last_undegraded":"2023-08-25T09:52:03.593285+0000","last_fullsized":"2023-08-25T09:52:03.593285+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.614837+0000","last_change":"2023-08-25T09:52:03.614418+0000","last_active":"2023-08-25T09:52:03.614837+0000","last_peered":"2023-08-25T09:52:03.614837+0000","last_clean":"2023-08-25T09:52:03.614837+0000","last_became_active":"2023-08-25T09:52:03.614127+0000","last_became_peered":"2023-08-25T09:52:03.614127+0000","last_unstale":"2023-08-25T09:52:03.614837+0000","last_undegraded":"2023-08-25T09:52:03.614837+0000","last_fullsized":"2023-08-25T09:52:03.614837+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:42:20.246360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.604894+0000","last_change":"2023-08-25T09:52:03.604894+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.604894+0000","last_undegraded":"2023-08-25T09:52:03.604894+0000","last_fullsized":"2023-08-25T09:52:03.604894+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.619131+0000","last_change":"2023-08-25T09:52:03.619038+0000","last_active":"2023-08-25T09:52:03.619131+0000","last_peered":"2023-08-25T09:52:03.619131+0000","last_clean":"2023-08-25T09:52:03.619131+0000","last_became_active":"2023-08-25T09:52:03.618767+0000","last_became_peered":"2023-08-25T09:52:03.618767+0000","last_unstale":"2023-08-25T09:52:03.619131+0000","last_undegraded":"2023-08-25T09:52:03.619131+0000","last_fullsized":"2023-08-25T09:52:03.619131+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:05:24.291732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":475,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.598927+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:52:03.598927+0000","last_peered":"2023-08-25T09:52:03.598927+0000","last_clean":"2023-08-25T09:52:03.598927+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:52:03.598927+0000","last_undegraded":"2023-08-25T09:52:03.598927+0000","last_fullsized":"2023-08-25T09:52:03.598927+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:07.239907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.602122+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:52:03.602122+0000","last_peered":"2023-08-25T09:52:03.602122+0000","last_clean":"2023-08-25T09:52:03.602122+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:52:03.602122+0000","last_undegraded":"2023-08-25T09:52:03.602122+0000","last_fullsized":"2023-08-25T09:52:03.602122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:33:51.823664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595499+0000","last_change":"2023-08-25T09:52:03.595499+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595499+0000","last_undegraded":"2023-08-25T09:52:03.595499+0000","last_fullsized":"2023-08-25T09:52:03.595499+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.609570+0000","last_change":"2023-08-25T09:52:03.608811+0000","last_active":"2023-08-25T09:52:03.609570+0000","last_peered":"2023-08-25T09:52:03.609570+0000","last_clean":"2023-08-25T09:52:03.609570+0000","last_became_active":"2023-08-25T09:52:03.608140+0000","last_became_peered":"2023-08-25T09:52:03.608140+0000","last_unstale":"2023-08-25T09:52:03.609570+0000","last_undegraded":"2023-08-25T09:52:03.609570+0000","last_fullsized":"2023-08-25T09:52:03.609570+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:24:40.118198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.599796+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:52:03.599796+0000","last_peered":"2023-08-25T09:52:03.599796+0000","last_clean":"2023-08-25T09:52:03.599796+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:52:03.599796+0000","last_undegraded":"2023-08-25T09:52:03.599796+0000","last_fullsized":"2023-08-25T09:52:03.599796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:11:15.025918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.596135+0000","last_change":"2023-08-25T09:52:03.596135+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.596135+0000","last_undegraded":"2023-08-25T09:52:03.596135+0000","last_fullsized":"2023-08-25T09:52:03.596135+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.616009+0000","last_change":"2023-08-25T09:52:03.615496+0000","last_active":"2023-08-25T09:52:03.616009+0000","last_peered":"2023-08-25T09:52:03.616009+0000","last_clean":"2023-08-25T09:52:03.616009+0000","last_became_active":"2023-08-25T09:52:03.615224+0000","last_became_peered":"2023-08-25T09:52:03.615224+0000","last_unstale":"2023-08-25T09:52:03.616009+0000","last_undegraded":"2023-08-25T09:52:03.616009+0000","last_fullsized":"2023-08-25T09:52:03.616009+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:00.452072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":355,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.590094+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:52:03.590094+0000","last_peered":"2023-08-25T09:52:03.590094+0000","last_clean":"2023-08-25T09:52:03.590094+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:52:03.590094+0000","last_undegraded":"2023-08-25T09:52:03.590094+0000","last_fullsized":"2023-08-25T09:52:03.590094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:42:51.224218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.595733+0000","last_change":"2023-08-25T09:52:03.595733+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.595733+0000","last_undegraded":"2023-08-25T09:52:03.595733+0000","last_fullsized":"2023-08-25T09:52:03.595733+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.616028+0000","last_change":"2023-08-25T09:52:03.615906+0000","last_active":"2023-08-25T09:52:03.616028+0000","last_peered":"2023-08-25T09:52:03.616028+0000","last_clean":"2023-08-25T09:52:03.616028+0000","last_became_active":"2023-08-25T09:52:03.615475+0000","last_became_peered":"2023-08-25T09:52:03.615475+0000","last_unstale":"2023-08-25T09:52:03.616028+0000","last_undegraded":"2023-08-25T09:52:03.616028+0000","last_fullsized":"2023-08-25T09:52:03.616028+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:00:02.156359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.600084+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:52:03.600084+0000","last_peered":"2023-08-25T09:52:03.600084+0000","last_clean":"2023-08-25T09:52:03.600084+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:52:03.600084+0000","last_undegraded":"2023-08-25T09:52:03.600084+0000","last_fullsized":"2023-08-25T09:52:03.600084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:56:48.511610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.605516+0000","last_change":"2023-08-25T09:52:03.605516+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.605516+0000","last_undegraded":"2023-08-25T09:52:03.605516+0000","last_fullsized":"2023-08-25T09:52:03.605516+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.613201+0000","last_change":"2023-08-25T09:52:03.613108+0000","last_active":"2023-08-25T09:52:03.613201+0000","last_peered":"2023-08-25T09:52:03.613201+0000","last_clean":"2023-08-25T09:52:03.613201+0000","last_became_active":"2023-08-25T09:52:03.612787+0000","last_became_peered":"2023-08-25T09:52:03.612787+0000","last_unstale":"2023-08-25T09:52:03.613201+0000","last_undegraded":"2023-08-25T09:52:03.613201+0000","last_fullsized":"2023-08-25T09:52:03.613201+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:30.915836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":321,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.590217+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:52:03.590217+0000","last_peered":"2023-08-25T09:52:03.590217+0000","last_clean":"2023-08-25T09:52:03.590217+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:52:03.590217+0000","last_undegraded":"2023-08-25T09:52:03.590217+0000","last_fullsized":"2023-08-25T09:52:03.590217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:17:35.995249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.616369+0000","last_change":"2023-08-25T09:52:03.616369+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.616369+0000","last_undegraded":"2023-08-25T09:52:03.616369+0000","last_fullsized":"2023-08-25T09:52:03.616369+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.615749+0000","last_change":"2023-08-25T09:52:03.615255+0000","last_active":"2023-08-25T09:52:03.615749+0000","last_peered":"2023-08-25T09:52:03.615749+0000","last_clean":"2023-08-25T09:52:03.615749+0000","last_became_active":"2023-08-25T09:52:03.613680+0000","last_became_peered":"2023-08-25T09:52:03.613680+0000","last_unstale":"2023-08-25T09:52:03.615749+0000","last_undegraded":"2023-08-25T09:52:03.615749+0000","last_fullsized":"2023-08-25T09:52:03.615749+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:49:11.723113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.599484+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:52:03.599484+0000","last_peered":"2023-08-25T09:52:03.599484+0000","last_clean":"2023-08-25T09:52:03.599484+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:52:03.599484+0000","last_undegraded":"2023-08-25T09:52:03.599484+0000","last_fullsized":"2023-08-25T09:52:03.599484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:55:37.067424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.601630+0000","last_change":"2023-08-25T09:52:03.601630+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.601630+0000","last_undegraded":"2023-08-25T09:52:03.601630+0000","last_fullsized":"2023-08-25T09:52:03.601630+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.608543+0000","last_change":"2023-08-25T09:52:03.608367+0000","last_active":"2023-08-25T09:52:03.608543+0000","last_peered":"2023-08-25T09:52:03.608543+0000","last_clean":"2023-08-25T09:52:03.608543+0000","last_became_active":"2023-08-25T09:52:03.605716+0000","last_became_peered":"2023-08-25T09:52:03.605716+0000","last_unstale":"2023-08-25T09:52:03.608543+0000","last_undegraded":"2023-08-25T09:52:03.608543+0000","last_fullsized":"2023-08-25T09:52:03.608543+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:18:13.745741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":321,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.599505+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:52:03.599505+0000","last_peered":"2023-08-25T09:52:03.599505+0000","last_clean":"2023-08-25T09:52:03.599505+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:52:03.599505+0000","last_undegraded":"2023-08-25T09:52:03.599505+0000","last_fullsized":"2023-08-25T09:52:03.599505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:08:03.681656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":196,"state":"creating+peering","last_fresh":"2023-08-25T09:52:03.594627+0000","last_change":"2023-08-25T09:52:03.594627+0000","last_active":"2023-08-25T09:52:03.577494+0000","last_peered":"2023-08-25T09:52:03.577494+0000","last_clean":"2023-08-25T09:52:03.577494+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T09:52:03.594627+0000","last_undegraded":"2023-08-25T09:52:03.594627+0000","last_fullsized":"2023-08-25T09:52:03.594627+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.612060+0000","last_change":"2023-08-25T09:52:03.611964+0000","last_active":"2023-08-25T09:52:03.612060+0000","last_peered":"2023-08-25T09:52:03.612060+0000","last_clean":"2023-08-25T09:52:03.612060+0000","last_became_active":"2023-08-25T09:52:03.611724+0000","last_became_peered":"2023-08-25T09:52:03.611724+0000","last_unstale":"2023-08-25T09:52:03.612060+0000","last_undegraded":"2023-08-25T09:52:03.612060+0000","last_fullsized":"2023-08-25T09:52:03.612060+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:55:08.582145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-08-25T09:52:03.598776+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:52:03.598776+0000","last_peered":"2023-08-25T09:52:03.598776+0000","last_clean":"2023-08-25T09:52:03.598776+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:52:03.598776+0000","last_undegraded":"2023-08-25T09:52:03.598776+0000","last_fullsized":"2023-08-25T09:52:03.598776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:30:43.545960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194168,"statfs":{"total":200000000,"available":198828593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.179,"15min":1.1259999999999999},"min":{"1min":0.754,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.829,"5min":1.829,"15min":10.747},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.1539999999999999,"15min":1.0980000000000001},"min":{"1min":0.69699999999999995,"5min":0.61699999999999999,"15min":0.43099999999999999},"max":{"1min":1.637,"5min":1.651,"15min":1.855},"last":1.3899999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.28,"15min":1.2370000000000001},"min":{"1min":0.66000000000000003,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.7390000000000001,"5min":1.823,"15min":11.010999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.2689999999999999,"15min":1.177},"min":{"1min":0.83699999999999997,"5min":0.70899999999999996,"15min":0.57999999999999996},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1830000000000001,"15min":1.137},"min":{"1min":0.85799999999999998,"5min":0.66800000000000004,"15min":0.50600000000000001},"max":{"1min":1.46,"5min":1.5529999999999999,"15min":1.619},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.323,"5min":1.304,"15min":1.27},"min":{"1min":0.99199999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.8149999999999999,"5min":1.873,"15min":1.98},"last":1.0069999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.23,"15min":1.21},"min":{"1min":0.81399999999999995,"5min":0.59799999999999998,"15min":0.56799999999999995},"max":{"1min":1.5680000000000001,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.456},{"interface":"front","average":{"1min":1.3560000000000001,"5min":1.385,"15min":1.363},"min":{"1min":1.018,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.7270000000000001,"5min":1.833,"15min":2.1030000000000002},"last":1.135}]},{"osd":4,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.268,"15min":1.2869999999999999},"min":{"1min":0.82999999999999996,"5min":0.51300000000000001,"15min":0.48599999999999999},"max":{"1min":1.7010000000000001,"5min":1.8879999999999999,"15min":7.1639999999999997},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3220000000000001,"15min":1.321},"min":{"1min":0.94599999999999995,"5min":0.53800000000000003,"15min":0.46300000000000002},"max":{"1min":1.74,"5min":1.798,"15min":1.8779999999999999},"last":1.5700000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.468,"5min":1.46,"15min":1.411},"min":{"1min":0.90700000000000003,"5min":0.64100000000000001,"15min":0.52700000000000002},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.522},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.468,"15min":1.4279999999999999},"min":{"1min":1.1060000000000001,"5min":0.67700000000000005,"15min":0.64100000000000001},"max":{"1min":1.907,"5min":2.0289999999999999,"15min":2.0739999999999998},"last":1.6890000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5389999999999999,"5min":1.5589999999999999,"15min":1.51},"min":{"1min":1.3400000000000001,"5min":0.72999999999999998,"15min":0.68799999999999994},"max":{"1min":1.984,"5min":2.0259999999999998,"15min":2.1720000000000002},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.488,"15min":1.456},"min":{"1min":0.90600000000000003,"5min":0.63500000000000001,"15min":0.61399999999999999},"max":{"1min":1.877,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6339999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.1539999999999999,"15min":1.1459999999999999},"min":{"1min":0.78800000000000003,"5min":0.68700000000000006,"15min":0.54400000000000004},"max":{"1min":1.4850000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.7310000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.1779999999999999,"15min":1.1659999999999999},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5980000000000001,"5min":1.992,"15min":5.5410000000000004},"last":0.97299999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.222,"15min":1.2110000000000001},"min":{"1min":0.78800000000000003,"5min":0.77000000000000002,"15min":0.54700000000000004},"max":{"1min":1.8380000000000001,"5min":2.198,"15min":5.6799999999999997},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1919999999999999,"15min":1.1739999999999999},"min":{"1min":0.79000000000000004,"5min":0.66800000000000004,"15min":0.50800000000000001},"max":{"1min":1.6220000000000001,"5min":2.3079999999999998,"15min":2.3079999999999998},"last":0.91100000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1739999999999999,"15min":1.147},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.526,"5min":1.631,"15min":1.708},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.288,"15min":1.288},"min":{"1min":0.88700000000000001,"5min":0.67000000000000004,"15min":0.47299999999999998},"max":{"1min":1.6140000000000001,"5min":1.7969999999999999,"15min":13.193},"last":1.093}]},{"osd":3,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.345,"15min":1.2949999999999999},"min":{"1min":0.93899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7010000000000001,"5min":2.3879999999999999,"15min":5.6239999999999997},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.234,"5min":1.264,"15min":1.2649999999999999},"min":{"1min":0.85099999999999998,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.3400000000000001,"15min":1.3100000000000001},"min":{"1min":0.93999999999999995,"5min":0.55300000000000005,"15min":0.42599999999999999},"max":{"1min":1.7609999999999999,"5min":2.1110000000000002,"15min":2.484},"last":1.165},{"interface":"front","average":{"1min":1.454,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.78900000000000003,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.101,"5min":2.5550000000000002,"15min":5.4699999999999998},"last":1.2969999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.3640000000000001,"15min":1.3169999999999999},"min":{"1min":0.75,"5min":0.65000000000000002,"15min":0.59599999999999997},"max":{"1min":1.831,"5min":1.8640000000000001,"15min":2.0049999999999999},"last":1.9670000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.4770000000000001,"15min":1.444},"min":{"1min":1.0980000000000001,"5min":0.61099999999999999,"15min":0.59099999999999997},"max":{"1min":1.716,"5min":2.4420000000000002,"15min":2.4420000000000002},"last":1.534}]},{"osd":7,"last update":"Fri Aug 25 09:50:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.518,"15min":1.518},"min":{"1min":0.89400000000000002,"5min":0.63200000000000001,"15min":0.622},"max":{"1min":1.917,"5min":2.6579999999999999,"15min":6.7709999999999999},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.49,"5min":1.532,"15min":1.494},"min":{"1min":0.872,"5min":0.81699999999999995,"15min":0.57399999999999995},"max":{"1min":2.1960000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":2.0590000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575011,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1720,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193592,"statfs":{"total":200000000,"available":198238225,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2529999999999999,"15min":1.196},"min":{"1min":0.753,"5min":0.64000000000000001,"15min":0.497},"max":{"1min":1.845,"5min":1.9770000000000001,"15min":2.081},"last":1.024},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2529999999999999,"15min":1.22},"min":{"1min":0.747,"5min":0.747,"15min":0.65400000000000003},"max":{"1min":1.621,"5min":1.8080000000000001,"15min":1.974},"last":1.1479999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.177,"15min":1.1719999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.6499999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.1559999999999999,"15min":1.153},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.492,"5min":1.704,"15min":2.1869999999999998},"last":1.111}]},{"osd":2,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.296,"15min":1.2370000000000001},"min":{"1min":0.874,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.772,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.204},{"interface":"front","average":{"1min":1.22,"5min":1.2869999999999999,"15min":1.252},"min":{"1min":0.75,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":0.83099999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.2290000000000001,"15min":1.2829999999999999},"min":{"1min":0.66600000000000004,"5min":0.65900000000000003,"15min":0.57999999999999996},"max":{"1min":1.597,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.3540000000000001,"15min":1.3200000000000001},"min":{"1min":0.87,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.7410000000000001,"5min":1.964,"15min":1.9650000000000001},"last":0.95299999999999996}]},{"osd":4,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.3169999999999999,"15min":1.3129999999999999},"min":{"1min":0.91400000000000003,"5min":0.58299999999999996,"15min":0.55500000000000005},"max":{"1min":1.671,"5min":1.921,"15min":2.625},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3220000000000001,"15min":1.3420000000000001},"min":{"1min":0.90500000000000003,"5min":0.61299999999999999,"15min":0.58799999999999997},"max":{"1min":1.7290000000000001,"5min":2.1230000000000002,"15min":2.2789999999999999},"last":1.2470000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4790000000000001,"15min":1.446},"min":{"1min":1.131,"5min":0.72099999999999997,"15min":0.58899999999999997},"max":{"1min":1.794,"5min":2.048,"15min":2.4820000000000002},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.5169999999999999,"15min":1.4750000000000001},"min":{"1min":1.268,"5min":0.65500000000000003,"15min":0.60399999999999998},"max":{"1min":1.901,"5min":2.044,"15min":2.5539999999999998},"last":1.4279999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:51:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.4910000000000001,"15min":1.46},"min":{"1min":1.099,"5min":0.85399999999999998,"15min":0.63},"max":{"1min":1.8069999999999999,"5min":2.2719999999999998,"15min":2.2719999999999998},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.585,"5min":1.5920000000000001,"15min":1.581},"min":{"1min":1.22,"5min":0.75600000000000001,"15min":0.63900000000000001},"max":{"1min":1.8899999999999999,"5min":2.3580000000000001,"15min":2.3700000000000001},"last":1.2869999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.28,"15min":1.224},"min":{"1min":0.624,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.589,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.405},{"interface":"front","average":{"1min":1.264,"5min":1.2709999999999999,"15min":1.2490000000000001},"min":{"1min":0.91600000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.615,"5min":2.5150000000000001,"15min":7.367},"last":1.135}]},{"osd":1,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.466,"5min":1.7330000000000001,"15min":2.032},"last":0.88},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.234,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.571,"5min":4.351,"15min":8.5850000000000009},"last":1.3200000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.214},"min":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.556,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.294,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5940000000000001,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.5209999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.284,"15min":1.2629999999999999},"min":{"1min":0.79400000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.665,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.351},{"interface":"front","average":{"1min":1.359,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.911},"last":1.548}]},{"osd":5,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3140000000000001,"15min":1.3009999999999999},"min":{"1min":0.67100000000000004,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.7370000000000001,"5min":5.048,"15min":5.048},"last":1.214},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.2729999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4179999999999999,"15min":1.4319999999999999},"min":{"1min":0.88100000000000001,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":1.927,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.597,"15min":1.579},"min":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.968,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.496,"15min":1.49},"min":{"1min":0.75700000000000001,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.865,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.494},{"interface":"front","average":{"1min":1.504,"5min":1.569,"15min":1.5920000000000001},"min":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8340000000000001,"5min":3.831,"15min":3.831},"last":1.4450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98599999999999999,"15min":0.91000000000000003},"min":{"1min":0.628,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.8180000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0649999999999999,"15min":0.98199999999999998},"min":{"1min":0.69999999999999996,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.579,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.089}]},{"osd":1,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.64600000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.258,"5min":1.413,"15min":1.702},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.087,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.623,"5min":1.6339999999999999,"15min":2.3399999999999999},"last":0.94399999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.034,"15min":0.96499999999999997},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.5980000000000001,"5min":1.623,"15min":1.919},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2050000000000001,"15min":1.1439999999999999},"min":{"1min":0.745,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.901,"5min":1.901,"15min":3.399},"last":1.137}]},{"osd":4,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3009999999999999,"15min":1.236},"min":{"1min":1.1399999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.204,"15min":1.2090000000000001},"min":{"1min":1.0109999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.405,"5min":1.79,"15min":2.1840000000000002},"last":1.341}]},{"osd":5,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3480000000000001},"min":{"1min":1.137,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":2.1509999999999998,"5min":2.1930000000000001,"15min":2.407},"last":1.284},{"interface":"front","average":{"1min":1.357,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.0349999999999999,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.804,"5min":1.895,"15min":1.895},"last":1.2430000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.302,"15min":1.282},"min":{"1min":1.054,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5980000000000001,"5min":1.875,"15min":2.3450000000000002},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.47,"15min":1.4199999999999999},"min":{"1min":1.28,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":2.2330000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.3700000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.464,"15min":1.405},"min":{"1min":1.0840000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.976,"5min":1.976,"15min":2.2989999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.3839999999999999},"min":{"1min":1.1259999999999999,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":1.7589999999999999,"5min":1.9350000000000001,"15min":2.1509999999999998},"last":1.444}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194168,"statfs":{"total":200000000,"available":198828593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.6930000000000001,"5min":1.706,"15min":2.383},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.92100000000000004,"15min":0.97699999999999998},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.38400000000000001},"max":{"1min":1.367,"5min":1.645,"15min":2.7309999999999999},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.95399999999999996,"15min":0.98299999999999998},"min":{"1min":0.47599999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.446,"5min":1.978,"15min":2.3180000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.02,"15min":1.04},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.528,"5min":1.704,"15min":2.8599999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.92600000000000005,"15min":1.0580000000000001},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.341,"5min":1.6839999999999999,"15min":18.821000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.077,"15min":1.097},"min":{"1min":0.54600000000000004,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.486,"5min":1.8999999999999999,"15min":2.2040000000000002},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2689999999999999,"15min":1.329},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9139999999999999,"5min":2.0059999999999998,"15min":3.0150000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.2549999999999999,"15min":1.294},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.8,"5min":1.8,"15min":2.7639999999999998},"last":1.6459999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.2569999999999999,"15min":1.296},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.4990000000000001,"5min":3.7559999999999998,"15min":5.1269999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.3400000000000001,"15min":1.371},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0920000000000001,"15min":2.4220000000000002},"last":1.3240000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.3979999999999999,"15min":1.403},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.8109999999999999,"5min":4.0170000000000003,"15min":4.0170000000000003},"last":1.7769999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.409,"15min":1.4299999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.76,"5min":3.4140000000000001,"15min":5.0599999999999996},"last":1.23}]},{"osd":7,"last update":"Fri Aug 25 09:51:01 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.367,"15min":1.3999999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.706,"5min":2.3149999999999999,"15min":2.968},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.3819999999999999,"15min":1.4379999999999999},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7330000000000001,"5min":2.96,"15min":5.2990000000000004},"last":1.0740000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575011,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1726,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193585,"statfs":{"total":200000000,"available":198231613,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.82699999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.29699999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97899999999999998,"15min":0.89800000000000002},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.7869999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.88100000000000001},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.35899999999999999},"max":{"1min":1.2809999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.022,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.389,"5min":1.6619999999999999,"15min":1.746},"last":1.153}]},{"osd":3,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.109},"min":{"1min":0.496,"5min":0.496,"15min":0.372},"max":{"1min":1.647,"5min":1.647,"15min":1.9450000000000001},"last":1.419},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0109999999999999},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.333,"5min":1.655,"15min":1.764},"last":1.0409999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.89300000000000002},{"interface":"front","average":{"1min":1.254,"5min":1.2929999999999999,"15min":1.276},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":1.6830000000000001,"5min":1.9419999999999999,"15min":2.008},"last":1.5529999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.2430000000000001},"min":{"1min":0.94699999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.665,"15min":1.9650000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3049999999999999,"15min":1.2450000000000001},"min":{"1min":0.82699999999999996,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.677,"5min":2.867,"15min":2.867},"last":0.97399999999999998}]},{"osd":6,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.363},"min":{"1min":1.016,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":1.633,"5min":2.0779999999999998,"15min":2.9940000000000002},"last":1.605},{"interface":"front","average":{"1min":1.331,"5min":1.411,"15min":1.3660000000000001},"min":{"1min":0.95099999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.603,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.5109999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4490000000000001,"15min":1.4139999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.79200000000000004},"max":{"1min":1.772,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.431,"15min":1.399},"min":{"1min":0.88200000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.738,"5min":1.913,"15min":2.218},"last":1.1870000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1150,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194162,"statfs":{"total":200000000,"available":198821981,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.88900000000000001,"15min":0.88500000000000001},"min":{"1min":0.41499999999999998,"5min":0.40699999999999997,"15min":0.313},"max":{"1min":1.2929999999999999,"5min":2.2759999999999998,"15min":11.257},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.94899999999999995,"15min":0.96099999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.405,"5min":1.829,"15min":6.4260000000000002},"last":1.107}]},{"osd":2,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.98899999999999999,"15min":0.95899999999999996},"min":{"1min":0.45300000000000001,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.522,"5min":1.8620000000000001,"15min":1.8620000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.94099999999999995,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.432,"5min":0.374,"15min":0.374},"max":{"1min":1.5860000000000001,"5min":1.714,"15min":3.4889999999999999},"last":1.0249999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.96499999999999997,"15min":0.92400000000000004},"min":{"1min":0.46500000000000002,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.532,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.1240000000000001,"15min":1.1160000000000001},"min":{"1min":0.48099999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5860000000000001,"5min":1.825,"15min":2.3809999999999998},"last":0.745}]},{"osd":4,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.278,"15min":1.2529999999999999},"min":{"1min":0.83599999999999997,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.782,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.3740000000000001},"min":{"1min":0.82399999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7589999999999999,"5min":2.0339999999999998,"15min":2.5499999999999998},"last":1.282}]},{"osd":5,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.3129999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.599,"5min":2.0939999999999999,"15min":3.3769999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.27,"15min":1.276},"min":{"1min":0.747,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6299999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.079}]},{"osd":6,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4119999999999999,"15min":1.3600000000000001},"min":{"1min":0.88900000000000001,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.9910000000000001,"5min":2.1480000000000001,"15min":2.6869999999999998},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3640000000000001,"15min":1.329},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.863,"5min":1.9710000000000001,"15min":2.331},"last":1.1839999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:51:07 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.423,"15min":1.405},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.669,"5min":2.4590000000000001,"15min":2.6429999999999998},"last":1.143},{"interface":"front","average":{"1min":1.385,"5min":1.3740000000000001,"15min":1.3380000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.847,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.157}]}]}],"pool_statfs":[]}} 2023-08-25T09:52:09.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:52:09.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.825+0000 7f7c00c7c640 1 Processor -- start 2023-08-25T09:52:09.828 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.825+0000 7f7c00c7c640 1 -- start start 2023-08-25T09:52:09.829 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.827+0000 7f7c00c7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 0x7f7bfc0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.827+0000 7f7c00c7c640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc1615b0 con 0x7f7bfc0ac870 2023-08-25T09:52:09.830 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.827+0000 7f7c00c7c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc06f050 con 0x7f7bfc0b2b40 2023-08-25T09:52:09.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.827+0000 7f7c00c7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc06f330 con 0x7f7bfc15cb40 2023-08-25T09:52:09.831 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.827+0000 7f7bf3fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7bfc0ac870 0x7f7bfc160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45776/0 (socket says 172.21.15.70:45776) 2023-08-25T09:52:09.832 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.827+0000 7f7bf3fff640 1 -- 172.21.15.70:0/2658271602 learned_addr learned my addr 172.21.15.70:0/2658271602 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:09.840 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736705075 0 0) 0x7f7bfc1615b0 con 0x7f7bfc0ac870 2023-08-25T09:52:09.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7be8003520 con 0x7f7bfc0ac870 2023-08-25T09:52:09.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bfa59c640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 0x7f7bfc0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2964158957 0 0) 0x7f7be8003520 con 0x7f7bfc0ac870 2023-08-25T09:52:09.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 msgr2=0x7f7bfc0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:09.841 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 0x7f7bfc0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 >> v1:172.21.15.155:6789/0 conn(0x7f7bfc0b2b40 legacy=0x7f7bfc160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc06f610 con 0x7f7bfc0ac870 2023-08-25T09:52:09.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f7be4001670 con 0x7f7bfc0ac870 2023-08-25T09:52:09.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 >> v1:172.21.15.70:6789/0 conn(0x7f7bfc0ac870 legacy=0x7f7bfc160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7be80039a0 0x7f7be8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 0x7f7be80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7be8004d00 0x7f7be8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc06f330 con 0x7f7be8004d00 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be800bb10 con 0x7f7be80039a0 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.828+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be800bca0 con 0x7f7bfc15cb40 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7be8004d00 0x7f7be8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfa59c640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7be80039a0 0x7f7be8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bf3fff640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 0x7f7be80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.844 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 msgr2=0x7f7be80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:09.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7bfc15cb40 0x7f7be80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7be80039a0 msgr2=0x7f7be8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:09.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7be80039a0 0x7f7be8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.829+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2658271602 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8003520 con 0x7f7be8004d00 2023-08-25T09:52:09.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.830+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7be8004d00 0x7f7be8005180 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4001e50 tx=0x7f7be80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af28e34c8e249a79 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:09.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4003b70 con 0x7f7be8004d00 2023-08-25T09:52:09.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4003cd0 con 0x7f7be8004d00 2023-08-25T09:52:09.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7bf9d9b640 1 -- 172.21.15.70:0/2658271602 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4005310 con 0x7f7be8004d00 2023-08-25T09:52:09.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2658271602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7be8004d00 msgr2=0x7f7be8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2658271602 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7be8004d00 0x7f7be8005180 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4001e50 tx=0x7f7be80057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2658271602 reap_dead start 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2658271602 shutdown_connections 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2658271602 >> 172.21.15.70:0/2658271602 conn(0x7f7bfc069850 msgr2=0x7f7bfc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2658271602 shutdown_connections 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.831+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2658271602 wait complete. 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.833+0000 7f7c00c7c640 1 Processor -- start 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.833+0000 7f7c00c7c640 1 -- start start 2023-08-25T09:52:09.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.834+0000 7f7c00c7c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7be8004d00 0x7f7bfc1600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.834+0000 7f7c00c7c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bfc150340 0x7f7bfc15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.834+0000 7f7c00c7c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 0x7f7bfc13e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.834+0000 7f7c00c7c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15fca0 con 0x7f7bfc15d460 2023-08-25T09:52:09.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.834+0000 7f7c00c7c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15d910 con 0x7f7bfc150340 2023-08-25T09:52:09.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.834+0000 7f7c00c7c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc15f370 con 0x7f7be8004d00 2023-08-25T09:52:09.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 0x7f7bfc13e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 0x7f7bfc13e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43076/0 (socket says 172.21.15.70:43076) 2023-08-25T09:52:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2194470338 learned_addr learned my addr 172.21.15.70:0/2194470338 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfa59c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7be8004d00 0x7f7bfc1600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bf3fff640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bfc150340 0x7f7bfc15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7be8004d00 msgr2=0x7f7bfc1600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7be8004d00 0x7f7bfc1600c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bfc150340 msgr2=0x7f7bfc15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bfc150340 0x7f7bfc15cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 -- 172.21.15.70:0/2194470338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc15f5e0 con 0x7f7bfc15d460 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfa59c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7be8004d00 0x7f7bfc1600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bfad9d640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 0x7f7bfc13e960 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f7bf402c020 tx=0x7f7bf40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.835+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4006d50 con 0x7f7bfc15d460 2023-08-25T09:52:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.836+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf4003e40 con 0x7f7bfc15d460 2023-08-25T09:52:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.836+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc15f8c0 con 0x7f7bfc15d460 2023-08-25T09:52:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.836+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4007270 con 0x7f7bfc15d460 2023-08-25T09:52:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.836+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc1507c0 con 0x7f7bfc15d460 2023-08-25T09:52:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.837+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc15fa50 con 0x7f7bfc15d460 2023-08-25T09:52:09.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.838+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7bf40073d0 con 0x7f7bfc15d460 2023-08-25T09:52:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.839+0000 7f7bf2ffd640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7bdc06a050 0x7f7bdc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.839+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7bf40a9cd0 con 0x7f7bfc15d460 2023-08-25T09:52:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.839+0000 7f7bfa59c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7bdc06a050 0x7f7bdc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:09.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.840+0000 7f7bfa59c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7bdc06a050 0x7f7bdc06c510 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc069fa0 tx=0x7f7bec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:09.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:09.843+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7bf4075320 con 0x7f7bfc15d460 2023-08-25T09:52:10.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.025+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bfc06fbd0 con 0x7f7bdc06a050 2023-08-25T09:52:10.048 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.045+0000 7f7bf2ffd640 1 -- 172.21.15.70:0/2194470338 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302974 (secure 0 0 0) 0x7f7bec00b670 con 0x7f7bdc06a050 2023-08-25T09:52:10.049 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:10.052 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:52:10.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7bdc06a050 msgr2=0x7f7bdc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7bdc06a050 0x7f7bdc06c510 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc069fa0 tx=0x7f7bec008040 comp rx=0 tx=0).stop 2023-08-25T09:52:10.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 msgr2=0x7f7bfc13e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 0x7f7bfc13e960 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f7bf402c020 tx=0x7f7bf40014a0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 shutdown_connections 2023-08-25T09:52:10.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7bdc06a050 0x7f7bdc06c510 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7bfc15d460 0x7f7bfc13e960 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7bfc150340 0x7f7bfc15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 --2- 172.21.15.70:0/2194470338 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7be8004d00 0x7f7bfc1600c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 >> 172.21.15.70:0/2194470338 conn(0x7f7bfc069850 msgr2=0x7f7bfc0ae190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:10.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 shutdown_connections 2023-08-25T09:52:10.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.051+0000 7f7c00c7c640 1 -- 172.21.15.70:0/2194470338 wait complete. 2023-08-25T09:52:10.071 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2023-08-25T09:52:09.592743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10468,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552031,"statfs":{"total":1600000000,"available":1589280044,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.005760"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978214+0000","last_change":"2023-08-25T09:52:04.628641+0000","last_active":"2023-08-25T09:52:06.978214+0000","last_peered":"2023-08-25T09:52:06.978214+0000","last_clean":"2023-08-25T09:52:06.978214+0000","last_became_active":"2023-08-25T09:52:04.627891+0000","last_became_peered":"2023-08-25T09:52:04.627891+0000","last_unstale":"2023-08-25T09:52:06.978214+0000","last_undegraded":"2023-08-25T09:52:06.978214+0000","last_fullsized":"2023-08-25T09:52:06.978214+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:41:47.154534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979875+0000","last_change":"2023-08-25T09:52:04.625251+0000","last_active":"2023-08-25T09:52:06.979875+0000","last_peered":"2023-08-25T09:52:06.979875+0000","last_clean":"2023-08-25T09:52:06.979875+0000","last_became_active":"2023-08-25T09:52:04.624800+0000","last_became_peered":"2023-08-25T09:52:04.624800+0000","last_unstale":"2023-08-25T09:52:06.979875+0000","last_undegraded":"2023-08-25T09:52:06.979875+0000","last_fullsized":"2023-08-25T09:52:06.979875+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:41:47.867907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978509+0000","last_change":"2023-08-25T09:52:04.617930+0000","last_active":"2023-08-25T09:52:06.978509+0000","last_peered":"2023-08-25T09:52:06.978509+0000","last_clean":"2023-08-25T09:52:06.978509+0000","last_became_active":"2023-08-25T09:52:04.614336+0000","last_became_peered":"2023-08-25T09:52:04.614336+0000","last_unstale":"2023-08-25T09:52:06.978509+0000","last_undegraded":"2023-08-25T09:52:06.978509+0000","last_fullsized":"2023-08-25T09:52:06.978509+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:06.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977490+0000","last_change":"2023-08-25T09:52:04.627289+0000","last_active":"2023-08-25T09:52:06.977490+0000","last_peered":"2023-08-25T09:52:06.977490+0000","last_clean":"2023-08-25T09:52:06.977490+0000","last_became_active":"2023-08-25T09:52:04.625811+0000","last_became_peered":"2023-08-25T09:52:04.625811+0000","last_unstale":"2023-08-25T09:52:06.977490+0000","last_undegraded":"2023-08-25T09:52:06.977490+0000","last_fullsized":"2023-08-25T09:52:06.977490+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:21:46.850988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598128+0000","last_change":"2023-08-25T09:52:04.617186+0000","last_active":"2023-08-25T09:52:05.598128+0000","last_peered":"2023-08-25T09:52:05.598128+0000","last_clean":"2023-08-25T09:52:05.598128+0000","last_became_active":"2023-08-25T09:52:04.616036+0000","last_became_peered":"2023-08-25T09:52:04.616036+0000","last_unstale":"2023-08-25T09:52:05.598128+0000","last_undegraded":"2023-08-25T09:52:05.598128+0000","last_fullsized":"2023-08-25T09:52:05.598128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:27.953431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978450+0000","last_change":"2023-08-25T09:52:04.628367+0000","last_active":"2023-08-25T09:52:06.978450+0000","last_peered":"2023-08-25T09:52:06.978450+0000","last_clean":"2023-08-25T09:52:06.978450+0000","last_became_active":"2023-08-25T09:52:04.626874+0000","last_became_peered":"2023-08-25T09:52:04.626874+0000","last_unstale":"2023-08-25T09:52:06.978450+0000","last_undegraded":"2023-08-25T09:52:06.978450+0000","last_fullsized":"2023-08-25T09:52:06.978450+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:04.237329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599734+0000","last_change":"2023-08-25T09:52:04.612395+0000","last_active":"2023-08-25T09:52:05.599734+0000","last_peered":"2023-08-25T09:52:05.599734+0000","last_clean":"2023-08-25T09:52:05.599734+0000","last_became_active":"2023-08-25T09:52:04.607152+0000","last_became_peered":"2023-08-25T09:52:04.607152+0000","last_unstale":"2023-08-25T09:52:05.599734+0000","last_undegraded":"2023-08-25T09:52:05.599734+0000","last_fullsized":"2023-08-25T09:52:05.599734+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:51.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978190+0000","last_change":"2023-08-25T09:52:04.624655+0000","last_active":"2023-08-25T09:52:06.978190+0000","last_peered":"2023-08-25T09:52:06.978190+0000","last_clean":"2023-08-25T09:52:06.978190+0000","last_became_active":"2023-08-25T09:52:04.624478+0000","last_became_peered":"2023-08-25T09:52:04.624478+0000","last_unstale":"2023-08-25T09:52:06.978190+0000","last_undegraded":"2023-08-25T09:52:06.978190+0000","last_fullsized":"2023-08-25T09:52:06.978190+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:34.884721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599300+0000","last_change":"2023-08-25T09:52:04.626361+0000","last_active":"2023-08-25T09:52:05.599300+0000","last_peered":"2023-08-25T09:52:05.599300+0000","last_clean":"2023-08-25T09:52:05.599300+0000","last_became_active":"2023-08-25T09:52:04.625812+0000","last_became_peered":"2023-08-25T09:52:04.625812+0000","last_unstale":"2023-08-25T09:52:05.599300+0000","last_undegraded":"2023-08-25T09:52:05.599300+0000","last_fullsized":"2023-08-25T09:52:05.599300+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:04.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598504+0000","last_change":"2023-08-25T09:52:04.612349+0000","last_active":"2023-08-25T09:52:05.598504+0000","last_peered":"2023-08-25T09:52:05.598504+0000","last_clean":"2023-08-25T09:52:05.598504+0000","last_became_active":"2023-08-25T09:52:04.611885+0000","last_became_peered":"2023-08-25T09:52:04.611885+0000","last_unstale":"2023-08-25T09:52:05.598504+0000","last_undegraded":"2023-08-25T09:52:05.598504+0000","last_fullsized":"2023-08-25T09:52:05.598504+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:36.629581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978698+0000","last_change":"2023-08-25T09:52:04.627125+0000","last_active":"2023-08-25T09:52:06.978698+0000","last_peered":"2023-08-25T09:52:06.978698+0000","last_clean":"2023-08-25T09:52:06.978698+0000","last_became_active":"2023-08-25T09:52:04.625250+0000","last_became_peered":"2023-08-25T09:52:04.625250+0000","last_unstale":"2023-08-25T09:52:06.978698+0000","last_undegraded":"2023-08-25T09:52:06.978698+0000","last_fullsized":"2023-08-25T09:52:06.978698+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:34.884721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977263+0000","last_change":"2023-08-25T09:52:04.624230+0000","last_active":"2023-08-25T09:52:06.977263+0000","last_peered":"2023-08-25T09:52:06.977263+0000","last_clean":"2023-08-25T09:52:06.977263+0000","last_became_active":"2023-08-25T09:52:04.623981+0000","last_became_peered":"2023-08-25T09:52:04.623981+0000","last_unstale":"2023-08-25T09:52:06.977263+0000","last_undegraded":"2023-08-25T09:52:06.977263+0000","last_fullsized":"2023-08-25T09:52:06.977263+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:18:08.663525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601907+0000","last_change":"2023-08-25T09:52:04.624126+0000","last_active":"2023-08-25T09:52:05.601907+0000","last_peered":"2023-08-25T09:52:05.601907+0000","last_clean":"2023-08-25T09:52:05.601907+0000","last_became_active":"2023-08-25T09:52:04.623878+0000","last_became_peered":"2023-08-25T09:52:04.623878+0000","last_unstale":"2023-08-25T09:52:05.601907+0000","last_undegraded":"2023-08-25T09:52:05.601907+0000","last_fullsized":"2023-08-25T09:52:05.601907+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:58.851337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977983+0000","last_change":"2023-08-25T09:52:04.620817+0000","last_active":"2023-08-25T09:52:06.977983+0000","last_peered":"2023-08-25T09:52:06.977983+0000","last_clean":"2023-08-25T09:52:06.977983+0000","last_became_active":"2023-08-25T09:52:04.620537+0000","last_became_peered":"2023-08-25T09:52:04.620537+0000","last_unstale":"2023-08-25T09:52:06.977983+0000","last_undegraded":"2023-08-25T09:52:06.977983+0000","last_fullsized":"2023-08-25T09:52:06.977983+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:32.675064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978306+0000","last_change":"2023-08-25T09:52:04.622813+0000","last_active":"2023-08-25T09:52:06.978306+0000","last_peered":"2023-08-25T09:52:06.978306+0000","last_clean":"2023-08-25T09:52:06.978306+0000","last_became_active":"2023-08-25T09:52:04.622347+0000","last_became_peered":"2023-08-25T09:52:04.622347+0000","last_unstale":"2023-08-25T09:52:06.978306+0000","last_undegraded":"2023-08-25T09:52:06.978306+0000","last_fullsized":"2023-08-25T09:52:06.978306+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:45:01.358105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978917+0000","last_change":"2023-08-25T09:52:04.627414+0000","last_active":"2023-08-25T09:52:06.978917+0000","last_peered":"2023-08-25T09:52:06.978917+0000","last_clean":"2023-08-25T09:52:06.978917+0000","last_became_active":"2023-08-25T09:52:04.625586+0000","last_became_peered":"2023-08-25T09:52:04.625586+0000","last_unstale":"2023-08-25T09:52:06.978917+0000","last_undegraded":"2023-08-25T09:52:06.978917+0000","last_fullsized":"2023-08-25T09:52:06.978917+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:23.051366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.597997+0000","last_change":"2023-08-25T09:52:04.622725+0000","last_active":"2023-08-25T09:52:05.597997+0000","last_peered":"2023-08-25T09:52:05.597997+0000","last_clean":"2023-08-25T09:52:05.597997+0000","last_became_active":"2023-08-25T09:52:04.622326+0000","last_became_peered":"2023-08-25T09:52:04.622326+0000","last_unstale":"2023-08-25T09:52:05.597997+0000","last_undegraded":"2023-08-25T09:52:05.597997+0000","last_fullsized":"2023-08-25T09:52:05.597997+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:40.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594716+0000","last_change":"2023-08-25T09:52:04.613623+0000","last_active":"2023-08-25T09:52:05.594716+0000","last_peered":"2023-08-25T09:52:05.594716+0000","last_clean":"2023-08-25T09:52:05.594716+0000","last_became_active":"2023-08-25T09:52:04.612966+0000","last_became_peered":"2023-08-25T09:52:04.612966+0000","last_unstale":"2023-08-25T09:52:05.594716+0000","last_undegraded":"2023-08-25T09:52:05.594716+0000","last_fullsized":"2023-08-25T09:52:05.594716+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:08.314372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979043+0000","last_change":"2023-08-25T09:52:04.627459+0000","last_active":"2023-08-25T09:52:06.979043+0000","last_peered":"2023-08-25T09:52:06.979043+0000","last_clean":"2023-08-25T09:52:06.979043+0000","last_became_active":"2023-08-25T09:52:04.626892+0000","last_became_peered":"2023-08-25T09:52:04.626892+0000","last_unstale":"2023-08-25T09:52:06.979043+0000","last_undegraded":"2023-08-25T09:52:06.979043+0000","last_fullsized":"2023-08-25T09:52:06.979043+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:00:20.842644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978283+0000","last_change":"2023-08-25T09:52:04.612691+0000","last_active":"2023-08-25T09:52:06.978283+0000","last_peered":"2023-08-25T09:52:06.978283+0000","last_clean":"2023-08-25T09:52:06.978283+0000","last_became_active":"2023-08-25T09:52:04.612199+0000","last_became_peered":"2023-08-25T09:52:04.612199+0000","last_unstale":"2023-08-25T09:52:06.978283+0000","last_undegraded":"2023-08-25T09:52:06.978283+0000","last_fullsized":"2023-08-25T09:52:06.978283+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:23.051366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599582+0000","last_change":"2023-08-25T09:52:04.626435+0000","last_active":"2023-08-25T09:52:05.599582+0000","last_peered":"2023-08-25T09:52:05.599582+0000","last_clean":"2023-08-25T09:52:05.599582+0000","last_became_active":"2023-08-25T09:52:04.626050+0000","last_became_peered":"2023-08-25T09:52:04.626050+0000","last_unstale":"2023-08-25T09:52:05.599582+0000","last_undegraded":"2023-08-25T09:52:05.599582+0000","last_fullsized":"2023-08-25T09:52:05.599582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:35:30.431066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977748+0000","last_change":"2023-08-25T09:52:04.625300+0000","last_active":"2023-08-25T09:52:06.977748+0000","last_peered":"2023-08-25T09:52:06.977748+0000","last_clean":"2023-08-25T09:52:06.977748+0000","last_became_active":"2023-08-25T09:52:04.624890+0000","last_became_peered":"2023-08-25T09:52:04.624890+0000","last_unstale":"2023-08-25T09:52:06.977748+0000","last_undegraded":"2023-08-25T09:52:06.977748+0000","last_fullsized":"2023-08-25T09:52:06.977748+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:04:08.579191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598557+0000","last_change":"2023-08-25T09:52:04.614691+0000","last_active":"2023-08-25T09:52:05.598557+0000","last_peered":"2023-08-25T09:52:05.598557+0000","last_clean":"2023-08-25T09:52:05.598557+0000","last_became_active":"2023-08-25T09:52:04.614305+0000","last_became_peered":"2023-08-25T09:52:04.614305+0000","last_unstale":"2023-08-25T09:52:05.598557+0000","last_undegraded":"2023-08-25T09:52:05.598557+0000","last_fullsized":"2023-08-25T09:52:05.598557+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:41:57.725402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979364+0000","last_change":"2023-08-25T09:52:04.627696+0000","last_active":"2023-08-25T09:52:06.979364+0000","last_peered":"2023-08-25T09:52:06.979364+0000","last_clean":"2023-08-25T09:52:06.979364+0000","last_became_active":"2023-08-25T09:52:04.627211+0000","last_became_peered":"2023-08-25T09:52:04.627211+0000","last_unstale":"2023-08-25T09:52:06.979364+0000","last_undegraded":"2023-08-25T09:52:06.979364+0000","last_fullsized":"2023-08-25T09:52:06.979364+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:16.624825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599254+0000","last_change":"2023-08-25T09:52:04.614199+0000","last_active":"2023-08-25T09:52:05.599254+0000","last_peered":"2023-08-25T09:52:05.599254+0000","last_clean":"2023-08-25T09:52:05.599254+0000","last_became_active":"2023-08-25T09:52:04.613179+0000","last_became_peered":"2023-08-25T09:52:04.613179+0000","last_unstale":"2023-08-25T09:52:05.599254+0000","last_undegraded":"2023-08-25T09:52:05.599254+0000","last_fullsized":"2023-08-25T09:52:05.599254+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:42.152837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979806+0000","last_change":"2023-08-25T09:52:04.625832+0000","last_active":"2023-08-25T09:52:06.979806+0000","last_peered":"2023-08-25T09:52:06.979806+0000","last_clean":"2023-08-25T09:52:06.979806+0000","last_became_active":"2023-08-25T09:52:04.625323+0000","last_became_peered":"2023-08-25T09:52:04.625323+0000","last_unstale":"2023-08-25T09:52:06.979806+0000","last_undegraded":"2023-08-25T09:52:06.979806+0000","last_fullsized":"2023-08-25T09:52:06.979806+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:32.060870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980040+0000","last_change":"2023-08-25T09:52:04.606510+0000","last_active":"2023-08-25T09:52:06.980040+0000","last_peered":"2023-08-25T09:52:06.980040+0000","last_clean":"2023-08-25T09:52:06.980040+0000","last_became_active":"2023-08-25T09:52:04.605382+0000","last_became_peered":"2023-08-25T09:52:04.605382+0000","last_unstale":"2023-08-25T09:52:06.980040+0000","last_undegraded":"2023-08-25T09:52:06.980040+0000","last_fullsized":"2023-08-25T09:52:06.980040+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:15.708169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600531+0000","last_change":"2023-08-25T09:52:04.616677+0000","last_active":"2023-08-25T09:52:05.600531+0000","last_peered":"2023-08-25T09:52:05.600531+0000","last_clean":"2023-08-25T09:52:05.600531+0000","last_became_active":"2023-08-25T09:52:04.614596+0000","last_became_peered":"2023-08-25T09:52:04.614596+0000","last_unstale":"2023-08-25T09:52:05.600531+0000","last_undegraded":"2023-08-25T09:52:05.600531+0000","last_fullsized":"2023-08-25T09:52:05.600531+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:53.023481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594927+0000","last_change":"2023-08-25T09:52:04.609767+0000","last_active":"2023-08-25T09:52:05.594927+0000","last_peered":"2023-08-25T09:52:05.594927+0000","last_clean":"2023-08-25T09:52:05.594927+0000","last_became_active":"2023-08-25T09:52:04.606031+0000","last_became_peered":"2023-08-25T09:52:04.606031+0000","last_unstale":"2023-08-25T09:52:05.594927+0000","last_undegraded":"2023-08-25T09:52:05.594927+0000","last_fullsized":"2023-08-25T09:52:05.594927+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:06.542127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979128+0000","last_change":"2023-08-25T09:52:04.622622+0000","last_active":"2023-08-25T09:52:06.979128+0000","last_peered":"2023-08-25T09:52:06.979128+0000","last_clean":"2023-08-25T09:52:06.979128+0000","last_became_active":"2023-08-25T09:52:04.621916+0000","last_became_peered":"2023-08-25T09:52:04.621916+0000","last_unstale":"2023-08-25T09:52:06.979128+0000","last_undegraded":"2023-08-25T09:52:06.979128+0000","last_fullsized":"2023-08-25T09:52:06.979128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:52.575789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976919+0000","last_change":"2023-08-25T09:52:04.629777+0000","last_active":"2023-08-25T09:52:06.976919+0000","last_peered":"2023-08-25T09:52:06.976919+0000","last_clean":"2023-08-25T09:52:06.976919+0000","last_became_active":"2023-08-25T09:52:04.628214+0000","last_became_peered":"2023-08-25T09:52:04.628214+0000","last_unstale":"2023-08-25T09:52:06.976919+0000","last_undegraded":"2023-08-25T09:52:06.976919+0000","last_fullsized":"2023-08-25T09:52:06.976919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:49:37.628859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980187+0000","last_change":"2023-08-25T09:52:04.606056+0000","last_active":"2023-08-25T09:52:06.980187+0000","last_peered":"2023-08-25T09:52:06.980187+0000","last_clean":"2023-08-25T09:52:06.980187+0000","last_became_active":"2023-08-25T09:52:04.604555+0000","last_became_peered":"2023-08-25T09:52:04.604555+0000","last_unstale":"2023-08-25T09:52:06.980187+0000","last_undegraded":"2023-08-25T09:52:06.980187+0000","last_fullsized":"2023-08-25T09:52:06.980187+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:49:02.585201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978773+0000","last_change":"2023-08-25T09:52:04.624540+0000","last_active":"2023-08-25T09:52:06.978773+0000","last_peered":"2023-08-25T09:52:06.978773+0000","last_clean":"2023-08-25T09:52:06.978773+0000","last_became_active":"2023-08-25T09:52:04.624033+0000","last_became_peered":"2023-08-25T09:52:04.624033+0000","last_unstale":"2023-08-25T09:52:06.978773+0000","last_undegraded":"2023-08-25T09:52:06.978773+0000","last_fullsized":"2023-08-25T09:52:06.978773+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:22.449654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"197'4","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979006+0000","last_change":"2023-08-25T09:52:03.605509+0000","last_active":"2023-08-25T09:52:06.979006+0000","last_peered":"2023-08-25T09:52:06.979006+0000","last_clean":"2023-08-25T09:52:06.979006+0000","last_became_active":"2023-08-25T09:52:03.596779+0000","last_became_peered":"2023-08-25T09:52:03.596779+0000","last_unstale":"2023-08-25T09:52:06.979006+0000","last_undegraded":"2023-08-25T09:52:06.979006+0000","last_fullsized":"2023-08-25T09:52:06.979006+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:55.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600442+0000","last_change":"2023-08-25T09:52:04.621176+0000","last_active":"2023-08-25T09:52:05.600442+0000","last_peered":"2023-08-25T09:52:05.600442+0000","last_clean":"2023-08-25T09:52:05.600442+0000","last_became_active":"2023-08-25T09:52:04.620648+0000","last_became_peered":"2023-08-25T09:52:04.620648+0000","last_unstale":"2023-08-25T09:52:05.600442+0000","last_undegraded":"2023-08-25T09:52:05.600442+0000","last_fullsized":"2023-08-25T09:52:05.600442+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:37.317841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977679+0000","last_change":"2023-08-25T09:52:03.618376+0000","last_active":"2023-08-25T09:52:06.977679+0000","last_peered":"2023-08-25T09:52:06.977679+0000","last_clean":"2023-08-25T09:52:06.977679+0000","last_became_active":"2023-08-25T09:52:03.617343+0000","last_became_peered":"2023-08-25T09:52:03.617343+0000","last_unstale":"2023-08-25T09:52:06.977679+0000","last_undegraded":"2023-08-25T09:52:06.977679+0000","last_fullsized":"2023-08-25T09:52:06.977679+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:45:00.348861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598202+0000","last_change":"2023-08-25T09:52:04.615114+0000","last_active":"2023-08-25T09:52:05.598202+0000","last_peered":"2023-08-25T09:52:05.598202+0000","last_clean":"2023-08-25T09:52:05.598202+0000","last_became_active":"2023-08-25T09:52:04.614691+0000","last_became_peered":"2023-08-25T09:52:04.614691+0000","last_unstale":"2023-08-25T09:52:05.598202+0000","last_undegraded":"2023-08-25T09:52:05.598202+0000","last_fullsized":"2023-08-25T09:52:05.598202+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:54.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976923+0000","last_change":"2023-08-25T09:52:03.615544+0000","last_active":"2023-08-25T09:52:06.976923+0000","last_peered":"2023-08-25T09:52:06.976923+0000","last_clean":"2023-08-25T09:52:06.976923+0000","last_became_active":"2023-08-25T09:52:03.615200+0000","last_became_peered":"2023-08-25T09:52:03.615200+0000","last_unstale":"2023-08-25T09:52:06.976923+0000","last_undegraded":"2023-08-25T09:52:06.976923+0000","last_fullsized":"2023-08-25T09:52:06.976923+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:24:11.361908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976773+0000","last_change":"2023-08-25T09:52:04.622820+0000","last_active":"2023-08-25T09:52:06.976773+0000","last_peered":"2023-08-25T09:52:06.976773+0000","last_clean":"2023-08-25T09:52:06.976773+0000","last_became_active":"2023-08-25T09:52:04.622556+0000","last_became_peered":"2023-08-25T09:52:04.622556+0000","last_unstale":"2023-08-25T09:52:06.976773+0000","last_undegraded":"2023-08-25T09:52:06.976773+0000","last_fullsized":"2023-08-25T09:52:06.976773+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:12.796779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599945+0000","last_change":"2023-08-25T09:52:03.615665+0000","last_active":"2023-08-25T09:52:05.599945+0000","last_peered":"2023-08-25T09:52:05.599945+0000","last_clean":"2023-08-25T09:52:05.599945+0000","last_became_active":"2023-08-25T09:52:03.615398+0000","last_became_peered":"2023-08-25T09:52:03.615398+0000","last_unstale":"2023-08-25T09:52:05.599945+0000","last_undegraded":"2023-08-25T09:52:05.599945+0000","last_fullsized":"2023-08-25T09:52:05.599945+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:24.261532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598463+0000","last_change":"2023-08-25T09:52:04.606215+0000","last_active":"2023-08-25T09:52:05.598463+0000","last_peered":"2023-08-25T09:52:05.598463+0000","last_clean":"2023-08-25T09:52:05.598463+0000","last_became_active":"2023-08-25T09:52:04.605569+0000","last_became_peered":"2023-08-25T09:52:04.605569+0000","last_unstale":"2023-08-25T09:52:05.598463+0000","last_undegraded":"2023-08-25T09:52:05.598463+0000","last_fullsized":"2023-08-25T09:52:05.598463+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:33.270874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600761+0000","last_change":"2023-08-25T09:52:03.612691+0000","last_active":"2023-08-25T09:52:05.600761+0000","last_peered":"2023-08-25T09:52:05.600761+0000","last_clean":"2023-08-25T09:52:05.600761+0000","last_became_active":"2023-08-25T09:52:03.609712+0000","last_became_peered":"2023-08-25T09:52:03.609712+0000","last_unstale":"2023-08-25T09:52:05.600761+0000","last_undegraded":"2023-08-25T09:52:05.600761+0000","last_fullsized":"2023-08-25T09:52:05.600761+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:18.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601189+0000","last_change":"2023-08-25T09:52:04.619419+0000","last_active":"2023-08-25T09:52:05.601189+0000","last_peered":"2023-08-25T09:52:05.601189+0000","last_clean":"2023-08-25T09:52:05.601189+0000","last_became_active":"2023-08-25T09:52:04.616364+0000","last_became_peered":"2023-08-25T09:52:04.616364+0000","last_unstale":"2023-08-25T09:52:05.601189+0000","last_undegraded":"2023-08-25T09:52:05.601189+0000","last_fullsized":"2023-08-25T09:52:05.601189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:19.369746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978543+0000","last_change":"2023-08-25T09:52:03.612785+0000","last_active":"2023-08-25T09:52:06.978543+0000","last_peered":"2023-08-25T09:52:06.978543+0000","last_clean":"2023-08-25T09:52:06.978543+0000","last_became_active":"2023-08-25T09:52:03.612129+0000","last_became_peered":"2023-08-25T09:52:03.612129+0000","last_unstale":"2023-08-25T09:52:06.978543+0000","last_undegraded":"2023-08-25T09:52:06.978543+0000","last_fullsized":"2023-08-25T09:52:06.978543+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:32.325636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598377+0000","last_change":"2023-08-25T09:52:04.622547+0000","last_active":"2023-08-25T09:52:05.598377+0000","last_peered":"2023-08-25T09:52:05.598377+0000","last_clean":"2023-08-25T09:52:05.598377+0000","last_became_active":"2023-08-25T09:52:04.621941+0000","last_became_peered":"2023-08-25T09:52:04.621941+0000","last_unstale":"2023-08-25T09:52:05.598377+0000","last_undegraded":"2023-08-25T09:52:05.598377+0000","last_fullsized":"2023-08-25T09:52:05.598377+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:49.426665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978061+0000","last_change":"2023-08-25T09:52:03.618689+0000","last_active":"2023-08-25T09:52:06.978061+0000","last_peered":"2023-08-25T09:52:06.978061+0000","last_clean":"2023-08-25T09:52:06.978061+0000","last_became_active":"2023-08-25T09:52:03.617671+0000","last_became_peered":"2023-08-25T09:52:03.617671+0000","last_unstale":"2023-08-25T09:52:06.978061+0000","last_undegraded":"2023-08-25T09:52:06.978061+0000","last_fullsized":"2023-08-25T09:52:06.978061+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:03.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594833+0000","last_change":"2023-08-25T09:52:04.626763+0000","last_active":"2023-08-25T09:52:05.594833+0000","last_peered":"2023-08-25T09:52:05.594833+0000","last_clean":"2023-08-25T09:52:05.594833+0000","last_became_active":"2023-08-25T09:52:04.626503+0000","last_became_peered":"2023-08-25T09:52:04.626503+0000","last_unstale":"2023-08-25T09:52:05.594833+0000","last_undegraded":"2023-08-25T09:52:05.594833+0000","last_fullsized":"2023-08-25T09:52:05.594833+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:22:38.126861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979938+0000","last_change":"2023-08-25T09:52:03.603589+0000","last_active":"2023-08-25T09:52:06.979938+0000","last_peered":"2023-08-25T09:52:06.979938+0000","last_clean":"2023-08-25T09:52:06.979938+0000","last_became_active":"2023-08-25T09:52:03.602798+0000","last_became_peered":"2023-08-25T09:52:03.602798+0000","last_unstale":"2023-08-25T09:52:06.979938+0000","last_undegraded":"2023-08-25T09:52:06.979938+0000","last_fullsized":"2023-08-25T09:52:06.979938+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:18.360502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979138+0000","last_change":"2023-08-25T09:52:04.617158+0000","last_active":"2023-08-25T09:52:06.979138+0000","last_peered":"2023-08-25T09:52:06.979138+0000","last_clean":"2023-08-25T09:52:06.979138+0000","last_became_active":"2023-08-25T09:52:04.616462+0000","last_became_peered":"2023-08-25T09:52:04.616462+0000","last_unstale":"2023-08-25T09:52:06.979138+0000","last_undegraded":"2023-08-25T09:52:06.979138+0000","last_fullsized":"2023-08-25T09:52:06.979138+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:15:24.095300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979469+0000","last_change":"2023-08-25T09:52:03.618945+0000","last_active":"2023-08-25T09:52:06.979469+0000","last_peered":"2023-08-25T09:52:06.979469+0000","last_clean":"2023-08-25T09:52:06.979469+0000","last_became_active":"2023-08-25T09:52:03.618057+0000","last_became_peered":"2023-08-25T09:52:03.618057+0000","last_unstale":"2023-08-25T09:52:06.979469+0000","last_undegraded":"2023-08-25T09:52:06.979469+0000","last_fullsized":"2023-08-25T09:52:06.979469+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:05.630395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977905+0000","last_change":"2023-08-25T09:52:04.613446+0000","last_active":"2023-08-25T09:52:06.977905+0000","last_peered":"2023-08-25T09:52:06.977905+0000","last_clean":"2023-08-25T09:52:06.977905+0000","last_became_active":"2023-08-25T09:52:04.612830+0000","last_became_peered":"2023-08-25T09:52:04.612830+0000","last_unstale":"2023-08-25T09:52:06.977905+0000","last_undegraded":"2023-08-25T09:52:06.977905+0000","last_fullsized":"2023-08-25T09:52:06.977905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:40.373369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598883+0000","last_change":"2023-08-25T09:52:03.617112+0000","last_active":"2023-08-25T09:52:05.598883+0000","last_peered":"2023-08-25T09:52:05.598883+0000","last_clean":"2023-08-25T09:52:05.598883+0000","last_became_active":"2023-08-25T09:52:03.616870+0000","last_became_peered":"2023-08-25T09:52:03.616870+0000","last_unstale":"2023-08-25T09:52:05.598883+0000","last_undegraded":"2023-08-25T09:52:05.598883+0000","last_fullsized":"2023-08-25T09:52:05.598883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:14.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977880+0000","last_change":"2023-08-25T09:52:04.618901+0000","last_active":"2023-08-25T09:52:06.977880+0000","last_peered":"2023-08-25T09:52:06.977880+0000","last_clean":"2023-08-25T09:52:06.977880+0000","last_became_active":"2023-08-25T09:52:04.618464+0000","last_became_peered":"2023-08-25T09:52:04.618464+0000","last_unstale":"2023-08-25T09:52:06.977880+0000","last_undegraded":"2023-08-25T09:52:06.977880+0000","last_fullsized":"2023-08-25T09:52:06.977880+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:34.504532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980224+0000","last_change":"2023-08-25T09:52:03.596983+0000","last_active":"2023-08-25T09:52:06.980224+0000","last_peered":"2023-08-25T09:52:06.980224+0000","last_clean":"2023-08-25T09:52:06.980224+0000","last_became_active":"2023-08-25T09:52:03.596298+0000","last_became_peered":"2023-08-25T09:52:03.596298+0000","last_unstale":"2023-08-25T09:52:06.980224+0000","last_undegraded":"2023-08-25T09:52:06.980224+0000","last_fullsized":"2023-08-25T09:52:06.980224+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:57.842093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979133+0000","last_change":"2023-08-25T09:52:04.625834+0000","last_active":"2023-08-25T09:52:06.979133+0000","last_peered":"2023-08-25T09:52:06.979133+0000","last_clean":"2023-08-25T09:52:06.979133+0000","last_became_active":"2023-08-25T09:52:04.624790+0000","last_became_peered":"2023-08-25T09:52:04.624790+0000","last_unstale":"2023-08-25T09:52:06.979133+0000","last_undegraded":"2023-08-25T09:52:06.979133+0000","last_fullsized":"2023-08-25T09:52:06.979133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:33.334879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979304+0000","last_change":"2023-08-25T09:52:03.614128+0000","last_active":"2023-08-25T09:52:06.979304+0000","last_peered":"2023-08-25T09:52:06.979304+0000","last_clean":"2023-08-25T09:52:06.979304+0000","last_became_active":"2023-08-25T09:52:03.610416+0000","last_became_peered":"2023-08-25T09:52:03.610416+0000","last_unstale":"2023-08-25T09:52:06.979304+0000","last_undegraded":"2023-08-25T09:52:06.979304+0000","last_fullsized":"2023-08-25T09:52:06.979304+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:00:50.255020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600006+0000","last_change":"2023-08-25T09:52:04.618371+0000","last_active":"2023-08-25T09:52:05.600006+0000","last_peered":"2023-08-25T09:52:05.600006+0000","last_clean":"2023-08-25T09:52:05.600006+0000","last_became_active":"2023-08-25T09:52:04.617925+0000","last_became_peered":"2023-08-25T09:52:04.617925+0000","last_unstale":"2023-08-25T09:52:05.600006+0000","last_undegraded":"2023-08-25T09:52:05.600006+0000","last_fullsized":"2023-08-25T09:52:05.600006+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:32.675064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978949+0000","last_change":"2023-08-25T09:52:03.617559+0000","last_active":"2023-08-25T09:52:06.978949+0000","last_peered":"2023-08-25T09:52:06.978949+0000","last_clean":"2023-08-25T09:52:06.978949+0000","last_became_active":"2023-08-25T09:52:03.616663+0000","last_became_peered":"2023-08-25T09:52:03.616663+0000","last_unstale":"2023-08-25T09:52:06.978949+0000","last_undegraded":"2023-08-25T09:52:06.978949+0000","last_fullsized":"2023-08-25T09:52:06.978949+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:52.014238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978320+0000","last_change":"2023-08-25T09:52:04.618659+0000","last_active":"2023-08-25T09:52:06.978320+0000","last_peered":"2023-08-25T09:52:06.978320+0000","last_clean":"2023-08-25T09:52:06.978320+0000","last_became_active":"2023-08-25T09:52:04.617978+0000","last_became_peered":"2023-08-25T09:52:04.617978+0000","last_unstale":"2023-08-25T09:52:06.978320+0000","last_undegraded":"2023-08-25T09:52:06.978320+0000","last_fullsized":"2023-08-25T09:52:06.978320+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:37.317841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979649+0000","last_change":"2023-08-25T09:52:03.609849+0000","last_active":"2023-08-25T09:52:06.979649+0000","last_peered":"2023-08-25T09:52:06.979649+0000","last_clean":"2023-08-25T09:52:06.979649+0000","last_became_active":"2023-08-25T09:52:03.609198+0000","last_became_peered":"2023-08-25T09:52:03.609198+0000","last_unstale":"2023-08-25T09:52:06.979649+0000","last_undegraded":"2023-08-25T09:52:06.979649+0000","last_fullsized":"2023-08-25T09:52:06.979649+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:14.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979290+0000","last_change":"2023-08-25T09:52:04.617431+0000","last_active":"2023-08-25T09:52:06.979290+0000","last_peered":"2023-08-25T09:52:06.979290+0000","last_clean":"2023-08-25T09:52:06.979290+0000","last_became_active":"2023-08-25T09:52:04.616896+0000","last_became_peered":"2023-08-25T09:52:04.616896+0000","last_unstale":"2023-08-25T09:52:06.979290+0000","last_undegraded":"2023-08-25T09:52:06.979290+0000","last_fullsized":"2023-08-25T09:52:06.979290+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:19.587563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599442+0000","last_change":"2023-08-25T09:52:03.613206+0000","last_active":"2023-08-25T09:52:05.599442+0000","last_peered":"2023-08-25T09:52:05.599442+0000","last_clean":"2023-08-25T09:52:05.599442+0000","last_became_active":"2023-08-25T09:52:03.612871+0000","last_became_peered":"2023-08-25T09:52:03.612871+0000","last_unstale":"2023-08-25T09:52:05.599442+0000","last_undegraded":"2023-08-25T09:52:05.599442+0000","last_fullsized":"2023-08-25T09:52:05.599442+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:13.982356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595647+0000","last_change":"2023-08-25T09:52:04.621653+0000","last_active":"2023-08-25T09:52:05.595647+0000","last_peered":"2023-08-25T09:52:05.595647+0000","last_clean":"2023-08-25T09:52:05.595647+0000","last_became_active":"2023-08-25T09:52:04.621306+0000","last_became_peered":"2023-08-25T09:52:04.621306+0000","last_unstale":"2023-08-25T09:52:05.595647+0000","last_undegraded":"2023-08-25T09:52:05.595647+0000","last_fullsized":"2023-08-25T09:52:05.595647+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:10:13.499449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980116+0000","last_change":"2023-08-25T09:52:03.614440+0000","last_active":"2023-08-25T09:52:06.980116+0000","last_peered":"2023-08-25T09:52:06.980116+0000","last_clean":"2023-08-25T09:52:06.980116+0000","last_became_active":"2023-08-25T09:52:03.613170+0000","last_became_peered":"2023-08-25T09:52:03.613170+0000","last_unstale":"2023-08-25T09:52:06.980116+0000","last_undegraded":"2023-08-25T09:52:06.980116+0000","last_fullsized":"2023-08-25T09:52:06.980116+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:24.408611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595199+0000","last_change":"2023-08-25T09:52:04.604860+0000","last_active":"2023-08-25T09:52:05.595199+0000","last_peered":"2023-08-25T09:52:05.595199+0000","last_clean":"2023-08-25T09:52:05.595199+0000","last_became_active":"2023-08-25T09:52:04.603337+0000","last_became_peered":"2023-08-25T09:52:04.603337+0000","last_unstale":"2023-08-25T09:52:05.595199+0000","last_undegraded":"2023-08-25T09:52:05.595199+0000","last_fullsized":"2023-08-25T09:52:05.595199+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:01.014394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978774+0000","last_change":"2023-08-25T09:52:03.606748+0000","last_active":"2023-08-25T09:52:06.978774+0000","last_peered":"2023-08-25T09:52:06.978774+0000","last_clean":"2023-08-25T09:52:06.978774+0000","last_became_active":"2023-08-25T09:52:03.606384+0000","last_became_peered":"2023-08-25T09:52:03.606384+0000","last_unstale":"2023-08-25T09:52:06.978774+0000","last_undegraded":"2023-08-25T09:52:06.978774+0000","last_fullsized":"2023-08-25T09:52:06.978774+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:55.782470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979062+0000","last_change":"2023-08-25T09:52:04.613626+0000","last_active":"2023-08-25T09:52:06.979062+0000","last_peered":"2023-08-25T09:52:06.979062+0000","last_clean":"2023-08-25T09:52:06.979062+0000","last_became_active":"2023-08-25T09:52:04.613221+0000","last_became_peered":"2023-08-25T09:52:04.613221+0000","last_unstale":"2023-08-25T09:52:06.979062+0000","last_undegraded":"2023-08-25T09:52:06.979062+0000","last_fullsized":"2023-08-25T09:52:06.979062+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:19:08.585858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"198'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979649+0000","last_change":"2023-08-25T09:52:03.598008+0000","last_active":"2023-08-25T09:52:06.979649+0000","last_peered":"2023-08-25T09:52:06.979649+0000","last_clean":"2023-08-25T09:52:06.979649+0000","last_became_active":"2023-08-25T09:52:03.597171+0000","last_became_peered":"2023-08-25T09:52:03.597171+0000","last_unstale":"2023-08-25T09:52:06.979649+0000","last_undegraded":"2023-08-25T09:52:06.979649+0000","last_fullsized":"2023-08-25T09:52:06.979649+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:29.871817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979809+0000","last_change":"2023-08-25T09:52:04.617324+0000","last_active":"2023-08-25T09:52:06.979809+0000","last_peered":"2023-08-25T09:52:06.979809+0000","last_clean":"2023-08-25T09:52:06.979809+0000","last_became_active":"2023-08-25T09:52:04.616867+0000","last_became_peered":"2023-08-25T09:52:04.616867+0000","last_unstale":"2023-08-25T09:52:06.979809+0000","last_undegraded":"2023-08-25T09:52:06.979809+0000","last_fullsized":"2023-08-25T09:52:06.979809+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:13:46.538086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980140+0000","last_change":"2023-08-25T09:52:03.619922+0000","last_active":"2023-08-25T09:52:06.980140+0000","last_peered":"2023-08-25T09:52:06.980140+0000","last_clean":"2023-08-25T09:52:06.980140+0000","last_became_active":"2023-08-25T09:52:03.619288+0000","last_became_peered":"2023-08-25T09:52:03.619288+0000","last_unstale":"2023-08-25T09:52:06.980140+0000","last_undegraded":"2023-08-25T09:52:06.980140+0000","last_fullsized":"2023-08-25T09:52:06.980140+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:47.888788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601489+0000","last_change":"2023-08-25T09:52:04.618818+0000","last_active":"2023-08-25T09:52:05.601489+0000","last_peered":"2023-08-25T09:52:05.601489+0000","last_clean":"2023-08-25T09:52:05.601489+0000","last_became_active":"2023-08-25T09:52:04.615160+0000","last_became_peered":"2023-08-25T09:52:04.615160+0000","last_unstale":"2023-08-25T09:52:05.601489+0000","last_undegraded":"2023-08-25T09:52:05.601489+0000","last_fullsized":"2023-08-25T09:52:05.601489+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:46.121852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979396+0000","last_change":"2023-08-25T09:52:03.614922+0000","last_active":"2023-08-25T09:52:06.979396+0000","last_peered":"2023-08-25T09:52:06.979396+0000","last_clean":"2023-08-25T09:52:06.979396+0000","last_became_active":"2023-08-25T09:52:03.614577+0000","last_became_peered":"2023-08-25T09:52:03.614577+0000","last_unstale":"2023-08-25T09:52:06.979396+0000","last_undegraded":"2023-08-25T09:52:06.979396+0000","last_fullsized":"2023-08-25T09:52:06.979396+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:23.368847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594731+0000","last_change":"2023-08-25T09:52:04.622816+0000","last_active":"2023-08-25T09:52:05.594731+0000","last_peered":"2023-08-25T09:52:05.594731+0000","last_clean":"2023-08-25T09:52:05.594731+0000","last_became_active":"2023-08-25T09:52:04.622527+0000","last_became_peered":"2023-08-25T09:52:04.622527+0000","last_unstale":"2023-08-25T09:52:05.594731+0000","last_undegraded":"2023-08-25T09:52:05.594731+0000","last_fullsized":"2023-08-25T09:52:05.594731+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:29.167368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980224+0000","last_change":"2023-08-25T09:52:03.614822+0000","last_active":"2023-08-25T09:52:06.980224+0000","last_peered":"2023-08-25T09:52:06.980224+0000","last_clean":"2023-08-25T09:52:06.980224+0000","last_became_active":"2023-08-25T09:52:03.613922+0000","last_became_peered":"2023-08-25T09:52:03.613922+0000","last_unstale":"2023-08-25T09:52:06.980224+0000","last_undegraded":"2023-08-25T09:52:06.980224+0000","last_fullsized":"2023-08-25T09:52:06.980224+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:10.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595023+0000","last_change":"2023-08-25T09:52:04.605224+0000","last_active":"2023-08-25T09:52:05.595023+0000","last_peered":"2023-08-25T09:52:05.595023+0000","last_clean":"2023-08-25T09:52:05.595023+0000","last_became_active":"2023-08-25T09:52:04.604198+0000","last_became_peered":"2023-08-25T09:52:04.604198+0000","last_unstale":"2023-08-25T09:52:05.595023+0000","last_undegraded":"2023-08-25T09:52:05.595023+0000","last_fullsized":"2023-08-25T09:52:05.595023+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:20.783977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601107+0000","last_change":"2023-08-25T09:52:03.610194+0000","last_active":"2023-08-25T09:52:05.601107+0000","last_peered":"2023-08-25T09:52:05.601107+0000","last_clean":"2023-08-25T09:52:05.601107+0000","last_became_active":"2023-08-25T09:52:03.609009+0000","last_became_peered":"2023-08-25T09:52:03.609009+0000","last_unstale":"2023-08-25T09:52:05.601107+0000","last_undegraded":"2023-08-25T09:52:05.601107+0000","last_fullsized":"2023-08-25T09:52:05.601107+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:31.051627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978645+0000","last_change":"2023-08-25T09:52:04.620996+0000","last_active":"2023-08-25T09:52:06.978645+0000","last_peered":"2023-08-25T09:52:06.978645+0000","last_clean":"2023-08-25T09:52:06.978645+0000","last_became_active":"2023-08-25T09:52:04.620726+0000","last_became_peered":"2023-08-25T09:52:04.620726+0000","last_unstale":"2023-08-25T09:52:06.978645+0000","last_undegraded":"2023-08-25T09:52:06.978645+0000","last_fullsized":"2023-08-25T09:52:06.978645+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:52.575789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979407+0000","last_change":"2023-08-25T09:52:03.608521+0000","last_active":"2023-08-25T09:52:06.979407+0000","last_peered":"2023-08-25T09:52:06.979407+0000","last_clean":"2023-08-25T09:52:06.979407+0000","last_became_active":"2023-08-25T09:52:03.605881+0000","last_became_peered":"2023-08-25T09:52:03.605881+0000","last_unstale":"2023-08-25T09:52:06.979407+0000","last_undegraded":"2023-08-25T09:52:06.979407+0000","last_fullsized":"2023-08-25T09:52:06.979407+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:12:17.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598820+0000","last_change":"2023-08-25T09:52:04.615664+0000","last_active":"2023-08-25T09:52:05.598820+0000","last_peered":"2023-08-25T09:52:05.598820+0000","last_clean":"2023-08-25T09:52:05.598820+0000","last_became_active":"2023-08-25T09:52:04.613567+0000","last_became_peered":"2023-08-25T09:52:04.613567+0000","last_unstale":"2023-08-25T09:52:05.598820+0000","last_undegraded":"2023-08-25T09:52:05.598820+0000","last_fullsized":"2023-08-25T09:52:05.598820+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:33.941978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600289+0000","last_change":"2023-08-25T09:52:03.614418+0000","last_active":"2023-08-25T09:52:05.600289+0000","last_peered":"2023-08-25T09:52:05.600289+0000","last_clean":"2023-08-25T09:52:05.600289+0000","last_became_active":"2023-08-25T09:52:03.614127+0000","last_became_peered":"2023-08-25T09:52:03.614127+0000","last_unstale":"2023-08-25T09:52:05.600289+0000","last_undegraded":"2023-08-25T09:52:05.600289+0000","last_fullsized":"2023-08-25T09:52:05.600289+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:31.211948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977137+0000","last_change":"2023-08-25T09:52:04.629117+0000","last_active":"2023-08-25T09:52:06.977137+0000","last_peered":"2023-08-25T09:52:06.977137+0000","last_clean":"2023-08-25T09:52:06.977137+0000","last_became_active":"2023-08-25T09:52:04.626222+0000","last_became_peered":"2023-08-25T09:52:04.626222+0000","last_unstale":"2023-08-25T09:52:06.977137+0000","last_undegraded":"2023-08-25T09:52:06.977137+0000","last_fullsized":"2023-08-25T09:52:06.977137+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:59:10.664615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598694+0000","last_change":"2023-08-25T09:52:03.619038+0000","last_active":"2023-08-25T09:52:05.598694+0000","last_peered":"2023-08-25T09:52:05.598694+0000","last_clean":"2023-08-25T09:52:05.598694+0000","last_became_active":"2023-08-25T09:52:03.618767+0000","last_became_peered":"2023-08-25T09:52:03.618767+0000","last_unstale":"2023-08-25T09:52:05.598694+0000","last_undegraded":"2023-08-25T09:52:05.598694+0000","last_fullsized":"2023-08-25T09:52:05.598694+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:33.870463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":479,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977545+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:52:06.977545+0000","last_peered":"2023-08-25T09:52:06.977545+0000","last_clean":"2023-08-25T09:52:06.977545+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:52:06.977545+0000","last_undegraded":"2023-08-25T09:52:06.977545+0000","last_fullsized":"2023-08-25T09:52:06.977545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:46.367732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595603+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:52:05.595603+0000","last_peered":"2023-08-25T09:52:05.595603+0000","last_clean":"2023-08-25T09:52:05.595603+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:52:05.595603+0000","last_undegraded":"2023-08-25T09:52:05.595603+0000","last_fullsized":"2023-08-25T09:52:05.595603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:03:12.303284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600876+0000","last_change":"2023-08-25T09:52:04.619104+0000","last_active":"2023-08-25T09:52:05.600876+0000","last_peered":"2023-08-25T09:52:05.600876+0000","last_clean":"2023-08-25T09:52:05.600876+0000","last_became_active":"2023-08-25T09:52:04.615804+0000","last_became_peered":"2023-08-25T09:52:04.615804+0000","last_unstale":"2023-08-25T09:52:05.600876+0000","last_undegraded":"2023-08-25T09:52:05.600876+0000","last_fullsized":"2023-08-25T09:52:05.600876+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:30.881061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"197'3","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979257+0000","last_change":"2023-08-25T09:52:03.608811+0000","last_active":"2023-08-25T09:52:06.979257+0000","last_peered":"2023-08-25T09:52:06.979257+0000","last_clean":"2023-08-25T09:52:06.979257+0000","last_became_active":"2023-08-25T09:52:03.608140+0000","last_became_peered":"2023-08-25T09:52:03.608140+0000","last_unstale":"2023-08-25T09:52:06.979257+0000","last_undegraded":"2023-08-25T09:52:06.979257+0000","last_fullsized":"2023-08-25T09:52:06.979257+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:48:39.537165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595573+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:52:05.595573+0000","last_peered":"2023-08-25T09:52:05.595573+0000","last_clean":"2023-08-25T09:52:05.595573+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:52:05.595573+0000","last_undegraded":"2023-08-25T09:52:05.595573+0000","last_fullsized":"2023-08-25T09:52:05.595573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:43:58.928366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600124+0000","last_change":"2023-08-25T09:52:04.616692+0000","last_active":"2023-08-25T09:52:05.600124+0000","last_peered":"2023-08-25T09:52:05.600124+0000","last_clean":"2023-08-25T09:52:05.600124+0000","last_became_active":"2023-08-25T09:52:04.610333+0000","last_became_peered":"2023-08-25T09:52:04.610333+0000","last_unstale":"2023-08-25T09:52:05.600124+0000","last_undegraded":"2023-08-25T09:52:05.600124+0000","last_fullsized":"2023-08-25T09:52:05.600124+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:21.268563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"197'6","reported_seq":19,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600016+0000","last_change":"2023-08-25T09:52:03.615496+0000","last_active":"2023-08-25T09:52:05.600016+0000","last_peered":"2023-08-25T09:52:05.600016+0000","last_clean":"2023-08-25T09:52:05.600016+0000","last_became_active":"2023-08-25T09:52:03.615224+0000","last_became_peered":"2023-08-25T09:52:03.615224+0000","last_unstale":"2023-08-25T09:52:05.600016+0000","last_undegraded":"2023-08-25T09:52:05.600016+0000","last_fullsized":"2023-08-25T09:52:05.600016+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:12:41.768761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":359,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.602617+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:52:05.602617+0000","last_peered":"2023-08-25T09:52:05.602617+0000","last_clean":"2023-08-25T09:52:05.602617+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:52:05.602617+0000","last_undegraded":"2023-08-25T09:52:05.602617+0000","last_fullsized":"2023-08-25T09:52:05.602617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:48.772597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979915+0000","last_change":"2023-08-25T09:52:04.609947+0000","last_active":"2023-08-25T09:52:06.979915+0000","last_peered":"2023-08-25T09:52:06.979915+0000","last_clean":"2023-08-25T09:52:06.979915+0000","last_became_active":"2023-08-25T09:52:04.609100+0000","last_became_peered":"2023-08-25T09:52:04.609100+0000","last_unstale":"2023-08-25T09:52:06.979915+0000","last_undegraded":"2023-08-25T09:52:06.979915+0000","last_fullsized":"2023-08-25T09:52:06.979915+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:45.487493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598743+0000","last_change":"2023-08-25T09:52:03.615906+0000","last_active":"2023-08-25T09:52:05.598743+0000","last_peered":"2023-08-25T09:52:05.598743+0000","last_clean":"2023-08-25T09:52:05.598743+0000","last_became_active":"2023-08-25T09:52:03.615475+0000","last_became_peered":"2023-08-25T09:52:03.615475+0000","last_unstale":"2023-08-25T09:52:05.598743+0000","last_undegraded":"2023-08-25T09:52:05.598743+0000","last_fullsized":"2023-08-25T09:52:05.598743+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:57.343819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.596068+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:52:05.596068+0000","last_peered":"2023-08-25T09:52:05.596068+0000","last_clean":"2023-08-25T09:52:05.596068+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:52:05.596068+0000","last_undegraded":"2023-08-25T09:52:05.596068+0000","last_fullsized":"2023-08-25T09:52:05.596068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:33:58.167750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976617+0000","last_change":"2023-08-25T09:52:04.626470+0000","last_active":"2023-08-25T09:52:06.976617+0000","last_peered":"2023-08-25T09:52:06.976617+0000","last_clean":"2023-08-25T09:52:06.976617+0000","last_became_active":"2023-08-25T09:52:04.625407+0000","last_became_peered":"2023-08-25T09:52:04.625407+0000","last_unstale":"2023-08-25T09:52:06.976617+0000","last_undegraded":"2023-08-25T09:52:06.976617+0000","last_fullsized":"2023-08-25T09:52:06.976617+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:27:19.683074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977016+0000","last_change":"2023-08-25T09:52:03.613108+0000","last_active":"2023-08-25T09:52:06.977016+0000","last_peered":"2023-08-25T09:52:06.977016+0000","last_clean":"2023-08-25T09:52:06.977016+0000","last_became_active":"2023-08-25T09:52:03.612787+0000","last_became_peered":"2023-08-25T09:52:03.612787+0000","last_unstale":"2023-08-25T09:52:06.977016+0000","last_undegraded":"2023-08-25T09:52:06.977016+0000","last_fullsized":"2023-08-25T09:52:06.977016+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:45.967657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980300+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:52:06.980300+0000","last_peered":"2023-08-25T09:52:06.980300+0000","last_clean":"2023-08-25T09:52:06.980300+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:52:06.980300+0000","last_undegraded":"2023-08-25T09:52:06.980300+0000","last_fullsized":"2023-08-25T09:52:06.980300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:10.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978259+0000","last_change":"2023-08-25T09:52:04.625453+0000","last_active":"2023-08-25T09:52:06.978259+0000","last_peered":"2023-08-25T09:52:06.978259+0000","last_clean":"2023-08-25T09:52:06.978259+0000","last_became_active":"2023-08-25T09:52:04.625181+0000","last_became_peered":"2023-08-25T09:52:04.625181+0000","last_unstale":"2023-08-25T09:52:06.978259+0000","last_undegraded":"2023-08-25T09:52:06.978259+0000","last_fullsized":"2023-08-25T09:52:06.978259+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:21.268563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"197'8","reported_seq":21,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.602337+0000","last_change":"2023-08-25T09:52:03.615255+0000","last_active":"2023-08-25T09:52:05.602337+0000","last_peered":"2023-08-25T09:52:05.602337+0000","last_clean":"2023-08-25T09:52:05.602337+0000","last_became_active":"2023-08-25T09:52:03.613680+0000","last_became_peered":"2023-08-25T09:52:03.613680+0000","last_unstale":"2023-08-25T09:52:05.602337+0000","last_undegraded":"2023-08-25T09:52:05.602337+0000","last_fullsized":"2023-08-25T09:52:05.602337+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:05:27.088191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595232+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:52:05.595232+0000","last_peered":"2023-08-25T09:52:05.595232+0000","last_clean":"2023-08-25T09:52:05.595232+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:52:05.595232+0000","last_undegraded":"2023-08-25T09:52:05.595232+0000","last_fullsized":"2023-08-25T09:52:05.595232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:32.237945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977164+0000","last_change":"2023-08-25T09:52:04.628438+0000","last_active":"2023-08-25T09:52:06.977164+0000","last_peered":"2023-08-25T09:52:06.977164+0000","last_clean":"2023-08-25T09:52:06.977164+0000","last_became_active":"2023-08-25T09:52:04.627892+0000","last_became_peered":"2023-08-25T09:52:04.627892+0000","last_unstale":"2023-08-25T09:52:06.977164+0000","last_undegraded":"2023-08-25T09:52:06.977164+0000","last_fullsized":"2023-08-25T09:52:06.977164+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:45.280799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980132+0000","last_change":"2023-08-25T09:52:03.608367+0000","last_active":"2023-08-25T09:52:06.980132+0000","last_peered":"2023-08-25T09:52:06.980132+0000","last_clean":"2023-08-25T09:52:06.980132+0000","last_became_active":"2023-08-25T09:52:03.605716+0000","last_became_peered":"2023-08-25T09:52:03.605716+0000","last_unstale":"2023-08-25T09:52:06.980132+0000","last_undegraded":"2023-08-25T09:52:06.980132+0000","last_fullsized":"2023-08-25T09:52:06.980132+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:45.112608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978871+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:52:06.978871+0000","last_peered":"2023-08-25T09:52:06.978871+0000","last_clean":"2023-08-25T09:52:06.978871+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:52:06.978871+0000","last_undegraded":"2023-08-25T09:52:06.978871+0000","last_fullsized":"2023-08-25T09:52:06.978871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:05.193960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599626+0000","last_change":"2023-08-25T09:52:04.616992+0000","last_active":"2023-08-25T09:52:05.599626+0000","last_peered":"2023-08-25T09:52:05.599626+0000","last_clean":"2023-08-25T09:52:05.599626+0000","last_became_active":"2023-08-25T09:52:04.613623+0000","last_became_peered":"2023-08-25T09:52:04.613623+0000","last_unstale":"2023-08-25T09:52:05.599626+0000","last_undegraded":"2023-08-25T09:52:05.599626+0000","last_fullsized":"2023-08-25T09:52:05.599626+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:27.441433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599106+0000","last_change":"2023-08-25T09:52:03.611964+0000","last_active":"2023-08-25T09:52:05.599106+0000","last_peered":"2023-08-25T09:52:05.599106+0000","last_clean":"2023-08-25T09:52:05.599106+0000","last_became_active":"2023-08-25T09:52:03.611724+0000","last_became_peered":"2023-08-25T09:52:03.611724+0000","last_unstale":"2023-08-25T09:52:05.599106+0000","last_undegraded":"2023-08-25T09:52:05.599106+0000","last_fullsized":"2023-08-25T09:52:05.599106+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:31.754265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595886+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:52:05.595886+0000","last_peered":"2023-08-25T09:52:05.595886+0000","last_clean":"2023-08-25T09:52:05.595886+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:52:05.595886+0000","last_undegraded":"2023-08-25T09:52:05.595886+0000","last_fullsized":"2023-08-25T09:52:05.595886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:56.976887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198806588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.35699999999999998},"max":{"1min":1.4930000000000001,"5min":1.829,"15min":10.747},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.173,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.43099999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.855},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.6970000000000001,"5min":1.823,"15min":11.010999999999999},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.2589999999999999,"15min":1.1819999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.496,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.206,"15min":1.147},"min":{"1min":0.86699999999999999,"5min":0.79400000000000004,"15min":0.50600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5529999999999999,"15min":1.619},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3180000000000001,"15min":1.2769999999999999},"min":{"1min":0.93999999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6779999999999999,"5min":1.873,"15min":1.98},"last":1.6379999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.258,"15min":1.2190000000000001},"min":{"1min":0.95599999999999996,"5min":0.79300000000000004,"15min":0.56799999999999995},"max":{"1min":1.506,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.224},{"interface":"front","average":{"1min":1.359,"5min":1.393,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.69299999999999995},"max":{"1min":1.778,"5min":1.833,"15min":2.1030000000000002},"last":1.5620000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.264,"15min":1.288},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.48599999999999999},"max":{"1min":1.732,"5min":1.845,"15min":7.1639999999999997},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.306,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.46300000000000002},"max":{"1min":1.7190000000000001,"5min":1.798,"15min":1.8779999999999999},"last":0.999}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.52700000000000002},"max":{"1min":1.714,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.331},{"interface":"front","average":{"1min":1.361,"5min":1.4450000000000001,"15min":1.429},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.64100000000000001},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":2.0739999999999998},"last":1.04}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5580000000000001,"15min":1.518},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.68799999999999994},"max":{"1min":1.79,"5min":1.984,"15min":2.1720000000000002},"last":1.49},{"interface":"front","average":{"1min":1.448,"5min":1.4870000000000001,"15min":1.462},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.5189999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198806588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.161,"15min":1.1459999999999999},"min":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.54400000000000004},"max":{"1min":1.7310000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.25,"5min":1.198,"15min":1.175},"min":{"1min":0.84199999999999997,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.627,"5min":1.645,"15min":5.5410000000000004},"last":0.997}]},{"osd":1,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2529999999999999,"15min":1.2210000000000001},"min":{"1min":0.82799999999999996,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6930000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.194,"15min":1.179},"min":{"1min":0.75800000000000001,"5min":0.66800000000000004,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.909,"15min":2.3079999999999998},"last":1.1259999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.153},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.708},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.67000000000000004,"15min":0.47299999999999998},"max":{"1min":1.7310000000000001,"5min":1.7969999999999999,"15min":13.193},"last":1.458}]},{"osd":3,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":1.0620000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.861,"5min":1.911,"15min":5.6239999999999997},"last":1.861},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.286,"15min":1.27},"min":{"1min":0.995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8200000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5880000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.325,"15min":1.3169999999999999},"min":{"1min":1.048,"5min":0.55300000000000005,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":1.867,"15min":2.484},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.405,"15min":1.381},"min":{"1min":0.99199999999999999,"5min":0.70099999999999996,"15min":0.58799999999999997},"max":{"1min":1.9650000000000001,"5min":2.101,"15min":5.4699999999999998},"last":1.9650000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.3260000000000001},"min":{"1min":0.95099999999999996,"5min":0.74199999999999999,"15min":0.59599999999999997},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.802},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.468,"15min":1.454},"min":{"1min":1,"5min":0.61099999999999999,"15min":0.59099999999999997},"max":{"1min":1.786,"5min":1.994,"15min":2.4420000000000002},"last":1.7509999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":1.179,"5min":0.80000000000000004,"15min":0.622},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":6.7709999999999999},"last":2.0329999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5169999999999999,"15min":1.498},"min":{"1min":1.105,"5min":0.85899999999999999,"15min":0.57399999999999995},"max":{"1min":2.0590000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.921}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193571,"statfs":{"total":200000000,"available":198217146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2669999999999999,"15min":1.2050000000000001},"min":{"1min":0.751,"5min":0.71999999999999997,"15min":0.497},"max":{"1min":1.7210000000000001,"5min":1.9770000000000001,"15min":2.081},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.252,"15min":1.2210000000000001},"min":{"1min":0.88800000000000001,"5min":0.747,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.8080000000000001,"15min":1.974},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1639999999999999,"15min":1.165},"min":{"1min":0.69799999999999995,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.6399999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.208,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.571,"5min":1.704,"15min":2.1869999999999998},"last":1.4990000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3200000000000001,"15min":1.252},"min":{"1min":0.91700000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8149999999999999,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.331},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.252},"min":{"1min":0.83099999999999996,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.5449999999999999,"5min":1.847,"15min":1.962},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2589999999999999,"15min":1.288},"min":{"1min":0.86299999999999999,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.71,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.121},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3500000000000001,"15min":1.325},"min":{"1min":0.95299999999999996,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.7330000000000001,"5min":1.964,"15min":1.9650000000000001},"last":1.569}]},{"osd":4,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.306,"15min":1.3149999999999999},"min":{"1min":0.83199999999999996,"5min":0.80800000000000005,"15min":0.55500000000000005},"max":{"1min":1.76,"5min":1.9059999999999999,"15min":2.625},"last":1.637},{"interface":"front","average":{"1min":1.304,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.58799999999999997},"max":{"1min":1.7150000000000001,"5min":1.913,"15min":2.2789999999999999},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.462,"15min":1.444},"min":{"1min":1.0569999999999999,"5min":0.95099999999999996,"15min":0.58899999999999997},"max":{"1min":1.76,"5min":2.028,"15min":2.4820000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.5249999999999999,"15min":1.486},"min":{"1min":1.0840000000000001,"5min":0.96899999999999997,"15min":0.60399999999999998},"max":{"1min":1.9039999999999999,"5min":2.044,"15min":2.5539999999999998},"last":1.7549999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4950000000000001,"15min":1.464},"min":{"1min":1.0129999999999999,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.585,"15min":1.579},"min":{"1min":1.2869999999999999,"5min":1.0289999999999999,"15min":0.63900000000000001},"max":{"1min":1.923,"5min":2.1040000000000001,"15min":2.3700000000000001},"last":1.7929999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.28,"15min":1.224},"min":{"1min":0.624,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.589,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.2709999999999999,"15min":1.2490000000000001},"min":{"1min":0.91600000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.615,"5min":2.5150000000000001,"15min":7.367},"last":1.3859999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.466,"5min":1.7330000000000001,"15min":2.032},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.234,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.571,"5min":4.351,"15min":8.5850000000000009},"last":1.3}]},{"osd":2,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.214},"min":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.556,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.294,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5940000000000001,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.04}]},{"osd":3,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.284,"15min":1.2629999999999999},"min":{"1min":0.79400000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.665,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.911},"last":1.4810000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3140000000000001,"15min":1.3009999999999999},"min":{"1min":0.67100000000000004,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.7370000000000001,"5min":5.048,"15min":5.048},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.1419999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4179999999999999,"15min":1.4319999999999999},"min":{"1min":0.88100000000000001,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":1.927,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.597,"15min":1.579},"min":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.968,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.496,"15min":1.49},"min":{"1min":0.75700000000000001,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.865,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.569,"15min":1.5920000000000001},"min":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8340000000000001,"5min":3.831,"15min":3.831},"last":1.097}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98599999999999999,"15min":0.91000000000000003},"min":{"1min":0.628,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.8180000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0649999999999999,"15min":0.98199999999999998},"min":{"1min":0.69999999999999996,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.579,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.94299999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.64600000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.258,"5min":1.413,"15min":1.702},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.087,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.623,"5min":1.6339999999999999,"15min":2.3399999999999999},"last":1.27}]},{"osd":2,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.034,"15min":0.96499999999999997},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.5980000000000001,"5min":1.623,"15min":1.919},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2050000000000001,"15min":1.1439999999999999},"min":{"1min":0.745,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.901,"5min":1.901,"15min":3.399},"last":1.5249999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3009999999999999,"15min":1.236},"min":{"1min":1.1399999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.395},{"interface":"front","average":{"1min":1.198,"5min":1.204,"15min":1.2090000000000001},"min":{"1min":1.0109999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.405,"5min":1.79,"15min":2.1840000000000002},"last":1.6539999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3480000000000001},"min":{"1min":1.137,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":2.1509999999999998,"5min":2.1930000000000001,"15min":2.407},"last":1.764},{"interface":"front","average":{"1min":1.357,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.0349999999999999,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.804,"5min":1.895,"15min":1.895},"last":1.806}]},{"osd":6,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.302,"15min":1.282},"min":{"1min":1.054,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5980000000000001,"5min":1.875,"15min":2.3450000000000002},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.47,"15min":1.4199999999999999},"min":{"1min":1.28,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":2.2330000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.9830000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.464,"15min":1.405},"min":{"1min":1.0840000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.976,"5min":1.976,"15min":2.2989999999999999},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.3839999999999999},"min":{"1min":1.1259999999999999,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":1.7589999999999999,"5min":1.9350000000000001,"15min":2.1509999999999998},"last":2.0139999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198807436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.64900000000000002,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.51,"5min":1.6930000000000001,"15min":2.383},"last":0.753},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93300000000000005,"15min":0.96299999999999997},"min":{"1min":0.47599999999999998,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":2.7309999999999999},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.97599999999999998,"15min":0.98499999999999999},"min":{"1min":0.66400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3009999999999999,"5min":1.635,"15min":2.3180000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0429999999999999,"15min":1.0389999999999999},"min":{"1min":0.56699999999999995,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.5800000000000001,"5min":1.704,"15min":2.8599999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":1.0429999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.294,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.121,"15min":1.101},"min":{"1min":0.84699999999999998,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.587,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.99199999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3080000000000001,"15min":1.321},"min":{"1min":0.94099999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":1.194},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":0.91200000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6459999999999999,"5min":1.8,"15min":2.7639999999999998},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2470000000000001,"15min":1.2809999999999999},"min":{"1min":0.89000000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.534,"5min":1.849,"15min":5.1269999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.1020000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.605,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":1.2649999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.397,"15min":1.403},"min":{"1min":1.1060000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.393,"15min":1.417},"min":{"1min":1.0109999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.617,"5min":2.028,"15min":5.0599999999999996},"last":1.0800000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.373,"15min":1.395},"min":{"1min":1.1699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.7210000000000001,"5min":2,"15min":2.968},"last":0.872},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3779999999999999,"15min":1.4239999999999999},"min":{"1min":1.071,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.569,"5min":1.96,"15min":5.2990000000000004},"last":0.96699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1740,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193572,"statfs":{"total":200000000,"available":198217742,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.82699999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.29699999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97899999999999998,"15min":0.89800000000000002},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.7869999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.88100000000000001},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.35899999999999999},"max":{"1min":1.2809999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.022,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.389,"5min":1.6619999999999999,"15min":1.746},"last":1.139}]},{"osd":3,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.109},"min":{"1min":0.496,"5min":0.496,"15min":0.372},"max":{"1min":1.647,"5min":1.647,"15min":1.9450000000000001},"last":1.278},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0109999999999999},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.333,"5min":1.655,"15min":1.764},"last":1.381}]},{"osd":4,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.2929999999999999,"15min":1.276},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":1.6830000000000001,"5min":1.9419999999999999,"15min":2.008},"last":1.351}]},{"osd":5,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.2430000000000001},"min":{"1min":0.94699999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.665,"15min":1.9650000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3049999999999999,"15min":1.2450000000000001},"min":{"1min":0.82699999999999996,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.677,"5min":2.867,"15min":2.867},"last":1.647}]},{"osd":6,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.363},"min":{"1min":1.016,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":1.633,"5min":2.0779999999999998,"15min":2.9940000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.411,"15min":1.3660000000000001},"min":{"1min":0.95099999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.603,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.5629999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4490000000000001,"15min":1.4139999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.79200000000000004},"max":{"1min":1.772,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.431,"15min":1.399},"min":{"1min":0.88200000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.738,"5min":1.913,"15min":2.218},"last":1.4950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808200,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.313},"max":{"1min":1.4450000000000001,"5min":2.2759999999999998,"15min":11.257},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.98499999999999999,"15min":0.96799999999999997},"min":{"1min":0.60199999999999998,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.4730000000000001,"5min":1.762,"15min":6.4260000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.96099999999999997},"min":{"1min":0.72399999999999998,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.343,"5min":1.522,"15min":1.8620000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":1.048},"min":{"1min":0.62,"5min":0.374,"15min":0.374},"max":{"1min":1.401,"5min":1.714,"15min":3.4889999999999999},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.98499999999999999,"15min":0.93000000000000005},"min":{"1min":0.65900000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.464,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.69199999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.629,"15min":2.3809999999999998},"last":1.4690000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.294,"15min":1.2569999999999999},"min":{"1min":0.89800000000000002,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.724,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.097,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.379}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.88800000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.919,"5min":1.919,"15min":3.3769999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.2769999999999999},"min":{"1min":1.038,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6439999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.573}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4219999999999999,"15min":1.3640000000000001},"min":{"1min":0.92400000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7370000000000001,"5min":2.056,"15min":2.6869999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.3759999999999999,"15min":1.335},"min":{"1min":0.999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7709999999999999,"5min":1.863,"15min":2.331},"last":1.508}]},{"osd":7,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.706,"5min":1.962,"15min":2.6429999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.389,"15min":1.3440000000000001},"min":{"1min":0.95699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.804,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.423}]}]}],"pool_statfs":[]}} 2023-08-25T09:52:10.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-25T09:52:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.321+0000 7fb87a287640 1 Processor -- start 2023-08-25T09:52:10.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.322+0000 7fb87a287640 1 -- start start 2023-08-25T09:52:10.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.323+0000 7fb87a287640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb8740ca780 0x7fb8740cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.323+0000 7fb87a287640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8740b12b0 con 0x7fb8740b3cf0 2023-08-25T09:52:10.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.323+0000 7fb87a287640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87406ae20 con 0x7fb87415acd0 2023-08-25T09:52:10.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.323+0000 7fb87a287640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8740b47e0 con 0x7fb8740ca780 2023-08-25T09:52:10.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb878824640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb8740b3cf0 0x7fb8740b40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45784/0 (socket says 172.21.15.70:45784) 2023-08-25T09:52:10.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb878824640 1 -- 172.21.15.70:0/3349959675 learned_addr learned my addr 172.21.15.70:0/3349959675 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:10.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb86bfff640 1 --2- 172.21.15.70:0/3349959675 >> v2:172.21.15.70:3300/0 conn(0x7fb8740ca780 0x7fb8740cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb8737fe640 1 -- 172.21.15.70:0/3349959675 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727051234 0 0) 0x7fb8740b12b0 con 0x7fb8740b3cf0 2023-08-25T09:52:10.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb8737fe640 1 -- 172.21.15.70:0/3349959675 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb860003520 con 0x7fb8740b3cf0 2023-08-25T09:52:10.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb86bfff640 1 -- 172.21.15.70:0/3349959675 >> v1:172.21.15.155:6789/0 conn(0x7fb87415acd0 legacy=0x7fb87415b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:10.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb86bfff640 1 -- 172.21.15.70:0/3349959675 >> v1:172.21.15.70:6789/0 conn(0x7fb8740b3cf0 legacy=0x7fb8740b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.324+0000 7fb86bfff640 1 -- 172.21.15.70:0/3349959675 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87415b7c0 con 0x7fb8740ca780 2023-08-25T09:52:10.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.325+0000 7fb86bfff640 1 --2- 172.21.15.70:0/3349959675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740ca780 0x7fb8740cac30 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0097a0 tx=0x7fb85c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f175b810893b7015 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:10.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.325+0000 7fb8737fe640 1 -- 172.21.15.70:0/3349959675 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c02e070 con 0x7fb8740ca780 2023-08-25T09:52:10.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb8737fe640 1 -- 172.21.15.70:0/3349959675 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb85c0315a0 con 0x7fb8740ca780 2023-08-25T09:52:10.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb8737fe640 1 -- 172.21.15.70:0/3349959675 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb85c031930 con 0x7fb8740ca780 2023-08-25T09:52:10.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 -- 172.21.15.70:0/3349959675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740ca780 msgr2=0x7fb8740cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 --2- 172.21.15.70:0/3349959675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740ca780 0x7fb8740cac30 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0097a0 tx=0x7fb85c02b840 comp rx=0 tx=0).stop 2023-08-25T09:52:10.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 -- 172.21.15.70:0/3349959675 shutdown_connections 2023-08-25T09:52:10.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 --2- 172.21.15.70:0/3349959675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740ca780 0x7fb8740cac30 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 -- 172.21.15.70:0/3349959675 >> 172.21.15.70:0/3349959675 conn(0x7fb8740ce760 msgr2=0x7fb8740ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:10.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 -- 172.21.15.70:0/3349959675 shutdown_connections 2023-08-25T09:52:10.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.326+0000 7fb87a287640 1 -- 172.21.15.70:0/3349959675 wait complete. 2023-08-25T09:52:10.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.328+0000 7fb87a287640 1 Processor -- start 2023-08-25T09:52:10.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.328+0000 7fb87a287640 1 -- start start 2023-08-25T09:52:10.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb87a287640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 0x7fb8740afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb87a287640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740ca780 0x7fb874142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb87a287640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87415acd0 0x7fb874144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb87a287640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8741438d0 con 0x7fb8740b3cf0 2023-08-25T09:52:10.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb87a287640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb874143bb0 con 0x7fb8740ca780 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb87a287640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb874143e90 con 0x7fb87415acd0 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb873fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 0x7fb8740afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb873fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 0x7fb8740afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43086/0 (socket says 172.21.15.70:43086) 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.329+0000 7fb873fff640 1 -- 172.21.15.70:0/1173218666 learned_addr learned my addr 172.21.15.70:0/1173218666 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb873fff640 1 -- 172.21.15.70:0/1173218666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87415acd0 msgr2=0x7fb874144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb873fff640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87415acd0 0x7fb874144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb873fff640 1 -- 172.21.15.70:0/1173218666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740ca780 msgr2=0x7fb874142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb873fff640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740ca780 0x7fb874142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb873fff640 1 -- 172.21.15.70:0/1173218666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c009400 con 0x7fb8740b3cf0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb873fff640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 0x7fb8740afd40 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b3e0 tx=0x7fb86400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864011830 con 0x7fb8740b3cf0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8640119b0 con 0x7fb8740b3cf0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864011da0 con 0x7fb8740b3cf0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb874144170 con 0x7fb8740b3cf0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.330+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb874150630 con 0x7fb8740b3cf0 2023-08-25T09:52:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.331+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb874144690 con 0x7fb8740b3cf0 2023-08-25T09:52:10.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.332+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb86401a400 con 0x7fb8740b3cf0 2023-08-25T09:52:10.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.333+0000 7fb8717fa640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb85406a050 0x7fb85406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.333+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb8640887d0 con 0x7fb8740b3cf0 2023-08-25T09:52:10.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.334+0000 7fb86bfff640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb85406a050 0x7fb85406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.334+0000 7fb86bfff640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb85406a050 0x7fb85406c510 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb85c036000 tx=0x7fb85c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:10.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.337+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb874144690 con 0x7fb8740b3cf0 2023-08-25T09:52:10.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.571+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb874144690 con 0x7fb8740b3cf0 2023-08-25T09:52:10.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.572+0000 7fb8717fa640 1 -- 172.21.15.70:0/1173218666 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1013 (secure 0 0 0) 0x7fb874144690 con 0x7fb8740b3cf0 2023-08-25T09:52:10.575 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:10.575 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":818,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":198,"num_osds":8,"num_up_osds":8,"osd_up_since":1692956318,"num_in_osds":8,"osd_in_since":1692956315,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":24,"data_bytes":592778,"bytes_used":10719956,"bytes_avail":1589280044,"bytes_total":1600000000,"write_bytes_sec":2387,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":91,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"a","status":"up:active","gid":9573}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2023-08-25T09:51:39.584865+0000","services":{}},"progress_events":{}} 2023-08-25T09:52:10.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb85406a050 msgr2=0x7fb85406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb85406a050 0x7fb85406c510 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb85c036000 tx=0x7fb85c030000 comp rx=0 tx=0).stop 2023-08-25T09:52:10.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 msgr2=0x7fb8740afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 0x7fb8740afd40 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7fb86400b3e0 tx=0x7fb86400b8a0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 shutdown_connections 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb85406a050 0x7fb85406c510 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb87415acd0 0x7fb874144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb8740ca780 0x7fb874142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 --2- 172.21.15.70:0/1173218666 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb8740b3cf0 0x7fb8740afd40 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 >> 172.21.15.70:0/1173218666 conn(0x7fb8740ce760 msgr2=0x7fb874153140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:10.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 shutdown_connections 2023-08-25T09:52:10.579 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.575+0000 7fb87a287640 1 -- 172.21.15.70:0/1173218666 wait complete. 2023-08-25T09:52:10.595 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-25T09:52:10.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:52:10.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.796+0000 7f28fcd6a640 1 Processor -- start 2023-08-25T09:52:10.799 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.796+0000 7f28fcd6a640 1 -- start start 2023-08-25T09:52:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.797+0000 7f28fcd6a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f28f80ce5b0 0x7f28f80ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28fcd6a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f28f80b2850 0x7f28f806f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28fcd6a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f80ceed0 con 0x7f28f815aa10 2023-08-25T09:52:10.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28fcd6a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f8159d10 con 0x7f28f80b2850 2023-08-25T09:52:10.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28fcd6a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f815d3f0 con 0x7f28f80ce5b0 2023-08-25T09:52:10.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28f6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f28f80b2850 0x7f28f806f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28f6d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f28f80b2850 0x7f28f806f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43098/0 (socket says 172.21.15.70:43098) 2023-08-25T09:52:10.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.798+0000 7f28f6d9d640 1 -- 172.21.15.70:0/273033604 learned_addr learned my addr 172.21.15.70:0/273033604 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:10.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.799+0000 7f28f659c640 1 --2- 172.21.15.70:0/273033604 >> v2:172.21.15.155:3300/0 conn(0x7f28f80ce5b0 0x7f28f80ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.799+0000 7f28f6d9d640 1 -- 172.21.15.70:0/273033604 >> v2:172.21.15.155:3300/0 conn(0x7f28f80ce5b0 msgr2=0x7f28f80ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:10.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.799+0000 7f28f6d9d640 1 --2- 172.21.15.70:0/273033604 >> v2:172.21.15.155:3300/0 conn(0x7f28f80ce5b0 0x7f28f80ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.799+0000 7f28f6d9d640 1 -- 172.21.15.70:0/273033604 >> v1:172.21.15.155:6789/0 conn(0x7f28f815aa10 legacy=0x7f28f80b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:10.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.799+0000 7f28f6d9d640 1 -- 172.21.15.70:0/273033604 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f80b2d20 con 0x7f28f80b2850 2023-08-25T09:52:10.805 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.800+0000 7f28f6d9d640 1 --2- 172.21.15.70:0/273033604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80b2850 0x7f28f806f9f0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f28f806f470 tx=0x7f28f002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70895a3450deac5b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:10.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.800+0000 7f28f5d9b640 1 -- 172.21.15.70:0/273033604 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f002c9f0 con 0x7f28f80b2850 2023-08-25T09:52:10.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.800+0000 7f28f5d9b640 1 -- 172.21.15.70:0/273033604 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f002cb50 con 0x7f28f80b2850 2023-08-25T09:52:10.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.800+0000 7f28f5d9b640 1 -- 172.21.15.70:0/273033604 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f00343f0 con 0x7f28f80b2850 2023-08-25T09:52:10.806 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 -- 172.21.15.70:0/273033604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80b2850 msgr2=0x7f28f806f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/273033604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80b2850 0x7f28f806f9f0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f28f806f470 tx=0x7f28f002be50 comp rx=0 tx=0).stop 2023-08-25T09:52:10.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 -- 172.21.15.70:0/273033604 shutdown_connections 2023-08-25T09:52:10.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/273033604 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80b2850 0x7f28f806f9f0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/273033604 >> v2:172.21.15.155:3300/0 conn(0x7f28f80ce5b0 0x7f28f80ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 -- 172.21.15.70:0/273033604 >> 172.21.15.70:0/273033604 conn(0x7f28f8069850 msgr2=0x7f28f8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 -- 172.21.15.70:0/273033604 shutdown_connections 2023-08-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.801+0000 7f28fcd6a640 1 -- 172.21.15.70:0/273033604 wait complete. 2023-08-25T09:52:10.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.803+0000 7f28fcd6a640 1 Processor -- start 2023-08-25T09:52:10.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.803+0000 7f28fcd6a640 1 -- start start 2023-08-25T09:52:10.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28fcd6a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28f80b2850 0x7f28f8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28fcd6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 0x7f28f8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28fcd6a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28f815aa10 0x7f28f80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28fcd6a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8154710 con 0x7f28f80ce5b0 2023-08-25T09:52:10.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28fcd6a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f814c850 con 0x7f28f80b2850 2023-08-25T09:52:10.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28fcd6a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80cd540 con 0x7f28f815aa10 2023-08-25T09:52:10.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 0x7f28f8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 0x7f28f8150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43100/0 (socket says 172.21.15.70:43100) 2023-08-25T09:52:10.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 -- 172.21.15.70:0/1116784298 learned_addr learned my addr 172.21.15.70:0/1116784298 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:10.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28f659c640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28f80b2850 0x7f28f8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 -- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28f815aa10 msgr2=0x7f28f80ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28f6d9d640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28f815aa10 0x7f28f80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28f815aa10 0x7f28f80ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.815 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 -- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28f80b2850 msgr2=0x7f28f8159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:10.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28f80b2850 0x7f28f8159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:10.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.804+0000 7f28effff640 1 -- 172.21.15.70:0/1116784298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0009b30 con 0x7f28f80ce5b0 2023-08-25T09:52:10.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.805+0000 7f28effff640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 0x7f28f8150e20 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b3f0 tx=0x7f28e000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:10.816 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.805+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0011830 con 0x7f28f80ce5b0 2023-08-25T09:52:10.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.805+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e0011990 con 0x7f28f80ce5b0 2023-08-25T09:52:10.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.805+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0011d40 con 0x7f28f80ce5b0 2023-08-25T09:52:10.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.805+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f8153d70 con 0x7f28f80ce5b0 2023-08-25T09:52:10.817 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.805+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f8154330 con 0x7f28f80ce5b0 2023-08-25T09:52:10.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.806+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f8154850 con 0x7f28f80ce5b0 2023-08-25T09:52:10.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.807+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f28e001a400 con 0x7f28f80ce5b0 2023-08-25T09:52:10.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.808+0000 7f28eeffd640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f28d806a050 0x7f28d806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:10.818 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.808+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f28e0088410 con 0x7f28f80ce5b0 2023-08-25T09:52:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.808+0000 7f28f659c640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f28d806a050 0x7f28d806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.809+0000 7f28f659c640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f28d806a050 0x7f28d806c510 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f28f80ccf80 tx=0x7f28e8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:10.820 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.812+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f28f8154850 con 0x7f28f80ce5b0 2023-08-25T09:52:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:10.993+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28f815d990 con 0x7f28d806a050 2023-08-25T09:52:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.017+0000 7f28eeffd640 1 -- 172.21.15.70:0/1116784298 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302974 (secure 0 0 0) 0x7f28e800c050 con 0x7f28d806a050 2023-08-25T09:52:11.021 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:11.025 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:52:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f28d806a050 msgr2=0x7f28d806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f28d806a050 0x7f28d806c510 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f28f80ccf80 tx=0x7f28e8008040 comp rx=0 tx=0).stop 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 msgr2=0x7f28f8150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 0x7f28f8150e20 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f28e000b3f0 tx=0x7f28e000b8b0 comp rx=0 tx=0).stop 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 shutdown_connections 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f28d806a050 0x7f28d806c510 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f28f815aa10 0x7f28f80ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f28f80ce5b0 0x7f28f8150e20 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 --2- 172.21.15.70:0/1116784298 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f28f80b2850 0x7f28f8159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.024+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 >> 172.21.15.70:0/1116784298 conn(0x7f28f8069850 msgr2=0x7f28f8070920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.025+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 shutdown_connections 2023-08-25T09:52:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:11.025+0000 7f28fcd6a640 1 -- 172.21.15.70:0/1116784298 wait complete. 2023-08-25T09:52:11.044 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":595,"stamp":"2023-08-25T09:52:09.592743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10468,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552031,"statfs":{"total":1600000000,"available":1589280044,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.005760"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978214+0000","last_change":"2023-08-25T09:52:04.628641+0000","last_active":"2023-08-25T09:52:06.978214+0000","last_peered":"2023-08-25T09:52:06.978214+0000","last_clean":"2023-08-25T09:52:06.978214+0000","last_became_active":"2023-08-25T09:52:04.627891+0000","last_became_peered":"2023-08-25T09:52:04.627891+0000","last_unstale":"2023-08-25T09:52:06.978214+0000","last_undegraded":"2023-08-25T09:52:06.978214+0000","last_fullsized":"2023-08-25T09:52:06.978214+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:41:47.154534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979875+0000","last_change":"2023-08-25T09:52:04.625251+0000","last_active":"2023-08-25T09:52:06.979875+0000","last_peered":"2023-08-25T09:52:06.979875+0000","last_clean":"2023-08-25T09:52:06.979875+0000","last_became_active":"2023-08-25T09:52:04.624800+0000","last_became_peered":"2023-08-25T09:52:04.624800+0000","last_unstale":"2023-08-25T09:52:06.979875+0000","last_undegraded":"2023-08-25T09:52:06.979875+0000","last_fullsized":"2023-08-25T09:52:06.979875+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:41:47.867907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978509+0000","last_change":"2023-08-25T09:52:04.617930+0000","last_active":"2023-08-25T09:52:06.978509+0000","last_peered":"2023-08-25T09:52:06.978509+0000","last_clean":"2023-08-25T09:52:06.978509+0000","last_became_active":"2023-08-25T09:52:04.614336+0000","last_became_peered":"2023-08-25T09:52:04.614336+0000","last_unstale":"2023-08-25T09:52:06.978509+0000","last_undegraded":"2023-08-25T09:52:06.978509+0000","last_fullsized":"2023-08-25T09:52:06.978509+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:06.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977490+0000","last_change":"2023-08-25T09:52:04.627289+0000","last_active":"2023-08-25T09:52:06.977490+0000","last_peered":"2023-08-25T09:52:06.977490+0000","last_clean":"2023-08-25T09:52:06.977490+0000","last_became_active":"2023-08-25T09:52:04.625811+0000","last_became_peered":"2023-08-25T09:52:04.625811+0000","last_unstale":"2023-08-25T09:52:06.977490+0000","last_undegraded":"2023-08-25T09:52:06.977490+0000","last_fullsized":"2023-08-25T09:52:06.977490+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:21:46.850988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598128+0000","last_change":"2023-08-25T09:52:04.617186+0000","last_active":"2023-08-25T09:52:05.598128+0000","last_peered":"2023-08-25T09:52:05.598128+0000","last_clean":"2023-08-25T09:52:05.598128+0000","last_became_active":"2023-08-25T09:52:04.616036+0000","last_became_peered":"2023-08-25T09:52:04.616036+0000","last_unstale":"2023-08-25T09:52:05.598128+0000","last_undegraded":"2023-08-25T09:52:05.598128+0000","last_fullsized":"2023-08-25T09:52:05.598128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:27.953431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978450+0000","last_change":"2023-08-25T09:52:04.628367+0000","last_active":"2023-08-25T09:52:06.978450+0000","last_peered":"2023-08-25T09:52:06.978450+0000","last_clean":"2023-08-25T09:52:06.978450+0000","last_became_active":"2023-08-25T09:52:04.626874+0000","last_became_peered":"2023-08-25T09:52:04.626874+0000","last_unstale":"2023-08-25T09:52:06.978450+0000","last_undegraded":"2023-08-25T09:52:06.978450+0000","last_fullsized":"2023-08-25T09:52:06.978450+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:04.237329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599734+0000","last_change":"2023-08-25T09:52:04.612395+0000","last_active":"2023-08-25T09:52:05.599734+0000","last_peered":"2023-08-25T09:52:05.599734+0000","last_clean":"2023-08-25T09:52:05.599734+0000","last_became_active":"2023-08-25T09:52:04.607152+0000","last_became_peered":"2023-08-25T09:52:04.607152+0000","last_unstale":"2023-08-25T09:52:05.599734+0000","last_undegraded":"2023-08-25T09:52:05.599734+0000","last_fullsized":"2023-08-25T09:52:05.599734+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:51.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978190+0000","last_change":"2023-08-25T09:52:04.624655+0000","last_active":"2023-08-25T09:52:06.978190+0000","last_peered":"2023-08-25T09:52:06.978190+0000","last_clean":"2023-08-25T09:52:06.978190+0000","last_became_active":"2023-08-25T09:52:04.624478+0000","last_became_peered":"2023-08-25T09:52:04.624478+0000","last_unstale":"2023-08-25T09:52:06.978190+0000","last_undegraded":"2023-08-25T09:52:06.978190+0000","last_fullsized":"2023-08-25T09:52:06.978190+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:34.884721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599300+0000","last_change":"2023-08-25T09:52:04.626361+0000","last_active":"2023-08-25T09:52:05.599300+0000","last_peered":"2023-08-25T09:52:05.599300+0000","last_clean":"2023-08-25T09:52:05.599300+0000","last_became_active":"2023-08-25T09:52:04.625812+0000","last_became_peered":"2023-08-25T09:52:04.625812+0000","last_unstale":"2023-08-25T09:52:05.599300+0000","last_undegraded":"2023-08-25T09:52:05.599300+0000","last_fullsized":"2023-08-25T09:52:05.599300+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:04.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598504+0000","last_change":"2023-08-25T09:52:04.612349+0000","last_active":"2023-08-25T09:52:05.598504+0000","last_peered":"2023-08-25T09:52:05.598504+0000","last_clean":"2023-08-25T09:52:05.598504+0000","last_became_active":"2023-08-25T09:52:04.611885+0000","last_became_peered":"2023-08-25T09:52:04.611885+0000","last_unstale":"2023-08-25T09:52:05.598504+0000","last_undegraded":"2023-08-25T09:52:05.598504+0000","last_fullsized":"2023-08-25T09:52:05.598504+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:36.629581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978698+0000","last_change":"2023-08-25T09:52:04.627125+0000","last_active":"2023-08-25T09:52:06.978698+0000","last_peered":"2023-08-25T09:52:06.978698+0000","last_clean":"2023-08-25T09:52:06.978698+0000","last_became_active":"2023-08-25T09:52:04.625250+0000","last_became_peered":"2023-08-25T09:52:04.625250+0000","last_unstale":"2023-08-25T09:52:06.978698+0000","last_undegraded":"2023-08-25T09:52:06.978698+0000","last_fullsized":"2023-08-25T09:52:06.978698+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:34.884721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977263+0000","last_change":"2023-08-25T09:52:04.624230+0000","last_active":"2023-08-25T09:52:06.977263+0000","last_peered":"2023-08-25T09:52:06.977263+0000","last_clean":"2023-08-25T09:52:06.977263+0000","last_became_active":"2023-08-25T09:52:04.623981+0000","last_became_peered":"2023-08-25T09:52:04.623981+0000","last_unstale":"2023-08-25T09:52:06.977263+0000","last_undegraded":"2023-08-25T09:52:06.977263+0000","last_fullsized":"2023-08-25T09:52:06.977263+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:18:08.663525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601907+0000","last_change":"2023-08-25T09:52:04.624126+0000","last_active":"2023-08-25T09:52:05.601907+0000","last_peered":"2023-08-25T09:52:05.601907+0000","last_clean":"2023-08-25T09:52:05.601907+0000","last_became_active":"2023-08-25T09:52:04.623878+0000","last_became_peered":"2023-08-25T09:52:04.623878+0000","last_unstale":"2023-08-25T09:52:05.601907+0000","last_undegraded":"2023-08-25T09:52:05.601907+0000","last_fullsized":"2023-08-25T09:52:05.601907+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:58.851337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977983+0000","last_change":"2023-08-25T09:52:04.620817+0000","last_active":"2023-08-25T09:52:06.977983+0000","last_peered":"2023-08-25T09:52:06.977983+0000","last_clean":"2023-08-25T09:52:06.977983+0000","last_became_active":"2023-08-25T09:52:04.620537+0000","last_became_peered":"2023-08-25T09:52:04.620537+0000","last_unstale":"2023-08-25T09:52:06.977983+0000","last_undegraded":"2023-08-25T09:52:06.977983+0000","last_fullsized":"2023-08-25T09:52:06.977983+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:32.675064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978306+0000","last_change":"2023-08-25T09:52:04.622813+0000","last_active":"2023-08-25T09:52:06.978306+0000","last_peered":"2023-08-25T09:52:06.978306+0000","last_clean":"2023-08-25T09:52:06.978306+0000","last_became_active":"2023-08-25T09:52:04.622347+0000","last_became_peered":"2023-08-25T09:52:04.622347+0000","last_unstale":"2023-08-25T09:52:06.978306+0000","last_undegraded":"2023-08-25T09:52:06.978306+0000","last_fullsized":"2023-08-25T09:52:06.978306+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:45:01.358105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978917+0000","last_change":"2023-08-25T09:52:04.627414+0000","last_active":"2023-08-25T09:52:06.978917+0000","last_peered":"2023-08-25T09:52:06.978917+0000","last_clean":"2023-08-25T09:52:06.978917+0000","last_became_active":"2023-08-25T09:52:04.625586+0000","last_became_peered":"2023-08-25T09:52:04.625586+0000","last_unstale":"2023-08-25T09:52:06.978917+0000","last_undegraded":"2023-08-25T09:52:06.978917+0000","last_fullsized":"2023-08-25T09:52:06.978917+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:23.051366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.597997+0000","last_change":"2023-08-25T09:52:04.622725+0000","last_active":"2023-08-25T09:52:05.597997+0000","last_peered":"2023-08-25T09:52:05.597997+0000","last_clean":"2023-08-25T09:52:05.597997+0000","last_became_active":"2023-08-25T09:52:04.622326+0000","last_became_peered":"2023-08-25T09:52:04.622326+0000","last_unstale":"2023-08-25T09:52:05.597997+0000","last_undegraded":"2023-08-25T09:52:05.597997+0000","last_fullsized":"2023-08-25T09:52:05.597997+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:40.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594716+0000","last_change":"2023-08-25T09:52:04.613623+0000","last_active":"2023-08-25T09:52:05.594716+0000","last_peered":"2023-08-25T09:52:05.594716+0000","last_clean":"2023-08-25T09:52:05.594716+0000","last_became_active":"2023-08-25T09:52:04.612966+0000","last_became_peered":"2023-08-25T09:52:04.612966+0000","last_unstale":"2023-08-25T09:52:05.594716+0000","last_undegraded":"2023-08-25T09:52:05.594716+0000","last_fullsized":"2023-08-25T09:52:05.594716+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:08.314372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979043+0000","last_change":"2023-08-25T09:52:04.627459+0000","last_active":"2023-08-25T09:52:06.979043+0000","last_peered":"2023-08-25T09:52:06.979043+0000","last_clean":"2023-08-25T09:52:06.979043+0000","last_became_active":"2023-08-25T09:52:04.626892+0000","last_became_peered":"2023-08-25T09:52:04.626892+0000","last_unstale":"2023-08-25T09:52:06.979043+0000","last_undegraded":"2023-08-25T09:52:06.979043+0000","last_fullsized":"2023-08-25T09:52:06.979043+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:00:20.842644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978283+0000","last_change":"2023-08-25T09:52:04.612691+0000","last_active":"2023-08-25T09:52:06.978283+0000","last_peered":"2023-08-25T09:52:06.978283+0000","last_clean":"2023-08-25T09:52:06.978283+0000","last_became_active":"2023-08-25T09:52:04.612199+0000","last_became_peered":"2023-08-25T09:52:04.612199+0000","last_unstale":"2023-08-25T09:52:06.978283+0000","last_undegraded":"2023-08-25T09:52:06.978283+0000","last_fullsized":"2023-08-25T09:52:06.978283+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:23.051366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599582+0000","last_change":"2023-08-25T09:52:04.626435+0000","last_active":"2023-08-25T09:52:05.599582+0000","last_peered":"2023-08-25T09:52:05.599582+0000","last_clean":"2023-08-25T09:52:05.599582+0000","last_became_active":"2023-08-25T09:52:04.626050+0000","last_became_peered":"2023-08-25T09:52:04.626050+0000","last_unstale":"2023-08-25T09:52:05.599582+0000","last_undegraded":"2023-08-25T09:52:05.599582+0000","last_fullsized":"2023-08-25T09:52:05.599582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:35:30.431066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977748+0000","last_change":"2023-08-25T09:52:04.625300+0000","last_active":"2023-08-25T09:52:06.977748+0000","last_peered":"2023-08-25T09:52:06.977748+0000","last_clean":"2023-08-25T09:52:06.977748+0000","last_became_active":"2023-08-25T09:52:04.624890+0000","last_became_peered":"2023-08-25T09:52:04.624890+0000","last_unstale":"2023-08-25T09:52:06.977748+0000","last_undegraded":"2023-08-25T09:52:06.977748+0000","last_fullsized":"2023-08-25T09:52:06.977748+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:04:08.579191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598557+0000","last_change":"2023-08-25T09:52:04.614691+0000","last_active":"2023-08-25T09:52:05.598557+0000","last_peered":"2023-08-25T09:52:05.598557+0000","last_clean":"2023-08-25T09:52:05.598557+0000","last_became_active":"2023-08-25T09:52:04.614305+0000","last_became_peered":"2023-08-25T09:52:04.614305+0000","last_unstale":"2023-08-25T09:52:05.598557+0000","last_undegraded":"2023-08-25T09:52:05.598557+0000","last_fullsized":"2023-08-25T09:52:05.598557+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:41:57.725402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979364+0000","last_change":"2023-08-25T09:52:04.627696+0000","last_active":"2023-08-25T09:52:06.979364+0000","last_peered":"2023-08-25T09:52:06.979364+0000","last_clean":"2023-08-25T09:52:06.979364+0000","last_became_active":"2023-08-25T09:52:04.627211+0000","last_became_peered":"2023-08-25T09:52:04.627211+0000","last_unstale":"2023-08-25T09:52:06.979364+0000","last_undegraded":"2023-08-25T09:52:06.979364+0000","last_fullsized":"2023-08-25T09:52:06.979364+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:16.624825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599254+0000","last_change":"2023-08-25T09:52:04.614199+0000","last_active":"2023-08-25T09:52:05.599254+0000","last_peered":"2023-08-25T09:52:05.599254+0000","last_clean":"2023-08-25T09:52:05.599254+0000","last_became_active":"2023-08-25T09:52:04.613179+0000","last_became_peered":"2023-08-25T09:52:04.613179+0000","last_unstale":"2023-08-25T09:52:05.599254+0000","last_undegraded":"2023-08-25T09:52:05.599254+0000","last_fullsized":"2023-08-25T09:52:05.599254+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:42.152837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979806+0000","last_change":"2023-08-25T09:52:04.625832+0000","last_active":"2023-08-25T09:52:06.979806+0000","last_peered":"2023-08-25T09:52:06.979806+0000","last_clean":"2023-08-25T09:52:06.979806+0000","last_became_active":"2023-08-25T09:52:04.625323+0000","last_became_peered":"2023-08-25T09:52:04.625323+0000","last_unstale":"2023-08-25T09:52:06.979806+0000","last_undegraded":"2023-08-25T09:52:06.979806+0000","last_fullsized":"2023-08-25T09:52:06.979806+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:32.060870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980040+0000","last_change":"2023-08-25T09:52:04.606510+0000","last_active":"2023-08-25T09:52:06.980040+0000","last_peered":"2023-08-25T09:52:06.980040+0000","last_clean":"2023-08-25T09:52:06.980040+0000","last_became_active":"2023-08-25T09:52:04.605382+0000","last_became_peered":"2023-08-25T09:52:04.605382+0000","last_unstale":"2023-08-25T09:52:06.980040+0000","last_undegraded":"2023-08-25T09:52:06.980040+0000","last_fullsized":"2023-08-25T09:52:06.980040+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:15.708169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600531+0000","last_change":"2023-08-25T09:52:04.616677+0000","last_active":"2023-08-25T09:52:05.600531+0000","last_peered":"2023-08-25T09:52:05.600531+0000","last_clean":"2023-08-25T09:52:05.600531+0000","last_became_active":"2023-08-25T09:52:04.614596+0000","last_became_peered":"2023-08-25T09:52:04.614596+0000","last_unstale":"2023-08-25T09:52:05.600531+0000","last_undegraded":"2023-08-25T09:52:05.600531+0000","last_fullsized":"2023-08-25T09:52:05.600531+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:53.023481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594927+0000","last_change":"2023-08-25T09:52:04.609767+0000","last_active":"2023-08-25T09:52:05.594927+0000","last_peered":"2023-08-25T09:52:05.594927+0000","last_clean":"2023-08-25T09:52:05.594927+0000","last_became_active":"2023-08-25T09:52:04.606031+0000","last_became_peered":"2023-08-25T09:52:04.606031+0000","last_unstale":"2023-08-25T09:52:05.594927+0000","last_undegraded":"2023-08-25T09:52:05.594927+0000","last_fullsized":"2023-08-25T09:52:05.594927+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:06.542127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979128+0000","last_change":"2023-08-25T09:52:04.622622+0000","last_active":"2023-08-25T09:52:06.979128+0000","last_peered":"2023-08-25T09:52:06.979128+0000","last_clean":"2023-08-25T09:52:06.979128+0000","last_became_active":"2023-08-25T09:52:04.621916+0000","last_became_peered":"2023-08-25T09:52:04.621916+0000","last_unstale":"2023-08-25T09:52:06.979128+0000","last_undegraded":"2023-08-25T09:52:06.979128+0000","last_fullsized":"2023-08-25T09:52:06.979128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:52.575789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976919+0000","last_change":"2023-08-25T09:52:04.629777+0000","last_active":"2023-08-25T09:52:06.976919+0000","last_peered":"2023-08-25T09:52:06.976919+0000","last_clean":"2023-08-25T09:52:06.976919+0000","last_became_active":"2023-08-25T09:52:04.628214+0000","last_became_peered":"2023-08-25T09:52:04.628214+0000","last_unstale":"2023-08-25T09:52:06.976919+0000","last_undegraded":"2023-08-25T09:52:06.976919+0000","last_fullsized":"2023-08-25T09:52:06.976919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:49:37.628859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980187+0000","last_change":"2023-08-25T09:52:04.606056+0000","last_active":"2023-08-25T09:52:06.980187+0000","last_peered":"2023-08-25T09:52:06.980187+0000","last_clean":"2023-08-25T09:52:06.980187+0000","last_became_active":"2023-08-25T09:52:04.604555+0000","last_became_peered":"2023-08-25T09:52:04.604555+0000","last_unstale":"2023-08-25T09:52:06.980187+0000","last_undegraded":"2023-08-25T09:52:06.980187+0000","last_fullsized":"2023-08-25T09:52:06.980187+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:49:02.585201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978773+0000","last_change":"2023-08-25T09:52:04.624540+0000","last_active":"2023-08-25T09:52:06.978773+0000","last_peered":"2023-08-25T09:52:06.978773+0000","last_clean":"2023-08-25T09:52:06.978773+0000","last_became_active":"2023-08-25T09:52:04.624033+0000","last_became_peered":"2023-08-25T09:52:04.624033+0000","last_unstale":"2023-08-25T09:52:06.978773+0000","last_undegraded":"2023-08-25T09:52:06.978773+0000","last_fullsized":"2023-08-25T09:52:06.978773+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:22.449654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"197'4","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979006+0000","last_change":"2023-08-25T09:52:03.605509+0000","last_active":"2023-08-25T09:52:06.979006+0000","last_peered":"2023-08-25T09:52:06.979006+0000","last_clean":"2023-08-25T09:52:06.979006+0000","last_became_active":"2023-08-25T09:52:03.596779+0000","last_became_peered":"2023-08-25T09:52:03.596779+0000","last_unstale":"2023-08-25T09:52:06.979006+0000","last_undegraded":"2023-08-25T09:52:06.979006+0000","last_fullsized":"2023-08-25T09:52:06.979006+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:55.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600442+0000","last_change":"2023-08-25T09:52:04.621176+0000","last_active":"2023-08-25T09:52:05.600442+0000","last_peered":"2023-08-25T09:52:05.600442+0000","last_clean":"2023-08-25T09:52:05.600442+0000","last_became_active":"2023-08-25T09:52:04.620648+0000","last_became_peered":"2023-08-25T09:52:04.620648+0000","last_unstale":"2023-08-25T09:52:05.600442+0000","last_undegraded":"2023-08-25T09:52:05.600442+0000","last_fullsized":"2023-08-25T09:52:05.600442+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:37.317841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977679+0000","last_change":"2023-08-25T09:52:03.618376+0000","last_active":"2023-08-25T09:52:06.977679+0000","last_peered":"2023-08-25T09:52:06.977679+0000","last_clean":"2023-08-25T09:52:06.977679+0000","last_became_active":"2023-08-25T09:52:03.617343+0000","last_became_peered":"2023-08-25T09:52:03.617343+0000","last_unstale":"2023-08-25T09:52:06.977679+0000","last_undegraded":"2023-08-25T09:52:06.977679+0000","last_fullsized":"2023-08-25T09:52:06.977679+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:45:00.348861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598202+0000","last_change":"2023-08-25T09:52:04.615114+0000","last_active":"2023-08-25T09:52:05.598202+0000","last_peered":"2023-08-25T09:52:05.598202+0000","last_clean":"2023-08-25T09:52:05.598202+0000","last_became_active":"2023-08-25T09:52:04.614691+0000","last_became_peered":"2023-08-25T09:52:04.614691+0000","last_unstale":"2023-08-25T09:52:05.598202+0000","last_undegraded":"2023-08-25T09:52:05.598202+0000","last_fullsized":"2023-08-25T09:52:05.598202+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:54.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976923+0000","last_change":"2023-08-25T09:52:03.615544+0000","last_active":"2023-08-25T09:52:06.976923+0000","last_peered":"2023-08-25T09:52:06.976923+0000","last_clean":"2023-08-25T09:52:06.976923+0000","last_became_active":"2023-08-25T09:52:03.615200+0000","last_became_peered":"2023-08-25T09:52:03.615200+0000","last_unstale":"2023-08-25T09:52:06.976923+0000","last_undegraded":"2023-08-25T09:52:06.976923+0000","last_fullsized":"2023-08-25T09:52:06.976923+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:24:11.361908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976773+0000","last_change":"2023-08-25T09:52:04.622820+0000","last_active":"2023-08-25T09:52:06.976773+0000","last_peered":"2023-08-25T09:52:06.976773+0000","last_clean":"2023-08-25T09:52:06.976773+0000","last_became_active":"2023-08-25T09:52:04.622556+0000","last_became_peered":"2023-08-25T09:52:04.622556+0000","last_unstale":"2023-08-25T09:52:06.976773+0000","last_undegraded":"2023-08-25T09:52:06.976773+0000","last_fullsized":"2023-08-25T09:52:06.976773+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:12.796779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599945+0000","last_change":"2023-08-25T09:52:03.615665+0000","last_active":"2023-08-25T09:52:05.599945+0000","last_peered":"2023-08-25T09:52:05.599945+0000","last_clean":"2023-08-25T09:52:05.599945+0000","last_became_active":"2023-08-25T09:52:03.615398+0000","last_became_peered":"2023-08-25T09:52:03.615398+0000","last_unstale":"2023-08-25T09:52:05.599945+0000","last_undegraded":"2023-08-25T09:52:05.599945+0000","last_fullsized":"2023-08-25T09:52:05.599945+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:24.261532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598463+0000","last_change":"2023-08-25T09:52:04.606215+0000","last_active":"2023-08-25T09:52:05.598463+0000","last_peered":"2023-08-25T09:52:05.598463+0000","last_clean":"2023-08-25T09:52:05.598463+0000","last_became_active":"2023-08-25T09:52:04.605569+0000","last_became_peered":"2023-08-25T09:52:04.605569+0000","last_unstale":"2023-08-25T09:52:05.598463+0000","last_undegraded":"2023-08-25T09:52:05.598463+0000","last_fullsized":"2023-08-25T09:52:05.598463+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:33.270874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600761+0000","last_change":"2023-08-25T09:52:03.612691+0000","last_active":"2023-08-25T09:52:05.600761+0000","last_peered":"2023-08-25T09:52:05.600761+0000","last_clean":"2023-08-25T09:52:05.600761+0000","last_became_active":"2023-08-25T09:52:03.609712+0000","last_became_peered":"2023-08-25T09:52:03.609712+0000","last_unstale":"2023-08-25T09:52:05.600761+0000","last_undegraded":"2023-08-25T09:52:05.600761+0000","last_fullsized":"2023-08-25T09:52:05.600761+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:18.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601189+0000","last_change":"2023-08-25T09:52:04.619419+0000","last_active":"2023-08-25T09:52:05.601189+0000","last_peered":"2023-08-25T09:52:05.601189+0000","last_clean":"2023-08-25T09:52:05.601189+0000","last_became_active":"2023-08-25T09:52:04.616364+0000","last_became_peered":"2023-08-25T09:52:04.616364+0000","last_unstale":"2023-08-25T09:52:05.601189+0000","last_undegraded":"2023-08-25T09:52:05.601189+0000","last_fullsized":"2023-08-25T09:52:05.601189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:19.369746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978543+0000","last_change":"2023-08-25T09:52:03.612785+0000","last_active":"2023-08-25T09:52:06.978543+0000","last_peered":"2023-08-25T09:52:06.978543+0000","last_clean":"2023-08-25T09:52:06.978543+0000","last_became_active":"2023-08-25T09:52:03.612129+0000","last_became_peered":"2023-08-25T09:52:03.612129+0000","last_unstale":"2023-08-25T09:52:06.978543+0000","last_undegraded":"2023-08-25T09:52:06.978543+0000","last_fullsized":"2023-08-25T09:52:06.978543+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:32.325636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598377+0000","last_change":"2023-08-25T09:52:04.622547+0000","last_active":"2023-08-25T09:52:05.598377+0000","last_peered":"2023-08-25T09:52:05.598377+0000","last_clean":"2023-08-25T09:52:05.598377+0000","last_became_active":"2023-08-25T09:52:04.621941+0000","last_became_peered":"2023-08-25T09:52:04.621941+0000","last_unstale":"2023-08-25T09:52:05.598377+0000","last_undegraded":"2023-08-25T09:52:05.598377+0000","last_fullsized":"2023-08-25T09:52:05.598377+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:49.426665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978061+0000","last_change":"2023-08-25T09:52:03.618689+0000","last_active":"2023-08-25T09:52:06.978061+0000","last_peered":"2023-08-25T09:52:06.978061+0000","last_clean":"2023-08-25T09:52:06.978061+0000","last_became_active":"2023-08-25T09:52:03.617671+0000","last_became_peered":"2023-08-25T09:52:03.617671+0000","last_unstale":"2023-08-25T09:52:06.978061+0000","last_undegraded":"2023-08-25T09:52:06.978061+0000","last_fullsized":"2023-08-25T09:52:06.978061+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:03.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594833+0000","last_change":"2023-08-25T09:52:04.626763+0000","last_active":"2023-08-25T09:52:05.594833+0000","last_peered":"2023-08-25T09:52:05.594833+0000","last_clean":"2023-08-25T09:52:05.594833+0000","last_became_active":"2023-08-25T09:52:04.626503+0000","last_became_peered":"2023-08-25T09:52:04.626503+0000","last_unstale":"2023-08-25T09:52:05.594833+0000","last_undegraded":"2023-08-25T09:52:05.594833+0000","last_fullsized":"2023-08-25T09:52:05.594833+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:22:38.126861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979938+0000","last_change":"2023-08-25T09:52:03.603589+0000","last_active":"2023-08-25T09:52:06.979938+0000","last_peered":"2023-08-25T09:52:06.979938+0000","last_clean":"2023-08-25T09:52:06.979938+0000","last_became_active":"2023-08-25T09:52:03.602798+0000","last_became_peered":"2023-08-25T09:52:03.602798+0000","last_unstale":"2023-08-25T09:52:06.979938+0000","last_undegraded":"2023-08-25T09:52:06.979938+0000","last_fullsized":"2023-08-25T09:52:06.979938+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:18.360502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979138+0000","last_change":"2023-08-25T09:52:04.617158+0000","last_active":"2023-08-25T09:52:06.979138+0000","last_peered":"2023-08-25T09:52:06.979138+0000","last_clean":"2023-08-25T09:52:06.979138+0000","last_became_active":"2023-08-25T09:52:04.616462+0000","last_became_peered":"2023-08-25T09:52:04.616462+0000","last_unstale":"2023-08-25T09:52:06.979138+0000","last_undegraded":"2023-08-25T09:52:06.979138+0000","last_fullsized":"2023-08-25T09:52:06.979138+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:15:24.095300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979469+0000","last_change":"2023-08-25T09:52:03.618945+0000","last_active":"2023-08-25T09:52:06.979469+0000","last_peered":"2023-08-25T09:52:06.979469+0000","last_clean":"2023-08-25T09:52:06.979469+0000","last_became_active":"2023-08-25T09:52:03.618057+0000","last_became_peered":"2023-08-25T09:52:03.618057+0000","last_unstale":"2023-08-25T09:52:06.979469+0000","last_undegraded":"2023-08-25T09:52:06.979469+0000","last_fullsized":"2023-08-25T09:52:06.979469+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:05.630395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977905+0000","last_change":"2023-08-25T09:52:04.613446+0000","last_active":"2023-08-25T09:52:06.977905+0000","last_peered":"2023-08-25T09:52:06.977905+0000","last_clean":"2023-08-25T09:52:06.977905+0000","last_became_active":"2023-08-25T09:52:04.612830+0000","last_became_peered":"2023-08-25T09:52:04.612830+0000","last_unstale":"2023-08-25T09:52:06.977905+0000","last_undegraded":"2023-08-25T09:52:06.977905+0000","last_fullsized":"2023-08-25T09:52:06.977905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:40.373369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598883+0000","last_change":"2023-08-25T09:52:03.617112+0000","last_active":"2023-08-25T09:52:05.598883+0000","last_peered":"2023-08-25T09:52:05.598883+0000","last_clean":"2023-08-25T09:52:05.598883+0000","last_became_active":"2023-08-25T09:52:03.616870+0000","last_became_peered":"2023-08-25T09:52:03.616870+0000","last_unstale":"2023-08-25T09:52:05.598883+0000","last_undegraded":"2023-08-25T09:52:05.598883+0000","last_fullsized":"2023-08-25T09:52:05.598883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:14.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977880+0000","last_change":"2023-08-25T09:52:04.618901+0000","last_active":"2023-08-25T09:52:06.977880+0000","last_peered":"2023-08-25T09:52:06.977880+0000","last_clean":"2023-08-25T09:52:06.977880+0000","last_became_active":"2023-08-25T09:52:04.618464+0000","last_became_peered":"2023-08-25T09:52:04.618464+0000","last_unstale":"2023-08-25T09:52:06.977880+0000","last_undegraded":"2023-08-25T09:52:06.977880+0000","last_fullsized":"2023-08-25T09:52:06.977880+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:34.504532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980224+0000","last_change":"2023-08-25T09:52:03.596983+0000","last_active":"2023-08-25T09:52:06.980224+0000","last_peered":"2023-08-25T09:52:06.980224+0000","last_clean":"2023-08-25T09:52:06.980224+0000","last_became_active":"2023-08-25T09:52:03.596298+0000","last_became_peered":"2023-08-25T09:52:03.596298+0000","last_unstale":"2023-08-25T09:52:06.980224+0000","last_undegraded":"2023-08-25T09:52:06.980224+0000","last_fullsized":"2023-08-25T09:52:06.980224+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:57.842093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979133+0000","last_change":"2023-08-25T09:52:04.625834+0000","last_active":"2023-08-25T09:52:06.979133+0000","last_peered":"2023-08-25T09:52:06.979133+0000","last_clean":"2023-08-25T09:52:06.979133+0000","last_became_active":"2023-08-25T09:52:04.624790+0000","last_became_peered":"2023-08-25T09:52:04.624790+0000","last_unstale":"2023-08-25T09:52:06.979133+0000","last_undegraded":"2023-08-25T09:52:06.979133+0000","last_fullsized":"2023-08-25T09:52:06.979133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:33.334879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979304+0000","last_change":"2023-08-25T09:52:03.614128+0000","last_active":"2023-08-25T09:52:06.979304+0000","last_peered":"2023-08-25T09:52:06.979304+0000","last_clean":"2023-08-25T09:52:06.979304+0000","last_became_active":"2023-08-25T09:52:03.610416+0000","last_became_peered":"2023-08-25T09:52:03.610416+0000","last_unstale":"2023-08-25T09:52:06.979304+0000","last_undegraded":"2023-08-25T09:52:06.979304+0000","last_fullsized":"2023-08-25T09:52:06.979304+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:00:50.255020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600006+0000","last_change":"2023-08-25T09:52:04.618371+0000","last_active":"2023-08-25T09:52:05.600006+0000","last_peered":"2023-08-25T09:52:05.600006+0000","last_clean":"2023-08-25T09:52:05.600006+0000","last_became_active":"2023-08-25T09:52:04.617925+0000","last_became_peered":"2023-08-25T09:52:04.617925+0000","last_unstale":"2023-08-25T09:52:05.600006+0000","last_undegraded":"2023-08-25T09:52:05.600006+0000","last_fullsized":"2023-08-25T09:52:05.600006+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:32.675064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978949+0000","last_change":"2023-08-25T09:52:03.617559+0000","last_active":"2023-08-25T09:52:06.978949+0000","last_peered":"2023-08-25T09:52:06.978949+0000","last_clean":"2023-08-25T09:52:06.978949+0000","last_became_active":"2023-08-25T09:52:03.616663+0000","last_became_peered":"2023-08-25T09:52:03.616663+0000","last_unstale":"2023-08-25T09:52:06.978949+0000","last_undegraded":"2023-08-25T09:52:06.978949+0000","last_fullsized":"2023-08-25T09:52:06.978949+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:52.014238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978320+0000","last_change":"2023-08-25T09:52:04.618659+0000","last_active":"2023-08-25T09:52:06.978320+0000","last_peered":"2023-08-25T09:52:06.978320+0000","last_clean":"2023-08-25T09:52:06.978320+0000","last_became_active":"2023-08-25T09:52:04.617978+0000","last_became_peered":"2023-08-25T09:52:04.617978+0000","last_unstale":"2023-08-25T09:52:06.978320+0000","last_undegraded":"2023-08-25T09:52:06.978320+0000","last_fullsized":"2023-08-25T09:52:06.978320+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:37.317841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979649+0000","last_change":"2023-08-25T09:52:03.609849+0000","last_active":"2023-08-25T09:52:06.979649+0000","last_peered":"2023-08-25T09:52:06.979649+0000","last_clean":"2023-08-25T09:52:06.979649+0000","last_became_active":"2023-08-25T09:52:03.609198+0000","last_became_peered":"2023-08-25T09:52:03.609198+0000","last_unstale":"2023-08-25T09:52:06.979649+0000","last_undegraded":"2023-08-25T09:52:06.979649+0000","last_fullsized":"2023-08-25T09:52:06.979649+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:14.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979290+0000","last_change":"2023-08-25T09:52:04.617431+0000","last_active":"2023-08-25T09:52:06.979290+0000","last_peered":"2023-08-25T09:52:06.979290+0000","last_clean":"2023-08-25T09:52:06.979290+0000","last_became_active":"2023-08-25T09:52:04.616896+0000","last_became_peered":"2023-08-25T09:52:04.616896+0000","last_unstale":"2023-08-25T09:52:06.979290+0000","last_undegraded":"2023-08-25T09:52:06.979290+0000","last_fullsized":"2023-08-25T09:52:06.979290+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:19.587563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599442+0000","last_change":"2023-08-25T09:52:03.613206+0000","last_active":"2023-08-25T09:52:05.599442+0000","last_peered":"2023-08-25T09:52:05.599442+0000","last_clean":"2023-08-25T09:52:05.599442+0000","last_became_active":"2023-08-25T09:52:03.612871+0000","last_became_peered":"2023-08-25T09:52:03.612871+0000","last_unstale":"2023-08-25T09:52:05.599442+0000","last_undegraded":"2023-08-25T09:52:05.599442+0000","last_fullsized":"2023-08-25T09:52:05.599442+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:13.982356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595647+0000","last_change":"2023-08-25T09:52:04.621653+0000","last_active":"2023-08-25T09:52:05.595647+0000","last_peered":"2023-08-25T09:52:05.595647+0000","last_clean":"2023-08-25T09:52:05.595647+0000","last_became_active":"2023-08-25T09:52:04.621306+0000","last_became_peered":"2023-08-25T09:52:04.621306+0000","last_unstale":"2023-08-25T09:52:05.595647+0000","last_undegraded":"2023-08-25T09:52:05.595647+0000","last_fullsized":"2023-08-25T09:52:05.595647+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:10:13.499449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980116+0000","last_change":"2023-08-25T09:52:03.614440+0000","last_active":"2023-08-25T09:52:06.980116+0000","last_peered":"2023-08-25T09:52:06.980116+0000","last_clean":"2023-08-25T09:52:06.980116+0000","last_became_active":"2023-08-25T09:52:03.613170+0000","last_became_peered":"2023-08-25T09:52:03.613170+0000","last_unstale":"2023-08-25T09:52:06.980116+0000","last_undegraded":"2023-08-25T09:52:06.980116+0000","last_fullsized":"2023-08-25T09:52:06.980116+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:24.408611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595199+0000","last_change":"2023-08-25T09:52:04.604860+0000","last_active":"2023-08-25T09:52:05.595199+0000","last_peered":"2023-08-25T09:52:05.595199+0000","last_clean":"2023-08-25T09:52:05.595199+0000","last_became_active":"2023-08-25T09:52:04.603337+0000","last_became_peered":"2023-08-25T09:52:04.603337+0000","last_unstale":"2023-08-25T09:52:05.595199+0000","last_undegraded":"2023-08-25T09:52:05.595199+0000","last_fullsized":"2023-08-25T09:52:05.595199+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:01.014394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978774+0000","last_change":"2023-08-25T09:52:03.606748+0000","last_active":"2023-08-25T09:52:06.978774+0000","last_peered":"2023-08-25T09:52:06.978774+0000","last_clean":"2023-08-25T09:52:06.978774+0000","last_became_active":"2023-08-25T09:52:03.606384+0000","last_became_peered":"2023-08-25T09:52:03.606384+0000","last_unstale":"2023-08-25T09:52:06.978774+0000","last_undegraded":"2023-08-25T09:52:06.978774+0000","last_fullsized":"2023-08-25T09:52:06.978774+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:55.782470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979062+0000","last_change":"2023-08-25T09:52:04.613626+0000","last_active":"2023-08-25T09:52:06.979062+0000","last_peered":"2023-08-25T09:52:06.979062+0000","last_clean":"2023-08-25T09:52:06.979062+0000","last_became_active":"2023-08-25T09:52:04.613221+0000","last_became_peered":"2023-08-25T09:52:04.613221+0000","last_unstale":"2023-08-25T09:52:06.979062+0000","last_undegraded":"2023-08-25T09:52:06.979062+0000","last_fullsized":"2023-08-25T09:52:06.979062+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:19:08.585858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"198'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979649+0000","last_change":"2023-08-25T09:52:03.598008+0000","last_active":"2023-08-25T09:52:06.979649+0000","last_peered":"2023-08-25T09:52:06.979649+0000","last_clean":"2023-08-25T09:52:06.979649+0000","last_became_active":"2023-08-25T09:52:03.597171+0000","last_became_peered":"2023-08-25T09:52:03.597171+0000","last_unstale":"2023-08-25T09:52:06.979649+0000","last_undegraded":"2023-08-25T09:52:06.979649+0000","last_fullsized":"2023-08-25T09:52:06.979649+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:29.871817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979809+0000","last_change":"2023-08-25T09:52:04.617324+0000","last_active":"2023-08-25T09:52:06.979809+0000","last_peered":"2023-08-25T09:52:06.979809+0000","last_clean":"2023-08-25T09:52:06.979809+0000","last_became_active":"2023-08-25T09:52:04.616867+0000","last_became_peered":"2023-08-25T09:52:04.616867+0000","last_unstale":"2023-08-25T09:52:06.979809+0000","last_undegraded":"2023-08-25T09:52:06.979809+0000","last_fullsized":"2023-08-25T09:52:06.979809+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:13:46.538086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980140+0000","last_change":"2023-08-25T09:52:03.619922+0000","last_active":"2023-08-25T09:52:06.980140+0000","last_peered":"2023-08-25T09:52:06.980140+0000","last_clean":"2023-08-25T09:52:06.980140+0000","last_became_active":"2023-08-25T09:52:03.619288+0000","last_became_peered":"2023-08-25T09:52:03.619288+0000","last_unstale":"2023-08-25T09:52:06.980140+0000","last_undegraded":"2023-08-25T09:52:06.980140+0000","last_fullsized":"2023-08-25T09:52:06.980140+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:47.888788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601489+0000","last_change":"2023-08-25T09:52:04.618818+0000","last_active":"2023-08-25T09:52:05.601489+0000","last_peered":"2023-08-25T09:52:05.601489+0000","last_clean":"2023-08-25T09:52:05.601489+0000","last_became_active":"2023-08-25T09:52:04.615160+0000","last_became_peered":"2023-08-25T09:52:04.615160+0000","last_unstale":"2023-08-25T09:52:05.601489+0000","last_undegraded":"2023-08-25T09:52:05.601489+0000","last_fullsized":"2023-08-25T09:52:05.601489+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:46.121852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979396+0000","last_change":"2023-08-25T09:52:03.614922+0000","last_active":"2023-08-25T09:52:06.979396+0000","last_peered":"2023-08-25T09:52:06.979396+0000","last_clean":"2023-08-25T09:52:06.979396+0000","last_became_active":"2023-08-25T09:52:03.614577+0000","last_became_peered":"2023-08-25T09:52:03.614577+0000","last_unstale":"2023-08-25T09:52:06.979396+0000","last_undegraded":"2023-08-25T09:52:06.979396+0000","last_fullsized":"2023-08-25T09:52:06.979396+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:23.368847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594731+0000","last_change":"2023-08-25T09:52:04.622816+0000","last_active":"2023-08-25T09:52:05.594731+0000","last_peered":"2023-08-25T09:52:05.594731+0000","last_clean":"2023-08-25T09:52:05.594731+0000","last_became_active":"2023-08-25T09:52:04.622527+0000","last_became_peered":"2023-08-25T09:52:04.622527+0000","last_unstale":"2023-08-25T09:52:05.594731+0000","last_undegraded":"2023-08-25T09:52:05.594731+0000","last_fullsized":"2023-08-25T09:52:05.594731+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:29.167368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980224+0000","last_change":"2023-08-25T09:52:03.614822+0000","last_active":"2023-08-25T09:52:06.980224+0000","last_peered":"2023-08-25T09:52:06.980224+0000","last_clean":"2023-08-25T09:52:06.980224+0000","last_became_active":"2023-08-25T09:52:03.613922+0000","last_became_peered":"2023-08-25T09:52:03.613922+0000","last_unstale":"2023-08-25T09:52:06.980224+0000","last_undegraded":"2023-08-25T09:52:06.980224+0000","last_fullsized":"2023-08-25T09:52:06.980224+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:10.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595023+0000","last_change":"2023-08-25T09:52:04.605224+0000","last_active":"2023-08-25T09:52:05.595023+0000","last_peered":"2023-08-25T09:52:05.595023+0000","last_clean":"2023-08-25T09:52:05.595023+0000","last_became_active":"2023-08-25T09:52:04.604198+0000","last_became_peered":"2023-08-25T09:52:04.604198+0000","last_unstale":"2023-08-25T09:52:05.595023+0000","last_undegraded":"2023-08-25T09:52:05.595023+0000","last_fullsized":"2023-08-25T09:52:05.595023+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:20.783977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601107+0000","last_change":"2023-08-25T09:52:03.610194+0000","last_active":"2023-08-25T09:52:05.601107+0000","last_peered":"2023-08-25T09:52:05.601107+0000","last_clean":"2023-08-25T09:52:05.601107+0000","last_became_active":"2023-08-25T09:52:03.609009+0000","last_became_peered":"2023-08-25T09:52:03.609009+0000","last_unstale":"2023-08-25T09:52:05.601107+0000","last_undegraded":"2023-08-25T09:52:05.601107+0000","last_fullsized":"2023-08-25T09:52:05.601107+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:31.051627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978645+0000","last_change":"2023-08-25T09:52:04.620996+0000","last_active":"2023-08-25T09:52:06.978645+0000","last_peered":"2023-08-25T09:52:06.978645+0000","last_clean":"2023-08-25T09:52:06.978645+0000","last_became_active":"2023-08-25T09:52:04.620726+0000","last_became_peered":"2023-08-25T09:52:04.620726+0000","last_unstale":"2023-08-25T09:52:06.978645+0000","last_undegraded":"2023-08-25T09:52:06.978645+0000","last_fullsized":"2023-08-25T09:52:06.978645+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:52.575789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979407+0000","last_change":"2023-08-25T09:52:03.608521+0000","last_active":"2023-08-25T09:52:06.979407+0000","last_peered":"2023-08-25T09:52:06.979407+0000","last_clean":"2023-08-25T09:52:06.979407+0000","last_became_active":"2023-08-25T09:52:03.605881+0000","last_became_peered":"2023-08-25T09:52:03.605881+0000","last_unstale":"2023-08-25T09:52:06.979407+0000","last_undegraded":"2023-08-25T09:52:06.979407+0000","last_fullsized":"2023-08-25T09:52:06.979407+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:12:17.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598820+0000","last_change":"2023-08-25T09:52:04.615664+0000","last_active":"2023-08-25T09:52:05.598820+0000","last_peered":"2023-08-25T09:52:05.598820+0000","last_clean":"2023-08-25T09:52:05.598820+0000","last_became_active":"2023-08-25T09:52:04.613567+0000","last_became_peered":"2023-08-25T09:52:04.613567+0000","last_unstale":"2023-08-25T09:52:05.598820+0000","last_undegraded":"2023-08-25T09:52:05.598820+0000","last_fullsized":"2023-08-25T09:52:05.598820+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:33.941978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600289+0000","last_change":"2023-08-25T09:52:03.614418+0000","last_active":"2023-08-25T09:52:05.600289+0000","last_peered":"2023-08-25T09:52:05.600289+0000","last_clean":"2023-08-25T09:52:05.600289+0000","last_became_active":"2023-08-25T09:52:03.614127+0000","last_became_peered":"2023-08-25T09:52:03.614127+0000","last_unstale":"2023-08-25T09:52:05.600289+0000","last_undegraded":"2023-08-25T09:52:05.600289+0000","last_fullsized":"2023-08-25T09:52:05.600289+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:31.211948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977137+0000","last_change":"2023-08-25T09:52:04.629117+0000","last_active":"2023-08-25T09:52:06.977137+0000","last_peered":"2023-08-25T09:52:06.977137+0000","last_clean":"2023-08-25T09:52:06.977137+0000","last_became_active":"2023-08-25T09:52:04.626222+0000","last_became_peered":"2023-08-25T09:52:04.626222+0000","last_unstale":"2023-08-25T09:52:06.977137+0000","last_undegraded":"2023-08-25T09:52:06.977137+0000","last_fullsized":"2023-08-25T09:52:06.977137+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:59:10.664615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598694+0000","last_change":"2023-08-25T09:52:03.619038+0000","last_active":"2023-08-25T09:52:05.598694+0000","last_peered":"2023-08-25T09:52:05.598694+0000","last_clean":"2023-08-25T09:52:05.598694+0000","last_became_active":"2023-08-25T09:52:03.618767+0000","last_became_peered":"2023-08-25T09:52:03.618767+0000","last_unstale":"2023-08-25T09:52:05.598694+0000","last_undegraded":"2023-08-25T09:52:05.598694+0000","last_fullsized":"2023-08-25T09:52:05.598694+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:33.870463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":479,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977545+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:52:06.977545+0000","last_peered":"2023-08-25T09:52:06.977545+0000","last_clean":"2023-08-25T09:52:06.977545+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:52:06.977545+0000","last_undegraded":"2023-08-25T09:52:06.977545+0000","last_fullsized":"2023-08-25T09:52:06.977545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:46.367732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595603+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:52:05.595603+0000","last_peered":"2023-08-25T09:52:05.595603+0000","last_clean":"2023-08-25T09:52:05.595603+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:52:05.595603+0000","last_undegraded":"2023-08-25T09:52:05.595603+0000","last_fullsized":"2023-08-25T09:52:05.595603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:03:12.303284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600876+0000","last_change":"2023-08-25T09:52:04.619104+0000","last_active":"2023-08-25T09:52:05.600876+0000","last_peered":"2023-08-25T09:52:05.600876+0000","last_clean":"2023-08-25T09:52:05.600876+0000","last_became_active":"2023-08-25T09:52:04.615804+0000","last_became_peered":"2023-08-25T09:52:04.615804+0000","last_unstale":"2023-08-25T09:52:05.600876+0000","last_undegraded":"2023-08-25T09:52:05.600876+0000","last_fullsized":"2023-08-25T09:52:05.600876+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:30.881061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"197'3","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979257+0000","last_change":"2023-08-25T09:52:03.608811+0000","last_active":"2023-08-25T09:52:06.979257+0000","last_peered":"2023-08-25T09:52:06.979257+0000","last_clean":"2023-08-25T09:52:06.979257+0000","last_became_active":"2023-08-25T09:52:03.608140+0000","last_became_peered":"2023-08-25T09:52:03.608140+0000","last_unstale":"2023-08-25T09:52:06.979257+0000","last_undegraded":"2023-08-25T09:52:06.979257+0000","last_fullsized":"2023-08-25T09:52:06.979257+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:48:39.537165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595573+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:52:05.595573+0000","last_peered":"2023-08-25T09:52:05.595573+0000","last_clean":"2023-08-25T09:52:05.595573+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:52:05.595573+0000","last_undegraded":"2023-08-25T09:52:05.595573+0000","last_fullsized":"2023-08-25T09:52:05.595573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:43:58.928366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600124+0000","last_change":"2023-08-25T09:52:04.616692+0000","last_active":"2023-08-25T09:52:05.600124+0000","last_peered":"2023-08-25T09:52:05.600124+0000","last_clean":"2023-08-25T09:52:05.600124+0000","last_became_active":"2023-08-25T09:52:04.610333+0000","last_became_peered":"2023-08-25T09:52:04.610333+0000","last_unstale":"2023-08-25T09:52:05.600124+0000","last_undegraded":"2023-08-25T09:52:05.600124+0000","last_fullsized":"2023-08-25T09:52:05.600124+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:21.268563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"197'6","reported_seq":19,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600016+0000","last_change":"2023-08-25T09:52:03.615496+0000","last_active":"2023-08-25T09:52:05.600016+0000","last_peered":"2023-08-25T09:52:05.600016+0000","last_clean":"2023-08-25T09:52:05.600016+0000","last_became_active":"2023-08-25T09:52:03.615224+0000","last_became_peered":"2023-08-25T09:52:03.615224+0000","last_unstale":"2023-08-25T09:52:05.600016+0000","last_undegraded":"2023-08-25T09:52:05.600016+0000","last_fullsized":"2023-08-25T09:52:05.600016+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:12:41.768761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":359,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.602617+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:52:05.602617+0000","last_peered":"2023-08-25T09:52:05.602617+0000","last_clean":"2023-08-25T09:52:05.602617+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:52:05.602617+0000","last_undegraded":"2023-08-25T09:52:05.602617+0000","last_fullsized":"2023-08-25T09:52:05.602617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:48.772597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979915+0000","last_change":"2023-08-25T09:52:04.609947+0000","last_active":"2023-08-25T09:52:06.979915+0000","last_peered":"2023-08-25T09:52:06.979915+0000","last_clean":"2023-08-25T09:52:06.979915+0000","last_became_active":"2023-08-25T09:52:04.609100+0000","last_became_peered":"2023-08-25T09:52:04.609100+0000","last_unstale":"2023-08-25T09:52:06.979915+0000","last_undegraded":"2023-08-25T09:52:06.979915+0000","last_fullsized":"2023-08-25T09:52:06.979915+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:45.487493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598743+0000","last_change":"2023-08-25T09:52:03.615906+0000","last_active":"2023-08-25T09:52:05.598743+0000","last_peered":"2023-08-25T09:52:05.598743+0000","last_clean":"2023-08-25T09:52:05.598743+0000","last_became_active":"2023-08-25T09:52:03.615475+0000","last_became_peered":"2023-08-25T09:52:03.615475+0000","last_unstale":"2023-08-25T09:52:05.598743+0000","last_undegraded":"2023-08-25T09:52:05.598743+0000","last_fullsized":"2023-08-25T09:52:05.598743+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:57.343819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.596068+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:52:05.596068+0000","last_peered":"2023-08-25T09:52:05.596068+0000","last_clean":"2023-08-25T09:52:05.596068+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:52:05.596068+0000","last_undegraded":"2023-08-25T09:52:05.596068+0000","last_fullsized":"2023-08-25T09:52:05.596068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:33:58.167750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976617+0000","last_change":"2023-08-25T09:52:04.626470+0000","last_active":"2023-08-25T09:52:06.976617+0000","last_peered":"2023-08-25T09:52:06.976617+0000","last_clean":"2023-08-25T09:52:06.976617+0000","last_became_active":"2023-08-25T09:52:04.625407+0000","last_became_peered":"2023-08-25T09:52:04.625407+0000","last_unstale":"2023-08-25T09:52:06.976617+0000","last_undegraded":"2023-08-25T09:52:06.976617+0000","last_fullsized":"2023-08-25T09:52:06.976617+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:27:19.683074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977016+0000","last_change":"2023-08-25T09:52:03.613108+0000","last_active":"2023-08-25T09:52:06.977016+0000","last_peered":"2023-08-25T09:52:06.977016+0000","last_clean":"2023-08-25T09:52:06.977016+0000","last_became_active":"2023-08-25T09:52:03.612787+0000","last_became_peered":"2023-08-25T09:52:03.612787+0000","last_unstale":"2023-08-25T09:52:06.977016+0000","last_undegraded":"2023-08-25T09:52:06.977016+0000","last_fullsized":"2023-08-25T09:52:06.977016+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:45.967657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980300+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:52:06.980300+0000","last_peered":"2023-08-25T09:52:06.980300+0000","last_clean":"2023-08-25T09:52:06.980300+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:52:06.980300+0000","last_undegraded":"2023-08-25T09:52:06.980300+0000","last_fullsized":"2023-08-25T09:52:06.980300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:10.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978259+0000","last_change":"2023-08-25T09:52:04.625453+0000","last_active":"2023-08-25T09:52:06.978259+0000","last_peered":"2023-08-25T09:52:06.978259+0000","last_clean":"2023-08-25T09:52:06.978259+0000","last_became_active":"2023-08-25T09:52:04.625181+0000","last_became_peered":"2023-08-25T09:52:04.625181+0000","last_unstale":"2023-08-25T09:52:06.978259+0000","last_undegraded":"2023-08-25T09:52:06.978259+0000","last_fullsized":"2023-08-25T09:52:06.978259+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:21.268563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"197'8","reported_seq":21,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.602337+0000","last_change":"2023-08-25T09:52:03.615255+0000","last_active":"2023-08-25T09:52:05.602337+0000","last_peered":"2023-08-25T09:52:05.602337+0000","last_clean":"2023-08-25T09:52:05.602337+0000","last_became_active":"2023-08-25T09:52:03.613680+0000","last_became_peered":"2023-08-25T09:52:03.613680+0000","last_unstale":"2023-08-25T09:52:05.602337+0000","last_undegraded":"2023-08-25T09:52:05.602337+0000","last_fullsized":"2023-08-25T09:52:05.602337+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:05:27.088191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595232+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:52:05.595232+0000","last_peered":"2023-08-25T09:52:05.595232+0000","last_clean":"2023-08-25T09:52:05.595232+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:52:05.595232+0000","last_undegraded":"2023-08-25T09:52:05.595232+0000","last_fullsized":"2023-08-25T09:52:05.595232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:32.237945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977164+0000","last_change":"2023-08-25T09:52:04.628438+0000","last_active":"2023-08-25T09:52:06.977164+0000","last_peered":"2023-08-25T09:52:06.977164+0000","last_clean":"2023-08-25T09:52:06.977164+0000","last_became_active":"2023-08-25T09:52:04.627892+0000","last_became_peered":"2023-08-25T09:52:04.627892+0000","last_unstale":"2023-08-25T09:52:06.977164+0000","last_undegraded":"2023-08-25T09:52:06.977164+0000","last_fullsized":"2023-08-25T09:52:06.977164+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:45.280799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980132+0000","last_change":"2023-08-25T09:52:03.608367+0000","last_active":"2023-08-25T09:52:06.980132+0000","last_peered":"2023-08-25T09:52:06.980132+0000","last_clean":"2023-08-25T09:52:06.980132+0000","last_became_active":"2023-08-25T09:52:03.605716+0000","last_became_peered":"2023-08-25T09:52:03.605716+0000","last_unstale":"2023-08-25T09:52:06.980132+0000","last_undegraded":"2023-08-25T09:52:06.980132+0000","last_fullsized":"2023-08-25T09:52:06.980132+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:45.112608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978871+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:52:06.978871+0000","last_peered":"2023-08-25T09:52:06.978871+0000","last_clean":"2023-08-25T09:52:06.978871+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:52:06.978871+0000","last_undegraded":"2023-08-25T09:52:06.978871+0000","last_fullsized":"2023-08-25T09:52:06.978871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:05.193960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599626+0000","last_change":"2023-08-25T09:52:04.616992+0000","last_active":"2023-08-25T09:52:05.599626+0000","last_peered":"2023-08-25T09:52:05.599626+0000","last_clean":"2023-08-25T09:52:05.599626+0000","last_became_active":"2023-08-25T09:52:04.613623+0000","last_became_peered":"2023-08-25T09:52:04.613623+0000","last_unstale":"2023-08-25T09:52:05.599626+0000","last_undegraded":"2023-08-25T09:52:05.599626+0000","last_fullsized":"2023-08-25T09:52:05.599626+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:27.441433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599106+0000","last_change":"2023-08-25T09:52:03.611964+0000","last_active":"2023-08-25T09:52:05.599106+0000","last_peered":"2023-08-25T09:52:05.599106+0000","last_clean":"2023-08-25T09:52:05.599106+0000","last_became_active":"2023-08-25T09:52:03.611724+0000","last_became_peered":"2023-08-25T09:52:03.611724+0000","last_unstale":"2023-08-25T09:52:05.599106+0000","last_undegraded":"2023-08-25T09:52:05.599106+0000","last_fullsized":"2023-08-25T09:52:05.599106+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:31.754265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595886+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:52:05.595886+0000","last_peered":"2023-08-25T09:52:05.595886+0000","last_clean":"2023-08-25T09:52:05.595886+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:52:05.595886+0000","last_undegraded":"2023-08-25T09:52:05.595886+0000","last_fullsized":"2023-08-25T09:52:05.595886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:56.976887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198806588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.35699999999999998},"max":{"1min":1.4930000000000001,"5min":1.829,"15min":10.747},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.173,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.43099999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.855},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.6970000000000001,"5min":1.823,"15min":11.010999999999999},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.2589999999999999,"15min":1.1819999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.496,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.206,"15min":1.147},"min":{"1min":0.86699999999999999,"5min":0.79400000000000004,"15min":0.50600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5529999999999999,"15min":1.619},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3180000000000001,"15min":1.2769999999999999},"min":{"1min":0.93999999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6779999999999999,"5min":1.873,"15min":1.98},"last":1.6379999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.258,"15min":1.2190000000000001},"min":{"1min":0.95599999999999996,"5min":0.79300000000000004,"15min":0.56799999999999995},"max":{"1min":1.506,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.224},{"interface":"front","average":{"1min":1.359,"5min":1.393,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.69299999999999995},"max":{"1min":1.778,"5min":1.833,"15min":2.1030000000000002},"last":1.5620000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.264,"15min":1.288},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.48599999999999999},"max":{"1min":1.732,"5min":1.845,"15min":7.1639999999999997},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.306,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.46300000000000002},"max":{"1min":1.7190000000000001,"5min":1.798,"15min":1.8779999999999999},"last":0.999}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.52700000000000002},"max":{"1min":1.714,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.331},{"interface":"front","average":{"1min":1.361,"5min":1.4450000000000001,"15min":1.429},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.64100000000000001},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":2.0739999999999998},"last":1.04}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5580000000000001,"15min":1.518},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.68799999999999994},"max":{"1min":1.79,"5min":1.984,"15min":2.1720000000000002},"last":1.49},{"interface":"front","average":{"1min":1.448,"5min":1.4870000000000001,"15min":1.462},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.5189999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198806588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.161,"15min":1.1459999999999999},"min":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.54400000000000004},"max":{"1min":1.7310000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.25,"5min":1.198,"15min":1.175},"min":{"1min":0.84199999999999997,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.627,"5min":1.645,"15min":5.5410000000000004},"last":0.997}]},{"osd":1,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2529999999999999,"15min":1.2210000000000001},"min":{"1min":0.82799999999999996,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6930000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.194,"15min":1.179},"min":{"1min":0.75800000000000001,"5min":0.66800000000000004,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.909,"15min":2.3079999999999998},"last":1.1259999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.153},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.708},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.67000000000000004,"15min":0.47299999999999998},"max":{"1min":1.7310000000000001,"5min":1.7969999999999999,"15min":13.193},"last":1.458}]},{"osd":3,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":1.0620000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.861,"5min":1.911,"15min":5.6239999999999997},"last":1.861},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.286,"15min":1.27},"min":{"1min":0.995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8200000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5880000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.325,"15min":1.3169999999999999},"min":{"1min":1.048,"5min":0.55300000000000005,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":1.867,"15min":2.484},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.405,"15min":1.381},"min":{"1min":0.99199999999999999,"5min":0.70099999999999996,"15min":0.58799999999999997},"max":{"1min":1.9650000000000001,"5min":2.101,"15min":5.4699999999999998},"last":1.9650000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.3260000000000001},"min":{"1min":0.95099999999999996,"5min":0.74199999999999999,"15min":0.59599999999999997},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.802},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.468,"15min":1.454},"min":{"1min":1,"5min":0.61099999999999999,"15min":0.59099999999999997},"max":{"1min":1.786,"5min":1.994,"15min":2.4420000000000002},"last":1.7509999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":1.179,"5min":0.80000000000000004,"15min":0.622},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":6.7709999999999999},"last":2.0329999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5169999999999999,"15min":1.498},"min":{"1min":1.105,"5min":0.85899999999999999,"15min":0.57399999999999995},"max":{"1min":2.0590000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.921}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193571,"statfs":{"total":200000000,"available":198217146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2669999999999999,"15min":1.2050000000000001},"min":{"1min":0.751,"5min":0.71999999999999997,"15min":0.497},"max":{"1min":1.7210000000000001,"5min":1.9770000000000001,"15min":2.081},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.252,"15min":1.2210000000000001},"min":{"1min":0.88800000000000001,"5min":0.747,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.8080000000000001,"15min":1.974},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1639999999999999,"15min":1.165},"min":{"1min":0.69799999999999995,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.6399999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.208,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.571,"5min":1.704,"15min":2.1869999999999998},"last":1.4990000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3200000000000001,"15min":1.252},"min":{"1min":0.91700000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8149999999999999,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.331},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.252},"min":{"1min":0.83099999999999996,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.5449999999999999,"5min":1.847,"15min":1.962},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2589999999999999,"15min":1.288},"min":{"1min":0.86299999999999999,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.71,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.121},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3500000000000001,"15min":1.325},"min":{"1min":0.95299999999999996,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.7330000000000001,"5min":1.964,"15min":1.9650000000000001},"last":1.569}]},{"osd":4,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.306,"15min":1.3149999999999999},"min":{"1min":0.83199999999999996,"5min":0.80800000000000005,"15min":0.55500000000000005},"max":{"1min":1.76,"5min":1.9059999999999999,"15min":2.625},"last":1.637},{"interface":"front","average":{"1min":1.304,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.58799999999999997},"max":{"1min":1.7150000000000001,"5min":1.913,"15min":2.2789999999999999},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.462,"15min":1.444},"min":{"1min":1.0569999999999999,"5min":0.95099999999999996,"15min":0.58899999999999997},"max":{"1min":1.76,"5min":2.028,"15min":2.4820000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.5249999999999999,"15min":1.486},"min":{"1min":1.0840000000000001,"5min":0.96899999999999997,"15min":0.60399999999999998},"max":{"1min":1.9039999999999999,"5min":2.044,"15min":2.5539999999999998},"last":1.7549999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4950000000000001,"15min":1.464},"min":{"1min":1.0129999999999999,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.585,"15min":1.579},"min":{"1min":1.2869999999999999,"5min":1.0289999999999999,"15min":0.63900000000000001},"max":{"1min":1.923,"5min":2.1040000000000001,"15min":2.3700000000000001},"last":1.7929999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.28,"15min":1.224},"min":{"1min":0.624,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.589,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.2709999999999999,"15min":1.2490000000000001},"min":{"1min":0.91600000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.615,"5min":2.5150000000000001,"15min":7.367},"last":1.3859999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.466,"5min":1.7330000000000001,"15min":2.032},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.234,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.571,"5min":4.351,"15min":8.5850000000000009},"last":1.3}]},{"osd":2,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.214},"min":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.556,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.294,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5940000000000001,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.04}]},{"osd":3,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.284,"15min":1.2629999999999999},"min":{"1min":0.79400000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.665,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.911},"last":1.4810000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3140000000000001,"15min":1.3009999999999999},"min":{"1min":0.67100000000000004,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.7370000000000001,"5min":5.048,"15min":5.048},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.1419999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4179999999999999,"15min":1.4319999999999999},"min":{"1min":0.88100000000000001,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":1.927,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.597,"15min":1.579},"min":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.968,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.496,"15min":1.49},"min":{"1min":0.75700000000000001,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.865,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.569,"15min":1.5920000000000001},"min":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8340000000000001,"5min":3.831,"15min":3.831},"last":1.097}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98599999999999999,"15min":0.91000000000000003},"min":{"1min":0.628,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.8180000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0649999999999999,"15min":0.98199999999999998},"min":{"1min":0.69999999999999996,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.579,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.94299999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.64600000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.258,"5min":1.413,"15min":1.702},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.087,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.623,"5min":1.6339999999999999,"15min":2.3399999999999999},"last":1.27}]},{"osd":2,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.034,"15min":0.96499999999999997},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.5980000000000001,"5min":1.623,"15min":1.919},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2050000000000001,"15min":1.1439999999999999},"min":{"1min":0.745,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.901,"5min":1.901,"15min":3.399},"last":1.5249999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3009999999999999,"15min":1.236},"min":{"1min":1.1399999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.395},{"interface":"front","average":{"1min":1.198,"5min":1.204,"15min":1.2090000000000001},"min":{"1min":1.0109999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.405,"5min":1.79,"15min":2.1840000000000002},"last":1.6539999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3480000000000001},"min":{"1min":1.137,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":2.1509999999999998,"5min":2.1930000000000001,"15min":2.407},"last":1.764},{"interface":"front","average":{"1min":1.357,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.0349999999999999,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.804,"5min":1.895,"15min":1.895},"last":1.806}]},{"osd":6,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.302,"15min":1.282},"min":{"1min":1.054,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5980000000000001,"5min":1.875,"15min":2.3450000000000002},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.47,"15min":1.4199999999999999},"min":{"1min":1.28,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":2.2330000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.9830000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.464,"15min":1.405},"min":{"1min":1.0840000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.976,"5min":1.976,"15min":2.2989999999999999},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.3839999999999999},"min":{"1min":1.1259999999999999,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":1.7589999999999999,"5min":1.9350000000000001,"15min":2.1509999999999998},"last":2.0139999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198807436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.64900000000000002,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.51,"5min":1.6930000000000001,"15min":2.383},"last":0.753},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93300000000000005,"15min":0.96299999999999997},"min":{"1min":0.47599999999999998,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":2.7309999999999999},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.97599999999999998,"15min":0.98499999999999999},"min":{"1min":0.66400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3009999999999999,"5min":1.635,"15min":2.3180000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0429999999999999,"15min":1.0389999999999999},"min":{"1min":0.56699999999999995,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.5800000000000001,"5min":1.704,"15min":2.8599999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":1.0429999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.294,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.121,"15min":1.101},"min":{"1min":0.84699999999999998,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.587,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.99199999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3080000000000001,"15min":1.321},"min":{"1min":0.94099999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":1.194},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":0.91200000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6459999999999999,"5min":1.8,"15min":2.7639999999999998},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2470000000000001,"15min":1.2809999999999999},"min":{"1min":0.89000000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.534,"5min":1.849,"15min":5.1269999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.1020000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.605,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":1.2649999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.397,"15min":1.403},"min":{"1min":1.1060000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.393,"15min":1.417},"min":{"1min":1.0109999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.617,"5min":2.028,"15min":5.0599999999999996},"last":1.0800000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.373,"15min":1.395},"min":{"1min":1.1699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.7210000000000001,"5min":2,"15min":2.968},"last":0.872},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3779999999999999,"15min":1.4239999999999999},"min":{"1min":1.071,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.569,"5min":1.96,"15min":5.2990000000000004},"last":0.96699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1740,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193572,"statfs":{"total":200000000,"available":198217742,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.82699999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.29699999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97899999999999998,"15min":0.89800000000000002},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.7869999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.88100000000000001},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.35899999999999999},"max":{"1min":1.2809999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.022,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.389,"5min":1.6619999999999999,"15min":1.746},"last":1.139}]},{"osd":3,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.109},"min":{"1min":0.496,"5min":0.496,"15min":0.372},"max":{"1min":1.647,"5min":1.647,"15min":1.9450000000000001},"last":1.278},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0109999999999999},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.333,"5min":1.655,"15min":1.764},"last":1.381}]},{"osd":4,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.2929999999999999,"15min":1.276},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":1.6830000000000001,"5min":1.9419999999999999,"15min":2.008},"last":1.351}]},{"osd":5,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.2430000000000001},"min":{"1min":0.94699999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.665,"15min":1.9650000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3049999999999999,"15min":1.2450000000000001},"min":{"1min":0.82699999999999996,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.677,"5min":2.867,"15min":2.867},"last":1.647}]},{"osd":6,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.363},"min":{"1min":1.016,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":1.633,"5min":2.0779999999999998,"15min":2.9940000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.411,"15min":1.3660000000000001},"min":{"1min":0.95099999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.603,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.5629999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4490000000000001,"15min":1.4139999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.79200000000000004},"max":{"1min":1.772,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.431,"15min":1.399},"min":{"1min":0.88200000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.738,"5min":1.913,"15min":2.218},"last":1.4950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808200,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.313},"max":{"1min":1.4450000000000001,"5min":2.2759999999999998,"15min":11.257},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.98499999999999999,"15min":0.96799999999999997},"min":{"1min":0.60199999999999998,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.4730000000000001,"5min":1.762,"15min":6.4260000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.96099999999999997},"min":{"1min":0.72399999999999998,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.343,"5min":1.522,"15min":1.8620000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":1.048},"min":{"1min":0.62,"5min":0.374,"15min":0.374},"max":{"1min":1.401,"5min":1.714,"15min":3.4889999999999999},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.98499999999999999,"15min":0.93000000000000005},"min":{"1min":0.65900000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.464,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.69199999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.629,"15min":2.3809999999999998},"last":1.4690000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.294,"15min":1.2569999999999999},"min":{"1min":0.89800000000000002,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.724,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.097,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.379}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.88800000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.919,"5min":1.919,"15min":3.3769999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.2769999999999999},"min":{"1min":1.038,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6439999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.573}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4219999999999999,"15min":1.3640000000000001},"min":{"1min":0.92400000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7370000000000001,"5min":2.056,"15min":2.6869999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.3759999999999999,"15min":1.335},"min":{"1min":0.999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7709999999999999,"5min":1.863,"15min":2.331},"last":1.508}]},{"osd":7,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.706,"5min":1.962,"15min":2.6429999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.389,"15min":1.3440000000000001},"min":{"1min":0.95699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.804,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.423}]}]}],"pool_statfs":[]}} 2023-08-25T09:52:14.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:52:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.324+0000 7f81f6d12640 1 Processor -- start 2023-08-25T09:52:14.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.325+0000 7f81f6d12640 1 -- start start 2023-08-25T09:52:14.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.326+0000 7f81f6d12640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81f00cea30 0x7f81f00b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.326+0000 7f81f6d12640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f006f050 con 0x7f81f00ac870 2023-08-25T09:52:14.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.326+0000 7f81f6d12640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f006f330 con 0x7f81f015cb40 2023-08-25T09:52:14.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.326+0000 7f81f6d12640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f006f610 con 0x7f81f00cea30 2023-08-25T09:52:14.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81effff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f81f00cea30 0x7f81f00b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81f52af640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f81f00ac870 0x7f81f00b2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45800/0 (socket says 172.21.15.70:45800) 2023-08-25T09:52:14.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81f52af640 1 -- 172.21.15.70:0/3425265998 learned_addr learned my addr 172.21.15.70:0/3425265998 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:14.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81effff640 1 -- 172.21.15.70:0/3425265998 >> v1:172.21.15.155:6789/0 conn(0x7f81f015cb40 legacy=0x7f81f00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81effff640 1 -- 172.21.15.70:0/3425265998 >> v1:172.21.15.70:6789/0 conn(0x7f81f00ac870 legacy=0x7f81f00b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81effff640 1 -- 172.21.15.70:0/3425265998 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f006f8f0 con 0x7f81f00cea30 2023-08-25T09:52:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.327+0000 7f81ef7fe640 1 -- 172.21.15.70:0/3425265998 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739634628 0 0) 0x7f81f006f050 con 0x7f81f00ac870 2023-08-25T09:52:14.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.328+0000 7f81effff640 1 --2- 172.21.15.70:0/3425265998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00b1b30 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0097a0 tx=0x7f81dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f27357ff1cacc2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81ef7fe640 1 -- 172.21.15.70:0/3425265998 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc031830 con 0x7f81f00cea30 2023-08-25T09:52:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81ef7fe640 1 -- 172.21.15.70:0/3425265998 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc031990 con 0x7f81f00cea30 2023-08-25T09:52:14.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81ef7fe640 1 -- 172.21.15.70:0/3425265998 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc031d20 con 0x7f81f00cea30 2023-08-25T09:52:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81f6d12640 1 -- 172.21.15.70:0/3425265998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 msgr2=0x7f81f00b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3425265998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00b1b30 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0097a0 tx=0x7f81dc02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81f6d12640 1 -- 172.21.15.70:0/3425265998 shutdown_connections 2023-08-25T09:52:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3425265998 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00b1b30 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.329+0000 7f81f6d12640 1 -- 172.21.15.70:0/3425265998 >> 172.21.15.70:0/3425265998 conn(0x7f81f0069850 msgr2=0x7f81f0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:14.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.330+0000 7f81f6d12640 1 -- 172.21.15.70:0/3425265998 shutdown_connections 2023-08-25T09:52:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.330+0000 7f81f6d12640 1 -- 172.21.15.70:0/3425265998 wait complete. 2023-08-25T09:52:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.331+0000 7f81f6d12640 1 Processor -- start 2023-08-25T09:52:14.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.332+0000 7f81f6d12640 1 -- start start 2023-08-25T09:52:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.332+0000 7f81f6d12640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ac870 0x7f81f014b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81f6d12640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81f6d12640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81f015cb40 0x7f81f00cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81f6d12640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00cb9a0 con 0x7f81f00cea30 2023-08-25T09:52:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81f6d12640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00cae30 con 0x7f81f00ac870 2023-08-25T09:52:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81f6d12640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00cb100 con 0x7f81f015cb40 2023-08-25T09:52:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43126/0 (socket says 172.21.15.70:43126) 2023-08-25T09:52:14.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 -- 172.21.15.70:0/3770264192 learned_addr learned my addr 172.21.15.70:0/3770264192 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:14.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 -- 172.21.15.70:0/3770264192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81f015cb40 msgr2=0x7f81f00cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81f015cb40 0x7f81f00cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 -- 172.21.15.70:0/3770264192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ac870 msgr2=0x7f81f014b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ac870 0x7f81f014b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.333+0000 7f81effff640 1 -- 172.21.15.70:0/3770264192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc009400 con 0x7f81f00cea30 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.334+0000 7f81effff640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00ca8f0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0098c0 tx=0x7f81dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.334+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc03ba70 con 0x7f81f00cea30 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.334+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc03bbd0 con 0x7f81f00cea30 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.334+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0453f0 con 0x7f81f00cea30 2023-08-25T09:52:14.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.334+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f00cb3e0 con 0x7f81f00cea30 2023-08-25T09:52:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.334+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f0157280 con 0x7f81f00cea30 2023-08-25T09:52:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.335+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f01570e0 con 0x7f81f00cea30 2023-08-25T09:52:14.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.336+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f81dc04e450 con 0x7f81f00cea30 2023-08-25T09:52:14.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.337+0000 7f81ed7fa640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f81d406a050 0x7f81d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.337+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f81dc0aa870 con 0x7f81f00cea30 2023-08-25T09:52:14.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.337+0000 7f81f4aae640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f81d406a050 0x7f81d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.338+0000 7f81f4aae640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f81d406a050 0x7f81d406c510 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:14.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.341+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f81f01570e0 con 0x7f81f00cea30 2023-08-25T09:52:14.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.529+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81f006ece0 con 0x7f81d406a050 2023-08-25T09:52:14.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.539+0000 7f81ed7fa640 1 -- 172.21.15.70:0/3770264192 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+302974 (secure 0 0 0) 0x7f81e4013430 con 0x7f81d406a050 2023-08-25T09:52:14.541 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:14.546 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:52:14.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f81d406a050 msgr2=0x7f81d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f81d406a050 0x7f81d406c510 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f81e4006fd0 tx=0x7f81e4006bc0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 msgr2=0x7f81f00ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00ca8f0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0098c0 tx=0x7f81dc0046c0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 shutdown_connections 2023-08-25T09:52:14.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f81d406a050 0x7f81d406c510 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f81f015cb40 0x7f81f00cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f81f00cea30 0x7f81f00ca8f0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 --2- 172.21.15.70:0/3770264192 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f81f00ac870 0x7f81f014b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 >> 172.21.15.70:0/3770264192 conn(0x7f81f0069850 msgr2=0x7f81f00acc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 shutdown_connections 2023-08-25T09:52:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.544+0000 7f81f6d12640 1 -- 172.21.15.70:0/3770264192 wait complete. 2023-08-25T09:52:14.565 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":597,"stamp":"2023-08-25T09:52:13.594411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10468,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552031,"statfs":{"total":1600000000,"available":1589280044,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.003955"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978214+0000","last_change":"2023-08-25T09:52:04.628641+0000","last_active":"2023-08-25T09:52:06.978214+0000","last_peered":"2023-08-25T09:52:06.978214+0000","last_clean":"2023-08-25T09:52:06.978214+0000","last_became_active":"2023-08-25T09:52:04.627891+0000","last_became_peered":"2023-08-25T09:52:04.627891+0000","last_unstale":"2023-08-25T09:52:06.978214+0000","last_undegraded":"2023-08-25T09:52:06.978214+0000","last_fullsized":"2023-08-25T09:52:06.978214+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:41:47.154534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979875+0000","last_change":"2023-08-25T09:52:04.625251+0000","last_active":"2023-08-25T09:52:06.979875+0000","last_peered":"2023-08-25T09:52:06.979875+0000","last_clean":"2023-08-25T09:52:06.979875+0000","last_became_active":"2023-08-25T09:52:04.624800+0000","last_became_peered":"2023-08-25T09:52:04.624800+0000","last_unstale":"2023-08-25T09:52:06.979875+0000","last_undegraded":"2023-08-25T09:52:06.979875+0000","last_fullsized":"2023-08-25T09:52:06.979875+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:41:47.867907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978509+0000","last_change":"2023-08-25T09:52:04.617930+0000","last_active":"2023-08-25T09:52:06.978509+0000","last_peered":"2023-08-25T09:52:06.978509+0000","last_clean":"2023-08-25T09:52:06.978509+0000","last_became_active":"2023-08-25T09:52:04.614336+0000","last_became_peered":"2023-08-25T09:52:04.614336+0000","last_unstale":"2023-08-25T09:52:06.978509+0000","last_undegraded":"2023-08-25T09:52:06.978509+0000","last_fullsized":"2023-08-25T09:52:06.978509+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:05:06.201084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977490+0000","last_change":"2023-08-25T09:52:04.627289+0000","last_active":"2023-08-25T09:52:06.977490+0000","last_peered":"2023-08-25T09:52:06.977490+0000","last_clean":"2023-08-25T09:52:06.977490+0000","last_became_active":"2023-08-25T09:52:04.625811+0000","last_became_peered":"2023-08-25T09:52:04.625811+0000","last_unstale":"2023-08-25T09:52:06.977490+0000","last_undegraded":"2023-08-25T09:52:06.977490+0000","last_fullsized":"2023-08-25T09:52:06.977490+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:21:46.850988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598128+0000","last_change":"2023-08-25T09:52:04.617186+0000","last_active":"2023-08-25T09:52:05.598128+0000","last_peered":"2023-08-25T09:52:05.598128+0000","last_clean":"2023-08-25T09:52:05.598128+0000","last_became_active":"2023-08-25T09:52:04.616036+0000","last_became_peered":"2023-08-25T09:52:04.616036+0000","last_unstale":"2023-08-25T09:52:05.598128+0000","last_undegraded":"2023-08-25T09:52:05.598128+0000","last_fullsized":"2023-08-25T09:52:05.598128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:27.953431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978450+0000","last_change":"2023-08-25T09:52:04.628367+0000","last_active":"2023-08-25T09:52:06.978450+0000","last_peered":"2023-08-25T09:52:06.978450+0000","last_clean":"2023-08-25T09:52:06.978450+0000","last_became_active":"2023-08-25T09:52:04.626874+0000","last_became_peered":"2023-08-25T09:52:04.626874+0000","last_unstale":"2023-08-25T09:52:06.978450+0000","last_undegraded":"2023-08-25T09:52:06.978450+0000","last_fullsized":"2023-08-25T09:52:06.978450+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:04.237329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599734+0000","last_change":"2023-08-25T09:52:04.612395+0000","last_active":"2023-08-25T09:52:05.599734+0000","last_peered":"2023-08-25T09:52:05.599734+0000","last_clean":"2023-08-25T09:52:05.599734+0000","last_became_active":"2023-08-25T09:52:04.607152+0000","last_became_peered":"2023-08-25T09:52:04.607152+0000","last_unstale":"2023-08-25T09:52:05.599734+0000","last_undegraded":"2023-08-25T09:52:05.599734+0000","last_fullsized":"2023-08-25T09:52:05.599734+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:26:51.307680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978190+0000","last_change":"2023-08-25T09:52:04.624655+0000","last_active":"2023-08-25T09:52:06.978190+0000","last_peered":"2023-08-25T09:52:06.978190+0000","last_clean":"2023-08-25T09:52:06.978190+0000","last_became_active":"2023-08-25T09:52:04.624478+0000","last_became_peered":"2023-08-25T09:52:04.624478+0000","last_unstale":"2023-08-25T09:52:06.978190+0000","last_undegraded":"2023-08-25T09:52:06.978190+0000","last_fullsized":"2023-08-25T09:52:06.978190+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:34.884721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599300+0000","last_change":"2023-08-25T09:52:04.626361+0000","last_active":"2023-08-25T09:52:05.599300+0000","last_peered":"2023-08-25T09:52:05.599300+0000","last_clean":"2023-08-25T09:52:05.599300+0000","last_became_active":"2023-08-25T09:52:04.625812+0000","last_became_peered":"2023-08-25T09:52:04.625812+0000","last_unstale":"2023-08-25T09:52:05.599300+0000","last_undegraded":"2023-08-25T09:52:05.599300+0000","last_fullsized":"2023-08-25T09:52:05.599300+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:50:04.366957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598504+0000","last_change":"2023-08-25T09:52:04.612349+0000","last_active":"2023-08-25T09:52:05.598504+0000","last_peered":"2023-08-25T09:52:05.598504+0000","last_clean":"2023-08-25T09:52:05.598504+0000","last_became_active":"2023-08-25T09:52:04.611885+0000","last_became_peered":"2023-08-25T09:52:04.611885+0000","last_unstale":"2023-08-25T09:52:05.598504+0000","last_undegraded":"2023-08-25T09:52:05.598504+0000","last_fullsized":"2023-08-25T09:52:05.598504+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:33:36.629581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978698+0000","last_change":"2023-08-25T09:52:04.627125+0000","last_active":"2023-08-25T09:52:06.978698+0000","last_peered":"2023-08-25T09:52:06.978698+0000","last_clean":"2023-08-25T09:52:06.978698+0000","last_became_active":"2023-08-25T09:52:04.625250+0000","last_became_peered":"2023-08-25T09:52:04.625250+0000","last_unstale":"2023-08-25T09:52:06.978698+0000","last_undegraded":"2023-08-25T09:52:06.978698+0000","last_fullsized":"2023-08-25T09:52:06.978698+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:16:34.884721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977263+0000","last_change":"2023-08-25T09:52:04.624230+0000","last_active":"2023-08-25T09:52:06.977263+0000","last_peered":"2023-08-25T09:52:06.977263+0000","last_clean":"2023-08-25T09:52:06.977263+0000","last_became_active":"2023-08-25T09:52:04.623981+0000","last_became_peered":"2023-08-25T09:52:04.623981+0000","last_unstale":"2023-08-25T09:52:06.977263+0000","last_undegraded":"2023-08-25T09:52:06.977263+0000","last_fullsized":"2023-08-25T09:52:06.977263+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:18:08.663525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601907+0000","last_change":"2023-08-25T09:52:04.624126+0000","last_active":"2023-08-25T09:52:05.601907+0000","last_peered":"2023-08-25T09:52:05.601907+0000","last_clean":"2023-08-25T09:52:05.601907+0000","last_became_active":"2023-08-25T09:52:04.623878+0000","last_became_peered":"2023-08-25T09:52:04.623878+0000","last_unstale":"2023-08-25T09:52:05.601907+0000","last_undegraded":"2023-08-25T09:52:05.601907+0000","last_fullsized":"2023-08-25T09:52:05.601907+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:58.851337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977983+0000","last_change":"2023-08-25T09:52:04.620817+0000","last_active":"2023-08-25T09:52:06.977983+0000","last_peered":"2023-08-25T09:52:06.977983+0000","last_clean":"2023-08-25T09:52:06.977983+0000","last_became_active":"2023-08-25T09:52:04.620537+0000","last_became_peered":"2023-08-25T09:52:04.620537+0000","last_unstale":"2023-08-25T09:52:06.977983+0000","last_undegraded":"2023-08-25T09:52:06.977983+0000","last_fullsized":"2023-08-25T09:52:06.977983+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:32.675064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978306+0000","last_change":"2023-08-25T09:52:04.622813+0000","last_active":"2023-08-25T09:52:06.978306+0000","last_peered":"2023-08-25T09:52:06.978306+0000","last_clean":"2023-08-25T09:52:06.978306+0000","last_became_active":"2023-08-25T09:52:04.622347+0000","last_became_peered":"2023-08-25T09:52:04.622347+0000","last_unstale":"2023-08-25T09:52:06.978306+0000","last_undegraded":"2023-08-25T09:52:06.978306+0000","last_fullsized":"2023-08-25T09:52:06.978306+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:45:01.358105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978917+0000","last_change":"2023-08-25T09:52:04.627414+0000","last_active":"2023-08-25T09:52:06.978917+0000","last_peered":"2023-08-25T09:52:06.978917+0000","last_clean":"2023-08-25T09:52:06.978917+0000","last_became_active":"2023-08-25T09:52:04.625586+0000","last_became_peered":"2023-08-25T09:52:04.625586+0000","last_unstale":"2023-08-25T09:52:06.978917+0000","last_undegraded":"2023-08-25T09:52:06.978917+0000","last_fullsized":"2023-08-25T09:52:06.978917+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:23.051366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.597997+0000","last_change":"2023-08-25T09:52:04.622725+0000","last_active":"2023-08-25T09:52:05.597997+0000","last_peered":"2023-08-25T09:52:05.597997+0000","last_clean":"2023-08-25T09:52:05.597997+0000","last_became_active":"2023-08-25T09:52:04.622326+0000","last_became_peered":"2023-08-25T09:52:04.622326+0000","last_unstale":"2023-08-25T09:52:05.597997+0000","last_undegraded":"2023-08-25T09:52:05.597997+0000","last_fullsized":"2023-08-25T09:52:05.597997+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:40.942518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594716+0000","last_change":"2023-08-25T09:52:04.613623+0000","last_active":"2023-08-25T09:52:05.594716+0000","last_peered":"2023-08-25T09:52:05.594716+0000","last_clean":"2023-08-25T09:52:05.594716+0000","last_became_active":"2023-08-25T09:52:04.612966+0000","last_became_peered":"2023-08-25T09:52:04.612966+0000","last_unstale":"2023-08-25T09:52:05.594716+0000","last_undegraded":"2023-08-25T09:52:05.594716+0000","last_fullsized":"2023-08-25T09:52:05.594716+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:21:08.314372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979043+0000","last_change":"2023-08-25T09:52:04.627459+0000","last_active":"2023-08-25T09:52:06.979043+0000","last_peered":"2023-08-25T09:52:06.979043+0000","last_clean":"2023-08-25T09:52:06.979043+0000","last_became_active":"2023-08-25T09:52:04.626892+0000","last_became_peered":"2023-08-25T09:52:04.626892+0000","last_unstale":"2023-08-25T09:52:06.979043+0000","last_undegraded":"2023-08-25T09:52:06.979043+0000","last_fullsized":"2023-08-25T09:52:06.979043+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:00:20.842644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978283+0000","last_change":"2023-08-25T09:52:04.612691+0000","last_active":"2023-08-25T09:52:06.978283+0000","last_peered":"2023-08-25T09:52:06.978283+0000","last_clean":"2023-08-25T09:52:06.978283+0000","last_became_active":"2023-08-25T09:52:04.612199+0000","last_became_peered":"2023-08-25T09:52:04.612199+0000","last_unstale":"2023-08-25T09:52:06.978283+0000","last_undegraded":"2023-08-25T09:52:06.978283+0000","last_fullsized":"2023-08-25T09:52:06.978283+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:25:23.051366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599582+0000","last_change":"2023-08-25T09:52:04.626435+0000","last_active":"2023-08-25T09:52:05.599582+0000","last_peered":"2023-08-25T09:52:05.599582+0000","last_clean":"2023-08-25T09:52:05.599582+0000","last_became_active":"2023-08-25T09:52:04.626050+0000","last_became_peered":"2023-08-25T09:52:04.626050+0000","last_unstale":"2023-08-25T09:52:05.599582+0000","last_undegraded":"2023-08-25T09:52:05.599582+0000","last_fullsized":"2023-08-25T09:52:05.599582+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:35:30.431066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977748+0000","last_change":"2023-08-25T09:52:04.625300+0000","last_active":"2023-08-25T09:52:06.977748+0000","last_peered":"2023-08-25T09:52:06.977748+0000","last_clean":"2023-08-25T09:52:06.977748+0000","last_became_active":"2023-08-25T09:52:04.624890+0000","last_became_peered":"2023-08-25T09:52:04.624890+0000","last_unstale":"2023-08-25T09:52:06.977748+0000","last_undegraded":"2023-08-25T09:52:06.977748+0000","last_fullsized":"2023-08-25T09:52:06.977748+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:04:08.579191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598557+0000","last_change":"2023-08-25T09:52:04.614691+0000","last_active":"2023-08-25T09:52:05.598557+0000","last_peered":"2023-08-25T09:52:05.598557+0000","last_clean":"2023-08-25T09:52:05.598557+0000","last_became_active":"2023-08-25T09:52:04.614305+0000","last_became_peered":"2023-08-25T09:52:04.614305+0000","last_unstale":"2023-08-25T09:52:05.598557+0000","last_undegraded":"2023-08-25T09:52:05.598557+0000","last_fullsized":"2023-08-25T09:52:05.598557+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:41:57.725402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979364+0000","last_change":"2023-08-25T09:52:04.627696+0000","last_active":"2023-08-25T09:52:06.979364+0000","last_peered":"2023-08-25T09:52:06.979364+0000","last_clean":"2023-08-25T09:52:06.979364+0000","last_became_active":"2023-08-25T09:52:04.627211+0000","last_became_peered":"2023-08-25T09:52:04.627211+0000","last_unstale":"2023-08-25T09:52:06.979364+0000","last_undegraded":"2023-08-25T09:52:06.979364+0000","last_fullsized":"2023-08-25T09:52:06.979364+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:16.624825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599254+0000","last_change":"2023-08-25T09:52:04.614199+0000","last_active":"2023-08-25T09:52:05.599254+0000","last_peered":"2023-08-25T09:52:05.599254+0000","last_clean":"2023-08-25T09:52:05.599254+0000","last_became_active":"2023-08-25T09:52:04.613179+0000","last_became_peered":"2023-08-25T09:52:04.613179+0000","last_unstale":"2023-08-25T09:52:05.599254+0000","last_undegraded":"2023-08-25T09:52:05.599254+0000","last_fullsized":"2023-08-25T09:52:05.599254+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:42.152837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979806+0000","last_change":"2023-08-25T09:52:04.625832+0000","last_active":"2023-08-25T09:52:06.979806+0000","last_peered":"2023-08-25T09:52:06.979806+0000","last_clean":"2023-08-25T09:52:06.979806+0000","last_became_active":"2023-08-25T09:52:04.625323+0000","last_became_peered":"2023-08-25T09:52:04.625323+0000","last_unstale":"2023-08-25T09:52:06.979806+0000","last_undegraded":"2023-08-25T09:52:06.979806+0000","last_fullsized":"2023-08-25T09:52:06.979806+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:32.060870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980040+0000","last_change":"2023-08-25T09:52:04.606510+0000","last_active":"2023-08-25T09:52:06.980040+0000","last_peered":"2023-08-25T09:52:06.980040+0000","last_clean":"2023-08-25T09:52:06.980040+0000","last_became_active":"2023-08-25T09:52:04.605382+0000","last_became_peered":"2023-08-25T09:52:04.605382+0000","last_unstale":"2023-08-25T09:52:06.980040+0000","last_undegraded":"2023-08-25T09:52:06.980040+0000","last_fullsized":"2023-08-25T09:52:06.980040+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:15.708169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600531+0000","last_change":"2023-08-25T09:52:04.616677+0000","last_active":"2023-08-25T09:52:05.600531+0000","last_peered":"2023-08-25T09:52:05.600531+0000","last_clean":"2023-08-25T09:52:05.600531+0000","last_became_active":"2023-08-25T09:52:04.614596+0000","last_became_peered":"2023-08-25T09:52:04.614596+0000","last_unstale":"2023-08-25T09:52:05.600531+0000","last_undegraded":"2023-08-25T09:52:05.600531+0000","last_fullsized":"2023-08-25T09:52:05.600531+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:53.023481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594927+0000","last_change":"2023-08-25T09:52:04.609767+0000","last_active":"2023-08-25T09:52:05.594927+0000","last_peered":"2023-08-25T09:52:05.594927+0000","last_clean":"2023-08-25T09:52:05.594927+0000","last_became_active":"2023-08-25T09:52:04.606031+0000","last_became_peered":"2023-08-25T09:52:04.606031+0000","last_unstale":"2023-08-25T09:52:05.594927+0000","last_undegraded":"2023-08-25T09:52:05.594927+0000","last_fullsized":"2023-08-25T09:52:05.594927+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:06.542127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979128+0000","last_change":"2023-08-25T09:52:04.622622+0000","last_active":"2023-08-25T09:52:06.979128+0000","last_peered":"2023-08-25T09:52:06.979128+0000","last_clean":"2023-08-25T09:52:06.979128+0000","last_became_active":"2023-08-25T09:52:04.621916+0000","last_became_peered":"2023-08-25T09:52:04.621916+0000","last_unstale":"2023-08-25T09:52:06.979128+0000","last_undegraded":"2023-08-25T09:52:06.979128+0000","last_fullsized":"2023-08-25T09:52:06.979128+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:52.575789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976919+0000","last_change":"2023-08-25T09:52:04.629777+0000","last_active":"2023-08-25T09:52:06.976919+0000","last_peered":"2023-08-25T09:52:06.976919+0000","last_clean":"2023-08-25T09:52:06.976919+0000","last_became_active":"2023-08-25T09:52:04.628214+0000","last_became_peered":"2023-08-25T09:52:04.628214+0000","last_unstale":"2023-08-25T09:52:06.976919+0000","last_undegraded":"2023-08-25T09:52:06.976919+0000","last_fullsized":"2023-08-25T09:52:06.976919+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:49:37.628859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980187+0000","last_change":"2023-08-25T09:52:04.606056+0000","last_active":"2023-08-25T09:52:06.980187+0000","last_peered":"2023-08-25T09:52:06.980187+0000","last_clean":"2023-08-25T09:52:06.980187+0000","last_became_active":"2023-08-25T09:52:04.604555+0000","last_became_peered":"2023-08-25T09:52:04.604555+0000","last_unstale":"2023-08-25T09:52:06.980187+0000","last_undegraded":"2023-08-25T09:52:06.980187+0000","last_fullsized":"2023-08-25T09:52:06.980187+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:49:02.585201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978773+0000","last_change":"2023-08-25T09:52:04.624540+0000","last_active":"2023-08-25T09:52:06.978773+0000","last_peered":"2023-08-25T09:52:06.978773+0000","last_clean":"2023-08-25T09:52:06.978773+0000","last_became_active":"2023-08-25T09:52:04.624033+0000","last_became_peered":"2023-08-25T09:52:04.624033+0000","last_unstale":"2023-08-25T09:52:06.978773+0000","last_undegraded":"2023-08-25T09:52:06.978773+0000","last_fullsized":"2023-08-25T09:52:06.978773+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:59:22.449654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"197'4","reported_seq":17,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979006+0000","last_change":"2023-08-25T09:52:03.605509+0000","last_active":"2023-08-25T09:52:06.979006+0000","last_peered":"2023-08-25T09:52:06.979006+0000","last_clean":"2023-08-25T09:52:06.979006+0000","last_became_active":"2023-08-25T09:52:03.596779+0000","last_became_peered":"2023-08-25T09:52:03.596779+0000","last_unstale":"2023-08-25T09:52:06.979006+0000","last_undegraded":"2023-08-25T09:52:06.979006+0000","last_fullsized":"2023-08-25T09:52:06.979006+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:08:55.813989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600442+0000","last_change":"2023-08-25T09:52:04.621176+0000","last_active":"2023-08-25T09:52:05.600442+0000","last_peered":"2023-08-25T09:52:05.600442+0000","last_clean":"2023-08-25T09:52:05.600442+0000","last_became_active":"2023-08-25T09:52:04.620648+0000","last_became_peered":"2023-08-25T09:52:04.620648+0000","last_unstale":"2023-08-25T09:52:05.600442+0000","last_undegraded":"2023-08-25T09:52:05.600442+0000","last_fullsized":"2023-08-25T09:52:05.600442+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:37.317841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977679+0000","last_change":"2023-08-25T09:52:03.618376+0000","last_active":"2023-08-25T09:52:06.977679+0000","last_peered":"2023-08-25T09:52:06.977679+0000","last_clean":"2023-08-25T09:52:06.977679+0000","last_became_active":"2023-08-25T09:52:03.617343+0000","last_became_peered":"2023-08-25T09:52:03.617343+0000","last_unstale":"2023-08-25T09:52:06.977679+0000","last_undegraded":"2023-08-25T09:52:06.977679+0000","last_fullsized":"2023-08-25T09:52:06.977679+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:45:00.348861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598202+0000","last_change":"2023-08-25T09:52:04.615114+0000","last_active":"2023-08-25T09:52:05.598202+0000","last_peered":"2023-08-25T09:52:05.598202+0000","last_clean":"2023-08-25T09:52:05.598202+0000","last_became_active":"2023-08-25T09:52:04.614691+0000","last_became_peered":"2023-08-25T09:52:04.614691+0000","last_unstale":"2023-08-25T09:52:05.598202+0000","last_undegraded":"2023-08-25T09:52:05.598202+0000","last_fullsized":"2023-08-25T09:52:05.598202+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:11:54.517313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1d","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976923+0000","last_change":"2023-08-25T09:52:03.615544+0000","last_active":"2023-08-25T09:52:06.976923+0000","last_peered":"2023-08-25T09:52:06.976923+0000","last_clean":"2023-08-25T09:52:06.976923+0000","last_became_active":"2023-08-25T09:52:03.615200+0000","last_became_peered":"2023-08-25T09:52:03.615200+0000","last_unstale":"2023-08-25T09:52:06.976923+0000","last_undegraded":"2023-08-25T09:52:06.976923+0000","last_fullsized":"2023-08-25T09:52:06.976923+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:24:11.361908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976773+0000","last_change":"2023-08-25T09:52:04.622820+0000","last_active":"2023-08-25T09:52:06.976773+0000","last_peered":"2023-08-25T09:52:06.976773+0000","last_clean":"2023-08-25T09:52:06.976773+0000","last_became_active":"2023-08-25T09:52:04.622556+0000","last_became_peered":"2023-08-25T09:52:04.622556+0000","last_unstale":"2023-08-25T09:52:06.976773+0000","last_undegraded":"2023-08-25T09:52:06.976773+0000","last_fullsized":"2023-08-25T09:52:06.976773+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:12.796779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599945+0000","last_change":"2023-08-25T09:52:03.615665+0000","last_active":"2023-08-25T09:52:05.599945+0000","last_peered":"2023-08-25T09:52:05.599945+0000","last_clean":"2023-08-25T09:52:05.599945+0000","last_became_active":"2023-08-25T09:52:03.615398+0000","last_became_peered":"2023-08-25T09:52:03.615398+0000","last_unstale":"2023-08-25T09:52:05.599945+0000","last_undegraded":"2023-08-25T09:52:05.599945+0000","last_fullsized":"2023-08-25T09:52:05.599945+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:32:24.261532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598463+0000","last_change":"2023-08-25T09:52:04.606215+0000","last_active":"2023-08-25T09:52:05.598463+0000","last_peered":"2023-08-25T09:52:05.598463+0000","last_clean":"2023-08-25T09:52:05.598463+0000","last_became_active":"2023-08-25T09:52:04.605569+0000","last_became_peered":"2023-08-25T09:52:04.605569+0000","last_unstale":"2023-08-25T09:52:05.598463+0000","last_undegraded":"2023-08-25T09:52:05.598463+0000","last_fullsized":"2023-08-25T09:52:05.598463+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:25:33.270874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1b","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600761+0000","last_change":"2023-08-25T09:52:03.612691+0000","last_active":"2023-08-25T09:52:05.600761+0000","last_peered":"2023-08-25T09:52:05.600761+0000","last_clean":"2023-08-25T09:52:05.600761+0000","last_became_active":"2023-08-25T09:52:03.609712+0000","last_became_peered":"2023-08-25T09:52:03.609712+0000","last_unstale":"2023-08-25T09:52:05.600761+0000","last_undegraded":"2023-08-25T09:52:05.600761+0000","last_fullsized":"2023-08-25T09:52:05.600761+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:18.578319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601189+0000","last_change":"2023-08-25T09:52:04.619419+0000","last_active":"2023-08-25T09:52:05.601189+0000","last_peered":"2023-08-25T09:52:05.601189+0000","last_clean":"2023-08-25T09:52:05.601189+0000","last_became_active":"2023-08-25T09:52:04.616364+0000","last_became_peered":"2023-08-25T09:52:04.616364+0000","last_unstale":"2023-08-25T09:52:05.601189+0000","last_undegraded":"2023-08-25T09:52:05.601189+0000","last_fullsized":"2023-08-25T09:52:05.601189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:19.369746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1a","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978543+0000","last_change":"2023-08-25T09:52:03.612785+0000","last_active":"2023-08-25T09:52:06.978543+0000","last_peered":"2023-08-25T09:52:06.978543+0000","last_clean":"2023-08-25T09:52:06.978543+0000","last_became_active":"2023-08-25T09:52:03.612129+0000","last_became_peered":"2023-08-25T09:52:03.612129+0000","last_unstale":"2023-08-25T09:52:06.978543+0000","last_undegraded":"2023-08-25T09:52:06.978543+0000","last_fullsized":"2023-08-25T09:52:06.978543+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:32.325636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598377+0000","last_change":"2023-08-25T09:52:04.622547+0000","last_active":"2023-08-25T09:52:05.598377+0000","last_peered":"2023-08-25T09:52:05.598377+0000","last_clean":"2023-08-25T09:52:05.598377+0000","last_became_active":"2023-08-25T09:52:04.621941+0000","last_became_peered":"2023-08-25T09:52:04.621941+0000","last_unstale":"2023-08-25T09:52:05.598377+0000","last_undegraded":"2023-08-25T09:52:05.598377+0000","last_fullsized":"2023-08-25T09:52:05.598377+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:09:49.426665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978061+0000","last_change":"2023-08-25T09:52:03.618689+0000","last_active":"2023-08-25T09:52:06.978061+0000","last_peered":"2023-08-25T09:52:06.978061+0000","last_clean":"2023-08-25T09:52:06.978061+0000","last_became_active":"2023-08-25T09:52:03.617671+0000","last_became_peered":"2023-08-25T09:52:03.617671+0000","last_unstale":"2023-08-25T09:52:06.978061+0000","last_undegraded":"2023-08-25T09:52:06.978061+0000","last_fullsized":"2023-08-25T09:52:06.978061+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:35:03.228085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594833+0000","last_change":"2023-08-25T09:52:04.626763+0000","last_active":"2023-08-25T09:52:05.594833+0000","last_peered":"2023-08-25T09:52:05.594833+0000","last_clean":"2023-08-25T09:52:05.594833+0000","last_became_active":"2023-08-25T09:52:04.626503+0000","last_became_peered":"2023-08-25T09:52:04.626503+0000","last_unstale":"2023-08-25T09:52:05.594833+0000","last_undegraded":"2023-08-25T09:52:05.594833+0000","last_fullsized":"2023-08-25T09:52:05.594833+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:22:38.126861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979938+0000","last_change":"2023-08-25T09:52:03.603589+0000","last_active":"2023-08-25T09:52:06.979938+0000","last_peered":"2023-08-25T09:52:06.979938+0000","last_clean":"2023-08-25T09:52:06.979938+0000","last_became_active":"2023-08-25T09:52:03.602798+0000","last_became_peered":"2023-08-25T09:52:03.602798+0000","last_unstale":"2023-08-25T09:52:06.979938+0000","last_undegraded":"2023-08-25T09:52:06.979938+0000","last_fullsized":"2023-08-25T09:52:06.979938+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:58:18.360502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979138+0000","last_change":"2023-08-25T09:52:04.617158+0000","last_active":"2023-08-25T09:52:06.979138+0000","last_peered":"2023-08-25T09:52:06.979138+0000","last_clean":"2023-08-25T09:52:06.979138+0000","last_became_active":"2023-08-25T09:52:04.616462+0000","last_became_peered":"2023-08-25T09:52:04.616462+0000","last_unstale":"2023-08-25T09:52:06.979138+0000","last_undegraded":"2023-08-25T09:52:06.979138+0000","last_fullsized":"2023-08-25T09:52:06.979138+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:15:24.095300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.f","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979469+0000","last_change":"2023-08-25T09:52:03.618945+0000","last_active":"2023-08-25T09:52:06.979469+0000","last_peered":"2023-08-25T09:52:06.979469+0000","last_clean":"2023-08-25T09:52:06.979469+0000","last_became_active":"2023-08-25T09:52:03.618057+0000","last_became_peered":"2023-08-25T09:52:03.618057+0000","last_unstale":"2023-08-25T09:52:06.979469+0000","last_undegraded":"2023-08-25T09:52:06.979469+0000","last_fullsized":"2023-08-25T09:52:06.979469+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:05.630395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977905+0000","last_change":"2023-08-25T09:52:04.613446+0000","last_active":"2023-08-25T09:52:06.977905+0000","last_peered":"2023-08-25T09:52:06.977905+0000","last_clean":"2023-08-25T09:52:06.977905+0000","last_became_active":"2023-08-25T09:52:04.612830+0000","last_became_peered":"2023-08-25T09:52:04.612830+0000","last_unstale":"2023-08-25T09:52:06.977905+0000","last_undegraded":"2023-08-25T09:52:06.977905+0000","last_fullsized":"2023-08-25T09:52:06.977905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:28:40.373369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598883+0000","last_change":"2023-08-25T09:52:03.617112+0000","last_active":"2023-08-25T09:52:05.598883+0000","last_peered":"2023-08-25T09:52:05.598883+0000","last_clean":"2023-08-25T09:52:05.598883+0000","last_became_active":"2023-08-25T09:52:03.616870+0000","last_became_peered":"2023-08-25T09:52:03.616870+0000","last_unstale":"2023-08-25T09:52:05.598883+0000","last_undegraded":"2023-08-25T09:52:05.598883+0000","last_fullsized":"2023-08-25T09:52:05.598883+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:32:14.394268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977880+0000","last_change":"2023-08-25T09:52:04.618901+0000","last_active":"2023-08-25T09:52:06.977880+0000","last_peered":"2023-08-25T09:52:06.977880+0000","last_clean":"2023-08-25T09:52:06.977880+0000","last_became_active":"2023-08-25T09:52:04.618464+0000","last_became_peered":"2023-08-25T09:52:04.618464+0000","last_unstale":"2023-08-25T09:52:06.977880+0000","last_undegraded":"2023-08-25T09:52:06.977880+0000","last_fullsized":"2023-08-25T09:52:06.977880+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:01:34.504532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"197'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980224+0000","last_change":"2023-08-25T09:52:03.596983+0000","last_active":"2023-08-25T09:52:06.980224+0000","last_peered":"2023-08-25T09:52:06.980224+0000","last_clean":"2023-08-25T09:52:06.980224+0000","last_became_active":"2023-08-25T09:52:03.596298+0000","last_became_peered":"2023-08-25T09:52:03.596298+0000","last_unstale":"2023-08-25T09:52:06.980224+0000","last_undegraded":"2023-08-25T09:52:06.980224+0000","last_fullsized":"2023-08-25T09:52:06.980224+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:04:57.842093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979133+0000","last_change":"2023-08-25T09:52:04.625834+0000","last_active":"2023-08-25T09:52:06.979133+0000","last_peered":"2023-08-25T09:52:06.979133+0000","last_clean":"2023-08-25T09:52:06.979133+0000","last_became_active":"2023-08-25T09:52:04.624790+0000","last_became_peered":"2023-08-25T09:52:04.624790+0000","last_unstale":"2023-08-25T09:52:06.979133+0000","last_undegraded":"2023-08-25T09:52:06.979133+0000","last_fullsized":"2023-08-25T09:52:06.979133+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:45:33.334879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979304+0000","last_change":"2023-08-25T09:52:03.614128+0000","last_active":"2023-08-25T09:52:06.979304+0000","last_peered":"2023-08-25T09:52:06.979304+0000","last_clean":"2023-08-25T09:52:06.979304+0000","last_became_active":"2023-08-25T09:52:03.610416+0000","last_became_peered":"2023-08-25T09:52:03.610416+0000","last_unstale":"2023-08-25T09:52:06.979304+0000","last_undegraded":"2023-08-25T09:52:06.979304+0000","last_fullsized":"2023-08-25T09:52:06.979304+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:00:50.255020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600006+0000","last_change":"2023-08-25T09:52:04.618371+0000","last_active":"2023-08-25T09:52:05.600006+0000","last_peered":"2023-08-25T09:52:05.600006+0000","last_clean":"2023-08-25T09:52:05.600006+0000","last_became_active":"2023-08-25T09:52:04.617925+0000","last_became_peered":"2023-08-25T09:52:04.617925+0000","last_unstale":"2023-08-25T09:52:05.600006+0000","last_undegraded":"2023-08-25T09:52:05.600006+0000","last_fullsized":"2023-08-25T09:52:05.600006+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:32.675064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.b","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978949+0000","last_change":"2023-08-25T09:52:03.617559+0000","last_active":"2023-08-25T09:52:06.978949+0000","last_peered":"2023-08-25T09:52:06.978949+0000","last_clean":"2023-08-25T09:52:06.978949+0000","last_became_active":"2023-08-25T09:52:03.616663+0000","last_became_peered":"2023-08-25T09:52:03.616663+0000","last_unstale":"2023-08-25T09:52:06.978949+0000","last_undegraded":"2023-08-25T09:52:06.978949+0000","last_fullsized":"2023-08-25T09:52:06.978949+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:14:52.014238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978320+0000","last_change":"2023-08-25T09:52:04.618659+0000","last_active":"2023-08-25T09:52:06.978320+0000","last_peered":"2023-08-25T09:52:06.978320+0000","last_clean":"2023-08-25T09:52:06.978320+0000","last_became_active":"2023-08-25T09:52:04.617978+0000","last_became_peered":"2023-08-25T09:52:04.617978+0000","last_unstale":"2023-08-25T09:52:06.978320+0000","last_undegraded":"2023-08-25T09:52:06.978320+0000","last_fullsized":"2023-08-25T09:52:06.978320+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:37.317841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979649+0000","last_change":"2023-08-25T09:52:03.609849+0000","last_active":"2023-08-25T09:52:06.979649+0000","last_peered":"2023-08-25T09:52:06.979649+0000","last_clean":"2023-08-25T09:52:06.979649+0000","last_became_active":"2023-08-25T09:52:03.609198+0000","last_became_peered":"2023-08-25T09:52:03.609198+0000","last_unstale":"2023-08-25T09:52:06.979649+0000","last_undegraded":"2023-08-25T09:52:06.979649+0000","last_fullsized":"2023-08-25T09:52:06.979649+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:09:14.999030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979290+0000","last_change":"2023-08-25T09:52:04.617431+0000","last_active":"2023-08-25T09:52:06.979290+0000","last_peered":"2023-08-25T09:52:06.979290+0000","last_clean":"2023-08-25T09:52:06.979290+0000","last_became_active":"2023-08-25T09:52:04.616896+0000","last_became_peered":"2023-08-25T09:52:04.616896+0000","last_unstale":"2023-08-25T09:52:06.979290+0000","last_undegraded":"2023-08-25T09:52:06.979290+0000","last_fullsized":"2023-08-25T09:52:06.979290+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:19.587563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599442+0000","last_change":"2023-08-25T09:52:03.613206+0000","last_active":"2023-08-25T09:52:05.599442+0000","last_peered":"2023-08-25T09:52:05.599442+0000","last_clean":"2023-08-25T09:52:05.599442+0000","last_became_active":"2023-08-25T09:52:03.612871+0000","last_became_peered":"2023-08-25T09:52:03.612871+0000","last_unstale":"2023-08-25T09:52:05.599442+0000","last_undegraded":"2023-08-25T09:52:05.599442+0000","last_fullsized":"2023-08-25T09:52:05.599442+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:40:13.982356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595647+0000","last_change":"2023-08-25T09:52:04.621653+0000","last_active":"2023-08-25T09:52:05.595647+0000","last_peered":"2023-08-25T09:52:05.595647+0000","last_clean":"2023-08-25T09:52:05.595647+0000","last_became_active":"2023-08-25T09:52:04.621306+0000","last_became_peered":"2023-08-25T09:52:04.621306+0000","last_unstale":"2023-08-25T09:52:05.595647+0000","last_undegraded":"2023-08-25T09:52:05.595647+0000","last_fullsized":"2023-08-25T09:52:05.595647+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:10:13.499449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980116+0000","last_change":"2023-08-25T09:52:03.614440+0000","last_active":"2023-08-25T09:52:06.980116+0000","last_peered":"2023-08-25T09:52:06.980116+0000","last_clean":"2023-08-25T09:52:06.980116+0000","last_became_active":"2023-08-25T09:52:03.613170+0000","last_became_peered":"2023-08-25T09:52:03.613170+0000","last_unstale":"2023-08-25T09:52:06.980116+0000","last_undegraded":"2023-08-25T09:52:06.980116+0000","last_fullsized":"2023-08-25T09:52:06.980116+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:24.408611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595199+0000","last_change":"2023-08-25T09:52:04.604860+0000","last_active":"2023-08-25T09:52:05.595199+0000","last_peered":"2023-08-25T09:52:05.595199+0000","last_clean":"2023-08-25T09:52:05.595199+0000","last_became_active":"2023-08-25T09:52:04.603337+0000","last_became_peered":"2023-08-25T09:52:04.603337+0000","last_unstale":"2023-08-25T09:52:05.595199+0000","last_undegraded":"2023-08-25T09:52:05.595199+0000","last_fullsized":"2023-08-25T09:52:05.595199+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:08:01.014394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978774+0000","last_change":"2023-08-25T09:52:03.606748+0000","last_active":"2023-08-25T09:52:06.978774+0000","last_peered":"2023-08-25T09:52:06.978774+0000","last_clean":"2023-08-25T09:52:06.978774+0000","last_became_active":"2023-08-25T09:52:03.606384+0000","last_became_peered":"2023-08-25T09:52:03.606384+0000","last_unstale":"2023-08-25T09:52:06.978774+0000","last_undegraded":"2023-08-25T09:52:06.978774+0000","last_fullsized":"2023-08-25T09:52:06.978774+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:55.782470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979062+0000","last_change":"2023-08-25T09:52:04.613626+0000","last_active":"2023-08-25T09:52:06.979062+0000","last_peered":"2023-08-25T09:52:06.979062+0000","last_clean":"2023-08-25T09:52:06.979062+0000","last_became_active":"2023-08-25T09:52:04.613221+0000","last_became_peered":"2023-08-25T09:52:04.613221+0000","last_unstale":"2023-08-25T09:52:06.979062+0000","last_undegraded":"2023-08-25T09:52:06.979062+0000","last_fullsized":"2023-08-25T09:52:06.979062+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:19:08.585858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.6","version":"198'1","reported_seq":14,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979649+0000","last_change":"2023-08-25T09:52:03.598008+0000","last_active":"2023-08-25T09:52:06.979649+0000","last_peered":"2023-08-25T09:52:06.979649+0000","last_clean":"2023-08-25T09:52:06.979649+0000","last_became_active":"2023-08-25T09:52:03.597171+0000","last_became_peered":"2023-08-25T09:52:03.597171+0000","last_unstale":"2023-08-25T09:52:06.979649+0000","last_undegraded":"2023-08-25T09:52:06.979649+0000","last_fullsized":"2023-08-25T09:52:06.979649+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:29.871817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979809+0000","last_change":"2023-08-25T09:52:04.617324+0000","last_active":"2023-08-25T09:52:06.979809+0000","last_peered":"2023-08-25T09:52:06.979809+0000","last_clean":"2023-08-25T09:52:06.979809+0000","last_became_active":"2023-08-25T09:52:04.616867+0000","last_became_peered":"2023-08-25T09:52:04.616867+0000","last_unstale":"2023-08-25T09:52:06.979809+0000","last_undegraded":"2023-08-25T09:52:06.979809+0000","last_fullsized":"2023-08-25T09:52:06.979809+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:13:46.538086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980140+0000","last_change":"2023-08-25T09:52:03.619922+0000","last_active":"2023-08-25T09:52:06.980140+0000","last_peered":"2023-08-25T09:52:06.980140+0000","last_clean":"2023-08-25T09:52:06.980140+0000","last_became_active":"2023-08-25T09:52:03.619288+0000","last_became_peered":"2023-08-25T09:52:03.619288+0000","last_unstale":"2023-08-25T09:52:06.980140+0000","last_undegraded":"2023-08-25T09:52:06.980140+0000","last_fullsized":"2023-08-25T09:52:06.980140+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:38:47.888788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601489+0000","last_change":"2023-08-25T09:52:04.618818+0000","last_active":"2023-08-25T09:52:05.601489+0000","last_peered":"2023-08-25T09:52:05.601489+0000","last_clean":"2023-08-25T09:52:05.601489+0000","last_became_active":"2023-08-25T09:52:04.615160+0000","last_became_peered":"2023-08-25T09:52:04.615160+0000","last_unstale":"2023-08-25T09:52:05.601489+0000","last_undegraded":"2023-08-25T09:52:05.601489+0000","last_fullsized":"2023-08-25T09:52:05.601489+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:46.121852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979396+0000","last_change":"2023-08-25T09:52:03.614922+0000","last_active":"2023-08-25T09:52:06.979396+0000","last_peered":"2023-08-25T09:52:06.979396+0000","last_clean":"2023-08-25T09:52:06.979396+0000","last_became_active":"2023-08-25T09:52:03.614577+0000","last_became_peered":"2023-08-25T09:52:03.614577+0000","last_unstale":"2023-08-25T09:52:06.979396+0000","last_undegraded":"2023-08-25T09:52:06.979396+0000","last_fullsized":"2023-08-25T09:52:06.979396+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:23.368847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.594731+0000","last_change":"2023-08-25T09:52:04.622816+0000","last_active":"2023-08-25T09:52:05.594731+0000","last_peered":"2023-08-25T09:52:05.594731+0000","last_clean":"2023-08-25T09:52:05.594731+0000","last_became_active":"2023-08-25T09:52:04.622527+0000","last_became_peered":"2023-08-25T09:52:04.622527+0000","last_unstale":"2023-08-25T09:52:05.594731+0000","last_undegraded":"2023-08-25T09:52:05.594731+0000","last_fullsized":"2023-08-25T09:52:05.594731+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:15:29.167368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980224+0000","last_change":"2023-08-25T09:52:03.614822+0000","last_active":"2023-08-25T09:52:06.980224+0000","last_peered":"2023-08-25T09:52:06.980224+0000","last_clean":"2023-08-25T09:52:06.980224+0000","last_became_active":"2023-08-25T09:52:03.613922+0000","last_became_peered":"2023-08-25T09:52:03.613922+0000","last_unstale":"2023-08-25T09:52:06.980224+0000","last_undegraded":"2023-08-25T09:52:06.980224+0000","last_fullsized":"2023-08-25T09:52:06.980224+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:23:10.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595023+0000","last_change":"2023-08-25T09:52:04.605224+0000","last_active":"2023-08-25T09:52:05.595023+0000","last_peered":"2023-08-25T09:52:05.595023+0000","last_clean":"2023-08-25T09:52:05.595023+0000","last_became_active":"2023-08-25T09:52:04.604198+0000","last_became_peered":"2023-08-25T09:52:04.604198+0000","last_unstale":"2023-08-25T09:52:05.595023+0000","last_undegraded":"2023-08-25T09:52:05.595023+0000","last_fullsized":"2023-08-25T09:52:05.595023+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:36:20.783977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.601107+0000","last_change":"2023-08-25T09:52:03.610194+0000","last_active":"2023-08-25T09:52:05.601107+0000","last_peered":"2023-08-25T09:52:05.601107+0000","last_clean":"2023-08-25T09:52:05.601107+0000","last_became_active":"2023-08-25T09:52:03.609009+0000","last_became_peered":"2023-08-25T09:52:03.609009+0000","last_unstale":"2023-08-25T09:52:05.601107+0000","last_undegraded":"2023-08-25T09:52:05.601107+0000","last_fullsized":"2023-08-25T09:52:05.601107+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:44:31.051627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978645+0000","last_change":"2023-08-25T09:52:04.620996+0000","last_active":"2023-08-25T09:52:06.978645+0000","last_peered":"2023-08-25T09:52:06.978645+0000","last_clean":"2023-08-25T09:52:06.978645+0000","last_became_active":"2023-08-25T09:52:04.620726+0000","last_became_peered":"2023-08-25T09:52:04.620726+0000","last_unstale":"2023-08-25T09:52:06.978645+0000","last_undegraded":"2023-08-25T09:52:06.978645+0000","last_fullsized":"2023-08-25T09:52:06.978645+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:45:52.575789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979407+0000","last_change":"2023-08-25T09:52:03.608521+0000","last_active":"2023-08-25T09:52:06.979407+0000","last_peered":"2023-08-25T09:52:06.979407+0000","last_clean":"2023-08-25T09:52:06.979407+0000","last_became_active":"2023-08-25T09:52:03.605881+0000","last_became_peered":"2023-08-25T09:52:03.605881+0000","last_unstale":"2023-08-25T09:52:06.979407+0000","last_undegraded":"2023-08-25T09:52:06.979407+0000","last_fullsized":"2023-08-25T09:52:06.979407+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:12:17.038291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598820+0000","last_change":"2023-08-25T09:52:04.615664+0000","last_active":"2023-08-25T09:52:05.598820+0000","last_peered":"2023-08-25T09:52:05.598820+0000","last_clean":"2023-08-25T09:52:05.598820+0000","last_became_active":"2023-08-25T09:52:04.613567+0000","last_became_peered":"2023-08-25T09:52:04.613567+0000","last_unstale":"2023-08-25T09:52:05.598820+0000","last_undegraded":"2023-08-25T09:52:05.598820+0000","last_fullsized":"2023-08-25T09:52:05.598820+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:33.941978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600289+0000","last_change":"2023-08-25T09:52:03.614418+0000","last_active":"2023-08-25T09:52:05.600289+0000","last_peered":"2023-08-25T09:52:05.600289+0000","last_clean":"2023-08-25T09:52:05.600289+0000","last_became_active":"2023-08-25T09:52:03.614127+0000","last_became_peered":"2023-08-25T09:52:03.614127+0000","last_unstale":"2023-08-25T09:52:05.600289+0000","last_undegraded":"2023-08-25T09:52:05.600289+0000","last_fullsized":"2023-08-25T09:52:05.600289+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:15:31.211948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977137+0000","last_change":"2023-08-25T09:52:04.629117+0000","last_active":"2023-08-25T09:52:06.977137+0000","last_peered":"2023-08-25T09:52:06.977137+0000","last_clean":"2023-08-25T09:52:06.977137+0000","last_became_active":"2023-08-25T09:52:04.626222+0000","last_became_peered":"2023-08-25T09:52:04.626222+0000","last_unstale":"2023-08-25T09:52:06.977137+0000","last_undegraded":"2023-08-25T09:52:06.977137+0000","last_fullsized":"2023-08-25T09:52:06.977137+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:59:10.664615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598694+0000","last_change":"2023-08-25T09:52:03.619038+0000","last_active":"2023-08-25T09:52:05.598694+0000","last_peered":"2023-08-25T09:52:05.598694+0000","last_clean":"2023-08-25T09:52:05.598694+0000","last_became_active":"2023-08-25T09:52:03.618767+0000","last_became_peered":"2023-08-25T09:52:03.618767+0000","last_unstale":"2023-08-25T09:52:05.598694+0000","last_undegraded":"2023-08-25T09:52:05.598694+0000","last_fullsized":"2023-08-25T09:52:05.598694+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:23:33.870463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":479,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977545+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:52:06.977545+0000","last_peered":"2023-08-25T09:52:06.977545+0000","last_clean":"2023-08-25T09:52:06.977545+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:52:06.977545+0000","last_undegraded":"2023-08-25T09:52:06.977545+0000","last_fullsized":"2023-08-25T09:52:06.977545+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:46.367732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595603+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:52:05.595603+0000","last_peered":"2023-08-25T09:52:05.595603+0000","last_clean":"2023-08-25T09:52:05.595603+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:52:05.595603+0000","last_undegraded":"2023-08-25T09:52:05.595603+0000","last_fullsized":"2023-08-25T09:52:05.595603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:03:12.303284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600876+0000","last_change":"2023-08-25T09:52:04.619104+0000","last_active":"2023-08-25T09:52:05.600876+0000","last_peered":"2023-08-25T09:52:05.600876+0000","last_clean":"2023-08-25T09:52:05.600876+0000","last_became_active":"2023-08-25T09:52:04.615804+0000","last_became_peered":"2023-08-25T09:52:04.615804+0000","last_unstale":"2023-08-25T09:52:05.600876+0000","last_undegraded":"2023-08-25T09:52:05.600876+0000","last_fullsized":"2023-08-25T09:52:05.600876+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:26:30.881061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.15","version":"197'3","reported_seq":16,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979257+0000","last_change":"2023-08-25T09:52:03.608811+0000","last_active":"2023-08-25T09:52:06.979257+0000","last_peered":"2023-08-25T09:52:06.979257+0000","last_clean":"2023-08-25T09:52:06.979257+0000","last_became_active":"2023-08-25T09:52:03.608140+0000","last_became_peered":"2023-08-25T09:52:03.608140+0000","last_unstale":"2023-08-25T09:52:06.979257+0000","last_undegraded":"2023-08-25T09:52:06.979257+0000","last_fullsized":"2023-08-25T09:52:06.979257+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:48:39.537165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595573+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:52:05.595573+0000","last_peered":"2023-08-25T09:52:05.595573+0000","last_clean":"2023-08-25T09:52:05.595573+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:52:05.595573+0000","last_undegraded":"2023-08-25T09:52:05.595573+0000","last_fullsized":"2023-08-25T09:52:05.595573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:43:58.928366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600124+0000","last_change":"2023-08-25T09:52:04.616692+0000","last_active":"2023-08-25T09:52:05.600124+0000","last_peered":"2023-08-25T09:52:05.600124+0000","last_clean":"2023-08-25T09:52:05.600124+0000","last_became_active":"2023-08-25T09:52:04.610333+0000","last_became_peered":"2023-08-25T09:52:04.610333+0000","last_unstale":"2023-08-25T09:52:05.600124+0000","last_undegraded":"2023-08-25T09:52:05.600124+0000","last_fullsized":"2023-08-25T09:52:05.600124+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:21.268563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"197'6","reported_seq":19,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.600016+0000","last_change":"2023-08-25T09:52:03.615496+0000","last_active":"2023-08-25T09:52:05.600016+0000","last_peered":"2023-08-25T09:52:05.600016+0000","last_clean":"2023-08-25T09:52:05.600016+0000","last_became_active":"2023-08-25T09:52:03.615224+0000","last_became_peered":"2023-08-25T09:52:03.615224+0000","last_unstale":"2023-08-25T09:52:05.600016+0000","last_undegraded":"2023-08-25T09:52:05.600016+0000","last_fullsized":"2023-08-25T09:52:05.600016+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:12:41.768761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":359,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.602617+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:52:05.602617+0000","last_peered":"2023-08-25T09:52:05.602617+0000","last_clean":"2023-08-25T09:52:05.602617+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:52:05.602617+0000","last_undegraded":"2023-08-25T09:52:05.602617+0000","last_fullsized":"2023-08-25T09:52:05.602617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:48.772597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.979915+0000","last_change":"2023-08-25T09:52:04.609947+0000","last_active":"2023-08-25T09:52:06.979915+0000","last_peered":"2023-08-25T09:52:06.979915+0000","last_clean":"2023-08-25T09:52:06.979915+0000","last_became_active":"2023-08-25T09:52:04.609100+0000","last_became_peered":"2023-08-25T09:52:04.609100+0000","last_unstale":"2023-08-25T09:52:06.979915+0000","last_undegraded":"2023-08-25T09:52:06.979915+0000","last_fullsized":"2023-08-25T09:52:06.979915+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:09:45.487493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.598743+0000","last_change":"2023-08-25T09:52:03.615906+0000","last_active":"2023-08-25T09:52:05.598743+0000","last_peered":"2023-08-25T09:52:05.598743+0000","last_clean":"2023-08-25T09:52:05.598743+0000","last_became_active":"2023-08-25T09:52:03.615475+0000","last_became_peered":"2023-08-25T09:52:03.615475+0000","last_unstale":"2023-08-25T09:52:05.598743+0000","last_undegraded":"2023-08-25T09:52:05.598743+0000","last_fullsized":"2023-08-25T09:52:05.598743+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:46:57.343819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.596068+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:52:05.596068+0000","last_peered":"2023-08-25T09:52:05.596068+0000","last_clean":"2023-08-25T09:52:05.596068+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:52:05.596068+0000","last_undegraded":"2023-08-25T09:52:05.596068+0000","last_fullsized":"2023-08-25T09:52:05.596068+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:33:58.167750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.976617+0000","last_change":"2023-08-25T09:52:04.626470+0000","last_active":"2023-08-25T09:52:06.976617+0000","last_peered":"2023-08-25T09:52:06.976617+0000","last_clean":"2023-08-25T09:52:06.976617+0000","last_became_active":"2023-08-25T09:52:04.625407+0000","last_became_peered":"2023-08-25T09:52:04.625407+0000","last_unstale":"2023-08-25T09:52:06.976617+0000","last_undegraded":"2023-08-25T09:52:06.976617+0000","last_fullsized":"2023-08-25T09:52:06.976617+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:27:19.683074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977016+0000","last_change":"2023-08-25T09:52:03.613108+0000","last_active":"2023-08-25T09:52:06.977016+0000","last_peered":"2023-08-25T09:52:06.977016+0000","last_clean":"2023-08-25T09:52:06.977016+0000","last_became_active":"2023-08-25T09:52:03.612787+0000","last_became_peered":"2023-08-25T09:52:03.612787+0000","last_unstale":"2023-08-25T09:52:06.977016+0000","last_undegraded":"2023-08-25T09:52:06.977016+0000","last_fullsized":"2023-08-25T09:52:06.977016+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:45.967657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":325,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980300+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:52:06.980300+0000","last_peered":"2023-08-25T09:52:06.980300+0000","last_clean":"2023-08-25T09:52:06.980300+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:52:06.980300+0000","last_undegraded":"2023-08-25T09:52:06.980300+0000","last_fullsized":"2023-08-25T09:52:06.980300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T19:52:10.510528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978259+0000","last_change":"2023-08-25T09:52:04.625453+0000","last_active":"2023-08-25T09:52:06.978259+0000","last_peered":"2023-08-25T09:52:06.978259+0000","last_clean":"2023-08-25T09:52:06.978259+0000","last_became_active":"2023-08-25T09:52:04.625181+0000","last_became_peered":"2023-08-25T09:52:04.625181+0000","last_unstale":"2023-08-25T09:52:06.978259+0000","last_undegraded":"2023-08-25T09:52:06.978259+0000","last_fullsized":"2023-08-25T09:52:06.978259+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:21:21.268563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.10","version":"197'8","reported_seq":21,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.602337+0000","last_change":"2023-08-25T09:52:03.615255+0000","last_active":"2023-08-25T09:52:05.602337+0000","last_peered":"2023-08-25T09:52:05.602337+0000","last_clean":"2023-08-25T09:52:05.602337+0000","last_became_active":"2023-08-25T09:52:03.613680+0000","last_became_peered":"2023-08-25T09:52:03.613680+0000","last_unstale":"2023-08-25T09:52:05.602337+0000","last_undegraded":"2023-08-25T09:52:05.602337+0000","last_fullsized":"2023-08-25T09:52:05.602337+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:05:27.088191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595232+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:52:05.595232+0000","last_peered":"2023-08-25T09:52:05.595232+0000","last_clean":"2023-08-25T09:52:05.595232+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:52:05.595232+0000","last_undegraded":"2023-08-25T09:52:05.595232+0000","last_fullsized":"2023-08-25T09:52:05.595232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:27:32.237945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.977164+0000","last_change":"2023-08-25T09:52:04.628438+0000","last_active":"2023-08-25T09:52:06.977164+0000","last_peered":"2023-08-25T09:52:06.977164+0000","last_clean":"2023-08-25T09:52:06.977164+0000","last_became_active":"2023-08-25T09:52:04.627892+0000","last_became_peered":"2023-08-25T09:52:04.627892+0000","last_unstale":"2023-08-25T09:52:06.977164+0000","last_undegraded":"2023-08-25T09:52:06.977164+0000","last_fullsized":"2023-08-25T09:52:06.977164+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:04:45.280799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.13","version":"197'2","reported_seq":15,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.980132+0000","last_change":"2023-08-25T09:52:03.608367+0000","last_active":"2023-08-25T09:52:06.980132+0000","last_peered":"2023-08-25T09:52:06.980132+0000","last_clean":"2023-08-25T09:52:06.980132+0000","last_became_active":"2023-08-25T09:52:03.605716+0000","last_became_peered":"2023-08-25T09:52:03.605716+0000","last_unstale":"2023-08-25T09:52:06.980132+0000","last_undegraded":"2023-08-25T09:52:06.980132+0000","last_fullsized":"2023-08-25T09:52:06.980132+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:34:45.112608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":325,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:06.978871+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:52:06.978871+0000","last_peered":"2023-08-25T09:52:06.978871+0000","last_clean":"2023-08-25T09:52:06.978871+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:52:06.978871+0000","last_undegraded":"2023-08-25T09:52:06.978871+0000","last_fullsized":"2023-08-25T09:52:06.978871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:55:05.193960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":11,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599626+0000","last_change":"2023-08-25T09:52:04.616992+0000","last_active":"2023-08-25T09:52:05.599626+0000","last_peered":"2023-08-25T09:52:05.599626+0000","last_clean":"2023-08-25T09:52:05.599626+0000","last_became_active":"2023-08-25T09:52:04.613623+0000","last_became_peered":"2023-08-25T09:52:04.613623+0000","last_unstale":"2023-08-25T09:52:05.599626+0000","last_undegraded":"2023-08-25T09:52:05.599626+0000","last_fullsized":"2023-08-25T09:52:05.599626+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":196,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:03.577494+0000","last_clean_scrub_stamp":"2023-08-25T09:52:03.577494+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:58:27.441433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.599106+0000","last_change":"2023-08-25T09:52:03.611964+0000","last_active":"2023-08-25T09:52:05.599106+0000","last_peered":"2023-08-25T09:52:05.599106+0000","last_clean":"2023-08-25T09:52:05.599106+0000","last_became_active":"2023-08-25T09:52:03.611724+0000","last_became_peered":"2023-08-25T09:52:03.611724+0000","last_unstale":"2023-08-25T09:52:05.599106+0000","last_undegraded":"2023-08-25T09:52:05.599106+0000","last_fullsized":"2023-08-25T09:52:05.599106+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:52:02.568250+0000","last_clean_scrub_stamp":"2023-08-25T09:52:02.568250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:54:31.754265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":357,"reported_epoch":198,"state":"active+clean","last_fresh":"2023-08-25T09:52:05.595886+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:52:05.595886+0000","last_peered":"2023-08-25T09:52:05.595886+0000","last_clean":"2023-08-25T09:52:05.595886+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:52:05.595886+0000","last_undegraded":"2023-08-25T09:52:05.595886+0000","last_fullsized":"2023-08-25T09:52:05.595886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:56.976887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198806588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.35699999999999998},"max":{"1min":1.4930000000000001,"5min":1.829,"15min":10.747},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.173,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.43099999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.855},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.6970000000000001,"5min":1.823,"15min":11.010999999999999},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.129,"5min":1.2589999999999999,"15min":1.1819999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.496,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":1.2789999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.206,"15min":1.147},"min":{"1min":0.86699999999999999,"5min":0.79400000000000004,"15min":0.50600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5529999999999999,"15min":1.619},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3180000000000001,"15min":1.2769999999999999},"min":{"1min":0.93999999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6779999999999999,"5min":1.873,"15min":1.98},"last":1.6379999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.258,"15min":1.2190000000000001},"min":{"1min":0.95599999999999996,"5min":0.79300000000000004,"15min":0.56799999999999995},"max":{"1min":1.506,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.224},{"interface":"front","average":{"1min":1.359,"5min":1.393,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.69299999999999995},"max":{"1min":1.778,"5min":1.833,"15min":2.1030000000000002},"last":1.5620000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.264,"15min":1.288},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.48599999999999999},"max":{"1min":1.732,"5min":1.845,"15min":7.1639999999999997},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.306,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.46300000000000002},"max":{"1min":1.7190000000000001,"5min":1.798,"15min":1.8779999999999999},"last":0.999}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.52700000000000002},"max":{"1min":1.714,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.331},{"interface":"front","average":{"1min":1.361,"5min":1.4450000000000001,"15min":1.429},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.64100000000000001},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":2.0739999999999998},"last":1.04}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5580000000000001,"15min":1.518},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.68799999999999994},"max":{"1min":1.79,"5min":1.984,"15min":2.1720000000000002},"last":1.49},{"interface":"front","average":{"1min":1.448,"5min":1.4870000000000001,"15min":1.462},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.5189999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1165,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198806588,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.161,"15min":1.1459999999999999},"min":{"1min":0.70499999999999996,"5min":0.68700000000000006,"15min":0.54400000000000004},"max":{"1min":1.7310000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.25,"5min":1.198,"15min":1.175},"min":{"1min":0.84199999999999997,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.627,"5min":1.645,"15min":5.5410000000000004},"last":0.997}]},{"osd":1,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.2529999999999999,"15min":1.2210000000000001},"min":{"1min":0.82799999999999996,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6930000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.194,"15min":1.179},"min":{"1min":0.75800000000000001,"5min":0.66800000000000004,"15min":0.50800000000000001},"max":{"1min":1.5529999999999999,"5min":1.909,"15min":2.3079999999999998},"last":1.1259999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1839999999999999,"15min":1.153},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.708},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3220000000000001,"15min":1.3},"min":{"1min":0.91300000000000003,"5min":0.67000000000000004,"15min":0.47299999999999998},"max":{"1min":1.7310000000000001,"5min":1.7969999999999999,"15min":13.193},"last":1.458}]},{"osd":3,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.359,"15min":1.3100000000000001},"min":{"1min":1.0620000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.861,"5min":1.911,"15min":5.6239999999999997},"last":1.861},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.286,"15min":1.27},"min":{"1min":0.995,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8200000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5880000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.325,"15min":1.3169999999999999},"min":{"1min":1.048,"5min":0.55300000000000005,"15min":0.42599999999999999},"max":{"1min":1.867,"5min":1.867,"15min":2.484},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.405,"15min":1.381},"min":{"1min":0.99199999999999999,"5min":0.70099999999999996,"15min":0.58799999999999997},"max":{"1min":1.9650000000000001,"5min":2.101,"15min":5.4699999999999998},"last":1.9650000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.359,"15min":1.3260000000000001},"min":{"1min":0.95099999999999996,"5min":0.74199999999999999,"15min":0.59599999999999997},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.802},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.468,"15min":1.454},"min":{"1min":1,"5min":0.61099999999999999,"15min":0.59099999999999997},"max":{"1min":1.786,"5min":1.994,"15min":2.4420000000000002},"last":1.7509999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:04 2023","interfaces":[{"interface":"back","average":{"1min":1.601,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":1.179,"5min":0.80000000000000004,"15min":0.622},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":6.7709999999999999},"last":2.0329999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5169999999999999,"15min":1.498},"min":{"1min":1.105,"5min":0.85899999999999999,"15min":0.57399999999999995},"max":{"1min":2.0590000000000002,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.921}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1741,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193571,"statfs":{"total":200000000,"available":198217146,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2669999999999999,"15min":1.2050000000000001},"min":{"1min":0.751,"5min":0.71999999999999997,"15min":0.497},"max":{"1min":1.7210000000000001,"5min":1.9770000000000001,"15min":2.081},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.252,"15min":1.2210000000000001},"min":{"1min":0.88800000000000001,"5min":0.747,"15min":0.65400000000000003},"max":{"1min":1.597,"5min":1.8080000000000001,"15min":1.974},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1639999999999999,"15min":1.165},"min":{"1min":0.69799999999999995,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.6399999999999999,"5min":1.8680000000000001,"15min":2.1779999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.208,"5min":1.1719999999999999,"15min":1.161},"min":{"1min":0.84199999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.571,"5min":1.704,"15min":2.1869999999999998},"last":1.4990000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3200000000000001,"15min":1.252},"min":{"1min":0.91700000000000004,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.8149999999999999,"5min":1.9810000000000001,"15min":2.3690000000000002},"last":1.331},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2729999999999999,"15min":1.252},"min":{"1min":0.83099999999999996,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.5449999999999999,"5min":1.847,"15min":1.962},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.2589999999999999,"15min":1.288},"min":{"1min":0.86299999999999999,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.71,"5min":1.8109999999999999,"15min":17.356999999999999},"last":1.121},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3500000000000001,"15min":1.325},"min":{"1min":0.95299999999999996,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.7330000000000001,"5min":1.964,"15min":1.9650000000000001},"last":1.569}]},{"osd":4,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.306,"15min":1.3149999999999999},"min":{"1min":0.83199999999999996,"5min":0.80800000000000005,"15min":0.55500000000000005},"max":{"1min":1.76,"5min":1.9059999999999999,"15min":2.625},"last":1.637},{"interface":"front","average":{"1min":1.304,"5min":1.3160000000000001,"15min":1.3360000000000001},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.58799999999999997},"max":{"1min":1.7150000000000001,"5min":1.913,"15min":2.2789999999999999},"last":1.2709999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.462,"15min":1.444},"min":{"1min":1.0569999999999999,"5min":0.95099999999999996,"15min":0.58899999999999997},"max":{"1min":1.76,"5min":2.028,"15min":2.4820000000000002},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.5249999999999999,"15min":1.486},"min":{"1min":1.0840000000000001,"5min":0.96899999999999997,"15min":0.60399999999999998},"max":{"1min":1.9039999999999999,"5min":2.044,"15min":2.5539999999999998},"last":1.7549999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4950000000000001,"15min":1.464},"min":{"1min":1.0129999999999999,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.585,"15min":1.579},"min":{"1min":1.2869999999999999,"5min":1.0289999999999999,"15min":0.63900000000000001},"max":{"1min":1.923,"5min":2.1040000000000001,"15min":2.3700000000000001},"last":1.7929999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808178,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.28,"15min":1.224},"min":{"1min":0.624,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.589,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.2709999999999999,"15min":1.2490000000000001},"min":{"1min":0.91600000000000004,"5min":0.434,"15min":0.434},"max":{"1min":1.615,"5min":2.5150000000000001,"15min":7.367},"last":1.3859999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.1379999999999999,"15min":1.1379999999999999},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.466,"5min":1.7330000000000001,"15min":2.032},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.234,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.571,"5min":4.351,"15min":8.5850000000000009},"last":1.3}]},{"osd":2,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.214},"min":{"1min":0.72599999999999998,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.556,"5min":1.8859999999999999,"15min":4.1669999999999998},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.294,"15min":1.256},"min":{"1min":0.71399999999999997,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.5940000000000001,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.04}]},{"osd":3,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.284,"15min":1.2629999999999999},"min":{"1min":0.79400000000000004,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.665,"5min":2.5680000000000001,"15min":7.2560000000000002},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.377,"15min":1.3779999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.911},"last":1.4810000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3140000000000001,"15min":1.3009999999999999},"min":{"1min":0.67100000000000004,"5min":0.59399999999999997,"15min":0.55200000000000005},"max":{"1min":1.7370000000000001,"5min":5.048,"15min":5.048},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3089999999999999,"15min":1.304},"min":{"1min":0.67500000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.71,"5min":2.6469999999999998,"15min":8.2409999999999997},"last":1.1419999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4179999999999999,"15min":1.4319999999999999},"min":{"1min":0.88100000000000001,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":1.927,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.597,"15min":1.579},"min":{"1min":0.69799999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.968,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4430000000000001,"5min":1.496,"15min":1.49},"min":{"1min":0.75700000000000001,"5min":0.65500000000000003,"15min":0.63800000000000001},"max":{"1min":1.865,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.569,"15min":1.5920000000000001},"min":{"1min":0.85799999999999998,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.8340000000000001,"5min":3.831,"15min":3.831},"last":1.097}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808166,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.98599999999999999,"15min":0.91000000000000003},"min":{"1min":0.628,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.8180000000000001,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.0649999999999999,"15min":0.98199999999999998},"min":{"1min":0.69999999999999996,"5min":0.45000000000000001,"15min":0.41699999999999998},"max":{"1min":1.579,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.94299999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.88100000000000001,"15min":0.85699999999999998},"min":{"1min":0.64600000000000002,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.258,"5min":1.413,"15min":1.702},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.087,"15min":1.0169999999999999},"min":{"1min":0.65000000000000002,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.623,"5min":1.6339999999999999,"15min":2.3399999999999999},"last":1.27}]},{"osd":2,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.034,"15min":0.96499999999999997},"min":{"1min":0.78600000000000003,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.5980000000000001,"5min":1.623,"15min":1.919},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2050000000000001,"15min":1.1439999999999999},"min":{"1min":0.745,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.901,"5min":1.901,"15min":3.399},"last":1.5249999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3009999999999999,"15min":1.236},"min":{"1min":1.1399999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.669,"5min":1.7330000000000001,"15min":2.1949999999999998},"last":1.395},{"interface":"front","average":{"1min":1.198,"5min":1.204,"15min":1.2090000000000001},"min":{"1min":1.0109999999999999,"5min":0.629,"15min":0.629},"max":{"1min":1.405,"5min":1.79,"15min":2.1840000000000002},"last":1.6539999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.405,"15min":1.3480000000000001},"min":{"1min":1.137,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":2.1509999999999998,"5min":2.1930000000000001,"15min":2.407},"last":1.764},{"interface":"front","average":{"1min":1.357,"5min":1.3500000000000001,"15min":1.304},"min":{"1min":1.0349999999999999,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.804,"5min":1.895,"15min":1.895},"last":1.806}]},{"osd":6,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.302,"15min":1.282},"min":{"1min":1.054,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5980000000000001,"5min":1.875,"15min":2.3450000000000002},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.47,"15min":1.4199999999999999},"min":{"1min":1.28,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":2.2330000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.9830000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.464,"15min":1.405},"min":{"1min":1.0840000000000001,"5min":0.871,"15min":0.83699999999999997},"max":{"1min":1.976,"5min":1.976,"15min":2.2989999999999999},"last":1.9390000000000001},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4379999999999999,"15min":1.3839999999999999},"min":{"1min":1.1259999999999999,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":1.7589999999999999,"5min":1.9350000000000001,"15min":2.1509999999999998},"last":2.0139999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1164,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194147,"statfs":{"total":200000000,"available":198807436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.85799999999999998,"15min":0.84599999999999997},"min":{"1min":0.64900000000000002,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.51,"5min":1.6930000000000001,"15min":2.383},"last":0.753},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.93300000000000005,"15min":0.96299999999999997},"min":{"1min":0.47599999999999998,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":1.3600000000000001,"5min":1.625,"15min":2.7309999999999999},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.97599999999999998,"15min":0.98499999999999999},"min":{"1min":0.66400000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3009999999999999,"5min":1.635,"15min":2.3180000000000001},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.0429999999999999,"15min":1.0389999999999999},"min":{"1min":0.56699999999999995,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.5800000000000001,"5min":1.704,"15min":2.8599999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93100000000000005,"15min":1.0429999999999999},"min":{"1min":0.61599999999999999,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.294,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.121,"15min":1.101},"min":{"1min":0.84699999999999998,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.587,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.99199999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3080000000000001,"15min":1.321},"min":{"1min":0.94099999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":1.194},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":0.91200000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6459999999999999,"5min":1.8,"15min":2.7639999999999998},"last":1.2450000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.2470000000000001,"15min":1.2809999999999999},"min":{"1min":0.89000000000000001,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.534,"5min":1.849,"15min":5.1269999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.359,"5min":1.371,"15min":1.3680000000000001},"min":{"1min":1.1020000000000001,"5min":0.625,"15min":0.625},"max":{"1min":1.605,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":1.2649999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.397,"15min":1.403},"min":{"1min":1.1060000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.7769999999999999,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.393,"15min":1.417},"min":{"1min":1.0109999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.617,"5min":2.028,"15min":5.0599999999999996},"last":1.0800000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:52:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.373,"15min":1.395},"min":{"1min":1.1699999999999999,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.7210000000000001,"5min":2,"15min":2.968},"last":0.872},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.3779999999999999,"15min":1.4239999999999999},"min":{"1min":1.071,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.569,"5min":1.96,"15min":5.2990000000000004},"last":0.96699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1740,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193572,"statfs":{"total":200000000,"available":198217742,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.82699999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.29699999999999999},"max":{"1min":1.4690000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.97899999999999998,"15min":0.89800000000000002},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.315},"max":{"1min":1.7869999999999999,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.98499999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.94099999999999995,"15min":0.88100000000000001},"min":{"1min":0.51100000000000001,"5min":0.39800000000000002,"15min":0.35899999999999999},"max":{"1min":1.2809999999999999,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.022,"15min":0.97499999999999998},"min":{"1min":0.53400000000000003,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.389,"5min":1.6619999999999999,"15min":1.746},"last":1.139}]},{"osd":3,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.147,"15min":1.109},"min":{"1min":0.496,"5min":0.496,"15min":0.372},"max":{"1min":1.647,"5min":1.647,"15min":1.9450000000000001},"last":1.278},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0409999999999999,"15min":1.0109999999999999},"min":{"1min":0.41799999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.333,"5min":1.655,"15min":1.764},"last":1.381}]},{"osd":4,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1779999999999999,"15min":1.1459999999999999},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.488,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.254,"5min":1.2929999999999999,"15min":1.276},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":1.6830000000000001,"5min":1.9419999999999999,"15min":2.008},"last":1.351}]},{"osd":5,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2829999999999999,"15min":1.2430000000000001},"min":{"1min":0.94699999999999995,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5269999999999999,"5min":1.665,"15min":1.9650000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.3049999999999999,"15min":1.2450000000000001},"min":{"1min":0.82699999999999996,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.677,"5min":2.867,"15min":2.867},"last":1.647}]},{"osd":6,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.375,"15min":1.363},"min":{"1min":1.016,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":1.633,"5min":2.0779999999999998,"15min":2.9940000000000002},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.411,"15min":1.3660000000000001},"min":{"1min":0.95099999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":1.603,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.5629999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4490000000000001,"15min":1.4139999999999999},"min":{"1min":0.871,"5min":0.871,"15min":0.79200000000000004},"max":{"1min":1.772,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.375,"5min":1.431,"15min":1.399},"min":{"1min":0.88200000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.738,"5min":1.913,"15min":2.218},"last":1.4950000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575013,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1163,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194148,"statfs":{"total":200000000,"available":198808200,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.313},"max":{"1min":1.4450000000000001,"5min":2.2759999999999998,"15min":11.257},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.98499999999999999,"15min":0.96799999999999997},"min":{"1min":0.60199999999999998,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.4730000000000001,"5min":1.762,"15min":6.4260000000000002},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.96099999999999997},"min":{"1min":0.72399999999999998,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.343,"5min":1.522,"15min":1.8620000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":1.048},"min":{"1min":0.62,"5min":0.374,"15min":0.374},"max":{"1min":1.401,"5min":1.714,"15min":3.4889999999999999},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.98499999999999999,"15min":0.93000000000000005},"min":{"1min":0.65900000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.464,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.69199999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.629,"15min":2.3809999999999998},"last":1.4690000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.294,"15min":1.2569999999999999},"min":{"1min":0.89800000000000002,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.724,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.097,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.379}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.88800000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.919,"5min":1.919,"15min":3.3769999999999998},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.2769999999999999},"min":{"1min":1.038,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6439999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.573}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4219999999999999,"15min":1.3640000000000001},"min":{"1min":0.92400000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7370000000000001,"5min":2.056,"15min":2.6869999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.3759999999999999,"15min":1.335},"min":{"1min":0.999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7709999999999999,"5min":1.863,"15min":2.331},"last":1.508}]},{"osd":7,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.706,"5min":1.962,"15min":2.6429999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.389,"15min":1.3440000000000001},"min":{"1min":0.95699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.804,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.423}]}]}],"pool_statfs":[]}} 2023-08-25T09:52:14.573 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T09:52:14.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T09:52:14.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.840+0000 7efd91863640 1 Processor -- start 2023-08-25T09:52:14.842 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.840+0000 7efd91863640 1 -- start start 2023-08-25T09:52:14.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.841+0000 7efd91863640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd8c159650 0x7efd8c15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.841+0000 7efd91863640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c0ccee0 0x7efd8c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd91863640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0acda0 con 0x7efd8c0b0300 2023-08-25T09:52:14.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd91863640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c0cd650 con 0x7efd8c159650 2023-08-25T09:52:14.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd91863640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0ac1d0 con 0x7efd8c0ccee0 2023-08-25T09:52:14.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd90861640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7efd8c159650 0x7efd8c15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd91062640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efd8c0b0300 0x7efd8c0c89b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45802/0 (socket says 172.21.15.70:45802) 2023-08-25T09:52:14.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd91062640 1 -- 172.21.15.70:0/3825943130 learned_addr learned my addr 172.21.15.70:0/3825943130 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:14.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd8bfff640 1 --2- 172.21.15.70:0/3825943130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c0ccee0 0x7efd8c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd90861640 1 -- 172.21.15.70:0/3825943130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c0ccee0 msgr2=0x7efd8c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:14.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd90861640 1 --2- 172.21.15.70:0/3825943130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c0ccee0 0x7efd8c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd90861640 1 -- 172.21.15.70:0/3825943130 >> v1:172.21.15.70:6789/0 conn(0x7efd8c0b0300 legacy=0x7efd8c0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd90861640 1 -- 172.21.15.70:0/3825943130 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0ac470 con 0x7efd8c159650 2023-08-25T09:52:14.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.842+0000 7efd8b7fe640 1 -- 172.21.15.70:0/3825943130 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164326227 0 0) 0x7efd8c0acda0 con 0x7efd8c0b0300 2023-08-25T09:52:14.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.844+0000 7efd90861640 1 --2- 172.21.15.70:0/3825943130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c159650 0x7efd8c15dd90 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7efd80009790 tx=0x7efd8002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da4051b20080383 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:14.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.844+0000 7efd8b7fe640 1 -- 172.21.15.70:0/3825943130 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80031800 con 0x7efd8c159650 2023-08-25T09:52:14.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.844+0000 7efd8b7fe640 1 -- 172.21.15.70:0/3825943130 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd80031960 con 0x7efd8c159650 2023-08-25T09:52:14.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.844+0000 7efd8b7fe640 1 -- 172.21.15.70:0/3825943130 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80031cf0 con 0x7efd8c159650 2023-08-25T09:52:14.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.844+0000 7efd91863640 1 -- 172.21.15.70:0/3825943130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c159650 msgr2=0x7efd8c15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.844+0000 7efd91863640 1 --2- 172.21.15.70:0/3825943130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c159650 0x7efd8c15dd90 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7efd80009790 tx=0x7efd8002b8a0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.845+0000 7efd91863640 1 -- 172.21.15.70:0/3825943130 shutdown_connections 2023-08-25T09:52:14.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.845+0000 7efd91863640 1 --2- 172.21.15.70:0/3825943130 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c0ccee0 0x7efd8c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.845+0000 7efd91863640 1 --2- 172.21.15.70:0/3825943130 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c159650 0x7efd8c15dd90 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.845+0000 7efd91863640 1 -- 172.21.15.70:0/3825943130 >> 172.21.15.70:0/3825943130 conn(0x7efd8c0ce810 msgr2=0x7efd8c0cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.845+0000 7efd91863640 1 -- 172.21.15.70:0/3825943130 shutdown_connections 2023-08-25T09:52:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.845+0000 7efd91863640 1 -- 172.21.15.70:0/3825943130 wait complete. 2023-08-25T09:52:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.846+0000 7efd91863640 1 Processor -- start 2023-08-25T09:52:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.846+0000 7efd91863640 1 -- start start 2023-08-25T09:52:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.847+0000 7efd91863640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 0x7efd8c0c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.847+0000 7efd91863640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c0ccee0 0x7efd8c097940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.847+0000 7efd91863640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c159650 0x7efd8c070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.847+0000 7efd91863640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c097e80 con 0x7efd8c0b0300 2023-08-25T09:52:14.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.847+0000 7efd91863640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c15c750 con 0x7efd8c0ccee0 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.847+0000 7efd91863640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c06be20 con 0x7efd8c159650 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 0x7efd8c0c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 0x7efd8c0c1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43130/0 (socket says 172.21.15.70:43130) 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 -- 172.21.15.70:0/3003645270 learned_addr learned my addr 172.21.15.70:0/3003645270 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 -- 172.21.15.70:0/3003645270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c159650 msgr2=0x7efd8c070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c159650 0x7efd8c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 -- 172.21.15.70:0/3003645270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c0ccee0 msgr2=0x7efd8c097940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:14.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c0ccee0 0x7efd8c097940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:14.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 -- 172.21.15.70:0/3003645270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800093a0 con 0x7efd8c0b0300 2023-08-25T09:52:14.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.848+0000 7efd90861640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 0x7efd8c0c1160 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7efd80034040 tx=0x7efd800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:14.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.849+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80005e60 con 0x7efd8c0b0300 2023-08-25T09:52:14.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.849+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8003a850 con 0x7efd8c0b0300 2023-08-25T09:52:14.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.849+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8003ac00 con 0x7efd8c0b0300 2023-08-25T09:52:14.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.849+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c06c100 con 0x7efd8c0b0300 2023-08-25T09:52:14.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.849+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c06baf0 con 0x7efd8c0b0300 2023-08-25T09:52:14.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.850+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c06c620 con 0x7efd8c0b0300 2023-08-25T09:52:14.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.851+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efd80006030 con 0x7efd8c0b0300 2023-08-25T09:52:14.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.852+0000 7efd897fa640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd7006a050 0x7efd7006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:14.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.852+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd8002e070 con 0x7efd8c0b0300 2023-08-25T09:52:14.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.852+0000 7efd8bfff640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd7006a050 0x7efd7006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:14.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.853+0000 7efd8bfff640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd7006a050 0x7efd7006c510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0ccbc0 tx=0x7efd78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:14.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:14.856+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efd8c06c620 con 0x7efd8c0b0300 2023-08-25T09:52:15.123 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.122+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7efd8c06c620 con 0x7efd8c0b0300 2023-08-25T09:52:15.129 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.128+0000 7efd897fa640 1 -- 172.21.15.70:0/3003645270 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7efd8c06c620 con 0x7efd8c0b0300 2023-08-25T09:52:15.130 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2023-08-25T09:52:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQCKduhke1cmAhAAb9LHdmcAWtPbtzWCfnaDYg== 2023-08-25T09:52:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: caps mds = "allow" 2023-08-25T09:52:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: caps mon = "allow r" 2023-08-25T09:52:15.130 INFO:teuthology.orchestra.run.smithi070.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T09:52:15.131 INFO:teuthology.orchestra.run.smithi070.stderr:updated caps for client.0 2023-08-25T09:52:15.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd7006a050 msgr2=0x7efd7006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd7006a050 0x7efd7006c510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0ccbc0 tx=0x7efd78008040 comp rx=0 tx=0).stop 2023-08-25T09:52:15.132 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 msgr2=0x7efd8c0c1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.133 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 0x7efd8c0c1160 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7efd80034040 tx=0x7efd800046c0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 shutdown_connections 2023-08-25T09:52:15.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efd7006a050 0x7efd7006c510 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efd8c159650 0x7efd8c070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efd8c0ccee0 0x7efd8c097940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.134 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 --2- 172.21.15.70:0/3003645270 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efd8c0b0300 0x7efd8c0c1160 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.135 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.131+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 >> 172.21.15.70:0/3003645270 conn(0x7efd8c0ce810 msgr2=0x7efd8c0b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:15.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.132+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 shutdown_connections 2023-08-25T09:52:15.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.132+0000 7efd91863640 1 -- 172.21.15.70:0/3003645270 wait complete. 2023-08-25T09:52:15.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:52:15.382 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.381+0000 7fa3f656d640 1 Processor -- start 2023-08-25T09:52:15.383 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.381+0000 7fa3f656d640 1 -- start start 2023-08-25T09:52:15.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.382+0000 7fa3f656d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3f00b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.382+0000 7fa3f656d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cadb0 0x7fa3f00cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.382+0000 7fa3f656d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f00b2aa0 con 0x7fa3f0159840 2023-08-25T09:52:15.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3f656d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f00c8b60 con 0x7fa3f00cadb0 2023-08-25T09:52:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3f656d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f015d890 con 0x7fa3f015d3d0 2023-08-25T09:52:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3f556b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa3f0159840 0x7fa3f00b2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45812/0 (socket says 172.21.15.70:45812) 2023-08-25T09:52:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3f556b640 1 -- 172.21.15.70:0/2283525080 learned_addr learned my addr 172.21.15.70:0/2283525080 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050914346 0 0) 0x7fa3f00b2aa0 con 0x7fa3f0159840 2023-08-25T09:52:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3dc005730 con 0x7fa3f0159840 2023-08-25T09:52:15.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3f4d6a640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3f00b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.383+0000 7fa3f5d6c640 1 --2- 172.21.15.70:0/2283525080 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cadb0 0x7fa3f00cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615359555 0 0) 0x7fa3dc005730 con 0x7fa3f0159840 2023-08-25T09:52:15.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 msgr2=0x7fa3f00b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:15.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3f00b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cadb0 msgr2=0x7fa3f00cb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:15.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 --2- 172.21.15.70:0/2283525080 >> v2:172.21.15.155:3300/0 conn(0x7fa3f00cadb0 0x7fa3f00cb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f015db70 con 0x7fa3f0159840 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fa3ec001670 con 0x7fa3f0159840 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 >> v1:172.21.15.70:6789/0 conn(0x7fa3f0159840 legacy=0x7fa3f00b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc005b80 0x7fa3dc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3dc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3dc006f10 0x7fa3dc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00c8b60 con 0x7fa3dc006f10 2023-08-25T09:52:15.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f015d890 con 0x7fa3dc005b80 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc00dbe0 con 0x7fa3f015d3d0 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.384+0000 7fa3f5d6c640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3dc006f10 0x7fa3dc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f556b640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc005b80 0x7fa3dc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f4d6a640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3dc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f5d6c640 1 -- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 msgr2=0x7fa3dc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f5d6c640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3dc0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f5d6c640 1 -- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc005b80 msgr2=0x7fa3dc005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f5d6c640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3dc005b80 0x7fa3dc005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f5d6c640 1 -- 172.21.15.70:0/2283525080 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc005730 con 0x7fa3dc006f10 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.385+0000 7fa3f4d6a640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3f015d3d0 0x7fa3dc0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.386+0000 7fa3f5d6c640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3dc006f10 0x7fa3dc007390 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8007360 tx=0x7fa3dc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2bc873203b6c7d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.386+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e802ccf0 con 0x7fa3dc006f10 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.386+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e802ce50 con 0x7fa3dc006f10 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.386+0000 7fa3e7fff640 1 -- 172.21.15.70:0/2283525080 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8034660 con 0x7fa3dc006f10 2023-08-25T09:52:15.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f656d640 1 -- 172.21.15.70:0/2283525080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3dc006f10 msgr2=0x7fa3dc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f656d640 1 --2- 172.21.15.70:0/2283525080 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3dc006f10 0x7fa3dc007390 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8007360 tx=0x7fa3dc007a00 comp rx=0 tx=0).stop 2023-08-25T09:52:15.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f5d6c640 1 -- 172.21.15.70:0/2283525080 reap_dead start 2023-08-25T09:52:15.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f656d640 1 -- 172.21.15.70:0/2283525080 shutdown_connections 2023-08-25T09:52:15.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f656d640 1 -- 172.21.15.70:0/2283525080 >> 172.21.15.70:0/2283525080 conn(0x7fa3f00affd0 msgr2=0x7fa3f00b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:15.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f656d640 1 -- 172.21.15.70:0/2283525080 shutdown_connections 2023-08-25T09:52:15.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.387+0000 7fa3f656d640 1 -- 172.21.15.70:0/2283525080 wait complete. 2023-08-25T09:52:15.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.388+0000 7fa3f656d640 1 Processor -- start 2023-08-25T09:52:15.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.389+0000 7fa3f656d640 1 -- start start 2023-08-25T09:52:15.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f656d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3dc006f10 0x7fa3f0155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f656d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 0x7fa3f014df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f656d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f015bf90 0x7fa3f015c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f656d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f015c950 con 0x7fa3f006fc20 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f656d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00700a0 con 0x7fa3f015bf90 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f656d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0070350 con 0x7fa3dc006f10 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 0x7fa3f014df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 0x7fa3f014df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43140/0 (socket says 172.21.15.70:43140) 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 -- 172.21.15.70:0/3177758675 learned_addr learned my addr 172.21.15.70:0/3177758675 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 -- 172.21.15.70:0/3177758675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3dc006f10 msgr2=0x7fa3f0155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3dc006f10 0x7fa3f0155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 -- 172.21.15.70:0/3177758675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f015bf90 msgr2=0x7fa3f015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f015bf90 0x7fa3f015c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.390+0000 7fa3f4d6a640 1 -- 172.21.15.70:0/3177758675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0070630 con 0x7fa3f006fc20 2023-08-25T09:52:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.391+0000 7fa3f4d6a640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 0x7fa3f014df60 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00108b0 tx=0x7fa3e0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.391+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000eae0 con 0x7fa3f006fc20 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.391+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e000ec40 con 0x7fa3f006fc20 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.391+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001c400 con 0x7fa3f006fc20 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.391+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f0070910 con 0x7fa3f006fc20 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.391+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f014e4a0 con 0x7fa3f006fc20 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.392+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f0070e30 con 0x7fa3f006fc20 2023-08-25T09:52:15.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.393+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa3e001c560 con 0x7fa3f006fc20 2023-08-25T09:52:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.394+0000 7fa3e5ffb640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3d406a050 0x7fa3d406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.394+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa3e0088bc0 con 0x7fa3f006fc20 2023-08-25T09:52:15.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.394+0000 7fa3f556b640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3d406a050 0x7fa3d406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:15.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.395+0000 7fa3f556b640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3d406a050 0x7fa3d406c510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec000f60 tx=0x7fa3ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:15.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.399+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa3e0057d50 con 0x7fa3f006fc20 2023-08-25T09:52:15.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.623+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f0066fb0 con 0x7fa3f006fc20 2023-08-25T09:52:15.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.625+0000 7fa3e5ffb640 1 -- 172.21.15.70:0/3177758675 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fa3e00578e0 con 0x7fa3f006fc20 2023-08-25T09:52:15.626 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:15.627 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:52:15.627 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:52:15.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3d406a050 msgr2=0x7fa3d406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3d406a050 0x7fa3d406c510 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec000f60 tx=0x7fa3ec009040 comp rx=0 tx=0).stop 2023-08-25T09:52:15.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 msgr2=0x7fa3f014df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:15.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 0x7fa3f014df60 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00108b0 tx=0x7fa3e0010d70 comp rx=0 tx=0).stop 2023-08-25T09:52:15.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 shutdown_connections 2023-08-25T09:52:15.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa3d406a050 0x7fa3d406c510 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3f015bf90 0x7fa3f015c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3f006fc20 0x7fa3f014df60 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 --2- 172.21.15.70:0/3177758675 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa3dc006f10 0x7fa3f0155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:15.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 >> 172.21.15.70:0/3177758675 conn(0x7fa3f00affd0 msgr2=0x7fa3f00cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:15.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 shutdown_connections 2023-08-25T09:52:15.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:15.627+0000 7fa3f656d640 1 -- 172.21.15.70:0/3177758675 wait complete. 2023-08-25T09:52:16.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T09:52:04.089417+0000', 'modified': '2023-08-25T09:52:07.665370+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9573': {'gid': 9573, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.70:6836/771430984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6834', 'nonce': 771430984}, {'type': 'v1', 'addr': '172.21.15.70:6836', 'nonce': 771430984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T09:52:16.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T09:52:16.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T09:52:16.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.275+0000 7f2b154f9640 1 Processor -- start 2023-08-25T09:52:16.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.275+0000 7f2b154f9640 1 -- start start 2023-08-25T09:52:16.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.276+0000 7f2b154f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 0x7f2b100cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.276+0000 7f2b154f9640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b100b12b0 con 0x7f2b100ca780 2023-08-25T09:52:16.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.276+0000 7f2b154f9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1006ae20 con 0x7f2b1015acd0 2023-08-25T09:52:16.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.276+0000 7f2b154f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100b47e0 con 0x7f2b100b4400 2023-08-25T09:52:16.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b067fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2b100ca780 0x7f2b100b3cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45828/0 (socket says 172.21.15.70:45828) 2023-08-25T09:52:16.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b067fc640 1 -- 172.21.15.70:0/4025725570 learned_addr learned my addr 172.21.15.70:0/4025725570 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:16.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874701221 0 0) 0x7f2b100b12b0 con 0x7f2b100ca780 2023-08-25T09:52:16.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af0003520 con 0x7f2b100ca780 2023-08-25T09:52:16.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 0x7f2b100cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949960942 0 0) 0x7f2af0003520 con 0x7f2b100ca780 2023-08-25T09:52:16.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 msgr2=0x7f2b100cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0e7fc640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 0x7f2b100cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.277+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 >> v1:172.21.15.155:6789/0 conn(0x7f2b1015acd0 legacy=0x7f2b1015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1015b7c0 con 0x7f2b100ca780 2023-08-25T09:52:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f2af8001670 con 0x7f2b100ca780 2023-08-25T09:52:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 >> v1:172.21.15.70:6789/0 conn(0x7f2b100ca780 legacy=0x7f2b100b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 0x7f2af00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.282 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0003fd0 0x7f2af00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af0004d00 0x7f2af0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100b47e0 con 0x7f2af0004d00 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af000bbc0 con 0x7f2af0003fd0 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af000bd80 con 0x7f2b100b4400 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af0004d00 0x7f2af0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b067fc640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0003fd0 0x7f2af00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.283 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 msgr2=0x7f2af00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100b4400 0x7f2af00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0003fd0 msgr2=0x7f2af00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0003fd0 0x7f2af00047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.284 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.278+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/4025725570 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0003520 con 0x7f2af0004d00 2023-08-25T09:52:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.279+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af0004d00 0x7f2af0005180 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f2b08007360 tx=0x7f2af00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd613a9cbf25f75 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0802ccf0 con 0x7f2af0004d00 2023-08-25T09:52:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0802ce50 con 0x7f2af0004d00 2023-08-25T09:52:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b0e7fc640 1 -- 172.21.15.70:0/4025725570 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08034660 con 0x7f2af0004d00 2023-08-25T09:52:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b154f9640 1 -- 172.21.15.70:0/4025725570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af0004d00 msgr2=0x7f2af0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b154f9640 1 --2- 172.21.15.70:0/4025725570 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2af0004d00 0x7f2af0005180 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f2b08007360 tx=0x7f2af00057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/4025725570 reap_dead start 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b154f9640 1 -- 172.21.15.70:0/4025725570 shutdown_connections 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b154f9640 1 -- 172.21.15.70:0/4025725570 >> 172.21.15.70:0/4025725570 conn(0x7f2b100ce760 msgr2=0x7f2b100ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b154f9640 1 -- 172.21.15.70:0/4025725570 shutdown_connections 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.280+0000 7f2b154f9640 1 -- 172.21.15.70:0/4025725570 wait complete. 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.282+0000 7f2b154f9640 1 Processor -- start 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.282+0000 7f2b154f9640 1 -- start start 2023-08-25T09:52:16.288 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b154f9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0004d00 0x7f2b101e1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b154f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100c5fa0 0x7f2b100caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b154f9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 0x7f2b10148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b154f9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b101487b0 con 0x7f2b10147df0 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b154f9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100c67f0 con 0x7f2af0004d00 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b154f9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100c6ad0 con 0x7f2b100c5fa0 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 0x7f2b10148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 0x7f2b10148270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43154/0 (socket says 172.21.15.70:43154) 2023-08-25T09:52:16.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.283+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/117153736 learned_addr learned my addr 172.21.15.70:0/117153736 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0effd640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0004d00 0x7f2b101e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100c5fa0 msgr2=0x7f2b100caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b067fc640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100c5fa0 0x7f2b100caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100c5fa0 0x7f2b100caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0004d00 msgr2=0x7f2b101e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0004d00 0x7f2b101e1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/117153736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b100c6db0 con 0x7f2b10147df0 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0effd640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0004d00 0x7f2b101e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 0x7f2b10148270 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f2b0802dfd0 tx=0x7f2b08004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b07fff640 1 -- 172.21.15.70:0/117153736 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0803d600 con 0x7f2b10147df0 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b07fff640 1 -- 172.21.15.70:0/117153736 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0803d780 con 0x7f2b10147df0 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b07fff640 1 -- 172.21.15.70:0/117153736 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0803db30 con 0x7f2b10147df0 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b100c7090 con 0x7f2b10147df0 2023-08-25T09:52:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.284+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b100c6420 con 0x7f2b10147df0 2023-08-25T09:52:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.287+0000 7f2b07fff640 1 -- 172.21.15.70:0/117153736 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2b0803b040 con 0x7f2b10147df0 2023-08-25T09:52:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.288+0000 7f2b07fff640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2af406a080 0x7f2af406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.288+0000 7f2b07fff640 1 -- 172.21.15.70:0/117153736 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b080a9780 con 0x7f2b10147df0 2023-08-25T09:52:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.288+0000 7f2b0effd640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2af406a080 0x7f2af406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.288+0000 7f2b1806d740 1 Processor -- start 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b0effd640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2af406a080 0x7f2af406c540 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2b000016f0 tx=0x7f2b00009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 -- start start 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19fe80 0x5598cb181f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19d9a0 0x5598cb1824a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 0x5598cb155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb0c5190 con 0x5598cb17f590 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb0c2880 con 0x5598cb19d9a0 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.289+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598cb0be170 con 0x5598cb19fe80 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 0x5598cb155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 0x5598cb155cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43164/0 (socket says 172.21.15.70:43164) 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/224643925 learned_addr learned my addr 172.21.15.70:0/224643925 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/224643925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19fe80 msgr2=0x5598cb181f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19fe80 0x5598cb181f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/224643925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19d9a0 msgr2=0x5598cb1824a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19d9a0 0x5598cb1824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/224643925 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598cb0bc7f0 con 0x5598cb17f590 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 0x5598cb155cd0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f2b08080300 tx=0x7f2b08000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9177547a1cedbed server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b04ff9640 1 -- 172.21.15.70:0/224643925 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0808e070 con 0x5598cb17f590 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b04ff9640 1 -- 172.21.15.70:0/224643925 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b08080830 con 0x5598cb17f590 2023-08-25T09:52:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.290+0000 7f2b04ff9640 1 -- 172.21.15.70:0/224643925 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08009be0 con 0x5598cb17f590 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 -- 172.21.15.70:0/224643925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 msgr2=0x5598cb155cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 0x5598cb155cd0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f2b08080300 tx=0x7f2b08000f40 comp rx=0 tx=0).stop 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 -- 172.21.15.70:0/224643925 shutdown_connections 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb17f590 0x5598cb155cd0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19d9a0 0x5598cb1824a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 --2- 172.21.15.70:0/224643925 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19fe80 0x5598cb181f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 -- 172.21.15.70:0/224643925 >> 172.21.15.70:0/224643925 conn(0x5598cb1543e0 msgr2=0x5598cb19cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 -- 172.21.15.70:0/224643925 shutdown_connections 2023-08-25T09:52:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.291+0000 7f2b1806d740 1 -- 172.21.15.70:0/224643925 wait complete. 2023-08-25T09:52:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.292+0000 7f2b1806d740 1 -- start start 2023-08-25T09:52:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.292+0000 7f2b1806d740 1 Processor -- start 2023-08-25T09:52:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19d9a0 0x5598cb17e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 0x5598cb19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb05dd20 con 0x5598cb1b52e0 2023-08-25T09:52:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb05d9c0 con 0x5598cb19fe80 2023-08-25T09:52:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598cae23f20 con 0x5598cb19d9a0 2023-08-25T09:52:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b1806d740 10 client.0 _ll_register_callbacks cb 0x5598cb075630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:52:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 0x5598cb19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b0f7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 0x5598cb19bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43166/0 (socket says 172.21.15.70:43166) 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.293+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/2138902661 learned_addr learned my addr 172.21.15.70:0/2138902661 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/2138902661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19d9a0 msgr2=0x5598cb17e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19d9a0 0x5598cb17e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/2138902661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 msgr2=0x5598cb1b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b0f7fe640 1 -- 172.21.15.70:0/2138902661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b100c6420 con 0x5598cb1b52e0 2023-08-25T09:52:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 0x5598cb19bb20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f2b08000f80 tx=0x7f2b080070f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e09af39a378bbd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2afdffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:52:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2afdffb640 1 -- 172.21.15.70:0/2138902661 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0808e030 con 0x5598cb1b52e0 2023-08-25T09:52:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2afdffb640 1 -- 172.21.15.70:0/2138902661 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b0808f070 con 0x5598cb1b52e0 2023-08-25T09:52:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598cb092540 con 0x5598cb1b52e0 2023-08-25T09:52:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2afdffb640 1 -- 172.21.15.70:0/2138902661 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b080803a0 con 0x5598cb1b52e0 2023-08-25T09:52:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b1806d740 10 client.9756 fetch_fsmap learned FSMap version 91 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.294+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5598cb0b5f30 con 0x5598cb1b52e0 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2afdffb640 1 -- 172.21.15.70:0/2138902661 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598cb092540 con 0x5598cb1b52e0 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2afdffb640 1 -- 172.21.15.70:0/2138902661 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b0807cb40 con 0x5598cb1b52e0 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b1806d740 10 client.9756 fetch_fsmap finished waiting for FSMap version 91 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b1806d740 10 client.9756 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b1806d740 20 client.9756 populate_metadata read hostname 'smithi070' 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb178f00 0x5598cb184c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b1806d740 4 client.9756 mds_command: new command op to 9573 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 --> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598cb0b7150 con 0x5598cb178f00 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.295+0000 7f2b0effd640 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb178f00 0x5598cb184c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.296+0000 7f2afdffb640 10 client.9756 ms_handle_connect on v2:172.21.15.70:6834/771430984 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.296+0000 7f2b0effd640 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb178f00 0x5598cb184c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd8c89d85566b85 server_cookie=3b43aafd420fe5de in_seq=0 out_seq=0 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.296+0000 7f2afdffb640 1 -- 172.21.15.70:0/2138902661 <== mds.0 v2:172.21.15.70:6834/771430984 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598cb0b7150 con 0x5598cb178f00 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.296+0000 7f2afdffb640 10 client.9756 handle_command_reply: tid=0 2023-08-25T09:52:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.296+0000 7f2b1806d740 1 client.9756 shutdown 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 msgr2=0x5598cb19bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 0x5598cb19bb20 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f2b08000f80 tx=0x7f2b080070f0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 shutdown_connections 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb178f00 0x5598cb184c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb1b52e0 0x5598cb19bb20 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2138902661 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb19d9a0 0x5598cb17e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2afdffb640 0 client.9756 ms_handle_reset on v2:172.21.15.70:6834/771430984 2023-08-25T09:52:16.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 >> 172.21.15.70:0/2138902661 conn(0x5598cb17f590 msgr2=0x5598cb19cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 shutdown_connections 2023-08-25T09:52:16.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 1 -- 172.21.15.70:0/2138902661 wait complete. 2023-08-25T09:52:16.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.297+0000 7f2b1806d740 20 client.9756 trim_cache size 0 max 16384 2023-08-25T09:52:16.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.313+0000 7f2b1806d740 1 Processor -- start 2023-08-25T09:52:16.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.313+0000 7f2b1806d740 1 -- start start 2023-08-25T09:52:16.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb183760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb17b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b1806d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb14dcc0 con 0x5598cb19d9a0 2023-08-25T09:52:16.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb17d740 con 0x5598cb19fe80 2023-08-25T09:52:16.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598cb17d900 con 0x5598cb1b52e0 2023-08-25T09:52:16.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b0effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb183760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b0effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb183760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43174/0 (socket says 172.21.15.70:43174) 2023-08-25T09:52:16.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.314+0000 7f2b0effd640 1 -- 172.21.15.70:0/1616337312 learned_addr learned my addr 172.21.15.70:0/1616337312 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b0effd640 1 -- 172.21.15.70:0/1616337312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 msgr2=0x5598cb1b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b0effd640 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b0effd640 1 -- 172.21.15.70:0/1616337312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 msgr2=0x5598cb17b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b0effd640 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb17b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b0effd640 1 -- 172.21.15.70:0/1616337312 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598cb05ca10 con 0x5598cb19d9a0 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b0effd640 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb183760 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000d840 tx=0x7f2b00001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6366838eb0f5a0e5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b057fa640 1 -- 172.21.15.70:0/1616337312 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00022270 con 0x5598cb19d9a0 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b057fa640 1 -- 172.21.15.70:0/1616337312 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b00026070 con 0x5598cb19d9a0 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.315+0000 7f2b057fa640 1 -- 172.21.15.70:0/1616337312 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b000225d0 con 0x5598cb19d9a0 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 -- 172.21.15.70:0/1616337312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 msgr2=0x5598cb183760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb183760 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f2b0000d840 tx=0x7f2b00001170 comp rx=0 tx=0).stop 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 -- 172.21.15.70:0/1616337312 shutdown_connections 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb17b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 --2- 172.21.15.70:0/1616337312 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb183760 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 -- 172.21.15.70:0/1616337312 >> 172.21.15.70:0/1616337312 conn(0x5598cb17f590 msgr2=0x5598cb17e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 -- 172.21.15.70:0/1616337312 shutdown_connections 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.316+0000 7f2b1806d740 1 -- 172.21.15.70:0/1616337312 wait complete. 2023-08-25T09:52:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.317+0000 7f2b1806d740 1 -- start start 2023-08-25T09:52:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.317+0000 7f2b1806d740 1 Processor -- start 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb19b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1bc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb1a8fa0 con 0x5598cb19d9a0 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598cb1bcbd0 con 0x5598cb19fe80 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598cb155e00 con 0x5598cb1b52e0 2023-08-25T09:52:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b1806d740 10 client.0 _ll_register_callbacks cb 0x7f2b0802cd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b0effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b0effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43186/0 (socket says 172.21.15.70:43186) 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.318+0000 7f2b0effd640 1 -- 172.21.15.70:0/2357841286 learned_addr learned my addr 172.21.15.70:0/2357841286 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0f7fe640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b067fc640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1bc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0effd640 1 -- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 msgr2=0x5598cb1b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0effd640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0effd640 1 -- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 msgr2=0x5598cb1bc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0effd640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1bc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0effd640 1 -- 172.21.15.70:0/2357841286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1006ae20 con 0x5598cb19d9a0 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b0effd640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb19b750 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009010 tx=0x7f2b00010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4a02bc82cf13e2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2afdffb640 10 client.0 ms_handle_connect on v2:172.21.15.70:3300/0 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2afdffb640 1 -- 172.21.15.70:0/2357841286 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00030070 con 0x5598cb19d9a0 2023-08-25T09:52:16.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b08080830 con 0x5598cb19d9a0 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.319+0000 7f2afdffb640 1 -- 172.21.15.70:0/2357841286 <== mon.0 v2:172.21.15.70:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2b00022380 con 0x5598cb19d9a0 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2afdffb640 1 -- 172.21.15.70:0/2357841286 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b00022540 con 0x5598cb19d9a0 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 10 client.9762 fetch_fsmap learned FSMap version 91 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5598cb081de0 con 0x5598cb19d9a0 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2afdffb640 1 -- 172.21.15.70:0/2357841286 <== mon.0 v2:172.21.15.70:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b08080830 con 0x5598cb19d9a0 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2afdffb640 1 -- 172.21.15.70:0/2357841286 <== mon.0 v2:172.21.15.70:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b00006820 con 0x5598cb19d9a0 2023-08-25T09:52:16.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 10 client.9762 fetch_fsmap finished waiting for FSMap version 91 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 10 client.9762 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 20 client.9762 populate_metadata read hostname 'smithi070' 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb1a8600 0x5598cb1a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 4 client.9762 mds_command: new command op to 9573 tid=0[{"prefix": "status"}] 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 --> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5598cad6d5e0 con 0x5598cb1a8600 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.320+0000 7f2b067fc640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb1a8600 0x5598cb1a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b067fc640 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb1a8600 0x5598cb1a89e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e82b5f0b50f258 server_cookie=6be23a6f734da9a3 in_seq=0 out_seq=0 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2afdffb640 10 client.9762 ms_handle_connect on v2:172.21.15.70:6834/771430984 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2afdffb640 1 -- 172.21.15.70:0/2357841286 <== mds.0 v2:172.21.15.70:6834/771430984 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5598cad6d5e0 con 0x5598cb1a8600 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2afdffb640 10 client.9762 handle_command_reply: tid=0 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 client.9762 shutdown 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 msgr2=0x5598cb19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb19b750 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009010 tx=0x7f2b00010000 comp rx=0 tx=0).stop 2023-08-25T09:52:16.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 shutdown_connections 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x5598cb1b52e0 0x5598cb1b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x5598cb19fe80 0x5598cb1bc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:6834/771430984,v1:172.21.15.70:6836/771430984] conn(0x5598cb1a8600 0x5598cb1a89e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 --2- 172.21.15.70:0/2357841286 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x5598cb19d9a0 0x5598cb19b750 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 >> 172.21.15.70:0/2357841286 conn(0x5598cb17f590 msgr2=0x5598cb183760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2afdffb640 0 client.9762 ms_handle_reset on v2:172.21.15.70:6834/771430984 2023-08-25T09:52:16.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 shutdown_connections 2023-08-25T09:52:16.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.321+0000 7f2b1806d740 1 -- 172.21.15.70:0/2357841286 wait complete. 2023-08-25T09:52:16.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b1806d740 20 client.9762 trim_cache size 0 max 16384 2023-08-25T09:52:16.331 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:16.331 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "whoami": 0, 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 9573, 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "want_state": "up:active", 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "state": "up:active", 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs", 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "rank_uptime": 11.691842847, 2023-08-25T09:52:16.332 INFO:teuthology.orchestra.run.smithi070.stdout: "mdsmap_epoch": 91, 2023-08-25T09:52:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch": 198, 2023-08-25T09:52:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "osdmap_epoch_barrier": 198, 2023-08-25T09:52:16.333 INFO:teuthology.orchestra.run.smithi070.stdout: "uptime": 17.087704684999999 2023-08-25T09:52:16.333 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2af406a080 msgr2=0x7f2af406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2af406a080 0x7f2af406c540 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2b000016f0 tx=0x7f2b00009040 comp rx=0 tx=0).stop 2023-08-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 msgr2=0x7f2b10148270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 0x7f2b10148270 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f2b0802dfd0 tx=0x7f2b08004ab0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 shutdown_connections 2023-08-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2af406a080 0x7f2af406c540 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2b10147df0 0x7f2b10148270 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2b100c5fa0 0x7f2b100caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 --2- 172.21.15.70:0/117153736 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2af0004d00 0x7f2b101e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 >> 172.21.15.70:0/117153736 conn(0x7f2b100ce760 msgr2=0x7f2b100ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 shutdown_connections 2023-08-25T09:52:16.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.322+0000 7f2b154f9640 1 -- 172.21.15.70:0/117153736 wait complete. 2023-08-25T09:52:16.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:52:16.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.547+0000 7f8b050ec640 1 Processor -- start 2023-08-25T09:52:16.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.547+0000 7f8b050ec640 1 -- start start 2023-08-25T09:52:16.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8b050ec640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8b0015aa10 0x7f8b000b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8b050ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8b0006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8b050ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b0015d3f0 con 0x7f8b000ce5b0 2023-08-25T09:52:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8b050ec640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b000b2b10 con 0x7f8b0015aa10 2023-08-25T09:52:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8b050ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b00159d10 con 0x7f8b000b2680 2023-08-25T09:52:16.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8afed9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8b000ce5b0 0x7f8b000ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45834/0 (socket says 172.21.15.70:45834) 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.549+0000 7f8afed9d640 1 -- 172.21.15.70:0/2527923473 learned_addr learned my addr 172.21.15.70:0/2527923473 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8aff59e640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8b0006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771170131 0 0) 0x7f8b0015d3f0 con 0x7f8b000ce5b0 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae8003520 con 0x7f8b000ce5b0 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afe59c640 1 --2- 172.21.15.70:0/2527923473 >> v2:172.21.15.155:3300/0 conn(0x7f8b0015aa10 0x7f8b000b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556421354 0 0) 0x7f8ae8003520 con 0x7f8b000ce5b0 2023-08-25T09:52:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 msgr2=0x7f8b0006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8b0006f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 >> v2:172.21.15.155:3300/0 conn(0x7f8b0015aa10 msgr2=0x7f8b000b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 --2- 172.21.15.70:0/2527923473 >> v2:172.21.15.155:3300/0 conn(0x7f8b0015aa10 0x7f8b000b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000b2d20 con 0x7f8b000ce5b0 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8af0001670 con 0x7f8b000ce5b0 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8aff59e640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8b0006f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:52:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 >> v1:172.21.15.70:6789/0 conn(0x7f8b000ce5b0 legacy=0x7f8b000ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ae8003970 0x7f8ae8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8ae80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8ae8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.550+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000b2b10 con 0x7f8ae8004d00 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b00159d10 con 0x7f8ae8003970 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae800ba80 con 0x7f8b000b2680 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8aff59e640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8ae8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8afe59c640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8ae80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8afed9d640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ae8003970 0x7f8ae8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8aff59e640 1 -- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 msgr2=0x7f8ae80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8aff59e640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000b2680 0x7f8ae80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8aff59e640 1 -- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ae8003970 msgr2=0x7f8ae8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8aff59e640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8ae8003970 0x7f8ae8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.551+0000 7f8aff59e640 1 -- 172.21.15.70:0/2527923473 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae8003520 con 0x7f8ae8004d00 2023-08-25T09:52:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.552+0000 7f8aff59e640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8ae8005180 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f8af80074e0 tx=0x7f8ae80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd32779323ca2c76 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.552+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8033620 con 0x7f8ae8004d00 2023-08-25T09:52:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af8033780 con 0x7f8ae8004d00 2023-08-25T09:52:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8afdd9b640 1 -- 172.21.15.70:0/2527923473 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af8033b10 con 0x7f8ae8004d00 2023-08-25T09:52:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8b050ec640 1 -- 172.21.15.70:0/2527923473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 msgr2=0x7f8ae8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8b050ec640 1 --2- 172.21.15.70:0/2527923473 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8ae8005180 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f8af80074e0 tx=0x7f8ae80057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8aff59e640 1 -- 172.21.15.70:0/2527923473 reap_dead start 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8b050ec640 1 -- 172.21.15.70:0/2527923473 shutdown_connections 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8b050ec640 1 -- 172.21.15.70:0/2527923473 >> 172.21.15.70:0/2527923473 conn(0x7f8b00069850 msgr2=0x7f8b00069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8b050ec640 1 -- 172.21.15.70:0/2527923473 shutdown_connections 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.553+0000 7f8b050ec640 1 -- 172.21.15.70:0/2527923473 wait complete. 2023-08-25T09:52:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.555+0000 7f8b050ec640 1 Processor -- start 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.555+0000 7f8b050ec640 1 -- start start 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8b050ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8b00152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8b050ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b00145160 0x7f8b000ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8b050ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000ccc90 0x7f8b000cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8b050ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000cd650 con 0x7f8ae8004d00 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8b050ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b000af720 con 0x7f8b00145160 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8b050ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b001455e0 con 0x7f8b000ccc90 2023-08-25T09:52:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8afed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8b00152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.556+0000 7f8afed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8b00152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43200/0 (socket says 172.21.15.70:43200) 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 -- 172.21.15.70:0/4222967646 learned_addr learned my addr 172.21.15.70:0/4222967646 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afe59c640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b00145160 0x7f8b000ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 -- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000ccc90 msgr2=0x7f8b000cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000ccc90 0x7f8b000cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 -- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b00145160 msgr2=0x7f8b000ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b00145160 0x7f8b000ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 -- 172.21.15.70:0/4222967646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b000aeb10 con 0x7f8ae8004d00 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afe59c640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b00145160 0x7f8b000ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8afed9d640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8b00152500 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e760 tx=0x7f8af000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.557+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000ee70 con 0x7f8ae8004d00 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.558+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af000c870 con 0x7f8ae8004d00 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.558+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000cc20 con 0x7f8ae8004d00 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.558+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b000aedf0 con 0x7f8ae8004d00 2023-08-25T09:52:16.563 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.558+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b000af3b0 con 0x7f8ae8004d00 2023-08-25T09:52:16.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.559+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b000af570 con 0x7f8ae8004d00 2023-08-25T09:52:16.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.560+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8af000cd80 con 0x7f8ae8004d00 2023-08-25T09:52:16.564 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.561+0000 7f8ae77fe640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8aec06a080 0x7f8aec06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:16.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.561+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8af0088580 con 0x7f8ae8004d00 2023-08-25T09:52:16.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.561+0000 7f8afe59c640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8aec06a080 0x7f8aec06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:16.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.562+0000 7f8afe59c640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8aec06a080 0x7f8aec06c540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8b000ad300 tx=0x7f8af4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:16.566 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.565+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8af0057660 con 0x7f8ae8004d00 2023-08-25T09:52:16.796 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.795+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b0015d690 con 0x7f8ae8004d00 2023-08-25T09:52:16.798 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.796+0000 7f8ae77fe640 1 -- 172.21.15.70:0/4222967646 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b0015d690 con 0x7f8ae8004d00 2023-08-25T09:52:16.798 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:52:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8aec06a080 msgr2=0x7f8aec06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8aec06a080 0x7f8aec06c540 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f8b000ad300 tx=0x7f8af4008040 comp rx=0 tx=0).stop 2023-08-25T09:52:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 msgr2=0x7f8b00152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8b00152500 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e760 tx=0x7f8af000ec20 comp rx=0 tx=0).stop 2023-08-25T09:52:16.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 shutdown_connections 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8aec06a080 0x7f8aec06c540 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b000ccc90 0x7f8b000cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b00145160 0x7f8b000ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 --2- 172.21.15.70:0/4222967646 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8ae8004d00 0x7f8b00152500 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 >> 172.21.15.70:0/4222967646 conn(0x7f8b00069850 msgr2=0x7f8b00070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.798+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 shutdown_connections 2023-08-25T09:52:16.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:16.799+0000 7f8b050ec640 1 -- 172.21.15.70:0/4222967646 wait complete. 2023-08-25T09:52:16.819 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:52:16.820 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:52:16.820 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi070.front.sepia.ceph.com 2023-08-25T09:52:16.820 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T09:52:16.821 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:16.821 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:52:16.821 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:52:16.821 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:52:16.822 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:52:16.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:16.822 DEBUG:teuthology.orchestra.run.smithi070:> ip addr 2023-08-25T09:52:16.875 INFO:teuthology.orchestra.run.smithi070.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:52:16.875 INFO:teuthology.orchestra.run.smithi070.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:52:16.875 INFO:teuthology.orchestra.run.smithi070.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:52:16.876 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.876 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 ::1/128 scope host 2023-08-25T09:52:16.876 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.876 INFO:teuthology.orchestra.run.smithi070.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:52:16.876 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b2 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:16.876 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f0 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:d8:a4:b3 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout: altname enp7s0f1 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e6 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0c:c4:7a:8f:d3:e7 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:16.877 INFO:teuthology.orchestra.run.smithi070.stdout: inet 172.21.15.70/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft 41539sec preferred_lft 41539sec 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:52:16.878 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::41c:e0ff:fec2:c356/64 scope link 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether 0a:85:ff:8e:64:d4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::885:ffff:fe8e:64d4/64 scope link 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:52:16.879 INFO:teuthology.orchestra.run.smithi070.stdout: link/ether d2:54:cd:ba:a7:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:52:16.880 INFO:teuthology.orchestra.run.smithi070.stdout: inet6 fe80::d054:cdff:feba:a7f0/64 scope link 2023-08-25T09:52:16.880 INFO:teuthology.orchestra.run.smithi070.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:16.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:16.881 DEBUG:teuthology.orchestra.run.smithi070:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:52:16.977 INFO:teuthology.orchestra.run.smithi070.stdout:1 2023-08-25T09:52:16.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:16.981 DEBUG:teuthology.orchestra.run.smithi070:> route 2023-08-25T09:52:17.037 INFO:teuthology.orchestra.run.smithi070.stdout:Kernel IP routing table 2023-08-25T09:52:17.037 INFO:teuthology.orchestra.run.smithi070.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:52:17.037 INFO:teuthology.orchestra.run.smithi070.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:52:17.038 INFO:teuthology.orchestra.run.smithi070.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:52:17.038 INFO:teuthology.orchestra.run.smithi070.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:52:17.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:17.038 DEBUG:teuthology.orchestra.run.smithi070:> (cd / && exec bash -c ' 2023-08-25T09:52:17.039 DEBUG:teuthology.orchestra.run.smithi070:> set -e 2023-08-25T09:52:17.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:52:17.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:52:17.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:52:17.040 DEBUG:teuthology.orchestra.run.smithi070:> ') 2023-08-25T09:52:17.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:17.222 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:52:17.273 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:52:17.273 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:52:17.273 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:52:17.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:17.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link set brx.0 up 2023-08-25T09:52:17.369 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-08-25T09:52:17.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:52:17.370 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:17.428 INFO:teuthology.orchestra.run.smithi070.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T09:52:17.428 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:52:17.429 DEBUG:teuthology.orchestra.run.smithi070:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:17.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:52:17.579 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:52:17.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:52:17.677 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:52:17.681 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:52:17.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:17.683 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:52:17.744 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:52:17.745 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-08-25T09:52:17.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo modprobe fuse 2023-08-25T09:52:17.830 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:52:17.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:52:17.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:52:17.883 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:52:17.907 INFO:teuthology.orchestra.run.smithi070.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:52:17.910 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:52:17.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:17.911 DEBUG:teuthology.orchestra.run.smithi070:> ls /sys/fs/fuse/connections 2023-08-25T09:52:17.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:2023-08-25T09:52:17.941+0000 7faeaf155480 -1 init, newargv = 0x56541996add0 newargc=15 2023-08-25T09:52:17.943 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[127931]: starting ceph client 2023-08-25T09:52:17.949 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[127931]: starting fuse 2023-08-25T09:52:17.981 INFO:teuthology.orchestra.run.smithi070.stdout:39 2023-08-25T09:52:17.982 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:52:17.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:52:17.983 DEBUG:teuthology.orchestra.run.smithi070:> import glob 2023-08-25T09:52:17.983 DEBUG:teuthology.orchestra.run.smithi070:> import re 2023-08-25T09:52:17.983 DEBUG:teuthology.orchestra.run.smithi070:> import os 2023-08-25T09:52:17.984 DEBUG:teuthology.orchestra.run.smithi070:> import subprocess 2023-08-25T09:52:17.984 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:52:17.984 DEBUG:teuthology.orchestra.run.smithi070:> def _find_admin_socket(client_name): 2023-08-25T09:52:17.985 DEBUG:teuthology.orchestra.run.smithi070:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-08-25T09:52:17.985 DEBUG:teuthology.orchestra.run.smithi070:> files = glob.glob(asok_path) 2023-08-25T09:52:17.985 DEBUG:teuthology.orchestra.run.smithi070:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-08-25T09:52:17.986 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:52:17.986 DEBUG:teuthology.orchestra.run.smithi070:> # Given a non-glob path, it better be there 2023-08-25T09:52:17.986 DEBUG:teuthology.orchestra.run.smithi070:> if "*" not in asok_path: 2023-08-25T09:52:17.987 DEBUG:teuthology.orchestra.run.smithi070:> assert(len(files) == 1) 2023-08-25T09:52:17.987 DEBUG:teuthology.orchestra.run.smithi070:> return files[0] 2023-08-25T09:52:17.987 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:52:17.988 DEBUG:teuthology.orchestra.run.smithi070:> for f in files: 2023-08-25T09:52:17.988 DEBUG:teuthology.orchestra.run.smithi070:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:52:17.988 DEBUG:teuthology.orchestra.run.smithi070:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:52:17.989 DEBUG:teuthology.orchestra.run.smithi070:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:52:17.989 DEBUG:teuthology.orchestra.run.smithi070:> contents = proc_f.read() 2023-08-25T09:52:17.989 DEBUG:teuthology.orchestra.run.smithi070:> if mountpoint in contents: 2023-08-25T09:52:17.989 DEBUG:teuthology.orchestra.run.smithi070:> return f 2023-08-25T09:52:17.990 DEBUG:teuthology.orchestra.run.smithi070:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:52:17.990 DEBUG:teuthology.orchestra.run.smithi070:> 2023-08-25T09:52:17.991 DEBUG:teuthology.orchestra.run.smithi070:> print(_find_admin_socket("client.0")) 2023-08-25T09:52:17.991 DEBUG:teuthology.orchestra.run.smithi070:> ' 2023-08-25T09:52:18.223 INFO:teuthology.orchestra.run.smithi070.stdout:/var/run/ceph/ceph-client.0.127931.asok 2023-08-25T09:52:18.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:52:18.240 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.127931.asok 2023-08-25T09:52:18.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:18.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.127931.asok status 2023-08-25T09:52:18.483 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": { 2023-08-25T09:52:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:52:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:52:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: "entity_id": "0", 2023-08-25T09:52:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: "hostname": "smithi070", 2023-08-25T09:52:18.484 INFO:teuthology.orchestra.run.smithi070.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "pid": "127931", 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "root": "/" 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_count": 0, 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "dentry_pinned_count": 0, 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "id": 9777, 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "inst": { 2023-08-25T09:52:18.485 INFO:teuthology.orchestra.run.smithi070.stdout: "name": { 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "client", 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: "num": 9777 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2869798035 2023-08-25T09:52:18.486 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": { 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2869798035 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi070.stdout: "inst_str": "client.9777 192.168.0.1:0/2869798035", 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addr_str": "192.168.0.1:0/2869798035", 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "inode_count": 1, 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_epoch": 91, 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch": 198, 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklisted": false, 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout: "fs_name": "cephfs" 2023-08-25T09:52:18.488 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:18.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:18.499 DEBUG:teuthology.orchestra.run.smithi070:> stat --file-system '--printf=%T 2023-08-25T09:52:18.499 DEBUG:teuthology.orchestra.run.smithi070:> ' -- /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:18.551 INFO:teuthology.orchestra.run.smithi070.stdout:fuseblk 2023-08-25T09:52:18.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:18.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:18.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:18.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T09:52:18.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.865+0000 7f0151c1f640 1 Processor -- start 2023-08-25T09:52:18.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.865+0000 7f0151c1f640 1 -- start start 2023-08-25T09:52:18.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f0151c1f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f014c15ec10 0x7f014c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:18.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f0151c1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15c6d0 0x7f014c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:18.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f0151c1f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f014c0b1b60 con 0x7f014c15e830 2023-08-25T09:52:18.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f0151c1f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f014c06bd80 con 0x7f014c15ec10 2023-08-25T09:52:18.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f0151c1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c06b7c0 con 0x7f014c15c6d0 2023-08-25T09:52:18.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f014affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f014c15ec10 0x7f014c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:18.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f014b7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f014c15e830 0x7f014c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45844/0 (socket says 172.21.15.70:45844) 2023-08-25T09:52:18.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.866+0000 7f014b7fe640 1 -- 172.21.15.70:0/2072429332 learned_addr learned my addr 172.21.15.70:0/2072429332 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:18.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014affd640 1 -- 172.21.15.70:0/2072429332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15c6d0 msgr2=0x7f014c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:18.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014bfff640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15c6d0 0x7f014c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:18.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014a7fc640 1 -- 172.21.15.70:0/2072429332 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319159812 0 0) 0x7f014c0b1b60 con 0x7f014c15e830 2023-08-25T09:52:18.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014affd640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15c6d0 0x7f014c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:18.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014affd640 1 -- 172.21.15.70:0/2072429332 >> v1:172.21.15.70:6789/0 conn(0x7f014c15e830 legacy=0x7f014c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:18.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014affd640 1 -- 172.21.15.70:0/2072429332 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c06baa0 con 0x7f014c15ec10 2023-08-25T09:52:18.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.867+0000 7f014bfff640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15c6d0 0x7f014c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:18.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.868+0000 7f014affd640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15ec10 0x7f014c15c190 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f01400097a0 tx=0x7f014002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0443157393374e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:18.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.868+0000 7f014a7fc640 1 -- 172.21.15.70:0/2072429332 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014002d040 con 0x7f014c15ec10 2023-08-25T09:52:18.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.868+0000 7f014a7fc640 1 -- 172.21.15.70:0/2072429332 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0140031810 con 0x7f014c15ec10 2023-08-25T09:52:18.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.868+0000 7f014a7fc640 1 -- 172.21.15.70:0/2072429332 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140031ba0 con 0x7f014c15ec10 2023-08-25T09:52:18.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 -- 172.21.15.70:0/2072429332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15ec10 msgr2=0x7f014c15c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:18.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15ec10 0x7f014c15c190 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f01400097a0 tx=0x7f014002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:52:18.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 -- 172.21.15.70:0/2072429332 shutdown_connections 2023-08-25T09:52:18.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15ec10 0x7f014c15c190 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:18.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 --2- 172.21.15.70:0/2072429332 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15c6d0 0x7f014c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:18.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 -- 172.21.15.70:0/2072429332 >> 172.21.15.70:0/2072429332 conn(0x7f014c069850 msgr2=0x7f014c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:18.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 -- 172.21.15.70:0/2072429332 shutdown_connections 2023-08-25T09:52:18.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.869+0000 7f0151c1f640 1 -- 172.21.15.70:0/2072429332 wait complete. 2023-08-25T09:52:18.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.871+0000 7f0151c1f640 1 Processor -- start 2023-08-25T09:52:18.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.871+0000 7f0151c1f640 1 -- start start 2023-08-25T09:52:18.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f0151c1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 0x7f014c1487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:18.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f0151c1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014c15e830 0x7f014c0a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:18.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f0151c1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15ec10 0x7f014c146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:18.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f0151c1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c1452c0 con 0x7f014c15c6d0 2023-08-25T09:52:18.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f0151c1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c145010 con 0x7f014c15e830 2023-08-25T09:52:18.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f0151c1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c148cf0 con 0x7f014c15ec10 2023-08-25T09:52:18.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f014b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 0x7f014c1487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:18.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f014b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 0x7f014c1487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:43224/0 (socket says 172.21.15.70:43224) 2023-08-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.872+0000 7f014b7fe640 1 -- 172.21.15.70:0/3481257488 learned_addr learned my addr 172.21.15.70:0/3481257488 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014affd640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014c15e830 0x7f014c0a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014b7fe640 1 -- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15ec10 msgr2=0x7f014c146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014b7fe640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15ec10 0x7f014c146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014b7fe640 1 -- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014c15e830 msgr2=0x7f014c0a9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014b7fe640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014c15e830 0x7f014c0a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:18.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014b7fe640 1 -- 172.21.15.70:0/3481257488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0140009400 con 0x7f014c15c6d0 2023-08-25T09:52:18.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f014b7fe640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 0x7f014c1487b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c890 tx=0x7f013c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:18.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00a640 con 0x7f014c15c6d0 2023-08-25T09:52:18.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013c00a7c0 con 0x7f014c15c6d0 2023-08-25T09:52:18.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c00ab70 con 0x7f014c15c6d0 2023-08-25T09:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014c1454d0 con 0x7f014c15c6d0 2023-08-25T09:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.873+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014c145a90 con 0x7f014c15c6d0 2023-08-25T09:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.875+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014c04e420 con 0x7f014c15c6d0 2023-08-25T09:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.875+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f013c00acd0 con 0x7f014c15c6d0 2023-08-25T09:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.877+0000 7f0133fff640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f013806a050 0x7f013806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:18.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.877+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f013c087b10 con 0x7f014c15c6d0 2023-08-25T09:52:18.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.877+0000 7f014affd640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f013806a050 0x7f013806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:18.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.878+0000 7f014affd640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f013806a050 0x7f013806c510 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0140036000 tx=0x7f014003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:18.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:18.882+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f013c0531f0 con 0x7f014c15c6d0 2023-08-25T09:52:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.111+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f014c15f280 con 0x7f014c15c6d0 2023-08-25T09:52:19.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.112+0000 7f0133fff640 1 -- 172.21.15.70:0/3481257488 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f013c018070 con 0x7f014c15c6d0 2023-08-25T09:52:19.114 INFO:teuthology.orchestra.run.smithi070.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T09:52:19.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f013806a050 msgr2=0x7f013806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:19.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f013806a050 0x7f013806c510 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0140036000 tx=0x7f014003c040 comp rx=0 tx=0).stop 2023-08-25T09:52:19.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 msgr2=0x7f014c1487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:19.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 0x7f014c1487b0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f013c00c890 tx=0x7f013c00cd50 comp rx=0 tx=0).stop 2023-08-25T09:52:19.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 shutdown_connections 2023-08-25T09:52:19.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f013806a050 0x7f013806c510 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:19.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f014c15ec10 0x7f014c146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:19.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f014c15e830 0x7f014c0a9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:19.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 --2- 172.21.15.70:0/3481257488 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f014c15c6d0 0x7f014c1487b0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:19.120 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 >> 172.21.15.70:0/3481257488 conn(0x7f014c069850 msgr2=0x7f014c15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:19.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 shutdown_connections 2023-08-25T09:52:19.121 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:19.115+0000 7f0151c1f640 1 -- 172.21.15.70:0/3481257488 wait complete. 2023-08-25T09:52:19.134 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T09:52:19.134 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T09:52:19.134 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi155.front.sepia.ceph.com 2023-08-25T09:52:19.135 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T09:52:19.135 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:19.135 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T09:52:19.136 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T09:52:19.136 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T09:52:19.136 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:52:19.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.137 DEBUG:teuthology.orchestra.run.smithi155:> ip addr 2023-08-25T09:52:19.196 INFO:teuthology.orchestra.run.smithi155.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T09:52:19.196 INFO:teuthology.orchestra.run.smithi155.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T09:52:19.196 INFO:teuthology.orchestra.run.smithi155.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T09:52:19.196 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 ::1/128 scope host 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fa brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f0 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:52:19.197 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:d9:a0:fb brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout: altname enp7s0f1 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ec brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 0c:c4:7a:88:6d:ed brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout: inet 172.21.15.155/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft 41550sec preferred_lft 41550sec 2023-08-25T09:52:19.198 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::ec4:7aff:fe88:6ded/64 scope link noprefixroute 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::c4f3:5fff:fe03:da1e/64 scope link 2023-08-25T09:52:19.199 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether 36:30:86:26:e0:a1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::3430:86ff:fe26:e0a1/64 scope link 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout: link/ether ba:b1:08:9d:4a:15 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout: inet6 fe80::b8b1:8ff:fe9d:4a15/64 scope link 2023-08-25T09:52:19.200 INFO:teuthology.orchestra.run.smithi155.stdout: valid_lft forever preferred_lft forever 2023-08-25T09:52:19.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.202 DEBUG:teuthology.orchestra.run.smithi155:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T09:52:19.289 INFO:teuthology.orchestra.run.smithi155.stdout:1 2023-08-25T09:52:19.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.294 DEBUG:teuthology.orchestra.run.smithi155:> route 2023-08-25T09:52:19.350 INFO:teuthology.orchestra.run.smithi155.stdout:Kernel IP routing table 2023-08-25T09:52:19.350 INFO:teuthology.orchestra.run.smithi155.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T09:52:19.350 INFO:teuthology.orchestra.run.smithi155.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-08-25T09:52:19.350 INFO:teuthology.orchestra.run.smithi155.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-08-25T09:52:19.350 INFO:teuthology.orchestra.run.smithi155.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T09:52:19.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.351 DEBUG:teuthology.orchestra.run.smithi155:> (cd / && exec bash -c ' 2023-08-25T09:52:19.352 DEBUG:teuthology.orchestra.run.smithi155:> set -e 2023-08-25T09:52:19.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-08-25T09:52:19.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-08-25T09:52:19.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-08-25T09:52:19.353 DEBUG:teuthology.orchestra.run.smithi155:> ') 2023-08-25T09:52:19.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.546 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:52:19.595 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:52:19.595 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:52:19.596 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:52:19.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.597 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link set brx.0 up 2023-08-25T09:52:19.685 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-08-25T09:52:19.685 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:52:19.686 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:19.739 INFO:teuthology.orchestra.run.smithi155.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T09:52:19.739 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T09:52:19.740 DEBUG:teuthology.orchestra.run.smithi155:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:19.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:52:19.886 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:52:19.887 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:52:19.980 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:52:19.983 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:52:19.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:19.984 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:52:20.040 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-08-25T09:52:20.041 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-08-25T09:52:20.050 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:52:20.135 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:52:20.136 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:52:20.210 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:52:20.212 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:52:20.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:20.213 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:52:20.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:2023-08-25T09:52:20.247+0000 7fef4b837480 -1 init, newargv = 0x5585eccf2fc0 newargc=15 2023-08-25T09:52:20.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[94138]: starting ceph client 2023-08-25T09:52:20.257 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[94138]: starting fuse 2023-08-25T09:52:21.358 DEBUG:teuthology.orchestra.run.smithi155:> sudo modprobe fuse 2023-08-25T09:52:21.466 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-08-25T09:52:21.466 DEBUG:teuthology.orchestra.run.smithi155:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-08-25T09:52:21.563 INFO:teuthology.orchestra.run.smithi155.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-08-25T09:52:21.567 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-08-25T09:52:21.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:21.568 DEBUG:teuthology.orchestra.run.smithi155:> ls /sys/fs/fuse/connections 2023-08-25T09:52:21.627 INFO:teuthology.orchestra.run.smithi155.stdout:39 2023-08-25T09:52:21.628 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-08-25T09:52:21.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo stdin-killer -- python3 -c ' 2023-08-25T09:52:21.629 DEBUG:teuthology.orchestra.run.smithi155:> import glob 2023-08-25T09:52:21.629 DEBUG:teuthology.orchestra.run.smithi155:> import re 2023-08-25T09:52:21.629 DEBUG:teuthology.orchestra.run.smithi155:> import os 2023-08-25T09:52:21.630 DEBUG:teuthology.orchestra.run.smithi155:> import subprocess 2023-08-25T09:52:21.630 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:52:21.630 DEBUG:teuthology.orchestra.run.smithi155:> def _find_admin_socket(client_name): 2023-08-25T09:52:21.630 DEBUG:teuthology.orchestra.run.smithi155:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-08-25T09:52:21.631 DEBUG:teuthology.orchestra.run.smithi155:> files = glob.glob(asok_path) 2023-08-25T09:52:21.631 DEBUG:teuthology.orchestra.run.smithi155:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-08-25T09:52:21.631 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:52:21.632 DEBUG:teuthology.orchestra.run.smithi155:> # Given a non-glob path, it better be there 2023-08-25T09:52:21.632 DEBUG:teuthology.orchestra.run.smithi155:> if "*" not in asok_path: 2023-08-25T09:52:21.632 DEBUG:teuthology.orchestra.run.smithi155:> assert(len(files) == 1) 2023-08-25T09:52:21.633 DEBUG:teuthology.orchestra.run.smithi155:> return files[0] 2023-08-25T09:52:21.633 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:52:21.633 DEBUG:teuthology.orchestra.run.smithi155:> for f in files: 2023-08-25T09:52:21.634 DEBUG:teuthology.orchestra.run.smithi155:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-08-25T09:52:21.634 DEBUG:teuthology.orchestra.run.smithi155:> if os.path.exists("/proc/{0}".format(pid)): 2023-08-25T09:52:21.634 DEBUG:teuthology.orchestra.run.smithi155:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-08-25T09:52:21.635 DEBUG:teuthology.orchestra.run.smithi155:> contents = proc_f.read() 2023-08-25T09:52:21.635 DEBUG:teuthology.orchestra.run.smithi155:> if mountpoint in contents: 2023-08-25T09:52:21.635 DEBUG:teuthology.orchestra.run.smithi155:> return f 2023-08-25T09:52:21.636 DEBUG:teuthology.orchestra.run.smithi155:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-08-25T09:52:21.636 DEBUG:teuthology.orchestra.run.smithi155:> 2023-08-25T09:52:21.636 DEBUG:teuthology.orchestra.run.smithi155:> print(_find_admin_socket("client.1")) 2023-08-25T09:52:21.637 DEBUG:teuthology.orchestra.run.smithi155:> ' 2023-08-25T09:52:21.821 INFO:teuthology.orchestra.run.smithi155.stdout:/var/run/ceph/ceph-client.1.94138.asok 2023-08-25T09:52:21.825 INFO:teuthology.orchestra.run.smithi155.stderr:2023-08-25T09:52:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-08-25T09:52:21.839 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94138.asok 2023-08-25T09:52:21.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:21.841 DEBUG:teuthology.orchestra.run.smithi155:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94138.asok status 2023-08-25T09:52:22.081 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2023-08-25T09:52:22.082 INFO:teuthology.orchestra.run.smithi155.stdout: "metadata": { 2023-08-25T09:52:22.082 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_sha1": "ba6b39e4f4f77fb95c747099b53db0467d435f56", 2023-08-25T09:52:22.082 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 18.0.0-5703-gba6b39e4 (ba6b39e4f4f77fb95c747099b53db0467d435f56) reef (dev)", 2023-08-25T09:52:22.082 INFO:teuthology.orchestra.run.smithi155.stdout: "entity_id": "1", 2023-08-25T09:52:22.082 INFO:teuthology.orchestra.run.smithi155.stdout: "hostname": "smithi155", 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: "pid": "94138", 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: "root": "/" 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_count": 0, 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: "dentry_pinned_count": 0, 2023-08-25T09:52:22.083 INFO:teuthology.orchestra.run.smithi155.stdout: "id": 5828, 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: "inst": { 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: "name": { 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "client", 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: "num": 5828 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:52:22.084 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:52:22.085 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:52:22.085 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1898542384 2023-08-25T09:52:22.085 INFO:teuthology.orchestra.run.smithi155.stdout: } 2023-08-25T09:52:22.085 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:52:22.085 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": { 2023-08-25T09:52:22.085 INFO:teuthology.orchestra.run.smithi155.stdout: "type": "v1", 2023-08-25T09:52:22.086 INFO:teuthology.orchestra.run.smithi155.stdout: "addr": "192.168.0.1:0", 2023-08-25T09:52:22.086 INFO:teuthology.orchestra.run.smithi155.stdout: "nonce": 1898542384 2023-08-25T09:52:22.086 INFO:teuthology.orchestra.run.smithi155.stdout: }, 2023-08-25T09:52:22.086 INFO:teuthology.orchestra.run.smithi155.stdout: "inst_str": "client.5828 192.168.0.1:0/1898542384", 2023-08-25T09:52:22.086 INFO:teuthology.orchestra.run.smithi155.stdout: "addr_str": "192.168.0.1:0/1898542384", 2023-08-25T09:52:22.086 INFO:teuthology.orchestra.run.smithi155.stdout: "inode_count": 1, 2023-08-25T09:52:22.087 INFO:teuthology.orchestra.run.smithi155.stdout: "mds_epoch": 91, 2023-08-25T09:52:22.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch": 198, 2023-08-25T09:52:22.087 INFO:teuthology.orchestra.run.smithi155.stdout: "osd_epoch_barrier": 0, 2023-08-25T09:52:22.087 INFO:teuthology.orchestra.run.smithi155.stdout: "blocklisted": false, 2023-08-25T09:52:22.087 INFO:teuthology.orchestra.run.smithi155.stdout: "fs_name": "cephfs" 2023-08-25T09:52:22.087 INFO:teuthology.orchestra.run.smithi155.stdout:} 2023-08-25T09:52:22.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:22.097 DEBUG:teuthology.orchestra.run.smithi155:> stat --file-system '--printf=%T 2023-08-25T09:52:22.097 DEBUG:teuthology.orchestra.run.smithi155:> ' -- /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:22.152 INFO:teuthology.orchestra.run.smithi155.stdout:fuseblk 2023-08-25T09:52:22.153 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:22.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:22.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:22.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:52:22.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.469+0000 7f46af59e640 1 Processor -- start 2023-08-25T09:52:22.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.469+0000 7f46af59e640 1 -- start start 2023-08-25T09:52:22.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46af59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f46b00b2af0 0x7f46b006ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46af59e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b006c840 con 0x7f46b015aa90 2023-08-25T09:52:22.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46af59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b0070ef0 con 0x7f46b006b9c0 2023-08-25T09:52:22.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46af59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b006bfa0 con 0x7f46b00b2af0 2023-08-25T09:52:22.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46ae59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f46b015aa90 0x7f46b00716b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54840/0 (socket says 172.21.15.70:54840) 2023-08-25T09:52:22.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46ae59c640 1 -- 172.21.15.70:0/1277009361 learned_addr learned my addr 172.21.15.70:0/1277009361 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:22.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598295535 0 0) 0x7f46b006c840 con 0x7f46b015aa90 2023-08-25T09:52:22.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f469c003520 con 0x7f46b015aa90 2023-08-25T09:52:22.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.471+0000 7f46aed9d640 1 --2- 172.21.15.70:0/1277009361 >> v2:172.21.15.155:3300/0 conn(0x7f46b00b2af0 0x7f46b006ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2765385368 0 0) 0x7f469c003520 con 0x7f46b015aa90 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 >> v2:172.21.15.155:3300/0 conn(0x7f46b00b2af0 msgr2=0x7f46b006ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 --2- 172.21.15.70:0/1277009361 >> v2:172.21.15.155:3300/0 conn(0x7f46b00b2af0 0x7f46b006ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 >> v1:172.21.15.155:6789/0 conn(0x7f46b006b9c0 legacy=0x7f46b00b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b006c280 con 0x7f46b015aa90 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f46a00019f0 con 0x7f46b015aa90 2023-08-25T09:52:22.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 >> v1:172.21.15.70:6789/0 conn(0x7f46b015aa90 legacy=0x7f46b00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c003970 0x7f469c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c0047c0 0x7f469c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c005180 0x7f469c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b006bfa0 con 0x7f469c0047c0 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c00c0d0 con 0x7f469c005180 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c00c290 con 0x7f469c003970 2023-08-25T09:52:22.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46add9b640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c0047c0 0x7f469c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.472+0000 7f46aed9d640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c005180 0x7f469c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46ae59c640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c003970 0x7f469c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46add9b640 1 -- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c003970 msgr2=0x7f469c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46add9b640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c003970 0x7f469c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46add9b640 1 -- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c005180 msgr2=0x7f469c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46add9b640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c005180 0x7f469c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46add9b640 1 -- 172.21.15.70:0/1277009361 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c003520 con 0x7f469c0047c0 2023-08-25T09:52:22.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46aed9d640 1 -- 172.21.15.70:0/1277009361 reap_dead start 2023-08-25T09:52:22.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.473+0000 7f46ae59c640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f469c003970 0x7f469c003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.474+0000 7f46add9b640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c0047c0 0x7f469c004c40 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4698001750 tx=0x7f469c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00ced49612caa0e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.474+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980052d0 con 0x7f469c0047c0 2023-08-25T09:52:22.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.474+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469802e050 con 0x7f469c0047c0 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.474+0000 7f46ad59a640 1 -- 172.21.15.70:0/1277009361 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698003830 con 0x7f469c0047c0 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 -- 172.21.15.70:0/1277009361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c0047c0 msgr2=0x7f469c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c0047c0 0x7f469c004c40 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4698001750 tx=0x7f469c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 -- 172.21.15.70:0/1277009361 shutdown_connections 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 --2- 172.21.15.70:0/1277009361 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f469c0047c0 0x7f469c004c40 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 -- 172.21.15.70:0/1277009361 >> 172.21.15.70:0/1277009361 conn(0x7f46b00cf5b0 msgr2=0x7f46b00cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 -- 172.21.15.70:0/1277009361 shutdown_connections 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.475+0000 7f46af59e640 1 -- 172.21.15.70:0/1277009361 wait complete. 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.477+0000 7f46af59e640 1 Processor -- start 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.477+0000 7f46af59e640 1 -- start start 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46af59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c0047c0 0x7f46b0154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46af59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46b015dba0 0x7f46b00c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46af59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 0x7f46b0072820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46af59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c44d0 con 0x7f46b00c50c0 2023-08-25T09:52:22.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46af59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c58f0 con 0x7f469c0047c0 2023-08-25T09:52:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46af59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b00c3210 con 0x7f46b015dba0 2023-08-25T09:52:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 0x7f46b0072820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 0x7f46b0072820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42292/0 (socket says 172.21.15.70:42292) 2023-08-25T09:52:22.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 -- 172.21.15.70:0/3001457443 learned_addr learned my addr 172.21.15.70:0/3001457443 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46ae59c640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c0047c0 0x7f46b0154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 -- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46b015dba0 msgr2=0x7f46b00c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46b015dba0 0x7f46b00c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 -- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c0047c0 msgr2=0x7f46b0154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c0047c0 0x7f46b0154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.478+0000 7f46aed9d640 1 -- 172.21.15.70:0/3001457443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b00c34b0 con 0x7f46b00c50c0 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.479+0000 7f46aed9d640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 0x7f46b0072820 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f46b0073900 tx=0x7f46a800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.479+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8014870 con 0x7f46b00c50c0 2023-08-25T09:52:22.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.479+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a80149f0 con 0x7f46b00c50c0 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.479+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8014da0 con 0x7f46b00c50c0 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.479+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b00c3790 con 0x7f46b00c50c0 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.479+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b00c5540 con 0x7f46b00c50c0 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.480+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b0066fb0 con 0x7f46b00c50c0 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.481+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f46a801d400 con 0x7f46b00c50c0 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.482+0000 7f46a6ffd640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f469006a080 0x7f469006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:22.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.482+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46a8088ab0 con 0x7f46b00c50c0 2023-08-25T09:52:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.482+0000 7f46ae59c640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f469006a080 0x7f469006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:22.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.483+0000 7f46ae59c640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f469006a080 0x7f469006c540 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f46a0001b20 tx=0x7f46a0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:22.487 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.486+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f46a8054190 con 0x7f46b00c50c0 2023-08-25T09:52:22.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.713+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46b00b4db0 con 0x7f46b00c50c0 2023-08-25T09:52:22.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.714+0000 7f46a6ffd640 1 -- 172.21.15.70:0/3001457443 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f46a80577b0 con 0x7f46b00c50c0 2023-08-25T09:52:22.716 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:22.716 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:52:22.717 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:52:22.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f469006a080 msgr2=0x7f469006c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f469006a080 0x7f469006c540 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f46a0001b20 tx=0x7f46a0009040 comp rx=0 tx=0).stop 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 msgr2=0x7f46b0072820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 0x7f46b0072820 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f46b0073900 tx=0x7f46a800f710 comp rx=0 tx=0).stop 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 shutdown_connections 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f469006a080 0x7f469006c540 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46b00c50c0 0x7f46b0072820 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f46b015dba0 0x7f46b00c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 --2- 172.21.15.70:0/3001457443 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f469c0047c0 0x7f46b0154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:22.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 >> 172.21.15.70:0/3001457443 conn(0x7f46b00cf5b0 msgr2=0x7f46b0159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:22.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 shutdown_connections 2023-08-25T09:52:22.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:22.717+0000 7f46af59e640 1 -- 172.21.15.70:0/3001457443 wait complete. 2023-08-25T09:52:22.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:52:22.971 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":9573,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.324632626,"mdsmap_epoch":91,"osdmap_epoch":198,"osdmap_epoch_barrier":198,"uptime":23.720506709999999} 2023-08-25T09:52:22.972 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 9573, "mdsmap_epoch": 91, "osdmap_epoch": 198, "osdmap_epoch_barrier": 198, "rank_uptime": 18.324632626, "state": "up:active", "uptime": 23.72050671, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:52:22.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-08-25T09:52:23.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.187+0000 7feb6f59e640 1 Processor -- start 2023-08-25T09:52:23.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.188+0000 7feb6f59e640 1 -- start start 2023-08-25T09:52:23.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.189+0000 7feb6f59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb7015ab70 0x7feb7006a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:23.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.189+0000 7feb6f59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb7015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:23.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb6f59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb7015b010 con 0x7feb7015a790 2023-08-25T09:52:23.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb6f59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7015b2f0 con 0x7feb7015ab70 2023-08-25T09:52:23.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb6f59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb7015b5d0 con 0x7feb70072f30 2023-08-25T09:52:23.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb7015ab70 0x7feb7006a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:23.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7feb7015ab70 0x7feb7006a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42298/0 (socket says 172.21.15.70:42298) 2023-08-25T09:52:23.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 -- 172.21.15.70:0/228958187 learned_addr learned my addr 172.21.15.70:0/228958187 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:23.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb6ed9d640 1 --2- 172.21.15.70:0/228958187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb7015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:23.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 -- 172.21.15.70:0/228958187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 msgr2=0x7feb7015bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:23.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 --2- 172.21.15.70:0/228958187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb7015bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 -- 172.21.15.70:0/228958187 >> v1:172.21.15.155:6789/0 conn(0x7feb7015a790 legacy=0x7feb70072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:23.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.190+0000 7feb65d9b640 1 -- 172.21.15.70:0/228958187 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7015b8b0 con 0x7feb7015ab70 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb65d9b640 1 --2- 172.21.15.70:0/228958187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb7006a260 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7feb58009870 tx=0x7feb5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909e15283287f572 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6dd9b640 1 -- 172.21.15.70:0/228958187 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58031640 con 0x7feb7015ab70 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6dd9b640 1 -- 172.21.15.70:0/228958187 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb580317a0 con 0x7feb7015ab70 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6dd9b640 1 -- 172.21.15.70:0/228958187 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb58031b30 con 0x7feb7015ab70 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6f59e640 1 -- 172.21.15.70:0/228958187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 msgr2=0x7feb7006a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6f59e640 1 --2- 172.21.15.70:0/228958187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb7006a260 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7feb58009870 tx=0x7feb5802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6f59e640 1 -- 172.21.15.70:0/228958187 shutdown_connections 2023-08-25T09:52:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6f59e640 1 --2- 172.21.15.70:0/228958187 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb7015bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.192+0000 7feb6f59e640 1 --2- 172.21.15.70:0/228958187 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb7006a260 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.193+0000 7feb6f59e640 1 -- 172.21.15.70:0/228958187 >> 172.21.15.70:0/228958187 conn(0x7feb700cc0f0 msgr2=0x7feb700cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:23.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.193+0000 7feb6f59e640 1 -- 172.21.15.70:0/228958187 shutdown_connections 2023-08-25T09:52:23.196 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.193+0000 7feb6f59e640 1 -- 172.21.15.70:0/228958187 wait complete. 2023-08-25T09:52:23.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.194+0000 7feb6f59e640 1 Processor -- start 2023-08-25T09:52:23.197 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.194+0000 7feb6f59e640 1 -- start start 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.195+0000 7feb6f59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb700cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.195+0000 7feb6f59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb7015a790 0x7feb700c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.195+0000 7feb6f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb70154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.195+0000 7feb6f59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb70154890 con 0x7feb7015ab70 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.195+0000 7feb6f59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb700b4380 con 0x7feb7015a790 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.195+0000 7feb6f59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb700b45d0 con 0x7feb70072f30 2023-08-25T09:52:23.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb70154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb70154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42302/0 (socket says 172.21.15.70:42302) 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 -- 172.21.15.70:0/2364964576 learned_addr learned my addr 172.21.15.70:0/2364964576 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 -- 172.21.15.70:0/2364964576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 msgr2=0x7feb700cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb700cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 -- 172.21.15.70:0/2364964576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb7015a790 msgr2=0x7feb700c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb7015a790 0x7feb700c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 -- 172.21.15.70:0/2364964576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb580094d0 con 0x7feb7015ab70 2023-08-25T09:52:23.199 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb6ed9d640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb70154350 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7feb68007e40 tx=0x7feb6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.196+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68014620 con 0x7feb7015ab70 2023-08-25T09:52:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.197+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6800ee90 con 0x7feb7015ab70 2023-08-25T09:52:23.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.197+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb680149e0 con 0x7feb7015ab70 2023-08-25T09:52:23.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.197+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb700b48b0 con 0x7feb7015ab70 2023-08-25T09:52:23.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.197+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb70154ea0 con 0x7feb7015ab70 2023-08-25T09:52:23.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.198+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb70154a20 con 0x7feb7015ab70 2023-08-25T09:52:23.201 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.199+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feb68014b40 con 0x7feb7015ab70 2023-08-25T09:52:23.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.200+0000 7feb677fe640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feb5006a050 0x7feb5006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:23.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.200+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb68088ff0 con 0x7feb7015ab70 2023-08-25T09:52:23.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.201+0000 7feb6e59c640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feb5006a050 0x7feb5006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:23.202 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.201+0000 7feb6e59c640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feb5006a050 0x7feb5006c510 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feb60003cc0 tx=0x7feb600079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:23.206 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.205+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feb680bb990 con 0x7feb7015ab70 2023-08-25T09:52:23.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.397+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7feb70155090 con 0x7feb7015ab70 2023-08-25T09:52:23.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.529+0000 7feb677fe640 1 -- 172.21.15.70:0/2364964576 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v199) v1 ==== 181+0+0 (secure 0 0 0) 0x7feb70154a20 con 0x7feb7015ab70 2023-08-25T09:52:23.531 INFO:teuthology.orchestra.run.smithi070.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-08-25T09:52:23.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.532+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feb5006a050 msgr2=0x7feb5006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.532+0000 7feb6f59e640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feb5006a050 0x7feb5006c510 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7feb60003cc0 tx=0x7feb600079b0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.532+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 msgr2=0x7feb70154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.532+0000 7feb6f59e640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb70154350 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7feb68007e40 tx=0x7feb6800e4b0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 shutdown_connections 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7feb5006a050 0x7feb5006c510 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7feb7015ab70 0x7feb70154350 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7feb7015a790 0x7feb700c9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 --2- 172.21.15.70:0/2364964576 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7feb70072f30 0x7feb700cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 >> 172.21.15.70:0/2364964576 conn(0x7feb700cc0f0 msgr2=0x7feb700c89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:23.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 shutdown_connections 2023-08-25T09:52:23.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:23.533+0000 7feb6f59e640 1 -- 172.21.15.70:0/2364964576 wait complete. 2023-08-25T09:52:23.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-08-25T09:52:23.781 INFO:teuthology.orchestra.run.smithi070.stdout:{"osd_mon_report_interval":"5"} 2023-08-25T09:52:23.782 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-08-25T09:52:23.782 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-08-25T09:52:23.783 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:52:23.783 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:23.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:23.836 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-08-25T09:52:24.231 INFO:teuthology.orchestra.run.smithi070.stderr:16+0 records in 2023-08-25T09:52:24.231 INFO:teuthology.orchestra.run.smithi070.stderr:16+0 records out 2023-08-25T09:52:24.231 INFO:teuthology.orchestra.run.smithi070.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.331425 s, 50.6 MB/s 2023-08-25T09:52:24.233 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:52:24.233 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:24.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:24.294 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-08-25T09:52:24.647 INFO:teuthology.orchestra.run.smithi070.stderr:17+0 records in 2023-08-25T09:52:24.647 INFO:teuthology.orchestra.run.smithi070.stderr:17+0 records out 2023-08-25T09:52:24.647 INFO:teuthology.orchestra.run.smithi070.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.291594 s, 61.1 MB/s 2023-08-25T09:52:24.649 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-08-25T09:52:24.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:24.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.862+0000 7f7f231e1640 1 Processor -- start 2023-08-25T09:52:24.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.862+0000 7f7f231e1640 1 -- start start 2023-08-25T09:52:24.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f231e1640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f1c0ce5b0 0x7f7f1c0ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:24.865 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f231e1640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c0ceed0 con 0x7f7f1c0b2850 2023-08-25T09:52:24.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f231e1640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c15d3f0 con 0x7f7f1c15aa10 2023-08-25T09:52:24.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f231e1640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c159d10 con 0x7f7f1c0ce5b0 2023-08-25T09:52:24.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f20f7d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7f1c0ce5b0 0x7f7f1c0ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:24.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f2177e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f7f1c0b2850 0x7f7f1c06f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:54848/0 (socket says 172.21.15.70:54848) 2023-08-25T09:52:24.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.864+0000 7f7f2177e640 1 -- 172.21.15.70:0/1131755398 learned_addr learned my addr 172.21.15.70:0/1131755398 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:24.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.865+0000 7f7f20f7d640 1 -- 172.21.15.70:0/1131755398 >> v1:172.21.15.155:6789/0 conn(0x7f7f1c15aa10 legacy=0x7f7f1c0b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:24.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.865+0000 7f7f20f7d640 1 -- 172.21.15.70:0/1131755398 >> v1:172.21.15.70:6789/0 conn(0x7f7f1c0b2850 legacy=0x7f7f1c06f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:24.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.865+0000 7f7f20f7d640 1 -- 172.21.15.70:0/1131755398 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0b2d20 con 0x7f7f1c0ce5b0 2023-08-25T09:52:24.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.865+0000 7f7f1b7fe640 1 -- 172.21.15.70:0/1131755398 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379381479 0 0) 0x7f7f1c0ceed0 con 0x7f7f1c0b2850 2023-08-25T09:52:24.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.866+0000 7f7f20f7d640 1 --2- 172.21.15.70:0/1131755398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c0ce5b0 0x7f7f1c0ce990 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0097a0 tx=0x7f7f0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a475fd516abe624c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:24.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.866+0000 7f7f1b7fe640 1 -- 172.21.15.70:0/1131755398 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c02e070 con 0x7f7f1c0ce5b0 2023-08-25T09:52:24.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.866+0000 7f7f1b7fe640 1 -- 172.21.15.70:0/1131755398 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0315a0 con 0x7f7f1c0ce5b0 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.866+0000 7f7f1b7fe640 1 -- 172.21.15.70:0/1131755398 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c031930 con 0x7f7f1c0ce5b0 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 -- 172.21.15.70:0/1131755398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c0ce5b0 msgr2=0x7f7f1c0ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 --2- 172.21.15.70:0/1131755398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c0ce5b0 0x7f7f1c0ce990 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0097a0 tx=0x7f7f0c02b840 comp rx=0 tx=0).stop 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 -- 172.21.15.70:0/1131755398 shutdown_connections 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 --2- 172.21.15.70:0/1131755398 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c0ce5b0 0x7f7f1c0ce990 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 -- 172.21.15.70:0/1131755398 >> 172.21.15.70:0/1131755398 conn(0x7f7f1c069850 msgr2=0x7f7f1c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:24.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 -- 172.21.15.70:0/1131755398 shutdown_connections 2023-08-25T09:52:24.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.867+0000 7f7f231e1640 1 -- 172.21.15.70:0/1131755398 wait complete. 2023-08-25T09:52:24.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.869+0000 7f7f231e1640 1 Processor -- start 2023-08-25T09:52:24.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.869+0000 7f7f231e1640 1 -- start start 2023-08-25T09:52:24.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f231e1640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f1c0b2850 0x7f7f1c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:24.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f231e1640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f1c0ce5b0 0x7f7f1c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:24.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f231e1640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 0x7f7f1c1f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:24.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f231e1640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c14f7d0 con 0x7f7f1c15aa10 2023-08-25T09:52:24.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f231e1640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c1f2010 con 0x7f7f1c0b2850 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f231e1640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0b0e50 con 0x7f7f1c0ce5b0 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f2177e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 0x7f7f1c1f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f2177e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 0x7f7f1c1f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:42314/0 (socket says 172.21.15.70:42314) 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.870+0000 7f7f2177e640 1 -- 172.21.15.70:0/2186719651 learned_addr learned my addr 172.21.15.70:0/2186719651 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f1bfff640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f1c0ce5b0 0x7f7f1c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f2177e640 1 -- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f1c0ce5b0 msgr2=0x7f7f1c14f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:24.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f20f7d640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f1c0b2850 0x7f7f1c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:24.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f2177e640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f1c0ce5b0 0x7f7f1c14f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:24.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f2177e640 1 -- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f1c0b2850 msgr2=0x7f7f1c0ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:24.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f2177e640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f1c0b2850 0x7f7f1c0ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:24.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f2177e640 1 -- 172.21.15.70:0/2186719651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009400 con 0x7f7f1c15aa10 2023-08-25T09:52:24.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f1bfff640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f1c0ce5b0 0x7f7f1c14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:24.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f20f7d640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f1c0b2850 0x7f7f1c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:24.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f2177e640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 0x7f7f1c1f2a10 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f7f14001560 tx=0x7f7f1400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:24.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400a640 con 0x7f7f1c15aa10 2023-08-25T09:52:24.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400a7a0 con 0x7f7f1c15aa10 2023-08-25T09:52:24.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400ab30 con 0x7f7f1c15aa10 2023-08-25T09:52:24.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c0b0890 con 0x7f7f1c15aa10 2023-08-25T09:52:24.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.871+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c1f2210 con 0x7f7f1c15aa10 2023-08-25T09:52:24.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.873+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c04e420 con 0x7f7f1c15aa10 2023-08-25T09:52:24.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.873+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7f1400ac90 con 0x7f7f1c15aa10 2023-08-25T09:52:24.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.875+0000 7f7f197fa640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7efc06a050 0x7f7efc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:24.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.875+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f14087b40 con 0x7f7f1c15aa10 2023-08-25T09:52:24.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.875+0000 7f7f20f7d640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7efc06a050 0x7f7efc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:24.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.876+0000 7f7f20f7d640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7efc06a050 0x7f7efc06c510 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c036000 tx=0x7f7f0c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:24.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:24.879+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f14053220 con 0x7f7f1c15aa10 2023-08-25T09:52:25.070 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.068+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f7f1c0cae50 con 0x7f7f1c15aa10 2023-08-25T09:52:25.073 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.072+0000 7f7f197fa640 1 -- 172.21.15.70:0/2186719651 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v199) v1 ==== 81+0+36967 (secure 0 0 0) 0x7f7f14018020 con 0x7f7f1c15aa10 2023-08-25T09:52:25.074 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:25.074 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 199, 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:23.511316+0000", 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:25.075 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:25.076 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:25.077 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.078 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:25.079 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:25.080 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:25.081 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:25.082 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:25.083 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:25.084 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:25.085 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:25.086 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:25.087 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:25.087 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:25.087 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:25.087 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:25.088 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:25.088 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:25.088 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:25.088 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:25.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.088 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:25.089 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.090 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:25.093 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:25.094 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:25.095 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:25.096 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:25.097 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:25.098 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:25.099 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:25.100 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:25.101 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:25.101 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:25.101 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:25.101 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:25.101 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:25.102 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:25.103 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:25.104 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:25.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.106 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:25.106 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:25.106 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:25.106 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:25.106 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:25.106 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:25.107 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:25.108 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.109 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:25.110 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:25.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:25.112 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:25.113 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:25.114 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:25.115 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:25.115 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:25.115 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:25.115 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:25.115 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:25.116 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:25.116 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:25.116 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:25.116 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:25.116 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:25.116 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:25.117 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "199", 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:25.118 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:25.119 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:25.120 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:25.121 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:25.122 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:25.123 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:25.125 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.126 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.127 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.128 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.129 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:25.130 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.134 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:25.135 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.137 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:25.139 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.140 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.141 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.142 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:25.143 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.144 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:25.145 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.146 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.147 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.148 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:25.149 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.152 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.152 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:25.152 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.152 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.154 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.156 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.160 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.161 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:25.162 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.163 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.163 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.163 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.164 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.165 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.166 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:25.166 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.166 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.167 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:25.167 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:25.167 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.167 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.168 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:25.168 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:25.168 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:25.169 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:25.169 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.169 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.170 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.170 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.171 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:25.171 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.171 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.172 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.172 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.172 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.173 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.173 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.173 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.173 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.174 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.174 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.174 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.175 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.175 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.175 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:25.176 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.177 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:25.177 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.177 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.177 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.178 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.178 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.179 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.179 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:25.179 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.179 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.180 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.181 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:25.182 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.182 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.182 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.182 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.182 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.183 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:25.184 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.185 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.186 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.187 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.188 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:25.189 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.192 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.193 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:25.194 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.195 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.196 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.197 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.198 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.200 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:25.204 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:25.206 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.208 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.209 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.210 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:25.211 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.214 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:25.215 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.216 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:25.216 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:25.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.216 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:25.216 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.216 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.217 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.219 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.220 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:25.222 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.223 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:25.224 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:25.226 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:25.227 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:25.228 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:25.229 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:25.229 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:25.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:25.229 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:25.229 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:25.230 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:25.230 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:25.230 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:25.230 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:25.230 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:25.231 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:25.231 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:25.231 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:25.231 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:25.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:25.232 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7efc06a050 msgr2=0x7f7efc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7efc06a050 0x7f7efc06c510 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c036000 tx=0x7f7f0c03d040 comp rx=0 tx=0).stop 2023-08-25T09:52:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 msgr2=0x7f7f1c1f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 0x7f7f1c1f2a10 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f7f14001560 tx=0x7f7f1400cc70 comp rx=0 tx=0).stop 2023-08-25T09:52:25.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 shutdown_connections 2023-08-25T09:52:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f7efc06a050 0x7f7efc06c510 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7f1c15aa10 0x7f7f1c1f2a10 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f7f1c0ce5b0 0x7f7f1c14f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 --2- 172.21.15.70:0/2186719651 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f7f1c0b2850 0x7f7f1c0ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:25.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.075+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 >> 172.21.15.70:0/2186719651 conn(0x7f7f1c069850 msgr2=0x7f7f1c06f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:25.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.076+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 shutdown_connections 2023-08-25T09:52:25.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:25.076+0000 7f7f231e1640 1 -- 172.21.15.70:0/2186719651 wait complete. 2023-08-25T09:52:25.237 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:52:30.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:30.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.462+0000 7f4646e58640 1 Processor -- start 2023-08-25T09:52:30.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.462+0000 7f4646e58640 1 -- start start 2023-08-25T09:52:30.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4646e58640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f46400cadb0 0x7f46400cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:30.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4646e58640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46400b2aa0 con 0x7f4640159840 2023-08-25T09:52:30.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4646e58640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46400b4450 con 0x7f464015d3d0 2023-08-25T09:52:30.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4646e58640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46400c8b60 con 0x7f46400cadb0 2023-08-25T09:52:30.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4645e56640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f4640159840 0x7f46400b2390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45438/0 (socket says 172.21.15.70:45438) 2023-08-25T09:52:30.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4645e56640 1 -- 172.21.15.70:0/312142929 learned_addr learned my addr 172.21.15.70:0/312142929 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:30.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.464+0000 7f4646657640 1 --2- 172.21.15.70:0/312142929 >> v2:172.21.15.70:3300/0 conn(0x7f46400cadb0 0x7f46400cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:30.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.465+0000 7f4644e54640 1 -- 172.21.15.70:0/312142929 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57016317 0 0) 0x7f46400b2aa0 con 0x7f4640159840 2023-08-25T09:52:30.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.465+0000 7f4646657640 1 -- 172.21.15.70:0/312142929 >> v1:172.21.15.155:6789/0 conn(0x7f464015d3d0 legacy=0x7f46400b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:30.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.465+0000 7f4646657640 1 -- 172.21.15.70:0/312142929 >> v1:172.21.15.70:6789/0 conn(0x7f4640159840 legacy=0x7f46400b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:30.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.465+0000 7f4646657640 1 -- 172.21.15.70:0/312142929 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464015d890 con 0x7f46400cadb0 2023-08-25T09:52:30.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.466+0000 7f4646657640 1 --2- 172.21.15.70:0/312142929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46400cadb0 0x7f46400cb1d0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f4638009d90 tx=0x7f463802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d445f984aaec5807 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:30.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.466+0000 7f4644e54640 1 -- 172.21.15.70:0/312142929 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463802cd00 con 0x7f46400cadb0 2023-08-25T09:52:30.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.466+0000 7f4644e54640 1 -- 172.21.15.70:0/312142929 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463802ce60 con 0x7f46400cadb0 2023-08-25T09:52:30.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.466+0000 7f4644e54640 1 -- 172.21.15.70:0/312142929 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638034660 con 0x7f46400cadb0 2023-08-25T09:52:30.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 -- 172.21.15.70:0/312142929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46400cadb0 msgr2=0x7f46400cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:30.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 --2- 172.21.15.70:0/312142929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46400cadb0 0x7f46400cb1d0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f4638009d90 tx=0x7f463802bf50 comp rx=0 tx=0).stop 2023-08-25T09:52:30.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 -- 172.21.15.70:0/312142929 shutdown_connections 2023-08-25T09:52:30.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 --2- 172.21.15.70:0/312142929 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f46400cadb0 0x7f46400cb1d0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 -- 172.21.15.70:0/312142929 >> 172.21.15.70:0/312142929 conn(0x7f46400affd0 msgr2=0x7f46400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:30.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 -- 172.21.15.70:0/312142929 shutdown_connections 2023-08-25T09:52:30.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.467+0000 7f4646e58640 1 -- 172.21.15.70:0/312142929 wait complete. 2023-08-25T09:52:30.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.469+0000 7f4646e58640 1 Processor -- start 2023-08-25T09:52:30.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.469+0000 7f4646e58640 1 -- start start 2023-08-25T09:52:30.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646e58640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46400cadb0 0x7f46400c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:30.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646e58640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4640159840 0x7f46400c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:30.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646e58640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 0x7f4640152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:30.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646e58640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46400c1f90 con 0x7f464015d3d0 2023-08-25T09:52:30.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646e58640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4640070da0 con 0x7f46400cadb0 2023-08-25T09:52:30.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646e58640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46400c21e0 con 0x7f4640159840 2023-08-25T09:52:30.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646657640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 0x7f4640152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646657640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 0x7f4640152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52818/0 (socket says 172.21.15.70:52818) 2023-08-25T09:52:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.470+0000 7f4646657640 1 -- 172.21.15.70:0/2514983415 learned_addr learned my addr 172.21.15.70:0/2514983415 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4645e56640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46400cadb0 0x7f46400c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4645655640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4640159840 0x7f46400c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4646657640 1 -- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4640159840 msgr2=0x7f46400c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:30.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4646657640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4640159840 0x7f46400c4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4646657640 1 -- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46400cadb0 msgr2=0x7f46400c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:30.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4646657640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46400cadb0 0x7f46400c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4646657640 1 -- 172.21.15.70:0/2514983415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4638009a70 con 0x7f464015d3d0 2023-08-25T09:52:30.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.471+0000 7f4646657640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 0x7f4640152810 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f4638009d90 tx=0x7f4638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:30.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.472+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463803d600 con 0x7f464015d3d0 2023-08-25T09:52:30.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.472+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463803d780 con 0x7f464015d3d0 2023-08-25T09:52:30.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.472+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463803db30 con 0x7f464015d3d0 2023-08-25T09:52:30.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.472+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46400c24c0 con 0x7f464015d3d0 2023-08-25T09:52:30.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.472+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46400c6400 con 0x7f464015d3d0 2023-08-25T09:52:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.473+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4640066fb0 con 0x7f464015d3d0 2023-08-25T09:52:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.474+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f463803b040 con 0x7f464015d3d0 2023-08-25T09:52:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.475+0000 7f46367fc640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f462406a050 0x7f462406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.475+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46380aa600 con 0x7f464015d3d0 2023-08-25T09:52:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.475+0000 7f4645e56640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f462406a050 0x7f462406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:30.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.476+0000 7f4645e56640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f462406a050 0x7f462406c510 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f463c000f60 tx=0x7f463c007b90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:30.480 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.479+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4638075ce0 con 0x7f464015d3d0 2023-08-25T09:52:30.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.668+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f46400c87e0 con 0x7f464015d3d0 2023-08-25T09:52:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.672+0000 7f46367fc640 1 -- 172.21.15.70:0/2514983415 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v199) v1 ==== 81+0+36967 (secure 0 0 0) 0x7f4638079300 con 0x7f464015d3d0 2023-08-25T09:52:30.674 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:30.674 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:30.674 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 199, 2023-08-25T09:52:30.675 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:30.675 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:30.675 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:23.511316+0000", 2023-08-25T09:52:30.675 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:30.675 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:30.676 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:30.676 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:30.676 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:30.676 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:30.677 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:30.678 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:30.678 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:30.678 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:30.678 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:30.679 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:30.679 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:30.679 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:30.679 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:30.679 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:30.680 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:30.680 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:30.680 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:30.680 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:30.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.681 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:30.681 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:30.681 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:30.682 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:30.682 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:30.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:30.682 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:30.683 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:30.683 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:30.683 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:30.683 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:30.683 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:30.683 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:30.684 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:30.685 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:30.685 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:30.685 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:30.685 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:30.685 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:30.685 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:30.686 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:30.687 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:30.688 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:30.689 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:30.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:30.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.690 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:30.690 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:30.691 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:30.691 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:30.691 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:30.691 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:30.692 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:30.692 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:30.692 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:30.693 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:30.693 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:30.693 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:30.694 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:30.694 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:30.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.694 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:30.695 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:30.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.695 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:30.695 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:30.696 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:30.696 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:30.696 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:30.697 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:30.697 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:30.697 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:30.697 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:30.697 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.698 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:30.698 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:30.699 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:30.699 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:30.699 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:30.700 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:30.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:30.702 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:30.703 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:30.704 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:30.705 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:30.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:30.707 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:30.708 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:30.709 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:30.710 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:30.711 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:30.712 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:30.713 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:30.714 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:30.715 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:30.716 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:30.717 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:30.718 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:30.719 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:30.719 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:30.719 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:30.719 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:30.719 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:30.719 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:30.720 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:30.721 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:30.723 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:30.724 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:30.725 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "199", 2023-08-25T09:52:30.726 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:30.727 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:30.728 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:30.729 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:30.730 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:30.731 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:30.732 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:30.733 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.734 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.735 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.736 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.741 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.743 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:30.744 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.746 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.747 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.748 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.751 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:30.752 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.753 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.753 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.754 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.754 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.755 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:30.755 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.756 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.756 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:30.756 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:30.756 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.757 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.757 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:30.758 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:30.758 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:30.758 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:30.758 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.759 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.759 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.759 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.760 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:30.760 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.760 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.760 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.760 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.761 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.761 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.761 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.761 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.761 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.762 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.762 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.762 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.762 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.762 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.764 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:30.765 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.765 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.765 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:30.765 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.766 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.767 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:30.767 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.767 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.769 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:30.770 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:30.771 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.772 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.772 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.772 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.772 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.773 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:30.774 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.775 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:30.776 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.777 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.781 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.782 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.783 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.784 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.786 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:30.788 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.789 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.790 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.791 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:30.791 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:30.791 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:30.791 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:30.792 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.793 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.794 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:30.795 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.795 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.795 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.796 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.797 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.799 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.799 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:30.799 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.800 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:30.800 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.800 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:30.802 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.803 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.804 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.805 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.808 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:30.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:30.813 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:30.814 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:30.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:30.816 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:30.819 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:30.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:30.827 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:30.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.829 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:30.830 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.831 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.832 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.834 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.835 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.837 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.838 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.840 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:30.841 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:30.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:30.843 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:30.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:30.845 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:30.846 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.674+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f462406a050 msgr2=0x7f462406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.674+0000 7f4646e58640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f462406a050 0x7f462406c510 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f463c000f60 tx=0x7f463c007b90 comp rx=0 tx=0).stop 2023-08-25T09:52:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.674+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 msgr2=0x7f4640152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.674+0000 7f4646e58640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 0x7f4640152810 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f4638009d90 tx=0x7f4638004a40 comp rx=0 tx=0).stop 2023-08-25T09:52:30.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 shutdown_connections 2023-08-25T09:52:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f462406a050 0x7f462406c510 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f464015d3d0 0x7f4640152810 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4640159840 0x7f46400c4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 --2- 172.21.15.70:0/2514983415 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f46400cadb0 0x7f46400c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 >> 172.21.15.70:0/2514983415 conn(0x7f46400affd0 msgr2=0x7f46400b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:30.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 shutdown_connections 2023-08-25T09:52:30.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:30.675+0000 7f4646e58640 1 -- 172.21.15.70:0/2514983415 wait complete. 2023-08-25T09:52:30.850 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:52:35.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:36.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.077+0000 7f04bafbd640 1 Processor -- start 2023-08-25T09:52:36.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.077+0000 7f04bafbd640 1 -- start start 2023-08-25T09:52:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.078+0000 7f04bafbd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04b415a7c0 0x7f04b40b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.078+0000 7f04bafbd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 0x7f04b40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04bafbd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b415e7c0 con 0x7f04b40b12f0 2023-08-25T09:52:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04bafbd640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b40cb6f0 con 0x7f04b415a7c0 2023-08-25T09:52:36.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04bafbd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b4071b50 con 0x7f04b40cad20 2023-08-25T09:52:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04b415a7c0 0x7f04b40b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f04b415a7c0 0x7f04b40b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52832/0 (socket says 172.21.15.70:52832) 2023-08-25T09:52:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 -- 172.21.15.70:0/551926628 learned_addr learned my addr 172.21.15.70:0/551926628 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:36.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 -- 172.21.15.70:0/551926628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 msgr2=0x7f04b40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:36.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 --2- 172.21.15.70:0/551926628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 0x7f04b40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 -- 172.21.15.70:0/551926628 >> v1:172.21.15.155:6789/0 conn(0x7f04b40b12f0 legacy=0x7f04b415ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:36.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.079+0000 7f04b9fbb640 1 -- 172.21.15.70:0/551926628 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b415e490 con 0x7f04b415a7c0 2023-08-25T09:52:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.080+0000 7f04b9fbb640 1 --2- 172.21.15.70:0/551926628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40b3560 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009870 tx=0x7f04b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbec1d20a538309 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.081+0000 7f04b8fb9640 1 -- 172.21.15.70:0/551926628 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0031640 con 0x7f04b415a7c0 2023-08-25T09:52:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.081+0000 7f04b8fb9640 1 -- 172.21.15.70:0/551926628 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00317a0 con 0x7f04b415a7c0 2023-08-25T09:52:36.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.081+0000 7f04b8fb9640 1 -- 172.21.15.70:0/551926628 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0031b30 con 0x7f04b415a7c0 2023-08-25T09:52:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.081+0000 7f04bafbd640 1 -- 172.21.15.70:0/551926628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 msgr2=0x7f04b40b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.081+0000 7f04bafbd640 1 --2- 172.21.15.70:0/551926628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40b3560 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009870 tx=0x7f04b002b8f0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.082+0000 7f04bafbd640 1 -- 172.21.15.70:0/551926628 shutdown_connections 2023-08-25T09:52:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.082+0000 7f04bafbd640 1 --2- 172.21.15.70:0/551926628 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 0x7f04b40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.082+0000 7f04bafbd640 1 --2- 172.21.15.70:0/551926628 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40b3560 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.082+0000 7f04bafbd640 1 -- 172.21.15.70:0/551926628 >> 172.21.15.70:0/551926628 conn(0x7f04b40affd0 msgr2=0x7f04b40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:36.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.082+0000 7f04bafbd640 1 -- 172.21.15.70:0/551926628 shutdown_connections 2023-08-25T09:52:36.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.082+0000 7f04bafbd640 1 -- 172.21.15.70:0/551926628 wait complete. 2023-08-25T09:52:36.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.083+0000 7f04bafbd640 1 Processor -- start 2023-08-25T09:52:36.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.084+0000 7f04bafbd640 1 -- start start 2023-08-25T09:52:36.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04bafbd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b40b12f0 0x7f04b4158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04bafbd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 0x7f04b40709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04bafbd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04bafbd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b406f8d0 con 0x7f04b415a7c0 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04bafbd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b406ec50 con 0x7f04b40b12f0 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04bafbd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b406ef30 con 0x7f04b40cad20 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40cd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52840/0 (socket says 172.21.15.70:52840) 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 -- 172.21.15.70:0/1669690760 learned_addr learned my addr 172.21.15.70:0/1669690760 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04b9fbb640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b40b12f0 0x7f04b4158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 -- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 msgr2=0x7f04b40709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 0x7f04b40709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 -- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b40b12f0 msgr2=0x7f04b4158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b40b12f0 0x7f04b4158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.088 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.085+0000 7f04ba7bc640 1 -- 172.21.15.70:0/1669690760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00094d0 con 0x7f04b415a7c0 2023-08-25T09:52:36.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.086+0000 7f04ba7bc640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40cd0c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b970 tx=0x7f04ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:36.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.086+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00cb40 con 0x7f04b415a7c0 2023-08-25T09:52:36.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.086+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac00ccc0 con 0x7f04b415a7c0 2023-08-25T09:52:36.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.086+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac014680 con 0x7f04b415a7c0 2023-08-25T09:52:36.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.086+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b406f210 con 0x7f04b415a7c0 2023-08-25T09:52:36.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.086+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b4070f30 con 0x7f04b415a7c0 2023-08-25T09:52:36.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.087+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b406fa10 con 0x7f04b415a7c0 2023-08-25T09:52:36.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.088+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f04ac01d3b0 con 0x7f04b415a7c0 2023-08-25T09:52:36.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.089+0000 7f04aa7fc640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f049806a080 0x7f049806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.089+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f04ac0884f0 con 0x7f04b415a7c0 2023-08-25T09:52:36.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.089+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f04980721d0 con 0x7f04b415a7c0 2023-08-25T09:52:36.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.089+0000 7f04b9fbb640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f049806a080 0x7f049806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.090+0000 7f04b9fbb640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f049806a080 0x7f049806c540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f04b0036000 tx=0x7f04b003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:36.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.093+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f04b406fa10 con 0x7f04b415a7c0 2023-08-25T09:52:36.285 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.283+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f04b406fa10 con 0x7f04b415a7c0 2023-08-25T09:52:36.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.285+0000 7f04aa7fc640 1 -- 172.21.15.70:0/1669690760 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v200) v1 ==== 81+0+36986 (secure 0 0 0) 0x7f04b406fa10 con 0x7f04b415a7c0 2023-08-25T09:52:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:36.287 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:36.287 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 200, 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:32.511764+0000", 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:36.288 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:36.289 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:36.292 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:36.293 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:36.294 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:36.295 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:36.296 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:36.297 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.298 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:36.299 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.300 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:36.301 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:36.302 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:36.303 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:36.304 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:36.305 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:36.306 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:36.307 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:36.308 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:36.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:36.310 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.311 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:36.312 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:36.313 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:36.314 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:36.315 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:36.316 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:36.317 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:36.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:36.319 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.320 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:36.321 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.322 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:36.323 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:36.325 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:36.326 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:36.327 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "200", 2023-08-25T09:52:36.328 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:36.329 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:36.330 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:36.331 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:36.332 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:36.333 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:36.335 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:36.336 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.337 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.338 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.339 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:36.340 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:36.342 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.343 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.345 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.347 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.348 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.349 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.350 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.351 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.353 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.354 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.355 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.356 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.357 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.358 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.360 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.361 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.362 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:36.363 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.364 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.365 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.366 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.367 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.368 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.370 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.371 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:36.373 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.374 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.375 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.376 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.377 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.378 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.380 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.383 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.384 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:36.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.386 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.387 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.388 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.390 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.392 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.393 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:36.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:36.396 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.397 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.398 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:36.399 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.402 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:36.405 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:36.406 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:36.407 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:36.408 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:36.409 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.412 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:36.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:36.415 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:36.416 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:36.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:36.418 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.419 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.420 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.421 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.423 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.424 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:36.425 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:36.427 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:36.428 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:36.429 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:36.430 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:36.431 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f049806a080 msgr2=0x7f049806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f049806a080 0x7f049806c540 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f04b0036000 tx=0x7f04b003c040 comp rx=0 tx=0).stop 2023-08-25T09:52:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 msgr2=0x7f04b40cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40cd0c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00b970 tx=0x7f04ac00be30 comp rx=0 tx=0).stop 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 shutdown_connections 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f049806a080 0x7f049806c540 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f04b415a7c0 0x7f04b40cd0c0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f04b40cad20 0x7f04b40709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 --2- 172.21.15.70:0/1669690760 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f04b40b12f0 0x7f04b4158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 >> 172.21.15.70:0/1669690760 conn(0x7f04b40affd0 msgr2=0x7f04b40ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:36.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 shutdown_connections 2023-08-25T09:52:36.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.288+0000 7f04bafbd640 1 -- 172.21.15.70:0/1669690760 wait complete. 2023-08-25T09:52:36.435 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-08-25T09:52:36.436 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:52:36.436 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:36.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:36.495 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-08-25T09:52:36.710 INFO:teuthology.orchestra.run.smithi070.stderr:dd: error writing 'large_file_b': No space left on device 2023-08-25T09:52:36.710 INFO:teuthology.orchestra.run.smithi070.stderr:dd: closing output file 'large_file_b': No space left on device 2023-08-25T09:52:36.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:52:36.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:36.931 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.929+0000 7f91b5846640 1 Processor -- start 2023-08-25T09:52:36.932 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.929+0000 7f91b5846640 1 -- start start 2023-08-25T09:52:36.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.931+0000 7f91b5846640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 0x7f91b00cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.931+0000 7f91b5846640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00cc790 con 0x7f91b00ce480 2023-08-25T09:52:36.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.931+0000 7f91b5846640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b00cec90 con 0x7f91b00b0ec0 2023-08-25T09:52:36.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.931+0000 7f91b5846640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00cdf90 con 0x7f91b015d6b0 2023-08-25T09:52:36.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.932+0000 7f91ae7fc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f91b00ce480 0x7f91b015da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45440/0 (socket says 172.21.15.70:45440) 2023-08-25T09:52:36.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.932+0000 7f91ae7fc640 1 -- 172.21.15.70:0/2451958718 learned_addr learned my addr 172.21.15.70:0/2451958718 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:36.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.932+0000 7f91aeffd640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 0x7f91b00cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.932+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96263256 0 0) 0x7f91b00cc790 con 0x7f91b00ce480 2023-08-25T09:52:36.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.932+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9198003520 con 0x7f91b00ce480 2023-08-25T09:52:36.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.932+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1439023842 0 0) 0x7f9198003520 con 0x7f91b00ce480 2023-08-25T09:52:36.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 msgr2=0x7f91b00cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:36.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 0x7f91b00cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 >> v1:172.21.15.155:6789/0 conn(0x7f91b00b0ec0 legacy=0x7f91b00c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b00cb490 con 0x7f91b00ce480 2023-08-25T09:52:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f91a4001670 con 0x7f91b00ce480 2023-08-25T09:52:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 >> v1:172.21.15.70:6789/0 conn(0x7f91b00ce480 legacy=0x7f91b015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 0x7f91980039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.939 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9198003f20 0x7f91980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9198004d00 0x7f9198005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00cdf90 con 0x7f9198004d00 2023-08-25T09:52:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919800ba60 con 0x7f9198003f20 2023-08-25T09:52:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919800bc20 con 0x7f91b015d6b0 2023-08-25T09:52:36.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91af7fe640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9198004d00 0x7f9198005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91aeffd640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 0x7f91980039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91af7fe640 1 -- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 msgr2=0x7f91980039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.933+0000 7f91af7fe640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f91b015d6b0 0x7f91980039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.934+0000 7f91ae7fc640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9198003f20 0x7f91980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.934+0000 7f91af7fe640 1 -- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9198003f20 msgr2=0x7f91980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:36.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.934+0000 7f91af7fe640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9198003f20 0x7f91980047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.934+0000 7f91af7fe640 1 -- 172.21.15.70:0/2451958718 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9198003520 con 0x7f9198004d00 2023-08-25T09:52:36.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91af7fe640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9198004d00 0x7f9198005180 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f91a8001e50 tx=0x7f91980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b0977e1895118c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8003b70 con 0x7f9198004d00 2023-08-25T09:52:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a8003cd0 con 0x7f9198004d00 2023-08-25T09:52:36.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91adffb640 1 -- 172.21.15.70:0/2451958718 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8005310 con 0x7f9198004d00 2023-08-25T09:52:36.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91b5846640 1 -- 172.21.15.70:0/2451958718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9198004d00 msgr2=0x7f9198005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91b5846640 1 --2- 172.21.15.70:0/2451958718 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9198004d00 0x7f9198005180 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f91a8001e50 tx=0x7f91980057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.935+0000 7f91af7fe640 1 -- 172.21.15.70:0/2451958718 reap_dead start 2023-08-25T09:52:36.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.936+0000 7f91b5846640 1 -- 172.21.15.70:0/2451958718 shutdown_connections 2023-08-25T09:52:36.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.936+0000 7f91b5846640 1 -- 172.21.15.70:0/2451958718 >> 172.21.15.70:0/2451958718 conn(0x7f91b0069c70 msgr2=0x7f91b006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.936+0000 7f91b5846640 1 -- 172.21.15.70:0/2451958718 shutdown_connections 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.936+0000 7f91b5846640 1 -- 172.21.15.70:0/2451958718 wait complete. 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.937+0000 7f91b5846640 1 Processor -- start 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 -- start start 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9198004d00 0x7f91b00b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 0x7f91b014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b006f720 0x7f91b006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b0070d40 con 0x7f91b014c920 2023-08-25T09:52:36.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b014a4f0 con 0x7f91b006f720 2023-08-25T09:52:36.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.938+0000 7f91b5846640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b00704a0 con 0x7f9198004d00 2023-08-25T09:52:36.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 0x7f91b014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 0x7f91b014cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52856/0 (socket says 172.21.15.70:52856) 2023-08-25T09:52:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 -- 172.21.15.70:0/1314474685 learned_addr learned my addr 172.21.15.70:0/1314474685 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91aeffd640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9198004d00 0x7f91b00b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 -- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9198004d00 msgr2=0x7f91b00b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9198004d00 0x7f91b00b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 -- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b006f720 msgr2=0x7f91b006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91af7fe640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b006f720 0x7f91b006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b006f720 0x7f91b006fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.939+0000 7f91ae7fc640 1 -- 172.21.15.70:0/1314474685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0070780 con 0x7f91b014c920 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.940+0000 7f91ae7fc640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 0x7f91b014cda0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f91a400d7f0 tx=0x7f91a400dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.940+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40074e0 con 0x7f91b014c920 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.940+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a4007640 con 0x7f91b014c920 2023-08-25T09:52:36.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.940+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004a00 con 0x7f91b014c920 2023-08-25T09:52:36.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.940+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b0070a60 con 0x7f91b014c920 2023-08-25T09:52:36.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.940+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b00700e0 con 0x7f91b014c920 2023-08-25T09:52:36.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.941+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b014a6f0 con 0x7f91b014c920 2023-08-25T09:52:36.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.942+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f91a4004b60 con 0x7f91b014c920 2023-08-25T09:52:36.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.943+0000 7f9197fff640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919c06a050 0x7f919c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:36.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.943+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91a4087f60 con 0x7f91b014c920 2023-08-25T09:52:36.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.943+0000 7f91aeffd640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919c06a050 0x7f919c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:36.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.944+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f919c0721a0 con 0x7f91b014c920 2023-08-25T09:52:36.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.944+0000 7f91aeffd640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919c06a050 0x7f919c06c510 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009910 tx=0x7f91a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:36.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:36.948+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f91a4019020 con 0x7f91b014c920 2023-08-25T09:52:37.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.141+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f91b004e420 con 0x7f91b014c920 2023-08-25T09:52:37.147 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.145+0000 7f9197fff640 1 -- 172.21.15.70:0/1314474685 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v200) v1 ==== 81+0+36986 (secure 0 0 0) 0x7f91b014a6f0 con 0x7f91b014c920 2023-08-25T09:52:37.148 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:37.148 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 200, 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:32.511764+0000", 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:37.149 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:37.150 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:37.151 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:37.152 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:37.153 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:37.154 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:37.155 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:37.156 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:37.157 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:37.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:37.159 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:37.160 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:37.161 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:37.162 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:37.163 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:37.164 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:37.165 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.166 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:37.167 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:37.168 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:37.169 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:37.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:37.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:37.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:37.173 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:37.174 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:37.175 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.176 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:37.177 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:37.178 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:37.179 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.180 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:37.181 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:37.182 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:37.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:37.184 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:37.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:37.186 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:37.187 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:37.188 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.189 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:37.190 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:37.191 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:37.192 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:37.193 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:37.195 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:37.196 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:37.197 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:37.198 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "200", 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:37.199 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:37.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:37.201 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:37.202 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:37.203 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:37.204 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:37.206 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.207 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.209 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.213 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.215 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.216 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:37.217 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:37.218 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.220 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.221 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.223 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:37.224 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:37.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:37.227 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.229 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:37.230 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.232 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.233 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:37.238 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.239 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:37.240 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.241 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:37.241 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:37.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.242 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.242 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:37.242 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:37.242 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:37.242 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:37.243 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.244 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.244 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.244 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.245 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.245 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.245 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.245 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.246 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.246 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.246 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.246 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:37.247 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.248 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.248 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.248 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.249 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.249 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:37.249 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.249 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.250 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.252 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.254 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:37.254 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:37.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.255 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.256 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:37.256 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.256 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.257 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.257 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.257 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.257 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.258 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.258 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.258 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.258 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.258 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.258 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.260 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:37.260 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.260 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.260 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.260 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.261 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.261 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.261 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:37.261 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.264 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:37.264 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.264 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.264 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:37.265 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.266 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.267 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.268 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.268 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:37.268 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.268 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.269 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:37.270 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.273 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.274 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.274 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:37.274 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.275 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.276 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.276 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:37.276 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.277 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:37.277 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:37.278 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:37.279 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.279 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.279 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.279 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.280 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.282 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.282 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:37.282 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.282 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.284 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.287 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.288 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.289 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.290 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.290 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:37.290 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:37.290 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:37.291 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.292 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.293 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:37.294 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.295 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.299 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:37.300 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.301 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:37.302 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.304 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.305 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.306 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.308 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.309 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:37.310 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:37.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:37.312 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:37.313 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:37.314 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:37.315 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:37.316 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:37.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919c06a050 msgr2=0x7f919c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919c06a050 0x7f919c06c510 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009910 tx=0x7f91a0008040 comp rx=0 tx=0).stop 2023-08-25T09:52:37.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 msgr2=0x7f91b014cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 0x7f91b014cda0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f91a400d7f0 tx=0x7f91a400dcb0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.317 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 shutdown_connections 2023-08-25T09:52:37.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f919c06a050 0x7f919c06c510 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f91b006f720 0x7f91b006fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f91b014c920 0x7f91b014cda0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 --2- 172.21.15.70:0/1314474685 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f9198004d00 0x7f91b00b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.147+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 >> 172.21.15.70:0/1314474685 conn(0x7f91b0069c70 msgr2=0x7f91b00addb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:37.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.148+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 shutdown_connections 2023-08-25T09:52:37.319 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.148+0000 7f91b5846640 1 -- 172.21.15.70:0/1314474685 wait complete. 2023-08-25T09:52:37.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:52:37.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.542+0000 7f329f23f640 1 Processor -- start 2023-08-25T09:52:37.545 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.543+0000 7f329f23f640 1 -- start start 2023-08-25T09:52:37.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.544+0000 7f329f23f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f329815e830 0x7f329806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:37.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329f23f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32980b1b60 con 0x7f329815ec10 2023-08-25T09:52:37.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329f23f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329806b7c0 con 0x7f329806b2b0 2023-08-25T09:52:37.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329f23f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f329806bd80 con 0x7f329815e830 2023-08-25T09:52:37.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329cfdb640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f329815e830 0x7f329806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:37.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329d7dc640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f329815ec10 0x7f32980b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45452/0 (socket says 172.21.15.70:45452) 2023-08-25T09:52:37.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329d7dc640 1 -- 172.21.15.70:0/1641448730 learned_addr learned my addr 172.21.15.70:0/1641448730 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:37.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329cfdb640 1 -- 172.21.15.70:0/1641448730 >> v1:172.21.15.155:6789/0 conn(0x7f329806b2b0 legacy=0x7f329815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:37.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329cfdb640 1 -- 172.21.15.70:0/1641448730 >> v1:172.21.15.70:6789/0 conn(0x7f329815ec10 legacy=0x7f32980b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.545+0000 7f329cfdb640 1 -- 172.21.15.70:0/1641448730 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329806baa0 con 0x7f329815e830 2023-08-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.547+0000 7f329cfdb640 1 --2- 172.21.15.70:0/1641448730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329815e830 0x7f329806ad70 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f3288009d90 tx=0x7f328802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4574043271e98b9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.547+0000 7f32977fe640 1 -- 172.21.15.70:0/1641448730 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328802cc60 con 0x7f329815e830 2023-08-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.547+0000 7f32977fe640 1 -- 172.21.15.70:0/1641448730 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328802cdc0 con 0x7f329815e830 2023-08-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.547+0000 7f32977fe640 1 -- 172.21.15.70:0/1641448730 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288034660 con 0x7f329815e830 2023-08-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.547+0000 7f329f23f640 1 -- 172.21.15.70:0/1641448730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329815e830 msgr2=0x7f329806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.547+0000 7f329f23f640 1 --2- 172.21.15.70:0/1641448730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329815e830 0x7f329806ad70 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f3288009d90 tx=0x7f328802beb0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.548+0000 7f329f23f640 1 -- 172.21.15.70:0/1641448730 shutdown_connections 2023-08-25T09:52:37.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.548+0000 7f329f23f640 1 --2- 172.21.15.70:0/1641448730 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329815e830 0x7f329806ad70 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.548+0000 7f329f23f640 1 -- 172.21.15.70:0/1641448730 >> 172.21.15.70:0/1641448730 conn(0x7f3298069850 msgr2=0x7f3298069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:37.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.548+0000 7f329f23f640 1 -- 172.21.15.70:0/1641448730 shutdown_connections 2023-08-25T09:52:37.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.548+0000 7f329f23f640 1 -- 172.21.15.70:0/1641448730 wait complete. 2023-08-25T09:52:37.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.550+0000 7f329f23f640 1 Processor -- start 2023-08-25T09:52:37.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.550+0000 7f329f23f640 1 -- start start 2023-08-25T09:52:37.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329f23f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 0x7f3298155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:37.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329f23f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329815e830 0x7f3298153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:37.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329f23f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f329815ec10 0x7f329815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:37.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329f23f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32981f0430 con 0x7f329806b2b0 2023-08-25T09:52:37.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329f23f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32981f06d0 con 0x7f329815e830 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329f23f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32981f09b0 con 0x7f329815ec10 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329cfdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 0x7f3298155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329cfdb640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 0x7f3298155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52884/0 (socket says 172.21.15.70:52884) 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.551+0000 7f329cfdb640 1 -- 172.21.15.70:0/2644544802 learned_addr learned my addr 172.21.15.70:0/2644544802 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f3297fff640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329815e830 0x7f3298153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329cfdb640 1 -- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f329815ec10 msgr2=0x7f329815cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329cfdb640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f329815ec10 0x7f329815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329cfdb640 1 -- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329815e830 msgr2=0x7f3298153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329cfdb640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329815e830 0x7f3298153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329cfdb640 1 -- 172.21.15.70:0/2644544802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288009a70 con 0x7f329806b2b0 2023-08-25T09:52:37.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329cfdb640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 0x7f3298155f60 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f3288037040 tx=0x7f3288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288040650 con 0x7f329806b2b0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32880407b0 con 0x7f329806b2b0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288040b60 con 0x7f329806b2b0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.552+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329815ca20 con 0x7f329806b2b0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.553+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3298156850 con 0x7f329806b2b0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.554+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f329804e420 con 0x7f329806b2b0 2023-08-25T09:52:37.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.554+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3288040cc0 con 0x7f329806b2b0 2023-08-25T09:52:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.555+0000 7f32957fa640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f327806a050 0x7f327806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.555+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32880ab600 con 0x7f329806b2b0 2023-08-25T09:52:37.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.556+0000 7f3297fff640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f327806a050 0x7f327806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:37.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.556+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f32780721a0 con 0x7f329806b2b0 2023-08-25T09:52:37.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.556+0000 7f3297fff640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f327806a050 0x7f327806c510 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3280006fd0 tx=0x7f3280008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:37.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.560+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3288076c50 con 0x7f329806b2b0 2023-08-25T09:52:37.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.787+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f329806c620 con 0x7f329806b2b0 2023-08-25T09:52:37.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.788+0000 7f32957fa640 1 -- 172.21.15.70:0/2644544802 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f328807a270 con 0x7f329806b2b0 2023-08-25T09:52:37.791 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:37.791 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:52:37.791 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:52:37.793 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f327806a050 msgr2=0x7f327806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f327806a050 0x7f327806c510 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3280006fd0 tx=0x7f3280008040 comp rx=0 tx=0).stop 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 msgr2=0x7f3298155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 0x7f3298155f60 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f3288037040 tx=0x7f3288004a40 comp rx=0 tx=0).stop 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 shutdown_connections 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f327806a050 0x7f327806c510 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f329815ec10 0x7f329815cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f329815e830 0x7f3298153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 --2- 172.21.15.70:0/2644544802 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f329806b2b0 0x7f3298155f60 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:37.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 >> 172.21.15.70:0/2644544802 conn(0x7f3298069850 msgr2=0x7f3298159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:37.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 shutdown_connections 2023-08-25T09:52:37.795 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:37.791+0000 7f329f23f640 1 -- 172.21.15.70:0/2644544802 wait complete. 2023-08-25T09:52:37.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:52:38.043 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":9573,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":33.397869514999996,"mdsmap_epoch":91,"osdmap_epoch":200,"osdmap_epoch_barrier":198,"uptime":38.793783824000002} 2023-08-25T09:52:38.044 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 9573, "mdsmap_epoch": 91, "osdmap_epoch": 200, "osdmap_epoch_barrier": 198, "rank_uptime": 33.397869515, "state": "up:active", "uptime": 38.793783824, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:52:38.044 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:52:38.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:38.045 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-08-25T09:52:38.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:38.105 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-08-25T09:52:38.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.382+0000 7f73c2e8e640 1 Processor -- start 2023-08-25T09:52:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.382+0000 7f73c2e8e640 1 -- start start 2023-08-25T09:52:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.383+0000 7f73c2e8e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73bc0b1ff0 0x7f73bc0b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:38.385 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.383+0000 7f73c2e8e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc0b1b60 con 0x7f73bc15e830 2023-08-25T09:52:38.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.383+0000 7f73c2e8e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc06b7c0 con 0x7f73bc15ec10 2023-08-25T09:52:38.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.383+0000 7f73c2e8e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc06bd80 con 0x7f73bc0b1ff0 2023-08-25T09:52:38.386 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.383+0000 7f73c142b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73bc0b1ff0 0x7f73bc0b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:38.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.383+0000 7f73c142b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f73bc0b1ff0 0x7f73bc0b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52890/0 (socket says 172.21.15.70:52890) 2023-08-25T09:52:38.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.384+0000 7f73c142b640 1 -- 172.21.15.70:0/3650287426 learned_addr learned my addr 172.21.15.70:0/3650287426 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:38.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.384+0000 7f73c0c2a640 1 --1- 172.21.15.70:0/3650287426 >> v1:172.21.15.70:6789/0 conn(0x7f73bc15e830 0x7f73bc06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:45454/0 (socket says 172.21.15.70:45454) 2023-08-25T09:52:38.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.384+0000 7f73c142b640 1 -- 172.21.15.70:0/3650287426 >> v1:172.21.15.155:6789/0 conn(0x7f73bc15ec10 legacy=0x7f73bc15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:38.387 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.384+0000 7f73c142b640 1 -- 172.21.15.70:0/3650287426 >> v1:172.21.15.70:6789/0 conn(0x7f73bc15e830 legacy=0x7f73bc06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:38.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.384+0000 7f73c142b640 1 -- 172.21.15.70:0/3650287426 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc06baa0 con 0x7f73bc0b1ff0 2023-08-25T09:52:38.388 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.384+0000 7f73b37fe640 1 -- 172.21.15.70:0/3650287426 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688845509 0 0) 0x7f73bc0b1b60 con 0x7f73bc15e830 2023-08-25T09:52:38.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.385+0000 7f73c142b640 1 --2- 172.21.15.70:0/3650287426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc0b1ff0 0x7f73bc0b2440 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009870 tx=0x7f73b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c9c5da24927562 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:38.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.385+0000 7f73b37fe640 1 -- 172.21.15.70:0/3650287426 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4030860 con 0x7f73bc0b1ff0 2023-08-25T09:52:38.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.385+0000 7f73b37fe640 1 -- 172.21.15.70:0/3650287426 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b40309c0 con 0x7f73bc0b1ff0 2023-08-25T09:52:38.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.385+0000 7f73b37fe640 1 -- 172.21.15.70:0/3650287426 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4030d50 con 0x7f73bc0b1ff0 2023-08-25T09:52:38.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3650287426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc0b1ff0 msgr2=0x7f73bc0b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:38.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3650287426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc0b1ff0 0x7f73bc0b2440 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009870 tx=0x7f73b402b9a0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3650287426 shutdown_connections 2023-08-25T09:52:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3650287426 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc0b1ff0 0x7f73bc0b2440 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3650287426 >> 172.21.15.70:0/3650287426 conn(0x7f73bc069850 msgr2=0x7f73bc069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3650287426 shutdown_connections 2023-08-25T09:52:38.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.386+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3650287426 wait complete. 2023-08-25T09:52:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.388+0000 7f73c2e8e640 1 Processor -- start 2023-08-25T09:52:38.391 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.388+0000 7f73c2e8e640 1 -- start start 2023-08-25T09:52:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c2e8e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc0b1ff0 0x7f73bc0ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c2e8e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 0x7f73bc1f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c2e8e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73bc15ec10 0x7f73bc155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:38.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c2e8e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc154870 con 0x7f73bc15e830 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c2e8e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc154b50 con 0x7f73bc15ec10 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c2e8e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc154e30 con 0x7f73bc0b1ff0 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 0x7f73bc1f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 0x7f73bc1f25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:52902/0 (socket says 172.21.15.70:52902) 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73b3fff640 1 -- 172.21.15.70:0/3398104964 learned_addr learned my addr 172.21.15.70:0/3398104964 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.389+0000 7f73c0c2a640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc0b1ff0 0x7f73bc0ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:38.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73c142b640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73bc15ec10 0x7f73bc155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b3fff640 1 -- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc0b1ff0 msgr2=0x7f73bc0ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc0b1ff0 0x7f73bc0ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b3fff640 1 -- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73bc15ec10 msgr2=0x7f73bc155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73bc15ec10 0x7f73bc155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b3fff640 1 -- 172.21.15.70:0/3398104964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b40094d0 con 0x7f73bc15e830 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73c0c2a640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc0b1ff0 0x7f73bc0ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73c142b640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73bc15ec10 0x7f73bc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:38.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b3fff640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 0x7f73bc1f25c0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b3c0 tx=0x7f73ac00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:38.395 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011830 con 0x7f73bc15e830 2023-08-25T09:52:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac011990 con 0x7f73bc15e830 2023-08-25T09:52:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc155110 con 0x7f73bc15e830 2023-08-25T09:52:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac011d80 con 0x7f73bc15e830 2023-08-25T09:52:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.390+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc154500 con 0x7f73bc15e830 2023-08-25T09:52:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.391+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc04e420 con 0x7f73bc15e830 2023-08-25T09:52:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.392+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f73ac01b460 con 0x7f73bc15e830 2023-08-25T09:52:38.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.393+0000 7f73b17fa640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:38.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.393+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f73ac0881b0 con 0x7f73bc15e830 2023-08-25T09:52:38.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.394+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7f73a80721a0 con 0x7f73bc15e830 2023-08-25T09:52:38.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.394+0000 7f73c0c2a640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:38.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.394+0000 7f73c0c2a640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f73b80012e0 tx=0x7f73b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:38.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.398+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f73ac053800 con 0x7f73bc15e830 2023-08-25T09:52:38.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.591+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f73bc06b450 con 0x7f73bc15e830 2023-08-25T09:52:38.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.594+0000 7f73b17fa640 1 -- 172.21.15.70:0/3398104964 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v200) v1 ==== 81+0+36986 (secure 0 0 0) 0x7f73ac056e20 con 0x7f73bc15e830 2023-08-25T09:52:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:38.598 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:38.598 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 200, 2023-08-25T09:52:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:38.599 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:32.511764+0000", 2023-08-25T09:52:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:38.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:38.601 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:38.602 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:38.603 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:38.603 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:38.603 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:38.603 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:38.603 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:38.603 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:38.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:38.605 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:38.606 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:38.607 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:38.608 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:38.608 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:38.608 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:38.608 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:38.608 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:38.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:38.609 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:38.610 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:38.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:38.612 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:38.613 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:38.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:38.615 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:38.616 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:38.617 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:38.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:38.619 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:38.620 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:38.621 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:38.622 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.623 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:38.624 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.625 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:38.626 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:38.627 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:38.628 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:38.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:38.630 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:38.630 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:38.630 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:38.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:38.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:38.631 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:38.631 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:38.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:38.632 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:38.633 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:38.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:38.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:38.636 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.637 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:38.638 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:38.640 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:38.641 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:38.642 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "200", 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:38.643 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:38.644 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:38.645 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:38.646 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:38.647 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:38.648 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.649 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:38.650 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:38.651 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.652 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:38.653 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.656 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:38.660 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:38.661 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.662 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:38.663 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.664 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.668 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.669 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:38.670 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.671 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.672 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:38.674 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.674 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:38.676 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.676 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.677 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.677 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.678 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:38.679 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.680 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:38.680 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.680 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.680 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.681 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.681 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:38.682 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.684 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:38.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:38.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:38.686 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.687 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.688 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.689 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.689 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.689 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.689 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.689 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.691 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.692 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:38.692 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.692 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:38.693 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.693 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.693 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.693 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:38.696 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:38.697 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.698 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.699 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:38.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:38.703 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.704 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.705 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:38.706 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.708 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:38.708 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:38.708 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:38.708 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:38.708 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.709 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.709 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.709 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.709 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.710 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.711 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:38.712 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.712 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.712 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.712 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:38.712 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.714 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:38.715 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.716 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:38.716 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:38.717 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.718 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.718 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:38.718 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:38.718 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:38.719 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.720 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.720 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.720 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.720 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.721 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.722 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.723 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.724 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:38.725 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.726 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.727 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:38.728 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.730 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:38.731 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:38.732 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.733 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:38.735 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.736 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.737 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:38.738 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.739 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:38.740 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:38.741 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:38.741 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.741 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:38.741 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:38.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.741 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.743 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.744 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:38.745 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.747 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.748 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:38.749 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:38.751 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:38.752 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:38.753 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:38.754 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:38.755 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:38.756 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:38.756 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:38.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 msgr2=0x7f73a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:38.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f73b80012e0 tx=0x7f73b8009040 comp rx=0 tx=0).stop 2023-08-25T09:52:38.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 msgr2=0x7f73bc1f25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:38.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 0x7f73bc1f25c0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00b3c0 tx=0x7f73ac00b880 comp rx=0 tx=0).stop 2023-08-25T09:52:38.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 shutdown_connections 2023-08-25T09:52:38.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f73a806a050 0x7f73a806c510 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f73bc15ec10 0x7f73bc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f73bc15e830 0x7f73bc1f25c0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 --2- 172.21.15.70:0/3398104964 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f73bc0b1ff0 0x7f73bc0ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:38.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 >> 172.21.15.70:0/3398104964 conn(0x7f73bc069850 msgr2=0x7f73bc0ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:38.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 shutdown_connections 2023-08-25T09:52:38.759 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:38.597+0000 7f73c2e8e640 1 -- 172.21.15.70:0/3398104964 wait complete. 2023-08-25T09:52:38.760 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:52:43.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:43.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.982+0000 7fb572d22640 1 Processor -- start 2023-08-25T09:52:43.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.983+0000 7fb572d22640 1 -- start start 2023-08-25T09:52:43.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb572d22640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fb56c15ec10 0x7fb56c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb572d22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 0x7fb56c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb572d22640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56c0b1b60 con 0x7fb56c15e830 2023-08-25T09:52:43.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb572d22640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c06bd80 con 0x7fb56c15ec10 2023-08-25T09:52:43.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb572d22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c06b7c0 con 0x7fb56c15c6d0 2023-08-25T09:52:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb570abe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb56c15e830 0x7fb56c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:39964/0 (socket says 172.21.15.70:39964) 2023-08-25T09:52:43.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb570abe640 1 -- 172.21.15.70:0/1872181909 learned_addr learned my addr 172.21.15.70:0/1872181909 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb5712bf640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 0x7fb56c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.985+0000 7fb56bfff640 1 --2- 172.21.15.70:0/1872181909 >> v2:172.21.15.155:3300/0 conn(0x7fb56c15ec10 0x7fb56c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883437337 0 0) 0x7fb56c0b1b60 con 0x7fb56c15e830 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb554003520 con 0x7fb56c15e830 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2391338066 0 0) 0x7fb554003520 con 0x7fb56c15e830 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 msgr2=0x7fb56c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:43.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 0x7fb56c0b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 >> v2:172.21.15.155:3300/0 conn(0x7fb56c15ec10 msgr2=0x7fb56c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 --2- 172.21.15.70:0/1872181909 >> v2:172.21.15.155:3300/0 conn(0x7fb56c15ec10 0x7fb56c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c06baa0 con 0x7fb56c15e830 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb55c0019f0 con 0x7fb56c15e830 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 >> v1:172.21.15.70:6789/0 conn(0x7fb56c15e830 legacy=0x7fb56c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb554003970 0x7fb554003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5540047c0 0x7fb554004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 0x7fb554005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c06bd80 con 0x7fb554003970 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c06b7c0 con 0x7fb5540047c0 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55400b980 con 0x7fb56c15c6d0 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.986+0000 7fb570abe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb554003970 0x7fb554003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb56bfff640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5540047c0 0x7fb554004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb570abe640 1 -- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 msgr2=0x7fb554005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb5712bf640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 0x7fb554005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb570abe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb56c15c6d0 0x7fb554005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb570abe640 1 -- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5540047c0 msgr2=0x7fb554004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb570abe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb5540047c0 0x7fb554004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.987+0000 7fb570abe640 1 -- 172.21.15.70:0/1872181909 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554003520 con 0x7fb554003970 2023-08-25T09:52:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.988+0000 7fb570abe640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb554003970 0x7fb554003d90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00a860 tx=0x7fb5540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610f1052c3b69b26 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.988+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c034630 con 0x7fb554003970 2023-08-25T09:52:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.988+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c02f070 con 0x7fb554003970 2023-08-25T09:52:43.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.988+0000 7fb56b7fe640 1 -- 172.21.15.70:0/1872181909 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c0349f0 con 0x7fb554003970 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb572d22640 1 -- 172.21.15.70:0/1872181909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb554003970 msgr2=0x7fb554003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb572d22640 1 --2- 172.21.15.70:0/1872181909 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb554003970 0x7fb554003d90 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fb55c00a860 tx=0x7fb5540057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb5712bf640 1 -- 172.21.15.70:0/1872181909 reap_dead start 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb572d22640 1 -- 172.21.15.70:0/1872181909 shutdown_connections 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb572d22640 1 -- 172.21.15.70:0/1872181909 >> 172.21.15.70:0/1872181909 conn(0x7fb56c069850 msgr2=0x7fb56c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb572d22640 1 -- 172.21.15.70:0/1872181909 shutdown_connections 2023-08-25T09:52:43.996 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.989+0000 7fb572d22640 1 -- 172.21.15.70:0/1872181909 wait complete. 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.991+0000 7fb572d22640 1 Processor -- start 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.991+0000 7fb572d22640 1 -- start start 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb572d22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb554003970 0x7fb56c0a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb572d22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 0x7fb56c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb572d22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c155b10 0x7fb56c155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb572d22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c1564d0 con 0x7fb56c1f1f10 2023-08-25T09:52:43.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb572d22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c144c50 con 0x7fb56c155b10 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb572d22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0a8a40 con 0x7fb554003970 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb56bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 0x7fb56c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb56bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 0x7fb56c0aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54460/0 (socket says 172.21.15.70:54460) 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.992+0000 7fb56bfff640 1 -- 172.21.15.70:0/2627612771 learned_addr learned my addr 172.21.15.70:0/2627612771 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb570abe640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb554003970 0x7fb56c0a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb5712bf640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c155b10 0x7fb56c155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb56bfff640 1 -- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb554003970 msgr2=0x7fb56c0a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:43.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb56bfff640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb554003970 0x7fb56c0a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb56bfff640 1 -- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c155b10 msgr2=0x7fb56c155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb56bfff640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c155b10 0x7fb56c155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb56bfff640 1 -- 172.21.15.70:0/2627612771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0a8d20 con 0x7fb56c1f1f10 2023-08-25T09:52:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb570abe640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb554003970 0x7fb56c0a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb5712bf640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c155b10 0x7fb56c155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:43.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb56bfff640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 0x7fb56c0aeed0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fb5600115d0 tx=0x7fb560011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:44.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560015870 con 0x7fb56c1f1f10 2023-08-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5600159d0 con 0x7fb56c1f1f10 2023-08-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c0a9000 con 0x7fb56c1f1f10 2023-08-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560015d80 con 0x7fb56c1f1f10 2023-08-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.993+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c1f2390 con 0x7fb56c1f1f10 2023-08-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.995+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c04e420 con 0x7fb56c1f1f10 2023-08-25T09:52:44.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.995+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb56000f480 con 0x7fb56c1f1f10 2023-08-25T09:52:44.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.996+0000 7fb5697fa640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb55806a080 0x7fb55806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:44.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.996+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb560088d90 con 0x7fb56c1f1f10 2023-08-25T09:52:44.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.997+0000 7fb570abe640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb55806a080 0x7fb55806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:44.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.997+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=201}) v3 -- 0x7fb5580721d0 con 0x7fb56c1f1f10 2023-08-25T09:52:44.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:43.997+0000 7fb570abe640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb55806a080 0x7fb55806c540 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034000 tx=0x7fb55c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:44.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.001+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb5600543e0 con 0x7fb56c1f1f10 2023-08-25T09:52:44.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.193+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb56c06c620 con 0x7fb56c1f1f10 2023-08-25T09:52:44.198 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.195+0000 7fb5697fa640 1 -- 172.21.15.70:0/2627612771 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v200) v1 ==== 81+0+36986 (secure 0 0 0) 0x7fb560057a00 con 0x7fb56c1f1f10 2023-08-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 200, 2023-08-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:44.199 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:32.511764+0000", 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:44.200 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:44.201 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:44.202 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:44.202 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:44.202 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:44.202 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:44.202 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:44.203 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:44.204 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:44.205 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:44.206 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:44.207 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:44.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:44.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:44.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:44.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:44.208 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:44.209 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:44.210 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:44.210 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:44.211 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.212 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:44.213 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:44.214 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:44.215 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:44.216 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:44.217 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:44.218 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:44.219 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:44.220 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:44.221 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:44.222 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:44.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:44.224 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:44.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:44.226 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:44.227 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:44.227 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.227 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:44.228 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:44.229 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:44.230 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:44.231 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:44.232 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:44.233 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:44.234 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:44.235 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:44.236 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:44.237 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.238 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:44.238 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:44.238 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:44.238 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:44.238 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1027, 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,full,full_quota", 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:44.240 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:44.241 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:44.242 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.243 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "200", 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:44.244 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:44.245 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:44.246 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:44.247 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:44.248 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.249 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:44.250 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.251 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.252 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.253 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:44.256 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.257 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.258 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.261 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.262 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.263 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.265 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.266 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.267 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.268 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.269 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:44.270 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:44.271 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.272 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.273 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.277 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.279 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.280 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:44.281 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.282 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.283 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.284 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:44.286 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.289 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.290 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:44.291 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.292 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.293 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.294 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.295 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:44.296 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.297 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.299 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:44.301 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.303 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:44.305 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.306 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.307 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.309 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.311 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.313 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.314 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.315 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.317 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:44.318 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.321 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:44.323 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:44.324 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.325 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.328 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.329 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.331 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:44.332 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.333 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:44.334 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:44.336 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.337 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.338 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:44.339 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.341 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:44.342 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:44.343 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:44.344 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:44.345 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:44.346 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:44.347 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:44.347 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:44.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb55806a080 msgr2=0x7fb55806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:44.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb55806a080 0x7fb55806c540 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb55c034000 tx=0x7fb55c009030 comp rx=0 tx=0).stop 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 msgr2=0x7fb56c0aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 0x7fb56c0aeed0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fb5600115d0 tx=0x7fb560011a90 comp rx=0 tx=0).stop 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 shutdown_connections 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb55806a080 0x7fb55806c540 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb56c155b10 0x7fb56c155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb56c1f1f10 0x7fb56c0aeed0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:44.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 --2- 172.21.15.70:0/2627612771 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb554003970 0x7fb56c0a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 >> 172.21.15.70:0/2627612771 conn(0x7fb56c069850 msgr2=0x7fb56c15ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 shutdown_connections 2023-08-25T09:52:44.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:44.199+0000 7fb572d22640 1 -- 172.21.15.70:0/2627612771 wait complete. 2023-08-25T09:52:44.350 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-08-25T09:52:48.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:52:48.204 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:52:49.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.574+0000 7f7241a06640 1 Processor -- start 2023-08-25T09:52:49.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.574+0000 7f7241a06640 1 -- start start 2023-08-25T09:52:49.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f7241a06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 0x7f723c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f7241a06640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f723c06b2b0 0x7f723c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f7241a06640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f723c0b1b60 con 0x7f723c15c6d0 2023-08-25T09:52:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f7241a06640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f723c06bd80 con 0x7f723c06b2b0 2023-08-25T09:52:49.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f7241a06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c06b7c0 con 0x7f723c15e830 2023-08-25T09:52:49.580 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f723affd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 0x7f723c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f723affd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 0x7f723c06ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46822/0 (socket says 172.21.15.70:46822) 2023-08-25T09:52:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.576+0000 7f723affd640 1 -- 172.21.15.70:0/1925333487 learned_addr learned my addr 172.21.15.70:0/1925333487 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:49.581 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.577+0000 7f7233fff640 1 --2- 172.21.15.70:0/1925333487 >> v2:172.21.15.155:3300/0 conn(0x7f723c06b2b0 0x7f723c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.577+0000 7f723affd640 1 -- 172.21.15.70:0/1925333487 >> v2:172.21.15.155:3300/0 conn(0x7f723c06b2b0 msgr2=0x7f723c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.577+0000 7f723affd640 1 --2- 172.21.15.70:0/1925333487 >> v2:172.21.15.155:3300/0 conn(0x7f723c06b2b0 0x7f723c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.577+0000 7f723affd640 1 -- 172.21.15.70:0/1925333487 >> v1:172.21.15.155:6789/0 conn(0x7f723c15c6d0 legacy=0x7f723c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:49.582 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.577+0000 7f723affd640 1 -- 172.21.15.70:0/1925333487 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c06baa0 con 0x7f723c15e830 2023-08-25T09:52:49.583 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.579+0000 7f723affd640 1 --2- 172.21.15.70:0/1925333487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 0x7f723c06ad70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f722c009870 tx=0x7f722c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebf843d040b37aa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.579+0000 7f723a7fc640 1 -- 172.21.15.70:0/1925333487 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c030840 con 0x7f723c15e830 2023-08-25T09:52:49.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f723a7fc640 1 -- 172.21.15.70:0/1925333487 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722c0309a0 con 0x7f723c15e830 2023-08-25T09:52:49.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f723a7fc640 1 -- 172.21.15.70:0/1925333487 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722c030d30 con 0x7f723c15e830 2023-08-25T09:52:49.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 -- 172.21.15.70:0/1925333487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 msgr2=0x7f723c06ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:49.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 --2- 172.21.15.70:0/1925333487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 0x7f723c06ad70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f722c009870 tx=0x7f722c02b980 comp rx=0 tx=0).stop 2023-08-25T09:52:49.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 -- 172.21.15.70:0/1925333487 shutdown_connections 2023-08-25T09:52:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 --2- 172.21.15.70:0/1925333487 >> v2:172.21.15.155:3300/0 conn(0x7f723c06b2b0 0x7f723c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 --2- 172.21.15.70:0/1925333487 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c15e830 0x7f723c06ad70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 -- 172.21.15.70:0/1925333487 >> 172.21.15.70:0/1925333487 conn(0x7f723c069850 msgr2=0x7f723c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 -- 172.21.15.70:0/1925333487 shutdown_connections 2023-08-25T09:52:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.580+0000 7f7241a06640 1 -- 172.21.15.70:0/1925333487 wait complete. 2023-08-25T09:52:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.582+0000 7f7241a06640 1 Processor -- start 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.582+0000 7f7241a06640 1 -- start start 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.583+0000 7f7241a06640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c06b2b0 0x7f723c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.583+0000 7f7241a06640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f723c15c6d0 0x7f723c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.583+0000 7f7241a06640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 0x7f723c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.583+0000 7f7241a06640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c156670 con 0x7f723c15e830 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.583+0000 7f7241a06640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c156fd0 con 0x7f723c15c6d0 2023-08-25T09:52:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.583+0000 7f7241a06640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c0ca820 con 0x7f723c06b2b0 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 0x7f723c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 0x7f723c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39470/0 (socket says 172.21.15.70:39470) 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 -- 172.21.15.70:0/1000433762 learned_addr learned my addr 172.21.15.70:0/1000433762 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f7233fff640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f723c15c6d0 0x7f723c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723affd640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c06b2b0 0x7f723c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 -- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c06b2b0 msgr2=0x7f723c155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:49.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c06b2b0 0x7f723c155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 -- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f723c15c6d0 msgr2=0x7f723c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:49.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f723c15c6d0 0x7f723c156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.584+0000 7f723b7fe640 1 -- 172.21.15.70:0/1000433762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c0094d0 con 0x7f723c15e830 2023-08-25T09:52:49.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.585+0000 7f723b7fe640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 0x7f723c0ca2e0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f723400c7d0 tx=0x7f723400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:49.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.585+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234003b70 con 0x7f723c15e830 2023-08-25T09:52:49.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.585+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234003cf0 con 0x7f723c15e830 2023-08-25T09:52:49.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.585+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723400aa90 con 0x7f723c15e830 2023-08-25T09:52:49.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.585+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723c158840 con 0x7f723c15e830 2023-08-25T09:52:49.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.585+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723c155fb0 con 0x7f723c15e830 2023-08-25T09:52:49.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.586+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f723c1567b0 con 0x7f723c15e830 2023-08-25T09:52:49.593 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.587+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f723400abf0 con 0x7f723c15e830 2023-08-25T09:52:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.588+0000 7f7238ff9640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f721c06a050 0x7f721c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.588+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f72340883d0 con 0x7f723c15e830 2023-08-25T09:52:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.588+0000 7f723affd640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f721c06a050 0x7f721c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.589+0000 7f723affd640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f721c06a050 0x7f721c06c510 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f722c036000 tx=0x7f722c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:49.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.593+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f723c1567b0 con 0x7f723c15e830 2023-08-25T09:52:49.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.784+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f723c1567b0 con 0x7f723c15e830 2023-08-25T09:52:49.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.788+0000 7f7238ff9640 1 -- 172.21.15.70:0/1000433762 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v201) v1 ==== 81+0+36967 (secure 0 0 0) 0x7f723c1567b0 con 0x7f723c15e830 2023-08-25T09:52:49.790 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:49.790 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:49.790 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 201, 2023-08-25T09:52:49.790 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:47.518383+0000", 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:49.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:49.792 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:49.793 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:49.794 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:49.795 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:49.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:49.797 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:49.798 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:49.799 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:49.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:49.801 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:49.802 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:49.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:49.804 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:49.805 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:49.806 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:49.807 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:49.808 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:49.809 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:49.810 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:49.811 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:49.812 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:49.813 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:49.814 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:49.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:49.816 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:49.817 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:49.818 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.819 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:49.820 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:49.821 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:49.822 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:49.823 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:49.824 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:49.825 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:49.826 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.827 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:49.828 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:49.829 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:49.830 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "201", 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:49.831 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:49.832 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:49.833 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:49.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:49.835 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:49.836 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:49.837 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:49.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.839 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.840 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.842 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:49.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:49.848 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.849 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.850 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:49.857 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:49.858 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.859 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.860 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.862 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.864 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:49.867 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.868 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.869 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.870 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.872 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:49.875 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.876 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:49.877 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.878 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.879 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.880 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.882 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.883 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.885 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.886 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:49.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.888 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.889 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.890 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:49.891 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.894 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.895 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.896 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.897 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:49.898 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.899 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.900 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.901 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.902 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:49.903 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.906 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.907 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:49.908 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:49.910 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.911 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.912 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:49.913 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.916 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:49.918 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.919 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.920 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.921 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:49.922 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.924 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.925 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.926 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:49.928 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:49.929 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:49.930 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:49.931 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:49.932 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:49.933 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:49.934 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:49.934 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:49.934 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.790+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f721c06a050 msgr2=0x7f721c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:49.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.790+0000 7f7241a06640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f721c06a050 0x7f721c06c510 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f722c036000 tx=0x7f722c03b040 comp rx=0 tx=0).stop 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.790+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 msgr2=0x7f723c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.790+0000 7f7241a06640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 0x7f723c0ca2e0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f723400c7d0 tx=0x7f723400cc90 comp rx=0 tx=0).stop 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 shutdown_connections 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f721c06a050 0x7f721c06c510 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f723c15e830 0x7f723c0ca2e0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f723c15c6d0 0x7f723c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.936 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 --2- 172.21.15.70:0/1000433762 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f723c06b2b0 0x7f723c155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 >> 172.21.15.70:0/1000433762 conn(0x7f723c069850 msgr2=0x7f723c0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 shutdown_connections 2023-08-25T09:52:49.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:49.791+0000 7f7241a06640 1 -- 172.21.15.70:0/1000433762 wait complete. 2023-08-25T09:52:49.937 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-08-25T09:52:49.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-08-25T09:52:50.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.159+0000 7efdd675d640 1 Processor -- start 2023-08-25T09:52:50.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.159+0000 7efdd675d640 1 -- start start 2023-08-25T09:52:50.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.160+0000 7efdd675d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7efdd015aa10 0x7efdd00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.160+0000 7efdd675d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00b2ad0 con 0x7efdd00ce5b0 2023-08-25T09:52:50.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.160+0000 7efdd675d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd015d3f0 con 0x7efdd00b2680 2023-08-25T09:52:50.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.160+0000 7efdd675d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd0159d10 con 0x7efdd015aa10 2023-08-25T09:52:50.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.161+0000 7efdcffff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7efdd00ce5b0 0x7efdd00ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34514/0 (socket says 172.21.15.70:34514) 2023-08-25T09:52:50.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.161+0000 7efdcffff640 1 -- 172.21.15.70:0/552476656 learned_addr learned my addr 172.21.15.70:0/552476656 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:50.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.161+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284901339 0 0) 0x7efdd00b2ad0 con 0x7efdd00ce5b0 2023-08-25T09:52:50.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.161+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdbc003520 con 0x7efdd00ce5b0 2023-08-25T09:52:50.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.161+0000 7efdcf7fe640 1 --2- 172.21.15.70:0/552476656 >> v2:172.21.15.155:3300/0 conn(0x7efdd015aa10 0x7efdd00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042676577 0 0) 0x7efdbc003520 con 0x7efdd00ce5b0 2023-08-25T09:52:50.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 >> v2:172.21.15.155:3300/0 conn(0x7efdd015aa10 msgr2=0x7efdd00b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:50.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 --2- 172.21.15.70:0/552476656 >> v2:172.21.15.155:3300/0 conn(0x7efdd015aa10 0x7efdd00b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 >> v1:172.21.15.155:6789/0 conn(0x7efdd00b2680 legacy=0x7efdd006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00b2d20 con 0x7efdd00ce5b0 2023-08-25T09:52:50.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7efdb8001670 con 0x7efdd00ce5b0 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 >> v1:172.21.15.70:6789/0 conn(0x7efdd00ce5b0 legacy=0x7efdd00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdbc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0047c0 0x7efdbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0051d0 0x7efdbc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0159d10 con 0x7efdbc0039a0 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc00c100 con 0x7efdbc0051d0 2023-08-25T09:52:50.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdbc00c2c0 con 0x7efdbc0047c0 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdcffff640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdbc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdcffff640 1 -- 172.21.15.70:0/552476656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0047c0 msgr2=0x7efdbc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.162+0000 7efdcffff640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdbc0047c0 0x7efdbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.163+0000 7efdcffff640 1 -- 172.21.15.70:0/552476656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0051d0 msgr2=0x7efdbc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.163+0000 7efdcffff640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdbc0051d0 0x7efdbc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.163+0000 7efdcffff640 1 -- 172.21.15.70:0/552476656 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc003520 con 0x7efdbc0039a0 2023-08-25T09:52:50.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.163+0000 7efdd4cfa640 1 -- 172.21.15.70:0/552476656 reap_dead start 2023-08-25T09:52:50.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.164+0000 7efdcffff640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdbc003dc0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c240 tx=0x7efdbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25a4ca2635464ac server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:50.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.164+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8005ad0 con 0x7efdbc0039a0 2023-08-25T09:52:50.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.164+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb8005c30 con 0x7efdbc0039a0 2023-08-25T09:52:50.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.164+0000 7efdceffd640 1 -- 172.21.15.70:0/552476656 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb802caf0 con 0x7efdbc0039a0 2023-08-25T09:52:50.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.164+0000 7efdd675d640 1 -- 172.21.15.70:0/552476656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 msgr2=0x7efdbc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.164+0000 7efdd675d640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdbc003dc0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c240 tx=0x7efdbc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.165+0000 7efdd675d640 1 -- 172.21.15.70:0/552476656 shutdown_connections 2023-08-25T09:52:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.165+0000 7efdd675d640 1 --2- 172.21.15.70:0/552476656 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdbc003dc0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.165+0000 7efdd675d640 1 -- 172.21.15.70:0/552476656 >> 172.21.15.70:0/552476656 conn(0x7efdd0069850 msgr2=0x7efdd0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.165+0000 7efdd675d640 1 -- 172.21.15.70:0/552476656 shutdown_connections 2023-08-25T09:52:50.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.165+0000 7efdd675d640 1 -- 172.21.15.70:0/552476656 wait complete. 2023-08-25T09:52:50.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.167+0000 7efdd675d640 1 Processor -- start 2023-08-25T09:52:50.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.167+0000 7efdd675d640 1 -- start start 2023-08-25T09:52:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdd675d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdd00b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdd675d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdd01568b0 0x7efdd013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdd675d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd00add20 0x7efdd00ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdd675d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00ae6e0 con 0x7efdbc0039a0 2023-08-25T09:52:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdd675d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd01f5e10 con 0x7efdd00add20 2023-08-25T09:52:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdd675d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0156d30 con 0x7efdd01568b0 2023-08-25T09:52:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdcffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdd00b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdcffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdd00b0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39488/0 (socket says 172.21.15.70:39488) 2023-08-25T09:52:50.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.168+0000 7efdcffff640 1 -- 172.21.15.70:0/3997898663 learned_addr learned my addr 172.21.15.70:0/3997898663 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcffff640 1 -- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdd01568b0 msgr2=0x7efdd013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcf7fe640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdd01568b0 0x7efdd013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdd4cfa640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd00add20 0x7efdd00ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcffff640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdd01568b0 0x7efdd013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcffff640 1 -- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd00add20 msgr2=0x7efdd00ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcffff640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd00add20 0x7efdd00ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcffff640 1 -- 172.21.15.70:0/3997898663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0157000 con 0x7efdbc0039a0 2023-08-25T09:52:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcf7fe640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdd01568b0 0x7efdd013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdcffff640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdd00b0d40 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c240 tx=0x7efdb8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8039070 con 0x7efdbc0039a0 2023-08-25T09:52:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb802ccc0 con 0x7efdbc0039a0 2023-08-25T09:52:50.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd01572e0 con 0x7efdbc0039a0 2023-08-25T09:52:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.169+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb803c040 con 0x7efdbc0039a0 2023-08-25T09:52:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.170+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd01578a0 con 0x7efdbc0039a0 2023-08-25T09:52:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.171+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd004e420 con 0x7efdbc0039a0 2023-08-25T09:52:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.172+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efdb8032070 con 0x7efdbc0039a0 2023-08-25T09:52:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.172+0000 7efdccff9640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efdb006a050 0x7efdb006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.173+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efdb80a9a70 con 0x7efdbc0039a0 2023-08-25T09:52:50.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.173+0000 7efdcf7fe640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efdb006a050 0x7efdb006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.173+0000 7efdcf7fe640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efdb006a050 0x7efdb006c510 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7efdd006b090 tx=0x7efdc000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:50.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.177+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efdb8075150 con 0x7efdbc0039a0 2023-08-25T09:52:50.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.369+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efdd00ad3a0 con 0x7efdbc0039a0 2023-08-25T09:52:50.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.372+0000 7efdccff9640 1 -- 172.21.15.70:0/3997898663 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v201) v1 ==== 81+0+36967 (secure 0 0 0) 0x7efdb8078770 con 0x7efdbc0039a0 2023-08-25T09:52:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:50.375 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:52:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 201, 2023-08-25T09:52:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", 2023-08-25T09:52:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2023-08-25T09:38:32.416535+0000", 2023-08-25T09:52:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2023-08-25T09:52:47.518383+0000", 2023-08-25T09:52:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2023-08-25T09:38:38.906428+0000", 2023-08-25T09:52:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2023-08-25T09:38:35.884043+0000", 2023-08-25T09:52:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-08-25T09:52:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2023-08-25T09:52:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2023-08-25T09:52:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2023-08-25T09:52:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2023-08-25T09:52:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2023-08-25T09:52:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2023-08-25T09:52:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 4, 2023-08-25T09:52:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.69999998807907104, 2023-08-25T09:52:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-08-25T09:52:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.60000002384185791, 2023-08-25T09:52:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2023-08-25T09:52:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 24, 2023-08-25T09:52:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2023-08-25T09:52:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2023-08-25T09:52:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "jewel", 2023-08-25T09:52:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "reef", 2023-08-25T09:52:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2023-08-25T09:52:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2023-08-25T09:52:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:39.384505+0000", 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 1, 2023-08-25T09:52:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 1, 2023-08-25T09:52:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 1, 2023-08-25T09:52:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 1, 2023-08-25T09:52:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 1, 2023-08-25T09:52:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 8, 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 8, 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.25, 2023-08-25T09:52:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "rbd", 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:38:45.046709+0000", 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 8, 2023-08-25T09:52:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 8, 2023-08-25T09:52:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 8, 2023-08-25T09:52:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 8, 2023-08-25T09:52:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 8, 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "121", 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2023-08-25T09:52:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.9800000190734863, 2023-08-25T09:52:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 4.9800000190734863, 2023-08-25T09:52:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2023-08-25T09:52:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.380000114440918, 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 4.380000114440918, 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 23, 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_metadata", 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:02.296119+0000", 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2023-08-25T09:52:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "197", 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4, 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 16, 2023-08-25T09:52:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_priority": 5 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "metadata": "cephfs" 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 24, 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "cephfs_data", 2023-08-25T09:52:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2023-08-25T09:52:03.097163+0000", 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 2, 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 1, 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2023-08-25T09:52:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2023-08-25T09:52:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2023-08-25T09:52:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-08-25T09:52:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2023-08-25T09:52:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2023-08-25T09:52:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 64, 2023-08-25T09:52:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 64, 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 64, 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 64, 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 64, 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2023-08-25T09:52:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "201", 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2023-08-25T09:52:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2023-08-25T09:52:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 33554432, 2023-08-25T09:52:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2023-08-25T09:52:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2023-08-25T09:52:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2023-08-25T09:52:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2023-08-25T09:52:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2023-08-25T09:52:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2023-08-25T09:52:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2023-08-25T09:52:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2023-08-25T09:52:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-08-25T09:52:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-08-25T09:52:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2023-08-25T09:52:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2023-08-25T09:52:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2023-08-25T09:52:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2023-08-25T09:52:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2023-08-25T09:52:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2023-08-25T09:52:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2023-08-25T09:52:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2023-08-25T09:52:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2023-08-25T09:52:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2023-08-25T09:52:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2023-08-25T09:52:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2023-08-25T09:52:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 64, 2023-08-25T09:52:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "target_size_ratio": 0.89999997615814209 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs": { 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "data": "cephfs" 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2023-08-25T09:52:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2023-08-25T09:52:50.437 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.437 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.437 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.437 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.437 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.438 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.439 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.440 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3728228047 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6809/3728228047", 2023-08-25T09:52:50.441 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6811/3728228047", 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6816/3728228047", 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6813/3728228047", 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.443 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6800", 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.445 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6801", 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.447 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.448 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2023-08-25T09:52:50.449 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.450 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1147342635 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.451 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6801/1147342635", 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6803/1147342635", 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6807/1147342635", 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6805/1147342635", 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.452 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.453 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.454 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.460 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.460 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2023-08-25T09:52:50.460 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.461 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.462 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.462 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2023-08-25T09:52:50.462 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 284471511 2023-08-25T09:52:50.462 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.462 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6817/284471511", 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6819/284471511", 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6823/284471511", 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6821/284471511", 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.463 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.464 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.465 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.466 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.466 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.466 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.466 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.466 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2023-08-25T09:52:50.467 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.468 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.469 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.470 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.471 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2023-08-25T09:52:50.472 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2671981919 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.473 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6825/2671981919", 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6827/2671981919", 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6831/2671981919", 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6829/2671981919", 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.474 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.475 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.476 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6800", 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6801", 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.478 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6802", 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.479 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6803", 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.480 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6806", 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.482 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6807", 2023-08-25T09:52:50.482 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.482 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.482 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.482 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6804", 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.483 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.484 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6805", 2023-08-25T09:52:50.484 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4222826791 2023-08-25T09:52:50.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.484 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6801/4222826791", 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6803/4222826791", 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6807/4222826791", 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6805/4222826791", 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.485 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.486 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.487 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.488 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.488 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.488 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6818", 2023-08-25T09:52:50.488 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.489 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.489 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6820", 2023-08-25T09:52:50.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.489 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.490 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6822", 2023-08-25T09:52:50.491 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.491 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6825", 2023-08-25T09:52:50.491 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.492 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6830", 2023-08-25T09:52:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.493 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6831", 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6827", 2023-08-25T09:52:50.495 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.495 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6829", 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1708871082 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6820/1708871082", 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6825/1708871082", 2023-08-25T09:52:50.496 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6831/1708871082", 2023-08-25T09:52:50.497 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6829/1708871082", 2023-08-25T09:52:50.497 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.497 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.497 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.497 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.498 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.499 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.499 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.499 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.499 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.499 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.499 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6816", 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.501 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.501 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6817", 2023-08-25T09:52:50.501 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.502 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6819", 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6821", 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.503 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.504 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.504 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.504 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6826", 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.505 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6828", 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6823", 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.507 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6824", 2023-08-25T09:52:50.508 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 4156222745 2023-08-25T09:52:50.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.508 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6817/4156222745", 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6821/4156222745", 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6828/4156222745", 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6824/4156222745", 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.509 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2023-08-25T09:52:50.510 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 196, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2023-08-25T09:52:50.511 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6808", 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.512 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6809", 2023-08-25T09:52:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.513 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6810", 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6811", 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.515 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6814", 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6815", 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.517 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6812", 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.155:6813", 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 324773794 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.519 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.155:6809/324773794", 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.155:6811/324773794", 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.155:6815/324773794", 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.155:6813/324773794", 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2023-08-25T09:52:50.520 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.521 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.191703+0000", 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.522 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2023-08-25T09:52:50.523 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.523 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.523 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.523 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.523 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.523 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.136564+0000", 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.524 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.525 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.525 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.525 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.160566+0000", 2023-08-25T09:52:50.525 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.526 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.240696+0000", 2023-08-25T09:52:50.527 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.527 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2023-08-25T09:52:50.527 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.527 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.528 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.528 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.528 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.126106+0000", 2023-08-25T09:52:50.528 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.529 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.530 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.173986+0000", 2023-08-25T09:52:50.530 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.530 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2023-08-25T09:52:50.530 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.180511+0000", 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2023-08-25T09:52:50.532 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2023-08-25T09:52:50.532 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2023-08-25T09:52:50.532 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2023-08-25T09:52:50.532 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2023-08-25T09:52:50.532 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540138322906710015, 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2023-08-25T09:38:37.131259+0000", 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2023-08-25T09:52:50.533 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [], 2023-08-25T09:52:50.534 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2023-08-25T09:52:50.534 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2023-08-25T09:52:50.534 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2023-08-25T09:52:50.534 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": {}, 2023-08-25T09:52:50.534 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2023-08-25T09:52:50.534 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2023-08-25T09:52:50.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2023-08-25T09:52:50.536 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2023-08-25T09:52:50.537 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2023-08-25T09:52:50.537 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2023-08-25T09:52:50.537 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2023-08-25T09:52:50.537 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2023-08-25T09:52:50.537 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2023-08-25T09:52:50.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2023-08-25T09:52:50.538 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:52:50.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.374+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efdb006a050 msgr2=0x7efdb006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efdb006a050 0x7efdb006c510 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7efdd006b090 tx=0x7efdc000b040 comp rx=0 tx=0).stop 2023-08-25T09:52:50.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 msgr2=0x7efdd00b0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdd00b0d40 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7efdb800c240 tx=0x7efdb8001a00 comp rx=0 tx=0).stop 2023-08-25T09:52:50.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 shutdown_connections 2023-08-25T09:52:50.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7efdb006a050 0x7efdb006c510 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7efdd00add20 0x7efdd00ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7efdd01568b0 0x7efdd013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 --2- 172.21.15.70:0/3997898663 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7efdbc0039a0 0x7efdd00b0d40 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 >> 172.21.15.70:0/3997898663 conn(0x7efdd0069850 msgr2=0x7efdd0070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:50.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 shutdown_connections 2023-08-25T09:52:50.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.375+0000 7efdd675d640 1 -- 172.21.15.70:0/3997898663 wait complete. 2023-08-25T09:52:50.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:52:50.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.766+0000 7f773b7c9640 1 Processor -- start 2023-08-25T09:52:50.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.766+0000 7f773b7c9640 1 -- start start 2023-08-25T09:52:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.768+0000 7f773b7c9640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f773415e830 0x7f773406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.768+0000 7f773b7c9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.768+0000 7f773b7c9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340b1b60 con 0x7f773406b2b0 2023-08-25T09:52:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.768+0000 7f773b7c9640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773406bd80 con 0x7f773415e830 2023-08-25T09:52:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.768+0000 7f773b7c9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773406b7c0 con 0x7f773415ec10 2023-08-25T09:52:50.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.769+0000 7f7739d66640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.769+0000 7f7739d66640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:46870/0 (socket says 172.21.15.70:46870) 2023-08-25T09:52:50.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.769+0000 7f7739d66640 1 -- 172.21.15.70:0/3656735484 learned_addr learned my addr 172.21.15.70:0/3656735484 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:50.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.769+0000 7f7739565640 1 --2- 172.21.15.70:0/3656735484 >> v2:172.21.15.155:3300/0 conn(0x7f773415e830 0x7f773406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.769+0000 7f772bfff640 1 -- 172.21.15.70:0/3656735484 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477359557 0 0) 0x7f77340b1b60 con 0x7f773406b2b0 2023-08-25T09:52:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.769+0000 7f772bfff640 1 -- 172.21.15.70:0/3656735484 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7720003520 con 0x7f773406b2b0 2023-08-25T09:52:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.770+0000 7f7739d66640 1 -- 172.21.15.70:0/3656735484 >> v2:172.21.15.155:3300/0 conn(0x7f773415e830 msgr2=0x7f773406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.770+0000 7f7739d66640 1 --2- 172.21.15.70:0/3656735484 >> v2:172.21.15.155:3300/0 conn(0x7f773415e830 0x7f773406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.770+0000 7f7739d66640 1 -- 172.21.15.70:0/3656735484 >> v1:172.21.15.155:6789/0 conn(0x7f773406b2b0 legacy=0x7f773415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.770+0000 7f7739d66640 1 -- 172.21.15.70:0/3656735484 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773406baa0 con 0x7f773415ec10 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.771+0000 7f7739d66640 1 --2- 172.21.15.70:0/3656735484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b1ff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f772c009e50 tx=0x7f772c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e87310b412dd343 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.771+0000 7f772bfff640 1 -- 172.21.15.70:0/3656735484 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c02cb00 con 0x7f773415ec10 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.771+0000 7f772bfff640 1 -- 172.21.15.70:0/3656735484 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c02cc60 con 0x7f773415ec10 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.771+0000 7f772bfff640 1 -- 172.21.15.70:0/3656735484 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c0343f0 con 0x7f773415ec10 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 -- 172.21.15.70:0/3656735484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 msgr2=0x7f77340b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 --2- 172.21.15.70:0/3656735484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b1ff0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f772c009e50 tx=0x7f772c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:52:50.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 -- 172.21.15.70:0/3656735484 shutdown_connections 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 --2- 172.21.15.70:0/3656735484 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b1ff0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 --2- 172.21.15.70:0/3656735484 >> v2:172.21.15.155:3300/0 conn(0x7f773415e830 0x7f773406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 -- 172.21.15.70:0/3656735484 >> 172.21.15.70:0/3656735484 conn(0x7f7734069850 msgr2=0x7f7734069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 -- 172.21.15.70:0/3656735484 shutdown_connections 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.772+0000 7f773b7c9640 1 -- 172.21.15.70:0/3656735484 wait complete. 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.774+0000 7f773b7c9640 1 Processor -- start 2023-08-25T09:52:50.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.774+0000 7f773b7c9640 1 -- start start 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f773b7c9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 0x7f77341f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f773b7c9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773415e830 0x7f77340b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f773b7c9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f773b7c9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340a8670 con 0x7f773406b2b0 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f773b7c9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340ca280 con 0x7f773415e830 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f773b7c9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340c9670 con 0x7f773415ec10 2023-08-25T09:52:50.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f7739565640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 0x7f77341f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.775+0000 7f7739565640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 0x7f77341f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39504/0 (socket says 172.21.15.70:39504) 2023-08-25T09:52:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 -- 172.21.15.70:0/554420645 learned_addr learned my addr 172.21.15.70:0/554420645 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739d66640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7738d64640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773415e830 0x7f77340b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 -- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 msgr2=0x7f77340b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 -- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773415e830 msgr2=0x7f77340b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773415e830 0x7f77340b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:50.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 -- 172.21.15.70:0/554420645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c009b30 con 0x7f773406b2b0 2023-08-25T09:52:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.776+0000 7f7739565640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 0x7f77341f5000 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f773000e9b0 tx=0x7f773000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.777+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730009670 con 0x7f773406b2b0 2023-08-25T09:52:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.777+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000c9a0 con 0x7f773406b2b0 2023-08-25T09:52:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.777+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000cd50 con 0x7f773406b2b0 2023-08-25T09:52:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.777+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773415ca20 con 0x7f773406b2b0 2023-08-25T09:52:50.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.777+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77340c9c30 con 0x7f773406b2b0 2023-08-25T09:52:50.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.778+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773404e420 con 0x7f773406b2b0 2023-08-25T09:52:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.779+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7730013470 con 0x7f773406b2b0 2023-08-25T09:52:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.780+0000 7f7729ffb640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f771806a050 0x7f771806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.780+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77300888f0 con 0x7f773406b2b0 2023-08-25T09:52:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.780+0000 7f7738d64640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f771806a050 0x7f771806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.781+0000 7f7738d64640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f771806a050 0x7f771806c510 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f77240010e0 tx=0x7f7724009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:50.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:50.784+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7730053eb0 con 0x7f773406b2b0 2023-08-25T09:52:51.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.010+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f773406c620 con 0x7f773406b2b0 2023-08-25T09:52:51.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.011+0000 7f7729ffb640 1 -- 172.21.15.70:0/554420645 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f7730021090 con 0x7f773406b2b0 2023-08-25T09:52:51.013 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:51.013 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:52:51.014 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:52:51.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f771806a050 msgr2=0x7f771806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:51.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f771806a050 0x7f771806c510 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f77240010e0 tx=0x7f7724009040 comp rx=0 tx=0).stop 2023-08-25T09:52:51.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 msgr2=0x7f77341f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:51.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 0x7f77341f5000 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f773000e9b0 tx=0x7f773000ee70 comp rx=0 tx=0).stop 2023-08-25T09:52:51.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 shutdown_connections 2023-08-25T09:52:51.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f771806a050 0x7f771806c510 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:51.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f773415ec10 0x7f77340b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:51.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f773415e830 0x7f77340b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:51.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 --2- 172.21.15.70:0/554420645 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f773406b2b0 0x7f77341f5000 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:51.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 >> 172.21.15.70:0/554420645 conn(0x7f7734069850 msgr2=0x7f773415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:51.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 shutdown_connections 2023-08-25T09:52:51.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:51.014+0000 7f773b7c9640 1 -- 172.21.15.70:0/554420645 wait complete. 2023-08-25T09:52:51.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-08-25T09:52:51.259 INFO:teuthology.orchestra.run.smithi070.stdout:{"cluster_fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","whoami":0,"id":9573,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":46.613445968999997,"mdsmap_epoch":91,"osdmap_epoch":201,"osdmap_epoch_barrier":200,"uptime":52.009362015000001} 2023-08-25T09:52:51.260 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "64dd9a7e-0cdf-46fb-8c89-fab611563c75", "fs_name": "cephfs", "id": 9573, "mdsmap_epoch": 91, "osdmap_epoch": 201, "osdmap_epoch_barrier": 200, "rank_uptime": 46.613445969, "state": "up:active", "uptime": 52.009362015, "want_state": "up:active", "whoami": 0 } 2023-08-25T09:52:51.260 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-08-25T09:52:51.261 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:52:51.261 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:51.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:51.311 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-08-25T09:52:52.412 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records in 2023-08-25T09:52:52.412 INFO:teuthology.orchestra.run.smithi070.stderr:50+0 records out 2023-08-25T09:52:52.412 INFO:teuthology.orchestra.run.smithi070.stderr:52428800 bytes (52 MB, 50 MiB) copied, 1.00123 s, 52.4 MB/s 2023-08-25T09:52:52.414 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T09:52:52.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:52:52.531 DEBUG:teuthology.orchestra.run.smithi155:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T09:52:52.663 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T09:52:52.664 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:52:52.665 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:52.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.945+0000 7fc3dd712640 1 Processor -- start 2023-08-25T09:52:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.945+0000 7fc3dd712640 1 -- start start 2023-08-25T09:52:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3dd712640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3d815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3dd712640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d80b1b60 con 0x7fc3d815e830 2023-08-25T09:52:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3dd712640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d806b7c0 con 0x7fc3d815c6d0 2023-08-25T09:52:52.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3dd712640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d806bd80 con 0x7fc3d815ec10 2023-08-25T09:52:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3d6ffd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc3d815e830 0x7fc3d806ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34526/0 (socket says 172.21.15.70:34526) 2023-08-25T09:52:52.949 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3d6ffd640 1 -- 172.21.15.70:0/555017047 learned_addr learned my addr 172.21.15.70:0/555017047 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.947+0000 7fc3cffff640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3d815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322520983 0 0) 0x7fc3d80b1b60 con 0x7fc3d815e830 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c4003520 con 0x7fc3d815e830 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3207404728 0 0) 0x7fc3c4003520 con 0x7fc3d815e830 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 msgr2=0x7fc3d815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3d815c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:52.950 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 >> v1:172.21.15.155:6789/0 conn(0x7fc3d815c6d0 legacy=0x7fc3d80b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d806baa0 con 0x7fc3d815e830 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fc3c8001670 con 0x7fc3d815e830 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 >> v1:172.21.15.70:6789/0 conn(0x7fc3d815e830 legacy=0x7fc3d806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40039a0 0x7fc3c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.951 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d806bd80 con 0x7fc3c4004d00 2023-08-25T09:52:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c400fec0 con 0x7fc3c40039a0 2023-08-25T09:52:52.952 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c4010080 con 0x7fc3d815ec10 2023-08-25T09:52:52.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.948+0000 7fc3d77fe640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3cffff640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.953 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d77fe640 1 -- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 msgr2=0x7fc3c40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d77fe640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3c40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d6ffd640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40039a0 0x7fc3c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d77fe640 1 -- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40039a0 msgr2=0x7fc3c4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d77fe640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40039a0 0x7fc3c4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d77fe640 1 -- 172.21.15.70:0/555017047 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4003520 con 0x7fc3c4004d00 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3cffff640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d815ec10 0x7fc3c40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:52.954 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.949+0000 7fc3d6ffd640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3c40039a0 0x7fc3c4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.950+0000 7fc3d77fe640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3c4005180 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0001e50 tx=0x7fc3c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c1948426f6360f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.950+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0003b70 con 0x7fc3c4004d00 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.950+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0003cd0 con 0x7fc3c4004d00 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.950+0000 7fc3d67fc640 1 -- 172.21.15.70:0/555017047 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0005310 con 0x7fc3c4004d00 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3dd712640 1 -- 172.21.15.70:0/555017047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 msgr2=0x7fc3c4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3dd712640 1 --2- 172.21.15.70:0/555017047 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3c4005180 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0001e50 tx=0x7fc3c40057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:52.955 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3d77fe640 1 -- 172.21.15.70:0/555017047 reap_dead start 2023-08-25T09:52:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3dd712640 1 -- 172.21.15.70:0/555017047 shutdown_connections 2023-08-25T09:52:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3dd712640 1 -- 172.21.15.70:0/555017047 >> 172.21.15.70:0/555017047 conn(0x7fc3d8069850 msgr2=0x7fc3d8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3dd712640 1 -- 172.21.15.70:0/555017047 shutdown_connections 2023-08-25T09:52:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.951+0000 7fc3dd712640 1 -- 172.21.15.70:0/555017047 wait complete. 2023-08-25T09:52:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.953+0000 7fc3dd712640 1 Processor -- start 2023-08-25T09:52:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.953+0000 7fc3dd712640 1 -- start start 2023-08-25T09:52:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3dd712640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3d81ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3dd712640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d813b870 0x7fc3d8155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3dd712640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d8155b70 0x7fc3d813d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.958 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3dd712640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d8156000 con 0x7fc3c4004d00 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3dd712640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d81eef00 con 0x7fc3d8155b70 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3dd712640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80ce2e0 con 0x7fc3d813b870 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3d6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3d81ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3d6ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3d81ee9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39512/0 (socket says 172.21.15.70:39512) 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3d6ffd640 1 -- 172.21.15.70:0/141986638 learned_addr learned my addr 172.21.15.70:0/141986638 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3cffff640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d813b870 0x7fc3d8155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.954+0000 7fc3d77fe640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d8155b70 0x7fc3d813d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.959 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3d6ffd640 1 -- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d813b870 msgr2=0x7fc3d8155630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3d6ffd640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d813b870 0x7fc3d8155630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3d6ffd640 1 -- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d8155b70 msgr2=0x7fc3d813d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3d6ffd640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d8155b70 0x7fc3d813d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3d6ffd640 1 -- 172.21.15.70:0/141986638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d813de50 con 0x7fc3c4004d00 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3cffff640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d813b870 0x7fc3d8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3d6ffd640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3d81ee9c0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800d8e0 tx=0x7fc3c800dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:52.960 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80074e0 con 0x7fc3c4004d00 2023-08-25T09:52:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c8007640 con 0x7fc3c4004d00 2023-08-25T09:52:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80049e0 con 0x7fc3c4004d00 2023-08-25T09:52:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.955+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d813e130 con 0x7fc3c4004d00 2023-08-25T09:52:52.961 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.956+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d813e6f0 con 0x7fc3c4004d00 2023-08-25T09:52:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.957+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d804e420 con 0x7fc3c4004d00 2023-08-25T09:52:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.958+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc3c8004b40 con 0x7fc3c4004d00 2023-08-25T09:52:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.959+0000 7fc3cf7fe640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3b806a050 0x7fc3b806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.959+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3c8087fe0 con 0x7fc3c4004d00 2023-08-25T09:52:52.962 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.959+0000 7fc3cffff640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3b806a050 0x7fc3b806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:52.963 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.960+0000 7fc3cffff640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3b806a050 0x7fc3b806c510 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc3d813bc90 tx=0x7fc3c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:52.965 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:52.964+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc3c80571b0 con 0x7fc3c4004d00 2023-08-25T09:52:53.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.152+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d806c620 con 0x7fc3c4004d00 2023-08-25T09:52:53.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.154+0000 7fc3cf7fe640 1 -- 172.21.15.70:0/141986638 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+18320 (secure 0 0 0) 0x7fc3c8019020 con 0x7fc3c4004d00 2023-08-25T09:52:53.156 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:53.156 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":201,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:47.518383+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3b806a050 msgr2=0x7fc3b806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3b806a050 0x7fc3b806c510 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc3d813bc90 tx=0x7fc3c0008040 comp rx=0 tx=0).stop 2023-08-25T09:52:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 msgr2=0x7fc3d81ee9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3d81ee9c0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800d8e0 tx=0x7fc3c800dda0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 shutdown_connections 2023-08-25T09:52:53.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc3b806a050 0x7fc3b806c510 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc3d8155b70 0x7fc3d813d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc3d813b870 0x7fc3d8155630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 --2- 172.21.15.70:0/141986638 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc3c4004d00 0x7fc3d81ee9c0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.156+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 >> 172.21.15.70:0/141986638 conn(0x7fc3d8069850 msgr2=0x7fc3d815a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:53.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.157+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 shutdown_connections 2023-08-25T09:52:53.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.157+0000 7fc3dd712640 1 -- 172.21.15.70:0/141986638 wait complete. 2023-08-25T09:52:53.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:52:02.296119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:52:03.097163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:52:53.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:52:53.398 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.397+0000 7f8b34d98640 1 Processor -- start 2023-08-25T09:52:53.399 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.397+0000 7f8b34d98640 1 -- start start 2023-08-25T09:52:53.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b34d98640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f8b3015ae50 0x7f8b300c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b34d98640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b300cc5f0 con 0x7f8b300cdc20 2023-08-25T09:52:53.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b34d98640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b30070650 con 0x7f8b30069f60 2023-08-25T09:52:53.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b34d98640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b30070930 con 0x7f8b3015ae50 2023-08-25T09:52:53.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b2dd9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f8b300cdc20 0x7f8b30069850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34536/0 (socket says 172.21.15.70:34536) 2023-08-25T09:52:53.402 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b2dd9b640 1 -- 172.21.15.70:0/4281022295 learned_addr learned my addr 172.21.15.70:0/4281022295 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:53.410 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/4281022295 >> v2:172.21.15.155:3300/0 conn(0x7f8b3015ae50 0x7f8b300c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.399+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395475152 0 0) 0x7f8b300cc5f0 con 0x7f8b300cdc20 2023-08-25T09:52:53.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b18003520 con 0x7f8b300cdc20 2023-08-25T09:52:53.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184596020 0 0) 0x7f8b18003520 con 0x7f8b300cdc20 2023-08-25T09:52:53.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 >> v2:172.21.15.155:3300/0 conn(0x7f8b3015ae50 msgr2=0x7f8b300c2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:53.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 --2- 172.21.15.70:0/4281022295 >> v2:172.21.15.155:3300/0 conn(0x7f8b3015ae50 0x7f8b300c2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 >> v1:172.21.15.155:6789/0 conn(0x7f8b30069f60 legacy=0x7f8b300ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30070c10 con 0x7f8b300cdc20 2023-08-25T09:52:53.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f8b24001670 con 0x7f8b300cdc20 2023-08-25T09:52:53.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 >> v1:172.21.15.70:6789/0 conn(0x7f8b300cdc20 legacy=0x7f8b30069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b18003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.412 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b180047c0 0x7f8b18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b18005180 0x7f8b1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b30070930 con 0x7f8b18003970 2023-08-25T09:52:53.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1800c0b0 con 0x7f8b18005180 2023-08-25T09:52:53.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1800c270 con 0x7f8b180047c0 2023-08-25T09:52:53.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.400+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b18003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2dd9b640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b180047c0 0x7f8b18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2e59c640 1 -- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b180047c0 msgr2=0x7f8b18004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:53.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b180047c0 0x7f8b18004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2e59c640 1 -- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b18005180 msgr2=0x7f8b1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:53.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2ed9d640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b18005180 0x7f8b1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b18005180 0x7f8b1800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2e59c640 1 -- 172.21.15.70:0/4281022295 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b18003520 con 0x7f8b18003970 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.401+0000 7f8b2ed9d640 1 -- 172.21.15.70:0/4281022295 reap_dead start 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.402+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b18003d90 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000ba30 tx=0x7f8b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da32900d7257fcb5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.402+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20032640 con 0x7f8b18003970 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.402+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b200327a0 con 0x7f8b18003970 2023-08-25T09:52:53.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.402+0000 7f8b2d59a640 1 -- 172.21.15.70:0/4281022295 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20032b30 con 0x7f8b18003970 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 -- 172.21.15.70:0/4281022295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 msgr2=0x7f8b18003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b18003d90 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7f8b2000ba30 tx=0x7f8b180057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 -- 172.21.15.70:0/4281022295 shutdown_connections 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 --2- 172.21.15.70:0/4281022295 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b18003d90 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 -- 172.21.15.70:0/4281022295 >> 172.21.15.70:0/4281022295 conn(0x7f8b300cb0d0 msgr2=0x7f8b300cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 -- 172.21.15.70:0/4281022295 shutdown_connections 2023-08-25T09:52:53.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.403+0000 7f8b34d98640 1 -- 172.21.15.70:0/4281022295 wait complete. 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.405+0000 7f8b34d98640 1 Processor -- start 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.405+0000 7f8b34d98640 1 -- start start 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b34d98640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b3014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b34d98640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b300b1c50 0x7f8b300b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b34d98640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b300c7490 0x7f8b300c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b34d98640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300b2e40 con 0x7f8b18003970 2023-08-25T09:52:53.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b34d98640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b300c7e50 con 0x7f8b300c7490 2023-08-25T09:52:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b34d98640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3014f240 con 0x7f8b300b1c50 2023-08-25T09:52:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.406+0000 7f8b2e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b3014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b3014ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39524/0 (socket says 172.21.15.70:39524) 2023-08-25T09:52:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 -- 172.21.15.70:0/961880326 learned_addr learned my addr 172.21.15.70:0/961880326 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 -- 172.21.15.70:0/961880326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b300b1c50 msgr2=0x7f8b300b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b300b1c50 0x7f8b300b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 -- 172.21.15.70:0/961880326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b300c7490 msgr2=0x7f8b300c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b300c7490 0x7f8b300c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 -- 172.21.15.70:0/961880326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300b21c0 con 0x7f8b18003970 2023-08-25T09:52:53.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.407+0000 7f8b2e59c640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b3014ed00 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8b20035040 tx=0x7f8b20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:53.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.408+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2003baf0 con 0x7f8b18003970 2023-08-25T09:52:53.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.408+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2003bc50 con 0x7f8b18003970 2023-08-25T09:52:53.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.408+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b20030410 con 0x7f8b18003970 2023-08-25T09:52:53.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.408+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b300b24a0 con 0x7f8b18003970 2023-08-25T09:52:53.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.408+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b300b2a60 con 0x7f8b18003970 2023-08-25T09:52:53.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.409+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b30066fb0 con 0x7f8b18003970 2023-08-25T09:52:53.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.410+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8b20030570 con 0x7f8b18003970 2023-08-25T09:52:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.411+0000 7f8b1effd640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8b1006a050 0x7f8b1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.411+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b200aaeb0 con 0x7f8b18003970 2023-08-25T09:52:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.411+0000 7f8b2dd9b640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8b1006a050 0x7f8b1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.412+0000 7f8b2dd9b640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8b1006a050 0x7f8b1006c510 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000ff0 tx=0x7f8b24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:53.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.415+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8b2007a020 con 0x7f8b18003970 2023-08-25T09:52:53.608 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.606+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b300caa70 con 0x7f8b18003970 2023-08-25T09:52:53.609 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.607+0000 7f8b1effd640 1 -- 172.21.15.70:0/961880326 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v201) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b20036020 con 0x7f8b18003970 2023-08-25T09:52:53.609 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:52:53.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8b1006a050 msgr2=0x7f8b1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8b1006a050 0x7f8b1006c510 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8b24000ff0 tx=0x7f8b24009040 comp rx=0 tx=0).stop 2023-08-25T09:52:53.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 msgr2=0x7f8b3014ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.611 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b3014ed00 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f8b20035040 tx=0x7f8b20004730 comp rx=0 tx=0).stop 2023-08-25T09:52:53.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 shutdown_connections 2023-08-25T09:52:53.612 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f8b1006a050 0x7f8b1006c510 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f8b300c7490 0x7f8b300c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f8b300b1c50 0x7f8b300b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 --2- 172.21.15.70:0/961880326 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f8b18003970 0x7f8b3014ed00 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 >> 172.21.15.70:0/961880326 conn(0x7f8b300cb0d0 msgr2=0x7f8b30159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:53.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 shutdown_connections 2023-08-25T09:52:53.613 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.610+0000 7f8b34d98640 1 -- 172.21.15.70:0/961880326 wait complete. 2023-08-25T09:52:53.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:52:53.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.843+0000 7f746be6a640 1 Processor -- start 2023-08-25T09:52:53.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.843+0000 7f746be6a640 1 -- start start 2023-08-25T09:52:53.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.844+0000 7f746be6a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f746415ae50 0x7f74640c2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.844+0000 7f746be6a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f74640cdc20 0x7f7464069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f746be6a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74640ce060 0x7f7464069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f746be6a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640cc5f0 con 0x7f746415ae50 2023-08-25T09:52:53.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f746be6a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7464070650 con 0x7f74640cdc20 2023-08-25T09:52:53.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f746be6a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7464070930 con 0x7f74640ce060 2023-08-25T09:52:53.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f7469c06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f746415ae50 0x7f74640c2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f7469c06640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f746415ae50 0x7f74640c2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39534/0 (socket says 172.21.15.70:39534) 2023-08-25T09:52:53.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f7469c06640 1 -- 172.21.15.70:0/1334814768 learned_addr learned my addr 172.21.15.70:0/1334814768 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:53.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f746a407640 1 --2- 172.21.15.70:0/1334814768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74640ce060 0x7f7464069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.845+0000 7f7469405640 1 --2- 172.21.15.70:0/1334814768 >> v2:172.21.15.155:3300/0 conn(0x7f74640cdc20 0x7f7464069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.846+0000 7f7469c06640 1 -- 172.21.15.70:0/1334814768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74640ce060 msgr2=0x7f7464069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.846+0000 7f7469c06640 1 --2- 172.21.15.70:0/1334814768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74640ce060 0x7f7464069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.846+0000 7f7469c06640 1 -- 172.21.15.70:0/1334814768 >> v2:172.21.15.155:3300/0 conn(0x7f74640cdc20 msgr2=0x7f7464069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:53.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.846+0000 7f7469c06640 1 --2- 172.21.15.70:0/1334814768 >> v2:172.21.15.155:3300/0 conn(0x7f74640cdc20 0x7f7464069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.846+0000 7f7469c06640 1 -- 172.21.15.70:0/1334814768 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464070c10 con 0x7f746415ae50 2023-08-25T09:52:53.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.847+0000 7f7469c06640 1 --2- 172.21.15.70:0/1334814768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746415ae50 0x7f74640c2110 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f74640cb030 tx=0x7f746002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45bb786b1703a48 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.847+0000 7f7468c04640 1 -- 172.21.15.70:0/1334814768 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460030860 con 0x7f746415ae50 2023-08-25T09:52:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f7468c04640 1 -- 172.21.15.70:0/1334814768 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74600309c0 con 0x7f746415ae50 2023-08-25T09:52:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f7468c04640 1 -- 172.21.15.70:0/1334814768 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460030d50 con 0x7f746415ae50 2023-08-25T09:52:53.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 -- 172.21.15.70:0/1334814768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746415ae50 msgr2=0x7f74640c2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 --2- 172.21.15.70:0/1334814768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746415ae50 0x7f74640c2110 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f74640cb030 tx=0x7f746002b9a0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 -- 172.21.15.70:0/1334814768 shutdown_connections 2023-08-25T09:52:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 --2- 172.21.15.70:0/1334814768 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f74640ce060 0x7f7464069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 --2- 172.21.15.70:0/1334814768 >> v2:172.21.15.155:3300/0 conn(0x7f74640cdc20 0x7f7464069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 --2- 172.21.15.70:0/1334814768 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f746415ae50 0x7f74640c2110 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.854 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 -- 172.21.15.70:0/1334814768 >> 172.21.15.70:0/1334814768 conn(0x7f74640cb0d0 msgr2=0x7f74640cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 -- 172.21.15.70:0/1334814768 shutdown_connections 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.848+0000 7f746be6a640 1 -- 172.21.15.70:0/1334814768 wait complete. 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.850+0000 7f746be6a640 1 Processor -- start 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.850+0000 7f746be6a640 1 -- start start 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f746be6a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74640cdc20 0x7f74641f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f746be6a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 0x7f74640ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f746be6a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f746415ae50 0x7f746414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f746be6a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74641f3940 con 0x7f74640ce060 2023-08-25T09:52:53.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f746be6a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74641f2c50 con 0x7f74640cdc20 2023-08-25T09:52:53.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f746be6a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74641f2f30 con 0x7f746415ae50 2023-08-25T09:52:53.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.851+0000 7f7469405640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 0x7f74640ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 0x7f74640ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39544/0 (socket says 172.21.15.70:39544) 2023-08-25T09:52:53.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 -- 172.21.15.70:0/2292013424 learned_addr learned my addr 172.21.15.70:0/2292013424 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f746a407640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f746415ae50 0x7f746414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469c06640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74640cdc20 0x7f74641f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 -- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f746415ae50 msgr2=0x7f746414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f746415ae50 0x7f746414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 -- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74640cdc20 msgr2=0x7f74641f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74640cdc20 0x7f74641f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:53.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.852+0000 7f7469405640 1 -- 172.21.15.70:0/2292013424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74600094d0 con 0x7f74640ce060 2023-08-25T09:52:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.853+0000 7f7469405640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 0x7f74640ae180 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f745400d5f0 tx=0x7f745400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.853+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74540125f0 con 0x7f74640ce060 2023-08-25T09:52:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.853+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7454012750 con 0x7f74640ce060 2023-08-25T09:52:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.853+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7454012b00 con 0x7f74640ce060 2023-08-25T09:52:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.853+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74641f3210 con 0x7f74640ce060 2023-08-25T09:52:53.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.853+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74641f28d0 con 0x7f74640ce060 2023-08-25T09:52:53.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.855+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7464066fb0 con 0x7f74640ce060 2023-08-25T09:52:53.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.855+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7454012c60 con 0x7f74640ce060 2023-08-25T09:52:53.860 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.856+0000 7f745a7fc640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f744806a050 0x7f744806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.856+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74540889c0 con 0x7f74640ce060 2023-08-25T09:52:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.857+0000 7f7469c06640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f744806a050 0x7f744806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:53.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.857+0000 7f7469c06640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f744806a050 0x7f744806c510 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7460036000 tx=0x7f7460008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:53.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:53.861+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7454054140 con 0x7f74640ce060 2023-08-25T09:52:54.050 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.049+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f74640af870 con 0x7f74640ce060 2023-08-25T09:52:54.051 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.050+0000 7f745a7fc640 1 -- 172.21.15.70:0/2292013424 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v201) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7454057760 con 0x7f74640ce060 2023-08-25T09:52:54.052 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:52:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f744806a050 msgr2=0x7f744806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.054 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f744806a050 0x7f744806c510 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7460036000 tx=0x7f7460008030 comp rx=0 tx=0).stop 2023-08-25T09:52:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 msgr2=0x7f74640ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 0x7f74640ae180 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f745400d5f0 tx=0x7f745400dab0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 shutdown_connections 2023-08-25T09:52:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f744806a050 0x7f744806c510 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f746415ae50 0x7f746414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.055 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f74640ce060 0x7f74640ae180 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 --2- 172.21.15.70:0/2292013424 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f74640cdc20 0x7f74641f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 >> 172.21.15.70:0/2292013424 conn(0x7f74640cb0d0 msgr2=0x7f74640ca290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 shutdown_connections 2023-08-25T09:52:54.056 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.053+0000 7f746be6a640 1 -- 172.21.15.70:0/2292013424 wait complete. 2023-08-25T09:52:54.072 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:52:54.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.288+0000 7f9590d99640 1 Processor -- start 2023-08-25T09:52:54.289 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.288+0000 7f9590d99640 1 -- start start 2023-08-25T09:52:54.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.289+0000 7f9590d99640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 0x7f958c0ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.290 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.289+0000 7f9590d99640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958c0c2550 con 0x7f958c15ae50 2023-08-25T09:52:54.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.289+0000 7f9590d99640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958c0ce330 con 0x7f958c0c2110 2023-08-25T09:52:54.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.289+0000 7f9590d99640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0cc5f0 con 0x7f958c069f60 2023-08-25T09:52:54.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.289+0000 7f958a59c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f958c15ae50 0x7f958c0cdc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34546/0 (socket says 172.21.15.70:34546) 2023-08-25T09:52:54.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.289+0000 7f958a59c640 1 -- 172.21.15.70:0/3526802578 learned_addr learned my addr 172.21.15.70:0/3526802578 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958ad9d640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 0x7f958c0ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585328782 0 0) 0x7f958c0c2550 con 0x7f958c15ae50 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9574003520 con 0x7f958c15ae50 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2039975119 0 0) 0x7f9574003520 con 0x7f958c15ae50 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 msgr2=0x7f958c0ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 0x7f958c0ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 >> v1:172.21.15.155:6789/0 conn(0x7f958c0c2110 legacy=0x7f958c069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c070650 con 0x7f958c15ae50 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f957c001670 con 0x7f958c15ae50 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 >> v1:172.21.15.70:6789/0 conn(0x7f958c15ae50 legacy=0x7f958c0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.290+0000 7f958959a640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 0x7f95740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958959a640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9574003f20 0x7f95740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958959a640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9574004d00 0x7f9574005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0cc5f0 con 0x7f9574004d00 2023-08-25T09:52:54.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957400baf0 con 0x7f9574003f20 2023-08-25T09:52:54.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957400bcb0 con 0x7f958c069f60 2023-08-25T09:52:54.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958ad9d640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9574004d00 0x7f9574005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958a59c640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 0x7f95740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f9589d9b640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9574003f20 0x7f95740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958ad9d640 1 -- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 msgr2=0x7f95740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:54.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958ad9d640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c069f60 0x7f95740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958ad9d640 1 -- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9574003f20 msgr2=0x7f95740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:54.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958ad9d640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f9574003f20 0x7f95740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.291+0000 7f958ad9d640 1 -- 172.21.15.70:0/3526802578 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9574003520 con 0x7f9574004d00 2023-08-25T09:52:54.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.293+0000 7f958ad9d640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9574004d00 0x7f9574005180 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f9584009f90 tx=0x7f95740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfeb37e8979dda87 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:54.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.293+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958402cbd0 con 0x7f9574004d00 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.293+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958400f040 con 0x7f9574004d00 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.293+0000 7f958959a640 1 -- 172.21.15.70:0/3526802578 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584034440 con 0x7f9574004d00 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f9590d99640 1 -- 172.21.15.70:0/3526802578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9574004d00 msgr2=0x7f9574005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f9590d99640 1 --2- 172.21.15.70:0/3526802578 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f9574004d00 0x7f9574005180 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f9584009f90 tx=0x7f95740057f0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f958ad9d640 1 -- 172.21.15.70:0/3526802578 reap_dead start 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f9590d99640 1 -- 172.21.15.70:0/3526802578 shutdown_connections 2023-08-25T09:52:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f9590d99640 1 -- 172.21.15.70:0/3526802578 >> 172.21.15.70:0/3526802578 conn(0x7f958c0cb0d0 msgr2=0x7f958c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:54.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f9590d99640 1 -- 172.21.15.70:0/3526802578 shutdown_connections 2023-08-25T09:52:54.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.294+0000 7f9590d99640 1 -- 172.21.15.70:0/3526802578 wait complete. 2023-08-25T09:52:54.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.296+0000 7f9590d99640 1 Processor -- start 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.296+0000 7f9590d99640 1 -- start start 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9590d99640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f958c151c00 0x7f958c0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9590d99640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 0x7f958c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9590d99640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c0b0120 0x7f958c0c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9590d99640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0c73e0 con 0x7f958c151fe0 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9590d99640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0c5c80 con 0x7f958c151c00 2023-08-25T09:52:54.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9590d99640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958c0b05a0 con 0x7f958c0b0120 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 0x7f958c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 0x7f958c0c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39558/0 (socket says 172.21.15.70:39558) 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 -- 172.21.15.70:0/4249282913 learned_addr learned my addr 172.21.15.70:0/4249282913 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f958a59c640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f958c151c00 0x7f958c0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f958ad9d640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c0b0120 0x7f958c0c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 -- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c0b0120 msgr2=0x7f958c0c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c0b0120 0x7f958c0c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 -- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f958c151c00 msgr2=0x7f958c0c8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f958c151c00 0x7f958c0c8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f9589d9b640 1 -- 172.21.15.70:0/4249282913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958c0c5ed0 con 0x7f958c151fe0 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.297+0000 7f958a59c640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f958c151c00 0x7f958c0c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.298+0000 7f9589d9b640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 0x7f958c0c7800 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f958000e550 tx=0x7f958000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.298+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95800052e0 con 0x7f958c151fe0 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.298+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9580005440 con 0x7f958c151fe0 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.298+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f958c0c61b0 con 0x7f958c151fe0 2023-08-25T09:52:54.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.298+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580003810 con 0x7f958c151fe0 2023-08-25T09:52:54.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.298+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958c0c6d30 con 0x7f958c151fe0 2023-08-25T09:52:54.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.299+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958c0c63f0 con 0x7f958c151fe0 2023-08-25T09:52:54.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.300+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9580003970 con 0x7f958c151fe0 2023-08-25T09:52:54.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.301+0000 7f957affd640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f956c06a050 0x7f956c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.301+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9580088330 con 0x7f958c151fe0 2023-08-25T09:52:54.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.301+0000 7f958a59c640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f956c06a050 0x7f956c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.302+0000 7f958a59c640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f956c06a050 0x7f956c06c510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f957c005970 tx=0x7f957c001470 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:54.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.305+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f958c0c63f0 con 0x7f958c151fe0 2023-08-25T09:52:54.495 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.493+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f958c0c63f0 con 0x7f958c151fe0 2023-08-25T09:52:54.496 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.494+0000 7f957affd640 1 -- 172.21.15.70:0/4249282913 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v201) v1 ==== 104+0+11 (secure 0 0 0) 0x7f958c0c63f0 con 0x7f958c151fe0 2023-08-25T09:52:54.496 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:52:54.498 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f956c06a050 msgr2=0x7f956c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f956c06a050 0x7f956c06c510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f957c005970 tx=0x7f957c001470 comp rx=0 tx=0).stop 2023-08-25T09:52:54.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 msgr2=0x7f958c0c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 0x7f958c0c7800 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f958000e550 tx=0x7f958000ea10 comp rx=0 tx=0).stop 2023-08-25T09:52:54.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 shutdown_connections 2023-08-25T09:52:54.499 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f956c06a050 0x7f956c06c510 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f958c0b0120 0x7f958c0c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f958c151fe0 0x7f958c0c7800 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 --2- 172.21.15.70:0/4249282913 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f958c151c00 0x7f958c0c8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 >> 172.21.15.70:0/4249282913 conn(0x7f958c0cb0d0 msgr2=0x7f958c0ac880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:54.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 shutdown_connections 2023-08-25T09:52:54.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.497+0000 7f9590d99640 1 -- 172.21.15.70:0/4249282913 wait complete. 2023-08-25T09:52:54.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:52:54.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.731+0000 7f0c2bfff640 1 Processor -- start 2023-08-25T09:52:54.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.731+0000 7f0c2bfff640 1 -- start start 2023-08-25T09:52:54.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2470 0x7f0c2c159930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c2c0c8d20 0x7f0c2c0c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2bfff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c2c159e70 con 0x7f0c2c0b2850 2023-08-25T09:52:54.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2bfff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2c15ad20 con 0x7f0c2c0c8d20 2023-08-25T09:52:54.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0cd800 con 0x7f0c2c0b2470 2023-08-25T09:52:54.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c2c0c8d20 0x7f0c2c0c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2b7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f0c2c0c8d20 0x7f0c2c0c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39572/0 (socket says 172.21.15.70:39572) 2023-08-25T09:52:54.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2b7fe640 1 -- 172.21.15.70:0/963216844 learned_addr learned my addr 172.21.15.70:0/963216844 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:54.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2affd640 1 --2- 172.21.15.70:0/963216844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2470 0x7f0c2c159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.733+0000 7f0c2b7fe640 1 -- 172.21.15.70:0/963216844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2470 msgr2=0x7f0c2c159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:54.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.734+0000 7f0c2b7fe640 1 --2- 172.21.15.70:0/963216844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2470 0x7f0c2c159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.734+0000 7f0c2b7fe640 1 -- 172.21.15.70:0/963216844 >> v1:172.21.15.70:6789/0 conn(0x7f0c2c0b2850 legacy=0x7f0c2c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.734+0000 7f0c2b7fe640 1 -- 172.21.15.70:0/963216844 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c15a110 con 0x7f0c2c0c8d20 2023-08-25T09:52:54.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.734+0000 7f0c2a7fc640 1 -- 172.21.15.70:0/963216844 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817962194 0 0) 0x7f0c2c159e70 con 0x7f0c2c0b2850 2023-08-25T09:52:54.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2b7fe640 1 --2- 172.21.15.70:0/963216844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c0c91b0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009e50 tx=0x7f0c2402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b24f80c23b0dcf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2a7fc640 1 -- 172.21.15.70:0/963216844 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2402cb00 con 0x7f0c2c0c8d20 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2a7fc640 1 -- 172.21.15.70:0/963216844 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2402cc60 con 0x7f0c2c0c8d20 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2a7fc640 1 -- 172.21.15.70:0/963216844 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240343f0 con 0x7f0c2c0c8d20 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2bfff640 1 -- 172.21.15.70:0/963216844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0c8d20 msgr2=0x7f0c2c0c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/963216844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c0c91b0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009e50 tx=0x7f0c2402bf60 comp rx=0 tx=0).stop 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2bfff640 1 -- 172.21.15.70:0/963216844 shutdown_connections 2023-08-25T09:52:54.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/963216844 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c0c91b0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/963216844 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2470 0x7f0c2c159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.735+0000 7f0c2bfff640 1 -- 172.21.15.70:0/963216844 >> 172.21.15.70:0/963216844 conn(0x7f0c2c0ce830 msgr2=0x7f0c2c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.736+0000 7f0c2bfff640 1 -- 172.21.15.70:0/963216844 shutdown_connections 2023-08-25T09:52:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.736+0000 7f0c2bfff640 1 -- 172.21.15.70:0/963216844 wait complete. 2023-08-25T09:52:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.737+0000 7f0c2bfff640 1 Processor -- start 2023-08-25T09:52:54.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.737+0000 7f0c2bfff640 1 -- start start 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.738+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 0x7f0c2c0ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.738+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2850 0x7f0c2c158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.738+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c15cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.738+0000 7f0c2bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0ac720 con 0x7f0c2c0b2470 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.738+0000 7f0c2bfff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0aca00 con 0x7f0c2c0c8d20 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.738+0000 7f0c2bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2c0acce0 con 0x7f0c2c0b2850 2023-08-25T09:52:54.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 0x7f0c2c0ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 0x7f0c2c0ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39582/0 (socket says 172.21.15.70:39582) 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 -- 172.21.15.70:0/2525072579 learned_addr learned my addr 172.21.15.70:0/2525072579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2b7fe640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c15cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c227fc640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2850 0x7f0c2c158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 -- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2850 msgr2=0x7f0c2c158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2850 0x7f0c2c158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.749 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 -- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c0c8d20 msgr2=0x7f0c2c15cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c15cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 -- 172.21.15.70:0/2525072579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24009b30 con 0x7f0c2c0b2470 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2b7fe640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c227fc640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2850 0x7f0c2c158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.739+0000 7f0c2affd640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 0x7f0c2c0ad7c0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00d7d0 tx=0x7f0c1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.740+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0096c0 con 0x7f0c2c0b2470 2023-08-25T09:52:54.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.740+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c009820 con 0x7f0c2c0b2470 2023-08-25T09:52:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.740+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c009bd0 con 0x7f0c2c0b2470 2023-08-25T09:52:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.740+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2c0acfc0 con 0x7f0c2c0b2470 2023-08-25T09:52:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.740+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2c0add00 con 0x7f0c2c0b2470 2023-08-25T09:52:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.741+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2c15a990 con 0x7f0c2c0b2470 2023-08-25T09:52:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.742+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0c1c009d30 con 0x7f0c2c0b2470 2023-08-25T09:52:54.751 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.743+0000 7f0c23fff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c1006a050 0x7f0c1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.743+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0c1c088df0 con 0x7f0c2c0b2470 2023-08-25T09:52:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.743+0000 7f0c227fc640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c1006a050 0x7f0c1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.744+0000 7f0c227fc640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c1006a050 0x7f0c1006c510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c1f53d0 tx=0x7f0c14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:54.752 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.747+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0c2c15a990 con 0x7f0c2c0b2470 2023-08-25T09:52:54.937 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.935+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c2c15a990 con 0x7f0c2c0b2470 2023-08-25T09:52:54.938 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.936+0000 7f0c23fff640 1 -- 172.21.15.70:0/2525072579 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v201) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c2c15a990 con 0x7f0c2c0b2470 2023-08-25T09:52:54.938 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:52:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.938+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c1006a050 msgr2=0x7f0c1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.940 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.938+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c1006a050 0x7f0c1006c510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c1f53d0 tx=0x7f0c14009040 comp rx=0 tx=0).stop 2023-08-25T09:52:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 msgr2=0x7f0c2c0ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 0x7f0c2c0ad7c0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00d7d0 tx=0x7f0c1c00dc90 comp rx=0 tx=0).stop 2023-08-25T09:52:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 shutdown_connections 2023-08-25T09:52:54.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0c1006a050 0x7f0c1006c510 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0c2c0c8d20 0x7f0c2c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.942 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0c2c0b2850 0x7f0c2c158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 --2- 172.21.15.70:0/2525072579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0c2c0b2470 0x7f0c2c0ad7c0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:54.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 >> 172.21.15.70:0/2525072579 conn(0x7f0c2c0ce830 msgr2=0x7f0c2c069850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:54.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 shutdown_connections 2023-08-25T09:52:54.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:54.939+0000 7f0c2bfff640 1 -- 172.21.15.70:0/2525072579 wait complete. 2023-08-25T09:52:54.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:52:55.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.169+0000 7f2d89ea6640 1 Processor -- start 2023-08-25T09:52:55.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.170+0000 7f2d89ea6640 1 -- start start 2023-08-25T09:52:55.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.171+0000 7f2d89ea6640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d8415acd0 0x7f2d8415b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.171+0000 7f2d89ea6640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d840b12b0 con 0x7f2d840b4400 2023-08-25T09:52:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.171+0000 7f2d89ea6640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8406ae20 con 0x7f2d840ca780 2023-08-25T09:52:55.174 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.171+0000 7f2d89ea6640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d840b47e0 con 0x7f2d8415acd0 2023-08-25T09:52:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.171+0000 7f2d837fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d8415acd0 0x7f2d8415b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d837fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f2d8415acd0 0x7f2d8415b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39586/0 (socket says 172.21.15.70:39586) 2023-08-25T09:52:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d837fe640 1 -- 172.21.15.70:0/726010013 learned_addr learned my addr 172.21.15.70:0/726010013 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d83fff640 1 --1- 172.21.15.70:0/726010013 >> v1:172.21.15.70:6789/0 conn(0x7f2d840b4400 0x7f2d840c9400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:34554/0 (socket says 172.21.15.70:34554) 2023-08-25T09:52:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d837fe640 1 -- 172.21.15.70:0/726010013 >> v1:172.21.15.155:6789/0 conn(0x7f2d840ca780 legacy=0x7f2d840b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d837fe640 1 -- 172.21.15.70:0/726010013 >> v1:172.21.15.70:6789/0 conn(0x7f2d840b4400 legacy=0x7f2d840c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:55.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d837fe640 1 -- 172.21.15.70:0/726010013 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8406b910 con 0x7f2d8415acd0 2023-08-25T09:52:55.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.172+0000 7f2d82ffd640 1 -- 172.21.15.70:0/726010013 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259326336 0 0) 0x7f2d840b12b0 con 0x7f2d840b4400 2023-08-25T09:52:55.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.173+0000 7f2d837fe640 1 --2- 172.21.15.70:0/726010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d8415b0b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009850 tx=0x7f2d7402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deaa80a11d1f4813 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:55.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.173+0000 7f2d82ffd640 1 -- 172.21.15.70:0/726010013 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74004230 con 0x7f2d8415acd0 2023-08-25T09:52:55.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.173+0000 7f2d82ffd640 1 -- 172.21.15.70:0/726010013 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d74004390 con 0x7f2d8415acd0 2023-08-25T09:52:55.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.173+0000 7f2d82ffd640 1 -- 172.21.15.70:0/726010013 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74003810 con 0x7f2d8415acd0 2023-08-25T09:52:55.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 -- 172.21.15.70:0/726010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 msgr2=0x7f2d8415b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:55.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/726010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d8415b0b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009850 tx=0x7f2d7402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 -- 172.21.15.70:0/726010013 shutdown_connections 2023-08-25T09:52:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/726010013 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d8415b0b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 -- 172.21.15.70:0/726010013 >> 172.21.15.70:0/726010013 conn(0x7f2d840ce760 msgr2=0x7f2d840ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 -- 172.21.15.70:0/726010013 shutdown_connections 2023-08-25T09:52:55.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.174+0000 7f2d89ea6640 1 -- 172.21.15.70:0/726010013 wait complete. 2023-08-25T09:52:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.176+0000 7f2d89ea6640 1 Processor -- start 2023-08-25T09:52:55.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.176+0000 7f2d89ea6640 1 -- start start 2023-08-25T09:52:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d89ea6640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d840b4400 0x7f2d840ccd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d89ea6640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d840ca780 0x7f2d8415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:55.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d89ea6640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d84150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d89ea6640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840cc310 con 0x7f2d8415acd0 2023-08-25T09:52:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d89ea6640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840cd2a0 con 0x7f2d840b4400 2023-08-25T09:52:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d89ea6640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841518f0 con 0x7f2d840ca780 2023-08-25T09:52:55.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d83fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d84150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d83fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d84150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39594/0 (socket says 172.21.15.70:39594) 2023-08-25T09:52:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.177+0000 7f2d83fff640 1 -- 172.21.15.70:0/1026554565 learned_addr learned my addr 172.21.15.70:0/1026554565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d837fe640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d840b4400 0x7f2d840ccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:55.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d83fff640 1 -- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d840ca780 msgr2=0x7f2d8415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d7bfff640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d840ca780 0x7f2d8415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d83fff640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d840ca780 0x7f2d8415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d83fff640 1 -- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d840b4400 msgr2=0x7f2d840ccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:55.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d83fff640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d840b4400 0x7f2d840ccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d83fff640 1 -- 172.21.15.70:0/1026554565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74009460 con 0x7f2d8415acd0 2023-08-25T09:52:55.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d837fe640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d840b4400 0x7f2d840ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:55.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d7bfff640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d840ca780 0x7f2d8415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:55.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d83fff640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d84150f10 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c7e0 tx=0x7f2d7c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:55.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00a640 con 0x7f2d8415acd0 2023-08-25T09:52:55.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7c00a7c0 con 0x7f2d8415acd0 2023-08-25T09:52:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00ab70 con 0x7f2d8415acd0 2023-08-25T09:52:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.178+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d84151bd0 con 0x7f2d8415acd0 2023-08-25T09:52:55.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.179+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d840cc840 con 0x7f2d8415acd0 2023-08-25T09:52:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.180+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d840cca60 con 0x7f2d8415acd0 2023-08-25T09:52:55.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.181+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2d7c00acd0 con 0x7f2d8415acd0 2023-08-25T09:52:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.182+0000 7f2d80ff9640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2d6406a050 0x7f2d6406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.182+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d7c087b60 con 0x7f2d8415acd0 2023-08-25T09:52:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.182+0000 7f2d837fe640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2d6406a050 0x7f2d6406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:55.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.182+0000 7f2d837fe640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2d6406a050 0x7f2d6406c510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2d74035000 tx=0x7f2d74008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:55.195 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.186+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2d7c053250 con 0x7f2d8415acd0 2023-08-25T09:52:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.387+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2d8404e420 con 0x7f2d8415acd0 2023-08-25T09:52:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.388+0000 7f2d80ff9640 1 -- 172.21.15.70:0/1026554565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v201) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2d7c018020 con 0x7f2d8415acd0 2023-08-25T09:52:55.390 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:52:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.390+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2d6406a050 msgr2=0x7f2d6406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.390+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2d6406a050 0x7f2d6406c510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2d74035000 tx=0x7f2d74008030 comp rx=0 tx=0).stop 2023-08-25T09:52:55.392 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.390+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 msgr2=0x7f2d84150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.390+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d84150f10 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c00c7e0 tx=0x7f2d7c00cca0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 shutdown_connections 2023-08-25T09:52:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2d6406a050 0x7f2d6406c510 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.393 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2d8415acd0 0x7f2d84150f10 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2d840ca780 0x7f2d8415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 --2- 172.21.15.70:0/1026554565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2d840b4400 0x7f2d840ccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 >> 172.21.15.70:0/1026554565 conn(0x7f2d840ce760 msgr2=0x7f2d8415d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 shutdown_connections 2023-08-25T09:52:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:55.391+0000 7f2d89ea6640 1 -- 172.21.15.70:0/1026554565 wait complete. 2023-08-25T09:52:55.410 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi070.front.sepia.ceph.com... 2023-08-25T09:52:55.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:55.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-08-25T09:52:55.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi070.stderr:ceph-fuse[127931]: fuse finished with error 0 and tester_r 0 2023-08-25T09:52:55.518 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:52:55.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:55.519 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T09:52:55.729 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:52:55.729 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:55.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:56.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.010+0000 7f526a0c5640 1 Processor -- start 2023-08-25T09:52:56.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.010+0000 7f526a0c5640 1 -- start start 2023-08-25T09:52:56.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.012+0000 7f526a0c5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415e830 0x7f526406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.012+0000 7f526a0c5640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f526406b2b0 0x7f526415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.012+0000 7f526a0c5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52640b1b60 con 0x7f526415c6d0 2023-08-25T09:52:56.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.012+0000 7f526a0c5640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f526406bd80 con 0x7f526406b2b0 2023-08-25T09:52:56.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.012+0000 7f526a0c5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526406b7c0 con 0x7f526415e830 2023-08-25T09:52:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f526406b2b0 0x7f526415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f526406b2b0 0x7f526415c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39596/0 (socket says 172.21.15.70:39596) 2023-08-25T09:52:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 -- 172.21.15.70:0/4092610552 learned_addr learned my addr 172.21.15.70:0/4092610552 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f52637fe640 1 --2- 172.21.15.70:0/4092610552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415e830 0x7f526406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 -- 172.21.15.70:0/4092610552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415e830 msgr2=0x7f526406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:56.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 --2- 172.21.15.70:0/4092610552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415e830 0x7f526406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 -- 172.21.15.70:0/4092610552 >> v1:172.21.15.155:6789/0 conn(0x7f526415c6d0 legacy=0x7f52640b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.013+0000 7f525affd640 1 -- 172.21.15.70:0/4092610552 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526406baa0 con 0x7f526406b2b0 2023-08-25T09:52:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.014+0000 7f525affd640 1 --2- 172.21.15.70:0/4092610552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f526415c190 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f524c009870 tx=0x7f524c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e686ee46aef2b27a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f5262ffd640 1 -- 172.21.15.70:0/4092610552 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c030840 con 0x7f526406b2b0 2023-08-25T09:52:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f5262ffd640 1 -- 172.21.15.70:0/4092610552 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c0309a0 con 0x7f526406b2b0 2023-08-25T09:52:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f5262ffd640 1 -- 172.21.15.70:0/4092610552 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c030d30 con 0x7f526406b2b0 2023-08-25T09:52:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 -- 172.21.15.70:0/4092610552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 msgr2=0x7f526415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 --2- 172.21.15.70:0/4092610552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f526415c190 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f524c009870 tx=0x7f524c02b980 comp rx=0 tx=0).stop 2023-08-25T09:52:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 -- 172.21.15.70:0/4092610552 shutdown_connections 2023-08-25T09:52:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 --2- 172.21.15.70:0/4092610552 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f526415c190 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 --2- 172.21.15.70:0/4092610552 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415e830 0x7f526406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 -- 172.21.15.70:0/4092610552 >> 172.21.15.70:0/4092610552 conn(0x7f5264069850 msgr2=0x7f5264069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 -- 172.21.15.70:0/4092610552 shutdown_connections 2023-08-25T09:52:56.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.015+0000 7f526a0c5640 1 -- 172.21.15.70:0/4092610552 wait complete. 2023-08-25T09:52:56.021 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.017+0000 7f526a0c5640 1 Processor -- start 2023-08-25T09:52:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.017+0000 7f526a0c5640 1 -- start start 2023-08-25T09:52:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f526a0c5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f52640aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f526a0c5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415c6d0 0x7f52641f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.022 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f526a0c5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f526415e830 0x7f52640cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f526a0c5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52641f66c0 con 0x7f526406b2b0 2023-08-25T09:52:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f526a0c5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52641f6100 con 0x7f526415e830 2023-08-25T09:52:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f526a0c5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52641f63e0 con 0x7f526415c6d0 2023-08-25T09:52:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f52637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f52640aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f52637fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f52640aa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39612/0 (socket says 172.21.15.70:39612) 2023-08-25T09:52:56.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.018+0000 7f52637fe640 1 -- 172.21.15.70:0/3737333444 learned_addr learned my addr 172.21.15.70:0/3737333444 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:56.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f52637fe640 1 -- 172.21.15.70:0/3737333444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415c6d0 msgr2=0x7f52641f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f52637fe640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415c6d0 0x7f52641f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f52637fe640 1 -- 172.21.15.70:0/3737333444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f526415e830 msgr2=0x7f52640cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f52637fe640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f526415e830 0x7f52640cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f52637fe640 1 -- 172.21.15.70:0/3737333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0094d0 con 0x7f526406b2b0 2023-08-25T09:52:56.025 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f52637fe640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f52640aa7c0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f525400d5f0 tx=0x7f525400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52540125f0 con 0x7f526406b2b0 2023-08-25T09:52:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5254012750 con 0x7f526406b2b0 2023-08-25T09:52:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5254012b00 con 0x7f526406b2b0 2023-08-25T09:52:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.019+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52640a9fc0 con 0x7f526406b2b0 2023-08-25T09:52:56.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.020+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52640aad00 con 0x7f526406b2b0 2023-08-25T09:52:56.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.021+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52640aaef0 con 0x7f526406b2b0 2023-08-25T09:52:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.021+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5254012c60 con 0x7f526406b2b0 2023-08-25T09:52:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.022+0000 7f5260ff9640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f524406a050 0x7f524406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.023+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5254088b10 con 0x7f526406b2b0 2023-08-25T09:52:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.023+0000 7f525affd640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f524406a050 0x7f524406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.024+0000 7f525affd640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f524406a050 0x7f524406c510 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f524c036000 tx=0x7f524c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.028+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5254054160 con 0x7f526406b2b0 2023-08-25T09:52:56.220 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.219+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f526404e420 con 0x7f526406b2b0 2023-08-25T09:52:56.223 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.222+0000 7f5260ff9640 1 -- 172.21.15.70:0/3737333444 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+18320 (secure 0 0 0) 0x7f5254057780 con 0x7f526406b2b0 2023-08-25T09:52:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:56.224 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":201,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:47.518383+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:56.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f524406a050 msgr2=0x7f524406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.226 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f524406a050 0x7f524406c510 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f524c036000 tx=0x7f524c03b040 comp rx=0 tx=0).stop 2023-08-25T09:52:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 msgr2=0x7f52640aa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f52640aa7c0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f525400d5f0 tx=0x7f525400dab0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 shutdown_connections 2023-08-25T09:52:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f524406a050 0x7f524406c510 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f526415e830 0x7f52640cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.227 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f526415c6d0 0x7f52641f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 --2- 172.21.15.70:0/3737333444 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f526406b2b0 0x7f52640aa7c0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 >> 172.21.15.70:0/3737333444 conn(0x7f5264069850 msgr2=0x7f526415b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 shutdown_connections 2023-08-25T09:52:56.228 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.225+0000 7f526a0c5640 1 -- 172.21.15.70:0/3737333444 wait complete. 2023-08-25T09:52:56.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:52:02.296119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:52:03.097163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:52:56.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:52:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.467+0000 7f31a948a640 1 Processor -- start 2023-08-25T09:52:56.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.468+0000 7f31a948a640 1 -- start start 2023-08-25T09:52:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.469+0000 7f31a948a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 0x7f31a40c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.469+0000 7f31a948a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f31a40b1ce0 0x7f31a40ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.469+0000 7f31a948a640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a40b2170 con 0x7f31a415a920 2023-08-25T09:52:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.469+0000 7f31a948a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a40ad8f0 con 0x7f31a40b1ce0 2023-08-25T09:52:56.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.469+0000 7f31a948a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40adbb0 con 0x7f31a40b0320 2023-08-25T09:52:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.470+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 0x7f31a40c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.470+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 0x7f31a40c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:47008/0 (socket says 172.21.15.70:47008) 2023-08-25T09:52:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.470+0000 7f31a3fff640 1 -- 172.21.15.70:0/2417909842 learned_addr learned my addr 172.21.15.70:0/2417909842 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:56.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.470+0000 7f31a8c89640 1 --2- 172.21.15.70:0/2417909842 >> v2:172.21.15.155:3300/0 conn(0x7f31a40b1ce0 0x7f31a40ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.471+0000 7f31a3fff640 1 -- 172.21.15.70:0/2417909842 >> v2:172.21.15.155:3300/0 conn(0x7f31a40b1ce0 msgr2=0x7f31a40ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:56.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.471+0000 7f31a3fff640 1 --2- 172.21.15.70:0/2417909842 >> v2:172.21.15.155:3300/0 conn(0x7f31a40b1ce0 0x7f31a40ad3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.471+0000 7f31a3fff640 1 -- 172.21.15.70:0/2417909842 >> v1:172.21.15.155:6789/0 conn(0x7f31a415a920 legacy=0x7f31a40b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.471+0000 7f31a3fff640 1 -- 172.21.15.70:0/2417909842 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40c90a0 con 0x7f31a40b0320 2023-08-25T09:52:56.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.471+0000 7f31a8c89640 1 --2- 172.21.15.70:0/2417909842 >> v2:172.21.15.155:3300/0 conn(0x7f31a40b1ce0 0x7f31a40ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:52:56.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.472+0000 7f31a3fff640 1 --2- 172.21.15.70:0/2417909842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 0x7f31a40c9e60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3194009850 tx=0x7f319402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735a37b702987e77 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.472+0000 7f31a37fe640 1 -- 172.21.15.70:0/2417909842 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194004230 con 0x7f31a40b0320 2023-08-25T09:52:56.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.472+0000 7f31a37fe640 1 -- 172.21.15.70:0/2417909842 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194004390 con 0x7f31a40b0320 2023-08-25T09:52:56.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.472+0000 7f31a37fe640 1 -- 172.21.15.70:0/2417909842 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194003810 con 0x7f31a40b0320 2023-08-25T09:52:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 -- 172.21.15.70:0/2417909842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 msgr2=0x7f31a40c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 --2- 172.21.15.70:0/2417909842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 0x7f31a40c9e60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3194009850 tx=0x7f319402b960 comp rx=0 tx=0).stop 2023-08-25T09:52:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 -- 172.21.15.70:0/2417909842 shutdown_connections 2023-08-25T09:52:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 --2- 172.21.15.70:0/2417909842 >> v2:172.21.15.155:3300/0 conn(0x7f31a40b1ce0 0x7f31a40ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 --2- 172.21.15.70:0/2417909842 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b0320 0x7f31a40c9e60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 -- 172.21.15.70:0/2417909842 >> 172.21.15.70:0/2417909842 conn(0x7f31a40ce830 msgr2=0x7f31a40cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 -- 172.21.15.70:0/2417909842 shutdown_connections 2023-08-25T09:52:56.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.473+0000 7f31a948a640 1 -- 172.21.15.70:0/2417909842 wait complete. 2023-08-25T09:52:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.475+0000 7f31a948a640 1 Processor -- start 2023-08-25T09:52:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.475+0000 7f31a948a640 1 -- start start 2023-08-25T09:52:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.476+0000 7f31a948a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 0x7f31a4157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.476+0000 7f31a948a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b1ce0 0x7f31a406e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.481 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.476+0000 7f31a948a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a415a920 0x7f31a4070f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.476+0000 7f31a948a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4157870 con 0x7f31a40b0320 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.476+0000 7f31a948a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a406f1b0 con 0x7f31a415a920 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.476+0000 7f31a948a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a406f490 con 0x7f31a40b1ce0 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 0x7f31a4157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 0x7f31a4157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39614/0 (socket says 172.21.15.70:39614) 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 -- 172.21.15.70:0/2638841809 learned_addr learned my addr 172.21.15.70:0/2638841809 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f319bfff640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b1ce0 0x7f31a406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.482 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a8c89640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a415a920 0x7f31a4070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 -- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b1ce0 msgr2=0x7f31a406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b1ce0 0x7f31a406e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 -- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a415a920 msgr2=0x7f31a4070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a415a920 0x7f31a4070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.477+0000 7f31a3fff640 1 -- 172.21.15.70:0/2638841809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c009a70 con 0x7f31a40b0320 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.478+0000 7f31a3fff640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 0x7f31a4157330 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f319402b9a0 tx=0x7f31940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.483 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.478+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319403e070 con 0x7f31a40b0320 2023-08-25T09:52:56.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.478+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3194004500 con 0x7f31a40b0320 2023-08-25T09:52:56.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.478+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3194009460 con 0x7f31a40b0320 2023-08-25T09:52:56.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.478+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3194003df0 con 0x7f31a40b0320 2023-08-25T09:52:56.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.478+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a406fa50 con 0x7f31a40b0320 2023-08-25T09:52:56.484 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.479+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a40cdf80 con 0x7f31a40b0320 2023-08-25T09:52:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.480+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3194006040 con 0x7f31a40b0320 2023-08-25T09:52:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.481+0000 7f31a17fa640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f318406a080 0x7f318406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.481+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f31940aa3d0 con 0x7f31a40b0320 2023-08-25T09:52:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.481+0000 7f319bfff640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f318406a080 0x7f318406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.485 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.482+0000 7f319bfff640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f318406a080 0x7f318406c540 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f31a40978b0 tx=0x7f318c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.486 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.485+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3194075a20 con 0x7f31a40b0320 2023-08-25T09:52:56.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.672+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f31a4061460 con 0x7f31a40b0320 2023-08-25T09:52:56.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.673+0000 7f31a17fa640 1 -- 172.21.15.70:0/2638841809 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v201) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3194079040 con 0x7f31a40b0320 2023-08-25T09:52:56.675 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:52:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f318406a080 msgr2=0x7f318406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f318406a080 0x7f318406c540 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f31a40978b0 tx=0x7f318c009040 comp rx=0 tx=0).stop 2023-08-25T09:52:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 msgr2=0x7f31a4157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 0x7f31a4157330 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f319402b9a0 tx=0x7f31940046c0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 shutdown_connections 2023-08-25T09:52:56.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f318406a080 0x7f318406c540 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f31a415a920 0x7f31a4070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f31a40b1ce0 0x7f31a406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 --2- 172.21.15.70:0/2638841809 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f31a40b0320 0x7f31a4157330 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 >> 172.21.15.70:0/2638841809 conn(0x7f31a40ce830 msgr2=0x7f31a40cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 shutdown_connections 2023-08-25T09:52:56.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.676+0000 7f31a948a640 1 -- 172.21.15.70:0/2638841809 wait complete. 2023-08-25T09:52:56.694 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:52:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.902+0000 7f4ca2f22640 1 Processor -- start 2023-08-25T09:52:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.903+0000 7f4ca2f22640 1 -- start start 2023-08-25T09:52:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.904+0000 7f4ca2f22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cb400 0x7f4c9c06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca2f22640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4c9c0cd7e0 0x7f4c9c0cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca2f22640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c0ccab0 con 0x7f4c9c0b0960 2023-08-25T09:52:56.908 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca2f22640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c06c0c0 con 0x7f4c9c0cd7e0 2023-08-25T09:52:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca2f22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c0ccd00 con 0x7f4c9c0cb400 2023-08-25T09:52:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4c9bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cb400 0x7f4c9c06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca0cbe640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f4c9c0b0960 0x7f4c9c0b0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:33004/0 (socket says 172.21.15.70:33004) 2023-08-25T09:52:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca0cbe640 1 -- 172.21.15.70:0/690358920 learned_addr learned my addr 172.21.15.70:0/690358920 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:52:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.905+0000 7f4ca14bf640 1 --2- 172.21.15.70:0/690358920 >> v2:172.21.15.155:3300/0 conn(0x7f4c9c0cd7e0 0x7f4c9c0cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.909 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4c9b7fe640 1 -- 172.21.15.70:0/690358920 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576800299 0 0) 0x7f4c9c0ccab0 con 0x7f4c9c0b0960 2023-08-25T09:52:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4c9b7fe640 1 -- 172.21.15.70:0/690358920 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c8c003520 con 0x7f4c9c0b0960 2023-08-25T09:52:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4c9bfff640 1 -- 172.21.15.70:0/690358920 >> v2:172.21.15.155:3300/0 conn(0x7f4c9c0cd7e0 msgr2=0x7f4c9c0cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4c9bfff640 1 --2- 172.21.15.70:0/690358920 >> v2:172.21.15.155:3300/0 conn(0x7f4c9c0cd7e0 0x7f4c9c0cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4c9bfff640 1 -- 172.21.15.70:0/690358920 >> v1:172.21.15.155:6789/0 conn(0x7f4c9c0b0960 legacy=0x7f4c9c0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4c9bfff640 1 -- 172.21.15.70:0/690358920 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c0ccfe0 con 0x7f4c9c0cb400 2023-08-25T09:52:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.906+0000 7f4ca14bf640 1 --2- 172.21.15.70:0/690358920 >> v2:172.21.15.155:3300/0 conn(0x7f4c9c0cd7e0 0x7f4c9c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:52:56.911 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.908+0000 7f4c9bfff640 1 --2- 172.21.15.70:0/690358920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cb400 0x7f4c9c06b870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c880097a0 tx=0x7f4c8802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=183bb2058d5aa099 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.908+0000 7f4c9b7fe640 1 -- 172.21.15.70:0/690358920 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88031780 con 0x7f4c9c0cb400 2023-08-25T09:52:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.908+0000 7f4c9b7fe640 1 -- 172.21.15.70:0/690358920 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c880318e0 con 0x7f4c9c0cb400 2023-08-25T09:52:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.908+0000 7f4c9b7fe640 1 -- 172.21.15.70:0/690358920 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88031c70 con 0x7f4c9c0cb400 2023-08-25T09:52:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 -- 172.21.15.70:0/690358920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cb400 msgr2=0x7f4c9c06b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.912 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/690358920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cb400 0x7f4c9c06b870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c880097a0 tx=0x7f4c8802b820 comp rx=0 tx=0).stop 2023-08-25T09:52:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 -- 172.21.15.70:0/690358920 shutdown_connections 2023-08-25T09:52:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/690358920 >> v2:172.21.15.155:3300/0 conn(0x7f4c9c0cd7e0 0x7f4c9c0cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/690358920 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cb400 0x7f4c9c06b870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 -- 172.21.15.70:0/690358920 >> 172.21.15.70:0/690358920 conn(0x7f4c9c0cb960 msgr2=0x7f4c9c0cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 -- 172.21.15.70:0/690358920 shutdown_connections 2023-08-25T09:52:56.913 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.909+0000 7f4ca2f22640 1 -- 172.21.15.70:0/690358920 wait complete. 2023-08-25T09:52:56.914 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.911+0000 7f4ca2f22640 1 Processor -- start 2023-08-25T09:52:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.912+0000 7f4ca2f22640 1 -- start start 2023-08-25T09:52:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca2f22640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 0x7f4c9c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca2f22640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c9c0cb400 0x7f4c9c0c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca2f22640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cd7e0 0x7f4c9c0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.915 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca2f22640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c15ef60 con 0x7f4c9c0b0960 2023-08-25T09:52:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca2f22640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c15ba10 con 0x7f4c9c0cb400 2023-08-25T09:52:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca2f22640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c15f1b0 con 0x7f4c9c0cd7e0 2023-08-25T09:52:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca0cbe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 0x7f4c9c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca0cbe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 0x7f4c9c15b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39626/0 (socket says 172.21.15.70:39626) 2023-08-25T09:52:56.916 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.913+0000 7f4ca0cbe640 1 -- 172.21.15.70:0/1284926790 learned_addr learned my addr 172.21.15.70:0/1284926790 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4c9bfff640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c9c0cb400 0x7f4c9c0c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca0cbe640 1 -- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cd7e0 msgr2=0x7f4c9c0c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca14bf640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cd7e0 0x7f4c9c0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca0cbe640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cd7e0 0x7f4c9c0c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca0cbe640 1 -- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c9c0cb400 msgr2=0x7f4c9c0c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:56.918 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca0cbe640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c9c0cb400 0x7f4c9c0c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca0cbe640 1 -- 172.21.15.70:0/1284926790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009a70 con 0x7f4c9c0b0960 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4c9bfff640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c9c0cb400 0x7f4c9c0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca0cbe640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 0x7f4c9c15b4d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c410 tx=0x7f4c9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c90005ce0 con 0x7f4c9c0b0960 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c90005e40 con 0x7f4c9c0b0960 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c88009400 con 0x7f4c9c0b0960 2023-08-25T09:52:56.919 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c900072f0 con 0x7f4c9c0b0960 2023-08-25T09:52:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.914+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c15f770 con 0x7f4c9c0b0960 2023-08-25T09:52:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.916+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c066fb0 con 0x7f4c9c0b0960 2023-08-25T09:52:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.917+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4c90007510 con 0x7f4c9c0b0960 2023-08-25T09:52:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.918+0000 7f4c997fa640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8006a050 0x7f4c8006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.918+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c90087bb0 con 0x7f4c9c0b0960 2023-08-25T09:52:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.918+0000 7f4c9bfff640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8006a050 0x7f4c8006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:56.921 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.918+0000 7f4c9bfff640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8006a050 0x7f4c8006c510 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4c88036000 tx=0x7f4c8803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:56.923 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:56.922+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4c900533f0 con 0x7f4c9c0b0960 2023-08-25T09:52:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.111+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c9c0b29c0 con 0x7f4c9c0b0960 2023-08-25T09:52:57.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.112+0000 7f4c997fa640 1 -- 172.21.15.70:0/1284926790 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v201) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c900170a0 con 0x7f4c9c0b0960 2023-08-25T09:52:57.114 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:52:57.116 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8006a050 msgr2=0x7f4c8006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8006a050 0x7f4c8006c510 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4c88036000 tx=0x7f4c8803d040 comp rx=0 tx=0).stop 2023-08-25T09:52:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 msgr2=0x7f4c9c15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 0x7f4c9c15b4d0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000c410 tx=0x7f4c9000c8d0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.117 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 shutdown_connections 2023-08-25T09:52:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4c8006a050 0x7f4c8006c510 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4c9c0cd7e0 0x7f4c9c0c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4c9c0cb400 0x7f4c9c0c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 --2- 172.21.15.70:0/1284926790 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4c9c0b0960 0x7f4c9c15b4d0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.118 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 >> 172.21.15.70:0/1284926790 conn(0x7f4c9c0cb960 msgr2=0x7f4c9c0ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:57.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 shutdown_connections 2023-08-25T09:52:57.119 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.114+0000 7f4ca2f22640 1 -- 172.21.15.70:0/1284926790 wait complete. 2023-08-25T09:52:57.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:52:57.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.356+0000 7f3e67b5d640 1 Processor -- start 2023-08-25T09:52:57.357 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.356+0000 7f3e67b5d640 1 -- start start 2023-08-25T09:52:57.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.357+0000 7f3e67b5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60072a90 0x7f3e6006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.359 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.357+0000 7f3e67b5d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e60072e70 0x7f3e60070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.357+0000 7f3e67b5d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e600cb7e0 con 0x7f3e600709b0 2023-08-25T09:52:57.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.357+0000 7f3e67b5d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e600cc080 con 0x7f3e60072e70 2023-08-25T09:52:57.360 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.357+0000 7f3e67b5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e600cbac0 con 0x7f3e60072a90 2023-08-25T09:52:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.358+0000 7f3e658f9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60072a90 0x7f3e6006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.362 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.358+0000 7f3e650f8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e60072e70 0x7f3e60070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.358+0000 7f3e650f8640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3e60072e70 0x7f3e60070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:33706/0 (socket says 172.21.15.70:33706) 2023-08-25T09:52:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.358+0000 7f3e650f8640 1 -- 172.21.15.70:0/2111684804 learned_addr learned my addr 172.21.15.70:0/2111684804 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.359+0000 7f3e650f8640 1 -- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60072a90 msgr2=0x7f3e6006aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:57.363 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.359+0000 7f3e650f8640 1 --2- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60072a90 0x7f3e6006aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.359+0000 7f3e650f8640 1 -- 172.21.15.70:0/2111684804 >> v1:172.21.15.155:6789/0 conn(0x7f3e600709b0 legacy=0x7f3e6015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.359+0000 7f3e650f8640 1 -- 172.21.15.70:0/2111684804 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e600cbda0 con 0x7f3e60072e70 2023-08-25T09:52:57.364 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.361+0000 7f3e650f8640 1 --2- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e60070470 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009870 tx=0x7f3e5402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd39d79aec1006ce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e648f7640 1 -- 172.21.15.70:0/2111684804 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54030840 con 0x7f3e60072e70 2023-08-25T09:52:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e648f7640 1 -- 172.21.15.70:0/2111684804 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e540309a0 con 0x7f3e60072e70 2023-08-25T09:52:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e648f7640 1 -- 172.21.15.70:0/2111684804 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54030d30 con 0x7f3e60072e70 2023-08-25T09:52:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e67b5d640 1 -- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 msgr2=0x7f3e60070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.365 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e60070470 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009870 tx=0x7f3e5402b980 comp rx=0 tx=0).stop 2023-08-25T09:52:57.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e67b5d640 1 -- 172.21.15.70:0/2111684804 shutdown_connections 2023-08-25T09:52:57.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e60070470 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/2111684804 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e60072a90 0x7f3e6006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.362+0000 7f3e67b5d640 1 -- 172.21.15.70:0/2111684804 >> 172.21.15.70:0/2111684804 conn(0x7f3e600cf9c0 msgr2=0x7f3e600cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:57.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.363+0000 7f3e67b5d640 1 -- 172.21.15.70:0/2111684804 shutdown_connections 2023-08-25T09:52:57.366 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.363+0000 7f3e67b5d640 1 -- 172.21.15.70:0/2111684804 wait complete. 2023-08-25T09:52:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.364+0000 7f3e67b5d640 1 Processor -- start 2023-08-25T09:52:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.365+0000 7f3e67b5d640 1 -- start start 2023-08-25T09:52:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e67b5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e600709b0 0x7f3e601ef930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e67b5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 0x7f3e60146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e67b5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e601478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.368 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e67b5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e601efe70 con 0x7f3e60072a90 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e67b5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60147370 con 0x7f3e60072e70 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e67b5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e60144780 con 0x7f3e600709b0 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e650f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 0x7f3e60146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e650f8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 0x7f3e60146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39642/0 (socket says 172.21.15.70:39642) 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e650f8640 1 -- 172.21.15.70:0/4262037052 learned_addr learned my addr 172.21.15.70:0/4262037052 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e660fa640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e601478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.369 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.366+0000 7f3e658f9640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e600709b0 0x7f3e601ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e650f8640 1 -- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e600709b0 msgr2=0x7f3e601ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e650f8640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e600709b0 0x7f3e601ef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e650f8640 1 -- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 msgr2=0x7f3e601478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e650f8640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e601478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.370 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e650f8640 1 -- 172.21.15.70:0/4262037052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540094d0 con 0x7f3e60072a90 2023-08-25T09:52:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e650f8640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 0x7f3e60146e30 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f3e5402b9c0 tx=0x7f3e54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5403e040 con 0x7f3e60072a90 2023-08-25T09:52:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e54039cb0 con 0x7f3e60072a90 2023-08-25T09:52:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e60144a60 con 0x7f3e60072a90 2023-08-25T09:52:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e54044630 con 0x7f3e60072a90 2023-08-25T09:52:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.367+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e60145020 con 0x7f3e60072a90 2023-08-25T09:52:57.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.368+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e60066fb0 con 0x7f3e60072a90 2023-08-25T09:52:57.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.369+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e54043390 con 0x7f3e60072a90 2023-08-25T09:52:57.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.370+0000 7f3e5e7fc640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e4c06a050 0x7f3e4c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.373 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.370+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e540aa980 con 0x7f3e60072a90 2023-08-25T09:52:57.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.370+0000 7f3e658f9640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e4c06a050 0x7f3e4c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.374 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.371+0000 7f3e658f9640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e4c06a050 0x7f3e4c06c510 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009ad0 tx=0x7f3e50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:57.376 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.375+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3e540761a0 con 0x7f3e60072a90 2023-08-25T09:52:57.570 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.569+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e6006a010 con 0x7f3e60072a90 2023-08-25T09:52:57.571 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.569+0000 7f3e5e7fc640 1 -- 172.21.15.70:0/4262037052 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v202) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3e540797c0 con 0x7f3e60072a90 2023-08-25T09:52:57.571 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:52:57.574 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e4c06a050 msgr2=0x7f3e4c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e4c06a050 0x7f3e4c06c510 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3e50009ad0 tx=0x7f3e50008040 comp rx=0 tx=0).stop 2023-08-25T09:52:57.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 msgr2=0x7f3e60146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.575 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 0x7f3e60146e30 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f3e5402b9c0 tx=0x7f3e54004730 comp rx=0 tx=0).stop 2023-08-25T09:52:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 shutdown_connections 2023-08-25T09:52:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3e4c06a050 0x7f3e4c06c510 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3e60072e70 0x7f3e601478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3e60072a90 0x7f3e60146e30 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.577 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 --2- 172.21.15.70:0/4262037052 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3e600709b0 0x7f3e601ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.572+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 >> 172.21.15.70:0/4262037052 conn(0x7f3e600cf9c0 msgr2=0x7f3e600c52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.573+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 shutdown_connections 2023-08-25T09:52:57.578 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.573+0000 7f3e67b5d640 1 -- 172.21.15.70:0/4262037052 wait complete. 2023-08-25T09:52:57.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:52:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.795+0000 7f4d3357f640 1 Processor -- start 2023-08-25T09:52:57.797 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.796+0000 7f4d3357f640 1 -- start start 2023-08-25T09:52:57.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.797+0000 7f4d3357f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f4d2c15ccd0 0x7f4d2c0d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.797+0000 7f4d3357f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d2c0d2400 0x7f4d2c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.800 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.797+0000 7f4d3357f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c0d09f0 0x7f4d2c0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.797+0000 7f4d3357f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c15d0b0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.797+0000 7f4d3357f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d2c0b58d0 con 0x7f4d2c15ccd0 2023-08-25T09:52:57.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.797+0000 7f4d3357f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0d0e80 con 0x7f4d2c0d09f0 2023-08-25T09:52:57.801 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d2c0d2400 0x7f4d2c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4d2c0d2400 0x7f4d2c0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39656/0 (socket says 172.21.15.70:39656) 2023-08-25T09:52:57.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 -- 172.21.15.70:0/3305749391 learned_addr learned my addr 172.21.15.70:0/3305749391 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:57.802 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d32d7e640 1 --2- 172.21.15.70:0/3305749391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c0d09f0 0x7f4d2c0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d3257d640 1 --2- 172.21.15.70:0/3305749391 >> v2:172.21.15.155:3300/0 conn(0x7f4d2c15ccd0 0x7f4d2c0d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 -- 172.21.15.70:0/3305749391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c0d09f0 msgr2=0x7f4d2c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:57.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 --2- 172.21.15.70:0/3305749391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c0d09f0 0x7f4d2c0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.803 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 -- 172.21.15.70:0/3305749391 >> v2:172.21.15.155:3300/0 conn(0x7f4d2c15ccd0 msgr2=0x7f4d2c0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:57.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 --2- 172.21.15.70:0/3305749391 >> v2:172.21.15.155:3300/0 conn(0x7f4d2c15ccd0 0x7f4d2c0d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.804 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.798+0000 7f4d31d7c640 1 -- 172.21.15.70:0/3305749391 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0d1160 con 0x7f4d2c0d2400 2023-08-25T09:52:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.799+0000 7f4d31d7c640 1 --2- 172.21.15.70:0/3305749391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0cf2e0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c08b9a0 tx=0x7f4d2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9523062f084c3b1b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3157b640 1 -- 172.21.15.70:0/3305749391 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031870 con 0x7f4d2c0d2400 2023-08-25T09:52:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3157b640 1 -- 172.21.15.70:0/3305749391 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d200319d0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3157b640 1 -- 172.21.15.70:0/3305749391 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d20031d60 con 0x7f4d2c0d2400 2023-08-25T09:52:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 -- 172.21.15.70:0/3305749391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 msgr2=0x7f4d2c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.807 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 --2- 172.21.15.70:0/3305749391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0cf2e0 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c08b9a0 tx=0x7f4d2002b910 comp rx=0 tx=0).stop 2023-08-25T09:52:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 -- 172.21.15.70:0/3305749391 shutdown_connections 2023-08-25T09:52:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 --2- 172.21.15.70:0/3305749391 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c0d09f0 0x7f4d2c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 --2- 172.21.15.70:0/3305749391 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0cf2e0 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 --2- 172.21.15.70:0/3305749391 >> v2:172.21.15.155:3300/0 conn(0x7f4d2c15ccd0 0x7f4d2c0d1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 -- 172.21.15.70:0/3305749391 >> 172.21.15.70:0/3305749391 conn(0x7f4d2c069850 msgr2=0x7f4d2c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:57.808 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 -- 172.21.15.70:0/3305749391 shutdown_connections 2023-08-25T09:52:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.800+0000 7f4d3357f640 1 -- 172.21.15.70:0/3305749391 wait complete. 2023-08-25T09:52:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.802+0000 7f4d3357f640 1 Processor -- start 2023-08-25T09:52:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.802+0000 7f4d3357f640 1 -- start start 2023-08-25T09:52:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3357f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3357f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c15ccd0 0x7f4d2c151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3357f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d2c15ef70 0x7f4d2c1e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3357f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0b2880 con 0x7f4d2c0d2400 2023-08-25T09:52:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3357f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0b1fe0 con 0x7f4d2c15ef70 2023-08-25T09:52:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3357f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2c0b22c0 con 0x7f4d2c15ccd0 2023-08-25T09:52:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3257d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.810 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3257d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39662/0 (socket says 172.21.15.70:39662) 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.803+0000 7f4d3257d640 1 -- 172.21.15.70:0/768569193 learned_addr learned my addr 172.21.15.70:0/768569193 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3257d640 1 -- 172.21.15.70:0/768569193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c15ccd0 msgr2=0x7f4d2c151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3257d640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c15ccd0 0x7f4d2c151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3257d640 1 -- 172.21.15.70:0/768569193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d2c15ef70 msgr2=0x7f4d2c1e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3257d640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d2c15ef70 0x7f4d2c1e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3257d640 1 -- 172.21.15.70:0/768569193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200094d0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.811 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3257d640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0b1790 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c007e40 tx=0x7f4d1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c014620 con 0x7f4d2c0d2400 2023-08-25T09:52:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1c00ee90 con 0x7f4d2c0d2400 2023-08-25T09:52:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c0149c0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d2c0b25a0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.804+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d2c0b1cd0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.812 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.806+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d2c1536f0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.806+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4d1c014b20 con 0x7f4d2c0d2400 2023-08-25T09:52:57.813 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.807+0000 7f4d2affd640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d1806a050 0x7f4d1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.807+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d1c088eb0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.808+0000 7f4d31d7c640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d1806a050 0x7f4d1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.808+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f4d180721a0 con 0x7f4d2c0d2400 2023-08-25T09:52:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.808+0000 7f4d31d7c640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d1806a050 0x7f4d1806c510 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4d20036000 tx=0x7f4d20030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:57.814 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.812+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4d1c054630 con 0x7f4d2c0d2400 2023-08-25T09:52:58.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:57.999+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d2c0d3a20 con 0x7f4d2c0d2400 2023-08-25T09:52:58.002 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.000+0000 7f4d2affd640 1 -- 172.21.15.70:0/768569193 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v202) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d1c057c50 con 0x7f4d2c0d2400 2023-08-25T09:52:58.002 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:52:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d1806a050 msgr2=0x7f4d1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d1806a050 0x7f4d1806c510 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f4d20036000 tx=0x7f4d20030000 comp rx=0 tx=0).stop 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 msgr2=0x7f4d2c0b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0b1790 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c007e40 tx=0x7f4d1c00e4b0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 shutdown_connections 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4d1806a050 0x7f4d1806c510 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4d2c15ef70 0x7f4d2c1e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4d2c15ccd0 0x7f4d2c151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 --2- 172.21.15.70:0/768569193 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4d2c0d2400 0x7f4d2c0b1790 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 >> 172.21.15.70:0/768569193 conn(0x7f4d2c069850 msgr2=0x7f4d2c0cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:58.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 shutdown_connections 2023-08-25T09:52:58.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.003+0000 7f4d3357f640 1 -- 172.21.15.70:0/768569193 wait complete. 2023-08-25T09:52:58.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T09:52:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.228+0000 7f80762a7640 1 Processor -- start 2023-08-25T09:52:58.230 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.229+0000 7f80762a7640 1 -- start start 2023-08-25T09:52:58.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.230+0000 7f80762a7640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f807015acd0 0x7f807015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:58.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.230+0000 7f80762a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80700ca780 0x7f80700cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:58.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.230+0000 7f80762a7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80700b12b0 con 0x7f80700b3cf0 2023-08-25T09:52:58.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.230+0000 7f80762a7640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807006c0e0 con 0x7f807015acd0 2023-08-25T09:52:58.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.230+0000 7f80762a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807006ae20 con 0x7f80700ca780 2023-08-25T09:52:58.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.230+0000 7f806ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f807015acd0 0x7f807015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f806ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f807015acd0 0x7f807015b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39672/0 (socket says 172.21.15.70:39672) 2023-08-25T09:52:58.234 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f806ffff640 1 -- 172.21.15.70:0/1892581756 learned_addr learned my addr 172.21.15.70:0/1892581756 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:58.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f806ffff640 1 -- 172.21.15.70:0/1892581756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80700ca780 msgr2=0x7f80700cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:58.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f80677fe640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80700ca780 0x7f80700cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:58.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f806ffff640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80700ca780 0x7f80700cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f806ffff640 1 -- 172.21.15.70:0/1892581756 >> v1:172.21.15.155:6789/0 conn(0x7f80700b3cf0 legacy=0x7f80700b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f806ffff640 1 -- 172.21.15.70:0/1892581756 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807006b910 con 0x7f807015acd0 2023-08-25T09:52:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.231+0000 7f80677fe640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80700ca780 0x7f80700cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.232+0000 7f806ffff640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807015acd0 0x7f807015b0b0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f8060009850 tx=0x7f806002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4297d2ea70d2008b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.232+0000 7f806f7fe640 1 -- 172.21.15.70:0/1892581756 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806002e070 con 0x7f807015acd0 2023-08-25T09:52:58.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f806f7fe640 1 -- 172.21.15.70:0/1892581756 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060004030 con 0x7f807015acd0 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f806f7fe640 1 -- 172.21.15.70:0/1892581756 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060004420 con 0x7f807015acd0 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 -- 172.21.15.70:0/1892581756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807015acd0 msgr2=0x7f807015b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807015acd0 0x7f807015b0b0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f8060009850 tx=0x7f806002b840 comp rx=0 tx=0).stop 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 -- 172.21.15.70:0/1892581756 shutdown_connections 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f80700ca780 0x7f80700cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 --2- 172.21.15.70:0/1892581756 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f807015acd0 0x7f807015b0b0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 -- 172.21.15.70:0/1892581756 >> 172.21.15.70:0/1892581756 conn(0x7f80700ce760 msgr2=0x7f80700ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:58.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 -- 172.21.15.70:0/1892581756 shutdown_connections 2023-08-25T09:52:58.239 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.233+0000 7f80762a7640 1 -- 172.21.15.70:0/1892581756 wait complete. 2023-08-25T09:52:58.240 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.235+0000 7f80762a7640 1 Processor -- start 2023-08-25T09:52:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.236+0000 7f80762a7640 1 -- start start 2023-08-25T09:52:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f80762a7640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 0x7f80701567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f80762a7640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80700ca780 0x7f80701528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f80762a7640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f807015acd0 0x7f8070144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f80762a7640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8070158e50 con 0x7f80700b3cf0 2023-08-25T09:52:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f80762a7640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8070156db0 con 0x7f80700ca780 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f80762a7640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8070157090 con 0x7f807015acd0 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f806ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 0x7f80701567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f806ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 0x7f80701567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39682/0 (socket says 172.21.15.70:39682) 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f806ffff640 1 -- 172.21.15.70:0/2649102565 learned_addr learned my addr 172.21.15.70:0/2649102565 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.237+0000 7f8074844640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f807015acd0 0x7f8070144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806ffff640 1 -- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f807015acd0 msgr2=0x7f8070144fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.242 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f80677fe640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80700ca780 0x7f80701528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806ffff640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f807015acd0 0x7f8070144fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806ffff640 1 -- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80700ca780 msgr2=0x7f80701528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806ffff640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80700ca780 0x7f80701528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806ffff640 1 -- 172.21.15.70:0/2649102565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009460 con 0x7f80700b3cf0 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f80677fe640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80700ca780 0x7f80701528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806ffff640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 0x7f80701567d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f8060035040 tx=0x7f80600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:58.243 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060003c20 con 0x7f80700b3cf0 2023-08-25T09:52:58.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060003d80 con 0x7f80700b3cf0 2023-08-25T09:52:58.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060006280 con 0x7f80700b3cf0 2023-08-25T09:52:58.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.238+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8070157370 con 0x7f80700b3cf0 2023-08-25T09:52:58.244 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.239+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701525f0 con 0x7f80700b3cf0 2023-08-25T09:52:58.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.240+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807004e420 con 0x7f80700b3cf0 2023-08-25T09:52:58.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.240+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f80600063e0 con 0x7f80700b3cf0 2023-08-25T09:52:58.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.241+0000 7f806d7fa640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f805406a050 0x7f805406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:58.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.241+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f80600aa6c0 con 0x7f80700b3cf0 2023-08-25T09:52:58.245 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.242+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f80540721a0 con 0x7f80700b3cf0 2023-08-25T09:52:58.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.242+0000 7f80677fe640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f805406a050 0x7f805406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:58.246 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.242+0000 7f80677fe640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f805406a050 0x7f805406c510 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:58.248 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.247+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8060075d10 con 0x7f80700b3cf0 2023-08-25T09:52:58.448 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.447+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8070153250 con 0x7f80700b3cf0 2023-08-25T09:52:58.449 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.447+0000 7f806d7fa640 1 -- 172.21.15.70:0/2649102565 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v202) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8060042080 con 0x7f80700b3cf0 2023-08-25T09:52:58.449 INFO:teuthology.orchestra.run.smithi070.stderr:listed 0 entries 2023-08-25T09:52:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f805406a050 msgr2=0x7f805406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f805406a050 0x7f805406c510 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f8058006fd0 tx=0x7f8058008040 comp rx=0 tx=0).stop 2023-08-25T09:52:58.452 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 msgr2=0x7f80701567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:58.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 0x7f80701567d0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f8060035040 tx=0x7f80600046c0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 shutdown_connections 2023-08-25T09:52:58.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f805406a050 0x7f805406c510 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f807015acd0 0x7f8070144fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.453 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f80700ca780 0x7f80701528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 --2- 172.21.15.70:0/2649102565 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f80700b3cf0 0x7f80701567d0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 >> 172.21.15.70:0/2649102565 conn(0x7f80700ce760 msgr2=0x7f807015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 shutdown_connections 2023-08-25T09:52:58.454 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:58.450+0000 7f80762a7640 1 -- 172.21.15.70:0/2649102565 wait complete. 2023-08-25T09:52:58.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi155.front.sepia.ceph.com... 2023-08-25T09:52:58.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:58.471 DEBUG:teuthology.orchestra.run.smithi155:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-08-25T09:52:58.571 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi155.stderr:ceph-fuse[94138]: fuse finished with error 0 and tester_r 0 2023-08-25T09:52:58.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:52:58.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:58.580 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T09:52:58.699 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:52:58.700 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:58.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:52:58.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:52:58.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:58.762 DEBUG:teuthology.orchestra.run.smithi070:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T09:52:58.815 INFO:teuthology.orchestra.run.smithi070.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-08-25T09:52:58.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:52:58.816 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:52:58.817 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:52:58.876 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:52:58.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:52:58.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:52:58.878 DEBUG:teuthology.orchestra.run.smithi155:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T09:52:58.930 INFO:teuthology.orchestra.run.smithi155.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-08-25T09:52:58.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:52:58.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:52:59.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.151+0000 7fe459979640 1 Processor -- start 2023-08-25T09:52:59.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.152+0000 7fe459979640 1 -- start start 2023-08-25T09:52:59.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.153+0000 7fe459979640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4540ce5b0 0x7fe4540ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.155 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.153+0000 7fe459979640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe45406f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.153+0000 7fe459979640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4540ceed0 con 0x7fe45415aa10 2023-08-25T09:52:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.153+0000 7fe459979640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe454159d10 con 0x7fe4540ce5b0 2023-08-25T09:52:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.153+0000 7fe459979640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45415d3f0 con 0x7fe4540b2850 2023-08-25T09:52:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4540ce5b0 0x7fe4540ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe4540ce5b0 0x7fe4540ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60218/0 (socket says 172.21.15.70:60218) 2023-08-25T09:52:59.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 -- 172.21.15.70:0/4056579710 learned_addr learned my addr 172.21.15.70:0/4056579710 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 -- 172.21.15.70:0/4056579710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 msgr2=0x7fe45406f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 --2- 172.21.15.70:0/4056579710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe45406f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 -- 172.21.15.70:0/4056579710 >> v1:172.21.15.70:6789/0 conn(0x7fe45415aa10 legacy=0x7fe4540b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.154+0000 7fe452ffd640 1 -- 172.21.15.70:0/4056579710 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540b2d20 con 0x7fe4540ce5b0 2023-08-25T09:52:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.155+0000 7fe452ffd640 1 --2- 172.21.15.70:0/4056579710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4540ce5b0 0x7fe4540ce990 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fe444009870 tx=0x7fe44402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45305ed1c0ac3aa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:59.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.155+0000 7fe4527fc640 1 -- 172.21.15.70:0/4056579710 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444031640 con 0x7fe4540ce5b0 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe4527fc640 1 -- 172.21.15.70:0/4056579710 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4440317a0 con 0x7fe4540ce5b0 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe4527fc640 1 -- 172.21.15.70:0/4056579710 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444031b30 con 0x7fe4540ce5b0 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 -- 172.21.15.70:0/4056579710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4540ce5b0 msgr2=0x7fe4540ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 --2- 172.21.15.70:0/4056579710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4540ce5b0 0x7fe4540ce990 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fe444009870 tx=0x7fe44402b8f0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 -- 172.21.15.70:0/4056579710 shutdown_connections 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 --2- 172.21.15.70:0/4056579710 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe45406f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 --2- 172.21.15.70:0/4056579710 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe4540ce5b0 0x7fe4540ce990 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 -- 172.21.15.70:0/4056579710 >> 172.21.15.70:0/4056579710 conn(0x7fe454069850 msgr2=0x7fe454069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:59.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 -- 172.21.15.70:0/4056579710 shutdown_connections 2023-08-25T09:52:59.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.156+0000 7fe459979640 1 -- 172.21.15.70:0/4056579710 wait complete. 2023-08-25T09:52:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.158+0000 7fe459979640 1 Processor -- start 2023-08-25T09:52:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.158+0000 7fe459979640 1 -- start start 2023-08-25T09:52:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe459979640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe454156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe459979640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4540ce5b0 0x7fe4541560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe459979640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 0x7fe4541565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe459979640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540a9280 con 0x7fe45415aa10 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe459979640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540a8600 con 0x7fe4540ce5b0 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe459979640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540a88e0 con 0x7fe4540b2850 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 0x7fe4541565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 0x7fe4541565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60226/0 (socket says 172.21.15.70:60226) 2023-08-25T09:52:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.159+0000 7fe4537fe640 1 -- 172.21.15.70:0/3687488388 learned_addr learned my addr 172.21.15.70:0/3687488388 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe452ffd640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe454156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe44bfff640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4540ce5b0 0x7fe4541560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe4537fe640 1 -- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 msgr2=0x7fe454156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe4537fe640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe454156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe4537fe640 1 -- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4540ce5b0 msgr2=0x7fe4541560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe4537fe640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4540ce5b0 0x7fe4541560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe4537fe640 1 -- 172.21.15.70:0/3687488388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440094d0 con 0x7fe45415aa10 2023-08-25T09:52:59.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe44bfff640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4540ce5b0 0x7fe4541560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:52:59.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe4537fe640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 0x7fe4541565f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b9e0 tx=0x7fe44c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.160+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00cb20 con 0x7fe45415aa10 2023-08-25T09:52:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.161+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00cca0 con 0x7fe45415aa10 2023-08-25T09:52:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.161+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c014680 con 0x7fe45415aa10 2023-08-25T09:52:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.161+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4540a8bc0 con 0x7fe45415aa10 2023-08-25T09:52:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.161+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45415ba70 con 0x7fe45415aa10 2023-08-25T09:52:59.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.162+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4540a93c0 con 0x7fe45415aa10 2023-08-25T09:52:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.163+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe44c01d3b0 con 0x7fe45415aa10 2023-08-25T09:52:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.164+0000 7fe450ff9640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe43406a050 0x7fe43406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.164+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe44c088480 con 0x7fe45415aa10 2023-08-25T09:52:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.164+0000 7fe452ffd640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe43406a050 0x7fe43406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.164+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7fe4340721a0 con 0x7fe45415aa10 2023-08-25T09:52:59.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.165+0000 7fe452ffd640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe43406a050 0x7fe43406c510 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe444036000 tx=0x7fe44403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:59.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.168+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe4540a93c0 con 0x7fe45415aa10 2023-08-25T09:52:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.395+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4540a93c0 con 0x7fe45415aa10 2023-08-25T09:52:59.397 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.396+0000 7fe450ff9640 1 -- 172.21.15.70:0/3687488388 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fe4540a93c0 con 0x7fe45415aa10 2023-08-25T09:52:59.397 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:59.398 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:52:59.398 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:52:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe43406a050 msgr2=0x7fe43406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe43406a050 0x7fe43406c510 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe444036000 tx=0x7fe44403c040 comp rx=0 tx=0).stop 2023-08-25T09:52:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 msgr2=0x7fe4541565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.400 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 0x7fe4541565f0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b9e0 tx=0x7fe44c00bea0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 shutdown_connections 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe43406a050 0x7fe43406c510 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe45415aa10 0x7fe4541565f0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe4540ce5b0 0x7fe4541560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.398+0000 7fe459979640 1 --2- 172.21.15.70:0/3687488388 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe4540b2850 0x7fe454156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.399+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 >> 172.21.15.70:0/3687488388 conn(0x7fe454069850 msgr2=0x7fe45415af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.399+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 shutdown_connections 2023-08-25T09:52:59.401 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.399+0000 7fe459979640 1 -- 172.21.15.70:0/3687488388 wait complete. 2023-08-25T09:52:59.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:52:59.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.624+0000 7fe9a206a640 1 Processor -- start 2023-08-25T09:52:59.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.624+0000 7fe9a206a640 1 -- start start 2023-08-25T09:52:59.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a206a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c0b2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a206a640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe99c15d3d0 0x7fe99c0b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a206a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe99c0cadb0 0x7fe99c0cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a206a640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c0c8b60 con 0x7fe99c0cadb0 2023-08-25T09:52:59.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a206a640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c15d890 con 0x7fe99c15d3d0 2023-08-25T09:52:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a206a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c15db70 con 0x7fe99c159840 2023-08-25T09:52:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a1869640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe99c0cadb0 0x7fe99c0cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a1869640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fe99c0cadb0 0x7fe99c0cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60232/0 (socket says 172.21.15.70:60232) 2023-08-25T09:52:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.626+0000 7fe9a1869640 1 -- 172.21.15.70:0/1322046708 learned_addr learned my addr 172.21.15.70:0/1322046708 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a1068640 1 --2- 172.21.15.70:0/1322046708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c0b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a1869640 1 -- 172.21.15.70:0/1322046708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 msgr2=0x7fe99c0b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a1869640 1 --2- 172.21.15.70:0/1322046708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c0b2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a1869640 1 -- 172.21.15.70:0/1322046708 >> v2:172.21.15.155:3300/0 conn(0x7fe99c15d3d0 msgr2=0x7fe99c0b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:52:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a0867640 1 --2- 172.21.15.70:0/1322046708 >> v2:172.21.15.155:3300/0 conn(0x7fe99c15d3d0 0x7fe99c0b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a1869640 1 --2- 172.21.15.70:0/1322046708 >> v2:172.21.15.155:3300/0 conn(0x7fe99c15d3d0 0x7fe99c0b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.627+0000 7fe9a1869640 1 -- 172.21.15.70:0/1322046708 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c15de50 con 0x7fe99c0cadb0 2023-08-25T09:52:59.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.628+0000 7fe9a1869640 1 --2- 172.21.15.70:0/1322046708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0cb1e0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fe99c06bef0 tx=0x7fe99402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5a0483985713e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:59.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.628+0000 7fe993fff640 1 -- 172.21.15.70:0/1322046708 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99402ca70 con 0x7fe99c0cadb0 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.628+0000 7fe993fff640 1 -- 172.21.15.70:0/1322046708 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe99402cbd0 con 0x7fe99c0cadb0 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.628+0000 7fe993fff640 1 -- 172.21.15.70:0/1322046708 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940343f0 con 0x7fe99c0cadb0 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 -- 172.21.15.70:0/1322046708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 msgr2=0x7fe99c0cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 --2- 172.21.15.70:0/1322046708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0cb1e0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fe99c06bef0 tx=0x7fe99402bed0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 -- 172.21.15.70:0/1322046708 shutdown_connections 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 --2- 172.21.15.70:0/1322046708 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0cb1e0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.637 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 --2- 172.21.15.70:0/1322046708 >> v2:172.21.15.155:3300/0 conn(0x7fe99c15d3d0 0x7fe99c0b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 --2- 172.21.15.70:0/1322046708 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c0b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 -- 172.21.15.70:0/1322046708 >> 172.21.15.70:0/1322046708 conn(0x7fe99c0affd0 msgr2=0x7fe99c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 -- 172.21.15.70:0/1322046708 shutdown_connections 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.629+0000 7fe9a206a640 1 -- 172.21.15.70:0/1322046708 wait complete. 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.631+0000 7fe9a206a640 1 Processor -- start 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.632+0000 7fe9a206a640 1 -- start start 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a206a640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.638 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a206a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a206a640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe99c15d3d0 0x7fe99c0af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a206a640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c0af210 con 0x7fe99c0cadb0 2023-08-25T09:52:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a206a640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c155600 con 0x7fe99c15d3d0 2023-08-25T09:52:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a206a640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c0ae870 con 0x7fe99c159840 2023-08-25T09:52:59.639 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a1068640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a1068640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60242/0 (socket says 172.21.15.70:60242) 2023-08-25T09:52:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a1068640 1 -- 172.21.15.70:0/4245996949 learned_addr learned my addr 172.21.15.70:0/4245996949 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:52:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a0867640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.633+0000 7fe9a1068640 1 -- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 msgr2=0x7fe99c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a1869640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe99c15d3d0 0x7fe99c0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.640 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a1068640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a1068640 1 -- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe99c15d3d0 msgr2=0x7fe99c0af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a1068640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe99c15d3d0 0x7fe99c0af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a1068640 1 -- 172.21.15.70:0/4245996949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994009b30 con 0x7fe99c0cadb0 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a1068640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0711a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c7f0 tx=0x7fe99800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998004220 con 0x7fe99c0cadb0 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe998004380 con 0x7fe99c0cadb0 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe998007860 con 0x7fe99c0cadb0 2023-08-25T09:52:59.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c0aeb50 con 0x7fe99c0cadb0 2023-08-25T09:52:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.634+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c0716e0 con 0x7fe99c0cadb0 2023-08-25T09:52:59.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.636+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c0af350 con 0x7fe99c0cadb0 2023-08-25T09:52:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.636+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe998007bd0 con 0x7fe99c0cadb0 2023-08-25T09:52:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.637+0000 7fe991ffb640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe98006a050 0x7fe98006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:52:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.637+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe9980885e0 con 0x7fe99c0cadb0 2023-08-25T09:52:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.638+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7fe9800721a0 con 0x7fe99c0cadb0 2023-08-25T09:52:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.638+0000 7fe9a0867640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe98006a050 0x7fe98006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:52:59.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.638+0000 7fe9a0867640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe98006a050 0x7fe98006c510 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0c50d0 tx=0x7fe98c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:52:59.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.642+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe99c0af350 con 0x7fe99c0cadb0 2023-08-25T09:52:59.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.832+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe99c0af350 con 0x7fe99c0cadb0 2023-08-25T09:52:59.835 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.834+0000 7fe991ffb640 1 -- 172.21.15.70:0/4245996949 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+18339 (secure 0 0 0) 0x7fe99c0af350 con 0x7fe99c0cadb0 2023-08-25T09:52:59.836 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:52:59.837 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":202,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:57.522412+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:52:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe98006a050 msgr2=0x7fe98006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe98006a050 0x7fe98006c510 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe99c0c50d0 tx=0x7fe98c008040 comp rx=0 tx=0).stop 2023-08-25T09:52:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 msgr2=0x7fe99c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:52:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0711a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fe99800c7f0 tx=0x7fe99800ccb0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 shutdown_connections 2023-08-25T09:52:59.838 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe98006a050 0x7fe98006c510 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe99c15d3d0 0x7fe99c0af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe99c159840 0x7fe99c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 --2- 172.21.15.70:0/4245996949 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe99c0cadb0 0x7fe99c0711a0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:52:59.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.836+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 >> 172.21.15.70:0/4245996949 conn(0x7fe99c0affd0 msgr2=0x7fe99c15a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:52:59.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.837+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 shutdown_connections 2023-08-25T09:52:59.839 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:52:59.837+0000 7fe9a206a640 1 -- 172.21.15.70:0/4245996949 wait complete. 2023-08-25T09:52:59.857 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T09:38:39.384505+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T09:38:45.046709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.980000019073486, 'score_stable': 4.980000019073486, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T09:52:02.296119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T09:52:03.097163+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '202', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T09:52:59.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T09:53:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.077+0000 7f355f4d2640 1 Processor -- start 2023-08-25T09:53:00.079 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.077+0000 7f355f4d2640 1 -- start start 2023-08-25T09:53:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.079+0000 7f355f4d2640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3558069850 0x7f355815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.079+0000 7f355f4d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f3558069c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.080 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.079+0000 7f355f4d2640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580ac880 con 0x7f3558160be0 2023-08-25T09:53:00.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.079+0000 7f355f4d2640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35580ad400 con 0x7f3558069850 2023-08-25T09:53:00.081 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.079+0000 7f355f4d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35580ad890 con 0x7f35580c9620 2023-08-25T09:53:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f3558069c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355d26e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f3558069850 0x7f355815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f3558069c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:43444/0 (socket says 172.21.15.70:43444) 2023-08-25T09:53:00.082 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 -- 172.21.15.70:0/1670619772 learned_addr learned my addr 172.21.15.70:0/1670619772 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 -- 172.21.15.70:0/1670619772 >> v2:172.21.15.155:3300/0 conn(0x7f3558069850 msgr2=0x7f355815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 --2- 172.21.15.70:0/1670619772 >> v2:172.21.15.155:3300/0 conn(0x7f3558069850 0x7f355815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.083 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 -- 172.21.15.70:0/1670619772 >> v1:172.21.15.155:6789/0 conn(0x7f3558160be0 legacy=0x7f3558161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.080+0000 7f355ca6d640 1 -- 172.21.15.70:0/1670619772 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580acad0 con 0x7f35580c9620 2023-08-25T09:53:00.084 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.082+0000 7f355ca6d640 1 --2- 172.21.15.70:0/1670619772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f3558069c30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3548009870 tx=0x7f354802ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f34800ae2e797d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.082+0000 7f354ffff640 1 -- 172.21.15.70:0/1670619772 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354802f830 con 0x7f35580c9620 2023-08-25T09:53:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.083+0000 7f354ffff640 1 -- 172.21.15.70:0/1670619772 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354802f990 con 0x7f35580c9620 2023-08-25T09:53:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.083+0000 7f354ffff640 1 -- 172.21.15.70:0/1670619772 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354802fd20 con 0x7f35580c9620 2023-08-25T09:53:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.083+0000 7f355f4d2640 1 -- 172.21.15.70:0/1670619772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 msgr2=0x7f3558069c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.085 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.083+0000 7f355f4d2640 1 --2- 172.21.15.70:0/1670619772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f3558069c30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3548009870 tx=0x7f354802ba10 comp rx=0 tx=0).stop 2023-08-25T09:53:00.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.084+0000 7f355f4d2640 1 -- 172.21.15.70:0/1670619772 shutdown_connections 2023-08-25T09:53:00.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.084+0000 7f355f4d2640 1 --2- 172.21.15.70:0/1670619772 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f3558069c30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.084+0000 7f355f4d2640 1 --2- 172.21.15.70:0/1670619772 >> v2:172.21.15.155:3300/0 conn(0x7f3558069850 0x7f355815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.084+0000 7f355f4d2640 1 -- 172.21.15.70:0/1670619772 >> 172.21.15.70:0/1670619772 conn(0x7f35580cb0d0 msgr2=0x7f35580cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.084+0000 7f355f4d2640 1 -- 172.21.15.70:0/1670619772 shutdown_connections 2023-08-25T09:53:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.084+0000 7f355f4d2640 1 -- 172.21.15.70:0/1670619772 wait complete. 2023-08-25T09:53:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.085+0000 7f355f4d2640 1 Processor -- start 2023-08-25T09:53:00.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.086+0000 7f355f4d2640 1 -- start start 2023-08-25T09:53:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355f4d2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 0x7f355814ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355f4d2640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f35580c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355f4d2640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3558160be0 0x7f35580c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.089 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355f4d2640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c7000 con 0x7f3558069850 2023-08-25T09:53:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355f4d2640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c6380 con 0x7f3558160be0 2023-08-25T09:53:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355f4d2640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35580c6660 con 0x7f35580c9620 2023-08-25T09:53:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355d26e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 0x7f355814ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355d26e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 0x7f355814ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60246/0 (socket says 172.21.15.70:60246) 2023-08-25T09:53:00.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355d26e640 1 -- 172.21.15.70:0/358458051 learned_addr learned my addr 172.21.15.70:0/358458051 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355ca6d640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f35580c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355da6f640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3558160be0 0x7f35580c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355d26e640 1 -- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 msgr2=0x7f35580c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355d26e640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f35580c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.087+0000 7f355d26e640 1 -- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3558160be0 msgr2=0x7f35580c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f355d26e640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3558160be0 0x7f35580c7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f355d26e640 1 -- 172.21.15.70:0/358458051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480094d0 con 0x7f3558069850 2023-08-25T09:53:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f355d26e640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 0x7f355814ea30 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f355400e9b0 tx=0x7f355400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3554009670 con 0x7f3558069850 2023-08-25T09:53:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355400c9a0 con 0x7f3558069850 2023-08-25T09:53:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355400cd50 con 0x7f3558069850 2023-08-25T09:53:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35580c6940 con 0x7f3558069850 2023-08-25T09:53:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.088+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355814ef70 con 0x7f3558069850 2023-08-25T09:53:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.089+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3558066fb0 con 0x7f3558069850 2023-08-25T09:53:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.090+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3554013470 con 0x7f3558069850 2023-08-25T09:53:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.091+0000 7f354dffb640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f353c06a080 0x7f353c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.091+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35540888f0 con 0x7f3558069850 2023-08-25T09:53:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.092+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f353c0721d0 con 0x7f3558069850 2023-08-25T09:53:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.092+0000 7f355ca6d640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f353c06a080 0x7f353c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.092+0000 7f355ca6d640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f353c06a080 0x7f353c06c540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f3548008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.097 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.096+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3554053ec0 con 0x7f3558069850 2023-08-25T09:53:00.286 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.285+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f355806a810 con 0x7f3558069850 2023-08-25T09:53:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.286+0000 7f354dffb640 1 -- 172.21.15.70:0/358458051 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v202) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35540170a0 con 0x7f3558069850 2023-08-25T09:53:00.287 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2023-08-25T09:53:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.288+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f353c06a080 msgr2=0x7f353c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.288+0000 7f355f4d2640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f353c06a080 0x7f353c06c540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3548036000 tx=0x7f3548008030 comp rx=0 tx=0).stop 2023-08-25T09:53:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.288+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 msgr2=0x7f355814ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.288+0000 7f355f4d2640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 0x7f355814ea30 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f355400e9b0 tx=0x7f355400ee70 comp rx=0 tx=0).stop 2023-08-25T09:53:00.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 shutdown_connections 2023-08-25T09:53:00.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f353c06a080 0x7f353c06c540 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3558160be0 0x7f35580c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.292 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f35580c9620 0x7f35580c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 --2- 172.21.15.70:0/358458051 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3558069850 0x7f355814ea30 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 >> 172.21.15.70:0/358458051 conn(0x7f35580cb0d0 msgr2=0x7f35580cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:00.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 shutdown_connections 2023-08-25T09:53:00.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.289+0000 7f355f4d2640 1 -- 172.21.15.70:0/358458051 wait complete. 2023-08-25T09:53:00.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T09:53:00.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.525+0000 7f709bfff640 1 Processor -- start 2023-08-25T09:53:00.526 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.525+0000 7f709bfff640 1 -- start start 2023-08-25T09:53:00.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.526+0000 7f709bfff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f709c0b0320 0x7f709c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.526+0000 7f709bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c0b1ce0 0x7f709c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.526+0000 7f709bfff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0ad8f0 con 0x7f709c15a920 2023-08-25T09:53:00.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.526+0000 7f709bfff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c0adbb0 con 0x7f709c0b0320 2023-08-25T09:53:00.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.526+0000 7f709bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c0c90a0 con 0x7f709c0b1ce0 2023-08-25T09:53:00.529 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f709c0b0320 0x7f709c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f709c0b0320 0x7f709c0c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60256/0 (socket says 172.21.15.70:60256) 2023-08-25T09:53:00.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 -- 172.21.15.70:0/1886704522 learned_addr learned my addr 172.21.15.70:0/1886704522 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:00.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709b7fe640 1 --2- 172.21.15.70:0/1886704522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c0b1ce0 0x7f709c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.530 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 -- 172.21.15.70:0/1886704522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c0b1ce0 msgr2=0x7f709c0ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 --2- 172.21.15.70:0/1886704522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c0b1ce0 0x7f709c0ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 -- 172.21.15.70:0/1886704522 >> v1:172.21.15.155:6789/0 conn(0x7f709c15a920 legacy=0x7f709c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.527+0000 7f709affd640 1 -- 172.21.15.70:0/1886704522 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0c9380 con 0x7f709c0b0320 2023-08-25T09:53:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.528+0000 7f709affd640 1 --2- 172.21.15.70:0/1886704522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c0c9e60 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f708c009850 tx=0x7f708c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36aef9fb58b3b8b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.531 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f7099ffb640 1 -- 172.21.15.70:0/1886704522 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c004230 con 0x7f709c0b0320 2023-08-25T09:53:00.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f7099ffb640 1 -- 172.21.15.70:0/1886704522 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c004390 con 0x7f709c0b0320 2023-08-25T09:53:00.532 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f7099ffb640 1 -- 172.21.15.70:0/1886704522 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c031830 con 0x7f709c0b0320 2023-08-25T09:53:00.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 -- 172.21.15.70:0/1886704522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 msgr2=0x7f709c0c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 --2- 172.21.15.70:0/1886704522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c0c9e60 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f708c009850 tx=0x7f708c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 -- 172.21.15.70:0/1886704522 shutdown_connections 2023-08-25T09:53:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 --2- 172.21.15.70:0/1886704522 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c0b1ce0 0x7f709c0ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 --2- 172.21.15.70:0/1886704522 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c0c9e60 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 -- 172.21.15.70:0/1886704522 >> 172.21.15.70:0/1886704522 conn(0x7f709c0ce830 msgr2=0x7f709c0cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 -- 172.21.15.70:0/1886704522 shutdown_connections 2023-08-25T09:53:00.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.529+0000 7f709bfff640 1 -- 172.21.15.70:0/1886704522 wait complete. 2023-08-25T09:53:00.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.531+0000 7f709bfff640 1 Processor -- start 2023-08-25T09:53:00.535 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.531+0000 7f709bfff640 1 -- start start 2023-08-25T09:53:00.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.532+0000 7f709bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.532+0000 7f709bfff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f709c0b1ce0 0x7f709c06fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.532+0000 7f709bfff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c15a920 0x7f709c070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.532+0000 7f709bfff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c06ee60 con 0x7f709c0b0320 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.532+0000 7f709bfff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c06f140 con 0x7f709c0b1ce0 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.532+0000 7f709bfff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c06f420 con 0x7f709c15a920 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60258/0 (socket says 172.21.15.70:60258) 2023-08-25T09:53:00.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 -- 172.21.15.70:0/1549249495 learned_addr learned my addr 172.21.15.70:0/1549249495 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:00.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709b7fe640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c15a920 0x7f709c070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 -- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c15a920 msgr2=0x7f709c070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709a7fc640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f709c0b1ce0 0x7f709c06fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c15a920 0x7f709c070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 -- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f709c0b1ce0 msgr2=0x7f709c06fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f709c0b1ce0 0x7f709c06fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709affd640 1 -- 172.21.15.70:0/1549249495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c009460 con 0x7f709c0b0320 2023-08-25T09:53:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709b7fe640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c15a920 0x7f709c070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.533+0000 7f709a7fc640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f709c0b1ce0 0x7f709c06fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.534+0000 7f709affd640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c157020 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f708c000c00 tx=0x7f708c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.539 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.534+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c031830 con 0x7f709c0b0320 2023-08-25T09:53:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.534+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c03f070 con 0x7f709c0b0320 2023-08-25T09:53:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.534+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c031bd0 con 0x7f709c0b0320 2023-08-25T09:53:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.534+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c06f700 con 0x7f709c0b0320 2023-08-25T09:53:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.534+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c0c4e80 con 0x7f709c0b0320 2023-08-25T09:53:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.535+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c06fc20 con 0x7f709c0b0320 2023-08-25T09:53:00.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.536+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f708c041030 con 0x7f709c0b0320 2023-08-25T09:53:00.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.537+0000 7f70837fe640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f708806a050 0x7f708806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.541 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.537+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f708c0aa860 con 0x7f709c0b0320 2023-08-25T09:53:00.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.538+0000 7f709a7fc640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f708806a050 0x7f708806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.538+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f70880721a0 con 0x7f709c0b0320 2023-08-25T09:53:00.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.538+0000 7f709a7fc640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f708806a050 0x7f708806c510 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f709c0c83f0 tx=0x7f7090008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.542 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.541+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f709c06fc20 con 0x7f709c0b0320 2023-08-25T09:53:00.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.735+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f709c06fc20 con 0x7f709c0b0320 2023-08-25T09:53:00.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.736+0000 7f70837fe640 1 -- 172.21.15.70:0/1549249495 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v202) v1 ==== 92+0+10 (secure 0 0 0) 0x7f709c06fc20 con 0x7f709c0b0320 2023-08-25T09:53:00.738 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 8 2023-08-25T09:53:00.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.738+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f708806a050 msgr2=0x7f708806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.738+0000 7f709bfff640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f708806a050 0x7f708806c510 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f709c0c83f0 tx=0x7f7090008040 comp rx=0 tx=0).stop 2023-08-25T09:53:00.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.738+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 msgr2=0x7f709c157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.738+0000 7f709bfff640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c157020 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f708c000c00 tx=0x7f708c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 shutdown_connections 2023-08-25T09:53:00.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f708806a050 0x7f708806c510 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f709c15a920 0x7f709c070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f709c0b1ce0 0x7f709c06fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.741 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 --2- 172.21.15.70:0/1549249495 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f709c0b0320 0x7f709c157020 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 >> 172.21.15.70:0/1549249495 conn(0x7f709c0ce830 msgr2=0x7f709c0aafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:00.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 shutdown_connections 2023-08-25T09:53:00.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.739+0000 7f709bfff640 1 -- 172.21.15.70:0/1549249495 wait complete. 2023-08-25T09:53:00.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T09:53:00.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.976+0000 7f079e1f3640 1 Processor -- start 2023-08-25T09:53:00.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.977+0000 7f079e1f3640 1 -- start start 2023-08-25T09:53:00.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.978+0000 7f079e1f3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07980cb280 0x7f07980706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.978+0000 7f079e1f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f079815af10 0x7f07980ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.978+0000 7f079e1f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0798070d20 con 0x7f07980b0960 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.978+0000 7f079e1f3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0798071000 con 0x7f07980cb280 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.978+0000 7f079e1f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07980712e0 con 0x7f079815af10 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07980cb280 0x7f07980706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f07980cb280 0x7f07980706f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60270/0 (socket says 172.21.15.70:60270) 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 -- 172.21.15.70:0/3770389583 learned_addr learned my addr 172.21.15.70:0/3770389583 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f0796ffd640 1 --2- 172.21.15.70:0/3770389583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f079815af10 0x7f07980ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 -- 172.21.15.70:0/3770389583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f079815af10 msgr2=0x7f07980ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:00.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 --2- 172.21.15.70:0/3770389583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f079815af10 0x7f07980ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 -- 172.21.15.70:0/3770389583 >> v1:172.21.15.155:6789/0 conn(0x7f07980b0960 legacy=0x7f07980cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:00.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.979+0000 7f07977fe640 1 -- 172.21.15.70:0/3770389583 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07980715c0 con 0x7f07980cb280 2023-08-25T09:53:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.980+0000 7f07977fe640 1 --2- 172.21.15.70:0/3770389583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07980cb280 0x7f07980706f0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f0788009e50 tx=0x7f078802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d236dc80d6bb500 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f07967fc640 1 -- 172.21.15.70:0/3770389583 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078802ca70 con 0x7f07980cb280 2023-08-25T09:53:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f07967fc640 1 -- 172.21.15.70:0/3770389583 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078802cbd0 con 0x7f07980cb280 2023-08-25T09:53:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f07967fc640 1 -- 172.21.15.70:0/3770389583 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880343f0 con 0x7f07980cb280 2023-08-25T09:53:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f079e1f3640 1 -- 172.21.15.70:0/3770389583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07980cb280 msgr2=0x7f07980706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f079e1f3640 1 --2- 172.21.15.70:0/3770389583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07980cb280 0x7f07980706f0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f0788009e50 tx=0x7f078802bed0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f079e1f3640 1 -- 172.21.15.70:0/3770389583 shutdown_connections 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f079e1f3640 1 --2- 172.21.15.70:0/3770389583 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f079815af10 0x7f07980ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f079e1f3640 1 --2- 172.21.15.70:0/3770389583 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f07980cb280 0x7f07980706f0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.981+0000 7f079e1f3640 1 -- 172.21.15.70:0/3770389583 >> 172.21.15.70:0/3770389583 conn(0x7f07980cb960 msgr2=0x7f07980cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.982+0000 7f079e1f3640 1 -- 172.21.15.70:0/3770389583 shutdown_connections 2023-08-25T09:53:00.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.982+0000 7f079e1f3640 1 -- 172.21.15.70:0/3770389583 wait complete. 2023-08-25T09:53:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.983+0000 7f079e1f3640 1 Processor -- start 2023-08-25T09:53:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 -- start start 2023-08-25T09:53:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07980b0960 0x7f0798099b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07980cb280 0x7f079809a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 0x7f07981f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079815c950 con 0x7f079815af10 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07981589d0 con 0x7f07980cb280 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.984+0000 7f079e1f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079809b190 con 0x7f07980b0960 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 0x7f07981f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 0x7f07981f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60282/0 (socket says 172.21.15.70:60282) 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 -- 172.21.15.70:0/2903083145 learned_addr learned my addr 172.21.15.70:0/2903083145 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f07977fe640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07980b0960 0x7f0798099b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0796ffd640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07980cb280 0x7f079809a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 -- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07980b0960 msgr2=0x7f0798099b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07980b0960 0x7f0798099b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 -- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07980cb280 msgr2=0x7f079809a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07980cb280 0x7f079809a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0797fff640 1 -- 172.21.15.70:0/2903083145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0788009b30 con 0x7f079815af10 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.985+0000 7f0796ffd640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07980cb280 0x7f079809a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:00.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.986+0000 7f0797fff640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 0x7f07981f22d0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f079000b410 tx=0x7f0790009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.986+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790011830 con 0x7f079815af10 2023-08-25T09:53:00.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.986+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0790011990 con 0x7f079815af10 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.986+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0790011d40 con 0x7f079815af10 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.986+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079809a8f0 con 0x7f079815af10 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.986+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079809aeb0 con 0x7f079815af10 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.987+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079815cac0 con 0x7f079815af10 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.988+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f079001a400 con 0x7f079815af10 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.989+0000 7f078ffff640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f077806a080 0x7f077806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:00.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.989+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0790088610 con 0x7f079815af10 2023-08-25T09:53:00.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.989+0000 7f07977fe640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f077806a080 0x7f077806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:00.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.989+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f07780721d0 con 0x7f079815af10 2023-08-25T09:53:00.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.990+0000 7f07977fe640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f077806a080 0x7f077806c540 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0788035000 tx=0x7f0788030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:00.995 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:00.994+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f079815cac0 con 0x7f079815af10 2023-08-25T09:53:01.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.181+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f079815ca90 con 0x7f079815af10 2023-08-25T09:53:01.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.182+0000 7f078ffff640 1 -- 172.21.15.70:0/2903083145 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v202) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0790053cf0 con 0x7f079815af10 2023-08-25T09:53:01.185 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 32 2023-08-25T09:53:01.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f077806a080 msgr2=0x7f077806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f077806a080 0x7f077806c540 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0788035000 tx=0x7f0788030000 comp rx=0 tx=0).stop 2023-08-25T09:53:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 msgr2=0x7f07981f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 0x7f07981f22d0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f079000b410 tx=0x7f0790009f90 comp rx=0 tx=0).stop 2023-08-25T09:53:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 shutdown_connections 2023-08-25T09:53:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f077806a080 0x7f077806c540 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f079815af10 0x7f07981f22d0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f07980cb280 0x7f079809a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 --2- 172.21.15.70:0/2903083145 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f07980b0960 0x7f0798099b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 >> 172.21.15.70:0/2903083145 conn(0x7f07980cb960 msgr2=0x7f079806ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:01.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 shutdown_connections 2023-08-25T09:53:01.193 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.185+0000 7f079e1f3640 1 -- 172.21.15.70:0/2903083145 wait complete. 2023-08-25T09:53:01.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T09:53:01.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.422+0000 7fd1786cc640 1 Processor -- start 2023-08-25T09:53:01.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.422+0000 7fd1786cc640 1 -- start start 2023-08-25T09:53:01.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.424+0000 7fd1786cc640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fd170072820 0x7fd1700aec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.424+0000 7fd1786cc640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700b38c0 con 0x7fd17006bc10 2023-08-25T09:53:01.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.424+0000 7fd1786cc640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd170160620 con 0x7fd1700ad280 2023-08-25T09:53:01.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.424+0000 7fd1786cc640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700af180 con 0x7fd170072820 2023-08-25T09:53:01.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.424+0000 7fd176c69640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fd17006bc10 0x7fd1700b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43386/0 (socket says 172.21.15.70:43386) 2023-08-25T09:53:01.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.424+0000 7fd176c69640 1 -- 172.21.15.70:0/2224427493 learned_addr learned my addr 172.21.15.70:0/2224427493 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:01.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd176468640 1 --2- 172.21.15.70:0/2224427493 >> v2:172.21.15.155:3300/0 conn(0x7fd170072820 0x7fd1700aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515698360 0 0) 0x7fd1700b38c0 con 0x7fd17006bc10 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd158003520 con 0x7fd17006bc10 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840194789 0 0) 0x7fd158003520 con 0x7fd17006bc10 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 >> v2:172.21.15.155:3300/0 conn(0x7fd170072820 msgr2=0x7fd1700aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 --2- 172.21.15.70:0/2224427493 >> v2:172.21.15.155:3300/0 conn(0x7fd170072820 0x7fd1700aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 >> v1:172.21.15.155:6789/0 conn(0x7fd1700ad280 legacy=0x7fd170072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700b2cb0 con 0x7fd17006bc10 2023-08-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fd168001670 con 0x7fd17006bc10 2023-08-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 >> v1:172.21.15.70:6789/0 conn(0x7fd17006bc10 legacy=0x7fd1700b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd158003970 0x7fd158003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1580047c0 0x7fd158004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158005180 0x7fd15800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700af180 con 0x7fd158003970 2023-08-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15800c120 con 0x7fd1580047c0 2023-08-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.425+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15800c2e0 con 0x7fd158005180 2023-08-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176468640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd158003970 0x7fd158003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176468640 1 -- 172.21.15.70:0/2224427493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158005180 msgr2=0x7fd15800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:01.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176468640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158005180 0x7fd15800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176468640 1 -- 172.21.15.70:0/2224427493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1580047c0 msgr2=0x7fd158004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:01.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176468640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd1580047c0 0x7fd158004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176468640 1 -- 172.21.15.70:0/2224427493 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158003520 con 0x7fd158003970 2023-08-25T09:53:01.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.426+0000 7fd176c69640 1 -- 172.21.15.70:0/2224427493 reap_dead start 2023-08-25T09:53:01.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.427+0000 7fd176468640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd158003970 0x7fd158003d90 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fc0 tx=0x7fd1580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d52b68e6d85e0b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.427+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16002cc20 con 0x7fd158003970 2023-08-25T09:53:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.427+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16000f040 con 0x7fd158003970 2023-08-25T09:53:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.427+0000 7fd175466640 1 -- 172.21.15.70:0/2224427493 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160034440 con 0x7fd158003970 2023-08-25T09:53:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 -- 172.21.15.70:0/2224427493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd158003970 msgr2=0x7fd158003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd158003970 0x7fd158003d90 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fc0 tx=0x7fd1580057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 -- 172.21.15.70:0/2224427493 shutdown_connections 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 --2- 172.21.15.70:0/2224427493 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd158003970 0x7fd158003d90 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 -- 172.21.15.70:0/2224427493 >> 172.21.15.70:0/2224427493 conn(0x7fd1700cfc70 msgr2=0x7fd1700d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 -- 172.21.15.70:0/2224427493 shutdown_connections 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.428+0000 7fd1786cc640 1 -- 172.21.15.70:0/2224427493 wait complete. 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.430+0000 7fd1786cc640 1 Processor -- start 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.430+0000 7fd1786cc640 1 -- start start 2023-08-25T09:53:01.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd1786cc640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158003970 0x7fd170153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd1786cc640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 0x7fd17015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd1786cc640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd170155ce0 0x7fd170156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd1786cc640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701537d0 con 0x7fd1701552a0 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd1786cc640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701566a0 con 0x7fd170155ce0 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd1786cc640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd170154e10 con 0x7fd158003970 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd175c67640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 0x7fd17015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd175c67640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 0x7fd17015a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60312/0 (socket says 172.21.15.70:60312) 2023-08-25T09:53:01.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd175c67640 1 -- 172.21.15.70:0/3055587540 learned_addr learned my addr 172.21.15.70:0/3055587540 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd176c69640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd170155ce0 0x7fd170156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.431+0000 7fd176468640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158003970 0x7fd170153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd175c67640 1 -- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158003970 msgr2=0x7fd170153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd175c67640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158003970 0x7fd170153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd175c67640 1 -- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd170155ce0 msgr2=0x7fd170156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd175c67640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd170155ce0 0x7fd170156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd175c67640 1 -- 172.21.15.70:0/3055587540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170154200 con 0x7fd1701552a0 2023-08-25T09:53:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd175c67640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 0x7fd17015a050 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd16400c530 tx=0x7fd16400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640052d0 con 0x7fd1701552a0 2023-08-25T09:53:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164005430 con 0x7fd1701552a0 2023-08-25T09:53:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164012870 con 0x7fd1701552a0 2023-08-25T09:53:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1701544e0 con 0x7fd1701552a0 2023-08-25T09:53:01.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.432+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd170154aa0 con 0x7fd1701552a0 2023-08-25T09:53:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.433+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170154c60 con 0x7fd1701552a0 2023-08-25T09:53:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.435+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd1640129d0 con 0x7fd1701552a0 2023-08-25T09:53:01.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.435+0000 7fd16effd640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd15c06a080 0x7fd15c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.436+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd164088160 con 0x7fd1701552a0 2023-08-25T09:53:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.436+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7fd15c0721d0 con 0x7fd1701552a0 2023-08-25T09:53:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.436+0000 7fd176468640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd15c06a080 0x7fd15c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.440 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.436+0000 7fd176468640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd15c06a080 0x7fd15c06c540 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd160036000 tx=0x7fd16002e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:01.441 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.440+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd1640537b0 con 0x7fd1701552a0 2023-08-25T09:53:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.627+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd170066fb0 con 0x7fd1701552a0 2023-08-25T09:53:01.630 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.628+0000 7fd16effd640 1 -- 172.21.15.70:0/3055587540 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v202) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd164056dd0 con 0x7fd1701552a0 2023-08-25T09:53:01.630 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 64 2023-08-25T09:53:01.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.630+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd15c06a080 msgr2=0x7fd15c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.630+0000 7fd1786cc640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd15c06a080 0x7fd15c06c540 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd160036000 tx=0x7fd16002e000 comp rx=0 tx=0).stop 2023-08-25T09:53:01.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.630+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 msgr2=0x7fd17015a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.630+0000 7fd1786cc640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 0x7fd17015a050 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7fd16400c530 tx=0x7fd16400c9f0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 shutdown_connections 2023-08-25T09:53:01.634 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd15c06a080 0x7fd15c06c540 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd170155ce0 0x7fd170156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd1701552a0 0x7fd17015a050 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 --2- 172.21.15.70:0/3055587540 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd158003970 0x7fd170153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 >> 172.21.15.70:0/3055587540 conn(0x7fd1700cfc70 msgr2=0x7fd1700af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 shutdown_connections 2023-08-25T09:53:01.635 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.631+0000 7fd1786cc640 1 -- 172.21.15.70:0/3055587540 wait complete. 2023-08-25T09:53:01.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:53:01.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.864+0000 7f4b5e4d3640 1 Processor -- start 2023-08-25T09:53:01.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.865+0000 7f4b5e4d3640 1 -- start start 2023-08-25T09:53:01.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.866+0000 7f4b5e4d3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4b5815e830 0x7f4b5806ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.866+0000 7f4b5e4d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5815ec10 0x7f4b580b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.866+0000 7f4b5e4d3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b580b1b60 con 0x7f4b5806b2b0 2023-08-25T09:53:01.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.866+0000 7f4b5e4d3640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5806bd80 con 0x7f4b5815e830 2023-08-25T09:53:01.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.866+0000 7f4b5e4d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5806b7c0 con 0x7f4b5815ec10 2023-08-25T09:53:01.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4b5815e830 0x7f4b5806ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f4b5815e830 0x7f4b5806ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60324/0 (socket says 172.21.15.70:60324) 2023-08-25T09:53:01.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 -- 172.21.15.70:0/3866892185 learned_addr learned my addr 172.21.15.70:0/3866892185 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:01.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 -- 172.21.15.70:0/3866892185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5815ec10 msgr2=0x7f4b580b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:01.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 --2- 172.21.15.70:0/3866892185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5815ec10 0x7f4b580b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 -- 172.21.15.70:0/3866892185 >> v1:172.21.15.155:6789/0 conn(0x7f4b5806b2b0 legacy=0x7f4b5815c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:01.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.867+0000 7f4b57fff640 1 -- 172.21.15.70:0/3866892185 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5806baa0 con 0x7f4b5815e830 2023-08-25T09:53:01.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.868+0000 7f4b57fff640 1 --2- 172.21.15.70:0/3866892185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815e830 0x7f4b5806ad70 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009e50 tx=0x7f4b4802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beefc31362242f2e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:01.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.868+0000 7f4b56ffd640 1 -- 172.21.15.70:0/3866892185 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4802ca70 con 0x7f4b5815e830 2023-08-25T09:53:01.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.868+0000 7f4b56ffd640 1 -- 172.21.15.70:0/3866892185 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4802cbd0 con 0x7f4b5815e830 2023-08-25T09:53:01.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.868+0000 7f4b56ffd640 1 -- 172.21.15.70:0/3866892185 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b480343f0 con 0x7f4b5815e830 2023-08-25T09:53:01.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/3866892185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815e830 msgr2=0x7f4b5806ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/3866892185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815e830 0x7f4b5806ad70 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009e50 tx=0x7f4b4802bed0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/3866892185 shutdown_connections 2023-08-25T09:53:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/3866892185 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5815ec10 0x7f4b580b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/3866892185 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815e830 0x7f4b5806ad70 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/3866892185 >> 172.21.15.70:0/3866892185 conn(0x7f4b58069850 msgr2=0x7f4b58069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:01.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.869+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/3866892185 shutdown_connections 2023-08-25T09:53:01.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.870+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/3866892185 wait complete. 2023-08-25T09:53:01.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.871+0000 7f4b5e4d3640 1 Processor -- start 2023-08-25T09:53:01.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.871+0000 7f4b5e4d3640 1 -- start start 2023-08-25T09:53:01.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.872+0000 7f4b5e4d3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5806b2b0 0x7f4b581f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.872+0000 7f4b5e4d3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b5815e830 0x7f4b581e4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.872+0000 7f4b5e4d3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 0x7f4b580cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.872+0000 7f4b5e4d3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581e5360 con 0x7f4b5815ec10 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.872+0000 7f4b5e4d3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580cc8d0 con 0x7f4b5815e830 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.872+0000 7f4b5e4d3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580ccbb0 con 0x7f4b5806b2b0 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 0x7f4b580cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 0x7f4b580cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60340/0 (socket says 172.21.15.70:60340) 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 -- 172.21.15.70:0/375498419 learned_addr learned my addr 172.21.15.70:0/375498419 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:01.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b57fff640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5806b2b0 0x7f4b581f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 -- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5806b2b0 msgr2=0x7f4b581f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5806b2b0 0x7f4b581f6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 -- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b5815e830 msgr2=0x7f4b581e4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b5815e830 0x7f4b581e4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 -- 172.21.15.70:0/375498419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b48009b30 con 0x7f4b5815ec10 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b5ca70640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 0x7f4b580cd970 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b410 tx=0x7f4b5000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:01.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.873+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50011830 con 0x7f4b5815ec10 2023-08-25T09:53:01.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.874+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b500119b0 con 0x7f4b5815ec10 2023-08-25T09:53:01.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.874+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50011d60 con 0x7f4b5815ec10 2023-08-25T09:53:01.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.874+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5815ca20 con 0x7f4b5815ec10 2023-08-25T09:53:01.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.874+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b580cd170 con 0x7f4b5815ec10 2023-08-25T09:53:01.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.875+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5804e420 con 0x7f4b5815ec10 2023-08-25T09:53:01.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.876+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4b5001a400 con 0x7f4b5815ec10 2023-08-25T09:53:01.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.877+0000 7f4b54ff9640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4b3806a080 0x7f4b3806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:01.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.877+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b50088760 con 0x7f4b5815ec10 2023-08-25T09:53:01.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.877+0000 7f4b57fff640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4b3806a080 0x7f4b3806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:01.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.877+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f4b380721d0 con 0x7f4b5815ec10 2023-08-25T09:53:01.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.878+0000 7f4b57fff640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4b3806a080 0x7f4b3806c540 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4b48035000 tx=0x7f4b48030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:01.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:01.881+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4b50053db0 con 0x7f4b5815ec10 2023-08-25T09:53:02.107 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.106+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b581e5590 con 0x7f4b5815ec10 2023-08-25T09:53:02.109 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.107+0000 7f4b54ff9640 1 -- 172.21.15.70:0/375498419 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f4b50015070 con 0x7f4b5815ec10 2023-08-25T09:53:02.109 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:02.109 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:53:02.110 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:53:02.111 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4b3806a080 msgr2=0x7f4b3806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4b3806a080 0x7f4b3806c540 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4b48035000 tx=0x7f4b48030000 comp rx=0 tx=0).stop 2023-08-25T09:53:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 msgr2=0x7f4b580cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 0x7f4b580cd970 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000b410 tx=0x7f4b5000b8d0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 shutdown_connections 2023-08-25T09:53:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f4b3806a080 0x7f4b3806c540 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.112 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f4b5815ec10 0x7f4b580cd970 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4b5815e830 0x7f4b581e4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 --2- 172.21.15.70:0/375498419 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4b5806b2b0 0x7f4b581f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 >> 172.21.15.70:0/375498419 conn(0x7f4b58069850 msgr2=0x7f4b580b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 shutdown_connections 2023-08-25T09:53:02.113 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.110+0000 7f4b5e4d3640 1 -- 172.21.15.70:0/375498419 wait complete. 2023-08-25T09:53:02.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.335+0000 7fa317d47640 1 Processor -- start 2023-08-25T09:53:02.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.336+0000 7fa317d47640 1 -- start start 2023-08-25T09:53:02.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa317d47640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa31015cb40 0x7fa3100ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa317d47640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3101615b0 con 0x7fa3100b2b40 2023-08-25T09:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa317d47640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31006f050 con 0x7fa3100ac870 2023-08-25T09:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa317d47640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31006f330 con 0x7fa31015cb40 2023-08-25T09:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa315ae3640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fa31015cb40 0x7fa3100ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa3162e4640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa3100b2b40 0x7fa310160ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43402/0 (socket says 172.21.15.70:43402) 2023-08-25T09:53:02.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.337+0000 7fa3162e4640 1 -- 172.21.15.70:0/1303647798 learned_addr learned my addr 172.21.15.70:0/1303647798 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:02.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.338+0000 7fa315ae3640 1 -- 172.21.15.70:0/1303647798 >> v1:172.21.15.155:6789/0 conn(0x7fa3100ac870 legacy=0x7fa310160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.338+0000 7fa315ae3640 1 -- 172.21.15.70:0/1303647798 >> v1:172.21.15.70:6789/0 conn(0x7fa3100b2b40 legacy=0x7fa310160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.338+0000 7fa315ae3640 1 -- 172.21.15.70:0/1303647798 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31006f610 con 0x7fa31015cb40 2023-08-25T09:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.338+0000 7fa314ae1640 1 -- 172.21.15.70:0/1303647798 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2820879678 0 0) 0x7fa3101615b0 con 0x7fa3100b2b40 2023-08-25T09:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.339+0000 7fa315ae3640 1 --2- 172.21.15.70:0/1303647798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31015cb40 0x7fa3100ce320 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fa300009d90 tx=0x7fa30002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa63c6116f360888 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.339+0000 7fa314ae1640 1 -- 172.21.15.70:0/1303647798 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30002cc60 con 0x7fa31015cb40 2023-08-25T09:53:02.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.339+0000 7fa314ae1640 1 -- 172.21.15.70:0/1303647798 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30002cdc0 con 0x7fa31015cb40 2023-08-25T09:53:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.339+0000 7fa314ae1640 1 -- 172.21.15.70:0/1303647798 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300034660 con 0x7fa31015cb40 2023-08-25T09:53:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 -- 172.21.15.70:0/1303647798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31015cb40 msgr2=0x7fa3100ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 --2- 172.21.15.70:0/1303647798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31015cb40 0x7fa3100ce320 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7fa300009d90 tx=0x7fa30002beb0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 -- 172.21.15.70:0/1303647798 shutdown_connections 2023-08-25T09:53:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 --2- 172.21.15.70:0/1303647798 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa31015cb40 0x7fa3100ce320 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 -- 172.21.15.70:0/1303647798 >> 172.21.15.70:0/1303647798 conn(0x7fa310069850 msgr2=0x7fa310069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 -- 172.21.15.70:0/1303647798 shutdown_connections 2023-08-25T09:53:02.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.340+0000 7fa317d47640 1 -- 172.21.15.70:0/1303647798 wait complete. 2023-08-25T09:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.342+0000 7fa317d47640 1 Processor -- start 2023-08-25T09:53:02.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.342+0000 7fa317d47640 1 -- start start 2023-08-25T09:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa317d47640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3100ac870 0x7fa31015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa317d47640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 0x7fa310140430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa317d47640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa31015cb40 0x7fa31015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa317d47640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31015ed20 con 0x7fa3100b2b40 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa317d47640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31015fb60 con 0x7fa3100ac870 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa317d47640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31015ef50 con 0x7fa31015cb40 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa3152e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 0x7fa310140430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa3152e2640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 0x7fa310140430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60360/0 (socket says 172.21.15.70:60360) 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.343+0000 7fa3152e2640 1 -- 172.21.15.70:0/1508208262 learned_addr learned my addr 172.21.15.70:0/1508208262 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa315ae3640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3100ac870 0x7fa31015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3162e4640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa31015cb40 0x7fa31015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3152e2640 1 -- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa31015cb40 msgr2=0x7fa31015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3152e2640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa31015cb40 0x7fa31015d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3152e2640 1 -- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3100ac870 msgr2=0x7fa31015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3152e2640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3100ac870 0x7fa31015fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3152e2640 1 -- 172.21.15.70:0/1508208262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300009a70 con 0x7fa3100b2b40 2023-08-25T09:53:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa3152e2640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 0x7fa310140430 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b3e0 tx=0x7fa30400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304011830 con 0x7fa3100b2b40 2023-08-25T09:53:02.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.344+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3040119b0 con 0x7fa3100b2b40 2023-08-25T09:53:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.345+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31015f230 con 0x7fa3100b2b40 2023-08-25T09:53:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.345+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304011d60 con 0x7fa3100b2b40 2023-08-25T09:53:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.345+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31015f7f0 con 0x7fa3100b2b40 2023-08-25T09:53:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.346+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31004e420 con 0x7fa3100b2b40 2023-08-25T09:53:02.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.347+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa30401a400 con 0x7fa3100b2b40 2023-08-25T09:53:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.347+0000 7fa30e7fc640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa2fc06a080 0x7fa2fc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.348+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa304088780 con 0x7fa3100b2b40 2023-08-25T09:53:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.348+0000 7fa315ae3640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa2fc06a080 0x7fa2fc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.348+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7fa2fc0721d0 con 0x7fa3100b2b40 2023-08-25T09:53:02.351 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.348+0000 7fa315ae3640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa2fc06a080 0x7fa2fc06c540 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa300036000 tx=0x7fa300030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:02.353 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.352+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa304053dd0 con 0x7fa3100b2b40 2023-08-25T09:53:02.544 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.543+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa31006fba0 con 0x7fa3100b2b40 2023-08-25T09:53:02.547 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.546+0000 7fa30e7fc640 1 -- 172.21.15.70:0/1508208262 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+18339 (secure 0 0 0) 0x7fa304015070 con 0x7fa3100b2b40 2023-08-25T09:53:02.547 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:02.548 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":202,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:57.522412+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:53:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.548+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa2fc06a080 msgr2=0x7fa2fc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.548+0000 7fa317d47640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa2fc06a080 0x7fa2fc06c540 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa300036000 tx=0x7fa300030000 comp rx=0 tx=0).stop 2023-08-25T09:53:02.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.548+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 msgr2=0x7fa310140430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.548+0000 7fa317d47640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 0x7fa310140430 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fa30400b3e0 tx=0x7fa30400b8a0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 shutdown_connections 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa2fc06a080 0x7fa2fc06c540 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa31015cb40 0x7fa31015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa3100b2b40 0x7fa310140430 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 --2- 172.21.15.70:0/1508208262 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa3100ac870 0x7fa31015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 >> 172.21.15.70:0/1508208262 conn(0x7fa310069850 msgr2=0x7fa31006c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 shutdown_connections 2023-08-25T09:53:02.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.549+0000 7fa317d47640 1 -- 172.21.15.70:0/1508208262 wait complete. 2023-08-25T09:53:02.568 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:53:02.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:53:02.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.775+0000 7f2c84ec5640 1 Processor -- start 2023-08-25T09:53:02.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.775+0000 7f2c84ec5640 1 -- start start 2023-08-25T09:53:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.776+0000 7f2c84ec5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 0x7f2c8006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.776+0000 7f2c84ec5640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f2c8015ec10 0x7f2c800b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c84ec5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c800b1b60 con 0x7f2c8006b2b0 2023-08-25T09:53:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c84ec5640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8006bd80 con 0x7f2c8015ec10 2023-08-25T09:53:02.778 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c84ec5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8006b7c0 con 0x7f2c8015e830 2023-08-25T09:53:02.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c7e59c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 0x7f2c8006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c7e59c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 0x7f2c8006ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:43522/0 (socket says 172.21.15.70:43522) 2023-08-25T09:53:02.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c7e59c640 1 -- 172.21.15.70:0/3709454867 learned_addr learned my addr 172.21.15.70:0/3709454867 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:02.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.777+0000 7f2c7ed9d640 1 --2- 172.21.15.70:0/3709454867 >> v2:172.21.15.155:3300/0 conn(0x7f2c8015ec10 0x7f2c800b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.778+0000 7f2c7e59c640 1 -- 172.21.15.70:0/3709454867 >> v2:172.21.15.155:3300/0 conn(0x7f2c8015ec10 msgr2=0x7f2c800b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:02.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.778+0000 7f2c7e59c640 1 --2- 172.21.15.70:0/3709454867 >> v2:172.21.15.155:3300/0 conn(0x7f2c8015ec10 0x7f2c800b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.778+0000 7f2c7e59c640 1 -- 172.21.15.70:0/3709454867 >> v1:172.21.15.155:6789/0 conn(0x7f2c8006b2b0 legacy=0x7f2c8015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.778+0000 7f2c7e59c640 1 -- 172.21.15.70:0/3709454867 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8006baa0 con 0x7f2c8015e830 2023-08-25T09:53:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.778+0000 7f2c7ed9d640 1 --2- 172.21.15.70:0/3709454867 >> v2:172.21.15.155:3300/0 conn(0x7f2c8015ec10 0x7f2c800b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:53:02.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.780+0000 7f2c7e59c640 1 --2- 172.21.15.70:0/3709454867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 0x7f2c8006ad70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138906f4c42cffd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.780+0000 7f2c7dd9b640 1 -- 172.21.15.70:0/3709454867 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70030840 con 0x7f2c8015e830 2023-08-25T09:53:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.780+0000 7f2c7dd9b640 1 -- 172.21.15.70:0/3709454867 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c700309a0 con 0x7f2c8015e830 2023-08-25T09:53:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.780+0000 7f2c7dd9b640 1 -- 172.21.15.70:0/3709454867 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70030d30 con 0x7f2c8015e830 2023-08-25T09:53:02.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 -- 172.21.15.70:0/3709454867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 msgr2=0x7f2c8006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/3709454867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 0x7f2c8006ad70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c7002b980 comp rx=0 tx=0).stop 2023-08-25T09:53:02.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 -- 172.21.15.70:0/3709454867 shutdown_connections 2023-08-25T09:53:02.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/3709454867 >> v2:172.21.15.155:3300/0 conn(0x7f2c8015ec10 0x7f2c800b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/3709454867 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015e830 0x7f2c8006ad70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 -- 172.21.15.70:0/3709454867 >> 172.21.15.70:0/3709454867 conn(0x7f2c80069850 msgr2=0x7f2c80069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:02.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 -- 172.21.15.70:0/3709454867 shutdown_connections 2023-08-25T09:53:02.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.781+0000 7f2c84ec5640 1 -- 172.21.15.70:0/3709454867 wait complete. 2023-08-25T09:53:02.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.783+0000 7f2c84ec5640 1 Processor -- start 2023-08-25T09:53:02.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.783+0000 7f2c84ec5640 1 -- start start 2023-08-25T09:53:02.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c84ec5640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8006b2b0 0x7f2c800c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c84ec5640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 0x7f2c8014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c84ec5640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015ec10 0x7f2c800c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c84ec5640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801e6c10 con 0x7f2c8015e830 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c84ec5640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801e6ef0 con 0x7f2c8006b2b0 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c84ec5640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c801e41d0 con 0x7f2c8015ec10 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 0x7f2c8014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c77fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 0x7f2c8014b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60374/0 (socket says 172.21.15.70:60374) 2023-08-25T09:53:02.786 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.784+0000 7f2c77fff640 1 -- 172.21.15.70:0/1365392845 learned_addr learned my addr 172.21.15.70:0/1365392845 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:02.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c7ed9d640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015ec10 0x7f2c800c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c7e59c640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8006b2b0 0x7f2c800c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c77fff640 1 -- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015ec10 msgr2=0x7f2c800c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c77fff640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015ec10 0x7f2c800c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c77fff640 1 -- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8006b2b0 msgr2=0x7f2c800c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c77fff640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8006b2b0 0x7f2c800c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c77fff640 1 -- 172.21.15.70:0/1365392845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009a70 con 0x7f2c8015e830 2023-08-25T09:53:02.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c77fff640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 0x7f2c8014b0e0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c440 tx=0x7f2c68001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:02.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c680052d0 con 0x7f2c8015e830 2023-08-25T09:53:02.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68005430 con 0x7f2c8015e830 2023-08-25T09:53:02.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c700094d0 con 0x7f2c8015e830 2023-08-25T09:53:02.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.785+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68007260 con 0x7f2c8015e830 2023-08-25T09:53:02.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.786+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8015ca20 con 0x7f2c8015e830 2023-08-25T09:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.787+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8004e420 con 0x7f2c8015e830 2023-08-25T09:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.788+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2c680073c0 con 0x7f2c8015e830 2023-08-25T09:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.788+0000 7f2c76ffd640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6006a050 0x7f2c6006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.789+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2c680879a0 con 0x7f2c8015e830 2023-08-25T09:53:02.790 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.789+0000 7f2c7e59c640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6006a050 0x7f2c6006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:02.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.789+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f2c600721a0 con 0x7f2c8015e830 2023-08-25T09:53:02.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.789+0000 7f2c7e59c640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6006a050 0x7f2c6006c510 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2c70036000 tx=0x7f2c7003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:02.794 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:02.793+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2c68053080 con 0x7f2c8015e830 2023-08-25T09:53:03.023 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.022+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c8006c620 con 0x7f2c8015e830 2023-08-25T09:53:03.024 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.023+0000 7f2c76ffd640 1 -- 172.21.15.70:0/1365392845 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7f2c680566a0 con 0x7f2c8015e830 2023-08-25T09:53:03.025 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:03.025 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:53:03.025 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:53:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6006a050 msgr2=0x7f2c6006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6006a050 0x7f2c6006c510 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2c70036000 tx=0x7f2c7003b040 comp rx=0 tx=0).stop 2023-08-25T09:53:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 msgr2=0x7f2c8014b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 0x7f2c8014b0e0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c440 tx=0x7f2c68001e70 comp rx=0 tx=0).stop 2023-08-25T09:53:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 shutdown_connections 2023-08-25T09:53:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f2c6006a050 0x7f2c6006c510 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.028 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f2c8015ec10 0x7f2c800c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f2c8015e830 0x7f2c8014b0e0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 --2- 172.21.15.70:0/1365392845 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2c8006b2b0 0x7f2c800c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 >> 172.21.15.70:0/1365392845 conn(0x7f2c80069850 msgr2=0x7f2c800b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:03.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 shutdown_connections 2023-08-25T09:53:03.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.026+0000 7f2c84ec5640 1 -- 172.21.15.70:0/1365392845 wait complete. 2023-08-25T09:53:03.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:53:03.255 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.254+0000 7fd48359e640 1 Processor -- start 2023-08-25T09:53:03.256 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.254+0000 7fd48359e640 1 -- start start 2023-08-25T09:53:03.257 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.255+0000 7fd48359e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 0x7fd4840b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.255+0000 7fd48359e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4840b12f0 0x7fd48415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd48359e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48415e7c0 con 0x7fd48415f190 2023-08-25T09:53:03.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd48359e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd484071b50 con 0x7fd4840b12f0 2023-08-25T09:53:03.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd48359e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48415e490 con 0x7fd48415a7c0 2023-08-25T09:53:03.258 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4840b12f0 0x7fd48415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fd4840b12f0 0x7fd48415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60380/0 (socket says 172.21.15.70:60380) 2023-08-25T09:53:03.259 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 -- 172.21.15.70:0/1581961431 learned_addr learned my addr 172.21.15.70:0/1581961431 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:03.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 -- 172.21.15.70:0/1581961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 msgr2=0x7fd4840b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:03.260 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 --2- 172.21.15.70:0/1581961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 0x7fd4840b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 -- 172.21.15.70:0/1581961431 >> v1:172.21.15.155:6789/0 conn(0x7fd48415f190 legacy=0x7fd4840cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.256+0000 7fd481d9b640 1 -- 172.21.15.70:0/1581961431 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840b3aa0 con 0x7fd4840b12f0 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.257+0000 7fd481d9b640 1 --2- 172.21.15.70:0/1581961431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4840b12f0 0x7fd48415ec50 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e50 tx=0x7fd47802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5837fd0226d1917 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48159a640 1 -- 172.21.15.70:0/1581961431 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47802ca80 con 0x7fd4840b12f0 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48159a640 1 -- 172.21.15.70:0/1581961431 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47802cbe0 con 0x7fd4840b12f0 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48159a640 1 -- 172.21.15.70:0/1581961431 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4780343f0 con 0x7fd4840b12f0 2023-08-25T09:53:03.261 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 -- 172.21.15.70:0/1581961431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4840b12f0 msgr2=0x7fd48415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 --2- 172.21.15.70:0/1581961431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4840b12f0 0x7fd48415ec50 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7fd478009e50 tx=0x7fd47802bee0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 -- 172.21.15.70:0/1581961431 shutdown_connections 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 --2- 172.21.15.70:0/1581961431 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd4840b12f0 0x7fd48415ec50 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 --2- 172.21.15.70:0/1581961431 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 0x7fd4840b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 -- 172.21.15.70:0/1581961431 >> 172.21.15.70:0/1581961431 conn(0x7fd4840affd0 msgr2=0x7fd4840b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 -- 172.21.15.70:0/1581961431 shutdown_connections 2023-08-25T09:53:03.262 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.258+0000 7fd48359e640 1 -- 172.21.15.70:0/1581961431 wait complete. 2023-08-25T09:53:03.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.260+0000 7fd48359e640 1 Processor -- start 2023-08-25T09:53:03.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.260+0000 7fd48359e640 1 -- start start 2023-08-25T09:53:03.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd48359e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4840b12f0 0x7fd4841587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd48359e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 0x7fd48415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd48359e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 0x7fd4840c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd48359e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840c2e80 con 0x7fd48415f190 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd48359e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840c1fd0 con 0x7fd4840b12f0 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd48359e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd484158ce0 con 0x7fd48415a7c0 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.261+0000 7fd482d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 0x7fd4840c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 0x7fd4840c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60386/0 (socket says 172.21.15.70:60386) 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 -- 172.21.15.70:0/3085485163 learned_addr learned my addr 172.21.15.70:0/3085485163 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 -- 172.21.15.70:0/3085485163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 msgr2=0x7fd48415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 0x7fd48415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 -- 172.21.15.70:0/3085485163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4840b12f0 msgr2=0x7fd4841587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4840b12f0 0x7fd4841587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 -- 172.21.15.70:0/3085485163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd478009b30 con 0x7fd48415f190 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd482d9d640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 0x7fd4840c7740 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fd47400c3e0 tx=0x7fd47400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474011790 con 0x7fd48415f190 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4740118f0 con 0x7fd48415f190 2023-08-25T09:53:03.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4840c2270 con 0x7fd48415f190 2023-08-25T09:53:03.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.262+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474011ce0 con 0x7fd48415f190 2023-08-25T09:53:03.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.263+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4840c2830 con 0x7fd48415f190 2023-08-25T09:53:03.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.264+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484066fb0 con 0x7fd48415f190 2023-08-25T09:53:03.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.265+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd47401c460 con 0x7fd48415f190 2023-08-25T09:53:03.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.266+0000 7fd472ffd640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd46406a080 0x7fd46406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.266+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4740884b0 con 0x7fd48415f190 2023-08-25T09:53:03.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.266+0000 7fd48259c640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd46406a080 0x7fd46406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.266+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7fd4640721d0 con 0x7fd48415f190 2023-08-25T09:53:03.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.267+0000 7fd48259c640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd46406a080 0x7fd46406c540 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009710 tx=0x7fd47c006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:03.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.270+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd474053b00 con 0x7fd48415f190 2023-08-25T09:53:03.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.495+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4840b1de0 con 0x7fd48415f190 2023-08-25T09:53:03.497 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.496+0000 7fd472ffd640 1 -- 172.21.15.70:0/3085485163 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4882 (secure 0 0 0) 0x7fd474057120 con 0x7fd48415f190 2023-08-25T09:53:03.498 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:03.498 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":91,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T09:52:04.089417+0000","modified":"2023-08-25T09:52:07.665370+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9573":{"gid":9573,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.70:6836/771430984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":771430984},{"type":"v1","addr":"172.21.15.70:6836","nonce":771430984}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-08-25T09:53:03.498 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 91 2023-08-25T09:53:03.500 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd46406a080 msgr2=0x7fd46406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd46406a080 0x7fd46406c540 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009710 tx=0x7fd47c006c90 comp rx=0 tx=0).stop 2023-08-25T09:53:03.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 msgr2=0x7fd4840c7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 0x7fd4840c7740 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fd47400c3e0 tx=0x7fd47400c8a0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 shutdown_connections 2023-08-25T09:53:03.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fd46406a080 0x7fd46406c540 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.501 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fd48415f190 0x7fd4840c7740 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fd48415a7c0 0x7fd48415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 --2- 172.21.15.70:0/3085485163 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fd4840b12f0 0x7fd4841587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 >> 172.21.15.70:0/3085485163 conn(0x7fd4840affd0 msgr2=0x7fd484070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:03.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 shutdown_connections 2023-08-25T09:53:03.502 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.499+0000 7fd48359e640 1 -- 172.21.15.70:0/3085485163 wait complete. 2023-08-25T09:53:03.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:53:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.731+0000 7f4160a1f640 1 Processor -- start 2023-08-25T09:53:03.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.731+0000 7f4160a1f640 1 -- start start 2023-08-25T09:53:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.733+0000 7f4160a1f640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f415c15e830 0x7f415c06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.733+0000 7f4160a1f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c0b1b60 con 0x7f415c06b2b0 2023-08-25T09:53:03.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.733+0000 7f4160a1f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c06b7c0 con 0x7f415c15ec10 2023-08-25T09:53:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.733+0000 7f4160a1f640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c06bd80 con 0x7f415c15e830 2023-08-25T09:53:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.733+0000 7f4159d9b640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f415c06b2b0 0x7f415c15c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43412/0 (socket says 172.21.15.70:43412) 2023-08-25T09:53:03.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.733+0000 7f4159d9b640 1 -- 172.21.15.70:0/4138100043 learned_addr learned my addr 172.21.15.70:0/4138100043 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:03.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88090993 0 0) 0x7f415c0b1b60 con 0x7f415c06b2b0 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4148003520 con 0x7f415c06b2b0 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415a59c640 1 --2- 172.21.15.70:0/4138100043 >> v2:172.21.15.155:3300/0 conn(0x7f415c15e830 0x7f415c06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202787418 0 0) 0x7f4148003520 con 0x7f415c06b2b0 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 >> v2:172.21.15.155:3300/0 conn(0x7f415c15e830 msgr2=0x7f415c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 --2- 172.21.15.70:0/4138100043 >> v2:172.21.15.155:3300/0 conn(0x7f415c15e830 0x7f415c06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 >> v1:172.21.15.155:6789/0 conn(0x7f415c15ec10 legacy=0x7f415c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.737 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c06baa0 con 0x7f415c06b2b0 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f4144001670 con 0x7f415c06b2b0 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 >> v1:172.21.15.70:6789/0 conn(0x7f415c06b2b0 legacy=0x7f415c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4148003970 0x7f4148003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f4148004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4148005180 0x7f414800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c06bd80 con 0x7f41480047c0 2023-08-25T09:53:03.738 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414800c0d0 con 0x7f4148005180 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f414800c290 con 0x7f4148003970 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.734+0000 7f4159d9b640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f4148004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f415a59c640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4148003970 0x7f4148003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f4159d9b640 1 -- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4148003970 msgr2=0x7f4148003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f4159d9b640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f4148003970 0x7f4148003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f4159d9b640 1 -- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4148005180 msgr2=0x7f414800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:03.739 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f4159d9b640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f4148005180 0x7f414800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f4159d9b640 1 -- 172.21.15.70:0/4138100043 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4148003520 con 0x7f41480047c0 2023-08-25T09:53:03.740 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.735+0000 7f415ad9d640 1 -- 172.21.15.70:0/4138100043 reap_dead start 2023-08-25T09:53:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.736+0000 7f4159d9b640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f4148004c40 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f414400c240 tx=0x7f41480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81deb87602c804de server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.736+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144005ad0 con 0x7f41480047c0 2023-08-25T09:53:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.736+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144005c30 con 0x7f41480047c0 2023-08-25T09:53:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.736+0000 7f415959a640 1 -- 172.21.15.70:0/4138100043 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414402caf0 con 0x7f41480047c0 2023-08-25T09:53:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 -- 172.21.15.70:0/4138100043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 msgr2=0x7f4148004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.742 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f4148004c40 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f414400c240 tx=0x7f41480057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 -- 172.21.15.70:0/4138100043 shutdown_connections 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 --2- 172.21.15.70:0/4138100043 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f4148004c40 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 -- 172.21.15.70:0/4138100043 >> 172.21.15.70:0/4138100043 conn(0x7f415c069850 msgr2=0x7f415c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 -- 172.21.15.70:0/4138100043 shutdown_connections 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.737+0000 7f4160a1f640 1 -- 172.21.15.70:0/4138100043 wait complete. 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.739+0000 7f4160a1f640 1 Processor -- start 2023-08-25T09:53:03.743 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.739+0000 7f4160a1f640 1 -- start start 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f4160a1f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f415c14e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f4160a1f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f415c157bb0 0x7f415c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f4160a1f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f415c0c6ea0 0x7f415c1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f4160a1f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c14e750 con 0x7f41480047c0 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f4160a1f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c158430 con 0x7f415c0c6ea0 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f4160a1f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0c58c0 con 0x7f415c157bb0 2023-08-25T09:53:03.744 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f415c14e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f415c14e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60406/0 (socket says 172.21.15.70:60406) 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 -- 172.21.15.70:0/2542296476 learned_addr learned my addr 172.21.15.70:0/2542296476 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 -- 172.21.15.70:0/2542296476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f415c157bb0 msgr2=0x7f415c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f415c157bb0 0x7f415c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 -- 172.21.15.70:0/2542296476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f415c0c6ea0 msgr2=0x7f415c1f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f415c0c6ea0 0x7f415c1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.745 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.740+0000 7f415a59c640 1 -- 172.21.15.70:0/2542296476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c0c5ba0 con 0x7f41480047c0 2023-08-25T09:53:03.746 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.741+0000 7f415a59c640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f415c14e210 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f414c00e540 tx=0x7f414c00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.741+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c013610 con 0x7f41480047c0 2023-08-25T09:53:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.741+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c013770 con 0x7f41480047c0 2023-08-25T09:53:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.741+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c013ae0 con 0x7f41480047c0 2023-08-25T09:53:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.741+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c0c5e80 con 0x7f41480047c0 2023-08-25T09:53:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.741+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c0c6440 con 0x7f41480047c0 2023-08-25T09:53:03.747 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.743+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c04e420 con 0x7f41480047c0 2023-08-25T09:53:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.743+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f414c013c40 con 0x7f41480047c0 2023-08-25T09:53:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.745+0000 7f4152ffd640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f413c06a080 0x7f413c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.745+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f414c088180 con 0x7f41480047c0 2023-08-25T09:53:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.745+0000 7f4159d9b640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f413c06a080 0x7f413c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.745+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f413c0721d0 con 0x7f41480047c0 2023-08-25T09:53:03.748 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.746+0000 7f4159d9b640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f413c06a080 0x7f413c06c540 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4144034000 tx=0x7f4144039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:03.750 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.749+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f414c053900 con 0x7f41480047c0 2023-08-25T09:53:03.941 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.939+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f415c06c620 con 0x7f41480047c0 2023-08-25T09:53:03.943 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.942+0000 7f4152ffd640 1 -- 172.21.15.70:0/2542296476 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+18339 (secure 0 0 0) 0x7f414c056f20 con 0x7f41480047c0 2023-08-25T09:53:03.944 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:03.945 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":202,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:52:57.522412+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-08-25T09:52:02.296119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-08-25T09:52:03.097163+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:53:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f413c06a080 msgr2=0x7f413c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f413c06a080 0x7f413c06c540 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4144034000 tx=0x7f4144039040 comp rx=0 tx=0).stop 2023-08-25T09:53:03.946 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 msgr2=0x7f415c14e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f415c14e210 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f414c00e540 tx=0x7f414c00ea00 comp rx=0 tx=0).stop 2023-08-25T09:53:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 shutdown_connections 2023-08-25T09:53:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f413c06a080 0x7f413c06c540 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.947 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f415c0c6ea0 0x7f415c1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f415c157bb0 0x7f415c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 --2- 172.21.15.70:0/2542296476 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f41480047c0 0x7f415c14e210 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 >> 172.21.15.70:0/2542296476 conn(0x7f415c069850 msgr2=0x7f415c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.945+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 shutdown_connections 2023-08-25T09:53:03.948 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:03.946+0000 7f4160a1f640 1 -- 172.21.15.70:0/2542296476 wait complete. 2023-08-25T09:53:03.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T09:53:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.176+0000 7f94affff640 1 Processor -- start 2023-08-25T09:53:04.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.177+0000 7f94affff640 1 -- start start 2023-08-25T09:53:04.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.178+0000 7f94affff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94b015cf00 0x7f94b0070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.178+0000 7f94affff640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b00b81d0 con 0x7f94b00cf210 2023-08-25T09:53:04.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.178+0000 7f94affff640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94b015d3c0 con 0x7f94b00b7df0 2023-08-25T09:53:04.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.178+0000 7f94affff640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94b01604f0 con 0x7f94b015cf00 2023-08-25T09:53:04.181 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94af7fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f94b00cf210 0x7f94b00cf660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43418/0 (socket says 172.21.15.70:43418) 2023-08-25T09:53:04.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94af7fe640 1 -- 172.21.15.70:0/3401784955 learned_addr learned my addr 172.21.15.70:0/3401784955 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:04.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94ae7fc640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f94b015cf00 0x7f94b0070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94ae7fc640 1 -- 172.21.15.70:0/3401784955 >> v1:172.21.15.155:6789/0 conn(0x7f94b00b7df0 legacy=0x7f94b015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94ae7fc640 1 -- 172.21.15.70:0/3401784955 >> v1:172.21.15.70:6789/0 conn(0x7f94b00cf210 legacy=0x7f94b00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.182 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94ae7fc640 1 -- 172.21.15.70:0/3401784955 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b015d6a0 con 0x7f94b015cf00 2023-08-25T09:53:04.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.179+0000 7f94adffb640 1 -- 172.21.15.70:0/3401784955 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783218057 0 0) 0x7f94b00b81d0 con 0x7f94b00cf210 2023-08-25T09:53:04.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.180+0000 7f94ae7fc640 1 --2- 172.21.15.70:0/3401784955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b0070880 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f94980097a0 tx=0x7f949802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b30d0abd6eb9fd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:04.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94adffb640 1 -- 172.21.15.70:0/3401784955 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498030610 con 0x7f94b015cf00 2023-08-25T09:53:04.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94adffb640 1 -- 172.21.15.70:0/3401784955 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498030770 con 0x7f94b015cf00 2023-08-25T09:53:04.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94adffb640 1 -- 172.21.15.70:0/3401784955 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498030b00 con 0x7f94b015cf00 2023-08-25T09:53:04.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 -- 172.21.15.70:0/3401784955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 msgr2=0x7f94b0070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 --2- 172.21.15.70:0/3401784955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b0070880 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f94980097a0 tx=0x7f949802b960 comp rx=0 tx=0).stop 2023-08-25T09:53:04.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 -- 172.21.15.70:0/3401784955 shutdown_connections 2023-08-25T09:53:04.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 --2- 172.21.15.70:0/3401784955 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b0070880 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 -- 172.21.15.70:0/3401784955 >> 172.21.15.70:0/3401784955 conn(0x7f94b00b43a0 msgr2=0x7f94b00b4780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:04.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 -- 172.21.15.70:0/3401784955 shutdown_connections 2023-08-25T09:53:04.185 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.181+0000 7f94affff640 1 -- 172.21.15.70:0/3401784955 wait complete. 2023-08-25T09:53:04.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.183+0000 7f94affff640 1 Processor -- start 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.183+0000 7f94affff640 1 -- start start 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.184+0000 7f94affff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94b00b7df0 0x7f94b0158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.184+0000 7f94affff640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b00cf210 0x7f94b00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.184+0000 7f94affff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b00b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.184+0000 7f94affff640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b00b3300 con 0x7f94b015cf00 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.184+0000 7f94affff640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b00b2680 con 0x7f94b00cf210 2023-08-25T09:53:04.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.184+0000 7f94affff640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b00b2960 con 0x7f94b00b7df0 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b00b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b00b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60436/0 (socket says 172.21.15.70:60436) 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 -- 172.21.15.70:0/4055095567 learned_addr learned my addr 172.21.15.70:0/4055095567 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 -- 172.21.15.70:0/4055095567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94b00b7df0 msgr2=0x7f94b0158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94b00b7df0 0x7f94b0158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 -- 172.21.15.70:0/4055095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b00cf210 msgr2=0x7f94b00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b00cf210 0x7f94b00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498009400 con 0x7f94b015cf00 2023-08-25T09:53:04.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.185+0000 7f94af7fe640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b00b3d10 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f94a800c450 tx=0x7f94a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:04.189 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.186+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80074e0 con 0x7f94b015cf00 2023-08-25T09:53:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.186+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94a8007660 con 0x7f94b015cf00 2023-08-25T09:53:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.186+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a8004a20 con 0x7f94b015cf00 2023-08-25T09:53:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.186+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b00b2c40 con 0x7f94b015cf00 2023-08-25T09:53:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.186+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b0159390 con 0x7f94b015cf00 2023-08-25T09:53:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.187+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b00b3440 con 0x7f94b015cf00 2023-08-25T09:53:04.190 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.188+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f94a8004b80 con 0x7f94b015cf00 2023-08-25T09:53:04.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.189+0000 7f94a77fe640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f949006a050 0x7f949006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.191 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.189+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94a80877a0 con 0x7f94b015cf00 2023-08-25T09:53:04.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.189+0000 7f94aeffd640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f949006a050 0x7f949006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.189+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=203}) v3 -- 0x7f94900721a0 con 0x7f94b015cf00 2023-08-25T09:53:04.192 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.190+0000 7f94aeffd640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f949006a050 0x7f949006c510 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f94a00012e0 tx=0x7f94a0007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:04.194 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.193+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94b00b3440 con 0x7f94b015cf00 2023-08-25T09:53:04.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.419+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f94b00b3440 con 0x7f94b015cf00 2023-08-25T09:53:04.421 INFO:tasks.ceph.mon.a.smithi070.stderr:2023-08-25T09:53:04.420+0000 7fc75f1e5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T09:53:04.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.425+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(203..203 src has 1..203) v4 ==== 374+0+0 (secure 0 0 0) 0x7f94a8004e30 con 0x7f94b015cf00 2023-08-25T09:53:04.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.425+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=204}) v3 -- 0x7f9490072a50 con 0x7f94b015cf00 2023-08-25T09:53:04.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.431+0000 7f94a77fe640 1 -- 172.21.15.70:0/4055095567 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f94b00b3440 con 0x7f94b015cf00 2023-08-25T09:53:04.433 INFO:teuthology.orchestra.run.smithi070.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T09:53:04.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f949006a050 msgr2=0x7f949006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f949006a050 0x7f949006c510 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f94a00012e0 tx=0x7f94a0007d30 comp rx=0 tx=0).stop 2023-08-25T09:53:04.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 msgr2=0x7f94b00b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b00b3d10 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f94a800c450 tx=0x7f94a800c910 comp rx=0 tx=0).stop 2023-08-25T09:53:04.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 shutdown_connections 2023-08-25T09:53:04.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f949006a050 0x7f949006c510 secure :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f94a00012e0 tx=0x7f94a0007d30 comp rx=0 tx=0).stop 2023-08-25T09:53:04.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f94b015cf00 0x7f94b00b3d10 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f94b00cf210 0x7f94b00b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 --2- 172.21.15.70:0/4055095567 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f94b00b7df0 0x7f94b0158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.435+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 >> 172.21.15.70:0/4055095567 conn(0x7f94b00b43a0 msgr2=0x7f94b00c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:04.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.436+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 shutdown_connections 2023-08-25T09:53:04.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.436+0000 7f94affff640 1 -- 172.21.15.70:0/4055095567 wait complete. 2023-08-25T09:53:04.458 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T09:53:04.464 INFO:tasks.ceph.mds.a.smithi070.stderr: -8544> 2023-08-25T09:52:09.552+0000 7f9914bc5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T09:53:04.534 INFO:tasks.ceph.mds.a.smithi070.stdout:starting mds.a at 2023-08-25T09:53:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.665+0000 7f6878838640 1 Processor -- start 2023-08-25T09:53:04.667 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.665+0000 7f6878838640 1 -- start start 2023-08-25T09:53:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6878838640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f687415ae90 0x7f68740ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.668 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6878838640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f687406f300 0x7f687406f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6878838640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687415be60 con 0x7f68740ce410 2023-08-25T09:53:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6878838640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687406ee70 con 0x7f687406f300 2023-08-25T09:53:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6878838640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68740ced30 con 0x7f687415ae90 2023-08-25T09:53:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6872d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f687406f300 0x7f687406f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6872d9d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f687406f300 0x7f687406f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60464/0 (socket says 172.21.15.70:60464) 2023-08-25T09:53:04.669 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6872d9d640 1 -- 172.21.15.70:0/4126416538 learned_addr learned my addr 172.21.15.70:0/4126416538 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:04.670 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.667+0000 7f6872d9d640 1 -- 172.21.15.70:0/4126416538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f687415ae90 msgr2=0x7f68740ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.668+0000 7f6871d9b640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f687415ae90 0x7f68740ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.668+0000 7f6872d9d640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f687415ae90 0x7f68740ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.668+0000 7f6872d9d640 1 -- 172.21.15.70:0/4126416538 >> v1:172.21.15.155:6789/0 conn(0x7f68740ce410 legacy=0x7f6874070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.668+0000 7f6872d9d640 1 -- 172.21.15.70:0/4126416538 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687415b2f0 con 0x7f687406f300 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.668+0000 7f6871d9b640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f687415ae90 0x7f68740ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.669+0000 7f6872d9d640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f687406f790 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f686c009870 tx=0x7f686c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca75ef90dfea3dc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.669+0000 7f687159a640 1 -- 172.21.15.70:0/4126416538 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c031640 con 0x7f687406f300 2023-08-25T09:53:04.671 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.669+0000 7f687159a640 1 -- 172.21.15.70:0/4126416538 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686c0317a0 con 0x7f687406f300 2023-08-25T09:53:04.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.669+0000 7f687159a640 1 -- 172.21.15.70:0/4126416538 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686c031b30 con 0x7f687406f300 2023-08-25T09:53:04.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 -- 172.21.15.70:0/4126416538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 msgr2=0x7f687406f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.672 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f687406f790 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f686c009870 tx=0x7f686c02b8f0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 -- 172.21.15.70:0/4126416538 shutdown_connections 2023-08-25T09:53:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f687406f790 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 --2- 172.21.15.70:0/4126416538 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f687415ae90 0x7f68740ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 -- 172.21.15.70:0/4126416538 >> 172.21.15.70:0/4126416538 conn(0x7f6874069850 msgr2=0x7f6874069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 -- 172.21.15.70:0/4126416538 shutdown_connections 2023-08-25T09:53:04.673 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.670+0000 7f6878838640 1 -- 172.21.15.70:0/4126416538 wait complete. 2023-08-25T09:53:04.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.672+0000 7f6878838640 1 Processor -- start 2023-08-25T09:53:04.674 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.672+0000 7f6878838640 1 -- start start 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6878838640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f68741f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6878838640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68740ce410 0x7f6874156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6878838640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f687415ae90 0x7f68741f14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6878838640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6874156a40 con 0x7f687406f300 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6878838640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741f1a00 con 0x7f687415ae90 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6878838640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68741f1cd0 con 0x7f68740ce410 2023-08-25T09:53:04.675 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f687259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f68741f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f687259c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f68741f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60478/0 (socket says 172.21.15.70:60478) 2023-08-25T09:53:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f687259c640 1 -- 172.21.15.70:0/1324392720 learned_addr learned my addr 172.21.15.70:0/1324392720 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:04.676 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.673+0000 7f6871d9b640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68740ce410 0x7f6874156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6872d9d640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f687415ae90 0x7f68741f14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f687259c640 1 -- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68740ce410 msgr2=0x7f6874156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f687259c640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68740ce410 0x7f6874156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f687259c640 1 -- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f687415ae90 msgr2=0x7f68741f14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f687259c640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f687415ae90 0x7f68741f14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f687259c640 1 -- 172.21.15.70:0/1324392720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0094d0 con 0x7f687406f300 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6872d9d640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f687415ae90 0x7f68741f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f687259c640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f68741f0f40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f686400c7c0 tx=0x7f686400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400aa50 con 0x7f687406f300 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686400abb0 con 0x7f687406f300 2023-08-25T09:53:04.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68741f1fb0 con 0x7f687406f300 2023-08-25T09:53:04.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864012420 con 0x7f687406f300 2023-08-25T09:53:04.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.674+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68741f2570 con 0x7f687406f300 2023-08-25T09:53:04.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.675+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687404e420 con 0x7f687406f300 2023-08-25T09:53:04.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.677+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6864012580 con 0x7f687406f300 2023-08-25T09:53:04.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.678+0000 7f6862ffd640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f685406a050 0x7f685406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:04.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.678+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f68640888f0 con 0x7f687406f300 2023-08-25T09:53:04.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.678+0000 7f6871d9b640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f685406a050 0x7f685406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:04.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.678+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=204}) v3 -- 0x7f68540722b0 con 0x7f687406f300 2023-08-25T09:53:04.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.679+0000 7f6871d9b640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f685406a050 0x7f685406c510 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f68740ad9b0 tx=0x7f6868008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:04.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.682+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6864053ee0 con 0x7f687406f300 2023-08-25T09:53:04.910 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:04.908+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6874070f00 con 0x7f687406f300 2023-08-25T09:53:05.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.432+0000 7f6862ffd640 1 -- 172.21.15.70:0/1324392720 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6864057500 con 0x7f687406f300 2023-08-25T09:53:05.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f685406a050 msgr2=0x7f685406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f685406a050 0x7f685406c510 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f68740ad9b0 tx=0x7f6868008040 comp rx=0 tx=0).stop 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 msgr2=0x7f68741f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f68741f0f40 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f686400c7c0 tx=0x7f686400cc80 comp rx=0 tx=0).stop 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 shutdown_connections 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f685406a050 0x7f685406c510 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f687415ae90 0x7f68741f14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.438 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f68740ce410 0x7f6874156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 --2- 172.21.15.70:0/1324392720 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f687406f300 0x7f68741f0f40 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 >> 172.21.15.70:0/1324392720 conn(0x7f6874069850 msgr2=0x7f687406b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:05.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 shutdown_connections 2023-08-25T09:53:05.439 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.436+0000 7f6878838640 1 -- 172.21.15.70:0/1324392720 wait complete. 2023-08-25T09:53:05.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T09:53:05.677 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.676+0000 7f69161f3640 1 Processor -- start 2023-08-25T09:53:05.678 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.676+0000 7f69161f3640 1 -- start start 2023-08-25T09:53:05.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.678+0000 7f69161f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100cda60 0x7f69100ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:05.679 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.678+0000 7f69161f3640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f69100ca820 0x7f691015af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:05.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.678+0000 7f69161f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69100cacb0 con 0x7f6910155120 2023-08-25T09:53:05.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.678+0000 7f69161f3640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69100caf30 con 0x7f69100ca820 2023-08-25T09:53:05.680 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.678+0000 7f69161f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691015e6f0 con 0x7f69100cda60 2023-08-25T09:53:05.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.679+0000 7f6907fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100cda60 0x7f69100ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:05.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.679+0000 7f6907fff640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100cda60 0x7f69100ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3301/0 says I am v2:172.21.15.70:43590/0 (socket says 172.21.15.70:43590) 2023-08-25T09:53:05.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.679+0000 7f6907fff640 1 -- 172.21.15.70:0/1194957466 learned_addr learned my addr 172.21.15.70:0/1194957466 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:05.681 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.679+0000 7f690ffff640 1 --2- 172.21.15.70:0/1194957466 >> v2:172.21.15.155:3300/0 conn(0x7f69100ca820 0x7f691015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.680+0000 7f690ffff640 1 -- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100cda60 msgr2=0x7f69100ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.680+0000 7f690ffff640 1 --2- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100cda60 0x7f69100ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.680+0000 7f690ffff640 1 -- 172.21.15.70:0/1194957466 >> v1:172.21.15.155:6789/0 conn(0x7f6910155120 legacy=0x7f69100cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.680+0000 7f690ffff640 1 -- 172.21.15.70:0/1194957466 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691015e9d0 con 0x7f69100ca820 2023-08-25T09:53:05.682 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.681+0000 7f690ffff640 1 --2- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69100ca820 0x7f691015af60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6908009d90 tx=0x7f690802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e506d1d4b32ac0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f690effd640 1 -- 172.21.15.70:0/1194957466 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690802cc50 con 0x7f69100ca820 2023-08-25T09:53:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f690effd640 1 -- 172.21.15.70:0/1194957466 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690802cdb0 con 0x7f69100ca820 2023-08-25T09:53:05.683 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f690effd640 1 -- 172.21.15.70:0/1194957466 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908034660 con 0x7f69100ca820 2023-08-25T09:53:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 -- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69100ca820 msgr2=0x7f691015af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 --2- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69100ca820 0x7f691015af60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6908009d90 tx=0x7f690802bea0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 -- 172.21.15.70:0/1194957466 shutdown_connections 2023-08-25T09:53:05.684 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 --2- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f69100ca820 0x7f691015af60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 --2- 172.21.15.70:0/1194957466 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100cda60 0x7f69100ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 -- 172.21.15.70:0/1194957466 >> 172.21.15.70:0/1194957466 conn(0x7f6910069c70 msgr2=0x7f691006a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 -- 172.21.15.70:0/1194957466 shutdown_connections 2023-08-25T09:53:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.682+0000 7f69161f3640 1 -- 172.21.15.70:0/1194957466 wait complete. 2023-08-25T09:53:05.685 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.684+0000 7f69161f3640 1 Processor -- start 2023-08-25T09:53:05.686 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.684+0000 7f69161f3640 1 -- start start 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f69161f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100ca820 0x7f6910143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f69161f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 0x7f691006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f69161f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6910155120 0x7f691006afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f69161f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910141e10 con 0x7f69100cda60 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f69161f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910141500 con 0x7f6910155120 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f69161f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6910157850 con 0x7f69100ca820 2023-08-25T09:53:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.685+0000 7f6907fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 0x7f691006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:05.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 0x7f691006aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60494/0 (socket says 172.21.15.70:60494) 2023-08-25T09:53:05.688 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 -- 172.21.15.70:0/3280959775 learned_addr learned my addr 172.21.15.70:0/3280959775 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f690f7fe640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100ca820 0x7f6910143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f690ffff640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6910155120 0x7f691006afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 -- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100ca820 msgr2=0x7f6910143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100ca820 0x7f6910143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.689 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 -- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6910155120 msgr2=0x7f691006afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6910155120 0x7f691006afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6908009a70 con 0x7f69100cda60 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f6907fff640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 0x7f691006aa70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f68f800e8b0 tx=0x7f68f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.686+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800c940 con 0x7f69100cda60 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.687+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800caa0 con 0x7f69100cda60 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.687+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69101444b0 con 0x7f69100cda60 2023-08-25T09:53:05.690 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.687+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800ce30 con 0x7f69100cda60 2023-08-25T09:53:05.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.687+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6910141750 con 0x7f69100cda60 2023-08-25T09:53:05.691 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.688+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691004e420 con 0x7f69100cda60 2023-08-25T09:53:05.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.689+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f68f8013470 con 0x7f69100cda60 2023-08-25T09:53:05.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.690+0000 7f690cff9640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f68f006a050 0x7f68f006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:05.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.690+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f68f8088760 con 0x7f69100cda60 2023-08-25T09:53:05.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.690+0000 7f690f7fe640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f68f006a050 0x7f68f006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:05.692 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.690+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=204}) v3 -- 0x7f68f00722b0 con 0x7f69100cda60 2023-08-25T09:53:05.693 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.690+0000 7f690f7fe640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f68f006a050 0x7f68f006c510 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6900000f60 tx=0x7f6900009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:05.695 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.694+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f68f8053de0 con 0x7f69100cda60 2023-08-25T09:53:05.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:05.882+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f69100715c0 con 0x7f69100cda60 2023-08-25T09:53:06.623 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.622+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(204..204 src has 1..204) v4 ==== 296+0+0 (secure 0 0 0) 0x7f68f8058130 con 0x7f69100cda60 2023-08-25T09:53:06.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.622+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7f68f00706e0 con 0x7f69100cda60 2023-08-25T09:53:06.636 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.633+0000 7f690cff9640 1 -- 172.21.15.70:0/3280959775 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v204) v1 ==== 181+0+0 (secure 0 0 0) 0x7f68f8021020 con 0x7f69100cda60 2023-08-25T09:53:06.637 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_metadata' removed 2023-08-25T09:53:06.641 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f68f006a050 msgr2=0x7f68f006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:06.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f68f006a050 0x7f68f006c510 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f6900000f60 tx=0x7f6900009040 comp rx=0 tx=0).stop 2023-08-25T09:53:06.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 msgr2=0x7f691006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:06.642 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 0x7f691006aa70 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f68f800e8b0 tx=0x7f68f800ed70 comp rx=0 tx=0).stop 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 shutdown_connections 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f68f006a050 0x7f68f006c510 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f6910155120 0x7f691006afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f69100cda60 0x7f691006aa70 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 --2- 172.21.15.70:0/3280959775 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f69100ca820 0x7f6910143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.640+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 >> 172.21.15.70:0/3280959775 conn(0x7f6910069c70 msgr2=0x7f69100cd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:06.643 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.641+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 shutdown_connections 2023-08-25T09:53:06.644 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.641+0000 7f69161f3640 1 -- 172.21.15.70:0/3280959775 wait complete. 2023-08-25T09:53:06.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T09:53:06.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.867+0000 7fb7eff42640 1 Processor -- start 2023-08-25T09:53:06.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.867+0000 7fb7eff42640 1 -- start start 2023-08-25T09:53:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7eff42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7e8142740 0x7fb7e80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7eff42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e806aa70 0x7fb7e806af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7eff42640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e815d320 con 0x7fb7e8142c30 2023-08-25T09:53:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7eff42640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e806b440 con 0x7fb7e8142740 2023-08-25T09:53:06.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7eff42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e80ccd40 con 0x7fb7e806aa70 2023-08-25T09:53:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7edcde640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fb7e8142740 0x7fb7e80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7ed4dd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb7e8142c30 0x7fb7e815db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43434/0 (socket says 172.21.15.70:43434) 2023-08-25T09:53:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7ed4dd640 1 -- 172.21.15.70:0/1470849833 learned_addr learned my addr 172.21.15.70:0/1470849833 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7edcde640 1 -- 172.21.15.70:0/1470849833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e806aa70 msgr2=0x7fb7e806af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:06.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7edcde640 1 --2- 172.21.15.70:0/1470849833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e806aa70 0x7fb7e806af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7edcde640 1 -- 172.21.15.70:0/1470849833 >> v1:172.21.15.70:6789/0 conn(0x7fb7e8142c30 legacy=0x7fb7e815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7eccdc640 1 -- 172.21.15.70:0/1470849833 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2130435275 0 0) 0x7fb7e815d320 con 0x7fb7e8142c30 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.869+0000 7fb7edcde640 1 -- 172.21.15.70:0/1470849833 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e80cd020 con 0x7fb7e8142740 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.870+0000 7fb7edcde640 1 --2- 172.21.15.70:0/1470849833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e80cc6b0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009dc0 tx=0x7fb7e402be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27a20a316039c61 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eccdc640 1 -- 172.21.15.70:0/1470849833 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e402cc40 con 0x7fb7e8142740 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eccdc640 1 -- 172.21.15.70:0/1470849833 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e402cda0 con 0x7fb7e8142740 2023-08-25T09:53:06.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eccdc640 1 -- 172.21.15.70:0/1470849833 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4034660 con 0x7fb7e8142740 2023-08-25T09:53:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eff42640 1 -- 172.21.15.70:0/1470849833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 msgr2=0x7fb7e80cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:06.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eff42640 1 --2- 172.21.15.70:0/1470849833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e80cc6b0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009dc0 tx=0x7fb7e402be90 comp rx=0 tx=0).stop 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eff42640 1 -- 172.21.15.70:0/1470849833 shutdown_connections 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eff42640 1 --2- 172.21.15.70:0/1470849833 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e806aa70 0x7fb7e806af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eff42640 1 --2- 172.21.15.70:0/1470849833 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e80cc6b0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.871+0000 7fb7eff42640 1 -- 172.21.15.70:0/1470849833 >> 172.21.15.70:0/1470849833 conn(0x7fb7e8069c70 msgr2=0x7fb7e806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.872+0000 7fb7eff42640 1 -- 172.21.15.70:0/1470849833 shutdown_connections 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.872+0000 7fb7eff42640 1 -- 172.21.15.70:0/1470849833 wait complete. 2023-08-25T09:53:06.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.873+0000 7fb7eff42640 1 Processor -- start 2023-08-25T09:53:06.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.873+0000 7fb7eff42640 1 -- start start 2023-08-25T09:53:06.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.874+0000 7fb7eff42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e806aa70 0x7fb7e8139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:06.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.874+0000 7fb7eff42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e806ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:06.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.874+0000 7fb7eff42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e8142c30 0x7fb7e8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.874+0000 7fb7eff42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815ca30 con 0x7fb7e8142740 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.874+0000 7fb7eff42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e813c7d0 con 0x7fb7e806aa70 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.874+0000 7fb7eff42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e815c470 con 0x7fb7e8142c30 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e806ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e806ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60524/0 (socket says 172.21.15.70:60524) 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 -- 172.21.15.70:0/676716849 learned_addr learned my addr 172.21.15.70:0/676716849 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 -- 172.21.15.70:0/676716849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e8142c30 msgr2=0x7fb7e8158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:06.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e8142c30 0x7fb7e8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 -- 172.21.15.70:0/676716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e806aa70 msgr2=0x7fb7e8139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e806aa70 0x7fb7e8139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e4009a70 con 0x7fb7e8142740 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.875+0000 7fb7ed4dd640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e806ff10 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800c430 tx=0x7fb7d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.876+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d80074e0 con 0x7fb7e8142740 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.876+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d8007660 con 0x7fb7e8142740 2023-08-25T09:53:06.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.876+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8004a20 con 0x7fb7e8142740 2023-08-25T09:53:06.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.876+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e815c750 con 0x7fb7e8142740 2023-08-25T09:53:06.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.876+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e813cd50 con 0x7fb7e8142740 2023-08-25T09:53:06.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.877+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e813cf70 con 0x7fb7e8142740 2023-08-25T09:53:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.878+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb7d8004b80 con 0x7fb7e8142740 2023-08-25T09:53:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.879+0000 7fb7de7fc640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7cc06a050 0x7fb7cc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.879+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fb7d80882b0 con 0x7fb7e8142740 2023-08-25T09:53:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.879+0000 7fb7edcde640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7cc06a050 0x7fb7cc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:06.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.879+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7fb7cc071440 con 0x7fb7e8142740 2023-08-25T09:53:06.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.880+0000 7fb7edcde640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7cc06a050 0x7fb7cc06c510 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4036000 tx=0x7fb7e402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:06.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:06.883+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7d8053a60 con 0x7fb7e8142740 2023-08-25T09:53:07.074 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.072+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7e804e420 con 0x7fb7e8142740 2023-08-25T09:53:07.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.626+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 7 ==== osd_map(205..205 src has 1..205) v4 ==== 296+0+0 (secure 0 0 0) 0x7fb7d8004e30 con 0x7fb7e8142740 2023-08-25T09:53:07.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.626+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=206}) v3 -- 0x7fb7cc0701d0 con 0x7fb7e8142740 2023-08-25T09:53:07.648 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.646+0000 7fb7de7fc640 1 -- 172.21.15.70:0/676716849 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v205) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb7d8057080 con 0x7fb7e8142740 2023-08-25T09:53:07.648 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'cephfs_data' removed 2023-08-25T09:53:07.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.653+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7cc06a050 msgr2=0x7fb7cc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:07.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.653+0000 7fb7eff42640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7cc06a050 0x7fb7cc06c510 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4036000 tx=0x7fb7e402e000 comp rx=0 tx=0).stop 2023-08-25T09:53:07.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.653+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 msgr2=0x7fb7e806ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:07.654 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.653+0000 7fb7eff42640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e806ff10 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800c430 tx=0x7fb7d800c8f0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.654+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 shutdown_connections 2023-08-25T09:53:07.655 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.654+0000 7fb7eff42640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb7cc06a050 0x7fb7cc06c510 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.654+0000 7fb7eff42640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb7e8142c30 0x7fb7e8158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.654+0000 7fb7eff42640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb7e8142740 0x7fb7e806ff10 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.654+0000 7fb7eff42640 1 --2- 172.21.15.70:0/676716849 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb7e806aa70 0x7fb7e8139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.656 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.654+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 >> 172.21.15.70:0/676716849 conn(0x7fb7e8069c70 msgr2=0x7fb7e80b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.655+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 shutdown_connections 2023-08-25T09:53:07.657 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.655+0000 7fb7eff42640 1 -- 172.21.15.70:0/676716849 wait complete. 2023-08-25T09:53:07.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T09:53:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.872+0000 7fc9a808e640 1 Processor -- start 2023-08-25T09:53:07.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.873+0000 7fc9a808e640 1 -- start start 2023-08-25T09:53:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.874+0000 7fc9a808e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc9a00ce410 0x7fc9a015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.874+0000 7fc9a808e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.874+0000 7fc9a808e640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a015be60 con 0x7fc9a006f300 2023-08-25T09:53:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.874+0000 7fc9a808e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a015b3d0 con 0x7fc9a00ce410 2023-08-25T09:53:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.874+0000 7fc9a808e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a006ee70 con 0x7fc9a0070770 2023-08-25T09:53:07.876 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.874+0000 7fc9a5e2a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc9a00ce410 0x7fc9a015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5e2a640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc9a00ce410 0x7fc9a015ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60540/0 (socket says 172.21.15.70:60540) 2023-08-25T09:53:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5e2a640 1 -- 172.21.15.70:0/3919826727 learned_addr learned my addr 172.21.15.70:0/3919826727 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:07.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5e2a640 1 -- 172.21.15.70:0/3919826727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 msgr2=0x7fc9a00ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5629640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5e2a640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5e2a640 1 -- 172.21.15.70:0/3919826727 >> v1:172.21.15.70:6789/0 conn(0x7fc9a006f300 legacy=0x7fc9a006aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5e2a640 1 -- 172.21.15.70:0/3919826727 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a00ced30 con 0x7fc9a00ce410 2023-08-25T09:53:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a4e28640 1 -- 172.21.15.70:0/3919826727 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438253986 0 0) 0x7fc9a015be60 con 0x7fc9a006f300 2023-08-25T09:53:07.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.875+0000 7fc9a5629640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.876+0000 7fc9a5e2a640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a015ae90 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009870 tx=0x7fc99c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c013bfbda0fc6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.876+0000 7fc9a4e28640 1 -- 172.21.15.70:0/3919826727 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c030840 con 0x7fc9a00ce410 2023-08-25T09:53:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.876+0000 7fc9a4e28640 1 -- 172.21.15.70:0/3919826727 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c0309a0 con 0x7fc9a00ce410 2023-08-25T09:53:07.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a4e28640 1 -- 172.21.15.70:0/3919826727 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c030d30 con 0x7fc9a00ce410 2023-08-25T09:53:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 -- 172.21.15.70:0/3919826727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 msgr2=0x7fc9a015ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a015ae90 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009870 tx=0x7fc99c02b980 comp rx=0 tx=0).stop 2023-08-25T09:53:07.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 -- 172.21.15.70:0/3919826727 shutdown_connections 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 --2- 172.21.15.70:0/3919826727 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a015ae90 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 -- 172.21.15.70:0/3919826727 >> 172.21.15.70:0/3919826727 conn(0x7fc9a0069850 msgr2=0x7fc9a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 -- 172.21.15.70:0/3919826727 shutdown_connections 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.877+0000 7fc9a808e640 1 -- 172.21.15.70:0/3919826727 wait complete. 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.879+0000 7fc9a808e640 1 Processor -- start 2023-08-25T09:53:07.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.879+0000 7fc9a808e640 1 -- start start 2023-08-25T09:53:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a808e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a006f300 0x7fc9a014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a808e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ad830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a808e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a00add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:07.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a808e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00ad210 con 0x7fc9a00ce410 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a808e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a01501b0 con 0x7fc9a006f300 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a808e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a00af9a0 con 0x7fc9a0070770 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.880+0000 7fc9a662b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a00add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a00add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60542/0 (socket says 172.21.15.70:60542) 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 -- 172.21.15.70:0/1533948218 learned_addr learned my addr 172.21.15.70:0/1533948218 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a5629640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a5e2a640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a006f300 0x7fc9a014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:07.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 -- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 msgr2=0x7fc9a00ad830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ad830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 -- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a006f300 msgr2=0x7fc9a014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a006f300 0x7fc9a014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 -- 172.21.15.70:0/1533948218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0094d0 con 0x7fc9a00ce410 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.881+0000 7fc9a662b640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a00add70 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fc998001560 tx=0x7fc99800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.882+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998003da0 con 0x7fc9a00ce410 2023-08-25T09:53:07.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.882+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc998005070 con 0x7fc9a00ce410 2023-08-25T09:53:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.882+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998005420 con 0x7fc9a00ce410 2023-08-25T09:53:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.882+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a0152360 con 0x7fc9a00ce410 2023-08-25T09:53:07.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.882+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a0152920 con 0x7fc9a00ce410 2023-08-25T09:53:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.883+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a01521b0 con 0x7fc9a00ce410 2023-08-25T09:53:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.884+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc998005580 con 0x7fc9a00ce410 2023-08-25T09:53:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.885+0000 7fc9967fc640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc98406a080 0x7fc98406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.885+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc998087b20 con 0x7fc9a00ce410 2023-08-25T09:53:07.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.885+0000 7fc9a5e2a640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc98406a080 0x7fc98406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:07.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.886+0000 7fc9a5e2a640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc98406a080 0x7fc98406c540 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc99c036000 tx=0x7fc99c03b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:07.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:07.889+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc998089e60 con 0x7fc9a00ce410 2023-08-25T09:53:08.078 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.077+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc9a0152b40 con 0x7fc9a00ce410 2023-08-25T09:53:08.087 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.086+0000 7fc9967fc640 1 -- 172.21.15.70:0/1533948218 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc998017020 con 0x7fc9a00ce410 2023-08-25T09:53:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc98406a080 msgr2=0x7fc98406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc98406a080 0x7fc98406c540 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc99c036000 tx=0x7fc99c03b040 comp rx=0 tx=0).stop 2023-08-25T09:53:08.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 msgr2=0x7fc9a00add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a00add70 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7fc998001560 tx=0x7fc99800cc70 comp rx=0 tx=0).stop 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 shutdown_connections 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc98406a080 0x7fc98406c540 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc9a00ce410 0x7fc9a00add70 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc9a0070770 0x7fc9a00ad830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 --2- 172.21.15.70:0/1533948218 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc9a006f300 0x7fc9a014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.091 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 >> 172.21.15.70:0/1533948218 conn(0x7fc9a0069850 msgr2=0x7fc9a006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:08.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 shutdown_connections 2023-08-25T09:53:08.092 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.089+0000 7fc9a808e640 1 -- 172.21.15.70:0/1533948218 wait complete. 2023-08-25T09:53:08.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-08-25T09:53:08.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.326+0000 7f3b39a21640 1 Processor -- start 2023-08-25T09:53:08.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.326+0000 7f3b39a21640 1 -- start start 2023-08-25T09:53:08.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.328+0000 7f3b39a21640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd3a0 0x7f3b340cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:08.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.328+0000 7f3b39a21640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b34159d10 con 0x7f3b340b3c80 2023-08-25T09:53:08.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.328+0000 7f3b39a21640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b340b2d10 con 0x7f3b34159930 2023-08-25T09:53:08.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.328+0000 7f3b39a21640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b340b2ff0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b33fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f3b340b3c80 0x7f3b340caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:43452/0 (socket says 172.21.15.70:43452) 2023-08-25T09:53:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b33fff640 1 -- 172.21.15.70:0/3008411315 learned_addr learned my addr 172.21.15.70:0/3008411315 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b39220640 1 --2- 172.21.15.70:0/3008411315 >> v2:172.21.15.70:3300/0 conn(0x7f3b340cd3a0 0x7f3b340cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:08.332 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b337fe640 1 -- 172.21.15.70:0/3008411315 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235032471 0 0) 0x7f3b34159d10 con 0x7f3b340b3c80 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b337fe640 1 -- 172.21.15.70:0/3008411315 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b24005730 con 0x7f3b340b3c80 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b39220640 1 -- 172.21.15.70:0/3008411315 >> v1:172.21.15.155:6789/0 conn(0x7f3b34159930 legacy=0x7f3b340b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b39220640 1 -- 172.21.15.70:0/3008411315 >> v1:172.21.15.70:6789/0 conn(0x7f3b340b3c80 legacy=0x7f3b340caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.329+0000 7f3b39220640 1 -- 172.21.15.70:0/3008411315 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340b32d0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.330+0000 7f3b39220640 1 --2- 172.21.15.70:0/3008411315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340cb200 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009d90 tx=0x7f3b2c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef835b6c73c4baae server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b337fe640 1 -- 172.21.15.70:0/3008411315 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c02cc60 con 0x7f3b340cd3a0 2023-08-25T09:53:08.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b337fe640 1 -- 172.21.15.70:0/3008411315 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c02cdc0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b337fe640 1 -- 172.21.15.70:0/3008411315 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c034660 con 0x7f3b340cd3a0 2023-08-25T09:53:08.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b39a21640 1 -- 172.21.15.70:0/3008411315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 msgr2=0x7f3b340cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b39a21640 1 --2- 172.21.15.70:0/3008411315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340cb200 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009d90 tx=0x7f3b2c02beb0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b39a21640 1 -- 172.21.15.70:0/3008411315 shutdown_connections 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b39a21640 1 --2- 172.21.15.70:0/3008411315 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340cb200 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.331+0000 7f3b39a21640 1 -- 172.21.15.70:0/3008411315 >> 172.21.15.70:0/3008411315 conn(0x7f3b340aff60 msgr2=0x7f3b340b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.332+0000 7f3b39a21640 1 -- 172.21.15.70:0/3008411315 shutdown_connections 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.332+0000 7f3b39a21640 1 -- 172.21.15.70:0/3008411315 wait complete. 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.333+0000 7f3b39a21640 1 Processor -- start 2023-08-25T09:53:08.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.333+0000 7f3b39a21640 1 -- start start 2023-08-25T09:53:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.334+0000 7f3b39a21640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b340b3c80 0x7f3b34156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.334+0000 7f3b39a21640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.334+0000 7f3b39a21640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b34159930 0x7f3b340c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.334+0000 7f3b39a21640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c8410 con 0x7f3b340cd3a0 2023-08-25T09:53:08.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.334+0000 7f3b39a21640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34156f40 con 0x7f3b34159930 2023-08-25T09:53:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.334+0000 7f3b39a21640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c4800 con 0x7f3b340b3c80 2023-08-25T09:53:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:60558/0 (socket says 172.21.15.70:60558) 2023-08-25T09:53:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 -- 172.21.15.70:0/1452818526 learned_addr learned my addr 172.21.15.70:0/1452818526 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b38a1f640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b340b3c80 0x7f3b34156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:08.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 -- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b340b3c80 msgr2=0x7f3b34156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b340b3c80 0x7f3b34156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 -- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b34159930 msgr2=0x7f3b340c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b34159930 0x7f3b340c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 -- 172.21.15.70:0/1452818526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c009a70 con 0x7f3b340cd3a0 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b38a1f640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b340b3c80 0x7f3b34156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.335+0000 7f3b33fff640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340c3f90 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000c500 tx=0x7f3b2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:08.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.336+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20005ce0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.336+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b20005e40 con 0x7f3b340cd3a0 2023-08-25T09:53:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.336+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b20007330 con 0x7f3b340cd3a0 2023-08-25T09:53:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.336+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b340c4ae0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.336+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b340c50a0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.337+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34066fb0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.338+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3b20007550 con 0x7f3b340cd3a0 2023-08-25T09:53:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.339+0000 7f3b317fa640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3b1806a050 0x7f3b1806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.339+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3b200878c0 con 0x7f3b340cd3a0 2023-08-25T09:53:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.339+0000 7f3b38a1f640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3b1806a050 0x7f3b1806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:08.342 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.340+0000 7f3b38a1f640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3b1806a050 0x7f3b1806c510 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3b28000f60 tx=0x7f3b28009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:08.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.343+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3b20053400 con 0x7f3b340cd3a0 2023-08-25T09:53:08.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.601+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f3b340ae430 con 0x7f3b340cd3a0 2023-08-25T09:53:08.661 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.659+0000 7f3b317fa640 1 -- 172.21.15.70:0/1452818526 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v704) v1 ==== 135+0+0 (secure 0 0 0) 0x7f3b20017020 con 0x7f3b340cd3a0 2023-08-25T09:53:08.663 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3b1806a050 msgr2=0x7f3b1806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3b1806a050 0x7f3b1806c510 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f3b28000f60 tx=0x7f3b28009040 comp rx=0 tx=0).stop 2023-08-25T09:53:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 msgr2=0x7f3b340c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340c3f90 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f3b2000c500 tx=0x7f3b2000c9c0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 shutdown_connections 2023-08-25T09:53:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f3b1806a050 0x7f3b1806c510 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.664 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f3b34159930 0x7f3b340c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f3b340cd3a0 0x7f3b340c3f90 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 --2- 172.21.15.70:0/1452818526 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f3b340b3c80 0x7f3b34156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 >> 172.21.15.70:0/1452818526 conn(0x7f3b340aff60 msgr2=0x7f3b340aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 shutdown_connections 2023-08-25T09:53:08.665 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:08.662+0000 7f3b39a21640 1 -- 172.21.15.70:0/1452818526 wait complete. 2023-08-25T09:53:08.682 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-08-25T09:53:08.682 INFO:tasks.cephfs_test_runner: 2023-08-25T09:53:08.682 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-25T09:53:08.683 INFO:tasks.cephfs_test_runner:Ran 10 tests in 809.173s 2023-08-25T09:53:08.683 INFO:tasks.cephfs_test_runner: 2023-08-25T09:53:08.683 INFO:tasks.cephfs_test_runner:OK 2023-08-25T09:53:08.683 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-08-25T09:53:08.693 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-08-25T09:53:08.743 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-08-25T09:53:08.744 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:53:08.744 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:53:08.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:53:08.802 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2023-08-25T09:53:08.802 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:53:08.855 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053656,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248792k,nr_inodes=812198,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497588k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-08-25T09:53:08.856 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:53:08.856 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:53:08.917 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:53:08.921 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2023-08-25T09:53:08.922 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T09:53:08.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053668,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248800k,nr_inodes=812200,mode=700,uid=1000,gid=1275 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497608k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-08-25T09:53:08.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:08.977 DEBUG:teuthology.orchestra.run.smithi070:> ip netns list 2023-08-25T09:53:09.037 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T09:53:09.037 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T09:53:09.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T09:53:09.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T09:53:09.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo ip link delete ceph-brx 2023-08-25T09:53:09.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.348 DEBUG:teuthology.orchestra.run.smithi155:> ip netns list 2023-08-25T09:53:09.408 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T09:53:09.409 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T09:53:09.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.410 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T09:53:09.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.512 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T09:53:09.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T09:53:09.603 DEBUG:teuthology.orchestra.run.smithi155:> sudo ip link delete ceph-brx 2023-08-25T09:53:09.746 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-08-25T09:53:09.757 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-25T09:53:09.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:53:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.979+0000 7fc75b59e640 1 Processor -- start 2023-08-25T09:53:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.979+0000 7fc75b59e640 1 -- start start 2023-08-25T09:53:09.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.980+0000 7fc75b59e640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc75c15a7c0 0x7fc75c0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.980+0000 7fc75b59e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc75c0b12f0 0x7fc75c15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.980+0000 7fc75b59e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc75c15e7c0 con 0x7fc75c15f190 2023-08-25T09:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.980+0000 7fc75b59e640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75c071b50 con 0x7fc75c0b12f0 2023-08-25T09:53:09.982 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.980+0000 7fc75b59e640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75c15e490 con 0x7fc75c15a7c0 2023-08-25T09:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc75c0b12f0 0x7fc75c15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc75c0b12f0 0x7fc75c15ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54384/0 (socket says 172.21.15.70:54384) 2023-08-25T09:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 -- 172.21.15.70:0/3488636337 learned_addr learned my addr 172.21.15.70:0/3488636337 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 -- 172.21.15.70:0/3488636337 >> v2:172.21.15.155:3300/0 conn(0x7fc75c15a7c0 msgr2=0x7fc75c0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:09.983 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 --2- 172.21.15.70:0/3488636337 >> v2:172.21.15.155:3300/0 conn(0x7fc75c15a7c0 0x7fc75c0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 -- 172.21.15.70:0/3488636337 >> v1:172.21.15.155:6789/0 conn(0x7fc75c15f190 legacy=0x7fc75c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:09.984 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.981+0000 7fc753fff640 1 -- 172.21.15.70:0/3488636337 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0b3aa0 con 0x7fc75c0b12f0 2023-08-25T09:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.982+0000 7fc753fff640 1 --2- 172.21.15.70:0/3488636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c0b12f0 0x7fc75c15ec50 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0ce280 tx=0x7fc74402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cca0687b3d8445 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.982+0000 7fc759d9b640 1 -- 172.21.15.70:0/3488636337 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744031870 con 0x7fc75c0b12f0 2023-08-25T09:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc759d9b640 1 -- 172.21.15.70:0/3488636337 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7440319d0 con 0x7fc75c0b12f0 2023-08-25T09:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc759d9b640 1 -- 172.21.15.70:0/3488636337 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744031d60 con 0x7fc75c0b12f0 2023-08-25T09:53:09.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 -- 172.21.15.70:0/3488636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c0b12f0 msgr2=0x7fc75c15ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3488636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c0b12f0 0x7fc75c15ec50 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0ce280 tx=0x7fc74402b910 comp rx=0 tx=0).stop 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 -- 172.21.15.70:0/3488636337 shutdown_connections 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3488636337 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c0b12f0 0x7fc75c15ec50 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3488636337 >> v2:172.21.15.155:3300/0 conn(0x7fc75c15a7c0 0x7fc75c0b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 -- 172.21.15.70:0/3488636337 >> 172.21.15.70:0/3488636337 conn(0x7fc75c0affd0 msgr2=0x7fc75c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.983+0000 7fc75b59e640 1 -- 172.21.15.70:0/3488636337 shutdown_connections 2023-08-25T09:53:09.986 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.984+0000 7fc75b59e640 1 -- 172.21.15.70:0/3488636337 wait complete. 2023-08-25T09:53:09.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.985+0000 7fc75b59e640 1 Processor -- start 2023-08-25T09:53:09.987 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.985+0000 7fc75b59e640 1 -- start start 2023-08-25T09:53:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.986+0000 7fc75b59e640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc75c0b12f0 0x7fc75c1f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.986+0000 7fc75b59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 0x7fc75c1f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.986+0000 7fc75b59e640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c15f190 0x7fc75c0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:09.988 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.986+0000 7fc75b59e640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c1f2f10 con 0x7fc75c15a7c0 2023-08-25T09:53:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.986+0000 7fc75b59e640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c1f3100 con 0x7fc75c15f190 2023-08-25T09:53:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.986+0000 7fc75b59e640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc75c0c4480 con 0x7fc75c0b12f0 2023-08-25T09:53:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 0x7fc75c1f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 0x7fc75c1f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54386/0 (socket says 172.21.15.70:54386) 2023-08-25T09:53:09.989 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 -- 172.21.15.70:0/3452489778 learned_addr learned my addr 172.21.15.70:0/3452489778 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 -- 172.21.15.70:0/3452489778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc75c0b12f0 msgr2=0x7fc75c1f29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc75c0b12f0 0x7fc75c1f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:09.990 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 -- 172.21.15.70:0/3452489778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c15f190 msgr2=0x7fc75c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c15f190 0x7fc75c0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.987+0000 7fc753fff640 1 -- 172.21.15.70:0/3452489778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440094d0 con 0x7fc75c15a7c0 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.988+0000 7fc753fff640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 0x7fc75c1f1840 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fc74402b950 tx=0x7fc7440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.988+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74403ad10 con 0x7fc75c15a7c0 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.988+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74403ae70 con 0x7fc75c15a7c0 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.988+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744044610 con 0x7fc75c15a7c0 2023-08-25T09:53:09.991 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.988+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75c0c3b50 con 0x7fc75c15a7c0 2023-08-25T09:53:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.988+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75c1f5d70 con 0x7fc75c15a7c0 2023-08-25T09:53:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.989+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75c1f32d0 con 0x7fc75c15a7c0 2023-08-25T09:53:09.992 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.990+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc744044770 con 0x7fc75c15a7c0 2023-08-25T09:53:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.991+0000 7fc752ffd640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc73c06a050 0x7fc73c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.991+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc74402d040 con 0x7fc75c15a7c0 2023-08-25T09:53:09.993 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.991+0000 7fc75a59c640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc73c06a050 0x7fc73c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:09.994 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.992+0000 7fc75a59c640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc73c06a050 0x7fc73c06c510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009710 tx=0x7fc74c006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:09.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:09.996+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc75c1f32d0 con 0x7fc75c15a7c0 2023-08-25T09:53:10.180 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.178+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc75c0b4040 con 0x7fc73c06a050 2023-08-25T09:53:10.183 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.182+0000 7fc752ffd640 1 -- 172.21.15.70:0/3452489778 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68035 (secure 0 0 0) 0x7fc74c00b480 con 0x7fc73c06a050 2023-08-25T09:53:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:10.185 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:53:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc73c06a050 msgr2=0x7fc73c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc73c06a050 0x7fc73c06c510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc74c009710 tx=0x7fc74c006c90 comp rx=0 tx=0).stop 2023-08-25T09:53:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 msgr2=0x7fc75c1f1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 0x7fc75c1f1840 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fc74402b950 tx=0x7fc7440046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 shutdown_connections 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc73c06a050 0x7fc73c06c510 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc75c15f190 0x7fc75c0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc75c15a7c0 0x7fc75c1f1840 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 --2- 172.21.15.70:0/3452489778 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc75c0b12f0 0x7fc75c1f29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.184+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 >> 172.21.15.70:0/3452489778 conn(0x7fc75c0affd0 msgr2=0x7fc75c0ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:10.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.185+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 shutdown_connections 2023-08-25T09:53:10.188 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.185+0000 7fc75b59e640 1 -- 172.21.15.70:0/3452489778 wait complete. 2023-08-25T09:53:10.204 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2023-08-25T09:53:09.615244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10818,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1551681,"statfs":{"total":1600000000,"available":1588922304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485516,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-130,"num_write_kb":-85085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002202"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":494,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.632435+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:53:07.632435+0000","last_peered":"2023-08-25T09:53:07.632435+0000","last_clean":"2023-08-25T09:53:07.632435+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:53:07.632435+0000","last_undegraded":"2023-08-25T09:53:07.632435+0000","last_fullsized":"2023-08-25T09:53:07.632435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:01.074556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.635287+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:53:07.635287+0000","last_peered":"2023-08-25T09:53:07.635287+0000","last_clean":"2023-08-25T09:53:07.635287+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:53:07.635287+0000","last_undegraded":"2023-08-25T09:53:07.635287+0000","last_fullsized":"2023-08-25T09:53:07.635287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:09.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":372,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.631062+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:53:07.631062+0000","last_peered":"2023-08-25T09:53:07.631062+0000","last_clean":"2023-08-25T09:53:07.631062+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:53:07.631062+0000","last_undegraded":"2023-08-25T09:53:07.631062+0000","last_fullsized":"2023-08-25T09:53:07.631062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:11.279207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":373,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.650798+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:53:07.650798+0000","last_peered":"2023-08-25T09:53:07.650798+0000","last_clean":"2023-08-25T09:53:07.650798+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:53:07.650798+0000","last_undegraded":"2023-08-25T09:53:07.650798+0000","last_fullsized":"2023-08-25T09:53:07.650798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:59.345933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.634006+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:53:07.634006+0000","last_peered":"2023-08-25T09:53:07.634006+0000","last_clean":"2023-08-25T09:53:07.634006+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:53:07.634006+0000","last_undegraded":"2023-08-25T09:53:07.634006+0000","last_fullsized":"2023-08-25T09:53:07.634006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:55.654458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.643927+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:53:07.643927+0000","last_peered":"2023-08-25T09:53:07.643927+0000","last_clean":"2023-08-25T09:53:07.643927+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:53:07.643927+0000","last_undegraded":"2023-08-25T09:53:07.643927+0000","last_fullsized":"2023-08-25T09:53:07.643927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:06.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.630628+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:53:07.630628+0000","last_peered":"2023-08-25T09:53:07.630628+0000","last_clean":"2023-08-25T09:53:07.630628+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:53:07.630628+0000","last_undegraded":"2023-08-25T09:53:07.630628+0000","last_fullsized":"2023-08-25T09:53:07.630628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:53.337656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633575+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:53:07.633575+0000","last_peered":"2023-08-25T09:53:07.633575+0000","last_clean":"2023-08-25T09:53:07.633575+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:53:07.633575+0000","last_undegraded":"2023-08-25T09:53:07.633575+0000","last_fullsized":"2023-08-25T09:53:07.633575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:17.959301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633711+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:53:07.633711+0000","last_peered":"2023-08-25T09:53:07.633711+0000","last_clean":"2023-08-25T09:53:07.633711+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:53:07.633711+0000","last_undegraded":"2023-08-25T09:53:07.633711+0000","last_fullsized":"2023-08-25T09:53:07.633711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:38.900516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.35699999999999998},"max":{"1min":1.4930000000000001,"5min":1.829,"15min":10.747},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.173,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.43099999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.855},"last":1.3080000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.6970000000000001,"5min":1.823,"15min":11.010999999999999},"last":1.23},{"interface":"front","average":{"1min":1.129,"5min":1.2589999999999999,"15min":1.1819999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.496,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":0.97699999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.206,"15min":1.147},"min":{"1min":0.86699999999999999,"5min":0.79400000000000004,"15min":0.50600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5529999999999999,"15min":1.619},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3180000000000001,"15min":1.2769999999999999},"min":{"1min":0.93999999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6779999999999999,"5min":1.873,"15min":1.98},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.258,"15min":1.2190000000000001},"min":{"1min":0.95599999999999996,"5min":0.79300000000000004,"15min":0.56799999999999995},"max":{"1min":1.506,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.393,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.69299999999999995},"max":{"1min":1.778,"5min":1.833,"15min":2.1030000000000002},"last":1.47}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.264,"15min":1.288},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.48599999999999999},"max":{"1min":1.732,"5min":1.845,"15min":7.1639999999999997},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.46300000000000002},"max":{"1min":1.7190000000000001,"5min":1.798,"15min":1.8779999999999999},"last":1.5269999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.52700000000000002},"max":{"1min":1.714,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.4450000000000001,"15min":1.429},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.64100000000000001},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":2.0739999999999998},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5580000000000001,"15min":1.518},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.68799999999999994},"max":{"1min":1.79,"5min":1.984,"15min":2.1720000000000002},"last":1.734},{"interface":"front","average":{"1min":1.448,"5min":1.4870000000000001,"15min":1.462},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6679999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1739999999999999,"15min":1.149},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.54400000000000004},"max":{"1min":1.591,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1950000000000001,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5509999999999999,"5min":1.645,"15min":5.5410000000000004},"last":1.4950000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.262,"15min":1.228},"min":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.194,"15min":1.1830000000000001},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6359999999999999,"5min":1.8100000000000001,"15min":2.3079999999999998},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1930000000000001,"15min":1.1619999999999999},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.534,"5min":1.696,"15min":1.708},"last":1.534},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.47299999999999998},"max":{"1min":1.7350000000000001,"5min":1.7889999999999999,"15min":13.193},"last":1.4490000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.369,"15min":1.319},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.68899999999999995},"max":{"1min":1.6519999999999999,"5min":1.911,"15min":5.6239999999999997},"last":1.369},{"interface":"front","average":{"1min":1.347,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":1.071,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8720000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.6639999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.3280000000000001},"min":{"1min":1.004,"5min":0.90500000000000003,"15min":0.42599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.484},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.419,"15min":1.3939999999999999},"min":{"1min":0.95999999999999996,"5min":0.78900000000000003,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":2.101,"15min":5.4699999999999998},"last":1.776}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3999999999999999,"15min":1.3440000000000001},"min":{"1min":0.94499999999999995,"5min":0.75,"15min":0.59599999999999997},"max":{"1min":1.875,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.4730000000000001},"min":{"1min":1.1639999999999999,"5min":1,"15min":0.59099999999999997},"max":{"1min":2.121,"5min":2.121,"15min":2.4420000000000002},"last":1.7050000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":1.157,"5min":0.89400000000000002,"15min":0.622},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":6.7709999999999999},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.548,"15min":1.5089999999999999},"min":{"1min":1.24,"5min":0.872,"15min":0.57399999999999995},"max":{"1min":2.181,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.802}]}]},{"osd":5,"up_from":13,"seq":55834575024,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2669999999999999,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.497},"max":{"1min":1.667,"5min":1.9770000000000001,"15min":2.081},"last":1.448},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.2210000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.65400000000000003},"max":{"1min":1.5309999999999999,"5min":1.6950000000000001,"15min":1.974},"last":1.3600000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.145,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.5569999999999999,"5min":1.6899999999999999,"15min":2.1779999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2010000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.76,"5min":1.76,"15min":2.1869999999999998},"last":1.623}]},{"osd":2,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.258},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.68400000000000005},"max":{"1min":1.909,"5min":1.909,"15min":2.3690000000000002},"last":1.298},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.873,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":1.6679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.236,"15min":1.28},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.53,"5min":1.7649999999999999,"15min":17.356999999999999},"last":1.53},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.343,"15min":1.3300000000000001},"min":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.696,"5min":1.7490000000000001,"15min":1.9650000000000001},"last":1.335}]},{"osd":4,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.55500000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.625},"last":1.038},{"interface":"front","average":{"1min":1.278,"5min":1.304,"15min":1.329},"min":{"1min":0.746,"5min":0.746,"15min":0.58799999999999997},"max":{"1min":1.556,"5min":1.7290000000000001,"15min":2.2789999999999999},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4450000000000001,"15min":1.4430000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.58899999999999997},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":2.4820000000000002},"last":1.399},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.516,"15min":1.4950000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.60399999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.5539999999999998},"last":1.2}]},{"osd":7,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.95499999999999996,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":1.6359999999999999,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.111},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5700000000000001,"15min":1.5780000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.63900000000000001},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.3700000000000001},"last":1.5780000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.274,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.624,"15min":0.34399999999999997},"max":{"1min":1.617,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.228,"5min":1.2529999999999999,"15min":1.258},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.434},"max":{"1min":2.3959999999999999,"5min":2.5150000000000001,"15min":7.367},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.159,"15min":1.1439999999999999},"min":{"1min":0.88,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.69499999999999995,"5min":0.56200000000000006,"15min":0.49299999999999999},"max":{"1min":1.7010000000000001,"5min":4.351,"15min":8.5850000000000009},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2010000000000001,"15min":1.208},"min":{"1min":0.89500000000000002,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":4.1669999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.266},"min":{"1min":0.92500000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"max":{"1min":1.899,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.103}]},{"osd":3,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.29,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.79400000000000004,"15min":0.61899999999999999},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":7.2560000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4059999999999999,"15min":1.385},"min":{"1min":1.0800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.137}]},{"osd":5,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.67100000000000004,"15min":0.55200000000000005},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":1.208},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.3400000000000001,"15min":1.3260000000000001},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.57099999999999995},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":8.2409999999999997},"last":1.177}]},{"osd":6,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4730000000000001,"15min":1.446},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":6.1980000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.298},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.6399999999999999,"15min":1.5920000000000001},"min":{"1min":1.0840000000000001,"5min":0.69799999999999995,"15min":0.67100000000000004},"max":{"1min":6.069,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.496,"15min":1.502},"min":{"1min":1.119,"5min":0.75700000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.5900000000000001},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.9900000000000002,"5min":3.831,"15min":3.831},"last":1.528}]}]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.4159999999999999,"5min":1.8180000000000001,"15min":1.8999999999999999},"last":1.026},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":0.998},"min":{"1min":0.71299999999999997,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":1.552,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.86799999999999999},"min":{"1min":0.66800000000000004,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.702},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.103,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.5740000000000001,"5min":1.631,"15min":2.3399999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.0589999999999999,"15min":0.98599999999999999},"min":{"1min":0.81799999999999995,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.919},"last":1.006},{"interface":"front","average":{"1min":1.268,"5min":1.2250000000000001,"15min":1.1539999999999999},"min":{"1min":0.94899999999999995,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.702,"5min":1.901,"15min":3.399},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3340000000000001,"15min":1.2490000000000001},"min":{"1min":1.0169999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.1949999999999998},"last":1.171},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.222,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6539999999999999,"5min":1.79,"15min":2.1840000000000002},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4079999999999999,"15min":1.355},"min":{"1min":1.1060000000000001,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":1.881,"5min":2.1930000000000001,"15min":2.407},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3779999999999999,"15min":1.3160000000000001},"min":{"1min":1.151,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.895},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3280000000000001,"15min":1.292},"min":{"1min":0.97999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.3450000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.49,"15min":1.427},"min":{"1min":1.1200000000000001,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.9830000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.498,"15min":1.4199999999999999},"min":{"1min":1.3009999999999999,"5min":0.95299999999999996,"15min":0.83699999999999997},"max":{"1min":1.9390000000000001,"5min":1.976,"15min":2.2989999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4570000000000001,"15min":1.3999999999999999},"min":{"1min":1.093,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.1509999999999998},"last":1.413}]}]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.2490000000000001,"5min":1.6930000000000001,"15min":2.383},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.7309999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97399999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3340000000000001,"5min":1.635,"15min":2.3180000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.391,"5min":1.704,"15min":2.8599999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":1.026},"min":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.214,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.127,"15min":1.1000000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.4430000000000001,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3180000000000001,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9219999999999999,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.288,"15min":1.286},"min":{"1min":0.77700000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6799999999999999,"5min":1.8,"15min":2.7639999999999998},"last":0.622}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2509999999999999,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6339999999999999,"5min":1.849,"15min":5.1269999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.341,"5min":1.373,"15min":1.363},"min":{"1min":0.878,"5min":0.625,"15min":0.625},"max":{"1min":1.7390000000000001,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.407,"15min":1.399},"min":{"1min":0.97799999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0059999999999998,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.389,"15min":1.4039999999999999},"min":{"1min":0.98799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6919999999999999,"5min":2.028,"15min":5.0599999999999996},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.381,"15min":1.3859999999999999},"min":{"1min":0.81699999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2,"15min":2.968},"last":0.753},{"interface":"front","average":{"1min":1.298,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.96,"15min":5.2990000000000004},"last":0.63800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.83099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.3240000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.504},{"interface":"front","average":{"1min":0.871,"5min":0.95199999999999996,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.333,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.749}]},{"osd":2,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95299999999999996,"15min":0.89400000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5940000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.036,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.474,"5min":1.6619999999999999,"15min":1.746},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.372},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.9450000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.04,"15min":1.02},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.657,"5min":1.657,"15min":1.764},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.175,"15min":1.1539999999999999},"min":{"1min":0.76200000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5209999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2949999999999999,"15min":1.286},"min":{"1min":0.85799999999999998,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0109999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.278,"15min":1.2509999999999999},"min":{"1min":0.70099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.3129999999999999,"15min":1.2549999999999999},"min":{"1min":0.86799999999999999,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":2.867,"15min":2.867},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.369,"15min":1.371},"min":{"1min":0.96899999999999997,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.9940000000000002},"last":1.419},{"interface":"front","average":{"1min":1.411,"5min":1.415,"15min":1.3759999999999999},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":2.036,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4419999999999999,"15min":1.419},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.79200000000000004},"max":{"1min":1.704,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.218},"last":1.3420000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.313},"max":{"1min":1.4450000000000001,"5min":2.2759999999999998,"15min":11.257},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.98499999999999999,"15min":0.96799999999999997},"min":{"1min":0.60199999999999998,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.4730000000000001,"5min":1.762,"15min":6.4260000000000002},"last":1.054}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.96099999999999997},"min":{"1min":0.72399999999999998,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.343,"5min":1.522,"15min":1.8620000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":1.048},"min":{"1min":0.62,"5min":0.374,"15min":0.374},"max":{"1min":1.401,"5min":1.714,"15min":3.4889999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.98499999999999999,"15min":0.93000000000000005},"min":{"1min":0.65900000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.464,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.873},{"interface":"front","average":{"1min":1.198,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.69199999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.629,"15min":2.3809999999999998},"last":1.016}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.294,"15min":1.2569999999999999},"min":{"1min":0.89800000000000002,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.724,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.097,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.88800000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.919,"5min":1.919,"15min":3.3769999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.2769999999999999},"min":{"1min":1.038,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6439999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.2909999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4219999999999999,"15min":1.3640000000000001},"min":{"1min":0.92400000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7370000000000001,"5min":2.056,"15min":2.6869999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3759999999999999,"15min":1.335},"min":{"1min":0.999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7709999999999999,"5min":1.863,"15min":2.331},"last":1.6879999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.706,"5min":1.962,"15min":2.6429999999999998},"last":1.544},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.389,"15min":1.3440000000000001},"min":{"1min":0.95699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.804,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.407}]}]}],"pool_statfs":[]}} 2023-08-25T09:53:10.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:53:10.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.411+0000 7f11850f3640 1 Processor -- start 2023-08-25T09:53:10.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.411+0000 7f11850f3640 1 -- start start 2023-08-25T09:53:10.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.413+0000 7f11850f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 0x7f11800ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.413+0000 7f11850f3640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11800ceed0 con 0x7f11800b2850 2023-08-25T09:53:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.413+0000 7f11850f3640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118015d3f0 con 0x7f118015aa10 2023-08-25T09:53:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.413+0000 7f11850f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1180159d10 con 0x7f11800ce5b0 2023-08-25T09:53:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.413+0000 7f117f59e640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f11800b2850 0x7f118006f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46222/0 (socket says 172.21.15.70:46222) 2023-08-25T09:53:10.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.413+0000 7f117f59e640 1 -- 172.21.15.70:0/713621592 learned_addr learned my addr 172.21.15.70:0/713621592 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117ed9d640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 0x7f11800ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536791333 0 0) 0x7f11800ceed0 con 0x7f11800b2850 2023-08-25T09:53:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003520 con 0x7f11800b2850 2023-08-25T09:53:10.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744734738 0 0) 0x7f1168003520 con 0x7f11800b2850 2023-08-25T09:53:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 msgr2=0x7f11800ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 0x7f11800ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 >> v1:172.21.15.155:6789/0 conn(0x7f118015aa10 legacy=0x7f11800b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11800b2d20 con 0x7f11800b2850 2023-08-25T09:53:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f11780019f0 con 0x7f11800b2850 2023-08-25T09:53:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 >> v1:172.21.15.70:6789/0 conn(0x7f11800b2850 legacy=0x7f118006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 0x7f11680039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168003f20 0x7f11680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1168004d00 0x7f1168005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180159d10 con 0x7f1168004d00 2023-08-25T09:53:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116800dc70 con 0x7f1168003f20 2023-08-25T09:53:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.414+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116800de30 con 0x7f11800ce5b0 2023-08-25T09:53:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117f59e640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1168004d00 0x7f1168005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117ed9d640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 0x7f11680039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117e59c640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168003f20 0x7f11680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117f59e640 1 -- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 msgr2=0x7f11680039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117f59e640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f11800ce5b0 0x7f11680039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117f59e640 1 -- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168003f20 msgr2=0x7f11680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117f59e640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168003f20 0x7f11680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.415+0000 7f117f59e640 1 -- 172.21.15.70:0/713621592 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168003520 con 0x7f1168004d00 2023-08-25T09:53:10.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.416+0000 7f117f59e640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1168004d00 0x7f1168005180 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f11800b2c50 tx=0x7f11680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d745f2c508676 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780072d0 con 0x7f1168004d00 2023-08-25T09:53:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117802f070 con 0x7f1168004d00 2023-08-25T09:53:10.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f117dd9b640 1 -- 172.21.15.70:0/713621592 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780076f0 con 0x7f1168004d00 2023-08-25T09:53:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f11850f3640 1 -- 172.21.15.70:0/713621592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1168004d00 msgr2=0x7f1168005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f11850f3640 1 --2- 172.21.15.70:0/713621592 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f1168004d00 0x7f1168005180 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f11800b2c50 tx=0x7f11680057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f117f59e640 1 -- 172.21.15.70:0/713621592 reap_dead start 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f11850f3640 1 -- 172.21.15.70:0/713621592 shutdown_connections 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f11850f3640 1 -- 172.21.15.70:0/713621592 >> 172.21.15.70:0/713621592 conn(0x7f1180069850 msgr2=0x7f1180069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f11850f3640 1 -- 172.21.15.70:0/713621592 shutdown_connections 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.417+0000 7f11850f3640 1 -- 172.21.15.70:0/713621592 wait complete. 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.419+0000 7f11850f3640 1 Processor -- start 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.419+0000 7f11850f3640 1 -- start start 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f11850f3640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168004d00 0x7f1180158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f11850f3640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f118015c0f0 0x7f11801555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f11850f3640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 0x7f1180155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f11850f3640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1180156050 con 0x7f118015c570 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f11850f3640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11801e5040 con 0x7f1168004d00 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f11850f3640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11801e4430 con 0x7f118015c0f0 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.420+0000 7f117f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 0x7f1180155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 0x7f1180155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54402/0 (socket says 172.21.15.70:54402) 2023-08-25T09:53:10.434 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 -- 172.21.15.70:0/939055746 learned_addr learned my addr 172.21.15.70:0/939055746 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117ed9d640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168004d00 0x7f1180158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117e59c640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f118015c0f0 0x7f11801555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 -- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f118015c0f0 msgr2=0x7f11801555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f118015c0f0 0x7f11801555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 -- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168004d00 msgr2=0x7f1180158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168004d00 0x7f1180158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 -- 172.21.15.70:0/939055746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11801e4710 con 0x7f118015c570 2023-08-25T09:53:10.435 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117ed9d640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168004d00 0x7f1180158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.421+0000 7f117f59e640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 0x7f1180155b10 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f117800a8f0 tx=0x7f1178001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.422+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178004d60 con 0x7f118015c570 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.422+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1178001250 con 0x7f118015c570 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.422+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11801e49f0 con 0x7f118015c570 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.422+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11780072d0 con 0x7f118015c570 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.422+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11801e4100 con 0x7f118015c570 2023-08-25T09:53:10.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.423+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f118015caa0 con 0x7f118015c570 2023-08-25T09:53:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.424+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1178031030 con 0x7f118015c570 2023-08-25T09:53:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.425+0000 7f11677fe640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f116c06a080 0x7f116c06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.425+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f11780a9dc0 con 0x7f118015c570 2023-08-25T09:53:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.425+0000 7f117ed9d640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f116c06a080 0x7f116c06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.426+0000 7f117ed9d640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f116c06a080 0x7f116c06c540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f118006c300 tx=0x7f1170008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:10.437 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.429+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1178075730 con 0x7f118015c570 2023-08-25T09:53:10.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.613+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f118015d990 con 0x7f116c06a080 2023-08-25T09:53:10.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.621+0000 7f11677fe640 1 -- 172.21.15.70:0/939055746 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68035 (secure 0 0 0) 0x7f1170009690 con 0x7f116c06a080 2023-08-25T09:53:10.623 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:10.623 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:53:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.623+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f116c06a080 msgr2=0x7f116c06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.623+0000 7f11850f3640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f116c06a080 0x7f116c06c540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f118006c300 tx=0x7f1170008040 comp rx=0 tx=0).stop 2023-08-25T09:53:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.623+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 msgr2=0x7f1180155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.623+0000 7f11850f3640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 0x7f1180155b10 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f117800a8f0 tx=0x7f1178001d80 comp rx=0 tx=0).stop 2023-08-25T09:53:10.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 shutdown_connections 2023-08-25T09:53:10.627 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f116c06a080 0x7f116c06c540 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f118015c570 0x7f1180155b10 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f118015c0f0 0x7f11801555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 --2- 172.21.15.70:0/939055746 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f1168004d00 0x7f1180158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 >> 172.21.15.70:0/939055746 conn(0x7f1180069850 msgr2=0x7f1180159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:10.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 shutdown_connections 2023-08-25T09:53:10.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.624+0000 7f11850f3640 1 -- 172.21.15.70:0/939055746 wait complete. 2023-08-25T09:53:10.643 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2023-08-25T09:53:09.615244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10818,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1551681,"statfs":{"total":1600000000,"available":1588922304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485516,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-130,"num_write_kb":-85085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002202"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":494,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.632435+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:53:07.632435+0000","last_peered":"2023-08-25T09:53:07.632435+0000","last_clean":"2023-08-25T09:53:07.632435+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:53:07.632435+0000","last_undegraded":"2023-08-25T09:53:07.632435+0000","last_fullsized":"2023-08-25T09:53:07.632435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:01.074556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.635287+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:53:07.635287+0000","last_peered":"2023-08-25T09:53:07.635287+0000","last_clean":"2023-08-25T09:53:07.635287+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:53:07.635287+0000","last_undegraded":"2023-08-25T09:53:07.635287+0000","last_fullsized":"2023-08-25T09:53:07.635287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:09.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":372,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.631062+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:53:07.631062+0000","last_peered":"2023-08-25T09:53:07.631062+0000","last_clean":"2023-08-25T09:53:07.631062+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:53:07.631062+0000","last_undegraded":"2023-08-25T09:53:07.631062+0000","last_fullsized":"2023-08-25T09:53:07.631062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:11.279207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":373,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.650798+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:53:07.650798+0000","last_peered":"2023-08-25T09:53:07.650798+0000","last_clean":"2023-08-25T09:53:07.650798+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:53:07.650798+0000","last_undegraded":"2023-08-25T09:53:07.650798+0000","last_fullsized":"2023-08-25T09:53:07.650798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:59.345933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.634006+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:53:07.634006+0000","last_peered":"2023-08-25T09:53:07.634006+0000","last_clean":"2023-08-25T09:53:07.634006+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:53:07.634006+0000","last_undegraded":"2023-08-25T09:53:07.634006+0000","last_fullsized":"2023-08-25T09:53:07.634006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:55.654458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.643927+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:53:07.643927+0000","last_peered":"2023-08-25T09:53:07.643927+0000","last_clean":"2023-08-25T09:53:07.643927+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:53:07.643927+0000","last_undegraded":"2023-08-25T09:53:07.643927+0000","last_fullsized":"2023-08-25T09:53:07.643927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:06.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.630628+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:53:07.630628+0000","last_peered":"2023-08-25T09:53:07.630628+0000","last_clean":"2023-08-25T09:53:07.630628+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:53:07.630628+0000","last_undegraded":"2023-08-25T09:53:07.630628+0000","last_fullsized":"2023-08-25T09:53:07.630628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:53.337656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633575+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:53:07.633575+0000","last_peered":"2023-08-25T09:53:07.633575+0000","last_clean":"2023-08-25T09:53:07.633575+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:53:07.633575+0000","last_undegraded":"2023-08-25T09:53:07.633575+0000","last_fullsized":"2023-08-25T09:53:07.633575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:17.959301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633711+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:53:07.633711+0000","last_peered":"2023-08-25T09:53:07.633711+0000","last_clean":"2023-08-25T09:53:07.633711+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:53:07.633711+0000","last_undegraded":"2023-08-25T09:53:07.633711+0000","last_fullsized":"2023-08-25T09:53:07.633711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:38.900516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.35699999999999998},"max":{"1min":1.4930000000000001,"5min":1.829,"15min":10.747},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.173,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.43099999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.855},"last":1.3080000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.6970000000000001,"5min":1.823,"15min":11.010999999999999},"last":1.23},{"interface":"front","average":{"1min":1.129,"5min":1.2589999999999999,"15min":1.1819999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.496,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":0.97699999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.206,"15min":1.147},"min":{"1min":0.86699999999999999,"5min":0.79400000000000004,"15min":0.50600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5529999999999999,"15min":1.619},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3180000000000001,"15min":1.2769999999999999},"min":{"1min":0.93999999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6779999999999999,"5min":1.873,"15min":1.98},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.258,"15min":1.2190000000000001},"min":{"1min":0.95599999999999996,"5min":0.79300000000000004,"15min":0.56799999999999995},"max":{"1min":1.506,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.393,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.69299999999999995},"max":{"1min":1.778,"5min":1.833,"15min":2.1030000000000002},"last":1.47}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.264,"15min":1.288},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.48599999999999999},"max":{"1min":1.732,"5min":1.845,"15min":7.1639999999999997},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.46300000000000002},"max":{"1min":1.7190000000000001,"5min":1.798,"15min":1.8779999999999999},"last":1.5269999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.52700000000000002},"max":{"1min":1.714,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.4450000000000001,"15min":1.429},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.64100000000000001},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":2.0739999999999998},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5580000000000001,"15min":1.518},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.68799999999999994},"max":{"1min":1.79,"5min":1.984,"15min":2.1720000000000002},"last":1.734},{"interface":"front","average":{"1min":1.448,"5min":1.4870000000000001,"15min":1.462},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6679999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1739999999999999,"15min":1.149},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.54400000000000004},"max":{"1min":1.591,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1950000000000001,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5509999999999999,"5min":1.645,"15min":5.5410000000000004},"last":1.4950000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.262,"15min":1.228},"min":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.194,"15min":1.1830000000000001},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6359999999999999,"5min":1.8100000000000001,"15min":2.3079999999999998},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1930000000000001,"15min":1.1619999999999999},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.534,"5min":1.696,"15min":1.708},"last":1.534},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.47299999999999998},"max":{"1min":1.7350000000000001,"5min":1.7889999999999999,"15min":13.193},"last":1.4490000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.369,"15min":1.319},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.68899999999999995},"max":{"1min":1.6519999999999999,"5min":1.911,"15min":5.6239999999999997},"last":1.369},{"interface":"front","average":{"1min":1.347,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":1.071,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8720000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.6639999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.3280000000000001},"min":{"1min":1.004,"5min":0.90500000000000003,"15min":0.42599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.484},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.419,"15min":1.3939999999999999},"min":{"1min":0.95999999999999996,"5min":0.78900000000000003,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":2.101,"15min":5.4699999999999998},"last":1.776}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3999999999999999,"15min":1.3440000000000001},"min":{"1min":0.94499999999999995,"5min":0.75,"15min":0.59599999999999997},"max":{"1min":1.875,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.4730000000000001},"min":{"1min":1.1639999999999999,"5min":1,"15min":0.59099999999999997},"max":{"1min":2.121,"5min":2.121,"15min":2.4420000000000002},"last":1.7050000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":1.157,"5min":0.89400000000000002,"15min":0.622},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":6.7709999999999999},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.548,"15min":1.5089999999999999},"min":{"1min":1.24,"5min":0.872,"15min":0.57399999999999995},"max":{"1min":2.181,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.802}]}]},{"osd":5,"up_from":13,"seq":55834575024,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2669999999999999,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.497},"max":{"1min":1.667,"5min":1.9770000000000001,"15min":2.081},"last":1.448},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.2210000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.65400000000000003},"max":{"1min":1.5309999999999999,"5min":1.6950000000000001,"15min":1.974},"last":1.3600000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.145,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.5569999999999999,"5min":1.6899999999999999,"15min":2.1779999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2010000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.76,"5min":1.76,"15min":2.1869999999999998},"last":1.623}]},{"osd":2,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.258},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.68400000000000005},"max":{"1min":1.909,"5min":1.909,"15min":2.3690000000000002},"last":1.298},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.873,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":1.6679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.236,"15min":1.28},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.53,"5min":1.7649999999999999,"15min":17.356999999999999},"last":1.53},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.343,"15min":1.3300000000000001},"min":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.696,"5min":1.7490000000000001,"15min":1.9650000000000001},"last":1.335}]},{"osd":4,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.55500000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.625},"last":1.038},{"interface":"front","average":{"1min":1.278,"5min":1.304,"15min":1.329},"min":{"1min":0.746,"5min":0.746,"15min":0.58799999999999997},"max":{"1min":1.556,"5min":1.7290000000000001,"15min":2.2789999999999999},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4450000000000001,"15min":1.4430000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.58899999999999997},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":2.4820000000000002},"last":1.399},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.516,"15min":1.4950000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.60399999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.5539999999999998},"last":1.2}]},{"osd":7,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.95499999999999996,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":1.6359999999999999,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.111},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5700000000000001,"15min":1.5780000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.63900000000000001},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.3700000000000001},"last":1.5780000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.274,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.624,"15min":0.34399999999999997},"max":{"1min":1.617,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.228,"5min":1.2529999999999999,"15min":1.258},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.434},"max":{"1min":2.3959999999999999,"5min":2.5150000000000001,"15min":7.367},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.159,"15min":1.1439999999999999},"min":{"1min":0.88,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.69499999999999995,"5min":0.56200000000000006,"15min":0.49299999999999999},"max":{"1min":1.7010000000000001,"5min":4.351,"15min":8.5850000000000009},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2010000000000001,"15min":1.208},"min":{"1min":0.89500000000000002,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":4.1669999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.266},"min":{"1min":0.92500000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"max":{"1min":1.899,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.103}]},{"osd":3,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.29,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.79400000000000004,"15min":0.61899999999999999},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":7.2560000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4059999999999999,"15min":1.385},"min":{"1min":1.0800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.137}]},{"osd":5,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.67100000000000004,"15min":0.55200000000000005},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":1.208},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.3400000000000001,"15min":1.3260000000000001},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.57099999999999995},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":8.2409999999999997},"last":1.177}]},{"osd":6,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4730000000000001,"15min":1.446},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":6.1980000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.298},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.6399999999999999,"15min":1.5920000000000001},"min":{"1min":1.0840000000000001,"5min":0.69799999999999995,"15min":0.67100000000000004},"max":{"1min":6.069,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.496,"15min":1.502},"min":{"1min":1.119,"5min":0.75700000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.5900000000000001},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.9900000000000002,"5min":3.831,"15min":3.831},"last":1.528}]}]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.4159999999999999,"5min":1.8180000000000001,"15min":1.8999999999999999},"last":1.026},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":0.998},"min":{"1min":0.71299999999999997,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":1.552,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.86799999999999999},"min":{"1min":0.66800000000000004,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.702},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.103,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.5740000000000001,"5min":1.631,"15min":2.3399999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.0589999999999999,"15min":0.98599999999999999},"min":{"1min":0.81799999999999995,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.919},"last":1.006},{"interface":"front","average":{"1min":1.268,"5min":1.2250000000000001,"15min":1.1539999999999999},"min":{"1min":0.94899999999999995,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.702,"5min":1.901,"15min":3.399},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3340000000000001,"15min":1.2490000000000001},"min":{"1min":1.0169999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.1949999999999998},"last":1.171},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.222,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6539999999999999,"5min":1.79,"15min":2.1840000000000002},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4079999999999999,"15min":1.355},"min":{"1min":1.1060000000000001,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":1.881,"5min":2.1930000000000001,"15min":2.407},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3779999999999999,"15min":1.3160000000000001},"min":{"1min":1.151,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.895},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3280000000000001,"15min":1.292},"min":{"1min":0.97999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.3450000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.49,"15min":1.427},"min":{"1min":1.1200000000000001,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.9830000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.498,"15min":1.4199999999999999},"min":{"1min":1.3009999999999999,"5min":0.95299999999999996,"15min":0.83699999999999997},"max":{"1min":1.9390000000000001,"5min":1.976,"15min":2.2989999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4570000000000001,"15min":1.3999999999999999},"min":{"1min":1.093,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.1509999999999998},"last":1.413}]}]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.2490000000000001,"5min":1.6930000000000001,"15min":2.383},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.7309999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97399999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3340000000000001,"5min":1.635,"15min":2.3180000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.391,"5min":1.704,"15min":2.8599999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":1.026},"min":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.214,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.127,"15min":1.1000000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.4430000000000001,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3180000000000001,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9219999999999999,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.288,"15min":1.286},"min":{"1min":0.77700000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6799999999999999,"5min":1.8,"15min":2.7639999999999998},"last":0.622}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2509999999999999,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6339999999999999,"5min":1.849,"15min":5.1269999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.341,"5min":1.373,"15min":1.363},"min":{"1min":0.878,"5min":0.625,"15min":0.625},"max":{"1min":1.7390000000000001,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.407,"15min":1.399},"min":{"1min":0.97799999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0059999999999998,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.389,"15min":1.4039999999999999},"min":{"1min":0.98799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6919999999999999,"5min":2.028,"15min":5.0599999999999996},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.381,"15min":1.3859999999999999},"min":{"1min":0.81699999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2,"15min":2.968},"last":0.753},{"interface":"front","average":{"1min":1.298,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.96,"15min":5.2990000000000004},"last":0.63800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.83099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.3240000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.504},{"interface":"front","average":{"1min":0.871,"5min":0.95199999999999996,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.333,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.749}]},{"osd":2,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95299999999999996,"15min":0.89400000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5940000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.036,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.474,"5min":1.6619999999999999,"15min":1.746},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.372},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.9450000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.04,"15min":1.02},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.657,"5min":1.657,"15min":1.764},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.175,"15min":1.1539999999999999},"min":{"1min":0.76200000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5209999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2949999999999999,"15min":1.286},"min":{"1min":0.85799999999999998,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0109999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.278,"15min":1.2509999999999999},"min":{"1min":0.70099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.3129999999999999,"15min":1.2549999999999999},"min":{"1min":0.86799999999999999,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":2.867,"15min":2.867},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.369,"15min":1.371},"min":{"1min":0.96899999999999997,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.9940000000000002},"last":1.419},{"interface":"front","average":{"1min":1.411,"5min":1.415,"15min":1.3759999999999999},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":2.036,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4419999999999999,"15min":1.419},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.79200000000000004},"max":{"1min":1.704,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.218},"last":1.3420000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.313},"max":{"1min":1.4450000000000001,"5min":2.2759999999999998,"15min":11.257},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.98499999999999999,"15min":0.96799999999999997},"min":{"1min":0.60199999999999998,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.4730000000000001,"5min":1.762,"15min":6.4260000000000002},"last":1.054}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.96099999999999997},"min":{"1min":0.72399999999999998,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.343,"5min":1.522,"15min":1.8620000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":1.048},"min":{"1min":0.62,"5min":0.374,"15min":0.374},"max":{"1min":1.401,"5min":1.714,"15min":3.4889999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.98499999999999999,"15min":0.93000000000000005},"min":{"1min":0.65900000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.464,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.873},{"interface":"front","average":{"1min":1.198,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.69199999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.629,"15min":2.3809999999999998},"last":1.016}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.294,"15min":1.2569999999999999},"min":{"1min":0.89800000000000002,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.724,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.097,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.88800000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.919,"5min":1.919,"15min":3.3769999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.2769999999999999},"min":{"1min":1.038,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6439999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.2909999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4219999999999999,"15min":1.3640000000000001},"min":{"1min":0.92400000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7370000000000001,"5min":2.056,"15min":2.6869999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3759999999999999,"15min":1.335},"min":{"1min":0.999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7709999999999999,"5min":1.863,"15min":2.331},"last":1.6879999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.706,"5min":1.962,"15min":2.6429999999999998},"last":1.544},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.389,"15min":1.3440000000000001},"min":{"1min":0.95699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.804,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.407}]}]}],"pool_statfs":[]}} 2023-08-25T09:53:10.646 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-25T09:53:10.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:53:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.855+0000 7f5bb3f42640 1 Processor -- start 2023-08-25T09:53:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.855+0000 7f5bb3f42640 1 -- start start 2023-08-25T09:53:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb3f42640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5bac15e830 0x7f5bac06ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb3f42640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f5bac06b2b0 0x7f5bac15c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb3f42640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac0b1b60 con 0x7f5bac15c6d0 2023-08-25T09:53:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb3f42640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac06bd80 con 0x7f5bac06b2b0 2023-08-25T09:53:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb3f42640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac06b7c0 con 0x7f5bac15e830 2023-08-25T09:53:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb24df640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5bac15c6d0 0x7f5bac0b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46226/0 (socket says 172.21.15.70:46226) 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb24df640 1 -- 172.21.15.70:0/3158014631 learned_addr learned my addr 172.21.15.70:0/3158014631 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.857+0000 7f5bb14dd640 1 --2- 172.21.15.70:0/3158014631 >> v2:172.21.15.70:3300/0 conn(0x7f5bac06b2b0 0x7f5bac15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.858+0000 7f5bb1cde640 1 --2- 172.21.15.70:0/3158014631 >> v2:172.21.15.155:3300/0 conn(0x7f5bac15e830 0x7f5bac06ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.858+0000 7f5bb0cdc640 1 -- 172.21.15.70:0/3158014631 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3743619629 0 0) 0x7f5bac0b1b60 con 0x7f5bac15c6d0 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.858+0000 7f5bb14dd640 1 -- 172.21.15.70:0/3158014631 >> v2:172.21.15.155:3300/0 conn(0x7f5bac15e830 msgr2=0x7f5bac06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.858+0000 7f5bb14dd640 1 --2- 172.21.15.70:0/3158014631 >> v2:172.21.15.155:3300/0 conn(0x7f5bac15e830 0x7f5bac06ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.858+0000 7f5bb14dd640 1 -- 172.21.15.70:0/3158014631 >> v1:172.21.15.70:6789/0 conn(0x7f5bac15c6d0 legacy=0x7f5bac0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.858+0000 7f5bb14dd640 1 -- 172.21.15.70:0/3158014631 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac06baa0 con 0x7f5bac06b2b0 2023-08-25T09:53:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.859+0000 7f5bb14dd640 1 --2- 172.21.15.70:0/3158014631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac06b2b0 0x7f5bac15c190 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f5bac06bf10 tx=0x7f5b9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39d4dae6a3c8bf7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.859+0000 7f5bb0cdc640 1 -- 172.21.15.70:0/3158014631 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c031830 con 0x7f5bac06b2b0 2023-08-25T09:53:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.859+0000 7f5bb0cdc640 1 -- 172.21.15.70:0/3158014631 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c031990 con 0x7f5bac06b2b0 2023-08-25T09:53:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.859+0000 7f5bb0cdc640 1 -- 172.21.15.70:0/3158014631 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c031d20 con 0x7f5bac06b2b0 2023-08-25T09:53:10.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 -- 172.21.15.70:0/3158014631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac06b2b0 msgr2=0x7f5bac15c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.866 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/3158014631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac06b2b0 0x7f5bac15c190 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f5bac06bf10 tx=0x7f5b9c02b8d0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 -- 172.21.15.70:0/3158014631 shutdown_connections 2023-08-25T09:53:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/3158014631 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac06b2b0 0x7f5bac15c190 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/3158014631 >> v2:172.21.15.155:3300/0 conn(0x7f5bac15e830 0x7f5bac06ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 -- 172.21.15.70:0/3158014631 >> 172.21.15.70:0/3158014631 conn(0x7f5bac069850 msgr2=0x7f5bac069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 -- 172.21.15.70:0/3158014631 shutdown_connections 2023-08-25T09:53:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.860+0000 7f5bb3f42640 1 -- 172.21.15.70:0/3158014631 wait complete. 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.862+0000 7f5bb3f42640 1 Processor -- start 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.862+0000 7f5bb3f42640 1 -- start start 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb3f42640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bac06b2b0 0x7f5bac157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb3f42640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 0x7f5bac06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb3f42640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bac15e830 0x7f5bac1f1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb3f42640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0ac870 con 0x7f5bac15c6d0 2023-08-25T09:53:10.868 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb3f42640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0aca80 con 0x7f5bac15e830 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb3f42640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0acd60 con 0x7f5bac06b2b0 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 0x7f5bac06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 0x7f5bac06f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54418/0 (socket says 172.21.15.70:54418) 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 -- 172.21.15.70:0/423764549 learned_addr learned my addr 172.21.15.70:0/423764549 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb24df640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bac15e830 0x7f5bac1f1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb1cde640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bac06b2b0 0x7f5bac157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 -- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bac06b2b0 msgr2=0x7f5bac157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bac06b2b0 0x7f5bac157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 -- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bac15e830 msgr2=0x7f5bac1f1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bac15e830 0x7f5bac1f1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb14dd640 1 -- 172.21.15.70:0/423764549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c009400 con 0x7f5bac15c6d0 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.863+0000 7f5bb24df640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bac15e830 0x7f5bac1f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.864+0000 7f5bb14dd640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 0x7f5bac06f6f0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009890 tx=0x7f5b9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.864+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c03ba70 con 0x7f5bac15c6d0 2023-08-25T09:53:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.864+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c03bbf0 con 0x7f5bac15c6d0 2023-08-25T09:53:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.864+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c045400 con 0x7f5bac15c6d0 2023-08-25T09:53:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.864+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac0ad040 con 0x7f5bac15c6d0 2023-08-25T09:53:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.864+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac157fc0 con 0x7f5bac15c6d0 2023-08-25T09:53:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.865+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac0ad560 con 0x7f5bac15c6d0 2023-08-25T09:53:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.866+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5b9c04e440 con 0x7f5bac15c6d0 2023-08-25T09:53:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.867+0000 7f5ba27fc640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b9006a050 0x7f5b9006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:10.872 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.867+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5b9c0aa5d0 con 0x7f5bac15c6d0 2023-08-25T09:53:10.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.867+0000 7f5bb1cde640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b9006a050 0x7f5b9006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:10.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.868+0000 7f5bb1cde640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b9006a050 0x7f5b9006c510 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8006fd0 tx=0x7f5ba8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:10.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:10.872+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5bac0ad560 con 0x7f5bac15c6d0 2023-08-25T09:53:11.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.058+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bac0acbf0 con 0x7f5b9006a050 2023-08-25T09:53:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.062+0000 7f5ba27fc640 1 -- 172.21.15.70:0/423764549 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68035 (secure 0 0 0) 0x7f5bac0acbf0 con 0x7f5b9006a050 2023-08-25T09:53:11.065 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:11.066 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:53:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b9006a050 msgr2=0x7f5b9006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b9006a050 0x7f5b9006c510 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8006fd0 tx=0x7f5ba8008040 comp rx=0 tx=0).stop 2023-08-25T09:53:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 msgr2=0x7f5bac06f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 0x7f5bac06f6f0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009890 tx=0x7f5b9c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 shutdown_connections 2023-08-25T09:53:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5b9006a050 0x7f5b9006c510 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5bac15e830 0x7f5bac1f1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5bac15c6d0 0x7f5bac06f6f0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 --2- 172.21.15.70:0/423764549 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5bac06b2b0 0x7f5bac157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 >> 172.21.15.70:0/423764549 conn(0x7f5bac069850 msgr2=0x7f5bac15a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 shutdown_connections 2023-08-25T09:53:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.065+0000 7f5bb3f42640 1 -- 172.21.15.70:0/423764549 wait complete. 2023-08-25T09:53:11.086 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":632,"stamp":"2023-08-25T09:53:09.615244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10818,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1551681,"statfs":{"total":1600000000,"available":1588922304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485516,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-130,"num_write_kb":-85085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002202"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":494,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.632435+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:53:07.632435+0000","last_peered":"2023-08-25T09:53:07.632435+0000","last_clean":"2023-08-25T09:53:07.632435+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:53:07.632435+0000","last_undegraded":"2023-08-25T09:53:07.632435+0000","last_fullsized":"2023-08-25T09:53:07.632435+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:25:01.074556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.635287+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:53:07.635287+0000","last_peered":"2023-08-25T09:53:07.635287+0000","last_clean":"2023-08-25T09:53:07.635287+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:53:07.635287+0000","last_undegraded":"2023-08-25T09:53:07.635287+0000","last_fullsized":"2023-08-25T09:53:07.635287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:09.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":372,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.631062+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:53:07.631062+0000","last_peered":"2023-08-25T09:53:07.631062+0000","last_clean":"2023-08-25T09:53:07.631062+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:53:07.631062+0000","last_undegraded":"2023-08-25T09:53:07.631062+0000","last_fullsized":"2023-08-25T09:53:07.631062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:11.279207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":373,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.650798+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:53:07.650798+0000","last_peered":"2023-08-25T09:53:07.650798+0000","last_clean":"2023-08-25T09:53:07.650798+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:53:07.650798+0000","last_undegraded":"2023-08-25T09:53:07.650798+0000","last_fullsized":"2023-08-25T09:53:07.650798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:59.345933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.634006+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:53:07.634006+0000","last_peered":"2023-08-25T09:53:07.634006+0000","last_clean":"2023-08-25T09:53:07.634006+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:53:07.634006+0000","last_undegraded":"2023-08-25T09:53:07.634006+0000","last_fullsized":"2023-08-25T09:53:07.634006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:55.654458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.643927+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:53:07.643927+0000","last_peered":"2023-08-25T09:53:07.643927+0000","last_clean":"2023-08-25T09:53:07.643927+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:53:07.643927+0000","last_undegraded":"2023-08-25T09:53:07.643927+0000","last_fullsized":"2023-08-25T09:53:07.643927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:06.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.630628+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:53:07.630628+0000","last_peered":"2023-08-25T09:53:07.630628+0000","last_clean":"2023-08-25T09:53:07.630628+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:53:07.630628+0000","last_undegraded":"2023-08-25T09:53:07.630628+0000","last_fullsized":"2023-08-25T09:53:07.630628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:53.337656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633575+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:53:07.633575+0000","last_peered":"2023-08-25T09:53:07.633575+0000","last_clean":"2023-08-25T09:53:07.633575+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:53:07.633575+0000","last_undegraded":"2023-08-25T09:53:07.633575+0000","last_fullsized":"2023-08-25T09:53:07.633575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:17.959301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633711+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:53:07.633711+0000","last_peered":"2023-08-25T09:53:07.633711+0000","last_clean":"2023-08-25T09:53:07.633711+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:53:07.633711+0000","last_undegraded":"2023-08-25T09:53:07.633711+0000","last_fullsized":"2023-08-25T09:53:07.633711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:38.900516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.169,"15min":1.1279999999999999},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.35699999999999998},"max":{"1min":1.4930000000000001,"5min":1.829,"15min":10.747},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.169,"5min":1.173,"15min":1.109},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.43099999999999999},"max":{"1min":1.669,"5min":1.669,"15min":1.855},"last":1.3080000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.3080000000000001,"15min":1.246},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.6970000000000001,"5min":1.823,"15min":11.010999999999999},"last":1.23},{"interface":"front","average":{"1min":1.129,"5min":1.2589999999999999,"15min":1.1819999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.496,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":0.97699999999999998}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.206,"15min":1.147},"min":{"1min":0.86699999999999999,"5min":0.79400000000000004,"15min":0.50600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5529999999999999,"15min":1.619},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.3180000000000001,"15min":1.2769999999999999},"min":{"1min":0.93999999999999995,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.6779999999999999,"5min":1.873,"15min":1.98},"last":0.89200000000000002}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.258,"15min":1.2190000000000001},"min":{"1min":0.95599999999999996,"5min":0.79300000000000004,"15min":0.56799999999999995},"max":{"1min":1.506,"5min":1.6759999999999999,"15min":1.7110000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.393,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.69299999999999995},"max":{"1min":1.778,"5min":1.833,"15min":2.1030000000000002},"last":1.47}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.264,"15min":1.288},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.48599999999999999},"max":{"1min":1.732,"5min":1.845,"15min":7.1639999999999997},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3169999999999999,"15min":1.3240000000000001},"min":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.46300000000000002},"max":{"1min":1.7190000000000001,"5min":1.798,"15min":1.8779999999999999},"last":1.5269999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.448,"15min":1.4159999999999999},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.52700000000000002},"max":{"1min":1.714,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.4450000000000001,"15min":1.429},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.64100000000000001},"max":{"1min":1.6890000000000001,"5min":1.907,"15min":2.0739999999999998},"last":1.5880000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5580000000000001,"15min":1.518},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.68799999999999994},"max":{"1min":1.79,"5min":1.984,"15min":2.1720000000000002},"last":1.734},{"interface":"front","average":{"1min":1.448,"5min":1.4870000000000001,"15min":1.462},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.61399999999999999},"max":{"1min":1.7689999999999999,"5min":2.3220000000000001,"15min":2.3220000000000001},"last":1.6679999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1739999999999999,"15min":1.149},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.54400000000000004},"max":{"1min":1.591,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.1759999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1950000000000001,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5509999999999999,"5min":1.645,"15min":5.5410000000000004},"last":1.4950000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.262,"15min":1.228},"min":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.139,"5min":1.194,"15min":1.1830000000000001},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6359999999999999,"5min":1.8100000000000001,"15min":2.3079999999999998},"last":1.0329999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1930000000000001,"15min":1.1619999999999999},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.534,"5min":1.696,"15min":1.708},"last":1.534},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.47299999999999998},"max":{"1min":1.7350000000000001,"5min":1.7889999999999999,"15min":13.193},"last":1.4490000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.369,"15min":1.319},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.68899999999999995},"max":{"1min":1.6519999999999999,"5min":1.911,"15min":5.6239999999999997},"last":1.369},{"interface":"front","average":{"1min":1.347,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":1.071,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8720000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.6639999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.3280000000000001},"min":{"1min":1.004,"5min":0.90500000000000003,"15min":0.42599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.484},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.482,"5min":1.419,"15min":1.3939999999999999},"min":{"1min":0.95999999999999996,"5min":0.78900000000000003,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":2.101,"15min":5.4699999999999998},"last":1.776}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3999999999999999,"15min":1.3440000000000001},"min":{"1min":0.94499999999999995,"5min":0.75,"15min":0.59599999999999997},"max":{"1min":1.875,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.7390000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.4730000000000001},"min":{"1min":1.1639999999999999,"5min":1,"15min":0.59099999999999997},"max":{"1min":2.121,"5min":2.121,"15min":2.4420000000000002},"last":1.7050000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":1.157,"5min":0.89400000000000002,"15min":0.622},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":6.7709999999999999},"last":1.8839999999999999},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.548,"15min":1.5089999999999999},"min":{"1min":1.24,"5min":0.872,"15min":0.57399999999999995},"max":{"1min":2.181,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.802}]}]},{"osd":5,"up_from":13,"seq":55834575024,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2669999999999999,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.497},"max":{"1min":1.667,"5min":1.9770000000000001,"15min":2.081},"last":1.448},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.2210000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.65400000000000003},"max":{"1min":1.5309999999999999,"5min":1.6950000000000001,"15min":1.974},"last":1.3600000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.145,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.5569999999999999,"5min":1.6899999999999999,"15min":2.1779999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2010000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.76,"5min":1.76,"15min":2.1869999999999998},"last":1.623}]},{"osd":2,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.258},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.68400000000000005},"max":{"1min":1.909,"5min":1.909,"15min":2.3690000000000002},"last":1.298},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.873,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":1.6679999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.236,"15min":1.28},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.53,"5min":1.7649999999999999,"15min":17.356999999999999},"last":1.53},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.343,"15min":1.3300000000000001},"min":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.696,"5min":1.7490000000000001,"15min":1.9650000000000001},"last":1.335}]},{"osd":4,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.55500000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.625},"last":1.038},{"interface":"front","average":{"1min":1.278,"5min":1.304,"15min":1.329},"min":{"1min":0.746,"5min":0.746,"15min":0.58799999999999997},"max":{"1min":1.556,"5min":1.7290000000000001,"15min":2.2789999999999999},"last":1.2509999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4450000000000001,"15min":1.4430000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.58899999999999997},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":2.4820000000000002},"last":1.399},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.516,"15min":1.4950000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.60399999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.5539999999999998},"last":1.2}]},{"osd":7,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.95499999999999996,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":1.6359999999999999,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.111},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5700000000000001,"15min":1.5780000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.63900000000000001},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.3700000000000001},"last":1.5780000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.274,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.624,"15min":0.34399999999999997},"max":{"1min":1.617,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.228,"5min":1.2529999999999999,"15min":1.258},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.434},"max":{"1min":2.3959999999999999,"5min":2.5150000000000001,"15min":7.367},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.159,"15min":1.1439999999999999},"min":{"1min":0.88,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.248,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.69499999999999995,"5min":0.56200000000000006,"15min":0.49299999999999999},"max":{"1min":1.7010000000000001,"5min":4.351,"15min":8.5850000000000009},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2010000000000001,"15min":1.208},"min":{"1min":0.89500000000000002,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":4.1669999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.266},"min":{"1min":0.92500000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"max":{"1min":1.899,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.103}]},{"osd":3,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.29,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.79400000000000004,"15min":0.61899999999999999},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":7.2560000000000002},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4059999999999999,"15min":1.385},"min":{"1min":1.0800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.137}]},{"osd":5,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.67100000000000004,"15min":0.55200000000000005},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":1.208},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.3400000000000001,"15min":1.3260000000000001},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.57099999999999995},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":8.2409999999999997},"last":1.177}]},{"osd":6,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4730000000000001,"15min":1.446},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":6.1980000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.298},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.6399999999999999,"15min":1.5920000000000001},"min":{"1min":1.0840000000000001,"5min":0.69799999999999995,"15min":0.67100000000000004},"max":{"1min":6.069,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.4530000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.496,"15min":1.502},"min":{"1min":1.119,"5min":0.75700000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.5900000000000001},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.9900000000000002,"5min":3.831,"15min":3.831},"last":1.528}]}]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.4159999999999999,"5min":1.8180000000000001,"15min":1.8999999999999999},"last":1.026},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":0.998},"min":{"1min":0.71299999999999997,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":1.552,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.86799999999999999},"min":{"1min":0.66800000000000004,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.702},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.103,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.5740000000000001,"5min":1.631,"15min":2.3399999999999999},"last":0.747}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.0589999999999999,"15min":0.98599999999999999},"min":{"1min":0.81799999999999995,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.919},"last":1.006},{"interface":"front","average":{"1min":1.268,"5min":1.2250000000000001,"15min":1.1539999999999999},"min":{"1min":0.94899999999999995,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.702,"5min":1.901,"15min":3.399},"last":1.2370000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3340000000000001,"15min":1.2490000000000001},"min":{"1min":1.0169999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.1949999999999998},"last":1.171},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.222,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6539999999999999,"5min":1.79,"15min":2.1840000000000002},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4079999999999999,"15min":1.355},"min":{"1min":1.1060000000000001,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":1.881,"5min":2.1930000000000001,"15min":2.407},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3779999999999999,"15min":1.3160000000000001},"min":{"1min":1.151,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.895},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3280000000000001,"15min":1.292},"min":{"1min":0.97999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.3450000000000002},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.49,"15min":1.427},"min":{"1min":1.1200000000000001,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.9830000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.498,"15min":1.4199999999999999},"min":{"1min":1.3009999999999999,"5min":0.95299999999999996,"15min":0.83699999999999997},"max":{"1min":1.9390000000000001,"5min":1.976,"15min":2.2989999999999999},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4570000000000001,"15min":1.3999999999999999},"min":{"1min":1.093,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.1509999999999998},"last":1.413}]}]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.2490000000000001,"5min":1.6930000000000001,"15min":2.383},"last":0.60299999999999998},{"interface":"front","average":{"1min":1.018,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.7309999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97399999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3340000000000001,"5min":1.635,"15min":2.3180000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.391,"5min":1.704,"15min":2.8599999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":1.026},"min":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.214,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.127,"15min":1.1000000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.4430000000000001,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3180000000000001,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9219999999999999,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.288,"15min":1.286},"min":{"1min":0.77700000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6799999999999999,"5min":1.8,"15min":2.7639999999999998},"last":0.622}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2509999999999999,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6339999999999999,"5min":1.849,"15min":5.1269999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.341,"5min":1.373,"15min":1.363},"min":{"1min":0.878,"5min":0.625,"15min":0.625},"max":{"1min":1.7390000000000001,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":0.77800000000000002}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.407,"15min":1.399},"min":{"1min":0.97799999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0059999999999998,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.389,"15min":1.4039999999999999},"min":{"1min":0.98799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6919999999999999,"5min":2.028,"15min":5.0599999999999996},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.381,"15min":1.3859999999999999},"min":{"1min":0.81699999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2,"15min":2.968},"last":0.753},{"interface":"front","average":{"1min":1.298,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.96,"15min":5.2990000000000004},"last":0.63800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.83099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.3240000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.504},{"interface":"front","average":{"1min":0.871,"5min":0.95199999999999996,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.333,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.749}]},{"osd":2,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95299999999999996,"15min":0.89400000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5940000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.036,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.474,"5min":1.6619999999999999,"15min":1.746},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.372},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.9450000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":1.05,"5min":1.04,"15min":1.02},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.657,"5min":1.657,"15min":1.764},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.175,"15min":1.1539999999999999},"min":{"1min":0.76200000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5209999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2949999999999999,"15min":1.286},"min":{"1min":0.85799999999999998,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.0109999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.278,"15min":1.2509999999999999},"min":{"1min":0.70099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.2350000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.3129999999999999,"15min":1.2549999999999999},"min":{"1min":0.86799999999999999,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":2.867,"15min":2.867},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.369,"15min":1.371},"min":{"1min":0.96899999999999997,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.9940000000000002},"last":1.419},{"interface":"front","average":{"1min":1.411,"5min":1.415,"15min":1.3759999999999999},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":2.036,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.0629999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4419999999999999,"15min":1.419},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.79200000000000004},"max":{"1min":1.704,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.218},"last":1.3420000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.88100000000000001,"15min":0.88500000000000001},"min":{"1min":0.45000000000000001,"5min":0.41499999999999998,"15min":0.313},"max":{"1min":1.4450000000000001,"5min":2.2759999999999998,"15min":11.257},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0580000000000001,"5min":0.98499999999999999,"15min":0.96799999999999997},"min":{"1min":0.60199999999999998,"5min":0.39100000000000001,"15min":0.32800000000000001},"max":{"1min":1.4730000000000001,"5min":1.762,"15min":6.4260000000000002},"last":1.054}]},{"osd":2,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.96099999999999997},"min":{"1min":0.72399999999999998,"5min":0.44,"15min":0.36199999999999999},"max":{"1min":1.343,"5min":1.522,"15min":1.8620000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.077,"15min":1.048},"min":{"1min":0.62,"5min":0.374,"15min":0.374},"max":{"1min":1.401,"5min":1.714,"15min":3.4889999999999999},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.98499999999999999,"15min":0.93000000000000005},"min":{"1min":0.65900000000000003,"5min":0.41599999999999998,"15min":0.40699999999999997},"max":{"1min":1.464,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.873},{"interface":"front","average":{"1min":1.198,"5min":1.1479999999999999,"15min":1.1200000000000001},"min":{"1min":0.69199999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5009999999999999,"5min":1.629,"15min":2.3809999999999998},"last":1.016}]},{"osd":4,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.294,"15min":1.2569999999999999},"min":{"1min":0.89800000000000002,"5min":0.62,"15min":0.59999999999999998},"max":{"1min":1.724,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.097,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8120000000000001,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.4770000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.3320000000000001,"15min":1.3100000000000001},"min":{"1min":0.88800000000000001,"5min":0.623,"15min":0.623},"max":{"1min":1.919,"5min":1.919,"15min":3.3769999999999998},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.284,"15min":1.2769999999999999},"min":{"1min":1.038,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6439999999999999,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.2909999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.4219999999999999,"15min":1.3640000000000001},"min":{"1min":0.92400000000000004,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7370000000000001,"5min":2.056,"15min":2.6869999999999998},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.3759999999999999,"15min":1.335},"min":{"1min":0.999,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.7709999999999999,"5min":1.863,"15min":2.331},"last":1.6879999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:52:08 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4370000000000001,"15min":1.4079999999999999},"min":{"1min":0.82299999999999995,"5min":0.77800000000000002,"15min":0.71099999999999997},"max":{"1min":1.706,"5min":1.962,"15min":2.6429999999999998},"last":1.544},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.389,"15min":1.3440000000000001},"min":{"1min":0.95699999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.804,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.407}]}]}],"pool_statfs":[]}} 2023-08-25T09:53:11.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T09:53:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.299+0000 7f10a7c87640 1 Processor -- start 2023-08-25T09:53:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.299+0000 7f10a7c87640 1 -- start start 2023-08-25T09:53:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.300+0000 7f10a7c87640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f10a015aa10 0x7f10a00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.300+0000 7f10a7c87640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00b2ad0 con 0x7f10a00ce5b0 2023-08-25T09:53:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.300+0000 7f10a7c87640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a015d3f0 con 0x7f10a00b2680 2023-08-25T09:53:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.300+0000 7f10a7c87640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a0159d10 con 0x7f10a015aa10 2023-08-25T09:53:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a5a23640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f10a00ce5b0 0x7f10a00ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46232/0 (socket says 172.21.15.70:46232) 2023-08-25T09:53:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a5a23640 1 -- 172.21.15.70:0/790628923 learned_addr learned my addr 172.21.15.70:0/790628923 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a5222640 1 --2- 172.21.15.70:0/790628923 >> v2:172.21.15.70:3300/0 conn(0x7f10a015aa10 0x7f10a00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a4a21640 1 -- 172.21.15.70:0/790628923 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621276380 0 0) 0x7f10a00b2ad0 con 0x7f10a00ce5b0 2023-08-25T09:53:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a4a21640 1 -- 172.21.15.70:0/790628923 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f108c003520 con 0x7f10a00ce5b0 2023-08-25T09:53:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a5222640 1 -- 172.21.15.70:0/790628923 >> v1:172.21.15.155:6789/0 conn(0x7f10a00b2680 legacy=0x7f10a006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a5222640 1 -- 172.21.15.70:0/790628923 >> v1:172.21.15.70:6789/0 conn(0x7f10a00ce5b0 legacy=0x7f10a00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.301+0000 7f10a5222640 1 -- 172.21.15.70:0/790628923 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00b2d20 con 0x7f10a015aa10 2023-08-25T09:53:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.302+0000 7f10a5222640 1 --2- 172.21.15.70:0/790628923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a00b2140 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f10900097d0 tx=0x7f109002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fbe09a7ec0c405 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a4a21640 1 -- 172.21.15.70:0/790628923 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090031780 con 0x7f10a015aa10 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a4a21640 1 -- 172.21.15.70:0/790628923 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10900318e0 con 0x7f10a015aa10 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a4a21640 1 -- 172.21.15.70:0/790628923 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1090031c70 con 0x7f10a015aa10 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 -- 172.21.15.70:0/790628923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 msgr2=0x7f10a00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 --2- 172.21.15.70:0/790628923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a00b2140 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f10900097d0 tx=0x7f109002b820 comp rx=0 tx=0).stop 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 -- 172.21.15.70:0/790628923 shutdown_connections 2023-08-25T09:53:11.309 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 --2- 172.21.15.70:0/790628923 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a00b2140 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 -- 172.21.15.70:0/790628923 >> 172.21.15.70:0/790628923 conn(0x7f10a0069850 msgr2=0x7f10a0069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 -- 172.21.15.70:0/790628923 shutdown_connections 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.303+0000 7f10a7c87640 1 -- 172.21.15.70:0/790628923 wait complete. 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.305+0000 7f10a7c87640 1 Processor -- start 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.305+0000 7f10a7c87640 1 -- start start 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.306+0000 7f10a7c87640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a00b2680 0x7f10a01f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.306+0000 7f10a7c87640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a00ce5b0 0x7f10a014e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.306+0000 7f10a7c87640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a006ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.306+0000 7f10a7c87640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01f4690 con 0x7f10a015aa10 2023-08-25T09:53:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.306+0000 7f10a7c87640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a01519f0 con 0x7f10a00ce5b0 2023-08-25T09:53:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.306+0000 7f10a7c87640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a014ee20 con 0x7f10a00b2680 2023-08-25T09:53:11.312 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a006ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a006ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54428/0 (socket says 172.21.15.70:54428) 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 -- 172.21.15.70:0/1430497564 learned_addr learned my addr 172.21.15.70:0/1430497564 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a5a23640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a00b2680 0x7f10a01f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a5222640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a00ce5b0 0x7f10a014e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 -- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a00b2680 msgr2=0x7f10a01f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a00b2680 0x7f10a01f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 -- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a00ce5b0 msgr2=0x7f10a014e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a00ce5b0 0x7f10a014e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 -- 172.21.15.70:0/1430497564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090009400 con 0x7f10a015aa10 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.307+0000 7f10a6224640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a006ad70 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f109800c780 tx=0x7f109800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.308+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098004b60 con 0x7f10a015aa10 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.308+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109800ce90 con 0x7f10a015aa10 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.308+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109800a850 con 0x7f10a015aa10 2023-08-25T09:53:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.308+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a01f0710 con 0x7f10a015aa10 2023-08-25T09:53:11.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.308+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a01f0cd0 con 0x7f10a015aa10 2023-08-25T09:53:11.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.309+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a004e420 con 0x7f10a015aa10 2023-08-25T09:53:11.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.310+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f109800a9b0 con 0x7f10a015aa10 2023-08-25T09:53:11.315 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.311+0000 7f10967fc640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a050 0x7f108406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:11.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.311+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f10980884e0 con 0x7f10a015aa10 2023-08-25T09:53:11.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.311+0000 7f10a5a23640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a050 0x7f108406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:11.316 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.312+0000 7f10a5a23640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a050 0x7f108406c510 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f109c0011e0 tx=0x7f109c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:11.318 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.316+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1098053e50 con 0x7f10a015aa10 2023-08-25T09:53:11.507 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.505+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10a00cae50 con 0x7f10a015aa10 2023-08-25T09:53:11.510 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.507+0000 7f10967fc640 1 -- 172.21.15.70:0/1430497564 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) v1 ==== 74+0+14792 (secure 0 0 0) 0x7f1098057470 con 0x7f10a015aa10 2023-08-25T09:53:11.510 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:11.510 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":205,"fsid":"64dd9a7e-0cdf-46fb-8c89-fab611563c75","created":"2023-08-25T09:38:32.416535+0000","modified":"2023-08-25T09:53:07.621588+0000","last_up_change":"2023-08-25T09:38:38.906428+0000","last_in_change":"2023-08-25T09:38:35.884043+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T09:38:39.384505+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T09:38:45.046709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":4.9800000190734863,"score_stable":4.9800000190734863,"optimal_score":0.87999999523162842,"raw_score_acting":4.380000114440918,"raw_score_stable":4.380000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6809","nonce":3728228047}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6811","nonce":3728228047}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6816","nonce":3728228047}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":3728228047},{"type":"v1","addr":"172.21.15.70:6813","nonce":3728228047}]},"public_addr":"172.21.15.70:6809/3728228047","cluster_addr":"172.21.15.70:6811/3728228047","heartbeat_back_addr":"172.21.15.70:6816/3728228047","heartbeat_front_addr":"172.21.15.70:6813/3728228047","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6801","nonce":1147342635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6803","nonce":1147342635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6807","nonce":1147342635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":1147342635},{"type":"v1","addr":"172.21.15.70:6805","nonce":1147342635}]},"public_addr":"172.21.15.70:6801/1147342635","cluster_addr":"172.21.15.70:6803/1147342635","heartbeat_back_addr":"172.21.15.70:6807/1147342635","heartbeat_front_addr":"172.21.15.70:6805/1147342635","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6815","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6817","nonce":284471511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6819","nonce":284471511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6823","nonce":284471511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":284471511},{"type":"v1","addr":"172.21.15.70:6821","nonce":284471511}]},"public_addr":"172.21.15.70:6817/284471511","cluster_addr":"172.21.15.70:6819/284471511","heartbeat_back_addr":"172.21.15.70:6823/284471511","heartbeat_front_addr":"172.21.15.70:6821/284471511","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6825","nonce":2671981919}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6827","nonce":2671981919}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6831","nonce":2671981919}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2671981919},{"type":"v1","addr":"172.21.15.70:6829","nonce":2671981919}]},"public_addr":"172.21.15.70:6825/2671981919","cluster_addr":"172.21.15.70:6827/2671981919","heartbeat_back_addr":"172.21.15.70:6831/2671981919","heartbeat_front_addr":"172.21.15.70:6829/2671981919","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6800","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6801","nonce":4222826791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6802","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6803","nonce":4222826791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6806","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6807","nonce":4222826791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6804","nonce":4222826791},{"type":"v1","addr":"172.21.15.155:6805","nonce":4222826791}]},"public_addr":"172.21.15.155:6801/4222826791","cluster_addr":"172.21.15.155:6803/4222826791","heartbeat_back_addr":"172.21.15.155:6807/4222826791","heartbeat_front_addr":"172.21.15.155:6805/4222826791","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6818","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6820","nonce":1708871082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6822","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6825","nonce":1708871082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6830","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6831","nonce":1708871082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6827","nonce":1708871082},{"type":"v1","addr":"172.21.15.155:6829","nonce":1708871082}]},"public_addr":"172.21.15.155:6820/1708871082","cluster_addr":"172.21.15.155:6825/1708871082","heartbeat_back_addr":"172.21.15.155:6831/1708871082","heartbeat_front_addr":"172.21.15.155:6829/1708871082","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6816","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6817","nonce":4156222745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6819","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6821","nonce":4156222745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6826","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6828","nonce":4156222745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6823","nonce":4156222745},{"type":"v1","addr":"172.21.15.155:6824","nonce":4156222745}]},"public_addr":"172.21.15.155:6817/4156222745","cluster_addr":"172.21.15.155:6821/4156222745","heartbeat_back_addr":"172.21.15.155:6828/4156222745","heartbeat_front_addr":"172.21.15.155:6824/4156222745","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6808","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6809","nonce":324773794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6810","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6811","nonce":324773794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6814","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6815","nonce":324773794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6812","nonce":324773794},{"type":"v1","addr":"172.21.15.155:6813","nonce":324773794}]},"public_addr":"172.21.15.155:6809/324773794","cluster_addr":"172.21.15.155:6811/324773794","heartbeat_back_addr":"172.21.15.155:6815/324773794","heartbeat_front_addr":"172.21.15.155:6813/324773794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.191703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.136564+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.160566+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.240696+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.126106+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.173986+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.180511+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T09:38:37.131259+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:6836/771430984":"2023-08-26T09:53:04.420951+0000","172.21.15.70:6834/771430984":"2023-08-26T09:53:04.420951+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T09:53:11.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a050 msgr2=0x7f108406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.512 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a050 0x7f108406c510 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f109c0011e0 tx=0x7f109c009040 comp rx=0 tx=0).stop 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 msgr2=0x7f10a006ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a006ad70 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f109800c780 tx=0x7f109800cc40 comp rx=0 tx=0).stop 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 shutdown_connections 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f108406a050 0x7f108406c510 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f10a015aa10 0x7f10a006ad70 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f10a00ce5b0 0x7f10a014e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 --2- 172.21.15.70:0/1430497564 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f10a00b2680 0x7f10a01f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:11.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 >> 172.21.15.70:0/1430497564 conn(0x7f10a0069850 msgr2=0x7f10a00cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:11.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 shutdown_connections 2023-08-25T09:53:11.514 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:11.510+0000 7f10a7c87640 1 -- 172.21.15.70:0/1430497564 wait complete. 2023-08-25T09:53:12.530 INFO:tasks.ceph:Scrubbing osd.0 2023-08-25T09:53:12.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:12.761 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.759+0000 7fc6c0acf640 1 Processor -- start 2023-08-25T09:53:12.762 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.759+0000 7fc6c0acf640 1 -- start start 2023-08-25T09:53:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.760+0000 7fc6c0acf640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6bc0b0030 0x7fc6bc0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6c0acf640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fc6bc0b1400 0x7fc6bc0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.763 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6c0acf640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc0c8c00 con 0x7fc6bc069850 2023-08-25T09:53:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6c0acf640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0c91c0 con 0x7fc6bc0b0030 2023-08-25T09:53:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6c0acf640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0b0950 con 0x7fc6bc0b1400 2023-08-25T09:53:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bb7fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fc6bc0b0030 0x7fc6bc0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:12.765 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bbfff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fc6bc069850 0x7fc6bc069ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46240/0 (socket says 172.21.15.70:46240) 2023-08-25T09:53:12.766 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bbfff640 1 -- 172.21.15.70:0/4128227540 learned_addr learned my addr 172.21.15.70:0/4128227540 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6baffd640 1 --2- 172.21.15.70:0/4128227540 >> v2:172.21.15.155:3300/0 conn(0x7fc6bc0b1400 0x7fc6bc0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/4128227540 >> v2:172.21.15.155:3300/0 conn(0x7fc6bc0b1400 msgr2=0x7fc6bc0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:12.767 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bb7fe640 1 --2- 172.21.15.70:0/4128227540 >> v2:172.21.15.155:3300/0 conn(0x7fc6bc0b1400 0x7fc6bc0b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/4128227540 >> v1:172.21.15.70:6789/0 conn(0x7fc6bc069850 legacy=0x7fc6bc069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.761+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/4128227540 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0b1e50 con 0x7fc6bc0b0030 2023-08-25T09:53:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.763+0000 7fc6bb7fe640 1 --2- 172.21.15.70:0/4128227540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc0b0030 0x7fc6bc0b0410 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06d050 tx=0x7fc6ac02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507a434753c66251 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.763+0000 7fc6ba7fc640 1 -- 172.21.15.70:0/4128227540 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac02e070 con 0x7fc6bc0b0030 2023-08-25T09:53:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.763+0000 7fc6ba7fc640 1 -- 172.21.15.70:0/4128227540 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac0315a0 con 0x7fc6bc0b0030 2023-08-25T09:53:12.768 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.763+0000 7fc6ba7fc640 1 -- 172.21.15.70:0/4128227540 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac031930 con 0x7fc6bc0b0030 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 -- 172.21.15.70:0/4128227540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc0b0030 msgr2=0x7fc6bc0b0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 --2- 172.21.15.70:0/4128227540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc0b0030 0x7fc6bc0b0410 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06d050 tx=0x7fc6ac02b840 comp rx=0 tx=0).stop 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 -- 172.21.15.70:0/4128227540 shutdown_connections 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 --2- 172.21.15.70:0/4128227540 >> v2:172.21.15.155:3300/0 conn(0x7fc6bc0b1400 0x7fc6bc0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 --2- 172.21.15.70:0/4128227540 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc0b0030 0x7fc6bc0b0410 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 -- 172.21.15.70:0/4128227540 >> 172.21.15.70:0/4128227540 conn(0x7fc6bc06b560 msgr2=0x7fc6bc0cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:12.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 -- 172.21.15.70:0/4128227540 shutdown_connections 2023-08-25T09:53:12.770 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.764+0000 7fc6c0acf640 1 -- 172.21.15.70:0/4128227540 wait complete. 2023-08-25T09:53:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.765+0000 7fc6c0acf640 1 Processor -- start 2023-08-25T09:53:12.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.766+0000 7fc6c0acf640 1 -- start start 2023-08-25T09:53:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6c0acf640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc069850 0x7fc6bc1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6c0acf640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0b0030 0x7fc6bc0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6c0acf640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc0b1400 0x7fc6bc155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6c0acf640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0c6840 con 0x7fc6bc069850 2023-08-25T09:53:12.772 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6c0acf640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0c65d0 con 0x7fc6bc0b1400 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6c0acf640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc0c6a90 con 0x7fc6bc0b0030 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc069850 0x7fc6bc1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc069850 0x7fc6bc1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54446/0 (socket says 172.21.15.70:54446) 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/3078821823 learned_addr learned my addr 172.21.15.70:0/3078821823 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6baffd640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0b0030 0x7fc6bc0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/3078821823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0b0030 msgr2=0x7fc6bc0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fc6bc0b0030 0x7fc6bc0c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/3078821823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc0b1400 msgr2=0x7fc6bc155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fc6bc0b1400 0x7fc6bc155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.767+0000 7fc6bb7fe640 1 -- 172.21.15.70:0/3078821823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ac009400 con 0x7fc6bc069850 2023-08-25T09:53:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.768+0000 7fc6bb7fe640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc069850 0x7fc6bc1556e0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac034040 tx=0x7fc6ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.768+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac032840 con 0x7fc6bc069850 2023-08-25T09:53:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.768+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac0329a0 con 0x7fc6bc069850 2023-08-25T09:53:12.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.768+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac032d50 con 0x7fc6bc069850 2023-08-25T09:53:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.768+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc0c6d70 con 0x7fc6bc069850 2023-08-25T09:53:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.768+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc0c7330 con 0x7fc6bc069850 2023-08-25T09:53:12.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.769+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc6bc0c8e50 con 0x7fc6bc069850 2023-08-25T09:53:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.770+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc6ac03b3f0 con 0x7fc6bc069850 2023-08-25T09:53:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.771+0000 7fc6a3fff640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6a806a050 0x7fc6a806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.771+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fc6ac0aa2e0 con 0x7fc6bc069850 2023-08-25T09:53:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.772+0000 7fc6baffd640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6a806a050 0x7fc6a806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:12.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.772+0000 7fc6a3fff640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7fc6a80704d0 0x7fc6a80728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.772+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 --> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6a8072fa0 con 0x7fc6a80704d0 2023-08-25T09:53:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.772+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ac0ac8c0 con 0x7fc6bc069850 2023-08-25T09:53:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.772+0000 7fc6bbfff640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7fc6a80704d0 0x7fc6a80728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.772+0000 7fc6baffd640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6a806a050 0x7fc6a806c510 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06f780 tx=0x7fc6b0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.773+0000 7fc6bbfff640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7fc6a80704d0 0x7fc6a80728f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:12.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.773+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== osd.0 v2:172.21.15.70:6808/3728228047 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fc6a8072fa0 con 0x7fc6a80704d0 2023-08-25T09:53:12.788 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.786+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 --> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc6bc0b2130 con 0x7fc6a80704d0 2023-08-25T09:53:12.789 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.786+0000 7fc6a3fff640 1 -- 172.21.15.70:0/3078821823 <== osd.0 v2:172.21.15.70:6808/3728228047 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7fc6b4001b70 con 0x7fc6a80704d0 2023-08-25T09:53:12.789 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:12.790 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:12.790 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7fc6a80704d0 msgr2=0x7fc6a80728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6808/3728228047,v1:172.21.15.70:6809/3728228047] conn(0x7fc6a80704d0 0x7fc6a80728f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6a806a050 msgr2=0x7fc6a806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fc6a806a050 0x7fc6a806c510 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc06f780 tx=0x7fc6b0008040 comp rx=0 tx=0).stop 2023-08-25T09:53:12.791 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc069850 msgr2=0x7fc6bc1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 --2- 172.21.15.70:0/3078821823 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fc6bc069850 0x7fc6bc1556e0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac034040 tx=0x7fc6ac0046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6bbfff640 1 -- 172.21.15.70:0/3078821823 reap_dead start 2023-08-25T09:53:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 shutdown_connections 2023-08-25T09:53:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 >> 172.21.15.70:0/3078821823 conn(0x7fc6bc06b560 msgr2=0x7fc6bc0ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 shutdown_connections 2023-08-25T09:53:12.792 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:12.787+0000 7fc6c0acf640 1 -- 172.21.15.70:0/3078821823 wait complete. 2023-08-25T09:53:12.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-08-25T09:53:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.024+0000 7f422c139640 1 Processor -- start 2023-08-25T09:53:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.025+0000 7f422c139640 1 -- start start 2023-08-25T09:53:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.026+0000 7f422c139640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f422415cbd0 0x7f42240cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.026+0000 7f422c139640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f42241606c0 0x7f42240cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.026+0000 7f422c139640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f422415cfb0 con 0x7f42240cdeb0 2023-08-25T09:53:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.026+0000 7f422c139640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42240cd4e0 con 0x7f42241606c0 2023-08-25T09:53:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.026+0000 7f422c139640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4224160f10 con 0x7f422415cbd0 2023-08-25T09:53:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f422a6d6640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f42240cdeb0 0x7f42240d2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46254/0 (socket says 172.21.15.70:46254) 2023-08-25T09:53:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f422a6d6640 1 -- 172.21.15.70:0/291653676 learned_addr learned my addr 172.21.15.70:0/291653676 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:13.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f42296d4640 1 --2- 172.21.15.70:0/291653676 >> v2:172.21.15.70:3300/0 conn(0x7f42241606c0 0x7f42240cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f4229ed5640 1 --2- 172.21.15.70:0/291653676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f422415cbd0 0x7f42240cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f4228ed3640 1 -- 172.21.15.70:0/291653676 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457086982 0 0) 0x7f422415cfb0 con 0x7f42240cdeb0 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f4228ed3640 1 -- 172.21.15.70:0/291653676 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4210003520 con 0x7f42240cdeb0 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f42296d4640 1 -- 172.21.15.70:0/291653676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f422415cbd0 msgr2=0x7f42240cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f42296d4640 1 --2- 172.21.15.70:0/291653676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f422415cbd0 0x7f42240cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f42296d4640 1 -- 172.21.15.70:0/291653676 >> v1:172.21.15.70:6789/0 conn(0x7f42240cdeb0 legacy=0x7f42240d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.027+0000 7f42296d4640 1 -- 172.21.15.70:0/291653676 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42241611f0 con 0x7f42241606c0 2023-08-25T09:53:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.028+0000 7f42296d4640 1 --2- 172.21.15.70:0/291653676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42241606c0 0x7f42240cd970 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f42180097a0 tx=0x7f421802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8439618f1584b382 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f4228ed3640 1 -- 172.21.15.70:0/291653676 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421802d040 con 0x7f42241606c0 2023-08-25T09:53:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f4228ed3640 1 -- 172.21.15.70:0/291653676 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4218031810 con 0x7f42241606c0 2023-08-25T09:53:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f4228ed3640 1 -- 172.21.15.70:0/291653676 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4218031ba0 con 0x7f42241606c0 2023-08-25T09:53:13.034 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 -- 172.21.15.70:0/291653676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42241606c0 msgr2=0x7f42240cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 --2- 172.21.15.70:0/291653676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42241606c0 0x7f42240cd970 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f42180097a0 tx=0x7f421802b8b0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 -- 172.21.15.70:0/291653676 shutdown_connections 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 --2- 172.21.15.70:0/291653676 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f42241606c0 0x7f42240cd970 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 --2- 172.21.15.70:0/291653676 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f422415cbd0 0x7f42240cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 -- 172.21.15.70:0/291653676 >> 172.21.15.70:0/291653676 conn(0x7f4224070260 msgr2=0x7f4224070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.029+0000 7f422c139640 1 -- 172.21.15.70:0/291653676 shutdown_connections 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.030+0000 7f422c139640 1 -- 172.21.15.70:0/291653676 wait complete. 2023-08-25T09:53:13.035 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.031+0000 7f422c139640 1 Processor -- start 2023-08-25T09:53:13.036 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.031+0000 7f422c139640 1 -- start start 2023-08-25T09:53:13.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.032+0000 7f422c139640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42240cdeb0 0x7f422415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.032+0000 7f422c139640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 0x7f4224159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.037 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.032+0000 7f422c139640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42241606c0 0x7f42240c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.032+0000 7f422c139640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422415a070 con 0x7f422415cbd0 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.032+0000 7f422c139640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240c65a0 con 0x7f42240cdeb0 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.032+0000 7f422c139640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240c6b60 con 0x7f42241606c0 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 0x7f4224159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 0x7f4224159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54466/0 (socket says 172.21.15.70:54466) 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 -- 172.21.15.70:0/1684739412 learned_addr learned my addr 172.21.15.70:0/1684739412 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 -- 172.21.15.70:0/1684739412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42241606c0 msgr2=0x7f42240c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.038 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42241606c0 0x7f42240c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 -- 172.21.15.70:0/1684739412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42240cdeb0 msgr2=0x7f422415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42240cdeb0 0x7f422415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 -- 172.21.15.70:0/1684739412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4218009400 con 0x7f422415cbd0 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.033+0000 7f42296d4640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 0x7f4224159b30 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f42180098c0 tx=0x7f42180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.034+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421803ad60 con 0x7f422415cbd0 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.034+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42180423f0 con 0x7f422415cbd0 2023-08-25T09:53:13.039 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.034+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42180427a0 con 0x7f422415cbd0 2023-08-25T09:53:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.034+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42240c8e80 con 0x7f422415cbd0 2023-08-25T09:53:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.034+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42240c9400 con 0x7f422415cbd0 2023-08-25T09:53:13.040 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.035+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f422404e420 con 0x7f422415cbd0 2023-08-25T09:53:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.036+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f42180393b0 con 0x7f422415cbd0 2023-08-25T09:53:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.037+0000 7f42167fc640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f420806a050 0x7f420806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.037+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f42180a9ff0 con 0x7f422415cbd0 2023-08-25T09:53:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.037+0000 7f4229ed5640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f420806a050 0x7f420806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.038+0000 7f4229ed5640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f420806a050 0x7f420806c510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f42200098f0 tx=0x7f4220008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.044 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.042+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f42180dc990 con 0x7f422415cbd0 2023-08-25T09:53:13.231 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.229+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f42240cd170 con 0x7f420806a050 2023-08-25T09:53:13.232 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.230+0000 7f42167fc640 1 -- 172.21.15.70:0/1684739412 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4220009e70 con 0x7f420806a050 2023-08-25T09:53:13.232 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 0 to deep-scrub 2023-08-25T09:53:13.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f420806a050 msgr2=0x7f420806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f420806a050 0x7f420806c510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f42200098f0 tx=0x7f4220008040 comp rx=0 tx=0).stop 2023-08-25T09:53:13.235 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 msgr2=0x7f4224159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 0x7f4224159b30 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f42180098c0 tx=0x7f42180046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 shutdown_connections 2023-08-25T09:53:13.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f420806a050 0x7f420806c510 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.236 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f42241606c0 0x7f42240c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f422415cbd0 0x7f4224159b30 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 --2- 172.21.15.70:0/1684739412 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f42240cdeb0 0x7f422415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 >> 172.21.15.70:0/1684739412 conn(0x7f4224070260 msgr2=0x7f4224156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:13.237 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 shutdown_connections 2023-08-25T09:53:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.232+0000 7f422c139640 1 -- 172.21.15.70:0/1684739412 wait complete. 2023-08-25T09:53:13.254 INFO:tasks.ceph:Scrubbing osd.1 2023-08-25T09:53:13.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:13.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.461+0000 7fbe8beee640 1 Processor -- start 2023-08-25T09:53:13.464 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.461+0000 7fbe8beee640 1 -- start start 2023-08-25T09:53:13.465 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.463+0000 7fbe8beee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe84142740 0x7fbe840cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.463+0000 7fbe8beee640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbe8406aa70 0x7fbe8406af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.463+0000 7fbe8beee640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8415d320 con 0x7fbe84142c30 2023-08-25T09:53:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.463+0000 7fbe8beee640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8406b440 con 0x7fbe8406aa70 2023-08-25T09:53:13.466 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.463+0000 7fbe8beee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840ccd40 con 0x7fbe84142740 2023-08-25T09:53:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbe8406aa70 0x7fbe8406af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fbe8406aa70 0x7fbe8406af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54474/0 (socket says 172.21.15.70:54474) 2023-08-25T09:53:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 -- 172.21.15.70:0/4121267088 learned_addr learned my addr 172.21.15.70:0/4121267088 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe89c8a640 1 --2- 172.21.15.70:0/4121267088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe84142740 0x7fbe840cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 -- 172.21.15.70:0/4121267088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe84142740 msgr2=0x7fbe840cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 --2- 172.21.15.70:0/4121267088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe84142740 0x7fbe840cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 -- 172.21.15.70:0/4121267088 >> v1:172.21.15.155:6789/0 conn(0x7fbe84142c30 legacy=0x7fbe8415db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.468 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.464+0000 7fbe8a48b640 1 -- 172.21.15.70:0/4121267088 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840cd020 con 0x7fbe8406aa70 2023-08-25T09:53:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.465+0000 7fbe8a48b640 1 --2- 172.21.15.70:0/4121267088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8406aa70 0x7fbe8406af00 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009e50 tx=0x7fbe7c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a702085e50e13dab server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe88c88640 1 -- 172.21.15.70:0/4121267088 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c02cb00 con 0x7fbe8406aa70 2023-08-25T09:53:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe88c88640 1 -- 172.21.15.70:0/4121267088 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c02cc60 con 0x7fbe8406aa70 2023-08-25T09:53:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe88c88640 1 -- 172.21.15.70:0/4121267088 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c0343f0 con 0x7fbe8406aa70 2023-08-25T09:53:13.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 -- 172.21.15.70:0/4121267088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8406aa70 msgr2=0x7fbe8406af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 --2- 172.21.15.70:0/4121267088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8406aa70 0x7fbe8406af00 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c009e50 tx=0x7fbe7c02bf60 comp rx=0 tx=0).stop 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 -- 172.21.15.70:0/4121267088 shutdown_connections 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 --2- 172.21.15.70:0/4121267088 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe8406aa70 0x7fbe8406af00 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 --2- 172.21.15.70:0/4121267088 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe84142740 0x7fbe840cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 -- 172.21.15.70:0/4121267088 >> 172.21.15.70:0/4121267088 conn(0x7fbe84069c70 msgr2=0x7fbe8406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.466+0000 7fbe8beee640 1 -- 172.21.15.70:0/4121267088 shutdown_connections 2023-08-25T09:53:13.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.467+0000 7fbe8beee640 1 -- 172.21.15.70:0/4121267088 wait complete. 2023-08-25T09:53:13.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.468+0000 7fbe8beee640 1 Processor -- start 2023-08-25T09:53:13.471 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.468+0000 7fbe8beee640 1 -- start start 2023-08-25T09:53:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe8beee640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe8406aa70 0x7fbe840b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe8beee640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84142740 0x7fbe84152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe8beee640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe84142c30 0x7fbe841585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.472 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe8beee640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84153420 con 0x7fbe84142740 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe8beee640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84157540 con 0x7fbe84142c30 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe8beee640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84157820 con 0x7fbe8406aa70 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.469+0000 7fbe89489640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84142740 0x7fbe84152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84142740 0x7fbe84152ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54478/0 (socket says 172.21.15.70:54478) 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 -- 172.21.15.70:0/3408442031 learned_addr learned my addr 172.21.15.70:0/3408442031 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe8a48b640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe84142c30 0x7fbe841585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89c8a640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe8406aa70 0x7fbe840b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 -- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe8406aa70 msgr2=0x7fbe840b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe8406aa70 0x7fbe840b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 -- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe84142c30 msgr2=0x7fbe841585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe84142c30 0x7fbe841585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 -- 172.21.15.70:0/3408442031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c009b30 con 0x7fbe84142740 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe8a48b640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fbe84142c30 0x7fbe841585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:13.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89c8a640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fbe8406aa70 0x7fbe840b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe89489640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84142740 0x7fbe84152ee0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c450 tx=0x7fbe7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.470+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe780052d0 con 0x7fbe84142740 2023-08-25T09:53:13.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.471+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78005430 con 0x7fbe84142740 2023-08-25T09:53:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.471+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78012870 con 0x7fbe84142740 2023-08-25T09:53:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.471+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe84157b00 con 0x7fbe84142740 2023-08-25T09:53:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.471+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe841580c0 con 0x7fbe84142740 2023-08-25T09:53:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.472+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe8415d5e0 con 0x7fbe84142740 2023-08-25T09:53:13.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.473+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbe780129d0 con 0x7fbe84142740 2023-08-25T09:53:13.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe827fc640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7006a050 0x7fbe7006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.477 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbe78087dd0 con 0x7fbe84142740 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe89c8a640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7006a050 0x7fbe7006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe827fc640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fbe700704d0 0x7fbe700728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 --> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe70072fa0 con 0x7fbe700704d0 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe7808b020 con 0x7fbe84142740 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.474+0000 7fbe8a48b640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fbe700704d0 0x7fbe700728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.475+0000 7fbe89c8a640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7006a050 0x7fbe7006c510 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406fc50 tx=0x7fbe74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.475+0000 7fbe8a48b640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fbe700704d0 0x7fbe700728f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.479 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.476+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== osd.1 v2:172.21.15.70:6800/1147342635 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fbe70072fa0 con 0x7fbe700704d0 2023-08-25T09:53:13.490 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.488+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 --> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbe840cd300 con 0x7fbe700704d0 2023-08-25T09:53:13.491 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe827fc640 1 -- 172.21.15.70:0/3408442031 <== osd.1 v2:172.21.15.70:6800/1147342635 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7fbe7c02cad0 con 0x7fbe700704d0 2023-08-25T09:53:13.491 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:13.492 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:13.492 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:13.492 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fbe700704d0 msgr2=0x7fbe700728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6800/1147342635,v1:172.21.15.70:6801/1147342635] conn(0x7fbe700704d0 0x7fbe700728f0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7006a050 msgr2=0x7fbe7006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fbe7006a050 0x7fbe7006c510 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe8406fc50 tx=0x7fbe74008040 comp rx=0 tx=0).stop 2023-08-25T09:53:13.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84142740 msgr2=0x7fbe84152ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 --2- 172.21.15.70:0/3408442031 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fbe84142740 0x7fbe84152ee0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800c450 tx=0x7fbe7800c910 comp rx=0 tx=0).stop 2023-08-25T09:53:13.493 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8a48b640 1 -- 172.21.15.70:0/3408442031 reap_dead start 2023-08-25T09:53:13.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 shutdown_connections 2023-08-25T09:53:13.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 >> 172.21.15.70:0/3408442031 conn(0x7fbe84069c70 msgr2=0x7fbe84140b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:13.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 shutdown_connections 2023-08-25T09:53:13.494 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.489+0000 7fbe8beee640 1 -- 172.21.15.70:0/3408442031 wait complete. 2023-08-25T09:53:13.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-08-25T09:53:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.717+0000 7f8219d5d640 1 Processor -- start 2023-08-25T09:53:13.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.717+0000 7f8219d5d640 1 -- start start 2023-08-25T09:53:13.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f8219d5d640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f82140cffb0 0x7f82140cd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f8219d5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82140cdeb0 0x7f82140d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f8219d5d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821415cfb0 con 0x7f821415cbd0 2023-08-25T09:53:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f8219d5d640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82140cd4e0 con 0x7f82140cffb0 2023-08-25T09:53:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f8219d5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8214160f10 con 0x7f82140cdeb0 2023-08-25T09:53:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f82137fe640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f821415cbd0 0x7f82141606c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46268/0 (socket says 172.21.15.70:46268) 2023-08-25T09:53:13.721 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.718+0000 7f82137fe640 1 -- 172.21.15.70:0/2790768437 learned_addr learned my addr 172.21.15.70:0/2790768437 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f8212ffd640 1 --2- 172.21.15.70:0/2790768437 >> v2:172.21.15.70:3300/0 conn(0x7f82140cffb0 0x7f82140cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.722 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f8213fff640 1 --2- 172.21.15.70:0/2790768437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82140cdeb0 0x7f82140d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f82127fc640 1 -- 172.21.15.70:0/2790768437 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479837159 0 0) 0x7f821415cfb0 con 0x7f821415cbd0 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f82127fc640 1 -- 172.21.15.70:0/2790768437 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81fc003520 con 0x7f821415cbd0 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f82127fc640 1 -- 172.21.15.70:0/2790768437 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467455506 0 0) 0x7f81fc003520 con 0x7f821415cbd0 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f8212ffd640 1 -- 172.21.15.70:0/2790768437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82140cdeb0 msgr2=0x7f82140d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f8212ffd640 1 --2- 172.21.15.70:0/2790768437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82140cdeb0 0x7f82140d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f8212ffd640 1 -- 172.21.15.70:0/2790768437 >> v1:172.21.15.70:6789/0 conn(0x7f821415cbd0 legacy=0x7f82141606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.723 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.719+0000 7f8212ffd640 1 -- 172.21.15.70:0/2790768437 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82141611f0 con 0x7f82140cffb0 2023-08-25T09:53:13.724 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.721+0000 7f8212ffd640 1 --2- 172.21.15.70:0/2790768437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f82140cd970 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f82080097a0 tx=0x7f820802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689b68444348a333 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.721+0000 7f82127fc640 1 -- 172.21.15.70:0/2790768437 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040019f0 con 0x7f82140cffb0 2023-08-25T09:53:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.721+0000 7f82127fc640 1 -- 172.21.15.70:0/2790768437 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820802e050 con 0x7f82140cffb0 2023-08-25T09:53:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.721+0000 7f82127fc640 1 -- 172.21.15.70:0/2790768437 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208030730 con 0x7f82140cffb0 2023-08-25T09:53:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 -- 172.21.15.70:0/2790768437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 msgr2=0x7f82140cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 --2- 172.21.15.70:0/2790768437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f82140cd970 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f82080097a0 tx=0x7f820802b790 comp rx=0 tx=0).stop 2023-08-25T09:53:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 -- 172.21.15.70:0/2790768437 shutdown_connections 2023-08-25T09:53:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 --2- 172.21.15.70:0/2790768437 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f82140cd970 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 --2- 172.21.15.70:0/2790768437 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f82140cdeb0 0x7f82140d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 -- 172.21.15.70:0/2790768437 >> 172.21.15.70:0/2790768437 conn(0x7f8214070260 msgr2=0x7f8214070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 -- 172.21.15.70:0/2790768437 shutdown_connections 2023-08-25T09:53:13.726 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.722+0000 7f8219d5d640 1 -- 172.21.15.70:0/2790768437 wait complete. 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.724+0000 7f8219d5d640 1 Processor -- start 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.724+0000 7f8219d5d640 1 -- start start 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8219d5d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82140cdeb0 0x7f821415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8219d5d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f8214156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8219d5d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f821415cbd0 0x7f82140cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8219d5d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214160190 con 0x7f82140cffb0 2023-08-25T09:53:13.730 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8219d5d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821415f650 con 0x7f82140cdeb0 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8219d5d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821415f8f0 con 0x7f821415cbd0 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8212ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f8214156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8212ffd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f8214156260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54502/0 (socket says 172.21.15.70:54502) 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.725+0000 7f8212ffd640 1 -- 172.21.15.70:0/1453920429 learned_addr learned my addr 172.21.15.70:0/1453920429 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8213fff640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f821415cbd0 0x7f82140cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f82137fe640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82140cdeb0 0x7f821415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8212ffd640 1 -- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f821415cbd0 msgr2=0x7f82140cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8212ffd640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f821415cbd0 0x7f82140cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8212ffd640 1 -- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82140cdeb0 msgr2=0x7f821415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8212ffd640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82140cdeb0 0x7f821415f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8212ffd640 1 -- 172.21.15.70:0/1453920429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8208009400 con 0x7f82140cffb0 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8213fff640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f821415cbd0 0x7f82140cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f82137fe640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82140cdeb0 0x7f821415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8212ffd640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f8214156260 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f82080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820803b810 con 0x7f82140cffb0 2023-08-25T09:53:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820803b970 con 0x7f82140cffb0 2023-08-25T09:53:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820803bd20 con 0x7f82140cffb0 2023-08-25T09:53:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.726+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821415fbd0 con 0x7f82140cffb0 2023-08-25T09:53:13.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.727+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82141567a0 con 0x7f82140cffb0 2023-08-25T09:53:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.728+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82140cdd20 con 0x7f82140cffb0 2023-08-25T09:53:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.728+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8208039040 con 0x7f82140cffb0 2023-08-25T09:53:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.729+0000 7f81fbfff640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f820006a050 0x7f820006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.729+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f82080a9970 con 0x7f82140cffb0 2023-08-25T09:53:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.730+0000 7f82137fe640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f820006a050 0x7f820006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:13.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.730+0000 7f82137fe640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f820006a050 0x7f820006c510 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8204005c00 tx=0x7f8204009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:13.736 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.734+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8208075370 con 0x7f82140cffb0 2023-08-25T09:53:13.929 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.927+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f82140cd170 con 0x7f820006a050 2023-08-25T09:53:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.928+0000 7f81fbfff640 1 -- 172.21.15.70:0/1453920429 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f820400ddd0 con 0x7f820006a050 2023-08-25T09:53:13.930 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 1 to deep-scrub 2023-08-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f820006a050 msgr2=0x7f820006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f820006a050 0x7f820006c510 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8204005c00 tx=0x7f8204009040 comp rx=0 tx=0).stop 2023-08-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 msgr2=0x7f8214156260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f8214156260 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f8208000c00 tx=0x7f82080046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.933 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 shutdown_connections 2023-08-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f820006a050 0x7f820006c510 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f821415cbd0 0x7f82140cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f82140cffb0 0x7f8214156260 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 --2- 172.21.15.70:0/1453920429 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f82140cdeb0 0x7f821415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:13.934 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 >> 172.21.15.70:0/1453920429 conn(0x7f8214070260 msgr2=0x7f82140d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 shutdown_connections 2023-08-25T09:53:13.935 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:13.931+0000 7f8219d5d640 1 -- 172.21.15.70:0/1453920429 wait complete. 2023-08-25T09:53:13.950 INFO:tasks.ceph:Scrubbing osd.2 2023-08-25T09:53:13.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.154+0000 7f296b30f640 1 Processor -- start 2023-08-25T09:53:14.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.154+0000 7f296b30f640 1 -- start start 2023-08-25T09:53:14.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f296b30f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640add40 0x7f29640ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f296b30f640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f296415ce70 0x7f29640ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f296b30f640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640ad940 con 0x7f29640ce520 2023-08-25T09:53:14.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f296b30f640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640ae660 con 0x7f296415ce70 2023-08-25T09:53:14.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f296b30f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640cccd0 con 0x7f29640add40 2023-08-25T09:53:14.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f29688aa640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f296415ce70 0x7f29640ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f29698ac640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f29640ce520 0x7f296406f3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46278/0 (socket says 172.21.15.70:46278) 2023-08-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.156+0000 7f29698ac640 1 -- 172.21.15.70:0/723612273 learned_addr learned my addr 172.21.15.70:0/723612273 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.157+0000 7f29690ab640 1 --2- 172.21.15.70:0/723612273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640add40 0x7f29640ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.157+0000 7f29688aa640 1 -- 172.21.15.70:0/723612273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640add40 msgr2=0x7f29640ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:14.160 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.157+0000 7f29688aa640 1 --2- 172.21.15.70:0/723612273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640add40 0x7f29640ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.157+0000 7f29688aa640 1 -- 172.21.15.70:0/723612273 >> v1:172.21.15.70:6789/0 conn(0x7f29640ce520 legacy=0x7f296406f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.161 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.157+0000 7f29688aa640 1 -- 172.21.15.70:0/723612273 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640ac920 con 0x7f296415ce70 2023-08-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.158+0000 7f29688aa640 1 --2- 172.21.15.70:0/723612273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f29640ae120 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f29540097a0 tx=0x7f295402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db99d37331c4fb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.158+0000 7f295bfff640 1 -- 172.21.15.70:0/723612273 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295402d040 con 0x7f296415ce70 2023-08-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.158+0000 7f295bfff640 1 -- 172.21.15.70:0/723612273 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2954031810 con 0x7f296415ce70 2023-08-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.158+0000 7f295bfff640 1 -- 172.21.15.70:0/723612273 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954031ba0 con 0x7f296415ce70 2023-08-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 -- 172.21.15.70:0/723612273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 msgr2=0x7f29640ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.162 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 --2- 172.21.15.70:0/723612273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f29640ae120 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f29540097a0 tx=0x7f295402b8b0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 -- 172.21.15.70:0/723612273 shutdown_connections 2023-08-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 --2- 172.21.15.70:0/723612273 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f29640ae120 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 --2- 172.21.15.70:0/723612273 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640add40 0x7f29640ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 -- 172.21.15.70:0/723612273 >> 172.21.15.70:0/723612273 conn(0x7f2964069850 msgr2=0x7f2964069c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 -- 172.21.15.70:0/723612273 shutdown_connections 2023-08-25T09:53:14.163 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.159+0000 7f296b30f640 1 -- 172.21.15.70:0/723612273 wait complete. 2023-08-25T09:53:14.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.161+0000 7f296b30f640 1 Processor -- start 2023-08-25T09:53:14.164 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.161+0000 7f296b30f640 1 -- start start 2023-08-25T09:53:14.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f296b30f640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29640add40 0x7f296415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f296b30f640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640ce520 0x7f29640b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.165 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f296b30f640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f2964155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f296b30f640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296415b2f0 con 0x7f296415ce70 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f296b30f640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640b48f0 con 0x7f29640add40 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f296b30f640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29640c6430 con 0x7f29640ce520 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.162+0000 7f29698ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f2964155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f2964155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54528/0 (socket says 172.21.15.70:54528) 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 -- 172.21.15.70:0/752404209 learned_addr learned my addr 172.21.15.70:0/752404209 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29690ab640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29640add40 0x7f296415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.166 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29688aa640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640ce520 0x7f29640b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 -- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640ce520 msgr2=0x7f29640b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640ce520 0x7f29640b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 -- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29640add40 msgr2=0x7f296415c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29640add40 0x7f296415c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 -- 172.21.15.70:0/752404209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009400 con 0x7f296415ce70 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29690ab640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f29640add40 0x7f296415c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:14.167 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29688aa640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f29640ce520 0x7f29640b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:14.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.163+0000 7f29698ac640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f2964155c80 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f295c00cbf0 tx=0x7f295c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.168 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.164+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00ad40 con 0x7f296415ce70 2023-08-25T09:53:14.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.164+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c012400 con 0x7f296415ce70 2023-08-25T09:53:14.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.164+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c00aea0 con 0x7f296415ce70 2023-08-25T09:53:14.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.164+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296415b540 con 0x7f296415ce70 2023-08-25T09:53:14.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.164+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296415bb00 con 0x7f296415ce70 2023-08-25T09:53:14.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.165+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29640acee0 con 0x7f296415ce70 2023-08-25T09:53:14.169 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.166+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f295c007070 con 0x7f296415ce70 2023-08-25T09:53:14.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f2959ffb640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f294806a050 0x7f294806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.170 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f295c0880f0 con 0x7f296415ce70 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f2959ffb640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f29480704d0 0x7f29480728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 --> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2948072fa0 con 0x7f29480704d0 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7f295c08a6d0 con 0x7f296415ce70 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f29690ab640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f294806a050 0x7f294806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.167+0000 7f29688aa640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f29480704d0 0x7f29480728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.168+0000 7f29690ab640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f294806a050 0x7f294806c510 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2960006fd0 tx=0x7f2960008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.168+0000 7f29688aa640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f29480704d0 0x7f29480728f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.169+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== osd.2 v2:172.21.15.70:6815/284471511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f2948072fa0 con 0x7f29480704d0 2023-08-25T09:53:14.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.182+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 --> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f29640acc00 con 0x7f29480704d0 2023-08-25T09:53:14.184 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.182+0000 7f2959ffb640 1 -- 172.21.15.70:0/752404209 <== osd.2 v2:172.21.15.70:6815/284471511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f2954031b90 con 0x7f29480704d0 2023-08-25T09:53:14.185 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:14.185 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:14.185 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:14.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f29480704d0 msgr2=0x7f29480728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6815/284471511,v1:172.21.15.70:6817/284471511] conn(0x7f29480704d0 0x7f29480728f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f294806a050 msgr2=0x7f294806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.186 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f294806a050 0x7f294806c510 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f2960006fd0 tx=0x7f2960008040 comp rx=0 tx=0).stop 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 msgr2=0x7f2964155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 --2- 172.21.15.70:0/752404209 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f296415ce70 0x7f2964155c80 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f295c00cbf0 tx=0x7f295c00a430 comp rx=0 tx=0).stop 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f29698ac640 1 -- 172.21.15.70:0/752404209 reap_dead start 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 shutdown_connections 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 >> 172.21.15.70:0/752404209 conn(0x7f2964069850 msgr2=0x7f29641591e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 shutdown_connections 2023-08-25T09:53:14.187 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.183+0000 7f296b30f640 1 -- 172.21.15.70:0/752404209 wait complete. 2023-08-25T09:53:14.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-08-25T09:53:14.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.415+0000 7fa84b376640 1 Processor -- start 2023-08-25T09:53:14.418 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.415+0000 7fa84b376640 1 -- start start 2023-08-25T09:53:14.419 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.417+0000 7fa84b376640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415ec10 0x7fa84415c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.417+0000 7fa84b376640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8440b1b60 con 0x7fa84415c6d0 2023-08-25T09:53:14.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.417+0000 7fa84b376640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84406b7c0 con 0x7fa84415e830 2023-08-25T09:53:14.420 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.417+0000 7fa84b376640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84406bd80 con 0x7fa84415ec10 2023-08-25T09:53:14.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.418+0000 7fa849913640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fa84415c6d0 0x7fa8440b1ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46292/0 (socket says 172.21.15.70:46292) 2023-08-25T09:53:14.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.418+0000 7fa849913640 1 -- 172.21.15.70:0/3160384908 learned_addr learned my addr 172.21.15.70:0/3160384908 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:14.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.418+0000 7fa848911640 1 --2- 172.21.15.70:0/3160384908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415ec10 0x7fa84415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.418+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831466820 0 0) 0x7fa8440b1b60 con 0x7fa84415c6d0 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.418+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa830003520 con 0x7fa84415c6d0 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.418+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622621066 0 0) 0x7fa84406b7c0 con 0x7fa84415e830 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.419+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8440b1b60 con 0x7fa84415e830 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.419+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894964739 0 0) 0x7fa830003520 con 0x7fa84415c6d0 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.419+0000 7fa848911640 1 -- 172.21.15.70:0/3160384908 >> v1:172.21.15.155:6789/0 conn(0x7fa84415e830 legacy=0x7fa84406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.419+0000 7fa848911640 1 -- 172.21.15.70:0/3160384908 >> v1:172.21.15.70:6789/0 conn(0x7fa84415c6d0 legacy=0x7fa8440b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.419+0000 7fa848911640 1 -- 172.21.15.70:0/3160384908 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84406baa0 con 0x7fa84415ec10 2023-08-25T09:53:14.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.420+0000 7fa848911640 1 --2- 172.21.15.70:0/3160384908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415ec10 0x7fa84415c190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097d0 tx=0x7fa83402b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8299cc90f42564c6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.421+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 <== mon.2 v2:172.21.15.155:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c001670 con 0x7fa84415ec10 2023-08-25T09:53:14.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.421+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 <== mon.2 v2:172.21.15.155:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83402e050 con 0x7fa84415ec10 2023-08-25T09:53:14.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.421+0000 7fa83bfff640 1 -- 172.21.15.70:0/3160384908 <== mon.2 v2:172.21.15.155:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834030710 con 0x7fa84415ec10 2023-08-25T09:53:14.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 -- 172.21.15.70:0/3160384908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415ec10 msgr2=0x7fa84415c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 --2- 172.21.15.70:0/3160384908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415ec10 0x7fa84415c190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa8340097d0 tx=0x7fa83402b770 comp rx=0 tx=0).stop 2023-08-25T09:53:14.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 -- 172.21.15.70:0/3160384908 shutdown_connections 2023-08-25T09:53:14.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 --2- 172.21.15.70:0/3160384908 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415ec10 0x7fa84415c190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 -- 172.21.15.70:0/3160384908 >> 172.21.15.70:0/3160384908 conn(0x7fa844069850 msgr2=0x7fa844069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:14.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 -- 172.21.15.70:0/3160384908 shutdown_connections 2023-08-25T09:53:14.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.422+0000 7fa84b376640 1 -- 172.21.15.70:0/3160384908 wait complete. 2023-08-25T09:53:14.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.424+0000 7fa84b376640 1 Processor -- start 2023-08-25T09:53:14.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.424+0000 7fa84b376640 1 -- start start 2023-08-25T09:53:14.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa84b376640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415c6d0 0x7fa8440c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa84b376640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84415e830 0x7fa8440c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa84b376640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 0x7fa8440ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa84b376640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440c3c40 con 0x7fa84415ec10 2023-08-25T09:53:14.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa84b376640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440c3660 con 0x7fa84415e830 2023-08-25T09:53:14.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa84b376640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8440c4610 con 0x7fa84415c6d0 2023-08-25T09:53:14.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa849913640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 0x7fa8440ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa849913640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 0x7fa8440ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54544/0 (socket says 172.21.15.70:54544) 2023-08-25T09:53:14.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.425+0000 7fa849913640 1 -- 172.21.15.70:0/572519585 learned_addr learned my addr 172.21.15.70:0/572519585 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:14.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa848911640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84415e830 0x7fa8440c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849112640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415c6d0 0x7fa8440c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849913640 1 -- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415c6d0 msgr2=0x7fa8440c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849913640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415c6d0 0x7fa8440c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849913640 1 -- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84415e830 msgr2=0x7fa8440c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849913640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84415e830 0x7fa8440c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849913640 1 -- 172.21.15.70:0/572519585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009400 con 0x7fa84415ec10 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849112640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415c6d0 0x7fa8440c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa849913640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 0x7fa8440ce1d0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c750 tx=0x7fa83c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c005ad0 con 0x7fa84415ec10 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c005c50 con 0x7fa84415ec10 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8440cead0 con 0x7fa84415ec10 2023-08-25T09:53:14.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c007070 con 0x7fa84415ec10 2023-08-25T09:53:14.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.426+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8440cedb0 con 0x7fa84415ec10 2023-08-25T09:53:14.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.427+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8440c3db0 con 0x7fa84415ec10 2023-08-25T09:53:14.432 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.428+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa83c0071d0 con 0x7fa84415ec10 2023-08-25T09:53:14.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.429+0000 7fa839ffb640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa82806a050 0x7fa82806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.429+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa83c087a90 con 0x7fa84415ec10 2023-08-25T09:53:14.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.430+0000 7fa849112640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa82806a050 0x7fa82806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.433 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.430+0000 7fa849112640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa82806a050 0x7fa82806c510 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa840007fd0 tx=0x7fa840009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.436 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.433+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa83c053490 con 0x7fa84415ec10 2023-08-25T09:53:14.628 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.625+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84406b480 con 0x7fa82806a050 2023-08-25T09:53:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.626+0000 7fa839ffb640 1 -- 172.21.15.70:0/572519585 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa840001b50 con 0x7fa82806a050 2023-08-25T09:53:14.629 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 2 to deep-scrub 2023-08-25T09:53:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa82806a050 msgr2=0x7fa82806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa82806a050 0x7fa82806c510 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa840007fd0 tx=0x7fa840009040 comp rx=0 tx=0).stop 2023-08-25T09:53:14.631 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 msgr2=0x7fa8440ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 0x7fa8440ce1d0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00c750 tx=0x7fa83c00cc10 comp rx=0 tx=0).stop 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 shutdown_connections 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fa82806a050 0x7fa82806c510 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fa84415ec10 0x7fa8440ce1d0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fa84415e830 0x7fa8440c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 --2- 172.21.15.70:0/572519585 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fa84415c6d0 0x7fa8440c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.632 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 >> 172.21.15.70:0/572519585 conn(0x7fa844069850 msgr2=0x7fa8440b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 shutdown_connections 2023-08-25T09:53:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.629+0000 7fa84b376640 1 -- 172.21.15.70:0/572519585 wait complete. 2023-08-25T09:53:14.649 INFO:tasks.ceph:Scrubbing osd.3 2023-08-25T09:53:14.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:14.873 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.871+0000 7fac21e71640 1 Processor -- start 2023-08-25T09:53:14.874 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.871+0000 7fac21e71640 1 -- start start 2023-08-25T09:53:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac21e71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 0x7fac1c0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac21e71640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c15d320 con 0x7fac1c142c30 2023-08-25T09:53:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac21e71640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c0ccd40 con 0x7fac1c06aa70 2023-08-25T09:53:14.875 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac21e71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0cd020 con 0x7fac1c142740 2023-08-25T09:53:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac13fff640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fac1c142c30 0x7fac1c15db00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46308/0 (socket says 172.21.15.70:46308) 2023-08-25T09:53:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac13fff640 1 -- 172.21.15.70:0/7367172 learned_addr learned my addr 172.21.15.70:0/7367172 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac1b7fe640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 0x7fac1c0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.873+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027692440 0 0) 0x7fac1c15d320 con 0x7fac1c142c30 2023-08-25T09:53:14.877 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac08003520 con 0x7fac1c142c30 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421617847 0 0) 0x7fac08003520 con 0x7fac1c142c30 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 msgr2=0x7fac1c0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 0x7fac1c0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 >> v1:172.21.15.155:6789/0 conn(0x7fac1c06aa70 legacy=0x7fac1c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0cd300 con 0x7fac1c142c30 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fac04001670 con 0x7fac1c142c30 2023-08-25T09:53:14.878 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 >> v1:172.21.15.70:6789/0 conn(0x7fac1c142c30 legacy=0x7fac1c15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 0x7fac080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac08003fd0 0x7fac080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac08004d00 0x7fac08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c0cd020 con 0x7fac08004d00 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0800bb40 con 0x7fac08003fd0 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0800df10 con 0x7fac1c142740 2023-08-25T09:53:14.879 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.874+0000 7fac1bfff640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac08004d00 0x7fac08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.875+0000 7fac1b7fe640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 0x7fac080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.875+0000 7fac1bfff640 1 -- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 msgr2=0x7fac080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.875+0000 7fac1bfff640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac1c142740 0x7fac080039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.875+0000 7fac1bfff640 1 -- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac08003fd0 msgr2=0x7fac080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.875+0000 7fac1bfff640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac08003fd0 0x7fac080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.875+0000 7fac1bfff640 1 -- 172.21.15.70:0/7367172 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac08003520 con 0x7fac08004d00 2023-08-25T09:53:14.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.876+0000 7fac1bfff640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac08004d00 0x7fac08005180 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fac14001e50 tx=0x7fac080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7dfd20a0f308e47 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.876+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14003b70 con 0x7fac08004d00 2023-08-25T09:53:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.876+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14003cd0 con 0x7fac08004d00 2023-08-25T09:53:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.876+0000 7fac1affd640 1 -- 172.21.15.70:0/7367172 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14005310 con 0x7fac08004d00 2023-08-25T09:53:14.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac21e71640 1 -- 172.21.15.70:0/7367172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac08004d00 msgr2=0x7fac08005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac21e71640 1 --2- 172.21.15.70:0/7367172 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac08004d00 0x7fac08005180 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fac14001e50 tx=0x7fac080057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac1bfff640 1 -- 172.21.15.70:0/7367172 reap_dead start 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac21e71640 1 -- 172.21.15.70:0/7367172 shutdown_connections 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac21e71640 1 -- 172.21.15.70:0/7367172 >> 172.21.15.70:0/7367172 conn(0x7fac1c069c70 msgr2=0x7fac1c06a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac21e71640 1 -- 172.21.15.70:0/7367172 shutdown_connections 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.877+0000 7fac21e71640 1 -- 172.21.15.70:0/7367172 wait complete. 2023-08-25T09:53:14.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.879+0000 7fac21e71640 1 Processor -- start 2023-08-25T09:53:14.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.879+0000 7fac21e71640 1 -- start start 2023-08-25T09:53:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac21e71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac08004d00 0x7fac1c15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac21e71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0b0030 0x7fac1c138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac21e71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac1c141650 0x7fac1c141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac21e71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c136060 con 0x7fac1c141650 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac21e71640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c15cbe0 con 0x7fac1c0b0030 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac21e71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c142010 con 0x7fac08004d00 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac1c141650 0x7fac1c141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac1c141650 0x7fac1c141ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54568/0 (socket says 172.21.15.70:54568) 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 -- 172.21.15.70:0/3901275579 learned_addr learned my addr 172.21.15.70:0/3901275579 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1b7fe640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac08004d00 0x7fac1c15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 -- 172.21.15.70:0/3901275579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac08004d00 msgr2=0x7fac1c15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac08004d00 0x7fac1c15c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 -- 172.21.15.70:0/3901275579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0b0030 msgr2=0x7fac1c138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac1c0b0030 0x7fac1c138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.880+0000 7fac1bfff640 1 -- 172.21.15.70:0/3901275579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c135290 con 0x7fac1c141650 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.881+0000 7fac1bfff640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac1c141650 0x7fac1c141ad0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fac14001f70 tx=0x7fac140014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.881+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1403e070 con 0x7fac1c141650 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.881+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14003e40 con 0x7fac1c141650 2023-08-25T09:53:14.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.881+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14006d90 con 0x7fac1c141650 2023-08-25T09:53:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.881+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c1354e0 con 0x7fac1c141650 2023-08-25T09:53:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.881+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c135aa0 con 0x7fac1c141650 2023-08-25T09:53:14.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.882+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac1c143450 con 0x7fac1c141650 2023-08-25T09:53:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.883+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fac14007030 con 0x7fac1c141650 2023-08-25T09:53:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.884+0000 7fac18ff9640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fabfc06a080 0x7fabfc06c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.884+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fac140a9610 con 0x7fac1c141650 2023-08-25T09:53:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.884+0000 7fac18ff9640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7fabfc070500 0x7fabfc072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:14.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.884+0000 7fac1b7fe640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fabfc06a080 0x7fabfc06c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.885+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 --> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabfc072fd0 con 0x7fabfc070500 2023-08-25T09:53:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.885+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac140abbf0 con 0x7fac1c141650 2023-08-25T09:53:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.885+0000 7fac13fff640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7fabfc070500 0x7fabfc072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.885+0000 7fac1b7fe640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fabfc06a080 0x7fabfc06c540 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fac0c006fd0 tx=0x7fac0c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.885+0000 7fac13fff640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7fabfc070500 0x7fabfc072920 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:14.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.886+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== osd.3 v2:172.21.15.70:6824/2671981919 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fabfc072fd0 con 0x7fabfc070500 2023-08-25T09:53:14.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.899+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 --> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fac1c15d5e0 con 0x7fabfc070500 2023-08-25T09:53:14.901 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.899+0000 7fac18ff9640 1 -- 172.21.15.70:0/3901275579 <== osd.3 v2:172.21.15.70:6824/2671981919 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7fac04007370 con 0x7fabfc070500 2023-08-25T09:53:14.902 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:14.902 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:14.902 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.899+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7fabfc070500 msgr2=0x7fabfc072920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6824/2671981919,v1:172.21.15.70:6825/2671981919] conn(0x7fabfc070500 0x7fabfc072920 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fabfc06a080 msgr2=0x7fabfc06c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fabfc06a080 0x7fabfc06c540 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fac0c006fd0 tx=0x7fac0c008040 comp rx=0 tx=0).stop 2023-08-25T09:53:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac1c141650 msgr2=0x7fac1c141ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:14.903 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 --2- 172.21.15.70:0/3901275579 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac1c141650 0x7fac1c141ad0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fac14001f70 tx=0x7fac140014a0 comp rx=0 tx=0).stop 2023-08-25T09:53:14.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac1bfff640 1 -- 172.21.15.70:0/3901275579 reap_dead start 2023-08-25T09:53:14.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 shutdown_connections 2023-08-25T09:53:14.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 >> 172.21.15.70:0/3901275579 conn(0x7fac1c069c70 msgr2=0x7fac1c0b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:14.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 shutdown_connections 2023-08-25T09:53:14.904 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:14.900+0000 7fac21e71640 1 -- 172.21.15.70:0/3901275579 wait complete. 2023-08-25T09:53:14.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-08-25T09:53:15.136 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.134+0000 7f5f5b9ab640 1 Processor -- start 2023-08-25T09:53:15.137 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.134+0000 7f5f5b9ab640 1 -- start start 2023-08-25T09:53:15.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.135+0000 7f5f5b9ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 0x7f5f5406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.135+0000 7f5f5b9ab640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f5f5406e0c0 0x7f5f54166760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.135+0000 7f5f5b9ab640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f5415cfe0 con 0x7f5f540cf580 2023-08-25T09:53:15.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f5b9ab640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f540cfdb0 con 0x7f5f5406e0c0 2023-08-25T09:53:15.138 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f5b9ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54069850 con 0x7f5f5415cc00 2023-08-25T09:53:15.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f58f46640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f5f540cf580 0x7f5f5406d9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46314/0 (socket says 172.21.15.70:46314) 2023-08-25T09:53:15.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f58f46640 1 -- 172.21.15.70:0/1799094659 learned_addr learned my addr 172.21.15.70:0/1799094659 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:15.140 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f59f48640 1 --2- 172.21.15.70:0/1799094659 >> v2:172.21.15.155:3300/0 conn(0x7f5f5406e0c0 0x7f5f54166760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f59747640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 0x7f5f5406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863732215 0 0) 0x7f5f5415cfe0 con 0x7f5f540cf580 2023-08-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.136+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f40003520 con 0x7f5f540cf580 2023-08-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1345024616 0 0) 0x7f5f40003520 con 0x7f5f540cf580 2023-08-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 msgr2=0x7f5f5406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.141 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 0x7f5f5406d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 >> v2:172.21.15.155:3300/0 conn(0x7f5f5406e0c0 msgr2=0x7f5f54166760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 --2- 172.21.15.70:0/1799094659 >> v2:172.21.15.155:3300/0 conn(0x7f5f5406e0c0 0x7f5f54166760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f54069b30 con 0x7f5f540cf580 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f5f44001670 con 0x7f5f540cf580 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 >> v1:172.21.15.70:6789/0 conn(0x7f5f540cf580 legacy=0x7f5f5406d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 0x7f5f40003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.142 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f40003ef0 0x7f5f400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f40004d00 0x7f5f40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540cfdb0 con 0x7f5f40004d00 2023-08-25T09:53:15.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f54069850 con 0x7f5f40003ef0 2023-08-25T09:53:15.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4000b980 con 0x7f5f5415cc00 2023-08-25T09:53:15.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.137+0000 7f5f59f48640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f40004d00 0x7f5f40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.143 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f59747640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 0x7f5f40003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f58f46640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f40003ef0 0x7f5f400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f59f48640 1 -- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 msgr2=0x7f5f40003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f59f48640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f5415cc00 0x7f5f40003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f59f48640 1 -- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f40003ef0 msgr2=0x7f5f400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f59f48640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f40003ef0 0x7f5f400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.144 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.138+0000 7f5f59f48640 1 -- 172.21.15.70:0/1799094659 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40003520 con 0x7f5f40004d00 2023-08-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.139+0000 7f5f59f48640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f40004d00 0x7f5f40005180 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009f90 tx=0x7f5f400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc707034a2254e0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.139+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c02cc00 con 0x7f5f40004d00 2023-08-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.139+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4c00f040 con 0x7f5f40004d00 2023-08-25T09:53:15.148 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.139+0000 7f5f4bfff640 1 -- 172.21.15.70:0/1799094659 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c034440 con 0x7f5f40004d00 2023-08-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1799094659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f40004d00 msgr2=0x7f5f40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1799094659 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f40004d00 0x7f5f40005180 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009f90 tx=0x7f5f400057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f59f48640 1 -- 172.21.15.70:0/1799094659 reap_dead start 2023-08-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1799094659 shutdown_connections 2023-08-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1799094659 >> 172.21.15.70:0/1799094659 conn(0x7f5f540d0860 msgr2=0x7f5f540d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:15.149 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1799094659 shutdown_connections 2023-08-25T09:53:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.140+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1799094659 wait complete. 2023-08-25T09:53:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.143+0000 7f5f5b9ab640 1 Processor -- start 2023-08-25T09:53:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.143+0000 7f5f5b9ab640 1 -- start start 2023-08-25T09:53:15.150 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f5b9ab640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f40004d00 0x7f5f540ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f5b9ab640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 0x7f5f541588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f5b9ab640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f54159690 0x7f5f540b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f5b9ab640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f541e2fe0 con 0x7f5f54159e40 2023-08-25T09:53:15.151 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f5b9ab640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540cd3c0 con 0x7f5f54159690 2023-08-25T09:53:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f5b9ab640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f540cd6a0 con 0x7f5f40004d00 2023-08-25T09:53:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.144+0000 7f5f58f46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 0x7f5f541588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.152 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 0x7f5f541588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54582/0 (socket says 172.21.15.70:54582) 2023-08-25T09:53:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 -- 172.21.15.70:0/1921104356 learned_addr learned my addr 172.21.15.70:0/1921104356 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f59747640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f40004d00 0x7f5f540ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f59f48640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f54159690 0x7f5f540b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 -- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f40004d00 msgr2=0x7f5f540ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.153 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f40004d00 0x7f5f540ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 -- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f54159690 msgr2=0x7f5f540b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.154 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f54159690 0x7f5f540b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 -- 172.21.15.70:0/1921104356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f540cd980 con 0x7f5f54159e40 2023-08-25T09:53:15.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.145+0000 7f5f58f46640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 0x7f5f541588e0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e840 tx=0x7f5f4400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.156 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.146+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400c8d0 con 0x7f5f54159e40 2023-08-25T09:53:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.146+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4400ca50 con 0x7f5f54159e40 2023-08-25T09:53:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.146+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f4400ce00 con 0x7f5f54159e40 2023-08-25T09:53:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.146+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f540b46e0 con 0x7f5f54159e40 2023-08-25T09:53:15.157 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.146+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f540b4390 con 0x7f5f54159e40 2023-08-25T09:53:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.147+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f540b4550 con 0x7f5f54159e40 2023-08-25T09:53:15.158 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.148+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5f4401d460 con 0x7f5f54159e40 2023-08-25T09:53:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.149+0000 7f5f49ffb640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3806a050 0x7f5f3806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.149+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5f44088390 con 0x7f5f54159e40 2023-08-25T09:53:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.149+0000 7f5f59747640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3806a050 0x7f5f3806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.150+0000 7f5f59747640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3806a050 0x7f5f3806c510 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009a00 tx=0x7f5f50008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.159 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.153+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5f44053d00 con 0x7f5f54159e40 2023-08-25T09:53:15.343 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.341+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f5406a0f0 con 0x7f5f3806a050 2023-08-25T09:53:15.344 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.342+0000 7f5f49ffb640 1 -- 172.21.15.70:0/1921104356 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5f50004030 con 0x7f5f3806a050 2023-08-25T09:53:15.344 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 3 to deep-scrub 2023-08-25T09:53:15.347 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3806a050 msgr2=0x7f5f3806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3806a050 0x7f5f3806c510 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5f50009a00 tx=0x7f5f50008040 comp rx=0 tx=0).stop 2023-08-25T09:53:15.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 msgr2=0x7f5f541588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 0x7f5f541588e0 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400e840 tx=0x7f5f4400ed00 comp rx=0 tx=0).stop 2023-08-25T09:53:15.348 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 shutdown_connections 2023-08-25T09:53:15.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f5f3806a050 0x7f5f3806c510 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.344+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f5f54159690 0x7f5f540b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.349 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.345+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f5f54159e40 0x7f5f541588e0 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.345+0000 7f5f5b9ab640 1 --2- 172.21.15.70:0/1921104356 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f5f40004d00 0x7f5f540ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.345+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 >> 172.21.15.70:0/1921104356 conn(0x7f5f540d0860 msgr2=0x7f5f540b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:15.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.345+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 shutdown_connections 2023-08-25T09:53:15.350 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.345+0000 7f5f5b9ab640 1 -- 172.21.15.70:0/1921104356 wait complete. 2023-08-25T09:53:15.366 INFO:tasks.ceph:Scrubbing osd.4 2023-08-25T09:53:15.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:15.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.583+0000 7f0d15779640 1 Processor -- start 2023-08-25T09:53:15.585 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.583+0000 7f0d15779640 1 -- start start 2023-08-25T09:53:15.586 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d15779640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f0d100d23a0 0x7f0d100cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d15779640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d100cd5e0 0x7f0d100d29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d15779640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d100cdad0 con 0x7f0d1015cd60 2023-08-25T09:53:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d15779640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d100d2810 con 0x7f0d100d23a0 2023-08-25T09:53:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d15779640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100d0d30 con 0x7f0d100cd5e0 2023-08-25T09:53:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d0effd640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f0d1015cd60 0x7f0d100d0620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46328/0 (socket says 172.21.15.70:46328) 2023-08-25T09:53:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.584+0000 7f0d0effd640 1 -- 172.21.15.70:0/2326523814 learned_addr learned my addr 172.21.15.70:0/2326523814 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908225634 0 0) 0x7f0d100cdad0 con 0x7f0d1015cd60 2023-08-25T09:53:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cf8003520 con 0x7f0d1015cd60 2023-08-25T09:53:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0f7fe640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d100cd5e0 0x7f0d100d29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2326523814 >> v2:172.21.15.155:3300/0 conn(0x7f0d100d23a0 0x7f0d100cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536288771 0 0) 0x7f0cf8003520 con 0x7f0d1015cd60 2023-08-25T09:53:15.589 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d100cd5e0 msgr2=0x7f0d100d29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d100cd5e0 0x7f0d100d29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 >> v2:172.21.15.155:3300/0 conn(0x7f0d100d23a0 msgr2=0x7f0d100cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 --2- 172.21.15.70:0/2326523814 >> v2:172.21.15.155:3300/0 conn(0x7f0d100d23a0 0x7f0d100cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d100b8370 con 0x7f0d1015cd60 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f0d00001670 con 0x7f0d1015cd60 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 >> v1:172.21.15.70:6789/0 conn(0x7f0d1015cd60 legacy=0x7f0d100d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.590 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cf80039a0 0x7f0cf8003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0cf8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cf8005180 0x7f0cf800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100d0d30 con 0x7f0cf80047c0 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf800bec0 con 0x7f0cf8005180 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.585+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf800c000 con 0x7f0cf80039a0 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0cf8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.591 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0effd640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cf80039a0 0x7f0cf8003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0e7fc640 1 -- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cf80039a0 msgr2=0x7f0cf8003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cf80039a0 0x7f0cf8003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0e7fc640 1 -- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cf8005180 msgr2=0x7f0cf800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0cf8005180 0x7f0cf800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0e7fc640 1 -- 172.21.15.70:0/2326523814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8003520 con 0x7f0cf80047c0 2023-08-25T09:53:15.592 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.586+0000 7f0d0effd640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0cf80039a0 0x7f0cf8003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.587+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0cf8004c40 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ba30 tx=0x7f0cf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f95e451752e40f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.587+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04032680 con 0x7f0cf80047c0 2023-08-25T09:53:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d040327e0 con 0x7f0cf80047c0 2023-08-25T09:53:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d0dffb640 1 -- 172.21.15.70:0/2326523814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04032b70 con 0x7f0cf80047c0 2023-08-25T09:53:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d15779640 1 -- 172.21.15.70:0/2326523814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 msgr2=0x7f0cf8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.594 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d15779640 1 --2- 172.21.15.70:0/2326523814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0cf8004c40 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ba30 tx=0x7f0cf80057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d0f7fe640 1 -- 172.21.15.70:0/2326523814 reap_dead start 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d15779640 1 -- 172.21.15.70:0/2326523814 shutdown_connections 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d15779640 1 -- 172.21.15.70:0/2326523814 >> 172.21.15.70:0/2326523814 conn(0x7f0d10070290 msgr2=0x7f0d100706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d15779640 1 -- 172.21.15.70:0/2326523814 shutdown_connections 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.588+0000 7f0d15779640 1 -- 172.21.15.70:0/2326523814 wait complete. 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.590+0000 7f0d15779640 1 Processor -- start 2023-08-25T09:53:15.595 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.590+0000 7f0d15779640 1 -- start start 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d15779640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0d101f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d15779640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d1015f9e0 0x7f0d100b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d15779640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d1015a8b0 0x7f0d1015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d15779640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100a9890 con 0x7f0cf80047c0 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d15779640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d100b1140 con 0x7f0d1015a8b0 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d15779640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1015b280 con 0x7f0d1015f9e0 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d0effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0d101f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.596 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d0effd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0d101f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54596/0 (socket says 172.21.15.70:54596) 2023-08-25T09:53:15.597 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.591+0000 7f0d0effd640 1 -- 172.21.15.70:0/2787356071 learned_addr learned my addr 172.21.15.70:0/2787356071 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:15.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d0effd640 1 -- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d1015f9e0 msgr2=0x7f0d100b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d0effd640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f0d1015f9e0 0x7f0d100b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d0effd640 1 -- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d1015a8b0 msgr2=0x7f0d1015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d0effd640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f0d1015a8b0 0x7f0d1015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.598 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d0effd640 1 -- 172.21.15.70:0/2787356071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d101f63c0 con 0x7f0cf80047c0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d0effd640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0d101f5e80 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000d910 tx=0x7f0d0000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000074e0 con 0x7f0cf80047c0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d00001d70 con 0x7f0cf80047c0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d000049e0 con 0x7f0cf80047c0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.592+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1015eac0 con 0x7f0cf80047c0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.593+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1015f080 con 0x7f0cf80047c0 2023-08-25T09:53:15.599 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.594+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0d100b8630 con 0x7f0cf80047c0 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.595+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0d00004b40 con 0x7f0cf80047c0 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.595+0000 7f0cf77fe640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0cfc06a050 0x7f0cfc06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.596+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0d00087c30 con 0x7f0cf80047c0 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.596+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0cfc06a050 0x7f0cfc06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.596+0000 7f0cf77fe640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7f0cfc0704d0 0x7f0cfc0728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.596+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 --> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cfc072fa0 con 0x7f0cfc0704d0 2023-08-25T09:53:15.600 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.596+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d0008c2a0 con 0x7f0cf80047c0 2023-08-25T09:53:15.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.596+0000 7f0d0e7fc640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0cfc06a050 0x7f0cfc06c510 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ba00 tx=0x7f0d04032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.597+0000 7f0d0f7fe640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7f0cfc0704d0 0x7f0cfc0728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.601 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.598+0000 7f0d0f7fe640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7f0cfc0704d0 0x7f0cfc0728f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.602 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.599+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== osd.4 v2:172.21.15.155:6800/4222826791 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f0cfc072fa0 con 0x7f0cfc0704d0 2023-08-25T09:53:15.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.613+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 --> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0d10161510 con 0x7f0cfc0704d0 2023-08-25T09:53:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0cf77fe640 1 -- 172.21.15.70:0/2787356071 <== osd.4 v2:172.21.15.155:6800/4222826791 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f0d0800f040 con 0x7f0cfc0704d0 2023-08-25T09:53:15.616 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:15.616 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:15.617 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:15.617 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7f0cfc0704d0 msgr2=0x7f0cfc0728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.155:6800/4222826791,v1:172.21.15.155:6801/4222826791] conn(0x7f0cfc0704d0 0x7f0cfc0728f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0cfc06a050 msgr2=0x7f0cfc06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f0cfc06a050 0x7f0cfc06c510 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400ba00 tx=0x7f0d04032000 comp rx=0 tx=0).stop 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 msgr2=0x7f0d101f5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 --2- 172.21.15.70:0/2787356071 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f0cf80047c0 0x7f0d101f5e80 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000d910 tx=0x7f0d0000ddd0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d0f7fe640 1 -- 172.21.15.70:0/2787356071 reap_dead start 2023-08-25T09:53:15.618 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 shutdown_connections 2023-08-25T09:53:15.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 >> 172.21.15.70:0/2787356071 conn(0x7f0d10070290 msgr2=0x7f0d1015bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:15.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.614+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 shutdown_connections 2023-08-25T09:53:15.619 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.615+0000 7f0d15779640 1 -- 172.21.15.70:0/2787356071 wait complete. 2023-08-25T09:53:15.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-08-25T09:53:15.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.841+0000 7fe6bb40d640 1 Processor -- start 2023-08-25T09:53:15.843 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.841+0000 7fe6bb40d640 1 -- start start 2023-08-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6bb40d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe6b40b2850 0x7fe6b4164780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6bb40d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b415d3f0 con 0x7fe6b40ce5b0 2023-08-25T09:53:15.845 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6bb40d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b4070a00 con 0x7fe6b415aa10 2023-08-25T09:53:15.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6bb40d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b4159d10 con 0x7fe6b40b2850 2023-08-25T09:53:15.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6b91a9640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe6b40ce5b0 0x7fe6b40ce990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46342/0 (socket says 172.21.15.70:46342) 2023-08-25T09:53:15.846 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6b91a9640 1 -- 172.21.15.70:0/299410113 learned_addr learned my addr 172.21.15.70:0/299410113 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.843+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944491307 0 0) 0x7fe6b415d3f0 con 0x7fe6b40ce5b0 2023-08-25T09:53:15.847 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a0003520 con 0x7fe6b40ce5b0 2023-08-25T09:53:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6b99aa640 1 --2- 172.21.15.70:0/299410113 >> v2:172.21.15.155:3300/0 conn(0x7fe6b40b2850 0x7fe6b4164780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3430870637 0 0) 0x7fe6a0003520 con 0x7fe6b40ce5b0 2023-08-25T09:53:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 >> v2:172.21.15.155:3300/0 conn(0x7fe6b40b2850 msgr2=0x7fe6b4164780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 --2- 172.21.15.70:0/299410113 >> v2:172.21.15.155:3300/0 conn(0x7fe6b40b2850 0x7fe6b4164780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 >> v1:172.21.15.155:6789/0 conn(0x7fe6b415aa10 legacy=0x7fe6b40b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.848 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40b2d20 con 0x7fe6b40ce5b0 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe6a4001670 con 0x7fe6b40ce5b0 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 >> v1:172.21.15.70:6789/0 conn(0x7fe6b40ce5b0 legacy=0x7fe6b40ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6a0003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a00047c0 0x7fe6a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6a00051d0 0x7fe6a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4159d10 con 0x7fe6a0003970 2023-08-25T09:53:15.849 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a000c100 con 0x7fe6a00047c0 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a000c2c0 con 0x7fe6a00051d0 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.844+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6a0003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b89a8640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a00047c0 0x7fe6a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b91a9640 1 -- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6a00051d0 msgr2=0x7fe6a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6a00051d0 0x7fe6a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b91a9640 1 -- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a00047c0 msgr2=0x7fe6a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6a00047c0 0x7fe6a0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.850 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b91a9640 1 -- 172.21.15.70:0/299410113 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a0003520 con 0x7fe6a0003970 2023-08-25T09:53:15.851 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.845+0000 7fe6b99aa640 1 -- 172.21.15.70:0/299410113 reap_dead start 2023-08-25T09:53:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.846+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6a0003db0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c240 tx=0x7fe6a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0d9961516d022d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.846+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4005ad0 con 0x7fe6a0003970 2023-08-25T09:53:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.846+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4005c30 con 0x7fe6a0003970 2023-08-25T09:53:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.846+0000 7fe6abfff640 1 -- 172.21.15.70:0/299410113 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a402cab0 con 0x7fe6a0003970 2023-08-25T09:53:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 -- 172.21.15.70:0/299410113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 msgr2=0x7fe6a0003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.852 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6a0003db0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c240 tx=0x7fe6a00057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 -- 172.21.15.70:0/299410113 shutdown_connections 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/299410113 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6a0003db0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 -- 172.21.15.70:0/299410113 >> 172.21.15.70:0/299410113 conn(0x7fe6b4069850 msgr2=0x7fe6b4069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 -- 172.21.15.70:0/299410113 shutdown_connections 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.847+0000 7fe6bb40d640 1 -- 172.21.15.70:0/299410113 wait complete. 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.849+0000 7fe6bb40d640 1 Processor -- start 2023-08-25T09:53:15.853 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.849+0000 7fe6bb40d640 1 -- start start 2023-08-25T09:53:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6bb40d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6b413e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6bb40d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6b413b530 0x7fe6b40ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6bb40d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6b40ae7e0 0x7fe6b40ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6bb40d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b413e2b0 con 0x7fe6a0003970 2023-08-25T09:53:15.855 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6bb40d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40aed40 con 0x7fe6b413b530 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6bb40d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b413dc60 con 0x7fe6b40ae7e0 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6b91a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6b413e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6b91a9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6b413e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54618/0 (socket says 172.21.15.70:54618) 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.850+0000 7fe6b91a9640 1 -- 172.21.15.70:0/2365170654 learned_addr learned my addr 172.21.15.70:0/2365170654 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b89a8640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6b413b530 0x7fe6b40ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b99aa640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6b40ae7e0 0x7fe6b40ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.856 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b91a9640 1 -- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6b40ae7e0 msgr2=0x7fe6b40ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6b40ae7e0 0x7fe6b40ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b91a9640 1 -- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6b413b530 msgr2=0x7fe6b40ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6b413b530 0x7fe6b40ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b91a9640 1 -- 172.21.15.70:0/2365170654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b413df40 con 0x7fe6a0003970 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6b91a9640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6b413e740 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fe6a402c410 tx=0x7fe6a402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a403c040 con 0x7fe6a0003970 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a402cc80 con 0x7fe6a0003970 2023-08-25T09:53:15.857 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4007070 con 0x7fe6a0003970 2023-08-25T09:53:15.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.851+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b414f3a0 con 0x7fe6a0003970 2023-08-25T09:53:15.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.852+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b40acdb0 con 0x7fe6a0003970 2023-08-25T09:53:15.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.853+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b413e0d0 con 0x7fe6a0003970 2023-08-25T09:53:15.858 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.854+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe6a4032070 con 0x7fe6a0003970 2023-08-25T09:53:15.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.854+0000 7fe6a9ffb640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe69806a050 0x7fe69806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:15.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.855+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe6a40a9570 con 0x7fe6a0003970 2023-08-25T09:53:15.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.855+0000 7fe6b89a8640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe69806a050 0x7fe69806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:15.859 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.855+0000 7fe6b89a8640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe69806a050 0x7fe69806c510 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0000c00 tx=0x7fe6b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:15.861 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:15.859+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6a4075140 con 0x7fe6a0003970 2023-08-25T09:53:16.052 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.050+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6b415d990 con 0x7fe69806a050 2023-08-25T09:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.050+0000 7fe6a9ffb640 1 -- 172.21.15.70:0/2365170654 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe6b000d040 con 0x7fe69806a050 2023-08-25T09:53:16.053 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 4 to deep-scrub 2023-08-25T09:53:16.058 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe69806a050 msgr2=0x7fe69806c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe69806a050 0x7fe69806c510 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0000c00 tx=0x7fe6b0009040 comp rx=0 tx=0).stop 2023-08-25T09:53:16.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 msgr2=0x7fe6b413e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6b413e740 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fe6a402c410 tx=0x7fe6a402e070 comp rx=0 tx=0).stop 2023-08-25T09:53:16.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 shutdown_connections 2023-08-25T09:53:16.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe69806a050 0x7fe69806c510 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.059 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe6b40ae7e0 0x7fe6b40ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe6b413b530 0x7fe6b40ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 --2- 172.21.15.70:0/2365170654 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe6a0003970 0x7fe6b413e740 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 >> 172.21.15.70:0/2365170654 conn(0x7fe6b4069850 msgr2=0x7fe6b40b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 shutdown_connections 2023-08-25T09:53:16.060 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.053+0000 7fe6bb40d640 1 -- 172.21.15.70:0/2365170654 wait complete. 2023-08-25T09:53:16.074 INFO:tasks.ceph:Scrubbing osd.5 2023-08-25T09:53:16.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.288+0000 7f7767a71640 1 Processor -- start 2023-08-25T09:53:16.291 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.289+0000 7f7767a71640 1 -- start start 2023-08-25T09:53:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.290+0000 7f7767a71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77600caab0 0x7f77600caf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.290+0000 7f7767a71640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7760072820 0x7f7760072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.290+0000 7f7767a71640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77600731f0 con 0x7f776015ce50 2023-08-25T09:53:16.293 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.290+0000 7f7767a71640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77600ca620 con 0x7f7760072820 2023-08-25T09:53:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.290+0000 7f7767a71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7760070930 con 0x7f77600caab0 2023-08-25T09:53:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7760072820 0x7f7760072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f7760072820 0x7f7760072cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54628/0 (socket says 172.21.15.70:54628) 2023-08-25T09:53:16.294 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 -- 172.21.15.70:0/3808072076 learned_addr learned my addr 172.21.15.70:0/3808072076 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776500c640 1 --2- 172.21.15.70:0/3808072076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77600caab0 0x7f77600caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 -- 172.21.15.70:0/3808072076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77600caab0 msgr2=0x7f77600caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 --2- 172.21.15.70:0/3808072076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77600caab0 0x7f77600caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.295 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 -- 172.21.15.70:0/3808072076 >> v1:172.21.15.155:6789/0 conn(0x7f776015ce50 legacy=0x7f7760069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.291+0000 7f776600e640 1 -- 172.21.15.70:0/3808072076 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77600c9fd0 con 0x7f7760072820 2023-08-25T09:53:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.292+0000 7f776600e640 1 --2- 172.21.15.70:0/3808072076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f7760072cb0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f7758009870 tx=0x7f775802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bd2c196a439c20 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.293+0000 7f776480b640 1 -- 172.21.15.70:0/3808072076 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758031640 con 0x7f7760072820 2023-08-25T09:53:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.293+0000 7f776480b640 1 -- 172.21.15.70:0/3808072076 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77580317a0 con 0x7f7760072820 2023-08-25T09:53:16.296 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.293+0000 7f776480b640 1 -- 172.21.15.70:0/3808072076 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758031b30 con 0x7f7760072820 2023-08-25T09:53:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.293+0000 7f7767a71640 1 -- 172.21.15.70:0/3808072076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 msgr2=0x7f7760072cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.293+0000 7f7767a71640 1 --2- 172.21.15.70:0/3808072076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f7760072cb0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f7758009870 tx=0x7f775802b8f0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.293+0000 7f7767a71640 1 -- 172.21.15.70:0/3808072076 shutdown_connections 2023-08-25T09:53:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.294+0000 7f7767a71640 1 --2- 172.21.15.70:0/3808072076 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f7760072cb0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.297 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.294+0000 7f7767a71640 1 --2- 172.21.15.70:0/3808072076 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f77600caab0 0x7f77600caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.294+0000 7f7767a71640 1 -- 172.21.15.70:0/3808072076 >> 172.21.15.70:0/3808072076 conn(0x7f77600cf870 msgr2=0x7f77600cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.294+0000 7f7767a71640 1 -- 172.21.15.70:0/3808072076 shutdown_connections 2023-08-25T09:53:16.298 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.294+0000 7f7767a71640 1 -- 172.21.15.70:0/3808072076 wait complete. 2023-08-25T09:53:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.295+0000 7f7767a71640 1 Processor -- start 2023-08-25T09:53:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.296+0000 7f7767a71640 1 -- start start 2023-08-25T09:53:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.296+0000 7f7767a71640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f77600b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.299 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.296+0000 7f7767a71640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77600caab0 0x7f776015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f7767a71640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ce50 0x7f77600c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f7767a71640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776015ec90 con 0x7f7760072820 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f7767a71640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f776015a680 con 0x7f77600caab0 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f7767a71640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f776006ab70 con 0x7f776015ce50 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f77600b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f77600b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54640/0 (socket says 172.21.15.70:54640) 2023-08-25T09:53:16.300 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 -- 172.21.15.70:0/3470828605 learned_addr learned my addr 172.21.15.70:0/3470828605 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:16.301 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 -- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ce50 msgr2=0x7f77600c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f776015ce50 0x7f77600c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 -- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77600caab0 msgr2=0x7f776015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f77600caab0 0x7f776015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 -- 172.21.15.70:0/3470828605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580094d0 con 0x7f7760072820 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.297+0000 7f776580d640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f77600b37b0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f775000c7c0 tx=0x7f775000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.298+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000aa50 con 0x7f7760072820 2023-08-25T09:53:16.302 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.298+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000abb0 con 0x7f7760072820 2023-08-25T09:53:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.298+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750012420 con 0x7f7760072820 2023-08-25T09:53:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.298+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776006ae30 con 0x7f7760072820 2023-08-25T09:53:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.298+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776015f180 con 0x7f7760072820 2023-08-25T09:53:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.299+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77600ca2b0 con 0x7f7760072820 2023-08-25T09:53:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.300+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7750012580 con 0x7f7760072820 2023-08-25T09:53:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.301+0000 7f775e7fc640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774c06a050 0x7f774c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.301+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f77500882c0 con 0x7f7760072820 2023-08-25T09:53:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.301+0000 7f775e7fc640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f774c0704d0 0x7f774c0728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.301+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 --> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f774c072fa0 con 0x7f774c0704d0 2023-08-25T09:53:16.304 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.301+0000 7f776500c640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774c06a050 0x7f774c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.301+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7f775008a8a0 con 0x7f7760072820 2023-08-25T09:53:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.302+0000 7f776500c640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774c06a050 0x7f774c06c510 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7754006fd0 tx=0x7f7754008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.305 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.302+0000 7f776600e640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f774c0704d0 0x7f774c0728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.306 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.303+0000 7f776600e640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f774c0704d0 0x7f774c0728f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.307 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.305+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== osd.5 v2:172.21.15.155:6818/1708871082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f774c072fa0 con 0x7f774c0704d0 2023-08-25T09:53:16.320 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.318+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 --> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f776004e420 con 0x7f774c0704d0 2023-08-25T09:53:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f775e7fc640 1 -- 172.21.15.70:0/3470828605 <== osd.5 v2:172.21.15.155:6818/1708871082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f776004e420 con 0x7f774c0704d0 2023-08-25T09:53:16.321 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:16.321 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:16.322 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f774c0704d0 msgr2=0x7f774c0728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f7767a71640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.155:6818/1708871082,v1:172.21.15.155:6820/1708871082] conn(0x7f774c0704d0 0x7f774c0728f0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774c06a050 msgr2=0x7f774c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f7767a71640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f774c06a050 0x7f774c06c510 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7754006fd0 tx=0x7f7754008040 comp rx=0 tx=0).stop 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 msgr2=0x7f77600b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f7767a71640 1 --2- 172.21.15.70:0/3470828605 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f7760072820 0x7f77600b37b0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7f775000c7c0 tx=0x7f775000cc80 comp rx=0 tx=0).stop 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.319+0000 7f776600e640 1 -- 172.21.15.70:0/3470828605 reap_dead start 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.320+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 shutdown_connections 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.320+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 >> 172.21.15.70:0/3470828605 conn(0x7f77600cf870 msgr2=0x7f77600cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:16.323 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.320+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 shutdown_connections 2023-08-25T09:53:16.324 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.320+0000 7f7767a71640 1 -- 172.21.15.70:0/3470828605 wait complete. 2023-08-25T09:53:16.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-08-25T09:53:16.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.545+0000 7ff9bec4d640 1 Processor -- start 2023-08-25T09:53:16.548 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.546+0000 7ff9bec4d640 1 -- start start 2023-08-25T09:53:16.549 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.547+0000 7ff9bec4d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80714f0 0x7ff9b8069850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.547+0000 7ff9bec4d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9b815bc10 0x7ff9b8069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.547+0000 7ff9bec4d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b815b780 con 0x7ff9b80ce790 2023-08-25T09:53:16.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.547+0000 7ff9bec4d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b815af40 con 0x7ff9b815bc10 2023-08-25T09:53:16.550 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.547+0000 7ff9bec4d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80cb820 con 0x7ff9b80714f0 2023-08-25T09:53:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.548+0000 7ff9bd1ea640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9b815bc10 0x7ff9b8069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.548+0000 7ff9bd1ea640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7ff9b815bc10 0x7ff9b8069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.155:3300/0 says I am v2:172.21.15.70:53218/0 (socket says 172.21.15.70:53218) 2023-08-25T09:53:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.548+0000 7ff9bd1ea640 1 -- 172.21.15.70:0/3778599793 learned_addr learned my addr 172.21.15.70:0/3778599793 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.548+0000 7ff9b7fff640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80714f0 0x7ff9b8069850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.549+0000 7ff9bd1ea640 1 -- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80714f0 msgr2=0x7ff9b8069850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:16.551 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.549+0000 7ff9bd1ea640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80714f0 0x7ff9b8069850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.549+0000 7ff9bd1ea640 1 -- 172.21.15.70:0/3778599793 >> v1:172.21.15.155:6789/0 conn(0x7ff9b80ce790 legacy=0x7ff9b80caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.549+0000 7ff9bd1ea640 1 -- 172.21.15.70:0/3778599793 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80cb2b0 con 0x7ff9b815bc10 2023-08-25T09:53:16.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.549+0000 7ff9b7fff640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80714f0 0x7ff9b8069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:53:16.552 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.550+0000 7ff9bd1ea640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b8069d90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009870 tx=0x7ff9b002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe1cb57d252f8592 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.550+0000 7ff9b77fe640 1 -- 172.21.15.70:0/3778599793 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0030840 con 0x7ff9b815bc10 2023-08-25T09:53:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9b77fe640 1 -- 172.21.15.70:0/3778599793 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b00309a0 con 0x7ff9b815bc10 2023-08-25T09:53:16.553 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9b77fe640 1 -- 172.21.15.70:0/3778599793 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0030d30 con 0x7ff9b815bc10 2023-08-25T09:53:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 -- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 msgr2=0x7ff9b8069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b8069d90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009870 tx=0x7ff9b002b980 comp rx=0 tx=0).stop 2023-08-25T09:53:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 -- 172.21.15.70:0/3778599793 shutdown_connections 2023-08-25T09:53:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b8069d90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/3778599793 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80714f0 0x7ff9b8069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 -- 172.21.15.70:0/3778599793 >> 172.21.15.70:0/3778599793 conn(0x7ff9b80cef00 msgr2=0x7ff9b80cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 -- 172.21.15.70:0/3778599793 shutdown_connections 2023-08-25T09:53:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.551+0000 7ff9bec4d640 1 -- 172.21.15.70:0/3778599793 wait complete. 2023-08-25T09:53:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.553+0000 7ff9bec4d640 1 Processor -- start 2023-08-25T09:53:16.555 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.553+0000 7ff9bec4d640 1 -- start start 2023-08-25T09:53:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bec4d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 0x7ff9b80b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.556 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bec4d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80ce790 0x7ff9b815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bec4d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b815c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bec4d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815ce30 con 0x7ff9b80714f0 2023-08-25T09:53:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bec4d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815d110 con 0x7ff9b815bc10 2023-08-25T09:53:16.557 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bec4d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815d3f0 con 0x7ff9b80ce790 2023-08-25T09:53:16.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.554+0000 7ff9bc9e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 0x7ff9b80b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 0x7ff9b80b3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54644/0 (socket says 172.21.15.70:54644) 2023-08-25T09:53:16.558 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 -- 172.21.15.70:0/1507041814 learned_addr learned my addr 172.21.15.70:0/1507041814 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9b7fff640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80ce790 0x7ff9b815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bd1ea640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 -- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80ce790 msgr2=0x7ff9b815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80ce790 0x7ff9b815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 -- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 msgr2=0x7ff9b815c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b815c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.559 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 -- 172.21.15.70:0/1507041814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0009400 con 0x7ff9b80714f0 2023-08-25T09:53:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9b7fff640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80ce790 0x7ff9b815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.555+0000 7ff9bc9e9640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 0x7ff9b80b3d20 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800c870 tx=0x7ff9a800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.556+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8003b70 con 0x7ff9b80714f0 2023-08-25T09:53:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.556+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8003cf0 con 0x7ff9b80714f0 2023-08-25T09:53:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.556+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a800aa90 con 0x7ff9b80714f0 2023-08-25T09:53:16.560 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.556+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00094d0 con 0x7ff9b80714f0 2023-08-25T09:53:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.556+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b815cb50 con 0x7ff9b80714f0 2023-08-25T09:53:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.557+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b804e420 con 0x7ff9b80714f0 2023-08-25T09:53:16.561 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.558+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff9a800abf0 con 0x7ff9b80714f0 2023-08-25T09:53:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.559+0000 7ff9b57fa640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff99806a080 0x7ff99806c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.559+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff9a8088100 con 0x7ff9b80714f0 2023-08-25T09:53:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.559+0000 7ff9b7fff640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff99806a080 0x7ff99806c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:16.562 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.560+0000 7ff9b7fff640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff99806a080 0x7ff99806c540 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff9b81e4ca0 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:16.565 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.563+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff9a8053b00 con 0x7ff9b80714f0 2023-08-25T09:53:16.753 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.751+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9b815d280 con 0x7ff99806a080 2023-08-25T09:53:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.751+0000 7ff9b57fa640 1 -- 172.21.15.70:0/1507041814 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff9b815d280 con 0x7ff99806a080 2023-08-25T09:53:16.754 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 5 to deep-scrub 2023-08-25T09:53:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff99806a080 msgr2=0x7ff99806c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff99806a080 0x7ff99806c540 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff9b81e4ca0 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 msgr2=0x7ff9b80b3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 0x7ff9b80b3d20 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800c870 tx=0x7ff9a800cd30 comp rx=0 tx=0).stop 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 shutdown_connections 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff99806a080 0x7ff99806c540 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff9b815bc10 0x7ff9b815c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.757 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff9b80ce790 0x7ff9b815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 --2- 172.21.15.70:0/1507041814 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff9b80714f0 0x7ff9b80b3d20 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 >> 172.21.15.70:0/1507041814 conn(0x7ff9b80cef00 msgr2=0x7ff9b80caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 shutdown_connections 2023-08-25T09:53:16.758 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.754+0000 7ff9bec4d640 1 -- 172.21.15.70:0/1507041814 wait complete. 2023-08-25T09:53:16.775 INFO:tasks.ceph:Scrubbing osd.6 2023-08-25T09:53:16.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:16.997 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.995+0000 7f238d0ec640 1 Processor -- start 2023-08-25T09:53:16.998 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.995+0000 7f238d0ec640 1 -- start start 2023-08-25T09:53:16.999 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f238d0ec640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23880cda60 0x7f23880ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f238d0ec640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238815e6f0 con 0x7f2388155120 2023-08-25T09:53:17.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f238d0ec640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f238815e9d0 con 0x7f23880ca820 2023-08-25T09:53:17.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f238d0ec640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880caf30 con 0x7f23880cda60 2023-08-25T09:53:17.000 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f237ffff640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f23880cda60 0x7f23880ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f2386d9d640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f2388155120 0x7f23880cd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46344/0 (socket says 172.21.15.70:46344) 2023-08-25T09:53:17.001 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f2386d9d640 1 -- 172.21.15.70:0/3927943069 learned_addr learned my addr 172.21.15.70:0/3927943069 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:17.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f237ffff640 1 -- 172.21.15.70:0/3927943069 >> v1:172.21.15.155:6789/0 conn(0x7f23880ca820 legacy=0x7f238815af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.003 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.997+0000 7f237ffff640 1 -- 172.21.15.70:0/3927943069 >> v1:172.21.15.70:6789/0 conn(0x7f2388155120 legacy=0x7f23880cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.998+0000 7f238659c640 1 -- 172.21.15.70:0/3927943069 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682425597 0 0) 0x7f238815e6f0 con 0x7f2388155120 2023-08-25T09:53:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.998+0000 7f237ffff640 1 -- 172.21.15.70:0/3927943069 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238815ecb0 con 0x7f23880cda60 2023-08-25T09:53:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.999+0000 7f237ffff640 1 --2- 172.21.15.70:0/3927943069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ca2e0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f23700097a0 tx=0x7f237002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39243c9b8f7dd55c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.004 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.999+0000 7f238659c640 1 -- 172.21.15.70:0/3927943069 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370031830 con 0x7f23880cda60 2023-08-25T09:53:17.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.999+0000 7f238659c640 1 -- 172.21.15.70:0/3927943069 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2370031990 con 0x7f23880cda60 2023-08-25T09:53:17.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.999+0000 7f238659c640 1 -- 172.21.15.70:0/3927943069 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370031d20 con 0x7f23880cda60 2023-08-25T09:53:17.005 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.999+0000 7f238d0ec640 1 -- 172.21.15.70:0/3927943069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 msgr2=0x7f23880ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:16.999+0000 7f238d0ec640 1 --2- 172.21.15.70:0/3927943069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ca2e0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f23700097a0 tx=0x7f237002b8d0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.000+0000 7f238d0ec640 1 -- 172.21.15.70:0/3927943069 shutdown_connections 2023-08-25T09:53:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.000+0000 7f238d0ec640 1 --2- 172.21.15.70:0/3927943069 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ca2e0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.006 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.000+0000 7f238d0ec640 1 -- 172.21.15.70:0/3927943069 >> 172.21.15.70:0/3927943069 conn(0x7f2388069c70 msgr2=0x7f238806a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.000+0000 7f238d0ec640 1 -- 172.21.15.70:0/3927943069 shutdown_connections 2023-08-25T09:53:17.007 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.000+0000 7f238d0ec640 1 -- 172.21.15.70:0/3927943069 wait complete. 2023-08-25T09:53:17.010 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.001+0000 7f238d0ec640 1 Processor -- start 2023-08-25T09:53:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 -- start start 2023-08-25T09:53:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23880ca820 0x7f2388143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ce640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.011 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2388155120 0x7f23880ceb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23881465f0 con 0x7f23880cda60 2023-08-25T09:53:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23881f21b0 con 0x7f2388155120 2023-08-25T09:53:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.002+0000 7f238d0ec640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23881f2490 con 0x7f23880ca820 2023-08-25T09:53:17.012 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54662/0 (socket says 172.21.15.70:54662) 2023-08-25T09:53:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 -- 172.21.15.70:0/2513951856 learned_addr learned my addr 172.21.15.70:0/2513951856 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 -- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23880ca820 msgr2=0x7f2388143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f23880ca820 0x7f2388143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 -- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2388155120 msgr2=0x7f23880ceb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f2388155120 0x7f23880ceb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.003+0000 7f237ffff640 1 -- 172.21.15.70:0/2513951856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2370009400 con 0x7f23880cda60 2023-08-25T09:53:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.004+0000 7f237ffff640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ce640 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f23700098c0 tx=0x7f23700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.004+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237003ba70 con 0x7f23880cda60 2023-08-25T09:53:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.004+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237003bbd0 con 0x7f23880cda60 2023-08-25T09:53:17.015 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.004+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23881f2770 con 0x7f23880cda60 2023-08-25T09:53:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.004+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700453f0 con 0x7f23880cda60 2023-08-25T09:53:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.004+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23881f2f70 con 0x7f23880cda60 2023-08-25T09:53:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.005+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23880cac30 con 0x7f23880cda60 2023-08-25T09:53:17.016 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.006+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f237004e450 con 0x7f23880cda60 2023-08-25T09:53:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.007+0000 7f237f7fe640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f236c06a050 0x7f236c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.007+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f23700aa560 con 0x7f23880cda60 2023-08-25T09:53:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.007+0000 7f237f7fe640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f236c0704d0 0x7f236c0728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.007+0000 7f2386d9d640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f236c06a050 0x7f236c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.007+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 --> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f236c072fa0 con 0x7f236c0704d0 2023-08-25T09:53:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.007+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23700acbf0 con 0x7f23880cda60 2023-08-25T09:53:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.008+0000 7f2386d9d640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f236c06a050 0x7f236c06c510 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2378007fd0 tx=0x7f2378007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.008+0000 7f238759e640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f236c0704d0 0x7f236c0728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.009+0000 7f238759e640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f236c0704d0 0x7f236c0728f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.020 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.011+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== osd.6 v2:172.21.15.155:6816/4156222745 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f236c072fa0 con 0x7f236c0704d0 2023-08-25T09:53:17.026 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.023+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 --> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f238815ef90 con 0x7f236c0704d0 2023-08-25T09:53:17.027 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f237f7fe640 1 -- 172.21.15.70:0/2513951856 <== osd.6 v2:172.21.15.155:6816/4156222745 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f2380006b50 con 0x7f236c0704d0 2023-08-25T09:53:17.027 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:17.028 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:17.028 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:17.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f236c0704d0 msgr2=0x7f236c0728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.029 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.155:6816/4156222745,v1:172.21.15.155:6817/4156222745] conn(0x7f236c0704d0 0x7f236c0728f0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f236c06a050 msgr2=0x7f236c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.030 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f236c06a050 0x7f236c06c510 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2378007fd0 tx=0x7f2378007960 comp rx=0 tx=0).stop 2023-08-25T09:53:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 msgr2=0x7f23880ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 --2- 172.21.15.70:0/2513951856 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f23880cda60 0x7f23880ce640 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f23700098c0 tx=0x7f23700046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.031 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238759e640 1 -- 172.21.15.70:0/2513951856 reap_dead start 2023-08-25T09:53:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 shutdown_connections 2023-08-25T09:53:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 >> 172.21.15.70:0/2513951856 conn(0x7f2388069c70 msgr2=0x7f23880654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.032 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 shutdown_connections 2023-08-25T09:53:17.033 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.025+0000 7f238d0ec640 1 -- 172.21.15.70:0/2513951856 wait complete. 2023-08-25T09:53:17.048 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-08-25T09:53:17.263 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.261+0000 7fb1d9471640 1 Processor -- start 2023-08-25T09:53:17.264 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.261+0000 7fb1d9471640 1 -- start start 2023-08-25T09:53:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.263+0000 7fb1d9471640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 0x7fb1d40cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.265 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.263+0000 7fb1d9471640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d40b2130 con 0x7fb1d4071b40 2023-08-25T09:53:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.263+0000 7fb1d9471640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d40b1520 con 0x7fb1d415a840 2023-08-25T09:53:17.266 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.263+0000 7fb1d9471640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40b1800 con 0x7fb1d40cec20 2023-08-25T09:53:17.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.263+0000 7fb1d8c70640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fb1d4071b40 0x7fb1d415e650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46350/0 (socket says 172.21.15.70:46350) 2023-08-25T09:53:17.267 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.263+0000 7fb1d8c70640 1 -- 172.21.15.70:0/2790103894 learned_addr learned my addr 172.21.15.70:0/2790103894 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:17.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1cb7fe640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 0x7fb1d40cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942014339 0 0) 0x7fb1d40b2130 con 0x7fb1d4071b40 2023-08-25T09:53:17.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1c0003520 con 0x7fb1d4071b40 2023-08-25T09:53:17.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1196724854 0 0) 0x7fb1c0003520 con 0x7fb1d4071b40 2023-08-25T09:53:17.268 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 msgr2=0x7fb1d40cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 0x7fb1d40cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 >> v1:172.21.15.155:6789/0 conn(0x7fb1d415a840 legacy=0x7fb1d40ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40b1ae0 con 0x7fb1d4071b40 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fb1cc001670 con 0x7fb1d4071b40 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 >> v1:172.21.15.70:6789/0 conn(0x7fb1d4071b40 legacy=0x7fb1d415e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1c00039a0 0x7fb1c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c00047c0 0x7fb1c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.269 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 0x7fb1c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d40b1800 con 0x7fb1c00039a0 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c000bae0 con 0x7fb1c00047c0 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.264+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c000bca0 con 0x7fb1d40cec20 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1c00039a0 0x7fb1c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d3fff640 1 -- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 msgr2=0x7fb1c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d8c70640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 0x7fb1c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.270 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1d40cec20 0x7fb1c00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d3fff640 1 -- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c00047c0 msgr2=0x7fb1c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1c00047c0 0x7fb1c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.271 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.265+0000 7fb1d3fff640 1 -- 172.21.15.70:0/2790103894 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c0003520 con 0x7fb1c00039a0 2023-08-25T09:53:17.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.266+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1c00039a0 0x7fb1c0003dc0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4001e50 tx=0x7fb1c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ded7cc78dbd807 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.266+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4003b70 con 0x7fb1c00039a0 2023-08-25T09:53:17.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.266+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c4003cd0 con 0x7fb1c00039a0 2023-08-25T09:53:17.273 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.266+0000 7fb1d37fe640 1 -- 172.21.15.70:0/2790103894 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c4005310 con 0x7fb1c00039a0 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d9471640 1 -- 172.21.15.70:0/2790103894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1c00039a0 msgr2=0x7fb1c0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2790103894 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1c00039a0 0x7fb1c0003dc0 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4001e50 tx=0x7fb1c00057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d8c70640 1 -- 172.21.15.70:0/2790103894 reap_dead start 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d9471640 1 -- 172.21.15.70:0/2790103894 shutdown_connections 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d9471640 1 -- 172.21.15.70:0/2790103894 >> 172.21.15.70:0/2790103894 conn(0x7fb1d4069eb0 msgr2=0x7fb1d40b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d9471640 1 -- 172.21.15.70:0/2790103894 shutdown_connections 2023-08-25T09:53:17.274 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.267+0000 7fb1d9471640 1 -- 172.21.15.70:0/2790103894 wait complete. 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.269+0000 7fb1d9471640 1 Processor -- start 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.269+0000 7fb1d9471640 1 -- start start 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d9471640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c00039a0 0x7fb1d4155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d9471640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d41f0210 0x7fb1d41503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d9471640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 0x7fb1d40af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d9471640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41556a0 con 0x7fb1d4156dd0 2023-08-25T09:53:17.275 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d9471640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d415bff0 con 0x7fb1d41f0210 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d9471640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41f0740 con 0x7fb1c00039a0 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d8c70640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 0x7fb1d40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d8c70640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 0x7fb1d40af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54672/0 (socket says 172.21.15.70:54672) 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.270+0000 7fb1d8c70640 1 -- 172.21.15.70:0/2684042974 learned_addr learned my addr 172.21.15.70:0/2684042974 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c00039a0 0x7fb1d4155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d8c70640 1 -- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c00039a0 msgr2=0x7fb1d4155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d8c70640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c00039a0 0x7fb1d4155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.276 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d8c70640 1 -- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d41f0210 msgr2=0x7fb1d41503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d8c70640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d41f0210 0x7fb1d41503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.277 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d8c70640 1 -- 172.21.15.70:0/2684042974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d41f0a20 con 0x7fb1d4156dd0 2023-08-25T09:53:17.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d8c70640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 0x7fb1d40af790 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000fb0 tx=0x7fb1cc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.278 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc007230 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0073b0 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d41f0d00 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0047e0 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.271+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d4150920 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.272+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d41f1220 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.273+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb1cc004940 con 0x7fb1d4156dd0 2023-08-25T09:53:17.279 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.274+0000 7fb1d17fa640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.275+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb1cc087f00 con 0x7fb1d4156dd0 2023-08-25T09:53:17.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.275+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.280 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.275+0000 7fb1d3fff640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4034000 tx=0x7fb1c4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.281 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.279+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb1cc053870 con 0x7fb1d4156dd0 2023-08-25T09:53:17.469 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.466+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1d40b1dc0 con 0x7fb1b406a050 2023-08-25T09:53:17.470 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.467+0000 7fb1d17fa640 1 -- 172.21.15.70:0/2684042974 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb1c4003e40 con 0x7fb1b406a050 2023-08-25T09:53:17.470 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 6 to deep-scrub 2023-08-25T09:53:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 msgr2=0x7fb1b406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.473 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4034000 tx=0x7fb1c4009030 comp rx=0 tx=0).stop 2023-08-25T09:53:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 msgr2=0x7fb1d40af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 0x7fb1d40af790 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc000fb0 tx=0x7fb1cc00dc00 comp rx=0 tx=0).stop 2023-08-25T09:53:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 shutdown_connections 2023-08-25T09:53:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fb1b406a050 0x7fb1b406c510 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fb1d4156dd0 0x7fb1d40af790 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fb1d41f0210 0x7fb1d41503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.475 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 --2- 172.21.15.70:0/2684042974 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fb1c00039a0 0x7fb1d4155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 >> 172.21.15.70:0/2684042974 conn(0x7fb1d4069eb0 msgr2=0x7fb1d4070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 shutdown_connections 2023-08-25T09:53:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.470+0000 7fb1d9471640 1 -- 172.21.15.70:0/2684042974 wait complete. 2023-08-25T09:53:17.492 INFO:tasks.ceph:Scrubbing osd.7 2023-08-25T09:53:17.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-08-25T09:53:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.698+0000 7fac32d34640 1 Processor -- start 2023-08-25T09:53:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.698+0000 7fac32d34640 1 -- start start 2023-08-25T09:53:17.703 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac32d34640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fac2c1597d0 0x7fac2c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac32d34640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac2c159bb0 0x7fac2c0b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac32d34640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c0b2d40 con 0x7fac2c0b2780 2023-08-25T09:53:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac32d34640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c0b3020 con 0x7fac2c159bb0 2023-08-25T09:53:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac32d34640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c0b3300 con 0x7fac2c1597d0 2023-08-25T09:53:17.705 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac31531640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac2c159bb0 0x7fac2c0b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac31531640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7fac2c159bb0 0x7fac2c0b0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54676/0 (socket says 172.21.15.70:54676) 2023-08-25T09:53:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.700+0000 7fac31531640 1 -- 172.21.15.70:0/1414641367 learned_addr learned my addr 172.21.15.70:0/1414641367 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.701+0000 7fac31d32640 1 --2- 172.21.15.70:0/1414641367 >> v2:172.21.15.155:3300/0 conn(0x7fac2c1597d0 0x7fac2c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.706 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.701+0000 7fac31531640 1 -- 172.21.15.70:0/1414641367 >> v2:172.21.15.155:3300/0 conn(0x7fac2c1597d0 msgr2=0x7fac2c0ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.701+0000 7fac31531640 1 --2- 172.21.15.70:0/1414641367 >> v2:172.21.15.155:3300/0 conn(0x7fac2c1597d0 0x7fac2c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.701+0000 7fac31531640 1 -- 172.21.15.70:0/1414641367 >> v1:172.21.15.155:6789/0 conn(0x7fac2c0b2780 legacy=0x7fac2c0c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.707 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.701+0000 7fac31531640 1 -- 172.21.15.70:0/1414641367 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c0b35e0 con 0x7fac2c159bb0 2023-08-25T09:53:17.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.702+0000 7fac31531640 1 --2- 172.21.15.70:0/1414641367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c159bb0 0x7fac2c0b0030 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fac2c06cd20 tx=0x7fac1c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82183e3c82dd43fa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.702+0000 7fac30d30640 1 -- 172.21.15.70:0/1414641367 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0317e0 con 0x7fac2c159bb0 2023-08-25T09:53:17.710 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.702+0000 7fac30d30640 1 -- 172.21.15.70:0/1414641367 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c031940 con 0x7fac2c159bb0 2023-08-25T09:53:17.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.702+0000 7fac30d30640 1 -- 172.21.15.70:0/1414641367 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c031cd0 con 0x7fac2c159bb0 2023-08-25T09:53:17.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 -- 172.21.15.70:0/1414641367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c159bb0 msgr2=0x7fac2c0b0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 --2- 172.21.15.70:0/1414641367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c159bb0 0x7fac2c0b0030 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fac2c06cd20 tx=0x7fac1c02b880 comp rx=0 tx=0).stop 2023-08-25T09:53:17.711 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 -- 172.21.15.70:0/1414641367 shutdown_connections 2023-08-25T09:53:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 --2- 172.21.15.70:0/1414641367 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c159bb0 0x7fac2c0b0030 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 --2- 172.21.15.70:0/1414641367 >> v2:172.21.15.155:3300/0 conn(0x7fac2c1597d0 0x7fac2c0ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 -- 172.21.15.70:0/1414641367 >> 172.21.15.70:0/1414641367 conn(0x7fac2c06b560 msgr2=0x7fac2c0cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.712 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 -- 172.21.15.70:0/1414641367 shutdown_connections 2023-08-25T09:53:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.703+0000 7fac32d34640 1 -- 172.21.15.70:0/1414641367 wait complete. 2023-08-25T09:53:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.705+0000 7fac32d34640 1 Processor -- start 2023-08-25T09:53:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.706+0000 7fac32d34640 1 -- start start 2023-08-25T09:53:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac32d34640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac2c0b2780 0x7fac2c14df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.713 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac32d34640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c1597d0 0x7fac2c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac32d34640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2c159bb0 0x7fac2c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac32d34640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c14e440 con 0x7fac2c1597d0 2023-08-25T09:53:17.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac32d34640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c15a740 con 0x7fac2c159bb0 2023-08-25T09:53:17.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac32d34640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0c4ee0 con 0x7fac2c0b2780 2023-08-25T09:53:17.714 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac31531640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c1597d0 0x7fac2c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac31531640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c1597d0 0x7fac2c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54678/0 (socket says 172.21.15.70:54678) 2023-08-25T09:53:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.707+0000 7fac31531640 1 -- 172.21.15.70:0/3696259293 learned_addr learned my addr 172.21.15.70:0/3696259293 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31d32640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac2c0b2780 0x7fac2c14df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac32533640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2c159bb0 0x7fac2c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31531640 1 -- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac2c0b2780 msgr2=0x7fac2c14df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31531640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fac2c0b2780 0x7fac2c14df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31531640 1 -- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2c159bb0 msgr2=0x7fac2c06ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31531640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fac2c159bb0 0x7fac2c06ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.716 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31531640 1 -- 172.21.15.70:0/3696259293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0094d0 con 0x7fac2c1597d0 2023-08-25T09:53:17.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac31531640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c1597d0 0x7fac2c15abd0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02b8c0 tx=0x7fac1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.708+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0329c0 con 0x7fac2c1597d0 2023-08-25T09:53:17.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.709+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c032b40 con 0x7fac2c1597d0 2023-08-25T09:53:17.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.709+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c1e2a60 con 0x7fac2c1597d0 2023-08-25T09:53:17.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.709+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0453f0 con 0x7fac2c1597d0 2023-08-25T09:53:17.718 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.709+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c15a3d0 con 0x7fac2c1597d0 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.710+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac2c0b38c0 con 0x7fac2c1597d0 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.711+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fac1c04e450 con 0x7fac2c1597d0 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.712+0000 7fac227fc640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac1006a050 0x7fac1006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.712+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fac1c0aa720 con 0x7fac2c1597d0 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.712+0000 7fac227fc640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7fac100704d0 0x7fac100728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.712+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 --> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac10072fa0 con 0x7fac100704d0 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.712+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_get_version_reply(handle=1 version=205) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac1c0aaa90 con 0x7fac2c1597d0 2023-08-25T09:53:17.719 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.712+0000 7fac31d32640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac1006a050 0x7fac1006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.713+0000 7fac31d32640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac1006a050 0x7fac1006c510 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fac2800a970 tx=0x7fac28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.713+0000 7fac32533640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7fac100704d0 0x7fac100728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.714+0000 7fac32533640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7fac100704d0 0x7fac100728f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.720 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.716+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== osd.7 v2:172.21.15.155:6808/324773794 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fac10072fa0 con 0x7fac100704d0 2023-08-25T09:53:17.731 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.729+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 --> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fac2c066fb0 con 0x7fac100704d0 2023-08-25T09:53:17.732 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac227fc640 1 -- 172.21.15.70:0/3696259293 <== osd.7 v2:172.21.15.155:6808/324773794 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7fac2c066fb0 con 0x7fac100704d0 2023-08-25T09:53:17.732 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2023-08-25T09:53:17.733 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-08-25T09:53:17.733 INFO:teuthology.orchestra.run.smithi070.stdout:} 2023-08-25T09:53:17.733 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7fac100704d0 msgr2=0x7fac100728f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.155:6808/324773794,v1:172.21.15.155:6809/324773794] conn(0x7fac100704d0 0x7fac100728f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac1006a050 msgr2=0x7fac1006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fac1006a050 0x7fac1006c510 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fac2800a970 tx=0x7fac28008040 comp rx=0 tx=0).stop 2023-08-25T09:53:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c1597d0 msgr2=0x7fac2c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 --2- 172.21.15.70:0/3696259293 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fac2c1597d0 0x7fac2c15abd0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fac1c02b8c0 tx=0x7fac1c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.734 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32533640 1 -- 172.21.15.70:0/3696259293 reap_dead start 2023-08-25T09:53:17.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 shutdown_connections 2023-08-25T09:53:17.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 >> 172.21.15.70:0/3696259293 conn(0x7fac2c06b560 msgr2=0x7fac2c0afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 shutdown_connections 2023-08-25T09:53:17.735 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.731+0000 7fac32d34640 1 -- 172.21.15.70:0/3696259293 wait complete. 2023-08-25T09:53:17.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-08-25T09:53:17.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.963+0000 7f0952dfd640 1 Processor -- start 2023-08-25T09:53:17.966 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.964+0000 7f0952dfd640 1 -- start start 2023-08-25T09:53:17.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.965+0000 7f0952dfd640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f094c15cb40 0x7f094c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.967 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.965+0000 7f0952dfd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c0b2b40 0x7f094c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.965+0000 7f0952dfd640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094c06f050 con 0x7f094c0ac870 2023-08-25T09:53:17.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.965+0000 7f0952dfd640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094c06f330 con 0x7f094c15cb40 2023-08-25T09:53:17.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.965+0000 7f0952dfd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c06f610 con 0x7f094c0b2b40 2023-08-25T09:53:17.968 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.966+0000 7f095139a640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c0b2b40 0x7f094c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.966+0000 7f094bfff640 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f094c0ac870 0x7f094c160790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.70:44700/0 (socket says 172.21.15.70:44700) 2023-08-25T09:53:17.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.966+0000 7f094bfff640 1 -- 172.21.15.70:0/3764475520 learned_addr learned my addr 172.21.15.70:0/3764475520 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:17.969 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.966+0000 7f0950b99640 1 --2- 172.21.15.70:0/3764475520 >> v2:172.21.15.155:3300/0 conn(0x7f094c15cb40 0x7f094c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f094b7fe640 1 -- 172.21.15.70:0/3764475520 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840429500 0 0) 0x7f094c06f050 con 0x7f094c0ac870 2023-08-25T09:53:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f094b7fe640 1 -- 172.21.15.70:0/3764475520 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093c003520 con 0x7f094c0ac870 2023-08-25T09:53:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f0950b99640 1 -- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c0b2b40 msgr2=0x7f094c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:17.970 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f0950b99640 1 --2- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c0b2b40 0x7f094c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f0950b99640 1 -- 172.21.15.70:0/3764475520 >> v1:172.21.15.155:6789/0 conn(0x7f094c0ac870 legacy=0x7f094c160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f0950b99640 1 -- 172.21.15.70:0/3764475520 --> v2:172.21.15.155:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c06f8f0 con 0x7f094c15cb40 2023-08-25T09:53:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.967+0000 7f095139a640 1 --2- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c0b2b40 0x7f094c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T09:53:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.968+0000 7f0950b99640 1 --2- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c15cb40 0x7f094c0ce320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f09400097a0 tx=0x7f094002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8924bceb7416e1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.969+0000 7f094b7fe640 1 -- 172.21.15.70:0/3764475520 <== mon.1 v2:172.21.15.155:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094002d040 con 0x7f094c15cb40 2023-08-25T09:53:17.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.969+0000 7f094b7fe640 1 -- 172.21.15.70:0/3764475520 <== mon.1 v2:172.21.15.155:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940031810 con 0x7f094c15cb40 2023-08-25T09:53:17.972 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.969+0000 7f094b7fe640 1 -- 172.21.15.70:0/3764475520 <== mon.1 v2:172.21.15.155:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940031ba0 con 0x7f094c15cb40 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 -- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c15cb40 msgr2=0x7f094c0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 --2- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c15cb40 0x7f094c0ce320 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f09400097a0 tx=0x7f094002b8b0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 -- 172.21.15.70:0/3764475520 shutdown_connections 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 --2- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c0b2b40 0x7f094c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 --2- 172.21.15.70:0/3764475520 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c15cb40 0x7f094c0ce320 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 -- 172.21.15.70:0/3764475520 >> 172.21.15.70:0/3764475520 conn(0x7f094c069850 msgr2=0x7f094c069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:17.973 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 -- 172.21.15.70:0/3764475520 shutdown_connections 2023-08-25T09:53:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.970+0000 7f0952dfd640 1 -- 172.21.15.70:0/3764475520 wait complete. 2023-08-25T09:53:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.972+0000 7f0952dfd640 1 Processor -- start 2023-08-25T09:53:17.974 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.972+0000 7f0952dfd640 1 -- start start 2023-08-25T09:53:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f0952dfd640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c0ac870 0x7f094c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f0952dfd640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 0x7f094c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.975 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f0952dfd640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c15cb40 0x7f094c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f0952dfd640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0b00c0 con 0x7f094c0b2b40 2023-08-25T09:53:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f0952dfd640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c150700 con 0x7f094c0ac870 2023-08-25T09:53:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f0952dfd640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c14dfa0 con 0x7f094c15cb40 2023-08-25T09:53:17.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f094bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 0x7f094c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f094bfff640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 0x7f094c14ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54686/0 (socket says 172.21.15.70:54686) 2023-08-25T09:53:17.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.973+0000 7f094bfff640 1 -- 172.21.15.70:0/348953124 learned_addr learned my addr 172.21.15.70:0/348953124 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:17.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f094bfff640 1 -- 172.21.15.70:0/348953124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c15cb40 msgr2=0x7f094c06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.977 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f094bfff640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c15cb40 0x7f094c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f094bfff640 1 -- 172.21.15.70:0/348953124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c0ac870 msgr2=0x7f094c1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f094bfff640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c0ac870 0x7f094c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f094bfff640 1 -- 172.21.15.70:0/348953124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944009a70 con 0x7f094c0b2b40 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f094bfff640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 0x7f094c14ced0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f093800c410 tx=0x7f093800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938005ce0 con 0x7f094c0b2b40 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0938005e40 con 0x7f094c0b2b40 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938007310 con 0x7f094c0b2b40 2023-08-25T09:53:17.978 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.974+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0940009400 con 0x7f094c0b2b40 2023-08-25T09:53:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.975+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c150ed0 con 0x7f094c0b2b40 2023-08-25T09:53:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.976+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c14e160 con 0x7f094c0b2b40 2023-08-25T09:53:17.979 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.977+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0938007470 con 0x7f094c0b2b40 2023-08-25T09:53:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.978+0000 7f09497fa640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f093006a050 0x7f093006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.978+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f09380878c0 con 0x7f094c0b2b40 2023-08-25T09:53:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.978+0000 7f0950b99640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f093006a050 0x7f093006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:17.981 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.979+0000 7f0950b99640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f093006a050 0x7f093006c510 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0940036000 tx=0x7f0940030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:17.985 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:17.982+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f09380532f0 con 0x7f094c0b2b40 2023-08-25T09:53:18.172 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.170+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f094c06ece0 con 0x7f093006a050 2023-08-25T09:53:18.173 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.171+0000 7f09497fa640 1 -- 172.21.15.70:0/348953124 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0940031a80 con 0x7f093006a050 2023-08-25T09:53:18.173 INFO:teuthology.orchestra.run.smithi070.stderr:instructed osd(s) 7 to deep-scrub 2023-08-25T09:53:18.176 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f093006a050 msgr2=0x7f093006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f093006a050 0x7f093006c510 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0940036000 tx=0x7f0940030000 comp rx=0 tx=0).stop 2023-08-25T09:53:18.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 msgr2=0x7f094c14ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.177 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 0x7f094c14ced0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7f093800c410 tx=0x7f093800c8d0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 shutdown_connections 2023-08-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f093006a050 0x7f093006c510 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f094c15cb40 0x7f094c06be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f094c0b2b40 0x7f094c14ced0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 --2- 172.21.15.70:0/348953124 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f094c0ac870 0x7f094c1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.173+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 >> 172.21.15.70:0/348953124 conn(0x7f094c069850 msgr2=0x7f094c15ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.174+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 shutdown_connections 2023-08-25T09:53:18.179 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.174+0000 7f0952dfd640 1 -- 172.21.15.70:0/348953124 wait complete. 2023-08-25T09:53:18.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:53:18.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:53:18.353 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:53:18.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.408+0000 7ff263bc9640 1 Processor -- start 2023-08-25T09:53:18.411 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.408+0000 7ff263bc9640 1 -- start start 2023-08-25T09:53:18.413 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff263bc9640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff25c15cbd0 0x7ff25c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:18.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff263bc9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0d2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:18.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff263bc9640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c15cfb0 con 0x7ff25c1606c0 2023-08-25T09:53:18.414 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff263bc9640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0d33d0 con 0x7ff25c15cbd0 2023-08-25T09:53:18.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff263bc9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0cd4e0 con 0x7ff25c0d2a00 2023-08-25T09:53:18.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff261965640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff25c15cbd0 0x7ff25c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:18.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff261965640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7ff25c15cbd0 0x7ff25c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54698/0 (socket says 172.21.15.70:54698) 2023-08-25T09:53:18.415 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.410+0000 7ff261965640 1 -- 172.21.15.70:0/2733889079 learned_addr learned my addr 172.21.15.70:0/2733889079 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:18.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.411+0000 7ff262166640 1 --2- 172.21.15.70:0/2733889079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0d2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:18.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.411+0000 7ff261965640 1 -- 172.21.15.70:0/2733889079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 msgr2=0x7ff25c0d2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:18.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.411+0000 7ff261965640 1 --2- 172.21.15.70:0/2733889079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0d2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.416 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.411+0000 7ff261965640 1 -- 172.21.15.70:0/2733889079 >> v1:172.21.15.155:6789/0 conn(0x7ff25c1606c0 legacy=0x7ff25c0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.417 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.411+0000 7ff261965640 1 -- 172.21.15.70:0/2733889079 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c160f10 con 0x7ff25c15cbd0 2023-08-25T09:53:18.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.412+0000 7ff261965640 1 --2- 172.21.15.70:0/2733889079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c0cffb0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7ff258009870 tx=0x7ff25802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634550642a2e16fa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:18.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.412+0000 7ff260963640 1 -- 172.21.15.70:0/2733889079 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258030840 con 0x7ff25c15cbd0 2023-08-25T09:53:18.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.412+0000 7ff260963640 1 -- 172.21.15.70:0/2733889079 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2580309a0 con 0x7ff25c15cbd0 2023-08-25T09:53:18.421 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.412+0000 7ff260963640 1 -- 172.21.15.70:0/2733889079 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258030d30 con 0x7ff25c15cbd0 2023-08-25T09:53:18.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 -- 172.21.15.70:0/2733889079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 msgr2=0x7ff25c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 --2- 172.21.15.70:0/2733889079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c0cffb0 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7ff258009870 tx=0x7ff25802b980 comp rx=0 tx=0).stop 2023-08-25T09:53:18.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 -- 172.21.15.70:0/2733889079 shutdown_connections 2023-08-25T09:53:18.422 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 --2- 172.21.15.70:0/2733889079 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0d2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 --2- 172.21.15.70:0/2733889079 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c0cffb0 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 -- 172.21.15.70:0/2733889079 >> 172.21.15.70:0/2733889079 conn(0x7ff25c070260 msgr2=0x7ff25c070670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:18.423 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 -- 172.21.15.70:0/2733889079 shutdown_connections 2023-08-25T09:53:18.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.413+0000 7ff263bc9640 1 -- 172.21.15.70:0/2733889079 wait complete. 2023-08-25T09:53:18.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.415+0000 7ff263bc9640 1 Processor -- start 2023-08-25T09:53:18.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.415+0000 7ff263bc9640 1 -- start start 2023-08-25T09:53:18.424 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.416+0000 7ff263bc9640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:18.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff263bc9640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:18.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff263bc9640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1606c0 0x7ff25c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:18.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff263bc9640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c151530 con 0x7ff25c15cbd0 2023-08-25T09:53:18.425 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff263bc9640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0ccb20 con 0x7ff25c1606c0 2023-08-25T09:53:18.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff263bc9640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c0cb870 con 0x7ff25c0d2a00 2023-08-25T09:53:18.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:18.426 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c150ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:54710/0 (socket says 172.21.15.70:54710) 2023-08-25T09:53:18.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 -- 172.21.15.70:0/3875785238 learned_addr learned my addr 172.21.15.70:0/3875785238 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:18.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261965640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:18.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff262166640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1606c0 0x7ff25c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:18.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 -- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 msgr2=0x7ff25c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.427 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 -- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1606c0 msgr2=0x7ff25c0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1606c0 0x7ff25c0cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff261164640 1 -- 172.21.15.70:0/3875785238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2580094d0 con 0x7ff25c15cbd0 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.417+0000 7ff262166640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1606c0 0x7ff25c0cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.418+0000 7ff261164640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c150ff0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c400 tx=0x7ff24c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.418+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c0052d0 con 0x7ff25c15cbd0 2023-08-25T09:53:18.428 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.418+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c005430 con 0x7ff25c15cbd0 2023-08-25T09:53:18.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.418+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c003830 con 0x7ff25c15cbd0 2023-08-25T09:53:18.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.418+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c0cc690 con 0x7ff25c15cbd0 2023-08-25T09:53:18.429 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.418+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c0cbd60 con 0x7ff25c15cbd0 2023-08-25T09:53:18.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.419+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c04e420 con 0x7ff25c15cbd0 2023-08-25T09:53:18.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.420+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff24c003990 con 0x7ff25c15cbd0 2023-08-25T09:53:18.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.421+0000 7ff2527fc640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff24006a050 0x7ff24006c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:18.430 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.421+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff24c087800 con 0x7ff25c15cbd0 2023-08-25T09:53:18.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.422+0000 7ff261965640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff24006a050 0x7ff24006c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:18.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.423+0000 7ff261965640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff24006a050 0x7ff24006c510 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff258036000 tx=0x7ff258008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:18.431 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.426+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff24c053220 con 0x7ff25c15cbd0 2023-08-25T09:53:18.615 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.612+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff25c1614d0 con 0x7ff24006a050 2023-08-25T09:53:18.622 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.619+0000 7ff2527fc640 1 -- 172.21.15.70:0/3875785238 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68027 (secure 0 0 0) 0x7ff258030b10 con 0x7ff24006a050 2023-08-25T09:53:18.622 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:18.623 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:53:18.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff24006a050 msgr2=0x7ff24006c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.624 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff24006a050 0x7ff24006c510 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff258036000 tx=0x7ff258008030 comp rx=0 tx=0).stop 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 msgr2=0x7ff25c150ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c150ff0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7ff24c00c400 tx=0x7ff24c00c8c0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 shutdown_connections 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7ff24006a050 0x7ff24006c510 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7ff25c1606c0 0x7ff25c0cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7ff25c15cbd0 0x7ff25c150ff0 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.625 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 --2- 172.21.15.70:0/3875785238 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7ff25c0d2a00 0x7ff25c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:18.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 >> 172.21.15.70:0/3875785238 conn(0x7ff25c070260 msgr2=0x7ff25c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:18.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.622+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 shutdown_connections 2023-08-25T09:53:18.626 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:18.623+0000 7ff263bc9640 1 -- 172.21.15.70:0/3875785238 wait complete. 2023-08-25T09:53:18.644 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":636,"stamp":"2023-08-25T09:53:17.617162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10818,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1551681,"statfs":{"total":1600000000,"available":1588922304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52428800,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-84992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002577"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":495,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.632553+0000","last_change":"2023-08-25T09:38:41.181882+0000","last_active":"2023-08-25T09:53:07.632553+0000","last_peered":"2023-08-25T09:53:07.632553+0000","last_clean":"2023-08-25T09:53:07.632553+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:53:07.632553+0000","last_undegraded":"2023-08-25T09:53:07.632553+0000","last_fullsized":"2023-08-25T09:53:07.632553+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:39.936341+0000","last_clean_scrub_stamp":"2023-08-25T09:38:39.936341+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.635287+0000","last_change":"2023-08-25T09:38:47.230112+0000","last_active":"2023-08-25T09:53:07.635287+0000","last_peered":"2023-08-25T09:53:07.635287+0000","last_clean":"2023-08-25T09:53:07.635287+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:53:07.635287+0000","last_undegraded":"2023-08-25T09:53:07.635287+0000","last_fullsized":"2023-08-25T09:53:07.635287+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:40:09.976193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":372,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.631062+0000","last_change":"2023-08-25T09:38:47.023886+0000","last_active":"2023-08-25T09:53:07.631062+0000","last_peered":"2023-08-25T09:53:07.631062+0000","last_clean":"2023-08-25T09:53:07.631062+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:53:07.631062+0000","last_undegraded":"2023-08-25T09:53:07.631062+0000","last_fullsized":"2023-08-25T09:53:07.631062+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:11.279207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":373,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.650798+0000","last_change":"2023-08-25T09:38:47.023446+0000","last_active":"2023-08-25T09:53:07.650798+0000","last_peered":"2023-08-25T09:53:07.650798+0000","last_clean":"2023-08-25T09:53:07.650798+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:53:07.650798+0000","last_undegraded":"2023-08-25T09:53:07.650798+0000","last_fullsized":"2023-08-25T09:53:07.650798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:20:59.345933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.634006+0000","last_change":"2023-08-25T09:38:47.025037+0000","last_active":"2023-08-25T09:53:07.634006+0000","last_peered":"2023-08-25T09:53:07.634006+0000","last_clean":"2023-08-25T09:53:07.634006+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:53:07.634006+0000","last_undegraded":"2023-08-25T09:53:07.634006+0000","last_fullsized":"2023-08-25T09:53:07.634006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:53:55.654458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.643927+0000","last_change":"2023-08-25T09:38:47.023652+0000","last_active":"2023-08-25T09:53:07.643927+0000","last_peered":"2023-08-25T09:53:07.643927+0000","last_clean":"2023-08-25T09:53:07.643927+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:53:07.643927+0000","last_undegraded":"2023-08-25T09:53:07.643927+0000","last_fullsized":"2023-08-25T09:53:07.643927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:48:06.450281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.630628+0000","last_change":"2023-08-25T09:38:47.022583+0000","last_active":"2023-08-25T09:53:07.630628+0000","last_peered":"2023-08-25T09:53:07.630628+0000","last_clean":"2023-08-25T09:53:07.630628+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:53:07.630628+0000","last_undegraded":"2023-08-25T09:53:07.630628+0000","last_fullsized":"2023-08-25T09:53:07.630628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:45:53.337656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":339,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633575+0000","last_change":"2023-08-25T09:38:47.151090+0000","last_active":"2023-08-25T09:53:07.633575+0000","last_peered":"2023-08-25T09:53:07.633575+0000","last_clean":"2023-08-25T09:53:07.633575+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:53:07.633575+0000","last_undegraded":"2023-08-25T09:53:07.633575+0000","last_fullsized":"2023-08-25T09:53:07.633575+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:02:17.959301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":371,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:07.633711+0000","last_change":"2023-08-25T09:38:47.150984+0000","last_active":"2023-08-25T09:53:07.633711+0000","last_peered":"2023-08-25T09:53:07.633711+0000","last_clean":"2023-08-25T09:53:07.633711+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:53:07.633711+0000","last_undegraded":"2023-08-25T09:53:07.633711+0000","last_fullsized":"2023-08-25T09:53:07.633711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:38:45.990587+0000","last_clean_scrub_stamp":"2023-08-25T09:38:45.990587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:42:38.900516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575025,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1399999999999999,"15min":1.135},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.35699999999999998},"max":{"1min":1.7430000000000001,"5min":1.829,"15min":10.747},"last":0.65900000000000003},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1679999999999999,"15min":1.1220000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.43099999999999999},"max":{"1min":1.605,"5min":1.669,"15min":1.855},"last":1.605}]},{"osd":1,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.2569999999999999,"15min":1.244},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.5569999999999999,"5min":1.823,"15min":11.010999999999999},"last":1.012},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.212,"15min":1.1890000000000001},"min":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.6100000000000001,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":1.431}]},{"osd":2,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.1539999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.50600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.179,"5min":1.288,"15min":1.276},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.621,"5min":1.873,"15min":1.98},"last":1.141}]},{"osd":3,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2629999999999999,"15min":1.2250000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.69299999999999995},"max":{"1min":1.651,"5min":1.833,"15min":2.1030000000000002},"last":1.278}]},{"osd":4,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.288},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":1.712,"5min":1.845,"15min":7.1639999999999997},"last":1.712},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.323},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.46300000000000002},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8779999999999999},"last":1.4119999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4059999999999999,"15min":1.419},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6919999999999999,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.375},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.427,"15min":1.4330000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.827,"5min":1.907,"15min":2.0739999999999998},"last":1.827}]},{"osd":6,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.538,"15min":1.522},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.929,"5min":1.984,"15min":2.1720000000000002},"last":1.929},{"interface":"front","average":{"1min":1.381,"5min":1.454,"15min":1.4630000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9530000000000001,"5min":2.1019999999999999,"15min":2.3220000000000001},"last":1.3160000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575025,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1739999999999999,"15min":1.149},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.54400000000000004},"max":{"1min":1.591,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1950000000000001,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5509999999999999,"5min":1.645,"15min":5.5410000000000004},"last":0.95199999999999996}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.262,"15min":1.228},"min":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.496},{"interface":"front","average":{"1min":1.139,"5min":1.194,"15min":1.1830000000000001},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6359999999999999,"5min":1.8100000000000001,"15min":2.3079999999999998},"last":0.91700000000000004}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1930000000000001,"15min":1.1619999999999999},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.534,"5min":1.696,"15min":1.708},"last":1.127},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.47299999999999998},"max":{"1min":1.7350000000000001,"5min":1.7889999999999999,"15min":13.193},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.369,"15min":1.319},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.68899999999999995},"max":{"1min":1.6519999999999999,"5min":1.911,"15min":5.6239999999999997},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":1.071,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8720000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.0009999999999999}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.3280000000000001},"min":{"1min":1.004,"5min":0.90500000000000003,"15min":0.42599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.484},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.482,"5min":1.419,"15min":1.3939999999999999},"min":{"1min":0.95999999999999996,"5min":0.78900000000000003,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":2.101,"15min":5.4699999999999998},"last":1.7909999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3999999999999999,"15min":1.3440000000000001},"min":{"1min":0.94499999999999995,"5min":0.75,"15min":0.59599999999999997},"max":{"1min":1.875,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.391},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.4730000000000001},"min":{"1min":1.1639999999999999,"5min":1,"15min":0.59099999999999997},"max":{"1min":2.121,"5min":2.121,"15min":2.4420000000000002},"last":1.716}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":1.157,"5min":0.89400000000000002,"15min":0.622},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":6.7709999999999999},"last":2.0379999999999998},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.548,"15min":1.5089999999999999},"min":{"1min":1.24,"5min":0.872,"15min":0.57399999999999995},"max":{"1min":2.181,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.6779999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575025,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2669999999999999,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.497},"max":{"1min":1.667,"5min":1.9770000000000001,"15min":2.081},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.2210000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.65400000000000003},"max":{"1min":1.5309999999999999,"5min":1.6950000000000001,"15min":1.974},"last":1.4419999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.145,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.5569999999999999,"5min":1.6899999999999999,"15min":2.1779999999999999},"last":1.351},{"interface":"front","average":{"1min":1.339,"5min":1.2010000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.76,"5min":1.76,"15min":2.1869999999999998},"last":1.034}]},{"osd":2,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.258},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.68400000000000005},"max":{"1min":1.909,"5min":1.909,"15min":2.3690000000000002},"last":1.401},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.873,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":1.204}]},{"osd":3,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.236,"15min":1.28},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.53,"5min":1.7649999999999999,"15min":17.356999999999999},"last":1.538},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.343,"15min":1.3300000000000001},"min":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.696,"5min":1.7490000000000001,"15min":1.9650000000000001},"last":1.498}]},{"osd":4,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.55500000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.625},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.278,"5min":1.304,"15min":1.329},"min":{"1min":0.746,"5min":0.746,"15min":0.58799999999999997},"max":{"1min":1.556,"5min":1.7290000000000001,"15min":2.2789999999999999},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4450000000000001,"15min":1.4430000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.58899999999999997},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":2.4820000000000002},"last":1.077},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.516,"15min":1.4950000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.60399999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.5539999999999998},"last":1.4730000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.95499999999999996,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":1.6359999999999999,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5700000000000001,"15min":1.5780000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.63900000000000001},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.3700000000000001},"last":1.593}]}]},{"osd":4,"up_from":13,"seq":55834575025,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.274,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.624,"15min":0.34399999999999997},"max":{"1min":1.617,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.228,"5min":1.2529999999999999,"15min":1.258},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.434},"max":{"1min":2.3959999999999999,"5min":2.5150000000000001,"15min":7.367},"last":1.1299999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.159,"15min":1.1439999999999999},"min":{"1min":0.88,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.69499999999999995,"5min":0.56200000000000006,"15min":0.49299999999999999},"max":{"1min":1.7010000000000001,"5min":4.351,"15min":8.5850000000000009},"last":1.2549999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2010000000000001,"15min":1.208},"min":{"1min":0.89500000000000002,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":4.1669999999999998},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.266},"min":{"1min":0.92500000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"max":{"1min":1.899,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.425}]},{"osd":3,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.29,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.79400000000000004,"15min":0.61899999999999999},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":7.2560000000000002},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4059999999999999,"15min":1.385},"min":{"1min":1.0800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.351}]},{"osd":5,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.67100000000000004,"15min":0.55200000000000005},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.3400000000000001,"15min":1.3260000000000001},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.57099999999999995},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":8.2409999999999997},"last":1.325}]},{"osd":6,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4730000000000001,"15min":1.446},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":6.1980000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.498},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.6399999999999999,"15min":1.5920000000000001},"min":{"1min":1.0840000000000001,"5min":0.69799999999999995,"15min":0.67100000000000004},"max":{"1min":6.069,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.712}]},{"osd":7,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.496,"15min":1.502},"min":{"1min":1.119,"5min":0.75700000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.21},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.5900000000000001},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.9900000000000002,"5min":3.831,"15min":3.831},"last":1.5600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575025,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.4159999999999999,"5min":1.8180000000000001,"15min":1.8999999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":0.998},"min":{"1min":0.71299999999999997,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":1.552,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.1539999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.86799999999999999},"min":{"1min":0.66800000000000004,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.702},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.103,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.5740000000000001,"5min":1.631,"15min":2.3399999999999999},"last":1.595}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.0589999999999999,"15min":0.98599999999999999},"min":{"1min":0.81799999999999995,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.919},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2250000000000001,"15min":1.1539999999999999},"min":{"1min":0.94899999999999995,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.702,"5min":1.901,"15min":3.399},"last":1.466}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3340000000000001,"15min":1.2490000000000001},"min":{"1min":1.0169999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.1949999999999998},"last":1.542},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.222,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6539999999999999,"5min":1.79,"15min":2.1840000000000002},"last":1.24}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4079999999999999,"15min":1.355},"min":{"1min":1.1060000000000001,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":1.881,"5min":2.1930000000000001,"15min":2.407},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3779999999999999,"15min":1.3160000000000001},"min":{"1min":1.151,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.895},"last":1.7430000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3280000000000001,"15min":1.292},"min":{"1min":0.97999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.3450000000000002},"last":1.429},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.49,"15min":1.427},"min":{"1min":1.1200000000000001,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.9830000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.498,"15min":1.4199999999999999},"min":{"1min":1.3009999999999999,"5min":0.95299999999999996,"15min":0.83699999999999997},"max":{"1min":1.9390000000000001,"5min":1.976,"15min":2.2989999999999999},"last":1.651},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4570000000000001,"15min":1.3999999999999999},"min":{"1min":1.093,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.1509999999999998},"last":1.5089999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575025,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.2490000000000001,"5min":1.6930000000000001,"15min":2.383},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.018,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.7309999999999999},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97399999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3340000000000001,"5min":1.635,"15min":2.3180000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.391,"5min":1.704,"15min":2.8599999999999999},"last":0.747}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":1.026},"min":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.214,"5min":1.5589999999999999,"15min":18.821000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.127,"15min":1.1000000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.4430000000000001,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":0.85299999999999998}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3180000000000001,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9219999999999999,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":1.115},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.288,"15min":1.286},"min":{"1min":0.77700000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6799999999999999,"5min":1.8,"15min":2.7639999999999998},"last":1.333}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2509999999999999,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6339999999999999,"5min":1.849,"15min":5.1269999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.373,"15min":1.363},"min":{"1min":0.878,"5min":0.625,"15min":0.625},"max":{"1min":1.7390000000000001,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":1.167}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.407,"15min":1.399},"min":{"1min":0.97799999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0059999999999998,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.389,"15min":1.4039999999999999},"min":{"1min":0.98799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6919999999999999,"5min":2.028,"15min":5.0599999999999996},"last":1.1899999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.381,"15min":1.3859999999999999},"min":{"1min":0.81699999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2,"15min":2.968},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.96,"15min":5.2990000000000004},"last":1.3540000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575025,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.83099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.3240000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.95199999999999996,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.333,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95299999999999996,"15min":0.89400000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5940000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.036,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.474,"5min":1.6619999999999999,"15min":1.746},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.372},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.9450000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.05,"5min":1.04,"15min":1.02},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.657,"5min":1.657,"15min":1.764},"last":0.66200000000000003}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.175,"15min":1.1539999999999999},"min":{"1min":0.76200000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5209999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.075},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2949999999999999,"15min":1.286},"min":{"1min":0.85799999999999998,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.278,"15min":1.2509999999999999},"min":{"1min":0.70099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.29,"5min":1.3129999999999999,"15min":1.2549999999999999},"min":{"1min":0.86799999999999999,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":2.867,"15min":2.867},"last":1.014}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.369,"15min":1.371},"min":{"1min":0.96899999999999997,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.9940000000000002},"last":1.137},{"interface":"front","average":{"1min":1.411,"5min":1.415,"15min":1.3759999999999999},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":2.036,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.121}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4419999999999999,"15min":1.419},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.79200000000000004},"max":{"1min":1.704,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.399,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.218},"last":1.093}]}]},{"osd":0,"up_from":13,"seq":55834575026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.313},"max":{"1min":1.2969999999999999,"5min":2.2759999999999998,"15min":11.257},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":0.96999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":1.694,"5min":1.762,"15min":6.4260000000000002},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.45300000000000001,"15min":0.36199999999999999},"max":{"1min":1.488,"5min":1.522,"15min":1.8620000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.0389999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.374},"max":{"1min":1.788,"5min":1.788,"15min":3.4889999999999999},"last":1.1579999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1,"15min":0.93100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.40699999999999997},"max":{"1min":1.448,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.212},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1459999999999999,"15min":1.115},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.39700000000000002},"max":{"1min":1.5449999999999999,"5min":1.5860000000000001,"15min":2.3809999999999998},"last":0.96599999999999997}]},{"osd":4,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.278,"15min":1.252},"min":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.59999999999999998},"max":{"1min":1.4930000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.115},{"interface":"front","average":{"1min":1.323,"5min":1.4239999999999999,"15min":1.377},"min":{"1min":0.94799999999999995,"5min":0.82399999999999995,"15min":0.65500000000000003},"max":{"1min":1.595,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.1850000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.351,"15min":1.306},"min":{"1min":0.73099999999999998,"5min":0.623,"15min":0.623},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":3.3769999999999998},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.292,"15min":1.2729999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.51400000000000001},"max":{"1min":1.829,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.28}]},{"osd":6,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4099999999999999,"15min":1.359},"min":{"1min":0.83599999999999997,"5min":0.81499999999999995,"15min":0.71399999999999997},"max":{"1min":1.8759999999999999,"5min":2.056,"15min":2.6869999999999998},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.401,"15min":1.3420000000000001},"min":{"1min":0.77300000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.331},"last":1.3069999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.446,"15min":1.407},"min":{"1min":0.88900000000000001,"5min":0.78900000000000003,"15min":0.71099999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.6429999999999998},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3899999999999999,"15min":1.3440000000000001},"min":{"1min":0.81999999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.9319999999999999,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.3320000000000001}]}]}],"pool_statfs":[]}} 2023-08-25T09:53:18.650 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-08-25T09:38:39.936341+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=39, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.651 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.651 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.652 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.652 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.652 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.653 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.653 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.653 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-08-25T09:38:45.990587+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=38, tm_sec=45, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=9, tm_min=53, tm_sec=11, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T09:53:18.654 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-08-25T09:53:38.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T09:53:38.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.879+0000 7f366e50c640 1 Processor -- start 2023-08-25T09:53:38.881 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.879+0000 7f366e50c640 1 -- start start 2023-08-25T09:53:38.882 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f366e50c640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f366815ec10 0x7f366815c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f366e50c640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7f366815c6d0 0x7f36680b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f366e50c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680b1b60 con 0x7f366815e830 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f366e50c640 1 -- --> v2:172.21.15.70:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366806bd80 con 0x7f366815ec10 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f366e50c640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366806b7c0 con 0x7f366815c6d0 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f36677fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f366815ec10 0x7f366815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f36677fe640 1 --2- >> v2:172.21.15.70:3300/0 conn(0x7f366815ec10 0x7f366815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33146/0 (socket says 172.21.15.70:33146) 2023-08-25T09:53:38.883 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.881+0000 7f36677fe640 1 -- 172.21.15.70:0/2247215979 learned_addr learned my addr 172.21.15.70:0/2247215979 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:38.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.882+0000 7f36677fe640 1 -- 172.21.15.70:0/2247215979 >> v2:172.21.15.155:3300/0 conn(0x7f366815c6d0 msgr2=0x7f36680b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:38.884 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.882+0000 7f36677fe640 1 --2- 172.21.15.70:0/2247215979 >> v2:172.21.15.155:3300/0 conn(0x7f366815c6d0 0x7f36680b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:38.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.882+0000 7f36677fe640 1 -- 172.21.15.70:0/2247215979 >> v1:172.21.15.155:6789/0 conn(0x7f366815e830 legacy=0x7f366806ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:38.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.882+0000 7f36677fe640 1 -- 172.21.15.70:0/2247215979 --> v2:172.21.15.70:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366806baa0 con 0x7f366815ec10 2023-08-25T09:53:38.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.883+0000 7f36677fe640 1 --2- 172.21.15.70:0/2247215979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815ec10 0x7f366815c190 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f36680cf460 tx=0x7f365c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395b917aefeabcb0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:38.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.883+0000 7f3666ffd640 1 -- 172.21.15.70:0/2247215979 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0317e0 con 0x7f366815ec10 2023-08-25T09:53:38.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.883+0000 7f3666ffd640 1 -- 172.21.15.70:0/2247215979 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c031940 con 0x7f366815ec10 2023-08-25T09:53:38.885 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.883+0000 7f3666ffd640 1 -- 172.21.15.70:0/2247215979 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c031cd0 con 0x7f366815ec10 2023-08-25T09:53:38.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 -- 172.21.15.70:0/2247215979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815ec10 msgr2=0x7f366815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:38.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 --2- 172.21.15.70:0/2247215979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815ec10 0x7f366815c190 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f36680cf460 tx=0x7f365c02b880 comp rx=0 tx=0).stop 2023-08-25T09:53:38.886 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 -- 172.21.15.70:0/2247215979 shutdown_connections 2023-08-25T09:53:38.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 --2- 172.21.15.70:0/2247215979 >> v2:172.21.15.155:3300/0 conn(0x7f366815c6d0 0x7f36680b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:38.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 --2- 172.21.15.70:0/2247215979 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815ec10 0x7f366815c190 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:38.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 -- 172.21.15.70:0/2247215979 >> 172.21.15.70:0/2247215979 conn(0x7f3668069850 msgr2=0x7f3668069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:38.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 -- 172.21.15.70:0/2247215979 shutdown_connections 2023-08-25T09:53:38.887 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.884+0000 7f366e50c640 1 -- 172.21.15.70:0/2247215979 wait complete. 2023-08-25T09:53:38.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.886+0000 7f366e50c640 1 Processor -- start 2023-08-25T09:53:38.888 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.886+0000 7f366e50c640 1 -- start start 2023-08-25T09:53:38.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.887+0000 7f366e50c640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f366815c6d0 0x7f3668146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:38.889 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.887+0000 7f366e50c640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 0x7f36681448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.887+0000 7f366e50c640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f366815ec10 0x7f36681450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.887+0000 7f366e50c640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668144e00 con 0x7f366815e830 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.887+0000 7f366e50c640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668146690 con 0x7f366815ec10 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.887+0000 7f366e50c640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3668145a80 con 0x7f366815c6d0 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 0x7f36681448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 0x7f36681448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:33162/0 (socket says 172.21.15.70:33162) 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 -- 172.21.15.70:0/1164931452 learned_addr learned my addr 172.21.15.70:0/1164931452 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:38.890 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f3667fff640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f366815c6d0 0x7f3668146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:38.891 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f366caa9640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f366815ec10 0x7f36681450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:38.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 -- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f366815c6d0 msgr2=0x7f3668146b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:38.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f366815c6d0 0x7f3668146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:38.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 -- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f366815ec10 msgr2=0x7f36681450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:38.892 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f366815ec10 0x7f36681450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.888+0000 7f36677fe640 1 -- 172.21.15.70:0/1164931452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0094d0 con 0x7f366815e830 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.889+0000 7f36677fe640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 0x7f36681448c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.889+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c032880 con 0x7f366815e830 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.889+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c0329e0 con 0x7f366815e830 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.889+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c032d90 con 0x7f366815e830 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.889+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3668145d60 con 0x7f366815e830 2023-08-25T09:53:38.893 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.889+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668146320 con 0x7f366815e830 2023-08-25T09:53:38.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.890+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366804e420 con 0x7f366815e830 2023-08-25T09:53:38.894 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.891+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f365c04d3f0 con 0x7f366815e830 2023-08-25T09:53:38.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.892+0000 7f3664ff9640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f365406a080 0x7f365406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:38.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.892+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f365c0aa210 con 0x7f366815e830 2023-08-25T09:53:38.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.892+0000 7f3667fff640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f365406a080 0x7f365406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:38.895 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.893+0000 7f3667fff640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f365406a080 0x7f365406c540 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3658007fd0 tx=0x7f3658009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:38.898 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:38.897+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f365c075b80 con 0x7f366815e830 2023-08-25T09:53:39.086 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.084+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 --> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f366806b480 con 0x7f365406a080 2023-08-25T09:53:39.090 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.089+0000 7f3664ff9640 1 -- 172.21.15.70:0/1164931452 <== mgr.4110 v2:172.21.15.70:6832/2630723694 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+68181 (secure 0 0 0) 0x7f3658001c60 con 0x7f365406a080 2023-08-25T09:53:39.091 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:39.092 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2023-08-25T09:53:39.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f365406a080 msgr2=0x7f365406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.093 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f365406a080 0x7f365406c540 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3658007fd0 tx=0x7f3658009040 comp rx=0 tx=0).stop 2023-08-25T09:53:39.094 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 msgr2=0x7f36681448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 0x7f36681448c0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c0046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 shutdown_connections 2023-08-25T09:53:39.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f365406a080 0x7f365406c540 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.095 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f366815ec10 0x7f36681450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f366815e830 0x7f36681448c0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 --2- 172.21.15.70:0/1164931452 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f366815c6d0 0x7f3668146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 >> 172.21.15.70:0/1164931452 conn(0x7f3668069850 msgr2=0x7f36680ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:39.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 shutdown_connections 2023-08-25T09:53:39.096 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.092+0000 7f366e50c640 1 -- 172.21.15.70:0/1164931452 wait complete. 2023-08-25T09:53:39.113 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":646,"stamp":"2023-08-25T09:53:37.623300+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10818,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1551681,"statfs":{"total":1600000000,"available":1588922304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"1.0","version":"205'119","reported_seq":505,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:14.908837+0000","last_change":"2023-08-25T09:53:14.908897+0000","last_active":"2023-08-25T09:53:14.908837+0000","last_peered":"2023-08-25T09:53:14.908837+0000","last_clean":"2023-08-25T09:53:14.908837+0000","last_became_active":"2023-08-25T09:38:41.181103+0000","last_became_peered":"2023-08-25T09:38:41.181103+0000","last_unstale":"2023-08-25T09:53:14.908837+0000","last_undegraded":"2023-08-25T09:53:14.908837+0000","last_fullsized":"2023-08-25T09:53:14.908837+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"205'119","last_scrub_stamp":"2023-08-25T09:53:14.908786+0000","last_deep_scrub":"205'119","last_deep_scrub_stamp":"2023-08-25T09:53:14.908786+0000","last_clean_scrub_stamp":"2023-08-25T09:53:14.908786+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:31:59.663694+0000","scrub_duration":0.0045568229999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":381,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:24.850137+0000","last_change":"2023-08-25T09:53:24.850242+0000","last_active":"2023-08-25T09:53:24.850137+0000","last_peered":"2023-08-25T09:53:24.850137+0000","last_clean":"2023-08-25T09:53:24.850137+0000","last_became_active":"2023-08-25T09:38:47.229909+0000","last_became_peered":"2023-08-25T09:38:47.229909+0000","last_unstale":"2023-08-25T09:53:24.850137+0000","last_undegraded":"2023-08-25T09:53:24.850137+0000","last_fullsized":"2023-08-25T09:53:24.850137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:24.850049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:24.850049+0000","last_clean_scrub_stamp":"2023-08-25T09:53:24.850049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:10.209114+0000","scrub_duration":0.0028141049999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":381,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:26.880951+0000","last_change":"2023-08-25T09:53:26.881055+0000","last_active":"2023-08-25T09:53:26.880951+0000","last_peered":"2023-08-25T09:53:26.880951+0000","last_clean":"2023-08-25T09:53:26.880951+0000","last_became_active":"2023-08-25T09:38:47.023260+0000","last_became_peered":"2023-08-25T09:38:47.023260+0000","last_unstale":"2023-08-25T09:53:26.880951+0000","last_undegraded":"2023-08-25T09:53:26.880951+0000","last_fullsized":"2023-08-25T09:53:26.880951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:26.880860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:26.880860+0000","last_clean_scrub_stamp":"2023-08-25T09:53:26.880860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T17:53:30.331422+0000","scrub_duration":0.003404771,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":383,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:21.619558+0000","last_change":"2023-08-25T09:53:21.619558+0000","last_active":"2023-08-25T09:53:21.619558+0000","last_peered":"2023-08-25T09:53:21.619558+0000","last_clean":"2023-08-25T09:53:21.619558+0000","last_became_active":"2023-08-25T09:38:47.022764+0000","last_became_peered":"2023-08-25T09:38:47.022764+0000","last_unstale":"2023-08-25T09:53:21.619558+0000","last_undegraded":"2023-08-25T09:53:21.619558+0000","last_fullsized":"2023-08-25T09:53:21.619558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:21.619251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:21.619251+0000","last_clean_scrub_stamp":"2023-08-25T09:53:21.619251+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T20:17:37.726505+0000","scrub_duration":0.0036445470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":381,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:28.885791+0000","last_change":"2023-08-25T09:53:28.885903+0000","last_active":"2023-08-25T09:53:28.885791+0000","last_peered":"2023-08-25T09:53:28.885791+0000","last_clean":"2023-08-25T09:53:28.885791+0000","last_became_active":"2023-08-25T09:38:47.024467+0000","last_became_peered":"2023-08-25T09:38:47.024467+0000","last_unstale":"2023-08-25T09:53:28.885791+0000","last_undegraded":"2023-08-25T09:53:28.885791+0000","last_fullsized":"2023-08-25T09:53:28.885791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:28.885699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:28.885699+0000","last_clean_scrub_stamp":"2023-08-25T09:53:28.885699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T16:36:31.337223+0000","scrub_duration":0.0032089839999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":349,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:20.174696+0000","last_change":"2023-08-25T09:53:20.174696+0000","last_active":"2023-08-25T09:53:20.174696+0000","last_peered":"2023-08-25T09:53:20.174696+0000","last_clean":"2023-08-25T09:53:20.174696+0000","last_became_active":"2023-08-25T09:38:47.023029+0000","last_became_peered":"2023-08-25T09:38:47.023029+0000","last_unstale":"2023-08-25T09:53:20.174696+0000","last_undegraded":"2023-08-25T09:53:20.174696+0000","last_fullsized":"2023-08-25T09:53:20.174696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:20.174490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:20.174490+0000","last_clean_scrub_stamp":"2023-08-25T09:53:20.174490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:02:30.846817+0000","scrub_duration":0.0028805279999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":381,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:21.937772+0000","last_change":"2023-08-25T09:53:21.937772+0000","last_active":"2023-08-25T09:53:21.937772+0000","last_peered":"2023-08-25T09:53:21.937772+0000","last_clean":"2023-08-25T09:53:21.937772+0000","last_became_active":"2023-08-25T09:38:47.022099+0000","last_became_peered":"2023-08-25T09:38:47.022099+0000","last_unstale":"2023-08-25T09:53:21.937772+0000","last_undegraded":"2023-08-25T09:53:21.937772+0000","last_fullsized":"2023-08-25T09:53:21.937772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:21.937533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:21.937533+0000","last_clean_scrub_stamp":"2023-08-25T09:53:21.937533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T21:14:33.258491+0000","scrub_duration":0.0031411439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":348,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:16.460456+0000","last_change":"2023-08-25T09:53:16.460560+0000","last_active":"2023-08-25T09:53:16.460456+0000","last_peered":"2023-08-25T09:53:16.460456+0000","last_clean":"2023-08-25T09:53:16.460456+0000","last_became_active":"2023-08-25T09:38:47.150875+0000","last_became_peered":"2023-08-25T09:38:47.150875+0000","last_unstale":"2023-08-25T09:53:16.460456+0000","last_undegraded":"2023-08-25T09:53:16.460456+0000","last_fullsized":"2023-08-25T09:53:16.460456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:16.460358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:16.460358+0000","last_clean_scrub_stamp":"2023-08-25T09:53:16.460358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T18:29:57.267378+0000","scrub_duration":0.002893839,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":380,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-08-25T09:53:23.880139+0000","last_change":"2023-08-25T09:53:23.880233+0000","last_active":"2023-08-25T09:53:23.880139+0000","last_peered":"2023-08-25T09:53:23.880139+0000","last_clean":"2023-08-25T09:53:23.880139+0000","last_became_active":"2023-08-25T09:38:47.150714+0000","last_became_peered":"2023-08-25T09:38:47.150714+0000","last_unstale":"2023-08-25T09:53:23.880139+0000","last_undegraded":"2023-08-25T09:53:23.880139+0000","last_fullsized":"2023-08-25T09:53:23.880139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T09:53:23.880049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T09:53:23.880049+0000","last_clean_scrub_stamp":"2023-08-25T09:53:23.880049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:48:59.575223+0000","scrub_duration":0.0026402679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.1399999999999999,"15min":1.135},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.35699999999999998},"max":{"1min":1.7430000000000001,"5min":1.829,"15min":10.747},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1679999999999999,"15min":1.1220000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.43099999999999999},"max":{"1min":1.605,"5min":1.669,"15min":1.855},"last":1.018}]},{"osd":1,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0680000000000001,"5min":1.2569999999999999,"15min":1.244},"min":{"1min":0.63300000000000001,"5min":0.61499999999999999,"15min":0.48999999999999999},"max":{"1min":1.5569999999999999,"5min":1.823,"15min":11.010999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.212,"15min":1.1890000000000001},"min":{"1min":0.72599999999999998,"5min":0.68600000000000005,"15min":0.57999999999999996},"max":{"1min":1.6100000000000001,"5min":1.7410000000000001,"15min":1.7769999999999999},"last":1.1919999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.1970000000000001,"15min":1.1539999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.50600000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.94399999999999995},{"interface":"front","average":{"1min":1.179,"5min":1.288,"15min":1.276},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.621,"5min":1.873,"15min":1.98},"last":1.3839999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2629999999999999,"15min":1.2250000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.56799999999999995},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3620000000000001,"15min":1.3640000000000001},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.69299999999999995},"max":{"1min":1.651,"5min":1.833,"15min":2.1030000000000002},"last":1.163}]},{"osd":4,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2629999999999999,"15min":1.288},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":1.712,"5min":1.845,"15min":7.1639999999999997},"last":1.107},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.323},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.46300000000000002},"max":{"1min":1.8149999999999999,"5min":1.8149999999999999,"15min":1.8779999999999999},"last":1.337}]},{"osd":5,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.4059999999999999,"15min":1.419},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.6919999999999999,"5min":2.0059999999999998,"15min":2.2389999999999999},"last":1.296},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.427,"15min":1.4330000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.827,"5min":1.907,"15min":2.0739999999999998},"last":1.7250000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4490000000000001,"5min":1.538,"15min":1.522},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.929,"5min":1.984,"15min":2.1720000000000002},"last":1.587},{"interface":"front","average":{"1min":1.381,"5min":1.454,"15min":1.4630000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.9530000000000001,"5min":2.1019999999999999,"15min":2.3220000000000001},"last":1.679}]}]},{"osd":6,"up_from":13,"seq":55834575029,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.1739999999999999,"15min":1.149},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.54400000000000004},"max":{"1min":1.591,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1950000000000001,"15min":1.1759999999999999},"min":{"1min":0.81000000000000005,"5min":0.68500000000000005,"15min":0.60799999999999998},"max":{"1min":1.5509999999999999,"5min":1.645,"15min":5.5410000000000004},"last":1.3320000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.262,"15min":1.228},"min":{"1min":0.88700000000000001,"5min":0.78800000000000003,"15min":0.54700000000000004},"max":{"1min":1.6140000000000001,"5min":1.8380000000000001,"15min":5.6799999999999997},"last":1.456},{"interface":"front","average":{"1min":1.139,"5min":1.194,"15min":1.1830000000000001},"min":{"1min":0.81299999999999994,"5min":0.75800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6359999999999999,"5min":1.8100000000000001,"15min":2.3079999999999998},"last":0.90000000000000002}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1930000000000001,"15min":1.1619999999999999},"min":{"1min":0.80800000000000005,"5min":0.72299999999999998,"15min":0.60399999999999998},"max":{"1min":1.534,"5min":1.696,"15min":1.708},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.47299999999999998},"max":{"1min":1.7350000000000001,"5min":1.7889999999999999,"15min":13.193},"last":1.5840000000000001}]},{"osd":3,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.369,"15min":1.319},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.68899999999999995},"max":{"1min":1.6519999999999999,"5min":1.911,"15min":5.6239999999999997},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.2969999999999999,"15min":1.276},"min":{"1min":1.071,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.8720000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":0.96099999999999997}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.331,"15min":1.3280000000000001},"min":{"1min":1.004,"5min":0.90500000000000003,"15min":0.42599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.484},"last":1.252},{"interface":"front","average":{"1min":1.482,"5min":1.419,"15min":1.3939999999999999},"min":{"1min":0.95999999999999996,"5min":0.78900000000000003,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":2.101,"15min":5.4699999999999998},"last":1.6619999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.3999999999999999,"15min":1.3440000000000001},"min":{"1min":0.94499999999999995,"5min":0.75,"15min":0.59599999999999997},"max":{"1min":1.875,"5min":1.9670000000000001,"15min":2.0049999999999999},"last":1.3},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.4730000000000001},"min":{"1min":1.1639999999999999,"5min":1,"15min":0.59099999999999997},"max":{"1min":2.121,"5min":2.121,"15min":2.4420000000000002},"last":1.4970000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.6379999999999999,"5min":1.5449999999999999,"15min":1.538},"min":{"1min":1.157,"5min":0.89400000000000002,"15min":0.622},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":6.7709999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.548,"15min":1.5089999999999999},"min":{"1min":1.24,"5min":0.872,"15min":0.57399999999999995},"max":{"1min":2.181,"5min":2.1960000000000002,"15min":2.1960000000000002},"last":1.379}]}]},{"osd":5,"up_from":13,"seq":55834575029,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2669999999999999,"15min":1.208},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.497},"max":{"1min":1.667,"5min":1.9770000000000001,"15min":2.081},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.2390000000000001,"15min":1.2210000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.65400000000000003},"max":{"1min":1.5309999999999999,"5min":1.6950000000000001,"15min":1.974},"last":1.5740000000000001}]},{"osd":1,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.145,"15min":1.157},"min":{"1min":0.72799999999999998,"5min":0.67000000000000004,"15min":0.60199999999999998},"max":{"1min":1.5569999999999999,"5min":1.6899999999999999,"15min":2.1779999999999999},"last":1.365},{"interface":"front","average":{"1min":1.339,"5min":1.2010000000000001,"15min":1.1779999999999999},"min":{"1min":0.84399999999999997,"5min":0.74099999999999999,"15min":0.55000000000000004},"max":{"1min":1.76,"5min":1.76,"15min":2.1869999999999998},"last":1.4019999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.3089999999999999,"15min":1.258},"min":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.68400000000000005},"max":{"1min":1.909,"5min":1.909,"15min":2.3690000000000002},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.256,"5min":1.2649999999999999,"15min":1.2549999999999999},"min":{"1min":0.873,"5min":0.75,"15min":0.72599999999999998},"max":{"1min":1.6679999999999999,"5min":1.847,"15min":1.962},"last":1.0649999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.236,"15min":1.28},"min":{"1min":0.70099999999999996,"5min":0.66600000000000004,"15min":0.57999999999999996},"max":{"1min":1.53,"5min":1.7649999999999999,"15min":17.356999999999999},"last":1.52},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.343,"15min":1.3300000000000001},"min":{"1min":0.85399999999999998,"5min":0.77100000000000002,"15min":0.65700000000000003},"max":{"1min":1.696,"5min":1.7490000000000001,"15min":1.9650000000000001},"last":1.196}]},{"osd":4,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3280000000000001,"15min":1.3220000000000001},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.55500000000000005},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.625},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.304,"15min":1.329},"min":{"1min":0.746,"5min":0.746,"15min":0.58799999999999997},"max":{"1min":1.556,"5min":1.7290000000000001,"15min":2.2789999999999999},"last":1.2989999999999999}]},{"osd":6,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4450000000000001,"15min":1.4430000000000001},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.58899999999999997},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":2.4820000000000002},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.516,"15min":1.4950000000000001},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.60399999999999998},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.5539999999999998},"last":1.5920000000000001}]},{"osd":7,"last update":"Fri Aug 25 09:53:06 2023","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.95499999999999996,"5min":0.91700000000000004,"15min":0.63},"max":{"1min":1.6359999999999999,"5min":2.1080000000000001,"15min":2.2719999999999998},"last":1.448},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5700000000000001,"15min":1.5780000000000001},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.63900000000000001},"max":{"1min":2.2040000000000002,"5min":2.2040000000000002,"15min":2.3700000000000001},"last":1.1479999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.274,"15min":1.228},"min":{"1min":0.71199999999999997,"5min":0.624,"15min":0.34399999999999997},"max":{"1min":1.617,"5min":4.4260000000000002,"15min":4.4260000000000002},"last":1.123},{"interface":"front","average":{"1min":1.228,"5min":1.2529999999999999,"15min":1.258},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.434},"max":{"1min":2.3959999999999999,"5min":2.5150000000000001,"15min":7.367},"last":1.629}]},{"osd":1,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.159,"15min":1.1439999999999999},"min":{"1min":0.88,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.248,"5min":1.256,"15min":1.2589999999999999},"min":{"1min":0.69499999999999995,"5min":0.56200000000000006,"15min":0.49299999999999999},"max":{"1min":1.7010000000000001,"5min":4.351,"15min":8.5850000000000009},"last":1.349}]},{"osd":2,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2010000000000001,"15min":1.208},"min":{"1min":0.89500000000000002,"5min":0.71999999999999997,"15min":0.68400000000000005},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":4.1669999999999998},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.2949999999999999,"15min":1.266},"min":{"1min":0.92500000000000004,"5min":0.71399999999999997,"15min":0.63600000000000001},"max":{"1min":1.899,"5min":4.7320000000000002,"15min":4.7320000000000002},"last":1.298}]},{"osd":3,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.29,"15min":1.274},"min":{"1min":0.89700000000000002,"5min":0.79400000000000004,"15min":0.61899999999999999},"max":{"1min":3.0550000000000002,"5min":3.0550000000000002,"15min":7.2560000000000002},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4059999999999999,"15min":1.385},"min":{"1min":1.0800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.2769999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.351,"15min":1.3140000000000001},"min":{"1min":0.81200000000000006,"5min":0.67100000000000004,"15min":0.55200000000000005},"max":{"1min":5.5910000000000002,"5min":5.5910000000000002,"15min":5.5910000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.5409999999999999,"5min":1.3400000000000001,"15min":1.3260000000000001},"min":{"1min":0.872,"5min":0.67500000000000004,"15min":0.57099999999999995},"max":{"1min":5.7389999999999999,"5min":5.7389999999999999,"15min":8.2409999999999997},"last":1.76}]},{"osd":6,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.6499999999999999,"5min":1.4730000000000001,"15min":1.446},"min":{"1min":0.85299999999999998,"5min":0.65800000000000003,"15min":0.52000000000000002},"max":{"1min":6.1980000000000004,"5min":6.2220000000000004,"15min":6.2220000000000004},"last":1.569},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.6399999999999999,"15min":1.5920000000000001},"min":{"1min":1.0840000000000001,"5min":0.69799999999999995,"15min":0.67100000000000004},"max":{"1min":6.069,"5min":7.0990000000000002,"15min":7.0990000000000002},"last":1.462}]},{"osd":7,"last update":"Fri Aug 25 09:53:03 2023","interfaces":[{"interface":"back","average":{"1min":1.528,"5min":1.496,"15min":1.502},"min":{"1min":1.119,"5min":0.75700000000000001,"15min":0.63800000000000001},"max":{"1min":1.9490000000000001,"5min":3.5409999999999999,"15min":8.1760000000000002},"last":1.861},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5740000000000001,"15min":1.5900000000000001},"min":{"1min":0.96599999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.9900000000000002,"5min":3.831,"15min":3.831},"last":1.7050000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.96299999999999997,"15min":0.91500000000000004},"min":{"1min":0.60299999999999998,"5min":0.36699999999999999,"15min":0.29299999999999998},"max":{"1min":1.4159999999999999,"5min":1.8180000000000001,"15min":1.8999999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.0840000000000001,"15min":0.998},"min":{"1min":0.71299999999999997,"5min":0.5,"15min":0.41699999999999998},"max":{"1min":1.552,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.254}]},{"osd":1,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.90600000000000003,"15min":0.86799999999999999},"min":{"1min":0.66800000000000004,"5min":0.43099999999999999,"15min":0.35699999999999998},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.702},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.103,"15min":1.0289999999999999},"min":{"1min":0.71799999999999997,"5min":0.52400000000000002,"15min":0.45700000000000002},"max":{"1min":1.5740000000000001,"5min":1.631,"15min":2.3399999999999999},"last":1.3200000000000001}]},{"osd":2,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.0589999999999999,"15min":0.98599999999999999},"min":{"1min":0.81799999999999995,"5min":0.48899999999999999,"15min":0.378},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.919},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.268,"5min":1.2250000000000001,"15min":1.1539999999999999},"min":{"1min":0.94899999999999995,"5min":0.61099999999999999,"15min":0.44900000000000001},"max":{"1min":1.702,"5min":1.901,"15min":3.399},"last":1.4590000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3340000000000001,"15min":1.2490000000000001},"min":{"1min":1.0169999999999999,"5min":0.78300000000000003,"15min":0.66800000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.1949999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.222,"15min":1.2150000000000001},"min":{"1min":0.95699999999999996,"5min":0.629,"15min":0.629},"max":{"1min":1.6539999999999999,"5min":1.79,"15min":2.1840000000000002},"last":1.175}]},{"osd":5,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4079999999999999,"15min":1.355},"min":{"1min":1.1060000000000001,"5min":0.78700000000000003,"15min":0.71899999999999997},"max":{"1min":1.881,"5min":2.1930000000000001,"15min":2.407},"last":1.522},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.3779999999999999,"15min":1.3160000000000001},"min":{"1min":1.151,"5min":0.66600000000000004,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":1.806,"15min":1.895},"last":1.702}]},{"osd":6,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.3280000000000001,"15min":1.292},"min":{"1min":0.97999999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.8799999999999999,"5min":1.8799999999999999,"15min":2.3450000000000002},"last":1.298},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.49,"15min":1.427},"min":{"1min":1.1200000000000001,"5min":0.82899999999999996,"15min":0.82199999999999995},"max":{"1min":1.9830000000000001,"5min":2.3210000000000002,"15min":3.496},"last":1.577}]},{"osd":7,"last update":"Fri Aug 25 09:53:07 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.498,"15min":1.4199999999999999},"min":{"1min":1.3009999999999999,"5min":0.95299999999999996,"15min":0.83699999999999997},"max":{"1min":1.9390000000000001,"5min":1.976,"15min":2.2989999999999999},"last":1.611},{"interface":"front","average":{"1min":1.4770000000000001,"5min":1.4570000000000001,"15min":1.3999999999999999},"min":{"1min":1.093,"5min":0.78400000000000003,"15min":0.68400000000000005},"max":{"1min":2.0139999999999998,"5min":2.0139999999999998,"15min":2.1509999999999998},"last":1.425}]}]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.86299999999999999,"15min":0.84499999999999997},"min":{"1min":0.42699999999999999,"5min":0.34300000000000003,"15min":0.29999999999999999},"max":{"1min":1.2490000000000001,"5min":1.6930000000000001,"15min":2.383},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.018,"5min":0.94199999999999995,"15min":0.95599999999999996},"min":{"1min":0.56299999999999994,"5min":0.42699999999999999,"15min":0.38400000000000001},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.7309999999999999},"last":1.4199999999999999}]},{"osd":1,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.97399999999999998,"15min":0.97999999999999998},"min":{"1min":0.503,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3340000000000001,"5min":1.635,"15min":2.3180000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0309999999999999},"min":{"1min":0.50900000000000001,"5min":0.45100000000000001,"15min":0.439},"max":{"1min":1.391,"5min":1.704,"15min":2.8599999999999999},"last":1.5389999999999999}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.92900000000000005,"15min":1.026},"min":{"1min":0.54300000000000004,"5min":0.40100000000000002,"15min":0.36099999999999999},"max":{"1min":1.214,"5min":1.5589999999999999,"15min":18.821000000000002},"last":1.962},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.127,"15min":1.1000000000000001},"min":{"1min":0.59499999999999997,"5min":0.54600000000000004,"15min":0.45500000000000002},"max":{"1min":1.4430000000000001,"5min":1.7390000000000001,"15min":2.2040000000000002},"last":1.264}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.3180000000000001,"15min":1.3100000000000001},"min":{"1min":0.90300000000000002,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.9219999999999999,"5min":1.9470000000000001,"15min":3.0150000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.288,"15min":1.286},"min":{"1min":0.77700000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.6799999999999999,"5min":1.8,"15min":2.7639999999999998},"last":1.623}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2509999999999999,"15min":1.2669999999999999},"min":{"1min":0.79700000000000004,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.6339999999999999,"5min":1.849,"15min":5.1269999999999998},"last":2.0539999999999998},{"interface":"front","average":{"1min":1.341,"5min":1.373,"15min":1.363},"min":{"1min":0.878,"5min":0.625,"15min":0.625},"max":{"1min":1.7390000000000001,"5min":1.8200000000000001,"15min":2.4220000000000002},"last":1.171}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.407,"15min":1.399},"min":{"1min":0.97799999999999998,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.0059999999999998,"5min":2.0840000000000001,"15min":4.0170000000000003},"last":1.7829999999999999},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.389,"15min":1.4039999999999999},"min":{"1min":0.98799999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6919999999999999,"5min":2.028,"15min":5.0599999999999996},"last":2.2189999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.381,"15min":1.3859999999999999},"min":{"1min":0.81699999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2,"15min":2.968},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.3640000000000001,"15min":1.4079999999999999},"min":{"1min":0.85999999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.96,"15min":5.2990000000000004},"last":2.1379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1784,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193527,"statfs":{"total":200000000,"available":198172512,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.85399999999999998,"15min":0.83099999999999996},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.3240000000000001,"5min":1.9470000000000001,"15min":1.9470000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.871,"5min":0.95199999999999996,"15min":0.90600000000000003},"min":{"1min":0.46800000000000003,"5min":0.315,"15min":0.315},"max":{"1min":1.333,"5min":2.2389999999999999,"15min":2.2389999999999999},"last":1.1859999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.95299999999999996,"15min":0.89400000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.5940000000000001,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.036,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.496,"15min":0.33600000000000002},"max":{"1min":1.474,"5min":1.6619999999999999,"15min":1.746},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1479999999999999,"15min":1.117},"min":{"1min":0.55100000000000005,"5min":0.496,"15min":0.372},"max":{"1min":1.6299999999999999,"5min":1.647,"15min":1.9450000000000001},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.04,"15min":1.02},"min":{"1min":0.39700000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.657,"5min":1.657,"15min":1.764},"last":1.3080000000000001}]},{"osd":4,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.175,"15min":1.1539999999999999},"min":{"1min":0.76200000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.5209999999999999,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2949999999999999,"15min":1.286},"min":{"1min":0.85799999999999998,"5min":0.67700000000000005,"15min":0.629},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.3490000000000002},"last":1.2290000000000001}]},{"osd":5,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.278,"15min":1.2509999999999999},"min":{"1min":0.70099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.1970000000000001,"5min":2.1970000000000001,"15min":2.1970000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.3129999999999999,"15min":1.2549999999999999},"min":{"1min":0.86799999999999999,"5min":0.66400000000000003,"15min":0.61199999999999999},"max":{"1min":1.647,"5min":2.867,"15min":2.867},"last":1.52}]},{"osd":6,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.369,"15min":1.371},"min":{"1min":0.96899999999999997,"5min":0.79300000000000004,"15min":0.73199999999999998},"max":{"1min":2.093,"5min":2.093,"15min":2.9940000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.415,"15min":1.3759999999999999},"min":{"1min":0.95399999999999996,"5min":0.86299999999999999,"15min":0.72099999999999997},"max":{"1min":2.036,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":1.2529999999999999}]},{"osd":7,"last update":"Fri Aug 25 09:53:05 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4419999999999999,"15min":1.419},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.79200000000000004},"max":{"1min":1.704,"5min":2.1360000000000001,"15min":2.9529999999999998},"last":1.411},{"interface":"front","average":{"1min":1.399,"5min":1.4139999999999999,"15min":1.4079999999999999},"min":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.218},"last":1.492}]}]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1208,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194104,"statfs":{"total":200000000,"available":198762880,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.86499999999999999,"15min":0.88300000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.313},"max":{"1min":1.2969999999999999,"5min":2.2759999999999998,"15min":11.257},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.999,"15min":0.96999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.32800000000000001},"max":{"1min":1.694,"5min":1.762,"15min":6.4260000000000002},"last":1.3169999999999999}]},{"osd":2,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.97699999999999998,"15min":0.95799999999999996},"min":{"1min":0.46899999999999997,"5min":0.45300000000000001,"15min":0.36199999999999999},"max":{"1min":1.488,"5min":1.522,"15min":1.8620000000000001},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0760000000000001,"15min":1.0389999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.374},"max":{"1min":1.788,"5min":1.788,"15min":3.4889999999999999},"last":0.871}]},{"osd":3,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1,"15min":0.93100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.40699999999999997},"max":{"1min":1.448,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.756},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.1459999999999999,"15min":1.115},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.39700000000000002},"max":{"1min":1.5449999999999999,"5min":1.5860000000000001,"15min":2.3809999999999998},"last":1.03}]},{"osd":4,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.278,"15min":1.252},"min":{"1min":0.71499999999999997,"5min":0.71399999999999997,"15min":0.59999999999999998},"max":{"1min":1.4930000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":1.6040000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.4239999999999999,"15min":1.377},"min":{"1min":0.94799999999999995,"5min":0.82399999999999995,"15min":0.65500000000000003},"max":{"1min":1.595,"5min":1.9770000000000001,"15min":2.5499999999999998},"last":1.5029999999999999}]},{"osd":5,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.351,"15min":1.306},"min":{"1min":0.73099999999999998,"5min":0.623,"15min":0.623},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":3.3769999999999998},"last":1.714},{"interface":"front","average":{"1min":1.256,"5min":1.292,"15min":1.2729999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.51400000000000001},"max":{"1min":1.829,"5min":2.3119999999999998,"15min":2.3220000000000001},"last":1.4390000000000001}]},{"osd":6,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4099999999999999,"15min":1.359},"min":{"1min":0.83599999999999997,"5min":0.81499999999999995,"15min":0.71399999999999997},"max":{"1min":1.8759999999999999,"5min":2.056,"15min":2.6869999999999998},"last":1.173},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.401,"15min":1.3420000000000001},"min":{"1min":0.77300000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.331},"last":1.548}]},{"osd":7,"last update":"Fri Aug 25 09:53:10 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.446,"15min":1.407},"min":{"1min":0.88900000000000001,"5min":0.78900000000000003,"15min":0.71099999999999997},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.6429999999999998},"last":1.659},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3899999999999999,"15min":1.3440000000000001},"min":{"1min":0.81999999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.9319999999999999,"5min":2.4140000000000001,"15min":2.7410000000000001},"last":1.889}]}]}],"pool_statfs":[]}} 2023-08-25T09:53:39.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-08-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.320+0000 7fe3b6b6d640 1 Processor -- start 2023-08-25T09:53:39.322 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.320+0000 7fe3b6b6d640 1 -- start start 2023-08-25T09:53:39.325 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b6b6d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 0x7fe3b00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b6b6d640 1 --2- >> v2:172.21.15.155:3300/0 conn(0x7fe3b00ca450 0x7fe3b00b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b6b6d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b015ac10 con 0x7fe3b00ad340 2023-08-25T09:53:39.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b6b6d640 1 -- --> v2:172.21.15.155:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b00adb40 con 0x7fe3b00ca450 2023-08-25T09:53:39.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b6b6d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00cac80 con 0x7fe3b00b0300 2023-08-25T09:53:39.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b636c640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7fe3b00ad340 0x7fe3b006e960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46932/0 (socket says 172.21.15.70:46932) 2023-08-25T09:53:39.326 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b636c640 1 -- 172.21.15.70:0/2681467971 learned_addr learned my addr 172.21.15.70:0/2681467971 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:39.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 0x7fe3b00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.322+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862148338 0 0) 0x7fe3b015ac10 con 0x7fe3b00ad340 2023-08-25T09:53:39.327 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c003520 con 0x7fe3b00ad340 2023-08-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b536a640 1 --2- 172.21.15.70:0/2681467971 >> v2:172.21.15.155:3300/0 conn(0x7fe3b00ca450 0x7fe3b00b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902700730 0 0) 0x7fe39c003520 con 0x7fe3b00ad340 2023-08-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 msgr2=0x7fe3b00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.328 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 0x7fe3b00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 >> v2:172.21.15.155:3300/0 conn(0x7fe3b00ca450 msgr2=0x7fe3b00b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 --2- 172.21.15.70:0/2681467971 >> v2:172.21.15.155:3300/0 conn(0x7fe3b00ca450 0x7fe3b00b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00b0c20 con 0x7fe3b00ad340 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7fe3a8001a10 con 0x7fe3b00ad340 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 0x7fe3b00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 >> v1:172.21.15.70:6789/0 conn(0x7fe3b00ad340 legacy=0x7fe3b006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.329 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe39c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe39c0047c0 0x7fe39c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 0x7fe39c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00adb40 con 0x7fe39c003970 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00cac80 con 0x7fe39c0047c0 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00bab0 con 0x7fe3b00b0300 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.323+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe39c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.324+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 msgr2=0x7fe39c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.330 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.324+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00b0300 0x7fe39c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.324+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe39c0047c0 msgr2=0x7fe39c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.324+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe39c0047c0 0x7fe39c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.331 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.324+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/2681467971 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c003520 con 0x7fe39c003970 2023-08-25T09:53:39.333 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.325+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe39c003d90 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009880 tx=0x7fe39c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aec96e0b99ad870 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.325+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac030400 con 0x7fe39c003970 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.325+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac030560 con 0x7fe39c003970 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.325+0000 7fe3b4b69640 1 -- 172.21.15.70:0/2681467971 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0308d0 con 0x7fe39c003970 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/2681467971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 msgr2=0x7fe39c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/2681467971 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe39c003d90 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009880 tx=0x7fe39c0057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b636c640 1 -- 172.21.15.70:0/2681467971 reap_dead start 2023-08-25T09:53:39.334 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/2681467971 shutdown_connections 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/2681467971 >> 172.21.15.70:0/2681467971 conn(0x7fe3b00ce810 msgr2=0x7fe3b00cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/2681467971 shutdown_connections 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.326+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/2681467971 wait complete. 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.328+0000 7fe3b6b6d640 1 Processor -- start 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.328+0000 7fe3b6b6d640 1 -- start start 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b6b6d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe3b0159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.335 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b6b6d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00c5dd0 0x7fe3b00977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b6b6d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b015c4b0 0x7fe3b015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b6b6d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00c65b0 con 0x7fe39c003970 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b6b6d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b015b7c0 con 0x7fe3b015c4b0 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b6b6d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b015c020 con 0x7fe3b00c5dd0 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe3b0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe3b0159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39510/0 (socket says 172.21.15.70:39510) 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/3846599173 learned_addr learned my addr 172.21.15.70:0/3846599173 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:39.336 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b536a640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00c5dd0 0x7fe3b00977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00c5dd0 msgr2=0x7fe3b00977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00c5dd0 0x7fe3b00977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b015c4b0 msgr2=0x7fe3b015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b015c4b0 0x7fe3b015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.329+0000 7fe3b5b6b640 1 -- 172.21.15.70:0/3846599173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b00c62d0 con 0x7fe39c003970 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.330+0000 7fe3b5b6b640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe3b0159330 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0099a0 tx=0x7fe3ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:39.337 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.330+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac03e040 con 0x7fe39c003970 2023-08-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.330+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac0399a0 con 0x7fe39c003970 2023-08-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.330+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac039d10 con 0x7fe39c003970 2023-08-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.330+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b015b9d0 con 0x7fe39c003970 2023-08-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.330+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b0159870 con 0x7fe39c003970 2023-08-25T09:53:39.338 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.332+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0061460 con 0x7fe39c003970 2023-08-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.332+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe3ac043440 con 0x7fe39c003970 2023-08-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.333+0000 7fe3a67fc640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe39406a080 0x7fe39406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.334+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe3ac0a9f30 con 0x7fe39c003970 2023-08-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.334+0000 7fe3b536a640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe39406a080 0x7fe39406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.339 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.335+0000 7fe3b536a640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe39406a080 0x7fe39406c540 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ab000 tx=0x7fe3a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:39.340 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.338+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3ac075a60 con 0x7fe39c003970 2023-08-25T09:53:39.528 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.527+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe3b00abb90 con 0x7fe39c003970 2023-08-25T09:53:39.533 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.532+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe3ac075a60 con 0x7fe39c003970 2023-08-25T09:53:39.534 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.532+0000 7fe3a67fc640 1 -- 172.21.15.70:0/3846599173 <== mon.0 v2:172.21.15.70:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe3ac02d020 con 0x7fe39c003970 2023-08-25T09:53:39.536 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe39406a080 msgr2=0x7fe39406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe39406a080 0x7fe39406c540 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ab000 tx=0x7fe3a0008040 comp rx=0 tx=0).stop 2023-08-25T09:53:39.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 msgr2=0x7fe3b0159330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe3b0159330 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0099a0 tx=0x7fe3ac0046c0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 shutdown_connections 2023-08-25T09:53:39.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7fe39406a080 0x7fe39406c540 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.537 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7fe3b015c4b0 0x7fe3b015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7fe3b00c5dd0 0x7fe3b00977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 --2- 172.21.15.70:0/3846599173 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7fe39c003970 0x7fe3b0159330 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 >> 172.21.15.70:0/3846599173 conn(0x7fe3b00ce810 msgr2=0x7fe3b00b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:39.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 shutdown_connections 2023-08-25T09:53:39.538 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.535+0000 7fe3b6b6d640 1 -- 172.21.15.70:0/3846599173 wait complete. 2023-08-25T09:53:39.556 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T09:53:39.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T09:53:39.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.767+0000 7f721875d640 1 Processor -- start 2023-08-25T09:53:39.769 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.768+0000 7f721875d640 1 -- start start 2023-08-25T09:53:39.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.769+0000 7f721875d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 0x7f72100b2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.769+0000 7f721875d640 1 -- --> v1:172.21.15.70:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721015d3f0 con 0x7f72100b2680 2023-08-25T09:53:39.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.769+0000 7f721875d640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72100b2d20 con 0x7f72100ce5b0 2023-08-25T09:53:39.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.769+0000 7f721875d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72100b2b10 con 0x7f721015aa10 2023-08-25T09:53:39.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.770+0000 7f7216cfa640 1 --1- >> v1:172.21.15.70:6789/0 conn(0x7f72100b2680 0x7f721006f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.70:6789/0 says I am v1:172.21.15.70:46940/0 (socket says 172.21.15.70:46940) 2023-08-25T09:53:39.771 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.770+0000 7f7216cfa640 1 -- 172.21.15.70:0/241442860 learned_addr learned my addr 172.21.15.70:0/241442860 (peer_addr_for_me v1:172.21.15.70:0/0) 2023-08-25T09:53:39.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.770+0000 7f7215cf8640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 0x7f72100b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.770+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 <== mon.0 v1:172.21.15.70:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116766229 0 0) 0x7f721015d3f0 con 0x7f72100b2680 2023-08-25T09:53:39.773 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.770+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 --> v1:172.21.15.70:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71fc003520 con 0x7f72100b2680 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 <== mon.0 v1:172.21.15.70:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104329837 0 0) 0x7f71fc003520 con 0x7f72100b2680 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 msgr2=0x7f72100b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 0x7f72100b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 >> v1:172.21.15.155:6789/0 conn(0x7f72100ce5b0 legacy=0x7f72100ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 --> v1:172.21.15.70:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721015d6b0 con 0x7f72100b2680 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 <== mon.0 v1:172.21.15.70:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3701798569 0 0) 0x7f72080019f0 con 0x7f72100b2680 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 >> v1:172.21.15.70:6789/0 conn(0x7f72100b2680 legacy=0x7f721006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.774 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71fc0039a0 0x7f71fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 0x7f71fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71fc004d00 0x7f71fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100b2b10 con 0x7f71fc0039a0 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc00ba60 con 0x7f71fc004d00 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc00de30 con 0x7f721015aa10 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72164f9640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71fc0039a0 0x7f71fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.775 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f7215cf8640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 0x7f71fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.771+0000 7f72164f9640 1 -- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 msgr2=0x7f71fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.772+0000 7f72164f9640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f721015aa10 0x7f71fc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.772+0000 7f72164f9640 1 -- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71fc004d00 msgr2=0x7f71fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T09:53:39.776 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.772+0000 7f7216cfa640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71fc004d00 0x7f71fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.772+0000 7f72164f9640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71fc004d00 0x7f71fc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.772+0000 7f72164f9640 1 -- 172.21.15.70:0/241442860 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc003520 con 0x7f71fc0039a0 2023-08-25T09:53:39.777 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.772+0000 7f7216cfa640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f71fc004d00 0x7f71fc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T09:53:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.773+0000 7f72164f9640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71fc0039a0 0x7f71fc003dc0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f720c001720 tx=0x7f71fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415395cba7d1b8ee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.773+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0052d0 con 0x7f71fc0039a0 2023-08-25T09:53:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.773+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f720c02e050 con 0x7f71fc0039a0 2023-08-25T09:53:39.779 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.773+0000 7f72154f7640 1 -- 172.21.15.70:0/241442860 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0035f0 con 0x7f71fc0039a0 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f721875d640 1 -- 172.21.15.70:0/241442860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71fc0039a0 msgr2=0x7f71fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f721875d640 1 --2- 172.21.15.70:0/241442860 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f71fc0039a0 0x7f71fc003dc0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f720c001720 tx=0x7f71fc0057f0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f7216cfa640 1 -- 172.21.15.70:0/241442860 reap_dead start 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f721875d640 1 -- 172.21.15.70:0/241442860 shutdown_connections 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f721875d640 1 -- 172.21.15.70:0/241442860 >> 172.21.15.70:0/241442860 conn(0x7f7210069850 msgr2=0x7f7210069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f721875d640 1 -- 172.21.15.70:0/241442860 shutdown_connections 2023-08-25T09:53:39.780 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.774+0000 7f721875d640 1 -- 172.21.15.70:0/241442860 wait complete. 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.776+0000 7f721875d640 1 Processor -- start 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.776+0000 7f721875d640 1 -- start start 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f721875d640 1 --2- >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71fc0039a0 0x7f721014e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f721875d640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 0x7f72101f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f721875d640 1 --2- >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72101503b0 0x7f7210150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f721875d640 1 -- --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721014f690 con 0x7f721014ab60 2023-08-25T09:53:39.781 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f721875d640 1 -- --> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72100cd6a0 con 0x7f72101503b0 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f721875d640 1 -- --> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7210150d80 con 0x7f71fc0039a0 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f7215cf8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 0x7f72101f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f7215cf8640 1 --2- >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 0x7f72101f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.70:3300/0 says I am v2:172.21.15.70:39522/0 (socket says 172.21.15.70:39522) 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.777+0000 7f7215cf8640 1 -- 172.21.15.70:0/475758692 learned_addr learned my addr 172.21.15.70:0/475758692 (peer_addr_for_me v2:172.21.15.70:0/0) 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7215cf8640 1 -- 172.21.15.70:0/475758692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71fc0039a0 msgr2=0x7f721014e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7215cf8640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71fc0039a0 0x7f721014e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7215cf8640 1 -- 172.21.15.70:0/475758692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72101503b0 msgr2=0x7f7210150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:39.782 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7215cf8640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72101503b0 0x7f7210150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:39.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7215cf8640 1 -- 172.21.15.70:0/475758692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72101f7c30 con 0x7f721014ab60 2023-08-25T09:53:39.783 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7215cf8640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 0x7f72101f76f0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f7200010890 tx=0x7f7200010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:39.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7200009610 con 0x7f721014ab60 2023-08-25T09:53:39.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f720000eb00 con 0x7f721014ab60 2023-08-25T09:53:39.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720000eeb0 con 0x7f721014ab60 2023-08-25T09:53:39.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721014ea10 con 0x7f721014ab60 2023-08-25T09:53:39.784 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.778+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721014efd0 con 0x7f721014ab60 2023-08-25T09:53:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.780+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721004e420 con 0x7f721014ab60 2023-08-25T09:53:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.780+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f720001c400 con 0x7f721014ab60 2023-08-25T09:53:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.781+0000 7f7206ffd640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71f406a080 0x7f71f406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T09:53:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.781+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f72000887d0 con 0x7f721014ab60 2023-08-25T09:53:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.782+0000 7f72164f9640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71f406a080 0x7f71f406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T09:53:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.782+0000 7f72164f9640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71f406a080 0x7f71f406c540 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f720c033000 tx=0x7f720c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T09:53:39.787 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:39.785+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7200054130 con 0x7f721014ab60 2023-08-25T09:53:40.013 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.012+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 --> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f721014f7d0 con 0x7f721014ab60 2023-08-25T09:53:40.014 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.013+0000 7f7206ffd640 1 -- 172.21.15.70:0/475758692 <== mon.0 v2:172.21.15.70:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3706 (secure 0 0 0) 0x7f7200021e90 con 0x7f721014ab60 2023-08-25T09:53:40.015 INFO:teuthology.orchestra.run.smithi070.stdout: 2023-08-25T09:53:40.015 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5759,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6835/1710720377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6833","nonce":1710720377},{"type":"v1","addr":"172.21.15.155:6835","nonce":1710720377}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6478,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6837/1550953700","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6835","nonce":1550953700},{"type":"v1","addr":"172.21.15.70:6837","nonce":1550953700}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":6481,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.155:6834/2118498856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.155:6832","nonce":2118498856},{"type":"v1","addr":"172.21.15.155:6834","nonce":2118498856}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":10017,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.70:6836/1044634698","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6834","nonce":1044634698},{"type":"v1","addr":"172.21.15.70:6836","nonce":1044634698}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[]} 2023-08-25T09:53:40.015 INFO:teuthology.orchestra.run.smithi070.stderr:dumped fsmap epoch 93 2023-08-25T09:53:40.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71f406a080 msgr2=0x7f71f406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:40.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71f406a080 0x7f71f406c540 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f720c033000 tx=0x7f720c02f000 comp rx=0 tx=0).stop 2023-08-25T09:53:40.017 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 msgr2=0x7f72101f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 0x7f72101f76f0 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f7200010890 tx=0x7f7200010d50 comp rx=0 tx=0).stop 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 shutdown_connections 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:6832/2630723694,v1:172.21.15.70:6833/2630723694] conn(0x7f71f406a080 0x7f71f406c540 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.155:3300/0,v1:172.21.15.155:6789/0] conn(0x7f72101503b0 0x7f7210150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] conn(0x7f721014ab60 0x7f72101f76f0 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 --2- 172.21.15.70:0/475758692 >> [v2:172.21.15.155:3301/0,v1:172.21.15.155:6790/0] conn(0x7f71fc0039a0 0x7f721014e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T09:53:40.018 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.015+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 >> 172.21.15.70:0/475758692 conn(0x7f7210069850 msgr2=0x7f72100b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T09:53:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.016+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 shutdown_connections 2023-08-25T09:53:40.019 INFO:teuthology.orchestra.run.smithi070.stderr:2023-08-25T09:53:40.016+0000 7f721875d640 1 -- 172.21.15.70:0/475758692 wait complete. 2023-08-25T09:53:40.034 DEBUG:tasks.cephfs.filesystem:already dead... 2023-08-25T09:53:40.035 INFO:teuthology.misc:Shutting down mds daemons... 2023-08-25T09:53:40.035 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T09:53:40.035 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:40.063 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T09:53:40.064 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T09:53:40.064 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:40.130 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T09:53:40.131 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T09:53:40.131 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:40.163 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T09:53:40.164 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T09:53:40.164 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:40.236 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T09:53:40.236 INFO:teuthology.misc:Shutting down osd daemons... 2023-08-25T09:53:40.237 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-08-25T09:53:40.237 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:40.325 INFO:tasks.ceph.osd.0:Stopped 2023-08-25T09:53:40.326 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-08-25T09:53:40.326 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:42.606 INFO:tasks.ceph.mgr.y.smithi070.stderr:2023-08-25T09:53:42.605+0000 7eff29f6a640 -1 calc_pg_upmaps abort due to max <= 0 2023-08-25T09:53:46.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-08-25T09:53:46.428 INFO:tasks.ceph.osd.1:Stopped 2023-08-25T09:53:46.429 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-08-25T09:53:46.429 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:46.527 INFO:tasks.ceph.osd.2:Stopped 2023-08-25T09:53:46.527 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-08-25T09:53:46.527 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:48.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:53:48.519 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:53:52.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-08-25T09:53:52.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-08-25T09:53:52.630 INFO:tasks.ceph.osd.3:Stopped 2023-08-25T09:53:52.630 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-08-25T09:53:52.631 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:53:57.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-08-25T09:53:57.902 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-08-25T09:53:58.733 INFO:tasks.ceph.osd.4:Stopped 2023-08-25T09:53:58.733 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-08-25T09:53:58.734 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:03.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-08-25T09:54:03.608 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-08-25T09:54:04.836 INFO:tasks.ceph.osd.5:Stopped 2023-08-25T09:54:04.837 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-08-25T09:54:04.837 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:04.906 INFO:tasks.ceph.osd.6:Stopped 2023-08-25T09:54:04.906 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-08-25T09:54:04.907 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:05.004 INFO:tasks.ceph.osd.7:Stopped 2023-08-25T09:54:05.005 INFO:teuthology.misc:Shutting down mgr daemons... 2023-08-25T09:54:05.005 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-08-25T09:54:05.006 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:05.082 INFO:tasks.ceph.mgr.y:Stopped 2023-08-25T09:54:05.083 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-08-25T09:54:05.083 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:05.154 INFO:tasks.ceph.mgr.x:Stopped 2023-08-25T09:54:05.155 INFO:teuthology.misc:Shutting down mon daemons... 2023-08-25T09:54:05.155 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-08-25T09:54:05.155 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:05.207 INFO:tasks.ceph.mon.a:Stopped 2023-08-25T09:54:05.207 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-08-25T09:54:05.208 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:05.240 INFO:tasks.ceph.mon.b:Stopped 2023-08-25T09:54:05.240 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-08-25T09:54:05.241 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T09:54:05.325 INFO:tasks.ceph.mon.c:Stopped 2023-08-25T09:54:05.326 INFO:tasks.ceph:Checking cluster log for badness... 2023-08-25T09:54:05.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-08-25T09:54:05.442 INFO:teuthology.orchestra.run.smithi070.stdout:2023-08-25T09:38:37.950823+0000 osd.3 (osd.3) 3 : cluster [WRN] OSD bench result of 99205.725890 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.3. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-08-25T09:54:05.443 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-08-25T09:54:05.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-08-25T09:54:05.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-08-25T09:54:05.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-08-25T09:54:05.723 INFO:teuthology.orchestra.run.smithi070.stdout:2023-08-25T09:38:37.950823+0000 osd.3 (osd.3) 3 : cluster [WRN] OSD bench result of 99205.725890 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.3. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-08-25T09:54:05.725 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:54:05.725 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-08-25T09:54:06.022 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:54:06.023 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-08-25T09:54:06.261 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:54:06.261 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-08-25T09:54:06.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:54:06.514 DEBUG:teuthology.orchestra.run.smithi070:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-08-25T09:54:06.749 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:54:06.749 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-08-25T09:54:07.131 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:54:07.132 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-08-25T09:54:07.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:54:07.381 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-08-25T09:54:07.669 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:54:07.669 DEBUG:teuthology.orchestra.run.smithi155:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-08-25T09:54:07.927 INFO:tasks.ceph:Archiving mon data... 2023-08-25T09:54:07.928 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/data/mon.a.tgz 2023-08-25T09:54:07.929 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2023-08-25T09:54:07.988 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.lY00i3v8FD 2023-08-25T09:54:07.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.lY00i3v8FD 2023-08-25T09:54:08.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.lY00i3v8FD 2023-08-25T09:54:08.377 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.lY00i3v8FD is 545KB 2023-08-25T09:54:08.490 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.lY00i3v8FD 2023-08-25T09:54:08.549 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/data/mon.b.tgz 2023-08-25T09:54:08.550 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2023-08-25T09:54:08.609 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.Y4JqlQbHio 2023-08-25T09:54:08.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Y4JqlQbHio 2023-08-25T09:54:08.610 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-08-25T09:54:08.856 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.Y4JqlQbHio 2023-08-25T09:54:09.016 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.Y4JqlQbHio is 576KB 2023-08-25T09:54:09.128 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.Y4JqlQbHio 2023-08-25T09:54:09.187 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/data/mon.c.tgz 2023-08-25T09:54:09.188 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2023-08-25T09:54:09.240 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.FLBcuUxrYW 2023-08-25T09:54:09.241 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.FLBcuUxrYW 2023-08-25T09:54:09.469 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.FLBcuUxrYW 2023-08-25T09:54:09.615 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.FLBcuUxrYW is 576KB 2023-08-25T09:54:09.721 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.FLBcuUxrYW 2023-08-25T09:54:09.779 INFO:tasks.ceph:Cleaning ceph cluster... 2023-08-25T09:54:09.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-25T09:54:09.790 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-25T09:54:09.889 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-08-25T09:54:09.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-25T09:54:09.896 DEBUG:teuthology.orchestra.run.smithi155:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-25T09:54:09.995 INFO:teuthology.orchestra.run.smithi070.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-25T09:54:09.997 INFO:teuthology.orchestra.run.smithi155.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-25T09:54:10.009 INFO:tasks.ceph:Archiving crash dumps... 2023-08-25T09:54:10.011 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/remote/smithi070/crash 2023-08-25T09:54:10.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-25T09:54:10.095 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/remote/smithi155/crash 2023-08-25T09:54:10.096 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-25T09:54:10.198 INFO:tasks.ceph:Shutting down logrotate 2023-08-25T09:54:10.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:54:10.208 DEBUG:teuthology.orchestra.run.smithi155:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T09:54:10.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-25T09:54:10.442 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-25T09:54:10.546 INFO:tasks.ceph:Compressing logs... 2023-08-25T09:54:10.547 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T09:54:10.556 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T09:54:10.646 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129135.log 2023-08-25T09:54:10.647 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96962.log 2023-08-25T09:54:10.648 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101434.log 2023-08-25T09:54:10.649 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129135.log.gz 2023-08-25T09:54:10.650 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103645.log 2023-08-25T09:54:10.651 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96962.log.gz 2023-08-25T09:54:10.651 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89968.log 2023-08-25T09:54:10.652 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101434.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.125962.log -- replaced with /var/log/ceph/ceph-client.admin.101434.log.gz 2023-08-25T09:54:10.653 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:10.653 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103645.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103645.log.gz 2023-08-25T09:54:10.653 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94250.log 2023-08-25T09:54:10.656 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.87948.log 2023-08-25T09:54:10.656 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.89968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.89302.log 2023-08-25T09:54:10.656 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.87948.log: 91.9% -- replaced with /var/log/ceph/ceph-client.1.89968.log.gz 2023-08-25T09:54:10.656 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-08-25T09:54:10.656 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.89302.log: 89.7% -- replaced with /var/log/ceph/ceph-client.1.87948.log.gz 2023-08-25T09:54:10.657 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85447.log 2023-08-25T09:54:10.657 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125962.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125962.log.gz 2023-08-25T09:54:10.657 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96238.log 2023-08-25T09:54:10.657 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115846.log 2023-08-25T09:54:10.657 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94250.log.gz 2023-08-25T09:54:10.657 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114936.log 2023-08-25T09:54:10.658 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96238.log.gz 2023-08-25T09:54:10.660 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.7.log: 93.4% -- replaced with /var/log/ceph/ceph-client.1.89302.log.gz 2023-08-25T09:54:10.660 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.88654.log 2023-08-25T09:54:10.660 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.86634.log 2023-08-25T09:54:10.660 INFO:teuthology.orchestra.run.smithi155.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.85447.log.gz 2023-08-25T09:54:10.660 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.88654.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-08-25T09:54:10.661 INFO:teuthology.orchestra.run.smithi155.stderr: 92.3%/var/log/ceph/ceph-client.1.86634.log: -- replaced with /var/log/ceph/ceph-client.1.88654.log.gz 2023-08-25T09:54:10.662 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112382.log 2023-08-25T09:54:10.662 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115846.log.gz 2023-08-25T09:54:10.662 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114936.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114936.log.gz 2023-08-25T09:54:10.662 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121896.log 2023-08-25T09:54:10.662 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96000.log 2023-08-25T09:54:10.662 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112382.log.gz 2023-08-25T09:54:10.663 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93560.log 2023-08-25T09:54:10.663 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121896.log.gz 2023-08-25T09:54:10.663 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96000.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96000.log.gz 2023-08-25T09:54:10.663 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87131.log 2023-08-25T09:54:10.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86711.log 2023-08-25T09:54:10.665 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93560.log.gz 2023-08-25T09:54:10.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87131.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87131.log.gz --verbose 2023-08-25T09:54:10.665 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.91927.log 2023-08-25T09:54:10.666 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93415.log 2023-08-25T09:54:10.666 INFO:teuthology.orchestra.run.smithi155.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.1.86634.log.gz 2023-08-25T09:54:10.666 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-08-25T09:54:10.667 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.93415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85658.log 2023-08-25T09:54:10.667 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.6.log: 93.3% -- replaced with /var/log/ceph/ceph-client.1.93415.log.gz 2023-08-25T09:54:10.667 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-08-25T09:54:10.667 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85658.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85658.log.gz 2023-08-25T09:54:10.667 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92718.log 2023-08-25T09:54:10.668 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.87155.log 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86711.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86711.log.gz 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86581.log 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91927.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.91927.log.gz 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85358.log 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86581.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86581.log.gz 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-08-25T09:54:10.670 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.85358.log: gzip 89.9% -- replaced with /var/log/ceph/ceph-client.admin.85358.log.gz -5 2023-08-25T09:54:10.671 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.116569.log 2023-08-25T09:54:10.672 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131059.log 2023-08-25T09:54:10.672 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116569.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116569.log.gz 2023-08-25T09:54:10.673 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100389.log 2023-08-25T09:54:10.674 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.92718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85738.log 2023-08-25T09:54:10.674 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.3.87155.log: 94.9% -- replaced with /var/log/ceph/ceph-client.1.92718.log.gz 2023-08-25T09:54:10.674 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91974.log 2023-08-25T09:54:10.674 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85738.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.85738.log.gz 2023-08-25T09:54:10.674 INFO:teuthology.orchestra.run.smithi155.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.3.87155.log.gz 2023-08-25T09:54:10.674 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-25T09:54:10.676 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.91974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94138.log 2023-08-25T09:54:10.677 INFO:teuthology.orchestra.run.smithi155.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.1.91974.log.gz 2023-08-25T09:54:10.677 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85527.log 2023-08-25T09:54:10.678 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131059.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131059.log.gz 2023-08-25T09:54:10.678 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123739.log 2023-08-25T09:54:10.678 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100389.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100389.log.gz 2023-08-25T09:54:10.678 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93770.log 2023-08-25T09:54:10.679 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123739.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123739.log.gz 2023-08-25T09:54:10.679 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104059.log 2023-08-25T09:54:10.679 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93770.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93770.log.gz 2023-08-25T09:54:10.679 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122022.log 2023-08-25T09:54:10.682 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101797.log 2023-08-25T09:54:10.683 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104059.log.gz 2023-08-25T09:54:10.683 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122022.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122022.log.gz 2023-08-25T09:54:10.683 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105670.log 2023-08-25T09:54:10.683 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101797.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101797.log.gz 2023-08-25T09:54:10.684 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91064.log 2023-08-25T09:54:10.684 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105670.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105670.log.gz 2023-08-25T09:54:10.684 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116067.log 2023-08-25T09:54:10.685 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.94138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85698.log 2023-08-25T09:54:10.685 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85527.log: 92.3% -- replaced with /var/log/ceph/ceph-client.1.94138.log.gz 2023-08-25T09:54:10.685 INFO:teuthology.orchestra.run.smithi155.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85527.log.gz 2023-08-25T09:54:10.686 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85618.log 2023-08-25T09:54:10.686 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85698.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-08-25T09:54:10.686 INFO:teuthology.orchestra.run.smithi155.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85698.log.gz 2023-08-25T09:54:10.686 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85618.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-08-25T09:54:10.686 INFO:teuthology.orchestra.run.smithi155.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85618.log.gz 2023-08-25T09:54:10.687 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85487.log 2023-08-25T09:54:10.687 INFO:teuthology.orchestra.run.smithi155.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-25T09:54:10.688 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-08-25T09:54:10.688 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85487.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-08-25T09:54:10.688 INFO:teuthology.orchestra.run.smithi155.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.85487.log.gz 2023-08-25T09:54:10.690 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91064.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91064.log.gz 2023-08-25T09:54:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120643.log 2023-08-25T09:54:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116067.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116067.log.gz 2023-08-25T09:54:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129213.log 2023-08-25T09:54:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120643.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120643.log.gzgzip 2023-08-25T09:54:10.691 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.110908.log 2023-08-25T09:54:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129213.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129213.log.gz 2023-08-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99068.log 2023-08-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123100.log 2023-08-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110908.log.gz 2023-08-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128393.log 2023-08-25T09:54:10.692 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.99068.log.gz 2023-08-25T09:54:10.693 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85569.log 2023-08-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123100.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123100.log.gz 2023-08-25T09:54:10.694 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129339.log 2023-08-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90938.log 2023-08-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128393.log.gz 2023-08-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129339.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129339.log.gz 2023-08-25T09:54:10.695 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106184.log 2023-08-25T09:54:10.696 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90938.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90938.log.gz 2023-08-25T09:54:10.697 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87302.log 2023-08-25T09:54:10.698 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102548.log 2023-08-25T09:54:10.698 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106184.log.gz 2023-08-25T09:54:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87302.log: 87.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.87302.log.gz 2023-08-25T09:54:10.699 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97373.log 2023-08-25T09:54:10.700 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113385.log 2023-08-25T09:54:10.700 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102548.log.gz 2023-08-25T09:54:10.701 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97373.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97373.log.gz 2023-08-25T09:54:10.702 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120559.log 2023-08-25T09:54:10.703 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123357.log 2023-08-25T09:54:10.703 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113385.log.gz 2023-08-25T09:54:10.704 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-08-25T09:54:10.704 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120559.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120559.log.gz 2023-08-25T09:54:10.704 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115263.log 2023-08-25T09:54:10.705 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115305.log 2023-08-25T09:54:10.705 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123357.log.gz 2023-08-25T09:54:10.706 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.admin.85569.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85569.log.gz 2023-08-25T09:54:10.706 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115263.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115263.log.gz 2023-08-25T09:54:10.706 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129423.log 2023-08-25T09:54:10.707 INFO:teuthology.orchestra.run.smithi155.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-08-25T09:54:10.707 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122577.log 2023-08-25T09:54:10.708 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115305.log.gz 2023-08-25T09:54:10.708 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129423.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129423.log.gz 2023-08-25T09:54:10.709 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98534.log 2023-08-25T09:54:10.709 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-25T09:54:10.711 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122577.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122577.log.gz 2023-08-25T09:54:10.711 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104783.log 2023-08-25T09:54:10.711 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98534.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98534.log.gz 2023-08-25T09:54:10.712 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124183.log 2023-08-25T09:54:10.712 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109349.log 2023-08-25T09:54:10.712 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104783.log.gz 2023-08-25T09:54:10.714 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124183.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124183.log.gz 2023-08-25T09:54:10.714 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127577.log 2023-08-25T09:54:10.715 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124711.log 2023-08-25T09:54:10.715 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109349.log.gz 2023-08-25T09:54:10.716 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101004.log 2023-08-25T09:54:10.716 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.127577.log.gz 2023-08-25T09:54:10.717 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124711.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124711.log.gz 2023-08-25T09:54:10.717 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109085.log 2023-08-25T09:54:10.719 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101004.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101004.log.gz 2023-08-25T09:54:10.720 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97247.log 2023-08-25T09:54:10.721 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109085.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109085.log.gzgzip 2023-08-25T09:54:10.721 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96042.log 2023-08-25T09:54:10.722 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125566.log 2023-08-25T09:54:10.722 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97247.log.gz 2023-08-25T09:54:10.723 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96042.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96042.log.gz 2023-08-25T09:54:10.723 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91190.log 2023-08-25T09:54:10.724 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102200.log 2023-08-25T09:54:10.724 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125566.log.gz 2023-08-25T09:54:10.725 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.90636.log 2023-08-25T09:54:10.726 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91190.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91190.log.gz 2023-08-25T09:54:10.726 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123655.log 2023-08-25T09:54:10.727 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102200.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102200.log.gz -5 2023-08-25T09:54:10.727 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.0.95809.log 2023-08-25T09:54:10.728 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123655.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123655.log.gz 2023-08-25T09:54:10.728 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117566.log 2023-08-25T09:54:10.729 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.95809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105540.log 2023-08-25T09:54:10.730 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117566.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117566.log.gz 2023-08-25T09:54:10.731 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91743.log 2023-08-25T09:54:10.731 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph.log: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-25T09:54:10.732 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105540.log: 91.8% -- replaced with /var/log/ceph/ceph-client.0.95809.log.gz 2023-08-25T09:54:10.732 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122106.log 2023-08-25T09:54:10.733 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.91335.log 2023-08-25T09:54:10.733 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105540.log.gz 2023-08-25T09:54:10.734 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91743.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91743.log.gz 2023-08-25T09:54:10.734 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130063.log 2023-08-25T09:54:10.735 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122106.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122106.log.gz 2023-08-25T09:54:10.735 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87985.log 2023-08-25T09:54:10.736 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130063.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.130063.log.gz --verbose 2023-08-25T09:54:10.736 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.114522.log 2023-08-25T09:54:10.737 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102284.log 2023-08-25T09:54:10.738 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87985.log.gz 2023-08-25T09:54:10.738 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114522.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114522.log.gz 2023-08-25T09:54:10.739 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121171.log 2023-08-25T09:54:10.740 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/ceph-client.1.90636.log: 91.8% -- replaced with /var/log/ceph/ceph-client.1.90636.log.gz/var/log/ceph/ceph-client.1.91335.log: 2023-08-25T09:54:10.740 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102284.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102284.log.gz 2023-08-25T09:54:10.741 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117524.log 2023-08-25T09:54:10.741 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.121171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124993.log 2023-08-25T09:54:10.743 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130933.log 2023-08-25T09:54:10.743 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117524.log.gz 2023-08-25T09:54:10.744 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122493.log 2023-08-25T09:54:10.744 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124993.log.gz 2023-08-25T09:54:10.745 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130933.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130933.log.gz 2023-08-25T09:54:10.746 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127157.log 2023-08-25T09:54:10.747 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105762.log 2023-08-25T09:54:10.748 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122493.log.gz 2023-08-25T09:54:10.748 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127157.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127157.log.gz 2023-08-25T09:54:10.749 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97709.log 2023-08-25T09:54:10.750 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94960.log 2023-08-25T09:54:10.751 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105762.log.gz 2023-08-25T09:54:10.752 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97709.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97709.log.gz 2023-08-25T09:54:10.752 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-08-25T09:54:10.753 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129792.log 2023-08-25T09:54:10.754 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94960.log.gz 2023-08-25T09:54:10.754 INFO:teuthology.orchestra.run.smithi155.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.1.91335.log.gz 2023-08-25T09:54:10.755 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88433.log 2023-08-25T09:54:10.756 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129792.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129792.log.gz 2023-08-25T09:54:10.757 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111775.log 2023-08-25T09:54:10.758 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88433.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.88433.log.gz 2023-08-25T09:54:10.759 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97499.log 2023-08-25T09:54:10.760 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98984.log 2023-08-25T09:54:10.760 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111775.log.gz 2023-08-25T09:54:10.762 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97499.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.97499.log.gz --verbose 2023-08-25T09:54:10.762 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.123611.log 2023-08-25T09:54:10.763 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98984.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98984.log.gz 2023-08-25T09:54:10.764 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90518.log 2023-08-25T09:54:10.765 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118521.log 2023-08-25T09:54:10.765 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123611.log.gz 2023-08-25T09:54:10.766 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90518.log: 93.9% -- replaced with /var/log/ceph/ceph-client.0.121171.log.gz 2023-08-25T09:54:10.767 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122064.log 2023-08-25T09:54:10.767 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90518.log.gz 2023-08-25T09:54:10.768 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107271.log 2023-08-25T09:54:10.768 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118521.log.gz 2023-08-25T09:54:10.769 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122064.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.122064.log.gz 2023-08-25T09:54:10.770 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94748.log 2023-08-25T09:54:10.771 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113909.log 2023-08-25T09:54:10.771 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107271.log.gz 2023-08-25T09:54:10.772 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94748.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94748.log.gz 2023-08-25T09:54:10.772 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87089.log 2023-08-25T09:54:10.773 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105456.log 2023-08-25T09:54:10.774 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113909.log.gz 2023-08-25T09:54:10.775 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87089.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.87089.log.gz 2023-08-25T09:54:10.775 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106016.log 2023-08-25T09:54:10.776 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105456.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.105456.log.gz 2023-08-25T09:54:10.776 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114978.log 2023-08-25T09:54:10.777 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106016.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106016.log.gz 2023-08-25T09:54:10.777 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92215.log 2023-08-25T09:54:10.778 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119238.log 2023-08-25T09:54:10.779 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114978.log.gz 2023-08-25T09:54:10.779 INFO:teuthology.orchestra.run.smithi155.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-08-25T09:54:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92215.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92215.log.gz 2023-08-25T09:54:10.780 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124033.log 2023-08-25T09:54:10.781 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104951.log 89.0% 2023-08-25T09:54:10.781 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119238.log.gz 2023-08-25T09:54:10.782 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124033.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124033.log.gz 2023-08-25T09:54:10.783 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94622.log 2023-08-25T09:54:10.784 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104951.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.104176.log 2023-08-25T09:54:10.784 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104951.log.gz 2023-08-25T09:54:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94622.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94622.log.gz 2023-08-25T09:54:10.785 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130849.log 2023-08-25T09:54:10.787 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88475.log 2023-08-25T09:54:10.787 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104176.log.gz 2023-08-25T09:54:10.788 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130849.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130849.log.gz 2023-08-25T09:54:10.788 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131246.log 2023-08-25T09:54:10.789 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119070.log 2023-08-25T09:54:10.789 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88475.log.gz 2023-08-25T09:54:10.790 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131246.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131246.log.gz 2023-08-25T09:54:10.790 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119616.log 2023-08-25T09:54:10.792 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121818.log 2023-08-25T09:54:10.792 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119070.log.gz 2023-08-25T09:54:10.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119616.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119616.log.gz 2023-08-25T09:54:10.793 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129960.log 2023-08-25T09:54:10.794 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121818.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-08-25T09:54:10.794 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121818.log.gz 2023-08-25T09:54:10.796 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129960.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.129960.log.gz 2023-08-25T09:54:10.796 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90308.log 2023-08-25T09:54:10.797 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102158.log 2023-08-25T09:54:10.798 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90308.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90308.log.gz 2023-08-25T09:54:10.799 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111523.log 2023-08-25T09:54:10.799 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-08-25T09:54:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102158.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102158.log.gz 2023-08-25T09:54:10.800 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122223.log 2023-08-25T09:54:10.802 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126723.log 2023-08-25T09:54:10.802 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122223.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111523.log.gz 2023-08-25T09:54:10.803 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122223.log.gz 2023-08-25T09:54:10.803 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93938.log 2023-08-25T09:54:10.804 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126723.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.126723.log.gz 2023-08-25T09:54:10.805 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104101.log 2023-08-25T09:54:10.806 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118599.log 2023-08-25T09:54:10.807 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93938.log.gz 2023-08-25T09:54:10.808 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104101.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104101.log.gz 2023-08-25T09:54:10.808 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88895.log 2023-08-25T09:54:10.809 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118599.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118599.log.gz 2023-08-25T09:54:10.810 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119280.log 2023-08-25T09:54:10.811 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123525.log 2023-08-25T09:54:10.812 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119280.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.88895.log.gz 2023-08-25T09:54:10.813 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119280.log.gz 2023-08-25T09:54:10.813 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125878.log 2023-08-25T09:54:10.814 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123525.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123525.log.gz 2023-08-25T09:54:10.814 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123991.log 2023-08-25T09:54:10.815 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126807.log 2023-08-25T09:54:10.816 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123991.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125878.log.gz 2023-08-25T09:54:10.817 INFO:teuthology.orchestra.run.smithi070.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.123991.log.gz 2023-08-25T09:54:10.817 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129540.log 2023-08-25T09:54:10.818 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126807.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126807.log.gz 2023-08-25T09:54:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112340.log 2023-08-25T09:54:10.820 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92341.log 2023-08-25T09:54:10.820 INFO:teuthology.orchestra.run.smithi070.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.129540.log.gz 2023-08-25T09:54:10.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112340.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112340.log.gz 2023-08-25T09:54:10.822 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122848.log 2023-08-25T09:54:10.823 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92341.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92341.log.gz 2023-08-25T09:54:10.823 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107229.log 2023-08-25T09:54:10.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118986.log 2023-08-25T09:54:10.825 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122848.log.gz 2023-08-25T09:54:10.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107229.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107229.log.gz 2023-08-25T09:54:10.826 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124651.log 2023-08-25T09:54:10.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118986.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118986.log.gz 2023-08-25T09:54:10.828 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97919.log 2023-08-25T09:54:10.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119112.log 2023-08-25T09:54:10.830 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124651.log.gz 2023-08-25T09:54:10.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97919.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97919.log.gz 2023-08-25T09:54:10.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117838.log 2023-08-25T09:54:10.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119112.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119112.log.gz 2023-08-25T09:54:10.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120389.log 2023-08-25T09:54:10.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110153.log 2023-08-25T09:54:10.834 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117838.log.gz 2023-08-25T09:54:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120389.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120389.log.gz 2023-08-25T09:54:10.835 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122721.log 2023-08-25T09:54:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110153.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110153.log.gz 2023-08-25T09:54:10.837 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105414.log 2023-08-25T09:54:10.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103891.log 2023-08-25T09:54:10.838 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122721.log.gz 2023-08-25T09:54:10.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105414.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105414.log.gz 2023-08-25T09:54:10.840 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104909.log 2023-08-25T09:54:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103891.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103891.log.gz 2023-08-25T09:54:10.841 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90198.log 2023-08-25T09:54:10.842 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116990.log 2023-08-25T09:54:10.843 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104909.log.gz 2023-08-25T09:54:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90198.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90198.log.gz 2023-08-25T09:54:10.844 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126172.log 2023-08-25T09:54:10.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96920.log 2023-08-25T09:54:10.846 INFO:teuthology.orchestra.run.smithi070.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.116990.log.gz 2023-08-25T09:54:10.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122367.log 2023-08-25T09:54:10.847 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126172.log.gz 2023-08-25T09:54:10.848 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118073.log 2023-08-25T09:54:10.849 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96920.log.gz 2023-08-25T09:54:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122367.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122367.log.gz 2023-08-25T09:54:10.850 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100803.log 2023-08-25T09:54:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118073.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118073.log.gz 2023-08-25T09:54:10.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105932.log 2023-08-25T09:54:10.853 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90350.log 2023-08-25T09:54:10.853 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100803.log.gz 2023-08-25T09:54:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105932.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105932.log.gz 2023-08-25T09:54:10.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119154.log 2023-08-25T09:54:10.857 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98198.log 2023-08-25T09:54:10.857 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90350.log.gz 2023-08-25T09:54:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119154.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119154.log.gz 2023-08-25T09:54:10.858 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93686.log 2023-08-25T09:54:10.859 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129009.log 2023-08-25T09:54:10.860 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98198.log.gz 2023-08-25T09:54:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93686.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93686.log.gz 2023-08-25T09:54:10.861 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111118.log 2023-08-25T09:54:10.862 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129009.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129009.log.gz 2023-08-25T09:54:10.863 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130105.log 2023-08-25T09:54:10.864 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99899.log 2023-08-25T09:54:10.864 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111118.log.gz 2023-08-25T09:54:10.865 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130105.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130105.log.gz 2023-08-25T09:54:10.866 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122806.log 2023-08-25T09:54:10.867 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88265.log 2023-08-25T09:54:10.868 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99899.log.gz 2023-08-25T09:54:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122806.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122806.log.gz 2023-08-25T09:54:10.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116439.log 2023-08-25T09:54:10.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86125.log 2023-08-25T09:54:10.870 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88265.log.gz 2023-08-25T09:54:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116439.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116439.log.gz 2023-08-25T09:54:10.871 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93980.log 2023-08-25T09:54:10.873 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86125.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.86125.log.gz /var/log/ceph/ceph-client.admin.87982.log 2023-08-25T09:54:10.873 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:10.875 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87639.log 2023-08-25T09:54:10.875 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93980.log.gz 2023-08-25T09:54:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87982.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87982.log.gz 2023-08-25T09:54:10.876 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94832.log 2023-08-25T09:54:10.878 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111817.log 2023-08-25T09:54:10.878 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87639.log.gz 2023-08-25T09:54:10.879 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90434.log 88.9% 2023-08-25T09:54:10.879 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94832.log.gz 2023-08-25T09:54:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115137.log 2023-08-25T09:54:10.881 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111817.log.gz 2023-08-25T09:54:10.882 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90434.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90434.log.gz 2023-08-25T09:54:10.883 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118944.log 2023-08-25T09:54:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91659.log 2023-08-25T09:54:10.884 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115137.log.gz 2023-08-25T09:54:10.885 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118944.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118944.log.gz 2023-08-25T09:54:10.886 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127029.log 2023-08-25T09:54:10.887 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91659.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91659.log.gz 2023-08-25T09:54:10.887 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120877.log 2023-08-25T09:54:10.888 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95130.log 2023-08-25T09:54:10.889 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127029.log.gz 2023-08-25T09:54:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120877.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120877.log.gz 2023-08-25T09:54:10.890 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90686.log 2023-08-25T09:54:10.892 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95130.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95130.log.gz 2023-08-25T09:54:10.892 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111901.log 2023-08-25T09:54:10.893 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90686.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.90686.log.gz 2023-08-25T09:54:10.893 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124753.log 2023-08-25T09:54:10.895 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90980.log 2023-08-25T09:54:10.895 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111901.log.gz 2023-08-25T09:54:10.897 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115431.log 2023-08-25T09:54:10.897 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124753.log.gz 2023-08-25T09:54:10.898 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90980.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90980.log.gz 2023-08-25T09:54:10.898 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109517.log 2023-08-25T09:54:10.899 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96180.log 2023-08-25T09:54:10.900 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115431.log.gz 2023-08-25T09:54:10.901 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109517.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109517.log.gz 2023-08-25T09:54:10.901 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90602.log 2023-08-25T09:54:10.903 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96180.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.96180.log.gz --verbose 2023-08-25T09:54:10.903 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.124117.log 2023-08-25T09:54:10.904 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90602.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90602.log.gz 2023-08-25T09:54:10.905 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125365.log 2023-08-25T09:54:10.906 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91148.log 2023-08-25T09:54:10.906 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125365.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.124117.log.gz 2023-08-25T09:54:10.907 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125365.log.gz 2023-08-25T09:54:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121692.log 2023-08-25T09:54:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91148.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91148.log.gz 2023-08-25T09:54:10.909 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123277.log 2023-08-25T09:54:10.910 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105210.log 2023-08-25T09:54:10.911 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121692.log.gz 2023-08-25T09:54:10.912 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123277.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123277.log.gz 2023-08-25T09:54:10.912 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128606.log 2023-08-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105210.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105210.log.gz 2023-08-25T09:54:10.914 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96632.log 2023-08-25T09:54:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128606.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128606.log.gz 2023-08-25T09:54:10.915 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98900.log 2023-08-25T09:54:10.917 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110776.log 2023-08-25T09:54:10.917 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.96632.log.gz 2023-08-25T09:54:10.918 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98900.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98900.log.gz 2023-08-25T09:54:10.918 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131101.log 2023-08-25T09:54:10.920 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105290.log 2023-08-25T09:54:10.921 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110776.log.gz 2023-08-25T09:54:10.922 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131328.log 2023-08-25T09:54:10.922 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131101.log.gz 2023-08-25T09:54:10.924 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105290.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105290.log.gz 2023-08-25T09:54:10.924 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115804.log 2023-08-25T09:54:10.926 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131328.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.83831.log 2023-08-25T09:54:10.926 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131328.log.gz 2023-08-25T09:54:10.927 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104218.log 2023-08-25T09:54:10.928 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115804.log.gz 2023-08-25T09:54:10.928 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.tmp-client.admin.83831.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.83831.log.gz 2023-08-25T09:54:10.930 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-08-25T09:54:10.930 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104218.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104218.log.gz 2023-08-25T09:54:10.931 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.117333.log 2023-08-25T09:54:10.933 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117736.log 2023-08-25T09:54:10.935 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.117333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111196.log 2023-08-25T09:54:10.937 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117736.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117736.log.gz 2023-08-25T09:54:10.937 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.119784.log 2023-08-25T09:54:10.938 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111196.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111196.log.gz 2023-08-25T09:54:10.939 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92257.log 2023-08-25T09:54:10.940 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119784.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119784.log.gz 2023-08-25T09:54:10.940 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122265.log 2023-08-25T09:54:10.942 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92257.log: gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92257.log.gz 2023-08-25T09:54:10.942 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.90392.log 2023-08-25T09:54:10.943 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122265.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122265.log.gz 2023-08-25T09:54:10.944 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127367.log 2023-08-25T09:54:10.945 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90392.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130723.log 2023-08-25T09:54:10.945 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90392.log.gz 2023-08-25T09:54:10.947 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127367.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127367.log.gz 2023-08-25T09:54:10.947 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110344.log 2023-08-25T09:54:10.948 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130723.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130723.log.gz 2023-08-25T09:54:10.949 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99857.log 2023-08-25T09:54:10.950 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110344.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110344.log.gz 2023-08-25T09:54:10.950 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105974.log 2023-08-25T09:54:10.952 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99857.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99857.log.gz 2023-08-25T09:54:10.952 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.120082.log 2023-08-25T09:54:10.953 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105974.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105974.log.gz 2023-08-25T09:54:10.954 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94292.log 2023-08-25T09:54:10.955 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120082.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108573.log 2023-08-25T09:54:10.955 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.120082.log.gz 2023-08-25T09:54:10.957 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94292.log: gzip 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.95088.log 2023-08-25T09:54:10.957 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94292.log.gz 2023-08-25T09:54:10.958 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108573.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108573.log.gz 2023-08-25T09:54:10.959 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115575.log 2023-08-25T09:54:10.960 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95088.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95088.log.gz 2023-08-25T09:54:10.960 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118311.log 2023-08-25T09:54:10.962 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115575.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88601.log 2023-08-25T09:54:10.962 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.115575.log.gz 2023-08-25T09:54:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118311.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113126.log 2023-08-25T09:54:10.964 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118311.log.gz 2023-08-25T09:54:10.965 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88601.log: gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88601.log.gz 2023-08-25T09:54:10.966 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102326.log 2023-08-25T09:54:10.967 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113126.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113126.log.gz 2023-08-25T09:54:10.967 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94454.log 2023-08-25T09:54:10.969 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102326.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102326.log.gz 2023-08-25T09:54:10.969 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.106142.log 2023-08-25T09:54:10.970 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94454.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94454.log.gz 2023-08-25T09:54:10.971 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109559.log 2023-08-25T09:54:10.972 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106142.log: gzip 88.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.113493.log 2023-08-25T09:54:10.972 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106142.log.gz 2023-08-25T09:54:10.974 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109559.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109559.log.gz 2023-08-25T09:54:10.974 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.128468.log 2023-08-25T09:54:10.975 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113493.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113493.log.gz 2023-08-25T09:54:10.975 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86497.log 2023-08-25T09:54:10.977 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128468.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128468.log.gz 2023-08-25T09:54:10.978 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101046.log 2023-08-25T09:54:10.979 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86497.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86497.log.gz 2023-08-25T09:54:10.979 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122409.log 2023-08-25T09:54:10.981 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91274.log 2023-08-25T09:54:10.981 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101046.log.gz 2023-08-25T09:54:10.982 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116229.log 2023-08-25T09:54:10.982 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122409.log.gz 2023-08-25T09:54:10.984 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104698.log 2023-08-25T09:54:10.984 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91274.log.gz 2023-08-25T09:54:10.985 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86837.log 2023-08-25T09:54:10.985 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116229.log.gz 2023-08-25T09:54:10.986 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126130.log 2023-08-25T09:54:10.987 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104698.log.gz 2023-08-25T09:54:10.988 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100845.log 2023-08-25T09:54:10.989 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86837.log.gz/var/log/ceph/ceph-client.admin.126130.log: 2023-08-25T09:54:10.990 INFO:teuthology.orchestra.run.smithi070.stderr:gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126130.log.gz -5 2023-08-25T09:54:10.990 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.131017.log 2023-08-25T09:54:10.991 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114109.log 2023-08-25T09:54:10.992 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100845.log.gz 2023-08-25T09:54:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93182.log 2023-08-25T09:54:10.993 INFO:teuthology.orchestra.run.smithi070.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.131017.log.gz 2023-08-25T09:54:10.994 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117955.log 2023-08-25T09:54:10.994 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114109.log.gz 2023-08-25T09:54:10.996 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112639.log 2023-08-25T09:54:10.996 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93182.log.gz 2023-08-25T09:54:10.997 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121608.log 2023-08-25T09:54:10.997 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117955.log.gz 2023-08-25T09:54:10.999 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115515.log 2023-08-25T09:54:10.999 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112639.log.gz 2023-08-25T09:54:11.005 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121608.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97205.log 2023-08-25T09:54:11.005 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.121608.log.gz 2023-08-25T09:54:11.005 INFO:teuthology.orchestra.run.smithi070.stderr: 94.8%/var/log/ceph/ceph-client.admin.115515.log: -- replaced with /var/log/ceph/ceph-client.0.117333.log.gz 2023-08-25T09:54:11.006 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115515.log.gz 2023-08-25T09:54:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97793.log 2023-08-25T09:54:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115762.log 2023-08-25T09:54:11.006 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97205.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97205.log.gz 2023-08-25T09:54:11.007 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97793.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.97793.log.gz -- 2023-08-25T09:54:11.007 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.108427.log 2023-08-25T09:54:11.009 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115762.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115762.log.gz 2023-08-25T09:54:11.009 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.109601.log 2023-08-25T09:54:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108427.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108427.log.gz 2023-08-25T09:54:11.010 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106310.log 2023-08-25T09:54:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86027.log 2023-08-25T09:54:11.012 INFO:teuthology.orchestra.run.smithi070.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.109601.log.gz 2023-08-25T09:54:11.012 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105720.log 2023-08-25T09:54:11.012 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.106310.log.gz 2023-08-25T09:54:11.013 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128510.log 2023-08-25T09:54:11.013 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86027.log.gz 2023-08-25T09:54:11.014 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110632.log 2023-08-25T09:54:11.015 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105720.log.gz 2023-08-25T09:54:11.016 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128510.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128510.log.gz 2023-08-25T09:54:11.016 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.112893.log 2023-08-25T09:54:11.017 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110632.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110632.log.gz 2023-08-25T09:54:11.018 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92625.log 2023-08-25T09:54:11.019 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116737.log 2023-08-25T09:54:11.020 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112893.log.gz 2023-08-25T09:54:11.021 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92625.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92625.log.gz 2023-08-25T09:54:11.021 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.115972.log 2023-08-25T09:54:11.022 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116737.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116737.log.gz 2023-08-25T09:54:11.023 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125281.log 2023-08-25T09:54:11.024 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94916.log 2023-08-25T09:54:11.025 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115972.log.gz 2023-08-25T09:54:11.026 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125281.log: gzip -5 --verbose -- 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125281.log.gz 2023-08-25T09:54:11.026 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.125524.log 2023-08-25T09:54:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94916.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94916.log.gz 2023-08-25T09:54:11.028 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98118.log 2023-08-25T09:54:11.029 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116527.log 2023-08-25T09:54:11.030 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125524.log.gz 2023-08-25T09:54:11.032 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98118.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98118.log.gz 2023-08-25T09:54:11.032 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.95532.log 2023-08-25T09:54:11.033 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116527.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116527.log.gz 2023-08-25T09:54:11.033 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106844.log 2023-08-25T09:54:11.034 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109307.log 2023-08-25T09:54:11.035 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95532.log.gz 2023-08-25T09:54:11.036 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101382.log 2023-08-25T09:54:11.036 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106844.log.gz 2023-08-25T09:54:11.037 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109307.log: gzip -5 --verbose 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109307.log.gz 2023-08-25T09:54:11.038 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.100088.log 2023-08-25T09:54:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101382.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101382.log.gz 2023-08-25T09:54:11.039 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119490.log 2023-08-25T09:54:11.041 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92425.log 2023-08-25T09:54:11.041 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100088.log.gz 2023-08-25T09:54:11.042 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119490.log: gzip -5 --verbose -- 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119490.log.gz 2023-08-25T09:54:11.042 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.87047.log 2023-08-25T09:54:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92425.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92425.log.gz 2023-08-25T09:54:11.044 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128648.log 2023-08-25T09:54:11.045 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118641.log 2023-08-25T09:54:11.046 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87047.log.gz 2023-08-25T09:54:11.047 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128648.log: gzip -5 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128648.log.gz 2023-08-25T09:54:11.048 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111481.log 2023-08-25T09:54:11.048 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118641.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118641.log.gz 2023-08-25T09:54:11.049 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107652.log 2023-08-25T09:54:11.050 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111481.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.123567.log 2023-08-25T09:54:11.051 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.111481.log.gz 2023-08-25T09:54:11.052 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107652.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107652.log.gz 2023-08-25T09:54:11.052 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87306.log 2023-08-25T09:54:11.053 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108511.log 2023-08-25T09:54:11.054 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123567.log.gz 2023-08-25T09:54:11.055 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121980.log 2023-08-25T09:54:11.055 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87306.log.gz 2023-08-25T09:54:11.057 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108511.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108511.log.gz 2023-08-25T09:54:11.057 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.120339.log 2023-08-25T09:54:11.058 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121980.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121980.log.gz 2023-08-25T09:54:11.058 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102760.log 2023-08-25T09:54:11.060 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87973.log 2023-08-25T09:54:11.060 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120339.log.gz 2023-08-25T09:54:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102760.log: gzip -5 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102760.log.gz 2023-08-25T09:54:11.062 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.130807.log 2023-08-25T09:54:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87973.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87973.log.gz 2023-08-25T09:54:11.063 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122890.log 2023-08-25T09:54:11.064 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98578.log 2023-08-25T09:54:11.065 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130807.log.gz 2023-08-25T09:54:11.066 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122890.log: gzip -5 --verbose -- 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122890.log.gz 2023-08-25T09:54:11.066 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.126893.log 2023-08-25T09:54:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98578.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98578.log.gz 2023-08-25T09:54:11.068 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88181.log 2023-08-25T09:54:11.069 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126893.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.126893.log.gz 2023-08-25T09:54:11.069 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96836.log 2023-08-25T09:54:11.071 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86286.log 2023-08-25T09:54:11.071 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88181.log.gz 2023-08-25T09:54:11.073 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96836.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96836.log.gz 2023-08-25T09:54:11.073 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.127451.log 2023-08-25T09:54:11.074 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86286.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86286.log.gz 2023-08-25T09:54:11.074 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98490.log 2023-08-25T09:54:11.076 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111649.log 2023-08-25T09:54:11.076 INFO:teuthology.orchestra.run.smithi070.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.127451.log.gz 2023-08-25T09:54:11.077 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98490.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98490.log.gz 2023-08-25T09:54:11.077 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.107349.log 2023-08-25T09:54:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111649.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111649.log.gz 2023-08-25T09:54:11.079 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111859.log 2023-08-25T09:54:11.081 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107577.log 2023-08-25T09:54:11.081 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107349.log.gz 2023-08-25T09:54:11.083 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111859.log: gzip -5 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111859.log.gz 2023-08-25T09:54:11.083 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.108998.log 2023-08-25T09:54:11.084 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107577.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107577.log.gz 2023-08-25T09:54:11.085 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87646.log 2023-08-25T09:54:11.086 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128309.log 2023-08-25T09:54:11.086 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108998.log.gz 2023-08-25T09:54:11.088 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87646.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87646.log.gz 2023-08-25T09:54:11.088 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.0.124460.log 2023-08-25T09:54:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128309.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128309.log.gz 2023-08-25T09:54:11.089 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104554.log 2023-08-25T09:54:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.124460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122535.log 2023-08-25T09:54:11.092 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100515.log 2023-08-25T09:54:11.093 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104554.log.gz 2023-08-25T09:54:11.094 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94790.log 2023-08-25T09:54:11.094 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122535.log.gz 2023-08-25T09:54:11.095 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92551.log 2023-08-25T09:54:11.096 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100515.log.gz 2023-08-25T09:54:11.097 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101340.log 2023-08-25T09:54:11.097 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94790.log.gz 2023-08-25T09:54:11.098 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92551.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92551.log.gz -5 2023-08-25T09:54:11.098 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98774.log 2023-08-25T09:54:11.100 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101340.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101340.log.gz 2023-08-25T09:54:11.101 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.90854.log 2023-08-25T09:54:11.101 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98774.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98774.log.gz 2023-08-25T09:54:11.102 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103975.log 2023-08-25T09:54:11.103 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90854.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90854.log.gz 2023-08-25T09:54:11.103 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111076.log 2023-08-25T09:54:11.106 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103975.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103975.log.gz 2023-08-25T09:54:11.106 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123399.log 2023-08-25T09:54:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111076.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111076.log.gz 2023-08-25T09:54:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97625.log 2023-08-25T09:54:11.108 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123399.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123399.log.gz 2023-08-25T09:54:11.109 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94124.log 2023-08-25T09:54:11.110 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97625.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97625.log.gz 2023-08-25T09:54:11.111 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122181.log 2023-08-25T09:54:11.112 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94124.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94124.log.gz 2023-08-25T09:54:11.113 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106058.log 2023-08-25T09:54:11.114 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122181.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122181.log.gz 2023-08-25T09:54:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87978.log 2023-08-25T09:54:11.115 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106058.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106058.log.gz 2023-08-25T09:54:11.116 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90560.log 2023-08-25T09:54:11.117 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87978.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87978.log.gz 2023-08-25T09:54:11.118 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97046.log 2023-08-25T09:54:11.119 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90560.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90560.log.gz 2023-08-25T09:54:11.120 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100028.log 2023-08-25T09:54:11.121 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97046.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97046.log.gz 2023-08-25T09:54:11.122 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123483.log 2023-08-25T09:54:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100028.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100028.log.gz 2023-08-25T09:54:11.123 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107187.log 2023-08-25T09:54:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123483.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123483.log.gz 2023-08-25T09:54:11.125 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101569.log 2023-08-25T09:54:11.127 INFO:teuthology.orchestra.run.smithi070.stderr: 93.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101256.log 2023-08-25T09:54:11.127 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.0.124460.log.gz 2023-08-25T09:54:11.128 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107187.log: /var/log/ceph/ceph-client.admin.101569.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107187.log.gz 2023-08-25T09:54:11.128 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108217.log 2023-08-25T09:54:11.129 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101569.log.gz 2023-08-25T09:54:11.130 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101256.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-08-25T09:54:11.130 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126214.log 2023-08-25T09:54:11.131 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101256.log.gz 2023-08-25T09:54:11.131 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106226.log 2023-08-25T09:54:11.132 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108217.log.gz 2023-08-25T09:54:11.132 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126214.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126214.log.gz 2023-08-25T09:54:11.132 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106268.log 2023-08-25T09:54:11.133 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106226.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106226.log.gz 2023-08-25T09:54:11.133 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118785.log 2023-08-25T09:54:11.134 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106268.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106268.log.gz 2023-08-25T09:54:11.135 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120473.log 2023-08-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120295.log 2023-08-25T09:54:11.136 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8%/var/log/ceph/ceph-client.admin.120473.log: -- replaced with /var/log/ceph/ceph-client.admin.118785.log.gz 2023-08-25T09:54:11.137 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120473.log.gz 2023-08-25T09:54:11.137 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129051.log 2023-08-25T09:54:11.139 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120295.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120295.log.gz 2023-08-25T09:54:11.139 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102464.log 2023-08-25T09:54:11.140 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129051.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129051.log.gz 2023-08-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97541.log 2023-08-25T09:54:11.141 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115888.log 2023-08-25T09:54:11.142 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102464.log.gz 2023-08-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97541.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97541.log.gz 2023-08-25T09:54:11.143 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97835.log 2023-08-25T09:54:11.144 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115888.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115888.log.gz 2023-08-25T09:54:11.145 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108914.log 2023-08-25T09:54:11.145 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87311.log 2023-08-25T09:54:11.146 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97835.log.gz 2023-08-25T09:54:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108914.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108914.log.gz 2023-08-25T09:54:11.147 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130891.log 2023-08-25T09:54:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87311.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87311.log.gz 2023-08-25T09:54:11.148 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86205.log 2023-08-25T09:54:11.150 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98730.log 2023-08-25T09:54:11.150 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130891.log.gz 2023-08-25T09:54:11.151 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86205.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86205.log.gz 2023-08-25T09:54:11.151 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92173.log 2023-08-25T09:54:11.152 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98730.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98730.log.gz 2023-08-25T09:54:11.153 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-25T09:54:11.153 INFO:teuthology.orchestra.run.smithi155.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-08-25T09:54:11.154 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90770.log 2023-08-25T09:54:11.154 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92173.log.gz 2023-08-25T09:54:11.155 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87648.log 2023-08-25T09:54:11.157 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121650.log 2023-08-25T09:54:11.157 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90770.log.gz 2023-08-25T09:54:11.158 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103368.log 2023-08-25T09:54:11.158 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.87648.log.gz 2023-08-25T09:54:11.159 INFO:teuthology.orchestra.run.smithi070.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-25T09:54:11.159 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121650.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.121650.log.gz 2023-08-25T09:54:11.160 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.122661.log 2023-08-25T09:54:11.161 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95340.log 2023-08-25T09:54:11.161 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103368.log.gz 2023-08-25T09:54:11.162 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122661.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122661.log.gz 2023-08-25T09:54:11.162 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130471.log 2023-08-25T09:54:11.163 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95340.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95340.log.gz 2023-08-25T09:54:11.164 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92835.log 2023-08-25T09:54:11.164 INFO:teuthology.orchestra.run.smithi155.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-08-25T09:54:11.165 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88997.log 2023-08-25T09:54:11.165 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130471.log.gz 2023-08-25T09:54:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92835.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92835.log.gz 2023-08-25T09:54:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111280.log 2023-08-25T09:54:11.167 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88997.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88997.log.gz 2023-08-25T09:54:11.168 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93812.log 2023-08-25T09:54:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111280.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111280.log.gz 2023-08-25T09:54:11.169 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86963.log 2023-08-25T09:54:11.170 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126340.log 2023-08-25T09:54:11.171 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93812.log.gz 2023-08-25T09:54:11.172 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94664.log 2023-08-25T09:54:11.172 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86963.log.gz 2023-08-25T09:54:11.173 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119742.log 2023-08-25T09:54:11.173 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126340.log.gz 2023-08-25T09:54:11.175 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94664.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94664.log.gz 2023-08-25T09:54:11.175 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127535.log 2023-08-25T09:54:11.176 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119742.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119742.log.gz 2023-08-25T09:54:11.177 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100920.log 2023-08-25T09:54:11.178 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97667.log 2023-08-25T09:54:11.178 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127535.log.gz 2023-08-25T09:54:11.179 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100920.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100920.log.gz 2023-08-25T09:54:11.180 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88643.log 2023-08-25T09:54:11.181 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97667.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97667.log.gz 2023-08-25T09:54:11.181 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120251.log 2023-08-25T09:54:11.182 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88643.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88643.log.gz 2023-08-25T09:54:11.183 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112557.log 2023-08-25T09:54:11.184 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120251.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120251.log.gz -5 2023-08-25T09:54:11.184 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.126046.log 2023-08-25T09:54:11.185 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112557.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112557.log.gz 2023-08-25T09:54:11.186 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130681.log 2023-08-25T09:54:11.187 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127241.log 2023-08-25T09:54:11.187 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126046.log.gz 2023-08-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130681.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130681.log.gz 2023-08-25T09:54:11.188 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100347.log 2023-08-25T09:54:11.189 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105890.log 2023-08-25T09:54:11.190 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127241.log.gz 2023-08-25T09:54:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100347.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100347.log.gz 2023-08-25T09:54:11.191 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105498.log 2023-08-25T09:54:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105890.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105890.log.gz 2023-08-25T09:54:11.192 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123907.log 2023-08-25T09:54:11.193 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104825.log 2023-08-25T09:54:11.194 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105498.log.gz 2023-08-25T09:54:11.195 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123907.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123907.log.gz 2023-08-25T09:54:11.195 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111322.log 2023-08-25T09:54:11.196 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104825.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104825.log.gz 2023-08-25T09:54:11.196 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130429.log 2023-08-25T09:54:11.197 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100431.log 2023-08-25T09:54:11.198 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111322.log.gz 2023-08-25T09:54:11.199 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130429.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130429.log.gz 2023-08-25T09:54:11.199 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119196.log 2023-08-25T09:54:11.200 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100431.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100431.log.gz 2023-08-25T09:54:11.201 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123697.log 2023-08-25T09:54:11.202 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117056.log 2023-08-25T09:54:11.202 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119196.log.gz 2023-08-25T09:54:11.203 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86083.log 2023-08-25T09:54:11.203 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123697.log.gz 2023-08-25T09:54:11.204 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117056.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117056.log.gz 2023-08-25T09:54:11.205 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91575.log 2023-08-25T09:54:11.205 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130513.log 2023-08-25T09:54:11.206 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86083.log.gz 2023-08-25T09:54:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91575.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91575.log.gz 2023-08-25T09:54:11.207 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127071.log 2023-08-25T09:54:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130513.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130513.log.gz 2023-08-25T09:54:11.208 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126765.log 2023-08-25T09:54:11.209 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105804.log 2023-08-25T09:54:11.210 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127071.log.gz 2023-08-25T09:54:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126765.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126765.log.gz 2023-08-25T09:54:11.211 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104512.log 2023-08-25T09:54:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105804.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105804.log.gz 2023-08-25T09:54:11.213 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87968.log 2023-08-25T09:54:11.214 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94208.log 2023-08-25T09:54:11.214 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104512.log.gz 2023-08-25T09:54:11.215 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87968.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87968.log.gz 2023-08-25T09:54:11.215 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129255.log 2023-08-25T09:54:11.217 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94208.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94208.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110461.log 2023-08-25T09:54:11.217 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.218 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129255.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129255.log.gz 2023-08-25T09:54:11.218 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113343.log 2023-08-25T09:54:11.219 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110461.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-08-25T09:54:11.220 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110461.log.gz 2023-08-25T09:54:11.221 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113343.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113343.log.gz 2023-08-25T09:54:11.221 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92011.log 2023-08-25T09:54:11.222 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90812.log 2023-08-25T09:54:11.224 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122764.log 2023-08-25T09:54:11.224 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92011.log.gz 2023-08-25T09:54:11.225 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93476.log 2023-08-25T09:54:11.225 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90812.log.gz 2023-08-25T09:54:11.227 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120769.log 2023-08-25T09:54:11.227 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122764.log.gz 2023-08-25T09:54:11.228 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92089.log 2023-08-25T09:54:11.228 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93476.log.gz 2023-08-25T09:54:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125920.log 2023-08-25T09:54:11.230 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120769.log.gz 2023-08-25T09:54:11.231 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105582.log 2023-08-25T09:54:11.231 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92089.log.gz 2023-08-25T09:54:11.232 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128750.log 2023-08-25T09:54:11.233 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125920.log.gz 2023-08-25T09:54:11.234 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114564.log 2023-08-25T09:54:11.234 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105582.log.gz 2023-08-25T09:54:11.235 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128750.log: gzip -5 88.5% --verbose -- /var/log/ceph/ceph-client.admin.108259.log 2023-08-25T09:54:11.236 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.128750.log.gz 2023-08-25T09:54:11.237 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96878.log 2023-08-25T09:54:11.237 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114564.log.gz 2023-08-25T09:54:11.238 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98688.log 2023-08-25T09:54:11.238 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108259.log.gz 2023-08-25T09:54:11.240 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96878.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96878.log.gz 2023-08-25T09:54:11.240 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.109265.log 2023-08-25T09:54:11.241 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98688.log.gz 2023-08-25T09:54:11.242 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96282.log 2023-08-25T09:54:11.243 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93224.log 2023-08-25T09:54:11.243 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109265.log.gz 2023-08-25T09:54:11.245 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91316.log 2023-08-25T09:54:11.245 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96282.log.gz 2023-08-25T09:54:11.246 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93224.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93224.log.gz 2023-08-25T09:54:11.246 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.95046.log 2023-08-25T09:54:11.248 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91316.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91316.log.gz 2023-08-25T09:54:11.248 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112681.log 2023-08-25T09:54:11.249 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112807.log 2023-08-25T09:54:11.249 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95046.log.gz 2023-08-25T09:54:11.251 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113259.log 2023-08-25T09:54:11.251 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112681.log.gz 2023-08-25T09:54:11.252 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88727.log 2023-08-25T09:54:11.253 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112807.log.gz 2023-08-25T09:54:11.253 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113259.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113259.log.gz 2023-08-25T09:54:11.254 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92978.log 2023-08-25T09:54:11.255 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91232.log 2023-08-25T09:54:11.255 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88727.log.gz 2023-08-25T09:54:11.257 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92978.log: gzip -5 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92978.log.gz 2023-08-25T09:54:11.257 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103687.log 2023-08-25T09:54:11.258 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91232.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91232.log.gz 2023-08-25T09:54:11.259 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129498.log 2023-08-25T09:54:11.260 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98406.log 2023-08-25T09:54:11.260 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103687.log.gz 2023-08-25T09:54:11.262 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129498.log: gzip -5 --verbose 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129498.log.gz 2023-08-25T09:54:11.262 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.112849.log 2023-08-25T09:54:11.263 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98406.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98406.log.gz 2023-08-25T09:54:11.263 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97877.log 2023-08-25T09:54:11.265 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112849.log.gz 2023-08-25T09:54:11.265 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104386.log 2023-08-25T09:54:11.266 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97877.log: gzip -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97877.log.gz 2023-08-25T09:54:11.267 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.129093.log 2023-08-25T09:54:11.268 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104386.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104386.log.gz 2023-08-25T09:54:11.268 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93434.log 2023-08-25T09:54:11.269 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126088.log 2023-08-25T09:54:11.270 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129093.log.gz 2023-08-25T09:54:11.271 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93434.log: gzip -5 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93434.log.gz 2023-08-25T09:54:11.272 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.104260.log 2023-08-25T09:54:11.272 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126088.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126088.log.gz 2023-08-25T09:54:11.273 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107391.log 2023-08-25T09:54:11.274 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93104.log 2023-08-25T09:54:11.274 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104260.log.gz 2023-08-25T09:54:11.276 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93350.log 2023-08-25T09:54:11.276 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107391.log.gz 2023-08-25T09:54:11.277 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93104.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93104.log.gz 2023-08-25T09:54:11.278 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.118479.log 2023-08-25T09:54:11.278 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93350.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93350.log.gz 2023-08-25T09:54:11.279 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112987.log 2023-08-25T09:54:11.280 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91022.log 2023-08-25T09:54:11.281 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118479.log.gz 2023-08-25T09:54:11.282 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112987.log: gzip -5 88.8% --verbose -- /var/log/ceph/ceph-client.admin.93854.log 2023-08-25T09:54:11.282 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112987.log.gz 2023-08-25T09:54:11.283 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91022.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91022.log.gz 2023-08-25T09:54:11.284 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119959.log 2023-08-25T09:54:11.285 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93854.log: gzip -5 --verbose 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93854.log.gz 2023-08-25T09:54:11.285 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.95424.log 2023-08-25T09:54:11.286 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119959.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119959.log.gz 2023-08-25T09:54:11.287 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86328.log 2023-08-25T09:54:11.288 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95424.log: gzip -5 --verbose 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95424.log.gz 2023-08-25T09:54:11.289 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.116947.log 2023-08-25T09:54:11.290 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86328.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.86328.log.gz 2023-08-25T09:54:11.290 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123781.log 2023-08-25T09:54:11.292 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85870.log 2023-08-25T09:54:11.292 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116947.log.gz 2023-08-25T09:54:11.293 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108030.log 2023-08-25T09:54:11.293 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123781.log.gz 2023-08-25T09:54:11.294 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.85870.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.85870.log.gz 2023-08-25T09:54:11.295 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.130387.log 2023-08-25T09:54:11.296 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108030.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108030.log.gz 2023-08-25T09:54:11.296 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119879.log 2023-08-25T09:54:11.297 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125119.log 2023-08-25T09:54:11.298 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130387.log.gz 2023-08-25T09:54:11.299 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119879.log: gzip 89.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.119879.log.gz -- 2023-08-25T09:54:11.299 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.102718.log 2023-08-25T09:54:11.301 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125119.log: gzip -5 --verbose 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125119.log.gz 2023-08-25T09:54:11.301 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.126937.log 2023-08-25T09:54:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102718.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102718.log.gz 2023-08-25T09:54:11.302 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118395.log 2023-08-25T09:54:11.304 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126937.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126937.log.gz 2023-08-25T09:54:11.304 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.94580.log 2023-08-25T09:54:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118395.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118395.log.gz 2023-08-25T09:54:11.305 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109179.log 2023-08-25T09:54:11.307 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97415.log 2023-08-25T09:54:11.307 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94580.log.gz 2023-08-25T09:54:11.308 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109179.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109179.log.gz 2023-08-25T09:54:11.309 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96716.log 2023-08-25T09:54:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97415.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97415.log.gz 2023-08-25T09:54:11.310 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114606.log 2023-08-25T09:54:11.311 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96716.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96716.log.gz -5 2023-08-25T09:54:11.312 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125650.log 2023-08-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114606.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114606.log.gz 2023-08-25T09:54:11.313 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.93392.log 2023-08-25T09:54:11.314 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125650.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125650.log.gz 2023-08-25T09:54:11.315 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113217.log 2023-08-25T09:54:11.316 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105115.log 2023-08-25T09:54:11.316 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93392.log.gz 2023-08-25T09:54:11.317 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92793.log 2023-08-25T09:54:11.318 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113217.log.gz 2023-08-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105115.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105115.log.gz 2023-08-25T09:54:11.319 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96378.log 2023-08-25T09:54:11.320 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92793.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92793.log.gz 2023-08-25T09:54:11.321 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119574.log 2023-08-25T09:54:11.322 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102886.log 2023-08-25T09:54:11.322 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96378.log.gz 2023-08-25T09:54:11.324 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119574.log: gzip 87.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.126004.log 2023-08-25T09:54:11.324 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119574.log.gz 2023-08-25T09:54:11.325 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125710.log 2023-08-25T09:54:11.326 INFO:teuthology.orchestra.run.smithi070.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.102886.log.gz 2023-08-25T09:54:11.327 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104867.log 2023-08-25T09:54:11.327 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126004.log.gz 2023-08-25T09:54:11.328 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107535.log 2023-08-25T09:54:11.328 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125710.log.gz 2023-08-25T09:54:11.330 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129750.log 2023-08-25T09:54:11.330 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104867.log.gz 2023-08-25T09:54:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96548.log 2023-08-25T09:54:11.331 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107535.log.gz 2023-08-25T09:54:11.333 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129750.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129750.log.gz 2023-08-25T09:54:11.333 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86669.log 2023-08-25T09:54:11.334 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119448.log 2023-08-25T09:54:11.334 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96548.log.gz 2023-08-25T09:54:11.335 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108164.log 2023-08-25T09:54:11.336 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86669.log.gz 2023-08-25T09:54:11.337 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119322.log 2023-08-25T09:54:11.337 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119448.log.gz 2023-08-25T09:54:11.338 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115179.log 2023-08-25T09:54:11.338 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108164.log.gz 2023-08-25T09:54:11.340 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125794.log 2023-08-25T09:54:11.340 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119322.log.gz 2023-08-25T09:54:11.341 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115179.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115179.log.gz 2023-08-25T09:54:11.341 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103771.log 2023-08-25T09:54:11.343 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127325.log 2023-08-25T09:54:11.343 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125794.log.gz 2023-08-25T09:54:11.344 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112937.log 2023-08-25T09:54:11.344 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103771.log.gz 2023-08-25T09:54:11.346 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101611.log 2023-08-25T09:54:11.346 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127325.log.gz 2023-08-25T09:54:11.347 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88559.log 2023-08-25T09:54:11.347 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112937.log.gz 2023-08-25T09:54:11.349 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88307.log 2023-08-25T09:54:11.349 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101611.log.gz 2023-08-25T09:54:11.350 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106886.log 2023-08-25T09:54:11.350 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.88559.log.gz 2023-08-25T09:54:11.351 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88307.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.109391.log 2023-08-25T09:54:11.352 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88307.log.gz 2023-08-25T09:54:11.353 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106886.log: gzip 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106886.log.gz 2023-08-25T09:54:11.353 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.115095.log 2023-08-25T09:54:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109391.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109391.log.gz 2023-08-25T09:54:11.355 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94166.log 2023-08-25T09:54:11.356 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115095.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115095.log.gz 2023-08-25T09:54:11.357 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.99592.log 2023-08-25T09:54:11.358 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94166.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94166.log.gz 2023-08-25T09:54:11.358 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99026.log 2023-08-25T09:54:11.359 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129708.log 2023-08-25T09:54:11.359 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99592.log.gz 2023-08-25T09:54:11.361 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99026.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99026.log.gz 2023-08-25T09:54:11.361 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111607.log 2023-08-25T09:54:11.362 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129708.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129708.log.gz 2023-08-25T09:54:11.363 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113029.log 2023-08-25T09:54:11.364 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111607.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111607.log.gz 2023-08-25T09:54:11.364 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87939.log 2023-08-25T09:54:11.366 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124909.log 2023-08-25T09:54:11.366 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113029.log.gz 2023-08-25T09:54:11.367 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87939.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87939.log.gz 2023-08-25T09:54:11.367 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.125482.log 2023-08-25T09:54:11.368 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124909.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124909.log.gz 2023-08-25T09:54:11.369 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131143.log 2023-08-25T09:54:11.370 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128077.log 2023-08-25T09:54:11.370 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125482.log.gz 2023-08-25T09:54:11.372 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131143.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131143.log.gz 2023-08-25T09:54:11.372 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.120685.log 2023-08-25T09:54:11.373 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128077.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128077.log.gz 2023-08-25T09:54:11.374 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101485.log 2023-08-25T09:54:11.375 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111733.log 2023-08-25T09:54:11.375 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120685.log.gz 2023-08-25T09:54:11.377 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101485.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101485.log.gz 2023-08-25T09:54:11.377 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.120727.log 2023-08-25T09:54:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111733.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111733.log.gz 2023-08-25T09:54:11.378 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103229.log 2023-08-25T09:54:11.380 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128967.log 2023-08-25T09:54:11.380 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120727.log.gz 2023-08-25T09:54:11.381 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.103229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111943.log 2023-08-25T09:54:11.383 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128967.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128967.log.gz 2023-08-25T09:54:11.383 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92383.log 2023-08-25T09:54:11.385 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111943.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111943.log.gz 2023-08-25T09:54:11.385 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91617.log 2023-08-25T09:54:11.386 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92383.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92383.log.gz 2023-08-25T09:54:11.386 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101172.log 2023-08-25T09:54:11.388 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91617.log: gzip -5 --verbose -- 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91617.log.gz 2023-08-25T09:54:11.388 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.123823.log 2023-08-25T09:54:11.389 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101172.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101172.log.gz 2023-08-25T09:54:11.390 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111238.log 2023-08-25T09:54:11.391 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123823.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123823.log.gz 2023-08-25T09:54:11.392 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113951.log 2023-08-25T09:54:11.393 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111238.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.111238.log.gz 2023-08-25T09:54:11.394 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118902.log 2023-08-25T09:54:11.394 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113951.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113951.log.gz 2023-08-25T09:54:11.395 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118115.log 2023-08-25T09:54:11.396 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118902.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118902.log.gz 2023-08-25T09:54:11.397 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113175.log 2023-08-25T09:54:11.398 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118115.log: gzip -5 --verbose -- 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118115.log.gz 2023-08-25T09:54:11.399 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.108469.log 2023-08-25T09:54:11.400 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113175.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113175.log.gz 2023-08-25T09:54:11.400 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126382.log 2023-08-25T09:54:11.401 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108469.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108469.log.gz 2023-08-25T09:54:11.402 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102844.log 2023-08-25T09:54:11.403 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126382.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126382.log.gz 2023-08-25T09:54:11.404 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89594.log 2023-08-25T09:54:11.405 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102844.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102844.log.gz 2023-08-25T09:54:11.405 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105011.log 2023-08-25T09:54:11.406 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.89594.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.89594.log.gz 2023-08-25T09:54:11.407 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88391.log 2023-08-25T09:54:11.408 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105011.log.gz 2023-08-25T09:54:11.409 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130147.log 2023-08-25T09:54:11.413 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88391.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122932.log 2023-08-25T09:54:11.413 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88391.log.gz 2023-08-25T09:54:11.413 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107736.log 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130147.log.gz 2023-08-25T09:54:11.413 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.413 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122932.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122932.log.gz 2023-08-25T09:54:11.414 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112298.log 2023-08-25T09:54:11.415 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107736.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107736.log.gz 2023-08-25T09:54:11.415 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100761.log 2023-08-25T09:54:11.416 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112298.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112298.log.gz 2023-08-25T09:54:11.417 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125035.log 2023-08-25T09:54:11.418 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100761.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100761.log.gz 2023-08-25T09:54:11.418 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92467.log 2023-08-25T09:54:11.420 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125035.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125035.log.gz 2023-08-25T09:54:11.420 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119658.log 2023-08-25T09:54:11.421 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92467.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92467.log.gz 2023-08-25T09:54:11.422 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107988.log 2023-08-25T09:54:11.423 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119658.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121311.log 2023-08-25T09:54:11.423 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.119658.log.gz 2023-08-25T09:54:11.425 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120209.log 2023-08-25T09:54:11.425 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107988.log.gz 2023-08-25T09:54:11.426 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121311.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121311.log.gz 2023-08-25T09:54:11.427 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117608.log 2023-08-25T09:54:11.428 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120209.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120209.log.gz 2023-08-25T09:54:11.428 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116397.log 2023-08-25T09:54:11.429 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117608.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117608.log.gz 2023-08-25T09:54:11.430 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108748.log 2023-08-25T09:54:11.431 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116397.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116397.log.gz 2023-08-25T09:54:11.432 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87643.log 2023-08-25T09:54:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108748.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108748.log.gz 2023-08-25T09:54:11.433 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130555.log 2023-08-25T09:54:11.435 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87643.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87643.log.gz 2023-08-25T09:54:11.435 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115678.log 2023-08-25T09:54:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130555.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.130555.log.gz 2023-08-25T09:54:11.437 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123865.log 2023-08-25T09:54:11.438 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115678.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115678.log.gz 2023-08-25T09:54:11.439 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102370.log 2023-08-25T09:54:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123865.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123865.log.gz 2023-08-25T09:54:11.440 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101671.log 2023-08-25T09:54:11.442 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102370.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102370.log.gz 2023-08-25T09:54:11.442 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116147.log 2023-08-25T09:54:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101671.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101671.log.gz 2023-08-25T09:54:11.444 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101088.log 2023-08-25T09:54:11.445 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116147.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.116147.log.gz 2023-08-25T09:54:11.446 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108956.log 2023-08-25T09:54:11.447 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101088.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101088.log.gz 2023-08-25T09:54:11.448 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116313.log 2023-08-25T09:54:11.449 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108956.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108956.log.gz 2023-08-25T09:54:11.449 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117913.log 2023-08-25T09:54:11.450 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116313.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116821.log 2023-08-25T09:54:11.451 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116313.log.gz 2023-08-25T09:54:11.452 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.117913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120125.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117913.log.gz 2023-08-25T09:54:11.452 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116821.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116821.log.gz 2023-08-25T09:54:11.454 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93062.log 2023-08-25T09:54:11.455 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120125.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120125.log.gz 2023-08-25T09:54:11.456 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128119.log 2023-08-25T09:54:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93062.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93062.log.gz 2023-08-25T09:54:11.458 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95256.log 2023-08-25T09:54:11.459 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128119.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128119.log.gz 2023-08-25T09:54:11.459 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87977.log 2023-08-25T09:54:11.461 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95256.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95256.log.gz 2023-08-25T09:54:11.462 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96590.log 2023-08-25T09:54:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87977.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87977.log.gz 2023-08-25T09:54:11.463 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112046.log 2023-08-25T09:54:11.464 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96590.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96590.log.gz 2023-08-25T09:54:11.465 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125752.log 2023-08-25T09:54:11.466 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112046.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112046.log.gz 2023-08-25T09:54:11.466 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.107103.log 2023-08-25T09:54:11.467 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125752.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125752.log.gz 2023-08-25T09:54:11.468 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94372.log 2023-08-25T09:54:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107103.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107103.log.gz 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103933.log 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94372.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94372.log.gz 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93518.log 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103933.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103933.log.gz 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92667.log 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93518.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93518.log.gz 2023-08-25T09:54:11.479 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130345.log 2023-08-25T09:54:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92667.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.92667.log.gz 2023-08-25T09:54:11.480 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94874.log 2023-08-25T09:54:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130345.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130345.log.gz 2023-08-25T09:54:11.480 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115473.log 2023-08-25T09:54:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94874.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94874.log.gz 2023-08-25T09:54:11.481 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93728.log 2023-08-25T09:54:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115473.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115473.log.gz 2023-08-25T09:54:11.481 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87299.log 2023-08-25T09:54:11.483 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93728.log: gzip -5 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93728.log.gz 2023-08-25T09:54:11.483 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.113427.log 2023-08-25T09:54:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87299.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87299.log.gz 2023-08-25T09:54:11.484 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129582.log 2023-08-25T09:54:11.486 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109945.log 2023-08-25T09:54:11.486 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.113427.log.gz 2023-08-25T09:54:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129582.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129582.log.gz 2023-08-25T09:54:11.487 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104596.log 2023-08-25T09:54:11.489 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104344.log 2023-08-25T09:54:11.490 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.109945.log: /var/log/ceph/ceph-client.admin.104596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100172.log 2023-08-25T09:54:11.491 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104596.log.gz 2023-08-25T09:54:11.492 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118827.log 2023-08-25T09:54:11.493 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104344.log.gz 2023-08-25T09:54:11.494 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100172.log.gz 2023-08-25T09:54:11.494 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107145.log 2023-08-25T09:54:11.495 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116863.log 2023-08-25T09:54:11.496 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118827.log.gz 2023-08-25T09:54:11.497 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100635.log 2023-08-25T09:54:11.498 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107145.log.gz 2023-08-25T09:54:11.499 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91801.log 2023-08-25T09:54:11.499 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116863.log.gz 2023-08-25T09:54:11.501 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91358.log 2023-08-25T09:54:11.502 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100635.log.gz 2023-08-25T09:54:11.502 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91801.log: gzip 88.6% -5 --verbose -- /var/log/ceph/ceph-client.admin.101214.log 2023-08-25T09:54:11.502 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91801.log.gz 2023-08-25T09:54:11.503 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99134.log 2023-08-25T09:54:11.504 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91358.log.gz 2023-08-25T09:54:11.504 INFO:teuthology.orchestra.run.smithi155.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-08-25T09:54:11.504 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101214.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101214.log.gz 2023-08-25T09:54:11.505 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90728.log 2023-08-25T09:54:11.507 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102242.log 2023-08-25T09:54:11.507 INFO:teuthology.orchestra.run.smithi070.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.99134.log.gz 2023-08-25T09:54:11.508 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90728.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119532.log 2023-08-25T09:54:11.509 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.510 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94706.log 2023-08-25T09:54:11.510 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102242.log.gz 2023-08-25T09:54:11.511 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119532.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119532.log.gz 2023-08-25T09:54:11.513 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105372.log 2023-08-25T09:54:11.514 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94706.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.94706.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.107820.log 2023-08-25T09:54:11.515 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.516 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116905.log 2023-08-25T09:54:11.516 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105372.log.gz 2023-08-25T09:54:11.517 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107820.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107820.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.113769.log 2023-08-25T09:54:11.518 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.519 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102036.log 2023-08-25T09:54:11.519 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116905.log.gz 2023-08-25T09:54:11.523 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.113769.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.113769.log.gz 2023-08-25T09:54:11.524 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108830.log 2023-08-25T09:54:11.526 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97961.log 2023-08-25T09:54:11.526 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102036.log.gz 2023-08-25T09:54:11.527 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99782.log 2023-08-25T09:54:11.528 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108830.log.gz 2023-08-25T09:54:11.529 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89436.log 2023-08-25T09:54:11.530 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99782.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97961.log.gz 2023-08-25T09:54:11.531 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99782.log.gz 2023-08-25T09:54:11.532 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130639.log 2023-08-25T09:54:11.533 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.89436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114648.log 2023-08-25T09:54:11.535 INFO:teuthology.orchestra.run.smithi070.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.89436.log.gz 2023-08-25T09:54:11.535 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130639.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130639.log.gz 2023-08-25T09:54:11.536 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95466.log 2023-08-25T09:54:11.537 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114648.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114648.log.gz 2023-08-25T09:54:11.539 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110992.log 2023-08-25T09:54:11.540 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93602.log 2023-08-25T09:54:11.541 INFO:teuthology.orchestra.run.smithi070.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.95466.log.gz 2023-08-25T09:54:11.542 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95382.log 2023-08-25T09:54:11.543 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7%/var/log/ceph/ceph-client.admin.93602.log: -- replaced with /var/log/ceph/ceph-client.admin.110992.log.gz 2023-08-25T09:54:11.544 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93602.log.gz 2023-08-25T09:54:11.545 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106376.log 2023-08-25T09:54:11.547 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105626.log 2023-08-25T09:54:11.547 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95382.log.gz 2023-08-25T09:54:11.548 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96674.log 2023-08-25T09:54:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105626.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106376.log.gz 2023-08-25T09:54:11.550 INFO:teuthology.orchestra.run.smithi070.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105626.log.gz 2023-08-25T09:54:11.551 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110590.log 2023-08-25T09:54:11.552 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120517.log 2023-08-25T09:54:11.553 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96674.log: /var/log/ceph/ceph-client.admin.110590.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96674.log.gz 2023-08-25T09:54:11.554 INFO:teuthology.orchestra.run.smithi070.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110590.log.gz 2023-08-25T09:54:11.555 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122451.log 2023-08-25T09:54:11.556 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101859.log 2023-08-25T09:54:11.557 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120517.log: /var/log/ceph/ceph-client.admin.122451.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120517.log.gz 2023-08-25T09:54:11.558 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122451.log.gz 2023-08-25T09:54:11.559 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88769.log 2023-08-25T09:54:11.560 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107433.log 2023-08-25T09:54:11.561 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101859.log: /var/log/ceph/ceph-client.admin.88769.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101859.log.gz 2023-08-25T09:54:11.562 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88769.log.gz 2023-08-25T09:54:11.563 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121353.log 2023-08-25T09:54:11.565 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107904.log 2023-08-25T09:54:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107433.log: /var/log/ceph/ceph-client.admin.121353.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107433.log.gz 2023-08-25T09:54:11.566 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121353.log.gz 2023-08-25T09:54:11.566 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115720.log 2023-08-25T09:54:11.568 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125197.log 2023-08-25T09:54:11.569 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107904.log: /var/log/ceph/ceph-client.admin.115720.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107904.log.gz 2023-08-25T09:54:11.570 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115720.log.gz 2023-08-25T09:54:11.571 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129297.log 2023-08-25T09:54:11.572 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95172.log 2023-08-25T09:54:11.573 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125197.log: /var/log/ceph/ceph-client.admin.129297.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125197.log.gz 2023-08-25T09:54:11.574 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129297.log.gz 2023-08-25T09:54:11.575 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99740.log 2023-08-25T09:54:11.576 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110734.log 2023-08-25T09:54:11.577 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95172.log: /var/log/ceph/ceph-client.admin.99740.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95172.log.gz 2023-08-25T09:54:11.578 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99740.log.gz 2023-08-25T09:54:11.579 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109433.log 2023-08-25T09:54:11.580 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88685.log 2023-08-25T09:54:11.581 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110734.log: /var/log/ceph/ceph-client.admin.109433.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110734.log.gz 2023-08-25T09:54:11.583 INFO:teuthology.orchestra.run.smithi070.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109433.log.gz 2023-08-25T09:54:11.583 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109223.log 2023-08-25T09:54:11.585 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96084.log 2023-08-25T09:54:11.586 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88685.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88685.log.gz 2023-08-25T09:54:11.587 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109223.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.109223.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.125077.log 2023-08-25T09:54:11.587 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.588 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116483.log 2023-08-25T09:54:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96084.log: /var/log/ceph/ceph-client.admin.125077.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96084.log.gz 2023-08-25T09:54:11.590 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125077.log.gz 2023-08-25T09:54:11.590 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117796.log 2023-08-25T09:54:11.592 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122307.log 2023-08-25T09:54:11.593 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116483.log: /var/log/ceph/ceph-client.admin.117796.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116483.log.gz 2023-08-25T09:54:11.594 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117796.log.gz 2023-08-25T09:54:11.595 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98280.log 2023-08-25T09:54:11.596 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86539.log 2023-08-25T09:54:11.597 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122307.log: /var/log/ceph/ceph-client.admin.98280.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122307.log.gz 2023-08-25T09:54:11.598 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98280.log.gz 2023-08-25T09:54:11.598 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118157.log 2023-08-25T09:54:11.600 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115053.log 2023-08-25T09:54:11.602 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86539.log: /var/log/ceph/ceph-client.admin.118157.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118157.log.gz 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86539.log.gz 2023-08-25T09:54:11.602 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.603 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100677.log 2023-08-25T09:54:11.604 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115347.log 2023-08-25T09:54:11.606 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115053.log: /var/log/ceph/ceph-client.admin.100677.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115053.log.gz 2023-08-25T09:54:11.606 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100677.log.gz 2023-08-25T09:54:11.607 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104017.log 2023-08-25T09:54:11.608 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125836.log 2023-08-25T09:54:11.610 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115347.log: /var/log/ceph/ceph-client.admin.104017.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115347.log.gz 2023-08-25T09:54:11.610 INFO:teuthology.orchestra.run.smithi155.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-08-25T09:54:11.611 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104017.log.gz 2023-08-25T09:54:11.611 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-08-25T09:54:11.612 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129381.log 2023-08-25T09:54:11.614 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125836.log: /var/log/ceph/ceph-mgr.y.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125836.log.gz 2023-08-25T09:54:11.615 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107694.log 2023-08-25T09:54:11.617 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129381.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129381.log.gz 2023-08-25T09:54:11.619 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97457.log 2023-08-25T09:54:11.621 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107694.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107694.log.gz 2023-08-25T09:54:11.622 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129666.log 2023-08-25T09:54:11.624 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97457.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97457.log.gz 2023-08-25T09:54:11.626 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98628.log 2023-08-25T09:54:11.628 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129666.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129666.log.gz 2023-08-25T09:54:11.629 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88853.log 2023-08-25T09:54:11.631 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98628.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98628.log.gz 2023-08-25T09:54:11.633 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93020.log 2023-08-25T09:54:11.635 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88853.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.88853.log.gz 2023-08-25T09:54:11.636 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115930.log 2023-08-25T09:54:11.639 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93020.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93020.log.gz 2023-08-25T09:54:11.640 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130765.log 2023-08-25T09:54:11.641 INFO:teuthology.orchestra.run.smithi070.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-08-25T09:54:11.643 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115930.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115930.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-08-25T09:54:11.643 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.645 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130765.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130765.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104470.log 2023-08-25T09:54:11.645 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129876.log 2023-08-25T09:54:11.648 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104470.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104470.log.gz 2023-08-25T09:54:11.650 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91533.log 2023-08-25T09:54:11.652 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129876.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129876.log.gz 2023-08-25T09:54:11.653 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127199.log 2023-08-25T09:54:11.655 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91533.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91533.log.gz 2023-08-25T09:54:11.656 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108872.log 2023-08-25T09:54:11.658 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127199.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127199.log.gz 2023-08-25T09:54:11.660 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92935.log 2023-08-25T09:54:11.662 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108872.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108872.log.gz 2023-08-25T09:54:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102802.log 2023-08-25T09:54:11.665 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92935.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92935.log.gz 2023-08-25T09:54:11.666 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96758.log 2023-08-25T09:54:11.669 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102802.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102802.log.gz 2023-08-25T09:54:11.670 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88349.log 2023-08-25T09:54:11.672 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.96758.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96758.log.gz 2023-08-25T09:54:11.673 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98816.log 2023-08-25T09:54:11.675 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88349.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88349.log.gz 2023-08-25T09:54:11.677 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102414.log 2023-08-25T09:54:11.679 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98816.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98816.log.gz 2023-08-25T09:54:11.680 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.90000.log 2023-08-25T09:54:11.682 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102414.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102414.log.gz 2023-08-25T09:54:11.684 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126680.log 2023-08-25T09:54:11.687 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.2.90000.log: 92.6% -- replaced with /var/log/ceph/ceph-client.2.90000.log.gz 2023-08-25T09:54:11.688 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85625.log 2023-08-25T09:54:11.690 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126680.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126680.log.gz 2023-08-25T09:54:11.692 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-08-25T09:54:11.692 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109129.log 2023-08-25T09:54:11.694 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.85625.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.85625.log.gz 2023-08-25T09:54:11.696 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105053.log 2023-08-25T09:54:11.698 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109129.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109129.log.gz 2023-08-25T09:54:11.700 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98322.log 2023-08-25T09:54:11.703 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105053.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105053.log.gz 2023-08-25T09:54:11.703 INFO:teuthology.orchestra.run.smithi070.stderr: 94.2% -- replaced with /var/log/ceph/ceph-client.0.103229.log.gz 2023-08-25T09:54:11.704 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101954.log 2023-08-25T09:54:11.704 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-25T09:54:11.706 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98322.log: /var/log/ceph/ceph-client.admin.101954.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98322.log.gz 2023-08-25T09:54:11.706 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101954.log.gz 2023-08-25T09:54:11.707 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128351.log 2023-08-25T09:54:11.708 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120811.log 2023-08-25T09:54:11.710 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128351.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128351.log.gz 2023-08-25T09:54:11.711 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100719.log 2023-08-25T09:54:11.712 INFO:teuthology.orchestra.run.smithi155.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-08-25T09:54:11.713 INFO:teuthology.orchestra.run.smithi070.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-25T09:54:11.714 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120811.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.120811.log.gz 2023-08-25T09:54:11.715 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128708.log 2023-08-25T09:54:11.716 INFO:teuthology.orchestra.run.smithi155.stderr: 2023-08-25T09:54:11.717 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m1.107s 2023-08-25T09:54:11.717 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m4.606s 2023-08-25T09:54:11.717 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.591s 2023-08-25T09:54:11.719 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103813.log 2023-08-25T09:54:11.719 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100719.log.gz 2023-08-25T09:54:11.720 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128708.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128708.log.gz 2023-08-25T09:54:11.721 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97121.log 2023-08-25T09:54:11.722 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103813.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91885.log 2023-08-25T09:54:11.723 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103813.log.gz 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-08-25T09:54:11.723 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.724 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100130.log 2023-08-25T09:54:11.725 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97121.log: /var/log/ceph/ceph-client.admin.91885.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97121.log.gz 2023-08-25T09:54:11.726 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104302.log 2023-08-25T09:54:11.726 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91885.log.gz 2023-08-25T09:54:11.727 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105848.log 2023-08-25T09:54:11.728 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8%/var/log/ceph/ceph-client.admin.104302.log: -- replaced with /var/log/ceph/ceph-client.admin.100130.log.gz 2023-08-25T09:54:11.728 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113301.log 2023-08-25T09:54:11.729 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104302.log.gz 2023-08-25T09:54:11.730 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.105848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129834.log 2023-08-25T09:54:11.730 INFO:teuthology.orchestra.run.smithi070.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105848.log.gz 2023-08-25T09:54:11.731 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100214.log 2023-08-25T09:54:11.732 INFO:teuthology.orchestra.run.smithi070.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113301.log.gz 2023-08-25T09:54:11.733 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129834.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129834.log.gz -5 2023-08-25T09:54:11.733 INFO:teuthology.orchestra.run.smithi070.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98942.log 2023-08-25T09:54:11.734 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120167.log 2023-08-25T09:54:11.735 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100214.log.gz 2023-08-25T09:54:11.736 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130975.log 2023-08-25T09:54:11.736 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98942.log.gz 2023-08-25T09:54:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123949.log 2023-08-25T09:54:11.738 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120167.log.gz 2023-08-25T09:54:11.739 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130975.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.130975.log.gz --verbose 2023-08-25T09:54:11.739 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.116695.log 2023-08-25T09:54:11.740 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102592.log 2023-08-25T09:54:11.741 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123949.log.gz 2023-08-25T09:54:11.742 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116695.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116695.log.gz 2023-08-25T09:54:11.742 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.97751.log 2023-08-25T09:54:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102592.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102592.log.gz 2023-08-25T09:54:11.743 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112172.log 2023-08-25T09:54:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97751.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97751.log.gz 2023-08-25T09:54:11.745 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93644.log 2023-08-25T09:54:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112172.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112172.log.gz 2023-08-25T09:54:11.746 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130597.log 2023-08-25T09:54:11.748 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93644.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93644.log.gz 2023-08-25T09:54:11.749 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.91106.log 2023-08-25T09:54:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130597.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130597.log.gz 2023-08-25T09:54:11.750 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91843.log 2023-08-25T09:54:11.751 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91106.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127493.log 2023-08-25T09:54:11.752 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.91106.log.gz 2023-08-25T09:54:11.753 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91843.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.91843.log.gz 2023-08-25T09:54:11.753 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.93896.log 2023-08-25T09:54:11.754 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127493.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127493.log.gz 2023-08-25T09:54:11.755 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111034.log 2023-08-25T09:54:11.756 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93896.log: gzip -5 --verbose 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93896.log.gz 2023-08-25T09:54:11.757 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.104638.log 2023-08-25T09:54:11.757 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111034.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111034.log.gz 2023-08-25T09:54:11.758 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92509.log 2023-08-25T09:54:11.759 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104638.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104638.log.gz 2023-08-25T09:54:11.760 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86879.log 2023-08-25T09:54:11.761 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92509.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127931.log 2023-08-25T09:54:11.761 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92509.log.gz 2023-08-25T09:54:11.763 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119700.log 2023-08-25T09:54:11.763 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.86879.log.gz 2023-08-25T09:54:11.764 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.127931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104428.log 2023-08-25T09:54:11.766 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123058.log 2023-08-25T09:54:11.766 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119700.log.gz 2023-08-25T09:54:11.768 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92751.log 2023-08-25T09:54:11.770 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104428.log: /var/log/ceph/ceph-client.admin.123058.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104428.log.gz 2023-08-25T09:54:11.770 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123058.log.gz 2023-08-25T09:54:11.770 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98364.log 2023-08-25T09:54:11.772 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103729.log 2023-08-25T09:54:11.772 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92751.log: /var/log/ceph/ceph-client.admin.98364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92751.log.gz 2023-08-25T09:54:11.773 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98364.log.gz 2023-08-25T09:54:11.773 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109475.log 2023-08-25T09:54:11.774 INFO:teuthology.orchestra.run.smithi070.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-08-25T09:54:11.775 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114151.log 2023-08-25T09:54:11.776 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109475.log: /var/log/ceph/ceph-client.admin.103729.log: 88.6% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103729.log.gz 2023-08-25T09:54:11.777 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109475.log.gz 2023-08-25T09:54:11.777 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92709.log 2023-08-25T09:54:11.778 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114151.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114151.log.gz 2023-08-25T09:54:11.779 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119028.log 2023-08-25T09:54:11.779 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92709.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92709.log.gz 2023-08-25T09:54:11.780 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98448.log 2023-08-25T09:54:11.781 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119028.log.gz 2023-08-25T09:54:11.782 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109040.log 2023-08-25T09:54:11.782 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98448.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98448.log.gz 2023-08-25T09:54:11.783 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128792.log 2023-08-25T09:54:11.784 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109040.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109040.log.gz 2023-08-25T09:54:11.784 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125323.log 2023-08-25T09:54:11.785 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128792.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128792.log.gz 2023-08-25T09:54:11.786 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127409.log 2023-08-25T09:54:11.787 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125323.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125323.log.gz 2023-08-25T09:54:11.787 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125608.log 2023-08-25T09:54:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127409.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127409.log.gz 2023-08-25T09:54:11.789 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107778.log 2023-08-25T09:54:11.790 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125608.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125608.log.gz 2023-08-25T09:54:11.790 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112003.log 2023-08-25T09:54:11.791 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107778.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107778.log.gz 2023-08-25T09:54:11.792 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93266.log 2023-08-25T09:54:11.793 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112003.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112003.log.gz 2023-08-25T09:54:11.794 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111364.log 2023-08-25T09:54:11.794 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93266.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93266.log.gz 2023-08-25T09:54:11.795 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129624.log 2023-08-25T09:54:11.796 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111364.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111364.log.gz 2023-08-25T09:54:11.797 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121776.log 2023-08-25T09:54:11.797 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129624.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129624.log.gz 2023-08-25T09:54:11.798 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127643.log 2023-08-25T09:54:11.800 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121776.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86795.log 2023-08-25T09:54:11.800 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.121776.log.gz 2023-08-25T09:54:11.801 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120040.log 2023-08-25T09:54:11.802 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127643.log.gz 2023-08-25T09:54:11.802 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86795.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86795.log.gz 2023-08-25T09:54:11.803 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111691.log 2023-08-25T09:54:11.804 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88517.log 2023-08-25T09:54:11.804 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120040.log.gz 2023-08-25T09:54:11.805 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114810.log 2023-08-25T09:54:11.806 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111691.log.gz 2023-08-25T09:54:11.807 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86625.log 2023-08-25T09:54:11.807 INFO:teuthology.orchestra.run.smithi070.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88517.log.gz 2023-08-25T09:54:11.808 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123016.log 2023-08-25T09:54:11.808 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114810.log.gz 2023-08-25T09:54:11.809 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86625.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.86625.log.gz --verbose 2023-08-25T09:54:11.810 INFO:teuthology.orchestra.run.smithi070.stderr: -- /var/log/ceph/ceph-client.admin.119406.log 2023-08-25T09:54:11.811 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123016.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131286.log 2023-08-25T09:54:11.811 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.123016.log.gz 2023-08-25T09:54:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102634.log 2023-08-25T09:54:11.813 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119406.log.gz 2023-08-25T09:54:11.813 INFO:teuthology.orchestra.run.smithi070.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-08-25T09:54:11.814 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131286.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.131286.log.gz 2023-08-25T09:54:11.814 INFO:teuthology.orchestra.run.smithi070.stderr: /var/log/ceph/ceph-client.admin.110419.log 2023-08-25T09:54:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102634.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102634.log.gz 2023-08-25T09:54:11.815 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109667.log 2023-08-25T09:54:11.817 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123195.log 2023-08-25T09:54:11.817 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110419.log.gz 2023-08-25T09:54:11.818 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.109667.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109667.log.gz 2023-08-25T09:54:11.818 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114690.log 2023-08-25T09:54:11.819 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98858.log 2023-08-25T09:54:11.820 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123195.log.gz 2023-08-25T09:54:11.820 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114690.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114690.log.gz 2023-08-25T09:54:11.820 INFO:teuthology.orchestra.run.smithi070.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.0.127931.log.gz 2023-08-25T09:54:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108385.log 2023-08-25T09:54:11.821 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129918.log 2023-08-25T09:54:11.822 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98858.log.gz 2023-08-25T09:54:11.822 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99550.log 2023-08-25T09:54:11.823 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108385.log.gz 2023-08-25T09:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.129918.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129918.log.gz 2023-08-25T09:54:11.824 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126477.log 2023-08-25T09:54:11.825 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.99550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122974.log 2023-08-25T09:54:11.825 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99550.log.gz 2023-08-25T09:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126477.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126477.log.gz 2023-08-25T09:54:11.826 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118743.log 2023-08-25T09:54:11.827 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122974.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-08-25T09:54:11.827 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122974.log.gz 2023-08-25T09:54:11.828 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118743.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118743.log.gz 2023-08-25T09:54:11.828 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123441.log 2023-08-25T09:54:11.829 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102506.log 2023-08-25T09:54:11.831 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.123441.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123441.log.gz 2023-08-25T09:54:11.831 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100962.log 2023-08-25T09:54:11.832 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102506.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102506.log.gz 2023-08-25T09:54:11.833 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115221.log 2023-08-25T09:54:11.834 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131185.log 88.7% 2023-08-25T09:54:11.834 INFO:teuthology.orchestra.run.smithi070.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100962.log.gz 2023-08-25T09:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115221.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.115221.log.gz 2023-08-25T09:54:11.835 INFO:teuthology.orchestra.run.smithi070.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.101755.log 2023-08-25T09:54:11.837 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.131185.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131185.log.gz 2023-08-25T09:54:11.837 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93308.log 2023-08-25T09:54:11.838 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101755.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101755.log.gz 2023-08-25T09:54:11.838 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97289.log 2023-08-25T09:54:11.839 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.93308.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93308.log.gz 2023-08-25T09:54:11.840 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94538.log 2023-08-25T09:54:11.841 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97289.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97289.log.gz 2023-08-25T09:54:11.842 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102676.log 2023-08-25T09:54:11.843 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94538.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94538.log.gz 2023-08-25T09:54:11.843 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97583.log 2023-08-25T09:54:11.844 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102676.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102676.log.gz 2023-08-25T09:54:11.845 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97331.log 2023-08-25T09:54:11.846 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97583.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97583.log.gz 2023-08-25T09:54:11.846 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115389.log 2023-08-25T09:54:11.847 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97331.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97331.log.gz 2023-08-25T09:54:11.848 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103603.log 2023-08-25T09:54:11.849 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115389.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115389.log.gz 2023-08-25T09:54:11.849 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126987.log 2023-08-25T09:54:11.850 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103603.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103603.log.gz 2023-08-25T09:54:11.851 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101298.log 2023-08-25T09:54:11.851 INFO:teuthology.orchestra.run.smithi070.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.0.109945.log.gz 2023-08-25T09:54:11.852 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126987.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126987.log.gz 2023-08-25T09:54:11.852 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116653.log 2023-08-25T09:54:11.854 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111565.log 2023-08-25T09:54:11.854 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101298.log.gz 2023-08-25T09:54:11.855 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116653.log.gz 2023-08-25T09:54:11.855 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104741.log 2023-08-25T09:54:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111565.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111565.log.gz 2023-08-25T09:54:11.857 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100473.log 2023-08-25T09:54:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.104741.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104741.log.gz 2023-08-25T09:54:11.858 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111406.log 2023-08-25T09:54:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100473.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100473.log.gz 2023-08-25T09:54:11.860 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103410.log 2023-08-25T09:54:11.861 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.111406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95004.log 2023-08-25T09:54:11.861 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111406.log.gz 2023-08-25T09:54:11.862 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.103410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101527.log 2023-08-25T09:54:11.863 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103410.log.gz 2023-08-25T09:54:11.864 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116779.log 2023-08-25T09:54:11.864 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95004.log.gz 2023-08-25T09:54:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101527.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101527.log.gz 2023-08-25T09:54:11.866 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126256.log 2023-08-25T09:54:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116779.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116779.log.gz 2023-08-25T09:54:11.867 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86371.log 2023-08-25T09:54:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126256.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126256.log.gz 2023-08-25T09:54:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95214.log 2023-08-25T09:54:11.870 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90644.log 2023-08-25T09:54:11.870 INFO:teuthology.orchestra.run.smithi070.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.86371.log.gz 2023-08-25T09:54:11.871 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101130.log 2023-08-25T09:54:11.872 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95214.log.gz 2023-08-25T09:54:11.873 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86753.log 2023-08-25T09:54:11.873 INFO:teuthology.orchestra.run.smithi070.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90644.log.gz 2023-08-25T09:54:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108301.log 2023-08-25T09:54:11.875 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101130.log.gz 2023-08-25T09:54:11.876 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86753.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.87313.log -- replaced with /var/log/ceph/ceph-client.admin.86753.log.gz 2023-08-25T09:54:11.876 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108301.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.108301.log.gz 2023-08-25T09:54:11.878 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92893.log 2023-08-25T09:54:11.879 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120601.log 2023-08-25T09:54:11.879 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87313.log.gz 2023-08-25T09:54:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91969.log 2023-08-25T09:54:11.881 INFO:teuthology.orchestra.run.smithi070.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92893.log.gz 2023-08-25T09:54:11.882 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87649.log 2023-08-25T09:54:11.882 INFO:teuthology.orchestra.run.smithi070.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120601.log.gz 2023-08-25T09:54:11.883 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91969.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91969.log.gz 2023-08-25T09:54:11.884 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126298.log 2023-08-25T09:54:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87649.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87649.log.gz 2023-08-25T09:54:11.885 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114013.log 2023-08-25T09:54:11.886 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126298.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126298.log.gz 2023-08-25T09:54:11.886 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86455.log 2023-08-25T09:54:11.888 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114013.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114013.log.gz 2023-08-25T09:54:11.888 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112765.log 2023-08-25T09:54:11.889 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121734.log 2023-08-25T09:54:11.889 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.86455.log.gz 2023-08-25T09:54:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112765.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112765.log.gz 2023-08-25T09:54:11.891 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114289.log 2023-08-25T09:54:11.892 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114768.log 2023-08-25T09:54:11.892 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121734.log.gz 2023-08-25T09:54:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114289.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.114289.log.gz 2023-08-25T09:54:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125239.log 2023-08-25T09:54:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114768.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114768.log.gz 2023-08-25T09:54:11.895 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127115.log 2023-08-25T09:54:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125239.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125239.log.gz 2023-08-25T09:54:11.897 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86413.log 2023-08-25T09:54:11.898 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127115.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127115.log.gz 2023-08-25T09:54:11.899 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101713.log 2023-08-25T09:54:11.900 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86413.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86413.log.gz 2023-08-25T09:54:11.900 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107862.log 2023-08-25T09:54:11.901 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.101713.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101713.log.gz 2023-08-25T09:54:11.902 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95298.log 2023-08-25T09:54:11.903 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107862.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107862.log.gz 2023-08-25T09:54:11.903 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92299.log 2023-08-25T09:54:11.904 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.95298.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95298.log.gz 2023-08-25T09:54:11.905 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87305.log 2023-08-25T09:54:11.906 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92299.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92299.log.gz 2023-08-25T09:54:11.906 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116611.log 2023-08-25T09:54:11.907 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87305.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87305.log.gz 2023-08-25T09:54:11.908 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108114.log 2023-08-25T09:54:11.908 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116611.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116611.log.gz 2023-08-25T09:54:11.909 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112088.log 2023-08-25T09:54:11.910 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108114.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108114.log.gz 2023-08-25T09:54:11.911 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128181.log 2023-08-25T09:54:11.913 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112088.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112088.log.gz 2023-08-25T09:54:11.913 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124075.log 2023-08-25T09:54:11.914 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128181.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128181.log.gz 2023-08-25T09:54:11.915 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94022.log 2023-08-25T09:54:11.916 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90476.log 2023-08-25T09:54:11.916 INFO:teuthology.orchestra.run.smithi070.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-08-25T09:54:11.916 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124075.log.gz 2023-08-25T09:54:11.917 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94022.log.gz 2023-08-25T09:54:11.917 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106100.log 2023-08-25T09:54:11.918 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90476.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90476.log.gz 2023-08-25T09:54:11.918 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118199.log 2023-08-25T09:54:11.919 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.106100.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106100.log.gz 2023-08-25T09:54:11.919 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107061.log 2023-08-25T09:54:11.920 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118199.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118199.log.gz 2023-08-25T09:54:11.920 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98023.log 2023-08-25T09:54:11.921 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107061.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107061.log.gz 2023-08-25T09:54:11.922 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90156.log 2023-08-25T09:54:11.922 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.98023.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98023.log.gz 2023-08-25T09:54:11.923 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126638.log 2023-08-25T09:54:11.923 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90156.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90156.log.gz 2023-08-25T09:54:11.924 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88811.log 2023-08-25T09:54:11.925 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126638.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.126638.log.gz 2023-08-25T09:54:11.925 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130189.log 2023-08-25T09:54:11.926 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88811.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88811.log.gz 2023-08-25T09:54:11.926 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112214.log 2023-08-25T09:54:11.927 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130189.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130189.log.gz 2023-08-25T09:54:11.927 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102116.log 2023-08-25T09:54:11.928 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112214.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112214.log.gz 2023-08-25T09:54:11.928 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114480.log 2023-08-25T09:54:11.929 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102116.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102116.log.gz 2023-08-25T09:54:11.929 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108343.log 2023-08-25T09:54:11.930 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114480.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114480.log.gz 2023-08-25T09:54:11.931 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94064.log 2023-08-25T09:54:11.931 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108343.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108343.log.gz 2023-08-25T09:54:11.932 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87636.log 2023-08-25T09:54:11.933 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94064.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94064.log.gz 2023-08-25T09:54:11.933 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114193.log 2023-08-25T09:54:11.934 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87636.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87636.log.gz 2023-08-25T09:54:11.934 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99410.log 2023-08-25T09:54:11.935 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114193.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114193.log.gz 2023-08-25T09:54:11.935 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97004.log 2023-08-25T09:54:11.936 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.99410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86921.log 2023-08-25T09:54:11.938 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97004.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97004.log.gz 2023-08-25T09:54:11.938 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110950.log 2023-08-25T09:54:11.939 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.86921.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86921.log.gz 2023-08-25T09:54:11.939 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90896.log 2023-08-25T09:54:11.940 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112256.log 2023-08-25T09:54:11.941 INFO:teuthology.orchestra.run.smithi070.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110950.log.gz 2023-08-25T09:54:11.942 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.90896.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90896.log.gz 2023-08-25T09:54:11.942 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110674.log 2023-08-25T09:54:11.943 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112256.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112256.log.gz 2023-08-25T09:54:11.943 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114894.log 2023-08-25T09:54:11.944 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87005.log 2023-08-25T09:54:11.945 INFO:teuthology.orchestra.run.smithi070.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110674.log.gz 2023-08-25T09:54:11.946 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114894.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114894.log.gz 2023-08-25T09:54:11.946 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108072.log 2023-08-25T09:54:11.947 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87005.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87005.log.gz 2023-08-25T09:54:11.947 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87632.log 2023-08-25T09:54:11.948 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113071.log 2023-08-25T09:54:11.949 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108072.log.gz 2023-08-25T09:54:11.949 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87632.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87632.log.gz 2023-08-25T09:54:11.950 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122619.log 2023-08-25T09:54:11.951 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.113071.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113071.log.gz 2023-08-25T09:54:11.951 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91701.log 2023-08-25T09:54:11.952 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.122619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120431.log 2023-08-25T09:54:11.953 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.122619.log.gz/var/log/ceph/ceph-client.admin.91701.log: 2023-08-25T09:54:11.953 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91701.log.gz 2023-08-25T09:54:11.954 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124951.log 2023-08-25T09:54:11.955 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.120431.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120431.log.gz 2023-08-25T09:54:11.955 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118031.log 2023-08-25T09:54:11.956 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.124951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126849.log 2023-08-25T09:54:11.957 INFO:teuthology.orchestra.run.smithi070.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124951.log.gz 2023-08-25T09:54:11.958 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118031.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118031.log.gz 2023-08-25T09:54:11.958 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112477.log 2023-08-25T09:54:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126849.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126849.log.gz 2023-08-25T09:54:11.959 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116355.log 2023-08-25T09:54:11.960 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92131.log 2023-08-25T09:54:11.961 INFO:teuthology.orchestra.run.smithi070.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112477.log.gz 2023-08-25T09:54:11.961 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116355.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116355.log.gz 2023-08-25T09:54:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106653.log 2023-08-25T09:54:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.92131.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.92131.log.gz 2023-08-25T09:54:11.963 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128925.log 2023-08-25T09:54:11.964 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.0.106653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94496.log 2023-08-25T09:54:11.965 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.128925.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128925.log.gz 2023-08-25T09:54:11.966 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87296.log 2023-08-25T09:54:11.967 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.94496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121415.log 2023-08-25T09:54:11.967 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94496.log.gz 2023-08-25T09:54:11.968 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87296.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.87296.log.gz 2023-08-25T09:54:11.968 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116271.log 2023-08-25T09:54:11.969 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107946.log 2023-08-25T09:54:11.970 INFO:teuthology.orchestra.run.smithi070.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121415.log.gz 2023-08-25T09:54:11.971 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.116271.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116271.log.gz 2023-08-25T09:54:11.971 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102952.log 2023-08-25T09:54:11.972 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87979.log 2023-08-25T09:54:11.972 INFO:teuthology.orchestra.run.smithi070.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107946.log.gz 2023-08-25T09:54:11.973 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.102952.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102952.log.gz 2023-08-25T09:54:11.973 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127283.log 2023-08-25T09:54:11.974 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112130.log 2023-08-25T09:54:11.975 INFO:teuthology.orchestra.run.smithi070.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.87979.log.gz 2023-08-25T09:54:11.976 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.127283.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127283.log.gz 2023-08-25T09:54:11.976 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97163.log 2023-08-25T09:54:11.977 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108668.log 2023-08-25T09:54:11.977 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112130.log.gz 2023-08-25T09:54:11.978 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.97163.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97163.log.gz 2023-08-25T09:54:11.978 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125407.log 2023-08-25T09:54:11.981 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.108668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118353.log 2023-08-25T09:54:11.981 INFO:teuthology.orchestra.run.smithi070.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108668.log.gz 2023-08-25T09:54:11.981 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.125407.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125407.log.gz 2023-08-25T09:54:11.982 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100557.log 2023-08-25T09:54:11.982 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126557.log 2023-08-25T09:54:11.984 INFO:teuthology.orchestra.run.smithi070.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118353.log.gz 2023-08-25T09:54:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.100557.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.100557.log.gz 2023-08-25T09:54:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87312.log 2023-08-25T09:54:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.126557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118701.log 2023-08-25T09:54:11.986 INFO:teuthology.orchestra.run.smithi070.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126557.log.gz 2023-08-25T09:54:11.986 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87312.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87312.log.gz 2023-08-25T09:54:11.987 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110195.log 2023-08-25T09:54:11.988 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112723.log 2023-08-25T09:54:11.988 INFO:teuthology.orchestra.run.smithi070.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118701.log.gz 2023-08-25T09:54:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.110195.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110195.log.gz 2023-08-25T09:54:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119364.log 2023-08-25T09:54:11.990 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.112723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130020.log 2023-08-25T09:54:11.991 INFO:teuthology.orchestra.run.smithi070.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112723.log.gz 2023-08-25T09:54:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.119364.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119364.log.gz 2023-08-25T09:54:11.992 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91420.log 2023-08-25T09:54:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.130020.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130020.log.gz 2023-08-25T09:54:11.993 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121938.log 2023-08-25T09:54:11.994 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.91420.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91420.log.gz 2023-08-25T09:54:11.995 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107493.log 2023-08-25T09:54:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.121938.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121938.log.gz 2023-08-25T09:54:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114852.log 2023-08-25T09:54:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.107493.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107493.log.gz 2023-08-25T09:54:11.997 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115618.log 2023-08-25T09:54:11.998 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.114852.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114852.log.gz 2023-08-25T09:54:11.999 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118437.log 2023-08-25T09:54:12.000 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.115618.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115618.log.gz 2023-08-25T09:54:12.000 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87634.log 2023-08-25T09:54:12.001 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.118437.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118437.log.gz 2023-08-25T09:54:12.002 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88223.log 2023-08-25T09:54:12.003 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.87634.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87634.log.gz 2023-08-25T09:54:12.004 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/ceph-client.admin.88223.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88223.log.gz 2023-08-25T09:54:12.191 INFO:teuthology.orchestra.run.smithi070.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.0.99410.log.gz 2023-08-25T09:54:12.268 INFO:teuthology.orchestra.run.smithi070.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.106653.log.gz 2023-08-25T09:54:12.649 INFO:teuthology.orchestra.run.smithi070.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-08-25T09:54:12.727 INFO:teuthology.orchestra.run.smithi070.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-08-25T09:54:12.731 INFO:teuthology.orchestra.run.smithi070.stderr: 2023-08-25T09:54:12.731 INFO:teuthology.orchestra.run.smithi070.stderr:real 0m2.127s 2023-08-25T09:54:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:user 0m7.018s 2023-08-25T09:54:12.732 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m3.130s 2023-08-25T09:54:12.733 INFO:tasks.ceph:Archiving logs... 2023-08-25T09:54:12.734 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/remote/smithi070/log 2023-08-25T09:54:12.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T09:54:14.270 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/remote/smithi155/log 2023-08-25T09:54:14.271 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T09:54:14.874 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-25T09:54:14.891 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-25T09:54:14.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T09:54:14.901 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T09:54:15.054 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-25T09:54:15.064 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-25T09:54:15.064 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:54:15.073 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T09:54:15.122 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2023-08-25T09:54:15.130 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2023-08-25T09:54:15.132 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:54:15.132 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2023-08-25T09:54:15.132 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +2865us[+2878us] +/- 87ms 2023-08-25T09:54:15.132 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv02.front.sepia.ceph.com 3 6 377 51 -2035us[-2021us] +/- 55ms 2023-08-25T09:54:15.132 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 53 +389us[ +402us] +/- 81ms 2023-08-25T09:54:15.133 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:54:15.140 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T09:54:15.140 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2023-08-25T09:54:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 51 +2845us[+2845us] +/- 87ms 2023-08-25T09:54:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -2058us[-2056us] +/- 55ms 2023-08-25T09:54:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +367us[ +367us] +/- 81ms 2023-08-25T09:54:15.141 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T09:54:15.142 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-25T09:54:15.151 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-25T09:54:15.152 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-25T09:54:15.182 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T09:54:15.343 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T09:54:15.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:54:15.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:54:15.712 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T09:54:15.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:54:15.825 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-25T09:54:15.835 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-25T09:54:15.874 INFO:teuthology.task.internal:Duration was 1761.057450 seconds 2023-08-25T09:54:15.874 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-25T09:54:15.923 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-25T09:54:15.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T09:54:15.933 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T09:54:16.084 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:54:16.107 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T09:54:16.557 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-25T09:54:16.557 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2023-08-25T09:54:16.558 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T09:54:16.625 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2023-08-25T09:54:16.625 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T09:54:16.717 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-25T09:54:16.718 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T09:54:16.727 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T09:54:16.821 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-25T09:54:16.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T09:54:16.828 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T09:54:17.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-25T09:54:17.669 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-25T09:54:17.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T09:54:17.762 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T09:54:17.866 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-25T09:54:17.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:54:17.889 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:54:17.995 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2023-08-25T09:54:18.006 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2023-08-25T09:54:18.062 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:54:18.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:54:18.114 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T09:54:18.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T09:54:18.170 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-25T09:54:18.180 INFO:teuthology.task.internal:Transferring archived files... 2023-08-25T09:54:18.181 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/remote/smithi070 2023-08-25T09:54:18.181 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T09:54:18.286 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315/remote/smithi155 2023-08-25T09:54:18.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T09:54:18.397 INFO:teuthology.task.internal:Removing archive directory... 2023-08-25T09:54:18.397 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T09:54:18.407 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T09:54:18.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-25T09:54:18.474 INFO:teuthology.task.internal:Not uploading archives. 2023-08-25T09:54:18.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-25T09:54:18.507 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-25T09:54:18.507 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T09:54:18.521 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T09:54:18.570 INFO:teuthology.orchestra.run.smithi070.stdout: 264347 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 09:54 /home/ubuntu/cephtest 2023-08-25T09:54:18.579 INFO:teuthology.orchestra.run.smithi155.stdout: 263665 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 09:54 /home/ubuntu/cephtest 2023-08-25T09:54:18.582 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-25T09:54:18.595 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-25T09:54:18.674 INFO:teuthology.nuke:Checking targets against current locks 2023-08-25T09:54:18.698 DEBUG:teuthology.nuke:shortname: smithi070 2023-08-25T09:54:18.698 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T09:54:18.721 DEBUG:teuthology.nuke:shortname: smithi155 2023-08-25T09:54:18.722 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T09:54:18.724 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 09:16:06.610146', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL3R/U3OG/FM9IFxeKz3TYfxUwwqbXNkJq2WN7SE1FnMDOfR60dexyJU4bMzBueqLKnZTZ/jAGKwpHHxo5L981I='} 2023-08-25T09:54:18.747 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_06:38:25-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379315', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 09:16:06.608777', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6by/2gfhYuyGvhCvl1oZimmuV0/1tQO9dMYvoeQ22S/c4hi+VkSUZiLArWbdLYvzk4bQdC3sIdT45aZwfMh+U='} 2023-08-25T09:54:18.769 INFO:teuthology.orchestra.console.smithi070:Power off 2023-08-25T09:54:18.771 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T09:54:18.797 DEBUG:teuthology.orchestra.console.smithi070:power off output: Chassis Power Control: Down/Off 2023-08-25T09:54:18.798 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:18.818 INFO:teuthology.orchestra.console.smithi155:Power off 2023-08-25T09:54:18.819 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T09:54:18.830 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-08-25T09:54:18.841 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2023-08-25T09:54:18.842 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:18.868 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-08-25T09:54:22.832 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:22.869 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:22.944 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-08-25T09:54:22.983 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-08-25T09:54:26.948 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:26.985 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:27.058 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is on 2023-08-25T09:54:27.098 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2023-08-25T09:54:31.061 DEBUG:teuthology.orchestra.console.smithi070:pexpect command: ipmitool -H smithi070.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:31.099 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T09:54:31.173 DEBUG:teuthology.orchestra.console.smithi070:check power output: Chassis Power is off 2023-08-25T09:54:31.211 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2023-08-25T09:54:31.275 INFO:teuthology.orchestra.console.smithi070:Power off completed 2023-08-25T09:54:31.312 INFO:teuthology.orchestra.console.smithi155:Power off completed 2023-08-25T09:54:31.420 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1761.057449579239 failure_reason: '"2023-08-25T09:38:37.950823+0000 osd.3 (osd.3) 3 : cluster [WRN] OSD bench result of 99205.725890 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.3. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default mon.a-kernel-sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 mon.b-kernel-sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 owner: scheduled_rishabh@teuthology success: false 2023-08-25T09:54:31.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T09:54:31.508 INFO:teuthology.run:FAIL